From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id BCBD9A0C58; Wed, 25 Aug 2021 11:32:37 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 3F71640041; Wed, 25 Aug 2021 11:32:37 +0200 (CEST) Received: from mga07.intel.com (mga07.intel.com [134.134.136.100]) by mails.dpdk.org (Postfix) with ESMTP id 3BF2B4003D; Wed, 25 Aug 2021 11:32:35 +0200 (CEST) X-IronPort-AV: E=McAfee;i="6200,9189,10086"; a="281204755" X-IronPort-AV: E=Sophos;i="5.84,350,1620716400"; d="scan'208";a="281204755" Received: from orsmga007.jf.intel.com ([10.7.209.58]) by orsmga105.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 25 Aug 2021 02:32:34 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.84,350,1620716400"; d="scan'208";a="464725508" Received: from orsmsx602.amr.corp.intel.com ([10.22.229.15]) by orsmga007.jf.intel.com with ESMTP; 25 Aug 2021 02:32:34 -0700 Received: from orsmsx612.amr.corp.intel.com (10.22.229.25) by ORSMSX602.amr.corp.intel.com (10.22.229.15) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2242.10; Wed, 25 Aug 2021 02:32:33 -0700 Received: from ORSEDG602.ED.cps.intel.com (10.7.248.7) by orsmsx612.amr.corp.intel.com (10.22.229.25) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2242.10 via Frontend Transport; Wed, 25 Aug 2021 02:32:33 -0700 Received: from NAM10-DM6-obe.outbound.protection.outlook.com (104.47.58.100) by edgegateway.intel.com (134.134.137.103) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.1.2242.10; Wed, 25 Aug 2021 02:32:33 -0700 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=Ih0T3e2Be6iJfpSF/475ks1Jbpb8wXuWHLBhNwIvrL2Np0WG4Euw4TGiNXo1KnH4pIW3OuFvpYYchZ8JjyD4DkHJ0Za2I1GXjyebMXS07hWbBNmQ1atPi5ApmvKvzOKGkaJc6AmC4YrtQyRnHps2i5Ekn96OnJCtVLS3YcVVP15cW0tBw45NcBNR5VOwgClf3cn+I4zhvam+2TrPnr51/KCswrvQZP8EEOcR0eQU7h/VsN+UxMZvpB9knHbGwxk31jzg3EUMjZ0GSpXIjNJJdGAu2CpI2G55j6wrLICV+GrkJZ73bAfwzw29fjQUHkGENAtAndlJH/svgXZX3gB3XA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=8hV4MDeC44A1bB7DnOgWF3FeF/pTv14w1We3ZA9E52o=; b=COH4NDwLn0w0PWB3so5oYYWd7vkwU5/ie3y5mmRjXChcsORXRF91AqvnEFHKu34HuMh+46u733zMsoeRkf3OasfCiC9PPzNxlHXeQi8kXJ/xsvbjqj14m/8N9YjWicGPNQSue9mEHIGCyDzmeDBj9dBnj5l1DlIoNSyF0Jr5beGkdvkdQAo8blHCQqHDrtaVMsmN7/LAJlnf3t00YpkjVLnSyYj5sDJZSRLNak1V8ZmvyjhE5gjTtwE0ccVNV6b8AZARz9TApS9+XUzQbk+0p9sVnba01vQLDxHDZNEVMJQUNXbz0wp4C9jkPdnyu6BBRifXqfYHnpAbbRzE9Rw4HA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=intel.com; dmarc=pass action=none header.from=intel.com; dkim=pass header.d=intel.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=intel.onmicrosoft.com; s=selector2-intel-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=8hV4MDeC44A1bB7DnOgWF3FeF/pTv14w1We3ZA9E52o=; b=JReYD+pwS4XnOX5MfzWKq9eM6GAggk4ZGOqek/Yk/xK/cMmfZUhbEJz60UR2S8VS2eO9w+9cpezWc6E/DNF9QB/67ocfYjZzruzg4wF5zfUWmLFUvtE6iHW6u4js8VC9QLCFU2zY7VXJF2wf9wiy8Y6IFH32tOeS1ThAVcdvP6g= Received: from PH0PR11MB4791.namprd11.prod.outlook.com (2603:10b6:510:43::10) by PH0PR11MB4965.namprd11.prod.outlook.com (2603:10b6:510:34::7) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4436.21; Wed, 25 Aug 2021 09:32:29 +0000 Received: from PH0PR11MB4791.namprd11.prod.outlook.com ([fe80::dc7a:b39d:76db:2a4d]) by PH0PR11MB4791.namprd11.prod.outlook.com ([fe80::dc7a:b39d:76db:2a4d%5]) with mapi id 15.20.4436.027; Wed, 25 Aug 2021 09:32:29 +0000 From: "Loftus, Ciara" To: Baruch Siach , "Zhang, Qi Z" CC: "dev@dpdk.org" , "stable@dpdk.org" Thread-Topic: [PATCH] net/af_xdp: fix zero copy Tx queue drain Thread-Index: AQHXmD/7cZlbF2xDUkmmGE2KZWok46uD858Q Date: Wed, 25 Aug 2021 09:32:29 +0000 Message-ID: References: <421e4003bb7f556588ae95f0ed32ce282906a000.1629737818.git.baruch@tkos.co.il> In-Reply-To: <421e4003bb7f556588ae95f0ed32ce282906a000.1629737818.git.baruch@tkos.co.il> Accept-Language: en-GB, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: dlp-product: dlpe-windows dlp-reaction: no-action dlp-version: 11.5.1.3 authentication-results: tkos.co.il; dkim=none (message not signed) header.d=none;tkos.co.il; dmarc=none action=none header.from=intel.com; x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: 9b67cae7-e1df-4ada-5207-08d967ab4205 x-ms-traffictypediagnostic: PH0PR11MB4965: x-ld-processed: 46c98d88-e344-4ed4-8496-4ed7712e255d,ExtAddr x-ms-exchange-transport-forked: True x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:10000; x-ms-exchange-senderadcheck: 1 x-ms-exchange-antispam-relay: 0 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: J5lCdcdj/ZFOeC49lAGC9ca1Vl5ypX9vtT7x9qgDAVvBWoVc8fGpRdyDOzbfinF5gkyLn9wN4MNyGydaeeddvPGM77A1kzRkCwFFxHHeEo327woTXewzcE2nKUXM3U8iKT+mnIQhlszQ2uXvx516GQ/gmbmqjZlnCU6IyJUJSUfQ1WxvyAhWITzK09fwQOOHp7O/jqpNdhfEBaTk1fg6s1K7I7BnEoB9CKioZQLv2aHv5i4WiBIWrlYiVPPyd1BeNxN8PNvb5/3zHLCRLitgaHvjXWZkqQmSo65vVznCIMFTrIOGqVr2y9fCBaqhmSEUWaYdXUHvOcO6pU1xhcl35yNeHSFga7G09EL5lyc3AzFokNoPZaQ6xZMbQDYtFzj17L2ZyJZC3OmruyaU0pf45RJPvbIVrpFUoJOGuKWaF+bKHl6fNN43Zd+H4QUPRJwBL7S3lIphc2utsKkFzJGNnDdpTyAShKtD7J0205oT4REH9DAbc2JPCAsDVspLpNx9l23YOf6M59AbXPizHpX6CKU94kCz2q0GOp/s8Vx1Hr5G5ZHb8OWt7obs4tWUeZX8rdMY9KIoLLuDZfmGeWOqFDB90vZ5XS8KPQelhaGT4gVC9sUS5gUy7qHc0vNmr2er2uOAaov7//LQI0qSQXdnP+a4A2kK+Ld9F6l4AShsRN3fwpOlr8PVy+p5EVkZktUUtMbkqnHfRe1RD1id+uJFvA== x-forefront-antispam-report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:PH0PR11MB4791.namprd11.prod.outlook.com; PTR:; CAT:NONE; SFS:(4636009)(39860400002)(346002)(376002)(366004)(136003)(396003)(55016002)(66476007)(76116006)(6636002)(26005)(83380400001)(38070700005)(71200400001)(66946007)(38100700002)(2906002)(316002)(33656002)(64756008)(66556008)(478600001)(9686003)(7696005)(66446008)(52536014)(186003)(122000001)(8936002)(8676002)(6506007)(5660300002)(86362001)(54906003)(110136005)(4326008); DIR:OUT; SFP:1102; x-ms-exchange-antispam-messagedata-chunkcount: 1 x-ms-exchange-antispam-messagedata-0: =?us-ascii?Q?cHPUYVSZydVTSnCSt6Ysn/3jJmHBM5I/E98zY5oA4kBriqq/4xngo6kx9U8g?= =?us-ascii?Q?GqFjfAf01LCGiF+EnNcgH9B/E45pk5ScRndlRk62YJuP6zclrucoy5z+Kdq6?= =?us-ascii?Q?PQDOVi/iTcnsdvm+jWAJDdUxRQx91vQTbci755PBXghmQTi8808YXgofsEaJ?= =?us-ascii?Q?R41GMwHXgpUz040p0Inlk2X4uEDdqLhKHEFKm+QU1clHIkQQt/Poztw0OtPZ?= =?us-ascii?Q?+RS/LfPDntnO7x6EWBb9//NSvlyDQhn0yK/qkVW+VrPvjFvz32fYbg63JHTh?= =?us-ascii?Q?QQOU1y5p9brpoY30761PylMbzaORozFFSFYu0E+zSC9TqJoc8pWTOrnPpZqE?= =?us-ascii?Q?BhGG+Htgak80oSyCQ6MvvGJ7r7pRqdXb0vuj7vXPR58qvfyNzAubWRKADFIF?= =?us-ascii?Q?Gk8hROwLLvuyHFRFiQu5Y3znZVlU6kaTGx6PJVpWpMRFc42BO3ywjOJu6yvT?= =?us-ascii?Q?8leVe1tXPzmycEGPPksC9KAl5Rj4x31lqjPw2iijoMcbezbw5JNV0xPhgEP6?= =?us-ascii?Q?ZKLVX0pLylgI3jvxG61bGM/rGjiCt/Fn3UgxW5mJePLXj2ZCQk7p/zhdhdU+?= =?us-ascii?Q?J0wVm+0Nk5rwxtp57jFHX/9mgQhHDIBLJH7TYHEtxuskWPDd+f0rG9UEXoi1?= =?us-ascii?Q?Rs+M4OBGZK8GD6ooIF06H54jflCKA/0EKY6zrx7AN5AEVyyeZPB0L260K2eU?= =?us-ascii?Q?FSVanNh/Wqyv96B6mOZMYdgNG0GIbHNLnHqwKYX8f7nMQnG5/zTuLV2FQfgT?= =?us-ascii?Q?kc5K4ftIPMV0K2D/Tsb4jzRSCoksbfM7Z2asWgZm9II7ZaAMMjpErufgadCA?= =?us-ascii?Q?TYvWAoVFfb2ZBv+mSrgy5bEPw9wGF5wcc6WLcRxPSOQ00oededdCv2GiKYQP?= =?us-ascii?Q?mp4JyBuO59f0K9pRjmZ/OAW0kJFM+xF+4aY+pwxJleYltwx+n+08nZnhOzvR?= =?us-ascii?Q?YtErBwMB/CW6NwDvhxGiqP+Q7lnddJoo3DH8y3GmtEzsZ9CRjsNO4dEphJiK?= =?us-ascii?Q?F/ECiievBDj8CR2D68xSWO+CoyVepnnMDZDcULNNMtlz/m3JKEwyo4GdYlnO?= =?us-ascii?Q?hrL8ZiL/icDeWi8X65ntdM55HbVWeTa2Y2Yi/xvtmUfktM+mWDndMKOzQuKx?= =?us-ascii?Q?mYjMAN/2yJNyQ8979x9IM/Mic6S/d6KuVzcrF2Zj1scWqQuzczcWxNr2koi+?= =?us-ascii?Q?IkpIT8vDOgQKArMuv2bdZh5Go/mQPrDL3lNpAl/pqaTpNipO4iHEJ/H1+Pik?= =?us-ascii?Q?zkkyPCtPD/ogXyexi5jEXm2h6QxCMj1kC4ISc57s6D3qFPapXsty/ZNjk50I?= =?us-ascii?Q?hr0FLc+v5IPScfPWHLbnJCVN?= Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: PH0PR11MB4791.namprd11.prod.outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: 9b67cae7-e1df-4ada-5207-08d967ab4205 X-MS-Exchange-CrossTenant-originalarrivaltime: 25 Aug 2021 09:32:29.2776 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 46c98d88-e344-4ed4-8496-4ed7712e255d X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: BZTtvuaONhyEKpMShl6TKrXfXLypsJmdkGy5c5dlkuyPlBf0M67byl0WNV7liv88Cxs/AO7ZexFcNg04wxxQ1g== X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH0PR11MB4965 X-OriginatorOrg: intel.com Subject: Re: [dpdk-dev] [PATCH] net/af_xdp: fix zero copy Tx queue drain X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" >=20 > Call xsk_ring_prod__submit() before kick_tx() so that the kernel > consumer sees the updated state of Tx ring. Otherwise, Tx packets are > stuck in the ring until the next call to af_xdp_tx_zc(). >=20 > Fixes: d8a210774e1d ("net/af_xdp: support unaligned umem chunks") > Cc: stable@dpdk.org >=20 > Signed-off-by: Baruch Siach > --- > drivers/net/af_xdp/rte_eth_af_xdp.c | 8 ++++---- > 1 file changed, 4 insertions(+), 4 deletions(-) >=20 > diff --git a/drivers/net/af_xdp/rte_eth_af_xdp.c > b/drivers/net/af_xdp/rte_eth_af_xdp.c > index 74ffa4511284..81998d86b4aa 100644 > --- a/drivers/net/af_xdp/rte_eth_af_xdp.c > +++ b/drivers/net/af_xdp/rte_eth_af_xdp.c > @@ -502,10 +502,11 @@ af_xdp_tx_zc(void *queue, struct rte_mbuf > **bufs, uint16_t nb_pkts) >=20 > if (mbuf->pool =3D=3D umem->mb_pool) { > if (!xsk_ring_prod__reserve(&txq->tx, 1, &idx_tx)) { > + xsk_ring_prod__submit(&txq->tx, count); We cannot submit 'count' to the tx ring both here and at 'out'. We could en= d up submitting too many. > kick_tx(txq, cq); The purpose of this kick is not necessarily to transmit new descriptors but= to drain the completion queue. No space in the completion queue may be wha= t is preventing the kernel from transmitting existing tx buffers and then i= n userspace causing the the xsk_ring_prod__reserve to fail. We are not trying to transmit new descriptors here. > if (!xsk_ring_prod__reserve(&txq->tx, 1, > &idx_tx)) > - goto out; > + goto out_skip_tx; > } > desc =3D xsk_ring_prod__tx_desc(&txq->tx, idx_tx); > desc->len =3D mbuf->pkt_len; > @@ -527,7 +528,6 @@ af_xdp_tx_zc(void *queue, struct rte_mbuf **bufs, > uint16_t nb_pkts) >=20 > if (!xsk_ring_prod__reserve(&txq->tx, 1, &idx_tx)) { > rte_pktmbuf_free(local_mbuf); > - kick_tx(txq, cq); > goto out; > } >=20 > @@ -551,11 +551,11 @@ af_xdp_tx_zc(void *queue, struct rte_mbuf > **bufs, uint16_t nb_pkts) > tx_bytes +=3D mbuf->pkt_len; > } >=20 > - kick_tx(txq, cq); > - > out: > xsk_ring_prod__submit(&txq->tx, count); > + kick_tx(txq, cq); I think this change is valid. We should kick here after the submit. Thanks for the patch. Could you please spin a v2 if you agree with the abov= e? Thanks, Ciara >=20 > +out_skip_tx: > txq->stats.tx_pkts +=3D count; > txq->stats.tx_bytes +=3D tx_bytes; > txq->stats.tx_dropped +=3D nb_pkts - count; > -- > 2.32.0