From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id A24AEA00C2; Mon, 28 Mar 2022 19:31:43 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 7C0A241104; Mon, 28 Mar 2022 19:31:43 +0200 (CEST) Received: from EUR04-HE1-obe.outbound.protection.outlook.com (mail-eopbgr70040.outbound.protection.outlook.com [40.107.7.40]) by mails.dpdk.org (Postfix) with ESMTP id 5BA0640041 for ; Mon, 28 Mar 2022 19:31:42 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com; s=selector2-armh-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=xq/1i83jtlTyolygsB+wkWS9vzu9Ybcv/v8HS32RqWU=; b=mkovns9WecmFTFLVL86fb7ThAXxkytR0PG8ZCHckNHmdMQHbY5dvf7iAJoA/oqURCBBC2XxAR//NBdkP1mG26CbPjgmTr3b69jirxUukMbexqk1i8a/+7CguDVfzsAKWWvIgibVnsVjSLA+TVc1vlFZdfkEIJlrLyqdQRV8EnPg= Received: from AS9PR06CA0201.eurprd06.prod.outlook.com (2603:10a6:20b:45d::19) by DB9PR08MB6841.eurprd08.prod.outlook.com (2603:10a6:10:2a9::15) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5102.18; Mon, 28 Mar 2022 17:31:36 +0000 Received: from VE1EUR03FT008.eop-EUR03.prod.protection.outlook.com (2603:10a6:20b:45d:cafe::cf) by AS9PR06CA0201.outlook.office365.com (2603:10a6:20b:45d::19) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5102.17 via Frontend Transport; Mon, 28 Mar 2022 17:31:36 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123) smtp.mailfrom=arm.com; dkim=pass (signature was verified) header.d=armh.onmicrosoft.com;dmarc=pass action=none header.from=arm.com; Received-SPF: Pass (protection.outlook.com: domain of arm.com designates 63.35.35.123 as permitted sender) receiver=protection.outlook.com; client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com; Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by VE1EUR03FT008.mail.protection.outlook.com (10.152.18.75) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5102.18 via Frontend Transport; Mon, 28 Mar 2022 17:31:35 +0000 Received: ("Tessian outbound 741ca6c82739:v113"); Mon, 28 Mar 2022 17:31:34 +0000 X-CR-MTA-TID: 64aa7808 Received: from 7e90b29ee8a9.3 by 64aa7808-outbound-1.mta.getcheckrecipient.com id 03B9DE2C-A116-4BA1-A93F-46E58A0B2CC9.1; Mon, 28 Mar 2022 17:31:24 +0000 Received: from EUR04-HE1-obe.outbound.protection.outlook.com by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id 7e90b29ee8a9.3 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384); Mon, 28 Mar 2022 17:31:24 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=Y5Jgbq+V+NEKNu0YhQOc2ZppuHZebGyVgmSzdB/hx4M99tOu/arnKU/in0quj/h4RdQeuv5EJlLz8hoMZRj8GMyO/4iuHytBojsmYZLEIPihDxqCGqq6DwCagSG6cu2dRe3OKKRftz2TmxtA6TKQOc1SzoJSYg5e9pF+3TJFcYyxBoWp+I1dvJrMkxVVfo497SGpmJIW03qztC+1AzT47E+FSv8+7m+LfWjmYd7UWONqfUfIEB0UU+jVbo9s1QMxQBACK4TfGxBMsaTokFZ8m6nWYn0H6WsU8myVW1gIx9F2PQhghtqJ8749wKDt+h/GQjMWMb1FcVE14GTDTyp37w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=xq/1i83jtlTyolygsB+wkWS9vzu9Ybcv/v8HS32RqWU=; b=LGYgy+kKohQb0BOjlDpatn8Az9Q30vJDdG8f8SZFpaNQaTljPczr1S75cr1Xz1Px6gi28JZTsnzGqxwIVM4cbatEP7F9LxfE9IvIe5L/fNcsEhur5Inx7JdhlRdu06CNvAyFhjJBIi0VPKPmZ8CqDidJEtuMe5oF6qF+T7ZCGTEb+wDlhpeAfPiOuTUH454HBYD06dJpNYB87lmTmb64FJWmy2++SXsM1R29T/TKh4aQNZUlWZcg0qoQuracDQkbJbDzwMkrpns/IXA9KIiEygNQSh8+ywsuRWpjW2858Z2IFSgFFlv92ixg5guLoobajthU+n+M16ewDxat132FTw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass header.d=arm.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com; s=selector2-armh-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=xq/1i83jtlTyolygsB+wkWS9vzu9Ybcv/v8HS32RqWU=; b=mkovns9WecmFTFLVL86fb7ThAXxkytR0PG8ZCHckNHmdMQHbY5dvf7iAJoA/oqURCBBC2XxAR//NBdkP1mG26CbPjgmTr3b69jirxUukMbexqk1i8a/+7CguDVfzsAKWWvIgibVnsVjSLA+TVc1vlFZdfkEIJlrLyqdQRV8EnPg= Received: from DBAPR08MB5752.eurprd08.prod.outlook.com (2603:10a6:10:1ac::21) by AM0PR08MB3090.eurprd08.prod.outlook.com (2603:10a6:208:56::23) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5102.16; Mon, 28 Mar 2022 17:31:13 +0000 Received: from DBAPR08MB5752.eurprd08.prod.outlook.com ([fe80::c9d4:b47c:637c:36b6]) by DBAPR08MB5752.eurprd08.prod.outlook.com ([fe80::c9d4:b47c:637c:36b6%9]) with mapi id 15.20.5102.022; Mon, 28 Mar 2022 17:31:13 +0000 From: Kathleen Capella To: Kathleen Capella , Jingjing Wu , Beilei Xing CC: "dev@dpdk.org" , nd , Honnappa Nagarahalli , Dharmik Thakkar Subject: RE: [PATCH] net/iavf: remove extra copy step in Rx bulk path Thread-Topic: [PATCH] net/iavf: remove extra copy step in Rx bulk path Thread-Index: AQHYP8wpJjE6AOgdgkOS9pV4EQBn7qzVE4bQ Date: Mon, 28 Mar 2022 17:31:12 +0000 Message-ID: References: <20220324221132.10055-1-kathleen.capella@arm.com> In-Reply-To: <20220324221132.10055-1-kathleen.capella@arm.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-ts-tracking-id: 3EED966C659A044A869AC0556A5D3BBB.0 x-checkrecipientchecked: true Authentication-Results-Original: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=arm.com; X-MS-Office365-Filtering-Correlation-Id: beb77c75-8ad3-4c32-7951-08da10e0cefe x-ms-traffictypediagnostic: AM0PR08MB3090:EE_|VE1EUR03FT008:EE_|DB9PR08MB6841:EE_ X-Microsoft-Antispam-PRVS: x-checkrecipientrouted: true nodisclaimer: true X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam-Untrusted: BCL:0; X-Microsoft-Antispam-Message-Info-Original: tCiKMVC05bCEe0Ach0YnBw84p5S/qypYbgzz2hX5Dj9B7qs7O7t0iAzOZo8994xoVI+JttnFWKdeWaijHQZ3/5CB0e+Z7qxp2nondQ5SWW+MVGeZQy0dH6bXenFh2fFOnxvQmaa8ZcJfCHvQPDDeW8dTS2AqdBDyZceEC+vz5Bk60gcm0f/HnLuVRyAt5d9z9A/6UN2K4Yfnpj55bOdau0bpiNAw5Y2gM5yKy1dx4A4IwmsIyQ/15VKawzsR2MBdFX3TeexCic5wTCRUuljdb29AKVV3bLXzjiGAep/BRy7sx5xS9KYGW/ej9Hj1lUWLc3E5h+cROW1Pt3USJWU0iStJCMUDB1Ca8h4D7h0odM86JRI2+0mais5hd1Uef6JuYLnFTceg3e3+31BUQsZ286NFC+zmWzkyQ+2gBvRnTODMql1nkhIKDuEMIz8xqxb5Coq2BPAjtaoxRNOjKWdImCD6k5FZcoUb9yHqDePwRgkES1SFmoYvTIUdtgtPSqRyM/PxgC/ylsByzpCj7Aai67QQBaSevjCde+27fHgPK2XT+Y6Q/HKR+muPWIr+GfBHaxF3xBNw/ZLft9AakQRdBwbgrzMMzGlC4E0R5lY/s3XwlRl4Pk3DyPOs7WMzp8zo6Bs9J9tjzTWEeJN+29YVpga/a+XXRY0H3GJ/dvv4ydQnCNCTcpUeZ6nX/PghDoYxwETliek+45J1y0v5pZRvNQ== X-Forefront-Antispam-Report-Untrusted: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:DBAPR08MB5752.eurprd08.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230001)(4636009)(366004)(4326008)(8676002)(55016003)(8936002)(76116006)(66556008)(66946007)(66446008)(5660300002)(66476007)(71200400001)(52536014)(38100700002)(64756008)(122000001)(316002)(83380400001)(2906002)(54906003)(26005)(186003)(110136005)(33656002)(7696005)(9686003)(508600001)(53546011)(38070700005)(86362001)(6506007); DIR:OUT; SFP:1101; Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM0PR08MB3090 Original-Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=arm.com; X-EOPAttributedMessage: 0 X-MS-Exchange-Transport-CrossTenantHeadersStripped: VE1EUR03FT008.eop-EUR03.prod.protection.outlook.com X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id-Prvs: d8f5f4dd-f46f-4611-c338-08da10e0c17a X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: CUNDU9vdW/WdVpHXMGPPqSRzspGc0a00CaAqu7wOT/DAQQv3cbeq5rL2Zs//TNNmHs6LsWzH20kHC/dvjDRlI6/UqCcibODxwLKC5IobPSNb4WO4i5fKtd+EpXR3lTHk+MM0oLETJjZHZQWj6Rzz+wBoojdWch39jUKew5XyODxu4cOzwzHioY3zOQCD8L3tZoE0dnHfi/s/nzivrBiZhiUS1te9UlnI8TJvE93QGEAb5rqO9FZWsOc0moY6yxu11QIAmkro/KQm3r2DvWbJAsB2fbcs6ENMx7aDspkErqjgn+J4gCTmj+9a0sYcIEXmom9JZJ6c1lwPT5bfGfinw06NDPsTm8SJR362oD36SwYDLzpi2e/lBWTLnUqVXZPOZC1eeDeS/5fI4PlUeqXqGS9LHWccbPwUFAaPwR2Aw31NqS2sFzlFt3bUC0f9QhQKT3D4TEFWCHKMxSPQ+5nCAzZVL08TyUY7m1UDNnotMgfNtH/mIetNgCEX3k7qMJdQzsyc2Kw5iP0TRlCzMl/CwmMrYk6DwEm/LCt1WwThcavifqK2uYLFxgNCJDSxKKfjpaUmLgQJ89A8QQEjPC/G7dUR23er7nu8xPsPKqLbIyc6p2mxBM+K+ZrS1zPyMYR7YwLy7rla2qk1i6NaRzHVIqoOjl4PJ0aveJ3BwQFT+iN5FMgcb+fHGjXLxqSjpaYO X-Forefront-Antispam-Report: CIP:63.35.35.123; CTRY:IE; LANG:en; SCL:1; SRV:; IPV:CAL; SFV:NSPM; H:64aa7808-outbound-1.mta.getcheckrecipient.com; PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com; CAT:NONE; SFS:(13230001)(4636009)(46966006)(36840700001)(40470700004)(7696005)(54906003)(316002)(6506007)(356005)(55016003)(83380400001)(336012)(110136005)(2906002)(36860700001)(8676002)(4326008)(52536014)(81166007)(82310400004)(53546011)(8936002)(186003)(86362001)(47076005)(70206006)(33656002)(508600001)(26005)(5660300002)(40460700003)(70586007)(9686003); DIR:OUT; SFP:1101; X-OriginatorOrg: arm.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 28 Mar 2022 17:31:35.6286 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: beb77c75-8ad3-4c32-7951-08da10e0cefe X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d; Ip=[63.35.35.123]; Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com] X-MS-Exchange-CrossTenant-AuthSource: VE1EUR03FT008.eop-EUR03.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: DB9PR08MB6841 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org The failure in ci/iol-broadcom-Functional on this patch seems to be unrelat= ed to the patch. > -----Original Message----- > From: Kathleen Capella > Sent: Thursday, March 24, 2022 5:12 PM > To: Jingjing Wu ; Beilei Xing > Cc: dev@dpdk.org; nd ; Honnappa Nagarahalli > ; Dharmik Thakkar > ; Kathleen Capella > > Subject: [PATCH] net/iavf: remove extra copy step in Rx bulk path >=20 > In the Rx bulk path, packets which are taken from the HW ring, are first > copied to the stage data structure and then later copied from the stage t= o the > rx_pkts array. For the number of packets requested immediately by the > receiving function, this two-step process adds extra overhead that is not > necessary. >=20 > Instead, put requested number of packets directly into the rx_pkts array = and > only stage excess packets. On N1SDP with 1 core/port, l3fwd saw up to 4% > performance improvement. On x86, no difference in performance was > observed. >=20 > Signed-off-by: Kathleen Capella > Suggested-by: Dharmik Thakkar > --- > drivers/net/iavf/iavf_rxtx.c | 74 ++++++++++++++++++++++++------------ > 1 file changed, 49 insertions(+), 25 deletions(-) >=20 > diff --git a/drivers/net/iavf/iavf_rxtx.c b/drivers/net/iavf/iavf_rxtx.c = index > 16e8d021f9..245dd225fd 100644 > --- a/drivers/net/iavf/iavf_rxtx.c > +++ b/drivers/net/iavf/iavf_rxtx.c > @@ -1813,7 +1813,9 @@ iavf_recv_scattered_pkts(void *rx_queue, struct > rte_mbuf **rx_pkts, >=20 > #define IAVF_LOOK_AHEAD 8 > static inline int > -iavf_rx_scan_hw_ring_flex_rxd(struct iavf_rx_queue *rxq) > +iavf_rx_scan_hw_ring_flex_rxd(struct iavf_rx_queue *rxq, > + struct rte_mbuf **rx_pkts, > + uint16_t nb_pkts) > { > volatile union iavf_rx_flex_desc *rxdp; > struct rte_mbuf **rxep; > @@ -1822,6 +1824,7 @@ iavf_rx_scan_hw_ring_flex_rxd(struct > iavf_rx_queue *rxq) > uint16_t pkt_len; > int32_t s[IAVF_LOOK_AHEAD], var, nb_dd; > int32_t i, j, nb_rx =3D 0; > + int32_t nb_staged =3D 0; > uint64_t pkt_flags; > const uint32_t *ptype_tbl =3D rxq->vsi->adapter->ptype_tbl; >=20 > @@ -1867,8 +1870,6 @@ iavf_rx_scan_hw_ring_flex_rxd(struct > iavf_rx_queue *rxq) #endif > } >=20 > - nb_rx +=3D nb_dd; > - > /* Translate descriptor info to mbuf parameters */ > for (j =3D 0; j < nb_dd; j++) { > IAVF_DUMP_RX_DESC(rxq, &rxdp[j], > @@ -1892,24 +1893,34 @@ iavf_rx_scan_hw_ring_flex_rxd(struct > iavf_rx_queue *rxq) > pkt_flags =3D > iavf_flex_rxd_error_to_pkt_flags(stat_err0); >=20 > mb->ol_flags |=3D pkt_flags; > - } >=20 > - for (j =3D 0; j < IAVF_LOOK_AHEAD; j++) > - rxq->rx_stage[i + j] =3D rxep[j]; > + /* Put up to nb_pkts directly into buffers */ > + if ((i + j) < nb_pkts) { > + rx_pkts[i + j] =3D rxep[j]; > + nb_rx++; > + } else { > + /* Stage excess pkts received */ > + rxq->rx_stage[nb_staged] =3D rxep[j]; > + nb_staged++; > + } > + } >=20 > if (nb_dd !=3D IAVF_LOOK_AHEAD) > break; > } >=20 > + /* Update rxq->rx_nb_avail to reflect number of staged pkts */ > + rxq->rx_nb_avail =3D nb_staged; > + > /* Clear software ring entries */ > - for (i =3D 0; i < nb_rx; i++) > + for (i =3D 0; i < (nb_rx + nb_staged); i++) > rxq->sw_ring[rxq->rx_tail + i] =3D NULL; >=20 > return nb_rx; > } >=20 > static inline int > -iavf_rx_scan_hw_ring(struct iavf_rx_queue *rxq) > +iavf_rx_scan_hw_ring(struct iavf_rx_queue *rxq, struct rte_mbuf > +**rx_pkts, uint16_t nb_pkts) > { > volatile union iavf_rx_desc *rxdp; > struct rte_mbuf **rxep; > @@ -1919,6 +1930,7 @@ iavf_rx_scan_hw_ring(struct iavf_rx_queue *rxq) > uint32_t rx_status; > int32_t s[IAVF_LOOK_AHEAD], var, nb_dd; > int32_t i, j, nb_rx =3D 0; > + int32_t nb_staged =3D 0; > uint64_t pkt_flags; > const uint32_t *ptype_tbl =3D rxq->vsi->adapter->ptype_tbl; >=20 > @@ -1970,8 +1982,6 @@ iavf_rx_scan_hw_ring(struct iavf_rx_queue *rxq) > #endif > } >=20 > - nb_rx +=3D nb_dd; > - > /* Translate descriptor info to mbuf parameters */ > for (j =3D 0; j < nb_dd; j++) { > IAVF_DUMP_RX_DESC(rxq, &rxdp[j], > @@ -2000,17 +2010,26 @@ iavf_rx_scan_hw_ring(struct iavf_rx_queue > *rxq) > pkt_flags |=3D iavf_rxd_build_fdir(&rxdp[j], > mb); >=20 > mb->ol_flags |=3D pkt_flags; > - } >=20 > - for (j =3D 0; j < IAVF_LOOK_AHEAD; j++) > - rxq->rx_stage[i + j] =3D rxep[j]; > + /* Put up to nb_pkts directly into buffers */ > + if ((i + j) < nb_pkts) { > + rx_pkts[i + j] =3D rxep[j]; > + nb_rx++; > + } else { /* Stage excess pkts received */ > + rxq->rx_stage[nb_staged] =3D rxep[j]; > + nb_staged++; > + } > + } >=20 > if (nb_dd !=3D IAVF_LOOK_AHEAD) > break; > } >=20 > + /* Update rxq->rx_nb_avail to reflect number of staged pkts */ > + rxq->rx_nb_avail =3D nb_staged; > + > /* Clear software ring entries */ > - for (i =3D 0; i < nb_rx; i++) > + for (i =3D 0; i < (nb_rx + nb_staged); i++) > rxq->sw_ring[rxq->rx_tail + i] =3D NULL; >=20 > return nb_rx; > @@ -2098,23 +2117,31 @@ rx_recv_pkts(void *rx_queue, struct rte_mbuf > **rx_pkts, uint16_t nb_pkts) > return iavf_rx_fill_from_stage(rxq, rx_pkts, nb_pkts); >=20 > if (rxq->rxdid >=3D IAVF_RXDID_FLEX_NIC && rxq->rxdid <=3D > IAVF_RXDID_LAST) > - nb_rx =3D (uint16_t)iavf_rx_scan_hw_ring_flex_rxd(rxq); > + nb_rx =3D (uint16_t)iavf_rx_scan_hw_ring_flex_rxd(rxq, > rx_pkts, > +nb_pkts); > else > - nb_rx =3D (uint16_t)iavf_rx_scan_hw_ring(rxq); > + nb_rx =3D (uint16_t)iavf_rx_scan_hw_ring(rxq, rx_pkts, > nb_pkts); > + > rxq->rx_next_avail =3D 0; > - rxq->rx_nb_avail =3D nb_rx; > - rxq->rx_tail =3D (uint16_t)(rxq->rx_tail + nb_rx); > + rxq->rx_tail =3D (uint16_t)(rxq->rx_tail + nb_rx + rxq->rx_nb_avail); >=20 > if (rxq->rx_tail > rxq->rx_free_trigger) { > if (iavf_rx_alloc_bufs(rxq) !=3D 0) { > - uint16_t i, j; > + uint16_t i, j, nb_staged; >=20 > /* TODO: count rx_mbuf_alloc_failed here */ >=20 > + nb_staged =3D rxq->rx_nb_avail; > rxq->rx_nb_avail =3D 0; > - rxq->rx_tail =3D (uint16_t)(rxq->rx_tail - nb_rx); > - for (i =3D 0, j =3D rxq->rx_tail; i < nb_rx; i++, j++) > + > + rxq->rx_tail =3D (uint16_t)(rxq->rx_tail - (nb_rx + > nb_staged)); > + for (i =3D 0, j =3D rxq->rx_tail; i < nb_rx; i++, j++) { > + rxq->sw_ring[j] =3D rx_pkts[i]; > + rx_pkts[i] =3D NULL; > + } > + for (i =3D 0, j =3D rxq->rx_tail + nb_rx; i < nb_staged; i++, > j++) { > rxq->sw_ring[j] =3D rxq->rx_stage[i]; > + rx_pkts[i] =3D NULL; > + } >=20 > return 0; > } > @@ -2127,10 +2154,7 @@ rx_recv_pkts(void *rx_queue, struct rte_mbuf > **rx_pkts, uint16_t nb_pkts) > rxq->port_id, rxq->queue_id, > rxq->rx_tail, nb_rx); >=20 > - if (rxq->rx_nb_avail) > - return iavf_rx_fill_from_stage(rxq, rx_pkts, nb_pkts); > - > - return 0; > + return nb_rx; > } >=20 > static uint16_t > -- > 2.31.1