From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 74AE2A0613 for ; Wed, 25 Sep 2019 11:29:27 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 155B22AA0; Wed, 25 Sep 2019 11:29:26 +0200 (CEST) Received: from EUR03-DB5-obe.outbound.protection.outlook.com (mail-eopbgr40063.outbound.protection.outlook.com [40.107.4.63]) by dpdk.org (Postfix) with ESMTP id 07D921DBE for ; Wed, 25 Sep 2019 11:29:23 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com; s=selector2-armh-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=nKKTbjUMvJE8+zWDIIFhDpt9hw5ZpRzvce9a8EJeWqY=; b=rxgKfOGImf8BOEb7HAiRx7sddTaHf4XxbhV5OuUq6lThLMas4AKVjprrUbh+b61PFpBAczf1EvcOPvYwFXxd6joOaZubq6Uuu7rfEgp5tCzTWwrhDT9lDVHqwWeEHgV4iPOQnL9Ee3feJQrmQzeZ4ot3e3v9M+IHHfpz0EY8wzw= Received: from DB6PR0802CA0043.eurprd08.prod.outlook.com (2603:10a6:4:a3::29) by DBBPR08MB4631.eurprd08.prod.outlook.com (2603:10a6:10:df::16) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2284.23; Wed, 25 Sep 2019 09:29:20 +0000 Received: from VE1EUR03FT009.eop-EUR03.prod.protection.outlook.com (2a01:111:f400:7e09::206) by DB6PR0802CA0043.outlook.office365.com (2603:10a6:4:a3::29) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2284.19 via Frontend Transport; Wed, 25 Sep 2019 09:29:20 +0000 Authentication-Results: spf=fail (sender IP is 63.35.35.123) smtp.mailfrom=arm.com; dpdk.org; dkim=pass (signature was verified) header.d=armh.onmicrosoft.com;dpdk.org; dmarc=none action=none header.from=arm.com; Received-SPF: Fail (protection.outlook.com: domain of arm.com does not designate 63.35.35.123 as permitted sender) receiver=protection.outlook.com; client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com; Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by VE1EUR03FT009.mail.protection.outlook.com (10.152.18.92) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2284.20 via Frontend Transport; Wed, 25 Sep 2019 09:29:19 +0000 Received: ("Tessian outbound 927f2cdd66cc:v33"); Wed, 25 Sep 2019 09:29:18 +0000 X-CR-MTA-TID: 64aa7808 Received: from dbf327c4ab3b.1 (ip-172-16-0-2.eu-west-1.compute.internal [104.47.2.56]) by 64aa7808-outbound-1.mta.getcheckrecipient.com id 506E7BEE-701A-4EA3-9173-898BDDE1AF4B.1; Wed, 25 Sep 2019 09:29:13 +0000 Received: from EUR01-DB5-obe.outbound.protection.outlook.com (mail-db5eur01lp2056.outbound.protection.outlook.com [104.47.2.56]) by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id dbf327c4ab3b.1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384); Wed, 25 Sep 2019 09:29:13 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=kCkzAmj7HI2TqHRNnJneiSRsHqjbsmbAuVW1d5//4fMRmve/QbZBkLouoIeFuctTao3dU4QNI78gHK24UobLf8HuzpOkgYbWLDe0g/bCgYkXeYHHlWsZUCNj3HjJXNb6KtDLRC2rLptbQ0JWtx0l1Gxe+EnQmwUj4UlijmmvXhgq2+wDK93GHDZ/NK4IKLQ1k2cmHSoYzYDgMgapk4L5/u8MCoXoM1fRtOFJNa6MnlV0VIY4/0+pRQrKclMNMbmypvoOTEml4oWn1cFExXkjchL9MZeUcxywY4TJNxOn5ToAheq1yvdY3OrPYEVQccirZasLHnNsgG8NhOYijrt7mg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=nKKTbjUMvJE8+zWDIIFhDpt9hw5ZpRzvce9a8EJeWqY=; b=SZNo38eOXWwxcu559cN4ShNGvqrrxpErZuoJIhLZYKXMuj6PMDQ5sj4e4Dkzq66p+NcBNoiCV2bkwyrDL4/dWKeRR91+mYZaAbACgX9ZcCl++Sgk9+ggtsoLm7qfsKnvn6yIV8U++Crs99fVe1I5xFtb3BIDrcUu5qjm07LtQp/Db/yBUkO+aokAnhiyLEqyDiCE2R6P+FqZMG3uDZN2Njxv7YGBj5fswBm8xcxIWblFz+In7EH+23eP4K4uvMohJ2NbbrMoOhfWc9OvNmjGK8dsaEXG00rTpQThuzvoN388gAZfqqsRbq4+eXVbbPnOxON24X+E3xynnoYO6yf2ZQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass header.d=arm.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com; s=selector2-armh-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=nKKTbjUMvJE8+zWDIIFhDpt9hw5ZpRzvce9a8EJeWqY=; b=rxgKfOGImf8BOEb7HAiRx7sddTaHf4XxbhV5OuUq6lThLMas4AKVjprrUbh+b61PFpBAczf1EvcOPvYwFXxd6joOaZubq6Uuu7rfEgp5tCzTWwrhDT9lDVHqwWeEHgV4iPOQnL9Ee3feJQrmQzeZ4ot3e3v9M+IHHfpz0EY8wzw= Received: from VI1PR08MB5376.eurprd08.prod.outlook.com (52.133.244.200) by VI1PR08MB3136.eurprd08.prod.outlook.com (52.133.15.29) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2284.25; Wed, 25 Sep 2019 09:29:12 +0000 Received: from VI1PR08MB5376.eurprd08.prod.outlook.com ([fe80::c09a:49d3:eab2:f707]) by VI1PR08MB5376.eurprd08.prod.outlook.com ([fe80::c09a:49d3:eab2:f707%3]) with mapi id 15.20.2284.023; Wed, 25 Sep 2019 09:29:12 +0000 From: "Gavin Hu (Arm Technology China)" To: "Liu, Yong" , "maxime.coquelin@redhat.com" , "Bie, Tiwei" , "Wang, Zhihong" CC: "dev@dpdk.org" , nd , nd Thread-Topic: [dpdk-dev] [PATCH v2 03/16] vhost: add burst enqueue function for packed ring Thread-Index: AQHVbsgy8n4dxJX8MUO7ryE7+kcY6ac5EPEQgAEiWgCAAeHvQA== Date: Wed, 25 Sep 2019 09:29:11 +0000 Message-ID: References: <20190905161421.55981-2-yong.liu@intel.com> <20190919163643.24130-1-yong.liu@intel.com> <20190919163643.24130-4-yong.liu@intel.com> <86228AFD5BCD8E4EBFD2B90117B5E81E633AA324@SHSMSX103.ccr.corp.intel.com> In-Reply-To: <86228AFD5BCD8E4EBFD2B90117B5E81E633AA324@SHSMSX103.ccr.corp.intel.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-ts-tracking-id: 93f79278-37d4-4ad0-adf8-798f691fc3ff.0 x-checkrecipientchecked: true Authentication-Results-Original: spf=none (sender IP is ) smtp.mailfrom=Gavin.Hu@arm.com; x-originating-ip: [113.29.88.7] x-ms-publictraffictype: Email X-MS-Office365-Filtering-Correlation-Id: 298ed4ea-9d86-47f4-ac72-08d7419ad7d9 X-MS-Office365-Filtering-HT: Tenant X-Microsoft-Antispam-Untrusted: BCL:0; PCL:0; RULEID:(2390118)(7020095)(4652040)(8989299)(5600167)(711020)(4605104)(1401327)(4618075)(4534185)(7168020)(4627221)(201703031133081)(201702281549075)(8990200)(2017052603328)(7193020); SRVR:VI1PR08MB3136; X-MS-TrafficTypeDiagnostic: VI1PR08MB3136:|DBBPR08MB4631: X-Microsoft-Antispam-PRVS: x-checkrecipientrouted: true x-ms-oob-tlc-oobclassifiers: OLM:7219;OLM:7219; x-forefront-prvs: 01713B2841 X-Forefront-Antispam-Report-Untrusted: SFV:NSPM; SFS:(10009020)(4636009)(136003)(39860400002)(346002)(396003)(376002)(366004)(199004)(189003)(13464003)(86362001)(53546011)(186003)(5660300002)(14454004)(74316002)(3846002)(6116002)(305945005)(7736002)(2906002)(8936002)(54906003)(110136005)(316002)(6506007)(2501003)(81166006)(102836004)(4326008)(81156014)(55016002)(66066001)(11346002)(6246003)(99286004)(25786009)(9686003)(486006)(76176011)(52536014)(446003)(33656002)(7696005)(229853002)(476003)(66946007)(66446008)(26005)(71200400001)(71190400001)(55236004)(6436002)(256004)(478600001)(14444005)(76116006)(66476007)(66556008)(64756008); DIR:OUT; SFP:1101; SCL:1; SRVR:VI1PR08MB3136; H:VI1PR08MB5376.eurprd08.prod.outlook.com; FPR:; SPF:None; LANG:en; PTR:InfoNoRecords; A:1; MX:1; received-spf: None (protection.outlook.com: arm.com does not designate permitted sender hosts) X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam-Message-Info-Original: 02R8csfOCgf+hbCvcA1YAPVDd2rEwvM0gJ6m6IM6VthwCPE4pRPQWJakgWsx73NYfMrPyyqKem922EL4EPnmk/5fpqU0fqpKJbAhsGrNYeu+/Xb5T4aGSyIyqEG31LQfDCzOd8YfnuIOGuYz81stpW2+L2bPjFoSSqeWfew0SeTnId6Xx3gNIkdd7r4ma98PPflFOm7+gYfsjRoIOuuEX70SMQART0bXtyY1ODqoj/4brXz/8sxEC6sbxY9vArgtoKJp9mlhD3+TIsykaH4hoRZl2BrK02CWTLRY5fzgZVHJEMLba7X5cPKHCeo42tQpWvT42/WavoPneDPNvb07wDSQZ8jC2g584TlUdkb4tHfNKeVDapjeoClP7FEcS6U292wDNILuZ3bQ90kZBcfCImkpjLUapUyWv8ZDAxrqcJM= x-ms-exchange-transport-forked: True Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR08MB3136 Original-Authentication-Results: spf=none (sender IP is ) smtp.mailfrom=Gavin.Hu@arm.com; X-EOPAttributedMessage: 0 X-MS-Exchange-Transport-CrossTenantHeadersStripped: VE1EUR03FT009.eop-EUR03.prod.protection.outlook.com X-Forefront-Antispam-Report: CIP:63.35.35.123; IPV:CAL; SCL:-1; CTRY:IE; EFV:NLI; SFV:NSPM; SFS:(10009020)(4636009)(376002)(346002)(39860400002)(136003)(396003)(1110001)(339900001)(13464003)(199004)(189003)(14444005)(76130400001)(126002)(14454004)(8936002)(478600001)(26826003)(476003)(86362001)(55016002)(2906002)(22756006)(9686003)(356004)(25786009)(81156014)(102836004)(81166006)(229853002)(446003)(70586007)(486006)(11346002)(52536014)(5660300002)(46406003)(8746002)(336012)(97756001)(7736002)(47776003)(70206006)(50466002)(2501003)(6246003)(305945005)(76176011)(99286004)(66066001)(186003)(4326008)(26005)(7696005)(36906005)(6116002)(110136005)(33656002)(53546011)(3846002)(54906003)(316002)(6506007)(23726003)(74316002); DIR:OUT; SFP:1101; SCL:1; SRVR:DBBPR08MB4631; H:64aa7808-outbound-1.mta.getcheckrecipient.com; FPR:; SPF:Fail; LANG:en; PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com; A:1; MX:1; X-MS-Office365-Filtering-Correlation-Id-Prvs: 66a6a33c-c11d-4caf-189d-08d7419ad343 X-Microsoft-Antispam: BCL:0; PCL:0; RULEID:(2390118)(7020095)(4652040)(8989299)(4534185)(4627221)(201703031133081)(201702281549075)(8990200)(5600167)(710020)(711020)(4605104)(1401327)(4618075)(2017052603328)(7193020); SRVR:DBBPR08MB4631; NoDisclaimer: True X-Forefront-PRVS: 01713B2841 X-Microsoft-Antispam-Message-Info: Bo/PDFSq2EizrLeD5N56OoI+cIJntHwvSWHBDzY0SijdWQDegLui3VwdlD4d9oaNNNM5EGdX/V7Sj5aHVy6ZOYG5+wHwTtiPQ/Oq/piPPvnmAg7zAG0Zp+Bm6pL67DEM39bq1+2IB29h9r8UvS4c69O8voM+kaDO19stEGfT6MMPhG+zU7BCbjo/9fnU2lvspHvLHZRjnuHSnecHdMfyvUVCVEkZz0VE96FdAOuDPCoXFbiDKbRa/aTnRIJly8Y0x0dWdZKwprPAGuF8XCcM8KDrrgUOzBjLWwhdNdx/IgkJPstMsHF7UDOaQ1gqr/d/Pnayv2+Mssi2SlFTWJ2PlhQyjbJCdLWwhlufg2ReKN2ad25FF06EFNIR6qMGKHQR7rYp7doXPdWZgp6gNTr13xX6gTGX1hd0Dhj1svy0YhI= X-OriginatorOrg: arm.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 25 Sep 2019 09:29:19.7289 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 298ed4ea-9d86-47f4-ac72-08d7419ad7d9 X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d; Ip=[63.35.35.123]; Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com] X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: DBBPR08MB4631 Subject: Re: [dpdk-dev] [PATCH v2 03/16] vhost: add burst enqueue function for packed ring X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Hi Marvin, > -----Original Message----- > From: Liu, Yong > Sent: Tuesday, September 24, 2019 11:31 AM > To: Gavin Hu (Arm Technology China) ; > maxime.coquelin@redhat.com; Bie, Tiwei ; Wang, > Zhihong > Cc: dev@dpdk.org; nd > Subject: RE: [dpdk-dev] [PATCH v2 03/16] vhost: add burst enqueue functio= n > for packed ring >=20 > Thanks, Gavin. My comments are inline. >=20 > > -----Original Message----- > > From: Gavin Hu (Arm Technology China) [mailto:Gavin.Hu@arm.com] > > Sent: Monday, September 23, 2019 7:09 PM > > To: Liu, Yong ; maxime.coquelin@redhat.com; Bie, > Tiwei > > ; Wang, Zhihong > > Cc: dev@dpdk.org; nd > > Subject: RE: [dpdk-dev] [PATCH v2 03/16] vhost: add burst enqueue > function > > for packed ring > > > > Hi Marvin, > > > > Is it possible to vectorize the processing? > > Other comments inline: > > /Gavin >=20 > Gavin, > According to our experiment, only vectorize some parts in [ed]nqueue > function can't benefit performance. > Functions like vhost_iova_to_vva and virtio_enqueue_offload can't be > easily vectorized as they are full of judgment conditions. >=20 > Thanks, > Marvin >=20 > > > -----Original Message----- > > > From: dev On Behalf Of Marvin Liu > > > Sent: Friday, September 20, 2019 12:37 AM > > > To: maxime.coquelin@redhat.com; tiwei.bie@intel.com; > > > zhihong.wang@intel.com > > > Cc: dev@dpdk.org; Marvin Liu > > > Subject: [dpdk-dev] [PATCH v2 03/16] vhost: add burst enqueue functio= n > > for > > > packed ring > > > > > > Burst enqueue function will first check whether descriptors are cache > > > aligned. It will also check prerequisites in the beginning. Burst > > > enqueue function not support chained mbufs, single packet enqueue > > > function will handle it. > > > > > > Signed-off-by: Marvin Liu > > > > > > diff --git a/lib/librte_vhost/vhost.h b/lib/librte_vhost/vhost.h > > > index 5074226f0..67889c80a 100644 > > > --- a/lib/librte_vhost/vhost.h > > > +++ b/lib/librte_vhost/vhost.h > > > @@ -39,6 +39,9 @@ > > > > > > #define VHOST_LOG_CACHE_NR 32 > > > > > > +#define PACKED_DESCS_BURST (RTE_CACHE_LINE_SIZE / \ > > > + sizeof(struct vring_packed_desc)) > > > + > > > #ifdef SUPPORT_GCC_UNROLL_PRAGMA > > > #define PRAGMA_PARAM "GCC unroll 4" > > > #endif > > > @@ -57,6 +60,8 @@ > > > #define UNROLL_PRAGMA(param) do {} while(0); > > > #endif > > > > > > +#define PACKED_BURST_MASK (PACKED_DESCS_BURST - 1) > > > + > > > /** > > > * Structure contains buffer address, length and descriptor index > > > * from vring to do scatter RX. > > > diff --git a/lib/librte_vhost/virtio_net.c > > b/lib/librte_vhost/virtio_net.c > > > index 2b5c47145..c664b27c5 100644 > > > --- a/lib/librte_vhost/virtio_net.c > > > +++ b/lib/librte_vhost/virtio_net.c > > > @@ -895,6 +895,84 @@ virtio_dev_rx_split(struct virtio_net *dev, > struct > > > vhost_virtqueue *vq, > > > return pkt_idx; > > > } > > > > > > +static __rte_unused __rte_always_inline int > > I remember "__rte_always_inline" should start at the first and separate > > line, otherwise you will get a style issue. > > /Gavin > > > +virtio_dev_rx_burst_packed(struct virtio_net *dev, struct > > vhost_virtqueue > > > *vq, > > > + struct rte_mbuf **pkts) > > > +{ > > > + bool wrap_counter =3D vq->avail_wrap_counter; > > > + struct vring_packed_desc *descs =3D vq->desc_packed; > > > + uint16_t avail_idx =3D vq->last_avail_idx; > > > + > > > + uint64_t desc_addrs[PACKED_DESCS_BURST]; > > > + struct virtio_net_hdr_mrg_rxbuf *hdrs[PACKED_DESCS_BURST]; > > > + uint32_t buf_offset =3D dev->vhost_hlen; > > > + uint64_t lens[PACKED_DESCS_BURST]; > > > + > > > + uint16_t i; > > > + > > > + if (unlikely(avail_idx & PACKED_BURST_MASK)) > > > + return -1; > > > + > > > + UNROLL_PRAGMA(PRAGMA_PARAM) > > > + for (i =3D 0; i < PACKED_DESCS_BURST; i++) { > > > + if (unlikely(pkts[i]->next !=3D NULL)) > > > + return -1; > > > + if (unlikely(!desc_is_avail(&descs[avail_idx + i], > > > + wrap_counter))) > > > + return -1; > > > + } > > > + > > > + rte_smp_rmb(); > > > + > > > + UNROLL_PRAGMA(PRAGMA_PARAM) > > > + for (i =3D 0; i < PACKED_DESCS_BURST; i++) > > > + lens[i] =3D descs[avail_idx + i].len; > > Looks like the code is a strong candidate for vectorization. > > > + > > > + UNROLL_PRAGMA(PRAGMA_PARAM) > > > + for (i =3D 0; i < PACKED_DESCS_BURST; i++) { > > > + if (unlikely(pkts[i]->pkt_len > (lens[i] - buf_offset))) > > > + return -1; > > > + } > > > + > > > + UNROLL_PRAGMA(PRAGMA_PARAM) > > > + for (i =3D 0; i < PACKED_DESCS_BURST; i++) > > > + desc_addrs[i] =3D vhost_iova_to_vva(dev, vq, > > > + descs[avail_idx + i].addr, > > > + &lens[i], > > > + VHOST_ACCESS_RW); > > > + UNROLL_PRAGMA(PRAGMA_PARAM) > > > + for (i =3D 0; i < PACKED_DESCS_BURST; i++) { > > > + if (unlikely(lens[i] !=3D descs[avail_idx + i].len)) > > > + return -1; > > > + } > > > + > > > + UNROLL_PRAGMA(PRAGMA_PARAM) > > > + for (i =3D 0; i < PACKED_DESCS_BURST; i++) { > > > + rte_prefetch0((void *)(uintptr_t)desc_addrs[i]); > > > + hdrs[i] =3D (struct virtio_net_hdr_mrg_rxbuf *)desc_addrs[i]; > > > + lens[i] =3D pkts[i]->pkt_len + dev->vhost_hlen; > > > + } > > > + > > > + UNROLL_PRAGMA(PRAGMA_PARAM) > > > + for (i =3D 0; i < PACKED_DESCS_BURST; i++) > > > + virtio_enqueue_offload(pkts[i], &hdrs[i]->hdr); > > > + > > A store barrier here is missing, last_avail_idx may be observed before = the > > above enqueue completion on weak memory order architectures. > > For x86, a compiler barrier is also required. > > > Thanks a lot for point out. I guess your mention is that need to add barr= ier > between memcpy and enqueue. > last_avail_idx is just local variable, no barrier is need to protect it. Sorry I was wrong, yes, last_avail_idx is a local variable(or we may call i= t meta data). Copying the headers and payload does not need to be ordered, we just need t= o ensure all these happen before updating the idx, which is the single sync= hronization point. In one word, no barriers are required here.=20 /Gavin >=20 > > > + vq->last_avail_idx +=3D PACKED_DESCS_BURST; > > > + if (vq->last_avail_idx >=3D vq->size) { > > > + vq->last_avail_idx -=3D vq->size; > > > + vq->avail_wrap_counter ^=3D 1; > > > + } > > > + > > > + UNROLL_PRAGMA(PRAGMA_PARAM) > > > + for (i =3D 0; i < PACKED_DESCS_BURST; i++) { > > > + rte_memcpy((void *)(uintptr_t)(desc_addrs[i] + buf_offset), > > > + rte_pktmbuf_mtod_offset(pkts[i], void *, 0), > > > + pkts[i]->pkt_len); > > > + } > > > + > > > + return 0; > > > +} > > > + > > > static __rte_unused int16_t > > > virtio_dev_rx_single_packed(struct virtio_net *dev, struct > > vhost_virtqueue > > > *vq, > > > struct rte_mbuf *pkt) > > > -- > > > 2.17.1