From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id DDE50A0613 for ; Wed, 25 Sep 2019 09:15:46 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 27C3E2D13; Wed, 25 Sep 2019 09:15:46 +0200 (CEST) Received: from EUR02-AM5-obe.outbound.protection.outlook.com (mail-eopbgr00072.outbound.protection.outlook.com [40.107.0.72]) by dpdk.org (Postfix) with ESMTP id 8A6382C30 for ; Wed, 25 Sep 2019 09:15:45 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com; s=selector2-armh-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=8DTTmEHHFYc3QlBYW/p53+In+Ol6RPs/O4hGr+olerk=; b=FFfIi1hJzCBWCq89Z2KDKLzqx0thsQQQt1U70lIeMiqU+XpOzq+aGMvk2uAvBKN8PEp6v2hhRI5S32viAaDwmdNgcDiWU/EF+b1ab7Gppr48gyNR9Tmhxm/xHkAKskyed0EDcVZUQuvwXs8Ox6xzBK2rN2ERR+6NZPdF8XbVun8= Received: from VI1PR08CA0256.eurprd08.prod.outlook.com (2603:10a6:803:dc::29) by AM5PR0801MB2035.eurprd08.prod.outlook.com (2603:10a6:203:4a::19) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2284.20; Wed, 25 Sep 2019 07:15:43 +0000 Received: from DB5EUR03FT057.eop-EUR03.prod.protection.outlook.com (2a01:111:f400:7e0a::204) by VI1PR08CA0256.outlook.office365.com (2603:10a6:803:dc::29) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2284.23 via Frontend Transport; Wed, 25 Sep 2019 07:15:43 +0000 Authentication-Results: spf=temperror (sender IP is 63.35.35.123) smtp.mailfrom=arm.com; dpdk.org; dkim=pass (signature was verified) header.d=armh.onmicrosoft.com;dpdk.org; dmarc=none action=none header.from=arm.com; Received-SPF: TempError (protection.outlook.com: error in processing during lookup of arm.com: DNS Timeout) Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by DB5EUR03FT057.mail.protection.outlook.com (10.152.20.235) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2284.20 via Frontend Transport; Wed, 25 Sep 2019 07:15:42 +0000 Received: ("Tessian outbound 0cf06bf5c60e:v33"); Wed, 25 Sep 2019 07:15:41 +0000 X-CR-MTA-TID: 64aa7808 Received: from 645bf51da9ef.1 (ip-172-16-0-2.eu-west-1.compute.internal [104.47.5.55]) by 64aa7808-outbound-1.mta.getcheckrecipient.com id 4F20C525-DEFA-42E3-ADCF-604AC9C8938B.1; Wed, 25 Sep 2019 07:15:36 +0000 Received: from EUR02-HE1-obe.outbound.protection.outlook.com (mail-he1eur02lp2055.outbound.protection.outlook.com [104.47.5.55]) by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id 645bf51da9ef.1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384); Wed, 25 Sep 2019 07:15:36 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=gt3k9wol8d9GiesV8gJepsSuf8dYOiKnU7j36JAUygZlRJDJOcGJ44YhCEYSAqk71LWr3lqX0MFBMu8Aa4ThklJS8Fn+5N10DegNvjxjc/PVJZ0zXQAuO66nDxqMl3RVPvGKXNtZQuhM4ADU5xkcKeALbJAT/qU1luDYb51cHh3v9o+ve+E98hNDdQNw0MfwOCd9pvOuEH4JPIFZWD4YUk1ueCzB4th3BKQmUIAf9r2pJpRHdT+ER5XQHzpbbETunfrHWiIooKmr/vhTlqArDev9bb9vD4MpU1O6SQBAalWq8CVb+/I+yqTnLV62XJgbrpiPJHjkcYk2RctYzMFcCg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=8DTTmEHHFYc3QlBYW/p53+In+Ol6RPs/O4hGr+olerk=; b=Jrhc4S65nKTbXgvq5rGiKQm18Uk2eURTsk819E0a9mmn0NWB0vMj7r5nDNYlzfBvMuDAycfp2cZdwdcmU6UmTxg6CjBJy5gKxzNw5VEHUc+HzYmtcwaO6VAHhRc2pBrqLc0/x5aMNeUTe8e3LbFaopK+/gUXgL2VSc8MQ60HeY4P2TEWZ+uUtGUHUQhZ+DJGYe4XJ9HWIKSoLzBybHkZKM7C5guATIN4evZ61uvCvTmhKOvSELQCxiISAb2sUNHZwZZ6CJcTi+GEXIThjHkY3MfePLKJTbLk55BtsGBIBxSkNuTd1IF/zMpKYiBcwaAmSdg+sAZJ9NaMcDtkE90heg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass header.d=arm.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com; s=selector2-armh-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=8DTTmEHHFYc3QlBYW/p53+In+Ol6RPs/O4hGr+olerk=; b=FFfIi1hJzCBWCq89Z2KDKLzqx0thsQQQt1U70lIeMiqU+XpOzq+aGMvk2uAvBKN8PEp6v2hhRI5S32viAaDwmdNgcDiWU/EF+b1ab7Gppr48gyNR9Tmhxm/xHkAKskyed0EDcVZUQuvwXs8Ox6xzBK2rN2ERR+6NZPdF8XbVun8= Received: from VI1PR08MB5376.eurprd08.prod.outlook.com (52.133.244.200) by VI1PR08MB5421.eurprd08.prod.outlook.com (52.133.245.139) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2284.20; Wed, 25 Sep 2019 07:15:34 +0000 Received: from VI1PR08MB5376.eurprd08.prod.outlook.com ([fe80::c09a:49d3:eab2:f707]) by VI1PR08MB5376.eurprd08.prod.outlook.com ([fe80::c09a:49d3:eab2:f707%3]) with mapi id 15.20.2284.023; Wed, 25 Sep 2019 07:15:34 +0000 From: "Gavin Hu (Arm Technology China)" To: "Liu, Yong" , "'maxime.coquelin@redhat.com'" , "Bie, Tiwei" , "Wang, Zhihong" CC: "'dev@dpdk.org'" , nd , nd Thread-Topic: [dpdk-dev] [PATCH v2 08/16] vhost: add flush function for burst enqueue Thread-Index: AQHVbshQyvDz2MRg6k67Hms4iIqIBqc7xaZggAAjgQCAABStAIAABT3w Date: Wed, 25 Sep 2019 07:15:33 +0000 Message-ID: References: <20190905161421.55981-2-yong.liu@intel.com> <20190919163643.24130-1-yong.liu@intel.com> <20190919163643.24130-9-yong.liu@intel.com> <86228AFD5BCD8E4EBFD2B90117B5E81E633B08DB@SHSMSX103.ccr.corp.intel.com> <86228AFD5BCD8E4EBFD2B90117B5E81E633B0C2B@SHSMSX103.ccr.corp.intel.com> In-Reply-To: <86228AFD5BCD8E4EBFD2B90117B5E81E633B0C2B@SHSMSX103.ccr.corp.intel.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-ts-tracking-id: c3e5d23d-d23d-4d86-a495-dc19cebcf5ab.0 x-checkrecipientchecked: true Authentication-Results-Original: spf=none (sender IP is ) smtp.mailfrom=Gavin.Hu@arm.com; x-originating-ip: [113.29.88.7] x-ms-publictraffictype: Email X-MS-Office365-Filtering-Correlation-Id: c8c0fc40-5c9c-4d47-9ff7-08d741882d53 X-MS-Office365-Filtering-HT: Tenant X-Microsoft-Antispam-Untrusted: BCL:0; PCL:0; RULEID:(2390118)(7020095)(4652040)(8989299)(4534185)(7168020)(4627221)(201703031133081)(201702281549075)(8990200)(5600167)(711020)(4605104)(1401327)(4618075)(2017052603328)(7193020); SRVR:VI1PR08MB5421; X-MS-TrafficTypeDiagnostic: VI1PR08MB5421:|AM5PR0801MB2035: X-Microsoft-Antispam-PRVS: x-checkrecipientrouted: true x-ms-oob-tlc-oobclassifiers: OLM:6108;OLM:6108; x-forefront-prvs: 01713B2841 X-Forefront-Antispam-Report-Untrusted: SFV:NSPM; SFS:(10009020)(4636009)(39860400002)(396003)(376002)(346002)(136003)(366004)(199004)(189003)(13464003)(25786009)(55016002)(229853002)(52536014)(81166006)(8936002)(6116002)(81156014)(3846002)(74316002)(6436002)(71190400001)(110136005)(305945005)(54906003)(5660300002)(71200400001)(316002)(66066001)(11346002)(476003)(7736002)(9686003)(256004)(14444005)(446003)(486006)(66476007)(76116006)(53546011)(76176011)(66946007)(14454004)(186003)(2906002)(86362001)(6246003)(64756008)(99286004)(4326008)(102836004)(66556008)(26005)(478600001)(7696005)(6506007)(55236004)(66446008)(33656002)(491001); DIR:OUT; SFP:1101; SCL:1; SRVR:VI1PR08MB5421; H:VI1PR08MB5376.eurprd08.prod.outlook.com; FPR:; SPF:None; LANG:en; PTR:InfoNoRecords; A:1; MX:1; received-spf: None (protection.outlook.com: arm.com does not designate permitted sender hosts) X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam-Message-Info-Original: CNEOSzU8zJRERbnEXjkaCl+VcZXV6/eWjVJ+17lqCuoVIDvJX8nimbxVjU/kfBiQUXX8dBxJGu9BMr5LQ+u0AyfEgF1xebpGWQrdezMCJz3ZT8jh17lUY3oNL7iWSZ+Gl3VcPir3TtnRH6VoqdEzwX4KSAZCJq+ZvoC5Y8Qn25P+s8+RbwA26m8jd18rEvEEHn0oM1N3/Kw990RTnnJM6zMbpVyJGQr6Efe6VTZEIhDobylsE69GM7cgMZ6ea3JZgk3wqiLoVZZAt3sP5sgGQjPgHxfCShZ6kqZRLIYz4MGyjtd7MYYv2Uj43SKhChqcuW5hAFzOUlmVoxhs9DUfRKG8IQIuihqpgtmUsC8ecl0h+QDjOI+wLhpJLoF7Td3eC+yhNJmwT3ytqiEzPwKs1VNAHUd5q0qmlGkN0qVyKYk= x-ms-exchange-transport-forked: True Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR08MB5421 Original-Authentication-Results: spf=none (sender IP is ) smtp.mailfrom=Gavin.Hu@arm.com; X-EOPAttributedMessage: 0 X-MS-Exchange-Transport-CrossTenantHeadersStripped: DB5EUR03FT057.eop-EUR03.prod.protection.outlook.com X-Forefront-Antispam-Report: CIP:63.35.35.123; IPV:CAL; SCL:-1; CTRY:IE; EFV:NLI; SFV:NSPM; SFS:(10009020)(4636009)(39860400002)(396003)(136003)(346002)(376002)(199004)(189003)(13464003)(11346002)(476003)(5660300002)(50466002)(126002)(229853002)(486006)(25786009)(446003)(2906002)(46406003)(99286004)(66066001)(81156014)(81166006)(7696005)(63350400001)(8746002)(356004)(33656002)(102836004)(186003)(26005)(76176011)(6506007)(53546011)(86362001)(22756006)(6246003)(336012)(8936002)(74316002)(6116002)(47776003)(14444005)(23726003)(110136005)(7736002)(316002)(54906003)(70586007)(70206006)(305945005)(3846002)(76130400001)(26826003)(478600001)(97756001)(14454004)(55016002)(9686003)(4326008)(52536014)(491001); DIR:OUT; SFP:1101; SCL:1; SRVR:AM5PR0801MB2035; H:64aa7808-outbound-1.mta.getcheckrecipient.com; FPR:; SPF:TempError; LANG:en; PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com; A:1; MX:1; X-MS-Office365-Filtering-Correlation-Id-Prvs: 581281d9-b20c-41a7-cced-08d741882826 X-Microsoft-Antispam: BCL:0; PCL:0; RULEID:(2390118)(7020095)(4652040)(8989299)(4534185)(4627221)(201703031133081)(201702281549075)(8990200)(5600167)(710020)(711020)(4605104)(1401327)(4618075)(2017052603328)(7193020); SRVR:AM5PR0801MB2035; NoDisclaimer: True X-Forefront-PRVS: 01713B2841 X-Microsoft-Antispam-Message-Info: BquIW/kRWvUQv6gdo2Y3biZ+PCYJheB1F7l/ncChEEqP3ky0mffKJwL9kCDV7EG7mW7B4hpkVxQQ/L2CswfPDPcj12a1h+RgZX9TITc5Bedud65L/3VATuYc1qnZFyzII5Tnq8DiZHs8WMuyQr1GQDXehwdU1dD4iYfuvHIF/WdCoVJyRReGi69DsG/x7+L/wDfncyDu6wgjJ/nBBq7MzMnyXyVMIOSHKH+JCURKwyC5wru/+ZYeeIVzkIlMo8Cp6hhxTPxLaWukzTV4WZA2H2Mzdtb/BX+X7hmmJwjwruHhCyZNd6rzytGb4vccR2jjEl2SOrc5bYSxtbFSd911pMzc9/Ng8SFtV8cOe5A1UBgV4ArwRmhRatzcHUp0yTbTISO8dgsaNVqzSAgQkc+SvcbAiyuXZJgfaFOH77kmpro= X-OriginatorOrg: arm.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 25 Sep 2019 07:15:42.7466 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: c8c0fc40-5c9c-4d47-9ff7-08d741882d53 X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d; Ip=[63.35.35.123]; Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com] X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM5PR0801MB2035 Subject: Re: [dpdk-dev] [PATCH v2 08/16] vhost: add flush function for burst enqueue X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Hi Marvin, > -----Original Message----- > From: Liu, Yong > Sent: Wednesday, September 25, 2019 2:52 PM > To: Gavin Hu (Arm Technology China) ; > 'maxime.coquelin@redhat.com' ; Bie, > Tiwei ; Wang, Zhihong > Cc: 'dev@dpdk.org' ; nd > Subject: RE: [dpdk-dev] [PATCH v2 08/16] vhost: add flush function for bu= rst > enqueue >=20 > > -----Original Message----- > > From: Liu, Yong > > Sent: Wednesday, September 25, 2019 1:38 PM > > To: Gavin Hu (Arm Technology China) ; > > maxime.coquelin@redhat.com; Bie, Tiwei ; Wang, > Zhihong > > > > Cc: dev@dpdk.org; nd > > Subject: RE: [dpdk-dev] [PATCH v2 08/16] vhost: add flush function for > > burst enqueue > > > > > > > > > -----Original Message----- > > > From: Gavin Hu (Arm Technology China) [mailto:Gavin.Hu@arm.com] > > > Sent: Wednesday, September 25, 2019 11:38 AM > > > To: Liu, Yong ; maxime.coquelin@redhat.com; Bie, > > Tiwei > > > ; Wang, Zhihong > > > Cc: dev@dpdk.org; nd > > > Subject: RE: [dpdk-dev] [PATCH v2 08/16] vhost: add flush function fo= r > > > burst enqueue > > > > > > Hi Marvin, > > > > > > One typo and one comment about the barrier. > > > > > > /Gavin > > > > > > > -----Original Message----- > > > > From: dev On Behalf Of Marvin Liu > > > > Sent: Friday, September 20, 2019 12:37 AM > > > > To: maxime.coquelin@redhat.com; tiwei.bie@intel.com; > > > > zhihong.wang@intel.com > > > > Cc: dev@dpdk.org; Marvin Liu > > > > Subject: [dpdk-dev] [PATCH v2 08/16] vhost: add flush function for > > burst > > > > enqueue > > > > > > > > Flush used flags when burst enqueue function is finished. Descripto= r's > > > > flags are pre-calculated as them will be reset by vhost. > > > s/them/they > > > > > > > Thanks. > > > > > > > > > > Signed-off-by: Marvin Liu > > > > > > > > diff --git a/lib/librte_vhost/vhost.h b/lib/librte_vhost/vhost.h > > > > index 000648dd4..9c42c7db0 100644 > > > > --- a/lib/librte_vhost/vhost.h > > > > +++ b/lib/librte_vhost/vhost.h > > > > @@ -39,6 +39,9 @@ > > > > > > > > #define VHOST_LOG_CACHE_NR 32 > > > > > > > > +#define VIRTIO_RX_USED_FLAG (0ULL | VRING_DESC_F_AVAIL | > > > > VRING_DESC_F_USED \ > > > > + | VRING_DESC_F_WRITE) > > > > +#define VIRTIO_RX_USED_WRAP_FLAG (VRING_DESC_F_WRITE) > > > > #define PACKED_DESCS_BURST (RTE_CACHE_LINE_SIZE / \ > > > > sizeof(struct vring_packed_desc)) > > > > > > > > diff --git a/lib/librte_vhost/virtio_net.c > > > b/lib/librte_vhost/virtio_net.c > > > > index e2787b72e..8e4036204 100644 > > > > --- a/lib/librte_vhost/virtio_net.c > > > > +++ b/lib/librte_vhost/virtio_net.c > > > > @@ -169,6 +169,51 @@ update_shadow_packed(struct > vhost_virtqueue > > > > *vq, > > > > vq->shadow_used_packed[i].count =3D count; > > > > } > > > > > > > > +static __rte_always_inline void > > > > +flush_burst_packed(struct virtio_net *dev, struct vhost_virtqueue = *vq, > > > > + uint64_t *lens, uint16_t *ids, uint16_t flags) > > > > +{ > > > > + uint16_t i; > > > > + > > > > + UNROLL_PRAGMA(PRAGMA_PARAM) > > > > + for (i =3D 0; i < PACKED_DESCS_BURST; i++) { > > > > + vq->desc_packed[vq->last_used_idx + i].id =3D ids[i]; > > > > + vq->desc_packed[vq->last_used_idx + i].len =3D lens[i]; > > > > + } > > > > + > > > > + UNROLL_PRAGMA(PRAGMA_PARAM) > > > > + for (i =3D 0; i < PACKED_DESCS_BURST; i++) { > > > > + rte_smp_wmb(); > > > Should this rte_smp_wmb() be moved above the loop? It guarantees the > > > orderings of updates of id, len happens before the flags, > > > But all the flags of different descriptors should not be ordered. > > > > > Hi Gavin, > > For each descriptor, virtio driver will first check flags and then chec= k > > read barrier, at the last driver will read id and length. > > So wmb here is to guarantee that id and length are updated before flags= . > > And afterwards wmb is to guarantee the sequence. > > > Gavin, > Checked with master branch, flags store sequence is not needed. > But in my environment, performance will be a litter better if ordered fla= gs > store. > I think it may be harmless to place wmb here. How about your idea? The smp barrier on x86 is a compiler barrier only, it ensure data consisten= cy, it will not help performance, The slight better performance should come from run-to-run variances or syst= em noise or sth else.=20 The barrier will dampen the performance on weak memory ordered platforms, l= ike aarch64.=20 /Gavin >=20 > > Thanks, > > Marvin > > > > > > + vq->desc_packed[vq->last_used_idx + i].flags =3D flags; > > > > + } > > > > + > > > > + vhost_log_cache_used_vring(dev, vq, vq->last_used_idx * > > > > + sizeof(struct vring_packed_desc), > > > > + sizeof(struct vring_packed_desc) * > > > > + PACKED_DESCS_BURST); > > > > + vhost_log_cache_sync(dev, vq); > > > > + > > > > + vq->last_used_idx +=3D PACKED_DESCS_BURST; > > > > + if (vq->last_used_idx >=3D vq->size) { > > > > + vq->used_wrap_counter ^=3D 1; > > > > + vq->last_used_idx -=3D vq->size; > > > > + } > > > > +} > > > > + > > > > +static __rte_always_inline void > > > > +flush_enqueue_burst_packed(struct virtio_net *dev, struct > > > > vhost_virtqueue *vq, > > > > + uint64_t *lens, uint16_t *ids) > > > > +{ > > > > + uint16_t flags =3D 0; > > > > + > > > > + if (vq->used_wrap_counter) > > > > + flags =3D VIRTIO_RX_USED_FLAG; > > > > + else > > > > + flags =3D VIRTIO_RX_USED_WRAP_FLAG; > > > > + > > > > + flush_burst_packed(dev, vq, lens, ids, flags); > > > > +} > > > > + > > > > static __rte_always_inline void > > > > update_enqueue_shadow_packed(struct vhost_virtqueue *vq, > uint16_t > > > > desc_idx, > > > > uint32_t len, uint16_t count) > > > > @@ -950,6 +995,7 @@ virtio_dev_rx_burst_packed(struct virtio_net > *dev, > > > > struct vhost_virtqueue *vq, > > > > struct virtio_net_hdr_mrg_rxbuf *hdrs[PACKED_DESCS_BURST]; > > > > uint32_t buf_offset =3D dev->vhost_hlen; > > > > uint64_t lens[PACKED_DESCS_BURST]; > > > > + uint16_t ids[PACKED_DESCS_BURST]; > > > > > > > > uint16_t i; > > > > > > > > @@ -1013,6 +1059,12 @@ virtio_dev_rx_burst_packed(struct > virtio_net > > > > *dev, struct vhost_virtqueue *vq, > > > > pkts[i]->pkt_len); > > > > } > > > > > > > > + UNROLL_PRAGMA(PRAGMA_PARAM) > > > > + for (i =3D 0; i < PACKED_DESCS_BURST; i++) > > > > + ids[i] =3D descs[avail_idx + i].id; > > > > + > > > > + flush_enqueue_burst_packed(dev, vq, lens, ids); > > > > + > > > > return 0; > > > > } > > > > > > > > -- > > > > 2.17.1