From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 196ACA00BE; Mon, 16 May 2022 08:11:04 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id AFA2440A7A; Mon, 16 May 2022 08:11:03 +0200 (CEST) Received: from mga03.intel.com (mga03.intel.com [134.134.136.65]) by mails.dpdk.org (Postfix) with ESMTP id 82A3140A79 for ; Mon, 16 May 2022 08:11:01 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1652681461; x=1684217461; h=from:to:cc:subject:date:message-id:references: in-reply-to:content-transfer-encoding:mime-version; bh=0bN2mgsJ5wUk4a91W6P0tBnM3cxl1LkrZ6l424mqouE=; b=jZ5w4hXgeJ5gB0U5ZnhnLMluBayheOiIaX1BbHBjuk0EzHo7J19+l6tF 3qae0kHS39jogYlV+wJHiufczr0xG/gXgQiJ0tpBQH8CuAEBBlXIqY29d Ov4oR4UHw8oUDWj72wx8I2lA0QEL4FlkrDZj+cNn5nZJ6kCtoWjVhn5F3 g2AkRktD9plt+si0sHR+sKYZjHisjcQj8PVVCyN66dukIbLZnAwdSeCZK JeLzwJeO7rewlLxdmMMO65Uu1VcieGJv+xBactHKgUgx1nic8DYjZeQUq Qw/FGHTmbGhXClxJWuAzgNh2qwj7gmCshysTSUF/AUAz9Hm9WQvWmt28h w==; X-IronPort-AV: E=McAfee;i="6400,9594,10348"; a="270694066" X-IronPort-AV: E=Sophos;i="5.91,229,1647327600"; d="scan'208";a="270694066" Received: from orsmga002.jf.intel.com ([10.7.209.21]) by orsmga103.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 15 May 2022 23:10:58 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.91,229,1647327600"; d="scan'208";a="555111575" Received: from fmsmsx601.amr.corp.intel.com ([10.18.126.81]) by orsmga002.jf.intel.com with ESMTP; 15 May 2022 23:10:57 -0700 Received: from fmsmsx609.amr.corp.intel.com (10.18.126.89) by fmsmsx601.amr.corp.intel.com (10.18.126.81) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2308.27; Sun, 15 May 2022 23:10:56 -0700 Received: from fmsmsx606.amr.corp.intel.com (10.18.126.86) by fmsmsx609.amr.corp.intel.com (10.18.126.89) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2308.27; Sun, 15 May 2022 23:10:56 -0700 Received: from FMSEDG603.ED.cps.intel.com (10.1.192.133) by fmsmsx606.amr.corp.intel.com (10.18.126.86) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2308.27 via Frontend Transport; Sun, 15 May 2022 23:10:56 -0700 Received: from NAM11-DM6-obe.outbound.protection.outlook.com (104.47.57.171) by edgegateway.intel.com (192.55.55.68) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.1.2308.27; Sun, 15 May 2022 23:10:56 -0700 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=dETsNNT0DFkxlyvemvVXFJOnZb4XJYsdzs++zkHOPYXBbALdggDGl0YJEXe04fpWFO6Ky2JLQvxMhqfdtbpUY706hW4854F7fV5Y+g25BrQV93BLOYHTKiZ44uVkyxW6fdV4FZfnP1pFKBkvVIg/h7cB6Vy0sYAqqENiQfuuNhJrLfyVEd/LfT3RVVXqDifwfkewu1TZflJMFu1LOblGIuID7ZngAOpAQVq16F1W1FYSbn2nFVlegS1e2VQ7bi76syOzogL0dxS1viX3kHKW1ljCwO97Q8u7nsWpd8zbBe4kqpBza/Gh6rBe+JSFUi6HxH3ukKHgtJtRKl2Z7qo4yg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=5oF0eZPvml/HESNEQwgDSPWuodOtZi+atxJyIrw+9C8=; b=Q5Vzcmm9JuUsGO4kD5steUVNoHrIDZ2A355txjMH6wMhM9xRwovaW6zpX24p2Ojdinmgcj57IGa35NxTyaj1BHPpM5gY+xiDpyYSlpUxDUHPwkMFDtzlk01CRr8xFxahzedQ0P8mzsTPnpGlw7PAn38Y0ifYNjR69xjkoOm6w9LqUHdRphWi/cl5A5zRG6X7mzbVA2zaSo8jmIero+qLzmr8VxsbbGWlPMjZo7N+p54E+kK4mfaK7vSHhmIrimmZ4Yf/vtNt3husnvcX6lo6XR3E7tPhE5uCY/ghl+xNI0PkJsvxiGGAExiYGTHHTcQ7gFRsrRN2Qq5EzwPmJJm3NA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=intel.com; dmarc=pass action=none header.from=intel.com; dkim=pass header.d=intel.com; arc=none Received: from BN9PR11MB5513.namprd11.prod.outlook.com (2603:10b6:408:102::11) by BN6PR11MB1444.namprd11.prod.outlook.com (2603:10b6:405:a::19) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5250.13; Mon, 16 May 2022 06:10:54 +0000 Received: from BN9PR11MB5513.namprd11.prod.outlook.com ([fe80::31c6:2ec8:2f71:42da]) by BN9PR11MB5513.namprd11.prod.outlook.com ([fe80::31c6:2ec8:2f71:42da%2]) with mapi id 15.20.5250.018; Mon, 16 May 2022 06:10:54 +0000 From: "Ding, Xuan" To: "Hu, Jiayu" , "maxime.coquelin@redhat.com" , "Xia, Chenbo" CC: "dev@dpdk.org" , "Jiang, Cheng1" , "Pai G, Sunil" , "liangma@liangbit.com" , "Wang, YuanX" Subject: RE: [PATCH v7 4/5] vhost: support async dequeue for split ring Thread-Topic: [PATCH v7 4/5] vhost: support async dequeue for split ring Thread-Index: AQHYaM9xHP9rv9UvY0STuwW87I8CFq0hAMIAgAAB4EA= Date: Mon, 16 May 2022 06:10:54 +0000 Message-ID: References: <20220407152546.38167-1-xuan.ding@intel.com> <20220516024325.96067-1-xuan.ding@intel.com> <20220516024325.96067-5-xuan.ding@intel.com> <4eedc57dfe484bd0b7d76735f944e9c9@intel.com> In-Reply-To: <4eedc57dfe484bd0b7d76735f944e9c9@intel.com> Accept-Language: zh-CN, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: dlp-product: dlpe-windows dlp-reaction: no-action dlp-version: 11.6.401.20 authentication-results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=intel.com; x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: 8d81ba3e-e76c-4a8d-6697-08da3702d5bc x-ms-traffictypediagnostic: BN6PR11MB1444:EE_ x-ld-processed: 46c98d88-e344-4ed4-8496-4ed7712e255d,ExtAddr x-microsoft-antispam-prvs: x-ms-exchange-senderadcheck: 1 x-ms-exchange-antispam-relay: 0 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: M7UAQzThppIGODnnrvksMMxdJBdAiqX9QaohoGa3v5lVjuVLsl6YCxh3xtfV4S5oE0K/jfI9uA/x0Rn0BllxnRQQi+6BoSvJ3+GA68SEWZPjSiCP1/mrFcUm9WeOiK1lAMpRLVI4hkFqPuP8nubgmr97jwlrDKJcOxE6GrS+6JCMbP35xLBvmy2tnWJ1XMYFeYM34COYEViaYQ+2QcI5GII0mnnBTv+WU4de1d+voOIlCWn+esZrZZCuXC3A9iuQtvbVbTCw7njlT4TWlaIdB9/ZdJ7bpgtcXSUmerEtay/+XIu4LplqYKCDXxFkFZ/dLLsiKZfnm34px9sDjynYIbY7I+QlR0HTNMExFPO0cLlCMKl3qxhJ6hhT97hYelisDAZHmJX9JGuqOflj2W5JEfTP7qk+5NuAN4zeOLmNujHoUUSgMbXTOpNRSm+kiC8afc5TG2AHFfWu4IrkcH8MrWvxOSUSqNxFV3MeEWbUf4nMxlo8qhMDH0eZzZgh/TsQm0v58J25uNdCriEfI/p59TPGthwlOi2dSKreGQbHTUnTlFNINq4BFRCQkNihPyNHqfrItZa48+jXIXtlbwtzunpwGTL2C1sJL8zaoTQpIycutRSysUBV3+2ukduC6LbCfV/HbkPtTM1UAEgd0KO2wRYAdaMzjmJGJiPlXy2Yxzq/tERDF6IvKjy+G35OXyxs75lNTTSgkwdvVQ1TLklaEQ== x-forefront-antispam-report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:BN9PR11MB5513.namprd11.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230001)(366004)(4326008)(86362001)(76116006)(2906002)(33656002)(38100700002)(7696005)(6506007)(66446008)(66556008)(508600001)(66946007)(64756008)(38070700005)(55016003)(53546011)(82960400001)(83380400001)(66476007)(71200400001)(122000001)(52536014)(8676002)(8936002)(54906003)(107886003)(186003)(6636002)(110136005)(5660300002)(9686003)(316002)(26005); DIR:OUT; SFP:1102; x-ms-exchange-antispam-messagedata-chunkcount: 1 x-ms-exchange-antispam-messagedata-0: =?us-ascii?Q?21PcpMTyZSE7Nc0H4HbaJ9T2bMnZAHjnImV1qeRonTx9I6kq64ccbKjCEYMe?= =?us-ascii?Q?DwxE0DIAh33pkpUBAodvEJSbk1ELTr8835j25tzjSqVwEUcGvpBrFz8tUYKO?= =?us-ascii?Q?PbLiZq6dzxjkoiUSEX6ycFaf636tHRlEmusMwGd9diqrP8MO6VU2DH6b5Emi?= =?us-ascii?Q?cnVEjXuI/3Cp7yJCU3jUGc9Iv2Bhe1lqd+vzVhSKC7uQSfzW9q00HQaA/IV1?= =?us-ascii?Q?3MqR5yYWBoNlL43AToSVq7lbxy6WaasjAkNfsP17vt6DZ3FOHdgSGVsrE5lB?= =?us-ascii?Q?PPZpvV0UCQfAb5JAjatwN4nb6X+8gRjzB0sa8OWC309mtngaa+22xYH/KbkM?= =?us-ascii?Q?sxWEv7QB0cxyrEDd5GKMTDP0wxOXC6nlMgKU+4qznUrwvkCL3JnONgrHnd5j?= =?us-ascii?Q?lDYaJtE4pZrNhzLEPWyb9XgGYRrv9U/BPLRO4XhWKk1yVdzhUBMVoAMUDuDk?= =?us-ascii?Q?PBISbGzunWh1E2lhWQFwL/M+SvHh69FE/XfAa+wFTQ3vRlDXvxsq8aq8o5Gb?= =?us-ascii?Q?DhlQfvr6z9Fl3FD2pf8nD/SK99Ax/A1UWybP92nLqkp1JsmIBoVnE7q3rn3d?= =?us-ascii?Q?e+nP8TDFi/SXXpRpC9ZoZVQKL7e7nN0cNg+vpC/BRmZ7CRpQ1B9BpijG7FMd?= =?us-ascii?Q?fWhZ9DMCWaFEmlQjXdQ6vqpedZuepPAzcIOp5CjmbjKk8rd/JzqrWMtgW4Hy?= =?us-ascii?Q?AyUcQCJGTYD/0vsjo2CiOtXR9Qm9aUYD3AG3Do/DKnTMdNcL8R7IAAuJ3wR6?= =?us-ascii?Q?/Yj9492lKWk3p6WWcRxml2Tja3MWG7VTA3EgKjKBCYVj00rDjlmgbBJIhB9Z?= =?us-ascii?Q?K26osYbHqs9k7d3FxZr0KLLk7k/0kgHBSzO0UdZU1ATzgb8OBno28Jw1CPCa?= =?us-ascii?Q?3JwZD97ZInWNUbCu8Ukak8BjxRsTx5+ZNCI/zWk/gbkDiH8FIIDDbvBTsxPw?= =?us-ascii?Q?Qy7QRKMOIWNPky7vCl58l1QzbdjOuUWC2A91U/LI/QI44vCdATJgoS8ZE2gT?= =?us-ascii?Q?VK5PqOtUlw6X7nl1759mwWYR3oHp50wxg/fRgo0sbuS7+I5S8g6+WVcRBXuR?= =?us-ascii?Q?OuLmebuy8sO65OUmI/Y3bApEHqryqc3RGj1yGxQp4fG9sg2XZWiOpytGpFRm?= =?us-ascii?Q?md3yusgWzpZrvE+4+ItUYRuMk4ZFwrY9I5PI6pxYHtxNk2ID6WPpmZNa9r8f?= =?us-ascii?Q?026YVtzQNculy/r8vwPoQ1POIA12+ZAx8IBiwqcL/bB6UlurLVF7+wWJG+0K?= =?us-ascii?Q?WCfPYnJQ0Tkp01AZB5D9cUtpqYbE7LiMmMrVMo/XGXd4XwBFv0VyVQ86dbo+?= =?us-ascii?Q?1Ux9JCPo6dE0bcPToZ/wjc8UB2SZXRs7UY8wPNf5ZT3NgGyokQS5Y/Neg03w?= =?us-ascii?Q?uLEIa0SGE6T8tlC5r3NPD5/GAlgSp8bqwp+S9DASgpsowMUz3tJjjbAAb0mF?= =?us-ascii?Q?r9UeNHc5YuO1it8ukkyB0U8fegwqZEKqyLZhcb0g3oSqaXVm7UKt/+/lkNWo?= =?us-ascii?Q?uI+t1g6p0MFRaN9ZwoSJkiAINxI2F0moDJ+iUrJ0Koquw4jjiseuladQ9ki8?= =?us-ascii?Q?7qnHJ7N+DiX5pgWUfVxll57qybGkPknGhz29AcUt/M9QKwKjOqW45QGRFiQ+?= =?us-ascii?Q?7ANkV0fRyZ5fKQb4XRaMPjwpbnG0ochffJOw6aDMwyPUMBTbWQK+q2IOxQ6c?= =?us-ascii?Q?8fGEMbOlNOjBW4zAOAYliKxT7obypM5/rJEAubUOyDGdywbtiJkPRmE5dPSr?= =?us-ascii?Q?zwwFmOlAQA=3D=3D?= Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: BN9PR11MB5513.namprd11.prod.outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: 8d81ba3e-e76c-4a8d-6697-08da3702d5bc X-MS-Exchange-CrossTenant-originalarrivaltime: 16 May 2022 06:10:54.1234 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 46c98d88-e344-4ed4-8496-4ed7712e255d X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: fkJUOtSViOfqU+9OIbKb6SzM0kk4LwIjZFF477ns84SJtUG3INt/ixVEqxBqvxQ/W2Xq3qz8meJJMwF+838vnQ== X-MS-Exchange-Transport-CrossTenantHeadersStamped: BN6PR11MB1444 X-OriginatorOrg: intel.com X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Hi Jiayu, > -----Original Message----- > From: Hu, Jiayu > Sent: Monday, May 16, 2022 1:53 PM > To: Ding, Xuan ; maxime.coquelin@redhat.com; Xia, > Chenbo > Cc: dev@dpdk.org; Jiang, Cheng1 ; Pai G, Sunil > ; liangma@liangbit.com; Wang, YuanX > > Subject: RE: [PATCH v7 4/5] vhost: support async dequeue for split ring >=20 > Hi Xuan, >=20 > > -----Original Message----- > > From: Ding, Xuan > > Sent: Monday, May 16, 2022 10:43 AM > > To: maxime.coquelin@redhat.com; Xia, Chenbo > > Cc: dev@dpdk.org; Hu, Jiayu ; Jiang, Cheng1 > > ; Pai G, Sunil ; > > liangma@liangbit.com; Ding, Xuan ; Wang, YuanX > > > > Subject: [PATCH v7 4/5] vhost: support async dequeue for split ring > > > > From: Xuan Ding > > > > This patch implements asynchronous dequeue data path for vhost split > > ring, a new API rte_vhost_async_try_dequeue_burst() is introduced. > > > > Signed-off-by: Xuan Ding > > Signed-off-by: Yuan Wang > > Tested-by: Yvonne Yang > > Reviewed-by: Maxime Coquelin > > --- > > doc/guides/prog_guide/vhost_lib.rst | 7 + > > doc/guides/rel_notes/release_22_07.rst | 5 + > > lib/vhost/rte_vhost_async.h | 37 +++ > > lib/vhost/version.map | 2 +- > > lib/vhost/virtio_net.c | 337 +++++++++++++++++++++++++ > > 5 files changed, 387 insertions(+), 1 deletion(-) > > > > diff --git a/doc/guides/prog_guide/vhost_lib.rst > > b/doc/guides/prog_guide/vhost_lib.rst > > index f287b76ebf..09c1c24b48 100644 > > --- a/doc/guides/prog_guide/vhost_lib.rst > > +++ b/doc/guides/prog_guide/vhost_lib.rst > > @@ -282,6 +282,13 @@ The following is an overview of some key Vhost > > API > > functions: > > Clear inflight packets which are submitted to DMA engine in vhost > > async data > > path. Completed packets are returned to applications through ``pkts`= `. > > > > +* ``rte_vhost_async_try_dequeue_burst(int vid, uint16_t queue_id, > > + struct rte_mempool *mbuf_pool, struct rte_mbuf **pkts, uint16_t > > +count, int *nr_inflight, uint16_t dma_id, uint16_t vchan_id)`` >=20 > In dmadev library, dma_id is int16_t, rather than uint16_t. Thanks for the reminder. I will fix it in v8. >=20 > > + > > + Receives (dequeues) ``count`` packets from guest to host in async > > + data path, and stored them at ``pkts``. > > + > > Vhost-user Implementations > > -------------------------- > > > > diff --git a/doc/guides/rel_notes/release_22_07.rst > > b/doc/guides/rel_notes/release_22_07.rst > > index 88b1e478d4..564d88623e 100644 > > --- a/doc/guides/rel_notes/release_22_07.rst > > +++ b/doc/guides/rel_notes/release_22_07.rst > > @@ -70,6 +70,11 @@ New Features > > Added an API which can get the number of inflight packets in > > vhost async data path without using lock. > > > > +* **Added vhost async dequeue API to receive pkts from guest.** > > + > > + Added vhost async dequeue API which can leverage DMA devices to > > + accelerate receiving pkts from guest. > > + > > Removed Items > > ------------- > > > > diff --git a/lib/vhost/rte_vhost_async.h b/lib/vhost/rte_vhost_async.h > > index > > 70234debf9..2789492e38 100644 > > --- a/lib/vhost/rte_vhost_async.h > > +++ b/lib/vhost/rte_vhost_async.h > > @@ -204,6 +204,43 @@ uint16_t > rte_vhost_clear_queue_thread_unsafe(int > > vid, uint16_t queue_id, __rte_experimental int > > rte_vhost_async_dma_configure(int16_t dma_id, uint16_t vchan_id); > > > > +/** > > + * @warning > > + * @b EXPERIMENTAL: this API may change, or be removed, without prior > > +notice > > + * > > + * This function tries to receive packets from the guest with > > +offloading > > + * copies to the async channel. The packets that are transfer > > +completed >=20 > In rte_vhost_async.h, DMA vChannel is referred as async channel or async > copy engine. But I think it's better to replace all with DMA vChannel to = be > consistent with DPDK. Thanks for your suggestion, I will refine the API description in next versi= on. >=20 > > + * are returned in "pkts". The other packets that their copies are > > +submitted to > > + * the async channel but not completed are called "in-flight packets". > > + * This function will not return in-flight packets until their copies > > +are > > + * completed by the async channel. > > + * > > + * @param vid > > + * ID of vhost device to dequeue data > > + * @param queue_id > > + * ID of virtqueue to dequeue data > > + * @param mbuf_pool > > + * Mbuf_pool where host mbuf is allocated > > + * @param pkts > > + * Blank array to keep successfully dequeued packets > > + * @param count > > + * Size of the packet array > > + * @param nr_inflight > > + * >=3D 0: The amount of in-flight packets >=20 > Better to add more descriptions about the meaning of "nr_inflight". Same as above. Regards, Xuan >=20 > Thanks, > Jiayu