From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id F40BCA0C41; Thu, 30 Sep 2021 08:54:17 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id E1C75410ED; Thu, 30 Sep 2021 08:54:17 +0200 (CEST) Received: from mga04.intel.com (mga04.intel.com [192.55.52.120]) by mails.dpdk.org (Postfix) with ESMTP id AF3D0410EC for ; Thu, 30 Sep 2021 08:54:15 +0200 (CEST) X-IronPort-AV: E=McAfee;i="6200,9189,10122"; a="223229434" X-IronPort-AV: E=Sophos;i="5.85,335,1624345200"; d="scan'208";a="223229434" Received: from orsmga001.jf.intel.com ([10.7.209.18]) by fmsmga104.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 29 Sep 2021 23:54:14 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.85,335,1624345200"; d="scan'208";a="521100413" Received: from orsmsx601.amr.corp.intel.com ([10.22.229.14]) by orsmga001.jf.intel.com with ESMTP; 29 Sep 2021 23:54:14 -0700 Received: from orsmsx608.amr.corp.intel.com (10.22.229.21) by ORSMSX601.amr.corp.intel.com (10.22.229.14) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2242.12; Wed, 29 Sep 2021 23:54:14 -0700 Received: from orsmsx611.amr.corp.intel.com (10.22.229.24) by ORSMSX608.amr.corp.intel.com (10.22.229.21) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2242.12; Wed, 29 Sep 2021 23:54:13 -0700 Received: from orsedg603.ED.cps.intel.com (10.7.248.4) by orsmsx611.amr.corp.intel.com (10.22.229.24) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2242.12 via Frontend Transport; Wed, 29 Sep 2021 23:54:13 -0700 Received: from NAM11-BN8-obe.outbound.protection.outlook.com (104.47.58.173) by edgegateway.intel.com (134.134.137.100) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.1.2242.12; Wed, 29 Sep 2021 23:54:13 -0700 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=daDMLsXLgWEKt+4drq3wREV/7CyUEOgBjP7RNMn+6HoVzYfm3cC2+ZmSUnaOOvTdCw1LWQZe87A+9YP20Gq0YtLVinHhj7tgjT3D0uplwhUXLN5+DXA0Mu8j7/NjumSj44veILqkUZAjtIdEj1eNIvn+jXeujTQsGPHO50L1EBmoHJ47+cYnj5IO5aOWwtELs94GZDGjUl7uGEdDsWP1hZssD4GTqgPsyyygKLgVymUoZDcd1NBctm6isPQJFJU1Nl65eVTHrnfZXIJsQwfWe8QluMRlmWr10BDiJBFRJTTHHoXJWlitS6JrhOgHpW8LoRhtgEW/Lg9zX0bD6dSNOg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=ihDYcdTCnq3Neq98bkb+7Ptd1vf8s0BnVIMdpbBZnvQ=; b=h24awQVw5N1KYt8VbILYtJqmw8LH+H01Fx5iGKgyC6oRu2X8JtWweITN1PgzVE317k+CfrDk/IdibPQfp6BXDlNQCXI07S275OtUlufoO6orF8sYi+nRZxQS1lLhkOmGWQLBLeQihstpeYrBMoHWODei9HA/ZYmpD1Bwp2gusdNbsmbr87vwGCcSInYTEkU26S+F+MHvKtAoYB26svPcaQSY7PSN2kPgBFoSOHPyCwtCLoYuIig5WfOMP8Z8L+NDF13QgSRVVtPCOy3GR+aWC1Td9nWACUf0TOnXiideqtHnZ/g5qk4AShltxAWVLmNulFsgyn9MUzvs9GOPcyesRg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=intel.com; dmarc=pass action=none header.from=intel.com; dkim=pass header.d=intel.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=intel.onmicrosoft.com; s=selector2-intel-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=ihDYcdTCnq3Neq98bkb+7Ptd1vf8s0BnVIMdpbBZnvQ=; b=w/SrwjRP9uHXJ+UzuWxXP002krC9T/HBd2Wzh0G073ygE5hrJR/cZhj9RhLFXzUxIi8LL8z4TPZEnriGtfmpLRcbUJJTLl7ENW4aD03bzE3j0MJwlChXhQEbP25LBk0DjRhysl2zFReLfy3mEpPOV8gVCT7NvpXcCgxxs/AGeHo= Received: from BN9PR11MB5513.namprd11.prod.outlook.com (2603:10b6:408:102::11) by BN6PR11MB1972.namprd11.prod.outlook.com (2603:10b6:404:105::21) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.15; Thu, 30 Sep 2021 06:54:11 +0000 Received: from BN9PR11MB5513.namprd11.prod.outlook.com ([fe80::99b5:c482:4da6:799e]) by BN9PR11MB5513.namprd11.prod.outlook.com ([fe80::99b5:c482:4da6:799e%7]) with mapi id 15.20.4544.021; Thu, 30 Sep 2021 06:54:11 +0000 From: "Ding, Xuan" To: "Wang, YuanX" , "dev@dpdk.org" CC: "maxime.coquelin@redhat.com" , "Xia, Chenbo" , "Hu, Jiayu" , "Jiang, Cheng1" , "Ma, WenwuX" , "Yang, YvonneX" , "Pai G, Sunil" Thread-Topic: [PATCH v3 2/2] vhost: add thread-safe API for clearing in-flight packets in async vhost Thread-Index: AQHXr5FODzjfu7H4V0WmsxXqW901mKu8JdPQ Date: Thu, 30 Sep 2021 06:54:10 +0000 Message-ID: References: <20210909065807.812145-1-yuanx.wang@intel.com> <20210922085546.54758-1-yuanx.wang@intel.com> <20210922085546.54758-3-yuanx.wang@intel.com> In-Reply-To: <20210922085546.54758-3-yuanx.wang@intel.com> Accept-Language: zh-CN, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: dlp-product: dlpe-windows dlp-reaction: no-action dlp-version: 11.6.200.16 authentication-results: intel.com; dkim=none (message not signed) header.d=none;intel.com; dmarc=none action=none header.from=intel.com; x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: 64119747-bb14-4bc9-1bb6-08d983df1b80 x-ms-traffictypediagnostic: BN6PR11MB1972: x-ld-processed: 46c98d88-e344-4ed4-8496-4ed7712e255d,ExtAddr x-ms-exchange-transport-forked: True x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:269; x-ms-exchange-senderadcheck: 1 x-ms-exchange-antispam-relay: 0 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: csrSuzEhrvXCwaYetZV98d86UcCxd9axMbtwBSuHHTj8ER1je18jydGLGstsWFZmnUY6ZOxukpjzVbuOyy3HdaZ5UaWhi+lI0+gIrjaeih9ebEFe2XuWHxzmxhkUlzDsaJcM0vxm3Jwwh1o88+Wj+EKPax4SfeOQ01iFBSGndsfBqAO5PgR/gIB4ENlOHrGgXP0+X4H2oXMa8tEXNwstGb2LvN2gPKrgcZ80BQoms8JkPb8wnZOKn9KvKb1XkdTOZbWgEqGCTsBO1ebx+LnWEpMjshVqs57Ey/z9RhYGxZAdhz9lJHsfM3cUwQH5zPASqJgLLLgNR49vdm8MSoj64JL0fRrAybVB8fst86gSPRDOdySDcXVk4rtaOLDXKpn1efk/hM2JTrsTwgMPNJeCukRrnNnL9OIUm9iOFpOFSxKQ7WJWzERpcPhyiN5tmTi+rvHYba913E+my6C1qIohFUkYGjzYi1x3PAMWamPv5i731epflaIq0Xc8RtA8L2SeEtzvGc4n6d5+RFdK9Ya3LSI+oF+3BMDR7+dPinNbVtRhFsf0cYQhKXwkKEMP7ojJnOD6wJ373zmJKBSs/BfDYhLn1s2UYg9vuR5JiYhkx41RmEYrOQtPIPG43SpqnH4Vt31IPGF7W8U2NrFFIcaIRrMwaZaoxBBOEZanRh5QFGy4L5E9HwgxRdHUGzRN4FPNPFv4CI8yQxjzLHLbOy3t2w== x-forefront-antispam-report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:BN9PR11MB5513.namprd11.prod.outlook.com; PTR:; CAT:NONE; SFS:(4636009)(366004)(186003)(66556008)(9686003)(55016002)(33656002)(8676002)(38070700005)(53546011)(52536014)(5660300002)(8936002)(508600001)(71200400001)(6506007)(66446008)(66946007)(64756008)(66476007)(107886003)(26005)(76116006)(4326008)(7696005)(110136005)(54906003)(316002)(83380400001)(38100700002)(122000001)(2906002)(86362001); DIR:OUT; SFP:1102; x-ms-exchange-antispam-messagedata-chunkcount: 1 x-ms-exchange-antispam-messagedata-0: =?Windows-1252?Q?66urFSikvgeCJj3hwmr3u2+NZTwcZ2neKTyIQIywhBQ6Z42LegZrcJFa?= =?Windows-1252?Q?LSxeEvtaw/sokxUbJvSTZFOldK6I5TYsHjjIgZYzElnNXs+jDghXulrQ?= =?Windows-1252?Q?5AMCQ7sfFyTszTJOJgu8LuZ3p60Rwm3RR9rTY9Nr9b/hOf/3nlDq2L+P?= =?Windows-1252?Q?sFwrIyKdci9z3yWlS/b42ethsDaq2XFdhk8MmUf9u0JaYK/Yiy/7Wft+?= =?Windows-1252?Q?QYerAf1vGZM9VMHwD2MxRdEgFgHH6+cOIQ0wxk0R/A0H0QyOaHa5fy03?= =?Windows-1252?Q?Sc4lmjOM8pVNugJIFb6oXFSmxVZyYvcvYWfvOoWkfdekoB86M+WnFAdU?= =?Windows-1252?Q?0cdptGsr/Y+Biq47IE8IwrsRPxSDgXVb0quU57HsQ/5txupTQStpLQLs?= =?Windows-1252?Q?ZGE1VbAm2YPw8tcbkrVlC/Mr/aeJQdwBSTVBHFE0qNsH14r2wwTU0xow?= =?Windows-1252?Q?P8MMl/APau/X8lDIUEYwEmpwFLm+SdV0iADOA2OMsoPHsdT+KBmeG5mT?= =?Windows-1252?Q?PQWJRqdhYawy+GaMr5AwPgMuwP/ymnx4mbTWjboeVD1kRJYnhCFY1neW?= =?Windows-1252?Q?/3K2UG33Y8wXGcM3zGI/TiDuTY45wqpzHYMmuAYtW1uvIZncY2jTliyU?= =?Windows-1252?Q?qVGDonn63hC7CbmtrmpBr8imTk2DwR+G6olO84e1vNlGCv0kRMLQb1+k?= =?Windows-1252?Q?HHg2PLwdGMBClUJalcrfwDdZspYsyq/VlnZFgOIFoqtXBYF7Pya+QcKn?= =?Windows-1252?Q?siLItQIN2JTsiyUVZ4WYPFesikWeVST88F2eqt4wBbGdj4KrDMiUOmyJ?= =?Windows-1252?Q?nIDzgXr0Oae72hfbnFWGvT4qDKrVF2M+7xuIevqVl13oLsGyoDCoY2Y4?= =?Windows-1252?Q?gtw7kL1hKYJreBbiv7cXSmUAz1HJV9WlTUXJ8fDUxGljy/vtec9tZHvn?= =?Windows-1252?Q?i8iBggoneBnpGCJSt8AOGBo3MxoxsEkVkj47B3LO7o9uqxTVaSQFiqr4?= =?Windows-1252?Q?+RTYyDYfzXlfj2xhxNvVxLnzbeRLahfvWrqOUBTFYyv6Zn5ONNYi9Qnd?= =?Windows-1252?Q?z+22C7mohKqwcWtGnHdwPwFBT4+ZcihotL6hqGifJwC0OdRz/D5KFwLs?= =?Windows-1252?Q?62Tek7/z3tqlrXXpkc8JPhw0DGOZjg7wtNI1TEQaXJoO2+LpOYj8YQyJ?= =?Windows-1252?Q?dzdNx/COKSbGIcuK6eQ0XRekAVEoAoVXCrb4/w5Kx5dfuLzCnGqXGQP5?= =?Windows-1252?Q?VQRg+mx9MMNYOBGlFa/APYvP6mErBVbFRwjMp3cIAebtO8Gio4wzuwyb?= =?Windows-1252?Q?k0nzc0KN1B/Z8Ofr7lxRwTBxEswKWR6rH6JuhOCZ8FkJiLpeACbV9fCG?= =?Windows-1252?Q?g9oleKWryC2++TrbyfuggjXHY4HqpdwtAnUa3nkHBLMM+0/zqdGx5WEt?= Content-Type: text/plain; charset="Windows-1252" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: BN9PR11MB5513.namprd11.prod.outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: 64119747-bb14-4bc9-1bb6-08d983df1b80 X-MS-Exchange-CrossTenant-originalarrivaltime: 30 Sep 2021 06:54:10.5161 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 46c98d88-e344-4ed4-8496-4ed7712e255d X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: IBS9/1/qGugPZPmtppKlqGfrwNsgUbaIV7tt4RDYAPeLEFFSyOD4NGSr6lbkfdgEB18dzxuC9JKlwgtgOBYTig== X-MS-Exchange-Transport-CrossTenantHeadersStamped: BN6PR11MB1972 X-OriginatorOrg: intel.com Subject: Re: [dpdk-dev] [PATCH v3 2/2] vhost: add thread-safe API for clearing in-flight packets in async vhost X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Hi Yuan, > -----Original Message----- > From: Wang, YuanX > Sent: Wednesday, September 22, 2021 4:56 PM > To: dev@dpdk.org > Cc: maxime.coquelin@redhat.com; Xia, Chenbo ; Hu, > Jiayu ; Ding, Xuan ; Jiang, > Cheng1 ; Ma, WenwuX ; > Yang, YvonneX ; Pai G, Sunil > > Subject: [PATCH v3 2/2] vhost: add thread-safe API for clearing in-flight > packets in async vhost >=20 > This patch adds thread safe version for > clearing in-flight packets function. Maybe the commit log can refined to be more accurate, like when will clear = in-flight packets needed? What is the difference between thread safe and unsafe version(whether use l= ock I think). >=20 > Signed-off-by: Yuan Wang > --- > doc/guides/prog_guide/vhost_lib.rst | 8 ++++- > lib/vhost/rte_vhost_async.h | 21 +++++++++++++ > lib/vhost/version.map | 1 + > lib/vhost/virtio_net.c | 49 +++++++++++++++++++++++++++++ > 4 files changed, 78 insertions(+), 1 deletion(-) Remember to add the new API explanation in 21.11 rel_note. Thanks, Xuan >=20 > diff --git a/doc/guides/prog_guide/vhost_lib.rst > b/doc/guides/prog_guide/vhost_lib.rst > index 9ed544db7a..bc21c879f3 100644 > --- a/doc/guides/prog_guide/vhost_lib.rst > +++ b/doc/guides/prog_guide/vhost_lib.rst > @@ -300,7 +300,13 @@ The following is an overview of some key Vhost API > functions: >=20 > * ``rte_vhost_clear_queue_thread_unsafe(vid, queue_id, **pkts, count)`` >=20 > - Clear inflight packets which are submitted to DMA engine in vhost asyn= c > data > + Clear in-flight packets which are submitted to async channel in vhost > + async data path without performing any locking. Completed packets are > + returned to applications through ``pkts``. > + > +* ``rte_vhost_clear_queue(vid, queue_id, **pkts, count)`` > + > + Clear in-flight packets which are submitted to async channel in vhost = async > data > path. Completed packets are returned to applications through ``pkts``. >=20 > * ``rte_vhost_async_try_dequeue_burst(vid, queue_id, mbuf_pool, pkts, > count, nr_inflight)`` > diff --git a/lib/vhost/rte_vhost_async.h b/lib/vhost/rte_vhost_async.h > index 5e2429ab70..a418e0a03d 100644 > --- a/lib/vhost/rte_vhost_async.h > +++ b/lib/vhost/rte_vhost_async.h > @@ -261,6 +261,27 @@ int rte_vhost_async_get_inflight(int vid, uint16_t > queue_id); > __rte_experimental > uint16_t rte_vhost_clear_queue_thread_unsafe(int vid, uint16_t queue_id, > struct rte_mbuf **pkts, uint16_t count); > + > +/** > + * This function checks async completion status and clear packets for > + * a specific vhost device queue. Packets which are inflight will be > + * returned in an array. > + * > + * @param vid > + * ID of vhost device to clear data > + * @param queue_id > + * Queue id to clear data > + * @param pkts > + * Blank array to get return packet pointer > + * @param count > + * Size of the packet array > + * @return > + * Number of packets returned > + */ > +__rte_experimental > +uint16_t rte_vhost_clear_queue(int vid, uint16_t queue_id, > + struct rte_mbuf **pkts, uint16_t count); > + > /** > * This function tries to receive packets from the guest with offloading > * copies to the async channel. The packets that are transfer completed > diff --git a/lib/vhost/version.map b/lib/vhost/version.map > index 8eb7e92c32..b87d5906b8 100644 > --- a/lib/vhost/version.map > +++ b/lib/vhost/version.map > @@ -88,4 +88,5 @@ EXPERIMENTAL { >=20 > # added in 21.11 > rte_vhost_async_try_dequeue_burst; > + rte_vhost_clear_queue; > }; > diff --git a/lib/vhost/virtio_net.c b/lib/vhost/virtio_net.c > index 21afcd1854..2bf8a511d5 100644 > --- a/lib/vhost/virtio_net.c > +++ b/lib/vhost/virtio_net.c > @@ -2154,6 +2154,55 @@ rte_vhost_clear_queue_thread_unsafe(int vid, > uint16_t queue_id, > return n_pkts_cpl; > } >=20 > +uint16_t > +rte_vhost_clear_queue(int vid, uint16_t queue_id, struct rte_mbuf **pkts= , > uint16_t count) > +{ > + struct virtio_net *dev =3D get_device(vid); > + struct vhost_virtqueue *vq; > + uint16_t n_pkts_cpl =3D 0; > + > + if (!dev) > + return 0; > + > + VHOST_LOG_DATA(DEBUG, "(%d) %s\n", dev->vid, __func__); > + if (unlikely(queue_id >=3D dev->nr_vring)) { > + VHOST_LOG_DATA(ERR, "(%d) %s: invalid virtqueue > idx %d.\n", > + dev->vid, __func__, queue_id); > + return 0; > + } > + > + vq =3D dev->virtqueue[queue_id]; > + > + if (unlikely(!vq->async_registered)) { > + VHOST_LOG_DATA(ERR, "(%d) %s: async not registered for > queue id %d.\n", > + dev->vid, __func__, queue_id); > + return 0; > + } > + > + if (!rte_spinlock_trylock(&vq->access_lock)) { > + VHOST_LOG_DATA(ERR, > + "(%d) %s: failed to clear async queue id %d, > virtqueue busy.\n", > + dev->vid, __func__, queue_id); > + return 0; > + } > + > + if (queue_id % 2 =3D=3D 0) > + n_pkts_cpl =3D vhost_poll_enqueue_completed(dev, > queue_id, pkts, count); > + else { > + if (unlikely(vq_is_packed(dev))) > + VHOST_LOG_DATA(ERR, > + "(%d) %s: async dequeue does not support > packed ring.\n", > + dev->vid, __func__); > + else > + n_pkts_cpl =3D > async_poll_dequeue_completed_split(dev, vq, queue_id, pkts, > + count, dev->flags & > VIRTIO_DEV_LEGACY_OL_FLAGS); > + } > + > + rte_spinlock_unlock(&vq->access_lock); > + > + return n_pkts_cpl; > +} > + > static __rte_always_inline uint32_t > virtio_dev_rx_async_submit(struct virtio_net *dev, uint16_t queue_id, > struct rte_mbuf **pkts, uint32_t count) > -- > 2.25.1