From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 9827DA0556; Mon, 17 Oct 2022 09:51:04 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 88F01427F1; Mon, 17 Oct 2022 09:51:04 +0200 (CEST) Received: from mga07.intel.com (mga07.intel.com [134.134.136.100]) by mails.dpdk.org (Postfix) with ESMTP id 9A3F041614 for ; Mon, 17 Oct 2022 09:51:02 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1665993062; x=1697529062; h=from:to:cc:subject:date:message-id:references: in-reply-to:content-transfer-encoding:mime-version; bh=8hJpBFK5/VlIhWW3XiZ98omDxq5se3SjQUqtUwm9Sa4=; b=ZMNS2Wz6jJB2rHw6VlxqHJNaxnxsdoPVV+fWE+LobrBeeuskIuIRFPxc gmcdE0FzKLZ54jvRGCgM20gTkFIoTqWpSmP4JPN63oDg95uukoCgyQ7TF H3fGGvh46E41l9j7Bn7peiPqecZBIIX5AM6RruAusSFcClCl+6Ri6ggyu NdguyA5loN9HQ6TaMZ3lBy79rGs1zqdeqNGXRIAR5f4V3BMXJ09vcS0KQ 0foOJmmqwezp9kU3egveF0SjfOzWTs8lncFBcN/201q7CXcTYSZtulHSd LvNtuR1BPwmiiFKSSgcqinC1smKXqoeScFjeJ0fWviOKCD68SJWv3OpkB A==; X-IronPort-AV: E=McAfee;i="6500,9779,10502"; a="369924326" X-IronPort-AV: E=Sophos;i="5.95,190,1661842800"; d="scan'208";a="369924326" Received: from fmsmga007.fm.intel.com ([10.253.24.52]) by orsmga105.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 17 Oct 2022 00:51:01 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6500,9779,10502"; a="630592368" X-IronPort-AV: E=Sophos;i="5.95,190,1661842800"; d="scan'208";a="630592368" Received: from fmsmsx603.amr.corp.intel.com ([10.18.126.83]) by fmsmga007.fm.intel.com with ESMTP; 17 Oct 2022 00:51:01 -0700 Received: from fmsmsx611.amr.corp.intel.com (10.18.126.91) by fmsmsx603.amr.corp.intel.com (10.18.126.83) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.31; Mon, 17 Oct 2022 00:51:01 -0700 Received: from fmsedg602.ED.cps.intel.com (10.1.192.136) by fmsmsx611.amr.corp.intel.com (10.18.126.91) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.31 via Frontend Transport; Mon, 17 Oct 2022 00:51:01 -0700 Received: from NAM10-DM6-obe.outbound.protection.outlook.com (104.47.58.102) by edgegateway.intel.com (192.55.55.71) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.1.2375.31; Mon, 17 Oct 2022 00:51:00 -0700 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=TX7UCEfvashjgZuoWAqPj5U1bmoV88zriUjnGVd1FuLwYm667UJHXxEO18Mi16fIdbjDBcKNshb7pXLdZxUlL8F229XJGt6Wl+AWFcSElSul3aZ5pD8i0j69Y5Bte9My7ZsET1ISmAzlYQydVANbCAwDABqJ8vgJh7gaQlHSpVOiTdOHdjg6vtFRc0o3tLmc3CbMboyp2/Ph9ek2oYzsV+XVn0HfiB4EfrRcurjBMeKgzsheCt+yOrB1D6dEhfB+0TVXHySvQObK/RRPtJWYBPEuMKuNLrHMecbZUC8ggXG9VdKR2rIi/gQUUg4tve5daDHUIEuFcqLyt6xR6sYWyg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=JFXj9Xh0zPyX3365a//s5lX0VGcpYkhBs78UoaUfhIw=; b=M6MwucjzZOieVBq0aoqMwDg3c4+h7ORekQnWi8R1HAz9Dsh43RRNhIZ/0HbMLqvuZCs7wxTuBbXt75iCiQnieDceEZSQ10fH8K6Y+gB/rghB26Y1e9bmAoQaswPA4BSGCpkzwXGKR0W+boBeBDC20NT+mBxXon5L9mKXlAFOClodlHWlcbGbktK2Z5tNPD3dm3V6ntkMEa5daroGDrKPHHwZhSOau7sHiH6ahqipkycw1abF/vFpldabJj7VgfbHUkQpSb1q97C3j7cMZSBBLRy49qGXzmPGDoUd/jfSmE4k8wMOj2WpZrY1RfcftYu9E+8RNTIcL3pgTInUIsogJg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=intel.com; dmarc=pass action=none header.from=intel.com; dkim=pass header.d=intel.com; arc=none Received: from PH0PR11MB5093.namprd11.prod.outlook.com (2603:10b6:510:3e::23) by DS7PR11MB5965.namprd11.prod.outlook.com (2603:10b6:8:70::16) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5723.33; Mon, 17 Oct 2022 07:50:59 +0000 Received: from PH0PR11MB5093.namprd11.prod.outlook.com ([fe80::8d5d:46e9:c3d:7723]) by PH0PR11MB5093.namprd11.prod.outlook.com ([fe80::8d5d:46e9:c3d:7723%3]) with mapi id 15.20.5723.032; Mon, 17 Oct 2022 07:50:59 +0000 From: "Liu, Changpeng" To: "Xia, Chenbo" , "dev@dpdk.org" CC: Maxime Coquelin , David Marchand Subject: RE: [PATCH v3] vhost: add new `rte_vhost_vring_call_nonblock` API Thread-Topic: [PATCH v3] vhost: add new `rte_vhost_vring_call_nonblock` API Thread-Index: AQHY4fg/KiMNKWN0l0SF2QuFbE69na4SM16AgAAC6OA= Date: Mon, 17 Oct 2022 07:50:59 +0000 Message-ID: References: <20221012064007.56040-1-changpeng.liu@intel.com> <20221017071457.28589-1-changpeng.liu@intel.com> In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: dlp-product: dlpe-windows dlp-version: 11.6.500.17 dlp-reaction: no-action authentication-results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=intel.com; x-ms-publictraffictype: Email x-ms-traffictypediagnostic: PH0PR11MB5093:EE_|DS7PR11MB5965:EE_ x-ms-office365-filtering-correlation-id: d6c1fd95-7f06-4fda-3007-08dab014549d x-ms-exchange-senderadcheck: 1 x-ms-exchange-antispam-relay: 0 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: CWsMVUHHczy18jGZ8435IXQHrAlnRIyCZmoEFtYM2toMPaNzxIwfFtqwaV/zi6qs1G7KhihwXvhrRkQVKhliyWYV5MkLBIens5Y1yfruvQYDuS+HzoCZa9DuchRd71Wtr/H2dotk6FCGYKSwvlOm3005oZ55vrmgOtBnRO8otDkrHPhspBbteIODv8VXbtvFyR80i+h8Qvd1jEfn5v1Gs1ls7i7KYiEn0j9Hxa2icBgN9hgrTMQEpQlKKGKnDIcjAGcE6PmiJR96n0dpHc3QVSpuyvxAXNeMrZp61OTkO+rpZBUJNxzKORcs30QN1uJUMNlE5aK0f3lOuz0go2THA3Lq5eV+Of5WTUG2vwQkdP5XqvMmhKA29o4LiXGq191LXS13eIPVXP8nciLgc+nRd41dU4y+NO8Mbx7UUeyyVjDXJAYwA2n8Uwslh/u92kqvp6fIkV6j0DITUqmb7BMt71ECyldruTiXYP4d71jBZpSjtpZzbhhLRMvgJbG4VgxNizlWQByiCHHf77s/eez8C2lDItMAzqJc/ysCkwjiGIWt8yhdkmpSe9OQfHMnADRERT67+UPyGnr8pPYAT04pdVnmNMEebGnITsBFT7yL8cdBnf3YGa2UgsgLrSLvGXrYcLG6lk0XQ1d4GI2XuWmo8B3wLE9i+Xt+NGW0t3IlR/NPekzNnXkosojIPqghg4BeMOIoQag4vf/9tMuSdJkKnLuEUZALq5dTjVCj4L/LHdDtHcNDXY06RoCi6uGrbPYy2Jp7sjNCOeBrP0SUjvqsn8iPvwT7cBRbtSppkPWQkRU= x-forefront-antispam-report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:PH0PR11MB5093.namprd11.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230022)(39860400002)(136003)(376002)(346002)(396003)(366004)(451199015)(26005)(53546011)(9686003)(6506007)(33656002)(478600001)(7696005)(76116006)(66946007)(8676002)(4326008)(71200400001)(110136005)(316002)(54906003)(82960400001)(86362001)(38070700005)(55016003)(38100700002)(122000001)(83380400001)(186003)(41300700001)(8936002)(5660300002)(66476007)(66556008)(66446008)(64756008)(52536014)(2906002)(81973001); DIR:OUT; SFP:1102; x-ms-exchange-antispam-messagedata-chunkcount: 1 x-ms-exchange-antispam-messagedata-0: =?Windows-1252?Q?CnVs0rgiCxY6QdpS3lZsfbO9TTDpqJ0qFb2jG4gY27uJlixM3kHCZ4Uf?= =?Windows-1252?Q?kZP7sk9uZLLWn5cN/R8jrqdfUTdCkbT3DICOdL0/iErd/VZhf6e0pj8l?= =?Windows-1252?Q?RYFdFlQrMWbZAkqXPleosLM4wlXTTcg01ShY1xSyt94lm/XHo+/0/qCZ?= =?Windows-1252?Q?em9NDtNc3w5eith2fzisahYR27QiUiZ+xhearP1sEs48ip8FJkjuSVI6?= =?Windows-1252?Q?oQV3CMayGPSpez06Tu8MRmsb0+pTQlgYqpv4Sz4v+q0URZAChwS/DHGI?= =?Windows-1252?Q?ei/5Tx1Nki3T5qlqvqtvK65/m9mmla3ELixt9kskn6TNB1OeDtF/XBcN?= =?Windows-1252?Q?e8/fkkXV2xdLkBrRLQ8xSSGS4KUHo+ql8nVR9A11AqoaNO9GPldKMq4L?= =?Windows-1252?Q?yRjJh3J2BZmWpneGvzP8+ZnC5dhVLXinnH5T3f+DmPtLZkmAWY71DODs?= =?Windows-1252?Q?eGOz7F5WY7tp1FZGXx9Tda5iEtbVOmsfDnDAEQoQNjqqvE+DFNwFyvtP?= =?Windows-1252?Q?fGRC1PXYj9sbBaj7nH/xmAXuiNsKybv9NuaT6QS8En3xcfbwoGKYAQjg?= =?Windows-1252?Q?j09g02KGDC4CJ9Lf/Cnqnm/m+oJcxHyoZtMT3O2lWrZ/5dRZONr9W9a5?= =?Windows-1252?Q?RstnejdgGyrmFGVIBxE2Wr3BI6u9lEF8kBR1dU1KO0ALiJaK7aEe6pWX?= =?Windows-1252?Q?OIvjBV+MTjBbFebJzP1gYSLyNI75Y19qwB7aG1S9bSkaYDTve4X8XEEl?= =?Windows-1252?Q?Ym8vOTxtKncsQx17hZYe4bW2Dp9ODdZ9ge8Rjo35mAfbTv5V+EjtzwbG?= =?Windows-1252?Q?aktQdnxBaPqdPjv1UYqvtG4Wj3OuSiaUxGAIgTzxehAdi4AUEu6QNvX0?= =?Windows-1252?Q?V2CaSpxkCd/wq6sF+8TfeUIRxol095JyY5+JbnQBMhYpUwjHHgbSrx6w?= =?Windows-1252?Q?lJZp+B80f5X9jnFcJ7uaG3aghtNcVbiwcEfoBtQpShjmKMB0sj8lk9st?= =?Windows-1252?Q?FC52FUaYWVzrKjvDm4ke1O7WGhc3ie6F9VuPg8FPbn5kt2P4MXlghJfV?= =?Windows-1252?Q?fozWlnlC/cqWEO9JRwR/krn3mKxtH7zFbnKIN8u8/jLYwugvxlnsF+3m?= =?Windows-1252?Q?QaMAPj0cDLkzVEETZ7iK345yylpL3hhjrSapwoQkQPagsyJVDeIraUyr?= =?Windows-1252?Q?vSrz5YEv3wsN1yOpBBe8oDtrjkyvWLwqPqFybQayZ/jkWTag32RQAeyx?= =?Windows-1252?Q?NSfq995EZQDAVlJVSIPPvTf+Nfvbs4IMwijgTr+Yx99wHUGQDE/ZdbvT?= =?Windows-1252?Q?0mkTeHF8hDQTtf7U6f/2GkNY0edlmexonm3qFWymfr6/ioA7vyLya1o0?= =?Windows-1252?Q?lI6Y/qmBpnZfNs6AWCa07KE+Rn3g9uwkIW2O9bdR1bC+rwgbqNAV/L0e?= =?Windows-1252?Q?wGP+r11Nu6XmVPtAYvYqziv8cwDC2lIMt7lcou8j0qFv+K0+sEzVGKOu?= =?Windows-1252?Q?p3sRXst7Q90maiBYxo6kf/tf6t/pHwtYUUgXmIpSnhue6w8XAeAevT6N?= =?Windows-1252?Q?ZVDd703WpG7U7/5u8QbWxTJPRGh1PCzTCco9CaB2qDmcKmxXxHFOGLnU?= =?Windows-1252?Q?U42bhrw4edXCARYYJpxV7VaYwcsKRy/9fHsNDFYtQbBmnpO9f43vuQ63?= =?Windows-1252?Q?sdF1sYFvzUT6UA4vb4pLyv7Lb5OPnHi2?= Content-Type: text/plain; charset="Windows-1252" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: PH0PR11MB5093.namprd11.prod.outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: d6c1fd95-7f06-4fda-3007-08dab014549d X-MS-Exchange-CrossTenant-originalarrivaltime: 17 Oct 2022 07:50:59.1299 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 46c98d88-e344-4ed4-8496-4ed7712e255d X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: rUI0Bv/n2HYgKCrEMcPc2/DHzQdR598soOHYeDkAr2Rdi3ypy2wDiNwWSba/tdDo9aZaUaYEHd1klllpxqnX+g== X-MS-Exchange-Transport-CrossTenantHeadersStamped: DS7PR11MB5965 X-OriginatorOrg: intel.com X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org > -----Original Message----- > From: Xia, Chenbo > Sent: Monday, October 17, 2022 3:40 PM > To: Liu, Changpeng ; dev@dpdk.org > Cc: Maxime Coquelin ; David Marchand > > Subject: RE: [PATCH v3] vhost: add new `rte_vhost_vring_call_nonblock` AP= I >=20 > Hi Changpeng, >=20 > > -----Original Message----- > > From: Liu, Changpeng > > Sent: Monday, October 17, 2022 3:15 PM > > To: dev@dpdk.org > > Cc: Liu, Changpeng ; Maxime Coquelin > > ; Xia, Chenbo ; David > > Marchand > > Subject: [PATCH v3] vhost: add new `rte_vhost_vring_call_nonblock` API > > > > Vhost-user library locks all VQ's access lock when processing > > vring based messages, such as SET_VRING_KICK and SET_VRING_CALL, > > and the data processing thread may already be started, e.g: SPDK > > vhost-blk and vhost-scsi will start the data processing thread > > when one vring is ready, then deadlock may happen when SPDK is > > posting interrupts to VM. Here, we add a new API which allows > > caller to try again later for this case. > > > > Bugzilla ID: 1015 > > Fixes: c5736998305d ("vhost: fix missing virtqueue lock protection") > > > > Signed-off-by: Changpeng Liu > > --- > > doc/guides/prog_guide/vhost_lib.rst | 6 ++++++ > > doc/guides/rel_notes/release_22_11.rst | 6 ++++++ > > lib/vhost/rte_vhost.h | 15 +++++++++++++ > > lib/vhost/version.map | 3 +++ > > lib/vhost/vhost.c | 30 ++++++++++++++++++++++++++ > > 5 files changed, 60 insertions(+) > > > > diff --git a/doc/guides/prog_guide/vhost_lib.rst > > b/doc/guides/prog_guide/vhost_lib.rst > > index bad4d819e1..52f0589f51 100644 > > --- a/doc/guides/prog_guide/vhost_lib.rst > > +++ b/doc/guides/prog_guide/vhost_lib.rst > > @@ -297,6 +297,12 @@ The following is an overview of some key Vhost API > > functions: > > Clear in-flight packets which are submitted to async channel in vhos= t > > async data > > path. Completed packets are returned to applications through ``pkts`= `. > > > > +* ``rte_vhost_vring_call_nonblock(int vid, uint16_t vring_idx)`` > > + > > + Notify the guest that used descriptors have been added to the vring. > > This function > > + will return -EAGAIN when vq's access lock is held by other thread, u= ser > > should try > > + again later. > > + > > * ``rte_vhost_vring_stats_get_names(int vid, uint16_t queue_id, struct > > rte_vhost_stat_name *names, unsigned int size)`` > > > > This function returns the names of the queue statistics. It requires > > diff --git a/doc/guides/rel_notes/release_22_11.rst > > b/doc/guides/rel_notes/release_22_11.rst > > index 2da8bc9661..9b3783ffd8 100644 > > --- a/doc/guides/rel_notes/release_22_11.rst > > +++ b/doc/guides/rel_notes/release_22_11.rst > > @@ -236,6 +236,12 @@ New Features > > > > strings $dpdk_binary_or_driver | sed -n 's/^PMD_INFO_STRING=3D //= p' > > > > +* **Added non-block notify API to the vhost library.** > > + > > + Added API to notify the guest that used descriptors have been added > > + to the vring in non-block way, user should check the return value of > > + this API. > > + >=20 > This may be better: >=20 > * **Added non-blocking notify API to the vhost library.** >=20 > Added ``rte_vhost_vring_call_nonblock`` API to notify the guest that us= ed > descriptors have been added to the vring in non-blocking way. User shou= ld > check the return value of this API and try again if needed. >=20 > And for new features, library should come before scripts, so you need to = move > this before ' Rewritten pmdinfo script. ' item. Thanks Chenbo, applied them with v4. >=20 > Thanks, > Chenbo >=20 > > > > Removed Items > > ------------- > > diff --git a/lib/vhost/rte_vhost.h b/lib/vhost/rte_vhost.h > > index bb7d86a432..d22b25cd4e 100644 > > --- a/lib/vhost/rte_vhost.h > > +++ b/lib/vhost/rte_vhost.h > > @@ -909,6 +909,21 @@ rte_vhost_clr_inflight_desc_packed(int vid, uint16= _t > > vring_idx, > > */ > > int rte_vhost_vring_call(int vid, uint16_t vring_idx); > > > > +/** > > + * Notify the guest that used descriptors have been added to the vring= . > > This > > + * function acts as a memory barrier. This function will return -EAGA= IN > > when > > + * vq's access lock is held by other thread, user should try again lat= er. > > + * > > + * @param vid > > + * vhost device ID > > + * @param vring_idx > > + * vring index > > + * @return > > + * 0 on success, -1 on failure, -EAGAIN for another retry > > + */ > > +__rte_experimental > > +int rte_vhost_vring_call_nonblock(int vid, uint16_t vring_idx); > > + > > /** > > * Get vhost RX queue avail count. > > * > > diff --git a/lib/vhost/version.map b/lib/vhost/version.map > > index 7a00b65740..c8c44b8326 100644 > > --- a/lib/vhost/version.map > > +++ b/lib/vhost/version.map > > @@ -94,6 +94,9 @@ EXPERIMENTAL { > > rte_vhost_async_try_dequeue_burst; > > rte_vhost_driver_get_vdpa_dev_type; > > rte_vhost_clear_queue; > > + > > + # added in 22.11 > > + rte_vhost_vring_call_nonblock; > > }; > > > > INTERNAL { > > diff --git a/lib/vhost/vhost.c b/lib/vhost/vhost.c > > index 8740aa2788..ed6efb003f 100644 > > --- a/lib/vhost/vhost.c > > +++ b/lib/vhost/vhost.c > > @@ -1317,6 +1317,36 @@ rte_vhost_vring_call(int vid, uint16_t vring_idx= ) > > return 0; > > } > > > > +int > > +rte_vhost_vring_call_nonblock(int vid, uint16_t vring_idx) > > +{ > > + struct virtio_net *dev; > > + struct vhost_virtqueue *vq; > > + > > + dev =3D get_device(vid); > > + if (!dev) > > + return -1; > > + > > + if (vring_idx >=3D VHOST_MAX_VRING) > > + return -1; > > + > > + vq =3D dev->virtqueue[vring_idx]; > > + if (!vq) > > + return -1; > > + > > + if (!rte_spinlock_trylock(&vq->access_lock)) > > + return -EAGAIN; > > + > > + if (vq_is_packed(dev)) > > + vhost_vring_call_packed(dev, vq); > > + else > > + vhost_vring_call_split(dev, vq); > > + > > + rte_spinlock_unlock(&vq->access_lock); > > + > > + return 0; > > +} > > + > > uint16_t > > rte_vhost_avail_entries(int vid, uint16_t queue_id) > > { > > -- > > 2.21.3