From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 77F8FA0556; Mon, 17 Oct 2022 09:17:29 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 5EC8C40143; Mon, 17 Oct 2022 09:17:29 +0200 (CEST) Received: from mga18.intel.com (mga18.intel.com [134.134.136.126]) by mails.dpdk.org (Postfix) with ESMTP id CD394400D7 for ; Mon, 17 Oct 2022 09:17:27 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1665991048; x=1697527048; h=from:to:cc:subject:date:message-id:references: in-reply-to:content-transfer-encoding:mime-version; bh=54pAcWI+AbF9WKGN4kEWiUmBoD33+qsPlfwWTuTIAs4=; b=QW8R8h4lzm1MYXeXL6YeW4p8jc9x51SXvFekjQE3Gju5M/v2D0f5W2Q0 uM1Cw7DMLajYaCZl1pRwtiChmy7lVCE66v+6R8Z/dXQgdk49miB1/z5NG k97LUoRhY5sPb+fLkUbVcdK/gMLsRSljnkKAIYF1Spt3VcWHTLnnTMSK3 rhg05qXY/4hPO30geFThXMf5Vw3wTlLNCnusuPqTQUG2mp6Rsw+LoK9ZJ bPQ9u5HDUF7O2z8Uw7jJVQEpxTGla25g5MApY73buxDZ83Zm1liAn4xhP 2Vb7iAGiFY830nlQ/9xqrdW4YNVWMZNEwTZByW8TyV/qcXsEY1J27clZB A==; X-IronPort-AV: E=McAfee;i="6500,9779,10502"; a="289016807" X-IronPort-AV: E=Sophos;i="5.95,190,1661842800"; d="scan'208";a="289016807" Received: from fmsmga003.fm.intel.com ([10.253.24.29]) by orsmga106.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 17 Oct 2022 00:17:26 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6500,9779,10502"; a="717395635" X-IronPort-AV: E=Sophos;i="5.95,190,1661842800"; d="scan'208";a="717395635" Received: from orsmsx602.amr.corp.intel.com ([10.22.229.15]) by FMSMGA003.fm.intel.com with ESMTP; 17 Oct 2022 00:17:26 -0700 Received: from orsmsx612.amr.corp.intel.com (10.22.229.25) by ORSMSX602.amr.corp.intel.com (10.22.229.15) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.31; Mon, 17 Oct 2022 00:17:25 -0700 Received: from orsedg603.ED.cps.intel.com (10.7.248.4) by orsmsx612.amr.corp.intel.com (10.22.229.25) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.31 via Frontend Transport; Mon, 17 Oct 2022 00:17:25 -0700 Received: from NAM10-MW2-obe.outbound.protection.outlook.com (104.47.55.107) by edgegateway.intel.com (134.134.137.100) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.1.2375.31; Mon, 17 Oct 2022 00:17:25 -0700 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=BHBVRwLJBN3hmaUuUG2jpGYVfJoUy9WHv0VXsJ4P0lp+A+kHbY1I0csWrKQrhWIBQUHp/aSIDsim6lXGdagqU/oXBodYO7LsKeWGNTV0YzO/2YB0wVH/ojB5+7htfjU8V1NKxqaX2hA+G37QH/stOFsuvflNhnlBuiMl8n2Mu71+MeaRH1+4zIxD8FP88MDy5jLwqrMAGUBD4qMNTX2Fw9rcUCPslmm4dBO9S0bxl2hg5rD6b1D/+NF2ARh//VgFPb/ybVJEzqgAEP+FqFn5k3TF8MQ1U1MB66mhZdOqofWTlC5pFb3aT6drtp8twBDvq6g96FVqcWy+lJKHk18qMA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=R7MmYs3FWPFFD6WuiQ3TihgXPTXadznEsp4suUFRlRM=; b=DAoRquDIpIH9e+VntumF6Cf5Mzf/JinPZpodC8Pspz152C1TkZO5xXSlV4WBv3BPUQxcX7djbnZIch6IBrtJ5Ng/LDNWur48FJJyaO/5q/yD2Qsq6XEWv++53LQ1plFaDI2IRB7n+S0dy57nugzY+7geSp0l5TZOCKn3udK4eiVVhiNpv5hjvoTYglvtAziIs+VbnXHuCAp6p18b/riuoTpYLXY8ZdWTlb1QuX8KjXIqtCCrbSWJZifdzzAv14DzJBLCMnynkZ093mfmqNM4H1kupYCBEZJbUQYCeazTI2yjAw3Jh8w+eQA/Tbqc66vWt3UAAJdl2cqmo2SJXIUiVg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=intel.com; dmarc=pass action=none header.from=intel.com; dkim=pass header.d=intel.com; arc=none Received: from PH0PR11MB5093.namprd11.prod.outlook.com (2603:10b6:510:3e::23) by MN0PR11MB5963.namprd11.prod.outlook.com (2603:10b6:208:372::10) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5676.36; Mon, 17 Oct 2022 07:17:23 +0000 Received: from PH0PR11MB5093.namprd11.prod.outlook.com ([fe80::8d5d:46e9:c3d:7723]) by PH0PR11MB5093.namprd11.prod.outlook.com ([fe80::8d5d:46e9:c3d:7723%3]) with mapi id 15.20.5723.032; Mon, 17 Oct 2022 07:17:23 +0000 From: "Liu, Changpeng" To: "Xia, Chenbo" , "dev@dpdk.org" CC: Maxime Coquelin , David Marchand Subject: RE: [PATCH v2] vhost: add new `rte_vhost_vring_call_nonblock` API Thread-Topic: [PATCH v2] vhost: add new `rte_vhost_vring_call_nonblock` API Thread-Index: AQHY3gWKI5fsu4mx0Uqo8VIKEvYhAq4SLGGAgAAIa8A= Date: Mon, 17 Oct 2022 07:17:23 +0000 Message-ID: References: <20220906022225.17215-1-changpeng.liu@intel.com> <20221012064007.56040-1-changpeng.liu@intel.com> In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: dlp-product: dlpe-windows dlp-version: 11.6.500.17 dlp-reaction: no-action authentication-results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=intel.com; x-ms-publictraffictype: Email x-ms-traffictypediagnostic: PH0PR11MB5093:EE_|MN0PR11MB5963:EE_ x-ms-office365-filtering-correlation-id: 0217e074-1cf3-430f-1ec1-08dab00fa34d x-ms-exchange-senderadcheck: 1 x-ms-exchange-antispam-relay: 0 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: kkAazpa2smBYwwlxSHwNRWOplilBCuBx8xjtcBGd6A54YuUE/u60rtt3oh823iEPWBmco6KlkAmpoKFklH/jA9AesnRQ9AHzITEFCve5GtLDA9hUfCS4m+GmxHQ2Qo6CiewTo7GtDewSxpexRhX8nS52UoOg8OVPAo5VLNx4MWJyp+idCXLzlLM/CjJjO7iVdkBTYmoOqBQdnFcqCr/qSvWkuuwPlt8hyGkRjedQ8eqXpwKkisXK2RZLAoAiDd6IieszeFwnVHUwVBeXUpL10lGykLFeGGz3E6Uw6tWoKv8n2BjqBENiZ1GJMI9p/aDlElUPBqzfi6TV8NzCMS2RmMKP1lajxjThRdEVrheogp+qufpbuz3EZqGJvlplBiag+yj21clnxfktxXyjEXNqODqLVBP95BaI3gGVj5Ag1Yn61fv0VpoEaUvhtakwALI8d8xAAf89CfR1GKeJvbC3LdgZPqSer+PSgy8dCJTzTOQvJb7y/cjyXt+Bd3P8BJiaWohlhiqHLYeosYraYhyadfnqr3kfPxeCSrGBUb9g31+UVJfkTCPlUemUDTPgTZNlcy2j6/LQITSiNZFl5nVZaP5S9scimEdgA5AQWKqVeqxPuqKhBJHljXy3vWoITJ+4OHJ0CpamLe7j0466VZDdCaPdihVNPvd4fUMfPK/0pJtV3DPNIQZp4REW9Xv80tEcfx3zjSan/UiLM4bGC2wFqYiJSBWkizQnLb4teFZZ9P2z0Zj0GMiZNMxNxCB5U7bP9zP2SaPnd4zlRSLw+bA+j9nIKybuJ0Xrs/Ebj/OEOYR9A9ftEjth6r2mGexa23qMqA1Z2Y45+lrgkUBKLOqoUA== x-forefront-antispam-report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:PH0PR11MB5093.namprd11.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230022)(136003)(376002)(346002)(396003)(366004)(39860400002)(451199015)(110136005)(54906003)(64756008)(76116006)(66946007)(8676002)(66476007)(53546011)(66446008)(66556008)(6506007)(7696005)(41300700001)(4326008)(5660300002)(55016003)(316002)(26005)(9686003)(86362001)(52536014)(8936002)(83380400001)(82960400001)(2906002)(38100700002)(186003)(33656002)(71200400001)(478600001)(966005)(38070700005)(122000001)(81973001); DIR:OUT; SFP:1102; x-ms-exchange-antispam-messagedata-chunkcount: 1 x-ms-exchange-antispam-messagedata-0: =?Windows-1252?Q?tZZMUTeFy3xCfvXZ32B4r4vIuOeOjv99ZjmWFl7Bjw+UdOQ7hLAavOjz?= =?Windows-1252?Q?RW575GefNp5mfdDoKbrAzaVpGVMq8Q7ltiafD4Qacvxz59wQxCX+HmhZ?= =?Windows-1252?Q?31dCny0XJ1CDc2ms24hb+/F4RpjCtTGX3DhM6aM6eW+hYrpifDZEjpD2?= =?Windows-1252?Q?+CGsl8XpKwk1D7JXMYDeWSH4huuE9p0fTfYzHpwsxc9us+ua2mYCODlQ?= =?Windows-1252?Q?NR3iAlzmTqIN6gbs01J1Biaa3ANoDxMukVIBOGIto/kR947jPCilQWKu?= =?Windows-1252?Q?SLiCtQmf6y3j7XWvw38bZr2wVGRuazI6iWpDwjQrK8BkL9yG8CLDDlZb?= =?Windows-1252?Q?en+tk3jZCKeRviQatBIfR9sx2CJVvM4gfc0lTf2WvQ0gju1u4zlloveu?= =?Windows-1252?Q?Jy/Jmo5cChWObMUJLWa4dle1cNA/EIxxtYQvbPxKlExXEf/ux5UBKiNa?= =?Windows-1252?Q?9jwmwQqeJ3DzwdccyJhBxpEbIe4dUMTD84zGcHu/wHQMXI27+bh7h5l2?= =?Windows-1252?Q?il9QjCMv4b9/KbLRb6+PNjPD+AMAKHzuGo51ipCkPGJuHlsbKWbFJyMP?= =?Windows-1252?Q?TvZ1UFjQi5aKEq2jR3mPuwmE2Q+BOcNxZ2TGsVSq0igTSukYGege3dv5?= =?Windows-1252?Q?mnO3vSjFo8Eeuu/Yf7G9hIRxw3mBxTCpKkGmtdoy9uxuOu1gLFElHxbn?= =?Windows-1252?Q?DUKk7whfDmOvQUuEmQUhb+u0a2PxWN77sGC3iIvZBbnEGdzXs6k41AvS?= =?Windows-1252?Q?IcSXgZKO3A3TKk44mrq2zsjpw+uR+G/Tx4KP7nm0evZgoZBKQMV7hL/W?= =?Windows-1252?Q?4ud9YNNRZGeu+0SIZxGO1B+sOJnGkXNnYx9yg2fa1P/5wep6SbM8XQ9a?= =?Windows-1252?Q?b4r5k8oBkFVdb/M5Tv4ydbuwr4hPdeuTWEBbpeYnJXuaz9Fuk8xvLoj+?= =?Windows-1252?Q?qETnciV2qw4Qb1tNNN7mmVkh4BIc+IW8qVWgdXntYLt6+BSBg/sUcr5U?= =?Windows-1252?Q?k4DAIuiLsTelRpFH5lrGEFASfa1ienHfcEGKsEOvt/fXO52SiuxOMehB?= =?Windows-1252?Q?h1+3QX7BqkMWH/5PDzfFYbAhU1e3tXnwJbQx/5ks6eiQTYF6sVFM6kVb?= =?Windows-1252?Q?hXcanISIbZ8orYMeDbylkUIZby9QEoMhRryelVYUoQeuq3UqGh8/5M0D?= =?Windows-1252?Q?9t6OrO1I4A/nHp32b/GYPHnSyPVdNNyRl2rsfo3JWe9ard+MucKxtjKD?= =?Windows-1252?Q?W813AZWccMDP+hHWifSDzoHCnaHKLn3T6HHUVThYA2QG8DBlFNRwNvEq?= =?Windows-1252?Q?grQa8KqbFka/xKO7us1ger+RMZ36crT0gomDtfxoAXGLEid2tF8AAF0r?= =?Windows-1252?Q?Jzh4Zebqwp6NFWASj0nIf8lqR0bEdGvOqP86/ZjPQpXJ4y8WnJPNNnZk?= =?Windows-1252?Q?xKS/ctqJmrRDPJsvGkGZnXUhftqg5WIjlctLt/5EFrdM/yYuPHxeU3Uc?= =?Windows-1252?Q?/QVYjgiMkosYMtQBUwpnZTqTmKsujYj2DhVngDPrcVJNDI8YIohAIyF7?= =?Windows-1252?Q?SLPnrDBTtM8EVbWX1401k0aLm1X/mRveWWH090u8yzKNJNy/B43eLdrY?= =?Windows-1252?Q?PEO/tvnaV6ZEdauRZ4xp1rdQPFw+m8e0pik8C9Rap0C0bTh2X64k3Pn2?= =?Windows-1252?Q?0sQ9HzvjF0s/sOhQsAMvTexrPrl0+dcm?= Content-Type: text/plain; charset="Windows-1252" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: PH0PR11MB5093.namprd11.prod.outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: 0217e074-1cf3-430f-1ec1-08dab00fa34d X-MS-Exchange-CrossTenant-originalarrivaltime: 17 Oct 2022 07:17:23.6928 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 46c98d88-e344-4ed4-8496-4ed7712e255d X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: IUyA5iY1vnM9XKFJu57Usjw136jCgG5AFOuz9Sz6Lb5Cmo13ntlnGlOtg7MVDUj+a4t00w3dv+BFRpuaZqItfQ== X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN0PR11MB5963 X-OriginatorOrg: intel.com X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org > -----Original Message----- > From: Xia, Chenbo > Sent: Monday, October 17, 2022 2:47 PM > To: Liu, Changpeng ; dev@dpdk.org > Cc: Maxime Coquelin ; David Marchand > > Subject: RE: [PATCH v2] vhost: add new `rte_vhost_vring_call_nonblock` AP= I >=20 > Hi Changpeng, >=20 > > -----Original Message----- > > From: Liu, Changpeng > > Sent: Wednesday, October 12, 2022 2:40 PM > > To: dev@dpdk.org > > Cc: Liu, Changpeng ; Maxime Coquelin > > ; Xia, Chenbo ; David > > Marchand > > Subject: [PATCH v2] vhost: add new `rte_vhost_vring_call_nonblock` API > > > > Vhost-user library locks all VQ's access lock when processing > > vring based messages, such as SET_VRING_KICK and SET_VRING_CALL, > > and the data processing thread may already be started, e.g: SPDK > > vhost-blk and vhost-scsi will start the data processing thread > > when one vring is ready, then deadlock may happen when SPDK is > > posting interrupts to VM. Here, we add a new API which allows > > caller to try again later for this case. > > > > Bugzilla ID: 1015 > > Fixes: c5736998305d ("vhost: fix missing virtqueue lock protection") > > > > Signed-off-by: Changpeng Liu > > --- > > lib/vhost/rte_vhost.h | 15 +++++++++++++++ > > lib/vhost/version.map | 3 +++ > > lib/vhost/vhost.c | 30 ++++++++++++++++++++++++++++++ > > 3 files changed, 48 insertions(+) >=20 > For new API, we need to update release_22_11.rst and vhost_lib.rst. Thanks Chenbo, a new v3 is sent for review. >=20 > You can refer to > http://patchwork.dpdk.org/project/dpdk/patch/20221013092708.4922-2- > xuan.ding@intel.com/ >=20 > Thanks, > Chenbo >=20 > > > > diff --git a/lib/vhost/rte_vhost.h b/lib/vhost/rte_vhost.h > > index bb7d86a432..d22b25cd4e 100644 > > --- a/lib/vhost/rte_vhost.h > > +++ b/lib/vhost/rte_vhost.h > > @@ -909,6 +909,21 @@ rte_vhost_clr_inflight_desc_packed(int vid, uint16= _t > > vring_idx, > > */ > > int rte_vhost_vring_call(int vid, uint16_t vring_idx); > > > > +/** > > + * Notify the guest that used descriptors have been added to the vring= . > > This > > + * function acts as a memory barrier. This function will return -EAGA= IN > > when > > + * vq's access lock is held by other thread, user should try again lat= er. > > + * > > + * @param vid > > + * vhost device ID > > + * @param vring_idx > > + * vring index > > + * @return > > + * 0 on success, -1 on failure, -EAGAIN for another retry > > + */ > > +__rte_experimental > > +int rte_vhost_vring_call_nonblock(int vid, uint16_t vring_idx); > > + > > /** > > * Get vhost RX queue avail count. > > * > > diff --git a/lib/vhost/version.map b/lib/vhost/version.map > > index 7a00b65740..c8c44b8326 100644 > > --- a/lib/vhost/version.map > > +++ b/lib/vhost/version.map > > @@ -94,6 +94,9 @@ EXPERIMENTAL { > > rte_vhost_async_try_dequeue_burst; > > rte_vhost_driver_get_vdpa_dev_type; > > rte_vhost_clear_queue; > > + > > + # added in 22.11 > > + rte_vhost_vring_call_nonblock; > > }; > > > > INTERNAL { > > diff --git a/lib/vhost/vhost.c b/lib/vhost/vhost.c > > index 8740aa2788..ed6efb003f 100644 > > --- a/lib/vhost/vhost.c > > +++ b/lib/vhost/vhost.c > > @@ -1317,6 +1317,36 @@ rte_vhost_vring_call(int vid, uint16_t vring_idx= ) > > return 0; > > } > > > > +int > > +rte_vhost_vring_call_nonblock(int vid, uint16_t vring_idx) > > +{ > > + struct virtio_net *dev; > > + struct vhost_virtqueue *vq; > > + > > + dev =3D get_device(vid); > > + if (!dev) > > + return -1; > > + > > + if (vring_idx >=3D VHOST_MAX_VRING) > > + return -1; > > + > > + vq =3D dev->virtqueue[vring_idx]; > > + if (!vq) > > + return -1; > > + > > + if (!rte_spinlock_trylock(&vq->access_lock)) > > + return -EAGAIN; > > + > > + if (vq_is_packed(dev)) > > + vhost_vring_call_packed(dev, vq); > > + else > > + vhost_vring_call_split(dev, vq); > > + > > + rte_spinlock_unlock(&vq->access_lock); > > + > > + return 0; > > +} > > + > > uint16_t > > rte_vhost_avail_entries(int vid, uint16_t queue_id) > > { > > -- > > 2.21.3