From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 14F4B42C57; Thu, 8 Jun 2023 04:11:10 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 06698427F2; Thu, 8 Jun 2023 04:11:10 +0200 (CEST) Received: from mga12.intel.com (mga12.intel.com [192.55.52.136]) by mails.dpdk.org (Postfix) with ESMTP id DFF8E40042 for ; Thu, 8 Jun 2023 04:11:07 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1686190268; x=1717726268; h=from:to:subject:date:message-id:references:in-reply-to: content-transfer-encoding:mime-version; bh=Km1nNM6pVYywNvBw+dx7kFrmm4ySpwycWuIJAwCxrJg=; b=aMIZuRoLQKNkyul6AI8+BGtKL1dntotQizgX32mNg5kazZeCgWjaHxiA M0Pplgj1/j6AEkqaOO011cJs7WIolujSv5uwUzwcjHRccvWz7DYvxja0r +zHBKVAhwyYieevWVvfp1NPUQJKLbiHeMp3NZ2dW4+ONG/zHKVVEGEv2W xLiyoRnB8Pv2yi/EIs2Dc9Qhoz3sx+zsEdSMATrZvtxmo21IeFEJrOChP Qk/dN8RxBBXjKhgf1y1bGBTHi5TyYGgIgu26qQeipThtjH2OegoQf6QZR 4pIgUaipvT1jOaDoFGL2VFD2fS1bWhwG2cDWmA+Qj6ixkGS0pBCn+V3mo w==; X-IronPort-AV: E=McAfee;i="6600,9927,10734"; a="336799280" X-IronPort-AV: E=Sophos;i="6.00,225,1681196400"; d="scan'208";a="336799280" Received: from orsmga002.jf.intel.com ([10.7.209.21]) by fmsmga106.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 07 Jun 2023 19:11:06 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10734"; a="709774511" X-IronPort-AV: E=Sophos;i="6.00,225,1681196400"; d="scan'208";a="709774511" Received: from orsmsx601.amr.corp.intel.com ([10.22.229.14]) by orsmga002.jf.intel.com with ESMTP; 07 Jun 2023 19:11:06 -0700 Received: from orsmsx612.amr.corp.intel.com (10.22.229.25) by ORSMSX601.amr.corp.intel.com (10.22.229.14) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.23; Wed, 7 Jun 2023 19:11:05 -0700 Received: from orsmsx610.amr.corp.intel.com (10.22.229.23) by ORSMSX612.amr.corp.intel.com (10.22.229.25) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.23; Wed, 7 Jun 2023 19:11:05 -0700 Received: from ORSEDG601.ED.cps.intel.com (10.7.248.6) by orsmsx610.amr.corp.intel.com (10.22.229.23) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.23 via Frontend Transport; Wed, 7 Jun 2023 19:11:05 -0700 Received: from NAM11-DM6-obe.outbound.protection.outlook.com (104.47.57.168) by edgegateway.intel.com (134.134.137.102) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.1.2507.23; Wed, 7 Jun 2023 19:11:04 -0700 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=BmjTKtE4soz2SGySbW5+wp0+UluSbfOmeaYQGQaHMd2cmzFNOzI6AiHmz1TRUocqSDytddRCddQLVK9qx4V7piQj4da+BkCXUL5PjwsqVIZL7MJbu8Z1JTpPQ8lP6zxZvb8u0hm2/0yFbwgNz30XtwgV3t2YVRgyX6+VGeEKPW2XnCYA6ZJBAt1nNuoGjZyK0IyNfe6Qsb4oc7yx2meY+JCFZQctkustfWpzoJuamFzECfr4+8dmWYRiRFft9ZeBdkKxBoRT1VX2chAmAuEUglfRVfU9ymghTLS+B6yOZxcG7dec9aP+jC6G1Zn1bCznzVFbknwgUb6tShNOjqTEHQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=m1+4LhcTA64KkgmmH+tECmXdJ/5QrxbsFinXR1ulhr0=; b=NNHY1jMR3q1QiqDj3xKqhG1VRNmPbAssYBoXtJ9uZtxErtrO//1UUlXFgcEMEXxYrsS3IAjHOclcH/SjRuWJDT/b6arRJVtKNYCHvw9C39326T7rXuiFmA7qXAujn2kMQ0OYnBo4VcZeYL7CNW4L+OPqCBnQvPR57Ql8tZ5eaL8rplqOsXb2ZmdLb5RsH0fsXrln/iS3XibZRvMpMB/nZndcANC/pL6SwDkTuhfaUyZq68mzW3haZi3d6iscKdGc2tehjYN5zDwtZU6R5yWRUCuxmc3KaPMQda10TZegFP3k+Amxi2aSoZqc8gZTMv7eFOiahjYGQ/8yICPgqxQv9Q== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=intel.com; dmarc=pass action=none header.from=intel.com; dkim=pass header.d=intel.com; arc=none Received: from SN6PR11MB3504.namprd11.prod.outlook.com (2603:10b6:805:d0::17) by CO1PR11MB4836.namprd11.prod.outlook.com (2603:10b6:303:9d::22) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6455.33; Thu, 8 Jun 2023 02:11:02 +0000 Received: from SN6PR11MB3504.namprd11.prod.outlook.com ([fe80::e508:44ff:3d34:60bc]) by SN6PR11MB3504.namprd11.prod.outlook.com ([fe80::e508:44ff:3d34:60bc%4]) with mapi id 15.20.6455.030; Thu, 8 Jun 2023 02:11:02 +0000 From: "Xia, Chenbo" To: Maxime Coquelin , "dev@dpdk.org" , "david.marchand@redhat.com" , "mkp@redhat.com" , "fbl@redhat.com" , "jasowang@redhat.com" , "Liang, Cunming" , "Xie, Yongji" , "echaudro@redhat.com" , "eperezma@redhat.com" , "amorenoz@redhat.com" , "lulu@redhat.com" Subject: RE: [PATCH v5 26/26] vhost: add VDUSE device stop Thread-Topic: [PATCH v5 26/26] vhost: add VDUSE device stop Thread-Index: AQHZmE/Ldrxa9dBxV0WC6ioLilDO0K+ALEZw Date: Thu, 8 Jun 2023 02:11:02 +0000 Message-ID: References: <20230606081852.71003-1-maxime.coquelin@redhat.com> <20230606081852.71003-27-maxime.coquelin@redhat.com> In-Reply-To: <20230606081852.71003-27-maxime.coquelin@redhat.com> Accept-Language: en-US, zh-CN Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=intel.com; x-ms-publictraffictype: Email x-ms-traffictypediagnostic: SN6PR11MB3504:EE_|CO1PR11MB4836:EE_ x-ms-office365-filtering-correlation-id: db6ac773-f4e7-4ec8-9202-08db67c59bef x-ld-processed: 46c98d88-e344-4ed4-8496-4ed7712e255d,ExtAddr x-ms-exchange-senderadcheck: 1 x-ms-exchange-antispam-relay: 0 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: 8YGbldXr3DO5GsGRC2rFBzegdWRT1utNlnCkvvSjdziVMDo9i6pzb9Bo0IByWiMWoPv2mlw1AExgc62sqm0lzYPJZtesHhEZIWE+lMfIDcq2iAlNqW6HYm3HK6xxVCVlJopjC3gO1T3OY2VuVthu1cNET00gpKdik/65N3i5rrEv3kyrQUi/G5FTe89oCyptzIDGfRvNaKtD3+/nuV2ZpCbCDE5GcqopF9WYnonCejr1PyU8CXB6sqdRzXrPYYYPeZn1GZS51xpXRsoR8P1ontYaDD8oVwxqyZGJhSW0ItDJHWpAG/JHKNCAA/BWyOOKLgc+DBolsqDl2iwjzFDCOlOkb30k52to57RrtZyXGB88EYNZoUDZkSb9whBEmXWjaQC46+JbhUoPsMJgQ7PR+OzYJLwAS2rQ0hr6fcm4iDSVGm7Ls8pBS2n2xj/0rAkr+z3qHLjx/5A5bQjfl50SZAd97S1e3JzydZ/W8E1uMR48FAWPjNM5k1YqKB9JQwnlOrNIkFS+H0zDpRE49AFxaW/RRVgo0XfOtdCmuhsWGWu8NcNMVHbarshS4ScWWBRDJHcC3Bme+cqTC5jlptHiJQ93xpfpBkFbDE1jOPEsK7m2FeRtTnCKK7uGnB7v2GHpYMSw/4ec21M/5KeNUg9dvQ== x-forefront-antispam-report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:SN6PR11MB3504.namprd11.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230028)(39860400002)(376002)(396003)(346002)(366004)(136003)(451199021)(6506007)(53546011)(9686003)(26005)(7416002)(52536014)(83380400001)(122000001)(86362001)(33656002)(186003)(82960400001)(316002)(55016003)(2906002)(5660300002)(8676002)(38070700005)(8936002)(41300700001)(38100700002)(66476007)(66556008)(76116006)(66446008)(110136005)(66946007)(921005)(64756008)(478600001)(71200400001)(7696005); DIR:OUT; SFP:1102; x-ms-exchange-antispam-messagedata-chunkcount: 1 x-ms-exchange-antispam-messagedata-0: =?us-ascii?Q?qZZDZUM8b0YJEwVJhMFKie9lO90pX8M9HEzwRFBBJasHhzVRn9GnT4YYkEPB?= =?us-ascii?Q?KffLSJVLKmmOLL3GJveo0WGMx3Y1Jn3OiHIeGj771wsZRNLqx0w4n3mw0mwR?= =?us-ascii?Q?+Mjr4nv2FCHsn/qKRWTQgRj32G3kHotGeKqnif4Dw4IHYTuhQWSQ7fKqKMKJ?= =?us-ascii?Q?PzYgZlEQFkb8fv71p+Yw+WhXs5J9w/KAZm/9zeEIm7mTv24Kpn5NmVtqbpxu?= =?us-ascii?Q?MFoS45v8TLLcTj1zzGVb2ZIKnnekUlbRaIcV0z9DTrSgw96rPwUvfVBgGszi?= =?us-ascii?Q?LYAv+JbvPlRjzlOld/IayfSBe7JQbL5ybV9v9K1mv//H7bsot6sA4MhxOV3a?= =?us-ascii?Q?iY/if6AqmTBqOJZ3gU5ePOXL+vynHn5l+g6GNWSw2bxI6lgO3IGwjHM4GUTF?= =?us-ascii?Q?CAqH19mA5CPGW/JDlzxtNM0YPAkr3b0KVfi+M343iI0tw3+88bi5gN/ZbUIW?= =?us-ascii?Q?iq+NJLX4lYYqZpZ83TdkNR2AlNIZ1yn/nhAzQbcHPTHTj/dWKEiPx88Sa8tJ?= =?us-ascii?Q?t96ijfEKMMIAggWXIDShOGRtvn0vr+v1YIG+d8P419ikALXOlAhsvrHRRb4V?= =?us-ascii?Q?ELjraBmx+myNZiKkmNgQCmb9bvbCcMcj5pzyY9s5i7cd3xUEfSLdGliaQU0C?= =?us-ascii?Q?gW2++OSUvLZAfSnT7ojTq5QfADjp5JS0wJGXALqacAtFr0O+H1BItlS/bau0?= =?us-ascii?Q?fdEIyPl9oR3y8+z6Ej6/Kti8DlbpoCNc/xRvCfa1l23pcHmvVY5JuPnqWyFs?= =?us-ascii?Q?VuYu/fbDcrk1nB5+BKSRdxaBP9sZZhzyhKT/qsbyFPIJVNgjvmsQR1QwifjD?= =?us-ascii?Q?rfVGtIjClYN+orv9xGjsB5fWamfbUbEjHY1DrJM7wllV/SCVikM5PC7H3uyR?= =?us-ascii?Q?tyHnWEQzGRP4Jsx9QB3S8C1Ii5YR6xQ004CAPb7KpzIARnjbjzFke8F59ujP?= =?us-ascii?Q?yZapcZs7gFTWouJswsVtvQA2zyHVc5Fd+ibdxiDEnvD7te9IuD0x/+/qi2wX?= =?us-ascii?Q?N8R58OisyADqArNuJtwOIEhBg0rMdKsi5Hd5AipnhgFLftPG9Zksg2gMyxAM?= =?us-ascii?Q?w4SgiDj7vVcYn2miSpcJOw9PThOY3KoblyvSU7ZRRF0V2EP5TdhokI8WFFLA?= =?us-ascii?Q?vncXuBUelURo7B0BH4DfA10UFbb97znLWHagDMkWbLJZXHrEs37e4xd8p3W0?= =?us-ascii?Q?GjuWWSu5EUBLx3MPnONECjVk+0iW/TN5Xa5tETvr870IX1vgMuQgXWd4UftC?= =?us-ascii?Q?Xi7jO5G1WhMwRHjA4dmlrN4y8638bhd2GNNy7sz1Wawf+8YW+vv+br/BPW5t?= =?us-ascii?Q?ImcLk3k+KgDDVhSZIfxCA2V+bIp3nOhykI8ws6E/W7TMOR1MB3XUfNI2oYsY?= =?us-ascii?Q?c6KRqddG20yAwVC7IIJPMczqIA1tnWWw+XQitz7poFLTR+MV3UjlAWCU+K4f?= =?us-ascii?Q?W/Ydb4okyt/6haPhTDW9dQsK/avP1zmfSlRV5oGlVX0bovyepBj1RraPBfJu?= =?us-ascii?Q?qCoOJtt41OmJ9Hk79S2z2eTNgqspFw46ikIiry83XcQZ0lGpG87CdlFctzGE?= =?us-ascii?Q?OyV5RU9csCIrGdkT/xdaj+uA+TkViZi4EhM6Sp80?= Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: SN6PR11MB3504.namprd11.prod.outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: db6ac773-f4e7-4ec8-9202-08db67c59bef X-MS-Exchange-CrossTenant-originalarrivaltime: 08 Jun 2023 02:11:02.5304 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 46c98d88-e344-4ed4-8496-4ed7712e255d X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: O7yGnm9SXw5sulbAkv+TfRN3ik4IseYRqSEbb3RqK0oc/xAxpLaL8fKB17g3sjyycTzzXtp6s+nx0J3i4M2Jbg== X-MS-Exchange-Transport-CrossTenantHeadersStamped: CO1PR11MB4836 X-OriginatorOrg: intel.com X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org > -----Original Message----- > From: Maxime Coquelin > Sent: Tuesday, June 6, 2023 4:19 PM > To: dev@dpdk.org; Xia, Chenbo ; > david.marchand@redhat.com; mkp@redhat.com; fbl@redhat.com; > jasowang@redhat.com; Liang, Cunming ; Xie, Yongj= i > ; echaudro@redhat.com; eperezma@redhat.com; > amorenoz@redhat.com; lulu@redhat.com > Cc: Maxime Coquelin > Subject: [PATCH v5 26/26] vhost: add VDUSE device stop >=20 > This patch adds VDUSE device stop and cleanup of its > virtqueues. >=20 > Signed-off-by: Maxime Coquelin > --- > doc/guides/rel_notes/release_23_07.rst | 7 +++ > lib/vhost/vduse.c | 72 +++++++++++++++++++++++--- > 2 files changed, 71 insertions(+), 8 deletions(-) >=20 > diff --git a/doc/guides/rel_notes/release_23_07.rst > b/doc/guides/rel_notes/release_23_07.rst > index 7034fb664c..0f44c859da 100644 > --- a/doc/guides/rel_notes/release_23_07.rst > +++ b/doc/guides/rel_notes/release_23_07.rst > @@ -67,6 +67,13 @@ New Features > Introduced ``rte_vhost_driver_set_max_queue_num()`` to be able to limi= t > the > maximum number of supported queue pairs, required for VDUSE support. >=20 > +* **Added VDUSE support into Vhost library.** > + > + VDUSE aims at implementing vDPA devices in userspace. It can be used a= s > an > + alternative to Vhost-user when using Vhost-vDPA, but also enable > providing a > + virtio-net netdev to the host when using Virtio-vDPA driver. A > limitation in > + this release is the lack of reconnection support. > + >=20 > Removed Items > ------------- > diff --git a/lib/vhost/vduse.c b/lib/vhost/vduse.c > index d3759077ff..a509daf80c 100644 > --- a/lib/vhost/vduse.c > +++ b/lib/vhost/vduse.c > @@ -252,6 +252,44 @@ vduse_vring_setup(struct virtio_net *dev, unsigned > int index) > } > } >=20 > +static void > +vduse_vring_cleanup(struct virtio_net *dev, unsigned int index) > +{ > + struct vhost_virtqueue *vq =3D dev->virtqueue[index]; > + struct vduse_vq_eventfd vq_efd; > + int ret; > + > + if (vq =3D=3D dev->cvq && vq->kickfd >=3D 0) { > + fdset_del(&vduse.fdset, vq->kickfd); > + fdset_pipe_notify(&vduse.fdset); > + } > + > + vq_efd.index =3D index; > + vq_efd.fd =3D VDUSE_EVENTFD_DEASSIGN; > + > + ret =3D ioctl(dev->vduse_dev_fd, VDUSE_VQ_SETUP_KICKFD, &vq_efd); > + if (ret) > + VHOST_LOG_CONFIG(dev->ifname, ERR, "Failed to cleanup kickfd > for VQ %u: %s\n", > + index, strerror(errno)); > + > + close(vq->kickfd); > + vq->kickfd =3D VIRTIO_UNINITIALIZED_EVENTFD; > + > + vring_invalidate(dev, vq); > + > + rte_free(vq->batch_copy_elems); > + vq->batch_copy_elems =3D NULL; > + > + rte_free(vq->shadow_used_split); > + vq->shadow_used_split =3D NULL; > + > + vq->enabled =3D false; > + vq->ready =3D false; > + vq->size =3D 0; > + vq->last_used_idx =3D 0; > + vq->last_avail_idx =3D 0; > +} > + > static void > vduse_device_start(struct virtio_net *dev) > { > @@ -304,6 +342,23 @@ vduse_device_start(struct virtio_net *dev) > } > } >=20 > +static void > +vduse_device_stop(struct virtio_net *dev) > +{ > + unsigned int i; > + > + VHOST_LOG_CONFIG(dev->ifname, INFO, "Stopping device...\n"); > + > + vhost_destroy_device_notify(dev); > + > + dev->flags &=3D ~VIRTIO_DEV_READY; > + > + for (i =3D 0; i < dev->nr_vring; i++) > + vduse_vring_cleanup(dev, i); > + > + vhost_user_iotlb_flush_all(dev); > +} > + > static void > vduse_events_handler(int fd, void *arg, int *remove __rte_unused) > { > @@ -311,6 +366,7 @@ vduse_events_handler(int fd, void *arg, int *remove > __rte_unused) > struct vduse_dev_request req; > struct vduse_dev_response resp; > struct vhost_virtqueue *vq; > + uint8_t old_status =3D dev->status; > int ret; >=20 > memset(&resp, 0, sizeof(resp)); > @@ -339,6 +395,7 @@ vduse_events_handler(int fd, void *arg, int *remove > __rte_unused) > case VDUSE_SET_STATUS: > VHOST_LOG_CONFIG(dev->ifname, INFO, "\tnew status: 0x%08x\n", > req.s.status); > + old_status =3D dev->status; > dev->status =3D req.s.status; > resp.result =3D VDUSE_REQ_RESULT_OK; > break; > @@ -363,8 +420,12 @@ vduse_events_handler(int fd, void *arg, int *remove > __rte_unused) > return; > } >=20 > - if (dev->status & VIRTIO_DEVICE_STATUS_DRIVER_OK) > - vduse_device_start(dev); > + if ((old_status ^ dev->status) & VIRTIO_DEVICE_STATUS_DRIVER_OK) { > + if (dev->status & VIRTIO_DEVICE_STATUS_DRIVER_OK) > + vduse_device_start(dev); > + else > + vduse_device_stop(dev); > + } >=20 > VHOST_LOG_CONFIG(dev->ifname, INFO, "Request %s (%u) handled > successfully\n", > vduse_req_id_to_str(req.type), req.type); > @@ -560,12 +621,7 @@ vduse_device_destroy(const char *path) > if (vid =3D=3D RTE_MAX_VHOST_DEVICE) > return -1; >=20 > - if (dev->cvq && dev->cvq->kickfd >=3D 0) { > - fdset_del(&vduse.fdset, dev->cvq->kickfd); > - fdset_pipe_notify(&vduse.fdset); > - close(dev->cvq->kickfd); > - dev->cvq->kickfd =3D VIRTIO_UNINITIALIZED_EVENTFD; > - } > + vduse_device_stop(dev); >=20 > fdset_del(&vduse.fdset, dev->vduse_dev_fd); > fdset_pipe_notify(&vduse.fdset); > -- > 2.40.1 Reviewed-by: Chenbo Xia =20