From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 666BD42C31; Mon, 5 Jun 2023 09:56:49 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 590C041104; Mon, 5 Jun 2023 09:56:49 +0200 (CEST) Received: from mga07.intel.com (mga07.intel.com [134.134.136.100]) by mails.dpdk.org (Postfix) with ESMTP id 98228410ED for ; Mon, 5 Jun 2023 09:56:47 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1685951807; x=1717487807; h=from:to:subject:date:message-id:references:in-reply-to: content-transfer-encoding:mime-version; bh=Cw43hzTnGzVfucGv1WFKvC+wfQRw7xceyBDAUqD8Btc=; b=ISb7l6/f+OIHoycVg0eK0MOTzsQjXKp1nxLrgttP6OJOrE26iQm77xQw TPzTyZ3ZEjEtRRC2OtOsmA0IAdxs3OIok4ngrbkOu+9gqXj+stv/z0grA DOIP4Y5XN2oCjWizsTRgFCkWuw2CAXbbs3mDyOnmtcPEU40AR9kDkgGzI TjaIK4f+Sfb5/7lhgv2yuho+JPfQh3GLrvoh2CE9oKJI8flXHpt7AYd94 GqNI0WO5RCIrL4og0kBYscsIHJOvZGj6aWu9C1zLPDTbJxqhq+Fgdn+nr Qe1SQT6vlINrJI1FlkRcjJ2U02lgPM3fgPlDkb4zFVh31Bg+ZSswpMN4H w==; X-IronPort-AV: E=McAfee;i="6600,9927,10731"; a="422133581" X-IronPort-AV: E=Sophos;i="6.00,217,1681196400"; d="scan'208";a="422133581" Received: from fmsmga004.fm.intel.com ([10.253.24.48]) by orsmga105.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 05 Jun 2023 00:56:46 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10731"; a="778446166" X-IronPort-AV: E=Sophos;i="6.00,217,1681196400"; d="scan'208";a="778446166" Received: from fmsmsx602.amr.corp.intel.com ([10.18.126.82]) by fmsmga004.fm.intel.com with ESMTP; 05 Jun 2023 00:56:45 -0700 Received: from fmsmsx610.amr.corp.intel.com (10.18.126.90) by fmsmsx602.amr.corp.intel.com (10.18.126.82) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.23; Mon, 5 Jun 2023 00:56:44 -0700 Received: from fmsedg602.ED.cps.intel.com (10.1.192.136) by fmsmsx610.amr.corp.intel.com (10.18.126.90) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.23 via Frontend Transport; Mon, 5 Jun 2023 00:56:44 -0700 Received: from NAM04-BN8-obe.outbound.protection.outlook.com (104.47.74.43) by edgegateway.intel.com (192.55.55.71) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.1.2507.23; Mon, 5 Jun 2023 00:56:43 -0700 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=MaslFbeqidBf5FG8YQbJryLio6E9+uVDwKouRIYsHia/lPgH7VoRjO9KQzO9szJzECNolnZQYer5EXOhnCkjRF1v2EPlJK0q32SxjxaOxrDIn3GmMjTc5aCtBn4Rc46wayCYSCWj6t4DZ4fDfAyIqNG0+nNyyQuDkKKipQq5wHuY8Q7Y3t8XxSeRpHn5DqxGv17u/Ui2/UMq7DkNtgFppnnbN5PTc1KWY2xhzcNRoz3c11z7ms8VohkHzEJgxKl1bg0mgAOUh4MzaVohXf4YhLwvhy5BUz4iMgCG2O66zUe13pQ0pEmdjuoFmjMUko94hXsBCbCXElMmXUnS7ITOgg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=ZZSG6oWqTX2obIlZLE9hxtDV1mGXRzCb8JMLyZs2++w=; b=kseXkNpfZqZiUgbP0PJnuKNyvBz8MwBjMsdz7iNTs1KKJgdmI1/O5wb7jY1bbtACRT6TCMLJuefuPEzizr+48aboU0aGTrqNr8Fg6ZmESyIBB2pyeNzpO/teTfsfB3V5g4PDFqtmgU2t4D2vF24coa2uy6Gv/Ok4XqzlS9O0zUsIIvwN/TBdEvzdFm64aBltWSnjxUeE9P5zpwaaqaP8cZGdSkeLnx3RTCDmXjj3x4sLK0DTP6q4/TajkYous5+GV4mClb88a5SkDkDggAUwe8r6pqZn719g3abUHzErBI21adX2iUPmML1XX5ODMpyadLDru7zLFj9khxe+EnzzGQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=intel.com; dmarc=pass action=none header.from=intel.com; dkim=pass header.d=intel.com; arc=none Received: from SN6PR11MB3504.namprd11.prod.outlook.com (2603:10b6:805:d0::17) by PH7PR11MB6859.namprd11.prod.outlook.com (2603:10b6:510:1ef::11) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6455.32; Mon, 5 Jun 2023 07:56:35 +0000 Received: from SN6PR11MB3504.namprd11.prod.outlook.com ([fe80::e508:44ff:3d34:60bc]) by SN6PR11MB3504.namprd11.prod.outlook.com ([fe80::e508:44ff:3d34:60bc%4]) with mapi id 15.20.6455.030; Mon, 5 Jun 2023 07:56:35 +0000 From: "Xia, Chenbo" To: Maxime Coquelin , "dev@dpdk.org" , "david.marchand@redhat.com" , "mkp@redhat.com" , "fbl@redhat.com" , "jasowang@redhat.com" , "Liang, Cunming" , "Xie, Yongji" , "echaudro@redhat.com" , "eperezma@redhat.com" , "amorenoz@redhat.com" , "lulu@redhat.com" Subject: RE: [PATCH v4 26/26] vhost: add VDUSE device stop Thread-Topic: [PATCH v4 26/26] vhost: add VDUSE device stop Thread-Index: AQHZlMUOupTNtWjbOUCMr6I0eRofnq973Oow Date: Mon, 5 Jun 2023 07:56:35 +0000 Message-ID: References: <20230601200812.672233-1-maxime.coquelin@redhat.com> <20230601200812.672233-27-maxime.coquelin@redhat.com> In-Reply-To: <20230601200812.672233-27-maxime.coquelin@redhat.com> Accept-Language: en-US, zh-CN Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=intel.com; x-ms-publictraffictype: Email x-ms-traffictypediagnostic: SN6PR11MB3504:EE_|PH7PR11MB6859:EE_ x-ms-office365-filtering-correlation-id: 33b9abfa-6058-456b-de2d-08db659a623e x-ld-processed: 46c98d88-e344-4ed4-8496-4ed7712e255d,ExtAddr x-ms-exchange-senderadcheck: 1 x-ms-exchange-antispam-relay: 0 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: w+dfRsVgDc8AS5Ls8NzasfDMwvgK4oX5jqdlG9CaBPvV4aMcSZ6Y1T5Q35hS4qcL9/yFDMwAVRs7DREpfNJTURAmV09SkFy4b1s9UkhytDtRCjIpsY6/HG77gyHaAQE2W7en9HzdmbJFPk6FVwSFIC76hV+FxyHI3w3OdVkc8ahgmiLxaM6IeKbtP5PMsIMh5yO01ukYkfS3womHPrBxE/yUw/XzsOk+EHgvFMvQBwhG66ndHjEoouRGy14sqHoB1UWgaKFZAqE/L7SBwpMU0CfK75dhh6BFcc2atUTwZ6KdOqZ8aybEnVydVpcP8DvcZbBomcPBjapL7XRybdccttuvMJHWVzEjeaUK0Xqv7hFRingDa1m0Z+RfteKyvHLloB0lcF7LlPH0JZdO9vmjZfoskJjv69AoQu9djnUx/6WxEk2lYCpadiKJmcbCEfBaBQCzdOBFkdgxY/Yuj0n8nNRjt/7tSfwpNE5NQNURNcczqK8SUfwgCK3ofbS8rAS4zNvzgvOfAqP6aN+HRjMopxzpiA3NCrenoXQ31VqFBy6bCJhmkJaB0O5vgeIe0mIYTwVQb0v/R/+3cO3bF475HZYKSOW7ipO75YfUpkXDE4aEMjWfZp8nXlR0QxisSXKVTXulXPEvYqwwtxSXIA7hJg== x-forefront-antispam-report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:SN6PR11MB3504.namprd11.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230028)(136003)(346002)(39860400002)(366004)(376002)(396003)(451199021)(6506007)(9686003)(186003)(26005)(53546011)(83380400001)(7416002)(7696005)(71200400001)(2906002)(8676002)(8936002)(82960400001)(110136005)(55016003)(478600001)(33656002)(5660300002)(38100700002)(122000001)(86362001)(316002)(38070700005)(41300700001)(66446008)(52536014)(66556008)(921005)(64756008)(66476007)(66946007)(76116006); DIR:OUT; SFP:1102; x-ms-exchange-antispam-messagedata-chunkcount: 1 x-ms-exchange-antispam-messagedata-0: =?us-ascii?Q?HAraRRqGYxvc4YDx+nlYuZ7xEOEgih9ae5tdA+Kok4Pvv7kPwshE7B2Yt4Yr?= =?us-ascii?Q?/5u3czvOy8HLKP3qv7R6bUyw9WR3oSyTpTwZobtnfbQSKsgrKIRZHebAKjkE?= =?us-ascii?Q?EQAeBx7QGfzjfDseyqSKvkxSuFyWOaMwQFPrePJ1qfyAn9dQSaUtIKEgX6yT?= =?us-ascii?Q?/MpkUY1khJQ+rVlT0gcoYUvJpg/z3JNs9X63QIXaBzvQYbphoS8jIzfsDt6Y?= =?us-ascii?Q?E5E/Mgqn2Eetqi4S+aKJ6elRJdJnD8pS37ko6RSFiBQYGGj9wF1jBUHjoIuR?= =?us-ascii?Q?KS7L+pWq4afFJ55X/vDlwZlDRfuebokV1fQz/Za054YxLS+gnKtv3xY2MyCj?= =?us-ascii?Q?Yba1FOWSni/LltW2BnpQJwr/kHjKw0InBN9XC32NGhnLdIqdrEt9P/K9ENP5?= =?us-ascii?Q?VqEnb7QKO4q3pvCh0DwfKMDMLIlHMFMw2d1rhLGj51srYtA4TtpWLSUBTspN?= =?us-ascii?Q?6lsMkXivIikYjgWQHKmjmjNC5Af6a6XX2xdPx4zxealBQ8W60I/l02kmJDdh?= =?us-ascii?Q?yE6cYW19IOkDd+E3lYCq8o+cYno4MRoiVsfdp2tSiKxX/Z2FQIBLssdAyX1u?= =?us-ascii?Q?sCKRksYWqRYBVflmDxGZu1Qc96yes6ZGFifT/SiOWQQBRZdVhR3bLz14+Rmr?= =?us-ascii?Q?UUmOBEAj7YDRSNJfWXDXA6IOWF8uK6TWh7d4mtUYSBSsgM8/sEJkeiAEtJCE?= =?us-ascii?Q?Thh9DG/vC6dYN1tRWiSbqFcCtz7LtvllKiEdKiA2jWJVXDEpYPyZNTmGkNG1?= =?us-ascii?Q?A8lTONle8qzJWGK4vtup0ey8pGxOvABWVOXeQ8TV9nFoSyX65YF93/geDzNT?= =?us-ascii?Q?WbvikMv4yQrqXqP92cYxi8kONdhLAhfFW9HptIsKP/HHTIhPT5hWZ3sRnuUZ?= =?us-ascii?Q?ns5MEejtJ7bfb6FZgY44Fvwt9QxM84VtbxQo4Xi+NxcqyDXY9LzLL1+R3d+E?= =?us-ascii?Q?DXX6YbWzoyqxCeqkKWX0SwZnrkPA9eA51X87S828QdZp41gpEvVhmduaWt0q?= =?us-ascii?Q?paOdzETEQsOVtfB9iwhg4uGNtcsysObMoMuFmlvd8lC5s+Fk7saAFpOvRs8G?= =?us-ascii?Q?6/fB0cVBMwIhG7BN9FxgzjVMT7K7Dx/2nWLn32mvR0V4oYBlYi3fBNlUobaJ?= =?us-ascii?Q?DBlYBjeHwPx2oD7UBjL7XPwuoi0FSf+nNMTGbQWDBAcKE36ebRikT3nE+0pr?= =?us-ascii?Q?TViLcB/j3tzoN140sKDoVwL68UJC1LN9Q8AoAGDK6Ef5aUcy1ivi5JDShATo?= =?us-ascii?Q?dhda+mvCJdxQZ2FFCXkxRBmf57pHiw7nyNAqqkfJErg6LoKKqKR9EPdVJ/R9?= =?us-ascii?Q?8ozmnSGE7o9/uZwzTHnjPI/fwIflV40n2CyBDqCeJu/mue2Boo4kpAqTzwaS?= =?us-ascii?Q?2K4zv5CyqrDMNFglB0iS6ujJ39TfE/JeP3i1Ti+BvDfjc0CoSNghFGFnkd6v?= =?us-ascii?Q?xsuhpIREtFhtni2E8BA9yhsWj+P1wr6CY5ARg6md2h6KQVzxnadKG35Kupb5?= =?us-ascii?Q?lON6Qr2ZpmEgZCTKhZojXbbRtw0SbNtGsEfmW+wjiwPCvHi95JidkFx2Ywn/?= =?us-ascii?Q?ifU8KVoA1dW7WG+o6kST8GbRwM/61ahG0IuDOCnZ?= Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: SN6PR11MB3504.namprd11.prod.outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: 33b9abfa-6058-456b-de2d-08db659a623e X-MS-Exchange-CrossTenant-originalarrivaltime: 05 Jun 2023 07:56:35.0696 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 46c98d88-e344-4ed4-8496-4ed7712e255d X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: 8nN3/fOJYHA/buG4L1tzEH2gLRwjuEVnnQZU/0g0jFYksRjc46zQ1cTQ8nlZXJFUov5UXqA4Pba4KkMaKZZomQ== X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH7PR11MB6859 X-OriginatorOrg: intel.com X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org > -----Original Message----- > From: Maxime Coquelin > Sent: Friday, June 2, 2023 4:08 AM > To: dev@dpdk.org; Xia, Chenbo ; > david.marchand@redhat.com; mkp@redhat.com; fbl@redhat.com; > jasowang@redhat.com; Liang, Cunming ; Xie, Yongj= i > ; echaudro@redhat.com; eperezma@redhat.com; > amorenoz@redhat.com; lulu@redhat.com > Cc: Maxime Coquelin > Subject: [PATCH v4 26/26] vhost: add VDUSE device stop >=20 > This patch adds VDUSE device stop and cleanup of its > virtqueues. >=20 > Signed-off-by: Maxime Coquelin > --- > doc/guides/rel_notes/release_23_07.rst | 6 +++ > lib/vhost/vduse.c | 72 +++++++++++++++++++++++--- > 2 files changed, 70 insertions(+), 8 deletions(-) >=20 > diff --git a/doc/guides/rel_notes/release_23_07.rst > b/doc/guides/rel_notes/release_23_07.rst > index 7034fb664c..6f43e8e633 100644 > --- a/doc/guides/rel_notes/release_23_07.rst > +++ b/doc/guides/rel_notes/release_23_07.rst > @@ -67,6 +67,12 @@ New Features > Introduced ``rte_vhost_driver_set_max_queue_num()`` to be able to limi= t > the > maximum number of supported queue pairs, required for VDUSE support. >=20 > +* **Added VDUSE support into Vhost library.** > + > + VDUSE aims at implementing vDPA devices in userspace. It can be used a= s > an > + alternative to Vhost-user when using Vhost-vDPA, but also enable > providing a > + virtio-net netdev to the host when using Virtio-vDPA driver. > + >=20 > Removed Items > ------------- > diff --git a/lib/vhost/vduse.c b/lib/vhost/vduse.c > index 699cfed9e3..f421b1cf4c 100644 > --- a/lib/vhost/vduse.c > +++ b/lib/vhost/vduse.c > @@ -252,6 +252,44 @@ vduse_vring_setup(struct virtio_net *dev, unsigned > int index) > } > } >=20 > +static void > +vduse_vring_cleanup(struct virtio_net *dev, unsigned int index) > +{ > + struct vhost_virtqueue *vq =3D dev->virtqueue[index]; > + struct vduse_vq_eventfd vq_efd; > + int ret; > + > + if (vq =3D=3D dev->cvq && vq->kickfd >=3D 0) { > + fdset_del(&vduse.fdset, vq->kickfd); > + fdset_pipe_notify(&vduse.fdset); > + } > + > + vq_efd.index =3D index; > + vq_efd.fd =3D VDUSE_EVENTFD_DEASSIGN; > + > + ret =3D ioctl(dev->vduse_dev_fd, VDUSE_VQ_SETUP_KICKFD, &vq_efd); > + if (ret) > + VHOST_LOG_CONFIG(dev->ifname, ERR, "Failed to cleanup kickfd > for VQ %u: %s\n", > + index, strerror(errno)); > + > + close(vq->kickfd); > + vq->kickfd =3D VIRTIO_UNINITIALIZED_EVENTFD; > + > + vring_invalidate(dev, vq); > + > + rte_free(vq->batch_copy_elems); > + vq->batch_copy_elems =3D NULL; > + > + rte_free(vq->shadow_used_split); > + vq->shadow_used_split =3D NULL; > + > + vq->enabled =3D false; > + vq->ready =3D false; > + vq->size =3D 0; > + vq->last_used_idx =3D 0; > + vq->last_avail_idx =3D 0; > +} > + > static void > vduse_device_start(struct virtio_net *dev) > { > @@ -304,6 +342,23 @@ vduse_device_start(struct virtio_net *dev) > } > } >=20 > +static void > +vduse_device_stop(struct virtio_net *dev) > +{ > + unsigned int i; > + > + VHOST_LOG_CONFIG(dev->ifname, INFO, "Stopping device...\n"); > + > + vhost_destroy_device_notify(dev); > + > + dev->flags &=3D ~VIRTIO_DEV_READY; > + > + for (i =3D 0; i < dev->nr_vring; i++) > + vduse_vring_cleanup(dev, i); > + > + vhost_user_iotlb_flush_all(dev); > +} > + > static void > vduse_events_handler(int fd, void *arg, int *remove __rte_unused) > { > @@ -311,6 +366,7 @@ vduse_events_handler(int fd, void *arg, int *remove > __rte_unused) > struct vduse_dev_request req; > struct vduse_dev_response resp; > struct vhost_virtqueue *vq; > + uint8_t old_status; > int ret; >=20 > memset(&resp, 0, sizeof(resp)); > @@ -339,10 +395,15 @@ vduse_events_handler(int fd, void *arg, int *remove > __rte_unused) > case VDUSE_SET_STATUS: > VHOST_LOG_CONFIG(dev->ifname, INFO, "\tnew status: 0x%08x\n", > req.s.status); > + old_status =3D dev->status; > dev->status =3D req.s.status; >=20 > - if (dev->status & VIRTIO_DEVICE_STATUS_DRIVER_OK) > - vduse_device_start(dev); > + if ((old_status ^ dev->status) & > VIRTIO_DEVICE_STATUS_DRIVER_OK) { > + if (dev->status & VIRTIO_DEVICE_STATUS_DRIVER_OK) > + vduse_device_start(dev); > + else > + vduse_device_stop(dev); > + } >=20 > resp.result =3D VDUSE_REQ_RESULT_OK; > break; > @@ -560,12 +621,7 @@ vduse_device_destroy(const char *path) > if (vid =3D=3D RTE_MAX_VHOST_DEVICE) > return -1; >=20 > - if (dev->cvq && dev->cvq->kickfd >=3D 0) { > - fdset_del(&vduse.fdset, dev->cvq->kickfd); > - fdset_pipe_notify(&vduse.fdset); > - close(dev->cvq->kickfd); > - dev->cvq->kickfd =3D VIRTIO_UNINITIALIZED_EVENTFD; > - } > + vduse_device_stop(dev); >=20 > fdset_del(&vduse.fdset, dev->vduse_dev_fd); > fdset_pipe_notify(&vduse.fdset); > -- > 2.40.1 Reviewed-by: Chenbo Xia =20