From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id D7C3842A9D; Tue, 9 May 2023 07:35:30 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id C24B642D3A; Tue, 9 May 2023 07:35:26 +0200 (CEST) Received: from mga17.intel.com (mga17.intel.com [192.55.52.151]) by mails.dpdk.org (Postfix) with ESMTP id 0A788410FA for ; Tue, 9 May 2023 07:35:22 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1683610523; x=1715146523; h=from:to:subject:date:message-id:references:in-reply-to: content-transfer-encoding:mime-version; bh=JIyXE9igrXn2ZIdt9W7zg/LrESHpwb1zyMnWsGvYxyQ=; b=h4pW1sCsGsB2/VC6dMSyXYzdodDf1la+1ufwkFb8/LYZxMp9g5QgxNoK mqJkg6/ylFSYHPmVyaAktl12GCXMV8ZD8E6hC7v11b1Vm/z5t4q5e2GG9 z9z9FkIWaQZutPGNUPYquoQEVzro7CcRwlVHHZzjkSvf0KxmSLg2MGA4r VseP1NjJJ45flc7P/hloOj2xkAyi6Mf5v2sbNBSUuvi2eOEpraobHqUej j2HqmKrwyddeBKtkiTut2IKPDEDaZl/mki5WYAr6+UJLhb2q8GiAhl0YM 9KOdfaaxR0/npLPMULsMjus/ZGuNos3P3dHMBlB7jfgu0pNmsHBHxu/4O w==; X-IronPort-AV: E=McAfee;i="6600,9927,10704"; a="330172940" X-IronPort-AV: E=Sophos;i="5.99,261,1677571200"; d="scan'208";a="330172940" Received: from fmsmga006.fm.intel.com ([10.253.24.20]) by fmsmga107.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 08 May 2023 22:35:14 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10704"; a="945134731" X-IronPort-AV: E=Sophos;i="5.99,261,1677571200"; d="scan'208";a="945134731" Received: from orsmsx602.amr.corp.intel.com ([10.22.229.15]) by fmsmga006.fm.intel.com with ESMTP; 08 May 2023 22:35:14 -0700 Received: from orsmsx610.amr.corp.intel.com (10.22.229.23) by ORSMSX602.amr.corp.intel.com (10.22.229.15) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.23; Mon, 8 May 2023 22:35:13 -0700 Received: from ORSEDG601.ED.cps.intel.com (10.7.248.6) by orsmsx610.amr.corp.intel.com (10.22.229.23) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.23 via Frontend Transport; Mon, 8 May 2023 22:35:13 -0700 Received: from NAM12-DM6-obe.outbound.protection.outlook.com (104.47.59.168) by edgegateway.intel.com (134.134.137.102) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.1.2507.23; Mon, 8 May 2023 22:35:13 -0700 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=KvsH2mSZlgsZX5bilsZ+OG6gXSdV6SEFsSbYXHS/+OAlxXecaDPUJu3vt55hl2NkQMr7xO0O1t6ETm/FNoazFuUwSnq9LR9+qiBDIKMmpuEnj+A/yOhYqB1Bj1BU4dscBFU8CxpUBOYjHkF09OmPwgRsAG4DqIFc5Z57LrJybxt8eq1FetJxauDmsc1aKr0/B5QV8UQy0hMVKSwXzHm55lZD2mp+vu94BrliNpiz2aK0GqIfqq0DMskK9EEZ6JsZP0KK8O6ZfVc6KbPNFM4klSwoT2BOCtoFxetu4/A86F0i6HhQ/xhVSXAwo5WtDS+kesorGhepWQ4/hPwgFP6oqg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=n9roLlE3ilBa3Jlkyj+v/34kDkDgqMq99o39KSsJ9po=; b=A4h6Hk9hE9LqZF6VuBG/dtVzOtDDF405JU8U/LR7nPgSAroc6q9wsb5ZHrz+s4vx6Uz/JiN5hMspDpr5j0sQafv2ggjNMK2wnVSStmsASZchSvChpszgmy7eq13FGnCb8RWIpdag4lIf6m1pvO54dJBy6/dy/pTfZHR11VB/tblSF/n+2lkI2nwvUwwuXpX+yH04NPHhmzh7V+PBpOx+sKr+GWz+HNxJGd0XZLk+j68QRA0io41A7NlvnN06EX57ChFgTgc2xzDNIsv+OUnYSpRlMlbxFpwoTFT/3eU5lbuZ2iTram9PQZSbuMbOoJYkFNlzroBr3W2zl6HGSQrLBA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=intel.com; dmarc=pass action=none header.from=intel.com; dkim=pass header.d=intel.com; arc=none Received: from SN6PR11MB3504.namprd11.prod.outlook.com (2603:10b6:805:d0::17) by SA1PR11MB6807.namprd11.prod.outlook.com (2603:10b6:806:24e::7) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6363.32; Tue, 9 May 2023 05:35:12 +0000 Received: from SN6PR11MB3504.namprd11.prod.outlook.com ([fe80::5797:849:7a10:560d]) by SN6PR11MB3504.namprd11.prod.outlook.com ([fe80::5797:849:7a10:560d%6]) with mapi id 15.20.6363.032; Tue, 9 May 2023 05:35:12 +0000 From: "Xia, Chenbo" To: Maxime Coquelin , "dev@dpdk.org" , "david.marchand@redhat.com" , "mkp@redhat.com" , "fbl@redhat.com" , "jasowang@redhat.com" , "Liang, Cunming" , "Xie, Yongji" , "echaudro@redhat.com" , "eperezma@redhat.com" , "amorenoz@redhat.com" Subject: RE: [RFC 26/27] vhost: add VDUSE device startup Thread-Topic: [RFC 26/27] vhost: add VDUSE device startup Thread-Index: AQHZY+etXJvj6lDV6Eqetdrbj41FRq9RqDHA Date: Tue, 9 May 2023 05:35:12 +0000 Message-ID: References: <20230331154259.1447831-1-maxime.coquelin@redhat.com> <20230331154259.1447831-27-maxime.coquelin@redhat.com> In-Reply-To: <20230331154259.1447831-27-maxime.coquelin@redhat.com> Accept-Language: en-US, zh-CN Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=intel.com; x-ms-publictraffictype: Email x-ms-traffictypediagnostic: SN6PR11MB3504:EE_|SA1PR11MB6807:EE_ x-ms-office365-filtering-correlation-id: fd3c0e91-352b-4fa5-1e1b-08db504f28d9 x-ld-processed: 46c98d88-e344-4ed4-8496-4ed7712e255d,ExtAddr x-ms-exchange-senderadcheck: 1 x-ms-exchange-antispam-relay: 0 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: bEGsZnKY8LjcUVyPZHCZMAAPnvk7TvY7u426Qa/Ewx/k1o/jq7XLUis3hQJZlX2vyNf9EfP7Wh4l88UdaDfc2UtUd9qMVFE7vM0oWrjhTTLWh9iZvgvSYq639+Wsk7ErJbz8OP0VOlWoMuw4cUkXL/AR5es6KDv3K1s19NsX/LXjRbPf/b88VJxFwaxsZSMuG+unAtyIyllqtFD9UqWn4/eY5y7py0iSMlGTetMTjuyvKaGo/qyIcRbgAzQCzuTqCtM4dJGp2Rop9zi37do8Aqx7stxZXwzZlf0LpbTrG/pKJ3wqIIqdRZpX2EAIUDff6vrWke5aNaZzOafYXSZhlM18he/2JA/q3WCI7F91rm3W7XS/Db1ox97hxeCgogKbOAm/NIddJNskAmWosozsOAH75EjCcrtDOfBA6jw+ujNa9Hs0QjGM4QnQyXS7+PnBogXPrVLvGNPsA1iOh+AAeG/cAehFVHzKtReWLWKwvM89Nn+IOKz8SaXuwNch9wuLjCgb/Y4fj+IkyXQZHkt8GxfYuOBAjVwntNsbkZwP5i+PtcOhzTp4+ktU+1Dbq3jbPcTG776QSR6brv8vjyNYRtKD275I0wQ1tmfAFALm4dZDr2QNOc1SWFFPQk6JMTSb4YkoR5vHaMLTscBM/KP/8Q== x-forefront-antispam-report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:SN6PR11MB3504.namprd11.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230028)(376002)(396003)(39860400002)(366004)(346002)(136003)(451199021)(33656002)(52536014)(66556008)(38100700002)(2906002)(7416002)(55016003)(316002)(86362001)(8676002)(66446008)(64756008)(76116006)(82960400001)(66946007)(5660300002)(66476007)(38070700005)(8936002)(921005)(41300700001)(122000001)(83380400001)(186003)(53546011)(6506007)(26005)(9686003)(478600001)(7696005)(110136005)(71200400001); DIR:OUT; SFP:1102; x-ms-exchange-antispam-messagedata-chunkcount: 1 x-ms-exchange-antispam-messagedata-0: =?us-ascii?Q?KPFmThC8kVmjDBAHVmwRCEHli5HNQ5go2MyTfouuu5s/sYC/wBY8lNruyohp?= =?us-ascii?Q?jEwmax3coTTzmKxmDuhGmW7PBr0qOr3czmMQvGaOAWww9vbpyMBY4r0Je3nb?= =?us-ascii?Q?UTsIAiKP/0P5mFyqlbBxBlUG27CTCFbeSZJazyFmM+whFU9yauTYoVxOHs5p?= =?us-ascii?Q?d76t2OiXWBdpX73vpAUFpAQ5+0GTdraIexHlfbUM0ZIL+M9ZtvK1OAgMwdx8?= =?us-ascii?Q?adf+NV53k4PqNHPZdEe+daGX0vc59+9BSgHm/Lb0HkA4KBm6qa7kXmy77sGG?= =?us-ascii?Q?nRc5eWSEhHkQsF1xEYGmPDwIK3zbPm1SYXn88frIr226dF06k0q/A5fJwBc+?= =?us-ascii?Q?JL1L+6ySAME5VWTr6UJQmUWdRqTGfhDKQ3f7SouV/2MQSOv6Ok292BcleMXY?= =?us-ascii?Q?55npOw0qE7DIV6dFbJl7XcIVd27dy+qAyKvKjrpLhxG2VdkvXte5MgiPGmJn?= =?us-ascii?Q?TgclvNvW8NAQQrTvPlpLDkMBpBAtHJNh0kw4qqnyUlZt+A8qkTEejc9voQex?= =?us-ascii?Q?fMwqo9rNOp3yQ4bvaddHg22QEsAwqTO1n2zrWV+C2L/SrzayknN+j22udWyK?= =?us-ascii?Q?DrRqZPdU34Ct6Q4ZoCG1Z7EkVUsyigs4GWfr5BiM1uEEne8ysIFjwU1WWE5c?= =?us-ascii?Q?pEI0iTTHm4EG0LmJGiKj6zqb9ncwR2evI/Y0T5kB/6O4Yf3Nn6Ieso7T1vhK?= =?us-ascii?Q?h0l3a+EYv4dI11ZK7w9IZ9aBj06+mPAKVqyTVI4rO6SWXQ80V6pbm5o3u7d/?= =?us-ascii?Q?hrsNFEInecLgx0fJRcvwVwjsA7qzg/r5lmkoPtsQGiJtcajHMnbp+h3pb/V4?= =?us-ascii?Q?nPOvSXyiBj0BZR/mk5BygVSZF2lk31gapYz+l2HJt1UKgpThWtdH0GGqDUfg?= =?us-ascii?Q?L49609NjG2vfgXmcgxnpHsEGd4rWlm7gc+B58d+J7/cmACCih/r3a8WoMfU0?= =?us-ascii?Q?KWzJKse8pgvqaWc2NE+Y+QRTeX/XljmgdWxjOhR+hVsdhptlndnn6BCK+yY6?= =?us-ascii?Q?IdEkedK/p/u02rhguPzj1oEG6otmWthLj8f2U2acOt+5qA/kT/jke6aMIuZk?= =?us-ascii?Q?gJStFecYK9o07ah1ZUeSg1Z/WxRwyn9x1Ykv3ymKTM//GChTOP20+2CwqYEn?= =?us-ascii?Q?2E1ev73Sd9z7osK7riCCGVHKQHnajfls3JF9UiBy1wj7+d1eqNfJRrZ7O5HD?= =?us-ascii?Q?U/536ZQYkgVFpk0djo25oHwRQi/RzjgDW19BcyQkz3yiwUKJfB3niBUT8Nwb?= =?us-ascii?Q?DFaO2IJVj02WX8zbHzU85X70MzX0cTSto9CDIwTgdL6WzoqHQIAUk1ISKkbr?= =?us-ascii?Q?kuAO8Km86wVaWuDksOPPl5kTH5ZncE3NgcStppcvIf77me9WVU4xzWSEP5P/?= =?us-ascii?Q?Y+/sdlBtX9+JWeUz/jnaAGjnrUSO+fsRy7z381e+s3fYz2xA21CwRT6E6mMl?= =?us-ascii?Q?Znqcjlz4tK8WF0ARe1LsWPI8TB3BjtD2ipdEohytuTPR5xBZwcvLukGwGnJF?= =?us-ascii?Q?nIrokNxMydNuFcqaLudxeJiOc5s0cfd5+KZuh5qXpl5PRAnykl3hfhv4hnF0?= =?us-ascii?Q?b80HrDlKM//zHIYhC9mVMlIOjrvTy3faIKmXzK2R?= Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: SN6PR11MB3504.namprd11.prod.outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: fd3c0e91-352b-4fa5-1e1b-08db504f28d9 X-MS-Exchange-CrossTenant-originalarrivaltime: 09 May 2023 05:35:12.0992 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 46c98d88-e344-4ed4-8496-4ed7712e255d X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: 26uim7ERA2FbNVlL9yIzNl3fbIvMsh9cwl7rmjmeRzVJZKgFK9HPkJYf/RkYJvytmbmVyaKDiQDFBm7iSEmvsA== X-MS-Exchange-Transport-CrossTenantHeadersStamped: SA1PR11MB6807 X-OriginatorOrg: intel.com X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org > -----Original Message----- > From: Maxime Coquelin > Sent: Friday, March 31, 2023 11:43 PM > To: dev@dpdk.org; david.marchand@redhat.com; Xia, Chenbo > ; mkp@redhat.com; fbl@redhat.com; > jasowang@redhat.com; Liang, Cunming ; Xie, Yongj= i > ; echaudro@redhat.com; eperezma@redhat.com; > amorenoz@redhat.com > Cc: Maxime Coquelin > Subject: [RFC 26/27] vhost: add VDUSE device startup >=20 > This patch adds the device and its virtqueues > initialization once the Virtio driver has set the DRIVER_OK > in the Virtio status register. >=20 > Signed-off-by: Maxime Coquelin > --- > lib/vhost/vduse.c | 118 ++++++++++++++++++++++++++++++++++++++++++++++ > 1 file changed, 118 insertions(+) >=20 > diff --git a/lib/vhost/vduse.c b/lib/vhost/vduse.c > index b5b9fa2eb1..1cd04b4872 100644 > --- a/lib/vhost/vduse.c > +++ b/lib/vhost/vduse.c > @@ -127,6 +127,120 @@ static struct vhost_backend_ops vduse_backend_ops = =3D > { > .inject_irq =3D vduse_inject_irq, > }; >=20 > +static void > +vduse_vring_setup(struct virtio_net *dev, unsigned int index) > +{ > + struct vhost_virtqueue *vq =3D dev->virtqueue[index]; > + struct vhost_vring_addr *ra =3D &vq->ring_addrs; > + struct vduse_vq_info vq_info; > + struct vduse_vq_eventfd vq_efd; > + int ret; > + > + vq_info.index =3D index; > + ret =3D ioctl(dev->vduse_dev_fd, VDUSE_VQ_GET_INFO, &vq_info); > + if (ret) { > + VHOST_LOG_CONFIG(dev->ifname, ERR, "Failed to get VQ %u > info: %s\n", > + index, strerror(errno)); > + return; > + } > + > + VHOST_LOG_CONFIG(dev->ifname, INFO, "VQ %u info:\n", index); > + VHOST_LOG_CONFIG(dev->ifname, INFO, "\tnum: %u\n", vq_info.num); > + VHOST_LOG_CONFIG(dev->ifname, INFO, "\tdesc_addr: %llx\n", > vq_info.desc_addr); > + VHOST_LOG_CONFIG(dev->ifname, INFO, "\tdriver_addr: %llx\n", > vq_info.driver_addr); > + VHOST_LOG_CONFIG(dev->ifname, INFO, "\tdevice_addr: %llx\n", > vq_info.device_addr); > + VHOST_LOG_CONFIG(dev->ifname, INFO, "\tavail_idx: %u\n", > vq_info.split.avail_index); > + VHOST_LOG_CONFIG(dev->ifname, INFO, "\tready: %u\n", vq_info.ready); > + > + vq->last_avail_idx =3D vq_info.split.avail_index; > + vq->size =3D vq_info.num; > + vq->ready =3D vq_info.ready; > + vq->enabled =3D true; > + ra->desc_user_addr =3D vq_info.desc_addr; > + ra->avail_user_addr =3D vq_info.driver_addr; > + ra->used_user_addr =3D vq_info.device_addr; > + > + vq->shadow_used_split =3D rte_malloc_socket(NULL, > + vq->size * sizeof(struct vring_used_elem), > + RTE_CACHE_LINE_SIZE, 0); > + vq->batch_copy_elems =3D rte_malloc_socket(NULL, > + vq->size * sizeof(struct batch_copy_elem), > + RTE_CACHE_LINE_SIZE, 0); > + > + vhost_user_iotlb_rd_lock(vq); > + if (vring_translate(dev, vq)) > + VHOST_LOG_CONFIG(dev->ifname, ERR, "Failed to translate > vring %d addresses\n", > + index); > + vhost_user_iotlb_rd_unlock(vq); > + > + vq->kickfd =3D eventfd(0, EFD_NONBLOCK | EFD_CLOEXEC); > + if (vq->kickfd < 0) { > + VHOST_LOG_CONFIG(dev->ifname, ERR, "Failed to init kickfd for > VQ %u: %s\n", > + index, strerror(errno)); > + vq->kickfd =3D VIRTIO_INVALID_EVENTFD; > + return; > + } > + > + vq_efd.index =3D index; > + vq_efd.fd =3D vq->kickfd; > + > + ret =3D ioctl(dev->vduse_dev_fd, VDUSE_VQ_SETUP_KICKFD, &vq_efd); > + if (ret) { > + VHOST_LOG_CONFIG(dev->ifname, ERR, "Failed to setup kickfd for > VQ %u: %s\n", > + index, strerror(errno)); > + close(vq->kickfd); > + vq->kickfd =3D VIRTIO_UNINITIALIZED_EVENTFD; > + return; > + } > +} > + > +static void > +vduse_device_start(struct virtio_net *dev) > +{ > + unsigned int i, ret; > + > + dev->notify_ops =3D vhost_driver_callback_get(dev->ifname); > + if (!dev->notify_ops) { > + VHOST_LOG_CONFIG(dev->ifname, ERR, > + "Failed to get callback ops for driver\n"); > + return; > + } > + > + ret =3D ioctl(dev->vduse_dev_fd, VDUSE_DEV_GET_FEATURES, &dev- > >features); > + if (ret) { > + VHOST_LOG_CONFIG(dev->ifname, ERR, "Failed to get > features: %s\n", > + strerror(errno)); > + return; > + } > + > + VHOST_LOG_CONFIG(dev->ifname, INFO, "negotiated Virtio features: > 0x%" PRIx64 "\n", > + dev->features); > + > + if (dev->features & > + ((1ULL << VIRTIO_NET_F_MRG_RXBUF) | > + (1ULL << VIRTIO_F_VERSION_1) | > + (1ULL << VIRTIO_F_RING_PACKED))) { > + dev->vhost_hlen =3D sizeof(struct virtio_net_hdr_mrg_rxbuf); > + } else { > + dev->vhost_hlen =3D sizeof(struct virtio_net_hdr); > + } > + > + for (i =3D 0; i < dev->nr_vring; i++) > + vduse_vring_setup(dev, i); > + > + dev->flags |=3D VIRTIO_DEV_READY; > + > + if (dev->notify_ops->new_device(dev->vid) =3D=3D 0) > + dev->flags |=3D VIRTIO_DEV_RUNNING; > + > + for (i =3D 0; i < dev->nr_vring; i++) { > + struct vhost_virtqueue *vq =3D dev->virtqueue[i]; > + > + if (dev->notify_ops->vring_state_changed) > + dev->notify_ops->vring_state_changed(dev->vid, i, vq- > >enabled); > + } > +} > + > static void > vduse_events_handler(int fd, void *arg, int *remove __rte_unused) > { > @@ -167,6 +281,10 @@ vduse_events_handler(int fd, void *arg, int *remove > __rte_unused) > VHOST_LOG_CONFIG(dev->ifname, INFO, "\tnew status: 0x%08x\n", > req.s.status); > dev->status =3D req.s.status; > + > + if (dev->status & VIRTIO_DEVICE_STATUS_DRIVER_OK) > + vduse_device_start(dev); > + > resp.result =3D VDUSE_REQ_RESULT_OK; > break; > case VDUSE_UPDATE_IOTLB: > -- > 2.39.2 Reviewed-by: Chenbo Xia =20