From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id A1D01A0350; Sun, 28 Jun 2020 05:08:20 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 7AC491C031; Sun, 28 Jun 2020 05:08:20 +0200 (CEST) Received: from mga05.intel.com (mga05.intel.com [192.55.52.43]) by dpdk.org (Postfix) with ESMTP id B15FC1C026 for ; Sun, 28 Jun 2020 05:08:18 +0200 (CEST) IronPort-SDR: 1pzqnwDqvV06F2pkcuDj4l5iUgHIamvleN6KmkyIixSWPQygvNAgK6i9kpYgZspqPyh8S1uHq0 zW+I3QSHUQ6w== X-IronPort-AV: E=McAfee;i="6000,8403,9665"; a="230549232" X-IronPort-AV: E=Sophos;i="5.75,290,1589266800"; d="scan'208";a="230549232" X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga004.fm.intel.com ([10.253.24.48]) by fmsmga105.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 27 Jun 2020 20:08:17 -0700 IronPort-SDR: RvM3DgArf0/C/TFznd94cebC7N/S9jHaOlABBkybQZiLD/i1YR/2vhv4KClBxtEuczOcgtSrk1 mDZbZSSYhpqw== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.75,290,1589266800"; d="scan'208";a="302714972" Received: from orsmsx102.amr.corp.intel.com ([10.22.225.129]) by fmsmga004.fm.intel.com with ESMTP; 27 Jun 2020 20:08:17 -0700 Received: from orsmsx152.amr.corp.intel.com (10.22.226.39) by ORSMSX102.amr.corp.intel.com (10.22.225.129) with Microsoft SMTP Server (TLS) id 14.3.439.0; Sat, 27 Jun 2020 20:08:17 -0700 Received: from ORSEDG001.ED.cps.intel.com (10.7.248.4) by ORSMSX152.amr.corp.intel.com (10.22.226.39) with Microsoft SMTP Server (TLS) id 14.3.439.0; Sat, 27 Jun 2020 20:08:17 -0700 Received: from NAM02-BL2-obe.outbound.protection.outlook.com (104.47.38.55) by edgegateway.intel.com (134.134.137.100) with Microsoft SMTP Server (TLS) id 14.3.439.0; Sat, 27 Jun 2020 20:08:16 -0700 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=jpKpDa7FNeVompDlylVbxV3Cfcb63JcrFrZDmr0x8MVOSkc/GNDf/u3uvxZzBcXWiDaN+gdKzRzEMIIkaU4AUxE0Oz/dItorFBPjTWYVElBCElYicRJfVA6qSKYNY91hrR00VBp2uBWRekbKDsJRCULbEFd23IKHzMGKLaKTiBJRyU8WHbkHld1ELjz52YbgiriHaX9biTX+KJpdzUZ5TPFRSiBKCZ0wnJu2XbjGTPDgo6bfPH25/pBjlu6sjha0CmVGkXg7uW39aoJlej4Q0EQUWvYWj6o0M6aIoUpt6Xy9H2+xUV0OOa7RgMBXefsmVjbhqdCfOY/DfRLShOaLcA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=Sxyh8bJ+F63GDwApUUazQcag4xqs1AWXmycDlvWmz14=; b=avXTvaQy8CYI+Pd1FbOxWozySRS5An2qIYiJM96LV5ud0zPtrigjUpTOpEOfBMDzRpPMxguy0bddl0F3hBvuGiBkTgLA2+5C20lBAJdxvC8iPknOS+YPJZNjrUhwPy19mYrHYe3u6GZsDX2f+3g/MMmJODL1Ssvnsv6Wo6SlwIo9k2twIgcb9Igjj974UZsdauqwruHY0uFgRlpDsXeYpn2fZ8jHpPHjtE/bjM5YXYNTnh/43DK+TC3Q4xqkfrDyVBMGzRgYI2jz7jKAlGjcfF7CcO3z8BGVXmasTkw8IR6QcjYffnVUBL9tVMXBNwrNAL9YgwanNHnAKv+Rku+aRw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=intel.com; dmarc=pass action=none header.from=intel.com; dkim=pass header.d=intel.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=intel.onmicrosoft.com; s=selector2-intel-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=Sxyh8bJ+F63GDwApUUazQcag4xqs1AWXmycDlvWmz14=; b=k9SzyvjbOtRI7bo96xQzrBwofsUiWK988+zh8g7phQ8uSEvPY/44oY8lnvJBzXYy07rWLBWmznpVhYiPV8yORpws54TGkswELRW1JTBlTZib4T+TpKmj/qx7TIlAohd4+gWEbUkmzxltY5Lq8j80i8RbOtodbBjP94dfQt/fcyU= Received: from MN2PR11MB4063.namprd11.prod.outlook.com (2603:10b6:208:13f::22) by MN2PR11MB3999.namprd11.prod.outlook.com (2603:10b6:208:154::32) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3131.23; Sun, 28 Jun 2020 03:08:15 +0000 Received: from MN2PR11MB4063.namprd11.prod.outlook.com ([fe80::7cde:8326:5010:c47e]) by MN2PR11MB4063.namprd11.prod.outlook.com ([fe80::7cde:8326:5010:c47e%7]) with mapi id 15.20.3131.026; Sun, 28 Jun 2020 03:08:15 +0000 From: "Xia, Chenbo" To: Matan Azrad , Maxime Coquelin CC: "dev@dpdk.org" , "Wang, Xiao W" Thread-Topic: [dpdk-dev] [PATCH v2 2/5] vhost: improve device readiness notifications Thread-Index: AQHWSvYA9n34wdObpE+eDRJGP5Kj+ajtXKUw Date: Sun, 28 Jun 2020 03:08:15 +0000 Message-ID: References: <1592497686-433697-1-git-send-email-matan@mellanox.com> <1593092298-52257-1-git-send-email-matan@mellanox.com> <1593092298-52257-3-git-send-email-matan@mellanox.com> In-Reply-To: <1593092298-52257-3-git-send-email-matan@mellanox.com> Accept-Language: zh-CN, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: mellanox.com; dkim=none (message not signed) header.d=none;mellanox.com; dmarc=none action=none header.from=intel.com; x-originating-ip: [192.198.147.218] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: 7d681abf-2e80-4e43-a5ff-08d81b108010 x-ms-traffictypediagnostic: MN2PR11MB3999: x-ms-exchange-transport-forked: True x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:10000; x-forefront-prvs: 0448A97BF2 x-ms-exchange-senderadcheck: 1 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: hPrUDpj9tKBrr2ve9km8Tnxj0fMYnvf4xck7dkqMWYmKk8lzTE+npPCuZGsJVIzsBg19n8aCV94QUkvshveQ0rUomuSrwKkruWV/r9g1JtFam+psIxB6QyTfvuUR2G1enrq0vr2TMvIIztuCBlL9OvJ68kegmzHO1YwC1deEGwHO5NhjeRj+NquQrHBvF7Xp9tFG9giPGme6cz0mZ4/wuFjB8fgGesa57x5Ymqt+Qt/xefSo6szaQOP5AygIpLpkUa4fUYyjJiwOMQbDNufUwTThBMRa3fdihyZiHE3CUC9h3UVTErs7+Oz4SECz3R6GPz2RS6KlcyxHxnzbn0du1A== x-forefront-antispam-report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:MN2PR11MB4063.namprd11.prod.outlook.com; PTR:; CAT:NONE; SFTY:; SFS:(4636009)(136003)(396003)(366004)(39860400002)(346002)(376002)(15650500001)(186003)(26005)(76116006)(110136005)(8936002)(64756008)(52536014)(86362001)(107886003)(66446008)(53546011)(66556008)(66476007)(5660300002)(33656002)(6506007)(54906003)(2906002)(66946007)(478600001)(7696005)(83380400001)(9686003)(4326008)(55016002)(71200400001)(316002); DIR:OUT; SFP:1102; x-ms-exchange-antispam-messagedata: M4QweGUbfHdYaE6Gsdr5P7kwnAC6NoYDi+arsOz8HXRp4EcxOjBuhRHuCTXXYRaX+kOcV0zqmmFzde+UOmu6jBaReowKNpHaUNgL7IIUVVYG13ox1vZlL3lx+ZcZDm5b2ie3pCBfepcF7omug21f5SfHEesZQqPB4nktgwdkQlbdL1O6UOIAXDEMUkRyDvPCGB2+Y1WwW0sMW7sVzMkPzirNyDWoCKk9tykbq7QeyLyGpIIErsNsTKbUjXaZBc7K79qUopcRzoJ+nTwVma9OqKbWscihjeRB8518prfGgA2PYWJHWEigHdGxC8hOmHJH+vLo4ZyntcAU4ZSHuq+p8F1fUMcvzRH5o0+LaszGFge+owSqFQlniNSfETKO8RYXBsgTOzP4Q/hTdMI0LJef88uUzesD8XLbPO5PoRDtGi4V8IDuppAAsc8oDYQNUbuvMK16UE9NzxygD5yL4GrjD8mZaB2jUgEMcT8xtX7nCNrxVoJoIBiWmwZ44hTF51h1 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: MN2PR11MB4063.namprd11.prod.outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: 7d681abf-2e80-4e43-a5ff-08d81b108010 X-MS-Exchange-CrossTenant-originalarrivaltime: 28 Jun 2020 03:08:15.3777 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 46c98d88-e344-4ed4-8496-4ed7712e255d X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: 3M9CtvjLetmZhYstqprqZ0cdNiY/Rr13i+GTNZUVZclTkVvooOCaSp0Poel8yg8tFZiVKMVn8pYggbMYNG3K8Q== X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN2PR11MB3999 X-OriginatorOrg: intel.com Subject: Re: [dpdk-dev] [PATCH v2 2/5] vhost: improve device readiness notifications X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" > -----Original Message----- > From: dev On Behalf Of Matan Azrad > Sent: Thursday, June 25, 2020 9:38 PM > To: Maxime Coquelin > Cc: dev@dpdk.org; Wang, Xiao W > Subject: [dpdk-dev] [PATCH v2 2/5] vhost: improve device readiness > notifications >=20 > Some guest drivers may not configure disabled virtio queues. >=20 > In this case, the vhost management never notifies the application and the= vDPA > device readiness because it waits to the device to be ready. >=20 > The current ready state means that all the virtio queues should be config= ured > regardless the enablement status. >=20 > In order to support this case, this patch changes the ready state: > The device is ready when at least 1 queue pair is configured and enabled. >=20 > So, now, the application and vDPA driver are notifies when the first queu= e pair is > configured and enabled. >=20 > Also the queue notifications will be triggered according to the new ready > definition. >=20 > Signed-off-by: Matan Azrad > --- > lib/librte_vhost/vhost.h | 1 + > lib/librte_vhost/vhost_user.c | 55 +++++++++++++++++++++++++++++--------= ---- > -- > 2 files changed, 38 insertions(+), 18 deletions(-) >=20 > diff --git a/lib/librte_vhost/vhost.h b/lib/librte_vhost/vhost.h index > 17f1e9a..8a74f33 100644 > --- a/lib/librte_vhost/vhost.h > +++ b/lib/librte_vhost/vhost.h > @@ -151,6 +151,7 @@ struct vhost_virtqueue { > int backend; > int enabled; > int access_ok; > + int ready; > rte_spinlock_t access_lock; >=20 > /* Used to notify the guest (trigger interrupt) */ diff --git > a/lib/librte_vhost/vhost_user.c b/lib/librte_vhost/vhost_user.c index > 8d8050b..b90fc78 100644 > --- a/lib/librte_vhost/vhost_user.c > +++ b/lib/librte_vhost/vhost_user.c > @@ -228,6 +228,21 @@ > dev->postcopy_listening =3D 0; > } >=20 > +static void > +vhost_user_notify_queue_state(struct virtio_net *dev, uint16_t index, > + int enable) > +{ > + int did =3D dev->vdpa_dev_id; > + struct rte_vdpa_device *vdpa_dev =3D rte_vdpa_get_device(did); > + > + if (vdpa_dev && vdpa_dev->ops->set_vring_state) > + vdpa_dev->ops->set_vring_state(dev->vid, index, enable); > + > + if (dev->notify_ops->vring_state_changed) > + dev->notify_ops->vring_state_changed(dev->vid, > + index, enable); > +} > + > /* > * This function just returns success at the moment unless > * the device hasn't been initialised. > @@ -1306,27 +1321,31 @@ >=20 > return rings_ok && > vq->kickfd !=3D VIRTIO_UNINITIALIZED_EVENTFD && > - vq->callfd !=3D VIRTIO_UNINITIALIZED_EVENTFD; > + vq->callfd !=3D VIRTIO_UNINITIALIZED_EVENTFD && > + vq->enabled; > } >=20 > +#define VIRTIO_DEV_NUM_VQS_TO_BE_READY 2u > + > static int > virtio_is_ready(struct virtio_net *dev) { > struct vhost_virtqueue *vq; > uint32_t i; >=20 > - if (dev->nr_vring =3D=3D 0) > + if (dev->nr_vring < VIRTIO_DEV_NUM_VQS_TO_BE_READY) > return 0; >=20 > - for (i =3D 0; i < dev->nr_vring; i++) { > + for (i =3D 0; i < VIRTIO_DEV_NUM_VQS_TO_BE_READY; i++) { > vq =3D dev->virtqueue[i]; >=20 > if (!vq_is_ready(dev, vq)) > return 0; > } >=20 > - VHOST_LOG_CONFIG(INFO, > - "virtio is now ready for processing.\n"); > + if (!(dev->flags & VIRTIO_DEV_RUNNING)) > + VHOST_LOG_CONFIG(INFO, > + "virtio is now ready for processing.\n"); > return 1; > } >=20 > @@ -1970,8 +1989,6 @@ static int vhost_user_set_vring_err(struct virtio_n= et > **pdev __rte_unused, > struct virtio_net *dev =3D *pdev; > int enable =3D (int)msg->payload.state.num; > int index =3D (int)msg->payload.state.index; > - struct rte_vdpa_device *vdpa_dev; > - int did =3D -1; >=20 > if (validate_msg_fds(msg, 0) !=3D 0) > return RTE_VHOST_MSG_RESULT_ERR; > @@ -1980,15 +1997,6 @@ static int vhost_user_set_vring_err(struct virtio_= net > **pdev __rte_unused, > "set queue enable: %d to qp idx: %d\n", > enable, index); >=20 > - did =3D dev->vdpa_dev_id; > - vdpa_dev =3D rte_vdpa_get_device(did); > - if (vdpa_dev && vdpa_dev->ops->set_vring_state) > - vdpa_dev->ops->set_vring_state(dev->vid, index, enable); > - > - if (dev->notify_ops->vring_state_changed) > - dev->notify_ops->vring_state_changed(dev->vid, > - index, enable); > - > /* On disable, rings have to be stopped being processed. */ > if (!enable && dev->dequeue_zero_copy) > drain_zmbuf_list(dev->virtqueue[index]); > @@ -2618,6 +2626,7 @@ typedef int (*vhost_message_handler_t)(struct > virtio_net **pdev, > int unlock_required =3D 0; > bool handled; > int request; > + uint32_t i; >=20 > dev =3D get_device(vid); > if (dev =3D=3D NULL) > @@ -2793,6 +2802,17 @@ typedef int (*vhost_message_handler_t)(struct > virtio_net **pdev, > return -1; > } >=20 > + for (i =3D 0; i < dev->nr_vring; i++) { > + struct vhost_virtqueue *vq =3D dev->virtqueue[i]; > + bool cur_ready =3D vq_is_ready(dev, vq); > + > + if (cur_ready !=3D (vq && vq->ready)) { > + vhost_user_notify_queue_state(dev, i, cur_ready); > + vq->ready =3D cur_ready; > + } > + } > + > + > if (!(dev->flags & VIRTIO_DEV_RUNNING) && virtio_is_ready(dev)) { > dev->flags |=3D VIRTIO_DEV_READY; >=20 > @@ -2810,8 +2830,7 @@ typedef int (*vhost_message_handler_t)(struct > virtio_net **pdev, > did =3D dev->vdpa_dev_id; > vdpa_dev =3D rte_vdpa_get_device(did); > if (vdpa_dev && virtio_is_ready(dev) && > - !(dev->flags & VIRTIO_DEV_VDPA_CONFIGURED) && > - msg.request.master =3D=3D VHOST_USER_SET_VRING_CALL) > { > + !(dev->flags & VIRTIO_DEV_VDPA_CONFIGURED)) { > if (vdpa_dev->ops->dev_conf) > vdpa_dev->ops->dev_conf(dev->vid); > dev->flags |=3D VIRTIO_DEV_VDPA_CONFIGURED; > -- > 1.8.3.1 Reviewed-by: Chenbo Xia