From: "Pei, Andy" <andy.pei@intel.com>
To: "Xia, Chenbo" <chenbo.xia@intel.com>, "dev@dpdk.org" <dev@dpdk.org>
Cc: "Xu, Rosen" <rosen.xu@intel.com>,
"Huang, Wei" <wei.huang@intel.com>,
"Cao, Gang" <gang.cao@intel.com>,
"maxime.coquelin@redhat.com" <maxime.coquelin@redhat.com>
Subject: RE: [PATCH v3 8/8] vhost: improve vDPA blk device readiness condition
Date: Thu, 13 Oct 2022 07:55:38 +0000 [thread overview]
Message-ID: <DM5PR11MB17398553ED144611DA683D908F259@DM5PR11MB1739.namprd11.prod.outlook.com> (raw)
In-Reply-To: <SN6PR11MB35045D0FD28EDDCA3368F2D89C229@SN6PR11MB3504.namprd11.prod.outlook.com>
Hi Chenbo,
Thanks for your reply, my reply is inline.
> -----Original Message-----
> From: Xia, Chenbo <chenbo.xia@intel.com>
> Sent: Wednesday, October 12, 2022 5:36 PM
> To: Pei, Andy <andy.pei@intel.com>; dev@dpdk.org
> Cc: Xu, Rosen <rosen.xu@intel.com>; Huang, Wei <wei.huang@intel.com>;
> Cao, Gang <gang.cao@intel.com>; maxime.coquelin@redhat.com
> Subject: RE: [PATCH v3 8/8] vhost: improve vDPA blk device readiness
> condition
>
> > -----Original Message-----
> > From: Pei, Andy <andy.pei@intel.com>
> > Sent: Friday, September 16, 2022 2:17 PM
> > To: dev@dpdk.org
> > Cc: Xia, Chenbo <chenbo.xia@intel.com>; Xu, Rosen
> > <rosen.xu@intel.com>; Huang, Wei <wei.huang@intel.com>; Cao, Gang
> > <gang.cao@intel.com>; maxime.coquelin@redhat.com
> > Subject: [PATCH v3 8/8] vhost: improve vDPA blk device readiness
> > condition
> >
> > In the virtio blk vDPA live migration use case, for the target VM,
> > before the live migration process, QEMU will set call fd to all queues
> > of vDPA back-end. QEMU and vDPA back-end stand by until live migration
> > starts. During live migration process, QEMU sets kick fd and new call
> > fd. However, after the kick fd is set to the vDPA back-end, the vDPA
> > back-end configures device and data path starts. The new call fd will
> > cause some kind of "re-configuration", this kind of "re-configuration"
> > cause IO drop.
> > After this patch, vDPA back-end configures device after kick fd and
> > call fd are well set and make sure no IO drops.
> > This patch only impact virtio blk vDPA device and does not impact net
> > device.
>
> IIUC, this is an improvement for MQ to make sure all call fds are well-set, but
> previously it only makes sure one is well-set. If I am right, the title and
> commit message should describe it in better way.
>
> Thanks,
> Chenbo
>
In the virtio blk vDPA live migration use case, for the target VM,
before the live migration process, QEMU will set call fd to all queues of vDPA back-end.
QEMU and vDPA back-end stand by until live migration starts.
During live migration process, QEMU sets kick fd to all queues one by one,
and new call fd all queues one by one.
However, after the kick fd is set to the vDPA back-end,
with the original call fd, the queue is ready.
Then the vDPA back-end configures device and data path starts.
The new call fd will cause some kind of "re-configuration",
this kind of "re-configuration" cause IO drop.
After this patch, vDPA back-end configures device after kick fd and
call fd are well set and make sure no IO drops.
This patch only impact virtio blk vDPA device and does not impact net device.
Is this new commit message good?
> >
> > Signed-off-by: Andy Pei <andy.pei@intel.com>
> > Signed-off-by: Huang Wei <wei.huang@intel.com>
> > ---
> > lib/vhost/vhost_user.c | 13 ++++++++++---
> > 1 file changed, 10 insertions(+), 3 deletions(-)
> >
> > diff --git a/lib/vhost/vhost_user.c b/lib/vhost/vhost_user.c index
> > 9169cf5..14ff266 100644
> > --- a/lib/vhost/vhost_user.c
> > +++ b/lib/vhost/vhost_user.c
> > @@ -2983,6 +2983,7 @@ static int is_vring_iotlb(struct virtio_net *dev,
> > uint32_t vdpa_type = 0;
> > uint32_t request;
> > uint32_t i;
> > + uint16_t blk_call_fd;
> >
> > dev = get_device(vid);
> > if (dev == NULL)
> > @@ -3210,9 +3211,15 @@ static int is_vring_iotlb(struct virtio_net *dev,
> > if (!vdpa_dev)
> > goto out;
> >
> > - if (vdpa_type == RTE_VHOST_VDPA_DEVICE_TYPE_BLK
> > - && request != VHOST_USER_SET_VRING_CALL)
> > - goto out;
> > + if (vdpa_type == RTE_VHOST_VDPA_DEVICE_TYPE_BLK) {
> > + if (request == VHOST_USER_SET_VRING_CALL) {
> > + blk_call_fd = ctx.msg.payload.u64 &
> > VHOST_USER_VRING_IDX_MASK;
> > + if (blk_call_fd != dev->nr_vring - 1)
> > + goto out;
> > + } else {
> > + goto out;
> > + }
> > + }
> >
> > if (!(dev->flags & VIRTIO_DEV_VDPA_CONFIGURED)) {
> > if (vdpa_dev->ops->dev_conf(dev->vid))
> > --
> > 1.8.3.1
next prev parent reply other threads:[~2022-10-13 7:55 UTC|newest]
Thread overview: 181+ messages / expand[flat|nested] mbox.gz Atom feed top
2022-08-23 4:34 [PATCH 0/8] add multi queue support to vDPA ifc driver Andy Pei
2022-08-23 4:34 ` [PATCH 1/8] vdpa/ifc: add new device ID Andy Pei
2022-09-08 5:54 ` [PATCH v2 0/8] vdpa/ifc: add multi queue support Andy Pei
2022-09-08 5:54 ` [PATCH v2 1/8] vdpa/ifc: add new device ID Andy Pei
2022-09-09 0:29 ` Xia, Chenbo
2022-09-09 7:44 ` Pei, Andy
2022-09-08 5:54 ` [PATCH v2 2/8] vdpa/ifc: add multi queue support Andy Pei
2022-09-09 0:29 ` Xia, Chenbo
2022-09-09 7:55 ` Pei, Andy
2022-09-08 5:54 ` [PATCH v2 3/8] vdpa/ifc: set max queues according to HW spec Andy Pei
2022-09-09 5:56 ` Xia, Chenbo
2022-09-14 3:59 ` Pei, Andy
2022-09-08 5:54 ` [PATCH v2 4/8] vdpa/ifc: write queue count to MQ register Andy Pei
2022-09-09 7:35 ` Xia, Chenbo
2022-09-09 8:31 ` Pei, Andy
2022-09-08 5:54 ` [PATCH v2 5/8] vdpa/ifc: only configure enabled queue Andy Pei
2022-09-14 1:59 ` Xia, Chenbo
2022-09-14 2:57 ` Huang, Wei
2022-09-08 5:54 ` [PATCH v2 6/8] vdpa/ifc: support dynamic enable/disable queue Andy Pei
2022-09-14 2:23 ` Xia, Chenbo
2022-09-14 3:04 ` Huang, Wei
2022-09-14 3:14 ` Xia, Chenbo
2022-09-14 5:29 ` Pei, Andy
2022-09-08 5:54 ` [PATCH v2 7/8] vhost: configure device when any queue is ready for BLK device Andy Pei
2022-09-14 2:50 ` Xia, Chenbo
2022-09-14 7:01 ` Pei, Andy
2022-09-08 5:54 ` [PATCH v2 8/8] vhost: vDPA BLK devices configure device when all queue callfds are set Andy Pei
2022-09-16 6:16 ` [PATCH v3 0/8] vdpa/ifc: add multi queue support Andy Pei
2022-09-16 6:16 ` [PATCH v3 1/8] vdpa/ifc: add new device ID for legacy network device Andy Pei
2022-10-11 16:49 ` Maxime Coquelin
2022-10-12 6:33 ` Pei, Andy
2022-10-12 2:59 ` Xia, Chenbo
2022-10-12 6:34 ` Pei, Andy
2022-09-16 6:16 ` [PATCH v3 2/8] vdpa/ifc: add multi-queue support Andy Pei
2022-10-12 3:01 ` Xia, Chenbo
2022-10-12 7:25 ` Pei, Andy
2022-09-16 6:16 ` [PATCH v3 3/8] vdpa/ifc: set max queues based on virtio spec Andy Pei
2022-10-12 6:08 ` Xia, Chenbo
2022-10-12 7:22 ` Pei, Andy
2022-10-12 7:22 ` Pei, Andy
2022-09-16 6:16 ` [PATCH v3 4/8] vdpa/ifc: write queue count to MQ register Andy Pei
2022-10-12 8:08 ` Xia, Chenbo
2022-10-12 8:14 ` Pei, Andy
2022-09-16 6:16 ` [PATCH v3 5/8] vdpa/ifc: only configure enabled queue Andy Pei
2022-10-12 8:12 ` Xia, Chenbo
2022-10-12 8:37 ` Pei, Andy
2022-09-16 6:16 ` [PATCH v3 6/8] vdpa/ifc: support dynamic enable/disable queue Andy Pei
2022-10-12 8:19 ` Xia, Chenbo
2022-10-12 11:00 ` Pei, Andy
2022-09-16 6:16 ` [PATCH v3 7/8] vhost: vDPA blk device gets ready when any queue is ready Andy Pei
2022-10-12 9:09 ` Xia, Chenbo
2022-10-12 12:13 ` Pei, Andy
2022-10-13 1:00 ` Xia, Chenbo
2022-10-13 3:05 ` Pei, Andy
2022-10-13 7:16 ` Maxime Coquelin
2022-10-13 9:00 ` Pei, Andy
2022-09-16 6:16 ` [PATCH v3 8/8] vhost: improve vDPA blk device readiness condition Andy Pei
2022-10-12 9:35 ` Xia, Chenbo
2022-10-13 7:55 ` Pei, Andy [this message]
2022-10-13 8:23 ` Pei, Andy
2022-10-13 8:44 ` [PATCH v4 0/8] vdpa/ifc: add multi queue support Andy Pei
2022-10-13 8:44 ` [PATCH v4 1/8] vdpa/ifc: add new device ID for legacy network device Andy Pei
2022-10-13 8:44 ` [PATCH v4 2/8] vdpa/ifc: add multi-queue support Andy Pei
2022-10-17 6:21 ` Xia, Chenbo
2022-10-17 6:28 ` Pei, Andy
2022-10-13 8:44 ` [PATCH v4 3/8] vdpa/ifc: set max queues based on virtio spec Andy Pei
2022-10-17 6:22 ` Xia, Chenbo
2022-10-17 6:29 ` Pei, Andy
2022-10-13 8:44 ` [PATCH v4 4/8] vdpa/ifc: write queue count to MQ register Andy Pei
2022-10-17 6:23 ` Xia, Chenbo
2022-10-17 6:36 ` Pei, Andy
2022-10-13 8:44 ` [PATCH v4 5/8] vdpa/ifc: only configure enabled queue Andy Pei
2022-10-17 6:24 ` Xia, Chenbo
2022-10-17 6:38 ` Pei, Andy
2022-10-13 8:44 ` [PATCH v4 6/8] vdpa/ifc: support dynamic enable/disable queue Andy Pei
2022-10-17 6:26 ` Xia, Chenbo
2022-10-17 6:41 ` Pei, Andy
2022-10-13 8:44 ` [PATCH v4 7/8] vhost: vDPA blk device gets ready when any queue is ready Andy Pei
2022-10-17 6:34 ` Xia, Chenbo
2022-10-17 6:43 ` Pei, Andy
2022-10-13 8:44 ` [PATCH v4 8/8] vhost: improve vDPA blk device configure condition Andy Pei
2022-10-17 6:35 ` Xia, Chenbo
2022-10-17 7:12 ` Pei, Andy
2022-10-17 7:13 ` [PATCH v5 0/8] vdpa/ifc: add multi queue support Andy Pei
2022-10-17 7:13 ` [PATCH v5 1/8] vdpa/ifc: add new device ID for legacy network device Andy Pei
2022-10-17 7:13 ` [PATCH v5 2/8] vdpa/ifc: add multi-queue support Andy Pei
2022-10-17 7:13 ` [PATCH v5 3/8] vdpa/ifc: set max queues based on virtio spec Andy Pei
2022-10-17 7:13 ` [PATCH v5 4/8] vdpa/ifc: write queue count to MQ register Andy Pei
2022-10-17 7:13 ` [PATCH v5 5/8] vdpa/ifc: only configure enabled queue Andy Pei
2022-10-17 7:13 ` [PATCH v5 6/8] vdpa/ifc: support dynamic enable/disable queue Andy Pei
2022-10-17 7:13 ` [PATCH v5 7/8] vhost: vDPA blk device gets ready when the first queue is ready Andy Pei
2022-10-17 7:54 ` Xia, Chenbo
2022-10-17 8:36 ` Maxime Coquelin
2022-10-17 8:42 ` Xia, Chenbo
2022-10-17 8:46 ` Maxime Coquelin
2022-10-17 7:13 ` [PATCH v5 8/8] vhost: improve vDPA blk device configure condition Andy Pei
2022-10-17 11:41 ` [PATCH v6 0/8] vdpa/ifc: add multi queue support Andy Pei
2022-10-17 11:41 ` [PATCH v6 1/8] vdpa/ifc: add new device ID for legacy network device Andy Pei
2022-10-17 11:41 ` [PATCH v6 2/8] vdpa/ifc: add multi-queue support Andy Pei
2022-10-17 11:41 ` [PATCH v6 3/8] vdpa/ifc: set max queues based on virtio spec Andy Pei
2022-10-17 11:41 ` [PATCH v6 4/8] vdpa/ifc: write queue count to MQ register Andy Pei
2022-10-17 11:41 ` [PATCH v6 5/8] vdpa/ifc: only configure enabled queue Andy Pei
2022-10-17 11:41 ` [PATCH v6 6/8] vdpa/ifc: support dynamic enable/disable queue Andy Pei
2022-10-17 11:41 ` [PATCH v6 7/8] vhost: vDPA blk device gets ready when the first queue is ready Andy Pei
2022-10-17 11:41 ` [PATCH v6 8/8] vhost: improve vDPA blk device configure condition Andy Pei
2022-10-18 6:19 ` [PATCH v7 00/12] vdpa/ifc: add multi queue support Andy Pei
2022-10-18 6:19 ` [PATCH v7 01/12] vdpa/ifc: add new device ID for legacy network device Andy Pei
2022-10-18 6:19 ` [PATCH v7 02/12] vdpa/ifc: add multi-queue support Andy Pei
2022-10-18 6:19 ` [PATCH v7 03/12] vdpa/ifc: set max queues based on virtio spec Andy Pei
2022-10-18 6:19 ` [PATCH v7 04/12] vdpa/ifc: write queue count to MQ register Andy Pei
2022-10-18 6:19 ` [PATCH v7 05/12] vdpa/ifc: only configure enabled queue Andy Pei
2022-10-18 6:19 ` [PATCH v7 06/12] vdpa/ifc: support dynamic enable/disable queue Andy Pei
2022-10-18 6:19 ` [PATCH v7 07/12] vdpa/ifc: change internal function name Andy Pei
2022-10-18 6:19 ` [PATCH v7 08/12] vdpa/ifc: add internal API to get device Andy Pei
2022-10-18 6:19 ` [PATCH v7 09/12] vdpa/ifc: change some driver logic Andy Pei
2022-10-18 6:19 ` [PATCH v7 10/12] vhost: add vdpa device type to rte vdpa device Andy Pei
2022-10-18 7:14 ` Maxime Coquelin
2022-10-18 12:14 ` Pei, Andy
2022-10-19 9:14 ` Xia, Chenbo
2022-10-19 9:19 ` Pei, Andy
2022-10-18 6:19 ` [PATCH v7 11/12] vhost: vDPA blk device gets ready when the first queue is ready Andy Pei
2022-10-18 6:19 ` [PATCH v7 12/12] vhost: improve vDPA blk device configure condition Andy Pei
2022-10-18 12:07 ` [PATCH v8 00/12] vdpa/ifc: add multi queue support Andy Pei
2022-10-18 12:07 ` [PATCH v8 01/12] vdpa/ifc: add new device ID for legacy network device Andy Pei
2022-10-18 12:07 ` [PATCH v8 02/12] vdpa/ifc: add multi-queue support Andy Pei
2022-10-18 12:07 ` [PATCH v8 03/12] vdpa/ifc: set max queues based on virtio spec Andy Pei
2022-10-18 12:07 ` [PATCH v8 04/12] vdpa/ifc: write queue count to MQ register Andy Pei
2022-10-18 12:07 ` [PATCH v8 05/12] vdpa/ifc: only configure enabled queue Andy Pei
2022-10-18 12:07 ` [PATCH v8 06/12] vdpa/ifc: support dynamic enable/disable queue Andy Pei
2022-10-18 12:07 ` [PATCH v8 07/12] vdpa/ifc: change internal function name Andy Pei
2022-10-18 13:44 ` Maxime Coquelin
2022-10-18 13:47 ` Pei, Andy
2022-10-19 6:59 ` Xia, Chenbo
2022-10-19 9:41 ` Pei, Andy
2022-10-18 12:07 ` [PATCH v8 08/12] vdpa/ifc: add internal API to get device Andy Pei
2022-10-18 13:48 ` Maxime Coquelin
2022-10-18 13:52 ` Pei, Andy
2022-10-19 7:03 ` Xia, Chenbo
2022-10-19 9:40 ` Pei, Andy
2022-10-18 12:07 ` [PATCH v8 09/12] vdpa/ifc: change some driver logic Andy Pei
2022-10-18 13:57 ` Maxime Coquelin
2022-10-18 14:01 ` Pei, Andy
2022-10-19 9:13 ` Xia, Chenbo
2022-10-19 9:21 ` Pei, Andy
2022-10-18 12:07 ` [PATCH v8 10/12] vhost: add type to rte vdpa device Andy Pei
2022-10-18 13:15 ` Maxime Coquelin
2022-10-18 13:42 ` Pei, Andy
2022-10-18 12:07 ` [PATCH v8 11/12] vhost: vDPA blk device gets ready when the first queue is ready Andy Pei
2022-10-18 14:09 ` Maxime Coquelin
2022-10-18 14:11 ` Pei, Andy
2022-10-19 9:14 ` Xia, Chenbo
2022-10-19 9:18 ` Pei, Andy
2022-10-18 12:07 ` [PATCH v8 12/12] vhost: improve vDPA blk device configure condition Andy Pei
2022-10-18 14:14 ` Maxime Coquelin
2022-10-18 14:16 ` Pei, Andy
2022-10-19 9:15 ` Xia, Chenbo
2022-10-19 9:19 ` Pei, Andy
2022-10-19 8:41 ` [PATCH v9 00/12] vdpa/ifc: add multi queue support Andy Pei
2022-10-19 8:41 ` [PATCH v9 01/12] vdpa/ifc: add new device ID for legacy network device Andy Pei
2022-10-19 8:41 ` [PATCH v9 02/12] vdpa/ifc: add multi-queue support Andy Pei
2022-10-19 8:41 ` [PATCH v9 03/12] vdpa/ifc: set max queues based on virtio spec Andy Pei
2022-10-19 8:41 ` [PATCH v9 04/12] vdpa/ifc: write queue count to MQ register Andy Pei
2022-10-19 8:41 ` [PATCH v9 05/12] vdpa/ifc: only configure enabled queue Andy Pei
2022-10-19 8:41 ` [PATCH v9 06/12] vdpa/ifc: support dynamic enable/disable queue Andy Pei
2022-10-19 8:41 ` [PATCH v9 07/12] vdpa/ifc: change internal function name Andy Pei
2022-10-19 8:41 ` [PATCH v9 08/12] vdpa/ifc: add internal API to get device Andy Pei
2022-10-19 8:41 ` [PATCH v9 09/12] vdpa/ifc: improve internal list logic Andy Pei
2022-10-20 3:21 ` Xia, Chenbo
2022-10-20 5:53 ` Pei, Andy
2022-10-19 8:41 ` [PATCH v9 10/12] vhost: add type to rte vdpa device Andy Pei
2022-10-19 8:41 ` [PATCH v9 11/12] vhost: vDPA blk device gets ready when the first queue is ready Andy Pei
2022-10-19 8:41 ` [PATCH v9 12/12] vhost: improve vDPA blk device configure condition Andy Pei
2022-10-26 9:00 ` [PATCH v9 00/12] vdpa/ifc: add multi queue support Xia, Chenbo
2022-10-26 9:26 ` Pei, Andy
2022-08-23 4:34 ` [PATCH 2/8] vdpa/ifc: add multi queue suppoort Andy Pei
2022-08-23 4:35 ` [PATCH 3/8] vdpa/ifc: set max queues according to HW spec Andy Pei
2022-08-23 4:35 ` [PATCH 4/8] vdpa/ifc: write queue count to MQ register Andy Pei
2022-08-23 4:35 ` [PATCH 5/8] vdpa/ifc: only configure enabled queue Andy Pei
2022-08-23 4:35 ` [PATCH 6/8] vdpa/ifc: set vring state callback update data path Andy Pei
2022-08-23 4:35 ` [PATCH 7/8] vhost: configure device when any queue is ready for BLK device Andy Pei
2022-08-23 4:35 ` [PATCH 8/8] vhost: vDPA BLK devices configure device when all queue callfds are set Andy Pei
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=DM5PR11MB17398553ED144611DA683D908F259@DM5PR11MB1739.namprd11.prod.outlook.com \
--to=andy.pei@intel.com \
--cc=chenbo.xia@intel.com \
--cc=dev@dpdk.org \
--cc=gang.cao@intel.com \
--cc=maxime.coquelin@redhat.com \
--cc=rosen.xu@intel.com \
--cc=wei.huang@intel.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).