From: "Pei, Andy" <andy.pei@intel.com>
To: "Pei, Andy" <andy.pei@intel.com>,
"Xia, Chenbo" <chenbo.xia@intel.com>,
"dev@dpdk.org" <dev@dpdk.org>
Cc: "Xu, Rosen" <rosen.xu@intel.com>,
"Huang, Wei" <wei.huang@intel.com>,
"Cao, Gang" <gang.cao@intel.com>,
"maxime.coquelin@redhat.com" <maxime.coquelin@redhat.com>
Subject: RE: [PATCH v3 8/8] vhost: improve vDPA blk device readiness condition
Date: Thu, 13 Oct 2022 08:23:33 +0000 [thread overview]
Message-ID: <DM5PR11MB1739FB700FC439BD790E85208F259@DM5PR11MB1739.namprd11.prod.outlook.com> (raw)
In-Reply-To: <DM5PR11MB17398553ED144611DA683D908F259@DM5PR11MB1739.namprd11.prod.outlook.com>
> -----Original Message-----
> From: Pei, Andy <andy.pei@intel.com>
> Sent: Thursday, October 13, 2022 3:56 PM
> To: Xia, Chenbo <Chenbo.Xia@intel.com>; dev@dpdk.org
> Cc: Xu, Rosen <rosen.xu@intel.com>; Huang, Wei <wei.huang@intel.com>;
> Cao, Gang <gang.cao@intel.com>; maxime.coquelin@redhat.com
> Subject: RE: [PATCH v3 8/8] vhost: improve vDPA blk device readiness
> condition
>
> Hi Chenbo,
>
> Thanks for your reply, my reply is inline.
>
> > -----Original Message-----
> > From: Xia, Chenbo <chenbo.xia@intel.com>
> > Sent: Wednesday, October 12, 2022 5:36 PM
> > To: Pei, Andy <andy.pei@intel.com>; dev@dpdk.org
> > Cc: Xu, Rosen <rosen.xu@intel.com>; Huang, Wei <wei.huang@intel.com>;
> > Cao, Gang <gang.cao@intel.com>; maxime.coquelin@redhat.com
> > Subject: RE: [PATCH v3 8/8] vhost: improve vDPA blk device readiness
> > condition
> >
> > > -----Original Message-----
> > > From: Pei, Andy <andy.pei@intel.com>
> > > Sent: Friday, September 16, 2022 2:17 PM
> > > To: dev@dpdk.org
> > > Cc: Xia, Chenbo <chenbo.xia@intel.com>; Xu, Rosen
> > > <rosen.xu@intel.com>; Huang, Wei <wei.huang@intel.com>; Cao, Gang
> > > <gang.cao@intel.com>; maxime.coquelin@redhat.com
> > > Subject: [PATCH v3 8/8] vhost: improve vDPA blk device readiness
> > > condition
> > >
> > > In the virtio blk vDPA live migration use case, for the target VM,
> > > before the live migration process, QEMU will set call fd to all
> > > queues of vDPA back-end. QEMU and vDPA back-end stand by until live
> > > migration starts. During live migration process, QEMU sets kick fd
> > > and new call fd. However, after the kick fd is set to the vDPA
> > > back-end, the vDPA back-end configures device and data path starts.
> > > The new call fd will cause some kind of "re-configuration", this kind of
> "re-configuration"
> > > cause IO drop.
> > > After this patch, vDPA back-end configures device after kick fd and
> > > call fd are well set and make sure no IO drops.
> > > This patch only impact virtio blk vDPA device and does not impact
> > > net device.
> >
> > IIUC, this is an improvement for MQ to make sure all call fds are
> > well-set, but previously it only makes sure one is well-set. If I am
> > right, the title and commit message should describe it in better way.
> >
> > Thanks,
> > Chenbo
> >
> In the virtio blk vDPA live migration use case, for the target VM, before the
> live migration process, QEMU will set call fd to all queues of vDPA back-end.
> QEMU and vDPA back-end stand by until live migration starts.
> During live migration process, QEMU sets kick fd to all queues one by one,
> and new call fd all queues one by one.
> However, after the kick fd is set to the vDPA back-end, with the original call fd,
> the queue is ready.
> Then the vDPA back-end configures device and data path starts.
> The new call fd will cause some kind of "re-configuration", this kind of "re-
> configuration" cause IO drop.
> After this patch, vDPA back-end configures device after kick fd and call fd are
> well set and make sure no IO drops.
> This patch only impact virtio blk vDPA device and does not impact net device.
>
> Is this new commit message good?
Explanation seems duplicate with former commit
" vhost: fix virtio block vDPA live migration IO drop".
So I will make it simple here,
"To support multi-queue, configure device after call fd of all queues are set."
I will use " vhost: improve vDPA blk device configure condition" for title
To reduce confusion.
> > >
> > > Signed-off-by: Andy Pei <andy.pei@intel.com>
> > > Signed-off-by: Huang Wei <wei.huang@intel.com>
> > > ---
> > > lib/vhost/vhost_user.c | 13 ++++++++++---
> > > 1 file changed, 10 insertions(+), 3 deletions(-)
> > >
> > > diff --git a/lib/vhost/vhost_user.c b/lib/vhost/vhost_user.c index
> > > 9169cf5..14ff266 100644
> > > --- a/lib/vhost/vhost_user.c
> > > +++ b/lib/vhost/vhost_user.c
> > > @@ -2983,6 +2983,7 @@ static int is_vring_iotlb(struct virtio_net *dev,
> > > uint32_t vdpa_type = 0;
> > > uint32_t request;
> > > uint32_t i;
> > > + uint16_t blk_call_fd;
> > >
> > > dev = get_device(vid);
> > > if (dev == NULL)
> > > @@ -3210,9 +3211,15 @@ static int is_vring_iotlb(struct virtio_net *dev,
> > > if (!vdpa_dev)
> > > goto out;
> > >
> > > - if (vdpa_type == RTE_VHOST_VDPA_DEVICE_TYPE_BLK
> > > - && request != VHOST_USER_SET_VRING_CALL)
> > > - goto out;
> > > + if (vdpa_type == RTE_VHOST_VDPA_DEVICE_TYPE_BLK) {
> > > + if (request == VHOST_USER_SET_VRING_CALL) {
> > > + blk_call_fd = ctx.msg.payload.u64 &
> > > VHOST_USER_VRING_IDX_MASK;
> > > + if (blk_call_fd != dev->nr_vring - 1)
> > > + goto out;
> > > + } else {
> > > + goto out;
> > > + }
> > > + }
> > >
> > > if (!(dev->flags & VIRTIO_DEV_VDPA_CONFIGURED)) {
> > > if (vdpa_dev->ops->dev_conf(dev->vid))
> > > --
> > > 1.8.3.1
next prev parent reply other threads:[~2022-10-13 8:23 UTC|newest]
Thread overview: 181+ messages / expand[flat|nested] mbox.gz Atom feed top
2022-08-23 4:34 [PATCH 0/8] add multi queue support to vDPA ifc driver Andy Pei
2022-08-23 4:34 ` [PATCH 1/8] vdpa/ifc: add new device ID Andy Pei
2022-09-08 5:54 ` [PATCH v2 0/8] vdpa/ifc: add multi queue support Andy Pei
2022-09-08 5:54 ` [PATCH v2 1/8] vdpa/ifc: add new device ID Andy Pei
2022-09-09 0:29 ` Xia, Chenbo
2022-09-09 7:44 ` Pei, Andy
2022-09-08 5:54 ` [PATCH v2 2/8] vdpa/ifc: add multi queue support Andy Pei
2022-09-09 0:29 ` Xia, Chenbo
2022-09-09 7:55 ` Pei, Andy
2022-09-08 5:54 ` [PATCH v2 3/8] vdpa/ifc: set max queues according to HW spec Andy Pei
2022-09-09 5:56 ` Xia, Chenbo
2022-09-14 3:59 ` Pei, Andy
2022-09-08 5:54 ` [PATCH v2 4/8] vdpa/ifc: write queue count to MQ register Andy Pei
2022-09-09 7:35 ` Xia, Chenbo
2022-09-09 8:31 ` Pei, Andy
2022-09-08 5:54 ` [PATCH v2 5/8] vdpa/ifc: only configure enabled queue Andy Pei
2022-09-14 1:59 ` Xia, Chenbo
2022-09-14 2:57 ` Huang, Wei
2022-09-08 5:54 ` [PATCH v2 6/8] vdpa/ifc: support dynamic enable/disable queue Andy Pei
2022-09-14 2:23 ` Xia, Chenbo
2022-09-14 3:04 ` Huang, Wei
2022-09-14 3:14 ` Xia, Chenbo
2022-09-14 5:29 ` Pei, Andy
2022-09-08 5:54 ` [PATCH v2 7/8] vhost: configure device when any queue is ready for BLK device Andy Pei
2022-09-14 2:50 ` Xia, Chenbo
2022-09-14 7:01 ` Pei, Andy
2022-09-08 5:54 ` [PATCH v2 8/8] vhost: vDPA BLK devices configure device when all queue callfds are set Andy Pei
2022-09-16 6:16 ` [PATCH v3 0/8] vdpa/ifc: add multi queue support Andy Pei
2022-09-16 6:16 ` [PATCH v3 1/8] vdpa/ifc: add new device ID for legacy network device Andy Pei
2022-10-11 16:49 ` Maxime Coquelin
2022-10-12 6:33 ` Pei, Andy
2022-10-12 2:59 ` Xia, Chenbo
2022-10-12 6:34 ` Pei, Andy
2022-09-16 6:16 ` [PATCH v3 2/8] vdpa/ifc: add multi-queue support Andy Pei
2022-10-12 3:01 ` Xia, Chenbo
2022-10-12 7:25 ` Pei, Andy
2022-09-16 6:16 ` [PATCH v3 3/8] vdpa/ifc: set max queues based on virtio spec Andy Pei
2022-10-12 6:08 ` Xia, Chenbo
2022-10-12 7:22 ` Pei, Andy
2022-10-12 7:22 ` Pei, Andy
2022-09-16 6:16 ` [PATCH v3 4/8] vdpa/ifc: write queue count to MQ register Andy Pei
2022-10-12 8:08 ` Xia, Chenbo
2022-10-12 8:14 ` Pei, Andy
2022-09-16 6:16 ` [PATCH v3 5/8] vdpa/ifc: only configure enabled queue Andy Pei
2022-10-12 8:12 ` Xia, Chenbo
2022-10-12 8:37 ` Pei, Andy
2022-09-16 6:16 ` [PATCH v3 6/8] vdpa/ifc: support dynamic enable/disable queue Andy Pei
2022-10-12 8:19 ` Xia, Chenbo
2022-10-12 11:00 ` Pei, Andy
2022-09-16 6:16 ` [PATCH v3 7/8] vhost: vDPA blk device gets ready when any queue is ready Andy Pei
2022-10-12 9:09 ` Xia, Chenbo
2022-10-12 12:13 ` Pei, Andy
2022-10-13 1:00 ` Xia, Chenbo
2022-10-13 3:05 ` Pei, Andy
2022-10-13 7:16 ` Maxime Coquelin
2022-10-13 9:00 ` Pei, Andy
2022-09-16 6:16 ` [PATCH v3 8/8] vhost: improve vDPA blk device readiness condition Andy Pei
2022-10-12 9:35 ` Xia, Chenbo
2022-10-13 7:55 ` Pei, Andy
2022-10-13 8:23 ` Pei, Andy [this message]
2022-10-13 8:44 ` [PATCH v4 0/8] vdpa/ifc: add multi queue support Andy Pei
2022-10-13 8:44 ` [PATCH v4 1/8] vdpa/ifc: add new device ID for legacy network device Andy Pei
2022-10-13 8:44 ` [PATCH v4 2/8] vdpa/ifc: add multi-queue support Andy Pei
2022-10-17 6:21 ` Xia, Chenbo
2022-10-17 6:28 ` Pei, Andy
2022-10-13 8:44 ` [PATCH v4 3/8] vdpa/ifc: set max queues based on virtio spec Andy Pei
2022-10-17 6:22 ` Xia, Chenbo
2022-10-17 6:29 ` Pei, Andy
2022-10-13 8:44 ` [PATCH v4 4/8] vdpa/ifc: write queue count to MQ register Andy Pei
2022-10-17 6:23 ` Xia, Chenbo
2022-10-17 6:36 ` Pei, Andy
2022-10-13 8:44 ` [PATCH v4 5/8] vdpa/ifc: only configure enabled queue Andy Pei
2022-10-17 6:24 ` Xia, Chenbo
2022-10-17 6:38 ` Pei, Andy
2022-10-13 8:44 ` [PATCH v4 6/8] vdpa/ifc: support dynamic enable/disable queue Andy Pei
2022-10-17 6:26 ` Xia, Chenbo
2022-10-17 6:41 ` Pei, Andy
2022-10-13 8:44 ` [PATCH v4 7/8] vhost: vDPA blk device gets ready when any queue is ready Andy Pei
2022-10-17 6:34 ` Xia, Chenbo
2022-10-17 6:43 ` Pei, Andy
2022-10-13 8:44 ` [PATCH v4 8/8] vhost: improve vDPA blk device configure condition Andy Pei
2022-10-17 6:35 ` Xia, Chenbo
2022-10-17 7:12 ` Pei, Andy
2022-10-17 7:13 ` [PATCH v5 0/8] vdpa/ifc: add multi queue support Andy Pei
2022-10-17 7:13 ` [PATCH v5 1/8] vdpa/ifc: add new device ID for legacy network device Andy Pei
2022-10-17 7:13 ` [PATCH v5 2/8] vdpa/ifc: add multi-queue support Andy Pei
2022-10-17 7:13 ` [PATCH v5 3/8] vdpa/ifc: set max queues based on virtio spec Andy Pei
2022-10-17 7:13 ` [PATCH v5 4/8] vdpa/ifc: write queue count to MQ register Andy Pei
2022-10-17 7:13 ` [PATCH v5 5/8] vdpa/ifc: only configure enabled queue Andy Pei
2022-10-17 7:13 ` [PATCH v5 6/8] vdpa/ifc: support dynamic enable/disable queue Andy Pei
2022-10-17 7:13 ` [PATCH v5 7/8] vhost: vDPA blk device gets ready when the first queue is ready Andy Pei
2022-10-17 7:54 ` Xia, Chenbo
2022-10-17 8:36 ` Maxime Coquelin
2022-10-17 8:42 ` Xia, Chenbo
2022-10-17 8:46 ` Maxime Coquelin
2022-10-17 7:13 ` [PATCH v5 8/8] vhost: improve vDPA blk device configure condition Andy Pei
2022-10-17 11:41 ` [PATCH v6 0/8] vdpa/ifc: add multi queue support Andy Pei
2022-10-17 11:41 ` [PATCH v6 1/8] vdpa/ifc: add new device ID for legacy network device Andy Pei
2022-10-17 11:41 ` [PATCH v6 2/8] vdpa/ifc: add multi-queue support Andy Pei
2022-10-17 11:41 ` [PATCH v6 3/8] vdpa/ifc: set max queues based on virtio spec Andy Pei
2022-10-17 11:41 ` [PATCH v6 4/8] vdpa/ifc: write queue count to MQ register Andy Pei
2022-10-17 11:41 ` [PATCH v6 5/8] vdpa/ifc: only configure enabled queue Andy Pei
2022-10-17 11:41 ` [PATCH v6 6/8] vdpa/ifc: support dynamic enable/disable queue Andy Pei
2022-10-17 11:41 ` [PATCH v6 7/8] vhost: vDPA blk device gets ready when the first queue is ready Andy Pei
2022-10-17 11:41 ` [PATCH v6 8/8] vhost: improve vDPA blk device configure condition Andy Pei
2022-10-18 6:19 ` [PATCH v7 00/12] vdpa/ifc: add multi queue support Andy Pei
2022-10-18 6:19 ` [PATCH v7 01/12] vdpa/ifc: add new device ID for legacy network device Andy Pei
2022-10-18 6:19 ` [PATCH v7 02/12] vdpa/ifc: add multi-queue support Andy Pei
2022-10-18 6:19 ` [PATCH v7 03/12] vdpa/ifc: set max queues based on virtio spec Andy Pei
2022-10-18 6:19 ` [PATCH v7 04/12] vdpa/ifc: write queue count to MQ register Andy Pei
2022-10-18 6:19 ` [PATCH v7 05/12] vdpa/ifc: only configure enabled queue Andy Pei
2022-10-18 6:19 ` [PATCH v7 06/12] vdpa/ifc: support dynamic enable/disable queue Andy Pei
2022-10-18 6:19 ` [PATCH v7 07/12] vdpa/ifc: change internal function name Andy Pei
2022-10-18 6:19 ` [PATCH v7 08/12] vdpa/ifc: add internal API to get device Andy Pei
2022-10-18 6:19 ` [PATCH v7 09/12] vdpa/ifc: change some driver logic Andy Pei
2022-10-18 6:19 ` [PATCH v7 10/12] vhost: add vdpa device type to rte vdpa device Andy Pei
2022-10-18 7:14 ` Maxime Coquelin
2022-10-18 12:14 ` Pei, Andy
2022-10-19 9:14 ` Xia, Chenbo
2022-10-19 9:19 ` Pei, Andy
2022-10-18 6:19 ` [PATCH v7 11/12] vhost: vDPA blk device gets ready when the first queue is ready Andy Pei
2022-10-18 6:19 ` [PATCH v7 12/12] vhost: improve vDPA blk device configure condition Andy Pei
2022-10-18 12:07 ` [PATCH v8 00/12] vdpa/ifc: add multi queue support Andy Pei
2022-10-18 12:07 ` [PATCH v8 01/12] vdpa/ifc: add new device ID for legacy network device Andy Pei
2022-10-18 12:07 ` [PATCH v8 02/12] vdpa/ifc: add multi-queue support Andy Pei
2022-10-18 12:07 ` [PATCH v8 03/12] vdpa/ifc: set max queues based on virtio spec Andy Pei
2022-10-18 12:07 ` [PATCH v8 04/12] vdpa/ifc: write queue count to MQ register Andy Pei
2022-10-18 12:07 ` [PATCH v8 05/12] vdpa/ifc: only configure enabled queue Andy Pei
2022-10-18 12:07 ` [PATCH v8 06/12] vdpa/ifc: support dynamic enable/disable queue Andy Pei
2022-10-18 12:07 ` [PATCH v8 07/12] vdpa/ifc: change internal function name Andy Pei
2022-10-18 13:44 ` Maxime Coquelin
2022-10-18 13:47 ` Pei, Andy
2022-10-19 6:59 ` Xia, Chenbo
2022-10-19 9:41 ` Pei, Andy
2022-10-18 12:07 ` [PATCH v8 08/12] vdpa/ifc: add internal API to get device Andy Pei
2022-10-18 13:48 ` Maxime Coquelin
2022-10-18 13:52 ` Pei, Andy
2022-10-19 7:03 ` Xia, Chenbo
2022-10-19 9:40 ` Pei, Andy
2022-10-18 12:07 ` [PATCH v8 09/12] vdpa/ifc: change some driver logic Andy Pei
2022-10-18 13:57 ` Maxime Coquelin
2022-10-18 14:01 ` Pei, Andy
2022-10-19 9:13 ` Xia, Chenbo
2022-10-19 9:21 ` Pei, Andy
2022-10-18 12:07 ` [PATCH v8 10/12] vhost: add type to rte vdpa device Andy Pei
2022-10-18 13:15 ` Maxime Coquelin
2022-10-18 13:42 ` Pei, Andy
2022-10-18 12:07 ` [PATCH v8 11/12] vhost: vDPA blk device gets ready when the first queue is ready Andy Pei
2022-10-18 14:09 ` Maxime Coquelin
2022-10-18 14:11 ` Pei, Andy
2022-10-19 9:14 ` Xia, Chenbo
2022-10-19 9:18 ` Pei, Andy
2022-10-18 12:07 ` [PATCH v8 12/12] vhost: improve vDPA blk device configure condition Andy Pei
2022-10-18 14:14 ` Maxime Coquelin
2022-10-18 14:16 ` Pei, Andy
2022-10-19 9:15 ` Xia, Chenbo
2022-10-19 9:19 ` Pei, Andy
2022-10-19 8:41 ` [PATCH v9 00/12] vdpa/ifc: add multi queue support Andy Pei
2022-10-19 8:41 ` [PATCH v9 01/12] vdpa/ifc: add new device ID for legacy network device Andy Pei
2022-10-19 8:41 ` [PATCH v9 02/12] vdpa/ifc: add multi-queue support Andy Pei
2022-10-19 8:41 ` [PATCH v9 03/12] vdpa/ifc: set max queues based on virtio spec Andy Pei
2022-10-19 8:41 ` [PATCH v9 04/12] vdpa/ifc: write queue count to MQ register Andy Pei
2022-10-19 8:41 ` [PATCH v9 05/12] vdpa/ifc: only configure enabled queue Andy Pei
2022-10-19 8:41 ` [PATCH v9 06/12] vdpa/ifc: support dynamic enable/disable queue Andy Pei
2022-10-19 8:41 ` [PATCH v9 07/12] vdpa/ifc: change internal function name Andy Pei
2022-10-19 8:41 ` [PATCH v9 08/12] vdpa/ifc: add internal API to get device Andy Pei
2022-10-19 8:41 ` [PATCH v9 09/12] vdpa/ifc: improve internal list logic Andy Pei
2022-10-20 3:21 ` Xia, Chenbo
2022-10-20 5:53 ` Pei, Andy
2022-10-19 8:41 ` [PATCH v9 10/12] vhost: add type to rte vdpa device Andy Pei
2022-10-19 8:41 ` [PATCH v9 11/12] vhost: vDPA blk device gets ready when the first queue is ready Andy Pei
2022-10-19 8:41 ` [PATCH v9 12/12] vhost: improve vDPA blk device configure condition Andy Pei
2022-10-26 9:00 ` [PATCH v9 00/12] vdpa/ifc: add multi queue support Xia, Chenbo
2022-10-26 9:26 ` Pei, Andy
2022-08-23 4:34 ` [PATCH 2/8] vdpa/ifc: add multi queue suppoort Andy Pei
2022-08-23 4:35 ` [PATCH 3/8] vdpa/ifc: set max queues according to HW spec Andy Pei
2022-08-23 4:35 ` [PATCH 4/8] vdpa/ifc: write queue count to MQ register Andy Pei
2022-08-23 4:35 ` [PATCH 5/8] vdpa/ifc: only configure enabled queue Andy Pei
2022-08-23 4:35 ` [PATCH 6/8] vdpa/ifc: set vring state callback update data path Andy Pei
2022-08-23 4:35 ` [PATCH 7/8] vhost: configure device when any queue is ready for BLK device Andy Pei
2022-08-23 4:35 ` [PATCH 8/8] vhost: vDPA BLK devices configure device when all queue callfds are set Andy Pei
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=DM5PR11MB1739FB700FC439BD790E85208F259@DM5PR11MB1739.namprd11.prod.outlook.com \
--to=andy.pei@intel.com \
--cc=chenbo.xia@intel.com \
--cc=dev@dpdk.org \
--cc=gang.cao@intel.com \
--cc=maxime.coquelin@redhat.com \
--cc=rosen.xu@intel.com \
--cc=wei.huang@intel.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).