[PATCH] vhost: fix virtio blk vDPA live migration IO drop

[PATCH] vhost: fix virtio blk vDPA live migration IO drop

[Andy Pei]

In the virtio blk vDPA live migration use case, before the live
migration process, QEMU will set call fd to vDPA back-end. QEMU
and vDPA back-end stand by until live migration starts.
During live migration process, QEMU sets kick fd and a new call
fd. However, after the kick fd is set to the vDPA back-end, the
vDPA back-end configures device and data path starts. The new
call fd will cause some kind of "re-configuration", this kind
of "re-configuration" cause IO drop.
After this patch, vDPA back-end configures device after kick fd
and call fd are well set and make sure no IO drops.
This patch only impact virtio blk vDPA device and does not impact
net device.

Fixes: 7015b6577178 ("vdpa/ifc: add block device SW live-migration")

Signed-off-by: Andy Pei <andy.pei@intel.com>
---
 lib/vhost/vhost_user.c | 15 +++++++++++++++
 1 file changed, 15 insertions(+)

diff --git a/lib/vhost/vhost_user.c b/lib/vhost/vhost_user.c
index 2b9a3b6..cc03f67 100644
--- a/lib/vhost/vhost_user.c
+++ b/lib/vhost/vhost_user.c
@@ -2946,6 +2946,7 @@ static int is_vring_iotlb(struct virtio_net *dev,
 	int ret;
 	int unlock_required = 0;
 	bool handled;
+	uint32_t vdpa_type = 0;
 	uint32_t request;
 	uint32_t i;
 
@@ -3152,6 +3153,20 @@ static int is_vring_iotlb(struct virtio_net *dev,
 	if (!vdpa_dev)
 		goto out;
 
+	if (vdpa_dev->ops->get_dev_type) {
+		ret = vdpa_dev->ops->get_dev_type(vdpa_dev, &vdpa_type);
+		if (ret) {
+			VHOST_LOG_CONFIG(ERR, "failed to get vdpa dev type.\n");
+			ret = -1;
+			goto out;
+		}
+	} else {
+		vdpa_type = RTE_VHOST_VDPA_DEVICE_TYPE_NET;
+	}
+	if (vdpa_type == RTE_VHOST_VDPA_DEVICE_TYPE_BLK
+		&& request != VHOST_USER_SET_VRING_CALL)
+		goto out;
+
 	if (!(dev->flags & VIRTIO_DEV_VDPA_CONFIGURED)) {
 		if (vdpa_dev->ops->dev_conf(dev->vid))
 			VHOST_LOG_CONFIG(ERR, "(%s) failed to configure vDPA device\n",
-- 
1.8.3.1

Re: [PATCH] vhost: fix virtio blk vDPA live migration IO drop

[Maxime Coquelin]

On 6/22/22 09:47, Andy Pei wrote:
> In the virtio blk vDPA live migration use case, before the live
> migration process, QEMU will set call fd to vDPA back-end. QEMU
> and vDPA back-end stand by until live migration starts.
> During live migration process, QEMU sets kick fd and a new call
> fd. However, after the kick fd is set to the vDPA back-end, the
> vDPA back-end configures device and data path starts. The new
> call fd will cause some kind of "re-configuration", this kind
> of "re-configuration" cause IO drop.
> After this patch, vDPA back-end configures device after kick fd
> and call fd are well set and make sure no IO drops.
> This patch only impact virtio blk vDPA device and does not impact
> net device.
> 
> Fixes: 7015b6577178 ("vdpa/ifc: add block device SW live-migration")
> 
> Signed-off-by: Andy Pei <andy.pei@intel.com>
> ---
>   lib/vhost/vhost_user.c | 15 +++++++++++++++
>   1 file changed, 15 insertions(+)
> 

Reviewed-by: Maxime Coquelin <maxime.coquelin@redhat.com>

Thanks,
Maxime

Re: [PATCH] vhost: fix virtio blk vDPA live migration IO drop

[Maxime Coquelin]

On 6/22/22 09:47, Andy Pei wrote:
> In the virtio blk vDPA live migration use case, before the live
> migration process, QEMU will set call fd to vDPA back-end. QEMU
> and vDPA back-end stand by until live migration starts.
> During live migration process, QEMU sets kick fd and a new call
> fd. However, after the kick fd is set to the vDPA back-end, the
> vDPA back-end configures device and data path starts. The new
> call fd will cause some kind of "re-configuration", this kind
> of "re-configuration" cause IO drop.
> After this patch, vDPA back-end configures device after kick fd
> and call fd are well set and make sure no IO drops.
> This patch only impact virtio blk vDPA device and does not impact
> net device.
> 
> Fixes: 7015b6577178 ("vdpa/ifc: add block device SW live-migration")
> 
> Signed-off-by: Andy Pei <andy.pei@intel.com>
> ---
>   lib/vhost/vhost_user.c | 15 +++++++++++++++
>   1 file changed, 15 insertions(+)
> 

Applied to dpdk-next-virtio/main.

Thanks,
Maxime

RE: [PATCH] vhost: fix virtio blk vDPA live migration IO drop

[Pei, Andy]

HI Maxime,

Thanks.

> -----Original Message-----
> From: Maxime Coquelin <maxime.coquelin@redhat.com>
> Sent: Friday, July 1, 2022 9:55 PM
> To: Pei, Andy <andy.pei@intel.com>; dev@dpdk.org
> Cc: Xia, Chenbo <chenbo.xia@intel.com>; Xu, Rosen <rosen.xu@intel.com>;
> Xiao, QimaiX <qimaix.xiao@intel.com>
> Subject: Re: [PATCH] vhost: fix virtio blk vDPA live migration IO drop
> 
> 
> 
> On 6/22/22 09:47, Andy Pei wrote:
> > In the virtio blk vDPA live migration use case, before the live
> > migration process, QEMU will set call fd to vDPA back-end. QEMU and
> > vDPA back-end stand by until live migration starts.
> > During live migration process, QEMU sets kick fd and a new call fd.
> > However, after the kick fd is set to the vDPA back-end, the vDPA
> > back-end configures device and data path starts. The new call fd will
> > cause some kind of "re-configuration", this kind of "re-configuration"
> > cause IO drop.
> > After this patch, vDPA back-end configures device after kick fd and
> > call fd are well set and make sure no IO drops.
> > This patch only impact virtio blk vDPA device and does not impact net
> > device.
> >
> > Fixes: 7015b6577178 ("vdpa/ifc: add block device SW live-migration")
> >
> > Signed-off-by: Andy Pei <andy.pei@intel.com>
> > ---
> >   lib/vhost/vhost_user.c | 15 +++++++++++++++
> >   1 file changed, 15 insertions(+)
> >
> 
> Applied to dpdk-next-virtio/main.
> 
> Thanks,
> Maxime