From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 012BFA00C2; Thu, 13 Oct 2022 09:55:45 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id ABF4142C94; Thu, 13 Oct 2022 09:55:45 +0200 (CEST) Received: from mga09.intel.com (mga09.intel.com [134.134.136.24]) by mails.dpdk.org (Postfix) with ESMTP id 2AD9742C27 for ; Thu, 13 Oct 2022 09:55:43 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1665647743; x=1697183743; h=from:to:cc:subject:date:message-id:references: in-reply-to:content-transfer-encoding:mime-version; bh=6mhSjU9mtD6Y2axHXYYttgpKZsivfB5p+LV6SFScKBE=; b=SVcaH4Zv+Ke1kaF2IxuoZVISWgTdcTJ8kWT0PvXrhjVy0sKGgVjkgXRG V5kWLIZinL/8lYOAsMFmc8HeMDPAdJ63Sl2Y4uqustC72FpCkQQVaY3At ZuQTcUNsbmwERLmLhsmBFQO5r4JPldWDfFuyZ+tkqlfEKLKqajRG5LXjh 83TyDAalxQR3JARkIMxsU4eZpsqDvkr6OiXN2z/5oANgHwQ+HnFvgJ0bJ OZbBs4FNRflULnXFppsBjO0DCn7Ggb2EkQrCS1LolozwJBsZ8fIBzLmTq JEifkbfm744fu10fqiZdDoBELW4y2RdxKrlfbNWmqTa4ERwAFH7cFYNVU Q==; X-IronPort-AV: E=McAfee;i="6500,9779,10498"; a="306075546" X-IronPort-AV: E=Sophos;i="5.95,180,1661842800"; d="scan'208";a="306075546" Received: from fmsmga007.fm.intel.com ([10.253.24.52]) by orsmga102.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 13 Oct 2022 00:55:42 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6500,9779,10498"; a="629428345" X-IronPort-AV: E=Sophos;i="5.95,180,1661842800"; d="scan'208";a="629428345" Received: from orsmsx602.amr.corp.intel.com ([10.22.229.15]) by fmsmga007.fm.intel.com with ESMTP; 13 Oct 2022 00:55:41 -0700 Received: from orsmsx609.amr.corp.intel.com (10.22.229.22) by ORSMSX602.amr.corp.intel.com (10.22.229.15) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.31; Thu, 13 Oct 2022 00:55:41 -0700 Received: from orsmsx612.amr.corp.intel.com (10.22.229.25) by ORSMSX609.amr.corp.intel.com (10.22.229.22) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.31; Thu, 13 Oct 2022 00:55:41 -0700 Received: from ORSEDG602.ED.cps.intel.com (10.7.248.7) by orsmsx612.amr.corp.intel.com (10.22.229.25) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.31 via Frontend Transport; Thu, 13 Oct 2022 00:55:41 -0700 Received: from NAM04-MW2-obe.outbound.protection.outlook.com (104.47.73.168) by edgegateway.intel.com (134.134.137.103) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.1.2375.31; Thu, 13 Oct 2022 00:55:40 -0700 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=Mw0We8ed4ENlbB0auPRmjUH29lpUBAZAfbsBnwtHDEGdJ6otp1GxCxT/GcUcNKll7FZcBC6aBscfrbpcXihGBc7a78MlK5seIV9khx9BDjHqfV/Ryf12fwCM68Apuz72NWmhJ1cnMYlPDiqaFDMO3sA92tq0GGAw3Fi6Li3P2bNzDOoVwsr6ACnePi3yTqSV0yXzE69kSfI6q/QTvRs7xraIAa36uXe8ftlmxUvFKrppzTFLBFK08jBH3KgKKT0o8qOpdcHEKkl0C8T58kjQ8Jfwd6B0+C4yKavn3m1+hhHG3qo7bvf9aPH8HMaKuCaVhT8M6OCsfuB+c3rK4ot2wQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=szpB72iH1/4BRouuHEsASOptjr2wTUBPCaF+i2O3eCE=; b=C6D5/n8pBFThfQwC6P8P98oqDqR9F+n1iaUz/jsDzsosMgfxwK8ak7tnZdyNa+ObUZxyxSr+rUjAXw7UFG/JLJubim19fKc3ESqRH7rIfmTXWsyPm3DxSQZOi3TgsLDt9BnCiehrKCIBd+ItybhTc46VKcLN75w/+/fnhdmYNkiH0qIVYvsEJpKr43EpsCBXEonGcN3CXB1wB9Fcv+iTSKfsRX6NyY96jD6mpwsShqvRl0nUtA4ar6Fx7nRVF1Rr7GMHK24qpJoVeDpAZY+ZOWuOabzsl8jXdrLTw+tV6j4x+AxCDgJYV/m1K9HFi+inejDcckzhJY4mxIpob9Rf0Q== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=intel.com; dmarc=pass action=none header.from=intel.com; dkim=pass header.d=intel.com; arc=none Received: from DM5PR11MB1739.namprd11.prod.outlook.com (2603:10b6:3:111::22) by MW4PR11MB7126.namprd11.prod.outlook.com (2603:10b6:303:22b::5) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5676.23; Thu, 13 Oct 2022 07:55:39 +0000 Received: from DM5PR11MB1739.namprd11.prod.outlook.com ([fe80::b8d7:6a95:bf26:e4ef]) by DM5PR11MB1739.namprd11.prod.outlook.com ([fe80::b8d7:6a95:bf26:e4ef%12]) with mapi id 15.20.5723.026; Thu, 13 Oct 2022 07:55:38 +0000 From: "Pei, Andy" To: "Xia, Chenbo" , "dev@dpdk.org" CC: "Xu, Rosen" , "Huang, Wei" , "Cao, Gang" , "maxime.coquelin@redhat.com" Subject: RE: [PATCH v3 8/8] vhost: improve vDPA blk device readiness condition Thread-Topic: [PATCH v3 8/8] vhost: improve vDPA blk device readiness condition Thread-Index: AQHYyZPyPMmG3ois2kyhSUbbYSnxUq4KpnqQgAF2Q2A= Date: Thu, 13 Oct 2022 07:55:38 +0000 Message-ID: References: <1661229305-240952-2-git-send-email-andy.pei@intel.com> <1663308990-621-1-git-send-email-andy.pei@intel.com> <1663308990-621-9-git-send-email-andy.pei@intel.com> In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: dlp-product: dlpe-windows dlp-version: 11.6.500.17 dlp-reaction: no-action authentication-results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=intel.com; x-ms-publictraffictype: Email x-ms-traffictypediagnostic: DM5PR11MB1739:EE_|MW4PR11MB7126:EE_ x-ms-office365-filtering-correlation-id: 9b7db8d4-3d9c-457a-4581-08daacf05183 x-ms-exchange-senderadcheck: 1 x-ms-exchange-antispam-relay: 0 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: zXJEHgbrG+A5JAO3CQE9U8jdmWQmZQPRcWNLtoP6b25iXSvjl/JwKnHHl7Ikgyb6V2tig03sbc583uCQZ4TLWtzrLFa5VOJqxiRvT9yxUMuB5uDW/zhonDu7npStb9dKTFXeP7lgqgQ4LlvbeH/GpioFnBSzX6osto/s6NsYaVA933eHhLdpKiYyPNbIuxCCk1chuwn7XdYk58Isvgc1OCIplCyiExn+WVh7KAjIHtpOMTUpgkDmOIkDe7EOENBJ9M2AniQBFnkWWKB2/QPkN9Sn7dyRw0OvtSgOk1G97NQCyhxeun3vKLMGohUPjts26eMG1NGtKmCEsJMm44thnjWy5C3UvuKU90XpdR3C2qBN5TLsPx1YxnTImfaHiCGsRyv6GN+K1nQ3ZkpBV9Ep+MYFRmbvFhQaLG9xrWWNAdsaT6whi3mqz5CBNX5eQ9eWcseoE0PMLo6wdekiDIg41QFmWf5vw69C4Sjha2fVn96VdwuZwCtl8FsQMJJHf0TuTeavnbFuro21XFOZ7syIQaPC4qSriZi+DUIqE2DldyX8kFLP+TAEX8w3YrLsimPRcoBrpIogqXUWRx+OhpmvOe4AKnuMe6qzaIrC6P+M2nXBmZQVU7loOlehEw0vH2cYFFqlKA1Kqzh+kDGixtXSvEadRAOvb4Idzz+ip8XIbsiLUJJ0CIp7RMl7icFnpdcfmVnSvUSARoRg97UxvbjY/Ra9X8ErOMSF5QbAK4xumrg9Km1dU4dTpPQR42XayJ6vQhh6mEvjNPbVwuKJJ+Mlug== x-forefront-antispam-report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:DM5PR11MB1739.namprd11.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230022)(39860400002)(396003)(136003)(346002)(376002)(366004)(451199015)(2906002)(66446008)(316002)(110136005)(33656002)(54906003)(122000001)(66556008)(66946007)(52536014)(8936002)(41300700001)(8676002)(71200400001)(64756008)(4326008)(76116006)(478600001)(82960400001)(38070700005)(83380400001)(53546011)(5660300002)(66476007)(186003)(55016003)(38100700002)(6506007)(86362001)(9686003)(7696005)(26005); DIR:OUT; SFP:1102; x-ms-exchange-antispam-messagedata-chunkcount: 1 x-ms-exchange-antispam-messagedata-0: =?us-ascii?Q?Y2BRo0oCMJZ3o7QCstnG2toRQz8nBHG/Lljcudobtim6NB8REa4xIWfHK28j?= =?us-ascii?Q?hOauXb9/u3QjTjDugtCmRDRbVhLa8C2TErRi7XJS71IDzuUtV+K0xe0hqqT+?= =?us-ascii?Q?skmAsdiLm7oAdUM22pwNGv/B0x0f/fiYro9JjwVhvCM72xI+tckReIxhFsu4?= =?us-ascii?Q?LajnXeFwZWcCFWhVElFdVmg8jZA1k1/ISsEQdz5YQXJCdPRAM7wz0cRgMEPp?= =?us-ascii?Q?zqlNEKZY5tD3YeNvjhQUmEmhtz8m02zRHLBSOh82MVSDOFiiOFDFITb/3Ru+?= =?us-ascii?Q?jzIHuUNESMJYvZd90YASuYTDSYUDbwoRN7fk1dzH6KK5KRuLue7LcWFIh8hM?= =?us-ascii?Q?8HzpFBDmMZDMs0DBCKpvFtoc4+vbuaYo95lQnKRMEkqcsgUPDWWSqVwHxGFj?= =?us-ascii?Q?bdcLj9ziHhdlysjlIV0KD9h4UVfOPiVwyNMFqo6UCzC5LkFKY+2+VvKBMLQi?= =?us-ascii?Q?a8clUsbhh+gMqekOIOVgkQZECY5kJSJdj0AKkb6tN40svb4CgsqR+24wraK6?= =?us-ascii?Q?NOZwS/aHQDX8mh1YM4XUf9ysq4Ip1D8JrEG2vhFKXAVAfhr1Ww6oe8Qs53bh?= =?us-ascii?Q?G9jqgZFaLWiS11vfis0g36CTN1wD2UrUks0/+Djb0igr0vxD78zjZU9l/gY2?= =?us-ascii?Q?nQzDgwyBlRT0OR22oWr1yGaIizvrjNkP4ygmwAeJ+0tlA1d+Fm7XRGG0shEe?= =?us-ascii?Q?Js9SHWTbfqFCU2r2elOvGnJdmqRoSIG3T0tPz7QbOFlWCKBmq4a9KldVEZHj?= =?us-ascii?Q?6/qQHVPfL2vp+W0Rc4r2F1ClLiemH/LwSrIJOCYTDQaGUsdPbM5oWKi3tAOQ?= =?us-ascii?Q?z97ed4zhGpc2HG7XYdAyOnLjkSkOQUjsHQcuKdMFjCnlEDZUk/bURq5d6wfn?= =?us-ascii?Q?KSY5C0M4VqWMJlojjNSeLqbf/XTVeM5FspqVoXiTSdMzWc0D0VSbX1AzJbGa?= =?us-ascii?Q?qEJQKVd79h0kQCR4ZKlIKs6tiL8Z7pEr799PqtcXyT8PpU1BfuclM2jBBh6P?= =?us-ascii?Q?zB8OKaKbTn92YmCUGQFWySvUors5M2+DMQVBxNq8c/zUdX5yDBAzsYmIb5Wh?= =?us-ascii?Q?ga+Zd1fHx+sONMqNpqvsU1ntNjmoy9McPy644JwluyY/iyzaBEj1TjWyxs/S?= =?us-ascii?Q?Py+3d2KRVkDlXlgLZqchF9lBbl7dNTxEZjdQ3WrcXZ7CZFyluuyIkHBkJNw1?= =?us-ascii?Q?0Frgd3GSBPY9q54sZeVCaNq8BBmFJHJqMf8obZeML5s1AtJiA6jHYj1pK9HK?= =?us-ascii?Q?9xYUEucNSueebzTjrhn6xtBgUlybnU7/y3fdgg4qFBYPMafYjIznLpqQieXy?= =?us-ascii?Q?XcBaIHbl4swnBD0W+fyU8rlfmxpmar2YgJdH832i1JuzLJefHIGFTnk9f3N+?= =?us-ascii?Q?vrqwuygfEC1WdX1VsR9NAf1IM5muMQPmt1A9RNqfWkD2eJ7b5sg/m9mS5Aii?= =?us-ascii?Q?sj/gXjnjFegs2Y0eLCbA8DGdohEIoSIbHFPCfPThkfuJ3SPL/ItbUv19AEAE?= =?us-ascii?Q?3Zaeu/Y1bnGQlD9Hivobwwsw1/R1vua1W/iYTRgbDzyQgeK3OHe/TqyEC+M3?= =?us-ascii?Q?1NE6QgxTbkzwjCXNmtKAJTBJ63zqu2bG2akxea7c?= Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: DM5PR11MB1739.namprd11.prod.outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: 9b7db8d4-3d9c-457a-4581-08daacf05183 X-MS-Exchange-CrossTenant-originalarrivaltime: 13 Oct 2022 07:55:38.5563 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 46c98d88-e344-4ed4-8496-4ed7712e255d X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: SzAJ27WoH2l3iwy/CBSn8P9oLXNjEb37PY3qHC5yMCWkuEszExEOKbtKwFnKznKCC6iYve2f0Aa3pfUZu+BE8A== X-MS-Exchange-Transport-CrossTenantHeadersStamped: MW4PR11MB7126 X-OriginatorOrg: intel.com X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Hi Chenbo, Thanks for your reply, my reply is inline. > -----Original Message----- > From: Xia, Chenbo > Sent: Wednesday, October 12, 2022 5:36 PM > To: Pei, Andy ; dev@dpdk.org > Cc: Xu, Rosen ; Huang, Wei ; > Cao, Gang ; maxime.coquelin@redhat.com > Subject: RE: [PATCH v3 8/8] vhost: improve vDPA blk device readiness > condition >=20 > > -----Original Message----- > > From: Pei, Andy > > Sent: Friday, September 16, 2022 2:17 PM > > To: dev@dpdk.org > > Cc: Xia, Chenbo ; Xu, Rosen > > ; Huang, Wei ; Cao, Gang > > ; maxime.coquelin@redhat.com > > Subject: [PATCH v3 8/8] vhost: improve vDPA blk device readiness > > condition > > > > In the virtio blk vDPA live migration use case, for the target VM, > > before the live migration process, QEMU will set call fd to all queues > > of vDPA back-end. QEMU and vDPA back-end stand by until live migration > > starts. During live migration process, QEMU sets kick fd and new call > > fd. However, after the kick fd is set to the vDPA back-end, the vDPA > > back-end configures device and data path starts. The new call fd will > > cause some kind of "re-configuration", this kind of "re-configuration" > > cause IO drop. > > After this patch, vDPA back-end configures device after kick fd and > > call fd are well set and make sure no IO drops. > > This patch only impact virtio blk vDPA device and does not impact net > > device. >=20 > IIUC, this is an improvement for MQ to make sure all call fds are well-se= t, but > previously it only makes sure one is well-set. If I am right, the title a= nd > commit message should describe it in better way. >=20 > Thanks, > Chenbo >=20 In the virtio blk vDPA live migration use case, for the target VM,=20 before the live migration process, QEMU will set call fd to all queues of v= DPA back-end. QEMU and vDPA back-end stand by until live migration starts. During live migration process, QEMU sets kick fd to all queues one by one, and new call fd all queues one by one. However, after the kick fd is set to the vDPA back-end, with the original call fd, the queue is ready. Then the vDPA back-end configures device and data path starts. The new call fd will cause some kind of "re-configuration", this kind of "re-configuration" cause IO drop. After this patch, vDPA back-end configures device after kick fd and=20 call fd are well set and make sure no IO drops. This patch only impact virtio blk vDPA device and does not impact net devic= e. Is this new commit message good? > > > > Signed-off-by: Andy Pei > > Signed-off-by: Huang Wei > > --- > > lib/vhost/vhost_user.c | 13 ++++++++++--- > > 1 file changed, 10 insertions(+), 3 deletions(-) > > > > diff --git a/lib/vhost/vhost_user.c b/lib/vhost/vhost_user.c index > > 9169cf5..14ff266 100644 > > --- a/lib/vhost/vhost_user.c > > +++ b/lib/vhost/vhost_user.c > > @@ -2983,6 +2983,7 @@ static int is_vring_iotlb(struct virtio_net *dev, > > uint32_t vdpa_type =3D 0; > > uint32_t request; > > uint32_t i; > > + uint16_t blk_call_fd; > > > > dev =3D get_device(vid); > > if (dev =3D=3D NULL) > > @@ -3210,9 +3211,15 @@ static int is_vring_iotlb(struct virtio_net *dev= , > > if (!vdpa_dev) > > goto out; > > > > - if (vdpa_type =3D=3D RTE_VHOST_VDPA_DEVICE_TYPE_BLK > > - && request !=3D VHOST_USER_SET_VRING_CALL) > > - goto out; > > + if (vdpa_type =3D=3D RTE_VHOST_VDPA_DEVICE_TYPE_BLK) { > > + if (request =3D=3D VHOST_USER_SET_VRING_CALL) { > > + blk_call_fd =3D ctx.msg.payload.u64 & > > VHOST_USER_VRING_IDX_MASK; > > + if (blk_call_fd !=3D dev->nr_vring - 1) > > + goto out; > > + } else { > > + goto out; > > + } > > + } > > > > if (!(dev->flags & VIRTIO_DEV_VDPA_CONFIGURED)) { > > if (vdpa_dev->ops->dev_conf(dev->vid)) > > -- > > 1.8.3.1