From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id A567BA00C2; Thu, 13 Oct 2022 10:23:40 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 4F8B142C95; Thu, 13 Oct 2022 10:23:40 +0200 (CEST) Received: from mga05.intel.com (mga05.intel.com [192.55.52.43]) by mails.dpdk.org (Postfix) with ESMTP id E508242C27 for ; Thu, 13 Oct 2022 10:23:37 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1665649418; x=1697185418; h=from:to:cc:subject:date:message-id:references: in-reply-to:content-transfer-encoding:mime-version; bh=/43P8BekMOrIFYLWn44hY/QhYOI0nNH9KlBUtCc282Y=; b=iHnq+XmG9cfWUPgXEd7xMgHCOeGNbNMPI1rGsv2sw+1koIBrh4LNqRyc GzuROvD2O4FdE0KIFJsajVcUz5x9XYJzxCPuN4OBpNd7ubbWLSQD3z3MH lIjpwGGQemtljRc8tJS1nHd7j7AjTqVDGqr0w4iSYZ4LOLc/7cnz0g01Y nXqD9+VwGSKfCZCvpXIJJb106uji60wQR/cxE7RW/emSlFBYEHlW0wKEX jkPn1gr1jeeJwFKDyKOMYUqjGEdkh/z7YD+ijhUdZ5Ag6K/IaATXvXrZ5 YFP1+S279ytzDfq1y8KCjdwxg6TmWTkdIzdGDfttNn7sm0TvEZA8s6ldc Q==; X-IronPort-AV: E=McAfee;i="6500,9779,10498"; a="391324780" X-IronPort-AV: E=Sophos;i="5.95,180,1661842800"; d="scan'208";a="391324780" Received: from fmsmga002.fm.intel.com ([10.253.24.26]) by fmsmga105.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 13 Oct 2022 01:23:36 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6500,9779,10498"; a="731768999" X-IronPort-AV: E=Sophos;i="5.95,180,1661842800"; d="scan'208";a="731768999" Received: from fmsmsx603.amr.corp.intel.com ([10.18.126.83]) by fmsmga002.fm.intel.com with ESMTP; 13 Oct 2022 01:23:36 -0700 Received: from fmsmsx611.amr.corp.intel.com (10.18.126.91) by fmsmsx603.amr.corp.intel.com (10.18.126.83) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.31; Thu, 13 Oct 2022 01:23:36 -0700 Received: from fmsmsx610.amr.corp.intel.com (10.18.126.90) by fmsmsx611.amr.corp.intel.com (10.18.126.91) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.31; Thu, 13 Oct 2022 01:23:35 -0700 Received: from fmsedg602.ED.cps.intel.com (10.1.192.136) by fmsmsx610.amr.corp.intel.com (10.18.126.90) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.31 via Frontend Transport; Thu, 13 Oct 2022 01:23:35 -0700 Received: from NAM11-CO1-obe.outbound.protection.outlook.com (104.47.56.176) by edgegateway.intel.com (192.55.55.71) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.1.2375.31; Thu, 13 Oct 2022 01:23:35 -0700 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=G9siVekUOXpsFpeTK4Q7pOhwmQLo1Lwq7su5cwkQCb//PihiMWOWfdGdFcUNYwzrFY11epnasNIgFPFImwGo1/wlzoEShAr6jTiq3G5lqbCyrsHCrw9ZKhw6MKmPkn9A1+L+C/hG/gaC/GzgCDrT1CLLTXl1/aib07GBY6vXDocZYoCYySBjsi5/qvnVsXaJn94BrJ+YFeM+gqKybuuJ7t24cCBBIeS1xI/FVb6mmE9MgwD9qW494NDDqGKl7sBguslP5zCTjgxinfvHRTBITYqxjUvLsDMMqZIHCOLQGwSmpjxmhrpuiJs+VpR2YJ1X8fAwVSvCVbX6c59fpIgGgA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=IkMbuqCFmsbj5/W51SAq+XMGVoShkSK1vTL3OWERc/8=; b=aGdZ8Yg/zFxlQS0E1flrmZu7ivkUQCeHlU8UEpT3oI5YUhfw6NXdN8corjEF7ItJIkerNQWmw7eIwmfQGgdzKBr75R6JZHc6EWZinUciR8NS8WRB88e/g3w6LTa7lPXDCw00yDeS6TfS4js5HLN+E2zT0Gs+vKgacH8Y4OXpwf1plt5umEo65GBAY8WZOMhaqgANDfbPubl4Y+12nj+cH1yfh/n5ag7DKxxrjYZ+fTAKUpOrSTjLmhxDc3N/50AboD7o8CKqBz0fhVIlWUiuIekVZ/OJuCskHAuvzOPxE5tDQ6N2nUluOcszs1JWOFXl9m2ajTrvkKzOXF6XwIqxdg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=intel.com; dmarc=pass action=none header.from=intel.com; dkim=pass header.d=intel.com; arc=none Received: from DM5PR11MB1739.namprd11.prod.outlook.com (2603:10b6:3:111::22) by SA1PR11MB6808.namprd11.prod.outlook.com (2603:10b6:806:24f::10) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5709.21; Thu, 13 Oct 2022 08:23:33 +0000 Received: from DM5PR11MB1739.namprd11.prod.outlook.com ([fe80::b8d7:6a95:bf26:e4ef]) by DM5PR11MB1739.namprd11.prod.outlook.com ([fe80::b8d7:6a95:bf26:e4ef%12]) with mapi id 15.20.5723.026; Thu, 13 Oct 2022 08:23:33 +0000 From: "Pei, Andy" To: "Pei, Andy" , "Xia, Chenbo" , "dev@dpdk.org" CC: "Xu, Rosen" , "Huang, Wei" , "Cao, Gang" , "maxime.coquelin@redhat.com" Subject: RE: [PATCH v3 8/8] vhost: improve vDPA blk device readiness condition Thread-Topic: [PATCH v3 8/8] vhost: improve vDPA blk device readiness condition Thread-Index: AQHYyZPyPMmG3ois2kyhSUbbYSnxUq4KpnqQgAF2Q2CAAAfsQA== Date: Thu, 13 Oct 2022 08:23:33 +0000 Message-ID: References: <1661229305-240952-2-git-send-email-andy.pei@intel.com> <1663308990-621-1-git-send-email-andy.pei@intel.com> <1663308990-621-9-git-send-email-andy.pei@intel.com> In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: dlp-product: dlpe-windows dlp-version: 11.6.500.17 dlp-reaction: no-action authentication-results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=intel.com; x-ms-publictraffictype: Email x-ms-traffictypediagnostic: DM5PR11MB1739:EE_|SA1PR11MB6808:EE_ x-ms-office365-filtering-correlation-id: 4fa733a0-7c07-4dc3-d5b5-08daacf437f5 x-ms-exchange-senderadcheck: 1 x-ms-exchange-antispam-relay: 0 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: 6jPZ1cXxMnEf4MvvAMveQWkDlLADzAvWXKU7x6NPR/AUN/pRCOQ7LVZuBD62dmeUFuNsWp8X1xpu2kMssahv3307Q0CEN/JciBU+h+bDztDYoeH2iIO0HCAZ26CTE+h52C8GoFEisDllW0La7Nn21o1ziTy6cW0fMF0esE6DEcSY725jXW0NNN9/SeJpESYvl3EqmKKctCzDIxA5jRXhQlKCk868AWDdxZfsvT4iNeUEE8o/tFy/Kg4J1HKNT5+PHlKROJ2SXPLx9OhYsTg59q5Nub9J6j+6K38grRc0/17vD13Z4CPNpOW789vcweZsFelm3wcX7sRUtKsbZIQURFLvSWE4j48h3qw+w2zLGWgujRwO9RVgtGZXXhD+D1ZGlSbTp6ZPyGvgA9dMTtuMdzpl8OUHWZcp7EqtC/5O5g2705ccagg10gH78KudB9ZY8ux3ipIrzGpk3154HBAFJQgXrGAOQlJXh0JYkxTtQk19QqobI0KvfmXiBekxQmobjmkajz7quQk1m31yufX0oM7xUc3cVrOU9jB9uL3ps45O9Vb7smsipAnEXSywJ/ESwMwPU5vzhy3cWmCPxt1m1NQeB52xfbcfDXfzUe8bUNcD5QTKgcMGHT13RMGQ2ucqctpphwR2JgPzUbjGMVJ3RVEEuUyTGIhXDhEwB6rH01VTUO7MLA9WUVjNE/hOEyPPMuXo4oEqJliMX5Q9wDf+9HhGWYXT/VOOauhcRwudKQgcLGwc7TIXYcxEL19+X8xnoiK9eySBf9qH7zyVKUvp7A== x-forefront-antispam-report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:DM5PR11MB1739.namprd11.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230022)(396003)(39860400002)(366004)(376002)(346002)(136003)(451199015)(54906003)(110136005)(53546011)(4326008)(316002)(6506007)(66476007)(66556008)(66946007)(83380400001)(33656002)(122000001)(71200400001)(7696005)(478600001)(64756008)(76116006)(82960400001)(186003)(66446008)(2940100002)(9686003)(86362001)(26005)(38070700005)(55016003)(52536014)(38100700002)(8936002)(5660300002)(8676002)(2906002)(41300700001); DIR:OUT; SFP:1102; x-ms-exchange-antispam-messagedata-chunkcount: 1 x-ms-exchange-antispam-messagedata-0: =?us-ascii?Q?rTm5u+c9IUzdgm7W7rB6c/lxNUMln80e+HvJn4qXA/zeY1IGljeg/B0FVYDt?= =?us-ascii?Q?fCIhbwrera+/+JlkQimWoBVO1Bt9SOckwQZSXLwUmko1vCDAlRDCiDnRoxLt?= =?us-ascii?Q?06GzFVLxzoLa2IJJU/LwacyXDsDwf7ZEg/adn9rADk2V0BLjiDP/SYnFpBSG?= =?us-ascii?Q?SCW8oCZqS8HE5212sNSCdqq2/ZHwOqOBgTP2w8pRGq4v2/CTssWe9JF0AljI?= =?us-ascii?Q?NXwIqdMl8Ckab1U83K0Ljc6niUJql4G6EfPKRwJYJ92cQyu98LM4K4E2pkbR?= =?us-ascii?Q?W8kF+MJt0Syf9mlM0Q9kx/2JL8t7NUPHrYFiAABYXCqqbdggbJASmjkFxan9?= =?us-ascii?Q?ArR8DR5fnvcSJpWQuL4vUoCz/39hZfk2QFX0zY08oTbobSV905rtC15YxLBg?= =?us-ascii?Q?GzJnKoVpR8vanefLa+6QYDmokZM6m7QGBujMi3lpuqzWFZgsznrVPuDchJtO?= =?us-ascii?Q?4mCnXTdhHxAKMl5BaRSKM1p5bt4N3WlNvMbFi8oEylUk1UVkJbHb+aG/PfiH?= =?us-ascii?Q?75nhUqwXOITEGRgPBg/ey4ol39T1PTd85s3GsE9nMmSvxxnEyoZ5tdo6btLr?= =?us-ascii?Q?9oGndZux9ps38RTdRef6rsRXpHDmLcxjfoynsRYBQz279nRZU8CuMq3AEnUl?= =?us-ascii?Q?/VOGB7H56GUBIuZUmNqJRQUEc7vkupq8IIOl5ZglfYXF0OX0l9kSciybEUtc?= =?us-ascii?Q?eGY1e5iSbOZsm+d/1zT6EW3pPPhy5fa4FBT8QWGtkpA2U50J6t3Gy/qCurT8?= =?us-ascii?Q?PD5rcSKajT3ZouJu7E+gMdwfZ3XQO6XYOYoXxWo33x4xNpHlUNOxjbHopWYK?= =?us-ascii?Q?vhScNJBhNrmjz8Mm2NLNuAucZmLp8kYu+iSFXipSpmaGdhFBwKedLm2GXTt9?= =?us-ascii?Q?p7SGOZGqy5BxGNdgPYIhp61tUYBYNIo29HsaQORHR7PHQ+16q9qzP3oXi6r0?= =?us-ascii?Q?yvkgrpjtQ/L2RTVIHQbRHh7uIXNYmSjgvITGOKrHfRPlF3jDkUCr/buO3A8h?= =?us-ascii?Q?1/W/BD4TIfSMSox+1Ei/5kZ5RF41jMkkxYpTEeKbFalV48mBqV//0EJRrpR2?= =?us-ascii?Q?ubXMCoOrEb7MCx6pxuByu85OBn4UGVUe1YvDDNgai2BIU9i7uCOZIx/JbddZ?= =?us-ascii?Q?fiBcoxG3X7fuCCk1QccBR//zQ82Fp73TRYY3BuUKyF/au7BjVRTkeJrkPNGS?= =?us-ascii?Q?16nyOdG6jLuS5UUONLRH8oBRx4adl4n0lylhx8jolQfLEF4gEHTcWYoH9Bt0?= =?us-ascii?Q?TnFoPNFRKLM8oIY1obWw3RZ8alpJwq2iOYEC4e8aE+dexv1ErnMs+3/oX3Av?= =?us-ascii?Q?W3xBJaJSHa7NPOEzKpMXuQaUN/A/F6N/bl1EQfyckJEvwH9cYIEPyazR4NmT?= =?us-ascii?Q?ZXWrTRhbhg4JfxCPLRMIMWXtGXTwsTpXsF1X9Fts8YmfQ1x4XHI8X3mQ1a4x?= =?us-ascii?Q?/Il9+u9Mc7HQbx6y81Kk2N2rtsnrKpoCMyfCRvlb+pe/Dw8vt4jcLcY8/yLd?= =?us-ascii?Q?tybrarp8SquAautO5X4O50gedaiCHH70z7FtDB6sTGkZWXAGdZd5keWp1har?= =?us-ascii?Q?vk0OTEyFO5VG080MprRzbcYuv+EhoqBZw99SRd63?= Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: DM5PR11MB1739.namprd11.prod.outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: 4fa733a0-7c07-4dc3-d5b5-08daacf437f5 X-MS-Exchange-CrossTenant-originalarrivaltime: 13 Oct 2022 08:23:33.6889 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 46c98d88-e344-4ed4-8496-4ed7712e255d X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: dgTmg7h+4zKnTX9xahh7sZUdLRlVL2NYL2qr8vRfdw3DE3eeXQ9lEalQU9kIx4KbGcnl2i5DFy/ewQzWdGLSnA== X-MS-Exchange-Transport-CrossTenantHeadersStamped: SA1PR11MB6808 X-OriginatorOrg: intel.com X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org > -----Original Message----- > From: Pei, Andy > Sent: Thursday, October 13, 2022 3:56 PM > To: Xia, Chenbo ; dev@dpdk.org > Cc: Xu, Rosen ; Huang, Wei ; > Cao, Gang ; maxime.coquelin@redhat.com > Subject: RE: [PATCH v3 8/8] vhost: improve vDPA blk device readiness > condition >=20 > Hi Chenbo, >=20 > Thanks for your reply, my reply is inline. >=20 > > -----Original Message----- > > From: Xia, Chenbo > > Sent: Wednesday, October 12, 2022 5:36 PM > > To: Pei, Andy ; dev@dpdk.org > > Cc: Xu, Rosen ; Huang, Wei ; > > Cao, Gang ; maxime.coquelin@redhat.com > > Subject: RE: [PATCH v3 8/8] vhost: improve vDPA blk device readiness > > condition > > > > > -----Original Message----- > > > From: Pei, Andy > > > Sent: Friday, September 16, 2022 2:17 PM > > > To: dev@dpdk.org > > > Cc: Xia, Chenbo ; Xu, Rosen > > > ; Huang, Wei ; Cao, Gang > > > ; maxime.coquelin@redhat.com > > > Subject: [PATCH v3 8/8] vhost: improve vDPA blk device readiness > > > condition > > > > > > In the virtio blk vDPA live migration use case, for the target VM, > > > before the live migration process, QEMU will set call fd to all > > > queues of vDPA back-end. QEMU and vDPA back-end stand by until live > > > migration starts. During live migration process, QEMU sets kick fd > > > and new call fd. However, after the kick fd is set to the vDPA > > > back-end, the vDPA back-end configures device and data path starts. > > > The new call fd will cause some kind of "re-configuration", this kind= of > "re-configuration" > > > cause IO drop. > > > After this patch, vDPA back-end configures device after kick fd and > > > call fd are well set and make sure no IO drops. > > > This patch only impact virtio blk vDPA device and does not impact > > > net device. > > > > IIUC, this is an improvement for MQ to make sure all call fds are > > well-set, but previously it only makes sure one is well-set. If I am > > right, the title and commit message should describe it in better way. > > > > Thanks, > > Chenbo > > > In the virtio blk vDPA live migration use case, for the target VM, before= the > live migration process, QEMU will set call fd to all queues of vDPA back-= end. > QEMU and vDPA back-end stand by until live migration starts. > During live migration process, QEMU sets kick fd to all queues one by one= , > and new call fd all queues one by one. > However, after the kick fd is set to the vDPA back-end, with the original= call fd, > the queue is ready. > Then the vDPA back-end configures device and data path starts. > The new call fd will cause some kind of "re-configuration", this kind of = "re- > configuration" cause IO drop. > After this patch, vDPA back-end configures device after kick fd and call = fd are > well set and make sure no IO drops. > This patch only impact virtio blk vDPA device and does not impact net dev= ice. >=20 > Is this new commit message good? Explanation seems duplicate with former commit " vhost: fix virtio block vDPA live migration IO drop". So I will make it simple here, "To support multi-queue, configure device after call fd of all queues are s= et." I will use " vhost: improve vDPA blk device configure condition" for title= =20 To reduce confusion. > > > > > > Signed-off-by: Andy Pei > > > Signed-off-by: Huang Wei > > > --- > > > lib/vhost/vhost_user.c | 13 ++++++++++--- > > > 1 file changed, 10 insertions(+), 3 deletions(-) > > > > > > diff --git a/lib/vhost/vhost_user.c b/lib/vhost/vhost_user.c index > > > 9169cf5..14ff266 100644 > > > --- a/lib/vhost/vhost_user.c > > > +++ b/lib/vhost/vhost_user.c > > > @@ -2983,6 +2983,7 @@ static int is_vring_iotlb(struct virtio_net *de= v, > > > uint32_t vdpa_type =3D 0; > > > uint32_t request; > > > uint32_t i; > > > + uint16_t blk_call_fd; > > > > > > dev =3D get_device(vid); > > > if (dev =3D=3D NULL) > > > @@ -3210,9 +3211,15 @@ static int is_vring_iotlb(struct virtio_net *d= ev, > > > if (!vdpa_dev) > > > goto out; > > > > > > - if (vdpa_type =3D=3D RTE_VHOST_VDPA_DEVICE_TYPE_BLK > > > - && request !=3D VHOST_USER_SET_VRING_CALL) > > > - goto out; > > > + if (vdpa_type =3D=3D RTE_VHOST_VDPA_DEVICE_TYPE_BLK) { > > > + if (request =3D=3D VHOST_USER_SET_VRING_CALL) { > > > + blk_call_fd =3D ctx.msg.payload.u64 & > > > VHOST_USER_VRING_IDX_MASK; > > > + if (blk_call_fd !=3D dev->nr_vring - 1) > > > + goto out; > > > + } else { > > > + goto out; > > > + } > > > + } > > > > > > if (!(dev->flags & VIRTIO_DEV_VDPA_CONFIGURED)) { > > > if (vdpa_dev->ops->dev_conf(dev->vid)) > > > -- > > > 1.8.3.1