From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 4CB24A00C3; Fri, 13 May 2022 05:32:37 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 09D2240DDE; Fri, 13 May 2022 05:32:37 +0200 (CEST) Received: from mga09.intel.com (mga09.intel.com [134.134.136.24]) by mails.dpdk.org (Postfix) with ESMTP id 979B040698 for ; Fri, 13 May 2022 05:32:35 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1652412755; x=1683948755; h=from:to:cc:subject:date:message-id:references: in-reply-to:content-transfer-encoding:mime-version; bh=MGMo3/zSZvxN6soSmBuiPzAtkgaFh5HEAeTqIJ35rVo=; b=cl6l9y5ok6KUgZFz0dObX5Y4JEUEcCeM1EydQczrSEjpe4o501cOE6u0 5H6Hjkn2R8AQbSV9VpjDCKWszvCEvNT3CBYV8z6xmMgtnkQaqe9QDxCKW Cg607GcDs/5lhIDbcSvUjz1eLYzIRTLU0kaK+8+8jJiVgmvSYmxgi55mi UYtx3kNxSPmu6rR6GdH9ECwjXrMge913MY5ro1twAx6Thv0oJHbWvS6pC FBjrfDRxXz5X7kflRTKv5vXmhc9XTQCMLP1H9UnIi+kYrT8Ql0kMTsI29 DYnB/Vyv6RqeRJ4/KdEnmvKkCaz9xUi6izPVx+fsTwkJ9wEpVDJKdUPEu A==; X-IronPort-AV: E=McAfee;i="6400,9594,10345"; a="269874224" X-IronPort-AV: E=Sophos;i="5.91,221,1647327600"; d="scan'208";a="269874224" Received: from fmsmga007.fm.intel.com ([10.253.24.52]) by orsmga102.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 12 May 2022 20:32:34 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.91,221,1647327600"; d="scan'208";a="572801138" Received: from fmsmsx603.amr.corp.intel.com ([10.18.126.83]) by fmsmga007.fm.intel.com with ESMTP; 12 May 2022 20:32:34 -0700 Received: from fmsmsx610.amr.corp.intel.com (10.18.126.90) by fmsmsx603.amr.corp.intel.com (10.18.126.83) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2308.27; Thu, 12 May 2022 20:32:33 -0700 Received: from fmsmsx612.amr.corp.intel.com (10.18.126.92) by fmsmsx610.amr.corp.intel.com (10.18.126.90) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2308.27; Thu, 12 May 2022 20:32:33 -0700 Received: from fmsmsx612.amr.corp.intel.com ([10.18.126.92]) by fmsmsx612.amr.corp.intel.com ([10.18.126.92]) with mapi id 15.01.2308.027; Thu, 12 May 2022 20:32:33 -0700 From: "Pei, Andy" To: "Xia, Chenbo" , "dev@dpdk.org" CC: "maxime.coquelin@redhat.com" , "Cao, Gang" , "Liu, Changpeng" Subject: RE: [PATCH v7 06/18] vdpa/ifc: add block device SW live-migration Thread-Topic: [PATCH v7 06/18] vdpa/ifc: add block device SW live-migration Thread-Index: AQHYWheDsJei7826WUGDCpro8PYkba0bSdbQgAD13CA= Date: Fri, 13 May 2022 03:32:33 +0000 Message-ID: <235853b3989949ccab972d28d217e7c7@intel.com> References: <1643093258-47258-2-git-send-email-andy.pei@intel.com> <1651048206-282372-1-git-send-email-andy.pei@intel.com> <1651048206-282372-7-git-send-email-andy.pei@intel.com> In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: dlp-product: dlpe-windows dlp-version: 11.6.401.20 dlp-reaction: no-action x-originating-ip: [10.239.127.36] Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Hi Chenbo, Thanks for your reply. Your suggestion is good. I will fix in next version. > -----Original Message----- > From: Xia, Chenbo > Sent: Thursday, May 12, 2022 8:55 PM > To: Pei, Andy ; dev@dpdk.org > Cc: maxime.coquelin@redhat.com; Cao, Gang ; Liu, > Changpeng > Subject: RE: [PATCH v7 06/18] vdpa/ifc: add block device SW live-migratio= n >=20 > > -----Original Message----- > > From: Pei, Andy > > Sent: Wednesday, April 27, 2022 4:30 PM > > To: dev@dpdk.org > > Cc: Xia, Chenbo ; maxime.coquelin@redhat.com; > > Cao, Gang ; Liu, Changpeng > > > > Subject: [PATCH v7 06/18] vdpa/ifc: add block device SW live-migration > > > > Add SW live-migration support to block device. > > > > Signed-off-by: Andy Pei > > --- > > drivers/vdpa/ifc/ifcvf_vdpa.c | 33 +++++++++++++++++++++++++++++---- > > 1 file changed, 29 insertions(+), 4 deletions(-) > > > > diff --git a/drivers/vdpa/ifc/ifcvf_vdpa.c > > b/drivers/vdpa/ifc/ifcvf_vdpa.c index 07fc3ca..8a260b7 100644 > > --- a/drivers/vdpa/ifc/ifcvf_vdpa.c > > +++ b/drivers/vdpa/ifc/ifcvf_vdpa.c > > @@ -312,6 +312,7 @@ struct rte_vdpa_dev_info { vdpa_ifcvf_stop(struct > > ifcvf_internal *internal) { > > struct ifcvf_hw *hw =3D &internal->hw; > > + struct rte_vhost_vring vq; > > uint32_t i; > > int vid; > > uint64_t features =3D 0; > > @@ -319,6 +320,22 @@ struct rte_vdpa_dev_info { > > uint64_t len; > > > > vid =3D internal->vid; > > + > > + /* to make sure no packet is lost for blk device > > + * do not stop until last_avail_idx =3D=3D last_used_idx > > + */ > > + if (internal->device_type =3D=3D IFCVF_BLK) { > > + for (i =3D 0; i < hw->nr_vring; i++) { > > + rte_vhost_get_vhost_vring(internal->vid, i, &vq); > > + while (vq.avail->idx !=3D vq.used->idx) { > > + ifcvf_notify_queue(hw, i); > > + usleep(10); > > + } > > + hw->vring[i].last_avail_idx =3D vq.avail->idx; > > + hw->vring[i].last_used_idx =3D vq.used->idx; > > + } > > + } > > + >=20 > This seems not match with the above comment about avoiding in-flight > packets. > But the change in patch 17 seems good. Why not just using the > implementation in patch 17? >=20 > Thanks, > Chenbo >=20 > > ifcvf_stop_hw(hw); > > > > for (i =3D 0; i < hw->nr_vring; i++) > > @@ -642,8 +659,10 @@ struct rte_vdpa_dev_info { > > } > > hw->vring[i].avail =3D gpa; > > > > - /* Direct I/O for Tx queue, relay for Rx queue */ > > - if (i & 1) { > > + /* NET: Direct I/O for Tx queue, relay for Rx queue > > + * BLK: relay every queue > > + */ > > + if ((internal->device_type =3D=3D IFCVF_NET) && (i & 1)) { > > gpa =3D hva_to_gpa(vid, (uint64_t)(uintptr_t)vq.used); > > if (gpa =3D=3D 0) { > > DRV_LOG(ERR, "Fail to get GPA for used > ring."); @@ -693,8 +712,12 > > @@ struct rte_vdpa_dev_info { > > > > for (i =3D 0; i < hw->nr_vring; i++) { > > /* synchronize remaining new used entries if any */ > > - if ((i & 1) =3D=3D 0) > > + if (internal->device_type =3D=3D IFCVF_NET) { > > + if ((i & 1) =3D=3D 0) > > + update_used_ring(internal, i); > > + } else if (internal->device_type =3D=3D IFCVF_BLK) { > > update_used_ring(internal, i); > > + } > > > > rte_vhost_get_vhost_vring(vid, i, &vq); > > len =3D IFCVF_USED_RING_LEN(vq.size); @@ -756,7 +779,9 > @@ struct > > rte_vdpa_dev_info { > > } > > } > > > > - for (qid =3D 0; qid < q_num; qid +=3D 2) { > > + for (qid =3D 0; qid < q_num; qid +=3D 1) { > > + if ((internal->device_type =3D=3D IFCVF_NET) && (qid & 1)) > > + continue; > > ev.events =3D EPOLLIN | EPOLLPRI; > > /* leave a flag to mark it's for interrupt */ > > ev.data.u64 =3D 1 | qid << 1 | > > -- > > 1.8.3.1 >=20