From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 1EBDAA0548; Fri, 9 Sep 2022 09:55:27 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 08F3440DDC; Fri, 9 Sep 2022 09:55:27 +0200 (CEST) Received: from mga01.intel.com (mga01.intel.com [192.55.52.88]) by mails.dpdk.org (Postfix) with ESMTP id BFE3440A7E for ; Fri, 9 Sep 2022 09:55:25 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1662710126; x=1694246126; h=from:to:cc:subject:date:message-id:references: in-reply-to:content-transfer-encoding:mime-version; bh=wc2qegD/z3YV3Ddqjx2JrwN7w587ZjCIVz/J6TFISYs=; b=E82pOGMUcH+qqMM/7yBK0loC20Hdslo5rcfLIpTEcUTHi5ragk0R0GUk O8rLHQbVksFd8JdqwhlpAA/AxHFtkAgIbl3kV29Hc1KZMzHI8AQb/ZuOV NszGwc8z3asu4wnBi+mlOyMdFt7AuKMwW3pG3Akj+rdIw3DIGxyldVHFt Ddp8hgVwSd2Wfb9omt0mPZOAaKTzb48M3c6YvNcmpyF2M+7CQt+CgjEJx veLZlNTiXVSMX/N4BCegnToOVcdMRHvCSHlKbDQp28DLS9o6HWjP8cQuv t+7O6TqPnh2CP7ocv7S9w34fcQOz6MPgTCsOiS1ifq19J+vKagjedX2KW A==; X-IronPort-AV: E=McAfee;i="6500,9779,10464"; a="323627264" X-IronPort-AV: E=Sophos;i="5.93,302,1654585200"; d="scan'208";a="323627264" Received: from orsmga001.jf.intel.com ([10.7.209.18]) by fmsmga101.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 09 Sep 2022 00:55:24 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.93,302,1654585200"; d="scan'208";a="648355864" Received: from fmsmsx602.amr.corp.intel.com ([10.18.126.82]) by orsmga001.jf.intel.com with ESMTP; 09 Sep 2022 00:55:24 -0700 Received: from fmsmsx611.amr.corp.intel.com (10.18.126.91) by fmsmsx602.amr.corp.intel.com (10.18.126.82) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.31; Fri, 9 Sep 2022 00:55:24 -0700 Received: from FMSEDG603.ED.cps.intel.com (10.1.192.133) by fmsmsx611.amr.corp.intel.com (10.18.126.91) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.31 via Frontend Transport; Fri, 9 Sep 2022 00:55:24 -0700 Received: from NAM10-BN7-obe.outbound.protection.outlook.com (104.47.70.106) by edgegateway.intel.com (192.55.55.68) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.1.2375.31; Fri, 9 Sep 2022 00:55:23 -0700 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=Md8z0AQ0wMvHN1oJZ8LlfXqSU5epalZHmFkxw8jZyO4Tb1uJ4fALYo2hVy5HDK2ZLEeb3+Tqs4S/wek2rtEfPuyRurpdE9KUROmVkV/+VJJJr7wDnr8w7f1c6mcrtN6IOmB2tzdgIA9zHYQWgBWKc9R0JMpFCVCobyE4tqsA02MTomMaFwnSAMfZet/GO1UxJKYEw7yl92g6CDU4l63rkIs5h/foPNim5mpjqC6Zlnu1Oiqz8GlSBvmXC0k8CXXLg0BZeq8oHmS37Z6FijkZLVz1cceW11ytmX1UKLlrUecIA6Bmd9eCwBsCVFZMnRs1FlLqjpR+E9VN+rbS6Uu7Vw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=SRuAC0c19HZmp1HWo6iWPW+7N6DaqPJtdgx8d/e8wzw=; b=PuO5Wim5jyNOwHXkTM28s0Y9giApGO0H7d8QnPwX6XpHh02dsSM0Kc4yUdHp9us2AaYmwImn5yx0OROlOxhz3qwh9ipdjLgDZt29TngFDpysz75IN+aP1RXcWGLmXoeCKldbiosvIw0CiF9sCe5E4+LHF6nnoX7SxScI/6VVzTqw1tAXF9V19Adh9P6rFlT/nr9qyF+d6SFIA3okoRYixr+ZJbtw3clqgX/9/8FnocxFBRlKYv4HFMZweX6wZVWd+fWG+JruQ9hRlQ+Wk4fg4cCLVbknzV3ZHNqIepzaAIDuczRDKkB5ZTpCjF8t9iksA8m5kUXnuPiXknY4ugUEZg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=intel.com; dmarc=pass action=none header.from=intel.com; dkim=pass header.d=intel.com; arc=none Received: from DM5PR11MB1739.namprd11.prod.outlook.com (2603:10b6:3:111::22) by MW3PR11MB4571.namprd11.prod.outlook.com (2603:10b6:303:59::19) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5588.14; Fri, 9 Sep 2022 07:55:22 +0000 Received: from DM5PR11MB1739.namprd11.prod.outlook.com ([fe80::4cf2:7f87:f969:3ef9]) by DM5PR11MB1739.namprd11.prod.outlook.com ([fe80::4cf2:7f87:f969:3ef9%3]) with mapi id 15.20.5612.019; Fri, 9 Sep 2022 07:55:22 +0000 From: "Pei, Andy" To: "Xia, Chenbo" , "dev@dpdk.org" CC: "Xu, Rosen" , "Huang, Wei" , "Cao, Gang" , "maxime.coquelin@redhat.com" , Huang Wei Subject: RE: [PATCH v2 2/8] vdpa/ifc: add multi queue support Thread-Topic: [PATCH v2 2/8] vdpa/ifc: add multi queue support Thread-Index: AQHYw+M1wYwKzByKp0exg8Nq4TOfSK3WuOMA Date: Fri, 9 Sep 2022 07:55:21 +0000 Message-ID: References: <1661229305-240952-2-git-send-email-andy.pei@intel.com> <1662616458-164613-1-git-send-email-andy.pei@intel.com> <1662616458-164613-3-git-send-email-andy.pei@intel.com> In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: dlp-product: dlpe-windows dlp-version: 11.6.500.17 dlp-reaction: no-action authentication-results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=intel.com; x-ms-publictraffictype: Email x-ms-traffictypediagnostic: DM5PR11MB1739:EE_|MW3PR11MB4571:EE_ x-ms-office365-filtering-correlation-id: 3c7c7b0c-e6b7-49f9-da65-08da9238a580 x-ms-exchange-senderadcheck: 1 x-ms-exchange-antispam-relay: 0 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: LQJ71eVEp0wMA6zBB8fXQi+StSFdeV/9ZHjXKUTSbq2ZP0eG0t4gf0TZzoyu9rMIl4gob2ZNV4lLUZ7By1qnsUyWiAI65x1RHM892EERNa+SEGY59EjaPrqi9TkTtur/LkNBLPZRPk53psmS0xIIj/YQo6lhWfi3LAUFAjR5MD2ZrjyIS9RuansjNPJ8jA3BHya2KY4ur9Ui9WodNLPslsVh5acxax+hUzV/+BfXWkZU/wxSfQIqmr/MUPYW6zsJZmg9oOj5L6vIUdS4ZBOIc0vWRgzS8zkP7XUAA1U01s2quPzm6F9QWZUxfkyXFR3THdPFHSN87nXhhuDGxeTsPItKjaZ3dwGg2py0a9pBKmfD7M41qy3EYsWrF5lggpnOdEveH2YRZXYXDf7WV9JhtN8sgZpb82ZcGqErivDM6Qc0X6Pd0K+PcK9nzCWYLd8BT0wmVrkEfg+pngG3yIzX79Vyp0I7MuvG4S7hy5s/QUcziWcF95Q3spdh7s2HokygKUoREkhBehHNB5oN1IYZCdQMF1mA4JeZ3XaBu9w6OltPElJ9r/AuUF0SYWZtcpUbCrKCMMSOEuzS3pmDOO8LiigR1w/PIusRy3o9kaRGG+PXs9gnhLvaLMAmCCnzVAVxpKn2OXzlIgE4UqtDywFeDqIFhWxlr3BEpyIfBjlrcLQwrHqFPLe0FrjoU1wuirLeGBMXMuU2NmVLsya6CtDeObAV9JSa4QllF8vNxPOqe2c0huFsfX3r9owlpFFhfZnQJMrA8OqFR/jRbftabkaROg== x-forefront-antispam-report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:DM5PR11MB1739.namprd11.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230016)(39860400002)(376002)(346002)(136003)(366004)(396003)(8676002)(33656002)(26005)(54906003)(316002)(66476007)(52536014)(76116006)(64756008)(2906002)(110136005)(4326008)(66446008)(66946007)(5660300002)(66556008)(478600001)(53546011)(6506007)(55016003)(41300700001)(7696005)(186003)(8936002)(38100700002)(71200400001)(82960400001)(9686003)(122000001)(107886003)(86362001)(38070700005)(83380400001); DIR:OUT; SFP:1102; x-ms-exchange-antispam-messagedata-chunkcount: 1 x-ms-exchange-antispam-messagedata-0: =?us-ascii?Q?CvrTOd5/cAZ9psR6wry4NkJE5RBrejvAsGjEIaVf0uNOlMG9pzOFRfMq9GoT?= =?us-ascii?Q?zJ3OlfnsXKA+n/ky0RIC+pkV9gDx1nJJHpKEWg2VEUqaROfHcsul68goLFeu?= =?us-ascii?Q?eDRCpV+JdKwwzB9KxTNlN9kqG+1HJGserDZjCL6mQ05Xi6G0nobRMbanN9mF?= =?us-ascii?Q?pEKnNbfEqHD2Aife0fu3QfAoeo9l4bedkxe70VhyNwif2BTmsBXycVUXS4ys?= =?us-ascii?Q?2YpmgTVTxu58TOxUvgd0KQzkIFTTe82gWqqCMK/wK30We5cBLh0I8ihokOEw?= =?us-ascii?Q?mbMIoHRHqoM61eI0wcVtLu8D5AJoR5fSVRhjVbCdkre2B4vJgap0A0ayZ3cx?= =?us-ascii?Q?J/s5Bo4cXDkCOZyCcM6BvqWQqiapmb5JkdaZr/ZHKkMTN5cAd1cG9eusKY1N?= =?us-ascii?Q?B3PSBOuYBq+CaVhTIoPMq2q5PXUHVn7Kx9YrXXTx2kfikrGCPpzjRsy/W2xU?= =?us-ascii?Q?Uww2jm2kyYCAwfCnBJI75jhXg7PXxQhdzj/2us2toz+47l14TEkCHrBk0mOb?= =?us-ascii?Q?uZnMStNazAEHrrxh1rrId6lW6XjUYK2CWjsO0rp8t9Ft4DuVnHiaH4RdkhaZ?= =?us-ascii?Q?yKvzMQVRMrJEeQsuCL3feNT2DM/nsEPb6TxS0mjx4UOikQ7/cBMz27weHTpL?= =?us-ascii?Q?6+7VMIyJmp1uxWRNfRiDTaTKfdMrNp0xCPcP4Y9TTuwm9U94wkTwQborkQnw?= =?us-ascii?Q?a2yTiZ2wjYR+Bi8t3m7uAo+TVdMV+lYprlyw0cY+d8aQLTSD/f3USDHNaXNb?= =?us-ascii?Q?zImRSjNsawyUt+ZJ4mMAwOKJ1s402RFzXR02+FjwYQRKgpi+QmdnQ3Zc7C6W?= =?us-ascii?Q?fAaFdFus2QCoZhg71BnUOetJqulgBZG+VEaOn1iCrj9mZZ6wdT16Ya2tv2+e?= =?us-ascii?Q?enO8xKkb1Hjt+V1UiwNJqcWm89YODSX/QSj7RtbcGWToVuUvT+E4b70NK3m5?= =?us-ascii?Q?7LSuC6jR5Z+7gMvSbjNqBYDhYW577uj/VAAYhniAatGWC7mA+ku8fTgLlREH?= =?us-ascii?Q?Y5aNRlb7AV8JglEK94wMpAtHWuGSRqxh4bpTIZ4AmSDHA/UDz+1IPGP76ZRD?= =?us-ascii?Q?vSkVaihD8mQ9y2jM7lXYVoHipe7nZWkmoFsXKWsDuVIObhLWLoyki0QGelWz?= =?us-ascii?Q?iAjhjnMWR5oWDk5TtSAKoAyopnYrLD+6+N6m9fcBt0H068sp4bQZc2w+LB7/?= =?us-ascii?Q?UK8dCB/xBNFJ4bRusBMv80fR15FJt98DNLTzFNdCevMNkrATWoWUhaks7im9?= =?us-ascii?Q?sB5dqwgLEd/YUGa+HdCO0rCryo0Db277rhf2Fr+Zh5TEw9oUiGXjnEHQSSM8?= =?us-ascii?Q?sRL99EKP1x3Px2Ay8WlQxjfZRvf1a/Tim6KXB9d4FpMjbQBascVfSOncgvEk?= =?us-ascii?Q?T1Il6EtJ+hhxKml24ojvr1aAJ3JHpNxiJWLz1DxyLKGdGZQG/HdR713dfNs/?= =?us-ascii?Q?VrkmbhlllNrOGEmoy+Q+TH5Pk4+CslV0/NaIGdVqcLQu2MBKTF2E2cs5pb8X?= =?us-ascii?Q?H4MVYESfLPICGJMeEaGqeiNfpRHyNanu1ZOhzNnLT9d/lXxDkBImDrqINmlT?= =?us-ascii?Q?6h607VwczOaWI1BbCao=3D?= Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: DM5PR11MB1739.namprd11.prod.outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: 3c7c7b0c-e6b7-49f9-da65-08da9238a580 X-MS-Exchange-CrossTenant-originalarrivaltime: 09 Sep 2022 07:55:21.8543 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 46c98d88-e344-4ed4-8496-4ed7712e255d X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: OolPLFvaS7KvhxLOAqPDNTXldFFqUeqZt4gW/pjnc9wYAixRh6eQA6rkMujQLGpY1pbd4G9utpzgK38DavGCAw== X-MS-Exchange-Transport-CrossTenantHeadersStamped: MW3PR11MB4571 X-OriginatorOrg: intel.com X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Hi Chenbo, Thanks for your reply. My reply is inline. > -----Original Message----- > From: Xia, Chenbo > Sent: Friday, September 9, 2022 8:29 AM > To: Pei, Andy ; dev@dpdk.org > Cc: Xu, Rosen ; Huang, Wei ; Cao= , > Gang ; maxime.coquelin@redhat.com; Huang Wei > > Subject: RE: [PATCH v2 2/8] vdpa/ifc: add multi queue support >=20 > > -----Original Message----- > > From: Pei, Andy > > Sent: Thursday, September 8, 2022 1:54 PM > > To: dev@dpdk.org > > Cc: Xia, Chenbo ; Xu, Rosen > > ; Huang, Wei ; Cao, Gang > > ; maxime.coquelin@redhat.com; Huang Wei > > > > Subject: [PATCH v2 2/8] vdpa/ifc: add multi queue support >=20 > multi-queue >=20 Sure. > > > > Enable VHOST_USER_PROTOCOL_F_MQ feature. > > ExposeIFCVF_MQ_OFFSET register to enable multi queue. >=20 > Please rephase it in a better way, at least add a space before > IFCVF_MP_OFFSET... >=20 I will make it "Expose IFCVF_MQ_OFFSET register to enable multi queue." > > > > Signed-off-by: Andy Pei > > Signed-off-by: Huang Wei > > --- > > drivers/vdpa/ifc/base/ifcvf.c | 5 +++++ > > drivers/vdpa/ifc/base/ifcvf.h | 2 ++ drivers/vdpa/ifc/ifcvf_vdpa.c | > > 1 + > > 3 files changed, 8 insertions(+) > > > > diff --git a/drivers/vdpa/ifc/base/ifcvf.c > > b/drivers/vdpa/ifc/base/ifcvf.c index f1e1474..34c8226 100644 > > --- a/drivers/vdpa/ifc/base/ifcvf.c > > +++ b/drivers/vdpa/ifc/base/ifcvf.c > > @@ -90,6 +90,11 @@ > > if (!hw->lm_cfg) > > WARNINGOUT("HW support live migration not support!\n"); > > > > + if (hw->mem_resource[4].addr) > > + hw->mq_cfg =3D hw->mem_resource[4].addr + > IFCVF_MQ_OFFSET; > > + else > > + hw->mq_cfg =3D NULL; >=20 > Could you help me understand the logic here? There are two cases that BAR= 4 > mmap-able and not mmap-able? >=20 For some hardware implementation, for example the BAR 4 of PF is NULL, whil= e BAR 4 of VF is not. This code makes sure hw->mq_cfg is a valid address. I am planning a optimization to check parameter consistent at init phase. For example, if hardware does not support HW-assist live migration, we cann= ot use parameter like "sw-live-migration=3D0". > Thanks, > Chenbo >=20 > > + > > if (hw->common_cfg =3D=3D NULL || hw->notify_base =3D=3D NULL || > > hw->isr =3D=3D NULL || hw->dev_cfg =3D=3D NULL) { > > DEBUGOUT("capability incomplete\n"); diff --git > > a/drivers/vdpa/ifc/base/ifcvf.h b/drivers/vdpa/ifc/base/ifcvf.h index > > 7ede738..ad505f1 100644 > > --- a/drivers/vdpa/ifc/base/ifcvf.h > > +++ b/drivers/vdpa/ifc/base/ifcvf.h > > @@ -50,6 +50,7 @@ > > > > #define IFCVF_LM_CFG_SIZE 0x40 > > #define IFCVF_LM_RING_STATE_OFFSET 0x20 > > +#define IFCVF_MQ_OFFSET 0x28 > > > > #define IFCVF_LM_LOGGING_CTRL 0x0 > > > > @@ -149,6 +150,7 @@ struct ifcvf_hw { > > u16 *notify_base; > > u16 *notify_addr[IFCVF_MAX_QUEUES * 2]; > > u8 *lm_cfg; > > + u8 *mq_cfg; > > struct vring_info vring[IFCVF_MAX_QUEUES * 2]; > > u8 nr_vring; > > int device_type; > > diff --git a/drivers/vdpa/ifc/ifcvf_vdpa.c > > b/drivers/vdpa/ifc/ifcvf_vdpa.c index 61d0250..2d165c0 100644 > > --- a/drivers/vdpa/ifc/ifcvf_vdpa.c > > +++ b/drivers/vdpa/ifc/ifcvf_vdpa.c > > @@ -1248,6 +1248,7 @@ struct rte_vdpa_dev_info { > > 1ULL << VHOST_USER_PROTOCOL_F_SLAVE_SEND_FD | \ > > 1ULL << VHOST_USER_PROTOCOL_F_HOST_NOTIFIER | \ > > 1ULL << VHOST_USER_PROTOCOL_F_LOG_SHMFD | \ > > + 1ULL << VHOST_USER_PROTOCOL_F_MQ | \ > > 1ULL << VHOST_USER_PROTOCOL_F_STATUS) > > > > #define VDPA_BLK_PROTOCOL_FEATURES \ > > -- > > 1.8.3.1