From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 7D03CA0032; Wed, 14 Sep 2022 07:29:40 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 22FD74021D; Wed, 14 Sep 2022 07:29:40 +0200 (CEST) Received: from mga17.intel.com (mga17.intel.com [192.55.52.151]) by mails.dpdk.org (Postfix) with ESMTP id 0BF2F40141 for ; Wed, 14 Sep 2022 07:29:37 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1663133378; x=1694669378; h=from:to:cc:subject:date:message-id:references: in-reply-to:content-transfer-encoding:mime-version; bh=8yw7Dlx2CgdAOvmRiMjmk7h0wJe+QHAzUedtieapEi4=; b=NPuvP2YPTP8TFtouGs//nZz2XnCMHpPVebIKR06/KkGRl68Xtrdw52lu eU5mAlu3plgOA49n21vICgoHE3f/3euYHQMq3wFdKGESX+6Fet5tFZ9xz uNgpkO822xB2pF9ai4H2p8sTqxz0khR5QEkl7kIHGfpDCWMmaqDWvpvYw Dseyw45Y3eSeo7UyE8YGaDQnEIezSJyxDBM2KPTUQX9x00HPJBwV2TBGP 56s6ePglQ67ySbDcQE/YNU2sjzqjy2sqxXHqxJ3TzXOsTmD2B1t51Ksav ecEbPwSOK8OX7pENq4lAXDJyatQt10aAy1UqTB8s5NARL5GgUeRa52VVG Q==; X-IronPort-AV: E=McAfee;i="6500,9779,10469"; a="278726069" X-IronPort-AV: E=Sophos;i="5.93,313,1654585200"; d="scan'208";a="278726069" Received: from fmsmga004.fm.intel.com ([10.253.24.48]) by fmsmga107.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 13 Sep 2022 22:29:37 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.93,313,1654585200"; d="scan'208";a="685175108" Received: from orsmsx603.amr.corp.intel.com ([10.22.229.16]) by fmsmga004.fm.intel.com with ESMTP; 13 Sep 2022 22:29:36 -0700 Received: from orsmsx609.amr.corp.intel.com (10.22.229.22) by ORSMSX603.amr.corp.intel.com (10.22.229.16) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.31; Tue, 13 Sep 2022 22:29:36 -0700 Received: from ORSEDG602.ED.cps.intel.com (10.7.248.7) by orsmsx609.amr.corp.intel.com (10.22.229.22) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.31 via Frontend Transport; Tue, 13 Sep 2022 22:29:36 -0700 Received: from NAM12-MW2-obe.outbound.protection.outlook.com (104.47.66.40) by edgegateway.intel.com (134.134.137.103) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.1.2375.31; Tue, 13 Sep 2022 22:29:33 -0700 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=U1i+4h5r/ImKqyBj2oLYsc2f29lqIwXRjxpWzBfeOCcdaU9tLiwIevfR3/yTxVKznp0C6ivhj/sSxI18y/3gDUpxIYrcQXhx75h0DvkScgzJ0lhvado4JxcYGf4GcNMO+cwXBQ2+jcbB3UhLFKrZdeshMW2xTTkOgHmdlTVhwNDcHw7D8EePqzGz5eIepJwb4Dt6TBBKlRly5Ui9qAr8RnE5NgKZbI1a07UyPKBG0mCEJPRDgZqh2ekEx6ABreEru4bXNE/Zwvxg2QGVEpqroTj0vSDpIQ5B7My0kKBYW9Z4+OY6jpV+gn8zEdhaKGwysQ+SPrHrmuRe38jiSYNa8Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=J2f0mtRfFvEaqG51SyKdA37oKqTq9hx0N0vkyaXV/KA=; b=jYddoTDWoo218n92DR9t5rP53whCh6hAAGWLDoAl4uG6T3ZlBNckUa4MtljmiR/HPDzrau5UQ1mWNQzVACokmLNluYH4rxBeJa72fNiUmDTnvwxPXS1JiTCpuIbAHFGddtY1E57uLiEV86q6H3qwfO2fQrc9QjQRxHp7iOi0hM+knvE3BYIe8YF6Zobk5t9aSgOGYFXxPmCuDF5vuBuQrKftlfWqBkweEC3wpbId0pxVQXunDDTm9L+oD9nUOszPkPLEYGesBTce+XXwzSQrWFj1IppP+PcflqNx+19M+PYPY/lIrTvQag1ZvYvgyBUN7r/Gyix4hdy5pwJch4H69w== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=intel.com; dmarc=pass action=none header.from=intel.com; dkim=pass header.d=intel.com; arc=none Received: from DM5PR11MB1739.namprd11.prod.outlook.com (2603:10b6:3:111::22) by SN7PR11MB7137.namprd11.prod.outlook.com (2603:10b6:806:2a0::14) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5612.14; Wed, 14 Sep 2022 05:29:31 +0000 Received: from DM5PR11MB1739.namprd11.prod.outlook.com ([fe80::4cf2:7f87:f969:3ef9]) by DM5PR11MB1739.namprd11.prod.outlook.com ([fe80::4cf2:7f87:f969:3ef9%3]) with mapi id 15.20.5612.022; Wed, 14 Sep 2022 05:29:30 +0000 From: "Pei, Andy" To: "Huang, Wei" , "Xia, Chenbo" , "dev@dpdk.org" CC: "Xu, Rosen" , "Cao, Gang" , "maxime.coquelin@redhat.com" , Huang Wei Subject: RE: [PATCH v2 6/8] vdpa/ifc: support dynamic enable/disable queue Thread-Topic: [PATCH v2 6/8] vdpa/ifc: support dynamic enable/disable queue Thread-Index: AQHYx+axwBpXRW2vDUGB+69nfDICF63eZO6Q Date: Wed, 14 Sep 2022 05:29:30 +0000 Message-ID: References: <1661229305-240952-2-git-send-email-andy.pei@intel.com> <1662616458-164613-1-git-send-email-andy.pei@intel.com> <1662616458-164613-7-git-send-email-andy.pei@intel.com> In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: dlp-product: dlpe-windows dlp-version: 11.6.500.17 dlp-reaction: no-action authentication-results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=intel.com; x-ms-publictraffictype: Email x-ms-traffictypediagnostic: DM5PR11MB1739:EE_|SN7PR11MB7137:EE_ x-ms-office365-filtering-correlation-id: c32c6d51-1548-4451-ced1-08da96121967 x-ms-exchange-senderadcheck: 1 x-ms-exchange-antispam-relay: 0 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: 3wBP1mGCS0j2SNIlisv4LF9uuTb6wtL/IQD8BihLlzhbGf0/Y5T8IEmk5/l8tVDUwctFzi2fAh5hNIZrSuGMwU3JC1HiM6ZSnEU8WD9+71kfHMqUgOnJtTk1Blt5qFdQxQhPDT0i+Dlz5OhUugjVGYSaRTAssOQBCzYBeuEmSRn5cE5joiFsFHBdZnSMKRz9/YKLqWMel9h1YS7jokCOuJS5lb0453QkXUMzQOqGOCxcig7ARs+C5Tplz9mhgfx5Xjmx4ANtYS0pfpk+Eo0J9BtCa7qoWe3DVPZ0bIy3LL5j03Cp3EvHT8lO78jHg/uRQtjSIUtiFxxh48TUW7J5dfuLnmyZikBSuHuI86J6xqzF0Ec5LrgiFaBtEIjyhCJmGagDvEpDHqoaa0fmG7JM9UKyVL9x+C7PUQGZKBy+c7l1m4zY+XDCRTkcrrOnk3j82+e148w6t7NkFZUV3OmOAn6XyLwil8C6vtu0CVAv2YpZqDRelFnhnNdniqFPBif5vximelhPJgWVy3xS9KRu955Iahh/Xjcjx1lSH44IOoJ7WgWz+iwf9R7U4+otDSH8Fl+Vf0VpAMdLlyVyOiuBkeSZ8gnzKzL/oTAlpQ52E9cDo9OeNTxio4Tr5q/AD6xBhJQljf6mME28dXBLtWLgVGPAlOpYhJOp7M8Yje5zH0HO0Kmg3sCK4Tz1SvZUiLOPda+bHInjf5cox5jk7XmhSLNXMMT3RZi4VRRgPnWQF7DREhrW2V/Bekmpjf/fAMQICC995NgU+U0BpASxID+z16rlzvcm+rpuOOIt62ZufuE= x-forefront-antispam-report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:DM5PR11MB1739.namprd11.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230022)(39860400002)(376002)(346002)(136003)(396003)(366004)(451199015)(316002)(110136005)(76116006)(186003)(66556008)(26005)(66476007)(54906003)(55016003)(64756008)(53546011)(66446008)(41300700001)(6506007)(478600001)(30864003)(71200400001)(52536014)(82960400001)(8936002)(5660300002)(8676002)(66946007)(122000001)(2906002)(38100700002)(107886003)(83380400001)(9686003)(86362001)(4326008)(7696005)(33656002)(38070700005)(309714004); DIR:OUT; SFP:1102; x-ms-exchange-antispam-messagedata-chunkcount: 1 x-ms-exchange-antispam-messagedata-0: =?us-ascii?Q?iWeJ1sCHicSM1uxbPMqFFWnM8O/r0A7YM1lFlsQDKIlu/QR4N1qnrlck9X2i?= =?us-ascii?Q?22Ebs1BYuJacshv4C7I6c8KPCpGQleBPUplJcHAa6e9ShnTzY0rssqtgF3K4?= =?us-ascii?Q?DmvwC5HH6Rij44IYTT6hfZ7eqPhQhPitoAhl7qcxYu3FwVOhJDcYYsJACxFt?= =?us-ascii?Q?/3ffDhFEcdKdouA9BOAYIb/BG2lKxeISFB6vjF65iJ4Ack8HwZN+hil6vlpT?= =?us-ascii?Q?jTzYTTDv0G5CKWu+Jq32D+hDOkpbfp42zxy+KqEdYLRuqSZyxejDMU1AGy8C?= =?us-ascii?Q?QsOcOauT6NNajtF33UU9yKGcahN60ayUEqdLZ8SssYTh4uetVcJTBcW4Bglr?= =?us-ascii?Q?g1qOtj/3Bs3bPq5kql3zp+Xy8cTSR98PkLTF4ag/nLaVl97JdCL260uxyHqS?= =?us-ascii?Q?Szwtk1XzUGe9yuSpSsqAYAbDbuULKYXdh9v5OtRDt5WaKSd+0aD2pkE5zuZv?= =?us-ascii?Q?RVETzwDK+4PbfkFWCTSCYxIvb0ZMC6hF4AaXenJrg7+gxglmMvflUsK8kasq?= =?us-ascii?Q?cgM/9kxOE2USn4YFeHE2b1kggjBp/g+xKbJ8BaXE0OU87pu7r57y38elx7yB?= =?us-ascii?Q?vQ+1vxWXq4HNcYtql/9Km8vDjS/4Y7lXuRgKQ8iv9At71UGIOl0gBD6XMwvO?= =?us-ascii?Q?pomZEOzLBsJAsA32XkTBXm6VzCmjHs7nKw1suZEVT7s52KbQOUVMk3H6MKHu?= =?us-ascii?Q?UyGcMK82z9hwumxQOf7VCPah+jkDjVhKWQ2bHxBBjzym8FaHDBocUYUPpaal?= =?us-ascii?Q?gJQvj6RNWU2eR08DED+3jkKo/pg7KoeoXtBGDFlwUHjjsLOsI31QSHejqXOD?= =?us-ascii?Q?UgtIw5aHFeWCSUVl04DFMYJI4F9U0kYuXaY513KQ1aBbg97ZFM6F6DAzJS9j?= =?us-ascii?Q?oiGgmUu0jWms8I9QUpJ316Yv0p3TnClWiohYD+pzdH1dfZNKkydRhk6iFY6h?= =?us-ascii?Q?BwD38MKT7H/oxfgRHdkePtx7SD3cJBYul5tCUsOyecgHa5U1fi0qP1xW83u/?= =?us-ascii?Q?4a6ZqIZu1/QILk5uKPKLZPZoZ1ouGp6jI0EM8sYMx8qsRak3QFzT+Yn3naso?= =?us-ascii?Q?iCkzJ3ZzBrCDqG3zUqq4WZsjOZ6KPJ1d7nTPOW0LFIW0PinEMtUtu8Xjl5WN?= =?us-ascii?Q?5TtZCsb1ynXWEqwUTTYxzTFocEbG+xPI2FbqbL8P+dxVUiLy2GT+U/lj9jjN?= =?us-ascii?Q?VW754Qw0f0nPXBa40kMS6ZSVPaE7uXirGKViMb66vgQ8XSzvhUr2o+RDg7HP?= =?us-ascii?Q?e/0VQF0cwRAAe+VS1zULjKv4T2NChAP3fMAxC3wsQgUIJgO300pDX33d/R1V?= =?us-ascii?Q?rsHlXvP1BA+iubOFq9P+Rng6CEIoNeadQHq95FcCmvqHOfh5+9X8vnwDR/71?= =?us-ascii?Q?4Gj9T2rg93ilcNidhXCi7dHB9lgC0WvhqxUe+zLIEI9+K0RJKZTwawpWy4w7?= =?us-ascii?Q?uhQCK6kVMSEUfXqgr4c6S/GFTu3TWi6+80KWl4s/xpCUt3vR3hoNIz9iQU7o?= =?us-ascii?Q?V6/JSSZJhvpvLeYQOk7gYVpN+JY0SRie3L5xIpR7DATMSggDxozWvnWEW0Mq?= =?us-ascii?Q?GOJtZnYkk9NK1W4NIQaFRNqCYC+4vonwhamD/C1Z?= Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: DM5PR11MB1739.namprd11.prod.outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: c32c6d51-1548-4451-ced1-08da96121967 X-MS-Exchange-CrossTenant-originalarrivaltime: 14 Sep 2022 05:29:30.5845 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 46c98d88-e344-4ed4-8496-4ed7712e255d X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: XOtH/bb06Lk5JuBPRgLtg3Q4H6p2z8YceZQmpNIr/eLrcCNuEZP60JNxNV0QEC1wBGqcjjXFCAsMHmqAxTmsMg== X-MS-Exchange-Transport-CrossTenantHeadersStamped: SN7PR11MB7137 X-OriginatorOrg: intel.com X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Hi Chenbo,=20 See my reply inline. > -----Original Message----- > From: Huang, Wei > Sent: Wednesday, September 14, 2022 11:04 AM > To: Xia, Chenbo ; Pei, Andy ; > dev@dpdk.org > Cc: Xu, Rosen ; Cao, Gang ; > maxime.coquelin@redhat.com; Huang Wei > Subject: RE: [PATCH v2 6/8] vdpa/ifc: support dynamic enable/disable queu= e >=20 >=20 >=20 > > -----Original Message----- > > From: Xia, Chenbo > > Sent: Wednesday, September 14, 2022 10:23 > > To: Pei, Andy ; dev@dpdk.org > > Cc: Xu, Rosen ; Huang, Wei ; > > Cao, Gang ; maxime.coquelin@redhat.com; Huang Wei > > > > Subject: RE: [PATCH v2 6/8] vdpa/ifc: support dynamic enable/disable > > queue > > > > > -----Original Message----- > > > From: Pei, Andy > > > Sent: Thursday, September 8, 2022 1:54 PM > > > To: dev@dpdk.org > > > Cc: Xia, Chenbo ; Xu, Rosen > > > ; Huang, Wei ; Cao, Gang > > > ; maxime.coquelin@redhat.com; Huang Wei > > > > > > Subject: [PATCH v2 6/8] vdpa/ifc: support dynamic enable/disable > > > queue > > > > > > From: Huang Wei > > > > > > Support dynamic enable or disable queue. > > > For front end, like QEMU, user can use ethtool to configurate queue. > > > For example, "ethtool -L eth0 combined 3" to enable 3 queues pairs. > > > > > > Signed-off-by: Huang Wei > > > Signed-off-by: Andy Pei > > > --- > > > drivers/vdpa/ifc/base/ifcvf.c | 101 > > > ++++++++++++++++++++++++++++++++++++ > > > drivers/vdpa/ifc/base/ifcvf.h | 6 +++ > > > drivers/vdpa/ifc/base/ifcvf_osdep.h | 1 + > > > drivers/vdpa/ifc/ifcvf_vdpa.c | 93 ++++++++++++++++++++++++++= +---- > > > -- > > > 4 files changed, 186 insertions(+), 15 deletions(-) > > > > > > diff --git a/drivers/vdpa/ifc/base/ifcvf.c > > > b/drivers/vdpa/ifc/base/ifcvf.c index 4875ea1..34f32f8 100644 > > > --- a/drivers/vdpa/ifc/base/ifcvf.c > > > +++ b/drivers/vdpa/ifc/base/ifcvf.c > > > @@ -230,6 +230,107 @@ > > > } > > > } > > > > > > +int > > > +ifcvf_enable_vring_hw(struct ifcvf_hw *hw, int i) { > > > + struct ifcvf_pci_common_cfg *cfg; > > > + u8 *lm_cfg; > > > + u16 notify_off; > > > + int msix_vector; > > > + > > > + if (!hw || (i >=3D (int)hw->nr_vring)) > > > + return -1; > > > > Seems HW will always be not NULL > As a external function, we should not assume the input argument is always= valid. > > > > > + > > > + cfg =3D hw->common_cfg; > > > + if (!cfg) { > > > + ERROUT("common_cfg in HW is NULL.\n"); > > > > I am thinking why you introduce this new log? Why not just use DRV_LOG > > that is already defined? > Because below type of log macros are already defined and used in original= code, > I just follow its tradition. > #define WARNINGOUT(S, args...) RTE_LOG(WARNING, PMD, S, ##args) > #define DEBUGOUT(S, args...) RTE_LOG(DEBUG, PMD, S, ##args) > #define ERROUT(S, args...) RTE_LOG(ERR, PMD, S, ##args) > > > > > + return -1; > > > + } > > > + > > > + ifcvf_enable_multiqueue(hw); > > > + > > > + IFCVF_WRITE_REG16(i, &cfg->queue_select); > > > + msix_vector =3D IFCVF_READ_REG16(&cfg->queue_msix_vector); > > > + if (msix_vector !=3D (i + 1)) { > > > + IFCVF_WRITE_REG16(i + 1, &cfg->queue_msix_vector); > > > + msix_vector =3D IFCVF_READ_REG16(&cfg->queue_msix_vector); > > > + if (msix_vector =3D=3D IFCVF_MSI_NO_VECTOR) { > > > + ERROUT("queue %u, msix vec alloc failed\n", i); > > > + return -1; > > > + } > > > + } > > > + > > > + io_write64_twopart(hw->vring[i].desc, &cfg->queue_desc_lo, > > > + &cfg->queue_desc_hi); > > > + io_write64_twopart(hw->vring[i].avail, &cfg->queue_avail_lo, > > > + &cfg->queue_avail_hi); > > > + io_write64_twopart(hw->vring[i].used, &cfg->queue_used_lo, > > > + &cfg->queue_used_hi); > > > + IFCVF_WRITE_REG16(hw->vring[i].size, &cfg->queue_size); > > > + > > > + lm_cfg =3D hw->lm_cfg; > > > + if (lm_cfg) { > > > + if (hw->device_type =3D=3D IFCVF_BLK) > > > + *(u32 *)(lm_cfg + IFCVF_LM_RING_STATE_OFFSET + > > > + i * IFCVF_LM_CFG_SIZE) =3D > > > + (u32)hw->vring[i].last_avail_idx | > > > + ((u32)hw->vring[i].last_used_idx << 16); > > > + else > > > + *(u32 *)(lm_cfg + IFCVF_LM_RING_STATE_OFFSET + > > > + (i / 2) * IFCVF_LM_CFG_SIZE + > > > + (i % 2) * 4) =3D > > > + (u32)hw->vring[i].last_avail_idx | > > > + ((u32)hw->vring[i].last_used_idx << 16); > > > + } > > > > So the register layout is different for blk and net? > That's sure. For the register layout differences, the story is as follow: When I add support for blk device, I tried to re-use the existing code. However, the BAR4 layout of blk HW is different. I keep code for net device unchanged, and add code for blk device. > > > > > + > > > + notify_off =3D IFCVF_READ_REG16(&cfg->queue_notify_off); > > > + hw->notify_addr[i] =3D (void *)((u8 *)hw->notify_base + > > > + notify_off * hw->notify_off_multiplier); > > > + IFCVF_WRITE_REG16(1, &cfg->queue_enable); > > > + > > > + return 0; > > > +} > > > + > > > +void > > > +ifcvf_disable_vring_hw(struct ifcvf_hw *hw, int i) { > > > + struct ifcvf_pci_common_cfg *cfg; > > > + u32 ring_state; > > > + u8 *lm_cfg; > > > + > > > + if (!hw || (i >=3D (int)hw->nr_vring)) > > > + return; > > > + > > > + cfg =3D hw->common_cfg; > > > + if (!cfg) { > > > + ERROUT("common_cfg in HW is NULL.\n"); > > > + return; > > > + } > > > + > > > + IFCVF_WRITE_REG16(i, &cfg->queue_select); > > > + IFCVF_WRITE_REG16(0, &cfg->queue_enable); > > > + > > > + lm_cfg =3D hw->lm_cfg; > > > + if (lm_cfg) { > > > + if (hw->device_type =3D=3D IFCVF_BLK) > > > + ring_state =3D *(u32 *)(lm_cfg + > > > + IFCVF_LM_RING_STATE_OFFSET + > > > + i * IFCVF_LM_CFG_SIZE); > > > + else > > > + ring_state =3D *(u32 *)(lm_cfg + > > > + IFCVF_LM_RING_STATE_OFFSET + > > > + (i / 2) * IFCVF_LM_CFG_SIZE + > > > + (i % 2) * 4); > > > + > > > + if (hw->device_type =3D=3D IFCVF_BLK) > > > + hw->vring[i].last_avail_idx =3D > > > + (u16)(ring_state & IFCVF_16_BIT_MASK); > > > + else > > > + hw->vring[i].last_avail_idx =3D (u16)(ring_state >> 16); > > > > Above two if-else should be combined. > It's a good suggestion. > > > > Thanks, > > Chenbo > > > > > + hw->vring[i].last_used_idx =3D (u16)(ring_state >> 16); > > > + } > > > +} > > > + > > > STATIC int > > > ifcvf_hw_enable(struct ifcvf_hw *hw) { diff --git > > > a/drivers/vdpa/ifc/base/ifcvf.h b/drivers/vdpa/ifc/base/ifcvf.h > > > index c17bf2a..e67d4e8 100644 > > > --- a/drivers/vdpa/ifc/base/ifcvf.h > > > +++ b/drivers/vdpa/ifc/base/ifcvf.h > > > @@ -164,6 +164,12 @@ struct ifcvf_hw { ifcvf_get_features(struct > > > ifcvf_hw *hw); > > > > > > int > > > +ifcvf_enable_vring_hw(struct ifcvf_hw *hw, int i); > > > + > > > +void > > > +ifcvf_disable_vring_hw(struct ifcvf_hw *hw, int i); > > > + > > > +int > > > ifcvf_start_hw(struct ifcvf_hw *hw); > > > > > > void > > > diff --git a/drivers/vdpa/ifc/base/ifcvf_osdep.h > > > b/drivers/vdpa/ifc/base/ifcvf_osdep.h > > > index 3d56769..4a1bfec 100644 > > > --- a/drivers/vdpa/ifc/base/ifcvf_osdep.h > > > +++ b/drivers/vdpa/ifc/base/ifcvf_osdep.h > > > @@ -16,6 +16,7 @@ > > > > > > #define WARNINGOUT(S, args...) RTE_LOG(WARNING, PMD, S, ##args) > > > #define DEBUGOUT(S, args...) RTE_LOG(DEBUG, PMD, S, ##args) > > > +#define ERROUT(S, args...) RTE_LOG(ERR, PMD, S, ##args) > > > #define STATIC static > > > > > > #define msec_delay(x) rte_delay_us_sleep(1000 * (x)) > > > diff --git a/drivers/vdpa/ifc/ifcvf_vdpa.c > > > b/drivers/vdpa/ifc/ifcvf_vdpa.c index 48f1a89..16fd0fd 100644 > > > --- a/drivers/vdpa/ifc/ifcvf_vdpa.c > > > +++ b/drivers/vdpa/ifc/ifcvf_vdpa.c > > > @@ -1288,13 +1288,59 @@ struct rte_vdpa_dev_info { } > > > > > > static int > > > +ifcvf_config_vring(struct ifcvf_internal *internal, int vring) { > > > + struct ifcvf_hw *hw =3D &internal->hw; > > > + int vid =3D internal->vid; > > > + struct rte_vhost_vring vq; > > > + uint64_t gpa; > > > + > > > + if (hw->vring[vring].enable) { > > > + rte_vhost_get_vhost_vring(vid, vring, &vq); > > > + gpa =3D hva_to_gpa(vid, (uint64_t)(uintptr_t)vq.desc); > > > + if (gpa =3D=3D 0) { > > > + DRV_LOG(ERR, "Fail to get GPA for descriptor ring."); > > > + return -1; > > > + } > > > + hw->vring[vring].desc =3D gpa; > > > + > > > + gpa =3D hva_to_gpa(vid, (uint64_t)(uintptr_t)vq.avail); > > > + if (gpa =3D=3D 0) { > > > + DRV_LOG(ERR, "Fail to get GPA for available ring."); > > > + return -1; > > > + } > > > + hw->vring[vring].avail =3D gpa; > > > + > > > + gpa =3D hva_to_gpa(vid, (uint64_t)(uintptr_t)vq.used); > > > + if (gpa =3D=3D 0) { > > > + DRV_LOG(ERR, "Fail to get GPA for used ring."); > > > + return -1; > > > + } > > > + hw->vring[vring].used =3D gpa; > > > + > > > + hw->vring[vring].size =3D vq.size; > > > + rte_vhost_get_vring_base(vid, vring, > > > + &hw->vring[vring].last_avail_idx, > > > + &hw->vring[vring].last_used_idx); > > > + ifcvf_enable_vring_hw(&internal->hw, vring); > > > + } else { > > > + ifcvf_disable_vring_hw(&internal->hw, vring); > > > + rte_vhost_set_vring_base(vid, vring, > > > + hw->vring[vring].last_avail_idx, > > > + hw->vring[vring].last_used_idx); > > > + } > > > + > > > + return 0; > > > +} > > > + > > > +static int > > > ifcvf_set_vring_state(int vid, int vring, int state) { > > > struct rte_vdpa_device *vdev; > > > struct internal_list *list; > > > struct ifcvf_internal *internal; > > > struct ifcvf_hw *hw; > > > - struct ifcvf_pci_common_cfg *cfg; > > > + bool enable =3D !!state; > > > int ret =3D 0; > > > > > > vdev =3D rte_vhost_get_vdpa_device(vid); @@ -1304,6 +1350,9 @@ > > struct > > > rte_vdpa_dev_info { > > > return -1; > > > } > > > > > > + DRV_LOG(INFO, "%s queue %d of vDPA device %s", > > > + enable ? "enable" : "disable", vring, vdev->device->name); > > > + > > > internal =3D list->internal; > > > if (vring < 0 || vring >=3D internal->max_queues * 2) { > > > DRV_LOG(ERR, "Vring index %d not correct", vring); @@ - > > 1311,27 > > > +1360,41 @@ struct rte_vdpa_dev_info { > > > } > > > > > > hw =3D &internal->hw; > > > + hw->vring[vring].enable =3D enable; > > > + > > > if (!internal->configured) > > > - goto exit; > > > + return 0; > > > > > > - cfg =3D hw->common_cfg; > > > - IFCVF_WRITE_REG16(vring, &cfg->queue_select); > > > - IFCVF_WRITE_REG16(!!state, &cfg->queue_enable); > > > + unset_notify_relay(internal); > > > > > > - if (!state && hw->vring[vring].enable) { > > > - ret =3D vdpa_disable_vfio_intr(internal); > > > - if (ret) > > > - return ret; > > > + ret =3D vdpa_enable_vfio_intr(internal, false); > > > + if (ret) { > > > + DRV_LOG(ERR, "failed to set vfio interrupt of vDPA device %s", > > > + vdev->device->name); > > > + return ret; > > > } > > > > > > - if (state && !hw->vring[vring].enable) { > > > - ret =3D vdpa_enable_vfio_intr(internal, false); > > > - if (ret) > > > - return ret; > > > + ret =3D ifcvf_config_vring(internal, vring); > > > + if (ret) { > > > + DRV_LOG(ERR, "failed to configure queue %d of vDPA > > device %s", > > > + vring, vdev->device->name); > > > + return ret; > > > + } > > > + > > > + ret =3D setup_notify_relay(internal); > > > + if (ret) { > > > + DRV_LOG(ERR, "failed to setup notify relay of vDPA device %s", > > > + vdev->device->name); > > > + return ret; > > > + } > > > + > > > + ret =3D rte_vhost_host_notifier_ctrl(vid, vring, enable); > > > + if (ret) { > > > + DRV_LOG(ERR, "vDPA device %s queue %d host notifier ctrl fail", > > > + vdev->device->name, vring); > > > + return ret; > > > } > > > > > > -exit: > > > - hw->vring[vring].enable =3D !!state; > > > return 0; > > > } > > > > > > -- > > > 1.8.3.1