From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 53A03A0547; Wed, 12 Oct 2022 10:19:57 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 4345E42FCE; Wed, 12 Oct 2022 10:19:57 +0200 (CEST) Received: from mga11.intel.com (mga11.intel.com [192.55.52.93]) by mails.dpdk.org (Postfix) with ESMTP id 0DB9842FCF for ; Wed, 12 Oct 2022 10:19:54 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1665562795; x=1697098795; h=from:to:cc:subject:date:message-id:references: in-reply-to:content-transfer-encoding:mime-version; bh=eNzFDwrgzyeLUQmW4njKxFdfYisM3zPEPViQX/0cY0c=; b=VctU7IkATjFVCTlXVTFeLCXTms/ka0+Kt3qaDZKD6sNVMsQuZnxY/s4y ytAMXovUxbjcBSKnSt5R5vqSx3zTO3P9n59Hm3AdRXLfriBZdOfIJNLY8 uWim4+YcrNJ8WuVIkchRcrL5Jv/jxlWjkpezqaEsiJLfC2WZYSNZAVUZA je7rxBykw8I9YSxYXcrgEa8kDEtc3VmEglJLz0eOYZrQt/3OrS6taSZF3 UnzZyMs5lrykFCVmIXplouxB2Vo0E7/X3nihAv9Cpnn+m8UsVTuQN+Pew bxQNS90L1G1NjWpJSBFlIREew0P+/jcJnyw0DI/mfXFtDHYWuu4T6uyIe g==; X-IronPort-AV: E=McAfee;i="6500,9779,10497"; a="302345009" X-IronPort-AV: E=Sophos;i="5.95,178,1661842800"; d="scan'208";a="302345009" Received: from fmsmga005.fm.intel.com ([10.253.24.32]) by fmsmga102.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 12 Oct 2022 01:19:54 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6500,9779,10497"; a="955664151" X-IronPort-AV: E=Sophos;i="5.95,178,1661842800"; d="scan'208";a="955664151" Received: from fmsmsx601.amr.corp.intel.com ([10.18.126.81]) by fmsmga005.fm.intel.com with ESMTP; 12 Oct 2022 01:19:54 -0700 Received: from fmsmsx607.amr.corp.intel.com (10.18.126.87) by fmsmsx601.amr.corp.intel.com (10.18.126.81) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.31; Wed, 12 Oct 2022 01:19:53 -0700 Received: from fmsmsx607.amr.corp.intel.com (10.18.126.87) by fmsmsx607.amr.corp.intel.com (10.18.126.87) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.31; Wed, 12 Oct 2022 01:19:53 -0700 Received: from FMSEDG603.ED.cps.intel.com (10.1.192.133) by fmsmsx607.amr.corp.intel.com (10.18.126.87) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.31 via Frontend Transport; Wed, 12 Oct 2022 01:19:53 -0700 Received: from NAM10-DM6-obe.outbound.protection.outlook.com (104.47.58.100) by edgegateway.intel.com (192.55.55.68) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.1.2375.31; Wed, 12 Oct 2022 01:19:53 -0700 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=ahWAF85UJTwSpt2INiWJzvc8U0RpQx2h/FTjQ7RXfpYKJm5Pw5OaBCLj702bd5Ao3KOO0ggLd1eBS5L+Zzd+bAwyWcEzy0ZwCJ1YeepMweTnyUuW4xxG/NeVazHhAwAYJmfravyg2SJKn/SxKbV6Dgui0RIvV37D6Z92MO5dSIwexeaons89ysmqLLfbQce2u4fVQXIyiq0ls0O2NIm+YwqSqFZg7HnEU+iDrVf5Kg7Yiydl8epOxO7rRazYGD+98zsh7ffskuvHEN+FLjnEHaX+6p5noKkgpKQkqQIDmjtAZrRMBNFd009Z8soO+4P9TICJI13HuSWtM/SQIwnbOQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=Fp+0GCzl4SQPweKmvjAHO2uZ0q9qZLPlWwOI2NOs3Vo=; b=cnnEEeFdvgbYlRJgItKnRiqMWNfpt/GrPUMtuTYY5phoL06SWcD05updj+MDnoUu+6DqYvkKKZIubl3sn7idEWg67Zpqx0bgjSvrnp1uTqiR8iDMUrt/Pifp00KLmccFTLgdMGzc3HWYzr7z1Wc5XvufhUSCDSKYwNt3vsB2bgU7841kAieELZDpTtuHN/d6ICWVH7UUf3Xw5+XHRzmHeSO06MauxBw8BNAA4mIHeDZ3XXE3SqEKT0guDJ/aVuHUDi8is3UPk5/DyZhXNmp22jQ07WOfk2abPbvvSIAjSI4XKVcsoKhM4qYRYVBOp0bnWE0amKxMiRQJBqLuIfGbZQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=intel.com; dmarc=pass action=none header.from=intel.com; dkim=pass header.d=intel.com; arc=none Received: from SN6PR11MB3504.namprd11.prod.outlook.com (2603:10b6:805:d0::17) by MW4PR11MB5935.namprd11.prod.outlook.com (2603:10b6:303:18a::15) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5676.32; Wed, 12 Oct 2022 08:19:51 +0000 Received: from SN6PR11MB3504.namprd11.prod.outlook.com ([fe80::39bf:57b1:4824:d40d]) by SN6PR11MB3504.namprd11.prod.outlook.com ([fe80::39bf:57b1:4824:d40d%6]) with mapi id 15.20.5709.022; Wed, 12 Oct 2022 08:19:51 +0000 From: "Xia, Chenbo" To: "Pei, Andy" , "dev@dpdk.org" CC: "Xu, Rosen" , "Huang, Wei" , "Cao, Gang" , "maxime.coquelin@redhat.com" Subject: RE: [PATCH v3 6/8] vdpa/ifc: support dynamic enable/disable queue Thread-Topic: [PATCH v3 6/8] vdpa/ifc: support dynamic enable/disable queue Thread-Index: AQHYyZPuQTJqMbvZWECb7Z+oG9bAra4KkyIQ Date: Wed, 12 Oct 2022 08:19:51 +0000 Message-ID: References: <1661229305-240952-2-git-send-email-andy.pei@intel.com> <1663308990-621-1-git-send-email-andy.pei@intel.com> <1663308990-621-7-git-send-email-andy.pei@intel.com> In-Reply-To: <1663308990-621-7-git-send-email-andy.pei@intel.com> Accept-Language: en-US, zh-CN Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: dlp-product: dlpe-windows dlp-reaction: no-action dlp-version: 11.6.500.17 authentication-results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=intel.com; x-ms-publictraffictype: Email x-ms-traffictypediagnostic: SN6PR11MB3504:EE_|MW4PR11MB5935:EE_ x-ms-office365-filtering-correlation-id: 06d20db3-2302-4b31-bc0f-08daac2a890c x-ms-exchange-senderadcheck: 1 x-ms-exchange-antispam-relay: 0 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: IZvsYFvH+cFmgMiYmHrPj7o2DbMxjND9c8D5H0KzVSh/cx7xbWz5PmVnrMPGrZpXvPhD7uiQ8TL7GPO/VS285xOMbqho6HKKILN9yRsSbAskU9j04uH8jcCPJN1Bf5zwtO/Gks6qlzS+/SWy4614vhVwiZ43L3bEhKX+dBkUkWXJmMrS2OOKUVfIUrgYvbMiAfoicTaboQOrR3s221dvRav/CLJ5h/lS+wTO778JSeY1Sj24VD4XGCu4I4pIN9qkOAI3e/iDGkpo1br7tD8OxYvuzNucgqEq7aoVksVwXivNSyUOgfun8fpTXfwtD4wtKwWzhmFITMW7W1ib1gpuvsAeQf+wkfyfFrRdpKtlXouNQTldEMXiI3H17TDarAKOEyx2f/AounFRbOk61liaMmIbu0d2VwDGXutFKTxTmLiyMltS0UXFfz39IedoMDfuDP9CDdG5Mrayc1Pyh6p2UW88ClbGbJ2Ar21sgFLxEbu34gapFurHiADY1aLV7ofwcIZEEWsfaYrek4k3D7Cu0NcS5T9VlqUChsTgyiXRzFKY0+Jww2AuF1UNfspqTNmhvc0EJBmrHDK1D5C6Z6XVKd29T/Pf37kXj1W1VtCOGJ6oV2eVU3r41NQ3hKPKo0HSNVcDNwTOndUOm5uRuTJOC9wmfud8sz6KaMumZp7893ChZS14yvaOooAqcnmxVmHHEcIw+P1FQALOyvd1rBWNvIQYPB4U1g7KDllAhumjN6eG4MQRoHywPiY3pTVWIIowcPwNPzr961ySSUR9bY4ijumxLHdsZ8oT5e+h7bQxRfE= x-forefront-antispam-report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:SN6PR11MB3504.namprd11.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230022)(136003)(366004)(396003)(376002)(39860400002)(346002)(451199015)(7696005)(82960400001)(478600001)(316002)(55016003)(38100700002)(53546011)(122000001)(5660300002)(186003)(52536014)(83380400001)(26005)(9686003)(8936002)(41300700001)(54906003)(2906002)(4326008)(66476007)(76116006)(110136005)(6506007)(8676002)(86362001)(71200400001)(66946007)(38070700005)(66446008)(64756008)(66556008)(33656002)(309714004); DIR:OUT; SFP:1102; x-ms-exchange-antispam-messagedata-chunkcount: 1 x-ms-exchange-antispam-messagedata-0: =?us-ascii?Q?hAPJQgVGvUBLl+T4Uaei3vTT8WRd53VkoN8VVK9NV6RCWnxLYpwiubSBIoDQ?= =?us-ascii?Q?3yJbDEwtd2Edv5YEx1pyt6iBsJLdH4nF0BE8QKpBPfOhnEGdB/Gx/2parO4M?= =?us-ascii?Q?B/dD/fclBCc6kH0m6Zbq28J8B8KxCjjPMX7zxb6g+vQt/TqmtC4oiZi+7Rcr?= =?us-ascii?Q?BlCIGDVu2LnvLRQUlzGFHY9Ow4ji5EquvyYKM1EPl8S6WsFAuKRcYI+PqtGz?= =?us-ascii?Q?Ube3T9zo6bZ0tqI7iBDmf9Knr3/SYlFGhrX22q10Cq/B/fqslOzlUQEPnwDg?= =?us-ascii?Q?FIWcvc9Kjch2rp00mO91Ax2GOfuTMZf1QlquU7uVwv8e2DRzCrGwd+4NDzgY?= =?us-ascii?Q?++m6J0C7b9UdcSCYAHLkksflqz9Vw1704Io87N/D7OWQcPsIWsH5Crs34+Ni?= =?us-ascii?Q?wD/yYUo6p5rVZ8NFqgOhAPeOjEePzg0TYYTvvg5q6HuNUU6B/Bba0+Fq7fRf?= =?us-ascii?Q?u2GWJqUz9kNCFrczKsR22DyLTXiakYH3a1IA3xBElzNpA0BSMyGtfFu7hXjT?= =?us-ascii?Q?JbUfqWfR0sB0lgjBpmMgYEyKU7kSoEE+bM2wE6YSQtDiU5IXoSgOPrOWSVDV?= =?us-ascii?Q?TmEZPxgbmV7NGzERdBKlha9Iv9pdLwkH2dmbkTA8qzVns8/xGKFcx4H3+azm?= =?us-ascii?Q?icYm/hCNFsZQy0BR9ppoFEsdXjmDu04LDFDaCq9eGrqf+yruIdTJhffG8sF6?= =?us-ascii?Q?yw2zdjdCjSh9Xq+3Wn6mEekPIK6jx5nr84mv3CWbXXv+xO3HNT8KS6JbHWAQ?= =?us-ascii?Q?BNwkGnIVIYtq9vq4IYsrrBFBYa2GoGF0LRR83ef2pf8viZYO8jyd2PNrwrLk?= =?us-ascii?Q?c4fWd3JgCUTtDu1QHx4o7hUszBhcllIiNW2c+en23ZTHP6a98Z/FRpgQ5FkJ?= =?us-ascii?Q?xwnUe5bgGleGewZqlr5RHoHYODJi13xRXMuBSjo7f2WhvtpA/ee7Cwgf0Wtx?= =?us-ascii?Q?rc9YWjMbwbj8oHRSnxR1MH22nMZOuZJxYn2jpn5owlK/nmlvUOooJ239JKfp?= =?us-ascii?Q?SpKCv1Enlpwh7KSlLbC9BxYoxlcVxR6a2pBcU09Oh+MY4dRqRpbN9ijgpva3?= =?us-ascii?Q?Ll3+wqVj8CVBsneM4MXjV8jM4QIuC8OJoi2WwnsQH0Y6wL0Cx4Q3IUZcnQGa?= =?us-ascii?Q?j5GXc3EIl8TTwAOzd9Egooc6OZzsa/g2oSXA3C4azBb2ObMKFYQP7y1O5kUL?= =?us-ascii?Q?vdpf9L/+7kjdPZbc3J1QoiMAUofS04fTCTWFtwnyGTrWRiZ/p9GBoIhdKam+?= =?us-ascii?Q?jF6XCAZEpWBCByt62ONEREudCNux1uh/fQgSr1brzVuS+Ky8NNEnTFcVw/kk?= =?us-ascii?Q?tMpV2eVwR62xTTG+1VsXtb54zSYUy+JwOeH24SsaIqVAYEKTRSKnY7b65gDc?= =?us-ascii?Q?L7Yw9RBCy03pgxfFpAX1iF1CWi1HrkKHrac4QjU0Qn2MbzlCwRCfst2dqhaI?= =?us-ascii?Q?GNWyrdGZMyAucn8JQOpM6X9u5Py2M9zJlIrMNzNYC1gy3ggnUwtZ++8Befe/?= =?us-ascii?Q?gUbX9Ux1ugotxUbALftYvjXl9AsZxt5Bb24hCjwXJg6ZMqXLslalP59GAg03?= =?us-ascii?Q?qa6RWhUGOx1XM/MsQd5MgrZGpEKIySCugtAYXIOx?= Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: SN6PR11MB3504.namprd11.prod.outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: 06d20db3-2302-4b31-bc0f-08daac2a890c X-MS-Exchange-CrossTenant-originalarrivaltime: 12 Oct 2022 08:19:51.3794 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 46c98d88-e344-4ed4-8496-4ed7712e255d X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: g4AxotZL8dt6M2nQLnBX0KO22hwQa8L3ePNUseoIabQjYDNLbuxQrz0JehoUptnEv6SigWB7f/KkL4s1xVM9DA== X-MS-Exchange-Transport-CrossTenantHeadersStamped: MW4PR11MB5935 X-OriginatorOrg: intel.com X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org > -----Original Message----- > From: Pei, Andy > Sent: Friday, September 16, 2022 2:16 PM > To: dev@dpdk.org > Cc: Xia, Chenbo ; Xu, Rosen ; > Huang, Wei ; Cao, Gang ; > maxime.coquelin@redhat.com > Subject: [PATCH v3 6/8] vdpa/ifc: support dynamic enable/disable queue >=20 > From: Huang Wei >=20 > Support dynamic enable or disable queue. > For front end, like QEMU, user can use ethtool to configurate queue. configure > For example, "ethtool -L eth0 combined 3" to enable 3 queues pairs. >=20 > Signed-off-by: Huang Wei > Signed-off-by: Andy Pei > --- > drivers/vdpa/ifc/base/ifcvf.c | 99 > +++++++++++++++++++++++++++++++++++++++++++ > drivers/vdpa/ifc/base/ifcvf.h | 6 +++ > drivers/vdpa/ifc/ifcvf_vdpa.c | 93 +++++++++++++++++++++++++++++++++----= - > -- > 3 files changed, 183 insertions(+), 15 deletions(-) >=20 > diff --git a/drivers/vdpa/ifc/base/ifcvf.c b/drivers/vdpa/ifc/base/ifcvf.= c > index 619b034..792d258 100644 > --- a/drivers/vdpa/ifc/base/ifcvf.c > +++ b/drivers/vdpa/ifc/base/ifcvf.c > @@ -227,6 +227,105 @@ > } > } >=20 > +int > +ifcvf_enable_vring_hw(struct ifcvf_hw *hw, int i) > +{ > + struct ifcvf_pci_common_cfg *cfg; > + u8 *lm_cfg; > + u16 notify_off; > + int msix_vector; > + > + if (i >=3D (int)hw->nr_vring) > + return -1; > + > + cfg =3D hw->common_cfg; > + if (!cfg) { > + WARNINGOUT("common_cfg in HW is NULL.\n"); This should be error log > + return -1; > + } > + > + ifcvf_enable_multiqueue(hw); > + > + IFCVF_WRITE_REG16(i, &cfg->queue_select); > + msix_vector =3D IFCVF_READ_REG16(&cfg->queue_msix_vector); > + if (msix_vector !=3D (i + 1)) { > + IFCVF_WRITE_REG16(i + 1, &cfg->queue_msix_vector); > + msix_vector =3D IFCVF_READ_REG16(&cfg->queue_msix_vector); > + if (msix_vector =3D=3D IFCVF_MSI_NO_VECTOR) { > + WARNINGOUT("queue %u, msix vec alloc failed\n", i); Ditto. And %u -> %d Same for the function ifcvf_disable_vring_hw. Thanks, Chenbo > + return -1; > + } > + } > + > + io_write64_twopart(hw->vring[i].desc, &cfg->queue_desc_lo, > + &cfg->queue_desc_hi); > + io_write64_twopart(hw->vring[i].avail, &cfg->queue_avail_lo, > + &cfg->queue_avail_hi); > + io_write64_twopart(hw->vring[i].used, &cfg->queue_used_lo, > + &cfg->queue_used_hi); > + IFCVF_WRITE_REG16(hw->vring[i].size, &cfg->queue_size); > + > + lm_cfg =3D hw->lm_cfg; > + if (lm_cfg) { > + if (hw->device_type =3D=3D IFCVF_BLK) > + *(u32 *)(lm_cfg + IFCVF_LM_RING_STATE_OFFSET + > + i * IFCVF_LM_CFG_SIZE) =3D > + (u32)hw->vring[i].last_avail_idx | > + ((u32)hw->vring[i].last_used_idx << 16); > + else > + *(u32 *)(lm_cfg + IFCVF_LM_RING_STATE_OFFSET + > + (i / 2) * IFCVF_LM_CFG_SIZE + > + (i % 2) * 4) =3D > + (u32)hw->vring[i].last_avail_idx | > + ((u32)hw->vring[i].last_used_idx << 16); > + } > + > + notify_off =3D IFCVF_READ_REG16(&cfg->queue_notify_off); > + hw->notify_addr[i] =3D (void *)((u8 *)hw->notify_base + > + notify_off * hw->notify_off_multiplier); > + IFCVF_WRITE_REG16(1, &cfg->queue_enable); > + > + return 0; > +} > + > +void > +ifcvf_disable_vring_hw(struct ifcvf_hw *hw, int i) > +{ > + struct ifcvf_pci_common_cfg *cfg; > + u32 ring_state; > + u8 *lm_cfg; > + > + if (i >=3D (int)hw->nr_vring) > + return; > + > + cfg =3D hw->common_cfg; > + if (!cfg) { > + WARNINGOUT("common_cfg in HW is NULL.\n"); > + return; > + } > + > + IFCVF_WRITE_REG16(i, &cfg->queue_select); > + IFCVF_WRITE_REG16(0, &cfg->queue_enable); > + > + lm_cfg =3D hw->lm_cfg; > + if (lm_cfg) { > + if (hw->device_type =3D=3D IFCVF_BLK) { > + ring_state =3D *(u32 *)(lm_cfg + > + IFCVF_LM_RING_STATE_OFFSET + > + i * IFCVF_LM_CFG_SIZE); > + hw->vring[i].last_avail_idx =3D > + (u16)(ring_state & IFCVF_16_BIT_MASK); > + } else { > + ring_state =3D *(u32 *)(lm_cfg + > + IFCVF_LM_RING_STATE_OFFSET + > + (i / 2) * IFCVF_LM_CFG_SIZE + > + (i % 2) * 4); > + hw->vring[i].last_avail_idx =3D (u16)(ring_state >> 16); > + } > + hw->vring[i].last_used_idx =3D (u16)(ring_state >> 16); > + } > +} > + > STATIC int > ifcvf_hw_enable(struct ifcvf_hw *hw) > { > diff --git a/drivers/vdpa/ifc/base/ifcvf.h b/drivers/vdpa/ifc/base/ifcvf.= h > index 1e133c0..3726da7 100644 > --- a/drivers/vdpa/ifc/base/ifcvf.h > +++ b/drivers/vdpa/ifc/base/ifcvf.h > @@ -164,6 +164,12 @@ struct ifcvf_hw { > ifcvf_get_features(struct ifcvf_hw *hw); >=20 > int > +ifcvf_enable_vring_hw(struct ifcvf_hw *hw, int i); > + > +void > +ifcvf_disable_vring_hw(struct ifcvf_hw *hw, int i); > + > +int > ifcvf_start_hw(struct ifcvf_hw *hw); >=20 > void > diff --git a/drivers/vdpa/ifc/ifcvf_vdpa.c b/drivers/vdpa/ifc/ifcvf_vdpa.= c > index b00afdb..32bc1c9 100644 > --- a/drivers/vdpa/ifc/ifcvf_vdpa.c > +++ b/drivers/vdpa/ifc/ifcvf_vdpa.c > @@ -1282,13 +1282,59 @@ struct rte_vdpa_dev_info { > } >=20 > static int > +ifcvf_config_vring(struct ifcvf_internal *internal, int vring) > +{ > + struct ifcvf_hw *hw =3D &internal->hw; > + int vid =3D internal->vid; > + struct rte_vhost_vring vq; > + uint64_t gpa; > + > + if (hw->vring[vring].enable) { > + rte_vhost_get_vhost_vring(vid, vring, &vq); > + gpa =3D hva_to_gpa(vid, (uint64_t)(uintptr_t)vq.desc); > + if (gpa =3D=3D 0) { > + DRV_LOG(ERR, "Fail to get GPA for descriptor ring."); > + return -1; > + } > + hw->vring[vring].desc =3D gpa; > + > + gpa =3D hva_to_gpa(vid, (uint64_t)(uintptr_t)vq.avail); > + if (gpa =3D=3D 0) { > + DRV_LOG(ERR, "Fail to get GPA for available ring."); > + return -1; > + } > + hw->vring[vring].avail =3D gpa; > + > + gpa =3D hva_to_gpa(vid, (uint64_t)(uintptr_t)vq.used); > + if (gpa =3D=3D 0) { > + DRV_LOG(ERR, "Fail to get GPA for used ring."); > + return -1; > + } > + hw->vring[vring].used =3D gpa; > + > + hw->vring[vring].size =3D vq.size; > + rte_vhost_get_vring_base(vid, vring, > + &hw->vring[vring].last_avail_idx, > + &hw->vring[vring].last_used_idx); > + ifcvf_enable_vring_hw(&internal->hw, vring); > + } else { > + ifcvf_disable_vring_hw(&internal->hw, vring); > + rte_vhost_set_vring_base(vid, vring, > + hw->vring[vring].last_avail_idx, > + hw->vring[vring].last_used_idx); > + } > + > + return 0; > +} > + > +static int > ifcvf_set_vring_state(int vid, int vring, int state) > { > struct rte_vdpa_device *vdev; > struct internal_list *list; > struct ifcvf_internal *internal; > struct ifcvf_hw *hw; > - struct ifcvf_pci_common_cfg *cfg; > + bool enable =3D !!state; > int ret =3D 0; >=20 > vdev =3D rte_vhost_get_vdpa_device(vid); > @@ -1298,6 +1344,9 @@ struct rte_vdpa_dev_info { > return -1; > } >=20 > + DRV_LOG(INFO, "%s queue %d of vDPA device %s", > + enable ? "enable" : "disable", vring, vdev->device->name); > + > internal =3D list->internal; > if (vring < 0 || vring >=3D internal->max_queues * 2) { > DRV_LOG(ERR, "Vring index %d not correct", vring); > @@ -1305,27 +1354,41 @@ struct rte_vdpa_dev_info { > } >=20 > hw =3D &internal->hw; > + hw->vring[vring].enable =3D enable; > + > if (!internal->configured) > - goto exit; > + return 0; >=20 > - cfg =3D hw->common_cfg; > - IFCVF_WRITE_REG16(vring, &cfg->queue_select); > - IFCVF_WRITE_REG16(!!state, &cfg->queue_enable); > + unset_notify_relay(internal); >=20 > - if (!state && hw->vring[vring].enable) { > - ret =3D vdpa_disable_vfio_intr(internal); > - if (ret) > - return ret; > + ret =3D vdpa_enable_vfio_intr(internal, false); > + if (ret) { > + DRV_LOG(ERR, "failed to set vfio interrupt of vDPA device %s", > + vdev->device->name); > + return ret; > } >=20 > - if (state && !hw->vring[vring].enable) { > - ret =3D vdpa_enable_vfio_intr(internal, false); > - if (ret) > - return ret; > + ret =3D ifcvf_config_vring(internal, vring); > + if (ret) { > + DRV_LOG(ERR, "failed to configure queue %d of vDPA device %s", > + vring, vdev->device->name); > + return ret; > + } > + > + ret =3D setup_notify_relay(internal); > + if (ret) { > + DRV_LOG(ERR, "failed to setup notify relay of vDPA device %s", > + vdev->device->name); > + return ret; > + } > + > + ret =3D rte_vhost_host_notifier_ctrl(vid, vring, enable); > + if (ret) { > + DRV_LOG(ERR, "vDPA device %s queue %d host notifier ctrl fail", > + vdev->device->name, vring); > + return ret; > } >=20 > -exit: > - hw->vring[vring].enable =3D !!state; > return 0; > } >=20 > -- > 1.8.3.1