[dpdk-dev] [PATCH] net/ice: simplify the use of DCF device reset

[dpdk-dev] [PATCH] net/ice: simplify the use of DCF device reset

[dapengx.yu]

From: Dapeng Yu <dapengx.yu@intel.com>

After DCF is reset by PF, the DCF device un-initialization cannot
function normally since the resource is already invalidated. So
reset DCF twice is necessary, the first reset re-initializes the DCF,
only then second reset can clean the filters successfully.

This patch detects the reset flag, which is set by PF on DCF reset,
if the flag is true, do DCF reset twice automatically.

Fixes: 1a86f4dbdf42 ("net/ice: support DCF device reset")
Cc: stable@dpdk.org

Signed-off-by: Dapeng Yu <dapengx.yu@intel.com>
---
 drivers/net/ice/ice_dcf_ethdev.c | 21 +++++++++++++++++++++
 1 file changed, 21 insertions(+)

diff --git a/drivers/net/ice/ice_dcf_ethdev.c b/drivers/net/ice/ice_dcf_ethdev.c
index 7cb8066416..e8057f5588 100644
--- a/drivers/net/ice/ice_dcf_ethdev.c
+++ b/drivers/net/ice/ice_dcf_ethdev.c
@@ -1031,8 +1031,29 @@ ice_dcf_tm_ops_get(struct rte_eth_dev *dev __rte_unused,
 static int
 ice_dcf_dev_reset(struct rte_eth_dev *dev)
 {
+	struct ice_dcf_adapter *ad = dev->data->dev_private;
+	struct iavf_hw *hw = &ad->real_hw.avf;
 	int ret;
 
+	if (!(IAVF_READ_REG(hw, IAVF_VF_ARQLEN1) &
+	      IAVF_VF_ARQLEN1_ARQENABLE_MASK)) {
+		if (!ad->real_hw.resetting)
+			ad->real_hw.resetting = true;
+		PMD_DRV_LOG(ERR, "The DCF has been reset by PF");
+
+		/*
+		 * Do the extra dev uninit/init to make DCF get resource.
+		 * Then the next uninit/init can clean filters successfully.
+		 */
+		ret = ice_dcf_dev_uninit(dev);
+		if (ret)
+			return ret;
+
+		ret = ice_dcf_dev_init(dev);
+		if (ret)
+			return ret;
+	}
+
 	ret = ice_dcf_dev_uninit(dev);
 	if (ret)
 		return ret;
-- 
2.27.0

[dpdk-dev] [PATCH v2] net/ice: simplify the use of DCF device reset

[dapengx.yu]

From: Dapeng Yu <dapengx.yu@intel.com>

After DCF is reset by PF, the DCF device un-initialization cannot
function normally since the resource is already invalidated. So
reset DCF twice is necessary, the first reset re-initializes the DCF,
only then second reset can clean the filters successfully.

This patch detects the reset flag, which is set by PF on DCF reset,
if the flag is true, do DCF reset twice automatically.

Fixes: 1a86f4dbdf42 ("net/ice: support DCF device reset")
Cc: stable@dpdk.org

Signed-off-by: Dapeng Yu <dapengx.yu@intel.com>
---
V2:
* Ignore the returned error of dev_uninit when DCF is reset by PF
---
 drivers/net/ice/ice_dcf_ethdev.c | 19 +++++++++++++++++++
 1 file changed, 19 insertions(+)

diff --git a/drivers/net/ice/ice_dcf_ethdev.c b/drivers/net/ice/ice_dcf_ethdev.c
index 7cb8066416..f51b7cbb8b 100644
--- a/drivers/net/ice/ice_dcf_ethdev.c
+++ b/drivers/net/ice/ice_dcf_ethdev.c
@@ -1031,8 +1031,27 @@ ice_dcf_tm_ops_get(struct rte_eth_dev *dev __rte_unused,
 static int
 ice_dcf_dev_reset(struct rte_eth_dev *dev)
 {
+	struct ice_dcf_adapter *ad = dev->data->dev_private;
+	struct iavf_hw *hw = &ad->real_hw.avf;
 	int ret;
 
+	if (!(IAVF_READ_REG(hw, IAVF_VF_ARQLEN1) &
+	      IAVF_VF_ARQLEN1_ARQENABLE_MASK)) {
+		if (!ad->real_hw.resetting)
+			ad->real_hw.resetting = true;
+		PMD_DRV_LOG(ERR, "The DCF has been reset by PF");
+
+		/*
+		 * Do the extra dev uninit/init to make DCF get resource.
+		 * Then the next uninit/init can clean filters successfully.
+		 */
+		ice_dcf_dev_uninit(dev);
+
+		ret = ice_dcf_dev_init(dev);
+		if (ret)
+			return ret;
+	}
+
 	ret = ice_dcf_dev_uninit(dev);
 	if (ret)
 		return ret;
-- 
2.27.0

[dpdk-dev] [PATCH v3] net/ice: simplify the use of DCF device reset

[dapengx.yu]

From: Dapeng Yu <dapengx.yu@intel.com>

After DCF is reset by PF, the DCF device un-initialization cannot
function normally since the resource is already invalidated. So
reset DCF twice is necessary, the first simplified reset
re-initializes the AdminQ of DCF, only then second reset can clean
the filters successfully.

This patch detects the reset flag, which is set by PF on DCF reset,
if the flag is true, do DCF reset twice automatically.

Fixes: 1a86f4dbdf42 ("net/ice: support DCF device reset")
Cc: stable@dpdk.org

Signed-off-by: Dapeng Yu <dapengx.yu@intel.com>
---
V2:
* Ignore the returned error of dev_uninit when DCF is reset by PF
V3:
* Add a reset function to re-initialize AdminQ resource
* Add a function to check the reset flag
---
 drivers/net/ice/ice_dcf.c        |  2 ++
 drivers/net/ice/ice_dcf_ethdev.c | 32 ++++++++++++++++++++++++++++++++
 2 files changed, 34 insertions(+)

diff --git a/drivers/net/ice/ice_dcf.c b/drivers/net/ice/ice_dcf.c
index 084f7a53db..366ff0a907 100644
--- a/drivers/net/ice/ice_dcf.c
+++ b/drivers/net/ice/ice_dcf.c
@@ -593,6 +593,8 @@ ice_dcf_init_hw(struct rte_eth_dev *eth_dev, struct ice_dcf_hw *hw)
 	struct rte_pci_device *pci_dev = RTE_ETH_DEV_TO_PCI(eth_dev);
 	int ret, size;
 
+	hw->resetting = false;
+
 	hw->avf.hw_addr = pci_dev->mem_resource[0].addr;
 	hw->avf.back = hw;
 
diff --git a/drivers/net/ice/ice_dcf_ethdev.c b/drivers/net/ice/ice_dcf_ethdev.c
index 7c71a48010..f47219ee17 100644
--- a/drivers/net/ice/ice_dcf_ethdev.c
+++ b/drivers/net/ice/ice_dcf_ethdev.c
@@ -1025,11 +1025,43 @@ ice_dcf_tm_ops_get(struct rte_eth_dev *dev __rte_unused,
 	return 0;
 }
 
+static inline void
+ice_dcf_simple_reset(struct rte_eth_dev *eth_dev, struct ice_dcf_hw *hw)
+{
+	ice_dcf_uninit_hw(eth_dev, hw);
+	ice_dcf_init_hw(eth_dev, hw);
+}
+
+/* Check if reset has been triggered by PF */
+static inline bool
+dcf_is_reset(struct rte_eth_dev *dev)
+{
+	struct ice_dcf_adapter *ad = dev->data->dev_private;
+	struct iavf_hw *hw = &ad->real_hw.avf;
+
+	return !(IAVF_READ_REG(hw, IAVF_VF_ARQLEN1) &
+		 IAVF_VF_ARQLEN1_ARQENABLE_MASK);
+}
+
 static int
 ice_dcf_dev_reset(struct rte_eth_dev *dev)
 {
+	struct ice_dcf_adapter *ad = dev->data->dev_private;
+	struct ice_dcf_hw *hw = &ad->real_hw;
 	int ret;
 
+	if (dcf_is_reset(dev)) {
+		if (!ad->real_hw.resetting)
+			ad->real_hw.resetting = true;
+		PMD_DRV_LOG(ERR, "The DCF has been reset by PF");
+
+		/*
+		 * Do the simplified reset to make DCF get AdminQ resource.
+		 * Then the next uninit/init can clean filters successfully.
+		 */
+		ice_dcf_simple_reset(dev, hw);
+	}
+
 	ret = ice_dcf_dev_uninit(dev);
 	if (ret)
 		return ret;
-- 
2.27.0

[dpdk-dev] [PATCH v4] net/ice: simplify the use of DCF device reset

[dapengx.yu]

From: Dapeng Yu <dapengx.yu@intel.com>

After DCF is reset by PF, the DCF device un-initialization cannot
function normally since the resource is already invalidated. So
reset DCF twice is necessary, the first simplified reset
re-initializes the AdminQ of DCF, only then second reset can clean
the filters successfully.

This patch detects the reset flag, which is set by PF on DCF reset,
if the flag is true, do DCF reset twice automatically.

Fixes: 1a86f4dbdf42 ("net/ice: support DCF device reset")
Cc: stable@dpdk.org

Signed-off-by: Dapeng Yu <dapengx.yu@intel.com>
---
V2:
* Ignore the returned error of dev_uninit when DCF is reset by PF
V3:
* Add a reset function to re-initialize AdminQ resource
* Add a function to check the reset flag
V4:
* Remove redundant reset flag setting
---
 drivers/net/ice/ice_dcf.c        |  2 ++
 drivers/net/ice/ice_dcf_ethdev.c | 33 +++++++++++++++++++++++++++++++-
 2 files changed, 34 insertions(+), 1 deletion(-)

diff --git a/drivers/net/ice/ice_dcf.c b/drivers/net/ice/ice_dcf.c
index 084f7a53db..366ff0a907 100644
--- a/drivers/net/ice/ice_dcf.c
+++ b/drivers/net/ice/ice_dcf.c
@@ -593,6 +593,8 @@ ice_dcf_init_hw(struct rte_eth_dev *eth_dev, struct ice_dcf_hw *hw)
 	struct rte_pci_device *pci_dev = RTE_ETH_DEV_TO_PCI(eth_dev);
 	int ret, size;
 
+	hw->resetting = false;
+
 	hw->avf.hw_addr = pci_dev->mem_resource[0].addr;
 	hw->avf.back = hw;
 
diff --git a/drivers/net/ice/ice_dcf_ethdev.c b/drivers/net/ice/ice_dcf_ethdev.c
index 7c71a48010..171b58a748 100644
--- a/drivers/net/ice/ice_dcf_ethdev.c
+++ b/drivers/net/ice/ice_dcf_ethdev.c
@@ -1025,11 +1025,43 @@ ice_dcf_tm_ops_get(struct rte_eth_dev *dev __rte_unused,
 	return 0;
 }
 
+static inline void
+ice_dcf_simple_reset(struct rte_eth_dev *eth_dev, struct ice_dcf_hw *hw)
+{
+	ice_dcf_uninit_hw(eth_dev, hw);
+	ice_dcf_init_hw(eth_dev, hw);
+}
+
+/* Check if reset has been triggered by PF */
+static inline bool
+dcf_is_reset(struct rte_eth_dev *dev)
+{
+	struct ice_dcf_adapter *ad = dev->data->dev_private;
+	struct iavf_hw *hw = &ad->real_hw.avf;
+
+	return !(IAVF_READ_REG(hw, IAVF_VF_ARQLEN1) &
+		 IAVF_VF_ARQLEN1_ARQENABLE_MASK);
+}
+
 static int
 ice_dcf_dev_reset(struct rte_eth_dev *dev)
 {
+	struct ice_dcf_adapter *ad = dev->data->dev_private;
+	struct ice_dcf_hw *hw = &ad->real_hw;
 	int ret;
 
+	if (dcf_is_reset(dev)) {
+		if (!ad->real_hw.resetting)
+			ad->real_hw.resetting = true;
+		PMD_DRV_LOG(ERR, "The DCF has been reset by PF");
+
+		/*
+		 * Do the simplified reset to make DCF get AdminQ resource.
+		 * Then the next uninit/init can clean filters successfully.
+		 */
+		ice_dcf_simple_reset(dev, hw);
+	}
+
 	ret = ice_dcf_dev_uninit(dev);
 	if (ret)
 		return ret;
@@ -1072,7 +1104,6 @@ ice_dcf_dev_init(struct rte_eth_dev *eth_dev)
 {
 	struct ice_dcf_adapter *adapter = eth_dev->data->dev_private;
 
-	adapter->real_hw.resetting = false;
 	eth_dev->dev_ops = &ice_dcf_eth_dev_ops;
 	eth_dev->rx_pkt_burst = ice_dcf_recv_pkts;
 	eth_dev->tx_pkt_burst = ice_dcf_xmit_pkts;
-- 
2.27.0

Re: [dpdk-dev] [PATCH v4] net/ice: simplify the use of DCF device reset

[Zhang, Qi Z]

> -----Original Message-----
> From: Yu, DapengX <dapengx.yu@intel.com>
> Sent: Tuesday, October 26, 2021 4:44 PM
> To: Yang, Qiming <qiming.yang@intel.com>; Zhang, Qi Z
> <qi.z.zhang@intel.com>
> Cc: dev@dpdk.org; Yu, DapengX <dapengx.yu@intel.com>; stable@dpdk.org
> Subject: [PATCH v4] net/ice: simplify the use of DCF device reset
> 
> From: Dapeng Yu <dapengx.yu@intel.com>
> 
> After DCF is reset by PF, the DCF device un-initialization cannot function
> normally since the resource is already invalidated. So reset DCF twice is
> necessary, the first simplified reset re-initializes the AdminQ of DCF, only then
> second reset can clean the filters successfully.
> 
> This patch detects the reset flag, which is set by PF on DCF reset, if the flag is
> true, do DCF reset twice automatically.
> 
> Fixes: 1a86f4dbdf42 ("net/ice: support DCF device reset")
> Cc: stable@dpdk.org
> 
> Signed-off-by: Dapeng Yu <dapengx.yu@intel.com>
> ---
> V2:
> * Ignore the returned error of dev_uninit when DCF is reset by PF
> V3:
> * Add a reset function to re-initialize AdminQ resource
> * Add a function to check the reset flag
> V4:
> * Remove redundant reset flag setting
> ---
>  drivers/net/ice/ice_dcf.c        |  2 ++
>  drivers/net/ice/ice_dcf_ethdev.c | 33 +++++++++++++++++++++++++++++++-
>  2 files changed, 34 insertions(+), 1 deletion(-)
> 
> diff --git a/drivers/net/ice/ice_dcf.c b/drivers/net/ice/ice_dcf.c index
> 084f7a53db..366ff0a907 100644
> --- a/drivers/net/ice/ice_dcf.c
> +++ b/drivers/net/ice/ice_dcf.c
> @@ -593,6 +593,8 @@ ice_dcf_init_hw(struct rte_eth_dev *eth_dev, struct
> ice_dcf_hw *hw)
>  	struct rte_pci_device *pci_dev = RTE_ETH_DEV_TO_PCI(eth_dev);
>  	int ret, size;
> 
> +	hw->resetting = false;
> +
>  	hw->avf.hw_addr = pci_dev->mem_resource[0].addr;
>  	hw->avf.back = hw;
> 
> diff --git a/drivers/net/ice/ice_dcf_ethdev.c b/drivers/net/ice/ice_dcf_ethdev.c
> index 7c71a48010..171b58a748 100644
> --- a/drivers/net/ice/ice_dcf_ethdev.c
> +++ b/drivers/net/ice/ice_dcf_ethdev.c
> @@ -1025,11 +1025,43 @@ ice_dcf_tm_ops_get(struct rte_eth_dev *dev
> __rte_unused,
>  	return 0;
>  }
> 
> +static inline void
> +ice_dcf_simple_reset(struct rte_eth_dev *eth_dev, struct ice_dcf_hw	

Better to rename the function to ice_dcf_reset_hw 

> +*hw) {
> +	ice_dcf_uninit_hw(eth_dev, hw);
> +	ice_dcf_init_hw(eth_dev, hw);
> +}
> +
> +/* Check if reset has been triggered by PF */ static inline bool
> +dcf_is_reset(struct rte_eth_dev *dev) {
> +	struct ice_dcf_adapter *ad = dev->data->dev_private;
> +	struct iavf_hw *hw = &ad->real_hw.avf;
> +
> +	return !(IAVF_READ_REG(hw, IAVF_VF_ARQLEN1) &
> +		 IAVF_VF_ARQLEN1_ARQENABLE_MASK);
> +}
> +
>  static int
>  ice_dcf_dev_reset(struct rte_eth_dev *dev)  {
> +	struct ice_dcf_adapter *ad = dev->data->dev_private;
> +	struct ice_dcf_hw *hw = &ad->real_hw;
>  	int ret;
> 
> +	if (dcf_is_reset(dev)) {
> +		if (!ad->real_hw.resetting)
> +			ad->real_hw.resetting = true;
> +		PMD_DRV_LOG(ERR, "The DCF has been reset by PF");
> +
> +		/*
> +		 * Do the simplified reset to make DCF get AdminQ resource.
> +		 * Then the next uninit/init can clean filters successfully.
> +		 */		

Can reword as below:

Simply reset hw to trigger an additional DCF enable/disable cycle which help to workaround
the issue that kernel driver may not clean up resource during previous reset.

> +		ice_dcf_simple_reset(dev, hw);
> +	}
> +
>  	ret = ice_dcf_dev_uninit(dev);
>  	if (ret)
>  		return ret;
> @@ -1072,7 +1104,6 @@ ice_dcf_dev_init(struct rte_eth_dev *eth_dev)  {
>  	struct ice_dcf_adapter *adapter = eth_dev->data->dev_private;
> 
> -	adapter->real_hw.resetting = false;
>  	eth_dev->dev_ops = &ice_dcf_eth_dev_ops;
>  	eth_dev->rx_pkt_burst = ice_dcf_recv_pkts;
>  	eth_dev->tx_pkt_burst = ice_dcf_xmit_pkts;
> --
> 2.27.0

[dpdk-dev] [PATCH v5] net/ice: simplify the use of DCF device reset

[dapengx.yu]

From: Dapeng Yu <dapengx.yu@intel.com>

After DCF is reset by PF, the DCF device un-initialization cannot
function normally since the kernel may not clean up resource.

This patch detects the reset flag, which is set by PF on DCF reset,
if the flag is true, reset hw to trigger an additional DCF
enable/disable cycle which helps to work around the issue that kernel
driver may not clean up resource during previous reset.

Fixes: 1a86f4dbdf42 ("net/ice: support DCF device reset")
Cc: stable@dpdk.org

Signed-off-by: Dapeng Yu <dapengx.yu@intel.com>
---
V2:
* Ignore the returned error of dev_uninit when DCF is reset by PF
V3:
* Add a reset function to re-initialize AdminQ resource
* Add a function to check the reset flag
V4:
* Remove redundant reset flag setting
V5:
* Rename function and modify comments
---
 drivers/net/ice/ice_dcf.c        |  2 ++
 drivers/net/ice/ice_dcf_ethdev.c | 34 +++++++++++++++++++++++++++++++-
 2 files changed, 35 insertions(+), 1 deletion(-)

diff --git a/drivers/net/ice/ice_dcf.c b/drivers/net/ice/ice_dcf.c
index 084f7a53db..366ff0a907 100644
--- a/drivers/net/ice/ice_dcf.c
+++ b/drivers/net/ice/ice_dcf.c
@@ -593,6 +593,8 @@ ice_dcf_init_hw(struct rte_eth_dev *eth_dev, struct ice_dcf_hw *hw)
 	struct rte_pci_device *pci_dev = RTE_ETH_DEV_TO_PCI(eth_dev);
 	int ret, size;
 
+	hw->resetting = false;
+
 	hw->avf.hw_addr = pci_dev->mem_resource[0].addr;
 	hw->avf.back = hw;
 
diff --git a/drivers/net/ice/ice_dcf_ethdev.c b/drivers/net/ice/ice_dcf_ethdev.c
index 7c71a48010..4d9484e994 100644
--- a/drivers/net/ice/ice_dcf_ethdev.c
+++ b/drivers/net/ice/ice_dcf_ethdev.c
@@ -1025,11 +1025,44 @@ ice_dcf_tm_ops_get(struct rte_eth_dev *dev __rte_unused,
 	return 0;
 }
 
+static inline void
+ice_dcf_reset_hw(struct rte_eth_dev *eth_dev, struct ice_dcf_hw *hw)
+{
+	ice_dcf_uninit_hw(eth_dev, hw);
+	ice_dcf_init_hw(eth_dev, hw);
+}
+
+/* Check if reset has been triggered by PF */
+static inline bool
+ice_dcf_is_reset(struct rte_eth_dev *dev)
+{
+	struct ice_dcf_adapter *ad = dev->data->dev_private;
+	struct iavf_hw *hw = &ad->real_hw.avf;
+
+	return !(IAVF_READ_REG(hw, IAVF_VF_ARQLEN1) &
+		 IAVF_VF_ARQLEN1_ARQENABLE_MASK);
+}
+
 static int
 ice_dcf_dev_reset(struct rte_eth_dev *dev)
 {
+	struct ice_dcf_adapter *ad = dev->data->dev_private;
+	struct ice_dcf_hw *hw = &ad->real_hw;
 	int ret;
 
+	if (ice_dcf_is_reset(dev)) {
+		if (!ad->real_hw.resetting)
+			ad->real_hw.resetting = true;
+		PMD_DRV_LOG(ERR, "The DCF has been reset by PF");
+
+		/*
+		 * Simply reset hw to trigger an additional DCF enable/disable
+		 * cycle which help to workaround the issue that kernel driver
+		 * may not clean up resource during previous reset.
+		 */
+		ice_dcf_reset_hw(dev, hw);
+	}
+
 	ret = ice_dcf_dev_uninit(dev);
 	if (ret)
 		return ret;
@@ -1072,7 +1105,6 @@ ice_dcf_dev_init(struct rte_eth_dev *eth_dev)
 {
 	struct ice_dcf_adapter *adapter = eth_dev->data->dev_private;
 
-	adapter->real_hw.resetting = false;
 	eth_dev->dev_ops = &ice_dcf_eth_dev_ops;
 	eth_dev->rx_pkt_burst = ice_dcf_recv_pkts;
 	eth_dev->tx_pkt_burst = ice_dcf_xmit_pkts;
-- 
2.27.0

Re: [dpdk-dev] [PATCH v5] net/ice: simplify the use of DCF device reset

[Zhang, Qi Z]

> -----Original Message-----
> From: Yu, DapengX <dapengx.yu@intel.com>
> Sent: Tuesday, October 26, 2021 5:53 PM
> To: Yang, Qiming <qiming.yang@intel.com>; Zhang, Qi Z
> <qi.z.zhang@intel.com>
> Cc: dev@dpdk.org; Yu, DapengX <dapengx.yu@intel.com>; stable@dpdk.org
> Subject: [PATCH v5] net/ice: simplify the use of DCF device reset

It's a workaround, so rename to 

"workaround DCF reset failure"

> 
> From: Dapeng Yu <dapengx.yu@intel.com>
> 
> After DCF is reset by PF, the DCF device un-initialization cannot function
> normally since the kernel may not clean up resource.
> 
> This patch detects the reset flag, which is set by PF on DCF reset, if the flag is
> true, reset hw to trigger an additional DCF enable/disable cycle which helps to
> work around the issue that kernel driver may not clean up resource during
> previous reset.

Reword to

After DCF is reset by PF, the DCF device un-initialization cannot
function normally, ignore the failure does not help since the kernel
does not clean up resource.

The patch workaround the issue by triggering an additional DCF enable/
disable cycle when a passive reset is detected.

> 
> Fixes: 1a86f4dbdf42 ("net/ice: support DCF device reset")
> Cc: stable@dpdk.org
> 
> Signed-off-by: Dapeng Yu <dapengx.yu@intel.com>

Acked-by: Qi Zhang <qi.z.zhang@intel.com>

Applied to dpdk-next-net-intel.

Thanks
Qi