[dpdk-dev] [PATCH] drivers/net: release port upon close

[dpdk-dev] [PATCH] drivers/net: release port upon close

[chenxux.di]

From: Di ChenxuX <chenxux.di@intel.com>

Set RTE_ETH_DEV_CLOSE_REMOVE upon probe so all the private
resources for the port can be freed by rte_eth_dev_close().
This patch cover all the intel drivers

Signed-off-by: Di ChenxuX <chenxux.di@intel.com>
---
 drivers/net/e1000/igb_ethdev.c   | 5 +++++
 drivers/net/fm10k/fm10k_ethdev.c | 5 +++++
 drivers/net/i40e/i40e_ethdev.c   | 5 +++++
 drivers/net/ice/ice_ethdev.c     | 5 +++++
 drivers/net/ixgbe/ixgbe_ethdev.c | 5 +++++
 5 files changed, 25 insertions(+)

diff --git a/drivers/net/e1000/igb_ethdev.c b/drivers/net/e1000/igb_ethdev.c
index fec2b4289..fe785d1d7 100644
--- a/drivers/net/e1000/igb_ethdev.c
+++ b/drivers/net/e1000/igb_ethdev.c
@@ -843,6 +843,11 @@ eth_igb_dev_init(struct rte_eth_dev *eth_dev)
 	rte_ether_addr_copy((struct rte_ether_addr *)hw->mac.addr,
 			&eth_dev->data->mac_addrs[0]);
 
+	/* Pass the information to the rte_eth_dev_close() that it should also
+	 * release the private port resources.
+	 */
+	dev->data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
+
 	/* initialize the vfta */
 	memset(shadow_vfta, 0, sizeof(*shadow_vfta));
 
diff --git a/drivers/net/fm10k/fm10k_ethdev.c b/drivers/net/fm10k/fm10k_ethdev.c
index db4d72129..b5a427c7b 100644
--- a/drivers/net/fm10k/fm10k_ethdev.c
+++ b/drivers/net/fm10k/fm10k_ethdev.c
@@ -3103,6 +3103,11 @@ eth_fm10k_dev_init(struct rte_eth_dev *dev)
 		&dev->data->mac_addrs[0]);
 	}
 
+	/* Pass the information to the rte_eth_dev_close() that it should also
+	 * release the private port resources.
+	 */
+	dev->data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
+
 	/* Reset the hw statistics */
 	fm10k_stats_reset(dev);
 
diff --git a/drivers/net/i40e/i40e_ethdev.c b/drivers/net/i40e/i40e_ethdev.c
index 4e40b7ab5..a0536f14c 100644
--- a/drivers/net/i40e/i40e_ethdev.c
+++ b/drivers/net/i40e/i40e_ethdev.c
@@ -1520,6 +1520,11 @@ eth_i40e_dev_init(struct rte_eth_dev *dev, void *init_params __rte_unused)
 	rte_ether_addr_copy((struct rte_ether_addr *)hw->mac.perm_addr,
 					&dev->data->mac_addrs[0]);
 
+	/* Pass the information to the rte_eth_dev_close() that it should also
+	 * release the private port resources.
+	 */
+	dev->data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
+
 	/* Init dcb to sw mode by default */
 	ret = i40e_dcb_init_configure(dev, TRUE);
 	if (ret != I40E_SUCCESS) {
diff --git a/drivers/net/ice/ice_ethdev.c b/drivers/net/ice/ice_ethdev.c
index 44a14cb8a..476c768ee 100644
--- a/drivers/net/ice/ice_ethdev.c
+++ b/drivers/net/ice/ice_ethdev.c
@@ -1488,6 +1488,11 @@ ice_dev_init(struct rte_eth_dev *dev)
 		goto err_init_mac;
 	}
 
+	/* Pass the information to the rte_eth_dev_close() that it should also
+	 * release the private port resources.
+	 */
+	dev->data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
+
 	ret = ice_res_pool_init(&pf->msix_pool, 1,
 				hw->func_caps.common_cap.num_msix_vectors - 1);
 	if (ret) {
diff --git a/drivers/net/ixgbe/ixgbe_ethdev.c b/drivers/net/ixgbe/ixgbe_ethdev.c
index 03fc1f717..3e2b537ec 100644
--- a/drivers/net/ixgbe/ixgbe_ethdev.c
+++ b/drivers/net/ixgbe/ixgbe_ethdev.c
@@ -1245,6 +1245,11 @@ eth_ixgbe_dev_init(struct rte_eth_dev *eth_dev, void *init_params __rte_unused)
 		return -ENOMEM;
 	}
 
+	/* Pass the information to the rte_eth_dev_close() that it should also
+	 * release the private port resources.
+	 */
+	dev->data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
+
 	/* initialize the vfta */
 	memset(shadow_vfta, 0, sizeof(*shadow_vfta));
 
-- 
2.17.1

Re: [dpdk-dev] [PATCH] drivers/net: release port upon close

[Ye Xiaolong]

Hi, chenxu

On 08/27, chenxux.di@intel.com wrote:
>From: Di ChenxuX <chenxux.di@intel.com>
>
>Set RTE_ETH_DEV_CLOSE_REMOVE upon probe so all the private
>resources for the port can be freed by rte_eth_dev_close().
>This patch cover all the intel drivers
>

Just setting RTE_ETH_DEV_CLOSE_REMOVE in probe stage is not enough, you can see
the advice in http://git.dpdk.org/dpdk/commit/?id=23ea57a2a

"When enabling RTE_ETH_DEV_CLOSE_REMOVE, the PMD must free all its private resources for the port 
in its dev_close function, it's advised to call the dev_close function in the remove function 
in order to support removing a device without closing its ports".

You can refer to other PMD which has finished this transition, like
696202ca5396 ("net/mvpp2: remove resources when port is closed")

Also, I'd suggest to split it to several patches, one patch for one PMD.

Thanks,
Xiaolong

>Signed-off-by: Di ChenxuX <chenxux.di@intel.com>
>---
> drivers/net/e1000/igb_ethdev.c   | 5 +++++
> drivers/net/fm10k/fm10k_ethdev.c | 5 +++++
> drivers/net/i40e/i40e_ethdev.c   | 5 +++++
> drivers/net/ice/ice_ethdev.c     | 5 +++++
> drivers/net/ixgbe/ixgbe_ethdev.c | 5 +++++
> 5 files changed, 25 insertions(+)
>
>diff --git a/drivers/net/e1000/igb_ethdev.c b/drivers/net/e1000/igb_ethdev.c
>index fec2b4289..fe785d1d7 100644
>--- a/drivers/net/e1000/igb_ethdev.c
>+++ b/drivers/net/e1000/igb_ethdev.c
>@@ -843,6 +843,11 @@ eth_igb_dev_init(struct rte_eth_dev *eth_dev)
> 	rte_ether_addr_copy((struct rte_ether_addr *)hw->mac.addr,
> 			&eth_dev->data->mac_addrs[0]);
> 
>+	/* Pass the information to the rte_eth_dev_close() that it should also
>+	 * release the private port resources.
>+	 */
>+	dev->data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
>+
> 	/* initialize the vfta */
> 	memset(shadow_vfta, 0, sizeof(*shadow_vfta));
> 
>diff --git a/drivers/net/fm10k/fm10k_ethdev.c b/drivers/net/fm10k/fm10k_ethdev.c
>index db4d72129..b5a427c7b 100644
>--- a/drivers/net/fm10k/fm10k_ethdev.c
>+++ b/drivers/net/fm10k/fm10k_ethdev.c
>@@ -3103,6 +3103,11 @@ eth_fm10k_dev_init(struct rte_eth_dev *dev)
> 		&dev->data->mac_addrs[0]);
> 	}
> 
>+	/* Pass the information to the rte_eth_dev_close() that it should also
>+	 * release the private port resources.
>+	 */
>+	dev->data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
>+
> 	/* Reset the hw statistics */
> 	fm10k_stats_reset(dev);
> 
>diff --git a/drivers/net/i40e/i40e_ethdev.c b/drivers/net/i40e/i40e_ethdev.c
>index 4e40b7ab5..a0536f14c 100644
>--- a/drivers/net/i40e/i40e_ethdev.c
>+++ b/drivers/net/i40e/i40e_ethdev.c
>@@ -1520,6 +1520,11 @@ eth_i40e_dev_init(struct rte_eth_dev *dev, void *init_params __rte_unused)
> 	rte_ether_addr_copy((struct rte_ether_addr *)hw->mac.perm_addr,
> 					&dev->data->mac_addrs[0]);
> 
>+	/* Pass the information to the rte_eth_dev_close() that it should also
>+	 * release the private port resources.
>+	 */
>+	dev->data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
>+
> 	/* Init dcb to sw mode by default */
> 	ret = i40e_dcb_init_configure(dev, TRUE);
> 	if (ret != I40E_SUCCESS) {
>diff --git a/drivers/net/ice/ice_ethdev.c b/drivers/net/ice/ice_ethdev.c
>index 44a14cb8a..476c768ee 100644
>--- a/drivers/net/ice/ice_ethdev.c
>+++ b/drivers/net/ice/ice_ethdev.c
>@@ -1488,6 +1488,11 @@ ice_dev_init(struct rte_eth_dev *dev)
> 		goto err_init_mac;
> 	}
> 
>+	/* Pass the information to the rte_eth_dev_close() that it should also
>+	 * release the private port resources.
>+	 */
>+	dev->data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
>+
> 	ret = ice_res_pool_init(&pf->msix_pool, 1,
> 				hw->func_caps.common_cap.num_msix_vectors - 1);
> 	if (ret) {
>diff --git a/drivers/net/ixgbe/ixgbe_ethdev.c b/drivers/net/ixgbe/ixgbe_ethdev.c
>index 03fc1f717..3e2b537ec 100644
>--- a/drivers/net/ixgbe/ixgbe_ethdev.c
>+++ b/drivers/net/ixgbe/ixgbe_ethdev.c
>@@ -1245,6 +1245,11 @@ eth_ixgbe_dev_init(struct rte_eth_dev *eth_dev, void *init_params __rte_unused)
> 		return -ENOMEM;
> 	}
> 
>+	/* Pass the information to the rte_eth_dev_close() that it should also
>+	 * release the private port resources.
>+	 */
>+	dev->data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
>+
> 	/* initialize the vfta */
> 	memset(shadow_vfta, 0, sizeof(*shadow_vfta));
> 
>-- 
>2.17.1
>

[dpdk-dev] [PATCH v3 0/5] drivers/net: release port upon close

[Di ChenxuX]

Set RTE_ETH_DEV_CLOSE_REMOVE upon probe so all the private resources 
 for the port can be freed by rte_eth_dev_close().
This patch cover all the intel drivers.

v3:
remove adapter closed flag in driver e1000, fm10k, ice, ixgbe.

Di ChenxuX (5):
  net/e1000: release port upon close
  net/fm10k: release port upon close
  net/i40e: release port upon close
  net/ice: release port upon close
  net/ixgbe: release port upon close

 doc/guides/rel_notes/release_19_11.rst |   5 +
 drivers/net/e1000/igb_ethdev.c         | 144 ++++++++++-----------
 drivers/net/fm10k/fm10k_ethdev.c       | 100 ++++++++-------
 drivers/net/i40e/i40e_ethdev.c         | 128 +++++++++----------
 drivers/net/i40e/i40e_ethdev_vf.c      |  25 ++--
 drivers/net/ice/ice_ethdev.c           |  27 ++--
 drivers/net/ixgbe/ixgbe_ethdev.c       | 167 +++++++++++++------------
 7 files changed, 307 insertions(+), 289 deletions(-)

-- 
2.17.1

[dpdk-dev] [PATCH v3 1/5] net/e1000: release port upon close

[Di ChenxuX]

Set RTE_ETH_DEV_CLOSE_REMOVE upon probe so all the private resources
 for the port can be freed by rte_eth_dev_close().

Signed-off-by: Di ChenxuX <chenxux.di@intel.com>
---
 doc/guides/rel_notes/release_19_11.rst |   5 +
 drivers/net/e1000/igb_ethdev.c         | 144 ++++++++++++-------------
 2 files changed, 77 insertions(+), 72 deletions(-)

diff --git a/doc/guides/rel_notes/release_19_11.rst b/doc/guides/rel_notes/release_19_11.rst
index 8490d897c..5d338afff 100644
--- a/doc/guides/rel_notes/release_19_11.rst
+++ b/doc/guides/rel_notes/release_19_11.rst
@@ -56,6 +56,11 @@ New Features
      Also, make sure to start the actual text at the margin.
      =========================================================
 
+* **Updated the Intel drivers.**
+
+  * Added support for the ``RTE_ETH_DEV_CLOSE_REMOVE`` flag for
+    e1000, fm10k, i40e, ice, ixgbe.
+
 
 Removed Items
 -------------
diff --git a/drivers/net/e1000/igb_ethdev.c b/drivers/net/e1000/igb_ethdev.c
index fec2b4289..11c831685 100644
--- a/drivers/net/e1000/igb_ethdev.c
+++ b/drivers/net/e1000/igb_ethdev.c
@@ -843,6 +843,11 @@ eth_igb_dev_init(struct rte_eth_dev *eth_dev)
 	rte_ether_addr_copy((struct rte_ether_addr *)hw->mac.addr,
 			&eth_dev->data->mac_addrs[0]);
 
+	/* Pass the information to the rte_eth_dev_close() that it should also
+	 * release the private port resources.
+	 */
+	eth_dev->data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
+
 	/* initialize the vfta */
 	memset(shadow_vfta, 0, sizeof(*shadow_vfta));
 
@@ -912,61 +917,12 @@ eth_igb_dev_init(struct rte_eth_dev *eth_dev)
 static int
 eth_igb_dev_uninit(struct rte_eth_dev *eth_dev)
 {
-	struct rte_pci_device *pci_dev;
-	struct rte_intr_handle *intr_handle;
-	struct e1000_hw *hw;
-	struct e1000_adapter *adapter =
-		E1000_DEV_PRIVATE(eth_dev->data->dev_private);
-	struct e1000_filter_info *filter_info =
-		E1000_DEV_PRIVATE_TO_FILTER_INFO(eth_dev->data->dev_private);
-
 	PMD_INIT_FUNC_TRACE();
 
 	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
 		return -EPERM;
 
-	hw = E1000_DEV_PRIVATE_TO_HW(eth_dev->data->dev_private);
-	pci_dev = RTE_ETH_DEV_TO_PCI(eth_dev);
-	intr_handle = &pci_dev->intr_handle;
-
-	if (adapter->stopped == 0)
-		eth_igb_close(eth_dev);
-
-	eth_dev->dev_ops = NULL;
-	eth_dev->rx_pkt_burst = NULL;
-	eth_dev->tx_pkt_burst = NULL;
-
-	/* Reset any pending lock */
-	igb_reset_swfw_lock(hw);
-
-	/* uninitialize PF if max_vfs not zero */
-	igb_pf_host_uninit(eth_dev);
-
-	/* disable uio intr before callback unregister */
-	rte_intr_disable(intr_handle);
-	rte_intr_callback_unregister(intr_handle,
-				     eth_igb_interrupt_handler, eth_dev);
-
-	/* clear the SYN filter info */
-	filter_info->syn_info = 0;
-
-	/* clear the ethertype filters info */
-	filter_info->ethertype_mask = 0;
-	memset(filter_info->ethertype_filters, 0,
-		E1000_MAX_ETQF_FILTERS * sizeof(struct igb_ethertype_filter));
-
-	/* clear the rss filter info */
-	memset(&filter_info->rss_info, 0,
-		sizeof(struct igb_rte_flow_rss_conf));
-
-	/* remove all ntuple filters of the device */
-	igb_ntuple_filter_uninit(eth_dev);
-
-	/* remove all flex filters of the device */
-	igb_flex_filter_uninit(eth_dev);
-
-	/* clear all the filters list */
-	igb_filterlist_flush(eth_dev);
+	eth_igb_close(eth_dev);
 
 	return 0;
 }
@@ -1038,6 +994,11 @@ eth_igbvf_dev_init(struct rte_eth_dev *eth_dev)
 		return -ENOMEM;
 	}
 
+	/* Pass the information to the rte_eth_dev_close() that it should also
+	 * release the private port resources.
+	 */
+	eth_dev->data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
+
 	/* Generate a random MAC address, if none was assigned by PF. */
 	if (rte_is_zero_ether_addr(perm_addr)) {
 		rte_eth_random_addr(perm_addr->addr_bytes);
@@ -1077,27 +1038,12 @@ eth_igbvf_dev_init(struct rte_eth_dev *eth_dev)
 static int
 eth_igbvf_dev_uninit(struct rte_eth_dev *eth_dev)
 {
-	struct e1000_adapter *adapter =
-		E1000_DEV_PRIVATE(eth_dev->data->dev_private);
-	struct rte_pci_device *pci_dev = RTE_ETH_DEV_TO_PCI(eth_dev);
-
 	PMD_INIT_FUNC_TRACE();
 
 	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
 		return -EPERM;
 
-	if (adapter->stopped == 0)
-		igbvf_dev_close(eth_dev);
-
-	eth_dev->dev_ops = NULL;
-	eth_dev->rx_pkt_burst = NULL;
-	eth_dev->tx_pkt_burst = NULL;
-
-	/* disable uio intr before callback unregister */
-	rte_intr_disable(&pci_dev->intr_handle);
-	rte_intr_callback_unregister(&pci_dev->intr_handle,
-				     eth_igbvf_interrupt_handler,
-				     (void *)eth_dev);
+	igbvf_dev_close(eth_dev);
 
 	return 0;
 }
@@ -1506,6 +1452,11 @@ eth_igb_stop(struct rte_eth_dev *dev)
 	struct rte_pci_device *pci_dev = RTE_ETH_DEV_TO_PCI(dev);
 	struct rte_eth_link link;
 	struct rte_intr_handle *intr_handle = &pci_dev->intr_handle;
+	struct e1000_adapter *adapter =
+		E1000_DEV_PRIVATE(dev->data->dev_private);
+
+	if (adapter->stopped)
+		return;
 
 	eth_igb_rxtx_control(dev, false);
 
@@ -1547,6 +1498,8 @@ eth_igb_stop(struct rte_eth_dev *dev)
 		rte_free(intr_handle->intr_vec);
 		intr_handle->intr_vec = NULL;
 	}
+
+	adapter->stopped = 1;
 }
 
 static int
@@ -1579,14 +1532,13 @@ static void
 eth_igb_close(struct rte_eth_dev *dev)
 {
 	struct e1000_hw *hw = E1000_DEV_PRIVATE_TO_HW(dev->data->dev_private);
-	struct e1000_adapter *adapter =
-		E1000_DEV_PRIVATE(dev->data->dev_private);
 	struct rte_eth_link link;
 	struct rte_pci_device *pci_dev = RTE_ETH_DEV_TO_PCI(dev);
 	struct rte_intr_handle *intr_handle = &pci_dev->intr_handle;
+	struct e1000_filter_info *filter_info =
+		E1000_DEV_PRIVATE_TO_FILTER_INFO(dev->data->dev_private);
 
 	eth_igb_stop(dev);
-	adapter->stopped = 1;
 
 	e1000_phy_hw_reset(hw);
 	igb_release_manageability(hw);
@@ -1610,6 +1562,40 @@ eth_igb_close(struct rte_eth_dev *dev)
 
 	memset(&link, 0, sizeof(link));
 	rte_eth_linkstatus_set(dev, &link);
+
+	dev->dev_ops = NULL;
+	dev->rx_pkt_burst = NULL;
+	dev->tx_pkt_burst = NULL;
+
+	/* Reset any pending lock */
+	igb_reset_swfw_lock(hw);
+
+	/* uninitialize PF if max_vfs not zero */
+	igb_pf_host_uninit(dev);
+
+	rte_intr_callback_unregister(intr_handle,
+				     eth_igb_interrupt_handler, dev);
+
+	/* clear the SYN filter info */
+	filter_info->syn_info = 0;
+
+	/* clear the ethertype filters info */
+	filter_info->ethertype_mask = 0;
+	memset(filter_info->ethertype_filters, 0,
+		E1000_MAX_ETQF_FILTERS * sizeof(struct igb_ethertype_filter));
+
+	/* clear the rss filter info */
+	memset(&filter_info->rss_info, 0,
+		sizeof(struct igb_rte_flow_rss_conf));
+
+	/* remove all ntuple filters of the device */
+	igb_ntuple_filter_uninit(dev);
+
+	/* remove all flex filters of the device */
+	igb_flex_filter_uninit(dev);
+
+	/* clear all the filters list */
+	igb_filterlist_flush(dev);
 }
 
 /*
@@ -3331,6 +3317,11 @@ igbvf_dev_stop(struct rte_eth_dev *dev)
 {
 	struct rte_pci_device *pci_dev = RTE_ETH_DEV_TO_PCI(dev);
 	struct rte_intr_handle *intr_handle = &pci_dev->intr_handle;
+	struct e1000_adapter *adapter =
+		E1000_DEV_PRIVATE(dev->data->dev_private);
+
+	if (adapter->stopped == 1)
+		return;
 
 	PMD_INIT_FUNC_TRACE();
 
@@ -3353,22 +3344,23 @@ igbvf_dev_stop(struct rte_eth_dev *dev)
 		rte_free(intr_handle->intr_vec);
 		intr_handle->intr_vec = NULL;
 	}
+
+	adapter->stopped = 1;
 }
 
 static void
 igbvf_dev_close(struct rte_eth_dev *dev)
 {
 	struct e1000_hw *hw = E1000_DEV_PRIVATE_TO_HW(dev->data->dev_private);
-	struct e1000_adapter *adapter =
-		E1000_DEV_PRIVATE(dev->data->dev_private);
 	struct rte_ether_addr addr;
+	struct rte_pci_device *pci_dev = RTE_ETH_DEV_TO_PCI(dev);
 
 	PMD_INIT_FUNC_TRACE();
 
 	e1000_reset_hw(hw);
 
 	igbvf_dev_stop(dev);
-	adapter->stopped = 1;
+
 	igb_dev_free_queues(dev);
 
 	/**
@@ -3379,6 +3371,14 @@ igbvf_dev_close(struct rte_eth_dev *dev)
 
 	memset(&addr, 0, sizeof(addr));
 	igbvf_default_mac_addr_set(dev, &addr);
+
+	dev->dev_ops = NULL;
+	dev->rx_pkt_burst = NULL;
+	dev->tx_pkt_burst = NULL;
+
+	rte_intr_callback_unregister(&pci_dev->intr_handle,
+				     eth_igbvf_interrupt_handler,
+				     (void *)dev);
 }
 
 static void
-- 
2.17.1

[dpdk-dev] [PATCH v3 2/5] net/fm10k: release port upon close

[Di ChenxuX]

Set RTE_ETH_DEV_CLOSE_REMOVE upon probe so all the private resources
 for the port can be freed by rte_eth_dev_close().

Signed-off-by: Di ChenxuX <chenxux.di@intel.com>
---
 drivers/net/fm10k/fm10k_ethdev.c | 100 ++++++++++++++++---------------
 1 file changed, 52 insertions(+), 48 deletions(-)

diff --git a/drivers/net/fm10k/fm10k_ethdev.c b/drivers/net/fm10k/fm10k_ethdev.c
index db4d72129..6885cfa4f 100644
--- a/drivers/net/fm10k/fm10k_ethdev.c
+++ b/drivers/net/fm10k/fm10k_ethdev.c
@@ -1194,28 +1194,6 @@ fm10k_dev_queue_release(struct rte_eth_dev *dev)
 	}
 }
 
-static void
-fm10k_dev_close(struct rte_eth_dev *dev)
-{
-	struct fm10k_hw *hw = FM10K_DEV_PRIVATE_TO_HW(dev->data->dev_private);
-
-	PMD_INIT_FUNC_TRACE();
-
-	fm10k_mbx_lock(hw);
-	hw->mac.ops.update_lport_state(hw, hw->mac.dglort_map,
-		MAX_LPORT_NUM, false);
-	fm10k_mbx_unlock(hw);
-
-	/* allow 100ms for device to quiesce */
-	rte_delay_us(FM10K_SWITCH_QUIESCE_US);
-
-	/* Stop mailbox service first */
-	fm10k_close_mbx_service(hw);
-	fm10k_dev_stop(dev);
-	fm10k_dev_queue_release(dev);
-	fm10k_stop_hw(hw);
-}
-
 static int
 fm10k_link_update(struct rte_eth_dev *dev,
 	__rte_unused int wait_to_complete)
@@ -2799,6 +2777,53 @@ fm10k_close_mbx_service(struct fm10k_hw *hw)
 	hw->mbx.ops.disconnect(hw, &hw->mbx);
 }
 
+static void
+fm10k_dev_close(struct rte_eth_dev *dev)
+{
+	struct fm10k_hw *hw = FM10K_DEV_PRIVATE_TO_HW(dev->data->dev_private);
+	struct rte_pci_device *pdev = RTE_ETH_DEV_TO_PCI(dev);
+	struct rte_intr_handle *intr_handle = &pdev->intr_handle;
+
+	PMD_INIT_FUNC_TRACE();
+
+	fm10k_mbx_lock(hw);
+	hw->mac.ops.update_lport_state(hw, hw->mac.dglort_map,
+		MAX_LPORT_NUM, false);
+	fm10k_mbx_unlock(hw);
+
+	/* allow 100ms for device to quiesce */
+	rte_delay_us(FM10K_SWITCH_QUIESCE_US);
+
+	/* Stop mailbox service first */
+	fm10k_close_mbx_service(hw);
+	fm10k_dev_stop(dev);
+	fm10k_dev_queue_release(dev);
+	fm10k_stop_hw(hw);
+
+	dev->dev_ops = NULL;
+	dev->rx_pkt_burst = NULL;
+	dev->tx_pkt_burst = NULL;
+
+	/* disable uio/vfio intr */
+	rte_intr_disable(intr_handle);
+
+	/*PF/VF has different interrupt handling mechanism */
+	if (hw->mac.type == fm10k_mac_pf) {
+		/* disable interrupt */
+		fm10k_dev_disable_intr_pf(dev);
+
+		/* unregister callback func to eal lib */
+		rte_intr_callback_unregister(intr_handle,
+			fm10k_dev_interrupt_handler_pf, (void *)dev);
+	} else {
+		/* disable interrupt */
+		fm10k_dev_disable_intr_vf(dev);
+
+		rte_intr_callback_unregister(intr_handle,
+			fm10k_dev_interrupt_handler_vf, (void *)dev);
+	}
+}
+
 static const struct eth_dev_ops fm10k_eth_dev_ops = {
 	.dev_configure		= fm10k_dev_configure,
 	.dev_start		= fm10k_dev_start,
@@ -3103,6 +3128,11 @@ eth_fm10k_dev_init(struct rte_eth_dev *dev)
 		&dev->data->mac_addrs[0]);
 	}
 
+	/* Pass the information to the rte_eth_dev_close() that it should also
+	 * release the private port resources.
+	 */
+	dev->data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
+
 	/* Reset the hw statistics */
 	fm10k_stats_reset(dev);
 
@@ -3205,9 +3235,6 @@ eth_fm10k_dev_init(struct rte_eth_dev *dev)
 static int
 eth_fm10k_dev_uninit(struct rte_eth_dev *dev)
 {
-	struct fm10k_hw *hw = FM10K_DEV_PRIVATE_TO_HW(dev->data->dev_private);
-	struct rte_pci_device *pdev = RTE_ETH_DEV_TO_PCI(dev);
-	struct rte_intr_handle *intr_handle = &pdev->intr_handle;
 	PMD_INIT_FUNC_TRACE();
 
 	/* only uninitialize in the primary process */
@@ -3217,29 +3244,6 @@ eth_fm10k_dev_uninit(struct rte_eth_dev *dev)
 	/* safe to close dev here */
 	fm10k_dev_close(dev);
 
-	dev->dev_ops = NULL;
-	dev->rx_pkt_burst = NULL;
-	dev->tx_pkt_burst = NULL;
-
-	/* disable uio/vfio intr */
-	rte_intr_disable(intr_handle);
-
-	/*PF/VF has different interrupt handling mechanism */
-	if (hw->mac.type == fm10k_mac_pf) {
-		/* disable interrupt */
-		fm10k_dev_disable_intr_pf(dev);
-
-		/* unregister callback func to eal lib */
-		rte_intr_callback_unregister(intr_handle,
-			fm10k_dev_interrupt_handler_pf, (void *)dev);
-	} else {
-		/* disable interrupt */
-		fm10k_dev_disable_intr_vf(dev);
-
-		rte_intr_callback_unregister(intr_handle,
-			fm10k_dev_interrupt_handler_vf, (void *)dev);
-	}
-
 	return 0;
 }
 
-- 
2.17.1

[dpdk-dev] [PATCH v3 3/5] net/i40e: release port upon close

[Di ChenxuX]

Set RTE_ETH_DEV_CLOSE_REMOVE upon probe so all the private resources
 for the port can be freed by rte_eth_dev_close().

Signed-off-by: Di ChenxuX <chenxux.di@intel.com>
---
 drivers/net/i40e/i40e_ethdev.c    | 128 ++++++++++++++----------------
 drivers/net/i40e/i40e_ethdev_vf.c |  25 +++---
 2 files changed, 77 insertions(+), 76 deletions(-)

diff --git a/drivers/net/i40e/i40e_ethdev.c b/drivers/net/i40e/i40e_ethdev.c
index 4e40b7ab5..f8180c850 100644
--- a/drivers/net/i40e/i40e_ethdev.c
+++ b/drivers/net/i40e/i40e_ethdev.c
@@ -1520,6 +1520,11 @@ eth_i40e_dev_init(struct rte_eth_dev *dev, void *init_params __rte_unused)
 	rte_ether_addr_copy((struct rte_ether_addr *)hw->mac.perm_addr,
 					&dev->data->mac_addrs[0]);
 
+	/* Pass the information to the rte_eth_dev_close() that it should also
+	 * release the private port resources.
+	 */
+	dev->data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
+
 	/* Init dcb to sw mode by default */
 	ret = i40e_dcb_init_configure(dev, TRUE);
 	if (ret != I40E_SUCCESS) {
@@ -1689,85 +1694,18 @@ void i40e_flex_payload_reg_set_default(struct i40e_hw *hw)
 static int
 eth_i40e_dev_uninit(struct rte_eth_dev *dev)
 {
-	struct i40e_pf *pf;
-	struct rte_pci_device *pci_dev;
-	struct rte_intr_handle *intr_handle;
 	struct i40e_hw *hw;
-	struct i40e_filter_control_settings settings;
-	struct rte_flow *p_flow;
-	int ret;
-	uint8_t aq_fail = 0;
-	int retries = 0;
 
 	PMD_INIT_FUNC_TRACE();
 
 	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
 		return 0;
 
-	pf = I40E_DEV_PRIVATE_TO_PF(dev->data->dev_private);
 	hw = I40E_DEV_PRIVATE_TO_HW(dev->data->dev_private);
-	pci_dev = RTE_ETH_DEV_TO_PCI(dev);
-	intr_handle = &pci_dev->intr_handle;
-
-	ret = rte_eth_switch_domain_free(pf->switch_domain_id);
-	if (ret)
-		PMD_INIT_LOG(WARNING, "failed to free switch domain: %d", ret);
 
 	if (hw->adapter_closed == 0)
 		i40e_dev_close(dev);
 
-	dev->dev_ops = NULL;
-	dev->rx_pkt_burst = NULL;
-	dev->tx_pkt_burst = NULL;
-
-	/* Clear PXE mode */
-	i40e_clear_pxe_mode(hw);
-
-	/* Unconfigure filter control */
-	memset(&settings, 0, sizeof(settings));
-	ret = i40e_set_filter_control(hw, &settings);
-	if (ret)
-		PMD_INIT_LOG(WARNING, "setup_pf_filter_control failed: %d",
-					ret);
-
-	/* Disable flow control */
-	hw->fc.requested_mode = I40E_FC_NONE;
-	i40e_set_fc(hw, &aq_fail, TRUE);
-
-	/* uninitialize pf host driver */
-	i40e_pf_host_uninit(dev);
-
-	/* disable uio intr before callback unregister */
-	rte_intr_disable(intr_handle);
-
-	/* unregister callback func to eal lib */
-	do {
-		ret = rte_intr_callback_unregister(intr_handle,
-				i40e_dev_interrupt_handler, dev);
-		if (ret >= 0) {
-			break;
-		} else if (ret != -EAGAIN) {
-			PMD_INIT_LOG(ERR,
-				 "intr callback unregister failed: %d",
-				 ret);
-			return ret;
-		}
-		i40e_msec_delay(500);
-	} while (retries++ < 5);
-
-	i40e_rm_ethtype_filter_list(pf);
-	i40e_rm_tunnel_filter_list(pf);
-	i40e_rm_fdir_filter_list(pf);
-
-	/* Remove all flows */
-	while ((p_flow = TAILQ_FIRST(&pf->flow_list))) {
-		TAILQ_REMOVE(&pf->flow_list, p_flow, node);
-		rte_free(p_flow);
-	}
-
-	/* Remove all Traffic Manager configuration */
-	i40e_tm_conf_uninit(dev);
-
 	return 0;
 }
 
@@ -2465,12 +2403,21 @@ i40e_dev_close(struct rte_eth_dev *dev)
 	struct rte_pci_device *pci_dev = RTE_ETH_DEV_TO_PCI(dev);
 	struct rte_intr_handle *intr_handle = &pci_dev->intr_handle;
 	struct i40e_mirror_rule *p_mirror;
+	struct i40e_filter_control_settings settings;
+	struct rte_flow *p_flow;
 	uint32_t reg;
 	int i;
 	int ret;
+	uint8_t aq_fail = 0;
+	int retries = 0;
 
 	PMD_INIT_FUNC_TRACE();
 
+	ret = rte_eth_switch_domain_free(pf->switch_domain_id);
+	if (ret)
+		PMD_INIT_LOG(WARNING, "failed to free switch domain: %d", ret);
+
+
 	i40e_dev_stop(dev);
 
 	/* Remove all mirror rules */
@@ -2535,6 +2482,53 @@ i40e_dev_close(struct rte_eth_dev *dev)
 			(reg | I40E_PFGEN_CTRL_PFSWR_MASK));
 	I40E_WRITE_FLUSH(hw);
 
+	dev->dev_ops = NULL;
+	dev->rx_pkt_burst = NULL;
+	dev->tx_pkt_burst = NULL;
+
+	/* Clear PXE mode */
+	i40e_clear_pxe_mode(hw);
+
+	/* Unconfigure filter control */
+	memset(&settings, 0, sizeof(settings));
+	ret = i40e_set_filter_control(hw, &settings);
+	if (ret)
+		PMD_INIT_LOG(WARNING, "setup_pf_filter_control failed: %d",
+					ret);
+
+	/* Disable flow control */
+	hw->fc.requested_mode = I40E_FC_NONE;
+	i40e_set_fc(hw, &aq_fail, TRUE);
+
+	/* uninitialize pf host driver */
+	i40e_pf_host_uninit(dev);
+
+	do {
+		ret = rte_intr_callback_unregister(intr_handle,
+				i40e_dev_interrupt_handler, dev);
+		if (ret >= 0) {
+			break;
+		} else if (ret != -EAGAIN) {
+			PMD_INIT_LOG(ERR,
+				 "intr callback unregister failed: %d",
+				 ret);
+		}
+		i40e_msec_delay(500);
+	} while (retries++ < 5);
+
+	i40e_rm_ethtype_filter_list(pf);
+	i40e_rm_tunnel_filter_list(pf);
+	i40e_rm_fdir_filter_list(pf);
+
+	/* Remove all flows */
+	while ((p_flow = TAILQ_FIRST(&pf->flow_list))) {
+		TAILQ_REMOVE(&pf->flow_list, p_flow, node);
+		rte_free(p_flow);
+	}
+
+	/* Remove all Traffic Manager configuration */
+	i40e_tm_conf_uninit(dev);
+
 	hw->adapter_closed = 1;
 }
 
diff --git a/drivers/net/i40e/i40e_ethdev_vf.c b/drivers/net/i40e/i40e_ethdev_vf.c
index 308fb9835..0916e01a6 100644
--- a/drivers/net/i40e/i40e_ethdev_vf.c
+++ b/drivers/net/i40e/i40e_ethdev_vf.c
@@ -1310,17 +1310,12 @@ i40evf_init_vf(struct rte_eth_dev *dev)
 static int
 i40evf_uninit_vf(struct rte_eth_dev *dev)
 {
-	struct i40e_vf *vf = I40EVF_DEV_PRIVATE_TO_VF(dev->data->dev_private);
 	struct i40e_hw *hw = I40E_DEV_PRIVATE_TO_HW(dev->data->dev_private);
 
 	PMD_INIT_FUNC_TRACE();
 
 	if (hw->adapter_closed == 0)
 		i40evf_dev_close(dev);
-	rte_free(vf->vf_res);
-	vf->vf_res = NULL;
-	rte_free(vf->aq_resp);
-	vf->aq_resp = NULL;
 
 	return 0;
 }
@@ -1498,6 +1493,11 @@ i40evf_dev_init(struct rte_eth_dev *eth_dev)
 	hw->adapter_stopped = 0;
 	hw->adapter_closed = 0;
 
+	/* Pass the information to the rte_eth_dev_close() that it should also
+	 * release the private port resources.
+	 */
+	eth_dev->data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
+
 	if(i40evf_init_vf(eth_dev) != 0) {
 		PMD_INIT_LOG(ERR, "Init vf failed");
 		return -1;
@@ -1534,10 +1534,6 @@ i40evf_dev_uninit(struct rte_eth_dev *eth_dev)
 	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
 		return -EPERM;
 
-	eth_dev->dev_ops = NULL;
-	eth_dev->rx_pkt_burst = NULL;
-	eth_dev->tx_pkt_burst = NULL;
-
 	if (i40evf_uninit_vf(eth_dev) != 0) {
 		PMD_INIT_LOG(ERR, "i40evf_uninit_vf failed");
 		return -1;
@@ -2324,6 +2320,7 @@ static void
 i40evf_dev_close(struct rte_eth_dev *dev)
 {
 	struct i40e_hw *hw = I40E_DEV_PRIVATE_TO_HW(dev->data->dev_private);
+	struct i40e_vf *vf = I40EVF_DEV_PRIVATE_TO_VF(dev->data->dev_private);
 
 	i40evf_dev_stop(dev);
 	i40e_dev_free_queues(dev);
@@ -2339,6 +2336,16 @@ i40evf_dev_close(struct rte_eth_dev *dev)
 	i40evf_reset_vf(dev);
 	i40e_shutdown_adminq(hw);
 	i40evf_disable_irq0(hw);
+
+	dev->dev_ops = NULL;
+	dev->rx_pkt_burst = NULL;
+	dev->tx_pkt_burst = NULL;
+
+	rte_free(vf->vf_res);
+	vf->vf_res = NULL;
+	rte_free(vf->aq_resp);
+	vf->aq_resp = NULL;
+
 	hw->adapter_closed = 1;
 }
 
-- 
2.17.1

[dpdk-dev] [PATCH v3 4/5] net/ice: release port upon close

[Di ChenxuX]

Set RTE_ETH_DEV_CLOSE_REMOVE upon probe so all the private resources
 for the port can be freed by rte_eth_dev_close().

Signed-off-by: Di ChenxuX <chenxux.di@intel.com>
---
 drivers/net/ice/ice_ethdev.c | 27 ++++++++++++++++-----------
 1 file changed, 16 insertions(+), 11 deletions(-)

diff --git a/drivers/net/ice/ice_ethdev.c b/drivers/net/ice/ice_ethdev.c
index 44a14cb8a..5d3d57b83 100644
--- a/drivers/net/ice/ice_ethdev.c
+++ b/drivers/net/ice/ice_ethdev.c
@@ -1488,6 +1488,11 @@ ice_dev_init(struct rte_eth_dev *dev)
 		goto err_init_mac;
 	}
 
+	/* Pass the information to the rte_eth_dev_close() that it should also
+	 * release the private port resources.
+	 */
+	dev->data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
+
 	ret = ice_res_pool_init(&pf->msix_pool, 1,
 				hw->func_caps.common_cap.num_msix_vectors - 1);
 	if (ret) {
@@ -1536,6 +1541,8 @@ ice_dev_init(struct rte_eth_dev *dev)
 
 	TAILQ_INIT(&pf->flow_list);
 
+	pf->adapter_stopped = 0;
+
 	return 0;
 
 err_pf_setup:
@@ -1653,6 +1660,9 @@ ice_dev_close(struct rte_eth_dev *dev)
 {
 	struct ice_pf *pf = ICE_DEV_PRIVATE_TO_PF(dev->data->dev_private);
 	struct ice_hw *hw = ICE_DEV_PRIVATE_TO_HW(dev->data->dev_private);
+	struct rte_pci_device *pci_dev = RTE_ETH_DEV_TO_PCI(dev);
+	struct rte_intr_handle *intr_handle = &pci_dev->intr_handle;
+	struct rte_flow *p_flow;
 
 	/* Since stop will make link down, then the link event will be
 	 * triggered, disable the irq firstly to avoid the port_infoe etc
@@ -1672,17 +1682,6 @@ ice_dev_close(struct rte_eth_dev *dev)
 	rte_free(hw->port_info);
 	hw->port_info = NULL;
 	ice_shutdown_all_ctrlq(hw);
-}
-
-static int
-ice_dev_uninit(struct rte_eth_dev *dev)
-{
-	struct rte_pci_device *pci_dev = RTE_ETH_DEV_TO_PCI(dev);
-	struct rte_intr_handle *intr_handle = &pci_dev->intr_handle;
-	struct ice_pf *pf = ICE_DEV_PRIVATE_TO_PF(dev->data->dev_private);
-	struct rte_flow *p_flow;
-
-	ice_dev_close(dev);
 
 	dev->dev_ops = NULL;
 	dev->rx_pkt_burst = NULL;
@@ -1704,6 +1703,12 @@ ice_dev_uninit(struct rte_eth_dev *dev)
 		ice_free_switch_filter_rule(p_flow->rule);
 		rte_free(p_flow);
 	}
+}
+
+static int
+ice_dev_uninit(struct rte_eth_dev *dev)
+{
+	ice_dev_close(dev);
 
 	return 0;
 }
-- 
2.17.1

[dpdk-dev] [PATCH v3 5/5] net/ixgbe: release port upon close

[Di ChenxuX]

Set RTE_ETH_DEV_CLOSE_REMOVE upon probe so all the private resources
 for the port can be freed by rte_eth_dev_close().

Signed-off-by: Di ChenxuX <chenxux.di@intel.com>
---
 drivers/net/ixgbe/ixgbe_ethdev.c | 167 ++++++++++++++++---------------
 1 file changed, 85 insertions(+), 82 deletions(-)

diff --git a/drivers/net/ixgbe/ixgbe_ethdev.c b/drivers/net/ixgbe/ixgbe_ethdev.c
index 03fc1f717..089c184ed 100644
--- a/drivers/net/ixgbe/ixgbe_ethdev.c
+++ b/drivers/net/ixgbe/ixgbe_ethdev.c
@@ -1245,6 +1245,11 @@ eth_ixgbe_dev_init(struct rte_eth_dev *eth_dev, void *init_params __rte_unused)
 		return -ENOMEM;
 	}
 
+	/* Pass the information to the rte_eth_dev_close() that it should also
+	 * release the private port resources.
+	 */
+	eth_dev->data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
+
 	/* initialize the vfta */
 	memset(shadow_vfta, 0, sizeof(*shadow_vfta));
 
@@ -1311,73 +1316,12 @@ eth_ixgbe_dev_init(struct rte_eth_dev *eth_dev, void *init_params __rte_unused)
 static int
 eth_ixgbe_dev_uninit(struct rte_eth_dev *eth_dev)
 {
-	struct rte_pci_device *pci_dev = RTE_ETH_DEV_TO_PCI(eth_dev);
-	struct rte_intr_handle *intr_handle = &pci_dev->intr_handle;
-	struct ixgbe_hw *hw;
-	int retries = 0;
-	int ret;
-
 	PMD_INIT_FUNC_TRACE();
 
 	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
 		return 0;
 
-	hw = IXGBE_DEV_PRIVATE_TO_HW(eth_dev->data->dev_private);
-
-	if (hw->adapter_stopped == 0)
-		ixgbe_dev_close(eth_dev);
-
-	eth_dev->dev_ops = NULL;
-	eth_dev->rx_pkt_burst = NULL;
-	eth_dev->tx_pkt_burst = NULL;
-
-	/* Unlock any pending hardware semaphore */
-	ixgbe_swfw_lock_reset(hw);
-
-	/* disable uio intr before callback unregister */
-	rte_intr_disable(intr_handle);
-
-	do {
-		ret = rte_intr_callback_unregister(intr_handle,
-				ixgbe_dev_interrupt_handler, eth_dev);
-		if (ret >= 0) {
-			break;
-		} else if (ret != -EAGAIN) {
-			PMD_INIT_LOG(ERR,
-				"intr callback unregister failed: %d",
-				ret);
-			return ret;
-		}
-		rte_delay_ms(100);
-	} while (retries++ < (10 + IXGBE_LINK_UP_TIME));
-
-	/* cancel the delay handler before remove dev */
-	rte_eal_alarm_cancel(ixgbe_dev_interrupt_delayed_handler, eth_dev);
-
-	/* cancel the link handler before remove dev */
-	rte_eal_alarm_cancel(ixgbe_dev_setup_link_alarm_handler, eth_dev);
-
-	/* uninitialize PF if max_vfs not zero */
-	ixgbe_pf_host_uninit(eth_dev);
-
-	/* remove all the fdir filters & hash */
-	ixgbe_fdir_filter_uninit(eth_dev);
-
-	/* remove all the L2 tunnel filters & hash */
-	ixgbe_l2_tn_filter_uninit(eth_dev);
-
-	/* Remove all ntuple filters of the device */
-	ixgbe_ntuple_filter_uninit(eth_dev);
-
-	/* clear all the filters list */
-	ixgbe_filterlist_flush();
-
-	/* Remove all Traffic Manager configuration */
-	ixgbe_tm_conf_uninit(eth_dev);
-
-#ifdef RTE_LIBRTE_SECURITY
-	rte_free(eth_dev->security_ctx);
-#endif
+	ixgbe_dev_close(eth_dev);
 
 	return 0;
 }
@@ -1711,6 +1655,11 @@ eth_ixgbevf_dev_init(struct rte_eth_dev *eth_dev)
 		return -ENOMEM;
 	}
 
+	/* Pass the information to the rte_eth_dev_close() that it should also
+	 * release the private port resources.
+	 */
+	eth_dev->data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
+
 	/* Generate a random MAC address, if none was assigned by PF. */
 	if (rte_is_zero_ether_addr(perm_addr)) {
 		generate_random_mac_addr(perm_addr);
@@ -1762,30 +1711,12 @@ eth_ixgbevf_dev_init(struct rte_eth_dev *eth_dev)
 static int
 eth_ixgbevf_dev_uninit(struct rte_eth_dev *eth_dev)
 {
-	struct rte_pci_device *pci_dev = RTE_ETH_DEV_TO_PCI(eth_dev);
-	struct rte_intr_handle *intr_handle = &pci_dev->intr_handle;
-	struct ixgbe_hw *hw;
-
 	PMD_INIT_FUNC_TRACE();
 
 	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
 		return 0;
 
-	hw = IXGBE_DEV_PRIVATE_TO_HW(eth_dev->data->dev_private);
-
-	if (hw->adapter_stopped == 0)
-		ixgbevf_dev_close(eth_dev);
-
-	eth_dev->dev_ops = NULL;
-	eth_dev->rx_pkt_burst = NULL;
-	eth_dev->tx_pkt_burst = NULL;
-
-	/* Disable the interrupts for VF */
-	ixgbevf_intr_disable(eth_dev);
-
-	rte_intr_disable(intr_handle);
-	rte_intr_callback_unregister(intr_handle,
-				     ixgbevf_dev_interrupt_handler, eth_dev);
+	ixgbevf_dev_close(eth_dev);
 
 	return 0;
 }
@@ -2876,6 +2807,9 @@ ixgbe_dev_stop(struct rte_eth_dev *dev)
 	struct ixgbe_tm_conf *tm_conf =
 		IXGBE_DEV_PRIVATE_TO_TM_CONF(dev->data->dev_private);
 
+	if (hw->adapter_stopped == 1)
+		return;
+
 	PMD_INIT_FUNC_TRACE();
 
 	rte_eal_alarm_cancel(ixgbe_dev_setup_link_alarm_handler, dev);
@@ -2928,6 +2862,8 @@ ixgbe_dev_stop(struct rte_eth_dev *dev)
 	tm_conf->committed = false;
 
 	adapter->rss_reta_updated = 0;
+
+	hw->adapter_stopped = 1;
 }
 
 /*
@@ -2998,13 +2934,16 @@ ixgbe_dev_close(struct rte_eth_dev *dev)
 {
 	struct ixgbe_hw *hw =
 		IXGBE_DEV_PRIVATE_TO_HW(dev->data->dev_private);
+	struct rte_pci_device *pci_dev = RTE_ETH_DEV_TO_PCI(dev);
+	struct rte_intr_handle *intr_handle = &pci_dev->intr_handle;
+	int retries = 0;
+	int ret;
 
 	PMD_INIT_FUNC_TRACE();
 
 	ixgbe_pf_reset_hw(hw);
 
 	ixgbe_dev_stop(dev);
-	hw->adapter_stopped = 1;
 
 	ixgbe_dev_free_queues(dev);
 
@@ -3012,6 +2951,55 @@ ixgbe_dev_close(struct rte_eth_dev *dev)
 
 	/* reprogram the RAR[0] in case user changed it. */
 	ixgbe_set_rar(hw, 0, hw->mac.addr, 0, IXGBE_RAH_AV);
+
+	dev->dev_ops = NULL;
+	dev->rx_pkt_burst = NULL;
+	dev->tx_pkt_burst = NULL;
+
+	/* Unlock any pending hardware semaphore */
+	ixgbe_swfw_lock_reset(hw);
+
+	/* disable uio intr before callback unregister */
+	rte_intr_disable(intr_handle);
+
+	do {
+		ret = rte_intr_callback_unregister(intr_handle,
+				ixgbe_dev_interrupt_handler, dev);
+		if (ret >= 0) {
+			break;
+		} else if (ret != -EAGAIN) {
+			PMD_INIT_LOG(ERR,
+				"intr callback unregister failed: %d",
+				ret);
+		}
+		rte_delay_ms(100);
+	} while (retries++ < (10 + IXGBE_LINK_UP_TIME));
+
+	/* cancel the delay handler before remove dev */
+	rte_eal_alarm_cancel(ixgbe_dev_interrupt_delayed_handler, dev);
+
+	/* uninitialize PF if max_vfs not zero */
+	ixgbe_pf_host_uninit(dev);
+
+	/* remove all the fdir filters & hash */
+	ixgbe_fdir_filter_uninit(dev);
+
+	/* remove all the L2 tunnel filters & hash */
+	ixgbe_l2_tn_filter_uninit(dev);
+
+	/* Remove all ntuple filters of the device */
+	ixgbe_ntuple_filter_uninit(dev);
+
+	/* clear all the filters list */
+	ixgbe_filterlist_flush();
+
+	/* Remove all Traffic Manager configuration */
+	ixgbe_tm_conf_uninit(dev);
+
+#ifdef RTE_LIBRTE_SECURITY
+	rte_free(dev->security_ctx);
+#endif
+
 }
 
 /*
@@ -5233,6 +5221,8 @@ ixgbevf_dev_start(struct rte_eth_dev *dev)
 	 */
 	ixgbevf_dev_link_update(dev, 0);
 
+	hw->adapter_stopped = 0;
+
 	return 0;
 }
 
@@ -5244,6 +5234,9 @@ ixgbevf_dev_stop(struct rte_eth_dev *dev)
 	struct rte_pci_device *pci_dev = RTE_ETH_DEV_TO_PCI(dev);
 	struct rte_intr_handle *intr_handle = &pci_dev->intr_handle;
 
+	if (hw->adapter_stopped == 1)
+		return;
+
 	PMD_INIT_FUNC_TRACE();
 
 	rte_eal_alarm_cancel(ixgbe_dev_setup_link_alarm_handler, dev);
@@ -5278,6 +5271,8 @@ static void
 ixgbevf_dev_close(struct rte_eth_dev *dev)
 {
 	struct ixgbe_hw *hw = IXGBE_DEV_PRIVATE_TO_HW(dev->data->dev_private);
+	struct rte_pci_device *pci_dev = RTE_ETH_DEV_TO_PCI(dev);
+	struct rte_intr_handle *intr_handle = &pci_dev->intr_handle;
 
 	PMD_INIT_FUNC_TRACE();
 
@@ -5293,6 +5288,14 @@ ixgbevf_dev_close(struct rte_eth_dev *dev)
 	 * after stop, close and detach of the VF
 	 **/
 	ixgbevf_remove_mac_addr(dev, 0);
+
+	dev->dev_ops = NULL;
+	dev->rx_pkt_burst = NULL;
+	dev->tx_pkt_burst = NULL;
+
+	rte_intr_disable(intr_handle);
+	rte_intr_callback_unregister(intr_handle,
+				     ixgbevf_dev_interrupt_handler, dev);
 }
 
 /*
-- 
2.17.1

[dpdk-dev] [PATCH v4 0/5] drivers/net: release port upon close

[Di ChenxuX]

Set RTE_ETH_DEV_CLOSE_REMOVE upon probe so all the private resources
for the port can be freed by rte_eth_dev_close().
This patch cover all the intel drivers.

---
V4 changes:
  Removed adaper stopped flag in function ice_dev_init.
V3 changes:
  Removed adapter closed flag in driver e1000, fm10k, ice, ixgbe.
V2 changes:
  Separated to 5 patches, each patch only change one driver.
  Added resource release in dev_close.

Di ChenxuX (5):
  net/e1000: release port upon close
  net/fm10k: release port upon close
  net/i40e: release port upon close
  net/ice: release port upon close
  net/ixgbe: release port upon close

 doc/guides/rel_notes/release_19_11.rst |   5 +
 drivers/net/e1000/igb_ethdev.c         | 144 ++++++++++-----------
 drivers/net/fm10k/fm10k_ethdev.c       | 100 ++++++++-------
 drivers/net/i40e/i40e_ethdev.c         | 128 +++++++++----------
 drivers/net/i40e/i40e_ethdev_vf.c      |  25 ++--
 drivers/net/ice/ice_ethdev.c           |  28 ++---
 drivers/net/ixgbe/ixgbe_ethdev.c       | 167 +++++++++++++------------
 7 files changed, 305 insertions(+), 292 deletions(-)

-- 
2.17.1

[dpdk-dev] [PATCH v4 1/5] net/e1000: release port upon close

[Di ChenxuX]

Set RTE_ETH_DEV_CLOSE_REMOVE upon probe so all the private resources
 for the port can be freed by rte_eth_dev_close().

Signed-off-by: Di ChenxuX <chenxux.di@intel.com>
---
 doc/guides/rel_notes/release_19_11.rst |   5 +
 drivers/net/e1000/igb_ethdev.c         | 144 ++++++++++++-------------
 2 files changed, 77 insertions(+), 72 deletions(-)

diff --git a/doc/guides/rel_notes/release_19_11.rst b/doc/guides/rel_notes/release_19_11.rst
index 27cfbd9e3..334756906 100644
--- a/doc/guides/rel_notes/release_19_11.rst
+++ b/doc/guides/rel_notes/release_19_11.rst
@@ -56,6 +56,11 @@ New Features
      Also, make sure to start the actual text at the margin.
      =========================================================
 
+* **Updated the Intel drivers.**
+
+  * Added support for the ``RTE_ETH_DEV_CLOSE_REMOVE`` flag for
+    e1000, fm10k, i40e, ice, ixgbe.
+
 
 Removed Items
 -------------
diff --git a/drivers/net/e1000/igb_ethdev.c b/drivers/net/e1000/igb_ethdev.c
index fec2b4289..c610042c3 100644
--- a/drivers/net/e1000/igb_ethdev.c
+++ b/drivers/net/e1000/igb_ethdev.c
@@ -843,6 +843,11 @@ eth_igb_dev_init(struct rte_eth_dev *eth_dev)
 	rte_ether_addr_copy((struct rte_ether_addr *)hw->mac.addr,
 			&eth_dev->data->mac_addrs[0]);
 
+	/* Pass the information to the rte_eth_dev_close() that it should also
+	 * release the private port resources.
+	 */
+	eth_dev->data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
+
 	/* initialize the vfta */
 	memset(shadow_vfta, 0, sizeof(*shadow_vfta));
 
@@ -912,61 +917,12 @@ eth_igb_dev_init(struct rte_eth_dev *eth_dev)
 static int
 eth_igb_dev_uninit(struct rte_eth_dev *eth_dev)
 {
-	struct rte_pci_device *pci_dev;
-	struct rte_intr_handle *intr_handle;
-	struct e1000_hw *hw;
-	struct e1000_adapter *adapter =
-		E1000_DEV_PRIVATE(eth_dev->data->dev_private);
-	struct e1000_filter_info *filter_info =
-		E1000_DEV_PRIVATE_TO_FILTER_INFO(eth_dev->data->dev_private);
-
 	PMD_INIT_FUNC_TRACE();
 
 	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
 		return -EPERM;
 
-	hw = E1000_DEV_PRIVATE_TO_HW(eth_dev->data->dev_private);
-	pci_dev = RTE_ETH_DEV_TO_PCI(eth_dev);
-	intr_handle = &pci_dev->intr_handle;
-
-	if (adapter->stopped == 0)
-		eth_igb_close(eth_dev);
-
-	eth_dev->dev_ops = NULL;
-	eth_dev->rx_pkt_burst = NULL;
-	eth_dev->tx_pkt_burst = NULL;
-
-	/* Reset any pending lock */
-	igb_reset_swfw_lock(hw);
-
-	/* uninitialize PF if max_vfs not zero */
-	igb_pf_host_uninit(eth_dev);
-
-	/* disable uio intr before callback unregister */
-	rte_intr_disable(intr_handle);
-	rte_intr_callback_unregister(intr_handle,
-				     eth_igb_interrupt_handler, eth_dev);
-
-	/* clear the SYN filter info */
-	filter_info->syn_info = 0;
-
-	/* clear the ethertype filters info */
-	filter_info->ethertype_mask = 0;
-	memset(filter_info->ethertype_filters, 0,
-		E1000_MAX_ETQF_FILTERS * sizeof(struct igb_ethertype_filter));
-
-	/* clear the rss filter info */
-	memset(&filter_info->rss_info, 0,
-		sizeof(struct igb_rte_flow_rss_conf));
-
-	/* remove all ntuple filters of the device */
-	igb_ntuple_filter_uninit(eth_dev);
-
-	/* remove all flex filters of the device */
-	igb_flex_filter_uninit(eth_dev);
-
-	/* clear all the filters list */
-	igb_filterlist_flush(eth_dev);
+	eth_igb_close(eth_dev);
 
 	return 0;
 }
@@ -1038,6 +994,11 @@ eth_igbvf_dev_init(struct rte_eth_dev *eth_dev)
 		return -ENOMEM;
 	}
 
+	/* Pass the information to the rte_eth_dev_close() that it should also
+	 * release the private port resources.
+	 */
+	eth_dev->data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
+
 	/* Generate a random MAC address, if none was assigned by PF. */
 	if (rte_is_zero_ether_addr(perm_addr)) {
 		rte_eth_random_addr(perm_addr->addr_bytes);
@@ -1077,27 +1038,12 @@ eth_igbvf_dev_init(struct rte_eth_dev *eth_dev)
 static int
 eth_igbvf_dev_uninit(struct rte_eth_dev *eth_dev)
 {
-	struct e1000_adapter *adapter =
-		E1000_DEV_PRIVATE(eth_dev->data->dev_private);
-	struct rte_pci_device *pci_dev = RTE_ETH_DEV_TO_PCI(eth_dev);
-
 	PMD_INIT_FUNC_TRACE();
 
 	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
 		return -EPERM;
 
-	if (adapter->stopped == 0)
-		igbvf_dev_close(eth_dev);
-
-	eth_dev->dev_ops = NULL;
-	eth_dev->rx_pkt_burst = NULL;
-	eth_dev->tx_pkt_burst = NULL;
-
-	/* disable uio intr before callback unregister */
-	rte_intr_disable(&pci_dev->intr_handle);
-	rte_intr_callback_unregister(&pci_dev->intr_handle,
-				     eth_igbvf_interrupt_handler,
-				     (void *)eth_dev);
+	igbvf_dev_close(eth_dev);
 
 	return 0;
 }
@@ -1506,6 +1452,11 @@ eth_igb_stop(struct rte_eth_dev *dev)
 	struct rte_pci_device *pci_dev = RTE_ETH_DEV_TO_PCI(dev);
 	struct rte_eth_link link;
 	struct rte_intr_handle *intr_handle = &pci_dev->intr_handle;
+	struct e1000_adapter *adapter =
+		E1000_DEV_PRIVATE(dev->data->dev_private);
+
+	if (adapter->stopped)
+		return;
 
 	eth_igb_rxtx_control(dev, false);
 
@@ -1547,6 +1498,8 @@ eth_igb_stop(struct rte_eth_dev *dev)
 		rte_free(intr_handle->intr_vec);
 		intr_handle->intr_vec = NULL;
 	}
+
+	adapter->stopped = true;
 }
 
 static int
@@ -1579,14 +1532,13 @@ static void
 eth_igb_close(struct rte_eth_dev *dev)
 {
 	struct e1000_hw *hw = E1000_DEV_PRIVATE_TO_HW(dev->data->dev_private);
-	struct e1000_adapter *adapter =
-		E1000_DEV_PRIVATE(dev->data->dev_private);
 	struct rte_eth_link link;
 	struct rte_pci_device *pci_dev = RTE_ETH_DEV_TO_PCI(dev);
 	struct rte_intr_handle *intr_handle = &pci_dev->intr_handle;
+	struct e1000_filter_info *filter_info =
+		E1000_DEV_PRIVATE_TO_FILTER_INFO(dev->data->dev_private);
 
 	eth_igb_stop(dev);
-	adapter->stopped = 1;
 
 	e1000_phy_hw_reset(hw);
 	igb_release_manageability(hw);
@@ -1610,6 +1562,40 @@ eth_igb_close(struct rte_eth_dev *dev)
 
 	memset(&link, 0, sizeof(link));
 	rte_eth_linkstatus_set(dev, &link);
+
+	dev->dev_ops = NULL;
+	dev->rx_pkt_burst = NULL;
+	dev->tx_pkt_burst = NULL;
+
+	/* Reset any pending lock */
+	igb_reset_swfw_lock(hw);
+
+	/* uninitialize PF if max_vfs not zero */
+	igb_pf_host_uninit(dev);
+
+	rte_intr_callback_unregister(intr_handle,
+				     eth_igb_interrupt_handler, dev);
+
+	/* clear the SYN filter info */
+	filter_info->syn_info = 0;
+
+	/* clear the ethertype filters info */
+	filter_info->ethertype_mask = 0;
+	memset(filter_info->ethertype_filters, 0,
+		E1000_MAX_ETQF_FILTERS * sizeof(struct igb_ethertype_filter));
+
+	/* clear the rss filter info */
+	memset(&filter_info->rss_info, 0,
+		sizeof(struct igb_rte_flow_rss_conf));
+
+	/* remove all ntuple filters of the device */
+	igb_ntuple_filter_uninit(dev);
+
+	/* remove all flex filters of the device */
+	igb_flex_filter_uninit(dev);
+
+	/* clear all the filters list */
+	igb_filterlist_flush(dev);
 }
 
 /*
@@ -3331,6 +3317,11 @@ igbvf_dev_stop(struct rte_eth_dev *dev)
 {
 	struct rte_pci_device *pci_dev = RTE_ETH_DEV_TO_PCI(dev);
 	struct rte_intr_handle *intr_handle = &pci_dev->intr_handle;
+	struct e1000_adapter *adapter =
+		E1000_DEV_PRIVATE(dev->data->dev_private);
+
+	if (adapter->stopped)
+		return;
 
 	PMD_INIT_FUNC_TRACE();
 
@@ -3353,22 +3344,23 @@ igbvf_dev_stop(struct rte_eth_dev *dev)
 		rte_free(intr_handle->intr_vec);
 		intr_handle->intr_vec = NULL;
 	}
+
+	adapter->stopped = true;
 }
 
 static void
 igbvf_dev_close(struct rte_eth_dev *dev)
 {
 	struct e1000_hw *hw = E1000_DEV_PRIVATE_TO_HW(dev->data->dev_private);
-	struct e1000_adapter *adapter =
-		E1000_DEV_PRIVATE(dev->data->dev_private);
 	struct rte_ether_addr addr;
+	struct rte_pci_device *pci_dev = RTE_ETH_DEV_TO_PCI(dev);
 
 	PMD_INIT_FUNC_TRACE();
 
 	e1000_reset_hw(hw);
 
 	igbvf_dev_stop(dev);
-	adapter->stopped = 1;
+
 	igb_dev_free_queues(dev);
 
 	/**
@@ -3379,6 +3371,14 @@ igbvf_dev_close(struct rte_eth_dev *dev)
 
 	memset(&addr, 0, sizeof(addr));
 	igbvf_default_mac_addr_set(dev, &addr);
+
+	dev->dev_ops = NULL;
+	dev->rx_pkt_burst = NULL;
+	dev->tx_pkt_burst = NULL;
+
+	rte_intr_callback_unregister(&pci_dev->intr_handle,
+				     eth_igbvf_interrupt_handler,
+				     (void *)dev);
 }
 
 static void
-- 
2.17.1

Re: [dpdk-dev] [PATCH v4 1/5] net/e1000: release port upon close

[Ye Xiaolong]

On 09/19, Di ChenxuX wrote:
>Set RTE_ETH_DEV_CLOSE_REMOVE upon probe so all the private resources
> for the port can be freed by rte_eth_dev_close().
>
>Signed-off-by: Di ChenxuX <chenxux.di@intel.com>
>---
> doc/guides/rel_notes/release_19_11.rst |   5 +
> drivers/net/e1000/igb_ethdev.c         | 144 ++++++++++++-------------

As I commented before, you also need to do the migration for the em_ethdev.c.

> 2 files changed, 77 insertions(+), 72 deletions(-)
>
>diff --git a/doc/guides/rel_notes/release_19_11.rst b/doc/guides/rel_notes/release_19_11.rst
>index 27cfbd9e3..334756906 100644
>--- a/doc/guides/rel_notes/release_19_11.rst
>+++ b/doc/guides/rel_notes/release_19_11.rst
>@@ -56,6 +56,11 @@ New Features
>      Also, make sure to start the actual text at the margin.
>      =========================================================
> 
>+* **Updated the Intel drivers.**
>+
>+  * Added support for the ``RTE_ETH_DEV_CLOSE_REMOVE`` flag for
>+    e1000, fm10k, i40e, ice, ixgbe.
>+

I think it makes more scene to separate release notes in respective patches,
as this patch just update e1000 pmd.

> 
> Removed Items
> -------------
>diff --git a/drivers/net/e1000/igb_ethdev.c b/drivers/net/e1000/igb_ethdev.c
>index fec2b4289..c610042c3 100644
>--- a/drivers/net/e1000/igb_ethdev.c
>+++ b/drivers/net/e1000/igb_ethdev.c
>@@ -843,6 +843,11 @@ eth_igb_dev_init(struct rte_eth_dev *eth_dev)
> 	rte_ether_addr_copy((struct rte_ether_addr *)hw->mac.addr,
> 			&eth_dev->data->mac_addrs[0]);
> 

[snip]

> static int
>@@ -1579,14 +1532,13 @@ static void
> eth_igb_close(struct rte_eth_dev *dev)
> {
> 	struct e1000_hw *hw = E1000_DEV_PRIVATE_TO_HW(dev->data->dev_private);
>-	struct e1000_adapter *adapter =
>-		E1000_DEV_PRIVATE(dev->data->dev_private);
> 	struct rte_eth_link link;
> 	struct rte_pci_device *pci_dev = RTE_ETH_DEV_TO_PCI(dev);
> 	struct rte_intr_handle *intr_handle = &pci_dev->intr_handle;
>+	struct e1000_filter_info *filter_info =
>+		E1000_DEV_PRIVATE_TO_FILTER_INFO(dev->data->dev_private);
> 
> 	eth_igb_stop(dev);
>-	adapter->stopped = 1;
> 
> 	e1000_phy_hw_reset(hw);
> 	igb_release_manageability(hw);
>@@ -1610,6 +1562,40 @@ eth_igb_close(struct rte_eth_dev *dev)
> 
> 	memset(&link, 0, sizeof(link));
> 	rte_eth_linkstatus_set(dev, &link);
>+
>+	dev->dev_ops = NULL;
>+	dev->rx_pkt_burst = NULL;
>+	dev->tx_pkt_burst = NULL;
>+
>+	/* Reset any pending lock */
>+	igb_reset_swfw_lock(hw);
>+
>+	/* uninitialize PF if max_vfs not zero */
>+	igb_pf_host_uninit(dev);
>+
>+	rte_intr_callback_unregister(intr_handle,
>+				     eth_igb_interrupt_handler, dev);

why rte_intr_disable(intr_handle) is missing here?

>+
>+	/* clear the SYN filter info */
>+	filter_info->syn_info = 0;
>+
>+	/* clear the ethertype filters info */
>+	filter_info->ethertype_mask = 0;
>+	memset(filter_info->ethertype_filters, 0,
>+		E1000_MAX_ETQF_FILTERS * sizeof(struct igb_ethertype_filter));
>+
>+	/* clear the rss filter info */
>+	memset(&filter_info->rss_info, 0,
>+		sizeof(struct igb_rte_flow_rss_conf));
>+
>+	/* remove all ntuple filters of the device */
>+	igb_ntuple_filter_uninit(dev);
>+
>+	/* remove all flex filters of the device */
>+	igb_flex_filter_uninit(dev);
>+
>+	/* clear all the filters list */
>+	igb_filterlist_flush(dev);
> }
> 
> /*
>@@ -3331,6 +3317,11 @@ igbvf_dev_stop(struct rte_eth_dev *dev)
> {
> 	struct rte_pci_device *pci_dev = RTE_ETH_DEV_TO_PCI(dev);
> 	struct rte_intr_handle *intr_handle = &pci_dev->intr_handle;
>+	struct e1000_adapter *adapter =
>+		E1000_DEV_PRIVATE(dev->data->dev_private);
>+
>+	if (adapter->stopped)
>+		return;
> 
> 	PMD_INIT_FUNC_TRACE();
> 
>@@ -3353,22 +3344,23 @@ igbvf_dev_stop(struct rte_eth_dev *dev)
> 		rte_free(intr_handle->intr_vec);
> 		intr_handle->intr_vec = NULL;
> 	}
>+
>+	adapter->stopped = true;
> }
> 
> static void
> igbvf_dev_close(struct rte_eth_dev *dev)
> {
> 	struct e1000_hw *hw = E1000_DEV_PRIVATE_TO_HW(dev->data->dev_private);
>-	struct e1000_adapter *adapter =
>-		E1000_DEV_PRIVATE(dev->data->dev_private);
> 	struct rte_ether_addr addr;
>+	struct rte_pci_device *pci_dev = RTE_ETH_DEV_TO_PCI(dev);
> 
> 	PMD_INIT_FUNC_TRACE();
> 
> 	e1000_reset_hw(hw);
> 
> 	igbvf_dev_stop(dev);
>-	adapter->stopped = 1;
>+
> 	igb_dev_free_queues(dev);
> 
> 	/**
>@@ -3379,6 +3371,14 @@ igbvf_dev_close(struct rte_eth_dev *dev)
> 
> 	memset(&addr, 0, sizeof(addr));
> 	igbvf_default_mac_addr_set(dev, &addr);
>+
>+	dev->dev_ops = NULL;
>+	dev->rx_pkt_burst = NULL;
>+	dev->tx_pkt_burst = NULL;
>+
>+	rte_intr_callback_unregister(&pci_dev->intr_handle,
>+				     eth_igbvf_interrupt_handler,
>+				     (void *)dev);

Ditto.

Thanks,
Xiaolong

> }
> 
> static void
>-- 
>2.17.1
>

Re: [dpdk-dev] [PATCH v4 1/5] net/e1000: release port upon close

[Di, ChenxuX]

> -----Original Message-----
> From: Ye, Xiaolong
> Sent: Wednesday, September 25, 2019 11:41 PM
> To: Di, ChenxuX <chenxux.di@intel.com>
> Cc: dev@dpdk.org; Yang, Qiming <qiming.yang@intel.com>
> Subject: Re: [dpdk-dev] [PATCH v4 1/5] net/e1000: release port upon close
> 
> On 09/19, Di ChenxuX wrote:
> >Set RTE_ETH_DEV_CLOSE_REMOVE upon probe so all the private resources
> >for the port can be freed by rte_eth_dev_close().
> >
> >Signed-off-by: Di ChenxuX <chenxux.di@intel.com>
> >---
> > doc/guides/rel_notes/release_19_11.rst |   5 +
> > drivers/net/e1000/igb_ethdev.c         | 144 ++++++++++++-------------
> 
> As I commented before, you also need to do the migration for the em_ethdev.c.

Ok , I will do it in the next version of patch.

> 
> > 2 files changed, 77 insertions(+), 72 deletions(-)
> >
> >diff --git a/doc/guides/rel_notes/release_19_11.rst
> >b/doc/guides/rel_notes/release_19_11.rst
> >index 27cfbd9e3..334756906 100644
> >--- a/doc/guides/rel_notes/release_19_11.rst
> >+++ b/doc/guides/rel_notes/release_19_11.rst
> >@@ -56,6 +56,11 @@ New Features
> >      Also, make sure to start the actual text at the margin.
> >      =========================================================
> >
> >+* **Updated the Intel drivers.**
> >+
> >+  * Added support for the ``RTE_ETH_DEV_CLOSE_REMOVE`` flag for
> >+    e1000, fm10k, i40e, ice, ixgbe.
> >+
> 
> I think it makes more scene to separate release notes in respective patches, as
> this patch just update e1000 pmd.
> 

OK. I will do it in the next version of patch.

> >
> > Removed Items
> > -------------
> >diff --git a/drivers/net/e1000/igb_ethdev.c
> >b/drivers/net/e1000/igb_ethdev.c index fec2b4289..c610042c3 100644
> >--- a/drivers/net/e1000/igb_ethdev.c
> >+++ b/drivers/net/e1000/igb_ethdev.c
> >@@ -843,6 +843,11 @@ eth_igb_dev_init(struct rte_eth_dev *eth_dev)
> > 	rte_ether_addr_copy((struct rte_ether_addr *)hw->mac.addr,
> > 			&eth_dev->data->mac_addrs[0]);
> >
> 
> [snip]
> 
> > static int
> >@@ -1579,14 +1532,13 @@ static void
> > eth_igb_close(struct rte_eth_dev *dev)  {
> > 	struct e1000_hw *hw = E1000_DEV_PRIVATE_TO_HW(dev->data-
> >dev_private);
> >-	struct e1000_adapter *adapter =
> >-		E1000_DEV_PRIVATE(dev->data->dev_private);
> > 	struct rte_eth_link link;
> > 	struct rte_pci_device *pci_dev = RTE_ETH_DEV_TO_PCI(dev);
> > 	struct rte_intr_handle *intr_handle = &pci_dev->intr_handle;
> >+	struct e1000_filter_info *filter_info =
> >+		E1000_DEV_PRIVATE_TO_FILTER_INFO(dev->data-
> >dev_private);
> >
> > 	eth_igb_stop(dev);
> >-	adapter->stopped = 1;
> >
> > 	e1000_phy_hw_reset(hw);
> > 	igb_release_manageability(hw);
> >@@ -1610,6 +1562,40 @@ eth_igb_close(struct rte_eth_dev *dev)
> >
> > 	memset(&link, 0, sizeof(link));
> > 	rte_eth_linkstatus_set(dev, &link);
> >+
> >+	dev->dev_ops = NULL;
> >+	dev->rx_pkt_burst = NULL;
> >+	dev->tx_pkt_burst = NULL;
> >+
> >+	/* Reset any pending lock */
> >+	igb_reset_swfw_lock(hw);
> >+
> >+	/* uninitialize PF if max_vfs not zero */
> >+	igb_pf_host_uninit(dev);
> >+
> >+	rte_intr_callback_unregister(intr_handle,
> >+				     eth_igb_interrupt_handler, dev);
> 
> why rte_intr_disable(intr_handle) is missing here?
> 

rte_intr_disable(intr_handle) has been called in eth_igb_stop(dev). So I delete it to prevent the repeat.

> >+

[snip]
 
> Ditto.
> 
> Thanks,
> Xiaolong
> 
> > }
> >
> > static void
> >--
> >2.17.1
> >

[dpdk-dev] [PATCH v4 2/5] net/fm10k: release port upon close

[Di ChenxuX]

Set RTE_ETH_DEV_CLOSE_REMOVE upon probe so all the private resources
 for the port can be freed by rte_eth_dev_close().

Signed-off-by: Di ChenxuX <chenxux.di@intel.com>
---
 drivers/net/fm10k/fm10k_ethdev.c | 100 ++++++++++++++++---------------
 1 file changed, 52 insertions(+), 48 deletions(-)

diff --git a/drivers/net/fm10k/fm10k_ethdev.c b/drivers/net/fm10k/fm10k_ethdev.c
index db4d72129..6885cfa4f 100644
--- a/drivers/net/fm10k/fm10k_ethdev.c
+++ b/drivers/net/fm10k/fm10k_ethdev.c
@@ -1194,28 +1194,6 @@ fm10k_dev_queue_release(struct rte_eth_dev *dev)
 	}
 }
 
-static void
-fm10k_dev_close(struct rte_eth_dev *dev)
-{
-	struct fm10k_hw *hw = FM10K_DEV_PRIVATE_TO_HW(dev->data->dev_private);
-
-	PMD_INIT_FUNC_TRACE();
-
-	fm10k_mbx_lock(hw);
-	hw->mac.ops.update_lport_state(hw, hw->mac.dglort_map,
-		MAX_LPORT_NUM, false);
-	fm10k_mbx_unlock(hw);
-
-	/* allow 100ms for device to quiesce */
-	rte_delay_us(FM10K_SWITCH_QUIESCE_US);
-
-	/* Stop mailbox service first */
-	fm10k_close_mbx_service(hw);
-	fm10k_dev_stop(dev);
-	fm10k_dev_queue_release(dev);
-	fm10k_stop_hw(hw);
-}
-
 static int
 fm10k_link_update(struct rte_eth_dev *dev,
 	__rte_unused int wait_to_complete)
@@ -2799,6 +2777,53 @@ fm10k_close_mbx_service(struct fm10k_hw *hw)
 	hw->mbx.ops.disconnect(hw, &hw->mbx);
 }
 
+static void
+fm10k_dev_close(struct rte_eth_dev *dev)
+{
+	struct fm10k_hw *hw = FM10K_DEV_PRIVATE_TO_HW(dev->data->dev_private);
+	struct rte_pci_device *pdev = RTE_ETH_DEV_TO_PCI(dev);
+	struct rte_intr_handle *intr_handle = &pdev->intr_handle;
+
+	PMD_INIT_FUNC_TRACE();
+
+	fm10k_mbx_lock(hw);
+	hw->mac.ops.update_lport_state(hw, hw->mac.dglort_map,
+		MAX_LPORT_NUM, false);
+	fm10k_mbx_unlock(hw);
+
+	/* allow 100ms for device to quiesce */
+	rte_delay_us(FM10K_SWITCH_QUIESCE_US);
+
+	/* Stop mailbox service first */
+	fm10k_close_mbx_service(hw);
+	fm10k_dev_stop(dev);
+	fm10k_dev_queue_release(dev);
+	fm10k_stop_hw(hw);
+
+	dev->dev_ops = NULL;
+	dev->rx_pkt_burst = NULL;
+	dev->tx_pkt_burst = NULL;
+
+	/* disable uio/vfio intr */
+	rte_intr_disable(intr_handle);
+
+	/*PF/VF has different interrupt handling mechanism */
+	if (hw->mac.type == fm10k_mac_pf) {
+		/* disable interrupt */
+		fm10k_dev_disable_intr_pf(dev);
+
+		/* unregister callback func to eal lib */
+		rte_intr_callback_unregister(intr_handle,
+			fm10k_dev_interrupt_handler_pf, (void *)dev);
+	} else {
+		/* disable interrupt */
+		fm10k_dev_disable_intr_vf(dev);
+
+		rte_intr_callback_unregister(intr_handle,
+			fm10k_dev_interrupt_handler_vf, (void *)dev);
+	}
+}
+
 static const struct eth_dev_ops fm10k_eth_dev_ops = {
 	.dev_configure		= fm10k_dev_configure,
 	.dev_start		= fm10k_dev_start,
@@ -3103,6 +3128,11 @@ eth_fm10k_dev_init(struct rte_eth_dev *dev)
 		&dev->data->mac_addrs[0]);
 	}
 
+	/* Pass the information to the rte_eth_dev_close() that it should also
+	 * release the private port resources.
+	 */
+	dev->data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
+
 	/* Reset the hw statistics */
 	fm10k_stats_reset(dev);
 
@@ -3205,9 +3235,6 @@ eth_fm10k_dev_init(struct rte_eth_dev *dev)
 static int
 eth_fm10k_dev_uninit(struct rte_eth_dev *dev)
 {
-	struct fm10k_hw *hw = FM10K_DEV_PRIVATE_TO_HW(dev->data->dev_private);
-	struct rte_pci_device *pdev = RTE_ETH_DEV_TO_PCI(dev);
-	struct rte_intr_handle *intr_handle = &pdev->intr_handle;
 	PMD_INIT_FUNC_TRACE();
 
 	/* only uninitialize in the primary process */
@@ -3217,29 +3244,6 @@ eth_fm10k_dev_uninit(struct rte_eth_dev *dev)
 	/* safe to close dev here */
 	fm10k_dev_close(dev);
 
-	dev->dev_ops = NULL;
-	dev->rx_pkt_burst = NULL;
-	dev->tx_pkt_burst = NULL;
-
-	/* disable uio/vfio intr */
-	rte_intr_disable(intr_handle);
-
-	/*PF/VF has different interrupt handling mechanism */
-	if (hw->mac.type == fm10k_mac_pf) {
-		/* disable interrupt */
-		fm10k_dev_disable_intr_pf(dev);
-
-		/* unregister callback func to eal lib */
-		rte_intr_callback_unregister(intr_handle,
-			fm10k_dev_interrupt_handler_pf, (void *)dev);
-	} else {
-		/* disable interrupt */
-		fm10k_dev_disable_intr_vf(dev);
-
-		rte_intr_callback_unregister(intr_handle,
-			fm10k_dev_interrupt_handler_vf, (void *)dev);
-	}
-
 	return 0;
 }
 
-- 
2.17.1

[dpdk-dev] [PATCH v4 3/5] net/i40e: release port upon close

[Di ChenxuX]

Set RTE_ETH_DEV_CLOSE_REMOVE upon probe so all the private resources
 for the port can be freed by rte_eth_dev_close().

Signed-off-by: Di ChenxuX <chenxux.di@intel.com>
---
 drivers/net/i40e/i40e_ethdev.c    | 128 ++++++++++++++----------------
 drivers/net/i40e/i40e_ethdev_vf.c |  25 +++---
 2 files changed, 77 insertions(+), 76 deletions(-)

diff --git a/drivers/net/i40e/i40e_ethdev.c b/drivers/net/i40e/i40e_ethdev.c
index 4e40b7ab5..f8180c850 100644
--- a/drivers/net/i40e/i40e_ethdev.c
+++ b/drivers/net/i40e/i40e_ethdev.c
@@ -1520,6 +1520,11 @@ eth_i40e_dev_init(struct rte_eth_dev *dev, void *init_params __rte_unused)
 	rte_ether_addr_copy((struct rte_ether_addr *)hw->mac.perm_addr,
 					&dev->data->mac_addrs[0]);
 
+	/* Pass the information to the rte_eth_dev_close() that it should also
+	 * release the private port resources.
+	 */
+	dev->data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
+
 	/* Init dcb to sw mode by default */
 	ret = i40e_dcb_init_configure(dev, TRUE);
 	if (ret != I40E_SUCCESS) {
@@ -1689,85 +1694,18 @@ void i40e_flex_payload_reg_set_default(struct i40e_hw *hw)
 static int
 eth_i40e_dev_uninit(struct rte_eth_dev *dev)
 {
-	struct i40e_pf *pf;
-	struct rte_pci_device *pci_dev;
-	struct rte_intr_handle *intr_handle;
 	struct i40e_hw *hw;
-	struct i40e_filter_control_settings settings;
-	struct rte_flow *p_flow;
-	int ret;
-	uint8_t aq_fail = 0;
-	int retries = 0;
 
 	PMD_INIT_FUNC_TRACE();
 
 	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
 		return 0;
 
-	pf = I40E_DEV_PRIVATE_TO_PF(dev->data->dev_private);
 	hw = I40E_DEV_PRIVATE_TO_HW(dev->data->dev_private);
-	pci_dev = RTE_ETH_DEV_TO_PCI(dev);
-	intr_handle = &pci_dev->intr_handle;
-
-	ret = rte_eth_switch_domain_free(pf->switch_domain_id);
-	if (ret)
-		PMD_INIT_LOG(WARNING, "failed to free switch domain: %d", ret);
 
 	if (hw->adapter_closed == 0)
 		i40e_dev_close(dev);
 
-	dev->dev_ops = NULL;
-	dev->rx_pkt_burst = NULL;
-	dev->tx_pkt_burst = NULL;
-
-	/* Clear PXE mode */
-	i40e_clear_pxe_mode(hw);
-
-	/* Unconfigure filter control */
-	memset(&settings, 0, sizeof(settings));
-	ret = i40e_set_filter_control(hw, &settings);
-	if (ret)
-		PMD_INIT_LOG(WARNING, "setup_pf_filter_control failed: %d",
-					ret);
-
-	/* Disable flow control */
-	hw->fc.requested_mode = I40E_FC_NONE;
-	i40e_set_fc(hw, &aq_fail, TRUE);
-
-	/* uninitialize pf host driver */
-	i40e_pf_host_uninit(dev);
-
-	/* disable uio intr before callback unregister */
-	rte_intr_disable(intr_handle);
-
-	/* unregister callback func to eal lib */
-	do {
-		ret = rte_intr_callback_unregister(intr_handle,
-				i40e_dev_interrupt_handler, dev);
-		if (ret >= 0) {
-			break;
-		} else if (ret != -EAGAIN) {
-			PMD_INIT_LOG(ERR,
-				 "intr callback unregister failed: %d",
-				 ret);
-			return ret;
-		}
-		i40e_msec_delay(500);
-	} while (retries++ < 5);
-
-	i40e_rm_ethtype_filter_list(pf);
-	i40e_rm_tunnel_filter_list(pf);
-	i40e_rm_fdir_filter_list(pf);
-
-	/* Remove all flows */
-	while ((p_flow = TAILQ_FIRST(&pf->flow_list))) {
-		TAILQ_REMOVE(&pf->flow_list, p_flow, node);
-		rte_free(p_flow);
-	}
-
-	/* Remove all Traffic Manager configuration */
-	i40e_tm_conf_uninit(dev);
-
 	return 0;
 }
 
@@ -2465,12 +2403,21 @@ i40e_dev_close(struct rte_eth_dev *dev)
 	struct rte_pci_device *pci_dev = RTE_ETH_DEV_TO_PCI(dev);
 	struct rte_intr_handle *intr_handle = &pci_dev->intr_handle;
 	struct i40e_mirror_rule *p_mirror;
+	struct i40e_filter_control_settings settings;
+	struct rte_flow *p_flow;
 	uint32_t reg;
 	int i;
 	int ret;
+	uint8_t aq_fail = 0;
+	int retries = 0;
 
 	PMD_INIT_FUNC_TRACE();
 
+	ret = rte_eth_switch_domain_free(pf->switch_domain_id);
+	if (ret)
+		PMD_INIT_LOG(WARNING, "failed to free switch domain: %d", ret);
+
+
 	i40e_dev_stop(dev);
 
 	/* Remove all mirror rules */
@@ -2535,6 +2482,53 @@ i40e_dev_close(struct rte_eth_dev *dev)
 			(reg | I40E_PFGEN_CTRL_PFSWR_MASK));
 	I40E_WRITE_FLUSH(hw);
 
+	dev->dev_ops = NULL;
+	dev->rx_pkt_burst = NULL;
+	dev->tx_pkt_burst = NULL;
+
+	/* Clear PXE mode */
+	i40e_clear_pxe_mode(hw);
+
+	/* Unconfigure filter control */
+	memset(&settings, 0, sizeof(settings));
+	ret = i40e_set_filter_control(hw, &settings);
+	if (ret)
+		PMD_INIT_LOG(WARNING, "setup_pf_filter_control failed: %d",
+					ret);
+
+	/* Disable flow control */
+	hw->fc.requested_mode = I40E_FC_NONE;
+	i40e_set_fc(hw, &aq_fail, TRUE);
+
+	/* uninitialize pf host driver */
+	i40e_pf_host_uninit(dev);
+
+	do {
+		ret = rte_intr_callback_unregister(intr_handle,
+				i40e_dev_interrupt_handler, dev);
+		if (ret >= 0) {
+			break;
+		} else if (ret != -EAGAIN) {
+			PMD_INIT_LOG(ERR,
+				 "intr callback unregister failed: %d",
+				 ret);
+		}
+		i40e_msec_delay(500);
+	} while (retries++ < 5);
+
+	i40e_rm_ethtype_filter_list(pf);
+	i40e_rm_tunnel_filter_list(pf);
+	i40e_rm_fdir_filter_list(pf);
+
+	/* Remove all flows */
+	while ((p_flow = TAILQ_FIRST(&pf->flow_list))) {
+		TAILQ_REMOVE(&pf->flow_list, p_flow, node);
+		rte_free(p_flow);
+	}
+
+	/* Remove all Traffic Manager configuration */
+	i40e_tm_conf_uninit(dev);
+
 	hw->adapter_closed = 1;
 }
 
diff --git a/drivers/net/i40e/i40e_ethdev_vf.c b/drivers/net/i40e/i40e_ethdev_vf.c
index c77b30c54..354dbcd0d 100644
--- a/drivers/net/i40e/i40e_ethdev_vf.c
+++ b/drivers/net/i40e/i40e_ethdev_vf.c
@@ -1310,17 +1310,12 @@ i40evf_init_vf(struct rte_eth_dev *dev)
 static int
 i40evf_uninit_vf(struct rte_eth_dev *dev)
 {
-	struct i40e_vf *vf = I40EVF_DEV_PRIVATE_TO_VF(dev->data->dev_private);
 	struct i40e_hw *hw = I40E_DEV_PRIVATE_TO_HW(dev->data->dev_private);
 
 	PMD_INIT_FUNC_TRACE();
 
 	if (hw->adapter_closed == 0)
 		i40evf_dev_close(dev);
-	rte_free(vf->vf_res);
-	vf->vf_res = NULL;
-	rte_free(vf->aq_resp);
-	vf->aq_resp = NULL;
 
 	return 0;
 }
@@ -1498,6 +1493,11 @@ i40evf_dev_init(struct rte_eth_dev *eth_dev)
 	hw->adapter_stopped = 0;
 	hw->adapter_closed = 0;
 
+	/* Pass the information to the rte_eth_dev_close() that it should also
+	 * release the private port resources.
+	 */
+	eth_dev->data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
+
 	if(i40evf_init_vf(eth_dev) != 0) {
 		PMD_INIT_LOG(ERR, "Init vf failed");
 		return -1;
@@ -1534,10 +1534,6 @@ i40evf_dev_uninit(struct rte_eth_dev *eth_dev)
 	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
 		return -EPERM;
 
-	eth_dev->dev_ops = NULL;
-	eth_dev->rx_pkt_burst = NULL;
-	eth_dev->tx_pkt_burst = NULL;
-
 	if (i40evf_uninit_vf(eth_dev) != 0) {
 		PMD_INIT_LOG(ERR, "i40evf_uninit_vf failed");
 		return -1;
@@ -2324,6 +2320,7 @@ static void
 i40evf_dev_close(struct rte_eth_dev *dev)
 {
 	struct i40e_hw *hw = I40E_DEV_PRIVATE_TO_HW(dev->data->dev_private);
+	struct i40e_vf *vf = I40EVF_DEV_PRIVATE_TO_VF(dev->data->dev_private);
 
 	i40evf_dev_stop(dev);
 	i40e_dev_free_queues(dev);
@@ -2339,6 +2336,16 @@ i40evf_dev_close(struct rte_eth_dev *dev)
 	i40evf_reset_vf(dev);
 	i40e_shutdown_adminq(hw);
 	i40evf_disable_irq0(hw);
+
+	dev->dev_ops = NULL;
+	dev->rx_pkt_burst = NULL;
+	dev->tx_pkt_burst = NULL;
+
+	rte_free(vf->vf_res);
+	vf->vf_res = NULL;
+	rte_free(vf->aq_resp);
+	vf->aq_resp = NULL;
+
 	hw->adapter_closed = 1;
 }
 
-- 
2.17.1

[dpdk-dev] [PATCH v4 4/5] net/ice: release port upon close

[Di ChenxuX]

Set RTE_ETH_DEV_CLOSE_REMOVE upon probe so all the private resources
 for the port can be freed by rte_eth_dev_close().

Signed-off-by: Di ChenxuX <chenxux.di@intel.com>
---
 drivers/net/ice/ice_ethdev.c | 28 ++++++++++++++--------------
 1 file changed, 14 insertions(+), 14 deletions(-)

diff --git a/drivers/net/ice/ice_ethdev.c b/drivers/net/ice/ice_ethdev.c
index 63997fdfb..7be454bcc 100644
--- a/drivers/net/ice/ice_ethdev.c
+++ b/drivers/net/ice/ice_ethdev.c
@@ -1497,6 +1497,11 @@ ice_dev_init(struct rte_eth_dev *dev)
 		goto err_init_mac;
 	}
 
+	/* Pass the information to the rte_eth_dev_close() that it should also
+	 * release the private port resources.
+	 */
+	dev->data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
+
 	ret = ice_res_pool_init(&pf->msix_pool, 1,
 				hw->func_caps.common_cap.num_msix_vectors - 1);
 	if (ret) {
@@ -1662,6 +1667,9 @@ ice_dev_close(struct rte_eth_dev *dev)
 {
 	struct ice_pf *pf = ICE_DEV_PRIVATE_TO_PF(dev->data->dev_private);
 	struct ice_hw *hw = ICE_DEV_PRIVATE_TO_HW(dev->data->dev_private);
+	struct rte_pci_device *pci_dev = RTE_ETH_DEV_TO_PCI(dev);
+	struct rte_intr_handle *intr_handle = &pci_dev->intr_handle;
+	struct rte_flow *p_flow;
 
 	/* Since stop will make link down, then the link event will be
 	 * triggered, disable the irq firstly to avoid the port_infoe etc
@@ -1681,20 +1689,6 @@ ice_dev_close(struct rte_eth_dev *dev)
 	rte_free(hw->port_info);
 	hw->port_info = NULL;
 	ice_shutdown_all_ctrlq(hw);
-}
-
-static int
-ice_dev_uninit(struct rte_eth_dev *dev)
-{
-	struct rte_pci_device *pci_dev = RTE_ETH_DEV_TO_PCI(dev);
-	struct rte_intr_handle *intr_handle = &pci_dev->intr_handle;
-	struct ice_pf *pf = ICE_DEV_PRIVATE_TO_PF(dev->data->dev_private);
-	struct rte_flow *p_flow;
-
-	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
-		return 0;
-
-	ice_dev_close(dev);
 
 	dev->dev_ops = NULL;
 	dev->rx_pkt_burst = NULL;
@@ -1716,6 +1710,12 @@ ice_dev_uninit(struct rte_eth_dev *dev)
 		ice_free_switch_filter_rule(p_flow->rule);
 		rte_free(p_flow);
 	}
+}
+
+static int
+ice_dev_uninit(struct rte_eth_dev *dev)
+{
+	ice_dev_close(dev);
 
 	return 0;
 }
-- 
2.17.1

[dpdk-dev] [PATCH v4 5/5] net/ixgbe: release port upon close

[Di ChenxuX]

Set RTE_ETH_DEV_CLOSE_REMOVE upon probe so all the private resources
 for the port can be freed by rte_eth_dev_close().

Signed-off-by: Di ChenxuX <chenxux.di@intel.com>
---
 drivers/net/ixgbe/ixgbe_ethdev.c | 167 ++++++++++++++++---------------
 1 file changed, 85 insertions(+), 82 deletions(-)

diff --git a/drivers/net/ixgbe/ixgbe_ethdev.c b/drivers/net/ixgbe/ixgbe_ethdev.c
index 7eb3d0567..edc702570 100644
--- a/drivers/net/ixgbe/ixgbe_ethdev.c
+++ b/drivers/net/ixgbe/ixgbe_ethdev.c
@@ -1245,6 +1245,11 @@ eth_ixgbe_dev_init(struct rte_eth_dev *eth_dev, void *init_params __rte_unused)
 		return -ENOMEM;
 	}
 
+	/* Pass the information to the rte_eth_dev_close() that it should also
+	 * release the private port resources.
+	 */
+	eth_dev->data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
+
 	/* initialize the vfta */
 	memset(shadow_vfta, 0, sizeof(*shadow_vfta));
 
@@ -1311,73 +1316,12 @@ eth_ixgbe_dev_init(struct rte_eth_dev *eth_dev, void *init_params __rte_unused)
 static int
 eth_ixgbe_dev_uninit(struct rte_eth_dev *eth_dev)
 {
-	struct rte_pci_device *pci_dev = RTE_ETH_DEV_TO_PCI(eth_dev);
-	struct rte_intr_handle *intr_handle = &pci_dev->intr_handle;
-	struct ixgbe_hw *hw;
-	int retries = 0;
-	int ret;
-
 	PMD_INIT_FUNC_TRACE();
 
 	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
 		return 0;
 
-	hw = IXGBE_DEV_PRIVATE_TO_HW(eth_dev->data->dev_private);
-
-	if (hw->adapter_stopped == 0)
-		ixgbe_dev_close(eth_dev);
-
-	eth_dev->dev_ops = NULL;
-	eth_dev->rx_pkt_burst = NULL;
-	eth_dev->tx_pkt_burst = NULL;
-
-	/* Unlock any pending hardware semaphore */
-	ixgbe_swfw_lock_reset(hw);
-
-	/* disable uio intr before callback unregister */
-	rte_intr_disable(intr_handle);
-
-	do {
-		ret = rte_intr_callback_unregister(intr_handle,
-				ixgbe_dev_interrupt_handler, eth_dev);
-		if (ret >= 0) {
-			break;
-		} else if (ret != -EAGAIN) {
-			PMD_INIT_LOG(ERR,
-				"intr callback unregister failed: %d",
-				ret);
-			return ret;
-		}
-		rte_delay_ms(100);
-	} while (retries++ < (10 + IXGBE_LINK_UP_TIME));
-
-	/* cancel the delay handler before remove dev */
-	rte_eal_alarm_cancel(ixgbe_dev_interrupt_delayed_handler, eth_dev);
-
-	/* cancel the link handler before remove dev */
-	rte_eal_alarm_cancel(ixgbe_dev_setup_link_alarm_handler, eth_dev);
-
-	/* uninitialize PF if max_vfs not zero */
-	ixgbe_pf_host_uninit(eth_dev);
-
-	/* remove all the fdir filters & hash */
-	ixgbe_fdir_filter_uninit(eth_dev);
-
-	/* remove all the L2 tunnel filters & hash */
-	ixgbe_l2_tn_filter_uninit(eth_dev);
-
-	/* Remove all ntuple filters of the device */
-	ixgbe_ntuple_filter_uninit(eth_dev);
-
-	/* clear all the filters list */
-	ixgbe_filterlist_flush();
-
-	/* Remove all Traffic Manager configuration */
-	ixgbe_tm_conf_uninit(eth_dev);
-
-#ifdef RTE_LIBRTE_SECURITY
-	rte_free(eth_dev->security_ctx);
-#endif
+	ixgbe_dev_close(eth_dev);
 
 	return 0;
 }
@@ -1711,6 +1655,11 @@ eth_ixgbevf_dev_init(struct rte_eth_dev *eth_dev)
 		return -ENOMEM;
 	}
 
+	/* Pass the information to the rte_eth_dev_close() that it should also
+	 * release the private port resources.
+	 */
+	eth_dev->data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
+
 	/* Generate a random MAC address, if none was assigned by PF. */
 	if (rte_is_zero_ether_addr(perm_addr)) {
 		generate_random_mac_addr(perm_addr);
@@ -1762,30 +1711,12 @@ eth_ixgbevf_dev_init(struct rte_eth_dev *eth_dev)
 static int
 eth_ixgbevf_dev_uninit(struct rte_eth_dev *eth_dev)
 {
-	struct rte_pci_device *pci_dev = RTE_ETH_DEV_TO_PCI(eth_dev);
-	struct rte_intr_handle *intr_handle = &pci_dev->intr_handle;
-	struct ixgbe_hw *hw;
-
 	PMD_INIT_FUNC_TRACE();
 
 	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
 		return 0;
 
-	hw = IXGBE_DEV_PRIVATE_TO_HW(eth_dev->data->dev_private);
-
-	if (hw->adapter_stopped == 0)
-		ixgbevf_dev_close(eth_dev);
-
-	eth_dev->dev_ops = NULL;
-	eth_dev->rx_pkt_burst = NULL;
-	eth_dev->tx_pkt_burst = NULL;
-
-	/* Disable the interrupts for VF */
-	ixgbevf_intr_disable(eth_dev);
-
-	rte_intr_disable(intr_handle);
-	rte_intr_callback_unregister(intr_handle,
-				     ixgbevf_dev_interrupt_handler, eth_dev);
+	ixgbevf_dev_close(eth_dev);
 
 	return 0;
 }
@@ -2876,6 +2807,9 @@ ixgbe_dev_stop(struct rte_eth_dev *dev)
 	struct ixgbe_tm_conf *tm_conf =
 		IXGBE_DEV_PRIVATE_TO_TM_CONF(dev->data->dev_private);
 
+	if (hw->adapter_stopped)
+		return;
+
 	PMD_INIT_FUNC_TRACE();
 
 	rte_eal_alarm_cancel(ixgbe_dev_setup_link_alarm_handler, dev);
@@ -2928,6 +2862,8 @@ ixgbe_dev_stop(struct rte_eth_dev *dev)
 	tm_conf->committed = false;
 
 	adapter->rss_reta_updated = 0;
+
+	hw->adapter_stopped = true;
 }
 
 /*
@@ -2998,13 +2934,16 @@ ixgbe_dev_close(struct rte_eth_dev *dev)
 {
 	struct ixgbe_hw *hw =
 		IXGBE_DEV_PRIVATE_TO_HW(dev->data->dev_private);
+	struct rte_pci_device *pci_dev = RTE_ETH_DEV_TO_PCI(dev);
+	struct rte_intr_handle *intr_handle = &pci_dev->intr_handle;
+	int retries = 0;
+	int ret;
 
 	PMD_INIT_FUNC_TRACE();
 
 	ixgbe_pf_reset_hw(hw);
 
 	ixgbe_dev_stop(dev);
-	hw->adapter_stopped = 1;
 
 	ixgbe_dev_free_queues(dev);
 
@@ -3012,6 +2951,55 @@ ixgbe_dev_close(struct rte_eth_dev *dev)
 
 	/* reprogram the RAR[0] in case user changed it. */
 	ixgbe_set_rar(hw, 0, hw->mac.addr, 0, IXGBE_RAH_AV);
+
+	dev->dev_ops = NULL;
+	dev->rx_pkt_burst = NULL;
+	dev->tx_pkt_burst = NULL;
+
+	/* Unlock any pending hardware semaphore */
+	ixgbe_swfw_lock_reset(hw);
+
+	/* disable uio intr before callback unregister */
+	rte_intr_disable(intr_handle);
+
+	do {
+		ret = rte_intr_callback_unregister(intr_handle,
+				ixgbe_dev_interrupt_handler, dev);
+		if (ret >= 0) {
+			break;
+		} else if (ret != -EAGAIN) {
+			PMD_INIT_LOG(ERR,
+				"intr callback unregister failed: %d",
+				ret);
+		}
+		rte_delay_ms(100);
+	} while (retries++ < (10 + IXGBE_LINK_UP_TIME));
+
+	/* cancel the delay handler before remove dev */
+	rte_eal_alarm_cancel(ixgbe_dev_interrupt_delayed_handler, dev);
+
+	/* uninitialize PF if max_vfs not zero */
+	ixgbe_pf_host_uninit(dev);
+
+	/* remove all the fdir filters & hash */
+	ixgbe_fdir_filter_uninit(dev);
+
+	/* remove all the L2 tunnel filters & hash */
+	ixgbe_l2_tn_filter_uninit(dev);
+
+	/* Remove all ntuple filters of the device */
+	ixgbe_ntuple_filter_uninit(dev);
+
+	/* clear all the filters list */
+	ixgbe_filterlist_flush();
+
+	/* Remove all Traffic Manager configuration */
+	ixgbe_tm_conf_uninit(dev);
+
+#ifdef RTE_LIBRTE_SECURITY
+	rte_free(dev->security_ctx);
+#endif
+
 }
 
 /*
@@ -5233,6 +5221,8 @@ ixgbevf_dev_start(struct rte_eth_dev *dev)
 	 */
 	ixgbevf_dev_link_update(dev, 0);
 
+	hw->adapter_stopped = false;
+
 	return 0;
 }
 
@@ -5244,6 +5234,9 @@ ixgbevf_dev_stop(struct rte_eth_dev *dev)
 	struct rte_pci_device *pci_dev = RTE_ETH_DEV_TO_PCI(dev);
 	struct rte_intr_handle *intr_handle = &pci_dev->intr_handle;
 
+	if (hw->adapter_stopped)
+		return;
+
 	PMD_INIT_FUNC_TRACE();
 
 	rte_eal_alarm_cancel(ixgbe_dev_setup_link_alarm_handler, dev);
@@ -5278,6 +5271,8 @@ static void
 ixgbevf_dev_close(struct rte_eth_dev *dev)
 {
 	struct ixgbe_hw *hw = IXGBE_DEV_PRIVATE_TO_HW(dev->data->dev_private);
+	struct rte_pci_device *pci_dev = RTE_ETH_DEV_TO_PCI(dev);
+	struct rte_intr_handle *intr_handle = &pci_dev->intr_handle;
 
 	PMD_INIT_FUNC_TRACE();
 
@@ -5293,6 +5288,14 @@ ixgbevf_dev_close(struct rte_eth_dev *dev)
 	 * after stop, close and detach of the VF
 	 **/
 	ixgbevf_remove_mac_addr(dev, 0);
+
+	dev->dev_ops = NULL;
+	dev->rx_pkt_burst = NULL;
+	dev->tx_pkt_burst = NULL;
+
+	rte_intr_disable(intr_handle);
+	rte_intr_callback_unregister(intr_handle,
+				     ixgbevf_dev_interrupt_handler, dev);
 }
 
 /*
-- 
2.17.1

Re: [dpdk-dev] [PATCH v4 0/5] drivers/net: release port upon close

[Yang, Qiming]

> -----Original Message-----
> From: Di, ChenxuX
> Sent: Thursday, September 19, 2019 10:48 AM
> To: dev@dpdk.org
> Cc: Yang, Qiming <qiming.yang@intel.com>; Di, ChenxuX
> <chenxux.di@intel.com>
> Subject: [PATCH v4 0/5] drivers/net: release port upon close
> 
> Set RTE_ETH_DEV_CLOSE_REMOVE upon probe so all the private resources
> for the port can be freed by rte_eth_dev_close().
> This patch cover all the intel drivers.
> 
> ---
> V4 changes:
>   Removed adaper stopped flag in function ice_dev_init.
> V3 changes:
>   Removed adapter closed flag in driver e1000, fm10k, ice, ixgbe.
> V2 changes:
>   Separated to 5 patches, each patch only change one driver.
>   Added resource release in dev_close.
> 
> Di ChenxuX (5):
>   net/e1000: release port upon close
>   net/fm10k: release port upon close
>   net/i40e: release port upon close
>   net/ice: release port upon close
>   net/ixgbe: release port upon close
> 
>  doc/guides/rel_notes/release_19_11.rst |   5 +
>  drivers/net/e1000/igb_ethdev.c         | 144 ++++++++++-----------
>  drivers/net/fm10k/fm10k_ethdev.c       | 100 ++++++++-------
>  drivers/net/i40e/i40e_ethdev.c         | 128 +++++++++----------
>  drivers/net/i40e/i40e_ethdev_vf.c      |  25 ++--
>  drivers/net/ice/ice_ethdev.c           |  28 ++---
>  drivers/net/ixgbe/ixgbe_ethdev.c       | 167 +++++++++++++------------
>  7 files changed, 305 insertions(+), 292 deletions(-)
> 
> --
> 2.17.1

Acked-by: Qiming Yang <qiming.yang@intel.com>

[dpdk-dev] [PATCH v5 0/5] drivers/net: release port upon close

[Di ChenxuX]

Set RTE_ETH_DEV_CLOSE_REMOVE upon probe so all the private resources
 for the port can be freed by rte_eth_dev_close().
This patch cover all the intel drivers.

---
V5 changes:
  Added code in em_ethdev.c.
  Separated the release note, each patch has its own note. 
V4 changes:
  Removed adaper stopped flag in function ice_dev_init.
V3 changes:
  Removed adapter closed flag in driver e1000, fm10k, ice, ixgbe.
V2 changes:
  Separated to 5 patches, each patch only change one driver.
  Added resource release in dev_close.

Di ChenxuX (5):
  net/e1000: release port upon close
  net/fm10k: release port upon close
  net/i40e: release port upon close
  net/ice: release port upon close
  net/ixgbe: release port upon close

 doc/guides/rel_notes/release_19_11.rst |  18 +++
 drivers/net/e1000/em_ethdev.c          |  33 ++---
 drivers/net/e1000/igb_ethdev.c         | 144 ++++++++++-----------
 drivers/net/fm10k/fm10k_ethdev.c       | 100 ++++++++-------
 drivers/net/i40e/i40e_ethdev.c         | 128 +++++++++----------
 drivers/net/i40e/i40e_ethdev_vf.c      |  25 ++--
 drivers/net/ice/ice_ethdev.c           |  30 +++--
 drivers/net/ixgbe/ixgbe_ethdev.c       | 167 +++++++++++++------------
 8 files changed, 335 insertions(+), 310 deletions(-)

-- 
2.17.1

[dpdk-dev] [PATCH v5 1/5] net/e1000: release port upon close

[Di ChenxuX]

Set RTE_ETH_DEV_CLOSE_REMOVE upon probe so all the private resources
 for the port can be freed by rte_eth_dev_close().

Signed-off-by: Di ChenxuX <chenxux.di@intel.com>
---
 doc/guides/rel_notes/release_19_11.rst |   5 +
 drivers/net/e1000/em_ethdev.c          |  33 +++---
 drivers/net/e1000/igb_ethdev.c         | 144 ++++++++++++-------------
 3 files changed, 94 insertions(+), 88 deletions(-)

diff --git a/doc/guides/rel_notes/release_19_11.rst b/doc/guides/rel_notes/release_19_11.rst
index 04cbc6e5f..99de7b0a3 100644
--- a/doc/guides/rel_notes/release_19_11.rst
+++ b/doc/guides/rel_notes/release_19_11.rst
@@ -64,6 +64,11 @@ New Features
   * Added support for handling Receive Flex Descriptor.
   * Added support for protocol extraction on per Rx queue.
 
+* **Updated the Intel e1000 driver.**
+
+  Added support for the ``RTE_ETH_DEV_CLOSE_REMOVE`` flag.
+
+
 Removed Items
 -------------
 
diff --git a/drivers/net/e1000/em_ethdev.c b/drivers/net/e1000/em_ethdev.c
index 8fe39176a..664f9f43d 100644
--- a/drivers/net/e1000/em_ethdev.c
+++ b/drivers/net/e1000/em_ethdev.c
@@ -297,6 +297,11 @@ eth_em_dev_init(struct rte_eth_dev *eth_dev)
 	rte_ether_addr_copy((struct rte_ether_addr *)hw->mac.addr,
 		eth_dev->data->mac_addrs);
 
+	/* Pass the information to the rte_eth_dev_close() that it should also
+	 * release the private port resources.
+	 */
+	eth_dev->data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
+
 	/* initialize the vfta */
 	memset(shadow_vfta, 0, sizeof(*shadow_vfta));
 
@@ -313,27 +318,12 @@ eth_em_dev_init(struct rte_eth_dev *eth_dev)
 static int
 eth_em_dev_uninit(struct rte_eth_dev *eth_dev)
 {
-	struct rte_pci_device *pci_dev = RTE_ETH_DEV_TO_PCI(eth_dev);
-	struct e1000_adapter *adapter =
-		E1000_DEV_PRIVATE(eth_dev->data->dev_private);
-	struct rte_intr_handle *intr_handle = &pci_dev->intr_handle;
-
 	PMD_INIT_FUNC_TRACE();
 
 	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
 		return -EPERM;
 
-	if (adapter->stopped == 0)
-		eth_em_close(eth_dev);
-
-	eth_dev->dev_ops = NULL;
-	eth_dev->rx_pkt_burst = NULL;
-	eth_dev->tx_pkt_burst = NULL;
-
-	/* disable uio intr before callback unregister */
-	rte_intr_disable(intr_handle);
-	rte_intr_callback_unregister(intr_handle,
-				     eth_em_interrupt_handler, eth_dev);
+	eth_em_close(eth_dev);
 
 	return 0;
 }
@@ -774,6 +764,8 @@ eth_em_close(struct rte_eth_dev *dev)
 	struct e1000_hw *hw = E1000_DEV_PRIVATE_TO_HW(dev->data->dev_private);
 	struct e1000_adapter *adapter =
 		E1000_DEV_PRIVATE(dev->data->dev_private);
+	struct rte_pci_device *pci_dev = RTE_ETH_DEV_TO_PCI(eth_dev);
+	struct rte_intr_handle *intr_handle = &pci_dev->intr_handle;
 
 	eth_em_stop(dev);
 	adapter->stopped = 1;
@@ -781,6 +773,15 @@ eth_em_close(struct rte_eth_dev *dev)
 	e1000_phy_hw_reset(hw);
 	em_release_manageability(hw);
 	em_hw_control_release(hw);
+
+	eth_dev->dev_ops = NULL;
+	eth_dev->rx_pkt_burst = NULL;
+	eth_dev->tx_pkt_burst = NULL;
+
+	/* disable uio intr before callback unregister */
+	rte_intr_disable(intr_handle);
+	rte_intr_callback_unregister(intr_handle,
+				     eth_em_interrupt_handler, eth_dev);
 }
 
 static int
diff --git a/drivers/net/e1000/igb_ethdev.c b/drivers/net/e1000/igb_ethdev.c
index a9f6de5d5..3f742193d 100644
--- a/drivers/net/e1000/igb_ethdev.c
+++ b/drivers/net/e1000/igb_ethdev.c
@@ -843,6 +843,11 @@ eth_igb_dev_init(struct rte_eth_dev *eth_dev)
 	rte_ether_addr_copy((struct rte_ether_addr *)hw->mac.addr,
 			&eth_dev->data->mac_addrs[0]);
 
+	/* Pass the information to the rte_eth_dev_close() that it should also
+	 * release the private port resources.
+	 */
+	eth_dev->data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
+
 	/* initialize the vfta */
 	memset(shadow_vfta, 0, sizeof(*shadow_vfta));
 
@@ -912,61 +917,12 @@ eth_igb_dev_init(struct rte_eth_dev *eth_dev)
 static int
 eth_igb_dev_uninit(struct rte_eth_dev *eth_dev)
 {
-	struct rte_pci_device *pci_dev;
-	struct rte_intr_handle *intr_handle;
-	struct e1000_hw *hw;
-	struct e1000_adapter *adapter =
-		E1000_DEV_PRIVATE(eth_dev->data->dev_private);
-	struct e1000_filter_info *filter_info =
-		E1000_DEV_PRIVATE_TO_FILTER_INFO(eth_dev->data->dev_private);
-
 	PMD_INIT_FUNC_TRACE();
 
 	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
 		return -EPERM;
 
-	hw = E1000_DEV_PRIVATE_TO_HW(eth_dev->data->dev_private);
-	pci_dev = RTE_ETH_DEV_TO_PCI(eth_dev);
-	intr_handle = &pci_dev->intr_handle;
-
-	if (adapter->stopped == 0)
-		eth_igb_close(eth_dev);
-
-	eth_dev->dev_ops = NULL;
-	eth_dev->rx_pkt_burst = NULL;
-	eth_dev->tx_pkt_burst = NULL;
-
-	/* Reset any pending lock */
-	igb_reset_swfw_lock(hw);
-
-	/* uninitialize PF if max_vfs not zero */
-	igb_pf_host_uninit(eth_dev);
-
-	/* disable uio intr before callback unregister */
-	rte_intr_disable(intr_handle);
-	rte_intr_callback_unregister(intr_handle,
-				     eth_igb_interrupt_handler, eth_dev);
-
-	/* clear the SYN filter info */
-	filter_info->syn_info = 0;
-
-	/* clear the ethertype filters info */
-	filter_info->ethertype_mask = 0;
-	memset(filter_info->ethertype_filters, 0,
-		E1000_MAX_ETQF_FILTERS * sizeof(struct igb_ethertype_filter));
-
-	/* clear the rss filter info */
-	memset(&filter_info->rss_info, 0,
-		sizeof(struct igb_rte_flow_rss_conf));
-
-	/* remove all ntuple filters of the device */
-	igb_ntuple_filter_uninit(eth_dev);
-
-	/* remove all flex filters of the device */
-	igb_flex_filter_uninit(eth_dev);
-
-	/* clear all the filters list */
-	igb_filterlist_flush(eth_dev);
+	eth_igb_close(eth_dev);
 
 	return 0;
 }
@@ -1038,6 +994,11 @@ eth_igbvf_dev_init(struct rte_eth_dev *eth_dev)
 		return -ENOMEM;
 	}
 
+	/* Pass the information to the rte_eth_dev_close() that it should also
+	 * release the private port resources.
+	 */
+	eth_dev->data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
+
 	/* Generate a random MAC address, if none was assigned by PF. */
 	if (rte_is_zero_ether_addr(perm_addr)) {
 		rte_eth_random_addr(perm_addr->addr_bytes);
@@ -1077,27 +1038,12 @@ eth_igbvf_dev_init(struct rte_eth_dev *eth_dev)
 static int
 eth_igbvf_dev_uninit(struct rte_eth_dev *eth_dev)
 {
-	struct e1000_adapter *adapter =
-		E1000_DEV_PRIVATE(eth_dev->data->dev_private);
-	struct rte_pci_device *pci_dev = RTE_ETH_DEV_TO_PCI(eth_dev);
-
 	PMD_INIT_FUNC_TRACE();
 
 	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
 		return -EPERM;
 
-	if (adapter->stopped == 0)
-		igbvf_dev_close(eth_dev);
-
-	eth_dev->dev_ops = NULL;
-	eth_dev->rx_pkt_burst = NULL;
-	eth_dev->tx_pkt_burst = NULL;
-
-	/* disable uio intr before callback unregister */
-	rte_intr_disable(&pci_dev->intr_handle);
-	rte_intr_callback_unregister(&pci_dev->intr_handle,
-				     eth_igbvf_interrupt_handler,
-				     (void *)eth_dev);
+	igbvf_dev_close(eth_dev);
 
 	return 0;
 }
@@ -1506,6 +1452,11 @@ eth_igb_stop(struct rte_eth_dev *dev)
 	struct rte_pci_device *pci_dev = RTE_ETH_DEV_TO_PCI(dev);
 	struct rte_eth_link link;
 	struct rte_intr_handle *intr_handle = &pci_dev->intr_handle;
+	struct e1000_adapter *adapter =
+		E1000_DEV_PRIVATE(dev->data->dev_private);
+
+	if (adapter->stopped)
+		return;
 
 	eth_igb_rxtx_control(dev, false);
 
@@ -1547,6 +1498,8 @@ eth_igb_stop(struct rte_eth_dev *dev)
 		rte_free(intr_handle->intr_vec);
 		intr_handle->intr_vec = NULL;
 	}
+
+	adapter->stopped = true;
 }
 
 static int
@@ -1579,14 +1532,13 @@ static void
 eth_igb_close(struct rte_eth_dev *dev)
 {
 	struct e1000_hw *hw = E1000_DEV_PRIVATE_TO_HW(dev->data->dev_private);
-	struct e1000_adapter *adapter =
-		E1000_DEV_PRIVATE(dev->data->dev_private);
 	struct rte_eth_link link;
 	struct rte_pci_device *pci_dev = RTE_ETH_DEV_TO_PCI(dev);
 	struct rte_intr_handle *intr_handle = &pci_dev->intr_handle;
+	struct e1000_filter_info *filter_info =
+		E1000_DEV_PRIVATE_TO_FILTER_INFO(dev->data->dev_private);
 
 	eth_igb_stop(dev);
-	adapter->stopped = 1;
 
 	e1000_phy_hw_reset(hw);
 	igb_release_manageability(hw);
@@ -1610,6 +1562,40 @@ eth_igb_close(struct rte_eth_dev *dev)
 
 	memset(&link, 0, sizeof(link));
 	rte_eth_linkstatus_set(dev, &link);
+
+	dev->dev_ops = NULL;
+	dev->rx_pkt_burst = NULL;
+	dev->tx_pkt_burst = NULL;
+
+	/* Reset any pending lock */
+	igb_reset_swfw_lock(hw);
+
+	/* uninitialize PF if max_vfs not zero */
+	igb_pf_host_uninit(dev);
+
+	rte_intr_callback_unregister(intr_handle,
+				     eth_igb_interrupt_handler, dev);
+
+	/* clear the SYN filter info */
+	filter_info->syn_info = 0;
+
+	/* clear the ethertype filters info */
+	filter_info->ethertype_mask = 0;
+	memset(filter_info->ethertype_filters, 0,
+		E1000_MAX_ETQF_FILTERS * sizeof(struct igb_ethertype_filter));
+
+	/* clear the rss filter info */
+	memset(&filter_info->rss_info, 0,
+		sizeof(struct igb_rte_flow_rss_conf));
+
+	/* remove all ntuple filters of the device */
+	igb_ntuple_filter_uninit(dev);
+
+	/* remove all flex filters of the device */
+	igb_flex_filter_uninit(dev);
+
+	/* clear all the filters list */
+	igb_filterlist_flush(dev);
 }
 
 /*
@@ -3344,6 +3330,11 @@ igbvf_dev_stop(struct rte_eth_dev *dev)
 {
 	struct rte_pci_device *pci_dev = RTE_ETH_DEV_TO_PCI(dev);
 	struct rte_intr_handle *intr_handle = &pci_dev->intr_handle;
+	struct e1000_adapter *adapter =
+		E1000_DEV_PRIVATE(dev->data->dev_private);
+
+	if (adapter->stopped)
+		return;
 
 	PMD_INIT_FUNC_TRACE();
 
@@ -3366,22 +3357,23 @@ igbvf_dev_stop(struct rte_eth_dev *dev)
 		rte_free(intr_handle->intr_vec);
 		intr_handle->intr_vec = NULL;
 	}
+
+	adapter->stopped = true;
 }
 
 static void
 igbvf_dev_close(struct rte_eth_dev *dev)
 {
 	struct e1000_hw *hw = E1000_DEV_PRIVATE_TO_HW(dev->data->dev_private);
-	struct e1000_adapter *adapter =
-		E1000_DEV_PRIVATE(dev->data->dev_private);
 	struct rte_ether_addr addr;
+	struct rte_pci_device *pci_dev = RTE_ETH_DEV_TO_PCI(dev);
 
 	PMD_INIT_FUNC_TRACE();
 
 	e1000_reset_hw(hw);
 
 	igbvf_dev_stop(dev);
-	adapter->stopped = 1;
+
 	igb_dev_free_queues(dev);
 
 	/**
@@ -3392,6 +3384,14 @@ igbvf_dev_close(struct rte_eth_dev *dev)
 
 	memset(&addr, 0, sizeof(addr));
 	igbvf_default_mac_addr_set(dev, &addr);
+
+	dev->dev_ops = NULL;
+	dev->rx_pkt_burst = NULL;
+	dev->tx_pkt_burst = NULL;
+
+	rte_intr_callback_unregister(&pci_dev->intr_handle,
+				     eth_igbvf_interrupt_handler,
+				     (void *)dev);
 }
 
 static int
-- 
2.17.1

[dpdk-dev] [PATCH v5 2/5] net/fm10k: release port upon close

[Di ChenxuX]

Set RTE_ETH_DEV_CLOSE_REMOVE upon probe so all the private resources
 for the port can be freed by rte_eth_dev_close().

Signed-off-by: Di ChenxuX <chenxux.di@intel.com>
---
 doc/guides/rel_notes/release_19_11.rst |   4 +
 drivers/net/fm10k/fm10k_ethdev.c       | 100 +++++++++++++------------
 2 files changed, 56 insertions(+), 48 deletions(-)

diff --git a/doc/guides/rel_notes/release_19_11.rst b/doc/guides/rel_notes/release_19_11.rst
index 99de7b0a3..177cd9b4a 100644
--- a/doc/guides/rel_notes/release_19_11.rst
+++ b/doc/guides/rel_notes/release_19_11.rst
@@ -68,6 +68,10 @@ New Features
 
   Added support for the ``RTE_ETH_DEV_CLOSE_REMOVE`` flag.
 
+* **Updated the Intel fm10k driver.**
+
+  Added support for the ``RTE_ETH_DEV_CLOSE_REMOVE`` flag.
+
 
 Removed Items
 -------------
diff --git a/drivers/net/fm10k/fm10k_ethdev.c b/drivers/net/fm10k/fm10k_ethdev.c
index f0f629008..f3048b6a6 100644
--- a/drivers/net/fm10k/fm10k_ethdev.c
+++ b/drivers/net/fm10k/fm10k_ethdev.c
@@ -1202,28 +1202,6 @@ fm10k_dev_queue_release(struct rte_eth_dev *dev)
 	}
 }
 
-static void
-fm10k_dev_close(struct rte_eth_dev *dev)
-{
-	struct fm10k_hw *hw = FM10K_DEV_PRIVATE_TO_HW(dev->data->dev_private);
-
-	PMD_INIT_FUNC_TRACE();
-
-	fm10k_mbx_lock(hw);
-	hw->mac.ops.update_lport_state(hw, hw->mac.dglort_map,
-		MAX_LPORT_NUM, false);
-	fm10k_mbx_unlock(hw);
-
-	/* allow 100ms for device to quiesce */
-	rte_delay_us(FM10K_SWITCH_QUIESCE_US);
-
-	/* Stop mailbox service first */
-	fm10k_close_mbx_service(hw);
-	fm10k_dev_stop(dev);
-	fm10k_dev_queue_release(dev);
-	fm10k_stop_hw(hw);
-}
-
 static int
 fm10k_link_update(struct rte_eth_dev *dev,
 	__rte_unused int wait_to_complete)
@@ -2809,6 +2787,53 @@ fm10k_close_mbx_service(struct fm10k_hw *hw)
 	hw->mbx.ops.disconnect(hw, &hw->mbx);
 }
 
+static void
+fm10k_dev_close(struct rte_eth_dev *dev)
+{
+	struct fm10k_hw *hw = FM10K_DEV_PRIVATE_TO_HW(dev->data->dev_private);
+	struct rte_pci_device *pdev = RTE_ETH_DEV_TO_PCI(dev);
+	struct rte_intr_handle *intr_handle = &pdev->intr_handle;
+
+	PMD_INIT_FUNC_TRACE();
+
+	fm10k_mbx_lock(hw);
+	hw->mac.ops.update_lport_state(hw, hw->mac.dglort_map,
+		MAX_LPORT_NUM, false);
+	fm10k_mbx_unlock(hw);
+
+	/* allow 100ms for device to quiesce */
+	rte_delay_us(FM10K_SWITCH_QUIESCE_US);
+
+	/* Stop mailbox service first */
+	fm10k_close_mbx_service(hw);
+	fm10k_dev_stop(dev);
+	fm10k_dev_queue_release(dev);
+	fm10k_stop_hw(hw);
+
+	dev->dev_ops = NULL;
+	dev->rx_pkt_burst = NULL;
+	dev->tx_pkt_burst = NULL;
+
+	/* disable uio/vfio intr */
+	rte_intr_disable(intr_handle);
+
+	/*PF/VF has different interrupt handling mechanism */
+	if (hw->mac.type == fm10k_mac_pf) {
+		/* disable interrupt */
+		fm10k_dev_disable_intr_pf(dev);
+
+		/* unregister callback func to eal lib */
+		rte_intr_callback_unregister(intr_handle,
+			fm10k_dev_interrupt_handler_pf, (void *)dev);
+	} else {
+		/* disable interrupt */
+		fm10k_dev_disable_intr_vf(dev);
+
+		rte_intr_callback_unregister(intr_handle,
+			fm10k_dev_interrupt_handler_vf, (void *)dev);
+	}
+}
+
 static const struct eth_dev_ops fm10k_eth_dev_ops = {
 	.dev_configure		= fm10k_dev_configure,
 	.dev_start		= fm10k_dev_start,
@@ -3113,6 +3138,11 @@ eth_fm10k_dev_init(struct rte_eth_dev *dev)
 		&dev->data->mac_addrs[0]);
 	}
 
+	/* Pass the information to the rte_eth_dev_close() that it should also
+	 * release the private port resources.
+	 */
+	dev->data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
+
 	/* Reset the hw statistics */
 	fm10k_stats_reset(dev);
 
@@ -3215,9 +3245,6 @@ eth_fm10k_dev_init(struct rte_eth_dev *dev)
 static int
 eth_fm10k_dev_uninit(struct rte_eth_dev *dev)
 {
-	struct fm10k_hw *hw = FM10K_DEV_PRIVATE_TO_HW(dev->data->dev_private);
-	struct rte_pci_device *pdev = RTE_ETH_DEV_TO_PCI(dev);
-	struct rte_intr_handle *intr_handle = &pdev->intr_handle;
 	PMD_INIT_FUNC_TRACE();
 
 	/* only uninitialize in the primary process */
@@ -3227,29 +3254,6 @@ eth_fm10k_dev_uninit(struct rte_eth_dev *dev)
 	/* safe to close dev here */
 	fm10k_dev_close(dev);
 
-	dev->dev_ops = NULL;
-	dev->rx_pkt_burst = NULL;
-	dev->tx_pkt_burst = NULL;
-
-	/* disable uio/vfio intr */
-	rte_intr_disable(intr_handle);
-
-	/*PF/VF has different interrupt handling mechanism */
-	if (hw->mac.type == fm10k_mac_pf) {
-		/* disable interrupt */
-		fm10k_dev_disable_intr_pf(dev);
-
-		/* unregister callback func to eal lib */
-		rte_intr_callback_unregister(intr_handle,
-			fm10k_dev_interrupt_handler_pf, (void *)dev);
-	} else {
-		/* disable interrupt */
-		fm10k_dev_disable_intr_vf(dev);
-
-		rte_intr_callback_unregister(intr_handle,
-			fm10k_dev_interrupt_handler_vf, (void *)dev);
-	}
-
 	return 0;
 }
 
-- 
2.17.1

[dpdk-dev] [PATCH v5 3/5] net/i40e: release port upon close

[Di ChenxuX]

Set RTE_ETH_DEV_CLOSE_REMOVE upon probe so all the private resources
 for the port can be freed by rte_eth_dev_close().

Signed-off-by: Di ChenxuX <chenxux.di@intel.com>
---
 doc/guides/rel_notes/release_19_11.rst |   4 +
 drivers/net/i40e/i40e_ethdev.c         | 128 ++++++++++++-------------
 drivers/net/i40e/i40e_ethdev_vf.c      |  25 +++--
 3 files changed, 81 insertions(+), 76 deletions(-)

diff --git a/doc/guides/rel_notes/release_19_11.rst b/doc/guides/rel_notes/release_19_11.rst
index 177cd9b4a..7e3290665 100644
--- a/doc/guides/rel_notes/release_19_11.rst
+++ b/doc/guides/rel_notes/release_19_11.rst
@@ -72,6 +72,10 @@ New Features
 
   Added support for the ``RTE_ETH_DEV_CLOSE_REMOVE`` flag.
 
+* **Updated the Intel i40e driver.**
+
+  Added support for the ``RTE_ETH_DEV_CLOSE_REMOVE`` flag.
+
 
 Removed Items
 -------------
diff --git a/drivers/net/i40e/i40e_ethdev.c b/drivers/net/i40e/i40e_ethdev.c
index 8b9e23659..4953d3aaa 100644
--- a/drivers/net/i40e/i40e_ethdev.c
+++ b/drivers/net/i40e/i40e_ethdev.c
@@ -1590,6 +1590,11 @@ eth_i40e_dev_init(struct rte_eth_dev *dev, void *init_params __rte_unused)
 	rte_ether_addr_copy((struct rte_ether_addr *)hw->mac.perm_addr,
 					&dev->data->mac_addrs[0]);
 
+	/* Pass the information to the rte_eth_dev_close() that it should also
+	 * release the private port resources.
+	 */
+	dev->data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
+
 	/* Init dcb to sw mode by default */
 	ret = i40e_dcb_init_configure(dev, TRUE);
 	if (ret != I40E_SUCCESS) {
@@ -1759,85 +1764,18 @@ void i40e_flex_payload_reg_set_default(struct i40e_hw *hw)
 static int
 eth_i40e_dev_uninit(struct rte_eth_dev *dev)
 {
-	struct i40e_pf *pf;
-	struct rte_pci_device *pci_dev;
-	struct rte_intr_handle *intr_handle;
 	struct i40e_hw *hw;
-	struct i40e_filter_control_settings settings;
-	struct rte_flow *p_flow;
-	int ret;
-	uint8_t aq_fail = 0;
-	int retries = 0;
 
 	PMD_INIT_FUNC_TRACE();
 
 	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
 		return 0;
 
-	pf = I40E_DEV_PRIVATE_TO_PF(dev->data->dev_private);
 	hw = I40E_DEV_PRIVATE_TO_HW(dev->data->dev_private);
-	pci_dev = RTE_ETH_DEV_TO_PCI(dev);
-	intr_handle = &pci_dev->intr_handle;
-
-	ret = rte_eth_switch_domain_free(pf->switch_domain_id);
-	if (ret)
-		PMD_INIT_LOG(WARNING, "failed to free switch domain: %d", ret);
 
 	if (hw->adapter_closed == 0)
 		i40e_dev_close(dev);
 
-	dev->dev_ops = NULL;
-	dev->rx_pkt_burst = NULL;
-	dev->tx_pkt_burst = NULL;
-
-	/* Clear PXE mode */
-	i40e_clear_pxe_mode(hw);
-
-	/* Unconfigure filter control */
-	memset(&settings, 0, sizeof(settings));
-	ret = i40e_set_filter_control(hw, &settings);
-	if (ret)
-		PMD_INIT_LOG(WARNING, "setup_pf_filter_control failed: %d",
-					ret);
-
-	/* Disable flow control */
-	hw->fc.requested_mode = I40E_FC_NONE;
-	i40e_set_fc(hw, &aq_fail, TRUE);
-
-	/* uninitialize pf host driver */
-	i40e_pf_host_uninit(dev);
-
-	/* disable uio intr before callback unregister */
-	rte_intr_disable(intr_handle);
-
-	/* unregister callback func to eal lib */
-	do {
-		ret = rte_intr_callback_unregister(intr_handle,
-				i40e_dev_interrupt_handler, dev);
-		if (ret >= 0) {
-			break;
-		} else if (ret != -EAGAIN) {
-			PMD_INIT_LOG(ERR,
-				 "intr callback unregister failed: %d",
-				 ret);
-			return ret;
-		}
-		i40e_msec_delay(500);
-	} while (retries++ < 5);
-
-	i40e_rm_ethtype_filter_list(pf);
-	i40e_rm_tunnel_filter_list(pf);
-	i40e_rm_fdir_filter_list(pf);
-
-	/* Remove all flows */
-	while ((p_flow = TAILQ_FIRST(&pf->flow_list))) {
-		TAILQ_REMOVE(&pf->flow_list, p_flow, node);
-		rte_free(p_flow);
-	}
-
-	/* Remove all Traffic Manager configuration */
-	i40e_tm_conf_uninit(dev);
-
 	return 0;
 }
 
@@ -2535,12 +2473,21 @@ i40e_dev_close(struct rte_eth_dev *dev)
 	struct rte_pci_device *pci_dev = RTE_ETH_DEV_TO_PCI(dev);
 	struct rte_intr_handle *intr_handle = &pci_dev->intr_handle;
 	struct i40e_mirror_rule *p_mirror;
+	struct i40e_filter_control_settings settings;
+	struct rte_flow *p_flow;
 	uint32_t reg;
 	int i;
 	int ret;
+	uint8_t aq_fail = 0;
+	int retries = 0;
 
 	PMD_INIT_FUNC_TRACE();
 
+	ret = rte_eth_switch_domain_free(pf->switch_domain_id);
+	if (ret)
+		PMD_INIT_LOG(WARNING, "failed to free switch domain: %d", ret);
+
+
 	i40e_dev_stop(dev);
 
 	/* Remove all mirror rules */
@@ -2605,6 +2552,53 @@ i40e_dev_close(struct rte_eth_dev *dev)
 			(reg | I40E_PFGEN_CTRL_PFSWR_MASK));
 	I40E_WRITE_FLUSH(hw);
 
+	dev->dev_ops = NULL;
+	dev->rx_pkt_burst = NULL;
+	dev->tx_pkt_burst = NULL;
+
+	/* Clear PXE mode */
+	i40e_clear_pxe_mode(hw);
+
+	/* Unconfigure filter control */
+	memset(&settings, 0, sizeof(settings));
+	ret = i40e_set_filter_control(hw, &settings);
+	if (ret)
+		PMD_INIT_LOG(WARNING, "setup_pf_filter_control failed: %d",
+					ret);
+
+	/* Disable flow control */
+	hw->fc.requested_mode = I40E_FC_NONE;
+	i40e_set_fc(hw, &aq_fail, TRUE);
+
+	/* uninitialize pf host driver */
+	i40e_pf_host_uninit(dev);
+
+	do {
+		ret = rte_intr_callback_unregister(intr_handle,
+				i40e_dev_interrupt_handler, dev);
+		if (ret >= 0) {
+			break;
+		} else if (ret != -EAGAIN) {
+			PMD_INIT_LOG(ERR,
+				 "intr callback unregister failed: %d",
+				 ret);
+		}
+		i40e_msec_delay(500);
+	} while (retries++ < 5);
+
+	i40e_rm_ethtype_filter_list(pf);
+	i40e_rm_tunnel_filter_list(pf);
+	i40e_rm_fdir_filter_list(pf);
+
+	/* Remove all flows */
+	while ((p_flow = TAILQ_FIRST(&pf->flow_list))) {
+		TAILQ_REMOVE(&pf->flow_list, p_flow, node);
+		rte_free(p_flow);
+	}
+
+	/* Remove all Traffic Manager configuration */
+	i40e_tm_conf_uninit(dev);
+
 	hw->adapter_closed = 1;
 }
 
diff --git a/drivers/net/i40e/i40e_ethdev_vf.c b/drivers/net/i40e/i40e_ethdev_vf.c
index 22e5e1409..4395a011e 100644
--- a/drivers/net/i40e/i40e_ethdev_vf.c
+++ b/drivers/net/i40e/i40e_ethdev_vf.c
@@ -1310,17 +1310,12 @@ i40evf_init_vf(struct rte_eth_dev *dev)
 static int
 i40evf_uninit_vf(struct rte_eth_dev *dev)
 {
-	struct i40e_vf *vf = I40EVF_DEV_PRIVATE_TO_VF(dev->data->dev_private);
 	struct i40e_hw *hw = I40E_DEV_PRIVATE_TO_HW(dev->data->dev_private);
 
 	PMD_INIT_FUNC_TRACE();
 
 	if (hw->adapter_closed == 0)
 		i40evf_dev_close(dev);
-	rte_free(vf->vf_res);
-	vf->vf_res = NULL;
-	rte_free(vf->aq_resp);
-	vf->aq_resp = NULL;
 
 	return 0;
 }
@@ -1498,6 +1493,11 @@ i40evf_dev_init(struct rte_eth_dev *eth_dev)
 	hw->adapter_stopped = 0;
 	hw->adapter_closed = 0;
 
+	/* Pass the information to the rte_eth_dev_close() that it should also
+	 * release the private port resources.
+	 */
+	eth_dev->data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
+
 	if(i40evf_init_vf(eth_dev) != 0) {
 		PMD_INIT_LOG(ERR, "Init vf failed");
 		return -1;
@@ -1534,10 +1534,6 @@ i40evf_dev_uninit(struct rte_eth_dev *eth_dev)
 	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
 		return -EPERM;
 
-	eth_dev->dev_ops = NULL;
-	eth_dev->rx_pkt_burst = NULL;
-	eth_dev->tx_pkt_burst = NULL;
-
 	if (i40evf_uninit_vf(eth_dev) != 0) {
 		PMD_INIT_LOG(ERR, "i40evf_uninit_vf failed");
 		return -1;
@@ -2334,6 +2330,7 @@ static void
 i40evf_dev_close(struct rte_eth_dev *dev)
 {
 	struct i40e_hw *hw = I40E_DEV_PRIVATE_TO_HW(dev->data->dev_private);
+	struct i40e_vf *vf = I40EVF_DEV_PRIVATE_TO_VF(dev->data->dev_private);
 
 	i40evf_dev_stop(dev);
 	i40e_dev_free_queues(dev);
@@ -2349,6 +2346,16 @@ i40evf_dev_close(struct rte_eth_dev *dev)
 	i40evf_reset_vf(dev);
 	i40e_shutdown_adminq(hw);
 	i40evf_disable_irq0(hw);
+
+	dev->dev_ops = NULL;
+	dev->rx_pkt_burst = NULL;
+	dev->tx_pkt_burst = NULL;
+
+	rte_free(vf->vf_res);
+	vf->vf_res = NULL;
+	rte_free(vf->aq_resp);
+	vf->aq_resp = NULL;
+
 	hw->adapter_closed = 1;
 }
 
-- 
2.17.1

[dpdk-dev] [PATCH v5 4/5] net/ice: release port upon close

[Di ChenxuX]

Set RTE_ETH_DEV_CLOSE_REMOVE upon probe so all the private resources
 for the port can be freed by rte_eth_dev_close().

Signed-off-by: Di ChenxuX <chenxux.di@intel.com>
---
 doc/guides/rel_notes/release_19_11.rst |  1 +
 drivers/net/ice/ice_ethdev.c           | 30 ++++++++++++--------------
 2 files changed, 15 insertions(+), 16 deletions(-)

diff --git a/doc/guides/rel_notes/release_19_11.rst b/doc/guides/rel_notes/release_19_11.rst
index 7e3290665..751606440 100644
--- a/doc/guides/rel_notes/release_19_11.rst
+++ b/doc/guides/rel_notes/release_19_11.rst
@@ -63,6 +63,7 @@ New Features
   * Added support for device-specific DDP package loading.
   * Added support for handling Receive Flex Descriptor.
   * Added support for protocol extraction on per Rx queue.
+  * Added support for the ``RTE_ETH_DEV_CLOSE_REMOVE`` flag.
 
 * **Updated the Intel e1000 driver.**
 
diff --git a/drivers/net/ice/ice_ethdev.c b/drivers/net/ice/ice_ethdev.c
index 0a514dc10..d3f4ebf38 100644
--- a/drivers/net/ice/ice_ethdev.c
+++ b/drivers/net/ice/ice_ethdev.c
@@ -1924,6 +1924,11 @@ ice_dev_init(struct rte_eth_dev *dev)
 		goto err_init_mac;
 	}
 
+	/* Pass the information to the rte_eth_dev_close() that it should also
+	 * release the private port resources.
+	 */
+	dev->data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
+
 	ret = ice_res_pool_init(&pf->msix_pool, 1,
 				hw->func_caps.common_cap.num_msix_vectors - 1);
 	if (ret) {
@@ -2090,6 +2095,9 @@ ice_dev_close(struct rte_eth_dev *dev)
 {
 	struct ice_pf *pf = ICE_DEV_PRIVATE_TO_PF(dev->data->dev_private);
 	struct ice_hw *hw = ICE_DEV_PRIVATE_TO_HW(dev->data->dev_private);
+	struct rte_pci_device *pci_dev = RTE_ETH_DEV_TO_PCI(dev);
+	struct rte_intr_handle *intr_handle = &pci_dev->intr_handle;
+	struct rte_flow *p_flow;
 
 	/* Since stop will make link down, then the link event will be
 	 * triggered, disable the irq firstly to avoid the port_infoe etc
@@ -2109,22 +2117,6 @@ ice_dev_close(struct rte_eth_dev *dev)
 	rte_free(hw->port_info);
 	hw->port_info = NULL;
 	ice_shutdown_all_ctrlq(hw);
-	rte_free(pf->proto_xtr);
-	pf->proto_xtr = NULL;
-}
-
-static int
-ice_dev_uninit(struct rte_eth_dev *dev)
-{
-	struct rte_pci_device *pci_dev = RTE_ETH_DEV_TO_PCI(dev);
-	struct rte_intr_handle *intr_handle = &pci_dev->intr_handle;
-	struct ice_pf *pf = ICE_DEV_PRIVATE_TO_PF(dev->data->dev_private);
-	struct rte_flow *p_flow;
-
-	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
-		return 0;
-
-	ice_dev_close(dev);
 
 	dev->dev_ops = NULL;
 	dev->rx_pkt_burst = NULL;
@@ -2146,6 +2138,12 @@ ice_dev_uninit(struct rte_eth_dev *dev)
 		ice_free_switch_filter_rule(p_flow->rule);
 		rte_free(p_flow);
 	}
+}
+
+static int
+ice_dev_uninit(struct rte_eth_dev *dev)
+{
+	ice_dev_close(dev);
 
 	return 0;
 }
-- 
2.17.1

[dpdk-dev] [PATCH v5 5/5] net/ixgbe: release port upon close

[Di ChenxuX]

Set RTE_ETH_DEV_CLOSE_REMOVE upon probe so all the private resources
 for the port can be freed by rte_eth_dev_close().

Signed-off-by: Di ChenxuX <chenxux.di@intel.com>
---
 doc/guides/rel_notes/release_19_11.rst |   4 +
 drivers/net/ixgbe/ixgbe_ethdev.c       | 167 +++++++++++++------------
 2 files changed, 89 insertions(+), 82 deletions(-)

diff --git a/doc/guides/rel_notes/release_19_11.rst b/doc/guides/rel_notes/release_19_11.rst
index 751606440..44011085d 100644
--- a/doc/guides/rel_notes/release_19_11.rst
+++ b/doc/guides/rel_notes/release_19_11.rst
@@ -77,6 +77,10 @@ New Features
 
   Added support for the ``RTE_ETH_DEV_CLOSE_REMOVE`` flag.
 
+* **Updated the Intel ixgbe driver.**
+
+  Added support for the ``RTE_ETH_DEV_CLOSE_REMOVE`` flag.
+
 
 Removed Items
 -------------
diff --git a/drivers/net/ixgbe/ixgbe_ethdev.c b/drivers/net/ixgbe/ixgbe_ethdev.c
index 023b267d7..e6879c4b0 100644
--- a/drivers/net/ixgbe/ixgbe_ethdev.c
+++ b/drivers/net/ixgbe/ixgbe_ethdev.c
@@ -1245,6 +1245,11 @@ eth_ixgbe_dev_init(struct rte_eth_dev *eth_dev, void *init_params __rte_unused)
 		return -ENOMEM;
 	}
 
+	/* Pass the information to the rte_eth_dev_close() that it should also
+	 * release the private port resources.
+	 */
+	eth_dev->data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
+
 	/* initialize the vfta */
 	memset(shadow_vfta, 0, sizeof(*shadow_vfta));
 
@@ -1311,73 +1316,12 @@ eth_ixgbe_dev_init(struct rte_eth_dev *eth_dev, void *init_params __rte_unused)
 static int
 eth_ixgbe_dev_uninit(struct rte_eth_dev *eth_dev)
 {
-	struct rte_pci_device *pci_dev = RTE_ETH_DEV_TO_PCI(eth_dev);
-	struct rte_intr_handle *intr_handle = &pci_dev->intr_handle;
-	struct ixgbe_hw *hw;
-	int retries = 0;
-	int ret;
-
 	PMD_INIT_FUNC_TRACE();
 
 	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
 		return 0;
 
-	hw = IXGBE_DEV_PRIVATE_TO_HW(eth_dev->data->dev_private);
-
-	if (hw->adapter_stopped == 0)
-		ixgbe_dev_close(eth_dev);
-
-	eth_dev->dev_ops = NULL;
-	eth_dev->rx_pkt_burst = NULL;
-	eth_dev->tx_pkt_burst = NULL;
-
-	/* Unlock any pending hardware semaphore */
-	ixgbe_swfw_lock_reset(hw);
-
-	/* disable uio intr before callback unregister */
-	rte_intr_disable(intr_handle);
-
-	do {
-		ret = rte_intr_callback_unregister(intr_handle,
-				ixgbe_dev_interrupt_handler, eth_dev);
-		if (ret >= 0) {
-			break;
-		} else if (ret != -EAGAIN) {
-			PMD_INIT_LOG(ERR,
-				"intr callback unregister failed: %d",
-				ret);
-			return ret;
-		}
-		rte_delay_ms(100);
-	} while (retries++ < (10 + IXGBE_LINK_UP_TIME));
-
-	/* cancel the delay handler before remove dev */
-	rte_eal_alarm_cancel(ixgbe_dev_interrupt_delayed_handler, eth_dev);
-
-	/* cancel the link handler before remove dev */
-	rte_eal_alarm_cancel(ixgbe_dev_setup_link_alarm_handler, eth_dev);
-
-	/* uninitialize PF if max_vfs not zero */
-	ixgbe_pf_host_uninit(eth_dev);
-
-	/* remove all the fdir filters & hash */
-	ixgbe_fdir_filter_uninit(eth_dev);
-
-	/* remove all the L2 tunnel filters & hash */
-	ixgbe_l2_tn_filter_uninit(eth_dev);
-
-	/* Remove all ntuple filters of the device */
-	ixgbe_ntuple_filter_uninit(eth_dev);
-
-	/* clear all the filters list */
-	ixgbe_filterlist_flush();
-
-	/* Remove all Traffic Manager configuration */
-	ixgbe_tm_conf_uninit(eth_dev);
-
-#ifdef RTE_LIBRTE_SECURITY
-	rte_free(eth_dev->security_ctx);
-#endif
+	ixgbe_dev_close(eth_dev);
 
 	return 0;
 }
@@ -1711,6 +1655,11 @@ eth_ixgbevf_dev_init(struct rte_eth_dev *eth_dev)
 		return -ENOMEM;
 	}
 
+	/* Pass the information to the rte_eth_dev_close() that it should also
+	 * release the private port resources.
+	 */
+	eth_dev->data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
+
 	/* Generate a random MAC address, if none was assigned by PF. */
 	if (rte_is_zero_ether_addr(perm_addr)) {
 		generate_random_mac_addr(perm_addr);
@@ -1762,30 +1711,12 @@ eth_ixgbevf_dev_init(struct rte_eth_dev *eth_dev)
 static int
 eth_ixgbevf_dev_uninit(struct rte_eth_dev *eth_dev)
 {
-	struct rte_pci_device *pci_dev = RTE_ETH_DEV_TO_PCI(eth_dev);
-	struct rte_intr_handle *intr_handle = &pci_dev->intr_handle;
-	struct ixgbe_hw *hw;
-
 	PMD_INIT_FUNC_TRACE();
 
 	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
 		return 0;
 
-	hw = IXGBE_DEV_PRIVATE_TO_HW(eth_dev->data->dev_private);
-
-	if (hw->adapter_stopped == 0)
-		ixgbevf_dev_close(eth_dev);
-
-	eth_dev->dev_ops = NULL;
-	eth_dev->rx_pkt_burst = NULL;
-	eth_dev->tx_pkt_burst = NULL;
-
-	/* Disable the interrupts for VF */
-	ixgbevf_intr_disable(eth_dev);
-
-	rte_intr_disable(intr_handle);
-	rte_intr_callback_unregister(intr_handle,
-				     ixgbevf_dev_interrupt_handler, eth_dev);
+	ixgbevf_dev_close(eth_dev);
 
 	return 0;
 }
@@ -2876,6 +2807,9 @@ ixgbe_dev_stop(struct rte_eth_dev *dev)
 	struct ixgbe_tm_conf *tm_conf =
 		IXGBE_DEV_PRIVATE_TO_TM_CONF(dev->data->dev_private);
 
+	if (hw->adapter_stopped)
+		return;
+
 	PMD_INIT_FUNC_TRACE();
 
 	rte_eal_alarm_cancel(ixgbe_dev_setup_link_alarm_handler, dev);
@@ -2928,6 +2862,8 @@ ixgbe_dev_stop(struct rte_eth_dev *dev)
 	tm_conf->committed = false;
 
 	adapter->rss_reta_updated = 0;
+
+	hw->adapter_stopped = true;
 }
 
 /*
@@ -2998,13 +2934,16 @@ ixgbe_dev_close(struct rte_eth_dev *dev)
 {
 	struct ixgbe_hw *hw =
 		IXGBE_DEV_PRIVATE_TO_HW(dev->data->dev_private);
+	struct rte_pci_device *pci_dev = RTE_ETH_DEV_TO_PCI(dev);
+	struct rte_intr_handle *intr_handle = &pci_dev->intr_handle;
+	int retries = 0;
+	int ret;
 
 	PMD_INIT_FUNC_TRACE();
 
 	ixgbe_pf_reset_hw(hw);
 
 	ixgbe_dev_stop(dev);
-	hw->adapter_stopped = 1;
 
 	ixgbe_dev_free_queues(dev);
 
@@ -3012,6 +2951,55 @@ ixgbe_dev_close(struct rte_eth_dev *dev)
 
 	/* reprogram the RAR[0] in case user changed it. */
 	ixgbe_set_rar(hw, 0, hw->mac.addr, 0, IXGBE_RAH_AV);
+
+	dev->dev_ops = NULL;
+	dev->rx_pkt_burst = NULL;
+	dev->tx_pkt_burst = NULL;
+
+	/* Unlock any pending hardware semaphore */
+	ixgbe_swfw_lock_reset(hw);
+
+	/* disable uio intr before callback unregister */
+	rte_intr_disable(intr_handle);
+
+	do {
+		ret = rte_intr_callback_unregister(intr_handle,
+				ixgbe_dev_interrupt_handler, dev);
+		if (ret >= 0) {
+			break;
+		} else if (ret != -EAGAIN) {
+			PMD_INIT_LOG(ERR,
+				"intr callback unregister failed: %d",
+				ret);
+		}
+		rte_delay_ms(100);
+	} while (retries++ < (10 + IXGBE_LINK_UP_TIME));
+
+	/* cancel the delay handler before remove dev */
+	rte_eal_alarm_cancel(ixgbe_dev_interrupt_delayed_handler, dev);
+
+	/* uninitialize PF if max_vfs not zero */
+	ixgbe_pf_host_uninit(dev);
+
+	/* remove all the fdir filters & hash */
+	ixgbe_fdir_filter_uninit(dev);
+
+	/* remove all the L2 tunnel filters & hash */
+	ixgbe_l2_tn_filter_uninit(dev);
+
+	/* Remove all ntuple filters of the device */
+	ixgbe_ntuple_filter_uninit(dev);
+
+	/* clear all the filters list */
+	ixgbe_filterlist_flush();
+
+	/* Remove all Traffic Manager configuration */
+	ixgbe_tm_conf_uninit(dev);
+
+#ifdef RTE_LIBRTE_SECURITY
+	rte_free(dev->security_ctx);
+#endif
+
 }
 
 /*
@@ -5244,6 +5232,8 @@ ixgbevf_dev_start(struct rte_eth_dev *dev)
 	 */
 	ixgbevf_dev_link_update(dev, 0);
 
+	hw->adapter_stopped = false;
+
 	return 0;
 }
 
@@ -5255,6 +5245,9 @@ ixgbevf_dev_stop(struct rte_eth_dev *dev)
 	struct rte_pci_device *pci_dev = RTE_ETH_DEV_TO_PCI(dev);
 	struct rte_intr_handle *intr_handle = &pci_dev->intr_handle;
 
+	if (hw->adapter_stopped)
+		return;
+
 	PMD_INIT_FUNC_TRACE();
 
 	rte_eal_alarm_cancel(ixgbe_dev_setup_link_alarm_handler, dev);
@@ -5289,6 +5282,8 @@ static void
 ixgbevf_dev_close(struct rte_eth_dev *dev)
 {
 	struct ixgbe_hw *hw = IXGBE_DEV_PRIVATE_TO_HW(dev->data->dev_private);
+	struct rte_pci_device *pci_dev = RTE_ETH_DEV_TO_PCI(dev);
+	struct rte_intr_handle *intr_handle = &pci_dev->intr_handle;
 
 	PMD_INIT_FUNC_TRACE();
 
@@ -5304,6 +5299,14 @@ ixgbevf_dev_close(struct rte_eth_dev *dev)
 	 * after stop, close and detach of the VF
 	 **/
 	ixgbevf_remove_mac_addr(dev, 0);
+
+	dev->dev_ops = NULL;
+	dev->rx_pkt_burst = NULL;
+	dev->tx_pkt_burst = NULL;
+
+	rte_intr_disable(intr_handle);
+	rte_intr_callback_unregister(intr_handle,
+				     ixgbevf_dev_interrupt_handler, dev);
 }
 
 /*
-- 
2.17.1

Re: [dpdk-dev] [PATCH v5 0/5] drivers/net: release port upon close

[Ye Xiaolong]

On 09/26, Di ChenxuX wrote:
>Set RTE_ETH_DEV_CLOSE_REMOVE upon probe so all the private resources
> for the port can be freed by rte_eth_dev_close().
>This patch cover all the intel drivers.
>
>---
>V5 changes:
>  Added code in em_ethdev.c.
>  Separated the release note, each patch has its own note. 
>V4 changes:
>  Removed adaper stopped flag in function ice_dev_init.
>V3 changes:
>  Removed adapter closed flag in driver e1000, fm10k, ice, ixgbe.
>V2 changes:
>  Separated to 5 patches, each patch only change one driver.
>  Added resource release in dev_close.
>
>Di ChenxuX (5):
>  net/e1000: release port upon close
>  net/fm10k: release port upon close
>  net/i40e: release port upon close
>  net/ice: release port upon close
>  net/ixgbe: release port upon close
>
> doc/guides/rel_notes/release_19_11.rst |  18 +++
> drivers/net/e1000/em_ethdev.c          |  33 ++---
> drivers/net/e1000/igb_ethdev.c         | 144 ++++++++++-----------
> drivers/net/fm10k/fm10k_ethdev.c       | 100 ++++++++-------
> drivers/net/i40e/i40e_ethdev.c         | 128 +++++++++----------
> drivers/net/i40e/i40e_ethdev_vf.c      |  25 ++--
> drivers/net/ice/ice_ethdev.c           |  30 +++--
> drivers/net/ixgbe/ixgbe_ethdev.c       | 167 +++++++++++++------------
> 8 files changed, 335 insertions(+), 310 deletions(-)
>
>-- 
>2.17.1
>

Series Reviewed-by: Xiaolong Ye <xiaolong.ye@intel.com>

[dpdk-dev] [PATCH v6 0/5] drivers/net: release port upon close

[Di ChenxuX]

Set RTE_ETH_DEV_CLOSE_REMOVE upon probe so all the private resources
 for the port can be freed by rte_eth_dev_close().
This patch cover all the intel drivers.

---
V6 changes:
  Fixed check error.
V5 changes:
  Added code in em_ethdev.c.
  Separated the release note, each patch has its own note. 
V4 changes:
  Removed adaper stopped flag in function ice_dev_init.
V3 changes:
  Removed adapter closed flag in driver e1000, fm10k, ice, ixgbe.
V2 changes:
  Separated to 5 patches, each patch only change one driver.
  Added resource release in dev_close.

Di ChenxuX (5):
  net/e1000: release port upon close
  net/fm10k: release port upon close
  net/i40e: release port upon close
  net/ice: release port upon close
  net/ixgbe: release port upon close

 doc/guides/rel_notes/release_19_11.rst |  18 +++
 drivers/net/e1000/em_ethdev.c          |  33 ++---
 drivers/net/e1000/igb_ethdev.c         | 144 ++++++++++-----------
 drivers/net/fm10k/fm10k_ethdev.c       | 100 ++++++++-------
 drivers/net/i40e/i40e_ethdev.c         | 128 +++++++++----------
 drivers/net/i40e/i40e_ethdev_vf.c      |  25 ++--
 drivers/net/ice/ice_ethdev.c           |  30 +++--
 drivers/net/ixgbe/ixgbe_ethdev.c       | 167 +++++++++++++------------
 8 files changed, 335 insertions(+), 310 deletions(-)

-- 
2.17.1

[dpdk-dev] [PATCH v6 1/5] net/e1000: release port upon close

[Di ChenxuX]

Set RTE_ETH_DEV_CLOSE_REMOVE upon probe so all the private resources
 for the port can be freed by rte_eth_dev_close().

Signed-off-by: Di ChenxuX <chenxux.di@intel.com>
---
 doc/guides/rel_notes/release_19_11.rst |   5 +
 drivers/net/e1000/em_ethdev.c          |  33 +++---
 drivers/net/e1000/igb_ethdev.c         | 144 ++++++++++++-------------
 3 files changed, 94 insertions(+), 88 deletions(-)

diff --git a/doc/guides/rel_notes/release_19_11.rst b/doc/guides/rel_notes/release_19_11.rst
index 04cbc6e5f..99de7b0a3 100644
--- a/doc/guides/rel_notes/release_19_11.rst
+++ b/doc/guides/rel_notes/release_19_11.rst
@@ -64,6 +64,11 @@ New Features
   * Added support for handling Receive Flex Descriptor.
   * Added support for protocol extraction on per Rx queue.
 
+* **Updated the Intel e1000 driver.**
+
+  Added support for the ``RTE_ETH_DEV_CLOSE_REMOVE`` flag.
+
+
 Removed Items
 -------------
 
diff --git a/drivers/net/e1000/em_ethdev.c b/drivers/net/e1000/em_ethdev.c
index 8fe39176a..c9ebacc3d 100644
--- a/drivers/net/e1000/em_ethdev.c
+++ b/drivers/net/e1000/em_ethdev.c
@@ -297,6 +297,11 @@ eth_em_dev_init(struct rte_eth_dev *eth_dev)
 	rte_ether_addr_copy((struct rte_ether_addr *)hw->mac.addr,
 		eth_dev->data->mac_addrs);
 
+	/* Pass the information to the rte_eth_dev_close() that it should also
+	 * release the private port resources.
+	 */
+	eth_dev->data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
+
 	/* initialize the vfta */
 	memset(shadow_vfta, 0, sizeof(*shadow_vfta));
 
@@ -313,27 +318,12 @@ eth_em_dev_init(struct rte_eth_dev *eth_dev)
 static int
 eth_em_dev_uninit(struct rte_eth_dev *eth_dev)
 {
-	struct rte_pci_device *pci_dev = RTE_ETH_DEV_TO_PCI(eth_dev);
-	struct e1000_adapter *adapter =
-		E1000_DEV_PRIVATE(eth_dev->data->dev_private);
-	struct rte_intr_handle *intr_handle = &pci_dev->intr_handle;
-
 	PMD_INIT_FUNC_TRACE();
 
 	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
 		return -EPERM;
 
-	if (adapter->stopped == 0)
-		eth_em_close(eth_dev);
-
-	eth_dev->dev_ops = NULL;
-	eth_dev->rx_pkt_burst = NULL;
-	eth_dev->tx_pkt_burst = NULL;
-
-	/* disable uio intr before callback unregister */
-	rte_intr_disable(intr_handle);
-	rte_intr_callback_unregister(intr_handle,
-				     eth_em_interrupt_handler, eth_dev);
+	eth_em_close(eth_dev);
 
 	return 0;
 }
@@ -774,6 +764,8 @@ eth_em_close(struct rte_eth_dev *dev)
 	struct e1000_hw *hw = E1000_DEV_PRIVATE_TO_HW(dev->data->dev_private);
 	struct e1000_adapter *adapter =
 		E1000_DEV_PRIVATE(dev->data->dev_private);
+	struct rte_pci_device *pci_dev = RTE_ETH_DEV_TO_PCI(dev);
+	struct rte_intr_handle *intr_handle = &pci_dev->intr_handle;
 
 	eth_em_stop(dev);
 	adapter->stopped = 1;
@@ -781,6 +773,15 @@ eth_em_close(struct rte_eth_dev *dev)
 	e1000_phy_hw_reset(hw);
 	em_release_manageability(hw);
 	em_hw_control_release(hw);
+
+	dev->dev_ops = NULL;
+	dev->rx_pkt_burst = NULL;
+	dev->tx_pkt_burst = NULL;
+
+	/* disable uio intr before callback unregister */
+	rte_intr_disable(intr_handle);
+	rte_intr_callback_unregister(intr_handle,
+				     eth_em_interrupt_handler, dev);
 }
 
 static int
diff --git a/drivers/net/e1000/igb_ethdev.c b/drivers/net/e1000/igb_ethdev.c
index a9f6de5d5..3f742193d 100644
--- a/drivers/net/e1000/igb_ethdev.c
+++ b/drivers/net/e1000/igb_ethdev.c
@@ -843,6 +843,11 @@ eth_igb_dev_init(struct rte_eth_dev *eth_dev)
 	rte_ether_addr_copy((struct rte_ether_addr *)hw->mac.addr,
 			&eth_dev->data->mac_addrs[0]);
 
+	/* Pass the information to the rte_eth_dev_close() that it should also
+	 * release the private port resources.
+	 */
+	eth_dev->data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
+
 	/* initialize the vfta */
 	memset(shadow_vfta, 0, sizeof(*shadow_vfta));
 
@@ -912,61 +917,12 @@ eth_igb_dev_init(struct rte_eth_dev *eth_dev)
 static int
 eth_igb_dev_uninit(struct rte_eth_dev *eth_dev)
 {
-	struct rte_pci_device *pci_dev;
-	struct rte_intr_handle *intr_handle;
-	struct e1000_hw *hw;
-	struct e1000_adapter *adapter =
-		E1000_DEV_PRIVATE(eth_dev->data->dev_private);
-	struct e1000_filter_info *filter_info =
-		E1000_DEV_PRIVATE_TO_FILTER_INFO(eth_dev->data->dev_private);
-
 	PMD_INIT_FUNC_TRACE();
 
 	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
 		return -EPERM;
 
-	hw = E1000_DEV_PRIVATE_TO_HW(eth_dev->data->dev_private);
-	pci_dev = RTE_ETH_DEV_TO_PCI(eth_dev);
-	intr_handle = &pci_dev->intr_handle;
-
-	if (adapter->stopped == 0)
-		eth_igb_close(eth_dev);
-
-	eth_dev->dev_ops = NULL;
-	eth_dev->rx_pkt_burst = NULL;
-	eth_dev->tx_pkt_burst = NULL;
-
-	/* Reset any pending lock */
-	igb_reset_swfw_lock(hw);
-
-	/* uninitialize PF if max_vfs not zero */
-	igb_pf_host_uninit(eth_dev);
-
-	/* disable uio intr before callback unregister */
-	rte_intr_disable(intr_handle);
-	rte_intr_callback_unregister(intr_handle,
-				     eth_igb_interrupt_handler, eth_dev);
-
-	/* clear the SYN filter info */
-	filter_info->syn_info = 0;
-
-	/* clear the ethertype filters info */
-	filter_info->ethertype_mask = 0;
-	memset(filter_info->ethertype_filters, 0,
-		E1000_MAX_ETQF_FILTERS * sizeof(struct igb_ethertype_filter));
-
-	/* clear the rss filter info */
-	memset(&filter_info->rss_info, 0,
-		sizeof(struct igb_rte_flow_rss_conf));
-
-	/* remove all ntuple filters of the device */
-	igb_ntuple_filter_uninit(eth_dev);
-
-	/* remove all flex filters of the device */
-	igb_flex_filter_uninit(eth_dev);
-
-	/* clear all the filters list */
-	igb_filterlist_flush(eth_dev);
+	eth_igb_close(eth_dev);
 
 	return 0;
 }
@@ -1038,6 +994,11 @@ eth_igbvf_dev_init(struct rte_eth_dev *eth_dev)
 		return -ENOMEM;
 	}
 
+	/* Pass the information to the rte_eth_dev_close() that it should also
+	 * release the private port resources.
+	 */
+	eth_dev->data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
+
 	/* Generate a random MAC address, if none was assigned by PF. */
 	if (rte_is_zero_ether_addr(perm_addr)) {
 		rte_eth_random_addr(perm_addr->addr_bytes);
@@ -1077,27 +1038,12 @@ eth_igbvf_dev_init(struct rte_eth_dev *eth_dev)
 static int
 eth_igbvf_dev_uninit(struct rte_eth_dev *eth_dev)
 {
-	struct e1000_adapter *adapter =
-		E1000_DEV_PRIVATE(eth_dev->data->dev_private);
-	struct rte_pci_device *pci_dev = RTE_ETH_DEV_TO_PCI(eth_dev);
-
 	PMD_INIT_FUNC_TRACE();
 
 	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
 		return -EPERM;
 
-	if (adapter->stopped == 0)
-		igbvf_dev_close(eth_dev);
-
-	eth_dev->dev_ops = NULL;
-	eth_dev->rx_pkt_burst = NULL;
-	eth_dev->tx_pkt_burst = NULL;
-
-	/* disable uio intr before callback unregister */
-	rte_intr_disable(&pci_dev->intr_handle);
-	rte_intr_callback_unregister(&pci_dev->intr_handle,
-				     eth_igbvf_interrupt_handler,
-				     (void *)eth_dev);
+	igbvf_dev_close(eth_dev);
 
 	return 0;
 }
@@ -1506,6 +1452,11 @@ eth_igb_stop(struct rte_eth_dev *dev)
 	struct rte_pci_device *pci_dev = RTE_ETH_DEV_TO_PCI(dev);
 	struct rte_eth_link link;
 	struct rte_intr_handle *intr_handle = &pci_dev->intr_handle;
+	struct e1000_adapter *adapter =
+		E1000_DEV_PRIVATE(dev->data->dev_private);
+
+	if (adapter->stopped)
+		return;
 
 	eth_igb_rxtx_control(dev, false);
 
@@ -1547,6 +1498,8 @@ eth_igb_stop(struct rte_eth_dev *dev)
 		rte_free(intr_handle->intr_vec);
 		intr_handle->intr_vec = NULL;
 	}
+
+	adapter->stopped = true;
 }
 
 static int
@@ -1579,14 +1532,13 @@ static void
 eth_igb_close(struct rte_eth_dev *dev)
 {
 	struct e1000_hw *hw = E1000_DEV_PRIVATE_TO_HW(dev->data->dev_private);
-	struct e1000_adapter *adapter =
-		E1000_DEV_PRIVATE(dev->data->dev_private);
 	struct rte_eth_link link;
 	struct rte_pci_device *pci_dev = RTE_ETH_DEV_TO_PCI(dev);
 	struct rte_intr_handle *intr_handle = &pci_dev->intr_handle;
+	struct e1000_filter_info *filter_info =
+		E1000_DEV_PRIVATE_TO_FILTER_INFO(dev->data->dev_private);
 
 	eth_igb_stop(dev);
-	adapter->stopped = 1;
 
 	e1000_phy_hw_reset(hw);
 	igb_release_manageability(hw);
@@ -1610,6 +1562,40 @@ eth_igb_close(struct rte_eth_dev *dev)
 
 	memset(&link, 0, sizeof(link));
 	rte_eth_linkstatus_set(dev, &link);
+
+	dev->dev_ops = NULL;
+	dev->rx_pkt_burst = NULL;
+	dev->tx_pkt_burst = NULL;
+
+	/* Reset any pending lock */
+	igb_reset_swfw_lock(hw);
+
+	/* uninitialize PF if max_vfs not zero */
+	igb_pf_host_uninit(dev);
+
+	rte_intr_callback_unregister(intr_handle,
+				     eth_igb_interrupt_handler, dev);
+
+	/* clear the SYN filter info */
+	filter_info->syn_info = 0;
+
+	/* clear the ethertype filters info */
+	filter_info->ethertype_mask = 0;
+	memset(filter_info->ethertype_filters, 0,
+		E1000_MAX_ETQF_FILTERS * sizeof(struct igb_ethertype_filter));
+
+	/* clear the rss filter info */
+	memset(&filter_info->rss_info, 0,
+		sizeof(struct igb_rte_flow_rss_conf));
+
+	/* remove all ntuple filters of the device */
+	igb_ntuple_filter_uninit(dev);
+
+	/* remove all flex filters of the device */
+	igb_flex_filter_uninit(dev);
+
+	/* clear all the filters list */
+	igb_filterlist_flush(dev);
 }
 
 /*
@@ -3344,6 +3330,11 @@ igbvf_dev_stop(struct rte_eth_dev *dev)
 {
 	struct rte_pci_device *pci_dev = RTE_ETH_DEV_TO_PCI(dev);
 	struct rte_intr_handle *intr_handle = &pci_dev->intr_handle;
+	struct e1000_adapter *adapter =
+		E1000_DEV_PRIVATE(dev->data->dev_private);
+
+	if (adapter->stopped)
+		return;
 
 	PMD_INIT_FUNC_TRACE();
 
@@ -3366,22 +3357,23 @@ igbvf_dev_stop(struct rte_eth_dev *dev)
 		rte_free(intr_handle->intr_vec);
 		intr_handle->intr_vec = NULL;
 	}
+
+	adapter->stopped = true;
 }
 
 static void
 igbvf_dev_close(struct rte_eth_dev *dev)
 {
 	struct e1000_hw *hw = E1000_DEV_PRIVATE_TO_HW(dev->data->dev_private);
-	struct e1000_adapter *adapter =
-		E1000_DEV_PRIVATE(dev->data->dev_private);
 	struct rte_ether_addr addr;
+	struct rte_pci_device *pci_dev = RTE_ETH_DEV_TO_PCI(dev);
 
 	PMD_INIT_FUNC_TRACE();
 
 	e1000_reset_hw(hw);
 
 	igbvf_dev_stop(dev);
-	adapter->stopped = 1;
+
 	igb_dev_free_queues(dev);
 
 	/**
@@ -3392,6 +3384,14 @@ igbvf_dev_close(struct rte_eth_dev *dev)
 
 	memset(&addr, 0, sizeof(addr));
 	igbvf_default_mac_addr_set(dev, &addr);
+
+	dev->dev_ops = NULL;
+	dev->rx_pkt_burst = NULL;
+	dev->tx_pkt_burst = NULL;
+
+	rte_intr_callback_unregister(&pci_dev->intr_handle,
+				     eth_igbvf_interrupt_handler,
+				     (void *)dev);
 }
 
 static int
-- 
2.17.1

[dpdk-dev] [PATCH v6 2/5] net/fm10k: release port upon close

[Di ChenxuX]

Set RTE_ETH_DEV_CLOSE_REMOVE upon probe so all the private resources
 for the port can be freed by rte_eth_dev_close().

Signed-off-by: Di ChenxuX <chenxux.di@intel.com>
---
 doc/guides/rel_notes/release_19_11.rst |   4 +
 drivers/net/fm10k/fm10k_ethdev.c       | 100 +++++++++++++------------
 2 files changed, 56 insertions(+), 48 deletions(-)

diff --git a/doc/guides/rel_notes/release_19_11.rst b/doc/guides/rel_notes/release_19_11.rst
index 99de7b0a3..177cd9b4a 100644
--- a/doc/guides/rel_notes/release_19_11.rst
+++ b/doc/guides/rel_notes/release_19_11.rst
@@ -68,6 +68,10 @@ New Features
 
   Added support for the ``RTE_ETH_DEV_CLOSE_REMOVE`` flag.
 
+* **Updated the Intel fm10k driver.**
+
+  Added support for the ``RTE_ETH_DEV_CLOSE_REMOVE`` flag.
+
 
 Removed Items
 -------------
diff --git a/drivers/net/fm10k/fm10k_ethdev.c b/drivers/net/fm10k/fm10k_ethdev.c
index f0f629008..f3048b6a6 100644
--- a/drivers/net/fm10k/fm10k_ethdev.c
+++ b/drivers/net/fm10k/fm10k_ethdev.c
@@ -1202,28 +1202,6 @@ fm10k_dev_queue_release(struct rte_eth_dev *dev)
 	}
 }
 
-static void
-fm10k_dev_close(struct rte_eth_dev *dev)
-{
-	struct fm10k_hw *hw = FM10K_DEV_PRIVATE_TO_HW(dev->data->dev_private);
-
-	PMD_INIT_FUNC_TRACE();
-
-	fm10k_mbx_lock(hw);
-	hw->mac.ops.update_lport_state(hw, hw->mac.dglort_map,
-		MAX_LPORT_NUM, false);
-	fm10k_mbx_unlock(hw);
-
-	/* allow 100ms for device to quiesce */
-	rte_delay_us(FM10K_SWITCH_QUIESCE_US);
-
-	/* Stop mailbox service first */
-	fm10k_close_mbx_service(hw);
-	fm10k_dev_stop(dev);
-	fm10k_dev_queue_release(dev);
-	fm10k_stop_hw(hw);
-}
-
 static int
 fm10k_link_update(struct rte_eth_dev *dev,
 	__rte_unused int wait_to_complete)
@@ -2809,6 +2787,53 @@ fm10k_close_mbx_service(struct fm10k_hw *hw)
 	hw->mbx.ops.disconnect(hw, &hw->mbx);
 }
 
+static void
+fm10k_dev_close(struct rte_eth_dev *dev)
+{
+	struct fm10k_hw *hw = FM10K_DEV_PRIVATE_TO_HW(dev->data->dev_private);
+	struct rte_pci_device *pdev = RTE_ETH_DEV_TO_PCI(dev);
+	struct rte_intr_handle *intr_handle = &pdev->intr_handle;
+
+	PMD_INIT_FUNC_TRACE();
+
+	fm10k_mbx_lock(hw);
+	hw->mac.ops.update_lport_state(hw, hw->mac.dglort_map,
+		MAX_LPORT_NUM, false);
+	fm10k_mbx_unlock(hw);
+
+	/* allow 100ms for device to quiesce */
+	rte_delay_us(FM10K_SWITCH_QUIESCE_US);
+
+	/* Stop mailbox service first */
+	fm10k_close_mbx_service(hw);
+	fm10k_dev_stop(dev);
+	fm10k_dev_queue_release(dev);
+	fm10k_stop_hw(hw);
+
+	dev->dev_ops = NULL;
+	dev->rx_pkt_burst = NULL;
+	dev->tx_pkt_burst = NULL;
+
+	/* disable uio/vfio intr */
+	rte_intr_disable(intr_handle);
+
+	/*PF/VF has different interrupt handling mechanism */
+	if (hw->mac.type == fm10k_mac_pf) {
+		/* disable interrupt */
+		fm10k_dev_disable_intr_pf(dev);
+
+		/* unregister callback func to eal lib */
+		rte_intr_callback_unregister(intr_handle,
+			fm10k_dev_interrupt_handler_pf, (void *)dev);
+	} else {
+		/* disable interrupt */
+		fm10k_dev_disable_intr_vf(dev);
+
+		rte_intr_callback_unregister(intr_handle,
+			fm10k_dev_interrupt_handler_vf, (void *)dev);
+	}
+}
+
 static const struct eth_dev_ops fm10k_eth_dev_ops = {
 	.dev_configure		= fm10k_dev_configure,
 	.dev_start		= fm10k_dev_start,
@@ -3113,6 +3138,11 @@ eth_fm10k_dev_init(struct rte_eth_dev *dev)
 		&dev->data->mac_addrs[0]);
 	}
 
+	/* Pass the information to the rte_eth_dev_close() that it should also
+	 * release the private port resources.
+	 */
+	dev->data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
+
 	/* Reset the hw statistics */
 	fm10k_stats_reset(dev);
 
@@ -3215,9 +3245,6 @@ eth_fm10k_dev_init(struct rte_eth_dev *dev)
 static int
 eth_fm10k_dev_uninit(struct rte_eth_dev *dev)
 {
-	struct fm10k_hw *hw = FM10K_DEV_PRIVATE_TO_HW(dev->data->dev_private);
-	struct rte_pci_device *pdev = RTE_ETH_DEV_TO_PCI(dev);
-	struct rte_intr_handle *intr_handle = &pdev->intr_handle;
 	PMD_INIT_FUNC_TRACE();
 
 	/* only uninitialize in the primary process */
@@ -3227,29 +3254,6 @@ eth_fm10k_dev_uninit(struct rte_eth_dev *dev)
 	/* safe to close dev here */
 	fm10k_dev_close(dev);
 
-	dev->dev_ops = NULL;
-	dev->rx_pkt_burst = NULL;
-	dev->tx_pkt_burst = NULL;
-
-	/* disable uio/vfio intr */
-	rte_intr_disable(intr_handle);
-
-	/*PF/VF has different interrupt handling mechanism */
-	if (hw->mac.type == fm10k_mac_pf) {
-		/* disable interrupt */
-		fm10k_dev_disable_intr_pf(dev);
-
-		/* unregister callback func to eal lib */
-		rte_intr_callback_unregister(intr_handle,
-			fm10k_dev_interrupt_handler_pf, (void *)dev);
-	} else {
-		/* disable interrupt */
-		fm10k_dev_disable_intr_vf(dev);
-
-		rte_intr_callback_unregister(intr_handle,
-			fm10k_dev_interrupt_handler_vf, (void *)dev);
-	}
-
 	return 0;
 }
 
-- 
2.17.1

[dpdk-dev] [PATCH v6 3/5] net/i40e: release port upon close

[Di ChenxuX]

Set RTE_ETH_DEV_CLOSE_REMOVE upon probe so all the private resources
 for the port can be freed by rte_eth_dev_close().

Signed-off-by: Di ChenxuX <chenxux.di@intel.com>
---
 doc/guides/rel_notes/release_19_11.rst |   4 +
 drivers/net/i40e/i40e_ethdev.c         | 128 ++++++++++++-------------
 drivers/net/i40e/i40e_ethdev_vf.c      |  25 +++--
 3 files changed, 81 insertions(+), 76 deletions(-)

diff --git a/doc/guides/rel_notes/release_19_11.rst b/doc/guides/rel_notes/release_19_11.rst
index 177cd9b4a..7e3290665 100644
--- a/doc/guides/rel_notes/release_19_11.rst
+++ b/doc/guides/rel_notes/release_19_11.rst
@@ -72,6 +72,10 @@ New Features
 
   Added support for the ``RTE_ETH_DEV_CLOSE_REMOVE`` flag.
 
+* **Updated the Intel i40e driver.**
+
+  Added support for the ``RTE_ETH_DEV_CLOSE_REMOVE`` flag.
+
 
 Removed Items
 -------------
diff --git a/drivers/net/i40e/i40e_ethdev.c b/drivers/net/i40e/i40e_ethdev.c
index 8b9e23659..4953d3aaa 100644
--- a/drivers/net/i40e/i40e_ethdev.c
+++ b/drivers/net/i40e/i40e_ethdev.c
@@ -1590,6 +1590,11 @@ eth_i40e_dev_init(struct rte_eth_dev *dev, void *init_params __rte_unused)
 	rte_ether_addr_copy((struct rte_ether_addr *)hw->mac.perm_addr,
 					&dev->data->mac_addrs[0]);
 
+	/* Pass the information to the rte_eth_dev_close() that it should also
+	 * release the private port resources.
+	 */
+	dev->data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
+
 	/* Init dcb to sw mode by default */
 	ret = i40e_dcb_init_configure(dev, TRUE);
 	if (ret != I40E_SUCCESS) {
@@ -1759,85 +1764,18 @@ void i40e_flex_payload_reg_set_default(struct i40e_hw *hw)
 static int
 eth_i40e_dev_uninit(struct rte_eth_dev *dev)
 {
-	struct i40e_pf *pf;
-	struct rte_pci_device *pci_dev;
-	struct rte_intr_handle *intr_handle;
 	struct i40e_hw *hw;
-	struct i40e_filter_control_settings settings;
-	struct rte_flow *p_flow;
-	int ret;
-	uint8_t aq_fail = 0;
-	int retries = 0;
 
 	PMD_INIT_FUNC_TRACE();
 
 	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
 		return 0;
 
-	pf = I40E_DEV_PRIVATE_TO_PF(dev->data->dev_private);
 	hw = I40E_DEV_PRIVATE_TO_HW(dev->data->dev_private);
-	pci_dev = RTE_ETH_DEV_TO_PCI(dev);
-	intr_handle = &pci_dev->intr_handle;
-
-	ret = rte_eth_switch_domain_free(pf->switch_domain_id);
-	if (ret)
-		PMD_INIT_LOG(WARNING, "failed to free switch domain: %d", ret);
 
 	if (hw->adapter_closed == 0)
 		i40e_dev_close(dev);
 
-	dev->dev_ops = NULL;
-	dev->rx_pkt_burst = NULL;
-	dev->tx_pkt_burst = NULL;
-
-	/* Clear PXE mode */
-	i40e_clear_pxe_mode(hw);
-
-	/* Unconfigure filter control */
-	memset(&settings, 0, sizeof(settings));
-	ret = i40e_set_filter_control(hw, &settings);
-	if (ret)
-		PMD_INIT_LOG(WARNING, "setup_pf_filter_control failed: %d",
-					ret);
-
-	/* Disable flow control */
-	hw->fc.requested_mode = I40E_FC_NONE;
-	i40e_set_fc(hw, &aq_fail, TRUE);
-
-	/* uninitialize pf host driver */
-	i40e_pf_host_uninit(dev);
-
-	/* disable uio intr before callback unregister */
-	rte_intr_disable(intr_handle);
-
-	/* unregister callback func to eal lib */
-	do {
-		ret = rte_intr_callback_unregister(intr_handle,
-				i40e_dev_interrupt_handler, dev);
-		if (ret >= 0) {
-			break;
-		} else if (ret != -EAGAIN) {
-			PMD_INIT_LOG(ERR,
-				 "intr callback unregister failed: %d",
-				 ret);
-			return ret;
-		}
-		i40e_msec_delay(500);
-	} while (retries++ < 5);
-
-	i40e_rm_ethtype_filter_list(pf);
-	i40e_rm_tunnel_filter_list(pf);
-	i40e_rm_fdir_filter_list(pf);
-
-	/* Remove all flows */
-	while ((p_flow = TAILQ_FIRST(&pf->flow_list))) {
-		TAILQ_REMOVE(&pf->flow_list, p_flow, node);
-		rte_free(p_flow);
-	}
-
-	/* Remove all Traffic Manager configuration */
-	i40e_tm_conf_uninit(dev);
-
 	return 0;
 }
 
@@ -2535,12 +2473,21 @@ i40e_dev_close(struct rte_eth_dev *dev)
 	struct rte_pci_device *pci_dev = RTE_ETH_DEV_TO_PCI(dev);
 	struct rte_intr_handle *intr_handle = &pci_dev->intr_handle;
 	struct i40e_mirror_rule *p_mirror;
+	struct i40e_filter_control_settings settings;
+	struct rte_flow *p_flow;
 	uint32_t reg;
 	int i;
 	int ret;
+	uint8_t aq_fail = 0;
+	int retries = 0;
 
 	PMD_INIT_FUNC_TRACE();
 
+	ret = rte_eth_switch_domain_free(pf->switch_domain_id);
+	if (ret)
+		PMD_INIT_LOG(WARNING, "failed to free switch domain: %d", ret);
+
+
 	i40e_dev_stop(dev);
 
 	/* Remove all mirror rules */
@@ -2605,6 +2552,53 @@ i40e_dev_close(struct rte_eth_dev *dev)
 			(reg | I40E_PFGEN_CTRL_PFSWR_MASK));
 	I40E_WRITE_FLUSH(hw);
 
+	dev->dev_ops = NULL;
+	dev->rx_pkt_burst = NULL;
+	dev->tx_pkt_burst = NULL;
+
+	/* Clear PXE mode */
+	i40e_clear_pxe_mode(hw);
+
+	/* Unconfigure filter control */
+	memset(&settings, 0, sizeof(settings));
+	ret = i40e_set_filter_control(hw, &settings);
+	if (ret)
+		PMD_INIT_LOG(WARNING, "setup_pf_filter_control failed: %d",
+					ret);
+
+	/* Disable flow control */
+	hw->fc.requested_mode = I40E_FC_NONE;
+	i40e_set_fc(hw, &aq_fail, TRUE);
+
+	/* uninitialize pf host driver */
+	i40e_pf_host_uninit(dev);
+
+	do {
+		ret = rte_intr_callback_unregister(intr_handle,
+				i40e_dev_interrupt_handler, dev);
+		if (ret >= 0) {
+			break;
+		} else if (ret != -EAGAIN) {
+			PMD_INIT_LOG(ERR,
+				 "intr callback unregister failed: %d",
+				 ret);
+		}
+		i40e_msec_delay(500);
+	} while (retries++ < 5);
+
+	i40e_rm_ethtype_filter_list(pf);
+	i40e_rm_tunnel_filter_list(pf);
+	i40e_rm_fdir_filter_list(pf);
+
+	/* Remove all flows */
+	while ((p_flow = TAILQ_FIRST(&pf->flow_list))) {
+		TAILQ_REMOVE(&pf->flow_list, p_flow, node);
+		rte_free(p_flow);
+	}
+
+	/* Remove all Traffic Manager configuration */
+	i40e_tm_conf_uninit(dev);
+
 	hw->adapter_closed = 1;
 }
 
diff --git a/drivers/net/i40e/i40e_ethdev_vf.c b/drivers/net/i40e/i40e_ethdev_vf.c
index 22e5e1409..4395a011e 100644
--- a/drivers/net/i40e/i40e_ethdev_vf.c
+++ b/drivers/net/i40e/i40e_ethdev_vf.c
@@ -1310,17 +1310,12 @@ i40evf_init_vf(struct rte_eth_dev *dev)
 static int
 i40evf_uninit_vf(struct rte_eth_dev *dev)
 {
-	struct i40e_vf *vf = I40EVF_DEV_PRIVATE_TO_VF(dev->data->dev_private);
 	struct i40e_hw *hw = I40E_DEV_PRIVATE_TO_HW(dev->data->dev_private);
 
 	PMD_INIT_FUNC_TRACE();
 
 	if (hw->adapter_closed == 0)
 		i40evf_dev_close(dev);
-	rte_free(vf->vf_res);
-	vf->vf_res = NULL;
-	rte_free(vf->aq_resp);
-	vf->aq_resp = NULL;
 
 	return 0;
 }
@@ -1498,6 +1493,11 @@ i40evf_dev_init(struct rte_eth_dev *eth_dev)
 	hw->adapter_stopped = 0;
 	hw->adapter_closed = 0;
 
+	/* Pass the information to the rte_eth_dev_close() that it should also
+	 * release the private port resources.
+	 */
+	eth_dev->data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
+
 	if(i40evf_init_vf(eth_dev) != 0) {
 		PMD_INIT_LOG(ERR, "Init vf failed");
 		return -1;
@@ -1534,10 +1534,6 @@ i40evf_dev_uninit(struct rte_eth_dev *eth_dev)
 	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
 		return -EPERM;
 
-	eth_dev->dev_ops = NULL;
-	eth_dev->rx_pkt_burst = NULL;
-	eth_dev->tx_pkt_burst = NULL;
-
 	if (i40evf_uninit_vf(eth_dev) != 0) {
 		PMD_INIT_LOG(ERR, "i40evf_uninit_vf failed");
 		return -1;
@@ -2334,6 +2330,7 @@ static void
 i40evf_dev_close(struct rte_eth_dev *dev)
 {
 	struct i40e_hw *hw = I40E_DEV_PRIVATE_TO_HW(dev->data->dev_private);
+	struct i40e_vf *vf = I40EVF_DEV_PRIVATE_TO_VF(dev->data->dev_private);
 
 	i40evf_dev_stop(dev);
 	i40e_dev_free_queues(dev);
@@ -2349,6 +2346,16 @@ i40evf_dev_close(struct rte_eth_dev *dev)
 	i40evf_reset_vf(dev);
 	i40e_shutdown_adminq(hw);
 	i40evf_disable_irq0(hw);
+
+	dev->dev_ops = NULL;
+	dev->rx_pkt_burst = NULL;
+	dev->tx_pkt_burst = NULL;
+
+	rte_free(vf->vf_res);
+	vf->vf_res = NULL;
+	rte_free(vf->aq_resp);
+	vf->aq_resp = NULL;
+
 	hw->adapter_closed = 1;
 }
 
-- 
2.17.1

[dpdk-dev] [PATCH v6 4/5] net/ice: release port upon close

[Di ChenxuX]

Set RTE_ETH_DEV_CLOSE_REMOVE upon probe so all the private resources
 for the port can be freed by rte_eth_dev_close().

Signed-off-by: Di ChenxuX <chenxux.di@intel.com>
---
 doc/guides/rel_notes/release_19_11.rst |  1 +
 drivers/net/ice/ice_ethdev.c           | 30 ++++++++++++--------------
 2 files changed, 15 insertions(+), 16 deletions(-)

diff --git a/doc/guides/rel_notes/release_19_11.rst b/doc/guides/rel_notes/release_19_11.rst
index 7e3290665..751606440 100644
--- a/doc/guides/rel_notes/release_19_11.rst
+++ b/doc/guides/rel_notes/release_19_11.rst
@@ -63,6 +63,7 @@ New Features
   * Added support for device-specific DDP package loading.
   * Added support for handling Receive Flex Descriptor.
   * Added support for protocol extraction on per Rx queue.
+  * Added support for the ``RTE_ETH_DEV_CLOSE_REMOVE`` flag.
 
 * **Updated the Intel e1000 driver.**
 
diff --git a/drivers/net/ice/ice_ethdev.c b/drivers/net/ice/ice_ethdev.c
index 0a514dc10..d3f4ebf38 100644
--- a/drivers/net/ice/ice_ethdev.c
+++ b/drivers/net/ice/ice_ethdev.c
@@ -1924,6 +1924,11 @@ ice_dev_init(struct rte_eth_dev *dev)
 		goto err_init_mac;
 	}
 
+	/* Pass the information to the rte_eth_dev_close() that it should also
+	 * release the private port resources.
+	 */
+	dev->data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
+
 	ret = ice_res_pool_init(&pf->msix_pool, 1,
 				hw->func_caps.common_cap.num_msix_vectors - 1);
 	if (ret) {
@@ -2090,6 +2095,9 @@ ice_dev_close(struct rte_eth_dev *dev)
 {
 	struct ice_pf *pf = ICE_DEV_PRIVATE_TO_PF(dev->data->dev_private);
 	struct ice_hw *hw = ICE_DEV_PRIVATE_TO_HW(dev->data->dev_private);
+	struct rte_pci_device *pci_dev = RTE_ETH_DEV_TO_PCI(dev);
+	struct rte_intr_handle *intr_handle = &pci_dev->intr_handle;
+	struct rte_flow *p_flow;
 
 	/* Since stop will make link down, then the link event will be
 	 * triggered, disable the irq firstly to avoid the port_infoe etc
@@ -2109,22 +2117,6 @@ ice_dev_close(struct rte_eth_dev *dev)
 	rte_free(hw->port_info);
 	hw->port_info = NULL;
 	ice_shutdown_all_ctrlq(hw);
-	rte_free(pf->proto_xtr);
-	pf->proto_xtr = NULL;
-}
-
-static int
-ice_dev_uninit(struct rte_eth_dev *dev)
-{
-	struct rte_pci_device *pci_dev = RTE_ETH_DEV_TO_PCI(dev);
-	struct rte_intr_handle *intr_handle = &pci_dev->intr_handle;
-	struct ice_pf *pf = ICE_DEV_PRIVATE_TO_PF(dev->data->dev_private);
-	struct rte_flow *p_flow;
-
-	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
-		return 0;
-
-	ice_dev_close(dev);
 
 	dev->dev_ops = NULL;
 	dev->rx_pkt_burst = NULL;
@@ -2146,6 +2138,12 @@ ice_dev_uninit(struct rte_eth_dev *dev)
 		ice_free_switch_filter_rule(p_flow->rule);
 		rte_free(p_flow);
 	}
+}
+
+static int
+ice_dev_uninit(struct rte_eth_dev *dev)
+{
+	ice_dev_close(dev);
 
 	return 0;
 }
-- 
2.17.1

[dpdk-dev] [PATCH v6 5/5] net/ixgbe: release port upon close

[Di ChenxuX]

Set RTE_ETH_DEV_CLOSE_REMOVE upon probe so all the private resources
 for the port can be freed by rte_eth_dev_close().

Signed-off-by: Di ChenxuX <chenxux.di@intel.com>
---
 doc/guides/rel_notes/release_19_11.rst |   4 +
 drivers/net/ixgbe/ixgbe_ethdev.c       | 167 +++++++++++++------------
 2 files changed, 89 insertions(+), 82 deletions(-)

diff --git a/doc/guides/rel_notes/release_19_11.rst b/doc/guides/rel_notes/release_19_11.rst
index 751606440..44011085d 100644
--- a/doc/guides/rel_notes/release_19_11.rst
+++ b/doc/guides/rel_notes/release_19_11.rst
@@ -77,6 +77,10 @@ New Features
 
   Added support for the ``RTE_ETH_DEV_CLOSE_REMOVE`` flag.
 
+* **Updated the Intel ixgbe driver.**
+
+  Added support for the ``RTE_ETH_DEV_CLOSE_REMOVE`` flag.
+
 
 Removed Items
 -------------
diff --git a/drivers/net/ixgbe/ixgbe_ethdev.c b/drivers/net/ixgbe/ixgbe_ethdev.c
index 023b267d7..e6879c4b0 100644
--- a/drivers/net/ixgbe/ixgbe_ethdev.c
+++ b/drivers/net/ixgbe/ixgbe_ethdev.c
@@ -1245,6 +1245,11 @@ eth_ixgbe_dev_init(struct rte_eth_dev *eth_dev, void *init_params __rte_unused)
 		return -ENOMEM;
 	}
 
+	/* Pass the information to the rte_eth_dev_close() that it should also
+	 * release the private port resources.
+	 */
+	eth_dev->data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
+
 	/* initialize the vfta */
 	memset(shadow_vfta, 0, sizeof(*shadow_vfta));
 
@@ -1311,73 +1316,12 @@ eth_ixgbe_dev_init(struct rte_eth_dev *eth_dev, void *init_params __rte_unused)
 static int
 eth_ixgbe_dev_uninit(struct rte_eth_dev *eth_dev)
 {
-	struct rte_pci_device *pci_dev = RTE_ETH_DEV_TO_PCI(eth_dev);
-	struct rte_intr_handle *intr_handle = &pci_dev->intr_handle;
-	struct ixgbe_hw *hw;
-	int retries = 0;
-	int ret;
-
 	PMD_INIT_FUNC_TRACE();
 
 	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
 		return 0;
 
-	hw = IXGBE_DEV_PRIVATE_TO_HW(eth_dev->data->dev_private);
-
-	if (hw->adapter_stopped == 0)
-		ixgbe_dev_close(eth_dev);
-
-	eth_dev->dev_ops = NULL;
-	eth_dev->rx_pkt_burst = NULL;
-	eth_dev->tx_pkt_burst = NULL;
-
-	/* Unlock any pending hardware semaphore */
-	ixgbe_swfw_lock_reset(hw);
-
-	/* disable uio intr before callback unregister */
-	rte_intr_disable(intr_handle);
-
-	do {
-		ret = rte_intr_callback_unregister(intr_handle,
-				ixgbe_dev_interrupt_handler, eth_dev);
-		if (ret >= 0) {
-			break;
-		} else if (ret != -EAGAIN) {
-			PMD_INIT_LOG(ERR,
-				"intr callback unregister failed: %d",
-				ret);
-			return ret;
-		}
-		rte_delay_ms(100);
-	} while (retries++ < (10 + IXGBE_LINK_UP_TIME));
-
-	/* cancel the delay handler before remove dev */
-	rte_eal_alarm_cancel(ixgbe_dev_interrupt_delayed_handler, eth_dev);
-
-	/* cancel the link handler before remove dev */
-	rte_eal_alarm_cancel(ixgbe_dev_setup_link_alarm_handler, eth_dev);
-
-	/* uninitialize PF if max_vfs not zero */
-	ixgbe_pf_host_uninit(eth_dev);
-
-	/* remove all the fdir filters & hash */
-	ixgbe_fdir_filter_uninit(eth_dev);
-
-	/* remove all the L2 tunnel filters & hash */
-	ixgbe_l2_tn_filter_uninit(eth_dev);
-
-	/* Remove all ntuple filters of the device */
-	ixgbe_ntuple_filter_uninit(eth_dev);
-
-	/* clear all the filters list */
-	ixgbe_filterlist_flush();
-
-	/* Remove all Traffic Manager configuration */
-	ixgbe_tm_conf_uninit(eth_dev);
-
-#ifdef RTE_LIBRTE_SECURITY
-	rte_free(eth_dev->security_ctx);
-#endif
+	ixgbe_dev_close(eth_dev);
 
 	return 0;
 }
@@ -1711,6 +1655,11 @@ eth_ixgbevf_dev_init(struct rte_eth_dev *eth_dev)
 		return -ENOMEM;
 	}
 
+	/* Pass the information to the rte_eth_dev_close() that it should also
+	 * release the private port resources.
+	 */
+	eth_dev->data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
+
 	/* Generate a random MAC address, if none was assigned by PF. */
 	if (rte_is_zero_ether_addr(perm_addr)) {
 		generate_random_mac_addr(perm_addr);
@@ -1762,30 +1711,12 @@ eth_ixgbevf_dev_init(struct rte_eth_dev *eth_dev)
 static int
 eth_ixgbevf_dev_uninit(struct rte_eth_dev *eth_dev)
 {
-	struct rte_pci_device *pci_dev = RTE_ETH_DEV_TO_PCI(eth_dev);
-	struct rte_intr_handle *intr_handle = &pci_dev->intr_handle;
-	struct ixgbe_hw *hw;
-
 	PMD_INIT_FUNC_TRACE();
 
 	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
 		return 0;
 
-	hw = IXGBE_DEV_PRIVATE_TO_HW(eth_dev->data->dev_private);
-
-	if (hw->adapter_stopped == 0)
-		ixgbevf_dev_close(eth_dev);
-
-	eth_dev->dev_ops = NULL;
-	eth_dev->rx_pkt_burst = NULL;
-	eth_dev->tx_pkt_burst = NULL;
-
-	/* Disable the interrupts for VF */
-	ixgbevf_intr_disable(eth_dev);
-
-	rte_intr_disable(intr_handle);
-	rte_intr_callback_unregister(intr_handle,
-				     ixgbevf_dev_interrupt_handler, eth_dev);
+	ixgbevf_dev_close(eth_dev);
 
 	return 0;
 }
@@ -2876,6 +2807,9 @@ ixgbe_dev_stop(struct rte_eth_dev *dev)
 	struct ixgbe_tm_conf *tm_conf =
 		IXGBE_DEV_PRIVATE_TO_TM_CONF(dev->data->dev_private);
 
+	if (hw->adapter_stopped)
+		return;
+
 	PMD_INIT_FUNC_TRACE();
 
 	rte_eal_alarm_cancel(ixgbe_dev_setup_link_alarm_handler, dev);
@@ -2928,6 +2862,8 @@ ixgbe_dev_stop(struct rte_eth_dev *dev)
 	tm_conf->committed = false;
 
 	adapter->rss_reta_updated = 0;
+
+	hw->adapter_stopped = true;
 }
 
 /*
@@ -2998,13 +2934,16 @@ ixgbe_dev_close(struct rte_eth_dev *dev)
 {
 	struct ixgbe_hw *hw =
 		IXGBE_DEV_PRIVATE_TO_HW(dev->data->dev_private);
+	struct rte_pci_device *pci_dev = RTE_ETH_DEV_TO_PCI(dev);
+	struct rte_intr_handle *intr_handle = &pci_dev->intr_handle;
+	int retries = 0;
+	int ret;
 
 	PMD_INIT_FUNC_TRACE();
 
 	ixgbe_pf_reset_hw(hw);
 
 	ixgbe_dev_stop(dev);
-	hw->adapter_stopped = 1;
 
 	ixgbe_dev_free_queues(dev);
 
@@ -3012,6 +2951,55 @@ ixgbe_dev_close(struct rte_eth_dev *dev)
 
 	/* reprogram the RAR[0] in case user changed it. */
 	ixgbe_set_rar(hw, 0, hw->mac.addr, 0, IXGBE_RAH_AV);
+
+	dev->dev_ops = NULL;
+	dev->rx_pkt_burst = NULL;
+	dev->tx_pkt_burst = NULL;
+
+	/* Unlock any pending hardware semaphore */
+	ixgbe_swfw_lock_reset(hw);
+
+	/* disable uio intr before callback unregister */
+	rte_intr_disable(intr_handle);
+
+	do {
+		ret = rte_intr_callback_unregister(intr_handle,
+				ixgbe_dev_interrupt_handler, dev);
+		if (ret >= 0) {
+			break;
+		} else if (ret != -EAGAIN) {
+			PMD_INIT_LOG(ERR,
+				"intr callback unregister failed: %d",
+				ret);
+		}
+		rte_delay_ms(100);
+	} while (retries++ < (10 + IXGBE_LINK_UP_TIME));
+
+	/* cancel the delay handler before remove dev */
+	rte_eal_alarm_cancel(ixgbe_dev_interrupt_delayed_handler, dev);
+
+	/* uninitialize PF if max_vfs not zero */
+	ixgbe_pf_host_uninit(dev);
+
+	/* remove all the fdir filters & hash */
+	ixgbe_fdir_filter_uninit(dev);
+
+	/* remove all the L2 tunnel filters & hash */
+	ixgbe_l2_tn_filter_uninit(dev);
+
+	/* Remove all ntuple filters of the device */
+	ixgbe_ntuple_filter_uninit(dev);
+
+	/* clear all the filters list */
+	ixgbe_filterlist_flush();
+
+	/* Remove all Traffic Manager configuration */
+	ixgbe_tm_conf_uninit(dev);
+
+#ifdef RTE_LIBRTE_SECURITY
+	rte_free(dev->security_ctx);
+#endif
+
 }
 
 /*
@@ -5244,6 +5232,8 @@ ixgbevf_dev_start(struct rte_eth_dev *dev)
 	 */
 	ixgbevf_dev_link_update(dev, 0);
 
+	hw->adapter_stopped = false;
+
 	return 0;
 }
 
@@ -5255,6 +5245,9 @@ ixgbevf_dev_stop(struct rte_eth_dev *dev)
 	struct rte_pci_device *pci_dev = RTE_ETH_DEV_TO_PCI(dev);
 	struct rte_intr_handle *intr_handle = &pci_dev->intr_handle;
 
+	if (hw->adapter_stopped)
+		return;
+
 	PMD_INIT_FUNC_TRACE();
 
 	rte_eal_alarm_cancel(ixgbe_dev_setup_link_alarm_handler, dev);
@@ -5289,6 +5282,8 @@ static void
 ixgbevf_dev_close(struct rte_eth_dev *dev)
 {
 	struct ixgbe_hw *hw = IXGBE_DEV_PRIVATE_TO_HW(dev->data->dev_private);
+	struct rte_pci_device *pci_dev = RTE_ETH_DEV_TO_PCI(dev);
+	struct rte_intr_handle *intr_handle = &pci_dev->intr_handle;
 
 	PMD_INIT_FUNC_TRACE();
 
@@ -5304,6 +5299,14 @@ ixgbevf_dev_close(struct rte_eth_dev *dev)
 	 * after stop, close and detach of the VF
 	 **/
 	ixgbevf_remove_mac_addr(dev, 0);
+
+	dev->dev_ops = NULL;
+	dev->rx_pkt_burst = NULL;
+	dev->tx_pkt_burst = NULL;
+
+	rte_intr_disable(intr_handle);
+	rte_intr_callback_unregister(intr_handle,
+				     ixgbevf_dev_interrupt_handler, dev);
 }
 
 /*
-- 
2.17.1

Re: [dpdk-dev] [PATCH v6 0/5] drivers/net: release port upon close

[Yang, Qiming]

> -----Original Message-----
> From: Di, ChenxuX
> Sent: Thursday, September 26, 2019 6:00 PM
> To: dev@dpdk.org
> Cc: Yang, Qiming <qiming.yang@intel.com>; Di, ChenxuX
> <chenxux.di@intel.com>
> Subject: [PATCH v6 0/5] drivers/net: release port upon close
> 
> Set RTE_ETH_DEV_CLOSE_REMOVE upon probe so all the private resources
> for the port can be freed by rte_eth_dev_close().
> This patch cover all the intel drivers.
> 
> ---
> V6 changes:
>   Fixed check error.
> V5 changes:
>   Added code in em_ethdev.c.
>   Separated the release note, each patch has its own note.
> V4 changes:
>   Removed adaper stopped flag in function ice_dev_init.
> V3 changes:
>   Removed adapter closed flag in driver e1000, fm10k, ice, ixgbe.
> V2 changes:
>   Separated to 5 patches, each patch only change one driver.
>   Added resource release in dev_close.
> 
> Di ChenxuX (5):
>   net/e1000: release port upon close
>   net/fm10k: release port upon close
>   net/i40e: release port upon close
>   net/ice: release port upon close
>   net/ixgbe: release port upon close
> 
>  doc/guides/rel_notes/release_19_11.rst |  18 +++
>  drivers/net/e1000/em_ethdev.c          |  33 ++---
>  drivers/net/e1000/igb_ethdev.c         | 144 ++++++++++-----------
>  drivers/net/fm10k/fm10k_ethdev.c       | 100 ++++++++-------
>  drivers/net/i40e/i40e_ethdev.c         | 128 +++++++++----------
>  drivers/net/i40e/i40e_ethdev_vf.c      |  25 ++--
>  drivers/net/ice/ice_ethdev.c           |  30 +++--
>  drivers/net/ixgbe/ixgbe_ethdev.c       | 167 +++++++++++++------------
>  8 files changed, 335 insertions(+), 310 deletions(-)
> 
> --
> 2.17.1

Acked-by: Qiming Yang <qiming.yang@intel.com>

[dpdk-dev] [PATCH v7 0/5] drivers/net: release port upon close

[Di ChenxuX]

Set RTE_ETH_DEV_CLOSE_REMOVE upon probe so all the private resources
 for the port can be freed by rte_eth_dev_close().
This patch cover all the intel drivers.

---
V7 changes:
  Modified code on the last commit.
V6 changes:
  Fixed check error.
V5 changes:
  Added code in em_ethdev.c.
  Separated the release note, each patch has its own note. 
V4 changes:
  Removed adaper stopped flag in function ice_dev_init.
V3 changes:
  Removed adapter closed flag in driver e1000, fm10k, ice, ixgbe.
V2 changes:
  Separated to 5 patches, each patch only change one driver.
  Added resource release in dev_close.

Acked-by: Qiming Yang <qiming.yang@intel.com>
Series Reviewed-by: Xiaolong Ye <xiaolong.ye@intel.com>

Di ChenxuX (5):
  net/e1000: release port upon close
  net/fm10k: release port upon close
  net/i40e: release port upon close
  net/ice: release port upon close
  net/ixgbe: release port upon close

 doc/guides/rel_notes/release_19_11.rst |  18 +++
 drivers/net/e1000/em_ethdev.c          |  33 ++---
 drivers/net/e1000/igb_ethdev.c         | 144 ++++++++++-----------
 drivers/net/fm10k/fm10k_ethdev.c       | 100 ++++++++-------
 drivers/net/i40e/i40e_ethdev.c         | 128 +++++++++----------
 drivers/net/i40e/i40e_ethdev_vf.c      |  25 ++--
 drivers/net/ice/ice_ethdev.c           |  30 +++--
 drivers/net/ixgbe/ixgbe_ethdev.c       | 167 +++++++++++++------------
 8 files changed, 335 insertions(+), 310 deletions(-)

-- 
2.17.1

[dpdk-dev] [PATCH v7 1/5] net/e1000: release port upon close

[Di ChenxuX]

Set RTE_ETH_DEV_CLOSE_REMOVE upon probe so all the private resources
 for the port can be freed by rte_eth_dev_close().

Signed-off-by: Di ChenxuX <chenxux.di@intel.com>
---
 doc/guides/rel_notes/release_19_11.rst |   5 +
 drivers/net/e1000/em_ethdev.c          |  33 +++---
 drivers/net/e1000/igb_ethdev.c         | 144 ++++++++++++-------------
 3 files changed, 94 insertions(+), 88 deletions(-)

diff --git a/doc/guides/rel_notes/release_19_11.rst b/doc/guides/rel_notes/release_19_11.rst
index 1b8bd6f35..37cd5fe16 100644
--- a/doc/guides/rel_notes/release_19_11.rst
+++ b/doc/guides/rel_notes/release_19_11.rst
@@ -64,6 +64,11 @@ New Features
   * Added support for handling Receive Flex Descriptor.
   * Added support for protocol extraction on per Rx queue.
 
+* **Updated the Intel e1000 driver.**
+
+  Added support for the ``RTE_ETH_DEV_CLOSE_REMOVE`` flag.
+
+
 Removed Items
 -------------
 
diff --git a/drivers/net/e1000/em_ethdev.c b/drivers/net/e1000/em_ethdev.c
index 3c2178ad6..9a88b50b2 100644
--- a/drivers/net/e1000/em_ethdev.c
+++ b/drivers/net/e1000/em_ethdev.c
@@ -297,6 +297,11 @@ eth_em_dev_init(struct rte_eth_dev *eth_dev)
 	rte_ether_addr_copy((struct rte_ether_addr *)hw->mac.addr,
 		eth_dev->data->mac_addrs);
 
+	/* Pass the information to the rte_eth_dev_close() that it should also
+	 * release the private port resources.
+	 */
+	eth_dev->data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
+
 	/* initialize the vfta */
 	memset(shadow_vfta, 0, sizeof(*shadow_vfta));
 
@@ -313,27 +318,12 @@ eth_em_dev_init(struct rte_eth_dev *eth_dev)
 static int
 eth_em_dev_uninit(struct rte_eth_dev *eth_dev)
 {
-	struct rte_pci_device *pci_dev = RTE_ETH_DEV_TO_PCI(eth_dev);
-	struct e1000_adapter *adapter =
-		E1000_DEV_PRIVATE(eth_dev->data->dev_private);
-	struct rte_intr_handle *intr_handle = &pci_dev->intr_handle;
-
 	PMD_INIT_FUNC_TRACE();
 
 	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
 		return -EPERM;
 
-	if (adapter->stopped == 0)
-		eth_em_close(eth_dev);
-
-	eth_dev->dev_ops = NULL;
-	eth_dev->rx_pkt_burst = NULL;
-	eth_dev->tx_pkt_burst = NULL;
-
-	/* disable uio intr before callback unregister */
-	rte_intr_disable(intr_handle);
-	rte_intr_callback_unregister(intr_handle,
-				     eth_em_interrupt_handler, eth_dev);
+	eth_em_close(eth_dev);
 
 	return 0;
 }
@@ -774,6 +764,8 @@ eth_em_close(struct rte_eth_dev *dev)
 	struct e1000_hw *hw = E1000_DEV_PRIVATE_TO_HW(dev->data->dev_private);
 	struct e1000_adapter *adapter =
 		E1000_DEV_PRIVATE(dev->data->dev_private);
+	struct rte_pci_device *pci_dev = RTE_ETH_DEV_TO_PCI(dev);
+	struct rte_intr_handle *intr_handle = &pci_dev->intr_handle;
 
 	eth_em_stop(dev);
 	adapter->stopped = 1;
@@ -781,6 +773,15 @@ eth_em_close(struct rte_eth_dev *dev)
 	e1000_phy_hw_reset(hw);
 	em_release_manageability(hw);
 	em_hw_control_release(hw);
+
+	dev->dev_ops = NULL;
+	dev->rx_pkt_burst = NULL;
+	dev->tx_pkt_burst = NULL;
+
+	/* disable uio intr before callback unregister */
+	rte_intr_disable(intr_handle);
+	rte_intr_callback_unregister(intr_handle,
+				     eth_em_interrupt_handler, dev);
 }
 
 static int
diff --git a/drivers/net/e1000/igb_ethdev.c b/drivers/net/e1000/igb_ethdev.c
index 80f1e13d7..ce7c9e664 100644
--- a/drivers/net/e1000/igb_ethdev.c
+++ b/drivers/net/e1000/igb_ethdev.c
@@ -843,6 +843,11 @@ eth_igb_dev_init(struct rte_eth_dev *eth_dev)
 	rte_ether_addr_copy((struct rte_ether_addr *)hw->mac.addr,
 			&eth_dev->data->mac_addrs[0]);
 
+	/* Pass the information to the rte_eth_dev_close() that it should also
+	 * release the private port resources.
+	 */
+	eth_dev->data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
+
 	/* initialize the vfta */
 	memset(shadow_vfta, 0, sizeof(*shadow_vfta));
 
@@ -912,61 +917,12 @@ eth_igb_dev_init(struct rte_eth_dev *eth_dev)
 static int
 eth_igb_dev_uninit(struct rte_eth_dev *eth_dev)
 {
-	struct rte_pci_device *pci_dev;
-	struct rte_intr_handle *intr_handle;
-	struct e1000_hw *hw;
-	struct e1000_adapter *adapter =
-		E1000_DEV_PRIVATE(eth_dev->data->dev_private);
-	struct e1000_filter_info *filter_info =
-		E1000_DEV_PRIVATE_TO_FILTER_INFO(eth_dev->data->dev_private);
-
 	PMD_INIT_FUNC_TRACE();
 
 	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
 		return -EPERM;
 
-	hw = E1000_DEV_PRIVATE_TO_HW(eth_dev->data->dev_private);
-	pci_dev = RTE_ETH_DEV_TO_PCI(eth_dev);
-	intr_handle = &pci_dev->intr_handle;
-
-	if (adapter->stopped == 0)
-		eth_igb_close(eth_dev);
-
-	eth_dev->dev_ops = NULL;
-	eth_dev->rx_pkt_burst = NULL;
-	eth_dev->tx_pkt_burst = NULL;
-
-	/* Reset any pending lock */
-	igb_reset_swfw_lock(hw);
-
-	/* uninitialize PF if max_vfs not zero */
-	igb_pf_host_uninit(eth_dev);
-
-	/* disable uio intr before callback unregister */
-	rte_intr_disable(intr_handle);
-	rte_intr_callback_unregister(intr_handle,
-				     eth_igb_interrupt_handler, eth_dev);
-
-	/* clear the SYN filter info */
-	filter_info->syn_info = 0;
-
-	/* clear the ethertype filters info */
-	filter_info->ethertype_mask = 0;
-	memset(filter_info->ethertype_filters, 0,
-		E1000_MAX_ETQF_FILTERS * sizeof(struct igb_ethertype_filter));
-
-	/* clear the rss filter info */
-	memset(&filter_info->rss_info, 0,
-		sizeof(struct igb_rte_flow_rss_conf));
-
-	/* remove all ntuple filters of the device */
-	igb_ntuple_filter_uninit(eth_dev);
-
-	/* remove all flex filters of the device */
-	igb_flex_filter_uninit(eth_dev);
-
-	/* clear all the filters list */
-	igb_filterlist_flush(eth_dev);
+	eth_igb_close(eth_dev);
 
 	return 0;
 }
@@ -1038,6 +994,11 @@ eth_igbvf_dev_init(struct rte_eth_dev *eth_dev)
 		return -ENOMEM;
 	}
 
+	/* Pass the information to the rte_eth_dev_close() that it should also
+	 * release the private port resources.
+	 */
+	eth_dev->data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
+
 	/* Generate a random MAC address, if none was assigned by PF. */
 	if (rte_is_zero_ether_addr(perm_addr)) {
 		rte_eth_random_addr(perm_addr->addr_bytes);
@@ -1077,27 +1038,12 @@ eth_igbvf_dev_init(struct rte_eth_dev *eth_dev)
 static int
 eth_igbvf_dev_uninit(struct rte_eth_dev *eth_dev)
 {
-	struct e1000_adapter *adapter =
-		E1000_DEV_PRIVATE(eth_dev->data->dev_private);
-	struct rte_pci_device *pci_dev = RTE_ETH_DEV_TO_PCI(eth_dev);
-
 	PMD_INIT_FUNC_TRACE();
 
 	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
 		return -EPERM;
 
-	if (adapter->stopped == 0)
-		igbvf_dev_close(eth_dev);
-
-	eth_dev->dev_ops = NULL;
-	eth_dev->rx_pkt_burst = NULL;
-	eth_dev->tx_pkt_burst = NULL;
-
-	/* disable uio intr before callback unregister */
-	rte_intr_disable(&pci_dev->intr_handle);
-	rte_intr_callback_unregister(&pci_dev->intr_handle,
-				     eth_igbvf_interrupt_handler,
-				     (void *)eth_dev);
+	igbvf_dev_close(eth_dev);
 
 	return 0;
 }
@@ -1506,6 +1452,11 @@ eth_igb_stop(struct rte_eth_dev *dev)
 	struct rte_pci_device *pci_dev = RTE_ETH_DEV_TO_PCI(dev);
 	struct rte_eth_link link;
 	struct rte_intr_handle *intr_handle = &pci_dev->intr_handle;
+	struct e1000_adapter *adapter =
+		E1000_DEV_PRIVATE(dev->data->dev_private);
+
+	if (adapter->stopped)
+		return;
 
 	eth_igb_rxtx_control(dev, false);
 
@@ -1547,6 +1498,8 @@ eth_igb_stop(struct rte_eth_dev *dev)
 		rte_free(intr_handle->intr_vec);
 		intr_handle->intr_vec = NULL;
 	}
+
+	adapter->stopped = true;
 }
 
 static int
@@ -1579,14 +1532,13 @@ static void
 eth_igb_close(struct rte_eth_dev *dev)
 {
 	struct e1000_hw *hw = E1000_DEV_PRIVATE_TO_HW(dev->data->dev_private);
-	struct e1000_adapter *adapter =
-		E1000_DEV_PRIVATE(dev->data->dev_private);
 	struct rte_eth_link link;
 	struct rte_pci_device *pci_dev = RTE_ETH_DEV_TO_PCI(dev);
 	struct rte_intr_handle *intr_handle = &pci_dev->intr_handle;
+	struct e1000_filter_info *filter_info =
+		E1000_DEV_PRIVATE_TO_FILTER_INFO(dev->data->dev_private);
 
 	eth_igb_stop(dev);
-	adapter->stopped = 1;
 
 	e1000_phy_hw_reset(hw);
 	igb_release_manageability(hw);
@@ -1610,6 +1562,40 @@ eth_igb_close(struct rte_eth_dev *dev)
 
 	memset(&link, 0, sizeof(link));
 	rte_eth_linkstatus_set(dev, &link);
+
+	dev->dev_ops = NULL;
+	dev->rx_pkt_burst = NULL;
+	dev->tx_pkt_burst = NULL;
+
+	/* Reset any pending lock */
+	igb_reset_swfw_lock(hw);
+
+	/* uninitialize PF if max_vfs not zero */
+	igb_pf_host_uninit(dev);
+
+	rte_intr_callback_unregister(intr_handle,
+				     eth_igb_interrupt_handler, dev);
+
+	/* clear the SYN filter info */
+	filter_info->syn_info = 0;
+
+	/* clear the ethertype filters info */
+	filter_info->ethertype_mask = 0;
+	memset(filter_info->ethertype_filters, 0,
+		E1000_MAX_ETQF_FILTERS * sizeof(struct igb_ethertype_filter));
+
+	/* clear the rss filter info */
+	memset(&filter_info->rss_info, 0,
+		sizeof(struct igb_rte_flow_rss_conf));
+
+	/* remove all ntuple filters of the device */
+	igb_ntuple_filter_uninit(dev);
+
+	/* remove all flex filters of the device */
+	igb_flex_filter_uninit(dev);
+
+	/* clear all the filters list */
+	igb_filterlist_flush(dev);
 }
 
 /*
@@ -3354,6 +3340,11 @@ igbvf_dev_stop(struct rte_eth_dev *dev)
 {
 	struct rte_pci_device *pci_dev = RTE_ETH_DEV_TO_PCI(dev);
 	struct rte_intr_handle *intr_handle = &pci_dev->intr_handle;
+	struct e1000_adapter *adapter =
+		E1000_DEV_PRIVATE(dev->data->dev_private);
+
+	if (adapter->stopped)
+		return;
 
 	PMD_INIT_FUNC_TRACE();
 
@@ -3376,22 +3367,23 @@ igbvf_dev_stop(struct rte_eth_dev *dev)
 		rte_free(intr_handle->intr_vec);
 		intr_handle->intr_vec = NULL;
 	}
+
+	adapter->stopped = true;
 }
 
 static void
 igbvf_dev_close(struct rte_eth_dev *dev)
 {
 	struct e1000_hw *hw = E1000_DEV_PRIVATE_TO_HW(dev->data->dev_private);
-	struct e1000_adapter *adapter =
-		E1000_DEV_PRIVATE(dev->data->dev_private);
 	struct rte_ether_addr addr;
+	struct rte_pci_device *pci_dev = RTE_ETH_DEV_TO_PCI(dev);
 
 	PMD_INIT_FUNC_TRACE();
 
 	e1000_reset_hw(hw);
 
 	igbvf_dev_stop(dev);
-	adapter->stopped = 1;
+
 	igb_dev_free_queues(dev);
 
 	/**
@@ -3402,6 +3394,14 @@ igbvf_dev_close(struct rte_eth_dev *dev)
 
 	memset(&addr, 0, sizeof(addr));
 	igbvf_default_mac_addr_set(dev, &addr);
+
+	dev->dev_ops = NULL;
+	dev->rx_pkt_burst = NULL;
+	dev->tx_pkt_burst = NULL;
+
+	rte_intr_callback_unregister(&pci_dev->intr_handle,
+				     eth_igbvf_interrupt_handler,
+				     (void *)dev);
 }
 
 static int
-- 
2.17.1

[dpdk-dev] [PATCH v7 2/5] net/fm10k: release port upon close

[Di ChenxuX]

Set RTE_ETH_DEV_CLOSE_REMOVE upon probe so all the private resources
 for the port can be freed by rte_eth_dev_close().

Signed-off-by: Di ChenxuX <chenxux.di@intel.com>
---
 doc/guides/rel_notes/release_19_11.rst |   4 +
 drivers/net/fm10k/fm10k_ethdev.c       | 100 +++++++++++++------------
 2 files changed, 56 insertions(+), 48 deletions(-)

diff --git a/doc/guides/rel_notes/release_19_11.rst b/doc/guides/rel_notes/release_19_11.rst
index 37cd5fe16..54a4f1001 100644
--- a/doc/guides/rel_notes/release_19_11.rst
+++ b/doc/guides/rel_notes/release_19_11.rst
@@ -68,6 +68,10 @@ New Features
 
   Added support for the ``RTE_ETH_DEV_CLOSE_REMOVE`` flag.
 
+* **Updated the Intel fm10k driver.**
+
+  Added support for the ``RTE_ETH_DEV_CLOSE_REMOVE`` flag.
+
 
 Removed Items
 -------------
diff --git a/drivers/net/fm10k/fm10k_ethdev.c b/drivers/net/fm10k/fm10k_ethdev.c
index 9fbedbfe6..dc8bc7545 100644
--- a/drivers/net/fm10k/fm10k_ethdev.c
+++ b/drivers/net/fm10k/fm10k_ethdev.c
@@ -1210,28 +1210,6 @@ fm10k_dev_queue_release(struct rte_eth_dev *dev)
 	}
 }
 
-static void
-fm10k_dev_close(struct rte_eth_dev *dev)
-{
-	struct fm10k_hw *hw = FM10K_DEV_PRIVATE_TO_HW(dev->data->dev_private);
-
-	PMD_INIT_FUNC_TRACE();
-
-	fm10k_mbx_lock(hw);
-	hw->mac.ops.update_lport_state(hw, hw->mac.dglort_map,
-		MAX_LPORT_NUM, false);
-	fm10k_mbx_unlock(hw);
-
-	/* allow 100ms for device to quiesce */
-	rte_delay_us(FM10K_SWITCH_QUIESCE_US);
-
-	/* Stop mailbox service first */
-	fm10k_close_mbx_service(hw);
-	fm10k_dev_stop(dev);
-	fm10k_dev_queue_release(dev);
-	fm10k_stop_hw(hw);
-}
-
 static int
 fm10k_link_update(struct rte_eth_dev *dev,
 	__rte_unused int wait_to_complete)
@@ -2819,6 +2797,53 @@ fm10k_close_mbx_service(struct fm10k_hw *hw)
 	hw->mbx.ops.disconnect(hw, &hw->mbx);
 }
 
+static void
+fm10k_dev_close(struct rte_eth_dev *dev)
+{
+	struct fm10k_hw *hw = FM10K_DEV_PRIVATE_TO_HW(dev->data->dev_private);
+	struct rte_pci_device *pdev = RTE_ETH_DEV_TO_PCI(dev);
+	struct rte_intr_handle *intr_handle = &pdev->intr_handle;
+
+	PMD_INIT_FUNC_TRACE();
+
+	fm10k_mbx_lock(hw);
+	hw->mac.ops.update_lport_state(hw, hw->mac.dglort_map,
+		MAX_LPORT_NUM, false);
+	fm10k_mbx_unlock(hw);
+
+	/* allow 100ms for device to quiesce */
+	rte_delay_us(FM10K_SWITCH_QUIESCE_US);
+
+	/* Stop mailbox service first */
+	fm10k_close_mbx_service(hw);
+	fm10k_dev_stop(dev);
+	fm10k_dev_queue_release(dev);
+	fm10k_stop_hw(hw);
+
+	dev->dev_ops = NULL;
+	dev->rx_pkt_burst = NULL;
+	dev->tx_pkt_burst = NULL;
+
+	/* disable uio/vfio intr */
+	rte_intr_disable(intr_handle);
+
+	/*PF/VF has different interrupt handling mechanism */
+	if (hw->mac.type == fm10k_mac_pf) {
+		/* disable interrupt */
+		fm10k_dev_disable_intr_pf(dev);
+
+		/* unregister callback func to eal lib */
+		rte_intr_callback_unregister(intr_handle,
+			fm10k_dev_interrupt_handler_pf, (void *)dev);
+	} else {
+		/* disable interrupt */
+		fm10k_dev_disable_intr_vf(dev);
+
+		rte_intr_callback_unregister(intr_handle,
+			fm10k_dev_interrupt_handler_vf, (void *)dev);
+	}
+}
+
 static const struct eth_dev_ops fm10k_eth_dev_ops = {
 	.dev_configure		= fm10k_dev_configure,
 	.dev_start		= fm10k_dev_start,
@@ -3123,6 +3148,11 @@ eth_fm10k_dev_init(struct rte_eth_dev *dev)
 		&dev->data->mac_addrs[0]);
 	}
 
+	/* Pass the information to the rte_eth_dev_close() that it should also
+	 * release the private port resources.
+	 */
+	dev->data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
+
 	/* Reset the hw statistics */
 	diag = fm10k_stats_reset(dev);
 	if (diag != 0) {
@@ -3229,9 +3259,6 @@ eth_fm10k_dev_init(struct rte_eth_dev *dev)
 static int
 eth_fm10k_dev_uninit(struct rte_eth_dev *dev)
 {
-	struct fm10k_hw *hw = FM10K_DEV_PRIVATE_TO_HW(dev->data->dev_private);
-	struct rte_pci_device *pdev = RTE_ETH_DEV_TO_PCI(dev);
-	struct rte_intr_handle *intr_handle = &pdev->intr_handle;
 	PMD_INIT_FUNC_TRACE();
 
 	/* only uninitialize in the primary process */
@@ -3241,29 +3268,6 @@ eth_fm10k_dev_uninit(struct rte_eth_dev *dev)
 	/* safe to close dev here */
 	fm10k_dev_close(dev);
 
-	dev->dev_ops = NULL;
-	dev->rx_pkt_burst = NULL;
-	dev->tx_pkt_burst = NULL;
-
-	/* disable uio/vfio intr */
-	rte_intr_disable(intr_handle);
-
-	/*PF/VF has different interrupt handling mechanism */
-	if (hw->mac.type == fm10k_mac_pf) {
-		/* disable interrupt */
-		fm10k_dev_disable_intr_pf(dev);
-
-		/* unregister callback func to eal lib */
-		rte_intr_callback_unregister(intr_handle,
-			fm10k_dev_interrupt_handler_pf, (void *)dev);
-	} else {
-		/* disable interrupt */
-		fm10k_dev_disable_intr_vf(dev);
-
-		rte_intr_callback_unregister(intr_handle,
-			fm10k_dev_interrupt_handler_vf, (void *)dev);
-	}
-
 	return 0;
 }
 
-- 
2.17.1

[dpdk-dev] [PATCH v7 3/5] net/i40e: release port upon close

[Di ChenxuX]

Set RTE_ETH_DEV_CLOSE_REMOVE upon probe so all the private resources
 for the port can be freed by rte_eth_dev_close().

Signed-off-by: Di ChenxuX <chenxux.di@intel.com>
---
 doc/guides/rel_notes/release_19_11.rst |   4 +
 drivers/net/i40e/i40e_ethdev.c         | 128 ++++++++++++-------------
 drivers/net/i40e/i40e_ethdev_vf.c      |  25 +++--
 3 files changed, 81 insertions(+), 76 deletions(-)

diff --git a/doc/guides/rel_notes/release_19_11.rst b/doc/guides/rel_notes/release_19_11.rst
index 54a4f1001..fc129cf3e 100644
--- a/doc/guides/rel_notes/release_19_11.rst
+++ b/doc/guides/rel_notes/release_19_11.rst
@@ -72,6 +72,10 @@ New Features
 
   Added support for the ``RTE_ETH_DEV_CLOSE_REMOVE`` flag.
 
+* **Updated the Intel i40e driver.**
+
+  Added support for the ``RTE_ETH_DEV_CLOSE_REMOVE`` flag.
+
 
 Removed Items
 -------------
diff --git a/drivers/net/i40e/i40e_ethdev.c b/drivers/net/i40e/i40e_ethdev.c
index 76cd9f510..c145e6407 100644
--- a/drivers/net/i40e/i40e_ethdev.c
+++ b/drivers/net/i40e/i40e_ethdev.c
@@ -1590,6 +1590,11 @@ eth_i40e_dev_init(struct rte_eth_dev *dev, void *init_params __rte_unused)
 	rte_ether_addr_copy((struct rte_ether_addr *)hw->mac.perm_addr,
 					&dev->data->mac_addrs[0]);
 
+	/* Pass the information to the rte_eth_dev_close() that it should also
+	 * release the private port resources.
+	 */
+	dev->data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
+
 	/* Init dcb to sw mode by default */
 	ret = i40e_dcb_init_configure(dev, TRUE);
 	if (ret != I40E_SUCCESS) {
@@ -1759,85 +1764,18 @@ void i40e_flex_payload_reg_set_default(struct i40e_hw *hw)
 static int
 eth_i40e_dev_uninit(struct rte_eth_dev *dev)
 {
-	struct i40e_pf *pf;
-	struct rte_pci_device *pci_dev;
-	struct rte_intr_handle *intr_handle;
 	struct i40e_hw *hw;
-	struct i40e_filter_control_settings settings;
-	struct rte_flow *p_flow;
-	int ret;
-	uint8_t aq_fail = 0;
-	int retries = 0;
 
 	PMD_INIT_FUNC_TRACE();
 
 	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
 		return 0;
 
-	pf = I40E_DEV_PRIVATE_TO_PF(dev->data->dev_private);
 	hw = I40E_DEV_PRIVATE_TO_HW(dev->data->dev_private);
-	pci_dev = RTE_ETH_DEV_TO_PCI(dev);
-	intr_handle = &pci_dev->intr_handle;
-
-	ret = rte_eth_switch_domain_free(pf->switch_domain_id);
-	if (ret)
-		PMD_INIT_LOG(WARNING, "failed to free switch domain: %d", ret);
 
 	if (hw->adapter_closed == 0)
 		i40e_dev_close(dev);
 
-	dev->dev_ops = NULL;
-	dev->rx_pkt_burst = NULL;
-	dev->tx_pkt_burst = NULL;
-
-	/* Clear PXE mode */
-	i40e_clear_pxe_mode(hw);
-
-	/* Unconfigure filter control */
-	memset(&settings, 0, sizeof(settings));
-	ret = i40e_set_filter_control(hw, &settings);
-	if (ret)
-		PMD_INIT_LOG(WARNING, "setup_pf_filter_control failed: %d",
-					ret);
-
-	/* Disable flow control */
-	hw->fc.requested_mode = I40E_FC_NONE;
-	i40e_set_fc(hw, &aq_fail, TRUE);
-
-	/* uninitialize pf host driver */
-	i40e_pf_host_uninit(dev);
-
-	/* disable uio intr before callback unregister */
-	rte_intr_disable(intr_handle);
-
-	/* unregister callback func to eal lib */
-	do {
-		ret = rte_intr_callback_unregister(intr_handle,
-				i40e_dev_interrupt_handler, dev);
-		if (ret >= 0) {
-			break;
-		} else if (ret != -EAGAIN) {
-			PMD_INIT_LOG(ERR,
-				 "intr callback unregister failed: %d",
-				 ret);
-			return ret;
-		}
-		i40e_msec_delay(500);
-	} while (retries++ < 5);
-
-	i40e_rm_ethtype_filter_list(pf);
-	i40e_rm_tunnel_filter_list(pf);
-	i40e_rm_fdir_filter_list(pf);
-
-	/* Remove all flows */
-	while ((p_flow = TAILQ_FIRST(&pf->flow_list))) {
-		TAILQ_REMOVE(&pf->flow_list, p_flow, node);
-		rte_free(p_flow);
-	}
-
-	/* Remove all Traffic Manager configuration */
-	i40e_tm_conf_uninit(dev);
-
 	return 0;
 }
 
@@ -2535,12 +2473,21 @@ i40e_dev_close(struct rte_eth_dev *dev)
 	struct rte_pci_device *pci_dev = RTE_ETH_DEV_TO_PCI(dev);
 	struct rte_intr_handle *intr_handle = &pci_dev->intr_handle;
 	struct i40e_mirror_rule *p_mirror;
+	struct i40e_filter_control_settings settings;
+	struct rte_flow *p_flow;
 	uint32_t reg;
 	int i;
 	int ret;
+	uint8_t aq_fail = 0;
+	int retries = 0;
 
 	PMD_INIT_FUNC_TRACE();
 
+	ret = rte_eth_switch_domain_free(pf->switch_domain_id);
+	if (ret)
+		PMD_INIT_LOG(WARNING, "failed to free switch domain: %d", ret);
+
+
 	i40e_dev_stop(dev);
 
 	/* Remove all mirror rules */
@@ -2605,6 +2552,53 @@ i40e_dev_close(struct rte_eth_dev *dev)
 			(reg | I40E_PFGEN_CTRL_PFSWR_MASK));
 	I40E_WRITE_FLUSH(hw);
 
+	dev->dev_ops = NULL;
+	dev->rx_pkt_burst = NULL;
+	dev->tx_pkt_burst = NULL;
+
+	/* Clear PXE mode */
+	i40e_clear_pxe_mode(hw);
+
+	/* Unconfigure filter control */
+	memset(&settings, 0, sizeof(settings));
+	ret = i40e_set_filter_control(hw, &settings);
+	if (ret)
+		PMD_INIT_LOG(WARNING, "setup_pf_filter_control failed: %d",
+					ret);
+
+	/* Disable flow control */
+	hw->fc.requested_mode = I40E_FC_NONE;
+	i40e_set_fc(hw, &aq_fail, TRUE);
+
+	/* uninitialize pf host driver */
+	i40e_pf_host_uninit(dev);
+
+	do {
+		ret = rte_intr_callback_unregister(intr_handle,
+				i40e_dev_interrupt_handler, dev);
+		if (ret >= 0) {
+			break;
+		} else if (ret != -EAGAIN) {
+			PMD_INIT_LOG(ERR,
+				 "intr callback unregister failed: %d",
+				 ret);
+		}
+		i40e_msec_delay(500);
+	} while (retries++ < 5);
+
+	i40e_rm_ethtype_filter_list(pf);
+	i40e_rm_tunnel_filter_list(pf);
+	i40e_rm_fdir_filter_list(pf);
+
+	/* Remove all flows */
+	while ((p_flow = TAILQ_FIRST(&pf->flow_list))) {
+		TAILQ_REMOVE(&pf->flow_list, p_flow, node);
+		rte_free(p_flow);
+	}
+
+	/* Remove all Traffic Manager configuration */
+	i40e_tm_conf_uninit(dev);
+
 	hw->adapter_closed = 1;
 }
 
diff --git a/drivers/net/i40e/i40e_ethdev_vf.c b/drivers/net/i40e/i40e_ethdev_vf.c
index ddbaaea04..7ffb111b8 100644
--- a/drivers/net/i40e/i40e_ethdev_vf.c
+++ b/drivers/net/i40e/i40e_ethdev_vf.c
@@ -1312,17 +1312,12 @@ i40evf_init_vf(struct rte_eth_dev *dev)
 static int
 i40evf_uninit_vf(struct rte_eth_dev *dev)
 {
-	struct i40e_vf *vf = I40EVF_DEV_PRIVATE_TO_VF(dev->data->dev_private);
 	struct i40e_hw *hw = I40E_DEV_PRIVATE_TO_HW(dev->data->dev_private);
 
 	PMD_INIT_FUNC_TRACE();
 
 	if (hw->adapter_closed == 0)
 		i40evf_dev_close(dev);
-	rte_free(vf->vf_res);
-	vf->vf_res = NULL;
-	rte_free(vf->aq_resp);
-	vf->aq_resp = NULL;
 
 	return 0;
 }
@@ -1500,6 +1495,11 @@ i40evf_dev_init(struct rte_eth_dev *eth_dev)
 	hw->adapter_stopped = 0;
 	hw->adapter_closed = 0;
 
+	/* Pass the information to the rte_eth_dev_close() that it should also
+	 * release the private port resources.
+	 */
+	eth_dev->data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
+
 	if(i40evf_init_vf(eth_dev) != 0) {
 		PMD_INIT_LOG(ERR, "Init vf failed");
 		return -1;
@@ -1536,10 +1536,6 @@ i40evf_dev_uninit(struct rte_eth_dev *eth_dev)
 	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
 		return -EPERM;
 
-	eth_dev->dev_ops = NULL;
-	eth_dev->rx_pkt_burst = NULL;
-	eth_dev->tx_pkt_burst = NULL;
-
 	if (i40evf_uninit_vf(eth_dev) != 0) {
 		PMD_INIT_LOG(ERR, "i40evf_uninit_vf failed");
 		return -1;
@@ -2344,6 +2340,7 @@ static void
 i40evf_dev_close(struct rte_eth_dev *dev)
 {
 	struct i40e_hw *hw = I40E_DEV_PRIVATE_TO_HW(dev->data->dev_private);
+	struct i40e_vf *vf = I40EVF_DEV_PRIVATE_TO_VF(dev->data->dev_private);
 
 	i40evf_dev_stop(dev);
 	i40e_dev_free_queues(dev);
@@ -2359,6 +2356,16 @@ i40evf_dev_close(struct rte_eth_dev *dev)
 	i40evf_reset_vf(dev);
 	i40e_shutdown_adminq(hw);
 	i40evf_disable_irq0(hw);
+
+	dev->dev_ops = NULL;
+	dev->rx_pkt_burst = NULL;
+	dev->tx_pkt_burst = NULL;
+
+	rte_free(vf->vf_res);
+	vf->vf_res = NULL;
+	rte_free(vf->aq_resp);
+	vf->aq_resp = NULL;
+
 	hw->adapter_closed = 1;
 }
 
-- 
2.17.1

[dpdk-dev] [PATCH v7 4/5] net/ice: release port upon close

[Di ChenxuX]

Set RTE_ETH_DEV_CLOSE_REMOVE upon probe so all the private resources
 for the port can be freed by rte_eth_dev_close().

Signed-off-by: Di ChenxuX <chenxux.di@intel.com>
---
 doc/guides/rel_notes/release_19_11.rst |  1 +
 drivers/net/ice/ice_ethdev.c           | 30 ++++++++++++--------------
 2 files changed, 15 insertions(+), 16 deletions(-)

diff --git a/doc/guides/rel_notes/release_19_11.rst b/doc/guides/rel_notes/release_19_11.rst
index fc129cf3e..ac8f3d0e5 100644
--- a/doc/guides/rel_notes/release_19_11.rst
+++ b/doc/guides/rel_notes/release_19_11.rst
@@ -63,6 +63,7 @@ New Features
   * Added support for device-specific DDP package loading.
   * Added support for handling Receive Flex Descriptor.
   * Added support for protocol extraction on per Rx queue.
+  * Added support for the ``RTE_ETH_DEV_CLOSE_REMOVE`` flag.
 
 * **Updated the Intel e1000 driver.**
 
diff --git a/drivers/net/ice/ice_ethdev.c b/drivers/net/ice/ice_ethdev.c
index c126d962c..dfccef65d 100644
--- a/drivers/net/ice/ice_ethdev.c
+++ b/drivers/net/ice/ice_ethdev.c
@@ -1924,6 +1924,11 @@ ice_dev_init(struct rte_eth_dev *dev)
 		goto err_init_mac;
 	}
 
+	/* Pass the information to the rte_eth_dev_close() that it should also
+	 * release the private port resources.
+	 */
+	dev->data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
+
 	ret = ice_res_pool_init(&pf->msix_pool, 1,
 				hw->func_caps.common_cap.num_msix_vectors - 1);
 	if (ret) {
@@ -2090,6 +2095,9 @@ ice_dev_close(struct rte_eth_dev *dev)
 {
 	struct ice_pf *pf = ICE_DEV_PRIVATE_TO_PF(dev->data->dev_private);
 	struct ice_hw *hw = ICE_DEV_PRIVATE_TO_HW(dev->data->dev_private);
+	struct rte_pci_device *pci_dev = RTE_ETH_DEV_TO_PCI(dev);
+	struct rte_intr_handle *intr_handle = &pci_dev->intr_handle;
+	struct rte_flow *p_flow;
 
 	/* Since stop will make link down, then the link event will be
 	 * triggered, disable the irq firstly to avoid the port_infoe etc
@@ -2109,22 +2117,6 @@ ice_dev_close(struct rte_eth_dev *dev)
 	rte_free(hw->port_info);
 	hw->port_info = NULL;
 	ice_shutdown_all_ctrlq(hw);
-	rte_free(pf->proto_xtr);
-	pf->proto_xtr = NULL;
-}
-
-static int
-ice_dev_uninit(struct rte_eth_dev *dev)
-{
-	struct rte_pci_device *pci_dev = RTE_ETH_DEV_TO_PCI(dev);
-	struct rte_intr_handle *intr_handle = &pci_dev->intr_handle;
-	struct ice_pf *pf = ICE_DEV_PRIVATE_TO_PF(dev->data->dev_private);
-	struct rte_flow *p_flow;
-
-	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
-		return 0;
-
-	ice_dev_close(dev);
 
 	dev->dev_ops = NULL;
 	dev->rx_pkt_burst = NULL;
@@ -2146,6 +2138,12 @@ ice_dev_uninit(struct rte_eth_dev *dev)
 		ice_free_switch_filter_rule(p_flow->rule);
 		rte_free(p_flow);
 	}
+}
+
+static int
+ice_dev_uninit(struct rte_eth_dev *dev)
+{
+	ice_dev_close(dev);
 
 	return 0;
 }
-- 
2.17.1

[dpdk-dev] [PATCH v7 5/5] net/ixgbe: release port upon close

[Di ChenxuX]

Set RTE_ETH_DEV_CLOSE_REMOVE upon probe so all the private resources
 for the port can be freed by rte_eth_dev_close().

Signed-off-by: Di ChenxuX <chenxux.di@intel.com>
---
 doc/guides/rel_notes/release_19_11.rst |   4 +
 drivers/net/ixgbe/ixgbe_ethdev.c       | 167 +++++++++++++------------
 2 files changed, 89 insertions(+), 82 deletions(-)

diff --git a/doc/guides/rel_notes/release_19_11.rst b/doc/guides/rel_notes/release_19_11.rst
index ac8f3d0e5..4dc237006 100644
--- a/doc/guides/rel_notes/release_19_11.rst
+++ b/doc/guides/rel_notes/release_19_11.rst
@@ -77,6 +77,10 @@ New Features
 
   Added support for the ``RTE_ETH_DEV_CLOSE_REMOVE`` flag.
 
+* **Updated the Intel ixgbe driver.**
+
+  Added support for the ``RTE_ETH_DEV_CLOSE_REMOVE`` flag.
+
 
 Removed Items
 -------------
diff --git a/drivers/net/ixgbe/ixgbe_ethdev.c b/drivers/net/ixgbe/ixgbe_ethdev.c
index 0aa039a71..7bec95fe5 100644
--- a/drivers/net/ixgbe/ixgbe_ethdev.c
+++ b/drivers/net/ixgbe/ixgbe_ethdev.c
@@ -1245,6 +1245,11 @@ eth_ixgbe_dev_init(struct rte_eth_dev *eth_dev, void *init_params __rte_unused)
 		return -ENOMEM;
 	}
 
+	/* Pass the information to the rte_eth_dev_close() that it should also
+	 * release the private port resources.
+	 */
+	eth_dev->data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
+
 	/* initialize the vfta */
 	memset(shadow_vfta, 0, sizeof(*shadow_vfta));
 
@@ -1311,73 +1316,12 @@ eth_ixgbe_dev_init(struct rte_eth_dev *eth_dev, void *init_params __rte_unused)
 static int
 eth_ixgbe_dev_uninit(struct rte_eth_dev *eth_dev)
 {
-	struct rte_pci_device *pci_dev = RTE_ETH_DEV_TO_PCI(eth_dev);
-	struct rte_intr_handle *intr_handle = &pci_dev->intr_handle;
-	struct ixgbe_hw *hw;
-	int retries = 0;
-	int ret;
-
 	PMD_INIT_FUNC_TRACE();
 
 	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
 		return 0;
 
-	hw = IXGBE_DEV_PRIVATE_TO_HW(eth_dev->data->dev_private);
-
-	if (hw->adapter_stopped == 0)
-		ixgbe_dev_close(eth_dev);
-
-	eth_dev->dev_ops = NULL;
-	eth_dev->rx_pkt_burst = NULL;
-	eth_dev->tx_pkt_burst = NULL;
-
-	/* Unlock any pending hardware semaphore */
-	ixgbe_swfw_lock_reset(hw);
-
-	/* disable uio intr before callback unregister */
-	rte_intr_disable(intr_handle);
-
-	do {
-		ret = rte_intr_callback_unregister(intr_handle,
-				ixgbe_dev_interrupt_handler, eth_dev);
-		if (ret >= 0) {
-			break;
-		} else if (ret != -EAGAIN) {
-			PMD_INIT_LOG(ERR,
-				"intr callback unregister failed: %d",
-				ret);
-			return ret;
-		}
-		rte_delay_ms(100);
-	} while (retries++ < (10 + IXGBE_LINK_UP_TIME));
-
-	/* cancel the delay handler before remove dev */
-	rte_eal_alarm_cancel(ixgbe_dev_interrupt_delayed_handler, eth_dev);
-
-	/* cancel the link handler before remove dev */
-	rte_eal_alarm_cancel(ixgbe_dev_setup_link_alarm_handler, eth_dev);
-
-	/* uninitialize PF if max_vfs not zero */
-	ixgbe_pf_host_uninit(eth_dev);
-
-	/* remove all the fdir filters & hash */
-	ixgbe_fdir_filter_uninit(eth_dev);
-
-	/* remove all the L2 tunnel filters & hash */
-	ixgbe_l2_tn_filter_uninit(eth_dev);
-
-	/* Remove all ntuple filters of the device */
-	ixgbe_ntuple_filter_uninit(eth_dev);
-
-	/* clear all the filters list */
-	ixgbe_filterlist_flush();
-
-	/* Remove all Traffic Manager configuration */
-	ixgbe_tm_conf_uninit(eth_dev);
-
-#ifdef RTE_LIBRTE_SECURITY
-	rte_free(eth_dev->security_ctx);
-#endif
+	ixgbe_dev_close(eth_dev);
 
 	return 0;
 }
@@ -1711,6 +1655,11 @@ eth_ixgbevf_dev_init(struct rte_eth_dev *eth_dev)
 		return -ENOMEM;
 	}
 
+	/* Pass the information to the rte_eth_dev_close() that it should also
+	 * release the private port resources.
+	 */
+	eth_dev->data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
+
 	/* Generate a random MAC address, if none was assigned by PF. */
 	if (rte_is_zero_ether_addr(perm_addr)) {
 		generate_random_mac_addr(perm_addr);
@@ -1762,30 +1711,12 @@ eth_ixgbevf_dev_init(struct rte_eth_dev *eth_dev)
 static int
 eth_ixgbevf_dev_uninit(struct rte_eth_dev *eth_dev)
 {
-	struct rte_pci_device *pci_dev = RTE_ETH_DEV_TO_PCI(eth_dev);
-	struct rte_intr_handle *intr_handle = &pci_dev->intr_handle;
-	struct ixgbe_hw *hw;
-
 	PMD_INIT_FUNC_TRACE();
 
 	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
 		return 0;
 
-	hw = IXGBE_DEV_PRIVATE_TO_HW(eth_dev->data->dev_private);
-
-	if (hw->adapter_stopped == 0)
-		ixgbevf_dev_close(eth_dev);
-
-	eth_dev->dev_ops = NULL;
-	eth_dev->rx_pkt_burst = NULL;
-	eth_dev->tx_pkt_burst = NULL;
-
-	/* Disable the interrupts for VF */
-	ixgbevf_intr_disable(eth_dev);
-
-	rte_intr_disable(intr_handle);
-	rte_intr_callback_unregister(intr_handle,
-				     ixgbevf_dev_interrupt_handler, eth_dev);
+	ixgbevf_dev_close(eth_dev);
 
 	return 0;
 }
@@ -2879,6 +2810,9 @@ ixgbe_dev_stop(struct rte_eth_dev *dev)
 	struct ixgbe_tm_conf *tm_conf =
 		IXGBE_DEV_PRIVATE_TO_TM_CONF(dev->data->dev_private);
 
+	if (hw->adapter_stopped)
+		return;
+
 	PMD_INIT_FUNC_TRACE();
 
 	rte_eal_alarm_cancel(ixgbe_dev_setup_link_alarm_handler, dev);
@@ -2931,6 +2865,8 @@ ixgbe_dev_stop(struct rte_eth_dev *dev)
 	tm_conf->committed = false;
 
 	adapter->rss_reta_updated = 0;
+
+	hw->adapter_stopped = true;
 }
 
 /*
@@ -3001,13 +2937,16 @@ ixgbe_dev_close(struct rte_eth_dev *dev)
 {
 	struct ixgbe_hw *hw =
 		IXGBE_DEV_PRIVATE_TO_HW(dev->data->dev_private);
+	struct rte_pci_device *pci_dev = RTE_ETH_DEV_TO_PCI(dev);
+	struct rte_intr_handle *intr_handle = &pci_dev->intr_handle;
+	int retries = 0;
+	int ret;
 
 	PMD_INIT_FUNC_TRACE();
 
 	ixgbe_pf_reset_hw(hw);
 
 	ixgbe_dev_stop(dev);
-	hw->adapter_stopped = 1;
 
 	ixgbe_dev_free_queues(dev);
 
@@ -3015,6 +2954,55 @@ ixgbe_dev_close(struct rte_eth_dev *dev)
 
 	/* reprogram the RAR[0] in case user changed it. */
 	ixgbe_set_rar(hw, 0, hw->mac.addr, 0, IXGBE_RAH_AV);
+
+	dev->dev_ops = NULL;
+	dev->rx_pkt_burst = NULL;
+	dev->tx_pkt_burst = NULL;
+
+	/* Unlock any pending hardware semaphore */
+	ixgbe_swfw_lock_reset(hw);
+
+	/* disable uio intr before callback unregister */
+	rte_intr_disable(intr_handle);
+
+	do {
+		ret = rte_intr_callback_unregister(intr_handle,
+				ixgbe_dev_interrupt_handler, dev);
+		if (ret >= 0) {
+			break;
+		} else if (ret != -EAGAIN) {
+			PMD_INIT_LOG(ERR,
+				"intr callback unregister failed: %d",
+				ret);
+		}
+		rte_delay_ms(100);
+	} while (retries++ < (10 + IXGBE_LINK_UP_TIME));
+
+	/* cancel the delay handler before remove dev */
+	rte_eal_alarm_cancel(ixgbe_dev_interrupt_delayed_handler, dev);
+
+	/* uninitialize PF if max_vfs not zero */
+	ixgbe_pf_host_uninit(dev);
+
+	/* remove all the fdir filters & hash */
+	ixgbe_fdir_filter_uninit(dev);
+
+	/* remove all the L2 tunnel filters & hash */
+	ixgbe_l2_tn_filter_uninit(dev);
+
+	/* Remove all ntuple filters of the device */
+	ixgbe_ntuple_filter_uninit(dev);
+
+	/* clear all the filters list */
+	ixgbe_filterlist_flush();
+
+	/* Remove all Traffic Manager configuration */
+	ixgbe_tm_conf_uninit(dev);
+
+#ifdef RTE_LIBRTE_SECURITY
+	rte_free(dev->security_ctx);
+#endif
+
 }
 
 /*
@@ -5257,6 +5245,8 @@ ixgbevf_dev_start(struct rte_eth_dev *dev)
 	 */
 	ixgbevf_dev_link_update(dev, 0);
 
+	hw->adapter_stopped = false;
+
 	return 0;
 }
 
@@ -5268,6 +5258,9 @@ ixgbevf_dev_stop(struct rte_eth_dev *dev)
 	struct rte_pci_device *pci_dev = RTE_ETH_DEV_TO_PCI(dev);
 	struct rte_intr_handle *intr_handle = &pci_dev->intr_handle;
 
+	if (hw->adapter_stopped)
+		return;
+
 	PMD_INIT_FUNC_TRACE();
 
 	rte_eal_alarm_cancel(ixgbe_dev_setup_link_alarm_handler, dev);
@@ -5302,6 +5295,8 @@ static void
 ixgbevf_dev_close(struct rte_eth_dev *dev)
 {
 	struct ixgbe_hw *hw = IXGBE_DEV_PRIVATE_TO_HW(dev->data->dev_private);
+	struct rte_pci_device *pci_dev = RTE_ETH_DEV_TO_PCI(dev);
+	struct rte_intr_handle *intr_handle = &pci_dev->intr_handle;
 
 	PMD_INIT_FUNC_TRACE();
 
@@ -5317,6 +5312,14 @@ ixgbevf_dev_close(struct rte_eth_dev *dev)
 	 * after stop, close and detach of the VF
 	 **/
 	ixgbevf_remove_mac_addr(dev, 0);
+
+	dev->dev_ops = NULL;
+	dev->rx_pkt_burst = NULL;
+	dev->tx_pkt_burst = NULL;
+
+	rte_intr_disable(intr_handle);
+	rte_intr_callback_unregister(intr_handle,
+				     ixgbevf_dev_interrupt_handler, dev);
 }
 
 /*
-- 
2.17.1

Re: [dpdk-dev] [PATCH v7 0/5] drivers/net: release port upon close

[Ye Xiaolong]

On 09/27, Di ChenxuX wrote:
>Set RTE_ETH_DEV_CLOSE_REMOVE upon probe so all the private resources
> for the port can be freed by rte_eth_dev_close().
>This patch cover all the intel drivers.
>
>---
>V7 changes:
>  Modified code on the last commit.
>V6 changes:
>  Fixed check error.
>V5 changes:
>  Added code in em_ethdev.c.
>  Separated the release note, each patch has its own note. 
>V4 changes:
>  Removed adaper stopped flag in function ice_dev_init.
>V3 changes:
>  Removed adapter closed flag in driver e1000, fm10k, ice, ixgbe.
>V2 changes:
>  Separated to 5 patches, each patch only change one driver.
>  Added resource release in dev_close.
>
>Acked-by: Qiming Yang <qiming.yang@intel.com>
>Series Reviewed-by: Xiaolong Ye <xiaolong.ye@intel.com>
>
>Di ChenxuX (5):
>  net/e1000: release port upon close
>  net/fm10k: release port upon close
>  net/i40e: release port upon close
>  net/ice: release port upon close
>  net/ixgbe: release port upon close
>
> doc/guides/rel_notes/release_19_11.rst |  18 +++
> drivers/net/e1000/em_ethdev.c          |  33 ++---
> drivers/net/e1000/igb_ethdev.c         | 144 ++++++++++-----------
> drivers/net/fm10k/fm10k_ethdev.c       | 100 ++++++++-------
> drivers/net/i40e/i40e_ethdev.c         | 128 +++++++++----------
> drivers/net/i40e/i40e_ethdev_vf.c      |  25 ++--
> drivers/net/ice/ice_ethdev.c           |  30 +++--
> drivers/net/ixgbe/ixgbe_ethdev.c       | 167 +++++++++++++------------
> 8 files changed, 335 insertions(+), 310 deletions(-)
>
>-- 
>2.17.1
>

Reviewed-by: Xiaolong Ye <xiaolong.ye@intel.com>

Applied to dpdk-next-net-intel.