[dpdk-dev] [PATCH 00/20] cleanup ethdev close operation

[dpdk-dev] [PATCH 00/20] cleanup ethdev close operation

[Thomas Monjalon]

This is the end of a process started two years ago,
to have a close which reliably releases an ethdev port
without the need of removing the device (which can have more ports).

Unfortunately, some drivers might be broken because did not follow
the migration recommendations. We cannot wait more,
this should be merged before the 20.11-rc1 release.


Steve Yang (1):
  net/iavf: release port upon close

Thomas Monjalon (18):
  ethdev: reset device and interrupt pointers on release
  ethdev: allow drivers to return error on close
  net/af_packet: release port upon close
  net/atlantic: release port upon close
  net/axgbe: release port upon close
  net/bonding: release port upon close
  net/failsafe: release port upon close
  net/mlx4: release port upon close
  net/null: release port upon close
  net/octeontx: release port upon close
  net/pcap: release port upon close
  net/ring: release port upon close
  net/softnic: release port upon close
  ethdev: remove old close behaviour
  drivers/net: accept removing device without any port
  drivers/net: remove redundant MAC addresses freeing
  app/testpmd: reset port status on close notification
  app/testpmd: align behaviour of multi-port detach

Yunjian Wang (1):
  net/tap: release port upon close

 MAINTAINERS                               | 22 ++++----
 app/test-pmd/config.c                     |  7 ++-
 app/test-pmd/testpmd.c                    | 56 +++++++------------
 app/test/virtual_pmd.c                    |  6 ++-
 doc/guides/rel_notes/deprecation.rst      |  6 ---
 drivers/net/af_packet/rte_eth_af_packet.c | 58 ++++++++++----------
 drivers/net/af_xdp/rte_eth_af_xdp.c       |  6 +--
 drivers/net/ark/ark_ethdev.c              | 11 ++--
 drivers/net/atlantic/atl_ethdev.c         | 66 +++++++++--------------
 drivers/net/avp/avp_ethdev.c              |  7 ++-
 drivers/net/axgbe/axgbe_ethdev.c          | 14 ++---
 drivers/net/bnx2x/bnx2x_ethdev.c          |  4 +-
 drivers/net/bnxt/bnxt_ethdev.c            |  9 ++--
 drivers/net/bnxt/bnxt_reps.c              |  7 +--
 drivers/net/bnxt/bnxt_reps.h              |  2 +-
 drivers/net/bonding/eth_bond_private.h    |  2 +-
 drivers/net/bonding/rte_eth_bond_pmd.c    | 37 +++++++------
 drivers/net/cxgbe/cxgbe_ethdev.c          |  6 ++-
 drivers/net/cxgbe/cxgbe_pfvf.h            |  2 +-
 drivers/net/dpaa/dpaa_ethdev.c            |  4 +-
 drivers/net/dpaa2/dpaa2_ethdev.c          |  6 ++-
 drivers/net/e1000/em_ethdev.c             | 11 ++--
 drivers/net/e1000/igb_ethdev.c            | 22 +++-----
 drivers/net/ena/ena_ethdev.c              | 12 ++---
 drivers/net/enetc/enetc_ethdev.c          |  4 +-
 drivers/net/enic/enic_ethdev.c            |  6 +--
 drivers/net/failsafe/failsafe.c           | 24 ++-------
 drivers/net/failsafe/failsafe_ops.c       | 60 +++++++++++++--------
 drivers/net/failsafe/failsafe_private.h   |  1 +
 drivers/net/fm10k/fm10k_ethdev.c          |  9 ++--
 drivers/net/hinic/hinic_pmd_ethdev.c      | 15 ++----
 drivers/net/hns3/hns3_ethdev.c            | 11 ++--
 drivers/net/hns3/hns3_ethdev_vf.c         | 11 ++--
 drivers/net/i40e/i40e_ethdev.c            | 10 ++--
 drivers/net/i40e/i40e_ethdev_vf.c         | 10 ++--
 drivers/net/iavf/iavf_ethdev.c            | 44 ++++++++-------
 drivers/net/ice/ice_dcf_ethdev.c          |  8 +--
 drivers/net/ice/ice_ethdev.c              | 14 ++---
 drivers/net/igc/igc_ethdev.c              | 11 ++--
 drivers/net/ionic/ionic_ethdev.c          | 10 ++--
 drivers/net/ipn3ke/ipn3ke_ethdev.c        |  6 +--
 drivers/net/ipn3ke/ipn3ke_representor.c   |  4 +-
 drivers/net/ixgbe/ixgbe_ethdev.c          | 21 +++-----
 drivers/net/kni/rte_eth_kni.c             | 22 ++++----
 drivers/net/liquidio/lio_ethdev.c         |  4 +-
 drivers/net/memif/rte_eth_memif.c         |  7 ++-
 drivers/net/mlx4/mlx4.c                   |  5 +-
 drivers/net/mlx5/linux/mlx5_os.c          |  2 -
 drivers/net/mlx5/mlx5.c                   |  9 ++--
 drivers/net/mlx5/mlx5.h                   |  2 +-
 drivers/net/mvneta/mvneta_ethdev.c        |  7 ++-
 drivers/net/mvpp2/mrvl_ethdev.c           |  7 ++-
 drivers/net/netvsc/hn_ethdev.c            |  9 ++--
 drivers/net/nfb/nfb_ethdev.c              |  8 +--
 drivers/net/nfp/nfp_net.c                 |  8 ++-
 drivers/net/null/rte_eth_null.c           | 23 +++++---
 drivers/net/octeontx/octeontx_ethdev.c    | 14 ++---
 drivers/net/octeontx2/otx2_ethdev.c       |  8 +--
 drivers/net/pcap/rte_eth_pcap.c           | 31 ++++++-----
 drivers/net/pfe/pfe_ethdev.c              | 14 ++---
 drivers/net/qede/qede_ethdev.c            |  4 +-
 drivers/net/ring/rte_eth_ring.c           | 50 ++++++++++-------
 drivers/net/sfc/sfc_ethdev.c              |  8 +--
 drivers/net/softnic/rte_eth_softnic.c     | 64 +++++++++++-----------
 drivers/net/szedata2/rte_eth_szedata2.c   | 20 +++----
 drivers/net/tap/rte_eth_tap.c             | 48 ++++++++++-------
 drivers/net/thunderx/nicvf_ethdev.c       |  4 +-
 drivers/net/vhost/rte_eth_vhost.c         |  8 +--
 drivers/net/virtio/virtio_ethdev.c        | 11 ++--
 drivers/net/vmxnet3/vmxnet3_ethdev.c      |  9 ++--
 lib/librte_ethdev/rte_ethdev.c            | 19 ++-----
 lib/librte_ethdev/rte_ethdev.h            |  8 +--
 lib/librte_ethdev/rte_ethdev_core.h       |  2 +-
 lib/librte_ethdev/rte_ethdev_pci.h        | 14 +----
 74 files changed, 503 insertions(+), 608 deletions(-)

-- 
2.28.0

[dpdk-dev] [PATCH 01/20] ethdev: reset device and interrupt pointers on release

[Thomas Monjalon]

The pointers .device and .intr_handle were already reset by the helper
rte_eth_dev_pci_generic_remove().
It is now made part of rte_eth_dev_release_port().

It makes rte_eth_dev_pci_release() meaningless,
so it is replaced with a call to rte_eth_dev_release_port().

Signed-off-by: Thomas Monjalon <thomas@monjalon.net>
---
 drivers/net/ark/ark_ethdev.c            |  2 +-
 drivers/net/octeontx2/otx2_ethdev.c     |  2 +-
 drivers/net/szedata2/rte_eth_szedata2.c |  6 +++---
 lib/librte_ethdev/rte_ethdev.c          |  2 ++
 lib/librte_ethdev/rte_ethdev_pci.h      | 14 ++------------
 5 files changed, 9 insertions(+), 17 deletions(-)

diff --git a/drivers/net/ark/ark_ethdev.c b/drivers/net/ark/ark_ethdev.c
index b32ccd8677..0321ec1264 100644
--- a/drivers/net/ark/ark_ethdev.c
+++ b/drivers/net/ark/ark_ethdev.c
@@ -105,7 +105,7 @@ eth_ark_pci_probe(struct rte_pci_driver *pci_drv __rte_unused,
 
 	ret = eth_ark_dev_init(eth_dev);
 	if (ret)
-		rte_eth_dev_pci_release(eth_dev);
+		rte_eth_dev_release_port(eth_dev);
 
 	return ret;
 }
diff --git a/drivers/net/octeontx2/otx2_ethdev.c b/drivers/net/octeontx2/otx2_ethdev.c
index 33b72bd4db..7425ee55be 100644
--- a/drivers/net/octeontx2/otx2_ethdev.c
+++ b/drivers/net/octeontx2/otx2_ethdev.c
@@ -2668,7 +2668,7 @@ nix_remove(struct rte_pci_device *pci_dev)
 		if (rc)
 			return rc;
 
-		rte_eth_dev_pci_release(eth_dev);
+		rte_eth_dev_release_port(eth_dev);
 	}
 
 	/* Nothing to be done for secondary processes */
diff --git a/drivers/net/szedata2/rte_eth_szedata2.c b/drivers/net/szedata2/rte_eth_szedata2.c
index 30c888cd96..a17c53577c 100644
--- a/drivers/net/szedata2/rte_eth_szedata2.c
+++ b/drivers/net/szedata2/rte_eth_szedata2.c
@@ -1802,7 +1802,7 @@ szedata2_eth_dev_release_interval(struct rte_eth_dev **eth_devs,
 
 	for (i = from; i < to; i++) {
 		rte_szedata2_eth_dev_uninit(eth_devs[i]);
-		rte_eth_dev_pci_release(eth_devs[i]);
+		rte_eth_dev_release_port(eth_devs[i]);
 	}
 }
 
@@ -1853,7 +1853,7 @@ static int szedata2_eth_pci_probe(struct rte_pci_driver *pci_drv __rte_unused,
 		if (ret != 0) {
 			PMD_INIT_LOG(ERR, "Failed to init eth_dev for port %u",
 					i);
-			rte_eth_dev_pci_release(eth_devs[i]);
+			rte_eth_dev_release_port(eth_devs[i]);
 			szedata2_eth_dev_release_interval(eth_devs, 0, i);
 			rte_free(list_entry);
 			return ret;
@@ -1922,7 +1922,7 @@ static int szedata2_eth_pci_remove(struct rte_pci_device *pci_dev)
 			retval = retval ? retval : ret;
 		}
 
-		rte_eth_dev_pci_release(eth_dev);
+		rte_eth_dev_release_port(eth_dev);
 	}
 
 	return retval;
diff --git a/lib/librte_ethdev/rte_ethdev.c b/lib/librte_ethdev/rte_ethdev.c
index 7858ad5f11..3617f39ca0 100644
--- a/lib/librte_ethdev/rte_ethdev.c
+++ b/lib/librte_ethdev/rte_ethdev.c
@@ -555,6 +555,8 @@ rte_eth_dev_release_port(struct rte_eth_dev *eth_dev)
 	rte_spinlock_lock(&rte_eth_dev_shared_data->ownership_lock);
 
 	eth_dev->state = RTE_ETH_DEV_UNUSED;
+	eth_dev->device = NULL;
+	eth_dev->intr_handle = NULL;
 
 	if (rte_eal_process_type() == RTE_PROC_PRIMARY) {
 		rte_free(eth_dev->data->rx_queues);
diff --git a/lib/librte_ethdev/rte_ethdev_pci.h b/lib/librte_ethdev/rte_ethdev_pci.h
index a999602fdd..114da807ff 100644
--- a/lib/librte_ethdev/rte_ethdev_pci.h
+++ b/lib/librte_ethdev/rte_ethdev_pci.h
@@ -134,16 +134,6 @@ rte_eth_dev_pci_allocate(struct rte_pci_device *dev, size_t private_data_size)
 	return eth_dev;
 }
 
-static inline void
-rte_eth_dev_pci_release(struct rte_eth_dev *eth_dev)
-{
-	eth_dev->device = NULL;
-	eth_dev->intr_handle = NULL;
-
-	/* free ether device */
-	rte_eth_dev_release_port(eth_dev);
-}
-
 typedef int (*eth_dev_pci_callback_t)(struct rte_eth_dev *eth_dev);
 
 /**
@@ -165,7 +155,7 @@ rte_eth_dev_pci_generic_probe(struct rte_pci_device *pci_dev,
 	RTE_FUNC_PTR_OR_ERR_RET(*dev_init, -EINVAL);
 	ret = dev_init(eth_dev);
 	if (ret)
-		rte_eth_dev_pci_release(eth_dev);
+		rte_eth_dev_release_port(eth_dev);
 	else
 		rte_eth_dev_probing_finish(eth_dev);
 
@@ -194,7 +184,7 @@ rte_eth_dev_pci_generic_remove(struct rte_pci_device *pci_dev,
 			return ret;
 	}
 
-	rte_eth_dev_pci_release(eth_dev);
+	rte_eth_dev_release_port(eth_dev);
 	return 0;
 }
 
-- 
2.28.0

Re: [dpdk-dev] [PATCH 01/20] ethdev: reset device and interrupt pointers on release

[Ferruh Yigit]

On 9/13/2020 11:06 PM, Thomas Monjalon wrote:
> The pointers .device and .intr_handle were already reset by the helper
> rte_eth_dev_pci_generic_remove().
> It is now made part of rte_eth_dev_release_port().
> 
> It makes rte_eth_dev_pci_release() meaningless,
> so it is replaced with a call to rte_eth_dev_release_port().
> 
> Signed-off-by: Thomas Monjalon <thomas@monjalon.net>

Reviewed-by: Ferruh Yigit <ferruh.yigit@intel.com>	

[dpdk-dev] [PATCH 02/20] ethdev: allow drivers to return error on close

[Thomas Monjalon]

The device operation .dev_close was returning void.
This driver interface is changed to return an int.

Note that the API rte_eth_dev_close() is still returning void,
although a deprecation notice is pending to change it as well.

Signed-off-by: Thomas Monjalon <thomas@monjalon.net>
---
 app/test/virtual_pmd.c                    |  6 ++++--
 drivers/net/af_packet/rte_eth_af_packet.c |  3 ++-
 drivers/net/af_xdp/rte_eth_af_xdp.c       |  4 +++-
 drivers/net/ark/ark_ethdev.c              |  6 ++++--
 drivers/net/atlantic/atl_ethdev.c         |  6 ++++--
 drivers/net/avp/avp_ethdev.c              |  5 +++--
 drivers/net/axgbe/axgbe_ethdev.c          |  5 +++--
 drivers/net/bnx2x/bnx2x_ethdev.c          |  4 +++-
 drivers/net/bnxt/bnxt_ethdev.c            |  4 +++-
 drivers/net/bnxt/bnxt_reps.c              |  3 ++-
 drivers/net/bnxt/bnxt_reps.h              |  2 +-
 drivers/net/bonding/eth_bond_private.h    |  2 +-
 drivers/net/bonding/rte_eth_bond_pmd.c    |  4 +++-
 drivers/net/cxgbe/cxgbe_ethdev.c          |  6 ++++--
 drivers/net/cxgbe/cxgbe_pfvf.h            |  2 +-
 drivers/net/dpaa/dpaa_ethdev.c            |  4 +++-
 drivers/net/dpaa2/dpaa2_ethdev.c          |  6 ++++--
 drivers/net/e1000/em_ethdev.c             |  6 ++++--
 drivers/net/e1000/igb_ethdev.c            | 12 ++++++++----
 drivers/net/ena/ena_ethdev.c              |  6 ++++--
 drivers/net/enetc/enetc_ethdev.c          |  4 +++-
 drivers/net/enic/enic_ethdev.c            |  4 +++-
 drivers/net/failsafe/failsafe_ops.c       |  3 ++-
 drivers/net/fm10k/fm10k_ethdev.c          |  4 +++-
 drivers/net/hinic/hinic_pmd_ethdev.c      |  6 ++++--
 drivers/net/hns3/hns3_ethdev.c            |  6 ++++--
 drivers/net/hns3/hns3_ethdev_vf.c         |  6 ++++--
 drivers/net/i40e/i40e_ethdev.c            |  5 +++--
 drivers/net/i40e/i40e_ethdev_vf.c         |  5 +++--
 drivers/net/iavf/iavf_ethdev.c            |  5 +++--
 drivers/net/ice/ice_dcf_ethdev.c          |  6 ++++--
 drivers/net/ice/ice_ethdev.c              |  6 ++++--
 drivers/net/igc/igc_ethdev.c              |  6 ++++--
 drivers/net/ionic/ionic_ethdev.c          | 10 ++++++----
 drivers/net/ipn3ke/ipn3ke_representor.c   |  4 +++-
 drivers/net/ixgbe/ixgbe_ethdev.c          | 11 +++++++----
 drivers/net/kni/rte_eth_kni.c             |  4 +++-
 drivers/net/liquidio/lio_ethdev.c         |  4 +++-
 drivers/net/memif/rte_eth_memif.c         |  4 +++-
 drivers/net/mlx4/mlx4.c                   |  3 ++-
 drivers/net/mlx5/mlx5.c                   |  9 +++++----
 drivers/net/mlx5/mlx5.h                   |  2 +-
 drivers/net/mvneta/mvneta_ethdev.c        |  4 +++-
 drivers/net/mvpp2/mrvl_ethdev.c           |  4 +++-
 drivers/net/netvsc/hn_ethdev.c            |  4 +++-
 drivers/net/nfb/nfb_ethdev.c              |  4 +++-
 drivers/net/nfp/nfp_net.c                 |  6 ++++--
 drivers/net/octeontx/octeontx_ethdev.c    |  4 +++-
 drivers/net/octeontx2/otx2_ethdev.c       |  5 +++--
 drivers/net/pcap/rte_eth_pcap.c           |  3 ++-
 drivers/net/pfe/pfe_ethdev.c              |  8 +++++---
 drivers/net/qede/qede_ethdev.c            |  4 +++-
 drivers/net/sfc/sfc_ethdev.c              |  4 +++-
 drivers/net/softnic/rte_eth_softnic.c     |  4 ++--
 drivers/net/szedata2/rte_eth_szedata2.c   |  4 +++-
 drivers/net/tap/rte_eth_tap.c             |  4 +++-
 drivers/net/thunderx/nicvf_ethdev.c       |  4 +++-
 drivers/net/vhost/rte_eth_vhost.c         |  6 ++++--
 drivers/net/virtio/virtio_ethdev.c        |  6 ++++--
 drivers/net/vmxnet3/vmxnet3_ethdev.c      |  6 ++++--
 lib/librte_ethdev/rte_ethdev_core.h       |  2 +-
 61 files changed, 200 insertions(+), 99 deletions(-)

diff --git a/app/test/virtual_pmd.c b/app/test/virtual_pmd.c
index 79156cb85a..596b608f4c 100644
--- a/app/test/virtual_pmd.c
+++ b/app/test/virtual_pmd.c
@@ -62,9 +62,11 @@ static void  virtual_ethdev_stop(struct rte_eth_dev *eth_dev __rte_unused)
 		rte_pktmbuf_free(pkt);
 }
 
-static void
+static int
 virtual_ethdev_close(struct rte_eth_dev *dev __rte_unused)
-{}
+{
+	return 0;
+}
 
 static int
 virtual_ethdev_configure_success(struct rte_eth_dev *dev __rte_unused)
diff --git a/drivers/net/af_packet/rte_eth_af_packet.c b/drivers/net/af_packet/rte_eth_af_packet.c
index b9723e9619..7d0ff1cbb3 100644
--- a/drivers/net/af_packet/rte_eth_af_packet.c
+++ b/drivers/net/af_packet/rte_eth_af_packet.c
@@ -376,9 +376,10 @@ eth_stats_reset(struct rte_eth_dev *dev)
 	return 0;
 }
 
-static void
+static int
 eth_dev_close(struct rte_eth_dev *dev __rte_unused)
 {
+	return 0;
 }
 
 static void
diff --git a/drivers/net/af_xdp/rte_eth_af_xdp.c b/drivers/net/af_xdp/rte_eth_af_xdp.c
index 936d4a7d5f..bac9729bf8 100644
--- a/drivers/net/af_xdp/rte_eth_af_xdp.c
+++ b/drivers/net/af_xdp/rte_eth_af_xdp.c
@@ -701,7 +701,7 @@ xdp_umem_destroy(struct xsk_umem_info *umem)
 	umem = NULL;
 }
 
-static void
+static int
 eth_dev_close(struct rte_eth_dev *dev)
 {
 	struct pmd_internals *internals = dev->data->dev_private;
@@ -731,6 +731,8 @@ eth_dev_close(struct rte_eth_dev *dev)
 	dev->data->mac_addrs = NULL;
 
 	remove_xdp_program(internals);
+
+	return 0;
 }
 
 static void
diff --git a/drivers/net/ark/ark_ethdev.c b/drivers/net/ark/ark_ethdev.c
index 0321ec1264..9e1f8218a8 100644
--- a/drivers/net/ark/ark_ethdev.c
+++ b/drivers/net/ark/ark_ethdev.c
@@ -30,7 +30,7 @@ static int eth_ark_dev_uninit(struct rte_eth_dev *eth_dev);
 static int eth_ark_dev_configure(struct rte_eth_dev *dev);
 static int eth_ark_dev_start(struct rte_eth_dev *dev);
 static void eth_ark_dev_stop(struct rte_eth_dev *dev);
-static void eth_ark_dev_close(struct rte_eth_dev *dev);
+static int eth_ark_dev_close(struct rte_eth_dev *dev);
 static int eth_ark_dev_info_get(struct rte_eth_dev *dev,
 				struct rte_eth_dev_info *dev_info);
 static int eth_ark_dev_link_update(struct rte_eth_dev *dev,
@@ -681,7 +681,7 @@ eth_ark_dev_stop(struct rte_eth_dev *dev)
 	}
 }
 
-static void
+static int
 eth_ark_dev_close(struct rte_eth_dev *dev)
 {
 	struct ark_adapter *ark = dev->data->dev_private;
@@ -711,6 +711,8 @@ eth_ark_dev_close(struct rte_eth_dev *dev)
 
 	rte_free(dev->data->mac_addrs);
 	dev->data->mac_addrs = 0;
+
+	return 0;
 }
 
 static int
diff --git a/drivers/net/atlantic/atl_ethdev.c b/drivers/net/atlantic/atl_ethdev.c
index d3b00ab295..1d76883c52 100644
--- a/drivers/net/atlantic/atl_ethdev.c
+++ b/drivers/net/atlantic/atl_ethdev.c
@@ -22,7 +22,7 @@ static int  atl_dev_start(struct rte_eth_dev *dev);
 static void atl_dev_stop(struct rte_eth_dev *dev);
 static int  atl_dev_set_link_up(struct rte_eth_dev *dev);
 static int  atl_dev_set_link_down(struct rte_eth_dev *dev);
-static void atl_dev_close(struct rte_eth_dev *dev);
+static int  atl_dev_close(struct rte_eth_dev *dev);
 static int  atl_dev_reset(struct rte_eth_dev *dev);
 static int  atl_dev_promiscuous_enable(struct rte_eth_dev *dev);
 static int  atl_dev_promiscuous_disable(struct rte_eth_dev *dev);
@@ -718,7 +718,7 @@ atl_dev_set_link_down(struct rte_eth_dev *dev)
 /*
  * Reset and stop device.
  */
-static void
+static int
 atl_dev_close(struct rte_eth_dev *dev)
 {
 	PMD_INIT_FUNC_TRACE();
@@ -726,6 +726,8 @@ atl_dev_close(struct rte_eth_dev *dev)
 	atl_dev_stop(dev);
 
 	atl_free_queues(dev);
+
+	return 0;
 }
 
 static int
diff --git a/drivers/net/avp/avp_ethdev.c b/drivers/net/avp/avp_ethdev.c
index 5b47f0924a..2eed6acc3f 100644
--- a/drivers/net/avp/avp_ethdev.c
+++ b/drivers/net/avp/avp_ethdev.c
@@ -38,7 +38,7 @@ static int avp_dev_create(struct rte_pci_device *pci_dev,
 static int avp_dev_configure(struct rte_eth_dev *dev);
 static int avp_dev_start(struct rte_eth_dev *dev);
 static void avp_dev_stop(struct rte_eth_dev *dev);
-static void avp_dev_close(struct rte_eth_dev *dev);
+static int avp_dev_close(struct rte_eth_dev *dev);
 static int avp_dev_info_get(struct rte_eth_dev *dev,
 			    struct rte_eth_dev_info *dev_info);
 static int avp_vlan_offload_set(struct rte_eth_dev *dev, int mask);
@@ -2103,7 +2103,7 @@ avp_dev_stop(struct rte_eth_dev *eth_dev)
 	rte_spinlock_unlock(&avp->lock);
 }
 
-static void
+static int
 avp_dev_close(struct rte_eth_dev *eth_dev)
 {
 	struct avp_dev *avp = AVP_DEV_PRIVATE_TO_HW(eth_dev->data->dev_private);
@@ -2139,6 +2139,7 @@ avp_dev_close(struct rte_eth_dev *eth_dev)
 
 unlock:
 	rte_spinlock_unlock(&avp->lock);
+	return 0;
 }
 
 static int
diff --git a/drivers/net/axgbe/axgbe_ethdev.c b/drivers/net/axgbe/axgbe_ethdev.c
index 0c25739f82..be6f7cbda6 100644
--- a/drivers/net/axgbe/axgbe_ethdev.c
+++ b/drivers/net/axgbe/axgbe_ethdev.c
@@ -15,7 +15,7 @@ static int  axgbe_dev_configure(struct rte_eth_dev *dev);
 static int  axgbe_dev_start(struct rte_eth_dev *dev);
 static void axgbe_dev_stop(struct rte_eth_dev *dev);
 static void axgbe_dev_interrupt_handler(void *param);
-static void axgbe_dev_close(struct rte_eth_dev *dev);
+static int axgbe_dev_close(struct rte_eth_dev *dev);
 static int axgbe_dev_promiscuous_enable(struct rte_eth_dev *dev);
 static int axgbe_dev_promiscuous_disable(struct rte_eth_dev *dev);
 static int axgbe_dev_allmulticast_enable(struct rte_eth_dev *dev);
@@ -379,10 +379,11 @@ axgbe_dev_stop(struct rte_eth_dev *dev)
 }
 
 /* Clear all resources like TX/RX queues. */
-static void
+static int
 axgbe_dev_close(struct rte_eth_dev *dev)
 {
 	axgbe_dev_clear_queues(dev);
+	return 0;
 }
 
 static int
diff --git a/drivers/net/bnx2x/bnx2x_ethdev.c b/drivers/net/bnx2x/bnx2x_ethdev.c
index b2ea5fafa7..73217c2138 100644
--- a/drivers/net/bnx2x/bnx2x_ethdev.c
+++ b/drivers/net/bnx2x/bnx2x_ethdev.c
@@ -280,7 +280,7 @@ bnx2x_dev_stop(struct rte_eth_dev *dev)
 	return;
 }
 
-static void
+static int
 bnx2x_dev_close(struct rte_eth_dev *dev)
 {
 	struct bnx2x_softc *sc = dev->data->dev_private;
@@ -295,6 +295,8 @@ bnx2x_dev_close(struct rte_eth_dev *dev)
 
 	/* free ilt */
 	bnx2x_free_ilt_mem(sc);
+
+	return 0;
 }
 
 static int
diff --git a/drivers/net/bnxt/bnxt_ethdev.c b/drivers/net/bnxt/bnxt_ethdev.c
index 510a0d9e0a..02e53bf04d 100644
--- a/drivers/net/bnxt/bnxt_ethdev.c
+++ b/drivers/net/bnxt/bnxt_ethdev.c
@@ -1329,7 +1329,7 @@ static void bnxt_dev_stop_op(struct rte_eth_dev *eth_dev)
 		bp->flow_stat->flow_count = 0;
 }
 
-static void bnxt_dev_close_op(struct rte_eth_dev *eth_dev)
+static int bnxt_dev_close_op(struct rte_eth_dev *eth_dev)
 {
 	struct bnxt *bp = eth_dev->data->dev_private;
 
@@ -1365,6 +1365,8 @@ static void bnxt_dev_close_op(struct rte_eth_dev *eth_dev)
 
 	rte_free(bp->grp_info);
 	bp->grp_info = NULL;
+
+	return 0;
 }
 
 static void bnxt_mac_addr_remove_op(struct rte_eth_dev *eth_dev,
diff --git a/drivers/net/bnxt/bnxt_reps.c b/drivers/net/bnxt/bnxt_reps.c
index 6fa9a30d27..fa014125e3 100644
--- a/drivers/net/bnxt/bnxt_reps.c
+++ b/drivers/net/bnxt/bnxt_reps.c
@@ -449,9 +449,10 @@ void bnxt_vf_rep_dev_stop_op(struct rte_eth_dev *eth_dev)
 	bnxt_vf_rep_free_rx_mbufs(vfr_bp);
 }
 
-void bnxt_vf_rep_dev_close_op(struct rte_eth_dev *eth_dev)
+int bnxt_vf_rep_dev_close_op(struct rte_eth_dev *eth_dev)
 {
 	bnxt_vf_representor_uninit(eth_dev);
+	return 0;
 }
 
 int bnxt_vf_rep_dev_info_get_op(struct rte_eth_dev *eth_dev,
diff --git a/drivers/net/bnxt/bnxt_reps.h b/drivers/net/bnxt/bnxt_reps.h
index 418b95afcd..0a0501f888 100644
--- a/drivers/net/bnxt/bnxt_reps.h
+++ b/drivers/net/bnxt/bnxt_reps.h
@@ -38,7 +38,7 @@ int bnxt_vf_rep_tx_queue_setup_op(struct rte_eth_dev *eth_dev,
 void bnxt_vf_rep_rx_queue_release_op(void *rx_queue);
 void bnxt_vf_rep_tx_queue_release_op(void *tx_queue);
 void bnxt_vf_rep_dev_stop_op(struct rte_eth_dev *eth_dev);
-void bnxt_vf_rep_dev_close_op(struct rte_eth_dev *eth_dev);
+int bnxt_vf_rep_dev_close_op(struct rte_eth_dev *eth_dev);
 int bnxt_vf_rep_stats_get_op(struct rte_eth_dev *eth_dev,
 			     struct rte_eth_stats *stats);
 int bnxt_vf_rep_stats_reset_op(struct rte_eth_dev *eth_dev);
diff --git a/drivers/net/bonding/eth_bond_private.h b/drivers/net/bonding/eth_bond_private.h
index c9b2d0fe46..0a0034705d 100644
--- a/drivers/net/bonding/eth_bond_private.h
+++ b/drivers/net/bonding/eth_bond_private.h
@@ -318,7 +318,7 @@ bond_tlb_activate_slave(struct bond_dev_private *internals);
 void
 bond_ethdev_stop(struct rte_eth_dev *eth_dev);
 
-void
+int
 bond_ethdev_close(struct rte_eth_dev *dev);
 
 #endif
diff --git a/drivers/net/bonding/rte_eth_bond_pmd.c b/drivers/net/bonding/rte_eth_bond_pmd.c
index cdbd8151ed..e1123deb83 100644
--- a/drivers/net/bonding/rte_eth_bond_pmd.c
+++ b/drivers/net/bonding/rte_eth_bond_pmd.c
@@ -2095,7 +2095,7 @@ bond_ethdev_stop(struct rte_eth_dev *eth_dev)
 	}
 }
 
-void
+int
 bond_ethdev_close(struct rte_eth_dev *dev)
 {
 	struct bond_dev_private *internals = dev->data->dev_private;
@@ -2119,6 +2119,8 @@ bond_ethdev_close(struct rte_eth_dev *dev)
 	bond_flow_ops.flush(dev, &ferror);
 	bond_ethdev_free_queues(dev);
 	rte_bitmap_reset(internals->vlan_filter_bmp);
+
+	return 0;
 }
 
 /* forward declaration */
diff --git a/drivers/net/cxgbe/cxgbe_ethdev.c b/drivers/net/cxgbe/cxgbe_ethdev.c
index 7c6016d5cf..d575a9b389 100644
--- a/drivers/net/cxgbe/cxgbe_ethdev.c
+++ b/drivers/net/cxgbe/cxgbe_ethdev.c
@@ -315,7 +315,7 @@ int cxgbe_dev_mtu_set(struct rte_eth_dev *eth_dev, uint16_t mtu)
 /*
  * Stop device.
  */
-void cxgbe_dev_close(struct rte_eth_dev *eth_dev)
+int cxgbe_dev_close(struct rte_eth_dev *eth_dev)
 {
 	struct port_info *pi = eth_dev->data->dev_private;
 	struct adapter *adapter = pi->adapter;
@@ -323,7 +323,7 @@ void cxgbe_dev_close(struct rte_eth_dev *eth_dev)
 	CXGBE_FUNC_TRACE();
 
 	if (!(adapter->flags & FULL_INIT_DONE))
-		return;
+		return 0;
 
 	cxgbe_down(pi);
 
@@ -332,6 +332,8 @@ void cxgbe_dev_close(struct rte_eth_dev *eth_dev)
 	 *  have been disabled
 	 */
 	t4_sge_eth_clear_queues(pi);
+
+	return 0;
 }
 
 /* Start the device.
diff --git a/drivers/net/cxgbe/cxgbe_pfvf.h b/drivers/net/cxgbe/cxgbe_pfvf.h
index 0b7c52aecf..69d91639e9 100644
--- a/drivers/net/cxgbe/cxgbe_pfvf.h
+++ b/drivers/net/cxgbe/cxgbe_pfvf.h
@@ -19,7 +19,7 @@
 void cxgbe_dev_rx_queue_release(void *q);
 void cxgbe_dev_tx_queue_release(void *q);
 void cxgbe_dev_stop(struct rte_eth_dev *eth_dev);
-void cxgbe_dev_close(struct rte_eth_dev *eth_dev);
+int cxgbe_dev_close(struct rte_eth_dev *eth_dev);
 int cxgbe_dev_info_get(struct rte_eth_dev *eth_dev,
 		       struct rte_eth_dev_info *device_info);
 int cxgbe_dev_promiscuous_enable(struct rte_eth_dev *eth_dev);
diff --git a/drivers/net/dpaa/dpaa_ethdev.c b/drivers/net/dpaa/dpaa_ethdev.c
index c15e2b5462..da8d36b06f 100644
--- a/drivers/net/dpaa/dpaa_ethdev.c
+++ b/drivers/net/dpaa/dpaa_ethdev.c
@@ -353,7 +353,7 @@ static void dpaa_eth_dev_stop(struct rte_eth_dev *dev)
 	dev->tx_pkt_burst = dpaa_eth_tx_drop_all;
 }
 
-static void dpaa_eth_dev_close(struct rte_eth_dev *dev)
+static int dpaa_eth_dev_close(struct rte_eth_dev *dev)
 {
 	struct fman_if *fif = dev->process_private;
 	struct __fman_if *__fif;
@@ -376,6 +376,8 @@ static void dpaa_eth_dev_close(struct rte_eth_dev *dev)
 					     dpaa_interrupt_handler,
 					     (void *)dev);
 	}
+
+	return 0;
 }
 
 static int
diff --git a/drivers/net/dpaa2/dpaa2_ethdev.c b/drivers/net/dpaa2/dpaa2_ethdev.c
index 02c254846c..037ee0d2c6 100644
--- a/drivers/net/dpaa2/dpaa2_ethdev.c
+++ b/drivers/net/dpaa2/dpaa2_ethdev.c
@@ -1239,7 +1239,7 @@ dpaa2_dev_stop(struct rte_eth_dev *dev)
 	rte_eth_linkstatus_set(dev, &link);
 }
 
-static void
+static int
 dpaa2_dev_close(struct rte_eth_dev *dev)
 {
 	struct dpaa2_dev_priv *priv = dev->data->dev_private;
@@ -1255,11 +1255,13 @@ dpaa2_dev_close(struct rte_eth_dev *dev)
 	ret = dpni_reset(dpni, CMD_PRI_LOW, priv->token);
 	if (ret) {
 		DPAA2_PMD_ERR("Failure cleaning dpni device: err=%d", ret);
-		return;
+		return -1;
 	}
 
 	memset(&link, 0, sizeof(link));
 	rte_eth_linkstatus_set(dev, &link);
+
+	return 0;
 }
 
 static int
diff --git a/drivers/net/e1000/em_ethdev.c b/drivers/net/e1000/em_ethdev.c
index 902b1cdca0..45d3949ff7 100644
--- a/drivers/net/e1000/em_ethdev.c
+++ b/drivers/net/e1000/em_ethdev.c
@@ -34,7 +34,7 @@
 static int eth_em_configure(struct rte_eth_dev *dev);
 static int eth_em_start(struct rte_eth_dev *dev);
 static void eth_em_stop(struct rte_eth_dev *dev);
-static void eth_em_close(struct rte_eth_dev *dev);
+static int eth_em_close(struct rte_eth_dev *dev);
 static int eth_em_promiscuous_enable(struct rte_eth_dev *dev);
 static int eth_em_promiscuous_disable(struct rte_eth_dev *dev);
 static int eth_em_allmulticast_enable(struct rte_eth_dev *dev);
@@ -758,7 +758,7 @@ eth_em_stop(struct rte_eth_dev *dev)
 	}
 }
 
-static void
+static int
 eth_em_close(struct rte_eth_dev *dev)
 {
 	struct e1000_hw *hw = E1000_DEV_PRIVATE_TO_HW(dev->data->dev_private);
@@ -782,6 +782,8 @@ eth_em_close(struct rte_eth_dev *dev)
 	rte_intr_disable(intr_handle);
 	rte_intr_callback_unregister(intr_handle,
 				     eth_em_interrupt_handler, dev);
+
+	return 0;
 }
 
 static int
diff --git a/drivers/net/e1000/igb_ethdev.c b/drivers/net/e1000/igb_ethdev.c
index a5551e8175..564216b0a4 100644
--- a/drivers/net/e1000/igb_ethdev.c
+++ b/drivers/net/e1000/igb_ethdev.c
@@ -77,7 +77,7 @@ static int  eth_igb_start(struct rte_eth_dev *dev);
 static void eth_igb_stop(struct rte_eth_dev *dev);
 static int  eth_igb_dev_set_link_up(struct rte_eth_dev *dev);
 static int  eth_igb_dev_set_link_down(struct rte_eth_dev *dev);
-static void eth_igb_close(struct rte_eth_dev *dev);
+static int eth_igb_close(struct rte_eth_dev *dev);
 static int eth_igb_reset(struct rte_eth_dev *dev);
 static int  eth_igb_promiscuous_enable(struct rte_eth_dev *dev);
 static int  eth_igb_promiscuous_disable(struct rte_eth_dev *dev);
@@ -155,7 +155,7 @@ static void igbvf_intr_disable(struct e1000_hw *hw);
 static int igbvf_dev_configure(struct rte_eth_dev *dev);
 static int igbvf_dev_start(struct rte_eth_dev *dev);
 static void igbvf_dev_stop(struct rte_eth_dev *dev);
-static void igbvf_dev_close(struct rte_eth_dev *dev);
+static int igbvf_dev_close(struct rte_eth_dev *dev);
 static int igbvf_promiscuous_enable(struct rte_eth_dev *dev);
 static int igbvf_promiscuous_disable(struct rte_eth_dev *dev);
 static int igbvf_allmulticast_enable(struct rte_eth_dev *dev);
@@ -1535,7 +1535,7 @@ eth_igb_dev_set_link_down(struct rte_eth_dev *dev)
 	return 0;
 }
 
-static void
+static int
 eth_igb_close(struct rte_eth_dev *dev)
 {
 	struct e1000_hw *hw = E1000_DEV_PRIVATE_TO_HW(dev->data->dev_private);
@@ -1604,6 +1604,8 @@ eth_igb_close(struct rte_eth_dev *dev)
 
 	/* clear all the filters list */
 	igb_filterlist_flush(dev);
+
+	return 0;
 }
 
 /*
@@ -3382,7 +3384,7 @@ igbvf_dev_stop(struct rte_eth_dev *dev)
 	adapter->stopped = true;
 }
 
-static void
+static int
 igbvf_dev_close(struct rte_eth_dev *dev)
 {
 	struct e1000_hw *hw = E1000_DEV_PRIVATE_TO_HW(dev->data->dev_private);
@@ -3413,6 +3415,8 @@ igbvf_dev_close(struct rte_eth_dev *dev)
 	rte_intr_callback_unregister(&pci_dev->intr_handle,
 				     eth_igbvf_interrupt_handler,
 				     (void *)dev);
+
+	return 0;
 }
 
 static int
diff --git a/drivers/net/ena/ena_ethdev.c b/drivers/net/ena/ena_ethdev.c
index da8c8ad922..ea979fddd1 100644
--- a/drivers/net/ena/ena_ethdev.c
+++ b/drivers/net/ena/ena_ethdev.c
@@ -187,7 +187,7 @@ static void ena_init_rings(struct ena_adapter *adapter,
 static int ena_mtu_set(struct rte_eth_dev *dev, uint16_t mtu);
 static int ena_start(struct rte_eth_dev *dev);
 static void ena_stop(struct rte_eth_dev *dev);
-static void ena_close(struct rte_eth_dev *dev);
+static int ena_close(struct rte_eth_dev *dev);
 static int ena_dev_reset(struct rte_eth_dev *dev);
 static int ena_stats_get(struct rte_eth_dev *dev, struct rte_eth_stats *stats);
 static void ena_rx_queue_release_all(struct rte_eth_dev *dev);
@@ -487,7 +487,7 @@ static void ena_config_debug_area(struct ena_adapter *adapter)
 	ena_com_delete_debug_area(&adapter->ena_dev);
 }
 
-static void ena_close(struct rte_eth_dev *dev)
+static int ena_close(struct rte_eth_dev *dev)
 {
 	struct rte_pci_device *pci_dev = RTE_ETH_DEV_TO_PCI(dev);
 	struct rte_intr_handle *intr_handle = &pci_dev->intr_handle;
@@ -513,6 +513,8 @@ static void ena_close(struct rte_eth_dev *dev)
 	 * release of the resource in the rte_eth_dev_release_port().
 	 */
 	dev->data->mac_addrs = NULL;
+
+	return 0;
 }
 
 static int
diff --git a/drivers/net/enetc/enetc_ethdev.c b/drivers/net/enetc/enetc_ethdev.c
index c6fb42740d..bdb32762ce 100644
--- a/drivers/net/enetc/enetc_ethdev.c
+++ b/drivers/net/enetc/enetc_ethdev.c
@@ -545,7 +545,7 @@ enetc_stats_reset(struct rte_eth_dev *dev)
 	return 0;
 }
 
-static void
+static int
 enetc_dev_close(struct rte_eth_dev *dev)
 {
 	uint16_t i;
@@ -564,6 +564,8 @@ enetc_dev_close(struct rte_eth_dev *dev)
 		dev->data->tx_queues[i] = NULL;
 	}
 	dev->data->nb_tx_queues = 0;
+
+	return 0;
 }
 
 static int
diff --git a/drivers/net/enic/enic_ethdev.c b/drivers/net/enic/enic_ethdev.c
index ca75919ee2..b14ed00085 100644
--- a/drivers/net/enic/enic_ethdev.c
+++ b/drivers/net/enic/enic_ethdev.c
@@ -445,12 +445,14 @@ static void enicpmd_dev_stop(struct rte_eth_dev *eth_dev)
 /*
  * Stop device.
  */
-static void enicpmd_dev_close(struct rte_eth_dev *eth_dev)
+static int enicpmd_dev_close(struct rte_eth_dev *eth_dev)
 {
 	struct enic *enic = pmd_priv(eth_dev);
 
 	ENICPMD_FUNC_TRACE();
 	enic_remove(enic);
+
+	return 0;
 }
 
 static int enicpmd_dev_link_update(struct rte_eth_dev *eth_dev,
diff --git a/drivers/net/failsafe/failsafe_ops.c b/drivers/net/failsafe/failsafe_ops.c
index 96f7e456f9..93ebd09114 100644
--- a/drivers/net/failsafe/failsafe_ops.c
+++ b/drivers/net/failsafe/failsafe_ops.c
@@ -240,7 +240,7 @@ fs_dev_set_link_down(struct rte_eth_dev *dev)
 }
 
 static void fs_dev_free_queues(struct rte_eth_dev *dev);
-static void
+static int
 fs_dev_close(struct rte_eth_dev *dev)
 {
 	struct sub_device *sdev;
@@ -259,6 +259,7 @@ fs_dev_close(struct rte_eth_dev *dev)
 	}
 	fs_dev_free_queues(dev);
 	fs_unlock(dev, 0);
+	return 0;
 }
 
 static int
diff --git a/drivers/net/fm10k/fm10k_ethdev.c b/drivers/net/fm10k/fm10k_ethdev.c
index b574693bca..82f46cebf5 100644
--- a/drivers/net/fm10k/fm10k_ethdev.c
+++ b/drivers/net/fm10k/fm10k_ethdev.c
@@ -2779,7 +2779,7 @@ fm10k_close_mbx_service(struct fm10k_hw *hw)
 	hw->mbx.ops.disconnect(hw, &hw->mbx);
 }
 
-static void
+static int
 fm10k_dev_close(struct rte_eth_dev *dev)
 {
 	struct fm10k_hw *hw = FM10K_DEV_PRIVATE_TO_HW(dev->data->dev_private);
@@ -2824,6 +2824,8 @@ fm10k_dev_close(struct rte_eth_dev *dev)
 		rte_intr_callback_unregister(intr_handle,
 			fm10k_dev_interrupt_handler_vf, (void *)dev);
 	}
+
+	return 0;
 }
 
 static const struct eth_dev_ops fm10k_eth_dev_ops = {
diff --git a/drivers/net/hinic/hinic_pmd_ethdev.c b/drivers/net/hinic/hinic_pmd_ethdev.c
index 67e6afcf7a..5f2e97d3bd 100644
--- a/drivers/net/hinic/hinic_pmd_ethdev.c
+++ b/drivers/net/hinic/hinic_pmd_ethdev.c
@@ -2936,7 +2936,7 @@ static void hinic_nic_dev_destroy(struct rte_eth_dev *eth_dev)
  * @param dev
  *   Pointer to Ethernet device structure.
  */
-static void hinic_dev_close(struct rte_eth_dev *dev)
+static int hinic_dev_close(struct rte_eth_dev *dev)
 {
 	struct hinic_nic_dev *nic_dev = HINIC_ETH_DEV_TO_PRIVATE_NIC_DEV(dev);
 
@@ -2944,7 +2944,7 @@ static void hinic_dev_close(struct rte_eth_dev *dev)
 					   &nic_dev->dev_status)) {
 		PMD_DRV_LOG(WARNING, "Device %s already closed",
 			    dev->data->name);
-		return;
+		return 0;
 	}
 
 	/* stop device first */
@@ -2971,6 +2971,8 @@ static void hinic_dev_close(struct rte_eth_dev *dev)
 
 	/* deinit nic hardware device */
 	hinic_nic_dev_destroy(dev);
+
+	return 0;
 }
 
 static const struct eth_dev_ops hinic_pmd_ops = {
diff --git a/drivers/net/hns3/hns3_ethdev.c b/drivers/net/hns3/hns3_ethdev.c
index 81e773046e..96806afd87 100644
--- a/drivers/net/hns3/hns3_ethdev.c
+++ b/drivers/net/hns3/hns3_ethdev.c
@@ -4747,7 +4747,7 @@ hns3_dev_stop(struct rte_eth_dev *dev)
 	rte_spinlock_unlock(&hw->lock);
 }
 
-static void
+static int
 hns3_dev_close(struct rte_eth_dev *eth_dev)
 {
 	struct hns3_adapter *hns = eth_dev->data->dev_private;
@@ -4756,7 +4756,7 @@ hns3_dev_close(struct rte_eth_dev *eth_dev)
 	if (rte_eal_process_type() != RTE_PROC_PRIMARY) {
 		rte_free(eth_dev->process_private);
 		eth_dev->process_private = NULL;
-		return;
+		return 0;
 	}
 
 	if (hw->adapter_state == HNS3_NIC_STARTED)
@@ -4776,6 +4776,8 @@ hns3_dev_close(struct rte_eth_dev *eth_dev)
 	eth_dev->process_private = NULL;
 	hns3_mp_uninit_primary();
 	hns3_warn(hw, "Close port %d finished", hw->data->port_id);
+
+	return 0;
 }
 
 static int
diff --git a/drivers/net/hns3/hns3_ethdev_vf.c b/drivers/net/hns3/hns3_ethdev_vf.c
index 1d2941f0e3..b732bbc732 100644
--- a/drivers/net/hns3/hns3_ethdev_vf.c
+++ b/drivers/net/hns3/hns3_ethdev_vf.c
@@ -1821,14 +1821,14 @@ hns3vf_dev_stop(struct rte_eth_dev *dev)
 	rte_spinlock_unlock(&hw->lock);
 }
 
-static void
+static int
 hns3vf_dev_close(struct rte_eth_dev *eth_dev)
 {
 	struct hns3_adapter *hns = eth_dev->data->dev_private;
 	struct hns3_hw *hw = &hns->hw;
 
 	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
-		return;
+		return 0;
 
 	if (hw->adapter_state == HNS3_NIC_STARTED)
 		hns3vf_dev_stop(eth_dev);
@@ -1846,6 +1846,8 @@ hns3vf_dev_close(struct rte_eth_dev *eth_dev)
 	eth_dev->process_private = NULL;
 	hns3_mp_uninit_primary();
 	hns3_warn(hw, "Close port %d finished", hw->data->port_id);
+
+	return 0;
 }
 
 static int
diff --git a/drivers/net/i40e/i40e_ethdev.c b/drivers/net/i40e/i40e_ethdev.c
index 11c02b1888..8dccb513d4 100644
--- a/drivers/net/i40e/i40e_ethdev.c
+++ b/drivers/net/i40e/i40e_ethdev.c
@@ -225,7 +225,7 @@ static int eth_i40e_dev_uninit(struct rte_eth_dev *eth_dev);
 static int i40e_dev_configure(struct rte_eth_dev *dev);
 static int i40e_dev_start(struct rte_eth_dev *dev);
 static void i40e_dev_stop(struct rte_eth_dev *dev);
-static void i40e_dev_close(struct rte_eth_dev *dev);
+static int i40e_dev_close(struct rte_eth_dev *dev);
 static int  i40e_dev_reset(struct rte_eth_dev *dev);
 static int i40e_dev_promiscuous_enable(struct rte_eth_dev *dev);
 static int i40e_dev_promiscuous_disable(struct rte_eth_dev *dev);
@@ -2608,7 +2608,7 @@ i40e_dev_stop(struct rte_eth_dev *dev)
 	pf->adapter->rss_reta_updated = 0;
 }
 
-static void
+static int
 i40e_dev_close(struct rte_eth_dev *dev)
 {
 	struct i40e_pf *pf = I40E_DEV_PRIVATE_TO_PF(dev->data->dev_private);
@@ -2748,6 +2748,7 @@ i40e_dev_close(struct rte_eth_dev *dev)
 	i40e_tm_conf_uninit(dev);
 
 	hw->adapter_closed = 1;
+	return 0;
 }
 
 /*
diff --git a/drivers/net/i40e/i40e_ethdev_vf.c b/drivers/net/i40e/i40e_ethdev_vf.c
index 69cab8e739..104eef639d 100644
--- a/drivers/net/i40e/i40e_ethdev_vf.c
+++ b/drivers/net/i40e/i40e_ethdev_vf.c
@@ -90,7 +90,7 @@ static int i40evf_dev_xstats_reset(struct rte_eth_dev *dev);
 static int i40evf_vlan_filter_set(struct rte_eth_dev *dev,
 				  uint16_t vlan_id, int on);
 static int i40evf_vlan_offload_set(struct rte_eth_dev *dev, int mask);
-static void i40evf_dev_close(struct rte_eth_dev *dev);
+static int i40evf_dev_close(struct rte_eth_dev *dev);
 static int i40evf_dev_reset(struct rte_eth_dev *dev);
 static int i40evf_check_vf_reset_done(struct rte_eth_dev *dev);
 static int i40evf_dev_promiscuous_enable(struct rte_eth_dev *dev);
@@ -2355,7 +2355,7 @@ i40evf_dev_stats_get(struct rte_eth_dev *dev, struct rte_eth_stats *stats)
 	return ret;
 }
 
-static void
+static int
 i40evf_dev_close(struct rte_eth_dev *dev)
 {
 	struct i40e_hw *hw = I40E_DEV_PRIVATE_TO_HW(dev->data->dev_private);
@@ -2387,6 +2387,7 @@ i40evf_dev_close(struct rte_eth_dev *dev)
 	vf->aq_resp = NULL;
 
 	hw->adapter_closed = 1;
+	return 0;
 }
 
 /*
diff --git a/drivers/net/iavf/iavf_ethdev.c b/drivers/net/iavf/iavf_ethdev.c
index c3aa4cd725..4860a94d4c 100644
--- a/drivers/net/iavf/iavf_ethdev.c
+++ b/drivers/net/iavf/iavf_ethdev.c
@@ -32,7 +32,7 @@
 static int iavf_dev_configure(struct rte_eth_dev *dev);
 static int iavf_dev_start(struct rte_eth_dev *dev);
 static void iavf_dev_stop(struct rte_eth_dev *dev);
-static void iavf_dev_close(struct rte_eth_dev *dev);
+static int iavf_dev_close(struct rte_eth_dev *dev);
 static int iavf_dev_reset(struct rte_eth_dev *dev);
 static int iavf_dev_info_get(struct rte_eth_dev *dev,
 			     struct rte_eth_dev_info *dev_info);
@@ -1408,7 +1408,7 @@ iavf_dev_init(struct rte_eth_dev *eth_dev)
 	return 0;
 }
 
-static void
+static int
 iavf_dev_close(struct rte_eth_dev *dev)
 {
 	struct iavf_hw *hw = IAVF_DEV_PRIVATE_TO_HW(dev->data->dev_private);
@@ -1428,6 +1428,7 @@ iavf_dev_close(struct rte_eth_dev *dev)
 	rte_intr_callback_unregister(intr_handle,
 				     iavf_dev_interrupt_handler, dev);
 	iavf_disable_irq0(hw);
+	return 0;
 }
 
 static int
diff --git a/drivers/net/ice/ice_dcf_ethdev.c b/drivers/net/ice/ice_dcf_ethdev.c
index 2faed3cc7a..5b626cb641 100644
--- a/drivers/net/ice/ice_dcf_ethdev.c
+++ b/drivers/net/ice/ice_dcf_ethdev.c
@@ -847,13 +847,13 @@ ice_dcf_stats_reset(struct rte_eth_dev *dev)
 	return 0;
 }
 
-static void
+static int
 ice_dcf_dev_close(struct rte_eth_dev *dev)
 {
 	struct ice_dcf_adapter *adapter = dev->data->dev_private;
 
 	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
-		return;
+		return 0;
 
 	dev->dev_ops = NULL;
 	dev->rx_pkt_burst = NULL;
@@ -861,6 +861,8 @@ ice_dcf_dev_close(struct rte_eth_dev *dev)
 
 	ice_dcf_uninit_parent_adapter(dev);
 	ice_dcf_uninit_hw(dev, &adapter->real_hw);
+
+	return 0;
 }
 
 static int
diff --git a/drivers/net/ice/ice_ethdev.c b/drivers/net/ice/ice_ethdev.c
index 8d435e8892..042ee6def0 100644
--- a/drivers/net/ice/ice_ethdev.c
+++ b/drivers/net/ice/ice_ethdev.c
@@ -74,7 +74,7 @@ static struct proto_xtr_ol_flag ice_proto_xtr_ol_flag_params[] = {
 static int ice_dev_configure(struct rte_eth_dev *dev);
 static int ice_dev_start(struct rte_eth_dev *dev);
 static void ice_dev_stop(struct rte_eth_dev *dev);
-static void ice_dev_close(struct rte_eth_dev *dev);
+static int ice_dev_close(struct rte_eth_dev *dev);
 static int ice_dev_reset(struct rte_eth_dev *dev);
 static int ice_dev_info_get(struct rte_eth_dev *dev,
 			    struct rte_eth_dev_info *dev_info);
@@ -2389,7 +2389,7 @@ ice_dev_stop(struct rte_eth_dev *dev)
 	pf->adapter_stopped = true;
 }
 
-static void
+static int
 ice_dev_close(struct rte_eth_dev *dev)
 {
 	struct ice_pf *pf = ICE_DEV_PRIVATE_TO_PF(dev->data->dev_private);
@@ -2437,6 +2437,8 @@ ice_dev_close(struct rte_eth_dev *dev)
 	/* unregister callback func from eal lib */
 	rte_intr_callback_unregister(intr_handle,
 				     ice_interrupt_handler, dev);
+
+	return 0;
 }
 
 static int
diff --git a/drivers/net/igc/igc_ethdev.c b/drivers/net/igc/igc_ethdev.c
index 6ab3ee909d..99014a369c 100644
--- a/drivers/net/igc/igc_ethdev.c
+++ b/drivers/net/igc/igc_ethdev.c
@@ -183,7 +183,7 @@ static void eth_igc_stop(struct rte_eth_dev *dev);
 static int eth_igc_start(struct rte_eth_dev *dev);
 static int eth_igc_set_link_up(struct rte_eth_dev *dev);
 static int eth_igc_set_link_down(struct rte_eth_dev *dev);
-static void eth_igc_close(struct rte_eth_dev *dev);
+static int eth_igc_close(struct rte_eth_dev *dev);
 static int eth_igc_reset(struct rte_eth_dev *dev);
 static int eth_igc_promiscuous_enable(struct rte_eth_dev *dev);
 static int eth_igc_promiscuous_disable(struct rte_eth_dev *dev);
@@ -1170,7 +1170,7 @@ igc_dev_free_queues(struct rte_eth_dev *dev)
 	dev->data->nb_tx_queues = 0;
 }
 
-static void
+static int
 eth_igc_close(struct rte_eth_dev *dev)
 {
 	struct rte_pci_device *pci_dev = RTE_ETH_DEV_TO_PCI(dev);
@@ -1204,6 +1204,8 @@ eth_igc_close(struct rte_eth_dev *dev)
 
 	/* Reset any pending lock */
 	igc_reset_swfw_lock(hw);
+
+	return 0;
 }
 
 static void
diff --git a/drivers/net/ionic/ionic_ethdev.c b/drivers/net/ionic/ionic_ethdev.c
index 800f6e5d64..1775fd29b7 100644
--- a/drivers/net/ionic/ionic_ethdev.c
+++ b/drivers/net/ionic/ionic_ethdev.c
@@ -25,7 +25,7 @@ static int  ionic_dev_configure(struct rte_eth_dev *dev);
 static int  ionic_dev_mtu_set(struct rte_eth_dev *dev, uint16_t mtu);
 static int  ionic_dev_start(struct rte_eth_dev *dev);
 static void ionic_dev_stop(struct rte_eth_dev *dev);
-static void ionic_dev_close(struct rte_eth_dev *dev);
+static int  ionic_dev_close(struct rte_eth_dev *dev);
 static int  ionic_dev_set_link_up(struct rte_eth_dev *dev);
 static int  ionic_dev_set_link_down(struct rte_eth_dev *dev);
 static int  ionic_dev_link_update(struct rte_eth_dev *eth_dev,
@@ -956,7 +956,7 @@ ionic_dev_stop(struct rte_eth_dev *eth_dev)
 /*
  * Reset and stop device.
  */
-static void
+static int
 ionic_dev_close(struct rte_eth_dev *eth_dev)
 {
 	struct ionic_lif *lif = IONIC_ETH_DEV_TO_LIF(eth_dev);
@@ -967,14 +967,16 @@ ionic_dev_close(struct rte_eth_dev *eth_dev)
 	err = ionic_lif_stop(lif);
 	if (err) {
 		IONIC_PRINT(ERR, "Cannot stop LIF: %d", err);
-		return;
+		return -1;
 	}
 
 	err = eth_ionic_dev_uninit(eth_dev);
 	if (err) {
 		IONIC_PRINT(ERR, "Cannot destroy LIF: %d", err);
-		return;
+		return -1;
 	}
+
+	return 0;
 }
 
 static int
diff --git a/drivers/net/ipn3ke/ipn3ke_representor.c b/drivers/net/ipn3ke/ipn3ke_representor.c
index b673c49149..d49abbf758 100644
--- a/drivers/net/ipn3ke/ipn3ke_representor.c
+++ b/drivers/net/ipn3ke/ipn3ke_representor.c
@@ -208,7 +208,7 @@ ipn3ke_rpst_dev_stop(struct rte_eth_dev *dev)
 	}
 }
 
-static void
+static int
 ipn3ke_rpst_dev_close(struct rte_eth_dev *dev)
 {
 	struct ipn3ke_hw *hw = IPN3KE_DEV_PRIVATE_TO_HW(dev);
@@ -221,6 +221,8 @@ ipn3ke_rpst_dev_close(struct rte_eth_dev *dev)
 		/* Disable the RX path */
 		ipn3ke_xmac_rx_disable(hw, rpst->port_id, 0);
 	}
+
+	return 0;
 }
 
 /*
diff --git a/drivers/net/ixgbe/ixgbe_ethdev.c b/drivers/net/ixgbe/ixgbe_ethdev.c
index fd0cb9b0e2..5c12a575c2 100644
--- a/drivers/net/ixgbe/ixgbe_ethdev.c
+++ b/drivers/net/ixgbe/ixgbe_ethdev.c
@@ -147,7 +147,7 @@ static int  ixgbe_dev_start(struct rte_eth_dev *dev);
 static void ixgbe_dev_stop(struct rte_eth_dev *dev);
 static int  ixgbe_dev_set_link_up(struct rte_eth_dev *dev);
 static int  ixgbe_dev_set_link_down(struct rte_eth_dev *dev);
-static void ixgbe_dev_close(struct rte_eth_dev *dev);
+static int  ixgbe_dev_close(struct rte_eth_dev *dev);
 static int  ixgbe_dev_reset(struct rte_eth_dev *dev);
 static int ixgbe_dev_promiscuous_enable(struct rte_eth_dev *dev);
 static int ixgbe_dev_promiscuous_disable(struct rte_eth_dev *dev);
@@ -251,7 +251,7 @@ static int  ixgbevf_dev_start(struct rte_eth_dev *dev);
 static int ixgbevf_dev_link_update(struct rte_eth_dev *dev,
 				   int wait_to_complete);
 static void ixgbevf_dev_stop(struct rte_eth_dev *dev);
-static void ixgbevf_dev_close(struct rte_eth_dev *dev);
+static int ixgbevf_dev_close(struct rte_eth_dev *dev);
 static int  ixgbevf_dev_reset(struct rte_eth_dev *dev);
 static void ixgbevf_intr_disable(struct rte_eth_dev *dev);
 static void ixgbevf_intr_enable(struct rte_eth_dev *dev);
@@ -2994,7 +2994,7 @@ ixgbe_dev_set_link_down(struct rte_eth_dev *dev)
 /*
  * Reset and stop device.
  */
-static void
+static int
 ixgbe_dev_close(struct rte_eth_dev *dev)
 {
 	struct ixgbe_hw *hw =
@@ -3065,6 +3065,7 @@ ixgbe_dev_close(struct rte_eth_dev *dev)
 	rte_free(dev->security_ctx);
 #endif
 
+	return 0;
 }
 
 /*
@@ -5449,7 +5450,7 @@ ixgbevf_dev_stop(struct rte_eth_dev *dev)
 	adapter->rss_reta_updated = 0;
 }
 
-static void
+static int
 ixgbevf_dev_close(struct rte_eth_dev *dev)
 {
 	struct ixgbe_hw *hw = IXGBE_DEV_PRIVATE_TO_HW(dev->data->dev_private);
@@ -5478,6 +5479,8 @@ ixgbevf_dev_close(struct rte_eth_dev *dev)
 	rte_intr_disable(intr_handle);
 	rte_intr_callback_unregister(intr_handle,
 				     ixgbevf_dev_interrupt_handler, dev);
+
+	return 0;
 }
 
 /*
diff --git a/drivers/net/kni/rte_eth_kni.c b/drivers/net/kni/rte_eth_kni.c
index 831fe96c96..aa365a57de 100644
--- a/drivers/net/kni/rte_eth_kni.c
+++ b/drivers/net/kni/rte_eth_kni.c
@@ -198,7 +198,7 @@ eth_kni_dev_stop(struct rte_eth_dev *dev)
 	dev->data->dev_link.link_status = 0;
 }
 
-static void
+static int
 eth_kni_close(struct rte_eth_dev *eth_dev)
 {
 	struct pmd_internals *internals;
@@ -214,6 +214,8 @@ eth_kni_close(struct rte_eth_dev *eth_dev)
 	if (ret)
 		PMD_LOG(WARNING, "Not able to release kni for %s",
 			eth_dev->data->name);
+
+	return 0;
 }
 
 static int
diff --git a/drivers/net/liquidio/lio_ethdev.c b/drivers/net/liquidio/lio_ethdev.c
index 2258838c4c..d7df0c7853 100644
--- a/drivers/net/liquidio/lio_ethdev.c
+++ b/drivers/net/liquidio/lio_ethdev.c
@@ -1550,7 +1550,7 @@ lio_dev_set_link_down(struct rte_eth_dev *eth_dev)
  * @return
  *    - nothing
  */
-static void
+static int
 lio_dev_close(struct rte_eth_dev *eth_dev)
 {
 	struct lio_device *lio_dev = LIO_DEV(eth_dev);
@@ -1581,6 +1581,8 @@ lio_dev_close(struct rte_eth_dev *eth_dev)
 
 	 /* Delete all queues */
 	lio_dev_clear_queues(eth_dev);
+
+	return 0;
 }
 
 /**
diff --git a/drivers/net/memif/rte_eth_memif.c b/drivers/net/memif/rte_eth_memif.c
index c1c7e9f8d8..c00b9702a9 100644
--- a/drivers/net/memif/rte_eth_memif.c
+++ b/drivers/net/memif/rte_eth_memif.c
@@ -1203,7 +1203,7 @@ memif_dev_start(struct rte_eth_dev *dev)
 	return ret;
 }
 
-static void
+static int
 memif_dev_close(struct rte_eth_dev *dev)
 {
 	struct pmd_internals *pmd = dev->data->dev_private;
@@ -1224,6 +1224,8 @@ memif_dev_close(struct rte_eth_dev *dev)
 	}
 
 	rte_free(dev->process_private);
+
+	return 0;
 }
 
 static int
diff --git a/drivers/net/mlx4/mlx4.c b/drivers/net/mlx4/mlx4.c
index 71061a720f..df59314b66 100644
--- a/drivers/net/mlx4/mlx4.c
+++ b/drivers/net/mlx4/mlx4.c
@@ -370,7 +370,7 @@ mlx4_dev_stop(struct rte_eth_dev *dev)
  * @param dev
  *   Pointer to Ethernet device structure.
  */
-static void
+static int
 mlx4_dev_close(struct rte_eth_dev *dev)
 {
 	struct mlx4_priv *priv = dev->data->dev_private;
@@ -400,6 +400,7 @@ mlx4_dev_close(struct rte_eth_dev *dev)
 		MLX4_ASSERT(priv->ctx == NULL);
 	mlx4_intr_uninstall(priv);
 	memset(priv, 0, sizeof(*priv));
+	return 0;
 }
 
 static const struct eth_dev_ops mlx4_dev_ops = {
diff --git a/drivers/net/mlx5/mlx5.c b/drivers/net/mlx5/mlx5.c
index 1e4c695f84..e8685b216f 100644
--- a/drivers/net/mlx5/mlx5.c
+++ b/drivers/net/mlx5/mlx5.c
@@ -1338,7 +1338,7 @@ mlx5_proc_priv_uninit(struct rte_eth_dev *dev)
  * @param dev
  *   Pointer to Ethernet device structure.
  */
-void
+int
 mlx5_dev_close(struct rte_eth_dev *dev)
 {
 	struct mlx5_priv *priv = dev->data->dev_private;
@@ -1348,14 +1348,14 @@ mlx5_dev_close(struct rte_eth_dev *dev)
 	if (rte_eal_process_type() == RTE_PROC_SECONDARY) {
 		/* Check if process_private released. */
 		if (!dev->process_private)
-			return;
+			return 0;
 		mlx5_tx_uar_uninit_secondary(dev);
 		mlx5_proc_priv_uninit(dev);
 		rte_eth_dev_release_port(dev);
-		return;
+		return 0;
 	}
 	if (!priv->sh)
-		return;
+		return 0;
 	DRV_LOG(DEBUG, "port %u closing device \"%s\"",
 		dev->data->port_id,
 		((priv->sh->ctx != NULL) ?
@@ -1478,6 +1478,7 @@ mlx5_dev_close(struct rte_eth_dev *dev)
 	 * it is freed when dev_private is freed.
 	 */
 	dev->data->mac_addrs = NULL;
+	return 0;
 }
 
 /**
diff --git a/drivers/net/mlx5/mlx5.h b/drivers/net/mlx5/mlx5.h
index 78d6eb7281..b1308ba594 100644
--- a/drivers/net/mlx5/mlx5.h
+++ b/drivers/net/mlx5/mlx5.h
@@ -767,7 +767,7 @@ int mlx5_proc_priv_init(struct rte_eth_dev *dev);
 int mlx5_udp_tunnel_port_add(struct rte_eth_dev *dev,
 			      struct rte_eth_udp_tunnel *udp_tunnel);
 uint16_t mlx5_eth_find_next(uint16_t port_id, struct rte_pci_device *pci_dev);
-void mlx5_dev_close(struct rte_eth_dev *dev);
+int mlx5_dev_close(struct rte_eth_dev *dev);
 
 /* Macro to iterate over all valid ports for mlx5 driver. */
 #define MLX5_ETH_FOREACH_DEV(port_id, pci_dev) \
diff --git a/drivers/net/mvneta/mvneta_ethdev.c b/drivers/net/mvneta/mvneta_ethdev.c
index 673a8e2b3a..b13e2c52f3 100644
--- a/drivers/net/mvneta/mvneta_ethdev.c
+++ b/drivers/net/mvneta/mvneta_ethdev.c
@@ -429,7 +429,7 @@ mvneta_dev_stop(struct rte_eth_dev *dev)
  * @param dev
  *   Pointer to Ethernet device structure.
  */
-static void
+static int
 mvneta_dev_close(struct rte_eth_dev *dev)
 {
 	struct mvneta_priv *priv = dev->data->dev_private;
@@ -455,6 +455,8 @@ mvneta_dev_close(struct rte_eth_dev *dev)
 		mvneta_neta_deinit();
 		rte_mvep_deinit(MVEP_MOD_T_NETA);
 	}
+
+	return 0;
 }
 
 /**
diff --git a/drivers/net/mvpp2/mrvl_ethdev.c b/drivers/net/mvpp2/mrvl_ethdev.c
index 96b27dfd30..b40169daba 100644
--- a/drivers/net/mvpp2/mrvl_ethdev.c
+++ b/drivers/net/mvpp2/mrvl_ethdev.c
@@ -855,7 +855,7 @@ mrvl_dev_stop(struct rte_eth_dev *dev)
  * @param dev
  *   Pointer to Ethernet device structure.
  */
-static void
+static int
 mrvl_dev_close(struct rte_eth_dev *dev)
 {
 	struct mrvl_priv *priv = dev->data->dev_private;
@@ -915,6 +915,8 @@ mrvl_dev_close(struct rte_eth_dev *dev)
 		mrvl_deinit_pp2();
 		rte_mvep_deinit(MVEP_MOD_T_PP2);
 	}
+
+	return 0;
 }
 
 /**
diff --git a/drivers/net/netvsc/hn_ethdev.c b/drivers/net/netvsc/hn_ethdev.c
index fd91c0e491..f02f546882 100644
--- a/drivers/net/netvsc/hn_ethdev.c
+++ b/drivers/net/netvsc/hn_ethdev.c
@@ -841,13 +841,15 @@ hn_dev_stop(struct rte_eth_dev *dev)
 	hn_vf_stop(dev);
 }
 
-static void
+static int
 hn_dev_close(struct rte_eth_dev *dev)
 {
 	PMD_INIT_FUNC_TRACE();
 
 	hn_vf_close(dev);
 	hn_dev_free_queues(dev);
+
+	return 0;
 }
 
 static const struct eth_dev_ops hn_eth_dev_ops = {
diff --git a/drivers/net/nfb/nfb_ethdev.c b/drivers/net/nfb/nfb_ethdev.c
index b039ab6fc2..c3c3d003f1 100644
--- a/drivers/net/nfb/nfb_ethdev.c
+++ b/drivers/net/nfb/nfb_ethdev.c
@@ -209,7 +209,7 @@ nfb_eth_dev_info(struct rte_eth_dev *dev,
  * @param dev
  *   Pointer to Ethernet device structure.
  */
-static void
+static int
 nfb_eth_dev_close(struct rte_eth_dev *dev)
 {
 	struct pmd_internals *internals = dev->data->dev_private;
@@ -235,6 +235,8 @@ nfb_eth_dev_close(struct rte_eth_dev *dev)
 
 	rte_free(dev->data->mac_addrs);
 	dev->data->mac_addrs = NULL;
+
+	return 0;
 }
 
 /**
diff --git a/drivers/net/nfp/nfp_net.c b/drivers/net/nfp/nfp_net.c
index 99946279db..c20d71cdc3 100644
--- a/drivers/net/nfp/nfp_net.c
+++ b/drivers/net/nfp/nfp_net.c
@@ -50,7 +50,7 @@
 #include <errno.h>
 
 /* Prototypes */
-static void nfp_net_close(struct rte_eth_dev *dev);
+static int nfp_net_close(struct rte_eth_dev *dev);
 static int nfp_net_configure(struct rte_eth_dev *dev);
 static void nfp_net_dev_interrupt_handler(void *param);
 static void nfp_net_dev_interrupt_delayed_handler(void *param);
@@ -864,7 +864,7 @@ nfp_net_set_link_down(struct rte_eth_dev *dev)
 }
 
 /* Reset and stop device. The device can not be restarted. */
-static void
+static int
 nfp_net_close(struct rte_eth_dev *dev)
 {
 	struct nfp_net_hw *hw;
@@ -906,6 +906,8 @@ nfp_net_close(struct rte_eth_dev *dev)
 	 * The ixgbe PMD driver disables the pcie master on the
 	 * device. The i40e does not...
 	 */
+
+	return 0;
 }
 
 static int
diff --git a/drivers/net/octeontx/octeontx_ethdev.c b/drivers/net/octeontx/octeontx_ethdev.c
index 76c69294d0..946844ca0b 100644
--- a/drivers/net/octeontx/octeontx_ethdev.c
+++ b/drivers/net/octeontx/octeontx_ethdev.c
@@ -478,7 +478,7 @@ octeontx_dev_configure(struct rte_eth_dev *dev)
 	return 0;
 }
 
-static void
+static int
 octeontx_dev_close(struct rte_eth_dev *dev)
 {
 	struct octeontx_txq *txq = NULL;
@@ -517,6 +517,8 @@ octeontx_dev_close(struct rte_eth_dev *dev)
 
 	dev->tx_pkt_burst = NULL;
 	dev->rx_pkt_burst = NULL;
+
+	return 0;
 }
 
 static int
diff --git a/drivers/net/octeontx2/otx2_ethdev.c b/drivers/net/octeontx2/otx2_ethdev.c
index 7425ee55be..d9bcf004c6 100644
--- a/drivers/net/octeontx2/otx2_ethdev.c
+++ b/drivers/net/octeontx2/otx2_ethdev.c
@@ -2222,7 +2222,7 @@ otx2_nix_dev_start(struct rte_eth_dev *eth_dev)
 }
 
 static int otx2_nix_dev_reset(struct rte_eth_dev *eth_dev);
-static void otx2_nix_dev_close(struct rte_eth_dev *eth_dev);
+static int otx2_nix_dev_close(struct rte_eth_dev *eth_dev);
 
 /* Initialize and register driver with DPDK Application */
 static const struct eth_dev_ops otx2_eth_dev_ops = {
@@ -2635,10 +2635,11 @@ otx2_eth_dev_uninit(struct rte_eth_dev *eth_dev, bool mbox_close)
 	return 0;
 }
 
-static void
+static int
 otx2_nix_dev_close(struct rte_eth_dev *eth_dev)
 {
 	otx2_eth_dev_uninit(eth_dev, true);
+	return 0;
 }
 
 static int
diff --git a/drivers/net/pcap/rte_eth_pcap.c b/drivers/net/pcap/rte_eth_pcap.c
index 668cbd1fc7..76e704a65a 100644
--- a/drivers/net/pcap/rte_eth_pcap.c
+++ b/drivers/net/pcap/rte_eth_pcap.c
@@ -728,7 +728,7 @@ eth_stats_reset(struct rte_eth_dev *dev)
 	return 0;
 }
 
-static void
+static int
 eth_dev_close(struct rte_eth_dev *dev)
 {
 	unsigned int i;
@@ -748,6 +748,7 @@ eth_dev_close(struct rte_eth_dev *dev)
 		}
 	}
 
+	return 0;
 }
 
 static void
diff --git a/drivers/net/pfe/pfe_ethdev.c b/drivers/net/pfe/pfe_ethdev.c
index 368acfa554..8867b61a9d 100644
--- a/drivers/net/pfe/pfe_ethdev.c
+++ b/drivers/net/pfe/pfe_ethdev.c
@@ -398,14 +398,14 @@ pfe_eth_exit(struct rte_eth_dev *dev, struct pfe *pfe)
 	pfe->nb_devs--;
 }
 
-static void
+static int
 pfe_eth_close(struct rte_eth_dev *dev)
 {
 	if (!dev)
-		return;
+		return -1;
 
 	if (!g_pfe)
-		return;
+		return -1;
 
 	pfe_eth_exit(dev, g_pfe);
 
@@ -415,6 +415,8 @@ pfe_eth_close(struct rte_eth_dev *dev)
 		rte_free(g_pfe);
 		g_pfe = NULL;
 	}
+
+	return 0;
 }
 
 static int
diff --git a/drivers/net/qede/qede_ethdev.c b/drivers/net/qede/qede_ethdev.c
index 70d48e48ef..7dc5fd1565 100644
--- a/drivers/net/qede/qede_ethdev.c
+++ b/drivers/net/qede/qede_ethdev.c
@@ -1542,7 +1542,7 @@ static void qede_poll_sp_sb_cb(void *param)
 	}
 }
 
-static void qede_dev_close(struct rte_eth_dev *eth_dev)
+static int qede_dev_close(struct rte_eth_dev *eth_dev)
 {
 	struct rte_pci_device *pci_dev = RTE_ETH_DEV_TO_PCI(eth_dev);
 	struct qede_dev *qdev = QEDE_INIT_QDEV(eth_dev);
@@ -1586,6 +1586,8 @@ static void qede_dev_close(struct rte_eth_dev *eth_dev)
 
 	if (ECORE_IS_CMT(edev))
 		rte_eal_alarm_cancel(qede_poll_sp_sb_cb, (void *)eth_dev);
+
+	return 0;
 }
 
 static int
diff --git a/drivers/net/sfc/sfc_ethdev.c b/drivers/net/sfc/sfc_ethdev.c
index acee3e48e4..f1b857358c 100644
--- a/drivers/net/sfc/sfc_ethdev.c
+++ b/drivers/net/sfc/sfc_ethdev.c
@@ -318,7 +318,7 @@ sfc_dev_set_link_down(struct rte_eth_dev *dev)
 	return 0;
 }
 
-static void
+static int
 sfc_dev_close(struct rte_eth_dev *dev)
 {
 	struct sfc_adapter *sa = sfc_adapter_by_eth_dev(dev);
@@ -364,6 +364,8 @@ sfc_dev_close(struct rte_eth_dev *dev)
 
 	dev->process_private = NULL;
 	free(sa);
+
+	return 0;
 }
 
 static int
diff --git a/drivers/net/softnic/rte_eth_softnic.c b/drivers/net/softnic/rte_eth_softnic.c
index 866d17a93c..491a308c11 100644
--- a/drivers/net/softnic/rte_eth_softnic.c
+++ b/drivers/net/softnic/rte_eth_softnic.c
@@ -201,10 +201,10 @@ pmd_dev_stop(struct rte_eth_dev *dev)
 	softnic_mtr_free(p);
 }
 
-static void
+static int
 pmd_dev_close(struct rte_eth_dev *dev __rte_unused)
 {
-	return;
+	return 0;
 }
 
 static int
diff --git a/drivers/net/szedata2/rte_eth_szedata2.c b/drivers/net/szedata2/rte_eth_szedata2.c
index a17c53577c..daa40cfd6c 100644
--- a/drivers/net/szedata2/rte_eth_szedata2.c
+++ b/drivers/net/szedata2/rte_eth_szedata2.c
@@ -1155,7 +1155,7 @@ eth_tx_queue_release(void *q)
 	}
 }
 
-static void
+static int
 eth_dev_close(struct rte_eth_dev *dev)
 {
 	struct pmd_internals *internals = dev->data->dev_private;
@@ -1180,6 +1180,8 @@ eth_dev_close(struct rte_eth_dev *dev)
 
 	rte_free(dev->data->mac_addrs);
 	dev->data->mac_addrs = NULL;
+
+	return 0;
 }
 
 static int
diff --git a/drivers/net/tap/rte_eth_tap.c b/drivers/net/tap/rte_eth_tap.c
index 339f24bf82..6bce90c531 100644
--- a/drivers/net/tap/rte_eth_tap.c
+++ b/drivers/net/tap/rte_eth_tap.c
@@ -1032,7 +1032,7 @@ tap_stats_reset(struct rte_eth_dev *dev)
 	return 0;
 }
 
-static void
+static int
 tap_dev_close(struct rte_eth_dev *dev)
 {
 	int i;
@@ -1078,6 +1078,8 @@ tap_dev_close(struct rte_eth_dev *dev)
 	 * Since TUN device has no more opened file descriptors
 	 * it will be removed from kernel
 	 */
+
+	return 0;
 }
 
 static void
diff --git a/drivers/net/thunderx/nicvf_ethdev.c b/drivers/net/thunderx/nicvf_ethdev.c
index d955a7ee23..649fe5c6e6 100644
--- a/drivers/net/thunderx/nicvf_ethdev.c
+++ b/drivers/net/thunderx/nicvf_ethdev.c
@@ -1852,7 +1852,7 @@ nicvf_vf_stop(struct rte_eth_dev *dev, struct nicvf *nic, bool cleanup)
 	}
 }
 
-static void
+static int
 nicvf_dev_close(struct rte_eth_dev *dev)
 {
 	size_t i;
@@ -1869,6 +1869,8 @@ nicvf_dev_close(struct rte_eth_dev *dev)
 
 		nicvf_periodic_alarm_stop(nicvf_vf_interrupt, nic->snicvf[i]);
 	}
+
+	return 0;
 }
 
 static int
diff --git a/drivers/net/vhost/rte_eth_vhost.c b/drivers/net/vhost/rte_eth_vhost.c
index e55278af69..fb3b9ce043 100644
--- a/drivers/net/vhost/rte_eth_vhost.c
+++ b/drivers/net/vhost/rte_eth_vhost.c
@@ -1164,7 +1164,7 @@ eth_dev_stop(struct rte_eth_dev *dev)
 	update_queuing_status(dev);
 }
 
-static void
+static int
 eth_dev_close(struct rte_eth_dev *dev)
 {
 	struct pmd_internal *internal;
@@ -1173,7 +1173,7 @@ eth_dev_close(struct rte_eth_dev *dev)
 
 	internal = dev->data->dev_private;
 	if (!internal)
-		return;
+		return 0;
 
 	eth_dev_stop(dev);
 
@@ -1201,6 +1201,8 @@ eth_dev_close(struct rte_eth_dev *dev)
 
 	rte_free(vring_states[dev->data->port_id]);
 	vring_states[dev->data->port_id] = NULL;
+
+	return 0;
 }
 
 static int
diff --git a/drivers/net/virtio/virtio_ethdev.c b/drivers/net/virtio/virtio_ethdev.c
index dc0093bdf0..1731fc6750 100644
--- a/drivers/net/virtio/virtio_ethdev.c
+++ b/drivers/net/virtio/virtio_ethdev.c
@@ -704,7 +704,7 @@ virtio_alloc_queues(struct rte_eth_dev *dev)
 
 static void virtio_queues_unbind_intr(struct rte_eth_dev *dev);
 
-static void
+static int
 virtio_dev_close(struct rte_eth_dev *dev)
 {
 	struct virtio_hw *hw = dev->data->dev_private;
@@ -713,7 +713,7 @@ virtio_dev_close(struct rte_eth_dev *dev)
 	PMD_INIT_LOG(DEBUG, "virtio_dev_close");
 
 	if (!hw->opened)
-		return;
+		return 0;
 	hw->opened = false;
 
 	/* reset the NIC */
@@ -743,6 +743,8 @@ virtio_dev_close(struct rte_eth_dev *dev)
 		if (!hw->modern)
 			rte_pci_ioport_unmap(VTPCI_IO(hw));
 	}
+
+	return 0;
 }
 
 static int
diff --git a/drivers/net/vmxnet3/vmxnet3_ethdev.c b/drivers/net/vmxnet3/vmxnet3_ethdev.c
index 815fba2f55..4efb48350c 100644
--- a/drivers/net/vmxnet3/vmxnet3_ethdev.c
+++ b/drivers/net/vmxnet3/vmxnet3_ethdev.c
@@ -64,7 +64,7 @@ static int eth_vmxnet3_dev_uninit(struct rte_eth_dev *eth_dev);
 static int vmxnet3_dev_configure(struct rte_eth_dev *dev);
 static int vmxnet3_dev_start(struct rte_eth_dev *dev);
 static void vmxnet3_dev_stop(struct rte_eth_dev *dev);
-static void vmxnet3_dev_close(struct rte_eth_dev *dev);
+static int vmxnet3_dev_close(struct rte_eth_dev *dev);
 static void vmxnet3_dev_set_rxmode(struct vmxnet3_hw *hw, uint32_t feature, int set);
 static int vmxnet3_dev_promiscuous_enable(struct rte_eth_dev *dev);
 static int vmxnet3_dev_promiscuous_disable(struct rte_eth_dev *dev);
@@ -888,13 +888,15 @@ vmxnet3_free_queues(struct rte_eth_dev *dev)
 /*
  * Reset and stop device.
  */
-static void
+static int
 vmxnet3_dev_close(struct rte_eth_dev *dev)
 {
 	PMD_INIT_FUNC_TRACE();
 
 	vmxnet3_dev_stop(dev);
 	vmxnet3_free_queues(dev);
+
+	return 0;
 }
 
 static void
diff --git a/lib/librte_ethdev/rte_ethdev_core.h b/lib/librte_ethdev/rte_ethdev_core.h
index 32407dd418..76308106bd 100644
--- a/lib/librte_ethdev/rte_ethdev_core.h
+++ b/lib/librte_ethdev/rte_ethdev_core.h
@@ -43,7 +43,7 @@ typedef int  (*eth_dev_set_link_up_t)(struct rte_eth_dev *dev);
 typedef int  (*eth_dev_set_link_down_t)(struct rte_eth_dev *dev);
 /**< @internal Function used to link down a configured Ethernet device. */
 
-typedef void (*eth_dev_close_t)(struct rte_eth_dev *dev);
+typedef int (*eth_dev_close_t)(struct rte_eth_dev *dev);
 /**< @internal Function used to close a configured Ethernet device. */
 
 typedef int (*eth_dev_reset_t)(struct rte_eth_dev *dev);
-- 
2.28.0

Re: [dpdk-dev] [PATCH 02/20] ethdev: allow drivers to return error on close

[Ferruh Yigit]

On 9/13/2020 11:06 PM, Thomas Monjalon wrote:
> The device operation .dev_close was returning void.
> This driver interface is changed to return an int.
> 
> Note that the API rte_eth_dev_close() is still returning void,
> although a deprecation notice is pending to change it as well.

Why not update the 'rte_eth_dev_close()' in this patch? After the 
updates below it should be easier.

> 
> Signed-off-by: Thomas Monjalon <thomas@monjalon.net>

<...>

Re: [dpdk-dev] [PATCH 02/20] ethdev: allow drivers to return error on close

[Thomas Monjalon]

23/09/2020 18:41, Ferruh Yigit:
> On 9/13/2020 11:06 PM, Thomas Monjalon wrote:
> > The device operation .dev_close was returning void.
> > This driver interface is changed to return an int.
> > 
> > Note that the API rte_eth_dev_close() is still returning void,
> > although a deprecation notice is pending to change it as well.
> 
> Why not update the 'rte_eth_dev_close()' in this patch? After the 
> updates below it should be easier.

Yes it is a step in the right direction.
Changing the API means adding some error checks in every apps.
The reason for not doing is that I did not commit for this task,
and I am not the author of the deprecation notice.
We can discuss how to follow up once this series is merged.

Re: [dpdk-dev] [PATCH 02/20] ethdev: allow drivers to return error on close

[Stephen Hemminger]

On Wed, 23 Sep 2020 22:53:46 +0200
Thomas Monjalon <thomas@monjalon.net> wrote:

> 23/09/2020 18:41, Ferruh Yigit:
> > On 9/13/2020 11:06 PM, Thomas Monjalon wrote:  
> > > The device operation .dev_close was returning void.
> > > This driver interface is changed to return an int.
> > > 
> > > Note that the API rte_eth_dev_close() is still returning void,
> > > although a deprecation notice is pending to change it as well.  
> > 
> > Why not update the 'rte_eth_dev_close()' in this patch? After the 
> > updates below it should be easier.  
> 
> Yes it is a step in the right direction.
> Changing the API means adding some error checks in every apps.
> The reason for not doing is that I did not commit for this task,
> and I am not the author of the deprecation notice.
> We can discuss how to follow up once this series is merged.

Not sure what application can really do if dev_close fails other than call rte_exit()?

Re: [dpdk-dev] [PATCH 02/20] ethdev: allow drivers to return error on close

[Thomas Monjalon]

23/09/2020 23:02, Stephen Hemminger:
> On Wed, 23 Sep 2020 22:53:46 +0200
> Thomas Monjalon <thomas@monjalon.net> wrote:
> > 23/09/2020 18:41, Ferruh Yigit:
> > > On 9/13/2020 11:06 PM, Thomas Monjalon wrote:  
> > > > The device operation .dev_close was returning void.
> > > > This driver interface is changed to return an int.
> > > > 
> > > > Note that the API rte_eth_dev_close() is still returning void,
> > > > although a deprecation notice is pending to change it as well.  
> > > 
> > > Why not update the 'rte_eth_dev_close()' in this patch? After the 
> > > updates below it should be easier.  
> > 
> > Yes it is a step in the right direction.
> > Changing the API means adding some error checks in every apps.
> > The reason for not doing is that I did not commit for this task,
> > and I am not the author of the deprecation notice.
> > We can discuss how to follow up once this series is merged.
> 
> Not sure what application can really do if dev_close fails other than call rte_exit()?

At least, it can show a message to the user.

Re: [dpdk-dev] [PATCH 02/20] ethdev: allow drivers to return error on close

[Stephen Hemminger]

On Wed, 23 Sep 2020 23:06:42 +0200
Thomas Monjalon <thomas@monjalon.net> wrote:

> 23/09/2020 23:02, Stephen Hemminger:
> > On Wed, 23 Sep 2020 22:53:46 +0200
> > Thomas Monjalon <thomas@monjalon.net> wrote:  
> > > 23/09/2020 18:41, Ferruh Yigit:  
> > > > On 9/13/2020 11:06 PM, Thomas Monjalon wrote:    
> > > > > The device operation .dev_close was returning void.
> > > > > This driver interface is changed to return an int.
> > > > > 
> > > > > Note that the API rte_eth_dev_close() is still returning void,
> > > > > although a deprecation notice is pending to change it as well.    
> > > > 
> > > > Why not update the 'rte_eth_dev_close()' in this patch? After the 
> > > > updates below it should be easier.    
> > > 
> > > Yes it is a step in the right direction.
> > > Changing the API means adding some error checks in every apps.
> > > The reason for not doing is that I did not commit for this task,
> > > and I am not the author of the deprecation notice.
> > > We can discuss how to follow up once this series is merged.  
> > 
> > Not sure what application can really do if dev_close fails other than call rte_exit()?  
> 
> At least, it can show a message to the user.

Agree, but no code checks return from close() system call now.
At least testpmd should be updated; the examples are fine.

Re: [dpdk-dev] [PATCH 02/20] ethdev: allow drivers to return error on close

[Thomas Monjalon]

23/09/2020 23:47, Stephen Hemminger:
> On Wed, 23 Sep 2020 23:06:42 +0200
> Thomas Monjalon <thomas@monjalon.net> wrote:
> 
> > 23/09/2020 23:02, Stephen Hemminger:
> > > On Wed, 23 Sep 2020 22:53:46 +0200
> > > Thomas Monjalon <thomas@monjalon.net> wrote:  
> > > > 23/09/2020 18:41, Ferruh Yigit:  
> > > > > On 9/13/2020 11:06 PM, Thomas Monjalon wrote:    
> > > > > > The device operation .dev_close was returning void.
> > > > > > This driver interface is changed to return an int.
> > > > > > 
> > > > > > Note that the API rte_eth_dev_close() is still returning void,
> > > > > > although a deprecation notice is pending to change it as well.    
> > > > > 
> > > > > Why not update the 'rte_eth_dev_close()' in this patch? After the 
> > > > > updates below it should be easier.    
> > > > 
> > > > Yes it is a step in the right direction.
> > > > Changing the API means adding some error checks in every apps.
> > > > The reason for not doing is that I did not commit for this task,
> > > > and I am not the author of the deprecation notice.
> > > > We can discuss how to follow up once this series is merged.  
> > > 
> > > Not sure what application can really do if dev_close fails other than call rte_exit()?  
> > 
> > At least, it can show a message to the user.
> 
> Agree, but no code checks return from close() system call now.
> At least testpmd should be updated; the examples are fine.

How do you explicitly silence the static code analyzers?
Do you recommend adding (void) in front of the call to close()?

Re: [dpdk-dev] [PATCH 02/20] ethdev: allow drivers to return error on close

[Stephen Hemminger]

On Wed, 23 Sep 2020 23:52:07 +0200
Thomas Monjalon <thomas@monjalon.net> wrote:

> > > > > Yes it is a step in the right direction.
> > > > > Changing the API means adding some error checks in every apps.
> > > > > The reason for not doing is that I did not commit for this task,
> > > > > and I am not the author of the deprecation notice.
> > > > > We can discuss how to follow up once this series is merged.    
> > > > 
> > > > Not sure what application can really do if dev_close fails other than call rte_exit()?    
> > > 
> > > At least, it can show a message to the user.  
> > 
> > Agree, but no code checks return from close() system call now.
> > At least testpmd should be updated; the examples are fine.  
> 
> How do you explicitly silence the static code analyzers?
> Do you recommend adding (void) in front of the call to close()?

Coverity usually gives something like, "return value of foo() is not checked
but checked 10 other places"

Re: [dpdk-dev] [PATCH 02/20] ethdev: allow drivers to return error on close

[Thomas Monjalon]

24/09/2020 00:02, Stephen Hemminger:
> On Wed, 23 Sep 2020 23:52:07 +0200
> Thomas Monjalon <thomas@monjalon.net> wrote:
> 
> > > > > > Yes it is a step in the right direction.
> > > > > > Changing the API means adding some error checks in every apps.
> > > > > > The reason for not doing is that I did not commit for this task,
> > > > > > and I am not the author of the deprecation notice.
> > > > > > We can discuss how to follow up once this series is merged.    
> > > > > 
> > > > > Not sure what application can really do if dev_close fails other than call rte_exit()?    
> > > > 
> > > > At least, it can show a message to the user.  
> > > 
> > > Agree, but no code checks return from close() system call now.
> > > At least testpmd should be updated; the examples are fine.  
> > 
> > How do you explicitly silence the static code analyzers?
> > Do you recommend adding (void) in front of the call to close()?
> 
> Coverity usually gives something like, "return value of foo() is not checked
> but checked 10 other places"

Yes so it should be silenced with (void) or ret =

Re: [dpdk-dev] [PATCH 02/20] ethdev: allow drivers to return error on close

[Bruce Richardson]

On Thu, Sep 24, 2020 at 12:35:58AM +0200, Thomas Monjalon wrote:
> 24/09/2020 00:02, Stephen Hemminger:
> > On Wed, 23 Sep 2020 23:52:07 +0200
> > Thomas Monjalon <thomas@monjalon.net> wrote:
> > 
> > > > > > > Yes it is a step in the right direction.
> > > > > > > Changing the API means adding some error checks in every apps.
> > > > > > > The reason for not doing is that I did not commit for this task,
> > > > > > > and I am not the author of the deprecation notice.
> > > > > > > We can discuss how to follow up once this series is merged.    
> > > > > > 
> > > > > > Not sure what application can really do if dev_close fails other than call rte_exit()?    
> > > > > 
> > > > > At least, it can show a message to the user.  
> > > > 
> > > > Agree, but no code checks return from close() system call now.
> > > > At least testpmd should be updated; the examples are fine.  
> > > 
> > > How do you explicitly silence the static code analyzers?
> > > Do you recommend adding (void) in front of the call to close()?
> > 
> > Coverity usually gives something like, "return value of foo() is not checked
> > but checked 10 other places"
> 
> Yes so it should be silenced with (void) or ret =
> 
Does coverity still give the warning if the majority of cases don't
actually check the return value?

Re: [dpdk-dev] [PATCH 02/20] ethdev: allow drivers to return error on close

[Thomas Monjalon]

24/09/2020 11:12, Bruce Richardson:
> On Thu, Sep 24, 2020 at 12:35:58AM +0200, Thomas Monjalon wrote:
> > 24/09/2020 00:02, Stephen Hemminger:
> > > On Wed, 23 Sep 2020 23:52:07 +0200
> > > Thomas Monjalon <thomas@monjalon.net> wrote:
> > > 
> > > > > > > > Yes it is a step in the right direction.
> > > > > > > > Changing the API means adding some error checks in every apps.
> > > > > > > > The reason for not doing is that I did not commit for this task,
> > > > > > > > and I am not the author of the deprecation notice.
> > > > > > > > We can discuss how to follow up once this series is merged.    
> > > > > > > 
> > > > > > > Not sure what application can really do if dev_close fails other than call rte_exit()?    
> > > > > > 
> > > > > > At least, it can show a message to the user.  
> > > > > 
> > > > > Agree, but no code checks return from close() system call now.
> > > > > At least testpmd should be updated; the examples are fine.  
> > > > 
> > > > How do you explicitly silence the static code analyzers?
> > > > Do you recommend adding (void) in front of the call to close()?
> > > 
> > > Coverity usually gives something like, "return value of foo() is not checked
> > > but checked 10 other places"
> > 
> > Yes so it should be silenced with (void) or ret =
> > 
> Does coverity still give the warning if the majority of cases don't
> actually check the return value?

No idea

Re: [dpdk-dev] [PATCH 02/20] ethdev: allow drivers to return error on close

[Ferruh Yigit]

On 9/24/2020 11:07 AM, Thomas Monjalon wrote:
> 24/09/2020 11:12, Bruce Richardson:
>> On Thu, Sep 24, 2020 at 12:35:58AM +0200, Thomas Monjalon wrote:
>>> 24/09/2020 00:02, Stephen Hemminger:
>>>> On Wed, 23 Sep 2020 23:52:07 +0200
>>>> Thomas Monjalon <thomas@monjalon.net> wrote:
>>>>
>>>>>>>>> Yes it is a step in the right direction.
>>>>>>>>> Changing the API means adding some error checks in every apps.
>>>>>>>>> The reason for not doing is that I did not commit for this task,
>>>>>>>>> and I am not the author of the deprecation notice.
>>>>>>>>> We can discuss how to follow up once this series is merged.
>>>>>>>>
>>>>>>>> Not sure what application can really do if dev_close fails other than call rte_exit()?
>>>>>>>
>>>>>>> At least, it can show a message to the user.
>>>>>>
>>>>>> Agree, but no code checks return from close() system call now.
>>>>>> At least testpmd should be updated; the examples are fine.
>>>>>
>>>>> How do you explicitly silence the static code analyzers?
>>>>> Do you recommend adding (void) in front of the call to close()?
>>>>
>>>> Coverity usually gives something like, "return value of foo() is not checked
>>>> but checked 10 other places"
>>>
>>> Yes so it should be silenced with (void) or ret =
>>>
>> Does coverity still give the warning if the majority of cases don't
>> actually check the return value?
> 
> No idea
> 

As far as I know it does, so just assigning return value to a variable 
and not checking it at all will generate warnings.

Re: [dpdk-dev] [PATCH 02/20] ethdev: allow drivers to return error on close

[Stephen Hemminger]

On Thu, 24 Sep 2020 10:12:46 +0100
Bruce Richardson <bruce.richardson@intel.com> wrote:

> On Thu, Sep 24, 2020 at 12:35:58AM +0200, Thomas Monjalon wrote:
> > 24/09/2020 00:02, Stephen Hemminger:  
> > > On Wed, 23 Sep 2020 23:52:07 +0200
> > > Thomas Monjalon <thomas@monjalon.net> wrote:
> > >   
> > > > > > > > Yes it is a step in the right direction.
> > > > > > > > Changing the API means adding some error checks in every apps.
> > > > > > > > The reason for not doing is that I did not commit for this task,
> > > > > > > > and I am not the author of the deprecation notice.
> > > > > > > > We can discuss how to follow up once this series is merged.      
> > > > > > > 
> > > > > > > Not sure what application can really do if dev_close fails other than call rte_exit()?      
> > > > > > 
> > > > > > At least, it can show a message to the user.    
> > > > > 
> > > > > Agree, but no code checks return from close() system call now.
> > > > > At least testpmd should be updated; the examples are fine.    
> > > > 
> > > > How do you explicitly silence the static code analyzers?
> > > > Do you recommend adding (void) in front of the call to close()?  
> > > 
> > > Coverity usually gives something like, "return value of foo() is not checked
> > > but checked 10 other places"  
> > 
> > Yes so it should be silenced with (void) or ret =
> >   
> Does coverity still give the warning if the majority of cases don't
> actually check the return value?

It looks like coverity only looks at some functions and if majority is checking.
For example, it never nags about just calling close().

[dpdk-dev] [PATCH 03/20] net/af_packet: release port upon close

[Thomas Monjalon]

The flag RTE_ETH_DEV_CLOSE_REMOVE is set so all port resources
can be freed by rte_eth_dev_close().

Freeing of private port resources is moved
from the ".remove(device)" to the ".dev_close(port)" operation.

Signed-off-by: Thomas Monjalon <thomas@monjalon.net>
---
 drivers/net/af_packet/rte_eth_af_packet.c | 56 ++++++++++++-----------
 1 file changed, 29 insertions(+), 27 deletions(-)

diff --git a/drivers/net/af_packet/rte_eth_af_packet.c b/drivers/net/af_packet/rte_eth_af_packet.c
index 7d0ff1cbb3..3ab6f4d388 100644
--- a/drivers/net/af_packet/rte_eth_af_packet.c
+++ b/drivers/net/af_packet/rte_eth_af_packet.c
@@ -377,8 +377,32 @@ eth_stats_reset(struct rte_eth_dev *dev)
 }
 
 static int
-eth_dev_close(struct rte_eth_dev *dev __rte_unused)
+eth_dev_close(struct rte_eth_dev *dev)
 {
+	struct pmd_internals *internals;
+	struct tpacket_req *req;
+	unsigned q;
+
+	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
+		return 0;
+
+	PMD_LOG(INFO, "Closing AF_PACKET ethdev on NUMA socket %u",
+		rte_socket_id());
+
+	internals = dev->data->dev_private;
+	req = &internals->req;
+	for (q = 0; q < internals->nb_queues; q++) {
+		munmap(internals->rx_queue[q].map,
+			2 * req->tp_block_size * req->tp_block_nr);
+		rte_free(internals->rx_queue[q].rd);
+		rte_free(internals->tx_queue[q].rd);
+	}
+	free(internals->if_name);
+	rte_free(internals->rx_queue);
+	rte_free(internals->tx_queue);
+
+	/* mac_addrs must not be freed alone because part of dev_private */
+	dev->data->mac_addrs = NULL;
 	return 0;
 }
 
@@ -835,6 +859,7 @@ rte_pmd_init_internals(struct rte_vdev_device *dev,
 	data->nb_tx_queues = (uint16_t)nb_queues;
 	data->dev_link = pmd_link;
 	data->mac_addrs = &(*internals)->eth_addr;
+	data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
 
 	(*eth_dev)->dev_ops = &ops;
 
@@ -1033,13 +1058,7 @@ rte_pmd_af_packet_probe(struct rte_vdev_device *dev)
 static int
 rte_pmd_af_packet_remove(struct rte_vdev_device *dev)
 {
-	struct rte_eth_dev *eth_dev = NULL;
-	struct pmd_internals *internals;
-	struct tpacket_req *req;
-	unsigned q;
-
-	PMD_LOG(INFO, "Closing AF_PACKET ethdev on numa socket %u",
-		rte_socket_id());
+	struct rte_eth_dev *eth_dev;
 
 	if (dev == NULL)
 		return -1;
@@ -1047,26 +1066,9 @@ rte_pmd_af_packet_remove(struct rte_vdev_device *dev)
 	/* find the ethdev entry */
 	eth_dev = rte_eth_dev_allocated(rte_vdev_device_name(dev));
 	if (eth_dev == NULL)
-		return -1;
-
-	/* mac_addrs must not be freed alone because part of dev_private */
-	eth_dev->data->mac_addrs = NULL;
-
-	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
-		return rte_eth_dev_release_port(eth_dev);
-
-	internals = eth_dev->data->dev_private;
-	req = &internals->req;
-	for (q = 0; q < internals->nb_queues; q++) {
-		munmap(internals->rx_queue[q].map,
-			2 * req->tp_block_size * req->tp_block_nr);
-		rte_free(internals->rx_queue[q].rd);
-		rte_free(internals->tx_queue[q].rd);
-	}
-	free(internals->if_name);
-	rte_free(internals->rx_queue);
-	rte_free(internals->tx_queue);
+		return 0; /* port already released */
 
+	eth_dev_close(eth_dev);
 	rte_eth_dev_release_port(eth_dev);
 
 	return 0;
-- 
2.28.0

Re: [dpdk-dev] [PATCH 03/20] net/af_packet: release port upon close

[Ferruh Yigit]

On 9/13/2020 11:06 PM, Thomas Monjalon wrote:
> The flag RTE_ETH_DEV_CLOSE_REMOVE is set so all port resources
> can be freed by rte_eth_dev_close().
> 
> Freeing of private port resources is moved
> from the ".remove(device)" to the ".dev_close(port)" operation.
> 
> Signed-off-by: Thomas Monjalon <thomas@monjalon.net>

Reviewed-by: Ferruh Yigit <ferruh.yigit@intel.com>

[dpdk-dev] [PATCH 04/20] net/atlantic: release port upon close

[Thomas Monjalon]

The flag RTE_ETH_DEV_CLOSE_REMOVE is set so all port resources
can be freed by rte_eth_dev_close().

Freeing of private port resources is moved
from the ".remove(device)" to the ".dev_close(port)" operation.

Signed-off-by: Thomas Monjalon <thomas@monjalon.net>
---
 drivers/net/atlantic/atl_ethdev.c | 62 ++++++++++++-------------------
 1 file changed, 24 insertions(+), 38 deletions(-)

diff --git a/drivers/net/atlantic/atl_ethdev.c b/drivers/net/atlantic/atl_ethdev.c
index 1d76883c52..16721a011f 100644
--- a/drivers/net/atlantic/atl_ethdev.c
+++ b/drivers/net/atlantic/atl_ethdev.c
@@ -15,8 +15,6 @@
 #include "hw_atl/hw_atl_b0_internal.h"
 
 static int eth_atl_dev_init(struct rte_eth_dev *eth_dev);
-static int eth_atl_dev_uninit(struct rte_eth_dev *eth_dev);
-
 static int  atl_dev_configure(struct rte_eth_dev *dev);
 static int  atl_dev_start(struct rte_eth_dev *dev);
 static void atl_dev_stop(struct rte_eth_dev *dev);
@@ -381,6 +379,8 @@ eth_atl_dev_init(struct rte_eth_dev *eth_dev)
 	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
 		return 0;
 
+	eth_dev->data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
+
 	/* Vendor and Device ID need to be set before init of shared code */
 	hw->device_id = pci_dev->id.device_id;
 	hw->vendor_id = pci_dev->id.vendor_id;
@@ -441,40 +441,6 @@ eth_atl_dev_init(struct rte_eth_dev *eth_dev)
 	return err;
 }
 
-static int
-eth_atl_dev_uninit(struct rte_eth_dev *eth_dev)
-{
-	struct rte_pci_device *pci_dev = RTE_ETH_DEV_TO_PCI(eth_dev);
-	struct rte_intr_handle *intr_handle = &pci_dev->intr_handle;
-	struct aq_hw_s *hw;
-
-	PMD_INIT_FUNC_TRACE();
-
-	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
-		return -EPERM;
-
-	hw = ATL_DEV_PRIVATE_TO_HW(eth_dev->data->dev_private);
-
-	if (hw->adapter_stopped == 0)
-		atl_dev_close(eth_dev);
-
-	eth_dev->dev_ops = NULL;
-	eth_dev->rx_pkt_burst = NULL;
-	eth_dev->tx_pkt_burst = NULL;
-
-	/* disable uio intr before callback unregister */
-	rte_intr_disable(intr_handle);
-	rte_intr_callback_unregister(intr_handle,
-				     atl_dev_interrupt_handler, eth_dev);
-
-	rte_free(eth_dev->data->mac_addrs);
-	eth_dev->data->mac_addrs = NULL;
-
-	pthread_mutex_destroy(&hw->mbox_mutex);
-
-	return 0;
-}
-
 static int
 eth_atl_pci_probe(struct rte_pci_driver *pci_drv __rte_unused,
 	struct rte_pci_device *pci_dev)
@@ -486,7 +452,7 @@ eth_atl_pci_probe(struct rte_pci_driver *pci_drv __rte_unused,
 static int
 eth_atl_pci_remove(struct rte_pci_device *pci_dev)
 {
-	return rte_eth_dev_pci_generic_remove(pci_dev, eth_atl_dev_uninit);
+	return rte_eth_dev_pci_generic_remove(pci_dev, atl_dev_close);
 }
 
 static int
@@ -721,12 +687,32 @@ atl_dev_set_link_down(struct rte_eth_dev *dev)
 static int
 atl_dev_close(struct rte_eth_dev *dev)
 {
+	struct rte_pci_device *pci_dev = RTE_ETH_DEV_TO_PCI(dev);
+	struct rte_intr_handle *intr_handle = &pci_dev->intr_handle;
+	struct aq_hw_s *hw;
+
 	PMD_INIT_FUNC_TRACE();
 
+	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
+		return -EPERM;
+
+	hw = ATL_DEV_PRIVATE_TO_HW(dev->data->dev_private);
+
 	atl_dev_stop(dev);
 
 	atl_free_queues(dev);
 
+	dev->dev_ops = NULL;
+	dev->rx_pkt_burst = NULL;
+	dev->tx_pkt_burst = NULL;
+
+	/* disable uio intr before callback unregister */
+	rte_intr_disable(intr_handle);
+	rte_intr_callback_unregister(intr_handle,
+				     atl_dev_interrupt_handler, dev);
+
+	pthread_mutex_destroy(&hw->mbox_mutex);
+
 	return 0;
 }
 
@@ -735,7 +721,7 @@ atl_dev_reset(struct rte_eth_dev *dev)
 {
 	int ret;
 
-	ret = eth_atl_dev_uninit(dev);
+	ret = atl_dev_close(dev);
 	if (ret)
 		return ret;
 
-- 
2.28.0

Re: [dpdk-dev] [PATCH 04/20] net/atlantic: release port upon close

[Igor Russkikh]

On 14/09/2020 1:06 am, Thomas Monjalon wrote:
> The flag RTE_ETH_DEV_CLOSE_REMOVE is set so all port resources
> can be freed by rte_eth_dev_close().
> 
> Freeing of private port resources is moved
> from the ".remove(device)" to the ".dev_close(port)" operation.
> 
> Signed-off-by: Thomas Monjalon <thomas@monjalon.net>

Thanks,

Acked-by: Igor Russkikh <irusskikh@marvell.com>

I'll verify this soon on HW.

Igor

Re: [dpdk-dev] [PATCH 04/20] net/atlantic: release port upon close

[Ferruh Yigit]

On 9/13/2020 11:06 PM, Thomas Monjalon wrote:
> The flag RTE_ETH_DEV_CLOSE_REMOVE is set so all port resources
> can be freed by rte_eth_dev_close().
> 
> Freeing of private port resources is moved
> from the ".remove(device)" to the ".dev_close(port)" operation.
> 
> Signed-off-by: Thomas Monjalon <thomas@monjalon.net>

<...>

> @@ -721,12 +687,32 @@ atl_dev_set_link_down(struct rte_eth_dev *dev)
>   static int
>   atl_dev_close(struct rte_eth_dev *dev)
>   {
> +	struct rte_pci_device *pci_dev = RTE_ETH_DEV_TO_PCI(dev);
> +	struct rte_intr_handle *intr_handle = &pci_dev->intr_handle;
> +	struct aq_hw_s *hw;
> +
>   	PMD_INIT_FUNC_TRACE();
>   
> +	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
> +		return -EPERM;

Is this case an error, or should it return 0.

> +
> +	hw = ATL_DEV_PRIVATE_TO_HW(dev->data->dev_private);
> +
>   	atl_dev_stop(dev);
>   
>   	atl_free_queues(dev);
>   
> +	dev->dev_ops = NULL;
> +	dev->rx_pkt_burst = NULL;
> +	dev->tx_pkt_burst = NULL;

What do you think moving above cleanup to 'rte_eth_dev_release_port()'?

Re: [dpdk-dev] [PATCH 04/20] net/atlantic: release port upon close

[Thomas Monjalon]

23/09/2020 18:42, Ferruh Yigit:
> On 9/13/2020 11:06 PM, Thomas Monjalon wrote:
> > The flag RTE_ETH_DEV_CLOSE_REMOVE is set so all port resources
> > can be freed by rte_eth_dev_close().
> > 
> > Freeing of private port resources is moved
> > from the ".remove(device)" to the ".dev_close(port)" operation.
> > 
> > Signed-off-by: Thomas Monjalon <thomas@monjalon.net>
> 
> <...>
> 
> > @@ -721,12 +687,32 @@ atl_dev_set_link_down(struct rte_eth_dev *dev)
> >   static int
> >   atl_dev_close(struct rte_eth_dev *dev)
> >   {
> > +	struct rte_pci_device *pci_dev = RTE_ETH_DEV_TO_PCI(dev);
> > +	struct rte_intr_handle *intr_handle = &pci_dev->intr_handle;
> > +	struct aq_hw_s *hw;
> > +
> >   	PMD_INIT_FUNC_TRACE();
> >   
> > +	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
> > +		return -EPERM;
> 
> Is this case an error, or should it return 0.

In other drivers, I think I chose 0.

> > +	dev->dev_ops = NULL;
> > +	dev->rx_pkt_burst = NULL;
> > +	dev->tx_pkt_burst = NULL;
> 
> What do you think moving above cleanup to 'rte_eth_dev_release_port()'?

I asked myself the same question.
I think we should but I propose to do this change in a different series.

[dpdk-dev] [PATCH 05/20] net/axgbe: release port upon close

[Thomas Monjalon]

The flag RTE_ETH_DEV_CLOSE_REMOVE is set so all port resources
can be freed by rte_eth_dev_close().

Freeing of private port resources is moved
from the ".remove(device)" to the ".dev_close(port)" operation.
The ".dev_close" callback is also called as part of the ".remove" one.

Signed-off-by: Thomas Monjalon <thomas@monjalon.net>
---
 drivers/net/axgbe/axgbe_ethdev.c | 14 +++-----------
 1 file changed, 3 insertions(+), 11 deletions(-)

diff --git a/drivers/net/axgbe/axgbe_ethdev.c b/drivers/net/axgbe/axgbe_ethdev.c
index be6f7cbda6..c5e70ea0a1 100644
--- a/drivers/net/axgbe/axgbe_ethdev.c
+++ b/drivers/net/axgbe/axgbe_ethdev.c
@@ -10,7 +10,6 @@
 #include "axgbe_regs.h"
 
 static int eth_axgbe_dev_init(struct rte_eth_dev *eth_dev);
-static int eth_axgbe_dev_uninit(struct rte_eth_dev *eth_dev);
 static int  axgbe_dev_configure(struct rte_eth_dev *dev);
 static int  axgbe_dev_start(struct rte_eth_dev *dev);
 static void axgbe_dev_stop(struct rte_eth_dev *dev);
@@ -378,14 +377,6 @@ axgbe_dev_stop(struct rte_eth_dev *dev)
 	rte_bit_relaxed_set32(AXGBE_DOWN, &pdata->dev_state);
 }
 
-/* Clear all resources like TX/RX queues. */
-static int
-axgbe_dev_close(struct rte_eth_dev *dev)
-{
-	axgbe_dev_clear_queues(dev);
-	return 0;
-}
-
 static int
 axgbe_dev_promiscuous_enable(struct rte_eth_dev *dev)
 {
@@ -1632,6 +1623,7 @@ eth_axgbe_dev_init(struct rte_eth_dev *eth_dev)
 	int ret;
 
 	eth_dev->dev_ops = &axgbe_eth_dev_ops;
+	eth_dev->data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
 
 	/*
 	 * For secondary processes, we don't initialise any further as primary
@@ -1794,7 +1786,7 @@ eth_axgbe_dev_init(struct rte_eth_dev *eth_dev)
 }
 
 static int
-eth_axgbe_dev_uninit(struct rte_eth_dev *eth_dev)
+axgbe_dev_close(struct rte_eth_dev *eth_dev)
 {
 	struct rte_pci_device *pci_dev;
 
@@ -1827,7 +1819,7 @@ static int eth_axgbe_pci_probe(struct rte_pci_driver *pci_drv __rte_unused,
 
 static int eth_axgbe_pci_remove(struct rte_pci_device *pci_dev)
 {
-	return rte_eth_dev_pci_generic_remove(pci_dev, eth_axgbe_dev_uninit);
+	return rte_eth_dev_pci_generic_remove(pci_dev, axgbe_dev_close);
 }
 
 static struct rte_pci_driver rte_axgbe_pmd = {
-- 
2.28.0

[dpdk-dev] [PATCH 06/20] net/bonding: release port upon close

[Thomas Monjalon]

The flag RTE_ETH_DEV_CLOSE_REMOVE is set so all port resources
can be freed by rte_eth_dev_close().

Freeing of private port resources is moved
from the ".remove(device)" to the ".dev_close(port)" operation.

Signed-off-by: Thomas Monjalon <thomas@monjalon.net>
---
 drivers/net/bonding/rte_eth_bond_pmd.c | 34 ++++++++++++--------------
 1 file changed, 16 insertions(+), 18 deletions(-)

diff --git a/drivers/net/bonding/rte_eth_bond_pmd.c b/drivers/net/bonding/rte_eth_bond_pmd.c
index e1123deb83..726b47055b 100644
--- a/drivers/net/bonding/rte_eth_bond_pmd.c
+++ b/drivers/net/bonding/rte_eth_bond_pmd.c
@@ -2103,6 +2103,9 @@ bond_ethdev_close(struct rte_eth_dev *dev)
 	int skipped = 0;
 	struct rte_flow_error ferror;
 
+	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
+		return 0;
+
 	RTE_BOND_LOG(INFO, "Closing bonded device %s", dev->device->name);
 	while (internals->slave_count != skipped) {
 		uint16_t port_id = internals->slaves[skipped].port_id;
@@ -2119,6 +2122,17 @@ bond_ethdev_close(struct rte_eth_dev *dev)
 	bond_flow_ops.flush(dev, &ferror);
 	bond_ethdev_free_queues(dev);
 	rte_bitmap_reset(internals->vlan_filter_bmp);
+	rte_bitmap_free(internals->vlan_filter_bmp);
+	rte_free(internals->vlan_filter_bmpmem);
+
+	/* Try to release mempool used in mode6. If the bond
+	 * device is not mode6, free the NULL is not problem.
+	 */
+	rte_mempool_free(internals->mode6.mempool);
+
+	dev->dev_ops = NULL;
+	dev->rx_pkt_burst = NULL;
+	dev->tx_pkt_burst = NULL;
 
 	return 0;
 }
@@ -3195,6 +3209,7 @@ bond_alloc(struct rte_vdev_device *dev, uint8_t mode)
 	}
 
 	internals = eth_dev->data->dev_private;
+	eth_dev->data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
 	eth_dev->data->nb_rx_queues = (uint16_t)1;
 	eth_dev->data->nb_tx_queues = (uint16_t)1;
 
@@ -3414,14 +3429,10 @@ bond_remove(struct rte_vdev_device *dev)
 	name = rte_vdev_device_name(dev);
 	RTE_BOND_LOG(INFO, "Uninitializing pmd_bond for %s", name);
 
-	/* now free all data allocation - for eth_dev structure,
-	 * dummy pci driver and internal (private) data
-	 */
-
 	/* find an ethdev entry */
 	eth_dev = rte_eth_dev_allocated(name);
 	if (eth_dev == NULL)
-		return -ENODEV;
+		return 0; /* port already released */
 
 	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
 		return rte_eth_dev_release_port(eth_dev);
@@ -3436,19 +3447,6 @@ bond_remove(struct rte_vdev_device *dev)
 		bond_ethdev_stop(eth_dev);
 		bond_ethdev_close(eth_dev);
 	}
-
-	eth_dev->dev_ops = NULL;
-	eth_dev->rx_pkt_burst = NULL;
-	eth_dev->tx_pkt_burst = NULL;
-
-	internals = eth_dev->data->dev_private;
-	/* Try to release mempool used in mode6. If the bond
-	 * device is not mode6, free the NULL is not problem.
-	 */
-	rte_mempool_free(internals->mode6.mempool);
-	rte_bitmap_free(internals->vlan_filter_bmp);
-	rte_free(internals->vlan_filter_bmpmem);
-
 	rte_eth_dev_release_port(eth_dev);
 
 	return 0;
-- 
2.28.0

[dpdk-dev] [PATCH 07/20] net/failsafe: release port upon close

[Thomas Monjalon]

The flag RTE_ETH_DEV_CLOSE_REMOVE is set so all port resources
can be freed by rte_eth_dev_close().

Freeing of private port resources is moved
from the ".remove(device)" to the ".dev_close(port)" operation.

Signed-off-by: Thomas Monjalon <thomas@monjalon.net>
---
 drivers/net/failsafe/failsafe.c         | 25 ++--------
 drivers/net/failsafe/failsafe_ops.c     | 61 +++++++++++++++----------
 drivers/net/failsafe/failsafe_private.h |  1 +
 3 files changed, 42 insertions(+), 45 deletions(-)

diff --git a/drivers/net/failsafe/failsafe.c b/drivers/net/failsafe/failsafe.c
index 4a4b7ceab6..44d47e8f72 100644
--- a/drivers/net/failsafe/failsafe.c
+++ b/drivers/net/failsafe/failsafe.c
@@ -60,12 +60,6 @@ fs_sub_device_alloc(struct rte_eth_dev *dev,
 	return 0;
 }
 
-static void
-fs_sub_device_free(struct rte_eth_dev *dev)
-{
-	rte_free(PRIV(dev)->subs);
-}
-
 static void fs_hotplug_alarm(void *arg);
 
 int
@@ -186,6 +180,7 @@ fs_eth_dev_create(struct rte_vdev_device *vdev)
 		ERROR("Unable to allocate rte_eth_dev");
 		return -1;
 	}
+	dev->data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
 	priv = PRIV(dev);
 	priv->data = dev->data;
 	priv->rxp = FS_RX_PROXY_INIT;
@@ -285,7 +280,7 @@ fs_eth_dev_create(struct rte_vdev_device *vdev)
 free_args:
 	failsafe_args_free(dev);
 free_subs:
-	fs_sub_device_free(dev);
+	rte_free(PRIV(dev)->subs);
 free_dev:
 	/* mac_addrs must not be freed alone because part of dev_private */
 	dev->data->mac_addrs = NULL;
@@ -301,20 +296,8 @@ fs_rte_eth_free(const char *name)
 
 	dev = rte_eth_dev_allocated(name);
 	if (dev == NULL)
-		return -ENODEV;
-	rte_eth_dev_callback_unregister(RTE_ETH_ALL, RTE_ETH_EVENT_NEW,
-					failsafe_eth_new_event_callback, dev);
-	ret = failsafe_eal_uninit(dev);
-	if (ret)
-		ERROR("Error while uninitializing sub-EAL");
-	failsafe_args_free(dev);
-	fs_sub_device_free(dev);
-	ret = pthread_mutex_destroy(&PRIV(dev)->hotplug_mutex);
-	if (ret)
-		ERROR("Error while destroying hotplug mutex");
-	rte_free(PRIV(dev)->mcast_addrs);
-	/* mac_addrs must not be freed alone because part of dev_private */
-	dev->data->mac_addrs = NULL;
+		return 0; /* port already released */
+	ret = failsafe_eth_dev_close(dev);
 	rte_eth_dev_release_port(dev);
 	return ret;
 }
diff --git a/drivers/net/failsafe/failsafe_ops.c b/drivers/net/failsafe/failsafe_ops.c
index 93ebd09114..fc8c11b28c 100644
--- a/drivers/net/failsafe/failsafe_ops.c
+++ b/drivers/net/failsafe/failsafe_ops.c
@@ -239,29 +239,6 @@ fs_dev_set_link_down(struct rte_eth_dev *dev)
 	return 0;
 }
 
-static void fs_dev_free_queues(struct rte_eth_dev *dev);
-static int
-fs_dev_close(struct rte_eth_dev *dev)
-{
-	struct sub_device *sdev;
-	uint8_t i;
-
-	fs_lock(dev, 0);
-	failsafe_hotplug_alarm_cancel(dev);
-	if (PRIV(dev)->state == DEV_STARTED)
-		dev->dev_ops->dev_stop(dev);
-	PRIV(dev)->state = DEV_ACTIVE - 1;
-	FOREACH_SUBDEV_STATE(sdev, i, dev, DEV_ACTIVE) {
-		DEBUG("Closing sub_device %d", i);
-		failsafe_eth_dev_unregister_callbacks(sdev);
-		rte_eth_dev_close(PORT_ID(sdev));
-		sdev->state = DEV_ACTIVE - 1;
-	}
-	fs_dev_free_queues(dev);
-	fs_unlock(dev, 0);
-	return 0;
-}
-
 static int
 fs_rx_queue_stop(struct rte_eth_dev *dev, uint16_t rx_queue_id)
 {
@@ -656,6 +633,42 @@ fs_dev_free_queues(struct rte_eth_dev *dev)
 	dev->data->nb_tx_queues = 0;
 }
 
+int
+failsafe_eth_dev_close(struct rte_eth_dev *dev)
+{
+	struct sub_device *sdev;
+	uint8_t i;
+	int ret;
+
+	fs_lock(dev, 0);
+	failsafe_hotplug_alarm_cancel(dev);
+	if (PRIV(dev)->state == DEV_STARTED)
+		dev->dev_ops->dev_stop(dev);
+	PRIV(dev)->state = DEV_ACTIVE - 1;
+	FOREACH_SUBDEV_STATE(sdev, i, dev, DEV_ACTIVE) {
+		DEBUG("Closing sub_device %d", i);
+		failsafe_eth_dev_unregister_callbacks(sdev);
+		rte_eth_dev_close(PORT_ID(sdev));
+		sdev->state = DEV_ACTIVE - 1;
+	}
+	fs_dev_free_queues(dev);
+	rte_eth_dev_callback_unregister(RTE_ETH_ALL, RTE_ETH_EVENT_NEW,
+					failsafe_eth_new_event_callback, dev);
+	ret = failsafe_eal_uninit(dev);
+	if (ret)
+		ERROR("Error while uninitializing sub-EAL");
+	failsafe_args_free(dev);
+	rte_free(PRIV(dev)->subs);
+	ret = pthread_mutex_destroy(&PRIV(dev)->hotplug_mutex);
+	if (ret)
+		ERROR("Error while destroying hotplug mutex");
+	rte_free(PRIV(dev)->mcast_addrs);
+	/* mac_addrs must not be freed alone because part of dev_private */
+	dev->data->mac_addrs = NULL;
+	fs_unlock(dev, 0);
+	return 0;
+}
+
 static int
 fs_promiscuous_enable(struct rte_eth_dev *dev)
 {
@@ -1484,7 +1497,7 @@ const struct eth_dev_ops failsafe_ops = {
 	.dev_stop = fs_dev_stop,
 	.dev_set_link_down = fs_dev_set_link_down,
 	.dev_set_link_up = fs_dev_set_link_up,
-	.dev_close = fs_dev_close,
+	.dev_close = failsafe_eth_dev_close,
 	.promiscuous_enable = fs_promiscuous_enable,
 	.promiscuous_disable = fs_promiscuous_disable,
 	.allmulticast_enable = fs_allmulticast_enable,
diff --git a/drivers/net/failsafe/failsafe_private.h b/drivers/net/failsafe/failsafe_private.h
index 651578a128..6af0ef8471 100644
--- a/drivers/net/failsafe/failsafe_private.h
+++ b/drivers/net/failsafe/failsafe_private.h
@@ -236,6 +236,7 @@ int failsafe_eal_uninit(struct rte_eth_dev *dev);
 
 int failsafe_eth_dev_state_sync(struct rte_eth_dev *dev);
 void failsafe_eth_dev_unregister_callbacks(struct sub_device *sdev);
+int failsafe_eth_dev_close(struct rte_eth_dev *dev);
 void failsafe_dev_remove(struct rte_eth_dev *dev);
 void failsafe_stats_increment(struct rte_eth_stats *to,
 				struct rte_eth_stats *from);
-- 
2.28.0

Re: [dpdk-dev] [PATCH 07/20] net/failsafe: release port upon close

[Thomas Monjalon]

> +int
> +failsafe_eth_dev_close(struct rte_eth_dev *dev)
> +{
> +       struct sub_device *sdev;
> +       uint8_t i;
> +       int ret;
> +
> +       fs_lock(dev, 0);
> +       failsafe_hotplug_alarm_cancel(dev);
> +       if (PRIV(dev)->state == DEV_STARTED)
> +               dev->dev_ops->dev_stop(dev);
> +       PRIV(dev)->state = DEV_ACTIVE - 1;
> +       FOREACH_SUBDEV_STATE(sdev, i, dev, DEV_ACTIVE) {
> +               DEBUG("Closing sub_device %d", i);
> +               failsafe_eth_dev_unregister_callbacks(sdev);
> +               rte_eth_dev_close(PORT_ID(sdev));
> +               sdev->state = DEV_ACTIVE - 1;
> +       }
> +       fs_dev_free_queues(dev);
> +       rte_eth_dev_callback_unregister(RTE_ETH_ALL, RTE_ETH_EVENT_NEW,
> +                                       failsafe_eth_new_event_callback,
> dev); +       ret = failsafe_eal_uninit(dev);
> +       if (ret)
> +               ERROR("Error while uninitializing sub-EAL");
> +       failsafe_args_free(dev);
> +       rte_free(PRIV(dev)->subs);
> +       ret = pthread_mutex_destroy(&PRIV(dev)->hotplug_mutex);
> +       if (ret)
> +               ERROR("Error while destroying hotplug mutex");

Note to self:
The mutex must be destroyed after it is unlocked below in fs_unlock().
Thanks Wael for the report.

> +       rte_free(PRIV(dev)->mcast_addrs);
> +       /* mac_addrs must not be freed alone because part of dev_private */
> +       dev->data->mac_addrs = NULL;
> +       fs_unlock(dev, 0);
> +       return 0;
> +}

[dpdk-dev] [PATCH 08/20] net/iavf: release port upon close

[Thomas Monjalon]

From: Steve Yang <stevex.yang@intel.com>

The flag RTE_ETH_DEV_CLOSE_REMOVE is set so all port resources
can be freed by rte_eth_dev_close().

Freeing of private port resources is moved
from the ".remove(device)" to the ".dev_close(port)" operation.

Signed-off-by: Steve Yang <stevex.yang@intel.com>
---
[Thomas] Note: freeing in secondary process is inconsistent
between .dev_close and .remove operations.
---
 drivers/net/iavf/iavf_ethdev.c | 46 ++++++++++++++++++++--------------
 1 file changed, 27 insertions(+), 19 deletions(-)

diff --git a/drivers/net/iavf/iavf_ethdev.c b/drivers/net/iavf/iavf_ethdev.c
index 4860a94d4c..f7552f33db 100644
--- a/drivers/net/iavf/iavf_ethdev.c
+++ b/drivers/net/iavf/iavf_ethdev.c
@@ -1362,6 +1362,11 @@ iavf_dev_init(struct rte_eth_dev *eth_dev)
 	adapter->eth_dev = eth_dev;
 	adapter->stopped = 1;
 
+	/* Pass the information to the rte_eth_dev_close() that it should also
+	 * release the private port resources.
+	 */
+	eth_dev->data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
+
 	if (iavf_init_vf(eth_dev) != 0) {
 		PMD_INIT_LOG(ERR, "Init vf failed");
 		return -1;
@@ -1416,6 +1421,7 @@ iavf_dev_close(struct rte_eth_dev *dev)
 	struct rte_intr_handle *intr_handle = &pci_dev->intr_handle;
 	struct iavf_adapter *adapter =
 		IAVF_DEV_PRIVATE_TO_ADAPTER(dev->data->dev_private);
+	struct iavf_info *vf = IAVF_DEV_PRIVATE_TO_VF(dev->data->dev_private);
 
 	iavf_dev_stop(dev);
 	iavf_flow_flush(dev, NULL);
@@ -1428,21 +1434,21 @@ iavf_dev_close(struct rte_eth_dev *dev)
 	rte_intr_callback_unregister(intr_handle,
 				     iavf_dev_interrupt_handler, dev);
 	iavf_disable_irq0(hw);
-	return 0;
-}
-
-static int
-iavf_dev_uninit(struct rte_eth_dev *dev)
-{
-	struct iavf_info *vf = IAVF_DEV_PRIVATE_TO_VF(dev->data->dev_private);
-
-	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
-		return -EPERM;
 
 	dev->dev_ops = NULL;
 	dev->rx_pkt_burst = NULL;
 	dev->tx_pkt_burst = NULL;
-	iavf_dev_close(dev);
+
+	if (vf->vf_res->vf_cap_flags & VIRTCHNL_VF_OFFLOAD_RSS_PF) {
+		if (vf->rss_lut) {
+			rte_free(vf->rss_lut);
+			vf->rss_lut = NULL;
+		}
+		if (vf->rss_key) {
+			rte_free(vf->rss_key);
+			vf->rss_key = NULL;
+		}
+	}
 
 	rte_free(vf->vf_res);
 	vf->vsi_res = NULL;
@@ -1451,14 +1457,16 @@ iavf_dev_uninit(struct rte_eth_dev *dev)
 	rte_free(vf->aq_resp);
 	vf->aq_resp = NULL;
 
-	if (vf->rss_lut) {
-		rte_free(vf->rss_lut);
-		vf->rss_lut = NULL;
-	}
-	if (vf->rss_key) {
-		rte_free(vf->rss_key);
-		vf->rss_key = NULL;
-	}
+	return 0;
+}
+
+static int
+iavf_dev_uninit(struct rte_eth_dev *dev)
+{
+	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
+		return -EPERM;
+
+	iavf_dev_close(dev);
 
 	return 0;
 }
-- 
2.28.0

[dpdk-dev] [PATCH 09/20] net/mlx4: release port upon close

[Thomas Monjalon]

The flag RTE_ETH_DEV_CLOSE_REMOVE is set so all port resources
can be freed by rte_eth_dev_close().

Signed-off-by: Thomas Monjalon <thomas@monjalon.net>
---
 drivers/net/mlx4/mlx4.c | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/drivers/net/mlx4/mlx4.c b/drivers/net/mlx4/mlx4.c
index df59314b66..ad7c805d67 100644
--- a/drivers/net/mlx4/mlx4.c
+++ b/drivers/net/mlx4/mlx4.c
@@ -400,6 +400,8 @@ mlx4_dev_close(struct rte_eth_dev *dev)
 		MLX4_ASSERT(priv->ctx == NULL);
 	mlx4_intr_uninstall(priv);
 	memset(priv, 0, sizeof(*priv));
+	/* mac_addrs must not be freed because part of dev_private */
+	dev->data->mac_addrs = NULL;
 	return 0;
 }
 
@@ -1025,6 +1027,7 @@ mlx4_pci_probe(struct rte_pci_driver *pci_drv, struct rte_pci_device *pci_dev)
 			ERROR("can not allocate rte ethdev");
 			goto port_error;
 		}
+		eth_dev->data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
 		eth_dev->data->dev_private = priv;
 		eth_dev->data->mac_addrs = priv->mac;
 		eth_dev->device = &pci_dev->device;
-- 
2.28.0

[dpdk-dev] [PATCH 10/20] net/null: release port upon close

[Thomas Monjalon]

The flag RTE_ETH_DEV_CLOSE_REMOVE is set so all port resources
can be freed by rte_eth_dev_close().

Signed-off-by: Thomas Monjalon <thomas@monjalon.net>
---
 drivers/net/null/rte_eth_null.c | 24 ++++++++++++++++--------
 1 file changed, 16 insertions(+), 8 deletions(-)

diff --git a/drivers/net/null/rte_eth_null.c b/drivers/net/null/rte_eth_null.c
index 0ce073fa4b..33997013e4 100644
--- a/drivers/net/null/rte_eth_null.c
+++ b/drivers/net/null/rte_eth_null.c
@@ -458,7 +458,20 @@ eth_mac_address_set(__rte_unused struct rte_eth_dev *dev,
 	return 0;
 }
 
+static int
+eth_dev_close(struct rte_eth_dev *dev)
+{
+	PMD_LOG(INFO, "Closing null ethdev on NUMA socket %u",
+			rte_socket_id());
+
+	/* mac_addrs must not be freed alone because part of dev_private */
+	dev->data->mac_addrs = NULL;
+
+	return 0;
+}
+
 static const struct eth_dev_ops ops = {
+	.dev_close = eth_dev_close,
 	.dev_start = eth_dev_start,
 	.dev_stop = eth_dev_stop,
 	.dev_configure = eth_dev_configure,
@@ -532,6 +545,7 @@ eth_dev_null_create(struct rte_vdev_device *dev, struct pmd_options *args)
 	data->mac_addrs = &internals->eth_addr;
 	data->promiscuous = 1;
 	data->all_multicast = 1;
+	data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
 
 	eth_dev->dev_ops = &ops;
 
@@ -701,18 +715,12 @@ rte_pmd_null_remove(struct rte_vdev_device *dev)
 	if (!dev)
 		return -EINVAL;
 
-	PMD_LOG(INFO, "Closing null ethdev on numa socket %u",
-			rte_socket_id());
-
 	/* find the ethdev entry */
 	eth_dev = rte_eth_dev_allocated(rte_vdev_device_name(dev));
 	if (eth_dev == NULL)
-		return -1;
-
-	if (rte_eal_process_type() == RTE_PROC_PRIMARY)
-		/* mac_addrs must not be freed alone because part of dev_private */
-		eth_dev->data->mac_addrs = NULL;
+		return 0; /* port already released */
 
+	eth_dev_close(eth_dev);
 	rte_eth_dev_release_port(eth_dev);
 
 	return 0;
-- 
2.28.0

Re: [dpdk-dev] [PATCH 10/20] net/null: release port upon close

[Ferruh Yigit]

On 9/13/2020 11:07 PM, Thomas Monjalon wrote:
> The flag RTE_ETH_DEV_CLOSE_REMOVE is set so all port resources
> can be freed by rte_eth_dev_close().
> 
> Signed-off-by: Thomas Monjalon <thomas@monjalon.net>
> ---
>   drivers/net/null/rte_eth_null.c | 24 ++++++++++++++++--------
>   1 file changed, 16 insertions(+), 8 deletions(-)
> 
> diff --git a/drivers/net/null/rte_eth_null.c b/drivers/net/null/rte_eth_null.c
> index 0ce073fa4b..33997013e4 100644
> --- a/drivers/net/null/rte_eth_null.c
> +++ b/drivers/net/null/rte_eth_null.c
> @@ -458,7 +458,20 @@ eth_mac_address_set(__rte_unused struct rte_eth_dev *dev,
>   	return 0;
>   }
>   
> +static int
> +eth_dev_close(struct rte_eth_dev *dev)
> +{
> +	PMD_LOG(INFO, "Closing null ethdev on NUMA socket %u",
> +			rte_socket_id());
> +
> +	/* mac_addrs must not be freed alone because part of dev_private */
> +	dev->data->mac_addrs = NULL;
> +
> +	return 0;
> +}

should check 'RTE_PROC_PRIMARY' in 'eth_dev_close()'?

Re: [dpdk-dev] [PATCH 10/20] net/null: release port upon close

[Thomas Monjalon]

23/09/2020 18:44, Ferruh Yigit:
> On 9/13/2020 11:07 PM, Thomas Monjalon wrote:
> > The flag RTE_ETH_DEV_CLOSE_REMOVE is set so all port resources
> > can be freed by rte_eth_dev_close().
> > 
> > Signed-off-by: Thomas Monjalon <thomas@monjalon.net>
> > ---
> > --- a/drivers/net/null/rte_eth_null.c
> > +++ b/drivers/net/null/rte_eth_null.c
> > +static int
> > +eth_dev_close(struct rte_eth_dev *dev)
> > +{
> > +	PMD_LOG(INFO, "Closing null ethdev on NUMA socket %u",
> > +			rte_socket_id());
> > +
> > +	/* mac_addrs must not be freed alone because part of dev_private */
> > +	dev->data->mac_addrs = NULL;
> > +
> > +	return 0;
> > +}
> 
> should check 'RTE_PROC_PRIMARY' in 'eth_dev_close()'?

Yes, looks to be a miss for this new function.

Re: [dpdk-dev] [PATCH 10/20] net/null: release port upon close

[Thomas Monjalon]

23/09/2020 22:47, Thomas Monjalon:
> 23/09/2020 18:44, Ferruh Yigit:
> > On 9/13/2020 11:07 PM, Thomas Monjalon wrote:
> > > The flag RTE_ETH_DEV_CLOSE_REMOVE is set so all port resources
> > > can be freed by rte_eth_dev_close().
> > > 
> > > Signed-off-by: Thomas Monjalon <thomas@monjalon.net>
> > > ---
> > > --- a/drivers/net/null/rte_eth_null.c
> > > +++ b/drivers/net/null/rte_eth_null.c
> > > +static int
> > > +eth_dev_close(struct rte_eth_dev *dev)
> > > +{
> > > +	PMD_LOG(INFO, "Closing null ethdev on NUMA socket %u",
> > > +			rte_socket_id());
> > > +
> > > +	/* mac_addrs must not be freed alone because part of dev_private */
> > > +	dev->data->mac_addrs = NULL;
> > > +
> > > +	return 0;
> > > +}
> > 
> > should check 'RTE_PROC_PRIMARY' in 'eth_dev_close()'?
> 
> Yes, looks to be a miss for this new function.

Sorry no, this function is not freeing any shared data,
so no restriction on secondary process.

Re: [dpdk-dev] [PATCH 10/20] net/null: release port upon close

[Ferruh Yigit]

On 9/24/2020 10:58 PM, Thomas Monjalon wrote:
> 23/09/2020 22:47, Thomas Monjalon:
>> 23/09/2020 18:44, Ferruh Yigit:
>>> On 9/13/2020 11:07 PM, Thomas Monjalon wrote:
>>>> The flag RTE_ETH_DEV_CLOSE_REMOVE is set so all port resources
>>>> can be freed by rte_eth_dev_close().
>>>>
>>>> Signed-off-by: Thomas Monjalon <thomas@monjalon.net>
>>>> ---
>>>> --- a/drivers/net/null/rte_eth_null.c
>>>> +++ b/drivers/net/null/rte_eth_null.c
>>>> +static int
>>>> +eth_dev_close(struct rte_eth_dev *dev)
>>>> +{
>>>> +	PMD_LOG(INFO, "Closing null ethdev on NUMA socket %u",
>>>> +			rte_socket_id());
>>>> +
>>>> +	/* mac_addrs must not be freed alone because part of dev_private */
>>>> +	dev->data->mac_addrs = NULL;
>>>> +
>>>> +	return 0;
>>>> +}
>>>
>>> should check 'RTE_PROC_PRIMARY' in 'eth_dev_close()'?
>>
>> Yes, looks to be a miss for this new function.
> 
> Sorry no, this function is not freeing any shared data,
> so no restriction on secondary process.
> 

"dev->data->mac_addrs = NULL", won't this change the shared data?

Re: [dpdk-dev] [PATCH 10/20] net/null: release port upon close

[Thomas Monjalon]

25/09/2020 10:52, Ferruh Yigit:
> On 9/24/2020 10:58 PM, Thomas Monjalon wrote:
> > 23/09/2020 22:47, Thomas Monjalon:
> >> 23/09/2020 18:44, Ferruh Yigit:
> >>> On 9/13/2020 11:07 PM, Thomas Monjalon wrote:
> >>>> The flag RTE_ETH_DEV_CLOSE_REMOVE is set so all port resources
> >>>> can be freed by rte_eth_dev_close().
> >>>>
> >>>> Signed-off-by: Thomas Monjalon <thomas@monjalon.net>
> >>>> ---
> >>>> --- a/drivers/net/null/rte_eth_null.c
> >>>> +++ b/drivers/net/null/rte_eth_null.c
> >>>> +static int
> >>>> +eth_dev_close(struct rte_eth_dev *dev)
> >>>> +{
> >>>> +	PMD_LOG(INFO, "Closing null ethdev on NUMA socket %u",
> >>>> +			rte_socket_id());
> >>>> +
> >>>> +	/* mac_addrs must not be freed alone because part of dev_private */
> >>>> +	dev->data->mac_addrs = NULL;
> >>>> +
> >>>> +	return 0;
> >>>> +}
> >>>
> >>> should check 'RTE_PROC_PRIMARY' in 'eth_dev_close()'?
> >>
> >> Yes, looks to be a miss for this new function.
> > 
> > Sorry no, this function is not freeing any shared data,
> > so no restriction on secondary process.
> 
> "dev->data->mac_addrs = NULL", won't this change the shared data?

Absolutely, you're right.
Will fix in v2.

[dpdk-dev] [PATCH 11/20] net/octeontx: release port upon close

[Thomas Monjalon]

The flag RTE_ETH_DEV_CLOSE_REMOVE is set so all port resources
can be freed by rte_eth_dev_close().

The callback ".dev_close(port)" is called also
from the ".remove(device)" operation.

Signed-off-by: Thomas Monjalon <thomas@monjalon.net>
---
 drivers/net/octeontx/octeontx_ethdev.c | 11 +++--------
 1 file changed, 3 insertions(+), 8 deletions(-)

diff --git a/drivers/net/octeontx/octeontx_ethdev.c b/drivers/net/octeontx/octeontx_ethdev.c
index 946844ca0b..6713c51504 100644
--- a/drivers/net/octeontx/octeontx_ethdev.c
+++ b/drivers/net/octeontx/octeontx_ethdev.c
@@ -509,10 +509,6 @@ octeontx_dev_close(struct rte_eth_dev *dev)
 		rte_free(txq);
 	}
 
-	/* Free MAC address table */
-	rte_free(dev->data->mac_addrs);
-	dev->data->mac_addrs = NULL;
-
 	octeontx_port_close(nic);
 
 	dev->tx_pkt_burst = NULL;
@@ -1379,6 +1375,7 @@ octeontx_create(struct rte_vdev_device *dev, int port, uint8_t evdev,
 	data->promiscuous = 0;
 	data->all_multicast = 0;
 	data->scattered_rx = 0;
+	data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
 
 	/* Get maximum number of supported MAC entries */
 	max_entries = octeontx_bgx_port_mac_entries_get(nic->port_id);
@@ -1466,10 +1463,9 @@ octeontx_remove(struct rte_vdev_device *dev)
 	for (i = 0; i < OCTEONTX_VDEV_DEFAULT_MAX_NR_PORT; i++) {
 		sprintf(octtx_name, "eth_octeontx_%d", i);
 
-		/* reserve an ethdev entry */
 		eth_dev = rte_eth_dev_allocated(octtx_name);
 		if (eth_dev == NULL)
-			return -ENODEV;
+			continue; /* port already released */
 
 		if (rte_eal_process_type() != RTE_PROC_PRIMARY) {
 			rte_eth_dev_release_port(eth_dev);
@@ -1479,9 +1475,8 @@ octeontx_remove(struct rte_vdev_device *dev)
 		nic = octeontx_pmd_priv(eth_dev);
 		rte_event_dev_stop(nic->evdev);
 		PMD_INIT_LOG(INFO, "Closing octeontx device %s", octtx_name);
-
+		octeontx_dev_close(eth_dev);
 		rte_eth_dev_release_port(eth_dev);
-		rte_event_dev_close(nic->evdev);
 	}
 
 	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
-- 
2.28.0

[dpdk-dev] [PATCH 12/20] net/pcap: release port upon close

[Thomas Monjalon]

The flag RTE_ETH_DEV_CLOSE_REMOVE is set so all port resources
can be freed by rte_eth_dev_close().

Freeing of private port resources is moved
from the ".remove(device)" to the ".dev_close(port)" operation.

Signed-off-by: Thomas Monjalon <thomas@monjalon.net>
---
 drivers/net/pcap/rte_eth_pcap.c | 29 ++++++++++++++---------------
 1 file changed, 14 insertions(+), 15 deletions(-)

diff --git a/drivers/net/pcap/rte_eth_pcap.c b/drivers/net/pcap/rte_eth_pcap.c
index 76e704a65a..a946fa9a1a 100644
--- a/drivers/net/pcap/rte_eth_pcap.c
+++ b/drivers/net/pcap/rte_eth_pcap.c
@@ -734,6 +734,12 @@ eth_dev_close(struct rte_eth_dev *dev)
 	unsigned int i;
 	struct pmd_internals *internals = dev->data->dev_private;
 
+	if (internals == NULL)
+		return 0;
+
+	PMD_LOG(INFO, "Closing pcap ethdev on NUMA socket %d",
+			rte_socket_id());
+
 	/* Device wide flag, but cleanup must be performed per queue. */
 	if (internals->infinite_rx) {
 		for (i = 0; i < dev->data->nb_rx_queues; i++) {
@@ -748,6 +754,12 @@ eth_dev_close(struct rte_eth_dev *dev)
 		}
 	}
 
+	rte_free(dev->process_private);
+
+	if (internals->phy_mac == 0)
+		/* not dynamically allocated, must not be freed */
+		dev->data->mac_addrs = NULL;
+
 	return 0;
 }
 
@@ -1322,6 +1334,7 @@ eth_from_pcaps(struct rte_vdev_device *vdev,
 	else
 		eth_dev->tx_pkt_burst = eth_tx_drop;
 
+	eth_dev->data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
 	rte_eth_dev_probing_finish(eth_dev);
 	return 0;
 }
@@ -1544,30 +1557,16 @@ pmd_pcap_probe(struct rte_vdev_device *dev)
 static int
 pmd_pcap_remove(struct rte_vdev_device *dev)
 {
-	struct pmd_internals *internals = NULL;
 	struct rte_eth_dev *eth_dev = NULL;
 
-	PMD_LOG(INFO, "Closing pcap ethdev on numa socket %d",
-			rte_socket_id());
-
 	if (!dev)
 		return -1;
 
-	/* reserve an ethdev entry */
 	eth_dev = rte_eth_dev_allocated(rte_vdev_device_name(dev));
 	if (eth_dev == NULL)
-		return -1;
-
-	if (rte_eal_process_type() == RTE_PROC_PRIMARY) {
-		internals = eth_dev->data->dev_private;
-		if (internals != NULL && internals->phy_mac == 0)
-			/* not dynamically allocated, must not be freed */
-			eth_dev->data->mac_addrs = NULL;
-	}
+		return 0; /* port already released */
 
 	eth_dev_close(eth_dev);
-
-	rte_free(eth_dev->process_private);
 	rte_eth_dev_release_port(eth_dev);
 
 	return 0;
-- 
2.28.0

Re: [dpdk-dev] [PATCH 12/20] net/pcap: release port upon close

[Ferruh Yigit]

On 9/13/2020 11:07 PM, Thomas Monjalon wrote:
> The flag RTE_ETH_DEV_CLOSE_REMOVE is set so all port resources
> can be freed by rte_eth_dev_close().
> 
> Freeing of private port resources is moved
> from the ".remove(device)" to the ".dev_close(port)" operation.
> 
> Signed-off-by: Thomas Monjalon <thomas@monjalon.net>
> ---
>   drivers/net/pcap/rte_eth_pcap.c | 29 ++++++++++++++---------------
>   1 file changed, 14 insertions(+), 15 deletions(-)
> 
> diff --git a/drivers/net/pcap/rte_eth_pcap.c b/drivers/net/pcap/rte_eth_pcap.c
> index 76e704a65a..a946fa9a1a 100644
> --- a/drivers/net/pcap/rte_eth_pcap.c
> +++ b/drivers/net/pcap/rte_eth_pcap.c
> @@ -734,6 +734,12 @@ eth_dev_close(struct rte_eth_dev *dev)
>   	unsigned int i;
>   	struct pmd_internals *internals = dev->data->dev_private;
>   
> +	if (internals == NULL)
> +		return 0;

Not sure if this check needed, can 'internals' be null at this stage?

But perhaps need to add 'RTE_PROC_PRIMARY' check.

> +
> +	PMD_LOG(INFO, "Closing pcap ethdev on NUMA socket %d",
> +			rte_socket_id());
> +
>   	/* Device wide flag, but cleanup must be performed per queue. */
>   	if (internals->infinite_rx) {
>   		for (i = 0; i < dev->data->nb_rx_queues; i++) {
> @@ -748,6 +754,12 @@ eth_dev_close(struct rte_eth_dev *dev)
>   		}
>   	}
>   
> +	rte_free(dev->process_private);
> +

Can we move freeing 'process_private' to 'rte_eth_dev_release_port()'

Re: [dpdk-dev] [PATCH 12/20] net/pcap: release port upon close

[Thomas Monjalon]

23/09/2020 18:44, Ferruh Yigit:
> On 9/13/2020 11:07 PM, Thomas Monjalon wrote:
> > The flag RTE_ETH_DEV_CLOSE_REMOVE is set so all port resources
> > can be freed by rte_eth_dev_close().
> > 
> > Freeing of private port resources is moved
> > from the ".remove(device)" to the ".dev_close(port)" operation.
> > 
> > Signed-off-by: Thomas Monjalon <thomas@monjalon.net>
> > ---
> >   drivers/net/pcap/rte_eth_pcap.c | 29 ++++++++++++++---------------
> >   1 file changed, 14 insertions(+), 15 deletions(-)
> > 
> > diff --git a/drivers/net/pcap/rte_eth_pcap.c b/drivers/net/pcap/rte_eth_pcap.c
> > index 76e704a65a..a946fa9a1a 100644
> > --- a/drivers/net/pcap/rte_eth_pcap.c
> > +++ b/drivers/net/pcap/rte_eth_pcap.c
> > @@ -734,6 +734,12 @@ eth_dev_close(struct rte_eth_dev *dev)
> >   	unsigned int i;
> >   	struct pmd_internals *internals = dev->data->dev_private;
> >   
> > +	if (internals == NULL)
> > +		return 0;
> 
> Not sure if this check needed, can 'internals' be null at this stage?

I think yes we need to protect against a double close.

> But perhaps need to add 'RTE_PROC_PRIMARY' check.

Yes but that's not the goal of this patch.

> > +	rte_free(dev->process_private);
> 
> Can we move freeing 'process_private' to 'rte_eth_dev_release_port()'

Yes we could.

Re: [dpdk-dev] [PATCH 12/20] net/pcap: release port upon close

[Ferruh Yigit]

On 9/23/2020 9:44 PM, Thomas Monjalon wrote:
> 23/09/2020 18:44, Ferruh Yigit:
>> On 9/13/2020 11:07 PM, Thomas Monjalon wrote:
>>> The flag RTE_ETH_DEV_CLOSE_REMOVE is set so all port resources
>>> can be freed by rte_eth_dev_close().
>>>
>>> Freeing of private port resources is moved
>>> from the ".remove(device)" to the ".dev_close(port)" operation.
>>>
>>> Signed-off-by: Thomas Monjalon <thomas@monjalon.net>
>>> ---
>>>    drivers/net/pcap/rte_eth_pcap.c | 29 ++++++++++++++---------------
>>>    1 file changed, 14 insertions(+), 15 deletions(-)
>>>
>>> diff --git a/drivers/net/pcap/rte_eth_pcap.c b/drivers/net/pcap/rte_eth_pcap.c
>>> index 76e704a65a..a946fa9a1a 100644
>>> --- a/drivers/net/pcap/rte_eth_pcap.c
>>> +++ b/drivers/net/pcap/rte_eth_pcap.c
>>> @@ -734,6 +734,12 @@ eth_dev_close(struct rte_eth_dev *dev)
>>>    	unsigned int i;
>>>    	struct pmd_internals *internals = dev->data->dev_private;
>>>    
>>> +	if (internals == NULL)
>>> +		return 0;
>>
>> Not sure if this check needed, can 'internals' be null at this stage?
> 
> I think yes we need to protect against a double close.
 >

'eth_dev_close()' can be called by 'pmd_pcap_remove()' or 
'rte_eth_dev_close()' both should be blocked to call 'eth_dev_close()' 
after first close.

And same thing for all PMDs, if they don't need, this one also shouldn't 
need.

> 
>> But perhaps need to add 'RTE_PROC_PRIMARY' check.
> 
> Yes but that's not the goal of this patch.
> 

Yes, the check should be there already, but now more stuff added to 
close dev_ops, and calling it from secondary will cause problem. Since 
you are already here, I think would be nice to add it.


>>> +	rte_free(dev->process_private);
>>
>> Can we move freeing 'process_private' to 'rte_eth_dev_release_port()'
> 
> Yes we could.
> 
> 

[dpdk-dev] [PATCH 13/20] net/ring: release port upon close

[Thomas Monjalon]

The flag RTE_ETH_DEV_CLOSE_REMOVE is set so all port resources
can be freed by rte_eth_dev_close().

Freeing of private port resources is moved
from the ".remove(device)" to the ".dev_close(port)" operation.

Signed-off-by: Thomas Monjalon <thomas@monjalon.net>
---
 drivers/net/ring/rte_eth_ring.c | 51 ++++++++++++++++++++-------------
 1 file changed, 31 insertions(+), 20 deletions(-)

diff --git a/drivers/net/ring/rte_eth_ring.c b/drivers/net/ring/rte_eth_ring.c
index 733c898259..afe3ec46cc 100644
--- a/drivers/net/ring/rte_eth_ring.c
+++ b/drivers/net/ring/rte_eth_ring.c
@@ -226,7 +226,35 @@ static int
 eth_link_update(struct rte_eth_dev *dev __rte_unused,
 		int wait_to_complete __rte_unused) { return 0; }
 
+static int
+eth_dev_close(struct rte_eth_dev *dev)
+{
+	struct pmd_internals *internals = NULL;
+	struct ring_queue *r = NULL;
+	uint16_t i;
+
+	eth_dev_stop(dev);
+
+	internals = dev->data->dev_private;
+	if (internals->action == DEV_CREATE) {
+		/*
+		 * it is only necessary to delete the rings in rx_queues because
+		 * they are the same used in tx_queues
+		 */
+		for (i = 0; i < dev->data->nb_rx_queues; i++) {
+			r = dev->data->rx_queues[i];
+			rte_ring_free(r->rng);
+		}
+	}
+
+	/* mac_addrs must not be freed alone because part of dev_private */
+	dev->data->mac_addrs = NULL;
+
+	return 0;
+}
+
 static const struct eth_dev_ops ops = {
+	.dev_close = eth_dev_close,
 	.dev_start = eth_dev_start,
 	.dev_stop = eth_dev_stop,
 	.dev_set_link_up = eth_dev_set_link_up,
@@ -328,6 +356,7 @@ do_eth_dev_ring_create(const char *name,
 	eth_dev->dev_ops = &ops;
 	data->kdrv = RTE_KDRV_NONE;
 	data->numa_node = numa_node;
+	data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
 
 	/* finally assign rx and tx ops */
 	eth_dev->rx_pkt_burst = eth_ring_rx;
@@ -659,9 +688,6 @@ rte_pmd_ring_remove(struct rte_vdev_device *dev)
 {
 	const char *name = rte_vdev_device_name(dev);
 	struct rte_eth_dev *eth_dev = NULL;
-	struct pmd_internals *internals = NULL;
-	struct ring_queue *r = NULL;
-	uint16_t i;
 
 	PMD_LOG(INFO, "Un-Initializing pmd_ring for %s", name);
 
@@ -671,24 +697,9 @@ rte_pmd_ring_remove(struct rte_vdev_device *dev)
 	/* find an ethdev entry */
 	eth_dev = rte_eth_dev_allocated(name);
 	if (eth_dev == NULL)
-		return -ENODEV;
+		return 0; /* port already released */
 
-	eth_dev_stop(eth_dev);
-
-	internals = eth_dev->data->dev_private;
-	if (internals->action == DEV_CREATE) {
-		/*
-		 * it is only necessary to delete the rings in rx_queues because
-		 * they are the same used in tx_queues
-		 */
-		for (i = 0; i < eth_dev->data->nb_rx_queues; i++) {
-			r = eth_dev->data->rx_queues[i];
-			rte_ring_free(r->rng);
-		}
-	}
-
-	/* mac_addrs must not be freed alone because part of dev_private */
-	eth_dev->data->mac_addrs = NULL;
+	eth_dev_close(eth_dev);
 	rte_eth_dev_release_port(eth_dev);
 	return 0;
 }
-- 
2.28.0

Re: [dpdk-dev] [PATCH 13/20] net/ring: release port upon close

[Bruce Richardson]

On Mon, Sep 14, 2020 at 12:07:04AM +0200, Thomas Monjalon wrote:
> The flag RTE_ETH_DEV_CLOSE_REMOVE is set so all port resources
> can be freed by rte_eth_dev_close().
> 
> Freeing of private port resources is moved
> from the ".remove(device)" to the ".dev_close(port)" operation.
> 
> Signed-off-by: Thomas Monjalon <thomas@monjalon.net>
> ---
Acked-by: Bruce Richardson <bruce.richardson@intel.com>

[dpdk-dev] [PATCH 14/20] net/softnic: release port upon close

[Thomas Monjalon]

The flag RTE_ETH_DEV_CLOSE_REMOVE is set so all port resources
can be freed by rte_eth_dev_close().

Freeing of private port resources is moved
from the ".remove(device)" to the ".dev_close(port)" operation.

Signed-off-by: Thomas Monjalon <thomas@monjalon.net>
---
 drivers/net/softnic/rte_eth_softnic.c | 63 ++++++++++++++-------------
 1 file changed, 32 insertions(+), 31 deletions(-)

diff --git a/drivers/net/softnic/rte_eth_softnic.c b/drivers/net/softnic/rte_eth_softnic.c
index 491a308c11..5b94e250d8 100644
--- a/drivers/net/softnic/rte_eth_softnic.c
+++ b/drivers/net/softnic/rte_eth_softnic.c
@@ -201,9 +201,37 @@ pmd_dev_stop(struct rte_eth_dev *dev)
 	softnic_mtr_free(p);
 }
 
+static void
+pmd_free(struct pmd_internals *p)
+{
+	if (p == NULL)
+		return;
+
+	if (p->params.conn_port)
+		softnic_conn_free(p->conn);
+
+	softnic_thread_free(p);
+	softnic_pipeline_free(p);
+	softnic_table_action_profile_free(p);
+	softnic_port_in_action_profile_free(p);
+	softnic_tap_free(p);
+	softnic_tmgr_free(p);
+	softnic_link_free(p);
+	softnic_swq_free(p);
+	softnic_mempool_free(p);
+
+	tm_hierarchy_free(p);
+	softnic_mtr_free(p);
+
+	rte_free(p);
+}
+
 static int
-pmd_dev_close(struct rte_eth_dev *dev __rte_unused)
+pmd_dev_close(struct rte_eth_dev *dev)
 {
+	pmd_free(dev->data->dev_private);
+	dev->data->dev_private = NULL; /* already freed */
+	dev->data->mac_addrs = NULL; /* statically allocated */
 	return 0;
 }
 
@@ -335,31 +363,6 @@ pmd_init(struct pmd_params *params)
 	return p;
 }
 
-static void
-pmd_free(struct pmd_internals *p)
-{
-	if (p == NULL)
-		return;
-
-	if (p->params.conn_port)
-		softnic_conn_free(p->conn);
-
-	softnic_thread_free(p);
-	softnic_pipeline_free(p);
-	softnic_table_action_profile_free(p);
-	softnic_port_in_action_profile_free(p);
-	softnic_tap_free(p);
-	softnic_tmgr_free(p);
-	softnic_link_free(p);
-	softnic_swq_free(p);
-	softnic_mempool_free(p);
-
-	tm_hierarchy_free(p);
-	softnic_mtr_free(p);
-
-	rte_free(p);
-}
-
 static struct rte_ether_addr eth_addr = {
 	.addr_bytes = {0},
 };
@@ -384,6 +387,7 @@ pmd_ethdev_register(struct rte_vdev_device *vdev,
 	dev->device = &vdev->device;
 
 	/* dev->data */
+	dev->data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
 	dev->data->dev_private = dev_private;
 	dev->data->dev_link.link_speed = ETH_SPEED_NUM_100G;
 	dev->data->dev_link.link_duplex = ETH_LINK_FULL_DUPLEX;
@@ -653,12 +657,9 @@ pmd_remove(struct rte_vdev_device *vdev)
 	/* Find the ethdev entry */
 	dev = rte_eth_dev_allocated(rte_vdev_device_name(vdev));
 	if (dev == NULL)
-		return -ENODEV;
+		return 0; /* port already released */
 
-	/* Free device data structures*/
-	pmd_free(dev->data->dev_private);
-	dev->data->dev_private = NULL; /* already freed */
-	dev->data->mac_addrs = NULL; /* statically allocated */
+	pmd_dev_close(dev);
 	rte_eth_dev_release_port(dev);
 
 	return 0;
-- 
2.28.0

Re: [dpdk-dev] [PATCH 14/20] net/softnic: release port upon close

[Dumitrescu, Cristian]

> -----Original Message-----
> From: Thomas Monjalon <thomas@monjalon.net>
> Sent: Sunday, September 13, 2020 11:07 PM
> To: dev@dpdk.org
> Cc: Yigit, Ferruh <ferruh.yigit@intel.com>; arybchenko@solarflare.com;
> Singh, Jasvinder <jasvinder.singh@intel.com>; Dumitrescu, Cristian
> <cristian.dumitrescu@intel.com>
> Subject: [PATCH 14/20] net/softnic: release port upon close
> 
> The flag RTE_ETH_DEV_CLOSE_REMOVE is set so all port resources
> can be freed by rte_eth_dev_close().
> 
> Freeing of private port resources is moved
> from the ".remove(device)" to the ".dev_close(port)" operation.
> 
> Signed-off-by: Thomas Monjalon <thomas@monjalon.net>
> ---

Thanks for doing this work, Thomas!

Acked-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com>

[dpdk-dev] [PATCH 15/20] net/tap: release port upon close

[Thomas Monjalon]

From: Yunjian Wang <wangyunjian@huawei.com>

The flag RTE_ETH_DEV_CLOSE_REMOVE is set so all port resources
can be freed by rte_eth_dev_close().

Freeing of private port resources is moved
from the ".remove(device)" to the ".dev_close(port)" operation.

Signed-off-by: Yunjian Wang <wangyunjian@huawei.com>
---
[Thomas] Note: freeing in secondary process is inconsistent
between .dev_close and .remove operations.
---
 drivers/net/tap/rte_eth_tap.c | 46 ++++++++++++++++++++---------------
 1 file changed, 26 insertions(+), 20 deletions(-)

diff --git a/drivers/net/tap/rte_eth_tap.c b/drivers/net/tap/rte_eth_tap.c
index 6bce90c531..2e325f55aa 100644
--- a/drivers/net/tap/rte_eth_tap.c
+++ b/drivers/net/tap/rte_eth_tap.c
@@ -72,6 +72,10 @@
 
 static int tap_devices_count;
 
+static const char *tuntap_types[ETH_TUNTAP_TYPE_MAX] = {
+	"UNKNOWN", "TUN", "TAP"
+};
+
 static const char *valid_arguments[] = {
 	ETH_TAP_IFACE_ARG,
 	ETH_TAP_REMOTE_ARG,
@@ -1040,6 +1044,9 @@ tap_dev_close(struct rte_eth_dev *dev)
 	struct pmd_process_private *process_private = dev->process_private;
 	struct rx_queue *rxq;
 
+	if (process_private == NULL)
+		return 0;
+
 	tap_link_set_down(dev);
 	if (internals->nlsk_fd != -1) {
 		tap_flow_flush(dev, NULL);
@@ -1079,6 +1086,23 @@ tap_dev_close(struct rte_eth_dev *dev)
 	 * it will be removed from kernel
 	 */
 
+	/* mac_addrs must not be freed alone because part of dev_private */
+	dev->data->mac_addrs = NULL;
+
+	internals = dev->data->dev_private;
+	TAP_LOG(DEBUG, "Closing %s Ethernet device on NUMA %u",
+		tuntap_types[internals->type], rte_socket_id());
+
+	if (internals->ioctl_sock != -1) {
+		close(internals->ioctl_sock);
+		internals->ioctl_sock = -1;
+	}
+	rte_free(dev->process_private);
+	dev->process_private = NULL;
+	if (tap_devices_count == 1)
+		rte_mp_action_unregister(TAP_MP_KEY);
+	tap_devices_count--;
+
 	return 0;
 }
 
@@ -1802,10 +1826,6 @@ static const struct eth_dev_ops ops = {
 	.filter_ctrl            = tap_dev_filter_ctrl,
 };
 
-static const char *tuntap_types[ETH_TUNTAP_TYPE_MAX] = {
-	"UNKNOWN", "TUN", "TAP"
-};
-
 static int
 eth_dev_tap_create(struct rte_vdev_device *vdev, const char *tap_name,
 		   char *remote_iface, struct rte_ether_addr *mac_addr,
@@ -1856,7 +1876,7 @@ eth_dev_tap_create(struct rte_vdev_device *vdev, const char *tap_name,
 	/* Setup some default values */
 	data = dev->data;
 	data->dev_private = pmd;
-	data->dev_flags = RTE_ETH_DEV_INTR_LSC;
+	data->dev_flags = RTE_ETH_DEV_INTR_LSC | RTE_ETH_DEV_CLOSE_REMOVE;
 	data->numa_node = numa_node;
 
 	data->dev_link = pmd_link;
@@ -2448,30 +2468,16 @@ static int
 rte_pmd_tap_remove(struct rte_vdev_device *dev)
 {
 	struct rte_eth_dev *eth_dev = NULL;
-	struct pmd_internals *internals;
 
 	/* find the ethdev entry */
 	eth_dev = rte_eth_dev_allocated(rte_vdev_device_name(dev));
 	if (!eth_dev)
-		return -ENODEV;
-
-	/* mac_addrs must not be freed alone because part of dev_private */
-	eth_dev->data->mac_addrs = NULL;
+		return 0;
 
 	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
 		return rte_eth_dev_release_port(eth_dev);
 
 	tap_dev_close(eth_dev);
-
-	internals = eth_dev->data->dev_private;
-	TAP_LOG(DEBUG, "Closing %s Ethernet device on numa %u",
-		tuntap_types[internals->type], rte_socket_id());
-
-	close(internals->ioctl_sock);
-	rte_free(eth_dev->process_private);
-	if (tap_devices_count == 1)
-		rte_mp_action_unregister(TAP_MP_KEY);
-	tap_devices_count--;
 	rte_eth_dev_release_port(eth_dev);
 
 	return 0;
-- 
2.28.0

[dpdk-dev] [PATCH 16/20] ethdev: remove old close behaviour

[Thomas Monjalon]

The temporary flag RTE_ETH_DEV_CLOSE_REMOVE is removed.
It was introduced in DPDK 18.11 in order to give time for PMDs to migrate.

The old behaviour was to free only queues when closing a port.
The new behaviour is calling rte_eth_dev_release_port() which does
three more tasks:
	- trigger event callback
	- reset state and few pointers
	- free all generic port resources

The private port resources must be released in the .dev_close callback.

The .remove callback should:
	- call .dev_close callback
	- call rte_eth_dev_release_port()
	- free multi-port device shared resources

Despite waiting two years, some drivers have not migrated,
so they may hit issues with the incompatible new behaviour.
After sending emails, adding logs, and announcing the deprecation,
the only last solution is to declare these drivers as unmaintained:
	bnx2x, cxgbe, dpaa, dpaa2, enetc, ionic,
	ipn3ke,	liquidio, nfp, pfe, qede
Below is a summary of what to implement in those drivers.

* The freeing of private port resources must be moved
from the ".remove(device)" function to the ".dev_close(port)" function.

* If a generic resource (.mac_addrs or .hash_mac_addrs) cannot be freed,
it must be set to NULL in ".dev_close" function to protect from
subsequent rte_eth_dev_release_port() freeing.

* Note 1:
The generic resources are freed in rte_eth_dev_release_port(),
after ".dev_close" is called in rte_eth_dev_close(), but not when
calling ".dev_close" directly from the ".remove" PMD function.
That's why rte_eth_dev_release_port() must still be called explicitly
from ".remove(device)" after calling the ".dev_close" PMD function.

* Note 2:
If a device can have multiple ports, the common resources must be freed
only in the ".remove(device)" function.

* Note 3:
The port is supposed to be in a stopped state when it is closed.
If it is not the case, it is free to the PMD implementation
how to react when trying to close a non-stopped port:
either try to stop it automatically or just return an error.

Cc: Rahul Lakkireddy <rahul.lakkireddy@chelsio.com>
Cc: Rosen Xu <rosen.xu@intel.com>
Cc: Shijith Thotton <sthotton@marvell.com>
Cc: Srisivasubramanian Srinivasan <srinivasan@marvell.com>
Cc: Rasesh Mody <rmody@marvell.com>
Cc: Shahed Shaikh <shshaikh@marvell.com>
Cc: Heinrich Kuhn <heinrich.kuhn@netronome.com>
Cc: Hemant Agrawal <hemant.agrawal@nxp.com>
Cc: Sachin Saxena <sachin.saxena@nxp.com>
Cc: Gagandeep Singh <g.singh@nxp.com>
Cc: Akhil Goyal <akhil.goyal@nxp.com>
Cc: Alfredo Cardigliano <cardigliano@ntop.org>

Signed-off-by: Thomas Monjalon <thomas@monjalon.net>
---
 MAINTAINERS                               | 22 +++++++++++-----------
 doc/guides/rel_notes/deprecation.rst      |  6 ------
 drivers/net/af_packet/rte_eth_af_packet.c |  1 -
 drivers/net/af_xdp/rte_eth_af_xdp.c       |  2 --
 drivers/net/ark/ark_ethdev.c              |  2 --
 drivers/net/atlantic/atl_ethdev.c         |  2 --
 drivers/net/avp/avp_ethdev.c              |  2 --
 drivers/net/axgbe/axgbe_ethdev.c          |  1 -
 drivers/net/bnxt/bnxt_ethdev.c            |  5 -----
 drivers/net/bnxt/bnxt_reps.c              |  4 ----
 drivers/net/bonding/rte_eth_bond_pmd.c    |  1 -
 drivers/net/e1000/em_ethdev.c             |  5 -----
 drivers/net/e1000/igb_ethdev.c            | 10 ----------
 drivers/net/ena/ena_ethdev.c              |  6 ------
 drivers/net/enic/enic_ethdev.c            |  2 --
 drivers/net/failsafe/failsafe.c           |  1 -
 drivers/net/fm10k/fm10k_ethdev.c          |  5 -----
 drivers/net/hinic/hinic_pmd_ethdev.c      |  6 ------
 drivers/net/hns3/hns3_ethdev.c            |  5 -----
 drivers/net/hns3/hns3_ethdev_vf.c         |  5 -----
 drivers/net/i40e/i40e_ethdev.c            |  5 -----
 drivers/net/i40e/i40e_ethdev_vf.c         |  5 -----
 drivers/net/iavf/iavf_ethdev.c            |  5 -----
 drivers/net/ice/ice_dcf_ethdev.c          |  2 --
 drivers/net/ice/ice_ethdev.c              |  5 -----
 drivers/net/igc/igc_ethdev.c              |  5 -----
 drivers/net/ixgbe/ixgbe_ethdev.c          | 10 ----------
 drivers/net/kni/rte_eth_kni.c             |  2 --
 drivers/net/memif/rte_eth_memif.c         |  3 ---
 drivers/net/mlx4/mlx4.c                   |  1 -
 drivers/net/mlx5/linux/mlx5_os.c          |  2 --
 drivers/net/mvneta/mvneta_ethdev.c        |  3 ---
 drivers/net/mvpp2/mrvl_ethdev.c           |  3 ---
 drivers/net/netvsc/hn_ethdev.c            |  3 ---
 drivers/net/nfb/nfb_ethdev.c              |  3 ---
 drivers/net/null/rte_eth_null.c           |  1 -
 drivers/net/octeontx/octeontx_ethdev.c    |  1 -
 drivers/net/octeontx2/otx2_ethdev.c       |  1 -
 drivers/net/pcap/rte_eth_pcap.c           |  1 -
 drivers/net/ring/rte_eth_ring.c           |  1 -
 drivers/net/sfc/sfc_ethdev.c              |  4 +---
 drivers/net/softnic/rte_eth_softnic.c     |  1 -
 drivers/net/szedata2/rte_eth_szedata2.c   |  3 ---
 drivers/net/tap/rte_eth_tap.c             |  2 +-
 drivers/net/vhost/rte_eth_vhost.c         |  2 +-
 drivers/net/virtio/virtio_ethdev.c        |  5 -----
 drivers/net/vmxnet3/vmxnet3_ethdev.c      |  3 ---
 lib/librte_ethdev/rte_ethdev.c            | 17 +----------------
 lib/librte_ethdev/rte_ethdev.h            |  8 +-------
 49 files changed, 16 insertions(+), 184 deletions(-)

diff --git a/MAINTAINERS b/MAINTAINERS
index 3b16d7a4b8..98aa24412e 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -605,7 +605,7 @@ F: drivers/net/thunderx/
 F: doc/guides/nics/thunderx.rst
 F: doc/guides/nics/features/thunderx.ini
 
-Cavium LiquidIO
+Cavium LiquidIO - UNMAINTAINED
 M: Shijith Thotton <sthotton@marvell.com>
 M: Srisivasubramanian Srinivasan <srinivasan@marvell.com>
 T: git://dpdk.org/next/dpdk-next-net-mrvl
@@ -622,7 +622,7 @@ F: drivers/net/octeontx/
 F: doc/guides/nics/octeontx.rst
 F: doc/guides/nics/features/octeontx.ini
 
-Chelsio cxgbe
+Chelsio cxgbe - UNMAINTAINED
 M: Rahul Lakkireddy <rahul.lakkireddy@chelsio.com>
 F: drivers/net/cxgbe/
 F: doc/guides/nics/cxgbe.rst
@@ -711,7 +711,7 @@ F: drivers/net/igc/
 F: doc/guides/nics/igc.rst
 F: doc/guides/nics/features/igc.ini
 
-Intel ipn3ke
+Intel ipn3ke - UNMAINTAINED
 M: Rosen Xu <rosen.xu@intel.com>
 T: git://dpdk.org/next/dpdk-next-net-intel
 F: drivers/net/ipn3ke/
@@ -794,13 +794,13 @@ F: drivers/net/nfb/
 F: doc/guides/nics/nfb.rst
 F: doc/guides/nics/features/nfb.ini
 
-Netronome nfp
+Netronome nfp - UNMAINTAINED
 M: Heinrich Kuhn <heinrich.kuhn@netronome.com>
 F: drivers/net/nfp/
 F: doc/guides/nics/nfp.rst
 F: doc/guides/nics/features/nfp*.ini
 
-NXP dpaa
+NXP dpaa - UNMAINTAINED
 M: Hemant Agrawal <hemant.agrawal@nxp.com>
 M: Sachin Saxena <sachin.saxena@nxp.com>
 F: drivers/mempool/dpaa/
@@ -808,7 +808,7 @@ F: drivers/net/dpaa/
 F: doc/guides/nics/dpaa.rst
 F: doc/guides/nics/features/dpaa.ini
 
-NXP dpaa2
+NXP dpaa2 - UNMAINTAINED
 M: Hemant Agrawal <hemant.agrawal@nxp.com>
 M: Sachin Saxena <sachin.saxena@nxp.com>
 F: drivers/mempool/dpaa2/
@@ -816,27 +816,27 @@ F: drivers/net/dpaa2/
 F: doc/guides/nics/dpaa2.rst
 F: doc/guides/nics/features/dpaa2.ini
 
-NXP enetc
+NXP enetc - UNMAINTAINED
 M: Gagandeep Singh <g.singh@nxp.com>
 M: Sachin Saxena <sachin.saxena@nxp.com>
 F: drivers/net/enetc/
 F: doc/guides/nics/enetc.rst
 F: doc/guides/nics/features/enetc.ini
 
-NXP pfe
+NXP pfe - UNMAINTAINED
 M: Gagandeep Singh <g.singh@nxp.com>
 M: Akhil Goyal <akhil.goyal@nxp.com>
 F: doc/guides/nics/pfe.rst
 F: drivers/net/pfe/
 F: doc/guides/nics/features/pfe.ini
 
-Pensando ionic
+Pensando ionic - UNMAINTAINED
 M: Alfredo Cardigliano <cardigliano@ntop.org>
 F: drivers/net/ionic/
 F: doc/guides/nics/ionic.rst
 F: doc/guides/nics/features/ionic.ini
 
-QLogic bnx2x
+QLogic bnx2x - UNMAINTAINED
 M: Rasesh Mody <rmody@marvell.com>
 M: Shahed Shaikh <shshaikh@marvell.com>
 T: git://dpdk.org/next/dpdk-next-net-mrvl
@@ -844,7 +844,7 @@ F: drivers/net/bnx2x/
 F: doc/guides/nics/bnx2x.rst
 F: doc/guides/nics/features/bnx2x*.ini
 
-QLogic qede PMD
+QLogic qede PMD - UNMAINTAINED
 M: Rasesh Mody <rmody@marvell.com>
 M: Shahed Shaikh <shshaikh@marvell.com>
 T: git://dpdk.org/next/dpdk-next-net-mrvl
diff --git a/doc/guides/rel_notes/deprecation.rst b/doc/guides/rel_notes/deprecation.rst
index 88d7d07613..a9d937a43c 100644
--- a/doc/guides/rel_notes/deprecation.rst
+++ b/doc/guides/rel_notes/deprecation.rst
@@ -189,12 +189,6 @@ Deprecation Notices
   - ``rte_eth_dev_stop``
   - ``rte_eth_dev_close``
 
-* ethdev: The temporary flag RTE_ETH_DEV_CLOSE_REMOVE will be removed in 20.11.
-  As a consequence, the new behaviour introduced in 18.11 will be effective
-  for all drivers: generic port resources are freed on close operation.
-  Private resources are expected to be released in the ``dev_close`` callback.
-  More details in http://inbox.dpdk.org/dev/5248162.j6AOsuQRmx@thomas/
-
 * ethdev: New offload flags ``DEV_RX_OFFLOAD_FLOW_MARK`` will be added in 19.11.
   This will allow application to enable or disable PMDs from updating
   ``rte_mbuf::hash::fdir``.
diff --git a/drivers/net/af_packet/rte_eth_af_packet.c b/drivers/net/af_packet/rte_eth_af_packet.c
index 3ab6f4d388..ff222c3dac 100644
--- a/drivers/net/af_packet/rte_eth_af_packet.c
+++ b/drivers/net/af_packet/rte_eth_af_packet.c
@@ -859,7 +859,6 @@ rte_pmd_init_internals(struct rte_vdev_device *dev,
 	data->nb_tx_queues = (uint16_t)nb_queues;
 	data->dev_link = pmd_link;
 	data->mac_addrs = &(*internals)->eth_addr;
-	data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
 
 	(*eth_dev)->dev_ops = &ops;
 
diff --git a/drivers/net/af_xdp/rte_eth_af_xdp.c b/drivers/net/af_xdp/rte_eth_af_xdp.c
index bac9729bf8..2eabc6437f 100644
--- a/drivers/net/af_xdp/rte_eth_af_xdp.c
+++ b/drivers/net/af_xdp/rte_eth_af_xdp.c
@@ -1269,8 +1269,6 @@ init_internals(struct rte_vdev_device *dev, const char *if_name,
 	eth_dev->dev_ops = &ops;
 	eth_dev->rx_pkt_burst = eth_af_xdp_rx;
 	eth_dev->tx_pkt_burst = eth_af_xdp_tx;
-	/* Let rte_eth_dev_close() release the port resources. */
-	eth_dev->data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
 
 #if defined(XDP_UMEM_UNALIGNED_CHUNK_FLAG)
 	AF_XDP_LOG(INFO, "Zero copy between umem and mbuf enabled.\n");
diff --git a/drivers/net/ark/ark_ethdev.c b/drivers/net/ark/ark_ethdev.c
index 9e1f8218a8..8012af75ee 100644
--- a/drivers/net/ark/ark_ethdev.c
+++ b/drivers/net/ark/ark_ethdev.c
@@ -261,8 +261,6 @@ eth_ark_dev_init(struct rte_eth_dev *dev)
 	/* Use dummy function until setup */
 	dev->rx_pkt_burst = &eth_ark_recv_pkts_noop;
 	dev->tx_pkt_burst = &eth_ark_xmit_pkts_noop;
-	/* Let rte_eth_dev_close() release the port resources */
-	dev->data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
 
 	ark->bar0 = (uint8_t *)pci_dev->mem_resource[0].addr;
 	ark->a_bar = (uint8_t *)pci_dev->mem_resource[2].addr;
diff --git a/drivers/net/atlantic/atl_ethdev.c b/drivers/net/atlantic/atl_ethdev.c
index 16721a011f..751c81b9c1 100644
--- a/drivers/net/atlantic/atl_ethdev.c
+++ b/drivers/net/atlantic/atl_ethdev.c
@@ -379,8 +379,6 @@ eth_atl_dev_init(struct rte_eth_dev *eth_dev)
 	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
 		return 0;
 
-	eth_dev->data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
-
 	/* Vendor and Device ID need to be set before init of shared code */
 	hw->device_id = pci_dev->id.device_id;
 	hw->vendor_id = pci_dev->id.vendor_id;
diff --git a/drivers/net/avp/avp_ethdev.c b/drivers/net/avp/avp_ethdev.c
index 2eed6acc3f..c730b7ab86 100644
--- a/drivers/net/avp/avp_ethdev.c
+++ b/drivers/net/avp/avp_ethdev.c
@@ -957,8 +957,6 @@ eth_avp_dev_init(struct rte_eth_dev *eth_dev)
 	eth_dev->dev_ops = &avp_eth_dev_ops;
 	eth_dev->rx_pkt_burst = &avp_recv_pkts;
 	eth_dev->tx_pkt_burst = &avp_xmit_pkts;
-	/* Let rte_eth_dev_close() release the port resources */
-	eth_dev->data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
 
 	if (rte_eal_process_type() != RTE_PROC_PRIMARY) {
 		/*
diff --git a/drivers/net/axgbe/axgbe_ethdev.c b/drivers/net/axgbe/axgbe_ethdev.c
index c5e70ea0a1..246e09016c 100644
--- a/drivers/net/axgbe/axgbe_ethdev.c
+++ b/drivers/net/axgbe/axgbe_ethdev.c
@@ -1623,7 +1623,6 @@ eth_axgbe_dev_init(struct rte_eth_dev *eth_dev)
 	int ret;
 
 	eth_dev->dev_ops = &axgbe_eth_dev_ops;
-	eth_dev->data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
 
 	/*
 	 * For secondary processes, we don't initialise any further as primary
diff --git a/drivers/net/bnxt/bnxt_ethdev.c b/drivers/net/bnxt/bnxt_ethdev.c
index 02e53bf04d..c580aff96b 100644
--- a/drivers/net/bnxt/bnxt_ethdev.c
+++ b/drivers/net/bnxt/bnxt_ethdev.c
@@ -5744,11 +5744,6 @@ bnxt_dev_init(struct rte_eth_dev *eth_dev, void *params __rte_unused)
 
 	bnxt_alloc_switch_domain(bp);
 
-	/* Pass the information to the rte_eth_dev_close() that it should also
-	 * release the private port resources.
-	 */
-	eth_dev->data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
-
 	PMD_DRV_LOG(INFO,
 		    DRV_MODULE_NAME "found at mem %" PRIX64 ", node addr %pM\n",
 		    pci_dev->mem_resource[0].phys_addr,
diff --git a/drivers/net/bnxt/bnxt_reps.c b/drivers/net/bnxt/bnxt_reps.c
index fa014125e3..ceb0604216 100644
--- a/drivers/net/bnxt/bnxt_reps.c
+++ b/drivers/net/bnxt/bnxt_reps.c
@@ -186,10 +186,6 @@ int bnxt_vf_representor_init(struct rte_eth_dev *eth_dev, void *params)
 	PMD_DRV_LOG(INFO, "calling bnxt_print_link_info\n");
 	bnxt_print_link_info(eth_dev);
 
-	/* Pass the information to the rte_eth_dev_close() that it should also
-	 * release the private port resources.
-	 */
-	eth_dev->data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
 	PMD_DRV_LOG(INFO,
 		    "Switch domain id %d: Representor Device %d init done\n",
 		    vf_rep_bp->switch_domain_id, vf_rep_bp->vf_id);
diff --git a/drivers/net/bonding/rte_eth_bond_pmd.c b/drivers/net/bonding/rte_eth_bond_pmd.c
index 726b47055b..f9603d16a5 100644
--- a/drivers/net/bonding/rte_eth_bond_pmd.c
+++ b/drivers/net/bonding/rte_eth_bond_pmd.c
@@ -3209,7 +3209,6 @@ bond_alloc(struct rte_vdev_device *dev, uint8_t mode)
 	}
 
 	internals = eth_dev->data->dev_private;
-	eth_dev->data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
 	eth_dev->data->nb_rx_queues = (uint16_t)1;
 	eth_dev->data->nb_tx_queues = (uint16_t)1;
 
diff --git a/drivers/net/e1000/em_ethdev.c b/drivers/net/e1000/em_ethdev.c
index 45d3949ff7..7d592e0313 100644
--- a/drivers/net/e1000/em_ethdev.c
+++ b/drivers/net/e1000/em_ethdev.c
@@ -297,11 +297,6 @@ eth_em_dev_init(struct rte_eth_dev *eth_dev)
 	rte_ether_addr_copy((struct rte_ether_addr *)hw->mac.addr,
 		eth_dev->data->mac_addrs);
 
-	/* Pass the information to the rte_eth_dev_close() that it should also
-	 * release the private port resources.
-	 */
-	eth_dev->data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
-
 	/* initialize the vfta */
 	memset(shadow_vfta, 0, sizeof(*shadow_vfta));
 
diff --git a/drivers/net/e1000/igb_ethdev.c b/drivers/net/e1000/igb_ethdev.c
index 564216b0a4..c7b2244890 100644
--- a/drivers/net/e1000/igb_ethdev.c
+++ b/drivers/net/e1000/igb_ethdev.c
@@ -844,11 +844,6 @@ eth_igb_dev_init(struct rte_eth_dev *eth_dev)
 	rte_ether_addr_copy((struct rte_ether_addr *)hw->mac.addr,
 			&eth_dev->data->mac_addrs[0]);
 
-	/* Pass the information to the rte_eth_dev_close() that it should also
-	 * release the private port resources.
-	 */
-	eth_dev->data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
-
 	/* initialize the vfta */
 	memset(shadow_vfta, 0, sizeof(*shadow_vfta));
 
@@ -997,11 +992,6 @@ eth_igbvf_dev_init(struct rte_eth_dev *eth_dev)
 		return -ENOMEM;
 	}
 
-	/* Pass the information to the rte_eth_dev_close() that it should also
-	 * release the private port resources.
-	 */
-	eth_dev->data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
-
 	/* Generate a random MAC address, if none was assigned by PF. */
 	if (rte_is_zero_ether_addr(perm_addr)) {
 		rte_eth_random_addr(perm_addr->addr_bytes);
diff --git a/drivers/net/ena/ena_ethdev.c b/drivers/net/ena/ena_ethdev.c
index ea979fddd1..9d91ff0708 100644
--- a/drivers/net/ena/ena_ethdev.c
+++ b/drivers/net/ena/ena_ethdev.c
@@ -1863,12 +1863,6 @@ static int eth_ena_dev_init(struct rte_eth_dev *eth_dev)
 			get_feat_ctx.dev_attr.mac_addr,
 			(struct rte_ether_addr *)adapter->mac_addr);
 
-	/*
-	 * Pass the information to the rte_eth_dev_close() that it should also
-	 * release the private port resources.
-	 */
-	eth_dev->data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
-
 	adapter->drv_stats = rte_zmalloc("adapter stats",
 					 sizeof(*adapter->drv_stats),
 					 RTE_CACHE_LINE_SIZE);
diff --git a/drivers/net/enic/enic_ethdev.c b/drivers/net/enic/enic_ethdev.c
index b14ed00085..0674c668b4 100644
--- a/drivers/net/enic/enic_ethdev.c
+++ b/drivers/net/enic/enic_ethdev.c
@@ -1293,8 +1293,6 @@ static int eth_enicpmd_dev_init(struct rte_eth_dev *eth_dev)
 	enic->port_id = eth_dev->data->port_id;
 	enic->rte_dev = eth_dev;
 	enic->dev_data = eth_dev->data;
-	/* Let rte_eth_dev_close() release the port resources */
-	eth_dev->data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
 
 	pdev = RTE_ETH_DEV_TO_PCI(eth_dev);
 	rte_eth_copy_pci_info(eth_dev, pdev);
diff --git a/drivers/net/failsafe/failsafe.c b/drivers/net/failsafe/failsafe.c
index 44d47e8f72..b921e101e6 100644
--- a/drivers/net/failsafe/failsafe.c
+++ b/drivers/net/failsafe/failsafe.c
@@ -180,7 +180,6 @@ fs_eth_dev_create(struct rte_vdev_device *vdev)
 		ERROR("Unable to allocate rte_eth_dev");
 		return -1;
 	}
-	dev->data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
 	priv = PRIV(dev);
 	priv->data = dev->data;
 	priv->rxp = FS_RX_PROXY_INIT;
diff --git a/drivers/net/fm10k/fm10k_ethdev.c b/drivers/net/fm10k/fm10k_ethdev.c
index 82f46cebf5..22e68be448 100644
--- a/drivers/net/fm10k/fm10k_ethdev.c
+++ b/drivers/net/fm10k/fm10k_ethdev.c
@@ -3132,11 +3132,6 @@ eth_fm10k_dev_init(struct rte_eth_dev *dev)
 		&dev->data->mac_addrs[0]);
 	}
 
-	/* Pass the information to the rte_eth_dev_close() that it should also
-	 * release the private port resources.
-	 */
-	dev->data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
-
 	/* Reset the hw statistics */
 	diag = fm10k_stats_reset(dev);
 	if (diag != 0) {
diff --git a/drivers/net/hinic/hinic_pmd_ethdev.c b/drivers/net/hinic/hinic_pmd_ethdev.c
index 5f2e97d3bd..623534fda4 100644
--- a/drivers/net/hinic/hinic_pmd_ethdev.c
+++ b/drivers/net/hinic/hinic_pmd_ethdev.c
@@ -3100,12 +3100,6 @@ static int hinic_func_init(struct rte_eth_dev *eth_dev)
 		goto mc_addr_fail;
 	}
 
-	/*
-	 * Pass the information to the rte_eth_dev_close() that it should also
-	 * release the private port resources.
-	 */
-	eth_dev->data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
-
 	/* create hardware nic_device */
 	rc = hinic_nic_dev_create(eth_dev);
 	if (rc) {
diff --git a/drivers/net/hns3/hns3_ethdev.c b/drivers/net/hns3/hns3_ethdev.c
index 96806afd87..bc853c6a8f 100644
--- a/drivers/net/hns3/hns3_ethdev.c
+++ b/drivers/net/hns3/hns3_ethdev.c
@@ -5528,11 +5528,6 @@ hns3_dev_init(struct rte_eth_dev *eth_dev)
 			    &eth_dev->data->mac_addrs[0]);
 
 	hw->adapter_state = HNS3_NIC_INITIALIZED;
-	/*
-	 * Pass the information to the rte_eth_dev_close() that it should also
-	 * release the private port resources.
-	 */
-	eth_dev->data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
 
 	if (rte_atomic16_read(&hns->hw.reset.schedule) == SCHEDULE_PENDING) {
 		hns3_err(hw, "Reschedule reset service after dev_init");
diff --git a/drivers/net/hns3/hns3_ethdev_vf.c b/drivers/net/hns3/hns3_ethdev_vf.c
index b732bbc732..7b97259c5e 100644
--- a/drivers/net/hns3/hns3_ethdev_vf.c
+++ b/drivers/net/hns3/hns3_ethdev_vf.c
@@ -2576,11 +2576,6 @@ hns3vf_dev_init(struct rte_eth_dev *eth_dev)
 	rte_ether_addr_copy((struct rte_ether_addr *)hw->mac.mac_addr,
 			    &eth_dev->data->mac_addrs[0]);
 	hw->adapter_state = HNS3_NIC_INITIALIZED;
-	/*
-	 * Pass the information to the rte_eth_dev_close() that it should also
-	 * release the private port resources.
-	 */
-	eth_dev->data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
 
 	if (rte_atomic16_read(&hns->hw.reset.schedule) == SCHEDULE_PENDING) {
 		hns3_err(hw, "Reschedule reset service after dev_init");
diff --git a/drivers/net/i40e/i40e_ethdev.c b/drivers/net/i40e/i40e_ethdev.c
index 8dccb513d4..dd8214d7be 100644
--- a/drivers/net/i40e/i40e_ethdev.c
+++ b/drivers/net/i40e/i40e_ethdev.c
@@ -1699,11 +1699,6 @@ eth_i40e_dev_init(struct rte_eth_dev *dev, void *init_params __rte_unused)
 	rte_ether_addr_copy((struct rte_ether_addr *)hw->mac.perm_addr,
 					&dev->data->mac_addrs[0]);
 
-	/* Pass the information to the rte_eth_dev_close() that it should also
-	 * release the private port resources.
-	 */
-	dev->data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
-
 	/* Init dcb to sw mode by default */
 	ret = i40e_dcb_init_configure(dev, TRUE);
 	if (ret != I40E_SUCCESS) {
diff --git a/drivers/net/i40e/i40e_ethdev_vf.c b/drivers/net/i40e/i40e_ethdev_vf.c
index 104eef639d..63dcff841e 100644
--- a/drivers/net/i40e/i40e_ethdev_vf.c
+++ b/drivers/net/i40e/i40e_ethdev_vf.c
@@ -1543,11 +1543,6 @@ i40evf_dev_init(struct rte_eth_dev *eth_dev)
 	hw->adapter_stopped = 1;
 	hw->adapter_closed = 0;
 
-	/* Pass the information to the rte_eth_dev_close() that it should also
-	 * release the private port resources.
-	 */
-	eth_dev->data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
-
 	if(i40evf_init_vf(eth_dev) != 0) {
 		PMD_INIT_LOG(ERR, "Init vf failed");
 		return -1;
diff --git a/drivers/net/iavf/iavf_ethdev.c b/drivers/net/iavf/iavf_ethdev.c
index f7552f33db..2a64204689 100644
--- a/drivers/net/iavf/iavf_ethdev.c
+++ b/drivers/net/iavf/iavf_ethdev.c
@@ -1362,11 +1362,6 @@ iavf_dev_init(struct rte_eth_dev *eth_dev)
 	adapter->eth_dev = eth_dev;
 	adapter->stopped = 1;
 
-	/* Pass the information to the rte_eth_dev_close() that it should also
-	 * release the private port resources.
-	 */
-	eth_dev->data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
-
 	if (iavf_init_vf(eth_dev) != 0) {
 		PMD_INIT_LOG(ERR, "Init vf failed");
 		return -1;
diff --git a/drivers/net/ice/ice_dcf_ethdev.c b/drivers/net/ice/ice_dcf_ethdev.c
index 5b626cb641..33dd0c44f2 100644
--- a/drivers/net/ice/ice_dcf_ethdev.c
+++ b/drivers/net/ice/ice_dcf_ethdev.c
@@ -908,8 +908,6 @@ ice_dcf_dev_init(struct rte_eth_dev *eth_dev)
 	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
 		return 0;
 
-	eth_dev->data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
-
 	adapter->real_hw.vc_event_msg_cb = ice_dcf_handle_pf_event_msg;
 	if (ice_dcf_init_hw(eth_dev, &adapter->real_hw) != 0) {
 		PMD_INIT_LOG(ERR, "Failed to init DCF hardware");
diff --git a/drivers/net/ice/ice_ethdev.c b/drivers/net/ice/ice_ethdev.c
index 042ee6def0..f0cb05cbc9 100644
--- a/drivers/net/ice/ice_ethdev.c
+++ b/drivers/net/ice/ice_ethdev.c
@@ -2206,11 +2206,6 @@ ice_dev_init(struct rte_eth_dev *dev)
 		goto err_init_mac;
 	}
 
-	/* Pass the information to the rte_eth_dev_close() that it should also
-	 * release the private port resources.
-	 */
-	dev->data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
-
 	ret = ice_res_pool_init(&pf->msix_pool, 1,
 				hw->func_caps.common_cap.num_msix_vectors - 1);
 	if (ret) {
diff --git a/drivers/net/igc/igc_ethdev.c b/drivers/net/igc/igc_ethdev.c
index 99014a369c..a9dda1f781 100644
--- a/drivers/net/igc/igc_ethdev.c
+++ b/drivers/net/igc/igc_ethdev.c
@@ -1324,11 +1324,6 @@ eth_igc_dev_init(struct rte_eth_dev *dev)
 		goto err_late;
 	}
 
-	/* Pass the information to the rte_eth_dev_close() that it should also
-	 * release the private port resources.
-	 */
-	dev->data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
-
 	hw->mac.get_link_status = 1;
 	igc->stopped = 0;
 
diff --git a/drivers/net/ixgbe/ixgbe_ethdev.c b/drivers/net/ixgbe/ixgbe_ethdev.c
index 5c12a575c2..195383ce1f 100644
--- a/drivers/net/ixgbe/ixgbe_ethdev.c
+++ b/drivers/net/ixgbe/ixgbe_ethdev.c
@@ -1251,11 +1251,6 @@ eth_ixgbe_dev_init(struct rte_eth_dev *eth_dev, void *init_params __rte_unused)
 		return -ENOMEM;
 	}
 
-	/* Pass the information to the rte_eth_dev_close() that it should also
-	 * release the private port resources.
-	 */
-	eth_dev->data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
-
 	/* initialize the vfta */
 	memset(shadow_vfta, 0, sizeof(*shadow_vfta));
 
@@ -1663,11 +1658,6 @@ eth_ixgbevf_dev_init(struct rte_eth_dev *eth_dev)
 		return -ENOMEM;
 	}
 
-	/* Pass the information to the rte_eth_dev_close() that it should also
-	 * release the private port resources.
-	 */
-	eth_dev->data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
-
 	/* Generate a random MAC address, if none was assigned by PF. */
 	if (rte_is_zero_ether_addr(perm_addr)) {
 		generate_random_mac_addr(perm_addr);
diff --git a/drivers/net/kni/rte_eth_kni.c b/drivers/net/kni/rte_eth_kni.c
index aa365a57de..45ab1b17a8 100644
--- a/drivers/net/kni/rte_eth_kni.c
+++ b/drivers/net/kni/rte_eth_kni.c
@@ -387,8 +387,6 @@ eth_kni_create(struct rte_vdev_device *vdev,
 	data->promiscuous = 1;
 	data->all_multicast = 1;
 
-	data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
-
 	rte_eth_random_addr(internals->eth_addr.addr_bytes);
 
 	eth_dev->dev_ops = &eth_kni_ops;
diff --git a/drivers/net/memif/rte_eth_memif.c b/drivers/net/memif/rte_eth_memif.c
index c00b9702a9..666dec77c6 100644
--- a/drivers/net/memif/rte_eth_memif.c
+++ b/drivers/net/memif/rte_eth_memif.c
@@ -1512,9 +1512,6 @@ memif_create(struct rte_vdev_device *vdev, enum memif_role_t role,
 		eth_dev->tx_pkt_burst = eth_memif_tx;
 	}
 
-
-	eth_dev->data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
-
 	rte_eth_dev_probing_finish(eth_dev);
 
 	return 0;
diff --git a/drivers/net/mlx4/mlx4.c b/drivers/net/mlx4/mlx4.c
index ad7c805d67..3e57875414 100644
--- a/drivers/net/mlx4/mlx4.c
+++ b/drivers/net/mlx4/mlx4.c
@@ -1027,7 +1027,6 @@ mlx4_pci_probe(struct rte_pci_driver *pci_drv, struct rte_pci_device *pci_dev)
 			ERROR("can not allocate rte ethdev");
 			goto port_error;
 		}
-		eth_dev->data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
 		eth_dev->data->dev_private = priv;
 		eth_dev->data->mac_addrs = priv->mac;
 		eth_dev->device = &pci_dev->device;
diff --git a/drivers/net/mlx5/linux/mlx5_os.c b/drivers/net/mlx5/linux/mlx5_os.c
index 69123e12c3..7fce24cafc 100644
--- a/drivers/net/mlx5/linux/mlx5_os.c
+++ b/drivers/net/mlx5/linux/mlx5_os.c
@@ -1136,8 +1136,6 @@ mlx5_dev_spawn(struct rte_device *dpdk_dev,
 		err = ENOMEM;
 		goto error;
 	}
-	/* Flag to call rte_eth_dev_release_port() in rte_eth_dev_close(). */
-	eth_dev->data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
 	if (priv->representor) {
 		eth_dev->data->dev_flags |= RTE_ETH_DEV_REPRESENTOR;
 		eth_dev->data->representor_id = priv->representor_id;
diff --git a/drivers/net/mvneta/mvneta_ethdev.c b/drivers/net/mvneta/mvneta_ethdev.c
index b13e2c52f3..f0a52e29fd 100644
--- a/drivers/net/mvneta/mvneta_ethdev.c
+++ b/drivers/net/mvneta/mvneta_ethdev.c
@@ -835,9 +835,6 @@ mvneta_eth_dev_create(struct rte_vdev_device *vdev, const char *name)
 	mvneta_set_tx_function(eth_dev);
 	eth_dev->dev_ops = &mvneta_ops;
 
-	/* Flag to call rte_eth_dev_release_port() in rte_eth_dev_close(). */
-	eth_dev->data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
-
 	rte_eth_dev_probing_finish(eth_dev);
 	return 0;
 out_free:
diff --git a/drivers/net/mvpp2/mrvl_ethdev.c b/drivers/net/mvpp2/mrvl_ethdev.c
index b40169daba..e115e7d0c1 100644
--- a/drivers/net/mvpp2/mrvl_ethdev.c
+++ b/drivers/net/mvpp2/mrvl_ethdev.c
@@ -2864,9 +2864,6 @@ mrvl_eth_dev_create(struct rte_vdev_device *vdev, const char *name)
 	mrvl_set_tx_function(eth_dev);
 	eth_dev->dev_ops = &mrvl_ops;
 
-	/* Flag to call rte_eth_dev_release_port() in rte_eth_dev_close(). */
-	eth_dev->data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
-
 	rte_eth_dev_probing_finish(eth_dev);
 	return 0;
 out_free:
diff --git a/drivers/net/netvsc/hn_ethdev.c b/drivers/net/netvsc/hn_ethdev.c
index f02f546882..c4a2dd9f4a 100644
--- a/drivers/net/netvsc/hn_ethdev.c
+++ b/drivers/net/netvsc/hn_ethdev.c
@@ -123,9 +123,6 @@ eth_dev_vmbus_allocate(struct rte_vmbus_device *dev, size_t private_data_size)
 	eth_dev->data->dev_flags |= RTE_ETH_DEV_INTR_LSC;
 	eth_dev->intr_handle = &dev->intr_handle;
 
-	/* allow ethdev to remove on close */
-	eth_dev->data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
-
 	return eth_dev;
 }
 
diff --git a/drivers/net/nfb/nfb_ethdev.c b/drivers/net/nfb/nfb_ethdev.c
index c3c3d003f1..d937ac6922 100644
--- a/drivers/net/nfb/nfb_ethdev.c
+++ b/drivers/net/nfb/nfb_ethdev.c
@@ -457,9 +457,6 @@ nfb_eth_dev_init(struct rte_eth_dev *dev)
 		rte_kvargs_free(kvlist);
 	}
 
-	/* Let rte_eth_dev_close() release the port resources */
-	dev->data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
-
 	/*
 	 * Get number of available DMA RX and TX queues, which is maximum
 	 * number of queues that can be created and store it in private device
diff --git a/drivers/net/null/rte_eth_null.c b/drivers/net/null/rte_eth_null.c
index 33997013e4..ab7ebe3ddc 100644
--- a/drivers/net/null/rte_eth_null.c
+++ b/drivers/net/null/rte_eth_null.c
@@ -545,7 +545,6 @@ eth_dev_null_create(struct rte_vdev_device *dev, struct pmd_options *args)
 	data->mac_addrs = &internals->eth_addr;
 	data->promiscuous = 1;
 	data->all_multicast = 1;
-	data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
 
 	eth_dev->dev_ops = &ops;
 
diff --git a/drivers/net/octeontx/octeontx_ethdev.c b/drivers/net/octeontx/octeontx_ethdev.c
index 6713c51504..328551b60f 100644
--- a/drivers/net/octeontx/octeontx_ethdev.c
+++ b/drivers/net/octeontx/octeontx_ethdev.c
@@ -1375,7 +1375,6 @@ octeontx_create(struct rte_vdev_device *dev, int port, uint8_t evdev,
 	data->promiscuous = 0;
 	data->all_multicast = 0;
 	data->scattered_rx = 0;
-	data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
 
 	/* Get maximum number of supported MAC entries */
 	max_entries = octeontx_bgx_port_mac_entries_get(nic->port_id);
diff --git a/drivers/net/octeontx2/otx2_ethdev.c b/drivers/net/octeontx2/otx2_ethdev.c
index d9bcf004c6..ca320047f6 100644
--- a/drivers/net/octeontx2/otx2_ethdev.c
+++ b/drivers/net/octeontx2/otx2_ethdev.c
@@ -2394,7 +2394,6 @@ otx2_eth_dev_init(struct rte_eth_dev *eth_dev)
 	pci_dev = RTE_ETH_DEV_TO_PCI(eth_dev);
 
 	rte_eth_copy_pci_info(eth_dev, pci_dev);
-	eth_dev->data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
 
 	/* Zero out everything after OTX2_DEV to allow proper dev_reset() */
 	memset(&dev->otx2_eth_dev_data_start, 0, sizeof(*dev) -
diff --git a/drivers/net/pcap/rte_eth_pcap.c b/drivers/net/pcap/rte_eth_pcap.c
index a946fa9a1a..fbbad5cbd1 100644
--- a/drivers/net/pcap/rte_eth_pcap.c
+++ b/drivers/net/pcap/rte_eth_pcap.c
@@ -1334,7 +1334,6 @@ eth_from_pcaps(struct rte_vdev_device *vdev,
 	else
 		eth_dev->tx_pkt_burst = eth_tx_drop;
 
-	eth_dev->data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
 	rte_eth_dev_probing_finish(eth_dev);
 	return 0;
 }
diff --git a/drivers/net/ring/rte_eth_ring.c b/drivers/net/ring/rte_eth_ring.c
index afe3ec46cc..8ed53c64af 100644
--- a/drivers/net/ring/rte_eth_ring.c
+++ b/drivers/net/ring/rte_eth_ring.c
@@ -356,7 +356,6 @@ do_eth_dev_ring_create(const char *name,
 	eth_dev->dev_ops = &ops;
 	data->kdrv = RTE_KDRV_NONE;
 	data->numa_node = numa_node;
-	data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
 
 	/* finally assign rx and tx ops */
 	eth_dev->rx_pkt_burst = eth_ring_rx;
diff --git a/drivers/net/sfc/sfc_ethdev.c b/drivers/net/sfc/sfc_ethdev.c
index f1b857358c..b609d808d7 100644
--- a/drivers/net/sfc/sfc_ethdev.c
+++ b/drivers/net/sfc/sfc_ethdev.c
@@ -343,7 +343,7 @@ sfc_dev_close(struct rte_eth_dev *dev)
 	}
 
 	/*
-	 * Cleanup all resources in accordance with RTE_ETH_DEV_CLOSE_REMOVE.
+	 * Cleanup all resources.
 	 * Rollback primary process sfc_eth_dev_init() below.
 	 */
 
@@ -2179,8 +2179,6 @@ sfc_eth_dev_init(struct rte_eth_dev *dev)
 
 	sfc_log_init(sa, "entry");
 
-	dev->data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
-
 	dev->data->mac_addrs = rte_zmalloc("sfc", RTE_ETHER_ADDR_LEN, 0);
 	if (dev->data->mac_addrs == NULL) {
 		rc = ENOMEM;
diff --git a/drivers/net/softnic/rte_eth_softnic.c b/drivers/net/softnic/rte_eth_softnic.c
index 5b94e250d8..0e5ec0cdf0 100644
--- a/drivers/net/softnic/rte_eth_softnic.c
+++ b/drivers/net/softnic/rte_eth_softnic.c
@@ -387,7 +387,6 @@ pmd_ethdev_register(struct rte_vdev_device *vdev,
 	dev->device = &vdev->device;
 
 	/* dev->data */
-	dev->data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
 	dev->data->dev_private = dev_private;
 	dev->data->dev_link.link_speed = ETH_SPEED_NUM_100G;
 	dev->data->dev_link.link_duplex = ETH_LINK_FULL_DUPLEX;
diff --git a/drivers/net/szedata2/rte_eth_szedata2.c b/drivers/net/szedata2/rte_eth_szedata2.c
index daa40cfd6c..4325b9a30d 100644
--- a/drivers/net/szedata2/rte_eth_szedata2.c
+++ b/drivers/net/szedata2/rte_eth_szedata2.c
@@ -1488,9 +1488,6 @@ rte_szedata2_eth_dev_init(struct rte_eth_dev *dev, struct port_info *pi)
 	PMD_INIT_LOG(INFO, "Initializing eth_dev %s (driver %s)", data->name,
 			RTE_STR(RTE_SZEDATA2_DRIVER_NAME));
 
-	/* Let rte_eth_dev_close() release the port resources */
-	dev->data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
-
 	/* Fill internal private structure. */
 	internals->dev = dev;
 	/* Get index of szedata2 device file and create path to device file */
diff --git a/drivers/net/tap/rte_eth_tap.c b/drivers/net/tap/rte_eth_tap.c
index 2e325f55aa..99f6dc118c 100644
--- a/drivers/net/tap/rte_eth_tap.c
+++ b/drivers/net/tap/rte_eth_tap.c
@@ -1876,7 +1876,7 @@ eth_dev_tap_create(struct rte_vdev_device *vdev, const char *tap_name,
 	/* Setup some default values */
 	data = dev->data;
 	data->dev_private = pmd;
-	data->dev_flags = RTE_ETH_DEV_INTR_LSC | RTE_ETH_DEV_CLOSE_REMOVE;
+	data->dev_flags = RTE_ETH_DEV_INTR_LSC;
 	data->numa_node = numa_node;
 
 	data->dev_link = pmd_link;
diff --git a/drivers/net/vhost/rte_eth_vhost.c b/drivers/net/vhost/rte_eth_vhost.c
index fb3b9ce043..36baf8f881 100644
--- a/drivers/net/vhost/rte_eth_vhost.c
+++ b/drivers/net/vhost/rte_eth_vhost.c
@@ -1444,7 +1444,7 @@ eth_dev_vhost_create(struct rte_vdev_device *dev, char *iface_name,
 	internal->flags = flags;
 	internal->disable_flags = disable_flags;
 	data->dev_link = pmd_link;
-	data->dev_flags = RTE_ETH_DEV_INTR_LSC | RTE_ETH_DEV_CLOSE_REMOVE;
+	data->dev_flags = RTE_ETH_DEV_INTR_LSC;
 	data->promiscuous = 1;
 	data->all_multicast = 1;
 
diff --git a/drivers/net/virtio/virtio_ethdev.c b/drivers/net/virtio/virtio_ethdev.c
index 1731fc6750..735c579a86 100644
--- a/drivers/net/virtio/virtio_ethdev.c
+++ b/drivers/net/virtio/virtio_ethdev.c
@@ -1923,11 +1923,6 @@ eth_virtio_dev_init(struct rte_eth_dev *eth_dev)
 	if (ret < 0)
 		return ret;
 	hw->speed = speed;
-	/*
-	 * Pass the information to the rte_eth_dev_close() that it should also
-	 * release the private port resources.
-	 */
-	eth_dev->data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
 
 	/* Allocate memory for storing MAC addresses */
 	eth_dev->data->mac_addrs = rte_zmalloc("virtio",
diff --git a/drivers/net/vmxnet3/vmxnet3_ethdev.c b/drivers/net/vmxnet3/vmxnet3_ethdev.c
index 4efb48350c..221f771023 100644
--- a/drivers/net/vmxnet3/vmxnet3_ethdev.c
+++ b/drivers/net/vmxnet3/vmxnet3_ethdev.c
@@ -321,9 +321,6 @@ eth_vmxnet3_dev_init(struct rte_eth_dev *eth_dev)
 		     hw->perm_addr[0], hw->perm_addr[1], hw->perm_addr[2],
 		     hw->perm_addr[3], hw->perm_addr[4], hw->perm_addr[5]);
 
-	/* Flag to call rte_eth_dev_release_port() in rte_eth_dev_close(). */
-	eth_dev->data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
-
 	/* Put device in Quiesce Mode */
 	VMXNET3_WRITE_BAR1_REG(hw, VMXNET3_REG_CMD, VMXNET3_CMD_QUIESCE_DEV);
 
diff --git a/lib/librte_ethdev/rte_ethdev.c b/lib/librte_ethdev/rte_ethdev.c
index 3617f39ca0..f1b0bdd3bc 100644
--- a/lib/librte_ethdev/rte_ethdev.c
+++ b/lib/librte_ethdev/rte_ethdev.c
@@ -1720,22 +1720,7 @@ rte_eth_dev_close(uint16_t port_id)
 	(*dev->dev_ops->dev_close)(dev);
 
 	rte_ethdev_trace_close(port_id);
-	/* check behaviour flag - temporary for PMD migration */
-	if ((dev->data->dev_flags & RTE_ETH_DEV_CLOSE_REMOVE) != 0) {
-		/* new behaviour: send event + reset state + free all data */
-		rte_eth_dev_release_port(dev);
-		return;
-	}
-	RTE_ETHDEV_LOG(DEBUG, "Port closing is using an old behaviour.\n"
-			"The driver %s should migrate to the new behaviour.\n",
-			dev->device->driver->name);
-	/* old behaviour: only free queue arrays */
-	dev->data->nb_rx_queues = 0;
-	rte_free(dev->data->rx_queues);
-	dev->data->rx_queues = NULL;
-	dev->data->nb_tx_queues = 0;
-	rte_free(dev->data->tx_queues);
-	dev->data->tx_queues = NULL;
+	rte_eth_dev_release_port(dev);
 }
 
 int
diff --git a/lib/librte_ethdev/rte_ethdev.h b/lib/librte_ethdev/rte_ethdev.h
index 70295d7ab7..f9775c2524 100644
--- a/lib/librte_ethdev/rte_ethdev.h
+++ b/lib/librte_ethdev/rte_ethdev.h
@@ -1621,11 +1621,6 @@ struct rte_eth_dev_owner {
 	char name[RTE_ETH_MAX_OWNER_NAME_LEN]; /**< The owner name. */
 };
 
-/**
- * Port is released (i.e. totally freed and data erased) on close.
- * Temporary flag for PMD migration to new rte_eth_dev_close() behaviour.
- */
-#define RTE_ETH_DEV_CLOSE_REMOVE 0x0001
 /** Device supports link state interrupt */
 #define RTE_ETH_DEV_INTR_LSC     0x0002
 /** Device is a bonded slave */
@@ -2250,8 +2245,7 @@ int rte_eth_dev_set_link_down(uint16_t port_id);
 
 /**
  * Close a stopped Ethernet device. The device cannot be restarted!
- * The function frees all port resources if the driver supports
- * the flag RTE_ETH_DEV_CLOSE_REMOVE.
+ * The function frees all port resources.
  *
  * @param port_id
  *   The port identifier of the Ethernet device.
-- 
2.28.0

Re: [dpdk-dev] [PATCH 16/20] ethdev: remove old close behaviour

[Ferruh Yigit]

On 9/13/2020 11:07 PM, Thomas Monjalon wrote:
> The temporary flag RTE_ETH_DEV_CLOSE_REMOVE is removed.
> It was introduced in DPDK 18.11 in order to give time for PMDs to migrate.
> 
> The old behaviour was to free only queues when closing a port.
> The new behaviour is calling rte_eth_dev_release_port() which does
> three more tasks:
> 	- trigger event callback
> 	- reset state and few pointers
> 	- free all generic port resources
> 
> The private port resources must be released in the .dev_close callback.
> 
> The .remove callback should:
> 	- call .dev_close callback
> 	- call rte_eth_dev_release_port()
> 	- free multi-port device shared resources
> 
> Despite waiting two years, some drivers have not migrated,
> so they may hit issues with the incompatible new behaviour.
> After sending emails, adding logs, and announcing the deprecation,
> the only last solution is to declare these drivers as unmaintained:
> 	bnx2x, cxgbe, dpaa, dpaa2, enetc, ionic,
> 	ipn3ke,	liquidio, nfp, pfe, qede
> Below is a summary of what to implement in those drivers.
> 
> * The freeing of private port resources must be moved
> from the ".remove(device)" function to the ".dev_close(port)" function.
> 
> * If a generic resource (.mac_addrs or .hash_mac_addrs) cannot be freed,
> it must be set to NULL in ".dev_close" function to protect from
> subsequent rte_eth_dev_release_port() freeing.
> 
> * Note 1:
> The generic resources are freed in rte_eth_dev_release_port(),
> after ".dev_close" is called in rte_eth_dev_close(), but not when
> calling ".dev_close" directly from the ".remove" PMD function.
> That's why rte_eth_dev_release_port() must still be called explicitly
> from ".remove(device)" after calling the ".dev_close" PMD function.
> 
> * Note 2:
> If a device can have multiple ports, the common resources must be freed
> only in the ".remove(device)" function.
> 
> * Note 3:
> The port is supposed to be in a stopped state when it is closed.
> If it is not the case, it is free to the PMD implementation
> how to react when trying to close a non-stopped port:
> either try to stop it automatically or just return an error.

For this note, 'rte_eth_dev_close()' assumes port stopped and sets 
"dev->data->dev_started = 0;" blindly.

should we verify this, or perhaps should call 'rte_eth_dev_stop()' 
within the 'rte_eth_dev_close()' to be sure.


Also many PMDs doesn't check primary process check in the close, which 
should be fixed too, I wonder if it can be fixed in this series?


> -Cavium LiquidIO
> +Cavium LiquidIO - UNMAINTAINED
>   M: Shijith Thotton <sthotton@marvell.com>
>   M: Srisivasubramanian Srinivasan <srinivasan@marvell.com>
>   T: git://dpdk.org/next/dpdk-next-net-mrvl
> @@ -622,7 +622,7 @@ F: drivers/net/octeontx/
>   F: doc/guides/nics/octeontx.rst
>   F: doc/guides/nics/features/octeontx.ini
>   
> -Chelsio cxgbe
> +Chelsio cxgbe - UNMAINTAINED
>   M: Rahul Lakkireddy <rahul.lakkireddy@chelsio.com>
>   F: drivers/net/cxgbe/
>   F: doc/guides/nics/cxgbe.rst
> @@ -711,7 +711,7 @@ F: drivers/net/igc/
>   F: doc/guides/nics/igc.rst
>   F: doc/guides/nics/features/igc.ini
>   
> -Intel ipn3ke
> +Intel ipn3ke - UNMAINTAINED
>   M: Rosen Xu <rosen.xu@intel.com>
>   T: git://dpdk.org/next/dpdk-next-net-intel
>   F: drivers/net/ipn3ke/
> @@ -794,13 +794,13 @@ F: drivers/net/nfb/
>   F: doc/guides/nics/nfb.rst
>   F: doc/guides/nics/features/nfb.ini
>   
> -Netronome nfp
> +Netronome nfp - UNMAINTAINED
>   M: Heinrich Kuhn <heinrich.kuhn@netronome.com>
>   F: drivers/net/nfp/
>   F: doc/guides/nics/nfp.rst
>   F: doc/guides/nics/features/nfp*.ini
>   
> -NXP dpaa
> +NXP dpaa - UNMAINTAINED
>   M: Hemant Agrawal <hemant.agrawal@nxp.com>
>   M: Sachin Saxena <sachin.saxena@nxp.com>
>   F: drivers/mempool/dpaa/
> @@ -808,7 +808,7 @@ F: drivers/net/dpaa/
>   F: doc/guides/nics/dpaa.rst
>   F: doc/guides/nics/features/dpaa.ini
>   
> -NXP dpaa2
> +NXP dpaa2 - UNMAINTAINED
>   M: Hemant Agrawal <hemant.agrawal@nxp.com>
>   M: Sachin Saxena <sachin.saxena@nxp.com>
>   F: drivers/mempool/dpaa2/
> @@ -816,27 +816,27 @@ F: drivers/net/dpaa2/
>   F: doc/guides/nics/dpaa2.rst
>   F: doc/guides/nics/features/dpaa2.ini
>   
> -NXP enetc
> +NXP enetc - UNMAINTAINED
>   M: Gagandeep Singh <g.singh@nxp.com>
>   M: Sachin Saxena <sachin.saxena@nxp.com>
>   F: drivers/net/enetc/
>   F: doc/guides/nics/enetc.rst
>   F: doc/guides/nics/features/enetc.ini
>   
> -NXP pfe
> +NXP pfe - UNMAINTAINED
>   M: Gagandeep Singh <g.singh@nxp.com>
>   M: Akhil Goyal <akhil.goyal@nxp.com>
>   F: doc/guides/nics/pfe.rst
>   F: drivers/net/pfe/
>   F: doc/guides/nics/features/pfe.ini
>   
> -Pensando ionic
> +Pensando ionic - UNMAINTAINED
>   M: Alfredo Cardigliano <cardigliano@ntop.org>
>   F: drivers/net/ionic/
>   F: doc/guides/nics/ionic.rst
>   F: doc/guides/nics/features/ionic.ini
>   
> -QLogic bnx2x
> +QLogic bnx2x - UNMAINTAINED
>   M: Rasesh Mody <rmody@marvell.com>
>   M: Shahed Shaikh <shshaikh@marvell.com>
>   T: git://dpdk.org/next/dpdk-next-net-mrvl
> @@ -844,7 +844,7 @@ F: drivers/net/bnx2x/
>   F: doc/guides/nics/bnx2x.rst
>   F: doc/guides/nics/features/bnx2x*.ini
>   
> -QLogic qede PMD
> +QLogic qede PMD - UNMAINTAINED

I guess 'cxgbe' already send the patch, but for rest, is there a 
deadline for the patches, before mark them as "UNMAINTAINED"?

Re: [dpdk-dev] [PATCH 16/20] ethdev: remove old close behaviour

[Thomas Monjalon]

23/09/2020 18:44, Ferruh Yigit:
> On 9/13/2020 11:07 PM, Thomas Monjalon wrote:
> > * Note 3:
> > The port is supposed to be in a stopped state when it is closed.
> > If it is not the case, it is free to the PMD implementation
> > how to react when trying to close a non-stopped port:
> > either try to stop it automatically or just return an error.
> 
> For this note, 'rte_eth_dev_close()' assumes port stopped and sets 
> "dev->data->dev_started = 0;" blindly.
> 
> should we verify this, or perhaps should call 'rte_eth_dev_stop()' 
> within the 'rte_eth_dev_close()' to be sure.

Good point. I wonder why dev_started is changed in the close.
It is set in rte_eth_dev_stop(), that should be enough.
But clearly it is a different issue which should be adressed separately.

> Also many PMDs doesn't check primary process check in the close, which 
> should be fixed too, I wonder if it can be fixed in this series?

You mean forbid close operation from secondary process?

Re: [dpdk-dev] [PATCH 16/20] ethdev: remove old close behaviour

[Ferruh Yigit]

On 9/23/2020 9:41 PM, Thomas Monjalon wrote:
> 23/09/2020 18:44, Ferruh Yigit:
>> On 9/13/2020 11:07 PM, Thomas Monjalon wrote:
>>> * Note 3:
>>> The port is supposed to be in a stopped state when it is closed.
>>> If it is not the case, it is free to the PMD implementation
>>> how to react when trying to close a non-stopped port:
>>> either try to stop it automatically or just return an error.
>>
>> For this note, 'rte_eth_dev_close()' assumes port stopped and sets
>> "dev->data->dev_started = 0;" blindly.
>>
>> should we verify this, or perhaps should call 'rte_eth_dev_stop()'
>> within the 'rte_eth_dev_close()' to be sure.
> 
> Good point. I wonder why dev_started is changed in the close.
> It is set in rte_eth_dev_stop(), that should be enough.
> But clearly it is a different issue which should be adressed separately.
> 

OK

>> Also many PMDs doesn't check primary process check in the close, which
>> should be fixed too, I wonder if it can be fixed in this series?
> 
> You mean forbid close operation from secondary process?
> 

No need to forbid I think, and the not shared resource cleanup for 
secondary can be done in close function, but PMDs should have checks for 
the close calls from secondary processes to not free everything.

Re: [dpdk-dev] [PATCH 16/20] ethdev: remove old close behaviour

[Rasesh Mody]

Hi Thomas, Ferruh,

>From: dev <dev-bounces@dpdk.org> On Behalf Of Ferruh Yigit
>Sent: Wednesday, September 23, 2020 9:45 AM
>
>On 9/13/2020 11:07 PM, Thomas Monjalon wrote:
>> The temporary flag RTE_ETH_DEV_CLOSE_REMOVE is removed.
>> It was introduced in DPDK 18.11 in order to give time for PMDs to migrate.
>>
>> The old behaviour was to free only queues when closing a port.
>> The new behaviour is calling rte_eth_dev_release_port() which does
>> three more tasks:
>> 	- trigger event callback
>> 	- reset state and few pointers
>> 	- free all generic port resources
>>
>> The private port resources must be released in the .dev_close callback.
>>
>> The .remove callback should:
>> 	- call .dev_close callback
>> 	- call rte_eth_dev_release_port()
>> 	- free multi-port device shared resources
>>
>> Despite waiting two years, some drivers have not migrated, so they may
>> hit issues with the incompatible new behaviour.
>> After sending emails, adding logs, and announcing the deprecation, the
>> only last solution is to declare these drivers as unmaintained:
>> 	bnx2x, cxgbe, dpaa, dpaa2, enetc, ionic,
>> 	ipn3ke,	liquidio, nfp, pfe, qede
>> Below is a summary of what to implement in those drivers.
>>
>> * The freeing of private port resources must be moved from the
>> ".remove(device)" function to the ".dev_close(port)" function.
>>
>> * If a generic resource (.mac_addrs or .hash_mac_addrs) cannot be
>> freed, it must be set to NULL in ".dev_close" function to protect from
>> subsequent rte_eth_dev_release_port() freeing.
>>
>> * Note 1:
>> The generic resources are freed in rte_eth_dev_release_port(), after
>> ".dev_close" is called in rte_eth_dev_close(), but not when calling
>> ".dev_close" directly from the ".remove" PMD function.
>> That's why rte_eth_dev_release_port() must still be called explicitly
>> from ".remove(device)" after calling the ".dev_close" PMD function.
>>
>> * Note 2:
>> If a device can have multiple ports, the common resources must be
>> freed only in the ".remove(device)" function.
>>
>> * Note 3:
>> The port is supposed to be in a stopped state when it is closed.
>> If it is not the case, it is free to the PMD implementation how to
>> react when trying to close a non-stopped port:
>> either try to stop it automatically or just return an error.
>
>For this note, 'rte_eth_dev_close()' assumes port stopped and sets "dev-
>>data->dev_started = 0;" blindly.
>
>should we verify this, or perhaps should call 'rte_eth_dev_stop()'
>within the 'rte_eth_dev_close()' to be sure.
>
>
>Also many PMDs doesn't check primary process check in the close, which
>should be fixed too, I wonder if it can be fixed in this series?
>
>
>> -Cavium LiquidIO
>> +Cavium LiquidIO - UNMAINTAINED
>>   M: Shijith Thotton <sthotton@marvell.com>
>>   M: Srisivasubramanian Srinivasan <srinivasan@marvell.com>
>>   T: git://dpdk.org/next/dpdk-next-net-mrvl
>> @@ -622,7 +622,7 @@ F: drivers/net/octeontx/
>>   F: doc/guides/nics/octeontx.rst
>>   F: doc/guides/nics/features/octeontx.ini
>>
>> -Chelsio cxgbe
>> +Chelsio cxgbe - UNMAINTAINED
>>   M: Rahul Lakkireddy <rahul.lakkireddy@chelsio.com>
>>   F: drivers/net/cxgbe/
>>   F: doc/guides/nics/cxgbe.rst
>> @@ -711,7 +711,7 @@ F: drivers/net/igc/
>>   F: doc/guides/nics/igc.rst
>>   F: doc/guides/nics/features/igc.ini
>>
>> -Intel ipn3ke
>> +Intel ipn3ke - UNMAINTAINED
>>   M: Rosen Xu <rosen.xu@intel.com>
>>   T: git://dpdk.org/next/dpdk-next-net-intel
>>   F: drivers/net/ipn3ke/
>> @@ -794,13 +794,13 @@ F: drivers/net/nfb/
>>   F: doc/guides/nics/nfb.rst
>>   F: doc/guides/nics/features/nfb.ini
>>
>> -Netronome nfp
>> +Netronome nfp - UNMAINTAINED
>>   M: Heinrich Kuhn <heinrich.kuhn@netronome.com>
>>   F: drivers/net/nfp/
>>   F: doc/guides/nics/nfp.rst
>>   F: doc/guides/nics/features/nfp*.ini
>>
>> -NXP dpaa
>> +NXP dpaa - UNMAINTAINED
>>   M: Hemant Agrawal <hemant.agrawal@nxp.com>
>>   M: Sachin Saxena <sachin.saxena@nxp.com>
>>   F: drivers/mempool/dpaa/
>> @@ -808,7 +808,7 @@ F: drivers/net/dpaa/
>>   F: doc/guides/nics/dpaa.rst
>>   F: doc/guides/nics/features/dpaa.ini
>>
>> -NXP dpaa2
>> +NXP dpaa2 - UNMAINTAINED
>>   M: Hemant Agrawal <hemant.agrawal@nxp.com>
>>   M: Sachin Saxena <sachin.saxena@nxp.com>
>>   F: drivers/mempool/dpaa2/
>> @@ -816,27 +816,27 @@ F: drivers/net/dpaa2/
>>   F: doc/guides/nics/dpaa2.rst
>>   F: doc/guides/nics/features/dpaa2.ini
>>
>> -NXP enetc
>> +NXP enetc - UNMAINTAINED
>>   M: Gagandeep Singh <g.singh@nxp.com>
>>   M: Sachin Saxena <sachin.saxena@nxp.com>
>>   F: drivers/net/enetc/
>>   F: doc/guides/nics/enetc.rst
>>   F: doc/guides/nics/features/enetc.ini
>>
>> -NXP pfe
>> +NXP pfe - UNMAINTAINED
>>   M: Gagandeep Singh <g.singh@nxp.com>
>>   M: Akhil Goyal <akhil.goyal@nxp.com>
>>   F: doc/guides/nics/pfe.rst
>>   F: drivers/net/pfe/
>>   F: doc/guides/nics/features/pfe.ini
>>
>> -Pensando ionic
>> +Pensando ionic - UNMAINTAINED
>>   M: Alfredo Cardigliano <cardigliano@ntop.org>
>>   F: drivers/net/ionic/
>>   F: doc/guides/nics/ionic.rst
>>   F: doc/guides/nics/features/ionic.ini
>>
>> -QLogic bnx2x
>> +QLogic bnx2x - UNMAINTAINED
>>   M: Rasesh Mody <rmody@marvell.com>
>>   M: Shahed Shaikh <shshaikh@marvell.com>
>>   T: git://dpdk.org/next/dpdk-next-net-mrvl
>> @@ -844,7 +844,7 @@ F: drivers/net/bnx2x/
>>   F: doc/guides/nics/bnx2x.rst
>>   F: doc/guides/nics/features/bnx2x*.ini
>>
>> -QLogic qede PMD
>> +QLogic qede PMD - UNMAINTAINED
>
>I guess 'cxgbe' already send the patch, but for rest, is there a
>deadline for the patches, before mark them as "UNMAINTAINED"?

The pending PMD changes just dropped off the radar for quite some time. The patches for bnx2x and qede PMDs are submitted for 20.11 release, please apply.

Thanks!
-Rasesh

[dpdk-dev] [PATCH 17/20] drivers/net: accept removing device without any port

[Thomas Monjalon]

The ports can be closed (i.e. completely released)
before removing the whole device.
Such case was wrongly considered an error by some drivers.

If the device supports only one port, there is nothing much
to free after the port is closed.

Signed-off-by: Thomas Monjalon <thomas@monjalon.net>
---
 drivers/net/ipn3ke/ipn3ke_ethdev.c      |  6 ++----
 drivers/net/kni/rte_eth_kni.c           | 16 +++++++---------
 drivers/net/netvsc/hn_ethdev.c          |  2 +-
 drivers/net/nfp/nfp_net.c               |  2 ++
 drivers/net/pfe/pfe_ethdev.c            |  6 ++----
 drivers/net/szedata2/rte_eth_szedata2.c |  6 ++----
 6 files changed, 16 insertions(+), 22 deletions(-)

diff --git a/drivers/net/ipn3ke/ipn3ke_ethdev.c b/drivers/net/ipn3ke/ipn3ke_ethdev.c
index 027be29bd8..4446d2af9e 100644
--- a/drivers/net/ipn3ke/ipn3ke_ethdev.c
+++ b/drivers/net/ipn3ke/ipn3ke_ethdev.c
@@ -562,10 +562,8 @@ static int ipn3ke_vswitch_remove(struct rte_afu_device *afu_dev)
 			afu_dev->device.name, i);
 
 		ethdev = rte_eth_dev_allocated(afu_dev->device.name);
-		if (!ethdev)
-			return -ENODEV;
-
-		rte_eth_dev_destroy(ethdev, ipn3ke_rpst_uninit);
+		if (ethdev != NULL)
+			rte_eth_dev_destroy(ethdev, ipn3ke_rpst_uninit);
 	}
 
 	ret = rte_eth_switch_domain_free(hw->switch_domain_id);
diff --git a/drivers/net/kni/rte_eth_kni.c b/drivers/net/kni/rte_eth_kni.c
index 45ab1b17a8..2a4058f7b0 100644
--- a/drivers/net/kni/rte_eth_kni.c
+++ b/drivers/net/kni/rte_eth_kni.c
@@ -488,17 +488,15 @@ eth_kni_remove(struct rte_vdev_device *vdev)
 
 	/* find the ethdev entry */
 	eth_dev = rte_eth_dev_allocated(name);
-	if (eth_dev == NULL)
-		return -1;
-
-	if (rte_eal_process_type() != RTE_PROC_PRIMARY) {
-		eth_kni_dev_stop(eth_dev);
-		return rte_eth_dev_release_port(eth_dev);
+	if (eth_dev != NULL) {
+		if (rte_eal_process_type() != RTE_PROC_PRIMARY) {
+			eth_kni_dev_stop(eth_dev);
+			return rte_eth_dev_release_port(eth_dev);
+		}
+		eth_kni_close(eth_dev);
+		rte_eth_dev_release_port(eth_dev);
 	}
 
-	eth_kni_close(eth_dev);
-	rte_eth_dev_release_port(eth_dev);
-
 	is_kni_initialized--;
 	if (is_kni_initialized == 0)
 		rte_kni_close();
diff --git a/drivers/net/netvsc/hn_ethdev.c b/drivers/net/netvsc/hn_ethdev.c
index c4a2dd9f4a..4c37dc520c 100644
--- a/drivers/net/netvsc/hn_ethdev.c
+++ b/drivers/net/netvsc/hn_ethdev.c
@@ -1092,7 +1092,7 @@ static int eth_hn_remove(struct rte_vmbus_device *dev)
 
 	eth_dev = rte_eth_dev_allocated(dev->device.name);
 	if (!eth_dev)
-		return -ENODEV;
+		return 0; /* port already released */
 
 	ret = eth_hn_dev_uninit(eth_dev);
 	if (ret)
diff --git a/drivers/net/nfp/nfp_net.c b/drivers/net/nfp/nfp_net.c
index c20d71cdc3..2f108099a3 100644
--- a/drivers/net/nfp/nfp_net.c
+++ b/drivers/net/nfp/nfp_net.c
@@ -3721,6 +3721,8 @@ static int eth_nfp_pci_remove(struct rte_pci_device *pci_dev)
 	int port = 0;
 
 	eth_dev = rte_eth_dev_allocated(pci_dev->device.name);
+	if (eth_dev == NULL)
+		return 0; /* port already released */
 	if ((pci_dev->id.device_id == PCI_DEVICE_ID_NFP4000_PF_NIC) ||
 	    (pci_dev->id.device_id == PCI_DEVICE_ID_NFP6000_PF_NIC)) {
 		port = get_pf_port_number(eth_dev->data->name);
diff --git a/drivers/net/pfe/pfe_ethdev.c b/drivers/net/pfe/pfe_ethdev.c
index 8867b61a9d..7a36303785 100644
--- a/drivers/net/pfe/pfe_ethdev.c
+++ b/drivers/net/pfe/pfe_ethdev.c
@@ -1158,10 +1158,8 @@ pmd_pfe_remove(struct rte_vdev_device *vdev)
 		return 0;
 
 	eth_dev = rte_eth_dev_allocated(name);
-	if (eth_dev == NULL)
-		return -ENODEV;
-
-	pfe_eth_exit(eth_dev, g_pfe);
+	if (eth_dev != NULL)
+		pfe_eth_exit(eth_dev, g_pfe);
 	munmap(g_pfe->cbus_baseaddr, g_pfe->cbus_size);
 
 	if (g_pfe->nb_devs == 0) {
diff --git a/drivers/net/szedata2/rte_eth_szedata2.c b/drivers/net/szedata2/rte_eth_szedata2.c
index 4325b9a30d..5f589dfa4c 100644
--- a/drivers/net/szedata2/rte_eth_szedata2.c
+++ b/drivers/net/szedata2/rte_eth_szedata2.c
@@ -1910,10 +1910,8 @@ static int szedata2_eth_pci_remove(struct rte_pci_device *pci_dev)
 				pci_dev->device.name, i);
 		PMD_DRV_LOG(DEBUG, "Removing eth_dev %s", name);
 		eth_dev = rte_eth_dev_allocated(name);
-		if (!eth_dev) {
-			PMD_DRV_LOG(ERR, "eth_dev %s not found", name);
-			retval = retval ? retval : -ENODEV;
-		}
+		if (eth_dev == NULL)
+			continue; /* port already released */
 
 		ret = rte_szedata2_eth_dev_uninit(eth_dev);
 		if (ret != 0) {
-- 
2.28.0

Re: [dpdk-dev] [PATCH 17/20] drivers/net: accept removing device without any port

[Ferruh Yigit]

On 9/13/2020 11:07 PM, Thomas Monjalon wrote:
> The ports can be closed (i.e. completely released)
> before removing the whole device.
> Such case was wrongly considered an error by some drivers.
> 
> If the device supports only one port, there is nothing much
> to free after the port is closed.
> 
> Signed-off-by: Thomas Monjalon <thomas@monjalon.net>
> ---
>   drivers/net/ipn3ke/ipn3ke_ethdev.c      |  6 ++----
>   drivers/net/kni/rte_eth_kni.c           | 16 +++++++---------
>   drivers/net/netvsc/hn_ethdev.c          |  2 +-
>   drivers/net/nfp/nfp_net.c               |  2 ++
>   drivers/net/pfe/pfe_ethdev.c            |  6 ++----
>   drivers/net/szedata2/rte_eth_szedata2.c |  6 ++----
>   6 files changed, 16 insertions(+), 22 deletions(-)

For net/kni,
Reviewed-by: Ferruh Yigit <ferruh.yigit@intel.com>

Re: [dpdk-dev] [PATCH 17/20] drivers/net: accept removing device without any port

[Xu, Rosen]

Hi,

> -----Original Message-----
> From: Thomas Monjalon <thomas@monjalon.net>
> Sent: Monday, September 14, 2020 6:07
> To: dev@dpdk.org
> Cc: Yigit, Ferruh <ferruh.yigit@intel.com>; arybchenko@solarflare.com; Xu,
> Rosen <rosen.xu@intel.com>; Stephen Hemminger
> <sthemmin@microsoft.com>; K. Y. Srinivasan <kys@microsoft.com>;
> Haiyang Zhang <haiyangz@microsoft.com>; Long Li <longli@microsoft.com>;
> Heinrich Kuhn <heinrich.kuhn@netronome.com>; Gagandeep Singh
> <g.singh@nxp.com>; Akhil Goyal <akhil.goyal@nxp.com>; Martin Spinler
> <spinler@cesnet.cz>; Burakov, Anatoly <anatoly.burakov@intel.com>
> Subject: [PATCH 17/20] drivers/net: accept removing device without any port
> 
> The ports can be closed (i.e. completely released) before removing the
> whole device.
> Such case was wrongly considered an error by some drivers.
> 
> If the device supports only one port, there is nothing much to free after the
> port is closed.
> 
> Signed-off-by: Thomas Monjalon <thomas@monjalon.net>
> ---
>  drivers/net/ipn3ke/ipn3ke_ethdev.c      |  6 ++----
>  drivers/net/kni/rte_eth_kni.c           | 16 +++++++---------
>  drivers/net/netvsc/hn_ethdev.c          |  2 +-
>  drivers/net/nfp/nfp_net.c               |  2 ++
>  drivers/net/pfe/pfe_ethdev.c            |  6 ++----
>  drivers/net/szedata2/rte_eth_szedata2.c |  6 ++----
>  6 files changed, 16 insertions(+), 22 deletions(-)
> 
> diff --git a/drivers/net/ipn3ke/ipn3ke_ethdev.c
> b/drivers/net/ipn3ke/ipn3ke_ethdev.c
> index 027be29bd8..4446d2af9e 100644
> --- a/drivers/net/ipn3ke/ipn3ke_ethdev.c
> +++ b/drivers/net/ipn3ke/ipn3ke_ethdev.c
> @@ -562,10 +562,8 @@ static int ipn3ke_vswitch_remove(struct
> rte_afu_device *afu_dev)
>  			afu_dev->device.name, i);
> 
>  		ethdev = rte_eth_dev_allocated(afu_dev->device.name);
> -		if (!ethdev)
> -			return -ENODEV;
> -
> -		rte_eth_dev_destroy(ethdev, ipn3ke_rpst_uninit);
> +		if (ethdev != NULL)
> +			rte_eth_dev_destroy(ethdev, ipn3ke_rpst_uninit);
>  	}
> 
>  	ret = rte_eth_switch_domain_free(hw->switch_domain_id);
> diff --git a/drivers/net/kni/rte_eth_kni.c b/drivers/net/kni/rte_eth_kni.c
> index 45ab1b17a8..2a4058f7b0 100644
> --- a/drivers/net/kni/rte_eth_kni.c
> +++ b/drivers/net/kni/rte_eth_kni.c
> @@ -488,17 +488,15 @@ eth_kni_remove(struct rte_vdev_device *vdev)
> 
>  	/* find the ethdev entry */
>  	eth_dev = rte_eth_dev_allocated(name);
> -	if (eth_dev == NULL)
> -		return -1;
> -
> -	if (rte_eal_process_type() != RTE_PROC_PRIMARY) {
> -		eth_kni_dev_stop(eth_dev);
> -		return rte_eth_dev_release_port(eth_dev);
> +	if (eth_dev != NULL) {
> +		if (rte_eal_process_type() != RTE_PROC_PRIMARY) {
> +			eth_kni_dev_stop(eth_dev);
> +			return rte_eth_dev_release_port(eth_dev);
> +		}
> +		eth_kni_close(eth_dev);
> +		rte_eth_dev_release_port(eth_dev);
>  	}
> 
> -	eth_kni_close(eth_dev);
> -	rte_eth_dev_release_port(eth_dev);
> -
>  	is_kni_initialized--;
>  	if (is_kni_initialized == 0)
>  		rte_kni_close();
> diff --git a/drivers/net/netvsc/hn_ethdev.c
> b/drivers/net/netvsc/hn_ethdev.c index c4a2dd9f4a..4c37dc520c 100644
> --- a/drivers/net/netvsc/hn_ethdev.c
> +++ b/drivers/net/netvsc/hn_ethdev.c
> @@ -1092,7 +1092,7 @@ static int eth_hn_remove(struct rte_vmbus_device
> *dev)
> 
>  	eth_dev = rte_eth_dev_allocated(dev->device.name);
>  	if (!eth_dev)
> -		return -ENODEV;
> +		return 0; /* port already released */
> 
>  	ret = eth_hn_dev_uninit(eth_dev);
>  	if (ret)
> diff --git a/drivers/net/nfp/nfp_net.c b/drivers/net/nfp/nfp_net.c index
> c20d71cdc3..2f108099a3 100644
> --- a/drivers/net/nfp/nfp_net.c
> +++ b/drivers/net/nfp/nfp_net.c
> @@ -3721,6 +3721,8 @@ static int eth_nfp_pci_remove(struct
> rte_pci_device *pci_dev)
>  	int port = 0;
> 
>  	eth_dev = rte_eth_dev_allocated(pci_dev->device.name);
> +	if (eth_dev == NULL)
> +		return 0; /* port already released */
>  	if ((pci_dev->id.device_id == PCI_DEVICE_ID_NFP4000_PF_NIC) ||
>  	    (pci_dev->id.device_id == PCI_DEVICE_ID_NFP6000_PF_NIC)) {
>  		port = get_pf_port_number(eth_dev->data->name);
> diff --git a/drivers/net/pfe/pfe_ethdev.c b/drivers/net/pfe/pfe_ethdev.c
> index 8867b61a9d..7a36303785 100644
> --- a/drivers/net/pfe/pfe_ethdev.c
> +++ b/drivers/net/pfe/pfe_ethdev.c
> @@ -1158,10 +1158,8 @@ pmd_pfe_remove(struct rte_vdev_device *vdev)
>  		return 0;
> 
>  	eth_dev = rte_eth_dev_allocated(name);
> -	if (eth_dev == NULL)
> -		return -ENODEV;
> -
> -	pfe_eth_exit(eth_dev, g_pfe);
> +	if (eth_dev != NULL)
> +		pfe_eth_exit(eth_dev, g_pfe);
>  	munmap(g_pfe->cbus_baseaddr, g_pfe->cbus_size);
> 
>  	if (g_pfe->nb_devs == 0) {
> diff --git a/drivers/net/szedata2/rte_eth_szedata2.c
> b/drivers/net/szedata2/rte_eth_szedata2.c
> index 4325b9a30d..5f589dfa4c 100644
> --- a/drivers/net/szedata2/rte_eth_szedata2.c
> +++ b/drivers/net/szedata2/rte_eth_szedata2.c
> @@ -1910,10 +1910,8 @@ static int szedata2_eth_pci_remove(struct
> rte_pci_device *pci_dev)
>  				pci_dev->device.name, i);
>  		PMD_DRV_LOG(DEBUG, "Removing eth_dev %s", name);
>  		eth_dev = rte_eth_dev_allocated(name);
> -		if (!eth_dev) {
> -			PMD_DRV_LOG(ERR, "eth_dev %s not found",
> name);
> -			retval = retval ? retval : -ENODEV;
> -		}
> +		if (eth_dev == NULL)
> +			continue; /* port already released */
> 
>  		ret = rte_szedata2_eth_dev_uninit(eth_dev);
>  		if (ret != 0) {
> --
> 2.28.0

For net/ipn3ke
Reviewed-by: Rosen Xu <rosen.xu@intel.com>

[dpdk-dev] [PATCH 18/20] drivers/net: remove redundant MAC addresses freeing

[Thomas Monjalon]

The MAC addresses array is already freed by rte_eth_dev_release_port().
The redundant freeing can be removed from the PMD port closing functions.

Signed-off-by: Thomas Monjalon <thomas@monjalon.net>
---
 drivers/net/ark/ark_ethdev.c            | 3 ---
 drivers/net/hinic/hinic_pmd_ethdev.c    | 3 ---
 drivers/net/ice/ice_ethdev.c            | 3 ---
 drivers/net/nfb/nfb_ethdev.c            | 3 ---
 drivers/net/szedata2/rte_eth_szedata2.c | 3 ---
 5 files changed, 15 deletions(-)

diff --git a/drivers/net/ark/ark_ethdev.c b/drivers/net/ark/ark_ethdev.c
index 8012af75ee..641a694717 100644
--- a/drivers/net/ark/ark_ethdev.c
+++ b/drivers/net/ark/ark_ethdev.c
@@ -707,9 +707,6 @@ eth_ark_dev_close(struct rte_eth_dev *dev)
 		dev->data->rx_queues[i] = 0;
 	}
 
-	rte_free(dev->data->mac_addrs);
-	dev->data->mac_addrs = 0;
-
 	return 0;
 }
 
diff --git a/drivers/net/hinic/hinic_pmd_ethdev.c b/drivers/net/hinic/hinic_pmd_ethdev.c
index 623534fda4..460093bf95 100644
--- a/drivers/net/hinic/hinic_pmd_ethdev.c
+++ b/drivers/net/hinic/hinic_pmd_ethdev.c
@@ -3225,9 +3225,6 @@ static int hinic_dev_uninit(struct rte_eth_dev *dev)
 
 	rte_free(nic_dev->mc_list);
 
-	rte_free(dev->data->mac_addrs);
-	dev->data->mac_addrs = NULL;
-
 	return HINIC_OK;
 }
 
diff --git a/drivers/net/ice/ice_ethdev.c b/drivers/net/ice/ice_ethdev.c
index f0cb05cbc9..1a7896f0da 100644
--- a/drivers/net/ice/ice_ethdev.c
+++ b/drivers/net/ice/ice_ethdev.c
@@ -2423,9 +2423,6 @@ ice_dev_close(struct rte_eth_dev *dev)
 	dev->rx_pkt_burst = NULL;
 	dev->tx_pkt_burst = NULL;
 
-	rte_free(dev->data->mac_addrs);
-	dev->data->mac_addrs = NULL;
-
 	/* disable uio intr before callback unregister */
 	rte_intr_disable(intr_handle);
 
diff --git a/drivers/net/nfb/nfb_ethdev.c b/drivers/net/nfb/nfb_ethdev.c
index d937ac6922..6fe7500475 100644
--- a/drivers/net/nfb/nfb_ethdev.c
+++ b/drivers/net/nfb/nfb_ethdev.c
@@ -233,9 +233,6 @@ nfb_eth_dev_close(struct rte_eth_dev *dev)
 	}
 	dev->data->nb_tx_queues = 0;
 
-	rte_free(dev->data->mac_addrs);
-	dev->data->mac_addrs = NULL;
-
 	return 0;
 }
 
diff --git a/drivers/net/szedata2/rte_eth_szedata2.c b/drivers/net/szedata2/rte_eth_szedata2.c
index 5f589dfa4c..9ae653e1bd 100644
--- a/drivers/net/szedata2/rte_eth_szedata2.c
+++ b/drivers/net/szedata2/rte_eth_szedata2.c
@@ -1178,9 +1178,6 @@ eth_dev_close(struct rte_eth_dev *dev)
 	}
 	dev->data->nb_tx_queues = 0;
 
-	rte_free(dev->data->mac_addrs);
-	dev->data->mac_addrs = NULL;
-
 	return 0;
 }
 
-- 
2.28.0

Re: [dpdk-dev] [PATCH 18/20] drivers/net: remove redundant MAC addresses freeing

[Ferruh Yigit]

On 9/13/2020 11:07 PM, Thomas Monjalon wrote:
> The MAC addresses array is already freed by rte_eth_dev_release_port().
> The redundant freeing can be removed from the PMD port closing functions.
> 
> Signed-off-by: Thomas Monjalon <thomas@monjalon.net>

Reviewed-by: Ferruh Yigit <ferruh.yigit@intel.com>

[dpdk-dev] [PATCH 19/20] app/testpmd: reset port status on close notification

[Thomas Monjalon]

Since rte_eth_dev_release_port() is called on all port close operations,
the event RTE_ETH_EVENT_DESTROY can be reliably used for resetting
the port status on the application side.

The intermediate state RTE_PORT_HANDLING is removed in close_port()
because a port can also be closed by a PMD in a device remove operation.

In case multiple ports are closed, calling remove_invalid_ports()
only once is enough.

Signed-off-by: Thomas Monjalon <thomas@monjalon.net>
---
 app/test-pmd/testpmd.c | 34 ++++++++--------------------------
 1 file changed, 8 insertions(+), 26 deletions(-)

diff --git a/app/test-pmd/testpmd.c b/app/test-pmd/testpmd.c
index 7842c3b781..31dc97239b 100644
--- a/app/test-pmd/testpmd.c
+++ b/app/test-pmd/testpmd.c
@@ -2694,23 +2694,12 @@ close_port(portid_t pid)
 			continue;
 		}
 
-		if (rte_atomic16_cmpset(&(port->port_status),
-			RTE_PORT_STOPPED, RTE_PORT_HANDLING) == 0) {
-			printf("Port %d is now not stopped\n", pi);
-			continue;
-		}
-
 		if (port->flow_list)
 			port_flow_flush(pi);
 		rte_eth_dev_close(pi);
-
-		remove_invalid_ports();
-
-		if (rte_atomic16_cmpset(&(port->port_status),
-			RTE_PORT_HANDLING, RTE_PORT_CLOSED) == 0)
-			printf("Port %d cannot be set to closed\n", pi);
 	}
 
+	remove_invalid_ports();
 	printf("Done\n");
 }
 
@@ -2841,12 +2830,7 @@ detach_device(struct rte_device *dev)
 		return;
 	}
 	RTE_ETH_FOREACH_DEV_OF(sibling, dev) {
-		/* reset mapping between old ports and removed device */
-		rte_eth_devices[sibling].device = NULL;
 		if (ports[sibling].port_status != RTE_PORT_CLOSED) {
-			/* sibling ports are forced to be closed */
-			ports[sibling].port_status = RTE_PORT_CLOSED;
-			printf("Port %u is closed\n", sibling);
 		}
 	}
 
@@ -2902,11 +2886,8 @@ detach_devargs(char *identifier)
 				return;
 			}
 
-			/* sibling ports are forced to be closed */
 			if (ports[port_id].flow_list)
 				port_flow_flush(port_id);
-			ports[port_id].port_status = RTE_PORT_CLOSED;
-			printf("Port %u is now closed\n", port_id);
 		}
 	}
 
@@ -3055,12 +3036,6 @@ check_all_ports_link_status(uint32_t port_mask)
 	}
 }
 
-/*
- * This callback is for remove a port for a device. It has limitation because
- * it is not for multiple port removal for a device.
- * TODO: the device detach invoke will plan to be removed from user side to
- * eal. And convert all PMDs to free port resources on ether device closing.
- */
 static void
 rmv_port_callback(void *arg)
 {
@@ -3118,6 +3093,13 @@ eth_event_callback(portid_t port_id, enum rte_eth_event_type type, void *param,
 				rmv_port_callback, (void *)(intptr_t)port_id))
 			fprintf(stderr, "Could not set up deferred device removal\n");
 		break;
+	case RTE_ETH_EVENT_DESTROY:
+		if (rte_atomic16_cmpset(&(ports[port_id].port_status),
+					RTE_PORT_STOPPED,
+					RTE_PORT_CLOSED) == 0)
+			printf("Port %d cannot be set to closed\n", port_id);
+		printf("Port %u is closed\n", port_id);
+		break;
 	default:
 		break;
 	}
-- 
2.28.0

Re: [dpdk-dev] [PATCH 19/20] app/testpmd: reset port status on close notification

[Ferruh Yigit]

On 9/13/2020 11:07 PM, Thomas Monjalon wrote:
> Since rte_eth_dev_release_port() is called on all port close operations,
> the event RTE_ETH_EVENT_DESTROY can be reliably used for resetting
> the port status on the application side.
> 
> The intermediate state RTE_PORT_HANDLING is removed in close_port()
> because a port can also be closed by a PMD in a device remove operation.
> 
> In case multiple ports are closed, calling remove_invalid_ports()
> only once is enough.
> 
> Signed-off-by: Thomas Monjalon <thomas@monjalon.net>

<...>

> @@ -3118,6 +3093,13 @@ eth_event_callback(portid_t port_id, enum rte_eth_event_type type, void *param,
>   				rmv_port_callback, (void *)(intptr_t)port_id))
>   			fprintf(stderr, "Could not set up deferred device removal\n");
>   		break;
> +	case RTE_ETH_EVENT_DESTROY:
> +		if (rte_atomic16_cmpset(&(ports[port_id].port_status),
> +					RTE_PORT_STOPPED,
> +					RTE_PORT_CLOSED) == 0)
> +			printf("Port %d cannot be set to closed\n", port_id);
> +		printf("Port %u is closed\n", port_id);
> +		break;

This is failing if a port closed without application port stop command, 
PMD may be doing port stop within the close function but since 
application didn't give the stop command, the port status is not 
'RTE_PORT_STOPPED', hence 'port_status' is not updated correctly.

Re: [dpdk-dev] [PATCH 19/20] app/testpmd: reset port status on close notification

[Thomas Monjalon]

23/09/2020 18:45, Ferruh Yigit:
> On 9/13/2020 11:07 PM, Thomas Monjalon wrote:
> > Since rte_eth_dev_release_port() is called on all port close operations,
> > the event RTE_ETH_EVENT_DESTROY can be reliably used for resetting
> > the port status on the application side.
> > 
> > The intermediate state RTE_PORT_HANDLING is removed in close_port()
> > because a port can also be closed by a PMD in a device remove operation.
> > 
> > In case multiple ports are closed, calling remove_invalid_ports()
> > only once is enough.
> > 
> > Signed-off-by: Thomas Monjalon <thomas@monjalon.net>
> 
> <...>
> 
> > @@ -3118,6 +3093,13 @@ eth_event_callback(portid_t port_id, enum rte_eth_event_type type, void *param,
> >   				rmv_port_callback, (void *)(intptr_t)port_id))
> >   			fprintf(stderr, "Could not set up deferred device removal\n");
> >   		break;
> > +	case RTE_ETH_EVENT_DESTROY:
> > +		if (rte_atomic16_cmpset(&(ports[port_id].port_status),
> > +					RTE_PORT_STOPPED,
> > +					RTE_PORT_CLOSED) == 0)
> > +			printf("Port %d cannot be set to closed\n", port_id);
> > +		printf("Port %u is closed\n", port_id);
> > +		break;
> 
> This is failing if a port closed without application port stop command, 
> PMD may be doing port stop within the close function but since 
> application didn't give the stop command, the port status is not 
> 'RTE_PORT_STOPPED', hence 'port_status' is not updated correctly.

Do you think we should give up with the atomic state transition,
and just assign the state as closed?

Re: [dpdk-dev] [PATCH 19/20] app/testpmd: reset port status on close notification

[Ferruh Yigit]

On 9/23/2020 9:32 PM, Thomas Monjalon wrote:
> 23/09/2020 18:45, Ferruh Yigit:
>> On 9/13/2020 11:07 PM, Thomas Monjalon wrote:
>>> Since rte_eth_dev_release_port() is called on all port close operations,
>>> the event RTE_ETH_EVENT_DESTROY can be reliably used for resetting
>>> the port status on the application side.
>>>
>>> The intermediate state RTE_PORT_HANDLING is removed in close_port()
>>> because a port can also be closed by a PMD in a device remove operation.
>>>
>>> In case multiple ports are closed, calling remove_invalid_ports()
>>> only once is enough.
>>>
>>> Signed-off-by: Thomas Monjalon <thomas@monjalon.net>
>>
>> <...>
>>
>>> @@ -3118,6 +3093,13 @@ eth_event_callback(portid_t port_id, enum rte_eth_event_type type, void *param,
>>>    				rmv_port_callback, (void *)(intptr_t)port_id))
>>>    			fprintf(stderr, "Could not set up deferred device removal\n");
>>>    		break;
>>> +	case RTE_ETH_EVENT_DESTROY:
>>> +		if (rte_atomic16_cmpset(&(ports[port_id].port_status),
>>> +					RTE_PORT_STOPPED,
>>> +					RTE_PORT_CLOSED) == 0)
>>> +			printf("Port %d cannot be set to closed\n", port_id);
>>> +		printf("Port %u is closed\n", port_id);
>>> +		break;
>>
>> This is failing if a port closed without application port stop command,
>> PMD may be doing port stop within the close function but since
>> application didn't give the stop command, the port status is not
>> 'RTE_PORT_STOPPED', hence 'port_status' is not updated correctly.
> 
> Do you think we should give up with the atomic state transition,
> and just assign the state as closed?
> 

It can be better, if the DESTROY event received it should be in closed 
state.

Right now device can be hot removed while running, or it may be closed 
before it has been stopped, both cases state transition assumption from 
stopped->closed will be wrong, and final state will be wrong.

Re: [dpdk-dev] [PATCH 19/20] app/testpmd: reset port status on close notification

[Thomas Monjalon]

24/09/2020 14:07, Ferruh Yigit:
> On 9/23/2020 9:32 PM, Thomas Monjalon wrote:
> > 23/09/2020 18:45, Ferruh Yigit:
> >> On 9/13/2020 11:07 PM, Thomas Monjalon wrote:
> >>> Since rte_eth_dev_release_port() is called on all port close operations,
> >>> the event RTE_ETH_EVENT_DESTROY can be reliably used for resetting
> >>> the port status on the application side.
> >>>
> >>> The intermediate state RTE_PORT_HANDLING is removed in close_port()
> >>> because a port can also be closed by a PMD in a device remove operation.
> >>>
> >>> In case multiple ports are closed, calling remove_invalid_ports()
> >>> only once is enough.
> >>>
> >>> Signed-off-by: Thomas Monjalon <thomas@monjalon.net>
> >>
> >> <...>
> >>
> >>> @@ -3118,6 +3093,13 @@ eth_event_callback(portid_t port_id, enum rte_eth_event_type type, void *param,
> >>>    				rmv_port_callback, (void *)(intptr_t)port_id))
> >>>    			fprintf(stderr, "Could not set up deferred device removal\n");
> >>>    		break;
> >>> +	case RTE_ETH_EVENT_DESTROY:
> >>> +		if (rte_atomic16_cmpset(&(ports[port_id].port_status),
> >>> +					RTE_PORT_STOPPED,
> >>> +					RTE_PORT_CLOSED) == 0)
> >>> +			printf("Port %d cannot be set to closed\n", port_id);
> >>> +		printf("Port %u is closed\n", port_id);
> >>> +		break;
> >>
> >> This is failing if a port closed without application port stop command,
> >> PMD may be doing port stop within the close function but since
> >> application didn't give the stop command, the port status is not
> >> 'RTE_PORT_STOPPED', hence 'port_status' is not updated correctly.
> > 
> > Do you think we should give up with the atomic state transition,
> > and just assign the state as closed?
> > 
> 
> It can be better, if the DESTROY event received it should be in closed 
> state.

I don't understand your proposal.
Note that we are not managing ethdev states here.
Despite the misleading RTE prefix, it is a testpmd state.

> Right now device can be hot removed while running, or it may be closed 
> before it has been stopped, both cases state transition assumption from 
> stopped->closed will be wrong, and final state will be wrong.

I think we should force the state, not matter what it was before.

Re: [dpdk-dev] [PATCH 19/20] app/testpmd: reset port status on close notification

[Ferruh Yigit]

On 9/24/2020 1:17 PM, Thomas Monjalon wrote:
> 24/09/2020 14:07, Ferruh Yigit:
>> On 9/23/2020 9:32 PM, Thomas Monjalon wrote:
>>> 23/09/2020 18:45, Ferruh Yigit:
>>>> On 9/13/2020 11:07 PM, Thomas Monjalon wrote:
>>>>> Since rte_eth_dev_release_port() is called on all port close operations,
>>>>> the event RTE_ETH_EVENT_DESTROY can be reliably used for resetting
>>>>> the port status on the application side.
>>>>>
>>>>> The intermediate state RTE_PORT_HANDLING is removed in close_port()
>>>>> because a port can also be closed by a PMD in a device remove operation.
>>>>>
>>>>> In case multiple ports are closed, calling remove_invalid_ports()
>>>>> only once is enough.
>>>>>
>>>>> Signed-off-by: Thomas Monjalon <thomas@monjalon.net>
>>>>
>>>> <...>
>>>>
>>>>> @@ -3118,6 +3093,13 @@ eth_event_callback(portid_t port_id, enum rte_eth_event_type type, void *param,
>>>>>     				rmv_port_callback, (void *)(intptr_t)port_id))
>>>>>     			fprintf(stderr, "Could not set up deferred device removal\n");
>>>>>     		break;
>>>>> +	case RTE_ETH_EVENT_DESTROY:
>>>>> +		if (rte_atomic16_cmpset(&(ports[port_id].port_status),
>>>>> +					RTE_PORT_STOPPED,
>>>>> +					RTE_PORT_CLOSED) == 0)
>>>>> +			printf("Port %d cannot be set to closed\n", port_id);
>>>>> +		printf("Port %u is closed\n", port_id);
>>>>> +		break;
>>>>
>>>> This is failing if a port closed without application port stop command,
>>>> PMD may be doing port stop within the close function but since
>>>> application didn't give the stop command, the port status is not
>>>> 'RTE_PORT_STOPPED', hence 'port_status' is not updated correctly.
>>>
>>> Do you think we should give up with the atomic state transition,
>>> and just assign the state as closed?
>>>
>>
>> It can be better, if the DESTROY event received it should be in closed
>> state.
> 
> I don't understand your proposal.
> Note that we are not managing ethdev states here.
> Despite the misleading RTE prefix, it is a testpmd state.
> 

I am aware this is testpmd state, but basically saying same thing, force 
the 'closed' state.

>> Right now device can be hot removed while running, or it may be closed
>> before it has been stopped, both cases state transition assumption from
>> stopped->closed will be wrong, and final state will be wrong.
> 
> I think we should force the state, not matter what it was before.
> 
> 

[dpdk-dev] [PATCH 20/20] app/testpmd: align behaviour of multi-port detach

[Thomas Monjalon]

A port can be closed in multiple situations:
	- close command calling close_port() -> rte_eth_dev_close()
	- exit calling close_port() -> rte_eth_dev_close()
	- hotplug calling close_port() -> rte_eth_dev_close()
	- hotplug calling detach_device() -> rte_dev_remove()
	- port detach command, detach_device() -> rte_dev_remove()
	- device detach command, detach_devargs() -> rte_eal_hotplug_remove()

The flow rules are flushed before each close.
It was already done in close_port(), detach_devargs() and
detach_port_device() which calls detach_device(),
but not in detach_device(). As a consequence, it was missing for siblings
of port detach command and unplugged device.
The check before calling port_flow_flush() is moved inside the function.

The state of the port to close is checked to be stopped.
As above, this check was missing in detach_device(),
impacting the cases of a multi-port device unplugged or detached
with the port detach command.

Signed-off-by: Thomas Monjalon <thomas@monjalon.net>
---
 app/test-pmd/config.c  |  7 +++++--
 app/test-pmd/testpmd.c | 22 +++++++++++-----------
 2 files changed, 16 insertions(+), 13 deletions(-)

diff --git a/app/test-pmd/config.c b/app/test-pmd/config.c
index 30bee33248..21513a2978 100644
--- a/app/test-pmd/config.c
+++ b/app/test-pmd/config.c
@@ -1588,9 +1588,12 @@ int
 port_flow_flush(portid_t port_id)
 {
 	struct rte_flow_error error;
-	struct rte_port *port;
+	struct rte_port *port = &ports[port_id];
 	int ret = 0;
 
+	if (port->flow_list == NULL)
+		return ret;
+
 	/* Poisoning to make sure PMDs update it in case of error. */
 	memset(&error, 0x44, sizeof(error));
 	if (rte_flow_flush(port_id, &error)) {
@@ -1599,7 +1602,7 @@ port_flow_flush(portid_t port_id)
 		    port_id == (portid_t)RTE_PORT_ALL)
 			return ret;
 	}
-	port = &ports[port_id];
+
 	while (port->flow_list) {
 		struct port_flow *pf = port->flow_list->next;
 
diff --git a/app/test-pmd/testpmd.c b/app/test-pmd/testpmd.c
index 31dc97239b..a76e9877f6 100644
--- a/app/test-pmd/testpmd.c
+++ b/app/test-pmd/testpmd.c
@@ -2694,8 +2694,7 @@ close_port(portid_t pid)
 			continue;
 		}
 
-		if (port->flow_list)
-			port_flow_flush(pi);
+		port_flow_flush(pi);
 		rte_eth_dev_close(pi);
 	}
 
@@ -2825,15 +2824,20 @@ detach_device(struct rte_device *dev)
 
 	printf("Removing a device...\n");
 
-	if (rte_dev_remove(dev) < 0) {
-		TESTPMD_LOG(ERR, "Failed to detach device %s\n", dev->name);
-		return;
-	}
 	RTE_ETH_FOREACH_DEV_OF(sibling, dev) {
 		if (ports[sibling].port_status != RTE_PORT_CLOSED) {
+			if (ports[sibling].port_status != RTE_PORT_STOPPED) {
+				printf("Port %u not stopped\n", sibling);
+				return;
+			}
+			port_flow_flush(sibling);
 		}
 	}
 
+	if (rte_dev_remove(dev) < 0) {
+		TESTPMD_LOG(ERR, "Failed to detach device %s\n", dev->name);
+		return;
+	}
 	remove_invalid_ports();
 
 	printf("Device is detached\n");
@@ -2854,8 +2858,6 @@ detach_port_device(portid_t port_id)
 			return;
 		}
 		printf("Port was not closed\n");
-		if (ports[port_id].flow_list)
-			port_flow_flush(port_id);
 	}
 
 	detach_device(rte_eth_devices[port_id].device);
@@ -2885,9 +2887,7 @@ detach_devargs(char *identifier)
 				rte_eth_iterator_cleanup(&iterator);
 				return;
 			}
-
-			if (ports[port_id].flow_list)
-				port_flow_flush(port_id);
+			port_flow_flush(port_id);
 		}
 	}
 
-- 
2.28.0

Re: [dpdk-dev] [PATCH 00/20] cleanup ethdev close operation

[Ferruh Yigit]

On 9/13/2020 11:06 PM, Thomas Monjalon wrote:
> This is the end of a process started two years ago,
> to have a close which reliably releases an ethdev port
> without the need of removing the device (which can have more ports).
> 
> Unfortunately, some drivers might be broken because did not follow
> the migration recommendations. We cannot wait more,
> this should be merged before the 20.11-rc1 release.
> 
> 
> Steve Yang (1):
>    net/iavf: release port upon close
> 
> Thomas Monjalon (18):
>    ethdev: reset device and interrupt pointers on release
>    ethdev: allow drivers to return error on close
>    net/af_packet: release port upon close
>    net/atlantic: release port upon close
>    net/axgbe: release port upon close
>    net/bonding: release port upon close
>    net/failsafe: release port upon close
>    net/mlx4: release port upon close
>    net/null: release port upon close
>    net/octeontx: release port upon close
>    net/pcap: release port upon close
>    net/ring: release port upon close
>    net/softnic: release port upon close
>    ethdev: remove old close behaviour
>    drivers/net: accept removing device without any port
>    drivers/net: remove redundant MAC addresses freeing
>    app/testpmd: reset port status on close notification
>    app/testpmd: align behaviour of multi-port detach
> 
> Yunjian Wang (1):
>    net/tap: release port upon close
> 

Hi Thomas,

this requires rebase, specially cxgbe & enic has relevant code changes 
after the set.

[dpdk-dev] [PATCH v2 00/25] cleanup ethdev close operation

[Thomas Monjalon]

This is the end of a process started two years ago,
to have a close which reliably releases an ethdev port
without the need of removing the device (which can have more ports).

Unfortunately, some drivers might be broken because did not follow
the migration recommendations. We cannot wait more,
this should be merged before the 20.11-rc1 release.


v2 (big thanks to Ferruh for the reviews):
  - rebase on top of cxgbe updates
  - integrate updated patches for tap, bnx2x, qede
  - return 0 in atlantic PMD
  - fix failsafe mutex free
  - remove useless check in pcap
  - fix testpmd port state transition
  - add primary process check
  - reset more pointers in rte_eth_dev_release_port()
  - remove dev_started reset in rte_eth_dev_close()
  - return int from rte_eth_dev_close()

Note for future: fix old memory leak when releasing port,
i.e. free callbacks link_intr_cbs, post_rx_burst_cbs, pre_tx_burst_cbs.

The last 3 patches are optional changes to ethdev close/release
which were requested by Ferruh but may take longer to be accepted:
  - memset 0
  - keep start state
  - int return


Rasesh Mody (2):
  net/bnx2x: release port upon close
  net/qede: release port upon close

Thomas Monjalon (22):
  ethdev: reset device and interrupt pointers on release
  ethdev: allow drivers to return error on close
  net/af_packet: release port upon close
  net/atlantic: release port upon close
  net/axgbe: release port upon close
  net/bonding: release port upon close
  net/failsafe: release port upon close
  net/mlx4: release port upon close
  net/null: release port upon close
  net/octeontx: release port upon close
  net/pcap: release port upon close
  net/ring: release port upon close
  net/softnic: release port upon close
  ethdev: remove old close behaviour
  drivers/net: accept removing device without any port
  drivers/net: check process type in close operation
  drivers/net: remove redundant MAC addresses freeing
  app/testpmd: reset port status on close notification
  app/testpmd: align behaviour of multi-port detach
  ethdev: remove forcing stopped state upon close
  ethdev: reset all when releasing a port
  ethdev: allow close function to return an error

Yunjian Wang (1):
  net/tap: release port upon close

 MAINTAINERS                               | 14 ++---
 app/test-pmd/config.c                     |  7 ++-
 app/test-pmd/testpmd.c                    | 53 ++++++------------
 app/test/virtual_pmd.c                    |  6 +-
 doc/guides/rel_notes/deprecation.rst      |  7 ---
 doc/guides/rel_notes/release_20_11.rst    |  4 +-
 drivers/net/af_packet/rte_eth_af_packet.c | 58 ++++++++++----------
 drivers/net/af_xdp/rte_eth_af_xdp.c       |  9 ++-
 drivers/net/ark/ark_ethdev.c              | 17 +++---
 drivers/net/atlantic/atl_ethdev.c         | 62 ++++++++-------------
 drivers/net/avp/avp_ethdev.c              | 10 ++--
 drivers/net/axgbe/axgbe_ethdev.c          | 17 +-----
 drivers/net/bnx2x/bnx2x_ethdev.c          | 16 +++++-
 drivers/net/bnxt/bnxt_ethdev.c            | 16 ++----
 drivers/net/bnxt/bnxt_reps.c              | 11 ++--
 drivers/net/bnxt/bnxt_reps.h              |  2 +-
 drivers/net/bonding/eth_bond_private.h    |  2 +-
 drivers/net/bonding/rte_eth_bond_pmd.c    | 33 +++++------
 drivers/net/cxgbe/cxgbe_ethdev.c          | 18 ++++--
 drivers/net/cxgbe/cxgbe_main.c            |  2 -
 drivers/net/cxgbe/cxgbe_pfvf.h            |  2 +-
 drivers/net/cxgbe/cxgbevf_ethdev.c        |  5 +-
 drivers/net/cxgbe/cxgbevf_main.c          |  2 -
 drivers/net/dpaa/dpaa_ethdev.c            | 11 ++--
 drivers/net/dpaa2/dpaa2_ethdev.c          | 13 +++--
 drivers/net/e1000/em_ethdev.c             | 18 +++---
 drivers/net/e1000/igb_ethdev.c            | 36 +++++-------
 drivers/net/ena/ena_ethdev.c              | 20 +++----
 drivers/net/enetc/enetc_ethdev.c          |  7 ++-
 drivers/net/enic/enic_ethdev.c            |  9 ++-
 drivers/net/enic/enic_vf_representor.c    |  8 +--
 drivers/net/failsafe/failsafe.c           | 24 +-------
 drivers/net/failsafe/failsafe_ether.c     |  6 +-
 drivers/net/failsafe/failsafe_ops.c       | 67 +++++++++++++++--------
 drivers/net/failsafe/failsafe_private.h   |  1 +
 drivers/net/fm10k/fm10k_ethdev.c          | 22 ++------
 drivers/net/hinic/hinic_pmd_ethdev.c      | 22 +++-----
 drivers/net/hns3/hns3_ethdev.c            | 15 ++---
 drivers/net/hns3/hns3_ethdev_vf.c         | 16 ++----
 drivers/net/i40e/i40e_ethdev.c            | 16 ++----
 drivers/net/i40e/i40e_ethdev_vf.c         | 17 ++----
 drivers/net/iavf/iavf_ethdev.c            | 18 +++---
 drivers/net/ice/ice_dcf_ethdev.c          | 12 ++--
 drivers/net/ice/ice_ethdev.c              | 21 +++----
 drivers/net/igc/igc_ethdev.c              | 17 ++----
 drivers/net/ionic/ionic_ethdev.c          | 17 +++---
 drivers/net/ipn3ke/ipn3ke_ethdev.c        |  6 +-
 drivers/net/ipn3ke/ipn3ke_representor.c   |  7 ++-
 drivers/net/ixgbe/ixgbe_ethdev.c          | 33 ++++-------
 drivers/net/kni/rte_eth_kni.c             | 25 +++++----
 drivers/net/liquidio/lio_ethdev.c         | 11 ++--
 drivers/net/memif/rte_eth_memif.c         | 11 ++--
 drivers/net/mlx4/mlx4.c                   |  7 ++-
 drivers/net/mlx5/linux/mlx5_os.c          |  2 -
 drivers/net/mlx5/mlx5.c                   | 16 +++---
 drivers/net/mlx5/mlx5.h                   |  2 +-
 drivers/net/mvneta/mvneta_ethdev.c        | 15 +++--
 drivers/net/mvpp2/mrvl_ethdev.c           | 15 +++--
 drivers/net/netvsc/hn_ethdev.c            | 19 +++----
 drivers/net/netvsc/hn_var.h               |  2 +-
 drivers/net/netvsc/hn_vf.c                |  7 ++-
 drivers/net/nfb/nfb_ethdev.c              | 11 ++--
 drivers/net/nfp/nfp_net.c                 | 11 +++-
 drivers/net/null/rte_eth_null.c           | 26 ++++++---
 drivers/net/octeontx/octeontx_ethdev.c    | 17 ++----
 drivers/net/octeontx2/otx2_ethdev.c       |  8 +--
 drivers/net/pcap/rte_eth_pcap.c           | 31 +++++------
 drivers/net/pfe/pfe_ethdev.c              | 17 +++---
 drivers/net/qede/qede_ethdev.c            | 20 +++----
 drivers/net/ring/rte_eth_ring.c           | 53 +++++++++++-------
 drivers/net/sfc/sfc_ethdev.c              | 35 +++++-------
 drivers/net/softnic/rte_eth_softnic.c     | 67 ++++++++++++-----------
 drivers/net/szedata2/rte_eth_szedata2.c   | 23 ++++----
 drivers/net/tap/rte_eth_tap.c             | 52 ++++++++++--------
 drivers/net/thunderx/nicvf_ethdev.c       | 11 ++--
 drivers/net/vhost/rte_eth_vhost.c         | 15 ++---
 drivers/net/virtio/virtio_ethdev.c        | 17 ++----
 drivers/net/virtio/virtio_user_ethdev.c   |  4 +-
 drivers/net/vmxnet3/vmxnet3_ethdev.c      | 16 ++----
 lib/librte_ethdev/rte_ethdev.c            | 31 +++--------
 lib/librte_ethdev/rte_ethdev.h            | 13 ++---
 lib/librte_ethdev/rte_ethdev_driver.h     |  2 +-
 lib/librte_ethdev/rte_ethdev_pci.h        | 14 +----
 83 files changed, 669 insertions(+), 789 deletions(-)

-- 
2.28.0

[dpdk-dev] [PATCH v2 01/25] ethdev: reset device and interrupt pointers on release

[Thomas Monjalon]

The pointers .device and .intr_handle were already reset by the helper
rte_eth_dev_pci_generic_remove().
It is now made part of rte_eth_dev_release_port().

It makes rte_eth_dev_pci_release() meaningless,
so it is replaced with a call to rte_eth_dev_release_port().

Signed-off-by: Thomas Monjalon <thomas@monjalon.net>
Reviewed-by: Ferruh Yigit <ferruh.yigit@intel.com>
---
 drivers/net/ark/ark_ethdev.c            |  2 +-
 drivers/net/octeontx2/otx2_ethdev.c     |  2 +-
 drivers/net/szedata2/rte_eth_szedata2.c |  6 +++---
 lib/librte_ethdev/rte_ethdev.c          |  2 ++
 lib/librte_ethdev/rte_ethdev_pci.h      | 14 ++------------
 5 files changed, 9 insertions(+), 17 deletions(-)

diff --git a/drivers/net/ark/ark_ethdev.c b/drivers/net/ark/ark_ethdev.c
index d472ae5643..e13415e95b 100644
--- a/drivers/net/ark/ark_ethdev.c
+++ b/drivers/net/ark/ark_ethdev.c
@@ -105,7 +105,7 @@ eth_ark_pci_probe(struct rte_pci_driver *pci_drv __rte_unused,
 
 	ret = eth_ark_dev_init(eth_dev);
 	if (ret)
-		rte_eth_dev_pci_release(eth_dev);
+		rte_eth_dev_release_port(eth_dev);
 
 	return ret;
 }
diff --git a/drivers/net/octeontx2/otx2_ethdev.c b/drivers/net/octeontx2/otx2_ethdev.c
index c06e32f26b..51a6f99877 100644
--- a/drivers/net/octeontx2/otx2_ethdev.c
+++ b/drivers/net/octeontx2/otx2_ethdev.c
@@ -2668,7 +2668,7 @@ nix_remove(struct rte_pci_device *pci_dev)
 		if (rc)
 			return rc;
 
-		rte_eth_dev_pci_release(eth_dev);
+		rte_eth_dev_release_port(eth_dev);
 	}
 
 	/* Nothing to be done for secondary processes */
diff --git a/drivers/net/szedata2/rte_eth_szedata2.c b/drivers/net/szedata2/rte_eth_szedata2.c
index 30c888cd96..a17c53577c 100644
--- a/drivers/net/szedata2/rte_eth_szedata2.c
+++ b/drivers/net/szedata2/rte_eth_szedata2.c
@@ -1802,7 +1802,7 @@ szedata2_eth_dev_release_interval(struct rte_eth_dev **eth_devs,
 
 	for (i = from; i < to; i++) {
 		rte_szedata2_eth_dev_uninit(eth_devs[i]);
-		rte_eth_dev_pci_release(eth_devs[i]);
+		rte_eth_dev_release_port(eth_devs[i]);
 	}
 }
 
@@ -1853,7 +1853,7 @@ static int szedata2_eth_pci_probe(struct rte_pci_driver *pci_drv __rte_unused,
 		if (ret != 0) {
 			PMD_INIT_LOG(ERR, "Failed to init eth_dev for port %u",
 					i);
-			rte_eth_dev_pci_release(eth_devs[i]);
+			rte_eth_dev_release_port(eth_devs[i]);
 			szedata2_eth_dev_release_interval(eth_devs, 0, i);
 			rte_free(list_entry);
 			return ret;
@@ -1922,7 +1922,7 @@ static int szedata2_eth_pci_remove(struct rte_pci_device *pci_dev)
 			retval = retval ? retval : ret;
 		}
 
-		rte_eth_dev_pci_release(eth_dev);
+		rte_eth_dev_release_port(eth_dev);
 	}
 
 	return retval;
diff --git a/lib/librte_ethdev/rte_ethdev.c b/lib/librte_ethdev/rte_ethdev.c
index dfe5c1b488..d12d74dd8c 100644
--- a/lib/librte_ethdev/rte_ethdev.c
+++ b/lib/librte_ethdev/rte_ethdev.c
@@ -555,6 +555,8 @@ rte_eth_dev_release_port(struct rte_eth_dev *eth_dev)
 	rte_spinlock_lock(&rte_eth_dev_shared_data->ownership_lock);
 
 	eth_dev->state = RTE_ETH_DEV_UNUSED;
+	eth_dev->device = NULL;
+	eth_dev->intr_handle = NULL;
 
 	if (rte_eal_process_type() == RTE_PROC_PRIMARY) {
 		rte_free(eth_dev->data->rx_queues);
diff --git a/lib/librte_ethdev/rte_ethdev_pci.h b/lib/librte_ethdev/rte_ethdev_pci.h
index 9c483468ba..bf715896ae 100644
--- a/lib/librte_ethdev/rte_ethdev_pci.h
+++ b/lib/librte_ethdev/rte_ethdev_pci.h
@@ -107,16 +107,6 @@ rte_eth_dev_pci_allocate(struct rte_pci_device *dev, size_t private_data_size)
 	return eth_dev;
 }
 
-static inline void
-rte_eth_dev_pci_release(struct rte_eth_dev *eth_dev)
-{
-	eth_dev->device = NULL;
-	eth_dev->intr_handle = NULL;
-
-	/* free ether device */
-	rte_eth_dev_release_port(eth_dev);
-}
-
 typedef int (*eth_dev_pci_callback_t)(struct rte_eth_dev *eth_dev);
 
 /**
@@ -138,7 +128,7 @@ rte_eth_dev_pci_generic_probe(struct rte_pci_device *pci_dev,
 	RTE_FUNC_PTR_OR_ERR_RET(*dev_init, -EINVAL);
 	ret = dev_init(eth_dev);
 	if (ret)
-		rte_eth_dev_pci_release(eth_dev);
+		rte_eth_dev_release_port(eth_dev);
 	else
 		rte_eth_dev_probing_finish(eth_dev);
 
@@ -167,7 +157,7 @@ rte_eth_dev_pci_generic_remove(struct rte_pci_device *pci_dev,
 			return ret;
 	}
 
-	rte_eth_dev_pci_release(eth_dev);
+	rte_eth_dev_release_port(eth_dev);
 	return 0;
 }
 
-- 
2.28.0

[dpdk-dev] [PATCH v2 02/25] ethdev: allow drivers to return error on close

[Thomas Monjalon]

The device operation .dev_close was returning void.
This driver interface is changed to return an int.

Note that the API rte_eth_dev_close() is still returning void,
although a deprecation notice is pending to change it as well.

Signed-off-by: Thomas Monjalon <thomas@monjalon.net>
---
 app/test/virtual_pmd.c                    |  6 ++++--
 drivers/net/af_packet/rte_eth_af_packet.c |  3 ++-
 drivers/net/af_xdp/rte_eth_af_xdp.c       |  4 +++-
 drivers/net/ark/ark_ethdev.c              |  6 ++++--
 drivers/net/atlantic/atl_ethdev.c         |  6 ++++--
 drivers/net/avp/avp_ethdev.c              |  5 +++--
 drivers/net/axgbe/axgbe_ethdev.c          |  5 +++--
 drivers/net/bnx2x/bnx2x_ethdev.c          |  4 +++-
 drivers/net/bnxt/bnxt_ethdev.c            |  4 +++-
 drivers/net/bnxt/bnxt_reps.c              |  3 ++-
 drivers/net/bnxt/bnxt_reps.h              |  2 +-
 drivers/net/bonding/eth_bond_private.h    |  2 +-
 drivers/net/bonding/rte_eth_bond_pmd.c    |  4 +++-
 drivers/net/cxgbe/cxgbe_ethdev.c          | 10 ++++++----
 drivers/net/cxgbe/cxgbe_pfvf.h            |  2 +-
 drivers/net/dpaa/dpaa_ethdev.c            |  4 +++-
 drivers/net/dpaa2/dpaa2_ethdev.c          |  6 ++++--
 drivers/net/e1000/em_ethdev.c             |  6 ++++--
 drivers/net/e1000/igb_ethdev.c            | 12 ++++++++----
 drivers/net/ena/ena_ethdev.c              |  6 ++++--
 drivers/net/enetc/enetc_ethdev.c          |  4 +++-
 drivers/net/enic/enic_ethdev.c            |  4 +++-
 drivers/net/enic/enic_vf_representor.c    |  5 +++--
 drivers/net/failsafe/failsafe_ops.c       |  3 ++-
 drivers/net/fm10k/fm10k_ethdev.c          |  4 +++-
 drivers/net/hinic/hinic_pmd_ethdev.c      |  6 ++++--
 drivers/net/hns3/hns3_ethdev.c            |  6 ++++--
 drivers/net/hns3/hns3_ethdev_vf.c         |  6 ++++--
 drivers/net/i40e/i40e_ethdev.c            |  5 +++--
 drivers/net/i40e/i40e_ethdev_vf.c         |  5 +++--
 drivers/net/iavf/iavf_ethdev.c            |  6 ++++--
 drivers/net/ice/ice_dcf_ethdev.c          |  6 ++++--
 drivers/net/ice/ice_ethdev.c              |  6 ++++--
 drivers/net/igc/igc_ethdev.c              |  6 ++++--
 drivers/net/ionic/ionic_ethdev.c          | 10 ++++++----
 drivers/net/ipn3ke/ipn3ke_representor.c   |  4 +++-
 drivers/net/ixgbe/ixgbe_ethdev.c          | 11 +++++++----
 drivers/net/kni/rte_eth_kni.c             |  4 +++-
 drivers/net/liquidio/lio_ethdev.c         |  4 +++-
 drivers/net/memif/rte_eth_memif.c         |  4 +++-
 drivers/net/mlx4/mlx4.c                   |  3 ++-
 drivers/net/mlx5/mlx5.c                   |  9 +++++----
 drivers/net/mlx5/mlx5.h                   |  2 +-
 drivers/net/mvneta/mvneta_ethdev.c        |  4 +++-
 drivers/net/mvpp2/mrvl_ethdev.c           |  4 +++-
 drivers/net/netvsc/hn_ethdev.c            |  4 +++-
 drivers/net/nfb/nfb_ethdev.c              |  4 +++-
 drivers/net/nfp/nfp_net.c                 |  6 ++++--
 drivers/net/octeontx/octeontx_ethdev.c    |  4 +++-
 drivers/net/octeontx2/otx2_ethdev.c       |  5 +++--
 drivers/net/pcap/rte_eth_pcap.c           |  3 ++-
 drivers/net/pfe/pfe_ethdev.c              |  8 +++++---
 drivers/net/qede/qede_ethdev.c            |  4 +++-
 drivers/net/sfc/sfc_ethdev.c              |  4 +++-
 drivers/net/softnic/rte_eth_softnic.c     |  4 ++--
 drivers/net/szedata2/rte_eth_szedata2.c   |  4 +++-
 drivers/net/tap/rte_eth_tap.c             |  4 +++-
 drivers/net/thunderx/nicvf_ethdev.c       |  4 +++-
 drivers/net/vhost/rte_eth_vhost.c         |  6 ++++--
 drivers/net/virtio/virtio_ethdev.c        |  6 ++++--
 drivers/net/vmxnet3/vmxnet3_ethdev.c      |  6 ++++--
 lib/librte_ethdev/rte_ethdev_driver.h     |  2 +-
 62 files changed, 206 insertions(+), 103 deletions(-)

diff --git a/app/test/virtual_pmd.c b/app/test/virtual_pmd.c
index b49089a33c..4bd4d1c9ad 100644
--- a/app/test/virtual_pmd.c
+++ b/app/test/virtual_pmd.c
@@ -62,9 +62,11 @@ static void  virtual_ethdev_stop(struct rte_eth_dev *eth_dev __rte_unused)
 		rte_pktmbuf_free(pkt);
 }
 
-static void
+static int
 virtual_ethdev_close(struct rte_eth_dev *dev __rte_unused)
-{}
+{
+	return 0;
+}
 
 static int
 virtual_ethdev_configure_success(struct rte_eth_dev *dev __rte_unused)
diff --git a/drivers/net/af_packet/rte_eth_af_packet.c b/drivers/net/af_packet/rte_eth_af_packet.c
index b9723e9619..7d0ff1cbb3 100644
--- a/drivers/net/af_packet/rte_eth_af_packet.c
+++ b/drivers/net/af_packet/rte_eth_af_packet.c
@@ -376,9 +376,10 @@ eth_stats_reset(struct rte_eth_dev *dev)
 	return 0;
 }
 
-static void
+static int
 eth_dev_close(struct rte_eth_dev *dev __rte_unused)
 {
+	return 0;
 }
 
 static void
diff --git a/drivers/net/af_xdp/rte_eth_af_xdp.c b/drivers/net/af_xdp/rte_eth_af_xdp.c
index b65ee449fc..badbce63fb 100644
--- a/drivers/net/af_xdp/rte_eth_af_xdp.c
+++ b/drivers/net/af_xdp/rte_eth_af_xdp.c
@@ -701,7 +701,7 @@ xdp_umem_destroy(struct xsk_umem_info *umem)
 	umem = NULL;
 }
 
-static void
+static int
 eth_dev_close(struct rte_eth_dev *dev)
 {
 	struct pmd_internals *internals = dev->data->dev_private;
@@ -731,6 +731,8 @@ eth_dev_close(struct rte_eth_dev *dev)
 	dev->data->mac_addrs = NULL;
 
 	remove_xdp_program(internals);
+
+	return 0;
 }
 
 static void
diff --git a/drivers/net/ark/ark_ethdev.c b/drivers/net/ark/ark_ethdev.c
index e13415e95b..5f2ed4b7b7 100644
--- a/drivers/net/ark/ark_ethdev.c
+++ b/drivers/net/ark/ark_ethdev.c
@@ -30,7 +30,7 @@ static int eth_ark_dev_uninit(struct rte_eth_dev *eth_dev);
 static int eth_ark_dev_configure(struct rte_eth_dev *dev);
 static int eth_ark_dev_start(struct rte_eth_dev *dev);
 static void eth_ark_dev_stop(struct rte_eth_dev *dev);
-static void eth_ark_dev_close(struct rte_eth_dev *dev);
+static int eth_ark_dev_close(struct rte_eth_dev *dev);
 static int eth_ark_dev_info_get(struct rte_eth_dev *dev,
 				struct rte_eth_dev_info *dev_info);
 static int eth_ark_dev_link_update(struct rte_eth_dev *dev,
@@ -676,7 +676,7 @@ eth_ark_dev_stop(struct rte_eth_dev *dev)
 	}
 }
 
-static void
+static int
 eth_ark_dev_close(struct rte_eth_dev *dev)
 {
 	struct ark_adapter *ark = dev->data->dev_private;
@@ -706,6 +706,8 @@ eth_ark_dev_close(struct rte_eth_dev *dev)
 
 	rte_free(dev->data->mac_addrs);
 	dev->data->mac_addrs = 0;
+
+	return 0;
 }
 
 static int
diff --git a/drivers/net/atlantic/atl_ethdev.c b/drivers/net/atlantic/atl_ethdev.c
index 2fca9140d7..254758fd66 100644
--- a/drivers/net/atlantic/atl_ethdev.c
+++ b/drivers/net/atlantic/atl_ethdev.c
@@ -22,7 +22,7 @@ static int  atl_dev_start(struct rte_eth_dev *dev);
 static void atl_dev_stop(struct rte_eth_dev *dev);
 static int  atl_dev_set_link_up(struct rte_eth_dev *dev);
 static int  atl_dev_set_link_down(struct rte_eth_dev *dev);
-static void atl_dev_close(struct rte_eth_dev *dev);
+static int  atl_dev_close(struct rte_eth_dev *dev);
 static int  atl_dev_reset(struct rte_eth_dev *dev);
 static int  atl_dev_promiscuous_enable(struct rte_eth_dev *dev);
 static int  atl_dev_promiscuous_disable(struct rte_eth_dev *dev);
@@ -719,7 +719,7 @@ atl_dev_set_link_down(struct rte_eth_dev *dev)
 /*
  * Reset and stop device.
  */
-static void
+static int
 atl_dev_close(struct rte_eth_dev *dev)
 {
 	PMD_INIT_FUNC_TRACE();
@@ -727,6 +727,8 @@ atl_dev_close(struct rte_eth_dev *dev)
 	atl_dev_stop(dev);
 
 	atl_free_queues(dev);
+
+	return 0;
 }
 
 static int
diff --git a/drivers/net/avp/avp_ethdev.c b/drivers/net/avp/avp_ethdev.c
index 5b47f0924a..2eed6acc3f 100644
--- a/drivers/net/avp/avp_ethdev.c
+++ b/drivers/net/avp/avp_ethdev.c
@@ -38,7 +38,7 @@ static int avp_dev_create(struct rte_pci_device *pci_dev,
 static int avp_dev_configure(struct rte_eth_dev *dev);
 static int avp_dev_start(struct rte_eth_dev *dev);
 static void avp_dev_stop(struct rte_eth_dev *dev);
-static void avp_dev_close(struct rte_eth_dev *dev);
+static int avp_dev_close(struct rte_eth_dev *dev);
 static int avp_dev_info_get(struct rte_eth_dev *dev,
 			    struct rte_eth_dev_info *dev_info);
 static int avp_vlan_offload_set(struct rte_eth_dev *dev, int mask);
@@ -2103,7 +2103,7 @@ avp_dev_stop(struct rte_eth_dev *eth_dev)
 	rte_spinlock_unlock(&avp->lock);
 }
 
-static void
+static int
 avp_dev_close(struct rte_eth_dev *eth_dev)
 {
 	struct avp_dev *avp = AVP_DEV_PRIVATE_TO_HW(eth_dev->data->dev_private);
@@ -2139,6 +2139,7 @@ avp_dev_close(struct rte_eth_dev *eth_dev)
 
 unlock:
 	rte_spinlock_unlock(&avp->lock);
+	return 0;
 }
 
 static int
diff --git a/drivers/net/axgbe/axgbe_ethdev.c b/drivers/net/axgbe/axgbe_ethdev.c
index 4f5a014b54..458d64bcef 100644
--- a/drivers/net/axgbe/axgbe_ethdev.c
+++ b/drivers/net/axgbe/axgbe_ethdev.c
@@ -16,7 +16,7 @@ static int  axgbe_dev_configure(struct rte_eth_dev *dev);
 static int  axgbe_dev_start(struct rte_eth_dev *dev);
 static void axgbe_dev_stop(struct rte_eth_dev *dev);
 static void axgbe_dev_interrupt_handler(void *param);
-static void axgbe_dev_close(struct rte_eth_dev *dev);
+static int axgbe_dev_close(struct rte_eth_dev *dev);
 static int axgbe_dev_promiscuous_enable(struct rte_eth_dev *dev);
 static int axgbe_dev_promiscuous_disable(struct rte_eth_dev *dev);
 static int axgbe_dev_allmulticast_enable(struct rte_eth_dev *dev);
@@ -410,10 +410,11 @@ axgbe_dev_stop(struct rte_eth_dev *dev)
 }
 
 /* Clear all resources like TX/RX queues. */
-static void
+static int
 axgbe_dev_close(struct rte_eth_dev *dev)
 {
 	axgbe_dev_clear_queues(dev);
+	return 0;
 }
 
 static int
diff --git a/drivers/net/bnx2x/bnx2x_ethdev.c b/drivers/net/bnx2x/bnx2x_ethdev.c
index b2ea5fafa7..73217c2138 100644
--- a/drivers/net/bnx2x/bnx2x_ethdev.c
+++ b/drivers/net/bnx2x/bnx2x_ethdev.c
@@ -280,7 +280,7 @@ bnx2x_dev_stop(struct rte_eth_dev *dev)
 	return;
 }
 
-static void
+static int
 bnx2x_dev_close(struct rte_eth_dev *dev)
 {
 	struct bnx2x_softc *sc = dev->data->dev_private;
@@ -295,6 +295,8 @@ bnx2x_dev_close(struct rte_eth_dev *dev)
 
 	/* free ilt */
 	bnx2x_free_ilt_mem(sc);
+
+	return 0;
 }
 
 static int
diff --git a/drivers/net/bnxt/bnxt_ethdev.c b/drivers/net/bnxt/bnxt_ethdev.c
index 05e9a6abbf..e726d078ec 100644
--- a/drivers/net/bnxt/bnxt_ethdev.c
+++ b/drivers/net/bnxt/bnxt_ethdev.c
@@ -1357,7 +1357,7 @@ static void bnxt_dev_stop_op(struct rte_eth_dev *eth_dev)
 		bp->flow_stat->flow_count = 0;
 }
 
-static void bnxt_dev_close_op(struct rte_eth_dev *eth_dev)
+static int bnxt_dev_close_op(struct rte_eth_dev *eth_dev)
 {
 	struct bnxt *bp = eth_dev->data->dev_private;
 
@@ -1392,6 +1392,8 @@ static void bnxt_dev_close_op(struct rte_eth_dev *eth_dev)
 
 	rte_free(bp->grp_info);
 	bp->grp_info = NULL;
+
+	return 0;
 }
 
 static void bnxt_mac_addr_remove_op(struct rte_eth_dev *eth_dev,
diff --git a/drivers/net/bnxt/bnxt_reps.c b/drivers/net/bnxt/bnxt_reps.c
index 17010f1ee2..2b6c0a277b 100644
--- a/drivers/net/bnxt/bnxt_reps.c
+++ b/drivers/net/bnxt/bnxt_reps.c
@@ -457,10 +457,11 @@ void bnxt_vf_rep_dev_stop_op(struct rte_eth_dev *eth_dev)
 	bnxt_vf_rep_free_rx_mbufs(vfr_bp);
 }
 
-void bnxt_vf_rep_dev_close_op(struct rte_eth_dev *eth_dev)
+int bnxt_vf_rep_dev_close_op(struct rte_eth_dev *eth_dev)
 {
 	BNXT_TF_DBG(DEBUG, "BNXT Port:%d VFR close\n", eth_dev->data->port_id);
 	bnxt_vf_representor_uninit(eth_dev);
+	return 0;
 }
 
 int bnxt_vf_rep_dev_info_get_op(struct rte_eth_dev *eth_dev,
diff --git a/drivers/net/bnxt/bnxt_reps.h b/drivers/net/bnxt/bnxt_reps.h
index 3239e03fc0..d9b9ee2757 100644
--- a/drivers/net/bnxt/bnxt_reps.h
+++ b/drivers/net/bnxt/bnxt_reps.h
@@ -45,7 +45,7 @@ int bnxt_vf_rep_tx_queue_setup_op(struct rte_eth_dev *eth_dev,
 void bnxt_vf_rep_rx_queue_release_op(void *rx_queue);
 void bnxt_vf_rep_tx_queue_release_op(void *tx_queue);
 void bnxt_vf_rep_dev_stop_op(struct rte_eth_dev *eth_dev);
-void bnxt_vf_rep_dev_close_op(struct rte_eth_dev *eth_dev);
+int bnxt_vf_rep_dev_close_op(struct rte_eth_dev *eth_dev);
 int bnxt_vf_rep_stats_get_op(struct rte_eth_dev *eth_dev,
 			     struct rte_eth_stats *stats);
 int bnxt_vf_rep_stats_reset_op(struct rte_eth_dev *eth_dev);
diff --git a/drivers/net/bonding/eth_bond_private.h b/drivers/net/bonding/eth_bond_private.h
index c9b2d0fe46..0a0034705d 100644
--- a/drivers/net/bonding/eth_bond_private.h
+++ b/drivers/net/bonding/eth_bond_private.h
@@ -318,7 +318,7 @@ bond_tlb_activate_slave(struct bond_dev_private *internals);
 void
 bond_ethdev_stop(struct rte_eth_dev *eth_dev);
 
-void
+int
 bond_ethdev_close(struct rte_eth_dev *dev);
 
 #endif
diff --git a/drivers/net/bonding/rte_eth_bond_pmd.c b/drivers/net/bonding/rte_eth_bond_pmd.c
index 4ffd2ca4bb..dff835ef92 100644
--- a/drivers/net/bonding/rte_eth_bond_pmd.c
+++ b/drivers/net/bonding/rte_eth_bond_pmd.c
@@ -2095,7 +2095,7 @@ bond_ethdev_stop(struct rte_eth_dev *eth_dev)
 	}
 }
 
-void
+int
 bond_ethdev_close(struct rte_eth_dev *dev)
 {
 	struct bond_dev_private *internals = dev->data->dev_private;
@@ -2119,6 +2119,8 @@ bond_ethdev_close(struct rte_eth_dev *dev)
 	bond_flow_ops.flush(dev, &ferror);
 	bond_ethdev_free_queues(dev);
 	rte_bitmap_reset(internals->vlan_filter_bmp);
+
+	return 0;
 }
 
 /* forward declaration */
diff --git a/drivers/net/cxgbe/cxgbe_ethdev.c b/drivers/net/cxgbe/cxgbe_ethdev.c
index 422e190daf..e4bbba5c32 100644
--- a/drivers/net/cxgbe/cxgbe_ethdev.c
+++ b/drivers/net/cxgbe/cxgbe_ethdev.c
@@ -318,7 +318,7 @@ int cxgbe_dev_mtu_set(struct rte_eth_dev *eth_dev, uint16_t mtu)
 /*
  * Stop device.
  */
-void cxgbe_dev_close(struct rte_eth_dev *eth_dev)
+int cxgbe_dev_close(struct rte_eth_dev *eth_dev)
 {
 	struct port_info *temp_pi, *pi = eth_dev->data->dev_private;
 	struct adapter *adapter = pi->adapter;
@@ -327,10 +327,10 @@ void cxgbe_dev_close(struct rte_eth_dev *eth_dev)
 	CXGBE_FUNC_TRACE();
 
 	if (!(adapter->flags & FULL_INIT_DONE))
-		return;
+		return 0;
 
 	if (!pi->viid)
-		return;
+		return 0;
 
 	cxgbe_down(pi);
 	t4_sge_eth_release_queues(pi);
@@ -343,11 +343,13 @@ void cxgbe_dev_close(struct rte_eth_dev *eth_dev)
 	for_each_port(adapter, i) {
 		temp_pi = adap2pinfo(adapter, i);
 		if (temp_pi->viid)
-			return;
+			return 0;
 	}
 
 	cxgbe_close(adapter);
 	rte_free(adapter);
+
+	return 0;
 }
 
 /* Start the device.
diff --git a/drivers/net/cxgbe/cxgbe_pfvf.h b/drivers/net/cxgbe/cxgbe_pfvf.h
index 0b7c52aecf..69d91639e9 100644
--- a/drivers/net/cxgbe/cxgbe_pfvf.h
+++ b/drivers/net/cxgbe/cxgbe_pfvf.h
@@ -19,7 +19,7 @@
 void cxgbe_dev_rx_queue_release(void *q);
 void cxgbe_dev_tx_queue_release(void *q);
 void cxgbe_dev_stop(struct rte_eth_dev *eth_dev);
-void cxgbe_dev_close(struct rte_eth_dev *eth_dev);
+int cxgbe_dev_close(struct rte_eth_dev *eth_dev);
 int cxgbe_dev_info_get(struct rte_eth_dev *eth_dev,
 		       struct rte_eth_dev_info *device_info);
 int cxgbe_dev_promiscuous_enable(struct rte_eth_dev *eth_dev);
diff --git a/drivers/net/dpaa/dpaa_ethdev.c b/drivers/net/dpaa/dpaa_ethdev.c
index 8a6b19b5ce..7fe06e1830 100644
--- a/drivers/net/dpaa/dpaa_ethdev.c
+++ b/drivers/net/dpaa/dpaa_ethdev.c
@@ -369,7 +369,7 @@ static void dpaa_eth_dev_stop(struct rte_eth_dev *dev)
 	dev->tx_pkt_burst = dpaa_eth_tx_drop_all;
 }
 
-static void dpaa_eth_dev_close(struct rte_eth_dev *dev)
+static int dpaa_eth_dev_close(struct rte_eth_dev *dev)
 {
 	struct fman_if *fif = dev->process_private;
 	struct __fman_if *__fif;
@@ -392,6 +392,8 @@ static void dpaa_eth_dev_close(struct rte_eth_dev *dev)
 					     dpaa_interrupt_handler,
 					     (void *)dev);
 	}
+
+	return 0;
 }
 
 static int
diff --git a/drivers/net/dpaa2/dpaa2_ethdev.c b/drivers/net/dpaa2/dpaa2_ethdev.c
index 02daa4d250..fb4165fa03 100644
--- a/drivers/net/dpaa2/dpaa2_ethdev.c
+++ b/drivers/net/dpaa2/dpaa2_ethdev.c
@@ -1236,7 +1236,7 @@ dpaa2_dev_stop(struct rte_eth_dev *dev)
 	rte_eth_linkstatus_set(dev, &link);
 }
 
-static void
+static int
 dpaa2_dev_close(struct rte_eth_dev *dev)
 {
 	struct dpaa2_dev_priv *priv = dev->data->dev_private;
@@ -1252,11 +1252,13 @@ dpaa2_dev_close(struct rte_eth_dev *dev)
 	ret = dpni_reset(dpni, CMD_PRI_LOW, priv->token);
 	if (ret) {
 		DPAA2_PMD_ERR("Failure cleaning dpni device: err=%d", ret);
-		return;
+		return -1;
 	}
 
 	memset(&link, 0, sizeof(link));
 	rte_eth_linkstatus_set(dev, &link);
+
+	return 0;
 }
 
 static int
diff --git a/drivers/net/e1000/em_ethdev.c b/drivers/net/e1000/em_ethdev.c
index 1dc360713a..2d0e071448 100644
--- a/drivers/net/e1000/em_ethdev.c
+++ b/drivers/net/e1000/em_ethdev.c
@@ -34,7 +34,7 @@
 static int eth_em_configure(struct rte_eth_dev *dev);
 static int eth_em_start(struct rte_eth_dev *dev);
 static void eth_em_stop(struct rte_eth_dev *dev);
-static void eth_em_close(struct rte_eth_dev *dev);
+static int eth_em_close(struct rte_eth_dev *dev);
 static int eth_em_promiscuous_enable(struct rte_eth_dev *dev);
 static int eth_em_promiscuous_disable(struct rte_eth_dev *dev);
 static int eth_em_allmulticast_enable(struct rte_eth_dev *dev);
@@ -758,7 +758,7 @@ eth_em_stop(struct rte_eth_dev *dev)
 	}
 }
 
-static void
+static int
 eth_em_close(struct rte_eth_dev *dev)
 {
 	struct e1000_hw *hw = E1000_DEV_PRIVATE_TO_HW(dev->data->dev_private);
@@ -782,6 +782,8 @@ eth_em_close(struct rte_eth_dev *dev)
 	rte_intr_disable(intr_handle);
 	rte_intr_callback_unregister(intr_handle,
 				     eth_em_interrupt_handler, dev);
+
+	return 0;
 }
 
 static int
diff --git a/drivers/net/e1000/igb_ethdev.c b/drivers/net/e1000/igb_ethdev.c
index 5ab74840aa..9e4aefe00c 100644
--- a/drivers/net/e1000/igb_ethdev.c
+++ b/drivers/net/e1000/igb_ethdev.c
@@ -77,7 +77,7 @@ static int  eth_igb_start(struct rte_eth_dev *dev);
 static void eth_igb_stop(struct rte_eth_dev *dev);
 static int  eth_igb_dev_set_link_up(struct rte_eth_dev *dev);
 static int  eth_igb_dev_set_link_down(struct rte_eth_dev *dev);
-static void eth_igb_close(struct rte_eth_dev *dev);
+static int eth_igb_close(struct rte_eth_dev *dev);
 static int eth_igb_reset(struct rte_eth_dev *dev);
 static int  eth_igb_promiscuous_enable(struct rte_eth_dev *dev);
 static int  eth_igb_promiscuous_disable(struct rte_eth_dev *dev);
@@ -155,7 +155,7 @@ static void igbvf_intr_disable(struct e1000_hw *hw);
 static int igbvf_dev_configure(struct rte_eth_dev *dev);
 static int igbvf_dev_start(struct rte_eth_dev *dev);
 static void igbvf_dev_stop(struct rte_eth_dev *dev);
-static void igbvf_dev_close(struct rte_eth_dev *dev);
+static int igbvf_dev_close(struct rte_eth_dev *dev);
 static int igbvf_promiscuous_enable(struct rte_eth_dev *dev);
 static int igbvf_promiscuous_disable(struct rte_eth_dev *dev);
 static int igbvf_allmulticast_enable(struct rte_eth_dev *dev);
@@ -1535,7 +1535,7 @@ eth_igb_dev_set_link_down(struct rte_eth_dev *dev)
 	return 0;
 }
 
-static void
+static int
 eth_igb_close(struct rte_eth_dev *dev)
 {
 	struct e1000_hw *hw = E1000_DEV_PRIVATE_TO_HW(dev->data->dev_private);
@@ -1604,6 +1604,8 @@ eth_igb_close(struct rte_eth_dev *dev)
 
 	/* clear all the filters list */
 	igb_filterlist_flush(dev);
+
+	return 0;
 }
 
 /*
@@ -3381,7 +3383,7 @@ igbvf_dev_stop(struct rte_eth_dev *dev)
 	adapter->stopped = true;
 }
 
-static void
+static int
 igbvf_dev_close(struct rte_eth_dev *dev)
 {
 	struct e1000_hw *hw = E1000_DEV_PRIVATE_TO_HW(dev->data->dev_private);
@@ -3412,6 +3414,8 @@ igbvf_dev_close(struct rte_eth_dev *dev)
 	rte_intr_callback_unregister(&pci_dev->intr_handle,
 				     eth_igbvf_interrupt_handler,
 				     (void *)dev);
+
+	return 0;
 }
 
 static int
diff --git a/drivers/net/ena/ena_ethdev.c b/drivers/net/ena/ena_ethdev.c
index 8077519735..ae6daad892 100644
--- a/drivers/net/ena/ena_ethdev.c
+++ b/drivers/net/ena/ena_ethdev.c
@@ -187,7 +187,7 @@ static void ena_init_rings(struct ena_adapter *adapter,
 static int ena_mtu_set(struct rte_eth_dev *dev, uint16_t mtu);
 static int ena_start(struct rte_eth_dev *dev);
 static void ena_stop(struct rte_eth_dev *dev);
-static void ena_close(struct rte_eth_dev *dev);
+static int ena_close(struct rte_eth_dev *dev);
 static int ena_dev_reset(struct rte_eth_dev *dev);
 static int ena_stats_get(struct rte_eth_dev *dev, struct rte_eth_stats *stats);
 static void ena_rx_queue_release_all(struct rte_eth_dev *dev);
@@ -487,7 +487,7 @@ static void ena_config_debug_area(struct ena_adapter *adapter)
 	ena_com_delete_debug_area(&adapter->ena_dev);
 }
 
-static void ena_close(struct rte_eth_dev *dev)
+static int ena_close(struct rte_eth_dev *dev)
 {
 	struct rte_pci_device *pci_dev = RTE_ETH_DEV_TO_PCI(dev);
 	struct rte_intr_handle *intr_handle = &pci_dev->intr_handle;
@@ -513,6 +513,8 @@ static void ena_close(struct rte_eth_dev *dev)
 	 * release of the resource in the rte_eth_dev_release_port().
 	 */
 	dev->data->mac_addrs = NULL;
+
+	return 0;
 }
 
 static int
diff --git a/drivers/net/enetc/enetc_ethdev.c b/drivers/net/enetc/enetc_ethdev.c
index c6fb42740d..bdb32762ce 100644
--- a/drivers/net/enetc/enetc_ethdev.c
+++ b/drivers/net/enetc/enetc_ethdev.c
@@ -545,7 +545,7 @@ enetc_stats_reset(struct rte_eth_dev *dev)
 	return 0;
 }
 
-static void
+static int
 enetc_dev_close(struct rte_eth_dev *dev)
 {
 	uint16_t i;
@@ -564,6 +564,8 @@ enetc_dev_close(struct rte_eth_dev *dev)
 		dev->data->tx_queues[i] = NULL;
 	}
 	dev->data->nb_tx_queues = 0;
+
+	return 0;
 }
 
 static int
diff --git a/drivers/net/enic/enic_ethdev.c b/drivers/net/enic/enic_ethdev.c
index 57e82845a4..ed03d53608 100644
--- a/drivers/net/enic/enic_ethdev.c
+++ b/drivers/net/enic/enic_ethdev.c
@@ -446,12 +446,14 @@ static void enicpmd_dev_stop(struct rte_eth_dev *eth_dev)
 /*
  * Stop device.
  */
-static void enicpmd_dev_close(struct rte_eth_dev *eth_dev)
+static int enicpmd_dev_close(struct rte_eth_dev *eth_dev)
 {
 	struct enic *enic = pmd_priv(eth_dev);
 
 	ENICPMD_FUNC_TRACE();
 	enic_remove(enic);
+
+	return 0;
 }
 
 static int enicpmd_dev_link_update(struct rte_eth_dev *eth_dev,
diff --git a/drivers/net/enic/enic_vf_representor.c b/drivers/net/enic/enic_vf_representor.c
index 5d34e1b460..c528be2b98 100644
--- a/drivers/net/enic/enic_vf_representor.c
+++ b/drivers/net/enic/enic_vf_representor.c
@@ -277,11 +277,12 @@ static void enic_vf_dev_stop(struct rte_eth_dev *eth_dev)
  * "close" is no-op for now and solely exists so that rte_eth_dev_close()
  * can finish its own cleanup without errors.
  */
-static void enic_vf_dev_close(struct rte_eth_dev *eth_dev __rte_unused)
+static int enic_vf_dev_close(struct rte_eth_dev *eth_dev __rte_unused)
 {
 	ENICPMD_FUNC_TRACE();
 	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
-		return;
+		return 0;
+	return 0;
 }
 
 static int
diff --git a/drivers/net/failsafe/failsafe_ops.c b/drivers/net/failsafe/failsafe_ops.c
index 96f7e456f9..93ebd09114 100644
--- a/drivers/net/failsafe/failsafe_ops.c
+++ b/drivers/net/failsafe/failsafe_ops.c
@@ -240,7 +240,7 @@ fs_dev_set_link_down(struct rte_eth_dev *dev)
 }
 
 static void fs_dev_free_queues(struct rte_eth_dev *dev);
-static void
+static int
 fs_dev_close(struct rte_eth_dev *dev)
 {
 	struct sub_device *sdev;
@@ -259,6 +259,7 @@ fs_dev_close(struct rte_eth_dev *dev)
 	}
 	fs_dev_free_queues(dev);
 	fs_unlock(dev, 0);
+	return 0;
 }
 
 static int
diff --git a/drivers/net/fm10k/fm10k_ethdev.c b/drivers/net/fm10k/fm10k_ethdev.c
index 5771d83b55..533f976709 100644
--- a/drivers/net/fm10k/fm10k_ethdev.c
+++ b/drivers/net/fm10k/fm10k_ethdev.c
@@ -2776,7 +2776,7 @@ fm10k_close_mbx_service(struct fm10k_hw *hw)
 	hw->mbx.ops.disconnect(hw, &hw->mbx);
 }
 
-static void
+static int
 fm10k_dev_close(struct rte_eth_dev *dev)
 {
 	struct fm10k_hw *hw = FM10K_DEV_PRIVATE_TO_HW(dev->data->dev_private);
@@ -2821,6 +2821,8 @@ fm10k_dev_close(struct rte_eth_dev *dev)
 		rte_intr_callback_unregister(intr_handle,
 			fm10k_dev_interrupt_handler_vf, (void *)dev);
 	}
+
+	return 0;
 }
 
 static const struct eth_dev_ops fm10k_eth_dev_ops = {
diff --git a/drivers/net/hinic/hinic_pmd_ethdev.c b/drivers/net/hinic/hinic_pmd_ethdev.c
index 67e6afcf7a..5f2e97d3bd 100644
--- a/drivers/net/hinic/hinic_pmd_ethdev.c
+++ b/drivers/net/hinic/hinic_pmd_ethdev.c
@@ -2936,7 +2936,7 @@ static void hinic_nic_dev_destroy(struct rte_eth_dev *eth_dev)
  * @param dev
  *   Pointer to Ethernet device structure.
  */
-static void hinic_dev_close(struct rte_eth_dev *dev)
+static int hinic_dev_close(struct rte_eth_dev *dev)
 {
 	struct hinic_nic_dev *nic_dev = HINIC_ETH_DEV_TO_PRIVATE_NIC_DEV(dev);
 
@@ -2944,7 +2944,7 @@ static void hinic_dev_close(struct rte_eth_dev *dev)
 					   &nic_dev->dev_status)) {
 		PMD_DRV_LOG(WARNING, "Device %s already closed",
 			    dev->data->name);
-		return;
+		return 0;
 	}
 
 	/* stop device first */
@@ -2971,6 +2971,8 @@ static void hinic_dev_close(struct rte_eth_dev *dev)
 
 	/* deinit nic hardware device */
 	hinic_nic_dev_destroy(dev);
+
+	return 0;
 }
 
 static const struct eth_dev_ops hinic_pmd_ops = {
diff --git a/drivers/net/hns3/hns3_ethdev.c b/drivers/net/hns3/hns3_ethdev.c
index 73d504253d..3b395a1ccf 100644
--- a/drivers/net/hns3/hns3_ethdev.c
+++ b/drivers/net/hns3/hns3_ethdev.c
@@ -4859,7 +4859,7 @@ hns3_dev_stop(struct rte_eth_dev *dev)
 	rte_spinlock_unlock(&hw->lock);
 }
 
-static void
+static int
 hns3_dev_close(struct rte_eth_dev *eth_dev)
 {
 	struct hns3_adapter *hns = eth_dev->data->dev_private;
@@ -4868,7 +4868,7 @@ hns3_dev_close(struct rte_eth_dev *eth_dev)
 	if (rte_eal_process_type() != RTE_PROC_PRIMARY) {
 		rte_free(eth_dev->process_private);
 		eth_dev->process_private = NULL;
-		return;
+		return 0;
 	}
 
 	if (hw->adapter_state == HNS3_NIC_STARTED)
@@ -4888,6 +4888,8 @@ hns3_dev_close(struct rte_eth_dev *eth_dev)
 	eth_dev->process_private = NULL;
 	hns3_mp_uninit_primary();
 	hns3_warn(hw, "Close port %d finished", hw->data->port_id);
+
+	return 0;
 }
 
 static int
diff --git a/drivers/net/hns3/hns3_ethdev_vf.c b/drivers/net/hns3/hns3_ethdev_vf.c
index 037a5be7e0..e0ca5a6ac2 100644
--- a/drivers/net/hns3/hns3_ethdev_vf.c
+++ b/drivers/net/hns3/hns3_ethdev_vf.c
@@ -1910,14 +1910,14 @@ hns3vf_dev_stop(struct rte_eth_dev *dev)
 	rte_spinlock_unlock(&hw->lock);
 }
 
-static void
+static int
 hns3vf_dev_close(struct rte_eth_dev *eth_dev)
 {
 	struct hns3_adapter *hns = eth_dev->data->dev_private;
 	struct hns3_hw *hw = &hns->hw;
 
 	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
-		return;
+		return 0;
 
 	if (hw->adapter_state == HNS3_NIC_STARTED)
 		hns3vf_dev_stop(eth_dev);
@@ -1935,6 +1935,8 @@ hns3vf_dev_close(struct rte_eth_dev *eth_dev)
 	eth_dev->process_private = NULL;
 	hns3_mp_uninit_primary();
 	hns3_warn(hw, "Close port %d finished", hw->data->port_id);
+
+	return 0;
 }
 
 static int
diff --git a/drivers/net/i40e/i40e_ethdev.c b/drivers/net/i40e/i40e_ethdev.c
index 563f21d9df..804f0128ab 100644
--- a/drivers/net/i40e/i40e_ethdev.c
+++ b/drivers/net/i40e/i40e_ethdev.c
@@ -225,7 +225,7 @@ static int eth_i40e_dev_uninit(struct rte_eth_dev *eth_dev);
 static int i40e_dev_configure(struct rte_eth_dev *dev);
 static int i40e_dev_start(struct rte_eth_dev *dev);
 static void i40e_dev_stop(struct rte_eth_dev *dev);
-static void i40e_dev_close(struct rte_eth_dev *dev);
+static int i40e_dev_close(struct rte_eth_dev *dev);
 static int  i40e_dev_reset(struct rte_eth_dev *dev);
 static int i40e_dev_promiscuous_enable(struct rte_eth_dev *dev);
 static int i40e_dev_promiscuous_disable(struct rte_eth_dev *dev);
@@ -2608,7 +2608,7 @@ i40e_dev_stop(struct rte_eth_dev *dev)
 	pf->adapter->rss_reta_updated = 0;
 }
 
-static void
+static int
 i40e_dev_close(struct rte_eth_dev *dev)
 {
 	struct i40e_pf *pf = I40E_DEV_PRIVATE_TO_PF(dev->data->dev_private);
@@ -2748,6 +2748,7 @@ i40e_dev_close(struct rte_eth_dev *dev)
 	i40e_tm_conf_uninit(dev);
 
 	hw->adapter_closed = 1;
+	return 0;
 }
 
 /*
diff --git a/drivers/net/i40e/i40e_ethdev_vf.c b/drivers/net/i40e/i40e_ethdev_vf.c
index 8531cf6b12..61aad8c415 100644
--- a/drivers/net/i40e/i40e_ethdev_vf.c
+++ b/drivers/net/i40e/i40e_ethdev_vf.c
@@ -90,7 +90,7 @@ static int i40evf_dev_xstats_reset(struct rte_eth_dev *dev);
 static int i40evf_vlan_filter_set(struct rte_eth_dev *dev,
 				  uint16_t vlan_id, int on);
 static int i40evf_vlan_offload_set(struct rte_eth_dev *dev, int mask);
-static void i40evf_dev_close(struct rte_eth_dev *dev);
+static int i40evf_dev_close(struct rte_eth_dev *dev);
 static int i40evf_dev_reset(struct rte_eth_dev *dev);
 static int i40evf_check_vf_reset_done(struct rte_eth_dev *dev);
 static int i40evf_dev_promiscuous_enable(struct rte_eth_dev *dev);
@@ -2401,7 +2401,7 @@ i40evf_dev_stats_get(struct rte_eth_dev *dev, struct rte_eth_stats *stats)
 	return ret;
 }
 
-static void
+static int
 i40evf_dev_close(struct rte_eth_dev *dev)
 {
 	struct i40e_hw *hw = I40E_DEV_PRIVATE_TO_HW(dev->data->dev_private);
@@ -2433,6 +2433,7 @@ i40evf_dev_close(struct rte_eth_dev *dev)
 	vf->aq_resp = NULL;
 
 	hw->adapter_closed = 1;
+	return 0;
 }
 
 /*
diff --git a/drivers/net/iavf/iavf_ethdev.c b/drivers/net/iavf/iavf_ethdev.c
index 440da7d76a..8f46c4588f 100644
--- a/drivers/net/iavf/iavf_ethdev.c
+++ b/drivers/net/iavf/iavf_ethdev.c
@@ -32,7 +32,7 @@
 static int iavf_dev_configure(struct rte_eth_dev *dev);
 static int iavf_dev_start(struct rte_eth_dev *dev);
 static void iavf_dev_stop(struct rte_eth_dev *dev);
-static void iavf_dev_close(struct rte_eth_dev *dev);
+static int iavf_dev_close(struct rte_eth_dev *dev);
 static int iavf_dev_reset(struct rte_eth_dev *dev);
 static int iavf_dev_info_get(struct rte_eth_dev *dev,
 			     struct rte_eth_dev_info *dev_info);
@@ -1463,7 +1463,7 @@ iavf_dev_init(struct rte_eth_dev *eth_dev)
 	return 0;
 }
 
-static void
+static int
 iavf_dev_close(struct rte_eth_dev *dev)
 {
 	struct iavf_hw *hw = IAVF_DEV_PRIVATE_TO_HW(dev->data->dev_private);
@@ -1506,6 +1506,8 @@ iavf_dev_close(struct rte_eth_dev *dev)
 
 	rte_free(vf->aq_resp);
 	vf->aq_resp = NULL;
+
+	return 0;
 }
 
 static int
diff --git a/drivers/net/ice/ice_dcf_ethdev.c b/drivers/net/ice/ice_dcf_ethdev.c
index 2faed3cc7a..5b626cb641 100644
--- a/drivers/net/ice/ice_dcf_ethdev.c
+++ b/drivers/net/ice/ice_dcf_ethdev.c
@@ -847,13 +847,13 @@ ice_dcf_stats_reset(struct rte_eth_dev *dev)
 	return 0;
 }
 
-static void
+static int
 ice_dcf_dev_close(struct rte_eth_dev *dev)
 {
 	struct ice_dcf_adapter *adapter = dev->data->dev_private;
 
 	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
-		return;
+		return 0;
 
 	dev->dev_ops = NULL;
 	dev->rx_pkt_burst = NULL;
@@ -861,6 +861,8 @@ ice_dcf_dev_close(struct rte_eth_dev *dev)
 
 	ice_dcf_uninit_parent_adapter(dev);
 	ice_dcf_uninit_hw(dev, &adapter->real_hw);
+
+	return 0;
 }
 
 static int
diff --git a/drivers/net/ice/ice_ethdev.c b/drivers/net/ice/ice_ethdev.c
index cfd357b055..b4ac5e3f81 100644
--- a/drivers/net/ice/ice_ethdev.c
+++ b/drivers/net/ice/ice_ethdev.c
@@ -77,7 +77,7 @@ static struct proto_xtr_ol_flag ice_proto_xtr_ol_flag_params[] = {
 static int ice_dev_configure(struct rte_eth_dev *dev);
 static int ice_dev_start(struct rte_eth_dev *dev);
 static void ice_dev_stop(struct rte_eth_dev *dev);
-static void ice_dev_close(struct rte_eth_dev *dev);
+static int ice_dev_close(struct rte_eth_dev *dev);
 static int ice_dev_reset(struct rte_eth_dev *dev);
 static int ice_dev_info_get(struct rte_eth_dev *dev,
 			    struct rte_eth_dev_info *dev_info);
@@ -2430,7 +2430,7 @@ ice_dev_stop(struct rte_eth_dev *dev)
 	pf->adapter_stopped = true;
 }
 
-static void
+static int
 ice_dev_close(struct rte_eth_dev *dev)
 {
 	struct ice_pf *pf = ICE_DEV_PRIVATE_TO_PF(dev->data->dev_private);
@@ -2478,6 +2478,8 @@ ice_dev_close(struct rte_eth_dev *dev)
 	/* unregister callback func from eal lib */
 	rte_intr_callback_unregister(intr_handle,
 				     ice_interrupt_handler, dev);
+
+	return 0;
 }
 
 static int
diff --git a/drivers/net/igc/igc_ethdev.c b/drivers/net/igc/igc_ethdev.c
index 810568bc54..11397c5c1d 100644
--- a/drivers/net/igc/igc_ethdev.c
+++ b/drivers/net/igc/igc_ethdev.c
@@ -183,7 +183,7 @@ static void eth_igc_stop(struct rte_eth_dev *dev);
 static int eth_igc_start(struct rte_eth_dev *dev);
 static int eth_igc_set_link_up(struct rte_eth_dev *dev);
 static int eth_igc_set_link_down(struct rte_eth_dev *dev);
-static void eth_igc_close(struct rte_eth_dev *dev);
+static int eth_igc_close(struct rte_eth_dev *dev);
 static int eth_igc_reset(struct rte_eth_dev *dev);
 static int eth_igc_promiscuous_enable(struct rte_eth_dev *dev);
 static int eth_igc_promiscuous_disable(struct rte_eth_dev *dev);
@@ -1165,7 +1165,7 @@ igc_dev_free_queues(struct rte_eth_dev *dev)
 	dev->data->nb_tx_queues = 0;
 }
 
-static void
+static int
 eth_igc_close(struct rte_eth_dev *dev)
 {
 	struct rte_pci_device *pci_dev = RTE_ETH_DEV_TO_PCI(dev);
@@ -1199,6 +1199,8 @@ eth_igc_close(struct rte_eth_dev *dev)
 
 	/* Reset any pending lock */
 	igc_reset_swfw_lock(hw);
+
+	return 0;
 }
 
 static void
diff --git a/drivers/net/ionic/ionic_ethdev.c b/drivers/net/ionic/ionic_ethdev.c
index 800f6e5d64..1775fd29b7 100644
--- a/drivers/net/ionic/ionic_ethdev.c
+++ b/drivers/net/ionic/ionic_ethdev.c
@@ -25,7 +25,7 @@ static int  ionic_dev_configure(struct rte_eth_dev *dev);
 static int  ionic_dev_mtu_set(struct rte_eth_dev *dev, uint16_t mtu);
 static int  ionic_dev_start(struct rte_eth_dev *dev);
 static void ionic_dev_stop(struct rte_eth_dev *dev);
-static void ionic_dev_close(struct rte_eth_dev *dev);
+static int  ionic_dev_close(struct rte_eth_dev *dev);
 static int  ionic_dev_set_link_up(struct rte_eth_dev *dev);
 static int  ionic_dev_set_link_down(struct rte_eth_dev *dev);
 static int  ionic_dev_link_update(struct rte_eth_dev *eth_dev,
@@ -956,7 +956,7 @@ ionic_dev_stop(struct rte_eth_dev *eth_dev)
 /*
  * Reset and stop device.
  */
-static void
+static int
 ionic_dev_close(struct rte_eth_dev *eth_dev)
 {
 	struct ionic_lif *lif = IONIC_ETH_DEV_TO_LIF(eth_dev);
@@ -967,14 +967,16 @@ ionic_dev_close(struct rte_eth_dev *eth_dev)
 	err = ionic_lif_stop(lif);
 	if (err) {
 		IONIC_PRINT(ERR, "Cannot stop LIF: %d", err);
-		return;
+		return -1;
 	}
 
 	err = eth_ionic_dev_uninit(eth_dev);
 	if (err) {
 		IONIC_PRINT(ERR, "Cannot destroy LIF: %d", err);
-		return;
+		return -1;
 	}
+
+	return 0;
 }
 
 static int
diff --git a/drivers/net/ipn3ke/ipn3ke_representor.c b/drivers/net/ipn3ke/ipn3ke_representor.c
index b673c49149..d49abbf758 100644
--- a/drivers/net/ipn3ke/ipn3ke_representor.c
+++ b/drivers/net/ipn3ke/ipn3ke_representor.c
@@ -208,7 +208,7 @@ ipn3ke_rpst_dev_stop(struct rte_eth_dev *dev)
 	}
 }
 
-static void
+static int
 ipn3ke_rpst_dev_close(struct rte_eth_dev *dev)
 {
 	struct ipn3ke_hw *hw = IPN3KE_DEV_PRIVATE_TO_HW(dev);
@@ -221,6 +221,8 @@ ipn3ke_rpst_dev_close(struct rte_eth_dev *dev)
 		/* Disable the RX path */
 		ipn3ke_xmac_rx_disable(hw, rpst->port_id, 0);
 	}
+
+	return 0;
 }
 
 /*
diff --git a/drivers/net/ixgbe/ixgbe_ethdev.c b/drivers/net/ixgbe/ixgbe_ethdev.c
index 0f065bbc04..a1369a0744 100644
--- a/drivers/net/ixgbe/ixgbe_ethdev.c
+++ b/drivers/net/ixgbe/ixgbe_ethdev.c
@@ -147,7 +147,7 @@ static int  ixgbe_dev_start(struct rte_eth_dev *dev);
 static void ixgbe_dev_stop(struct rte_eth_dev *dev);
 static int  ixgbe_dev_set_link_up(struct rte_eth_dev *dev);
 static int  ixgbe_dev_set_link_down(struct rte_eth_dev *dev);
-static void ixgbe_dev_close(struct rte_eth_dev *dev);
+static int  ixgbe_dev_close(struct rte_eth_dev *dev);
 static int  ixgbe_dev_reset(struct rte_eth_dev *dev);
 static int ixgbe_dev_promiscuous_enable(struct rte_eth_dev *dev);
 static int ixgbe_dev_promiscuous_disable(struct rte_eth_dev *dev);
@@ -251,7 +251,7 @@ static int  ixgbevf_dev_start(struct rte_eth_dev *dev);
 static int ixgbevf_dev_link_update(struct rte_eth_dev *dev,
 				   int wait_to_complete);
 static void ixgbevf_dev_stop(struct rte_eth_dev *dev);
-static void ixgbevf_dev_close(struct rte_eth_dev *dev);
+static int ixgbevf_dev_close(struct rte_eth_dev *dev);
 static int  ixgbevf_dev_reset(struct rte_eth_dev *dev);
 static void ixgbevf_intr_disable(struct rte_eth_dev *dev);
 static void ixgbevf_intr_enable(struct rte_eth_dev *dev);
@@ -2994,7 +2994,7 @@ ixgbe_dev_set_link_down(struct rte_eth_dev *dev)
 /*
  * Reset and stop device.
  */
-static void
+static int
 ixgbe_dev_close(struct rte_eth_dev *dev)
 {
 	struct ixgbe_hw *hw =
@@ -3065,6 +3065,7 @@ ixgbe_dev_close(struct rte_eth_dev *dev)
 	rte_free(dev->security_ctx);
 #endif
 
+	return 0;
 }
 
 /*
@@ -5443,7 +5444,7 @@ ixgbevf_dev_stop(struct rte_eth_dev *dev)
 	adapter->rss_reta_updated = 0;
 }
 
-static void
+static int
 ixgbevf_dev_close(struct rte_eth_dev *dev)
 {
 	struct ixgbe_hw *hw = IXGBE_DEV_PRIVATE_TO_HW(dev->data->dev_private);
@@ -5472,6 +5473,8 @@ ixgbevf_dev_close(struct rte_eth_dev *dev)
 	rte_intr_disable(intr_handle);
 	rte_intr_callback_unregister(intr_handle,
 				     ixgbevf_dev_interrupt_handler, dev);
+
+	return 0;
 }
 
 /*
diff --git a/drivers/net/kni/rte_eth_kni.c b/drivers/net/kni/rte_eth_kni.c
index 831fe96c96..aa365a57de 100644
--- a/drivers/net/kni/rte_eth_kni.c
+++ b/drivers/net/kni/rte_eth_kni.c
@@ -198,7 +198,7 @@ eth_kni_dev_stop(struct rte_eth_dev *dev)
 	dev->data->dev_link.link_status = 0;
 }
 
-static void
+static int
 eth_kni_close(struct rte_eth_dev *eth_dev)
 {
 	struct pmd_internals *internals;
@@ -214,6 +214,8 @@ eth_kni_close(struct rte_eth_dev *eth_dev)
 	if (ret)
 		PMD_LOG(WARNING, "Not able to release kni for %s",
 			eth_dev->data->name);
+
+	return 0;
 }
 
 static int
diff --git a/drivers/net/liquidio/lio_ethdev.c b/drivers/net/liquidio/lio_ethdev.c
index 2c2b27e62a..93e2ed5670 100644
--- a/drivers/net/liquidio/lio_ethdev.c
+++ b/drivers/net/liquidio/lio_ethdev.c
@@ -1550,7 +1550,7 @@ lio_dev_set_link_down(struct rte_eth_dev *eth_dev)
  * @return
  *    - nothing
  */
-static void
+static int
 lio_dev_close(struct rte_eth_dev *eth_dev)
 {
 	struct lio_device *lio_dev = LIO_DEV(eth_dev);
@@ -1581,6 +1581,8 @@ lio_dev_close(struct rte_eth_dev *eth_dev)
 
 	 /* Delete all queues */
 	lio_dev_clear_queues(eth_dev);
+
+	return 0;
 }
 
 /**
diff --git a/drivers/net/memif/rte_eth_memif.c b/drivers/net/memif/rte_eth_memif.c
index a19c0f3e6b..1a2411c838 100644
--- a/drivers/net/memif/rte_eth_memif.c
+++ b/drivers/net/memif/rte_eth_memif.c
@@ -1218,7 +1218,7 @@ memif_dev_start(struct rte_eth_dev *dev)
 	return ret;
 }
 
-static void
+static int
 memif_dev_close(struct rte_eth_dev *dev)
 {
 	struct pmd_internals *pmd = dev->data->dev_private;
@@ -1239,6 +1239,8 @@ memif_dev_close(struct rte_eth_dev *dev)
 	}
 
 	rte_free(dev->process_private);
+
+	return 0;
 }
 
 static int
diff --git a/drivers/net/mlx4/mlx4.c b/drivers/net/mlx4/mlx4.c
index 71061a720f..df59314b66 100644
--- a/drivers/net/mlx4/mlx4.c
+++ b/drivers/net/mlx4/mlx4.c
@@ -370,7 +370,7 @@ mlx4_dev_stop(struct rte_eth_dev *dev)
  * @param dev
  *   Pointer to Ethernet device structure.
  */
-static void
+static int
 mlx4_dev_close(struct rte_eth_dev *dev)
 {
 	struct mlx4_priv *priv = dev->data->dev_private;
@@ -400,6 +400,7 @@ mlx4_dev_close(struct rte_eth_dev *dev)
 		MLX4_ASSERT(priv->ctx == NULL);
 	mlx4_intr_uninstall(priv);
 	memset(priv, 0, sizeof(*priv));
+	return 0;
 }
 
 static const struct eth_dev_ops mlx4_dev_ops = {
diff --git a/drivers/net/mlx5/mlx5.c b/drivers/net/mlx5/mlx5.c
index 4a807fb4fd..01ead6e6af 100644
--- a/drivers/net/mlx5/mlx5.c
+++ b/drivers/net/mlx5/mlx5.c
@@ -1341,7 +1341,7 @@ mlx5_proc_priv_uninit(struct rte_eth_dev *dev)
  * @param dev
  *   Pointer to Ethernet device structure.
  */
-void
+int
 mlx5_dev_close(struct rte_eth_dev *dev)
 {
 	struct mlx5_priv *priv = dev->data->dev_private;
@@ -1351,14 +1351,14 @@ mlx5_dev_close(struct rte_eth_dev *dev)
 	if (rte_eal_process_type() == RTE_PROC_SECONDARY) {
 		/* Check if process_private released. */
 		if (!dev->process_private)
-			return;
+			return 0;
 		mlx5_tx_uar_uninit_secondary(dev);
 		mlx5_proc_priv_uninit(dev);
 		rte_eth_dev_release_port(dev);
-		return;
+		return 0;
 	}
 	if (!priv->sh)
-		return;
+		return 0;
 	DRV_LOG(DEBUG, "port %u closing device \"%s\"",
 		dev->data->port_id,
 		((priv->sh->ctx != NULL) ?
@@ -1479,6 +1479,7 @@ mlx5_dev_close(struct rte_eth_dev *dev)
 	 * it is freed when dev_private is freed.
 	 */
 	dev->data->mac_addrs = NULL;
+	return 0;
 }
 
 /**
diff --git a/drivers/net/mlx5/mlx5.h b/drivers/net/mlx5/mlx5.h
index 865e72d318..56f0e25b4c 100644
--- a/drivers/net/mlx5/mlx5.h
+++ b/drivers/net/mlx5/mlx5.h
@@ -842,7 +842,7 @@ int mlx5_proc_priv_init(struct rte_eth_dev *dev);
 int mlx5_udp_tunnel_port_add(struct rte_eth_dev *dev,
 			      struct rte_eth_udp_tunnel *udp_tunnel);
 uint16_t mlx5_eth_find_next(uint16_t port_id, struct rte_pci_device *pci_dev);
-void mlx5_dev_close(struct rte_eth_dev *dev);
+int mlx5_dev_close(struct rte_eth_dev *dev);
 
 /* Macro to iterate over all valid ports for mlx5 driver. */
 #define MLX5_ETH_FOREACH_DEV(port_id, pci_dev) \
diff --git a/drivers/net/mvneta/mvneta_ethdev.c b/drivers/net/mvneta/mvneta_ethdev.c
index 5bf3ebc4bd..eebcdb840e 100644
--- a/drivers/net/mvneta/mvneta_ethdev.c
+++ b/drivers/net/mvneta/mvneta_ethdev.c
@@ -429,7 +429,7 @@ mvneta_dev_stop(struct rte_eth_dev *dev)
  * @param dev
  *   Pointer to Ethernet device structure.
  */
-static void
+static int
 mvneta_dev_close(struct rte_eth_dev *dev)
 {
 	struct mvneta_priv *priv = dev->data->dev_private;
@@ -455,6 +455,8 @@ mvneta_dev_close(struct rte_eth_dev *dev)
 		mvneta_neta_deinit();
 		rte_mvep_deinit(MVEP_MOD_T_NETA);
 	}
+
+	return 0;
 }
 
 /**
diff --git a/drivers/net/mvpp2/mrvl_ethdev.c b/drivers/net/mvpp2/mrvl_ethdev.c
index f5e77d07bf..27f7cfb056 100644
--- a/drivers/net/mvpp2/mrvl_ethdev.c
+++ b/drivers/net/mvpp2/mrvl_ethdev.c
@@ -855,7 +855,7 @@ mrvl_dev_stop(struct rte_eth_dev *dev)
  * @param dev
  *   Pointer to Ethernet device structure.
  */
-static void
+static int
 mrvl_dev_close(struct rte_eth_dev *dev)
 {
 	struct mrvl_priv *priv = dev->data->dev_private;
@@ -915,6 +915,8 @@ mrvl_dev_close(struct rte_eth_dev *dev)
 		mrvl_deinit_pp2();
 		rte_mvep_deinit(MVEP_MOD_T_PP2);
 	}
+
+	return 0;
 }
 
 /**
diff --git a/drivers/net/netvsc/hn_ethdev.c b/drivers/net/netvsc/hn_ethdev.c
index 229c1b9149..8968036ea3 100644
--- a/drivers/net/netvsc/hn_ethdev.c
+++ b/drivers/net/netvsc/hn_ethdev.c
@@ -841,13 +841,15 @@ hn_dev_stop(struct rte_eth_dev *dev)
 	hn_vf_stop(dev);
 }
 
-static void
+static int
 hn_dev_close(struct rte_eth_dev *dev)
 {
 	PMD_INIT_FUNC_TRACE();
 
 	hn_vf_close(dev);
 	hn_dev_free_queues(dev);
+
+	return 0;
 }
 
 static const struct eth_dev_ops hn_eth_dev_ops = {
diff --git a/drivers/net/nfb/nfb_ethdev.c b/drivers/net/nfb/nfb_ethdev.c
index b039ab6fc2..c3c3d003f1 100644
--- a/drivers/net/nfb/nfb_ethdev.c
+++ b/drivers/net/nfb/nfb_ethdev.c
@@ -209,7 +209,7 @@ nfb_eth_dev_info(struct rte_eth_dev *dev,
  * @param dev
  *   Pointer to Ethernet device structure.
  */
-static void
+static int
 nfb_eth_dev_close(struct rte_eth_dev *dev)
 {
 	struct pmd_internals *internals = dev->data->dev_private;
@@ -235,6 +235,8 @@ nfb_eth_dev_close(struct rte_eth_dev *dev)
 
 	rte_free(dev->data->mac_addrs);
 	dev->data->mac_addrs = NULL;
+
+	return 0;
 }
 
 /**
diff --git a/drivers/net/nfp/nfp_net.c b/drivers/net/nfp/nfp_net.c
index f5e4cd339d..9509dc8bd6 100644
--- a/drivers/net/nfp/nfp_net.c
+++ b/drivers/net/nfp/nfp_net.c
@@ -50,7 +50,7 @@
 #include <errno.h>
 
 /* Prototypes */
-static void nfp_net_close(struct rte_eth_dev *dev);
+static int nfp_net_close(struct rte_eth_dev *dev);
 static int nfp_net_configure(struct rte_eth_dev *dev);
 static void nfp_net_dev_interrupt_handler(void *param);
 static void nfp_net_dev_interrupt_delayed_handler(void *param);
@@ -864,7 +864,7 @@ nfp_net_set_link_down(struct rte_eth_dev *dev)
 }
 
 /* Reset and stop device. The device can not be restarted. */
-static void
+static int
 nfp_net_close(struct rte_eth_dev *dev)
 {
 	struct nfp_net_hw *hw;
@@ -906,6 +906,8 @@ nfp_net_close(struct rte_eth_dev *dev)
 	 * The ixgbe PMD driver disables the pcie master on the
 	 * device. The i40e does not...
 	 */
+
+	return 0;
 }
 
 static int
diff --git a/drivers/net/octeontx/octeontx_ethdev.c b/drivers/net/octeontx/octeontx_ethdev.c
index ff2e22cd44..10eeeac642 100644
--- a/drivers/net/octeontx/octeontx_ethdev.c
+++ b/drivers/net/octeontx/octeontx_ethdev.c
@@ -478,7 +478,7 @@ octeontx_dev_configure(struct rte_eth_dev *dev)
 	return 0;
 }
 
-static void
+static int
 octeontx_dev_close(struct rte_eth_dev *dev)
 {
 	struct octeontx_txq *txq = NULL;
@@ -517,6 +517,8 @@ octeontx_dev_close(struct rte_eth_dev *dev)
 
 	dev->tx_pkt_burst = NULL;
 	dev->rx_pkt_burst = NULL;
+
+	return 0;
 }
 
 static int
diff --git a/drivers/net/octeontx2/otx2_ethdev.c b/drivers/net/octeontx2/otx2_ethdev.c
index 51a6f99877..abe5f03628 100644
--- a/drivers/net/octeontx2/otx2_ethdev.c
+++ b/drivers/net/octeontx2/otx2_ethdev.c
@@ -2222,7 +2222,7 @@ otx2_nix_dev_start(struct rte_eth_dev *eth_dev)
 }
 
 static int otx2_nix_dev_reset(struct rte_eth_dev *eth_dev);
-static void otx2_nix_dev_close(struct rte_eth_dev *eth_dev);
+static int otx2_nix_dev_close(struct rte_eth_dev *eth_dev);
 
 /* Initialize and register driver with DPDK Application */
 static const struct eth_dev_ops otx2_eth_dev_ops = {
@@ -2635,10 +2635,11 @@ otx2_eth_dev_uninit(struct rte_eth_dev *eth_dev, bool mbox_close)
 	return 0;
 }
 
-static void
+static int
 otx2_nix_dev_close(struct rte_eth_dev *eth_dev)
 {
 	otx2_eth_dev_uninit(eth_dev, true);
+	return 0;
 }
 
 static int
diff --git a/drivers/net/pcap/rte_eth_pcap.c b/drivers/net/pcap/rte_eth_pcap.c
index 668cbd1fc7..76e704a65a 100644
--- a/drivers/net/pcap/rte_eth_pcap.c
+++ b/drivers/net/pcap/rte_eth_pcap.c
@@ -728,7 +728,7 @@ eth_stats_reset(struct rte_eth_dev *dev)
 	return 0;
 }
 
-static void
+static int
 eth_dev_close(struct rte_eth_dev *dev)
 {
 	unsigned int i;
@@ -748,6 +748,7 @@ eth_dev_close(struct rte_eth_dev *dev)
 		}
 	}
 
+	return 0;
 }
 
 static void
diff --git a/drivers/net/pfe/pfe_ethdev.c b/drivers/net/pfe/pfe_ethdev.c
index 97356d2b0b..187a0019ff 100644
--- a/drivers/net/pfe/pfe_ethdev.c
+++ b/drivers/net/pfe/pfe_ethdev.c
@@ -398,14 +398,14 @@ pfe_eth_exit(struct rte_eth_dev *dev, struct pfe *pfe)
 	pfe->nb_devs--;
 }
 
-static void
+static int
 pfe_eth_close(struct rte_eth_dev *dev)
 {
 	if (!dev)
-		return;
+		return -1;
 
 	if (!g_pfe)
-		return;
+		return -1;
 
 	pfe_eth_exit(dev, g_pfe);
 
@@ -415,6 +415,8 @@ pfe_eth_close(struct rte_eth_dev *dev)
 		rte_free(g_pfe);
 		g_pfe = NULL;
 	}
+
+	return 0;
 }
 
 static int
diff --git a/drivers/net/qede/qede_ethdev.c b/drivers/net/qede/qede_ethdev.c
index 59f1746ee9..9535df183f 100644
--- a/drivers/net/qede/qede_ethdev.c
+++ b/drivers/net/qede/qede_ethdev.c
@@ -1542,7 +1542,7 @@ static void qede_poll_sp_sb_cb(void *param)
 	}
 }
 
-static void qede_dev_close(struct rte_eth_dev *eth_dev)
+static int qede_dev_close(struct rte_eth_dev *eth_dev)
 {
 	struct rte_pci_device *pci_dev = RTE_ETH_DEV_TO_PCI(eth_dev);
 	struct qede_dev *qdev = QEDE_INIT_QDEV(eth_dev);
@@ -1586,6 +1586,8 @@ static void qede_dev_close(struct rte_eth_dev *eth_dev)
 
 	if (ECORE_IS_CMT(edev))
 		rte_eal_alarm_cancel(qede_poll_sp_sb_cb, (void *)eth_dev);
+
+	return 0;
 }
 
 static int
diff --git a/drivers/net/sfc/sfc_ethdev.c b/drivers/net/sfc/sfc_ethdev.c
index 1a58e0df84..eaa48627d6 100644
--- a/drivers/net/sfc/sfc_ethdev.c
+++ b/drivers/net/sfc/sfc_ethdev.c
@@ -318,7 +318,7 @@ sfc_dev_set_link_down(struct rte_eth_dev *dev)
 	return 0;
 }
 
-static void
+static int
 sfc_dev_close(struct rte_eth_dev *dev)
 {
 	struct sfc_adapter *sa = sfc_adapter_by_eth_dev(dev);
@@ -364,6 +364,8 @@ sfc_dev_close(struct rte_eth_dev *dev)
 
 	dev->process_private = NULL;
 	free(sa);
+
+	return 0;
 }
 
 static int
diff --git a/drivers/net/softnic/rte_eth_softnic.c b/drivers/net/softnic/rte_eth_softnic.c
index e453fd1486..038abd98da 100644
--- a/drivers/net/softnic/rte_eth_softnic.c
+++ b/drivers/net/softnic/rte_eth_softnic.c
@@ -201,10 +201,10 @@ pmd_dev_stop(struct rte_eth_dev *dev)
 	softnic_mtr_free(p);
 }
 
-static void
+static int
 pmd_dev_close(struct rte_eth_dev *dev __rte_unused)
 {
-	return;
+	return 0;
 }
 
 static int
diff --git a/drivers/net/szedata2/rte_eth_szedata2.c b/drivers/net/szedata2/rte_eth_szedata2.c
index a17c53577c..daa40cfd6c 100644
--- a/drivers/net/szedata2/rte_eth_szedata2.c
+++ b/drivers/net/szedata2/rte_eth_szedata2.c
@@ -1155,7 +1155,7 @@ eth_tx_queue_release(void *q)
 	}
 }
 
-static void
+static int
 eth_dev_close(struct rte_eth_dev *dev)
 {
 	struct pmd_internals *internals = dev->data->dev_private;
@@ -1180,6 +1180,8 @@ eth_dev_close(struct rte_eth_dev *dev)
 
 	rte_free(dev->data->mac_addrs);
 	dev->data->mac_addrs = NULL;
+
+	return 0;
 }
 
 static int
diff --git a/drivers/net/tap/rte_eth_tap.c b/drivers/net/tap/rte_eth_tap.c
index df16aa4ea7..2d1e4e365a 100644
--- a/drivers/net/tap/rte_eth_tap.c
+++ b/drivers/net/tap/rte_eth_tap.c
@@ -1067,7 +1067,7 @@ tap_stats_reset(struct rte_eth_dev *dev)
 	return 0;
 }
 
-static void
+static int
 tap_dev_close(struct rte_eth_dev *dev)
 {
 	int i;
@@ -1116,6 +1116,8 @@ tap_dev_close(struct rte_eth_dev *dev)
 	 * Since TUN device has no more opened file descriptors
 	 * it will be removed from kernel
 	 */
+
+	return 0;
 }
 
 static void
diff --git a/drivers/net/thunderx/nicvf_ethdev.c b/drivers/net/thunderx/nicvf_ethdev.c
index 83d9488360..3d7348771a 100644
--- a/drivers/net/thunderx/nicvf_ethdev.c
+++ b/drivers/net/thunderx/nicvf_ethdev.c
@@ -1852,7 +1852,7 @@ nicvf_vf_stop(struct rte_eth_dev *dev, struct nicvf *nic, bool cleanup)
 	}
 }
 
-static void
+static int
 nicvf_dev_close(struct rte_eth_dev *dev)
 {
 	size_t i;
@@ -1869,6 +1869,8 @@ nicvf_dev_close(struct rte_eth_dev *dev)
 
 		nicvf_periodic_alarm_stop(nicvf_vf_interrupt, nic->snicvf[i]);
 	}
+
+	return 0;
 }
 
 static int
diff --git a/drivers/net/vhost/rte_eth_vhost.c b/drivers/net/vhost/rte_eth_vhost.c
index ce32be9ce3..f67f4db812 100644
--- a/drivers/net/vhost/rte_eth_vhost.c
+++ b/drivers/net/vhost/rte_eth_vhost.c
@@ -1164,7 +1164,7 @@ eth_dev_stop(struct rte_eth_dev *dev)
 	update_queuing_status(dev);
 }
 
-static void
+static int
 eth_dev_close(struct rte_eth_dev *dev)
 {
 	struct pmd_internal *internal;
@@ -1173,7 +1173,7 @@ eth_dev_close(struct rte_eth_dev *dev)
 
 	internal = dev->data->dev_private;
 	if (!internal)
-		return;
+		return 0;
 
 	eth_dev_stop(dev);
 
@@ -1201,6 +1201,8 @@ eth_dev_close(struct rte_eth_dev *dev)
 
 	rte_free(vring_states[dev->data->port_id]);
 	vring_states[dev->data->port_id] = NULL;
+
+	return 0;
 }
 
 static int
diff --git a/drivers/net/virtio/virtio_ethdev.c b/drivers/net/virtio/virtio_ethdev.c
index 013a2904e6..0787337a01 100644
--- a/drivers/net/virtio/virtio_ethdev.c
+++ b/drivers/net/virtio/virtio_ethdev.c
@@ -704,7 +704,7 @@ virtio_alloc_queues(struct rte_eth_dev *dev)
 
 static void virtio_queues_unbind_intr(struct rte_eth_dev *dev);
 
-static void
+static int
 virtio_dev_close(struct rte_eth_dev *dev)
 {
 	struct virtio_hw *hw = dev->data->dev_private;
@@ -713,7 +713,7 @@ virtio_dev_close(struct rte_eth_dev *dev)
 	PMD_INIT_LOG(DEBUG, "virtio_dev_close");
 
 	if (!hw->opened)
-		return;
+		return 0;
 	hw->opened = false;
 
 	/* reset the NIC */
@@ -743,6 +743,8 @@ virtio_dev_close(struct rte_eth_dev *dev)
 		if (!hw->modern)
 			rte_pci_ioport_unmap(VTPCI_IO(hw));
 	}
+
+	return 0;
 }
 
 static int
diff --git a/drivers/net/vmxnet3/vmxnet3_ethdev.c b/drivers/net/vmxnet3/vmxnet3_ethdev.c
index fa80e75a5b..17fed0ed6e 100644
--- a/drivers/net/vmxnet3/vmxnet3_ethdev.c
+++ b/drivers/net/vmxnet3/vmxnet3_ethdev.c
@@ -64,7 +64,7 @@ static int eth_vmxnet3_dev_uninit(struct rte_eth_dev *eth_dev);
 static int vmxnet3_dev_configure(struct rte_eth_dev *dev);
 static int vmxnet3_dev_start(struct rte_eth_dev *dev);
 static void vmxnet3_dev_stop(struct rte_eth_dev *dev);
-static void vmxnet3_dev_close(struct rte_eth_dev *dev);
+static int vmxnet3_dev_close(struct rte_eth_dev *dev);
 static void vmxnet3_dev_set_rxmode(struct vmxnet3_hw *hw, uint32_t feature, int set);
 static int vmxnet3_dev_promiscuous_enable(struct rte_eth_dev *dev);
 static int vmxnet3_dev_promiscuous_disable(struct rte_eth_dev *dev);
@@ -888,13 +888,15 @@ vmxnet3_free_queues(struct rte_eth_dev *dev)
 /*
  * Reset and stop device.
  */
-static void
+static int
 vmxnet3_dev_close(struct rte_eth_dev *dev)
 {
 	PMD_INIT_FUNC_TRACE();
 
 	vmxnet3_dev_stop(dev);
 	vmxnet3_free_queues(dev);
+
+	return 0;
 }
 
 static void
diff --git a/lib/librte_ethdev/rte_ethdev_driver.h b/lib/librte_ethdev/rte_ethdev_driver.h
index 23cc1e0959..6abfd58d2f 100644
--- a/lib/librte_ethdev/rte_ethdev_driver.h
+++ b/lib/librte_ethdev/rte_ethdev_driver.h
@@ -42,7 +42,7 @@ typedef int  (*eth_dev_set_link_up_t)(struct rte_eth_dev *dev);
 typedef int  (*eth_dev_set_link_down_t)(struct rte_eth_dev *dev);
 /**< @internal Function used to link down a configured Ethernet device. */
 
-typedef void (*eth_dev_close_t)(struct rte_eth_dev *dev);
+typedef int (*eth_dev_close_t)(struct rte_eth_dev *dev);
 /**< @internal Function used to close a configured Ethernet device. */
 
 typedef int (*eth_dev_reset_t)(struct rte_eth_dev *dev);
-- 
2.28.0

Re: [dpdk-dev] [PATCH v2 02/25] ethdev: allow drivers to return error on close

[Xu, Rosen]

Hi,

> -----Original Message-----
> From: Thomas Monjalon <thomas@monjalon.net>
> Sent: Monday, September 28, 2020 7:42
> To: dev@dpdk.org
> Cc: Yigit, Ferruh <ferruh.yigit@intel.com>; arybchenko@solarflare.com; John
> W. Linville <linville@tuxdriver.com>; Loftus, Ciara <ciara.loftus@intel.com>;
> Zhang, Qi Z <qi.z.zhang@intel.com>; Shepard Siegel
> <shepard.siegel@atomicrules.com>; Ed Czeck <ed.czeck@atomicrules.com>;
> John Miller <john.miller@atomicrules.com>; Igor Russkikh
> <igor.russkikh@aquantia.com>; Pavel Belous <pavel.belous@aquantia.com>;
> Steven Webster <steven.webster@windriver.com>; Matt Peters
> <matt.peters@windriver.com>; Somalapuram Amaranath
> <asomalap@amd.com>; Rasesh Mody <rmody@marvell.com>; Shahed
> Shaikh <shshaikh@marvell.com>; Ajit Khaparde
> <ajit.khaparde@broadcom.com>; Somnath Kotur
> <somnath.kotur@broadcom.com>; Chas Williams <chas3@att.com>; Wei Hu
> (Xavier) <xavier.huwei@huawei.com>; Rahul Lakkireddy
> <rahul.lakkireddy@chelsio.com>; Hemant Agrawal
> <hemant.agrawal@nxp.com>; Sachin Saxena <sachin.saxena@oss.nxp.com>;
> Guo, Jia <jia.guo@intel.com>; Wang, Haiyue <haiyue.wang@intel.com>;
> Marcin Wojtas <mw@semihalf.com>; Michal Krawczyk <mk@semihalf.com>;
> Guy Tzalik <gtzalik@amazon.com>; Evgeny Schemeilin
> <evgenys@amazon.com>; Igor Chauskin <igorch@amazon.com>;
> Gagandeep Singh <g.singh@nxp.com>; John Daley <johndale@cisco.com>;
> Hyong Youb Kim <hyonkim@cisco.com>; Gaetan Rivet <grive@u256.net>;
> Wang, Xiao W <xiao.w.wang@intel.com>; Ziyang Xuan
> <xuanziyang2@huawei.com>; Xiaoyun Wang
> <cloud.wangxiaoyun@huawei.com>; Guoyang Zhou
> <zhouguoyang@huawei.com>; Min Hu (Connor) <humin29@huawei.com>;
> Yisen Zhuang <yisen.zhuang@huawei.com>; Xing, Beilei
> <beilei.xing@intel.com>; Wu, Jingjing <jingjing.wu@intel.com>; Yang, Qiming
> <qiming.yang@intel.com>; Alfredo Cardigliano <cardigliano@ntop.org>; Xu,
> Rosen <rosen.xu@intel.com>; Shijith Thotton <sthotton@marvell.com>;
> Srisivasubramanian Srinivasan <srinivasan@marvell.com>; Jakub Grajciar
> <jgrajcia@cisco.com>; Matan Azrad <matan@nvidia.com>; Shahaf Shuler
> <shahafs@nvidia.com>; Viacheslav Ovsiienko <viacheslavo@nvidia.com>;
> Zyta Szpak <zr@semihalf.com>; Liron Himi <lironh@marvell.com>; Stephen
> Hemminger <sthemmin@microsoft.com>; K. Y. Srinivasan
> <kys@microsoft.com>; Haiyang Zhang <haiyangz@microsoft.com>; Long Li
> <longli@microsoft.com>; Martin Spinler <spinler@cesnet.cz>; Heinrich Kuhn
> <heinrich.kuhn@netronome.com>; Harman Kalra <hkalra@marvell.com>;
> Jerin Jacob <jerinj@marvell.com>; Nithin Dabilpuram
> <ndabilpuram@marvell.com>; Kiran Kumar K <kirankumark@marvell.com>;
> Akhil Goyal <akhil.goyal@nxp.com>; Singh, Jasvinder
> <jasvinder.singh@intel.com>; Dumitrescu, Cristian
> <cristian.dumitrescu@intel.com>; Wiles, Keith <keith.wiles@intel.com>;
> Maciej Czekaj <mczekaj@marvell.com>; Maxime Coquelin
> <maxime.coquelin@redhat.com>; Xia, Chenbo <chenbo.xia@intel.com>;
> Wang, Zhihong <zhihong.wang@intel.com>; Yong Wang
> <yongwang@vmware.com>
> Subject: [PATCH v2 02/25] ethdev: allow drivers to return error on close
> 
> The device operation .dev_close was returning void.
> This driver interface is changed to return an int.
> 
> Note that the API rte_eth_dev_close() is still returning void,
> although a deprecation notice is pending to change it as well.
> 
> Signed-off-by: Thomas Monjalon <thomas@monjalon.net>
> ---
>  app/test/virtual_pmd.c                    |  6 ++++--
>  drivers/net/af_packet/rte_eth_af_packet.c |  3 ++-
>  drivers/net/af_xdp/rte_eth_af_xdp.c       |  4 +++-
>  drivers/net/ark/ark_ethdev.c              |  6 ++++--
>  drivers/net/atlantic/atl_ethdev.c         |  6 ++++--
>  drivers/net/avp/avp_ethdev.c              |  5 +++--
>  drivers/net/axgbe/axgbe_ethdev.c          |  5 +++--
>  drivers/net/bnx2x/bnx2x_ethdev.c          |  4 +++-
>  drivers/net/bnxt/bnxt_ethdev.c            |  4 +++-
>  drivers/net/bnxt/bnxt_reps.c              |  3 ++-
>  drivers/net/bnxt/bnxt_reps.h              |  2 +-
>  drivers/net/bonding/eth_bond_private.h    |  2 +-
>  drivers/net/bonding/rte_eth_bond_pmd.c    |  4 +++-
>  drivers/net/cxgbe/cxgbe_ethdev.c          | 10 ++++++----
>  drivers/net/cxgbe/cxgbe_pfvf.h            |  2 +-
>  drivers/net/dpaa/dpaa_ethdev.c            |  4 +++-
>  drivers/net/dpaa2/dpaa2_ethdev.c          |  6 ++++--
>  drivers/net/e1000/em_ethdev.c             |  6 ++++--
>  drivers/net/e1000/igb_ethdev.c            | 12 ++++++++----
>  drivers/net/ena/ena_ethdev.c              |  6 ++++--
>  drivers/net/enetc/enetc_ethdev.c          |  4 +++-
>  drivers/net/enic/enic_ethdev.c            |  4 +++-
>  drivers/net/enic/enic_vf_representor.c    |  5 +++--
>  drivers/net/failsafe/failsafe_ops.c       |  3 ++-
>  drivers/net/fm10k/fm10k_ethdev.c          |  4 +++-
>  drivers/net/hinic/hinic_pmd_ethdev.c      |  6 ++++--
>  drivers/net/hns3/hns3_ethdev.c            |  6 ++++--
>  drivers/net/hns3/hns3_ethdev_vf.c         |  6 ++++--
>  drivers/net/i40e/i40e_ethdev.c            |  5 +++--
>  drivers/net/i40e/i40e_ethdev_vf.c         |  5 +++--
>  drivers/net/iavf/iavf_ethdev.c            |  6 ++++--
>  drivers/net/ice/ice_dcf_ethdev.c          |  6 ++++--
>  drivers/net/ice/ice_ethdev.c              |  6 ++++--
>  drivers/net/igc/igc_ethdev.c              |  6 ++++--
>  drivers/net/ionic/ionic_ethdev.c          | 10 ++++++----
>  drivers/net/ipn3ke/ipn3ke_representor.c   |  4 +++-
>  drivers/net/ixgbe/ixgbe_ethdev.c          | 11 +++++++----
>  drivers/net/kni/rte_eth_kni.c             |  4 +++-
>  drivers/net/liquidio/lio_ethdev.c         |  4 +++-
>  drivers/net/memif/rte_eth_memif.c         |  4 +++-
>  drivers/net/mlx4/mlx4.c                   |  3 ++-
>  drivers/net/mlx5/mlx5.c                   |  9 +++++----
>  drivers/net/mlx5/mlx5.h                   |  2 +-
>  drivers/net/mvneta/mvneta_ethdev.c        |  4 +++-
>  drivers/net/mvpp2/mrvl_ethdev.c           |  4 +++-
>  drivers/net/netvsc/hn_ethdev.c            |  4 +++-
>  drivers/net/nfb/nfb_ethdev.c              |  4 +++-
>  drivers/net/nfp/nfp_net.c                 |  6 ++++--
>  drivers/net/octeontx/octeontx_ethdev.c    |  4 +++-
>  drivers/net/octeontx2/otx2_ethdev.c       |  5 +++--
>  drivers/net/pcap/rte_eth_pcap.c           |  3 ++-
>  drivers/net/pfe/pfe_ethdev.c              |  8 +++++---
>  drivers/net/qede/qede_ethdev.c            |  4 +++-
>  drivers/net/sfc/sfc_ethdev.c              |  4 +++-
>  drivers/net/softnic/rte_eth_softnic.c     |  4 ++--
>  drivers/net/szedata2/rte_eth_szedata2.c   |  4 +++-
>  drivers/net/tap/rte_eth_tap.c             |  4 +++-
>  drivers/net/thunderx/nicvf_ethdev.c       |  4 +++-
>  drivers/net/vhost/rte_eth_vhost.c         |  6 ++++--
>  drivers/net/virtio/virtio_ethdev.c        |  6 ++++--
>  drivers/net/vmxnet3/vmxnet3_ethdev.c      |  6 ++++--
>  lib/librte_ethdev/rte_ethdev_driver.h     |  2 +-
>  62 files changed, 206 insertions(+), 103 deletions(-)
> 
>  static int
> diff --git a/drivers/net/ipn3ke/ipn3ke_representor.c
> b/drivers/net/ipn3ke/ipn3ke_representor.c
> index b673c49149..d49abbf758 100644
> --- a/drivers/net/ipn3ke/ipn3ke_representor.c
> +++ b/drivers/net/ipn3ke/ipn3ke_representor.c
> @@ -208,7 +208,7 @@ ipn3ke_rpst_dev_stop(struct rte_eth_dev *dev)
>  	}
>  }
> 
> -static void
> +static int
>  ipn3ke_rpst_dev_close(struct rte_eth_dev *dev)
>  {
>  	struct ipn3ke_hw *hw = IPN3KE_DEV_PRIVATE_TO_HW(dev);
> @@ -221,6 +221,8 @@ ipn3ke_rpst_dev_close(struct rte_eth_dev *dev)
>  		/* Disable the RX path */
>  		ipn3ke_xmac_rx_disable(hw, rpst->port_id, 0);
>  	}
> +
> +	return 0;
>  }
> 
>  /*
> 2.28.0

For net/ipn3ke
Reviewed-by: Rosen Xu <rosen.xu@intel.com>

Re: [dpdk-dev] [PATCH v2 02/25] ethdev: allow drivers to return error on close

[Sachin Saxena (OSS)]

For dpaa, dpaa2, enetc and pfe.

Reviewed-by: Sachin Saxena<sachin.saxena@oss.nxp.com>


On 28-Sep-20 5:12 AM, Thomas Monjalon wrote:
> The device operation .dev_close was returning void.
> This driver interface is changed to return an int.
>
> Note that the API rte_eth_dev_close() is still returning void,
> although a deprecation notice is pending to change it as well.
>
> Signed-off-by: Thomas Monjalon <thomas@monjalon.net>
> ---
>   app/test/virtual_pmd.c                    |  6 ++++--
>   drivers/net/af_packet/rte_eth_af_packet.c |  3 ++-
>   drivers/net/af_xdp/rte_eth_af_xdp.c       |  4 +++-
>   drivers/net/ark/ark_ethdev.c              |  6 ++++--
>   drivers/net/atlantic/atl_ethdev.c         |  6 ++++--
>   drivers/net/avp/avp_ethdev.c              |  5 +++--
>   drivers/net/axgbe/axgbe_ethdev.c          |  5 +++--
>   drivers/net/bnx2x/bnx2x_ethdev.c          |  4 +++-
>   drivers/net/bnxt/bnxt_ethdev.c            |  4 +++-
>   drivers/net/bnxt/bnxt_reps.c              |  3 ++-
>   drivers/net/bnxt/bnxt_reps.h              |  2 +-
>   drivers/net/bonding/eth_bond_private.h    |  2 +-
>   drivers/net/bonding/rte_eth_bond_pmd.c    |  4 +++-
>   drivers/net/cxgbe/cxgbe_ethdev.c          | 10 ++++++----
>   drivers/net/cxgbe/cxgbe_pfvf.h            |  2 +-
>   drivers/net/dpaa/dpaa_ethdev.c            |  4 +++-
>   drivers/net/dpaa2/dpaa2_ethdev.c          |  6 ++++--
>   drivers/net/e1000/em_ethdev.c             |  6 ++++--
>   drivers/net/e1000/igb_ethdev.c            | 12 ++++++++----
>   drivers/net/ena/ena_ethdev.c              |  6 ++++--
>   drivers/net/enetc/enetc_ethdev.c          |  4 +++-
>   drivers/net/enic/enic_ethdev.c            |  4 +++-
>   drivers/net/enic/enic_vf_representor.c    |  5 +++--
>   drivers/net/failsafe/failsafe_ops.c       |  3 ++-
>   drivers/net/fm10k/fm10k_ethdev.c          |  4 +++-
>   drivers/net/hinic/hinic_pmd_ethdev.c      |  6 ++++--
>   drivers/net/hns3/hns3_ethdev.c            |  6 ++++--
>   drivers/net/hns3/hns3_ethdev_vf.c         |  6 ++++--
>   drivers/net/i40e/i40e_ethdev.c            |  5 +++--
>   drivers/net/i40e/i40e_ethdev_vf.c         |  5 +++--
>   drivers/net/iavf/iavf_ethdev.c            |  6 ++++--
>   drivers/net/ice/ice_dcf_ethdev.c          |  6 ++++--
>   drivers/net/ice/ice_ethdev.c              |  6 ++++--
>   drivers/net/igc/igc_ethdev.c              |  6 ++++--
>   drivers/net/ionic/ionic_ethdev.c          | 10 ++++++----
>   drivers/net/ipn3ke/ipn3ke_representor.c   |  4 +++-
>   drivers/net/ixgbe/ixgbe_ethdev.c          | 11 +++++++----
>   drivers/net/kni/rte_eth_kni.c             |  4 +++-
>   drivers/net/liquidio/lio_ethdev.c         |  4 +++-
>   drivers/net/memif/rte_eth_memif.c         |  4 +++-
>   drivers/net/mlx4/mlx4.c                   |  3 ++-
>   drivers/net/mlx5/mlx5.c                   |  9 +++++----
>   drivers/net/mlx5/mlx5.h                   |  2 +-
>   drivers/net/mvneta/mvneta_ethdev.c        |  4 +++-
>   drivers/net/mvpp2/mrvl_ethdev.c           |  4 +++-
>   drivers/net/netvsc/hn_ethdev.c            |  4 +++-
>   drivers/net/nfb/nfb_ethdev.c              |  4 +++-
>   drivers/net/nfp/nfp_net.c                 |  6 ++++--
>   drivers/net/octeontx/octeontx_ethdev.c    |  4 +++-
>   drivers/net/octeontx2/otx2_ethdev.c       |  5 +++--
>   drivers/net/pcap/rte_eth_pcap.c           |  3 ++-
>   drivers/net/pfe/pfe_ethdev.c              |  8 +++++---
>   drivers/net/qede/qede_ethdev.c            |  4 +++-
>   drivers/net/sfc/sfc_ethdev.c              |  4 +++-
>   drivers/net/softnic/rte_eth_softnic.c     |  4 ++--
>   drivers/net/szedata2/rte_eth_szedata2.c   |  4 +++-
>   drivers/net/tap/rte_eth_tap.c             |  4 +++-
>   drivers/net/thunderx/nicvf_ethdev.c       |  4 +++-
>   drivers/net/vhost/rte_eth_vhost.c         |  6 ++++--
>   drivers/net/virtio/virtio_ethdev.c        |  6 ++++--
>   drivers/net/vmxnet3/vmxnet3_ethdev.c      |  6 ++++--
>   lib/librte_ethdev/rte_ethdev_driver.h     |  2 +-
>   62 files changed, 206 insertions(+), 103 deletions(-)
>
> diff --git a/app/test/virtual_pmd.c b/app/test/virtual_pmd.c
> index b49089a33c..4bd4d1c9ad 100644
> --- a/app/test/virtual_pmd.c
> +++ b/app/test/virtual_pmd.c
> @@ -62,9 +62,11 @@ static void  virtual_ethdev_stop(struct rte_eth_dev *eth_dev __rte_unused)
>   		rte_pktmbuf_free(pkt);
>   }
>   
> -static void
> +static int
>   virtual_ethdev_close(struct rte_eth_dev *dev __rte_unused)
> -{}
> +{
> +	return 0;
> +}
>   
>   static int
>   virtual_ethdev_configure_success(struct rte_eth_dev *dev __rte_unused)
> diff --git a/drivers/net/af_packet/rte_eth_af_packet.c b/drivers/net/af_packet/rte_eth_af_packet.c
> index b9723e9619..7d0ff1cbb3 100644
> --- a/drivers/net/af_packet/rte_eth_af_packet.c
> +++ b/drivers/net/af_packet/rte_eth_af_packet.c
> @@ -376,9 +376,10 @@ eth_stats_reset(struct rte_eth_dev *dev)
>   	return 0;
>   }
>   
> -static void
> +static int
>   eth_dev_close(struct rte_eth_dev *dev __rte_unused)
>   {
> +	return 0;
>   }
>   
>   static void
> diff --git a/drivers/net/af_xdp/rte_eth_af_xdp.c b/drivers/net/af_xdp/rte_eth_af_xdp.c
> index b65ee449fc..badbce63fb 100644
> --- a/drivers/net/af_xdp/rte_eth_af_xdp.c
> +++ b/drivers/net/af_xdp/rte_eth_af_xdp.c
> @@ -701,7 +701,7 @@ xdp_umem_destroy(struct xsk_umem_info *umem)
>   	umem = NULL;
>   }
>   
> -static void
> +static int
>   eth_dev_close(struct rte_eth_dev *dev)
>   {
>   	struct pmd_internals *internals = dev->data->dev_private;
> @@ -731,6 +731,8 @@ eth_dev_close(struct rte_eth_dev *dev)
>   	dev->data->mac_addrs = NULL;
>   
>   	remove_xdp_program(internals);
> +
> +	return 0;
>   }
>   
>   static void
> diff --git a/drivers/net/ark/ark_ethdev.c b/drivers/net/ark/ark_ethdev.c
> index e13415e95b..5f2ed4b7b7 100644
> --- a/drivers/net/ark/ark_ethdev.c
> +++ b/drivers/net/ark/ark_ethdev.c
> @@ -30,7 +30,7 @@ static int eth_ark_dev_uninit(struct rte_eth_dev *eth_dev);
>   static int eth_ark_dev_configure(struct rte_eth_dev *dev);
>   static int eth_ark_dev_start(struct rte_eth_dev *dev);
>   static void eth_ark_dev_stop(struct rte_eth_dev *dev);
> -static void eth_ark_dev_close(struct rte_eth_dev *dev);
> +static int eth_ark_dev_close(struct rte_eth_dev *dev);
>   static int eth_ark_dev_info_get(struct rte_eth_dev *dev,
>   				struct rte_eth_dev_info *dev_info);
>   static int eth_ark_dev_link_update(struct rte_eth_dev *dev,
> @@ -676,7 +676,7 @@ eth_ark_dev_stop(struct rte_eth_dev *dev)
>   	}
>   }
>   
> -static void
> +static int
>   eth_ark_dev_close(struct rte_eth_dev *dev)
>   {
>   	struct ark_adapter *ark = dev->data->dev_private;
> @@ -706,6 +706,8 @@ eth_ark_dev_close(struct rte_eth_dev *dev)
>   
>   	rte_free(dev->data->mac_addrs);
>   	dev->data->mac_addrs = 0;
> +
> +	return 0;
>   }
>   
>   static int
> diff --git a/drivers/net/atlantic/atl_ethdev.c b/drivers/net/atlantic/atl_ethdev.c
> index 2fca9140d7..254758fd66 100644
> --- a/drivers/net/atlantic/atl_ethdev.c
> +++ b/drivers/net/atlantic/atl_ethdev.c
> @@ -22,7 +22,7 @@ static int  atl_dev_start(struct rte_eth_dev *dev);
>   static void atl_dev_stop(struct rte_eth_dev *dev);
>   static int  atl_dev_set_link_up(struct rte_eth_dev *dev);
>   static int  atl_dev_set_link_down(struct rte_eth_dev *dev);
> -static void atl_dev_close(struct rte_eth_dev *dev);
> +static int  atl_dev_close(struct rte_eth_dev *dev);
>   static int  atl_dev_reset(struct rte_eth_dev *dev);
>   static int  atl_dev_promiscuous_enable(struct rte_eth_dev *dev);
>   static int  atl_dev_promiscuous_disable(struct rte_eth_dev *dev);
> @@ -719,7 +719,7 @@ atl_dev_set_link_down(struct rte_eth_dev *dev)
>   /*
>    * Reset and stop device.
>    */
> -static void
> +static int
>   atl_dev_close(struct rte_eth_dev *dev)
>   {
>   	PMD_INIT_FUNC_TRACE();
> @@ -727,6 +727,8 @@ atl_dev_close(struct rte_eth_dev *dev)
>   	atl_dev_stop(dev);
>   
>   	atl_free_queues(dev);
> +
> +	return 0;
>   }
>   
>   static int
> diff --git a/drivers/net/avp/avp_ethdev.c b/drivers/net/avp/avp_ethdev.c
> index 5b47f0924a..2eed6acc3f 100644
> --- a/drivers/net/avp/avp_ethdev.c
> +++ b/drivers/net/avp/avp_ethdev.c
> @@ -38,7 +38,7 @@ static int avp_dev_create(struct rte_pci_device *pci_dev,
>   static int avp_dev_configure(struct rte_eth_dev *dev);
>   static int avp_dev_start(struct rte_eth_dev *dev);
>   static void avp_dev_stop(struct rte_eth_dev *dev);
> -static void avp_dev_close(struct rte_eth_dev *dev);
> +static int avp_dev_close(struct rte_eth_dev *dev);
>   static int avp_dev_info_get(struct rte_eth_dev *dev,
>   			    struct rte_eth_dev_info *dev_info);
>   static int avp_vlan_offload_set(struct rte_eth_dev *dev, int mask);
> @@ -2103,7 +2103,7 @@ avp_dev_stop(struct rte_eth_dev *eth_dev)
>   	rte_spinlock_unlock(&avp->lock);
>   }
>   
> -static void
> +static int
>   avp_dev_close(struct rte_eth_dev *eth_dev)
>   {
>   	struct avp_dev *avp = AVP_DEV_PRIVATE_TO_HW(eth_dev->data->dev_private);
> @@ -2139,6 +2139,7 @@ avp_dev_close(struct rte_eth_dev *eth_dev)
>   
>   unlock:
>   	rte_spinlock_unlock(&avp->lock);
> +	return 0;
>   }
>   
>   static int
> diff --git a/drivers/net/axgbe/axgbe_ethdev.c b/drivers/net/axgbe/axgbe_ethdev.c
> index 4f5a014b54..458d64bcef 100644
> --- a/drivers/net/axgbe/axgbe_ethdev.c
> +++ b/drivers/net/axgbe/axgbe_ethdev.c
> @@ -16,7 +16,7 @@ static int  axgbe_dev_configure(struct rte_eth_dev *dev);
>   static int  axgbe_dev_start(struct rte_eth_dev *dev);
>   static void axgbe_dev_stop(struct rte_eth_dev *dev);
>   static void axgbe_dev_interrupt_handler(void *param);
> -static void axgbe_dev_close(struct rte_eth_dev *dev);
> +static int axgbe_dev_close(struct rte_eth_dev *dev);
>   static int axgbe_dev_promiscuous_enable(struct rte_eth_dev *dev);
>   static int axgbe_dev_promiscuous_disable(struct rte_eth_dev *dev);
>   static int axgbe_dev_allmulticast_enable(struct rte_eth_dev *dev);
> @@ -410,10 +410,11 @@ axgbe_dev_stop(struct rte_eth_dev *dev)
>   }
>   
>   /* Clear all resources like TX/RX queues. */
> -static void
> +static int
>   axgbe_dev_close(struct rte_eth_dev *dev)
>   {
>   	axgbe_dev_clear_queues(dev);
> +	return 0;
>   }
>   
>   static int
> diff --git a/drivers/net/bnx2x/bnx2x_ethdev.c b/drivers/net/bnx2x/bnx2x_ethdev.c
> index b2ea5fafa7..73217c2138 100644
> --- a/drivers/net/bnx2x/bnx2x_ethdev.c
> +++ b/drivers/net/bnx2x/bnx2x_ethdev.c
> @@ -280,7 +280,7 @@ bnx2x_dev_stop(struct rte_eth_dev *dev)
>   	return;
>   }
>   
> -static void
> +static int
>   bnx2x_dev_close(struct rte_eth_dev *dev)
>   {
>   	struct bnx2x_softc *sc = dev->data->dev_private;
> @@ -295,6 +295,8 @@ bnx2x_dev_close(struct rte_eth_dev *dev)
>   
>   	/* free ilt */
>   	bnx2x_free_ilt_mem(sc);
> +
> +	return 0;
>   }
>   
>   static int
> diff --git a/drivers/net/bnxt/bnxt_ethdev.c b/drivers/net/bnxt/bnxt_ethdev.c
> index 05e9a6abbf..e726d078ec 100644
> --- a/drivers/net/bnxt/bnxt_ethdev.c
> +++ b/drivers/net/bnxt/bnxt_ethdev.c
> @@ -1357,7 +1357,7 @@ static void bnxt_dev_stop_op(struct rte_eth_dev *eth_dev)
>   		bp->flow_stat->flow_count = 0;
>   }
>   
> -static void bnxt_dev_close_op(struct rte_eth_dev *eth_dev)
> +static int bnxt_dev_close_op(struct rte_eth_dev *eth_dev)
>   {
>   	struct bnxt *bp = eth_dev->data->dev_private;
>   
> @@ -1392,6 +1392,8 @@ static void bnxt_dev_close_op(struct rte_eth_dev *eth_dev)
>   
>   	rte_free(bp->grp_info);
>   	bp->grp_info = NULL;
> +
> +	return 0;
>   }
>   
>   static void bnxt_mac_addr_remove_op(struct rte_eth_dev *eth_dev,
> diff --git a/drivers/net/bnxt/bnxt_reps.c b/drivers/net/bnxt/bnxt_reps.c
> index 17010f1ee2..2b6c0a277b 100644
> --- a/drivers/net/bnxt/bnxt_reps.c
> +++ b/drivers/net/bnxt/bnxt_reps.c
> @@ -457,10 +457,11 @@ void bnxt_vf_rep_dev_stop_op(struct rte_eth_dev *eth_dev)
>   	bnxt_vf_rep_free_rx_mbufs(vfr_bp);
>   }
>   
> -void bnxt_vf_rep_dev_close_op(struct rte_eth_dev *eth_dev)
> +int bnxt_vf_rep_dev_close_op(struct rte_eth_dev *eth_dev)
>   {
>   	BNXT_TF_DBG(DEBUG, "BNXT Port:%d VFR close\n", eth_dev->data->port_id);
>   	bnxt_vf_representor_uninit(eth_dev);
> +	return 0;
>   }
>   
>   int bnxt_vf_rep_dev_info_get_op(struct rte_eth_dev *eth_dev,
> diff --git a/drivers/net/bnxt/bnxt_reps.h b/drivers/net/bnxt/bnxt_reps.h
> index 3239e03fc0..d9b9ee2757 100644
> --- a/drivers/net/bnxt/bnxt_reps.h
> +++ b/drivers/net/bnxt/bnxt_reps.h
> @@ -45,7 +45,7 @@ int bnxt_vf_rep_tx_queue_setup_op(struct rte_eth_dev *eth_dev,
>   void bnxt_vf_rep_rx_queue_release_op(void *rx_queue);
>   void bnxt_vf_rep_tx_queue_release_op(void *tx_queue);
>   void bnxt_vf_rep_dev_stop_op(struct rte_eth_dev *eth_dev);
> -void bnxt_vf_rep_dev_close_op(struct rte_eth_dev *eth_dev);
> +int bnxt_vf_rep_dev_close_op(struct rte_eth_dev *eth_dev);
>   int bnxt_vf_rep_stats_get_op(struct rte_eth_dev *eth_dev,
>   			     struct rte_eth_stats *stats);
>   int bnxt_vf_rep_stats_reset_op(struct rte_eth_dev *eth_dev);
> diff --git a/drivers/net/bonding/eth_bond_private.h b/drivers/net/bonding/eth_bond_private.h
> index c9b2d0fe46..0a0034705d 100644
> --- a/drivers/net/bonding/eth_bond_private.h
> +++ b/drivers/net/bonding/eth_bond_private.h
> @@ -318,7 +318,7 @@ bond_tlb_activate_slave(struct bond_dev_private *internals);
>   void
>   bond_ethdev_stop(struct rte_eth_dev *eth_dev);
>   
> -void
> +int
>   bond_ethdev_close(struct rte_eth_dev *dev);
>   
>   #endif
> diff --git a/drivers/net/bonding/rte_eth_bond_pmd.c b/drivers/net/bonding/rte_eth_bond_pmd.c
> index 4ffd2ca4bb..dff835ef92 100644
> --- a/drivers/net/bonding/rte_eth_bond_pmd.c
> +++ b/drivers/net/bonding/rte_eth_bond_pmd.c
> @@ -2095,7 +2095,7 @@ bond_ethdev_stop(struct rte_eth_dev *eth_dev)
>   	}
>   }
>   
> -void
> +int
>   bond_ethdev_close(struct rte_eth_dev *dev)
>   {
>   	struct bond_dev_private *internals = dev->data->dev_private;
> @@ -2119,6 +2119,8 @@ bond_ethdev_close(struct rte_eth_dev *dev)
>   	bond_flow_ops.flush(dev, &ferror);
>   	bond_ethdev_free_queues(dev);
>   	rte_bitmap_reset(internals->vlan_filter_bmp);
> +
> +	return 0;
>   }
>   
>   /* forward declaration */
> diff --git a/drivers/net/cxgbe/cxgbe_ethdev.c b/drivers/net/cxgbe/cxgbe_ethdev.c
> index 422e190daf..e4bbba5c32 100644
> --- a/drivers/net/cxgbe/cxgbe_ethdev.c
> +++ b/drivers/net/cxgbe/cxgbe_ethdev.c
> @@ -318,7 +318,7 @@ int cxgbe_dev_mtu_set(struct rte_eth_dev *eth_dev, uint16_t mtu)
>   /*
>    * Stop device.
>    */
> -void cxgbe_dev_close(struct rte_eth_dev *eth_dev)
> +int cxgbe_dev_close(struct rte_eth_dev *eth_dev)
>   {
>   	struct port_info *temp_pi, *pi = eth_dev->data->dev_private;
>   	struct adapter *adapter = pi->adapter;
> @@ -327,10 +327,10 @@ void cxgbe_dev_close(struct rte_eth_dev *eth_dev)
>   	CXGBE_FUNC_TRACE();
>   
>   	if (!(adapter->flags & FULL_INIT_DONE))
> -		return;
> +		return 0;
>   
>   	if (!pi->viid)
> -		return;
> +		return 0;
>   
>   	cxgbe_down(pi);
>   	t4_sge_eth_release_queues(pi);
> @@ -343,11 +343,13 @@ void cxgbe_dev_close(struct rte_eth_dev *eth_dev)
>   	for_each_port(adapter, i) {
>   		temp_pi = adap2pinfo(adapter, i);
>   		if (temp_pi->viid)
> -			return;
> +			return 0;
>   	}
>   
>   	cxgbe_close(adapter);
>   	rte_free(adapter);
> +
> +	return 0;
>   }
>   
>   /* Start the device.
> diff --git a/drivers/net/cxgbe/cxgbe_pfvf.h b/drivers/net/cxgbe/cxgbe_pfvf.h
> index 0b7c52aecf..69d91639e9 100644
> --- a/drivers/net/cxgbe/cxgbe_pfvf.h
> +++ b/drivers/net/cxgbe/cxgbe_pfvf.h
> @@ -19,7 +19,7 @@
>   void cxgbe_dev_rx_queue_release(void *q);
>   void cxgbe_dev_tx_queue_release(void *q);
>   void cxgbe_dev_stop(struct rte_eth_dev *eth_dev);
> -void cxgbe_dev_close(struct rte_eth_dev *eth_dev);
> +int cxgbe_dev_close(struct rte_eth_dev *eth_dev);
>   int cxgbe_dev_info_get(struct rte_eth_dev *eth_dev,
>   		       struct rte_eth_dev_info *device_info);
>   int cxgbe_dev_promiscuous_enable(struct rte_eth_dev *eth_dev);
> diff --git a/drivers/net/dpaa/dpaa_ethdev.c b/drivers/net/dpaa/dpaa_ethdev.c
> index 8a6b19b5ce..7fe06e1830 100644
> --- a/drivers/net/dpaa/dpaa_ethdev.c
> +++ b/drivers/net/dpaa/dpaa_ethdev.c
> @@ -369,7 +369,7 @@ static void dpaa_eth_dev_stop(struct rte_eth_dev *dev)
>   	dev->tx_pkt_burst = dpaa_eth_tx_drop_all;
>   }
>   
> -static void dpaa_eth_dev_close(struct rte_eth_dev *dev)
> +static int dpaa_eth_dev_close(struct rte_eth_dev *dev)
>   {
>   	struct fman_if *fif = dev->process_private;
>   	struct __fman_if *__fif;
> @@ -392,6 +392,8 @@ static void dpaa_eth_dev_close(struct rte_eth_dev *dev)
>   					     dpaa_interrupt_handler,
>   					     (void *)dev);
>   	}
> +
> +	return 0;
>   }
>   
>   static int
> diff --git a/drivers/net/dpaa2/dpaa2_ethdev.c b/drivers/net/dpaa2/dpaa2_ethdev.c
> index 02daa4d250..fb4165fa03 100644
> --- a/drivers/net/dpaa2/dpaa2_ethdev.c
> +++ b/drivers/net/dpaa2/dpaa2_ethdev.c
> @@ -1236,7 +1236,7 @@ dpaa2_dev_stop(struct rte_eth_dev *dev)
>   	rte_eth_linkstatus_set(dev, &link);
>   }
>   
> -static void
> +static int
>   dpaa2_dev_close(struct rte_eth_dev *dev)
>   {
>   	struct dpaa2_dev_priv *priv = dev->data->dev_private;
> @@ -1252,11 +1252,13 @@ dpaa2_dev_close(struct rte_eth_dev *dev)
>   	ret = dpni_reset(dpni, CMD_PRI_LOW, priv->token);
>   	if (ret) {
>   		DPAA2_PMD_ERR("Failure cleaning dpni device: err=%d", ret);
> -		return;
> +		return -1;
>   	}
>   
>   	memset(&link, 0, sizeof(link));
>   	rte_eth_linkstatus_set(dev, &link);
> +
> +	return 0;
>   }
>   
>   static int
> diff --git a/drivers/net/e1000/em_ethdev.c b/drivers/net/e1000/em_ethdev.c
> index 1dc360713a..2d0e071448 100644
> --- a/drivers/net/e1000/em_ethdev.c
> +++ b/drivers/net/e1000/em_ethdev.c
> @@ -34,7 +34,7 @@
>   static int eth_em_configure(struct rte_eth_dev *dev);
>   static int eth_em_start(struct rte_eth_dev *dev);
>   static void eth_em_stop(struct rte_eth_dev *dev);
> -static void eth_em_close(struct rte_eth_dev *dev);
> +static int eth_em_close(struct rte_eth_dev *dev);
>   static int eth_em_promiscuous_enable(struct rte_eth_dev *dev);
>   static int eth_em_promiscuous_disable(struct rte_eth_dev *dev);
>   static int eth_em_allmulticast_enable(struct rte_eth_dev *dev);
> @@ -758,7 +758,7 @@ eth_em_stop(struct rte_eth_dev *dev)
>   	}
>   }
>   
> -static void
> +static int
>   eth_em_close(struct rte_eth_dev *dev)
>   {
>   	struct e1000_hw *hw = E1000_DEV_PRIVATE_TO_HW(dev->data->dev_private);
> @@ -782,6 +782,8 @@ eth_em_close(struct rte_eth_dev *dev)
>   	rte_intr_disable(intr_handle);
>   	rte_intr_callback_unregister(intr_handle,
>   				     eth_em_interrupt_handler, dev);
> +
> +	return 0;
>   }
>   
>   static int
> diff --git a/drivers/net/e1000/igb_ethdev.c b/drivers/net/e1000/igb_ethdev.c
> index 5ab74840aa..9e4aefe00c 100644
> --- a/drivers/net/e1000/igb_ethdev.c
> +++ b/drivers/net/e1000/igb_ethdev.c
> @@ -77,7 +77,7 @@ static int  eth_igb_start(struct rte_eth_dev *dev);
>   static void eth_igb_stop(struct rte_eth_dev *dev);
>   static int  eth_igb_dev_set_link_up(struct rte_eth_dev *dev);
>   static int  eth_igb_dev_set_link_down(struct rte_eth_dev *dev);
> -static void eth_igb_close(struct rte_eth_dev *dev);
> +static int eth_igb_close(struct rte_eth_dev *dev);
>   static int eth_igb_reset(struct rte_eth_dev *dev);
>   static int  eth_igb_promiscuous_enable(struct rte_eth_dev *dev);
>   static int  eth_igb_promiscuous_disable(struct rte_eth_dev *dev);
> @@ -155,7 +155,7 @@ static void igbvf_intr_disable(struct e1000_hw *hw);
>   static int igbvf_dev_configure(struct rte_eth_dev *dev);
>   static int igbvf_dev_start(struct rte_eth_dev *dev);
>   static void igbvf_dev_stop(struct rte_eth_dev *dev);
> -static void igbvf_dev_close(struct rte_eth_dev *dev);
> +static int igbvf_dev_close(struct rte_eth_dev *dev);
>   static int igbvf_promiscuous_enable(struct rte_eth_dev *dev);
>   static int igbvf_promiscuous_disable(struct rte_eth_dev *dev);
>   static int igbvf_allmulticast_enable(struct rte_eth_dev *dev);
> @@ -1535,7 +1535,7 @@ eth_igb_dev_set_link_down(struct rte_eth_dev *dev)
>   	return 0;
>   }
>   
> -static void
> +static int
>   eth_igb_close(struct rte_eth_dev *dev)
>   {
>   	struct e1000_hw *hw = E1000_DEV_PRIVATE_TO_HW(dev->data->dev_private);
> @@ -1604,6 +1604,8 @@ eth_igb_close(struct rte_eth_dev *dev)
>   
>   	/* clear all the filters list */
>   	igb_filterlist_flush(dev);
> +
> +	return 0;
>   }
>   
>   /*
> @@ -3381,7 +3383,7 @@ igbvf_dev_stop(struct rte_eth_dev *dev)
>   	adapter->stopped = true;
>   }
>   
> -static void
> +static int
>   igbvf_dev_close(struct rte_eth_dev *dev)
>   {
>   	struct e1000_hw *hw = E1000_DEV_PRIVATE_TO_HW(dev->data->dev_private);
> @@ -3412,6 +3414,8 @@ igbvf_dev_close(struct rte_eth_dev *dev)
>   	rte_intr_callback_unregister(&pci_dev->intr_handle,
>   				     eth_igbvf_interrupt_handler,
>   				     (void *)dev);
> +
> +	return 0;
>   }
>   
>   static int
> diff --git a/drivers/net/ena/ena_ethdev.c b/drivers/net/ena/ena_ethdev.c
> index 8077519735..ae6daad892 100644
> --- a/drivers/net/ena/ena_ethdev.c
> +++ b/drivers/net/ena/ena_ethdev.c
> @@ -187,7 +187,7 @@ static void ena_init_rings(struct ena_adapter *adapter,
>   static int ena_mtu_set(struct rte_eth_dev *dev, uint16_t mtu);
>   static int ena_start(struct rte_eth_dev *dev);
>   static void ena_stop(struct rte_eth_dev *dev);
> -static void ena_close(struct rte_eth_dev *dev);
> +static int ena_close(struct rte_eth_dev *dev);
>   static int ena_dev_reset(struct rte_eth_dev *dev);
>   static int ena_stats_get(struct rte_eth_dev *dev, struct rte_eth_stats *stats);
>   static void ena_rx_queue_release_all(struct rte_eth_dev *dev);
> @@ -487,7 +487,7 @@ static void ena_config_debug_area(struct ena_adapter *adapter)
>   	ena_com_delete_debug_area(&adapter->ena_dev);
>   }
>   
> -static void ena_close(struct rte_eth_dev *dev)
> +static int ena_close(struct rte_eth_dev *dev)
>   {
>   	struct rte_pci_device *pci_dev = RTE_ETH_DEV_TO_PCI(dev);
>   	struct rte_intr_handle *intr_handle = &pci_dev->intr_handle;
> @@ -513,6 +513,8 @@ static void ena_close(struct rte_eth_dev *dev)
>   	 * release of the resource in the rte_eth_dev_release_port().
>   	 */
>   	dev->data->mac_addrs = NULL;
> +
> +	return 0;
>   }
>   
>   static int
> diff --git a/drivers/net/enetc/enetc_ethdev.c b/drivers/net/enetc/enetc_ethdev.c
> index c6fb42740d..bdb32762ce 100644
> --- a/drivers/net/enetc/enetc_ethdev.c
> +++ b/drivers/net/enetc/enetc_ethdev.c
> @@ -545,7 +545,7 @@ enetc_stats_reset(struct rte_eth_dev *dev)
>   	return 0;
>   }
>   
> -static void
> +static int
>   enetc_dev_close(struct rte_eth_dev *dev)
>   {
>   	uint16_t i;
> @@ -564,6 +564,8 @@ enetc_dev_close(struct rte_eth_dev *dev)
>   		dev->data->tx_queues[i] = NULL;
>   	}
>   	dev->data->nb_tx_queues = 0;
> +
> +	return 0;
>   }
>   
>   static int
> diff --git a/drivers/net/enic/enic_ethdev.c b/drivers/net/enic/enic_ethdev.c
> index 57e82845a4..ed03d53608 100644
> --- a/drivers/net/enic/enic_ethdev.c
> +++ b/drivers/net/enic/enic_ethdev.c
> @@ -446,12 +446,14 @@ static void enicpmd_dev_stop(struct rte_eth_dev *eth_dev)
>   /*
>    * Stop device.
>    */
> -static void enicpmd_dev_close(struct rte_eth_dev *eth_dev)
> +static int enicpmd_dev_close(struct rte_eth_dev *eth_dev)
>   {
>   	struct enic *enic = pmd_priv(eth_dev);
>   
>   	ENICPMD_FUNC_TRACE();
>   	enic_remove(enic);
> +
> +	return 0;
>   }
>   
>   static int enicpmd_dev_link_update(struct rte_eth_dev *eth_dev,
> diff --git a/drivers/net/enic/enic_vf_representor.c b/drivers/net/enic/enic_vf_representor.c
> index 5d34e1b460..c528be2b98 100644
> --- a/drivers/net/enic/enic_vf_representor.c
> +++ b/drivers/net/enic/enic_vf_representor.c
> @@ -277,11 +277,12 @@ static void enic_vf_dev_stop(struct rte_eth_dev *eth_dev)
>    * "close" is no-op for now and solely exists so that rte_eth_dev_close()
>    * can finish its own cleanup without errors.
>    */
> -static void enic_vf_dev_close(struct rte_eth_dev *eth_dev __rte_unused)
> +static int enic_vf_dev_close(struct rte_eth_dev *eth_dev __rte_unused)
>   {
>   	ENICPMD_FUNC_TRACE();
>   	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
> -		return;
> +		return 0;
> +	return 0;
>   }
>   
>   static int
> diff --git a/drivers/net/failsafe/failsafe_ops.c b/drivers/net/failsafe/failsafe_ops.c
> index 96f7e456f9..93ebd09114 100644
> --- a/drivers/net/failsafe/failsafe_ops.c
> +++ b/drivers/net/failsafe/failsafe_ops.c
> @@ -240,7 +240,7 @@ fs_dev_set_link_down(struct rte_eth_dev *dev)
>   }
>   
>   static void fs_dev_free_queues(struct rte_eth_dev *dev);
> -static void
> +static int
>   fs_dev_close(struct rte_eth_dev *dev)
>   {
>   	struct sub_device *sdev;
> @@ -259,6 +259,7 @@ fs_dev_close(struct rte_eth_dev *dev)
>   	}
>   	fs_dev_free_queues(dev);
>   	fs_unlock(dev, 0);
> +	return 0;
>   }
>   
>   static int
> diff --git a/drivers/net/fm10k/fm10k_ethdev.c b/drivers/net/fm10k/fm10k_ethdev.c
> index 5771d83b55..533f976709 100644
> --- a/drivers/net/fm10k/fm10k_ethdev.c
> +++ b/drivers/net/fm10k/fm10k_ethdev.c
> @@ -2776,7 +2776,7 @@ fm10k_close_mbx_service(struct fm10k_hw *hw)
>   	hw->mbx.ops.disconnect(hw, &hw->mbx);
>   }
>   
> -static void
> +static int
>   fm10k_dev_close(struct rte_eth_dev *dev)
>   {
>   	struct fm10k_hw *hw = FM10K_DEV_PRIVATE_TO_HW(dev->data->dev_private);
> @@ -2821,6 +2821,8 @@ fm10k_dev_close(struct rte_eth_dev *dev)
>   		rte_intr_callback_unregister(intr_handle,
>   			fm10k_dev_interrupt_handler_vf, (void *)dev);
>   	}
> +
> +	return 0;
>   }
>   
>   static const struct eth_dev_ops fm10k_eth_dev_ops = {
> diff --git a/drivers/net/hinic/hinic_pmd_ethdev.c b/drivers/net/hinic/hinic_pmd_ethdev.c
> index 67e6afcf7a..5f2e97d3bd 100644
> --- a/drivers/net/hinic/hinic_pmd_ethdev.c
> +++ b/drivers/net/hinic/hinic_pmd_ethdev.c
> @@ -2936,7 +2936,7 @@ static void hinic_nic_dev_destroy(struct rte_eth_dev *eth_dev)
>    * @param dev
>    *   Pointer to Ethernet device structure.
>    */
> -static void hinic_dev_close(struct rte_eth_dev *dev)
> +static int hinic_dev_close(struct rte_eth_dev *dev)
>   {
>   	struct hinic_nic_dev *nic_dev = HINIC_ETH_DEV_TO_PRIVATE_NIC_DEV(dev);
>   
> @@ -2944,7 +2944,7 @@ static void hinic_dev_close(struct rte_eth_dev *dev)
>   					   &nic_dev->dev_status)) {
>   		PMD_DRV_LOG(WARNING, "Device %s already closed",
>   			    dev->data->name);
> -		return;
> +		return 0;
>   	}
>   
>   	/* stop device first */
> @@ -2971,6 +2971,8 @@ static void hinic_dev_close(struct rte_eth_dev *dev)
>   
>   	/* deinit nic hardware device */
>   	hinic_nic_dev_destroy(dev);
> +
> +	return 0;
>   }
>   
>   static const struct eth_dev_ops hinic_pmd_ops = {
> diff --git a/drivers/net/hns3/hns3_ethdev.c b/drivers/net/hns3/hns3_ethdev.c
> index 73d504253d..3b395a1ccf 100644
> --- a/drivers/net/hns3/hns3_ethdev.c
> +++ b/drivers/net/hns3/hns3_ethdev.c
> @@ -4859,7 +4859,7 @@ hns3_dev_stop(struct rte_eth_dev *dev)
>   	rte_spinlock_unlock(&hw->lock);
>   }
>   
> -static void
> +static int
>   hns3_dev_close(struct rte_eth_dev *eth_dev)
>   {
>   	struct hns3_adapter *hns = eth_dev->data->dev_private;
> @@ -4868,7 +4868,7 @@ hns3_dev_close(struct rte_eth_dev *eth_dev)
>   	if (rte_eal_process_type() != RTE_PROC_PRIMARY) {
>   		rte_free(eth_dev->process_private);
>   		eth_dev->process_private = NULL;
> -		return;
> +		return 0;
>   	}
>   
>   	if (hw->adapter_state == HNS3_NIC_STARTED)
> @@ -4888,6 +4888,8 @@ hns3_dev_close(struct rte_eth_dev *eth_dev)
>   	eth_dev->process_private = NULL;
>   	hns3_mp_uninit_primary();
>   	hns3_warn(hw, "Close port %d finished", hw->data->port_id);
> +
> +	return 0;
>   }
>   
>   static int
> diff --git a/drivers/net/hns3/hns3_ethdev_vf.c b/drivers/net/hns3/hns3_ethdev_vf.c
> index 037a5be7e0..e0ca5a6ac2 100644
> --- a/drivers/net/hns3/hns3_ethdev_vf.c
> +++ b/drivers/net/hns3/hns3_ethdev_vf.c
> @@ -1910,14 +1910,14 @@ hns3vf_dev_stop(struct rte_eth_dev *dev)
>   	rte_spinlock_unlock(&hw->lock);
>   }
>   
> -static void
> +static int
>   hns3vf_dev_close(struct rte_eth_dev *eth_dev)
>   {
>   	struct hns3_adapter *hns = eth_dev->data->dev_private;
>   	struct hns3_hw *hw = &hns->hw;
>   
>   	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
> -		return;
> +		return 0;
>   
>   	if (hw->adapter_state == HNS3_NIC_STARTED)
>   		hns3vf_dev_stop(eth_dev);
> @@ -1935,6 +1935,8 @@ hns3vf_dev_close(struct rte_eth_dev *eth_dev)
>   	eth_dev->process_private = NULL;
>   	hns3_mp_uninit_primary();
>   	hns3_warn(hw, "Close port %d finished", hw->data->port_id);
> +
> +	return 0;
>   }
>   
>   static int
> diff --git a/drivers/net/i40e/i40e_ethdev.c b/drivers/net/i40e/i40e_ethdev.c
> index 563f21d9df..804f0128ab 100644
> --- a/drivers/net/i40e/i40e_ethdev.c
> +++ b/drivers/net/i40e/i40e_ethdev.c
> @@ -225,7 +225,7 @@ static int eth_i40e_dev_uninit(struct rte_eth_dev *eth_dev);
>   static int i40e_dev_configure(struct rte_eth_dev *dev);
>   static int i40e_dev_start(struct rte_eth_dev *dev);
>   static void i40e_dev_stop(struct rte_eth_dev *dev);
> -static void i40e_dev_close(struct rte_eth_dev *dev);
> +static int i40e_dev_close(struct rte_eth_dev *dev);
>   static int  i40e_dev_reset(struct rte_eth_dev *dev);
>   static int i40e_dev_promiscuous_enable(struct rte_eth_dev *dev);
>   static int i40e_dev_promiscuous_disable(struct rte_eth_dev *dev);
> @@ -2608,7 +2608,7 @@ i40e_dev_stop(struct rte_eth_dev *dev)
>   	pf->adapter->rss_reta_updated = 0;
>   }
>   
> -static void
> +static int
>   i40e_dev_close(struct rte_eth_dev *dev)
>   {
>   	struct i40e_pf *pf = I40E_DEV_PRIVATE_TO_PF(dev->data->dev_private);
> @@ -2748,6 +2748,7 @@ i40e_dev_close(struct rte_eth_dev *dev)
>   	i40e_tm_conf_uninit(dev);
>   
>   	hw->adapter_closed = 1;
> +	return 0;
>   }
>   
>   /*
> diff --git a/drivers/net/i40e/i40e_ethdev_vf.c b/drivers/net/i40e/i40e_ethdev_vf.c
> index 8531cf6b12..61aad8c415 100644
> --- a/drivers/net/i40e/i40e_ethdev_vf.c
> +++ b/drivers/net/i40e/i40e_ethdev_vf.c
> @@ -90,7 +90,7 @@ static int i40evf_dev_xstats_reset(struct rte_eth_dev *dev);
>   static int i40evf_vlan_filter_set(struct rte_eth_dev *dev,
>   				  uint16_t vlan_id, int on);
>   static int i40evf_vlan_offload_set(struct rte_eth_dev *dev, int mask);
> -static void i40evf_dev_close(struct rte_eth_dev *dev);
> +static int i40evf_dev_close(struct rte_eth_dev *dev);
>   static int i40evf_dev_reset(struct rte_eth_dev *dev);
>   static int i40evf_check_vf_reset_done(struct rte_eth_dev *dev);
>   static int i40evf_dev_promiscuous_enable(struct rte_eth_dev *dev);
> @@ -2401,7 +2401,7 @@ i40evf_dev_stats_get(struct rte_eth_dev *dev, struct rte_eth_stats *stats)
>   	return ret;
>   }
>   
> -static void
> +static int
>   i40evf_dev_close(struct rte_eth_dev *dev)
>   {
>   	struct i40e_hw *hw = I40E_DEV_PRIVATE_TO_HW(dev->data->dev_private);
> @@ -2433,6 +2433,7 @@ i40evf_dev_close(struct rte_eth_dev *dev)
>   	vf->aq_resp = NULL;
>   
>   	hw->adapter_closed = 1;
> +	return 0;
>   }
>   
>   /*
> diff --git a/drivers/net/iavf/iavf_ethdev.c b/drivers/net/iavf/iavf_ethdev.c
> index 440da7d76a..8f46c4588f 100644
> --- a/drivers/net/iavf/iavf_ethdev.c
> +++ b/drivers/net/iavf/iavf_ethdev.c
> @@ -32,7 +32,7 @@
>   static int iavf_dev_configure(struct rte_eth_dev *dev);
>   static int iavf_dev_start(struct rte_eth_dev *dev);
>   static void iavf_dev_stop(struct rte_eth_dev *dev);
> -static void iavf_dev_close(struct rte_eth_dev *dev);
> +static int iavf_dev_close(struct rte_eth_dev *dev);
>   static int iavf_dev_reset(struct rte_eth_dev *dev);
>   static int iavf_dev_info_get(struct rte_eth_dev *dev,
>   			     struct rte_eth_dev_info *dev_info);
> @@ -1463,7 +1463,7 @@ iavf_dev_init(struct rte_eth_dev *eth_dev)
>   	return 0;
>   }
>   
> -static void
> +static int
>   iavf_dev_close(struct rte_eth_dev *dev)
>   {
>   	struct iavf_hw *hw = IAVF_DEV_PRIVATE_TO_HW(dev->data->dev_private);
> @@ -1506,6 +1506,8 @@ iavf_dev_close(struct rte_eth_dev *dev)
>   
>   	rte_free(vf->aq_resp);
>   	vf->aq_resp = NULL;
> +
> +	return 0;
>   }
>   
>   static int
> diff --git a/drivers/net/ice/ice_dcf_ethdev.c b/drivers/net/ice/ice_dcf_ethdev.c
> index 2faed3cc7a..5b626cb641 100644
> --- a/drivers/net/ice/ice_dcf_ethdev.c
> +++ b/drivers/net/ice/ice_dcf_ethdev.c
> @@ -847,13 +847,13 @@ ice_dcf_stats_reset(struct rte_eth_dev *dev)
>   	return 0;
>   }
>   
> -static void
> +static int
>   ice_dcf_dev_close(struct rte_eth_dev *dev)
>   {
>   	struct ice_dcf_adapter *adapter = dev->data->dev_private;
>   
>   	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
> -		return;
> +		return 0;
>   
>   	dev->dev_ops = NULL;
>   	dev->rx_pkt_burst = NULL;
> @@ -861,6 +861,8 @@ ice_dcf_dev_close(struct rte_eth_dev *dev)
>   
>   	ice_dcf_uninit_parent_adapter(dev);
>   	ice_dcf_uninit_hw(dev, &adapter->real_hw);
> +
> +	return 0;
>   }
>   
>   static int
> diff --git a/drivers/net/ice/ice_ethdev.c b/drivers/net/ice/ice_ethdev.c
> index cfd357b055..b4ac5e3f81 100644
> --- a/drivers/net/ice/ice_ethdev.c
> +++ b/drivers/net/ice/ice_ethdev.c
> @@ -77,7 +77,7 @@ static struct proto_xtr_ol_flag ice_proto_xtr_ol_flag_params[] = {
>   static int ice_dev_configure(struct rte_eth_dev *dev);
>   static int ice_dev_start(struct rte_eth_dev *dev);
>   static void ice_dev_stop(struct rte_eth_dev *dev);
> -static void ice_dev_close(struct rte_eth_dev *dev);
> +static int ice_dev_close(struct rte_eth_dev *dev);
>   static int ice_dev_reset(struct rte_eth_dev *dev);
>   static int ice_dev_info_get(struct rte_eth_dev *dev,
>   			    struct rte_eth_dev_info *dev_info);
> @@ -2430,7 +2430,7 @@ ice_dev_stop(struct rte_eth_dev *dev)
>   	pf->adapter_stopped = true;
>   }
>   
> -static void
> +static int
>   ice_dev_close(struct rte_eth_dev *dev)
>   {
>   	struct ice_pf *pf = ICE_DEV_PRIVATE_TO_PF(dev->data->dev_private);
> @@ -2478,6 +2478,8 @@ ice_dev_close(struct rte_eth_dev *dev)
>   	/* unregister callback func from eal lib */
>   	rte_intr_callback_unregister(intr_handle,
>   				     ice_interrupt_handler, dev);
> +
> +	return 0;
>   }
>   
>   static int
> diff --git a/drivers/net/igc/igc_ethdev.c b/drivers/net/igc/igc_ethdev.c
> index 810568bc54..11397c5c1d 100644
> --- a/drivers/net/igc/igc_ethdev.c
> +++ b/drivers/net/igc/igc_ethdev.c
> @@ -183,7 +183,7 @@ static void eth_igc_stop(struct rte_eth_dev *dev);
>   static int eth_igc_start(struct rte_eth_dev *dev);
>   static int eth_igc_set_link_up(struct rte_eth_dev *dev);
>   static int eth_igc_set_link_down(struct rte_eth_dev *dev);
> -static void eth_igc_close(struct rte_eth_dev *dev);
> +static int eth_igc_close(struct rte_eth_dev *dev);
>   static int eth_igc_reset(struct rte_eth_dev *dev);
>   static int eth_igc_promiscuous_enable(struct rte_eth_dev *dev);
>   static int eth_igc_promiscuous_disable(struct rte_eth_dev *dev);
> @@ -1165,7 +1165,7 @@ igc_dev_free_queues(struct rte_eth_dev *dev)
>   	dev->data->nb_tx_queues = 0;
>   }
>   
> -static void
> +static int
>   eth_igc_close(struct rte_eth_dev *dev)
>   {
>   	struct rte_pci_device *pci_dev = RTE_ETH_DEV_TO_PCI(dev);
> @@ -1199,6 +1199,8 @@ eth_igc_close(struct rte_eth_dev *dev)
>   
>   	/* Reset any pending lock */
>   	igc_reset_swfw_lock(hw);
> +
> +	return 0;
>   }
>   
>   static void
> diff --git a/drivers/net/ionic/ionic_ethdev.c b/drivers/net/ionic/ionic_ethdev.c
> index 800f6e5d64..1775fd29b7 100644
> --- a/drivers/net/ionic/ionic_ethdev.c
> +++ b/drivers/net/ionic/ionic_ethdev.c
> @@ -25,7 +25,7 @@ static int  ionic_dev_configure(struct rte_eth_dev *dev);
>   static int  ionic_dev_mtu_set(struct rte_eth_dev *dev, uint16_t mtu);
>   static int  ionic_dev_start(struct rte_eth_dev *dev);
>   static void ionic_dev_stop(struct rte_eth_dev *dev);
> -static void ionic_dev_close(struct rte_eth_dev *dev);
> +static int  ionic_dev_close(struct rte_eth_dev *dev);
>   static int  ionic_dev_set_link_up(struct rte_eth_dev *dev);
>   static int  ionic_dev_set_link_down(struct rte_eth_dev *dev);
>   static int  ionic_dev_link_update(struct rte_eth_dev *eth_dev,
> @@ -956,7 +956,7 @@ ionic_dev_stop(struct rte_eth_dev *eth_dev)
>   /*
>    * Reset and stop device.
>    */
> -static void
> +static int
>   ionic_dev_close(struct rte_eth_dev *eth_dev)
>   {
>   	struct ionic_lif *lif = IONIC_ETH_DEV_TO_LIF(eth_dev);
> @@ -967,14 +967,16 @@ ionic_dev_close(struct rte_eth_dev *eth_dev)
>   	err = ionic_lif_stop(lif);
>   	if (err) {
>   		IONIC_PRINT(ERR, "Cannot stop LIF: %d", err);
> -		return;
> +		return -1;
>   	}
>   
>   	err = eth_ionic_dev_uninit(eth_dev);
>   	if (err) {
>   		IONIC_PRINT(ERR, "Cannot destroy LIF: %d", err);
> -		return;
> +		return -1;
>   	}
> +
> +	return 0;
>   }
>   
>   static int
> diff --git a/drivers/net/ipn3ke/ipn3ke_representor.c b/drivers/net/ipn3ke/ipn3ke_representor.c
> index b673c49149..d49abbf758 100644
> --- a/drivers/net/ipn3ke/ipn3ke_representor.c
> +++ b/drivers/net/ipn3ke/ipn3ke_representor.c
> @@ -208,7 +208,7 @@ ipn3ke_rpst_dev_stop(struct rte_eth_dev *dev)
>   	}
>   }
>   
> -static void
> +static int
>   ipn3ke_rpst_dev_close(struct rte_eth_dev *dev)
>   {
>   	struct ipn3ke_hw *hw = IPN3KE_DEV_PRIVATE_TO_HW(dev);
> @@ -221,6 +221,8 @@ ipn3ke_rpst_dev_close(struct rte_eth_dev *dev)
>   		/* Disable the RX path */
>   		ipn3ke_xmac_rx_disable(hw, rpst->port_id, 0);
>   	}
> +
> +	return 0;
>   }
>   
>   /*
> diff --git a/drivers/net/ixgbe/ixgbe_ethdev.c b/drivers/net/ixgbe/ixgbe_ethdev.c
> index 0f065bbc04..a1369a0744 100644
> --- a/drivers/net/ixgbe/ixgbe_ethdev.c
> +++ b/drivers/net/ixgbe/ixgbe_ethdev.c
> @@ -147,7 +147,7 @@ static int  ixgbe_dev_start(struct rte_eth_dev *dev);
>   static void ixgbe_dev_stop(struct rte_eth_dev *dev);
>   static int  ixgbe_dev_set_link_up(struct rte_eth_dev *dev);
>   static int  ixgbe_dev_set_link_down(struct rte_eth_dev *dev);
> -static void ixgbe_dev_close(struct rte_eth_dev *dev);
> +static int  ixgbe_dev_close(struct rte_eth_dev *dev);
>   static int  ixgbe_dev_reset(struct rte_eth_dev *dev);
>   static int ixgbe_dev_promiscuous_enable(struct rte_eth_dev *dev);
>   static int ixgbe_dev_promiscuous_disable(struct rte_eth_dev *dev);
> @@ -251,7 +251,7 @@ static int  ixgbevf_dev_start(struct rte_eth_dev *dev);
>   static int ixgbevf_dev_link_update(struct rte_eth_dev *dev,
>   				   int wait_to_complete);
>   static void ixgbevf_dev_stop(struct rte_eth_dev *dev);
> -static void ixgbevf_dev_close(struct rte_eth_dev *dev);
> +static int ixgbevf_dev_close(struct rte_eth_dev *dev);
>   static int  ixgbevf_dev_reset(struct rte_eth_dev *dev);
>   static void ixgbevf_intr_disable(struct rte_eth_dev *dev);
>   static void ixgbevf_intr_enable(struct rte_eth_dev *dev);
> @@ -2994,7 +2994,7 @@ ixgbe_dev_set_link_down(struct rte_eth_dev *dev)
>   /*
>    * Reset and stop device.
>    */
> -static void
> +static int
>   ixgbe_dev_close(struct rte_eth_dev *dev)
>   {
>   	struct ixgbe_hw *hw =
> @@ -3065,6 +3065,7 @@ ixgbe_dev_close(struct rte_eth_dev *dev)
>   	rte_free(dev->security_ctx);
>   #endif
>   
> +	return 0;
>   }
>   
>   /*
> @@ -5443,7 +5444,7 @@ ixgbevf_dev_stop(struct rte_eth_dev *dev)
>   	adapter->rss_reta_updated = 0;
>   }
>   
> -static void
> +static int
>   ixgbevf_dev_close(struct rte_eth_dev *dev)
>   {
>   	struct ixgbe_hw *hw = IXGBE_DEV_PRIVATE_TO_HW(dev->data->dev_private);
> @@ -5472,6 +5473,8 @@ ixgbevf_dev_close(struct rte_eth_dev *dev)
>   	rte_intr_disable(intr_handle);
>   	rte_intr_callback_unregister(intr_handle,
>   				     ixgbevf_dev_interrupt_handler, dev);
> +
> +	return 0;
>   }
>   
>   /*
> diff --git a/drivers/net/kni/rte_eth_kni.c b/drivers/net/kni/rte_eth_kni.c
> index 831fe96c96..aa365a57de 100644
> --- a/drivers/net/kni/rte_eth_kni.c
> +++ b/drivers/net/kni/rte_eth_kni.c
> @@ -198,7 +198,7 @@ eth_kni_dev_stop(struct rte_eth_dev *dev)
>   	dev->data->dev_link.link_status = 0;
>   }
>   
> -static void
> +static int
>   eth_kni_close(struct rte_eth_dev *eth_dev)
>   {
>   	struct pmd_internals *internals;
> @@ -214,6 +214,8 @@ eth_kni_close(struct rte_eth_dev *eth_dev)
>   	if (ret)
>   		PMD_LOG(WARNING, "Not able to release kni for %s",
>   			eth_dev->data->name);
> +
> +	return 0;
>   }
>   
>   static int
> diff --git a/drivers/net/liquidio/lio_ethdev.c b/drivers/net/liquidio/lio_ethdev.c
> index 2c2b27e62a..93e2ed5670 100644
> --- a/drivers/net/liquidio/lio_ethdev.c
> +++ b/drivers/net/liquidio/lio_ethdev.c
> @@ -1550,7 +1550,7 @@ lio_dev_set_link_down(struct rte_eth_dev *eth_dev)
>    * @return
>    *    - nothing
>    */
> -static void
> +static int
>   lio_dev_close(struct rte_eth_dev *eth_dev)
>   {
>   	struct lio_device *lio_dev = LIO_DEV(eth_dev);
> @@ -1581,6 +1581,8 @@ lio_dev_close(struct rte_eth_dev *eth_dev)
>   
>   	 /* Delete all queues */
>   	lio_dev_clear_queues(eth_dev);
> +
> +	return 0;
>   }
>   
>   /**
> diff --git a/drivers/net/memif/rte_eth_memif.c b/drivers/net/memif/rte_eth_memif.c
> index a19c0f3e6b..1a2411c838 100644
> --- a/drivers/net/memif/rte_eth_memif.c
> +++ b/drivers/net/memif/rte_eth_memif.c
> @@ -1218,7 +1218,7 @@ memif_dev_start(struct rte_eth_dev *dev)
>   	return ret;
>   }
>   
> -static void
> +static int
>   memif_dev_close(struct rte_eth_dev *dev)
>   {
>   	struct pmd_internals *pmd = dev->data->dev_private;
> @@ -1239,6 +1239,8 @@ memif_dev_close(struct rte_eth_dev *dev)
>   	}
>   
>   	rte_free(dev->process_private);
> +
> +	return 0;
>   }
>   
>   static int
> diff --git a/drivers/net/mlx4/mlx4.c b/drivers/net/mlx4/mlx4.c
> index 71061a720f..df59314b66 100644
> --- a/drivers/net/mlx4/mlx4.c
> +++ b/drivers/net/mlx4/mlx4.c
> @@ -370,7 +370,7 @@ mlx4_dev_stop(struct rte_eth_dev *dev)
>    * @param dev
>    *   Pointer to Ethernet device structure.
>    */
> -static void
> +static int
>   mlx4_dev_close(struct rte_eth_dev *dev)
>   {
>   	struct mlx4_priv *priv = dev->data->dev_private;
> @@ -400,6 +400,7 @@ mlx4_dev_close(struct rte_eth_dev *dev)
>   		MLX4_ASSERT(priv->ctx == NULL);
>   	mlx4_intr_uninstall(priv);
>   	memset(priv, 0, sizeof(*priv));
> +	return 0;
>   }
>   
>   static const struct eth_dev_ops mlx4_dev_ops = {
> diff --git a/drivers/net/mlx5/mlx5.c b/drivers/net/mlx5/mlx5.c
> index 4a807fb4fd..01ead6e6af 100644
> --- a/drivers/net/mlx5/mlx5.c
> +++ b/drivers/net/mlx5/mlx5.c
> @@ -1341,7 +1341,7 @@ mlx5_proc_priv_uninit(struct rte_eth_dev *dev)
>    * @param dev
>    *   Pointer to Ethernet device structure.
>    */
> -void
> +int
>   mlx5_dev_close(struct rte_eth_dev *dev)
>   {
>   	struct mlx5_priv *priv = dev->data->dev_private;
> @@ -1351,14 +1351,14 @@ mlx5_dev_close(struct rte_eth_dev *dev)
>   	if (rte_eal_process_type() == RTE_PROC_SECONDARY) {
>   		/* Check if process_private released. */
>   		if (!dev->process_private)
> -			return;
> +			return 0;
>   		mlx5_tx_uar_uninit_secondary(dev);
>   		mlx5_proc_priv_uninit(dev);
>   		rte_eth_dev_release_port(dev);
> -		return;
> +		return 0;
>   	}
>   	if (!priv->sh)
> -		return;
> +		return 0;
>   	DRV_LOG(DEBUG, "port %u closing device \"%s\"",
>   		dev->data->port_id,
>   		((priv->sh->ctx != NULL) ?
> @@ -1479,6 +1479,7 @@ mlx5_dev_close(struct rte_eth_dev *dev)
>   	 * it is freed when dev_private is freed.
>   	 */
>   	dev->data->mac_addrs = NULL;
> +	return 0;
>   }
>   
>   /**
> diff --git a/drivers/net/mlx5/mlx5.h b/drivers/net/mlx5/mlx5.h
> index 865e72d318..56f0e25b4c 100644
> --- a/drivers/net/mlx5/mlx5.h
> +++ b/drivers/net/mlx5/mlx5.h
> @@ -842,7 +842,7 @@ int mlx5_proc_priv_init(struct rte_eth_dev *dev);
>   int mlx5_udp_tunnel_port_add(struct rte_eth_dev *dev,
>   			      struct rte_eth_udp_tunnel *udp_tunnel);
>   uint16_t mlx5_eth_find_next(uint16_t port_id, struct rte_pci_device *pci_dev);
> -void mlx5_dev_close(struct rte_eth_dev *dev);
> +int mlx5_dev_close(struct rte_eth_dev *dev);
>   
>   /* Macro to iterate over all valid ports for mlx5 driver. */
>   #define MLX5_ETH_FOREACH_DEV(port_id, pci_dev) \
> diff --git a/drivers/net/mvneta/mvneta_ethdev.c b/drivers/net/mvneta/mvneta_ethdev.c
> index 5bf3ebc4bd..eebcdb840e 100644
> --- a/drivers/net/mvneta/mvneta_ethdev.c
> +++ b/drivers/net/mvneta/mvneta_ethdev.c
> @@ -429,7 +429,7 @@ mvneta_dev_stop(struct rte_eth_dev *dev)
>    * @param dev
>    *   Pointer to Ethernet device structure.
>    */
> -static void
> +static int
>   mvneta_dev_close(struct rte_eth_dev *dev)
>   {
>   	struct mvneta_priv *priv = dev->data->dev_private;
> @@ -455,6 +455,8 @@ mvneta_dev_close(struct rte_eth_dev *dev)
>   		mvneta_neta_deinit();
>   		rte_mvep_deinit(MVEP_MOD_T_NETA);
>   	}
> +
> +	return 0;
>   }
>   
>   /**
> diff --git a/drivers/net/mvpp2/mrvl_ethdev.c b/drivers/net/mvpp2/mrvl_ethdev.c
> index f5e77d07bf..27f7cfb056 100644
> --- a/drivers/net/mvpp2/mrvl_ethdev.c
> +++ b/drivers/net/mvpp2/mrvl_ethdev.c
> @@ -855,7 +855,7 @@ mrvl_dev_stop(struct rte_eth_dev *dev)
>    * @param dev
>    *   Pointer to Ethernet device structure.
>    */
> -static void
> +static int
>   mrvl_dev_close(struct rte_eth_dev *dev)
>   {
>   	struct mrvl_priv *priv = dev->data->dev_private;
> @@ -915,6 +915,8 @@ mrvl_dev_close(struct rte_eth_dev *dev)
>   		mrvl_deinit_pp2();
>   		rte_mvep_deinit(MVEP_MOD_T_PP2);
>   	}
> +
> +	return 0;
>   }
>   
>   /**
> diff --git a/drivers/net/netvsc/hn_ethdev.c b/drivers/net/netvsc/hn_ethdev.c
> index 229c1b9149..8968036ea3 100644
> --- a/drivers/net/netvsc/hn_ethdev.c
> +++ b/drivers/net/netvsc/hn_ethdev.c
> @@ -841,13 +841,15 @@ hn_dev_stop(struct rte_eth_dev *dev)
>   	hn_vf_stop(dev);
>   }
>   
> -static void
> +static int
>   hn_dev_close(struct rte_eth_dev *dev)
>   {
>   	PMD_INIT_FUNC_TRACE();
>   
>   	hn_vf_close(dev);
>   	hn_dev_free_queues(dev);
> +
> +	return 0;
>   }
>   
>   static const struct eth_dev_ops hn_eth_dev_ops = {
> diff --git a/drivers/net/nfb/nfb_ethdev.c b/drivers/net/nfb/nfb_ethdev.c
> index b039ab6fc2..c3c3d003f1 100644
> --- a/drivers/net/nfb/nfb_ethdev.c
> +++ b/drivers/net/nfb/nfb_ethdev.c
> @@ -209,7 +209,7 @@ nfb_eth_dev_info(struct rte_eth_dev *dev,
>    * @param dev
>    *   Pointer to Ethernet device structure.
>    */
> -static void
> +static int
>   nfb_eth_dev_close(struct rte_eth_dev *dev)
>   {
>   	struct pmd_internals *internals = dev->data->dev_private;
> @@ -235,6 +235,8 @@ nfb_eth_dev_close(struct rte_eth_dev *dev)
>   
>   	rte_free(dev->data->mac_addrs);
>   	dev->data->mac_addrs = NULL;
> +
> +	return 0;
>   }
>   
>   /**
> diff --git a/drivers/net/nfp/nfp_net.c b/drivers/net/nfp/nfp_net.c
> index f5e4cd339d..9509dc8bd6 100644
> --- a/drivers/net/nfp/nfp_net.c
> +++ b/drivers/net/nfp/nfp_net.c
> @@ -50,7 +50,7 @@
>   #include <errno.h>
>   
>   /* Prototypes */
> -static void nfp_net_close(struct rte_eth_dev *dev);
> +static int nfp_net_close(struct rte_eth_dev *dev);
>   static int nfp_net_configure(struct rte_eth_dev *dev);
>   static void nfp_net_dev_interrupt_handler(void *param);
>   static void nfp_net_dev_interrupt_delayed_handler(void *param);
> @@ -864,7 +864,7 @@ nfp_net_set_link_down(struct rte_eth_dev *dev)
>   }
>   
>   /* Reset and stop device. The device can not be restarted. */
> -static void
> +static int
>   nfp_net_close(struct rte_eth_dev *dev)
>   {
>   	struct nfp_net_hw *hw;
> @@ -906,6 +906,8 @@ nfp_net_close(struct rte_eth_dev *dev)
>   	 * The ixgbe PMD driver disables the pcie master on the
>   	 * device. The i40e does not...
>   	 */
> +
> +	return 0;
>   }
>   
>   static int
> diff --git a/drivers/net/octeontx/octeontx_ethdev.c b/drivers/net/octeontx/octeontx_ethdev.c
> index ff2e22cd44..10eeeac642 100644
> --- a/drivers/net/octeontx/octeontx_ethdev.c
> +++ b/drivers/net/octeontx/octeontx_ethdev.c
> @@ -478,7 +478,7 @@ octeontx_dev_configure(struct rte_eth_dev *dev)
>   	return 0;
>   }
>   
> -static void
> +static int
>   octeontx_dev_close(struct rte_eth_dev *dev)
>   {
>   	struct octeontx_txq *txq = NULL;
> @@ -517,6 +517,8 @@ octeontx_dev_close(struct rte_eth_dev *dev)
>   
>   	dev->tx_pkt_burst = NULL;
>   	dev->rx_pkt_burst = NULL;
> +
> +	return 0;
>   }
>   
>   static int
> diff --git a/drivers/net/octeontx2/otx2_ethdev.c b/drivers/net/octeontx2/otx2_ethdev.c
> index 51a6f99877..abe5f03628 100644
> --- a/drivers/net/octeontx2/otx2_ethdev.c
> +++ b/drivers/net/octeontx2/otx2_ethdev.c
> @@ -2222,7 +2222,7 @@ otx2_nix_dev_start(struct rte_eth_dev *eth_dev)
>   }
>   
>   static int otx2_nix_dev_reset(struct rte_eth_dev *eth_dev);
> -static void otx2_nix_dev_close(struct rte_eth_dev *eth_dev);
> +static int otx2_nix_dev_close(struct rte_eth_dev *eth_dev);
>   
>   /* Initialize and register driver with DPDK Application */
>   static const struct eth_dev_ops otx2_eth_dev_ops = {
> @@ -2635,10 +2635,11 @@ otx2_eth_dev_uninit(struct rte_eth_dev *eth_dev, bool mbox_close)
>   	return 0;
>   }
>   
> -static void
> +static int
>   otx2_nix_dev_close(struct rte_eth_dev *eth_dev)
>   {
>   	otx2_eth_dev_uninit(eth_dev, true);
> +	return 0;
>   }
>   
>   static int
> diff --git a/drivers/net/pcap/rte_eth_pcap.c b/drivers/net/pcap/rte_eth_pcap.c
> index 668cbd1fc7..76e704a65a 100644
> --- a/drivers/net/pcap/rte_eth_pcap.c
> +++ b/drivers/net/pcap/rte_eth_pcap.c
> @@ -728,7 +728,7 @@ eth_stats_reset(struct rte_eth_dev *dev)
>   	return 0;
>   }
>   
> -static void
> +static int
>   eth_dev_close(struct rte_eth_dev *dev)
>   {
>   	unsigned int i;
> @@ -748,6 +748,7 @@ eth_dev_close(struct rte_eth_dev *dev)
>   		}
>   	}
>   
> +	return 0;
>   }
>   
>   static void
> diff --git a/drivers/net/pfe/pfe_ethdev.c b/drivers/net/pfe/pfe_ethdev.c
> index 97356d2b0b..187a0019ff 100644
> --- a/drivers/net/pfe/pfe_ethdev.c
> +++ b/drivers/net/pfe/pfe_ethdev.c
> @@ -398,14 +398,14 @@ pfe_eth_exit(struct rte_eth_dev *dev, struct pfe *pfe)
>   	pfe->nb_devs--;
>   }
>   
> -static void
> +static int
>   pfe_eth_close(struct rte_eth_dev *dev)
>   {
>   	if (!dev)
> -		return;
> +		return -1;
>   
>   	if (!g_pfe)
> -		return;
> +		return -1;
>   
>   	pfe_eth_exit(dev, g_pfe);
>   
> @@ -415,6 +415,8 @@ pfe_eth_close(struct rte_eth_dev *dev)
>   		rte_free(g_pfe);
>   		g_pfe = NULL;
>   	}
> +
> +	return 0;
>   }
>   
>   static int
> diff --git a/drivers/net/qede/qede_ethdev.c b/drivers/net/qede/qede_ethdev.c
> index 59f1746ee9..9535df183f 100644
> --- a/drivers/net/qede/qede_ethdev.c
> +++ b/drivers/net/qede/qede_ethdev.c
> @@ -1542,7 +1542,7 @@ static void qede_poll_sp_sb_cb(void *param)
>   	}
>   }
>   
> -static void qede_dev_close(struct rte_eth_dev *eth_dev)
> +static int qede_dev_close(struct rte_eth_dev *eth_dev)
>   {
>   	struct rte_pci_device *pci_dev = RTE_ETH_DEV_TO_PCI(eth_dev);
>   	struct qede_dev *qdev = QEDE_INIT_QDEV(eth_dev);
> @@ -1586,6 +1586,8 @@ static void qede_dev_close(struct rte_eth_dev *eth_dev)
>   
>   	if (ECORE_IS_CMT(edev))
>   		rte_eal_alarm_cancel(qede_poll_sp_sb_cb, (void *)eth_dev);
> +
> +	return 0;
>   }
>   
>   static int
> diff --git a/drivers/net/sfc/sfc_ethdev.c b/drivers/net/sfc/sfc_ethdev.c
> index 1a58e0df84..eaa48627d6 100644
> --- a/drivers/net/sfc/sfc_ethdev.c
> +++ b/drivers/net/sfc/sfc_ethdev.c
> @@ -318,7 +318,7 @@ sfc_dev_set_link_down(struct rte_eth_dev *dev)
>   	return 0;
>   }
>   
> -static void
> +static int
>   sfc_dev_close(struct rte_eth_dev *dev)
>   {
>   	struct sfc_adapter *sa = sfc_adapter_by_eth_dev(dev);
> @@ -364,6 +364,8 @@ sfc_dev_close(struct rte_eth_dev *dev)
>   
>   	dev->process_private = NULL;
>   	free(sa);
> +
> +	return 0;
>   }
>   
>   static int
> diff --git a/drivers/net/softnic/rte_eth_softnic.c b/drivers/net/softnic/rte_eth_softnic.c
> index e453fd1486..038abd98da 100644
> --- a/drivers/net/softnic/rte_eth_softnic.c
> +++ b/drivers/net/softnic/rte_eth_softnic.c
> @@ -201,10 +201,10 @@ pmd_dev_stop(struct rte_eth_dev *dev)
>   	softnic_mtr_free(p);
>   }
>   
> -static void
> +static int
>   pmd_dev_close(struct rte_eth_dev *dev __rte_unused)
>   {
> -	return;
> +	return 0;
>   }
>   
>   static int
> diff --git a/drivers/net/szedata2/rte_eth_szedata2.c b/drivers/net/szedata2/rte_eth_szedata2.c
> index a17c53577c..daa40cfd6c 100644
> --- a/drivers/net/szedata2/rte_eth_szedata2.c
> +++ b/drivers/net/szedata2/rte_eth_szedata2.c
> @@ -1155,7 +1155,7 @@ eth_tx_queue_release(void *q)
>   	}
>   }
>   
> -static void
> +static int
>   eth_dev_close(struct rte_eth_dev *dev)
>   {
>   	struct pmd_internals *internals = dev->data->dev_private;
> @@ -1180,6 +1180,8 @@ eth_dev_close(struct rte_eth_dev *dev)
>   
>   	rte_free(dev->data->mac_addrs);
>   	dev->data->mac_addrs = NULL;
> +
> +	return 0;
>   }
>   
>   static int
> diff --git a/drivers/net/tap/rte_eth_tap.c b/drivers/net/tap/rte_eth_tap.c
> index df16aa4ea7..2d1e4e365a 100644
> --- a/drivers/net/tap/rte_eth_tap.c
> +++ b/drivers/net/tap/rte_eth_tap.c
> @@ -1067,7 +1067,7 @@ tap_stats_reset(struct rte_eth_dev *dev)
>   	return 0;
>   }
>   
> -static void
> +static int
>   tap_dev_close(struct rte_eth_dev *dev)
>   {
>   	int i;
> @@ -1116,6 +1116,8 @@ tap_dev_close(struct rte_eth_dev *dev)
>   	 * Since TUN device has no more opened file descriptors
>   	 * it will be removed from kernel
>   	 */
> +
> +	return 0;
>   }
>   
>   static void
> diff --git a/drivers/net/thunderx/nicvf_ethdev.c b/drivers/net/thunderx/nicvf_ethdev.c
> index 83d9488360..3d7348771a 100644
> --- a/drivers/net/thunderx/nicvf_ethdev.c
> +++ b/drivers/net/thunderx/nicvf_ethdev.c
> @@ -1852,7 +1852,7 @@ nicvf_vf_stop(struct rte_eth_dev *dev, struct nicvf *nic, bool cleanup)
>   	}
>   }
>   
> -static void
> +static int
>   nicvf_dev_close(struct rte_eth_dev *dev)
>   {
>   	size_t i;
> @@ -1869,6 +1869,8 @@ nicvf_dev_close(struct rte_eth_dev *dev)
>   
>   		nicvf_periodic_alarm_stop(nicvf_vf_interrupt, nic->snicvf[i]);
>   	}
> +
> +	return 0;
>   }
>   
>   static int
> diff --git a/drivers/net/vhost/rte_eth_vhost.c b/drivers/net/vhost/rte_eth_vhost.c
> index ce32be9ce3..f67f4db812 100644
> --- a/drivers/net/vhost/rte_eth_vhost.c
> +++ b/drivers/net/vhost/rte_eth_vhost.c
> @@ -1164,7 +1164,7 @@ eth_dev_stop(struct rte_eth_dev *dev)
>   	update_queuing_status(dev);
>   }
>   
> -static void
> +static int
>   eth_dev_close(struct rte_eth_dev *dev)
>   {
>   	struct pmd_internal *internal;
> @@ -1173,7 +1173,7 @@ eth_dev_close(struct rte_eth_dev *dev)
>   
>   	internal = dev->data->dev_private;
>   	if (!internal)
> -		return;
> +		return 0;
>   
>   	eth_dev_stop(dev);
>   
> @@ -1201,6 +1201,8 @@ eth_dev_close(struct rte_eth_dev *dev)
>   
>   	rte_free(vring_states[dev->data->port_id]);
>   	vring_states[dev->data->port_id] = NULL;
> +
> +	return 0;
>   }
>   
>   static int
> diff --git a/drivers/net/virtio/virtio_ethdev.c b/drivers/net/virtio/virtio_ethdev.c
> index 013a2904e6..0787337a01 100644
> --- a/drivers/net/virtio/virtio_ethdev.c
> +++ b/drivers/net/virtio/virtio_ethdev.c
> @@ -704,7 +704,7 @@ virtio_alloc_queues(struct rte_eth_dev *dev)
>   
>   static void virtio_queues_unbind_intr(struct rte_eth_dev *dev);
>   
> -static void
> +static int
>   virtio_dev_close(struct rte_eth_dev *dev)
>   {
>   	struct virtio_hw *hw = dev->data->dev_private;
> @@ -713,7 +713,7 @@ virtio_dev_close(struct rte_eth_dev *dev)
>   	PMD_INIT_LOG(DEBUG, "virtio_dev_close");
>   
>   	if (!hw->opened)
> -		return;
> +		return 0;
>   	hw->opened = false;
>   
>   	/* reset the NIC */
> @@ -743,6 +743,8 @@ virtio_dev_close(struct rte_eth_dev *dev)
>   		if (!hw->modern)
>   			rte_pci_ioport_unmap(VTPCI_IO(hw));
>   	}
> +
> +	return 0;
>   }
>   
>   static int
> diff --git a/drivers/net/vmxnet3/vmxnet3_ethdev.c b/drivers/net/vmxnet3/vmxnet3_ethdev.c
> index fa80e75a5b..17fed0ed6e 100644
> --- a/drivers/net/vmxnet3/vmxnet3_ethdev.c
> +++ b/drivers/net/vmxnet3/vmxnet3_ethdev.c
> @@ -64,7 +64,7 @@ static int eth_vmxnet3_dev_uninit(struct rte_eth_dev *eth_dev);
>   static int vmxnet3_dev_configure(struct rte_eth_dev *dev);
>   static int vmxnet3_dev_start(struct rte_eth_dev *dev);
>   static void vmxnet3_dev_stop(struct rte_eth_dev *dev);
> -static void vmxnet3_dev_close(struct rte_eth_dev *dev);
> +static int vmxnet3_dev_close(struct rte_eth_dev *dev);
>   static void vmxnet3_dev_set_rxmode(struct vmxnet3_hw *hw, uint32_t feature, int set);
>   static int vmxnet3_dev_promiscuous_enable(struct rte_eth_dev *dev);
>   static int vmxnet3_dev_promiscuous_disable(struct rte_eth_dev *dev);
> @@ -888,13 +888,15 @@ vmxnet3_free_queues(struct rte_eth_dev *dev)
>   /*
>    * Reset and stop device.
>    */
> -static void
> +static int
>   vmxnet3_dev_close(struct rte_eth_dev *dev)
>   {
>   	PMD_INIT_FUNC_TRACE();
>   
>   	vmxnet3_dev_stop(dev);
>   	vmxnet3_free_queues(dev);
> +
> +	return 0;
>   }
>   
>   static void
> diff --git a/lib/librte_ethdev/rte_ethdev_driver.h b/lib/librte_ethdev/rte_ethdev_driver.h
> index 23cc1e0959..6abfd58d2f 100644
> --- a/lib/librte_ethdev/rte_ethdev_driver.h
> +++ b/lib/librte_ethdev/rte_ethdev_driver.h
> @@ -42,7 +42,7 @@ typedef int  (*eth_dev_set_link_up_t)(struct rte_eth_dev *dev);
>   typedef int  (*eth_dev_set_link_down_t)(struct rte_eth_dev *dev);
>   /**< @internal Function used to link down a configured Ethernet device. */
>   
> -typedef void (*eth_dev_close_t)(struct rte_eth_dev *dev);
> +typedef int (*eth_dev_close_t)(struct rte_eth_dev *dev);
>   /**< @internal Function used to close a configured Ethernet device. */
>   
>   typedef int (*eth_dev_reset_t)(struct rte_eth_dev *dev);

Re: [dpdk-dev] [EXT] [PATCH v2 02/25] ethdev: allow drivers to return error on close

[Liron Himi]

For mvpp2, mvneta.

Reviewed-by: Liron Himi<lironh@marvell.com>

-----Original Message-----
From: Thomas Monjalon <thomas@monjalon.net> 
Sent: Monday, 28 September 2020 02:42
To: dev@dpdk.org
Cc: ferruh.yigit@intel.com; arybchenko@solarflare.com; John W. Linville <linville@tuxdriver.com>; Ciara Loftus <ciara.loftus@intel.com>; Qi Zhang <qi.z.zhang@intel.com>; Shepard Siegel <shepard.siegel@atomicrules.com>; Ed Czeck <ed.czeck@atomicrules.com>; John Miller <john.miller@atomicrules.com>; Igor Russkikh <igor.russkikh@aquantia.com>; Pavel Belous <pavel.belous@aquantia.com>; Steven Webster <steven.webster@windriver.com>; Matt Peters <matt.peters@windriver.com>; Somalapuram Amaranath <asomalap@amd.com>; Rasesh Mody <rmody@marvell.com>; Shahed Shaikh <shshaikh@marvell.com>; Ajit Khaparde <ajit.khaparde@broadcom.com>; Somnath Kotur <somnath.kotur@broadcom.com>; Chas Williams <chas3@att.com>; Wei Hu (Xavier) <xavier.huwei@huawei.com>; Rahul Lakkireddy <rahul.lakkireddy@chelsio.com>; Hemant Agrawal <hemant.agrawal@nxp.com>; Sachin Saxena <sachin.saxena@oss.nxp.com>; Jeff Guo <jia.guo@intel.com>; Haiyue Wang <haiyue.wang@intel.com>; Marcin Wojtas <mw@semihalf.com>; Michal Krawczyk <mk@semihalf.com>; Guy Tzalik <gtzalik@amazon.com>; Evgeny Schemeilin <evgenys@amazon.com>; Igor Chauskin <igorch@amazon.com>; Gagandeep Singh <g.singh@nxp.com>; John Daley <johndale@cisco.com>; Hyong Youb Kim <hyonkim@cisco.com>; Gaetan Rivet <grive@u256.net>; Xiao Wang <xiao.w.wang@intel.com>; Ziyang Xuan <xuanziyang2@huawei.com>; Xiaoyun Wang <cloud.wangxiaoyun@huawei.com>; Guoyang Zhou <zhouguoyang@huawei.com>; Min Hu (Connor) <humin29@huawei.com>; Yisen Zhuang <yisen.zhuang@huawei.com>; Beilei Xing <beilei.xing@intel.com>; Jingjing Wu <jingjing.wu@intel.com>; Qiming Yang <qiming.yang@intel.com>; Alfredo Cardigliano <cardigliano@ntop.org>; Rosen Xu <rosen.xu@intel.com>; Shijith Thotton <sthotton@marvell.com>; Srisivasubramanian Srinivasan <srinivasan@marvell.com>; Jakub Grajciar <jgrajcia@cisco.com>; Matan Azrad <matan@nvidia.com>; Shahaf Shuler <shahafs@nvidia.com>; Viacheslav Ovsiienko <viacheslavo@nvidia.com>; Zyta Szpak <zr@semihalf.com>; Liron Himi <lironh@marvell.com>; Stephen Hemminger <sthemmin@microsoft.com>; K. Y. Srinivasan <kys@microsoft.com>; Haiyang Zhang <haiyangz@microsoft.com>; Long Li <longli@microsoft.com>; Martin Spinler <spinler@cesnet.cz>; Heinrich Kuhn <heinrich.kuhn@netronome.com>; Harman Kalra <hkalra@marvell.com>; Jerin Jacob Kollanukkaran <jerinj@marvell.com>; Nithin Kumar Dabilpuram <ndabilpuram@marvell.com>; Kiran Kumar Kokkilagadda <kirankumark@marvell.com>; Akhil Goyal <akhil.goyal@nxp.com>; Jasvinder Singh <jasvinder.singh@intel.com>; Cristian Dumitrescu <cristian.dumitrescu@intel.com>; Keith Wiles <keith.wiles@intel.com>; Maciej Czekaj [C] <mczekaj@marvell.com>; Maxime Coquelin <maxime.coquelin@redhat.com>; Chenbo Xia <chenbo.xia@intel.com>; Zhihong Wang <zhihong.wang@intel.com>; Yong Wang <yongwang@vmware.com>
Subject: [EXT] [PATCH v2 02/25] ethdev: allow drivers to return error on close

External Email

----------------------------------------------------------------------
The device operation .dev_close was returning void.
This driver interface is changed to return an int.

Note that the API rte_eth_dev_close() is still returning void,
although a deprecation notice is pending to change it as well.

Signed-off-by: Thomas Monjalon <thomas@monjalon.net>
---
 app/test/virtual_pmd.c                    |  6 ++++--
 drivers/net/af_packet/rte_eth_af_packet.c |  3 ++-
 drivers/net/af_xdp/rte_eth_af_xdp.c       |  4 +++-
 drivers/net/ark/ark_ethdev.c              |  6 ++++--
 drivers/net/atlantic/atl_ethdev.c         |  6 ++++--
 drivers/net/avp/avp_ethdev.c              |  5 +++--
 drivers/net/axgbe/axgbe_ethdev.c          |  5 +++--
 drivers/net/bnx2x/bnx2x_ethdev.c          |  4 +++-
 drivers/net/bnxt/bnxt_ethdev.c            |  4 +++-
 drivers/net/bnxt/bnxt_reps.c              |  3 ++-
 drivers/net/bnxt/bnxt_reps.h              |  2 +-
 drivers/net/bonding/eth_bond_private.h    |  2 +-
 drivers/net/bonding/rte_eth_bond_pmd.c    |  4 +++-
 drivers/net/cxgbe/cxgbe_ethdev.c          | 10 ++++++----
 drivers/net/cxgbe/cxgbe_pfvf.h            |  2 +-
 drivers/net/dpaa/dpaa_ethdev.c            |  4 +++-
 drivers/net/dpaa2/dpaa2_ethdev.c          |  6 ++++--
 drivers/net/e1000/em_ethdev.c             |  6 ++++--
 drivers/net/e1000/igb_ethdev.c            | 12 ++++++++----
 drivers/net/ena/ena_ethdev.c              |  6 ++++--
 drivers/net/enetc/enetc_ethdev.c          |  4 +++-
 drivers/net/enic/enic_ethdev.c            |  4 +++-
 drivers/net/enic/enic_vf_representor.c    |  5 +++--
 drivers/net/failsafe/failsafe_ops.c       |  3 ++-
 drivers/net/fm10k/fm10k_ethdev.c          |  4 +++-
 drivers/net/hinic/hinic_pmd_ethdev.c      |  6 ++++--
 drivers/net/hns3/hns3_ethdev.c            |  6 ++++--
 drivers/net/hns3/hns3_ethdev_vf.c         |  6 ++++--
 drivers/net/i40e/i40e_ethdev.c            |  5 +++--
 drivers/net/i40e/i40e_ethdev_vf.c         |  5 +++--
 drivers/net/iavf/iavf_ethdev.c            |  6 ++++--
 drivers/net/ice/ice_dcf_ethdev.c          |  6 ++++--
 drivers/net/ice/ice_ethdev.c              |  6 ++++--
 drivers/net/igc/igc_ethdev.c              |  6 ++++--
 drivers/net/ionic/ionic_ethdev.c          | 10 ++++++----
 drivers/net/ipn3ke/ipn3ke_representor.c   |  4 +++-
 drivers/net/ixgbe/ixgbe_ethdev.c          | 11 +++++++----
 drivers/net/kni/rte_eth_kni.c             |  4 +++-
 drivers/net/liquidio/lio_ethdev.c         |  4 +++-
 drivers/net/memif/rte_eth_memif.c         |  4 +++-
 drivers/net/mlx4/mlx4.c                   |  3 ++-
 drivers/net/mlx5/mlx5.c                   |  9 +++++----
 drivers/net/mlx5/mlx5.h                   |  2 +-
 drivers/net/mvneta/mvneta_ethdev.c        |  4 +++-
 drivers/net/mvpp2/mrvl_ethdev.c           |  4 +++-
 drivers/net/netvsc/hn_ethdev.c            |  4 +++-
 drivers/net/nfb/nfb_ethdev.c              |  4 +++-
 drivers/net/nfp/nfp_net.c                 |  6 ++++--
 drivers/net/octeontx/octeontx_ethdev.c    |  4 +++-
 drivers/net/octeontx2/otx2_ethdev.c       |  5 +++--
 drivers/net/pcap/rte_eth_pcap.c           |  3 ++-
 drivers/net/pfe/pfe_ethdev.c              |  8 +++++---
 drivers/net/qede/qede_ethdev.c            |  4 +++-
 drivers/net/sfc/sfc_ethdev.c              |  4 +++-
 drivers/net/softnic/rte_eth_softnic.c     |  4 ++--
 drivers/net/szedata2/rte_eth_szedata2.c   |  4 +++-
 drivers/net/tap/rte_eth_tap.c             |  4 +++-
 drivers/net/thunderx/nicvf_ethdev.c       |  4 +++-
 drivers/net/vhost/rte_eth_vhost.c         |  6 ++++--
 drivers/net/virtio/virtio_ethdev.c        |  6 ++++--
 drivers/net/vmxnet3/vmxnet3_ethdev.c      |  6 ++++--
 lib/librte_ethdev/rte_ethdev_driver.h     |  2 +-
 62 files changed, 206 insertions(+), 103 deletions(-)

diff --git a/app/test/virtual_pmd.c b/app/test/virtual_pmd.c
index b49089a33c..4bd4d1c9ad 100644
--- a/app/test/virtual_pmd.c
+++ b/app/test/virtual_pmd.c
@@ -62,9 +62,11 @@ static void  virtual_ethdev_stop(struct rte_eth_dev *eth_dev __rte_unused)
 		rte_pktmbuf_free(pkt);
 }
 
-static void
+static int
 virtual_ethdev_close(struct rte_eth_dev *dev __rte_unused)
-{}
+{
+	return 0;
+}
 
 static int
 virtual_ethdev_configure_success(struct rte_eth_dev *dev __rte_unused)
diff --git a/drivers/net/af_packet/rte_eth_af_packet.c b/drivers/net/af_packet/rte_eth_af_packet.c
index b9723e9619..7d0ff1cbb3 100644
--- a/drivers/net/af_packet/rte_eth_af_packet.c
+++ b/drivers/net/af_packet/rte_eth_af_packet.c
@@ -376,9 +376,10 @@ eth_stats_reset(struct rte_eth_dev *dev)
 	return 0;
 }
 
-static void
+static int
 eth_dev_close(struct rte_eth_dev *dev __rte_unused)
 {
+	return 0;
 }
 
 static void
diff --git a/drivers/net/af_xdp/rte_eth_af_xdp.c b/drivers/net/af_xdp/rte_eth_af_xdp.c
index b65ee449fc..badbce63fb 100644
--- a/drivers/net/af_xdp/rte_eth_af_xdp.c
+++ b/drivers/net/af_xdp/rte_eth_af_xdp.c
@@ -701,7 +701,7 @@ xdp_umem_destroy(struct xsk_umem_info *umem)
 	umem = NULL;
 }
 
-static void
+static int
 eth_dev_close(struct rte_eth_dev *dev)
 {
 	struct pmd_internals *internals = dev->data->dev_private;
@@ -731,6 +731,8 @@ eth_dev_close(struct rte_eth_dev *dev)
 	dev->data->mac_addrs = NULL;
 
 	remove_xdp_program(internals);
+
+	return 0;
 }
 
 static void
diff --git a/drivers/net/ark/ark_ethdev.c b/drivers/net/ark/ark_ethdev.c
index e13415e95b..5f2ed4b7b7 100644
--- a/drivers/net/ark/ark_ethdev.c
+++ b/drivers/net/ark/ark_ethdev.c
@@ -30,7 +30,7 @@ static int eth_ark_dev_uninit(struct rte_eth_dev *eth_dev);
 static int eth_ark_dev_configure(struct rte_eth_dev *dev);
 static int eth_ark_dev_start(struct rte_eth_dev *dev);
 static void eth_ark_dev_stop(struct rte_eth_dev *dev);
-static void eth_ark_dev_close(struct rte_eth_dev *dev);
+static int eth_ark_dev_close(struct rte_eth_dev *dev);
 static int eth_ark_dev_info_get(struct rte_eth_dev *dev,
 				struct rte_eth_dev_info *dev_info);
 static int eth_ark_dev_link_update(struct rte_eth_dev *dev,
@@ -676,7 +676,7 @@ eth_ark_dev_stop(struct rte_eth_dev *dev)
 	}
 }
 
-static void
+static int
 eth_ark_dev_close(struct rte_eth_dev *dev)
 {
 	struct ark_adapter *ark = dev->data->dev_private;
@@ -706,6 +706,8 @@ eth_ark_dev_close(struct rte_eth_dev *dev)
 
 	rte_free(dev->data->mac_addrs);
 	dev->data->mac_addrs = 0;
+
+	return 0;
 }
 
 static int
diff --git a/drivers/net/atlantic/atl_ethdev.c b/drivers/net/atlantic/atl_ethdev.c
index 2fca9140d7..254758fd66 100644
--- a/drivers/net/atlantic/atl_ethdev.c
+++ b/drivers/net/atlantic/atl_ethdev.c
@@ -22,7 +22,7 @@ static int  atl_dev_start(struct rte_eth_dev *dev);
 static void atl_dev_stop(struct rte_eth_dev *dev);
 static int  atl_dev_set_link_up(struct rte_eth_dev *dev);
 static int  atl_dev_set_link_down(struct rte_eth_dev *dev);
-static void atl_dev_close(struct rte_eth_dev *dev);
+static int  atl_dev_close(struct rte_eth_dev *dev);
 static int  atl_dev_reset(struct rte_eth_dev *dev);
 static int  atl_dev_promiscuous_enable(struct rte_eth_dev *dev);
 static int  atl_dev_promiscuous_disable(struct rte_eth_dev *dev);
@@ -719,7 +719,7 @@ atl_dev_set_link_down(struct rte_eth_dev *dev)
 /*
  * Reset and stop device.
  */
-static void
+static int
 atl_dev_close(struct rte_eth_dev *dev)
 {
 	PMD_INIT_FUNC_TRACE();
@@ -727,6 +727,8 @@ atl_dev_close(struct rte_eth_dev *dev)
 	atl_dev_stop(dev);
 
 	atl_free_queues(dev);
+
+	return 0;
 }
 
 static int
diff --git a/drivers/net/avp/avp_ethdev.c b/drivers/net/avp/avp_ethdev.c
index 5b47f0924a..2eed6acc3f 100644
--- a/drivers/net/avp/avp_ethdev.c
+++ b/drivers/net/avp/avp_ethdev.c
@@ -38,7 +38,7 @@ static int avp_dev_create(struct rte_pci_device *pci_dev,
 static int avp_dev_configure(struct rte_eth_dev *dev);
 static int avp_dev_start(struct rte_eth_dev *dev);
 static void avp_dev_stop(struct rte_eth_dev *dev);
-static void avp_dev_close(struct rte_eth_dev *dev);
+static int avp_dev_close(struct rte_eth_dev *dev);
 static int avp_dev_info_get(struct rte_eth_dev *dev,
 			    struct rte_eth_dev_info *dev_info);
 static int avp_vlan_offload_set(struct rte_eth_dev *dev, int mask);
@@ -2103,7 +2103,7 @@ avp_dev_stop(struct rte_eth_dev *eth_dev)
 	rte_spinlock_unlock(&avp->lock);
 }
 
-static void
+static int
 avp_dev_close(struct rte_eth_dev *eth_dev)
 {
 	struct avp_dev *avp = AVP_DEV_PRIVATE_TO_HW(eth_dev->data->dev_private);
@@ -2139,6 +2139,7 @@ avp_dev_close(struct rte_eth_dev *eth_dev)
 
 unlock:
 	rte_spinlock_unlock(&avp->lock);
+	return 0;
 }
 
 static int
diff --git a/drivers/net/axgbe/axgbe_ethdev.c b/drivers/net/axgbe/axgbe_ethdev.c
index 4f5a014b54..458d64bcef 100644
--- a/drivers/net/axgbe/axgbe_ethdev.c
+++ b/drivers/net/axgbe/axgbe_ethdev.c
@@ -16,7 +16,7 @@ static int  axgbe_dev_configure(struct rte_eth_dev *dev);
 static int  axgbe_dev_start(struct rte_eth_dev *dev);
 static void axgbe_dev_stop(struct rte_eth_dev *dev);
 static void axgbe_dev_interrupt_handler(void *param);
-static void axgbe_dev_close(struct rte_eth_dev *dev);
+static int axgbe_dev_close(struct rte_eth_dev *dev);
 static int axgbe_dev_promiscuous_enable(struct rte_eth_dev *dev);
 static int axgbe_dev_promiscuous_disable(struct rte_eth_dev *dev);
 static int axgbe_dev_allmulticast_enable(struct rte_eth_dev *dev);
@@ -410,10 +410,11 @@ axgbe_dev_stop(struct rte_eth_dev *dev)
 }
 
 /* Clear all resources like TX/RX queues. */
-static void
+static int
 axgbe_dev_close(struct rte_eth_dev *dev)
 {
 	axgbe_dev_clear_queues(dev);
+	return 0;
 }
 
 static int
diff --git a/drivers/net/bnx2x/bnx2x_ethdev.c b/drivers/net/bnx2x/bnx2x_ethdev.c
index b2ea5fafa7..73217c2138 100644
--- a/drivers/net/bnx2x/bnx2x_ethdev.c
+++ b/drivers/net/bnx2x/bnx2x_ethdev.c
@@ -280,7 +280,7 @@ bnx2x_dev_stop(struct rte_eth_dev *dev)
 	return;
 }
 
-static void
+static int
 bnx2x_dev_close(struct rte_eth_dev *dev)
 {
 	struct bnx2x_softc *sc = dev->data->dev_private;
@@ -295,6 +295,8 @@ bnx2x_dev_close(struct rte_eth_dev *dev)
 
 	/* free ilt */
 	bnx2x_free_ilt_mem(sc);
+
+	return 0;
 }
 
 static int
diff --git a/drivers/net/bnxt/bnxt_ethdev.c b/drivers/net/bnxt/bnxt_ethdev.c
index 05e9a6abbf..e726d078ec 100644
--- a/drivers/net/bnxt/bnxt_ethdev.c
+++ b/drivers/net/bnxt/bnxt_ethdev.c
@@ -1357,7 +1357,7 @@ static void bnxt_dev_stop_op(struct rte_eth_dev *eth_dev)
 		bp->flow_stat->flow_count = 0;
 }
 
-static void bnxt_dev_close_op(struct rte_eth_dev *eth_dev)
+static int bnxt_dev_close_op(struct rte_eth_dev *eth_dev)
 {
 	struct bnxt *bp = eth_dev->data->dev_private;
 
@@ -1392,6 +1392,8 @@ static void bnxt_dev_close_op(struct rte_eth_dev *eth_dev)
 
 	rte_free(bp->grp_info);
 	bp->grp_info = NULL;
+
+	return 0;
 }
 
 static void bnxt_mac_addr_remove_op(struct rte_eth_dev *eth_dev,
diff --git a/drivers/net/bnxt/bnxt_reps.c b/drivers/net/bnxt/bnxt_reps.c
index 17010f1ee2..2b6c0a277b 100644
--- a/drivers/net/bnxt/bnxt_reps.c
+++ b/drivers/net/bnxt/bnxt_reps.c
@@ -457,10 +457,11 @@ void bnxt_vf_rep_dev_stop_op(struct rte_eth_dev *eth_dev)
 	bnxt_vf_rep_free_rx_mbufs(vfr_bp);
 }
 
-void bnxt_vf_rep_dev_close_op(struct rte_eth_dev *eth_dev)
+int bnxt_vf_rep_dev_close_op(struct rte_eth_dev *eth_dev)
 {
 	BNXT_TF_DBG(DEBUG, "BNXT Port:%d VFR close\n", eth_dev->data->port_id);
 	bnxt_vf_representor_uninit(eth_dev);
+	return 0;
 }
 
 int bnxt_vf_rep_dev_info_get_op(struct rte_eth_dev *eth_dev,
diff --git a/drivers/net/bnxt/bnxt_reps.h b/drivers/net/bnxt/bnxt_reps.h
index 3239e03fc0..d9b9ee2757 100644
--- a/drivers/net/bnxt/bnxt_reps.h
+++ b/drivers/net/bnxt/bnxt_reps.h
@@ -45,7 +45,7 @@ int bnxt_vf_rep_tx_queue_setup_op(struct rte_eth_dev *eth_dev,
 void bnxt_vf_rep_rx_queue_release_op(void *rx_queue);
 void bnxt_vf_rep_tx_queue_release_op(void *tx_queue);
 void bnxt_vf_rep_dev_stop_op(struct rte_eth_dev *eth_dev);
-void bnxt_vf_rep_dev_close_op(struct rte_eth_dev *eth_dev);
+int bnxt_vf_rep_dev_close_op(struct rte_eth_dev *eth_dev);
 int bnxt_vf_rep_stats_get_op(struct rte_eth_dev *eth_dev,
 			     struct rte_eth_stats *stats);
 int bnxt_vf_rep_stats_reset_op(struct rte_eth_dev *eth_dev);
diff --git a/drivers/net/bonding/eth_bond_private.h b/drivers/net/bonding/eth_bond_private.h
index c9b2d0fe46..0a0034705d 100644
--- a/drivers/net/bonding/eth_bond_private.h
+++ b/drivers/net/bonding/eth_bond_private.h
@@ -318,7 +318,7 @@ bond_tlb_activate_slave(struct bond_dev_private *internals);
 void
 bond_ethdev_stop(struct rte_eth_dev *eth_dev);
 
-void
+int
 bond_ethdev_close(struct rte_eth_dev *dev);
 
 #endif
diff --git a/drivers/net/bonding/rte_eth_bond_pmd.c b/drivers/net/bonding/rte_eth_bond_pmd.c
index 4ffd2ca4bb..dff835ef92 100644
--- a/drivers/net/bonding/rte_eth_bond_pmd.c
+++ b/drivers/net/bonding/rte_eth_bond_pmd.c
@@ -2095,7 +2095,7 @@ bond_ethdev_stop(struct rte_eth_dev *eth_dev)
 	}
 }
 
-void
+int
 bond_ethdev_close(struct rte_eth_dev *dev)
 {
 	struct bond_dev_private *internals = dev->data->dev_private;
@@ -2119,6 +2119,8 @@ bond_ethdev_close(struct rte_eth_dev *dev)
 	bond_flow_ops.flush(dev, &ferror);
 	bond_ethdev_free_queues(dev);
 	rte_bitmap_reset(internals->vlan_filter_bmp);
+
+	return 0;
 }
 
 /* forward declaration */
diff --git a/drivers/net/cxgbe/cxgbe_ethdev.c b/drivers/net/cxgbe/cxgbe_ethdev.c
index 422e190daf..e4bbba5c32 100644
--- a/drivers/net/cxgbe/cxgbe_ethdev.c
+++ b/drivers/net/cxgbe/cxgbe_ethdev.c
@@ -318,7 +318,7 @@ int cxgbe_dev_mtu_set(struct rte_eth_dev *eth_dev, uint16_t mtu)
 /*
  * Stop device.
  */
-void cxgbe_dev_close(struct rte_eth_dev *eth_dev)
+int cxgbe_dev_close(struct rte_eth_dev *eth_dev)
 {
 	struct port_info *temp_pi, *pi = eth_dev->data->dev_private;
 	struct adapter *adapter = pi->adapter;
@@ -327,10 +327,10 @@ void cxgbe_dev_close(struct rte_eth_dev *eth_dev)
 	CXGBE_FUNC_TRACE();
 
 	if (!(adapter->flags & FULL_INIT_DONE))
-		return;
+		return 0;
 
 	if (!pi->viid)
-		return;
+		return 0;
 
 	cxgbe_down(pi);
 	t4_sge_eth_release_queues(pi);
@@ -343,11 +343,13 @@ void cxgbe_dev_close(struct rte_eth_dev *eth_dev)
 	for_each_port(adapter, i) {
 		temp_pi = adap2pinfo(adapter, i);
 		if (temp_pi->viid)
-			return;
+			return 0;
 	}
 
 	cxgbe_close(adapter);
 	rte_free(adapter);
+
+	return 0;
 }
 
 /* Start the device.
diff --git a/drivers/net/cxgbe/cxgbe_pfvf.h b/drivers/net/cxgbe/cxgbe_pfvf.h
index 0b7c52aecf..69d91639e9 100644
--- a/drivers/net/cxgbe/cxgbe_pfvf.h
+++ b/drivers/net/cxgbe/cxgbe_pfvf.h
@@ -19,7 +19,7 @@
 void cxgbe_dev_rx_queue_release(void *q);
 void cxgbe_dev_tx_queue_release(void *q);
 void cxgbe_dev_stop(struct rte_eth_dev *eth_dev);
-void cxgbe_dev_close(struct rte_eth_dev *eth_dev);
+int cxgbe_dev_close(struct rte_eth_dev *eth_dev);
 int cxgbe_dev_info_get(struct rte_eth_dev *eth_dev,
 		       struct rte_eth_dev_info *device_info);
 int cxgbe_dev_promiscuous_enable(struct rte_eth_dev *eth_dev);
diff --git a/drivers/net/dpaa/dpaa_ethdev.c b/drivers/net/dpaa/dpaa_ethdev.c
index 8a6b19b5ce..7fe06e1830 100644
--- a/drivers/net/dpaa/dpaa_ethdev.c
+++ b/drivers/net/dpaa/dpaa_ethdev.c
@@ -369,7 +369,7 @@ static void dpaa_eth_dev_stop(struct rte_eth_dev *dev)
 	dev->tx_pkt_burst = dpaa_eth_tx_drop_all;
 }
 
-static void dpaa_eth_dev_close(struct rte_eth_dev *dev)
+static int dpaa_eth_dev_close(struct rte_eth_dev *dev)
 {
 	struct fman_if *fif = dev->process_private;
 	struct __fman_if *__fif;
@@ -392,6 +392,8 @@ static void dpaa_eth_dev_close(struct rte_eth_dev *dev)
 					     dpaa_interrupt_handler,
 					     (void *)dev);
 	}
+
+	return 0;
 }
 
 static int
diff --git a/drivers/net/dpaa2/dpaa2_ethdev.c b/drivers/net/dpaa2/dpaa2_ethdev.c
index 02daa4d250..fb4165fa03 100644
--- a/drivers/net/dpaa2/dpaa2_ethdev.c
+++ b/drivers/net/dpaa2/dpaa2_ethdev.c
@@ -1236,7 +1236,7 @@ dpaa2_dev_stop(struct rte_eth_dev *dev)
 	rte_eth_linkstatus_set(dev, &link);
 }
 
-static void
+static int
 dpaa2_dev_close(struct rte_eth_dev *dev)
 {
 	struct dpaa2_dev_priv *priv = dev->data->dev_private;
@@ -1252,11 +1252,13 @@ dpaa2_dev_close(struct rte_eth_dev *dev)
 	ret = dpni_reset(dpni, CMD_PRI_LOW, priv->token);
 	if (ret) {
 		DPAA2_PMD_ERR("Failure cleaning dpni device: err=%d", ret);
-		return;
+		return -1;
 	}
 
 	memset(&link, 0, sizeof(link));
 	rte_eth_linkstatus_set(dev, &link);
+
+	return 0;
 }
 
 static int
diff --git a/drivers/net/e1000/em_ethdev.c b/drivers/net/e1000/em_ethdev.c
index 1dc360713a..2d0e071448 100644
--- a/drivers/net/e1000/em_ethdev.c
+++ b/drivers/net/e1000/em_ethdev.c
@@ -34,7 +34,7 @@
 static int eth_em_configure(struct rte_eth_dev *dev);
 static int eth_em_start(struct rte_eth_dev *dev);
 static void eth_em_stop(struct rte_eth_dev *dev);
-static void eth_em_close(struct rte_eth_dev *dev);
+static int eth_em_close(struct rte_eth_dev *dev);
 static int eth_em_promiscuous_enable(struct rte_eth_dev *dev);
 static int eth_em_promiscuous_disable(struct rte_eth_dev *dev);
 static int eth_em_allmulticast_enable(struct rte_eth_dev *dev);
@@ -758,7 +758,7 @@ eth_em_stop(struct rte_eth_dev *dev)
 	}
 }
 
-static void
+static int
 eth_em_close(struct rte_eth_dev *dev)
 {
 	struct e1000_hw *hw = E1000_DEV_PRIVATE_TO_HW(dev->data->dev_private);
@@ -782,6 +782,8 @@ eth_em_close(struct rte_eth_dev *dev)
 	rte_intr_disable(intr_handle);
 	rte_intr_callback_unregister(intr_handle,
 				     eth_em_interrupt_handler, dev);
+
+	return 0;
 }
 
 static int
diff --git a/drivers/net/e1000/igb_ethdev.c b/drivers/net/e1000/igb_ethdev.c
index 5ab74840aa..9e4aefe00c 100644
--- a/drivers/net/e1000/igb_ethdev.c
+++ b/drivers/net/e1000/igb_ethdev.c
@@ -77,7 +77,7 @@ static int  eth_igb_start(struct rte_eth_dev *dev);
 static void eth_igb_stop(struct rte_eth_dev *dev);
 static int  eth_igb_dev_set_link_up(struct rte_eth_dev *dev);
 static int  eth_igb_dev_set_link_down(struct rte_eth_dev *dev);
-static void eth_igb_close(struct rte_eth_dev *dev);
+static int eth_igb_close(struct rte_eth_dev *dev);
 static int eth_igb_reset(struct rte_eth_dev *dev);
 static int  eth_igb_promiscuous_enable(struct rte_eth_dev *dev);
 static int  eth_igb_promiscuous_disable(struct rte_eth_dev *dev);
@@ -155,7 +155,7 @@ static void igbvf_intr_disable(struct e1000_hw *hw);
 static int igbvf_dev_configure(struct rte_eth_dev *dev);
 static int igbvf_dev_start(struct rte_eth_dev *dev);
 static void igbvf_dev_stop(struct rte_eth_dev *dev);
-static void igbvf_dev_close(struct rte_eth_dev *dev);
+static int igbvf_dev_close(struct rte_eth_dev *dev);
 static int igbvf_promiscuous_enable(struct rte_eth_dev *dev);
 static int igbvf_promiscuous_disable(struct rte_eth_dev *dev);
 static int igbvf_allmulticast_enable(struct rte_eth_dev *dev);
@@ -1535,7 +1535,7 @@ eth_igb_dev_set_link_down(struct rte_eth_dev *dev)
 	return 0;
 }
 
-static void
+static int
 eth_igb_close(struct rte_eth_dev *dev)
 {
 	struct e1000_hw *hw = E1000_DEV_PRIVATE_TO_HW(dev->data->dev_private);
@@ -1604,6 +1604,8 @@ eth_igb_close(struct rte_eth_dev *dev)
 
 	/* clear all the filters list */
 	igb_filterlist_flush(dev);
+
+	return 0;
 }
 
 /*
@@ -3381,7 +3383,7 @@ igbvf_dev_stop(struct rte_eth_dev *dev)
 	adapter->stopped = true;
 }
 
-static void
+static int
 igbvf_dev_close(struct rte_eth_dev *dev)
 {
 	struct e1000_hw *hw = E1000_DEV_PRIVATE_TO_HW(dev->data->dev_private);
@@ -3412,6 +3414,8 @@ igbvf_dev_close(struct rte_eth_dev *dev)
 	rte_intr_callback_unregister(&pci_dev->intr_handle,
 				     eth_igbvf_interrupt_handler,
 				     (void *)dev);
+
+	return 0;
 }
 
 static int
diff --git a/drivers/net/ena/ena_ethdev.c b/drivers/net/ena/ena_ethdev.c
index 8077519735..ae6daad892 100644
--- a/drivers/net/ena/ena_ethdev.c
+++ b/drivers/net/ena/ena_ethdev.c
@@ -187,7 +187,7 @@ static void ena_init_rings(struct ena_adapter *adapter,
 static int ena_mtu_set(struct rte_eth_dev *dev, uint16_t mtu);
 static int ena_start(struct rte_eth_dev *dev);
 static void ena_stop(struct rte_eth_dev *dev);
-static void ena_close(struct rte_eth_dev *dev);
+static int ena_close(struct rte_eth_dev *dev);
 static int ena_dev_reset(struct rte_eth_dev *dev);
 static int ena_stats_get(struct rte_eth_dev *dev, struct rte_eth_stats *stats);
 static void ena_rx_queue_release_all(struct rte_eth_dev *dev);
@@ -487,7 +487,7 @@ static void ena_config_debug_area(struct ena_adapter *adapter)
 	ena_com_delete_debug_area(&adapter->ena_dev);
 }
 
-static void ena_close(struct rte_eth_dev *dev)
+static int ena_close(struct rte_eth_dev *dev)
 {
 	struct rte_pci_device *pci_dev = RTE_ETH_DEV_TO_PCI(dev);
 	struct rte_intr_handle *intr_handle = &pci_dev->intr_handle;
@@ -513,6 +513,8 @@ static void ena_close(struct rte_eth_dev *dev)
 	 * release of the resource in the rte_eth_dev_release_port().
 	 */
 	dev->data->mac_addrs = NULL;
+
+	return 0;
 }
 
 static int
diff --git a/drivers/net/enetc/enetc_ethdev.c b/drivers/net/enetc/enetc_ethdev.c
index c6fb42740d..bdb32762ce 100644
--- a/drivers/net/enetc/enetc_ethdev.c
+++ b/drivers/net/enetc/enetc_ethdev.c
@@ -545,7 +545,7 @@ enetc_stats_reset(struct rte_eth_dev *dev)
 	return 0;
 }
 
-static void
+static int
 enetc_dev_close(struct rte_eth_dev *dev)
 {
 	uint16_t i;
@@ -564,6 +564,8 @@ enetc_dev_close(struct rte_eth_dev *dev)
 		dev->data->tx_queues[i] = NULL;
 	}
 	dev->data->nb_tx_queues = 0;
+
+	return 0;
 }
 
 static int
diff --git a/drivers/net/enic/enic_ethdev.c b/drivers/net/enic/enic_ethdev.c
index 57e82845a4..ed03d53608 100644
--- a/drivers/net/enic/enic_ethdev.c
+++ b/drivers/net/enic/enic_ethdev.c
@@ -446,12 +446,14 @@ static void enicpmd_dev_stop(struct rte_eth_dev *eth_dev)
 /*
  * Stop device.
  */
-static void enicpmd_dev_close(struct rte_eth_dev *eth_dev)
+static int enicpmd_dev_close(struct rte_eth_dev *eth_dev)
 {
 	struct enic *enic = pmd_priv(eth_dev);
 
 	ENICPMD_FUNC_TRACE();
 	enic_remove(enic);
+
+	return 0;
 }
 
 static int enicpmd_dev_link_update(struct rte_eth_dev *eth_dev,
diff --git a/drivers/net/enic/enic_vf_representor.c b/drivers/net/enic/enic_vf_representor.c
index 5d34e1b460..c528be2b98 100644
--- a/drivers/net/enic/enic_vf_representor.c
+++ b/drivers/net/enic/enic_vf_representor.c
@@ -277,11 +277,12 @@ static void enic_vf_dev_stop(struct rte_eth_dev *eth_dev)
  * "close" is no-op for now and solely exists so that rte_eth_dev_close()
  * can finish its own cleanup without errors.
  */
-static void enic_vf_dev_close(struct rte_eth_dev *eth_dev __rte_unused)
+static int enic_vf_dev_close(struct rte_eth_dev *eth_dev __rte_unused)
 {
 	ENICPMD_FUNC_TRACE();
 	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
-		return;
+		return 0;
+	return 0;
 }
 
 static int
diff --git a/drivers/net/failsafe/failsafe_ops.c b/drivers/net/failsafe/failsafe_ops.c
index 96f7e456f9..93ebd09114 100644
--- a/drivers/net/failsafe/failsafe_ops.c
+++ b/drivers/net/failsafe/failsafe_ops.c
@@ -240,7 +240,7 @@ fs_dev_set_link_down(struct rte_eth_dev *dev)
 }
 
 static void fs_dev_free_queues(struct rte_eth_dev *dev);
-static void
+static int
 fs_dev_close(struct rte_eth_dev *dev)
 {
 	struct sub_device *sdev;
@@ -259,6 +259,7 @@ fs_dev_close(struct rte_eth_dev *dev)
 	}
 	fs_dev_free_queues(dev);
 	fs_unlock(dev, 0);
+	return 0;
 }
 
 static int
diff --git a/drivers/net/fm10k/fm10k_ethdev.c b/drivers/net/fm10k/fm10k_ethdev.c
index 5771d83b55..533f976709 100644
--- a/drivers/net/fm10k/fm10k_ethdev.c
+++ b/drivers/net/fm10k/fm10k_ethdev.c
@@ -2776,7 +2776,7 @@ fm10k_close_mbx_service(struct fm10k_hw *hw)
 	hw->mbx.ops.disconnect(hw, &hw->mbx);
 }
 
-static void
+static int
 fm10k_dev_close(struct rte_eth_dev *dev)
 {
 	struct fm10k_hw *hw = FM10K_DEV_PRIVATE_TO_HW(dev->data->dev_private);
@@ -2821,6 +2821,8 @@ fm10k_dev_close(struct rte_eth_dev *dev)
 		rte_intr_callback_unregister(intr_handle,
 			fm10k_dev_interrupt_handler_vf, (void *)dev);
 	}
+
+	return 0;
 }
 
 static const struct eth_dev_ops fm10k_eth_dev_ops = {
diff --git a/drivers/net/hinic/hinic_pmd_ethdev.c b/drivers/net/hinic/hinic_pmd_ethdev.c
index 67e6afcf7a..5f2e97d3bd 100644
--- a/drivers/net/hinic/hinic_pmd_ethdev.c
+++ b/drivers/net/hinic/hinic_pmd_ethdev.c
@@ -2936,7 +2936,7 @@ static void hinic_nic_dev_destroy(struct rte_eth_dev *eth_dev)
  * @param dev
  *   Pointer to Ethernet device structure.
  */
-static void hinic_dev_close(struct rte_eth_dev *dev)
+static int hinic_dev_close(struct rte_eth_dev *dev)
 {
 	struct hinic_nic_dev *nic_dev = HINIC_ETH_DEV_TO_PRIVATE_NIC_DEV(dev);
 
@@ -2944,7 +2944,7 @@ static void hinic_dev_close(struct rte_eth_dev *dev)
 					   &nic_dev->dev_status)) {
 		PMD_DRV_LOG(WARNING, "Device %s already closed",
 			    dev->data->name);
-		return;
+		return 0;
 	}
 
 	/* stop device first */
@@ -2971,6 +2971,8 @@ static void hinic_dev_close(struct rte_eth_dev *dev)
 
 	/* deinit nic hardware device */
 	hinic_nic_dev_destroy(dev);
+
+	return 0;
 }
 
 static const struct eth_dev_ops hinic_pmd_ops = {
diff --git a/drivers/net/hns3/hns3_ethdev.c b/drivers/net/hns3/hns3_ethdev.c
index 73d504253d..3b395a1ccf 100644
--- a/drivers/net/hns3/hns3_ethdev.c
+++ b/drivers/net/hns3/hns3_ethdev.c
@@ -4859,7 +4859,7 @@ hns3_dev_stop(struct rte_eth_dev *dev)
 	rte_spinlock_unlock(&hw->lock);
 }
 
-static void
+static int
 hns3_dev_close(struct rte_eth_dev *eth_dev)
 {
 	struct hns3_adapter *hns = eth_dev->data->dev_private;
@@ -4868,7 +4868,7 @@ hns3_dev_close(struct rte_eth_dev *eth_dev)
 	if (rte_eal_process_type() != RTE_PROC_PRIMARY) {
 		rte_free(eth_dev->process_private);
 		eth_dev->process_private = NULL;
-		return;
+		return 0;
 	}
 
 	if (hw->adapter_state == HNS3_NIC_STARTED)
@@ -4888,6 +4888,8 @@ hns3_dev_close(struct rte_eth_dev *eth_dev)
 	eth_dev->process_private = NULL;
 	hns3_mp_uninit_primary();
 	hns3_warn(hw, "Close port %d finished", hw->data->port_id);
+
+	return 0;
 }
 
 static int
diff --git a/drivers/net/hns3/hns3_ethdev_vf.c b/drivers/net/hns3/hns3_ethdev_vf.c
index 037a5be7e0..e0ca5a6ac2 100644
--- a/drivers/net/hns3/hns3_ethdev_vf.c
+++ b/drivers/net/hns3/hns3_ethdev_vf.c
@@ -1910,14 +1910,14 @@ hns3vf_dev_stop(struct rte_eth_dev *dev)
 	rte_spinlock_unlock(&hw->lock);
 }
 
-static void
+static int
 hns3vf_dev_close(struct rte_eth_dev *eth_dev)
 {
 	struct hns3_adapter *hns = eth_dev->data->dev_private;
 	struct hns3_hw *hw = &hns->hw;
 
 	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
-		return;
+		return 0;
 
 	if (hw->adapter_state == HNS3_NIC_STARTED)
 		hns3vf_dev_stop(eth_dev);
@@ -1935,6 +1935,8 @@ hns3vf_dev_close(struct rte_eth_dev *eth_dev)
 	eth_dev->process_private = NULL;
 	hns3_mp_uninit_primary();
 	hns3_warn(hw, "Close port %d finished", hw->data->port_id);
+
+	return 0;
 }
 
 static int
diff --git a/drivers/net/i40e/i40e_ethdev.c b/drivers/net/i40e/i40e_ethdev.c
index 563f21d9df..804f0128ab 100644
--- a/drivers/net/i40e/i40e_ethdev.c
+++ b/drivers/net/i40e/i40e_ethdev.c
@@ -225,7 +225,7 @@ static int eth_i40e_dev_uninit(struct rte_eth_dev *eth_dev);
 static int i40e_dev_configure(struct rte_eth_dev *dev);
 static int i40e_dev_start(struct rte_eth_dev *dev);
 static void i40e_dev_stop(struct rte_eth_dev *dev);
-static void i40e_dev_close(struct rte_eth_dev *dev);
+static int i40e_dev_close(struct rte_eth_dev *dev);
 static int  i40e_dev_reset(struct rte_eth_dev *dev);
 static int i40e_dev_promiscuous_enable(struct rte_eth_dev *dev);
 static int i40e_dev_promiscuous_disable(struct rte_eth_dev *dev);
@@ -2608,7 +2608,7 @@ i40e_dev_stop(struct rte_eth_dev *dev)
 	pf->adapter->rss_reta_updated = 0;
 }
 
-static void
+static int
 i40e_dev_close(struct rte_eth_dev *dev)
 {
 	struct i40e_pf *pf = I40E_DEV_PRIVATE_TO_PF(dev->data->dev_private);
@@ -2748,6 +2748,7 @@ i40e_dev_close(struct rte_eth_dev *dev)
 	i40e_tm_conf_uninit(dev);
 
 	hw->adapter_closed = 1;
+	return 0;
 }
 
 /*
diff --git a/drivers/net/i40e/i40e_ethdev_vf.c b/drivers/net/i40e/i40e_ethdev_vf.c
index 8531cf6b12..61aad8c415 100644
--- a/drivers/net/i40e/i40e_ethdev_vf.c
+++ b/drivers/net/i40e/i40e_ethdev_vf.c
@@ -90,7 +90,7 @@ static int i40evf_dev_xstats_reset(struct rte_eth_dev *dev);
 static int i40evf_vlan_filter_set(struct rte_eth_dev *dev,
 				  uint16_t vlan_id, int on);
 static int i40evf_vlan_offload_set(struct rte_eth_dev *dev, int mask);
-static void i40evf_dev_close(struct rte_eth_dev *dev);
+static int i40evf_dev_close(struct rte_eth_dev *dev);
 static int i40evf_dev_reset(struct rte_eth_dev *dev);
 static int i40evf_check_vf_reset_done(struct rte_eth_dev *dev);
 static int i40evf_dev_promiscuous_enable(struct rte_eth_dev *dev);
@@ -2401,7 +2401,7 @@ i40evf_dev_stats_get(struct rte_eth_dev *dev, struct rte_eth_stats *stats)
 	return ret;
 }
 
-static void
+static int
 i40evf_dev_close(struct rte_eth_dev *dev)
 {
 	struct i40e_hw *hw = I40E_DEV_PRIVATE_TO_HW(dev->data->dev_private);
@@ -2433,6 +2433,7 @@ i40evf_dev_close(struct rte_eth_dev *dev)
 	vf->aq_resp = NULL;
 
 	hw->adapter_closed = 1;
+	return 0;
 }
 
 /*
diff --git a/drivers/net/iavf/iavf_ethdev.c b/drivers/net/iavf/iavf_ethdev.c
index 440da7d76a..8f46c4588f 100644
--- a/drivers/net/iavf/iavf_ethdev.c
+++ b/drivers/net/iavf/iavf_ethdev.c
@@ -32,7 +32,7 @@
 static int iavf_dev_configure(struct rte_eth_dev *dev);
 static int iavf_dev_start(struct rte_eth_dev *dev);
 static void iavf_dev_stop(struct rte_eth_dev *dev);
-static void iavf_dev_close(struct rte_eth_dev *dev);
+static int iavf_dev_close(struct rte_eth_dev *dev);
 static int iavf_dev_reset(struct rte_eth_dev *dev);
 static int iavf_dev_info_get(struct rte_eth_dev *dev,
 			     struct rte_eth_dev_info *dev_info);
@@ -1463,7 +1463,7 @@ iavf_dev_init(struct rte_eth_dev *eth_dev)
 	return 0;
 }
 
-static void
+static int
 iavf_dev_close(struct rte_eth_dev *dev)
 {
 	struct iavf_hw *hw = IAVF_DEV_PRIVATE_TO_HW(dev->data->dev_private);
@@ -1506,6 +1506,8 @@ iavf_dev_close(struct rte_eth_dev *dev)
 
 	rte_free(vf->aq_resp);
 	vf->aq_resp = NULL;
+
+	return 0;
 }
 
 static int
diff --git a/drivers/net/ice/ice_dcf_ethdev.c b/drivers/net/ice/ice_dcf_ethdev.c
index 2faed3cc7a..5b626cb641 100644
--- a/drivers/net/ice/ice_dcf_ethdev.c
+++ b/drivers/net/ice/ice_dcf_ethdev.c
@@ -847,13 +847,13 @@ ice_dcf_stats_reset(struct rte_eth_dev *dev)
 	return 0;
 }
 
-static void
+static int
 ice_dcf_dev_close(struct rte_eth_dev *dev)
 {
 	struct ice_dcf_adapter *adapter = dev->data->dev_private;
 
 	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
-		return;
+		return 0;
 
 	dev->dev_ops = NULL;
 	dev->rx_pkt_burst = NULL;
@@ -861,6 +861,8 @@ ice_dcf_dev_close(struct rte_eth_dev *dev)
 
 	ice_dcf_uninit_parent_adapter(dev);
 	ice_dcf_uninit_hw(dev, &adapter->real_hw);
+
+	return 0;
 }
 
 static int
diff --git a/drivers/net/ice/ice_ethdev.c b/drivers/net/ice/ice_ethdev.c
index cfd357b055..b4ac5e3f81 100644
--- a/drivers/net/ice/ice_ethdev.c
+++ b/drivers/net/ice/ice_ethdev.c
@@ -77,7 +77,7 @@ static struct proto_xtr_ol_flag ice_proto_xtr_ol_flag_params[] = {
 static int ice_dev_configure(struct rte_eth_dev *dev);
 static int ice_dev_start(struct rte_eth_dev *dev);
 static void ice_dev_stop(struct rte_eth_dev *dev);
-static void ice_dev_close(struct rte_eth_dev *dev);
+static int ice_dev_close(struct rte_eth_dev *dev);
 static int ice_dev_reset(struct rte_eth_dev *dev);
 static int ice_dev_info_get(struct rte_eth_dev *dev,
 			    struct rte_eth_dev_info *dev_info);
@@ -2430,7 +2430,7 @@ ice_dev_stop(struct rte_eth_dev *dev)
 	pf->adapter_stopped = true;
 }
 
-static void
+static int
 ice_dev_close(struct rte_eth_dev *dev)
 {
 	struct ice_pf *pf = ICE_DEV_PRIVATE_TO_PF(dev->data->dev_private);
@@ -2478,6 +2478,8 @@ ice_dev_close(struct rte_eth_dev *dev)
 	/* unregister callback func from eal lib */
 	rte_intr_callback_unregister(intr_handle,
 				     ice_interrupt_handler, dev);
+
+	return 0;
 }
 
 static int
diff --git a/drivers/net/igc/igc_ethdev.c b/drivers/net/igc/igc_ethdev.c
index 810568bc54..11397c5c1d 100644
--- a/drivers/net/igc/igc_ethdev.c
+++ b/drivers/net/igc/igc_ethdev.c
@@ -183,7 +183,7 @@ static void eth_igc_stop(struct rte_eth_dev *dev);
 static int eth_igc_start(struct rte_eth_dev *dev);
 static int eth_igc_set_link_up(struct rte_eth_dev *dev);
 static int eth_igc_set_link_down(struct rte_eth_dev *dev);
-static void eth_igc_close(struct rte_eth_dev *dev);
+static int eth_igc_close(struct rte_eth_dev *dev);
 static int eth_igc_reset(struct rte_eth_dev *dev);
 static int eth_igc_promiscuous_enable(struct rte_eth_dev *dev);
 static int eth_igc_promiscuous_disable(struct rte_eth_dev *dev);
@@ -1165,7 +1165,7 @@ igc_dev_free_queues(struct rte_eth_dev *dev)
 	dev->data->nb_tx_queues = 0;
 }
 
-static void
+static int
 eth_igc_close(struct rte_eth_dev *dev)
 {
 	struct rte_pci_device *pci_dev = RTE_ETH_DEV_TO_PCI(dev);
@@ -1199,6 +1199,8 @@ eth_igc_close(struct rte_eth_dev *dev)
 
 	/* Reset any pending lock */
 	igc_reset_swfw_lock(hw);
+
+	return 0;
 }
 
 static void
diff --git a/drivers/net/ionic/ionic_ethdev.c b/drivers/net/ionic/ionic_ethdev.c
index 800f6e5d64..1775fd29b7 100644
--- a/drivers/net/ionic/ionic_ethdev.c
+++ b/drivers/net/ionic/ionic_ethdev.c
@@ -25,7 +25,7 @@ static int  ionic_dev_configure(struct rte_eth_dev *dev);
 static int  ionic_dev_mtu_set(struct rte_eth_dev *dev, uint16_t mtu);
 static int  ionic_dev_start(struct rte_eth_dev *dev);
 static void ionic_dev_stop(struct rte_eth_dev *dev);
-static void ionic_dev_close(struct rte_eth_dev *dev);
+static int  ionic_dev_close(struct rte_eth_dev *dev);
 static int  ionic_dev_set_link_up(struct rte_eth_dev *dev);
 static int  ionic_dev_set_link_down(struct rte_eth_dev *dev);
 static int  ionic_dev_link_update(struct rte_eth_dev *eth_dev,
@@ -956,7 +956,7 @@ ionic_dev_stop(struct rte_eth_dev *eth_dev)
 /*
  * Reset and stop device.
  */
-static void
+static int
 ionic_dev_close(struct rte_eth_dev *eth_dev)
 {
 	struct ionic_lif *lif = IONIC_ETH_DEV_TO_LIF(eth_dev);
@@ -967,14 +967,16 @@ ionic_dev_close(struct rte_eth_dev *eth_dev)
 	err = ionic_lif_stop(lif);
 	if (err) {
 		IONIC_PRINT(ERR, "Cannot stop LIF: %d", err);
-		return;
+		return -1;
 	}
 
 	err = eth_ionic_dev_uninit(eth_dev);
 	if (err) {
 		IONIC_PRINT(ERR, "Cannot destroy LIF: %d", err);
-		return;
+		return -1;
 	}
+
+	return 0;
 }
 
 static int
diff --git a/drivers/net/ipn3ke/ipn3ke_representor.c b/drivers/net/ipn3ke/ipn3ke_representor.c
index b673c49149..d49abbf758 100644
--- a/drivers/net/ipn3ke/ipn3ke_representor.c
+++ b/drivers/net/ipn3ke/ipn3ke_representor.c
@@ -208,7 +208,7 @@ ipn3ke_rpst_dev_stop(struct rte_eth_dev *dev)
 	}
 }
 
-static void
+static int
 ipn3ke_rpst_dev_close(struct rte_eth_dev *dev)
 {
 	struct ipn3ke_hw *hw = IPN3KE_DEV_PRIVATE_TO_HW(dev);
@@ -221,6 +221,8 @@ ipn3ke_rpst_dev_close(struct rte_eth_dev *dev)
 		/* Disable the RX path */
 		ipn3ke_xmac_rx_disable(hw, rpst->port_id, 0);
 	}
+
+	return 0;
 }
 
 /*
diff --git a/drivers/net/ixgbe/ixgbe_ethdev.c b/drivers/net/ixgbe/ixgbe_ethdev.c
index 0f065bbc04..a1369a0744 100644
--- a/drivers/net/ixgbe/ixgbe_ethdev.c
+++ b/drivers/net/ixgbe/ixgbe_ethdev.c
@@ -147,7 +147,7 @@ static int  ixgbe_dev_start(struct rte_eth_dev *dev);
 static void ixgbe_dev_stop(struct rte_eth_dev *dev);
 static int  ixgbe_dev_set_link_up(struct rte_eth_dev *dev);
 static int  ixgbe_dev_set_link_down(struct rte_eth_dev *dev);
-static void ixgbe_dev_close(struct rte_eth_dev *dev);
+static int  ixgbe_dev_close(struct rte_eth_dev *dev);
 static int  ixgbe_dev_reset(struct rte_eth_dev *dev);
 static int ixgbe_dev_promiscuous_enable(struct rte_eth_dev *dev);
 static int ixgbe_dev_promiscuous_disable(struct rte_eth_dev *dev);
@@ -251,7 +251,7 @@ static int  ixgbevf_dev_start(struct rte_eth_dev *dev);
 static int ixgbevf_dev_link_update(struct rte_eth_dev *dev,
 				   int wait_to_complete);
 static void ixgbevf_dev_stop(struct rte_eth_dev *dev);
-static void ixgbevf_dev_close(struct rte_eth_dev *dev);
+static int ixgbevf_dev_close(struct rte_eth_dev *dev);
 static int  ixgbevf_dev_reset(struct rte_eth_dev *dev);
 static void ixgbevf_intr_disable(struct rte_eth_dev *dev);
 static void ixgbevf_intr_enable(struct rte_eth_dev *dev);
@@ -2994,7 +2994,7 @@ ixgbe_dev_set_link_down(struct rte_eth_dev *dev)
 /*
  * Reset and stop device.
  */
-static void
+static int
 ixgbe_dev_close(struct rte_eth_dev *dev)
 {
 	struct ixgbe_hw *hw =
@@ -3065,6 +3065,7 @@ ixgbe_dev_close(struct rte_eth_dev *dev)
 	rte_free(dev->security_ctx);
 #endif
 
+	return 0;
 }
 
 /*
@@ -5443,7 +5444,7 @@ ixgbevf_dev_stop(struct rte_eth_dev *dev)
 	adapter->rss_reta_updated = 0;
 }
 
-static void
+static int
 ixgbevf_dev_close(struct rte_eth_dev *dev)
 {
 	struct ixgbe_hw *hw = IXGBE_DEV_PRIVATE_TO_HW(dev->data->dev_private);
@@ -5472,6 +5473,8 @@ ixgbevf_dev_close(struct rte_eth_dev *dev)
 	rte_intr_disable(intr_handle);
 	rte_intr_callback_unregister(intr_handle,
 				     ixgbevf_dev_interrupt_handler, dev);
+
+	return 0;
 }
 
 /*
diff --git a/drivers/net/kni/rte_eth_kni.c b/drivers/net/kni/rte_eth_kni.c
index 831fe96c96..aa365a57de 100644
--- a/drivers/net/kni/rte_eth_kni.c
+++ b/drivers/net/kni/rte_eth_kni.c
@@ -198,7 +198,7 @@ eth_kni_dev_stop(struct rte_eth_dev *dev)
 	dev->data->dev_link.link_status = 0;
 }
 
-static void
+static int
 eth_kni_close(struct rte_eth_dev *eth_dev)
 {
 	struct pmd_internals *internals;
@@ -214,6 +214,8 @@ eth_kni_close(struct rte_eth_dev *eth_dev)
 	if (ret)
 		PMD_LOG(WARNING, "Not able to release kni for %s",
 			eth_dev->data->name);
+
+	return 0;
 }
 
 static int
diff --git a/drivers/net/liquidio/lio_ethdev.c b/drivers/net/liquidio/lio_ethdev.c
index 2c2b27e62a..93e2ed5670 100644
--- a/drivers/net/liquidio/lio_ethdev.c
+++ b/drivers/net/liquidio/lio_ethdev.c
@@ -1550,7 +1550,7 @@ lio_dev_set_link_down(struct rte_eth_dev *eth_dev)
  * @return
  *    - nothing
  */
-static void
+static int
 lio_dev_close(struct rte_eth_dev *eth_dev)
 {
 	struct lio_device *lio_dev = LIO_DEV(eth_dev);
@@ -1581,6 +1581,8 @@ lio_dev_close(struct rte_eth_dev *eth_dev)
 
 	 /* Delete all queues */
 	lio_dev_clear_queues(eth_dev);
+
+	return 0;
 }
 
 /**
diff --git a/drivers/net/memif/rte_eth_memif.c b/drivers/net/memif/rte_eth_memif.c
index a19c0f3e6b..1a2411c838 100644
--- a/drivers/net/memif/rte_eth_memif.c
+++ b/drivers/net/memif/rte_eth_memif.c
@@ -1218,7 +1218,7 @@ memif_dev_start(struct rte_eth_dev *dev)
 	return ret;
 }
 
-static void
+static int
 memif_dev_close(struct rte_eth_dev *dev)
 {
 	struct pmd_internals *pmd = dev->data->dev_private;
@@ -1239,6 +1239,8 @@ memif_dev_close(struct rte_eth_dev *dev)
 	}
 
 	rte_free(dev->process_private);
+
+	return 0;
 }
 
 static int
diff --git a/drivers/net/mlx4/mlx4.c b/drivers/net/mlx4/mlx4.c
index 71061a720f..df59314b66 100644
--- a/drivers/net/mlx4/mlx4.c
+++ b/drivers/net/mlx4/mlx4.c
@@ -370,7 +370,7 @@ mlx4_dev_stop(struct rte_eth_dev *dev)
  * @param dev
  *   Pointer to Ethernet device structure.
  */
-static void
+static int
 mlx4_dev_close(struct rte_eth_dev *dev)
 {
 	struct mlx4_priv *priv = dev->data->dev_private;
@@ -400,6 +400,7 @@ mlx4_dev_close(struct rte_eth_dev *dev)
 		MLX4_ASSERT(priv->ctx == NULL);
 	mlx4_intr_uninstall(priv);
 	memset(priv, 0, sizeof(*priv));
+	return 0;
 }
 
 static const struct eth_dev_ops mlx4_dev_ops = {
diff --git a/drivers/net/mlx5/mlx5.c b/drivers/net/mlx5/mlx5.c
index 4a807fb4fd..01ead6e6af 100644
--- a/drivers/net/mlx5/mlx5.c
+++ b/drivers/net/mlx5/mlx5.c
@@ -1341,7 +1341,7 @@ mlx5_proc_priv_uninit(struct rte_eth_dev *dev)
  * @param dev
  *   Pointer to Ethernet device structure.
  */
-void
+int
 mlx5_dev_close(struct rte_eth_dev *dev)
 {
 	struct mlx5_priv *priv = dev->data->dev_private;
@@ -1351,14 +1351,14 @@ mlx5_dev_close(struct rte_eth_dev *dev)
 	if (rte_eal_process_type() == RTE_PROC_SECONDARY) {
 		/* Check if process_private released. */
 		if (!dev->process_private)
-			return;
+			return 0;
 		mlx5_tx_uar_uninit_secondary(dev);
 		mlx5_proc_priv_uninit(dev);
 		rte_eth_dev_release_port(dev);
-		return;
+		return 0;
 	}
 	if (!priv->sh)
-		return;
+		return 0;
 	DRV_LOG(DEBUG, "port %u closing device \"%s\"",
 		dev->data->port_id,
 		((priv->sh->ctx != NULL) ?
@@ -1479,6 +1479,7 @@ mlx5_dev_close(struct rte_eth_dev *dev)
 	 * it is freed when dev_private is freed.
 	 */
 	dev->data->mac_addrs = NULL;
+	return 0;
 }
 
 /**
diff --git a/drivers/net/mlx5/mlx5.h b/drivers/net/mlx5/mlx5.h
index 865e72d318..56f0e25b4c 100644
--- a/drivers/net/mlx5/mlx5.h
+++ b/drivers/net/mlx5/mlx5.h
@@ -842,7 +842,7 @@ int mlx5_proc_priv_init(struct rte_eth_dev *dev);
 int mlx5_udp_tunnel_port_add(struct rte_eth_dev *dev,
 			      struct rte_eth_udp_tunnel *udp_tunnel);
 uint16_t mlx5_eth_find_next(uint16_t port_id, struct rte_pci_device *pci_dev);
-void mlx5_dev_close(struct rte_eth_dev *dev);
+int mlx5_dev_close(struct rte_eth_dev *dev);
 
 /* Macro to iterate over all valid ports for mlx5 driver. */
 #define MLX5_ETH_FOREACH_DEV(port_id, pci_dev) \
diff --git a/drivers/net/mvneta/mvneta_ethdev.c b/drivers/net/mvneta/mvneta_ethdev.c
index 5bf3ebc4bd..eebcdb840e 100644
--- a/drivers/net/mvneta/mvneta_ethdev.c
+++ b/drivers/net/mvneta/mvneta_ethdev.c
@@ -429,7 +429,7 @@ mvneta_dev_stop(struct rte_eth_dev *dev)
  * @param dev
  *   Pointer to Ethernet device structure.
  */
-static void
+static int
 mvneta_dev_close(struct rte_eth_dev *dev)
 {
 	struct mvneta_priv *priv = dev->data->dev_private;
@@ -455,6 +455,8 @@ mvneta_dev_close(struct rte_eth_dev *dev)
 		mvneta_neta_deinit();
 		rte_mvep_deinit(MVEP_MOD_T_NETA);
 	}
+
+	return 0;
 }
 
 /**
diff --git a/drivers/net/mvpp2/mrvl_ethdev.c b/drivers/net/mvpp2/mrvl_ethdev.c
index f5e77d07bf..27f7cfb056 100644
--- a/drivers/net/mvpp2/mrvl_ethdev.c
+++ b/drivers/net/mvpp2/mrvl_ethdev.c
@@ -855,7 +855,7 @@ mrvl_dev_stop(struct rte_eth_dev *dev)
  * @param dev
  *   Pointer to Ethernet device structure.
  */
-static void
+static int
 mrvl_dev_close(struct rte_eth_dev *dev)
 {
 	struct mrvl_priv *priv = dev->data->dev_private;
@@ -915,6 +915,8 @@ mrvl_dev_close(struct rte_eth_dev *dev)
 		mrvl_deinit_pp2();
 		rte_mvep_deinit(MVEP_MOD_T_PP2);
 	}
+
+	return 0;
 }
 
 /**
diff --git a/drivers/net/netvsc/hn_ethdev.c b/drivers/net/netvsc/hn_ethdev.c
index 229c1b9149..8968036ea3 100644
--- a/drivers/net/netvsc/hn_ethdev.c
+++ b/drivers/net/netvsc/hn_ethdev.c
@@ -841,13 +841,15 @@ hn_dev_stop(struct rte_eth_dev *dev)
 	hn_vf_stop(dev);
 }
 
-static void
+static int
 hn_dev_close(struct rte_eth_dev *dev)
 {
 	PMD_INIT_FUNC_TRACE();
 
 	hn_vf_close(dev);
 	hn_dev_free_queues(dev);
+
+	return 0;
 }
 
 static const struct eth_dev_ops hn_eth_dev_ops = {
diff --git a/drivers/net/nfb/nfb_ethdev.c b/drivers/net/nfb/nfb_ethdev.c
index b039ab6fc2..c3c3d003f1 100644
--- a/drivers/net/nfb/nfb_ethdev.c
+++ b/drivers/net/nfb/nfb_ethdev.c
@@ -209,7 +209,7 @@ nfb_eth_dev_info(struct rte_eth_dev *dev,
  * @param dev
  *   Pointer to Ethernet device structure.
  */
-static void
+static int
 nfb_eth_dev_close(struct rte_eth_dev *dev)
 {
 	struct pmd_internals *internals = dev->data->dev_private;
@@ -235,6 +235,8 @@ nfb_eth_dev_close(struct rte_eth_dev *dev)
 
 	rte_free(dev->data->mac_addrs);
 	dev->data->mac_addrs = NULL;
+
+	return 0;
 }
 
 /**
diff --git a/drivers/net/nfp/nfp_net.c b/drivers/net/nfp/nfp_net.c
index f5e4cd339d..9509dc8bd6 100644
--- a/drivers/net/nfp/nfp_net.c
+++ b/drivers/net/nfp/nfp_net.c
@@ -50,7 +50,7 @@
 #include <errno.h>
 
 /* Prototypes */
-static void nfp_net_close(struct rte_eth_dev *dev);
+static int nfp_net_close(struct rte_eth_dev *dev);
 static int nfp_net_configure(struct rte_eth_dev *dev);
 static void nfp_net_dev_interrupt_handler(void *param);
 static void nfp_net_dev_interrupt_delayed_handler(void *param);
@@ -864,7 +864,7 @@ nfp_net_set_link_down(struct rte_eth_dev *dev)
 }
 
 /* Reset and stop device. The device can not be restarted. */
-static void
+static int
 nfp_net_close(struct rte_eth_dev *dev)
 {
 	struct nfp_net_hw *hw;
@@ -906,6 +906,8 @@ nfp_net_close(struct rte_eth_dev *dev)
 	 * The ixgbe PMD driver disables the pcie master on the
 	 * device. The i40e does not...
 	 */
+
+	return 0;
 }
 
 static int
diff --git a/drivers/net/octeontx/octeontx_ethdev.c b/drivers/net/octeontx/octeontx_ethdev.c
index ff2e22cd44..10eeeac642 100644
--- a/drivers/net/octeontx/octeontx_ethdev.c
+++ b/drivers/net/octeontx/octeontx_ethdev.c
@@ -478,7 +478,7 @@ octeontx_dev_configure(struct rte_eth_dev *dev)
 	return 0;
 }
 
-static void
+static int
 octeontx_dev_close(struct rte_eth_dev *dev)
 {
 	struct octeontx_txq *txq = NULL;
@@ -517,6 +517,8 @@ octeontx_dev_close(struct rte_eth_dev *dev)
 
 	dev->tx_pkt_burst = NULL;
 	dev->rx_pkt_burst = NULL;
+
+	return 0;
 }
 
 static int
diff --git a/drivers/net/octeontx2/otx2_ethdev.c b/drivers/net/octeontx2/otx2_ethdev.c
index 51a6f99877..abe5f03628 100644
--- a/drivers/net/octeontx2/otx2_ethdev.c
+++ b/drivers/net/octeontx2/otx2_ethdev.c
@@ -2222,7 +2222,7 @@ otx2_nix_dev_start(struct rte_eth_dev *eth_dev)
 }
 
 static int otx2_nix_dev_reset(struct rte_eth_dev *eth_dev);
-static void otx2_nix_dev_close(struct rte_eth_dev *eth_dev);
+static int otx2_nix_dev_close(struct rte_eth_dev *eth_dev);
 
 /* Initialize and register driver with DPDK Application */
 static const struct eth_dev_ops otx2_eth_dev_ops = {
@@ -2635,10 +2635,11 @@ otx2_eth_dev_uninit(struct rte_eth_dev *eth_dev, bool mbox_close)
 	return 0;
 }
 
-static void
+static int
 otx2_nix_dev_close(struct rte_eth_dev *eth_dev)
 {
 	otx2_eth_dev_uninit(eth_dev, true);
+	return 0;
 }
 
 static int
diff --git a/drivers/net/pcap/rte_eth_pcap.c b/drivers/net/pcap/rte_eth_pcap.c
index 668cbd1fc7..76e704a65a 100644
--- a/drivers/net/pcap/rte_eth_pcap.c
+++ b/drivers/net/pcap/rte_eth_pcap.c
@@ -728,7 +728,7 @@ eth_stats_reset(struct rte_eth_dev *dev)
 	return 0;
 }
 
-static void
+static int
 eth_dev_close(struct rte_eth_dev *dev)
 {
 	unsigned int i;
@@ -748,6 +748,7 @@ eth_dev_close(struct rte_eth_dev *dev)
 		}
 	}
 
+	return 0;
 }
 
 static void
diff --git a/drivers/net/pfe/pfe_ethdev.c b/drivers/net/pfe/pfe_ethdev.c
index 97356d2b0b..187a0019ff 100644
--- a/drivers/net/pfe/pfe_ethdev.c
+++ b/drivers/net/pfe/pfe_ethdev.c
@@ -398,14 +398,14 @@ pfe_eth_exit(struct rte_eth_dev *dev, struct pfe *pfe)
 	pfe->nb_devs--;
 }
 
-static void
+static int
 pfe_eth_close(struct rte_eth_dev *dev)
 {
 	if (!dev)
-		return;
+		return -1;
 
 	if (!g_pfe)
-		return;
+		return -1;
 
 	pfe_eth_exit(dev, g_pfe);
 
@@ -415,6 +415,8 @@ pfe_eth_close(struct rte_eth_dev *dev)
 		rte_free(g_pfe);
 		g_pfe = NULL;
 	}
+
+	return 0;
 }
 
 static int
diff --git a/drivers/net/qede/qede_ethdev.c b/drivers/net/qede/qede_ethdev.c
index 59f1746ee9..9535df183f 100644
--- a/drivers/net/qede/qede_ethdev.c
+++ b/drivers/net/qede/qede_ethdev.c
@@ -1542,7 +1542,7 @@ static void qede_poll_sp_sb_cb(void *param)
 	}
 }
 
-static void qede_dev_close(struct rte_eth_dev *eth_dev)
+static int qede_dev_close(struct rte_eth_dev *eth_dev)
 {
 	struct rte_pci_device *pci_dev = RTE_ETH_DEV_TO_PCI(eth_dev);
 	struct qede_dev *qdev = QEDE_INIT_QDEV(eth_dev);
@@ -1586,6 +1586,8 @@ static void qede_dev_close(struct rte_eth_dev *eth_dev)
 
 	if (ECORE_IS_CMT(edev))
 		rte_eal_alarm_cancel(qede_poll_sp_sb_cb, (void *)eth_dev);
+
+	return 0;
 }
 
 static int
diff --git a/drivers/net/sfc/sfc_ethdev.c b/drivers/net/sfc/sfc_ethdev.c
index 1a58e0df84..eaa48627d6 100644
--- a/drivers/net/sfc/sfc_ethdev.c
+++ b/drivers/net/sfc/sfc_ethdev.c
@@ -318,7 +318,7 @@ sfc_dev_set_link_down(struct rte_eth_dev *dev)
 	return 0;
 }
 
-static void
+static int
 sfc_dev_close(struct rte_eth_dev *dev)
 {
 	struct sfc_adapter *sa = sfc_adapter_by_eth_dev(dev);
@@ -364,6 +364,8 @@ sfc_dev_close(struct rte_eth_dev *dev)
 
 	dev->process_private = NULL;
 	free(sa);
+
+	return 0;
 }
 
 static int
diff --git a/drivers/net/softnic/rte_eth_softnic.c b/drivers/net/softnic/rte_eth_softnic.c
index e453fd1486..038abd98da 100644
--- a/drivers/net/softnic/rte_eth_softnic.c
+++ b/drivers/net/softnic/rte_eth_softnic.c
@@ -201,10 +201,10 @@ pmd_dev_stop(struct rte_eth_dev *dev)
 	softnic_mtr_free(p);
 }
 
-static void
+static int
 pmd_dev_close(struct rte_eth_dev *dev __rte_unused)
 {
-	return;
+	return 0;
 }
 
 static int
diff --git a/drivers/net/szedata2/rte_eth_szedata2.c b/drivers/net/szedata2/rte_eth_szedata2.c
index a17c53577c..daa40cfd6c 100644
--- a/drivers/net/szedata2/rte_eth_szedata2.c
+++ b/drivers/net/szedata2/rte_eth_szedata2.c
@@ -1155,7 +1155,7 @@ eth_tx_queue_release(void *q)
 	}
 }
 
-static void
+static int
 eth_dev_close(struct rte_eth_dev *dev)
 {
 	struct pmd_internals *internals = dev->data->dev_private;
@@ -1180,6 +1180,8 @@ eth_dev_close(struct rte_eth_dev *dev)
 
 	rte_free(dev->data->mac_addrs);
 	dev->data->mac_addrs = NULL;
+
+	return 0;
 }
 
 static int
diff --git a/drivers/net/tap/rte_eth_tap.c b/drivers/net/tap/rte_eth_tap.c
index df16aa4ea7..2d1e4e365a 100644
--- a/drivers/net/tap/rte_eth_tap.c
+++ b/drivers/net/tap/rte_eth_tap.c
@@ -1067,7 +1067,7 @@ tap_stats_reset(struct rte_eth_dev *dev)
 	return 0;
 }
 
-static void
+static int
 tap_dev_close(struct rte_eth_dev *dev)
 {
 	int i;
@@ -1116,6 +1116,8 @@ tap_dev_close(struct rte_eth_dev *dev)
 	 * Since TUN device has no more opened file descriptors
 	 * it will be removed from kernel
 	 */
+
+	return 0;
 }
 
 static void
diff --git a/drivers/net/thunderx/nicvf_ethdev.c b/drivers/net/thunderx/nicvf_ethdev.c
index 83d9488360..3d7348771a 100644
--- a/drivers/net/thunderx/nicvf_ethdev.c
+++ b/drivers/net/thunderx/nicvf_ethdev.c
@@ -1852,7 +1852,7 @@ nicvf_vf_stop(struct rte_eth_dev *dev, struct nicvf *nic, bool cleanup)
 	}
 }
 
-static void
+static int
 nicvf_dev_close(struct rte_eth_dev *dev)
 {
 	size_t i;
@@ -1869,6 +1869,8 @@ nicvf_dev_close(struct rte_eth_dev *dev)
 
 		nicvf_periodic_alarm_stop(nicvf_vf_interrupt, nic->snicvf[i]);
 	}
+
+	return 0;
 }
 
 static int
diff --git a/drivers/net/vhost/rte_eth_vhost.c b/drivers/net/vhost/rte_eth_vhost.c
index ce32be9ce3..f67f4db812 100644
--- a/drivers/net/vhost/rte_eth_vhost.c
+++ b/drivers/net/vhost/rte_eth_vhost.c
@@ -1164,7 +1164,7 @@ eth_dev_stop(struct rte_eth_dev *dev)
 	update_queuing_status(dev);
 }
 
-static void
+static int
 eth_dev_close(struct rte_eth_dev *dev)
 {
 	struct pmd_internal *internal;
@@ -1173,7 +1173,7 @@ eth_dev_close(struct rte_eth_dev *dev)
 
 	internal = dev->data->dev_private;
 	if (!internal)
-		return;
+		return 0;
 
 	eth_dev_stop(dev);
 
@@ -1201,6 +1201,8 @@ eth_dev_close(struct rte_eth_dev *dev)
 
 	rte_free(vring_states[dev->data->port_id]);
 	vring_states[dev->data->port_id] = NULL;
+
+	return 0;
 }
 
 static int
diff --git a/drivers/net/virtio/virtio_ethdev.c b/drivers/net/virtio/virtio_ethdev.c
index 013a2904e6..0787337a01 100644
--- a/drivers/net/virtio/virtio_ethdev.c
+++ b/drivers/net/virtio/virtio_ethdev.c
@@ -704,7 +704,7 @@ virtio_alloc_queues(struct rte_eth_dev *dev)
 
 static void virtio_queues_unbind_intr(struct rte_eth_dev *dev);
 
-static void
+static int
 virtio_dev_close(struct rte_eth_dev *dev)
 {
 	struct virtio_hw *hw = dev->data->dev_private;
@@ -713,7 +713,7 @@ virtio_dev_close(struct rte_eth_dev *dev)
 	PMD_INIT_LOG(DEBUG, "virtio_dev_close");
 
 	if (!hw->opened)
-		return;
+		return 0;
 	hw->opened = false;
 
 	/* reset the NIC */
@@ -743,6 +743,8 @@ virtio_dev_close(struct rte_eth_dev *dev)
 		if (!hw->modern)
 			rte_pci_ioport_unmap(VTPCI_IO(hw));
 	}
+
+	return 0;
 }
 
 static int
diff --git a/drivers/net/vmxnet3/vmxnet3_ethdev.c b/drivers/net/vmxnet3/vmxnet3_ethdev.c
index fa80e75a5b..17fed0ed6e 100644
--- a/drivers/net/vmxnet3/vmxnet3_ethdev.c
+++ b/drivers/net/vmxnet3/vmxnet3_ethdev.c
@@ -64,7 +64,7 @@ static int eth_vmxnet3_dev_uninit(struct rte_eth_dev *eth_dev);
 static int vmxnet3_dev_configure(struct rte_eth_dev *dev);
 static int vmxnet3_dev_start(struct rte_eth_dev *dev);
 static void vmxnet3_dev_stop(struct rte_eth_dev *dev);
-static void vmxnet3_dev_close(struct rte_eth_dev *dev);
+static int vmxnet3_dev_close(struct rte_eth_dev *dev);
 static void vmxnet3_dev_set_rxmode(struct vmxnet3_hw *hw, uint32_t feature, int set);
 static int vmxnet3_dev_promiscuous_enable(struct rte_eth_dev *dev);
 static int vmxnet3_dev_promiscuous_disable(struct rte_eth_dev *dev);
@@ -888,13 +888,15 @@ vmxnet3_free_queues(struct rte_eth_dev *dev)
 /*
  * Reset and stop device.
  */
-static void
+static int
 vmxnet3_dev_close(struct rte_eth_dev *dev)
 {
 	PMD_INIT_FUNC_TRACE();
 
 	vmxnet3_dev_stop(dev);
 	vmxnet3_free_queues(dev);
+
+	return 0;
 }
 
 static void
diff --git a/lib/librte_ethdev/rte_ethdev_driver.h b/lib/librte_ethdev/rte_ethdev_driver.h
index 23cc1e0959..6abfd58d2f 100644
--- a/lib/librte_ethdev/rte_ethdev_driver.h
+++ b/lib/librte_ethdev/rte_ethdev_driver.h
@@ -42,7 +42,7 @@ typedef int  (*eth_dev_set_link_up_t)(struct rte_eth_dev *dev);
 typedef int  (*eth_dev_set_link_down_t)(struct rte_eth_dev *dev);
 /**< @internal Function used to link down a configured Ethernet device. */
 
-typedef void (*eth_dev_close_t)(struct rte_eth_dev *dev);
+typedef int (*eth_dev_close_t)(struct rte_eth_dev *dev);
 /**< @internal Function used to close a configured Ethernet device. */
 
 typedef int (*eth_dev_reset_t)(struct rte_eth_dev *dev);
-- 
2.28.0

[dpdk-dev] [PATCH v2 03/25] net/af_packet: release port upon close

[Thomas Monjalon]

The flag RTE_ETH_DEV_CLOSE_REMOVE is set so all port resources
can be freed by rte_eth_dev_close().

Freeing of private port resources is moved
from the ".remove(device)" to the ".dev_close(port)" operation.

Signed-off-by: Thomas Monjalon <thomas@monjalon.net>
Reviewed-by: Ferruh Yigit <ferruh.yigit@intel.com>
---
 drivers/net/af_packet/rte_eth_af_packet.c | 56 ++++++++++++-----------
 1 file changed, 29 insertions(+), 27 deletions(-)

diff --git a/drivers/net/af_packet/rte_eth_af_packet.c b/drivers/net/af_packet/rte_eth_af_packet.c
index 7d0ff1cbb3..12c202cba4 100644
--- a/drivers/net/af_packet/rte_eth_af_packet.c
+++ b/drivers/net/af_packet/rte_eth_af_packet.c
@@ -377,8 +377,32 @@ eth_stats_reset(struct rte_eth_dev *dev)
 }
 
 static int
-eth_dev_close(struct rte_eth_dev *dev __rte_unused)
+eth_dev_close(struct rte_eth_dev *dev)
 {
+	struct pmd_internals *internals;
+	struct tpacket_req *req;
+	unsigned int q;
+
+	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
+		return 0;
+
+	PMD_LOG(INFO, "Closing AF_PACKET ethdev on NUMA socket %u",
+		rte_socket_id());
+
+	internals = dev->data->dev_private;
+	req = &internals->req;
+	for (q = 0; q < internals->nb_queues; q++) {
+		munmap(internals->rx_queue[q].map,
+			2 * req->tp_block_size * req->tp_block_nr);
+		rte_free(internals->rx_queue[q].rd);
+		rte_free(internals->tx_queue[q].rd);
+	}
+	free(internals->if_name);
+	rte_free(internals->rx_queue);
+	rte_free(internals->tx_queue);
+
+	/* mac_addrs must not be freed alone because part of dev_private */
+	dev->data->mac_addrs = NULL;
 	return 0;
 }
 
@@ -835,6 +859,7 @@ rte_pmd_init_internals(struct rte_vdev_device *dev,
 	data->nb_tx_queues = (uint16_t)nb_queues;
 	data->dev_link = pmd_link;
 	data->mac_addrs = &(*internals)->eth_addr;
+	data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
 
 	(*eth_dev)->dev_ops = &ops;
 
@@ -1033,13 +1058,7 @@ rte_pmd_af_packet_probe(struct rte_vdev_device *dev)
 static int
 rte_pmd_af_packet_remove(struct rte_vdev_device *dev)
 {
-	struct rte_eth_dev *eth_dev = NULL;
-	struct pmd_internals *internals;
-	struct tpacket_req *req;
-	unsigned q;
-
-	PMD_LOG(INFO, "Closing AF_PACKET ethdev on numa socket %u",
-		rte_socket_id());
+	struct rte_eth_dev *eth_dev;
 
 	if (dev == NULL)
 		return -1;
@@ -1047,26 +1066,9 @@ rte_pmd_af_packet_remove(struct rte_vdev_device *dev)
 	/* find the ethdev entry */
 	eth_dev = rte_eth_dev_allocated(rte_vdev_device_name(dev));
 	if (eth_dev == NULL)
-		return -1;
-
-	/* mac_addrs must not be freed alone because part of dev_private */
-	eth_dev->data->mac_addrs = NULL;
-
-	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
-		return rte_eth_dev_release_port(eth_dev);
-
-	internals = eth_dev->data->dev_private;
-	req = &internals->req;
-	for (q = 0; q < internals->nb_queues; q++) {
-		munmap(internals->rx_queue[q].map,
-			2 * req->tp_block_size * req->tp_block_nr);
-		rte_free(internals->rx_queue[q].rd);
-		rte_free(internals->tx_queue[q].rd);
-	}
-	free(internals->if_name);
-	rte_free(internals->rx_queue);
-	rte_free(internals->tx_queue);
+		return 0; /* port already released */
 
+	eth_dev_close(eth_dev);
 	rte_eth_dev_release_port(eth_dev);
 
 	return 0;
-- 
2.28.0

[dpdk-dev] [PATCH v2 04/25] net/atlantic: release port upon close

[Thomas Monjalon]

The flag RTE_ETH_DEV_CLOSE_REMOVE is set so all port resources
can be freed by rte_eth_dev_close().

Freeing of private port resources is moved
from the ".remove(device)" to the ".dev_close(port)" operation.

Signed-off-by: Thomas Monjalon <thomas@monjalon.net>
Acked-by: Igor Russkikh <irusskikh@marvell.com>
---
 drivers/net/atlantic/atl_ethdev.c | 62 ++++++++++++-------------------
 1 file changed, 24 insertions(+), 38 deletions(-)

diff --git a/drivers/net/atlantic/atl_ethdev.c b/drivers/net/atlantic/atl_ethdev.c
index 254758fd66..d3babeff94 100644
--- a/drivers/net/atlantic/atl_ethdev.c
+++ b/drivers/net/atlantic/atl_ethdev.c
@@ -15,8 +15,6 @@
 #include "hw_atl/hw_atl_b0_internal.h"
 
 static int eth_atl_dev_init(struct rte_eth_dev *eth_dev);
-static int eth_atl_dev_uninit(struct rte_eth_dev *eth_dev);
-
 static int  atl_dev_configure(struct rte_eth_dev *dev);
 static int  atl_dev_start(struct rte_eth_dev *dev);
 static void atl_dev_stop(struct rte_eth_dev *dev);
@@ -382,6 +380,8 @@ eth_atl_dev_init(struct rte_eth_dev *eth_dev)
 	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
 		return 0;
 
+	eth_dev->data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
+
 	/* Vendor and Device ID need to be set before init of shared code */
 	hw->device_id = pci_dev->id.device_id;
 	hw->vendor_id = pci_dev->id.vendor_id;
@@ -442,40 +442,6 @@ eth_atl_dev_init(struct rte_eth_dev *eth_dev)
 	return err;
 }
 
-static int
-eth_atl_dev_uninit(struct rte_eth_dev *eth_dev)
-{
-	struct rte_pci_device *pci_dev = RTE_ETH_DEV_TO_PCI(eth_dev);
-	struct rte_intr_handle *intr_handle = &pci_dev->intr_handle;
-	struct aq_hw_s *hw;
-
-	PMD_INIT_FUNC_TRACE();
-
-	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
-		return -EPERM;
-
-	hw = ATL_DEV_PRIVATE_TO_HW(eth_dev->data->dev_private);
-
-	if (hw->adapter_stopped == 0)
-		atl_dev_close(eth_dev);
-
-	eth_dev->dev_ops = NULL;
-	eth_dev->rx_pkt_burst = NULL;
-	eth_dev->tx_pkt_burst = NULL;
-
-	/* disable uio intr before callback unregister */
-	rte_intr_disable(intr_handle);
-	rte_intr_callback_unregister(intr_handle,
-				     atl_dev_interrupt_handler, eth_dev);
-
-	rte_free(eth_dev->data->mac_addrs);
-	eth_dev->data->mac_addrs = NULL;
-
-	pthread_mutex_destroy(&hw->mbox_mutex);
-
-	return 0;
-}
-
 static int
 eth_atl_pci_probe(struct rte_pci_driver *pci_drv __rte_unused,
 	struct rte_pci_device *pci_dev)
@@ -487,7 +453,7 @@ eth_atl_pci_probe(struct rte_pci_driver *pci_drv __rte_unused,
 static int
 eth_atl_pci_remove(struct rte_pci_device *pci_dev)
 {
-	return rte_eth_dev_pci_generic_remove(pci_dev, eth_atl_dev_uninit);
+	return rte_eth_dev_pci_generic_remove(pci_dev, atl_dev_close);
 }
 
 static int
@@ -722,12 +688,32 @@ atl_dev_set_link_down(struct rte_eth_dev *dev)
 static int
 atl_dev_close(struct rte_eth_dev *dev)
 {
+	struct rte_pci_device *pci_dev = RTE_ETH_DEV_TO_PCI(dev);
+	struct rte_intr_handle *intr_handle = &pci_dev->intr_handle;
+	struct aq_hw_s *hw;
+
 	PMD_INIT_FUNC_TRACE();
 
+	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
+		return 0;
+
+	hw = ATL_DEV_PRIVATE_TO_HW(dev->data->dev_private);
+
 	atl_dev_stop(dev);
 
 	atl_free_queues(dev);
 
+	dev->dev_ops = NULL;
+	dev->rx_pkt_burst = NULL;
+	dev->tx_pkt_burst = NULL;
+
+	/* disable uio intr before callback unregister */
+	rte_intr_disable(intr_handle);
+	rte_intr_callback_unregister(intr_handle,
+				     atl_dev_interrupt_handler, dev);
+
+	pthread_mutex_destroy(&hw->mbox_mutex);
+
 	return 0;
 }
 
@@ -736,7 +722,7 @@ atl_dev_reset(struct rte_eth_dev *dev)
 {
 	int ret;
 
-	ret = eth_atl_dev_uninit(dev);
+	ret = atl_dev_close(dev);
 	if (ret)
 		return ret;
 
-- 
2.28.0

[dpdk-dev] [PATCH v2 05/25] net/axgbe: release port upon close

[Thomas Monjalon]

The flag RTE_ETH_DEV_CLOSE_REMOVE is set so all port resources
can be freed by rte_eth_dev_close().

Freeing of private port resources is moved
from the ".remove(device)" to the ".dev_close(port)" operation.
The ".dev_close" callback is also called as part of the ".remove" one.

Signed-off-by: Thomas Monjalon <thomas@monjalon.net>
---
 drivers/net/axgbe/axgbe_ethdev.c | 14 +++-----------
 1 file changed, 3 insertions(+), 11 deletions(-)

diff --git a/drivers/net/axgbe/axgbe_ethdev.c b/drivers/net/axgbe/axgbe_ethdev.c
index 458d64bcef..2dd64180ad 100644
--- a/drivers/net/axgbe/axgbe_ethdev.c
+++ b/drivers/net/axgbe/axgbe_ethdev.c
@@ -11,7 +11,6 @@
 #include "rte_time.h"
 
 static int eth_axgbe_dev_init(struct rte_eth_dev *eth_dev);
-static int eth_axgbe_dev_uninit(struct rte_eth_dev *eth_dev);
 static int  axgbe_dev_configure(struct rte_eth_dev *dev);
 static int  axgbe_dev_start(struct rte_eth_dev *dev);
 static void axgbe_dev_stop(struct rte_eth_dev *dev);
@@ -409,14 +408,6 @@ axgbe_dev_stop(struct rte_eth_dev *dev)
 	rte_bit_relaxed_set32(AXGBE_DOWN, &pdata->dev_state);
 }
 
-/* Clear all resources like TX/RX queues. */
-static int
-axgbe_dev_close(struct rte_eth_dev *dev)
-{
-	axgbe_dev_clear_queues(dev);
-	return 0;
-}
-
 static int
 axgbe_dev_promiscuous_enable(struct rte_eth_dev *dev)
 {
@@ -1968,6 +1959,7 @@ eth_axgbe_dev_init(struct rte_eth_dev *eth_dev)
 	int ret;
 
 	eth_dev->dev_ops = &axgbe_eth_dev_ops;
+	eth_dev->data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
 
 	eth_dev->rx_descriptor_status = axgbe_dev_rx_descriptor_status;
 	eth_dev->tx_descriptor_status = axgbe_dev_tx_descriptor_status;
@@ -2133,7 +2125,7 @@ eth_axgbe_dev_init(struct rte_eth_dev *eth_dev)
 }
 
 static int
-eth_axgbe_dev_uninit(struct rte_eth_dev *eth_dev)
+axgbe_dev_close(struct rte_eth_dev *eth_dev)
 {
 	struct rte_pci_device *pci_dev;
 
@@ -2166,7 +2158,7 @@ static int eth_axgbe_pci_probe(struct rte_pci_driver *pci_drv __rte_unused,
 
 static int eth_axgbe_pci_remove(struct rte_pci_device *pci_dev)
 {
-	return rte_eth_dev_pci_generic_remove(pci_dev, eth_axgbe_dev_uninit);
+	return rte_eth_dev_pci_generic_remove(pci_dev, axgbe_dev_close);
 }
 
 static struct rte_pci_driver rte_axgbe_pmd = {
-- 
2.28.0

[dpdk-dev] [PATCH v2 06/25] net/bnx2x: release port upon close

[Thomas Monjalon]

From: Rasesh Mody <rmody@marvell.com>

Set RTE_ETH_DEV_CLOSE_REMOVE upon probe so all the private resources
for the port can be freed by rte_eth_dev_close(). With this change the
private port resources are released in the .dev_close callback.

Signed-off-by: Rasesh Mody <rmody@marvell.com>
---
 drivers/net/bnx2x/bnx2x_ethdev.c | 17 +++++++++++++++--
 1 file changed, 15 insertions(+), 2 deletions(-)

diff --git a/drivers/net/bnx2x/bnx2x_ethdev.c b/drivers/net/bnx2x/bnx2x_ethdev.c
index 73217c2138..8dc46384af 100644
--- a/drivers/net/bnx2x/bnx2x_ethdev.c
+++ b/drivers/net/bnx2x/bnx2x_ethdev.c
@@ -287,6 +287,10 @@ bnx2x_dev_close(struct rte_eth_dev *dev)
 
 	PMD_INIT_FUNC_TRACE(sc);
 
+	/* only close in case of the primary process */
+	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
+		return 0;
+
 	if (IS_VF(sc))
 		bnx2x_vf_close(sc);
 
@@ -296,6 +300,9 @@ bnx2x_dev_close(struct rte_eth_dev *dev)
 	/* free ilt */
 	bnx2x_free_ilt_mem(sc);
 
+	/* mac_addrs must not be freed alone because part of dev_private */
+	dev->data->mac_addrs = NULL;
+
 	return 0;
 }
 
@@ -728,6 +735,11 @@ bnx2x_common_dev_init(struct rte_eth_dev *eth_dev, int is_vf)
 			goto out;
 	}
 
+	/* Pass the information to the rte_eth_dev_close() that it should also
+	 * release the private port resources.
+	 */
+	eth_dev->data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
+
 	return 0;
 
 out:
@@ -755,8 +767,9 @@ eth_bnx2xvf_dev_init(struct rte_eth_dev *eth_dev)
 
 static int eth_bnx2x_dev_uninit(struct rte_eth_dev *eth_dev)
 {
-	/* mac_addrs must not be freed alone because part of dev_private */
-	eth_dev->data->mac_addrs = NULL;
+	struct bnx2x_softc *sc = eth_dev->data->dev_private;
+	PMD_INIT_FUNC_TRACE(sc);
+	bnx2x_dev_close(eth_dev);
 	return 0;
 }
 
-- 
2.28.0

[dpdk-dev] [PATCH v2 07/25] net/bonding: release port upon close

[Thomas Monjalon]

The flag RTE_ETH_DEV_CLOSE_REMOVE is set so all port resources
can be freed by rte_eth_dev_close().

Freeing of private port resources is moved
from the ".remove(device)" to the ".dev_close(port)" operation.

Signed-off-by: Thomas Monjalon <thomas@monjalon.net>
---
 drivers/net/bonding/rte_eth_bond_pmd.c | 34 ++++++++++++--------------
 1 file changed, 16 insertions(+), 18 deletions(-)

diff --git a/drivers/net/bonding/rte_eth_bond_pmd.c b/drivers/net/bonding/rte_eth_bond_pmd.c
index dff835ef92..a9ed5d8e12 100644
--- a/drivers/net/bonding/rte_eth_bond_pmd.c
+++ b/drivers/net/bonding/rte_eth_bond_pmd.c
@@ -2103,6 +2103,9 @@ bond_ethdev_close(struct rte_eth_dev *dev)
 	int skipped = 0;
 	struct rte_flow_error ferror;
 
+	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
+		return 0;
+
 	RTE_BOND_LOG(INFO, "Closing bonded device %s", dev->device->name);
 	while (internals->slave_count != skipped) {
 		uint16_t port_id = internals->slaves[skipped].port_id;
@@ -2119,6 +2122,17 @@ bond_ethdev_close(struct rte_eth_dev *dev)
 	bond_flow_ops.flush(dev, &ferror);
 	bond_ethdev_free_queues(dev);
 	rte_bitmap_reset(internals->vlan_filter_bmp);
+	rte_bitmap_free(internals->vlan_filter_bmp);
+	rte_free(internals->vlan_filter_bmpmem);
+
+	/* Try to release mempool used in mode6. If the bond
+	 * device is not mode6, free the NULL is not problem.
+	 */
+	rte_mempool_free(internals->mode6.mempool);
+
+	dev->dev_ops = NULL;
+	dev->rx_pkt_burst = NULL;
+	dev->tx_pkt_burst = NULL;
 
 	return 0;
 }
@@ -3195,6 +3209,7 @@ bond_alloc(struct rte_vdev_device *dev, uint8_t mode)
 	}
 
 	internals = eth_dev->data->dev_private;
+	eth_dev->data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
 	eth_dev->data->nb_rx_queues = (uint16_t)1;
 	eth_dev->data->nb_tx_queues = (uint16_t)1;
 
@@ -3414,14 +3429,10 @@ bond_remove(struct rte_vdev_device *dev)
 	name = rte_vdev_device_name(dev);
 	RTE_BOND_LOG(INFO, "Uninitializing pmd_bond for %s", name);
 
-	/* now free all data allocation - for eth_dev structure,
-	 * dummy pci driver and internal (private) data
-	 */
-
 	/* find an ethdev entry */
 	eth_dev = rte_eth_dev_allocated(name);
 	if (eth_dev == NULL)
-		return -ENODEV;
+		return 0; /* port already released */
 
 	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
 		return rte_eth_dev_release_port(eth_dev);
@@ -3436,19 +3447,6 @@ bond_remove(struct rte_vdev_device *dev)
 		bond_ethdev_stop(eth_dev);
 		bond_ethdev_close(eth_dev);
 	}
-
-	eth_dev->dev_ops = NULL;
-	eth_dev->rx_pkt_burst = NULL;
-	eth_dev->tx_pkt_burst = NULL;
-
-	internals = eth_dev->data->dev_private;
-	/* Try to release mempool used in mode6. If the bond
-	 * device is not mode6, free the NULL is not problem.
-	 */
-	rte_mempool_free(internals->mode6.mempool);
-	rte_bitmap_free(internals->vlan_filter_bmp);
-	rte_free(internals->vlan_filter_bmpmem);
-
 	rte_eth_dev_release_port(eth_dev);
 
 	return 0;
-- 
2.28.0

[dpdk-dev] [PATCH v2 08/25] net/failsafe: release port upon close

[Thomas Monjalon]

The flag RTE_ETH_DEV_CLOSE_REMOVE is set so all port resources
can be freed by rte_eth_dev_close().

Freeing of private port resources is moved
from the ".remove(device)" to the ".dev_close(port)" operation.

Signed-off-by: Thomas Monjalon <thomas@monjalon.net>
---
 drivers/net/failsafe/failsafe.c         | 25 ++--------
 drivers/net/failsafe/failsafe_ops.c     | 65 ++++++++++++++++---------
 drivers/net/failsafe/failsafe_private.h |  1 +
 3 files changed, 46 insertions(+), 45 deletions(-)

diff --git a/drivers/net/failsafe/failsafe.c b/drivers/net/failsafe/failsafe.c
index 4a4b7ceab6..44d47e8f72 100644
--- a/drivers/net/failsafe/failsafe.c
+++ b/drivers/net/failsafe/failsafe.c
@@ -60,12 +60,6 @@ fs_sub_device_alloc(struct rte_eth_dev *dev,
 	return 0;
 }
 
-static void
-fs_sub_device_free(struct rte_eth_dev *dev)
-{
-	rte_free(PRIV(dev)->subs);
-}
-
 static void fs_hotplug_alarm(void *arg);
 
 int
@@ -186,6 +180,7 @@ fs_eth_dev_create(struct rte_vdev_device *vdev)
 		ERROR("Unable to allocate rte_eth_dev");
 		return -1;
 	}
+	dev->data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
 	priv = PRIV(dev);
 	priv->data = dev->data;
 	priv->rxp = FS_RX_PROXY_INIT;
@@ -285,7 +280,7 @@ fs_eth_dev_create(struct rte_vdev_device *vdev)
 free_args:
 	failsafe_args_free(dev);
 free_subs:
-	fs_sub_device_free(dev);
+	rte_free(PRIV(dev)->subs);
 free_dev:
 	/* mac_addrs must not be freed alone because part of dev_private */
 	dev->data->mac_addrs = NULL;
@@ -301,20 +296,8 @@ fs_rte_eth_free(const char *name)
 
 	dev = rte_eth_dev_allocated(name);
 	if (dev == NULL)
-		return -ENODEV;
-	rte_eth_dev_callback_unregister(RTE_ETH_ALL, RTE_ETH_EVENT_NEW,
-					failsafe_eth_new_event_callback, dev);
-	ret = failsafe_eal_uninit(dev);
-	if (ret)
-		ERROR("Error while uninitializing sub-EAL");
-	failsafe_args_free(dev);
-	fs_sub_device_free(dev);
-	ret = pthread_mutex_destroy(&PRIV(dev)->hotplug_mutex);
-	if (ret)
-		ERROR("Error while destroying hotplug mutex");
-	rte_free(PRIV(dev)->mcast_addrs);
-	/* mac_addrs must not be freed alone because part of dev_private */
-	dev->data->mac_addrs = NULL;
+		return 0; /* port already released */
+	ret = failsafe_eth_dev_close(dev);
 	rte_eth_dev_release_port(dev);
 	return ret;
 }
diff --git a/drivers/net/failsafe/failsafe_ops.c b/drivers/net/failsafe/failsafe_ops.c
index 93ebd09114..0ce7dfc8a6 100644
--- a/drivers/net/failsafe/failsafe_ops.c
+++ b/drivers/net/failsafe/failsafe_ops.c
@@ -239,29 +239,6 @@ fs_dev_set_link_down(struct rte_eth_dev *dev)
 	return 0;
 }
 
-static void fs_dev_free_queues(struct rte_eth_dev *dev);
-static int
-fs_dev_close(struct rte_eth_dev *dev)
-{
-	struct sub_device *sdev;
-	uint8_t i;
-
-	fs_lock(dev, 0);
-	failsafe_hotplug_alarm_cancel(dev);
-	if (PRIV(dev)->state == DEV_STARTED)
-		dev->dev_ops->dev_stop(dev);
-	PRIV(dev)->state = DEV_ACTIVE - 1;
-	FOREACH_SUBDEV_STATE(sdev, i, dev, DEV_ACTIVE) {
-		DEBUG("Closing sub_device %d", i);
-		failsafe_eth_dev_unregister_callbacks(sdev);
-		rte_eth_dev_close(PORT_ID(sdev));
-		sdev->state = DEV_ACTIVE - 1;
-	}
-	fs_dev_free_queues(dev);
-	fs_unlock(dev, 0);
-	return 0;
-}
-
 static int
 fs_rx_queue_stop(struct rte_eth_dev *dev, uint16_t rx_queue_id)
 {
@@ -656,6 +633,46 @@ fs_dev_free_queues(struct rte_eth_dev *dev)
 	dev->data->nb_tx_queues = 0;
 }
 
+int
+failsafe_eth_dev_close(struct rte_eth_dev *dev)
+{
+	struct sub_device *sdev;
+	uint8_t i;
+	int ret;
+
+	fs_lock(dev, 0);
+	failsafe_hotplug_alarm_cancel(dev);
+	if (PRIV(dev)->state == DEV_STARTED)
+		dev->dev_ops->dev_stop(dev);
+	PRIV(dev)->state = DEV_ACTIVE - 1;
+	FOREACH_SUBDEV_STATE(sdev, i, dev, DEV_ACTIVE) {
+		DEBUG("Closing sub_device %d", i);
+		failsafe_eth_dev_unregister_callbacks(sdev);
+		rte_eth_dev_close(PORT_ID(sdev));
+		sdev->state = DEV_ACTIVE - 1;
+	}
+	rte_eth_dev_callback_unregister(RTE_ETH_ALL, RTE_ETH_EVENT_NEW,
+					failsafe_eth_new_event_callback, dev);
+	if (rte_eal_process_type() != RTE_PROC_PRIMARY) {
+		fs_unlock(dev, 0);
+		return 0;
+	}
+	fs_dev_free_queues(dev);
+	ret = failsafe_eal_uninit(dev);
+	if (ret)
+		ERROR("Error while uninitializing sub-EAL");
+	failsafe_args_free(dev);
+	rte_free(PRIV(dev)->subs);
+	rte_free(PRIV(dev)->mcast_addrs);
+	/* mac_addrs must not be freed alone because part of dev_private */
+	dev->data->mac_addrs = NULL;
+	fs_unlock(dev, 0);
+	ret = pthread_mutex_destroy(&PRIV(dev)->hotplug_mutex);
+	if (ret)
+		ERROR("Error while destroying hotplug mutex");
+	return 0;
+}
+
 static int
 fs_promiscuous_enable(struct rte_eth_dev *dev)
 {
@@ -1484,7 +1501,7 @@ const struct eth_dev_ops failsafe_ops = {
 	.dev_stop = fs_dev_stop,
 	.dev_set_link_down = fs_dev_set_link_down,
 	.dev_set_link_up = fs_dev_set_link_up,
-	.dev_close = fs_dev_close,
+	.dev_close = failsafe_eth_dev_close,
 	.promiscuous_enable = fs_promiscuous_enable,
 	.promiscuous_disable = fs_promiscuous_disable,
 	.allmulticast_enable = fs_allmulticast_enable,
diff --git a/drivers/net/failsafe/failsafe_private.h b/drivers/net/failsafe/failsafe_private.h
index 651578a128..6af0ef8471 100644
--- a/drivers/net/failsafe/failsafe_private.h
+++ b/drivers/net/failsafe/failsafe_private.h
@@ -236,6 +236,7 @@ int failsafe_eal_uninit(struct rte_eth_dev *dev);
 
 int failsafe_eth_dev_state_sync(struct rte_eth_dev *dev);
 void failsafe_eth_dev_unregister_callbacks(struct sub_device *sdev);
+int failsafe_eth_dev_close(struct rte_eth_dev *dev);
 void failsafe_dev_remove(struct rte_eth_dev *dev);
 void failsafe_stats_increment(struct rte_eth_stats *to,
 				struct rte_eth_stats *from);
-- 
2.28.0

[dpdk-dev] [PATCH v2 09/25] net/mlx4: release port upon close

[Thomas Monjalon]

The flag RTE_ETH_DEV_CLOSE_REMOVE is set so all port resources
can be freed by rte_eth_dev_close().

Signed-off-by: Thomas Monjalon <thomas@monjalon.net>
---
 drivers/net/mlx4/mlx4.c | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/drivers/net/mlx4/mlx4.c b/drivers/net/mlx4/mlx4.c
index df59314b66..ad7c805d67 100644
--- a/drivers/net/mlx4/mlx4.c
+++ b/drivers/net/mlx4/mlx4.c
@@ -400,6 +400,8 @@ mlx4_dev_close(struct rte_eth_dev *dev)
 		MLX4_ASSERT(priv->ctx == NULL);
 	mlx4_intr_uninstall(priv);
 	memset(priv, 0, sizeof(*priv));
+	/* mac_addrs must not be freed because part of dev_private */
+	dev->data->mac_addrs = NULL;
 	return 0;
 }
 
@@ -1025,6 +1027,7 @@ mlx4_pci_probe(struct rte_pci_driver *pci_drv, struct rte_pci_device *pci_dev)
 			ERROR("can not allocate rte ethdev");
 			goto port_error;
 		}
+		eth_dev->data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
 		eth_dev->data->dev_private = priv;
 		eth_dev->data->mac_addrs = priv->mac;
 		eth_dev->device = &pci_dev->device;
-- 
2.28.0

[dpdk-dev] [PATCH v2 10/25] net/null: release port upon close

[Thomas Monjalon]

The flag RTE_ETH_DEV_CLOSE_REMOVE is set so all port resources
can be freed by rte_eth_dev_close().

Signed-off-by: Thomas Monjalon <thomas@monjalon.net>
---
 drivers/net/null/rte_eth_null.c | 27 +++++++++++++++++++--------
 1 file changed, 19 insertions(+), 8 deletions(-)

diff --git a/drivers/net/null/rte_eth_null.c b/drivers/net/null/rte_eth_null.c
index 0ce073fa4b..84c1d0c951 100644
--- a/drivers/net/null/rte_eth_null.c
+++ b/drivers/net/null/rte_eth_null.c
@@ -458,7 +458,23 @@ eth_mac_address_set(__rte_unused struct rte_eth_dev *dev,
 	return 0;
 }
 
+static int
+eth_dev_close(struct rte_eth_dev *dev)
+{
+	PMD_LOG(INFO, "Closing null ethdev on NUMA socket %u",
+			rte_socket_id());
+
+	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
+		return 0;
+
+	/* mac_addrs must not be freed alone because part of dev_private */
+	dev->data->mac_addrs = NULL;
+
+	return 0;
+}
+
 static const struct eth_dev_ops ops = {
+	.dev_close = eth_dev_close,
 	.dev_start = eth_dev_start,
 	.dev_stop = eth_dev_stop,
 	.dev_configure = eth_dev_configure,
@@ -532,6 +548,7 @@ eth_dev_null_create(struct rte_vdev_device *dev, struct pmd_options *args)
 	data->mac_addrs = &internals->eth_addr;
 	data->promiscuous = 1;
 	data->all_multicast = 1;
+	data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
 
 	eth_dev->dev_ops = &ops;
 
@@ -701,18 +718,12 @@ rte_pmd_null_remove(struct rte_vdev_device *dev)
 	if (!dev)
 		return -EINVAL;
 
-	PMD_LOG(INFO, "Closing null ethdev on numa socket %u",
-			rte_socket_id());
-
 	/* find the ethdev entry */
 	eth_dev = rte_eth_dev_allocated(rte_vdev_device_name(dev));
 	if (eth_dev == NULL)
-		return -1;
-
-	if (rte_eal_process_type() == RTE_PROC_PRIMARY)
-		/* mac_addrs must not be freed alone because part of dev_private */
-		eth_dev->data->mac_addrs = NULL;
+		return 0; /* port already released */
 
+	eth_dev_close(eth_dev);
 	rte_eth_dev_release_port(eth_dev);
 
 	return 0;
-- 
2.28.0

[dpdk-dev] [PATCH v2 11/25] net/octeontx: release port upon close

[Thomas Monjalon]

The flag RTE_ETH_DEV_CLOSE_REMOVE is set so all port resources
can be freed by rte_eth_dev_close().

The callback ".dev_close(port)" is called also
from the ".remove(device)" operation.

Signed-off-by: Thomas Monjalon <thomas@monjalon.net>
---
 drivers/net/octeontx/octeontx_ethdev.c | 11 +++--------
 1 file changed, 3 insertions(+), 8 deletions(-)

diff --git a/drivers/net/octeontx/octeontx_ethdev.c b/drivers/net/octeontx/octeontx_ethdev.c
index 10eeeac642..48ce3e1621 100644
--- a/drivers/net/octeontx/octeontx_ethdev.c
+++ b/drivers/net/octeontx/octeontx_ethdev.c
@@ -509,10 +509,6 @@ octeontx_dev_close(struct rte_eth_dev *dev)
 		rte_free(txq);
 	}
 
-	/* Free MAC address table */
-	rte_free(dev->data->mac_addrs);
-	dev->data->mac_addrs = NULL;
-
 	octeontx_port_close(nic);
 
 	dev->tx_pkt_burst = NULL;
@@ -1378,6 +1374,7 @@ octeontx_create(struct rte_vdev_device *dev, int port, uint8_t evdev,
 	data->promiscuous = 0;
 	data->all_multicast = 0;
 	data->scattered_rx = 0;
+	data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
 
 	/* Get maximum number of supported MAC entries */
 	max_entries = octeontx_bgx_port_mac_entries_get(nic->port_id);
@@ -1465,10 +1462,9 @@ octeontx_remove(struct rte_vdev_device *dev)
 	for (i = 0; i < OCTEONTX_VDEV_DEFAULT_MAX_NR_PORT; i++) {
 		sprintf(octtx_name, "eth_octeontx_%d", i);
 
-		/* reserve an ethdev entry */
 		eth_dev = rte_eth_dev_allocated(octtx_name);
 		if (eth_dev == NULL)
-			return -ENODEV;
+			continue; /* port already released */
 
 		if (rte_eal_process_type() != RTE_PROC_PRIMARY) {
 			rte_eth_dev_release_port(eth_dev);
@@ -1478,9 +1474,8 @@ octeontx_remove(struct rte_vdev_device *dev)
 		nic = octeontx_pmd_priv(eth_dev);
 		rte_event_dev_stop(nic->evdev);
 		PMD_INIT_LOG(INFO, "Closing octeontx device %s", octtx_name);
-
+		octeontx_dev_close(eth_dev);
 		rte_eth_dev_release_port(eth_dev);
-		rte_event_dev_close(nic->evdev);
 	}
 
 	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
-- 
2.28.0

[dpdk-dev] [PATCH v2 12/25] net/pcap: release port upon close

[Thomas Monjalon]

The flag RTE_ETH_DEV_CLOSE_REMOVE is set so all port resources
can be freed by rte_eth_dev_close().

Freeing of private port resources is moved
from the ".remove(device)" to the ".dev_close(port)" operation.

Signed-off-by: Thomas Monjalon <thomas@monjalon.net>
---
 drivers/net/pcap/rte_eth_pcap.c | 29 ++++++++++++++---------------
 1 file changed, 14 insertions(+), 15 deletions(-)

diff --git a/drivers/net/pcap/rte_eth_pcap.c b/drivers/net/pcap/rte_eth_pcap.c
index 76e704a65a..909eef8cce 100644
--- a/drivers/net/pcap/rte_eth_pcap.c
+++ b/drivers/net/pcap/rte_eth_pcap.c
@@ -734,6 +734,14 @@ eth_dev_close(struct rte_eth_dev *dev)
 	unsigned int i;
 	struct pmd_internals *internals = dev->data->dev_private;
 
+	PMD_LOG(INFO, "Closing pcap ethdev on NUMA socket %d",
+			rte_socket_id());
+
+	rte_free(dev->process_private);
+
+	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
+		return 0;
+
 	/* Device wide flag, but cleanup must be performed per queue. */
 	if (internals->infinite_rx) {
 		for (i = 0; i < dev->data->nb_rx_queues; i++) {
@@ -748,6 +756,10 @@ eth_dev_close(struct rte_eth_dev *dev)
 		}
 	}
 
+	if (internals->phy_mac == 0)
+		/* not dynamically allocated, must not be freed */
+		dev->data->mac_addrs = NULL;
+
 	return 0;
 }
 
@@ -1322,6 +1334,7 @@ eth_from_pcaps(struct rte_vdev_device *vdev,
 	else
 		eth_dev->tx_pkt_burst = eth_tx_drop;
 
+	eth_dev->data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
 	rte_eth_dev_probing_finish(eth_dev);
 	return 0;
 }
@@ -1544,30 +1557,16 @@ pmd_pcap_probe(struct rte_vdev_device *dev)
 static int
 pmd_pcap_remove(struct rte_vdev_device *dev)
 {
-	struct pmd_internals *internals = NULL;
 	struct rte_eth_dev *eth_dev = NULL;
 
-	PMD_LOG(INFO, "Closing pcap ethdev on numa socket %d",
-			rte_socket_id());
-
 	if (!dev)
 		return -1;
 
-	/* reserve an ethdev entry */
 	eth_dev = rte_eth_dev_allocated(rte_vdev_device_name(dev));
 	if (eth_dev == NULL)
-		return -1;
-
-	if (rte_eal_process_type() == RTE_PROC_PRIMARY) {
-		internals = eth_dev->data->dev_private;
-		if (internals != NULL && internals->phy_mac == 0)
-			/* not dynamically allocated, must not be freed */
-			eth_dev->data->mac_addrs = NULL;
-	}
+		return 0; /* port already released */
 
 	eth_dev_close(eth_dev);
-
-	rte_free(eth_dev->process_private);
 	rte_eth_dev_release_port(eth_dev);
 
 	return 0;
-- 
2.28.0

[dpdk-dev] [PATCH v2 13/25] net/qede: release port upon close

[Thomas Monjalon]

From: Rasesh Mody <rmody@marvell.com>

Set RTE_ETH_DEV_CLOSE_REMOVE upon probe so all the private resources
for the port can be freed by rte_eth_dev_close(). With this change the
private port resources are released in the .dev_close callback.

Signed-off-by: Rasesh Mody <rmody@marvell.com>
---
 drivers/net/qede/qede_ethdev.c | 25 +++++++++++++------------
 1 file changed, 13 insertions(+), 12 deletions(-)

diff --git a/drivers/net/qede/qede_ethdev.c b/drivers/net/qede/qede_ethdev.c
index 9535df183f..43fe68c2bf 100644
--- a/drivers/net/qede/qede_ethdev.c
+++ b/drivers/net/qede/qede_ethdev.c
@@ -1550,6 +1550,10 @@ static int qede_dev_close(struct rte_eth_dev *eth_dev)
 
 	PMD_INIT_FUNC_TRACE(edev);
 
+	/* only close in case of the primary process */
+	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
+		return 0;
+
 	/* dev_stop() shall cleanup fp resources in hw but without releasing
 	 * dma memories and sw structures so that dev_start() can be called
 	 * by the app without reconfiguration. However, in dev_close() we
@@ -1587,6 +1591,10 @@ static int qede_dev_close(struct rte_eth_dev *eth_dev)
 	if (ECORE_IS_CMT(edev))
 		rte_eal_alarm_cancel(qede_poll_sp_sb_cb, (void *)eth_dev);
 
+	eth_dev->dev_ops = NULL;
+	eth_dev->rx_pkt_burst = NULL;
+	eth_dev->tx_pkt_burst = NULL;
+
 	return 0;
 }
 
@@ -2703,6 +2711,11 @@ static int qede_common_dev_init(struct rte_eth_dev *eth_dev, bool is_vf)
 		adapter->ipgre.enable = false;
 	}
 
+	/* Pass the information to the rte_eth_dev_close() that it should also
+	 * release the private port resources.
+	 */
+	eth_dev->data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
+
 	DP_INFO(edev, "MAC address : %02x:%02x:%02x:%02x:%02x:%02x\n",
 		adapter->primary_mac.addr_bytes[0],
 		adapter->primary_mac.addr_bytes[1],
@@ -2737,20 +2750,8 @@ static int qede_dev_common_uninit(struct rte_eth_dev *eth_dev)
 {
 	struct qede_dev *qdev = eth_dev->data->dev_private;
 	struct ecore_dev *edev = &qdev->edev;
-
 	PMD_INIT_FUNC_TRACE(edev);
-
-	/* only uninitialize in the primary process */
-	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
-		return 0;
-
-	/* safe to close dev here */
 	qede_dev_close(eth_dev);
-
-	eth_dev->dev_ops = NULL;
-	eth_dev->rx_pkt_burst = NULL;
-	eth_dev->tx_pkt_burst = NULL;
-
 	return 0;
 }
 
-- 
2.28.0

[dpdk-dev] [PATCH v2 14/25] net/ring: release port upon close

[Thomas Monjalon]

The flag RTE_ETH_DEV_CLOSE_REMOVE is set so all port resources
can be freed by rte_eth_dev_close().

Freeing of private port resources is moved
from the ".remove(device)" to the ".dev_close(port)" operation.

Nothing is closed in a secondary process.

Signed-off-by: Thomas Monjalon <thomas@monjalon.net>
Acked-by: Bruce Richardson <bruce.richardson@intel.com>
---
 drivers/net/ring/rte_eth_ring.c | 54 +++++++++++++++++++++------------
 1 file changed, 34 insertions(+), 20 deletions(-)

diff --git a/drivers/net/ring/rte_eth_ring.c b/drivers/net/ring/rte_eth_ring.c
index 40fe1ca4ba..ed0fdeb28f 100644
--- a/drivers/net/ring/rte_eth_ring.c
+++ b/drivers/net/ring/rte_eth_ring.c
@@ -226,7 +226,38 @@ static int
 eth_link_update(struct rte_eth_dev *dev __rte_unused,
 		int wait_to_complete __rte_unused) { return 0; }
 
+static int
+eth_dev_close(struct rte_eth_dev *dev)
+{
+	struct pmd_internals *internals = NULL;
+	struct ring_queue *r = NULL;
+	uint16_t i;
+
+	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
+		return 0;
+
+	eth_dev_stop(dev);
+
+	internals = dev->data->dev_private;
+	if (internals->action == DEV_CREATE) {
+		/*
+		 * it is only necessary to delete the rings in rx_queues because
+		 * they are the same used in tx_queues
+		 */
+		for (i = 0; i < dev->data->nb_rx_queues; i++) {
+			r = dev->data->rx_queues[i];
+			rte_ring_free(r->rng);
+		}
+	}
+
+	/* mac_addrs must not be freed alone because part of dev_private */
+	dev->data->mac_addrs = NULL;
+
+	return 0;
+}
+
 static const struct eth_dev_ops ops = {
+	.dev_close = eth_dev_close,
 	.dev_start = eth_dev_start,
 	.dev_stop = eth_dev_stop,
 	.dev_set_link_up = eth_dev_set_link_up,
@@ -327,6 +358,7 @@ do_eth_dev_ring_create(const char *name,
 
 	eth_dev->dev_ops = &ops;
 	data->numa_node = numa_node;
+	data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
 
 	/* finally assign rx and tx ops */
 	eth_dev->rx_pkt_burst = eth_ring_rx;
@@ -658,9 +690,6 @@ rte_pmd_ring_remove(struct rte_vdev_device *dev)
 {
 	const char *name = rte_vdev_device_name(dev);
 	struct rte_eth_dev *eth_dev = NULL;
-	struct pmd_internals *internals = NULL;
-	struct ring_queue *r = NULL;
-	uint16_t i;
 
 	PMD_LOG(INFO, "Un-Initializing pmd_ring for %s", name);
 
@@ -670,24 +699,9 @@ rte_pmd_ring_remove(struct rte_vdev_device *dev)
 	/* find an ethdev entry */
 	eth_dev = rte_eth_dev_allocated(name);
 	if (eth_dev == NULL)
-		return -ENODEV;
+		return 0; /* port already released */
 
-	eth_dev_stop(eth_dev);
-
-	internals = eth_dev->data->dev_private;
-	if (internals->action == DEV_CREATE) {
-		/*
-		 * it is only necessary to delete the rings in rx_queues because
-		 * they are the same used in tx_queues
-		 */
-		for (i = 0; i < eth_dev->data->nb_rx_queues; i++) {
-			r = eth_dev->data->rx_queues[i];
-			rte_ring_free(r->rng);
-		}
-	}
-
-	/* mac_addrs must not be freed alone because part of dev_private */
-	eth_dev->data->mac_addrs = NULL;
+	eth_dev_close(eth_dev);
 	rte_eth_dev_release_port(eth_dev);
 	return 0;
 }
-- 
2.28.0

[dpdk-dev] [PATCH v2 15/25] net/softnic: release port upon close

[Thomas Monjalon]

The flag RTE_ETH_DEV_CLOSE_REMOVE is set so all port resources
can be freed by rte_eth_dev_close().

Freeing of private port resources is moved
from the ".remove(device)" to the ".dev_close(port)" operation.

Nothing is closed in a secondary process.

Signed-off-by: Thomas Monjalon <thomas@monjalon.net>
Acked-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com>
---
 drivers/net/softnic/rte_eth_softnic.c | 66 ++++++++++++++-------------
 1 file changed, 35 insertions(+), 31 deletions(-)

diff --git a/drivers/net/softnic/rte_eth_softnic.c b/drivers/net/softnic/rte_eth_softnic.c
index 038abd98da..ad9e2aa86d 100644
--- a/drivers/net/softnic/rte_eth_softnic.c
+++ b/drivers/net/softnic/rte_eth_softnic.c
@@ -201,9 +201,40 @@ pmd_dev_stop(struct rte_eth_dev *dev)
 	softnic_mtr_free(p);
 }
 
+static void
+pmd_free(struct pmd_internals *p)
+{
+	if (p == NULL)
+		return;
+
+	if (p->params.conn_port)
+		softnic_conn_free(p->conn);
+
+	softnic_thread_free(p);
+	softnic_pipeline_free(p);
+	softnic_table_action_profile_free(p);
+	softnic_port_in_action_profile_free(p);
+	softnic_tap_free(p);
+	softnic_tmgr_free(p);
+	softnic_link_free(p);
+	softnic_swq_free(p);
+	softnic_mempool_free(p);
+
+	tm_hierarchy_free(p);
+	softnic_mtr_free(p);
+
+	rte_free(p);
+}
+
 static int
-pmd_dev_close(struct rte_eth_dev *dev __rte_unused)
+pmd_dev_close(struct rte_eth_dev *dev)
 {
+	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
+		return 0;
+
+	pmd_free(dev->data->dev_private);
+	dev->data->dev_private = NULL; /* already freed */
+	dev->data->mac_addrs = NULL; /* statically allocated */
 	return 0;
 }
 
@@ -335,31 +366,6 @@ pmd_init(struct pmd_params *params)
 	return p;
 }
 
-static void
-pmd_free(struct pmd_internals *p)
-{
-	if (p == NULL)
-		return;
-
-	if (p->params.conn_port)
-		softnic_conn_free(p->conn);
-
-	softnic_thread_free(p);
-	softnic_pipeline_free(p);
-	softnic_table_action_profile_free(p);
-	softnic_port_in_action_profile_free(p);
-	softnic_tap_free(p);
-	softnic_tmgr_free(p);
-	softnic_link_free(p);
-	softnic_swq_free(p);
-	softnic_mempool_free(p);
-
-	tm_hierarchy_free(p);
-	softnic_mtr_free(p);
-
-	rte_free(p);
-}
-
 static struct rte_ether_addr eth_addr = {
 	.addr_bytes = {0},
 };
@@ -384,6 +390,7 @@ pmd_ethdev_register(struct rte_vdev_device *vdev,
 	dev->device = &vdev->device;
 
 	/* dev->data */
+	dev->data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
 	dev->data->dev_private = dev_private;
 	dev->data->dev_link.link_speed = ETH_SPEED_NUM_100G;
 	dev->data->dev_link.link_duplex = ETH_LINK_FULL_DUPLEX;
@@ -652,12 +659,9 @@ pmd_remove(struct rte_vdev_device *vdev)
 	/* Find the ethdev entry */
 	dev = rte_eth_dev_allocated(rte_vdev_device_name(vdev));
 	if (dev == NULL)
-		return -ENODEV;
+		return 0; /* port already released */
 
-	/* Free device data structures*/
-	pmd_free(dev->data->dev_private);
-	dev->data->dev_private = NULL; /* already freed */
-	dev->data->mac_addrs = NULL; /* statically allocated */
+	pmd_dev_close(dev);
 	rte_eth_dev_release_port(dev);
 
 	return 0;
-- 
2.28.0

[dpdk-dev] [PATCH v2 16/25] net/tap: release port upon close

[Thomas Monjalon]

From: Yunjian Wang <wangyunjian@huawei.com>

The flag RTE_ETH_DEV_CLOSE_REMOVE is set so all port resources
can be freed by rte_eth_dev_close().

Freeing of private port resources is moved
from the ".remove(device)" to the ".dev_close(port)" operation.

Signed-off-by: Yunjian Wang <wangyunjian@huawei.com>
Signed-off-by: Thomas Monjalon <thomas@monjalon.net>
---
 drivers/net/tap/rte_eth_tap.c | 51 +++++++++++++++++++----------------
 1 file changed, 28 insertions(+), 23 deletions(-)

diff --git a/drivers/net/tap/rte_eth_tap.c b/drivers/net/tap/rte_eth_tap.c
index 2d1e4e365a..0d217f1486 100644
--- a/drivers/net/tap/rte_eth_tap.c
+++ b/drivers/net/tap/rte_eth_tap.c
@@ -72,6 +72,10 @@
 
 static int tap_devices_count;
 
+static const char *tuntap_types[ETH_TUNTAP_TYPE_MAX] = {
+	"UNKNOWN", "TUN", "TAP"
+};
+
 static const char *valid_arguments[] = {
 	ETH_TAP_IFACE_ARG,
 	ETH_TAP_REMOTE_ARG,
@@ -1075,6 +1079,11 @@ tap_dev_close(struct rte_eth_dev *dev)
 	struct pmd_process_private *process_private = dev->process_private;
 	struct rx_queue *rxq;
 
+	if (rte_eal_process_type() != RTE_PROC_PRIMARY) {
+		rte_free(dev->process_private);
+		return 0;
+	}
+
 	tap_link_set_down(dev);
 	if (internals->nlsk_fd != -1) {
 		tap_flow_flush(dev, NULL);
@@ -1112,6 +1121,23 @@ tap_dev_close(struct rte_eth_dev *dev)
 		close(internals->ka_fd);
 		internals->ka_fd = -1;
 	}
+
+	/* mac_addrs must not be freed alone because part of dev_private */
+	dev->data->mac_addrs = NULL;
+
+	internals = dev->data->dev_private;
+	TAP_LOG(DEBUG, "Closing %s Ethernet device on numa %u",
+		tuntap_types[internals->type], rte_socket_id());
+
+	if (internals->ioctl_sock != -1) {
+		close(internals->ioctl_sock);
+		internals->ioctl_sock = -1;
+	}
+	rte_free(dev->process_private);
+	dev->process_private = NULL;
+	if (tap_devices_count == 1)
+		rte_mp_action_unregister(TAP_MP_KEY);
+	tap_devices_count--;
 	/*
 	 * Since TUN device has no more opened file descriptors
 	 * it will be removed from kernel
@@ -1845,10 +1871,6 @@ static const struct eth_dev_ops ops = {
 	.filter_ctrl            = tap_dev_filter_ctrl,
 };
 
-static const char *tuntap_types[ETH_TUNTAP_TYPE_MAX] = {
-	"UNKNOWN", "TUN", "TAP"
-};
-
 static int
 eth_dev_tap_create(struct rte_vdev_device *vdev, const char *tap_name,
 		   char *remote_iface, struct rte_ether_addr *mac_addr,
@@ -1900,7 +1922,7 @@ eth_dev_tap_create(struct rte_vdev_device *vdev, const char *tap_name,
 	/* Setup some default values */
 	data = dev->data;
 	data->dev_private = pmd;
-	data->dev_flags = RTE_ETH_DEV_INTR_LSC;
+	data->dev_flags = RTE_ETH_DEV_INTR_LSC | RTE_ETH_DEV_CLOSE_REMOVE;
 	data->numa_node = numa_node;
 
 	data->dev_link = pmd_link;
@@ -2492,30 +2514,13 @@ static int
 rte_pmd_tap_remove(struct rte_vdev_device *dev)
 {
 	struct rte_eth_dev *eth_dev = NULL;
-	struct pmd_internals *internals;
 
 	/* find the ethdev entry */
 	eth_dev = rte_eth_dev_allocated(rte_vdev_device_name(dev));
 	if (!eth_dev)
-		return -ENODEV;
-
-	/* mac_addrs must not be freed alone because part of dev_private */
-	eth_dev->data->mac_addrs = NULL;
-
-	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
-		return rte_eth_dev_release_port(eth_dev);
+		return 0;
 
 	tap_dev_close(eth_dev);
-
-	internals = eth_dev->data->dev_private;
-	TAP_LOG(DEBUG, "Closing %s Ethernet device on numa %u",
-		tuntap_types[internals->type], rte_socket_id());
-
-	close(internals->ioctl_sock);
-	rte_free(eth_dev->process_private);
-	if (tap_devices_count == 1)
-		rte_mp_action_unregister(TAP_MP_KEY);
-	tap_devices_count--;
 	rte_eth_dev_release_port(eth_dev);
 
 	return 0;
-- 
2.28.0

[dpdk-dev] [PATCH v2 17/25] ethdev: remove old close behaviour

[Thomas Monjalon]

The temporary flag RTE_ETH_DEV_CLOSE_REMOVE is removed.
It was introduced in DPDK 18.11 in order to give time for PMDs to migrate.

The old behaviour was to free only queues when closing a port.
The new behaviour is calling rte_eth_dev_release_port() which does
three more tasks:
	- trigger event callback
	- reset state and few pointers
	- free all generic port resources

The private port resources must be released in the .dev_close callback.

The .remove callback should:
	- call .dev_close callback
	- call rte_eth_dev_release_port()
	- free multi-port device shared resources

Despite waiting two years, some drivers have not migrated,
so they may hit issues with the incompatible new behaviour.
After sending emails, adding logs, and announcing the deprecation,
the only last solution is to declare these drivers as unmaintained:
	dpaa, dpaa2, enetc, ionic, liquidio, nfp, pfe
Below is a summary of what to implement in those drivers.

* The freeing of private port resources must be moved
from the ".remove(device)" function to the ".dev_close(port)" function.

* If a generic resource (.mac_addrs or .hash_mac_addrs) cannot be freed,
it must be set to NULL in ".dev_close" function to protect from
subsequent rte_eth_dev_release_port() freeing.

* Note 1:
The generic resources are freed in rte_eth_dev_release_port(),
after ".dev_close" is called in rte_eth_dev_close(), but not when
calling ".dev_close" directly from the ".remove" PMD function.
That's why rte_eth_dev_release_port() must still be called explicitly
from ".remove(device)" after calling the ".dev_close" PMD function.

* Note 2:
If a device can have multiple ports, the common resources must be freed
only in the ".remove(device)" function.

* Note 3:
The port is supposed to be in a stopped state when it is closed.
If it is not the case, it is free to the PMD implementation
how to react when trying to close a non-stopped port:
either try to stop it automatically or just return an error.

Cc: Shijith Thotton <sthotton@marvell.com>
Cc: Srisivasubramanian Srinivasan <srinivasan@marvell.com>
Cc: Heinrich Kuhn <heinrich.kuhn@netronome.com>
Cc: Hemant Agrawal <hemant.agrawal@nxp.com>
Cc: Sachin Saxena <sachin.saxena@nxp.com>
Cc: Gagandeep Singh <g.singh@nxp.com>
Cc: Akhil Goyal <akhil.goyal@nxp.com>
Cc: Alfredo Cardigliano <cardigliano@ntop.org>

Signed-off-by: Thomas Monjalon <thomas@monjalon.net>
---
 MAINTAINERS                               | 14 +++++++-------
 doc/guides/rel_notes/deprecation.rst      |  6 ------
 drivers/net/af_packet/rte_eth_af_packet.c |  1 -
 drivers/net/af_xdp/rte_eth_af_xdp.c       |  2 --
 drivers/net/ark/ark_ethdev.c              |  2 --
 drivers/net/atlantic/atl_ethdev.c         |  2 --
 drivers/net/avp/avp_ethdev.c              |  2 --
 drivers/net/axgbe/axgbe_ethdev.c          |  1 -
 drivers/net/bnx2x/bnx2x_ethdev.c          |  5 -----
 drivers/net/bnxt/bnxt_ethdev.c            |  5 -----
 drivers/net/bnxt/bnxt_reps.c              |  4 ----
 drivers/net/bonding/rte_eth_bond_pmd.c    |  1 -
 drivers/net/cxgbe/cxgbe_main.c            |  2 --
 drivers/net/cxgbe/cxgbevf_main.c          |  2 --
 drivers/net/e1000/em_ethdev.c             |  5 -----
 drivers/net/e1000/igb_ethdev.c            | 10 ----------
 drivers/net/ena/ena_ethdev.c              |  6 ------
 drivers/net/enic/enic_ethdev.c            |  2 --
 drivers/net/enic/enic_vf_representor.c    |  3 +--
 drivers/net/failsafe/failsafe.c           |  1 -
 drivers/net/fm10k/fm10k_ethdev.c          |  5 -----
 drivers/net/hinic/hinic_pmd_ethdev.c      |  6 ------
 drivers/net/hns3/hns3_ethdev.c            |  5 -----
 drivers/net/hns3/hns3_ethdev_vf.c         |  5 -----
 drivers/net/i40e/i40e_ethdev.c            |  5 -----
 drivers/net/i40e/i40e_ethdev_vf.c         |  5 -----
 drivers/net/iavf/iavf_ethdev.c            |  5 -----
 drivers/net/ice/ice_dcf_ethdev.c          |  2 --
 drivers/net/ice/ice_ethdev.c              |  5 -----
 drivers/net/igc/igc_ethdev.c              |  5 -----
 drivers/net/ixgbe/ixgbe_ethdev.c          | 10 ----------
 drivers/net/kni/rte_eth_kni.c             |  2 --
 drivers/net/memif/rte_eth_memif.c         |  3 ---
 drivers/net/mlx4/mlx4.c                   |  1 -
 drivers/net/mlx5/linux/mlx5_os.c          |  2 --
 drivers/net/mvneta/mvneta_ethdev.c        |  3 ---
 drivers/net/mvpp2/mrvl_ethdev.c           |  3 ---
 drivers/net/netvsc/hn_ethdev.c            |  3 ---
 drivers/net/nfb/nfb_ethdev.c              |  3 ---
 drivers/net/null/rte_eth_null.c           |  1 -
 drivers/net/octeontx/octeontx_ethdev.c    |  1 -
 drivers/net/octeontx2/otx2_ethdev.c       |  1 -
 drivers/net/pcap/rte_eth_pcap.c           |  1 -
 drivers/net/qede/qede_ethdev.c            |  5 -----
 drivers/net/ring/rte_eth_ring.c           |  1 -
 drivers/net/sfc/sfc_ethdev.c              |  4 +---
 drivers/net/softnic/rte_eth_softnic.c     |  1 -
 drivers/net/szedata2/rte_eth_szedata2.c   |  3 ---
 drivers/net/tap/rte_eth_tap.c             |  2 +-
 drivers/net/vhost/rte_eth_vhost.c         |  2 +-
 drivers/net/virtio/virtio_ethdev.c        |  5 -----
 drivers/net/vmxnet3/vmxnet3_ethdev.c      |  3 ---
 lib/librte_ethdev/rte_ethdev.c            | 17 +----------------
 lib/librte_ethdev/rte_ethdev.h            |  8 +-------
 54 files changed, 13 insertions(+), 196 deletions(-)

diff --git a/MAINTAINERS b/MAINTAINERS
index 49a6dfa7a5..71921675ab 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -604,7 +604,7 @@ F: drivers/net/thunderx/
 F: doc/guides/nics/thunderx.rst
 F: doc/guides/nics/features/thunderx.ini
 
-Cavium LiquidIO
+Cavium LiquidIO - UNMAINTAINED
 M: Shijith Thotton <sthotton@marvell.com>
 M: Srisivasubramanian Srinivasan <srinivasan@marvell.com>
 T: git://dpdk.org/next/dpdk-next-net-mrvl
@@ -793,13 +793,13 @@ F: drivers/net/nfb/
 F: doc/guides/nics/nfb.rst
 F: doc/guides/nics/features/nfb.ini
 
-Netronome nfp
+Netronome nfp - UNMAINTAINED
 M: Heinrich Kuhn <heinrich.kuhn@netronome.com>
 F: drivers/net/nfp/
 F: doc/guides/nics/nfp.rst
 F: doc/guides/nics/features/nfp*.ini
 
-NXP dpaa
+NXP dpaa - UNMAINTAINED
 M: Hemant Agrawal <hemant.agrawal@nxp.com>
 M: Sachin Saxena <sachin.saxena@oss.nxp.com>
 F: drivers/mempool/dpaa/
@@ -807,7 +807,7 @@ F: drivers/net/dpaa/
 F: doc/guides/nics/dpaa.rst
 F: doc/guides/nics/features/dpaa.ini
 
-NXP dpaa2
+NXP dpaa2 - UNMAINTAINED
 M: Hemant Agrawal <hemant.agrawal@nxp.com>
 M: Sachin Saxena <sachin.saxena@oss.nxp.com>
 F: drivers/mempool/dpaa2/
@@ -815,21 +815,21 @@ F: drivers/net/dpaa2/
 F: doc/guides/nics/dpaa2.rst
 F: doc/guides/nics/features/dpaa2.ini
 
-NXP enetc
+NXP enetc - UNMAINTAINED
 M: Gagandeep Singh <g.singh@nxp.com>
 M: Sachin Saxena <sachin.saxena@oss.nxp.com>
 F: drivers/net/enetc/
 F: doc/guides/nics/enetc.rst
 F: doc/guides/nics/features/enetc.ini
 
-NXP pfe
+NXP pfe - UNMAINTAINED
 M: Gagandeep Singh <g.singh@nxp.com>
 M: Akhil Goyal <akhil.goyal@nxp.com>
 F: doc/guides/nics/pfe.rst
 F: drivers/net/pfe/
 F: doc/guides/nics/features/pfe.ini
 
-Pensando ionic
+Pensando ionic - UNMAINTAINED
 M: Alfredo Cardigliano <cardigliano@ntop.org>
 F: drivers/net/ionic/
 F: doc/guides/nics/ionic.rst
diff --git a/doc/guides/rel_notes/deprecation.rst b/doc/guides/rel_notes/deprecation.rst
index 67caedb11b..a907358078 100644
--- a/doc/guides/rel_notes/deprecation.rst
+++ b/doc/guides/rel_notes/deprecation.rst
@@ -142,12 +142,6 @@ Deprecation Notices
   - ``rte_eth_dev_stop``
   - ``rte_eth_dev_close``
 
-* ethdev: The temporary flag RTE_ETH_DEV_CLOSE_REMOVE will be removed in 20.11.
-  As a consequence, the new behaviour introduced in 18.11 will be effective
-  for all drivers: generic port resources are freed on close operation.
-  Private resources are expected to be released in the ``dev_close`` callback.
-  More details in http://inbox.dpdk.org/dev/5248162.j6AOsuQRmx@thomas/
-
 * ethdev: New offload flags ``DEV_RX_OFFLOAD_FLOW_MARK`` will be added in 19.11.
   This will allow application to enable or disable PMDs from updating
   ``rte_mbuf::hash::fdir``.
diff --git a/drivers/net/af_packet/rte_eth_af_packet.c b/drivers/net/af_packet/rte_eth_af_packet.c
index 12c202cba4..25876224f8 100644
--- a/drivers/net/af_packet/rte_eth_af_packet.c
+++ b/drivers/net/af_packet/rte_eth_af_packet.c
@@ -859,7 +859,6 @@ rte_pmd_init_internals(struct rte_vdev_device *dev,
 	data->nb_tx_queues = (uint16_t)nb_queues;
 	data->dev_link = pmd_link;
 	data->mac_addrs = &(*internals)->eth_addr;
-	data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
 
 	(*eth_dev)->dev_ops = &ops;
 
diff --git a/drivers/net/af_xdp/rte_eth_af_xdp.c b/drivers/net/af_xdp/rte_eth_af_xdp.c
index badbce63fb..60add9ead4 100644
--- a/drivers/net/af_xdp/rte_eth_af_xdp.c
+++ b/drivers/net/af_xdp/rte_eth_af_xdp.c
@@ -1274,8 +1274,6 @@ init_internals(struct rte_vdev_device *dev, const char *if_name,
 	eth_dev->dev_ops = &ops;
 	eth_dev->rx_pkt_burst = eth_af_xdp_rx;
 	eth_dev->tx_pkt_burst = eth_af_xdp_tx;
-	/* Let rte_eth_dev_close() release the port resources. */
-	eth_dev->data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
 
 #if defined(XDP_UMEM_UNALIGNED_CHUNK_FLAG)
 	AF_XDP_LOG(INFO, "Zero copy between umem and mbuf enabled.\n");
diff --git a/drivers/net/ark/ark_ethdev.c b/drivers/net/ark/ark_ethdev.c
index 5f2ed4b7b7..83dc4ecd2c 100644
--- a/drivers/net/ark/ark_ethdev.c
+++ b/drivers/net/ark/ark_ethdev.c
@@ -260,8 +260,6 @@ eth_ark_dev_init(struct rte_eth_dev *dev)
 	/* Use dummy function until setup */
 	dev->rx_pkt_burst = &eth_ark_recv_pkts_noop;
 	dev->tx_pkt_burst = &eth_ark_xmit_pkts_noop;
-	/* Let rte_eth_dev_close() release the port resources */
-	dev->data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
 
 	ark->bar0 = (uint8_t *)pci_dev->mem_resource[0].addr;
 	ark->a_bar = (uint8_t *)pci_dev->mem_resource[2].addr;
diff --git a/drivers/net/atlantic/atl_ethdev.c b/drivers/net/atlantic/atl_ethdev.c
index d3babeff94..2217511ca0 100644
--- a/drivers/net/atlantic/atl_ethdev.c
+++ b/drivers/net/atlantic/atl_ethdev.c
@@ -380,8 +380,6 @@ eth_atl_dev_init(struct rte_eth_dev *eth_dev)
 	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
 		return 0;
 
-	eth_dev->data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
-
 	/* Vendor and Device ID need to be set before init of shared code */
 	hw->device_id = pci_dev->id.device_id;
 	hw->vendor_id = pci_dev->id.vendor_id;
diff --git a/drivers/net/avp/avp_ethdev.c b/drivers/net/avp/avp_ethdev.c
index 2eed6acc3f..c730b7ab86 100644
--- a/drivers/net/avp/avp_ethdev.c
+++ b/drivers/net/avp/avp_ethdev.c
@@ -957,8 +957,6 @@ eth_avp_dev_init(struct rte_eth_dev *eth_dev)
 	eth_dev->dev_ops = &avp_eth_dev_ops;
 	eth_dev->rx_pkt_burst = &avp_recv_pkts;
 	eth_dev->tx_pkt_burst = &avp_xmit_pkts;
-	/* Let rte_eth_dev_close() release the port resources */
-	eth_dev->data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
 
 	if (rte_eal_process_type() != RTE_PROC_PRIMARY) {
 		/*
diff --git a/drivers/net/axgbe/axgbe_ethdev.c b/drivers/net/axgbe/axgbe_ethdev.c
index 2dd64180ad..cf085487cc 100644
--- a/drivers/net/axgbe/axgbe_ethdev.c
+++ b/drivers/net/axgbe/axgbe_ethdev.c
@@ -1959,7 +1959,6 @@ eth_axgbe_dev_init(struct rte_eth_dev *eth_dev)
 	int ret;
 
 	eth_dev->dev_ops = &axgbe_eth_dev_ops;
-	eth_dev->data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
 
 	eth_dev->rx_descriptor_status = axgbe_dev_rx_descriptor_status;
 	eth_dev->tx_descriptor_status = axgbe_dev_tx_descriptor_status;
diff --git a/drivers/net/bnx2x/bnx2x_ethdev.c b/drivers/net/bnx2x/bnx2x_ethdev.c
index 8dc46384af..40225b2f44 100644
--- a/drivers/net/bnx2x/bnx2x_ethdev.c
+++ b/drivers/net/bnx2x/bnx2x_ethdev.c
@@ -735,11 +735,6 @@ bnx2x_common_dev_init(struct rte_eth_dev *eth_dev, int is_vf)
 			goto out;
 	}
 
-	/* Pass the information to the rte_eth_dev_close() that it should also
-	 * release the private port resources.
-	 */
-	eth_dev->data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
-
 	return 0;
 
 out:
diff --git a/drivers/net/bnxt/bnxt_ethdev.c b/drivers/net/bnxt/bnxt_ethdev.c
index e726d078ec..a77bab661d 100644
--- a/drivers/net/bnxt/bnxt_ethdev.c
+++ b/drivers/net/bnxt/bnxt_ethdev.c
@@ -5802,11 +5802,6 @@ bnxt_dev_init(struct rte_eth_dev *eth_dev, void *params __rte_unused)
 
 	bnxt_alloc_switch_domain(bp);
 
-	/* Pass the information to the rte_eth_dev_close() that it should also
-	 * release the private port resources.
-	 */
-	eth_dev->data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
-
 	PMD_DRV_LOG(INFO,
 		    DRV_MODULE_NAME "found at mem %" PRIX64 ", node addr %pM\n",
 		    pci_dev->mem_resource[0].phys_addr,
diff --git a/drivers/net/bnxt/bnxt_reps.c b/drivers/net/bnxt/bnxt_reps.c
index 2b6c0a277b..df8680c113 100644
--- a/drivers/net/bnxt/bnxt_reps.c
+++ b/drivers/net/bnxt/bnxt_reps.c
@@ -203,10 +203,6 @@ int bnxt_vf_representor_init(struct rte_eth_dev *eth_dev, void *params)
 	PMD_DRV_LOG(INFO, "calling bnxt_print_link_info\n");
 	bnxt_print_link_info(eth_dev);
 
-	/* Pass the information to the rte_eth_dev_close() that it should also
-	 * release the private port resources.
-	 */
-	eth_dev->data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
 	PMD_DRV_LOG(INFO,
 		    "Switch domain id %d: Representor Device %d init done\n",
 		    vf_rep_bp->switch_domain_id, vf_rep_bp->vf_id);
diff --git a/drivers/net/bonding/rte_eth_bond_pmd.c b/drivers/net/bonding/rte_eth_bond_pmd.c
index a9ed5d8e12..1f761c7c9e 100644
--- a/drivers/net/bonding/rte_eth_bond_pmd.c
+++ b/drivers/net/bonding/rte_eth_bond_pmd.c
@@ -3209,7 +3209,6 @@ bond_alloc(struct rte_vdev_device *dev, uint8_t mode)
 	}
 
 	internals = eth_dev->data->dev_private;
-	eth_dev->data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
 	eth_dev->data->nb_rx_queues = (uint16_t)1;
 	eth_dev->data->nb_tx_queues = (uint16_t)1;
 
diff --git a/drivers/net/cxgbe/cxgbe_main.c b/drivers/net/cxgbe/cxgbe_main.c
index da4ae25d47..53b08a64af 100644
--- a/drivers/net/cxgbe/cxgbe_main.c
+++ b/drivers/net/cxgbe/cxgbe_main.c
@@ -2142,8 +2142,6 @@ int cxgbe_probe(struct adapter *adapter)
 			goto out_free;
 		}
 
-		pi->eth_dev->data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
-
 		if (i > 0) {
 			/* First port will be notified by upper layer */
 			rte_eth_dev_probing_finish(eth_dev);
diff --git a/drivers/net/cxgbe/cxgbevf_main.c b/drivers/net/cxgbe/cxgbevf_main.c
index 35d873402b..9ee060504f 100644
--- a/drivers/net/cxgbe/cxgbevf_main.c
+++ b/drivers/net/cxgbe/cxgbevf_main.c
@@ -261,8 +261,6 @@ int cxgbevf_probe(struct adapter *adapter)
 			goto out_free;
 		}
 
-		pi->eth_dev->data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
-
 		if (i > 0) {
 			/* First port will be notified by upper layer */
 			rte_eth_dev_probing_finish(eth_dev);
diff --git a/drivers/net/e1000/em_ethdev.c b/drivers/net/e1000/em_ethdev.c
index 2d0e071448..6a6ae0e9d5 100644
--- a/drivers/net/e1000/em_ethdev.c
+++ b/drivers/net/e1000/em_ethdev.c
@@ -297,11 +297,6 @@ eth_em_dev_init(struct rte_eth_dev *eth_dev)
 	rte_ether_addr_copy((struct rte_ether_addr *)hw->mac.addr,
 		eth_dev->data->mac_addrs);
 
-	/* Pass the information to the rte_eth_dev_close() that it should also
-	 * release the private port resources.
-	 */
-	eth_dev->data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
-
 	/* initialize the vfta */
 	memset(shadow_vfta, 0, sizeof(*shadow_vfta));
 
diff --git a/drivers/net/e1000/igb_ethdev.c b/drivers/net/e1000/igb_ethdev.c
index 9e4aefe00c..f8778207ef 100644
--- a/drivers/net/e1000/igb_ethdev.c
+++ b/drivers/net/e1000/igb_ethdev.c
@@ -841,11 +841,6 @@ eth_igb_dev_init(struct rte_eth_dev *eth_dev)
 	rte_ether_addr_copy((struct rte_ether_addr *)hw->mac.addr,
 			&eth_dev->data->mac_addrs[0]);
 
-	/* Pass the information to the rte_eth_dev_close() that it should also
-	 * release the private port resources.
-	 */
-	eth_dev->data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
-
 	/* initialize the vfta */
 	memset(shadow_vfta, 0, sizeof(*shadow_vfta));
 
@@ -997,11 +992,6 @@ eth_igbvf_dev_init(struct rte_eth_dev *eth_dev)
 		return -ENOMEM;
 	}
 
-	/* Pass the information to the rte_eth_dev_close() that it should also
-	 * release the private port resources.
-	 */
-	eth_dev->data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
-
 	/* Generate a random MAC address, if none was assigned by PF. */
 	if (rte_is_zero_ether_addr(perm_addr)) {
 		rte_eth_random_addr(perm_addr->addr_bytes);
diff --git a/drivers/net/ena/ena_ethdev.c b/drivers/net/ena/ena_ethdev.c
index ae6daad892..cf1c0b9795 100644
--- a/drivers/net/ena/ena_ethdev.c
+++ b/drivers/net/ena/ena_ethdev.c
@@ -1863,12 +1863,6 @@ static int eth_ena_dev_init(struct rte_eth_dev *eth_dev)
 			get_feat_ctx.dev_attr.mac_addr,
 			(struct rte_ether_addr *)adapter->mac_addr);
 
-	/*
-	 * Pass the information to the rte_eth_dev_close() that it should also
-	 * release the private port resources.
-	 */
-	eth_dev->data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
-
 	adapter->drv_stats = rte_zmalloc("adapter stats",
 					 sizeof(*adapter->drv_stats),
 					 RTE_CACHE_LINE_SIZE);
diff --git a/drivers/net/enic/enic_ethdev.c b/drivers/net/enic/enic_ethdev.c
index ed03d53608..27f60b45b8 100644
--- a/drivers/net/enic/enic_ethdev.c
+++ b/drivers/net/enic/enic_ethdev.c
@@ -1292,8 +1292,6 @@ static int eth_enic_dev_init(struct rte_eth_dev *eth_dev,
 	enic->port_id = eth_dev->data->port_id;
 	enic->rte_dev = eth_dev;
 	enic->dev_data = eth_dev->data;
-	/* Let rte_eth_dev_close() release the port resources */
-	eth_dev->data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
 
 	pdev = RTE_ETH_DEV_TO_PCI(eth_dev);
 	rte_eth_copy_pci_info(eth_dev, pdev);
diff --git a/drivers/net/enic/enic_vf_representor.c b/drivers/net/enic/enic_vf_representor.c
index c528be2b98..169c611a68 100644
--- a/drivers/net/enic/enic_vf_representor.c
+++ b/drivers/net/enic/enic_vf_representor.c
@@ -670,8 +670,7 @@ int enic_vf_representor_init(struct rte_eth_dev *eth_dev, void *init_params)
 
 	eth_dev->device->driver = pf->rte_dev->device->driver;
 	eth_dev->dev_ops = &enic_vf_representor_dev_ops;
-	eth_dev->data->dev_flags |= RTE_ETH_DEV_REPRESENTOR
-		| RTE_ETH_DEV_CLOSE_REMOVE;
+	eth_dev->data->dev_flags |= RTE_ETH_DEV_REPRESENTOR;
 	eth_dev->data->representor_id = vf->vf_id;
 	eth_dev->data->mac_addrs = rte_zmalloc("enic_mac_addr_vf",
 		sizeof(struct rte_ether_addr) *
diff --git a/drivers/net/failsafe/failsafe.c b/drivers/net/failsafe/failsafe.c
index 44d47e8f72..b921e101e6 100644
--- a/drivers/net/failsafe/failsafe.c
+++ b/drivers/net/failsafe/failsafe.c
@@ -180,7 +180,6 @@ fs_eth_dev_create(struct rte_vdev_device *vdev)
 		ERROR("Unable to allocate rte_eth_dev");
 		return -1;
 	}
-	dev->data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
 	priv = PRIV(dev);
 	priv->data = dev->data;
 	priv->rxp = FS_RX_PROXY_INIT;
diff --git a/drivers/net/fm10k/fm10k_ethdev.c b/drivers/net/fm10k/fm10k_ethdev.c
index 533f976709..23f4d04068 100644
--- a/drivers/net/fm10k/fm10k_ethdev.c
+++ b/drivers/net/fm10k/fm10k_ethdev.c
@@ -3129,11 +3129,6 @@ eth_fm10k_dev_init(struct rte_eth_dev *dev)
 		&dev->data->mac_addrs[0]);
 	}
 
-	/* Pass the information to the rte_eth_dev_close() that it should also
-	 * release the private port resources.
-	 */
-	dev->data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
-
 	/* Reset the hw statistics */
 	diag = fm10k_stats_reset(dev);
 	if (diag != 0) {
diff --git a/drivers/net/hinic/hinic_pmd_ethdev.c b/drivers/net/hinic/hinic_pmd_ethdev.c
index 5f2e97d3bd..623534fda4 100644
--- a/drivers/net/hinic/hinic_pmd_ethdev.c
+++ b/drivers/net/hinic/hinic_pmd_ethdev.c
@@ -3100,12 +3100,6 @@ static int hinic_func_init(struct rte_eth_dev *eth_dev)
 		goto mc_addr_fail;
 	}
 
-	/*
-	 * Pass the information to the rte_eth_dev_close() that it should also
-	 * release the private port resources.
-	 */
-	eth_dev->data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
-
 	/* create hardware nic_device */
 	rc = hinic_nic_dev_create(eth_dev);
 	if (rc) {
diff --git a/drivers/net/hns3/hns3_ethdev.c b/drivers/net/hns3/hns3_ethdev.c
index 3b395a1ccf..dd03bd62b2 100644
--- a/drivers/net/hns3/hns3_ethdev.c
+++ b/drivers/net/hns3/hns3_ethdev.c
@@ -5678,11 +5678,6 @@ hns3_dev_init(struct rte_eth_dev *eth_dev)
 			    &eth_dev->data->mac_addrs[0]);
 
 	hw->adapter_state = HNS3_NIC_INITIALIZED;
-	/*
-	 * Pass the information to the rte_eth_dev_close() that it should also
-	 * release the private port resources.
-	 */
-	eth_dev->data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
 
 	if (rte_atomic16_read(&hns->hw.reset.schedule) == SCHEDULE_PENDING) {
 		hns3_err(hw, "Reschedule reset service after dev_init");
diff --git a/drivers/net/hns3/hns3_ethdev_vf.c b/drivers/net/hns3/hns3_ethdev_vf.c
index e0ca5a6ac2..436d864b5f 100644
--- a/drivers/net/hns3/hns3_ethdev_vf.c
+++ b/drivers/net/hns3/hns3_ethdev_vf.c
@@ -2684,11 +2684,6 @@ hns3vf_dev_init(struct rte_eth_dev *eth_dev)
 			    &eth_dev->data->mac_addrs[0]);
 
 	hw->adapter_state = HNS3_NIC_INITIALIZED;
-	/*
-	 * Pass the information to the rte_eth_dev_close() that it should also
-	 * release the private port resources.
-	 */
-	eth_dev->data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
 
 	if (rte_atomic16_read(&hns->hw.reset.schedule) == SCHEDULE_PENDING) {
 		hns3_err(hw, "Reschedule reset service after dev_init");
diff --git a/drivers/net/i40e/i40e_ethdev.c b/drivers/net/i40e/i40e_ethdev.c
index 804f0128ab..6fb88148b8 100644
--- a/drivers/net/i40e/i40e_ethdev.c
+++ b/drivers/net/i40e/i40e_ethdev.c
@@ -1699,11 +1699,6 @@ eth_i40e_dev_init(struct rte_eth_dev *dev, void *init_params __rte_unused)
 	rte_ether_addr_copy((struct rte_ether_addr *)hw->mac.perm_addr,
 					&dev->data->mac_addrs[0]);
 
-	/* Pass the information to the rte_eth_dev_close() that it should also
-	 * release the private port resources.
-	 */
-	dev->data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
-
 	/* Init dcb to sw mode by default */
 	ret = i40e_dcb_init_configure(dev, TRUE);
 	if (ret != I40E_SUCCESS) {
diff --git a/drivers/net/i40e/i40e_ethdev_vf.c b/drivers/net/i40e/i40e_ethdev_vf.c
index 61aad8c415..4aaf41956c 100644
--- a/drivers/net/i40e/i40e_ethdev_vf.c
+++ b/drivers/net/i40e/i40e_ethdev_vf.c
@@ -1586,11 +1586,6 @@ i40evf_dev_init(struct rte_eth_dev *eth_dev)
 	hw->adapter_stopped = 1;
 	hw->adapter_closed = 0;
 
-	/* Pass the information to the rte_eth_dev_close() that it should also
-	 * release the private port resources.
-	 */
-	eth_dev->data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
-
 	if(i40evf_init_vf(eth_dev) != 0) {
 		PMD_INIT_LOG(ERR, "Init vf failed");
 		return -1;
diff --git a/drivers/net/iavf/iavf_ethdev.c b/drivers/net/iavf/iavf_ethdev.c
index 8f46c4588f..512ade2ad0 100644
--- a/drivers/net/iavf/iavf_ethdev.c
+++ b/drivers/net/iavf/iavf_ethdev.c
@@ -1412,11 +1412,6 @@ iavf_dev_init(struct rte_eth_dev *eth_dev)
 	adapter->eth_dev = eth_dev;
 	adapter->stopped = 1;
 
-	/* Pass the information to the rte_eth_dev_close() that it should also
-	 * release the private port resources.
-	 */
-	eth_dev->data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
-
 	if (iavf_init_vf(eth_dev) != 0) {
 		PMD_INIT_LOG(ERR, "Init vf failed");
 		return -1;
diff --git a/drivers/net/ice/ice_dcf_ethdev.c b/drivers/net/ice/ice_dcf_ethdev.c
index 5b626cb641..33dd0c44f2 100644
--- a/drivers/net/ice/ice_dcf_ethdev.c
+++ b/drivers/net/ice/ice_dcf_ethdev.c
@@ -908,8 +908,6 @@ ice_dcf_dev_init(struct rte_eth_dev *eth_dev)
 	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
 		return 0;
 
-	eth_dev->data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
-
 	adapter->real_hw.vc_event_msg_cb = ice_dcf_handle_pf_event_msg;
 	if (ice_dcf_init_hw(eth_dev, &adapter->real_hw) != 0) {
 		PMD_INIT_LOG(ERR, "Failed to init DCF hardware");
diff --git a/drivers/net/ice/ice_ethdev.c b/drivers/net/ice/ice_ethdev.c
index b4ac5e3f81..85a3ca6a2d 100644
--- a/drivers/net/ice/ice_ethdev.c
+++ b/drivers/net/ice/ice_ethdev.c
@@ -2245,11 +2245,6 @@ ice_dev_init(struct rte_eth_dev *dev)
 		goto err_init_mac;
 	}
 
-	/* Pass the information to the rte_eth_dev_close() that it should also
-	 * release the private port resources.
-	 */
-	dev->data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
-
 	ret = ice_res_pool_init(&pf->msix_pool, 1,
 				hw->func_caps.common_cap.num_msix_vectors - 1);
 	if (ret) {
diff --git a/drivers/net/igc/igc_ethdev.c b/drivers/net/igc/igc_ethdev.c
index 11397c5c1d..9d27fc0d07 100644
--- a/drivers/net/igc/igc_ethdev.c
+++ b/drivers/net/igc/igc_ethdev.c
@@ -1323,11 +1323,6 @@ eth_igc_dev_init(struct rte_eth_dev *dev)
 		goto err_late;
 	}
 
-	/* Pass the information to the rte_eth_dev_close() that it should also
-	 * release the private port resources.
-	 */
-	dev->data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
-
 	hw->mac.get_link_status = 1;
 	igc->stopped = 0;
 
diff --git a/drivers/net/ixgbe/ixgbe_ethdev.c b/drivers/net/ixgbe/ixgbe_ethdev.c
index a1369a0744..426b7c9fe5 100644
--- a/drivers/net/ixgbe/ixgbe_ethdev.c
+++ b/drivers/net/ixgbe/ixgbe_ethdev.c
@@ -1248,11 +1248,6 @@ eth_ixgbe_dev_init(struct rte_eth_dev *eth_dev, void *init_params __rte_unused)
 		return -ENOMEM;
 	}
 
-	/* Pass the information to the rte_eth_dev_close() that it should also
-	 * release the private port resources.
-	 */
-	eth_dev->data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
-
 	/* initialize the vfta */
 	memset(shadow_vfta, 0, sizeof(*shadow_vfta));
 
@@ -1663,11 +1658,6 @@ eth_ixgbevf_dev_init(struct rte_eth_dev *eth_dev)
 		return -ENOMEM;
 	}
 
-	/* Pass the information to the rte_eth_dev_close() that it should also
-	 * release the private port resources.
-	 */
-	eth_dev->data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
-
 	/* Generate a random MAC address, if none was assigned by PF. */
 	if (rte_is_zero_ether_addr(perm_addr)) {
 		generate_random_mac_addr(perm_addr);
diff --git a/drivers/net/kni/rte_eth_kni.c b/drivers/net/kni/rte_eth_kni.c
index aa365a57de..45ab1b17a8 100644
--- a/drivers/net/kni/rte_eth_kni.c
+++ b/drivers/net/kni/rte_eth_kni.c
@@ -387,8 +387,6 @@ eth_kni_create(struct rte_vdev_device *vdev,
 	data->promiscuous = 1;
 	data->all_multicast = 1;
 
-	data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
-
 	rte_eth_random_addr(internals->eth_addr.addr_bytes);
 
 	eth_dev->dev_ops = &eth_kni_ops;
diff --git a/drivers/net/memif/rte_eth_memif.c b/drivers/net/memif/rte_eth_memif.c
index 1a2411c838..ff8a58081f 100644
--- a/drivers/net/memif/rte_eth_memif.c
+++ b/drivers/net/memif/rte_eth_memif.c
@@ -1527,9 +1527,6 @@ memif_create(struct rte_vdev_device *vdev, enum memif_role_t role,
 		eth_dev->tx_pkt_burst = eth_memif_tx;
 	}
 
-
-	eth_dev->data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
-
 	rte_eth_dev_probing_finish(eth_dev);
 
 	return 0;
diff --git a/drivers/net/mlx4/mlx4.c b/drivers/net/mlx4/mlx4.c
index ad7c805d67..3e57875414 100644
--- a/drivers/net/mlx4/mlx4.c
+++ b/drivers/net/mlx4/mlx4.c
@@ -1027,7 +1027,6 @@ mlx4_pci_probe(struct rte_pci_driver *pci_drv, struct rte_pci_device *pci_dev)
 			ERROR("can not allocate rte ethdev");
 			goto port_error;
 		}
-		eth_dev->data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
 		eth_dev->data->dev_private = priv;
 		eth_dev->data->mac_addrs = priv->mac;
 		eth_dev->device = &pci_dev->device;
diff --git a/drivers/net/mlx5/linux/mlx5_os.c b/drivers/net/mlx5/linux/mlx5_os.c
index 0511a55a28..8e498314ef 100644
--- a/drivers/net/mlx5/linux/mlx5_os.c
+++ b/drivers/net/mlx5/linux/mlx5_os.c
@@ -1155,8 +1155,6 @@ mlx5_dev_spawn(struct rte_device *dpdk_dev,
 		err = ENOMEM;
 		goto error;
 	}
-	/* Flag to call rte_eth_dev_release_port() in rte_eth_dev_close(). */
-	eth_dev->data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
 	if (priv->representor) {
 		eth_dev->data->dev_flags |= RTE_ETH_DEV_REPRESENTOR;
 		eth_dev->data->representor_id = priv->representor_id;
diff --git a/drivers/net/mvneta/mvneta_ethdev.c b/drivers/net/mvneta/mvneta_ethdev.c
index eebcdb840e..db142bec23 100644
--- a/drivers/net/mvneta/mvneta_ethdev.c
+++ b/drivers/net/mvneta/mvneta_ethdev.c
@@ -834,9 +834,6 @@ mvneta_eth_dev_create(struct rte_vdev_device *vdev, const char *name)
 	mvneta_set_tx_function(eth_dev);
 	eth_dev->dev_ops = &mvneta_ops;
 
-	/* Flag to call rte_eth_dev_release_port() in rte_eth_dev_close(). */
-	eth_dev->data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
-
 	rte_eth_dev_probing_finish(eth_dev);
 	return 0;
 out_free:
diff --git a/drivers/net/mvpp2/mrvl_ethdev.c b/drivers/net/mvpp2/mrvl_ethdev.c
index 27f7cfb056..cfb97e4f8f 100644
--- a/drivers/net/mvpp2/mrvl_ethdev.c
+++ b/drivers/net/mvpp2/mrvl_ethdev.c
@@ -2863,9 +2863,6 @@ mrvl_eth_dev_create(struct rte_vdev_device *vdev, const char *name)
 	mrvl_set_tx_function(eth_dev);
 	eth_dev->dev_ops = &mrvl_ops;
 
-	/* Flag to call rte_eth_dev_release_port() in rte_eth_dev_close(). */
-	eth_dev->data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
-
 	rte_eth_dev_probing_finish(eth_dev);
 	return 0;
 out_free:
diff --git a/drivers/net/netvsc/hn_ethdev.c b/drivers/net/netvsc/hn_ethdev.c
index 8968036ea3..15d6e9762d 100644
--- a/drivers/net/netvsc/hn_ethdev.c
+++ b/drivers/net/netvsc/hn_ethdev.c
@@ -123,9 +123,6 @@ eth_dev_vmbus_allocate(struct rte_vmbus_device *dev, size_t private_data_size)
 	eth_dev->data->dev_flags |= RTE_ETH_DEV_INTR_LSC;
 	eth_dev->intr_handle = &dev->intr_handle;
 
-	/* allow ethdev to remove on close */
-	eth_dev->data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
-
 	return eth_dev;
 }
 
diff --git a/drivers/net/nfb/nfb_ethdev.c b/drivers/net/nfb/nfb_ethdev.c
index c3c3d003f1..d937ac6922 100644
--- a/drivers/net/nfb/nfb_ethdev.c
+++ b/drivers/net/nfb/nfb_ethdev.c
@@ -457,9 +457,6 @@ nfb_eth_dev_init(struct rte_eth_dev *dev)
 		rte_kvargs_free(kvlist);
 	}
 
-	/* Let rte_eth_dev_close() release the port resources */
-	dev->data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
-
 	/*
 	 * Get number of available DMA RX and TX queues, which is maximum
 	 * number of queues that can be created and store it in private device
diff --git a/drivers/net/null/rte_eth_null.c b/drivers/net/null/rte_eth_null.c
index 84c1d0c951..7c3c76a897 100644
--- a/drivers/net/null/rte_eth_null.c
+++ b/drivers/net/null/rte_eth_null.c
@@ -548,7 +548,6 @@ eth_dev_null_create(struct rte_vdev_device *dev, struct pmd_options *args)
 	data->mac_addrs = &internals->eth_addr;
 	data->promiscuous = 1;
 	data->all_multicast = 1;
-	data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
 
 	eth_dev->dev_ops = &ops;
 
diff --git a/drivers/net/octeontx/octeontx_ethdev.c b/drivers/net/octeontx/octeontx_ethdev.c
index 48ce3e1621..a263f45399 100644
--- a/drivers/net/octeontx/octeontx_ethdev.c
+++ b/drivers/net/octeontx/octeontx_ethdev.c
@@ -1374,7 +1374,6 @@ octeontx_create(struct rte_vdev_device *dev, int port, uint8_t evdev,
 	data->promiscuous = 0;
 	data->all_multicast = 0;
 	data->scattered_rx = 0;
-	data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
 
 	/* Get maximum number of supported MAC entries */
 	max_entries = octeontx_bgx_port_mac_entries_get(nic->port_id);
diff --git a/drivers/net/octeontx2/otx2_ethdev.c b/drivers/net/octeontx2/otx2_ethdev.c
index abe5f03628..581ac89381 100644
--- a/drivers/net/octeontx2/otx2_ethdev.c
+++ b/drivers/net/octeontx2/otx2_ethdev.c
@@ -2394,7 +2394,6 @@ otx2_eth_dev_init(struct rte_eth_dev *eth_dev)
 	pci_dev = RTE_ETH_DEV_TO_PCI(eth_dev);
 
 	rte_eth_copy_pci_info(eth_dev, pci_dev);
-	eth_dev->data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
 
 	/* Zero out everything after OTX2_DEV to allow proper dev_reset() */
 	memset(&dev->otx2_eth_dev_data_start, 0, sizeof(*dev) -
diff --git a/drivers/net/pcap/rte_eth_pcap.c b/drivers/net/pcap/rte_eth_pcap.c
index 909eef8cce..da4988064a 100644
--- a/drivers/net/pcap/rte_eth_pcap.c
+++ b/drivers/net/pcap/rte_eth_pcap.c
@@ -1334,7 +1334,6 @@ eth_from_pcaps(struct rte_vdev_device *vdev,
 	else
 		eth_dev->tx_pkt_burst = eth_tx_drop;
 
-	eth_dev->data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
 	rte_eth_dev_probing_finish(eth_dev);
 	return 0;
 }
diff --git a/drivers/net/qede/qede_ethdev.c b/drivers/net/qede/qede_ethdev.c
index 43fe68c2bf..cd578709fb 100644
--- a/drivers/net/qede/qede_ethdev.c
+++ b/drivers/net/qede/qede_ethdev.c
@@ -2711,11 +2711,6 @@ static int qede_common_dev_init(struct rte_eth_dev *eth_dev, bool is_vf)
 		adapter->ipgre.enable = false;
 	}
 
-	/* Pass the information to the rte_eth_dev_close() that it should also
-	 * release the private port resources.
-	 */
-	eth_dev->data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
-
 	DP_INFO(edev, "MAC address : %02x:%02x:%02x:%02x:%02x:%02x\n",
 		adapter->primary_mac.addr_bytes[0],
 		adapter->primary_mac.addr_bytes[1],
diff --git a/drivers/net/ring/rte_eth_ring.c b/drivers/net/ring/rte_eth_ring.c
index ed0fdeb28f..12046f5a00 100644
--- a/drivers/net/ring/rte_eth_ring.c
+++ b/drivers/net/ring/rte_eth_ring.c
@@ -358,7 +358,6 @@ do_eth_dev_ring_create(const char *name,
 
 	eth_dev->dev_ops = &ops;
 	data->numa_node = numa_node;
-	data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
 
 	/* finally assign rx and tx ops */
 	eth_dev->rx_pkt_burst = eth_ring_rx;
diff --git a/drivers/net/sfc/sfc_ethdev.c b/drivers/net/sfc/sfc_ethdev.c
index eaa48627d6..1c72e40b75 100644
--- a/drivers/net/sfc/sfc_ethdev.c
+++ b/drivers/net/sfc/sfc_ethdev.c
@@ -343,7 +343,7 @@ sfc_dev_close(struct rte_eth_dev *dev)
 	}
 
 	/*
-	 * Cleanup all resources in accordance with RTE_ETH_DEV_CLOSE_REMOVE.
+	 * Cleanup all resources.
 	 * Rollback primary process sfc_eth_dev_init() below.
 	 */
 
@@ -2179,8 +2179,6 @@ sfc_eth_dev_init(struct rte_eth_dev *dev)
 
 	sfc_log_init(sa, "entry");
 
-	dev->data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
-
 	dev->data->mac_addrs = rte_zmalloc("sfc", RTE_ETHER_ADDR_LEN, 0);
 	if (dev->data->mac_addrs == NULL) {
 		rc = ENOMEM;
diff --git a/drivers/net/softnic/rte_eth_softnic.c b/drivers/net/softnic/rte_eth_softnic.c
index ad9e2aa86d..e942df78b6 100644
--- a/drivers/net/softnic/rte_eth_softnic.c
+++ b/drivers/net/softnic/rte_eth_softnic.c
@@ -390,7 +390,6 @@ pmd_ethdev_register(struct rte_vdev_device *vdev,
 	dev->device = &vdev->device;
 
 	/* dev->data */
-	dev->data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
 	dev->data->dev_private = dev_private;
 	dev->data->dev_link.link_speed = ETH_SPEED_NUM_100G;
 	dev->data->dev_link.link_duplex = ETH_LINK_FULL_DUPLEX;
diff --git a/drivers/net/szedata2/rte_eth_szedata2.c b/drivers/net/szedata2/rte_eth_szedata2.c
index daa40cfd6c..4325b9a30d 100644
--- a/drivers/net/szedata2/rte_eth_szedata2.c
+++ b/drivers/net/szedata2/rte_eth_szedata2.c
@@ -1488,9 +1488,6 @@ rte_szedata2_eth_dev_init(struct rte_eth_dev *dev, struct port_info *pi)
 	PMD_INIT_LOG(INFO, "Initializing eth_dev %s (driver %s)", data->name,
 			RTE_STR(RTE_SZEDATA2_DRIVER_NAME));
 
-	/* Let rte_eth_dev_close() release the port resources */
-	dev->data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
-
 	/* Fill internal private structure. */
 	internals->dev = dev;
 	/* Get index of szedata2 device file and create path to device file */
diff --git a/drivers/net/tap/rte_eth_tap.c b/drivers/net/tap/rte_eth_tap.c
index 0d217f1486..b127ce62dc 100644
--- a/drivers/net/tap/rte_eth_tap.c
+++ b/drivers/net/tap/rte_eth_tap.c
@@ -1922,7 +1922,7 @@ eth_dev_tap_create(struct rte_vdev_device *vdev, const char *tap_name,
 	/* Setup some default values */
 	data = dev->data;
 	data->dev_private = pmd;
-	data->dev_flags = RTE_ETH_DEV_INTR_LSC | RTE_ETH_DEV_CLOSE_REMOVE;
+	data->dev_flags = RTE_ETH_DEV_INTR_LSC;
 	data->numa_node = numa_node;
 
 	data->dev_link = pmd_link;
diff --git a/drivers/net/vhost/rte_eth_vhost.c b/drivers/net/vhost/rte_eth_vhost.c
index f67f4db812..45552ef742 100644
--- a/drivers/net/vhost/rte_eth_vhost.c
+++ b/drivers/net/vhost/rte_eth_vhost.c
@@ -1443,7 +1443,7 @@ eth_dev_vhost_create(struct rte_vdev_device *dev, char *iface_name,
 	internal->flags = flags;
 	internal->disable_flags = disable_flags;
 	data->dev_link = pmd_link;
-	data->dev_flags = RTE_ETH_DEV_INTR_LSC | RTE_ETH_DEV_CLOSE_REMOVE;
+	data->dev_flags = RTE_ETH_DEV_INTR_LSC;
 	data->promiscuous = 1;
 	data->all_multicast = 1;
 
diff --git a/drivers/net/virtio/virtio_ethdev.c b/drivers/net/virtio/virtio_ethdev.c
index 0787337a01..b6ed5829bd 100644
--- a/drivers/net/virtio/virtio_ethdev.c
+++ b/drivers/net/virtio/virtio_ethdev.c
@@ -1924,11 +1924,6 @@ eth_virtio_dev_init(struct rte_eth_dev *eth_dev)
 	if (ret < 0)
 		return ret;
 	hw->speed = speed;
-	/*
-	 * Pass the information to the rte_eth_dev_close() that it should also
-	 * release the private port resources.
-	 */
-	eth_dev->data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
 
 	/* Allocate memory for storing MAC addresses */
 	eth_dev->data->mac_addrs = rte_zmalloc("virtio",
diff --git a/drivers/net/vmxnet3/vmxnet3_ethdev.c b/drivers/net/vmxnet3/vmxnet3_ethdev.c
index 17fed0ed6e..34a169d2c0 100644
--- a/drivers/net/vmxnet3/vmxnet3_ethdev.c
+++ b/drivers/net/vmxnet3/vmxnet3_ethdev.c
@@ -321,9 +321,6 @@ eth_vmxnet3_dev_init(struct rte_eth_dev *eth_dev)
 		     hw->perm_addr[0], hw->perm_addr[1], hw->perm_addr[2],
 		     hw->perm_addr[3], hw->perm_addr[4], hw->perm_addr[5]);
 
-	/* Flag to call rte_eth_dev_release_port() in rte_eth_dev_close(). */
-	eth_dev->data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
-
 	/* Put device in Quiesce Mode */
 	VMXNET3_WRITE_BAR1_REG(hw, VMXNET3_REG_CMD, VMXNET3_CMD_QUIESCE_DEV);
 
diff --git a/lib/librte_ethdev/rte_ethdev.c b/lib/librte_ethdev/rte_ethdev.c
index d12d74dd8c..d7668114ca 100644
--- a/lib/librte_ethdev/rte_ethdev.c
+++ b/lib/librte_ethdev/rte_ethdev.c
@@ -1720,22 +1720,7 @@ rte_eth_dev_close(uint16_t port_id)
 	(*dev->dev_ops->dev_close)(dev);
 
 	rte_ethdev_trace_close(port_id);
-	/* check behaviour flag - temporary for PMD migration */
-	if ((dev->data->dev_flags & RTE_ETH_DEV_CLOSE_REMOVE) != 0) {
-		/* new behaviour: send event + reset state + free all data */
-		rte_eth_dev_release_port(dev);
-		return;
-	}
-	RTE_ETHDEV_LOG(DEBUG, "Port closing is using an old behaviour.\n"
-			"The driver %s should migrate to the new behaviour.\n",
-			dev->device->driver->name);
-	/* old behaviour: only free queue arrays */
-	dev->data->nb_rx_queues = 0;
-	rte_free(dev->data->rx_queues);
-	dev->data->rx_queues = NULL;
-	dev->data->nb_tx_queues = 0;
-	rte_free(dev->data->tx_queues);
-	dev->data->tx_queues = NULL;
+	rte_eth_dev_release_port(dev);
 }
 
 int
diff --git a/lib/librte_ethdev/rte_ethdev.h b/lib/librte_ethdev/rte_ethdev.h
index 645a18664d..24d898ae89 100644
--- a/lib/librte_ethdev/rte_ethdev.h
+++ b/lib/librte_ethdev/rte_ethdev.h
@@ -1654,11 +1654,6 @@ struct rte_eth_dev_owner {
 	char name[RTE_ETH_MAX_OWNER_NAME_LEN]; /**< The owner name. */
 };
 
-/**
- * Port is released (i.e. totally freed and data erased) on close.
- * Temporary flag for PMD migration to new rte_eth_dev_close() behaviour.
- */
-#define RTE_ETH_DEV_CLOSE_REMOVE 0x0001
 /** Device supports link state interrupt */
 #define RTE_ETH_DEV_INTR_LSC     0x0002
 /** Device is a bonded slave */
@@ -2282,8 +2277,7 @@ int rte_eth_dev_set_link_down(uint16_t port_id);
 
 /**
  * Close a stopped Ethernet device. The device cannot be restarted!
- * The function frees all port resources if the driver supports
- * the flag RTE_ETH_DEV_CLOSE_REMOVE.
+ * The function frees all port resources.
  *
  * @param port_id
  *   The port identifier of the Ethernet device.
-- 
2.28.0

Re: [dpdk-dev] [EXT] [PATCH v2 17/25] ethdev: remove old close behaviour

[Liron Himi]

For mvpp2, mvneta.

Reviewed-by: Liron Himi<lironh@marvell.com>

-----Original Message-----
From: Thomas Monjalon <thomas@monjalon.net> 
Sent: Monday, 28 September 2020 02:43
To: dev@dpdk.org
Cc: ferruh.yigit@intel.com; arybchenko@solarflare.com; Shijith Thotton <sthotton@marvell.com>; Srisivasubramanian Srinivasan <srinivasan@marvell.com>; Heinrich Kuhn <heinrich.kuhn@netronome.com>; Hemant Agrawal <hemant.agrawal@nxp.com>; Sachin Saxena <sachin.saxena@nxp.com>; Gagandeep Singh <g.singh@nxp.com>; Akhil Goyal <akhil.goyal@nxp.com>; Alfredo Cardigliano <cardigliano@ntop.org>; Ray Kinsella <mdr@ashroe.eu>; Neil Horman <nhorman@tuxdriver.com>; John W. Linville <linville@tuxdriver.com>; Ciara Loftus <ciara.loftus@intel.com>; Qi Zhang <qi.z.zhang@intel.com>; Shepard Siegel <shepard.siegel@atomicrules.com>; Ed Czeck <ed.czeck@atomicrules.com>; John Miller <john.miller@atomicrules.com>; Igor Russkikh <igor.russkikh@aquantia.com>; Pavel Belous <pavel.belous@aquantia.com>; Steven Webster <steven.webster@windriver.com>; Matt Peters <matt.peters@windriver.com>; Somalapuram Amaranath <asomalap@amd.com>; Rasesh Mody <rmody@marvell.com>; Shahed Shaikh <shshaikh@marvell.com>; Ajit Khaparde <ajit.khaparde@broadcom.com>; Somnath Kotur <somnath.kotur@broadcom.com>; Chas Williams <chas3@att.com>; Wei Hu (Xavier) <xavier.huwei@huawei.com>; Rahul Lakkireddy <rahul.lakkireddy@chelsio.com>; Jeff Guo <jia.guo@intel.com>; Haiyue Wang <haiyue.wang@intel.com>; Marcin Wojtas <mw@semihalf.com>; Michal Krawczyk <mk@semihalf.com>; Guy Tzalik <gtzalik@amazon.com>; Evgeny Schemeilin <evgenys@amazon.com>; Igor Chauskin <igorch@amazon.com>; John Daley <johndale@cisco.com>; Hyong Youb Kim <hyonkim@cisco.com>; Gaetan Rivet <grive@u256.net>; Xiao Wang <xiao.w.wang@intel.com>; Ziyang Xuan <xuanziyang2@huawei.com>; Xiaoyun Wang <cloud.wangxiaoyun@huawei.com>; Guoyang Zhou <zhouguoyang@huawei.com>; Min Hu (Connor) <humin29@huawei.com>; Yisen Zhuang <yisen.zhuang@huawei.com>; Beilei Xing <beilei.xing@intel.com>; Jingjing Wu <jingjing.wu@intel.com>; Qiming Yang <qiming.yang@intel.com>; Jakub Grajciar <jgrajcia@cisco.com>; Matan Azrad <matan@nvidia.com>; Shahaf Shuler <shahafs@nvidia.com>; Viacheslav Ovsiienko <viacheslavo@nvidia.com>; Zyta Szpak <zr@semihalf.com>; Liron Himi <lironh@marvell.com>; Stephen Hemminger <sthemmin@microsoft.com>; K. Y. Srinivasan <kys@microsoft.com>; Haiyang Zhang <haiyangz@microsoft.com>; Long Li <longli@microsoft.com>; Martin Spinler <spinler@cesnet.cz>; Tetsuya Mukawa <mtetsuyah@gmail.com>; Harman Kalra <hkalra@marvell.com>; Jerin Jacob Kollanukkaran <jerinj@marvell.com>; Nithin Kumar Dabilpuram <ndabilpuram@marvell.com>; Kiran Kumar Kokkilagadda <kirankumark@marvell.com>; Bruce Richardson <bruce.richardson@intel.com>; Jasvinder Singh <jasvinder.singh@intel.com>; Cristian Dumitrescu <cristian.dumitrescu@intel.com>; Keith Wiles <keith.wiles@intel.com>; Maxime Coquelin <maxime.coquelin@redhat.com>; Chenbo Xia <chenbo.xia@intel.com>; Zhihong Wang <zhihong.wang@intel.com>; Yong Wang <yongwang@vmware.com>
Subject: [EXT] [PATCH v2 17/25] ethdev: remove old close behaviour

External Email

----------------------------------------------------------------------
The temporary flag RTE_ETH_DEV_CLOSE_REMOVE is removed.
It was introduced in DPDK 18.11 in order to give time for PMDs to migrate.

The old behaviour was to free only queues when closing a port.
The new behaviour is calling rte_eth_dev_release_port() which does three more tasks:
	- trigger event callback
	- reset state and few pointers
	- free all generic port resources

The private port resources must be released in the .dev_close callback.

The .remove callback should:
	- call .dev_close callback
	- call rte_eth_dev_release_port()
	- free multi-port device shared resources

Despite waiting two years, some drivers have not migrated, so they may hit issues with the incompatible new behaviour.
After sending emails, adding logs, and announcing the deprecation, the only last solution is to declare these drivers as unmaintained:
	dpaa, dpaa2, enetc, ionic, liquidio, nfp, pfe Below is a summary of what to implement in those drivers.

* The freeing of private port resources must be moved from the ".remove(device)" function to the ".dev_close(port)" function.

* If a generic resource (.mac_addrs or .hash_mac_addrs) cannot be freed, it must be set to NULL in ".dev_close" function to protect from subsequent rte_eth_dev_release_port() freeing.

* Note 1:
The generic resources are freed in rte_eth_dev_release_port(), after ".dev_close" is called in rte_eth_dev_close(), but not when calling ".dev_close" directly from the ".remove" PMD function.
That's why rte_eth_dev_release_port() must still be called explicitly from ".remove(device)" after calling the ".dev_close" PMD function.

* Note 2:
If a device can have multiple ports, the common resources must be freed only in the ".remove(device)" function.

* Note 3:
The port is supposed to be in a stopped state when it is closed.
If it is not the case, it is free to the PMD implementation how to react when trying to close a non-stopped port:
either try to stop it automatically or just return an error.

Cc: Shijith Thotton <sthotton@marvell.com>
Cc: Srisivasubramanian Srinivasan <srinivasan@marvell.com>
Cc: Heinrich Kuhn <heinrich.kuhn@netronome.com>
Cc: Hemant Agrawal <hemant.agrawal@nxp.com>
Cc: Sachin Saxena <sachin.saxena@nxp.com>
Cc: Gagandeep Singh <g.singh@nxp.com>
Cc: Akhil Goyal <akhil.goyal@nxp.com>
Cc: Alfredo Cardigliano <cardigliano@ntop.org>

Signed-off-by: Thomas Monjalon <thomas@monjalon.net>
---
 MAINTAINERS                               | 14 +++++++-------
 doc/guides/rel_notes/deprecation.rst      |  6 ------
 drivers/net/af_packet/rte_eth_af_packet.c |  1 -
 drivers/net/af_xdp/rte_eth_af_xdp.c       |  2 --
 drivers/net/ark/ark_ethdev.c              |  2 --
 drivers/net/atlantic/atl_ethdev.c         |  2 --
 drivers/net/avp/avp_ethdev.c              |  2 --
 drivers/net/axgbe/axgbe_ethdev.c          |  1 -
 drivers/net/bnx2x/bnx2x_ethdev.c          |  5 -----
 drivers/net/bnxt/bnxt_ethdev.c            |  5 -----
 drivers/net/bnxt/bnxt_reps.c              |  4 ----
 drivers/net/bonding/rte_eth_bond_pmd.c    |  1 -
 drivers/net/cxgbe/cxgbe_main.c            |  2 --
 drivers/net/cxgbe/cxgbevf_main.c          |  2 --
 drivers/net/e1000/em_ethdev.c             |  5 -----
 drivers/net/e1000/igb_ethdev.c            | 10 ----------
 drivers/net/ena/ena_ethdev.c              |  6 ------
 drivers/net/enic/enic_ethdev.c            |  2 --
 drivers/net/enic/enic_vf_representor.c    |  3 +--
 drivers/net/failsafe/failsafe.c           |  1 -
 drivers/net/fm10k/fm10k_ethdev.c          |  5 -----
 drivers/net/hinic/hinic_pmd_ethdev.c      |  6 ------
 drivers/net/hns3/hns3_ethdev.c            |  5 -----
 drivers/net/hns3/hns3_ethdev_vf.c         |  5 -----
 drivers/net/i40e/i40e_ethdev.c            |  5 -----
 drivers/net/i40e/i40e_ethdev_vf.c         |  5 -----
 drivers/net/iavf/iavf_ethdev.c            |  5 -----
 drivers/net/ice/ice_dcf_ethdev.c          |  2 --
 drivers/net/ice/ice_ethdev.c              |  5 -----
 drivers/net/igc/igc_ethdev.c              |  5 -----
 drivers/net/ixgbe/ixgbe_ethdev.c          | 10 ----------
 drivers/net/kni/rte_eth_kni.c             |  2 --
 drivers/net/memif/rte_eth_memif.c         |  3 ---
 drivers/net/mlx4/mlx4.c                   |  1 -
 drivers/net/mlx5/linux/mlx5_os.c          |  2 --
 drivers/net/mvneta/mvneta_ethdev.c        |  3 ---
 drivers/net/mvpp2/mrvl_ethdev.c           |  3 ---
 drivers/net/netvsc/hn_ethdev.c            |  3 ---
 drivers/net/nfb/nfb_ethdev.c              |  3 ---
 drivers/net/null/rte_eth_null.c           |  1 -
 drivers/net/octeontx/octeontx_ethdev.c    |  1 -
 drivers/net/octeontx2/otx2_ethdev.c       |  1 -
 drivers/net/pcap/rte_eth_pcap.c           |  1 -
 drivers/net/qede/qede_ethdev.c            |  5 -----
 drivers/net/ring/rte_eth_ring.c           |  1 -
 drivers/net/sfc/sfc_ethdev.c              |  4 +---
 drivers/net/softnic/rte_eth_softnic.c     |  1 -
 drivers/net/szedata2/rte_eth_szedata2.c   |  3 ---
 drivers/net/tap/rte_eth_tap.c             |  2 +-
 drivers/net/vhost/rte_eth_vhost.c         |  2 +-
 drivers/net/virtio/virtio_ethdev.c        |  5 -----
 drivers/net/vmxnet3/vmxnet3_ethdev.c      |  3 ---
 lib/librte_ethdev/rte_ethdev.c            | 17 +----------------
 lib/librte_ethdev/rte_ethdev.h            |  8 +-------
 54 files changed, 13 insertions(+), 196 deletions(-)

diff --git a/MAINTAINERS b/MAINTAINERS
index 49a6dfa7a5..71921675ab 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -604,7 +604,7 @@ F: drivers/net/thunderx/
 F: doc/guides/nics/thunderx.rst
 F: doc/guides/nics/features/thunderx.ini
 
-Cavium LiquidIO
+Cavium LiquidIO - UNMAINTAINED
 M: Shijith Thotton <sthotton@marvell.com>
 M: Srisivasubramanian Srinivasan <srinivasan@marvell.com>
 T: git://dpdk.org/next/dpdk-next-net-mrvl
@@ -793,13 +793,13 @@ F: drivers/net/nfb/
 F: doc/guides/nics/nfb.rst
 F: doc/guides/nics/features/nfb.ini
 
-Netronome nfp
+Netronome nfp - UNMAINTAINED
 M: Heinrich Kuhn <heinrich.kuhn@netronome.com>
 F: drivers/net/nfp/
 F: doc/guides/nics/nfp.rst
 F: doc/guides/nics/features/nfp*.ini
 
-NXP dpaa
+NXP dpaa - UNMAINTAINED
 M: Hemant Agrawal <hemant.agrawal@nxp.com>
 M: Sachin Saxena <sachin.saxena@oss.nxp.com>
 F: drivers/mempool/dpaa/
@@ -807,7 +807,7 @@ F: drivers/net/dpaa/
 F: doc/guides/nics/dpaa.rst
 F: doc/guides/nics/features/dpaa.ini
 
-NXP dpaa2
+NXP dpaa2 - UNMAINTAINED
 M: Hemant Agrawal <hemant.agrawal@nxp.com>
 M: Sachin Saxena <sachin.saxena@oss.nxp.com>
 F: drivers/mempool/dpaa2/
@@ -815,21 +815,21 @@ F: drivers/net/dpaa2/
 F: doc/guides/nics/dpaa2.rst
 F: doc/guides/nics/features/dpaa2.ini
 
-NXP enetc
+NXP enetc - UNMAINTAINED
 M: Gagandeep Singh <g.singh@nxp.com>
 M: Sachin Saxena <sachin.saxena@oss.nxp.com>
 F: drivers/net/enetc/
 F: doc/guides/nics/enetc.rst
 F: doc/guides/nics/features/enetc.ini
 
-NXP pfe
+NXP pfe - UNMAINTAINED
 M: Gagandeep Singh <g.singh@nxp.com>
 M: Akhil Goyal <akhil.goyal@nxp.com>
 F: doc/guides/nics/pfe.rst
 F: drivers/net/pfe/
 F: doc/guides/nics/features/pfe.ini
 
-Pensando ionic
+Pensando ionic - UNMAINTAINED
 M: Alfredo Cardigliano <cardigliano@ntop.org>
 F: drivers/net/ionic/
 F: doc/guides/nics/ionic.rst
diff --git a/doc/guides/rel_notes/deprecation.rst b/doc/guides/rel_notes/deprecation.rst
index 67caedb11b..a907358078 100644
--- a/doc/guides/rel_notes/deprecation.rst
+++ b/doc/guides/rel_notes/deprecation.rst
@@ -142,12 +142,6 @@ Deprecation Notices
   - ``rte_eth_dev_stop``
   - ``rte_eth_dev_close``
 
-* ethdev: The temporary flag RTE_ETH_DEV_CLOSE_REMOVE will be removed in 20.11.
-  As a consequence, the new behaviour introduced in 18.11 will be effective
-  for all drivers: generic port resources are freed on close operation.
-  Private resources are expected to be released in the ``dev_close`` callback.
-  More details in https://urldefense.proofpoint.com/v2/url?u=http-3A__inbox.dpdk.org_dev_5248162.j6AOsuQRmx-40thomas_&d=DwIDAg&c=nKjWec2b6R0mOyPaz7xtfQ&r=M6lEONY3jVnAkBjt0yC6oAsT7Jn2HI7LhNHCQYlnx1Q&m=QncMCWuK2mCmjDK6KuQ0SIksIJuMUZrd7AB4hjpCmog&s=h2LLe8xxafiu6df0c3gMeMYu7tQtP_x3abE-uz73nSI&e=
-
 * ethdev: New offload flags ``DEV_RX_OFFLOAD_FLOW_MARK`` will be added in 19.11.
   This will allow application to enable or disable PMDs from updating
   ``rte_mbuf::hash::fdir``.
diff --git a/drivers/net/af_packet/rte_eth_af_packet.c b/drivers/net/af_packet/rte_eth_af_packet.c
index 12c202cba4..25876224f8 100644
--- a/drivers/net/af_packet/rte_eth_af_packet.c
+++ b/drivers/net/af_packet/rte_eth_af_packet.c
@@ -859,7 +859,6 @@ rte_pmd_init_internals(struct rte_vdev_device *dev,
 	data->nb_tx_queues = (uint16_t)nb_queues;
 	data->dev_link = pmd_link;
 	data->mac_addrs = &(*internals)->eth_addr;
-	data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
 
 	(*eth_dev)->dev_ops = &ops;
 
diff --git a/drivers/net/af_xdp/rte_eth_af_xdp.c b/drivers/net/af_xdp/rte_eth_af_xdp.c
index badbce63fb..60add9ead4 100644
--- a/drivers/net/af_xdp/rte_eth_af_xdp.c
+++ b/drivers/net/af_xdp/rte_eth_af_xdp.c
@@ -1274,8 +1274,6 @@ init_internals(struct rte_vdev_device *dev, const char *if_name,
 	eth_dev->dev_ops = &ops;
 	eth_dev->rx_pkt_burst = eth_af_xdp_rx;
 	eth_dev->tx_pkt_burst = eth_af_xdp_tx;
-	/* Let rte_eth_dev_close() release the port resources. */
-	eth_dev->data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
 
 #if defined(XDP_UMEM_UNALIGNED_CHUNK_FLAG)
 	AF_XDP_LOG(INFO, "Zero copy between umem and mbuf enabled.\n"); diff --git a/drivers/net/ark/ark_ethdev.c b/drivers/net/ark/ark_ethdev.c index 5f2ed4b7b7..83dc4ecd2c 100644
--- a/drivers/net/ark/ark_ethdev.c
+++ b/drivers/net/ark/ark_ethdev.c
@@ -260,8 +260,6 @@ eth_ark_dev_init(struct rte_eth_dev *dev)
 	/* Use dummy function until setup */
 	dev->rx_pkt_burst = &eth_ark_recv_pkts_noop;
 	dev->tx_pkt_burst = &eth_ark_xmit_pkts_noop;
-	/* Let rte_eth_dev_close() release the port resources */
-	dev->data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
 
 	ark->bar0 = (uint8_t *)pci_dev->mem_resource[0].addr;
 	ark->a_bar = (uint8_t *)pci_dev->mem_resource[2].addr; diff --git a/drivers/net/atlantic/atl_ethdev.c b/drivers/net/atlantic/atl_ethdev.c
index d3babeff94..2217511ca0 100644
--- a/drivers/net/atlantic/atl_ethdev.c
+++ b/drivers/net/atlantic/atl_ethdev.c
@@ -380,8 +380,6 @@ eth_atl_dev_init(struct rte_eth_dev *eth_dev)
 	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
 		return 0;
 
-	eth_dev->data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
-
 	/* Vendor and Device ID need to be set before init of shared code */
 	hw->device_id = pci_dev->id.device_id;
 	hw->vendor_id = pci_dev->id.vendor_id; diff --git a/drivers/net/avp/avp_ethdev.c b/drivers/net/avp/avp_ethdev.c index 2eed6acc3f..c730b7ab86 100644
--- a/drivers/net/avp/avp_ethdev.c
+++ b/drivers/net/avp/avp_ethdev.c
@@ -957,8 +957,6 @@ eth_avp_dev_init(struct rte_eth_dev *eth_dev)
 	eth_dev->dev_ops = &avp_eth_dev_ops;
 	eth_dev->rx_pkt_burst = &avp_recv_pkts;
 	eth_dev->tx_pkt_burst = &avp_xmit_pkts;
-	/* Let rte_eth_dev_close() release the port resources */
-	eth_dev->data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
 
 	if (rte_eal_process_type() != RTE_PROC_PRIMARY) {
 		/*
diff --git a/drivers/net/axgbe/axgbe_ethdev.c b/drivers/net/axgbe/axgbe_ethdev.c
index 2dd64180ad..cf085487cc 100644
--- a/drivers/net/axgbe/axgbe_ethdev.c
+++ b/drivers/net/axgbe/axgbe_ethdev.c
@@ -1959,7 +1959,6 @@ eth_axgbe_dev_init(struct rte_eth_dev *eth_dev)
 	int ret;
 
 	eth_dev->dev_ops = &axgbe_eth_dev_ops;
-	eth_dev->data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
 
 	eth_dev->rx_descriptor_status = axgbe_dev_rx_descriptor_status;
 	eth_dev->tx_descriptor_status = axgbe_dev_tx_descriptor_status; diff --git a/drivers/net/bnx2x/bnx2x_ethdev.c b/drivers/net/bnx2x/bnx2x_ethdev.c
index 8dc46384af..40225b2f44 100644
--- a/drivers/net/bnx2x/bnx2x_ethdev.c
+++ b/drivers/net/bnx2x/bnx2x_ethdev.c
@@ -735,11 +735,6 @@ bnx2x_common_dev_init(struct rte_eth_dev *eth_dev, int is_vf)
 			goto out;
 	}
 
-	/* Pass the information to the rte_eth_dev_close() that it should also
-	 * release the private port resources.
-	 */
-	eth_dev->data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
-
 	return 0;
 
 out:
diff --git a/drivers/net/bnxt/bnxt_ethdev.c b/drivers/net/bnxt/bnxt_ethdev.c index e726d078ec..a77bab661d 100644
--- a/drivers/net/bnxt/bnxt_ethdev.c
+++ b/drivers/net/bnxt/bnxt_ethdev.c
@@ -5802,11 +5802,6 @@ bnxt_dev_init(struct rte_eth_dev *eth_dev, void *params __rte_unused)
 
 	bnxt_alloc_switch_domain(bp);
 
-	/* Pass the information to the rte_eth_dev_close() that it should also
-	 * release the private port resources.
-	 */
-	eth_dev->data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
-
 	PMD_DRV_LOG(INFO,
 		    DRV_MODULE_NAME "found at mem %" PRIX64 ", node addr %pM\n",
 		    pci_dev->mem_resource[0].phys_addr,
diff --git a/drivers/net/bnxt/bnxt_reps.c b/drivers/net/bnxt/bnxt_reps.c index 2b6c0a277b..df8680c113 100644
--- a/drivers/net/bnxt/bnxt_reps.c
+++ b/drivers/net/bnxt/bnxt_reps.c
@@ -203,10 +203,6 @@ int bnxt_vf_representor_init(struct rte_eth_dev *eth_dev, void *params)
 	PMD_DRV_LOG(INFO, "calling bnxt_print_link_info\n");
 	bnxt_print_link_info(eth_dev);
 
-	/* Pass the information to the rte_eth_dev_close() that it should also
-	 * release the private port resources.
-	 */
-	eth_dev->data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
 	PMD_DRV_LOG(INFO,
 		    "Switch domain id %d: Representor Device %d init done\n",
 		    vf_rep_bp->switch_domain_id, vf_rep_bp->vf_id); diff --git a/drivers/net/bonding/rte_eth_bond_pmd.c b/drivers/net/bonding/rte_eth_bond_pmd.c
index a9ed5d8e12..1f761c7c9e 100644
--- a/drivers/net/bonding/rte_eth_bond_pmd.c
+++ b/drivers/net/bonding/rte_eth_bond_pmd.c
@@ -3209,7 +3209,6 @@ bond_alloc(struct rte_vdev_device *dev, uint8_t mode)
 	}
 
 	internals = eth_dev->data->dev_private;
-	eth_dev->data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
 	eth_dev->data->nb_rx_queues = (uint16_t)1;
 	eth_dev->data->nb_tx_queues = (uint16_t)1;
 
diff --git a/drivers/net/cxgbe/cxgbe_main.c b/drivers/net/cxgbe/cxgbe_main.c index da4ae25d47..53b08a64af 100644
--- a/drivers/net/cxgbe/cxgbe_main.c
+++ b/drivers/net/cxgbe/cxgbe_main.c
@@ -2142,8 +2142,6 @@ int cxgbe_probe(struct adapter *adapter)
 			goto out_free;
 		}
 
-		pi->eth_dev->data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
-
 		if (i > 0) {
 			/* First port will be notified by upper layer */
 			rte_eth_dev_probing_finish(eth_dev);
diff --git a/drivers/net/cxgbe/cxgbevf_main.c b/drivers/net/cxgbe/cxgbevf_main.c
index 35d873402b..9ee060504f 100644
--- a/drivers/net/cxgbe/cxgbevf_main.c
+++ b/drivers/net/cxgbe/cxgbevf_main.c
@@ -261,8 +261,6 @@ int cxgbevf_probe(struct adapter *adapter)
 			goto out_free;
 		}
 
-		pi->eth_dev->data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
-
 		if (i > 0) {
 			/* First port will be notified by upper layer */
 			rte_eth_dev_probing_finish(eth_dev);
diff --git a/drivers/net/e1000/em_ethdev.c b/drivers/net/e1000/em_ethdev.c index 2d0e071448..6a6ae0e9d5 100644
--- a/drivers/net/e1000/em_ethdev.c
+++ b/drivers/net/e1000/em_ethdev.c
@@ -297,11 +297,6 @@ eth_em_dev_init(struct rte_eth_dev *eth_dev)
 	rte_ether_addr_copy((struct rte_ether_addr *)hw->mac.addr,
 		eth_dev->data->mac_addrs);
 
-	/* Pass the information to the rte_eth_dev_close() that it should also
-	 * release the private port resources.
-	 */
-	eth_dev->data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
-
 	/* initialize the vfta */
 	memset(shadow_vfta, 0, sizeof(*shadow_vfta));
 
diff --git a/drivers/net/e1000/igb_ethdev.c b/drivers/net/e1000/igb_ethdev.c index 9e4aefe00c..f8778207ef 100644
--- a/drivers/net/e1000/igb_ethdev.c
+++ b/drivers/net/e1000/igb_ethdev.c
@@ -841,11 +841,6 @@ eth_igb_dev_init(struct rte_eth_dev *eth_dev)
 	rte_ether_addr_copy((struct rte_ether_addr *)hw->mac.addr,
 			&eth_dev->data->mac_addrs[0]);
 
-	/* Pass the information to the rte_eth_dev_close() that it should also
-	 * release the private port resources.
-	 */
-	eth_dev->data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
-
 	/* initialize the vfta */
 	memset(shadow_vfta, 0, sizeof(*shadow_vfta));
 
@@ -997,11 +992,6 @@ eth_igbvf_dev_init(struct rte_eth_dev *eth_dev)
 		return -ENOMEM;
 	}
 
-	/* Pass the information to the rte_eth_dev_close() that it should also
-	 * release the private port resources.
-	 */
-	eth_dev->data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
-
 	/* Generate a random MAC address, if none was assigned by PF. */
 	if (rte_is_zero_ether_addr(perm_addr)) {
 		rte_eth_random_addr(perm_addr->addr_bytes);
diff --git a/drivers/net/ena/ena_ethdev.c b/drivers/net/ena/ena_ethdev.c index ae6daad892..cf1c0b9795 100644
--- a/drivers/net/ena/ena_ethdev.c
+++ b/drivers/net/ena/ena_ethdev.c
@@ -1863,12 +1863,6 @@ static int eth_ena_dev_init(struct rte_eth_dev *eth_dev)
 			get_feat_ctx.dev_attr.mac_addr,
 			(struct rte_ether_addr *)adapter->mac_addr);
 
-	/*
-	 * Pass the information to the rte_eth_dev_close() that it should also
-	 * release the private port resources.
-	 */
-	eth_dev->data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
-
 	adapter->drv_stats = rte_zmalloc("adapter stats",
 					 sizeof(*adapter->drv_stats),
 					 RTE_CACHE_LINE_SIZE);
diff --git a/drivers/net/enic/enic_ethdev.c b/drivers/net/enic/enic_ethdev.c index ed03d53608..27f60b45b8 100644
--- a/drivers/net/enic/enic_ethdev.c
+++ b/drivers/net/enic/enic_ethdev.c
@@ -1292,8 +1292,6 @@ static int eth_enic_dev_init(struct rte_eth_dev *eth_dev,
 	enic->port_id = eth_dev->data->port_id;
 	enic->rte_dev = eth_dev;
 	enic->dev_data = eth_dev->data;
-	/* Let rte_eth_dev_close() release the port resources */
-	eth_dev->data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
 
 	pdev = RTE_ETH_DEV_TO_PCI(eth_dev);
 	rte_eth_copy_pci_info(eth_dev, pdev);
diff --git a/drivers/net/enic/enic_vf_representor.c b/drivers/net/enic/enic_vf_representor.c
index c528be2b98..169c611a68 100644
--- a/drivers/net/enic/enic_vf_representor.c
+++ b/drivers/net/enic/enic_vf_representor.c
@@ -670,8 +670,7 @@ int enic_vf_representor_init(struct rte_eth_dev *eth_dev, void *init_params)
 
 	eth_dev->device->driver = pf->rte_dev->device->driver;
 	eth_dev->dev_ops = &enic_vf_representor_dev_ops;
-	eth_dev->data->dev_flags |= RTE_ETH_DEV_REPRESENTOR
-		| RTE_ETH_DEV_CLOSE_REMOVE;
+	eth_dev->data->dev_flags |= RTE_ETH_DEV_REPRESENTOR;
 	eth_dev->data->representor_id = vf->vf_id;
 	eth_dev->data->mac_addrs = rte_zmalloc("enic_mac_addr_vf",
 		sizeof(struct rte_ether_addr) *
diff --git a/drivers/net/failsafe/failsafe.c b/drivers/net/failsafe/failsafe.c index 44d47e8f72..b921e101e6 100644
--- a/drivers/net/failsafe/failsafe.c
+++ b/drivers/net/failsafe/failsafe.c
@@ -180,7 +180,6 @@ fs_eth_dev_create(struct rte_vdev_device *vdev)
 		ERROR("Unable to allocate rte_eth_dev");
 		return -1;
 	}
-	dev->data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
 	priv = PRIV(dev);
 	priv->data = dev->data;
 	priv->rxp = FS_RX_PROXY_INIT;
diff --git a/drivers/net/fm10k/fm10k_ethdev.c b/drivers/net/fm10k/fm10k_ethdev.c
index 533f976709..23f4d04068 100644
--- a/drivers/net/fm10k/fm10k_ethdev.c
+++ b/drivers/net/fm10k/fm10k_ethdev.c
@@ -3129,11 +3129,6 @@ eth_fm10k_dev_init(struct rte_eth_dev *dev)
 		&dev->data->mac_addrs[0]);
 	}
 
-	/* Pass the information to the rte_eth_dev_close() that it should also
-	 * release the private port resources.
-	 */
-	dev->data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
-
 	/* Reset the hw statistics */
 	diag = fm10k_stats_reset(dev);
 	if (diag != 0) {
diff --git a/drivers/net/hinic/hinic_pmd_ethdev.c b/drivers/net/hinic/hinic_pmd_ethdev.c
index 5f2e97d3bd..623534fda4 100644
--- a/drivers/net/hinic/hinic_pmd_ethdev.c
+++ b/drivers/net/hinic/hinic_pmd_ethdev.c
@@ -3100,12 +3100,6 @@ static int hinic_func_init(struct rte_eth_dev *eth_dev)
 		goto mc_addr_fail;
 	}
 
-	/*
-	 * Pass the information to the rte_eth_dev_close() that it should also
-	 * release the private port resources.
-	 */
-	eth_dev->data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
-
 	/* create hardware nic_device */
 	rc = hinic_nic_dev_create(eth_dev);
 	if (rc) {
diff --git a/drivers/net/hns3/hns3_ethdev.c b/drivers/net/hns3/hns3_ethdev.c index 3b395a1ccf..dd03bd62b2 100644
--- a/drivers/net/hns3/hns3_ethdev.c
+++ b/drivers/net/hns3/hns3_ethdev.c
@@ -5678,11 +5678,6 @@ hns3_dev_init(struct rte_eth_dev *eth_dev)
 			    &eth_dev->data->mac_addrs[0]);
 
 	hw->adapter_state = HNS3_NIC_INITIALIZED;
-	/*
-	 * Pass the information to the rte_eth_dev_close() that it should also
-	 * release the private port resources.
-	 */
-	eth_dev->data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
 
 	if (rte_atomic16_read(&hns->hw.reset.schedule) == SCHEDULE_PENDING) {
 		hns3_err(hw, "Reschedule reset service after dev_init"); diff --git a/drivers/net/hns3/hns3_ethdev_vf.c b/drivers/net/hns3/hns3_ethdev_vf.c
index e0ca5a6ac2..436d864b5f 100644
--- a/drivers/net/hns3/hns3_ethdev_vf.c
+++ b/drivers/net/hns3/hns3_ethdev_vf.c
@@ -2684,11 +2684,6 @@ hns3vf_dev_init(struct rte_eth_dev *eth_dev)
 			    &eth_dev->data->mac_addrs[0]);
 
 	hw->adapter_state = HNS3_NIC_INITIALIZED;
-	/*
-	 * Pass the information to the rte_eth_dev_close() that it should also
-	 * release the private port resources.
-	 */
-	eth_dev->data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
 
 	if (rte_atomic16_read(&hns->hw.reset.schedule) == SCHEDULE_PENDING) {
 		hns3_err(hw, "Reschedule reset service after dev_init"); diff --git a/drivers/net/i40e/i40e_ethdev.c b/drivers/net/i40e/i40e_ethdev.c index 804f0128ab..6fb88148b8 100644
--- a/drivers/net/i40e/i40e_ethdev.c
+++ b/drivers/net/i40e/i40e_ethdev.c
@@ -1699,11 +1699,6 @@ eth_i40e_dev_init(struct rte_eth_dev *dev, void *init_params __rte_unused)
 	rte_ether_addr_copy((struct rte_ether_addr *)hw->mac.perm_addr,
 					&dev->data->mac_addrs[0]);
 
-	/* Pass the information to the rte_eth_dev_close() that it should also
-	 * release the private port resources.
-	 */
-	dev->data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
-
 	/* Init dcb to sw mode by default */
 	ret = i40e_dcb_init_configure(dev, TRUE);
 	if (ret != I40E_SUCCESS) {
diff --git a/drivers/net/i40e/i40e_ethdev_vf.c b/drivers/net/i40e/i40e_ethdev_vf.c
index 61aad8c415..4aaf41956c 100644
--- a/drivers/net/i40e/i40e_ethdev_vf.c
+++ b/drivers/net/i40e/i40e_ethdev_vf.c
@@ -1586,11 +1586,6 @@ i40evf_dev_init(struct rte_eth_dev *eth_dev)
 	hw->adapter_stopped = 1;
 	hw->adapter_closed = 0;
 
-	/* Pass the information to the rte_eth_dev_close() that it should also
-	 * release the private port resources.
-	 */
-	eth_dev->data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
-
 	if(i40evf_init_vf(eth_dev) != 0) {
 		PMD_INIT_LOG(ERR, "Init vf failed");
 		return -1;
diff --git a/drivers/net/iavf/iavf_ethdev.c b/drivers/net/iavf/iavf_ethdev.c index 8f46c4588f..512ade2ad0 100644
--- a/drivers/net/iavf/iavf_ethdev.c
+++ b/drivers/net/iavf/iavf_ethdev.c
@@ -1412,11 +1412,6 @@ iavf_dev_init(struct rte_eth_dev *eth_dev)
 	adapter->eth_dev = eth_dev;
 	adapter->stopped = 1;
 
-	/* Pass the information to the rte_eth_dev_close() that it should also
-	 * release the private port resources.
-	 */
-	eth_dev->data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
-
 	if (iavf_init_vf(eth_dev) != 0) {
 		PMD_INIT_LOG(ERR, "Init vf failed");
 		return -1;
diff --git a/drivers/net/ice/ice_dcf_ethdev.c b/drivers/net/ice/ice_dcf_ethdev.c
index 5b626cb641..33dd0c44f2 100644
--- a/drivers/net/ice/ice_dcf_ethdev.c
+++ b/drivers/net/ice/ice_dcf_ethdev.c
@@ -908,8 +908,6 @@ ice_dcf_dev_init(struct rte_eth_dev *eth_dev)
 	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
 		return 0;
 
-	eth_dev->data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
-
 	adapter->real_hw.vc_event_msg_cb = ice_dcf_handle_pf_event_msg;
 	if (ice_dcf_init_hw(eth_dev, &adapter->real_hw) != 0) {
 		PMD_INIT_LOG(ERR, "Failed to init DCF hardware"); diff --git a/drivers/net/ice/ice_ethdev.c b/drivers/net/ice/ice_ethdev.c index b4ac5e3f81..85a3ca6a2d 100644
--- a/drivers/net/ice/ice_ethdev.c
+++ b/drivers/net/ice/ice_ethdev.c
@@ -2245,11 +2245,6 @@ ice_dev_init(struct rte_eth_dev *dev)
 		goto err_init_mac;
 	}
 
-	/* Pass the information to the rte_eth_dev_close() that it should also
-	 * release the private port resources.
-	 */
-	dev->data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
-
 	ret = ice_res_pool_init(&pf->msix_pool, 1,
 				hw->func_caps.common_cap.num_msix_vectors - 1);
 	if (ret) {
diff --git a/drivers/net/igc/igc_ethdev.c b/drivers/net/igc/igc_ethdev.c index 11397c5c1d..9d27fc0d07 100644
--- a/drivers/net/igc/igc_ethdev.c
+++ b/drivers/net/igc/igc_ethdev.c
@@ -1323,11 +1323,6 @@ eth_igc_dev_init(struct rte_eth_dev *dev)
 		goto err_late;
 	}
 
-	/* Pass the information to the rte_eth_dev_close() that it should also
-	 * release the private port resources.
-	 */
-	dev->data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
-
 	hw->mac.get_link_status = 1;
 	igc->stopped = 0;
 
diff --git a/drivers/net/ixgbe/ixgbe_ethdev.c b/drivers/net/ixgbe/ixgbe_ethdev.c
index a1369a0744..426b7c9fe5 100644
--- a/drivers/net/ixgbe/ixgbe_ethdev.c
+++ b/drivers/net/ixgbe/ixgbe_ethdev.c
@@ -1248,11 +1248,6 @@ eth_ixgbe_dev_init(struct rte_eth_dev *eth_dev, void *init_params __rte_unused)
 		return -ENOMEM;
 	}
 
-	/* Pass the information to the rte_eth_dev_close() that it should also
-	 * release the private port resources.
-	 */
-	eth_dev->data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
-
 	/* initialize the vfta */
 	memset(shadow_vfta, 0, sizeof(*shadow_vfta));
 
@@ -1663,11 +1658,6 @@ eth_ixgbevf_dev_init(struct rte_eth_dev *eth_dev)
 		return -ENOMEM;
 	}
 
-	/* Pass the information to the rte_eth_dev_close() that it should also
-	 * release the private port resources.
-	 */
-	eth_dev->data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
-
 	/* Generate a random MAC address, if none was assigned by PF. */
 	if (rte_is_zero_ether_addr(perm_addr)) {
 		generate_random_mac_addr(perm_addr);
diff --git a/drivers/net/kni/rte_eth_kni.c b/drivers/net/kni/rte_eth_kni.c index aa365a57de..45ab1b17a8 100644
--- a/drivers/net/kni/rte_eth_kni.c
+++ b/drivers/net/kni/rte_eth_kni.c
@@ -387,8 +387,6 @@ eth_kni_create(struct rte_vdev_device *vdev,
 	data->promiscuous = 1;
 	data->all_multicast = 1;
 
-	data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
-
 	rte_eth_random_addr(internals->eth_addr.addr_bytes);
 
 	eth_dev->dev_ops = &eth_kni_ops;
diff --git a/drivers/net/memif/rte_eth_memif.c b/drivers/net/memif/rte_eth_memif.c
index 1a2411c838..ff8a58081f 100644
--- a/drivers/net/memif/rte_eth_memif.c
+++ b/drivers/net/memif/rte_eth_memif.c
@@ -1527,9 +1527,6 @@ memif_create(struct rte_vdev_device *vdev, enum memif_role_t role,
 		eth_dev->tx_pkt_burst = eth_memif_tx;
 	}
 
-
-	eth_dev->data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
-
 	rte_eth_dev_probing_finish(eth_dev);
 
 	return 0;
diff --git a/drivers/net/mlx4/mlx4.c b/drivers/net/mlx4/mlx4.c index ad7c805d67..3e57875414 100644
--- a/drivers/net/mlx4/mlx4.c
+++ b/drivers/net/mlx4/mlx4.c
@@ -1027,7 +1027,6 @@ mlx4_pci_probe(struct rte_pci_driver *pci_drv, struct rte_pci_device *pci_dev)
 			ERROR("can not allocate rte ethdev");
 			goto port_error;
 		}
-		eth_dev->data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
 		eth_dev->data->dev_private = priv;
 		eth_dev->data->mac_addrs = priv->mac;
 		eth_dev->device = &pci_dev->device;
diff --git a/drivers/net/mlx5/linux/mlx5_os.c b/drivers/net/mlx5/linux/mlx5_os.c
index 0511a55a28..8e498314ef 100644
--- a/drivers/net/mlx5/linux/mlx5_os.c
+++ b/drivers/net/mlx5/linux/mlx5_os.c
@@ -1155,8 +1155,6 @@ mlx5_dev_spawn(struct rte_device *dpdk_dev,
 		err = ENOMEM;
 		goto error;
 	}
-	/* Flag to call rte_eth_dev_release_port() in rte_eth_dev_close(). */
-	eth_dev->data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
 	if (priv->representor) {
 		eth_dev->data->dev_flags |= RTE_ETH_DEV_REPRESENTOR;
 		eth_dev->data->representor_id = priv->representor_id; diff --git a/drivers/net/mvneta/mvneta_ethdev.c b/drivers/net/mvneta/mvneta_ethdev.c
index eebcdb840e..db142bec23 100644
--- a/drivers/net/mvneta/mvneta_ethdev.c
+++ b/drivers/net/mvneta/mvneta_ethdev.c
@@ -834,9 +834,6 @@ mvneta_eth_dev_create(struct rte_vdev_device *vdev, const char *name)
 	mvneta_set_tx_function(eth_dev);
 	eth_dev->dev_ops = &mvneta_ops;
 
-	/* Flag to call rte_eth_dev_release_port() in rte_eth_dev_close(). */
-	eth_dev->data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
-
 	rte_eth_dev_probing_finish(eth_dev);
 	return 0;
 out_free:
diff --git a/drivers/net/mvpp2/mrvl_ethdev.c b/drivers/net/mvpp2/mrvl_ethdev.c index 27f7cfb056..cfb97e4f8f 100644
--- a/drivers/net/mvpp2/mrvl_ethdev.c
+++ b/drivers/net/mvpp2/mrvl_ethdev.c
@@ -2863,9 +2863,6 @@ mrvl_eth_dev_create(struct rte_vdev_device *vdev, const char *name)
 	mrvl_set_tx_function(eth_dev);
 	eth_dev->dev_ops = &mrvl_ops;
 
-	/* Flag to call rte_eth_dev_release_port() in rte_eth_dev_close(). */
-	eth_dev->data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
-
 	rte_eth_dev_probing_finish(eth_dev);
 	return 0;
 out_free:
diff --git a/drivers/net/netvsc/hn_ethdev.c b/drivers/net/netvsc/hn_ethdev.c index 8968036ea3..15d6e9762d 100644
--- a/drivers/net/netvsc/hn_ethdev.c
+++ b/drivers/net/netvsc/hn_ethdev.c
@@ -123,9 +123,6 @@ eth_dev_vmbus_allocate(struct rte_vmbus_device *dev, size_t private_data_size)
 	eth_dev->data->dev_flags |= RTE_ETH_DEV_INTR_LSC;
 	eth_dev->intr_handle = &dev->intr_handle;
 
-	/* allow ethdev to remove on close */
-	eth_dev->data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
-
 	return eth_dev;
 }
 
diff --git a/drivers/net/nfb/nfb_ethdev.c b/drivers/net/nfb/nfb_ethdev.c index c3c3d003f1..d937ac6922 100644
--- a/drivers/net/nfb/nfb_ethdev.c
+++ b/drivers/net/nfb/nfb_ethdev.c
@@ -457,9 +457,6 @@ nfb_eth_dev_init(struct rte_eth_dev *dev)
 		rte_kvargs_free(kvlist);
 	}
 
-	/* Let rte_eth_dev_close() release the port resources */
-	dev->data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
-
 	/*
 	 * Get number of available DMA RX and TX queues, which is maximum
 	 * number of queues that can be created and store it in private device diff --git a/drivers/net/null/rte_eth_null.c b/drivers/net/null/rte_eth_null.c index 84c1d0c951..7c3c76a897 100644
--- a/drivers/net/null/rte_eth_null.c
+++ b/drivers/net/null/rte_eth_null.c
@@ -548,7 +548,6 @@ eth_dev_null_create(struct rte_vdev_device *dev, struct pmd_options *args)
 	data->mac_addrs = &internals->eth_addr;
 	data->promiscuous = 1;
 	data->all_multicast = 1;
-	data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
 
 	eth_dev->dev_ops = &ops;
 
diff --git a/drivers/net/octeontx/octeontx_ethdev.c b/drivers/net/octeontx/octeontx_ethdev.c
index 48ce3e1621..a263f45399 100644
--- a/drivers/net/octeontx/octeontx_ethdev.c
+++ b/drivers/net/octeontx/octeontx_ethdev.c
@@ -1374,7 +1374,6 @@ octeontx_create(struct rte_vdev_device *dev, int port, uint8_t evdev,
 	data->promiscuous = 0;
 	data->all_multicast = 0;
 	data->scattered_rx = 0;
-	data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
 
 	/* Get maximum number of supported MAC entries */
 	max_entries = octeontx_bgx_port_mac_entries_get(nic->port_id);
diff --git a/drivers/net/octeontx2/otx2_ethdev.c b/drivers/net/octeontx2/otx2_ethdev.c
index abe5f03628..581ac89381 100644
--- a/drivers/net/octeontx2/otx2_ethdev.c
+++ b/drivers/net/octeontx2/otx2_ethdev.c
@@ -2394,7 +2394,6 @@ otx2_eth_dev_init(struct rte_eth_dev *eth_dev)
 	pci_dev = RTE_ETH_DEV_TO_PCI(eth_dev);
 
 	rte_eth_copy_pci_info(eth_dev, pci_dev);
-	eth_dev->data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
 
 	/* Zero out everything after OTX2_DEV to allow proper dev_reset() */
 	memset(&dev->otx2_eth_dev_data_start, 0, sizeof(*dev) - diff --git a/drivers/net/pcap/rte_eth_pcap.c b/drivers/net/pcap/rte_eth_pcap.c index 909eef8cce..da4988064a 100644
--- a/drivers/net/pcap/rte_eth_pcap.c
+++ b/drivers/net/pcap/rte_eth_pcap.c
@@ -1334,7 +1334,6 @@ eth_from_pcaps(struct rte_vdev_device *vdev,
 	else
 		eth_dev->tx_pkt_burst = eth_tx_drop;
 
-	eth_dev->data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
 	rte_eth_dev_probing_finish(eth_dev);
 	return 0;
 }
diff --git a/drivers/net/qede/qede_ethdev.c b/drivers/net/qede/qede_ethdev.c index 43fe68c2bf..cd578709fb 100644
--- a/drivers/net/qede/qede_ethdev.c
+++ b/drivers/net/qede/qede_ethdev.c
@@ -2711,11 +2711,6 @@ static int qede_common_dev_init(struct rte_eth_dev *eth_dev, bool is_vf)
 		adapter->ipgre.enable = false;
 	}
 
-	/* Pass the information to the rte_eth_dev_close() that it should also
-	 * release the private port resources.
-	 */
-	eth_dev->data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
-
 	DP_INFO(edev, "MAC address : %02x:%02x:%02x:%02x:%02x:%02x\n",
 		adapter->primary_mac.addr_bytes[0],
 		adapter->primary_mac.addr_bytes[1],
diff --git a/drivers/net/ring/rte_eth_ring.c b/drivers/net/ring/rte_eth_ring.c index ed0fdeb28f..12046f5a00 100644
--- a/drivers/net/ring/rte_eth_ring.c
+++ b/drivers/net/ring/rte_eth_ring.c
@@ -358,7 +358,6 @@ do_eth_dev_ring_create(const char *name,
 
 	eth_dev->dev_ops = &ops;
 	data->numa_node = numa_node;
-	data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
 
 	/* finally assign rx and tx ops */
 	eth_dev->rx_pkt_burst = eth_ring_rx;
diff --git a/drivers/net/sfc/sfc_ethdev.c b/drivers/net/sfc/sfc_ethdev.c index eaa48627d6..1c72e40b75 100644
--- a/drivers/net/sfc/sfc_ethdev.c
+++ b/drivers/net/sfc/sfc_ethdev.c
@@ -343,7 +343,7 @@ sfc_dev_close(struct rte_eth_dev *dev)
 	}
 
 	/*
-	 * Cleanup all resources in accordance with RTE_ETH_DEV_CLOSE_REMOVE.
+	 * Cleanup all resources.
 	 * Rollback primary process sfc_eth_dev_init() below.
 	 */
 
@@ -2179,8 +2179,6 @@ sfc_eth_dev_init(struct rte_eth_dev *dev)
 
 	sfc_log_init(sa, "entry");
 
-	dev->data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
-
 	dev->data->mac_addrs = rte_zmalloc("sfc", RTE_ETHER_ADDR_LEN, 0);
 	if (dev->data->mac_addrs == NULL) {
 		rc = ENOMEM;
diff --git a/drivers/net/softnic/rte_eth_softnic.c b/drivers/net/softnic/rte_eth_softnic.c
index ad9e2aa86d..e942df78b6 100644
--- a/drivers/net/softnic/rte_eth_softnic.c
+++ b/drivers/net/softnic/rte_eth_softnic.c
@@ -390,7 +390,6 @@ pmd_ethdev_register(struct rte_vdev_device *vdev,
 	dev->device = &vdev->device;
 
 	/* dev->data */
-	dev->data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
 	dev->data->dev_private = dev_private;
 	dev->data->dev_link.link_speed = ETH_SPEED_NUM_100G;
 	dev->data->dev_link.link_duplex = ETH_LINK_FULL_DUPLEX; diff --git a/drivers/net/szedata2/rte_eth_szedata2.c b/drivers/net/szedata2/rte_eth_szedata2.c
index daa40cfd6c..4325b9a30d 100644
--- a/drivers/net/szedata2/rte_eth_szedata2.c
+++ b/drivers/net/szedata2/rte_eth_szedata2.c
@@ -1488,9 +1488,6 @@ rte_szedata2_eth_dev_init(struct rte_eth_dev *dev, struct port_info *pi)
 	PMD_INIT_LOG(INFO, "Initializing eth_dev %s (driver %s)", data->name,
 			RTE_STR(RTE_SZEDATA2_DRIVER_NAME));
 
-	/* Let rte_eth_dev_close() release the port resources */
-	dev->data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
-
 	/* Fill internal private structure. */
 	internals->dev = dev;
 	/* Get index of szedata2 device file and create path to device file */ diff --git a/drivers/net/tap/rte_eth_tap.c b/drivers/net/tap/rte_eth_tap.c index 0d217f1486..b127ce62dc 100644
--- a/drivers/net/tap/rte_eth_tap.c
+++ b/drivers/net/tap/rte_eth_tap.c
@@ -1922,7 +1922,7 @@ eth_dev_tap_create(struct rte_vdev_device *vdev, const char *tap_name,
 	/* Setup some default values */
 	data = dev->data;
 	data->dev_private = pmd;
-	data->dev_flags = RTE_ETH_DEV_INTR_LSC | RTE_ETH_DEV_CLOSE_REMOVE;
+	data->dev_flags = RTE_ETH_DEV_INTR_LSC;
 	data->numa_node = numa_node;
 
 	data->dev_link = pmd_link;
diff --git a/drivers/net/vhost/rte_eth_vhost.c b/drivers/net/vhost/rte_eth_vhost.c
index f67f4db812..45552ef742 100644
--- a/drivers/net/vhost/rte_eth_vhost.c
+++ b/drivers/net/vhost/rte_eth_vhost.c
@@ -1443,7 +1443,7 @@ eth_dev_vhost_create(struct rte_vdev_device *dev, char *iface_name,
 	internal->flags = flags;
 	internal->disable_flags = disable_flags;
 	data->dev_link = pmd_link;
-	data->dev_flags = RTE_ETH_DEV_INTR_LSC | RTE_ETH_DEV_CLOSE_REMOVE;
+	data->dev_flags = RTE_ETH_DEV_INTR_LSC;
 	data->promiscuous = 1;
 	data->all_multicast = 1;
 
diff --git a/drivers/net/virtio/virtio_ethdev.c b/drivers/net/virtio/virtio_ethdev.c
index 0787337a01..b6ed5829bd 100644
--- a/drivers/net/virtio/virtio_ethdev.c
+++ b/drivers/net/virtio/virtio_ethdev.c
@@ -1924,11 +1924,6 @@ eth_virtio_dev_init(struct rte_eth_dev *eth_dev)
 	if (ret < 0)
 		return ret;
 	hw->speed = speed;
-	/*
-	 * Pass the information to the rte_eth_dev_close() that it should also
-	 * release the private port resources.
-	 */
-	eth_dev->data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
 
 	/* Allocate memory for storing MAC addresses */
 	eth_dev->data->mac_addrs = rte_zmalloc("virtio", diff --git a/drivers/net/vmxnet3/vmxnet3_ethdev.c b/drivers/net/vmxnet3/vmxnet3_ethdev.c
index 17fed0ed6e..34a169d2c0 100644
--- a/drivers/net/vmxnet3/vmxnet3_ethdev.c
+++ b/drivers/net/vmxnet3/vmxnet3_ethdev.c
@@ -321,9 +321,6 @@ eth_vmxnet3_dev_init(struct rte_eth_dev *eth_dev)
 		     hw->perm_addr[0], hw->perm_addr[1], hw->perm_addr[2],
 		     hw->perm_addr[3], hw->perm_addr[4], hw->perm_addr[5]);
 
-	/* Flag to call rte_eth_dev_release_port() in rte_eth_dev_close(). */
-	eth_dev->data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
-
 	/* Put device in Quiesce Mode */
 	VMXNET3_WRITE_BAR1_REG(hw, VMXNET3_REG_CMD, VMXNET3_CMD_QUIESCE_DEV);
 
diff --git a/lib/librte_ethdev/rte_ethdev.c b/lib/librte_ethdev/rte_ethdev.c index d12d74dd8c..d7668114ca 100644
--- a/lib/librte_ethdev/rte_ethdev.c
+++ b/lib/librte_ethdev/rte_ethdev.c
@@ -1720,22 +1720,7 @@ rte_eth_dev_close(uint16_t port_id)
 	(*dev->dev_ops->dev_close)(dev);
 
 	rte_ethdev_trace_close(port_id);
-	/* check behaviour flag - temporary for PMD migration */
-	if ((dev->data->dev_flags & RTE_ETH_DEV_CLOSE_REMOVE) != 0) {
-		/* new behaviour: send event + reset state + free all data */
-		rte_eth_dev_release_port(dev);
-		return;
-	}
-	RTE_ETHDEV_LOG(DEBUG, "Port closing is using an old behaviour.\n"
-			"The driver %s should migrate to the new behaviour.\n",
-			dev->device->driver->name);
-	/* old behaviour: only free queue arrays */
-	dev->data->nb_rx_queues = 0;
-	rte_free(dev->data->rx_queues);
-	dev->data->rx_queues = NULL;
-	dev->data->nb_tx_queues = 0;
-	rte_free(dev->data->tx_queues);
-	dev->data->tx_queues = NULL;
+	rte_eth_dev_release_port(dev);
 }
 
 int
diff --git a/lib/librte_ethdev/rte_ethdev.h b/lib/librte_ethdev/rte_ethdev.h index 645a18664d..24d898ae89 100644
--- a/lib/librte_ethdev/rte_ethdev.h
+++ b/lib/librte_ethdev/rte_ethdev.h
@@ -1654,11 +1654,6 @@ struct rte_eth_dev_owner {
 	char name[RTE_ETH_MAX_OWNER_NAME_LEN]; /**< The owner name. */  };
 
-/**
- * Port is released (i.e. totally freed and data erased) on close.
- * Temporary flag for PMD migration to new rte_eth_dev_close() behaviour.
- */
-#define RTE_ETH_DEV_CLOSE_REMOVE 0x0001
 /** Device supports link state interrupt */
 #define RTE_ETH_DEV_INTR_LSC     0x0002
 /** Device is a bonded slave */
@@ -2282,8 +2277,7 @@ int rte_eth_dev_set_link_down(uint16_t port_id);
 
 /**
  * Close a stopped Ethernet device. The device cannot be restarted!
- * The function frees all port resources if the driver supports
- * the flag RTE_ETH_DEV_CLOSE_REMOVE.
+ * The function frees all port resources.
  *
  * @param port_id
  *   The port identifier of the Ethernet device.
--
2.28.0

[dpdk-dev] [PATCH v2 18/25] drivers/net: accept removing device without any port

[Thomas Monjalon]

The ports can be closed (i.e. completely released)
before removing the whole device.
Such case was wrongly considered an error by some drivers.

If the device supports only one port, there is nothing much
to free after the port is closed.

Signed-off-by: Thomas Monjalon <thomas@monjalon.net>
Reviewed-by: Ferruh Yigit <ferruh.yigit@intel.com>
Reviewed-by: Rosen Xu <rosen.xu@intel.com>
---
 drivers/net/ipn3ke/ipn3ke_ethdev.c      |  6 ++----
 drivers/net/kni/rte_eth_kni.c           | 16 +++++++---------
 drivers/net/netvsc/hn_ethdev.c          |  2 +-
 drivers/net/nfp/nfp_net.c               |  2 ++
 drivers/net/pfe/pfe_ethdev.c            |  6 ++----
 drivers/net/szedata2/rte_eth_szedata2.c |  6 ++----
 6 files changed, 16 insertions(+), 22 deletions(-)

diff --git a/drivers/net/ipn3ke/ipn3ke_ethdev.c b/drivers/net/ipn3ke/ipn3ke_ethdev.c
index 027be29bd8..4446d2af9e 100644
--- a/drivers/net/ipn3ke/ipn3ke_ethdev.c
+++ b/drivers/net/ipn3ke/ipn3ke_ethdev.c
@@ -562,10 +562,8 @@ static int ipn3ke_vswitch_remove(struct rte_afu_device *afu_dev)
 			afu_dev->device.name, i);
 
 		ethdev = rte_eth_dev_allocated(afu_dev->device.name);
-		if (!ethdev)
-			return -ENODEV;
-
-		rte_eth_dev_destroy(ethdev, ipn3ke_rpst_uninit);
+		if (ethdev != NULL)
+			rte_eth_dev_destroy(ethdev, ipn3ke_rpst_uninit);
 	}
 
 	ret = rte_eth_switch_domain_free(hw->switch_domain_id);
diff --git a/drivers/net/kni/rte_eth_kni.c b/drivers/net/kni/rte_eth_kni.c
index 45ab1b17a8..2a4058f7b0 100644
--- a/drivers/net/kni/rte_eth_kni.c
+++ b/drivers/net/kni/rte_eth_kni.c
@@ -488,17 +488,15 @@ eth_kni_remove(struct rte_vdev_device *vdev)
 
 	/* find the ethdev entry */
 	eth_dev = rte_eth_dev_allocated(name);
-	if (eth_dev == NULL)
-		return -1;
-
-	if (rte_eal_process_type() != RTE_PROC_PRIMARY) {
-		eth_kni_dev_stop(eth_dev);
-		return rte_eth_dev_release_port(eth_dev);
+	if (eth_dev != NULL) {
+		if (rte_eal_process_type() != RTE_PROC_PRIMARY) {
+			eth_kni_dev_stop(eth_dev);
+			return rte_eth_dev_release_port(eth_dev);
+		}
+		eth_kni_close(eth_dev);
+		rte_eth_dev_release_port(eth_dev);
 	}
 
-	eth_kni_close(eth_dev);
-	rte_eth_dev_release_port(eth_dev);
-
 	is_kni_initialized--;
 	if (is_kni_initialized == 0)
 		rte_kni_close();
diff --git a/drivers/net/netvsc/hn_ethdev.c b/drivers/net/netvsc/hn_ethdev.c
index 15d6e9762d..19a9eb6bc2 100644
--- a/drivers/net/netvsc/hn_ethdev.c
+++ b/drivers/net/netvsc/hn_ethdev.c
@@ -1092,7 +1092,7 @@ static int eth_hn_remove(struct rte_vmbus_device *dev)
 
 	eth_dev = rte_eth_dev_allocated(dev->device.name);
 	if (!eth_dev)
-		return -ENODEV;
+		return 0; /* port already released */
 
 	ret = eth_hn_dev_uninit(eth_dev);
 	if (ret)
diff --git a/drivers/net/nfp/nfp_net.c b/drivers/net/nfp/nfp_net.c
index 9509dc8bd6..ce25cf1ed4 100644
--- a/drivers/net/nfp/nfp_net.c
+++ b/drivers/net/nfp/nfp_net.c
@@ -3739,6 +3739,8 @@ static int eth_nfp_pci_remove(struct rte_pci_device *pci_dev)
 	int port = 0;
 
 	eth_dev = rte_eth_dev_allocated(pci_dev->device.name);
+	if (eth_dev == NULL)
+		return 0; /* port already released */
 	if ((pci_dev->id.device_id == PCI_DEVICE_ID_NFP4000_PF_NIC) ||
 	    (pci_dev->id.device_id == PCI_DEVICE_ID_NFP6000_PF_NIC)) {
 		port = get_pf_port_number(eth_dev->data->name);
diff --git a/drivers/net/pfe/pfe_ethdev.c b/drivers/net/pfe/pfe_ethdev.c
index 187a0019ff..9d5415d9b1 100644
--- a/drivers/net/pfe/pfe_ethdev.c
+++ b/drivers/net/pfe/pfe_ethdev.c
@@ -1155,10 +1155,8 @@ pmd_pfe_remove(struct rte_vdev_device *vdev)
 		return 0;
 
 	eth_dev = rte_eth_dev_allocated(name);
-	if (eth_dev == NULL)
-		return -ENODEV;
-
-	pfe_eth_exit(eth_dev, g_pfe);
+	if (eth_dev != NULL)
+		pfe_eth_exit(eth_dev, g_pfe);
 	munmap(g_pfe->cbus_baseaddr, g_pfe->cbus_size);
 
 	if (g_pfe->nb_devs == 0) {
diff --git a/drivers/net/szedata2/rte_eth_szedata2.c b/drivers/net/szedata2/rte_eth_szedata2.c
index 4325b9a30d..5f589dfa4c 100644
--- a/drivers/net/szedata2/rte_eth_szedata2.c
+++ b/drivers/net/szedata2/rte_eth_szedata2.c
@@ -1910,10 +1910,8 @@ static int szedata2_eth_pci_remove(struct rte_pci_device *pci_dev)
 				pci_dev->device.name, i);
 		PMD_DRV_LOG(DEBUG, "Removing eth_dev %s", name);
 		eth_dev = rte_eth_dev_allocated(name);
-		if (!eth_dev) {
-			PMD_DRV_LOG(ERR, "eth_dev %s not found", name);
-			retval = retval ? retval : -ENODEV;
-		}
+		if (eth_dev == NULL)
+			continue; /* port already released */
 
 		ret = rte_szedata2_eth_dev_uninit(eth_dev);
 		if (ret != 0) {
-- 
2.28.0

Re: [dpdk-dev] [PATCH v2 18/25] drivers/net: accept removing device without any port

[Xu, Rosen]

Hi,

> -----Original Message-----
> From: Thomas Monjalon <thomas@monjalon.net>
> Sent: Monday, September 28, 2020 7:43
> To: dev@dpdk.org
> Cc: Yigit, Ferruh <ferruh.yigit@intel.com>; arybchenko@solarflare.com; Xu,
> Rosen <rosen.xu@intel.com>; Stephen Hemminger
> <sthemmin@microsoft.com>; K. Y. Srinivasan <kys@microsoft.com>;
> Haiyang Zhang <haiyangz@microsoft.com>; Long Li <longli@microsoft.com>;
> Heinrich Kuhn <heinrich.kuhn@netronome.com>; Gagandeep Singh
> <g.singh@nxp.com>; Akhil Goyal <akhil.goyal@nxp.com>; Martin Spinler
> <spinler@cesnet.cz>; Burakov, Anatoly <anatoly.burakov@intel.com>
> Subject: [PATCH v2 18/25] drivers/net: accept removing device without any
> port
> 
> The ports can be closed (i.e. completely released) before removing the
> whole device.
> Such case was wrongly considered an error by some drivers.
> 
> If the device supports only one port, there is nothing much to free after the
> port is closed.
> 
> Signed-off-by: Thomas Monjalon <thomas@monjalon.net>
> Reviewed-by: Ferruh Yigit <ferruh.yigit@intel.com>
> Reviewed-by: Rosen Xu <rosen.xu@intel.com>
> ---
>  drivers/net/ipn3ke/ipn3ke_ethdev.c      |  6 ++----
>  drivers/net/kni/rte_eth_kni.c           | 16 +++++++---------
>  drivers/net/netvsc/hn_ethdev.c          |  2 +-
>  drivers/net/nfp/nfp_net.c               |  2 ++
>  drivers/net/pfe/pfe_ethdev.c            |  6 ++----
>  drivers/net/szedata2/rte_eth_szedata2.c |  6 ++----
>  6 files changed, 16 insertions(+), 22 deletions(-)
> 
> diff --git a/drivers/net/ipn3ke/ipn3ke_ethdev.c
> b/drivers/net/ipn3ke/ipn3ke_ethdev.c
> index 027be29bd8..4446d2af9e 100644
> --- a/drivers/net/ipn3ke/ipn3ke_ethdev.c
> +++ b/drivers/net/ipn3ke/ipn3ke_ethdev.c
> @@ -562,10 +562,8 @@ static int ipn3ke_vswitch_remove(struct
> rte_afu_device *afu_dev)
>  			afu_dev->device.name, i);
> 
>  		ethdev = rte_eth_dev_allocated(afu_dev->device.name);
> -		if (!ethdev)
> -			return -ENODEV;
> -
> -		rte_eth_dev_destroy(ethdev, ipn3ke_rpst_uninit);
> +		if (ethdev != NULL)
> +			rte_eth_dev_destroy(ethdev, ipn3ke_rpst_uninit);
>  	}
> 
>  	ret = rte_eth_switch_domain_free(hw->switch_domain_id);
> diff --git a/drivers/net/kni/rte_eth_kni.c b/drivers/net/kni/rte_eth_kni.c
> index 45ab1b17a8..2a4058f7b0 100644
> --- a/drivers/net/kni/rte_eth_kni.c
> +++ b/drivers/net/kni/rte_eth_kni.c
> @@ -488,17 +488,15 @@ eth_kni_remove(struct rte_vdev_device *vdev)
> 
>  	/* find the ethdev entry */
>  	eth_dev = rte_eth_dev_allocated(name);
> -	if (eth_dev == NULL)
> -		return -1;
> -
> -	if (rte_eal_process_type() != RTE_PROC_PRIMARY) {
> -		eth_kni_dev_stop(eth_dev);
> -		return rte_eth_dev_release_port(eth_dev);
> +	if (eth_dev != NULL) {
> +		if (rte_eal_process_type() != RTE_PROC_PRIMARY) {
> +			eth_kni_dev_stop(eth_dev);
> +			return rte_eth_dev_release_port(eth_dev);
> +		}
> +		eth_kni_close(eth_dev);
> +		rte_eth_dev_release_port(eth_dev);
>  	}
> 
> -	eth_kni_close(eth_dev);
> -	rte_eth_dev_release_port(eth_dev);
> -
>  	is_kni_initialized--;
>  	if (is_kni_initialized == 0)
>  		rte_kni_close();
> diff --git a/drivers/net/netvsc/hn_ethdev.c
> b/drivers/net/netvsc/hn_ethdev.c index 15d6e9762d..19a9eb6bc2 100644
> --- a/drivers/net/netvsc/hn_ethdev.c
> +++ b/drivers/net/netvsc/hn_ethdev.c
> @@ -1092,7 +1092,7 @@ static int eth_hn_remove(struct rte_vmbus_device
> *dev)
> 
>  	eth_dev = rte_eth_dev_allocated(dev->device.name);
>  	if (!eth_dev)
> -		return -ENODEV;
> +		return 0; /* port already released */
> 
>  	ret = eth_hn_dev_uninit(eth_dev);
>  	if (ret)
> diff --git a/drivers/net/nfp/nfp_net.c b/drivers/net/nfp/nfp_net.c index
> 9509dc8bd6..ce25cf1ed4 100644
> --- a/drivers/net/nfp/nfp_net.c
> +++ b/drivers/net/nfp/nfp_net.c
> @@ -3739,6 +3739,8 @@ static int eth_nfp_pci_remove(struct
> rte_pci_device *pci_dev)
>  	int port = 0;
> 
>  	eth_dev = rte_eth_dev_allocated(pci_dev->device.name);
> +	if (eth_dev == NULL)
> +		return 0; /* port already released */
>  	if ((pci_dev->id.device_id == PCI_DEVICE_ID_NFP4000_PF_NIC) ||
>  	    (pci_dev->id.device_id == PCI_DEVICE_ID_NFP6000_PF_NIC)) {
>  		port = get_pf_port_number(eth_dev->data->name);
> diff --git a/drivers/net/pfe/pfe_ethdev.c b/drivers/net/pfe/pfe_ethdev.c
> index 187a0019ff..9d5415d9b1 100644
> --- a/drivers/net/pfe/pfe_ethdev.c
> +++ b/drivers/net/pfe/pfe_ethdev.c
> @@ -1155,10 +1155,8 @@ pmd_pfe_remove(struct rte_vdev_device *vdev)
>  		return 0;
> 
>  	eth_dev = rte_eth_dev_allocated(name);
> -	if (eth_dev == NULL)
> -		return -ENODEV;
> -
> -	pfe_eth_exit(eth_dev, g_pfe);
> +	if (eth_dev != NULL)
> +		pfe_eth_exit(eth_dev, g_pfe);
>  	munmap(g_pfe->cbus_baseaddr, g_pfe->cbus_size);
> 
>  	if (g_pfe->nb_devs == 0) {
> diff --git a/drivers/net/szedata2/rte_eth_szedata2.c
> b/drivers/net/szedata2/rte_eth_szedata2.c
> index 4325b9a30d..5f589dfa4c 100644
> --- a/drivers/net/szedata2/rte_eth_szedata2.c
> +++ b/drivers/net/szedata2/rte_eth_szedata2.c
> @@ -1910,10 +1910,8 @@ static int szedata2_eth_pci_remove(struct
> rte_pci_device *pci_dev)
>  				pci_dev->device.name, i);
>  		PMD_DRV_LOG(DEBUG, "Removing eth_dev %s", name);
>  		eth_dev = rte_eth_dev_allocated(name);
> -		if (!eth_dev) {
> -			PMD_DRV_LOG(ERR, "eth_dev %s not found",
> name);
> -			retval = retval ? retval : -ENODEV;
> -		}
> +		if (eth_dev == NULL)
> +			continue; /* port already released */
> 
>  		ret = rte_szedata2_eth_dev_uninit(eth_dev);
>  		if (ret != 0) {
> --
> 2.28.0

For net/ipn3ke
Reviewed-by: Rosen Xu <rosen.xu@intel.com>

Re: [dpdk-dev] [PATCH v2 18/25] drivers/net: accept removing device without any port

[Sachin Saxena (OSS)]

For "net/pfe"

Reviewed-by: Sachin Saxena<sachin.saxena@oss.nxp.com>

On 28-Sep-20 5:12 AM, Thomas Monjalon wrote:
> The ports can be closed (i.e. completely released)
> before removing the whole device.
> Such case was wrongly considered an error by some drivers.
>
> If the device supports only one port, there is nothing much
> to free after the port is closed.
>
> Signed-off-by: Thomas Monjalon <thomas@monjalon.net>
> Reviewed-by: Ferruh Yigit <ferruh.yigit@intel.com>
> Reviewed-by: Rosen Xu <rosen.xu@intel.com>
> ---
>   drivers/net/ipn3ke/ipn3ke_ethdev.c      |  6 ++----
>   drivers/net/kni/rte_eth_kni.c           | 16 +++++++---------
>   drivers/net/netvsc/hn_ethdev.c          |  2 +-
>   drivers/net/nfp/nfp_net.c               |  2 ++
>   drivers/net/pfe/pfe_ethdev.c            |  6 ++----
>   drivers/net/szedata2/rte_eth_szedata2.c |  6 ++----
>   6 files changed, 16 insertions(+), 22 deletions(-)
>
> diff --git a/drivers/net/ipn3ke/ipn3ke_ethdev.c b/drivers/net/ipn3ke/ipn3ke_ethdev.c
> index 027be29bd8..4446d2af9e 100644
> --- a/drivers/net/ipn3ke/ipn3ke_ethdev.c
> +++ b/drivers/net/ipn3ke/ipn3ke_ethdev.c
> @@ -562,10 +562,8 @@ static int ipn3ke_vswitch_remove(struct rte_afu_device *afu_dev)
>   			afu_dev->device.name, i);
>   
>   		ethdev = rte_eth_dev_allocated(afu_dev->device.name);
> -		if (!ethdev)
> -			return -ENODEV;
> -
> -		rte_eth_dev_destroy(ethdev, ipn3ke_rpst_uninit);
> +		if (ethdev != NULL)
> +			rte_eth_dev_destroy(ethdev, ipn3ke_rpst_uninit);
>   	}
>   
>   	ret = rte_eth_switch_domain_free(hw->switch_domain_id);
> diff --git a/drivers/net/kni/rte_eth_kni.c b/drivers/net/kni/rte_eth_kni.c
> index 45ab1b17a8..2a4058f7b0 100644
> --- a/drivers/net/kni/rte_eth_kni.c
> +++ b/drivers/net/kni/rte_eth_kni.c
> @@ -488,17 +488,15 @@ eth_kni_remove(struct rte_vdev_device *vdev)
>   
>   	/* find the ethdev entry */
>   	eth_dev = rte_eth_dev_allocated(name);
> -	if (eth_dev == NULL)
> -		return -1;
> -
> -	if (rte_eal_process_type() != RTE_PROC_PRIMARY) {
> -		eth_kni_dev_stop(eth_dev);
> -		return rte_eth_dev_release_port(eth_dev);
> +	if (eth_dev != NULL) {
> +		if (rte_eal_process_type() != RTE_PROC_PRIMARY) {
> +			eth_kni_dev_stop(eth_dev);
> +			return rte_eth_dev_release_port(eth_dev);
> +		}
> +		eth_kni_close(eth_dev);
> +		rte_eth_dev_release_port(eth_dev);
>   	}
>   
> -	eth_kni_close(eth_dev);
> -	rte_eth_dev_release_port(eth_dev);
> -
>   	is_kni_initialized--;
>   	if (is_kni_initialized == 0)
>   		rte_kni_close();
> diff --git a/drivers/net/netvsc/hn_ethdev.c b/drivers/net/netvsc/hn_ethdev.c
> index 15d6e9762d..19a9eb6bc2 100644
> --- a/drivers/net/netvsc/hn_ethdev.c
> +++ b/drivers/net/netvsc/hn_ethdev.c
> @@ -1092,7 +1092,7 @@ static int eth_hn_remove(struct rte_vmbus_device *dev)
>   
>   	eth_dev = rte_eth_dev_allocated(dev->device.name);
>   	if (!eth_dev)
> -		return -ENODEV;
> +		return 0; /* port already released */
>   
>   	ret = eth_hn_dev_uninit(eth_dev);
>   	if (ret)
> diff --git a/drivers/net/nfp/nfp_net.c b/drivers/net/nfp/nfp_net.c
> index 9509dc8bd6..ce25cf1ed4 100644
> --- a/drivers/net/nfp/nfp_net.c
> +++ b/drivers/net/nfp/nfp_net.c
> @@ -3739,6 +3739,8 @@ static int eth_nfp_pci_remove(struct rte_pci_device *pci_dev)
>   	int port = 0;
>   
>   	eth_dev = rte_eth_dev_allocated(pci_dev->device.name);
> +	if (eth_dev == NULL)
> +		return 0; /* port already released */
>   	if ((pci_dev->id.device_id == PCI_DEVICE_ID_NFP4000_PF_NIC) ||
>   	    (pci_dev->id.device_id == PCI_DEVICE_ID_NFP6000_PF_NIC)) {
>   		port = get_pf_port_number(eth_dev->data->name);
> diff --git a/drivers/net/pfe/pfe_ethdev.c b/drivers/net/pfe/pfe_ethdev.c
> index 187a0019ff..9d5415d9b1 100644
> --- a/drivers/net/pfe/pfe_ethdev.c
> +++ b/drivers/net/pfe/pfe_ethdev.c
> @@ -1155,10 +1155,8 @@ pmd_pfe_remove(struct rte_vdev_device *vdev)
>   		return 0;
>   
>   	eth_dev = rte_eth_dev_allocated(name);
> -	if (eth_dev == NULL)
> -		return -ENODEV;
> -
> -	pfe_eth_exit(eth_dev, g_pfe);
> +	if (eth_dev != NULL)
> +		pfe_eth_exit(eth_dev, g_pfe);
>   	munmap(g_pfe->cbus_baseaddr, g_pfe->cbus_size);
>   
>   	if (g_pfe->nb_devs == 0) {
> diff --git a/drivers/net/szedata2/rte_eth_szedata2.c b/drivers/net/szedata2/rte_eth_szedata2.c
> index 4325b9a30d..5f589dfa4c 100644
> --- a/drivers/net/szedata2/rte_eth_szedata2.c
> +++ b/drivers/net/szedata2/rte_eth_szedata2.c
> @@ -1910,10 +1910,8 @@ static int szedata2_eth_pci_remove(struct rte_pci_device *pci_dev)
>   				pci_dev->device.name, i);
>   		PMD_DRV_LOG(DEBUG, "Removing eth_dev %s", name);
>   		eth_dev = rte_eth_dev_allocated(name);
> -		if (!eth_dev) {
> -			PMD_DRV_LOG(ERR, "eth_dev %s not found", name);
> -			retval = retval ? retval : -ENODEV;
> -		}
> +		if (eth_dev == NULL)
> +			continue; /* port already released */
>   
>   		ret = rte_szedata2_eth_dev_uninit(eth_dev);
>   		if (ret != 0) {

[dpdk-dev] [PATCH v2 19/25] drivers/net: check process type in close operation

[Thomas Monjalon]

The secondary processes are not allowed to release shared resources.
Only process-private ressources should be freed in a secondary process.
Most of the time, there is no process-private ressource,
so the close operation is just forbidden in a secondary process.

After adding proper check in the port close functions,
some redundant checks in the device remove functions are dropped.

Signed-off-by: Thomas Monjalon <thomas@monjalon.net>
---
 drivers/net/af_xdp/rte_eth_af_xdp.c     |  3 +++
 drivers/net/ark/ark_ethdev.c            |  3 +++
 drivers/net/avp/avp_ethdev.c            |  3 +++
 drivers/net/bnxt/bnxt_ethdev.c          |  3 +++
 drivers/net/bnxt/bnxt_reps.c            |  3 +++
 drivers/net/cxgbe/cxgbe_ethdev.c        |  3 +++
 drivers/net/dpaa/dpaa_ethdev.c          |  3 +++
 drivers/net/dpaa2/dpaa2_ethdev.c        |  3 +++
 drivers/net/e1000/em_ethdev.c           |  3 +++
 drivers/net/e1000/igb_ethdev.c          |  6 +++++
 drivers/net/ena/ena_ethdev.c            |  3 +++
 drivers/net/enetc/enetc_ethdev.c        |  3 +++
 drivers/net/enic/enic_ethdev.c          |  3 +++
 drivers/net/fm10k/fm10k_ethdev.c        |  9 ++-----
 drivers/net/hinic/hinic_pmd_ethdev.c    |  3 +++
 drivers/net/i40e/i40e_ethdev.c          |  2 ++
 drivers/net/i40e/i40e_ethdev_vf.c       |  3 +++
 drivers/net/iavf/iavf_ethdev.c          |  3 +++
 drivers/net/ice/ice_ethdev.c            |  3 +++
 drivers/net/igc/igc_ethdev.c            |  6 ++---
 drivers/net/ionic/ionic_ethdev.c        |  2 ++
 drivers/net/ipn3ke/ipn3ke_representor.c |  3 +++
 drivers/net/ixgbe/ixgbe_ethdev.c        |  4 ++++
 drivers/net/kni/rte_eth_kni.c           |  3 +++
 drivers/net/liquidio/lio_ethdev.c       |  3 +++
 drivers/net/mlx4/mlx4.c                 |  2 ++
 drivers/net/mvneta/mvneta_ethdev.c      |  3 +++
 drivers/net/mvpp2/mrvl_ethdev.c         |  3 +++
 drivers/net/netvsc/hn_ethdev.c          |  2 ++
 drivers/net/nfb/nfb_ethdev.c            |  3 +++
 drivers/net/nfp/nfp_net.c               |  3 +++
 drivers/net/octeontx/octeontx_ethdev.c  |  2 ++
 drivers/net/pfe/pfe_ethdev.c            |  3 +++
 drivers/net/sfc/sfc_ethdev.c            | 32 ++++++++++++-------------
 drivers/net/szedata2/rte_eth_szedata2.c |  3 +++
 drivers/net/thunderx/nicvf_ethdev.c     |  7 +++---
 drivers/net/vhost/rte_eth_vhost.c       |  7 +++---
 drivers/net/virtio/virtio_ethdev.c      |  2 ++
 drivers/net/vmxnet3/vmxnet3_ethdev.c    |  2 ++
 39 files changed, 125 insertions(+), 35 deletions(-)

diff --git a/drivers/net/af_xdp/rte_eth_af_xdp.c b/drivers/net/af_xdp/rte_eth_af_xdp.c
index 60add9ead4..b289076e01 100644
--- a/drivers/net/af_xdp/rte_eth_af_xdp.c
+++ b/drivers/net/af_xdp/rte_eth_af_xdp.c
@@ -708,6 +708,9 @@ eth_dev_close(struct rte_eth_dev *dev)
 	struct pkt_rx_queue *rxq;
 	int i;
 
+	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
+		return 0;
+
 	AF_XDP_LOG(INFO, "Closing AF_XDP ethdev on numa socket %u\n",
 		rte_socket_id());
 
diff --git a/drivers/net/ark/ark_ethdev.c b/drivers/net/ark/ark_ethdev.c
index 83dc4ecd2c..3e96445fdb 100644
--- a/drivers/net/ark/ark_ethdev.c
+++ b/drivers/net/ark/ark_ethdev.c
@@ -680,6 +680,9 @@ eth_ark_dev_close(struct rte_eth_dev *dev)
 	struct ark_adapter *ark = dev->data->dev_private;
 	uint16_t i;
 
+	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
+		return 0;
+
 	if (ark->user_ext.dev_close)
 		ark->user_ext.dev_close(dev,
 		 ark->user_data[dev->data->port_id]);
diff --git a/drivers/net/avp/avp_ethdev.c b/drivers/net/avp/avp_ethdev.c
index c730b7ab86..95fdb57451 100644
--- a/drivers/net/avp/avp_ethdev.c
+++ b/drivers/net/avp/avp_ethdev.c
@@ -2107,6 +2107,9 @@ avp_dev_close(struct rte_eth_dev *eth_dev)
 	struct avp_dev *avp = AVP_DEV_PRIVATE_TO_HW(eth_dev->data->dev_private);
 	int ret;
 
+	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
+		return 0;
+
 	rte_spinlock_lock(&avp->lock);
 	if (avp->flags & AVP_F_DETACHED) {
 		PMD_DRV_LOG(ERR, "Operation not supported during VM live migration\n");
diff --git a/drivers/net/bnxt/bnxt_ethdev.c b/drivers/net/bnxt/bnxt_ethdev.c
index a77bab661d..d45347e8ca 100644
--- a/drivers/net/bnxt/bnxt_ethdev.c
+++ b/drivers/net/bnxt/bnxt_ethdev.c
@@ -1361,6 +1361,9 @@ static int bnxt_dev_close_op(struct rte_eth_dev *eth_dev)
 {
 	struct bnxt *bp = eth_dev->data->dev_private;
 
+	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
+		return 0;
+
 	/* cancel the recovery handler before remove dev */
 	rte_eal_alarm_cancel(bnxt_dev_reset_and_resume, (void *)bp);
 	rte_eal_alarm_cancel(bnxt_dev_recover, (void *)bp);
diff --git a/drivers/net/bnxt/bnxt_reps.c b/drivers/net/bnxt/bnxt_reps.c
index df8680c113..c419fe63b8 100644
--- a/drivers/net/bnxt/bnxt_reps.c
+++ b/drivers/net/bnxt/bnxt_reps.c
@@ -220,6 +220,9 @@ int bnxt_vf_representor_uninit(struct rte_eth_dev *eth_dev)
 		(struct bnxt_vf_representor *)eth_dev->data->dev_private;
 	uint16_t vf_id;
 
+	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
+		return 0;
+
 	PMD_DRV_LOG(DEBUG, "BNXT Port:%d VFR uninit\n", eth_dev->data->port_id);
 	eth_dev->data->mac_addrs = NULL;
 	eth_dev->dev_ops = NULL;
diff --git a/drivers/net/cxgbe/cxgbe_ethdev.c b/drivers/net/cxgbe/cxgbe_ethdev.c
index e4bbba5c32..16beb2d435 100644
--- a/drivers/net/cxgbe/cxgbe_ethdev.c
+++ b/drivers/net/cxgbe/cxgbe_ethdev.c
@@ -326,6 +326,9 @@ int cxgbe_dev_close(struct rte_eth_dev *eth_dev)
 
 	CXGBE_FUNC_TRACE();
 
+	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
+		return 0;
+
 	if (!(adapter->flags & FULL_INIT_DONE))
 		return 0;
 
diff --git a/drivers/net/dpaa/dpaa_ethdev.c b/drivers/net/dpaa/dpaa_ethdev.c
index 7fe06e1830..01632e591c 100644
--- a/drivers/net/dpaa/dpaa_ethdev.c
+++ b/drivers/net/dpaa/dpaa_ethdev.c
@@ -379,6 +379,9 @@ static int dpaa_eth_dev_close(struct rte_eth_dev *dev)
 
 	PMD_INIT_FUNC_TRACE();
 
+	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
+		return 0;
+
 	dpaa_dev = container_of(rdev, struct rte_dpaa_device, device);
 	intr_handle = &dpaa_dev->intr_handle;
 	__fif = container_of(fif, struct __fman_if, __if);
diff --git a/drivers/net/dpaa2/dpaa2_ethdev.c b/drivers/net/dpaa2/dpaa2_ethdev.c
index fb4165fa03..eb84d7231d 100644
--- a/drivers/net/dpaa2/dpaa2_ethdev.c
+++ b/drivers/net/dpaa2/dpaa2_ethdev.c
@@ -1246,6 +1246,9 @@ dpaa2_dev_close(struct rte_eth_dev *dev)
 
 	PMD_INIT_FUNC_TRACE();
 
+	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
+		return 0;
+
 	dpaa2_flow_clean(dev);
 
 	/* Clean the device first */
diff --git a/drivers/net/e1000/em_ethdev.c b/drivers/net/e1000/em_ethdev.c
index 6a6ae0e9d5..d050eb478a 100644
--- a/drivers/net/e1000/em_ethdev.c
+++ b/drivers/net/e1000/em_ethdev.c
@@ -762,6 +762,9 @@ eth_em_close(struct rte_eth_dev *dev)
 	struct rte_pci_device *pci_dev = RTE_ETH_DEV_TO_PCI(dev);
 	struct rte_intr_handle *intr_handle = &pci_dev->intr_handle;
 
+	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
+		return 0;
+
 	eth_em_stop(dev);
 	adapter->stopped = 1;
 	em_dev_free_queues(dev);
diff --git a/drivers/net/e1000/igb_ethdev.c b/drivers/net/e1000/igb_ethdev.c
index f8778207ef..cb3d97e2a3 100644
--- a/drivers/net/e1000/igb_ethdev.c
+++ b/drivers/net/e1000/igb_ethdev.c
@@ -1535,6 +1535,9 @@ eth_igb_close(struct rte_eth_dev *dev)
 	struct e1000_filter_info *filter_info =
 		E1000_DEV_PRIVATE_TO_FILTER_INFO(dev->data->dev_private);
 
+	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
+		return 0;
+
 	eth_igb_stop(dev);
 
 	e1000_phy_hw_reset(hw);
@@ -3382,6 +3385,9 @@ igbvf_dev_close(struct rte_eth_dev *dev)
 
 	PMD_INIT_FUNC_TRACE();
 
+	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
+		return 0;
+
 	e1000_reset_hw(hw);
 
 	igbvf_dev_stop(dev);
diff --git a/drivers/net/ena/ena_ethdev.c b/drivers/net/ena/ena_ethdev.c
index cf1c0b9795..07ae9bab32 100644
--- a/drivers/net/ena/ena_ethdev.c
+++ b/drivers/net/ena/ena_ethdev.c
@@ -493,6 +493,9 @@ static int ena_close(struct rte_eth_dev *dev)
 	struct rte_intr_handle *intr_handle = &pci_dev->intr_handle;
 	struct ena_adapter *adapter = dev->data->dev_private;
 
+	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
+		return 0;
+
 	if (adapter->state == ENA_ADAPTER_STATE_RUNNING)
 		ena_stop(dev);
 	adapter->state = ENA_ADAPTER_STATE_CLOSED;
diff --git a/drivers/net/enetc/enetc_ethdev.c b/drivers/net/enetc/enetc_ethdev.c
index bdb32762ce..d42add3efb 100644
--- a/drivers/net/enetc/enetc_ethdev.c
+++ b/drivers/net/enetc/enetc_ethdev.c
@@ -551,6 +551,9 @@ enetc_dev_close(struct rte_eth_dev *dev)
 	uint16_t i;
 
 	PMD_INIT_FUNC_TRACE();
+	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
+		return 0;
+
 	enetc_dev_stop(dev);
 
 	for (i = 0; i < dev->data->nb_rx_queues; i++) {
diff --git a/drivers/net/enic/enic_ethdev.c b/drivers/net/enic/enic_ethdev.c
index 27f60b45b8..60ee5e01de 100644
--- a/drivers/net/enic/enic_ethdev.c
+++ b/drivers/net/enic/enic_ethdev.c
@@ -451,6 +451,9 @@ static int enicpmd_dev_close(struct rte_eth_dev *eth_dev)
 	struct enic *enic = pmd_priv(eth_dev);
 
 	ENICPMD_FUNC_TRACE();
+	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
+		return 0;
+
 	enic_remove(enic);
 
 	return 0;
diff --git a/drivers/net/fm10k/fm10k_ethdev.c b/drivers/net/fm10k/fm10k_ethdev.c
index 23f4d04068..17cf0328c6 100644
--- a/drivers/net/fm10k/fm10k_ethdev.c
+++ b/drivers/net/fm10k/fm10k_ethdev.c
@@ -2784,6 +2784,8 @@ fm10k_dev_close(struct rte_eth_dev *dev)
 	struct rte_intr_handle *intr_handle = &pdev->intr_handle;
 
 	PMD_INIT_FUNC_TRACE();
+	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
+		return 0;
 
 	fm10k_mbx_lock(hw);
 	hw->mac.ops.update_lport_state(hw, hw->mac.dglort_map,
@@ -3236,14 +3238,7 @@ static int
 eth_fm10k_dev_uninit(struct rte_eth_dev *dev)
 {
 	PMD_INIT_FUNC_TRACE();
-
-	/* only uninitialize in the primary process */
-	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
-		return 0;
-
-	/* safe to close dev here */
 	fm10k_dev_close(dev);
-
 	return 0;
 }
 
diff --git a/drivers/net/hinic/hinic_pmd_ethdev.c b/drivers/net/hinic/hinic_pmd_ethdev.c
index 623534fda4..1b175aeace 100644
--- a/drivers/net/hinic/hinic_pmd_ethdev.c
+++ b/drivers/net/hinic/hinic_pmd_ethdev.c
@@ -2940,6 +2940,9 @@ static int hinic_dev_close(struct rte_eth_dev *dev)
 {
 	struct hinic_nic_dev *nic_dev = HINIC_ETH_DEV_TO_PRIVATE_NIC_DEV(dev);
 
+	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
+		return 0;
+
 	if (rte_bit_relaxed_test_and_set32(HINIC_DEV_CLOSE,
 					   &nic_dev->dev_status)) {
 		PMD_DRV_LOG(WARNING, "Device %s already closed",
diff --git a/drivers/net/i40e/i40e_ethdev.c b/drivers/net/i40e/i40e_ethdev.c
index 6fb88148b8..feeed2dec0 100644
--- a/drivers/net/i40e/i40e_ethdev.c
+++ b/drivers/net/i40e/i40e_ethdev.c
@@ -2620,6 +2620,8 @@ i40e_dev_close(struct rte_eth_dev *dev)
 	int retries = 0;
 
 	PMD_INIT_FUNC_TRACE();
+	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
+		return 0;
 
 	ret = rte_eth_switch_domain_free(pf->switch_domain_id);
 	if (ret)
diff --git a/drivers/net/i40e/i40e_ethdev_vf.c b/drivers/net/i40e/i40e_ethdev_vf.c
index 4aaf41956c..4d6510d1ff 100644
--- a/drivers/net/i40e/i40e_ethdev_vf.c
+++ b/drivers/net/i40e/i40e_ethdev_vf.c
@@ -2402,6 +2402,9 @@ i40evf_dev_close(struct rte_eth_dev *dev)
 	struct i40e_hw *hw = I40E_DEV_PRIVATE_TO_HW(dev->data->dev_private);
 	struct i40e_vf *vf = I40EVF_DEV_PRIVATE_TO_VF(dev->data->dev_private);
 
+	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
+		return 0;
+
 	i40evf_dev_stop(dev);
 	i40e_dev_free_queues(dev);
 	/*
diff --git a/drivers/net/iavf/iavf_ethdev.c b/drivers/net/iavf/iavf_ethdev.c
index 512ade2ad0..a5b1433306 100644
--- a/drivers/net/iavf/iavf_ethdev.c
+++ b/drivers/net/iavf/iavf_ethdev.c
@@ -1468,6 +1468,9 @@ iavf_dev_close(struct rte_eth_dev *dev)
 		IAVF_DEV_PRIVATE_TO_ADAPTER(dev->data->dev_private);
 	struct iavf_info *vf = IAVF_DEV_PRIVATE_TO_VF(dev->data->dev_private);
 
+	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
+		return 0;
+
 	iavf_dev_stop(dev);
 	iavf_flow_flush(dev, NULL);
 	iavf_flow_uninit(adapter);
diff --git a/drivers/net/ice/ice_ethdev.c b/drivers/net/ice/ice_ethdev.c
index 85a3ca6a2d..96159c92f2 100644
--- a/drivers/net/ice/ice_ethdev.c
+++ b/drivers/net/ice/ice_ethdev.c
@@ -2435,6 +2435,9 @@ ice_dev_close(struct rte_eth_dev *dev)
 	struct ice_adapter *ad =
 		ICE_DEV_PRIVATE_TO_ADAPTER(dev->data->dev_private);
 
+	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
+		return 0;
+
 	/* Since stop will make link down, then the link event will be
 	 * triggered, disable the irq firstly to avoid the port_infoe etc
 	 * resources deallocation causing the interrupt service thread
diff --git a/drivers/net/igc/igc_ethdev.c b/drivers/net/igc/igc_ethdev.c
index 9d27fc0d07..7f5066df4b 100644
--- a/drivers/net/igc/igc_ethdev.c
+++ b/drivers/net/igc/igc_ethdev.c
@@ -1175,6 +1175,8 @@ eth_igc_close(struct rte_eth_dev *dev)
 	int retry = 0;
 
 	PMD_INIT_FUNC_TRACE();
+	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
+		return 0;
 
 	if (!adapter->stopped)
 		eth_igc_stop(dev);
@@ -1363,10 +1365,6 @@ static int
 eth_igc_dev_uninit(__rte_unused struct rte_eth_dev *eth_dev)
 {
 	PMD_INIT_FUNC_TRACE();
-
-	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
-		return 0;
-
 	eth_igc_close(eth_dev);
 	return 0;
 }
diff --git a/drivers/net/ionic/ionic_ethdev.c b/drivers/net/ionic/ionic_ethdev.c
index 1775fd29b7..ef7d06e526 100644
--- a/drivers/net/ionic/ionic_ethdev.c
+++ b/drivers/net/ionic/ionic_ethdev.c
@@ -963,6 +963,8 @@ ionic_dev_close(struct rte_eth_dev *eth_dev)
 	int err;
 
 	IONIC_PRINT_CALL();
+	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
+		return 0;
 
 	err = ionic_lif_stop(lif);
 	if (err) {
diff --git a/drivers/net/ipn3ke/ipn3ke_representor.c b/drivers/net/ipn3ke/ipn3ke_representor.c
index d49abbf758..b9fb4d4e46 100644
--- a/drivers/net/ipn3ke/ipn3ke_representor.c
+++ b/drivers/net/ipn3ke/ipn3ke_representor.c
@@ -214,6 +214,9 @@ ipn3ke_rpst_dev_close(struct rte_eth_dev *dev)
 	struct ipn3ke_hw *hw = IPN3KE_DEV_PRIVATE_TO_HW(dev);
 	struct ipn3ke_rpst *rpst = IPN3KE_DEV_PRIVATE_TO_RPST(dev);
 
+	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
+		return 0;
+
 	if (hw->retimer.mac_type == IFPGA_RAWDEV_RETIMER_MAC_TYPE_10GE_XFI) {
 		/* Disable the TX path */
 		ipn3ke_xmac_tx_disable(hw, rpst->port_id, 0);
diff --git a/drivers/net/ixgbe/ixgbe_ethdev.c b/drivers/net/ixgbe/ixgbe_ethdev.c
index 426b7c9fe5..f6e383b1c2 100644
--- a/drivers/net/ixgbe/ixgbe_ethdev.c
+++ b/drivers/net/ixgbe/ixgbe_ethdev.c
@@ -2995,6 +2995,8 @@ ixgbe_dev_close(struct rte_eth_dev *dev)
 	int ret;
 
 	PMD_INIT_FUNC_TRACE();
+	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
+		return 0;
 
 	ixgbe_pf_reset_hw(hw);
 
@@ -5442,6 +5444,8 @@ ixgbevf_dev_close(struct rte_eth_dev *dev)
 	struct rte_intr_handle *intr_handle = &pci_dev->intr_handle;
 
 	PMD_INIT_FUNC_TRACE();
+	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
+		return 0;
 
 	ixgbe_reset_hw(hw);
 
diff --git a/drivers/net/kni/rte_eth_kni.c b/drivers/net/kni/rte_eth_kni.c
index 2a4058f7b0..be747adf86 100644
--- a/drivers/net/kni/rte_eth_kni.c
+++ b/drivers/net/kni/rte_eth_kni.c
@@ -204,6 +204,9 @@ eth_kni_close(struct rte_eth_dev *eth_dev)
 	struct pmd_internals *internals;
 	int ret;
 
+	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
+		return 0;
+
 	eth_kni_dev_stop(eth_dev);
 
 	/* mac_addrs must not be freed alone because part of dev_private */
diff --git a/drivers/net/liquidio/lio_ethdev.c b/drivers/net/liquidio/lio_ethdev.c
index 93e2ed5670..1a41f27198 100644
--- a/drivers/net/liquidio/lio_ethdev.c
+++ b/drivers/net/liquidio/lio_ethdev.c
@@ -1555,6 +1555,9 @@ lio_dev_close(struct rte_eth_dev *eth_dev)
 {
 	struct lio_device *lio_dev = LIO_DEV(eth_dev);
 
+	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
+		return 0;
+
 	lio_dev_info(lio_dev, "closing port %d\n", eth_dev->data->port_id);
 
 	if (lio_dev->intf_open)
diff --git a/drivers/net/mlx4/mlx4.c b/drivers/net/mlx4/mlx4.c
index 3e57875414..cfcfb8a8fc 100644
--- a/drivers/net/mlx4/mlx4.c
+++ b/drivers/net/mlx4/mlx4.c
@@ -376,6 +376,8 @@ mlx4_dev_close(struct rte_eth_dev *dev)
 	struct mlx4_priv *priv = dev->data->dev_private;
 	unsigned int i;
 
+	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
+		return 0;
 	DEBUG("%p: closing device \"%s\"",
 	      (void *)dev,
 	      ((priv->ctx != NULL) ? priv->ctx->device->name : ""));
diff --git a/drivers/net/mvneta/mvneta_ethdev.c b/drivers/net/mvneta/mvneta_ethdev.c
index db142bec23..607771149a 100644
--- a/drivers/net/mvneta/mvneta_ethdev.c
+++ b/drivers/net/mvneta/mvneta_ethdev.c
@@ -435,6 +435,9 @@ mvneta_dev_close(struct rte_eth_dev *dev)
 	struct mvneta_priv *priv = dev->data->dev_private;
 	int i;
 
+	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
+		return 0;
+
 	if (priv->ppio)
 		mvneta_dev_stop(dev);
 
diff --git a/drivers/net/mvpp2/mrvl_ethdev.c b/drivers/net/mvpp2/mrvl_ethdev.c
index cfb97e4f8f..a230a96840 100644
--- a/drivers/net/mvpp2/mrvl_ethdev.c
+++ b/drivers/net/mvpp2/mrvl_ethdev.c
@@ -861,6 +861,9 @@ mrvl_dev_close(struct rte_eth_dev *dev)
 	struct mrvl_priv *priv = dev->data->dev_private;
 	size_t i;
 
+	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
+		return 0;
+
 	mrvl_flush_rx_queues(dev);
 	mrvl_flush_tx_shadow_queues(dev);
 	mrvl_flow_deinit(dev);
diff --git a/drivers/net/netvsc/hn_ethdev.c b/drivers/net/netvsc/hn_ethdev.c
index 19a9eb6bc2..5ae2d469c8 100644
--- a/drivers/net/netvsc/hn_ethdev.c
+++ b/drivers/net/netvsc/hn_ethdev.c
@@ -842,6 +842,8 @@ static int
 hn_dev_close(struct rte_eth_dev *dev)
 {
 	PMD_INIT_FUNC_TRACE();
+	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
+		return 0;
 
 	hn_vf_close(dev);
 	hn_dev_free_queues(dev);
diff --git a/drivers/net/nfb/nfb_ethdev.c b/drivers/net/nfb/nfb_ethdev.c
index d937ac6922..7ee7294143 100644
--- a/drivers/net/nfb/nfb_ethdev.c
+++ b/drivers/net/nfb/nfb_ethdev.c
@@ -217,6 +217,9 @@ nfb_eth_dev_close(struct rte_eth_dev *dev)
 	uint16_t nb_rx = dev->data->nb_rx_queues;
 	uint16_t nb_tx = dev->data->nb_tx_queues;
 
+	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
+		return 0;
+
 	nfb_eth_dev_stop(dev);
 
 	nfb_nc_rxmac_deinit(internals->rxmac, internals->max_rxmac);
diff --git a/drivers/net/nfp/nfp_net.c b/drivers/net/nfp/nfp_net.c
index ce25cf1ed4..c1da66e3d6 100644
--- a/drivers/net/nfp/nfp_net.c
+++ b/drivers/net/nfp/nfp_net.c
@@ -871,6 +871,9 @@ nfp_net_close(struct rte_eth_dev *dev)
 	struct rte_pci_device *pci_dev;
 	int i;
 
+	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
+		return 0;
+
 	PMD_INIT_LOG(DEBUG, "Close");
 
 	hw = NFP_NET_DEV_PRIVATE_TO_HW(dev->data->dev_private);
diff --git a/drivers/net/octeontx/octeontx_ethdev.c b/drivers/net/octeontx/octeontx_ethdev.c
index a263f45399..aa9ef3bb70 100644
--- a/drivers/net/octeontx/octeontx_ethdev.c
+++ b/drivers/net/octeontx/octeontx_ethdev.c
@@ -487,6 +487,8 @@ octeontx_dev_close(struct rte_eth_dev *dev)
 	int ret;
 
 	PMD_INIT_FUNC_TRACE();
+	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
+		return 0;
 
 	rte_event_dev_close(nic->evdev);
 
diff --git a/drivers/net/pfe/pfe_ethdev.c b/drivers/net/pfe/pfe_ethdev.c
index 9d5415d9b1..a7e9d97ba9 100644
--- a/drivers/net/pfe/pfe_ethdev.c
+++ b/drivers/net/pfe/pfe_ethdev.c
@@ -407,6 +407,9 @@ pfe_eth_close(struct rte_eth_dev *dev)
 	if (!g_pfe)
 		return -1;
 
+	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
+		return 0;
+
 	pfe_eth_exit(dev, g_pfe);
 
 	if (g_pfe->nb_devs == 0) {
diff --git a/drivers/net/sfc/sfc_ethdev.c b/drivers/net/sfc/sfc_ethdev.c
index 1c72e40b75..0558f1808d 100644
--- a/drivers/net/sfc/sfc_ethdev.c
+++ b/drivers/net/sfc/sfc_ethdev.c
@@ -318,6 +318,17 @@ sfc_dev_set_link_down(struct rte_eth_dev *dev)
 	return 0;
 }
 
+static void
+sfc_eth_dev_secondary_clear_ops(struct rte_eth_dev *dev)
+{
+	free(dev->process_private);
+	dev->process_private = NULL;
+	dev->dev_ops = NULL;
+	dev->tx_pkt_prepare = NULL;
+	dev->tx_pkt_burst = NULL;
+	dev->rx_pkt_burst = NULL;
+}
+
 static int
 sfc_dev_close(struct rte_eth_dev *dev)
 {
@@ -325,6 +336,11 @@ sfc_dev_close(struct rte_eth_dev *dev)
 
 	sfc_log_init(sa, "entry");
 
+	if (rte_eal_process_type() != RTE_PROC_PRIMARY) {
+		sfc_eth_dev_secondary_clear_ops(dev);
+		return 0;
+	}
+
 	sfc_adapter_lock(sa);
 	switch (sa->state) {
 	case SFC_ADAPTER_STARTED:
@@ -2101,17 +2117,6 @@ sfc_eth_dev_secondary_init(struct rte_eth_dev *dev, uint32_t logtype_main)
 	return rc;
 }
 
-static void
-sfc_eth_dev_secondary_clear_ops(struct rte_eth_dev *dev)
-{
-	free(dev->process_private);
-	dev->process_private = NULL;
-	dev->dev_ops = NULL;
-	dev->tx_pkt_prepare = NULL;
-	dev->tx_pkt_burst = NULL;
-	dev->rx_pkt_burst = NULL;
-}
-
 static void
 sfc_register_dp(void)
 {
@@ -2245,11 +2250,6 @@ sfc_eth_dev_init(struct rte_eth_dev *dev)
 static int
 sfc_eth_dev_uninit(struct rte_eth_dev *dev)
 {
-	if (rte_eal_process_type() != RTE_PROC_PRIMARY) {
-		sfc_eth_dev_secondary_clear_ops(dev);
-		return 0;
-	}
-
 	sfc_dev_close(dev);
 
 	return 0;
diff --git a/drivers/net/szedata2/rte_eth_szedata2.c b/drivers/net/szedata2/rte_eth_szedata2.c
index 5f589dfa4c..7e9fafdcf7 100644
--- a/drivers/net/szedata2/rte_eth_szedata2.c
+++ b/drivers/net/szedata2/rte_eth_szedata2.c
@@ -1163,6 +1163,9 @@ eth_dev_close(struct rte_eth_dev *dev)
 	uint16_t nb_rx = dev->data->nb_rx_queues;
 	uint16_t nb_tx = dev->data->nb_tx_queues;
 
+	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
+		return 0;
+
 	eth_dev_stop(dev);
 
 	free(internals->sze_dev_path);
diff --git a/drivers/net/thunderx/nicvf_ethdev.c b/drivers/net/thunderx/nicvf_ethdev.c
index 3d7348771a..cc6eb4ba24 100644
--- a/drivers/net/thunderx/nicvf_ethdev.c
+++ b/drivers/net/thunderx/nicvf_ethdev.c
@@ -1859,6 +1859,8 @@ nicvf_dev_close(struct rte_eth_dev *dev)
 	struct nicvf *nic = nicvf_pmd_priv(dev);
 
 	PMD_INIT_FUNC_TRACE();
+	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
+		return 0;
 
 	nicvf_dev_stop_cleanup(dev, true);
 	nicvf_periodic_alarm_stop(nicvf_interrupt, dev);
@@ -2119,10 +2121,7 @@ static int
 nicvf_eth_dev_uninit(struct rte_eth_dev *dev)
 {
 	PMD_INIT_FUNC_TRACE();
-
-	if (rte_eal_process_type() == RTE_PROC_PRIMARY)
-		nicvf_dev_close(dev);
-
+	nicvf_dev_close(dev);
 	return 0;
 }
 static int
diff --git a/drivers/net/vhost/rte_eth_vhost.c b/drivers/net/vhost/rte_eth_vhost.c
index 45552ef742..32ad27fe4e 100644
--- a/drivers/net/vhost/rte_eth_vhost.c
+++ b/drivers/net/vhost/rte_eth_vhost.c
@@ -1171,6 +1171,9 @@ eth_dev_close(struct rte_eth_dev *dev)
 	struct internal_list *list;
 	unsigned int i;
 
+	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
+		return 0;
+
 	internal = dev->data->dev_private;
 	if (!internal)
 		return 0;
@@ -1655,11 +1658,7 @@ rte_pmd_vhost_remove(struct rte_vdev_device *dev)
 	if (eth_dev == NULL)
 		return 0;
 
-	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
-		return rte_eth_dev_release_port(eth_dev);
-
 	eth_dev_close(eth_dev);
-
 	rte_eth_dev_release_port(eth_dev);
 
 	return 0;
diff --git a/drivers/net/virtio/virtio_ethdev.c b/drivers/net/virtio/virtio_ethdev.c
index b6ed5829bd..f2117675b3 100644
--- a/drivers/net/virtio/virtio_ethdev.c
+++ b/drivers/net/virtio/virtio_ethdev.c
@@ -711,6 +711,8 @@ virtio_dev_close(struct rte_eth_dev *dev)
 	struct rte_intr_conf *intr_conf = &dev->data->dev_conf.intr_conf;
 
 	PMD_INIT_LOG(DEBUG, "virtio_dev_close");
+	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
+		return 0;
 
 	if (!hw->opened)
 		return 0;
diff --git a/drivers/net/vmxnet3/vmxnet3_ethdev.c b/drivers/net/vmxnet3/vmxnet3_ethdev.c
index 34a169d2c0..fa950e1ba0 100644
--- a/drivers/net/vmxnet3/vmxnet3_ethdev.c
+++ b/drivers/net/vmxnet3/vmxnet3_ethdev.c
@@ -889,6 +889,8 @@ static int
 vmxnet3_dev_close(struct rte_eth_dev *dev)
 {
 	PMD_INIT_FUNC_TRACE();
+	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
+		return 0;
 
 	vmxnet3_dev_stop(dev);
 	vmxnet3_free_queues(dev);
-- 
2.28.0

Re: [dpdk-dev] [PATCH v2 19/25] drivers/net: check process type in close operation

[Thomas Monjalon]

> The secondary processes are not allowed to release shared resources.
> Only process-private ressources should be freed in a secondary process.
> Most of the time, there is no process-private ressource,
> so the close operation is just forbidden in a secondary process.

Missed a (double) typo in this commit log:
	"ressource" -> "resource"

Re: [dpdk-dev] [PATCH v2 19/25] drivers/net: check process type in close operation

[Xu, Rosen]

Hi,

> -----Original Message-----
> From: Thomas Monjalon <thomas@monjalon.net>
> Sent: Monday, September 28, 2020 7:43
> To: dev@dpdk.org
> Cc: Yigit, Ferruh <ferruh.yigit@intel.com>; arybchenko@solarflare.com;
> Loftus, Ciara <ciara.loftus@intel.com>; Zhang, Qi Z <qi.z.zhang@intel.com>;
> Shepard Siegel <shepard.siegel@atomicrules.com>; Ed Czeck
> <ed.czeck@atomicrules.com>; John Miller <john.miller@atomicrules.com>;
> Steven Webster <steven.webster@windriver.com>; Matt Peters
> <matt.peters@windriver.com>; Ajit Khaparde
> <ajit.khaparde@broadcom.com>; Somnath Kotur
> <somnath.kotur@broadcom.com>; Rahul Lakkireddy
> <rahul.lakkireddy@chelsio.com>; Hemant Agrawal
> <hemant.agrawal@nxp.com>; Sachin Saxena <sachin.saxena@oss.nxp.com>;
> Guo, Jia <jia.guo@intel.com>; Wang, Haiyue <haiyue.wang@intel.com>;
> Marcin Wojtas <mw@semihalf.com>; Michal Krawczyk <mk@semihalf.com>;
> Guy Tzalik <gtzalik@amazon.com>; Evgeny Schemeilin
> <evgenys@amazon.com>; Igor Chauskin <igorch@amazon.com>;
> Gagandeep Singh <g.singh@nxp.com>; John Daley <johndale@cisco.com>;
> Hyong Youb Kim <hyonkim@cisco.com>; Wang, Xiao W
> <xiao.w.wang@intel.com>; Ziyang Xuan <xuanziyang2@huawei.com>;
> Xiaoyun Wang <cloud.wangxiaoyun@huawei.com>; Guoyang Zhou
> <zhouguoyang@huawei.com>; Xing, Beilei <beilei.xing@intel.com>; Wu,
> Jingjing <jingjing.wu@intel.com>; Yang, Qiming <qiming.yang@intel.com>;
> Alfredo Cardigliano <cardigliano@ntop.org>; Xu, Rosen
> <rosen.xu@intel.com>; Shijith Thotton <sthotton@marvell.com>;
> Srisivasubramanian Srinivasan <srinivasan@marvell.com>; Matan Azrad
> <matan@nvidia.com>; Shahaf Shuler <shahafs@nvidia.com>; Zyta Szpak
> <zr@semihalf.com>; Liron Himi <lironh@marvell.com>; Stephen Hemminger
> <sthemmin@microsoft.com>; K. Y. Srinivasan <kys@microsoft.com>;
> Haiyang Zhang <haiyangz@microsoft.com>; Long Li <longli@microsoft.com>;
> Martin Spinler <spinler@cesnet.cz>; Heinrich Kuhn
> <heinrich.kuhn@netronome.com>; Harman Kalra <hkalra@marvell.com>;
> Akhil Goyal <akhil.goyal@nxp.com>; Jerin Jacob <jerinj@marvell.com>;
> Maciej Czekaj <mczekaj@marvell.com>; Maxime Coquelin
> <maxime.coquelin@redhat.com>; Xia, Chenbo <chenbo.xia@intel.com>;
> Wang, Zhihong <zhihong.wang@intel.com>; Yong Wang
> <yongwang@vmware.com>; Burakov, Anatoly <anatoly.burakov@intel.com>
> Subject: [PATCH v2 19/25] drivers/net: check process type in close operation
> 
> The secondary processes are not allowed to release shared resources.
> Only process-private ressources should be freed in a secondary process.
> Most of the time, there is no process-private ressource, so the close
> operation is just forbidden in a secondary process.
> 
> After adding proper check in the port close functions, some redundant
> checks in the device remove functions are dropped.
> 
> Signed-off-by: Thomas Monjalon <thomas@monjalon.net>
> ---
>  drivers/net/af_xdp/rte_eth_af_xdp.c     |  3 +++
>  drivers/net/ark/ark_ethdev.c            |  3 +++
>  drivers/net/avp/avp_ethdev.c            |  3 +++
>  drivers/net/bnxt/bnxt_ethdev.c          |  3 +++
>  drivers/net/bnxt/bnxt_reps.c            |  3 +++
>  drivers/net/cxgbe/cxgbe_ethdev.c        |  3 +++
>  drivers/net/dpaa/dpaa_ethdev.c          |  3 +++
>  drivers/net/dpaa2/dpaa2_ethdev.c        |  3 +++
>  drivers/net/e1000/em_ethdev.c           |  3 +++
>  drivers/net/e1000/igb_ethdev.c          |  6 +++++
>  drivers/net/ena/ena_ethdev.c            |  3 +++
>  drivers/net/enetc/enetc_ethdev.c        |  3 +++
>  drivers/net/enic/enic_ethdev.c          |  3 +++
>  drivers/net/fm10k/fm10k_ethdev.c        |  9 ++-----
>  drivers/net/hinic/hinic_pmd_ethdev.c    |  3 +++
>  drivers/net/i40e/i40e_ethdev.c          |  2 ++
>  drivers/net/i40e/i40e_ethdev_vf.c       |  3 +++
>  drivers/net/iavf/iavf_ethdev.c          |  3 +++
>  drivers/net/ice/ice_ethdev.c            |  3 +++
>  drivers/net/igc/igc_ethdev.c            |  6 ++---
>  drivers/net/ionic/ionic_ethdev.c        |  2 ++
>  drivers/net/ipn3ke/ipn3ke_representor.c |  3 +++
>  drivers/net/ixgbe/ixgbe_ethdev.c        |  4 ++++
>  drivers/net/kni/rte_eth_kni.c           |  3 +++
>  drivers/net/liquidio/lio_ethdev.c       |  3 +++
>  drivers/net/mlx4/mlx4.c                 |  2 ++
>  drivers/net/mvneta/mvneta_ethdev.c      |  3 +++
>  drivers/net/mvpp2/mrvl_ethdev.c         |  3 +++
>  drivers/net/netvsc/hn_ethdev.c          |  2 ++
>  drivers/net/nfb/nfb_ethdev.c            |  3 +++
>  drivers/net/nfp/nfp_net.c               |  3 +++
>  drivers/net/octeontx/octeontx_ethdev.c  |  2 ++
>  drivers/net/pfe/pfe_ethdev.c            |  3 +++
>  drivers/net/sfc/sfc_ethdev.c            | 32 ++++++++++++-------------
>  drivers/net/szedata2/rte_eth_szedata2.c |  3 +++
>  drivers/net/thunderx/nicvf_ethdev.c     |  7 +++---
>  drivers/net/vhost/rte_eth_vhost.c       |  7 +++---
>  drivers/net/virtio/virtio_ethdev.c      |  2 ++
>  drivers/net/vmxnet3/vmxnet3_ethdev.c    |  2 ++
>  39 files changed, 125 insertions(+), 35 deletions(-)
> 
> diff --git a/drivers/net/ipn3ke/ipn3ke_representor.c
> b/drivers/net/ipn3ke/ipn3ke_representor.c
> index d49abbf758..b9fb4d4e46 100644
> --- a/drivers/net/ipn3ke/ipn3ke_representor.c
> +++ b/drivers/net/ipn3ke/ipn3ke_representor.c
> @@ -214,6 +214,9 @@ ipn3ke_rpst_dev_close(struct rte_eth_dev *dev)
>  	struct ipn3ke_hw *hw = IPN3KE_DEV_PRIVATE_TO_HW(dev);
>  	struct ipn3ke_rpst *rpst = IPN3KE_DEV_PRIVATE_TO_RPST(dev);
> 
> +	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
> +		return 0;
> +
>  	if (hw->retimer.mac_type ==
> IFPGA_RAWDEV_RETIMER_MAC_TYPE_10GE_XFI) {
>  		/* Disable the TX path */
>  		ipn3ke_xmac_tx_disable(hw, rpst->port_id, 0); diff --git
> 2.28.0

For net/ipn3ke
Reviewed-by: Rosen Xu <rosen.xu@intel.com>

Re: [dpdk-dev] [PATCH v2 19/25] drivers/net: check process type in close operation

[Sachin Saxena (OSS)]

For dpaa, dpaa2, enetc and pfe

Reviewed-by: Sachin Saxena<sachin.saxena@oss.nxp.com>

On 28-Sep-20 5:12 AM, Thomas Monjalon wrote:
> The secondary processes are not allowed to release shared resources.
> Only process-private ressources should be freed in a secondary process.
> Most of the time, there is no process-private ressource,
> so the close operation is just forbidden in a secondary process.
>
> After adding proper check in the port close functions,
> some redundant checks in the device remove functions are dropped.
>
> Signed-off-by: Thomas Monjalon <thomas@monjalon.net>
> ---
>   drivers/net/af_xdp/rte_eth_af_xdp.c     |  3 +++
>   drivers/net/ark/ark_ethdev.c            |  3 +++
>   drivers/net/avp/avp_ethdev.c            |  3 +++
>   drivers/net/bnxt/bnxt_ethdev.c          |  3 +++
>   drivers/net/bnxt/bnxt_reps.c            |  3 +++
>   drivers/net/cxgbe/cxgbe_ethdev.c        |  3 +++
>   drivers/net/dpaa/dpaa_ethdev.c          |  3 +++
>   drivers/net/dpaa2/dpaa2_ethdev.c        |  3 +++
>   drivers/net/e1000/em_ethdev.c           |  3 +++
>   drivers/net/e1000/igb_ethdev.c          |  6 +++++
>   drivers/net/ena/ena_ethdev.c            |  3 +++
>   drivers/net/enetc/enetc_ethdev.c        |  3 +++
>   drivers/net/enic/enic_ethdev.c          |  3 +++
>   drivers/net/fm10k/fm10k_ethdev.c        |  9 ++-----
>   drivers/net/hinic/hinic_pmd_ethdev.c    |  3 +++
>   drivers/net/i40e/i40e_ethdev.c          |  2 ++
>   drivers/net/i40e/i40e_ethdev_vf.c       |  3 +++
>   drivers/net/iavf/iavf_ethdev.c          |  3 +++
>   drivers/net/ice/ice_ethdev.c            |  3 +++
>   drivers/net/igc/igc_ethdev.c            |  6 ++---
>   drivers/net/ionic/ionic_ethdev.c        |  2 ++
>   drivers/net/ipn3ke/ipn3ke_representor.c |  3 +++
>   drivers/net/ixgbe/ixgbe_ethdev.c        |  4 ++++
>   drivers/net/kni/rte_eth_kni.c           |  3 +++
>   drivers/net/liquidio/lio_ethdev.c       |  3 +++
>   drivers/net/mlx4/mlx4.c                 |  2 ++
>   drivers/net/mvneta/mvneta_ethdev.c      |  3 +++
>   drivers/net/mvpp2/mrvl_ethdev.c         |  3 +++
>   drivers/net/netvsc/hn_ethdev.c          |  2 ++
>   drivers/net/nfb/nfb_ethdev.c            |  3 +++
>   drivers/net/nfp/nfp_net.c               |  3 +++
>   drivers/net/octeontx/octeontx_ethdev.c  |  2 ++
>   drivers/net/pfe/pfe_ethdev.c            |  3 +++
>   drivers/net/sfc/sfc_ethdev.c            | 32 ++++++++++++-------------
>   drivers/net/szedata2/rte_eth_szedata2.c |  3 +++
>   drivers/net/thunderx/nicvf_ethdev.c     |  7 +++---
>   drivers/net/vhost/rte_eth_vhost.c       |  7 +++---
>   drivers/net/virtio/virtio_ethdev.c      |  2 ++
>   drivers/net/vmxnet3/vmxnet3_ethdev.c    |  2 ++
>   39 files changed, 125 insertions(+), 35 deletions(-)
>
> diff --git a/drivers/net/af_xdp/rte_eth_af_xdp.c b/drivers/net/af_xdp/rte_eth_af_xdp.c
> index 60add9ead4..b289076e01 100644
> --- a/drivers/net/af_xdp/rte_eth_af_xdp.c
> +++ b/drivers/net/af_xdp/rte_eth_af_xdp.c
> @@ -708,6 +708,9 @@ eth_dev_close(struct rte_eth_dev *dev)
>   	struct pkt_rx_queue *rxq;
>   	int i;
>   
> +	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
> +		return 0;
> +
>   	AF_XDP_LOG(INFO, "Closing AF_XDP ethdev on numa socket %u\n",
>   		rte_socket_id());
>   
> diff --git a/drivers/net/ark/ark_ethdev.c b/drivers/net/ark/ark_ethdev.c
> index 83dc4ecd2c..3e96445fdb 100644
> --- a/drivers/net/ark/ark_ethdev.c
> +++ b/drivers/net/ark/ark_ethdev.c
> @@ -680,6 +680,9 @@ eth_ark_dev_close(struct rte_eth_dev *dev)
>   	struct ark_adapter *ark = dev->data->dev_private;
>   	uint16_t i;
>   
> +	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
> +		return 0;
> +
>   	if (ark->user_ext.dev_close)
>   		ark->user_ext.dev_close(dev,
>   		 ark->user_data[dev->data->port_id]);
> diff --git a/drivers/net/avp/avp_ethdev.c b/drivers/net/avp/avp_ethdev.c
> index c730b7ab86..95fdb57451 100644
> --- a/drivers/net/avp/avp_ethdev.c
> +++ b/drivers/net/avp/avp_ethdev.c
> @@ -2107,6 +2107,9 @@ avp_dev_close(struct rte_eth_dev *eth_dev)
>   	struct avp_dev *avp = AVP_DEV_PRIVATE_TO_HW(eth_dev->data->dev_private);
>   	int ret;
>   
> +	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
> +		return 0;
> +
>   	rte_spinlock_lock(&avp->lock);
>   	if (avp->flags & AVP_F_DETACHED) {
>   		PMD_DRV_LOG(ERR, "Operation not supported during VM live migration\n");
> diff --git a/drivers/net/bnxt/bnxt_ethdev.c b/drivers/net/bnxt/bnxt_ethdev.c
> index a77bab661d..d45347e8ca 100644
> --- a/drivers/net/bnxt/bnxt_ethdev.c
> +++ b/drivers/net/bnxt/bnxt_ethdev.c
> @@ -1361,6 +1361,9 @@ static int bnxt_dev_close_op(struct rte_eth_dev *eth_dev)
>   {
>   	struct bnxt *bp = eth_dev->data->dev_private;
>   
> +	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
> +		return 0;
> +
>   	/* cancel the recovery handler before remove dev */
>   	rte_eal_alarm_cancel(bnxt_dev_reset_and_resume, (void *)bp);
>   	rte_eal_alarm_cancel(bnxt_dev_recover, (void *)bp);
> diff --git a/drivers/net/bnxt/bnxt_reps.c b/drivers/net/bnxt/bnxt_reps.c
> index df8680c113..c419fe63b8 100644
> --- a/drivers/net/bnxt/bnxt_reps.c
> +++ b/drivers/net/bnxt/bnxt_reps.c
> @@ -220,6 +220,9 @@ int bnxt_vf_representor_uninit(struct rte_eth_dev *eth_dev)
>   		(struct bnxt_vf_representor *)eth_dev->data->dev_private;
>   	uint16_t vf_id;
>   
> +	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
> +		return 0;
> +
>   	PMD_DRV_LOG(DEBUG, "BNXT Port:%d VFR uninit\n", eth_dev->data->port_id);
>   	eth_dev->data->mac_addrs = NULL;
>   	eth_dev->dev_ops = NULL;
> diff --git a/drivers/net/cxgbe/cxgbe_ethdev.c b/drivers/net/cxgbe/cxgbe_ethdev.c
> index e4bbba5c32..16beb2d435 100644
> --- a/drivers/net/cxgbe/cxgbe_ethdev.c
> +++ b/drivers/net/cxgbe/cxgbe_ethdev.c
> @@ -326,6 +326,9 @@ int cxgbe_dev_close(struct rte_eth_dev *eth_dev)
>   
>   	CXGBE_FUNC_TRACE();
>   
> +	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
> +		return 0;
> +
>   	if (!(adapter->flags & FULL_INIT_DONE))
>   		return 0;
>   
> diff --git a/drivers/net/dpaa/dpaa_ethdev.c b/drivers/net/dpaa/dpaa_ethdev.c
> index 7fe06e1830..01632e591c 100644
> --- a/drivers/net/dpaa/dpaa_ethdev.c
> +++ b/drivers/net/dpaa/dpaa_ethdev.c
> @@ -379,6 +379,9 @@ static int dpaa_eth_dev_close(struct rte_eth_dev *dev)
>   
>   	PMD_INIT_FUNC_TRACE();
>   
> +	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
> +		return 0;
> +
>   	dpaa_dev = container_of(rdev, struct rte_dpaa_device, device);
>   	intr_handle = &dpaa_dev->intr_handle;
>   	__fif = container_of(fif, struct __fman_if, __if);
> diff --git a/drivers/net/dpaa2/dpaa2_ethdev.c b/drivers/net/dpaa2/dpaa2_ethdev.c
> index fb4165fa03..eb84d7231d 100644
> --- a/drivers/net/dpaa2/dpaa2_ethdev.c
> +++ b/drivers/net/dpaa2/dpaa2_ethdev.c
> @@ -1246,6 +1246,9 @@ dpaa2_dev_close(struct rte_eth_dev *dev)
>   
>   	PMD_INIT_FUNC_TRACE();
>   
> +	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
> +		return 0;
> +
>   	dpaa2_flow_clean(dev);
>   
>   	/* Clean the device first */
> diff --git a/drivers/net/e1000/em_ethdev.c b/drivers/net/e1000/em_ethdev.c
> index 6a6ae0e9d5..d050eb478a 100644
> --- a/drivers/net/e1000/em_ethdev.c
> +++ b/drivers/net/e1000/em_ethdev.c
> @@ -762,6 +762,9 @@ eth_em_close(struct rte_eth_dev *dev)
>   	struct rte_pci_device *pci_dev = RTE_ETH_DEV_TO_PCI(dev);
>   	struct rte_intr_handle *intr_handle = &pci_dev->intr_handle;
>   
> +	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
> +		return 0;
> +
>   	eth_em_stop(dev);
>   	adapter->stopped = 1;
>   	em_dev_free_queues(dev);
> diff --git a/drivers/net/e1000/igb_ethdev.c b/drivers/net/e1000/igb_ethdev.c
> index f8778207ef..cb3d97e2a3 100644
> --- a/drivers/net/e1000/igb_ethdev.c
> +++ b/drivers/net/e1000/igb_ethdev.c
> @@ -1535,6 +1535,9 @@ eth_igb_close(struct rte_eth_dev *dev)
>   	struct e1000_filter_info *filter_info =
>   		E1000_DEV_PRIVATE_TO_FILTER_INFO(dev->data->dev_private);
>   
> +	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
> +		return 0;
> +
>   	eth_igb_stop(dev);
>   
>   	e1000_phy_hw_reset(hw);
> @@ -3382,6 +3385,9 @@ igbvf_dev_close(struct rte_eth_dev *dev)
>   
>   	PMD_INIT_FUNC_TRACE();
>   
> +	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
> +		return 0;
> +
>   	e1000_reset_hw(hw);
>   
>   	igbvf_dev_stop(dev);
> diff --git a/drivers/net/ena/ena_ethdev.c b/drivers/net/ena/ena_ethdev.c
> index cf1c0b9795..07ae9bab32 100644
> --- a/drivers/net/ena/ena_ethdev.c
> +++ b/drivers/net/ena/ena_ethdev.c
> @@ -493,6 +493,9 @@ static int ena_close(struct rte_eth_dev *dev)
>   	struct rte_intr_handle *intr_handle = &pci_dev->intr_handle;
>   	struct ena_adapter *adapter = dev->data->dev_private;
>   
> +	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
> +		return 0;
> +
>   	if (adapter->state == ENA_ADAPTER_STATE_RUNNING)
>   		ena_stop(dev);
>   	adapter->state = ENA_ADAPTER_STATE_CLOSED;
> diff --git a/drivers/net/enetc/enetc_ethdev.c b/drivers/net/enetc/enetc_ethdev.c
> index bdb32762ce..d42add3efb 100644
> --- a/drivers/net/enetc/enetc_ethdev.c
> +++ b/drivers/net/enetc/enetc_ethdev.c
> @@ -551,6 +551,9 @@ enetc_dev_close(struct rte_eth_dev *dev)
>   	uint16_t i;
>   
>   	PMD_INIT_FUNC_TRACE();
> +	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
> +		return 0;
> +
>   	enetc_dev_stop(dev);
>   
>   	for (i = 0; i < dev->data->nb_rx_queues; i++) {
> diff --git a/drivers/net/enic/enic_ethdev.c b/drivers/net/enic/enic_ethdev.c
> index 27f60b45b8..60ee5e01de 100644
> --- a/drivers/net/enic/enic_ethdev.c
> +++ b/drivers/net/enic/enic_ethdev.c
> @@ -451,6 +451,9 @@ static int enicpmd_dev_close(struct rte_eth_dev *eth_dev)
>   	struct enic *enic = pmd_priv(eth_dev);
>   
>   	ENICPMD_FUNC_TRACE();
> +	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
> +		return 0;
> +
>   	enic_remove(enic);
>   
>   	return 0;
> diff --git a/drivers/net/fm10k/fm10k_ethdev.c b/drivers/net/fm10k/fm10k_ethdev.c
> index 23f4d04068..17cf0328c6 100644
> --- a/drivers/net/fm10k/fm10k_ethdev.c
> +++ b/drivers/net/fm10k/fm10k_ethdev.c
> @@ -2784,6 +2784,8 @@ fm10k_dev_close(struct rte_eth_dev *dev)
>   	struct rte_intr_handle *intr_handle = &pdev->intr_handle;
>   
>   	PMD_INIT_FUNC_TRACE();
> +	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
> +		return 0;
>   
>   	fm10k_mbx_lock(hw);
>   	hw->mac.ops.update_lport_state(hw, hw->mac.dglort_map,
> @@ -3236,14 +3238,7 @@ static int
>   eth_fm10k_dev_uninit(struct rte_eth_dev *dev)
>   {
>   	PMD_INIT_FUNC_TRACE();
> -
> -	/* only uninitialize in the primary process */
> -	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
> -		return 0;
> -
> -	/* safe to close dev here */
>   	fm10k_dev_close(dev);
> -
>   	return 0;
>   }
>   
> diff --git a/drivers/net/hinic/hinic_pmd_ethdev.c b/drivers/net/hinic/hinic_pmd_ethdev.c
> index 623534fda4..1b175aeace 100644
> --- a/drivers/net/hinic/hinic_pmd_ethdev.c
> +++ b/drivers/net/hinic/hinic_pmd_ethdev.c
> @@ -2940,6 +2940,9 @@ static int hinic_dev_close(struct rte_eth_dev *dev)
>   {
>   	struct hinic_nic_dev *nic_dev = HINIC_ETH_DEV_TO_PRIVATE_NIC_DEV(dev);
>   
> +	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
> +		return 0;
> +
>   	if (rte_bit_relaxed_test_and_set32(HINIC_DEV_CLOSE,
>   					   &nic_dev->dev_status)) {
>   		PMD_DRV_LOG(WARNING, "Device %s already closed",
> diff --git a/drivers/net/i40e/i40e_ethdev.c b/drivers/net/i40e/i40e_ethdev.c
> index 6fb88148b8..feeed2dec0 100644
> --- a/drivers/net/i40e/i40e_ethdev.c
> +++ b/drivers/net/i40e/i40e_ethdev.c
> @@ -2620,6 +2620,8 @@ i40e_dev_close(struct rte_eth_dev *dev)
>   	int retries = 0;
>   
>   	PMD_INIT_FUNC_TRACE();
> +	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
> +		return 0;
>   
>   	ret = rte_eth_switch_domain_free(pf->switch_domain_id);
>   	if (ret)
> diff --git a/drivers/net/i40e/i40e_ethdev_vf.c b/drivers/net/i40e/i40e_ethdev_vf.c
> index 4aaf41956c..4d6510d1ff 100644
> --- a/drivers/net/i40e/i40e_ethdev_vf.c
> +++ b/drivers/net/i40e/i40e_ethdev_vf.c
> @@ -2402,6 +2402,9 @@ i40evf_dev_close(struct rte_eth_dev *dev)
>   	struct i40e_hw *hw = I40E_DEV_PRIVATE_TO_HW(dev->data->dev_private);
>   	struct i40e_vf *vf = I40EVF_DEV_PRIVATE_TO_VF(dev->data->dev_private);
>   
> +	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
> +		return 0;
> +
>   	i40evf_dev_stop(dev);
>   	i40e_dev_free_queues(dev);
>   	/*
> diff --git a/drivers/net/iavf/iavf_ethdev.c b/drivers/net/iavf/iavf_ethdev.c
> index 512ade2ad0..a5b1433306 100644
> --- a/drivers/net/iavf/iavf_ethdev.c
> +++ b/drivers/net/iavf/iavf_ethdev.c
> @@ -1468,6 +1468,9 @@ iavf_dev_close(struct rte_eth_dev *dev)
>   		IAVF_DEV_PRIVATE_TO_ADAPTER(dev->data->dev_private);
>   	struct iavf_info *vf = IAVF_DEV_PRIVATE_TO_VF(dev->data->dev_private);
>   
> +	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
> +		return 0;
> +
>   	iavf_dev_stop(dev);
>   	iavf_flow_flush(dev, NULL);
>   	iavf_flow_uninit(adapter);
> diff --git a/drivers/net/ice/ice_ethdev.c b/drivers/net/ice/ice_ethdev.c
> index 85a3ca6a2d..96159c92f2 100644
> --- a/drivers/net/ice/ice_ethdev.c
> +++ b/drivers/net/ice/ice_ethdev.c
> @@ -2435,6 +2435,9 @@ ice_dev_close(struct rte_eth_dev *dev)
>   	struct ice_adapter *ad =
>   		ICE_DEV_PRIVATE_TO_ADAPTER(dev->data->dev_private);
>   
> +	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
> +		return 0;
> +
>   	/* Since stop will make link down, then the link event will be
>   	 * triggered, disable the irq firstly to avoid the port_infoe etc
>   	 * resources deallocation causing the interrupt service thread
> diff --git a/drivers/net/igc/igc_ethdev.c b/drivers/net/igc/igc_ethdev.c
> index 9d27fc0d07..7f5066df4b 100644
> --- a/drivers/net/igc/igc_ethdev.c
> +++ b/drivers/net/igc/igc_ethdev.c
> @@ -1175,6 +1175,8 @@ eth_igc_close(struct rte_eth_dev *dev)
>   	int retry = 0;
>   
>   	PMD_INIT_FUNC_TRACE();
> +	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
> +		return 0;
>   
>   	if (!adapter->stopped)
>   		eth_igc_stop(dev);
> @@ -1363,10 +1365,6 @@ static int
>   eth_igc_dev_uninit(__rte_unused struct rte_eth_dev *eth_dev)
>   {
>   	PMD_INIT_FUNC_TRACE();
> -
> -	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
> -		return 0;
> -
>   	eth_igc_close(eth_dev);
>   	return 0;
>   }
> diff --git a/drivers/net/ionic/ionic_ethdev.c b/drivers/net/ionic/ionic_ethdev.c
> index 1775fd29b7..ef7d06e526 100644
> --- a/drivers/net/ionic/ionic_ethdev.c
> +++ b/drivers/net/ionic/ionic_ethdev.c
> @@ -963,6 +963,8 @@ ionic_dev_close(struct rte_eth_dev *eth_dev)
>   	int err;
>   
>   	IONIC_PRINT_CALL();
> +	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
> +		return 0;
>   
>   	err = ionic_lif_stop(lif);
>   	if (err) {
> diff --git a/drivers/net/ipn3ke/ipn3ke_representor.c b/drivers/net/ipn3ke/ipn3ke_representor.c
> index d49abbf758..b9fb4d4e46 100644
> --- a/drivers/net/ipn3ke/ipn3ke_representor.c
> +++ b/drivers/net/ipn3ke/ipn3ke_representor.c
> @@ -214,6 +214,9 @@ ipn3ke_rpst_dev_close(struct rte_eth_dev *dev)
>   	struct ipn3ke_hw *hw = IPN3KE_DEV_PRIVATE_TO_HW(dev);
>   	struct ipn3ke_rpst *rpst = IPN3KE_DEV_PRIVATE_TO_RPST(dev);
>   
> +	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
> +		return 0;
> +
>   	if (hw->retimer.mac_type == IFPGA_RAWDEV_RETIMER_MAC_TYPE_10GE_XFI) {
>   		/* Disable the TX path */
>   		ipn3ke_xmac_tx_disable(hw, rpst->port_id, 0);
> diff --git a/drivers/net/ixgbe/ixgbe_ethdev.c b/drivers/net/ixgbe/ixgbe_ethdev.c
> index 426b7c9fe5..f6e383b1c2 100644
> --- a/drivers/net/ixgbe/ixgbe_ethdev.c
> +++ b/drivers/net/ixgbe/ixgbe_ethdev.c
> @@ -2995,6 +2995,8 @@ ixgbe_dev_close(struct rte_eth_dev *dev)
>   	int ret;
>   
>   	PMD_INIT_FUNC_TRACE();
> +	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
> +		return 0;
>   
>   	ixgbe_pf_reset_hw(hw);
>   
> @@ -5442,6 +5444,8 @@ ixgbevf_dev_close(struct rte_eth_dev *dev)
>   	struct rte_intr_handle *intr_handle = &pci_dev->intr_handle;
>   
>   	PMD_INIT_FUNC_TRACE();
> +	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
> +		return 0;
>   
>   	ixgbe_reset_hw(hw);
>   
> diff --git a/drivers/net/kni/rte_eth_kni.c b/drivers/net/kni/rte_eth_kni.c
> index 2a4058f7b0..be747adf86 100644
> --- a/drivers/net/kni/rte_eth_kni.c
> +++ b/drivers/net/kni/rte_eth_kni.c
> @@ -204,6 +204,9 @@ eth_kni_close(struct rte_eth_dev *eth_dev)
>   	struct pmd_internals *internals;
>   	int ret;
>   
> +	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
> +		return 0;
> +
>   	eth_kni_dev_stop(eth_dev);
>   
>   	/* mac_addrs must not be freed alone because part of dev_private */
> diff --git a/drivers/net/liquidio/lio_ethdev.c b/drivers/net/liquidio/lio_ethdev.c
> index 93e2ed5670..1a41f27198 100644
> --- a/drivers/net/liquidio/lio_ethdev.c
> +++ b/drivers/net/liquidio/lio_ethdev.c
> @@ -1555,6 +1555,9 @@ lio_dev_close(struct rte_eth_dev *eth_dev)
>   {
>   	struct lio_device *lio_dev = LIO_DEV(eth_dev);
>   
> +	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
> +		return 0;
> +
>   	lio_dev_info(lio_dev, "closing port %d\n", eth_dev->data->port_id);
>   
>   	if (lio_dev->intf_open)
> diff --git a/drivers/net/mlx4/mlx4.c b/drivers/net/mlx4/mlx4.c
> index 3e57875414..cfcfb8a8fc 100644
> --- a/drivers/net/mlx4/mlx4.c
> +++ b/drivers/net/mlx4/mlx4.c
> @@ -376,6 +376,8 @@ mlx4_dev_close(struct rte_eth_dev *dev)
>   	struct mlx4_priv *priv = dev->data->dev_private;
>   	unsigned int i;
>   
> +	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
> +		return 0;
>   	DEBUG("%p: closing device \"%s\"",
>   	      (void *)dev,
>   	      ((priv->ctx != NULL) ? priv->ctx->device->name : ""));
> diff --git a/drivers/net/mvneta/mvneta_ethdev.c b/drivers/net/mvneta/mvneta_ethdev.c
> index db142bec23..607771149a 100644
> --- a/drivers/net/mvneta/mvneta_ethdev.c
> +++ b/drivers/net/mvneta/mvneta_ethdev.c
> @@ -435,6 +435,9 @@ mvneta_dev_close(struct rte_eth_dev *dev)
>   	struct mvneta_priv *priv = dev->data->dev_private;
>   	int i;
>   
> +	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
> +		return 0;
> +
>   	if (priv->ppio)
>   		mvneta_dev_stop(dev);
>   
> diff --git a/drivers/net/mvpp2/mrvl_ethdev.c b/drivers/net/mvpp2/mrvl_ethdev.c
> index cfb97e4f8f..a230a96840 100644
> --- a/drivers/net/mvpp2/mrvl_ethdev.c
> +++ b/drivers/net/mvpp2/mrvl_ethdev.c
> @@ -861,6 +861,9 @@ mrvl_dev_close(struct rte_eth_dev *dev)
>   	struct mrvl_priv *priv = dev->data->dev_private;
>   	size_t i;
>   
> +	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
> +		return 0;
> +
>   	mrvl_flush_rx_queues(dev);
>   	mrvl_flush_tx_shadow_queues(dev);
>   	mrvl_flow_deinit(dev);
> diff --git a/drivers/net/netvsc/hn_ethdev.c b/drivers/net/netvsc/hn_ethdev.c
> index 19a9eb6bc2..5ae2d469c8 100644
> --- a/drivers/net/netvsc/hn_ethdev.c
> +++ b/drivers/net/netvsc/hn_ethdev.c
> @@ -842,6 +842,8 @@ static int
>   hn_dev_close(struct rte_eth_dev *dev)
>   {
>   	PMD_INIT_FUNC_TRACE();
> +	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
> +		return 0;
>   
>   	hn_vf_close(dev);
>   	hn_dev_free_queues(dev);
> diff --git a/drivers/net/nfb/nfb_ethdev.c b/drivers/net/nfb/nfb_ethdev.c
> index d937ac6922..7ee7294143 100644
> --- a/drivers/net/nfb/nfb_ethdev.c
> +++ b/drivers/net/nfb/nfb_ethdev.c
> @@ -217,6 +217,9 @@ nfb_eth_dev_close(struct rte_eth_dev *dev)
>   	uint16_t nb_rx = dev->data->nb_rx_queues;
>   	uint16_t nb_tx = dev->data->nb_tx_queues;
>   
> +	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
> +		return 0;
> +
>   	nfb_eth_dev_stop(dev);
>   
>   	nfb_nc_rxmac_deinit(internals->rxmac, internals->max_rxmac);
> diff --git a/drivers/net/nfp/nfp_net.c b/drivers/net/nfp/nfp_net.c
> index ce25cf1ed4..c1da66e3d6 100644
> --- a/drivers/net/nfp/nfp_net.c
> +++ b/drivers/net/nfp/nfp_net.c
> @@ -871,6 +871,9 @@ nfp_net_close(struct rte_eth_dev *dev)
>   	struct rte_pci_device *pci_dev;
>   	int i;
>   
> +	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
> +		return 0;
> +
>   	PMD_INIT_LOG(DEBUG, "Close");
>   
>   	hw = NFP_NET_DEV_PRIVATE_TO_HW(dev->data->dev_private);
> diff --git a/drivers/net/octeontx/octeontx_ethdev.c b/drivers/net/octeontx/octeontx_ethdev.c
> index a263f45399..aa9ef3bb70 100644
> --- a/drivers/net/octeontx/octeontx_ethdev.c
> +++ b/drivers/net/octeontx/octeontx_ethdev.c
> @@ -487,6 +487,8 @@ octeontx_dev_close(struct rte_eth_dev *dev)
>   	int ret;
>   
>   	PMD_INIT_FUNC_TRACE();
> +	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
> +		return 0;
>   
>   	rte_event_dev_close(nic->evdev);
>   
> diff --git a/drivers/net/pfe/pfe_ethdev.c b/drivers/net/pfe/pfe_ethdev.c
> index 9d5415d9b1..a7e9d97ba9 100644
> --- a/drivers/net/pfe/pfe_ethdev.c
> +++ b/drivers/net/pfe/pfe_ethdev.c
> @@ -407,6 +407,9 @@ pfe_eth_close(struct rte_eth_dev *dev)
>   	if (!g_pfe)
>   		return -1;
>   
> +	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
> +		return 0;
> +
>   	pfe_eth_exit(dev, g_pfe);
>   
>   	if (g_pfe->nb_devs == 0) {
> diff --git a/drivers/net/sfc/sfc_ethdev.c b/drivers/net/sfc/sfc_ethdev.c
> index 1c72e40b75..0558f1808d 100644
> --- a/drivers/net/sfc/sfc_ethdev.c
> +++ b/drivers/net/sfc/sfc_ethdev.c
> @@ -318,6 +318,17 @@ sfc_dev_set_link_down(struct rte_eth_dev *dev)
>   	return 0;
>   }
>   
> +static void
> +sfc_eth_dev_secondary_clear_ops(struct rte_eth_dev *dev)
> +{
> +	free(dev->process_private);
> +	dev->process_private = NULL;
> +	dev->dev_ops = NULL;
> +	dev->tx_pkt_prepare = NULL;
> +	dev->tx_pkt_burst = NULL;
> +	dev->rx_pkt_burst = NULL;
> +}
> +
>   static int
>   sfc_dev_close(struct rte_eth_dev *dev)
>   {
> @@ -325,6 +336,11 @@ sfc_dev_close(struct rte_eth_dev *dev)
>   
>   	sfc_log_init(sa, "entry");
>   
> +	if (rte_eal_process_type() != RTE_PROC_PRIMARY) {
> +		sfc_eth_dev_secondary_clear_ops(dev);
> +		return 0;
> +	}
> +
>   	sfc_adapter_lock(sa);
>   	switch (sa->state) {
>   	case SFC_ADAPTER_STARTED:
> @@ -2101,17 +2117,6 @@ sfc_eth_dev_secondary_init(struct rte_eth_dev *dev, uint32_t logtype_main)
>   	return rc;
>   }
>   
> -static void
> -sfc_eth_dev_secondary_clear_ops(struct rte_eth_dev *dev)
> -{
> -	free(dev->process_private);
> -	dev->process_private = NULL;
> -	dev->dev_ops = NULL;
> -	dev->tx_pkt_prepare = NULL;
> -	dev->tx_pkt_burst = NULL;
> -	dev->rx_pkt_burst = NULL;
> -}
> -
>   static void
>   sfc_register_dp(void)
>   {
> @@ -2245,11 +2250,6 @@ sfc_eth_dev_init(struct rte_eth_dev *dev)
>   static int
>   sfc_eth_dev_uninit(struct rte_eth_dev *dev)
>   {
> -	if (rte_eal_process_type() != RTE_PROC_PRIMARY) {
> -		sfc_eth_dev_secondary_clear_ops(dev);
> -		return 0;
> -	}
> -
>   	sfc_dev_close(dev);
>   
>   	return 0;
> diff --git a/drivers/net/szedata2/rte_eth_szedata2.c b/drivers/net/szedata2/rte_eth_szedata2.c
> index 5f589dfa4c..7e9fafdcf7 100644
> --- a/drivers/net/szedata2/rte_eth_szedata2.c
> +++ b/drivers/net/szedata2/rte_eth_szedata2.c
> @@ -1163,6 +1163,9 @@ eth_dev_close(struct rte_eth_dev *dev)
>   	uint16_t nb_rx = dev->data->nb_rx_queues;
>   	uint16_t nb_tx = dev->data->nb_tx_queues;
>   
> +	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
> +		return 0;
> +
>   	eth_dev_stop(dev);
>   
>   	free(internals->sze_dev_path);
> diff --git a/drivers/net/thunderx/nicvf_ethdev.c b/drivers/net/thunderx/nicvf_ethdev.c
> index 3d7348771a..cc6eb4ba24 100644
> --- a/drivers/net/thunderx/nicvf_ethdev.c
> +++ b/drivers/net/thunderx/nicvf_ethdev.c
> @@ -1859,6 +1859,8 @@ nicvf_dev_close(struct rte_eth_dev *dev)
>   	struct nicvf *nic = nicvf_pmd_priv(dev);
>   
>   	PMD_INIT_FUNC_TRACE();
> +	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
> +		return 0;
>   
>   	nicvf_dev_stop_cleanup(dev, true);
>   	nicvf_periodic_alarm_stop(nicvf_interrupt, dev);
> @@ -2119,10 +2121,7 @@ static int
>   nicvf_eth_dev_uninit(struct rte_eth_dev *dev)
>   {
>   	PMD_INIT_FUNC_TRACE();
> -
> -	if (rte_eal_process_type() == RTE_PROC_PRIMARY)
> -		nicvf_dev_close(dev);
> -
> +	nicvf_dev_close(dev);
>   	return 0;
>   }
>   static int
> diff --git a/drivers/net/vhost/rte_eth_vhost.c b/drivers/net/vhost/rte_eth_vhost.c
> index 45552ef742..32ad27fe4e 100644
> --- a/drivers/net/vhost/rte_eth_vhost.c
> +++ b/drivers/net/vhost/rte_eth_vhost.c
> @@ -1171,6 +1171,9 @@ eth_dev_close(struct rte_eth_dev *dev)
>   	struct internal_list *list;
>   	unsigned int i;
>   
> +	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
> +		return 0;
> +
>   	internal = dev->data->dev_private;
>   	if (!internal)
>   		return 0;
> @@ -1655,11 +1658,7 @@ rte_pmd_vhost_remove(struct rte_vdev_device *dev)
>   	if (eth_dev == NULL)
>   		return 0;
>   
> -	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
> -		return rte_eth_dev_release_port(eth_dev);
> -
>   	eth_dev_close(eth_dev);
> -
>   	rte_eth_dev_release_port(eth_dev);
>   
>   	return 0;
> diff --git a/drivers/net/virtio/virtio_ethdev.c b/drivers/net/virtio/virtio_ethdev.c
> index b6ed5829bd..f2117675b3 100644
> --- a/drivers/net/virtio/virtio_ethdev.c
> +++ b/drivers/net/virtio/virtio_ethdev.c
> @@ -711,6 +711,8 @@ virtio_dev_close(struct rte_eth_dev *dev)
>   	struct rte_intr_conf *intr_conf = &dev->data->dev_conf.intr_conf;
>   
>   	PMD_INIT_LOG(DEBUG, "virtio_dev_close");
> +	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
> +		return 0;
>   
>   	if (!hw->opened)
>   		return 0;
> diff --git a/drivers/net/vmxnet3/vmxnet3_ethdev.c b/drivers/net/vmxnet3/vmxnet3_ethdev.c
> index 34a169d2c0..fa950e1ba0 100644
> --- a/drivers/net/vmxnet3/vmxnet3_ethdev.c
> +++ b/drivers/net/vmxnet3/vmxnet3_ethdev.c
> @@ -889,6 +889,8 @@ static int
>   vmxnet3_dev_close(struct rte_eth_dev *dev)
>   {
>   	PMD_INIT_FUNC_TRACE();
> +	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
> +		return 0;
>   
>   	vmxnet3_dev_stop(dev);
>   	vmxnet3_free_queues(dev);

Re: [dpdk-dev] [PATCH v2 19/25] drivers/net: check process type in close operation

[Ajit Khaparde]

On Mon, Sep 28, 2020 at 2:56 AM Sachin Saxena (OSS)
<sachin.saxena@oss.nxp.com> wrote:
>
> For dpaa, dpaa2, enetc and pfe
>
> Reviewed-by: Sachin Saxena <sachin.saxena@oss.nxp.com>
>
> On 28-Sep-20 5:12 AM, Thomas Monjalon wrote:
>
> The secondary processes are not allowed to release shared resources.
> Only process-private ressources should be freed in a secondary process.
> Most of the time, there is no process-private ressource,
> so the close operation is just forbidden in a secondary process.
>
> After adding proper check in the port close functions,
> some redundant checks in the device remove functions are dropped.
>
> Signed-off-by: Thomas Monjalon <thomas@monjalon.net>
> ---
>  drivers/net/af_xdp/rte_eth_af_xdp.c     |  3 +++
>  drivers/net/ark/ark_ethdev.c            |  3 +++
>  drivers/net/avp/avp_ethdev.c            |  3 +++
>  drivers/net/bnxt/bnxt_ethdev.c          |  3 +++
>  drivers/net/bnxt/bnxt_reps.c            |  3 +++
For bnxt,
Reviewed-by: Ajit Khaparde <ajit.khaparde@broadcom.com>

Re: [dpdk-dev] [EXT] [PATCH v2 19/25] drivers/net: check process type in close operation

[Liron Himi]

For mvpp2, mvneta.

Reviewed-by: Liron Himi<lironh@marvell.com>

-----Original Message-----
From: Thomas Monjalon <thomas@monjalon.net> 
Sent: Monday, 28 September 2020 02:43
To: dev@dpdk.org
Cc: ferruh.yigit@intel.com; arybchenko@solarflare.com; Ciara Loftus <ciara.loftus@intel.com>; Qi Zhang <qi.z.zhang@intel.com>; Shepard Siegel <shepard.siegel@atomicrules.com>; Ed Czeck <ed.czeck@atomicrules.com>; John Miller <john.miller@atomicrules.com>; Steven Webster <steven.webster@windriver.com>; Matt Peters <matt.peters@windriver.com>; Ajit Khaparde <ajit.khaparde@broadcom.com>; Somnath Kotur <somnath.kotur@broadcom.com>; Rahul Lakkireddy <rahul.lakkireddy@chelsio.com>; Hemant Agrawal <hemant.agrawal@nxp.com>; Sachin Saxena <sachin.saxena@oss.nxp.com>; Jeff Guo <jia.guo@intel.com>; Haiyue Wang <haiyue.wang@intel.com>; Marcin Wojtas <mw@semihalf.com>; Michal Krawczyk <mk@semihalf.com>; Guy Tzalik <gtzalik@amazon.com>; Evgeny Schemeilin <evgenys@amazon.com>; Igor Chauskin <igorch@amazon.com>; Gagandeep Singh <g.singh@nxp.com>; John Daley <johndale@cisco.com>; Hyong Youb Kim <hyonkim@cisco.com>; Xiao Wang <xiao.w.wang@intel.com>; Ziyang Xuan <xuanziyang2@huawei.com>; Xiaoyun Wang <cloud.wangxiaoyun@huawei.com>; Guoyang Zhou <zhouguoyang@huawei.com>; Beilei Xing <beilei.xing@intel.com>; Jingjing Wu <jingjing.wu@intel.com>; Qiming Yang <qiming.yang@intel.com>; Alfredo Cardigliano <cardigliano@ntop.org>; Rosen Xu <rosen.xu@intel.com>; Shijith Thotton <sthotton@marvell.com>; Srisivasubramanian Srinivasan <srinivasan@marvell.com>; Matan Azrad <matan@nvidia.com>; Shahaf Shuler <shahafs@nvidia.com>; Zyta Szpak <zr@semihalf.com>; Liron Himi <lironh@marvell.com>; Stephen Hemminger <sthemmin@microsoft.com>; K. Y. Srinivasan <kys@microsoft.com>; Haiyang Zhang <haiyangz@microsoft.com>; Long Li <longli@microsoft.com>; Martin Spinler <spinler@cesnet.cz>; Heinrich Kuhn <heinrich.kuhn@netronome.com>; Harman Kalra <hkalra@marvell.com>; Akhil Goyal <akhil.goyal@nxp.com>; Jerin Jacob Kollanukkaran <jerinj@marvell.com>; Maciej Czekaj [C] <mczekaj@marvell.com>; Maxime Coquelin <maxime.coquelin@redhat.com>; Chenbo Xia <chenbo.xia@intel.com>; Zhihong Wang <zhihong.wang@intel.com>; Yong Wang <yongwang@vmware.com>; Anatoly Burakov <anatoly.burakov@intel.com>
Subject: [EXT] [PATCH v2 19/25] drivers/net: check process type in close operation

External Email

----------------------------------------------------------------------
The secondary processes are not allowed to release shared resources.
Only process-private ressources should be freed in a secondary process.
Most of the time, there is no process-private ressource, so the close operation is just forbidden in a secondary process.

After adding proper check in the port close functions, some redundant checks in the device remove functions are dropped.

Signed-off-by: Thomas Monjalon <thomas@monjalon.net>
---
 drivers/net/af_xdp/rte_eth_af_xdp.c     |  3 +++
 drivers/net/ark/ark_ethdev.c            |  3 +++
 drivers/net/avp/avp_ethdev.c            |  3 +++
 drivers/net/bnxt/bnxt_ethdev.c          |  3 +++
 drivers/net/bnxt/bnxt_reps.c            |  3 +++
 drivers/net/cxgbe/cxgbe_ethdev.c        |  3 +++
 drivers/net/dpaa/dpaa_ethdev.c          |  3 +++
 drivers/net/dpaa2/dpaa2_ethdev.c        |  3 +++
 drivers/net/e1000/em_ethdev.c           |  3 +++
 drivers/net/e1000/igb_ethdev.c          |  6 +++++
 drivers/net/ena/ena_ethdev.c            |  3 +++
 drivers/net/enetc/enetc_ethdev.c        |  3 +++
 drivers/net/enic/enic_ethdev.c          |  3 +++
 drivers/net/fm10k/fm10k_ethdev.c        |  9 ++-----
 drivers/net/hinic/hinic_pmd_ethdev.c    |  3 +++
 drivers/net/i40e/i40e_ethdev.c          |  2 ++
 drivers/net/i40e/i40e_ethdev_vf.c       |  3 +++
 drivers/net/iavf/iavf_ethdev.c          |  3 +++
 drivers/net/ice/ice_ethdev.c            |  3 +++
 drivers/net/igc/igc_ethdev.c            |  6 ++---
 drivers/net/ionic/ionic_ethdev.c        |  2 ++
 drivers/net/ipn3ke/ipn3ke_representor.c |  3 +++
 drivers/net/ixgbe/ixgbe_ethdev.c        |  4 ++++
 drivers/net/kni/rte_eth_kni.c           |  3 +++
 drivers/net/liquidio/lio_ethdev.c       |  3 +++
 drivers/net/mlx4/mlx4.c                 |  2 ++
 drivers/net/mvneta/mvneta_ethdev.c      |  3 +++
 drivers/net/mvpp2/mrvl_ethdev.c         |  3 +++
 drivers/net/netvsc/hn_ethdev.c          |  2 ++
 drivers/net/nfb/nfb_ethdev.c            |  3 +++
 drivers/net/nfp/nfp_net.c               |  3 +++
 drivers/net/octeontx/octeontx_ethdev.c  |  2 ++
 drivers/net/pfe/pfe_ethdev.c            |  3 +++
 drivers/net/sfc/sfc_ethdev.c            | 32 ++++++++++++-------------
 drivers/net/szedata2/rte_eth_szedata2.c |  3 +++
 drivers/net/thunderx/nicvf_ethdev.c     |  7 +++---
 drivers/net/vhost/rte_eth_vhost.c       |  7 +++---
 drivers/net/virtio/virtio_ethdev.c      |  2 ++
 drivers/net/vmxnet3/vmxnet3_ethdev.c    |  2 ++
 39 files changed, 125 insertions(+), 35 deletions(-)

diff --git a/drivers/net/af_xdp/rte_eth_af_xdp.c b/drivers/net/af_xdp/rte_eth_af_xdp.c
index 60add9ead4..b289076e01 100644
--- a/drivers/net/af_xdp/rte_eth_af_xdp.c
+++ b/drivers/net/af_xdp/rte_eth_af_xdp.c
@@ -708,6 +708,9 @@ eth_dev_close(struct rte_eth_dev *dev)
 	struct pkt_rx_queue *rxq;
 	int i;
 
+	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
+		return 0;
+
 	AF_XDP_LOG(INFO, "Closing AF_XDP ethdev on numa socket %u\n",
 		rte_socket_id());
 
diff --git a/drivers/net/ark/ark_ethdev.c b/drivers/net/ark/ark_ethdev.c index 83dc4ecd2c..3e96445fdb 100644
--- a/drivers/net/ark/ark_ethdev.c
+++ b/drivers/net/ark/ark_ethdev.c
@@ -680,6 +680,9 @@ eth_ark_dev_close(struct rte_eth_dev *dev)
 	struct ark_adapter *ark = dev->data->dev_private;
 	uint16_t i;
 
+	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
+		return 0;
+
 	if (ark->user_ext.dev_close)
 		ark->user_ext.dev_close(dev,
 		 ark->user_data[dev->data->port_id]);
diff --git a/drivers/net/avp/avp_ethdev.c b/drivers/net/avp/avp_ethdev.c index c730b7ab86..95fdb57451 100644
--- a/drivers/net/avp/avp_ethdev.c
+++ b/drivers/net/avp/avp_ethdev.c
@@ -2107,6 +2107,9 @@ avp_dev_close(struct rte_eth_dev *eth_dev)
 	struct avp_dev *avp = AVP_DEV_PRIVATE_TO_HW(eth_dev->data->dev_private);
 	int ret;
 
+	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
+		return 0;
+
 	rte_spinlock_lock(&avp->lock);
 	if (avp->flags & AVP_F_DETACHED) {
 		PMD_DRV_LOG(ERR, "Operation not supported during VM live migration\n"); diff --git a/drivers/net/bnxt/bnxt_ethdev.c b/drivers/net/bnxt/bnxt_ethdev.c index a77bab661d..d45347e8ca 100644
--- a/drivers/net/bnxt/bnxt_ethdev.c
+++ b/drivers/net/bnxt/bnxt_ethdev.c
@@ -1361,6 +1361,9 @@ static int bnxt_dev_close_op(struct rte_eth_dev *eth_dev)  {
 	struct bnxt *bp = eth_dev->data->dev_private;
 
+	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
+		return 0;
+
 	/* cancel the recovery handler before remove dev */
 	rte_eal_alarm_cancel(bnxt_dev_reset_and_resume, (void *)bp);
 	rte_eal_alarm_cancel(bnxt_dev_recover, (void *)bp); diff --git a/drivers/net/bnxt/bnxt_reps.c b/drivers/net/bnxt/bnxt_reps.c index df8680c113..c419fe63b8 100644
--- a/drivers/net/bnxt/bnxt_reps.c
+++ b/drivers/net/bnxt/bnxt_reps.c
@@ -220,6 +220,9 @@ int bnxt_vf_representor_uninit(struct rte_eth_dev *eth_dev)
 		(struct bnxt_vf_representor *)eth_dev->data->dev_private;
 	uint16_t vf_id;
 
+	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
+		return 0;
+
 	PMD_DRV_LOG(DEBUG, "BNXT Port:%d VFR uninit\n", eth_dev->data->port_id);
 	eth_dev->data->mac_addrs = NULL;
 	eth_dev->dev_ops = NULL;
diff --git a/drivers/net/cxgbe/cxgbe_ethdev.c b/drivers/net/cxgbe/cxgbe_ethdev.c
index e4bbba5c32..16beb2d435 100644
--- a/drivers/net/cxgbe/cxgbe_ethdev.c
+++ b/drivers/net/cxgbe/cxgbe_ethdev.c
@@ -326,6 +326,9 @@ int cxgbe_dev_close(struct rte_eth_dev *eth_dev)
 
 	CXGBE_FUNC_TRACE();
 
+	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
+		return 0;
+
 	if (!(adapter->flags & FULL_INIT_DONE))
 		return 0;
 
diff --git a/drivers/net/dpaa/dpaa_ethdev.c b/drivers/net/dpaa/dpaa_ethdev.c index 7fe06e1830..01632e591c 100644
--- a/drivers/net/dpaa/dpaa_ethdev.c
+++ b/drivers/net/dpaa/dpaa_ethdev.c
@@ -379,6 +379,9 @@ static int dpaa_eth_dev_close(struct rte_eth_dev *dev)
 
 	PMD_INIT_FUNC_TRACE();
 
+	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
+		return 0;
+
 	dpaa_dev = container_of(rdev, struct rte_dpaa_device, device);
 	intr_handle = &dpaa_dev->intr_handle;
 	__fif = container_of(fif, struct __fman_if, __if); diff --git a/drivers/net/dpaa2/dpaa2_ethdev.c b/drivers/net/dpaa2/dpaa2_ethdev.c
index fb4165fa03..eb84d7231d 100644
--- a/drivers/net/dpaa2/dpaa2_ethdev.c
+++ b/drivers/net/dpaa2/dpaa2_ethdev.c
@@ -1246,6 +1246,9 @@ dpaa2_dev_close(struct rte_eth_dev *dev)
 
 	PMD_INIT_FUNC_TRACE();
 
+	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
+		return 0;
+
 	dpaa2_flow_clean(dev);
 
 	/* Clean the device first */
diff --git a/drivers/net/e1000/em_ethdev.c b/drivers/net/e1000/em_ethdev.c index 6a6ae0e9d5..d050eb478a 100644
--- a/drivers/net/e1000/em_ethdev.c
+++ b/drivers/net/e1000/em_ethdev.c
@@ -762,6 +762,9 @@ eth_em_close(struct rte_eth_dev *dev)
 	struct rte_pci_device *pci_dev = RTE_ETH_DEV_TO_PCI(dev);
 	struct rte_intr_handle *intr_handle = &pci_dev->intr_handle;
 
+	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
+		return 0;
+
 	eth_em_stop(dev);
 	adapter->stopped = 1;
 	em_dev_free_queues(dev);
diff --git a/drivers/net/e1000/igb_ethdev.c b/drivers/net/e1000/igb_ethdev.c index f8778207ef..cb3d97e2a3 100644
--- a/drivers/net/e1000/igb_ethdev.c
+++ b/drivers/net/e1000/igb_ethdev.c
@@ -1535,6 +1535,9 @@ eth_igb_close(struct rte_eth_dev *dev)
 	struct e1000_filter_info *filter_info =
 		E1000_DEV_PRIVATE_TO_FILTER_INFO(dev->data->dev_private);
 
+	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
+		return 0;
+
 	eth_igb_stop(dev);
 
 	e1000_phy_hw_reset(hw);
@@ -3382,6 +3385,9 @@ igbvf_dev_close(struct rte_eth_dev *dev)
 
 	PMD_INIT_FUNC_TRACE();
 
+	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
+		return 0;
+
 	e1000_reset_hw(hw);
 
 	igbvf_dev_stop(dev);
diff --git a/drivers/net/ena/ena_ethdev.c b/drivers/net/ena/ena_ethdev.c index cf1c0b9795..07ae9bab32 100644
--- a/drivers/net/ena/ena_ethdev.c
+++ b/drivers/net/ena/ena_ethdev.c
@@ -493,6 +493,9 @@ static int ena_close(struct rte_eth_dev *dev)
 	struct rte_intr_handle *intr_handle = &pci_dev->intr_handle;
 	struct ena_adapter *adapter = dev->data->dev_private;
 
+	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
+		return 0;
+
 	if (adapter->state == ENA_ADAPTER_STATE_RUNNING)
 		ena_stop(dev);
 	adapter->state = ENA_ADAPTER_STATE_CLOSED; diff --git a/drivers/net/enetc/enetc_ethdev.c b/drivers/net/enetc/enetc_ethdev.c
index bdb32762ce..d42add3efb 100644
--- a/drivers/net/enetc/enetc_ethdev.c
+++ b/drivers/net/enetc/enetc_ethdev.c
@@ -551,6 +551,9 @@ enetc_dev_close(struct rte_eth_dev *dev)
 	uint16_t i;
 
 	PMD_INIT_FUNC_TRACE();
+	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
+		return 0;
+
 	enetc_dev_stop(dev);
 
 	for (i = 0; i < dev->data->nb_rx_queues; i++) { diff --git a/drivers/net/enic/enic_ethdev.c b/drivers/net/enic/enic_ethdev.c index 27f60b45b8..60ee5e01de 100644
--- a/drivers/net/enic/enic_ethdev.c
+++ b/drivers/net/enic/enic_ethdev.c
@@ -451,6 +451,9 @@ static int enicpmd_dev_close(struct rte_eth_dev *eth_dev)
 	struct enic *enic = pmd_priv(eth_dev);
 
 	ENICPMD_FUNC_TRACE();
+	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
+		return 0;
+
 	enic_remove(enic);
 
 	return 0;
diff --git a/drivers/net/fm10k/fm10k_ethdev.c b/drivers/net/fm10k/fm10k_ethdev.c
index 23f4d04068..17cf0328c6 100644
--- a/drivers/net/fm10k/fm10k_ethdev.c
+++ b/drivers/net/fm10k/fm10k_ethdev.c
@@ -2784,6 +2784,8 @@ fm10k_dev_close(struct rte_eth_dev *dev)
 	struct rte_intr_handle *intr_handle = &pdev->intr_handle;
 
 	PMD_INIT_FUNC_TRACE();
+	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
+		return 0;
 
 	fm10k_mbx_lock(hw);
 	hw->mac.ops.update_lport_state(hw, hw->mac.dglort_map, @@ -3236,14 +3238,7 @@ static int  eth_fm10k_dev_uninit(struct rte_eth_dev *dev)  {
 	PMD_INIT_FUNC_TRACE();
-
-	/* only uninitialize in the primary process */
-	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
-		return 0;
-
-	/* safe to close dev here */
 	fm10k_dev_close(dev);
-
 	return 0;
 }
 
diff --git a/drivers/net/hinic/hinic_pmd_ethdev.c b/drivers/net/hinic/hinic_pmd_ethdev.c
index 623534fda4..1b175aeace 100644
--- a/drivers/net/hinic/hinic_pmd_ethdev.c
+++ b/drivers/net/hinic/hinic_pmd_ethdev.c
@@ -2940,6 +2940,9 @@ static int hinic_dev_close(struct rte_eth_dev *dev)  {
 	struct hinic_nic_dev *nic_dev = HINIC_ETH_DEV_TO_PRIVATE_NIC_DEV(dev);
 
+	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
+		return 0;
+
 	if (rte_bit_relaxed_test_and_set32(HINIC_DEV_CLOSE,
 					   &nic_dev->dev_status)) {
 		PMD_DRV_LOG(WARNING, "Device %s already closed", diff --git a/drivers/net/i40e/i40e_ethdev.c b/drivers/net/i40e/i40e_ethdev.c index 6fb88148b8..feeed2dec0 100644
--- a/drivers/net/i40e/i40e_ethdev.c
+++ b/drivers/net/i40e/i40e_ethdev.c
@@ -2620,6 +2620,8 @@ i40e_dev_close(struct rte_eth_dev *dev)
 	int retries = 0;
 
 	PMD_INIT_FUNC_TRACE();
+	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
+		return 0;
 
 	ret = rte_eth_switch_domain_free(pf->switch_domain_id);
 	if (ret)
diff --git a/drivers/net/i40e/i40e_ethdev_vf.c b/drivers/net/i40e/i40e_ethdev_vf.c
index 4aaf41956c..4d6510d1ff 100644
--- a/drivers/net/i40e/i40e_ethdev_vf.c
+++ b/drivers/net/i40e/i40e_ethdev_vf.c
@@ -2402,6 +2402,9 @@ i40evf_dev_close(struct rte_eth_dev *dev)
 	struct i40e_hw *hw = I40E_DEV_PRIVATE_TO_HW(dev->data->dev_private);
 	struct i40e_vf *vf = I40EVF_DEV_PRIVATE_TO_VF(dev->data->dev_private);
 
+	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
+		return 0;
+
 	i40evf_dev_stop(dev);
 	i40e_dev_free_queues(dev);
 	/*
diff --git a/drivers/net/iavf/iavf_ethdev.c b/drivers/net/iavf/iavf_ethdev.c index 512ade2ad0..a5b1433306 100644
--- a/drivers/net/iavf/iavf_ethdev.c
+++ b/drivers/net/iavf/iavf_ethdev.c
@@ -1468,6 +1468,9 @@ iavf_dev_close(struct rte_eth_dev *dev)
 		IAVF_DEV_PRIVATE_TO_ADAPTER(dev->data->dev_private);
 	struct iavf_info *vf = IAVF_DEV_PRIVATE_TO_VF(dev->data->dev_private);
 
+	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
+		return 0;
+
 	iavf_dev_stop(dev);
 	iavf_flow_flush(dev, NULL);
 	iavf_flow_uninit(adapter);
diff --git a/drivers/net/ice/ice_ethdev.c b/drivers/net/ice/ice_ethdev.c index 85a3ca6a2d..96159c92f2 100644
--- a/drivers/net/ice/ice_ethdev.c
+++ b/drivers/net/ice/ice_ethdev.c
@@ -2435,6 +2435,9 @@ ice_dev_close(struct rte_eth_dev *dev)
 	struct ice_adapter *ad =
 		ICE_DEV_PRIVATE_TO_ADAPTER(dev->data->dev_private);
 
+	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
+		return 0;
+
 	/* Since stop will make link down, then the link event will be
 	 * triggered, disable the irq firstly to avoid the port_infoe etc
 	 * resources deallocation causing the interrupt service thread diff --git a/drivers/net/igc/igc_ethdev.c b/drivers/net/igc/igc_ethdev.c index 9d27fc0d07..7f5066df4b 100644
--- a/drivers/net/igc/igc_ethdev.c
+++ b/drivers/net/igc/igc_ethdev.c
@@ -1175,6 +1175,8 @@ eth_igc_close(struct rte_eth_dev *dev)
 	int retry = 0;
 
 	PMD_INIT_FUNC_TRACE();
+	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
+		return 0;
 
 	if (!adapter->stopped)
 		eth_igc_stop(dev);
@@ -1363,10 +1365,6 @@ static int
 eth_igc_dev_uninit(__rte_unused struct rte_eth_dev *eth_dev)  {
 	PMD_INIT_FUNC_TRACE();
-
-	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
-		return 0;
-
 	eth_igc_close(eth_dev);
 	return 0;
 }
diff --git a/drivers/net/ionic/ionic_ethdev.c b/drivers/net/ionic/ionic_ethdev.c
index 1775fd29b7..ef7d06e526 100644
--- a/drivers/net/ionic/ionic_ethdev.c
+++ b/drivers/net/ionic/ionic_ethdev.c
@@ -963,6 +963,8 @@ ionic_dev_close(struct rte_eth_dev *eth_dev)
 	int err;
 
 	IONIC_PRINT_CALL();
+	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
+		return 0;
 
 	err = ionic_lif_stop(lif);
 	if (err) {
diff --git a/drivers/net/ipn3ke/ipn3ke_representor.c b/drivers/net/ipn3ke/ipn3ke_representor.c
index d49abbf758..b9fb4d4e46 100644
--- a/drivers/net/ipn3ke/ipn3ke_representor.c
+++ b/drivers/net/ipn3ke/ipn3ke_representor.c
@@ -214,6 +214,9 @@ ipn3ke_rpst_dev_close(struct rte_eth_dev *dev)
 	struct ipn3ke_hw *hw = IPN3KE_DEV_PRIVATE_TO_HW(dev);
 	struct ipn3ke_rpst *rpst = IPN3KE_DEV_PRIVATE_TO_RPST(dev);
 
+	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
+		return 0;
+
 	if (hw->retimer.mac_type == IFPGA_RAWDEV_RETIMER_MAC_TYPE_10GE_XFI) {
 		/* Disable the TX path */
 		ipn3ke_xmac_tx_disable(hw, rpst->port_id, 0); diff --git a/drivers/net/ixgbe/ixgbe_ethdev.c b/drivers/net/ixgbe/ixgbe_ethdev.c
index 426b7c9fe5..f6e383b1c2 100644
--- a/drivers/net/ixgbe/ixgbe_ethdev.c
+++ b/drivers/net/ixgbe/ixgbe_ethdev.c
@@ -2995,6 +2995,8 @@ ixgbe_dev_close(struct rte_eth_dev *dev)
 	int ret;
 
 	PMD_INIT_FUNC_TRACE();
+	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
+		return 0;
 
 	ixgbe_pf_reset_hw(hw);
 
@@ -5442,6 +5444,8 @@ ixgbevf_dev_close(struct rte_eth_dev *dev)
 	struct rte_intr_handle *intr_handle = &pci_dev->intr_handle;
 
 	PMD_INIT_FUNC_TRACE();
+	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
+		return 0;
 
 	ixgbe_reset_hw(hw);
 
diff --git a/drivers/net/kni/rte_eth_kni.c b/drivers/net/kni/rte_eth_kni.c index 2a4058f7b0..be747adf86 100644
--- a/drivers/net/kni/rte_eth_kni.c
+++ b/drivers/net/kni/rte_eth_kni.c
@@ -204,6 +204,9 @@ eth_kni_close(struct rte_eth_dev *eth_dev)
 	struct pmd_internals *internals;
 	int ret;
 
+	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
+		return 0;
+
 	eth_kni_dev_stop(eth_dev);
 
 	/* mac_addrs must not be freed alone because part of dev_private */ diff --git a/drivers/net/liquidio/lio_ethdev.c b/drivers/net/liquidio/lio_ethdev.c
index 93e2ed5670..1a41f27198 100644
--- a/drivers/net/liquidio/lio_ethdev.c
+++ b/drivers/net/liquidio/lio_ethdev.c
@@ -1555,6 +1555,9 @@ lio_dev_close(struct rte_eth_dev *eth_dev)  {
 	struct lio_device *lio_dev = LIO_DEV(eth_dev);
 
+	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
+		return 0;
+
 	lio_dev_info(lio_dev, "closing port %d\n", eth_dev->data->port_id);
 
 	if (lio_dev->intf_open)
diff --git a/drivers/net/mlx4/mlx4.c b/drivers/net/mlx4/mlx4.c index 3e57875414..cfcfb8a8fc 100644
--- a/drivers/net/mlx4/mlx4.c
+++ b/drivers/net/mlx4/mlx4.c
@@ -376,6 +376,8 @@ mlx4_dev_close(struct rte_eth_dev *dev)
 	struct mlx4_priv *priv = dev->data->dev_private;
 	unsigned int i;
 
+	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
+		return 0;
 	DEBUG("%p: closing device \"%s\"",
 	      (void *)dev,
 	      ((priv->ctx != NULL) ? priv->ctx->device->name : "")); diff --git a/drivers/net/mvneta/mvneta_ethdev.c b/drivers/net/mvneta/mvneta_ethdev.c
index db142bec23..607771149a 100644
--- a/drivers/net/mvneta/mvneta_ethdev.c
+++ b/drivers/net/mvneta/mvneta_ethdev.c
@@ -435,6 +435,9 @@ mvneta_dev_close(struct rte_eth_dev *dev)
 	struct mvneta_priv *priv = dev->data->dev_private;
 	int i;
 
+	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
+		return 0;
+
 	if (priv->ppio)
 		mvneta_dev_stop(dev);
 
diff --git a/drivers/net/mvpp2/mrvl_ethdev.c b/drivers/net/mvpp2/mrvl_ethdev.c index cfb97e4f8f..a230a96840 100644
--- a/drivers/net/mvpp2/mrvl_ethdev.c
+++ b/drivers/net/mvpp2/mrvl_ethdev.c
@@ -861,6 +861,9 @@ mrvl_dev_close(struct rte_eth_dev *dev)
 	struct mrvl_priv *priv = dev->data->dev_private;
 	size_t i;
 
+	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
+		return 0;
+
 	mrvl_flush_rx_queues(dev);
 	mrvl_flush_tx_shadow_queues(dev);
 	mrvl_flow_deinit(dev);
diff --git a/drivers/net/netvsc/hn_ethdev.c b/drivers/net/netvsc/hn_ethdev.c index 19a9eb6bc2..5ae2d469c8 100644
--- a/drivers/net/netvsc/hn_ethdev.c
+++ b/drivers/net/netvsc/hn_ethdev.c
@@ -842,6 +842,8 @@ static int
 hn_dev_close(struct rte_eth_dev *dev)
 {
 	PMD_INIT_FUNC_TRACE();
+	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
+		return 0;
 
 	hn_vf_close(dev);
 	hn_dev_free_queues(dev);
diff --git a/drivers/net/nfb/nfb_ethdev.c b/drivers/net/nfb/nfb_ethdev.c index d937ac6922..7ee7294143 100644
--- a/drivers/net/nfb/nfb_ethdev.c
+++ b/drivers/net/nfb/nfb_ethdev.c
@@ -217,6 +217,9 @@ nfb_eth_dev_close(struct rte_eth_dev *dev)
 	uint16_t nb_rx = dev->data->nb_rx_queues;
 	uint16_t nb_tx = dev->data->nb_tx_queues;
 
+	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
+		return 0;
+
 	nfb_eth_dev_stop(dev);
 
 	nfb_nc_rxmac_deinit(internals->rxmac, internals->max_rxmac); diff --git a/drivers/net/nfp/nfp_net.c b/drivers/net/nfp/nfp_net.c index ce25cf1ed4..c1da66e3d6 100644
--- a/drivers/net/nfp/nfp_net.c
+++ b/drivers/net/nfp/nfp_net.c
@@ -871,6 +871,9 @@ nfp_net_close(struct rte_eth_dev *dev)
 	struct rte_pci_device *pci_dev;
 	int i;
 
+	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
+		return 0;
+
 	PMD_INIT_LOG(DEBUG, "Close");
 
 	hw = NFP_NET_DEV_PRIVATE_TO_HW(dev->data->dev_private);
diff --git a/drivers/net/octeontx/octeontx_ethdev.c b/drivers/net/octeontx/octeontx_ethdev.c
index a263f45399..aa9ef3bb70 100644
--- a/drivers/net/octeontx/octeontx_ethdev.c
+++ b/drivers/net/octeontx/octeontx_ethdev.c
@@ -487,6 +487,8 @@ octeontx_dev_close(struct rte_eth_dev *dev)
 	int ret;
 
 	PMD_INIT_FUNC_TRACE();
+	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
+		return 0;
 
 	rte_event_dev_close(nic->evdev);
 
diff --git a/drivers/net/pfe/pfe_ethdev.c b/drivers/net/pfe/pfe_ethdev.c index 9d5415d9b1..a7e9d97ba9 100644
--- a/drivers/net/pfe/pfe_ethdev.c
+++ b/drivers/net/pfe/pfe_ethdev.c
@@ -407,6 +407,9 @@ pfe_eth_close(struct rte_eth_dev *dev)
 	if (!g_pfe)
 		return -1;
 
+	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
+		return 0;
+
 	pfe_eth_exit(dev, g_pfe);
 
 	if (g_pfe->nb_devs == 0) {
diff --git a/drivers/net/sfc/sfc_ethdev.c b/drivers/net/sfc/sfc_ethdev.c index 1c72e40b75..0558f1808d 100644
--- a/drivers/net/sfc/sfc_ethdev.c
+++ b/drivers/net/sfc/sfc_ethdev.c
@@ -318,6 +318,17 @@ sfc_dev_set_link_down(struct rte_eth_dev *dev)
 	return 0;
 }
 
+static void
+sfc_eth_dev_secondary_clear_ops(struct rte_eth_dev *dev) {
+	free(dev->process_private);
+	dev->process_private = NULL;
+	dev->dev_ops = NULL;
+	dev->tx_pkt_prepare = NULL;
+	dev->tx_pkt_burst = NULL;
+	dev->rx_pkt_burst = NULL;
+}
+
 static int
 sfc_dev_close(struct rte_eth_dev *dev)
 {
@@ -325,6 +336,11 @@ sfc_dev_close(struct rte_eth_dev *dev)
 
 	sfc_log_init(sa, "entry");
 
+	if (rte_eal_process_type() != RTE_PROC_PRIMARY) {
+		sfc_eth_dev_secondary_clear_ops(dev);
+		return 0;
+	}
+
 	sfc_adapter_lock(sa);
 	switch (sa->state) {
 	case SFC_ADAPTER_STARTED:
@@ -2101,17 +2117,6 @@ sfc_eth_dev_secondary_init(struct rte_eth_dev *dev, uint32_t logtype_main)
 	return rc;
 }
 
-static void
-sfc_eth_dev_secondary_clear_ops(struct rte_eth_dev *dev) -{
-	free(dev->process_private);
-	dev->process_private = NULL;
-	dev->dev_ops = NULL;
-	dev->tx_pkt_prepare = NULL;
-	dev->tx_pkt_burst = NULL;
-	dev->rx_pkt_burst = NULL;
-}
-
 static void
 sfc_register_dp(void)
 {
@@ -2245,11 +2250,6 @@ sfc_eth_dev_init(struct rte_eth_dev *dev)  static int  sfc_eth_dev_uninit(struct rte_eth_dev *dev)  {
-	if (rte_eal_process_type() != RTE_PROC_PRIMARY) {
-		sfc_eth_dev_secondary_clear_ops(dev);
-		return 0;
-	}
-
 	sfc_dev_close(dev);
 
 	return 0;
diff --git a/drivers/net/szedata2/rte_eth_szedata2.c b/drivers/net/szedata2/rte_eth_szedata2.c
index 5f589dfa4c..7e9fafdcf7 100644
--- a/drivers/net/szedata2/rte_eth_szedata2.c
+++ b/drivers/net/szedata2/rte_eth_szedata2.c
@@ -1163,6 +1163,9 @@ eth_dev_close(struct rte_eth_dev *dev)
 	uint16_t nb_rx = dev->data->nb_rx_queues;
 	uint16_t nb_tx = dev->data->nb_tx_queues;
 
+	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
+		return 0;
+
 	eth_dev_stop(dev);
 
 	free(internals->sze_dev_path);
diff --git a/drivers/net/thunderx/nicvf_ethdev.c b/drivers/net/thunderx/nicvf_ethdev.c
index 3d7348771a..cc6eb4ba24 100644
--- a/drivers/net/thunderx/nicvf_ethdev.c
+++ b/drivers/net/thunderx/nicvf_ethdev.c
@@ -1859,6 +1859,8 @@ nicvf_dev_close(struct rte_eth_dev *dev)
 	struct nicvf *nic = nicvf_pmd_priv(dev);
 
 	PMD_INIT_FUNC_TRACE();
+	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
+		return 0;
 
 	nicvf_dev_stop_cleanup(dev, true);
 	nicvf_periodic_alarm_stop(nicvf_interrupt, dev); @@ -2119,10 +2121,7 @@ static int  nicvf_eth_dev_uninit(struct rte_eth_dev *dev)  {
 	PMD_INIT_FUNC_TRACE();
-
-	if (rte_eal_process_type() == RTE_PROC_PRIMARY)
-		nicvf_dev_close(dev);
-
+	nicvf_dev_close(dev);
 	return 0;
 }
 static int
diff --git a/drivers/net/vhost/rte_eth_vhost.c b/drivers/net/vhost/rte_eth_vhost.c
index 45552ef742..32ad27fe4e 100644
--- a/drivers/net/vhost/rte_eth_vhost.c
+++ b/drivers/net/vhost/rte_eth_vhost.c
@@ -1171,6 +1171,9 @@ eth_dev_close(struct rte_eth_dev *dev)
 	struct internal_list *list;
 	unsigned int i;
 
+	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
+		return 0;
+
 	internal = dev->data->dev_private;
 	if (!internal)
 		return 0;
@@ -1655,11 +1658,7 @@ rte_pmd_vhost_remove(struct rte_vdev_device *dev)
 	if (eth_dev == NULL)
 		return 0;
 
-	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
-		return rte_eth_dev_release_port(eth_dev);
-
 	eth_dev_close(eth_dev);
-
 	rte_eth_dev_release_port(eth_dev);
 
 	return 0;
diff --git a/drivers/net/virtio/virtio_ethdev.c b/drivers/net/virtio/virtio_ethdev.c
index b6ed5829bd..f2117675b3 100644
--- a/drivers/net/virtio/virtio_ethdev.c
+++ b/drivers/net/virtio/virtio_ethdev.c
@@ -711,6 +711,8 @@ virtio_dev_close(struct rte_eth_dev *dev)
 	struct rte_intr_conf *intr_conf = &dev->data->dev_conf.intr_conf;
 
 	PMD_INIT_LOG(DEBUG, "virtio_dev_close");
+	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
+		return 0;
 
 	if (!hw->opened)
 		return 0;
diff --git a/drivers/net/vmxnet3/vmxnet3_ethdev.c b/drivers/net/vmxnet3/vmxnet3_ethdev.c
index 34a169d2c0..fa950e1ba0 100644
--- a/drivers/net/vmxnet3/vmxnet3_ethdev.c
+++ b/drivers/net/vmxnet3/vmxnet3_ethdev.c
@@ -889,6 +889,8 @@ static int
 vmxnet3_dev_close(struct rte_eth_dev *dev)  {
 	PMD_INIT_FUNC_TRACE();
+	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
+		return 0;
 
 	vmxnet3_dev_stop(dev);
 	vmxnet3_free_queues(dev);
--
2.28.0

Re: [dpdk-dev] [PATCH v2 19/25] drivers/net: check process type in close operation

[Stephen Hemminger]

On Mon, 28 Sep 2020 01:42:43 +0200
Thomas Monjalon <thomas@monjalon.net> wrote:

> The secondary processes are not allowed to release shared resources.
> Only process-private ressources should be freed in a secondary process.
> Most of the time, there is no process-private ressource,
> so the close operation is just forbidden in a secondary process.
> 
> After adding proper check in the port close functions,
> some redundant checks in the device remove functions are dropped.
> 
> Signed-off-by: Thomas Monjalon <thomas@monjalon.net>

Since this is a common issue across many drivers, my preference would be that the
check should be in common code of eth_dev_close.  It looks like only a few drivers
would allow close from secondary, and that would not be that helpful to a user.

Re: [dpdk-dev] [PATCH v2 19/25] drivers/net: check process type in close operation

[Thomas Monjalon]

28/09/2020 20:51, Stephen Hemminger:
> On Mon, 28 Sep 2020 01:42:43 +0200
> Thomas Monjalon <thomas@monjalon.net> wrote:
> 
> > The secondary processes are not allowed to release shared resources.
> > Only process-private ressources should be freed in a secondary process.
> > Most of the time, there is no process-private ressource,
> > so the close operation is just forbidden in a secondary process.
> > 
> > After adding proper check in the port close functions,
> > some redundant checks in the device remove functions are dropped.
> > 
> > Signed-off-by: Thomas Monjalon <thomas@monjalon.net>
> 
> Since this is a common issue across many drivers, my preference would be that the
> check should be in common code of eth_dev_close.  It looks like only a few drivers
> would allow close from secondary, and that would not be that helpful to a user.

It needs to be managed in the PMD to free secondary process resources.

[dpdk-dev] [PATCH v2 20/25] drivers/net: remove redundant MAC addresses freeing

[Thomas Monjalon]

The MAC addresses array is already freed by rte_eth_dev_release_port().
The redundant freeing can be removed from the PMD port closing functions.

Signed-off-by: Thomas Monjalon <thomas@monjalon.net>
Reviewed-by: Ferruh Yigit <ferruh.yigit@intel.com>
---
 drivers/net/ark/ark_ethdev.c            | 3 ---
 drivers/net/hinic/hinic_pmd_ethdev.c    | 3 ---
 drivers/net/ice/ice_ethdev.c            | 3 ---
 drivers/net/nfb/nfb_ethdev.c            | 3 ---
 drivers/net/szedata2/rte_eth_szedata2.c | 3 ---
 5 files changed, 15 deletions(-)

diff --git a/drivers/net/ark/ark_ethdev.c b/drivers/net/ark/ark_ethdev.c
index 3e96445fdb..e3b1347769 100644
--- a/drivers/net/ark/ark_ethdev.c
+++ b/drivers/net/ark/ark_ethdev.c
@@ -705,9 +705,6 @@ eth_ark_dev_close(struct rte_eth_dev *dev)
 		dev->data->rx_queues[i] = 0;
 	}
 
-	rte_free(dev->data->mac_addrs);
-	dev->data->mac_addrs = 0;
-
 	return 0;
 }
 
diff --git a/drivers/net/hinic/hinic_pmd_ethdev.c b/drivers/net/hinic/hinic_pmd_ethdev.c
index 1b175aeace..c5b06fae73 100644
--- a/drivers/net/hinic/hinic_pmd_ethdev.c
+++ b/drivers/net/hinic/hinic_pmd_ethdev.c
@@ -3228,9 +3228,6 @@ static int hinic_dev_uninit(struct rte_eth_dev *dev)
 
 	rte_free(nic_dev->mc_list);
 
-	rte_free(dev->data->mac_addrs);
-	dev->data->mac_addrs = NULL;
-
 	return HINIC_OK;
 }
 
diff --git a/drivers/net/ice/ice_ethdev.c b/drivers/net/ice/ice_ethdev.c
index 96159c92f2..73af19ce81 100644
--- a/drivers/net/ice/ice_ethdev.c
+++ b/drivers/net/ice/ice_ethdev.c
@@ -2467,9 +2467,6 @@ ice_dev_close(struct rte_eth_dev *dev)
 	dev->rx_pkt_burst = NULL;
 	dev->tx_pkt_burst = NULL;
 
-	rte_free(dev->data->mac_addrs);
-	dev->data->mac_addrs = NULL;
-
 	/* disable uio intr before callback unregister */
 	rte_intr_disable(intr_handle);
 
diff --git a/drivers/net/nfb/nfb_ethdev.c b/drivers/net/nfb/nfb_ethdev.c
index 7ee7294143..a9a8bc878d 100644
--- a/drivers/net/nfb/nfb_ethdev.c
+++ b/drivers/net/nfb/nfb_ethdev.c
@@ -236,9 +236,6 @@ nfb_eth_dev_close(struct rte_eth_dev *dev)
 	}
 	dev->data->nb_tx_queues = 0;
 
-	rte_free(dev->data->mac_addrs);
-	dev->data->mac_addrs = NULL;
-
 	return 0;
 }
 
diff --git a/drivers/net/szedata2/rte_eth_szedata2.c b/drivers/net/szedata2/rte_eth_szedata2.c
index 7e9fafdcf7..0eecec1e8c 100644
--- a/drivers/net/szedata2/rte_eth_szedata2.c
+++ b/drivers/net/szedata2/rte_eth_szedata2.c
@@ -1181,9 +1181,6 @@ eth_dev_close(struct rte_eth_dev *dev)
 	}
 	dev->data->nb_tx_queues = 0;
 
-	rte_free(dev->data->mac_addrs);
-	dev->data->mac_addrs = NULL;
-
 	return 0;
 }
 
-- 
2.28.0

[dpdk-dev] [PATCH v2 21/25] app/testpmd: reset port status on close notification

[Thomas Monjalon]

Since rte_eth_dev_release_port() is called on all port close operations,
the event RTE_ETH_EVENT_DESTROY can be reliably used for resetting
the port status on the application side.

The intermediate state RTE_PORT_HANDLING is removed in close_port()
because a port can also be closed by a PMD in a device remove operation.

In case multiple ports are closed, calling remove_invalid_ports()
only once is enough.

Signed-off-by: Thomas Monjalon <thomas@monjalon.net>
---
 app/test-pmd/testpmd.c | 31 +++++--------------------------
 1 file changed, 5 insertions(+), 26 deletions(-)

diff --git a/app/test-pmd/testpmd.c b/app/test-pmd/testpmd.c
index fe6450cc0d..1e10d2e2e4 100644
--- a/app/test-pmd/testpmd.c
+++ b/app/test-pmd/testpmd.c
@@ -2695,23 +2695,12 @@ close_port(portid_t pid)
 			continue;
 		}
 
-		if (rte_atomic16_cmpset(&(port->port_status),
-			RTE_PORT_STOPPED, RTE_PORT_HANDLING) == 0) {
-			printf("Port %d is now not stopped\n", pi);
-			continue;
-		}
-
 		if (port->flow_list)
 			port_flow_flush(pi);
 		rte_eth_dev_close(pi);
-
-		remove_invalid_ports();
-
-		if (rte_atomic16_cmpset(&(port->port_status),
-			RTE_PORT_HANDLING, RTE_PORT_CLOSED) == 0)
-			printf("Port %d cannot be set to closed\n", pi);
 	}
 
+	remove_invalid_ports();
 	printf("Done\n");
 }
 
@@ -2842,12 +2831,7 @@ detach_device(struct rte_device *dev)
 		return;
 	}
 	RTE_ETH_FOREACH_DEV_OF(sibling, dev) {
-		/* reset mapping between old ports and removed device */
-		rte_eth_devices[sibling].device = NULL;
 		if (ports[sibling].port_status != RTE_PORT_CLOSED) {
-			/* sibling ports are forced to be closed */
-			ports[sibling].port_status = RTE_PORT_CLOSED;
-			printf("Port %u is closed\n", sibling);
 		}
 	}
 
@@ -2903,11 +2887,8 @@ detach_devargs(char *identifier)
 				return;
 			}
 
-			/* sibling ports are forced to be closed */
 			if (ports[port_id].flow_list)
 				port_flow_flush(port_id);
-			ports[port_id].port_status = RTE_PORT_CLOSED;
-			printf("Port %u is now closed\n", port_id);
 		}
 	}
 
@@ -3052,12 +3033,6 @@ check_all_ports_link_status(uint32_t port_mask)
 	}
 }
 
-/*
- * This callback is for remove a port for a device. It has limitation because
- * it is not for multiple port removal for a device.
- * TODO: the device detach invoke will plan to be removed from user side to
- * eal. And convert all PMDs to free port resources on ether device closing.
- */
 static void
 rmv_port_callback(void *arg)
 {
@@ -3115,6 +3090,10 @@ eth_event_callback(portid_t port_id, enum rte_eth_event_type type, void *param,
 				rmv_port_callback, (void *)(intptr_t)port_id))
 			fprintf(stderr, "Could not set up deferred device removal\n");
 		break;
+	case RTE_ETH_EVENT_DESTROY:
+		ports[port_id].port_status = RTE_PORT_CLOSED;
+		printf("Port %u is closed\n", port_id);
+		break;
 	default:
 		break;
 	}
-- 
2.28.0

[dpdk-dev] [PATCH v2 22/25] app/testpmd: align behaviour of multi-port detach

[Thomas Monjalon]

A port can be closed in multiple situations:
	- close command calling close_port() -> rte_eth_dev_close()
	- exit calling close_port() -> rte_eth_dev_close()
	- hotplug calling close_port() -> rte_eth_dev_close()
	- hotplug calling detach_device() -> rte_dev_remove()
	- port detach command, detach_device() -> rte_dev_remove()
	- device detach command, detach_devargs() -> rte_eal_hotplug_remove()

The flow rules are flushed before each close.
It was already done in close_port(), detach_devargs() and
detach_port_device() which calls detach_device(),
but not in detach_device(). As a consequence, it was missing for siblings
of port detach command and unplugged device.
The check before calling port_flow_flush() is moved inside the function.

The state of the port to close is checked to be stopped.
As above, this check was missing in detach_device(),
impacting the cases of a multi-port device unplugged or detached
with the port detach command.

Signed-off-by: Thomas Monjalon <thomas@monjalon.net>
---
 app/test-pmd/config.c  |  7 +++++--
 app/test-pmd/testpmd.c | 22 +++++++++++-----------
 2 files changed, 16 insertions(+), 13 deletions(-)

diff --git a/app/test-pmd/config.c b/app/test-pmd/config.c
index 2d9a456467..4691ae1f7b 100644
--- a/app/test-pmd/config.c
+++ b/app/test-pmd/config.c
@@ -1590,9 +1590,12 @@ int
 port_flow_flush(portid_t port_id)
 {
 	struct rte_flow_error error;
-	struct rte_port *port;
+	struct rte_port *port = &ports[port_id];
 	int ret = 0;
 
+	if (port->flow_list == NULL)
+		return ret;
+
 	/* Poisoning to make sure PMDs update it in case of error. */
 	memset(&error, 0x44, sizeof(error));
 	if (rte_flow_flush(port_id, &error)) {
@@ -1601,7 +1604,7 @@ port_flow_flush(portid_t port_id)
 		    port_id == (portid_t)RTE_PORT_ALL)
 			return ret;
 	}
-	port = &ports[port_id];
+
 	while (port->flow_list) {
 		struct port_flow *pf = port->flow_list->next;
 
diff --git a/app/test-pmd/testpmd.c b/app/test-pmd/testpmd.c
index 1e10d2e2e4..ccba71c076 100644
--- a/app/test-pmd/testpmd.c
+++ b/app/test-pmd/testpmd.c
@@ -2695,8 +2695,7 @@ close_port(portid_t pid)
 			continue;
 		}
 
-		if (port->flow_list)
-			port_flow_flush(pi);
+		port_flow_flush(pi);
 		rte_eth_dev_close(pi);
 	}
 
@@ -2826,15 +2825,20 @@ detach_device(struct rte_device *dev)
 
 	printf("Removing a device...\n");
 
-	if (rte_dev_remove(dev) < 0) {
-		TESTPMD_LOG(ERR, "Failed to detach device %s\n", dev->name);
-		return;
-	}
 	RTE_ETH_FOREACH_DEV_OF(sibling, dev) {
 		if (ports[sibling].port_status != RTE_PORT_CLOSED) {
+			if (ports[sibling].port_status != RTE_PORT_STOPPED) {
+				printf("Port %u not stopped\n", sibling);
+				return;
+			}
+			port_flow_flush(sibling);
 		}
 	}
 
+	if (rte_dev_remove(dev) < 0) {
+		TESTPMD_LOG(ERR, "Failed to detach device %s\n", dev->name);
+		return;
+	}
 	remove_invalid_ports();
 
 	printf("Device is detached\n");
@@ -2855,8 +2859,6 @@ detach_port_device(portid_t port_id)
 			return;
 		}
 		printf("Port was not closed\n");
-		if (ports[port_id].flow_list)
-			port_flow_flush(port_id);
 	}
 
 	detach_device(rte_eth_devices[port_id].device);
@@ -2886,9 +2888,7 @@ detach_devargs(char *identifier)
 				rte_eth_iterator_cleanup(&iterator);
 				return;
 			}
-
-			if (ports[port_id].flow_list)
-				port_flow_flush(port_id);
+			port_flow_flush(port_id);
 		}
 	}
 
-- 
2.28.0

[dpdk-dev] [PATCH v2 23/25] ethdev: remove forcing stopped state upon close

[Thomas Monjalon]

When closing a port, it is supposed to be already stopped,
and marked as such with "dev_started" state zeroed.

Resetting "dev_started" before calling the driver close operation
was hiding the case of not properly stopped port being closed.
The flag "dev_started" is not changed anymore in "rte_eth_dev_close()".

Signed-off-by: Thomas Monjalon <thomas@monjalon.net>
---
 lib/librte_ethdev/rte_ethdev.c | 1 -
 1 file changed, 1 deletion(-)

diff --git a/lib/librte_ethdev/rte_ethdev.c b/lib/librte_ethdev/rte_ethdev.c
index d7668114ca..0b8e8e3e8d 100644
--- a/lib/librte_ethdev/rte_ethdev.c
+++ b/lib/librte_ethdev/rte_ethdev.c
@@ -1716,7 +1716,6 @@ rte_eth_dev_close(uint16_t port_id)
 	dev = &rte_eth_devices[port_id];
 
 	RTE_FUNC_PTR_OR_RET(*dev->dev_ops->dev_close);
-	dev->data->dev_started = 0;
 	(*dev->dev_ops->dev_close)(dev);
 
 	rte_ethdev_trace_close(port_id);
-- 
2.28.0

[dpdk-dev] [PATCH v2 24/25] ethdev: reset all when releasing a port

[Thomas Monjalon]

The function rte_eth_dev_release_port() was resetting partially
the struct rte_eth_dev. The drivers were completing it
with more pointers set to NULL in the close or remove operations.

A full memset is done so most of those assignments become useless.

Signed-off-by: Thomas Monjalon <thomas@monjalon.net>
---
 drivers/net/ark/ark_ethdev.c           | 3 ---
 drivers/net/atlantic/atl_ethdev.c      | 4 ----
 drivers/net/axgbe/axgbe_ethdev.c       | 3 ---
 drivers/net/bnxt/bnxt_ethdev.c         | 4 ----
 drivers/net/bnxt/bnxt_reps.c           | 1 -
 drivers/net/bonding/rte_eth_bond_pmd.c | 4 ----
 drivers/net/dpaa/dpaa_ethdev.c         | 4 ----
 drivers/net/dpaa2/dpaa2_ethdev.c       | 4 ----
 drivers/net/e1000/em_ethdev.c          | 4 ----
 drivers/net/e1000/igb_ethdev.c         | 8 --------
 drivers/net/ena/ena_ethdev.c           | 5 -----
 drivers/net/fm10k/fm10k_ethdev.c       | 4 ----
 drivers/net/hinic/hinic_pmd_ethdev.c   | 4 ----
 drivers/net/hns3/hns3_ethdev.c         | 4 ----
 drivers/net/hns3/hns3_ethdev_vf.c      | 5 -----
 drivers/net/i40e/i40e_ethdev.c         | 4 ----
 drivers/net/i40e/i40e_ethdev_vf.c      | 4 ----
 drivers/net/iavf/iavf_ethdev.c         | 4 ----
 drivers/net/ice/ice_dcf_ethdev.c       | 4 ----
 drivers/net/ice/ice_ethdev.c           | 4 ----
 drivers/net/ionic/ionic_ethdev.c       | 5 -----
 drivers/net/ixgbe/ixgbe_ethdev.c       | 8 --------
 drivers/net/liquidio/lio_ethdev.c      | 4 ----
 drivers/net/netvsc/hn_ethdev.c         | 4 ----
 drivers/net/octeontx/octeontx_ethdev.c | 3 ---
 drivers/net/qede/qede_ethdev.c         | 4 ----
 drivers/net/sfc/sfc_ethdev.c           | 7 +------
 drivers/net/tap/rte_eth_tap.c          | 1 -
 drivers/net/virtio/virtio_ethdev.c     | 4 ----
 drivers/net/vmxnet3/vmxnet3_ethdev.c   | 5 -----
 lib/librte_ethdev/rte_ethdev.c         | 4 +---
 31 files changed, 2 insertions(+), 128 deletions(-)

diff --git a/drivers/net/ark/ark_ethdev.c b/drivers/net/ark/ark_ethdev.c
index e3b1347769..cfcfe5c42f 100644
--- a/drivers/net/ark/ark_ethdev.c
+++ b/drivers/net/ark/ark_ethdev.c
@@ -504,9 +504,6 @@ eth_ark_dev_uninit(struct rte_eth_dev *dev)
 	ark_pktgen_uninit(ark->pg);
 	ark_pktchkr_uninit(ark->pc);
 
-	dev->dev_ops = NULL;
-	dev->rx_pkt_burst = NULL;
-	dev->tx_pkt_burst = NULL;
 	return 0;
 }
 
diff --git a/drivers/net/atlantic/atl_ethdev.c b/drivers/net/atlantic/atl_ethdev.c
index 2217511ca0..5cc41c2b61 100644
--- a/drivers/net/atlantic/atl_ethdev.c
+++ b/drivers/net/atlantic/atl_ethdev.c
@@ -701,10 +701,6 @@ atl_dev_close(struct rte_eth_dev *dev)
 
 	atl_free_queues(dev);
 
-	dev->dev_ops = NULL;
-	dev->rx_pkt_burst = NULL;
-	dev->tx_pkt_burst = NULL;
-
 	/* disable uio intr before callback unregister */
 	rte_intr_disable(intr_handle);
 	rte_intr_callback_unregister(intr_handle,
diff --git a/drivers/net/axgbe/axgbe_ethdev.c b/drivers/net/axgbe/axgbe_ethdev.c
index cf085487cc..39087f36f0 100644
--- a/drivers/net/axgbe/axgbe_ethdev.c
+++ b/drivers/net/axgbe/axgbe_ethdev.c
@@ -2134,9 +2134,6 @@ axgbe_dev_close(struct rte_eth_dev *eth_dev)
 		return 0;
 
 	pci_dev = RTE_DEV_TO_PCI(eth_dev->device);
-	eth_dev->dev_ops = NULL;
-	eth_dev->rx_pkt_burst = NULL;
-	eth_dev->tx_pkt_burst = NULL;
 	axgbe_dev_clear_queues(eth_dev);
 
 	/* disable uio intr before callback unregister */
diff --git a/drivers/net/bnxt/bnxt_ethdev.c b/drivers/net/bnxt/bnxt_ethdev.c
index d45347e8ca..e1c430b2cc 100644
--- a/drivers/net/bnxt/bnxt_ethdev.c
+++ b/drivers/net/bnxt/bnxt_ethdev.c
@@ -1382,10 +1382,6 @@ static int bnxt_dev_close_op(struct rte_eth_dev *eth_dev)
 	bnxt_free_pf_info(bp);
 	bnxt_free_parent_info(bp);
 
-	eth_dev->dev_ops = NULL;
-	eth_dev->rx_pkt_burst = NULL;
-	eth_dev->tx_pkt_burst = NULL;
-
 	rte_memzone_free((const struct rte_memzone *)bp->tx_mem_zone);
 	bp->tx_mem_zone = NULL;
 	rte_memzone_free((const struct rte_memzone *)bp->rx_mem_zone);
diff --git a/drivers/net/bnxt/bnxt_reps.c b/drivers/net/bnxt/bnxt_reps.c
index c419fe63b8..d523c867eb 100644
--- a/drivers/net/bnxt/bnxt_reps.c
+++ b/drivers/net/bnxt/bnxt_reps.c
@@ -225,7 +225,6 @@ int bnxt_vf_representor_uninit(struct rte_eth_dev *eth_dev)
 
 	PMD_DRV_LOG(DEBUG, "BNXT Port:%d VFR uninit\n", eth_dev->data->port_id);
 	eth_dev->data->mac_addrs = NULL;
-	eth_dev->dev_ops = NULL;
 
 	parent_bp = rep->parent_dev->data->dev_private;
 	if (!parent_bp) {
diff --git a/drivers/net/bonding/rte_eth_bond_pmd.c b/drivers/net/bonding/rte_eth_bond_pmd.c
index 1f761c7c9e..ce8fe254af 100644
--- a/drivers/net/bonding/rte_eth_bond_pmd.c
+++ b/drivers/net/bonding/rte_eth_bond_pmd.c
@@ -2130,10 +2130,6 @@ bond_ethdev_close(struct rte_eth_dev *dev)
 	 */
 	rte_mempool_free(internals->mode6.mempool);
 
-	dev->dev_ops = NULL;
-	dev->rx_pkt_burst = NULL;
-	dev->tx_pkt_burst = NULL;
-
 	return 0;
 }
 
diff --git a/drivers/net/dpaa/dpaa_ethdev.c b/drivers/net/dpaa/dpaa_ethdev.c
index 01632e591c..c029dd4f3b 100644
--- a/drivers/net/dpaa/dpaa_ethdev.c
+++ b/drivers/net/dpaa/dpaa_ethdev.c
@@ -2037,10 +2037,6 @@ dpaa_dev_uninit(struct rte_eth_dev *dev)
 	rte_free(dpaa_intf->tx_queues);
 	dpaa_intf->tx_queues = NULL;
 
-	dev->dev_ops = NULL;
-	dev->rx_pkt_burst = NULL;
-	dev->tx_pkt_burst = NULL;
-
 	return 0;
 }
 
diff --git a/drivers/net/dpaa2/dpaa2_ethdev.c b/drivers/net/dpaa2/dpaa2_ethdev.c
index eb84d7231d..c81e75d538 100644
--- a/drivers/net/dpaa2/dpaa2_ethdev.c
+++ b/drivers/net/dpaa2/dpaa2_ethdev.c
@@ -2756,10 +2756,6 @@ dpaa2_dev_uninit(struct rte_eth_dev *eth_dev)
 	if (priv->extract.qos_extract_param)
 		rte_free((void *)(size_t)priv->extract.qos_extract_param);
 
-	eth_dev->dev_ops = NULL;
-	eth_dev->rx_pkt_burst = NULL;
-	eth_dev->tx_pkt_burst = NULL;
-
 	DPAA2_PMD_INFO("%s: netdev deleted", eth_dev->data->name);
 	return 0;
 }
diff --git a/drivers/net/e1000/em_ethdev.c b/drivers/net/e1000/em_ethdev.c
index d050eb478a..151b7543d8 100644
--- a/drivers/net/e1000/em_ethdev.c
+++ b/drivers/net/e1000/em_ethdev.c
@@ -772,10 +772,6 @@ eth_em_close(struct rte_eth_dev *dev)
 	em_release_manageability(hw);
 	em_hw_control_release(hw);
 
-	dev->dev_ops = NULL;
-	dev->rx_pkt_burst = NULL;
-	dev->tx_pkt_burst = NULL;
-
 	/* disable uio intr before callback unregister */
 	rte_intr_disable(intr_handle);
 	rte_intr_callback_unregister(intr_handle,
diff --git a/drivers/net/e1000/igb_ethdev.c b/drivers/net/e1000/igb_ethdev.c
index cb3d97e2a3..a7fcbaf295 100644
--- a/drivers/net/e1000/igb_ethdev.c
+++ b/drivers/net/e1000/igb_ethdev.c
@@ -1564,10 +1564,6 @@ eth_igb_close(struct rte_eth_dev *dev)
 	memset(&link, 0, sizeof(link));
 	rte_eth_linkstatus_set(dev, &link);
 
-	dev->dev_ops = NULL;
-	dev->rx_pkt_burst = NULL;
-	dev->tx_pkt_burst = NULL;
-
 	/* Reset any pending lock */
 	igb_reset_swfw_lock(hw);
 
@@ -3403,10 +3399,6 @@ igbvf_dev_close(struct rte_eth_dev *dev)
 	memset(&addr, 0, sizeof(addr));
 	igbvf_default_mac_addr_set(dev, &addr);
 
-	dev->dev_ops = NULL;
-	dev->rx_pkt_burst = NULL;
-	dev->tx_pkt_burst = NULL;
-
 	rte_intr_callback_unregister(&pci_dev->intr_handle,
 				     eth_igbvf_interrupt_handler,
 				     (void *)dev);
diff --git a/drivers/net/ena/ena_ethdev.c b/drivers/net/ena/ena_ethdev.c
index 07ae9bab32..b42f12b870 100644
--- a/drivers/net/ena/ena_ethdev.c
+++ b/drivers/net/ena/ena_ethdev.c
@@ -1933,11 +1933,6 @@ static int eth_ena_dev_uninit(struct rte_eth_dev *eth_dev)
 
 	ena_destroy_device(eth_dev);
 
-	eth_dev->dev_ops = NULL;
-	eth_dev->rx_pkt_burst = NULL;
-	eth_dev->tx_pkt_burst = NULL;
-	eth_dev->tx_pkt_prepare = NULL;
-
 	return 0;
 }
 
diff --git a/drivers/net/fm10k/fm10k_ethdev.c b/drivers/net/fm10k/fm10k_ethdev.c
index 17cf0328c6..2d51f50faa 100644
--- a/drivers/net/fm10k/fm10k_ethdev.c
+++ b/drivers/net/fm10k/fm10k_ethdev.c
@@ -2801,10 +2801,6 @@ fm10k_dev_close(struct rte_eth_dev *dev)
 	fm10k_dev_queue_release(dev);
 	fm10k_stop_hw(hw);
 
-	dev->dev_ops = NULL;
-	dev->rx_pkt_burst = NULL;
-	dev->tx_pkt_burst = NULL;
-
 	/* disable uio/vfio intr */
 	rte_intr_disable(intr_handle);
 
diff --git a/drivers/net/hinic/hinic_pmd_ethdev.c b/drivers/net/hinic/hinic_pmd_ethdev.c
index c5b06fae73..57084e59c1 100644
--- a/drivers/net/hinic/hinic_pmd_ethdev.c
+++ b/drivers/net/hinic/hinic_pmd_ethdev.c
@@ -3222,10 +3222,6 @@ static int hinic_dev_uninit(struct rte_eth_dev *dev)
 
 	hinic_dev_close(dev);
 
-	dev->dev_ops = NULL;
-	dev->rx_pkt_burst = NULL;
-	dev->tx_pkt_burst = NULL;
-
 	rte_free(nic_dev->mc_list);
 
 	return HINIC_OK;
diff --git a/drivers/net/hns3/hns3_ethdev.c b/drivers/net/hns3/hns3_ethdev.c
index dd03bd62b2..ba40d489d6 100644
--- a/drivers/net/hns3/hns3_ethdev.c
+++ b/drivers/net/hns3/hns3_ethdev.c
@@ -5721,10 +5721,6 @@ hns3_dev_uninit(struct rte_eth_dev *eth_dev)
 	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
 		return -EPERM;
 
-	eth_dev->dev_ops = NULL;
-	eth_dev->rx_pkt_burst = NULL;
-	eth_dev->tx_pkt_burst = NULL;
-	eth_dev->tx_pkt_prepare = NULL;
 	if (hw->adapter_state < HNS3_NIC_CLOSING)
 		hns3_dev_close(eth_dev);
 
diff --git a/drivers/net/hns3/hns3_ethdev_vf.c b/drivers/net/hns3/hns3_ethdev_vf.c
index 436d864b5f..6a04cd9c2c 100644
--- a/drivers/net/hns3/hns3_ethdev_vf.c
+++ b/drivers/net/hns3/hns3_ethdev_vf.c
@@ -2728,11 +2728,6 @@ hns3vf_dev_uninit(struct rte_eth_dev *eth_dev)
 	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
 		return -EPERM;
 
-	eth_dev->dev_ops = NULL;
-	eth_dev->rx_pkt_burst = NULL;
-	eth_dev->tx_pkt_burst = NULL;
-	eth_dev->tx_pkt_prepare = NULL;
-
 	if (hw->adapter_state < HNS3_NIC_CLOSING)
 		hns3vf_dev_close(eth_dev);
 
diff --git a/drivers/net/i40e/i40e_ethdev.c b/drivers/net/i40e/i40e_ethdev.c
index feeed2dec0..a05d29dcfc 100644
--- a/drivers/net/i40e/i40e_ethdev.c
+++ b/drivers/net/i40e/i40e_ethdev.c
@@ -2692,10 +2692,6 @@ i40e_dev_close(struct rte_eth_dev *dev)
 			(reg | I40E_PFGEN_CTRL_PFSWR_MASK));
 	I40E_WRITE_FLUSH(hw);
 
-	dev->dev_ops = NULL;
-	dev->rx_pkt_burst = NULL;
-	dev->tx_pkt_burst = NULL;
-
 	/* Clear PXE mode */
 	i40e_clear_pxe_mode(hw);
 
diff --git a/drivers/net/i40e/i40e_ethdev_vf.c b/drivers/net/i40e/i40e_ethdev_vf.c
index 4d6510d1ff..2001854ae6 100644
--- a/drivers/net/i40e/i40e_ethdev_vf.c
+++ b/drivers/net/i40e/i40e_ethdev_vf.c
@@ -2421,10 +2421,6 @@ i40evf_dev_close(struct rte_eth_dev *dev)
 	i40e_shutdown_adminq(hw);
 	i40evf_disable_irq0(hw);
 
-	dev->dev_ops = NULL;
-	dev->rx_pkt_burst = NULL;
-	dev->tx_pkt_burst = NULL;
-
 	rte_free(vf->vf_res);
 	vf->vf_res = NULL;
 	rte_free(vf->aq_resp);
diff --git a/drivers/net/iavf/iavf_ethdev.c b/drivers/net/iavf/iavf_ethdev.c
index a5b1433306..02391acfd1 100644
--- a/drivers/net/iavf/iavf_ethdev.c
+++ b/drivers/net/iavf/iavf_ethdev.c
@@ -1483,10 +1483,6 @@ iavf_dev_close(struct rte_eth_dev *dev)
 				     iavf_dev_interrupt_handler, dev);
 	iavf_disable_irq0(hw);
 
-	dev->dev_ops = NULL;
-	dev->rx_pkt_burst = NULL;
-	dev->tx_pkt_burst = NULL;
-
 	if (vf->vf_res->vf_cap_flags & VIRTCHNL_VF_OFFLOAD_RSS_PF) {
 		if (vf->rss_lut) {
 			rte_free(vf->rss_lut);
diff --git a/drivers/net/ice/ice_dcf_ethdev.c b/drivers/net/ice/ice_dcf_ethdev.c
index 33dd0c44f2..f2056e2aba 100644
--- a/drivers/net/ice/ice_dcf_ethdev.c
+++ b/drivers/net/ice/ice_dcf_ethdev.c
@@ -855,10 +855,6 @@ ice_dcf_dev_close(struct rte_eth_dev *dev)
 	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
 		return 0;
 
-	dev->dev_ops = NULL;
-	dev->rx_pkt_burst = NULL;
-	dev->tx_pkt_burst = NULL;
-
 	ice_dcf_uninit_parent_adapter(dev);
 	ice_dcf_uninit_hw(dev, &adapter->real_hw);
 
diff --git a/drivers/net/ice/ice_ethdev.c b/drivers/net/ice/ice_ethdev.c
index 73af19ce81..4f977bd4c4 100644
--- a/drivers/net/ice/ice_ethdev.c
+++ b/drivers/net/ice/ice_ethdev.c
@@ -2463,10 +2463,6 @@ ice_dev_close(struct rte_eth_dev *dev)
 	rte_free(pf->proto_xtr);
 	pf->proto_xtr = NULL;
 
-	dev->dev_ops = NULL;
-	dev->rx_pkt_burst = NULL;
-	dev->tx_pkt_burst = NULL;
-
 	/* disable uio intr before callback unregister */
 	rte_intr_disable(intr_handle);
 
diff --git a/drivers/net/ionic/ionic_ethdev.c b/drivers/net/ionic/ionic_ethdev.c
index ef7d06e526..1387253dc8 100644
--- a/drivers/net/ionic/ionic_ethdev.c
+++ b/drivers/net/ionic/ionic_ethdev.c
@@ -1065,11 +1065,6 @@ eth_ionic_dev_uninit(struct rte_eth_dev *eth_dev)
 	ionic_lif_deinit(lif);
 	ionic_lif_free(lif);
 
-	eth_dev->dev_ops = NULL;
-	eth_dev->rx_pkt_burst = NULL;
-	eth_dev->tx_pkt_burst = NULL;
-	eth_dev->tx_pkt_prepare = NULL;
-
 	return 0;
 }
 
diff --git a/drivers/net/ixgbe/ixgbe_ethdev.c b/drivers/net/ixgbe/ixgbe_ethdev.c
index f6e383b1c2..1759a3778c 100644
--- a/drivers/net/ixgbe/ixgbe_ethdev.c
+++ b/drivers/net/ixgbe/ixgbe_ethdev.c
@@ -3009,10 +3009,6 @@ ixgbe_dev_close(struct rte_eth_dev *dev)
 	/* reprogram the RAR[0] in case user changed it. */
 	ixgbe_set_rar(hw, 0, hw->mac.addr, 0, IXGBE_RAH_AV);
 
-	dev->dev_ops = NULL;
-	dev->rx_pkt_burst = NULL;
-	dev->tx_pkt_burst = NULL;
-
 	/* Unlock any pending hardware semaphore */
 	ixgbe_swfw_lock_reset(hw);
 
@@ -5460,10 +5456,6 @@ ixgbevf_dev_close(struct rte_eth_dev *dev)
 	 **/
 	ixgbevf_remove_mac_addr(dev, 0);
 
-	dev->dev_ops = NULL;
-	dev->rx_pkt_burst = NULL;
-	dev->tx_pkt_burst = NULL;
-
 	rte_intr_disable(intr_handle);
 	rte_intr_callback_unregister(intr_handle,
 				     ixgbevf_dev_interrupt_handler, dev);
diff --git a/drivers/net/liquidio/lio_ethdev.c b/drivers/net/liquidio/lio_ethdev.c
index 1a41f27198..9c4da467eb 100644
--- a/drivers/net/liquidio/lio_ethdev.c
+++ b/drivers/net/liquidio/lio_ethdev.c
@@ -2067,10 +2067,6 @@ lio_eth_dev_uninit(struct rte_eth_dev *eth_dev)
 	/* lio_free_sc_buffer_pool */
 	lio_free_sc_buffer_pool(lio_dev);
 
-	eth_dev->dev_ops = NULL;
-	eth_dev->rx_pkt_burst = NULL;
-	eth_dev->tx_pkt_burst = NULL;
-
 	return 0;
 }
 
diff --git a/drivers/net/netvsc/hn_ethdev.c b/drivers/net/netvsc/hn_ethdev.c
index 5ae2d469c8..9af64821a1 100644
--- a/drivers/net/netvsc/hn_ethdev.c
+++ b/drivers/net/netvsc/hn_ethdev.c
@@ -1049,10 +1049,6 @@ eth_hn_dev_uninit(struct rte_eth_dev *eth_dev)
 	hn_dev_stop(eth_dev);
 	hn_dev_close(eth_dev);
 
-	eth_dev->dev_ops = NULL;
-	eth_dev->tx_pkt_burst = NULL;
-	eth_dev->rx_pkt_burst = NULL;
-
 	hn_detach(hv);
 	hn_chim_uninit(eth_dev);
 	rte_vmbus_chan_close(hv->primary->chan);
diff --git a/drivers/net/octeontx/octeontx_ethdev.c b/drivers/net/octeontx/octeontx_ethdev.c
index aa9ef3bb70..0ea6de448a 100644
--- a/drivers/net/octeontx/octeontx_ethdev.c
+++ b/drivers/net/octeontx/octeontx_ethdev.c
@@ -513,9 +513,6 @@ octeontx_dev_close(struct rte_eth_dev *dev)
 
 	octeontx_port_close(nic);
 
-	dev->tx_pkt_burst = NULL;
-	dev->rx_pkt_burst = NULL;
-
 	return 0;
 }
 
diff --git a/drivers/net/qede/qede_ethdev.c b/drivers/net/qede/qede_ethdev.c
index cd578709fb..4055c74c2d 100644
--- a/drivers/net/qede/qede_ethdev.c
+++ b/drivers/net/qede/qede_ethdev.c
@@ -1591,10 +1591,6 @@ static int qede_dev_close(struct rte_eth_dev *eth_dev)
 	if (ECORE_IS_CMT(edev))
 		rte_eal_alarm_cancel(qede_poll_sp_sb_cb, (void *)eth_dev);
 
-	eth_dev->dev_ops = NULL;
-	eth_dev->rx_pkt_burst = NULL;
-	eth_dev->tx_pkt_burst = NULL;
-
 	return 0;
 }
 
diff --git a/drivers/net/sfc/sfc_ethdev.c b/drivers/net/sfc/sfc_ethdev.c
index 0558f1808d..9ee0b994eb 100644
--- a/drivers/net/sfc/sfc_ethdev.c
+++ b/drivers/net/sfc/sfc_ethdev.c
@@ -322,11 +322,7 @@ static void
 sfc_eth_dev_secondary_clear_ops(struct rte_eth_dev *dev)
 {
 	free(dev->process_private);
-	dev->process_private = NULL;
-	dev->dev_ops = NULL;
-	dev->tx_pkt_prepare = NULL;
-	dev->tx_pkt_burst = NULL;
-	dev->rx_pkt_burst = NULL;
+	rte_eth_dev_release_port(dev);
 }
 
 static int
@@ -378,7 +374,6 @@ sfc_dev_close(struct rte_eth_dev *dev)
 	/* Required for logging, so cleanup last */
 	sa->eth_dev = NULL;
 
-	dev->process_private = NULL;
 	free(sa);
 
 	return 0;
diff --git a/drivers/net/tap/rte_eth_tap.c b/drivers/net/tap/rte_eth_tap.c
index b127ce62dc..0598de00ed 100644
--- a/drivers/net/tap/rte_eth_tap.c
+++ b/drivers/net/tap/rte_eth_tap.c
@@ -1134,7 +1134,6 @@ tap_dev_close(struct rte_eth_dev *dev)
 		internals->ioctl_sock = -1;
 	}
 	rte_free(dev->process_private);
-	dev->process_private = NULL;
 	if (tap_devices_count == 1)
 		rte_mp_action_unregister(TAP_MP_KEY);
 	tap_devices_count--;
diff --git a/drivers/net/virtio/virtio_ethdev.c b/drivers/net/virtio/virtio_ethdev.c
index f2117675b3..3b3b2f6a7e 100644
--- a/drivers/net/virtio/virtio_ethdev.c
+++ b/drivers/net/virtio/virtio_ethdev.c
@@ -1995,10 +1995,6 @@ eth_virtio_dev_uninit(struct rte_eth_dev *eth_dev)
 	virtio_dev_stop(eth_dev);
 	virtio_dev_close(eth_dev);
 
-	eth_dev->dev_ops = NULL;
-	eth_dev->tx_pkt_burst = NULL;
-	eth_dev->rx_pkt_burst = NULL;
-
 	PMD_INIT_LOG(DEBUG, "dev_uninit completed");
 
 	return 0;
diff --git a/drivers/net/vmxnet3/vmxnet3_ethdev.c b/drivers/net/vmxnet3/vmxnet3_ethdev.c
index fa950e1ba0..12a782d987 100644
--- a/drivers/net/vmxnet3/vmxnet3_ethdev.c
+++ b/drivers/net/vmxnet3/vmxnet3_ethdev.c
@@ -368,11 +368,6 @@ eth_vmxnet3_dev_uninit(struct rte_eth_dev *eth_dev)
 		return -EBUSY;
 	}
 
-	eth_dev->dev_ops = NULL;
-	eth_dev->rx_pkt_burst = NULL;
-	eth_dev->tx_pkt_burst = NULL;
-	eth_dev->tx_pkt_prepare = NULL;
-
 	return 0;
 }
 
diff --git a/lib/librte_ethdev/rte_ethdev.c b/lib/librte_ethdev/rte_ethdev.c
index 0b8e8e3e8d..5c0e8f170e 100644
--- a/lib/librte_ethdev/rte_ethdev.c
+++ b/lib/librte_ethdev/rte_ethdev.c
@@ -554,9 +554,7 @@ rte_eth_dev_release_port(struct rte_eth_dev *eth_dev)
 
 	rte_spinlock_lock(&rte_eth_dev_shared_data->ownership_lock);
 
-	eth_dev->state = RTE_ETH_DEV_UNUSED;
-	eth_dev->device = NULL;
-	eth_dev->intr_handle = NULL;
+	memset(eth_dev, 0, sizeof(*eth_dev));
 
 	if (rte_eal_process_type() == RTE_PROC_PRIMARY) {
 		rte_free(eth_dev->data->rx_queues);
-- 
2.28.0

[dpdk-dev] [PATCH v2 25/25] ethdev: allow close function to return an error

[Thomas Monjalon]

The API function rte_eth_dev_close() was returning void.
The return type is changed to int for notifying of errors.

If an error happens during a close operation,
the status of the port is undefined,
a maximum of resources having been freed.

Signed-off-by: Thomas Monjalon <thomas@monjalon.net>
---
 doc/guides/rel_notes/deprecation.rst    |  1 -
 doc/guides/rel_notes/release_20_11.rst  |  4 +++-
 drivers/net/cxgbe/cxgbe_ethdev.c        |  5 +++--
 drivers/net/cxgbe/cxgbevf_ethdev.c      |  5 +++--
 drivers/net/failsafe/failsafe_ether.c   |  6 +++++-
 drivers/net/failsafe/failsafe_ops.c     |  5 ++++-
 drivers/net/memif/rte_eth_memif.c       |  4 +---
 drivers/net/mlx5/mlx5.c                 |  7 ++++---
 drivers/net/mvneta/mvneta_ethdev.c      |  5 +++--
 drivers/net/mvpp2/mrvl_ethdev.c         |  5 +++--
 drivers/net/netvsc/hn_ethdev.c          |  6 ++++--
 drivers/net/netvsc/hn_var.h             |  2 +-
 drivers/net/netvsc/hn_vf.c              |  7 +++++--
 drivers/net/virtio/virtio_user_ethdev.c |  4 +---
 lib/librte_ethdev/rte_ethdev.c          | 13 ++++++++-----
 lib/librte_ethdev/rte_ethdev.h          |  5 ++++-
 16 files changed, 52 insertions(+), 32 deletions(-)

diff --git a/doc/guides/rel_notes/deprecation.rst b/doc/guides/rel_notes/deprecation.rst
index a907358078..bfa43990f0 100644
--- a/doc/guides/rel_notes/deprecation.rst
+++ b/doc/guides/rel_notes/deprecation.rst
@@ -140,7 +140,6 @@ Deprecation Notices
   invalid port ID, unsupported operation, failed operation):
 
   - ``rte_eth_dev_stop``
-  - ``rte_eth_dev_close``
 
 * ethdev: New offload flags ``DEV_RX_OFFLOAD_FLOW_MARK`` will be added in 19.11.
   This will allow application to enable or disable PMDs from updating
diff --git a/doc/guides/rel_notes/release_20_11.rst b/doc/guides/rel_notes/release_20_11.rst
index f377ab8e87..ec5a3937e8 100644
--- a/doc/guides/rel_notes/release_20_11.rst
+++ b/doc/guides/rel_notes/release_20_11.rst
@@ -152,7 +152,9 @@ API Changes
 
 * ethdev: ``rte_eth_rx_descriptor_done()`` API has been deprecated.
 
-* Renamed internal ethdev APIs:
+* ethdev: Added ``int`` return type to ``rte_eth_dev_close()``.
+
+* ethdev: Renamed internal functions:
 
   * ``_rte_eth_dev_callback_process()`` -> ``rte_eth_dev_callback_process()``
   * ``_rte_eth_dev_reset`` -> ``rte_eth_dev_internal_reset()``
diff --git a/drivers/net/cxgbe/cxgbe_ethdev.c b/drivers/net/cxgbe/cxgbe_ethdev.c
index 16beb2d435..fe488231a7 100644
--- a/drivers/net/cxgbe/cxgbe_ethdev.c
+++ b/drivers/net/cxgbe/cxgbe_ethdev.c
@@ -1296,12 +1296,13 @@ static int eth_cxgbe_dev_uninit(struct rte_eth_dev *eth_dev)
 {
 	struct rte_pci_device *pci_dev = RTE_ETH_DEV_TO_PCI(eth_dev);
 	uint16_t port_id;
+	int err = 0;
 
 	/* Free up other ports and all resources */
 	RTE_ETH_FOREACH_DEV_OF(port_id, &pci_dev->device)
-		rte_eth_dev_close(port_id);
+		err |= rte_eth_dev_close(port_id);
 
-	return 0;
+	return err == 0 ? 0 : -EIO;
 }
 
 static int eth_cxgbe_pci_probe(struct rte_pci_driver *pci_drv __rte_unused,
diff --git a/drivers/net/cxgbe/cxgbevf_ethdev.c b/drivers/net/cxgbe/cxgbevf_ethdev.c
index 947fcdd406..c2918f5356 100644
--- a/drivers/net/cxgbe/cxgbevf_ethdev.c
+++ b/drivers/net/cxgbe/cxgbevf_ethdev.c
@@ -183,12 +183,13 @@ static int eth_cxgbevf_dev_uninit(struct rte_eth_dev *eth_dev)
 {
 	struct rte_pci_device *pci_dev = RTE_ETH_DEV_TO_PCI(eth_dev);
 	uint16_t port_id;
+	int err = 0;
 
 	/* Free up other ports and all resources */
 	RTE_ETH_FOREACH_DEV_OF(port_id, &pci_dev->device)
-		rte_eth_dev_close(port_id);
+		err |= rte_eth_dev_close(port_id);
 
-	return 0;
+	return err == 0 ? 0 : -EIO;
 }
 
 static int eth_cxgbevf_pci_probe(struct rte_pci_driver *pci_drv __rte_unused,
diff --git a/drivers/net/failsafe/failsafe_ether.c b/drivers/net/failsafe/failsafe_ether.c
index 7c68bbdec0..950d35dac4 100644
--- a/drivers/net/failsafe/failsafe_ether.c
+++ b/drivers/net/failsafe/failsafe_ether.c
@@ -287,7 +287,11 @@ fs_dev_remove(struct sub_device *sdev)
 		/* fallthrough */
 	case DEV_ACTIVE:
 		failsafe_eth_dev_unregister_callbacks(sdev);
-		rte_eth_dev_close(PORT_ID(sdev));
+		ret = rte_eth_dev_close(PORT_ID(sdev));
+		if (ret < 0) {
+			ERROR("Port close failed for sub-device %u",
+			      PORT_ID(sdev));
+		}
 		sdev->state = DEV_PROBED;
 		/* fallthrough */
 	case DEV_PROBED:
diff --git a/drivers/net/failsafe/failsafe_ops.c b/drivers/net/failsafe/failsafe_ops.c
index 0ce7dfc8a6..79b24ec996 100644
--- a/drivers/net/failsafe/failsafe_ops.c
+++ b/drivers/net/failsafe/failsafe_ops.c
@@ -648,7 +648,10 @@ failsafe_eth_dev_close(struct rte_eth_dev *dev)
 	FOREACH_SUBDEV_STATE(sdev, i, dev, DEV_ACTIVE) {
 		DEBUG("Closing sub_device %d", i);
 		failsafe_eth_dev_unregister_callbacks(sdev);
-		rte_eth_dev_close(PORT_ID(sdev));
+		ret = rte_eth_dev_close(PORT_ID(sdev));
+		if (ret)
+			ERROR("Error while closing sub-device %u",
+					PORT_ID(sdev));
 		sdev->state = DEV_ACTIVE - 1;
 	}
 	rte_eth_dev_callback_unregister(RTE_ETH_ALL, RTE_ETH_EVENT_NEW,
diff --git a/drivers/net/memif/rte_eth_memif.c b/drivers/net/memif/rte_eth_memif.c
index ff8a58081f..33bf5c68a3 100644
--- a/drivers/net/memif/rte_eth_memif.c
+++ b/drivers/net/memif/rte_eth_memif.c
@@ -1798,9 +1798,7 @@ rte_pmd_memif_remove(struct rte_vdev_device *vdev)
 	if (eth_dev == NULL)
 		return 0;
 
-	rte_eth_dev_close(eth_dev->data->port_id);
-
-	return 0;
+	return rte_eth_dev_close(eth_dev->data->port_id);
 }
 
 static struct rte_vdev_driver pmd_memif_drv = {
diff --git a/drivers/net/mlx5/mlx5.c b/drivers/net/mlx5/mlx5.c
index 01ead6e6af..2e2f0b274c 100644
--- a/drivers/net/mlx5/mlx5.c
+++ b/drivers/net/mlx5/mlx5.c
@@ -2010,6 +2010,7 @@ static int
 mlx5_pci_remove(struct rte_pci_device *pci_dev)
 {
 	uint16_t port_id;
+	int ret = 0;
 
 	RTE_ETH_FOREACH_DEV_OF(port_id, &pci_dev->device) {
 		/*
@@ -2017,11 +2018,11 @@ mlx5_pci_remove(struct rte_pci_device *pci_dev)
 		 * call the close function explicitly for secondary process.
 		 */
 		if (rte_eal_process_type() == RTE_PROC_SECONDARY)
-			mlx5_dev_close(&rte_eth_devices[port_id]);
+			ret |= mlx5_dev_close(&rte_eth_devices[port_id]);
 		else
-			rte_eth_dev_close(port_id);
+			ret |= rte_eth_dev_close(port_id);
 	}
-	return 0;
+	return ret == 0 ? 0 : -EIO;
 }
 
 static const struct rte_pci_id mlx5_pci_id_map[] = {
diff --git a/drivers/net/mvneta/mvneta_ethdev.c b/drivers/net/mvneta/mvneta_ethdev.c
index 607771149a..13d4b6af6b 100644
--- a/drivers/net/mvneta/mvneta_ethdev.c
+++ b/drivers/net/mvneta/mvneta_ethdev.c
@@ -964,14 +964,15 @@ static int
 rte_pmd_mvneta_remove(struct rte_vdev_device *vdev)
 {
 	uint16_t port_id;
+	int ret = 0;
 
 	RTE_ETH_FOREACH_DEV(port_id) {
 		if (rte_eth_devices[port_id].device != &vdev->device)
 			continue;
-		rte_eth_dev_close(port_id);
+		ret = rte_eth_dev_close(port_id);
 	}
 
-	return 0;
+	return ret == 0 ? 0 : -EIO;
 }
 
 static struct rte_vdev_driver pmd_mvneta_drv = {
diff --git a/drivers/net/mvpp2/mrvl_ethdev.c b/drivers/net/mvpp2/mrvl_ethdev.c
index a230a96840..acc8c70a95 100644
--- a/drivers/net/mvpp2/mrvl_ethdev.c
+++ b/drivers/net/mvpp2/mrvl_ethdev.c
@@ -3022,14 +3022,15 @@ static int
 rte_pmd_mrvl_remove(struct rte_vdev_device *vdev)
 {
 	uint16_t port_id;
+	int ret = 0;
 
 	RTE_ETH_FOREACH_DEV(port_id) {
 		if (rte_eth_devices[port_id].device != &vdev->device)
 			continue;
-		rte_eth_dev_close(port_id);
+		ret = rte_eth_dev_close(port_id);
 	}
 
-	return 0;
+	return ret == 0 ? 0 : -EIO;
 }
 
 static struct rte_vdev_driver pmd_mrvl_drv = {
diff --git a/drivers/net/netvsc/hn_ethdev.c b/drivers/net/netvsc/hn_ethdev.c
index 9af64821a1..3fa39dbebc 100644
--- a/drivers/net/netvsc/hn_ethdev.c
+++ b/drivers/net/netvsc/hn_ethdev.c
@@ -841,14 +841,16 @@ hn_dev_stop(struct rte_eth_dev *dev)
 static int
 hn_dev_close(struct rte_eth_dev *dev)
 {
+	int ret;
+
 	PMD_INIT_FUNC_TRACE();
 	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
 		return 0;
 
-	hn_vf_close(dev);
+	ret = hn_vf_close(dev);
 	hn_dev_free_queues(dev);
 
-	return 0;
+	return ret;
 }
 
 static const struct eth_dev_ops hn_eth_dev_ops = {
diff --git a/drivers/net/netvsc/hn_var.h b/drivers/net/netvsc/hn_var.h
index 4b63f87607..74f30669ac 100644
--- a/drivers/net/netvsc/hn_var.h
+++ b/drivers/net/netvsc/hn_var.h
@@ -217,7 +217,7 @@ const uint32_t *hn_vf_supported_ptypes(struct rte_eth_dev *dev);
 int	hn_vf_start(struct rte_eth_dev *dev);
 void	hn_vf_reset(struct rte_eth_dev *dev);
 void	hn_vf_stop(struct rte_eth_dev *dev);
-void	hn_vf_close(struct rte_eth_dev *dev);
+int	hn_vf_close(struct rte_eth_dev *dev);
 
 int	hn_vf_allmulticast_enable(struct rte_eth_dev *dev);
 int	hn_vf_allmulticast_disable(struct rte_eth_dev *dev);
diff --git a/drivers/net/netvsc/hn_vf.c b/drivers/net/netvsc/hn_vf.c
index f5f15c0462..d29eee7627 100644
--- a/drivers/net/netvsc/hn_vf.c
+++ b/drivers/net/netvsc/hn_vf.c
@@ -316,18 +316,21 @@ void hn_vf_reset(struct rte_eth_dev *dev)
 	VF_ETHDEV_FUNC(dev, rte_eth_dev_reset);
 }
 
-void hn_vf_close(struct rte_eth_dev *dev)
+int hn_vf_close(struct rte_eth_dev *dev)
 {
 	struct hn_data *hv = dev->data->dev_private;
 	uint16_t vf_port;
+	int ret = 0;
 
 	rte_rwlock_read_lock(&hv->vf_lock);
 	vf_port = hv->vf_port;
 	if (vf_port != HN_INVALID_PORT)
-		rte_eth_dev_close(vf_port);
+		ret = rte_eth_dev_close(vf_port);
 
 	hv->vf_port = HN_INVALID_PORT;
 	rte_rwlock_read_unlock(&hv->vf_lock);
+
+	return ret;
 }
 
 int hn_vf_stats_reset(struct rte_eth_dev *dev)
diff --git a/drivers/net/virtio/virtio_user_ethdev.c b/drivers/net/virtio/virtio_user_ethdev.c
index 87f6cb6950..865f73807f 100644
--- a/drivers/net/virtio/virtio_user_ethdev.c
+++ b/drivers/net/virtio/virtio_user_ethdev.c
@@ -813,9 +813,7 @@ virtio_user_pmd_remove(struct rte_vdev_device *vdev)
 		return rte_eth_dev_release_port(eth_dev);
 
 	/* make sure the device is stopped, queues freed */
-	rte_eth_dev_close(eth_dev->data->port_id);
-
-	return 0;
+	return rte_eth_dev_close(eth_dev->data->port_id);
 }
 
 static struct rte_vdev_driver virtio_user_driver = {
diff --git a/lib/librte_ethdev/rte_ethdev.c b/lib/librte_ethdev/rte_ethdev.c
index 5c0e8f170e..6091630a2f 100644
--- a/lib/librte_ethdev/rte_ethdev.c
+++ b/lib/librte_ethdev/rte_ethdev.c
@@ -1705,19 +1705,22 @@ rte_eth_dev_set_link_down(uint16_t port_id)
 	return eth_err(port_id, (*dev->dev_ops->dev_set_link_down)(dev));
 }
 
-void
+int
 rte_eth_dev_close(uint16_t port_id)
 {
 	struct rte_eth_dev *dev;
+	int ret;
 
-	RTE_ETH_VALID_PORTID_OR_RET(port_id);
+	RTE_ETH_VALID_PORTID_OR_ERR_RET(port_id, -EINVAL);
 	dev = &rte_eth_devices[port_id];
 
-	RTE_FUNC_PTR_OR_RET(*dev->dev_ops->dev_close);
-	(*dev->dev_ops->dev_close)(dev);
+	RTE_FUNC_PTR_OR_ERR_RET(*dev->dev_ops->dev_close, -ENOTSUP);
+	ret = (*dev->dev_ops->dev_close)(dev);
 
 	rte_ethdev_trace_close(port_id);
-	rte_eth_dev_release_port(dev);
+	ret = rte_eth_dev_release_port(dev);
+
+	return ret;
 }
 
 int
diff --git a/lib/librte_ethdev/rte_ethdev.h b/lib/librte_ethdev/rte_ethdev.h
index 24d898ae89..4380d8ecf6 100644
--- a/lib/librte_ethdev/rte_ethdev.h
+++ b/lib/librte_ethdev/rte_ethdev.h
@@ -2281,8 +2281,11 @@ int rte_eth_dev_set_link_down(uint16_t port_id);
  *
  * @param port_id
  *   The port identifier of the Ethernet device.
+ * @return
+ *   - Zero if the port is closed successfully.
+ *   - Negative if something went wrong.
  */
-void rte_eth_dev_close(uint16_t port_id);
+int rte_eth_dev_close(uint16_t port_id);
 
 /**
  * Reset a Ethernet device and keep its port id.
-- 
2.28.0

Re: [dpdk-dev] [EXT] [PATCH v2 25/25] ethdev: allow close function to return an error

[Liron Himi]

For mvpp2, mvneta.

Reviewed-by: Liron Himi<lironh@marvell.com>

-----Original Message-----
From: Thomas Monjalon <thomas@monjalon.net> 
Sent: Monday, 28 September 2020 02:43
To: dev@dpdk.org
Cc: ferruh.yigit@intel.com; arybchenko@solarflare.com; Ray Kinsella <mdr@ashroe.eu>; Neil Horman <nhorman@tuxdriver.com>; Rahul Lakkireddy <rahul.lakkireddy@chelsio.com>; Gaetan Rivet <grive@u256.net>; Jakub Grajciar <jgrajcia@cisco.com>; Matan Azrad <matan@nvidia.com>; Shahaf Shuler <shahafs@nvidia.com>; Viacheslav Ovsiienko <viacheslavo@nvidia.com>; Zyta Szpak <zr@semihalf.com>; Liron Himi <lironh@marvell.com>; Stephen Hemminger <sthemmin@microsoft.com>; K. Y. Srinivasan <kys@microsoft.com>; Haiyang Zhang <haiyangz@microsoft.com>; Long Li <longli@microsoft.com>; Maxime Coquelin <maxime.coquelin@redhat.com>; Chenbo Xia <chenbo.xia@intel.com>; Zhihong Wang <zhihong.wang@intel.com>
Subject: [EXT] [PATCH v2 25/25] ethdev: allow close function to return an error

External Email

----------------------------------------------------------------------
The API function rte_eth_dev_close() was returning void.
The return type is changed to int for notifying of errors.

If an error happens during a close operation, the status of the port is undefined, a maximum of resources having been freed.

Signed-off-by: Thomas Monjalon <thomas@monjalon.net>
---
 doc/guides/rel_notes/deprecation.rst    |  1 -
 doc/guides/rel_notes/release_20_11.rst  |  4 +++-
 drivers/net/cxgbe/cxgbe_ethdev.c        |  5 +++--
 drivers/net/cxgbe/cxgbevf_ethdev.c      |  5 +++--
 drivers/net/failsafe/failsafe_ether.c   |  6 +++++-
 drivers/net/failsafe/failsafe_ops.c     |  5 ++++-
 drivers/net/memif/rte_eth_memif.c       |  4 +---
 drivers/net/mlx5/mlx5.c                 |  7 ++++---
 drivers/net/mvneta/mvneta_ethdev.c      |  5 +++--
 drivers/net/mvpp2/mrvl_ethdev.c         |  5 +++--
 drivers/net/netvsc/hn_ethdev.c          |  6 ++++--
 drivers/net/netvsc/hn_var.h             |  2 +-
 drivers/net/netvsc/hn_vf.c              |  7 +++++--
 drivers/net/virtio/virtio_user_ethdev.c |  4 +---
 lib/librte_ethdev/rte_ethdev.c          | 13 ++++++++-----
 lib/librte_ethdev/rte_ethdev.h          |  5 ++++-
 16 files changed, 52 insertions(+), 32 deletions(-)

diff --git a/doc/guides/rel_notes/deprecation.rst b/doc/guides/rel_notes/deprecation.rst
index a907358078..bfa43990f0 100644
--- a/doc/guides/rel_notes/deprecation.rst
+++ b/doc/guides/rel_notes/deprecation.rst
@@ -140,7 +140,6 @@ Deprecation Notices
   invalid port ID, unsupported operation, failed operation):
 
   - ``rte_eth_dev_stop``
-  - ``rte_eth_dev_close``
 
 * ethdev: New offload flags ``DEV_RX_OFFLOAD_FLOW_MARK`` will be added in 19.11.
   This will allow application to enable or disable PMDs from updating diff --git a/doc/guides/rel_notes/release_20_11.rst b/doc/guides/rel_notes/release_20_11.rst
index f377ab8e87..ec5a3937e8 100644
--- a/doc/guides/rel_notes/release_20_11.rst
+++ b/doc/guides/rel_notes/release_20_11.rst
@@ -152,7 +152,9 @@ API Changes
 
 * ethdev: ``rte_eth_rx_descriptor_done()`` API has been deprecated.
 
-* Renamed internal ethdev APIs:
+* ethdev: Added ``int`` return type to ``rte_eth_dev_close()``.
+
+* ethdev: Renamed internal functions:
 
   * ``_rte_eth_dev_callback_process()`` -> ``rte_eth_dev_callback_process()``
   * ``_rte_eth_dev_reset`` -> ``rte_eth_dev_internal_reset()`` diff --git a/drivers/net/cxgbe/cxgbe_ethdev.c b/drivers/net/cxgbe/cxgbe_ethdev.c
index 16beb2d435..fe488231a7 100644
--- a/drivers/net/cxgbe/cxgbe_ethdev.c
+++ b/drivers/net/cxgbe/cxgbe_ethdev.c
@@ -1296,12 +1296,13 @@ static int eth_cxgbe_dev_uninit(struct rte_eth_dev *eth_dev)  {
 	struct rte_pci_device *pci_dev = RTE_ETH_DEV_TO_PCI(eth_dev);
 	uint16_t port_id;
+	int err = 0;
 
 	/* Free up other ports and all resources */
 	RTE_ETH_FOREACH_DEV_OF(port_id, &pci_dev->device)
-		rte_eth_dev_close(port_id);
+		err |= rte_eth_dev_close(port_id);
 
-	return 0;
+	return err == 0 ? 0 : -EIO;
 }
 
 static int eth_cxgbe_pci_probe(struct rte_pci_driver *pci_drv __rte_unused, diff --git a/drivers/net/cxgbe/cxgbevf_ethdev.c b/drivers/net/cxgbe/cxgbevf_ethdev.c
index 947fcdd406..c2918f5356 100644
--- a/drivers/net/cxgbe/cxgbevf_ethdev.c
+++ b/drivers/net/cxgbe/cxgbevf_ethdev.c
@@ -183,12 +183,13 @@ static int eth_cxgbevf_dev_uninit(struct rte_eth_dev *eth_dev)  {
 	struct rte_pci_device *pci_dev = RTE_ETH_DEV_TO_PCI(eth_dev);
 	uint16_t port_id;
+	int err = 0;
 
 	/* Free up other ports and all resources */
 	RTE_ETH_FOREACH_DEV_OF(port_id, &pci_dev->device)
-		rte_eth_dev_close(port_id);
+		err |= rte_eth_dev_close(port_id);
 
-	return 0;
+	return err == 0 ? 0 : -EIO;
 }
 
 static int eth_cxgbevf_pci_probe(struct rte_pci_driver *pci_drv __rte_unused, diff --git a/drivers/net/failsafe/failsafe_ether.c b/drivers/net/failsafe/failsafe_ether.c
index 7c68bbdec0..950d35dac4 100644
--- a/drivers/net/failsafe/failsafe_ether.c
+++ b/drivers/net/failsafe/failsafe_ether.c
@@ -287,7 +287,11 @@ fs_dev_remove(struct sub_device *sdev)
 		/* fallthrough */
 	case DEV_ACTIVE:
 		failsafe_eth_dev_unregister_callbacks(sdev);
-		rte_eth_dev_close(PORT_ID(sdev));
+		ret = rte_eth_dev_close(PORT_ID(sdev));
+		if (ret < 0) {
+			ERROR("Port close failed for sub-device %u",
+			      PORT_ID(sdev));
+		}
 		sdev->state = DEV_PROBED;
 		/* fallthrough */
 	case DEV_PROBED:
diff --git a/drivers/net/failsafe/failsafe_ops.c b/drivers/net/failsafe/failsafe_ops.c
index 0ce7dfc8a6..79b24ec996 100644
--- a/drivers/net/failsafe/failsafe_ops.c
+++ b/drivers/net/failsafe/failsafe_ops.c
@@ -648,7 +648,10 @@ failsafe_eth_dev_close(struct rte_eth_dev *dev)
 	FOREACH_SUBDEV_STATE(sdev, i, dev, DEV_ACTIVE) {
 		DEBUG("Closing sub_device %d", i);
 		failsafe_eth_dev_unregister_callbacks(sdev);
-		rte_eth_dev_close(PORT_ID(sdev));
+		ret = rte_eth_dev_close(PORT_ID(sdev));
+		if (ret)
+			ERROR("Error while closing sub-device %u",
+					PORT_ID(sdev));
 		sdev->state = DEV_ACTIVE - 1;
 	}
 	rte_eth_dev_callback_unregister(RTE_ETH_ALL, RTE_ETH_EVENT_NEW, diff --git a/drivers/net/memif/rte_eth_memif.c b/drivers/net/memif/rte_eth_memif.c
index ff8a58081f..33bf5c68a3 100644
--- a/drivers/net/memif/rte_eth_memif.c
+++ b/drivers/net/memif/rte_eth_memif.c
@@ -1798,9 +1798,7 @@ rte_pmd_memif_remove(struct rte_vdev_device *vdev)
 	if (eth_dev == NULL)
 		return 0;
 
-	rte_eth_dev_close(eth_dev->data->port_id);
-
-	return 0;
+	return rte_eth_dev_close(eth_dev->data->port_id);
 }
 
 static struct rte_vdev_driver pmd_memif_drv = { diff --git a/drivers/net/mlx5/mlx5.c b/drivers/net/mlx5/mlx5.c index 01ead6e6af..2e2f0b274c 100644
--- a/drivers/net/mlx5/mlx5.c
+++ b/drivers/net/mlx5/mlx5.c
@@ -2010,6 +2010,7 @@ static int
 mlx5_pci_remove(struct rte_pci_device *pci_dev)  {
 	uint16_t port_id;
+	int ret = 0;
 
 	RTE_ETH_FOREACH_DEV_OF(port_id, &pci_dev->device) {
 		/*
@@ -2017,11 +2018,11 @@ mlx5_pci_remove(struct rte_pci_device *pci_dev)
 		 * call the close function explicitly for secondary process.
 		 */
 		if (rte_eal_process_type() == RTE_PROC_SECONDARY)
-			mlx5_dev_close(&rte_eth_devices[port_id]);
+			ret |= mlx5_dev_close(&rte_eth_devices[port_id]);
 		else
-			rte_eth_dev_close(port_id);
+			ret |= rte_eth_dev_close(port_id);
 	}
-	return 0;
+	return ret == 0 ? 0 : -EIO;
 }
 
 static const struct rte_pci_id mlx5_pci_id_map[] = { diff --git a/drivers/net/mvneta/mvneta_ethdev.c b/drivers/net/mvneta/mvneta_ethdev.c
index 607771149a..13d4b6af6b 100644
--- a/drivers/net/mvneta/mvneta_ethdev.c
+++ b/drivers/net/mvneta/mvneta_ethdev.c
@@ -964,14 +964,15 @@ static int
 rte_pmd_mvneta_remove(struct rte_vdev_device *vdev)  {
 	uint16_t port_id;
+	int ret = 0;
 
 	RTE_ETH_FOREACH_DEV(port_id) {
 		if (rte_eth_devices[port_id].device != &vdev->device)
 			continue;
-		rte_eth_dev_close(port_id);
+		ret = rte_eth_dev_close(port_id);
 	}
 
-	return 0;
+	return ret == 0 ? 0 : -EIO;
 }
 
 static struct rte_vdev_driver pmd_mvneta_drv = { diff --git a/drivers/net/mvpp2/mrvl_ethdev.c b/drivers/net/mvpp2/mrvl_ethdev.c index a230a96840..acc8c70a95 100644
--- a/drivers/net/mvpp2/mrvl_ethdev.c
+++ b/drivers/net/mvpp2/mrvl_ethdev.c
@@ -3022,14 +3022,15 @@ static int
 rte_pmd_mrvl_remove(struct rte_vdev_device *vdev)  {
 	uint16_t port_id;
+	int ret = 0;
 
 	RTE_ETH_FOREACH_DEV(port_id) {
 		if (rte_eth_devices[port_id].device != &vdev->device)
 			continue;
-		rte_eth_dev_close(port_id);
+		ret = rte_eth_dev_close(port_id);
 	}
 
-	return 0;
+	return ret == 0 ? 0 : -EIO;
 }
 
 static struct rte_vdev_driver pmd_mrvl_drv = { diff --git a/drivers/net/netvsc/hn_ethdev.c b/drivers/net/netvsc/hn_ethdev.c index 9af64821a1..3fa39dbebc 100644
--- a/drivers/net/netvsc/hn_ethdev.c
+++ b/drivers/net/netvsc/hn_ethdev.c
@@ -841,14 +841,16 @@ hn_dev_stop(struct rte_eth_dev *dev)  static int  hn_dev_close(struct rte_eth_dev *dev)  {
+	int ret;
+
 	PMD_INIT_FUNC_TRACE();
 	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
 		return 0;
 
-	hn_vf_close(dev);
+	ret = hn_vf_close(dev);
 	hn_dev_free_queues(dev);
 
-	return 0;
+	return ret;
 }
 
 static const struct eth_dev_ops hn_eth_dev_ops = { diff --git a/drivers/net/netvsc/hn_var.h b/drivers/net/netvsc/hn_var.h index 4b63f87607..74f30669ac 100644
--- a/drivers/net/netvsc/hn_var.h
+++ b/drivers/net/netvsc/hn_var.h
@@ -217,7 +217,7 @@ const uint32_t *hn_vf_supported_ptypes(struct rte_eth_dev *dev);
 int	hn_vf_start(struct rte_eth_dev *dev);
 void	hn_vf_reset(struct rte_eth_dev *dev);
 void	hn_vf_stop(struct rte_eth_dev *dev);
-void	hn_vf_close(struct rte_eth_dev *dev);
+int	hn_vf_close(struct rte_eth_dev *dev);
 
 int	hn_vf_allmulticast_enable(struct rte_eth_dev *dev);
 int	hn_vf_allmulticast_disable(struct rte_eth_dev *dev);
diff --git a/drivers/net/netvsc/hn_vf.c b/drivers/net/netvsc/hn_vf.c index f5f15c0462..d29eee7627 100644
--- a/drivers/net/netvsc/hn_vf.c
+++ b/drivers/net/netvsc/hn_vf.c
@@ -316,18 +316,21 @@ void hn_vf_reset(struct rte_eth_dev *dev)
 	VF_ETHDEV_FUNC(dev, rte_eth_dev_reset);  }
 
-void hn_vf_close(struct rte_eth_dev *dev)
+int hn_vf_close(struct rte_eth_dev *dev)
 {
 	struct hn_data *hv = dev->data->dev_private;
 	uint16_t vf_port;
+	int ret = 0;
 
 	rte_rwlock_read_lock(&hv->vf_lock);
 	vf_port = hv->vf_port;
 	if (vf_port != HN_INVALID_PORT)
-		rte_eth_dev_close(vf_port);
+		ret = rte_eth_dev_close(vf_port);
 
 	hv->vf_port = HN_INVALID_PORT;
 	rte_rwlock_read_unlock(&hv->vf_lock);
+
+	return ret;
 }
 
 int hn_vf_stats_reset(struct rte_eth_dev *dev) diff --git a/drivers/net/virtio/virtio_user_ethdev.c b/drivers/net/virtio/virtio_user_ethdev.c
index 87f6cb6950..865f73807f 100644
--- a/drivers/net/virtio/virtio_user_ethdev.c
+++ b/drivers/net/virtio/virtio_user_ethdev.c
@@ -813,9 +813,7 @@ virtio_user_pmd_remove(struct rte_vdev_device *vdev)
 		return rte_eth_dev_release_port(eth_dev);
 
 	/* make sure the device is stopped, queues freed */
-	rte_eth_dev_close(eth_dev->data->port_id);
-
-	return 0;
+	return rte_eth_dev_close(eth_dev->data->port_id);
 }
 
 static struct rte_vdev_driver virtio_user_driver = { diff --git a/lib/librte_ethdev/rte_ethdev.c b/lib/librte_ethdev/rte_ethdev.c index 5c0e8f170e..6091630a2f 100644
--- a/lib/librte_ethdev/rte_ethdev.c
+++ b/lib/librte_ethdev/rte_ethdev.c
@@ -1705,19 +1705,22 @@ rte_eth_dev_set_link_down(uint16_t port_id)
 	return eth_err(port_id, (*dev->dev_ops->dev_set_link_down)(dev));
 }
 
-void
+int
 rte_eth_dev_close(uint16_t port_id)
 {
 	struct rte_eth_dev *dev;
+	int ret;
 
-	RTE_ETH_VALID_PORTID_OR_RET(port_id);
+	RTE_ETH_VALID_PORTID_OR_ERR_RET(port_id, -EINVAL);
 	dev = &rte_eth_devices[port_id];
 
-	RTE_FUNC_PTR_OR_RET(*dev->dev_ops->dev_close);
-	(*dev->dev_ops->dev_close)(dev);
+	RTE_FUNC_PTR_OR_ERR_RET(*dev->dev_ops->dev_close, -ENOTSUP);
+	ret = (*dev->dev_ops->dev_close)(dev);
 
 	rte_ethdev_trace_close(port_id);
-	rte_eth_dev_release_port(dev);
+	ret = rte_eth_dev_release_port(dev);
+
+	return ret;
 }
 
 int
diff --git a/lib/librte_ethdev/rte_ethdev.h b/lib/librte_ethdev/rte_ethdev.h index 24d898ae89..4380d8ecf6 100644
--- a/lib/librte_ethdev/rte_ethdev.h
+++ b/lib/librte_ethdev/rte_ethdev.h
@@ -2281,8 +2281,11 @@ int rte_eth_dev_set_link_down(uint16_t port_id);
  *
  * @param port_id
  *   The port identifier of the Ethernet device.
+ * @return
+ *   - Zero if the port is closed successfully.
+ *   - Negative if something went wrong.
  */
-void rte_eth_dev_close(uint16_t port_id);
+int rte_eth_dev_close(uint16_t port_id);
 
 /**
  * Reset a Ethernet device and keep its port id.
--
2.28.0

[dpdk-dev] [PATCH v3 00/29] cleanup ethdev close operation

[Thomas Monjalon]

This is the end of a process started two years ago,
to have a close which reliably releases an ethdev port
without the need of removing the device (which can have more ports).

Unfortunately, some drivers might be broken because did not follow
the migration recommendations. We cannot wait more,
this should be merged before the 20.11-rc1 release.


v3:
  - integrate NXP patches for dpaa, dpaa2, enetc and pfe
  - fix crash in rte_eth_dev_release_port() because of early memset 0

v2 (big thanks to Ferruh for the reviews):
  - rebase on top of cxgbe updates
  - integrate updated patches for tap, bnx2x, qede
  - return 0 in atlantic PMD
  - fix failsafe mutex free
  - remove useless check in pcap
  - fix testpmd port state transition
  - add primary process check
  - reset more pointers in rte_eth_dev_release_port()
  - remove dev_started reset in rte_eth_dev_close()
  - return int from rte_eth_dev_close()

Note for future: fix old memory leak when releasing port,
i.e. free callbacks link_intr_cbs, post_rx_burst_cbs, pre_tx_burst_cbs.

The last 3 patches are optional changes to ethdev close/release
which were requested by Ferruh but may take longer to be accepted:
  - memset 0
  - keep start state
  - int return


Rasesh Mody (2):
  net/bnx2x: release port upon close
  net/qede: release port upon close

Sachin Saxena (4):
  net/dpaa: release port upon close
  net/dpaa2: release port upon close
  net/enetc: release port upon close
  net/pfe: release port upon close

Thomas Monjalon (22):
  ethdev: reset device and interrupt pointers on release
  ethdev: allow drivers to return error on close
  net/af_packet: release port upon close
  net/atlantic: release port upon close
  net/axgbe: release port upon close
  net/bonding: release port upon close
  net/failsafe: release port upon close
  net/mlx4: release port upon close
  net/null: release port upon close
  net/octeontx: release port upon close
  net/pcap: release port upon close
  net/ring: release port upon close
  net/softnic: release port upon close
  ethdev: remove old close behaviour
  drivers/net: accept removing device without any port
  drivers/net: check process type in close operation
  drivers/net: remove redundant MAC addresses freeing
  app/testpmd: reset port status on close notification
  app/testpmd: align behaviour of multi-port detach
  ethdev: remove forcing stopped state upon close
  ethdev: reset all when releasing a port
  ethdev: allow close function to return an error

Yunjian Wang (1):
  net/tap: release port upon close

 MAINTAINERS                               |   6 +-
 app/test-pmd/config.c                     |   7 +-
 app/test-pmd/testpmd.c                    |  53 +++------
 app/test/virtual_pmd.c                    |   6 +-
 doc/guides/rel_notes/deprecation.rst      |   7 --
 doc/guides/rel_notes/release_20_11.rst    |   4 +-
 drivers/net/af_packet/rte_eth_af_packet.c |  58 +++++-----
 drivers/net/af_xdp/rte_eth_af_xdp.c       |   9 +-
 drivers/net/ark/ark_ethdev.c              |  17 ++-
 drivers/net/atlantic/atl_ethdev.c         |  62 ++++-------
 drivers/net/avp/avp_ethdev.c              |  10 +-
 drivers/net/axgbe/axgbe_ethdev.c          |  17 +--
 drivers/net/bnx2x/bnx2x_ethdev.c          |  16 ++-
 drivers/net/bnxt/bnxt_ethdev.c            |  16 ++-
 drivers/net/bnxt/bnxt_reps.c              |  11 +-
 drivers/net/bnxt/bnxt_reps.h              |   2 +-
 drivers/net/bonding/eth_bond_private.h    |   2 +-
 drivers/net/bonding/rte_eth_bond_pmd.c    |  33 +++---
 drivers/net/cxgbe/cxgbe_ethdev.c          |  18 ++--
 drivers/net/cxgbe/cxgbe_main.c            |   2 -
 drivers/net/cxgbe/cxgbe_pfvf.h            |   2 +-
 drivers/net/cxgbe/cxgbevf_ethdev.c        |   5 +-
 drivers/net/cxgbe/cxgbevf_main.c          |   2 -
 drivers/net/dpaa/dpaa_ethdev.c            | 124 ++++++++++------------
 drivers/net/dpaa2/dpaa2_ethdev.c          |  96 +++++++----------
 drivers/net/e1000/em_ethdev.c             |  18 ++--
 drivers/net/e1000/igb_ethdev.c            |  36 +++----
 drivers/net/ena/ena_ethdev.c              |  20 ++--
 drivers/net/enetc/enetc_ethdev.c          |  17 +--
 drivers/net/enic/enic_ethdev.c            |   9 +-
 drivers/net/enic/enic_vf_representor.c    |   8 +-
 drivers/net/failsafe/failsafe.c           |  24 +----
 drivers/net/failsafe/failsafe_ether.c     |   6 +-
 drivers/net/failsafe/failsafe_ops.c       |  67 ++++++++----
 drivers/net/failsafe/failsafe_private.h   |   1 +
 drivers/net/fm10k/fm10k_ethdev.c          |  22 +---
 drivers/net/hinic/hinic_pmd_ethdev.c      |  22 ++--
 drivers/net/hns3/hns3_ethdev.c            |  15 +--
 drivers/net/hns3/hns3_ethdev_vf.c         |  16 +--
 drivers/net/i40e/i40e_ethdev.c            |  16 +--
 drivers/net/i40e/i40e_ethdev_vf.c         |  17 ++-
 drivers/net/iavf/iavf_ethdev.c            |  18 ++--
 drivers/net/ice/ice_dcf_ethdev.c          |  12 +--
 drivers/net/ice/ice_ethdev.c              |  21 ++--
 drivers/net/igc/igc_ethdev.c              |  17 ++-
 drivers/net/ionic/ionic_ethdev.c          |  17 ++-
 drivers/net/ipn3ke/ipn3ke_ethdev.c        |   6 +-
 drivers/net/ipn3ke/ipn3ke_representor.c   |   7 +-
 drivers/net/ixgbe/ixgbe_ethdev.c          |  33 ++----
 drivers/net/kni/rte_eth_kni.c             |  25 ++---
 drivers/net/liquidio/lio_ethdev.c         |  11 +-
 drivers/net/memif/rte_eth_memif.c         |  11 +-
 drivers/net/mlx4/mlx4.c                   |   7 +-
 drivers/net/mlx5/linux/mlx5_os.c          |   2 -
 drivers/net/mlx5/mlx5.c                   |  16 +--
 drivers/net/mlx5/mlx5.h                   |   2 +-
 drivers/net/mvneta/mvneta_ethdev.c        |  15 +--
 drivers/net/mvpp2/mrvl_ethdev.c           |  15 +--
 drivers/net/netvsc/hn_ethdev.c            |  19 ++--
 drivers/net/netvsc/hn_var.h               |   2 +-
 drivers/net/netvsc/hn_vf.c                |   7 +-
 drivers/net/nfb/nfb_ethdev.c              |  11 +-
 drivers/net/nfp/nfp_net.c                 |  11 +-
 drivers/net/null/rte_eth_null.c           |  26 +++--
 drivers/net/octeontx/octeontx_ethdev.c    |  17 ++-
 drivers/net/octeontx2/otx2_ethdev.c       |   8 +-
 drivers/net/pcap/rte_eth_pcap.c           |  31 +++---
 drivers/net/pfe/pfe_ethdev.c              |  51 ++++-----
 drivers/net/qede/qede_ethdev.c            |  20 ++--
 drivers/net/ring/rte_eth_ring.c           |  53 +++++----
 drivers/net/sfc/sfc_ethdev.c              |  35 +++---
 drivers/net/softnic/rte_eth_softnic.c     |  67 ++++++------
 drivers/net/szedata2/rte_eth_szedata2.c   |  23 ++--
 drivers/net/tap/rte_eth_tap.c             |  52 +++++----
 drivers/net/thunderx/nicvf_ethdev.c       |  11 +-
 drivers/net/vhost/rte_eth_vhost.c         |  15 +--
 drivers/net/virtio/virtio_ethdev.c        |  17 ++-
 drivers/net/virtio/virtio_user_ethdev.c   |   4 +-
 drivers/net/vmxnet3/vmxnet3_ethdev.c      |  16 ++-
 lib/librte_ethdev/rte_ethdev.c            |  32 ++----
 lib/librte_ethdev/rte_ethdev.h            |  13 +--
 lib/librte_ethdev/rte_ethdev_driver.h     |   2 +-
 lib/librte_ethdev/rte_ethdev_pci.h        |  14 +--
 83 files changed, 763 insertions(+), 928 deletions(-)

-- 
2.28.0

[dpdk-dev] [PATCH v3 01/29] ethdev: reset device and interrupt pointers on release

[Thomas Monjalon]

The pointers .device and .intr_handle were already reset by the helper
rte_eth_dev_pci_generic_remove().
It is now made part of rte_eth_dev_release_port().

It makes rte_eth_dev_pci_release() meaningless,
so it is replaced with a call to rte_eth_dev_release_port().

Signed-off-by: Thomas Monjalon <thomas@monjalon.net>
Reviewed-by: Ferruh Yigit <ferruh.yigit@intel.com>
---
 drivers/net/ark/ark_ethdev.c            |  2 +-
 drivers/net/octeontx2/otx2_ethdev.c     |  2 +-
 drivers/net/szedata2/rte_eth_szedata2.c |  6 +++---
 lib/librte_ethdev/rte_ethdev.c          |  2 ++
 lib/librte_ethdev/rte_ethdev_pci.h      | 14 ++------------
 5 files changed, 9 insertions(+), 17 deletions(-)

diff --git a/drivers/net/ark/ark_ethdev.c b/drivers/net/ark/ark_ethdev.c
index d472ae5643..e13415e95b 100644
--- a/drivers/net/ark/ark_ethdev.c
+++ b/drivers/net/ark/ark_ethdev.c
@@ -105,7 +105,7 @@ eth_ark_pci_probe(struct rte_pci_driver *pci_drv __rte_unused,
 
 	ret = eth_ark_dev_init(eth_dev);
 	if (ret)
-		rte_eth_dev_pci_release(eth_dev);
+		rte_eth_dev_release_port(eth_dev);
 
 	return ret;
 }
diff --git a/drivers/net/octeontx2/otx2_ethdev.c b/drivers/net/octeontx2/otx2_ethdev.c
index c06e32f26b..51a6f99877 100644
--- a/drivers/net/octeontx2/otx2_ethdev.c
+++ b/drivers/net/octeontx2/otx2_ethdev.c
@@ -2668,7 +2668,7 @@ nix_remove(struct rte_pci_device *pci_dev)
 		if (rc)
 			return rc;
 
-		rte_eth_dev_pci_release(eth_dev);
+		rte_eth_dev_release_port(eth_dev);
 	}
 
 	/* Nothing to be done for secondary processes */
diff --git a/drivers/net/szedata2/rte_eth_szedata2.c b/drivers/net/szedata2/rte_eth_szedata2.c
index 30c888cd96..a17c53577c 100644
--- a/drivers/net/szedata2/rte_eth_szedata2.c
+++ b/drivers/net/szedata2/rte_eth_szedata2.c
@@ -1802,7 +1802,7 @@ szedata2_eth_dev_release_interval(struct rte_eth_dev **eth_devs,
 
 	for (i = from; i < to; i++) {
 		rte_szedata2_eth_dev_uninit(eth_devs[i]);
-		rte_eth_dev_pci_release(eth_devs[i]);
+		rte_eth_dev_release_port(eth_devs[i]);
 	}
 }
 
@@ -1853,7 +1853,7 @@ static int szedata2_eth_pci_probe(struct rte_pci_driver *pci_drv __rte_unused,
 		if (ret != 0) {
 			PMD_INIT_LOG(ERR, "Failed to init eth_dev for port %u",
 					i);
-			rte_eth_dev_pci_release(eth_devs[i]);
+			rte_eth_dev_release_port(eth_devs[i]);
 			szedata2_eth_dev_release_interval(eth_devs, 0, i);
 			rte_free(list_entry);
 			return ret;
@@ -1922,7 +1922,7 @@ static int szedata2_eth_pci_remove(struct rte_pci_device *pci_dev)
 			retval = retval ? retval : ret;
 		}
 
-		rte_eth_dev_pci_release(eth_dev);
+		rte_eth_dev_release_port(eth_dev);
 	}
 
 	return retval;
diff --git a/lib/librte_ethdev/rte_ethdev.c b/lib/librte_ethdev/rte_ethdev.c
index dfe5c1b488..d12d74dd8c 100644
--- a/lib/librte_ethdev/rte_ethdev.c
+++ b/lib/librte_ethdev/rte_ethdev.c
@@ -555,6 +555,8 @@ rte_eth_dev_release_port(struct rte_eth_dev *eth_dev)
 	rte_spinlock_lock(&rte_eth_dev_shared_data->ownership_lock);
 
 	eth_dev->state = RTE_ETH_DEV_UNUSED;
+	eth_dev->device = NULL;
+	eth_dev->intr_handle = NULL;
 
 	if (rte_eal_process_type() == RTE_PROC_PRIMARY) {
 		rte_free(eth_dev->data->rx_queues);
diff --git a/lib/librte_ethdev/rte_ethdev_pci.h b/lib/librte_ethdev/rte_ethdev_pci.h
index 9c483468ba..bf715896ae 100644
--- a/lib/librte_ethdev/rte_ethdev_pci.h
+++ b/lib/librte_ethdev/rte_ethdev_pci.h
@@ -107,16 +107,6 @@ rte_eth_dev_pci_allocate(struct rte_pci_device *dev, size_t private_data_size)
 	return eth_dev;
 }
 
-static inline void
-rte_eth_dev_pci_release(struct rte_eth_dev *eth_dev)
-{
-	eth_dev->device = NULL;
-	eth_dev->intr_handle = NULL;
-
-	/* free ether device */
-	rte_eth_dev_release_port(eth_dev);
-}
-
 typedef int (*eth_dev_pci_callback_t)(struct rte_eth_dev *eth_dev);
 
 /**
@@ -138,7 +128,7 @@ rte_eth_dev_pci_generic_probe(struct rte_pci_device *pci_dev,
 	RTE_FUNC_PTR_OR_ERR_RET(*dev_init, -EINVAL);
 	ret = dev_init(eth_dev);
 	if (ret)
-		rte_eth_dev_pci_release(eth_dev);
+		rte_eth_dev_release_port(eth_dev);
 	else
 		rte_eth_dev_probing_finish(eth_dev);
 
@@ -167,7 +157,7 @@ rte_eth_dev_pci_generic_remove(struct rte_pci_device *pci_dev,
 			return ret;
 	}
 
-	rte_eth_dev_pci_release(eth_dev);
+	rte_eth_dev_release_port(eth_dev);
 	return 0;
 }
 
-- 
2.28.0

Re: [dpdk-dev] [PATCH v3 01/29] ethdev: reset device and interrupt pointers on release

[Andrew Rybchenko]

On 9/29/20 2:14 AM, Thomas Monjalon wrote:
> The pointers .device and .intr_handle were already reset by the helper
> rte_eth_dev_pci_generic_remove().
> It is now made part of rte_eth_dev_release_port().
> 
> It makes rte_eth_dev_pci_release() meaningless,
> so it is replaced with a call to rte_eth_dev_release_port().
> 
> Signed-off-by: Thomas Monjalon <thomas@monjalon.net>
> Reviewed-by: Ferruh Yigit <ferruh.yigit@intel.com>

Reviewed-by: Andrew Rybchenko <arybchenko@solarflare.com>

[dpdk-dev] [PATCH v3 02/29] ethdev: allow drivers to return error on close

[Thomas Monjalon]

The device operation .dev_close was returning void.
This driver interface is changed to return an int.

Note that the API rte_eth_dev_close() is still returning void,
although a deprecation notice is pending to change it as well.

Signed-off-by: Thomas Monjalon <thomas@monjalon.net>
Reviewed-by: Rosen Xu <rosen.xu@intel.com>
Reviewed-by: Sachin Saxena <sachin.saxena@oss.nxp.com>
Reviewed-by: Liron Himi <lironh@marvell.com>
---
 app/test/virtual_pmd.c                    |  6 ++++--
 drivers/net/af_packet/rte_eth_af_packet.c |  3 ++-
 drivers/net/af_xdp/rte_eth_af_xdp.c       |  4 +++-
 drivers/net/ark/ark_ethdev.c              |  6 ++++--
 drivers/net/atlantic/atl_ethdev.c         |  6 ++++--
 drivers/net/avp/avp_ethdev.c              |  5 +++--
 drivers/net/axgbe/axgbe_ethdev.c          |  5 +++--
 drivers/net/bnx2x/bnx2x_ethdev.c          |  4 +++-
 drivers/net/bnxt/bnxt_ethdev.c            |  4 +++-
 drivers/net/bnxt/bnxt_reps.c              |  3 ++-
 drivers/net/bnxt/bnxt_reps.h              |  2 +-
 drivers/net/bonding/eth_bond_private.h    |  2 +-
 drivers/net/bonding/rte_eth_bond_pmd.c    |  4 +++-
 drivers/net/cxgbe/cxgbe_ethdev.c          | 10 ++++++----
 drivers/net/cxgbe/cxgbe_pfvf.h            |  2 +-
 drivers/net/dpaa/dpaa_ethdev.c            |  4 +++-
 drivers/net/dpaa2/dpaa2_ethdev.c          |  6 ++++--
 drivers/net/e1000/em_ethdev.c             |  6 ++++--
 drivers/net/e1000/igb_ethdev.c            | 12 ++++++++----
 drivers/net/ena/ena_ethdev.c              |  6 ++++--
 drivers/net/enetc/enetc_ethdev.c          |  4 +++-
 drivers/net/enic/enic_ethdev.c            |  4 +++-
 drivers/net/enic/enic_vf_representor.c    |  5 +++--
 drivers/net/failsafe/failsafe_ops.c       |  3 ++-
 drivers/net/fm10k/fm10k_ethdev.c          |  4 +++-
 drivers/net/hinic/hinic_pmd_ethdev.c      |  6 ++++--
 drivers/net/hns3/hns3_ethdev.c            |  6 ++++--
 drivers/net/hns3/hns3_ethdev_vf.c         |  6 ++++--
 drivers/net/i40e/i40e_ethdev.c            |  5 +++--
 drivers/net/i40e/i40e_ethdev_vf.c         |  5 +++--
 drivers/net/iavf/iavf_ethdev.c            |  6 ++++--
 drivers/net/ice/ice_dcf_ethdev.c          |  6 ++++--
 drivers/net/ice/ice_ethdev.c              |  6 ++++--
 drivers/net/igc/igc_ethdev.c              |  6 ++++--
 drivers/net/ionic/ionic_ethdev.c          | 10 ++++++----
 drivers/net/ipn3ke/ipn3ke_representor.c   |  4 +++-
 drivers/net/ixgbe/ixgbe_ethdev.c          | 11 +++++++----
 drivers/net/kni/rte_eth_kni.c             |  4 +++-
 drivers/net/liquidio/lio_ethdev.c         |  4 +++-
 drivers/net/memif/rte_eth_memif.c         |  4 +++-
 drivers/net/mlx4/mlx4.c                   |  3 ++-
 drivers/net/mlx5/mlx5.c                   |  9 +++++----
 drivers/net/mlx5/mlx5.h                   |  2 +-
 drivers/net/mvneta/mvneta_ethdev.c        |  4 +++-
 drivers/net/mvpp2/mrvl_ethdev.c           |  4 +++-
 drivers/net/netvsc/hn_ethdev.c            |  4 +++-
 drivers/net/nfb/nfb_ethdev.c              |  4 +++-
 drivers/net/nfp/nfp_net.c                 |  6 ++++--
 drivers/net/octeontx/octeontx_ethdev.c    |  4 +++-
 drivers/net/octeontx2/otx2_ethdev.c       |  5 +++--
 drivers/net/pcap/rte_eth_pcap.c           |  3 ++-
 drivers/net/pfe/pfe_ethdev.c              |  8 +++++---
 drivers/net/qede/qede_ethdev.c            |  4 +++-
 drivers/net/sfc/sfc_ethdev.c              |  4 +++-
 drivers/net/softnic/rte_eth_softnic.c     |  4 ++--
 drivers/net/szedata2/rte_eth_szedata2.c   |  4 +++-
 drivers/net/tap/rte_eth_tap.c             |  4 +++-
 drivers/net/thunderx/nicvf_ethdev.c       |  4 +++-
 drivers/net/vhost/rte_eth_vhost.c         |  6 ++++--
 drivers/net/virtio/virtio_ethdev.c        |  6 ++++--
 drivers/net/vmxnet3/vmxnet3_ethdev.c      |  6 ++++--
 lib/librte_ethdev/rte_ethdev_driver.h     |  2 +-
 62 files changed, 206 insertions(+), 103 deletions(-)

diff --git a/app/test/virtual_pmd.c b/app/test/virtual_pmd.c
index b49089a33c..4bd4d1c9ad 100644
--- a/app/test/virtual_pmd.c
+++ b/app/test/virtual_pmd.c
@@ -62,9 +62,11 @@ static void  virtual_ethdev_stop(struct rte_eth_dev *eth_dev __rte_unused)
 		rte_pktmbuf_free(pkt);
 }
 
-static void
+static int
 virtual_ethdev_close(struct rte_eth_dev *dev __rte_unused)
-{}
+{
+	return 0;
+}
 
 static int
 virtual_ethdev_configure_success(struct rte_eth_dev *dev __rte_unused)
diff --git a/drivers/net/af_packet/rte_eth_af_packet.c b/drivers/net/af_packet/rte_eth_af_packet.c
index b9723e9619..7d0ff1cbb3 100644
--- a/drivers/net/af_packet/rte_eth_af_packet.c
+++ b/drivers/net/af_packet/rte_eth_af_packet.c
@@ -376,9 +376,10 @@ eth_stats_reset(struct rte_eth_dev *dev)
 	return 0;
 }
 
-static void
+static int
 eth_dev_close(struct rte_eth_dev *dev __rte_unused)
 {
+	return 0;
 }
 
 static void
diff --git a/drivers/net/af_xdp/rte_eth_af_xdp.c b/drivers/net/af_xdp/rte_eth_af_xdp.c
index b65ee449fc..badbce63fb 100644
--- a/drivers/net/af_xdp/rte_eth_af_xdp.c
+++ b/drivers/net/af_xdp/rte_eth_af_xdp.c
@@ -701,7 +701,7 @@ xdp_umem_destroy(struct xsk_umem_info *umem)
 	umem = NULL;
 }
 
-static void
+static int
 eth_dev_close(struct rte_eth_dev *dev)
 {
 	struct pmd_internals *internals = dev->data->dev_private;
@@ -731,6 +731,8 @@ eth_dev_close(struct rte_eth_dev *dev)
 	dev->data->mac_addrs = NULL;
 
 	remove_xdp_program(internals);
+
+	return 0;
 }
 
 static void
diff --git a/drivers/net/ark/ark_ethdev.c b/drivers/net/ark/ark_ethdev.c
index e13415e95b..5f2ed4b7b7 100644
--- a/drivers/net/ark/ark_ethdev.c
+++ b/drivers/net/ark/ark_ethdev.c
@@ -30,7 +30,7 @@ static int eth_ark_dev_uninit(struct rte_eth_dev *eth_dev);
 static int eth_ark_dev_configure(struct rte_eth_dev *dev);
 static int eth_ark_dev_start(struct rte_eth_dev *dev);
 static void eth_ark_dev_stop(struct rte_eth_dev *dev);
-static void eth_ark_dev_close(struct rte_eth_dev *dev);
+static int eth_ark_dev_close(struct rte_eth_dev *dev);
 static int eth_ark_dev_info_get(struct rte_eth_dev *dev,
 				struct rte_eth_dev_info *dev_info);
 static int eth_ark_dev_link_update(struct rte_eth_dev *dev,
@@ -676,7 +676,7 @@ eth_ark_dev_stop(struct rte_eth_dev *dev)
 	}
 }
 
-static void
+static int
 eth_ark_dev_close(struct rte_eth_dev *dev)
 {
 	struct ark_adapter *ark = dev->data->dev_private;
@@ -706,6 +706,8 @@ eth_ark_dev_close(struct rte_eth_dev *dev)
 
 	rte_free(dev->data->mac_addrs);
 	dev->data->mac_addrs = 0;
+
+	return 0;
 }
 
 static int
diff --git a/drivers/net/atlantic/atl_ethdev.c b/drivers/net/atlantic/atl_ethdev.c
index 2fca9140d7..254758fd66 100644
--- a/drivers/net/atlantic/atl_ethdev.c
+++ b/drivers/net/atlantic/atl_ethdev.c
@@ -22,7 +22,7 @@ static int  atl_dev_start(struct rte_eth_dev *dev);
 static void atl_dev_stop(struct rte_eth_dev *dev);
 static int  atl_dev_set_link_up(struct rte_eth_dev *dev);
 static int  atl_dev_set_link_down(struct rte_eth_dev *dev);
-static void atl_dev_close(struct rte_eth_dev *dev);
+static int  atl_dev_close(struct rte_eth_dev *dev);
 static int  atl_dev_reset(struct rte_eth_dev *dev);
 static int  atl_dev_promiscuous_enable(struct rte_eth_dev *dev);
 static int  atl_dev_promiscuous_disable(struct rte_eth_dev *dev);
@@ -719,7 +719,7 @@ atl_dev_set_link_down(struct rte_eth_dev *dev)
 /*
  * Reset and stop device.
  */
-static void
+static int
 atl_dev_close(struct rte_eth_dev *dev)
 {
 	PMD_INIT_FUNC_TRACE();
@@ -727,6 +727,8 @@ atl_dev_close(struct rte_eth_dev *dev)
 	atl_dev_stop(dev);
 
 	atl_free_queues(dev);
+
+	return 0;
 }
 
 static int
diff --git a/drivers/net/avp/avp_ethdev.c b/drivers/net/avp/avp_ethdev.c
index 5b47f0924a..2eed6acc3f 100644
--- a/drivers/net/avp/avp_ethdev.c
+++ b/drivers/net/avp/avp_ethdev.c
@@ -38,7 +38,7 @@ static int avp_dev_create(struct rte_pci_device *pci_dev,
 static int avp_dev_configure(struct rte_eth_dev *dev);
 static int avp_dev_start(struct rte_eth_dev *dev);
 static void avp_dev_stop(struct rte_eth_dev *dev);
-static void avp_dev_close(struct rte_eth_dev *dev);
+static int avp_dev_close(struct rte_eth_dev *dev);
 static int avp_dev_info_get(struct rte_eth_dev *dev,
 			    struct rte_eth_dev_info *dev_info);
 static int avp_vlan_offload_set(struct rte_eth_dev *dev, int mask);
@@ -2103,7 +2103,7 @@ avp_dev_stop(struct rte_eth_dev *eth_dev)
 	rte_spinlock_unlock(&avp->lock);
 }
 
-static void
+static int
 avp_dev_close(struct rte_eth_dev *eth_dev)
 {
 	struct avp_dev *avp = AVP_DEV_PRIVATE_TO_HW(eth_dev->data->dev_private);
@@ -2139,6 +2139,7 @@ avp_dev_close(struct rte_eth_dev *eth_dev)
 
 unlock:
 	rte_spinlock_unlock(&avp->lock);
+	return 0;
 }
 
 static int
diff --git a/drivers/net/axgbe/axgbe_ethdev.c b/drivers/net/axgbe/axgbe_ethdev.c
index 4f5a014b54..458d64bcef 100644
--- a/drivers/net/axgbe/axgbe_ethdev.c
+++ b/drivers/net/axgbe/axgbe_ethdev.c
@@ -16,7 +16,7 @@ static int  axgbe_dev_configure(struct rte_eth_dev *dev);
 static int  axgbe_dev_start(struct rte_eth_dev *dev);
 static void axgbe_dev_stop(struct rte_eth_dev *dev);
 static void axgbe_dev_interrupt_handler(void *param);
-static void axgbe_dev_close(struct rte_eth_dev *dev);
+static int axgbe_dev_close(struct rte_eth_dev *dev);
 static int axgbe_dev_promiscuous_enable(struct rte_eth_dev *dev);
 static int axgbe_dev_promiscuous_disable(struct rte_eth_dev *dev);
 static int axgbe_dev_allmulticast_enable(struct rte_eth_dev *dev);
@@ -410,10 +410,11 @@ axgbe_dev_stop(struct rte_eth_dev *dev)
 }
 
 /* Clear all resources like TX/RX queues. */
-static void
+static int
 axgbe_dev_close(struct rte_eth_dev *dev)
 {
 	axgbe_dev_clear_queues(dev);
+	return 0;
 }
 
 static int
diff --git a/drivers/net/bnx2x/bnx2x_ethdev.c b/drivers/net/bnx2x/bnx2x_ethdev.c
index b2ea5fafa7..73217c2138 100644
--- a/drivers/net/bnx2x/bnx2x_ethdev.c
+++ b/drivers/net/bnx2x/bnx2x_ethdev.c
@@ -280,7 +280,7 @@ bnx2x_dev_stop(struct rte_eth_dev *dev)
 	return;
 }
 
-static void
+static int
 bnx2x_dev_close(struct rte_eth_dev *dev)
 {
 	struct bnx2x_softc *sc = dev->data->dev_private;
@@ -295,6 +295,8 @@ bnx2x_dev_close(struct rte_eth_dev *dev)
 
 	/* free ilt */
 	bnx2x_free_ilt_mem(sc);
+
+	return 0;
 }
 
 static int
diff --git a/drivers/net/bnxt/bnxt_ethdev.c b/drivers/net/bnxt/bnxt_ethdev.c
index 05e9a6abbf..e726d078ec 100644
--- a/drivers/net/bnxt/bnxt_ethdev.c
+++ b/drivers/net/bnxt/bnxt_ethdev.c
@@ -1357,7 +1357,7 @@ static void bnxt_dev_stop_op(struct rte_eth_dev *eth_dev)
 		bp->flow_stat->flow_count = 0;
 }
 
-static void bnxt_dev_close_op(struct rte_eth_dev *eth_dev)
+static int bnxt_dev_close_op(struct rte_eth_dev *eth_dev)
 {
 	struct bnxt *bp = eth_dev->data->dev_private;
 
@@ -1392,6 +1392,8 @@ static void bnxt_dev_close_op(struct rte_eth_dev *eth_dev)
 
 	rte_free(bp->grp_info);
 	bp->grp_info = NULL;
+
+	return 0;
 }
 
 static void bnxt_mac_addr_remove_op(struct rte_eth_dev *eth_dev,
diff --git a/drivers/net/bnxt/bnxt_reps.c b/drivers/net/bnxt/bnxt_reps.c
index 17010f1ee2..2b6c0a277b 100644
--- a/drivers/net/bnxt/bnxt_reps.c
+++ b/drivers/net/bnxt/bnxt_reps.c
@@ -457,10 +457,11 @@ void bnxt_vf_rep_dev_stop_op(struct rte_eth_dev *eth_dev)
 	bnxt_vf_rep_free_rx_mbufs(vfr_bp);
 }
 
-void bnxt_vf_rep_dev_close_op(struct rte_eth_dev *eth_dev)
+int bnxt_vf_rep_dev_close_op(struct rte_eth_dev *eth_dev)
 {
 	BNXT_TF_DBG(DEBUG, "BNXT Port:%d VFR close\n", eth_dev->data->port_id);
 	bnxt_vf_representor_uninit(eth_dev);
+	return 0;
 }
 
 int bnxt_vf_rep_dev_info_get_op(struct rte_eth_dev *eth_dev,
diff --git a/drivers/net/bnxt/bnxt_reps.h b/drivers/net/bnxt/bnxt_reps.h
index 3239e03fc0..d9b9ee2757 100644
--- a/drivers/net/bnxt/bnxt_reps.h
+++ b/drivers/net/bnxt/bnxt_reps.h
@@ -45,7 +45,7 @@ int bnxt_vf_rep_tx_queue_setup_op(struct rte_eth_dev *eth_dev,
 void bnxt_vf_rep_rx_queue_release_op(void *rx_queue);
 void bnxt_vf_rep_tx_queue_release_op(void *tx_queue);
 void bnxt_vf_rep_dev_stop_op(struct rte_eth_dev *eth_dev);
-void bnxt_vf_rep_dev_close_op(struct rte_eth_dev *eth_dev);
+int bnxt_vf_rep_dev_close_op(struct rte_eth_dev *eth_dev);
 int bnxt_vf_rep_stats_get_op(struct rte_eth_dev *eth_dev,
 			     struct rte_eth_stats *stats);
 int bnxt_vf_rep_stats_reset_op(struct rte_eth_dev *eth_dev);
diff --git a/drivers/net/bonding/eth_bond_private.h b/drivers/net/bonding/eth_bond_private.h
index c9b2d0fe46..0a0034705d 100644
--- a/drivers/net/bonding/eth_bond_private.h
+++ b/drivers/net/bonding/eth_bond_private.h
@@ -318,7 +318,7 @@ bond_tlb_activate_slave(struct bond_dev_private *internals);
 void
 bond_ethdev_stop(struct rte_eth_dev *eth_dev);
 
-void
+int
 bond_ethdev_close(struct rte_eth_dev *dev);
 
 #endif
diff --git a/drivers/net/bonding/rte_eth_bond_pmd.c b/drivers/net/bonding/rte_eth_bond_pmd.c
index 4ffd2ca4bb..dff835ef92 100644
--- a/drivers/net/bonding/rte_eth_bond_pmd.c
+++ b/drivers/net/bonding/rte_eth_bond_pmd.c
@@ -2095,7 +2095,7 @@ bond_ethdev_stop(struct rte_eth_dev *eth_dev)
 	}
 }
 
-void
+int
 bond_ethdev_close(struct rte_eth_dev *dev)
 {
 	struct bond_dev_private *internals = dev->data->dev_private;
@@ -2119,6 +2119,8 @@ bond_ethdev_close(struct rte_eth_dev *dev)
 	bond_flow_ops.flush(dev, &ferror);
 	bond_ethdev_free_queues(dev);
 	rte_bitmap_reset(internals->vlan_filter_bmp);
+
+	return 0;
 }
 
 /* forward declaration */
diff --git a/drivers/net/cxgbe/cxgbe_ethdev.c b/drivers/net/cxgbe/cxgbe_ethdev.c
index 422e190daf..e4bbba5c32 100644
--- a/drivers/net/cxgbe/cxgbe_ethdev.c
+++ b/drivers/net/cxgbe/cxgbe_ethdev.c
@@ -318,7 +318,7 @@ int cxgbe_dev_mtu_set(struct rte_eth_dev *eth_dev, uint16_t mtu)
 /*
  * Stop device.
  */
-void cxgbe_dev_close(struct rte_eth_dev *eth_dev)
+int cxgbe_dev_close(struct rte_eth_dev *eth_dev)
 {
 	struct port_info *temp_pi, *pi = eth_dev->data->dev_private;
 	struct adapter *adapter = pi->adapter;
@@ -327,10 +327,10 @@ void cxgbe_dev_close(struct rte_eth_dev *eth_dev)
 	CXGBE_FUNC_TRACE();
 
 	if (!(adapter->flags & FULL_INIT_DONE))
-		return;
+		return 0;
 
 	if (!pi->viid)
-		return;
+		return 0;
 
 	cxgbe_down(pi);
 	t4_sge_eth_release_queues(pi);
@@ -343,11 +343,13 @@ void cxgbe_dev_close(struct rte_eth_dev *eth_dev)
 	for_each_port(adapter, i) {
 		temp_pi = adap2pinfo(adapter, i);
 		if (temp_pi->viid)
-			return;
+			return 0;
 	}
 
 	cxgbe_close(adapter);
 	rte_free(adapter);
+
+	return 0;
 }
 
 /* Start the device.
diff --git a/drivers/net/cxgbe/cxgbe_pfvf.h b/drivers/net/cxgbe/cxgbe_pfvf.h
index 0b7c52aecf..69d91639e9 100644
--- a/drivers/net/cxgbe/cxgbe_pfvf.h
+++ b/drivers/net/cxgbe/cxgbe_pfvf.h
@@ -19,7 +19,7 @@
 void cxgbe_dev_rx_queue_release(void *q);
 void cxgbe_dev_tx_queue_release(void *q);
 void cxgbe_dev_stop(struct rte_eth_dev *eth_dev);
-void cxgbe_dev_close(struct rte_eth_dev *eth_dev);
+int cxgbe_dev_close(struct rte_eth_dev *eth_dev);
 int cxgbe_dev_info_get(struct rte_eth_dev *eth_dev,
 		       struct rte_eth_dev_info *device_info);
 int cxgbe_dev_promiscuous_enable(struct rte_eth_dev *eth_dev);
diff --git a/drivers/net/dpaa/dpaa_ethdev.c b/drivers/net/dpaa/dpaa_ethdev.c
index 8a6b19b5ce..7fe06e1830 100644
--- a/drivers/net/dpaa/dpaa_ethdev.c
+++ b/drivers/net/dpaa/dpaa_ethdev.c
@@ -369,7 +369,7 @@ static void dpaa_eth_dev_stop(struct rte_eth_dev *dev)
 	dev->tx_pkt_burst = dpaa_eth_tx_drop_all;
 }
 
-static void dpaa_eth_dev_close(struct rte_eth_dev *dev)
+static int dpaa_eth_dev_close(struct rte_eth_dev *dev)
 {
 	struct fman_if *fif = dev->process_private;
 	struct __fman_if *__fif;
@@ -392,6 +392,8 @@ static void dpaa_eth_dev_close(struct rte_eth_dev *dev)
 					     dpaa_interrupt_handler,
 					     (void *)dev);
 	}
+
+	return 0;
 }
 
 static int
diff --git a/drivers/net/dpaa2/dpaa2_ethdev.c b/drivers/net/dpaa2/dpaa2_ethdev.c
index 02daa4d250..fb4165fa03 100644
--- a/drivers/net/dpaa2/dpaa2_ethdev.c
+++ b/drivers/net/dpaa2/dpaa2_ethdev.c
@@ -1236,7 +1236,7 @@ dpaa2_dev_stop(struct rte_eth_dev *dev)
 	rte_eth_linkstatus_set(dev, &link);
 }
 
-static void
+static int
 dpaa2_dev_close(struct rte_eth_dev *dev)
 {
 	struct dpaa2_dev_priv *priv = dev->data->dev_private;
@@ -1252,11 +1252,13 @@ dpaa2_dev_close(struct rte_eth_dev *dev)
 	ret = dpni_reset(dpni, CMD_PRI_LOW, priv->token);
 	if (ret) {
 		DPAA2_PMD_ERR("Failure cleaning dpni device: err=%d", ret);
-		return;
+		return -1;
 	}
 
 	memset(&link, 0, sizeof(link));
 	rte_eth_linkstatus_set(dev, &link);
+
+	return 0;
 }
 
 static int
diff --git a/drivers/net/e1000/em_ethdev.c b/drivers/net/e1000/em_ethdev.c
index 1dc360713a..2d0e071448 100644
--- a/drivers/net/e1000/em_ethdev.c
+++ b/drivers/net/e1000/em_ethdev.c
@@ -34,7 +34,7 @@
 static int eth_em_configure(struct rte_eth_dev *dev);
 static int eth_em_start(struct rte_eth_dev *dev);
 static void eth_em_stop(struct rte_eth_dev *dev);
-static void eth_em_close(struct rte_eth_dev *dev);
+static int eth_em_close(struct rte_eth_dev *dev);
 static int eth_em_promiscuous_enable(struct rte_eth_dev *dev);
 static int eth_em_promiscuous_disable(struct rte_eth_dev *dev);
 static int eth_em_allmulticast_enable(struct rte_eth_dev *dev);
@@ -758,7 +758,7 @@ eth_em_stop(struct rte_eth_dev *dev)
 	}
 }
 
-static void
+static int
 eth_em_close(struct rte_eth_dev *dev)
 {
 	struct e1000_hw *hw = E1000_DEV_PRIVATE_TO_HW(dev->data->dev_private);
@@ -782,6 +782,8 @@ eth_em_close(struct rte_eth_dev *dev)
 	rte_intr_disable(intr_handle);
 	rte_intr_callback_unregister(intr_handle,
 				     eth_em_interrupt_handler, dev);
+
+	return 0;
 }
 
 static int
diff --git a/drivers/net/e1000/igb_ethdev.c b/drivers/net/e1000/igb_ethdev.c
index 5ab74840aa..9e4aefe00c 100644
--- a/drivers/net/e1000/igb_ethdev.c
+++ b/drivers/net/e1000/igb_ethdev.c
@@ -77,7 +77,7 @@ static int  eth_igb_start(struct rte_eth_dev *dev);
 static void eth_igb_stop(struct rte_eth_dev *dev);
 static int  eth_igb_dev_set_link_up(struct rte_eth_dev *dev);
 static int  eth_igb_dev_set_link_down(struct rte_eth_dev *dev);
-static void eth_igb_close(struct rte_eth_dev *dev);
+static int eth_igb_close(struct rte_eth_dev *dev);
 static int eth_igb_reset(struct rte_eth_dev *dev);
 static int  eth_igb_promiscuous_enable(struct rte_eth_dev *dev);
 static int  eth_igb_promiscuous_disable(struct rte_eth_dev *dev);
@@ -155,7 +155,7 @@ static void igbvf_intr_disable(struct e1000_hw *hw);
 static int igbvf_dev_configure(struct rte_eth_dev *dev);
 static int igbvf_dev_start(struct rte_eth_dev *dev);
 static void igbvf_dev_stop(struct rte_eth_dev *dev);
-static void igbvf_dev_close(struct rte_eth_dev *dev);
+static int igbvf_dev_close(struct rte_eth_dev *dev);
 static int igbvf_promiscuous_enable(struct rte_eth_dev *dev);
 static int igbvf_promiscuous_disable(struct rte_eth_dev *dev);
 static int igbvf_allmulticast_enable(struct rte_eth_dev *dev);
@@ -1535,7 +1535,7 @@ eth_igb_dev_set_link_down(struct rte_eth_dev *dev)
 	return 0;
 }
 
-static void
+static int
 eth_igb_close(struct rte_eth_dev *dev)
 {
 	struct e1000_hw *hw = E1000_DEV_PRIVATE_TO_HW(dev->data->dev_private);
@@ -1604,6 +1604,8 @@ eth_igb_close(struct rte_eth_dev *dev)
 
 	/* clear all the filters list */
 	igb_filterlist_flush(dev);
+
+	return 0;
 }
 
 /*
@@ -3381,7 +3383,7 @@ igbvf_dev_stop(struct rte_eth_dev *dev)
 	adapter->stopped = true;
 }
 
-static void
+static int
 igbvf_dev_close(struct rte_eth_dev *dev)
 {
 	struct e1000_hw *hw = E1000_DEV_PRIVATE_TO_HW(dev->data->dev_private);
@@ -3412,6 +3414,8 @@ igbvf_dev_close(struct rte_eth_dev *dev)
 	rte_intr_callback_unregister(&pci_dev->intr_handle,
 				     eth_igbvf_interrupt_handler,
 				     (void *)dev);
+
+	return 0;
 }
 
 static int
diff --git a/drivers/net/ena/ena_ethdev.c b/drivers/net/ena/ena_ethdev.c
index 8077519735..ae6daad892 100644
--- a/drivers/net/ena/ena_ethdev.c
+++ b/drivers/net/ena/ena_ethdev.c
@@ -187,7 +187,7 @@ static void ena_init_rings(struct ena_adapter *adapter,
 static int ena_mtu_set(struct rte_eth_dev *dev, uint16_t mtu);
 static int ena_start(struct rte_eth_dev *dev);
 static void ena_stop(struct rte_eth_dev *dev);
-static void ena_close(struct rte_eth_dev *dev);
+static int ena_close(struct rte_eth_dev *dev);
 static int ena_dev_reset(struct rte_eth_dev *dev);
 static int ena_stats_get(struct rte_eth_dev *dev, struct rte_eth_stats *stats);
 static void ena_rx_queue_release_all(struct rte_eth_dev *dev);
@@ -487,7 +487,7 @@ static void ena_config_debug_area(struct ena_adapter *adapter)
 	ena_com_delete_debug_area(&adapter->ena_dev);
 }
 
-static void ena_close(struct rte_eth_dev *dev)
+static int ena_close(struct rte_eth_dev *dev)
 {
 	struct rte_pci_device *pci_dev = RTE_ETH_DEV_TO_PCI(dev);
 	struct rte_intr_handle *intr_handle = &pci_dev->intr_handle;
@@ -513,6 +513,8 @@ static void ena_close(struct rte_eth_dev *dev)
 	 * release of the resource in the rte_eth_dev_release_port().
 	 */
 	dev->data->mac_addrs = NULL;
+
+	return 0;
 }
 
 static int
diff --git a/drivers/net/enetc/enetc_ethdev.c b/drivers/net/enetc/enetc_ethdev.c
index c6fb42740d..bdb32762ce 100644
--- a/drivers/net/enetc/enetc_ethdev.c
+++ b/drivers/net/enetc/enetc_ethdev.c
@@ -545,7 +545,7 @@ enetc_stats_reset(struct rte_eth_dev *dev)
 	return 0;
 }
 
-static void
+static int
 enetc_dev_close(struct rte_eth_dev *dev)
 {
 	uint16_t i;
@@ -564,6 +564,8 @@ enetc_dev_close(struct rte_eth_dev *dev)
 		dev->data->tx_queues[i] = NULL;
 	}
 	dev->data->nb_tx_queues = 0;
+
+	return 0;
 }
 
 static int
diff --git a/drivers/net/enic/enic_ethdev.c b/drivers/net/enic/enic_ethdev.c
index 57e82845a4..ed03d53608 100644
--- a/drivers/net/enic/enic_ethdev.c
+++ b/drivers/net/enic/enic_ethdev.c
@@ -446,12 +446,14 @@ static void enicpmd_dev_stop(struct rte_eth_dev *eth_dev)
 /*
  * Stop device.
  */
-static void enicpmd_dev_close(struct rte_eth_dev *eth_dev)
+static int enicpmd_dev_close(struct rte_eth_dev *eth_dev)
 {
 	struct enic *enic = pmd_priv(eth_dev);
 
 	ENICPMD_FUNC_TRACE();
 	enic_remove(enic);
+
+	return 0;
 }
 
 static int enicpmd_dev_link_update(struct rte_eth_dev *eth_dev,
diff --git a/drivers/net/enic/enic_vf_representor.c b/drivers/net/enic/enic_vf_representor.c
index 5d34e1b460..c528be2b98 100644
--- a/drivers/net/enic/enic_vf_representor.c
+++ b/drivers/net/enic/enic_vf_representor.c
@@ -277,11 +277,12 @@ static void enic_vf_dev_stop(struct rte_eth_dev *eth_dev)
  * "close" is no-op for now and solely exists so that rte_eth_dev_close()
  * can finish its own cleanup without errors.
  */
-static void enic_vf_dev_close(struct rte_eth_dev *eth_dev __rte_unused)
+static int enic_vf_dev_close(struct rte_eth_dev *eth_dev __rte_unused)
 {
 	ENICPMD_FUNC_TRACE();
 	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
-		return;
+		return 0;
+	return 0;
 }
 
 static int
diff --git a/drivers/net/failsafe/failsafe_ops.c b/drivers/net/failsafe/failsafe_ops.c
index 96f7e456f9..93ebd09114 100644
--- a/drivers/net/failsafe/failsafe_ops.c
+++ b/drivers/net/failsafe/failsafe_ops.c
@@ -240,7 +240,7 @@ fs_dev_set_link_down(struct rte_eth_dev *dev)
 }
 
 static void fs_dev_free_queues(struct rte_eth_dev *dev);
-static void
+static int
 fs_dev_close(struct rte_eth_dev *dev)
 {
 	struct sub_device *sdev;
@@ -259,6 +259,7 @@ fs_dev_close(struct rte_eth_dev *dev)
 	}
 	fs_dev_free_queues(dev);
 	fs_unlock(dev, 0);
+	return 0;
 }
 
 static int
diff --git a/drivers/net/fm10k/fm10k_ethdev.c b/drivers/net/fm10k/fm10k_ethdev.c
index 5771d83b55..533f976709 100644
--- a/drivers/net/fm10k/fm10k_ethdev.c
+++ b/drivers/net/fm10k/fm10k_ethdev.c
@@ -2776,7 +2776,7 @@ fm10k_close_mbx_service(struct fm10k_hw *hw)
 	hw->mbx.ops.disconnect(hw, &hw->mbx);
 }
 
-static void
+static int
 fm10k_dev_close(struct rte_eth_dev *dev)
 {
 	struct fm10k_hw *hw = FM10K_DEV_PRIVATE_TO_HW(dev->data->dev_private);
@@ -2821,6 +2821,8 @@ fm10k_dev_close(struct rte_eth_dev *dev)
 		rte_intr_callback_unregister(intr_handle,
 			fm10k_dev_interrupt_handler_vf, (void *)dev);
 	}
+
+	return 0;
 }
 
 static const struct eth_dev_ops fm10k_eth_dev_ops = {
diff --git a/drivers/net/hinic/hinic_pmd_ethdev.c b/drivers/net/hinic/hinic_pmd_ethdev.c
index 67e6afcf7a..5f2e97d3bd 100644
--- a/drivers/net/hinic/hinic_pmd_ethdev.c
+++ b/drivers/net/hinic/hinic_pmd_ethdev.c
@@ -2936,7 +2936,7 @@ static void hinic_nic_dev_destroy(struct rte_eth_dev *eth_dev)
  * @param dev
  *   Pointer to Ethernet device structure.
  */
-static void hinic_dev_close(struct rte_eth_dev *dev)
+static int hinic_dev_close(struct rte_eth_dev *dev)
 {
 	struct hinic_nic_dev *nic_dev = HINIC_ETH_DEV_TO_PRIVATE_NIC_DEV(dev);
 
@@ -2944,7 +2944,7 @@ static void hinic_dev_close(struct rte_eth_dev *dev)
 					   &nic_dev->dev_status)) {
 		PMD_DRV_LOG(WARNING, "Device %s already closed",
 			    dev->data->name);
-		return;
+		return 0;
 	}
 
 	/* stop device first */
@@ -2971,6 +2971,8 @@ static void hinic_dev_close(struct rte_eth_dev *dev)
 
 	/* deinit nic hardware device */
 	hinic_nic_dev_destroy(dev);
+
+	return 0;
 }
 
 static const struct eth_dev_ops hinic_pmd_ops = {
diff --git a/drivers/net/hns3/hns3_ethdev.c b/drivers/net/hns3/hns3_ethdev.c
index 73d504253d..3b395a1ccf 100644
--- a/drivers/net/hns3/hns3_ethdev.c
+++ b/drivers/net/hns3/hns3_ethdev.c
@@ -4859,7 +4859,7 @@ hns3_dev_stop(struct rte_eth_dev *dev)
 	rte_spinlock_unlock(&hw->lock);
 }
 
-static void
+static int
 hns3_dev_close(struct rte_eth_dev *eth_dev)
 {
 	struct hns3_adapter *hns = eth_dev->data->dev_private;
@@ -4868,7 +4868,7 @@ hns3_dev_close(struct rte_eth_dev *eth_dev)
 	if (rte_eal_process_type() != RTE_PROC_PRIMARY) {
 		rte_free(eth_dev->process_private);
 		eth_dev->process_private = NULL;
-		return;
+		return 0;
 	}
 
 	if (hw->adapter_state == HNS3_NIC_STARTED)
@@ -4888,6 +4888,8 @@ hns3_dev_close(struct rte_eth_dev *eth_dev)
 	eth_dev->process_private = NULL;
 	hns3_mp_uninit_primary();
 	hns3_warn(hw, "Close port %d finished", hw->data->port_id);
+
+	return 0;
 }
 
 static int
diff --git a/drivers/net/hns3/hns3_ethdev_vf.c b/drivers/net/hns3/hns3_ethdev_vf.c
index 037a5be7e0..e0ca5a6ac2 100644
--- a/drivers/net/hns3/hns3_ethdev_vf.c
+++ b/drivers/net/hns3/hns3_ethdev_vf.c
@@ -1910,14 +1910,14 @@ hns3vf_dev_stop(struct rte_eth_dev *dev)
 	rte_spinlock_unlock(&hw->lock);
 }
 
-static void
+static int
 hns3vf_dev_close(struct rte_eth_dev *eth_dev)
 {
 	struct hns3_adapter *hns = eth_dev->data->dev_private;
 	struct hns3_hw *hw = &hns->hw;
 
 	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
-		return;
+		return 0;
 
 	if (hw->adapter_state == HNS3_NIC_STARTED)
 		hns3vf_dev_stop(eth_dev);
@@ -1935,6 +1935,8 @@ hns3vf_dev_close(struct rte_eth_dev *eth_dev)
 	eth_dev->process_private = NULL;
 	hns3_mp_uninit_primary();
 	hns3_warn(hw, "Close port %d finished", hw->data->port_id);
+
+	return 0;
 }
 
 static int
diff --git a/drivers/net/i40e/i40e_ethdev.c b/drivers/net/i40e/i40e_ethdev.c
index 563f21d9df..804f0128ab 100644
--- a/drivers/net/i40e/i40e_ethdev.c
+++ b/drivers/net/i40e/i40e_ethdev.c
@@ -225,7 +225,7 @@ static int eth_i40e_dev_uninit(struct rte_eth_dev *eth_dev);
 static int i40e_dev_configure(struct rte_eth_dev *dev);
 static int i40e_dev_start(struct rte_eth_dev *dev);
 static void i40e_dev_stop(struct rte_eth_dev *dev);
-static void i40e_dev_close(struct rte_eth_dev *dev);
+static int i40e_dev_close(struct rte_eth_dev *dev);
 static int  i40e_dev_reset(struct rte_eth_dev *dev);
 static int i40e_dev_promiscuous_enable(struct rte_eth_dev *dev);
 static int i40e_dev_promiscuous_disable(struct rte_eth_dev *dev);
@@ -2608,7 +2608,7 @@ i40e_dev_stop(struct rte_eth_dev *dev)
 	pf->adapter->rss_reta_updated = 0;
 }
 
-static void
+static int
 i40e_dev_close(struct rte_eth_dev *dev)
 {
 	struct i40e_pf *pf = I40E_DEV_PRIVATE_TO_PF(dev->data->dev_private);
@@ -2748,6 +2748,7 @@ i40e_dev_close(struct rte_eth_dev *dev)
 	i40e_tm_conf_uninit(dev);
 
 	hw->adapter_closed = 1;
+	return 0;
 }
 
 /*
diff --git a/drivers/net/i40e/i40e_ethdev_vf.c b/drivers/net/i40e/i40e_ethdev_vf.c
index 8531cf6b12..61aad8c415 100644
--- a/drivers/net/i40e/i40e_ethdev_vf.c
+++ b/drivers/net/i40e/i40e_ethdev_vf.c
@@ -90,7 +90,7 @@ static int i40evf_dev_xstats_reset(struct rte_eth_dev *dev);
 static int i40evf_vlan_filter_set(struct rte_eth_dev *dev,
 				  uint16_t vlan_id, int on);
 static int i40evf_vlan_offload_set(struct rte_eth_dev *dev, int mask);
-static void i40evf_dev_close(struct rte_eth_dev *dev);
+static int i40evf_dev_close(struct rte_eth_dev *dev);
 static int i40evf_dev_reset(struct rte_eth_dev *dev);
 static int i40evf_check_vf_reset_done(struct rte_eth_dev *dev);
 static int i40evf_dev_promiscuous_enable(struct rte_eth_dev *dev);
@@ -2401,7 +2401,7 @@ i40evf_dev_stats_get(struct rte_eth_dev *dev, struct rte_eth_stats *stats)
 	return ret;
 }
 
-static void
+static int
 i40evf_dev_close(struct rte_eth_dev *dev)
 {
 	struct i40e_hw *hw = I40E_DEV_PRIVATE_TO_HW(dev->data->dev_private);
@@ -2433,6 +2433,7 @@ i40evf_dev_close(struct rte_eth_dev *dev)
 	vf->aq_resp = NULL;
 
 	hw->adapter_closed = 1;
+	return 0;
 }
 
 /*
diff --git a/drivers/net/iavf/iavf_ethdev.c b/drivers/net/iavf/iavf_ethdev.c
index 440da7d76a..8f46c4588f 100644
--- a/drivers/net/iavf/iavf_ethdev.c
+++ b/drivers/net/iavf/iavf_ethdev.c
@@ -32,7 +32,7 @@
 static int iavf_dev_configure(struct rte_eth_dev *dev);
 static int iavf_dev_start(struct rte_eth_dev *dev);
 static void iavf_dev_stop(struct rte_eth_dev *dev);
-static void iavf_dev_close(struct rte_eth_dev *dev);
+static int iavf_dev_close(struct rte_eth_dev *dev);
 static int iavf_dev_reset(struct rte_eth_dev *dev);
 static int iavf_dev_info_get(struct rte_eth_dev *dev,
 			     struct rte_eth_dev_info *dev_info);
@@ -1463,7 +1463,7 @@ iavf_dev_init(struct rte_eth_dev *eth_dev)
 	return 0;
 }
 
-static void
+static int
 iavf_dev_close(struct rte_eth_dev *dev)
 {
 	struct iavf_hw *hw = IAVF_DEV_PRIVATE_TO_HW(dev->data->dev_private);
@@ -1506,6 +1506,8 @@ iavf_dev_close(struct rte_eth_dev *dev)
 
 	rte_free(vf->aq_resp);
 	vf->aq_resp = NULL;
+
+	return 0;
 }
 
 static int
diff --git a/drivers/net/ice/ice_dcf_ethdev.c b/drivers/net/ice/ice_dcf_ethdev.c
index 2faed3cc7a..5b626cb641 100644
--- a/drivers/net/ice/ice_dcf_ethdev.c
+++ b/drivers/net/ice/ice_dcf_ethdev.c
@@ -847,13 +847,13 @@ ice_dcf_stats_reset(struct rte_eth_dev *dev)
 	return 0;
 }
 
-static void
+static int
 ice_dcf_dev_close(struct rte_eth_dev *dev)
 {
 	struct ice_dcf_adapter *adapter = dev->data->dev_private;
 
 	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
-		return;
+		return 0;
 
 	dev->dev_ops = NULL;
 	dev->rx_pkt_burst = NULL;
@@ -861,6 +861,8 @@ ice_dcf_dev_close(struct rte_eth_dev *dev)
 
 	ice_dcf_uninit_parent_adapter(dev);
 	ice_dcf_uninit_hw(dev, &adapter->real_hw);
+
+	return 0;
 }
 
 static int
diff --git a/drivers/net/ice/ice_ethdev.c b/drivers/net/ice/ice_ethdev.c
index cfd357b055..b4ac5e3f81 100644
--- a/drivers/net/ice/ice_ethdev.c
+++ b/drivers/net/ice/ice_ethdev.c
@@ -77,7 +77,7 @@ static struct proto_xtr_ol_flag ice_proto_xtr_ol_flag_params[] = {
 static int ice_dev_configure(struct rte_eth_dev *dev);
 static int ice_dev_start(struct rte_eth_dev *dev);
 static void ice_dev_stop(struct rte_eth_dev *dev);
-static void ice_dev_close(struct rte_eth_dev *dev);
+static int ice_dev_close(struct rte_eth_dev *dev);
 static int ice_dev_reset(struct rte_eth_dev *dev);
 static int ice_dev_info_get(struct rte_eth_dev *dev,
 			    struct rte_eth_dev_info *dev_info);
@@ -2430,7 +2430,7 @@ ice_dev_stop(struct rte_eth_dev *dev)
 	pf->adapter_stopped = true;
 }
 
-static void
+static int
 ice_dev_close(struct rte_eth_dev *dev)
 {
 	struct ice_pf *pf = ICE_DEV_PRIVATE_TO_PF(dev->data->dev_private);
@@ -2478,6 +2478,8 @@ ice_dev_close(struct rte_eth_dev *dev)
 	/* unregister callback func from eal lib */
 	rte_intr_callback_unregister(intr_handle,
 				     ice_interrupt_handler, dev);
+
+	return 0;
 }
 
 static int
diff --git a/drivers/net/igc/igc_ethdev.c b/drivers/net/igc/igc_ethdev.c
index 810568bc54..11397c5c1d 100644
--- a/drivers/net/igc/igc_ethdev.c
+++ b/drivers/net/igc/igc_ethdev.c
@@ -183,7 +183,7 @@ static void eth_igc_stop(struct rte_eth_dev *dev);
 static int eth_igc_start(struct rte_eth_dev *dev);
 static int eth_igc_set_link_up(struct rte_eth_dev *dev);
 static int eth_igc_set_link_down(struct rte_eth_dev *dev);
-static void eth_igc_close(struct rte_eth_dev *dev);
+static int eth_igc_close(struct rte_eth_dev *dev);
 static int eth_igc_reset(struct rte_eth_dev *dev);
 static int eth_igc_promiscuous_enable(struct rte_eth_dev *dev);
 static int eth_igc_promiscuous_disable(struct rte_eth_dev *dev);
@@ -1165,7 +1165,7 @@ igc_dev_free_queues(struct rte_eth_dev *dev)
 	dev->data->nb_tx_queues = 0;
 }
 
-static void
+static int
 eth_igc_close(struct rte_eth_dev *dev)
 {
 	struct rte_pci_device *pci_dev = RTE_ETH_DEV_TO_PCI(dev);
@@ -1199,6 +1199,8 @@ eth_igc_close(struct rte_eth_dev *dev)
 
 	/* Reset any pending lock */
 	igc_reset_swfw_lock(hw);
+
+	return 0;
 }
 
 static void
diff --git a/drivers/net/ionic/ionic_ethdev.c b/drivers/net/ionic/ionic_ethdev.c
index 800f6e5d64..1775fd29b7 100644
--- a/drivers/net/ionic/ionic_ethdev.c
+++ b/drivers/net/ionic/ionic_ethdev.c
@@ -25,7 +25,7 @@ static int  ionic_dev_configure(struct rte_eth_dev *dev);
 static int  ionic_dev_mtu_set(struct rte_eth_dev *dev, uint16_t mtu);
 static int  ionic_dev_start(struct rte_eth_dev *dev);
 static void ionic_dev_stop(struct rte_eth_dev *dev);
-static void ionic_dev_close(struct rte_eth_dev *dev);
+static int  ionic_dev_close(struct rte_eth_dev *dev);
 static int  ionic_dev_set_link_up(struct rte_eth_dev *dev);
 static int  ionic_dev_set_link_down(struct rte_eth_dev *dev);
 static int  ionic_dev_link_update(struct rte_eth_dev *eth_dev,
@@ -956,7 +956,7 @@ ionic_dev_stop(struct rte_eth_dev *eth_dev)
 /*
  * Reset and stop device.
  */
-static void
+static int
 ionic_dev_close(struct rte_eth_dev *eth_dev)
 {
 	struct ionic_lif *lif = IONIC_ETH_DEV_TO_LIF(eth_dev);
@@ -967,14 +967,16 @@ ionic_dev_close(struct rte_eth_dev *eth_dev)
 	err = ionic_lif_stop(lif);
 	if (err) {
 		IONIC_PRINT(ERR, "Cannot stop LIF: %d", err);
-		return;
+		return -1;
 	}
 
 	err = eth_ionic_dev_uninit(eth_dev);
 	if (err) {
 		IONIC_PRINT(ERR, "Cannot destroy LIF: %d", err);
-		return;
+		return -1;
 	}
+
+	return 0;
 }
 
 static int
diff --git a/drivers/net/ipn3ke/ipn3ke_representor.c b/drivers/net/ipn3ke/ipn3ke_representor.c
index b673c49149..d49abbf758 100644
--- a/drivers/net/ipn3ke/ipn3ke_representor.c
+++ b/drivers/net/ipn3ke/ipn3ke_representor.c
@@ -208,7 +208,7 @@ ipn3ke_rpst_dev_stop(struct rte_eth_dev *dev)
 	}
 }
 
-static void
+static int
 ipn3ke_rpst_dev_close(struct rte_eth_dev *dev)
 {
 	struct ipn3ke_hw *hw = IPN3KE_DEV_PRIVATE_TO_HW(dev);
@@ -221,6 +221,8 @@ ipn3ke_rpst_dev_close(struct rte_eth_dev *dev)
 		/* Disable the RX path */
 		ipn3ke_xmac_rx_disable(hw, rpst->port_id, 0);
 	}
+
+	return 0;
 }
 
 /*
diff --git a/drivers/net/ixgbe/ixgbe_ethdev.c b/drivers/net/ixgbe/ixgbe_ethdev.c
index c74467e068..5a863ca4bf 100644
--- a/drivers/net/ixgbe/ixgbe_ethdev.c
+++ b/drivers/net/ixgbe/ixgbe_ethdev.c
@@ -147,7 +147,7 @@ static int  ixgbe_dev_start(struct rte_eth_dev *dev);
 static void ixgbe_dev_stop(struct rte_eth_dev *dev);
 static int  ixgbe_dev_set_link_up(struct rte_eth_dev *dev);
 static int  ixgbe_dev_set_link_down(struct rte_eth_dev *dev);
-static void ixgbe_dev_close(struct rte_eth_dev *dev);
+static int  ixgbe_dev_close(struct rte_eth_dev *dev);
 static int  ixgbe_dev_reset(struct rte_eth_dev *dev);
 static int ixgbe_dev_promiscuous_enable(struct rte_eth_dev *dev);
 static int ixgbe_dev_promiscuous_disable(struct rte_eth_dev *dev);
@@ -251,7 +251,7 @@ static int  ixgbevf_dev_start(struct rte_eth_dev *dev);
 static int ixgbevf_dev_link_update(struct rte_eth_dev *dev,
 				   int wait_to_complete);
 static void ixgbevf_dev_stop(struct rte_eth_dev *dev);
-static void ixgbevf_dev_close(struct rte_eth_dev *dev);
+static int ixgbevf_dev_close(struct rte_eth_dev *dev);
 static int  ixgbevf_dev_reset(struct rte_eth_dev *dev);
 static void ixgbevf_intr_disable(struct rte_eth_dev *dev);
 static void ixgbevf_intr_enable(struct rte_eth_dev *dev);
@@ -2994,7 +2994,7 @@ ixgbe_dev_set_link_down(struct rte_eth_dev *dev)
 /*
  * Reset and stop device.
  */
-static void
+static int
 ixgbe_dev_close(struct rte_eth_dev *dev)
 {
 	struct ixgbe_hw *hw =
@@ -3065,6 +3065,7 @@ ixgbe_dev_close(struct rte_eth_dev *dev)
 	rte_free(dev->security_ctx);
 #endif
 
+	return 0;
 }
 
 /*
@@ -5443,7 +5444,7 @@ ixgbevf_dev_stop(struct rte_eth_dev *dev)
 	adapter->rss_reta_updated = 0;
 }
 
-static void
+static int
 ixgbevf_dev_close(struct rte_eth_dev *dev)
 {
 	struct ixgbe_hw *hw = IXGBE_DEV_PRIVATE_TO_HW(dev->data->dev_private);
@@ -5472,6 +5473,8 @@ ixgbevf_dev_close(struct rte_eth_dev *dev)
 	rte_intr_disable(intr_handle);
 	rte_intr_callback_unregister(intr_handle,
 				     ixgbevf_dev_interrupt_handler, dev);
+
+	return 0;
 }
 
 /*
diff --git a/drivers/net/kni/rte_eth_kni.c b/drivers/net/kni/rte_eth_kni.c
index 831fe96c96..aa365a57de 100644
--- a/drivers/net/kni/rte_eth_kni.c
+++ b/drivers/net/kni/rte_eth_kni.c
@@ -198,7 +198,7 @@ eth_kni_dev_stop(struct rte_eth_dev *dev)
 	dev->data->dev_link.link_status = 0;
 }
 
-static void
+static int
 eth_kni_close(struct rte_eth_dev *eth_dev)
 {
 	struct pmd_internals *internals;
@@ -214,6 +214,8 @@ eth_kni_close(struct rte_eth_dev *eth_dev)
 	if (ret)
 		PMD_LOG(WARNING, "Not able to release kni for %s",
 			eth_dev->data->name);
+
+	return 0;
 }
 
 static int
diff --git a/drivers/net/liquidio/lio_ethdev.c b/drivers/net/liquidio/lio_ethdev.c
index 2c2b27e62a..93e2ed5670 100644
--- a/drivers/net/liquidio/lio_ethdev.c
+++ b/drivers/net/liquidio/lio_ethdev.c
@@ -1550,7 +1550,7 @@ lio_dev_set_link_down(struct rte_eth_dev *eth_dev)
  * @return
  *    - nothing
  */
-static void
+static int
 lio_dev_close(struct rte_eth_dev *eth_dev)
 {
 	struct lio_device *lio_dev = LIO_DEV(eth_dev);
@@ -1581,6 +1581,8 @@ lio_dev_close(struct rte_eth_dev *eth_dev)
 
 	 /* Delete all queues */
 	lio_dev_clear_queues(eth_dev);
+
+	return 0;
 }
 
 /**
diff --git a/drivers/net/memif/rte_eth_memif.c b/drivers/net/memif/rte_eth_memif.c
index a19c0f3e6b..1a2411c838 100644
--- a/drivers/net/memif/rte_eth_memif.c
+++ b/drivers/net/memif/rte_eth_memif.c
@@ -1218,7 +1218,7 @@ memif_dev_start(struct rte_eth_dev *dev)
 	return ret;
 }
 
-static void
+static int
 memif_dev_close(struct rte_eth_dev *dev)
 {
 	struct pmd_internals *pmd = dev->data->dev_private;
@@ -1239,6 +1239,8 @@ memif_dev_close(struct rte_eth_dev *dev)
 	}
 
 	rte_free(dev->process_private);
+
+	return 0;
 }
 
 static int
diff --git a/drivers/net/mlx4/mlx4.c b/drivers/net/mlx4/mlx4.c
index 71061a720f..df59314b66 100644
--- a/drivers/net/mlx4/mlx4.c
+++ b/drivers/net/mlx4/mlx4.c
@@ -370,7 +370,7 @@ mlx4_dev_stop(struct rte_eth_dev *dev)
  * @param dev
  *   Pointer to Ethernet device structure.
  */
-static void
+static int
 mlx4_dev_close(struct rte_eth_dev *dev)
 {
 	struct mlx4_priv *priv = dev->data->dev_private;
@@ -400,6 +400,7 @@ mlx4_dev_close(struct rte_eth_dev *dev)
 		MLX4_ASSERT(priv->ctx == NULL);
 	mlx4_intr_uninstall(priv);
 	memset(priv, 0, sizeof(*priv));
+	return 0;
 }
 
 static const struct eth_dev_ops mlx4_dev_ops = {
diff --git a/drivers/net/mlx5/mlx5.c b/drivers/net/mlx5/mlx5.c
index 4a807fb4fd..01ead6e6af 100644
--- a/drivers/net/mlx5/mlx5.c
+++ b/drivers/net/mlx5/mlx5.c
@@ -1341,7 +1341,7 @@ mlx5_proc_priv_uninit(struct rte_eth_dev *dev)
  * @param dev
  *   Pointer to Ethernet device structure.
  */
-void
+int
 mlx5_dev_close(struct rte_eth_dev *dev)
 {
 	struct mlx5_priv *priv = dev->data->dev_private;
@@ -1351,14 +1351,14 @@ mlx5_dev_close(struct rte_eth_dev *dev)
 	if (rte_eal_process_type() == RTE_PROC_SECONDARY) {
 		/* Check if process_private released. */
 		if (!dev->process_private)
-			return;
+			return 0;
 		mlx5_tx_uar_uninit_secondary(dev);
 		mlx5_proc_priv_uninit(dev);
 		rte_eth_dev_release_port(dev);
-		return;
+		return 0;
 	}
 	if (!priv->sh)
-		return;
+		return 0;
 	DRV_LOG(DEBUG, "port %u closing device \"%s\"",
 		dev->data->port_id,
 		((priv->sh->ctx != NULL) ?
@@ -1479,6 +1479,7 @@ mlx5_dev_close(struct rte_eth_dev *dev)
 	 * it is freed when dev_private is freed.
 	 */
 	dev->data->mac_addrs = NULL;
+	return 0;
 }
 
 /**
diff --git a/drivers/net/mlx5/mlx5.h b/drivers/net/mlx5/mlx5.h
index 865e72d318..56f0e25b4c 100644
--- a/drivers/net/mlx5/mlx5.h
+++ b/drivers/net/mlx5/mlx5.h
@@ -842,7 +842,7 @@ int mlx5_proc_priv_init(struct rte_eth_dev *dev);
 int mlx5_udp_tunnel_port_add(struct rte_eth_dev *dev,
 			      struct rte_eth_udp_tunnel *udp_tunnel);
 uint16_t mlx5_eth_find_next(uint16_t port_id, struct rte_pci_device *pci_dev);
-void mlx5_dev_close(struct rte_eth_dev *dev);
+int mlx5_dev_close(struct rte_eth_dev *dev);
 
 /* Macro to iterate over all valid ports for mlx5 driver. */
 #define MLX5_ETH_FOREACH_DEV(port_id, pci_dev) \
diff --git a/drivers/net/mvneta/mvneta_ethdev.c b/drivers/net/mvneta/mvneta_ethdev.c
index 5bf3ebc4bd..eebcdb840e 100644
--- a/drivers/net/mvneta/mvneta_ethdev.c
+++ b/drivers/net/mvneta/mvneta_ethdev.c
@@ -429,7 +429,7 @@ mvneta_dev_stop(struct rte_eth_dev *dev)
  * @param dev
  *   Pointer to Ethernet device structure.
  */
-static void
+static int
 mvneta_dev_close(struct rte_eth_dev *dev)
 {
 	struct mvneta_priv *priv = dev->data->dev_private;
@@ -455,6 +455,8 @@ mvneta_dev_close(struct rte_eth_dev *dev)
 		mvneta_neta_deinit();
 		rte_mvep_deinit(MVEP_MOD_T_NETA);
 	}
+
+	return 0;
 }
 
 /**
diff --git a/drivers/net/mvpp2/mrvl_ethdev.c b/drivers/net/mvpp2/mrvl_ethdev.c
index f5e77d07bf..27f7cfb056 100644
--- a/drivers/net/mvpp2/mrvl_ethdev.c
+++ b/drivers/net/mvpp2/mrvl_ethdev.c
@@ -855,7 +855,7 @@ mrvl_dev_stop(struct rte_eth_dev *dev)
  * @param dev
  *   Pointer to Ethernet device structure.
  */
-static void
+static int
 mrvl_dev_close(struct rte_eth_dev *dev)
 {
 	struct mrvl_priv *priv = dev->data->dev_private;
@@ -915,6 +915,8 @@ mrvl_dev_close(struct rte_eth_dev *dev)
 		mrvl_deinit_pp2();
 		rte_mvep_deinit(MVEP_MOD_T_PP2);
 	}
+
+	return 0;
 }
 
 /**
diff --git a/drivers/net/netvsc/hn_ethdev.c b/drivers/net/netvsc/hn_ethdev.c
index 229c1b9149..8968036ea3 100644
--- a/drivers/net/netvsc/hn_ethdev.c
+++ b/drivers/net/netvsc/hn_ethdev.c
@@ -841,13 +841,15 @@ hn_dev_stop(struct rte_eth_dev *dev)
 	hn_vf_stop(dev);
 }
 
-static void
+static int
 hn_dev_close(struct rte_eth_dev *dev)
 {
 	PMD_INIT_FUNC_TRACE();
 
 	hn_vf_close(dev);
 	hn_dev_free_queues(dev);
+
+	return 0;
 }
 
 static const struct eth_dev_ops hn_eth_dev_ops = {
diff --git a/drivers/net/nfb/nfb_ethdev.c b/drivers/net/nfb/nfb_ethdev.c
index b039ab6fc2..c3c3d003f1 100644
--- a/drivers/net/nfb/nfb_ethdev.c
+++ b/drivers/net/nfb/nfb_ethdev.c
@@ -209,7 +209,7 @@ nfb_eth_dev_info(struct rte_eth_dev *dev,
  * @param dev
  *   Pointer to Ethernet device structure.
  */
-static void
+static int
 nfb_eth_dev_close(struct rte_eth_dev *dev)
 {
 	struct pmd_internals *internals = dev->data->dev_private;
@@ -235,6 +235,8 @@ nfb_eth_dev_close(struct rte_eth_dev *dev)
 
 	rte_free(dev->data->mac_addrs);
 	dev->data->mac_addrs = NULL;
+
+	return 0;
 }
 
 /**
diff --git a/drivers/net/nfp/nfp_net.c b/drivers/net/nfp/nfp_net.c
index f5e4cd339d..9509dc8bd6 100644
--- a/drivers/net/nfp/nfp_net.c
+++ b/drivers/net/nfp/nfp_net.c
@@ -50,7 +50,7 @@
 #include <errno.h>
 
 /* Prototypes */
-static void nfp_net_close(struct rte_eth_dev *dev);
+static int nfp_net_close(struct rte_eth_dev *dev);
 static int nfp_net_configure(struct rte_eth_dev *dev);
 static void nfp_net_dev_interrupt_handler(void *param);
 static void nfp_net_dev_interrupt_delayed_handler(void *param);
@@ -864,7 +864,7 @@ nfp_net_set_link_down(struct rte_eth_dev *dev)
 }
 
 /* Reset and stop device. The device can not be restarted. */
-static void
+static int
 nfp_net_close(struct rte_eth_dev *dev)
 {
 	struct nfp_net_hw *hw;
@@ -906,6 +906,8 @@ nfp_net_close(struct rte_eth_dev *dev)
 	 * The ixgbe PMD driver disables the pcie master on the
 	 * device. The i40e does not...
 	 */
+
+	return 0;
 }
 
 static int
diff --git a/drivers/net/octeontx/octeontx_ethdev.c b/drivers/net/octeontx/octeontx_ethdev.c
index ff2e22cd44..10eeeac642 100644
--- a/drivers/net/octeontx/octeontx_ethdev.c
+++ b/drivers/net/octeontx/octeontx_ethdev.c
@@ -478,7 +478,7 @@ octeontx_dev_configure(struct rte_eth_dev *dev)
 	return 0;
 }
 
-static void
+static int
 octeontx_dev_close(struct rte_eth_dev *dev)
 {
 	struct octeontx_txq *txq = NULL;
@@ -517,6 +517,8 @@ octeontx_dev_close(struct rte_eth_dev *dev)
 
 	dev->tx_pkt_burst = NULL;
 	dev->rx_pkt_burst = NULL;
+
+	return 0;
 }
 
 static int
diff --git a/drivers/net/octeontx2/otx2_ethdev.c b/drivers/net/octeontx2/otx2_ethdev.c
index 51a6f99877..abe5f03628 100644
--- a/drivers/net/octeontx2/otx2_ethdev.c
+++ b/drivers/net/octeontx2/otx2_ethdev.c
@@ -2222,7 +2222,7 @@ otx2_nix_dev_start(struct rte_eth_dev *eth_dev)
 }
 
 static int otx2_nix_dev_reset(struct rte_eth_dev *eth_dev);
-static void otx2_nix_dev_close(struct rte_eth_dev *eth_dev);
+static int otx2_nix_dev_close(struct rte_eth_dev *eth_dev);
 
 /* Initialize and register driver with DPDK Application */
 static const struct eth_dev_ops otx2_eth_dev_ops = {
@@ -2635,10 +2635,11 @@ otx2_eth_dev_uninit(struct rte_eth_dev *eth_dev, bool mbox_close)
 	return 0;
 }
 
-static void
+static int
 otx2_nix_dev_close(struct rte_eth_dev *eth_dev)
 {
 	otx2_eth_dev_uninit(eth_dev, true);
+	return 0;
 }
 
 static int
diff --git a/drivers/net/pcap/rte_eth_pcap.c b/drivers/net/pcap/rte_eth_pcap.c
index 668cbd1fc7..76e704a65a 100644
--- a/drivers/net/pcap/rte_eth_pcap.c
+++ b/drivers/net/pcap/rte_eth_pcap.c
@@ -728,7 +728,7 @@ eth_stats_reset(struct rte_eth_dev *dev)
 	return 0;
 }
 
-static void
+static int
 eth_dev_close(struct rte_eth_dev *dev)
 {
 	unsigned int i;
@@ -748,6 +748,7 @@ eth_dev_close(struct rte_eth_dev *dev)
 		}
 	}
 
+	return 0;
 }
 
 static void
diff --git a/drivers/net/pfe/pfe_ethdev.c b/drivers/net/pfe/pfe_ethdev.c
index 97356d2b0b..187a0019ff 100644
--- a/drivers/net/pfe/pfe_ethdev.c
+++ b/drivers/net/pfe/pfe_ethdev.c
@@ -398,14 +398,14 @@ pfe_eth_exit(struct rte_eth_dev *dev, struct pfe *pfe)
 	pfe->nb_devs--;
 }
 
-static void
+static int
 pfe_eth_close(struct rte_eth_dev *dev)
 {
 	if (!dev)
-		return;
+		return -1;
 
 	if (!g_pfe)
-		return;
+		return -1;
 
 	pfe_eth_exit(dev, g_pfe);
 
@@ -415,6 +415,8 @@ pfe_eth_close(struct rte_eth_dev *dev)
 		rte_free(g_pfe);
 		g_pfe = NULL;
 	}
+
+	return 0;
 }
 
 static int
diff --git a/drivers/net/qede/qede_ethdev.c b/drivers/net/qede/qede_ethdev.c
index 59f1746ee9..9535df183f 100644
--- a/drivers/net/qede/qede_ethdev.c
+++ b/drivers/net/qede/qede_ethdev.c
@@ -1542,7 +1542,7 @@ static void qede_poll_sp_sb_cb(void *param)
 	}
 }
 
-static void qede_dev_close(struct rte_eth_dev *eth_dev)
+static int qede_dev_close(struct rte_eth_dev *eth_dev)
 {
 	struct rte_pci_device *pci_dev = RTE_ETH_DEV_TO_PCI(eth_dev);
 	struct qede_dev *qdev = QEDE_INIT_QDEV(eth_dev);
@@ -1586,6 +1586,8 @@ static void qede_dev_close(struct rte_eth_dev *eth_dev)
 
 	if (ECORE_IS_CMT(edev))
 		rte_eal_alarm_cancel(qede_poll_sp_sb_cb, (void *)eth_dev);
+
+	return 0;
 }
 
 static int
diff --git a/drivers/net/sfc/sfc_ethdev.c b/drivers/net/sfc/sfc_ethdev.c
index 1a58e0df84..eaa48627d6 100644
--- a/drivers/net/sfc/sfc_ethdev.c
+++ b/drivers/net/sfc/sfc_ethdev.c
@@ -318,7 +318,7 @@ sfc_dev_set_link_down(struct rte_eth_dev *dev)
 	return 0;
 }
 
-static void
+static int
 sfc_dev_close(struct rte_eth_dev *dev)
 {
 	struct sfc_adapter *sa = sfc_adapter_by_eth_dev(dev);
@@ -364,6 +364,8 @@ sfc_dev_close(struct rte_eth_dev *dev)
 
 	dev->process_private = NULL;
 	free(sa);
+
+	return 0;
 }
 
 static int
diff --git a/drivers/net/softnic/rte_eth_softnic.c b/drivers/net/softnic/rte_eth_softnic.c
index e453fd1486..038abd98da 100644
--- a/drivers/net/softnic/rte_eth_softnic.c
+++ b/drivers/net/softnic/rte_eth_softnic.c
@@ -201,10 +201,10 @@ pmd_dev_stop(struct rte_eth_dev *dev)
 	softnic_mtr_free(p);
 }
 
-static void
+static int
 pmd_dev_close(struct rte_eth_dev *dev __rte_unused)
 {
-	return;
+	return 0;
 }
 
 static int
diff --git a/drivers/net/szedata2/rte_eth_szedata2.c b/drivers/net/szedata2/rte_eth_szedata2.c
index a17c53577c..daa40cfd6c 100644
--- a/drivers/net/szedata2/rte_eth_szedata2.c
+++ b/drivers/net/szedata2/rte_eth_szedata2.c
@@ -1155,7 +1155,7 @@ eth_tx_queue_release(void *q)
 	}
 }
 
-static void
+static int
 eth_dev_close(struct rte_eth_dev *dev)
 {
 	struct pmd_internals *internals = dev->data->dev_private;
@@ -1180,6 +1180,8 @@ eth_dev_close(struct rte_eth_dev *dev)
 
 	rte_free(dev->data->mac_addrs);
 	dev->data->mac_addrs = NULL;
+
+	return 0;
 }
 
 static int
diff --git a/drivers/net/tap/rte_eth_tap.c b/drivers/net/tap/rte_eth_tap.c
index df16aa4ea7..2d1e4e365a 100644
--- a/drivers/net/tap/rte_eth_tap.c
+++ b/drivers/net/tap/rte_eth_tap.c
@@ -1067,7 +1067,7 @@ tap_stats_reset(struct rte_eth_dev *dev)
 	return 0;
 }
 
-static void
+static int
 tap_dev_close(struct rte_eth_dev *dev)
 {
 	int i;
@@ -1116,6 +1116,8 @@ tap_dev_close(struct rte_eth_dev *dev)
 	 * Since TUN device has no more opened file descriptors
 	 * it will be removed from kernel
 	 */
+
+	return 0;
 }
 
 static void
diff --git a/drivers/net/thunderx/nicvf_ethdev.c b/drivers/net/thunderx/nicvf_ethdev.c
index 83d9488360..3d7348771a 100644
--- a/drivers/net/thunderx/nicvf_ethdev.c
+++ b/drivers/net/thunderx/nicvf_ethdev.c
@@ -1852,7 +1852,7 @@ nicvf_vf_stop(struct rte_eth_dev *dev, struct nicvf *nic, bool cleanup)
 	}
 }
 
-static void
+static int
 nicvf_dev_close(struct rte_eth_dev *dev)
 {
 	size_t i;
@@ -1869,6 +1869,8 @@ nicvf_dev_close(struct rte_eth_dev *dev)
 
 		nicvf_periodic_alarm_stop(nicvf_vf_interrupt, nic->snicvf[i]);
 	}
+
+	return 0;
 }
 
 static int
diff --git a/drivers/net/vhost/rte_eth_vhost.c b/drivers/net/vhost/rte_eth_vhost.c
index ce32be9ce3..f67f4db812 100644
--- a/drivers/net/vhost/rte_eth_vhost.c
+++ b/drivers/net/vhost/rte_eth_vhost.c
@@ -1164,7 +1164,7 @@ eth_dev_stop(struct rte_eth_dev *dev)
 	update_queuing_status(dev);
 }
 
-static void
+static int
 eth_dev_close(struct rte_eth_dev *dev)
 {
 	struct pmd_internal *internal;
@@ -1173,7 +1173,7 @@ eth_dev_close(struct rte_eth_dev *dev)
 
 	internal = dev->data->dev_private;
 	if (!internal)
-		return;
+		return 0;
 
 	eth_dev_stop(dev);
 
@@ -1201,6 +1201,8 @@ eth_dev_close(struct rte_eth_dev *dev)
 
 	rte_free(vring_states[dev->data->port_id]);
 	vring_states[dev->data->port_id] = NULL;
+
+	return 0;
 }
 
 static int
diff --git a/drivers/net/virtio/virtio_ethdev.c b/drivers/net/virtio/virtio_ethdev.c
index 013a2904e6..0787337a01 100644
--- a/drivers/net/virtio/virtio_ethdev.c
+++ b/drivers/net/virtio/virtio_ethdev.c
@@ -704,7 +704,7 @@ virtio_alloc_queues(struct rte_eth_dev *dev)
 
 static void virtio_queues_unbind_intr(struct rte_eth_dev *dev);
 
-static void
+static int
 virtio_dev_close(struct rte_eth_dev *dev)
 {
 	struct virtio_hw *hw = dev->data->dev_private;
@@ -713,7 +713,7 @@ virtio_dev_close(struct rte_eth_dev *dev)
 	PMD_INIT_LOG(DEBUG, "virtio_dev_close");
 
 	if (!hw->opened)
-		return;
+		return 0;
 	hw->opened = false;
 
 	/* reset the NIC */
@@ -743,6 +743,8 @@ virtio_dev_close(struct rte_eth_dev *dev)
 		if (!hw->modern)
 			rte_pci_ioport_unmap(VTPCI_IO(hw));
 	}
+
+	return 0;
 }
 
 static int
diff --git a/drivers/net/vmxnet3/vmxnet3_ethdev.c b/drivers/net/vmxnet3/vmxnet3_ethdev.c
index fa80e75a5b..17fed0ed6e 100644
--- a/drivers/net/vmxnet3/vmxnet3_ethdev.c
+++ b/drivers/net/vmxnet3/vmxnet3_ethdev.c
@@ -64,7 +64,7 @@ static int eth_vmxnet3_dev_uninit(struct rte_eth_dev *eth_dev);
 static int vmxnet3_dev_configure(struct rte_eth_dev *dev);
 static int vmxnet3_dev_start(struct rte_eth_dev *dev);
 static void vmxnet3_dev_stop(struct rte_eth_dev *dev);
-static void vmxnet3_dev_close(struct rte_eth_dev *dev);
+static int vmxnet3_dev_close(struct rte_eth_dev *dev);
 static void vmxnet3_dev_set_rxmode(struct vmxnet3_hw *hw, uint32_t feature, int set);
 static int vmxnet3_dev_promiscuous_enable(struct rte_eth_dev *dev);
 static int vmxnet3_dev_promiscuous_disable(struct rte_eth_dev *dev);
@@ -888,13 +888,15 @@ vmxnet3_free_queues(struct rte_eth_dev *dev)
 /*
  * Reset and stop device.
  */
-static void
+static int
 vmxnet3_dev_close(struct rte_eth_dev *dev)
 {
 	PMD_INIT_FUNC_TRACE();
 
 	vmxnet3_dev_stop(dev);
 	vmxnet3_free_queues(dev);
+
+	return 0;
 }
 
 static void
diff --git a/lib/librte_ethdev/rte_ethdev_driver.h b/lib/librte_ethdev/rte_ethdev_driver.h
index 04ac8e923c..c3062c246c 100644
--- a/lib/librte_ethdev/rte_ethdev_driver.h
+++ b/lib/librte_ethdev/rte_ethdev_driver.h
@@ -42,7 +42,7 @@ typedef int  (*eth_dev_set_link_up_t)(struct rte_eth_dev *dev);
 typedef int  (*eth_dev_set_link_down_t)(struct rte_eth_dev *dev);
 /**< @internal Function used to link down a configured Ethernet device. */
 
-typedef void (*eth_dev_close_t)(struct rte_eth_dev *dev);
+typedef int (*eth_dev_close_t)(struct rte_eth_dev *dev);
 /**< @internal Function used to close a configured Ethernet device. */
 
 typedef int (*eth_dev_reset_t)(struct rte_eth_dev *dev);
-- 
2.28.0

Re: [dpdk-dev] [PATCH v3 02/29] ethdev: allow drivers to return error on close

[Wang, Haiyue]

> -----Original Message-----
> From: Thomas Monjalon <thomas@monjalon.net>
> Sent: Tuesday, September 29, 2020 07:14
> To: dev@dpdk.org
> Cc: Yigit, Ferruh <ferruh.yigit@intel.com>; arybchenko@solarflare.com; Xu, Rosen <rosen.xu@intel.com>;
> Sachin Saxena <sachin.saxena@oss.nxp.com>; Liron Himi <lironh@marvell.com>; John W. Linville
> <linville@tuxdriver.com>; Loftus, Ciara <ciara.loftus@intel.com>; Zhang, Qi Z <qi.z.zhang@intel.com>;
> Shepard Siegel <shepard.siegel@atomicrules.com>; Ed Czeck <ed.czeck@atomicrules.com>; John Miller
> <john.miller@atomicrules.com>; Igor Russkikh <igor.russkikh@aquantia.com>; Pavel Belous
> <pavel.belous@aquantia.com>; Steven Webster <steven.webster@windriver.com>; Matt Peters
> <matt.peters@windriver.com>; Somalapuram Amaranath <asomalap@amd.com>; Rasesh Mody <rmody@marvell.com>;
> Shahed Shaikh <shshaikh@marvell.com>; Ajit Khaparde <ajit.khaparde@broadcom.com>; Somnath Kotur
> <somnath.kotur@broadcom.com>; Chas Williams <chas3@att.com>; Wei Hu (Xavier) <xavier.huwei@huawei.com>;
> Rahul Lakkireddy <rahul.lakkireddy@chelsio.com>; Hemant Agrawal <hemant.agrawal@nxp.com>; Guo, Jia
> <jia.guo@intel.com>; Wang, Haiyue <haiyue.wang@intel.com>; Marcin Wojtas <mw@semihalf.com>; Michal
> Krawczyk <mk@semihalf.com>; Guy Tzalik <gtzalik@amazon.com>; Evgeny Schemeilin <evgenys@amazon.com>;
> Igor Chauskin <igorch@amazon.com>; Gagandeep Singh <g.singh@nxp.com>; John Daley <johndale@cisco.com>;
> Hyong Youb Kim <hyonkim@cisco.com>; Gaetan Rivet <grive@u256.net>; Wang, Xiao W
> <xiao.w.wang@intel.com>; Ziyang Xuan <xuanziyang2@huawei.com>; Xiaoyun Wang
> <cloud.wangxiaoyun@huawei.com>; Guoyang Zhou <zhouguoyang@huawei.com>; Min Hu (Connor)
> <humin29@huawei.com>; Yisen Zhuang <yisen.zhuang@huawei.com>; Xing, Beilei <beilei.xing@intel.com>; Wu,
> Jingjing <jingjing.wu@intel.com>; Yang, Qiming <qiming.yang@intel.com>; Alfredo Cardigliano
> <cardigliano@ntop.org>; Shijith Thotton <sthotton@marvell.com>; Srisivasubramanian Srinivasan
> <srinivasan@marvell.com>; Jakub Grajciar <jgrajcia@cisco.com>; Matan Azrad <matan@nvidia.com>; Shahaf
> Shuler <shahafs@nvidia.com>; Viacheslav Ovsiienko <viacheslavo@nvidia.com>; Zyta Szpak
> <zr@semihalf.com>; Stephen Hemminger <sthemmin@microsoft.com>; K. Y. Srinivasan <kys@microsoft.com>;
> Haiyang Zhang <haiyangz@microsoft.com>; Long Li <longli@microsoft.com>; Martin Spinler
> <spinler@cesnet.cz>; Heinrich Kuhn <heinrich.kuhn@netronome.com>; Harman Kalra <hkalra@marvell.com>;
> Jerin Jacob <jerinj@marvell.com>; Nithin Dabilpuram <ndabilpuram@marvell.com>; Kiran Kumar K
> <kirankumark@marvell.com>; Akhil Goyal <akhil.goyal@nxp.com>; Singh, Jasvinder
> <jasvinder.singh@intel.com>; Dumitrescu, Cristian <cristian.dumitrescu@intel.com>; Wiles, Keith
> <keith.wiles@intel.com>; Maciej Czekaj <mczekaj@marvell.com>; Maxime Coquelin
> <maxime.coquelin@redhat.com>; Xia, Chenbo <chenbo.xia@intel.com>; Wang, Zhihong
> <zhihong.wang@intel.com>; Yong Wang <yongwang@vmware.com>
> Subject: [PATCH v3 02/29] ethdev: allow drivers to return error on close
> 
> The device operation .dev_close was returning void.
> This driver interface is changed to return an int.
> 
> Note that the API rte_eth_dev_close() is still returning void,
> although a deprecation notice is pending to change it as well.
> 
> Signed-off-by: Thomas Monjalon <thomas@monjalon.net>
> Reviewed-by: Rosen Xu <rosen.xu@intel.com>
> Reviewed-by: Sachin Saxena <sachin.saxena@oss.nxp.com>
> Reviewed-by: Liron Himi <lironh@marvell.com>
> ---

>  drivers/net/e1000/em_ethdev.c             |  6 ++++--
>  drivers/net/e1000/igb_ethdev.c            | 12 ++++++++----
>  drivers/net/igc/igc_ethdev.c              |  6 ++++--
>  drivers/net/ixgbe/ixgbe_ethdev.c          | 11 +++++++----

For e1000/igc/ixgbe

Reviewed-by: Haiyue Wang <haiyue.wang@intel.com>

> 2.28.0

Re: [dpdk-dev] [PATCH v3 02/29] ethdev: allow drivers to return error on close

[Guo, Jia]

For drivers/net/i40e,

Acked-by: Jeff Guo <jia.guo@intel.com>

> -----Original Message-----
> From: Thomas Monjalon <thomas@monjalon.net>
> Sent: Tuesday, September 29, 2020 7:14 AM
> To: dev@dpdk.org
> Cc: Yigit, Ferruh <ferruh.yigit@intel.com>; arybchenko@solarflare.com; Xu,
> Rosen <rosen.xu@intel.com>; Sachin Saxena <sachin.saxena@oss.nxp.com>;
> Liron Himi <lironh@marvell.com>; John W. Linville <linville@tuxdriver.com>;
> Loftus, Ciara <ciara.loftus@intel.com>; Zhang, Qi Z <qi.z.zhang@intel.com>;
> Shepard Siegel <shepard.siegel@atomicrules.com>; Ed Czeck
> <ed.czeck@atomicrules.com>; John Miller <john.miller@atomicrules.com>;
> Igor Russkikh <igor.russkikh@aquantia.com>; Pavel Belous
> <pavel.belous@aquantia.com>; Steven Webster
> <steven.webster@windriver.com>; Matt Peters
> <matt.peters@windriver.com>; Somalapuram Amaranath
> <asomalap@amd.com>; Rasesh Mody <rmody@marvell.com>; Shahed
> Shaikh <shshaikh@marvell.com>; Ajit Khaparde
> <ajit.khaparde@broadcom.com>; Somnath Kotur
> <somnath.kotur@broadcom.com>; Chas Williams <chas3@att.com>; Wei Hu
> (Xavier) <xavier.huwei@huawei.com>; Rahul Lakkireddy
> <rahul.lakkireddy@chelsio.com>; Hemant Agrawal
> <hemant.agrawal@nxp.com>; Guo, Jia <jia.guo@intel.com>; Wang, Haiyue
> <haiyue.wang@intel.com>; Marcin Wojtas <mw@semihalf.com>; Michal
> Krawczyk <mk@semihalf.com>; Guy Tzalik <gtzalik@amazon.com>; Evgeny
> Schemeilin <evgenys@amazon.com>; Igor Chauskin <igorch@amazon.com>;
> Gagandeep Singh <g.singh@nxp.com>; John Daley <johndale@cisco.com>;
> Hyong Youb Kim <hyonkim@cisco.com>; Gaetan Rivet <grive@u256.net>;
> Wang, Xiao W <xiao.w.wang@intel.com>; Ziyang Xuan
> <xuanziyang2@huawei.com>; Xiaoyun Wang
> <cloud.wangxiaoyun@huawei.com>; Guoyang Zhou
> <zhouguoyang@huawei.com>; Min Hu (Connor) <humin29@huawei.com>;
> Yisen Zhuang <yisen.zhuang@huawei.com>; Xing, Beilei
> <beilei.xing@intel.com>; Wu, Jingjing <jingjing.wu@intel.com>; Yang, Qiming
> <qiming.yang@intel.com>; Alfredo Cardigliano <cardigliano@ntop.org>;
> Shijith Thotton <sthotton@marvell.com>; Srisivasubramanian Srinivasan
> <srinivasan@marvell.com>; Jakub Grajciar <jgrajcia@cisco.com>; Matan
> Azrad <matan@nvidia.com>; Shahaf Shuler <shahafs@nvidia.com>;
> Viacheslav Ovsiienko <viacheslavo@nvidia.com>; Zyta Szpak
> <zr@semihalf.com>; Stephen Hemminger <sthemmin@microsoft.com>; K. Y.
> Srinivasan <kys@microsoft.com>; Haiyang Zhang <haiyangz@microsoft.com>;
> Long Li <longli@microsoft.com>; Martin Spinler <spinler@cesnet.cz>;
> Heinrich Kuhn <heinrich.kuhn@netronome.com>; Harman Kalra
> <hkalra@marvell.com>; Jerin Jacob <jerinj@marvell.com>; Nithin Dabilpuram
> <ndabilpuram@marvell.com>; Kiran Kumar K <kirankumark@marvell.com>;
> Akhil Goyal <akhil.goyal@nxp.com>; Singh, Jasvinder
> <jasvinder.singh@intel.com>; Dumitrescu, Cristian
> <cristian.dumitrescu@intel.com>; Wiles, Keith <keith.wiles@intel.com>;
> Maciej Czekaj <mczekaj@marvell.com>; Maxime Coquelin
> <maxime.coquelin@redhat.com>; Xia, Chenbo <chenbo.xia@intel.com>;
> Wang, Zhihong <zhihong.wang@intel.com>; Yong Wang
> <yongwang@vmware.com>
> Subject: [PATCH v3 02/29] ethdev: allow drivers to return error on close
> 
> The device operation .dev_close was returning void.
> This driver interface is changed to return an int.
> 
> Note that the API rte_eth_dev_close() is still returning void,
> although a deprecation notice is pending to change it as well.
> 
> Signed-off-by: Thomas Monjalon <thomas@monjalon.net>
> Reviewed-by: Rosen Xu <rosen.xu@intel.com>
> Reviewed-by: Sachin Saxena <sachin.saxena@oss.nxp.com>
> Reviewed-by: Liron Himi <lironh@marvell.com>
> ---
>  app/test/virtual_pmd.c                    |  6 ++++--
>  drivers/net/af_packet/rte_eth_af_packet.c |  3 ++-
>  drivers/net/af_xdp/rte_eth_af_xdp.c       |  4 +++-
>  drivers/net/ark/ark_ethdev.c              |  6 ++++--
>  drivers/net/atlantic/atl_ethdev.c         |  6 ++++--
>  drivers/net/avp/avp_ethdev.c              |  5 +++--
>  drivers/net/axgbe/axgbe_ethdev.c          |  5 +++--
>  drivers/net/bnx2x/bnx2x_ethdev.c          |  4 +++-
>  drivers/net/bnxt/bnxt_ethdev.c            |  4 +++-
>  drivers/net/bnxt/bnxt_reps.c              |  3 ++-
>  drivers/net/bnxt/bnxt_reps.h              |  2 +-
>  drivers/net/bonding/eth_bond_private.h    |  2 +-
>  drivers/net/bonding/rte_eth_bond_pmd.c    |  4 +++-
>  drivers/net/cxgbe/cxgbe_ethdev.c          | 10 ++++++----
>  drivers/net/cxgbe/cxgbe_pfvf.h            |  2 +-
>  drivers/net/dpaa/dpaa_ethdev.c            |  4 +++-
>  drivers/net/dpaa2/dpaa2_ethdev.c          |  6 ++++--
>  drivers/net/e1000/em_ethdev.c             |  6 ++++--
>  drivers/net/e1000/igb_ethdev.c            | 12 ++++++++----
>  drivers/net/ena/ena_ethdev.c              |  6 ++++--
>  drivers/net/enetc/enetc_ethdev.c          |  4 +++-
>  drivers/net/enic/enic_ethdev.c            |  4 +++-
>  drivers/net/enic/enic_vf_representor.c    |  5 +++--
>  drivers/net/failsafe/failsafe_ops.c       |  3 ++-
>  drivers/net/fm10k/fm10k_ethdev.c          |  4 +++-
>  drivers/net/hinic/hinic_pmd_ethdev.c      |  6 ++++--
>  drivers/net/hns3/hns3_ethdev.c            |  6 ++++--
>  drivers/net/hns3/hns3_ethdev_vf.c         |  6 ++++--
>  drivers/net/i40e/i40e_ethdev.c            |  5 +++--
>  drivers/net/i40e/i40e_ethdev_vf.c         |  5 +++--
>  drivers/net/iavf/iavf_ethdev.c            |  6 ++++--
>  drivers/net/ice/ice_dcf_ethdev.c          |  6 ++++--
>  drivers/net/ice/ice_ethdev.c              |  6 ++++--
>  drivers/net/igc/igc_ethdev.c              |  6 ++++--
>  drivers/net/ionic/ionic_ethdev.c          | 10 ++++++----
>  drivers/net/ipn3ke/ipn3ke_representor.c   |  4 +++-
>  drivers/net/ixgbe/ixgbe_ethdev.c          | 11 +++++++----
>  drivers/net/kni/rte_eth_kni.c             |  4 +++-
>  drivers/net/liquidio/lio_ethdev.c         |  4 +++-
>  drivers/net/memif/rte_eth_memif.c         |  4 +++-
>  drivers/net/mlx4/mlx4.c                   |  3 ++-
>  drivers/net/mlx5/mlx5.c                   |  9 +++++----
>  drivers/net/mlx5/mlx5.h                   |  2 +-
>  drivers/net/mvneta/mvneta_ethdev.c        |  4 +++-
>  drivers/net/mvpp2/mrvl_ethdev.c           |  4 +++-
>  drivers/net/netvsc/hn_ethdev.c            |  4 +++-
>  drivers/net/nfb/nfb_ethdev.c              |  4 +++-
>  drivers/net/nfp/nfp_net.c                 |  6 ++++--
>  drivers/net/octeontx/octeontx_ethdev.c    |  4 +++-
>  drivers/net/octeontx2/otx2_ethdev.c       |  5 +++--
>  drivers/net/pcap/rte_eth_pcap.c           |  3 ++-
>  drivers/net/pfe/pfe_ethdev.c              |  8 +++++---
>  drivers/net/qede/qede_ethdev.c            |  4 +++-
>  drivers/net/sfc/sfc_ethdev.c              |  4 +++-
>  drivers/net/softnic/rte_eth_softnic.c     |  4 ++--
>  drivers/net/szedata2/rte_eth_szedata2.c   |  4 +++-
>  drivers/net/tap/rte_eth_tap.c             |  4 +++-
>  drivers/net/thunderx/nicvf_ethdev.c       |  4 +++-
>  drivers/net/vhost/rte_eth_vhost.c         |  6 ++++--
>  drivers/net/virtio/virtio_ethdev.c        |  6 ++++--
>  drivers/net/vmxnet3/vmxnet3_ethdev.c      |  6 ++++--
>  lib/librte_ethdev/rte_ethdev_driver.h     |  2 +-
>  62 files changed, 206 insertions(+), 103 deletions(-)
> 
> diff --git a/app/test/virtual_pmd.c b/app/test/virtual_pmd.c
> index b49089a33c..4bd4d1c9ad 100644
> --- a/app/test/virtual_pmd.c
> +++ b/app/test/virtual_pmd.c
> @@ -62,9 +62,11 @@ static void  virtual_ethdev_stop(struct rte_eth_dev
> *eth_dev __rte_unused)
>  		rte_pktmbuf_free(pkt);
>  }
> 
> -static void
> +static int
>  virtual_ethdev_close(struct rte_eth_dev *dev __rte_unused)
> -{}
> +{
> +	return 0;
> +}
> 
>  static int
>  virtual_ethdev_configure_success(struct rte_eth_dev *dev __rte_unused)
> diff --git a/drivers/net/af_packet/rte_eth_af_packet.c
> b/drivers/net/af_packet/rte_eth_af_packet.c
> index b9723e9619..7d0ff1cbb3 100644
> --- a/drivers/net/af_packet/rte_eth_af_packet.c
> +++ b/drivers/net/af_packet/rte_eth_af_packet.c
> @@ -376,9 +376,10 @@ eth_stats_reset(struct rte_eth_dev *dev)
>  	return 0;
>  }
> 
> -static void
> +static int
>  eth_dev_close(struct rte_eth_dev *dev __rte_unused)
>  {
> +	return 0;
>  }
> 
>  static void
> diff --git a/drivers/net/af_xdp/rte_eth_af_xdp.c
> b/drivers/net/af_xdp/rte_eth_af_xdp.c
> index b65ee449fc..badbce63fb 100644
> --- a/drivers/net/af_xdp/rte_eth_af_xdp.c
> +++ b/drivers/net/af_xdp/rte_eth_af_xdp.c
> @@ -701,7 +701,7 @@ xdp_umem_destroy(struct xsk_umem_info *umem)
>  	umem = NULL;
>  }
> 
> -static void
> +static int
>  eth_dev_close(struct rte_eth_dev *dev)
>  {
>  	struct pmd_internals *internals = dev->data->dev_private;
> @@ -731,6 +731,8 @@ eth_dev_close(struct rte_eth_dev *dev)
>  	dev->data->mac_addrs = NULL;
> 
>  	remove_xdp_program(internals);
> +
> +	return 0;
>  }
> 
>  static void
> diff --git a/drivers/net/ark/ark_ethdev.c b/drivers/net/ark/ark_ethdev.c
> index e13415e95b..5f2ed4b7b7 100644
> --- a/drivers/net/ark/ark_ethdev.c
> +++ b/drivers/net/ark/ark_ethdev.c
> @@ -30,7 +30,7 @@ static int eth_ark_dev_uninit(struct rte_eth_dev
> *eth_dev);
>  static int eth_ark_dev_configure(struct rte_eth_dev *dev);
>  static int eth_ark_dev_start(struct rte_eth_dev *dev);
>  static void eth_ark_dev_stop(struct rte_eth_dev *dev);
> -static void eth_ark_dev_close(struct rte_eth_dev *dev);
> +static int eth_ark_dev_close(struct rte_eth_dev *dev);
>  static int eth_ark_dev_info_get(struct rte_eth_dev *dev,
>  				struct rte_eth_dev_info *dev_info);
>  static int eth_ark_dev_link_update(struct rte_eth_dev *dev,
> @@ -676,7 +676,7 @@ eth_ark_dev_stop(struct rte_eth_dev *dev)
>  	}
>  }
> 
> -static void
> +static int
>  eth_ark_dev_close(struct rte_eth_dev *dev)
>  {
>  	struct ark_adapter *ark = dev->data->dev_private;
> @@ -706,6 +706,8 @@ eth_ark_dev_close(struct rte_eth_dev *dev)
> 
>  	rte_free(dev->data->mac_addrs);
>  	dev->data->mac_addrs = 0;
> +
> +	return 0;
>  }
> 
>  static int
> diff --git a/drivers/net/atlantic/atl_ethdev.c
> b/drivers/net/atlantic/atl_ethdev.c
> index 2fca9140d7..254758fd66 100644
> --- a/drivers/net/atlantic/atl_ethdev.c
> +++ b/drivers/net/atlantic/atl_ethdev.c
> @@ -22,7 +22,7 @@ static int  atl_dev_start(struct rte_eth_dev *dev);
>  static void atl_dev_stop(struct rte_eth_dev *dev);
>  static int  atl_dev_set_link_up(struct rte_eth_dev *dev);
>  static int  atl_dev_set_link_down(struct rte_eth_dev *dev);
> -static void atl_dev_close(struct rte_eth_dev *dev);
> +static int  atl_dev_close(struct rte_eth_dev *dev);
>  static int  atl_dev_reset(struct rte_eth_dev *dev);
>  static int  atl_dev_promiscuous_enable(struct rte_eth_dev *dev);
>  static int  atl_dev_promiscuous_disable(struct rte_eth_dev *dev);
> @@ -719,7 +719,7 @@ atl_dev_set_link_down(struct rte_eth_dev *dev)
>  /*
>   * Reset and stop device.
>   */
> -static void
> +static int
>  atl_dev_close(struct rte_eth_dev *dev)
>  {
>  	PMD_INIT_FUNC_TRACE();
> @@ -727,6 +727,8 @@ atl_dev_close(struct rte_eth_dev *dev)
>  	atl_dev_stop(dev);
> 
>  	atl_free_queues(dev);
> +
> +	return 0;
>  }
> 
>  static int
> diff --git a/drivers/net/avp/avp_ethdev.c b/drivers/net/avp/avp_ethdev.c
> index 5b47f0924a..2eed6acc3f 100644
> --- a/drivers/net/avp/avp_ethdev.c
> +++ b/drivers/net/avp/avp_ethdev.c
> @@ -38,7 +38,7 @@ static int avp_dev_create(struct rte_pci_device
> *pci_dev,
>  static int avp_dev_configure(struct rte_eth_dev *dev);
>  static int avp_dev_start(struct rte_eth_dev *dev);
>  static void avp_dev_stop(struct rte_eth_dev *dev);
> -static void avp_dev_close(struct rte_eth_dev *dev);
> +static int avp_dev_close(struct rte_eth_dev *dev);
>  static int avp_dev_info_get(struct rte_eth_dev *dev,
>  			    struct rte_eth_dev_info *dev_info);
>  static int avp_vlan_offload_set(struct rte_eth_dev *dev, int mask);
> @@ -2103,7 +2103,7 @@ avp_dev_stop(struct rte_eth_dev *eth_dev)
>  	rte_spinlock_unlock(&avp->lock);
>  }
> 
> -static void
> +static int
>  avp_dev_close(struct rte_eth_dev *eth_dev)
>  {
>  	struct avp_dev *avp = AVP_DEV_PRIVATE_TO_HW(eth_dev->data-
> >dev_private);
> @@ -2139,6 +2139,7 @@ avp_dev_close(struct rte_eth_dev *eth_dev)
> 
>  unlock:
>  	rte_spinlock_unlock(&avp->lock);
> +	return 0;
>  }
> 
>  static int
> diff --git a/drivers/net/axgbe/axgbe_ethdev.c
> b/drivers/net/axgbe/axgbe_ethdev.c
> index 4f5a014b54..458d64bcef 100644
> --- a/drivers/net/axgbe/axgbe_ethdev.c
> +++ b/drivers/net/axgbe/axgbe_ethdev.c
> @@ -16,7 +16,7 @@ static int  axgbe_dev_configure(struct rte_eth_dev
> *dev);
>  static int  axgbe_dev_start(struct rte_eth_dev *dev);
>  static void axgbe_dev_stop(struct rte_eth_dev *dev);
>  static void axgbe_dev_interrupt_handler(void *param);
> -static void axgbe_dev_close(struct rte_eth_dev *dev);
> +static int axgbe_dev_close(struct rte_eth_dev *dev);
>  static int axgbe_dev_promiscuous_enable(struct rte_eth_dev *dev);
>  static int axgbe_dev_promiscuous_disable(struct rte_eth_dev *dev);
>  static int axgbe_dev_allmulticast_enable(struct rte_eth_dev *dev);
> @@ -410,10 +410,11 @@ axgbe_dev_stop(struct rte_eth_dev *dev)
>  }
> 
>  /* Clear all resources like TX/RX queues. */
> -static void
> +static int
>  axgbe_dev_close(struct rte_eth_dev *dev)
>  {
>  	axgbe_dev_clear_queues(dev);
> +	return 0;
>  }
> 
>  static int
> diff --git a/drivers/net/bnx2x/bnx2x_ethdev.c
> b/drivers/net/bnx2x/bnx2x_ethdev.c
> index b2ea5fafa7..73217c2138 100644
> --- a/drivers/net/bnx2x/bnx2x_ethdev.c
> +++ b/drivers/net/bnx2x/bnx2x_ethdev.c
> @@ -280,7 +280,7 @@ bnx2x_dev_stop(struct rte_eth_dev *dev)
>  	return;
>  }
> 
> -static void
> +static int
>  bnx2x_dev_close(struct rte_eth_dev *dev)
>  {
>  	struct bnx2x_softc *sc = dev->data->dev_private;
> @@ -295,6 +295,8 @@ bnx2x_dev_close(struct rte_eth_dev *dev)
> 
>  	/* free ilt */
>  	bnx2x_free_ilt_mem(sc);
> +
> +	return 0;
>  }
> 
>  static int
> diff --git a/drivers/net/bnxt/bnxt_ethdev.c
> b/drivers/net/bnxt/bnxt_ethdev.c
> index 05e9a6abbf..e726d078ec 100644
> --- a/drivers/net/bnxt/bnxt_ethdev.c
> +++ b/drivers/net/bnxt/bnxt_ethdev.c
> @@ -1357,7 +1357,7 @@ static void bnxt_dev_stop_op(struct rte_eth_dev
> *eth_dev)
>  		bp->flow_stat->flow_count = 0;
>  }
> 
> -static void bnxt_dev_close_op(struct rte_eth_dev *eth_dev)
> +static int bnxt_dev_close_op(struct rte_eth_dev *eth_dev)
>  {
>  	struct bnxt *bp = eth_dev->data->dev_private;
> 
> @@ -1392,6 +1392,8 @@ static void bnxt_dev_close_op(struct rte_eth_dev
> *eth_dev)
> 
>  	rte_free(bp->grp_info);
>  	bp->grp_info = NULL;
> +
> +	return 0;
>  }
> 
>  static void bnxt_mac_addr_remove_op(struct rte_eth_dev *eth_dev,
> diff --git a/drivers/net/bnxt/bnxt_reps.c b/drivers/net/bnxt/bnxt_reps.c
> index 17010f1ee2..2b6c0a277b 100644
> --- a/drivers/net/bnxt/bnxt_reps.c
> +++ b/drivers/net/bnxt/bnxt_reps.c
> @@ -457,10 +457,11 @@ void bnxt_vf_rep_dev_stop_op(struct
> rte_eth_dev *eth_dev)
>  	bnxt_vf_rep_free_rx_mbufs(vfr_bp);
>  }
> 
> -void bnxt_vf_rep_dev_close_op(struct rte_eth_dev *eth_dev)
> +int bnxt_vf_rep_dev_close_op(struct rte_eth_dev *eth_dev)
>  {
>  	BNXT_TF_DBG(DEBUG, "BNXT Port:%d VFR close\n", eth_dev->data-
> >port_id);
>  	bnxt_vf_representor_uninit(eth_dev);
> +	return 0;
>  }
> 
>  int bnxt_vf_rep_dev_info_get_op(struct rte_eth_dev *eth_dev,
> diff --git a/drivers/net/bnxt/bnxt_reps.h b/drivers/net/bnxt/bnxt_reps.h
> index 3239e03fc0..d9b9ee2757 100644
> --- a/drivers/net/bnxt/bnxt_reps.h
> +++ b/drivers/net/bnxt/bnxt_reps.h
> @@ -45,7 +45,7 @@ int bnxt_vf_rep_tx_queue_setup_op(struct
> rte_eth_dev *eth_dev,
>  void bnxt_vf_rep_rx_queue_release_op(void *rx_queue);
>  void bnxt_vf_rep_tx_queue_release_op(void *tx_queue);
>  void bnxt_vf_rep_dev_stop_op(struct rte_eth_dev *eth_dev);
> -void bnxt_vf_rep_dev_close_op(struct rte_eth_dev *eth_dev);
> +int bnxt_vf_rep_dev_close_op(struct rte_eth_dev *eth_dev);
>  int bnxt_vf_rep_stats_get_op(struct rte_eth_dev *eth_dev,
>  			     struct rte_eth_stats *stats);
>  int bnxt_vf_rep_stats_reset_op(struct rte_eth_dev *eth_dev);
> diff --git a/drivers/net/bonding/eth_bond_private.h
> b/drivers/net/bonding/eth_bond_private.h
> index c9b2d0fe46..0a0034705d 100644
> --- a/drivers/net/bonding/eth_bond_private.h
> +++ b/drivers/net/bonding/eth_bond_private.h
> @@ -318,7 +318,7 @@ bond_tlb_activate_slave(struct bond_dev_private
> *internals);
>  void
>  bond_ethdev_stop(struct rte_eth_dev *eth_dev);
> 
> -void
> +int
>  bond_ethdev_close(struct rte_eth_dev *dev);
> 
>  #endif
> diff --git a/drivers/net/bonding/rte_eth_bond_pmd.c
> b/drivers/net/bonding/rte_eth_bond_pmd.c
> index 4ffd2ca4bb..dff835ef92 100644
> --- a/drivers/net/bonding/rte_eth_bond_pmd.c
> +++ b/drivers/net/bonding/rte_eth_bond_pmd.c
> @@ -2095,7 +2095,7 @@ bond_ethdev_stop(struct rte_eth_dev *eth_dev)
>  	}
>  }
> 
> -void
> +int
>  bond_ethdev_close(struct rte_eth_dev *dev)
>  {
>  	struct bond_dev_private *internals = dev->data->dev_private;
> @@ -2119,6 +2119,8 @@ bond_ethdev_close(struct rte_eth_dev *dev)
>  	bond_flow_ops.flush(dev, &ferror);
>  	bond_ethdev_free_queues(dev);
>  	rte_bitmap_reset(internals->vlan_filter_bmp);
> +
> +	return 0;
>  }
> 
>  /* forward declaration */
> diff --git a/drivers/net/cxgbe/cxgbe_ethdev.c
> b/drivers/net/cxgbe/cxgbe_ethdev.c
> index 422e190daf..e4bbba5c32 100644
> --- a/drivers/net/cxgbe/cxgbe_ethdev.c
> +++ b/drivers/net/cxgbe/cxgbe_ethdev.c
> @@ -318,7 +318,7 @@ int cxgbe_dev_mtu_set(struct rte_eth_dev
> *eth_dev, uint16_t mtu)
>  /*
>   * Stop device.
>   */
> -void cxgbe_dev_close(struct rte_eth_dev *eth_dev)
> +int cxgbe_dev_close(struct rte_eth_dev *eth_dev)
>  {
>  	struct port_info *temp_pi, *pi = eth_dev->data->dev_private;
>  	struct adapter *adapter = pi->adapter;
> @@ -327,10 +327,10 @@ void cxgbe_dev_close(struct rte_eth_dev
> *eth_dev)
>  	CXGBE_FUNC_TRACE();
> 
>  	if (!(adapter->flags & FULL_INIT_DONE))
> -		return;
> +		return 0;
> 
>  	if (!pi->viid)
> -		return;
> +		return 0;
> 
>  	cxgbe_down(pi);
>  	t4_sge_eth_release_queues(pi);
> @@ -343,11 +343,13 @@ void cxgbe_dev_close(struct rte_eth_dev
> *eth_dev)
>  	for_each_port(adapter, i) {
>  		temp_pi = adap2pinfo(adapter, i);
>  		if (temp_pi->viid)
> -			return;
> +			return 0;
>  	}
> 
>  	cxgbe_close(adapter);
>  	rte_free(adapter);
> +
> +	return 0;
>  }
> 
>  /* Start the device.
> diff --git a/drivers/net/cxgbe/cxgbe_pfvf.h
> b/drivers/net/cxgbe/cxgbe_pfvf.h
> index 0b7c52aecf..69d91639e9 100644
> --- a/drivers/net/cxgbe/cxgbe_pfvf.h
> +++ b/drivers/net/cxgbe/cxgbe_pfvf.h
> @@ -19,7 +19,7 @@
>  void cxgbe_dev_rx_queue_release(void *q);
>  void cxgbe_dev_tx_queue_release(void *q);
>  void cxgbe_dev_stop(struct rte_eth_dev *eth_dev);
> -void cxgbe_dev_close(struct rte_eth_dev *eth_dev);
> +int cxgbe_dev_close(struct rte_eth_dev *eth_dev);
>  int cxgbe_dev_info_get(struct rte_eth_dev *eth_dev,
>  		       struct rte_eth_dev_info *device_info);
>  int cxgbe_dev_promiscuous_enable(struct rte_eth_dev *eth_dev);
> diff --git a/drivers/net/dpaa/dpaa_ethdev.c
> b/drivers/net/dpaa/dpaa_ethdev.c
> index 8a6b19b5ce..7fe06e1830 100644
> --- a/drivers/net/dpaa/dpaa_ethdev.c
> +++ b/drivers/net/dpaa/dpaa_ethdev.c
> @@ -369,7 +369,7 @@ static void dpaa_eth_dev_stop(struct rte_eth_dev
> *dev)
>  	dev->tx_pkt_burst = dpaa_eth_tx_drop_all;
>  }
> 
> -static void dpaa_eth_dev_close(struct rte_eth_dev *dev)
> +static int dpaa_eth_dev_close(struct rte_eth_dev *dev)
>  {
>  	struct fman_if *fif = dev->process_private;
>  	struct __fman_if *__fif;
> @@ -392,6 +392,8 @@ static void dpaa_eth_dev_close(struct rte_eth_dev
> *dev)
>  					     dpaa_interrupt_handler,
>  					     (void *)dev);
>  	}
> +
> +	return 0;
>  }
> 
>  static int
> diff --git a/drivers/net/dpaa2/dpaa2_ethdev.c
> b/drivers/net/dpaa2/dpaa2_ethdev.c
> index 02daa4d250..fb4165fa03 100644
> --- a/drivers/net/dpaa2/dpaa2_ethdev.c
> +++ b/drivers/net/dpaa2/dpaa2_ethdev.c
> @@ -1236,7 +1236,7 @@ dpaa2_dev_stop(struct rte_eth_dev *dev)
>  	rte_eth_linkstatus_set(dev, &link);
>  }
> 
> -static void
> +static int
>  dpaa2_dev_close(struct rte_eth_dev *dev)
>  {
>  	struct dpaa2_dev_priv *priv = dev->data->dev_private;
> @@ -1252,11 +1252,13 @@ dpaa2_dev_close(struct rte_eth_dev *dev)
>  	ret = dpni_reset(dpni, CMD_PRI_LOW, priv->token);
>  	if (ret) {
>  		DPAA2_PMD_ERR("Failure cleaning dpni device: err=%d",
> ret);
> -		return;
> +		return -1;
>  	}
> 
>  	memset(&link, 0, sizeof(link));
>  	rte_eth_linkstatus_set(dev, &link);
> +
> +	return 0;
>  }
> 
>  static int
> diff --git a/drivers/net/e1000/em_ethdev.c
> b/drivers/net/e1000/em_ethdev.c
> index 1dc360713a..2d0e071448 100644
> --- a/drivers/net/e1000/em_ethdev.c
> +++ b/drivers/net/e1000/em_ethdev.c
> @@ -34,7 +34,7 @@
>  static int eth_em_configure(struct rte_eth_dev *dev);
>  static int eth_em_start(struct rte_eth_dev *dev);
>  static void eth_em_stop(struct rte_eth_dev *dev);
> -static void eth_em_close(struct rte_eth_dev *dev);
> +static int eth_em_close(struct rte_eth_dev *dev);
>  static int eth_em_promiscuous_enable(struct rte_eth_dev *dev);
>  static int eth_em_promiscuous_disable(struct rte_eth_dev *dev);
>  static int eth_em_allmulticast_enable(struct rte_eth_dev *dev);
> @@ -758,7 +758,7 @@ eth_em_stop(struct rte_eth_dev *dev)
>  	}
>  }
> 
> -static void
> +static int
>  eth_em_close(struct rte_eth_dev *dev)
>  {
>  	struct e1000_hw *hw = E1000_DEV_PRIVATE_TO_HW(dev->data-
> >dev_private);
> @@ -782,6 +782,8 @@ eth_em_close(struct rte_eth_dev *dev)
>  	rte_intr_disable(intr_handle);
>  	rte_intr_callback_unregister(intr_handle,
>  				     eth_em_interrupt_handler, dev);
> +
> +	return 0;
>  }
> 
>  static int
> diff --git a/drivers/net/e1000/igb_ethdev.c
> b/drivers/net/e1000/igb_ethdev.c
> index 5ab74840aa..9e4aefe00c 100644
> --- a/drivers/net/e1000/igb_ethdev.c
> +++ b/drivers/net/e1000/igb_ethdev.c
> @@ -77,7 +77,7 @@ static int  eth_igb_start(struct rte_eth_dev *dev);
>  static void eth_igb_stop(struct rte_eth_dev *dev);
>  static int  eth_igb_dev_set_link_up(struct rte_eth_dev *dev);
>  static int  eth_igb_dev_set_link_down(struct rte_eth_dev *dev);
> -static void eth_igb_close(struct rte_eth_dev *dev);
> +static int eth_igb_close(struct rte_eth_dev *dev);
>  static int eth_igb_reset(struct rte_eth_dev *dev);
>  static int  eth_igb_promiscuous_enable(struct rte_eth_dev *dev);
>  static int  eth_igb_promiscuous_disable(struct rte_eth_dev *dev);
> @@ -155,7 +155,7 @@ static void igbvf_intr_disable(struct e1000_hw *hw);
>  static int igbvf_dev_configure(struct rte_eth_dev *dev);
>  static int igbvf_dev_start(struct rte_eth_dev *dev);
>  static void igbvf_dev_stop(struct rte_eth_dev *dev);
> -static void igbvf_dev_close(struct rte_eth_dev *dev);
> +static int igbvf_dev_close(struct rte_eth_dev *dev);
>  static int igbvf_promiscuous_enable(struct rte_eth_dev *dev);
>  static int igbvf_promiscuous_disable(struct rte_eth_dev *dev);
>  static int igbvf_allmulticast_enable(struct rte_eth_dev *dev);
> @@ -1535,7 +1535,7 @@ eth_igb_dev_set_link_down(struct rte_eth_dev
> *dev)
>  	return 0;
>  }
> 
> -static void
> +static int
>  eth_igb_close(struct rte_eth_dev *dev)
>  {
>  	struct e1000_hw *hw = E1000_DEV_PRIVATE_TO_HW(dev->data-
> >dev_private);
> @@ -1604,6 +1604,8 @@ eth_igb_close(struct rte_eth_dev *dev)
> 
>  	/* clear all the filters list */
>  	igb_filterlist_flush(dev);
> +
> +	return 0;
>  }
> 
>  /*
> @@ -3381,7 +3383,7 @@ igbvf_dev_stop(struct rte_eth_dev *dev)
>  	adapter->stopped = true;
>  }
> 
> -static void
> +static int
>  igbvf_dev_close(struct rte_eth_dev *dev)
>  {
>  	struct e1000_hw *hw = E1000_DEV_PRIVATE_TO_HW(dev->data-
> >dev_private);
> @@ -3412,6 +3414,8 @@ igbvf_dev_close(struct rte_eth_dev *dev)
>  	rte_intr_callback_unregister(&pci_dev->intr_handle,
>  				     eth_igbvf_interrupt_handler,
>  				     (void *)dev);
> +
> +	return 0;
>  }
> 
>  static int
> diff --git a/drivers/net/ena/ena_ethdev.c b/drivers/net/ena/ena_ethdev.c
> index 8077519735..ae6daad892 100644
> --- a/drivers/net/ena/ena_ethdev.c
> +++ b/drivers/net/ena/ena_ethdev.c
> @@ -187,7 +187,7 @@ static void ena_init_rings(struct ena_adapter
> *adapter,
>  static int ena_mtu_set(struct rte_eth_dev *dev, uint16_t mtu);
>  static int ena_start(struct rte_eth_dev *dev);
>  static void ena_stop(struct rte_eth_dev *dev);
> -static void ena_close(struct rte_eth_dev *dev);
> +static int ena_close(struct rte_eth_dev *dev);
>  static int ena_dev_reset(struct rte_eth_dev *dev);
>  static int ena_stats_get(struct rte_eth_dev *dev, struct rte_eth_stats
> *stats);
>  static void ena_rx_queue_release_all(struct rte_eth_dev *dev);
> @@ -487,7 +487,7 @@ static void ena_config_debug_area(struct
> ena_adapter *adapter)
>  	ena_com_delete_debug_area(&adapter->ena_dev);
>  }
> 
> -static void ena_close(struct rte_eth_dev *dev)
> +static int ena_close(struct rte_eth_dev *dev)
>  {
>  	struct rte_pci_device *pci_dev = RTE_ETH_DEV_TO_PCI(dev);
>  	struct rte_intr_handle *intr_handle = &pci_dev->intr_handle;
> @@ -513,6 +513,8 @@ static void ena_close(struct rte_eth_dev *dev)
>  	 * release of the resource in the rte_eth_dev_release_port().
>  	 */
>  	dev->data->mac_addrs = NULL;
> +
> +	return 0;
>  }
> 
>  static int
> diff --git a/drivers/net/enetc/enetc_ethdev.c
> b/drivers/net/enetc/enetc_ethdev.c
> index c6fb42740d..bdb32762ce 100644
> --- a/drivers/net/enetc/enetc_ethdev.c
> +++ b/drivers/net/enetc/enetc_ethdev.c
> @@ -545,7 +545,7 @@ enetc_stats_reset(struct rte_eth_dev *dev)
>  	return 0;
>  }
> 
> -static void
> +static int
>  enetc_dev_close(struct rte_eth_dev *dev)
>  {
>  	uint16_t i;
> @@ -564,6 +564,8 @@ enetc_dev_close(struct rte_eth_dev *dev)
>  		dev->data->tx_queues[i] = NULL;
>  	}
>  	dev->data->nb_tx_queues = 0;
> +
> +	return 0;
>  }
> 
>  static int
> diff --git a/drivers/net/enic/enic_ethdev.c b/drivers/net/enic/enic_ethdev.c
> index 57e82845a4..ed03d53608 100644
> --- a/drivers/net/enic/enic_ethdev.c
> +++ b/drivers/net/enic/enic_ethdev.c
> @@ -446,12 +446,14 @@ static void enicpmd_dev_stop(struct rte_eth_dev
> *eth_dev)
>  /*
>   * Stop device.
>   */
> -static void enicpmd_dev_close(struct rte_eth_dev *eth_dev)
> +static int enicpmd_dev_close(struct rte_eth_dev *eth_dev)
>  {
>  	struct enic *enic = pmd_priv(eth_dev);
> 
>  	ENICPMD_FUNC_TRACE();
>  	enic_remove(enic);
> +
> +	return 0;
>  }
> 
>  static int enicpmd_dev_link_update(struct rte_eth_dev *eth_dev,
> diff --git a/drivers/net/enic/enic_vf_representor.c
> b/drivers/net/enic/enic_vf_representor.c
> index 5d34e1b460..c528be2b98 100644
> --- a/drivers/net/enic/enic_vf_representor.c
> +++ b/drivers/net/enic/enic_vf_representor.c
> @@ -277,11 +277,12 @@ static void enic_vf_dev_stop(struct rte_eth_dev
> *eth_dev)
>   * "close" is no-op for now and solely exists so that rte_eth_dev_close()
>   * can finish its own cleanup without errors.
>   */
> -static void enic_vf_dev_close(struct rte_eth_dev *eth_dev __rte_unused)
> +static int enic_vf_dev_close(struct rte_eth_dev *eth_dev __rte_unused)
>  {
>  	ENICPMD_FUNC_TRACE();
>  	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
> -		return;
> +		return 0;
> +	return 0;
>  }
> 
>  static int
> diff --git a/drivers/net/failsafe/failsafe_ops.c
> b/drivers/net/failsafe/failsafe_ops.c
> index 96f7e456f9..93ebd09114 100644
> --- a/drivers/net/failsafe/failsafe_ops.c
> +++ b/drivers/net/failsafe/failsafe_ops.c
> @@ -240,7 +240,7 @@ fs_dev_set_link_down(struct rte_eth_dev *dev)
>  }
> 
>  static void fs_dev_free_queues(struct rte_eth_dev *dev);
> -static void
> +static int
>  fs_dev_close(struct rte_eth_dev *dev)
>  {
>  	struct sub_device *sdev;
> @@ -259,6 +259,7 @@ fs_dev_close(struct rte_eth_dev *dev)
>  	}
>  	fs_dev_free_queues(dev);
>  	fs_unlock(dev, 0);
> +	return 0;
>  }
> 
>  static int
> diff --git a/drivers/net/fm10k/fm10k_ethdev.c
> b/drivers/net/fm10k/fm10k_ethdev.c
> index 5771d83b55..533f976709 100644
> --- a/drivers/net/fm10k/fm10k_ethdev.c
> +++ b/drivers/net/fm10k/fm10k_ethdev.c
> @@ -2776,7 +2776,7 @@ fm10k_close_mbx_service(struct fm10k_hw *hw)
>  	hw->mbx.ops.disconnect(hw, &hw->mbx);
>  }
> 
> -static void
> +static int
>  fm10k_dev_close(struct rte_eth_dev *dev)
>  {
>  	struct fm10k_hw *hw = FM10K_DEV_PRIVATE_TO_HW(dev->data-
> >dev_private);
> @@ -2821,6 +2821,8 @@ fm10k_dev_close(struct rte_eth_dev *dev)
>  		rte_intr_callback_unregister(intr_handle,
>  			fm10k_dev_interrupt_handler_vf, (void *)dev);
>  	}
> +
> +	return 0;
>  }
> 
>  static const struct eth_dev_ops fm10k_eth_dev_ops = {
> diff --git a/drivers/net/hinic/hinic_pmd_ethdev.c
> b/drivers/net/hinic/hinic_pmd_ethdev.c
> index 67e6afcf7a..5f2e97d3bd 100644
> --- a/drivers/net/hinic/hinic_pmd_ethdev.c
> +++ b/drivers/net/hinic/hinic_pmd_ethdev.c
> @@ -2936,7 +2936,7 @@ static void hinic_nic_dev_destroy(struct
> rte_eth_dev *eth_dev)
>   * @param dev
>   *   Pointer to Ethernet device structure.
>   */
> -static void hinic_dev_close(struct rte_eth_dev *dev)
> +static int hinic_dev_close(struct rte_eth_dev *dev)
>  {
>  	struct hinic_nic_dev *nic_dev =
> HINIC_ETH_DEV_TO_PRIVATE_NIC_DEV(dev);
> 
> @@ -2944,7 +2944,7 @@ static void hinic_dev_close(struct rte_eth_dev
> *dev)
>  					   &nic_dev->dev_status)) {
>  		PMD_DRV_LOG(WARNING, "Device %s already closed",
>  			    dev->data->name);
> -		return;
> +		return 0;
>  	}
> 
>  	/* stop device first */
> @@ -2971,6 +2971,8 @@ static void hinic_dev_close(struct rte_eth_dev
> *dev)
> 
>  	/* deinit nic hardware device */
>  	hinic_nic_dev_destroy(dev);
> +
> +	return 0;
>  }
> 
>  static const struct eth_dev_ops hinic_pmd_ops = {
> diff --git a/drivers/net/hns3/hns3_ethdev.c
> b/drivers/net/hns3/hns3_ethdev.c
> index 73d504253d..3b395a1ccf 100644
> --- a/drivers/net/hns3/hns3_ethdev.c
> +++ b/drivers/net/hns3/hns3_ethdev.c
> @@ -4859,7 +4859,7 @@ hns3_dev_stop(struct rte_eth_dev *dev)
>  	rte_spinlock_unlock(&hw->lock);
>  }
> 
> -static void
> +static int
>  hns3_dev_close(struct rte_eth_dev *eth_dev)
>  {
>  	struct hns3_adapter *hns = eth_dev->data->dev_private;
> @@ -4868,7 +4868,7 @@ hns3_dev_close(struct rte_eth_dev *eth_dev)
>  	if (rte_eal_process_type() != RTE_PROC_PRIMARY) {
>  		rte_free(eth_dev->process_private);
>  		eth_dev->process_private = NULL;
> -		return;
> +		return 0;
>  	}
> 
>  	if (hw->adapter_state == HNS3_NIC_STARTED)
> @@ -4888,6 +4888,8 @@ hns3_dev_close(struct rte_eth_dev *eth_dev)
>  	eth_dev->process_private = NULL;
>  	hns3_mp_uninit_primary();
>  	hns3_warn(hw, "Close port %d finished", hw->data->port_id);
> +
> +	return 0;
>  }
> 
>  static int
> diff --git a/drivers/net/hns3/hns3_ethdev_vf.c
> b/drivers/net/hns3/hns3_ethdev_vf.c
> index 037a5be7e0..e0ca5a6ac2 100644
> --- a/drivers/net/hns3/hns3_ethdev_vf.c
> +++ b/drivers/net/hns3/hns3_ethdev_vf.c
> @@ -1910,14 +1910,14 @@ hns3vf_dev_stop(struct rte_eth_dev *dev)
>  	rte_spinlock_unlock(&hw->lock);
>  }
> 
> -static void
> +static int
>  hns3vf_dev_close(struct rte_eth_dev *eth_dev)
>  {
>  	struct hns3_adapter *hns = eth_dev->data->dev_private;
>  	struct hns3_hw *hw = &hns->hw;
> 
>  	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
> -		return;
> +		return 0;
> 
>  	if (hw->adapter_state == HNS3_NIC_STARTED)
>  		hns3vf_dev_stop(eth_dev);
> @@ -1935,6 +1935,8 @@ hns3vf_dev_close(struct rte_eth_dev *eth_dev)
>  	eth_dev->process_private = NULL;
>  	hns3_mp_uninit_primary();
>  	hns3_warn(hw, "Close port %d finished", hw->data->port_id);
> +
> +	return 0;
>  }
> 
>  static int
> diff --git a/drivers/net/i40e/i40e_ethdev.c b/drivers/net/i40e/i40e_ethdev.c
> index 563f21d9df..804f0128ab 100644
> --- a/drivers/net/i40e/i40e_ethdev.c
> +++ b/drivers/net/i40e/i40e_ethdev.c
> @@ -225,7 +225,7 @@ static int eth_i40e_dev_uninit(struct rte_eth_dev
> *eth_dev);
>  static int i40e_dev_configure(struct rte_eth_dev *dev);
>  static int i40e_dev_start(struct rte_eth_dev *dev);
>  static void i40e_dev_stop(struct rte_eth_dev *dev);
> -static void i40e_dev_close(struct rte_eth_dev *dev);
> +static int i40e_dev_close(struct rte_eth_dev *dev);
>  static int  i40e_dev_reset(struct rte_eth_dev *dev);
>  static int i40e_dev_promiscuous_enable(struct rte_eth_dev *dev);
>  static int i40e_dev_promiscuous_disable(struct rte_eth_dev *dev);
> @@ -2608,7 +2608,7 @@ i40e_dev_stop(struct rte_eth_dev *dev)
>  	pf->adapter->rss_reta_updated = 0;
>  }
> 
> -static void
> +static int
>  i40e_dev_close(struct rte_eth_dev *dev)
>  {
>  	struct i40e_pf *pf = I40E_DEV_PRIVATE_TO_PF(dev->data-
> >dev_private);
> @@ -2748,6 +2748,7 @@ i40e_dev_close(struct rte_eth_dev *dev)
>  	i40e_tm_conf_uninit(dev);
> 
>  	hw->adapter_closed = 1;
> +	return 0;
>  }
> 
>  /*
> diff --git a/drivers/net/i40e/i40e_ethdev_vf.c
> b/drivers/net/i40e/i40e_ethdev_vf.c
> index 8531cf6b12..61aad8c415 100644
> --- a/drivers/net/i40e/i40e_ethdev_vf.c
> +++ b/drivers/net/i40e/i40e_ethdev_vf.c
> @@ -90,7 +90,7 @@ static int i40evf_dev_xstats_reset(struct rte_eth_dev
> *dev);
>  static int i40evf_vlan_filter_set(struct rte_eth_dev *dev,
>  				  uint16_t vlan_id, int on);
>  static int i40evf_vlan_offload_set(struct rte_eth_dev *dev, int mask);
> -static void i40evf_dev_close(struct rte_eth_dev *dev);
> +static int i40evf_dev_close(struct rte_eth_dev *dev);
>  static int i40evf_dev_reset(struct rte_eth_dev *dev);
>  static int i40evf_check_vf_reset_done(struct rte_eth_dev *dev);
>  static int i40evf_dev_promiscuous_enable(struct rte_eth_dev *dev);
> @@ -2401,7 +2401,7 @@ i40evf_dev_stats_get(struct rte_eth_dev *dev,
> struct rte_eth_stats *stats)
>  	return ret;
>  }
> 
> -static void
> +static int
>  i40evf_dev_close(struct rte_eth_dev *dev)
>  {
>  	struct i40e_hw *hw = I40E_DEV_PRIVATE_TO_HW(dev->data-
> >dev_private);
> @@ -2433,6 +2433,7 @@ i40evf_dev_close(struct rte_eth_dev *dev)
>  	vf->aq_resp = NULL;
> 
>  	hw->adapter_closed = 1;
> +	return 0;
>  }
> 
>  /*
> diff --git a/drivers/net/iavf/iavf_ethdev.c b/drivers/net/iavf/iavf_ethdev.c
> index 440da7d76a..8f46c4588f 100644
> --- a/drivers/net/iavf/iavf_ethdev.c
> +++ b/drivers/net/iavf/iavf_ethdev.c
> @@ -32,7 +32,7 @@
>  static int iavf_dev_configure(struct rte_eth_dev *dev);
>  static int iavf_dev_start(struct rte_eth_dev *dev);
>  static void iavf_dev_stop(struct rte_eth_dev *dev);
> -static void iavf_dev_close(struct rte_eth_dev *dev);
> +static int iavf_dev_close(struct rte_eth_dev *dev);
>  static int iavf_dev_reset(struct rte_eth_dev *dev);
>  static int iavf_dev_info_get(struct rte_eth_dev *dev,
>  			     struct rte_eth_dev_info *dev_info);
> @@ -1463,7 +1463,7 @@ iavf_dev_init(struct rte_eth_dev *eth_dev)
>  	return 0;
>  }
> 
> -static void
> +static int
>  iavf_dev_close(struct rte_eth_dev *dev)
>  {
>  	struct iavf_hw *hw = IAVF_DEV_PRIVATE_TO_HW(dev->data-
> >dev_private);
> @@ -1506,6 +1506,8 @@ iavf_dev_close(struct rte_eth_dev *dev)
> 
>  	rte_free(vf->aq_resp);
>  	vf->aq_resp = NULL;
> +
> +	return 0;
>  }
> 
>  static int
> diff --git a/drivers/net/ice/ice_dcf_ethdev.c
> b/drivers/net/ice/ice_dcf_ethdev.c
> index 2faed3cc7a..5b626cb641 100644
> --- a/drivers/net/ice/ice_dcf_ethdev.c
> +++ b/drivers/net/ice/ice_dcf_ethdev.c
> @@ -847,13 +847,13 @@ ice_dcf_stats_reset(struct rte_eth_dev *dev)
>  	return 0;
>  }
> 
> -static void
> +static int
>  ice_dcf_dev_close(struct rte_eth_dev *dev)
>  {
>  	struct ice_dcf_adapter *adapter = dev->data->dev_private;
> 
>  	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
> -		return;
> +		return 0;
> 
>  	dev->dev_ops = NULL;
>  	dev->rx_pkt_burst = NULL;
> @@ -861,6 +861,8 @@ ice_dcf_dev_close(struct rte_eth_dev *dev)
> 
>  	ice_dcf_uninit_parent_adapter(dev);
>  	ice_dcf_uninit_hw(dev, &adapter->real_hw);
> +
> +	return 0;
>  }
> 
>  static int
> diff --git a/drivers/net/ice/ice_ethdev.c b/drivers/net/ice/ice_ethdev.c
> index cfd357b055..b4ac5e3f81 100644
> --- a/drivers/net/ice/ice_ethdev.c
> +++ b/drivers/net/ice/ice_ethdev.c
> @@ -77,7 +77,7 @@ static struct proto_xtr_ol_flag
> ice_proto_xtr_ol_flag_params[] = {
>  static int ice_dev_configure(struct rte_eth_dev *dev);
>  static int ice_dev_start(struct rte_eth_dev *dev);
>  static void ice_dev_stop(struct rte_eth_dev *dev);
> -static void ice_dev_close(struct rte_eth_dev *dev);
> +static int ice_dev_close(struct rte_eth_dev *dev);
>  static int ice_dev_reset(struct rte_eth_dev *dev);
>  static int ice_dev_info_get(struct rte_eth_dev *dev,
>  			    struct rte_eth_dev_info *dev_info);
> @@ -2430,7 +2430,7 @@ ice_dev_stop(struct rte_eth_dev *dev)
>  	pf->adapter_stopped = true;
>  }
> 
> -static void
> +static int
>  ice_dev_close(struct rte_eth_dev *dev)
>  {
>  	struct ice_pf *pf = ICE_DEV_PRIVATE_TO_PF(dev->data-
> >dev_private);
> @@ -2478,6 +2478,8 @@ ice_dev_close(struct rte_eth_dev *dev)
>  	/* unregister callback func from eal lib */
>  	rte_intr_callback_unregister(intr_handle,
>  				     ice_interrupt_handler, dev);
> +
> +	return 0;
>  }
> 
>  static int
> diff --git a/drivers/net/igc/igc_ethdev.c b/drivers/net/igc/igc_ethdev.c
> index 810568bc54..11397c5c1d 100644
> --- a/drivers/net/igc/igc_ethdev.c
> +++ b/drivers/net/igc/igc_ethdev.c
> @@ -183,7 +183,7 @@ static void eth_igc_stop(struct rte_eth_dev *dev);
>  static int eth_igc_start(struct rte_eth_dev *dev);
>  static int eth_igc_set_link_up(struct rte_eth_dev *dev);
>  static int eth_igc_set_link_down(struct rte_eth_dev *dev);
> -static void eth_igc_close(struct rte_eth_dev *dev);
> +static int eth_igc_close(struct rte_eth_dev *dev);
>  static int eth_igc_reset(struct rte_eth_dev *dev);
>  static int eth_igc_promiscuous_enable(struct rte_eth_dev *dev);
>  static int eth_igc_promiscuous_disable(struct rte_eth_dev *dev);
> @@ -1165,7 +1165,7 @@ igc_dev_free_queues(struct rte_eth_dev *dev)
>  	dev->data->nb_tx_queues = 0;
>  }
> 
> -static void
> +static int
>  eth_igc_close(struct rte_eth_dev *dev)
>  {
>  	struct rte_pci_device *pci_dev = RTE_ETH_DEV_TO_PCI(dev);
> @@ -1199,6 +1199,8 @@ eth_igc_close(struct rte_eth_dev *dev)
> 
>  	/* Reset any pending lock */
>  	igc_reset_swfw_lock(hw);
> +
> +	return 0;
>  }
> 
>  static void
> diff --git a/drivers/net/ionic/ionic_ethdev.c
> b/drivers/net/ionic/ionic_ethdev.c
> index 800f6e5d64..1775fd29b7 100644
> --- a/drivers/net/ionic/ionic_ethdev.c
> +++ b/drivers/net/ionic/ionic_ethdev.c
> @@ -25,7 +25,7 @@ static int  ionic_dev_configure(struct rte_eth_dev *dev);
>  static int  ionic_dev_mtu_set(struct rte_eth_dev *dev, uint16_t mtu);
>  static int  ionic_dev_start(struct rte_eth_dev *dev);
>  static void ionic_dev_stop(struct rte_eth_dev *dev);
> -static void ionic_dev_close(struct rte_eth_dev *dev);
> +static int  ionic_dev_close(struct rte_eth_dev *dev);
>  static int  ionic_dev_set_link_up(struct rte_eth_dev *dev);
>  static int  ionic_dev_set_link_down(struct rte_eth_dev *dev);
>  static int  ionic_dev_link_update(struct rte_eth_dev *eth_dev,
> @@ -956,7 +956,7 @@ ionic_dev_stop(struct rte_eth_dev *eth_dev)
>  /*
>   * Reset and stop device.
>   */
> -static void
> +static int
>  ionic_dev_close(struct rte_eth_dev *eth_dev)
>  {
>  	struct ionic_lif *lif = IONIC_ETH_DEV_TO_LIF(eth_dev);
> @@ -967,14 +967,16 @@ ionic_dev_close(struct rte_eth_dev *eth_dev)
>  	err = ionic_lif_stop(lif);
>  	if (err) {
>  		IONIC_PRINT(ERR, "Cannot stop LIF: %d", err);
> -		return;
> +		return -1;
>  	}
> 
>  	err = eth_ionic_dev_uninit(eth_dev);
>  	if (err) {
>  		IONIC_PRINT(ERR, "Cannot destroy LIF: %d", err);
> -		return;
> +		return -1;
>  	}
> +
> +	return 0;
>  }
> 
>  static int
> diff --git a/drivers/net/ipn3ke/ipn3ke_representor.c
> b/drivers/net/ipn3ke/ipn3ke_representor.c
> index b673c49149..d49abbf758 100644
> --- a/drivers/net/ipn3ke/ipn3ke_representor.c
> +++ b/drivers/net/ipn3ke/ipn3ke_representor.c
> @@ -208,7 +208,7 @@ ipn3ke_rpst_dev_stop(struct rte_eth_dev *dev)
>  	}
>  }
> 
> -static void
> +static int
>  ipn3ke_rpst_dev_close(struct rte_eth_dev *dev)
>  {
>  	struct ipn3ke_hw *hw = IPN3KE_DEV_PRIVATE_TO_HW(dev);
> @@ -221,6 +221,8 @@ ipn3ke_rpst_dev_close(struct rte_eth_dev *dev)
>  		/* Disable the RX path */
>  		ipn3ke_xmac_rx_disable(hw, rpst->port_id, 0);
>  	}
> +
> +	return 0;
>  }
> 
>  /*
> diff --git a/drivers/net/ixgbe/ixgbe_ethdev.c
> b/drivers/net/ixgbe/ixgbe_ethdev.c
> index c74467e068..5a863ca4bf 100644
> --- a/drivers/net/ixgbe/ixgbe_ethdev.c
> +++ b/drivers/net/ixgbe/ixgbe_ethdev.c
> @@ -147,7 +147,7 @@ static int  ixgbe_dev_start(struct rte_eth_dev *dev);
>  static void ixgbe_dev_stop(struct rte_eth_dev *dev);
>  static int  ixgbe_dev_set_link_up(struct rte_eth_dev *dev);
>  static int  ixgbe_dev_set_link_down(struct rte_eth_dev *dev);
> -static void ixgbe_dev_close(struct rte_eth_dev *dev);
> +static int  ixgbe_dev_close(struct rte_eth_dev *dev);
>  static int  ixgbe_dev_reset(struct rte_eth_dev *dev);
>  static int ixgbe_dev_promiscuous_enable(struct rte_eth_dev *dev);
>  static int ixgbe_dev_promiscuous_disable(struct rte_eth_dev *dev);
> @@ -251,7 +251,7 @@ static int  ixgbevf_dev_start(struct rte_eth_dev *dev);
>  static int ixgbevf_dev_link_update(struct rte_eth_dev *dev,
>  				   int wait_to_complete);
>  static void ixgbevf_dev_stop(struct rte_eth_dev *dev);
> -static void ixgbevf_dev_close(struct rte_eth_dev *dev);
> +static int ixgbevf_dev_close(struct rte_eth_dev *dev);
>  static int  ixgbevf_dev_reset(struct rte_eth_dev *dev);
>  static void ixgbevf_intr_disable(struct rte_eth_dev *dev);
>  static void ixgbevf_intr_enable(struct rte_eth_dev *dev);
> @@ -2994,7 +2994,7 @@ ixgbe_dev_set_link_down(struct rte_eth_dev
> *dev)
>  /*
>   * Reset and stop device.
>   */
> -static void
> +static int
>  ixgbe_dev_close(struct rte_eth_dev *dev)
>  {
>  	struct ixgbe_hw *hw =
> @@ -3065,6 +3065,7 @@ ixgbe_dev_close(struct rte_eth_dev *dev)
>  	rte_free(dev->security_ctx);
>  #endif
> 
> +	return 0;
>  }
> 
>  /*
> @@ -5443,7 +5444,7 @@ ixgbevf_dev_stop(struct rte_eth_dev *dev)
>  	adapter->rss_reta_updated = 0;
>  }
> 
> -static void
> +static int
>  ixgbevf_dev_close(struct rte_eth_dev *dev)
>  {
>  	struct ixgbe_hw *hw = IXGBE_DEV_PRIVATE_TO_HW(dev->data-
> >dev_private);
> @@ -5472,6 +5473,8 @@ ixgbevf_dev_close(struct rte_eth_dev *dev)
>  	rte_intr_disable(intr_handle);
>  	rte_intr_callback_unregister(intr_handle,
>  				     ixgbevf_dev_interrupt_handler, dev);
> +
> +	return 0;
>  }
> 
>  /*
> diff --git a/drivers/net/kni/rte_eth_kni.c b/drivers/net/kni/rte_eth_kni.c
> index 831fe96c96..aa365a57de 100644
> --- a/drivers/net/kni/rte_eth_kni.c
> +++ b/drivers/net/kni/rte_eth_kni.c
> @@ -198,7 +198,7 @@ eth_kni_dev_stop(struct rte_eth_dev *dev)
>  	dev->data->dev_link.link_status = 0;
>  }
> 
> -static void
> +static int
>  eth_kni_close(struct rte_eth_dev *eth_dev)
>  {
>  	struct pmd_internals *internals;
> @@ -214,6 +214,8 @@ eth_kni_close(struct rte_eth_dev *eth_dev)
>  	if (ret)
>  		PMD_LOG(WARNING, "Not able to release kni for %s",
>  			eth_dev->data->name);
> +
> +	return 0;
>  }
> 
>  static int
> diff --git a/drivers/net/liquidio/lio_ethdev.c
> b/drivers/net/liquidio/lio_ethdev.c
> index 2c2b27e62a..93e2ed5670 100644
> --- a/drivers/net/liquidio/lio_ethdev.c
> +++ b/drivers/net/liquidio/lio_ethdev.c
> @@ -1550,7 +1550,7 @@ lio_dev_set_link_down(struct rte_eth_dev
> *eth_dev)
>   * @return
>   *    - nothing
>   */
> -static void
> +static int
>  lio_dev_close(struct rte_eth_dev *eth_dev)
>  {
>  	struct lio_device *lio_dev = LIO_DEV(eth_dev);
> @@ -1581,6 +1581,8 @@ lio_dev_close(struct rte_eth_dev *eth_dev)
> 
>  	 /* Delete all queues */
>  	lio_dev_clear_queues(eth_dev);
> +
> +	return 0;
>  }
> 
>  /**
> diff --git a/drivers/net/memif/rte_eth_memif.c
> b/drivers/net/memif/rte_eth_memif.c
> index a19c0f3e6b..1a2411c838 100644
> --- a/drivers/net/memif/rte_eth_memif.c
> +++ b/drivers/net/memif/rte_eth_memif.c
> @@ -1218,7 +1218,7 @@ memif_dev_start(struct rte_eth_dev *dev)
>  	return ret;
>  }
> 
> -static void
> +static int
>  memif_dev_close(struct rte_eth_dev *dev)
>  {
>  	struct pmd_internals *pmd = dev->data->dev_private;
> @@ -1239,6 +1239,8 @@ memif_dev_close(struct rte_eth_dev *dev)
>  	}
> 
>  	rte_free(dev->process_private);
> +
> +	return 0;
>  }
> 
>  static int
> diff --git a/drivers/net/mlx4/mlx4.c b/drivers/net/mlx4/mlx4.c
> index 71061a720f..df59314b66 100644
> --- a/drivers/net/mlx4/mlx4.c
> +++ b/drivers/net/mlx4/mlx4.c
> @@ -370,7 +370,7 @@ mlx4_dev_stop(struct rte_eth_dev *dev)
>   * @param dev
>   *   Pointer to Ethernet device structure.
>   */
> -static void
> +static int
>  mlx4_dev_close(struct rte_eth_dev *dev)
>  {
>  	struct mlx4_priv *priv = dev->data->dev_private;
> @@ -400,6 +400,7 @@ mlx4_dev_close(struct rte_eth_dev *dev)
>  		MLX4_ASSERT(priv->ctx == NULL);
>  	mlx4_intr_uninstall(priv);
>  	memset(priv, 0, sizeof(*priv));
> +	return 0;
>  }
> 
>  static const struct eth_dev_ops mlx4_dev_ops = {
> diff --git a/drivers/net/mlx5/mlx5.c b/drivers/net/mlx5/mlx5.c
> index 4a807fb4fd..01ead6e6af 100644
> --- a/drivers/net/mlx5/mlx5.c
> +++ b/drivers/net/mlx5/mlx5.c
> @@ -1341,7 +1341,7 @@ mlx5_proc_priv_uninit(struct rte_eth_dev *dev)
>   * @param dev
>   *   Pointer to Ethernet device structure.
>   */
> -void
> +int
>  mlx5_dev_close(struct rte_eth_dev *dev)
>  {
>  	struct mlx5_priv *priv = dev->data->dev_private;
> @@ -1351,14 +1351,14 @@ mlx5_dev_close(struct rte_eth_dev *dev)
>  	if (rte_eal_process_type() == RTE_PROC_SECONDARY) {
>  		/* Check if process_private released. */
>  		if (!dev->process_private)
> -			return;
> +			return 0;
>  		mlx5_tx_uar_uninit_secondary(dev);
>  		mlx5_proc_priv_uninit(dev);
>  		rte_eth_dev_release_port(dev);
> -		return;
> +		return 0;
>  	}
>  	if (!priv->sh)
> -		return;
> +		return 0;
>  	DRV_LOG(DEBUG, "port %u closing device \"%s\"",
>  		dev->data->port_id,
>  		((priv->sh->ctx != NULL) ?
> @@ -1479,6 +1479,7 @@ mlx5_dev_close(struct rte_eth_dev *dev)
>  	 * it is freed when dev_private is freed.
>  	 */
>  	dev->data->mac_addrs = NULL;
> +	return 0;
>  }
> 
>  /**
> diff --git a/drivers/net/mlx5/mlx5.h b/drivers/net/mlx5/mlx5.h
> index 865e72d318..56f0e25b4c 100644
> --- a/drivers/net/mlx5/mlx5.h
> +++ b/drivers/net/mlx5/mlx5.h
> @@ -842,7 +842,7 @@ int mlx5_proc_priv_init(struct rte_eth_dev *dev);
>  int mlx5_udp_tunnel_port_add(struct rte_eth_dev *dev,
>  			      struct rte_eth_udp_tunnel *udp_tunnel);
>  uint16_t mlx5_eth_find_next(uint16_t port_id, struct rte_pci_device
> *pci_dev);
> -void mlx5_dev_close(struct rte_eth_dev *dev);
> +int mlx5_dev_close(struct rte_eth_dev *dev);
> 
>  /* Macro to iterate over all valid ports for mlx5 driver. */
>  #define MLX5_ETH_FOREACH_DEV(port_id, pci_dev) \
> diff --git a/drivers/net/mvneta/mvneta_ethdev.c
> b/drivers/net/mvneta/mvneta_ethdev.c
> index 5bf3ebc4bd..eebcdb840e 100644
> --- a/drivers/net/mvneta/mvneta_ethdev.c
> +++ b/drivers/net/mvneta/mvneta_ethdev.c
> @@ -429,7 +429,7 @@ mvneta_dev_stop(struct rte_eth_dev *dev)
>   * @param dev
>   *   Pointer to Ethernet device structure.
>   */
> -static void
> +static int
>  mvneta_dev_close(struct rte_eth_dev *dev)
>  {
>  	struct mvneta_priv *priv = dev->data->dev_private;
> @@ -455,6 +455,8 @@ mvneta_dev_close(struct rte_eth_dev *dev)
>  		mvneta_neta_deinit();
>  		rte_mvep_deinit(MVEP_MOD_T_NETA);
>  	}
> +
> +	return 0;
>  }
> 
>  /**
> diff --git a/drivers/net/mvpp2/mrvl_ethdev.c
> b/drivers/net/mvpp2/mrvl_ethdev.c
> index f5e77d07bf..27f7cfb056 100644
> --- a/drivers/net/mvpp2/mrvl_ethdev.c
> +++ b/drivers/net/mvpp2/mrvl_ethdev.c
> @@ -855,7 +855,7 @@ mrvl_dev_stop(struct rte_eth_dev *dev)
>   * @param dev
>   *   Pointer to Ethernet device structure.
>   */
> -static void
> +static int
>  mrvl_dev_close(struct rte_eth_dev *dev)
>  {
>  	struct mrvl_priv *priv = dev->data->dev_private;
> @@ -915,6 +915,8 @@ mrvl_dev_close(struct rte_eth_dev *dev)
>  		mrvl_deinit_pp2();
>  		rte_mvep_deinit(MVEP_MOD_T_PP2);
>  	}
> +
> +	return 0;
>  }
> 
>  /**
> diff --git a/drivers/net/netvsc/hn_ethdev.c
> b/drivers/net/netvsc/hn_ethdev.c
> index 229c1b9149..8968036ea3 100644
> --- a/drivers/net/netvsc/hn_ethdev.c
> +++ b/drivers/net/netvsc/hn_ethdev.c
> @@ -841,13 +841,15 @@ hn_dev_stop(struct rte_eth_dev *dev)
>  	hn_vf_stop(dev);
>  }
> 
> -static void
> +static int
>  hn_dev_close(struct rte_eth_dev *dev)
>  {
>  	PMD_INIT_FUNC_TRACE();
> 
>  	hn_vf_close(dev);
>  	hn_dev_free_queues(dev);
> +
> +	return 0;
>  }
> 
>  static const struct eth_dev_ops hn_eth_dev_ops = {
> diff --git a/drivers/net/nfb/nfb_ethdev.c b/drivers/net/nfb/nfb_ethdev.c
> index b039ab6fc2..c3c3d003f1 100644
> --- a/drivers/net/nfb/nfb_ethdev.c
> +++ b/drivers/net/nfb/nfb_ethdev.c
> @@ -209,7 +209,7 @@ nfb_eth_dev_info(struct rte_eth_dev *dev,
>   * @param dev
>   *   Pointer to Ethernet device structure.
>   */
> -static void
> +static int
>  nfb_eth_dev_close(struct rte_eth_dev *dev)
>  {
>  	struct pmd_internals *internals = dev->data->dev_private;
> @@ -235,6 +235,8 @@ nfb_eth_dev_close(struct rte_eth_dev *dev)
> 
>  	rte_free(dev->data->mac_addrs);
>  	dev->data->mac_addrs = NULL;
> +
> +	return 0;
>  }
> 
>  /**
> diff --git a/drivers/net/nfp/nfp_net.c b/drivers/net/nfp/nfp_net.c
> index f5e4cd339d..9509dc8bd6 100644
> --- a/drivers/net/nfp/nfp_net.c
> +++ b/drivers/net/nfp/nfp_net.c
> @@ -50,7 +50,7 @@
>  #include <errno.h>
> 
>  /* Prototypes */
> -static void nfp_net_close(struct rte_eth_dev *dev);
> +static int nfp_net_close(struct rte_eth_dev *dev);
>  static int nfp_net_configure(struct rte_eth_dev *dev);
>  static void nfp_net_dev_interrupt_handler(void *param);
>  static void nfp_net_dev_interrupt_delayed_handler(void *param);
> @@ -864,7 +864,7 @@ nfp_net_set_link_down(struct rte_eth_dev *dev)
>  }
> 
>  /* Reset and stop device. The device can not be restarted. */
> -static void
> +static int
>  nfp_net_close(struct rte_eth_dev *dev)
>  {
>  	struct nfp_net_hw *hw;
> @@ -906,6 +906,8 @@ nfp_net_close(struct rte_eth_dev *dev)
>  	 * The ixgbe PMD driver disables the pcie master on the
>  	 * device. The i40e does not...
>  	 */
> +
> +	return 0;
>  }
> 
>  static int
> diff --git a/drivers/net/octeontx/octeontx_ethdev.c
> b/drivers/net/octeontx/octeontx_ethdev.c
> index ff2e22cd44..10eeeac642 100644
> --- a/drivers/net/octeontx/octeontx_ethdev.c
> +++ b/drivers/net/octeontx/octeontx_ethdev.c
> @@ -478,7 +478,7 @@ octeontx_dev_configure(struct rte_eth_dev *dev)
>  	return 0;
>  }
> 
> -static void
> +static int
>  octeontx_dev_close(struct rte_eth_dev *dev)
>  {
>  	struct octeontx_txq *txq = NULL;
> @@ -517,6 +517,8 @@ octeontx_dev_close(struct rte_eth_dev *dev)
> 
>  	dev->tx_pkt_burst = NULL;
>  	dev->rx_pkt_burst = NULL;
> +
> +	return 0;
>  }
> 
>  static int
> diff --git a/drivers/net/octeontx2/otx2_ethdev.c
> b/drivers/net/octeontx2/otx2_ethdev.c
> index 51a6f99877..abe5f03628 100644
> --- a/drivers/net/octeontx2/otx2_ethdev.c
> +++ b/drivers/net/octeontx2/otx2_ethdev.c
> @@ -2222,7 +2222,7 @@ otx2_nix_dev_start(struct rte_eth_dev *eth_dev)
>  }
> 
>  static int otx2_nix_dev_reset(struct rte_eth_dev *eth_dev);
> -static void otx2_nix_dev_close(struct rte_eth_dev *eth_dev);
> +static int otx2_nix_dev_close(struct rte_eth_dev *eth_dev);
> 
>  /* Initialize and register driver with DPDK Application */
>  static const struct eth_dev_ops otx2_eth_dev_ops = {
> @@ -2635,10 +2635,11 @@ otx2_eth_dev_uninit(struct rte_eth_dev
> *eth_dev, bool mbox_close)
>  	return 0;
>  }
> 
> -static void
> +static int
>  otx2_nix_dev_close(struct rte_eth_dev *eth_dev)
>  {
>  	otx2_eth_dev_uninit(eth_dev, true);
> +	return 0;
>  }
> 
>  static int
> diff --git a/drivers/net/pcap/rte_eth_pcap.c
> b/drivers/net/pcap/rte_eth_pcap.c
> index 668cbd1fc7..76e704a65a 100644
> --- a/drivers/net/pcap/rte_eth_pcap.c
> +++ b/drivers/net/pcap/rte_eth_pcap.c
> @@ -728,7 +728,7 @@ eth_stats_reset(struct rte_eth_dev *dev)
>  	return 0;
>  }
> 
> -static void
> +static int
>  eth_dev_close(struct rte_eth_dev *dev)
>  {
>  	unsigned int i;
> @@ -748,6 +748,7 @@ eth_dev_close(struct rte_eth_dev *dev)
>  		}
>  	}
> 
> +	return 0;
>  }
> 
>  static void
> diff --git a/drivers/net/pfe/pfe_ethdev.c b/drivers/net/pfe/pfe_ethdev.c
> index 97356d2b0b..187a0019ff 100644
> --- a/drivers/net/pfe/pfe_ethdev.c
> +++ b/drivers/net/pfe/pfe_ethdev.c
> @@ -398,14 +398,14 @@ pfe_eth_exit(struct rte_eth_dev *dev, struct pfe
> *pfe)
>  	pfe->nb_devs--;
>  }
> 
> -static void
> +static int
>  pfe_eth_close(struct rte_eth_dev *dev)
>  {
>  	if (!dev)
> -		return;
> +		return -1;
> 
>  	if (!g_pfe)
> -		return;
> +		return -1;
> 
>  	pfe_eth_exit(dev, g_pfe);
> 
> @@ -415,6 +415,8 @@ pfe_eth_close(struct rte_eth_dev *dev)
>  		rte_free(g_pfe);
>  		g_pfe = NULL;
>  	}
> +
> +	return 0;
>  }
> 
>  static int
> diff --git a/drivers/net/qede/qede_ethdev.c
> b/drivers/net/qede/qede_ethdev.c
> index 59f1746ee9..9535df183f 100644
> --- a/drivers/net/qede/qede_ethdev.c
> +++ b/drivers/net/qede/qede_ethdev.c
> @@ -1542,7 +1542,7 @@ static void qede_poll_sp_sb_cb(void *param)
>  	}
>  }
> 
> -static void qede_dev_close(struct rte_eth_dev *eth_dev)
> +static int qede_dev_close(struct rte_eth_dev *eth_dev)
>  {
>  	struct rte_pci_device *pci_dev = RTE_ETH_DEV_TO_PCI(eth_dev);
>  	struct qede_dev *qdev = QEDE_INIT_QDEV(eth_dev);
> @@ -1586,6 +1586,8 @@ static void qede_dev_close(struct rte_eth_dev
> *eth_dev)
> 
>  	if (ECORE_IS_CMT(edev))
>  		rte_eal_alarm_cancel(qede_poll_sp_sb_cb, (void
> *)eth_dev);
> +
> +	return 0;
>  }
> 
>  static int
> diff --git a/drivers/net/sfc/sfc_ethdev.c b/drivers/net/sfc/sfc_ethdev.c
> index 1a58e0df84..eaa48627d6 100644
> --- a/drivers/net/sfc/sfc_ethdev.c
> +++ b/drivers/net/sfc/sfc_ethdev.c
> @@ -318,7 +318,7 @@ sfc_dev_set_link_down(struct rte_eth_dev *dev)
>  	return 0;
>  }
> 
> -static void
> +static int
>  sfc_dev_close(struct rte_eth_dev *dev)
>  {
>  	struct sfc_adapter *sa = sfc_adapter_by_eth_dev(dev);
> @@ -364,6 +364,8 @@ sfc_dev_close(struct rte_eth_dev *dev)
> 
>  	dev->process_private = NULL;
>  	free(sa);
> +
> +	return 0;
>  }
> 
>  static int
> diff --git a/drivers/net/softnic/rte_eth_softnic.c
> b/drivers/net/softnic/rte_eth_softnic.c
> index e453fd1486..038abd98da 100644
> --- a/drivers/net/softnic/rte_eth_softnic.c
> +++ b/drivers/net/softnic/rte_eth_softnic.c
> @@ -201,10 +201,10 @@ pmd_dev_stop(struct rte_eth_dev *dev)
>  	softnic_mtr_free(p);
>  }
> 
> -static void
> +static int
>  pmd_dev_close(struct rte_eth_dev *dev __rte_unused)
>  {
> -	return;
> +	return 0;
>  }
> 
>  static int
> diff --git a/drivers/net/szedata2/rte_eth_szedata2.c
> b/drivers/net/szedata2/rte_eth_szedata2.c
> index a17c53577c..daa40cfd6c 100644
> --- a/drivers/net/szedata2/rte_eth_szedata2.c
> +++ b/drivers/net/szedata2/rte_eth_szedata2.c
> @@ -1155,7 +1155,7 @@ eth_tx_queue_release(void *q)
>  	}
>  }
> 
> -static void
> +static int
>  eth_dev_close(struct rte_eth_dev *dev)
>  {
>  	struct pmd_internals *internals = dev->data->dev_private;
> @@ -1180,6 +1180,8 @@ eth_dev_close(struct rte_eth_dev *dev)
> 
>  	rte_free(dev->data->mac_addrs);
>  	dev->data->mac_addrs = NULL;
> +
> +	return 0;
>  }
> 
>  static int
> diff --git a/drivers/net/tap/rte_eth_tap.c b/drivers/net/tap/rte_eth_tap.c
> index df16aa4ea7..2d1e4e365a 100644
> --- a/drivers/net/tap/rte_eth_tap.c
> +++ b/drivers/net/tap/rte_eth_tap.c
> @@ -1067,7 +1067,7 @@ tap_stats_reset(struct rte_eth_dev *dev)
>  	return 0;
>  }
> 
> -static void
> +static int
>  tap_dev_close(struct rte_eth_dev *dev)
>  {
>  	int i;
> @@ -1116,6 +1116,8 @@ tap_dev_close(struct rte_eth_dev *dev)
>  	 * Since TUN device has no more opened file descriptors
>  	 * it will be removed from kernel
>  	 */
> +
> +	return 0;
>  }
> 
>  static void
> diff --git a/drivers/net/thunderx/nicvf_ethdev.c
> b/drivers/net/thunderx/nicvf_ethdev.c
> index 83d9488360..3d7348771a 100644
> --- a/drivers/net/thunderx/nicvf_ethdev.c
> +++ b/drivers/net/thunderx/nicvf_ethdev.c
> @@ -1852,7 +1852,7 @@ nicvf_vf_stop(struct rte_eth_dev *dev, struct nicvf
> *nic, bool cleanup)
>  	}
>  }
> 
> -static void
> +static int
>  nicvf_dev_close(struct rte_eth_dev *dev)
>  {
>  	size_t i;
> @@ -1869,6 +1869,8 @@ nicvf_dev_close(struct rte_eth_dev *dev)
> 
>  		nicvf_periodic_alarm_stop(nicvf_vf_interrupt, nic->snicvf[i]);
>  	}
> +
> +	return 0;
>  }
> 
>  static int
> diff --git a/drivers/net/vhost/rte_eth_vhost.c
> b/drivers/net/vhost/rte_eth_vhost.c
> index ce32be9ce3..f67f4db812 100644
> --- a/drivers/net/vhost/rte_eth_vhost.c
> +++ b/drivers/net/vhost/rte_eth_vhost.c
> @@ -1164,7 +1164,7 @@ eth_dev_stop(struct rte_eth_dev *dev)
>  	update_queuing_status(dev);
>  }
> 
> -static void
> +static int
>  eth_dev_close(struct rte_eth_dev *dev)
>  {
>  	struct pmd_internal *internal;
> @@ -1173,7 +1173,7 @@ eth_dev_close(struct rte_eth_dev *dev)
> 
>  	internal = dev->data->dev_private;
>  	if (!internal)
> -		return;
> +		return 0;
> 
>  	eth_dev_stop(dev);
> 
> @@ -1201,6 +1201,8 @@ eth_dev_close(struct rte_eth_dev *dev)
> 
>  	rte_free(vring_states[dev->data->port_id]);
>  	vring_states[dev->data->port_id] = NULL;
> +
> +	return 0;
>  }
> 
>  static int
> diff --git a/drivers/net/virtio/virtio_ethdev.c
> b/drivers/net/virtio/virtio_ethdev.c
> index 013a2904e6..0787337a01 100644
> --- a/drivers/net/virtio/virtio_ethdev.c
> +++ b/drivers/net/virtio/virtio_ethdev.c
> @@ -704,7 +704,7 @@ virtio_alloc_queues(struct rte_eth_dev *dev)
> 
>  static void virtio_queues_unbind_intr(struct rte_eth_dev *dev);
> 
> -static void
> +static int
>  virtio_dev_close(struct rte_eth_dev *dev)
>  {
>  	struct virtio_hw *hw = dev->data->dev_private;
> @@ -713,7 +713,7 @@ virtio_dev_close(struct rte_eth_dev *dev)
>  	PMD_INIT_LOG(DEBUG, "virtio_dev_close");
> 
>  	if (!hw->opened)
> -		return;
> +		return 0;
>  	hw->opened = false;
> 
>  	/* reset the NIC */
> @@ -743,6 +743,8 @@ virtio_dev_close(struct rte_eth_dev *dev)
>  		if (!hw->modern)
>  			rte_pci_ioport_unmap(VTPCI_IO(hw));
>  	}
> +
> +	return 0;
>  }
> 
>  static int
> diff --git a/drivers/net/vmxnet3/vmxnet3_ethdev.c
> b/drivers/net/vmxnet3/vmxnet3_ethdev.c
> index fa80e75a5b..17fed0ed6e 100644
> --- a/drivers/net/vmxnet3/vmxnet3_ethdev.c
> +++ b/drivers/net/vmxnet3/vmxnet3_ethdev.c
> @@ -64,7 +64,7 @@ static int eth_vmxnet3_dev_uninit(struct rte_eth_dev
> *eth_dev);
>  static int vmxnet3_dev_configure(struct rte_eth_dev *dev);
>  static int vmxnet3_dev_start(struct rte_eth_dev *dev);
>  static void vmxnet3_dev_stop(struct rte_eth_dev *dev);
> -static void vmxnet3_dev_close(struct rte_eth_dev *dev);
> +static int vmxnet3_dev_close(struct rte_eth_dev *dev);
>  static void vmxnet3_dev_set_rxmode(struct vmxnet3_hw *hw, uint32_t
> feature, int set);
>  static int vmxnet3_dev_promiscuous_enable(struct rte_eth_dev *dev);
>  static int vmxnet3_dev_promiscuous_disable(struct rte_eth_dev *dev);
> @@ -888,13 +888,15 @@ vmxnet3_free_queues(struct rte_eth_dev *dev)
>  /*
>   * Reset and stop device.
>   */
> -static void
> +static int
>  vmxnet3_dev_close(struct rte_eth_dev *dev)
>  {
>  	PMD_INIT_FUNC_TRACE();
> 
>  	vmxnet3_dev_stop(dev);
>  	vmxnet3_free_queues(dev);
> +
> +	return 0;
>  }
> 
>  static void
> diff --git a/lib/librte_ethdev/rte_ethdev_driver.h
> b/lib/librte_ethdev/rte_ethdev_driver.h
> index 04ac8e923c..c3062c246c 100644
> --- a/lib/librte_ethdev/rte_ethdev_driver.h
> +++ b/lib/librte_ethdev/rte_ethdev_driver.h
> @@ -42,7 +42,7 @@ typedef int  (*eth_dev_set_link_up_t)(struct
> rte_eth_dev *dev);
>  typedef int  (*eth_dev_set_link_down_t)(struct rte_eth_dev *dev);
>  /**< @internal Function used to link down a configured Ethernet device. */
> 
> -typedef void (*eth_dev_close_t)(struct rte_eth_dev *dev);
> +typedef int (*eth_dev_close_t)(struct rte_eth_dev *dev);
>  /**< @internal Function used to close a configured Ethernet device. */
> 
>  typedef int (*eth_dev_reset_t)(struct rte_eth_dev *dev);
> --
> 2.28.0

Re: [dpdk-dev] [PATCH v3 02/29] ethdev: allow drivers to return error on close

[Andrew Rybchenko]

On 9/29/20 2:14 AM, Thomas Monjalon wrote:
> The device operation .dev_close was returning void.
> This driver interface is changed to return an int.
> 
> Note that the API rte_eth_dev_close() is still returning void,
> although a deprecation notice is pending to change it as well.
> 
> Signed-off-by: Thomas Monjalon <thomas@monjalon.net>
> Reviewed-by: Rosen Xu <rosen.xu@intel.com>
> Reviewed-by: Sachin Saxena <sachin.saxena@oss.nxp.com>
> Reviewed-by: Liron Himi <lironh@marvell.com>

Reviewed-by: Andrew Rybchenko <arybchenko@solarflare.com>

Re: [dpdk-dev] [PATCH v3 02/29] ethdev: allow drivers to return error on close

[Ferruh Yigit]

On 9/29/2020 11:53 AM, Andrew Rybchenko wrote:
> On 9/29/20 2:14 AM, Thomas Monjalon wrote:
>> The device operation .dev_close was returning void.
>> This driver interface is changed to return an int.
>>
>> Note that the API rte_eth_dev_close() is still returning void,
>> although a deprecation notice is pending to change it as well.
>>
>> Signed-off-by: Thomas Monjalon <thomas@monjalon.net>
>> Reviewed-by: Rosen Xu <rosen.xu@intel.com>
>> Reviewed-by: Sachin Saxena <sachin.saxena@oss.nxp.com>
>> Reviewed-by: Liron Himi <lironh@marvell.com>
> 
> Reviewed-by: Andrew Rybchenko <arybchenko@solarflare.com>
> 

Reviewed-by: Ferruh Yigit <ferruh.yigit@intel.com>

[dpdk-dev] [PATCH v3 03/29] net/af_packet: release port upon close

[Thomas Monjalon]

The flag RTE_ETH_DEV_CLOSE_REMOVE is set so all port resources
can be freed by rte_eth_dev_close().

Freeing of private port resources is moved
from the ".remove(device)" to the ".dev_close(port)" operation.

Signed-off-by: Thomas Monjalon <thomas@monjalon.net>
Reviewed-by: Ferruh Yigit <ferruh.yigit@intel.com>
---
 drivers/net/af_packet/rte_eth_af_packet.c | 56 ++++++++++++-----------
 1 file changed, 29 insertions(+), 27 deletions(-)

diff --git a/drivers/net/af_packet/rte_eth_af_packet.c b/drivers/net/af_packet/rte_eth_af_packet.c
index 7d0ff1cbb3..12c202cba4 100644
--- a/drivers/net/af_packet/rte_eth_af_packet.c
+++ b/drivers/net/af_packet/rte_eth_af_packet.c
@@ -377,8 +377,32 @@ eth_stats_reset(struct rte_eth_dev *dev)
 }
 
 static int
-eth_dev_close(struct rte_eth_dev *dev __rte_unused)
+eth_dev_close(struct rte_eth_dev *dev)
 {
+	struct pmd_internals *internals;
+	struct tpacket_req *req;
+	unsigned int q;
+
+	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
+		return 0;
+
+	PMD_LOG(INFO, "Closing AF_PACKET ethdev on NUMA socket %u",
+		rte_socket_id());
+
+	internals = dev->data->dev_private;
+	req = &internals->req;
+	for (q = 0; q < internals->nb_queues; q++) {
+		munmap(internals->rx_queue[q].map,
+			2 * req->tp_block_size * req->tp_block_nr);
+		rte_free(internals->rx_queue[q].rd);
+		rte_free(internals->tx_queue[q].rd);
+	}
+	free(internals->if_name);
+	rte_free(internals->rx_queue);
+	rte_free(internals->tx_queue);
+
+	/* mac_addrs must not be freed alone because part of dev_private */
+	dev->data->mac_addrs = NULL;
 	return 0;
 }
 
@@ -835,6 +859,7 @@ rte_pmd_init_internals(struct rte_vdev_device *dev,
 	data->nb_tx_queues = (uint16_t)nb_queues;
 	data->dev_link = pmd_link;
 	data->mac_addrs = &(*internals)->eth_addr;
+	data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
 
 	(*eth_dev)->dev_ops = &ops;
 
@@ -1033,13 +1058,7 @@ rte_pmd_af_packet_probe(struct rte_vdev_device *dev)
 static int
 rte_pmd_af_packet_remove(struct rte_vdev_device *dev)
 {
-	struct rte_eth_dev *eth_dev = NULL;
-	struct pmd_internals *internals;
-	struct tpacket_req *req;
-	unsigned q;
-
-	PMD_LOG(INFO, "Closing AF_PACKET ethdev on numa socket %u",
-		rte_socket_id());
+	struct rte_eth_dev *eth_dev;
 
 	if (dev == NULL)
 		return -1;
@@ -1047,26 +1066,9 @@ rte_pmd_af_packet_remove(struct rte_vdev_device *dev)
 	/* find the ethdev entry */
 	eth_dev = rte_eth_dev_allocated(rte_vdev_device_name(dev));
 	if (eth_dev == NULL)
-		return -1;
-
-	/* mac_addrs must not be freed alone because part of dev_private */
-	eth_dev->data->mac_addrs = NULL;
-
-	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
-		return rte_eth_dev_release_port(eth_dev);
-
-	internals = eth_dev->data->dev_private;
-	req = &internals->req;
-	for (q = 0; q < internals->nb_queues; q++) {
-		munmap(internals->rx_queue[q].map,
-			2 * req->tp_block_size * req->tp_block_nr);
-		rte_free(internals->rx_queue[q].rd);
-		rte_free(internals->tx_queue[q].rd);
-	}
-	free(internals->if_name);
-	rte_free(internals->rx_queue);
-	rte_free(internals->tx_queue);
+		return 0; /* port already released */
 
+	eth_dev_close(eth_dev);
 	rte_eth_dev_release_port(eth_dev);
 
 	return 0;
-- 
2.28.0

[dpdk-dev] [PATCH v3 04/29] net/atlantic: release port upon close

[Thomas Monjalon]

The flag RTE_ETH_DEV_CLOSE_REMOVE is set so all port resources
can be freed by rte_eth_dev_close().

Freeing of private port resources is moved
from the ".remove(device)" to the ".dev_close(port)" operation.

Signed-off-by: Thomas Monjalon <thomas@monjalon.net>
Acked-by: Igor Russkikh <irusskikh@marvell.com>
---
 drivers/net/atlantic/atl_ethdev.c | 62 ++++++++++++-------------------
 1 file changed, 24 insertions(+), 38 deletions(-)

diff --git a/drivers/net/atlantic/atl_ethdev.c b/drivers/net/atlantic/atl_ethdev.c
index 254758fd66..d3babeff94 100644
--- a/drivers/net/atlantic/atl_ethdev.c
+++ b/drivers/net/atlantic/atl_ethdev.c
@@ -15,8 +15,6 @@
 #include "hw_atl/hw_atl_b0_internal.h"
 
 static int eth_atl_dev_init(struct rte_eth_dev *eth_dev);
-static int eth_atl_dev_uninit(struct rte_eth_dev *eth_dev);
-
 static int  atl_dev_configure(struct rte_eth_dev *dev);
 static int  atl_dev_start(struct rte_eth_dev *dev);
 static void atl_dev_stop(struct rte_eth_dev *dev);
@@ -382,6 +380,8 @@ eth_atl_dev_init(struct rte_eth_dev *eth_dev)
 	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
 		return 0;
 
+	eth_dev->data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
+
 	/* Vendor and Device ID need to be set before init of shared code */
 	hw->device_id = pci_dev->id.device_id;
 	hw->vendor_id = pci_dev->id.vendor_id;
@@ -442,40 +442,6 @@ eth_atl_dev_init(struct rte_eth_dev *eth_dev)
 	return err;
 }
 
-static int
-eth_atl_dev_uninit(struct rte_eth_dev *eth_dev)
-{
-	struct rte_pci_device *pci_dev = RTE_ETH_DEV_TO_PCI(eth_dev);
-	struct rte_intr_handle *intr_handle = &pci_dev->intr_handle;
-	struct aq_hw_s *hw;
-
-	PMD_INIT_FUNC_TRACE();
-
-	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
-		return -EPERM;
-
-	hw = ATL_DEV_PRIVATE_TO_HW(eth_dev->data->dev_private);
-
-	if (hw->adapter_stopped == 0)
-		atl_dev_close(eth_dev);
-
-	eth_dev->dev_ops = NULL;
-	eth_dev->rx_pkt_burst = NULL;
-	eth_dev->tx_pkt_burst = NULL;
-
-	/* disable uio intr before callback unregister */
-	rte_intr_disable(intr_handle);
-	rte_intr_callback_unregister(intr_handle,
-				     atl_dev_interrupt_handler, eth_dev);
-
-	rte_free(eth_dev->data->mac_addrs);
-	eth_dev->data->mac_addrs = NULL;
-
-	pthread_mutex_destroy(&hw->mbox_mutex);
-
-	return 0;
-}
-
 static int
 eth_atl_pci_probe(struct rte_pci_driver *pci_drv __rte_unused,
 	struct rte_pci_device *pci_dev)
@@ -487,7 +453,7 @@ eth_atl_pci_probe(struct rte_pci_driver *pci_drv __rte_unused,
 static int
 eth_atl_pci_remove(struct rte_pci_device *pci_dev)
 {
-	return rte_eth_dev_pci_generic_remove(pci_dev, eth_atl_dev_uninit);
+	return rte_eth_dev_pci_generic_remove(pci_dev, atl_dev_close);
 }
 
 static int
@@ -722,12 +688,32 @@ atl_dev_set_link_down(struct rte_eth_dev *dev)
 static int
 atl_dev_close(struct rte_eth_dev *dev)
 {
+	struct rte_pci_device *pci_dev = RTE_ETH_DEV_TO_PCI(dev);
+	struct rte_intr_handle *intr_handle = &pci_dev->intr_handle;
+	struct aq_hw_s *hw;
+
 	PMD_INIT_FUNC_TRACE();
 
+	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
+		return 0;
+
+	hw = ATL_DEV_PRIVATE_TO_HW(dev->data->dev_private);
+
 	atl_dev_stop(dev);
 
 	atl_free_queues(dev);
 
+	dev->dev_ops = NULL;
+	dev->rx_pkt_burst = NULL;
+	dev->tx_pkt_burst = NULL;
+
+	/* disable uio intr before callback unregister */
+	rte_intr_disable(intr_handle);
+	rte_intr_callback_unregister(intr_handle,
+				     atl_dev_interrupt_handler, dev);
+
+	pthread_mutex_destroy(&hw->mbox_mutex);
+
 	return 0;
 }
 
@@ -736,7 +722,7 @@ atl_dev_reset(struct rte_eth_dev *dev)
 {
 	int ret;
 
-	ret = eth_atl_dev_uninit(dev);
+	ret = atl_dev_close(dev);
 	if (ret)
 		return ret;
 
-- 
2.28.0

[dpdk-dev] [PATCH v3 05/29] net/axgbe: release port upon close

[Thomas Monjalon]

The flag RTE_ETH_DEV_CLOSE_REMOVE is set so all port resources
can be freed by rte_eth_dev_close().

Freeing of private port resources is moved
from the ".remove(device)" to the ".dev_close(port)" operation.
The ".dev_close" callback is also called as part of the ".remove" one.

Signed-off-by: Thomas Monjalon <thomas@monjalon.net>
---
 drivers/net/axgbe/axgbe_ethdev.c | 14 +++-----------
 1 file changed, 3 insertions(+), 11 deletions(-)

diff --git a/drivers/net/axgbe/axgbe_ethdev.c b/drivers/net/axgbe/axgbe_ethdev.c
index 458d64bcef..2dd64180ad 100644
--- a/drivers/net/axgbe/axgbe_ethdev.c
+++ b/drivers/net/axgbe/axgbe_ethdev.c
@@ -11,7 +11,6 @@
 #include "rte_time.h"
 
 static int eth_axgbe_dev_init(struct rte_eth_dev *eth_dev);
-static int eth_axgbe_dev_uninit(struct rte_eth_dev *eth_dev);
 static int  axgbe_dev_configure(struct rte_eth_dev *dev);
 static int  axgbe_dev_start(struct rte_eth_dev *dev);
 static void axgbe_dev_stop(struct rte_eth_dev *dev);
@@ -409,14 +408,6 @@ axgbe_dev_stop(struct rte_eth_dev *dev)
 	rte_bit_relaxed_set32(AXGBE_DOWN, &pdata->dev_state);
 }
 
-/* Clear all resources like TX/RX queues. */
-static int
-axgbe_dev_close(struct rte_eth_dev *dev)
-{
-	axgbe_dev_clear_queues(dev);
-	return 0;
-}
-
 static int
 axgbe_dev_promiscuous_enable(struct rte_eth_dev *dev)
 {
@@ -1968,6 +1959,7 @@ eth_axgbe_dev_init(struct rte_eth_dev *eth_dev)
 	int ret;
 
 	eth_dev->dev_ops = &axgbe_eth_dev_ops;
+	eth_dev->data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
 
 	eth_dev->rx_descriptor_status = axgbe_dev_rx_descriptor_status;
 	eth_dev->tx_descriptor_status = axgbe_dev_tx_descriptor_status;
@@ -2133,7 +2125,7 @@ eth_axgbe_dev_init(struct rte_eth_dev *eth_dev)
 }
 
 static int
-eth_axgbe_dev_uninit(struct rte_eth_dev *eth_dev)
+axgbe_dev_close(struct rte_eth_dev *eth_dev)
 {
 	struct rte_pci_device *pci_dev;
 
@@ -2166,7 +2158,7 @@ static int eth_axgbe_pci_probe(struct rte_pci_driver *pci_drv __rte_unused,
 
 static int eth_axgbe_pci_remove(struct rte_pci_device *pci_dev)
 {
-	return rte_eth_dev_pci_generic_remove(pci_dev, eth_axgbe_dev_uninit);
+	return rte_eth_dev_pci_generic_remove(pci_dev, axgbe_dev_close);
 }
 
 static struct rte_pci_driver rte_axgbe_pmd = {
-- 
2.28.0

[dpdk-dev] [PATCH v3 06/29] net/bnx2x: release port upon close

[Thomas Monjalon]

From: Rasesh Mody <rmody@marvell.com>

Set RTE_ETH_DEV_CLOSE_REMOVE upon probe so all the private resources
for the port can be freed by rte_eth_dev_close(). With this change the
private port resources are released in the .dev_close callback.

Signed-off-by: Rasesh Mody <rmody@marvell.com>
---
 drivers/net/bnx2x/bnx2x_ethdev.c | 17 +++++++++++++++--
 1 file changed, 15 insertions(+), 2 deletions(-)

diff --git a/drivers/net/bnx2x/bnx2x_ethdev.c b/drivers/net/bnx2x/bnx2x_ethdev.c
index 73217c2138..8dc46384af 100644
--- a/drivers/net/bnx2x/bnx2x_ethdev.c
+++ b/drivers/net/bnx2x/bnx2x_ethdev.c
@@ -287,6 +287,10 @@ bnx2x_dev_close(struct rte_eth_dev *dev)
 
 	PMD_INIT_FUNC_TRACE(sc);
 
+	/* only close in case of the primary process */
+	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
+		return 0;
+
 	if (IS_VF(sc))
 		bnx2x_vf_close(sc);
 
@@ -296,6 +300,9 @@ bnx2x_dev_close(struct rte_eth_dev *dev)
 	/* free ilt */
 	bnx2x_free_ilt_mem(sc);
 
+	/* mac_addrs must not be freed alone because part of dev_private */
+	dev->data->mac_addrs = NULL;
+
 	return 0;
 }
 
@@ -728,6 +735,11 @@ bnx2x_common_dev_init(struct rte_eth_dev *eth_dev, int is_vf)
 			goto out;
 	}
 
+	/* Pass the information to the rte_eth_dev_close() that it should also
+	 * release the private port resources.
+	 */
+	eth_dev->data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
+
 	return 0;
 
 out:
@@ -755,8 +767,9 @@ eth_bnx2xvf_dev_init(struct rte_eth_dev *eth_dev)
 
 static int eth_bnx2x_dev_uninit(struct rte_eth_dev *eth_dev)
 {
-	/* mac_addrs must not be freed alone because part of dev_private */
-	eth_dev->data->mac_addrs = NULL;
+	struct bnx2x_softc *sc = eth_dev->data->dev_private;
+	PMD_INIT_FUNC_TRACE(sc);
+	bnx2x_dev_close(eth_dev);
 	return 0;
 }
 
-- 
2.28.0

[dpdk-dev] [PATCH v3 07/29] net/bonding: release port upon close

[Thomas Monjalon]

The flag RTE_ETH_DEV_CLOSE_REMOVE is set so all port resources
can be freed by rte_eth_dev_close().

Freeing of private port resources is moved
from the ".remove(device)" to the ".dev_close(port)" operation.

Signed-off-by: Thomas Monjalon <thomas@monjalon.net>
---
 drivers/net/bonding/rte_eth_bond_pmd.c | 34 ++++++++++++--------------
 1 file changed, 16 insertions(+), 18 deletions(-)

diff --git a/drivers/net/bonding/rte_eth_bond_pmd.c b/drivers/net/bonding/rte_eth_bond_pmd.c
index dff835ef92..a9ed5d8e12 100644
--- a/drivers/net/bonding/rte_eth_bond_pmd.c
+++ b/drivers/net/bonding/rte_eth_bond_pmd.c
@@ -2103,6 +2103,9 @@ bond_ethdev_close(struct rte_eth_dev *dev)
 	int skipped = 0;
 	struct rte_flow_error ferror;
 
+	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
+		return 0;
+
 	RTE_BOND_LOG(INFO, "Closing bonded device %s", dev->device->name);
 	while (internals->slave_count != skipped) {
 		uint16_t port_id = internals->slaves[skipped].port_id;
@@ -2119,6 +2122,17 @@ bond_ethdev_close(struct rte_eth_dev *dev)
 	bond_flow_ops.flush(dev, &ferror);
 	bond_ethdev_free_queues(dev);
 	rte_bitmap_reset(internals->vlan_filter_bmp);
+	rte_bitmap_free(internals->vlan_filter_bmp);
+	rte_free(internals->vlan_filter_bmpmem);
+
+	/* Try to release mempool used in mode6. If the bond
+	 * device is not mode6, free the NULL is not problem.
+	 */
+	rte_mempool_free(internals->mode6.mempool);
+
+	dev->dev_ops = NULL;
+	dev->rx_pkt_burst = NULL;
+	dev->tx_pkt_burst = NULL;
 
 	return 0;
 }
@@ -3195,6 +3209,7 @@ bond_alloc(struct rte_vdev_device *dev, uint8_t mode)
 	}
 
 	internals = eth_dev->data->dev_private;
+	eth_dev->data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
 	eth_dev->data->nb_rx_queues = (uint16_t)1;
 	eth_dev->data->nb_tx_queues = (uint16_t)1;
 
@@ -3414,14 +3429,10 @@ bond_remove(struct rte_vdev_device *dev)
 	name = rte_vdev_device_name(dev);
 	RTE_BOND_LOG(INFO, "Uninitializing pmd_bond for %s", name);
 
-	/* now free all data allocation - for eth_dev structure,
-	 * dummy pci driver and internal (private) data
-	 */
-
 	/* find an ethdev entry */
 	eth_dev = rte_eth_dev_allocated(name);
 	if (eth_dev == NULL)
-		return -ENODEV;
+		return 0; /* port already released */
 
 	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
 		return rte_eth_dev_release_port(eth_dev);
@@ -3436,19 +3447,6 @@ bond_remove(struct rte_vdev_device *dev)
 		bond_ethdev_stop(eth_dev);
 		bond_ethdev_close(eth_dev);
 	}
-
-	eth_dev->dev_ops = NULL;
-	eth_dev->rx_pkt_burst = NULL;
-	eth_dev->tx_pkt_burst = NULL;
-
-	internals = eth_dev->data->dev_private;
-	/* Try to release mempool used in mode6. If the bond
-	 * device is not mode6, free the NULL is not problem.
-	 */
-	rte_mempool_free(internals->mode6.mempool);
-	rte_bitmap_free(internals->vlan_filter_bmp);
-	rte_free(internals->vlan_filter_bmpmem);
-
 	rte_eth_dev_release_port(eth_dev);
 
 	return 0;
-- 
2.28.0

[dpdk-dev] [PATCH v3 08/29] net/dpaa: release port upon close

[Thomas Monjalon]

From: Sachin Saxena <sachin.saxena@oss.nxp.com>

With removal of old close behavior, the private
port resources must be released in the .dev_close callback.
Freeing of port private resources is moved from
the ".remove(device)" to the ".dev_close(port)" operation

Signed-off-by: Sachin Saxena <sachin.saxena@oss.nxp.com>
---
 drivers/net/dpaa/dpaa_ethdev.c | 125 +++++++++++++++------------------
 1 file changed, 57 insertions(+), 68 deletions(-)

diff --git a/drivers/net/dpaa/dpaa_ethdev.c b/drivers/net/dpaa/dpaa_ethdev.c
index 7fe06e1830..8f986683e2 100644
--- a/drivers/net/dpaa/dpaa_ethdev.c
+++ b/drivers/net/dpaa/dpaa_ethdev.c
@@ -376,9 +376,25 @@ static int dpaa_eth_dev_close(struct rte_eth_dev *dev)
 	struct rte_device *rdev = dev->device;
 	struct rte_dpaa_device *dpaa_dev;
 	struct rte_intr_handle *intr_handle;
+	struct dpaa_if *dpaa_intf = dev->data->dev_private;
+	int loop;
 
 	PMD_INIT_FUNC_TRACE();
 
+	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
+		return 0;
+
+	if (!dpaa_intf) {
+		DPAA_PMD_WARN("Already closed or not started");
+		return -1;
+	}
+
+	/* DPAA FM deconfig */
+	if (!(default_q || fmc_q)) {
+		if (dpaa_fm_deconfig(dpaa_intf, dev->process_private))
+			DPAA_PMD_WARN("DPAA FM deconfig failed\n");
+	}
+
 	dpaa_dev = container_of(rdev, struct rte_dpaa_device, device);
 	intr_handle = &dpaa_dev->intr_handle;
 	__fif = container_of(fif, struct __fman_if, __if);
@@ -393,6 +409,42 @@ static int dpaa_eth_dev_close(struct rte_eth_dev *dev)
 					     (void *)dev);
 	}
 
+	/* release configuration memory */
+	if (dpaa_intf->fc_conf)
+		rte_free(dpaa_intf->fc_conf);
+
+	/* Release RX congestion Groups */
+	if (dpaa_intf->cgr_rx) {
+		for (loop = 0; loop < dpaa_intf->nb_rx_queues; loop++)
+			qman_delete_cgr(&dpaa_intf->cgr_rx[loop]);
+
+		qman_release_cgrid_range(dpaa_intf->cgr_rx[loop].cgrid,
+					 dpaa_intf->nb_rx_queues);
+	}
+
+	rte_free(dpaa_intf->cgr_rx);
+	dpaa_intf->cgr_rx = NULL;
+	/* Release TX congestion Groups */
+	if (dpaa_intf->cgr_tx) {
+		for (loop = 0; loop < MAX_DPAA_CORES; loop++)
+			qman_delete_cgr(&dpaa_intf->cgr_tx[loop]);
+
+		qman_release_cgrid_range(dpaa_intf->cgr_tx[loop].cgrid,
+					 MAX_DPAA_CORES);
+		rte_free(dpaa_intf->cgr_tx);
+		dpaa_intf->cgr_tx = NULL;
+	}
+
+	rte_free(dpaa_intf->rx_queues);
+	dpaa_intf->rx_queues = NULL;
+
+	rte_free(dpaa_intf->tx_queues);
+	dpaa_intf->tx_queues = NULL;
+
+	dev->dev_ops = NULL;
+	dev->rx_pkt_burst = NULL;
+	dev->tx_pkt_burst = NULL;
+
 	return 0;
 }
 
@@ -1977,70 +2029,6 @@ dpaa_dev_init(struct rte_eth_dev *eth_dev)
 	return ret;
 }
 
-static int
-dpaa_dev_uninit(struct rte_eth_dev *dev)
-{
-	struct dpaa_if *dpaa_intf = dev->data->dev_private;
-	int loop;
-
-	PMD_INIT_FUNC_TRACE();
-
-	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
-		return -EPERM;
-
-	if (!dpaa_intf) {
-		DPAA_PMD_WARN("Already closed or not started");
-		return -1;
-	}
-
-	/* DPAA FM deconfig */
-	if (!(default_q || fmc_q)) {
-		if (dpaa_fm_deconfig(dpaa_intf, dev->process_private))
-			DPAA_PMD_WARN("DPAA FM deconfig failed\n");
-	}
-
-	dpaa_eth_dev_close(dev);
-
-	/* release configuration memory */
-	if (dpaa_intf->fc_conf)
-		rte_free(dpaa_intf->fc_conf);
-
-	/* Release RX congestion Groups */
-	if (dpaa_intf->cgr_rx) {
-		for (loop = 0; loop < dpaa_intf->nb_rx_queues; loop++)
-			qman_delete_cgr(&dpaa_intf->cgr_rx[loop]);
-
-		qman_release_cgrid_range(dpaa_intf->cgr_rx[loop].cgrid,
-					 dpaa_intf->nb_rx_queues);
-	}
-
-	rte_free(dpaa_intf->cgr_rx);
-	dpaa_intf->cgr_rx = NULL;
-
-	/* Release TX congestion Groups */
-	if (dpaa_intf->cgr_tx) {
-		for (loop = 0; loop < MAX_DPAA_CORES; loop++)
-			qman_delete_cgr(&dpaa_intf->cgr_tx[loop]);
-
-		qman_release_cgrid_range(dpaa_intf->cgr_tx[loop].cgrid,
-					 MAX_DPAA_CORES);
-		rte_free(dpaa_intf->cgr_tx);
-		dpaa_intf->cgr_tx = NULL;
-	}
-
-	rte_free(dpaa_intf->rx_queues);
-	dpaa_intf->rx_queues = NULL;
-
-	rte_free(dpaa_intf->tx_queues);
-	dpaa_intf->tx_queues = NULL;
-
-	dev->dev_ops = NULL;
-	dev->rx_pkt_burst = NULL;
-	dev->tx_pkt_burst = NULL;
-
-	return 0;
-}
-
 static int
 rte_dpaa_probe(struct rte_dpaa_driver *dpaa_drv,
 	       struct rte_dpaa_device *dpaa_dev)
@@ -2139,6 +2127,7 @@ rte_dpaa_probe(struct rte_dpaa_driver *dpaa_drv,
 
 	qman_ern_register_cb(dpaa_free_mbuf);
 
+	eth_dev->data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
 	if (dpaa_drv->drv_flags & RTE_DPAA_DRV_INTR_LSC)
 		eth_dev->data->dev_flags |= RTE_ETH_DEV_INTR_LSC;
 
@@ -2157,15 +2146,15 @@ static int
 rte_dpaa_remove(struct rte_dpaa_device *dpaa_dev)
 {
 	struct rte_eth_dev *eth_dev;
+	int ret;
 
 	PMD_INIT_FUNC_TRACE();
 
 	eth_dev = dpaa_dev->eth_dev;
-	dpaa_dev_uninit(eth_dev);
+	dpaa_eth_dev_close(eth_dev);
+	ret = rte_eth_dev_release_port(eth_dev);
 
-	rte_eth_dev_release_port(eth_dev);
-
-	return 0;
+	return ret;
 }
 
 static void __attribute__((destructor(102))) dpaa_finish(void)
-- 
2.28.0

Re: [dpdk-dev] [PATCH v3 08/29] net/dpaa: release port upon close

[Hemant Agrawal]

Acked-by: Hemant Agrawal <hemant.agrawal@nxp.com>

[dpdk-dev] [PATCH v3 09/29] net/dpaa2: release port upon close

[Thomas Monjalon]

From: Sachin Saxena <sachin.saxena@oss.nxp.com>

With removal of old close behavior, the private
port resources must be released in the .dev_close callback.
Freeing of port private resources is moved from
the ".remove(device)" to the ".dev_close(port)" operation

Signed-off-by: Sachin Saxena <sachin.saxena@oss.nxp.com>
---
 drivers/net/dpaa2/dpaa2_ethdev.c | 95 ++++++++++++++------------------
 1 file changed, 40 insertions(+), 55 deletions(-)

diff --git a/drivers/net/dpaa2/dpaa2_ethdev.c b/drivers/net/dpaa2/dpaa2_ethdev.c
index fb4165fa03..7bd3208a19 100644
--- a/drivers/net/dpaa2/dpaa2_ethdev.c
+++ b/drivers/net/dpaa2/dpaa2_ethdev.c
@@ -99,7 +99,6 @@ static const enum rte_filter_op dpaa2_supported_filter_ops[] = {
 };
 
 static struct rte_dpaa2_driver rte_dpaa2_pmd;
-static int dpaa2_dev_uninit(struct rte_eth_dev *eth_dev);
 static int dpaa2_dev_link_update(struct rte_eth_dev *dev,
 				 int wait_to_complete);
 static int dpaa2_dev_set_link_up(struct rte_eth_dev *dev);
@@ -1241,13 +1240,20 @@ dpaa2_dev_close(struct rte_eth_dev *dev)
 {
 	struct dpaa2_dev_priv *priv = dev->data->dev_private;
 	struct fsl_mc_io *dpni = (struct fsl_mc_io *)dev->process_private;
-	int ret;
+	int i, ret;
 	struct rte_eth_link link;
 
 	PMD_INIT_FUNC_TRACE();
 
+	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
+		return 0;
+
+	if (!dpni) {
+		DPAA2_PMD_WARN("Already closed or not started");
+		return -1;
+	}
+
 	dpaa2_flow_clean(dev);
-
 	/* Clean the device first */
 	ret = dpni_reset(dpni, CMD_PRI_LOW, priv->token);
 	if (ret) {
@@ -1258,6 +1264,31 @@ dpaa2_dev_close(struct rte_eth_dev *dev)
 	memset(&link, 0, sizeof(link));
 	rte_eth_linkstatus_set(dev, &link);
 
+	/* Free private queues memory */
+	dpaa2_free_rx_tx_queues(dev);
+	/* Close the device at underlying layer*/
+	ret = dpni_close(dpni, CMD_PRI_LOW, priv->token);
+	if (ret) {
+		DPAA2_PMD_ERR("Failure closing dpni device with err code %d",
+			      ret);
+	}
+
+	/* Free the allocated memory for ethernet private data and dpni*/
+	priv->hw = NULL;
+	dev->process_private = NULL;
+	rte_free(dpni);
+
+	for (i = 0; i < MAX_TCS; i++)
+		rte_free((void *)(size_t)priv->extract.tc_extract_param[i]);
+
+	if (priv->extract.qos_extract_param)
+		rte_free((void *)(size_t)priv->extract.qos_extract_param);
+
+	dev->dev_ops = NULL;
+	dev->rx_pkt_burst = NULL;
+	dev->tx_pkt_burst = NULL;
+
+	DPAA2_PMD_INFO("%s: netdev deleted", dev->data->name);
 	return 0;
 }
 
@@ -2709,56 +2740,9 @@ dpaa2_dev_init(struct rte_eth_dev *eth_dev)
 	RTE_LOG(INFO, PMD, "%s: netdev created\n", eth_dev->data->name);
 	return 0;
 init_err:
-	dpaa2_dev_uninit(eth_dev);
-	return ret;
-}
-
-static int
-dpaa2_dev_uninit(struct rte_eth_dev *eth_dev)
-{
-	struct dpaa2_dev_priv *priv = eth_dev->data->dev_private;
-	struct fsl_mc_io *dpni = (struct fsl_mc_io *)eth_dev->process_private;
-	int i, ret;
-
-	PMD_INIT_FUNC_TRACE();
-
-	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
-		return 0;
-
-	if (!dpni) {
-		DPAA2_PMD_WARN("Already closed or not started");
-		return -1;
-	}
-
 	dpaa2_dev_close(eth_dev);
 
-	dpaa2_free_rx_tx_queues(eth_dev);
-
-	/* Close the device at underlying layer*/
-	ret = dpni_close(dpni, CMD_PRI_LOW, priv->token);
-	if (ret) {
-		DPAA2_PMD_ERR(
-			     "Failure closing dpni device with err code %d",
-			     ret);
-	}
-
-	/* Free the allocated memory for ethernet private data and dpni*/
-	priv->hw = NULL;
-	eth_dev->process_private = NULL;
-	rte_free(dpni);
-
-	for (i = 0; i < MAX_TCS; i++)
-		rte_free((void *)(size_t)priv->extract.tc_extract_param[i]);
-
-	if (priv->extract.qos_extract_param)
-		rte_free((void *)(size_t)priv->extract.qos_extract_param);
-
-	eth_dev->dev_ops = NULL;
-	eth_dev->rx_pkt_burst = NULL;
-	eth_dev->tx_pkt_burst = NULL;
-
-	DPAA2_PMD_INFO("%s: netdev deleted", eth_dev->data->name);
-	return 0;
+	return ret;
 }
 
 static int
@@ -2809,6 +2793,7 @@ rte_dpaa2_probe(struct rte_dpaa2_driver *dpaa2_drv,
 	dpaa2_dev->eth_dev = eth_dev;
 	eth_dev->data->rx_mbuf_alloc_failed = 0;
 
+	eth_dev->data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
 	if (dpaa2_drv->drv_flags & RTE_DPAA2_DRV_INTR_LSC)
 		eth_dev->data->dev_flags |= RTE_ETH_DEV_INTR_LSC;
 
@@ -2827,13 +2812,13 @@ static int
 rte_dpaa2_remove(struct rte_dpaa2_device *dpaa2_dev)
 {
 	struct rte_eth_dev *eth_dev;
+	int ret;
 
 	eth_dev = dpaa2_dev->eth_dev;
-	dpaa2_dev_uninit(eth_dev);
+	dpaa2_dev_close(eth_dev);
+	ret = rte_eth_dev_release_port(eth_dev);
 
-	rte_eth_dev_release_port(eth_dev);
-
-	return 0;
+	return ret;
 }
 
 static struct rte_dpaa2_driver rte_dpaa2_pmd = {
-- 
2.28.0

Re: [dpdk-dev] [PATCH v3 09/29] net/dpaa2: release port upon close

[Hemant Agrawal]

Acked-by: Hemant Agrawal <hemant.agrawal@nxp.com>

[dpdk-dev] [PATCH v3 10/29] net/enetc: release port upon close

[Thomas Monjalon]

From: Sachin Saxena <sachin.saxena@oss.nxp.com>

With removal of old close behavior, the private
port resources must be released in the .dev_close callback.
Freeing of port private resources is moved from
the ".remove(device)" to the ".dev_close(port)" operation

Signed-off-by: Sachin Saxena <sachin.saxena@oss.nxp.com>
---
 drivers/net/enetc/enetc_ethdev.c | 11 ++++++-----
 1 file changed, 6 insertions(+), 5 deletions(-)

diff --git a/drivers/net/enetc/enetc_ethdev.c b/drivers/net/enetc/enetc_ethdev.c
index bdb32762ce..afb658df3d 100644
--- a/drivers/net/enetc/enetc_ethdev.c
+++ b/drivers/net/enetc/enetc_ethdev.c
@@ -565,6 +565,9 @@ enetc_dev_close(struct rte_eth_dev *dev)
 	}
 	dev->data->nb_tx_queues = 0;
 
+	if (rte_eal_iova_mode() == RTE_IOVA_PA)
+		dpaax_iova_table_depopulate();
+
 	return 0;
 }
 
@@ -874,6 +877,7 @@ enetc_dev_init(struct rte_eth_dev *eth_dev)
 	eth_dev->dev_ops = &enetc_ops;
 	eth_dev->rx_pkt_burst = &enetc_recv_pkts;
 	eth_dev->tx_pkt_burst = &enetc_xmit_pkts;
+	eth_dev->data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
 
 	/* Retrieving and storing the HW base address of device */
 	hw->hw.reg = (void *)pci_dev->mem_resource[0].addr;
@@ -916,14 +920,11 @@ enetc_dev_init(struct rte_eth_dev *eth_dev)
 }
 
 static int
-enetc_dev_uninit(struct rte_eth_dev *eth_dev __rte_unused)
+enetc_dev_uninit(struct rte_eth_dev *eth_dev)
 {
 	PMD_INIT_FUNC_TRACE();
 
-	if (rte_eal_iova_mode() == RTE_IOVA_PA)
-		dpaax_iova_table_depopulate();
-
-	return 0;
+	return enetc_dev_close(eth_dev);
 }
 
 static int
-- 
2.28.0

[dpdk-dev] [PATCH v3 11/29] net/failsafe: release port upon close

[Thomas Monjalon]

The flag RTE_ETH_DEV_CLOSE_REMOVE is set so all port resources
can be freed by rte_eth_dev_close().

Freeing of private port resources is moved
from the ".remove(device)" to the ".dev_close(port)" operation.

Signed-off-by: Thomas Monjalon <thomas@monjalon.net>
---
 drivers/net/failsafe/failsafe.c         | 25 ++--------
 drivers/net/failsafe/failsafe_ops.c     | 65 ++++++++++++++++---------
 drivers/net/failsafe/failsafe_private.h |  1 +
 3 files changed, 46 insertions(+), 45 deletions(-)

diff --git a/drivers/net/failsafe/failsafe.c b/drivers/net/failsafe/failsafe.c
index 4a4b7ceab6..44d47e8f72 100644
--- a/drivers/net/failsafe/failsafe.c
+++ b/drivers/net/failsafe/failsafe.c
@@ -60,12 +60,6 @@ fs_sub_device_alloc(struct rte_eth_dev *dev,
 	return 0;
 }
 
-static void
-fs_sub_device_free(struct rte_eth_dev *dev)
-{
-	rte_free(PRIV(dev)->subs);
-}
-
 static void fs_hotplug_alarm(void *arg);
 
 int
@@ -186,6 +180,7 @@ fs_eth_dev_create(struct rte_vdev_device *vdev)
 		ERROR("Unable to allocate rte_eth_dev");
 		return -1;
 	}
+	dev->data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
 	priv = PRIV(dev);
 	priv->data = dev->data;
 	priv->rxp = FS_RX_PROXY_INIT;
@@ -285,7 +280,7 @@ fs_eth_dev_create(struct rte_vdev_device *vdev)
 free_args:
 	failsafe_args_free(dev);
 free_subs:
-	fs_sub_device_free(dev);
+	rte_free(PRIV(dev)->subs);
 free_dev:
 	/* mac_addrs must not be freed alone because part of dev_private */
 	dev->data->mac_addrs = NULL;
@@ -301,20 +296,8 @@ fs_rte_eth_free(const char *name)
 
 	dev = rte_eth_dev_allocated(name);
 	if (dev == NULL)
-		return -ENODEV;
-	rte_eth_dev_callback_unregister(RTE_ETH_ALL, RTE_ETH_EVENT_NEW,
-					failsafe_eth_new_event_callback, dev);
-	ret = failsafe_eal_uninit(dev);
-	if (ret)
-		ERROR("Error while uninitializing sub-EAL");
-	failsafe_args_free(dev);
-	fs_sub_device_free(dev);
-	ret = pthread_mutex_destroy(&PRIV(dev)->hotplug_mutex);
-	if (ret)
-		ERROR("Error while destroying hotplug mutex");
-	rte_free(PRIV(dev)->mcast_addrs);
-	/* mac_addrs must not be freed alone because part of dev_private */
-	dev->data->mac_addrs = NULL;
+		return 0; /* port already released */
+	ret = failsafe_eth_dev_close(dev);
 	rte_eth_dev_release_port(dev);
 	return ret;
 }
diff --git a/drivers/net/failsafe/failsafe_ops.c b/drivers/net/failsafe/failsafe_ops.c
index 93ebd09114..0ce7dfc8a6 100644
--- a/drivers/net/failsafe/failsafe_ops.c
+++ b/drivers/net/failsafe/failsafe_ops.c
@@ -239,29 +239,6 @@ fs_dev_set_link_down(struct rte_eth_dev *dev)
 	return 0;
 }
 
-static void fs_dev_free_queues(struct rte_eth_dev *dev);
-static int
-fs_dev_close(struct rte_eth_dev *dev)
-{
-	struct sub_device *sdev;
-	uint8_t i;
-
-	fs_lock(dev, 0);
-	failsafe_hotplug_alarm_cancel(dev);
-	if (PRIV(dev)->state == DEV_STARTED)
-		dev->dev_ops->dev_stop(dev);
-	PRIV(dev)->state = DEV_ACTIVE - 1;
-	FOREACH_SUBDEV_STATE(sdev, i, dev, DEV_ACTIVE) {
-		DEBUG("Closing sub_device %d", i);
-		failsafe_eth_dev_unregister_callbacks(sdev);
-		rte_eth_dev_close(PORT_ID(sdev));
-		sdev->state = DEV_ACTIVE - 1;
-	}
-	fs_dev_free_queues(dev);
-	fs_unlock(dev, 0);
-	return 0;
-}
-
 static int
 fs_rx_queue_stop(struct rte_eth_dev *dev, uint16_t rx_queue_id)
 {
@@ -656,6 +633,46 @@ fs_dev_free_queues(struct rte_eth_dev *dev)
 	dev->data->nb_tx_queues = 0;
 }
 
+int
+failsafe_eth_dev_close(struct rte_eth_dev *dev)
+{
+	struct sub_device *sdev;
+	uint8_t i;
+	int ret;
+
+	fs_lock(dev, 0);
+	failsafe_hotplug_alarm_cancel(dev);
+	if (PRIV(dev)->state == DEV_STARTED)
+		dev->dev_ops->dev_stop(dev);
+	PRIV(dev)->state = DEV_ACTIVE - 1;
+	FOREACH_SUBDEV_STATE(sdev, i, dev, DEV_ACTIVE) {
+		DEBUG("Closing sub_device %d", i);
+		failsafe_eth_dev_unregister_callbacks(sdev);
+		rte_eth_dev_close(PORT_ID(sdev));
+		sdev->state = DEV_ACTIVE - 1;
+	}
+	rte_eth_dev_callback_unregister(RTE_ETH_ALL, RTE_ETH_EVENT_NEW,
+					failsafe_eth_new_event_callback, dev);
+	if (rte_eal_process_type() != RTE_PROC_PRIMARY) {
+		fs_unlock(dev, 0);
+		return 0;
+	}
+	fs_dev_free_queues(dev);
+	ret = failsafe_eal_uninit(dev);
+	if (ret)
+		ERROR("Error while uninitializing sub-EAL");
+	failsafe_args_free(dev);
+	rte_free(PRIV(dev)->subs);
+	rte_free(PRIV(dev)->mcast_addrs);
+	/* mac_addrs must not be freed alone because part of dev_private */
+	dev->data->mac_addrs = NULL;
+	fs_unlock(dev, 0);
+	ret = pthread_mutex_destroy(&PRIV(dev)->hotplug_mutex);
+	if (ret)
+		ERROR("Error while destroying hotplug mutex");
+	return 0;
+}
+
 static int
 fs_promiscuous_enable(struct rte_eth_dev *dev)
 {
@@ -1484,7 +1501,7 @@ const struct eth_dev_ops failsafe_ops = {
 	.dev_stop = fs_dev_stop,
 	.dev_set_link_down = fs_dev_set_link_down,
 	.dev_set_link_up = fs_dev_set_link_up,
-	.dev_close = fs_dev_close,
+	.dev_close = failsafe_eth_dev_close,
 	.promiscuous_enable = fs_promiscuous_enable,
 	.promiscuous_disable = fs_promiscuous_disable,
 	.allmulticast_enable = fs_allmulticast_enable,
diff --git a/drivers/net/failsafe/failsafe_private.h b/drivers/net/failsafe/failsafe_private.h
index 651578a128..6af0ef8471 100644
--- a/drivers/net/failsafe/failsafe_private.h
+++ b/drivers/net/failsafe/failsafe_private.h
@@ -236,6 +236,7 @@ int failsafe_eal_uninit(struct rte_eth_dev *dev);
 
 int failsafe_eth_dev_state_sync(struct rte_eth_dev *dev);
 void failsafe_eth_dev_unregister_callbacks(struct sub_device *sdev);
+int failsafe_eth_dev_close(struct rte_eth_dev *dev);
 void failsafe_dev_remove(struct rte_eth_dev *dev);
 void failsafe_stats_increment(struct rte_eth_stats *to,
 				struct rte_eth_stats *from);
-- 
2.28.0

Re: [dpdk-dev] [PATCH v3 11/29] net/failsafe: release port upon close

[Gaëtan Rivet]

Hi Thomas,

On 29/09/20 01:14 +0200, Thomas Monjalon wrote:
> The flag RTE_ETH_DEV_CLOSE_REMOVE is set so all port resources
> can be freed by rte_eth_dev_close().
> 
> Freeing of private port resources is moved
> from the ".remove(device)" to the ".dev_close(port)" operation.
> 
> Signed-off-by: Thomas Monjalon <thomas@monjalon.net>

Thanks for doing this. LGTM:

Acked-by: Gaetan Rivet <grive@u256.net>

-- 
Gaëtan

[dpdk-dev] [PATCH v3 12/29] net/mlx4: release port upon close

[Thomas Monjalon]

The flag RTE_ETH_DEV_CLOSE_REMOVE is set so all port resources
can be freed by rte_eth_dev_close().

Signed-off-by: Thomas Monjalon <thomas@monjalon.net>
---
 drivers/net/mlx4/mlx4.c | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/drivers/net/mlx4/mlx4.c b/drivers/net/mlx4/mlx4.c
index df59314b66..ad7c805d67 100644
--- a/drivers/net/mlx4/mlx4.c
+++ b/drivers/net/mlx4/mlx4.c
@@ -400,6 +400,8 @@ mlx4_dev_close(struct rte_eth_dev *dev)
 		MLX4_ASSERT(priv->ctx == NULL);
 	mlx4_intr_uninstall(priv);
 	memset(priv, 0, sizeof(*priv));
+	/* mac_addrs must not be freed because part of dev_private */
+	dev->data->mac_addrs = NULL;
 	return 0;
 }
 
@@ -1025,6 +1027,7 @@ mlx4_pci_probe(struct rte_pci_driver *pci_drv, struct rte_pci_device *pci_dev)
 			ERROR("can not allocate rte ethdev");
 			goto port_error;
 		}
+		eth_dev->data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
 		eth_dev->data->dev_private = priv;
 		eth_dev->data->mac_addrs = priv->mac;
 		eth_dev->device = &pci_dev->device;
-- 
2.28.0

[dpdk-dev] [PATCH v3 13/29] net/null: release port upon close

[Thomas Monjalon]

The flag RTE_ETH_DEV_CLOSE_REMOVE is set so all port resources
can be freed by rte_eth_dev_close().

Signed-off-by: Thomas Monjalon <thomas@monjalon.net>
---
 drivers/net/null/rte_eth_null.c | 27 +++++++++++++++++++--------
 1 file changed, 19 insertions(+), 8 deletions(-)

diff --git a/drivers/net/null/rte_eth_null.c b/drivers/net/null/rte_eth_null.c
index 0ce073fa4b..84c1d0c951 100644
--- a/drivers/net/null/rte_eth_null.c
+++ b/drivers/net/null/rte_eth_null.c
@@ -458,7 +458,23 @@ eth_mac_address_set(__rte_unused struct rte_eth_dev *dev,
 	return 0;
 }
 
+static int
+eth_dev_close(struct rte_eth_dev *dev)
+{
+	PMD_LOG(INFO, "Closing null ethdev on NUMA socket %u",
+			rte_socket_id());
+
+	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
+		return 0;
+
+	/* mac_addrs must not be freed alone because part of dev_private */
+	dev->data->mac_addrs = NULL;
+
+	return 0;
+}
+
 static const struct eth_dev_ops ops = {
+	.dev_close = eth_dev_close,
 	.dev_start = eth_dev_start,
 	.dev_stop = eth_dev_stop,
 	.dev_configure = eth_dev_configure,
@@ -532,6 +548,7 @@ eth_dev_null_create(struct rte_vdev_device *dev, struct pmd_options *args)
 	data->mac_addrs = &internals->eth_addr;
 	data->promiscuous = 1;
 	data->all_multicast = 1;
+	data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
 
 	eth_dev->dev_ops = &ops;
 
@@ -701,18 +718,12 @@ rte_pmd_null_remove(struct rte_vdev_device *dev)
 	if (!dev)
 		return -EINVAL;
 
-	PMD_LOG(INFO, "Closing null ethdev on numa socket %u",
-			rte_socket_id());
-
 	/* find the ethdev entry */
 	eth_dev = rte_eth_dev_allocated(rte_vdev_device_name(dev));
 	if (eth_dev == NULL)
-		return -1;
-
-	if (rte_eal_process_type() == RTE_PROC_PRIMARY)
-		/* mac_addrs must not be freed alone because part of dev_private */
-		eth_dev->data->mac_addrs = NULL;
+		return 0; /* port already released */
 
+	eth_dev_close(eth_dev);
 	rte_eth_dev_release_port(eth_dev);
 
 	return 0;
-- 
2.28.0

Re: [dpdk-dev] [PATCH v3 13/29] net/null: release port upon close

[Ferruh Yigit]

On 9/29/2020 12:14 AM, Thomas Monjalon wrote:
> The flag RTE_ETH_DEV_CLOSE_REMOVE is set so all port resources
> can be freed by rte_eth_dev_close().
> 
> Signed-off-by: Thomas Monjalon <thomas@monjalon.net>

Reviewed-by: Ferruh Yigit <ferruh.yigit@intel.com>

[dpdk-dev] [PATCH v3 14/29] net/octeontx: release port upon close

[Thomas Monjalon]

The flag RTE_ETH_DEV_CLOSE_REMOVE is set so all port resources
can be freed by rte_eth_dev_close().

The callback ".dev_close(port)" is called also
from the ".remove(device)" operation.

Signed-off-by: Thomas Monjalon <thomas@monjalon.net>
---
 drivers/net/octeontx/octeontx_ethdev.c | 11 +++--------
 1 file changed, 3 insertions(+), 8 deletions(-)

diff --git a/drivers/net/octeontx/octeontx_ethdev.c b/drivers/net/octeontx/octeontx_ethdev.c
index 10eeeac642..48ce3e1621 100644
--- a/drivers/net/octeontx/octeontx_ethdev.c
+++ b/drivers/net/octeontx/octeontx_ethdev.c
@@ -509,10 +509,6 @@ octeontx_dev_close(struct rte_eth_dev *dev)
 		rte_free(txq);
 	}
 
-	/* Free MAC address table */
-	rte_free(dev->data->mac_addrs);
-	dev->data->mac_addrs = NULL;
-
 	octeontx_port_close(nic);
 
 	dev->tx_pkt_burst = NULL;
@@ -1378,6 +1374,7 @@ octeontx_create(struct rte_vdev_device *dev, int port, uint8_t evdev,
 	data->promiscuous = 0;
 	data->all_multicast = 0;
 	data->scattered_rx = 0;
+	data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
 
 	/* Get maximum number of supported MAC entries */
 	max_entries = octeontx_bgx_port_mac_entries_get(nic->port_id);
@@ -1465,10 +1462,9 @@ octeontx_remove(struct rte_vdev_device *dev)
 	for (i = 0; i < OCTEONTX_VDEV_DEFAULT_MAX_NR_PORT; i++) {
 		sprintf(octtx_name, "eth_octeontx_%d", i);
 
-		/* reserve an ethdev entry */
 		eth_dev = rte_eth_dev_allocated(octtx_name);
 		if (eth_dev == NULL)
-			return -ENODEV;
+			continue; /* port already released */
 
 		if (rte_eal_process_type() != RTE_PROC_PRIMARY) {
 			rte_eth_dev_release_port(eth_dev);
@@ -1478,9 +1474,8 @@ octeontx_remove(struct rte_vdev_device *dev)
 		nic = octeontx_pmd_priv(eth_dev);
 		rte_event_dev_stop(nic->evdev);
 		PMD_INIT_LOG(INFO, "Closing octeontx device %s", octtx_name);
-
+		octeontx_dev_close(eth_dev);
 		rte_eth_dev_release_port(eth_dev);
-		rte_event_dev_close(nic->evdev);
 	}
 
 	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
-- 
2.28.0

[dpdk-dev] [PATCH v3 15/29] net/pcap: release port upon close

[Thomas Monjalon]

The flag RTE_ETH_DEV_CLOSE_REMOVE is set so all port resources
can be freed by rte_eth_dev_close().

Freeing of private port resources is moved
from the ".remove(device)" to the ".dev_close(port)" operation.

Signed-off-by: Thomas Monjalon <thomas@monjalon.net>
---
 drivers/net/pcap/rte_eth_pcap.c | 29 ++++++++++++++---------------
 1 file changed, 14 insertions(+), 15 deletions(-)

diff --git a/drivers/net/pcap/rte_eth_pcap.c b/drivers/net/pcap/rte_eth_pcap.c
index 76e704a65a..909eef8cce 100644
--- a/drivers/net/pcap/rte_eth_pcap.c
+++ b/drivers/net/pcap/rte_eth_pcap.c
@@ -734,6 +734,14 @@ eth_dev_close(struct rte_eth_dev *dev)
 	unsigned int i;
 	struct pmd_internals *internals = dev->data->dev_private;
 
+	PMD_LOG(INFO, "Closing pcap ethdev on NUMA socket %d",
+			rte_socket_id());
+
+	rte_free(dev->process_private);
+
+	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
+		return 0;
+
 	/* Device wide flag, but cleanup must be performed per queue. */
 	if (internals->infinite_rx) {
 		for (i = 0; i < dev->data->nb_rx_queues; i++) {
@@ -748,6 +756,10 @@ eth_dev_close(struct rte_eth_dev *dev)
 		}
 	}
 
+	if (internals->phy_mac == 0)
+		/* not dynamically allocated, must not be freed */
+		dev->data->mac_addrs = NULL;
+
 	return 0;
 }
 
@@ -1322,6 +1334,7 @@ eth_from_pcaps(struct rte_vdev_device *vdev,
 	else
 		eth_dev->tx_pkt_burst = eth_tx_drop;
 
+	eth_dev->data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
 	rte_eth_dev_probing_finish(eth_dev);
 	return 0;
 }
@@ -1544,30 +1557,16 @@ pmd_pcap_probe(struct rte_vdev_device *dev)
 static int
 pmd_pcap_remove(struct rte_vdev_device *dev)
 {
-	struct pmd_internals *internals = NULL;
 	struct rte_eth_dev *eth_dev = NULL;
 
-	PMD_LOG(INFO, "Closing pcap ethdev on numa socket %d",
-			rte_socket_id());
-
 	if (!dev)
 		return -1;
 
-	/* reserve an ethdev entry */
 	eth_dev = rte_eth_dev_allocated(rte_vdev_device_name(dev));
 	if (eth_dev == NULL)
-		return -1;
-
-	if (rte_eal_process_type() == RTE_PROC_PRIMARY) {
-		internals = eth_dev->data->dev_private;
-		if (internals != NULL && internals->phy_mac == 0)
-			/* not dynamically allocated, must not be freed */
-			eth_dev->data->mac_addrs = NULL;
-	}
+		return 0; /* port already released */
 
 	eth_dev_close(eth_dev);
-
-	rte_free(eth_dev->process_private);
 	rte_eth_dev_release_port(eth_dev);
 
 	return 0;
-- 
2.28.0

Re: [dpdk-dev] [PATCH v3 15/29] net/pcap: release port upon close

[Ferruh Yigit]

On 9/29/2020 12:14 AM, Thomas Monjalon wrote:
> The flag RTE_ETH_DEV_CLOSE_REMOVE is set so all port resources
> can be freed by rte_eth_dev_close().
> 
> Freeing of private port resources is moved
> from the ".remove(device)" to the ".dev_close(port)" operation.
> 
> Signed-off-by: Thomas Monjalon <thomas@monjalon.net>

Reviewed-by: Ferruh Yigit <ferruh.yigit@intel.com>

[dpdk-dev] [PATCH v3 16/29] net/pfe: release port upon close

[Thomas Monjalon]

From: Sachin Saxena <sachin.saxena@oss.nxp.com>

With removal of old close behavior, the private
port resources must be released in the .dev_close callback.
Freeing of port private resources is moved from
the ".remove(device)" to the ".dev_close(port)" operation

Signed-off-by: Sachin Saxena <sachin.saxena@oss.nxp.com>
---
 drivers/net/pfe/pfe_ethdev.c | 42 ++++++++++++++----------------------
 1 file changed, 16 insertions(+), 26 deletions(-)

diff --git a/drivers/net/pfe/pfe_ethdev.c b/drivers/net/pfe/pfe_ethdev.c
index 187a0019ff..55f2fea4b2 100644
--- a/drivers/net/pfe/pfe_ethdev.c
+++ b/drivers/net/pfe/pfe_ethdev.c
@@ -385,29 +385,23 @@ pfe_eth_stop(struct rte_eth_dev *dev/*, int wake*/)
 	dev->tx_pkt_burst = &pfe_dummy_xmit_pkts;
 }
 
-static void
-pfe_eth_exit(struct rte_eth_dev *dev, struct pfe *pfe)
-{
-	PMD_INIT_FUNC_TRACE();
-
-	pfe_eth_stop(dev);
-	/* Close the device file for link status */
-	pfe_eth_close_cdev(dev->data->dev_private);
-
-	rte_eth_dev_release_port(dev);
-	pfe->nb_devs--;
-}
-
 static int
 pfe_eth_close(struct rte_eth_dev *dev)
 {
+	PMD_INIT_FUNC_TRACE();
+
 	if (!dev)
 		return -1;
 
 	if (!g_pfe)
 		return -1;
 
-	pfe_eth_exit(dev, g_pfe);
+	pfe_eth_stop(dev);
+	/* Close the device file for link status */
+	pfe_eth_close_cdev(dev->data->dev_private);
+
+	munmap(g_pfe->cbus_baseaddr, g_pfe->cbus_size);
+	g_pfe->nb_devs--;
 
 	if (g_pfe->nb_devs == 0) {
 		pfe_hif_exit(g_pfe);
@@ -852,6 +846,8 @@ pfe_eth_init(struct rte_vdev_device *vdev, struct pfe *pfe, int id)
 	eth_dev->data->nb_rx_queues = 1;
 	eth_dev->data->nb_tx_queues = 1;
 
+	eth_dev->data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
+
 	/* For link status, open the PFE CDEV; Error from this function
 	 * is silently ignored; In case of error, the link status will not
 	 * be available.
@@ -1144,6 +1140,7 @@ pmd_pfe_remove(struct rte_vdev_device *vdev)
 {
 	const char *name;
 	struct rte_eth_dev *eth_dev = NULL;
+	int ret = 0;
 
 	name = rte_vdev_device_name(vdev);
 	if (name == NULL)
@@ -1155,19 +1152,12 @@ pmd_pfe_remove(struct rte_vdev_device *vdev)
 		return 0;
 
 	eth_dev = rte_eth_dev_allocated(name);
-	if (eth_dev == NULL)
-		return -ENODEV;
-
-	pfe_eth_exit(eth_dev, g_pfe);
-	munmap(g_pfe->cbus_baseaddr, g_pfe->cbus_size);
-
-	if (g_pfe->nb_devs == 0) {
-		pfe_hif_exit(g_pfe);
-		pfe_hif_lib_exit(g_pfe);
-		rte_free(g_pfe);
-		g_pfe = NULL;
+	if (eth_dev) {
+		pfe_eth_close(eth_dev);
+		ret = rte_eth_dev_release_port(eth_dev);
 	}
-	return 0;
+
+	return ret;
 }
 
 static
-- 
2.28.0

[dpdk-dev] [PATCH v3 17/29] net/qede: release port upon close

[Thomas Monjalon]

From: Rasesh Mody <rmody@marvell.com>

Set RTE_ETH_DEV_CLOSE_REMOVE upon probe so all the private resources
for the port can be freed by rte_eth_dev_close(). With this change the
private port resources are released in the .dev_close callback.

Signed-off-by: Rasesh Mody <rmody@marvell.com>
---
 drivers/net/qede/qede_ethdev.c | 25 +++++++++++++------------
 1 file changed, 13 insertions(+), 12 deletions(-)

diff --git a/drivers/net/qede/qede_ethdev.c b/drivers/net/qede/qede_ethdev.c
index 9535df183f..43fe68c2bf 100644
--- a/drivers/net/qede/qede_ethdev.c
+++ b/drivers/net/qede/qede_ethdev.c
@@ -1550,6 +1550,10 @@ static int qede_dev_close(struct rte_eth_dev *eth_dev)
 
 	PMD_INIT_FUNC_TRACE(edev);
 
+	/* only close in case of the primary process */
+	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
+		return 0;
+
 	/* dev_stop() shall cleanup fp resources in hw but without releasing
 	 * dma memories and sw structures so that dev_start() can be called
 	 * by the app without reconfiguration. However, in dev_close() we
@@ -1587,6 +1591,10 @@ static int qede_dev_close(struct rte_eth_dev *eth_dev)
 	if (ECORE_IS_CMT(edev))
 		rte_eal_alarm_cancel(qede_poll_sp_sb_cb, (void *)eth_dev);
 
+	eth_dev->dev_ops = NULL;
+	eth_dev->rx_pkt_burst = NULL;
+	eth_dev->tx_pkt_burst = NULL;
+
 	return 0;
 }
 
@@ -2703,6 +2711,11 @@ static int qede_common_dev_init(struct rte_eth_dev *eth_dev, bool is_vf)
 		adapter->ipgre.enable = false;
 	}
 
+	/* Pass the information to the rte_eth_dev_close() that it should also
+	 * release the private port resources.
+	 */
+	eth_dev->data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
+
 	DP_INFO(edev, "MAC address : %02x:%02x:%02x:%02x:%02x:%02x\n",
 		adapter->primary_mac.addr_bytes[0],
 		adapter->primary_mac.addr_bytes[1],
@@ -2737,20 +2750,8 @@ static int qede_dev_common_uninit(struct rte_eth_dev *eth_dev)
 {
 	struct qede_dev *qdev = eth_dev->data->dev_private;
 	struct ecore_dev *edev = &qdev->edev;
-
 	PMD_INIT_FUNC_TRACE(edev);
-
-	/* only uninitialize in the primary process */
-	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
-		return 0;
-
-	/* safe to close dev here */
 	qede_dev_close(eth_dev);
-
-	eth_dev->dev_ops = NULL;
-	eth_dev->rx_pkt_burst = NULL;
-	eth_dev->tx_pkt_burst = NULL;
-
 	return 0;
 }
 
-- 
2.28.0

[dpdk-dev] [PATCH v3 18/29] net/ring: release port upon close

[Thomas Monjalon]

The flag RTE_ETH_DEV_CLOSE_REMOVE is set so all port resources
can be freed by rte_eth_dev_close().

Freeing of private port resources is moved
from the ".remove(device)" to the ".dev_close(port)" operation.

Nothing is closed in a secondary process.

Signed-off-by: Thomas Monjalon <thomas@monjalon.net>
Acked-by: Bruce Richardson <bruce.richardson@intel.com>
---
 drivers/net/ring/rte_eth_ring.c | 54 +++++++++++++++++++++------------
 1 file changed, 34 insertions(+), 20 deletions(-)

diff --git a/drivers/net/ring/rte_eth_ring.c b/drivers/net/ring/rte_eth_ring.c
index 40fe1ca4ba..ed0fdeb28f 100644
--- a/drivers/net/ring/rte_eth_ring.c
+++ b/drivers/net/ring/rte_eth_ring.c
@@ -226,7 +226,38 @@ static int
 eth_link_update(struct rte_eth_dev *dev __rte_unused,
 		int wait_to_complete __rte_unused) { return 0; }
 
+static int
+eth_dev_close(struct rte_eth_dev *dev)
+{
+	struct pmd_internals *internals = NULL;
+	struct ring_queue *r = NULL;
+	uint16_t i;
+
+	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
+		return 0;
+
+	eth_dev_stop(dev);
+
+	internals = dev->data->dev_private;
+	if (internals->action == DEV_CREATE) {
+		/*
+		 * it is only necessary to delete the rings in rx_queues because
+		 * they are the same used in tx_queues
+		 */
+		for (i = 0; i < dev->data->nb_rx_queues; i++) {
+			r = dev->data->rx_queues[i];
+			rte_ring_free(r->rng);
+		}
+	}
+
+	/* mac_addrs must not be freed alone because part of dev_private */
+	dev->data->mac_addrs = NULL;
+
+	return 0;
+}
+
 static const struct eth_dev_ops ops = {
+	.dev_close = eth_dev_close,
 	.dev_start = eth_dev_start,
 	.dev_stop = eth_dev_stop,
 	.dev_set_link_up = eth_dev_set_link_up,
@@ -327,6 +358,7 @@ do_eth_dev_ring_create(const char *name,
 
 	eth_dev->dev_ops = &ops;
 	data->numa_node = numa_node;
+	data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
 
 	/* finally assign rx and tx ops */
 	eth_dev->rx_pkt_burst = eth_ring_rx;
@@ -658,9 +690,6 @@ rte_pmd_ring_remove(struct rte_vdev_device *dev)
 {
 	const char *name = rte_vdev_device_name(dev);
 	struct rte_eth_dev *eth_dev = NULL;
-	struct pmd_internals *internals = NULL;
-	struct ring_queue *r = NULL;
-	uint16_t i;
 
 	PMD_LOG(INFO, "Un-Initializing pmd_ring for %s", name);
 
@@ -670,24 +699,9 @@ rte_pmd_ring_remove(struct rte_vdev_device *dev)
 	/* find an ethdev entry */
 	eth_dev = rte_eth_dev_allocated(name);
 	if (eth_dev == NULL)
-		return -ENODEV;
+		return 0; /* port already released */
 
-	eth_dev_stop(eth_dev);
-
-	internals = eth_dev->data->dev_private;
-	if (internals->action == DEV_CREATE) {
-		/*
-		 * it is only necessary to delete the rings in rx_queues because
-		 * they are the same used in tx_queues
-		 */
-		for (i = 0; i < eth_dev->data->nb_rx_queues; i++) {
-			r = eth_dev->data->rx_queues[i];
-			rte_ring_free(r->rng);
-		}
-	}
-
-	/* mac_addrs must not be freed alone because part of dev_private */
-	eth_dev->data->mac_addrs = NULL;
+	eth_dev_close(eth_dev);
 	rte_eth_dev_release_port(eth_dev);
 	return 0;
 }
-- 
2.28.0

[dpdk-dev] [PATCH v3 19/29] net/softnic: release port upon close

[Thomas Monjalon]

The flag RTE_ETH_DEV_CLOSE_REMOVE is set so all port resources
can be freed by rte_eth_dev_close().

Freeing of private port resources is moved
from the ".remove(device)" to the ".dev_close(port)" operation.

Nothing is closed in a secondary process.

Signed-off-by: Thomas Monjalon <thomas@monjalon.net>
Acked-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com>
---
 drivers/net/softnic/rte_eth_softnic.c | 66 ++++++++++++++-------------
 1 file changed, 35 insertions(+), 31 deletions(-)

diff --git a/drivers/net/softnic/rte_eth_softnic.c b/drivers/net/softnic/rte_eth_softnic.c
index 038abd98da..ad9e2aa86d 100644
--- a/drivers/net/softnic/rte_eth_softnic.c
+++ b/drivers/net/softnic/rte_eth_softnic.c
@@ -201,9 +201,40 @@ pmd_dev_stop(struct rte_eth_dev *dev)
 	softnic_mtr_free(p);
 }
 
+static void
+pmd_free(struct pmd_internals *p)
+{
+	if (p == NULL)
+		return;
+
+	if (p->params.conn_port)
+		softnic_conn_free(p->conn);
+
+	softnic_thread_free(p);
+	softnic_pipeline_free(p);
+	softnic_table_action_profile_free(p);
+	softnic_port_in_action_profile_free(p);
+	softnic_tap_free(p);
+	softnic_tmgr_free(p);
+	softnic_link_free(p);
+	softnic_swq_free(p);
+	softnic_mempool_free(p);
+
+	tm_hierarchy_free(p);
+	softnic_mtr_free(p);
+
+	rte_free(p);
+}
+
 static int
-pmd_dev_close(struct rte_eth_dev *dev __rte_unused)
+pmd_dev_close(struct rte_eth_dev *dev)
 {
+	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
+		return 0;
+
+	pmd_free(dev->data->dev_private);
+	dev->data->dev_private = NULL; /* already freed */
+	dev->data->mac_addrs = NULL; /* statically allocated */
 	return 0;
 }
 
@@ -335,31 +366,6 @@ pmd_init(struct pmd_params *params)
 	return p;
 }
 
-static void
-pmd_free(struct pmd_internals *p)
-{
-	if (p == NULL)
-		return;
-
-	if (p->params.conn_port)
-		softnic_conn_free(p->conn);
-
-	softnic_thread_free(p);
-	softnic_pipeline_free(p);
-	softnic_table_action_profile_free(p);
-	softnic_port_in_action_profile_free(p);
-	softnic_tap_free(p);
-	softnic_tmgr_free(p);
-	softnic_link_free(p);
-	softnic_swq_free(p);
-	softnic_mempool_free(p);
-
-	tm_hierarchy_free(p);
-	softnic_mtr_free(p);
-
-	rte_free(p);
-}
-
 static struct rte_ether_addr eth_addr = {
 	.addr_bytes = {0},
 };
@@ -384,6 +390,7 @@ pmd_ethdev_register(struct rte_vdev_device *vdev,
 	dev->device = &vdev->device;
 
 	/* dev->data */
+	dev->data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
 	dev->data->dev_private = dev_private;
 	dev->data->dev_link.link_speed = ETH_SPEED_NUM_100G;
 	dev->data->dev_link.link_duplex = ETH_LINK_FULL_DUPLEX;
@@ -652,12 +659,9 @@ pmd_remove(struct rte_vdev_device *vdev)
 	/* Find the ethdev entry */
 	dev = rte_eth_dev_allocated(rte_vdev_device_name(vdev));
 	if (dev == NULL)
-		return -ENODEV;
+		return 0; /* port already released */
 
-	/* Free device data structures*/
-	pmd_free(dev->data->dev_private);
-	dev->data->dev_private = NULL; /* already freed */
-	dev->data->mac_addrs = NULL; /* statically allocated */
+	pmd_dev_close(dev);
 	rte_eth_dev_release_port(dev);
 
 	return 0;
-- 
2.28.0

[dpdk-dev] [PATCH v3 20/29] net/tap: release port upon close

[Thomas Monjalon]

From: Yunjian Wang <wangyunjian@huawei.com>

The flag RTE_ETH_DEV_CLOSE_REMOVE is set so all port resources
can be freed by rte_eth_dev_close().

Freeing of private port resources is moved
from the ".remove(device)" to the ".dev_close(port)" operation.

Signed-off-by: Yunjian Wang <wangyunjian@huawei.com>
Signed-off-by: Thomas Monjalon <thomas@monjalon.net>
---
 drivers/net/tap/rte_eth_tap.c | 51 +++++++++++++++++++----------------
 1 file changed, 28 insertions(+), 23 deletions(-)

diff --git a/drivers/net/tap/rte_eth_tap.c b/drivers/net/tap/rte_eth_tap.c
index 2d1e4e365a..0d217f1486 100644
--- a/drivers/net/tap/rte_eth_tap.c
+++ b/drivers/net/tap/rte_eth_tap.c
@@ -72,6 +72,10 @@
 
 static int tap_devices_count;
 
+static const char *tuntap_types[ETH_TUNTAP_TYPE_MAX] = {
+	"UNKNOWN", "TUN", "TAP"
+};
+
 static const char *valid_arguments[] = {
 	ETH_TAP_IFACE_ARG,
 	ETH_TAP_REMOTE_ARG,
@@ -1075,6 +1079,11 @@ tap_dev_close(struct rte_eth_dev *dev)
 	struct pmd_process_private *process_private = dev->process_private;
 	struct rx_queue *rxq;
 
+	if (rte_eal_process_type() != RTE_PROC_PRIMARY) {
+		rte_free(dev->process_private);
+		return 0;
+	}
+
 	tap_link_set_down(dev);
 	if (internals->nlsk_fd != -1) {
 		tap_flow_flush(dev, NULL);
@@ -1112,6 +1121,23 @@ tap_dev_close(struct rte_eth_dev *dev)
 		close(internals->ka_fd);
 		internals->ka_fd = -1;
 	}
+
+	/* mac_addrs must not be freed alone because part of dev_private */
+	dev->data->mac_addrs = NULL;
+
+	internals = dev->data->dev_private;
+	TAP_LOG(DEBUG, "Closing %s Ethernet device on numa %u",
+		tuntap_types[internals->type], rte_socket_id());
+
+	if (internals->ioctl_sock != -1) {
+		close(internals->ioctl_sock);
+		internals->ioctl_sock = -1;
+	}
+	rte_free(dev->process_private);
+	dev->process_private = NULL;
+	if (tap_devices_count == 1)
+		rte_mp_action_unregister(TAP_MP_KEY);
+	tap_devices_count--;
 	/*
 	 * Since TUN device has no more opened file descriptors
 	 * it will be removed from kernel
@@ -1845,10 +1871,6 @@ static const struct eth_dev_ops ops = {
 	.filter_ctrl            = tap_dev_filter_ctrl,
 };
 
-static const char *tuntap_types[ETH_TUNTAP_TYPE_MAX] = {
-	"UNKNOWN", "TUN", "TAP"
-};
-
 static int
 eth_dev_tap_create(struct rte_vdev_device *vdev, const char *tap_name,
 		   char *remote_iface, struct rte_ether_addr *mac_addr,
@@ -1900,7 +1922,7 @@ eth_dev_tap_create(struct rte_vdev_device *vdev, const char *tap_name,
 	/* Setup some default values */
 	data = dev->data;
 	data->dev_private = pmd;
-	data->dev_flags = RTE_ETH_DEV_INTR_LSC;
+	data->dev_flags = RTE_ETH_DEV_INTR_LSC | RTE_ETH_DEV_CLOSE_REMOVE;
 	data->numa_node = numa_node;
 
 	data->dev_link = pmd_link;
@@ -2492,30 +2514,13 @@ static int
 rte_pmd_tap_remove(struct rte_vdev_device *dev)
 {
 	struct rte_eth_dev *eth_dev = NULL;
-	struct pmd_internals *internals;
 
 	/* find the ethdev entry */
 	eth_dev = rte_eth_dev_allocated(rte_vdev_device_name(dev));
 	if (!eth_dev)
-		return -ENODEV;
-
-	/* mac_addrs must not be freed alone because part of dev_private */
-	eth_dev->data->mac_addrs = NULL;
-
-	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
-		return rte_eth_dev_release_port(eth_dev);
+		return 0;
 
 	tap_dev_close(eth_dev);
-
-	internals = eth_dev->data->dev_private;
-	TAP_LOG(DEBUG, "Closing %s Ethernet device on numa %u",
-		tuntap_types[internals->type], rte_socket_id());
-
-	close(internals->ioctl_sock);
-	rte_free(eth_dev->process_private);
-	if (tap_devices_count == 1)
-		rte_mp_action_unregister(TAP_MP_KEY);
-	tap_devices_count--;
 	rte_eth_dev_release_port(eth_dev);
 
 	return 0;
-- 
2.28.0

Re: [dpdk-dev] [PATCH v3 20/29] net/tap: release port upon close

[wangyunjian]

I have checked and tested it, no issue found.

Thanks,
Yunjian

> -----Original Message-----
> From: Thomas Monjalon [mailto:thomas@monjalon.net]
> Sent: Tuesday, September 29, 2020 7:14 AM
> To: dev@dpdk.org
> Cc: ferruh.yigit@intel.com; arybchenko@solarflare.com; wangyunjian
> <wangyunjian@huawei.com>; Keith Wiles <keith.wiles@intel.com>; Anatoly
> Burakov <anatoly.burakov@intel.com>
> Subject: [PATCH v3 20/29] net/tap: release port upon close
> 
> From: Yunjian Wang <wangyunjian@huawei.com>
> 
> The flag RTE_ETH_DEV_CLOSE_REMOVE is set so all port resources can be
> freed by rte_eth_dev_close().
> 
> Freeing of private port resources is moved from the ".remove(device)" to the
> ".dev_close(port)" operation.
> 
> Signed-off-by: Yunjian Wang <wangyunjian@huawei.com>
> Signed-off-by: Thomas Monjalon <thomas@monjalon.net>
> ---
>  drivers/net/tap/rte_eth_tap.c | 51 +++++++++++++++++++----------------
>  1 file changed, 28 insertions(+), 23 deletions(-)
> 
> diff --git a/drivers/net/tap/rte_eth_tap.c b/drivers/net/tap/rte_eth_tap.c index
> 2d1e4e365a..0d217f1486 100644
> --- a/drivers/net/tap/rte_eth_tap.c
> +++ b/drivers/net/tap/rte_eth_tap.c
> @@ -72,6 +72,10 @@
> 
>  static int tap_devices_count;
> 
> +static const char *tuntap_types[ETH_TUNTAP_TYPE_MAX] = {
> +	"UNKNOWN", "TUN", "TAP"
> +};
> +
>  static const char *valid_arguments[] = {
>  	ETH_TAP_IFACE_ARG,
>  	ETH_TAP_REMOTE_ARG,
> @@ -1075,6 +1079,11 @@ tap_dev_close(struct rte_eth_dev *dev)
>  	struct pmd_process_private *process_private = dev->process_private;
>  	struct rx_queue *rxq;
> 
> +	if (rte_eal_process_type() != RTE_PROC_PRIMARY) {
> +		rte_free(dev->process_private);
> +		return 0;
> +	}
> +
>  	tap_link_set_down(dev);
>  	if (internals->nlsk_fd != -1) {
>  		tap_flow_flush(dev, NULL);
> @@ -1112,6 +1121,23 @@ tap_dev_close(struct rte_eth_dev *dev)
>  		close(internals->ka_fd);
>  		internals->ka_fd = -1;
>  	}
> +
> +	/* mac_addrs must not be freed alone because part of dev_private */
> +	dev->data->mac_addrs = NULL;
> +
> +	internals = dev->data->dev_private;
> +	TAP_LOG(DEBUG, "Closing %s Ethernet device on numa %u",
> +		tuntap_types[internals->type], rte_socket_id());
> +
> +	if (internals->ioctl_sock != -1) {
> +		close(internals->ioctl_sock);
> +		internals->ioctl_sock = -1;
> +	}
> +	rte_free(dev->process_private);
> +	dev->process_private = NULL;
> +	if (tap_devices_count == 1)
> +		rte_mp_action_unregister(TAP_MP_KEY);
> +	tap_devices_count--;
>  	/*
>  	 * Since TUN device has no more opened file descriptors
>  	 * it will be removed from kernel
> @@ -1845,10 +1871,6 @@ static const struct eth_dev_ops ops = {
>  	.filter_ctrl            = tap_dev_filter_ctrl,
>  };
> 
> -static const char *tuntap_types[ETH_TUNTAP_TYPE_MAX] = {
> -	"UNKNOWN", "TUN", "TAP"
> -};
> -
>  static int
>  eth_dev_tap_create(struct rte_vdev_device *vdev, const char *tap_name,
>  		   char *remote_iface, struct rte_ether_addr *mac_addr, @@
> -1900,7 +1922,7 @@ eth_dev_tap_create(struct rte_vdev_device *vdev, const
> char *tap_name,
>  	/* Setup some default values */
>  	data = dev->data;
>  	data->dev_private = pmd;
> -	data->dev_flags = RTE_ETH_DEV_INTR_LSC;
> +	data->dev_flags = RTE_ETH_DEV_INTR_LSC |
> RTE_ETH_DEV_CLOSE_REMOVE;
>  	data->numa_node = numa_node;
> 
>  	data->dev_link = pmd_link;
> @@ -2492,30 +2514,13 @@ static int
>  rte_pmd_tap_remove(struct rte_vdev_device *dev)  {
>  	struct rte_eth_dev *eth_dev = NULL;
> -	struct pmd_internals *internals;
> 
>  	/* find the ethdev entry */
>  	eth_dev = rte_eth_dev_allocated(rte_vdev_device_name(dev));
>  	if (!eth_dev)
> -		return -ENODEV;
> -
> -	/* mac_addrs must not be freed alone because part of dev_private */
> -	eth_dev->data->mac_addrs = NULL;
> -
> -	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
> -		return rte_eth_dev_release_port(eth_dev);
> +		return 0;
> 
>  	tap_dev_close(eth_dev);
> -
> -	internals = eth_dev->data->dev_private;
> -	TAP_LOG(DEBUG, "Closing %s Ethernet device on numa %u",
> -		tuntap_types[internals->type], rte_socket_id());
> -
> -	close(internals->ioctl_sock);
> -	rte_free(eth_dev->process_private);
> -	if (tap_devices_count == 1)
> -		rte_mp_action_unregister(TAP_MP_KEY);
> -	tap_devices_count--;
>  	rte_eth_dev_release_port(eth_dev);
> 
>  	return 0;
> --
> 2.28.0

[dpdk-dev] [PATCH v3 21/29] ethdev: remove old close behaviour

[Thomas Monjalon]

The temporary flag RTE_ETH_DEV_CLOSE_REMOVE is removed.
It was introduced in DPDK 18.11 in order to give time for PMDs to migrate.

The old behaviour was to free only queues when closing a port.
The new behaviour is calling rte_eth_dev_release_port() which does
three more tasks:
	- trigger event callback
	- reset state and few pointers
	- free all generic port resources

The private port resources must be released in the .dev_close callback.

The .remove callback should:
	- call .dev_close callback
	- call rte_eth_dev_release_port()
	- free multi-port device shared resources

Despite waiting two years, some drivers have not migrated,
so they may hit issues with the incompatible new behaviour.
After sending emails, adding logs, and announcing the deprecation,
the only last solution is to declare these drivers as unmaintained:
	ionic, liquidio, nfp
Below is a summary of what to implement in those drivers.

* The freeing of private port resources must be moved
from the ".remove(device)" function to the ".dev_close(port)" function.

* If a generic resource (.mac_addrs or .hash_mac_addrs) cannot be freed,
it must be set to NULL in ".dev_close" function to protect from
subsequent rte_eth_dev_release_port() freeing.

* Note 1:
The generic resources are freed in rte_eth_dev_release_port(),
after ".dev_close" is called in rte_eth_dev_close(), but not when
calling ".dev_close" directly from the ".remove" PMD function.
That's why rte_eth_dev_release_port() must still be called explicitly
from ".remove(device)" after calling the ".dev_close" PMD function.

* Note 2:
If a device can have multiple ports, the common resources must be freed
only in the ".remove(device)" function.

* Note 3:
The port is supposed to be in a stopped state when it is closed.
If it is not the case, it is free to the PMD implementation
how to react when trying to close a non-stopped port:
either try to stop it automatically or just return an error.

Cc: Shijith Thotton <sthotton@marvell.com>
Cc: Srisivasubramanian Srinivasan <srinivasan@marvell.com>
Cc: Heinrich Kuhn <heinrich.kuhn@netronome.com>
Cc: Alfredo Cardigliano <cardigliano@ntop.org>

Signed-off-by: Thomas Monjalon <thomas@monjalon.net>
Reviewed-by: Liron Himi <lironh@marvell.com>
---
 MAINTAINERS                               |  6 +++---
 doc/guides/rel_notes/deprecation.rst      |  6 ------
 drivers/net/af_packet/rte_eth_af_packet.c |  1 -
 drivers/net/af_xdp/rte_eth_af_xdp.c       |  2 --
 drivers/net/ark/ark_ethdev.c              |  2 --
 drivers/net/atlantic/atl_ethdev.c         |  2 --
 drivers/net/avp/avp_ethdev.c              |  2 --
 drivers/net/axgbe/axgbe_ethdev.c          |  1 -
 drivers/net/bnx2x/bnx2x_ethdev.c          |  5 -----
 drivers/net/bnxt/bnxt_ethdev.c            |  5 -----
 drivers/net/bnxt/bnxt_reps.c              |  4 ----
 drivers/net/bonding/rte_eth_bond_pmd.c    |  1 -
 drivers/net/cxgbe/cxgbe_main.c            |  2 --
 drivers/net/cxgbe/cxgbevf_main.c          |  2 --
 drivers/net/dpaa/dpaa_ethdev.c            |  1 -
 drivers/net/dpaa2/dpaa2_ethdev.c          |  1 -
 drivers/net/e1000/em_ethdev.c             |  5 -----
 drivers/net/e1000/igb_ethdev.c            | 10 ----------
 drivers/net/ena/ena_ethdev.c              |  6 ------
 drivers/net/enetc/enetc_ethdev.c          |  1 -
 drivers/net/enic/enic_ethdev.c            |  2 --
 drivers/net/enic/enic_vf_representor.c    |  3 +--
 drivers/net/failsafe/failsafe.c           |  1 -
 drivers/net/fm10k/fm10k_ethdev.c          |  5 -----
 drivers/net/hinic/hinic_pmd_ethdev.c      |  6 ------
 drivers/net/hns3/hns3_ethdev.c            |  5 -----
 drivers/net/hns3/hns3_ethdev_vf.c         |  5 -----
 drivers/net/i40e/i40e_ethdev.c            |  5 -----
 drivers/net/i40e/i40e_ethdev_vf.c         |  5 -----
 drivers/net/iavf/iavf_ethdev.c            |  5 -----
 drivers/net/ice/ice_dcf_ethdev.c          |  2 --
 drivers/net/ice/ice_ethdev.c              |  5 -----
 drivers/net/igc/igc_ethdev.c              |  5 -----
 drivers/net/ixgbe/ixgbe_ethdev.c          | 10 ----------
 drivers/net/kni/rte_eth_kni.c             |  2 --
 drivers/net/memif/rte_eth_memif.c         |  3 ---
 drivers/net/mlx4/mlx4.c                   |  1 -
 drivers/net/mlx5/linux/mlx5_os.c          |  2 --
 drivers/net/mvneta/mvneta_ethdev.c        |  3 ---
 drivers/net/mvpp2/mrvl_ethdev.c           |  3 ---
 drivers/net/netvsc/hn_ethdev.c            |  3 ---
 drivers/net/nfb/nfb_ethdev.c              |  3 ---
 drivers/net/null/rte_eth_null.c           |  1 -
 drivers/net/octeontx/octeontx_ethdev.c    |  1 -
 drivers/net/octeontx2/otx2_ethdev.c       |  1 -
 drivers/net/pcap/rte_eth_pcap.c           |  1 -
 drivers/net/pfe/pfe_ethdev.c              |  2 --
 drivers/net/qede/qede_ethdev.c            |  5 -----
 drivers/net/ring/rte_eth_ring.c           |  1 -
 drivers/net/sfc/sfc_ethdev.c              |  4 +---
 drivers/net/softnic/rte_eth_softnic.c     |  1 -
 drivers/net/szedata2/rte_eth_szedata2.c   |  3 ---
 drivers/net/tap/rte_eth_tap.c             |  2 +-
 drivers/net/vhost/rte_eth_vhost.c         |  2 +-
 drivers/net/virtio/virtio_ethdev.c        |  5 -----
 drivers/net/vmxnet3/vmxnet3_ethdev.c      |  3 ---
 lib/librte_ethdev/rte_ethdev.c            | 17 +----------------
 lib/librte_ethdev/rte_ethdev.h            |  8 +-------
 58 files changed, 9 insertions(+), 197 deletions(-)

diff --git a/MAINTAINERS b/MAINTAINERS
index 49a6dfa7a5..6aaf80d483 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -604,7 +604,7 @@ F: drivers/net/thunderx/
 F: doc/guides/nics/thunderx.rst
 F: doc/guides/nics/features/thunderx.ini
 
-Cavium LiquidIO
+Cavium LiquidIO - UNMAINTAINED
 M: Shijith Thotton <sthotton@marvell.com>
 M: Srisivasubramanian Srinivasan <srinivasan@marvell.com>
 T: git://dpdk.org/next/dpdk-next-net-mrvl
@@ -793,7 +793,7 @@ F: drivers/net/nfb/
 F: doc/guides/nics/nfb.rst
 F: doc/guides/nics/features/nfb.ini
 
-Netronome nfp
+Netronome nfp - UNMAINTAINED
 M: Heinrich Kuhn <heinrich.kuhn@netronome.com>
 F: drivers/net/nfp/
 F: doc/guides/nics/nfp.rst
@@ -829,7 +829,7 @@ F: doc/guides/nics/pfe.rst
 F: drivers/net/pfe/
 F: doc/guides/nics/features/pfe.ini
 
-Pensando ionic
+Pensando ionic - UNMAINTAINED
 M: Alfredo Cardigliano <cardigliano@ntop.org>
 F: drivers/net/ionic/
 F: doc/guides/nics/ionic.rst
diff --git a/doc/guides/rel_notes/deprecation.rst b/doc/guides/rel_notes/deprecation.rst
index 9691f2c57d..d07b4eeb47 100644
--- a/doc/guides/rel_notes/deprecation.rst
+++ b/doc/guides/rel_notes/deprecation.rst
@@ -137,12 +137,6 @@ Deprecation Notices
   - ``rte_eth_dev_stop``
   - ``rte_eth_dev_close``
 
-* ethdev: The temporary flag RTE_ETH_DEV_CLOSE_REMOVE will be removed in 20.11.
-  As a consequence, the new behaviour introduced in 18.11 will be effective
-  for all drivers: generic port resources are freed on close operation.
-  Private resources are expected to be released in the ``dev_close`` callback.
-  More details in http://inbox.dpdk.org/dev/5248162.j6AOsuQRmx@thomas/
-
 * ethdev: New offload flags ``DEV_RX_OFFLOAD_FLOW_MARK`` will be added in 19.11.
   This will allow application to enable or disable PMDs from updating
   ``rte_mbuf::hash::fdir``.
diff --git a/drivers/net/af_packet/rte_eth_af_packet.c b/drivers/net/af_packet/rte_eth_af_packet.c
index 12c202cba4..25876224f8 100644
--- a/drivers/net/af_packet/rte_eth_af_packet.c
+++ b/drivers/net/af_packet/rte_eth_af_packet.c
@@ -859,7 +859,6 @@ rte_pmd_init_internals(struct rte_vdev_device *dev,
 	data->nb_tx_queues = (uint16_t)nb_queues;
 	data->dev_link = pmd_link;
 	data->mac_addrs = &(*internals)->eth_addr;
-	data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
 
 	(*eth_dev)->dev_ops = &ops;
 
diff --git a/drivers/net/af_xdp/rte_eth_af_xdp.c b/drivers/net/af_xdp/rte_eth_af_xdp.c
index badbce63fb..60add9ead4 100644
--- a/drivers/net/af_xdp/rte_eth_af_xdp.c
+++ b/drivers/net/af_xdp/rte_eth_af_xdp.c
@@ -1274,8 +1274,6 @@ init_internals(struct rte_vdev_device *dev, const char *if_name,
 	eth_dev->dev_ops = &ops;
 	eth_dev->rx_pkt_burst = eth_af_xdp_rx;
 	eth_dev->tx_pkt_burst = eth_af_xdp_tx;
-	/* Let rte_eth_dev_close() release the port resources. */
-	eth_dev->data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
 
 #if defined(XDP_UMEM_UNALIGNED_CHUNK_FLAG)
 	AF_XDP_LOG(INFO, "Zero copy between umem and mbuf enabled.\n");
diff --git a/drivers/net/ark/ark_ethdev.c b/drivers/net/ark/ark_ethdev.c
index 5f2ed4b7b7..83dc4ecd2c 100644
--- a/drivers/net/ark/ark_ethdev.c
+++ b/drivers/net/ark/ark_ethdev.c
@@ -260,8 +260,6 @@ eth_ark_dev_init(struct rte_eth_dev *dev)
 	/* Use dummy function until setup */
 	dev->rx_pkt_burst = &eth_ark_recv_pkts_noop;
 	dev->tx_pkt_burst = &eth_ark_xmit_pkts_noop;
-	/* Let rte_eth_dev_close() release the port resources */
-	dev->data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
 
 	ark->bar0 = (uint8_t *)pci_dev->mem_resource[0].addr;
 	ark->a_bar = (uint8_t *)pci_dev->mem_resource[2].addr;
diff --git a/drivers/net/atlantic/atl_ethdev.c b/drivers/net/atlantic/atl_ethdev.c
index d3babeff94..2217511ca0 100644
--- a/drivers/net/atlantic/atl_ethdev.c
+++ b/drivers/net/atlantic/atl_ethdev.c
@@ -380,8 +380,6 @@ eth_atl_dev_init(struct rte_eth_dev *eth_dev)
 	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
 		return 0;
 
-	eth_dev->data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
-
 	/* Vendor and Device ID need to be set before init of shared code */
 	hw->device_id = pci_dev->id.device_id;
 	hw->vendor_id = pci_dev->id.vendor_id;
diff --git a/drivers/net/avp/avp_ethdev.c b/drivers/net/avp/avp_ethdev.c
index 2eed6acc3f..c730b7ab86 100644
--- a/drivers/net/avp/avp_ethdev.c
+++ b/drivers/net/avp/avp_ethdev.c
@@ -957,8 +957,6 @@ eth_avp_dev_init(struct rte_eth_dev *eth_dev)
 	eth_dev->dev_ops = &avp_eth_dev_ops;
 	eth_dev->rx_pkt_burst = &avp_recv_pkts;
 	eth_dev->tx_pkt_burst = &avp_xmit_pkts;
-	/* Let rte_eth_dev_close() release the port resources */
-	eth_dev->data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
 
 	if (rte_eal_process_type() != RTE_PROC_PRIMARY) {
 		/*
diff --git a/drivers/net/axgbe/axgbe_ethdev.c b/drivers/net/axgbe/axgbe_ethdev.c
index 2dd64180ad..cf085487cc 100644
--- a/drivers/net/axgbe/axgbe_ethdev.c
+++ b/drivers/net/axgbe/axgbe_ethdev.c
@@ -1959,7 +1959,6 @@ eth_axgbe_dev_init(struct rte_eth_dev *eth_dev)
 	int ret;
 
 	eth_dev->dev_ops = &axgbe_eth_dev_ops;
-	eth_dev->data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
 
 	eth_dev->rx_descriptor_status = axgbe_dev_rx_descriptor_status;
 	eth_dev->tx_descriptor_status = axgbe_dev_tx_descriptor_status;
diff --git a/drivers/net/bnx2x/bnx2x_ethdev.c b/drivers/net/bnx2x/bnx2x_ethdev.c
index 8dc46384af..40225b2f44 100644
--- a/drivers/net/bnx2x/bnx2x_ethdev.c
+++ b/drivers/net/bnx2x/bnx2x_ethdev.c
@@ -735,11 +735,6 @@ bnx2x_common_dev_init(struct rte_eth_dev *eth_dev, int is_vf)
 			goto out;
 	}
 
-	/* Pass the information to the rte_eth_dev_close() that it should also
-	 * release the private port resources.
-	 */
-	eth_dev->data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
-
 	return 0;
 
 out:
diff --git a/drivers/net/bnxt/bnxt_ethdev.c b/drivers/net/bnxt/bnxt_ethdev.c
index e726d078ec..a77bab661d 100644
--- a/drivers/net/bnxt/bnxt_ethdev.c
+++ b/drivers/net/bnxt/bnxt_ethdev.c
@@ -5802,11 +5802,6 @@ bnxt_dev_init(struct rte_eth_dev *eth_dev, void *params __rte_unused)
 
 	bnxt_alloc_switch_domain(bp);
 
-	/* Pass the information to the rte_eth_dev_close() that it should also
-	 * release the private port resources.
-	 */
-	eth_dev->data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
-
 	PMD_DRV_LOG(INFO,
 		    DRV_MODULE_NAME "found at mem %" PRIX64 ", node addr %pM\n",
 		    pci_dev->mem_resource[0].phys_addr,
diff --git a/drivers/net/bnxt/bnxt_reps.c b/drivers/net/bnxt/bnxt_reps.c
index 2b6c0a277b..df8680c113 100644
--- a/drivers/net/bnxt/bnxt_reps.c
+++ b/drivers/net/bnxt/bnxt_reps.c
@@ -203,10 +203,6 @@ int bnxt_vf_representor_init(struct rte_eth_dev *eth_dev, void *params)
 	PMD_DRV_LOG(INFO, "calling bnxt_print_link_info\n");
 	bnxt_print_link_info(eth_dev);
 
-	/* Pass the information to the rte_eth_dev_close() that it should also
-	 * release the private port resources.
-	 */
-	eth_dev->data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
 	PMD_DRV_LOG(INFO,
 		    "Switch domain id %d: Representor Device %d init done\n",
 		    vf_rep_bp->switch_domain_id, vf_rep_bp->vf_id);
diff --git a/drivers/net/bonding/rte_eth_bond_pmd.c b/drivers/net/bonding/rte_eth_bond_pmd.c
index a9ed5d8e12..1f761c7c9e 100644
--- a/drivers/net/bonding/rte_eth_bond_pmd.c
+++ b/drivers/net/bonding/rte_eth_bond_pmd.c
@@ -3209,7 +3209,6 @@ bond_alloc(struct rte_vdev_device *dev, uint8_t mode)
 	}
 
 	internals = eth_dev->data->dev_private;
-	eth_dev->data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
 	eth_dev->data->nb_rx_queues = (uint16_t)1;
 	eth_dev->data->nb_tx_queues = (uint16_t)1;
 
diff --git a/drivers/net/cxgbe/cxgbe_main.c b/drivers/net/cxgbe/cxgbe_main.c
index da4ae25d47..53b08a64af 100644
--- a/drivers/net/cxgbe/cxgbe_main.c
+++ b/drivers/net/cxgbe/cxgbe_main.c
@@ -2142,8 +2142,6 @@ int cxgbe_probe(struct adapter *adapter)
 			goto out_free;
 		}
 
-		pi->eth_dev->data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
-
 		if (i > 0) {
 			/* First port will be notified by upper layer */
 			rte_eth_dev_probing_finish(eth_dev);
diff --git a/drivers/net/cxgbe/cxgbevf_main.c b/drivers/net/cxgbe/cxgbevf_main.c
index 35d873402b..9ee060504f 100644
--- a/drivers/net/cxgbe/cxgbevf_main.c
+++ b/drivers/net/cxgbe/cxgbevf_main.c
@@ -261,8 +261,6 @@ int cxgbevf_probe(struct adapter *adapter)
 			goto out_free;
 		}
 
-		pi->eth_dev->data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
-
 		if (i > 0) {
 			/* First port will be notified by upper layer */
 			rte_eth_dev_probing_finish(eth_dev);
diff --git a/drivers/net/dpaa/dpaa_ethdev.c b/drivers/net/dpaa/dpaa_ethdev.c
index 8f986683e2..a01c8f3fc6 100644
--- a/drivers/net/dpaa/dpaa_ethdev.c
+++ b/drivers/net/dpaa/dpaa_ethdev.c
@@ -2127,7 +2127,6 @@ rte_dpaa_probe(struct rte_dpaa_driver *dpaa_drv,
 
 	qman_ern_register_cb(dpaa_free_mbuf);
 
-	eth_dev->data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
 	if (dpaa_drv->drv_flags & RTE_DPAA_DRV_INTR_LSC)
 		eth_dev->data->dev_flags |= RTE_ETH_DEV_INTR_LSC;
 
diff --git a/drivers/net/dpaa2/dpaa2_ethdev.c b/drivers/net/dpaa2/dpaa2_ethdev.c
index 7bd3208a19..d8624514df 100644
--- a/drivers/net/dpaa2/dpaa2_ethdev.c
+++ b/drivers/net/dpaa2/dpaa2_ethdev.c
@@ -2793,7 +2793,6 @@ rte_dpaa2_probe(struct rte_dpaa2_driver *dpaa2_drv,
 	dpaa2_dev->eth_dev = eth_dev;
 	eth_dev->data->rx_mbuf_alloc_failed = 0;
 
-	eth_dev->data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
 	if (dpaa2_drv->drv_flags & RTE_DPAA2_DRV_INTR_LSC)
 		eth_dev->data->dev_flags |= RTE_ETH_DEV_INTR_LSC;
 
diff --git a/drivers/net/e1000/em_ethdev.c b/drivers/net/e1000/em_ethdev.c
index 2d0e071448..6a6ae0e9d5 100644
--- a/drivers/net/e1000/em_ethdev.c
+++ b/drivers/net/e1000/em_ethdev.c
@@ -297,11 +297,6 @@ eth_em_dev_init(struct rte_eth_dev *eth_dev)
 	rte_ether_addr_copy((struct rte_ether_addr *)hw->mac.addr,
 		eth_dev->data->mac_addrs);
 
-	/* Pass the information to the rte_eth_dev_close() that it should also
-	 * release the private port resources.
-	 */
-	eth_dev->data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
-
 	/* initialize the vfta */
 	memset(shadow_vfta, 0, sizeof(*shadow_vfta));
 
diff --git a/drivers/net/e1000/igb_ethdev.c b/drivers/net/e1000/igb_ethdev.c
index 9e4aefe00c..f8778207ef 100644
--- a/drivers/net/e1000/igb_ethdev.c
+++ b/drivers/net/e1000/igb_ethdev.c
@@ -841,11 +841,6 @@ eth_igb_dev_init(struct rte_eth_dev *eth_dev)
 	rte_ether_addr_copy((struct rte_ether_addr *)hw->mac.addr,
 			&eth_dev->data->mac_addrs[0]);
 
-	/* Pass the information to the rte_eth_dev_close() that it should also
-	 * release the private port resources.
-	 */
-	eth_dev->data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
-
 	/* initialize the vfta */
 	memset(shadow_vfta, 0, sizeof(*shadow_vfta));
 
@@ -997,11 +992,6 @@ eth_igbvf_dev_init(struct rte_eth_dev *eth_dev)
 		return -ENOMEM;
 	}
 
-	/* Pass the information to the rte_eth_dev_close() that it should also
-	 * release the private port resources.
-	 */
-	eth_dev->data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
-
 	/* Generate a random MAC address, if none was assigned by PF. */
 	if (rte_is_zero_ether_addr(perm_addr)) {
 		rte_eth_random_addr(perm_addr->addr_bytes);
diff --git a/drivers/net/ena/ena_ethdev.c b/drivers/net/ena/ena_ethdev.c
index ae6daad892..cf1c0b9795 100644
--- a/drivers/net/ena/ena_ethdev.c
+++ b/drivers/net/ena/ena_ethdev.c
@@ -1863,12 +1863,6 @@ static int eth_ena_dev_init(struct rte_eth_dev *eth_dev)
 			get_feat_ctx.dev_attr.mac_addr,
 			(struct rte_ether_addr *)adapter->mac_addr);
 
-	/*
-	 * Pass the information to the rte_eth_dev_close() that it should also
-	 * release the private port resources.
-	 */
-	eth_dev->data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
-
 	adapter->drv_stats = rte_zmalloc("adapter stats",
 					 sizeof(*adapter->drv_stats),
 					 RTE_CACHE_LINE_SIZE);
diff --git a/drivers/net/enetc/enetc_ethdev.c b/drivers/net/enetc/enetc_ethdev.c
index afb658df3d..325c93b92a 100644
--- a/drivers/net/enetc/enetc_ethdev.c
+++ b/drivers/net/enetc/enetc_ethdev.c
@@ -877,7 +877,6 @@ enetc_dev_init(struct rte_eth_dev *eth_dev)
 	eth_dev->dev_ops = &enetc_ops;
 	eth_dev->rx_pkt_burst = &enetc_recv_pkts;
 	eth_dev->tx_pkt_burst = &enetc_xmit_pkts;
-	eth_dev->data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
 
 	/* Retrieving and storing the HW base address of device */
 	hw->hw.reg = (void *)pci_dev->mem_resource[0].addr;
diff --git a/drivers/net/enic/enic_ethdev.c b/drivers/net/enic/enic_ethdev.c
index ed03d53608..27f60b45b8 100644
--- a/drivers/net/enic/enic_ethdev.c
+++ b/drivers/net/enic/enic_ethdev.c
@@ -1292,8 +1292,6 @@ static int eth_enic_dev_init(struct rte_eth_dev *eth_dev,
 	enic->port_id = eth_dev->data->port_id;
 	enic->rte_dev = eth_dev;
 	enic->dev_data = eth_dev->data;
-	/* Let rte_eth_dev_close() release the port resources */
-	eth_dev->data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
 
 	pdev = RTE_ETH_DEV_TO_PCI(eth_dev);
 	rte_eth_copy_pci_info(eth_dev, pdev);
diff --git a/drivers/net/enic/enic_vf_representor.c b/drivers/net/enic/enic_vf_representor.c
index c528be2b98..169c611a68 100644
--- a/drivers/net/enic/enic_vf_representor.c
+++ b/drivers/net/enic/enic_vf_representor.c
@@ -670,8 +670,7 @@ int enic_vf_representor_init(struct rte_eth_dev *eth_dev, void *init_params)
 
 	eth_dev->device->driver = pf->rte_dev->device->driver;
 	eth_dev->dev_ops = &enic_vf_representor_dev_ops;
-	eth_dev->data->dev_flags |= RTE_ETH_DEV_REPRESENTOR
-		| RTE_ETH_DEV_CLOSE_REMOVE;
+	eth_dev->data->dev_flags |= RTE_ETH_DEV_REPRESENTOR;
 	eth_dev->data->representor_id = vf->vf_id;
 	eth_dev->data->mac_addrs = rte_zmalloc("enic_mac_addr_vf",
 		sizeof(struct rte_ether_addr) *
diff --git a/drivers/net/failsafe/failsafe.c b/drivers/net/failsafe/failsafe.c
index 44d47e8f72..b921e101e6 100644
--- a/drivers/net/failsafe/failsafe.c
+++ b/drivers/net/failsafe/failsafe.c
@@ -180,7 +180,6 @@ fs_eth_dev_create(struct rte_vdev_device *vdev)
 		ERROR("Unable to allocate rte_eth_dev");
 		return -1;
 	}
-	dev->data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
 	priv = PRIV(dev);
 	priv->data = dev->data;
 	priv->rxp = FS_RX_PROXY_INIT;
diff --git a/drivers/net/fm10k/fm10k_ethdev.c b/drivers/net/fm10k/fm10k_ethdev.c
index 533f976709..23f4d04068 100644
--- a/drivers/net/fm10k/fm10k_ethdev.c
+++ b/drivers/net/fm10k/fm10k_ethdev.c
@@ -3129,11 +3129,6 @@ eth_fm10k_dev_init(struct rte_eth_dev *dev)
 		&dev->data->mac_addrs[0]);
 	}
 
-	/* Pass the information to the rte_eth_dev_close() that it should also
-	 * release the private port resources.
-	 */
-	dev->data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
-
 	/* Reset the hw statistics */
 	diag = fm10k_stats_reset(dev);
 	if (diag != 0) {
diff --git a/drivers/net/hinic/hinic_pmd_ethdev.c b/drivers/net/hinic/hinic_pmd_ethdev.c
index 5f2e97d3bd..623534fda4 100644
--- a/drivers/net/hinic/hinic_pmd_ethdev.c
+++ b/drivers/net/hinic/hinic_pmd_ethdev.c
@@ -3100,12 +3100,6 @@ static int hinic_func_init(struct rte_eth_dev *eth_dev)
 		goto mc_addr_fail;
 	}
 
-	/*
-	 * Pass the information to the rte_eth_dev_close() that it should also
-	 * release the private port resources.
-	 */
-	eth_dev->data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
-
 	/* create hardware nic_device */
 	rc = hinic_nic_dev_create(eth_dev);
 	if (rc) {
diff --git a/drivers/net/hns3/hns3_ethdev.c b/drivers/net/hns3/hns3_ethdev.c
index 3b395a1ccf..dd03bd62b2 100644
--- a/drivers/net/hns3/hns3_ethdev.c
+++ b/drivers/net/hns3/hns3_ethdev.c
@@ -5678,11 +5678,6 @@ hns3_dev_init(struct rte_eth_dev *eth_dev)
 			    &eth_dev->data->mac_addrs[0]);
 
 	hw->adapter_state = HNS3_NIC_INITIALIZED;
-	/*
-	 * Pass the information to the rte_eth_dev_close() that it should also
-	 * release the private port resources.
-	 */
-	eth_dev->data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
 
 	if (rte_atomic16_read(&hns->hw.reset.schedule) == SCHEDULE_PENDING) {
 		hns3_err(hw, "Reschedule reset service after dev_init");
diff --git a/drivers/net/hns3/hns3_ethdev_vf.c b/drivers/net/hns3/hns3_ethdev_vf.c
index e0ca5a6ac2..436d864b5f 100644
--- a/drivers/net/hns3/hns3_ethdev_vf.c
+++ b/drivers/net/hns3/hns3_ethdev_vf.c
@@ -2684,11 +2684,6 @@ hns3vf_dev_init(struct rte_eth_dev *eth_dev)
 			    &eth_dev->data->mac_addrs[0]);
 
 	hw->adapter_state = HNS3_NIC_INITIALIZED;
-	/*
-	 * Pass the information to the rte_eth_dev_close() that it should also
-	 * release the private port resources.
-	 */
-	eth_dev->data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
 
 	if (rte_atomic16_read(&hns->hw.reset.schedule) == SCHEDULE_PENDING) {
 		hns3_err(hw, "Reschedule reset service after dev_init");
diff --git a/drivers/net/i40e/i40e_ethdev.c b/drivers/net/i40e/i40e_ethdev.c
index 804f0128ab..6fb88148b8 100644
--- a/drivers/net/i40e/i40e_ethdev.c
+++ b/drivers/net/i40e/i40e_ethdev.c
@@ -1699,11 +1699,6 @@ eth_i40e_dev_init(struct rte_eth_dev *dev, void *init_params __rte_unused)
 	rte_ether_addr_copy((struct rte_ether_addr *)hw->mac.perm_addr,
 					&dev->data->mac_addrs[0]);
 
-	/* Pass the information to the rte_eth_dev_close() that it should also
-	 * release the private port resources.
-	 */
-	dev->data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
-
 	/* Init dcb to sw mode by default */
 	ret = i40e_dcb_init_configure(dev, TRUE);
 	if (ret != I40E_SUCCESS) {
diff --git a/drivers/net/i40e/i40e_ethdev_vf.c b/drivers/net/i40e/i40e_ethdev_vf.c
index 61aad8c415..4aaf41956c 100644
--- a/drivers/net/i40e/i40e_ethdev_vf.c
+++ b/drivers/net/i40e/i40e_ethdev_vf.c
@@ -1586,11 +1586,6 @@ i40evf_dev_init(struct rte_eth_dev *eth_dev)
 	hw->adapter_stopped = 1;
 	hw->adapter_closed = 0;
 
-	/* Pass the information to the rte_eth_dev_close() that it should also
-	 * release the private port resources.
-	 */
-	eth_dev->data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
-
 	if(i40evf_init_vf(eth_dev) != 0) {
 		PMD_INIT_LOG(ERR, "Init vf failed");
 		return -1;
diff --git a/drivers/net/iavf/iavf_ethdev.c b/drivers/net/iavf/iavf_ethdev.c
index 8f46c4588f..512ade2ad0 100644
--- a/drivers/net/iavf/iavf_ethdev.c
+++ b/drivers/net/iavf/iavf_ethdev.c
@@ -1412,11 +1412,6 @@ iavf_dev_init(struct rte_eth_dev *eth_dev)
 	adapter->eth_dev = eth_dev;
 	adapter->stopped = 1;
 
-	/* Pass the information to the rte_eth_dev_close() that it should also
-	 * release the private port resources.
-	 */
-	eth_dev->data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
-
 	if (iavf_init_vf(eth_dev) != 0) {
 		PMD_INIT_LOG(ERR, "Init vf failed");
 		return -1;
diff --git a/drivers/net/ice/ice_dcf_ethdev.c b/drivers/net/ice/ice_dcf_ethdev.c
index 5b626cb641..33dd0c44f2 100644
--- a/drivers/net/ice/ice_dcf_ethdev.c
+++ b/drivers/net/ice/ice_dcf_ethdev.c
@@ -908,8 +908,6 @@ ice_dcf_dev_init(struct rte_eth_dev *eth_dev)
 	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
 		return 0;
 
-	eth_dev->data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
-
 	adapter->real_hw.vc_event_msg_cb = ice_dcf_handle_pf_event_msg;
 	if (ice_dcf_init_hw(eth_dev, &adapter->real_hw) != 0) {
 		PMD_INIT_LOG(ERR, "Failed to init DCF hardware");
diff --git a/drivers/net/ice/ice_ethdev.c b/drivers/net/ice/ice_ethdev.c
index b4ac5e3f81..85a3ca6a2d 100644
--- a/drivers/net/ice/ice_ethdev.c
+++ b/drivers/net/ice/ice_ethdev.c
@@ -2245,11 +2245,6 @@ ice_dev_init(struct rte_eth_dev *dev)
 		goto err_init_mac;
 	}
 
-	/* Pass the information to the rte_eth_dev_close() that it should also
-	 * release the private port resources.
-	 */
-	dev->data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
-
 	ret = ice_res_pool_init(&pf->msix_pool, 1,
 				hw->func_caps.common_cap.num_msix_vectors - 1);
 	if (ret) {
diff --git a/drivers/net/igc/igc_ethdev.c b/drivers/net/igc/igc_ethdev.c
index 11397c5c1d..9d27fc0d07 100644
--- a/drivers/net/igc/igc_ethdev.c
+++ b/drivers/net/igc/igc_ethdev.c
@@ -1323,11 +1323,6 @@ eth_igc_dev_init(struct rte_eth_dev *dev)
 		goto err_late;
 	}
 
-	/* Pass the information to the rte_eth_dev_close() that it should also
-	 * release the private port resources.
-	 */
-	dev->data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
-
 	hw->mac.get_link_status = 1;
 	igc->stopped = 0;
 
diff --git a/drivers/net/ixgbe/ixgbe_ethdev.c b/drivers/net/ixgbe/ixgbe_ethdev.c
index 5a863ca4bf..600e9d6928 100644
--- a/drivers/net/ixgbe/ixgbe_ethdev.c
+++ b/drivers/net/ixgbe/ixgbe_ethdev.c
@@ -1248,11 +1248,6 @@ eth_ixgbe_dev_init(struct rte_eth_dev *eth_dev, void *init_params __rte_unused)
 		return -ENOMEM;
 	}
 
-	/* Pass the information to the rte_eth_dev_close() that it should also
-	 * release the private port resources.
-	 */
-	eth_dev->data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
-
 	/* initialize the vfta */
 	memset(shadow_vfta, 0, sizeof(*shadow_vfta));
 
@@ -1663,11 +1658,6 @@ eth_ixgbevf_dev_init(struct rte_eth_dev *eth_dev)
 		return -ENOMEM;
 	}
 
-	/* Pass the information to the rte_eth_dev_close() that it should also
-	 * release the private port resources.
-	 */
-	eth_dev->data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
-
 	/* Generate a random MAC address, if none was assigned by PF. */
 	if (rte_is_zero_ether_addr(perm_addr)) {
 		generate_random_mac_addr(perm_addr);
diff --git a/drivers/net/kni/rte_eth_kni.c b/drivers/net/kni/rte_eth_kni.c
index aa365a57de..45ab1b17a8 100644
--- a/drivers/net/kni/rte_eth_kni.c
+++ b/drivers/net/kni/rte_eth_kni.c
@@ -387,8 +387,6 @@ eth_kni_create(struct rte_vdev_device *vdev,
 	data->promiscuous = 1;
 	data->all_multicast = 1;
 
-	data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
-
 	rte_eth_random_addr(internals->eth_addr.addr_bytes);
 
 	eth_dev->dev_ops = &eth_kni_ops;
diff --git a/drivers/net/memif/rte_eth_memif.c b/drivers/net/memif/rte_eth_memif.c
index 1a2411c838..ff8a58081f 100644
--- a/drivers/net/memif/rte_eth_memif.c
+++ b/drivers/net/memif/rte_eth_memif.c
@@ -1527,9 +1527,6 @@ memif_create(struct rte_vdev_device *vdev, enum memif_role_t role,
 		eth_dev->tx_pkt_burst = eth_memif_tx;
 	}
 
-
-	eth_dev->data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
-
 	rte_eth_dev_probing_finish(eth_dev);
 
 	return 0;
diff --git a/drivers/net/mlx4/mlx4.c b/drivers/net/mlx4/mlx4.c
index ad7c805d67..3e57875414 100644
--- a/drivers/net/mlx4/mlx4.c
+++ b/drivers/net/mlx4/mlx4.c
@@ -1027,7 +1027,6 @@ mlx4_pci_probe(struct rte_pci_driver *pci_drv, struct rte_pci_device *pci_dev)
 			ERROR("can not allocate rte ethdev");
 			goto port_error;
 		}
-		eth_dev->data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
 		eth_dev->data->dev_private = priv;
 		eth_dev->data->mac_addrs = priv->mac;
 		eth_dev->device = &pci_dev->device;
diff --git a/drivers/net/mlx5/linux/mlx5_os.c b/drivers/net/mlx5/linux/mlx5_os.c
index 0511a55a28..8e498314ef 100644
--- a/drivers/net/mlx5/linux/mlx5_os.c
+++ b/drivers/net/mlx5/linux/mlx5_os.c
@@ -1155,8 +1155,6 @@ mlx5_dev_spawn(struct rte_device *dpdk_dev,
 		err = ENOMEM;
 		goto error;
 	}
-	/* Flag to call rte_eth_dev_release_port() in rte_eth_dev_close(). */
-	eth_dev->data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
 	if (priv->representor) {
 		eth_dev->data->dev_flags |= RTE_ETH_DEV_REPRESENTOR;
 		eth_dev->data->representor_id = priv->representor_id;
diff --git a/drivers/net/mvneta/mvneta_ethdev.c b/drivers/net/mvneta/mvneta_ethdev.c
index eebcdb840e..db142bec23 100644
--- a/drivers/net/mvneta/mvneta_ethdev.c
+++ b/drivers/net/mvneta/mvneta_ethdev.c
@@ -834,9 +834,6 @@ mvneta_eth_dev_create(struct rte_vdev_device *vdev, const char *name)
 	mvneta_set_tx_function(eth_dev);
 	eth_dev->dev_ops = &mvneta_ops;
 
-	/* Flag to call rte_eth_dev_release_port() in rte_eth_dev_close(). */
-	eth_dev->data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
-
 	rte_eth_dev_probing_finish(eth_dev);
 	return 0;
 out_free:
diff --git a/drivers/net/mvpp2/mrvl_ethdev.c b/drivers/net/mvpp2/mrvl_ethdev.c
index 27f7cfb056..cfb97e4f8f 100644
--- a/drivers/net/mvpp2/mrvl_ethdev.c
+++ b/drivers/net/mvpp2/mrvl_ethdev.c
@@ -2863,9 +2863,6 @@ mrvl_eth_dev_create(struct rte_vdev_device *vdev, const char *name)
 	mrvl_set_tx_function(eth_dev);
 	eth_dev->dev_ops = &mrvl_ops;
 
-	/* Flag to call rte_eth_dev_release_port() in rte_eth_dev_close(). */
-	eth_dev->data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
-
 	rte_eth_dev_probing_finish(eth_dev);
 	return 0;
 out_free:
diff --git a/drivers/net/netvsc/hn_ethdev.c b/drivers/net/netvsc/hn_ethdev.c
index 8968036ea3..15d6e9762d 100644
--- a/drivers/net/netvsc/hn_ethdev.c
+++ b/drivers/net/netvsc/hn_ethdev.c
@@ -123,9 +123,6 @@ eth_dev_vmbus_allocate(struct rte_vmbus_device *dev, size_t private_data_size)
 	eth_dev->data->dev_flags |= RTE_ETH_DEV_INTR_LSC;
 	eth_dev->intr_handle = &dev->intr_handle;
 
-	/* allow ethdev to remove on close */
-	eth_dev->data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
-
 	return eth_dev;
 }
 
diff --git a/drivers/net/nfb/nfb_ethdev.c b/drivers/net/nfb/nfb_ethdev.c
index c3c3d003f1..d937ac6922 100644
--- a/drivers/net/nfb/nfb_ethdev.c
+++ b/drivers/net/nfb/nfb_ethdev.c
@@ -457,9 +457,6 @@ nfb_eth_dev_init(struct rte_eth_dev *dev)
 		rte_kvargs_free(kvlist);
 	}
 
-	/* Let rte_eth_dev_close() release the port resources */
-	dev->data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
-
 	/*
 	 * Get number of available DMA RX and TX queues, which is maximum
 	 * number of queues that can be created and store it in private device
diff --git a/drivers/net/null/rte_eth_null.c b/drivers/net/null/rte_eth_null.c
index 84c1d0c951..7c3c76a897 100644
--- a/drivers/net/null/rte_eth_null.c
+++ b/drivers/net/null/rte_eth_null.c
@@ -548,7 +548,6 @@ eth_dev_null_create(struct rte_vdev_device *dev, struct pmd_options *args)
 	data->mac_addrs = &internals->eth_addr;
 	data->promiscuous = 1;
 	data->all_multicast = 1;
-	data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
 
 	eth_dev->dev_ops = &ops;
 
diff --git a/drivers/net/octeontx/octeontx_ethdev.c b/drivers/net/octeontx/octeontx_ethdev.c
index 48ce3e1621..a263f45399 100644
--- a/drivers/net/octeontx/octeontx_ethdev.c
+++ b/drivers/net/octeontx/octeontx_ethdev.c
@@ -1374,7 +1374,6 @@ octeontx_create(struct rte_vdev_device *dev, int port, uint8_t evdev,
 	data->promiscuous = 0;
 	data->all_multicast = 0;
 	data->scattered_rx = 0;
-	data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
 
 	/* Get maximum number of supported MAC entries */
 	max_entries = octeontx_bgx_port_mac_entries_get(nic->port_id);
diff --git a/drivers/net/octeontx2/otx2_ethdev.c b/drivers/net/octeontx2/otx2_ethdev.c
index abe5f03628..581ac89381 100644
--- a/drivers/net/octeontx2/otx2_ethdev.c
+++ b/drivers/net/octeontx2/otx2_ethdev.c
@@ -2394,7 +2394,6 @@ otx2_eth_dev_init(struct rte_eth_dev *eth_dev)
 	pci_dev = RTE_ETH_DEV_TO_PCI(eth_dev);
 
 	rte_eth_copy_pci_info(eth_dev, pci_dev);
-	eth_dev->data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
 
 	/* Zero out everything after OTX2_DEV to allow proper dev_reset() */
 	memset(&dev->otx2_eth_dev_data_start, 0, sizeof(*dev) -
diff --git a/drivers/net/pcap/rte_eth_pcap.c b/drivers/net/pcap/rte_eth_pcap.c
index 909eef8cce..da4988064a 100644
--- a/drivers/net/pcap/rte_eth_pcap.c
+++ b/drivers/net/pcap/rte_eth_pcap.c
@@ -1334,7 +1334,6 @@ eth_from_pcaps(struct rte_vdev_device *vdev,
 	else
 		eth_dev->tx_pkt_burst = eth_tx_drop;
 
-	eth_dev->data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
 	rte_eth_dev_probing_finish(eth_dev);
 	return 0;
 }
diff --git a/drivers/net/pfe/pfe_ethdev.c b/drivers/net/pfe/pfe_ethdev.c
index 55f2fea4b2..bb2ae0df34 100644
--- a/drivers/net/pfe/pfe_ethdev.c
+++ b/drivers/net/pfe/pfe_ethdev.c
@@ -846,8 +846,6 @@ pfe_eth_init(struct rte_vdev_device *vdev, struct pfe *pfe, int id)
 	eth_dev->data->nb_rx_queues = 1;
 	eth_dev->data->nb_tx_queues = 1;
 
-	eth_dev->data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
-
 	/* For link status, open the PFE CDEV; Error from this function
 	 * is silently ignored; In case of error, the link status will not
 	 * be available.
diff --git a/drivers/net/qede/qede_ethdev.c b/drivers/net/qede/qede_ethdev.c
index 43fe68c2bf..cd578709fb 100644
--- a/drivers/net/qede/qede_ethdev.c
+++ b/drivers/net/qede/qede_ethdev.c
@@ -2711,11 +2711,6 @@ static int qede_common_dev_init(struct rte_eth_dev *eth_dev, bool is_vf)
 		adapter->ipgre.enable = false;
 	}
 
-	/* Pass the information to the rte_eth_dev_close() that it should also
-	 * release the private port resources.
-	 */
-	eth_dev->data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
-
 	DP_INFO(edev, "MAC address : %02x:%02x:%02x:%02x:%02x:%02x\n",
 		adapter->primary_mac.addr_bytes[0],
 		adapter->primary_mac.addr_bytes[1],
diff --git a/drivers/net/ring/rte_eth_ring.c b/drivers/net/ring/rte_eth_ring.c
index ed0fdeb28f..12046f5a00 100644
--- a/drivers/net/ring/rte_eth_ring.c
+++ b/drivers/net/ring/rte_eth_ring.c
@@ -358,7 +358,6 @@ do_eth_dev_ring_create(const char *name,
 
 	eth_dev->dev_ops = &ops;
 	data->numa_node = numa_node;
-	data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
 
 	/* finally assign rx and tx ops */
 	eth_dev->rx_pkt_burst = eth_ring_rx;
diff --git a/drivers/net/sfc/sfc_ethdev.c b/drivers/net/sfc/sfc_ethdev.c
index eaa48627d6..1c72e40b75 100644
--- a/drivers/net/sfc/sfc_ethdev.c
+++ b/drivers/net/sfc/sfc_ethdev.c
@@ -343,7 +343,7 @@ sfc_dev_close(struct rte_eth_dev *dev)
 	}
 
 	/*
-	 * Cleanup all resources in accordance with RTE_ETH_DEV_CLOSE_REMOVE.
+	 * Cleanup all resources.
 	 * Rollback primary process sfc_eth_dev_init() below.
 	 */
 
@@ -2179,8 +2179,6 @@ sfc_eth_dev_init(struct rte_eth_dev *dev)
 
 	sfc_log_init(sa, "entry");
 
-	dev->data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
-
 	dev->data->mac_addrs = rte_zmalloc("sfc", RTE_ETHER_ADDR_LEN, 0);
 	if (dev->data->mac_addrs == NULL) {
 		rc = ENOMEM;
diff --git a/drivers/net/softnic/rte_eth_softnic.c b/drivers/net/softnic/rte_eth_softnic.c
index ad9e2aa86d..e942df78b6 100644
--- a/drivers/net/softnic/rte_eth_softnic.c
+++ b/drivers/net/softnic/rte_eth_softnic.c
@@ -390,7 +390,6 @@ pmd_ethdev_register(struct rte_vdev_device *vdev,
 	dev->device = &vdev->device;
 
 	/* dev->data */
-	dev->data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
 	dev->data->dev_private = dev_private;
 	dev->data->dev_link.link_speed = ETH_SPEED_NUM_100G;
 	dev->data->dev_link.link_duplex = ETH_LINK_FULL_DUPLEX;
diff --git a/drivers/net/szedata2/rte_eth_szedata2.c b/drivers/net/szedata2/rte_eth_szedata2.c
index daa40cfd6c..4325b9a30d 100644
--- a/drivers/net/szedata2/rte_eth_szedata2.c
+++ b/drivers/net/szedata2/rte_eth_szedata2.c
@@ -1488,9 +1488,6 @@ rte_szedata2_eth_dev_init(struct rte_eth_dev *dev, struct port_info *pi)
 	PMD_INIT_LOG(INFO, "Initializing eth_dev %s (driver %s)", data->name,
 			RTE_STR(RTE_SZEDATA2_DRIVER_NAME));
 
-	/* Let rte_eth_dev_close() release the port resources */
-	dev->data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
-
 	/* Fill internal private structure. */
 	internals->dev = dev;
 	/* Get index of szedata2 device file and create path to device file */
diff --git a/drivers/net/tap/rte_eth_tap.c b/drivers/net/tap/rte_eth_tap.c
index 0d217f1486..b127ce62dc 100644
--- a/drivers/net/tap/rte_eth_tap.c
+++ b/drivers/net/tap/rte_eth_tap.c
@@ -1922,7 +1922,7 @@ eth_dev_tap_create(struct rte_vdev_device *vdev, const char *tap_name,
 	/* Setup some default values */
 	data = dev->data;
 	data->dev_private = pmd;
-	data->dev_flags = RTE_ETH_DEV_INTR_LSC | RTE_ETH_DEV_CLOSE_REMOVE;
+	data->dev_flags = RTE_ETH_DEV_INTR_LSC;
 	data->numa_node = numa_node;
 
 	data->dev_link = pmd_link;
diff --git a/drivers/net/vhost/rte_eth_vhost.c b/drivers/net/vhost/rte_eth_vhost.c
index f67f4db812..45552ef742 100644
--- a/drivers/net/vhost/rte_eth_vhost.c
+++ b/drivers/net/vhost/rte_eth_vhost.c
@@ -1443,7 +1443,7 @@ eth_dev_vhost_create(struct rte_vdev_device *dev, char *iface_name,
 	internal->flags = flags;
 	internal->disable_flags = disable_flags;
 	data->dev_link = pmd_link;
-	data->dev_flags = RTE_ETH_DEV_INTR_LSC | RTE_ETH_DEV_CLOSE_REMOVE;
+	data->dev_flags = RTE_ETH_DEV_INTR_LSC;
 	data->promiscuous = 1;
 	data->all_multicast = 1;
 
diff --git a/drivers/net/virtio/virtio_ethdev.c b/drivers/net/virtio/virtio_ethdev.c
index 0787337a01..b6ed5829bd 100644
--- a/drivers/net/virtio/virtio_ethdev.c
+++ b/drivers/net/virtio/virtio_ethdev.c
@@ -1924,11 +1924,6 @@ eth_virtio_dev_init(struct rte_eth_dev *eth_dev)
 	if (ret < 0)
 		return ret;
 	hw->speed = speed;
-	/*
-	 * Pass the information to the rte_eth_dev_close() that it should also
-	 * release the private port resources.
-	 */
-	eth_dev->data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
 
 	/* Allocate memory for storing MAC addresses */
 	eth_dev->data->mac_addrs = rte_zmalloc("virtio",
diff --git a/drivers/net/vmxnet3/vmxnet3_ethdev.c b/drivers/net/vmxnet3/vmxnet3_ethdev.c
index 17fed0ed6e..34a169d2c0 100644
--- a/drivers/net/vmxnet3/vmxnet3_ethdev.c
+++ b/drivers/net/vmxnet3/vmxnet3_ethdev.c
@@ -321,9 +321,6 @@ eth_vmxnet3_dev_init(struct rte_eth_dev *eth_dev)
 		     hw->perm_addr[0], hw->perm_addr[1], hw->perm_addr[2],
 		     hw->perm_addr[3], hw->perm_addr[4], hw->perm_addr[5]);
 
-	/* Flag to call rte_eth_dev_release_port() in rte_eth_dev_close(). */
-	eth_dev->data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
-
 	/* Put device in Quiesce Mode */
 	VMXNET3_WRITE_BAR1_REG(hw, VMXNET3_REG_CMD, VMXNET3_CMD_QUIESCE_DEV);
 
diff --git a/lib/librte_ethdev/rte_ethdev.c b/lib/librte_ethdev/rte_ethdev.c
index d12d74dd8c..d7668114ca 100644
--- a/lib/librte_ethdev/rte_ethdev.c
+++ b/lib/librte_ethdev/rte_ethdev.c
@@ -1720,22 +1720,7 @@ rte_eth_dev_close(uint16_t port_id)
 	(*dev->dev_ops->dev_close)(dev);
 
 	rte_ethdev_trace_close(port_id);
-	/* check behaviour flag - temporary for PMD migration */
-	if ((dev->data->dev_flags & RTE_ETH_DEV_CLOSE_REMOVE) != 0) {
-		/* new behaviour: send event + reset state + free all data */
-		rte_eth_dev_release_port(dev);
-		return;
-	}
-	RTE_ETHDEV_LOG(DEBUG, "Port closing is using an old behaviour.\n"
-			"The driver %s should migrate to the new behaviour.\n",
-			dev->device->driver->name);
-	/* old behaviour: only free queue arrays */
-	dev->data->nb_rx_queues = 0;
-	rte_free(dev->data->rx_queues);
-	dev->data->rx_queues = NULL;
-	dev->data->nb_tx_queues = 0;
-	rte_free(dev->data->tx_queues);
-	dev->data->tx_queues = NULL;
+	rte_eth_dev_release_port(dev);
 }
 
 int
diff --git a/lib/librte_ethdev/rte_ethdev.h b/lib/librte_ethdev/rte_ethdev.h
index 645a18664d..24d898ae89 100644
--- a/lib/librte_ethdev/rte_ethdev.h
+++ b/lib/librte_ethdev/rte_ethdev.h
@@ -1654,11 +1654,6 @@ struct rte_eth_dev_owner {
 	char name[RTE_ETH_MAX_OWNER_NAME_LEN]; /**< The owner name. */
 };
 
-/**
- * Port is released (i.e. totally freed and data erased) on close.
- * Temporary flag for PMD migration to new rte_eth_dev_close() behaviour.
- */
-#define RTE_ETH_DEV_CLOSE_REMOVE 0x0001
 /** Device supports link state interrupt */
 #define RTE_ETH_DEV_INTR_LSC     0x0002
 /** Device is a bonded slave */
@@ -2282,8 +2277,7 @@ int rte_eth_dev_set_link_down(uint16_t port_id);
 
 /**
  * Close a stopped Ethernet device. The device cannot be restarted!
- * The function frees all port resources if the driver supports
- * the flag RTE_ETH_DEV_CLOSE_REMOVE.
+ * The function frees all port resources.
  *
  * @param port_id
  *   The port identifier of the Ethernet device.
-- 
2.28.0

Re: [dpdk-dev] [PATCH v3 21/29] ethdev: remove old close behaviour

[Wang, Haiyue]

> -----Original Message-----
> From: Thomas Monjalon <thomas@monjalon.net>
> Sent: Tuesday, September 29, 2020 07:14
> To: dev@dpdk.org
> Cc: Yigit, Ferruh <ferruh.yigit@intel.com>; arybchenko@solarflare.com; Shijith Thotton
> <sthotton@marvell.com>; Srisivasubramanian Srinivasan <srinivasan@marvell.com>; Heinrich Kuhn
> <heinrich.kuhn@netronome.com>; Alfredo Cardigliano <cardigliano@ntop.org>; Liron Himi
> <lironh@marvell.com>; Ray Kinsella <mdr@ashroe.eu>; Neil Horman <nhorman@tuxdriver.com>; John W.
> Linville <linville@tuxdriver.com>; Loftus, Ciara <ciara.loftus@intel.com>; Zhang, Qi Z
> <qi.z.zhang@intel.com>; Shepard Siegel <shepard.siegel@atomicrules.com>; Ed Czeck
> <ed.czeck@atomicrules.com>; John Miller <john.miller@atomicrules.com>; Igor Russkikh
> <igor.russkikh@aquantia.com>; Pavel Belous <pavel.belous@aquantia.com>; Steven Webster
> <steven.webster@windriver.com>; Matt Peters <matt.peters@windriver.com>; Somalapuram Amaranath
> <asomalap@amd.com>; Rasesh Mody <rmody@marvell.com>; Shahed Shaikh <shshaikh@marvell.com>; Ajit
> Khaparde <ajit.khaparde@broadcom.com>; Somnath Kotur <somnath.kotur@broadcom.com>; Chas Williams
> <chas3@att.com>; Wei Hu (Xavier) <xavier.huwei@huawei.com>; Rahul Lakkireddy
> <rahul.lakkireddy@chelsio.com>; Hemant Agrawal <hemant.agrawal@nxp.com>; Sachin Saxena
> <sachin.saxena@oss.nxp.com>; Guo, Jia <jia.guo@intel.com>; Wang, Haiyue <haiyue.wang@intel.com>;
> Marcin Wojtas <mw@semihalf.com>; Michal Krawczyk <mk@semihalf.com>; Guy Tzalik <gtzalik@amazon.com>;
> Evgeny Schemeilin <evgenys@amazon.com>; Igor Chauskin <igorch@amazon.com>; Gagandeep Singh
> <g.singh@nxp.com>; John Daley <johndale@cisco.com>; Hyong Youb Kim <hyonkim@cisco.com>; Gaetan Rivet
> <grive@u256.net>; Wang, Xiao W <xiao.w.wang@intel.com>; Ziyang Xuan <xuanziyang2@huawei.com>; Xiaoyun
> Wang <cloud.wangxiaoyun@huawei.com>; Guoyang Zhou <zhouguoyang@huawei.com>; Min Hu (Connor)
> <humin29@huawei.com>; Yisen Zhuang <yisen.zhuang@huawei.com>; Xing, Beilei <beilei.xing@intel.com>; Wu,
> Jingjing <jingjing.wu@intel.com>; Yang, Qiming <qiming.yang@intel.com>; Jakub Grajciar
> <jgrajcia@cisco.com>; Matan Azrad <matan@nvidia.com>; Shahaf Shuler <shahafs@nvidia.com>; Viacheslav
> Ovsiienko <viacheslavo@nvidia.com>; Zyta Szpak <zr@semihalf.com>; Stephen Hemminger
> <sthemmin@microsoft.com>; K. Y. Srinivasan <kys@microsoft.com>; Haiyang Zhang <haiyangz@microsoft.com>;
> Long Li <longli@microsoft.com>; Martin Spinler <spinler@cesnet.cz>; Tetsuya Mukawa
> <mtetsuyah@gmail.com>; Harman Kalra <hkalra@marvell.com>; Jerin Jacob <jerinj@marvell.com>; Nithin
> Dabilpuram <ndabilpuram@marvell.com>; Kiran Kumar K <kirankumark@marvell.com>; Akhil Goyal
> <akhil.goyal@nxp.com>; Richardson, Bruce <bruce.richardson@intel.com>; Singh, Jasvinder
> <jasvinder.singh@intel.com>; Dumitrescu, Cristian <cristian.dumitrescu@intel.com>; Wiles, Keith
> <keith.wiles@intel.com>; Maxime Coquelin <maxime.coquelin@redhat.com>; Xia, Chenbo
> <chenbo.xia@intel.com>; Wang, Zhihong <zhihong.wang@intel.com>; Yong Wang <yongwang@vmware.com>
> Subject: [PATCH v3 21/29] ethdev: remove old close behaviour
> 
> The temporary flag RTE_ETH_DEV_CLOSE_REMOVE is removed.
> It was introduced in DPDK 18.11 in order to give time for PMDs to migrate.
> 
> The old behaviour was to free only queues when closing a port.
> The new behaviour is calling rte_eth_dev_release_port() which does
> three more tasks:
> 	- trigger event callback
> 	- reset state and few pointers
> 	- free all generic port resources
> 
> The private port resources must be released in the .dev_close callback.
> 
> The .remove callback should:
> 	- call .dev_close callback
> 	- call rte_eth_dev_release_port()
> 	- free multi-port device shared resources
> 
> Despite waiting two years, some drivers have not migrated,
> so they may hit issues with the incompatible new behaviour.
> After sending emails, adding logs, and announcing the deprecation,
> the only last solution is to declare these drivers as unmaintained:
> 	ionic, liquidio, nfp
> Below is a summary of what to implement in those drivers.
> 
> * The freeing of private port resources must be moved
> from the ".remove(device)" function to the ".dev_close(port)" function.
> 
> * If a generic resource (.mac_addrs or .hash_mac_addrs) cannot be freed,
> it must be set to NULL in ".dev_close" function to protect from
> subsequent rte_eth_dev_release_port() freeing.
> 
> * Note 1:
> The generic resources are freed in rte_eth_dev_release_port(),
> after ".dev_close" is called in rte_eth_dev_close(), but not when
> calling ".dev_close" directly from the ".remove" PMD function.
> That's why rte_eth_dev_release_port() must still be called explicitly
> from ".remove(device)" after calling the ".dev_close" PMD function.
> 
> * Note 2:
> If a device can have multiple ports, the common resources must be freed
> only in the ".remove(device)" function.
> 
> * Note 3:
> The port is supposed to be in a stopped state when it is closed.
> If it is not the case, it is free to the PMD implementation
> how to react when trying to close a non-stopped port:
> either try to stop it automatically or just return an error.
> 
> Cc: Shijith Thotton <sthotton@marvell.com>
> Cc: Srisivasubramanian Srinivasan <srinivasan@marvell.com>
> Cc: Heinrich Kuhn <heinrich.kuhn@netronome.com>
> Cc: Alfredo Cardigliano <cardigliano@ntop.org>
> 
> Signed-off-by: Thomas Monjalon <thomas@monjalon.net>
> Reviewed-by: Liron Himi <lironh@marvell.com>
> ---

>  drivers/net/e1000/em_ethdev.c             |  5 -----
>  drivers/net/e1000/igb_ethdev.c            | 10 ----------
>  drivers/net/igc/igc_ethdev.c              |  5 -----
>  drivers/net/ixgbe/ixgbe_ethdev.c          | 10 ----------

For e1000/igc/ixgbe

Reviewed-by: Haiyue Wang <haiyue.wang@intel.com>

> 2.28.0

Re: [dpdk-dev] [PATCH v3 21/29] ethdev: remove old close behaviour

[Guo, Jia]

For drivers/net/i40e,

Acked-by: Jeff Guo <jia.guo@intel.com>

> -----Original Message-----
> From: Thomas Monjalon <thomas@monjalon.net>
> Sent: Tuesday, September 29, 2020 7:14 AM
> To: dev@dpdk.org
> Cc: Yigit, Ferruh <ferruh.yigit@intel.com>; arybchenko@solarflare.com;
> Shijith Thotton <sthotton@marvell.com>; Srisivasubramanian Srinivasan
> <srinivasan@marvell.com>; Heinrich Kuhn
> <heinrich.kuhn@netronome.com>; Alfredo Cardigliano
> <cardigliano@ntop.org>; Liron Himi <lironh@marvell.com>; Ray Kinsella
> <mdr@ashroe.eu>; Neil Horman <nhorman@tuxdriver.com>; John W.
> Linville <linville@tuxdriver.com>; Loftus, Ciara <ciara.loftus@intel.com>;
> Zhang, Qi Z <qi.z.zhang@intel.com>; Shepard Siegel
> <shepard.siegel@atomicrules.com>; Ed Czeck <ed.czeck@atomicrules.com>;
> John Miller <john.miller@atomicrules.com>; Igor Russkikh
> <igor.russkikh@aquantia.com>; Pavel Belous <pavel.belous@aquantia.com>;
> Steven Webster <steven.webster@windriver.com>; Matt Peters
> <matt.peters@windriver.com>; Somalapuram Amaranath
> <asomalap@amd.com>; Rasesh Mody <rmody@marvell.com>; Shahed
> Shaikh <shshaikh@marvell.com>; Ajit Khaparde
> <ajit.khaparde@broadcom.com>; Somnath Kotur
> <somnath.kotur@broadcom.com>; Chas Williams <chas3@att.com>; Wei Hu
> (Xavier) <xavier.huwei@huawei.com>; Rahul Lakkireddy
> <rahul.lakkireddy@chelsio.com>; Hemant Agrawal
> <hemant.agrawal@nxp.com>; Sachin Saxena <sachin.saxena@oss.nxp.com>;
> Guo, Jia <jia.guo@intel.com>; Wang, Haiyue <haiyue.wang@intel.com>;
> Marcin Wojtas <mw@semihalf.com>; Michal Krawczyk <mk@semihalf.com>;
> Guy Tzalik <gtzalik@amazon.com>; Evgeny Schemeilin
> <evgenys@amazon.com>; Igor Chauskin <igorch@amazon.com>;
> Gagandeep Singh <g.singh@nxp.com>; John Daley <johndale@cisco.com>;
> Hyong Youb Kim <hyonkim@cisco.com>; Gaetan Rivet <grive@u256.net>;
> Wang, Xiao W <xiao.w.wang@intel.com>; Ziyang Xuan
> <xuanziyang2@huawei.com>; Xiaoyun Wang
> <cloud.wangxiaoyun@huawei.com>; Guoyang Zhou
> <zhouguoyang@huawei.com>; Min Hu (Connor) <humin29@huawei.com>;
> Yisen Zhuang <yisen.zhuang@huawei.com>; Xing, Beilei
> <beilei.xing@intel.com>; Wu, Jingjing <jingjing.wu@intel.com>; Yang, Qiming
> <qiming.yang@intel.com>; Jakub Grajciar <jgrajcia@cisco.com>; Matan
> Azrad <matan@nvidia.com>; Shahaf Shuler <shahafs@nvidia.com>;
> Viacheslav Ovsiienko <viacheslavo@nvidia.com>; Zyta Szpak
> <zr@semihalf.com>; Stephen Hemminger <sthemmin@microsoft.com>; K. Y.
> Srinivasan <kys@microsoft.com>; Haiyang Zhang <haiyangz@microsoft.com>;
> Long Li <longli@microsoft.com>; Martin Spinler <spinler@cesnet.cz>;
> Tetsuya Mukawa <mtetsuyah@gmail.com>; Harman Kalra
> <hkalra@marvell.com>; Jerin Jacob <jerinj@marvell.com>; Nithin Dabilpuram
> <ndabilpuram@marvell.com>; Kiran Kumar K <kirankumark@marvell.com>;
> Akhil Goyal <akhil.goyal@nxp.com>; Richardson, Bruce
> <bruce.richardson@intel.com>; Singh, Jasvinder
> <jasvinder.singh@intel.com>; Dumitrescu, Cristian
> <cristian.dumitrescu@intel.com>; Wiles, Keith <keith.wiles@intel.com>;
> Maxime Coquelin <maxime.coquelin@redhat.com>; Xia, Chenbo
> <chenbo.xia@intel.com>; Wang, Zhihong <zhihong.wang@intel.com>; Yong
> Wang <yongwang@vmware.com>
> Subject: [PATCH v3 21/29] ethdev: remove old close behaviour
> 
> The temporary flag RTE_ETH_DEV_CLOSE_REMOVE is removed.
> It was introduced in DPDK 18.11 in order to give time for PMDs to migrate.
> 
> The old behaviour was to free only queues when closing a port.
> The new behaviour is calling rte_eth_dev_release_port() which does three
> more tasks:
> 	- trigger event callback
> 	- reset state and few pointers
> 	- free all generic port resources
> 
> The private port resources must be released in the .dev_close callback.
> 
> The .remove callback should:
> 	- call .dev_close callback
> 	- call rte_eth_dev_release_port()
> 	- free multi-port device shared resources
> 
> Despite waiting two years, some drivers have not migrated, so they may hit
> issues with the incompatible new behaviour.
> After sending emails, adding logs, and announcing the deprecation, the only
> last solution is to declare these drivers as unmaintained:
> 	ionic, liquidio, nfp
> Below is a summary of what to implement in those drivers.
> 
> * The freeing of private port resources must be moved from the
> ".remove(device)" function to the ".dev_close(port)" function.
> 
> * If a generic resource (.mac_addrs or .hash_mac_addrs) cannot be freed, it
> must be set to NULL in ".dev_close" function to protect from subsequent
> rte_eth_dev_release_port() freeing.
> 
> * Note 1:
> The generic resources are freed in rte_eth_dev_release_port(), after
> ".dev_close" is called in rte_eth_dev_close(), but not when calling
> ".dev_close" directly from the ".remove" PMD function.
> That's why rte_eth_dev_release_port() must still be called explicitly from
> ".remove(device)" after calling the ".dev_close" PMD function.
> 
> * Note 2:
> If a device can have multiple ports, the common resources must be freed
> only in the ".remove(device)" function.
> 
> * Note 3:
> The port is supposed to be in a stopped state when it is closed.
> If it is not the case, it is free to the PMD implementation how to react when
> trying to close a non-stopped port:
> either try to stop it automatically or just return an error.
> 
> Cc: Shijith Thotton <sthotton@marvell.com>
> Cc: Srisivasubramanian Srinivasan <srinivasan@marvell.com>
> Cc: Heinrich Kuhn <heinrich.kuhn@netronome.com>
> Cc: Alfredo Cardigliano <cardigliano@ntop.org>
> 
> Signed-off-by: Thomas Monjalon <thomas@monjalon.net>
> Reviewed-by: Liron Himi <lironh@marvell.com>
> ---
>  MAINTAINERS                               |  6 +++---
>  doc/guides/rel_notes/deprecation.rst      |  6 ------
>  drivers/net/af_packet/rte_eth_af_packet.c |  1 -
>  drivers/net/af_xdp/rte_eth_af_xdp.c       |  2 --
>  drivers/net/ark/ark_ethdev.c              |  2 --
>  drivers/net/atlantic/atl_ethdev.c         |  2 --
>  drivers/net/avp/avp_ethdev.c              |  2 --
>  drivers/net/axgbe/axgbe_ethdev.c          |  1 -
>  drivers/net/bnx2x/bnx2x_ethdev.c          |  5 -----
>  drivers/net/bnxt/bnxt_ethdev.c            |  5 -----
>  drivers/net/bnxt/bnxt_reps.c              |  4 ----
>  drivers/net/bonding/rte_eth_bond_pmd.c    |  1 -
>  drivers/net/cxgbe/cxgbe_main.c            |  2 --
>  drivers/net/cxgbe/cxgbevf_main.c          |  2 --
>  drivers/net/dpaa/dpaa_ethdev.c            |  1 -
>  drivers/net/dpaa2/dpaa2_ethdev.c          |  1 -
>  drivers/net/e1000/em_ethdev.c             |  5 -----
>  drivers/net/e1000/igb_ethdev.c            | 10 ----------
>  drivers/net/ena/ena_ethdev.c              |  6 ------
>  drivers/net/enetc/enetc_ethdev.c          |  1 -
>  drivers/net/enic/enic_ethdev.c            |  2 --
>  drivers/net/enic/enic_vf_representor.c    |  3 +--
>  drivers/net/failsafe/failsafe.c           |  1 -
>  drivers/net/fm10k/fm10k_ethdev.c          |  5 -----
>  drivers/net/hinic/hinic_pmd_ethdev.c      |  6 ------
>  drivers/net/hns3/hns3_ethdev.c            |  5 -----
>  drivers/net/hns3/hns3_ethdev_vf.c         |  5 -----
>  drivers/net/i40e/i40e_ethdev.c            |  5 -----
>  drivers/net/i40e/i40e_ethdev_vf.c         |  5 -----
>  drivers/net/iavf/iavf_ethdev.c            |  5 -----
>  drivers/net/ice/ice_dcf_ethdev.c          |  2 --
>  drivers/net/ice/ice_ethdev.c              |  5 -----
>  drivers/net/igc/igc_ethdev.c              |  5 -----
>  drivers/net/ixgbe/ixgbe_ethdev.c          | 10 ----------
>  drivers/net/kni/rte_eth_kni.c             |  2 --
>  drivers/net/memif/rte_eth_memif.c         |  3 ---
>  drivers/net/mlx4/mlx4.c                   |  1 -
>  drivers/net/mlx5/linux/mlx5_os.c          |  2 --
>  drivers/net/mvneta/mvneta_ethdev.c        |  3 ---
>  drivers/net/mvpp2/mrvl_ethdev.c           |  3 ---
>  drivers/net/netvsc/hn_ethdev.c            |  3 ---
>  drivers/net/nfb/nfb_ethdev.c              |  3 ---
>  drivers/net/null/rte_eth_null.c           |  1 -
>  drivers/net/octeontx/octeontx_ethdev.c    |  1 -
>  drivers/net/octeontx2/otx2_ethdev.c       |  1 -
>  drivers/net/pcap/rte_eth_pcap.c           |  1 -
>  drivers/net/pfe/pfe_ethdev.c              |  2 --
>  drivers/net/qede/qede_ethdev.c            |  5 -----
>  drivers/net/ring/rte_eth_ring.c           |  1 -
>  drivers/net/sfc/sfc_ethdev.c              |  4 +---
>  drivers/net/softnic/rte_eth_softnic.c     |  1 -
>  drivers/net/szedata2/rte_eth_szedata2.c   |  3 ---
>  drivers/net/tap/rte_eth_tap.c             |  2 +-
>  drivers/net/vhost/rte_eth_vhost.c         |  2 +-
>  drivers/net/virtio/virtio_ethdev.c        |  5 -----
>  drivers/net/vmxnet3/vmxnet3_ethdev.c      |  3 ---
>  lib/librte_ethdev/rte_ethdev.c            | 17 +----------------
>  lib/librte_ethdev/rte_ethdev.h            |  8 +-------
>  58 files changed, 9 insertions(+), 197 deletions(-)
> 
> diff --git a/MAINTAINERS b/MAINTAINERS
> index 49a6dfa7a5..6aaf80d483 100644
> --- a/MAINTAINERS
> +++ b/MAINTAINERS
> @@ -604,7 +604,7 @@ F: drivers/net/thunderx/
>  F: doc/guides/nics/thunderx.rst
>  F: doc/guides/nics/features/thunderx.ini
> 
> -Cavium LiquidIO
> +Cavium LiquidIO - UNMAINTAINED
>  M: Shijith Thotton <sthotton@marvell.com>
>  M: Srisivasubramanian Srinivasan <srinivasan@marvell.com>
>  T: git://dpdk.org/next/dpdk-next-net-mrvl
> @@ -793,7 +793,7 @@ F: drivers/net/nfb/
>  F: doc/guides/nics/nfb.rst
>  F: doc/guides/nics/features/nfb.ini
> 
> -Netronome nfp
> +Netronome nfp - UNMAINTAINED
>  M: Heinrich Kuhn <heinrich.kuhn@netronome.com>
>  F: drivers/net/nfp/
>  F: doc/guides/nics/nfp.rst
> @@ -829,7 +829,7 @@ F: doc/guides/nics/pfe.rst
>  F: drivers/net/pfe/
>  F: doc/guides/nics/features/pfe.ini
> 
> -Pensando ionic
> +Pensando ionic - UNMAINTAINED
>  M: Alfredo Cardigliano <cardigliano@ntop.org>
>  F: drivers/net/ionic/
>  F: doc/guides/nics/ionic.rst
> diff --git a/doc/guides/rel_notes/deprecation.rst
> b/doc/guides/rel_notes/deprecation.rst
> index 9691f2c57d..d07b4eeb47 100644
> --- a/doc/guides/rel_notes/deprecation.rst
> +++ b/doc/guides/rel_notes/deprecation.rst
> @@ -137,12 +137,6 @@ Deprecation Notices
>    - ``rte_eth_dev_stop``
>    - ``rte_eth_dev_close``
> 
> -* ethdev: The temporary flag RTE_ETH_DEV_CLOSE_REMOVE will be
> removed in 20.11.
> -  As a consequence, the new behaviour introduced in 18.11 will be effective
> -  for all drivers: generic port resources are freed on close operation.
> -  Private resources are expected to be released in the ``dev_close`` callback.
> -  More details in
> http://inbox.dpdk.org/dev/5248162.j6AOsuQRmx@thomas/
> -
>  * ethdev: New offload flags ``DEV_RX_OFFLOAD_FLOW_MARK`` will be
> added in 19.11.
>    This will allow application to enable or disable PMDs from updating
>    ``rte_mbuf::hash::fdir``.
> diff --git a/drivers/net/af_packet/rte_eth_af_packet.c
> b/drivers/net/af_packet/rte_eth_af_packet.c
> index 12c202cba4..25876224f8 100644
> --- a/drivers/net/af_packet/rte_eth_af_packet.c
> +++ b/drivers/net/af_packet/rte_eth_af_packet.c
> @@ -859,7 +859,6 @@ rte_pmd_init_internals(struct rte_vdev_device *dev,
>  	data->nb_tx_queues = (uint16_t)nb_queues;
>  	data->dev_link = pmd_link;
>  	data->mac_addrs = &(*internals)->eth_addr;
> -	data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
> 
>  	(*eth_dev)->dev_ops = &ops;
> 
> diff --git a/drivers/net/af_xdp/rte_eth_af_xdp.c
> b/drivers/net/af_xdp/rte_eth_af_xdp.c
> index badbce63fb..60add9ead4 100644
> --- a/drivers/net/af_xdp/rte_eth_af_xdp.c
> +++ b/drivers/net/af_xdp/rte_eth_af_xdp.c
> @@ -1274,8 +1274,6 @@ init_internals(struct rte_vdev_device *dev, const
> char *if_name,
>  	eth_dev->dev_ops = &ops;
>  	eth_dev->rx_pkt_burst = eth_af_xdp_rx;
>  	eth_dev->tx_pkt_burst = eth_af_xdp_tx;
> -	/* Let rte_eth_dev_close() release the port resources. */
> -	eth_dev->data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
> 
>  #if defined(XDP_UMEM_UNALIGNED_CHUNK_FLAG)
>  	AF_XDP_LOG(INFO, "Zero copy between umem and mbuf
> enabled.\n"); diff --git a/drivers/net/ark/ark_ethdev.c
> b/drivers/net/ark/ark_ethdev.c index 5f2ed4b7b7..83dc4ecd2c 100644
> --- a/drivers/net/ark/ark_ethdev.c
> +++ b/drivers/net/ark/ark_ethdev.c
> @@ -260,8 +260,6 @@ eth_ark_dev_init(struct rte_eth_dev *dev)
>  	/* Use dummy function until setup */
>  	dev->rx_pkt_burst = &eth_ark_recv_pkts_noop;
>  	dev->tx_pkt_burst = &eth_ark_xmit_pkts_noop;
> -	/* Let rte_eth_dev_close() release the port resources */
> -	dev->data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
> 
>  	ark->bar0 = (uint8_t *)pci_dev->mem_resource[0].addr;
>  	ark->a_bar = (uint8_t *)pci_dev->mem_resource[2].addr; diff --git
> a/drivers/net/atlantic/atl_ethdev.c b/drivers/net/atlantic/atl_ethdev.c
> index d3babeff94..2217511ca0 100644
> --- a/drivers/net/atlantic/atl_ethdev.c
> +++ b/drivers/net/atlantic/atl_ethdev.c
> @@ -380,8 +380,6 @@ eth_atl_dev_init(struct rte_eth_dev *eth_dev)
>  	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
>  		return 0;
> 
> -	eth_dev->data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
> -
>  	/* Vendor and Device ID need to be set before init of shared code */
>  	hw->device_id = pci_dev->id.device_id;
>  	hw->vendor_id = pci_dev->id.vendor_id; diff --git
> a/drivers/net/avp/avp_ethdev.c b/drivers/net/avp/avp_ethdev.c index
> 2eed6acc3f..c730b7ab86 100644
> --- a/drivers/net/avp/avp_ethdev.c
> +++ b/drivers/net/avp/avp_ethdev.c
> @@ -957,8 +957,6 @@ eth_avp_dev_init(struct rte_eth_dev *eth_dev)
>  	eth_dev->dev_ops = &avp_eth_dev_ops;
>  	eth_dev->rx_pkt_burst = &avp_recv_pkts;
>  	eth_dev->tx_pkt_burst = &avp_xmit_pkts;
> -	/* Let rte_eth_dev_close() release the port resources */
> -	eth_dev->data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
> 
>  	if (rte_eal_process_type() != RTE_PROC_PRIMARY) {
>  		/*
> diff --git a/drivers/net/axgbe/axgbe_ethdev.c
> b/drivers/net/axgbe/axgbe_ethdev.c
> index 2dd64180ad..cf085487cc 100644
> --- a/drivers/net/axgbe/axgbe_ethdev.c
> +++ b/drivers/net/axgbe/axgbe_ethdev.c
> @@ -1959,7 +1959,6 @@ eth_axgbe_dev_init(struct rte_eth_dev *eth_dev)
>  	int ret;
> 
>  	eth_dev->dev_ops = &axgbe_eth_dev_ops;
> -	eth_dev->data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
> 
>  	eth_dev->rx_descriptor_status = axgbe_dev_rx_descriptor_status;
>  	eth_dev->tx_descriptor_status = axgbe_dev_tx_descriptor_status;
> diff --git a/drivers/net/bnx2x/bnx2x_ethdev.c
> b/drivers/net/bnx2x/bnx2x_ethdev.c
> index 8dc46384af..40225b2f44 100644
> --- a/drivers/net/bnx2x/bnx2x_ethdev.c
> +++ b/drivers/net/bnx2x/bnx2x_ethdev.c
> @@ -735,11 +735,6 @@ bnx2x_common_dev_init(struct rte_eth_dev
> *eth_dev, int is_vf)
>  			goto out;
>  	}
> 
> -	/* Pass the information to the rte_eth_dev_close() that it should
> also
> -	 * release the private port resources.
> -	 */
> -	eth_dev->data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
> -
>  	return 0;
> 
>  out:
> diff --git a/drivers/net/bnxt/bnxt_ethdev.c
> b/drivers/net/bnxt/bnxt_ethdev.c index e726d078ec..a77bab661d 100644
> --- a/drivers/net/bnxt/bnxt_ethdev.c
> +++ b/drivers/net/bnxt/bnxt_ethdev.c
> @@ -5802,11 +5802,6 @@ bnxt_dev_init(struct rte_eth_dev *eth_dev, void
> *params __rte_unused)
> 
>  	bnxt_alloc_switch_domain(bp);
> 
> -	/* Pass the information to the rte_eth_dev_close() that it should
> also
> -	 * release the private port resources.
> -	 */
> -	eth_dev->data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
> -
>  	PMD_DRV_LOG(INFO,
>  		    DRV_MODULE_NAME "found at mem %" PRIX64 ", node
> addr %pM\n",
>  		    pci_dev->mem_resource[0].phys_addr,
> diff --git a/drivers/net/bnxt/bnxt_reps.c b/drivers/net/bnxt/bnxt_reps.c
> index 2b6c0a277b..df8680c113 100644
> --- a/drivers/net/bnxt/bnxt_reps.c
> +++ b/drivers/net/bnxt/bnxt_reps.c
> @@ -203,10 +203,6 @@ int bnxt_vf_representor_init(struct rte_eth_dev
> *eth_dev, void *params)
>  	PMD_DRV_LOG(INFO, "calling bnxt_print_link_info\n");
>  	bnxt_print_link_info(eth_dev);
> 
> -	/* Pass the information to the rte_eth_dev_close() that it should
> also
> -	 * release the private port resources.
> -	 */
> -	eth_dev->data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
>  	PMD_DRV_LOG(INFO,
>  		    "Switch domain id %d: Representor Device %d init done\n",
>  		    vf_rep_bp->switch_domain_id, vf_rep_bp->vf_id); diff --
> git a/drivers/net/bonding/rte_eth_bond_pmd.c
> b/drivers/net/bonding/rte_eth_bond_pmd.c
> index a9ed5d8e12..1f761c7c9e 100644
> --- a/drivers/net/bonding/rte_eth_bond_pmd.c
> +++ b/drivers/net/bonding/rte_eth_bond_pmd.c
> @@ -3209,7 +3209,6 @@ bond_alloc(struct rte_vdev_device *dev, uint8_t
> mode)
>  	}
> 
>  	internals = eth_dev->data->dev_private;
> -	eth_dev->data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
>  	eth_dev->data->nb_rx_queues = (uint16_t)1;
>  	eth_dev->data->nb_tx_queues = (uint16_t)1;
> 
> diff --git a/drivers/net/cxgbe/cxgbe_main.c
> b/drivers/net/cxgbe/cxgbe_main.c index da4ae25d47..53b08a64af 100644
> --- a/drivers/net/cxgbe/cxgbe_main.c
> +++ b/drivers/net/cxgbe/cxgbe_main.c
> @@ -2142,8 +2142,6 @@ int cxgbe_probe(struct adapter *adapter)
>  			goto out_free;
>  		}
> 
> -		pi->eth_dev->data->dev_flags |=
> RTE_ETH_DEV_CLOSE_REMOVE;
> -
>  		if (i > 0) {
>  			/* First port will be notified by upper layer */
>  			rte_eth_dev_probing_finish(eth_dev);
> diff --git a/drivers/net/cxgbe/cxgbevf_main.c
> b/drivers/net/cxgbe/cxgbevf_main.c
> index 35d873402b..9ee060504f 100644
> --- a/drivers/net/cxgbe/cxgbevf_main.c
> +++ b/drivers/net/cxgbe/cxgbevf_main.c
> @@ -261,8 +261,6 @@ int cxgbevf_probe(struct adapter *adapter)
>  			goto out_free;
>  		}
> 
> -		pi->eth_dev->data->dev_flags |=
> RTE_ETH_DEV_CLOSE_REMOVE;
> -
>  		if (i > 0) {
>  			/* First port will be notified by upper layer */
>  			rte_eth_dev_probing_finish(eth_dev);
> diff --git a/drivers/net/dpaa/dpaa_ethdev.c
> b/drivers/net/dpaa/dpaa_ethdev.c index 8f986683e2..a01c8f3fc6 100644
> --- a/drivers/net/dpaa/dpaa_ethdev.c
> +++ b/drivers/net/dpaa/dpaa_ethdev.c
> @@ -2127,7 +2127,6 @@ rte_dpaa_probe(struct rte_dpaa_driver *dpaa_drv,
> 
>  	qman_ern_register_cb(dpaa_free_mbuf);
> 
> -	eth_dev->data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
>  	if (dpaa_drv->drv_flags & RTE_DPAA_DRV_INTR_LSC)
>  		eth_dev->data->dev_flags |= RTE_ETH_DEV_INTR_LSC;
> 
> diff --git a/drivers/net/dpaa2/dpaa2_ethdev.c
> b/drivers/net/dpaa2/dpaa2_ethdev.c
> index 7bd3208a19..d8624514df 100644
> --- a/drivers/net/dpaa2/dpaa2_ethdev.c
> +++ b/drivers/net/dpaa2/dpaa2_ethdev.c
> @@ -2793,7 +2793,6 @@ rte_dpaa2_probe(struct rte_dpaa2_driver
> *dpaa2_drv,
>  	dpaa2_dev->eth_dev = eth_dev;
>  	eth_dev->data->rx_mbuf_alloc_failed = 0;
> 
> -	eth_dev->data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
>  	if (dpaa2_drv->drv_flags & RTE_DPAA2_DRV_INTR_LSC)
>  		eth_dev->data->dev_flags |= RTE_ETH_DEV_INTR_LSC;
> 
> diff --git a/drivers/net/e1000/em_ethdev.c
> b/drivers/net/e1000/em_ethdev.c index 2d0e071448..6a6ae0e9d5 100644
> --- a/drivers/net/e1000/em_ethdev.c
> +++ b/drivers/net/e1000/em_ethdev.c
> @@ -297,11 +297,6 @@ eth_em_dev_init(struct rte_eth_dev *eth_dev)
>  	rte_ether_addr_copy((struct rte_ether_addr *)hw->mac.addr,
>  		eth_dev->data->mac_addrs);
> 
> -	/* Pass the information to the rte_eth_dev_close() that it should
> also
> -	 * release the private port resources.
> -	 */
> -	eth_dev->data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
> -
>  	/* initialize the vfta */
>  	memset(shadow_vfta, 0, sizeof(*shadow_vfta));
> 
> diff --git a/drivers/net/e1000/igb_ethdev.c
> b/drivers/net/e1000/igb_ethdev.c index 9e4aefe00c..f8778207ef 100644
> --- a/drivers/net/e1000/igb_ethdev.c
> +++ b/drivers/net/e1000/igb_ethdev.c
> @@ -841,11 +841,6 @@ eth_igb_dev_init(struct rte_eth_dev *eth_dev)
>  	rte_ether_addr_copy((struct rte_ether_addr *)hw->mac.addr,
>  			&eth_dev->data->mac_addrs[0]);
> 
> -	/* Pass the information to the rte_eth_dev_close() that it should
> also
> -	 * release the private port resources.
> -	 */
> -	eth_dev->data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
> -
>  	/* initialize the vfta */
>  	memset(shadow_vfta, 0, sizeof(*shadow_vfta));
> 
> @@ -997,11 +992,6 @@ eth_igbvf_dev_init(struct rte_eth_dev *eth_dev)
>  		return -ENOMEM;
>  	}
> 
> -	/* Pass the information to the rte_eth_dev_close() that it should
> also
> -	 * release the private port resources.
> -	 */
> -	eth_dev->data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
> -
>  	/* Generate a random MAC address, if none was assigned by PF. */
>  	if (rte_is_zero_ether_addr(perm_addr)) {
>  		rte_eth_random_addr(perm_addr->addr_bytes);
> diff --git a/drivers/net/ena/ena_ethdev.c b/drivers/net/ena/ena_ethdev.c
> index ae6daad892..cf1c0b9795 100644
> --- a/drivers/net/ena/ena_ethdev.c
> +++ b/drivers/net/ena/ena_ethdev.c
> @@ -1863,12 +1863,6 @@ static int eth_ena_dev_init(struct rte_eth_dev
> *eth_dev)
>  			get_feat_ctx.dev_attr.mac_addr,
>  			(struct rte_ether_addr *)adapter->mac_addr);
> 
> -	/*
> -	 * Pass the information to the rte_eth_dev_close() that it should also
> -	 * release the private port resources.
> -	 */
> -	eth_dev->data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
> -
>  	adapter->drv_stats = rte_zmalloc("adapter stats",
>  					 sizeof(*adapter->drv_stats),
>  					 RTE_CACHE_LINE_SIZE);
> diff --git a/drivers/net/enetc/enetc_ethdev.c
> b/drivers/net/enetc/enetc_ethdev.c
> index afb658df3d..325c93b92a 100644
> --- a/drivers/net/enetc/enetc_ethdev.c
> +++ b/drivers/net/enetc/enetc_ethdev.c
> @@ -877,7 +877,6 @@ enetc_dev_init(struct rte_eth_dev *eth_dev)
>  	eth_dev->dev_ops = &enetc_ops;
>  	eth_dev->rx_pkt_burst = &enetc_recv_pkts;
>  	eth_dev->tx_pkt_burst = &enetc_xmit_pkts;
> -	eth_dev->data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
> 
>  	/* Retrieving and storing the HW base address of device */
>  	hw->hw.reg = (void *)pci_dev->mem_resource[0].addr; diff --git
> a/drivers/net/enic/enic_ethdev.c b/drivers/net/enic/enic_ethdev.c index
> ed03d53608..27f60b45b8 100644
> --- a/drivers/net/enic/enic_ethdev.c
> +++ b/drivers/net/enic/enic_ethdev.c
> @@ -1292,8 +1292,6 @@ static int eth_enic_dev_init(struct rte_eth_dev
> *eth_dev,
>  	enic->port_id = eth_dev->data->port_id;
>  	enic->rte_dev = eth_dev;
>  	enic->dev_data = eth_dev->data;
> -	/* Let rte_eth_dev_close() release the port resources */
> -	eth_dev->data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
> 
>  	pdev = RTE_ETH_DEV_TO_PCI(eth_dev);
>  	rte_eth_copy_pci_info(eth_dev, pdev);
> diff --git a/drivers/net/enic/enic_vf_representor.c
> b/drivers/net/enic/enic_vf_representor.c
> index c528be2b98..169c611a68 100644
> --- a/drivers/net/enic/enic_vf_representor.c
> +++ b/drivers/net/enic/enic_vf_representor.c
> @@ -670,8 +670,7 @@ int enic_vf_representor_init(struct rte_eth_dev
> *eth_dev, void *init_params)
> 
>  	eth_dev->device->driver = pf->rte_dev->device->driver;
>  	eth_dev->dev_ops = &enic_vf_representor_dev_ops;
> -	eth_dev->data->dev_flags |= RTE_ETH_DEV_REPRESENTOR
> -		| RTE_ETH_DEV_CLOSE_REMOVE;
> +	eth_dev->data->dev_flags |= RTE_ETH_DEV_REPRESENTOR;
>  	eth_dev->data->representor_id = vf->vf_id;
>  	eth_dev->data->mac_addrs = rte_zmalloc("enic_mac_addr_vf",
>  		sizeof(struct rte_ether_addr) *
> diff --git a/drivers/net/failsafe/failsafe.c b/drivers/net/failsafe/failsafe.c
> index 44d47e8f72..b921e101e6 100644
> --- a/drivers/net/failsafe/failsafe.c
> +++ b/drivers/net/failsafe/failsafe.c
> @@ -180,7 +180,6 @@ fs_eth_dev_create(struct rte_vdev_device *vdev)
>  		ERROR("Unable to allocate rte_eth_dev");
>  		return -1;
>  	}
> -	dev->data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
>  	priv = PRIV(dev);
>  	priv->data = dev->data;
>  	priv->rxp = FS_RX_PROXY_INIT;
> diff --git a/drivers/net/fm10k/fm10k_ethdev.c
> b/drivers/net/fm10k/fm10k_ethdev.c
> index 533f976709..23f4d04068 100644
> --- a/drivers/net/fm10k/fm10k_ethdev.c
> +++ b/drivers/net/fm10k/fm10k_ethdev.c
> @@ -3129,11 +3129,6 @@ eth_fm10k_dev_init(struct rte_eth_dev *dev)
>  		&dev->data->mac_addrs[0]);
>  	}
> 
> -	/* Pass the information to the rte_eth_dev_close() that it should
> also
> -	 * release the private port resources.
> -	 */
> -	dev->data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
> -
>  	/* Reset the hw statistics */
>  	diag = fm10k_stats_reset(dev);
>  	if (diag != 0) {
> diff --git a/drivers/net/hinic/hinic_pmd_ethdev.c
> b/drivers/net/hinic/hinic_pmd_ethdev.c
> index 5f2e97d3bd..623534fda4 100644
> --- a/drivers/net/hinic/hinic_pmd_ethdev.c
> +++ b/drivers/net/hinic/hinic_pmd_ethdev.c
> @@ -3100,12 +3100,6 @@ static int hinic_func_init(struct rte_eth_dev
> *eth_dev)
>  		goto mc_addr_fail;
>  	}
> 
> -	/*
> -	 * Pass the information to the rte_eth_dev_close() that it should also
> -	 * release the private port resources.
> -	 */
> -	eth_dev->data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
> -
>  	/* create hardware nic_device */
>  	rc = hinic_nic_dev_create(eth_dev);
>  	if (rc) {
> diff --git a/drivers/net/hns3/hns3_ethdev.c
> b/drivers/net/hns3/hns3_ethdev.c index 3b395a1ccf..dd03bd62b2 100644
> --- a/drivers/net/hns3/hns3_ethdev.c
> +++ b/drivers/net/hns3/hns3_ethdev.c
> @@ -5678,11 +5678,6 @@ hns3_dev_init(struct rte_eth_dev *eth_dev)
>  			    &eth_dev->data->mac_addrs[0]);
> 
>  	hw->adapter_state = HNS3_NIC_INITIALIZED;
> -	/*
> -	 * Pass the information to the rte_eth_dev_close() that it should also
> -	 * release the private port resources.
> -	 */
> -	eth_dev->data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
> 
>  	if (rte_atomic16_read(&hns->hw.reset.schedule) ==
> SCHEDULE_PENDING) {
>  		hns3_err(hw, "Reschedule reset service after dev_init"); diff
> --git a/drivers/net/hns3/hns3_ethdev_vf.c
> b/drivers/net/hns3/hns3_ethdev_vf.c
> index e0ca5a6ac2..436d864b5f 100644
> --- a/drivers/net/hns3/hns3_ethdev_vf.c
> +++ b/drivers/net/hns3/hns3_ethdev_vf.c
> @@ -2684,11 +2684,6 @@ hns3vf_dev_init(struct rte_eth_dev *eth_dev)
>  			    &eth_dev->data->mac_addrs[0]);
> 
>  	hw->adapter_state = HNS3_NIC_INITIALIZED;
> -	/*
> -	 * Pass the information to the rte_eth_dev_close() that it should also
> -	 * release the private port resources.
> -	 */
> -	eth_dev->data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
> 
>  	if (rte_atomic16_read(&hns->hw.reset.schedule) ==
> SCHEDULE_PENDING) {
>  		hns3_err(hw, "Reschedule reset service after dev_init"); diff
> --git a/drivers/net/i40e/i40e_ethdev.c b/drivers/net/i40e/i40e_ethdev.c
> index 804f0128ab..6fb88148b8 100644
> --- a/drivers/net/i40e/i40e_ethdev.c
> +++ b/drivers/net/i40e/i40e_ethdev.c
> @@ -1699,11 +1699,6 @@ eth_i40e_dev_init(struct rte_eth_dev *dev, void
> *init_params __rte_unused)
>  	rte_ether_addr_copy((struct rte_ether_addr *)hw-
> >mac.perm_addr,
>  					&dev->data->mac_addrs[0]);
> 
> -	/* Pass the information to the rte_eth_dev_close() that it should
> also
> -	 * release the private port resources.
> -	 */
> -	dev->data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
> -
>  	/* Init dcb to sw mode by default */
>  	ret = i40e_dcb_init_configure(dev, TRUE);
>  	if (ret != I40E_SUCCESS) {
> diff --git a/drivers/net/i40e/i40e_ethdev_vf.c
> b/drivers/net/i40e/i40e_ethdev_vf.c
> index 61aad8c415..4aaf41956c 100644
> --- a/drivers/net/i40e/i40e_ethdev_vf.c
> +++ b/drivers/net/i40e/i40e_ethdev_vf.c
> @@ -1586,11 +1586,6 @@ i40evf_dev_init(struct rte_eth_dev *eth_dev)
>  	hw->adapter_stopped = 1;
>  	hw->adapter_closed = 0;
> 
> -	/* Pass the information to the rte_eth_dev_close() that it should
> also
> -	 * release the private port resources.
> -	 */
> -	eth_dev->data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
> -
>  	if(i40evf_init_vf(eth_dev) != 0) {
>  		PMD_INIT_LOG(ERR, "Init vf failed");
>  		return -1;
> diff --git a/drivers/net/iavf/iavf_ethdev.c b/drivers/net/iavf/iavf_ethdev.c
> index 8f46c4588f..512ade2ad0 100644
> --- a/drivers/net/iavf/iavf_ethdev.c
> +++ b/drivers/net/iavf/iavf_ethdev.c
> @@ -1412,11 +1412,6 @@ iavf_dev_init(struct rte_eth_dev *eth_dev)
>  	adapter->eth_dev = eth_dev;
>  	adapter->stopped = 1;
> 
> -	/* Pass the information to the rte_eth_dev_close() that it should
> also
> -	 * release the private port resources.
> -	 */
> -	eth_dev->data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
> -
>  	if (iavf_init_vf(eth_dev) != 0) {
>  		PMD_INIT_LOG(ERR, "Init vf failed");
>  		return -1;
> diff --git a/drivers/net/ice/ice_dcf_ethdev.c
> b/drivers/net/ice/ice_dcf_ethdev.c
> index 5b626cb641..33dd0c44f2 100644
> --- a/drivers/net/ice/ice_dcf_ethdev.c
> +++ b/drivers/net/ice/ice_dcf_ethdev.c
> @@ -908,8 +908,6 @@ ice_dcf_dev_init(struct rte_eth_dev *eth_dev)
>  	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
>  		return 0;
> 
> -	eth_dev->data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
> -
>  	adapter->real_hw.vc_event_msg_cb =
> ice_dcf_handle_pf_event_msg;
>  	if (ice_dcf_init_hw(eth_dev, &adapter->real_hw) != 0) {
>  		PMD_INIT_LOG(ERR, "Failed to init DCF hardware"); diff --git
> a/drivers/net/ice/ice_ethdev.c b/drivers/net/ice/ice_ethdev.c index
> b4ac5e3f81..85a3ca6a2d 100644
> --- a/drivers/net/ice/ice_ethdev.c
> +++ b/drivers/net/ice/ice_ethdev.c
> @@ -2245,11 +2245,6 @@ ice_dev_init(struct rte_eth_dev *dev)
>  		goto err_init_mac;
>  	}
> 
> -	/* Pass the information to the rte_eth_dev_close() that it should
> also
> -	 * release the private port resources.
> -	 */
> -	dev->data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
> -
>  	ret = ice_res_pool_init(&pf->msix_pool, 1,
>  				hw-
> >func_caps.common_cap.num_msix_vectors - 1);
>  	if (ret) {
> diff --git a/drivers/net/igc/igc_ethdev.c b/drivers/net/igc/igc_ethdev.c index
> 11397c5c1d..9d27fc0d07 100644
> --- a/drivers/net/igc/igc_ethdev.c
> +++ b/drivers/net/igc/igc_ethdev.c
> @@ -1323,11 +1323,6 @@ eth_igc_dev_init(struct rte_eth_dev *dev)
>  		goto err_late;
>  	}
> 
> -	/* Pass the information to the rte_eth_dev_close() that it should
> also
> -	 * release the private port resources.
> -	 */
> -	dev->data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
> -
>  	hw->mac.get_link_status = 1;
>  	igc->stopped = 0;
> 
> diff --git a/drivers/net/ixgbe/ixgbe_ethdev.c
> b/drivers/net/ixgbe/ixgbe_ethdev.c
> index 5a863ca4bf..600e9d6928 100644
> --- a/drivers/net/ixgbe/ixgbe_ethdev.c
> +++ b/drivers/net/ixgbe/ixgbe_ethdev.c
> @@ -1248,11 +1248,6 @@ eth_ixgbe_dev_init(struct rte_eth_dev *eth_dev,
> void *init_params __rte_unused)
>  		return -ENOMEM;
>  	}
> 
> -	/* Pass the information to the rte_eth_dev_close() that it should
> also
> -	 * release the private port resources.
> -	 */
> -	eth_dev->data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
> -
>  	/* initialize the vfta */
>  	memset(shadow_vfta, 0, sizeof(*shadow_vfta));
> 
> @@ -1663,11 +1658,6 @@ eth_ixgbevf_dev_init(struct rte_eth_dev
> *eth_dev)
>  		return -ENOMEM;
>  	}
> 
> -	/* Pass the information to the rte_eth_dev_close() that it should
> also
> -	 * release the private port resources.
> -	 */
> -	eth_dev->data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
> -
>  	/* Generate a random MAC address, if none was assigned by PF. */
>  	if (rte_is_zero_ether_addr(perm_addr)) {
>  		generate_random_mac_addr(perm_addr);
> diff --git a/drivers/net/kni/rte_eth_kni.c b/drivers/net/kni/rte_eth_kni.c
> index aa365a57de..45ab1b17a8 100644
> --- a/drivers/net/kni/rte_eth_kni.c
> +++ b/drivers/net/kni/rte_eth_kni.c
> @@ -387,8 +387,6 @@ eth_kni_create(struct rte_vdev_device *vdev,
>  	data->promiscuous = 1;
>  	data->all_multicast = 1;
> 
> -	data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
> -
>  	rte_eth_random_addr(internals->eth_addr.addr_bytes);
> 
>  	eth_dev->dev_ops = &eth_kni_ops;
> diff --git a/drivers/net/memif/rte_eth_memif.c
> b/drivers/net/memif/rte_eth_memif.c
> index 1a2411c838..ff8a58081f 100644
> --- a/drivers/net/memif/rte_eth_memif.c
> +++ b/drivers/net/memif/rte_eth_memif.c
> @@ -1527,9 +1527,6 @@ memif_create(struct rte_vdev_device *vdev,
> enum memif_role_t role,
>  		eth_dev->tx_pkt_burst = eth_memif_tx;
>  	}
> 
> -
> -	eth_dev->data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
> -
>  	rte_eth_dev_probing_finish(eth_dev);
> 
>  	return 0;
> diff --git a/drivers/net/mlx4/mlx4.c b/drivers/net/mlx4/mlx4.c index
> ad7c805d67..3e57875414 100644
> --- a/drivers/net/mlx4/mlx4.c
> +++ b/drivers/net/mlx4/mlx4.c
> @@ -1027,7 +1027,6 @@ mlx4_pci_probe(struct rte_pci_driver *pci_drv,
> struct rte_pci_device *pci_dev)
>  			ERROR("can not allocate rte ethdev");
>  			goto port_error;
>  		}
> -		eth_dev->data->dev_flags |=
> RTE_ETH_DEV_CLOSE_REMOVE;
>  		eth_dev->data->dev_private = priv;
>  		eth_dev->data->mac_addrs = priv->mac;
>  		eth_dev->device = &pci_dev->device;
> diff --git a/drivers/net/mlx5/linux/mlx5_os.c
> b/drivers/net/mlx5/linux/mlx5_os.c
> index 0511a55a28..8e498314ef 100644
> --- a/drivers/net/mlx5/linux/mlx5_os.c
> +++ b/drivers/net/mlx5/linux/mlx5_os.c
> @@ -1155,8 +1155,6 @@ mlx5_dev_spawn(struct rte_device *dpdk_dev,
>  		err = ENOMEM;
>  		goto error;
>  	}
> -	/* Flag to call rte_eth_dev_release_port() in rte_eth_dev_close(). */
> -	eth_dev->data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
>  	if (priv->representor) {
>  		eth_dev->data->dev_flags |= RTE_ETH_DEV_REPRESENTOR;
>  		eth_dev->data->representor_id = priv->representor_id; diff
> --git a/drivers/net/mvneta/mvneta_ethdev.c
> b/drivers/net/mvneta/mvneta_ethdev.c
> index eebcdb840e..db142bec23 100644
> --- a/drivers/net/mvneta/mvneta_ethdev.c
> +++ b/drivers/net/mvneta/mvneta_ethdev.c
> @@ -834,9 +834,6 @@ mvneta_eth_dev_create(struct rte_vdev_device
> *vdev, const char *name)
>  	mvneta_set_tx_function(eth_dev);
>  	eth_dev->dev_ops = &mvneta_ops;
> 
> -	/* Flag to call rte_eth_dev_release_port() in rte_eth_dev_close(). */
> -	eth_dev->data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
> -
>  	rte_eth_dev_probing_finish(eth_dev);
>  	return 0;
>  out_free:
> diff --git a/drivers/net/mvpp2/mrvl_ethdev.c
> b/drivers/net/mvpp2/mrvl_ethdev.c index 27f7cfb056..cfb97e4f8f 100644
> --- a/drivers/net/mvpp2/mrvl_ethdev.c
> +++ b/drivers/net/mvpp2/mrvl_ethdev.c
> @@ -2863,9 +2863,6 @@ mrvl_eth_dev_create(struct rte_vdev_device
> *vdev, const char *name)
>  	mrvl_set_tx_function(eth_dev);
>  	eth_dev->dev_ops = &mrvl_ops;
> 
> -	/* Flag to call rte_eth_dev_release_port() in rte_eth_dev_close(). */
> -	eth_dev->data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
> -
>  	rte_eth_dev_probing_finish(eth_dev);
>  	return 0;
>  out_free:
> diff --git a/drivers/net/netvsc/hn_ethdev.c
> b/drivers/net/netvsc/hn_ethdev.c index 8968036ea3..15d6e9762d 100644
> --- a/drivers/net/netvsc/hn_ethdev.c
> +++ b/drivers/net/netvsc/hn_ethdev.c
> @@ -123,9 +123,6 @@ eth_dev_vmbus_allocate(struct rte_vmbus_device
> *dev, size_t private_data_size)
>  	eth_dev->data->dev_flags |= RTE_ETH_DEV_INTR_LSC;
>  	eth_dev->intr_handle = &dev->intr_handle;
> 
> -	/* allow ethdev to remove on close */
> -	eth_dev->data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
> -
>  	return eth_dev;
>  }
> 
> diff --git a/drivers/net/nfb/nfb_ethdev.c b/drivers/net/nfb/nfb_ethdev.c
> index c3c3d003f1..d937ac6922 100644
> --- a/drivers/net/nfb/nfb_ethdev.c
> +++ b/drivers/net/nfb/nfb_ethdev.c
> @@ -457,9 +457,6 @@ nfb_eth_dev_init(struct rte_eth_dev *dev)
>  		rte_kvargs_free(kvlist);
>  	}
> 
> -	/* Let rte_eth_dev_close() release the port resources */
> -	dev->data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
> -
>  	/*
>  	 * Get number of available DMA RX and TX queues, which is
> maximum
>  	 * number of queues that can be created and store it in private
> device diff --git a/drivers/net/null/rte_eth_null.c
> b/drivers/net/null/rte_eth_null.c index 84c1d0c951..7c3c76a897 100644
> --- a/drivers/net/null/rte_eth_null.c
> +++ b/drivers/net/null/rte_eth_null.c
> @@ -548,7 +548,6 @@ eth_dev_null_create(struct rte_vdev_device *dev,
> struct pmd_options *args)
>  	data->mac_addrs = &internals->eth_addr;
>  	data->promiscuous = 1;
>  	data->all_multicast = 1;
> -	data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
> 
>  	eth_dev->dev_ops = &ops;
> 
> diff --git a/drivers/net/octeontx/octeontx_ethdev.c
> b/drivers/net/octeontx/octeontx_ethdev.c
> index 48ce3e1621..a263f45399 100644
> --- a/drivers/net/octeontx/octeontx_ethdev.c
> +++ b/drivers/net/octeontx/octeontx_ethdev.c
> @@ -1374,7 +1374,6 @@ octeontx_create(struct rte_vdev_device *dev, int
> port, uint8_t evdev,
>  	data->promiscuous = 0;
>  	data->all_multicast = 0;
>  	data->scattered_rx = 0;
> -	data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
> 
>  	/* Get maximum number of supported MAC entries */
>  	max_entries = octeontx_bgx_port_mac_entries_get(nic->port_id);
> diff --git a/drivers/net/octeontx2/otx2_ethdev.c
> b/drivers/net/octeontx2/otx2_ethdev.c
> index abe5f03628..581ac89381 100644
> --- a/drivers/net/octeontx2/otx2_ethdev.c
> +++ b/drivers/net/octeontx2/otx2_ethdev.c
> @@ -2394,7 +2394,6 @@ otx2_eth_dev_init(struct rte_eth_dev *eth_dev)
>  	pci_dev = RTE_ETH_DEV_TO_PCI(eth_dev);
> 
>  	rte_eth_copy_pci_info(eth_dev, pci_dev);
> -	eth_dev->data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
> 
>  	/* Zero out everything after OTX2_DEV to allow proper dev_reset()
> */
>  	memset(&dev->otx2_eth_dev_data_start, 0, sizeof(*dev) - diff --git
> a/drivers/net/pcap/rte_eth_pcap.c b/drivers/net/pcap/rte_eth_pcap.c
> index 909eef8cce..da4988064a 100644
> --- a/drivers/net/pcap/rte_eth_pcap.c
> +++ b/drivers/net/pcap/rte_eth_pcap.c
> @@ -1334,7 +1334,6 @@ eth_from_pcaps(struct rte_vdev_device *vdev,
>  	else
>  		eth_dev->tx_pkt_burst = eth_tx_drop;
> 
> -	eth_dev->data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
>  	rte_eth_dev_probing_finish(eth_dev);
>  	return 0;
>  }
> diff --git a/drivers/net/pfe/pfe_ethdev.c b/drivers/net/pfe/pfe_ethdev.c
> index 55f2fea4b2..bb2ae0df34 100644
> --- a/drivers/net/pfe/pfe_ethdev.c
> +++ b/drivers/net/pfe/pfe_ethdev.c
> @@ -846,8 +846,6 @@ pfe_eth_init(struct rte_vdev_device *vdev, struct
> pfe *pfe, int id)
>  	eth_dev->data->nb_rx_queues = 1;
>  	eth_dev->data->nb_tx_queues = 1;
> 
> -	eth_dev->data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
> -
>  	/* For link status, open the PFE CDEV; Error from this function
>  	 * is silently ignored; In case of error, the link status will not
>  	 * be available.
> diff --git a/drivers/net/qede/qede_ethdev.c
> b/drivers/net/qede/qede_ethdev.c index 43fe68c2bf..cd578709fb 100644
> --- a/drivers/net/qede/qede_ethdev.c
> +++ b/drivers/net/qede/qede_ethdev.c
> @@ -2711,11 +2711,6 @@ static int qede_common_dev_init(struct
> rte_eth_dev *eth_dev, bool is_vf)
>  		adapter->ipgre.enable = false;
>  	}
> 
> -	/* Pass the information to the rte_eth_dev_close() that it should
> also
> -	 * release the private port resources.
> -	 */
> -	eth_dev->data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
> -
>  	DP_INFO(edev, "MAC
> address : %02x:%02x:%02x:%02x:%02x:%02x\n",
>  		adapter->primary_mac.addr_bytes[0],
>  		adapter->primary_mac.addr_bytes[1],
> diff --git a/drivers/net/ring/rte_eth_ring.c b/drivers/net/ring/rte_eth_ring.c
> index ed0fdeb28f..12046f5a00 100644
> --- a/drivers/net/ring/rte_eth_ring.c
> +++ b/drivers/net/ring/rte_eth_ring.c
> @@ -358,7 +358,6 @@ do_eth_dev_ring_create(const char *name,
> 
>  	eth_dev->dev_ops = &ops;
>  	data->numa_node = numa_node;
> -	data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
> 
>  	/* finally assign rx and tx ops */
>  	eth_dev->rx_pkt_burst = eth_ring_rx;
> diff --git a/drivers/net/sfc/sfc_ethdev.c b/drivers/net/sfc/sfc_ethdev.c
> index eaa48627d6..1c72e40b75 100644
> --- a/drivers/net/sfc/sfc_ethdev.c
> +++ b/drivers/net/sfc/sfc_ethdev.c
> @@ -343,7 +343,7 @@ sfc_dev_close(struct rte_eth_dev *dev)
>  	}
> 
>  	/*
> -	 * Cleanup all resources in accordance with
> RTE_ETH_DEV_CLOSE_REMOVE.
> +	 * Cleanup all resources.
>  	 * Rollback primary process sfc_eth_dev_init() below.
>  	 */
> 
> @@ -2179,8 +2179,6 @@ sfc_eth_dev_init(struct rte_eth_dev *dev)
> 
>  	sfc_log_init(sa, "entry");
> 
> -	dev->data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
> -
>  	dev->data->mac_addrs = rte_zmalloc("sfc", RTE_ETHER_ADDR_LEN,
> 0);
>  	if (dev->data->mac_addrs == NULL) {
>  		rc = ENOMEM;
> diff --git a/drivers/net/softnic/rte_eth_softnic.c
> b/drivers/net/softnic/rte_eth_softnic.c
> index ad9e2aa86d..e942df78b6 100644
> --- a/drivers/net/softnic/rte_eth_softnic.c
> +++ b/drivers/net/softnic/rte_eth_softnic.c
> @@ -390,7 +390,6 @@ pmd_ethdev_register(struct rte_vdev_device *vdev,
>  	dev->device = &vdev->device;
> 
>  	/* dev->data */
> -	dev->data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
>  	dev->data->dev_private = dev_private;
>  	dev->data->dev_link.link_speed = ETH_SPEED_NUM_100G;
>  	dev->data->dev_link.link_duplex = ETH_LINK_FULL_DUPLEX; diff --
> git a/drivers/net/szedata2/rte_eth_szedata2.c
> b/drivers/net/szedata2/rte_eth_szedata2.c
> index daa40cfd6c..4325b9a30d 100644
> --- a/drivers/net/szedata2/rte_eth_szedata2.c
> +++ b/drivers/net/szedata2/rte_eth_szedata2.c
> @@ -1488,9 +1488,6 @@ rte_szedata2_eth_dev_init(struct rte_eth_dev
> *dev, struct port_info *pi)
>  	PMD_INIT_LOG(INFO, "Initializing eth_dev %s (driver %s)", data-
> >name,
>  			RTE_STR(RTE_SZEDATA2_DRIVER_NAME));
> 
> -	/* Let rte_eth_dev_close() release the port resources */
> -	dev->data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
> -
>  	/* Fill internal private structure. */
>  	internals->dev = dev;
>  	/* Get index of szedata2 device file and create path to device file */
> diff --git a/drivers/net/tap/rte_eth_tap.c b/drivers/net/tap/rte_eth_tap.c
> index 0d217f1486..b127ce62dc 100644
> --- a/drivers/net/tap/rte_eth_tap.c
> +++ b/drivers/net/tap/rte_eth_tap.c
> @@ -1922,7 +1922,7 @@ eth_dev_tap_create(struct rte_vdev_device *vdev,
> const char *tap_name,
>  	/* Setup some default values */
>  	data = dev->data;
>  	data->dev_private = pmd;
> -	data->dev_flags = RTE_ETH_DEV_INTR_LSC |
> RTE_ETH_DEV_CLOSE_REMOVE;
> +	data->dev_flags = RTE_ETH_DEV_INTR_LSC;
>  	data->numa_node = numa_node;
> 
>  	data->dev_link = pmd_link;
> diff --git a/drivers/net/vhost/rte_eth_vhost.c
> b/drivers/net/vhost/rte_eth_vhost.c
> index f67f4db812..45552ef742 100644
> --- a/drivers/net/vhost/rte_eth_vhost.c
> +++ b/drivers/net/vhost/rte_eth_vhost.c
> @@ -1443,7 +1443,7 @@ eth_dev_vhost_create(struct rte_vdev_device
> *dev, char *iface_name,
>  	internal->flags = flags;
>  	internal->disable_flags = disable_flags;
>  	data->dev_link = pmd_link;
> -	data->dev_flags = RTE_ETH_DEV_INTR_LSC |
> RTE_ETH_DEV_CLOSE_REMOVE;
> +	data->dev_flags = RTE_ETH_DEV_INTR_LSC;
>  	data->promiscuous = 1;
>  	data->all_multicast = 1;
> 
> diff --git a/drivers/net/virtio/virtio_ethdev.c
> b/drivers/net/virtio/virtio_ethdev.c
> index 0787337a01..b6ed5829bd 100644
> --- a/drivers/net/virtio/virtio_ethdev.c
> +++ b/drivers/net/virtio/virtio_ethdev.c
> @@ -1924,11 +1924,6 @@ eth_virtio_dev_init(struct rte_eth_dev *eth_dev)
>  	if (ret < 0)
>  		return ret;
>  	hw->speed = speed;
> -	/*
> -	 * Pass the information to the rte_eth_dev_close() that it should also
> -	 * release the private port resources.
> -	 */
> -	eth_dev->data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
> 
>  	/* Allocate memory for storing MAC addresses */
>  	eth_dev->data->mac_addrs = rte_zmalloc("virtio", diff --git
> a/drivers/net/vmxnet3/vmxnet3_ethdev.c
> b/drivers/net/vmxnet3/vmxnet3_ethdev.c
> index 17fed0ed6e..34a169d2c0 100644
> --- a/drivers/net/vmxnet3/vmxnet3_ethdev.c
> +++ b/drivers/net/vmxnet3/vmxnet3_ethdev.c
> @@ -321,9 +321,6 @@ eth_vmxnet3_dev_init(struct rte_eth_dev *eth_dev)
>  		     hw->perm_addr[0], hw->perm_addr[1], hw-
> >perm_addr[2],
>  		     hw->perm_addr[3], hw->perm_addr[4], hw-
> >perm_addr[5]);
> 
> -	/* Flag to call rte_eth_dev_release_port() in rte_eth_dev_close(). */
> -	eth_dev->data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
> -
>  	/* Put device in Quiesce Mode */
>  	VMXNET3_WRITE_BAR1_REG(hw, VMXNET3_REG_CMD,
> VMXNET3_CMD_QUIESCE_DEV);
> 
> diff --git a/lib/librte_ethdev/rte_ethdev.c b/lib/librte_ethdev/rte_ethdev.c
> index d12d74dd8c..d7668114ca 100644
> --- a/lib/librte_ethdev/rte_ethdev.c
> +++ b/lib/librte_ethdev/rte_ethdev.c
> @@ -1720,22 +1720,7 @@ rte_eth_dev_close(uint16_t port_id)
>  	(*dev->dev_ops->dev_close)(dev);
> 
>  	rte_ethdev_trace_close(port_id);
> -	/* check behaviour flag - temporary for PMD migration */
> -	if ((dev->data->dev_flags & RTE_ETH_DEV_CLOSE_REMOVE) != 0) {
> -		/* new behaviour: send event + reset state + free all data */
> -		rte_eth_dev_release_port(dev);
> -		return;
> -	}
> -	RTE_ETHDEV_LOG(DEBUG, "Port closing is using an old behaviour.\n"
> -			"The driver %s should migrate to the new
> behaviour.\n",
> -			dev->device->driver->name);
> -	/* old behaviour: only free queue arrays */
> -	dev->data->nb_rx_queues = 0;
> -	rte_free(dev->data->rx_queues);
> -	dev->data->rx_queues = NULL;
> -	dev->data->nb_tx_queues = 0;
> -	rte_free(dev->data->tx_queues);
> -	dev->data->tx_queues = NULL;
> +	rte_eth_dev_release_port(dev);
>  }
> 
>  int
> diff --git a/lib/librte_ethdev/rte_ethdev.h b/lib/librte_ethdev/rte_ethdev.h
> index 645a18664d..24d898ae89 100644
> --- a/lib/librte_ethdev/rte_ethdev.h
> +++ b/lib/librte_ethdev/rte_ethdev.h
> @@ -1654,11 +1654,6 @@ struct rte_eth_dev_owner {
>  	char name[RTE_ETH_MAX_OWNER_NAME_LEN]; /**< The owner
> name. */  };
> 
> -/**
> - * Port is released (i.e. totally freed and data erased) on close.
> - * Temporary flag for PMD migration to new rte_eth_dev_close() behaviour.
> - */
> -#define RTE_ETH_DEV_CLOSE_REMOVE 0x0001
>  /** Device supports link state interrupt */
>  #define RTE_ETH_DEV_INTR_LSC     0x0002
>  /** Device is a bonded slave */
> @@ -2282,8 +2277,7 @@ int rte_eth_dev_set_link_down(uint16_t port_id);
> 
>  /**
>   * Close a stopped Ethernet device. The device cannot be restarted!
> - * The function frees all port resources if the driver supports
> - * the flag RTE_ETH_DEV_CLOSE_REMOVE.
> + * The function frees all port resources.
>   *
>   * @param port_id
>   *   The port identifier of the Ethernet device.
> --
> 2.28.0

Re: [dpdk-dev] [PATCH v3 21/29] ethdev: remove old close behaviour

[Andrew Rybchenko]

On 9/29/20 2:14 AM, Thomas Monjalon wrote:
> The temporary flag RTE_ETH_DEV_CLOSE_REMOVE is removed.
> It was introduced in DPDK 18.11 in order to give time for PMDs to migrate.
> 
> The old behaviour was to free only queues when closing a port.
> The new behaviour is calling rte_eth_dev_release_port() which does
> three more tasks:
> 	- trigger event callback
> 	- reset state and few pointers
> 	- free all generic port resources
> 
> The private port resources must be released in the .dev_close callback.
> 
> The .remove callback should:
> 	- call .dev_close callback
> 	- call rte_eth_dev_release_port()
> 	- free multi-port device shared resources
> 
> Despite waiting two years, some drivers have not migrated,
> so they may hit issues with the incompatible new behaviour.
> After sending emails, adding logs, and announcing the deprecation,
> the only last solution is to declare these drivers as unmaintained:
> 	ionic, liquidio, nfp
> Below is a summary of what to implement in those drivers.
> 
> * The freeing of private port resources must be moved
> from the ".remove(device)" function to the ".dev_close(port)" function.
> 
> * If a generic resource (.mac_addrs or .hash_mac_addrs) cannot be freed,
> it must be set to NULL in ".dev_close" function to protect from
> subsequent rte_eth_dev_release_port() freeing.
> 
> * Note 1:
> The generic resources are freed in rte_eth_dev_release_port(),
> after ".dev_close" is called in rte_eth_dev_close(), but not when
> calling ".dev_close" directly from the ".remove" PMD function.
> That's why rte_eth_dev_release_port() must still be called explicitly
> from ".remove(device)" after calling the ".dev_close" PMD function.
> 
> * Note 2:
> If a device can have multiple ports, the common resources must be freed
> only in the ".remove(device)" function.
> 
> * Note 3:
> The port is supposed to be in a stopped state when it is closed.
> If it is not the case, it is free to the PMD implementation
> how to react when trying to close a non-stopped port:
> either try to stop it automatically or just return an error.
> 
> Cc: Shijith Thotton <sthotton@marvell.com>
> Cc: Srisivasubramanian Srinivasan <srinivasan@marvell.com>
> Cc: Heinrich Kuhn <heinrich.kuhn@netronome.com>
> Cc: Alfredo Cardigliano <cardigliano@ntop.org>
> 
> Signed-off-by: Thomas Monjalon <thomas@monjalon.net>
> Reviewed-by: Liron Himi <lironh@marvell.com>

Acked-by: Andrew Rybchenko <arybchenko@solarflare.com>

Re: [dpdk-dev] [PATCH v3 21/29] ethdev: remove old close behaviour

[Ferruh Yigit]

On 9/29/2020 12:14 AM, Thomas Monjalon wrote:
> The temporary flag RTE_ETH_DEV_CLOSE_REMOVE is removed.
> It was introduced in DPDK 18.11 in order to give time for PMDs to migrate.
> 
> The old behaviour was to free only queues when closing a port.
> The new behaviour is calling rte_eth_dev_release_port() which does
> three more tasks:
> 	- trigger event callback
> 	- reset state and few pointers
> 	- free all generic port resources
> 
> The private port resources must be released in the .dev_close callback.
> 
> The .remove callback should:
> 	- call .dev_close callback
> 	- call rte_eth_dev_release_port()
> 	- free multi-port device shared resources
> 
> Despite waiting two years, some drivers have not migrated,
> so they may hit issues with the incompatible new behaviour.
> After sending emails, adding logs, and announcing the deprecation,
> the only last solution is to declare these drivers as unmaintained:
> 	ionic, liquidio, nfp
> Below is a summary of what to implement in those drivers.
> 
> * The freeing of private port resources must be moved
> from the ".remove(device)" function to the ".dev_close(port)" function.
> 
> * If a generic resource (.mac_addrs or .hash_mac_addrs) cannot be freed,
> it must be set to NULL in ".dev_close" function to protect from
> subsequent rte_eth_dev_release_port() freeing.
> 
> * Note 1:
> The generic resources are freed in rte_eth_dev_release_port(),
> after ".dev_close" is called in rte_eth_dev_close(), but not when
> calling ".dev_close" directly from the ".remove" PMD function.
> That's why rte_eth_dev_release_port() must still be called explicitly
> from ".remove(device)" after calling the ".dev_close" PMD function.
> 
> * Note 2:
> If a device can have multiple ports, the common resources must be freed
> only in the ".remove(device)" function.
> 
> * Note 3:
> The port is supposed to be in a stopped state when it is closed.
> If it is not the case, it is free to the PMD implementation
> how to react when trying to close a non-stopped port:
> either try to stop it automatically or just return an error.
> 
> Cc: Shijith Thotton <sthotton@marvell.com>
> Cc: Srisivasubramanian Srinivasan <srinivasan@marvell.com>
> Cc: Heinrich Kuhn <heinrich.kuhn@netronome.com>
> Cc: Alfredo Cardigliano <cardigliano@ntop.org>
> 
> Signed-off-by: Thomas Monjalon <thomas@monjalon.net>
> Reviewed-by: Liron Himi <lironh@marvell.com>


Reviewed-by: Ferruh Yigit <ferruh.yigit@intel.com>

> 
> diff --git a/MAINTAINERS b/MAINTAINERS
> index 49a6dfa7a5..6aaf80d483 100644
> --- a/MAINTAINERS
> +++ b/MAINTAINERS
> @@ -604,7 +604,7 @@ F: drivers/net/thunderx/
>   F: doc/guides/nics/thunderx.rst
>   F: doc/guides/nics/features/thunderx.ini
>   
> -Cavium LiquidIO
> +Cavium LiquidIO - UNMAINTAINED
>   M: Shijith Thotton <sthotton@marvell.com>
>   M: Srisivasubramanian Srinivasan <srinivasan@marvell.com>
>   T: git://dpdk.org/next/dpdk-next-net-mrvl
> @@ -793,7 +793,7 @@ F: drivers/net/nfb/
>   F: doc/guides/nics/nfb.rst
>   F: doc/guides/nics/features/nfb.ini
>   
> -Netronome nfp
> +Netronome nfp - UNMAINTAINED
>   M: Heinrich Kuhn <heinrich.kuhn@netronome.com>
>   F: drivers/net/nfp/
>   F: doc/guides/nics/nfp.rst
> @@ -829,7 +829,7 @@ F: doc/guides/nics/pfe.rst
>   F: drivers/net/pfe/
>   F: doc/guides/nics/features/pfe.ini
>   
> -Pensando ionic
> +Pensando ionic - UNMAINTAINED
>   M: Alfredo Cardigliano <cardigliano@ntop.org>
>   F: drivers/net/ionic/
>   F: doc/guides/nics/ionic.rst

Just to highlight above drivers that are marked as 'UNMAINTAINED'.

[dpdk-dev] [PATCH v3 22/29] drivers/net: accept removing device without any port

[Thomas Monjalon]

The ports can be closed (i.e. completely released)
before removing the whole device.
Such case was wrongly considered an error by some drivers.

If the device supports only one port, there is nothing much
to free after the port is closed.

Signed-off-by: Thomas Monjalon <thomas@monjalon.net>
Reviewed-by: Ferruh Yigit <ferruh.yigit@intel.com>
Reviewed-by: Rosen Xu <rosen.xu@intel.com>
Reviewed-by: Sachin Saxena <sachin.saxena@oss.nxp.com>
---
 drivers/net/ipn3ke/ipn3ke_ethdev.c      |  6 ++----
 drivers/net/kni/rte_eth_kni.c           | 16 +++++++---------
 drivers/net/netvsc/hn_ethdev.c          |  2 +-
 drivers/net/nfp/nfp_net.c               |  2 ++
 drivers/net/szedata2/rte_eth_szedata2.c |  6 ++----
 5 files changed, 14 insertions(+), 18 deletions(-)

diff --git a/drivers/net/ipn3ke/ipn3ke_ethdev.c b/drivers/net/ipn3ke/ipn3ke_ethdev.c
index 027be29bd8..4446d2af9e 100644
--- a/drivers/net/ipn3ke/ipn3ke_ethdev.c
+++ b/drivers/net/ipn3ke/ipn3ke_ethdev.c
@@ -562,10 +562,8 @@ static int ipn3ke_vswitch_remove(struct rte_afu_device *afu_dev)
 			afu_dev->device.name, i);
 
 		ethdev = rte_eth_dev_allocated(afu_dev->device.name);
-		if (!ethdev)
-			return -ENODEV;
-
-		rte_eth_dev_destroy(ethdev, ipn3ke_rpst_uninit);
+		if (ethdev != NULL)
+			rte_eth_dev_destroy(ethdev, ipn3ke_rpst_uninit);
 	}
 
 	ret = rte_eth_switch_domain_free(hw->switch_domain_id);
diff --git a/drivers/net/kni/rte_eth_kni.c b/drivers/net/kni/rte_eth_kni.c
index 45ab1b17a8..2a4058f7b0 100644
--- a/drivers/net/kni/rte_eth_kni.c
+++ b/drivers/net/kni/rte_eth_kni.c
@@ -488,17 +488,15 @@ eth_kni_remove(struct rte_vdev_device *vdev)
 
 	/* find the ethdev entry */
 	eth_dev = rte_eth_dev_allocated(name);
-	if (eth_dev == NULL)
-		return -1;
-
-	if (rte_eal_process_type() != RTE_PROC_PRIMARY) {
-		eth_kni_dev_stop(eth_dev);
-		return rte_eth_dev_release_port(eth_dev);
+	if (eth_dev != NULL) {
+		if (rte_eal_process_type() != RTE_PROC_PRIMARY) {
+			eth_kni_dev_stop(eth_dev);
+			return rte_eth_dev_release_port(eth_dev);
+		}
+		eth_kni_close(eth_dev);
+		rte_eth_dev_release_port(eth_dev);
 	}
 
-	eth_kni_close(eth_dev);
-	rte_eth_dev_release_port(eth_dev);
-
 	is_kni_initialized--;
 	if (is_kni_initialized == 0)
 		rte_kni_close();
diff --git a/drivers/net/netvsc/hn_ethdev.c b/drivers/net/netvsc/hn_ethdev.c
index 15d6e9762d..19a9eb6bc2 100644
--- a/drivers/net/netvsc/hn_ethdev.c
+++ b/drivers/net/netvsc/hn_ethdev.c
@@ -1092,7 +1092,7 @@ static int eth_hn_remove(struct rte_vmbus_device *dev)
 
 	eth_dev = rte_eth_dev_allocated(dev->device.name);
 	if (!eth_dev)
-		return -ENODEV;
+		return 0; /* port already released */
 
 	ret = eth_hn_dev_uninit(eth_dev);
 	if (ret)
diff --git a/drivers/net/nfp/nfp_net.c b/drivers/net/nfp/nfp_net.c
index 9509dc8bd6..ce25cf1ed4 100644
--- a/drivers/net/nfp/nfp_net.c
+++ b/drivers/net/nfp/nfp_net.c
@@ -3739,6 +3739,8 @@ static int eth_nfp_pci_remove(struct rte_pci_device *pci_dev)
 	int port = 0;
 
 	eth_dev = rte_eth_dev_allocated(pci_dev->device.name);
+	if (eth_dev == NULL)
+		return 0; /* port already released */
 	if ((pci_dev->id.device_id == PCI_DEVICE_ID_NFP4000_PF_NIC) ||
 	    (pci_dev->id.device_id == PCI_DEVICE_ID_NFP6000_PF_NIC)) {
 		port = get_pf_port_number(eth_dev->data->name);
diff --git a/drivers/net/szedata2/rte_eth_szedata2.c b/drivers/net/szedata2/rte_eth_szedata2.c
index 4325b9a30d..5f589dfa4c 100644
--- a/drivers/net/szedata2/rte_eth_szedata2.c
+++ b/drivers/net/szedata2/rte_eth_szedata2.c
@@ -1910,10 +1910,8 @@ static int szedata2_eth_pci_remove(struct rte_pci_device *pci_dev)
 				pci_dev->device.name, i);
 		PMD_DRV_LOG(DEBUG, "Removing eth_dev %s", name);
 		eth_dev = rte_eth_dev_allocated(name);
-		if (!eth_dev) {
-			PMD_DRV_LOG(ERR, "eth_dev %s not found", name);
-			retval = retval ? retval : -ENODEV;
-		}
+		if (eth_dev == NULL)
+			continue; /* port already released */
 
 		ret = rte_szedata2_eth_dev_uninit(eth_dev);
 		if (ret != 0) {
-- 
2.28.0

[dpdk-dev] [PATCH v3 23/29] drivers/net: check process type in close operation

[Thomas Monjalon]

The secondary processes are not allowed to release shared resources.
Only process-private resources should be freed in a secondary process.
Most of the time, there is no process-private resource,
so the close operation is just forbidden in a secondary process.

After adding proper check in the port close functions,
some redundant checks in the device remove functions are dropped.

Signed-off-by: Thomas Monjalon <thomas@monjalon.net>
Reviewed-by: Rosen Xu <rosen.xu@intel.com>
Reviewed-by: Sachin Saxena <sachin.saxena@oss.nxp.com>
Reviewed-by: Ajit Khaparde <ajit.khaparde@broadcom.com>
Reviewed-by: Liron Himi <lironh@marvell.com>
---
 drivers/net/af_xdp/rte_eth_af_xdp.c     |  3 +++
 drivers/net/ark/ark_ethdev.c            |  3 +++
 drivers/net/avp/avp_ethdev.c            |  3 +++
 drivers/net/bnxt/bnxt_ethdev.c          |  3 +++
 drivers/net/bnxt/bnxt_reps.c            |  3 +++
 drivers/net/cxgbe/cxgbe_ethdev.c        |  3 +++
 drivers/net/e1000/em_ethdev.c           |  3 +++
 drivers/net/e1000/igb_ethdev.c          |  6 +++++
 drivers/net/ena/ena_ethdev.c            |  3 +++
 drivers/net/enetc/enetc_ethdev.c        |  3 +++
 drivers/net/enic/enic_ethdev.c          |  3 +++
 drivers/net/fm10k/fm10k_ethdev.c        |  9 ++-----
 drivers/net/hinic/hinic_pmd_ethdev.c    |  3 +++
 drivers/net/i40e/i40e_ethdev.c          |  2 ++
 drivers/net/i40e/i40e_ethdev_vf.c       |  3 +++
 drivers/net/iavf/iavf_ethdev.c          |  3 +++
 drivers/net/ice/ice_ethdev.c            |  3 +++
 drivers/net/igc/igc_ethdev.c            |  6 ++---
 drivers/net/ionic/ionic_ethdev.c        |  2 ++
 drivers/net/ipn3ke/ipn3ke_representor.c |  3 +++
 drivers/net/ixgbe/ixgbe_ethdev.c        |  4 ++++
 drivers/net/kni/rte_eth_kni.c           |  3 +++
 drivers/net/liquidio/lio_ethdev.c       |  3 +++
 drivers/net/mlx4/mlx4.c                 |  2 ++
 drivers/net/mvneta/mvneta_ethdev.c      |  3 +++
 drivers/net/mvpp2/mrvl_ethdev.c         |  3 +++
 drivers/net/netvsc/hn_ethdev.c          |  2 ++
 drivers/net/nfb/nfb_ethdev.c            |  3 +++
 drivers/net/nfp/nfp_net.c               |  3 +++
 drivers/net/octeontx/octeontx_ethdev.c  |  2 ++
 drivers/net/pfe/pfe_ethdev.c            |  3 +++
 drivers/net/sfc/sfc_ethdev.c            | 32 ++++++++++++-------------
 drivers/net/szedata2/rte_eth_szedata2.c |  3 +++
 drivers/net/thunderx/nicvf_ethdev.c     |  7 +++---
 drivers/net/vhost/rte_eth_vhost.c       |  7 +++---
 drivers/net/virtio/virtio_ethdev.c      |  2 ++
 drivers/net/vmxnet3/vmxnet3_ethdev.c    |  2 ++
 37 files changed, 119 insertions(+), 35 deletions(-)

diff --git a/drivers/net/af_xdp/rte_eth_af_xdp.c b/drivers/net/af_xdp/rte_eth_af_xdp.c
index 60add9ead4..b289076e01 100644
--- a/drivers/net/af_xdp/rte_eth_af_xdp.c
+++ b/drivers/net/af_xdp/rte_eth_af_xdp.c
@@ -708,6 +708,9 @@ eth_dev_close(struct rte_eth_dev *dev)
 	struct pkt_rx_queue *rxq;
 	int i;
 
+	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
+		return 0;
+
 	AF_XDP_LOG(INFO, "Closing AF_XDP ethdev on numa socket %u\n",
 		rte_socket_id());
 
diff --git a/drivers/net/ark/ark_ethdev.c b/drivers/net/ark/ark_ethdev.c
index 83dc4ecd2c..3e96445fdb 100644
--- a/drivers/net/ark/ark_ethdev.c
+++ b/drivers/net/ark/ark_ethdev.c
@@ -680,6 +680,9 @@ eth_ark_dev_close(struct rte_eth_dev *dev)
 	struct ark_adapter *ark = dev->data->dev_private;
 	uint16_t i;
 
+	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
+		return 0;
+
 	if (ark->user_ext.dev_close)
 		ark->user_ext.dev_close(dev,
 		 ark->user_data[dev->data->port_id]);
diff --git a/drivers/net/avp/avp_ethdev.c b/drivers/net/avp/avp_ethdev.c
index c730b7ab86..95fdb57451 100644
--- a/drivers/net/avp/avp_ethdev.c
+++ b/drivers/net/avp/avp_ethdev.c
@@ -2107,6 +2107,9 @@ avp_dev_close(struct rte_eth_dev *eth_dev)
 	struct avp_dev *avp = AVP_DEV_PRIVATE_TO_HW(eth_dev->data->dev_private);
 	int ret;
 
+	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
+		return 0;
+
 	rte_spinlock_lock(&avp->lock);
 	if (avp->flags & AVP_F_DETACHED) {
 		PMD_DRV_LOG(ERR, "Operation not supported during VM live migration\n");
diff --git a/drivers/net/bnxt/bnxt_ethdev.c b/drivers/net/bnxt/bnxt_ethdev.c
index a77bab661d..d45347e8ca 100644
--- a/drivers/net/bnxt/bnxt_ethdev.c
+++ b/drivers/net/bnxt/bnxt_ethdev.c
@@ -1361,6 +1361,9 @@ static int bnxt_dev_close_op(struct rte_eth_dev *eth_dev)
 {
 	struct bnxt *bp = eth_dev->data->dev_private;
 
+	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
+		return 0;
+
 	/* cancel the recovery handler before remove dev */
 	rte_eal_alarm_cancel(bnxt_dev_reset_and_resume, (void *)bp);
 	rte_eal_alarm_cancel(bnxt_dev_recover, (void *)bp);
diff --git a/drivers/net/bnxt/bnxt_reps.c b/drivers/net/bnxt/bnxt_reps.c
index df8680c113..c419fe63b8 100644
--- a/drivers/net/bnxt/bnxt_reps.c
+++ b/drivers/net/bnxt/bnxt_reps.c
@@ -220,6 +220,9 @@ int bnxt_vf_representor_uninit(struct rte_eth_dev *eth_dev)
 		(struct bnxt_vf_representor *)eth_dev->data->dev_private;
 	uint16_t vf_id;
 
+	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
+		return 0;
+
 	PMD_DRV_LOG(DEBUG, "BNXT Port:%d VFR uninit\n", eth_dev->data->port_id);
 	eth_dev->data->mac_addrs = NULL;
 	eth_dev->dev_ops = NULL;
diff --git a/drivers/net/cxgbe/cxgbe_ethdev.c b/drivers/net/cxgbe/cxgbe_ethdev.c
index e4bbba5c32..16beb2d435 100644
--- a/drivers/net/cxgbe/cxgbe_ethdev.c
+++ b/drivers/net/cxgbe/cxgbe_ethdev.c
@@ -326,6 +326,9 @@ int cxgbe_dev_close(struct rte_eth_dev *eth_dev)
 
 	CXGBE_FUNC_TRACE();
 
+	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
+		return 0;
+
 	if (!(adapter->flags & FULL_INIT_DONE))
 		return 0;
 
diff --git a/drivers/net/e1000/em_ethdev.c b/drivers/net/e1000/em_ethdev.c
index 6a6ae0e9d5..d050eb478a 100644
--- a/drivers/net/e1000/em_ethdev.c
+++ b/drivers/net/e1000/em_ethdev.c
@@ -762,6 +762,9 @@ eth_em_close(struct rte_eth_dev *dev)
 	struct rte_pci_device *pci_dev = RTE_ETH_DEV_TO_PCI(dev);
 	struct rte_intr_handle *intr_handle = &pci_dev->intr_handle;
 
+	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
+		return 0;
+
 	eth_em_stop(dev);
 	adapter->stopped = 1;
 	em_dev_free_queues(dev);
diff --git a/drivers/net/e1000/igb_ethdev.c b/drivers/net/e1000/igb_ethdev.c
index f8778207ef..cb3d97e2a3 100644
--- a/drivers/net/e1000/igb_ethdev.c
+++ b/drivers/net/e1000/igb_ethdev.c
@@ -1535,6 +1535,9 @@ eth_igb_close(struct rte_eth_dev *dev)
 	struct e1000_filter_info *filter_info =
 		E1000_DEV_PRIVATE_TO_FILTER_INFO(dev->data->dev_private);
 
+	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
+		return 0;
+
 	eth_igb_stop(dev);
 
 	e1000_phy_hw_reset(hw);
@@ -3382,6 +3385,9 @@ igbvf_dev_close(struct rte_eth_dev *dev)
 
 	PMD_INIT_FUNC_TRACE();
 
+	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
+		return 0;
+
 	e1000_reset_hw(hw);
 
 	igbvf_dev_stop(dev);
diff --git a/drivers/net/ena/ena_ethdev.c b/drivers/net/ena/ena_ethdev.c
index cf1c0b9795..07ae9bab32 100644
--- a/drivers/net/ena/ena_ethdev.c
+++ b/drivers/net/ena/ena_ethdev.c
@@ -493,6 +493,9 @@ static int ena_close(struct rte_eth_dev *dev)
 	struct rte_intr_handle *intr_handle = &pci_dev->intr_handle;
 	struct ena_adapter *adapter = dev->data->dev_private;
 
+	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
+		return 0;
+
 	if (adapter->state == ENA_ADAPTER_STATE_RUNNING)
 		ena_stop(dev);
 	adapter->state = ENA_ADAPTER_STATE_CLOSED;
diff --git a/drivers/net/enetc/enetc_ethdev.c b/drivers/net/enetc/enetc_ethdev.c
index 325c93b92a..b3dec7e64d 100644
--- a/drivers/net/enetc/enetc_ethdev.c
+++ b/drivers/net/enetc/enetc_ethdev.c
@@ -551,6 +551,9 @@ enetc_dev_close(struct rte_eth_dev *dev)
 	uint16_t i;
 
 	PMD_INIT_FUNC_TRACE();
+	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
+		return 0;
+
 	enetc_dev_stop(dev);
 
 	for (i = 0; i < dev->data->nb_rx_queues; i++) {
diff --git a/drivers/net/enic/enic_ethdev.c b/drivers/net/enic/enic_ethdev.c
index 27f60b45b8..60ee5e01de 100644
--- a/drivers/net/enic/enic_ethdev.c
+++ b/drivers/net/enic/enic_ethdev.c
@@ -451,6 +451,9 @@ static int enicpmd_dev_close(struct rte_eth_dev *eth_dev)
 	struct enic *enic = pmd_priv(eth_dev);
 
 	ENICPMD_FUNC_TRACE();
+	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
+		return 0;
+
 	enic_remove(enic);
 
 	return 0;
diff --git a/drivers/net/fm10k/fm10k_ethdev.c b/drivers/net/fm10k/fm10k_ethdev.c
index 23f4d04068..17cf0328c6 100644
--- a/drivers/net/fm10k/fm10k_ethdev.c
+++ b/drivers/net/fm10k/fm10k_ethdev.c
@@ -2784,6 +2784,8 @@ fm10k_dev_close(struct rte_eth_dev *dev)
 	struct rte_intr_handle *intr_handle = &pdev->intr_handle;
 
 	PMD_INIT_FUNC_TRACE();
+	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
+		return 0;
 
 	fm10k_mbx_lock(hw);
 	hw->mac.ops.update_lport_state(hw, hw->mac.dglort_map,
@@ -3236,14 +3238,7 @@ static int
 eth_fm10k_dev_uninit(struct rte_eth_dev *dev)
 {
 	PMD_INIT_FUNC_TRACE();
-
-	/* only uninitialize in the primary process */
-	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
-		return 0;
-
-	/* safe to close dev here */
 	fm10k_dev_close(dev);
-
 	return 0;
 }
 
diff --git a/drivers/net/hinic/hinic_pmd_ethdev.c b/drivers/net/hinic/hinic_pmd_ethdev.c
index 623534fda4..1b175aeace 100644
--- a/drivers/net/hinic/hinic_pmd_ethdev.c
+++ b/drivers/net/hinic/hinic_pmd_ethdev.c
@@ -2940,6 +2940,9 @@ static int hinic_dev_close(struct rte_eth_dev *dev)
 {
 	struct hinic_nic_dev *nic_dev = HINIC_ETH_DEV_TO_PRIVATE_NIC_DEV(dev);
 
+	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
+		return 0;
+
 	if (rte_bit_relaxed_test_and_set32(HINIC_DEV_CLOSE,
 					   &nic_dev->dev_status)) {
 		PMD_DRV_LOG(WARNING, "Device %s already closed",
diff --git a/drivers/net/i40e/i40e_ethdev.c b/drivers/net/i40e/i40e_ethdev.c
index 6fb88148b8..feeed2dec0 100644
--- a/drivers/net/i40e/i40e_ethdev.c
+++ b/drivers/net/i40e/i40e_ethdev.c
@@ -2620,6 +2620,8 @@ i40e_dev_close(struct rte_eth_dev *dev)
 	int retries = 0;
 
 	PMD_INIT_FUNC_TRACE();
+	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
+		return 0;
 
 	ret = rte_eth_switch_domain_free(pf->switch_domain_id);
 	if (ret)
diff --git a/drivers/net/i40e/i40e_ethdev_vf.c b/drivers/net/i40e/i40e_ethdev_vf.c
index 4aaf41956c..4d6510d1ff 100644
--- a/drivers/net/i40e/i40e_ethdev_vf.c
+++ b/drivers/net/i40e/i40e_ethdev_vf.c
@@ -2402,6 +2402,9 @@ i40evf_dev_close(struct rte_eth_dev *dev)
 	struct i40e_hw *hw = I40E_DEV_PRIVATE_TO_HW(dev->data->dev_private);
 	struct i40e_vf *vf = I40EVF_DEV_PRIVATE_TO_VF(dev->data->dev_private);
 
+	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
+		return 0;
+
 	i40evf_dev_stop(dev);
 	i40e_dev_free_queues(dev);
 	/*
diff --git a/drivers/net/iavf/iavf_ethdev.c b/drivers/net/iavf/iavf_ethdev.c
index 512ade2ad0..a5b1433306 100644
--- a/drivers/net/iavf/iavf_ethdev.c
+++ b/drivers/net/iavf/iavf_ethdev.c
@@ -1468,6 +1468,9 @@ iavf_dev_close(struct rte_eth_dev *dev)
 		IAVF_DEV_PRIVATE_TO_ADAPTER(dev->data->dev_private);
 	struct iavf_info *vf = IAVF_DEV_PRIVATE_TO_VF(dev->data->dev_private);
 
+	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
+		return 0;
+
 	iavf_dev_stop(dev);
 	iavf_flow_flush(dev, NULL);
 	iavf_flow_uninit(adapter);
diff --git a/drivers/net/ice/ice_ethdev.c b/drivers/net/ice/ice_ethdev.c
index 85a3ca6a2d..96159c92f2 100644
--- a/drivers/net/ice/ice_ethdev.c
+++ b/drivers/net/ice/ice_ethdev.c
@@ -2435,6 +2435,9 @@ ice_dev_close(struct rte_eth_dev *dev)
 	struct ice_adapter *ad =
 		ICE_DEV_PRIVATE_TO_ADAPTER(dev->data->dev_private);
 
+	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
+		return 0;
+
 	/* Since stop will make link down, then the link event will be
 	 * triggered, disable the irq firstly to avoid the port_infoe etc
 	 * resources deallocation causing the interrupt service thread
diff --git a/drivers/net/igc/igc_ethdev.c b/drivers/net/igc/igc_ethdev.c
index 9d27fc0d07..7f5066df4b 100644
--- a/drivers/net/igc/igc_ethdev.c
+++ b/drivers/net/igc/igc_ethdev.c
@@ -1175,6 +1175,8 @@ eth_igc_close(struct rte_eth_dev *dev)
 	int retry = 0;
 
 	PMD_INIT_FUNC_TRACE();
+	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
+		return 0;
 
 	if (!adapter->stopped)
 		eth_igc_stop(dev);
@@ -1363,10 +1365,6 @@ static int
 eth_igc_dev_uninit(__rte_unused struct rte_eth_dev *eth_dev)
 {
 	PMD_INIT_FUNC_TRACE();
-
-	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
-		return 0;
-
 	eth_igc_close(eth_dev);
 	return 0;
 }
diff --git a/drivers/net/ionic/ionic_ethdev.c b/drivers/net/ionic/ionic_ethdev.c
index 1775fd29b7..ef7d06e526 100644
--- a/drivers/net/ionic/ionic_ethdev.c
+++ b/drivers/net/ionic/ionic_ethdev.c
@@ -963,6 +963,8 @@ ionic_dev_close(struct rte_eth_dev *eth_dev)
 	int err;
 
 	IONIC_PRINT_CALL();
+	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
+		return 0;
 
 	err = ionic_lif_stop(lif);
 	if (err) {
diff --git a/drivers/net/ipn3ke/ipn3ke_representor.c b/drivers/net/ipn3ke/ipn3ke_representor.c
index d49abbf758..b9fb4d4e46 100644
--- a/drivers/net/ipn3ke/ipn3ke_representor.c
+++ b/drivers/net/ipn3ke/ipn3ke_representor.c
@@ -214,6 +214,9 @@ ipn3ke_rpst_dev_close(struct rte_eth_dev *dev)
 	struct ipn3ke_hw *hw = IPN3KE_DEV_PRIVATE_TO_HW(dev);
 	struct ipn3ke_rpst *rpst = IPN3KE_DEV_PRIVATE_TO_RPST(dev);
 
+	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
+		return 0;
+
 	if (hw->retimer.mac_type == IFPGA_RAWDEV_RETIMER_MAC_TYPE_10GE_XFI) {
 		/* Disable the TX path */
 		ipn3ke_xmac_tx_disable(hw, rpst->port_id, 0);
diff --git a/drivers/net/ixgbe/ixgbe_ethdev.c b/drivers/net/ixgbe/ixgbe_ethdev.c
index 600e9d6928..84c1b0f7a1 100644
--- a/drivers/net/ixgbe/ixgbe_ethdev.c
+++ b/drivers/net/ixgbe/ixgbe_ethdev.c
@@ -2995,6 +2995,8 @@ ixgbe_dev_close(struct rte_eth_dev *dev)
 	int ret;
 
 	PMD_INIT_FUNC_TRACE();
+	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
+		return 0;
 
 	ixgbe_pf_reset_hw(hw);
 
@@ -5442,6 +5444,8 @@ ixgbevf_dev_close(struct rte_eth_dev *dev)
 	struct rte_intr_handle *intr_handle = &pci_dev->intr_handle;
 
 	PMD_INIT_FUNC_TRACE();
+	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
+		return 0;
 
 	ixgbe_reset_hw(hw);
 
diff --git a/drivers/net/kni/rte_eth_kni.c b/drivers/net/kni/rte_eth_kni.c
index 2a4058f7b0..be747adf86 100644
--- a/drivers/net/kni/rte_eth_kni.c
+++ b/drivers/net/kni/rte_eth_kni.c
@@ -204,6 +204,9 @@ eth_kni_close(struct rte_eth_dev *eth_dev)
 	struct pmd_internals *internals;
 	int ret;
 
+	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
+		return 0;
+
 	eth_kni_dev_stop(eth_dev);
 
 	/* mac_addrs must not be freed alone because part of dev_private */
diff --git a/drivers/net/liquidio/lio_ethdev.c b/drivers/net/liquidio/lio_ethdev.c
index 93e2ed5670..1a41f27198 100644
--- a/drivers/net/liquidio/lio_ethdev.c
+++ b/drivers/net/liquidio/lio_ethdev.c
@@ -1555,6 +1555,9 @@ lio_dev_close(struct rte_eth_dev *eth_dev)
 {
 	struct lio_device *lio_dev = LIO_DEV(eth_dev);
 
+	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
+		return 0;
+
 	lio_dev_info(lio_dev, "closing port %d\n", eth_dev->data->port_id);
 
 	if (lio_dev->intf_open)
diff --git a/drivers/net/mlx4/mlx4.c b/drivers/net/mlx4/mlx4.c
index 3e57875414..cfcfb8a8fc 100644
--- a/drivers/net/mlx4/mlx4.c
+++ b/drivers/net/mlx4/mlx4.c
@@ -376,6 +376,8 @@ mlx4_dev_close(struct rte_eth_dev *dev)
 	struct mlx4_priv *priv = dev->data->dev_private;
 	unsigned int i;
 
+	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
+		return 0;
 	DEBUG("%p: closing device \"%s\"",
 	      (void *)dev,
 	      ((priv->ctx != NULL) ? priv->ctx->device->name : ""));
diff --git a/drivers/net/mvneta/mvneta_ethdev.c b/drivers/net/mvneta/mvneta_ethdev.c
index db142bec23..607771149a 100644
--- a/drivers/net/mvneta/mvneta_ethdev.c
+++ b/drivers/net/mvneta/mvneta_ethdev.c
@@ -435,6 +435,9 @@ mvneta_dev_close(struct rte_eth_dev *dev)
 	struct mvneta_priv *priv = dev->data->dev_private;
 	int i;
 
+	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
+		return 0;
+
 	if (priv->ppio)
 		mvneta_dev_stop(dev);
 
diff --git a/drivers/net/mvpp2/mrvl_ethdev.c b/drivers/net/mvpp2/mrvl_ethdev.c
index cfb97e4f8f..a230a96840 100644
--- a/drivers/net/mvpp2/mrvl_ethdev.c
+++ b/drivers/net/mvpp2/mrvl_ethdev.c
@@ -861,6 +861,9 @@ mrvl_dev_close(struct rte_eth_dev *dev)
 	struct mrvl_priv *priv = dev->data->dev_private;
 	size_t i;
 
+	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
+		return 0;
+
 	mrvl_flush_rx_queues(dev);
 	mrvl_flush_tx_shadow_queues(dev);
 	mrvl_flow_deinit(dev);
diff --git a/drivers/net/netvsc/hn_ethdev.c b/drivers/net/netvsc/hn_ethdev.c
index 19a9eb6bc2..5ae2d469c8 100644
--- a/drivers/net/netvsc/hn_ethdev.c
+++ b/drivers/net/netvsc/hn_ethdev.c
@@ -842,6 +842,8 @@ static int
 hn_dev_close(struct rte_eth_dev *dev)
 {
 	PMD_INIT_FUNC_TRACE();
+	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
+		return 0;
 
 	hn_vf_close(dev);
 	hn_dev_free_queues(dev);
diff --git a/drivers/net/nfb/nfb_ethdev.c b/drivers/net/nfb/nfb_ethdev.c
index d937ac6922..7ee7294143 100644
--- a/drivers/net/nfb/nfb_ethdev.c
+++ b/drivers/net/nfb/nfb_ethdev.c
@@ -217,6 +217,9 @@ nfb_eth_dev_close(struct rte_eth_dev *dev)
 	uint16_t nb_rx = dev->data->nb_rx_queues;
 	uint16_t nb_tx = dev->data->nb_tx_queues;
 
+	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
+		return 0;
+
 	nfb_eth_dev_stop(dev);
 
 	nfb_nc_rxmac_deinit(internals->rxmac, internals->max_rxmac);
diff --git a/drivers/net/nfp/nfp_net.c b/drivers/net/nfp/nfp_net.c
index ce25cf1ed4..c1da66e3d6 100644
--- a/drivers/net/nfp/nfp_net.c
+++ b/drivers/net/nfp/nfp_net.c
@@ -871,6 +871,9 @@ nfp_net_close(struct rte_eth_dev *dev)
 	struct rte_pci_device *pci_dev;
 	int i;
 
+	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
+		return 0;
+
 	PMD_INIT_LOG(DEBUG, "Close");
 
 	hw = NFP_NET_DEV_PRIVATE_TO_HW(dev->data->dev_private);
diff --git a/drivers/net/octeontx/octeontx_ethdev.c b/drivers/net/octeontx/octeontx_ethdev.c
index a263f45399..aa9ef3bb70 100644
--- a/drivers/net/octeontx/octeontx_ethdev.c
+++ b/drivers/net/octeontx/octeontx_ethdev.c
@@ -487,6 +487,8 @@ octeontx_dev_close(struct rte_eth_dev *dev)
 	int ret;
 
 	PMD_INIT_FUNC_TRACE();
+	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
+		return 0;
 
 	rte_event_dev_close(nic->evdev);
 
diff --git a/drivers/net/pfe/pfe_ethdev.c b/drivers/net/pfe/pfe_ethdev.c
index bb2ae0df34..f0de1c8a2e 100644
--- a/drivers/net/pfe/pfe_ethdev.c
+++ b/drivers/net/pfe/pfe_ethdev.c
@@ -396,6 +396,9 @@ pfe_eth_close(struct rte_eth_dev *dev)
 	if (!g_pfe)
 		return -1;
 
+	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
+		return 0;
+
 	pfe_eth_stop(dev);
 	/* Close the device file for link status */
 	pfe_eth_close_cdev(dev->data->dev_private);
diff --git a/drivers/net/sfc/sfc_ethdev.c b/drivers/net/sfc/sfc_ethdev.c
index 1c72e40b75..0558f1808d 100644
--- a/drivers/net/sfc/sfc_ethdev.c
+++ b/drivers/net/sfc/sfc_ethdev.c
@@ -318,6 +318,17 @@ sfc_dev_set_link_down(struct rte_eth_dev *dev)
 	return 0;
 }
 
+static void
+sfc_eth_dev_secondary_clear_ops(struct rte_eth_dev *dev)
+{
+	free(dev->process_private);
+	dev->process_private = NULL;
+	dev->dev_ops = NULL;
+	dev->tx_pkt_prepare = NULL;
+	dev->tx_pkt_burst = NULL;
+	dev->rx_pkt_burst = NULL;
+}
+
 static int
 sfc_dev_close(struct rte_eth_dev *dev)
 {
@@ -325,6 +336,11 @@ sfc_dev_close(struct rte_eth_dev *dev)
 
 	sfc_log_init(sa, "entry");
 
+	if (rte_eal_process_type() != RTE_PROC_PRIMARY) {
+		sfc_eth_dev_secondary_clear_ops(dev);
+		return 0;
+	}
+
 	sfc_adapter_lock(sa);
 	switch (sa->state) {
 	case SFC_ADAPTER_STARTED:
@@ -2101,17 +2117,6 @@ sfc_eth_dev_secondary_init(struct rte_eth_dev *dev, uint32_t logtype_main)
 	return rc;
 }
 
-static void
-sfc_eth_dev_secondary_clear_ops(struct rte_eth_dev *dev)
-{
-	free(dev->process_private);
-	dev->process_private = NULL;
-	dev->dev_ops = NULL;
-	dev->tx_pkt_prepare = NULL;
-	dev->tx_pkt_burst = NULL;
-	dev->rx_pkt_burst = NULL;
-}
-
 static void
 sfc_register_dp(void)
 {
@@ -2245,11 +2250,6 @@ sfc_eth_dev_init(struct rte_eth_dev *dev)
 static int
 sfc_eth_dev_uninit(struct rte_eth_dev *dev)
 {
-	if (rte_eal_process_type() != RTE_PROC_PRIMARY) {
-		sfc_eth_dev_secondary_clear_ops(dev);
-		return 0;
-	}
-
 	sfc_dev_close(dev);
 
 	return 0;
diff --git a/drivers/net/szedata2/rte_eth_szedata2.c b/drivers/net/szedata2/rte_eth_szedata2.c
index 5f589dfa4c..7e9fafdcf7 100644
--- a/drivers/net/szedata2/rte_eth_szedata2.c
+++ b/drivers/net/szedata2/rte_eth_szedata2.c
@@ -1163,6 +1163,9 @@ eth_dev_close(struct rte_eth_dev *dev)
 	uint16_t nb_rx = dev->data->nb_rx_queues;
 	uint16_t nb_tx = dev->data->nb_tx_queues;
 
+	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
+		return 0;
+
 	eth_dev_stop(dev);
 
 	free(internals->sze_dev_path);
diff --git a/drivers/net/thunderx/nicvf_ethdev.c b/drivers/net/thunderx/nicvf_ethdev.c
index 3d7348771a..cc6eb4ba24 100644
--- a/drivers/net/thunderx/nicvf_ethdev.c
+++ b/drivers/net/thunderx/nicvf_ethdev.c
@@ -1859,6 +1859,8 @@ nicvf_dev_close(struct rte_eth_dev *dev)
 	struct nicvf *nic = nicvf_pmd_priv(dev);
 
 	PMD_INIT_FUNC_TRACE();
+	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
+		return 0;
 
 	nicvf_dev_stop_cleanup(dev, true);
 	nicvf_periodic_alarm_stop(nicvf_interrupt, dev);
@@ -2119,10 +2121,7 @@ static int
 nicvf_eth_dev_uninit(struct rte_eth_dev *dev)
 {
 	PMD_INIT_FUNC_TRACE();
-
-	if (rte_eal_process_type() == RTE_PROC_PRIMARY)
-		nicvf_dev_close(dev);
-
+	nicvf_dev_close(dev);
 	return 0;
 }
 static int
diff --git a/drivers/net/vhost/rte_eth_vhost.c b/drivers/net/vhost/rte_eth_vhost.c
index 45552ef742..32ad27fe4e 100644
--- a/drivers/net/vhost/rte_eth_vhost.c
+++ b/drivers/net/vhost/rte_eth_vhost.c
@@ -1171,6 +1171,9 @@ eth_dev_close(struct rte_eth_dev *dev)
 	struct internal_list *list;
 	unsigned int i;
 
+	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
+		return 0;
+
 	internal = dev->data->dev_private;
 	if (!internal)
 		return 0;
@@ -1655,11 +1658,7 @@ rte_pmd_vhost_remove(struct rte_vdev_device *dev)
 	if (eth_dev == NULL)
 		return 0;
 
-	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
-		return rte_eth_dev_release_port(eth_dev);
-
 	eth_dev_close(eth_dev);
-
 	rte_eth_dev_release_port(eth_dev);
 
 	return 0;
diff --git a/drivers/net/virtio/virtio_ethdev.c b/drivers/net/virtio/virtio_ethdev.c
index b6ed5829bd..f2117675b3 100644
--- a/drivers/net/virtio/virtio_ethdev.c
+++ b/drivers/net/virtio/virtio_ethdev.c
@@ -711,6 +711,8 @@ virtio_dev_close(struct rte_eth_dev *dev)
 	struct rte_intr_conf *intr_conf = &dev->data->dev_conf.intr_conf;
 
 	PMD_INIT_LOG(DEBUG, "virtio_dev_close");
+	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
+		return 0;
 
 	if (!hw->opened)
 		return 0;
diff --git a/drivers/net/vmxnet3/vmxnet3_ethdev.c b/drivers/net/vmxnet3/vmxnet3_ethdev.c
index 34a169d2c0..fa950e1ba0 100644
--- a/drivers/net/vmxnet3/vmxnet3_ethdev.c
+++ b/drivers/net/vmxnet3/vmxnet3_ethdev.c
@@ -889,6 +889,8 @@ static int
 vmxnet3_dev_close(struct rte_eth_dev *dev)
 {
 	PMD_INIT_FUNC_TRACE();
+	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
+		return 0;
 
 	vmxnet3_dev_stop(dev);
 	vmxnet3_free_queues(dev);
-- 
2.28.0

Re: [dpdk-dev] [PATCH v3 23/29] drivers/net: check process type in close operation

[Wang, Haiyue]

> -----Original Message-----
> From: Thomas Monjalon <thomas@monjalon.net>
> Sent: Tuesday, September 29, 2020 07:15
> To: dev@dpdk.org
> Cc: Yigit, Ferruh <ferruh.yigit@intel.com>; arybchenko@solarflare.com; Xu, Rosen <rosen.xu@intel.com>;
> Sachin Saxena <sachin.saxena@oss.nxp.com>; Ajit Khaparde <ajit.khaparde@broadcom.com>; Liron Himi
> <lironh@marvell.com>; Loftus, Ciara <ciara.loftus@intel.com>; Zhang, Qi Z <qi.z.zhang@intel.com>;
> Shepard Siegel <shepard.siegel@atomicrules.com>; Ed Czeck <ed.czeck@atomicrules.com>; John Miller
> <john.miller@atomicrules.com>; Steven Webster <steven.webster@windriver.com>; Matt Peters
> <matt.peters@windriver.com>; Somnath Kotur <somnath.kotur@broadcom.com>; Rahul Lakkireddy
> <rahul.lakkireddy@chelsio.com>; Guo, Jia <jia.guo@intel.com>; Wang, Haiyue <haiyue.wang@intel.com>;
> Marcin Wojtas <mw@semihalf.com>; Michal Krawczyk <mk@semihalf.com>; Guy Tzalik <gtzalik@amazon.com>;
> Evgeny Schemeilin <evgenys@amazon.com>; Igor Chauskin <igorch@amazon.com>; Gagandeep Singh
> <g.singh@nxp.com>; John Daley <johndale@cisco.com>; Hyong Youb Kim <hyonkim@cisco.com>; Wang, Xiao W
> <xiao.w.wang@intel.com>; Ziyang Xuan <xuanziyang2@huawei.com>; Xiaoyun Wang
> <cloud.wangxiaoyun@huawei.com>; Guoyang Zhou <zhouguoyang@huawei.com>; Xing, Beilei
> <beilei.xing@intel.com>; Wu, Jingjing <jingjing.wu@intel.com>; Yang, Qiming <qiming.yang@intel.com>;
> Alfredo Cardigliano <cardigliano@ntop.org>; Shijith Thotton <sthotton@marvell.com>; Srisivasubramanian
> Srinivasan <srinivasan@marvell.com>; Matan Azrad <matan@nvidia.com>; Shahaf Shuler
> <shahafs@nvidia.com>; Zyta Szpak <zr@semihalf.com>; Stephen Hemminger <sthemmin@microsoft.com>; K. Y.
> Srinivasan <kys@microsoft.com>; Haiyang Zhang <haiyangz@microsoft.com>; Long Li <longli@microsoft.com>;
> Martin Spinler <spinler@cesnet.cz>; Heinrich Kuhn <heinrich.kuhn@netronome.com>; Harman Kalra
> <hkalra@marvell.com>; Akhil Goyal <akhil.goyal@nxp.com>; Jerin Jacob <jerinj@marvell.com>; Maciej
> Czekaj <mczekaj@marvell.com>; Maxime Coquelin <maxime.coquelin@redhat.com>; Xia, Chenbo
> <chenbo.xia@intel.com>; Wang, Zhihong <zhihong.wang@intel.com>; Yong Wang <yongwang@vmware.com>;
> Burakov, Anatoly <anatoly.burakov@intel.com>
> Subject: [PATCH v3 23/29] drivers/net: check process type in close operation
> 
> The secondary processes are not allowed to release shared resources.
> Only process-private resources should be freed in a secondary process.
> Most of the time, there is no process-private resource,
> so the close operation is just forbidden in a secondary process.
> 
> After adding proper check in the port close functions,
> some redundant checks in the device remove functions are dropped.
> 
> Signed-off-by: Thomas Monjalon <thomas@monjalon.net>
> Reviewed-by: Rosen Xu <rosen.xu@intel.com>
> Reviewed-by: Sachin Saxena <sachin.saxena@oss.nxp.com>
> Reviewed-by: Ajit Khaparde <ajit.khaparde@broadcom.com>
> Reviewed-by: Liron Himi <lironh@marvell.com>
> ---


>  drivers/net/e1000/em_ethdev.c           |  3 +++
>  drivers/net/e1000/igb_ethdev.c          |  6 +++++
>  drivers/net/igc/igc_ethdev.c            |  6 ++---
>  drivers/net/ixgbe/ixgbe_ethdev.c        |  4 ++++

For e1000/igc/ixgbe

Reviewed-by: Haiyue Wang <haiyue.wang@intel.com>

> --
> 2.28.0

Re: [dpdk-dev] [PATCH v3 23/29] drivers/net: check process type in close operation

[Guo, Jia]

For drivers/net/i40e,

Acked-by: Jeff Guo <jia.guo@intel.com>

> -----Original Message-----
> From: Thomas Monjalon <thomas@monjalon.net>
> Sent: Tuesday, September 29, 2020 7:15 AM
> To: dev@dpdk.org
> Cc: Yigit, Ferruh <ferruh.yigit@intel.com>; arybchenko@solarflare.com; Xu,
> Rosen <rosen.xu@intel.com>; Sachin Saxena <sachin.saxena@oss.nxp.com>;
> Ajit Khaparde <ajit.khaparde@broadcom.com>; Liron Himi
> <lironh@marvell.com>; Loftus, Ciara <ciara.loftus@intel.com>; Zhang, Qi Z
> <qi.z.zhang@intel.com>; Shepard Siegel <shepard.siegel@atomicrules.com>;
> Ed Czeck <ed.czeck@atomicrules.com>; John Miller
> <john.miller@atomicrules.com>; Steven Webster
> <steven.webster@windriver.com>; Matt Peters
> <matt.peters@windriver.com>; Somnath Kotur
> <somnath.kotur@broadcom.com>; Rahul Lakkireddy
> <rahul.lakkireddy@chelsio.com>; Guo, Jia <jia.guo@intel.com>; Wang,
> Haiyue <haiyue.wang@intel.com>; Marcin Wojtas <mw@semihalf.com>;
> Michal Krawczyk <mk@semihalf.com>; Guy Tzalik <gtzalik@amazon.com>;
> Evgeny Schemeilin <evgenys@amazon.com>; Igor Chauskin
> <igorch@amazon.com>; Gagandeep Singh <g.singh@nxp.com>; John Daley
> <johndale@cisco.com>; Hyong Youb Kim <hyonkim@cisco.com>; Wang, Xiao
> W <xiao.w.wang@intel.com>; Ziyang Xuan <xuanziyang2@huawei.com>;
> Xiaoyun Wang <cloud.wangxiaoyun@huawei.com>; Guoyang Zhou
> <zhouguoyang@huawei.com>; Xing, Beilei <beilei.xing@intel.com>; Wu,
> Jingjing <jingjing.wu@intel.com>; Yang, Qiming <qiming.yang@intel.com>;
> Alfredo Cardigliano <cardigliano@ntop.org>; Shijith Thotton
> <sthotton@marvell.com>; Srisivasubramanian Srinivasan
> <srinivasan@marvell.com>; Matan Azrad <matan@nvidia.com>; Shahaf
> Shuler <shahafs@nvidia.com>; Zyta Szpak <zr@semihalf.com>; Stephen
> Hemminger <sthemmin@microsoft.com>; K. Y. Srinivasan
> <kys@microsoft.com>; Haiyang Zhang <haiyangz@microsoft.com>; Long Li
> <longli@microsoft.com>; Martin Spinler <spinler@cesnet.cz>; Heinrich Kuhn
> <heinrich.kuhn@netronome.com>; Harman Kalra <hkalra@marvell.com>;
> Akhil Goyal <akhil.goyal@nxp.com>; Jerin Jacob <jerinj@marvell.com>;
> Maciej Czekaj <mczekaj@marvell.com>; Maxime Coquelin
> <maxime.coquelin@redhat.com>; Xia, Chenbo <chenbo.xia@intel.com>;
> Wang, Zhihong <zhihong.wang@intel.com>; Yong Wang
> <yongwang@vmware.com>; Burakov, Anatoly <anatoly.burakov@intel.com>
> Subject: [PATCH v3 23/29] drivers/net: check process type in close operation
> 
> The secondary processes are not allowed to release shared resources.
> Only process-private resources should be freed in a secondary process.
> Most of the time, there is no process-private resource, so the close
> operation is just forbidden in a secondary process.
> 
> After adding proper check in the port close functions, some redundant
> checks in the device remove functions are dropped.
> 
> Signed-off-by: Thomas Monjalon <thomas@monjalon.net>
> Reviewed-by: Rosen Xu <rosen.xu@intel.com>
> Reviewed-by: Sachin Saxena <sachin.saxena@oss.nxp.com>
> Reviewed-by: Ajit Khaparde <ajit.khaparde@broadcom.com>
> Reviewed-by: Liron Himi <lironh@marvell.com>
> ---
>  drivers/net/af_xdp/rte_eth_af_xdp.c     |  3 +++
>  drivers/net/ark/ark_ethdev.c            |  3 +++
>  drivers/net/avp/avp_ethdev.c            |  3 +++
>  drivers/net/bnxt/bnxt_ethdev.c          |  3 +++
>  drivers/net/bnxt/bnxt_reps.c            |  3 +++
>  drivers/net/cxgbe/cxgbe_ethdev.c        |  3 +++
>  drivers/net/e1000/em_ethdev.c           |  3 +++
>  drivers/net/e1000/igb_ethdev.c          |  6 +++++
>  drivers/net/ena/ena_ethdev.c            |  3 +++
>  drivers/net/enetc/enetc_ethdev.c        |  3 +++
>  drivers/net/enic/enic_ethdev.c          |  3 +++
>  drivers/net/fm10k/fm10k_ethdev.c        |  9 ++-----
>  drivers/net/hinic/hinic_pmd_ethdev.c    |  3 +++
>  drivers/net/i40e/i40e_ethdev.c          |  2 ++
>  drivers/net/i40e/i40e_ethdev_vf.c       |  3 +++
>  drivers/net/iavf/iavf_ethdev.c          |  3 +++
>  drivers/net/ice/ice_ethdev.c            |  3 +++
>  drivers/net/igc/igc_ethdev.c            |  6 ++---
>  drivers/net/ionic/ionic_ethdev.c        |  2 ++
>  drivers/net/ipn3ke/ipn3ke_representor.c |  3 +++
>  drivers/net/ixgbe/ixgbe_ethdev.c        |  4 ++++
>  drivers/net/kni/rte_eth_kni.c           |  3 +++
>  drivers/net/liquidio/lio_ethdev.c       |  3 +++
>  drivers/net/mlx4/mlx4.c                 |  2 ++
>  drivers/net/mvneta/mvneta_ethdev.c      |  3 +++
>  drivers/net/mvpp2/mrvl_ethdev.c         |  3 +++
>  drivers/net/netvsc/hn_ethdev.c          |  2 ++
>  drivers/net/nfb/nfb_ethdev.c            |  3 +++
>  drivers/net/nfp/nfp_net.c               |  3 +++
>  drivers/net/octeontx/octeontx_ethdev.c  |  2 ++
>  drivers/net/pfe/pfe_ethdev.c            |  3 +++
>  drivers/net/sfc/sfc_ethdev.c            | 32 ++++++++++++-------------
>  drivers/net/szedata2/rte_eth_szedata2.c |  3 +++
>  drivers/net/thunderx/nicvf_ethdev.c     |  7 +++---
>  drivers/net/vhost/rte_eth_vhost.c       |  7 +++---
>  drivers/net/virtio/virtio_ethdev.c      |  2 ++
>  drivers/net/vmxnet3/vmxnet3_ethdev.c    |  2 ++
>  37 files changed, 119 insertions(+), 35 deletions(-)
> 
> diff --git a/drivers/net/af_xdp/rte_eth_af_xdp.c
> b/drivers/net/af_xdp/rte_eth_af_xdp.c
> index 60add9ead4..b289076e01 100644
> --- a/drivers/net/af_xdp/rte_eth_af_xdp.c
> +++ b/drivers/net/af_xdp/rte_eth_af_xdp.c
> @@ -708,6 +708,9 @@ eth_dev_close(struct rte_eth_dev *dev)
>  	struct pkt_rx_queue *rxq;
>  	int i;
> 
> +	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
> +		return 0;
> +
>  	AF_XDP_LOG(INFO, "Closing AF_XDP ethdev on numa socket %u\n",
>  		rte_socket_id());
> 
> diff --git a/drivers/net/ark/ark_ethdev.c b/drivers/net/ark/ark_ethdev.c
> index 83dc4ecd2c..3e96445fdb 100644
> --- a/drivers/net/ark/ark_ethdev.c
> +++ b/drivers/net/ark/ark_ethdev.c
> @@ -680,6 +680,9 @@ eth_ark_dev_close(struct rte_eth_dev *dev)
>  	struct ark_adapter *ark = dev->data->dev_private;
>  	uint16_t i;
> 
> +	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
> +		return 0;
> +
>  	if (ark->user_ext.dev_close)
>  		ark->user_ext.dev_close(dev,
>  		 ark->user_data[dev->data->port_id]);
> diff --git a/drivers/net/avp/avp_ethdev.c b/drivers/net/avp/avp_ethdev.c
> index c730b7ab86..95fdb57451 100644
> --- a/drivers/net/avp/avp_ethdev.c
> +++ b/drivers/net/avp/avp_ethdev.c
> @@ -2107,6 +2107,9 @@ avp_dev_close(struct rte_eth_dev *eth_dev)
>  	struct avp_dev *avp = AVP_DEV_PRIVATE_TO_HW(eth_dev->data-
> >dev_private);
>  	int ret;
> 
> +	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
> +		return 0;
> +
>  	rte_spinlock_lock(&avp->lock);
>  	if (avp->flags & AVP_F_DETACHED) {
>  		PMD_DRV_LOG(ERR, "Operation not supported during VM
> live migration\n"); diff --git a/drivers/net/bnxt/bnxt_ethdev.c
> b/drivers/net/bnxt/bnxt_ethdev.c index a77bab661d..d45347e8ca 100644
> --- a/drivers/net/bnxt/bnxt_ethdev.c
> +++ b/drivers/net/bnxt/bnxt_ethdev.c
> @@ -1361,6 +1361,9 @@ static int bnxt_dev_close_op(struct rte_eth_dev
> *eth_dev)  {
>  	struct bnxt *bp = eth_dev->data->dev_private;
> 
> +	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
> +		return 0;
> +
>  	/* cancel the recovery handler before remove dev */
>  	rte_eal_alarm_cancel(bnxt_dev_reset_and_resume, (void *)bp);
>  	rte_eal_alarm_cancel(bnxt_dev_recover, (void *)bp); diff --git
> a/drivers/net/bnxt/bnxt_reps.c b/drivers/net/bnxt/bnxt_reps.c index
> df8680c113..c419fe63b8 100644
> --- a/drivers/net/bnxt/bnxt_reps.c
> +++ b/drivers/net/bnxt/bnxt_reps.c
> @@ -220,6 +220,9 @@ int bnxt_vf_representor_uninit(struct rte_eth_dev
> *eth_dev)
>  		(struct bnxt_vf_representor *)eth_dev->data->dev_private;
>  	uint16_t vf_id;
> 
> +	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
> +		return 0;
> +
>  	PMD_DRV_LOG(DEBUG, "BNXT Port:%d VFR uninit\n", eth_dev-
> >data->port_id);
>  	eth_dev->data->mac_addrs = NULL;
>  	eth_dev->dev_ops = NULL;
> diff --git a/drivers/net/cxgbe/cxgbe_ethdev.c
> b/drivers/net/cxgbe/cxgbe_ethdev.c
> index e4bbba5c32..16beb2d435 100644
> --- a/drivers/net/cxgbe/cxgbe_ethdev.c
> +++ b/drivers/net/cxgbe/cxgbe_ethdev.c
> @@ -326,6 +326,9 @@ int cxgbe_dev_close(struct rte_eth_dev *eth_dev)
> 
>  	CXGBE_FUNC_TRACE();
> 
> +	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
> +		return 0;
> +
>  	if (!(adapter->flags & FULL_INIT_DONE))
>  		return 0;
> 
> diff --git a/drivers/net/e1000/em_ethdev.c
> b/drivers/net/e1000/em_ethdev.c index 6a6ae0e9d5..d050eb478a 100644
> --- a/drivers/net/e1000/em_ethdev.c
> +++ b/drivers/net/e1000/em_ethdev.c
> @@ -762,6 +762,9 @@ eth_em_close(struct rte_eth_dev *dev)
>  	struct rte_pci_device *pci_dev = RTE_ETH_DEV_TO_PCI(dev);
>  	struct rte_intr_handle *intr_handle = &pci_dev->intr_handle;
> 
> +	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
> +		return 0;
> +
>  	eth_em_stop(dev);
>  	adapter->stopped = 1;
>  	em_dev_free_queues(dev);
> diff --git a/drivers/net/e1000/igb_ethdev.c
> b/drivers/net/e1000/igb_ethdev.c index f8778207ef..cb3d97e2a3 100644
> --- a/drivers/net/e1000/igb_ethdev.c
> +++ b/drivers/net/e1000/igb_ethdev.c
> @@ -1535,6 +1535,9 @@ eth_igb_close(struct rte_eth_dev *dev)
>  	struct e1000_filter_info *filter_info =
>  		E1000_DEV_PRIVATE_TO_FILTER_INFO(dev->data-
> >dev_private);
> 
> +	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
> +		return 0;
> +
>  	eth_igb_stop(dev);
> 
>  	e1000_phy_hw_reset(hw);
> @@ -3382,6 +3385,9 @@ igbvf_dev_close(struct rte_eth_dev *dev)
> 
>  	PMD_INIT_FUNC_TRACE();
> 
> +	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
> +		return 0;
> +
>  	e1000_reset_hw(hw);
> 
>  	igbvf_dev_stop(dev);
> diff --git a/drivers/net/ena/ena_ethdev.c b/drivers/net/ena/ena_ethdev.c
> index cf1c0b9795..07ae9bab32 100644
> --- a/drivers/net/ena/ena_ethdev.c
> +++ b/drivers/net/ena/ena_ethdev.c
> @@ -493,6 +493,9 @@ static int ena_close(struct rte_eth_dev *dev)
>  	struct rte_intr_handle *intr_handle = &pci_dev->intr_handle;
>  	struct ena_adapter *adapter = dev->data->dev_private;
> 
> +	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
> +		return 0;
> +
>  	if (adapter->state == ENA_ADAPTER_STATE_RUNNING)
>  		ena_stop(dev);
>  	adapter->state = ENA_ADAPTER_STATE_CLOSED; diff --git
> a/drivers/net/enetc/enetc_ethdev.c b/drivers/net/enetc/enetc_ethdev.c
> index 325c93b92a..b3dec7e64d 100644
> --- a/drivers/net/enetc/enetc_ethdev.c
> +++ b/drivers/net/enetc/enetc_ethdev.c
> @@ -551,6 +551,9 @@ enetc_dev_close(struct rte_eth_dev *dev)
>  	uint16_t i;
> 
>  	PMD_INIT_FUNC_TRACE();
> +	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
> +		return 0;
> +
>  	enetc_dev_stop(dev);
> 
>  	for (i = 0; i < dev->data->nb_rx_queues; i++) { diff --git
> a/drivers/net/enic/enic_ethdev.c b/drivers/net/enic/enic_ethdev.c index
> 27f60b45b8..60ee5e01de 100644
> --- a/drivers/net/enic/enic_ethdev.c
> +++ b/drivers/net/enic/enic_ethdev.c
> @@ -451,6 +451,9 @@ static int enicpmd_dev_close(struct rte_eth_dev
> *eth_dev)
>  	struct enic *enic = pmd_priv(eth_dev);
> 
>  	ENICPMD_FUNC_TRACE();
> +	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
> +		return 0;
> +
>  	enic_remove(enic);
> 
>  	return 0;
> diff --git a/drivers/net/fm10k/fm10k_ethdev.c
> b/drivers/net/fm10k/fm10k_ethdev.c
> index 23f4d04068..17cf0328c6 100644
> --- a/drivers/net/fm10k/fm10k_ethdev.c
> +++ b/drivers/net/fm10k/fm10k_ethdev.c
> @@ -2784,6 +2784,8 @@ fm10k_dev_close(struct rte_eth_dev *dev)
>  	struct rte_intr_handle *intr_handle = &pdev->intr_handle;
> 
>  	PMD_INIT_FUNC_TRACE();
> +	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
> +		return 0;
> 
>  	fm10k_mbx_lock(hw);
>  	hw->mac.ops.update_lport_state(hw, hw->mac.dglort_map, @@ -
> 3236,14 +3238,7 @@ static int  eth_fm10k_dev_uninit(struct rte_eth_dev
> *dev)  {
>  	PMD_INIT_FUNC_TRACE();
> -
> -	/* only uninitialize in the primary process */
> -	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
> -		return 0;
> -
> -	/* safe to close dev here */
>  	fm10k_dev_close(dev);
> -
>  	return 0;
>  }
> 
> diff --git a/drivers/net/hinic/hinic_pmd_ethdev.c
> b/drivers/net/hinic/hinic_pmd_ethdev.c
> index 623534fda4..1b175aeace 100644
> --- a/drivers/net/hinic/hinic_pmd_ethdev.c
> +++ b/drivers/net/hinic/hinic_pmd_ethdev.c
> @@ -2940,6 +2940,9 @@ static int hinic_dev_close(struct rte_eth_dev *dev)
> {
>  	struct hinic_nic_dev *nic_dev =
> HINIC_ETH_DEV_TO_PRIVATE_NIC_DEV(dev);
> 
> +	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
> +		return 0;
> +
>  	if (rte_bit_relaxed_test_and_set32(HINIC_DEV_CLOSE,
>  					   &nic_dev->dev_status)) {
>  		PMD_DRV_LOG(WARNING, "Device %s already closed", diff -
> -git a/drivers/net/i40e/i40e_ethdev.c b/drivers/net/i40e/i40e_ethdev.c
> index 6fb88148b8..feeed2dec0 100644
> --- a/drivers/net/i40e/i40e_ethdev.c
> +++ b/drivers/net/i40e/i40e_ethdev.c
> @@ -2620,6 +2620,8 @@ i40e_dev_close(struct rte_eth_dev *dev)
>  	int retries = 0;
> 
>  	PMD_INIT_FUNC_TRACE();
> +	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
> +		return 0;
> 
>  	ret = rte_eth_switch_domain_free(pf->switch_domain_id);
>  	if (ret)
> diff --git a/drivers/net/i40e/i40e_ethdev_vf.c
> b/drivers/net/i40e/i40e_ethdev_vf.c
> index 4aaf41956c..4d6510d1ff 100644
> --- a/drivers/net/i40e/i40e_ethdev_vf.c
> +++ b/drivers/net/i40e/i40e_ethdev_vf.c
> @@ -2402,6 +2402,9 @@ i40evf_dev_close(struct rte_eth_dev *dev)
>  	struct i40e_hw *hw = I40E_DEV_PRIVATE_TO_HW(dev->data-
> >dev_private);
>  	struct i40e_vf *vf = I40EVF_DEV_PRIVATE_TO_VF(dev->data-
> >dev_private);
> 
> +	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
> +		return 0;
> +
>  	i40evf_dev_stop(dev);
>  	i40e_dev_free_queues(dev);
>  	/*
> diff --git a/drivers/net/iavf/iavf_ethdev.c b/drivers/net/iavf/iavf_ethdev.c
> index 512ade2ad0..a5b1433306 100644
> --- a/drivers/net/iavf/iavf_ethdev.c
> +++ b/drivers/net/iavf/iavf_ethdev.c
> @@ -1468,6 +1468,9 @@ iavf_dev_close(struct rte_eth_dev *dev)
>  		IAVF_DEV_PRIVATE_TO_ADAPTER(dev->data->dev_private);
>  	struct iavf_info *vf = IAVF_DEV_PRIVATE_TO_VF(dev->data-
> >dev_private);
> 
> +	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
> +		return 0;
> +
>  	iavf_dev_stop(dev);
>  	iavf_flow_flush(dev, NULL);
>  	iavf_flow_uninit(adapter);
> diff --git a/drivers/net/ice/ice_ethdev.c b/drivers/net/ice/ice_ethdev.c
> index 85a3ca6a2d..96159c92f2 100644
> --- a/drivers/net/ice/ice_ethdev.c
> +++ b/drivers/net/ice/ice_ethdev.c
> @@ -2435,6 +2435,9 @@ ice_dev_close(struct rte_eth_dev *dev)
>  	struct ice_adapter *ad =
>  		ICE_DEV_PRIVATE_TO_ADAPTER(dev->data->dev_private);
> 
> +	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
> +		return 0;
> +
>  	/* Since stop will make link down, then the link event will be
>  	 * triggered, disable the irq firstly to avoid the port_infoe etc
>  	 * resources deallocation causing the interrupt service thread diff --
> git a/drivers/net/igc/igc_ethdev.c b/drivers/net/igc/igc_ethdev.c index
> 9d27fc0d07..7f5066df4b 100644
> --- a/drivers/net/igc/igc_ethdev.c
> +++ b/drivers/net/igc/igc_ethdev.c
> @@ -1175,6 +1175,8 @@ eth_igc_close(struct rte_eth_dev *dev)
>  	int retry = 0;
> 
>  	PMD_INIT_FUNC_TRACE();
> +	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
> +		return 0;
> 
>  	if (!adapter->stopped)
>  		eth_igc_stop(dev);
> @@ -1363,10 +1365,6 @@ static int
>  eth_igc_dev_uninit(__rte_unused struct rte_eth_dev *eth_dev)  {
>  	PMD_INIT_FUNC_TRACE();
> -
> -	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
> -		return 0;
> -
>  	eth_igc_close(eth_dev);
>  	return 0;
>  }
> diff --git a/drivers/net/ionic/ionic_ethdev.c
> b/drivers/net/ionic/ionic_ethdev.c
> index 1775fd29b7..ef7d06e526 100644
> --- a/drivers/net/ionic/ionic_ethdev.c
> +++ b/drivers/net/ionic/ionic_ethdev.c
> @@ -963,6 +963,8 @@ ionic_dev_close(struct rte_eth_dev *eth_dev)
>  	int err;
> 
>  	IONIC_PRINT_CALL();
> +	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
> +		return 0;
> 
>  	err = ionic_lif_stop(lif);
>  	if (err) {
> diff --git a/drivers/net/ipn3ke/ipn3ke_representor.c
> b/drivers/net/ipn3ke/ipn3ke_representor.c
> index d49abbf758..b9fb4d4e46 100644
> --- a/drivers/net/ipn3ke/ipn3ke_representor.c
> +++ b/drivers/net/ipn3ke/ipn3ke_representor.c
> @@ -214,6 +214,9 @@ ipn3ke_rpst_dev_close(struct rte_eth_dev *dev)
>  	struct ipn3ke_hw *hw = IPN3KE_DEV_PRIVATE_TO_HW(dev);
>  	struct ipn3ke_rpst *rpst = IPN3KE_DEV_PRIVATE_TO_RPST(dev);
> 
> +	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
> +		return 0;
> +
>  	if (hw->retimer.mac_type ==
> IFPGA_RAWDEV_RETIMER_MAC_TYPE_10GE_XFI) {
>  		/* Disable the TX path */
>  		ipn3ke_xmac_tx_disable(hw, rpst->port_id, 0); diff --git
> a/drivers/net/ixgbe/ixgbe_ethdev.c b/drivers/net/ixgbe/ixgbe_ethdev.c
> index 600e9d6928..84c1b0f7a1 100644
> --- a/drivers/net/ixgbe/ixgbe_ethdev.c
> +++ b/drivers/net/ixgbe/ixgbe_ethdev.c
> @@ -2995,6 +2995,8 @@ ixgbe_dev_close(struct rte_eth_dev *dev)
>  	int ret;
> 
>  	PMD_INIT_FUNC_TRACE();
> +	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
> +		return 0;
> 
>  	ixgbe_pf_reset_hw(hw);
> 
> @@ -5442,6 +5444,8 @@ ixgbevf_dev_close(struct rte_eth_dev *dev)
>  	struct rte_intr_handle *intr_handle = &pci_dev->intr_handle;
> 
>  	PMD_INIT_FUNC_TRACE();
> +	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
> +		return 0;
> 
>  	ixgbe_reset_hw(hw);
> 
> diff --git a/drivers/net/kni/rte_eth_kni.c b/drivers/net/kni/rte_eth_kni.c
> index 2a4058f7b0..be747adf86 100644
> --- a/drivers/net/kni/rte_eth_kni.c
> +++ b/drivers/net/kni/rte_eth_kni.c
> @@ -204,6 +204,9 @@ eth_kni_close(struct rte_eth_dev *eth_dev)
>  	struct pmd_internals *internals;
>  	int ret;
> 
> +	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
> +		return 0;
> +
>  	eth_kni_dev_stop(eth_dev);
> 
>  	/* mac_addrs must not be freed alone because part of dev_private
> */ diff --git a/drivers/net/liquidio/lio_ethdev.c
> b/drivers/net/liquidio/lio_ethdev.c
> index 93e2ed5670..1a41f27198 100644
> --- a/drivers/net/liquidio/lio_ethdev.c
> +++ b/drivers/net/liquidio/lio_ethdev.c
> @@ -1555,6 +1555,9 @@ lio_dev_close(struct rte_eth_dev *eth_dev)  {
>  	struct lio_device *lio_dev = LIO_DEV(eth_dev);
> 
> +	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
> +		return 0;
> +
>  	lio_dev_info(lio_dev, "closing port %d\n", eth_dev->data->port_id);
> 
>  	if (lio_dev->intf_open)
> diff --git a/drivers/net/mlx4/mlx4.c b/drivers/net/mlx4/mlx4.c index
> 3e57875414..cfcfb8a8fc 100644
> --- a/drivers/net/mlx4/mlx4.c
> +++ b/drivers/net/mlx4/mlx4.c
> @@ -376,6 +376,8 @@ mlx4_dev_close(struct rte_eth_dev *dev)
>  	struct mlx4_priv *priv = dev->data->dev_private;
>  	unsigned int i;
> 
> +	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
> +		return 0;
>  	DEBUG("%p: closing device \"%s\"",
>  	      (void *)dev,
>  	      ((priv->ctx != NULL) ? priv->ctx->device->name : "")); diff --git
> a/drivers/net/mvneta/mvneta_ethdev.c
> b/drivers/net/mvneta/mvneta_ethdev.c
> index db142bec23..607771149a 100644
> --- a/drivers/net/mvneta/mvneta_ethdev.c
> +++ b/drivers/net/mvneta/mvneta_ethdev.c
> @@ -435,6 +435,9 @@ mvneta_dev_close(struct rte_eth_dev *dev)
>  	struct mvneta_priv *priv = dev->data->dev_private;
>  	int i;
> 
> +	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
> +		return 0;
> +
>  	if (priv->ppio)
>  		mvneta_dev_stop(dev);
> 
> diff --git a/drivers/net/mvpp2/mrvl_ethdev.c
> b/drivers/net/mvpp2/mrvl_ethdev.c index cfb97e4f8f..a230a96840 100644
> --- a/drivers/net/mvpp2/mrvl_ethdev.c
> +++ b/drivers/net/mvpp2/mrvl_ethdev.c
> @@ -861,6 +861,9 @@ mrvl_dev_close(struct rte_eth_dev *dev)
>  	struct mrvl_priv *priv = dev->data->dev_private;
>  	size_t i;
> 
> +	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
> +		return 0;
> +
>  	mrvl_flush_rx_queues(dev);
>  	mrvl_flush_tx_shadow_queues(dev);
>  	mrvl_flow_deinit(dev);
> diff --git a/drivers/net/netvsc/hn_ethdev.c
> b/drivers/net/netvsc/hn_ethdev.c index 19a9eb6bc2..5ae2d469c8 100644
> --- a/drivers/net/netvsc/hn_ethdev.c
> +++ b/drivers/net/netvsc/hn_ethdev.c
> @@ -842,6 +842,8 @@ static int
>  hn_dev_close(struct rte_eth_dev *dev)
>  {
>  	PMD_INIT_FUNC_TRACE();
> +	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
> +		return 0;
> 
>  	hn_vf_close(dev);
>  	hn_dev_free_queues(dev);
> diff --git a/drivers/net/nfb/nfb_ethdev.c b/drivers/net/nfb/nfb_ethdev.c
> index d937ac6922..7ee7294143 100644
> --- a/drivers/net/nfb/nfb_ethdev.c
> +++ b/drivers/net/nfb/nfb_ethdev.c
> @@ -217,6 +217,9 @@ nfb_eth_dev_close(struct rte_eth_dev *dev)
>  	uint16_t nb_rx = dev->data->nb_rx_queues;
>  	uint16_t nb_tx = dev->data->nb_tx_queues;
> 
> +	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
> +		return 0;
> +
>  	nfb_eth_dev_stop(dev);
> 
>  	nfb_nc_rxmac_deinit(internals->rxmac, internals->max_rxmac); diff
> --git a/drivers/net/nfp/nfp_net.c b/drivers/net/nfp/nfp_net.c index
> ce25cf1ed4..c1da66e3d6 100644
> --- a/drivers/net/nfp/nfp_net.c
> +++ b/drivers/net/nfp/nfp_net.c
> @@ -871,6 +871,9 @@ nfp_net_close(struct rte_eth_dev *dev)
>  	struct rte_pci_device *pci_dev;
>  	int i;
> 
> +	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
> +		return 0;
> +
>  	PMD_INIT_LOG(DEBUG, "Close");
> 
>  	hw = NFP_NET_DEV_PRIVATE_TO_HW(dev->data->dev_private);
> diff --git a/drivers/net/octeontx/octeontx_ethdev.c
> b/drivers/net/octeontx/octeontx_ethdev.c
> index a263f45399..aa9ef3bb70 100644
> --- a/drivers/net/octeontx/octeontx_ethdev.c
> +++ b/drivers/net/octeontx/octeontx_ethdev.c
> @@ -487,6 +487,8 @@ octeontx_dev_close(struct rte_eth_dev *dev)
>  	int ret;
> 
>  	PMD_INIT_FUNC_TRACE();
> +	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
> +		return 0;
> 
>  	rte_event_dev_close(nic->evdev);
> 
> diff --git a/drivers/net/pfe/pfe_ethdev.c b/drivers/net/pfe/pfe_ethdev.c
> index bb2ae0df34..f0de1c8a2e 100644
> --- a/drivers/net/pfe/pfe_ethdev.c
> +++ b/drivers/net/pfe/pfe_ethdev.c
> @@ -396,6 +396,9 @@ pfe_eth_close(struct rte_eth_dev *dev)
>  	if (!g_pfe)
>  		return -1;
> 
> +	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
> +		return 0;
> +
>  	pfe_eth_stop(dev);
>  	/* Close the device file for link status */
>  	pfe_eth_close_cdev(dev->data->dev_private);
> diff --git a/drivers/net/sfc/sfc_ethdev.c b/drivers/net/sfc/sfc_ethdev.c
> index 1c72e40b75..0558f1808d 100644
> --- a/drivers/net/sfc/sfc_ethdev.c
> +++ b/drivers/net/sfc/sfc_ethdev.c
> @@ -318,6 +318,17 @@ sfc_dev_set_link_down(struct rte_eth_dev *dev)
>  	return 0;
>  }
> 
> +static void
> +sfc_eth_dev_secondary_clear_ops(struct rte_eth_dev *dev) {
> +	free(dev->process_private);
> +	dev->process_private = NULL;
> +	dev->dev_ops = NULL;
> +	dev->tx_pkt_prepare = NULL;
> +	dev->tx_pkt_burst = NULL;
> +	dev->rx_pkt_burst = NULL;
> +}
> +
>  static int
>  sfc_dev_close(struct rte_eth_dev *dev)
>  {
> @@ -325,6 +336,11 @@ sfc_dev_close(struct rte_eth_dev *dev)
> 
>  	sfc_log_init(sa, "entry");
> 
> +	if (rte_eal_process_type() != RTE_PROC_PRIMARY) {
> +		sfc_eth_dev_secondary_clear_ops(dev);
> +		return 0;
> +	}
> +
>  	sfc_adapter_lock(sa);
>  	switch (sa->state) {
>  	case SFC_ADAPTER_STARTED:
> @@ -2101,17 +2117,6 @@ sfc_eth_dev_secondary_init(struct rte_eth_dev
> *dev, uint32_t logtype_main)
>  	return rc;
>  }
> 
> -static void
> -sfc_eth_dev_secondary_clear_ops(struct rte_eth_dev *dev) -{
> -	free(dev->process_private);
> -	dev->process_private = NULL;
> -	dev->dev_ops = NULL;
> -	dev->tx_pkt_prepare = NULL;
> -	dev->tx_pkt_burst = NULL;
> -	dev->rx_pkt_burst = NULL;
> -}
> -
>  static void
>  sfc_register_dp(void)
>  {
> @@ -2245,11 +2250,6 @@ sfc_eth_dev_init(struct rte_eth_dev *dev)  static
> int  sfc_eth_dev_uninit(struct rte_eth_dev *dev)  {
> -	if (rte_eal_process_type() != RTE_PROC_PRIMARY) {
> -		sfc_eth_dev_secondary_clear_ops(dev);
> -		return 0;
> -	}
> -
>  	sfc_dev_close(dev);
> 
>  	return 0;
> diff --git a/drivers/net/szedata2/rte_eth_szedata2.c
> b/drivers/net/szedata2/rte_eth_szedata2.c
> index 5f589dfa4c..7e9fafdcf7 100644
> --- a/drivers/net/szedata2/rte_eth_szedata2.c
> +++ b/drivers/net/szedata2/rte_eth_szedata2.c
> @@ -1163,6 +1163,9 @@ eth_dev_close(struct rte_eth_dev *dev)
>  	uint16_t nb_rx = dev->data->nb_rx_queues;
>  	uint16_t nb_tx = dev->data->nb_tx_queues;
> 
> +	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
> +		return 0;
> +
>  	eth_dev_stop(dev);
> 
>  	free(internals->sze_dev_path);
> diff --git a/drivers/net/thunderx/nicvf_ethdev.c
> b/drivers/net/thunderx/nicvf_ethdev.c
> index 3d7348771a..cc6eb4ba24 100644
> --- a/drivers/net/thunderx/nicvf_ethdev.c
> +++ b/drivers/net/thunderx/nicvf_ethdev.c
> @@ -1859,6 +1859,8 @@ nicvf_dev_close(struct rte_eth_dev *dev)
>  	struct nicvf *nic = nicvf_pmd_priv(dev);
> 
>  	PMD_INIT_FUNC_TRACE();
> +	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
> +		return 0;
> 
>  	nicvf_dev_stop_cleanup(dev, true);
>  	nicvf_periodic_alarm_stop(nicvf_interrupt, dev); @@ -2119,10
> +2121,7 @@ static int  nicvf_eth_dev_uninit(struct rte_eth_dev *dev)  {
>  	PMD_INIT_FUNC_TRACE();
> -
> -	if (rte_eal_process_type() == RTE_PROC_PRIMARY)
> -		nicvf_dev_close(dev);
> -
> +	nicvf_dev_close(dev);
>  	return 0;
>  }
>  static int
> diff --git a/drivers/net/vhost/rte_eth_vhost.c
> b/drivers/net/vhost/rte_eth_vhost.c
> index 45552ef742..32ad27fe4e 100644
> --- a/drivers/net/vhost/rte_eth_vhost.c
> +++ b/drivers/net/vhost/rte_eth_vhost.c
> @@ -1171,6 +1171,9 @@ eth_dev_close(struct rte_eth_dev *dev)
>  	struct internal_list *list;
>  	unsigned int i;
> 
> +	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
> +		return 0;
> +
>  	internal = dev->data->dev_private;
>  	if (!internal)
>  		return 0;
> @@ -1655,11 +1658,7 @@ rte_pmd_vhost_remove(struct rte_vdev_device
> *dev)
>  	if (eth_dev == NULL)
>  		return 0;
> 
> -	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
> -		return rte_eth_dev_release_port(eth_dev);
> -
>  	eth_dev_close(eth_dev);
> -
>  	rte_eth_dev_release_port(eth_dev);
> 
>  	return 0;
> diff --git a/drivers/net/virtio/virtio_ethdev.c
> b/drivers/net/virtio/virtio_ethdev.c
> index b6ed5829bd..f2117675b3 100644
> --- a/drivers/net/virtio/virtio_ethdev.c
> +++ b/drivers/net/virtio/virtio_ethdev.c
> @@ -711,6 +711,8 @@ virtio_dev_close(struct rte_eth_dev *dev)
>  	struct rte_intr_conf *intr_conf = &dev->data->dev_conf.intr_conf;
> 
>  	PMD_INIT_LOG(DEBUG, "virtio_dev_close");
> +	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
> +		return 0;
> 
>  	if (!hw->opened)
>  		return 0;
> diff --git a/drivers/net/vmxnet3/vmxnet3_ethdev.c
> b/drivers/net/vmxnet3/vmxnet3_ethdev.c
> index 34a169d2c0..fa950e1ba0 100644
> --- a/drivers/net/vmxnet3/vmxnet3_ethdev.c
> +++ b/drivers/net/vmxnet3/vmxnet3_ethdev.c
> @@ -889,6 +889,8 @@ static int
>  vmxnet3_dev_close(struct rte_eth_dev *dev)  {
>  	PMD_INIT_FUNC_TRACE();
> +	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
> +		return 0;
> 
>  	vmxnet3_dev_stop(dev);
>  	vmxnet3_free_queues(dev);
> --
> 2.28.0

Re: [dpdk-dev] [PATCH v3 23/29] drivers/net: check process type in close operation

[Andrew Rybchenko]

On 9/29/20 2:14 AM, Thomas Monjalon wrote:
> The secondary processes are not allowed to release shared resources.
> Only process-private resources should be freed in a secondary process.
> Most of the time, there is no process-private resource,
> so the close operation is just forbidden in a secondary process.
> 
> After adding proper check in the port close functions,
> some redundant checks in the device remove functions are dropped.
> 
> Signed-off-by: Thomas Monjalon <thomas@monjalon.net>
> Reviewed-by: Rosen Xu <rosen.xu@intel.com>
> Reviewed-by: Sachin Saxena <sachin.saxena@oss.nxp.com>
> Reviewed-by: Ajit Khaparde <ajit.khaparde@broadcom.com>
> Reviewed-by: Liron Himi <lironh@marvell.com>

For net/sfc:
Reviewed-by: Andrew Rybchenko <arybchenko@solarflare.com>

[dpdk-dev] [PATCH v3 24/29] drivers/net: remove redundant MAC addresses freeing

[Thomas Monjalon]

The MAC addresses array is already freed by rte_eth_dev_release_port().
The redundant freeing can be removed from the PMD port closing functions.

Signed-off-by: Thomas Monjalon <thomas@monjalon.net>
Reviewed-by: Ferruh Yigit <ferruh.yigit@intel.com>
---
 drivers/net/ark/ark_ethdev.c            | 3 ---
 drivers/net/hinic/hinic_pmd_ethdev.c    | 3 ---
 drivers/net/ice/ice_ethdev.c            | 3 ---
 drivers/net/nfb/nfb_ethdev.c            | 3 ---
 drivers/net/szedata2/rte_eth_szedata2.c | 3 ---
 5 files changed, 15 deletions(-)

diff --git a/drivers/net/ark/ark_ethdev.c b/drivers/net/ark/ark_ethdev.c
index 3e96445fdb..e3b1347769 100644
--- a/drivers/net/ark/ark_ethdev.c
+++ b/drivers/net/ark/ark_ethdev.c
@@ -705,9 +705,6 @@ eth_ark_dev_close(struct rte_eth_dev *dev)
 		dev->data->rx_queues[i] = 0;
 	}
 
-	rte_free(dev->data->mac_addrs);
-	dev->data->mac_addrs = 0;
-
 	return 0;
 }
 
diff --git a/drivers/net/hinic/hinic_pmd_ethdev.c b/drivers/net/hinic/hinic_pmd_ethdev.c
index 1b175aeace..c5b06fae73 100644
--- a/drivers/net/hinic/hinic_pmd_ethdev.c
+++ b/drivers/net/hinic/hinic_pmd_ethdev.c
@@ -3228,9 +3228,6 @@ static int hinic_dev_uninit(struct rte_eth_dev *dev)
 
 	rte_free(nic_dev->mc_list);
 
-	rte_free(dev->data->mac_addrs);
-	dev->data->mac_addrs = NULL;
-
 	return HINIC_OK;
 }
 
diff --git a/drivers/net/ice/ice_ethdev.c b/drivers/net/ice/ice_ethdev.c
index 96159c92f2..73af19ce81 100644
--- a/drivers/net/ice/ice_ethdev.c
+++ b/drivers/net/ice/ice_ethdev.c
@@ -2467,9 +2467,6 @@ ice_dev_close(struct rte_eth_dev *dev)
 	dev->rx_pkt_burst = NULL;
 	dev->tx_pkt_burst = NULL;
 
-	rte_free(dev->data->mac_addrs);
-	dev->data->mac_addrs = NULL;
-
 	/* disable uio intr before callback unregister */
 	rte_intr_disable(intr_handle);
 
diff --git a/drivers/net/nfb/nfb_ethdev.c b/drivers/net/nfb/nfb_ethdev.c
index 7ee7294143..a9a8bc878d 100644
--- a/drivers/net/nfb/nfb_ethdev.c
+++ b/drivers/net/nfb/nfb_ethdev.c
@@ -236,9 +236,6 @@ nfb_eth_dev_close(struct rte_eth_dev *dev)
 	}
 	dev->data->nb_tx_queues = 0;
 
-	rte_free(dev->data->mac_addrs);
-	dev->data->mac_addrs = NULL;
-
 	return 0;
 }
 
diff --git a/drivers/net/szedata2/rte_eth_szedata2.c b/drivers/net/szedata2/rte_eth_szedata2.c
index 7e9fafdcf7..0eecec1e8c 100644
--- a/drivers/net/szedata2/rte_eth_szedata2.c
+++ b/drivers/net/szedata2/rte_eth_szedata2.c
@@ -1181,9 +1181,6 @@ eth_dev_close(struct rte_eth_dev *dev)
 	}
 	dev->data->nb_tx_queues = 0;
 
-	rte_free(dev->data->mac_addrs);
-	dev->data->mac_addrs = NULL;
-
 	return 0;
 }
 
-- 
2.28.0

[dpdk-dev] [PATCH v3 25/29] app/testpmd: reset port status on close notification

[Thomas Monjalon]

Since rte_eth_dev_release_port() is called on all port close operations,
the event RTE_ETH_EVENT_DESTROY can be reliably used for resetting
the port status on the application side.

The intermediate state RTE_PORT_HANDLING is removed in close_port()
because a port can also be closed by a PMD in a device remove operation.

In case multiple ports are closed, calling remove_invalid_ports()
only once is enough.

Signed-off-by: Thomas Monjalon <thomas@monjalon.net>
---
 app/test-pmd/testpmd.c | 31 +++++--------------------------
 1 file changed, 5 insertions(+), 26 deletions(-)

diff --git a/app/test-pmd/testpmd.c b/app/test-pmd/testpmd.c
index fe6450cc0d..1e10d2e2e4 100644
--- a/app/test-pmd/testpmd.c
+++ b/app/test-pmd/testpmd.c
@@ -2695,23 +2695,12 @@ close_port(portid_t pid)
 			continue;
 		}
 
-		if (rte_atomic16_cmpset(&(port->port_status),
-			RTE_PORT_STOPPED, RTE_PORT_HANDLING) == 0) {
-			printf("Port %d is now not stopped\n", pi);
-			continue;
-		}
-
 		if (port->flow_list)
 			port_flow_flush(pi);
 		rte_eth_dev_close(pi);
-
-		remove_invalid_ports();
-
-		if (rte_atomic16_cmpset(&(port->port_status),
-			RTE_PORT_HANDLING, RTE_PORT_CLOSED) == 0)
-			printf("Port %d cannot be set to closed\n", pi);
 	}
 
+	remove_invalid_ports();
 	printf("Done\n");
 }
 
@@ -2842,12 +2831,7 @@ detach_device(struct rte_device *dev)
 		return;
 	}
 	RTE_ETH_FOREACH_DEV_OF(sibling, dev) {
-		/* reset mapping between old ports and removed device */
-		rte_eth_devices[sibling].device = NULL;
 		if (ports[sibling].port_status != RTE_PORT_CLOSED) {
-			/* sibling ports are forced to be closed */
-			ports[sibling].port_status = RTE_PORT_CLOSED;
-			printf("Port %u is closed\n", sibling);
 		}
 	}
 
@@ -2903,11 +2887,8 @@ detach_devargs(char *identifier)
 				return;
 			}
 
-			/* sibling ports are forced to be closed */
 			if (ports[port_id].flow_list)
 				port_flow_flush(port_id);
-			ports[port_id].port_status = RTE_PORT_CLOSED;
-			printf("Port %u is now closed\n", port_id);
 		}
 	}
 
@@ -3052,12 +3033,6 @@ check_all_ports_link_status(uint32_t port_mask)
 	}
 }
 
-/*
- * This callback is for remove a port for a device. It has limitation because
- * it is not for multiple port removal for a device.
- * TODO: the device detach invoke will plan to be removed from user side to
- * eal. And convert all PMDs to free port resources on ether device closing.
- */
 static void
 rmv_port_callback(void *arg)
 {
@@ -3115,6 +3090,10 @@ eth_event_callback(portid_t port_id, enum rte_eth_event_type type, void *param,
 				rmv_port_callback, (void *)(intptr_t)port_id))
 			fprintf(stderr, "Could not set up deferred device removal\n");
 		break;
+	case RTE_ETH_EVENT_DESTROY:
+		ports[port_id].port_status = RTE_PORT_CLOSED;
+		printf("Port %u is closed\n", port_id);
+		break;
 	default:
 		break;
 	}
-- 
2.28.0

Re: [dpdk-dev] [PATCH v3 25/29] app/testpmd: reset port status on close notification

[Ferruh Yigit]

On 9/29/2020 12:14 AM, Thomas Monjalon wrote:
> Since rte_eth_dev_release_port() is called on all port close operations,
> the event RTE_ETH_EVENT_DESTROY can be reliably used for resetting
> the port status on the application side.
> 
> The intermediate state RTE_PORT_HANDLING is removed in close_port()
> because a port can also be closed by a PMD in a device remove operation.
> 
> In case multiple ports are closed, calling remove_invalid_ports()
> only once is enough.
> 
> Signed-off-by: Thomas Monjalon <thomas@monjalon.net>
>

Reviewed-by: Ferruh Yigit <ferruh.yigit@intel.com>

[dpdk-dev] [PATCH v3 26/29] app/testpmd: align behaviour of multi-port detach

[Thomas Monjalon]

A port can be closed in multiple situations:
	- close command calling close_port() -> rte_eth_dev_close()
	- exit calling close_port() -> rte_eth_dev_close()
	- hotplug calling close_port() -> rte_eth_dev_close()
	- hotplug calling detach_device() -> rte_dev_remove()
	- port detach command, detach_device() -> rte_dev_remove()
	- device detach command, detach_devargs() -> rte_eal_hotplug_remove()

The flow rules are flushed before each close.
It was already done in close_port(), detach_devargs() and
detach_port_device() which calls detach_device(),
but not in detach_device(). As a consequence, it was missing for siblings
of port detach command and unplugged device.
The check before calling port_flow_flush() is moved inside the function.

The state of the port to close is checked to be stopped.
As above, this check was missing in detach_device(),
impacting the cases of a multi-port device unplugged or detached
with the port detach command.

Signed-off-by: Thomas Monjalon <thomas@monjalon.net>
---
 app/test-pmd/config.c  |  7 +++++--
 app/test-pmd/testpmd.c | 22 +++++++++++-----------
 2 files changed, 16 insertions(+), 13 deletions(-)

diff --git a/app/test-pmd/config.c b/app/test-pmd/config.c
index 2d9a456467..4691ae1f7b 100644
--- a/app/test-pmd/config.c
+++ b/app/test-pmd/config.c
@@ -1590,9 +1590,12 @@ int
 port_flow_flush(portid_t port_id)
 {
 	struct rte_flow_error error;
-	struct rte_port *port;
+	struct rte_port *port = &ports[port_id];
 	int ret = 0;
 
+	if (port->flow_list == NULL)
+		return ret;
+
 	/* Poisoning to make sure PMDs update it in case of error. */
 	memset(&error, 0x44, sizeof(error));
 	if (rte_flow_flush(port_id, &error)) {
@@ -1601,7 +1604,7 @@ port_flow_flush(portid_t port_id)
 		    port_id == (portid_t)RTE_PORT_ALL)
 			return ret;
 	}
-	port = &ports[port_id];
+
 	while (port->flow_list) {
 		struct port_flow *pf = port->flow_list->next;
 
diff --git a/app/test-pmd/testpmd.c b/app/test-pmd/testpmd.c
index 1e10d2e2e4..ccba71c076 100644
--- a/app/test-pmd/testpmd.c
+++ b/app/test-pmd/testpmd.c
@@ -2695,8 +2695,7 @@ close_port(portid_t pid)
 			continue;
 		}
 
-		if (port->flow_list)
-			port_flow_flush(pi);
+		port_flow_flush(pi);
 		rte_eth_dev_close(pi);
 	}
 
@@ -2826,15 +2825,20 @@ detach_device(struct rte_device *dev)
 
 	printf("Removing a device...\n");
 
-	if (rte_dev_remove(dev) < 0) {
-		TESTPMD_LOG(ERR, "Failed to detach device %s\n", dev->name);
-		return;
-	}
 	RTE_ETH_FOREACH_DEV_OF(sibling, dev) {
 		if (ports[sibling].port_status != RTE_PORT_CLOSED) {
+			if (ports[sibling].port_status != RTE_PORT_STOPPED) {
+				printf("Port %u not stopped\n", sibling);
+				return;
+			}
+			port_flow_flush(sibling);
 		}
 	}
 
+	if (rte_dev_remove(dev) < 0) {
+		TESTPMD_LOG(ERR, "Failed to detach device %s\n", dev->name);
+		return;
+	}
 	remove_invalid_ports();
 
 	printf("Device is detached\n");
@@ -2855,8 +2859,6 @@ detach_port_device(portid_t port_id)
 			return;
 		}
 		printf("Port was not closed\n");
-		if (ports[port_id].flow_list)
-			port_flow_flush(port_id);
 	}
 
 	detach_device(rte_eth_devices[port_id].device);
@@ -2886,9 +2888,7 @@ detach_devargs(char *identifier)
 				rte_eth_iterator_cleanup(&iterator);
 				return;
 			}
-
-			if (ports[port_id].flow_list)
-				port_flow_flush(port_id);
+			port_flow_flush(port_id);
 		}
 	}
 
-- 
2.28.0

Re: [dpdk-dev] [PATCH v3 26/29] app/testpmd: align behaviour of multi-port detach

[Ferruh Yigit]

On 9/29/2020 12:14 AM, Thomas Monjalon wrote:
> A port can be closed in multiple situations:
> 	- close command calling close_port() -> rte_eth_dev_close()
> 	- exit calling close_port() -> rte_eth_dev_close()
> 	- hotplug calling close_port() -> rte_eth_dev_close()
> 	- hotplug calling detach_device() -> rte_dev_remove()
> 	- port detach command, detach_device() -> rte_dev_remove()
> 	- device detach command, detach_devargs() -> rte_eal_hotplug_remove()
> 
> The flow rules are flushed before each close.
> It was already done in close_port(), detach_devargs() and
> detach_port_device() which calls detach_device(),
> but not in detach_device(). As a consequence, it was missing for siblings
> of port detach command and unplugged device.
> The check before calling port_flow_flush() is moved inside the function.
> 
> The state of the port to close is checked to be stopped.
> As above, this check was missing in detach_device(),
> impacting the cases of a multi-port device unplugged or detached
> with the port detach command.
> 
> Signed-off-by: Thomas Monjalon <thomas@monjalon.net>
>

Reviewed-by: Ferruh Yigit <ferruh.yigit@intel.com>

[dpdk-dev] [PATCH v3 27/29] ethdev: remove forcing stopped state upon close

[Thomas Monjalon]

When closing a port, it is supposed to be already stopped,
and marked as such with "dev_started" state zeroed.

Resetting "dev_started" before calling the driver close operation
was hiding the case of not properly stopped port being closed.
The flag "dev_started" is not changed anymore in "rte_eth_dev_close()".

Signed-off-by: Thomas Monjalon <thomas@monjalon.net>
---
 lib/librte_ethdev/rte_ethdev.c | 1 -
 1 file changed, 1 deletion(-)

diff --git a/lib/librte_ethdev/rte_ethdev.c b/lib/librte_ethdev/rte_ethdev.c
index d7668114ca..0b8e8e3e8d 100644
--- a/lib/librte_ethdev/rte_ethdev.c
+++ b/lib/librte_ethdev/rte_ethdev.c
@@ -1716,7 +1716,6 @@ rte_eth_dev_close(uint16_t port_id)
 	dev = &rte_eth_devices[port_id];
 
 	RTE_FUNC_PTR_OR_RET(*dev->dev_ops->dev_close);
-	dev->data->dev_started = 0;
 	(*dev->dev_ops->dev_close)(dev);
 
 	rte_ethdev_trace_close(port_id);
-- 
2.28.0

Re: [dpdk-dev] [PATCH v3 27/29] ethdev: remove forcing stopped state upon close

[Andrew Rybchenko]

On 9/29/20 2:14 AM, Thomas Monjalon wrote:
> When closing a port, it is supposed to be already stopped,
> and marked as such with "dev_started" state zeroed.
> 
> Resetting "dev_started" before calling the driver close operation
> was hiding the case of not properly stopped port being closed.
> The flag "dev_started" is not changed anymore in "rte_eth_dev_close()".
> 
> Signed-off-by: Thomas Monjalon <thomas@monjalon.net>

Reviewed-by: Andrew Rybchenko <arybchenko@solarflare.com>

Re: [dpdk-dev] [PATCH v3 27/29] ethdev: remove forcing stopped state upon close

[Ferruh Yigit]

On 9/29/2020 12:14 AM, Thomas Monjalon wrote:
> When closing a port, it is supposed to be already stopped,
> and marked as such with "dev_started" state zeroed.
> 
> Resetting "dev_started" before calling the driver close operation
> was hiding the case of not properly stopped port being closed.
> The flag "dev_started" is not changed anymore in "rte_eth_dev_close()".
> 
> Signed-off-by: Thomas Monjalon <thomas@monjalon.net>
> ---
>   lib/librte_ethdev/rte_ethdev.c | 1 -
>   1 file changed, 1 deletion(-)
> 
> diff --git a/lib/librte_ethdev/rte_ethdev.c b/lib/librte_ethdev/rte_ethdev.c
> index d7668114ca..0b8e8e3e8d 100644
> --- a/lib/librte_ethdev/rte_ethdev.c
> +++ b/lib/librte_ethdev/rte_ethdev.c
> @@ -1716,7 +1716,6 @@ rte_eth_dev_close(uint16_t port_id)
>   	dev = &rte_eth_devices[port_id];
>   
>   	RTE_FUNC_PTR_OR_RET(*dev->dev_ops->dev_close);
> -	dev->data->dev_started = 0;
>   	(*dev->dev_ops->dev_close)(dev);
>   
>   	rte_ethdev_trace_close(port_id);
> 

The driver 'remove()' function may be calling the driver 'stop()' dev_ops 
internally, so the device will be stopped properly but the 'dev_started' status 
won't be updated because ethdev API is not called.

This API assumes device stopped and updates the state accordingly, it is not 
good but removing it also won't be good for the case device already stopped.

What do you think calling 'rte_eth_dev_stop()' from 'rte_eth_dev_close()'?
Although not sure how to handle driver 'remove()' case.

Re: [dpdk-dev] [PATCH v3 27/29] ethdev: remove forcing stopped state upon close

[Thomas Monjalon]

29/09/2020 18:01, Ferruh Yigit:
> On 9/29/2020 12:14 AM, Thomas Monjalon wrote:
> > When closing a port, it is supposed to be already stopped,
> > and marked as such with "dev_started" state zeroed.
> > 
> > Resetting "dev_started" before calling the driver close operation
> > was hiding the case of not properly stopped port being closed.
> > The flag "dev_started" is not changed anymore in "rte_eth_dev_close()".
> > 
> > Signed-off-by: Thomas Monjalon <thomas@monjalon.net>
> > ---
> >   lib/librte_ethdev/rte_ethdev.c | 1 -
> >   1 file changed, 1 deletion(-)
> > 
> > diff --git a/lib/librte_ethdev/rte_ethdev.c b/lib/librte_ethdev/rte_ethdev.c
> > index d7668114ca..0b8e8e3e8d 100644
> > --- a/lib/librte_ethdev/rte_ethdev.c
> > +++ b/lib/librte_ethdev/rte_ethdev.c
> > @@ -1716,7 +1716,6 @@ rte_eth_dev_close(uint16_t port_id)
> >   	dev = &rte_eth_devices[port_id];
> >   
> >   	RTE_FUNC_PTR_OR_RET(*dev->dev_ops->dev_close);
> > -	dev->data->dev_started = 0;
> >   	(*dev->dev_ops->dev_close)(dev);
> >   
> >   	rte_ethdev_trace_close(port_id);
> > 
> 
> The driver 'remove()' function may be calling the driver 'stop()' dev_ops 
> internally, so the device will be stopped properly but the 'dev_started' status 
> won't be updated because ethdev API is not called.

If the driver is managing it internally, it should reset the state as well.

> This API assumes device stopped and updates the state accordingly, it is not 
> good but removing it also won't be good for the case device already stopped.
> 
> What do you think calling 'rte_eth_dev_stop()' from 'rte_eth_dev_close()'?

I think it would be confusing.
Better to let the application and the driver manage "stop"
at their best.

> Although not sure how to handle driver 'remove()' case.

What are you referring to exactly?

Re: [dpdk-dev] [PATCH v3 27/29] ethdev: remove forcing stopped state upon close

[Ferruh Yigit]

On 9/29/2020 5:06 PM, Thomas Monjalon wrote:
> 29/09/2020 18:01, Ferruh Yigit:
>> On 9/29/2020 12:14 AM, Thomas Monjalon wrote:
>>> When closing a port, it is supposed to be already stopped,
>>> and marked as such with "dev_started" state zeroed.
>>>
>>> Resetting "dev_started" before calling the driver close operation
>>> was hiding the case of not properly stopped port being closed.
>>> The flag "dev_started" is not changed anymore in "rte_eth_dev_close()".
>>>
>>> Signed-off-by: Thomas Monjalon <thomas@monjalon.net>
>>> ---
>>>    lib/librte_ethdev/rte_ethdev.c | 1 -
>>>    1 file changed, 1 deletion(-)
>>>
>>> diff --git a/lib/librte_ethdev/rte_ethdev.c b/lib/librte_ethdev/rte_ethdev.c
>>> index d7668114ca..0b8e8e3e8d 100644
>>> --- a/lib/librte_ethdev/rte_ethdev.c
>>> +++ b/lib/librte_ethdev/rte_ethdev.c
>>> @@ -1716,7 +1716,6 @@ rte_eth_dev_close(uint16_t port_id)
>>>    	dev = &rte_eth_devices[port_id];
>>>    
>>>    	RTE_FUNC_PTR_OR_RET(*dev->dev_ops->dev_close);
>>> -	dev->data->dev_started = 0;
>>>    	(*dev->dev_ops->dev_close)(dev);
>>>    
>>>    	rte_ethdev_trace_close(port_id);
>>>
>>
>> The driver 'remove()' function may be calling the driver 'stop()' dev_ops
>> internally, so the device will be stopped properly but the 'dev_started' status
>> won't be updated because ethdev API is not called.
> 
> If the driver is managing it internally, it should reset the state as well.
> 

I think many PMD don't update the 'dev_started' right now.

>> This API assumes device stopped and updates the state accordingly, it is not
>> good but removing it also won't be good for the case device already stopped.
>>
>> What do you think calling 'rte_eth_dev_stop()' from 'rte_eth_dev_close()'?
> 
> I think it would be confusing.
> Better to let the application and the driver manage "stop"
> at their best.
> 

OK

>> Although not sure how to handle driver 'remove()' case.
> 
> What are you referring to exactly?
> 

This was questioning how to manage 'rte_eth_dev_stop()' call on driver 
'remove()' path. Not valid after above comment.

[dpdk-dev] [PATCH v3 28/29] ethdev: reset all when releasing a port

[Thomas Monjalon]

The function rte_eth_dev_release_port() was resetting partially
the struct rte_eth_dev. The drivers were completing it
with more pointers set to NULL in the close or remove operations.

A full memset is done so most of those assignments become useless.

Signed-off-by: Thomas Monjalon <thomas@monjalon.net>
---
 drivers/net/ark/ark_ethdev.c           | 3 ---
 drivers/net/atlantic/atl_ethdev.c      | 4 ----
 drivers/net/axgbe/axgbe_ethdev.c       | 3 ---
 drivers/net/bnxt/bnxt_ethdev.c         | 4 ----
 drivers/net/bnxt/bnxt_reps.c           | 1 -
 drivers/net/bonding/rte_eth_bond_pmd.c | 4 ----
 drivers/net/dpaa/dpaa_ethdev.c         | 4 ----
 drivers/net/dpaa2/dpaa2_ethdev.c       | 4 ----
 drivers/net/e1000/em_ethdev.c          | 4 ----
 drivers/net/e1000/igb_ethdev.c         | 8 --------
 drivers/net/ena/ena_ethdev.c           | 5 -----
 drivers/net/fm10k/fm10k_ethdev.c       | 4 ----
 drivers/net/hinic/hinic_pmd_ethdev.c   | 4 ----
 drivers/net/hns3/hns3_ethdev.c         | 4 ----
 drivers/net/hns3/hns3_ethdev_vf.c      | 5 -----
 drivers/net/i40e/i40e_ethdev.c         | 4 ----
 drivers/net/i40e/i40e_ethdev_vf.c      | 4 ----
 drivers/net/iavf/iavf_ethdev.c         | 4 ----
 drivers/net/ice/ice_dcf_ethdev.c       | 4 ----
 drivers/net/ice/ice_ethdev.c           | 4 ----
 drivers/net/ionic/ionic_ethdev.c       | 5 -----
 drivers/net/ixgbe/ixgbe_ethdev.c       | 8 --------
 drivers/net/liquidio/lio_ethdev.c      | 4 ----
 drivers/net/netvsc/hn_ethdev.c         | 4 ----
 drivers/net/octeontx/octeontx_ethdev.c | 3 ---
 drivers/net/qede/qede_ethdev.c         | 4 ----
 drivers/net/sfc/sfc_ethdev.c           | 7 +------
 drivers/net/tap/rte_eth_tap.c          | 1 -
 drivers/net/virtio/virtio_ethdev.c     | 4 ----
 drivers/net/vmxnet3/vmxnet3_ethdev.c   | 5 -----
 lib/librte_ethdev/rte_ethdev.c         | 5 +----
 31 files changed, 2 insertions(+), 129 deletions(-)

diff --git a/drivers/net/ark/ark_ethdev.c b/drivers/net/ark/ark_ethdev.c
index e3b1347769..cfcfe5c42f 100644
--- a/drivers/net/ark/ark_ethdev.c
+++ b/drivers/net/ark/ark_ethdev.c
@@ -504,9 +504,6 @@ eth_ark_dev_uninit(struct rte_eth_dev *dev)
 	ark_pktgen_uninit(ark->pg);
 	ark_pktchkr_uninit(ark->pc);
 
-	dev->dev_ops = NULL;
-	dev->rx_pkt_burst = NULL;
-	dev->tx_pkt_burst = NULL;
 	return 0;
 }
 
diff --git a/drivers/net/atlantic/atl_ethdev.c b/drivers/net/atlantic/atl_ethdev.c
index 2217511ca0..5cc41c2b61 100644
--- a/drivers/net/atlantic/atl_ethdev.c
+++ b/drivers/net/atlantic/atl_ethdev.c
@@ -701,10 +701,6 @@ atl_dev_close(struct rte_eth_dev *dev)
 
 	atl_free_queues(dev);
 
-	dev->dev_ops = NULL;
-	dev->rx_pkt_burst = NULL;
-	dev->tx_pkt_burst = NULL;
-
 	/* disable uio intr before callback unregister */
 	rte_intr_disable(intr_handle);
 	rte_intr_callback_unregister(intr_handle,
diff --git a/drivers/net/axgbe/axgbe_ethdev.c b/drivers/net/axgbe/axgbe_ethdev.c
index cf085487cc..39087f36f0 100644
--- a/drivers/net/axgbe/axgbe_ethdev.c
+++ b/drivers/net/axgbe/axgbe_ethdev.c
@@ -2134,9 +2134,6 @@ axgbe_dev_close(struct rte_eth_dev *eth_dev)
 		return 0;
 
 	pci_dev = RTE_DEV_TO_PCI(eth_dev->device);
-	eth_dev->dev_ops = NULL;
-	eth_dev->rx_pkt_burst = NULL;
-	eth_dev->tx_pkt_burst = NULL;
 	axgbe_dev_clear_queues(eth_dev);
 
 	/* disable uio intr before callback unregister */
diff --git a/drivers/net/bnxt/bnxt_ethdev.c b/drivers/net/bnxt/bnxt_ethdev.c
index d45347e8ca..e1c430b2cc 100644
--- a/drivers/net/bnxt/bnxt_ethdev.c
+++ b/drivers/net/bnxt/bnxt_ethdev.c
@@ -1382,10 +1382,6 @@ static int bnxt_dev_close_op(struct rte_eth_dev *eth_dev)
 	bnxt_free_pf_info(bp);
 	bnxt_free_parent_info(bp);
 
-	eth_dev->dev_ops = NULL;
-	eth_dev->rx_pkt_burst = NULL;
-	eth_dev->tx_pkt_burst = NULL;
-
 	rte_memzone_free((const struct rte_memzone *)bp->tx_mem_zone);
 	bp->tx_mem_zone = NULL;
 	rte_memzone_free((const struct rte_memzone *)bp->rx_mem_zone);
diff --git a/drivers/net/bnxt/bnxt_reps.c b/drivers/net/bnxt/bnxt_reps.c
index c419fe63b8..d523c867eb 100644
--- a/drivers/net/bnxt/bnxt_reps.c
+++ b/drivers/net/bnxt/bnxt_reps.c
@@ -225,7 +225,6 @@ int bnxt_vf_representor_uninit(struct rte_eth_dev *eth_dev)
 
 	PMD_DRV_LOG(DEBUG, "BNXT Port:%d VFR uninit\n", eth_dev->data->port_id);
 	eth_dev->data->mac_addrs = NULL;
-	eth_dev->dev_ops = NULL;
 
 	parent_bp = rep->parent_dev->data->dev_private;
 	if (!parent_bp) {
diff --git a/drivers/net/bonding/rte_eth_bond_pmd.c b/drivers/net/bonding/rte_eth_bond_pmd.c
index 1f761c7c9e..ce8fe254af 100644
--- a/drivers/net/bonding/rte_eth_bond_pmd.c
+++ b/drivers/net/bonding/rte_eth_bond_pmd.c
@@ -2130,10 +2130,6 @@ bond_ethdev_close(struct rte_eth_dev *dev)
 	 */
 	rte_mempool_free(internals->mode6.mempool);
 
-	dev->dev_ops = NULL;
-	dev->rx_pkt_burst = NULL;
-	dev->tx_pkt_burst = NULL;
-
 	return 0;
 }
 
diff --git a/drivers/net/dpaa/dpaa_ethdev.c b/drivers/net/dpaa/dpaa_ethdev.c
index a01c8f3fc6..7e6a954d7b 100644
--- a/drivers/net/dpaa/dpaa_ethdev.c
+++ b/drivers/net/dpaa/dpaa_ethdev.c
@@ -441,10 +441,6 @@ static int dpaa_eth_dev_close(struct rte_eth_dev *dev)
 	rte_free(dpaa_intf->tx_queues);
 	dpaa_intf->tx_queues = NULL;
 
-	dev->dev_ops = NULL;
-	dev->rx_pkt_burst = NULL;
-	dev->tx_pkt_burst = NULL;
-
 	return 0;
 }
 
diff --git a/drivers/net/dpaa2/dpaa2_ethdev.c b/drivers/net/dpaa2/dpaa2_ethdev.c
index d8624514df..ba99a524f9 100644
--- a/drivers/net/dpaa2/dpaa2_ethdev.c
+++ b/drivers/net/dpaa2/dpaa2_ethdev.c
@@ -1284,10 +1284,6 @@ dpaa2_dev_close(struct rte_eth_dev *dev)
 	if (priv->extract.qos_extract_param)
 		rte_free((void *)(size_t)priv->extract.qos_extract_param);
 
-	dev->dev_ops = NULL;
-	dev->rx_pkt_burst = NULL;
-	dev->tx_pkt_burst = NULL;
-
 	DPAA2_PMD_INFO("%s: netdev deleted", dev->data->name);
 	return 0;
 }
diff --git a/drivers/net/e1000/em_ethdev.c b/drivers/net/e1000/em_ethdev.c
index d050eb478a..151b7543d8 100644
--- a/drivers/net/e1000/em_ethdev.c
+++ b/drivers/net/e1000/em_ethdev.c
@@ -772,10 +772,6 @@ eth_em_close(struct rte_eth_dev *dev)
 	em_release_manageability(hw);
 	em_hw_control_release(hw);
 
-	dev->dev_ops = NULL;
-	dev->rx_pkt_burst = NULL;
-	dev->tx_pkt_burst = NULL;
-
 	/* disable uio intr before callback unregister */
 	rte_intr_disable(intr_handle);
 	rte_intr_callback_unregister(intr_handle,
diff --git a/drivers/net/e1000/igb_ethdev.c b/drivers/net/e1000/igb_ethdev.c
index cb3d97e2a3..a7fcbaf295 100644
--- a/drivers/net/e1000/igb_ethdev.c
+++ b/drivers/net/e1000/igb_ethdev.c
@@ -1564,10 +1564,6 @@ eth_igb_close(struct rte_eth_dev *dev)
 	memset(&link, 0, sizeof(link));
 	rte_eth_linkstatus_set(dev, &link);
 
-	dev->dev_ops = NULL;
-	dev->rx_pkt_burst = NULL;
-	dev->tx_pkt_burst = NULL;
-
 	/* Reset any pending lock */
 	igb_reset_swfw_lock(hw);
 
@@ -3403,10 +3399,6 @@ igbvf_dev_close(struct rte_eth_dev *dev)
 	memset(&addr, 0, sizeof(addr));
 	igbvf_default_mac_addr_set(dev, &addr);
 
-	dev->dev_ops = NULL;
-	dev->rx_pkt_burst = NULL;
-	dev->tx_pkt_burst = NULL;
-
 	rte_intr_callback_unregister(&pci_dev->intr_handle,
 				     eth_igbvf_interrupt_handler,
 				     (void *)dev);
diff --git a/drivers/net/ena/ena_ethdev.c b/drivers/net/ena/ena_ethdev.c
index 07ae9bab32..b42f12b870 100644
--- a/drivers/net/ena/ena_ethdev.c
+++ b/drivers/net/ena/ena_ethdev.c
@@ -1933,11 +1933,6 @@ static int eth_ena_dev_uninit(struct rte_eth_dev *eth_dev)
 
 	ena_destroy_device(eth_dev);
 
-	eth_dev->dev_ops = NULL;
-	eth_dev->rx_pkt_burst = NULL;
-	eth_dev->tx_pkt_burst = NULL;
-	eth_dev->tx_pkt_prepare = NULL;
-
 	return 0;
 }
 
diff --git a/drivers/net/fm10k/fm10k_ethdev.c b/drivers/net/fm10k/fm10k_ethdev.c
index 17cf0328c6..2d51f50faa 100644
--- a/drivers/net/fm10k/fm10k_ethdev.c
+++ b/drivers/net/fm10k/fm10k_ethdev.c
@@ -2801,10 +2801,6 @@ fm10k_dev_close(struct rte_eth_dev *dev)
 	fm10k_dev_queue_release(dev);
 	fm10k_stop_hw(hw);
 
-	dev->dev_ops = NULL;
-	dev->rx_pkt_burst = NULL;
-	dev->tx_pkt_burst = NULL;
-
 	/* disable uio/vfio intr */
 	rte_intr_disable(intr_handle);
 
diff --git a/drivers/net/hinic/hinic_pmd_ethdev.c b/drivers/net/hinic/hinic_pmd_ethdev.c
index c5b06fae73..57084e59c1 100644
--- a/drivers/net/hinic/hinic_pmd_ethdev.c
+++ b/drivers/net/hinic/hinic_pmd_ethdev.c
@@ -3222,10 +3222,6 @@ static int hinic_dev_uninit(struct rte_eth_dev *dev)
 
 	hinic_dev_close(dev);
 
-	dev->dev_ops = NULL;
-	dev->rx_pkt_burst = NULL;
-	dev->tx_pkt_burst = NULL;
-
 	rte_free(nic_dev->mc_list);
 
 	return HINIC_OK;
diff --git a/drivers/net/hns3/hns3_ethdev.c b/drivers/net/hns3/hns3_ethdev.c
index dd03bd62b2..ba40d489d6 100644
--- a/drivers/net/hns3/hns3_ethdev.c
+++ b/drivers/net/hns3/hns3_ethdev.c
@@ -5721,10 +5721,6 @@ hns3_dev_uninit(struct rte_eth_dev *eth_dev)
 	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
 		return -EPERM;
 
-	eth_dev->dev_ops = NULL;
-	eth_dev->rx_pkt_burst = NULL;
-	eth_dev->tx_pkt_burst = NULL;
-	eth_dev->tx_pkt_prepare = NULL;
 	if (hw->adapter_state < HNS3_NIC_CLOSING)
 		hns3_dev_close(eth_dev);
 
diff --git a/drivers/net/hns3/hns3_ethdev_vf.c b/drivers/net/hns3/hns3_ethdev_vf.c
index 436d864b5f..6a04cd9c2c 100644
--- a/drivers/net/hns3/hns3_ethdev_vf.c
+++ b/drivers/net/hns3/hns3_ethdev_vf.c
@@ -2728,11 +2728,6 @@ hns3vf_dev_uninit(struct rte_eth_dev *eth_dev)
 	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
 		return -EPERM;
 
-	eth_dev->dev_ops = NULL;
-	eth_dev->rx_pkt_burst = NULL;
-	eth_dev->tx_pkt_burst = NULL;
-	eth_dev->tx_pkt_prepare = NULL;
-
 	if (hw->adapter_state < HNS3_NIC_CLOSING)
 		hns3vf_dev_close(eth_dev);
 
diff --git a/drivers/net/i40e/i40e_ethdev.c b/drivers/net/i40e/i40e_ethdev.c
index feeed2dec0..a05d29dcfc 100644
--- a/drivers/net/i40e/i40e_ethdev.c
+++ b/drivers/net/i40e/i40e_ethdev.c
@@ -2692,10 +2692,6 @@ i40e_dev_close(struct rte_eth_dev *dev)
 			(reg | I40E_PFGEN_CTRL_PFSWR_MASK));
 	I40E_WRITE_FLUSH(hw);
 
-	dev->dev_ops = NULL;
-	dev->rx_pkt_burst = NULL;
-	dev->tx_pkt_burst = NULL;
-
 	/* Clear PXE mode */
 	i40e_clear_pxe_mode(hw);
 
diff --git a/drivers/net/i40e/i40e_ethdev_vf.c b/drivers/net/i40e/i40e_ethdev_vf.c
index 4d6510d1ff..2001854ae6 100644
--- a/drivers/net/i40e/i40e_ethdev_vf.c
+++ b/drivers/net/i40e/i40e_ethdev_vf.c
@@ -2421,10 +2421,6 @@ i40evf_dev_close(struct rte_eth_dev *dev)
 	i40e_shutdown_adminq(hw);
 	i40evf_disable_irq0(hw);
 
-	dev->dev_ops = NULL;
-	dev->rx_pkt_burst = NULL;
-	dev->tx_pkt_burst = NULL;
-
 	rte_free(vf->vf_res);
 	vf->vf_res = NULL;
 	rte_free(vf->aq_resp);
diff --git a/drivers/net/iavf/iavf_ethdev.c b/drivers/net/iavf/iavf_ethdev.c
index a5b1433306..02391acfd1 100644
--- a/drivers/net/iavf/iavf_ethdev.c
+++ b/drivers/net/iavf/iavf_ethdev.c
@@ -1483,10 +1483,6 @@ iavf_dev_close(struct rte_eth_dev *dev)
 				     iavf_dev_interrupt_handler, dev);
 	iavf_disable_irq0(hw);
 
-	dev->dev_ops = NULL;
-	dev->rx_pkt_burst = NULL;
-	dev->tx_pkt_burst = NULL;
-
 	if (vf->vf_res->vf_cap_flags & VIRTCHNL_VF_OFFLOAD_RSS_PF) {
 		if (vf->rss_lut) {
 			rte_free(vf->rss_lut);
diff --git a/drivers/net/ice/ice_dcf_ethdev.c b/drivers/net/ice/ice_dcf_ethdev.c
index 33dd0c44f2..f2056e2aba 100644
--- a/drivers/net/ice/ice_dcf_ethdev.c
+++ b/drivers/net/ice/ice_dcf_ethdev.c
@@ -855,10 +855,6 @@ ice_dcf_dev_close(struct rte_eth_dev *dev)
 	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
 		return 0;
 
-	dev->dev_ops = NULL;
-	dev->rx_pkt_burst = NULL;
-	dev->tx_pkt_burst = NULL;
-
 	ice_dcf_uninit_parent_adapter(dev);
 	ice_dcf_uninit_hw(dev, &adapter->real_hw);
 
diff --git a/drivers/net/ice/ice_ethdev.c b/drivers/net/ice/ice_ethdev.c
index 73af19ce81..4f977bd4c4 100644
--- a/drivers/net/ice/ice_ethdev.c
+++ b/drivers/net/ice/ice_ethdev.c
@@ -2463,10 +2463,6 @@ ice_dev_close(struct rte_eth_dev *dev)
 	rte_free(pf->proto_xtr);
 	pf->proto_xtr = NULL;
 
-	dev->dev_ops = NULL;
-	dev->rx_pkt_burst = NULL;
-	dev->tx_pkt_burst = NULL;
-
 	/* disable uio intr before callback unregister */
 	rte_intr_disable(intr_handle);
 
diff --git a/drivers/net/ionic/ionic_ethdev.c b/drivers/net/ionic/ionic_ethdev.c
index ef7d06e526..1387253dc8 100644
--- a/drivers/net/ionic/ionic_ethdev.c
+++ b/drivers/net/ionic/ionic_ethdev.c
@@ -1065,11 +1065,6 @@ eth_ionic_dev_uninit(struct rte_eth_dev *eth_dev)
 	ionic_lif_deinit(lif);
 	ionic_lif_free(lif);
 
-	eth_dev->dev_ops = NULL;
-	eth_dev->rx_pkt_burst = NULL;
-	eth_dev->tx_pkt_burst = NULL;
-	eth_dev->tx_pkt_prepare = NULL;
-
 	return 0;
 }
 
diff --git a/drivers/net/ixgbe/ixgbe_ethdev.c b/drivers/net/ixgbe/ixgbe_ethdev.c
index 84c1b0f7a1..be8a79009d 100644
--- a/drivers/net/ixgbe/ixgbe_ethdev.c
+++ b/drivers/net/ixgbe/ixgbe_ethdev.c
@@ -3009,10 +3009,6 @@ ixgbe_dev_close(struct rte_eth_dev *dev)
 	/* reprogram the RAR[0] in case user changed it. */
 	ixgbe_set_rar(hw, 0, hw->mac.addr, 0, IXGBE_RAH_AV);
 
-	dev->dev_ops = NULL;
-	dev->rx_pkt_burst = NULL;
-	dev->tx_pkt_burst = NULL;
-
 	/* Unlock any pending hardware semaphore */
 	ixgbe_swfw_lock_reset(hw);
 
@@ -5460,10 +5456,6 @@ ixgbevf_dev_close(struct rte_eth_dev *dev)
 	 **/
 	ixgbevf_remove_mac_addr(dev, 0);
 
-	dev->dev_ops = NULL;
-	dev->rx_pkt_burst = NULL;
-	dev->tx_pkt_burst = NULL;
-
 	rte_intr_disable(intr_handle);
 	rte_intr_callback_unregister(intr_handle,
 				     ixgbevf_dev_interrupt_handler, dev);
diff --git a/drivers/net/liquidio/lio_ethdev.c b/drivers/net/liquidio/lio_ethdev.c
index 1a41f27198..9c4da467eb 100644
--- a/drivers/net/liquidio/lio_ethdev.c
+++ b/drivers/net/liquidio/lio_ethdev.c
@@ -2067,10 +2067,6 @@ lio_eth_dev_uninit(struct rte_eth_dev *eth_dev)
 	/* lio_free_sc_buffer_pool */
 	lio_free_sc_buffer_pool(lio_dev);
 
-	eth_dev->dev_ops = NULL;
-	eth_dev->rx_pkt_burst = NULL;
-	eth_dev->tx_pkt_burst = NULL;
-
 	return 0;
 }
 
diff --git a/drivers/net/netvsc/hn_ethdev.c b/drivers/net/netvsc/hn_ethdev.c
index 5ae2d469c8..9af64821a1 100644
--- a/drivers/net/netvsc/hn_ethdev.c
+++ b/drivers/net/netvsc/hn_ethdev.c
@@ -1049,10 +1049,6 @@ eth_hn_dev_uninit(struct rte_eth_dev *eth_dev)
 	hn_dev_stop(eth_dev);
 	hn_dev_close(eth_dev);
 
-	eth_dev->dev_ops = NULL;
-	eth_dev->tx_pkt_burst = NULL;
-	eth_dev->rx_pkt_burst = NULL;
-
 	hn_detach(hv);
 	hn_chim_uninit(eth_dev);
 	rte_vmbus_chan_close(hv->primary->chan);
diff --git a/drivers/net/octeontx/octeontx_ethdev.c b/drivers/net/octeontx/octeontx_ethdev.c
index aa9ef3bb70..0ea6de448a 100644
--- a/drivers/net/octeontx/octeontx_ethdev.c
+++ b/drivers/net/octeontx/octeontx_ethdev.c
@@ -513,9 +513,6 @@ octeontx_dev_close(struct rte_eth_dev *dev)
 
 	octeontx_port_close(nic);
 
-	dev->tx_pkt_burst = NULL;
-	dev->rx_pkt_burst = NULL;
-
 	return 0;
 }
 
diff --git a/drivers/net/qede/qede_ethdev.c b/drivers/net/qede/qede_ethdev.c
index cd578709fb..4055c74c2d 100644
--- a/drivers/net/qede/qede_ethdev.c
+++ b/drivers/net/qede/qede_ethdev.c
@@ -1591,10 +1591,6 @@ static int qede_dev_close(struct rte_eth_dev *eth_dev)
 	if (ECORE_IS_CMT(edev))
 		rte_eal_alarm_cancel(qede_poll_sp_sb_cb, (void *)eth_dev);
 
-	eth_dev->dev_ops = NULL;
-	eth_dev->rx_pkt_burst = NULL;
-	eth_dev->tx_pkt_burst = NULL;
-
 	return 0;
 }
 
diff --git a/drivers/net/sfc/sfc_ethdev.c b/drivers/net/sfc/sfc_ethdev.c
index 0558f1808d..9ee0b994eb 100644
--- a/drivers/net/sfc/sfc_ethdev.c
+++ b/drivers/net/sfc/sfc_ethdev.c
@@ -322,11 +322,7 @@ static void
 sfc_eth_dev_secondary_clear_ops(struct rte_eth_dev *dev)
 {
 	free(dev->process_private);
-	dev->process_private = NULL;
-	dev->dev_ops = NULL;
-	dev->tx_pkt_prepare = NULL;
-	dev->tx_pkt_burst = NULL;
-	dev->rx_pkt_burst = NULL;
+	rte_eth_dev_release_port(dev);
 }
 
 static int
@@ -378,7 +374,6 @@ sfc_dev_close(struct rte_eth_dev *dev)
 	/* Required for logging, so cleanup last */
 	sa->eth_dev = NULL;
 
-	dev->process_private = NULL;
 	free(sa);
 
 	return 0;
diff --git a/drivers/net/tap/rte_eth_tap.c b/drivers/net/tap/rte_eth_tap.c
index b127ce62dc..0598de00ed 100644
--- a/drivers/net/tap/rte_eth_tap.c
+++ b/drivers/net/tap/rte_eth_tap.c
@@ -1134,7 +1134,6 @@ tap_dev_close(struct rte_eth_dev *dev)
 		internals->ioctl_sock = -1;
 	}
 	rte_free(dev->process_private);
-	dev->process_private = NULL;
 	if (tap_devices_count == 1)
 		rte_mp_action_unregister(TAP_MP_KEY);
 	tap_devices_count--;
diff --git a/drivers/net/virtio/virtio_ethdev.c b/drivers/net/virtio/virtio_ethdev.c
index f2117675b3..3b3b2f6a7e 100644
--- a/drivers/net/virtio/virtio_ethdev.c
+++ b/drivers/net/virtio/virtio_ethdev.c
@@ -1995,10 +1995,6 @@ eth_virtio_dev_uninit(struct rte_eth_dev *eth_dev)
 	virtio_dev_stop(eth_dev);
 	virtio_dev_close(eth_dev);
 
-	eth_dev->dev_ops = NULL;
-	eth_dev->tx_pkt_burst = NULL;
-	eth_dev->rx_pkt_burst = NULL;
-
 	PMD_INIT_LOG(DEBUG, "dev_uninit completed");
 
 	return 0;
diff --git a/drivers/net/vmxnet3/vmxnet3_ethdev.c b/drivers/net/vmxnet3/vmxnet3_ethdev.c
index fa950e1ba0..12a782d987 100644
--- a/drivers/net/vmxnet3/vmxnet3_ethdev.c
+++ b/drivers/net/vmxnet3/vmxnet3_ethdev.c
@@ -368,11 +368,6 @@ eth_vmxnet3_dev_uninit(struct rte_eth_dev *eth_dev)
 		return -EBUSY;
 	}
 
-	eth_dev->dev_ops = NULL;
-	eth_dev->rx_pkt_burst = NULL;
-	eth_dev->tx_pkt_burst = NULL;
-	eth_dev->tx_pkt_prepare = NULL;
-
 	return 0;
 }
 
diff --git a/lib/librte_ethdev/rte_ethdev.c b/lib/librte_ethdev/rte_ethdev.c
index 0b8e8e3e8d..e68943b461 100644
--- a/lib/librte_ethdev/rte_ethdev.c
+++ b/lib/librte_ethdev/rte_ethdev.c
@@ -554,10 +554,6 @@ rte_eth_dev_release_port(struct rte_eth_dev *eth_dev)
 
 	rte_spinlock_lock(&rte_eth_dev_shared_data->ownership_lock);
 
-	eth_dev->state = RTE_ETH_DEV_UNUSED;
-	eth_dev->device = NULL;
-	eth_dev->intr_handle = NULL;
-
 	if (rte_eal_process_type() == RTE_PROC_PRIMARY) {
 		rte_free(eth_dev->data->rx_queues);
 		rte_free(eth_dev->data->tx_queues);
@@ -566,6 +562,7 @@ rte_eth_dev_release_port(struct rte_eth_dev *eth_dev)
 		rte_free(eth_dev->data->dev_private);
 		memset(eth_dev->data, 0, sizeof(struct rte_eth_dev_data));
 	}
+	memset(eth_dev, 0, sizeof(struct rte_eth_dev));
 
 	rte_spinlock_unlock(&rte_eth_dev_shared_data->ownership_lock);
 
-- 
2.28.0

Re: [dpdk-dev] [PATCH v3 28/29] ethdev: reset all when releasing a port

[Wang, Haiyue]

> -----Original Message-----
> From: Thomas Monjalon <thomas@monjalon.net>
> Sent: Tuesday, September 29, 2020 07:15
> To: dev@dpdk.org
> Cc: Yigit, Ferruh <ferruh.yigit@intel.com>; arybchenko@solarflare.com; Shepard Siegel
> <shepard.siegel@atomicrules.com>; Ed Czeck <ed.czeck@atomicrules.com>; John Miller
> <john.miller@atomicrules.com>; Igor Russkikh <igor.russkikh@aquantia.com>; Pavel Belous
> <pavel.belous@aquantia.com>; Somalapuram Amaranath <asomalap@amd.com>; Ajit Khaparde
> <ajit.khaparde@broadcom.com>; Somnath Kotur <somnath.kotur@broadcom.com>; Chas Williams
> <chas3@att.com>; Wei Hu (Xavier) <xavier.huwei@huawei.com>; Hemant Agrawal <hemant.agrawal@nxp.com>;
> Sachin Saxena <sachin.saxena@oss.nxp.com>; Guo, Jia <jia.guo@intel.com>; Wang, Haiyue
> <haiyue.wang@intel.com>; Marcin Wojtas <mw@semihalf.com>; Michal Krawczyk <mk@semihalf.com>; Guy
> Tzalik <gtzalik@amazon.com>; Evgeny Schemeilin <evgenys@amazon.com>; Igor Chauskin <igorch@amazon.com>;
> Zhang, Qi Z <qi.z.zhang@intel.com>; Wang, Xiao W <xiao.w.wang@intel.com>; Ziyang Xuan
> <xuanziyang2@huawei.com>; Xiaoyun Wang <cloud.wangxiaoyun@huawei.com>; Guoyang Zhou
> <zhouguoyang@huawei.com>; Min Hu (Connor) <humin29@huawei.com>; Yisen Zhuang <yisen.zhuang@huawei.com>;
> Xing, Beilei <beilei.xing@intel.com>; Wu, Jingjing <jingjing.wu@intel.com>; Yang, Qiming
> <qiming.yang@intel.com>; Alfredo Cardigliano <cardigliano@ntop.org>; Shijith Thotton
> <sthotton@marvell.com>; Srisivasubramanian Srinivasan <srinivasan@marvell.com>; Stephen Hemminger
> <sthemmin@microsoft.com>; K. Y. Srinivasan <kys@microsoft.com>; Haiyang Zhang <haiyangz@microsoft.com>;
> Long Li <longli@microsoft.com>; Harman Kalra <hkalra@marvell.com>; Rasesh Mody <rmody@marvell.com>;
> Shahed Shaikh <shshaikh@marvell.com>; Wiles, Keith <keith.wiles@intel.com>; Maxime Coquelin
> <maxime.coquelin@redhat.com>; Xia, Chenbo <chenbo.xia@intel.com>; Wang, Zhihong
> <zhihong.wang@intel.com>; Yong Wang <yongwang@vmware.com>
> Subject: [PATCH v3 28/29] ethdev: reset all when releasing a port
> 
> The function rte_eth_dev_release_port() was resetting partially
> the struct rte_eth_dev. The drivers were completing it
> with more pointers set to NULL in the close or remove operations.
> 
> A full memset is done so most of those assignments become useless.
> 
> Signed-off-by: Thomas Monjalon <thomas@monjalon.net>
> ---


>  drivers/net/e1000/em_ethdev.c          | 4 ----
>  drivers/net/e1000/igb_ethdev.c         | 8 --------
>  drivers/net/ixgbe/ixgbe_ethdev.c       | 8 --------

For e1000/ixgbe

Reviewed-by: Haiyue Wang <haiyue.wang@intel.com>

> --
> 2.28.0

Re: [dpdk-dev] [PATCH v3 28/29] ethdev: reset all when releasing a port

[Guo, Jia]

For drivers/net/i40e,

Acked-by: Jeff Guo <jia.guo@intel.com>

> -----Original Message-----
> From: Thomas Monjalon <thomas@monjalon.net>
> Sent: Tuesday, September 29, 2020 7:15 AM
> To: dev@dpdk.org
> Cc: Yigit, Ferruh <ferruh.yigit@intel.com>; arybchenko@solarflare.com;
> Shepard Siegel <shepard.siegel@atomicrules.com>; Ed Czeck
> <ed.czeck@atomicrules.com>; John Miller <john.miller@atomicrules.com>;
> Igor Russkikh <igor.russkikh@aquantia.com>; Pavel Belous
> <pavel.belous@aquantia.com>; Somalapuram Amaranath
> <asomalap@amd.com>; Ajit Khaparde <ajit.khaparde@broadcom.com>;
> Somnath Kotur <somnath.kotur@broadcom.com>; Chas Williams
> <chas3@att.com>; Wei Hu (Xavier) <xavier.huwei@huawei.com>; Hemant
> Agrawal <hemant.agrawal@nxp.com>; Sachin Saxena
> <sachin.saxena@oss.nxp.com>; Guo, Jia <jia.guo@intel.com>; Wang, Haiyue
> <haiyue.wang@intel.com>; Marcin Wojtas <mw@semihalf.com>; Michal
> Krawczyk <mk@semihalf.com>; Guy Tzalik <gtzalik@amazon.com>; Evgeny
> Schemeilin <evgenys@amazon.com>; Igor Chauskin <igorch@amazon.com>;
> Zhang, Qi Z <qi.z.zhang@intel.com>; Wang, Xiao W
> <xiao.w.wang@intel.com>; Ziyang Xuan <xuanziyang2@huawei.com>;
> Xiaoyun Wang <cloud.wangxiaoyun@huawei.com>; Guoyang Zhou
> <zhouguoyang@huawei.com>; Min Hu (Connor) <humin29@huawei.com>;
> Yisen Zhuang <yisen.zhuang@huawei.com>; Xing, Beilei
> <beilei.xing@intel.com>; Wu, Jingjing <jingjing.wu@intel.com>; Yang, Qiming
> <qiming.yang@intel.com>; Alfredo Cardigliano <cardigliano@ntop.org>;
> Shijith Thotton <sthotton@marvell.com>; Srisivasubramanian Srinivasan
> <srinivasan@marvell.com>; Stephen Hemminger
> <sthemmin@microsoft.com>; K. Y. Srinivasan <kys@microsoft.com>;
> Haiyang Zhang <haiyangz@microsoft.com>; Long Li <longli@microsoft.com>;
> Harman Kalra <hkalra@marvell.com>; Rasesh Mody <rmody@marvell.com>;
> Shahed Shaikh <shshaikh@marvell.com>; Wiles, Keith
> <keith.wiles@intel.com>; Maxime Coquelin
> <maxime.coquelin@redhat.com>; Xia, Chenbo <chenbo.xia@intel.com>;
> Wang, Zhihong <zhihong.wang@intel.com>; Yong Wang
> <yongwang@vmware.com>
> Subject: [PATCH v3 28/29] ethdev: reset all when releasing a port
> 
> The function rte_eth_dev_release_port() was resetting partially the struct
> rte_eth_dev. The drivers were completing it with more pointers set to NULL
> in the close or remove operations.
> 
> A full memset is done so most of those assignments become useless.
> 
> Signed-off-by: Thomas Monjalon <thomas@monjalon.net>
> ---
>  drivers/net/ark/ark_ethdev.c           | 3 ---
>  drivers/net/atlantic/atl_ethdev.c      | 4 ----
>  drivers/net/axgbe/axgbe_ethdev.c       | 3 ---
>  drivers/net/bnxt/bnxt_ethdev.c         | 4 ----
>  drivers/net/bnxt/bnxt_reps.c           | 1 -
>  drivers/net/bonding/rte_eth_bond_pmd.c | 4 ----
>  drivers/net/dpaa/dpaa_ethdev.c         | 4 ----
>  drivers/net/dpaa2/dpaa2_ethdev.c       | 4 ----
>  drivers/net/e1000/em_ethdev.c          | 4 ----
>  drivers/net/e1000/igb_ethdev.c         | 8 --------
>  drivers/net/ena/ena_ethdev.c           | 5 -----
>  drivers/net/fm10k/fm10k_ethdev.c       | 4 ----
>  drivers/net/hinic/hinic_pmd_ethdev.c   | 4 ----
>  drivers/net/hns3/hns3_ethdev.c         | 4 ----
>  drivers/net/hns3/hns3_ethdev_vf.c      | 5 -----
>  drivers/net/i40e/i40e_ethdev.c         | 4 ----
>  drivers/net/i40e/i40e_ethdev_vf.c      | 4 ----
>  drivers/net/iavf/iavf_ethdev.c         | 4 ----
>  drivers/net/ice/ice_dcf_ethdev.c       | 4 ----
>  drivers/net/ice/ice_ethdev.c           | 4 ----
>  drivers/net/ionic/ionic_ethdev.c       | 5 -----
>  drivers/net/ixgbe/ixgbe_ethdev.c       | 8 --------
>  drivers/net/liquidio/lio_ethdev.c      | 4 ----
>  drivers/net/netvsc/hn_ethdev.c         | 4 ----
>  drivers/net/octeontx/octeontx_ethdev.c | 3 ---
>  drivers/net/qede/qede_ethdev.c         | 4 ----
>  drivers/net/sfc/sfc_ethdev.c           | 7 +------
>  drivers/net/tap/rte_eth_tap.c          | 1 -
>  drivers/net/virtio/virtio_ethdev.c     | 4 ----
>  drivers/net/vmxnet3/vmxnet3_ethdev.c   | 5 -----
>  lib/librte_ethdev/rte_ethdev.c         | 5 +----
>  31 files changed, 2 insertions(+), 129 deletions(-)
> 
> diff --git a/drivers/net/ark/ark_ethdev.c b/drivers/net/ark/ark_ethdev.c
> index e3b1347769..cfcfe5c42f 100644
> --- a/drivers/net/ark/ark_ethdev.c
> +++ b/drivers/net/ark/ark_ethdev.c
> @@ -504,9 +504,6 @@ eth_ark_dev_uninit(struct rte_eth_dev *dev)
>  	ark_pktgen_uninit(ark->pg);
>  	ark_pktchkr_uninit(ark->pc);
> 
> -	dev->dev_ops = NULL;
> -	dev->rx_pkt_burst = NULL;
> -	dev->tx_pkt_burst = NULL;
>  	return 0;
>  }
> 
> diff --git a/drivers/net/atlantic/atl_ethdev.c
> b/drivers/net/atlantic/atl_ethdev.c
> index 2217511ca0..5cc41c2b61 100644
> --- a/drivers/net/atlantic/atl_ethdev.c
> +++ b/drivers/net/atlantic/atl_ethdev.c
> @@ -701,10 +701,6 @@ atl_dev_close(struct rte_eth_dev *dev)
> 
>  	atl_free_queues(dev);
> 
> -	dev->dev_ops = NULL;
> -	dev->rx_pkt_burst = NULL;
> -	dev->tx_pkt_burst = NULL;
> -
>  	/* disable uio intr before callback unregister */
>  	rte_intr_disable(intr_handle);
>  	rte_intr_callback_unregister(intr_handle,
> diff --git a/drivers/net/axgbe/axgbe_ethdev.c
> b/drivers/net/axgbe/axgbe_ethdev.c
> index cf085487cc..39087f36f0 100644
> --- a/drivers/net/axgbe/axgbe_ethdev.c
> +++ b/drivers/net/axgbe/axgbe_ethdev.c
> @@ -2134,9 +2134,6 @@ axgbe_dev_close(struct rte_eth_dev *eth_dev)
>  		return 0;
> 
>  	pci_dev = RTE_DEV_TO_PCI(eth_dev->device);
> -	eth_dev->dev_ops = NULL;
> -	eth_dev->rx_pkt_burst = NULL;
> -	eth_dev->tx_pkt_burst = NULL;
>  	axgbe_dev_clear_queues(eth_dev);
> 
>  	/* disable uio intr before callback unregister */ diff --git
> a/drivers/net/bnxt/bnxt_ethdev.c b/drivers/net/bnxt/bnxt_ethdev.c index
> d45347e8ca..e1c430b2cc 100644
> --- a/drivers/net/bnxt/bnxt_ethdev.c
> +++ b/drivers/net/bnxt/bnxt_ethdev.c
> @@ -1382,10 +1382,6 @@ static int bnxt_dev_close_op(struct rte_eth_dev
> *eth_dev)
>  	bnxt_free_pf_info(bp);
>  	bnxt_free_parent_info(bp);
> 
> -	eth_dev->dev_ops = NULL;
> -	eth_dev->rx_pkt_burst = NULL;
> -	eth_dev->tx_pkt_burst = NULL;
> -
>  	rte_memzone_free((const struct rte_memzone *)bp-
> >tx_mem_zone);
>  	bp->tx_mem_zone = NULL;
>  	rte_memzone_free((const struct rte_memzone *)bp-
> >rx_mem_zone); diff --git a/drivers/net/bnxt/bnxt_reps.c
> b/drivers/net/bnxt/bnxt_reps.c index c419fe63b8..d523c867eb 100644
> --- a/drivers/net/bnxt/bnxt_reps.c
> +++ b/drivers/net/bnxt/bnxt_reps.c
> @@ -225,7 +225,6 @@ int bnxt_vf_representor_uninit(struct rte_eth_dev
> *eth_dev)
> 
>  	PMD_DRV_LOG(DEBUG, "BNXT Port:%d VFR uninit\n", eth_dev-
> >data->port_id);
>  	eth_dev->data->mac_addrs = NULL;
> -	eth_dev->dev_ops = NULL;
> 
>  	parent_bp = rep->parent_dev->data->dev_private;
>  	if (!parent_bp) {
> diff --git a/drivers/net/bonding/rte_eth_bond_pmd.c
> b/drivers/net/bonding/rte_eth_bond_pmd.c
> index 1f761c7c9e..ce8fe254af 100644
> --- a/drivers/net/bonding/rte_eth_bond_pmd.c
> +++ b/drivers/net/bonding/rte_eth_bond_pmd.c
> @@ -2130,10 +2130,6 @@ bond_ethdev_close(struct rte_eth_dev *dev)
>  	 */
>  	rte_mempool_free(internals->mode6.mempool);
> 
> -	dev->dev_ops = NULL;
> -	dev->rx_pkt_burst = NULL;
> -	dev->tx_pkt_burst = NULL;
> -
>  	return 0;
>  }
> 
> diff --git a/drivers/net/dpaa/dpaa_ethdev.c
> b/drivers/net/dpaa/dpaa_ethdev.c index a01c8f3fc6..7e6a954d7b 100644
> --- a/drivers/net/dpaa/dpaa_ethdev.c
> +++ b/drivers/net/dpaa/dpaa_ethdev.c
> @@ -441,10 +441,6 @@ static int dpaa_eth_dev_close(struct rte_eth_dev
> *dev)
>  	rte_free(dpaa_intf->tx_queues);
>  	dpaa_intf->tx_queues = NULL;
> 
> -	dev->dev_ops = NULL;
> -	dev->rx_pkt_burst = NULL;
> -	dev->tx_pkt_burst = NULL;
> -
>  	return 0;
>  }
> 
> diff --git a/drivers/net/dpaa2/dpaa2_ethdev.c
> b/drivers/net/dpaa2/dpaa2_ethdev.c
> index d8624514df..ba99a524f9 100644
> --- a/drivers/net/dpaa2/dpaa2_ethdev.c
> +++ b/drivers/net/dpaa2/dpaa2_ethdev.c
> @@ -1284,10 +1284,6 @@ dpaa2_dev_close(struct rte_eth_dev *dev)
>  	if (priv->extract.qos_extract_param)
>  		rte_free((void *)(size_t)priv->extract.qos_extract_param);
> 
> -	dev->dev_ops = NULL;
> -	dev->rx_pkt_burst = NULL;
> -	dev->tx_pkt_burst = NULL;
> -
>  	DPAA2_PMD_INFO("%s: netdev deleted", dev->data->name);
>  	return 0;
>  }
> diff --git a/drivers/net/e1000/em_ethdev.c
> b/drivers/net/e1000/em_ethdev.c index d050eb478a..151b7543d8 100644
> --- a/drivers/net/e1000/em_ethdev.c
> +++ b/drivers/net/e1000/em_ethdev.c
> @@ -772,10 +772,6 @@ eth_em_close(struct rte_eth_dev *dev)
>  	em_release_manageability(hw);
>  	em_hw_control_release(hw);
> 
> -	dev->dev_ops = NULL;
> -	dev->rx_pkt_burst = NULL;
> -	dev->tx_pkt_burst = NULL;
> -
>  	/* disable uio intr before callback unregister */
>  	rte_intr_disable(intr_handle);
>  	rte_intr_callback_unregister(intr_handle,
> diff --git a/drivers/net/e1000/igb_ethdev.c
> b/drivers/net/e1000/igb_ethdev.c index cb3d97e2a3..a7fcbaf295 100644
> --- a/drivers/net/e1000/igb_ethdev.c
> +++ b/drivers/net/e1000/igb_ethdev.c
> @@ -1564,10 +1564,6 @@ eth_igb_close(struct rte_eth_dev *dev)
>  	memset(&link, 0, sizeof(link));
>  	rte_eth_linkstatus_set(dev, &link);
> 
> -	dev->dev_ops = NULL;
> -	dev->rx_pkt_burst = NULL;
> -	dev->tx_pkt_burst = NULL;
> -
>  	/* Reset any pending lock */
>  	igb_reset_swfw_lock(hw);
> 
> @@ -3403,10 +3399,6 @@ igbvf_dev_close(struct rte_eth_dev *dev)
>  	memset(&addr, 0, sizeof(addr));
>  	igbvf_default_mac_addr_set(dev, &addr);
> 
> -	dev->dev_ops = NULL;
> -	dev->rx_pkt_burst = NULL;
> -	dev->tx_pkt_burst = NULL;
> -
>  	rte_intr_callback_unregister(&pci_dev->intr_handle,
>  				     eth_igbvf_interrupt_handler,
>  				     (void *)dev);
> diff --git a/drivers/net/ena/ena_ethdev.c b/drivers/net/ena/ena_ethdev.c
> index 07ae9bab32..b42f12b870 100644
> --- a/drivers/net/ena/ena_ethdev.c
> +++ b/drivers/net/ena/ena_ethdev.c
> @@ -1933,11 +1933,6 @@ static int eth_ena_dev_uninit(struct rte_eth_dev
> *eth_dev)
> 
>  	ena_destroy_device(eth_dev);
> 
> -	eth_dev->dev_ops = NULL;
> -	eth_dev->rx_pkt_burst = NULL;
> -	eth_dev->tx_pkt_burst = NULL;
> -	eth_dev->tx_pkt_prepare = NULL;
> -
>  	return 0;
>  }
> 
> diff --git a/drivers/net/fm10k/fm10k_ethdev.c
> b/drivers/net/fm10k/fm10k_ethdev.c
> index 17cf0328c6..2d51f50faa 100644
> --- a/drivers/net/fm10k/fm10k_ethdev.c
> +++ b/drivers/net/fm10k/fm10k_ethdev.c
> @@ -2801,10 +2801,6 @@ fm10k_dev_close(struct rte_eth_dev *dev)
>  	fm10k_dev_queue_release(dev);
>  	fm10k_stop_hw(hw);
> 
> -	dev->dev_ops = NULL;
> -	dev->rx_pkt_burst = NULL;
> -	dev->tx_pkt_burst = NULL;
> -
>  	/* disable uio/vfio intr */
>  	rte_intr_disable(intr_handle);
> 
> diff --git a/drivers/net/hinic/hinic_pmd_ethdev.c
> b/drivers/net/hinic/hinic_pmd_ethdev.c
> index c5b06fae73..57084e59c1 100644
> --- a/drivers/net/hinic/hinic_pmd_ethdev.c
> +++ b/drivers/net/hinic/hinic_pmd_ethdev.c
> @@ -3222,10 +3222,6 @@ static int hinic_dev_uninit(struct rte_eth_dev
> *dev)
> 
>  	hinic_dev_close(dev);
> 
> -	dev->dev_ops = NULL;
> -	dev->rx_pkt_burst = NULL;
> -	dev->tx_pkt_burst = NULL;
> -
>  	rte_free(nic_dev->mc_list);
> 
>  	return HINIC_OK;
> diff --git a/drivers/net/hns3/hns3_ethdev.c
> b/drivers/net/hns3/hns3_ethdev.c index dd03bd62b2..ba40d489d6 100644
> --- a/drivers/net/hns3/hns3_ethdev.c
> +++ b/drivers/net/hns3/hns3_ethdev.c
> @@ -5721,10 +5721,6 @@ hns3_dev_uninit(struct rte_eth_dev *eth_dev)
>  	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
>  		return -EPERM;
> 
> -	eth_dev->dev_ops = NULL;
> -	eth_dev->rx_pkt_burst = NULL;
> -	eth_dev->tx_pkt_burst = NULL;
> -	eth_dev->tx_pkt_prepare = NULL;
>  	if (hw->adapter_state < HNS3_NIC_CLOSING)
>  		hns3_dev_close(eth_dev);
> 
> diff --git a/drivers/net/hns3/hns3_ethdev_vf.c
> b/drivers/net/hns3/hns3_ethdev_vf.c
> index 436d864b5f..6a04cd9c2c 100644
> --- a/drivers/net/hns3/hns3_ethdev_vf.c
> +++ b/drivers/net/hns3/hns3_ethdev_vf.c
> @@ -2728,11 +2728,6 @@ hns3vf_dev_uninit(struct rte_eth_dev *eth_dev)
>  	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
>  		return -EPERM;
> 
> -	eth_dev->dev_ops = NULL;
> -	eth_dev->rx_pkt_burst = NULL;
> -	eth_dev->tx_pkt_burst = NULL;
> -	eth_dev->tx_pkt_prepare = NULL;
> -
>  	if (hw->adapter_state < HNS3_NIC_CLOSING)
>  		hns3vf_dev_close(eth_dev);
> 
> diff --git a/drivers/net/i40e/i40e_ethdev.c b/drivers/net/i40e/i40e_ethdev.c
> index feeed2dec0..a05d29dcfc 100644
> --- a/drivers/net/i40e/i40e_ethdev.c
> +++ b/drivers/net/i40e/i40e_ethdev.c
> @@ -2692,10 +2692,6 @@ i40e_dev_close(struct rte_eth_dev *dev)
>  			(reg | I40E_PFGEN_CTRL_PFSWR_MASK));
>  	I40E_WRITE_FLUSH(hw);
> 
> -	dev->dev_ops = NULL;
> -	dev->rx_pkt_burst = NULL;
> -	dev->tx_pkt_burst = NULL;
> -
>  	/* Clear PXE mode */
>  	i40e_clear_pxe_mode(hw);
> 
> diff --git a/drivers/net/i40e/i40e_ethdev_vf.c
> b/drivers/net/i40e/i40e_ethdev_vf.c
> index 4d6510d1ff..2001854ae6 100644
> --- a/drivers/net/i40e/i40e_ethdev_vf.c
> +++ b/drivers/net/i40e/i40e_ethdev_vf.c
> @@ -2421,10 +2421,6 @@ i40evf_dev_close(struct rte_eth_dev *dev)
>  	i40e_shutdown_adminq(hw);
>  	i40evf_disable_irq0(hw);
> 
> -	dev->dev_ops = NULL;
> -	dev->rx_pkt_burst = NULL;
> -	dev->tx_pkt_burst = NULL;
> -
>  	rte_free(vf->vf_res);
>  	vf->vf_res = NULL;
>  	rte_free(vf->aq_resp);
> diff --git a/drivers/net/iavf/iavf_ethdev.c b/drivers/net/iavf/iavf_ethdev.c
> index a5b1433306..02391acfd1 100644
> --- a/drivers/net/iavf/iavf_ethdev.c
> +++ b/drivers/net/iavf/iavf_ethdev.c
> @@ -1483,10 +1483,6 @@ iavf_dev_close(struct rte_eth_dev *dev)
>  				     iavf_dev_interrupt_handler, dev);
>  	iavf_disable_irq0(hw);
> 
> -	dev->dev_ops = NULL;
> -	dev->rx_pkt_burst = NULL;
> -	dev->tx_pkt_burst = NULL;
> -
>  	if (vf->vf_res->vf_cap_flags & VIRTCHNL_VF_OFFLOAD_RSS_PF) {
>  		if (vf->rss_lut) {
>  			rte_free(vf->rss_lut);
> diff --git a/drivers/net/ice/ice_dcf_ethdev.c
> b/drivers/net/ice/ice_dcf_ethdev.c
> index 33dd0c44f2..f2056e2aba 100644
> --- a/drivers/net/ice/ice_dcf_ethdev.c
> +++ b/drivers/net/ice/ice_dcf_ethdev.c
> @@ -855,10 +855,6 @@ ice_dcf_dev_close(struct rte_eth_dev *dev)
>  	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
>  		return 0;
> 
> -	dev->dev_ops = NULL;
> -	dev->rx_pkt_burst = NULL;
> -	dev->tx_pkt_burst = NULL;
> -
>  	ice_dcf_uninit_parent_adapter(dev);
>  	ice_dcf_uninit_hw(dev, &adapter->real_hw);
> 
> diff --git a/drivers/net/ice/ice_ethdev.c b/drivers/net/ice/ice_ethdev.c
> index 73af19ce81..4f977bd4c4 100644
> --- a/drivers/net/ice/ice_ethdev.c
> +++ b/drivers/net/ice/ice_ethdev.c
> @@ -2463,10 +2463,6 @@ ice_dev_close(struct rte_eth_dev *dev)
>  	rte_free(pf->proto_xtr);
>  	pf->proto_xtr = NULL;
> 
> -	dev->dev_ops = NULL;
> -	dev->rx_pkt_burst = NULL;
> -	dev->tx_pkt_burst = NULL;
> -
>  	/* disable uio intr before callback unregister */
>  	rte_intr_disable(intr_handle);
> 
> diff --git a/drivers/net/ionic/ionic_ethdev.c
> b/drivers/net/ionic/ionic_ethdev.c
> index ef7d06e526..1387253dc8 100644
> --- a/drivers/net/ionic/ionic_ethdev.c
> +++ b/drivers/net/ionic/ionic_ethdev.c
> @@ -1065,11 +1065,6 @@ eth_ionic_dev_uninit(struct rte_eth_dev
> *eth_dev)
>  	ionic_lif_deinit(lif);
>  	ionic_lif_free(lif);
> 
> -	eth_dev->dev_ops = NULL;
> -	eth_dev->rx_pkt_burst = NULL;
> -	eth_dev->tx_pkt_burst = NULL;
> -	eth_dev->tx_pkt_prepare = NULL;
> -
>  	return 0;
>  }
> 
> diff --git a/drivers/net/ixgbe/ixgbe_ethdev.c
> b/drivers/net/ixgbe/ixgbe_ethdev.c
> index 84c1b0f7a1..be8a79009d 100644
> --- a/drivers/net/ixgbe/ixgbe_ethdev.c
> +++ b/drivers/net/ixgbe/ixgbe_ethdev.c
> @@ -3009,10 +3009,6 @@ ixgbe_dev_close(struct rte_eth_dev *dev)
>  	/* reprogram the RAR[0] in case user changed it. */
>  	ixgbe_set_rar(hw, 0, hw->mac.addr, 0, IXGBE_RAH_AV);
> 
> -	dev->dev_ops = NULL;
> -	dev->rx_pkt_burst = NULL;
> -	dev->tx_pkt_burst = NULL;
> -
>  	/* Unlock any pending hardware semaphore */
>  	ixgbe_swfw_lock_reset(hw);
> 
> @@ -5460,10 +5456,6 @@ ixgbevf_dev_close(struct rte_eth_dev *dev)
>  	 **/
>  	ixgbevf_remove_mac_addr(dev, 0);
> 
> -	dev->dev_ops = NULL;
> -	dev->rx_pkt_burst = NULL;
> -	dev->tx_pkt_burst = NULL;
> -
>  	rte_intr_disable(intr_handle);
>  	rte_intr_callback_unregister(intr_handle,
>  				     ixgbevf_dev_interrupt_handler, dev); diff
> --git a/drivers/net/liquidio/lio_ethdev.c b/drivers/net/liquidio/lio_ethdev.c
> index 1a41f27198..9c4da467eb 100644
> --- a/drivers/net/liquidio/lio_ethdev.c
> +++ b/drivers/net/liquidio/lio_ethdev.c
> @@ -2067,10 +2067,6 @@ lio_eth_dev_uninit(struct rte_eth_dev *eth_dev)
>  	/* lio_free_sc_buffer_pool */
>  	lio_free_sc_buffer_pool(lio_dev);
> 
> -	eth_dev->dev_ops = NULL;
> -	eth_dev->rx_pkt_burst = NULL;
> -	eth_dev->tx_pkt_burst = NULL;
> -
>  	return 0;
>  }
> 
> diff --git a/drivers/net/netvsc/hn_ethdev.c
> b/drivers/net/netvsc/hn_ethdev.c index 5ae2d469c8..9af64821a1 100644
> --- a/drivers/net/netvsc/hn_ethdev.c
> +++ b/drivers/net/netvsc/hn_ethdev.c
> @@ -1049,10 +1049,6 @@ eth_hn_dev_uninit(struct rte_eth_dev *eth_dev)
>  	hn_dev_stop(eth_dev);
>  	hn_dev_close(eth_dev);
> 
> -	eth_dev->dev_ops = NULL;
> -	eth_dev->tx_pkt_burst = NULL;
> -	eth_dev->rx_pkt_burst = NULL;
> -
>  	hn_detach(hv);
>  	hn_chim_uninit(eth_dev);
>  	rte_vmbus_chan_close(hv->primary->chan);
> diff --git a/drivers/net/octeontx/octeontx_ethdev.c
> b/drivers/net/octeontx/octeontx_ethdev.c
> index aa9ef3bb70..0ea6de448a 100644
> --- a/drivers/net/octeontx/octeontx_ethdev.c
> +++ b/drivers/net/octeontx/octeontx_ethdev.c
> @@ -513,9 +513,6 @@ octeontx_dev_close(struct rte_eth_dev *dev)
> 
>  	octeontx_port_close(nic);
> 
> -	dev->tx_pkt_burst = NULL;
> -	dev->rx_pkt_burst = NULL;
> -
>  	return 0;
>  }
> 
> diff --git a/drivers/net/qede/qede_ethdev.c
> b/drivers/net/qede/qede_ethdev.c index cd578709fb..4055c74c2d 100644
> --- a/drivers/net/qede/qede_ethdev.c
> +++ b/drivers/net/qede/qede_ethdev.c
> @@ -1591,10 +1591,6 @@ static int qede_dev_close(struct rte_eth_dev
> *eth_dev)
>  	if (ECORE_IS_CMT(edev))
>  		rte_eal_alarm_cancel(qede_poll_sp_sb_cb, (void
> *)eth_dev);
> 
> -	eth_dev->dev_ops = NULL;
> -	eth_dev->rx_pkt_burst = NULL;
> -	eth_dev->tx_pkt_burst = NULL;
> -
>  	return 0;
>  }
> 
> diff --git a/drivers/net/sfc/sfc_ethdev.c b/drivers/net/sfc/sfc_ethdev.c
> index 0558f1808d..9ee0b994eb 100644
> --- a/drivers/net/sfc/sfc_ethdev.c
> +++ b/drivers/net/sfc/sfc_ethdev.c
> @@ -322,11 +322,7 @@ static void
>  sfc_eth_dev_secondary_clear_ops(struct rte_eth_dev *dev)  {
>  	free(dev->process_private);
> -	dev->process_private = NULL;
> -	dev->dev_ops = NULL;
> -	dev->tx_pkt_prepare = NULL;
> -	dev->tx_pkt_burst = NULL;
> -	dev->rx_pkt_burst = NULL;
> +	rte_eth_dev_release_port(dev);
>  }
> 
>  static int
> @@ -378,7 +374,6 @@ sfc_dev_close(struct rte_eth_dev *dev)
>  	/* Required for logging, so cleanup last */
>  	sa->eth_dev = NULL;
> 
> -	dev->process_private = NULL;
>  	free(sa);
> 
>  	return 0;
> diff --git a/drivers/net/tap/rte_eth_tap.c b/drivers/net/tap/rte_eth_tap.c
> index b127ce62dc..0598de00ed 100644
> --- a/drivers/net/tap/rte_eth_tap.c
> +++ b/drivers/net/tap/rte_eth_tap.c
> @@ -1134,7 +1134,6 @@ tap_dev_close(struct rte_eth_dev *dev)
>  		internals->ioctl_sock = -1;
>  	}
>  	rte_free(dev->process_private);
> -	dev->process_private = NULL;
>  	if (tap_devices_count == 1)
>  		rte_mp_action_unregister(TAP_MP_KEY);
>  	tap_devices_count--;
> diff --git a/drivers/net/virtio/virtio_ethdev.c
> b/drivers/net/virtio/virtio_ethdev.c
> index f2117675b3..3b3b2f6a7e 100644
> --- a/drivers/net/virtio/virtio_ethdev.c
> +++ b/drivers/net/virtio/virtio_ethdev.c
> @@ -1995,10 +1995,6 @@ eth_virtio_dev_uninit(struct rte_eth_dev
> *eth_dev)
>  	virtio_dev_stop(eth_dev);
>  	virtio_dev_close(eth_dev);
> 
> -	eth_dev->dev_ops = NULL;
> -	eth_dev->tx_pkt_burst = NULL;
> -	eth_dev->rx_pkt_burst = NULL;
> -
>  	PMD_INIT_LOG(DEBUG, "dev_uninit completed");
> 
>  	return 0;
> diff --git a/drivers/net/vmxnet3/vmxnet3_ethdev.c
> b/drivers/net/vmxnet3/vmxnet3_ethdev.c
> index fa950e1ba0..12a782d987 100644
> --- a/drivers/net/vmxnet3/vmxnet3_ethdev.c
> +++ b/drivers/net/vmxnet3/vmxnet3_ethdev.c
> @@ -368,11 +368,6 @@ eth_vmxnet3_dev_uninit(struct rte_eth_dev
> *eth_dev)
>  		return -EBUSY;
>  	}
> 
> -	eth_dev->dev_ops = NULL;
> -	eth_dev->rx_pkt_burst = NULL;
> -	eth_dev->tx_pkt_burst = NULL;
> -	eth_dev->tx_pkt_prepare = NULL;
> -
>  	return 0;
>  }
> 
> diff --git a/lib/librte_ethdev/rte_ethdev.c b/lib/librte_ethdev/rte_ethdev.c
> index 0b8e8e3e8d..e68943b461 100644
> --- a/lib/librte_ethdev/rte_ethdev.c
> +++ b/lib/librte_ethdev/rte_ethdev.c
> @@ -554,10 +554,6 @@ rte_eth_dev_release_port(struct rte_eth_dev
> *eth_dev)
> 
>  	rte_spinlock_lock(&rte_eth_dev_shared_data->ownership_lock);
> 
> -	eth_dev->state = RTE_ETH_DEV_UNUSED;
> -	eth_dev->device = NULL;
> -	eth_dev->intr_handle = NULL;
> -
>  	if (rte_eal_process_type() == RTE_PROC_PRIMARY) {
>  		rte_free(eth_dev->data->rx_queues);
>  		rte_free(eth_dev->data->tx_queues);
> @@ -566,6 +562,7 @@ rte_eth_dev_release_port(struct rte_eth_dev
> *eth_dev)
>  		rte_free(eth_dev->data->dev_private);
>  		memset(eth_dev->data, 0, sizeof(struct rte_eth_dev_data));
>  	}
> +	memset(eth_dev, 0, sizeof(struct rte_eth_dev));
> 
>  	rte_spinlock_unlock(&rte_eth_dev_shared_data->ownership_lock);
> 
> --
> 2.28.0

Re: [dpdk-dev] [PATCH v3 28/29] ethdev: reset all when releasing a port

[Maxime Coquelin]

On 9/29/20 1:14 AM, Thomas Monjalon wrote:
> The function rte_eth_dev_release_port() was resetting partially
> the struct rte_eth_dev. The drivers were completing it
> with more pointers set to NULL in the close or remove operations.
> 
> A full memset is done so most of those assignments become useless.
> 
> Signed-off-by: Thomas Monjalon <thomas@monjalon.net>
> ---
>  drivers/net/ark/ark_ethdev.c           | 3 ---
>  drivers/net/atlantic/atl_ethdev.c      | 4 ----
>  drivers/net/axgbe/axgbe_ethdev.c       | 3 ---
>  drivers/net/bnxt/bnxt_ethdev.c         | 4 ----
>  drivers/net/bnxt/bnxt_reps.c           | 1 -
>  drivers/net/bonding/rte_eth_bond_pmd.c | 4 ----
>  drivers/net/dpaa/dpaa_ethdev.c         | 4 ----
>  drivers/net/dpaa2/dpaa2_ethdev.c       | 4 ----
>  drivers/net/e1000/em_ethdev.c          | 4 ----
>  drivers/net/e1000/igb_ethdev.c         | 8 --------
>  drivers/net/ena/ena_ethdev.c           | 5 -----
>  drivers/net/fm10k/fm10k_ethdev.c       | 4 ----
>  drivers/net/hinic/hinic_pmd_ethdev.c   | 4 ----
>  drivers/net/hns3/hns3_ethdev.c         | 4 ----
>  drivers/net/hns3/hns3_ethdev_vf.c      | 5 -----
>  drivers/net/i40e/i40e_ethdev.c         | 4 ----
>  drivers/net/i40e/i40e_ethdev_vf.c      | 4 ----
>  drivers/net/iavf/iavf_ethdev.c         | 4 ----
>  drivers/net/ice/ice_dcf_ethdev.c       | 4 ----
>  drivers/net/ice/ice_ethdev.c           | 4 ----
>  drivers/net/ionic/ionic_ethdev.c       | 5 -----
>  drivers/net/ixgbe/ixgbe_ethdev.c       | 8 --------
>  drivers/net/liquidio/lio_ethdev.c      | 4 ----
>  drivers/net/netvsc/hn_ethdev.c         | 4 ----
>  drivers/net/octeontx/octeontx_ethdev.c | 3 ---
>  drivers/net/qede/qede_ethdev.c         | 4 ----
>  drivers/net/sfc/sfc_ethdev.c           | 7 +------
>  drivers/net/tap/rte_eth_tap.c          | 1 -
>  drivers/net/virtio/virtio_ethdev.c     | 4 ----
>  drivers/net/vmxnet3/vmxnet3_ethdev.c   | 5 -----
>  lib/librte_ethdev/rte_ethdev.c         | 5 +----
>  31 files changed, 2 insertions(+), 129 deletions(-)

With this patch, I get following segfault at init time with Virtio PMD:

        Program received signal SIGSEGV, Segmentation fault.
        0x0000000000854c9b in rte_eth_dev_callback_register (port_id=32,
            event=RTE_ETH_EVENT_UNKNOWN, cb_fn=0x4b24de
<eth_event_callback>,
            cb_arg=0x0) at ../lib/librte_ethdev/rte_ethdev.c:4042
        4042
TAILQ_INSERT_TAIL(&(dev->link_intr_cbs),

        (gdb) bt full
        #0  0x0000000000854c9b in rte_eth_dev_callback_register (port_id=32,
            event=RTE_ETH_EVENT_UNKNOWN, cb_fn=0x4b24de
<eth_event_callback>,
            cb_arg=0x0) at ../lib/librte_ethdev/rte_ethdev.c:4042
                dev = 0x2fe9980 <rte_eth_devices>
                user_cb = 0x1003b0300
                next_port = 0
                last_port = 31
        #1  0x00000000004b268f in register_eth_event_callback ()
            at ../app/test-pmd/testpmd.c:3111
                ret = 0
                event = RTE_ETH_EVENT_UNKNOWN
        #2  0x00000000004b3a7e in main (argc=2, argv=0x7fffffffeb18)
            at ../app/test-pmd/testpmd.c:3658
                diag = 0
                port_id = 0
                count = 0
                ret = -5584

        (gdb) l 4042
        4037                                    sizeof(struct
rte_eth_dev_callback), 0);
        4038                            if (user_cb != NULL) {
        4039                                    user_cb->cb_fn = cb_fn;
        4040                                    user_cb->cb_arg = cb_arg;
        4041                                    user_cb->event = event;
        4042
TAILQ_INSERT_TAIL(&(dev->link_intr_cbs),
        4043
user_cb, next);
        4044                            } else {
        4045
rte_spinlock_unlock(&rte_eth_dev_cb_lock);
        4046
rte_eth_dev_callback_unregister(port_id, event,


(gdb) p rte_eth_devices[0]
$2 = {
  rx_pkt_burst = 0x0,
  tx_pkt_burst = 0x0,
  tx_pkt_prepare = 0x0,
  rx_queue_count = 0x0,
  rx_descriptor_done = 0x1589664 <virtio_dev_rx_queue_done>,
  rx_descriptor_status = 0x0,
  tx_descriptor_status = 0x0,
  data = 0x1003b3700,
  process_private = 0x0,
  dev_ops = 0x29621a0 <virtio_eth_dev_ops>,
  device = 0x3174dc0,
  intr_handle = 0x3174ea0,
  link_intr_cbs = {
    tqh_first = 0x0,
    tqh_last = 0x0
  },
  post_rx_burst_cbs = {0x0 <repeats 1024 times>},
  pre_tx_burst_cbs = {0x0 <repeats 1024 times>},
  state = RTE_ETH_DEV_ATTACHED,
  security_ctx = 0x0,
  reserved_64s = {0, 0, 0, 0},
  reserved_ptrs = {0x0, 0x0, 0x0, 0x0}
}

When reverted, it looks OK.

Thanks,
Maxime

Re: [dpdk-dev] [PATCH v3 28/29] ethdev: reset all when releasing a port

[Thomas Monjalon]

29/09/2020 12:26, Maxime Coquelin:
> On 9/29/20 1:14 AM, Thomas Monjalon wrote:
> > The function rte_eth_dev_release_port() was resetting partially
> > the struct rte_eth_dev. The drivers were completing it
> > with more pointers set to NULL in the close or remove operations.
> > 
> > A full memset is done so most of those assignments become useless.
[...]
> With this patch, I get following segfault at init time with Virtio PMD:
> 
>         Program received signal SIGSEGV, Segmentation fault.
>         0x0000000000854c9b in rte_eth_dev_callback_register (port_id=32,
>             event=RTE_ETH_EVENT_UNKNOWN, cb_fn=0x4b24de
> <eth_event_callback>,
>             cb_arg=0x0) at ../lib/librte_ethdev/rte_ethdev.c:4042
>         4042
> TAILQ_INSERT_TAIL(&(dev->link_intr_cbs),

Yes this is because after closing a port, everything is resetted,
including .link_intr_cbs which is set only once in a constructor:
	http://git.dpdk.org/dpdk/commit/?id=9ec0b3869d8

I can change this patch to selectively set pointers to NULL.

Or if we prefer a big memset 0, we need to rework how RTE_ETH_ALL
is managed to register a callback for any event.
Instead of setting the callback for all ports, we could have
a special catch-call callback list which is called for all events.
This way we could revert initializing .link_intr_cbs in eth_dev_get().

Re: [dpdk-dev] [PATCH v3 28/29] ethdev: reset all when releasing a port

[Wang, Haiyue]

> -----Original Message-----
> From: Thomas Monjalon <thomas@monjalon.net>
> Sent: Tuesday, September 29, 2020 18:36
> To: Maxime Coquelin <maxime.coquelin@redhat.com>
> Cc: dev@dpdk.org; Yigit, Ferruh <ferruh.yigit@intel.com>; arybchenko@solarflare.com; Shepard Siegel
> <shepard.siegel@atomicrules.com>; Ed Czeck <ed.czeck@atomicrules.com>; John Miller
> <john.miller@atomicrules.com>; Igor Russkikh <igor.russkikh@aquantia.com>; Pavel Belous
> <pavel.belous@aquantia.com>; Somalapuram Amaranath <asomalap@amd.com>; Ajit Khaparde
> <ajit.khaparde@broadcom.com>; Somnath Kotur <somnath.kotur@broadcom.com>; Chas Williams
> <chas3@att.com>; Wei Hu (Xavier) <xavier.huwei@huawei.com>; Hemant Agrawal <hemant.agrawal@nxp.com>;
> Sachin Saxena <sachin.saxena@oss.nxp.com>; Guo, Jia <jia.guo@intel.com>; Wang, Haiyue
> <haiyue.wang@intel.com>; Marcin Wojtas <mw@semihalf.com>; Michal Krawczyk <mk@semihalf.com>; Guy
> Tzalik <gtzalik@amazon.com>; Evgeny Schemeilin <evgenys@amazon.com>; Igor Chauskin <igorch@amazon.com>;
> Zhang, Qi Z <qi.z.zhang@intel.com>; Wang, Xiao W <xiao.w.wang@intel.com>; Ziyang Xuan
> <xuanziyang2@huawei.com>; Xiaoyun Wang <cloud.wangxiaoyun@huawei.com>; Guoyang Zhou
> <zhouguoyang@huawei.com>; Min Hu (Connor) <humin29@huawei.com>; Yisen Zhuang <yisen.zhuang@huawei.com>;
> Xing, Beilei <beilei.xing@intel.com>; Wu, Jingjing <jingjing.wu@intel.com>; Yang, Qiming
> <qiming.yang@intel.com>; Alfredo Cardigliano <cardigliano@ntop.org>; Shijith Thotton
> <sthotton@marvell.com>; Srisivasubramanian Srinivasan <srinivasan@marvell.com>; Stephen Hemminger
> <sthemmin@microsoft.com>; K. Y. Srinivasan <kys@microsoft.com>; Haiyang Zhang <haiyangz@microsoft.com>;
> Long Li <longli@microsoft.com>; Harman Kalra <hkalra@marvell.com>; Rasesh Mody <rmody@marvell.com>;
> Shahed Shaikh <shshaikh@marvell.com>; Wiles, Keith <keith.wiles@intel.com>; Xia, Chenbo
> <chenbo.xia@intel.com>; Wang, Zhihong <zhihong.wang@intel.com>; Yong Wang <yongwang@vmware.com>;
> matan@nvidia.com
> Subject: Re: [PATCH v3 28/29] ethdev: reset all when releasing a port
> 
> 29/09/2020 12:26, Maxime Coquelin:
> > On 9/29/20 1:14 AM, Thomas Monjalon wrote:
> > > The function rte_eth_dev_release_port() was resetting partially
> > > the struct rte_eth_dev. The drivers were completing it
> > > with more pointers set to NULL in the close or remove operations.
> > >
> > > A full memset is done so most of those assignments become useless.
> [...]
> > With this patch, I get following segfault at init time with Virtio PMD:
> >
> >         Program received signal SIGSEGV, Segmentation fault.
> >         0x0000000000854c9b in rte_eth_dev_callback_register (port_id=32,
> >             event=RTE_ETH_EVENT_UNKNOWN, cb_fn=0x4b24de
> > <eth_event_callback>,
> >             cb_arg=0x0) at ../lib/librte_ethdev/rte_ethdev.c:4042
> >         4042
> > TAILQ_INSERT_TAIL(&(dev->link_intr_cbs),
> 
> Yes this is because after closing a port, everything is resetted,
> including .link_intr_cbs which is set only once in a constructor:
> 	http://git.dpdk.org/dpdk/commit/?id=9ec0b3869d8
> 
> I can change this patch to selectively set pointers to NULL.
> 
> Or if we prefer a big memset 0, we need to rework how RTE_ETH_ALL
> is managed to register a callback for any event.
> Instead of setting the callback for all ports, we could have
> a special catch-call callback list which is called for all events.
> This way we could revert initializing .link_intr_cbs in eth_dev_get().
> 
> 

Move 'struct rte_eth_dev_cb_list link_intr_cbs' to the end of 'struct rte_eth_dev',
and starting from link_intr_cbs, these members will be kept after closed ? :-)

memset(eth_dev, 0, offsetof(struct rte_eth_dev, link_intr_cbs));

Re: [dpdk-dev] [PATCH v3 28/29] ethdev: reset all when releasing a port

[Ferruh Yigit]

On 9/29/2020 12:58 PM, Wang, Haiyue wrote:
>> -----Original Message-----
>> From: Thomas Monjalon <thomas@monjalon.net>
>> Sent: Tuesday, September 29, 2020 18:36
>> To: Maxime Coquelin <maxime.coquelin@redhat.com>
>> Cc: dev@dpdk.org; Yigit, Ferruh <ferruh.yigit@intel.com>; arybchenko@solarflare.com; Shepard Siegel
>> <shepard.siegel@atomicrules.com>; Ed Czeck <ed.czeck@atomicrules.com>; John Miller
>> <john.miller@atomicrules.com>; Igor Russkikh <igor.russkikh@aquantia.com>; Pavel Belous
>> <pavel.belous@aquantia.com>; Somalapuram Amaranath <asomalap@amd.com>; Ajit Khaparde
>> <ajit.khaparde@broadcom.com>; Somnath Kotur <somnath.kotur@broadcom.com>; Chas Williams
>> <chas3@att.com>; Wei Hu (Xavier) <xavier.huwei@huawei.com>; Hemant Agrawal <hemant.agrawal@nxp.com>;
>> Sachin Saxena <sachin.saxena@oss.nxp.com>; Guo, Jia <jia.guo@intel.com>; Wang, Haiyue
>> <haiyue.wang@intel.com>; Marcin Wojtas <mw@semihalf.com>; Michal Krawczyk <mk@semihalf.com>; Guy
>> Tzalik <gtzalik@amazon.com>; Evgeny Schemeilin <evgenys@amazon.com>; Igor Chauskin <igorch@amazon.com>;
>> Zhang, Qi Z <qi.z.zhang@intel.com>; Wang, Xiao W <xiao.w.wang@intel.com>; Ziyang Xuan
>> <xuanziyang2@huawei.com>; Xiaoyun Wang <cloud.wangxiaoyun@huawei.com>; Guoyang Zhou
>> <zhouguoyang@huawei.com>; Min Hu (Connor) <humin29@huawei.com>; Yisen Zhuang <yisen.zhuang@huawei.com>;
>> Xing, Beilei <beilei.xing@intel.com>; Wu, Jingjing <jingjing.wu@intel.com>; Yang, Qiming
>> <qiming.yang@intel.com>; Alfredo Cardigliano <cardigliano@ntop.org>; Shijith Thotton
>> <sthotton@marvell.com>; Srisivasubramanian Srinivasan <srinivasan@marvell.com>; Stephen Hemminger
>> <sthemmin@microsoft.com>; K. Y. Srinivasan <kys@microsoft.com>; Haiyang Zhang <haiyangz@microsoft.com>;
>> Long Li <longli@microsoft.com>; Harman Kalra <hkalra@marvell.com>; Rasesh Mody <rmody@marvell.com>;
>> Shahed Shaikh <shshaikh@marvell.com>; Wiles, Keith <keith.wiles@intel.com>; Xia, Chenbo
>> <chenbo.xia@intel.com>; Wang, Zhihong <zhihong.wang@intel.com>; Yong Wang <yongwang@vmware.com>;
>> matan@nvidia.com
>> Subject: Re: [PATCH v3 28/29] ethdev: reset all when releasing a port
>>
>> 29/09/2020 12:26, Maxime Coquelin:
>>> On 9/29/20 1:14 AM, Thomas Monjalon wrote:
>>>> The function rte_eth_dev_release_port() was resetting partially
>>>> the struct rte_eth_dev. The drivers were completing it
>>>> with more pointers set to NULL in the close or remove operations.
>>>>
>>>> A full memset is done so most of those assignments become useless.
>> [...]
>>> With this patch, I get following segfault at init time with Virtio PMD:
>>>
>>>          Program received signal SIGSEGV, Segmentation fault.
>>>          0x0000000000854c9b in rte_eth_dev_callback_register (port_id=32,
>>>              event=RTE_ETH_EVENT_UNKNOWN, cb_fn=0x4b24de
>>> <eth_event_callback>,
>>>              cb_arg=0x0) at ../lib/librte_ethdev/rte_ethdev.c:4042
>>>          4042
>>> TAILQ_INSERT_TAIL(&(dev->link_intr_cbs),
>>
>> Yes this is because after closing a port, everything is resetted,
>> including .link_intr_cbs which is set only once in a constructor:
>> http://git.dpdk.org/dpdk/commit/?id=9ec0b3869d8
>>
>> I can change this patch to selectively set pointers to NULL.
>>
>> Or if we prefer a big memset 0, we need to rework how RTE_ETH_ALL
>> is managed to register a callback for any event.
>> Instead of setting the callback for all ports, we could have
>> a special catch-call callback list which is called for all events.
>> This way we could revert initializing .link_intr_cbs in eth_dev_get().
>>
>>
> 
> Move 'struct rte_eth_dev_cb_list link_intr_cbs' to the end of 'struct rte_eth_dev',
> and starting from link_intr_cbs, these members will be kept after closed ? :-)
> 
> memset(eth_dev, 0, offsetof(struct rte_eth_dev, link_intr_cbs));
> 

This is similar version of a previous patch:
https://patches.dpdk.org/patch/65795/

That one is also waiting because I remember it was not safe for hotplug.

Instead of a big memset, I am for selectively set pointers to NULL.

Re: [dpdk-dev] [PATCH v3 28/29] ethdev: reset all when releasing a port

[Thomas Monjalon]

29/09/2020 17:50, Ferruh Yigit:
> On 9/29/2020 12:58 PM, Wang, Haiyue wrote:
> > From: Thomas Monjalon <thomas@monjalon.net>
> >> 29/09/2020 12:26, Maxime Coquelin:
> >>> On 9/29/20 1:14 AM, Thomas Monjalon wrote:
> >>>> The function rte_eth_dev_release_port() was resetting partially
> >>>> the struct rte_eth_dev. The drivers were completing it
> >>>> with more pointers set to NULL in the close or remove operations.
> >>>>
> >>>> A full memset is done so most of those assignments become useless.
> >> [...]
> >>> With this patch, I get following segfault at init time with Virtio PMD:
> >>>
> >>>          Program received signal SIGSEGV, Segmentation fault.
> >>>          0x0000000000854c9b in rte_eth_dev_callback_register (port_id=32,
> >>>              event=RTE_ETH_EVENT_UNKNOWN, cb_fn=0x4b24de
> >>> <eth_event_callback>,
> >>>              cb_arg=0x0) at ../lib/librte_ethdev/rte_ethdev.c:4042
> >>>          4042
> >>> TAILQ_INSERT_TAIL(&(dev->link_intr_cbs),
> >>
> >> Yes this is because after closing a port, everything is resetted,
> >> including .link_intr_cbs which is set only once in a constructor:
> >> http://git.dpdk.org/dpdk/commit/?id=9ec0b3869d8
> >>
> >> I can change this patch to selectively set pointers to NULL.
> >>
> >> Or if we prefer a big memset 0, we need to rework how RTE_ETH_ALL
> >> is managed to register a callback for any event.
> >> Instead of setting the callback for all ports, we could have
> >> a special catch-call callback list which is called for all events.
> >> This way we could revert initializing .link_intr_cbs in eth_dev_get().
> >>
> >>
> > 
> > Move 'struct rte_eth_dev_cb_list link_intr_cbs' to the end of 'struct rte_eth_dev',
> > and starting from link_intr_cbs, these members will be kept after closed ? :-)
> > 
> > memset(eth_dev, 0, offsetof(struct rte_eth_dev, link_intr_cbs));
> > 
> 
> This is similar version of a previous patch:
> https://patches.dpdk.org/patch/65795/

I forgot this patch :)

> That one is also waiting because I remember it was not safe for hotplug.

I don't think there is a safety issue.
It makes harder to play with dangling pointers, which is good.
Note: the .device pointer was already resetted by
rte_eth_dev_pci_generic_remove(), and generalized in patch 1.

> Instead of a big memset, I am for selectively set pointers to NULL.

I will prepare the patch with selective resets.

Re: [dpdk-dev] [PATCH v3 28/29] ethdev: reset all when releasing a port

[Ferruh Yigit]

On 9/29/2020 5:02 PM, Thomas Monjalon wrote:
> 29/09/2020 17:50, Ferruh Yigit:
>> On 9/29/2020 12:58 PM, Wang, Haiyue wrote:
>>> From: Thomas Monjalon <thomas@monjalon.net>
>>>> 29/09/2020 12:26, Maxime Coquelin:
>>>>> On 9/29/20 1:14 AM, Thomas Monjalon wrote:
>>>>>> The function rte_eth_dev_release_port() was resetting partially
>>>>>> the struct rte_eth_dev. The drivers were completing it
>>>>>> with more pointers set to NULL in the close or remove operations.
>>>>>>
>>>>>> A full memset is done so most of those assignments become useless.
>>>> [...]
>>>>> With this patch, I get following segfault at init time with Virtio PMD:
>>>>>
>>>>>           Program received signal SIGSEGV, Segmentation fault.
>>>>>           0x0000000000854c9b in rte_eth_dev_callback_register (port_id=32,
>>>>>               event=RTE_ETH_EVENT_UNKNOWN, cb_fn=0x4b24de
>>>>> <eth_event_callback>,
>>>>>               cb_arg=0x0) at ../lib/librte_ethdev/rte_ethdev.c:4042
>>>>>           4042
>>>>> TAILQ_INSERT_TAIL(&(dev->link_intr_cbs),
>>>>
>>>> Yes this is because after closing a port, everything is resetted,
>>>> including .link_intr_cbs which is set only once in a constructor:
>>>> http://git.dpdk.org/dpdk/commit/?id=9ec0b3869d8
>>>>
>>>> I can change this patch to selectively set pointers to NULL.
>>>>
>>>> Or if we prefer a big memset 0, we need to rework how RTE_ETH_ALL
>>>> is managed to register a callback for any event.
>>>> Instead of setting the callback for all ports, we could have
>>>> a special catch-call callback list which is called for all events.
>>>> This way we could revert initializing .link_intr_cbs in eth_dev_get().
>>>>
>>>>
>>>
>>> Move 'struct rte_eth_dev_cb_list link_intr_cbs' to the end of 'struct rte_eth_dev',
>>> and starting from link_intr_cbs, these members will be kept after closed ? :-)
>>>
>>> memset(eth_dev, 0, offsetof(struct rte_eth_dev, link_intr_cbs));
>>>
>>
>> This is similar version of a previous patch:
>> https://patches.dpdk.org/patch/65795/
> 
> I forgot this patch :)
> 
>> That one is also waiting because I remember it was not safe for hotplug.
> 
> I don't think there is a safety issue.
> It makes harder to play with dangling pointers, which is good.
> Note: the .device pointer was already resetted by
> rte_eth_dev_pci_generic_remove(), and generalized in patch 1.
> 

As far as I remember, resetting '.device' in the 'close()' was causing some 
issues on hot remove. Yes patch one already does it, I will test the hot remove 
after close with it.

>> Instead of a big memset, I am for selectively set pointers to NULL.
> 
> I will prepare the patch with selective resets.
> 
> 

Re: [dpdk-dev] [PATCH v3 28/29] ethdev: reset all when releasing a port

[Ferruh Yigit]

On 9/29/2020 5:35 PM, Ferruh Yigit wrote:
> On 9/29/2020 5:02 PM, Thomas Monjalon wrote:
>> 29/09/2020 17:50, Ferruh Yigit:
>>> On 9/29/2020 12:58 PM, Wang, Haiyue wrote:
>>>> From: Thomas Monjalon <thomas@monjalon.net>
>>>>> 29/09/2020 12:26, Maxime Coquelin:
>>>>>> On 9/29/20 1:14 AM, Thomas Monjalon wrote:
>>>>>>> The function rte_eth_dev_release_port() was resetting partially
>>>>>>> the struct rte_eth_dev. The drivers were completing it
>>>>>>> with more pointers set to NULL in the close or remove operations.
>>>>>>>
>>>>>>> A full memset is done so most of those assignments become useless.
>>>>> [...]
>>>>>> With this patch, I get following segfault at init time with Virtio PMD:
>>>>>>
>>>>>>           Program received signal SIGSEGV, Segmentation fault.
>>>>>>           0x0000000000854c9b in rte_eth_dev_callback_register (port_id=32,
>>>>>>               event=RTE_ETH_EVENT_UNKNOWN, cb_fn=0x4b24de
>>>>>> <eth_event_callback>,
>>>>>>               cb_arg=0x0) at ../lib/librte_ethdev/rte_ethdev.c:4042
>>>>>>           4042
>>>>>> TAILQ_INSERT_TAIL(&(dev->link_intr_cbs),
>>>>>
>>>>> Yes this is because after closing a port, everything is resetted,
>>>>> including .link_intr_cbs which is set only once in a constructor:
>>>>> http://git.dpdk.org/dpdk/commit/?id=9ec0b3869d8
>>>>>
>>>>> I can change this patch to selectively set pointers to NULL.
>>>>>
>>>>> Or if we prefer a big memset 0, we need to rework how RTE_ETH_ALL
>>>>> is managed to register a callback for any event.
>>>>> Instead of setting the callback for all ports, we could have
>>>>> a special catch-call callback list which is called for all events.
>>>>> This way we could revert initializing .link_intr_cbs in eth_dev_get().
>>>>>
>>>>>
>>>>
>>>> Move 'struct rte_eth_dev_cb_list link_intr_cbs' to the end of 'struct 
>>>> rte_eth_dev',
>>>> and starting from link_intr_cbs, these members will be kept after closed ? :-)
>>>>
>>>> memset(eth_dev, 0, offsetof(struct rte_eth_dev, link_intr_cbs));
>>>>
>>>
>>> This is similar version of a previous patch:
>>> https://patches.dpdk.org/patch/65795/
>>
>> I forgot this patch :)
>>
>>> That one is also waiting because I remember it was not safe for hotplug.
>>
>> I don't think there is a safety issue.
>> It makes harder to play with dangling pointers, which is good.
>> Note: the .device pointer was already resetted by
>> rte_eth_dev_pci_generic_remove(), and generalized in patch 1.
>>
> 
> As far as I remember, resetting '.device' in the 'close()' was causing some 
> issues on hot remove. Yes patch one already does it, I will test the hot remove 
> after close with it.
>

I didn't get any error with 1/29, will proceed with the patchset.

>>> Instead of a big memset, I am for selectively set pointers to NULL.
>>
>> I will prepare the patch with selective resets.
>>
>>
> 

Re: [dpdk-dev] [PATCH v3 28/29] ethdev: reset all when releasing a port

[Andrew Rybchenko]

On 9/29/20 2:14 AM, Thomas Monjalon wrote:
> The function rte_eth_dev_release_port() was resetting partially
> the struct rte_eth_dev. The drivers were completing it
> with more pointers set to NULL in the close or remove operations.
> 
> A full memset is done so most of those assignments become useless.
> 
> Signed-off-by: Thomas Monjalon <thomas@monjalon.net>

Reviewed-by: Andrew Rybchenko <arybchenko@solarflare.com>

[dpdk-dev] [PATCH v3 29/29] ethdev: allow close function to return an error

[Thomas Monjalon]

The API function rte_eth_dev_close() was returning void.
The return type is changed to int for notifying of errors.

If an error happens during a close operation,
the status of the port is undefined,
a maximum of resources having been freed.

Signed-off-by: Thomas Monjalon <thomas@monjalon.net>
Reviewed-by: Liron Himi <lironh@marvell.com>
---
 doc/guides/rel_notes/deprecation.rst    |  1 -
 doc/guides/rel_notes/release_20_11.rst  |  4 +++-
 drivers/net/cxgbe/cxgbe_ethdev.c        |  5 +++--
 drivers/net/cxgbe/cxgbevf_ethdev.c      |  5 +++--
 drivers/net/failsafe/failsafe_ether.c   |  6 +++++-
 drivers/net/failsafe/failsafe_ops.c     |  5 ++++-
 drivers/net/memif/rte_eth_memif.c       |  4 +---
 drivers/net/mlx5/mlx5.c                 |  7 ++++---
 drivers/net/mvneta/mvneta_ethdev.c      |  5 +++--
 drivers/net/mvpp2/mrvl_ethdev.c         |  5 +++--
 drivers/net/netvsc/hn_ethdev.c          |  6 ++++--
 drivers/net/netvsc/hn_var.h             |  2 +-
 drivers/net/netvsc/hn_vf.c              |  7 +++++--
 drivers/net/virtio/virtio_user_ethdev.c |  4 +---
 lib/librte_ethdev/rte_ethdev.c          | 13 ++++++++-----
 lib/librte_ethdev/rte_ethdev.h          |  5 ++++-
 16 files changed, 52 insertions(+), 32 deletions(-)

diff --git a/doc/guides/rel_notes/deprecation.rst b/doc/guides/rel_notes/deprecation.rst
index d07b4eeb47..1a61bd671a 100644
--- a/doc/guides/rel_notes/deprecation.rst
+++ b/doc/guides/rel_notes/deprecation.rst
@@ -135,7 +135,6 @@ Deprecation Notices
   invalid port ID, unsupported operation, failed operation):
 
   - ``rte_eth_dev_stop``
-  - ``rte_eth_dev_close``
 
 * ethdev: New offload flags ``DEV_RX_OFFLOAD_FLOW_MARK`` will be added in 19.11.
   This will allow application to enable or disable PMDs from updating
diff --git a/doc/guides/rel_notes/release_20_11.rst b/doc/guides/rel_notes/release_20_11.rst
index 4eb3224a76..f98dc80b6c 100644
--- a/doc/guides/rel_notes/release_20_11.rst
+++ b/doc/guides/rel_notes/release_20_11.rst
@@ -156,7 +156,9 @@ API Changes
 
 * ethdev: ``rte_eth_rx_descriptor_done()`` API has been deprecated.
 
-* Renamed internal ethdev APIs:
+* ethdev: Added ``int`` return type to ``rte_eth_dev_close()``.
+
+* ethdev: Renamed internal functions:
 
   * ``_rte_eth_dev_callback_process()`` -> ``rte_eth_dev_callback_process()``
   * ``_rte_eth_dev_reset`` -> ``rte_eth_dev_internal_reset()``
diff --git a/drivers/net/cxgbe/cxgbe_ethdev.c b/drivers/net/cxgbe/cxgbe_ethdev.c
index 16beb2d435..fe488231a7 100644
--- a/drivers/net/cxgbe/cxgbe_ethdev.c
+++ b/drivers/net/cxgbe/cxgbe_ethdev.c
@@ -1296,12 +1296,13 @@ static int eth_cxgbe_dev_uninit(struct rte_eth_dev *eth_dev)
 {
 	struct rte_pci_device *pci_dev = RTE_ETH_DEV_TO_PCI(eth_dev);
 	uint16_t port_id;
+	int err = 0;
 
 	/* Free up other ports and all resources */
 	RTE_ETH_FOREACH_DEV_OF(port_id, &pci_dev->device)
-		rte_eth_dev_close(port_id);
+		err |= rte_eth_dev_close(port_id);
 
-	return 0;
+	return err == 0 ? 0 : -EIO;
 }
 
 static int eth_cxgbe_pci_probe(struct rte_pci_driver *pci_drv __rte_unused,
diff --git a/drivers/net/cxgbe/cxgbevf_ethdev.c b/drivers/net/cxgbe/cxgbevf_ethdev.c
index 947fcdd406..c2918f5356 100644
--- a/drivers/net/cxgbe/cxgbevf_ethdev.c
+++ b/drivers/net/cxgbe/cxgbevf_ethdev.c
@@ -183,12 +183,13 @@ static int eth_cxgbevf_dev_uninit(struct rte_eth_dev *eth_dev)
 {
 	struct rte_pci_device *pci_dev = RTE_ETH_DEV_TO_PCI(eth_dev);
 	uint16_t port_id;
+	int err = 0;
 
 	/* Free up other ports and all resources */
 	RTE_ETH_FOREACH_DEV_OF(port_id, &pci_dev->device)
-		rte_eth_dev_close(port_id);
+		err |= rte_eth_dev_close(port_id);
 
-	return 0;
+	return err == 0 ? 0 : -EIO;
 }
 
 static int eth_cxgbevf_pci_probe(struct rte_pci_driver *pci_drv __rte_unused,
diff --git a/drivers/net/failsafe/failsafe_ether.c b/drivers/net/failsafe/failsafe_ether.c
index 7c68bbdec0..950d35dac4 100644
--- a/drivers/net/failsafe/failsafe_ether.c
+++ b/drivers/net/failsafe/failsafe_ether.c
@@ -287,7 +287,11 @@ fs_dev_remove(struct sub_device *sdev)
 		/* fallthrough */
 	case DEV_ACTIVE:
 		failsafe_eth_dev_unregister_callbacks(sdev);
-		rte_eth_dev_close(PORT_ID(sdev));
+		ret = rte_eth_dev_close(PORT_ID(sdev));
+		if (ret < 0) {
+			ERROR("Port close failed for sub-device %u",
+			      PORT_ID(sdev));
+		}
 		sdev->state = DEV_PROBED;
 		/* fallthrough */
 	case DEV_PROBED:
diff --git a/drivers/net/failsafe/failsafe_ops.c b/drivers/net/failsafe/failsafe_ops.c
index 0ce7dfc8a6..79b24ec996 100644
--- a/drivers/net/failsafe/failsafe_ops.c
+++ b/drivers/net/failsafe/failsafe_ops.c
@@ -648,7 +648,10 @@ failsafe_eth_dev_close(struct rte_eth_dev *dev)
 	FOREACH_SUBDEV_STATE(sdev, i, dev, DEV_ACTIVE) {
 		DEBUG("Closing sub_device %d", i);
 		failsafe_eth_dev_unregister_callbacks(sdev);
-		rte_eth_dev_close(PORT_ID(sdev));
+		ret = rte_eth_dev_close(PORT_ID(sdev));
+		if (ret)
+			ERROR("Error while closing sub-device %u",
+					PORT_ID(sdev));
 		sdev->state = DEV_ACTIVE - 1;
 	}
 	rte_eth_dev_callback_unregister(RTE_ETH_ALL, RTE_ETH_EVENT_NEW,
diff --git a/drivers/net/memif/rte_eth_memif.c b/drivers/net/memif/rte_eth_memif.c
index ff8a58081f..33bf5c68a3 100644
--- a/drivers/net/memif/rte_eth_memif.c
+++ b/drivers/net/memif/rte_eth_memif.c
@@ -1798,9 +1798,7 @@ rte_pmd_memif_remove(struct rte_vdev_device *vdev)
 	if (eth_dev == NULL)
 		return 0;
 
-	rte_eth_dev_close(eth_dev->data->port_id);
-
-	return 0;
+	return rte_eth_dev_close(eth_dev->data->port_id);
 }
 
 static struct rte_vdev_driver pmd_memif_drv = {
diff --git a/drivers/net/mlx5/mlx5.c b/drivers/net/mlx5/mlx5.c
index 01ead6e6af..2e2f0b274c 100644
--- a/drivers/net/mlx5/mlx5.c
+++ b/drivers/net/mlx5/mlx5.c
@@ -2010,6 +2010,7 @@ static int
 mlx5_pci_remove(struct rte_pci_device *pci_dev)
 {
 	uint16_t port_id;
+	int ret = 0;
 
 	RTE_ETH_FOREACH_DEV_OF(port_id, &pci_dev->device) {
 		/*
@@ -2017,11 +2018,11 @@ mlx5_pci_remove(struct rte_pci_device *pci_dev)
 		 * call the close function explicitly for secondary process.
 		 */
 		if (rte_eal_process_type() == RTE_PROC_SECONDARY)
-			mlx5_dev_close(&rte_eth_devices[port_id]);
+			ret |= mlx5_dev_close(&rte_eth_devices[port_id]);
 		else
-			rte_eth_dev_close(port_id);
+			ret |= rte_eth_dev_close(port_id);
 	}
-	return 0;
+	return ret == 0 ? 0 : -EIO;
 }
 
 static const struct rte_pci_id mlx5_pci_id_map[] = {
diff --git a/drivers/net/mvneta/mvneta_ethdev.c b/drivers/net/mvneta/mvneta_ethdev.c
index 607771149a..13d4b6af6b 100644
--- a/drivers/net/mvneta/mvneta_ethdev.c
+++ b/drivers/net/mvneta/mvneta_ethdev.c
@@ -964,14 +964,15 @@ static int
 rte_pmd_mvneta_remove(struct rte_vdev_device *vdev)
 {
 	uint16_t port_id;
+	int ret = 0;
 
 	RTE_ETH_FOREACH_DEV(port_id) {
 		if (rte_eth_devices[port_id].device != &vdev->device)
 			continue;
-		rte_eth_dev_close(port_id);
+		ret = rte_eth_dev_close(port_id);
 	}
 
-	return 0;
+	return ret == 0 ? 0 : -EIO;
 }
 
 static struct rte_vdev_driver pmd_mvneta_drv = {
diff --git a/drivers/net/mvpp2/mrvl_ethdev.c b/drivers/net/mvpp2/mrvl_ethdev.c
index a230a96840..acc8c70a95 100644
--- a/drivers/net/mvpp2/mrvl_ethdev.c
+++ b/drivers/net/mvpp2/mrvl_ethdev.c
@@ -3022,14 +3022,15 @@ static int
 rte_pmd_mrvl_remove(struct rte_vdev_device *vdev)
 {
 	uint16_t port_id;
+	int ret = 0;
 
 	RTE_ETH_FOREACH_DEV(port_id) {
 		if (rte_eth_devices[port_id].device != &vdev->device)
 			continue;
-		rte_eth_dev_close(port_id);
+		ret = rte_eth_dev_close(port_id);
 	}
 
-	return 0;
+	return ret == 0 ? 0 : -EIO;
 }
 
 static struct rte_vdev_driver pmd_mrvl_drv = {
diff --git a/drivers/net/netvsc/hn_ethdev.c b/drivers/net/netvsc/hn_ethdev.c
index 9af64821a1..3fa39dbebc 100644
--- a/drivers/net/netvsc/hn_ethdev.c
+++ b/drivers/net/netvsc/hn_ethdev.c
@@ -841,14 +841,16 @@ hn_dev_stop(struct rte_eth_dev *dev)
 static int
 hn_dev_close(struct rte_eth_dev *dev)
 {
+	int ret;
+
 	PMD_INIT_FUNC_TRACE();
 	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
 		return 0;
 
-	hn_vf_close(dev);
+	ret = hn_vf_close(dev);
 	hn_dev_free_queues(dev);
 
-	return 0;
+	return ret;
 }
 
 static const struct eth_dev_ops hn_eth_dev_ops = {
diff --git a/drivers/net/netvsc/hn_var.h b/drivers/net/netvsc/hn_var.h
index 4b63f87607..74f30669ac 100644
--- a/drivers/net/netvsc/hn_var.h
+++ b/drivers/net/netvsc/hn_var.h
@@ -217,7 +217,7 @@ const uint32_t *hn_vf_supported_ptypes(struct rte_eth_dev *dev);
 int	hn_vf_start(struct rte_eth_dev *dev);
 void	hn_vf_reset(struct rte_eth_dev *dev);
 void	hn_vf_stop(struct rte_eth_dev *dev);
-void	hn_vf_close(struct rte_eth_dev *dev);
+int	hn_vf_close(struct rte_eth_dev *dev);
 
 int	hn_vf_allmulticast_enable(struct rte_eth_dev *dev);
 int	hn_vf_allmulticast_disable(struct rte_eth_dev *dev);
diff --git a/drivers/net/netvsc/hn_vf.c b/drivers/net/netvsc/hn_vf.c
index f5f15c0462..d29eee7627 100644
--- a/drivers/net/netvsc/hn_vf.c
+++ b/drivers/net/netvsc/hn_vf.c
@@ -316,18 +316,21 @@ void hn_vf_reset(struct rte_eth_dev *dev)
 	VF_ETHDEV_FUNC(dev, rte_eth_dev_reset);
 }
 
-void hn_vf_close(struct rte_eth_dev *dev)
+int hn_vf_close(struct rte_eth_dev *dev)
 {
 	struct hn_data *hv = dev->data->dev_private;
 	uint16_t vf_port;
+	int ret = 0;
 
 	rte_rwlock_read_lock(&hv->vf_lock);
 	vf_port = hv->vf_port;
 	if (vf_port != HN_INVALID_PORT)
-		rte_eth_dev_close(vf_port);
+		ret = rte_eth_dev_close(vf_port);
 
 	hv->vf_port = HN_INVALID_PORT;
 	rte_rwlock_read_unlock(&hv->vf_lock);
+
+	return ret;
 }
 
 int hn_vf_stats_reset(struct rte_eth_dev *dev)
diff --git a/drivers/net/virtio/virtio_user_ethdev.c b/drivers/net/virtio/virtio_user_ethdev.c
index 87f6cb6950..865f73807f 100644
--- a/drivers/net/virtio/virtio_user_ethdev.c
+++ b/drivers/net/virtio/virtio_user_ethdev.c
@@ -813,9 +813,7 @@ virtio_user_pmd_remove(struct rte_vdev_device *vdev)
 		return rte_eth_dev_release_port(eth_dev);
 
 	/* make sure the device is stopped, queues freed */
-	rte_eth_dev_close(eth_dev->data->port_id);
-
-	return 0;
+	return rte_eth_dev_close(eth_dev->data->port_id);
 }
 
 static struct rte_vdev_driver virtio_user_driver = {
diff --git a/lib/librte_ethdev/rte_ethdev.c b/lib/librte_ethdev/rte_ethdev.c
index e68943b461..1459ac1ad5 100644
--- a/lib/librte_ethdev/rte_ethdev.c
+++ b/lib/librte_ethdev/rte_ethdev.c
@@ -1704,19 +1704,22 @@ rte_eth_dev_set_link_down(uint16_t port_id)
 	return eth_err(port_id, (*dev->dev_ops->dev_set_link_down)(dev));
 }
 
-void
+int
 rte_eth_dev_close(uint16_t port_id)
 {
 	struct rte_eth_dev *dev;
+	int ret;
 
-	RTE_ETH_VALID_PORTID_OR_RET(port_id);
+	RTE_ETH_VALID_PORTID_OR_ERR_RET(port_id, -EINVAL);
 	dev = &rte_eth_devices[port_id];
 
-	RTE_FUNC_PTR_OR_RET(*dev->dev_ops->dev_close);
-	(*dev->dev_ops->dev_close)(dev);
+	RTE_FUNC_PTR_OR_ERR_RET(*dev->dev_ops->dev_close, -ENOTSUP);
+	ret = (*dev->dev_ops->dev_close)(dev);
 
 	rte_ethdev_trace_close(port_id);
-	rte_eth_dev_release_port(dev);
+	ret = rte_eth_dev_release_port(dev);
+
+	return ret;
 }
 
 int
diff --git a/lib/librte_ethdev/rte_ethdev.h b/lib/librte_ethdev/rte_ethdev.h
index 24d898ae89..4380d8ecf6 100644
--- a/lib/librte_ethdev/rte_ethdev.h
+++ b/lib/librte_ethdev/rte_ethdev.h
@@ -2281,8 +2281,11 @@ int rte_eth_dev_set_link_down(uint16_t port_id);
  *
  * @param port_id
  *   The port identifier of the Ethernet device.
+ * @return
+ *   - Zero if the port is closed successfully.
+ *   - Negative if something went wrong.
  */
-void rte_eth_dev_close(uint16_t port_id);
+int rte_eth_dev_close(uint16_t port_id);
 
 /**
  * Reset a Ethernet device and keep its port id.
-- 
2.28.0

Re: [dpdk-dev] [PATCH v3 29/29] ethdev: allow close function to return an error

[Andrew Rybchenko]

On 9/29/20 2:14 AM, Thomas Monjalon wrote:
> The API function rte_eth_dev_close() was returning void.
> The return type is changed to int for notifying of errors.
> 
> If an error happens during a close operation,
> the status of the port is undefined,
> a maximum of resources having been freed.
> 
> Signed-off-by: Thomas Monjalon <thomas@monjalon.net>
> Reviewed-by: Liron Himi <lironh@marvell.com>

[snip]

> diff --git a/doc/guides/rel_notes/deprecation.rst b/doc/guides/rel_notes/deprecation.rst
> index d07b4eeb47..1a61bd671a 100644
> --- a/doc/guides/rel_notes/deprecation.rst
> +++ b/doc/guides/rel_notes/deprecation.rst
> @@ -135,7 +135,6 @@ Deprecation Notices
>    invalid port ID, unsupported operation, failed operation):
>  
>    - ``rte_eth_dev_stop``
> -  - ``rte_eth_dev_close``

Many thanks for continuing my unfinished job.

>  
>  * ethdev: New offload flags ``DEV_RX_OFFLOAD_FLOW_MARK`` will be added in 19.11.
>    This will allow application to enable or disable PMDs from updating

[snip]

> diff --git a/drivers/net/mvneta/mvneta_ethdev.c b/drivers/net/mvneta/mvneta_ethdev.c
> index 607771149a..13d4b6af6b 100644
> --- a/drivers/net/mvneta/mvneta_ethdev.c
> +++ b/drivers/net/mvneta/mvneta_ethdev.c
> @@ -964,14 +964,15 @@ static int
>  rte_pmd_mvneta_remove(struct rte_vdev_device *vdev)
>  {
>  	uint16_t port_id;
> +	int ret = 0;
>  
>  	RTE_ETH_FOREACH_DEV(port_id) {
>  		if (rte_eth_devices[port_id].device != &vdev->device)
>  			continue;
> -		rte_eth_dev_close(port_id);
> +		ret = rte_eth_dev_close(port_id);

I guess |= should be used here as well (similar as above).

>  	}
>  
> -	return 0;
> +	return ret == 0 ? 0 : -EIO;
>  }
>  
>  static struct rte_vdev_driver pmd_mvneta_drv = {
> diff --git a/drivers/net/mvpp2/mrvl_ethdev.c b/drivers/net/mvpp2/mrvl_ethdev.c
> index a230a96840..acc8c70a95 100644
> --- a/drivers/net/mvpp2/mrvl_ethdev.c
> +++ b/drivers/net/mvpp2/mrvl_ethdev.c
> @@ -3022,14 +3022,15 @@ static int
>  rte_pmd_mrvl_remove(struct rte_vdev_device *vdev)
>  {
>  	uint16_t port_id;
> +	int ret = 0;
>  
>  	RTE_ETH_FOREACH_DEV(port_id) {
>  		if (rte_eth_devices[port_id].device != &vdev->device)
>  			continue;
> -		rte_eth_dev_close(port_id);
> +		ret = rte_eth_dev_close(port_id);

I guess |= should be used here as well (similar as above).

>  	}
>  
> -	return 0;
> +	return ret == 0 ? 0 : -EIO;
>  }
>  
>  static struct rte_vdev_driver pmd_mrvl_drv = {

[snip]

> diff --git a/lib/librte_ethdev/rte_ethdev.c b/lib/librte_ethdev/rte_ethdev.c
> index e68943b461..1459ac1ad5 100644
> --- a/lib/librte_ethdev/rte_ethdev.c
> +++ b/lib/librte_ethdev/rte_ethdev.c
> @@ -1704,19 +1704,22 @@ rte_eth_dev_set_link_down(uint16_t port_id)
>  	return eth_err(port_id, (*dev->dev_ops->dev_set_link_down)(dev));
>  }
>  
> -void
> +int
>  rte_eth_dev_close(uint16_t port_id)
>  {
>  	struct rte_eth_dev *dev;
> +	int ret;
>  
> -	RTE_ETH_VALID_PORTID_OR_RET(port_id);
> +	RTE_ETH_VALID_PORTID_OR_ERR_RET(port_id, -EINVAL);
>  	dev = &rte_eth_devices[port_id];
>  
> -	RTE_FUNC_PTR_OR_RET(*dev->dev_ops->dev_close);
> -	(*dev->dev_ops->dev_close)(dev);
> +	RTE_FUNC_PTR_OR_ERR_RET(*dev->dev_ops->dev_close, -ENOTSUP);
> +	ret = (*dev->dev_ops->dev_close)(dev);
>  
>  	rte_ethdev_trace_close(port_id);
> -	rte_eth_dev_release_port(dev);
> +	ret = rte_eth_dev_release_port(dev);

It does not look good that it overwrites close return status.

> +
> +	return ret;
>  }
>  
>  int

[snip]

Re: [dpdk-dev] [PATCH v3 29/29] ethdev: allow close function to return an error

[Thomas Monjalon]

29/09/2020 13:05, Andrew Rybchenko:
> On 9/29/20 2:14 AM, Thomas Monjalon wrote:
> > The API function rte_eth_dev_close() was returning void.
> > The return type is changed to int for notifying of errors.
> > 
> > If an error happens during a close operation,
> > the status of the port is undefined,
> > a maximum of resources having been freed.
[...]
> > @@ -135,7 +135,6 @@ Deprecation Notices
> >    invalid port ID, unsupported operation, failed operation):
> >  
> >    - ``rte_eth_dev_stop``
> > -  - ``rte_eth_dev_close``
> 
> Many thanks for continuing my unfinished job.

It is not finished.
Would you take rte_eth_dev_stop?

> >  rte_pmd_mvneta_remove(struct rte_vdev_device *vdev)
> >  {
> >  	uint16_t port_id;
> > +	int ret = 0;
> >  
> >  	RTE_ETH_FOREACH_DEV(port_id) {
> >  		if (rte_eth_devices[port_id].device != &vdev->device)
> >  			continue;
> > -		rte_eth_dev_close(port_id);
> > +		ret = rte_eth_dev_close(port_id);
> 
> I guess |= should be used here as well (similar as above).

Yes

> > -void
> > +int
> >  rte_eth_dev_close(uint16_t port_id)
> >  {
> >  	struct rte_eth_dev *dev;
> > +	int ret;
> >  
> > -	RTE_ETH_VALID_PORTID_OR_RET(port_id);
> > +	RTE_ETH_VALID_PORTID_OR_ERR_RET(port_id, -EINVAL);
> >  	dev = &rte_eth_devices[port_id];
> >  
> > -	RTE_FUNC_PTR_OR_RET(*dev->dev_ops->dev_close);
> > -	(*dev->dev_ops->dev_close)(dev);
> > +	RTE_FUNC_PTR_OR_ERR_RET(*dev->dev_ops->dev_close, -ENOTSUP);
> > +	ret = (*dev->dev_ops->dev_close)(dev);
> >  
> >  	rte_ethdev_trace_close(port_id);
> > -	rte_eth_dev_release_port(dev);
> > +	ret = rte_eth_dev_release_port(dev);
> 
> It does not look good that it overwrites close return status.

Indeed, it seems I did not finish this patch,
or maybe I was drunk ;)

What do you think is the best strategy?
I think we must call rte_eth_dev_release_port()
even if dev_close returns an error.
Then which error to return? I would return the first non-zero one.

Re: [dpdk-dev] [PATCH v3 29/29] ethdev: allow close function to return an error

[Andrew Rybchenko]

On 9/29/20 2:47 PM, Thomas Monjalon wrote:
> 29/09/2020 13:05, Andrew Rybchenko:
>> On 9/29/20 2:14 AM, Thomas Monjalon wrote:
>>> The API function rte_eth_dev_close() was returning void.
>>> The return type is changed to int for notifying of errors.
>>>
>>> If an error happens during a close operation,
>>> the status of the port is undefined,
>>> a maximum of resources having been freed.
> [...]
>>> @@ -135,7 +135,6 @@ Deprecation Notices
>>>    invalid port ID, unsupported operation, failed operation):
>>>  
>>>    - ``rte_eth_dev_stop``
>>> -  - ``rte_eth_dev_close``
>>
>> Many thanks for continuing my unfinished job.
> 
> It is not finished.
> Would you take rte_eth_dev_stop?

Sorry, I can't promise that I find time. I'll try.
If anyone else is ready, welcome.

>>> -void
>>> +int
>>>  rte_eth_dev_close(uint16_t port_id)
>>>  {
>>>  	struct rte_eth_dev *dev;
>>> +	int ret;
>>>  
>>> -	RTE_ETH_VALID_PORTID_OR_RET(port_id);
>>> +	RTE_ETH_VALID_PORTID_OR_ERR_RET(port_id, -EINVAL);
>>>  	dev = &rte_eth_devices[port_id];
>>>  
>>> -	RTE_FUNC_PTR_OR_RET(*dev->dev_ops->dev_close);
>>> -	(*dev->dev_ops->dev_close)(dev);
>>> +	RTE_FUNC_PTR_OR_ERR_RET(*dev->dev_ops->dev_close, -ENOTSUP);
>>> +	ret = (*dev->dev_ops->dev_close)(dev);
>>>  
>>>  	rte_ethdev_trace_close(port_id);
>>> -	rte_eth_dev_release_port(dev);
>>> +	ret = rte_eth_dev_release_port(dev);
>>
>> It does not look good that it overwrites close return status.
> 
> Indeed, it seems I did not finish this patch,
> or maybe I was drunk ;)
> 
> What do you think is the best strategy?
> I think we must call rte_eth_dev_release_port()
> even if dev_close returns an error.
> Then which error to return? I would return the first non-zero one.

Yes, exactly. Return the first non-zero error code.

Re: [dpdk-dev] [PATCH v3 00/29] cleanup ethdev close operation

[Stephen Hemminger]

On Tue, 29 Sep 2020 01:14:08 +0200
Thomas Monjalon <thomas@monjalon.net> wrote:

> This is the end of a process started two years ago,
> to have a close which reliably releases an ethdev port
> without the need of removing the device (which can have more ports).
> 
> Unfortunately, some drivers might be broken because did not follow
> the migration recommendations. We cannot wait more,
> this should be merged before the 20.11-rc1 release.
> 
> 
> v3:
>   - integrate NXP patches for dpaa, dpaa2, enetc and pfe
>   - fix crash in rte_eth_dev_release_port() because of early memset 0
> 
> v2 (big thanks to Ferruh for the reviews):
>   - rebase on top of cxgbe updates
>   - integrate updated patches for tap, bnx2x, qede
>   - return 0 in atlantic PMD
>   - fix failsafe mutex free
>   - remove useless check in pcap
>   - fix testpmd port state transition
>   - add primary process check
>   - reset more pointers in rte_eth_dev_release_port()
>   - remove dev_started reset in rte_eth_dev_close()
>   - return int from rte_eth_dev_close()
> 
> Note for future: fix old memory leak when releasing port,
> i.e. free callbacks link_intr_cbs, post_rx_burst_cbs, pre_tx_burst_cbs.
> 
> The last 3 patches are optional changes to ethdev close/release
> which were requested by Ferruh but may take longer to be accepted:
>   - memset 0
>   - keep start state
>   - int return
> 
> 
> Rasesh Mody (2):
>   net/bnx2x: release port upon close
>   net/qede: release port upon close
> 
> Sachin Saxena (4):
>   net/dpaa: release port upon close
>   net/dpaa2: release port upon close
>   net/enetc: release port upon close
>   net/pfe: release port upon close
> 
> Thomas Monjalon (22):
>   ethdev: reset device and interrupt pointers on release
>   ethdev: allow drivers to return error on close
>   net/af_packet: release port upon close
>   net/atlantic: release port upon close
>   net/axgbe: release port upon close
>   net/bonding: release port upon close
>   net/failsafe: release port upon close
>   net/mlx4: release port upon close
>   net/null: release port upon close
>   net/octeontx: release port upon close
>   net/pcap: release port upon close
>   net/ring: release port upon close
>   net/softnic: release port upon close
>   ethdev: remove old close behaviour
>   drivers/net: accept removing device without any port
>   drivers/net: check process type in close operation
>   drivers/net: remove redundant MAC addresses freeing
>   app/testpmd: reset port status on close notification
>   app/testpmd: align behaviour of multi-port detach
>   ethdev: remove forcing stopped state upon close
>   ethdev: reset all when releasing a port
>   ethdev: allow close function to return an error
> 
> Yunjian Wang (1):
>   net/tap: release port upon close
> 
>  MAINTAINERS                               |   6 +-
>  app/test-pmd/config.c                     |   7 +-
>  app/test-pmd/testpmd.c                    |  53 +++------
>  app/test/virtual_pmd.c                    |   6 +-
>  doc/guides/rel_notes/deprecation.rst      |   7 --
>  doc/guides/rel_notes/release_20_11.rst    |   4 +-
>  drivers/net/af_packet/rte_eth_af_packet.c |  58 +++++-----
>  drivers/net/af_xdp/rte_eth_af_xdp.c       |   9 +-
>  drivers/net/ark/ark_ethdev.c              |  17 ++-
>  drivers/net/atlantic/atl_ethdev.c         |  62 ++++-------
>  drivers/net/avp/avp_ethdev.c              |  10 +-
>  drivers/net/axgbe/axgbe_ethdev.c          |  17 +--
>  drivers/net/bnx2x/bnx2x_ethdev.c          |  16 ++-
>  drivers/net/bnxt/bnxt_ethdev.c            |  16 ++-
>  drivers/net/bnxt/bnxt_reps.c              |  11 +-
>  drivers/net/bnxt/bnxt_reps.h              |   2 +-
>  drivers/net/bonding/eth_bond_private.h    |   2 +-
>  drivers/net/bonding/rte_eth_bond_pmd.c    |  33 +++---
>  drivers/net/cxgbe/cxgbe_ethdev.c          |  18 ++--
>  drivers/net/cxgbe/cxgbe_main.c            |   2 -
>  drivers/net/cxgbe/cxgbe_pfvf.h            |   2 +-
>  drivers/net/cxgbe/cxgbevf_ethdev.c        |   5 +-
>  drivers/net/cxgbe/cxgbevf_main.c          |   2 -
>  drivers/net/dpaa/dpaa_ethdev.c            | 124 ++++++++++------------
>  drivers/net/dpaa2/dpaa2_ethdev.c          |  96 +++++++----------
>  drivers/net/e1000/em_ethdev.c             |  18 ++--
>  drivers/net/e1000/igb_ethdev.c            |  36 +++----
>  drivers/net/ena/ena_ethdev.c              |  20 ++--
>  drivers/net/enetc/enetc_ethdev.c          |  17 +--
>  drivers/net/enic/enic_ethdev.c            |   9 +-
>  drivers/net/enic/enic_vf_representor.c    |   8 +-
>  drivers/net/failsafe/failsafe.c           |  24 +----
>  drivers/net/failsafe/failsafe_ether.c     |   6 +-
>  drivers/net/failsafe/failsafe_ops.c       |  67 ++++++++----
>  drivers/net/failsafe/failsafe_private.h   |   1 +
>  drivers/net/fm10k/fm10k_ethdev.c          |  22 +---
>  drivers/net/hinic/hinic_pmd_ethdev.c      |  22 ++--
>  drivers/net/hns3/hns3_ethdev.c            |  15 +--
>  drivers/net/hns3/hns3_ethdev_vf.c         |  16 +--
>  drivers/net/i40e/i40e_ethdev.c            |  16 +--
>  drivers/net/i40e/i40e_ethdev_vf.c         |  17 ++-
>  drivers/net/iavf/iavf_ethdev.c            |  18 ++--
>  drivers/net/ice/ice_dcf_ethdev.c          |  12 +--
>  drivers/net/ice/ice_ethdev.c              |  21 ++--
>  drivers/net/igc/igc_ethdev.c              |  17 ++-
>  drivers/net/ionic/ionic_ethdev.c          |  17 ++-
>  drivers/net/ipn3ke/ipn3ke_ethdev.c        |   6 +-
>  drivers/net/ipn3ke/ipn3ke_representor.c   |   7 +-
>  drivers/net/ixgbe/ixgbe_ethdev.c          |  33 ++----
>  drivers/net/kni/rte_eth_kni.c             |  25 ++---
>  drivers/net/liquidio/lio_ethdev.c         |  11 +-
>  drivers/net/memif/rte_eth_memif.c         |  11 +-
>  drivers/net/mlx4/mlx4.c                   |   7 +-
>  drivers/net/mlx5/linux/mlx5_os.c          |   2 -
>  drivers/net/mlx5/mlx5.c                   |  16 +--
>  drivers/net/mlx5/mlx5.h                   |   2 +-
>  drivers/net/mvneta/mvneta_ethdev.c        |  15 +--
>  drivers/net/mvpp2/mrvl_ethdev.c           |  15 +--
>  drivers/net/netvsc/hn_ethdev.c            |  19 ++--
>  drivers/net/netvsc/hn_var.h               |   2 +-
>  drivers/net/netvsc/hn_vf.c                |   7 +-
>  drivers/net/nfb/nfb_ethdev.c              |  11 +-
>  drivers/net/nfp/nfp_net.c                 |  11 +-
>  drivers/net/null/rte_eth_null.c           |  26 +++--
>  drivers/net/octeontx/octeontx_ethdev.c    |  17 ++-
>  drivers/net/octeontx2/otx2_ethdev.c       |   8 +-
>  drivers/net/pcap/rte_eth_pcap.c           |  31 +++---
>  drivers/net/pfe/pfe_ethdev.c              |  51 ++++-----
>  drivers/net/qede/qede_ethdev.c            |  20 ++--
>  drivers/net/ring/rte_eth_ring.c           |  53 +++++----
>  drivers/net/sfc/sfc_ethdev.c              |  35 +++---
>  drivers/net/softnic/rte_eth_softnic.c     |  67 ++++++------
>  drivers/net/szedata2/rte_eth_szedata2.c   |  23 ++--
>  drivers/net/tap/rte_eth_tap.c             |  52 +++++----
>  drivers/net/thunderx/nicvf_ethdev.c       |  11 +-
>  drivers/net/vhost/rte_eth_vhost.c         |  15 +--
>  drivers/net/virtio/virtio_ethdev.c        |  17 ++-
>  drivers/net/virtio/virtio_user_ethdev.c   |   4 +-
>  drivers/net/vmxnet3/vmxnet3_ethdev.c      |  16 ++-
>  lib/librte_ethdev/rte_ethdev.c            |  32 ++----
>  lib/librte_ethdev/rte_ethdev.h            |  13 +--
>  lib/librte_ethdev/rte_ethdev_driver.h     |   2 +-
>  lib/librte_ethdev/rte_ethdev_pci.h        |  14 +--
>  83 files changed, 763 insertions(+), 928 deletions(-)
> 

Looks good, thanks for cleaning up the drivers. It is great to make sure all
the devices work right.

Acked-by: Stephen Hemminger <stephen@networkplumber.org>

Re: [dpdk-dev] [PATCH v3 00/29] cleanup ethdev close operation

[Ferruh Yigit]

On 9/29/2020 12:14 AM, Thomas Monjalon wrote:
> This is the end of a process started two years ago,
> to have a close which reliably releases an ethdev port
> without the need of removing the device (which can have more ports).
> 
> Unfortunately, some drivers might be broken because did not follow
> the migration recommendations. We cannot wait more,
> this should be merged before the 20.11-rc1 release.
> 
> 
> v3:
>    - integrate NXP patches for dpaa, dpaa2, enetc and pfe
>    - fix crash in rte_eth_dev_release_port() because of early memset 0
> 
> v2 (big thanks to Ferruh for the reviews):
>    - rebase on top of cxgbe updates
>    - integrate updated patches for tap, bnx2x, qede
>    - return 0 in atlantic PMD
>    - fix failsafe mutex free
>    - remove useless check in pcap
>    - fix testpmd port state transition
>    - add primary process check
>    - reset more pointers in rte_eth_dev_release_port()
>    - remove dev_started reset in rte_eth_dev_close()
>    - return int from rte_eth_dev_close()
> 
> Note for future: fix old memory leak when releasing port,
> i.e. free callbacks link_intr_cbs, post_rx_burst_cbs, pre_tx_burst_cbs.
> 
> The last 3 patches are optional changes to ethdev close/release
> which were requested by Ferruh but may take longer to be accepted:
>    - memset 0
>    - keep start state
>    - int return
> 
> 
> Rasesh Mody (2):
>    net/bnx2x: release port upon close
>    net/qede: release port upon close
> 
> Sachin Saxena (4):
>    net/dpaa: release port upon close
>    net/dpaa2: release port upon close
>    net/enetc: release port upon close
>    net/pfe: release port upon close
> 
> Thomas Monjalon (22):
>    ethdev: reset device and interrupt pointers on release
>    ethdev: allow drivers to return error on close
>    net/af_packet: release port upon close
>    net/atlantic: release port upon close
>    net/axgbe: release port upon close
>    net/bonding: release port upon close
>    net/failsafe: release port upon close
>    net/mlx4: release port upon close
>    net/null: release port upon close
>    net/octeontx: release port upon close
>    net/pcap: release port upon close
>    net/ring: release port upon close
>    net/softnic: release port upon close
>    ethdev: remove old close behaviour
>    drivers/net: accept removing device without any port
>    drivers/net: check process type in close operation
>    drivers/net: remove redundant MAC addresses freeing
>    app/testpmd: reset port status on close notification
>    app/testpmd: align behaviour of multi-port detach
>    ethdev: remove forcing stopped state upon close
>    ethdev: reset all when releasing a port
>    ethdev: allow close function to return an error
> 
> Yunjian Wang (1):
>    net/tap: release port upon close
> 


Since some discussions going on them last three patches not merged, they can be 
sent separately.

Except from last three patches, 27/29, 28/29, 29/29,
Series applied to dpdk-next-net/main, thanks.

[dpdk-dev] [PATCH v4 0/3] cleanup ethdev close operation

[Thomas Monjalon]

Most of the patches from the v3 have been merged already.
The remaining 3 patches are reworked in this v4:
  - reset dev_started in some PMDs
  - selectively reset pointers in rte_eth_dev_release_port()
  - fix error handling

Thomas Monjalon (3):
  ethdev: remove forcing stopped state upon close
  ethdev: reset all when releasing a port
  ethdev: allow close function to return an error

 doc/guides/rel_notes/deprecation.rst    |  1 -
 doc/guides/rel_notes/release_20_11.rst  |  4 +++-
 drivers/net/ark/ark_ethdev.c            |  4 +---
 drivers/net/atlantic/atl_ethdev.c       |  5 +----
 drivers/net/axgbe/axgbe_ethdev.c        |  3 ---
 drivers/net/bnxt/bnxt_ethdev.c          |  4 ----
 drivers/net/bnxt/bnxt_reps.c            |  1 -
 drivers/net/bonding/rte_eth_bond_pmd.c  |  4 ----
 drivers/net/cxgbe/cxgbe_ethdev.c        |  5 +++--
 drivers/net/cxgbe/cxgbevf_ethdev.c      |  5 +++--
 drivers/net/dpaa/dpaa_ethdev.c          |  5 +----
 drivers/net/dpaa2/dpaa2_ethdev.c        |  4 ----
 drivers/net/e1000/em_ethdev.c           |  6 ++----
 drivers/net/e1000/igb_ethdev.c          |  9 +--------
 drivers/net/ena/ena_ethdev.c            |  6 +-----
 drivers/net/enetc/enetc_ethdev.c        |  1 +
 drivers/net/failsafe/failsafe_ether.c   |  6 +++++-
 drivers/net/failsafe/failsafe_ops.c     | 25 ++++++++++++++++--------
 drivers/net/fm10k/fm10k_ethdev.c        |  5 +----
 drivers/net/hinic/hinic_pmd_ethdev.c    |  6 ++----
 drivers/net/hns3/hns3_ethdev.c          |  5 +----
 drivers/net/hns3/hns3_ethdev_vf.c       |  6 +-----
 drivers/net/i40e/i40e_ethdev.c          |  5 +----
 drivers/net/i40e/i40e_ethdev_vf.c       |  6 +-----
 drivers/net/iavf/iavf_ethdev.c          |  5 +----
 drivers/net/ice/ice_dcf_ethdev.c        |  4 ----
 drivers/net/ice/ice_ethdev.c            |  5 +----
 drivers/net/igc/igc_ethdev.c            |  1 +
 drivers/net/ionic/ionic_ethdev.c        |  5 -----
 drivers/net/ixgbe/ixgbe_ethdev.c        |  9 +--------
 drivers/net/kni/rte_eth_kni.c           |  1 +
 drivers/net/liquidio/lio_ethdev.c       |  5 +----
 drivers/net/memif/rte_eth_memif.c       |  4 +---
 drivers/net/mlx5/mlx5.c                 |  7 ++++---
 drivers/net/mvneta/mvneta_ethdev.c      |  7 +++++--
 drivers/net/mvpp2/mrvl_ethdev.c         |  5 +++--
 drivers/net/netvsc/hn_ethdev.c          | 11 +++++------
 drivers/net/netvsc/hn_var.h             |  2 +-
 drivers/net/netvsc/hn_vf.c              |  7 +++++--
 drivers/net/nfb/nfb_ethdev.c            |  2 ++
 drivers/net/octeontx/octeontx_ethdev.c  |  3 ---
 drivers/net/pfe/pfe_ethdev.c            |  2 ++
 drivers/net/qede/qede_ethdev.c          |  5 +----
 drivers/net/ring/rte_eth_ring.c         |  1 +
 drivers/net/sfc/sfc_ethdev.c            |  7 +------
 drivers/net/szedata2/rte_eth_szedata2.c |  2 ++
 drivers/net/tap/rte_eth_tap.c           |  1 -
 drivers/net/thunderx/nicvf_ethdev.c     |  1 +
 drivers/net/vhost/rte_eth_vhost.c       |  1 +
 drivers/net/virtio/virtio_ethdev.c      |  5 +----
 drivers/net/virtio/virtio_user_ethdev.c |  4 +---
 drivers/net/vmxnet3/vmxnet3_ethdev.c    |  6 +-----
 lib/librte_ethdev/rte_ethdev.c          | 26 +++++++++++++++++++------
 lib/librte_ethdev/rte_ethdev.h          |  5 ++++-
 54 files changed, 114 insertions(+), 166 deletions(-)

-- 
2.28.0

[dpdk-dev] [PATCH v4 1/3] ethdev: remove forcing stopped state upon close

[Thomas Monjalon]

When closing a port, it is supposed to be already stopped,
and marked as such with "dev_started" state zeroed by the stop API.

Resetting "dev_started" before calling the driver close operation
was hiding the case of not properly stopped port being closed.
The flag "dev_started" is not changed anymore in "rte_eth_dev_close()".

In case the "dev_stop" function is called from "dev_close",
bypassing "rte_eth_dev_stop()" API,
the "dev_started" state must be explicitly reset in the PMD
in order to keep the same behaviour.

Signed-off-by: Thomas Monjalon <thomas@monjalon.net>
Acked-by: Stephen Hemminger <stephen@networkplumber.org>
Reviewed-by: Andrew Rybchenko <arybchenko@solarflare.com>
---
 drivers/net/ark/ark_ethdev.c            | 1 +
 drivers/net/atlantic/atl_ethdev.c       | 1 +
 drivers/net/dpaa/dpaa_ethdev.c          | 1 +
 drivers/net/e1000/em_ethdev.c           | 2 ++
 drivers/net/e1000/igb_ethdev.c          | 1 +
 drivers/net/ena/ena_ethdev.c            | 1 +
 drivers/net/enetc/enetc_ethdev.c        | 1 +
 drivers/net/fm10k/fm10k_ethdev.c        | 1 +
 drivers/net/hinic/hinic_pmd_ethdev.c    | 2 ++
 drivers/net/hns3/hns3_ethdev.c          | 1 +
 drivers/net/hns3/hns3_ethdev_vf.c       | 1 +
 drivers/net/i40e/i40e_ethdev.c          | 1 +
 drivers/net/i40e/i40e_ethdev_vf.c       | 2 +-
 drivers/net/iavf/iavf_ethdev.c          | 1 +
 drivers/net/ice/ice_ethdev.c            | 1 +
 drivers/net/igc/igc_ethdev.c            | 1 +
 drivers/net/ixgbe/ixgbe_ethdev.c        | 1 +
 drivers/net/kni/rte_eth_kni.c           | 1 +
 drivers/net/liquidio/lio_ethdev.c       | 1 +
 drivers/net/mvneta/mvneta_ethdev.c      | 2 ++
 drivers/net/netvsc/hn_ethdev.c          | 1 +
 drivers/net/nfb/nfb_ethdev.c            | 2 ++
 drivers/net/pfe/pfe_ethdev.c            | 2 ++
 drivers/net/qede/qede_ethdev.c          | 1 +
 drivers/net/ring/rte_eth_ring.c         | 1 +
 drivers/net/szedata2/rte_eth_szedata2.c | 2 ++
 drivers/net/thunderx/nicvf_ethdev.c     | 1 +
 drivers/net/vhost/rte_eth_vhost.c       | 1 +
 drivers/net/virtio/virtio_ethdev.c      | 1 +
 drivers/net/vmxnet3/vmxnet3_ethdev.c    | 1 +
 lib/librte_ethdev/rte_ethdev.c          | 1 -
 31 files changed, 36 insertions(+), 2 deletions(-)

diff --git a/drivers/net/ark/ark_ethdev.c b/drivers/net/ark/ark_ethdev.c
index e3b1347769..f1355581f1 100644
--- a/drivers/net/ark/ark_ethdev.c
+++ b/drivers/net/ark/ark_ethdev.c
@@ -595,6 +595,7 @@ eth_ark_dev_stop(struct rte_eth_dev *dev)
 	if (ark->started == 0)
 		return;
 	ark->started = 0;
+	dev->data->dev_started = 0;
 
 	/* Stop the extension first */
 	if (ark->user_ext.dev_stop)
diff --git a/drivers/net/atlantic/atl_ethdev.c b/drivers/net/atlantic/atl_ethdev.c
index 2217511ca0..f2e48e1c11 100644
--- a/drivers/net/atlantic/atl_ethdev.c
+++ b/drivers/net/atlantic/atl_ethdev.c
@@ -609,6 +609,7 @@ atl_dev_stop(struct rte_eth_dev *dev)
 	struct rte_intr_handle *intr_handle = &pci_dev->intr_handle;
 
 	PMD_INIT_FUNC_TRACE();
+	dev->data->dev_started = 0;
 
 	/* disable interrupts */
 	atl_disable_intr(hw);
diff --git a/drivers/net/dpaa/dpaa_ethdev.c b/drivers/net/dpaa/dpaa_ethdev.c
index a01c8f3fc6..0ac9ef9fa1 100644
--- a/drivers/net/dpaa/dpaa_ethdev.c
+++ b/drivers/net/dpaa/dpaa_ethdev.c
@@ -363,6 +363,7 @@ static void dpaa_eth_dev_stop(struct rte_eth_dev *dev)
 	struct fman_if *fif = dev->process_private;
 
 	PMD_INIT_FUNC_TRACE();
+	dev->data->dev_started = 0;
 
 	if (!fif->is_shared_mac)
 		fman_if_disable_rx(fif);
diff --git a/drivers/net/e1000/em_ethdev.c b/drivers/net/e1000/em_ethdev.c
index d050eb478a..dd9238a616 100644
--- a/drivers/net/e1000/em_ethdev.c
+++ b/drivers/net/e1000/em_ethdev.c
@@ -717,6 +717,8 @@ eth_em_stop(struct rte_eth_dev *dev)
 	struct rte_pci_device *pci_dev = RTE_ETH_DEV_TO_PCI(dev);
 	struct rte_intr_handle *intr_handle = &pci_dev->intr_handle;
 
+	dev->data->dev_started = 0;
+
 	eth_em_rxtx_control(dev, false);
 	em_rxq_intr_disable(hw);
 	em_lsc_intr_disable(hw);
diff --git a/drivers/net/e1000/igb_ethdev.c b/drivers/net/e1000/igb_ethdev.c
index cb3d97e2a3..76746c9484 100644
--- a/drivers/net/e1000/igb_ethdev.c
+++ b/drivers/net/e1000/igb_ethdev.c
@@ -1497,6 +1497,7 @@ eth_igb_stop(struct rte_eth_dev *dev)
 	}
 
 	adapter->stopped = true;
+	dev->data->dev_started = 0;
 }
 
 static int
diff --git a/drivers/net/ena/ena_ethdev.c b/drivers/net/ena/ena_ethdev.c
index d32fa43837..e02d340c33 100644
--- a/drivers/net/ena/ena_ethdev.c
+++ b/drivers/net/ena/ena_ethdev.c
@@ -1118,6 +1118,7 @@ static void ena_stop(struct rte_eth_dev *dev)
 
 	++adapter->dev_stats.dev_stop;
 	adapter->state = ENA_ADAPTER_STATE_STOPPED;
+	dev->data->dev_started = 0;
 }
 
 static int ena_create_io_queue(struct ena_ring *ring)
diff --git a/drivers/net/enetc/enetc_ethdev.c b/drivers/net/enetc/enetc_ethdev.c
index b3dec7e64d..dddbc89939 100644
--- a/drivers/net/enetc/enetc_ethdev.c
+++ b/drivers/net/enetc/enetc_ethdev.c
@@ -54,6 +54,7 @@ enetc_dev_stop(struct rte_eth_dev *dev)
 	uint32_t val;
 
 	PMD_INIT_FUNC_TRACE();
+	dev->data->dev_started = 0;
 	/* Disable port */
 	val = enetc_port_rd(enetc_hw, ENETC_PMR);
 	enetc_port_wr(enetc_hw, ENETC_PMR, val & (~ENETC_PMR_EN));
diff --git a/drivers/net/fm10k/fm10k_ethdev.c b/drivers/net/fm10k/fm10k_ethdev.c
index 3096370718..51b36a1d7b 100644
--- a/drivers/net/fm10k/fm10k_ethdev.c
+++ b/drivers/net/fm10k/fm10k_ethdev.c
@@ -1161,6 +1161,7 @@ fm10k_dev_stop(struct rte_eth_dev *dev)
 	int i;
 
 	PMD_INIT_FUNC_TRACE();
+	dev->data->dev_started = 0;
 
 	if (dev->data->tx_queues)
 		for (i = 0; i < dev->data->nb_tx_queues; i++)
diff --git a/drivers/net/hinic/hinic_pmd_ethdev.c b/drivers/net/hinic/hinic_pmd_ethdev.c
index 466c8362b9..daf8f33cf3 100644
--- a/drivers/net/hinic/hinic_pmd_ethdev.c
+++ b/drivers/net/hinic/hinic_pmd_ethdev.c
@@ -1189,6 +1189,8 @@ static void hinic_dev_stop(struct rte_eth_dev *dev)
 	name = dev->data->name;
 	port_id = dev->data->port_id;
 
+	dev->data->dev_started = 0;
+
 	if (!rte_bit_relaxed_test_and_clear32(HINIC_DEV_START,
 					      &nic_dev->dev_status)) {
 		PMD_DRV_LOG(INFO, "Device %s already stopped", name);
diff --git a/drivers/net/hns3/hns3_ethdev.c b/drivers/net/hns3/hns3_ethdev.c
index c247583beb..a607824162 100644
--- a/drivers/net/hns3/hns3_ethdev.c
+++ b/drivers/net/hns3/hns3_ethdev.c
@@ -4855,6 +4855,7 @@ hns3_dev_stop(struct rte_eth_dev *dev)
 	struct hns3_hw *hw = &hns->hw;
 
 	PMD_INIT_FUNC_TRACE();
+	dev->data->dev_started = 0;
 
 	hw->adapter_state = HNS3_NIC_STOPPING;
 	hns3_set_rxtx_function(dev);
diff --git a/drivers/net/hns3/hns3_ethdev_vf.c b/drivers/net/hns3/hns3_ethdev_vf.c
index fec609c845..173983a91f 100644
--- a/drivers/net/hns3/hns3_ethdev_vf.c
+++ b/drivers/net/hns3/hns3_ethdev_vf.c
@@ -1892,6 +1892,7 @@ hns3vf_dev_stop(struct rte_eth_dev *dev)
 	struct hns3_hw *hw = &hns->hw;
 
 	PMD_INIT_FUNC_TRACE();
+	dev->data->dev_started = 0;
 
 	hw->adapter_state = HNS3_NIC_STOPPING;
 	hns3_set_rxtx_function(dev);
diff --git a/drivers/net/i40e/i40e_ethdev.c b/drivers/net/i40e/i40e_ethdev.c
index 943cfe71dc..825de51fbb 100644
--- a/drivers/net/i40e/i40e_ethdev.c
+++ b/drivers/net/i40e/i40e_ethdev.c
@@ -2599,6 +2599,7 @@ i40e_dev_stop(struct rte_eth_dev *dev)
 	pf->tm_conf.committed = false;
 
 	hw->adapter_stopped = 1;
+	dev->data->dev_started = 0;
 
 	pf->adapter->rss_reta_updated = 0;
 }
diff --git a/drivers/net/i40e/i40e_ethdev_vf.c b/drivers/net/i40e/i40e_ethdev_vf.c
index 4d6510d1ff..40c9745433 100644
--- a/drivers/net/i40e/i40e_ethdev_vf.c
+++ b/drivers/net/i40e/i40e_ethdev_vf.c
@@ -2207,7 +2207,7 @@ i40evf_dev_stop(struct rte_eth_dev *dev)
 	i40evf_add_del_mc_addr_list(dev, vf->mc_addrs, vf->mc_addrs_num,
 				FALSE);
 	hw->adapter_stopped = 1;
-
+	dev->data->dev_started = 0;
 }
 
 static int
diff --git a/drivers/net/iavf/iavf_ethdev.c b/drivers/net/iavf/iavf_ethdev.c
index a5b1433306..90949521b1 100644
--- a/drivers/net/iavf/iavf_ethdev.c
+++ b/drivers/net/iavf/iavf_ethdev.c
@@ -529,6 +529,7 @@ iavf_dev_stop(struct rte_eth_dev *dev)
 				  false);
 
 	adapter->stopped = 1;
+	dev->data->dev_started = 0;
 }
 
 static int
diff --git a/drivers/net/ice/ice_ethdev.c b/drivers/net/ice/ice_ethdev.c
index d8ce09d28f..1db4015c23 100644
--- a/drivers/net/ice/ice_ethdev.c
+++ b/drivers/net/ice/ice_ethdev.c
@@ -2376,6 +2376,7 @@ ice_dev_stop(struct rte_eth_dev *dev)
 	}
 
 	pf->adapter_stopped = true;
+	dev->data->dev_started = 0;
 }
 
 static int
diff --git a/drivers/net/igc/igc_ethdev.c b/drivers/net/igc/igc_ethdev.c
index 7f5066df4b..73b7d20ac2 100644
--- a/drivers/net/igc/igc_ethdev.c
+++ b/drivers/net/igc/igc_ethdev.c
@@ -616,6 +616,7 @@ eth_igc_stop(struct rte_eth_dev *dev)
 	struct rte_intr_handle *intr_handle = &pci_dev->intr_handle;
 	struct rte_eth_link link;
 
+	dev->data->dev_started = 0;
 	adapter->stopped = 1;
 
 	/* disable receive and transmit */
diff --git a/drivers/net/ixgbe/ixgbe_ethdev.c b/drivers/net/ixgbe/ixgbe_ethdev.c
index 0b98e210e7..bd83b0e31a 100644
--- a/drivers/net/ixgbe/ixgbe_ethdev.c
+++ b/drivers/net/ixgbe/ixgbe_ethdev.c
@@ -2917,6 +2917,7 @@ ixgbe_dev_stop(struct rte_eth_dev *dev)
 	adapter->rss_reta_updated = 0;
 
 	hw->adapter_stopped = true;
+	dev->data->dev_started = 0;
 }
 
 /*
diff --git a/drivers/net/kni/rte_eth_kni.c b/drivers/net/kni/rte_eth_kni.c
index be747adf86..329a55039f 100644
--- a/drivers/net/kni/rte_eth_kni.c
+++ b/drivers/net/kni/rte_eth_kni.c
@@ -196,6 +196,7 @@ eth_kni_dev_stop(struct rte_eth_dev *dev)
 	}
 
 	dev->data->dev_link.link_status = 0;
+	dev->data->dev_started = 0;
 }
 
 static int
diff --git a/drivers/net/liquidio/lio_ethdev.c b/drivers/net/liquidio/lio_ethdev.c
index 1a41f27198..52bcefc819 100644
--- a/drivers/net/liquidio/lio_ethdev.c
+++ b/drivers/net/liquidio/lio_ethdev.c
@@ -1471,6 +1471,7 @@ lio_dev_stop(struct rte_eth_dev *eth_dev)
 	struct lio_device *lio_dev = LIO_DEV(eth_dev);
 
 	lio_dev_info(lio_dev, "Stopping port %d\n", eth_dev->data->port_id);
+	eth_dev->data->dev_started = 0;
 	lio_dev->intf_open = 0;
 	rte_mb();
 
diff --git a/drivers/net/mvneta/mvneta_ethdev.c b/drivers/net/mvneta/mvneta_ethdev.c
index 607771149a..3c0332ab4d 100644
--- a/drivers/net/mvneta/mvneta_ethdev.c
+++ b/drivers/net/mvneta/mvneta_ethdev.c
@@ -413,6 +413,8 @@ mvneta_dev_stop(struct rte_eth_dev *dev)
 {
 	struct mvneta_priv *priv = dev->data->dev_private;
 
+	dev->data->dev_started = 0;
+
 	if (!priv->ppio)
 		return;
 
diff --git a/drivers/net/netvsc/hn_ethdev.c b/drivers/net/netvsc/hn_ethdev.c
index 5ae2d469c8..be3713dfa0 100644
--- a/drivers/net/netvsc/hn_ethdev.c
+++ b/drivers/net/netvsc/hn_ethdev.c
@@ -833,6 +833,7 @@ hn_dev_stop(struct rte_eth_dev *dev)
 	struct hn_data *hv = dev->data->dev_private;
 
 	PMD_INIT_FUNC_TRACE();
+	dev->data->dev_started = 0;
 
 	hn_rndis_set_rxfilter(hv, 0);
 	hn_vf_stop(dev);
diff --git a/drivers/net/nfb/nfb_ethdev.c b/drivers/net/nfb/nfb_ethdev.c
index a9a8bc878d..44e57ee6c8 100644
--- a/drivers/net/nfb/nfb_ethdev.c
+++ b/drivers/net/nfb/nfb_ethdev.c
@@ -158,6 +158,8 @@ nfb_eth_dev_stop(struct rte_eth_dev *dev)
 	uint16_t nb_rx = dev->data->nb_rx_queues;
 	uint16_t nb_tx = dev->data->nb_tx_queues;
 
+	dev->data->dev_started = 0;
+
 	for (i = 0; i < nb_tx; i++)
 		nfb_eth_tx_queue_stop(dev, i);
 
diff --git a/drivers/net/pfe/pfe_ethdev.c b/drivers/net/pfe/pfe_ethdev.c
index f0de1c8a2e..c4dd0a5f69 100644
--- a/drivers/net/pfe/pfe_ethdev.c
+++ b/drivers/net/pfe/pfe_ethdev.c
@@ -378,6 +378,8 @@ pfe_eth_stop(struct rte_eth_dev *dev/*, int wake*/)
 {
 	struct pfe_eth_priv_s *priv = dev->data->dev_private;
 
+	dev->data->dev_started = 0;
+
 	gemac_disable(priv->EMAC_baseaddr);
 	gpi_disable(priv->GPI_baseaddr);
 
diff --git a/drivers/net/qede/qede_ethdev.c b/drivers/net/qede/qede_ethdev.c
index 548497f3ae..88389c9e4f 100644
--- a/drivers/net/qede/qede_ethdev.c
+++ b/drivers/net/qede/qede_ethdev.c
@@ -1169,6 +1169,7 @@ static void qede_dev_stop(struct rte_eth_dev *eth_dev)
 	struct ecore_dev *edev = QEDE_INIT_EDEV(qdev);
 
 	PMD_INIT_FUNC_TRACE(edev);
+	eth_dev->data->dev_started = 0;
 
 	/* Bring the link down */
 	qede_dev_set_link_state(eth_dev, false);
diff --git a/drivers/net/ring/rte_eth_ring.c b/drivers/net/ring/rte_eth_ring.c
index 83c5502123..6dceab0c59 100644
--- a/drivers/net/ring/rte_eth_ring.c
+++ b/drivers/net/ring/rte_eth_ring.c
@@ -108,6 +108,7 @@ eth_dev_start(struct rte_eth_dev *dev)
 static void
 eth_dev_stop(struct rte_eth_dev *dev)
 {
+	dev->data->dev_started = 0;
 	dev->data->dev_link.link_status = ETH_LINK_DOWN;
 }
 
diff --git a/drivers/net/szedata2/rte_eth_szedata2.c b/drivers/net/szedata2/rte_eth_szedata2.c
index 0eecec1e8c..c89ff16fc1 100644
--- a/drivers/net/szedata2/rte_eth_szedata2.c
+++ b/drivers/net/szedata2/rte_eth_szedata2.c
@@ -1020,6 +1020,8 @@ eth_dev_stop(struct rte_eth_dev *dev)
 	uint16_t nb_rx = dev->data->nb_rx_queues;
 	uint16_t nb_tx = dev->data->nb_tx_queues;
 
+	dev->data->dev_started = 0;
+
 	for (i = 0; i < nb_tx; i++)
 		eth_tx_queue_stop(dev, i);
 
diff --git a/drivers/net/thunderx/nicvf_ethdev.c b/drivers/net/thunderx/nicvf_ethdev.c
index cc6eb4ba24..b87e1e5640 100644
--- a/drivers/net/thunderx/nicvf_ethdev.c
+++ b/drivers/net/thunderx/nicvf_ethdev.c
@@ -1766,6 +1766,7 @@ nicvf_dev_stop_cleanup(struct rte_eth_dev *dev, bool cleanup)
 	struct nicvf *nic = nicvf_pmd_priv(dev);
 
 	PMD_INIT_FUNC_TRACE();
+	dev->data->dev_started = 0;
 
 	/* Teardown secondary vf first */
 	for (i = 0; i < nic->sqs_count; i++) {
diff --git a/drivers/net/vhost/rte_eth_vhost.c b/drivers/net/vhost/rte_eth_vhost.c
index 66efecb320..5328cfcd02 100644
--- a/drivers/net/vhost/rte_eth_vhost.c
+++ b/drivers/net/vhost/rte_eth_vhost.c
@@ -1158,6 +1158,7 @@ eth_dev_stop(struct rte_eth_dev *dev)
 {
 	struct pmd_internal *internal = dev->data->dev_private;
 
+	dev->data->dev_started = 0;
 	rte_atomic32_set(&internal->started, 0);
 	update_queuing_status(dev);
 }
diff --git a/drivers/net/virtio/virtio_ethdev.c b/drivers/net/virtio/virtio_ethdev.c
index 0236c756dc..ea6496acce 100644
--- a/drivers/net/virtio/virtio_ethdev.c
+++ b/drivers/net/virtio/virtio_ethdev.c
@@ -2523,6 +2523,7 @@ virtio_dev_stop(struct rte_eth_dev *dev)
 	struct rte_intr_conf *intr_conf = &dev->data->dev_conf.intr_conf;
 
 	PMD_INIT_LOG(DEBUG, "stop");
+	dev->data->dev_started = 0;
 
 	rte_spinlock_lock(&hw->state_lock);
 	if (!hw->started)
diff --git a/drivers/net/vmxnet3/vmxnet3_ethdev.c b/drivers/net/vmxnet3/vmxnet3_ethdev.c
index fa950e1ba0..828043a65a 100644
--- a/drivers/net/vmxnet3/vmxnet3_ethdev.c
+++ b/drivers/net/vmxnet3/vmxnet3_ethdev.c
@@ -858,6 +858,7 @@ vmxnet3_dev_stop(struct rte_eth_dev *dev)
 	rte_eth_linkstatus_set(dev, &link);
 
 	hw->adapter_stopped = 1;
+	dev->data->dev_started = 0;
 }
 
 static void
diff --git a/lib/librte_ethdev/rte_ethdev.c b/lib/librte_ethdev/rte_ethdev.c
index d7668114ca..0b8e8e3e8d 100644
--- a/lib/librte_ethdev/rte_ethdev.c
+++ b/lib/librte_ethdev/rte_ethdev.c
@@ -1716,7 +1716,6 @@ rte_eth_dev_close(uint16_t port_id)
 	dev = &rte_eth_devices[port_id];
 
 	RTE_FUNC_PTR_OR_RET(*dev->dev_ops->dev_close);
-	dev->data->dev_started = 0;
 	(*dev->dev_ops->dev_close)(dev);
 
 	rte_ethdev_trace_close(port_id);
-- 
2.28.0

[dpdk-dev] [PATCH v4 2/3] ethdev: reset all when releasing a port

[Thomas Monjalon]

The function rte_eth_dev_release_port() is partially resetting
the struct rte_eth_dev. The drivers were completing this reset
with more pointers set to NULL in the close or remove operations.

More pointers are reset at ethdev level,
and some redundant assignments are removed from PMDs.

Signed-off-by: Thomas Monjalon <thomas@monjalon.net>
Acked-by: Stephen Hemminger <stephen@networkplumber.org>
Reviewed-by: Haiyue Wang <haiyue.wang@intel.com>
Acked-by: Jeff Guo <jia.guo@intel.com>
Reviewed-by: Andrew Rybchenko <arybchenko@solarflare.com>
---
 drivers/net/ark/ark_ethdev.c           | 3 ---
 drivers/net/atlantic/atl_ethdev.c      | 4 ----
 drivers/net/axgbe/axgbe_ethdev.c       | 3 ---
 drivers/net/bnxt/bnxt_ethdev.c         | 4 ----
 drivers/net/bnxt/bnxt_reps.c           | 1 -
 drivers/net/bonding/rte_eth_bond_pmd.c | 4 ----
 drivers/net/dpaa/dpaa_ethdev.c         | 4 ----
 drivers/net/dpaa2/dpaa2_ethdev.c       | 4 ----
 drivers/net/e1000/em_ethdev.c          | 4 ----
 drivers/net/e1000/igb_ethdev.c         | 8 --------
 drivers/net/ena/ena_ethdev.c           | 5 -----
 drivers/net/fm10k/fm10k_ethdev.c       | 4 ----
 drivers/net/hinic/hinic_pmd_ethdev.c   | 4 ----
 drivers/net/hns3/hns3_ethdev.c         | 4 ----
 drivers/net/hns3/hns3_ethdev_vf.c      | 5 -----
 drivers/net/i40e/i40e_ethdev.c         | 4 ----
 drivers/net/i40e/i40e_ethdev_vf.c      | 4 ----
 drivers/net/iavf/iavf_ethdev.c         | 4 ----
 drivers/net/ice/ice_dcf_ethdev.c       | 4 ----
 drivers/net/ice/ice_ethdev.c           | 4 ----
 drivers/net/ionic/ionic_ethdev.c       | 5 -----
 drivers/net/ixgbe/ixgbe_ethdev.c       | 8 --------
 drivers/net/liquidio/lio_ethdev.c      | 4 ----
 drivers/net/netvsc/hn_ethdev.c         | 4 ----
 drivers/net/octeontx/octeontx_ethdev.c | 3 ---
 drivers/net/qede/qede_ethdev.c         | 4 ----
 drivers/net/sfc/sfc_ethdev.c           | 7 +------
 drivers/net/tap/rte_eth_tap.c          | 1 -
 drivers/net/virtio/virtio_ethdev.c     | 4 ----
 drivers/net/vmxnet3/vmxnet3_ethdev.c   | 5 -----
 lib/librte_ethdev/rte_ethdev.c         | 9 +++++++++
 31 files changed, 10 insertions(+), 125 deletions(-)

diff --git a/drivers/net/ark/ark_ethdev.c b/drivers/net/ark/ark_ethdev.c
index f1355581f1..dc14f0fb1a 100644
--- a/drivers/net/ark/ark_ethdev.c
+++ b/drivers/net/ark/ark_ethdev.c
@@ -504,9 +504,6 @@ eth_ark_dev_uninit(struct rte_eth_dev *dev)
 	ark_pktgen_uninit(ark->pg);
 	ark_pktchkr_uninit(ark->pc);
 
-	dev->dev_ops = NULL;
-	dev->rx_pkt_burst = NULL;
-	dev->tx_pkt_burst = NULL;
 	return 0;
 }
 
diff --git a/drivers/net/atlantic/atl_ethdev.c b/drivers/net/atlantic/atl_ethdev.c
index f2e48e1c11..d96865d5b4 100644
--- a/drivers/net/atlantic/atl_ethdev.c
+++ b/drivers/net/atlantic/atl_ethdev.c
@@ -702,10 +702,6 @@ atl_dev_close(struct rte_eth_dev *dev)
 
 	atl_free_queues(dev);
 
-	dev->dev_ops = NULL;
-	dev->rx_pkt_burst = NULL;
-	dev->tx_pkt_burst = NULL;
-
 	/* disable uio intr before callback unregister */
 	rte_intr_disable(intr_handle);
 	rte_intr_callback_unregister(intr_handle,
diff --git a/drivers/net/axgbe/axgbe_ethdev.c b/drivers/net/axgbe/axgbe_ethdev.c
index cf085487cc..39087f36f0 100644
--- a/drivers/net/axgbe/axgbe_ethdev.c
+++ b/drivers/net/axgbe/axgbe_ethdev.c
@@ -2134,9 +2134,6 @@ axgbe_dev_close(struct rte_eth_dev *eth_dev)
 		return 0;
 
 	pci_dev = RTE_DEV_TO_PCI(eth_dev->device);
-	eth_dev->dev_ops = NULL;
-	eth_dev->rx_pkt_burst = NULL;
-	eth_dev->tx_pkt_burst = NULL;
 	axgbe_dev_clear_queues(eth_dev);
 
 	/* disable uio intr before callback unregister */
diff --git a/drivers/net/bnxt/bnxt_ethdev.c b/drivers/net/bnxt/bnxt_ethdev.c
index a6812c1a66..2789079b19 100644
--- a/drivers/net/bnxt/bnxt_ethdev.c
+++ b/drivers/net/bnxt/bnxt_ethdev.c
@@ -1429,10 +1429,6 @@ static int bnxt_dev_close_op(struct rte_eth_dev *eth_dev)
 	bnxt_free_pf_info(bp);
 	bnxt_free_parent_info(bp);
 
-	eth_dev->dev_ops = NULL;
-	eth_dev->rx_pkt_burst = NULL;
-	eth_dev->tx_pkt_burst = NULL;
-
 	rte_memzone_free((const struct rte_memzone *)bp->tx_mem_zone);
 	bp->tx_mem_zone = NULL;
 	rte_memzone_free((const struct rte_memzone *)bp->rx_mem_zone);
diff --git a/drivers/net/bnxt/bnxt_reps.c b/drivers/net/bnxt/bnxt_reps.c
index 74a76fce57..012ccc73fe 100644
--- a/drivers/net/bnxt/bnxt_reps.c
+++ b/drivers/net/bnxt/bnxt_reps.c
@@ -255,7 +255,6 @@ int bnxt_representor_uninit(struct rte_eth_dev *eth_dev)
 
 	PMD_DRV_LOG(DEBUG, "BNXT Port:%d VFR uninit\n", eth_dev->data->port_id);
 	eth_dev->data->mac_addrs = NULL;
-	eth_dev->dev_ops = NULL;
 
 	parent_bp = rep->parent_dev->data->dev_private;
 	if (!parent_bp) {
diff --git a/drivers/net/bonding/rte_eth_bond_pmd.c b/drivers/net/bonding/rte_eth_bond_pmd.c
index 1f761c7c9e..ce8fe254af 100644
--- a/drivers/net/bonding/rte_eth_bond_pmd.c
+++ b/drivers/net/bonding/rte_eth_bond_pmd.c
@@ -2130,10 +2130,6 @@ bond_ethdev_close(struct rte_eth_dev *dev)
 	 */
 	rte_mempool_free(internals->mode6.mempool);
 
-	dev->dev_ops = NULL;
-	dev->rx_pkt_burst = NULL;
-	dev->tx_pkt_burst = NULL;
-
 	return 0;
 }
 
diff --git a/drivers/net/dpaa/dpaa_ethdev.c b/drivers/net/dpaa/dpaa_ethdev.c
index 0ac9ef9fa1..0a46859d9e 100644
--- a/drivers/net/dpaa/dpaa_ethdev.c
+++ b/drivers/net/dpaa/dpaa_ethdev.c
@@ -442,10 +442,6 @@ static int dpaa_eth_dev_close(struct rte_eth_dev *dev)
 	rte_free(dpaa_intf->tx_queues);
 	dpaa_intf->tx_queues = NULL;
 
-	dev->dev_ops = NULL;
-	dev->rx_pkt_burst = NULL;
-	dev->tx_pkt_burst = NULL;
-
 	return 0;
 }
 
diff --git a/drivers/net/dpaa2/dpaa2_ethdev.c b/drivers/net/dpaa2/dpaa2_ethdev.c
index d8624514df..ba99a524f9 100644
--- a/drivers/net/dpaa2/dpaa2_ethdev.c
+++ b/drivers/net/dpaa2/dpaa2_ethdev.c
@@ -1284,10 +1284,6 @@ dpaa2_dev_close(struct rte_eth_dev *dev)
 	if (priv->extract.qos_extract_param)
 		rte_free((void *)(size_t)priv->extract.qos_extract_param);
 
-	dev->dev_ops = NULL;
-	dev->rx_pkt_burst = NULL;
-	dev->tx_pkt_burst = NULL;
-
 	DPAA2_PMD_INFO("%s: netdev deleted", dev->data->name);
 	return 0;
 }
diff --git a/drivers/net/e1000/em_ethdev.c b/drivers/net/e1000/em_ethdev.c
index dd9238a616..304c040421 100644
--- a/drivers/net/e1000/em_ethdev.c
+++ b/drivers/net/e1000/em_ethdev.c
@@ -774,10 +774,6 @@ eth_em_close(struct rte_eth_dev *dev)
 	em_release_manageability(hw);
 	em_hw_control_release(hw);
 
-	dev->dev_ops = NULL;
-	dev->rx_pkt_burst = NULL;
-	dev->tx_pkt_burst = NULL;
-
 	/* disable uio intr before callback unregister */
 	rte_intr_disable(intr_handle);
 	rte_intr_callback_unregister(intr_handle,
diff --git a/drivers/net/e1000/igb_ethdev.c b/drivers/net/e1000/igb_ethdev.c
index 76746c9484..f7c62f473d 100644
--- a/drivers/net/e1000/igb_ethdev.c
+++ b/drivers/net/e1000/igb_ethdev.c
@@ -1565,10 +1565,6 @@ eth_igb_close(struct rte_eth_dev *dev)
 	memset(&link, 0, sizeof(link));
 	rte_eth_linkstatus_set(dev, &link);
 
-	dev->dev_ops = NULL;
-	dev->rx_pkt_burst = NULL;
-	dev->tx_pkt_burst = NULL;
-
 	/* Reset any pending lock */
 	igb_reset_swfw_lock(hw);
 
@@ -3404,10 +3400,6 @@ igbvf_dev_close(struct rte_eth_dev *dev)
 	memset(&addr, 0, sizeof(addr));
 	igbvf_default_mac_addr_set(dev, &addr);
 
-	dev->dev_ops = NULL;
-	dev->rx_pkt_burst = NULL;
-	dev->tx_pkt_burst = NULL;
-
 	rte_intr_callback_unregister(&pci_dev->intr_handle,
 				     eth_igbvf_interrupt_handler,
 				     (void *)dev);
diff --git a/drivers/net/ena/ena_ethdev.c b/drivers/net/ena/ena_ethdev.c
index e02d340c33..bdb8fe79c6 100644
--- a/drivers/net/ena/ena_ethdev.c
+++ b/drivers/net/ena/ena_ethdev.c
@@ -1956,11 +1956,6 @@ static int eth_ena_dev_uninit(struct rte_eth_dev *eth_dev)
 
 	ena_destroy_device(eth_dev);
 
-	eth_dev->dev_ops = NULL;
-	eth_dev->rx_pkt_burst = NULL;
-	eth_dev->tx_pkt_burst = NULL;
-	eth_dev->tx_pkt_prepare = NULL;
-
 	return 0;
 }
 
diff --git a/drivers/net/fm10k/fm10k_ethdev.c b/drivers/net/fm10k/fm10k_ethdev.c
index 51b36a1d7b..949fe8fea3 100644
--- a/drivers/net/fm10k/fm10k_ethdev.c
+++ b/drivers/net/fm10k/fm10k_ethdev.c
@@ -2803,10 +2803,6 @@ fm10k_dev_close(struct rte_eth_dev *dev)
 	fm10k_dev_queue_release(dev);
 	fm10k_stop_hw(hw);
 
-	dev->dev_ops = NULL;
-	dev->rx_pkt_burst = NULL;
-	dev->tx_pkt_burst = NULL;
-
 	/* disable uio/vfio intr */
 	rte_intr_disable(intr_handle);
 
diff --git a/drivers/net/hinic/hinic_pmd_ethdev.c b/drivers/net/hinic/hinic_pmd_ethdev.c
index daf8f33cf3..2504228d94 100644
--- a/drivers/net/hinic/hinic_pmd_ethdev.c
+++ b/drivers/net/hinic/hinic_pmd_ethdev.c
@@ -3253,10 +3253,6 @@ static int hinic_dev_uninit(struct rte_eth_dev *dev)
 
 	hinic_dev_close(dev);
 
-	dev->dev_ops = NULL;
-	dev->rx_pkt_burst = NULL;
-	dev->tx_pkt_burst = NULL;
-
 	rte_free(nic_dev->mc_list);
 
 	return HINIC_OK;
diff --git a/drivers/net/hns3/hns3_ethdev.c b/drivers/net/hns3/hns3_ethdev.c
index a607824162..2a6ee797b7 100644
--- a/drivers/net/hns3/hns3_ethdev.c
+++ b/drivers/net/hns3/hns3_ethdev.c
@@ -5739,10 +5739,6 @@ hns3_dev_uninit(struct rte_eth_dev *eth_dev)
 	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
 		return -EPERM;
 
-	eth_dev->dev_ops = NULL;
-	eth_dev->rx_pkt_burst = NULL;
-	eth_dev->tx_pkt_burst = NULL;
-	eth_dev->tx_pkt_prepare = NULL;
 	if (hw->adapter_state < HNS3_NIC_CLOSING)
 		hns3_dev_close(eth_dev);
 
diff --git a/drivers/net/hns3/hns3_ethdev_vf.c b/drivers/net/hns3/hns3_ethdev_vf.c
index 173983a91f..041c6a3d93 100644
--- a/drivers/net/hns3/hns3_ethdev_vf.c
+++ b/drivers/net/hns3/hns3_ethdev_vf.c
@@ -2747,11 +2747,6 @@ hns3vf_dev_uninit(struct rte_eth_dev *eth_dev)
 	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
 		return -EPERM;
 
-	eth_dev->dev_ops = NULL;
-	eth_dev->rx_pkt_burst = NULL;
-	eth_dev->tx_pkt_burst = NULL;
-	eth_dev->tx_pkt_prepare = NULL;
-
 	if (hw->adapter_state < HNS3_NIC_CLOSING)
 		hns3vf_dev_close(eth_dev);
 
diff --git a/drivers/net/i40e/i40e_ethdev.c b/drivers/net/i40e/i40e_ethdev.c
index 825de51fbb..605a1792b6 100644
--- a/drivers/net/i40e/i40e_ethdev.c
+++ b/drivers/net/i40e/i40e_ethdev.c
@@ -2693,10 +2693,6 @@ i40e_dev_close(struct rte_eth_dev *dev)
 			(reg | I40E_PFGEN_CTRL_PFSWR_MASK));
 	I40E_WRITE_FLUSH(hw);
 
-	dev->dev_ops = NULL;
-	dev->rx_pkt_burst = NULL;
-	dev->tx_pkt_burst = NULL;
-
 	/* Clear PXE mode */
 	i40e_clear_pxe_mode(hw);
 
diff --git a/drivers/net/i40e/i40e_ethdev_vf.c b/drivers/net/i40e/i40e_ethdev_vf.c
index 40c9745433..3dcd50c428 100644
--- a/drivers/net/i40e/i40e_ethdev_vf.c
+++ b/drivers/net/i40e/i40e_ethdev_vf.c
@@ -2421,10 +2421,6 @@ i40evf_dev_close(struct rte_eth_dev *dev)
 	i40e_shutdown_adminq(hw);
 	i40evf_disable_irq0(hw);
 
-	dev->dev_ops = NULL;
-	dev->rx_pkt_burst = NULL;
-	dev->tx_pkt_burst = NULL;
-
 	rte_free(vf->vf_res);
 	vf->vf_res = NULL;
 	rte_free(vf->aq_resp);
diff --git a/drivers/net/iavf/iavf_ethdev.c b/drivers/net/iavf/iavf_ethdev.c
index 90949521b1..241ab62a49 100644
--- a/drivers/net/iavf/iavf_ethdev.c
+++ b/drivers/net/iavf/iavf_ethdev.c
@@ -1484,10 +1484,6 @@ iavf_dev_close(struct rte_eth_dev *dev)
 				     iavf_dev_interrupt_handler, dev);
 	iavf_disable_irq0(hw);
 
-	dev->dev_ops = NULL;
-	dev->rx_pkt_burst = NULL;
-	dev->tx_pkt_burst = NULL;
-
 	if (vf->vf_res->vf_cap_flags & VIRTCHNL_VF_OFFLOAD_RSS_PF) {
 		if (vf->rss_lut) {
 			rte_free(vf->rss_lut);
diff --git a/drivers/net/ice/ice_dcf_ethdev.c b/drivers/net/ice/ice_dcf_ethdev.c
index 33dd0c44f2..f2056e2aba 100644
--- a/drivers/net/ice/ice_dcf_ethdev.c
+++ b/drivers/net/ice/ice_dcf_ethdev.c
@@ -855,10 +855,6 @@ ice_dcf_dev_close(struct rte_eth_dev *dev)
 	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
 		return 0;
 
-	dev->dev_ops = NULL;
-	dev->rx_pkt_burst = NULL;
-	dev->tx_pkt_burst = NULL;
-
 	ice_dcf_uninit_parent_adapter(dev);
 	ice_dcf_uninit_hw(dev, &adapter->real_hw);
 
diff --git a/drivers/net/ice/ice_ethdev.c b/drivers/net/ice/ice_ethdev.c
index 1db4015c23..33b2828703 100644
--- a/drivers/net/ice/ice_ethdev.c
+++ b/drivers/net/ice/ice_ethdev.c
@@ -2417,10 +2417,6 @@ ice_dev_close(struct rte_eth_dev *dev)
 	rte_free(pf->proto_xtr);
 	pf->proto_xtr = NULL;
 
-	dev->dev_ops = NULL;
-	dev->rx_pkt_burst = NULL;
-	dev->tx_pkt_burst = NULL;
-
 	/* disable uio intr before callback unregister */
 	rte_intr_disable(intr_handle);
 
diff --git a/drivers/net/ionic/ionic_ethdev.c b/drivers/net/ionic/ionic_ethdev.c
index ef7d06e526..1387253dc8 100644
--- a/drivers/net/ionic/ionic_ethdev.c
+++ b/drivers/net/ionic/ionic_ethdev.c
@@ -1065,11 +1065,6 @@ eth_ionic_dev_uninit(struct rte_eth_dev *eth_dev)
 	ionic_lif_deinit(lif);
 	ionic_lif_free(lif);
 
-	eth_dev->dev_ops = NULL;
-	eth_dev->rx_pkt_burst = NULL;
-	eth_dev->tx_pkt_burst = NULL;
-	eth_dev->tx_pkt_prepare = NULL;
-
 	return 0;
 }
 
diff --git a/drivers/net/ixgbe/ixgbe_ethdev.c b/drivers/net/ixgbe/ixgbe_ethdev.c
index bd83b0e31a..a7f8a31488 100644
--- a/drivers/net/ixgbe/ixgbe_ethdev.c
+++ b/drivers/net/ixgbe/ixgbe_ethdev.c
@@ -3010,10 +3010,6 @@ ixgbe_dev_close(struct rte_eth_dev *dev)
 	/* reprogram the RAR[0] in case user changed it. */
 	ixgbe_set_rar(hw, 0, hw->mac.addr, 0, IXGBE_RAH_AV);
 
-	dev->dev_ops = NULL;
-	dev->rx_pkt_burst = NULL;
-	dev->tx_pkt_burst = NULL;
-
 	/* Unlock any pending hardware semaphore */
 	ixgbe_swfw_lock_reset(hw);
 
@@ -5467,10 +5463,6 @@ ixgbevf_dev_close(struct rte_eth_dev *dev)
 	 **/
 	ixgbevf_remove_mac_addr(dev, 0);
 
-	dev->dev_ops = NULL;
-	dev->rx_pkt_burst = NULL;
-	dev->tx_pkt_burst = NULL;
-
 	rte_intr_disable(intr_handle);
 	rte_intr_callback_unregister(intr_handle,
 				     ixgbevf_dev_interrupt_handler, dev);
diff --git a/drivers/net/liquidio/lio_ethdev.c b/drivers/net/liquidio/lio_ethdev.c
index 52bcefc819..1b01502e49 100644
--- a/drivers/net/liquidio/lio_ethdev.c
+++ b/drivers/net/liquidio/lio_ethdev.c
@@ -2068,10 +2068,6 @@ lio_eth_dev_uninit(struct rte_eth_dev *eth_dev)
 	/* lio_free_sc_buffer_pool */
 	lio_free_sc_buffer_pool(lio_dev);
 
-	eth_dev->dev_ops = NULL;
-	eth_dev->rx_pkt_burst = NULL;
-	eth_dev->tx_pkt_burst = NULL;
-
 	return 0;
 }
 
diff --git a/drivers/net/netvsc/hn_ethdev.c b/drivers/net/netvsc/hn_ethdev.c
index be3713dfa0..8743e6e69a 100644
--- a/drivers/net/netvsc/hn_ethdev.c
+++ b/drivers/net/netvsc/hn_ethdev.c
@@ -1050,10 +1050,6 @@ eth_hn_dev_uninit(struct rte_eth_dev *eth_dev)
 	hn_dev_stop(eth_dev);
 	hn_dev_close(eth_dev);
 
-	eth_dev->dev_ops = NULL;
-	eth_dev->tx_pkt_burst = NULL;
-	eth_dev->rx_pkt_burst = NULL;
-
 	hn_detach(hv);
 	hn_chim_uninit(eth_dev);
 	rte_vmbus_chan_close(hv->primary->chan);
diff --git a/drivers/net/octeontx/octeontx_ethdev.c b/drivers/net/octeontx/octeontx_ethdev.c
index aa9ef3bb70..0ea6de448a 100644
--- a/drivers/net/octeontx/octeontx_ethdev.c
+++ b/drivers/net/octeontx/octeontx_ethdev.c
@@ -513,9 +513,6 @@ octeontx_dev_close(struct rte_eth_dev *dev)
 
 	octeontx_port_close(nic);
 
-	dev->tx_pkt_burst = NULL;
-	dev->rx_pkt_burst = NULL;
-
 	return 0;
 }
 
diff --git a/drivers/net/qede/qede_ethdev.c b/drivers/net/qede/qede_ethdev.c
index 88389c9e4f..0dd2f78820 100644
--- a/drivers/net/qede/qede_ethdev.c
+++ b/drivers/net/qede/qede_ethdev.c
@@ -1594,10 +1594,6 @@ static int qede_dev_close(struct rte_eth_dev *eth_dev)
 	if (ECORE_IS_CMT(edev))
 		rte_eal_alarm_cancel(qede_poll_sp_sb_cb, (void *)eth_dev);
 
-	eth_dev->dev_ops = NULL;
-	eth_dev->rx_pkt_burst = NULL;
-	eth_dev->tx_pkt_burst = NULL;
-
 	return 0;
 }
 
diff --git a/drivers/net/sfc/sfc_ethdev.c b/drivers/net/sfc/sfc_ethdev.c
index f41d0f5fe2..620e541a10 100644
--- a/drivers/net/sfc/sfc_ethdev.c
+++ b/drivers/net/sfc/sfc_ethdev.c
@@ -324,11 +324,7 @@ static void
 sfc_eth_dev_secondary_clear_ops(struct rte_eth_dev *dev)
 {
 	free(dev->process_private);
-	dev->process_private = NULL;
-	dev->dev_ops = NULL;
-	dev->tx_pkt_prepare = NULL;
-	dev->tx_pkt_burst = NULL;
-	dev->rx_pkt_burst = NULL;
+	rte_eth_dev_release_port(dev);
 }
 
 static int
@@ -380,7 +376,6 @@ sfc_dev_close(struct rte_eth_dev *dev)
 	/* Required for logging, so cleanup last */
 	sa->eth_dev = NULL;
 
-	dev->process_private = NULL;
 	free(sa);
 
 	return 0;
diff --git a/drivers/net/tap/rte_eth_tap.c b/drivers/net/tap/rte_eth_tap.c
index b127ce62dc..0598de00ed 100644
--- a/drivers/net/tap/rte_eth_tap.c
+++ b/drivers/net/tap/rte_eth_tap.c
@@ -1134,7 +1134,6 @@ tap_dev_close(struct rte_eth_dev *dev)
 		internals->ioctl_sock = -1;
 	}
 	rte_free(dev->process_private);
-	dev->process_private = NULL;
 	if (tap_devices_count == 1)
 		rte_mp_action_unregister(TAP_MP_KEY);
 	tap_devices_count--;
diff --git a/drivers/net/virtio/virtio_ethdev.c b/drivers/net/virtio/virtio_ethdev.c
index ea6496acce..8683763aa8 100644
--- a/drivers/net/virtio/virtio_ethdev.c
+++ b/drivers/net/virtio/virtio_ethdev.c
@@ -2001,10 +2001,6 @@ eth_virtio_dev_uninit(struct rte_eth_dev *eth_dev)
 	virtio_dev_stop(eth_dev);
 	virtio_dev_close(eth_dev);
 
-	eth_dev->dev_ops = NULL;
-	eth_dev->tx_pkt_burst = NULL;
-	eth_dev->rx_pkt_burst = NULL;
-
 	PMD_INIT_LOG(DEBUG, "dev_uninit completed");
 
 	return 0;
diff --git a/drivers/net/vmxnet3/vmxnet3_ethdev.c b/drivers/net/vmxnet3/vmxnet3_ethdev.c
index 828043a65a..1d070fbd79 100644
--- a/drivers/net/vmxnet3/vmxnet3_ethdev.c
+++ b/drivers/net/vmxnet3/vmxnet3_ethdev.c
@@ -368,11 +368,6 @@ eth_vmxnet3_dev_uninit(struct rte_eth_dev *eth_dev)
 		return -EBUSY;
 	}
 
-	eth_dev->dev_ops = NULL;
-	eth_dev->rx_pkt_burst = NULL;
-	eth_dev->tx_pkt_burst = NULL;
-	eth_dev->tx_pkt_prepare = NULL;
-
 	return 0;
 }
 
diff --git a/lib/librte_ethdev/rte_ethdev.c b/lib/librte_ethdev/rte_ethdev.c
index 0b8e8e3e8d..7b33faa58b 100644
--- a/lib/librte_ethdev/rte_ethdev.c
+++ b/lib/librte_ethdev/rte_ethdev.c
@@ -556,7 +556,16 @@ rte_eth_dev_release_port(struct rte_eth_dev *eth_dev)
 
 	eth_dev->state = RTE_ETH_DEV_UNUSED;
 	eth_dev->device = NULL;
+	eth_dev->process_private = NULL;
 	eth_dev->intr_handle = NULL;
+	eth_dev->rx_pkt_burst = NULL;
+	eth_dev->tx_pkt_burst = NULL;
+	eth_dev->tx_pkt_prepare = NULL;
+	eth_dev->rx_queue_count = NULL;
+	eth_dev->rx_descriptor_done = NULL;
+	eth_dev->rx_descriptor_status = NULL;
+	eth_dev->tx_descriptor_status = NULL;
+	eth_dev->dev_ops = NULL;
 
 	if (rte_eal_process_type() == RTE_PROC_PRIMARY) {
 		rte_free(eth_dev->data->rx_queues);
-- 
2.28.0

[dpdk-dev] [PATCH v4 3/3] ethdev: allow close function to return an error

[Thomas Monjalon]

The API function rte_eth_dev_close() was returning void.
The return type is changed to int for notifying of errors.

If an error happens during a close operation,
the status of the port is undefined,
a maximum of resources having been freed.

Signed-off-by: Thomas Monjalon <thomas@monjalon.net>
Reviewed-by: Liron Himi <lironh@marvell.com>
Acked-by: Stephen Hemminger <stephen@networkplumber.org>
---
 doc/guides/rel_notes/deprecation.rst    |  1 -
 doc/guides/rel_notes/release_20_11.rst  |  4 +++-
 drivers/net/cxgbe/cxgbe_ethdev.c        |  5 +++--
 drivers/net/cxgbe/cxgbevf_ethdev.c      |  5 +++--
 drivers/net/failsafe/failsafe_ether.c   |  6 +++++-
 drivers/net/failsafe/failsafe_ops.c     | 25 +++++++++++++++++--------
 drivers/net/memif/rte_eth_memif.c       |  4 +---
 drivers/net/mlx5/mlx5.c                 |  7 ++++---
 drivers/net/mvneta/mvneta_ethdev.c      |  5 +++--
 drivers/net/mvpp2/mrvl_ethdev.c         |  5 +++--
 drivers/net/netvsc/hn_ethdev.c          |  6 ++++--
 drivers/net/netvsc/hn_var.h             |  2 +-
 drivers/net/netvsc/hn_vf.c              |  7 +++++--
 drivers/net/virtio/virtio_user_ethdev.c |  4 +---
 lib/librte_ethdev/rte_ethdev.c          | 16 +++++++++++-----
 lib/librte_ethdev/rte_ethdev.h          |  5 ++++-
 16 files changed, 68 insertions(+), 39 deletions(-)

diff --git a/doc/guides/rel_notes/deprecation.rst b/doc/guides/rel_notes/deprecation.rst
index 0be208edca..e0b302e080 100644
--- a/doc/guides/rel_notes/deprecation.rst
+++ b/doc/guides/rel_notes/deprecation.rst
@@ -135,7 +135,6 @@ Deprecation Notices
   invalid port ID, unsupported operation, failed operation):
 
   - ``rte_eth_dev_stop``
-  - ``rte_eth_dev_close``
 
 * ethdev: New offload flags ``DEV_RX_OFFLOAD_FLOW_MARK`` will be added in 19.11.
   This will allow application to enable or disable PMDs from updating
diff --git a/doc/guides/rel_notes/release_20_11.rst b/doc/guides/rel_notes/release_20_11.rst
index 4e61431c6c..617e54e981 100644
--- a/doc/guides/rel_notes/release_20_11.rst
+++ b/doc/guides/rel_notes/release_20_11.rst
@@ -189,7 +189,9 @@ API Changes
 
 * ethdev: ``rte_eth_rx_descriptor_done()`` API has been deprecated.
 
-* Renamed internal ethdev APIs:
+* ethdev: Added ``int`` return type to ``rte_eth_dev_close()``.
+
+* ethdev: Renamed internal functions:
 
   * ``_rte_eth_dev_callback_process()`` -> ``rte_eth_dev_callback_process()``
   * ``_rte_eth_dev_reset`` -> ``rte_eth_dev_internal_reset()``
diff --git a/drivers/net/cxgbe/cxgbe_ethdev.c b/drivers/net/cxgbe/cxgbe_ethdev.c
index 16beb2d435..fe488231a7 100644
--- a/drivers/net/cxgbe/cxgbe_ethdev.c
+++ b/drivers/net/cxgbe/cxgbe_ethdev.c
@@ -1296,12 +1296,13 @@ static int eth_cxgbe_dev_uninit(struct rte_eth_dev *eth_dev)
 {
 	struct rte_pci_device *pci_dev = RTE_ETH_DEV_TO_PCI(eth_dev);
 	uint16_t port_id;
+	int err = 0;
 
 	/* Free up other ports and all resources */
 	RTE_ETH_FOREACH_DEV_OF(port_id, &pci_dev->device)
-		rte_eth_dev_close(port_id);
+		err |= rte_eth_dev_close(port_id);
 
-	return 0;
+	return err == 0 ? 0 : -EIO;
 }
 
 static int eth_cxgbe_pci_probe(struct rte_pci_driver *pci_drv __rte_unused,
diff --git a/drivers/net/cxgbe/cxgbevf_ethdev.c b/drivers/net/cxgbe/cxgbevf_ethdev.c
index 947fcdd406..c2918f5356 100644
--- a/drivers/net/cxgbe/cxgbevf_ethdev.c
+++ b/drivers/net/cxgbe/cxgbevf_ethdev.c
@@ -183,12 +183,13 @@ static int eth_cxgbevf_dev_uninit(struct rte_eth_dev *eth_dev)
 {
 	struct rte_pci_device *pci_dev = RTE_ETH_DEV_TO_PCI(eth_dev);
 	uint16_t port_id;
+	int err = 0;
 
 	/* Free up other ports and all resources */
 	RTE_ETH_FOREACH_DEV_OF(port_id, &pci_dev->device)
-		rte_eth_dev_close(port_id);
+		err |= rte_eth_dev_close(port_id);
 
-	return 0;
+	return err == 0 ? 0 : -EIO;
 }
 
 static int eth_cxgbevf_pci_probe(struct rte_pci_driver *pci_drv __rte_unused,
diff --git a/drivers/net/failsafe/failsafe_ether.c b/drivers/net/failsafe/failsafe_ether.c
index 7c68bbdec0..950d35dac4 100644
--- a/drivers/net/failsafe/failsafe_ether.c
+++ b/drivers/net/failsafe/failsafe_ether.c
@@ -287,7 +287,11 @@ fs_dev_remove(struct sub_device *sdev)
 		/* fallthrough */
 	case DEV_ACTIVE:
 		failsafe_eth_dev_unregister_callbacks(sdev);
-		rte_eth_dev_close(PORT_ID(sdev));
+		ret = rte_eth_dev_close(PORT_ID(sdev));
+		if (ret < 0) {
+			ERROR("Port close failed for sub-device %u",
+			      PORT_ID(sdev));
+		}
 		sdev->state = DEV_PROBED;
 		/* fallthrough */
 	case DEV_PROBED:
diff --git a/drivers/net/failsafe/failsafe_ops.c b/drivers/net/failsafe/failsafe_ops.c
index 0ce7dfc8a6..5c606ff077 100644
--- a/drivers/net/failsafe/failsafe_ops.c
+++ b/drivers/net/failsafe/failsafe_ops.c
@@ -638,7 +638,7 @@ failsafe_eth_dev_close(struct rte_eth_dev *dev)
 {
 	struct sub_device *sdev;
 	uint8_t i;
-	int ret;
+	int err, ret = 0;
 
 	fs_lock(dev, 0);
 	failsafe_hotplug_alarm_cancel(dev);
@@ -648,29 +648,38 @@ failsafe_eth_dev_close(struct rte_eth_dev *dev)
 	FOREACH_SUBDEV_STATE(sdev, i, dev, DEV_ACTIVE) {
 		DEBUG("Closing sub_device %d", i);
 		failsafe_eth_dev_unregister_callbacks(sdev);
-		rte_eth_dev_close(PORT_ID(sdev));
+		err = rte_eth_dev_close(PORT_ID(sdev));
+		if (err) {
+			ret = ret ? ret : err;
+			ERROR("Error while closing sub-device %u",
+					PORT_ID(sdev));
+		}
 		sdev->state = DEV_ACTIVE - 1;
 	}
 	rte_eth_dev_callback_unregister(RTE_ETH_ALL, RTE_ETH_EVENT_NEW,
 					failsafe_eth_new_event_callback, dev);
 	if (rte_eal_process_type() != RTE_PROC_PRIMARY) {
 		fs_unlock(dev, 0);
-		return 0;
+		return ret;
 	}
 	fs_dev_free_queues(dev);
-	ret = failsafe_eal_uninit(dev);
-	if (ret)
+	err = failsafe_eal_uninit(dev);
+	if (err) {
+		ret = ret ? ret : err;
 		ERROR("Error while uninitializing sub-EAL");
+	}
 	failsafe_args_free(dev);
 	rte_free(PRIV(dev)->subs);
 	rte_free(PRIV(dev)->mcast_addrs);
 	/* mac_addrs must not be freed alone because part of dev_private */
 	dev->data->mac_addrs = NULL;
 	fs_unlock(dev, 0);
-	ret = pthread_mutex_destroy(&PRIV(dev)->hotplug_mutex);
-	if (ret)
+	err = pthread_mutex_destroy(&PRIV(dev)->hotplug_mutex);
+	if (err) {
+		ret = ret ? ret : err;
 		ERROR("Error while destroying hotplug mutex");
-	return 0;
+	}
+	return ret;
 }
 
 static int
diff --git a/drivers/net/memif/rte_eth_memif.c b/drivers/net/memif/rte_eth_memif.c
index ff8a58081f..33bf5c68a3 100644
--- a/drivers/net/memif/rte_eth_memif.c
+++ b/drivers/net/memif/rte_eth_memif.c
@@ -1798,9 +1798,7 @@ rte_pmd_memif_remove(struct rte_vdev_device *vdev)
 	if (eth_dev == NULL)
 		return 0;
 
-	rte_eth_dev_close(eth_dev->data->port_id);
-
-	return 0;
+	return rte_eth_dev_close(eth_dev->data->port_id);
 }
 
 static struct rte_vdev_driver pmd_memif_drv = {
diff --git a/drivers/net/mlx5/mlx5.c b/drivers/net/mlx5/mlx5.c
index 01ead6e6af..2e2f0b274c 100644
--- a/drivers/net/mlx5/mlx5.c
+++ b/drivers/net/mlx5/mlx5.c
@@ -2010,6 +2010,7 @@ static int
 mlx5_pci_remove(struct rte_pci_device *pci_dev)
 {
 	uint16_t port_id;
+	int ret = 0;
 
 	RTE_ETH_FOREACH_DEV_OF(port_id, &pci_dev->device) {
 		/*
@@ -2017,11 +2018,11 @@ mlx5_pci_remove(struct rte_pci_device *pci_dev)
 		 * call the close function explicitly for secondary process.
 		 */
 		if (rte_eal_process_type() == RTE_PROC_SECONDARY)
-			mlx5_dev_close(&rte_eth_devices[port_id]);
+			ret |= mlx5_dev_close(&rte_eth_devices[port_id]);
 		else
-			rte_eth_dev_close(port_id);
+			ret |= rte_eth_dev_close(port_id);
 	}
-	return 0;
+	return ret == 0 ? 0 : -EIO;
 }
 
 static const struct rte_pci_id mlx5_pci_id_map[] = {
diff --git a/drivers/net/mvneta/mvneta_ethdev.c b/drivers/net/mvneta/mvneta_ethdev.c
index 3c0332ab4d..1574bf35a8 100644
--- a/drivers/net/mvneta/mvneta_ethdev.c
+++ b/drivers/net/mvneta/mvneta_ethdev.c
@@ -966,14 +966,15 @@ static int
 rte_pmd_mvneta_remove(struct rte_vdev_device *vdev)
 {
 	uint16_t port_id;
+	int ret = 0;
 
 	RTE_ETH_FOREACH_DEV(port_id) {
 		if (rte_eth_devices[port_id].device != &vdev->device)
 			continue;
-		rte_eth_dev_close(port_id);
+		ret |= rte_eth_dev_close(port_id);
 	}
 
-	return 0;
+	return ret == 0 ? 0 : -EIO;
 }
 
 static struct rte_vdev_driver pmd_mvneta_drv = {
diff --git a/drivers/net/mvpp2/mrvl_ethdev.c b/drivers/net/mvpp2/mrvl_ethdev.c
index a230a96840..df9f336fab 100644
--- a/drivers/net/mvpp2/mrvl_ethdev.c
+++ b/drivers/net/mvpp2/mrvl_ethdev.c
@@ -3022,14 +3022,15 @@ static int
 rte_pmd_mrvl_remove(struct rte_vdev_device *vdev)
 {
 	uint16_t port_id;
+	int ret = 0;
 
 	RTE_ETH_FOREACH_DEV(port_id) {
 		if (rte_eth_devices[port_id].device != &vdev->device)
 			continue;
-		rte_eth_dev_close(port_id);
+		ret |= rte_eth_dev_close(port_id);
 	}
 
-	return 0;
+	return ret == 0 ? 0 : -EIO;
 }
 
 static struct rte_vdev_driver pmd_mrvl_drv = {
diff --git a/drivers/net/netvsc/hn_ethdev.c b/drivers/net/netvsc/hn_ethdev.c
index 8743e6e69a..40e6d25f34 100644
--- a/drivers/net/netvsc/hn_ethdev.c
+++ b/drivers/net/netvsc/hn_ethdev.c
@@ -842,14 +842,16 @@ hn_dev_stop(struct rte_eth_dev *dev)
 static int
 hn_dev_close(struct rte_eth_dev *dev)
 {
+	int ret;
+
 	PMD_INIT_FUNC_TRACE();
 	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
 		return 0;
 
-	hn_vf_close(dev);
+	ret = hn_vf_close(dev);
 	hn_dev_free_queues(dev);
 
-	return 0;
+	return ret;
 }
 
 static const struct eth_dev_ops hn_eth_dev_ops = {
diff --git a/drivers/net/netvsc/hn_var.h b/drivers/net/netvsc/hn_var.h
index 4b63f87607..74f30669ac 100644
--- a/drivers/net/netvsc/hn_var.h
+++ b/drivers/net/netvsc/hn_var.h
@@ -217,7 +217,7 @@ const uint32_t *hn_vf_supported_ptypes(struct rte_eth_dev *dev);
 int	hn_vf_start(struct rte_eth_dev *dev);
 void	hn_vf_reset(struct rte_eth_dev *dev);
 void	hn_vf_stop(struct rte_eth_dev *dev);
-void	hn_vf_close(struct rte_eth_dev *dev);
+int	hn_vf_close(struct rte_eth_dev *dev);
 
 int	hn_vf_allmulticast_enable(struct rte_eth_dev *dev);
 int	hn_vf_allmulticast_disable(struct rte_eth_dev *dev);
diff --git a/drivers/net/netvsc/hn_vf.c b/drivers/net/netvsc/hn_vf.c
index f5f15c0462..d29eee7627 100644
--- a/drivers/net/netvsc/hn_vf.c
+++ b/drivers/net/netvsc/hn_vf.c
@@ -316,18 +316,21 @@ void hn_vf_reset(struct rte_eth_dev *dev)
 	VF_ETHDEV_FUNC(dev, rte_eth_dev_reset);
 }
 
-void hn_vf_close(struct rte_eth_dev *dev)
+int hn_vf_close(struct rte_eth_dev *dev)
 {
 	struct hn_data *hv = dev->data->dev_private;
 	uint16_t vf_port;
+	int ret = 0;
 
 	rte_rwlock_read_lock(&hv->vf_lock);
 	vf_port = hv->vf_port;
 	if (vf_port != HN_INVALID_PORT)
-		rte_eth_dev_close(vf_port);
+		ret = rte_eth_dev_close(vf_port);
 
 	hv->vf_port = HN_INVALID_PORT;
 	rte_rwlock_read_unlock(&hv->vf_lock);
+
+	return ret;
 }
 
 int hn_vf_stats_reset(struct rte_eth_dev *dev)
diff --git a/drivers/net/virtio/virtio_user_ethdev.c b/drivers/net/virtio/virtio_user_ethdev.c
index f56fc238c4..042665bc09 100644
--- a/drivers/net/virtio/virtio_user_ethdev.c
+++ b/drivers/net/virtio/virtio_user_ethdev.c
@@ -881,9 +881,7 @@ virtio_user_pmd_remove(struct rte_vdev_device *vdev)
 		return rte_eth_dev_release_port(eth_dev);
 
 	/* make sure the device is stopped, queues freed */
-	rte_eth_dev_close(eth_dev->data->port_id);
-
-	return 0;
+	return rte_eth_dev_close(eth_dev->data->port_id);
 }
 
 static int virtio_user_pmd_dma_map(struct rte_vdev_device *vdev, void *addr,
diff --git a/lib/librte_ethdev/rte_ethdev.c b/lib/librte_ethdev/rte_ethdev.c
index 7b33faa58b..8331273370 100644
--- a/lib/librte_ethdev/rte_ethdev.c
+++ b/lib/librte_ethdev/rte_ethdev.c
@@ -1716,19 +1716,25 @@ rte_eth_dev_set_link_down(uint16_t port_id)
 	return eth_err(port_id, (*dev->dev_ops->dev_set_link_down)(dev));
 }
 
-void
+int
 rte_eth_dev_close(uint16_t port_id)
 {
 	struct rte_eth_dev *dev;
+	int firsterr, binerr;
+	int *lasterr = &firsterr;
 
-	RTE_ETH_VALID_PORTID_OR_RET(port_id);
+	RTE_ETH_VALID_PORTID_OR_ERR_RET(port_id, -EINVAL);
 	dev = &rte_eth_devices[port_id];
 
-	RTE_FUNC_PTR_OR_RET(*dev->dev_ops->dev_close);
-	(*dev->dev_ops->dev_close)(dev);
+	RTE_FUNC_PTR_OR_ERR_RET(*dev->dev_ops->dev_close, -ENOTSUP);
+	*lasterr = (*dev->dev_ops->dev_close)(dev);
+	if (*lasterr != 0)
+		lasterr = &binerr;
 
 	rte_ethdev_trace_close(port_id);
-	rte_eth_dev_release_port(dev);
+	*lasterr = rte_eth_dev_release_port(dev);
+
+	return firsterr;
 }
 
 int
diff --git a/lib/librte_ethdev/rte_ethdev.h b/lib/librte_ethdev/rte_ethdev.h
index d2bf74f128..92f03d9778 100644
--- a/lib/librte_ethdev/rte_ethdev.h
+++ b/lib/librte_ethdev/rte_ethdev.h
@@ -2281,8 +2281,11 @@ int rte_eth_dev_set_link_down(uint16_t port_id);
  *
  * @param port_id
  *   The port identifier of the Ethernet device.
+ * @return
+ *   - Zero if the port is closed successfully.
+ *   - Negative if something went wrong.
  */
-void rte_eth_dev_close(uint16_t port_id);
+int rte_eth_dev_close(uint16_t port_id);
 
 /**
  * Reset a Ethernet device and keep its port id.
-- 
2.28.0

Re: [dpdk-dev] [PATCH v4 3/3] ethdev: allow close function to return an error

[Ferruh Yigit]

On 10/5/2020 6:08 PM, Thomas Monjalon wrote:
> The API function rte_eth_dev_close() was returning void.
> The return type is changed to int for notifying of errors.
> 
> If an error happens during a close operation,
> the status of the port is undefined,
> a maximum of resources having been freed.
> 
> Signed-off-by: Thomas Monjalon <thomas@monjalon.net>
> Reviewed-by: Liron Himi <lironh@marvell.com>
> Acked-by: Stephen Hemminger <stephen@networkplumber.org>

<...>

> -void
> +int
>   rte_eth_dev_close(uint16_t port_id)
>   {
>   	struct rte_eth_dev *dev;
> +	int firsterr, binerr;
> +	int *lasterr = &firsterr;
>   
> -	RTE_ETH_VALID_PORTID_OR_RET(port_id);
> +	RTE_ETH_VALID_PORTID_OR_ERR_RET(port_id, -EINVAL);
>   	dev = &rte_eth_devices[port_id];
>   
> -	RTE_FUNC_PTR_OR_RET(*dev->dev_ops->dev_close);
> -	(*dev->dev_ops->dev_close)(dev);
> +	RTE_FUNC_PTR_OR_ERR_RET(*dev->dev_ops->dev_close, -ENOTSUP);
> +	*lasterr = (*dev->dev_ops->dev_close)(dev);
> +	if (*lasterr != 0)
> +		lasterr = &binerr;
>   
>   	rte_ethdev_trace_close(port_id);
> -	rte_eth_dev_release_port(dev);
> +	*lasterr = rte_eth_dev_release_port(dev);
> +
> +	return firsterr;
>   }

This may be personal taste but above error handling looks like unnecessary 
complex, what do you think something like below:

close_err = (*dev->dev_ops->dev_close)(dev);
release_err = rte_eth_dev_release_port(dev);
return close_err ? close_err : release_err;

Re: [dpdk-dev] [PATCH v4 3/3] ethdev: allow close function to return an error

[Thomas Monjalon]

06/10/2020 11:43, Ferruh Yigit:
> On 10/5/2020 6:08 PM, Thomas Monjalon wrote:
> > The API function rte_eth_dev_close() was returning void.
> > The return type is changed to int for notifying of errors.
> > 
> > If an error happens during a close operation,
> > the status of the port is undefined,
> > a maximum of resources having been freed.
> > 
> > Signed-off-by: Thomas Monjalon <thomas@monjalon.net>
> > Reviewed-by: Liron Himi <lironh@marvell.com>
> > Acked-by: Stephen Hemminger <stephen@networkplumber.org>
> 
> <...>
> 
> > -void
> > +int
> >   rte_eth_dev_close(uint16_t port_id)
> >   {
> >   	struct rte_eth_dev *dev;
> > +	int firsterr, binerr;
> > +	int *lasterr = &firsterr;
> >   
> > -	RTE_ETH_VALID_PORTID_OR_RET(port_id);
> > +	RTE_ETH_VALID_PORTID_OR_ERR_RET(port_id, -EINVAL);
> >   	dev = &rte_eth_devices[port_id];
> >   
> > -	RTE_FUNC_PTR_OR_RET(*dev->dev_ops->dev_close);
> > -	(*dev->dev_ops->dev_close)(dev);
> > +	RTE_FUNC_PTR_OR_ERR_RET(*dev->dev_ops->dev_close, -ENOTSUP);
> > +	*lasterr = (*dev->dev_ops->dev_close)(dev);
> > +	if (*lasterr != 0)
> > +		lasterr = &binerr;
> >   
> >   	rte_ethdev_trace_close(port_id);
> > -	rte_eth_dev_release_port(dev);
> > +	*lasterr = rte_eth_dev_release_port(dev);
> > +
> > +	return firsterr;
> >   }
> 
> This may be personal taste but above error handling looks like unnecessary 
> complex, what do you think something like below:
> 
> close_err = (*dev->dev_ops->dev_close)(dev);
> release_err = rte_eth_dev_release_port(dev);
> return close_err ? close_err : release_err;

This is what I did first. Then thought about this "elegant" handling.
The pointer solution is just one more line and is more future proof.

I'm fine with any choice. Andrew?

Re: [dpdk-dev] [PATCH v4 3/3] ethdev: allow close function to return an error

[Andrew Rybchenko]

On 10/6/20 1:57 PM, Thomas Monjalon wrote:
> 06/10/2020 11:43, Ferruh Yigit:
>> On 10/5/2020 6:08 PM, Thomas Monjalon wrote:
>>> The API function rte_eth_dev_close() was returning void.
>>> The return type is changed to int for notifying of errors.
>>>
>>> If an error happens during a close operation,
>>> the status of the port is undefined,
>>> a maximum of resources having been freed.
>>>
>>> Signed-off-by: Thomas Monjalon <thomas@monjalon.net>
>>> Reviewed-by: Liron Himi <lironh@marvell.com>
>>> Acked-by: Stephen Hemminger <stephen@networkplumber.org>
>>
>> <...>
>>
>>> -void
>>> +int
>>>   rte_eth_dev_close(uint16_t port_id)
>>>   {
>>>   	struct rte_eth_dev *dev;
>>> +	int firsterr, binerr;
>>> +	int *lasterr = &firsterr;
>>>   
>>> -	RTE_ETH_VALID_PORTID_OR_RET(port_id);
>>> +	RTE_ETH_VALID_PORTID_OR_ERR_RET(port_id, -EINVAL);

Shouldn't it be -ENODEV in fact?
IMHO it would be nice to make it uniform across ethdev API.

>>>   	dev = &rte_eth_devices[port_id];
>>>   
>>> -	RTE_FUNC_PTR_OR_RET(*dev->dev_ops->dev_close);
>>> -	(*dev->dev_ops->dev_close)(dev);
>>> +	RTE_FUNC_PTR_OR_ERR_RET(*dev->dev_ops->dev_close, -ENOTSUP);
>>> +	*lasterr = (*dev->dev_ops->dev_close)(dev);
>>> +	if (*lasterr != 0)
>>> +		lasterr = &binerr;
>>>   
>>>   	rte_ethdev_trace_close(port_id);
>>> -	rte_eth_dev_release_port(dev);
>>> +	*lasterr = rte_eth_dev_release_port(dev);
>>> +
>>> +	return firsterr;
>>>   }
>>
>> This may be personal taste but above error handling looks like unnecessary 
>> complex, what do you think something like below:
>>
>> close_err = (*dev->dev_ops->dev_close)(dev);
>> release_err = rte_eth_dev_release_port(dev);
>> return close_err ? close_err : release_err;
> 
> This is what I did first. Then thought about this "elegant" handling.
> The pointer solution is just one more line and is more future proof.
> 
> I'm fine with any choice. Andrew?
> 

Hm, really hard to make a choice. I tend to choose Thomas's
version. Yes, I agree that it is a bit over-complicated in
this particular case, but thoughts to be future-proof
overweight for me. Plus it adds a pattern on how to handle
such cases in the future.

Re: [dpdk-dev] [PATCH v4 3/3] ethdev: allow close function to return an error

[Thomas Monjalon]

13/10/2020 10:40, Andrew Rybchenko:
> On 10/6/20 1:57 PM, Thomas Monjalon wrote:
> > 06/10/2020 11:43, Ferruh Yigit:
> >> On 10/5/2020 6:08 PM, Thomas Monjalon wrote:
> >>> The API function rte_eth_dev_close() was returning void.
> >>> The return type is changed to int for notifying of errors.
> >>>
> >>> If an error happens during a close operation,
> >>> the status of the port is undefined,
> >>> a maximum of resources having been freed.
> >>>
> >>> Signed-off-by: Thomas Monjalon <thomas@monjalon.net>
> >>> Reviewed-by: Liron Himi <lironh@marvell.com>
> >>> Acked-by: Stephen Hemminger <stephen@networkplumber.org>
> >>
> >> <...>
> >>
> >>> -void
> >>> +int
> >>>   rte_eth_dev_close(uint16_t port_id)
> >>>   {
> >>>   	struct rte_eth_dev *dev;
> >>> +	int firsterr, binerr;
> >>> +	int *lasterr = &firsterr;
> >>>   
> >>> -	RTE_ETH_VALID_PORTID_OR_RET(port_id);
> >>> +	RTE_ETH_VALID_PORTID_OR_ERR_RET(port_id, -EINVAL);
> 
> Shouldn't it be -ENODEV in fact?

Yes you're right.

> IMHO it would be nice to make it uniform across ethdev API.

Yes, do you want to make the patch to change other occurences to ENODEV?


> >>>   	dev = &rte_eth_devices[port_id];
> >>>   
> >>> -	RTE_FUNC_PTR_OR_RET(*dev->dev_ops->dev_close);
> >>> -	(*dev->dev_ops->dev_close)(dev);
> >>> +	RTE_FUNC_PTR_OR_ERR_RET(*dev->dev_ops->dev_close, -ENOTSUP);
> >>> +	*lasterr = (*dev->dev_ops->dev_close)(dev);
> >>> +	if (*lasterr != 0)
> >>> +		lasterr = &binerr;
> >>>   
> >>>   	rte_ethdev_trace_close(port_id);
> >>> -	rte_eth_dev_release_port(dev);
> >>> +	*lasterr = rte_eth_dev_release_port(dev);
> >>> +
> >>> +	return firsterr;
> >>>   }
> >>
> >> This may be personal taste but above error handling looks like unnecessary 
> >> complex, what do you think something like below:
> >>
> >> close_err = (*dev->dev_ops->dev_close)(dev);
> >> release_err = rte_eth_dev_release_port(dev);
> >> return close_err ? close_err : release_err;
> > 
> > This is what I did first. Then thought about this "elegant" handling.
> > The pointer solution is just one more line and is more future proof.
> > 
> > I'm fine with any choice. Andrew?
> > 
> 
> Hm, really hard to make a choice. I tend to choose Thomas's
> version. Yes, I agree that it is a bit over-complicated in
> this particular case, but thoughts to be future-proof
> overweight for me. Plus it adds a pattern on how to handle
> such cases in the future.

Yes it's an elegant pattern :)

Re: [dpdk-dev] [PATCH v4 3/3] ethdev: allow close function to return an error

[Ferruh Yigit]

On 10/13/2020 9:55 AM, Thomas Monjalon wrote:

<...>

>>>>>    	dev = &rte_eth_devices[port_id];
>>>>>    
>>>>> -	RTE_FUNC_PTR_OR_RET(*dev->dev_ops->dev_close);
>>>>> -	(*dev->dev_ops->dev_close)(dev);
>>>>> +	RTE_FUNC_PTR_OR_ERR_RET(*dev->dev_ops->dev_close, -ENOTSUP);
>>>>> +	*lasterr = (*dev->dev_ops->dev_close)(dev);
>>>>> +	if (*lasterr != 0)
>>>>> +		lasterr = &binerr;
>>>>>    
>>>>>    	rte_ethdev_trace_close(port_id);
>>>>> -	rte_eth_dev_release_port(dev);
>>>>> +	*lasterr = rte_eth_dev_release_port(dev);
>>>>> +
>>>>> +	return firsterr;
>>>>>    }
>>>>
>>>> This may be personal taste but above error handling looks like unnecessary
>>>> complex, what do you think something like below:
>>>>
>>>> close_err = (*dev->dev_ops->dev_close)(dev);
>>>> release_err = rte_eth_dev_release_port(dev);
>>>> return close_err ? close_err : release_err;
>>>
>>> This is what I did first. Then thought about this "elegant" handling.
>>> The pointer solution is just one more line and is more future proof.
>>>
>>> I'm fine with any choice. Andrew?
>>>
>>
>> Hm, really hard to make a choice. I tend to choose Thomas's
>> version. Yes, I agree that it is a bit over-complicated in
>> this particular case, but thoughts to be future-proof
>> overweight for me. Plus it adds a pattern on how to handle
>> such cases in the future.
> 
> Yes it's an elegant pattern :)
> 

OK :)

[dpdk-dev] [PATCH v5 0/3] cleanup ethdev close operation

[Thomas Monjalon]

Most of the patches from the v3 have been merged already.
The remaining 3 patches are reworked in this v4:
  - reset dev_started in some PMDs
  - selectively reset pointers in rte_eth_dev_release_port()
  - fix error handling

v5: change error code to ENODEV if wrong port id


Thomas Monjalon (3):
  ethdev: remove forcing stopped state upon close
  ethdev: reset all when releasing a port
  ethdev: allow close function to return an error

 doc/guides/rel_notes/deprecation.rst    |  1 -
 doc/guides/rel_notes/release_20_11.rst  |  4 +++-
 drivers/net/ark/ark_ethdev.c            |  4 +---
 drivers/net/atlantic/atl_ethdev.c       |  5 +----
 drivers/net/axgbe/axgbe_ethdev.c        |  3 ---
 drivers/net/bnxt/bnxt_ethdev.c          |  4 ----
 drivers/net/bnxt/bnxt_reps.c            |  1 -
 drivers/net/bonding/rte_eth_bond_pmd.c  |  4 ----
 drivers/net/cxgbe/cxgbe_ethdev.c        |  5 +++--
 drivers/net/cxgbe/cxgbevf_ethdev.c      |  5 +++--
 drivers/net/dpaa/dpaa_ethdev.c          |  5 +----
 drivers/net/dpaa2/dpaa2_ethdev.c        |  4 ----
 drivers/net/e1000/em_ethdev.c           |  6 ++----
 drivers/net/e1000/igb_ethdev.c          |  9 +--------
 drivers/net/ena/ena_ethdev.c            |  6 +-----
 drivers/net/enetc/enetc_ethdev.c        |  1 +
 drivers/net/failsafe/failsafe_ether.c   |  6 +++++-
 drivers/net/failsafe/failsafe_ops.c     | 25 ++++++++++++++++--------
 drivers/net/fm10k/fm10k_ethdev.c        |  5 +----
 drivers/net/hinic/hinic_pmd_ethdev.c    |  6 ++----
 drivers/net/hns3/hns3_ethdev.c          |  5 +----
 drivers/net/hns3/hns3_ethdev_vf.c       |  6 +-----
 drivers/net/i40e/i40e_ethdev.c          |  5 +----
 drivers/net/i40e/i40e_ethdev_vf.c       |  6 +-----
 drivers/net/iavf/iavf_ethdev.c          |  5 +----
 drivers/net/ice/ice_dcf_ethdev.c        |  4 ----
 drivers/net/ice/ice_ethdev.c            |  5 +----
 drivers/net/igc/igc_ethdev.c            |  1 +
 drivers/net/ionic/ionic_ethdev.c        |  5 -----
 drivers/net/ixgbe/ixgbe_ethdev.c        |  9 +--------
 drivers/net/kni/rte_eth_kni.c           |  1 +
 drivers/net/liquidio/lio_ethdev.c       |  5 +----
 drivers/net/memif/rte_eth_memif.c       |  4 +---
 drivers/net/mlx5/mlx5.c                 |  7 ++++---
 drivers/net/mvneta/mvneta_ethdev.c      |  7 +++++--
 drivers/net/mvpp2/mrvl_ethdev.c         |  5 +++--
 drivers/net/netvsc/hn_ethdev.c          | 11 +++++------
 drivers/net/netvsc/hn_var.h             |  2 +-
 drivers/net/netvsc/hn_vf.c              |  7 +++++--
 drivers/net/nfb/nfb_ethdev.c            |  2 ++
 drivers/net/octeontx/octeontx_ethdev.c  |  3 ---
 drivers/net/pfe/pfe_ethdev.c            |  2 ++
 drivers/net/qede/qede_ethdev.c          |  5 +----
 drivers/net/ring/rte_eth_ring.c         |  1 +
 drivers/net/sfc/sfc_ethdev.c            |  7 +------
 drivers/net/szedata2/rte_eth_szedata2.c |  2 ++
 drivers/net/tap/rte_eth_tap.c           |  1 -
 drivers/net/thunderx/nicvf_ethdev.c     |  1 +
 drivers/net/vhost/rte_eth_vhost.c       |  1 +
 drivers/net/virtio/virtio_ethdev.c      |  5 +----
 drivers/net/virtio/virtio_user_ethdev.c |  4 +---
 drivers/net/vmxnet3/vmxnet3_ethdev.c    |  6 +-----
 lib/librte_ethdev/rte_ethdev.c          | 26 +++++++++++++++++++------
 lib/librte_ethdev/rte_ethdev.h          |  5 ++++-
 54 files changed, 114 insertions(+), 166 deletions(-)

-- 
2.28.0

[dpdk-dev] [PATCH v5 1/3] ethdev: remove forcing stopped state upon close

[Thomas Monjalon]

When closing a port, it is supposed to be already stopped,
and marked as such with "dev_started" state zeroed by the stop API.

Resetting "dev_started" before calling the driver close operation
was hiding the case of not properly stopped port being closed.
The flag "dev_started" is not changed anymore in "rte_eth_dev_close()".

In case the "dev_stop" function is called from "dev_close",
bypassing "rte_eth_dev_stop()" API,
the "dev_started" state must be explicitly reset in the PMD
in order to keep the same behaviour.

Signed-off-by: Thomas Monjalon <thomas@monjalon.net>
Acked-by: Stephen Hemminger <stephen@networkplumber.org>
Reviewed-by: Andrew Rybchenko <arybchenko@solarflare.com>
---
 drivers/net/ark/ark_ethdev.c            | 1 +
 drivers/net/atlantic/atl_ethdev.c       | 1 +
 drivers/net/dpaa/dpaa_ethdev.c          | 1 +
 drivers/net/e1000/em_ethdev.c           | 2 ++
 drivers/net/e1000/igb_ethdev.c          | 1 +
 drivers/net/ena/ena_ethdev.c            | 1 +
 drivers/net/enetc/enetc_ethdev.c        | 1 +
 drivers/net/fm10k/fm10k_ethdev.c        | 1 +
 drivers/net/hinic/hinic_pmd_ethdev.c    | 2 ++
 drivers/net/hns3/hns3_ethdev.c          | 1 +
 drivers/net/hns3/hns3_ethdev_vf.c       | 1 +
 drivers/net/i40e/i40e_ethdev.c          | 1 +
 drivers/net/i40e/i40e_ethdev_vf.c       | 2 +-
 drivers/net/iavf/iavf_ethdev.c          | 1 +
 drivers/net/ice/ice_ethdev.c            | 1 +
 drivers/net/igc/igc_ethdev.c            | 1 +
 drivers/net/ixgbe/ixgbe_ethdev.c        | 1 +
 drivers/net/kni/rte_eth_kni.c           | 1 +
 drivers/net/liquidio/lio_ethdev.c       | 1 +
 drivers/net/mvneta/mvneta_ethdev.c      | 2 ++
 drivers/net/netvsc/hn_ethdev.c          | 1 +
 drivers/net/nfb/nfb_ethdev.c            | 2 ++
 drivers/net/pfe/pfe_ethdev.c            | 2 ++
 drivers/net/qede/qede_ethdev.c          | 1 +
 drivers/net/ring/rte_eth_ring.c         | 1 +
 drivers/net/szedata2/rte_eth_szedata2.c | 2 ++
 drivers/net/thunderx/nicvf_ethdev.c     | 1 +
 drivers/net/vhost/rte_eth_vhost.c       | 1 +
 drivers/net/virtio/virtio_ethdev.c      | 1 +
 drivers/net/vmxnet3/vmxnet3_ethdev.c    | 1 +
 lib/librte_ethdev/rte_ethdev.c          | 1 -
 31 files changed, 36 insertions(+), 2 deletions(-)

diff --git a/drivers/net/ark/ark_ethdev.c b/drivers/net/ark/ark_ethdev.c
index e3b1347769..f1355581f1 100644
--- a/drivers/net/ark/ark_ethdev.c
+++ b/drivers/net/ark/ark_ethdev.c
@@ -595,6 +595,7 @@ eth_ark_dev_stop(struct rte_eth_dev *dev)
 	if (ark->started == 0)
 		return;
 	ark->started = 0;
+	dev->data->dev_started = 0;
 
 	/* Stop the extension first */
 	if (ark->user_ext.dev_stop)
diff --git a/drivers/net/atlantic/atl_ethdev.c b/drivers/net/atlantic/atl_ethdev.c
index 2217511ca0..f2e48e1c11 100644
--- a/drivers/net/atlantic/atl_ethdev.c
+++ b/drivers/net/atlantic/atl_ethdev.c
@@ -609,6 +609,7 @@ atl_dev_stop(struct rte_eth_dev *dev)
 	struct rte_intr_handle *intr_handle = &pci_dev->intr_handle;
 
 	PMD_INIT_FUNC_TRACE();
+	dev->data->dev_started = 0;
 
 	/* disable interrupts */
 	atl_disable_intr(hw);
diff --git a/drivers/net/dpaa/dpaa_ethdev.c b/drivers/net/dpaa/dpaa_ethdev.c
index af47c196ae..952e8f72ad 100644
--- a/drivers/net/dpaa/dpaa_ethdev.c
+++ b/drivers/net/dpaa/dpaa_ethdev.c
@@ -420,6 +420,7 @@ static void dpaa_eth_dev_stop(struct rte_eth_dev *dev)
 	struct fman_if *fif = dev->process_private;
 
 	PMD_INIT_FUNC_TRACE();
+	dev->data->dev_started = 0;
 
 	if (!fif->is_shared_mac)
 		fman_if_disable_rx(fif);
diff --git a/drivers/net/e1000/em_ethdev.c b/drivers/net/e1000/em_ethdev.c
index d050eb478a..dd9238a616 100644
--- a/drivers/net/e1000/em_ethdev.c
+++ b/drivers/net/e1000/em_ethdev.c
@@ -717,6 +717,8 @@ eth_em_stop(struct rte_eth_dev *dev)
 	struct rte_pci_device *pci_dev = RTE_ETH_DEV_TO_PCI(dev);
 	struct rte_intr_handle *intr_handle = &pci_dev->intr_handle;
 
+	dev->data->dev_started = 0;
+
 	eth_em_rxtx_control(dev, false);
 	em_rxq_intr_disable(hw);
 	em_lsc_intr_disable(hw);
diff --git a/drivers/net/e1000/igb_ethdev.c b/drivers/net/e1000/igb_ethdev.c
index cb3d97e2a3..76746c9484 100644
--- a/drivers/net/e1000/igb_ethdev.c
+++ b/drivers/net/e1000/igb_ethdev.c
@@ -1497,6 +1497,7 @@ eth_igb_stop(struct rte_eth_dev *dev)
 	}
 
 	adapter->stopped = true;
+	dev->data->dev_started = 0;
 }
 
 static int
diff --git a/drivers/net/ena/ena_ethdev.c b/drivers/net/ena/ena_ethdev.c
index d32fa43837..e02d340c33 100644
--- a/drivers/net/ena/ena_ethdev.c
+++ b/drivers/net/ena/ena_ethdev.c
@@ -1118,6 +1118,7 @@ static void ena_stop(struct rte_eth_dev *dev)
 
 	++adapter->dev_stats.dev_stop;
 	adapter->state = ENA_ADAPTER_STATE_STOPPED;
+	dev->data->dev_started = 0;
 }
 
 static int ena_create_io_queue(struct ena_ring *ring)
diff --git a/drivers/net/enetc/enetc_ethdev.c b/drivers/net/enetc/enetc_ethdev.c
index b3dec7e64d..dddbc89939 100644
--- a/drivers/net/enetc/enetc_ethdev.c
+++ b/drivers/net/enetc/enetc_ethdev.c
@@ -54,6 +54,7 @@ enetc_dev_stop(struct rte_eth_dev *dev)
 	uint32_t val;
 
 	PMD_INIT_FUNC_TRACE();
+	dev->data->dev_started = 0;
 	/* Disable port */
 	val = enetc_port_rd(enetc_hw, ENETC_PMR);
 	enetc_port_wr(enetc_hw, ENETC_PMR, val & (~ENETC_PMR_EN));
diff --git a/drivers/net/fm10k/fm10k_ethdev.c b/drivers/net/fm10k/fm10k_ethdev.c
index c4a6fdf7f0..0c13fd5c37 100644
--- a/drivers/net/fm10k/fm10k_ethdev.c
+++ b/drivers/net/fm10k/fm10k_ethdev.c
@@ -1161,6 +1161,7 @@ fm10k_dev_stop(struct rte_eth_dev *dev)
 	int i;
 
 	PMD_INIT_FUNC_TRACE();
+	dev->data->dev_started = 0;
 
 	if (dev->data->tx_queues)
 		for (i = 0; i < dev->data->nb_tx_queues; i++)
diff --git a/drivers/net/hinic/hinic_pmd_ethdev.c b/drivers/net/hinic/hinic_pmd_ethdev.c
index 466c8362b9..daf8f33cf3 100644
--- a/drivers/net/hinic/hinic_pmd_ethdev.c
+++ b/drivers/net/hinic/hinic_pmd_ethdev.c
@@ -1189,6 +1189,8 @@ static void hinic_dev_stop(struct rte_eth_dev *dev)
 	name = dev->data->name;
 	port_id = dev->data->port_id;
 
+	dev->data->dev_started = 0;
+
 	if (!rte_bit_relaxed_test_and_clear32(HINIC_DEV_START,
 					      &nic_dev->dev_status)) {
 		PMD_DRV_LOG(INFO, "Device %s already stopped", name);
diff --git a/drivers/net/hns3/hns3_ethdev.c b/drivers/net/hns3/hns3_ethdev.c
index ce5bae538d..8fcc0f18c5 100644
--- a/drivers/net/hns3/hns3_ethdev.c
+++ b/drivers/net/hns3/hns3_ethdev.c
@@ -4997,6 +4997,7 @@ hns3_dev_stop(struct rte_eth_dev *dev)
 	struct hns3_hw *hw = &hns->hw;
 
 	PMD_INIT_FUNC_TRACE();
+	dev->data->dev_started = 0;
 
 	hw->adapter_state = HNS3_NIC_STOPPING;
 	hns3_set_rxtx_function(dev);
diff --git a/drivers/net/hns3/hns3_ethdev_vf.c b/drivers/net/hns3/hns3_ethdev_vf.c
index 1a19c0e6e6..b2038395a2 100644
--- a/drivers/net/hns3/hns3_ethdev_vf.c
+++ b/drivers/net/hns3/hns3_ethdev_vf.c
@@ -1952,6 +1952,7 @@ hns3vf_dev_stop(struct rte_eth_dev *dev)
 	struct hns3_hw *hw = &hns->hw;
 
 	PMD_INIT_FUNC_TRACE();
+	dev->data->dev_started = 0;
 
 	hw->adapter_state = HNS3_NIC_STOPPING;
 	hns3_set_rxtx_function(dev);
diff --git a/drivers/net/i40e/i40e_ethdev.c b/drivers/net/i40e/i40e_ethdev.c
index 943cfe71dc..825de51fbb 100644
--- a/drivers/net/i40e/i40e_ethdev.c
+++ b/drivers/net/i40e/i40e_ethdev.c
@@ -2599,6 +2599,7 @@ i40e_dev_stop(struct rte_eth_dev *dev)
 	pf->tm_conf.committed = false;
 
 	hw->adapter_stopped = 1;
+	dev->data->dev_started = 0;
 
 	pf->adapter->rss_reta_updated = 0;
 }
diff --git a/drivers/net/i40e/i40e_ethdev_vf.c b/drivers/net/i40e/i40e_ethdev_vf.c
index 4d6510d1ff..40c9745433 100644
--- a/drivers/net/i40e/i40e_ethdev_vf.c
+++ b/drivers/net/i40e/i40e_ethdev_vf.c
@@ -2207,7 +2207,7 @@ i40evf_dev_stop(struct rte_eth_dev *dev)
 	i40evf_add_del_mc_addr_list(dev, vf->mc_addrs, vf->mc_addrs_num,
 				FALSE);
 	hw->adapter_stopped = 1;
-
+	dev->data->dev_started = 0;
 }
 
 static int
diff --git a/drivers/net/iavf/iavf_ethdev.c b/drivers/net/iavf/iavf_ethdev.c
index f5e6e852ae..8bd2c048a1 100644
--- a/drivers/net/iavf/iavf_ethdev.c
+++ b/drivers/net/iavf/iavf_ethdev.c
@@ -562,6 +562,7 @@ iavf_dev_stop(struct rte_eth_dev *dev)
 				  false);
 
 	adapter->stopped = 1;
+	dev->data->dev_started = 0;
 }
 
 static int
diff --git a/drivers/net/ice/ice_ethdev.c b/drivers/net/ice/ice_ethdev.c
index d8ce09d28f..1db4015c23 100644
--- a/drivers/net/ice/ice_ethdev.c
+++ b/drivers/net/ice/ice_ethdev.c
@@ -2376,6 +2376,7 @@ ice_dev_stop(struct rte_eth_dev *dev)
 	}
 
 	pf->adapter_stopped = true;
+	dev->data->dev_started = 0;
 }
 
 static int
diff --git a/drivers/net/igc/igc_ethdev.c b/drivers/net/igc/igc_ethdev.c
index 7f5066df4b..73b7d20ac2 100644
--- a/drivers/net/igc/igc_ethdev.c
+++ b/drivers/net/igc/igc_ethdev.c
@@ -616,6 +616,7 @@ eth_igc_stop(struct rte_eth_dev *dev)
 	struct rte_intr_handle *intr_handle = &pci_dev->intr_handle;
 	struct rte_eth_link link;
 
+	dev->data->dev_started = 0;
 	adapter->stopped = 1;
 
 	/* disable receive and transmit */
diff --git a/drivers/net/ixgbe/ixgbe_ethdev.c b/drivers/net/ixgbe/ixgbe_ethdev.c
index 0b98e210e7..bd83b0e31a 100644
--- a/drivers/net/ixgbe/ixgbe_ethdev.c
+++ b/drivers/net/ixgbe/ixgbe_ethdev.c
@@ -2917,6 +2917,7 @@ ixgbe_dev_stop(struct rte_eth_dev *dev)
 	adapter->rss_reta_updated = 0;
 
 	hw->adapter_stopped = true;
+	dev->data->dev_started = 0;
 }
 
 /*
diff --git a/drivers/net/kni/rte_eth_kni.c b/drivers/net/kni/rte_eth_kni.c
index be747adf86..329a55039f 100644
--- a/drivers/net/kni/rte_eth_kni.c
+++ b/drivers/net/kni/rte_eth_kni.c
@@ -196,6 +196,7 @@ eth_kni_dev_stop(struct rte_eth_dev *dev)
 	}
 
 	dev->data->dev_link.link_status = 0;
+	dev->data->dev_started = 0;
 }
 
 static int
diff --git a/drivers/net/liquidio/lio_ethdev.c b/drivers/net/liquidio/lio_ethdev.c
index 1a41f27198..52bcefc819 100644
--- a/drivers/net/liquidio/lio_ethdev.c
+++ b/drivers/net/liquidio/lio_ethdev.c
@@ -1471,6 +1471,7 @@ lio_dev_stop(struct rte_eth_dev *eth_dev)
 	struct lio_device *lio_dev = LIO_DEV(eth_dev);
 
 	lio_dev_info(lio_dev, "Stopping port %d\n", eth_dev->data->port_id);
+	eth_dev->data->dev_started = 0;
 	lio_dev->intf_open = 0;
 	rte_mb();
 
diff --git a/drivers/net/mvneta/mvneta_ethdev.c b/drivers/net/mvneta/mvneta_ethdev.c
index 607771149a..3c0332ab4d 100644
--- a/drivers/net/mvneta/mvneta_ethdev.c
+++ b/drivers/net/mvneta/mvneta_ethdev.c
@@ -413,6 +413,8 @@ mvneta_dev_stop(struct rte_eth_dev *dev)
 {
 	struct mvneta_priv *priv = dev->data->dev_private;
 
+	dev->data->dev_started = 0;
+
 	if (!priv->ppio)
 		return;
 
diff --git a/drivers/net/netvsc/hn_ethdev.c b/drivers/net/netvsc/hn_ethdev.c
index 5ae2d469c8..be3713dfa0 100644
--- a/drivers/net/netvsc/hn_ethdev.c
+++ b/drivers/net/netvsc/hn_ethdev.c
@@ -833,6 +833,7 @@ hn_dev_stop(struct rte_eth_dev *dev)
 	struct hn_data *hv = dev->data->dev_private;
 
 	PMD_INIT_FUNC_TRACE();
+	dev->data->dev_started = 0;
 
 	hn_rndis_set_rxfilter(hv, 0);
 	hn_vf_stop(dev);
diff --git a/drivers/net/nfb/nfb_ethdev.c b/drivers/net/nfb/nfb_ethdev.c
index a9a8bc878d..44e57ee6c8 100644
--- a/drivers/net/nfb/nfb_ethdev.c
+++ b/drivers/net/nfb/nfb_ethdev.c
@@ -158,6 +158,8 @@ nfb_eth_dev_stop(struct rte_eth_dev *dev)
 	uint16_t nb_rx = dev->data->nb_rx_queues;
 	uint16_t nb_tx = dev->data->nb_tx_queues;
 
+	dev->data->dev_started = 0;
+
 	for (i = 0; i < nb_tx; i++)
 		nfb_eth_tx_queue_stop(dev, i);
 
diff --git a/drivers/net/pfe/pfe_ethdev.c b/drivers/net/pfe/pfe_ethdev.c
index f0de1c8a2e..c4dd0a5f69 100644
--- a/drivers/net/pfe/pfe_ethdev.c
+++ b/drivers/net/pfe/pfe_ethdev.c
@@ -378,6 +378,8 @@ pfe_eth_stop(struct rte_eth_dev *dev/*, int wake*/)
 {
 	struct pfe_eth_priv_s *priv = dev->data->dev_private;
 
+	dev->data->dev_started = 0;
+
 	gemac_disable(priv->EMAC_baseaddr);
 	gpi_disable(priv->GPI_baseaddr);
 
diff --git a/drivers/net/qede/qede_ethdev.c b/drivers/net/qede/qede_ethdev.c
index 548497f3ae..88389c9e4f 100644
--- a/drivers/net/qede/qede_ethdev.c
+++ b/drivers/net/qede/qede_ethdev.c
@@ -1169,6 +1169,7 @@ static void qede_dev_stop(struct rte_eth_dev *eth_dev)
 	struct ecore_dev *edev = QEDE_INIT_EDEV(qdev);
 
 	PMD_INIT_FUNC_TRACE(edev);
+	eth_dev->data->dev_started = 0;
 
 	/* Bring the link down */
 	qede_dev_set_link_state(eth_dev, false);
diff --git a/drivers/net/ring/rte_eth_ring.c b/drivers/net/ring/rte_eth_ring.c
index 83c5502123..6dceab0c59 100644
--- a/drivers/net/ring/rte_eth_ring.c
+++ b/drivers/net/ring/rte_eth_ring.c
@@ -108,6 +108,7 @@ eth_dev_start(struct rte_eth_dev *dev)
 static void
 eth_dev_stop(struct rte_eth_dev *dev)
 {
+	dev->data->dev_started = 0;
 	dev->data->dev_link.link_status = ETH_LINK_DOWN;
 }
 
diff --git a/drivers/net/szedata2/rte_eth_szedata2.c b/drivers/net/szedata2/rte_eth_szedata2.c
index 0eecec1e8c..c89ff16fc1 100644
--- a/drivers/net/szedata2/rte_eth_szedata2.c
+++ b/drivers/net/szedata2/rte_eth_szedata2.c
@@ -1020,6 +1020,8 @@ eth_dev_stop(struct rte_eth_dev *dev)
 	uint16_t nb_rx = dev->data->nb_rx_queues;
 	uint16_t nb_tx = dev->data->nb_tx_queues;
 
+	dev->data->dev_started = 0;
+
 	for (i = 0; i < nb_tx; i++)
 		eth_tx_queue_stop(dev, i);
 
diff --git a/drivers/net/thunderx/nicvf_ethdev.c b/drivers/net/thunderx/nicvf_ethdev.c
index cc6eb4ba24..b87e1e5640 100644
--- a/drivers/net/thunderx/nicvf_ethdev.c
+++ b/drivers/net/thunderx/nicvf_ethdev.c
@@ -1766,6 +1766,7 @@ nicvf_dev_stop_cleanup(struct rte_eth_dev *dev, bool cleanup)
 	struct nicvf *nic = nicvf_pmd_priv(dev);
 
 	PMD_INIT_FUNC_TRACE();
+	dev->data->dev_started = 0;
 
 	/* Teardown secondary vf first */
 	for (i = 0; i < nic->sqs_count; i++) {
diff --git a/drivers/net/vhost/rte_eth_vhost.c b/drivers/net/vhost/rte_eth_vhost.c
index 66efecb320..5328cfcd02 100644
--- a/drivers/net/vhost/rte_eth_vhost.c
+++ b/drivers/net/vhost/rte_eth_vhost.c
@@ -1158,6 +1158,7 @@ eth_dev_stop(struct rte_eth_dev *dev)
 {
 	struct pmd_internal *internal = dev->data->dev_private;
 
+	dev->data->dev_started = 0;
 	rte_atomic32_set(&internal->started, 0);
 	update_queuing_status(dev);
 }
diff --git a/drivers/net/virtio/virtio_ethdev.c b/drivers/net/virtio/virtio_ethdev.c
index 0236c756dc..ea6496acce 100644
--- a/drivers/net/virtio/virtio_ethdev.c
+++ b/drivers/net/virtio/virtio_ethdev.c
@@ -2523,6 +2523,7 @@ virtio_dev_stop(struct rte_eth_dev *dev)
 	struct rte_intr_conf *intr_conf = &dev->data->dev_conf.intr_conf;
 
 	PMD_INIT_LOG(DEBUG, "stop");
+	dev->data->dev_started = 0;
 
 	rte_spinlock_lock(&hw->state_lock);
 	if (!hw->started)
diff --git a/drivers/net/vmxnet3/vmxnet3_ethdev.c b/drivers/net/vmxnet3/vmxnet3_ethdev.c
index fa950e1ba0..828043a65a 100644
--- a/drivers/net/vmxnet3/vmxnet3_ethdev.c
+++ b/drivers/net/vmxnet3/vmxnet3_ethdev.c
@@ -858,6 +858,7 @@ vmxnet3_dev_stop(struct rte_eth_dev *dev)
 	rte_eth_linkstatus_set(dev, &link);
 
 	hw->adapter_stopped = 1;
+	dev->data->dev_started = 0;
 }
 
 static void
diff --git a/lib/librte_ethdev/rte_ethdev.c b/lib/librte_ethdev/rte_ethdev.c
index 59beb8aec2..4537badff8 100644
--- a/lib/librte_ethdev/rte_ethdev.c
+++ b/lib/librte_ethdev/rte_ethdev.c
@@ -1716,7 +1716,6 @@ rte_eth_dev_close(uint16_t port_id)
 	dev = &rte_eth_devices[port_id];
 
 	RTE_FUNC_PTR_OR_RET(*dev->dev_ops->dev_close);
-	dev->data->dev_started = 0;
 	(*dev->dev_ops->dev_close)(dev);
 
 	rte_ethdev_trace_close(port_id);
-- 
2.28.0

Re: [dpdk-dev] [PATCH v5 1/3] ethdev: remove forcing stopped state upon close

[Ferruh Yigit]

On 10/13/2020 11:06 AM, Thomas Monjalon wrote:
> When closing a port, it is supposed to be already stopped,
> and marked as such with "dev_started" state zeroed by the stop API.
> 
> Resetting "dev_started" before calling the driver close operation
> was hiding the case of not properly stopped port being closed.
> The flag "dev_started" is not changed anymore in "rte_eth_dev_close()".
> 
> In case the "dev_stop" function is called from "dev_close",
> bypassing "rte_eth_dev_stop()" API,
> the "dev_started" state must be explicitly reset in the PMD
> in order to keep the same behaviour.
> 
> Signed-off-by: Thomas Monjalon <thomas@monjalon.net>
> Acked-by: Stephen Hemminger <stephen@networkplumber.org>
> Reviewed-by: Andrew Rybchenko <arybchenko@solarflare.com>

<...>

> @@ -1497,6 +1497,7 @@ eth_igb_stop(struct rte_eth_dev *dev)
>   	}
>   
>   	adapter->stopped = true;
> +	dev->data->dev_started = 0;
>   }

'igbvf_dev_stop()' may be missed.

<...>

> @@ -2917,6 +2917,7 @@ ixgbe_dev_stop(struct rte_eth_dev *dev)
>   	adapter->rss_reta_updated = 0;
>   
>   	hw->adapter_stopped = true;
> +	dev->data->dev_started = 0;
>   }

'ixgbevf_dev_stop()' may be missed.


Also many virtual PMDs doesn't call 'dev_stop()' from the 'dev_close()', for 
many cases they may not needed to, since there is no device to stop.
But for the sake of the correct status, should 'dev_started' set to '0' in 
'dev_close()' dev_ops, or do you think can we ignore this?

Re: [dpdk-dev] [PATCH v5 1/3] ethdev: remove forcing stopped state upon close

[Thomas Monjalon]

13/10/2020 14:36, Ferruh Yigit:
> On 10/13/2020 11:06 AM, Thomas Monjalon wrote:
> > When closing a port, it is supposed to be already stopped,
> > and marked as such with "dev_started" state zeroed by the stop API.
> > 
> > Resetting "dev_started" before calling the driver close operation
> > was hiding the case of not properly stopped port being closed.
> > The flag "dev_started" is not changed anymore in "rte_eth_dev_close()".
> > 
> > In case the "dev_stop" function is called from "dev_close",
> > bypassing "rte_eth_dev_stop()" API,
> > the "dev_started" state must be explicitly reset in the PMD
> > in order to keep the same behaviour.
> > 
> > Signed-off-by: Thomas Monjalon <thomas@monjalon.net>
> > Acked-by: Stephen Hemminger <stephen@networkplumber.org>
> > Reviewed-by: Andrew Rybchenko <arybchenko@solarflare.com>
> 
> <...>
> 
> > @@ -1497,6 +1497,7 @@ eth_igb_stop(struct rte_eth_dev *dev)
> >   	}
> >   
> >   	adapter->stopped = true;
> > +	dev->data->dev_started = 0;
> >   }
> 
> 'igbvf_dev_stop()' may be missed.

Will check

> <...>
> 
> > @@ -2917,6 +2917,7 @@ ixgbe_dev_stop(struct rte_eth_dev *dev)
> >   	adapter->rss_reta_updated = 0;
> >   
> >   	hw->adapter_stopped = true;
> > +	dev->data->dev_started = 0;
> >   }
> 
> 'ixgbevf_dev_stop()' may be missed.

Will check

> Also many virtual PMDs doesn't call 'dev_stop()' from the 'dev_close()', for 
> many cases they may not needed to, since there is no device to stop.
> But for the sake of the correct status, should 'dev_started' set to '0' in 
> 'dev_close()' dev_ops, or do you think can we ignore this?

I think we can ignore for this patch,
but maybe these PMDs could be updated separately
to comply with the stop/close sequence.

Re: [dpdk-dev] [PATCH v5 1/3] ethdev: remove forcing stopped state upon close

[Ferruh Yigit]

On 10/13/2020 11:06 AM, Thomas Monjalon wrote:
> When closing a port, it is supposed to be already stopped,
> and marked as such with "dev_started" state zeroed by the stop API.
> 
> Resetting "dev_started" before calling the driver close operation
> was hiding the case of not properly stopped port being closed.
> The flag "dev_started" is not changed anymore in "rte_eth_dev_close()".
> 
> In case the "dev_stop" function is called from "dev_close",
> bypassing "rte_eth_dev_stop()" API,
> the "dev_started" state must be explicitly reset in the PMD
> in order to keep the same behaviour.
> 
> Signed-off-by: Thomas Monjalon <thomas@monjalon.net>
> Acked-by: Stephen Hemminger <stephen@networkplumber.org>
> Reviewed-by: Andrew Rybchenko <arybchenko@solarflare.com>
> ---
>   drivers/net/ark/ark_ethdev.c            | 1 +
>   drivers/net/atlantic/atl_ethdev.c       | 1 +
>   drivers/net/dpaa/dpaa_ethdev.c          | 1 +
>   drivers/net/e1000/em_ethdev.c           | 2 ++
>   drivers/net/e1000/igb_ethdev.c          | 1 +
>   drivers/net/ena/ena_ethdev.c            | 1 +
>   drivers/net/enetc/enetc_ethdev.c        | 1 +
>   drivers/net/fm10k/fm10k_ethdev.c        | 1 +
>   drivers/net/hinic/hinic_pmd_ethdev.c    | 2 ++
>   drivers/net/hns3/hns3_ethdev.c          | 1 +
>   drivers/net/hns3/hns3_ethdev_vf.c       | 1 +
>   drivers/net/i40e/i40e_ethdev.c          | 1 +
>   drivers/net/i40e/i40e_ethdev_vf.c       | 2 +-
>   drivers/net/iavf/iavf_ethdev.c          | 1 +
>   drivers/net/ice/ice_ethdev.c            | 1 +
>   drivers/net/igc/igc_ethdev.c            | 1 +
>   drivers/net/ixgbe/ixgbe_ethdev.c        | 1 +
>   drivers/net/kni/rte_eth_kni.c           | 1 +
>   drivers/net/liquidio/lio_ethdev.c       | 1 +
>   drivers/net/mvneta/mvneta_ethdev.c      | 2 ++
>   drivers/net/netvsc/hn_ethdev.c          | 1 +
>   drivers/net/nfb/nfb_ethdev.c            | 2 ++
>   drivers/net/pfe/pfe_ethdev.c            | 2 ++
>   drivers/net/qede/qede_ethdev.c          | 1 +
>   drivers/net/ring/rte_eth_ring.c         | 1 +
>   drivers/net/szedata2/rte_eth_szedata2.c | 2 ++
>   drivers/net/thunderx/nicvf_ethdev.c     | 1 +
>   drivers/net/vhost/rte_eth_vhost.c       | 1 +
>   drivers/net/virtio/virtio_ethdev.c      | 1 +
>   drivers/net/vmxnet3/vmxnet3_ethdev.c    | 1 +
>   lib/librte_ethdev/rte_ethdev.c          | 1 -
>   31 files changed, 36 insertions(+), 2 deletions(-)

Following non-virtual PMDs doesn't call 'dev_stop()' from 'dev_close()', 
maintainers of the PMDs cc'ed.

avp
axgbe
bnx2x
cxgbe
dpaa2
enic
ice_dcf
ionic
ipn3ke
mlx4
mlx5
mvpp2
nfp
octeontx
octeontx2
sfc

Can you please double check for your driver:

1) Device stopped properly before it has been closed?

2) The device stopped state ('dev->data->dev_started') is correct if device 
closed without explicitly stopped first.

Thanks,
ferruh

Re: [dpdk-dev] [PATCH v5 1/3] ethdev: remove forcing stopped state upon close

[Thomas Monjalon]

13/10/2020 14:45, Ferruh Yigit:
> On 10/13/2020 11:06 AM, Thomas Monjalon wrote:
> > When closing a port, it is supposed to be already stopped,
> > and marked as such with "dev_started" state zeroed by the stop API.
> > 
> > Resetting "dev_started" before calling the driver close operation
> > was hiding the case of not properly stopped port being closed.
> > The flag "dev_started" is not changed anymore in "rte_eth_dev_close()".
> > 
> > In case the "dev_stop" function is called from "dev_close",
> > bypassing "rte_eth_dev_stop()" API,
> > the "dev_started" state must be explicitly reset in the PMD
> > in order to keep the same behaviour.
> > 
> > Signed-off-by: Thomas Monjalon <thomas@monjalon.net>
> > Acked-by: Stephen Hemminger <stephen@networkplumber.org>
> > Reviewed-by: Andrew Rybchenko <arybchenko@solarflare.com>
[...]
> Following non-virtual PMDs doesn't call 'dev_stop()' from 'dev_close()', 
> maintainers of the PMDs cc'ed.
> 
> avp
> axgbe
> bnx2x
> cxgbe
> dpaa2
> enic
> ice_dcf
> ionic
> ipn3ke
> mlx4
> mlx5
> mvpp2
> nfp
> octeontx
> octeontx2
> sfc
> 
> Can you please double check for your driver:
> 
> 1) Device stopped properly before it has been closed?
> 
> 2) The device stopped state ('dev->data->dev_started') is correct if device 
> closed without explicitly stopped first.

The application is supposed to call stop before close,
so I don't know what has to be checked.
The automatic stop in close is an extra feature per PMD choice.

Re: [dpdk-dev] [PATCH v5 1/3] ethdev: remove forcing stopped state upon close

[Ferruh Yigit]

On 10/13/2020 1:51 PM, Thomas Monjalon wrote:
> 13/10/2020 14:45, Ferruh Yigit:
>> On 10/13/2020 11:06 AM, Thomas Monjalon wrote:
>>> When closing a port, it is supposed to be already stopped,
>>> and marked as such with "dev_started" state zeroed by the stop API.
>>>
>>> Resetting "dev_started" before calling the driver close operation
>>> was hiding the case of not properly stopped port being closed.
>>> The flag "dev_started" is not changed anymore in "rte_eth_dev_close()".
>>>
>>> In case the "dev_stop" function is called from "dev_close",
>>> bypassing "rte_eth_dev_stop()" API,
>>> the "dev_started" state must be explicitly reset in the PMD
>>> in order to keep the same behaviour.
>>>
>>> Signed-off-by: Thomas Monjalon <thomas@monjalon.net>
>>> Acked-by: Stephen Hemminger <stephen@networkplumber.org>
>>> Reviewed-by: Andrew Rybchenko <arybchenko@solarflare.com>
> [...]
>> Following non-virtual PMDs doesn't call 'dev_stop()' from 'dev_close()',
>> maintainers of the PMDs cc'ed.
>>
>> avp
>> axgbe
>> bnx2x
>> cxgbe
>> dpaa2
>> enic
>> ice_dcf
>> ionic
>> ipn3ke
>> mlx4
>> mlx5
>> mvpp2
>> nfp
>> octeontx
>> octeontx2
>> sfc
>>
>> Can you please double check for your driver:
>>
>> 1) Device stopped properly before it has been closed?
>>
>> 2) The device stopped state ('dev->data->dev_started') is correct if device
>> closed without explicitly stopped first.
> 
> The application is supposed to call stop before close,
> so I don't know what has to be checked.
> The automatic stop in close is an extra feature per PMD choice.
> 

A device can be hot-removed without stop, that is why I was thinking 'close()' 
should stop the device (if it is not already).

Even current 'rte_eth_dev_close()' API doesn't check if the port stopped or not.

If application does stop -> close/remove in order, that would work, but for the 
case it didn't, PMD ensuring the device stopped before close is safer.

Re: [dpdk-dev] [PATCH v5 1/3] ethdev: remove forcing stopped state upon close

[Thomas Monjalon]

13/10/2020 19:54, Ferruh Yigit:
> On 10/13/2020 1:51 PM, Thomas Monjalon wrote:
> > 13/10/2020 14:45, Ferruh Yigit:
> >> On 10/13/2020 11:06 AM, Thomas Monjalon wrote:
> >>> When closing a port, it is supposed to be already stopped,
> >>> and marked as such with "dev_started" state zeroed by the stop API.
> >>>
> >>> Resetting "dev_started" before calling the driver close operation
> >>> was hiding the case of not properly stopped port being closed.
> >>> The flag "dev_started" is not changed anymore in "rte_eth_dev_close()".
> >>>
> >>> In case the "dev_stop" function is called from "dev_close",
> >>> bypassing "rte_eth_dev_stop()" API,
> >>> the "dev_started" state must be explicitly reset in the PMD
> >>> in order to keep the same behaviour.
> >>>
> >>> Signed-off-by: Thomas Monjalon <thomas@monjalon.net>
> >>> Acked-by: Stephen Hemminger <stephen@networkplumber.org>
> >>> Reviewed-by: Andrew Rybchenko <arybchenko@solarflare.com>
> > [...]
> >> Following non-virtual PMDs doesn't call 'dev_stop()' from 'dev_close()',
> >> maintainers of the PMDs cc'ed.
> >>
> >> avp
> >> axgbe
> >> bnx2x
> >> cxgbe
> >> dpaa2
> >> enic
> >> ice_dcf
> >> ionic
> >> ipn3ke
> >> mlx4
> >> mlx5
> >> mvpp2
> >> nfp
> >> octeontx
> >> octeontx2
> >> sfc
> >>
> >> Can you please double check for your driver:
> >>
> >> 1) Device stopped properly before it has been closed?
> >>
> >> 2) The device stopped state ('dev->data->dev_started') is correct if device
> >> closed without explicitly stopped first.
> > 
> > The application is supposed to call stop before close,
> > so I don't know what has to be checked.
> > The automatic stop in close is an extra feature per PMD choice.
> > 
> 
> A device can be hot-removed without stop, that is why I was thinking 'close()' 
> should stop the device (if it is not already).
> 
> Even current 'rte_eth_dev_close()' API doesn't check if the port stopped or not.
> 
> If application does stop -> close/remove in order, that would work, but for the 
> case it didn't, PMD ensuring the device stopped before close is safer.

Yes that's probably a case which not well tested.

[dpdk-dev] [PATCH v5 2/3] ethdev: reset all when releasing a port

[Thomas Monjalon]

The function rte_eth_dev_release_port() is partially resetting
the struct rte_eth_dev. The drivers were completing this reset
with more pointers set to NULL in the close or remove operations.

More pointers are reset at ethdev level,
and some redundant assignments are removed from PMDs.

Signed-off-by: Thomas Monjalon <thomas@monjalon.net>
Acked-by: Stephen Hemminger <stephen@networkplumber.org>
Reviewed-by: Haiyue Wang <haiyue.wang@intel.com>
Acked-by: Jeff Guo <jia.guo@intel.com>
Reviewed-by: Andrew Rybchenko <arybchenko@solarflare.com>
---
 drivers/net/ark/ark_ethdev.c           | 3 ---
 drivers/net/atlantic/atl_ethdev.c      | 4 ----
 drivers/net/axgbe/axgbe_ethdev.c       | 3 ---
 drivers/net/bnxt/bnxt_ethdev.c         | 4 ----
 drivers/net/bnxt/bnxt_reps.c           | 1 -
 drivers/net/bonding/rte_eth_bond_pmd.c | 4 ----
 drivers/net/dpaa/dpaa_ethdev.c         | 4 ----
 drivers/net/dpaa2/dpaa2_ethdev.c       | 4 ----
 drivers/net/e1000/em_ethdev.c          | 4 ----
 drivers/net/e1000/igb_ethdev.c         | 8 --------
 drivers/net/ena/ena_ethdev.c           | 5 -----
 drivers/net/fm10k/fm10k_ethdev.c       | 4 ----
 drivers/net/hinic/hinic_pmd_ethdev.c   | 4 ----
 drivers/net/hns3/hns3_ethdev.c         | 4 ----
 drivers/net/hns3/hns3_ethdev_vf.c      | 5 -----
 drivers/net/i40e/i40e_ethdev.c         | 4 ----
 drivers/net/i40e/i40e_ethdev_vf.c      | 4 ----
 drivers/net/iavf/iavf_ethdev.c         | 4 ----
 drivers/net/ice/ice_dcf_ethdev.c       | 4 ----
 drivers/net/ice/ice_ethdev.c           | 4 ----
 drivers/net/ionic/ionic_ethdev.c       | 5 -----
 drivers/net/ixgbe/ixgbe_ethdev.c       | 8 --------
 drivers/net/liquidio/lio_ethdev.c      | 4 ----
 drivers/net/netvsc/hn_ethdev.c         | 4 ----
 drivers/net/octeontx/octeontx_ethdev.c | 3 ---
 drivers/net/qede/qede_ethdev.c         | 4 ----
 drivers/net/sfc/sfc_ethdev.c           | 7 +------
 drivers/net/tap/rte_eth_tap.c          | 1 -
 drivers/net/virtio/virtio_ethdev.c     | 4 ----
 drivers/net/vmxnet3/vmxnet3_ethdev.c   | 5 -----
 lib/librte_ethdev/rte_ethdev.c         | 9 +++++++++
 31 files changed, 10 insertions(+), 125 deletions(-)

diff --git a/drivers/net/ark/ark_ethdev.c b/drivers/net/ark/ark_ethdev.c
index f1355581f1..dc14f0fb1a 100644
--- a/drivers/net/ark/ark_ethdev.c
+++ b/drivers/net/ark/ark_ethdev.c
@@ -504,9 +504,6 @@ eth_ark_dev_uninit(struct rte_eth_dev *dev)
 	ark_pktgen_uninit(ark->pg);
 	ark_pktchkr_uninit(ark->pc);
 
-	dev->dev_ops = NULL;
-	dev->rx_pkt_burst = NULL;
-	dev->tx_pkt_burst = NULL;
 	return 0;
 }
 
diff --git a/drivers/net/atlantic/atl_ethdev.c b/drivers/net/atlantic/atl_ethdev.c
index f2e48e1c11..d96865d5b4 100644
--- a/drivers/net/atlantic/atl_ethdev.c
+++ b/drivers/net/atlantic/atl_ethdev.c
@@ -702,10 +702,6 @@ atl_dev_close(struct rte_eth_dev *dev)
 
 	atl_free_queues(dev);
 
-	dev->dev_ops = NULL;
-	dev->rx_pkt_burst = NULL;
-	dev->tx_pkt_burst = NULL;
-
 	/* disable uio intr before callback unregister */
 	rte_intr_disable(intr_handle);
 	rte_intr_callback_unregister(intr_handle,
diff --git a/drivers/net/axgbe/axgbe_ethdev.c b/drivers/net/axgbe/axgbe_ethdev.c
index cf085487cc..39087f36f0 100644
--- a/drivers/net/axgbe/axgbe_ethdev.c
+++ b/drivers/net/axgbe/axgbe_ethdev.c
@@ -2134,9 +2134,6 @@ axgbe_dev_close(struct rte_eth_dev *eth_dev)
 		return 0;
 
 	pci_dev = RTE_DEV_TO_PCI(eth_dev->device);
-	eth_dev->dev_ops = NULL;
-	eth_dev->rx_pkt_burst = NULL;
-	eth_dev->tx_pkt_burst = NULL;
 	axgbe_dev_clear_queues(eth_dev);
 
 	/* disable uio intr before callback unregister */
diff --git a/drivers/net/bnxt/bnxt_ethdev.c b/drivers/net/bnxt/bnxt_ethdev.c
index 8b63134c39..3d32b578e8 100644
--- a/drivers/net/bnxt/bnxt_ethdev.c
+++ b/drivers/net/bnxt/bnxt_ethdev.c
@@ -1433,10 +1433,6 @@ static int bnxt_dev_close_op(struct rte_eth_dev *eth_dev)
 	bnxt_free_pf_info(bp);
 	bnxt_free_parent_info(bp);
 
-	eth_dev->dev_ops = NULL;
-	eth_dev->rx_pkt_burst = NULL;
-	eth_dev->tx_pkt_burst = NULL;
-
 	rte_memzone_free((const struct rte_memzone *)bp->tx_mem_zone);
 	bp->tx_mem_zone = NULL;
 	rte_memzone_free((const struct rte_memzone *)bp->rx_mem_zone);
diff --git a/drivers/net/bnxt/bnxt_reps.c b/drivers/net/bnxt/bnxt_reps.c
index 74a76fce57..012ccc73fe 100644
--- a/drivers/net/bnxt/bnxt_reps.c
+++ b/drivers/net/bnxt/bnxt_reps.c
@@ -255,7 +255,6 @@ int bnxt_representor_uninit(struct rte_eth_dev *eth_dev)
 
 	PMD_DRV_LOG(DEBUG, "BNXT Port:%d VFR uninit\n", eth_dev->data->port_id);
 	eth_dev->data->mac_addrs = NULL;
-	eth_dev->dev_ops = NULL;
 
 	parent_bp = rep->parent_dev->data->dev_private;
 	if (!parent_bp) {
diff --git a/drivers/net/bonding/rte_eth_bond_pmd.c b/drivers/net/bonding/rte_eth_bond_pmd.c
index 1f761c7c9e..ce8fe254af 100644
--- a/drivers/net/bonding/rte_eth_bond_pmd.c
+++ b/drivers/net/bonding/rte_eth_bond_pmd.c
@@ -2130,10 +2130,6 @@ bond_ethdev_close(struct rte_eth_dev *dev)
 	 */
 	rte_mempool_free(internals->mode6.mempool);
 
-	dev->dev_ops = NULL;
-	dev->rx_pkt_burst = NULL;
-	dev->tx_pkt_burst = NULL;
-
 	return 0;
 }
 
diff --git a/drivers/net/dpaa/dpaa_ethdev.c b/drivers/net/dpaa/dpaa_ethdev.c
index 952e8f72ad..6d27d742af 100644
--- a/drivers/net/dpaa/dpaa_ethdev.c
+++ b/drivers/net/dpaa/dpaa_ethdev.c
@@ -504,10 +504,6 @@ static int dpaa_eth_dev_close(struct rte_eth_dev *dev)
 	rte_free(dpaa_intf->tx_queues);
 	dpaa_intf->tx_queues = NULL;
 
-	dev->dev_ops = NULL;
-	dev->rx_pkt_burst = NULL;
-	dev->tx_pkt_burst = NULL;
-
 	return 0;
 }
 
diff --git a/drivers/net/dpaa2/dpaa2_ethdev.c b/drivers/net/dpaa2/dpaa2_ethdev.c
index d8624514df..ba99a524f9 100644
--- a/drivers/net/dpaa2/dpaa2_ethdev.c
+++ b/drivers/net/dpaa2/dpaa2_ethdev.c
@@ -1284,10 +1284,6 @@ dpaa2_dev_close(struct rte_eth_dev *dev)
 	if (priv->extract.qos_extract_param)
 		rte_free((void *)(size_t)priv->extract.qos_extract_param);
 
-	dev->dev_ops = NULL;
-	dev->rx_pkt_burst = NULL;
-	dev->tx_pkt_burst = NULL;
-
 	DPAA2_PMD_INFO("%s: netdev deleted", dev->data->name);
 	return 0;
 }
diff --git a/drivers/net/e1000/em_ethdev.c b/drivers/net/e1000/em_ethdev.c
index dd9238a616..304c040421 100644
--- a/drivers/net/e1000/em_ethdev.c
+++ b/drivers/net/e1000/em_ethdev.c
@@ -774,10 +774,6 @@ eth_em_close(struct rte_eth_dev *dev)
 	em_release_manageability(hw);
 	em_hw_control_release(hw);
 
-	dev->dev_ops = NULL;
-	dev->rx_pkt_burst = NULL;
-	dev->tx_pkt_burst = NULL;
-
 	/* disable uio intr before callback unregister */
 	rte_intr_disable(intr_handle);
 	rte_intr_callback_unregister(intr_handle,
diff --git a/drivers/net/e1000/igb_ethdev.c b/drivers/net/e1000/igb_ethdev.c
index 76746c9484..f7c62f473d 100644
--- a/drivers/net/e1000/igb_ethdev.c
+++ b/drivers/net/e1000/igb_ethdev.c
@@ -1565,10 +1565,6 @@ eth_igb_close(struct rte_eth_dev *dev)
 	memset(&link, 0, sizeof(link));
 	rte_eth_linkstatus_set(dev, &link);
 
-	dev->dev_ops = NULL;
-	dev->rx_pkt_burst = NULL;
-	dev->tx_pkt_burst = NULL;
-
 	/* Reset any pending lock */
 	igb_reset_swfw_lock(hw);
 
@@ -3404,10 +3400,6 @@ igbvf_dev_close(struct rte_eth_dev *dev)
 	memset(&addr, 0, sizeof(addr));
 	igbvf_default_mac_addr_set(dev, &addr);
 
-	dev->dev_ops = NULL;
-	dev->rx_pkt_burst = NULL;
-	dev->tx_pkt_burst = NULL;
-
 	rte_intr_callback_unregister(&pci_dev->intr_handle,
 				     eth_igbvf_interrupt_handler,
 				     (void *)dev);
diff --git a/drivers/net/ena/ena_ethdev.c b/drivers/net/ena/ena_ethdev.c
index e02d340c33..bdb8fe79c6 100644
--- a/drivers/net/ena/ena_ethdev.c
+++ b/drivers/net/ena/ena_ethdev.c
@@ -1956,11 +1956,6 @@ static int eth_ena_dev_uninit(struct rte_eth_dev *eth_dev)
 
 	ena_destroy_device(eth_dev);
 
-	eth_dev->dev_ops = NULL;
-	eth_dev->rx_pkt_burst = NULL;
-	eth_dev->tx_pkt_burst = NULL;
-	eth_dev->tx_pkt_prepare = NULL;
-
 	return 0;
 }
 
diff --git a/drivers/net/fm10k/fm10k_ethdev.c b/drivers/net/fm10k/fm10k_ethdev.c
index 0c13fd5c37..bada69c64a 100644
--- a/drivers/net/fm10k/fm10k_ethdev.c
+++ b/drivers/net/fm10k/fm10k_ethdev.c
@@ -2805,10 +2805,6 @@ fm10k_dev_close(struct rte_eth_dev *dev)
 	fm10k_dev_queue_release(dev);
 	fm10k_stop_hw(hw);
 
-	dev->dev_ops = NULL;
-	dev->rx_pkt_burst = NULL;
-	dev->tx_pkt_burst = NULL;
-
 	/* disable uio/vfio intr */
 	rte_intr_disable(intr_handle);
 
diff --git a/drivers/net/hinic/hinic_pmd_ethdev.c b/drivers/net/hinic/hinic_pmd_ethdev.c
index daf8f33cf3..2504228d94 100644
--- a/drivers/net/hinic/hinic_pmd_ethdev.c
+++ b/drivers/net/hinic/hinic_pmd_ethdev.c
@@ -3253,10 +3253,6 @@ static int hinic_dev_uninit(struct rte_eth_dev *dev)
 
 	hinic_dev_close(dev);
 
-	dev->dev_ops = NULL;
-	dev->rx_pkt_burst = NULL;
-	dev->tx_pkt_burst = NULL;
-
 	rte_free(nic_dev->mc_list);
 
 	return HINIC_OK;
diff --git a/drivers/net/hns3/hns3_ethdev.c b/drivers/net/hns3/hns3_ethdev.c
index 8fcc0f18c5..c850e5eb14 100644
--- a/drivers/net/hns3/hns3_ethdev.c
+++ b/drivers/net/hns3/hns3_ethdev.c
@@ -6201,10 +6201,6 @@ hns3_dev_uninit(struct rte_eth_dev *eth_dev)
 	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
 		return -EPERM;
 
-	eth_dev->dev_ops = NULL;
-	eth_dev->rx_pkt_burst = NULL;
-	eth_dev->tx_pkt_burst = NULL;
-	eth_dev->tx_pkt_prepare = NULL;
 	if (hw->adapter_state < HNS3_NIC_CLOSING)
 		hns3_dev_close(eth_dev);
 
diff --git a/drivers/net/hns3/hns3_ethdev_vf.c b/drivers/net/hns3/hns3_ethdev_vf.c
index b2038395a2..6e169583a0 100644
--- a/drivers/net/hns3/hns3_ethdev_vf.c
+++ b/drivers/net/hns3/hns3_ethdev_vf.c
@@ -2847,11 +2847,6 @@ hns3vf_dev_uninit(struct rte_eth_dev *eth_dev)
 	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
 		return -EPERM;
 
-	eth_dev->dev_ops = NULL;
-	eth_dev->rx_pkt_burst = NULL;
-	eth_dev->tx_pkt_burst = NULL;
-	eth_dev->tx_pkt_prepare = NULL;
-
 	if (hw->adapter_state < HNS3_NIC_CLOSING)
 		hns3vf_dev_close(eth_dev);
 
diff --git a/drivers/net/i40e/i40e_ethdev.c b/drivers/net/i40e/i40e_ethdev.c
index 825de51fbb..605a1792b6 100644
--- a/drivers/net/i40e/i40e_ethdev.c
+++ b/drivers/net/i40e/i40e_ethdev.c
@@ -2693,10 +2693,6 @@ i40e_dev_close(struct rte_eth_dev *dev)
 			(reg | I40E_PFGEN_CTRL_PFSWR_MASK));
 	I40E_WRITE_FLUSH(hw);
 
-	dev->dev_ops = NULL;
-	dev->rx_pkt_burst = NULL;
-	dev->tx_pkt_burst = NULL;
-
 	/* Clear PXE mode */
 	i40e_clear_pxe_mode(hw);
 
diff --git a/drivers/net/i40e/i40e_ethdev_vf.c b/drivers/net/i40e/i40e_ethdev_vf.c
index 40c9745433..3dcd50c428 100644
--- a/drivers/net/i40e/i40e_ethdev_vf.c
+++ b/drivers/net/i40e/i40e_ethdev_vf.c
@@ -2421,10 +2421,6 @@ i40evf_dev_close(struct rte_eth_dev *dev)
 	i40e_shutdown_adminq(hw);
 	i40evf_disable_irq0(hw);
 
-	dev->dev_ops = NULL;
-	dev->rx_pkt_burst = NULL;
-	dev->tx_pkt_burst = NULL;
-
 	rte_free(vf->vf_res);
 	vf->vf_res = NULL;
 	rte_free(vf->aq_resp);
diff --git a/drivers/net/iavf/iavf_ethdev.c b/drivers/net/iavf/iavf_ethdev.c
index 8bd2c048a1..c951e75358 100644
--- a/drivers/net/iavf/iavf_ethdev.c
+++ b/drivers/net/iavf/iavf_ethdev.c
@@ -1525,10 +1525,6 @@ iavf_dev_close(struct rte_eth_dev *dev)
 				     iavf_dev_interrupt_handler, dev);
 	iavf_disable_irq0(hw);
 
-	dev->dev_ops = NULL;
-	dev->rx_pkt_burst = NULL;
-	dev->tx_pkt_burst = NULL;
-
 	if (vf->vf_res->vf_cap_flags & VIRTCHNL_VF_OFFLOAD_RSS_PF) {
 		if (vf->rss_lut) {
 			rte_free(vf->rss_lut);
diff --git a/drivers/net/ice/ice_dcf_ethdev.c b/drivers/net/ice/ice_dcf_ethdev.c
index 33dd0c44f2..f2056e2aba 100644
--- a/drivers/net/ice/ice_dcf_ethdev.c
+++ b/drivers/net/ice/ice_dcf_ethdev.c
@@ -855,10 +855,6 @@ ice_dcf_dev_close(struct rte_eth_dev *dev)
 	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
 		return 0;
 
-	dev->dev_ops = NULL;
-	dev->rx_pkt_burst = NULL;
-	dev->tx_pkt_burst = NULL;
-
 	ice_dcf_uninit_parent_adapter(dev);
 	ice_dcf_uninit_hw(dev, &adapter->real_hw);
 
diff --git a/drivers/net/ice/ice_ethdev.c b/drivers/net/ice/ice_ethdev.c
index 1db4015c23..33b2828703 100644
--- a/drivers/net/ice/ice_ethdev.c
+++ b/drivers/net/ice/ice_ethdev.c
@@ -2417,10 +2417,6 @@ ice_dev_close(struct rte_eth_dev *dev)
 	rte_free(pf->proto_xtr);
 	pf->proto_xtr = NULL;
 
-	dev->dev_ops = NULL;
-	dev->rx_pkt_burst = NULL;
-	dev->tx_pkt_burst = NULL;
-
 	/* disable uio intr before callback unregister */
 	rte_intr_disable(intr_handle);
 
diff --git a/drivers/net/ionic/ionic_ethdev.c b/drivers/net/ionic/ionic_ethdev.c
index ef7d06e526..1387253dc8 100644
--- a/drivers/net/ionic/ionic_ethdev.c
+++ b/drivers/net/ionic/ionic_ethdev.c
@@ -1065,11 +1065,6 @@ eth_ionic_dev_uninit(struct rte_eth_dev *eth_dev)
 	ionic_lif_deinit(lif);
 	ionic_lif_free(lif);
 
-	eth_dev->dev_ops = NULL;
-	eth_dev->rx_pkt_burst = NULL;
-	eth_dev->tx_pkt_burst = NULL;
-	eth_dev->tx_pkt_prepare = NULL;
-
 	return 0;
 }
 
diff --git a/drivers/net/ixgbe/ixgbe_ethdev.c b/drivers/net/ixgbe/ixgbe_ethdev.c
index bd83b0e31a..a7f8a31488 100644
--- a/drivers/net/ixgbe/ixgbe_ethdev.c
+++ b/drivers/net/ixgbe/ixgbe_ethdev.c
@@ -3010,10 +3010,6 @@ ixgbe_dev_close(struct rte_eth_dev *dev)
 	/* reprogram the RAR[0] in case user changed it. */
 	ixgbe_set_rar(hw, 0, hw->mac.addr, 0, IXGBE_RAH_AV);
 
-	dev->dev_ops = NULL;
-	dev->rx_pkt_burst = NULL;
-	dev->tx_pkt_burst = NULL;
-
 	/* Unlock any pending hardware semaphore */
 	ixgbe_swfw_lock_reset(hw);
 
@@ -5467,10 +5463,6 @@ ixgbevf_dev_close(struct rte_eth_dev *dev)
 	 **/
 	ixgbevf_remove_mac_addr(dev, 0);
 
-	dev->dev_ops = NULL;
-	dev->rx_pkt_burst = NULL;
-	dev->tx_pkt_burst = NULL;
-
 	rte_intr_disable(intr_handle);
 	rte_intr_callback_unregister(intr_handle,
 				     ixgbevf_dev_interrupt_handler, dev);
diff --git a/drivers/net/liquidio/lio_ethdev.c b/drivers/net/liquidio/lio_ethdev.c
index 52bcefc819..1b01502e49 100644
--- a/drivers/net/liquidio/lio_ethdev.c
+++ b/drivers/net/liquidio/lio_ethdev.c
@@ -2068,10 +2068,6 @@ lio_eth_dev_uninit(struct rte_eth_dev *eth_dev)
 	/* lio_free_sc_buffer_pool */
 	lio_free_sc_buffer_pool(lio_dev);
 
-	eth_dev->dev_ops = NULL;
-	eth_dev->rx_pkt_burst = NULL;
-	eth_dev->tx_pkt_burst = NULL;
-
 	return 0;
 }
 
diff --git a/drivers/net/netvsc/hn_ethdev.c b/drivers/net/netvsc/hn_ethdev.c
index be3713dfa0..8743e6e69a 100644
--- a/drivers/net/netvsc/hn_ethdev.c
+++ b/drivers/net/netvsc/hn_ethdev.c
@@ -1050,10 +1050,6 @@ eth_hn_dev_uninit(struct rte_eth_dev *eth_dev)
 	hn_dev_stop(eth_dev);
 	hn_dev_close(eth_dev);
 
-	eth_dev->dev_ops = NULL;
-	eth_dev->tx_pkt_burst = NULL;
-	eth_dev->rx_pkt_burst = NULL;
-
 	hn_detach(hv);
 	hn_chim_uninit(eth_dev);
 	rte_vmbus_chan_close(hv->primary->chan);
diff --git a/drivers/net/octeontx/octeontx_ethdev.c b/drivers/net/octeontx/octeontx_ethdev.c
index aa9ef3bb70..0ea6de448a 100644
--- a/drivers/net/octeontx/octeontx_ethdev.c
+++ b/drivers/net/octeontx/octeontx_ethdev.c
@@ -513,9 +513,6 @@ octeontx_dev_close(struct rte_eth_dev *dev)
 
 	octeontx_port_close(nic);
 
-	dev->tx_pkt_burst = NULL;
-	dev->rx_pkt_burst = NULL;
-
 	return 0;
 }
 
diff --git a/drivers/net/qede/qede_ethdev.c b/drivers/net/qede/qede_ethdev.c
index 88389c9e4f..0dd2f78820 100644
--- a/drivers/net/qede/qede_ethdev.c
+++ b/drivers/net/qede/qede_ethdev.c
@@ -1594,10 +1594,6 @@ static int qede_dev_close(struct rte_eth_dev *eth_dev)
 	if (ECORE_IS_CMT(edev))
 		rte_eal_alarm_cancel(qede_poll_sp_sb_cb, (void *)eth_dev);
 
-	eth_dev->dev_ops = NULL;
-	eth_dev->rx_pkt_burst = NULL;
-	eth_dev->tx_pkt_burst = NULL;
-
 	return 0;
 }
 
diff --git a/drivers/net/sfc/sfc_ethdev.c b/drivers/net/sfc/sfc_ethdev.c
index f41d0f5fe2..620e541a10 100644
--- a/drivers/net/sfc/sfc_ethdev.c
+++ b/drivers/net/sfc/sfc_ethdev.c
@@ -324,11 +324,7 @@ static void
 sfc_eth_dev_secondary_clear_ops(struct rte_eth_dev *dev)
 {
 	free(dev->process_private);
-	dev->process_private = NULL;
-	dev->dev_ops = NULL;
-	dev->tx_pkt_prepare = NULL;
-	dev->tx_pkt_burst = NULL;
-	dev->rx_pkt_burst = NULL;
+	rte_eth_dev_release_port(dev);
 }
 
 static int
@@ -380,7 +376,6 @@ sfc_dev_close(struct rte_eth_dev *dev)
 	/* Required for logging, so cleanup last */
 	sa->eth_dev = NULL;
 
-	dev->process_private = NULL;
 	free(sa);
 
 	return 0;
diff --git a/drivers/net/tap/rte_eth_tap.c b/drivers/net/tap/rte_eth_tap.c
index b127ce62dc..0598de00ed 100644
--- a/drivers/net/tap/rte_eth_tap.c
+++ b/drivers/net/tap/rte_eth_tap.c
@@ -1134,7 +1134,6 @@ tap_dev_close(struct rte_eth_dev *dev)
 		internals->ioctl_sock = -1;
 	}
 	rte_free(dev->process_private);
-	dev->process_private = NULL;
 	if (tap_devices_count == 1)
 		rte_mp_action_unregister(TAP_MP_KEY);
 	tap_devices_count--;
diff --git a/drivers/net/virtio/virtio_ethdev.c b/drivers/net/virtio/virtio_ethdev.c
index ea6496acce..8683763aa8 100644
--- a/drivers/net/virtio/virtio_ethdev.c
+++ b/drivers/net/virtio/virtio_ethdev.c
@@ -2001,10 +2001,6 @@ eth_virtio_dev_uninit(struct rte_eth_dev *eth_dev)
 	virtio_dev_stop(eth_dev);
 	virtio_dev_close(eth_dev);
 
-	eth_dev->dev_ops = NULL;
-	eth_dev->tx_pkt_burst = NULL;
-	eth_dev->rx_pkt_burst = NULL;
-
 	PMD_INIT_LOG(DEBUG, "dev_uninit completed");
 
 	return 0;
diff --git a/drivers/net/vmxnet3/vmxnet3_ethdev.c b/drivers/net/vmxnet3/vmxnet3_ethdev.c
index 828043a65a..1d070fbd79 100644
--- a/drivers/net/vmxnet3/vmxnet3_ethdev.c
+++ b/drivers/net/vmxnet3/vmxnet3_ethdev.c
@@ -368,11 +368,6 @@ eth_vmxnet3_dev_uninit(struct rte_eth_dev *eth_dev)
 		return -EBUSY;
 	}
 
-	eth_dev->dev_ops = NULL;
-	eth_dev->rx_pkt_burst = NULL;
-	eth_dev->tx_pkt_burst = NULL;
-	eth_dev->tx_pkt_prepare = NULL;
-
 	return 0;
 }
 
diff --git a/lib/librte_ethdev/rte_ethdev.c b/lib/librte_ethdev/rte_ethdev.c
index 4537badff8..73c3d6f7fe 100644
--- a/lib/librte_ethdev/rte_ethdev.c
+++ b/lib/librte_ethdev/rte_ethdev.c
@@ -556,7 +556,16 @@ rte_eth_dev_release_port(struct rte_eth_dev *eth_dev)
 
 	eth_dev->state = RTE_ETH_DEV_UNUSED;
 	eth_dev->device = NULL;
+	eth_dev->process_private = NULL;
 	eth_dev->intr_handle = NULL;
+	eth_dev->rx_pkt_burst = NULL;
+	eth_dev->tx_pkt_burst = NULL;
+	eth_dev->tx_pkt_prepare = NULL;
+	eth_dev->rx_queue_count = NULL;
+	eth_dev->rx_descriptor_done = NULL;
+	eth_dev->rx_descriptor_status = NULL;
+	eth_dev->tx_descriptor_status = NULL;
+	eth_dev->dev_ops = NULL;
 
 	if (rte_eal_process_type() == RTE_PROC_PRIMARY) {
 		rte_free(eth_dev->data->rx_queues);
-- 
2.28.0

Re: [dpdk-dev] [PATCH v5 2/3] ethdev: reset all when releasing a port

[Ferruh Yigit]

On 10/13/2020 11:06 AM, Thomas Monjalon wrote:
> The function rte_eth_dev_release_port() is partially resetting
> the struct rte_eth_dev. The drivers were completing this reset
> with more pointers set to NULL in the close or remove operations.
> 
> More pointers are reset at ethdev level,
> and some redundant assignments are removed from PMDs.
> 
> Signed-off-by: Thomas Monjalon <thomas@monjalon.net>
> Acked-by: Stephen Hemminger <stephen@networkplumber.org>
> Reviewed-by: Haiyue Wang <haiyue.wang@intel.com>
> Acked-by: Jeff Guo <jia.guo@intel.com>
> Reviewed-by: Andrew Rybchenko <arybchenko@solarflare.com>

Reviewed-by: Ferruh Yigit <ferruh.yigit@intel.com>

[dpdk-dev] [PATCH v5 3/3] ethdev: allow close function to return an error

[Thomas Monjalon]

The API function rte_eth_dev_close() was returning void.
The return type is changed to int for notifying of errors.

If an error happens during a close operation,
the status of the port is undefined,
a maximum of resources having been freed.

Signed-off-by: Thomas Monjalon <thomas@monjalon.net>
Reviewed-by: Liron Himi <lironh@marvell.com>
Acked-by: Stephen Hemminger <stephen@networkplumber.org>
---
 doc/guides/rel_notes/deprecation.rst    |  1 -
 doc/guides/rel_notes/release_20_11.rst  |  4 +++-
 drivers/net/cxgbe/cxgbe_ethdev.c        |  5 +++--
 drivers/net/cxgbe/cxgbevf_ethdev.c      |  5 +++--
 drivers/net/failsafe/failsafe_ether.c   |  6 +++++-
 drivers/net/failsafe/failsafe_ops.c     | 25 +++++++++++++++++--------
 drivers/net/memif/rte_eth_memif.c       |  4 +---
 drivers/net/mlx5/mlx5.c                 |  7 ++++---
 drivers/net/mvneta/mvneta_ethdev.c      |  5 +++--
 drivers/net/mvpp2/mrvl_ethdev.c         |  5 +++--
 drivers/net/netvsc/hn_ethdev.c          |  6 ++++--
 drivers/net/netvsc/hn_var.h             |  2 +-
 drivers/net/netvsc/hn_vf.c              |  7 +++++--
 drivers/net/virtio/virtio_user_ethdev.c |  4 +---
 lib/librte_ethdev/rte_ethdev.c          | 16 +++++++++++-----
 lib/librte_ethdev/rte_ethdev.h          |  5 ++++-
 16 files changed, 68 insertions(+), 39 deletions(-)

diff --git a/doc/guides/rel_notes/deprecation.rst b/doc/guides/rel_notes/deprecation.rst
index 584e720879..81cce65f0d 100644
--- a/doc/guides/rel_notes/deprecation.rst
+++ b/doc/guides/rel_notes/deprecation.rst
@@ -128,7 +128,6 @@ Deprecation Notices
   invalid port ID, unsupported operation, failed operation):
 
   - ``rte_eth_dev_stop``
-  - ``rte_eth_dev_close``
 
 * ethdev: New offload flags ``DEV_RX_OFFLOAD_FLOW_MARK`` will be added in 19.11.
   This will allow application to enable or disable PMDs from updating
diff --git a/doc/guides/rel_notes/release_20_11.rst b/doc/guides/rel_notes/release_20_11.rst
index 57e3edcdda..b7cd89a4a2 100644
--- a/doc/guides/rel_notes/release_20_11.rst
+++ b/doc/guides/rel_notes/release_20_11.rst
@@ -232,7 +232,9 @@ API Changes
 
 * ethdev: ``rte_eth_rx_descriptor_done()`` API has been deprecated.
 
-* Renamed internal ethdev APIs:
+* ethdev: Added ``int`` return type to ``rte_eth_dev_close()``.
+
+* ethdev: Renamed internal functions:
 
   * ``_rte_eth_dev_callback_process()`` -> ``rte_eth_dev_callback_process()``
   * ``_rte_eth_dev_reset`` -> ``rte_eth_dev_internal_reset()``
diff --git a/drivers/net/cxgbe/cxgbe_ethdev.c b/drivers/net/cxgbe/cxgbe_ethdev.c
index 16beb2d435..fe488231a7 100644
--- a/drivers/net/cxgbe/cxgbe_ethdev.c
+++ b/drivers/net/cxgbe/cxgbe_ethdev.c
@@ -1296,12 +1296,13 @@ static int eth_cxgbe_dev_uninit(struct rte_eth_dev *eth_dev)
 {
 	struct rte_pci_device *pci_dev = RTE_ETH_DEV_TO_PCI(eth_dev);
 	uint16_t port_id;
+	int err = 0;
 
 	/* Free up other ports and all resources */
 	RTE_ETH_FOREACH_DEV_OF(port_id, &pci_dev->device)
-		rte_eth_dev_close(port_id);
+		err |= rte_eth_dev_close(port_id);
 
-	return 0;
+	return err == 0 ? 0 : -EIO;
 }
 
 static int eth_cxgbe_pci_probe(struct rte_pci_driver *pci_drv __rte_unused,
diff --git a/drivers/net/cxgbe/cxgbevf_ethdev.c b/drivers/net/cxgbe/cxgbevf_ethdev.c
index 947fcdd406..c2918f5356 100644
--- a/drivers/net/cxgbe/cxgbevf_ethdev.c
+++ b/drivers/net/cxgbe/cxgbevf_ethdev.c
@@ -183,12 +183,13 @@ static int eth_cxgbevf_dev_uninit(struct rte_eth_dev *eth_dev)
 {
 	struct rte_pci_device *pci_dev = RTE_ETH_DEV_TO_PCI(eth_dev);
 	uint16_t port_id;
+	int err = 0;
 
 	/* Free up other ports and all resources */
 	RTE_ETH_FOREACH_DEV_OF(port_id, &pci_dev->device)
-		rte_eth_dev_close(port_id);
+		err |= rte_eth_dev_close(port_id);
 
-	return 0;
+	return err == 0 ? 0 : -EIO;
 }
 
 static int eth_cxgbevf_pci_probe(struct rte_pci_driver *pci_drv __rte_unused,
diff --git a/drivers/net/failsafe/failsafe_ether.c b/drivers/net/failsafe/failsafe_ether.c
index 7c68bbdec0..950d35dac4 100644
--- a/drivers/net/failsafe/failsafe_ether.c
+++ b/drivers/net/failsafe/failsafe_ether.c
@@ -287,7 +287,11 @@ fs_dev_remove(struct sub_device *sdev)
 		/* fallthrough */
 	case DEV_ACTIVE:
 		failsafe_eth_dev_unregister_callbacks(sdev);
-		rte_eth_dev_close(PORT_ID(sdev));
+		ret = rte_eth_dev_close(PORT_ID(sdev));
+		if (ret < 0) {
+			ERROR("Port close failed for sub-device %u",
+			      PORT_ID(sdev));
+		}
 		sdev->state = DEV_PROBED;
 		/* fallthrough */
 	case DEV_PROBED:
diff --git a/drivers/net/failsafe/failsafe_ops.c b/drivers/net/failsafe/failsafe_ops.c
index 0ce7dfc8a6..5c606ff077 100644
--- a/drivers/net/failsafe/failsafe_ops.c
+++ b/drivers/net/failsafe/failsafe_ops.c
@@ -638,7 +638,7 @@ failsafe_eth_dev_close(struct rte_eth_dev *dev)
 {
 	struct sub_device *sdev;
 	uint8_t i;
-	int ret;
+	int err, ret = 0;
 
 	fs_lock(dev, 0);
 	failsafe_hotplug_alarm_cancel(dev);
@@ -648,29 +648,38 @@ failsafe_eth_dev_close(struct rte_eth_dev *dev)
 	FOREACH_SUBDEV_STATE(sdev, i, dev, DEV_ACTIVE) {
 		DEBUG("Closing sub_device %d", i);
 		failsafe_eth_dev_unregister_callbacks(sdev);
-		rte_eth_dev_close(PORT_ID(sdev));
+		err = rte_eth_dev_close(PORT_ID(sdev));
+		if (err) {
+			ret = ret ? ret : err;
+			ERROR("Error while closing sub-device %u",
+					PORT_ID(sdev));
+		}
 		sdev->state = DEV_ACTIVE - 1;
 	}
 	rte_eth_dev_callback_unregister(RTE_ETH_ALL, RTE_ETH_EVENT_NEW,
 					failsafe_eth_new_event_callback, dev);
 	if (rte_eal_process_type() != RTE_PROC_PRIMARY) {
 		fs_unlock(dev, 0);
-		return 0;
+		return ret;
 	}
 	fs_dev_free_queues(dev);
-	ret = failsafe_eal_uninit(dev);
-	if (ret)
+	err = failsafe_eal_uninit(dev);
+	if (err) {
+		ret = ret ? ret : err;
 		ERROR("Error while uninitializing sub-EAL");
+	}
 	failsafe_args_free(dev);
 	rte_free(PRIV(dev)->subs);
 	rte_free(PRIV(dev)->mcast_addrs);
 	/* mac_addrs must not be freed alone because part of dev_private */
 	dev->data->mac_addrs = NULL;
 	fs_unlock(dev, 0);
-	ret = pthread_mutex_destroy(&PRIV(dev)->hotplug_mutex);
-	if (ret)
+	err = pthread_mutex_destroy(&PRIV(dev)->hotplug_mutex);
+	if (err) {
+		ret = ret ? ret : err;
 		ERROR("Error while destroying hotplug mutex");
-	return 0;
+	}
+	return ret;
 }
 
 static int
diff --git a/drivers/net/memif/rte_eth_memif.c b/drivers/net/memif/rte_eth_memif.c
index ff8a58081f..33bf5c68a3 100644
--- a/drivers/net/memif/rte_eth_memif.c
+++ b/drivers/net/memif/rte_eth_memif.c
@@ -1798,9 +1798,7 @@ rte_pmd_memif_remove(struct rte_vdev_device *vdev)
 	if (eth_dev == NULL)
 		return 0;
 
-	rte_eth_dev_close(eth_dev->data->port_id);
-
-	return 0;
+	return rte_eth_dev_close(eth_dev->data->port_id);
 }
 
 static struct rte_vdev_driver pmd_memif_drv = {
diff --git a/drivers/net/mlx5/mlx5.c b/drivers/net/mlx5/mlx5.c
index e5ca392fed..dc55988b10 100644
--- a/drivers/net/mlx5/mlx5.c
+++ b/drivers/net/mlx5/mlx5.c
@@ -2020,6 +2020,7 @@ static int
 mlx5_pci_remove(struct rte_pci_device *pci_dev)
 {
 	uint16_t port_id;
+	int ret = 0;
 
 	RTE_ETH_FOREACH_DEV_OF(port_id, &pci_dev->device) {
 		/*
@@ -2027,11 +2028,11 @@ mlx5_pci_remove(struct rte_pci_device *pci_dev)
 		 * call the close function explicitly for secondary process.
 		 */
 		if (rte_eal_process_type() == RTE_PROC_SECONDARY)
-			mlx5_dev_close(&rte_eth_devices[port_id]);
+			ret |= mlx5_dev_close(&rte_eth_devices[port_id]);
 		else
-			rte_eth_dev_close(port_id);
+			ret |= rte_eth_dev_close(port_id);
 	}
-	return 0;
+	return ret == 0 ? 0 : -EIO;
 }
 
 static const struct rte_pci_id mlx5_pci_id_map[] = {
diff --git a/drivers/net/mvneta/mvneta_ethdev.c b/drivers/net/mvneta/mvneta_ethdev.c
index 3c0332ab4d..1574bf35a8 100644
--- a/drivers/net/mvneta/mvneta_ethdev.c
+++ b/drivers/net/mvneta/mvneta_ethdev.c
@@ -966,14 +966,15 @@ static int
 rte_pmd_mvneta_remove(struct rte_vdev_device *vdev)
 {
 	uint16_t port_id;
+	int ret = 0;
 
 	RTE_ETH_FOREACH_DEV(port_id) {
 		if (rte_eth_devices[port_id].device != &vdev->device)
 			continue;
-		rte_eth_dev_close(port_id);
+		ret |= rte_eth_dev_close(port_id);
 	}
 
-	return 0;
+	return ret == 0 ? 0 : -EIO;
 }
 
 static struct rte_vdev_driver pmd_mvneta_drv = {
diff --git a/drivers/net/mvpp2/mrvl_ethdev.c b/drivers/net/mvpp2/mrvl_ethdev.c
index a230a96840..df9f336fab 100644
--- a/drivers/net/mvpp2/mrvl_ethdev.c
+++ b/drivers/net/mvpp2/mrvl_ethdev.c
@@ -3022,14 +3022,15 @@ static int
 rte_pmd_mrvl_remove(struct rte_vdev_device *vdev)
 {
 	uint16_t port_id;
+	int ret = 0;
 
 	RTE_ETH_FOREACH_DEV(port_id) {
 		if (rte_eth_devices[port_id].device != &vdev->device)
 			continue;
-		rte_eth_dev_close(port_id);
+		ret |= rte_eth_dev_close(port_id);
 	}
 
-	return 0;
+	return ret == 0 ? 0 : -EIO;
 }
 
 static struct rte_vdev_driver pmd_mrvl_drv = {
diff --git a/drivers/net/netvsc/hn_ethdev.c b/drivers/net/netvsc/hn_ethdev.c
index 8743e6e69a..40e6d25f34 100644
--- a/drivers/net/netvsc/hn_ethdev.c
+++ b/drivers/net/netvsc/hn_ethdev.c
@@ -842,14 +842,16 @@ hn_dev_stop(struct rte_eth_dev *dev)
 static int
 hn_dev_close(struct rte_eth_dev *dev)
 {
+	int ret;
+
 	PMD_INIT_FUNC_TRACE();
 	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
 		return 0;
 
-	hn_vf_close(dev);
+	ret = hn_vf_close(dev);
 	hn_dev_free_queues(dev);
 
-	return 0;
+	return ret;
 }
 
 static const struct eth_dev_ops hn_eth_dev_ops = {
diff --git a/drivers/net/netvsc/hn_var.h b/drivers/net/netvsc/hn_var.h
index 4b63f87607..74f30669ac 100644
--- a/drivers/net/netvsc/hn_var.h
+++ b/drivers/net/netvsc/hn_var.h
@@ -217,7 +217,7 @@ const uint32_t *hn_vf_supported_ptypes(struct rte_eth_dev *dev);
 int	hn_vf_start(struct rte_eth_dev *dev);
 void	hn_vf_reset(struct rte_eth_dev *dev);
 void	hn_vf_stop(struct rte_eth_dev *dev);
-void	hn_vf_close(struct rte_eth_dev *dev);
+int	hn_vf_close(struct rte_eth_dev *dev);
 
 int	hn_vf_allmulticast_enable(struct rte_eth_dev *dev);
 int	hn_vf_allmulticast_disable(struct rte_eth_dev *dev);
diff --git a/drivers/net/netvsc/hn_vf.c b/drivers/net/netvsc/hn_vf.c
index f5f15c0462..d29eee7627 100644
--- a/drivers/net/netvsc/hn_vf.c
+++ b/drivers/net/netvsc/hn_vf.c
@@ -316,18 +316,21 @@ void hn_vf_reset(struct rte_eth_dev *dev)
 	VF_ETHDEV_FUNC(dev, rte_eth_dev_reset);
 }
 
-void hn_vf_close(struct rte_eth_dev *dev)
+int hn_vf_close(struct rte_eth_dev *dev)
 {
 	struct hn_data *hv = dev->data->dev_private;
 	uint16_t vf_port;
+	int ret = 0;
 
 	rte_rwlock_read_lock(&hv->vf_lock);
 	vf_port = hv->vf_port;
 	if (vf_port != HN_INVALID_PORT)
-		rte_eth_dev_close(vf_port);
+		ret = rte_eth_dev_close(vf_port);
 
 	hv->vf_port = HN_INVALID_PORT;
 	rte_rwlock_read_unlock(&hv->vf_lock);
+
+	return ret;
 }
 
 int hn_vf_stats_reset(struct rte_eth_dev *dev)
diff --git a/drivers/net/virtio/virtio_user_ethdev.c b/drivers/net/virtio/virtio_user_ethdev.c
index f56fc238c4..042665bc09 100644
--- a/drivers/net/virtio/virtio_user_ethdev.c
+++ b/drivers/net/virtio/virtio_user_ethdev.c
@@ -881,9 +881,7 @@ virtio_user_pmd_remove(struct rte_vdev_device *vdev)
 		return rte_eth_dev_release_port(eth_dev);
 
 	/* make sure the device is stopped, queues freed */
-	rte_eth_dev_close(eth_dev->data->port_id);
-
-	return 0;
+	return rte_eth_dev_close(eth_dev->data->port_id);
 }
 
 static int virtio_user_pmd_dma_map(struct rte_vdev_device *vdev, void *addr,
diff --git a/lib/librte_ethdev/rte_ethdev.c b/lib/librte_ethdev/rte_ethdev.c
index 73c3d6f7fe..ad12d4f07a 100644
--- a/lib/librte_ethdev/rte_ethdev.c
+++ b/lib/librte_ethdev/rte_ethdev.c
@@ -1716,19 +1716,25 @@ rte_eth_dev_set_link_down(uint16_t port_id)
 	return eth_err(port_id, (*dev->dev_ops->dev_set_link_down)(dev));
 }
 
-void
+int
 rte_eth_dev_close(uint16_t port_id)
 {
 	struct rte_eth_dev *dev;
+	int firsterr, binerr;
+	int *lasterr = &firsterr;
 
-	RTE_ETH_VALID_PORTID_OR_RET(port_id);
+	RTE_ETH_VALID_PORTID_OR_ERR_RET(port_id, -ENODEV);
 	dev = &rte_eth_devices[port_id];
 
-	RTE_FUNC_PTR_OR_RET(*dev->dev_ops->dev_close);
-	(*dev->dev_ops->dev_close)(dev);
+	RTE_FUNC_PTR_OR_ERR_RET(*dev->dev_ops->dev_close, -ENOTSUP);
+	*lasterr = (*dev->dev_ops->dev_close)(dev);
+	if (*lasterr != 0)
+		lasterr = &binerr;
 
 	rte_ethdev_trace_close(port_id);
-	rte_eth_dev_release_port(dev);
+	*lasterr = rte_eth_dev_release_port(dev);
+
+	return firsterr;
 }
 
 int
diff --git a/lib/librte_ethdev/rte_ethdev.h b/lib/librte_ethdev/rte_ethdev.h
index 3a31f94367..ca524ea264 100644
--- a/lib/librte_ethdev/rte_ethdev.h
+++ b/lib/librte_ethdev/rte_ethdev.h
@@ -2304,8 +2304,11 @@ int rte_eth_dev_set_link_down(uint16_t port_id);
  *
  * @param port_id
  *   The port identifier of the Ethernet device.
+ * @return
+ *   - Zero if the port is closed successfully.
+ *   - Negative if something went wrong.
  */
-void rte_eth_dev_close(uint16_t port_id);
+int rte_eth_dev_close(uint16_t port_id);
 
 /**
  * Reset a Ethernet device and keep its port id.
-- 
2.28.0

Re: [dpdk-dev] [PATCH v5 3/3] ethdev: allow close function to return an error

[Andrew Rybchenko]

On 10/13/20 1:06 PM, Thomas Monjalon wrote:
> The API function rte_eth_dev_close() was returning void.
> The return type is changed to int for notifying of errors.
> 
> If an error happens during a close operation,
> the status of the port is undefined,
> a maximum of resources having been freed.
> 
> Signed-off-by: Thomas Monjalon <thomas@monjalon.net>
> Reviewed-by: Liron Himi <lironh@marvell.com>
> Acked-by: Stephen Hemminger <stephen@networkplumber.org>

Acked-by: Andrew Rybchenko <andrew.rybchenko@oktetlabs.ru>

[snip]

> diff --git a/lib/librte_ethdev/rte_ethdev.c b/lib/librte_ethdev/rte_ethdev.c
> index 73c3d6f7fe..ad12d4f07a 100644
> --- a/lib/librte_ethdev/rte_ethdev.c
> +++ b/lib/librte_ethdev/rte_ethdev.c
> @@ -1716,19 +1716,25 @@ rte_eth_dev_set_link_down(uint16_t port_id)
>  	return eth_err(port_id, (*dev->dev_ops->dev_set_link_down)(dev));
>  }
>  
> -void
> +int
>  rte_eth_dev_close(uint16_t port_id)
>  {
>  	struct rte_eth_dev *dev;
> +	int firsterr, binerr;
> +	int *lasterr = &firsterr;
>  
> -	RTE_ETH_VALID_PORTID_OR_RET(port_id);
> +	RTE_ETH_VALID_PORTID_OR_ERR_RET(port_id, -ENODEV);
>  	dev = &rte_eth_devices[port_id];
>  
> -	RTE_FUNC_PTR_OR_RET(*dev->dev_ops->dev_close);
> -	(*dev->dev_ops->dev_close)(dev);
> +	RTE_FUNC_PTR_OR_ERR_RET(*dev->dev_ops->dev_close, -ENOTSUP);
> +	*lasterr = (*dev->dev_ops->dev_close)(dev);
> +	if (*lasterr != 0)
> +		lasterr = &binerr;
>  
>  	rte_ethdev_trace_close(port_id);
> -	rte_eth_dev_release_port(dev);
> +	*lasterr = rte_eth_dev_release_port(dev);
> +
> +	return firsterr;

Shouldn't eth_err() be used here?

Re: [dpdk-dev] [PATCH v5 3/3] ethdev: allow close function to return an error

[Thomas Monjalon]

13/10/2020 12:41, Andrew Rybchenko:
> On 10/13/20 1:06 PM, Thomas Monjalon wrote:
> > The API function rte_eth_dev_close() was returning void.
> > The return type is changed to int for notifying of errors.
> > 
> > If an error happens during a close operation,
> > the status of the port is undefined,
> > a maximum of resources having been freed.
> > 
> > Signed-off-by: Thomas Monjalon <thomas@monjalon.net>
> > Reviewed-by: Liron Himi <lironh@marvell.com>
> > Acked-by: Stephen Hemminger <stephen@networkplumber.org>
> 
> Acked-by: Andrew Rybchenko <andrew.rybchenko@oktetlabs.ru>
> 
> [snip]
> 
> > diff --git a/lib/librte_ethdev/rte_ethdev.c b/lib/librte_ethdev/rte_ethdev.c
> > index 73c3d6f7fe..ad12d4f07a 100644
> > --- a/lib/librte_ethdev/rte_ethdev.c
> > +++ b/lib/librte_ethdev/rte_ethdev.c
> > @@ -1716,19 +1716,25 @@ rte_eth_dev_set_link_down(uint16_t port_id)
> >  	return eth_err(port_id, (*dev->dev_ops->dev_set_link_down)(dev));
> >  }
> >  
> > -void
> > +int
> >  rte_eth_dev_close(uint16_t port_id)
> >  {
> >  	struct rte_eth_dev *dev;
> > +	int firsterr, binerr;
> > +	int *lasterr = &firsterr;
> >  
> > -	RTE_ETH_VALID_PORTID_OR_RET(port_id);
> > +	RTE_ETH_VALID_PORTID_OR_ERR_RET(port_id, -ENODEV);
> >  	dev = &rte_eth_devices[port_id];
> >  
> > -	RTE_FUNC_PTR_OR_RET(*dev->dev_ops->dev_close);
> > -	(*dev->dev_ops->dev_close)(dev);
> > +	RTE_FUNC_PTR_OR_ERR_RET(*dev->dev_ops->dev_close, -ENOTSUP);
> > +	*lasterr = (*dev->dev_ops->dev_close)(dev);
> > +	if (*lasterr != 0)
> > +		lasterr = &binerr;
> >  
> >  	rte_ethdev_trace_close(port_id);
> > -	rte_eth_dev_release_port(dev);
> > +	*lasterr = rte_eth_dev_release_port(dev);
> > +
> > +	return firsterr;
> 
> Shouldn't eth_err() be used here?

Yes you're right (again)
Will do a v6, thanks for the review.

Re: [dpdk-dev] [PATCH v5 3/3] ethdev: allow close function to return an error

[Ferruh Yigit]

On 10/13/2020 11:06 AM, Thomas Monjalon wrote:
> The API function rte_eth_dev_close() was returning void.
> The return type is changed to int for notifying of errors.
> 
> If an error happens during a close operation,
> the status of the port is undefined,
> a maximum of resources having been freed.
> 
> Signed-off-by: Thomas Monjalon <thomas@monjalon.net>
> Reviewed-by: Liron Himi <lironh@marvell.com>
> Acked-by: Stephen Hemminger <stephen@networkplumber.org>

Reviewed-by: Ferruh Yigit <ferruh.yigit@intel.com>

[dpdk-dev] [PATCH v6 0/3] cleanup ethdev close operation

[Thomas Monjalon]

Most of the patches from the v3 have been merged already.
The remaining 3 patches are reworked in v4:
  - reset dev_started in some PMDs
  - selectively reset pointers in rte_eth_dev_release_port()
  - fix error handling

v5: change error code to ENODEV if wrong port id
v6: - use eth_err() in rte_eth_dev_close()
    - fix missed igbvf and ixgbevf


Thomas Monjalon (3):
  ethdev: remove forcing stopped state upon close
  ethdev: reset all when releasing a port
  ethdev: allow close function to return an error

 doc/guides/rel_notes/deprecation.rst    |  1 -
 doc/guides/rel_notes/release_20_11.rst  |  4 +++-
 drivers/net/ark/ark_ethdev.c            |  4 +---
 drivers/net/atlantic/atl_ethdev.c       |  5 +----
 drivers/net/axgbe/axgbe_ethdev.c        |  3 ---
 drivers/net/bnxt/bnxt_ethdev.c          |  4 ----
 drivers/net/bnxt/bnxt_reps.c            |  1 -
 drivers/net/bonding/rte_eth_bond_pmd.c  |  4 ----
 drivers/net/cxgbe/cxgbe_ethdev.c        |  5 +++--
 drivers/net/cxgbe/cxgbevf_ethdev.c      |  5 +++--
 drivers/net/dpaa/dpaa_ethdev.c          |  5 +----
 drivers/net/dpaa2/dpaa2_ethdev.c        |  4 ----
 drivers/net/e1000/em_ethdev.c           |  6 ++----
 drivers/net/e1000/igb_ethdev.c          | 10 ++--------
 drivers/net/ena/ena_ethdev.c            |  6 +-----
 drivers/net/enetc/enetc_ethdev.c        |  1 +
 drivers/net/failsafe/failsafe_ether.c   |  6 +++++-
 drivers/net/failsafe/failsafe_ops.c     | 25 ++++++++++++++++--------
 drivers/net/fm10k/fm10k_ethdev.c        |  5 +----
 drivers/net/hinic/hinic_pmd_ethdev.c    |  6 ++----
 drivers/net/hns3/hns3_ethdev.c          |  5 +----
 drivers/net/hns3/hns3_ethdev_vf.c       |  6 +-----
 drivers/net/i40e/i40e_ethdev.c          |  5 +----
 drivers/net/i40e/i40e_ethdev_vf.c       |  6 +-----
 drivers/net/iavf/iavf_ethdev.c          |  5 +----
 drivers/net/ice/ice_dcf_ethdev.c        |  4 ----
 drivers/net/ice/ice_ethdev.c            |  5 +----
 drivers/net/igc/igc_ethdev.c            |  1 +
 drivers/net/ionic/ionic_ethdev.c        |  5 -----
 drivers/net/ixgbe/ixgbe_ethdev.c        | 10 ++--------
 drivers/net/kni/rte_eth_kni.c           |  1 +
 drivers/net/liquidio/lio_ethdev.c       |  5 +----
 drivers/net/memif/rte_eth_memif.c       |  4 +---
 drivers/net/mlx5/mlx5.c                 |  7 ++++---
 drivers/net/mvneta/mvneta_ethdev.c      |  7 +++++--
 drivers/net/mvpp2/mrvl_ethdev.c         |  5 +++--
 drivers/net/netvsc/hn_ethdev.c          | 11 +++++------
 drivers/net/netvsc/hn_var.h             |  2 +-
 drivers/net/netvsc/hn_vf.c              |  7 +++++--
 drivers/net/nfb/nfb_ethdev.c            |  2 ++
 drivers/net/octeontx/octeontx_ethdev.c  |  3 ---
 drivers/net/pfe/pfe_ethdev.c            |  2 ++
 drivers/net/qede/qede_ethdev.c          |  5 +----
 drivers/net/ring/rte_eth_ring.c         |  1 +
 drivers/net/sfc/sfc_ethdev.c            |  7 +------
 drivers/net/szedata2/rte_eth_szedata2.c |  2 ++
 drivers/net/tap/rte_eth_tap.c           |  1 -
 drivers/net/thunderx/nicvf_ethdev.c     |  1 +
 drivers/net/vhost/rte_eth_vhost.c       |  1 +
 drivers/net/virtio/virtio_ethdev.c      |  5 +----
 drivers/net/virtio/virtio_user_ethdev.c |  4 +---
 drivers/net/vmxnet3/vmxnet3_ethdev.c    |  6 +-----
 lib/librte_ethdev/rte_ethdev.c          | 26 +++++++++++++++++++------
 lib/librte_ethdev/rte_ethdev.h          |  5 ++++-
 54 files changed, 116 insertions(+), 166 deletions(-)

-- 
2.28.0

[dpdk-dev] [PATCH v6 1/3] ethdev: remove forcing stopped state upon close

[Thomas Monjalon]

When closing a port, it is supposed to be already stopped,
and marked as such with "dev_started" state zeroed by the stop API.

Resetting "dev_started" before calling the driver close operation
was hiding the case of not properly stopped port being closed.
The flag "dev_started" is not changed anymore in "rte_eth_dev_close()".

In case the "dev_stop" function is called from "dev_close",
bypassing "rte_eth_dev_stop()" API,
the "dev_started" state must be explicitly reset in the PMD
in order to keep the same behaviour.

Signed-off-by: Thomas Monjalon <thomas@monjalon.net>
Acked-by: Stephen Hemminger <stephen@networkplumber.org>
Reviewed-by: Andrew Rybchenko <arybchenko@solarflare.com>
---
 drivers/net/ark/ark_ethdev.c            | 1 +
 drivers/net/atlantic/atl_ethdev.c       | 1 +
 drivers/net/dpaa/dpaa_ethdev.c          | 1 +
 drivers/net/e1000/em_ethdev.c           | 2 ++
 drivers/net/e1000/igb_ethdev.c          | 2 ++
 drivers/net/ena/ena_ethdev.c            | 1 +
 drivers/net/enetc/enetc_ethdev.c        | 1 +
 drivers/net/fm10k/fm10k_ethdev.c        | 1 +
 drivers/net/hinic/hinic_pmd_ethdev.c    | 2 ++
 drivers/net/hns3/hns3_ethdev.c          | 1 +
 drivers/net/hns3/hns3_ethdev_vf.c       | 1 +
 drivers/net/i40e/i40e_ethdev.c          | 1 +
 drivers/net/i40e/i40e_ethdev_vf.c       | 2 +-
 drivers/net/iavf/iavf_ethdev.c          | 1 +
 drivers/net/ice/ice_ethdev.c            | 1 +
 drivers/net/igc/igc_ethdev.c            | 1 +
 drivers/net/ixgbe/ixgbe_ethdev.c        | 2 ++
 drivers/net/kni/rte_eth_kni.c           | 1 +
 drivers/net/liquidio/lio_ethdev.c       | 1 +
 drivers/net/mvneta/mvneta_ethdev.c      | 2 ++
 drivers/net/netvsc/hn_ethdev.c          | 1 +
 drivers/net/nfb/nfb_ethdev.c            | 2 ++
 drivers/net/pfe/pfe_ethdev.c            | 2 ++
 drivers/net/qede/qede_ethdev.c          | 1 +
 drivers/net/ring/rte_eth_ring.c         | 1 +
 drivers/net/szedata2/rte_eth_szedata2.c | 2 ++
 drivers/net/thunderx/nicvf_ethdev.c     | 1 +
 drivers/net/vhost/rte_eth_vhost.c       | 1 +
 drivers/net/virtio/virtio_ethdev.c      | 1 +
 drivers/net/vmxnet3/vmxnet3_ethdev.c    | 1 +
 lib/librte_ethdev/rte_ethdev.c          | 1 -
 31 files changed, 38 insertions(+), 2 deletions(-)

diff --git a/drivers/net/ark/ark_ethdev.c b/drivers/net/ark/ark_ethdev.c
index e3b1347769..f1355581f1 100644
--- a/drivers/net/ark/ark_ethdev.c
+++ b/drivers/net/ark/ark_ethdev.c
@@ -595,6 +595,7 @@ eth_ark_dev_stop(struct rte_eth_dev *dev)
 	if (ark->started == 0)
 		return;
 	ark->started = 0;
+	dev->data->dev_started = 0;
 
 	/* Stop the extension first */
 	if (ark->user_ext.dev_stop)
diff --git a/drivers/net/atlantic/atl_ethdev.c b/drivers/net/atlantic/atl_ethdev.c
index 2217511ca0..f2e48e1c11 100644
--- a/drivers/net/atlantic/atl_ethdev.c
+++ b/drivers/net/atlantic/atl_ethdev.c
@@ -609,6 +609,7 @@ atl_dev_stop(struct rte_eth_dev *dev)
 	struct rte_intr_handle *intr_handle = &pci_dev->intr_handle;
 
 	PMD_INIT_FUNC_TRACE();
+	dev->data->dev_started = 0;
 
 	/* disable interrupts */
 	atl_disable_intr(hw);
diff --git a/drivers/net/dpaa/dpaa_ethdev.c b/drivers/net/dpaa/dpaa_ethdev.c
index af47c196ae..952e8f72ad 100644
--- a/drivers/net/dpaa/dpaa_ethdev.c
+++ b/drivers/net/dpaa/dpaa_ethdev.c
@@ -420,6 +420,7 @@ static void dpaa_eth_dev_stop(struct rte_eth_dev *dev)
 	struct fman_if *fif = dev->process_private;
 
 	PMD_INIT_FUNC_TRACE();
+	dev->data->dev_started = 0;
 
 	if (!fif->is_shared_mac)
 		fman_if_disable_rx(fif);
diff --git a/drivers/net/e1000/em_ethdev.c b/drivers/net/e1000/em_ethdev.c
index d050eb478a..dd9238a616 100644
--- a/drivers/net/e1000/em_ethdev.c
+++ b/drivers/net/e1000/em_ethdev.c
@@ -717,6 +717,8 @@ eth_em_stop(struct rte_eth_dev *dev)
 	struct rte_pci_device *pci_dev = RTE_ETH_DEV_TO_PCI(dev);
 	struct rte_intr_handle *intr_handle = &pci_dev->intr_handle;
 
+	dev->data->dev_started = 0;
+
 	eth_em_rxtx_control(dev, false);
 	em_rxq_intr_disable(hw);
 	em_lsc_intr_disable(hw);
diff --git a/drivers/net/e1000/igb_ethdev.c b/drivers/net/e1000/igb_ethdev.c
index cb3d97e2a3..f697b0e52e 100644
--- a/drivers/net/e1000/igb_ethdev.c
+++ b/drivers/net/e1000/igb_ethdev.c
@@ -1497,6 +1497,7 @@ eth_igb_stop(struct rte_eth_dev *dev)
 	}
 
 	adapter->stopped = true;
+	dev->data->dev_started = 0;
 }
 
 static int
@@ -3374,6 +3375,7 @@ igbvf_dev_stop(struct rte_eth_dev *dev)
 	}
 
 	adapter->stopped = true;
+	dev->data->dev_started = 0;
 }
 
 static int
diff --git a/drivers/net/ena/ena_ethdev.c b/drivers/net/ena/ena_ethdev.c
index d32fa43837..e02d340c33 100644
--- a/drivers/net/ena/ena_ethdev.c
+++ b/drivers/net/ena/ena_ethdev.c
@@ -1118,6 +1118,7 @@ static void ena_stop(struct rte_eth_dev *dev)
 
 	++adapter->dev_stats.dev_stop;
 	adapter->state = ENA_ADAPTER_STATE_STOPPED;
+	dev->data->dev_started = 0;
 }
 
 static int ena_create_io_queue(struct ena_ring *ring)
diff --git a/drivers/net/enetc/enetc_ethdev.c b/drivers/net/enetc/enetc_ethdev.c
index b3dec7e64d..dddbc89939 100644
--- a/drivers/net/enetc/enetc_ethdev.c
+++ b/drivers/net/enetc/enetc_ethdev.c
@@ -54,6 +54,7 @@ enetc_dev_stop(struct rte_eth_dev *dev)
 	uint32_t val;
 
 	PMD_INIT_FUNC_TRACE();
+	dev->data->dev_started = 0;
 	/* Disable port */
 	val = enetc_port_rd(enetc_hw, ENETC_PMR);
 	enetc_port_wr(enetc_hw, ENETC_PMR, val & (~ENETC_PMR_EN));
diff --git a/drivers/net/fm10k/fm10k_ethdev.c b/drivers/net/fm10k/fm10k_ethdev.c
index c4a6fdf7f0..0c13fd5c37 100644
--- a/drivers/net/fm10k/fm10k_ethdev.c
+++ b/drivers/net/fm10k/fm10k_ethdev.c
@@ -1161,6 +1161,7 @@ fm10k_dev_stop(struct rte_eth_dev *dev)
 	int i;
 
 	PMD_INIT_FUNC_TRACE();
+	dev->data->dev_started = 0;
 
 	if (dev->data->tx_queues)
 		for (i = 0; i < dev->data->nb_tx_queues; i++)
diff --git a/drivers/net/hinic/hinic_pmd_ethdev.c b/drivers/net/hinic/hinic_pmd_ethdev.c
index 466c8362b9..daf8f33cf3 100644
--- a/drivers/net/hinic/hinic_pmd_ethdev.c
+++ b/drivers/net/hinic/hinic_pmd_ethdev.c
@@ -1189,6 +1189,8 @@ static void hinic_dev_stop(struct rte_eth_dev *dev)
 	name = dev->data->name;
 	port_id = dev->data->port_id;
 
+	dev->data->dev_started = 0;
+
 	if (!rte_bit_relaxed_test_and_clear32(HINIC_DEV_START,
 					      &nic_dev->dev_status)) {
 		PMD_DRV_LOG(INFO, "Device %s already stopped", name);
diff --git a/drivers/net/hns3/hns3_ethdev.c b/drivers/net/hns3/hns3_ethdev.c
index ce5bae538d..8fcc0f18c5 100644
--- a/drivers/net/hns3/hns3_ethdev.c
+++ b/drivers/net/hns3/hns3_ethdev.c
@@ -4997,6 +4997,7 @@ hns3_dev_stop(struct rte_eth_dev *dev)
 	struct hns3_hw *hw = &hns->hw;
 
 	PMD_INIT_FUNC_TRACE();
+	dev->data->dev_started = 0;
 
 	hw->adapter_state = HNS3_NIC_STOPPING;
 	hns3_set_rxtx_function(dev);
diff --git a/drivers/net/hns3/hns3_ethdev_vf.c b/drivers/net/hns3/hns3_ethdev_vf.c
index 1a19c0e6e6..b2038395a2 100644
--- a/drivers/net/hns3/hns3_ethdev_vf.c
+++ b/drivers/net/hns3/hns3_ethdev_vf.c
@@ -1952,6 +1952,7 @@ hns3vf_dev_stop(struct rte_eth_dev *dev)
 	struct hns3_hw *hw = &hns->hw;
 
 	PMD_INIT_FUNC_TRACE();
+	dev->data->dev_started = 0;
 
 	hw->adapter_state = HNS3_NIC_STOPPING;
 	hns3_set_rxtx_function(dev);
diff --git a/drivers/net/i40e/i40e_ethdev.c b/drivers/net/i40e/i40e_ethdev.c
index 943cfe71dc..825de51fbb 100644
--- a/drivers/net/i40e/i40e_ethdev.c
+++ b/drivers/net/i40e/i40e_ethdev.c
@@ -2599,6 +2599,7 @@ i40e_dev_stop(struct rte_eth_dev *dev)
 	pf->tm_conf.committed = false;
 
 	hw->adapter_stopped = 1;
+	dev->data->dev_started = 0;
 
 	pf->adapter->rss_reta_updated = 0;
 }
diff --git a/drivers/net/i40e/i40e_ethdev_vf.c b/drivers/net/i40e/i40e_ethdev_vf.c
index 4d6510d1ff..40c9745433 100644
--- a/drivers/net/i40e/i40e_ethdev_vf.c
+++ b/drivers/net/i40e/i40e_ethdev_vf.c
@@ -2207,7 +2207,7 @@ i40evf_dev_stop(struct rte_eth_dev *dev)
 	i40evf_add_del_mc_addr_list(dev, vf->mc_addrs, vf->mc_addrs_num,
 				FALSE);
 	hw->adapter_stopped = 1;
-
+	dev->data->dev_started = 0;
 }
 
 static int
diff --git a/drivers/net/iavf/iavf_ethdev.c b/drivers/net/iavf/iavf_ethdev.c
index f5e6e852ae..8bd2c048a1 100644
--- a/drivers/net/iavf/iavf_ethdev.c
+++ b/drivers/net/iavf/iavf_ethdev.c
@@ -562,6 +562,7 @@ iavf_dev_stop(struct rte_eth_dev *dev)
 				  false);
 
 	adapter->stopped = 1;
+	dev->data->dev_started = 0;
 }
 
 static int
diff --git a/drivers/net/ice/ice_ethdev.c b/drivers/net/ice/ice_ethdev.c
index d8ce09d28f..1db4015c23 100644
--- a/drivers/net/ice/ice_ethdev.c
+++ b/drivers/net/ice/ice_ethdev.c
@@ -2376,6 +2376,7 @@ ice_dev_stop(struct rte_eth_dev *dev)
 	}
 
 	pf->adapter_stopped = true;
+	dev->data->dev_started = 0;
 }
 
 static int
diff --git a/drivers/net/igc/igc_ethdev.c b/drivers/net/igc/igc_ethdev.c
index 7f5066df4b..73b7d20ac2 100644
--- a/drivers/net/igc/igc_ethdev.c
+++ b/drivers/net/igc/igc_ethdev.c
@@ -616,6 +616,7 @@ eth_igc_stop(struct rte_eth_dev *dev)
 	struct rte_intr_handle *intr_handle = &pci_dev->intr_handle;
 	struct rte_eth_link link;
 
+	dev->data->dev_started = 0;
 	adapter->stopped = 1;
 
 	/* disable receive and transmit */
diff --git a/drivers/net/ixgbe/ixgbe_ethdev.c b/drivers/net/ixgbe/ixgbe_ethdev.c
index 0b98e210e7..a45c59644b 100644
--- a/drivers/net/ixgbe/ixgbe_ethdev.c
+++ b/drivers/net/ixgbe/ixgbe_ethdev.c
@@ -2917,6 +2917,7 @@ ixgbe_dev_stop(struct rte_eth_dev *dev)
 	adapter->rss_reta_updated = 0;
 
 	hw->adapter_stopped = true;
+	dev->data->dev_started = 0;
 }
 
 /*
@@ -5418,6 +5419,7 @@ ixgbevf_dev_stop(struct rte_eth_dev *dev)
 
 	ixgbevf_intr_disable(dev);
 
+	dev->data->dev_started = 0;
 	hw->adapter_stopped = 1;
 	ixgbe_stop_adapter(hw);
 
diff --git a/drivers/net/kni/rte_eth_kni.c b/drivers/net/kni/rte_eth_kni.c
index be747adf86..329a55039f 100644
--- a/drivers/net/kni/rte_eth_kni.c
+++ b/drivers/net/kni/rte_eth_kni.c
@@ -196,6 +196,7 @@ eth_kni_dev_stop(struct rte_eth_dev *dev)
 	}
 
 	dev->data->dev_link.link_status = 0;
+	dev->data->dev_started = 0;
 }
 
 static int
diff --git a/drivers/net/liquidio/lio_ethdev.c b/drivers/net/liquidio/lio_ethdev.c
index 1a41f27198..52bcefc819 100644
--- a/drivers/net/liquidio/lio_ethdev.c
+++ b/drivers/net/liquidio/lio_ethdev.c
@@ -1471,6 +1471,7 @@ lio_dev_stop(struct rte_eth_dev *eth_dev)
 	struct lio_device *lio_dev = LIO_DEV(eth_dev);
 
 	lio_dev_info(lio_dev, "Stopping port %d\n", eth_dev->data->port_id);
+	eth_dev->data->dev_started = 0;
 	lio_dev->intf_open = 0;
 	rte_mb();
 
diff --git a/drivers/net/mvneta/mvneta_ethdev.c b/drivers/net/mvneta/mvneta_ethdev.c
index 607771149a..3c0332ab4d 100644
--- a/drivers/net/mvneta/mvneta_ethdev.c
+++ b/drivers/net/mvneta/mvneta_ethdev.c
@@ -413,6 +413,8 @@ mvneta_dev_stop(struct rte_eth_dev *dev)
 {
 	struct mvneta_priv *priv = dev->data->dev_private;
 
+	dev->data->dev_started = 0;
+
 	if (!priv->ppio)
 		return;
 
diff --git a/drivers/net/netvsc/hn_ethdev.c b/drivers/net/netvsc/hn_ethdev.c
index 5ae2d469c8..be3713dfa0 100644
--- a/drivers/net/netvsc/hn_ethdev.c
+++ b/drivers/net/netvsc/hn_ethdev.c
@@ -833,6 +833,7 @@ hn_dev_stop(struct rte_eth_dev *dev)
 	struct hn_data *hv = dev->data->dev_private;
 
 	PMD_INIT_FUNC_TRACE();
+	dev->data->dev_started = 0;
 
 	hn_rndis_set_rxfilter(hv, 0);
 	hn_vf_stop(dev);
diff --git a/drivers/net/nfb/nfb_ethdev.c b/drivers/net/nfb/nfb_ethdev.c
index a9a8bc878d..44e57ee6c8 100644
--- a/drivers/net/nfb/nfb_ethdev.c
+++ b/drivers/net/nfb/nfb_ethdev.c
@@ -158,6 +158,8 @@ nfb_eth_dev_stop(struct rte_eth_dev *dev)
 	uint16_t nb_rx = dev->data->nb_rx_queues;
 	uint16_t nb_tx = dev->data->nb_tx_queues;
 
+	dev->data->dev_started = 0;
+
 	for (i = 0; i < nb_tx; i++)
 		nfb_eth_tx_queue_stop(dev, i);
 
diff --git a/drivers/net/pfe/pfe_ethdev.c b/drivers/net/pfe/pfe_ethdev.c
index f0de1c8a2e..c4dd0a5f69 100644
--- a/drivers/net/pfe/pfe_ethdev.c
+++ b/drivers/net/pfe/pfe_ethdev.c
@@ -378,6 +378,8 @@ pfe_eth_stop(struct rte_eth_dev *dev/*, int wake*/)
 {
 	struct pfe_eth_priv_s *priv = dev->data->dev_private;
 
+	dev->data->dev_started = 0;
+
 	gemac_disable(priv->EMAC_baseaddr);
 	gpi_disable(priv->GPI_baseaddr);
 
diff --git a/drivers/net/qede/qede_ethdev.c b/drivers/net/qede/qede_ethdev.c
index 548497f3ae..88389c9e4f 100644
--- a/drivers/net/qede/qede_ethdev.c
+++ b/drivers/net/qede/qede_ethdev.c
@@ -1169,6 +1169,7 @@ static void qede_dev_stop(struct rte_eth_dev *eth_dev)
 	struct ecore_dev *edev = QEDE_INIT_EDEV(qdev);
 
 	PMD_INIT_FUNC_TRACE(edev);
+	eth_dev->data->dev_started = 0;
 
 	/* Bring the link down */
 	qede_dev_set_link_state(eth_dev, false);
diff --git a/drivers/net/ring/rte_eth_ring.c b/drivers/net/ring/rte_eth_ring.c
index 83c5502123..6dceab0c59 100644
--- a/drivers/net/ring/rte_eth_ring.c
+++ b/drivers/net/ring/rte_eth_ring.c
@@ -108,6 +108,7 @@ eth_dev_start(struct rte_eth_dev *dev)
 static void
 eth_dev_stop(struct rte_eth_dev *dev)
 {
+	dev->data->dev_started = 0;
 	dev->data->dev_link.link_status = ETH_LINK_DOWN;
 }
 
diff --git a/drivers/net/szedata2/rte_eth_szedata2.c b/drivers/net/szedata2/rte_eth_szedata2.c
index 0eecec1e8c..c89ff16fc1 100644
--- a/drivers/net/szedata2/rte_eth_szedata2.c
+++ b/drivers/net/szedata2/rte_eth_szedata2.c
@@ -1020,6 +1020,8 @@ eth_dev_stop(struct rte_eth_dev *dev)
 	uint16_t nb_rx = dev->data->nb_rx_queues;
 	uint16_t nb_tx = dev->data->nb_tx_queues;
 
+	dev->data->dev_started = 0;
+
 	for (i = 0; i < nb_tx; i++)
 		eth_tx_queue_stop(dev, i);
 
diff --git a/drivers/net/thunderx/nicvf_ethdev.c b/drivers/net/thunderx/nicvf_ethdev.c
index cc6eb4ba24..b87e1e5640 100644
--- a/drivers/net/thunderx/nicvf_ethdev.c
+++ b/drivers/net/thunderx/nicvf_ethdev.c
@@ -1766,6 +1766,7 @@ nicvf_dev_stop_cleanup(struct rte_eth_dev *dev, bool cleanup)
 	struct nicvf *nic = nicvf_pmd_priv(dev);
 
 	PMD_INIT_FUNC_TRACE();
+	dev->data->dev_started = 0;
 
 	/* Teardown secondary vf first */
 	for (i = 0; i < nic->sqs_count; i++) {
diff --git a/drivers/net/vhost/rte_eth_vhost.c b/drivers/net/vhost/rte_eth_vhost.c
index 66efecb320..5328cfcd02 100644
--- a/drivers/net/vhost/rte_eth_vhost.c
+++ b/drivers/net/vhost/rte_eth_vhost.c
@@ -1158,6 +1158,7 @@ eth_dev_stop(struct rte_eth_dev *dev)
 {
 	struct pmd_internal *internal = dev->data->dev_private;
 
+	dev->data->dev_started = 0;
 	rte_atomic32_set(&internal->started, 0);
 	update_queuing_status(dev);
 }
diff --git a/drivers/net/virtio/virtio_ethdev.c b/drivers/net/virtio/virtio_ethdev.c
index 65eefa2a31..aa8de55fe9 100644
--- a/drivers/net/virtio/virtio_ethdev.c
+++ b/drivers/net/virtio/virtio_ethdev.c
@@ -2523,6 +2523,7 @@ virtio_dev_stop(struct rte_eth_dev *dev)
 	struct rte_intr_conf *intr_conf = &dev->data->dev_conf.intr_conf;
 
 	PMD_INIT_LOG(DEBUG, "stop");
+	dev->data->dev_started = 0;
 
 	rte_spinlock_lock(&hw->state_lock);
 	if (!hw->started)
diff --git a/drivers/net/vmxnet3/vmxnet3_ethdev.c b/drivers/net/vmxnet3/vmxnet3_ethdev.c
index fa950e1ba0..828043a65a 100644
--- a/drivers/net/vmxnet3/vmxnet3_ethdev.c
+++ b/drivers/net/vmxnet3/vmxnet3_ethdev.c
@@ -858,6 +858,7 @@ vmxnet3_dev_stop(struct rte_eth_dev *dev)
 	rte_eth_linkstatus_set(dev, &link);
 
 	hw->adapter_stopped = 1;
+	dev->data->dev_started = 0;
 }
 
 static void
diff --git a/lib/librte_ethdev/rte_ethdev.c b/lib/librte_ethdev/rte_ethdev.c
index c8ed890686..bec12f6814 100644
--- a/lib/librte_ethdev/rte_ethdev.c
+++ b/lib/librte_ethdev/rte_ethdev.c
@@ -1718,7 +1718,6 @@ rte_eth_dev_close(uint16_t port_id)
 	dev = &rte_eth_devices[port_id];
 
 	RTE_FUNC_PTR_OR_RET(*dev->dev_ops->dev_close);
-	dev->data->dev_started = 0;
 	(*dev->dev_ops->dev_close)(dev);
 
 	rte_ethdev_trace_close(port_id);
-- 
2.28.0

[dpdk-dev] [PATCH v6 2/3] ethdev: reset all when releasing a port

[Thomas Monjalon]

The function rte_eth_dev_release_port() is partially resetting
the struct rte_eth_dev. The drivers were completing this reset
with more pointers set to NULL in the close or remove operations.

More pointers are reset at ethdev level,
and some redundant assignments are removed from PMDs.

Signed-off-by: Thomas Monjalon <thomas@monjalon.net>
Acked-by: Stephen Hemminger <stephen@networkplumber.org>
Reviewed-by: Haiyue Wang <haiyue.wang@intel.com>
Acked-by: Jeff Guo <jia.guo@intel.com>
Reviewed-by: Andrew Rybchenko <arybchenko@solarflare.com>
Reviewed-by: Ferruh Yigit <ferruh.yigit@intel.com>
---
 drivers/net/ark/ark_ethdev.c           | 3 ---
 drivers/net/atlantic/atl_ethdev.c      | 4 ----
 drivers/net/axgbe/axgbe_ethdev.c       | 3 ---
 drivers/net/bnxt/bnxt_ethdev.c         | 4 ----
 drivers/net/bnxt/bnxt_reps.c           | 1 -
 drivers/net/bonding/rte_eth_bond_pmd.c | 4 ----
 drivers/net/dpaa/dpaa_ethdev.c         | 4 ----
 drivers/net/dpaa2/dpaa2_ethdev.c       | 4 ----
 drivers/net/e1000/em_ethdev.c          | 4 ----
 drivers/net/e1000/igb_ethdev.c         | 8 --------
 drivers/net/ena/ena_ethdev.c           | 5 -----
 drivers/net/fm10k/fm10k_ethdev.c       | 4 ----
 drivers/net/hinic/hinic_pmd_ethdev.c   | 4 ----
 drivers/net/hns3/hns3_ethdev.c         | 4 ----
 drivers/net/hns3/hns3_ethdev_vf.c      | 5 -----
 drivers/net/i40e/i40e_ethdev.c         | 4 ----
 drivers/net/i40e/i40e_ethdev_vf.c      | 4 ----
 drivers/net/iavf/iavf_ethdev.c         | 4 ----
 drivers/net/ice/ice_dcf_ethdev.c       | 4 ----
 drivers/net/ice/ice_ethdev.c           | 4 ----
 drivers/net/ionic/ionic_ethdev.c       | 5 -----
 drivers/net/ixgbe/ixgbe_ethdev.c       | 8 --------
 drivers/net/liquidio/lio_ethdev.c      | 4 ----
 drivers/net/netvsc/hn_ethdev.c         | 4 ----
 drivers/net/octeontx/octeontx_ethdev.c | 3 ---
 drivers/net/qede/qede_ethdev.c         | 4 ----
 drivers/net/sfc/sfc_ethdev.c           | 7 +------
 drivers/net/tap/rte_eth_tap.c          | 1 -
 drivers/net/virtio/virtio_ethdev.c     | 4 ----
 drivers/net/vmxnet3/vmxnet3_ethdev.c   | 5 -----
 lib/librte_ethdev/rte_ethdev.c         | 9 +++++++++
 31 files changed, 10 insertions(+), 125 deletions(-)

diff --git a/drivers/net/ark/ark_ethdev.c b/drivers/net/ark/ark_ethdev.c
index f1355581f1..dc14f0fb1a 100644
--- a/drivers/net/ark/ark_ethdev.c
+++ b/drivers/net/ark/ark_ethdev.c
@@ -504,9 +504,6 @@ eth_ark_dev_uninit(struct rte_eth_dev *dev)
 	ark_pktgen_uninit(ark->pg);
 	ark_pktchkr_uninit(ark->pc);
 
-	dev->dev_ops = NULL;
-	dev->rx_pkt_burst = NULL;
-	dev->tx_pkt_burst = NULL;
 	return 0;
 }
 
diff --git a/drivers/net/atlantic/atl_ethdev.c b/drivers/net/atlantic/atl_ethdev.c
index f2e48e1c11..d96865d5b4 100644
--- a/drivers/net/atlantic/atl_ethdev.c
+++ b/drivers/net/atlantic/atl_ethdev.c
@@ -702,10 +702,6 @@ atl_dev_close(struct rte_eth_dev *dev)
 
 	atl_free_queues(dev);
 
-	dev->dev_ops = NULL;
-	dev->rx_pkt_burst = NULL;
-	dev->tx_pkt_burst = NULL;
-
 	/* disable uio intr before callback unregister */
 	rte_intr_disable(intr_handle);
 	rte_intr_callback_unregister(intr_handle,
diff --git a/drivers/net/axgbe/axgbe_ethdev.c b/drivers/net/axgbe/axgbe_ethdev.c
index cf085487cc..39087f36f0 100644
--- a/drivers/net/axgbe/axgbe_ethdev.c
+++ b/drivers/net/axgbe/axgbe_ethdev.c
@@ -2134,9 +2134,6 @@ axgbe_dev_close(struct rte_eth_dev *eth_dev)
 		return 0;
 
 	pci_dev = RTE_DEV_TO_PCI(eth_dev->device);
-	eth_dev->dev_ops = NULL;
-	eth_dev->rx_pkt_burst = NULL;
-	eth_dev->tx_pkt_burst = NULL;
 	axgbe_dev_clear_queues(eth_dev);
 
 	/* disable uio intr before callback unregister */
diff --git a/drivers/net/bnxt/bnxt_ethdev.c b/drivers/net/bnxt/bnxt_ethdev.c
index 8b63134c39..3d32b578e8 100644
--- a/drivers/net/bnxt/bnxt_ethdev.c
+++ b/drivers/net/bnxt/bnxt_ethdev.c
@@ -1433,10 +1433,6 @@ static int bnxt_dev_close_op(struct rte_eth_dev *eth_dev)
 	bnxt_free_pf_info(bp);
 	bnxt_free_parent_info(bp);
 
-	eth_dev->dev_ops = NULL;
-	eth_dev->rx_pkt_burst = NULL;
-	eth_dev->tx_pkt_burst = NULL;
-
 	rte_memzone_free((const struct rte_memzone *)bp->tx_mem_zone);
 	bp->tx_mem_zone = NULL;
 	rte_memzone_free((const struct rte_memzone *)bp->rx_mem_zone);
diff --git a/drivers/net/bnxt/bnxt_reps.c b/drivers/net/bnxt/bnxt_reps.c
index 74a76fce57..012ccc73fe 100644
--- a/drivers/net/bnxt/bnxt_reps.c
+++ b/drivers/net/bnxt/bnxt_reps.c
@@ -255,7 +255,6 @@ int bnxt_representor_uninit(struct rte_eth_dev *eth_dev)
 
 	PMD_DRV_LOG(DEBUG, "BNXT Port:%d VFR uninit\n", eth_dev->data->port_id);
 	eth_dev->data->mac_addrs = NULL;
-	eth_dev->dev_ops = NULL;
 
 	parent_bp = rep->parent_dev->data->dev_private;
 	if (!parent_bp) {
diff --git a/drivers/net/bonding/rte_eth_bond_pmd.c b/drivers/net/bonding/rte_eth_bond_pmd.c
index 1f761c7c9e..ce8fe254af 100644
--- a/drivers/net/bonding/rte_eth_bond_pmd.c
+++ b/drivers/net/bonding/rte_eth_bond_pmd.c
@@ -2130,10 +2130,6 @@ bond_ethdev_close(struct rte_eth_dev *dev)
 	 */
 	rte_mempool_free(internals->mode6.mempool);
 
-	dev->dev_ops = NULL;
-	dev->rx_pkt_burst = NULL;
-	dev->tx_pkt_burst = NULL;
-
 	return 0;
 }
 
diff --git a/drivers/net/dpaa/dpaa_ethdev.c b/drivers/net/dpaa/dpaa_ethdev.c
index 952e8f72ad..6d27d742af 100644
--- a/drivers/net/dpaa/dpaa_ethdev.c
+++ b/drivers/net/dpaa/dpaa_ethdev.c
@@ -504,10 +504,6 @@ static int dpaa_eth_dev_close(struct rte_eth_dev *dev)
 	rte_free(dpaa_intf->tx_queues);
 	dpaa_intf->tx_queues = NULL;
 
-	dev->dev_ops = NULL;
-	dev->rx_pkt_burst = NULL;
-	dev->tx_pkt_burst = NULL;
-
 	return 0;
 }
 
diff --git a/drivers/net/dpaa2/dpaa2_ethdev.c b/drivers/net/dpaa2/dpaa2_ethdev.c
index d8624514df..ba99a524f9 100644
--- a/drivers/net/dpaa2/dpaa2_ethdev.c
+++ b/drivers/net/dpaa2/dpaa2_ethdev.c
@@ -1284,10 +1284,6 @@ dpaa2_dev_close(struct rte_eth_dev *dev)
 	if (priv->extract.qos_extract_param)
 		rte_free((void *)(size_t)priv->extract.qos_extract_param);
 
-	dev->dev_ops = NULL;
-	dev->rx_pkt_burst = NULL;
-	dev->tx_pkt_burst = NULL;
-
 	DPAA2_PMD_INFO("%s: netdev deleted", dev->data->name);
 	return 0;
 }
diff --git a/drivers/net/e1000/em_ethdev.c b/drivers/net/e1000/em_ethdev.c
index dd9238a616..304c040421 100644
--- a/drivers/net/e1000/em_ethdev.c
+++ b/drivers/net/e1000/em_ethdev.c
@@ -774,10 +774,6 @@ eth_em_close(struct rte_eth_dev *dev)
 	em_release_manageability(hw);
 	em_hw_control_release(hw);
 
-	dev->dev_ops = NULL;
-	dev->rx_pkt_burst = NULL;
-	dev->tx_pkt_burst = NULL;
-
 	/* disable uio intr before callback unregister */
 	rte_intr_disable(intr_handle);
 	rte_intr_callback_unregister(intr_handle,
diff --git a/drivers/net/e1000/igb_ethdev.c b/drivers/net/e1000/igb_ethdev.c
index f697b0e52e..9a3f9cb669 100644
--- a/drivers/net/e1000/igb_ethdev.c
+++ b/drivers/net/e1000/igb_ethdev.c
@@ -1565,10 +1565,6 @@ eth_igb_close(struct rte_eth_dev *dev)
 	memset(&link, 0, sizeof(link));
 	rte_eth_linkstatus_set(dev, &link);
 
-	dev->dev_ops = NULL;
-	dev->rx_pkt_burst = NULL;
-	dev->tx_pkt_burst = NULL;
-
 	/* Reset any pending lock */
 	igb_reset_swfw_lock(hw);
 
@@ -3405,10 +3401,6 @@ igbvf_dev_close(struct rte_eth_dev *dev)
 	memset(&addr, 0, sizeof(addr));
 	igbvf_default_mac_addr_set(dev, &addr);
 
-	dev->dev_ops = NULL;
-	dev->rx_pkt_burst = NULL;
-	dev->tx_pkt_burst = NULL;
-
 	rte_intr_callback_unregister(&pci_dev->intr_handle,
 				     eth_igbvf_interrupt_handler,
 				     (void *)dev);
diff --git a/drivers/net/ena/ena_ethdev.c b/drivers/net/ena/ena_ethdev.c
index e02d340c33..bdb8fe79c6 100644
--- a/drivers/net/ena/ena_ethdev.c
+++ b/drivers/net/ena/ena_ethdev.c
@@ -1956,11 +1956,6 @@ static int eth_ena_dev_uninit(struct rte_eth_dev *eth_dev)
 
 	ena_destroy_device(eth_dev);
 
-	eth_dev->dev_ops = NULL;
-	eth_dev->rx_pkt_burst = NULL;
-	eth_dev->tx_pkt_burst = NULL;
-	eth_dev->tx_pkt_prepare = NULL;
-
 	return 0;
 }
 
diff --git a/drivers/net/fm10k/fm10k_ethdev.c b/drivers/net/fm10k/fm10k_ethdev.c
index 0c13fd5c37..bada69c64a 100644
--- a/drivers/net/fm10k/fm10k_ethdev.c
+++ b/drivers/net/fm10k/fm10k_ethdev.c
@@ -2805,10 +2805,6 @@ fm10k_dev_close(struct rte_eth_dev *dev)
 	fm10k_dev_queue_release(dev);
 	fm10k_stop_hw(hw);
 
-	dev->dev_ops = NULL;
-	dev->rx_pkt_burst = NULL;
-	dev->tx_pkt_burst = NULL;
-
 	/* disable uio/vfio intr */
 	rte_intr_disable(intr_handle);
 
diff --git a/drivers/net/hinic/hinic_pmd_ethdev.c b/drivers/net/hinic/hinic_pmd_ethdev.c
index daf8f33cf3..2504228d94 100644
--- a/drivers/net/hinic/hinic_pmd_ethdev.c
+++ b/drivers/net/hinic/hinic_pmd_ethdev.c
@@ -3253,10 +3253,6 @@ static int hinic_dev_uninit(struct rte_eth_dev *dev)
 
 	hinic_dev_close(dev);
 
-	dev->dev_ops = NULL;
-	dev->rx_pkt_burst = NULL;
-	dev->tx_pkt_burst = NULL;
-
 	rte_free(nic_dev->mc_list);
 
 	return HINIC_OK;
diff --git a/drivers/net/hns3/hns3_ethdev.c b/drivers/net/hns3/hns3_ethdev.c
index 8fcc0f18c5..c850e5eb14 100644
--- a/drivers/net/hns3/hns3_ethdev.c
+++ b/drivers/net/hns3/hns3_ethdev.c
@@ -6201,10 +6201,6 @@ hns3_dev_uninit(struct rte_eth_dev *eth_dev)
 	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
 		return -EPERM;
 
-	eth_dev->dev_ops = NULL;
-	eth_dev->rx_pkt_burst = NULL;
-	eth_dev->tx_pkt_burst = NULL;
-	eth_dev->tx_pkt_prepare = NULL;
 	if (hw->adapter_state < HNS3_NIC_CLOSING)
 		hns3_dev_close(eth_dev);
 
diff --git a/drivers/net/hns3/hns3_ethdev_vf.c b/drivers/net/hns3/hns3_ethdev_vf.c
index b2038395a2..6e169583a0 100644
--- a/drivers/net/hns3/hns3_ethdev_vf.c
+++ b/drivers/net/hns3/hns3_ethdev_vf.c
@@ -2847,11 +2847,6 @@ hns3vf_dev_uninit(struct rte_eth_dev *eth_dev)
 	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
 		return -EPERM;
 
-	eth_dev->dev_ops = NULL;
-	eth_dev->rx_pkt_burst = NULL;
-	eth_dev->tx_pkt_burst = NULL;
-	eth_dev->tx_pkt_prepare = NULL;
-
 	if (hw->adapter_state < HNS3_NIC_CLOSING)
 		hns3vf_dev_close(eth_dev);
 
diff --git a/drivers/net/i40e/i40e_ethdev.c b/drivers/net/i40e/i40e_ethdev.c
index 825de51fbb..605a1792b6 100644
--- a/drivers/net/i40e/i40e_ethdev.c
+++ b/drivers/net/i40e/i40e_ethdev.c
@@ -2693,10 +2693,6 @@ i40e_dev_close(struct rte_eth_dev *dev)
 			(reg | I40E_PFGEN_CTRL_PFSWR_MASK));
 	I40E_WRITE_FLUSH(hw);
 
-	dev->dev_ops = NULL;
-	dev->rx_pkt_burst = NULL;
-	dev->tx_pkt_burst = NULL;
-
 	/* Clear PXE mode */
 	i40e_clear_pxe_mode(hw);
 
diff --git a/drivers/net/i40e/i40e_ethdev_vf.c b/drivers/net/i40e/i40e_ethdev_vf.c
index 40c9745433..3dcd50c428 100644
--- a/drivers/net/i40e/i40e_ethdev_vf.c
+++ b/drivers/net/i40e/i40e_ethdev_vf.c
@@ -2421,10 +2421,6 @@ i40evf_dev_close(struct rte_eth_dev *dev)
 	i40e_shutdown_adminq(hw);
 	i40evf_disable_irq0(hw);
 
-	dev->dev_ops = NULL;
-	dev->rx_pkt_burst = NULL;
-	dev->tx_pkt_burst = NULL;
-
 	rte_free(vf->vf_res);
 	vf->vf_res = NULL;
 	rte_free(vf->aq_resp);
diff --git a/drivers/net/iavf/iavf_ethdev.c b/drivers/net/iavf/iavf_ethdev.c
index 8bd2c048a1..c951e75358 100644
--- a/drivers/net/iavf/iavf_ethdev.c
+++ b/drivers/net/iavf/iavf_ethdev.c
@@ -1525,10 +1525,6 @@ iavf_dev_close(struct rte_eth_dev *dev)
 				     iavf_dev_interrupt_handler, dev);
 	iavf_disable_irq0(hw);
 
-	dev->dev_ops = NULL;
-	dev->rx_pkt_burst = NULL;
-	dev->tx_pkt_burst = NULL;
-
 	if (vf->vf_res->vf_cap_flags & VIRTCHNL_VF_OFFLOAD_RSS_PF) {
 		if (vf->rss_lut) {
 			rte_free(vf->rss_lut);
diff --git a/drivers/net/ice/ice_dcf_ethdev.c b/drivers/net/ice/ice_dcf_ethdev.c
index 33dd0c44f2..f2056e2aba 100644
--- a/drivers/net/ice/ice_dcf_ethdev.c
+++ b/drivers/net/ice/ice_dcf_ethdev.c
@@ -855,10 +855,6 @@ ice_dcf_dev_close(struct rte_eth_dev *dev)
 	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
 		return 0;
 
-	dev->dev_ops = NULL;
-	dev->rx_pkt_burst = NULL;
-	dev->tx_pkt_burst = NULL;
-
 	ice_dcf_uninit_parent_adapter(dev);
 	ice_dcf_uninit_hw(dev, &adapter->real_hw);
 
diff --git a/drivers/net/ice/ice_ethdev.c b/drivers/net/ice/ice_ethdev.c
index 1db4015c23..33b2828703 100644
--- a/drivers/net/ice/ice_ethdev.c
+++ b/drivers/net/ice/ice_ethdev.c
@@ -2417,10 +2417,6 @@ ice_dev_close(struct rte_eth_dev *dev)
 	rte_free(pf->proto_xtr);
 	pf->proto_xtr = NULL;
 
-	dev->dev_ops = NULL;
-	dev->rx_pkt_burst = NULL;
-	dev->tx_pkt_burst = NULL;
-
 	/* disable uio intr before callback unregister */
 	rte_intr_disable(intr_handle);
 
diff --git a/drivers/net/ionic/ionic_ethdev.c b/drivers/net/ionic/ionic_ethdev.c
index ef7d06e526..1387253dc8 100644
--- a/drivers/net/ionic/ionic_ethdev.c
+++ b/drivers/net/ionic/ionic_ethdev.c
@@ -1065,11 +1065,6 @@ eth_ionic_dev_uninit(struct rte_eth_dev *eth_dev)
 	ionic_lif_deinit(lif);
 	ionic_lif_free(lif);
 
-	eth_dev->dev_ops = NULL;
-	eth_dev->rx_pkt_burst = NULL;
-	eth_dev->tx_pkt_burst = NULL;
-	eth_dev->tx_pkt_prepare = NULL;
-
 	return 0;
 }
 
diff --git a/drivers/net/ixgbe/ixgbe_ethdev.c b/drivers/net/ixgbe/ixgbe_ethdev.c
index a45c59644b..841c6877b2 100644
--- a/drivers/net/ixgbe/ixgbe_ethdev.c
+++ b/drivers/net/ixgbe/ixgbe_ethdev.c
@@ -3010,10 +3010,6 @@ ixgbe_dev_close(struct rte_eth_dev *dev)
 	/* reprogram the RAR[0] in case user changed it. */
 	ixgbe_set_rar(hw, 0, hw->mac.addr, 0, IXGBE_RAH_AV);
 
-	dev->dev_ops = NULL;
-	dev->rx_pkt_burst = NULL;
-	dev->tx_pkt_burst = NULL;
-
 	/* Unlock any pending hardware semaphore */
 	ixgbe_swfw_lock_reset(hw);
 
@@ -5468,10 +5464,6 @@ ixgbevf_dev_close(struct rte_eth_dev *dev)
 	 **/
 	ixgbevf_remove_mac_addr(dev, 0);
 
-	dev->dev_ops = NULL;
-	dev->rx_pkt_burst = NULL;
-	dev->tx_pkt_burst = NULL;
-
 	rte_intr_disable(intr_handle);
 	rte_intr_callback_unregister(intr_handle,
 				     ixgbevf_dev_interrupt_handler, dev);
diff --git a/drivers/net/liquidio/lio_ethdev.c b/drivers/net/liquidio/lio_ethdev.c
index 52bcefc819..1b01502e49 100644
--- a/drivers/net/liquidio/lio_ethdev.c
+++ b/drivers/net/liquidio/lio_ethdev.c
@@ -2068,10 +2068,6 @@ lio_eth_dev_uninit(struct rte_eth_dev *eth_dev)
 	/* lio_free_sc_buffer_pool */
 	lio_free_sc_buffer_pool(lio_dev);
 
-	eth_dev->dev_ops = NULL;
-	eth_dev->rx_pkt_burst = NULL;
-	eth_dev->tx_pkt_burst = NULL;
-
 	return 0;
 }
 
diff --git a/drivers/net/netvsc/hn_ethdev.c b/drivers/net/netvsc/hn_ethdev.c
index be3713dfa0..8743e6e69a 100644
--- a/drivers/net/netvsc/hn_ethdev.c
+++ b/drivers/net/netvsc/hn_ethdev.c
@@ -1050,10 +1050,6 @@ eth_hn_dev_uninit(struct rte_eth_dev *eth_dev)
 	hn_dev_stop(eth_dev);
 	hn_dev_close(eth_dev);
 
-	eth_dev->dev_ops = NULL;
-	eth_dev->tx_pkt_burst = NULL;
-	eth_dev->rx_pkt_burst = NULL;
-
 	hn_detach(hv);
 	hn_chim_uninit(eth_dev);
 	rte_vmbus_chan_close(hv->primary->chan);
diff --git a/drivers/net/octeontx/octeontx_ethdev.c b/drivers/net/octeontx/octeontx_ethdev.c
index aa9ef3bb70..0ea6de448a 100644
--- a/drivers/net/octeontx/octeontx_ethdev.c
+++ b/drivers/net/octeontx/octeontx_ethdev.c
@@ -513,9 +513,6 @@ octeontx_dev_close(struct rte_eth_dev *dev)
 
 	octeontx_port_close(nic);
 
-	dev->tx_pkt_burst = NULL;
-	dev->rx_pkt_burst = NULL;
-
 	return 0;
 }
 
diff --git a/drivers/net/qede/qede_ethdev.c b/drivers/net/qede/qede_ethdev.c
index 88389c9e4f..0dd2f78820 100644
--- a/drivers/net/qede/qede_ethdev.c
+++ b/drivers/net/qede/qede_ethdev.c
@@ -1594,10 +1594,6 @@ static int qede_dev_close(struct rte_eth_dev *eth_dev)
 	if (ECORE_IS_CMT(edev))
 		rte_eal_alarm_cancel(qede_poll_sp_sb_cb, (void *)eth_dev);
 
-	eth_dev->dev_ops = NULL;
-	eth_dev->rx_pkt_burst = NULL;
-	eth_dev->tx_pkt_burst = NULL;
-
 	return 0;
 }
 
diff --git a/drivers/net/sfc/sfc_ethdev.c b/drivers/net/sfc/sfc_ethdev.c
index f41d0f5fe2..620e541a10 100644
--- a/drivers/net/sfc/sfc_ethdev.c
+++ b/drivers/net/sfc/sfc_ethdev.c
@@ -324,11 +324,7 @@ static void
 sfc_eth_dev_secondary_clear_ops(struct rte_eth_dev *dev)
 {
 	free(dev->process_private);
-	dev->process_private = NULL;
-	dev->dev_ops = NULL;
-	dev->tx_pkt_prepare = NULL;
-	dev->tx_pkt_burst = NULL;
-	dev->rx_pkt_burst = NULL;
+	rte_eth_dev_release_port(dev);
 }
 
 static int
@@ -380,7 +376,6 @@ sfc_dev_close(struct rte_eth_dev *dev)
 	/* Required for logging, so cleanup last */
 	sa->eth_dev = NULL;
 
-	dev->process_private = NULL;
 	free(sa);
 
 	return 0;
diff --git a/drivers/net/tap/rte_eth_tap.c b/drivers/net/tap/rte_eth_tap.c
index b127ce62dc..0598de00ed 100644
--- a/drivers/net/tap/rte_eth_tap.c
+++ b/drivers/net/tap/rte_eth_tap.c
@@ -1134,7 +1134,6 @@ tap_dev_close(struct rte_eth_dev *dev)
 		internals->ioctl_sock = -1;
 	}
 	rte_free(dev->process_private);
-	dev->process_private = NULL;
 	if (tap_devices_count == 1)
 		rte_mp_action_unregister(TAP_MP_KEY);
 	tap_devices_count--;
diff --git a/drivers/net/virtio/virtio_ethdev.c b/drivers/net/virtio/virtio_ethdev.c
index aa8de55fe9..e492e822c4 100644
--- a/drivers/net/virtio/virtio_ethdev.c
+++ b/drivers/net/virtio/virtio_ethdev.c
@@ -2001,10 +2001,6 @@ eth_virtio_dev_uninit(struct rte_eth_dev *eth_dev)
 	virtio_dev_stop(eth_dev);
 	virtio_dev_close(eth_dev);
 
-	eth_dev->dev_ops = NULL;
-	eth_dev->tx_pkt_burst = NULL;
-	eth_dev->rx_pkt_burst = NULL;
-
 	PMD_INIT_LOG(DEBUG, "dev_uninit completed");
 
 	return 0;
diff --git a/drivers/net/vmxnet3/vmxnet3_ethdev.c b/drivers/net/vmxnet3/vmxnet3_ethdev.c
index 828043a65a..1d070fbd79 100644
--- a/drivers/net/vmxnet3/vmxnet3_ethdev.c
+++ b/drivers/net/vmxnet3/vmxnet3_ethdev.c
@@ -368,11 +368,6 @@ eth_vmxnet3_dev_uninit(struct rte_eth_dev *eth_dev)
 		return -EBUSY;
 	}
 
-	eth_dev->dev_ops = NULL;
-	eth_dev->rx_pkt_burst = NULL;
-	eth_dev->tx_pkt_burst = NULL;
-	eth_dev->tx_pkt_prepare = NULL;
-
 	return 0;
 }
 
diff --git a/lib/librte_ethdev/rte_ethdev.c b/lib/librte_ethdev/rte_ethdev.c
index bec12f6814..3ae2253b9d 100644
--- a/lib/librte_ethdev/rte_ethdev.c
+++ b/lib/librte_ethdev/rte_ethdev.c
@@ -557,7 +557,16 @@ rte_eth_dev_release_port(struct rte_eth_dev *eth_dev)
 
 	eth_dev->state = RTE_ETH_DEV_UNUSED;
 	eth_dev->device = NULL;
+	eth_dev->process_private = NULL;
 	eth_dev->intr_handle = NULL;
+	eth_dev->rx_pkt_burst = NULL;
+	eth_dev->tx_pkt_burst = NULL;
+	eth_dev->tx_pkt_prepare = NULL;
+	eth_dev->rx_queue_count = NULL;
+	eth_dev->rx_descriptor_done = NULL;
+	eth_dev->rx_descriptor_status = NULL;
+	eth_dev->tx_descriptor_status = NULL;
+	eth_dev->dev_ops = NULL;
 
 	if (rte_eal_process_type() == RTE_PROC_PRIMARY) {
 		rte_free(eth_dev->data->rx_queues);
-- 
2.28.0

Re: [dpdk-dev] [PATCH v6 2/3] ethdev: reset all when releasing a port

[Ajit Khaparde]

On Fri, Oct 16, 2020 at 6:33 AM Thomas Monjalon <thomas@monjalon.net> wrote:
>
> The function rte_eth_dev_release_port() is partially resetting
> the struct rte_eth_dev. The drivers were completing this reset
> with more pointers set to NULL in the close or remove operations.
>
> More pointers are reset at ethdev level,
> and some redundant assignments are removed from PMDs.
>
> Signed-off-by: Thomas Monjalon <thomas@monjalon.net>
> Acked-by: Stephen Hemminger <stephen@networkplumber.org>
> Reviewed-by: Haiyue Wang <haiyue.wang@intel.com>
> Acked-by: Jeff Guo <jia.guo@intel.com>
> Reviewed-by: Andrew Rybchenko <arybchenko@solarflare.com>
> Reviewed-by: Ferruh Yigit <ferruh.yigit@intel.com>
Acked-by: Ajit Khaparde <ajit.khaparde@broadcom.com>

> ---
>  drivers/net/ark/ark_ethdev.c           | 3 ---
>  drivers/net/atlantic/atl_ethdev.c      | 4 ----
>  drivers/net/axgbe/axgbe_ethdev.c       | 3 ---
>  drivers/net/bnxt/bnxt_ethdev.c         | 4 ----
>  drivers/net/bnxt/bnxt_reps.c           | 1 -
>  drivers/net/bonding/rte_eth_bond_pmd.c | 4 ----
>  drivers/net/dpaa/dpaa_ethdev.c         | 4 ----
>  drivers/net/dpaa2/dpaa2_ethdev.c       | 4 ----
>  drivers/net/e1000/em_ethdev.c          | 4 ----
>  drivers/net/e1000/igb_ethdev.c         | 8 --------
>  drivers/net/ena/ena_ethdev.c           | 5 -----
>  drivers/net/fm10k/fm10k_ethdev.c       | 4 ----
>  drivers/net/hinic/hinic_pmd_ethdev.c   | 4 ----
>  drivers/net/hns3/hns3_ethdev.c         | 4 ----
>  drivers/net/hns3/hns3_ethdev_vf.c      | 5 -----
>  drivers/net/i40e/i40e_ethdev.c         | 4 ----
>  drivers/net/i40e/i40e_ethdev_vf.c      | 4 ----
>  drivers/net/iavf/iavf_ethdev.c         | 4 ----
>  drivers/net/ice/ice_dcf_ethdev.c       | 4 ----
>  drivers/net/ice/ice_ethdev.c           | 4 ----
>  drivers/net/ionic/ionic_ethdev.c       | 5 -----
>  drivers/net/ixgbe/ixgbe_ethdev.c       | 8 --------
>  drivers/net/liquidio/lio_ethdev.c      | 4 ----
>  drivers/net/netvsc/hn_ethdev.c         | 4 ----
>  drivers/net/octeontx/octeontx_ethdev.c | 3 ---
>  drivers/net/qede/qede_ethdev.c         | 4 ----
>  drivers/net/sfc/sfc_ethdev.c           | 7 +------
>  drivers/net/tap/rte_eth_tap.c          | 1 -
>  drivers/net/virtio/virtio_ethdev.c     | 4 ----
>  drivers/net/vmxnet3/vmxnet3_ethdev.c   | 5 -----
>  lib/librte_ethdev/rte_ethdev.c         | 9 +++++++++
>  31 files changed, 10 insertions(+), 125 deletions(-)
>
> diff --git a/drivers/net/ark/ark_ethdev.c b/drivers/net/ark/ark_ethdev.c
> index f1355581f1..dc14f0fb1a 100644
> --- a/drivers/net/ark/ark_ethdev.c
> +++ b/drivers/net/ark/ark_ethdev.c
> @@ -504,9 +504,6 @@ eth_ark_dev_uninit(struct rte_eth_dev *dev)
>         ark_pktgen_uninit(ark->pg);
>         ark_pktchkr_uninit(ark->pc);
>
> -       dev->dev_ops = NULL;
> -       dev->rx_pkt_burst = NULL;
> -       dev->tx_pkt_burst = NULL;
>         return 0;
>  }
>
> diff --git a/drivers/net/atlantic/atl_ethdev.c b/drivers/net/atlantic/atl_ethdev.c
> index f2e48e1c11..d96865d5b4 100644
> --- a/drivers/net/atlantic/atl_ethdev.c
> +++ b/drivers/net/atlantic/atl_ethdev.c
> @@ -702,10 +702,6 @@ atl_dev_close(struct rte_eth_dev *dev)
>
>         atl_free_queues(dev);
>
> -       dev->dev_ops = NULL;
> -       dev->rx_pkt_burst = NULL;
> -       dev->tx_pkt_burst = NULL;
> -
>         /* disable uio intr before callback unregister */
>         rte_intr_disable(intr_handle);
>         rte_intr_callback_unregister(intr_handle,
> diff --git a/drivers/net/axgbe/axgbe_ethdev.c b/drivers/net/axgbe/axgbe_ethdev.c
> index cf085487cc..39087f36f0 100644
> --- a/drivers/net/axgbe/axgbe_ethdev.c
> +++ b/drivers/net/axgbe/axgbe_ethdev.c
> @@ -2134,9 +2134,6 @@ axgbe_dev_close(struct rte_eth_dev *eth_dev)
>                 return 0;
>
>         pci_dev = RTE_DEV_TO_PCI(eth_dev->device);
> -       eth_dev->dev_ops = NULL;
> -       eth_dev->rx_pkt_burst = NULL;
> -       eth_dev->tx_pkt_burst = NULL;
>         axgbe_dev_clear_queues(eth_dev);
>
>         /* disable uio intr before callback unregister */
> diff --git a/drivers/net/bnxt/bnxt_ethdev.c b/drivers/net/bnxt/bnxt_ethdev.c
> index 8b63134c39..3d32b578e8 100644
> --- a/drivers/net/bnxt/bnxt_ethdev.c
> +++ b/drivers/net/bnxt/bnxt_ethdev.c
> @@ -1433,10 +1433,6 @@ static int bnxt_dev_close_op(struct rte_eth_dev *eth_dev)
>         bnxt_free_pf_info(bp);
>         bnxt_free_parent_info(bp);
>
> -       eth_dev->dev_ops = NULL;
> -       eth_dev->rx_pkt_burst = NULL;
> -       eth_dev->tx_pkt_burst = NULL;
> -
>         rte_memzone_free((const struct rte_memzone *)bp->tx_mem_zone);
>         bp->tx_mem_zone = NULL;
>         rte_memzone_free((const struct rte_memzone *)bp->rx_mem_zone);
> diff --git a/drivers/net/bnxt/bnxt_reps.c b/drivers/net/bnxt/bnxt_reps.c
> index 74a76fce57..012ccc73fe 100644
> --- a/drivers/net/bnxt/bnxt_reps.c
> +++ b/drivers/net/bnxt/bnxt_reps.c
> @@ -255,7 +255,6 @@ int bnxt_representor_uninit(struct rte_eth_dev *eth_dev)
>
>         PMD_DRV_LOG(DEBUG, "BNXT Port:%d VFR uninit\n", eth_dev->data->port_id);
>         eth_dev->data->mac_addrs = NULL;
> -       eth_dev->dev_ops = NULL;
>
>         parent_bp = rep->parent_dev->data->dev_private;
>         if (!parent_bp) {
> diff --git a/drivers/net/bonding/rte_eth_bond_pmd.c b/drivers/net/bonding/rte_eth_bond_pmd.c
> index 1f761c7c9e..ce8fe254af 100644
> --- a/drivers/net/bonding/rte_eth_bond_pmd.c
> +++ b/drivers/net/bonding/rte_eth_bond_pmd.c
> @@ -2130,10 +2130,6 @@ bond_ethdev_close(struct rte_eth_dev *dev)
>          */
>         rte_mempool_free(internals->mode6.mempool);
>
> -       dev->dev_ops = NULL;
> -       dev->rx_pkt_burst = NULL;
> -       dev->tx_pkt_burst = NULL;
> -
>         return 0;
>  }
>
> diff --git a/drivers/net/dpaa/dpaa_ethdev.c b/drivers/net/dpaa/dpaa_ethdev.c
> index 952e8f72ad..6d27d742af 100644
> --- a/drivers/net/dpaa/dpaa_ethdev.c
> +++ b/drivers/net/dpaa/dpaa_ethdev.c
> @@ -504,10 +504,6 @@ static int dpaa_eth_dev_close(struct rte_eth_dev *dev)
>         rte_free(dpaa_intf->tx_queues);
>         dpaa_intf->tx_queues = NULL;
>
> -       dev->dev_ops = NULL;
> -       dev->rx_pkt_burst = NULL;
> -       dev->tx_pkt_burst = NULL;
> -
>         return 0;
>  }
>
> diff --git a/drivers/net/dpaa2/dpaa2_ethdev.c b/drivers/net/dpaa2/dpaa2_ethdev.c
> index d8624514df..ba99a524f9 100644
> --- a/drivers/net/dpaa2/dpaa2_ethdev.c
> +++ b/drivers/net/dpaa2/dpaa2_ethdev.c
> @@ -1284,10 +1284,6 @@ dpaa2_dev_close(struct rte_eth_dev *dev)
>         if (priv->extract.qos_extract_param)
>                 rte_free((void *)(size_t)priv->extract.qos_extract_param);
>
> -       dev->dev_ops = NULL;
> -       dev->rx_pkt_burst = NULL;
> -       dev->tx_pkt_burst = NULL;
> -
>         DPAA2_PMD_INFO("%s: netdev deleted", dev->data->name);
>         return 0;
>  }
> diff --git a/drivers/net/e1000/em_ethdev.c b/drivers/net/e1000/em_ethdev.c
> index dd9238a616..304c040421 100644
> --- a/drivers/net/e1000/em_ethdev.c
> +++ b/drivers/net/e1000/em_ethdev.c
> @@ -774,10 +774,6 @@ eth_em_close(struct rte_eth_dev *dev)
>         em_release_manageability(hw);
>         em_hw_control_release(hw);
>
> -       dev->dev_ops = NULL;
> -       dev->rx_pkt_burst = NULL;
> -       dev->tx_pkt_burst = NULL;
> -
>         /* disable uio intr before callback unregister */
>         rte_intr_disable(intr_handle);
>         rte_intr_callback_unregister(intr_handle,
> diff --git a/drivers/net/e1000/igb_ethdev.c b/drivers/net/e1000/igb_ethdev.c
> index f697b0e52e..9a3f9cb669 100644
> --- a/drivers/net/e1000/igb_ethdev.c
> +++ b/drivers/net/e1000/igb_ethdev.c
> @@ -1565,10 +1565,6 @@ eth_igb_close(struct rte_eth_dev *dev)
>         memset(&link, 0, sizeof(link));
>         rte_eth_linkstatus_set(dev, &link);
>
> -       dev->dev_ops = NULL;
> -       dev->rx_pkt_burst = NULL;
> -       dev->tx_pkt_burst = NULL;
> -
>         /* Reset any pending lock */
>         igb_reset_swfw_lock(hw);
>
> @@ -3405,10 +3401,6 @@ igbvf_dev_close(struct rte_eth_dev *dev)
>         memset(&addr, 0, sizeof(addr));
>         igbvf_default_mac_addr_set(dev, &addr);
>
> -       dev->dev_ops = NULL;
> -       dev->rx_pkt_burst = NULL;
> -       dev->tx_pkt_burst = NULL;
> -
>         rte_intr_callback_unregister(&pci_dev->intr_handle,
>                                      eth_igbvf_interrupt_handler,
>                                      (void *)dev);
> diff --git a/drivers/net/ena/ena_ethdev.c b/drivers/net/ena/ena_ethdev.c
> index e02d340c33..bdb8fe79c6 100644
> --- a/drivers/net/ena/ena_ethdev.c
> +++ b/drivers/net/ena/ena_ethdev.c
> @@ -1956,11 +1956,6 @@ static int eth_ena_dev_uninit(struct rte_eth_dev *eth_dev)
>
>         ena_destroy_device(eth_dev);
>
> -       eth_dev->dev_ops = NULL;
> -       eth_dev->rx_pkt_burst = NULL;
> -       eth_dev->tx_pkt_burst = NULL;
> -       eth_dev->tx_pkt_prepare = NULL;
> -
>         return 0;
>  }
>
> diff --git a/drivers/net/fm10k/fm10k_ethdev.c b/drivers/net/fm10k/fm10k_ethdev.c
> index 0c13fd5c37..bada69c64a 100644
> --- a/drivers/net/fm10k/fm10k_ethdev.c
> +++ b/drivers/net/fm10k/fm10k_ethdev.c
> @@ -2805,10 +2805,6 @@ fm10k_dev_close(struct rte_eth_dev *dev)
>         fm10k_dev_queue_release(dev);
>         fm10k_stop_hw(hw);
>
> -       dev->dev_ops = NULL;
> -       dev->rx_pkt_burst = NULL;
> -       dev->tx_pkt_burst = NULL;
> -
>         /* disable uio/vfio intr */
>         rte_intr_disable(intr_handle);
>
> diff --git a/drivers/net/hinic/hinic_pmd_ethdev.c b/drivers/net/hinic/hinic_pmd_ethdev.c
> index daf8f33cf3..2504228d94 100644
> --- a/drivers/net/hinic/hinic_pmd_ethdev.c
> +++ b/drivers/net/hinic/hinic_pmd_ethdev.c
> @@ -3253,10 +3253,6 @@ static int hinic_dev_uninit(struct rte_eth_dev *dev)
>
>         hinic_dev_close(dev);
>
> -       dev->dev_ops = NULL;
> -       dev->rx_pkt_burst = NULL;
> -       dev->tx_pkt_burst = NULL;
> -
>         rte_free(nic_dev->mc_list);
>
>         return HINIC_OK;
> diff --git a/drivers/net/hns3/hns3_ethdev.c b/drivers/net/hns3/hns3_ethdev.c
> index 8fcc0f18c5..c850e5eb14 100644
> --- a/drivers/net/hns3/hns3_ethdev.c
> +++ b/drivers/net/hns3/hns3_ethdev.c
> @@ -6201,10 +6201,6 @@ hns3_dev_uninit(struct rte_eth_dev *eth_dev)
>         if (rte_eal_process_type() != RTE_PROC_PRIMARY)
>                 return -EPERM;
>
> -       eth_dev->dev_ops = NULL;
> -       eth_dev->rx_pkt_burst = NULL;
> -       eth_dev->tx_pkt_burst = NULL;
> -       eth_dev->tx_pkt_prepare = NULL;
>         if (hw->adapter_state < HNS3_NIC_CLOSING)
>                 hns3_dev_close(eth_dev);
>
> diff --git a/drivers/net/hns3/hns3_ethdev_vf.c b/drivers/net/hns3/hns3_ethdev_vf.c
> index b2038395a2..6e169583a0 100644
> --- a/drivers/net/hns3/hns3_ethdev_vf.c
> +++ b/drivers/net/hns3/hns3_ethdev_vf.c
> @@ -2847,11 +2847,6 @@ hns3vf_dev_uninit(struct rte_eth_dev *eth_dev)
>         if (rte_eal_process_type() != RTE_PROC_PRIMARY)
>                 return -EPERM;
>
> -       eth_dev->dev_ops = NULL;
> -       eth_dev->rx_pkt_burst = NULL;
> -       eth_dev->tx_pkt_burst = NULL;
> -       eth_dev->tx_pkt_prepare = NULL;
> -
>         if (hw->adapter_state < HNS3_NIC_CLOSING)
>                 hns3vf_dev_close(eth_dev);
>
> diff --git a/drivers/net/i40e/i40e_ethdev.c b/drivers/net/i40e/i40e_ethdev.c
> index 825de51fbb..605a1792b6 100644
> --- a/drivers/net/i40e/i40e_ethdev.c
> +++ b/drivers/net/i40e/i40e_ethdev.c
> @@ -2693,10 +2693,6 @@ i40e_dev_close(struct rte_eth_dev *dev)
>                         (reg | I40E_PFGEN_CTRL_PFSWR_MASK));
>         I40E_WRITE_FLUSH(hw);
>
> -       dev->dev_ops = NULL;
> -       dev->rx_pkt_burst = NULL;
> -       dev->tx_pkt_burst = NULL;
> -
>         /* Clear PXE mode */
>         i40e_clear_pxe_mode(hw);
>
> diff --git a/drivers/net/i40e/i40e_ethdev_vf.c b/drivers/net/i40e/i40e_ethdev_vf.c
> index 40c9745433..3dcd50c428 100644
> --- a/drivers/net/i40e/i40e_ethdev_vf.c
> +++ b/drivers/net/i40e/i40e_ethdev_vf.c
> @@ -2421,10 +2421,6 @@ i40evf_dev_close(struct rte_eth_dev *dev)
>         i40e_shutdown_adminq(hw);
>         i40evf_disable_irq0(hw);
>
> -       dev->dev_ops = NULL;
> -       dev->rx_pkt_burst = NULL;
> -       dev->tx_pkt_burst = NULL;
> -
>         rte_free(vf->vf_res);
>         vf->vf_res = NULL;
>         rte_free(vf->aq_resp);
> diff --git a/drivers/net/iavf/iavf_ethdev.c b/drivers/net/iavf/iavf_ethdev.c
> index 8bd2c048a1..c951e75358 100644
> --- a/drivers/net/iavf/iavf_ethdev.c
> +++ b/drivers/net/iavf/iavf_ethdev.c
> @@ -1525,10 +1525,6 @@ iavf_dev_close(struct rte_eth_dev *dev)
>                                      iavf_dev_interrupt_handler, dev);
>         iavf_disable_irq0(hw);
>
> -       dev->dev_ops = NULL;
> -       dev->rx_pkt_burst = NULL;
> -       dev->tx_pkt_burst = NULL;
> -
>         if (vf->vf_res->vf_cap_flags & VIRTCHNL_VF_OFFLOAD_RSS_PF) {
>                 if (vf->rss_lut) {
>                         rte_free(vf->rss_lut);
> diff --git a/drivers/net/ice/ice_dcf_ethdev.c b/drivers/net/ice/ice_dcf_ethdev.c
> index 33dd0c44f2..f2056e2aba 100644
> --- a/drivers/net/ice/ice_dcf_ethdev.c
> +++ b/drivers/net/ice/ice_dcf_ethdev.c
> @@ -855,10 +855,6 @@ ice_dcf_dev_close(struct rte_eth_dev *dev)
>         if (rte_eal_process_type() != RTE_PROC_PRIMARY)
>                 return 0;
>
> -       dev->dev_ops = NULL;
> -       dev->rx_pkt_burst = NULL;
> -       dev->tx_pkt_burst = NULL;
> -
>         ice_dcf_uninit_parent_adapter(dev);
>         ice_dcf_uninit_hw(dev, &adapter->real_hw);
>
> diff --git a/drivers/net/ice/ice_ethdev.c b/drivers/net/ice/ice_ethdev.c
> index 1db4015c23..33b2828703 100644
> --- a/drivers/net/ice/ice_ethdev.c
> +++ b/drivers/net/ice/ice_ethdev.c
> @@ -2417,10 +2417,6 @@ ice_dev_close(struct rte_eth_dev *dev)
>         rte_free(pf->proto_xtr);
>         pf->proto_xtr = NULL;
>
> -       dev->dev_ops = NULL;
> -       dev->rx_pkt_burst = NULL;
> -       dev->tx_pkt_burst = NULL;
> -
>         /* disable uio intr before callback unregister */
>         rte_intr_disable(intr_handle);
>
> diff --git a/drivers/net/ionic/ionic_ethdev.c b/drivers/net/ionic/ionic_ethdev.c
> index ef7d06e526..1387253dc8 100644
> --- a/drivers/net/ionic/ionic_ethdev.c
> +++ b/drivers/net/ionic/ionic_ethdev.c
> @@ -1065,11 +1065,6 @@ eth_ionic_dev_uninit(struct rte_eth_dev *eth_dev)
>         ionic_lif_deinit(lif);
>         ionic_lif_free(lif);
>
> -       eth_dev->dev_ops = NULL;
> -       eth_dev->rx_pkt_burst = NULL;
> -       eth_dev->tx_pkt_burst = NULL;
> -       eth_dev->tx_pkt_prepare = NULL;
> -
>         return 0;
>  }
>
> diff --git a/drivers/net/ixgbe/ixgbe_ethdev.c b/drivers/net/ixgbe/ixgbe_ethdev.c
> index a45c59644b..841c6877b2 100644
> --- a/drivers/net/ixgbe/ixgbe_ethdev.c
> +++ b/drivers/net/ixgbe/ixgbe_ethdev.c
> @@ -3010,10 +3010,6 @@ ixgbe_dev_close(struct rte_eth_dev *dev)
>         /* reprogram the RAR[0] in case user changed it. */
>         ixgbe_set_rar(hw, 0, hw->mac.addr, 0, IXGBE_RAH_AV);
>
> -       dev->dev_ops = NULL;
> -       dev->rx_pkt_burst = NULL;
> -       dev->tx_pkt_burst = NULL;
> -
>         /* Unlock any pending hardware semaphore */
>         ixgbe_swfw_lock_reset(hw);
>
> @@ -5468,10 +5464,6 @@ ixgbevf_dev_close(struct rte_eth_dev *dev)
>          **/
>         ixgbevf_remove_mac_addr(dev, 0);
>
> -       dev->dev_ops = NULL;
> -       dev->rx_pkt_burst = NULL;
> -       dev->tx_pkt_burst = NULL;
> -
>         rte_intr_disable(intr_handle);
>         rte_intr_callback_unregister(intr_handle,
>                                      ixgbevf_dev_interrupt_handler, dev);
> diff --git a/drivers/net/liquidio/lio_ethdev.c b/drivers/net/liquidio/lio_ethdev.c
> index 52bcefc819..1b01502e49 100644
> --- a/drivers/net/liquidio/lio_ethdev.c
> +++ b/drivers/net/liquidio/lio_ethdev.c
> @@ -2068,10 +2068,6 @@ lio_eth_dev_uninit(struct rte_eth_dev *eth_dev)
>         /* lio_free_sc_buffer_pool */
>         lio_free_sc_buffer_pool(lio_dev);
>
> -       eth_dev->dev_ops = NULL;
> -       eth_dev->rx_pkt_burst = NULL;
> -       eth_dev->tx_pkt_burst = NULL;
> -
>         return 0;
>  }
>
> diff --git a/drivers/net/netvsc/hn_ethdev.c b/drivers/net/netvsc/hn_ethdev.c
> index be3713dfa0..8743e6e69a 100644
> --- a/drivers/net/netvsc/hn_ethdev.c
> +++ b/drivers/net/netvsc/hn_ethdev.c
> @@ -1050,10 +1050,6 @@ eth_hn_dev_uninit(struct rte_eth_dev *eth_dev)
>         hn_dev_stop(eth_dev);
>         hn_dev_close(eth_dev);
>
> -       eth_dev->dev_ops = NULL;
> -       eth_dev->tx_pkt_burst = NULL;
> -       eth_dev->rx_pkt_burst = NULL;
> -
>         hn_detach(hv);
>         hn_chim_uninit(eth_dev);
>         rte_vmbus_chan_close(hv->primary->chan);
> diff --git a/drivers/net/octeontx/octeontx_ethdev.c b/drivers/net/octeontx/octeontx_ethdev.c
> index aa9ef3bb70..0ea6de448a 100644
> --- a/drivers/net/octeontx/octeontx_ethdev.c
> +++ b/drivers/net/octeontx/octeontx_ethdev.c
> @@ -513,9 +513,6 @@ octeontx_dev_close(struct rte_eth_dev *dev)
>
>         octeontx_port_close(nic);
>
> -       dev->tx_pkt_burst = NULL;
> -       dev->rx_pkt_burst = NULL;
> -
>         return 0;
>  }
>
> diff --git a/drivers/net/qede/qede_ethdev.c b/drivers/net/qede/qede_ethdev.c
> index 88389c9e4f..0dd2f78820 100644
> --- a/drivers/net/qede/qede_ethdev.c
> +++ b/drivers/net/qede/qede_ethdev.c
> @@ -1594,10 +1594,6 @@ static int qede_dev_close(struct rte_eth_dev *eth_dev)
>         if (ECORE_IS_CMT(edev))
>                 rte_eal_alarm_cancel(qede_poll_sp_sb_cb, (void *)eth_dev);
>
> -       eth_dev->dev_ops = NULL;
> -       eth_dev->rx_pkt_burst = NULL;
> -       eth_dev->tx_pkt_burst = NULL;
> -
>         return 0;
>  }
>
> diff --git a/drivers/net/sfc/sfc_ethdev.c b/drivers/net/sfc/sfc_ethdev.c
> index f41d0f5fe2..620e541a10 100644
> --- a/drivers/net/sfc/sfc_ethdev.c
> +++ b/drivers/net/sfc/sfc_ethdev.c
> @@ -324,11 +324,7 @@ static void
>  sfc_eth_dev_secondary_clear_ops(struct rte_eth_dev *dev)
>  {
>         free(dev->process_private);
> -       dev->process_private = NULL;
> -       dev->dev_ops = NULL;
> -       dev->tx_pkt_prepare = NULL;
> -       dev->tx_pkt_burst = NULL;
> -       dev->rx_pkt_burst = NULL;
> +       rte_eth_dev_release_port(dev);
>  }
>
>  static int
> @@ -380,7 +376,6 @@ sfc_dev_close(struct rte_eth_dev *dev)
>         /* Required for logging, so cleanup last */
>         sa->eth_dev = NULL;
>
> -       dev->process_private = NULL;
>         free(sa);
>
>         return 0;
> diff --git a/drivers/net/tap/rte_eth_tap.c b/drivers/net/tap/rte_eth_tap.c
> index b127ce62dc..0598de00ed 100644
> --- a/drivers/net/tap/rte_eth_tap.c
> +++ b/drivers/net/tap/rte_eth_tap.c
> @@ -1134,7 +1134,6 @@ tap_dev_close(struct rte_eth_dev *dev)
>                 internals->ioctl_sock = -1;
>         }
>         rte_free(dev->process_private);
> -       dev->process_private = NULL;
>         if (tap_devices_count == 1)
>                 rte_mp_action_unregister(TAP_MP_KEY);
>         tap_devices_count--;
> diff --git a/drivers/net/virtio/virtio_ethdev.c b/drivers/net/virtio/virtio_ethdev.c
> index aa8de55fe9..e492e822c4 100644
> --- a/drivers/net/virtio/virtio_ethdev.c
> +++ b/drivers/net/virtio/virtio_ethdev.c
> @@ -2001,10 +2001,6 @@ eth_virtio_dev_uninit(struct rte_eth_dev *eth_dev)
>         virtio_dev_stop(eth_dev);
>         virtio_dev_close(eth_dev);
>
> -       eth_dev->dev_ops = NULL;
> -       eth_dev->tx_pkt_burst = NULL;
> -       eth_dev->rx_pkt_burst = NULL;
> -
>         PMD_INIT_LOG(DEBUG, "dev_uninit completed");
>
>         return 0;
> diff --git a/drivers/net/vmxnet3/vmxnet3_ethdev.c b/drivers/net/vmxnet3/vmxnet3_ethdev.c
> index 828043a65a..1d070fbd79 100644
> --- a/drivers/net/vmxnet3/vmxnet3_ethdev.c
> +++ b/drivers/net/vmxnet3/vmxnet3_ethdev.c
> @@ -368,11 +368,6 @@ eth_vmxnet3_dev_uninit(struct rte_eth_dev *eth_dev)
>                 return -EBUSY;
>         }
>
> -       eth_dev->dev_ops = NULL;
> -       eth_dev->rx_pkt_burst = NULL;
> -       eth_dev->tx_pkt_burst = NULL;
> -       eth_dev->tx_pkt_prepare = NULL;
> -
>         return 0;
>  }
>
> diff --git a/lib/librte_ethdev/rte_ethdev.c b/lib/librte_ethdev/rte_ethdev.c
> index bec12f6814..3ae2253b9d 100644
> --- a/lib/librte_ethdev/rte_ethdev.c
> +++ b/lib/librte_ethdev/rte_ethdev.c
> @@ -557,7 +557,16 @@ rte_eth_dev_release_port(struct rte_eth_dev *eth_dev)
>
>         eth_dev->state = RTE_ETH_DEV_UNUSED;
>         eth_dev->device = NULL;
> +       eth_dev->process_private = NULL;
>         eth_dev->intr_handle = NULL;
> +       eth_dev->rx_pkt_burst = NULL;
> +       eth_dev->tx_pkt_burst = NULL;
> +       eth_dev->tx_pkt_prepare = NULL;
> +       eth_dev->rx_queue_count = NULL;
> +       eth_dev->rx_descriptor_done = NULL;
> +       eth_dev->rx_descriptor_status = NULL;
> +       eth_dev->tx_descriptor_status = NULL;
> +       eth_dev->dev_ops = NULL;
>
>         if (rte_eal_process_type() == RTE_PROC_PRIMARY) {
>                 rte_free(eth_dev->data->rx_queues);
> --
> 2.28.0
>

[dpdk-dev] [PATCH v6 3/3] ethdev: allow close function to return an error

[Thomas Monjalon]

The API function rte_eth_dev_close() was returning void.
The return type is changed to int for notifying of errors.

If an error happens during a close operation,
the status of the port is undefined,
a maximum of resources having been freed.

Signed-off-by: Thomas Monjalon <thomas@monjalon.net>
Reviewed-by: Liron Himi <lironh@marvell.com>
Acked-by: Stephen Hemminger <stephen@networkplumber.org>
Acked-by: Andrew Rybchenko <andrew.rybchenko@oktetlabs.ru>
Reviewed-by: Ferruh Yigit <ferruh.yigit@intel.com>
---
 doc/guides/rel_notes/deprecation.rst    |  1 -
 doc/guides/rel_notes/release_20_11.rst  |  4 +++-
 drivers/net/cxgbe/cxgbe_ethdev.c        |  5 +++--
 drivers/net/cxgbe/cxgbevf_ethdev.c      |  5 +++--
 drivers/net/failsafe/failsafe_ether.c   |  6 +++++-
 drivers/net/failsafe/failsafe_ops.c     | 25 +++++++++++++++++--------
 drivers/net/memif/rte_eth_memif.c       |  4 +---
 drivers/net/mlx5/mlx5.c                 |  7 ++++---
 drivers/net/mvneta/mvneta_ethdev.c      |  5 +++--
 drivers/net/mvpp2/mrvl_ethdev.c         |  5 +++--
 drivers/net/netvsc/hn_ethdev.c          |  6 ++++--
 drivers/net/netvsc/hn_var.h             |  2 +-
 drivers/net/netvsc/hn_vf.c              |  7 +++++--
 drivers/net/virtio/virtio_user_ethdev.c |  4 +---
 lib/librte_ethdev/rte_ethdev.c          | 16 +++++++++++-----
 lib/librte_ethdev/rte_ethdev.h          |  5 ++++-
 16 files changed, 68 insertions(+), 39 deletions(-)

diff --git a/doc/guides/rel_notes/deprecation.rst b/doc/guides/rel_notes/deprecation.rst
index 6cbfd11847..6e7bd14a46 100644
--- a/doc/guides/rel_notes/deprecation.rst
+++ b/doc/guides/rel_notes/deprecation.rst
@@ -128,7 +128,6 @@ Deprecation Notices
   invalid port ID, unsupported operation, failed operation):
 
   - ``rte_eth_dev_stop``
-  - ``rte_eth_dev_close``
 
 * ethdev: New offload flags ``DEV_RX_OFFLOAD_FLOW_MARK`` will be added in 19.11.
   This will allow application to enable or disable PMDs from updating
diff --git a/doc/guides/rel_notes/release_20_11.rst b/doc/guides/rel_notes/release_20_11.rst
index 48717ee536..b53cff2adc 100644
--- a/doc/guides/rel_notes/release_20_11.rst
+++ b/doc/guides/rel_notes/release_20_11.rst
@@ -369,7 +369,9 @@ API Changes
 
 * ethdev: ``rte_eth_rx_descriptor_done()`` API has been deprecated.
 
-* Renamed internal ethdev APIs:
+* ethdev: Added ``int`` return type to ``rte_eth_dev_close()``.
+
+* ethdev: Renamed internal functions:
 
   * ``_rte_eth_dev_callback_process()`` -> ``rte_eth_dev_callback_process()``
   * ``_rte_eth_dev_reset`` -> ``rte_eth_dev_internal_reset()``
diff --git a/drivers/net/cxgbe/cxgbe_ethdev.c b/drivers/net/cxgbe/cxgbe_ethdev.c
index 16beb2d435..fe488231a7 100644
--- a/drivers/net/cxgbe/cxgbe_ethdev.c
+++ b/drivers/net/cxgbe/cxgbe_ethdev.c
@@ -1296,12 +1296,13 @@ static int eth_cxgbe_dev_uninit(struct rte_eth_dev *eth_dev)
 {
 	struct rte_pci_device *pci_dev = RTE_ETH_DEV_TO_PCI(eth_dev);
 	uint16_t port_id;
+	int err = 0;
 
 	/* Free up other ports and all resources */
 	RTE_ETH_FOREACH_DEV_OF(port_id, &pci_dev->device)
-		rte_eth_dev_close(port_id);
+		err |= rte_eth_dev_close(port_id);
 
-	return 0;
+	return err == 0 ? 0 : -EIO;
 }
 
 static int eth_cxgbe_pci_probe(struct rte_pci_driver *pci_drv __rte_unused,
diff --git a/drivers/net/cxgbe/cxgbevf_ethdev.c b/drivers/net/cxgbe/cxgbevf_ethdev.c
index 947fcdd406..c2918f5356 100644
--- a/drivers/net/cxgbe/cxgbevf_ethdev.c
+++ b/drivers/net/cxgbe/cxgbevf_ethdev.c
@@ -183,12 +183,13 @@ static int eth_cxgbevf_dev_uninit(struct rte_eth_dev *eth_dev)
 {
 	struct rte_pci_device *pci_dev = RTE_ETH_DEV_TO_PCI(eth_dev);
 	uint16_t port_id;
+	int err = 0;
 
 	/* Free up other ports and all resources */
 	RTE_ETH_FOREACH_DEV_OF(port_id, &pci_dev->device)
-		rte_eth_dev_close(port_id);
+		err |= rte_eth_dev_close(port_id);
 
-	return 0;
+	return err == 0 ? 0 : -EIO;
 }
 
 static int eth_cxgbevf_pci_probe(struct rte_pci_driver *pci_drv __rte_unused,
diff --git a/drivers/net/failsafe/failsafe_ether.c b/drivers/net/failsafe/failsafe_ether.c
index 7c68bbdec0..950d35dac4 100644
--- a/drivers/net/failsafe/failsafe_ether.c
+++ b/drivers/net/failsafe/failsafe_ether.c
@@ -287,7 +287,11 @@ fs_dev_remove(struct sub_device *sdev)
 		/* fallthrough */
 	case DEV_ACTIVE:
 		failsafe_eth_dev_unregister_callbacks(sdev);
-		rte_eth_dev_close(PORT_ID(sdev));
+		ret = rte_eth_dev_close(PORT_ID(sdev));
+		if (ret < 0) {
+			ERROR("Port close failed for sub-device %u",
+			      PORT_ID(sdev));
+		}
 		sdev->state = DEV_PROBED;
 		/* fallthrough */
 	case DEV_PROBED:
diff --git a/drivers/net/failsafe/failsafe_ops.c b/drivers/net/failsafe/failsafe_ops.c
index 0ce7dfc8a6..5c606ff077 100644
--- a/drivers/net/failsafe/failsafe_ops.c
+++ b/drivers/net/failsafe/failsafe_ops.c
@@ -638,7 +638,7 @@ failsafe_eth_dev_close(struct rte_eth_dev *dev)
 {
 	struct sub_device *sdev;
 	uint8_t i;
-	int ret;
+	int err, ret = 0;
 
 	fs_lock(dev, 0);
 	failsafe_hotplug_alarm_cancel(dev);
@@ -648,29 +648,38 @@ failsafe_eth_dev_close(struct rte_eth_dev *dev)
 	FOREACH_SUBDEV_STATE(sdev, i, dev, DEV_ACTIVE) {
 		DEBUG("Closing sub_device %d", i);
 		failsafe_eth_dev_unregister_callbacks(sdev);
-		rte_eth_dev_close(PORT_ID(sdev));
+		err = rte_eth_dev_close(PORT_ID(sdev));
+		if (err) {
+			ret = ret ? ret : err;
+			ERROR("Error while closing sub-device %u",
+					PORT_ID(sdev));
+		}
 		sdev->state = DEV_ACTIVE - 1;
 	}
 	rte_eth_dev_callback_unregister(RTE_ETH_ALL, RTE_ETH_EVENT_NEW,
 					failsafe_eth_new_event_callback, dev);
 	if (rte_eal_process_type() != RTE_PROC_PRIMARY) {
 		fs_unlock(dev, 0);
-		return 0;
+		return ret;
 	}
 	fs_dev_free_queues(dev);
-	ret = failsafe_eal_uninit(dev);
-	if (ret)
+	err = failsafe_eal_uninit(dev);
+	if (err) {
+		ret = ret ? ret : err;
 		ERROR("Error while uninitializing sub-EAL");
+	}
 	failsafe_args_free(dev);
 	rte_free(PRIV(dev)->subs);
 	rte_free(PRIV(dev)->mcast_addrs);
 	/* mac_addrs must not be freed alone because part of dev_private */
 	dev->data->mac_addrs = NULL;
 	fs_unlock(dev, 0);
-	ret = pthread_mutex_destroy(&PRIV(dev)->hotplug_mutex);
-	if (ret)
+	err = pthread_mutex_destroy(&PRIV(dev)->hotplug_mutex);
+	if (err) {
+		ret = ret ? ret : err;
 		ERROR("Error while destroying hotplug mutex");
-	return 0;
+	}
+	return ret;
 }
 
 static int
diff --git a/drivers/net/memif/rte_eth_memif.c b/drivers/net/memif/rte_eth_memif.c
index ff8a58081f..33bf5c68a3 100644
--- a/drivers/net/memif/rte_eth_memif.c
+++ b/drivers/net/memif/rte_eth_memif.c
@@ -1798,9 +1798,7 @@ rte_pmd_memif_remove(struct rte_vdev_device *vdev)
 	if (eth_dev == NULL)
 		return 0;
 
-	rte_eth_dev_close(eth_dev->data->port_id);
-
-	return 0;
+	return rte_eth_dev_close(eth_dev->data->port_id);
 }
 
 static struct rte_vdev_driver pmd_memif_drv = {
diff --git a/drivers/net/mlx5/mlx5.c b/drivers/net/mlx5/mlx5.c
index e5ca392fed..dc55988b10 100644
--- a/drivers/net/mlx5/mlx5.c
+++ b/drivers/net/mlx5/mlx5.c
@@ -2020,6 +2020,7 @@ static int
 mlx5_pci_remove(struct rte_pci_device *pci_dev)
 {
 	uint16_t port_id;
+	int ret = 0;
 
 	RTE_ETH_FOREACH_DEV_OF(port_id, &pci_dev->device) {
 		/*
@@ -2027,11 +2028,11 @@ mlx5_pci_remove(struct rte_pci_device *pci_dev)
 		 * call the close function explicitly for secondary process.
 		 */
 		if (rte_eal_process_type() == RTE_PROC_SECONDARY)
-			mlx5_dev_close(&rte_eth_devices[port_id]);
+			ret |= mlx5_dev_close(&rte_eth_devices[port_id]);
 		else
-			rte_eth_dev_close(port_id);
+			ret |= rte_eth_dev_close(port_id);
 	}
-	return 0;
+	return ret == 0 ? 0 : -EIO;
 }
 
 static const struct rte_pci_id mlx5_pci_id_map[] = {
diff --git a/drivers/net/mvneta/mvneta_ethdev.c b/drivers/net/mvneta/mvneta_ethdev.c
index 3c0332ab4d..1574bf35a8 100644
--- a/drivers/net/mvneta/mvneta_ethdev.c
+++ b/drivers/net/mvneta/mvneta_ethdev.c
@@ -966,14 +966,15 @@ static int
 rte_pmd_mvneta_remove(struct rte_vdev_device *vdev)
 {
 	uint16_t port_id;
+	int ret = 0;
 
 	RTE_ETH_FOREACH_DEV(port_id) {
 		if (rte_eth_devices[port_id].device != &vdev->device)
 			continue;
-		rte_eth_dev_close(port_id);
+		ret |= rte_eth_dev_close(port_id);
 	}
 
-	return 0;
+	return ret == 0 ? 0 : -EIO;
 }
 
 static struct rte_vdev_driver pmd_mvneta_drv = {
diff --git a/drivers/net/mvpp2/mrvl_ethdev.c b/drivers/net/mvpp2/mrvl_ethdev.c
index a230a96840..df9f336fab 100644
--- a/drivers/net/mvpp2/mrvl_ethdev.c
+++ b/drivers/net/mvpp2/mrvl_ethdev.c
@@ -3022,14 +3022,15 @@ static int
 rte_pmd_mrvl_remove(struct rte_vdev_device *vdev)
 {
 	uint16_t port_id;
+	int ret = 0;
 
 	RTE_ETH_FOREACH_DEV(port_id) {
 		if (rte_eth_devices[port_id].device != &vdev->device)
 			continue;
-		rte_eth_dev_close(port_id);
+		ret |= rte_eth_dev_close(port_id);
 	}
 
-	return 0;
+	return ret == 0 ? 0 : -EIO;
 }
 
 static struct rte_vdev_driver pmd_mrvl_drv = {
diff --git a/drivers/net/netvsc/hn_ethdev.c b/drivers/net/netvsc/hn_ethdev.c
index 8743e6e69a..40e6d25f34 100644
--- a/drivers/net/netvsc/hn_ethdev.c
+++ b/drivers/net/netvsc/hn_ethdev.c
@@ -842,14 +842,16 @@ hn_dev_stop(struct rte_eth_dev *dev)
 static int
 hn_dev_close(struct rte_eth_dev *dev)
 {
+	int ret;
+
 	PMD_INIT_FUNC_TRACE();
 	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
 		return 0;
 
-	hn_vf_close(dev);
+	ret = hn_vf_close(dev);
 	hn_dev_free_queues(dev);
 
-	return 0;
+	return ret;
 }
 
 static const struct eth_dev_ops hn_eth_dev_ops = {
diff --git a/drivers/net/netvsc/hn_var.h b/drivers/net/netvsc/hn_var.h
index 4b63f87607..74f30669ac 100644
--- a/drivers/net/netvsc/hn_var.h
+++ b/drivers/net/netvsc/hn_var.h
@@ -217,7 +217,7 @@ const uint32_t *hn_vf_supported_ptypes(struct rte_eth_dev *dev);
 int	hn_vf_start(struct rte_eth_dev *dev);
 void	hn_vf_reset(struct rte_eth_dev *dev);
 void	hn_vf_stop(struct rte_eth_dev *dev);
-void	hn_vf_close(struct rte_eth_dev *dev);
+int	hn_vf_close(struct rte_eth_dev *dev);
 
 int	hn_vf_allmulticast_enable(struct rte_eth_dev *dev);
 int	hn_vf_allmulticast_disable(struct rte_eth_dev *dev);
diff --git a/drivers/net/netvsc/hn_vf.c b/drivers/net/netvsc/hn_vf.c
index f5f15c0462..d29eee7627 100644
--- a/drivers/net/netvsc/hn_vf.c
+++ b/drivers/net/netvsc/hn_vf.c
@@ -316,18 +316,21 @@ void hn_vf_reset(struct rte_eth_dev *dev)
 	VF_ETHDEV_FUNC(dev, rte_eth_dev_reset);
 }
 
-void hn_vf_close(struct rte_eth_dev *dev)
+int hn_vf_close(struct rte_eth_dev *dev)
 {
 	struct hn_data *hv = dev->data->dev_private;
 	uint16_t vf_port;
+	int ret = 0;
 
 	rte_rwlock_read_lock(&hv->vf_lock);
 	vf_port = hv->vf_port;
 	if (vf_port != HN_INVALID_PORT)
-		rte_eth_dev_close(vf_port);
+		ret = rte_eth_dev_close(vf_port);
 
 	hv->vf_port = HN_INVALID_PORT;
 	rte_rwlock_read_unlock(&hv->vf_lock);
+
+	return ret;
 }
 
 int hn_vf_stats_reset(struct rte_eth_dev *dev)
diff --git a/drivers/net/virtio/virtio_user_ethdev.c b/drivers/net/virtio/virtio_user_ethdev.c
index f56fc238c4..042665bc09 100644
--- a/drivers/net/virtio/virtio_user_ethdev.c
+++ b/drivers/net/virtio/virtio_user_ethdev.c
@@ -881,9 +881,7 @@ virtio_user_pmd_remove(struct rte_vdev_device *vdev)
 		return rte_eth_dev_release_port(eth_dev);
 
 	/* make sure the device is stopped, queues freed */
-	rte_eth_dev_close(eth_dev->data->port_id);
-
-	return 0;
+	return rte_eth_dev_close(eth_dev->data->port_id);
 }
 
 static int virtio_user_pmd_dma_map(struct rte_vdev_device *vdev, void *addr,
diff --git a/lib/librte_ethdev/rte_ethdev.c b/lib/librte_ethdev/rte_ethdev.c
index 3ae2253b9d..11ad32ca61 100644
--- a/lib/librte_ethdev/rte_ethdev.c
+++ b/lib/librte_ethdev/rte_ethdev.c
@@ -1718,19 +1718,25 @@ rte_eth_dev_set_link_down(uint16_t port_id)
 	return eth_err(port_id, (*dev->dev_ops->dev_set_link_down)(dev));
 }
 
-void
+int
 rte_eth_dev_close(uint16_t port_id)
 {
 	struct rte_eth_dev *dev;
+	int firsterr, binerr;
+	int *lasterr = &firsterr;
 
-	RTE_ETH_VALID_PORTID_OR_RET(port_id);
+	RTE_ETH_VALID_PORTID_OR_ERR_RET(port_id, -ENODEV);
 	dev = &rte_eth_devices[port_id];
 
-	RTE_FUNC_PTR_OR_RET(*dev->dev_ops->dev_close);
-	(*dev->dev_ops->dev_close)(dev);
+	RTE_FUNC_PTR_OR_ERR_RET(*dev->dev_ops->dev_close, -ENOTSUP);
+	*lasterr = (*dev->dev_ops->dev_close)(dev);
+	if (*lasterr != 0)
+		lasterr = &binerr;
 
 	rte_ethdev_trace_close(port_id);
-	rte_eth_dev_release_port(dev);
+	*lasterr = rte_eth_dev_release_port(dev);
+
+	return eth_err(port_id, firsterr);
 }
 
 int
diff --git a/lib/librte_ethdev/rte_ethdev.h b/lib/librte_ethdev/rte_ethdev.h
index 500b2ff766..849bcaf85e 100644
--- a/lib/librte_ethdev/rte_ethdev.h
+++ b/lib/librte_ethdev/rte_ethdev.h
@@ -2306,8 +2306,11 @@ int rte_eth_dev_set_link_down(uint16_t port_id);
  *
  * @param port_id
  *   The port identifier of the Ethernet device.
+ * @return
+ *   - Zero if the port is closed successfully.
+ *   - Negative if something went wrong.
  */
-void rte_eth_dev_close(uint16_t port_id);
+int rte_eth_dev_close(uint16_t port_id);
 
 /**
  * Reset a Ethernet device and keep its port id.
-- 
2.28.0

Re: [dpdk-dev] [PATCH v6 0/3] cleanup ethdev close operation

[Ferruh Yigit]

On 10/16/2020 2:32 PM, Thomas Monjalon wrote:
> Most of the patches from the v3 have been merged already.
> The remaining 3 patches are reworked in v4:
>    - reset dev_started in some PMDs
>    - selectively reset pointers in rte_eth_dev_release_port()
>    - fix error handling
> 
> v5: change error code to ENODEV if wrong port id
> v6: - use eth_err() in rte_eth_dev_close()
>      - fix missed igbvf and ixgbevf
> 
> 
> Thomas Monjalon (3):
>    ethdev: remove forcing stopped state upon close
>    ethdev: reset all when releasing a port
>    ethdev: allow close function to return an error
> 

Series applied to dpdk-next-net/main, thanks.

Re: [dpdk-dev] [PATCH v6 0/3] cleanup ethdev close operation

[Bruce Richardson]

On Fri, Oct 16, 2020 at 06:55:19PM +0100, Ferruh Yigit wrote:
> On 10/16/2020 2:32 PM, Thomas Monjalon wrote:
> > Most of the patches from the v3 have been merged already.  The
> > remaining 3 patches are reworked in v4: - reset dev_started in some
> > PMDs - selectively reset pointers in rte_eth_dev_release_port() - fix
> > error handling
> > 
> > v5: change error code to ENODEV if wrong port id v6: - use eth_err() in
> > rte_eth_dev_close() - fix missed igbvf and ixgbevf
> > 
> > 
> > Thomas Monjalon (3): ethdev: remove forcing stopped state upon close
> > ethdev: reset all when releasing a port ethdev: allow close function to
> > return an error
> > 
> 
> Series applied to dpdk-next-net/main, thanks.
>
Interesting bit of trivia: 

According to "git shortlog -sne" this patchset pushes Thomas over the 1000
patches mark in DPDK, with patch 2 of 3 being his 1000th accepted DPDK
patch. Congrats on being the first to make this milestone!

~/dpdk.org$ git shortlog -sne | head -1
  1001  Thomas Monjalon <thomas@monjalon.net>

Regards,
/Bruce

PS: for anyone checking my numbers, Thomas' commits appear under two
different emails, so you need the following line in .mailmap to merge them:
"Thomas Monjalon <thomas@monjalon.net> <thomas.monjalon@6wind.com>"

PPS: Perhaps we should look to maintain a .mailmap file in the DPDK repo,
since there are quite a number of contributors to DPDK who have used
multiple email addresses over time - including both main tree committers!