DPDK patches and discussions
 help / color / mirror / Atom feed
* [dpdk-dev] [PATCH v4 00/11] Port Hotplug Framework
       [not found] <1418106629-22227-2-git-send-email-mukawa@igel.co.j>
@ 2015-01-19 10:40 ` Tetsuya Mukawa
  2015-01-19 10:40   ` [dpdk-dev] [PATCH v4 01/11] eal/pci, ethdev: Remove assumption that port will not be detached Tetsuya Mukawa
                     ` (12 more replies)
  2015-01-19 10:41 ` [dpdk-dev] [PATCH v4] librte_pmd_pcap: Add port hotplug support Tetsuya Mukawa
  2015-01-19 10:42 ` [dpdk-dev] [PATCH v4] testpmd: " Tetsuya Mukawa
  2 siblings, 13 replies; 362+ messages in thread
From: Tetsuya Mukawa @ 2015-01-19 10:40 UTC (permalink / raw)
  To: dev

This patch series adds a dynamic port hotplug framework to DPDK.
With the patches, DPDK apps can attach or detach ports at runtime.

The basic concept of the port hotplug is like followings.
- DPDK apps must have responsibility to manage ports.
  DPDK apps only know which ports are attached or detached at the moment.
  The port hotplug framework is implemented to allow DPDK apps to manage ports.
  For example, when DPDK apps call port attach function, attached port number
  will be returned. Also DPDK apps can detach port by port number.
- Kernel support is needed for attaching or detaching physical device ports.
  To attach new device, the device will be recognized by kernel at first and
  controlled by kernel driver. Then user can bind the device to igb_uio
  by 'dpdk_nic_bind.py'. Finally, DPDK apps can call the port hotplug
  functions to attach ports.
  For detaching, steps are vice versa.
- Before detach ports, ports must be stopped and closed.
  DPDK application must call rte_eth_dev_stop() and rte_eth_dev_close() before
  detaching ports. These function will call finalization codes of PMDs.
  But so far, no PMD frees all resources allocated by initialization.
  It means PMDs are needed to be fixed to support the port hotplug.
  'RTE_PCI_DRV_DETACHABLE' is a new flag indicating a PMD supports detaching.
  Without this flag, detaching will be failed.
- Mustn't affect legacy DPDK apps.
  No DPDK EAL behavior is changed, if the port hotplug functions are't called.
  So all legacy DPDK apps can still work without modifications.

And a few limitations.
- The port hotplug functions are not thread safe.
  DPDK apps should handle it.
- Only support Linux and igb_uio so far.
  BSD and VFIO is not supported. I will send VFIO patches at least, but I don't
  have a plan to submit BSD patch so far.


Here is port hotplug APIs.
-------------------------------------------------------------------------------
/**
 * Attach a new device.
 *
 * @param devargs
 *   A pointer to a strings array describing the new device
 *   to be attached. The strings should be a pci address like
 *   '0000:01:00.0' or virtual device name like 'eth_pcap0'.
 * @param port_id
 *  A pointer to a port identifier actually attached.
 * @return
 *  0 on success and port_id is filled, negative on error
 */
int rte_eal_dev_attach(const char *devargs, uint8_t *port_id);

/**
 * Detach a device.
 *
 * @param port_id
 *   The port identifier of the device to detach.
 * @param addr
 *  A pointer to a device name actually detached.
 * @return
 *  0 on success and devname is filled, negative on error
 */
int rte_eal_dev_detach(uint8_t port_id, char *devname);
-------------------------------------------------------------------------------

This patch series are for DPDK EAL. To use port hotplug function by DPDK apps,
each PMD should be fixed to support 'RTE_PCI_DRV_DETACHABLE' flag. Please check
a patch for pcap PMD.

Also please check testpmd patch. It will show you how to fix your legacy
applications to support port hotplug feature.


PATCH v4 changes
 - Merge patches to review easier.
 - Fix indent of 'if' statement.
 - Fix calculation method of eal_compare_pci_addr().
 - Fix header file declaration.
 - Add header file to determine if hotplug can be enabled.
   (Thanks to Qiu, Michael)
 - Use braces with 'for' loop.
 - Add paramerter checking.
 - Fix sanity check code
 - Fix comments of rte_eth_dev_type.
 - Change function names.
   (Thanks to Iremonger, Bernard)

PATCH v3 changes:
 - Fix enum definition used in rte_ethdev.c.
   (Thanks to Zhang, Helin)

PATCH v2 changes:
 - Replace rte_eal_dev_attach_pdev(), rte_eal_dev_detach_pdev,
   rte_eal_dev_attach_vdev() and rte_eal_dev_detach_vdev() to
   rte_eal_dev_attach() and rte_eal_dev_detach().
 - Add parameter values checking.
 - Refashion a few functions.
   (Thanks to Iremonger, Bernard)

PATCH v1 Changes:
 - Fix error checking code of librte_eth APIs.
 - Fix issue that port from pcap PMD cannot be detached correctly.
 - Fix issue that testpmd could hang after forwarding, if attaching and detaching
   is repeatedly.
 - Fix if-condition of rte_eth_dev_get_port_by_addr().
   (Thanks to Mark Enright)

RFC PATCH v2 Changes:
- remove 'rte_eth_dev_validate_port()', and cleanup codes.


Tetsuya Mukawa (11):
  eal/pci,ethdev: Remove assumption that port will not be detached
  eal/pci: Consolidate pci address comparison APIs
  ethdev: Add rte_eth_dev_free to free specified device
  eal,ethdev: Add a function and function pointers to close ether device
  ethdev: Add functions that will be used by port hotplug functions
  eal/linux/pci: Add functions for unmapping igb_uio resources
  eal/pci: Add a function to remove the entry of devargs list
  eal/pci: Cleanup pci driver initialization code
  ethdev: Add one dev_type paramerter to rte_eth_dev_allocate
  eal/pci: Add rte_eal_dev_attach/detach() functions
  eal: Enable port hotplug framework in Linux

 app/test/virtual_pmd.c                          |   2 +-
 config/common_linuxapp                          |   5 +
 lib/librte_eal/bsdapp/eal/eal_pci.c             |  16 +-
 lib/librte_eal/common/Makefile                  |   1 +
 lib/librte_eal/common/eal_common_dev.c          | 273 +++++++++++++++
 lib/librte_eal/common/eal_common_devargs.c      |  57 ++++
 lib/librte_eal/common/eal_common_pci.c          |  92 ++++-
 lib/librte_eal/common/eal_private.h             |  36 ++
 lib/librte_eal/common/include/rte_dev.h         |  33 ++
 lib/librte_eal/common/include/rte_dev_hotplug.h |  44 +++
 lib/librte_eal/common/include/rte_devargs.h     |  18 +
 lib/librte_eal/common/include/rte_pci.h         |  76 +++++
 lib/librte_eal/linuxapp/eal/Makefile            |   1 +
 lib/librte_eal/linuxapp/eal/eal_pci.c           | 136 ++++++--
 lib/librte_eal/linuxapp/eal/eal_pci_init.h      |   8 +
 lib/librte_eal/linuxapp/eal/eal_pci_uio.c       |  67 +++-
 lib/librte_ether/rte_ethdev.c                   | 424 ++++++++++++++++++------
 lib/librte_ether/rte_ethdev.h                   | 145 +++++++-
 lib/librte_pmd_af_packet/rte_eth_af_packet.c    |   2 +-
 lib/librte_pmd_bond/rte_eth_bond_api.c          |   2 +-
 lib/librte_pmd_pcap/rte_eth_pcap.c              |   2 +-
 lib/librte_pmd_ring/rte_eth_ring.c              |   2 +-
 lib/librte_pmd_xenvirt/rte_eth_xenvirt.c        |   2 +-
 23 files changed, 1283 insertions(+), 161 deletions(-)
 create mode 100644 lib/librte_eal/common/include/rte_dev_hotplug.h

-- 
1.9.1

^ permalink raw reply	[flat|nested] 362+ messages in thread

* [dpdk-dev] [PATCH v4 01/11] eal/pci, ethdev: Remove assumption that port will not be detached
  2015-01-19 10:40 ` [dpdk-dev] [PATCH v4 00/11] Port Hotplug Framework Tetsuya Mukawa
@ 2015-01-19 10:40   ` Tetsuya Mukawa
       [not found]     ` <8CEF83825BEC744B83065625E567D7C2049D7C19@IRSMSX108.ger.corp.intel.com>
  2015-01-19 10:40   ` [dpdk-dev] [PATCH v4 02/11] eal/pci: Consolidate pci address comparison APIs Tetsuya Mukawa
                     ` (11 subsequent siblings)
  12 siblings, 1 reply; 362+ messages in thread
From: Tetsuya Mukawa @ 2015-01-19 10:40 UTC (permalink / raw)
  To: dev

To remove assumption, do like followings.

This patch adds "RTE_PCI_DRV_DETACHABLE" to drv_flags of rte_pci_driver
structure. The flags indicates the driver can detach devices at runtime.
Also remove assumption that port will not be detached.

To remove the assumption.
- Add 'attached' member to rte_eth_dev structure.
  This member is used for indicating the port is attached, or not.
- Add rte_eth_dev_allocate_new_port().
  This function is used for allocating new port.

v4:
- Use braces with 'for' loop.
- Fix indent of 'if' statement.

Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp>
---
 lib/librte_eal/common/include/rte_pci.h |   2 +
 lib/librte_ether/rte_ethdev.c           | 248 ++++++++++++++++++--------------
 lib/librte_ether/rte_ethdev.h           |   5 +
 3 files changed, 151 insertions(+), 104 deletions(-)

diff --git a/lib/librte_eal/common/include/rte_pci.h b/lib/librte_eal/common/include/rte_pci.h
index 66ed793..dd1da28 100644
--- a/lib/librte_eal/common/include/rte_pci.h
+++ b/lib/librte_eal/common/include/rte_pci.h
@@ -199,6 +199,8 @@ struct rte_pci_driver {
 #define RTE_PCI_DRV_FORCE_UNBIND 0x0004
 /** Device driver supports link state interrupt */
 #define RTE_PCI_DRV_INTR_LSC	0x0008
+/** Device driver supports detaching capability */
+#define RTE_PCI_DRV_DETACHABLE	0x0010
 
 /**< Internal use only - Macro used by pci addr parsing functions **/
 #define GET_PCIADDR_FIELD(in, fd, lim, dlm)                   \
diff --git a/lib/librte_ether/rte_ethdev.c b/lib/librte_ether/rte_ethdev.c
index 077d430..46cabaf 100644
--- a/lib/librte_ether/rte_ethdev.c
+++ b/lib/librte_ether/rte_ethdev.c
@@ -175,6 +175,16 @@ enum {
 	STAT_QMAP_RX
 };
 
+enum {
+	DEV_INVALID = 0,
+	DEV_VALID,
+};
+
+enum {
+	DEV_DISCONNECTED = 0,
+	DEV_CONNECTED
+};
+
 static inline void
 rte_eth_dev_data_alloc(void)
 {
@@ -201,19 +211,34 @@ rte_eth_dev_allocated(const char *name)
 {
 	unsigned i;
 
-	for (i = 0; i < nb_ports; i++) {
-		if (strcmp(rte_eth_devices[i].data->name, name) == 0)
+	for (i = 0; i < RTE_MAX_ETHPORTS; i++) {
+		if ((rte_eth_devices[i].attached == DEV_CONNECTED) &&
+		    strcmp(rte_eth_devices[i].data->name, name) == 0)
 			return &rte_eth_devices[i];
 	}
 	return NULL;
 }
 
+static uint8_t
+rte_eth_dev_allocate_new_port(void)
+{
+	unsigned i;
+
+	for (i = 0; i < RTE_MAX_ETHPORTS; i++) {
+		if (rte_eth_devices[i].attached == DEV_DISCONNECTED)
+			return i;
+	}
+	return RTE_MAX_ETHPORTS;
+}
+
 struct rte_eth_dev *
 rte_eth_dev_allocate(const char *name)
 {
+	uint8_t port_id;
 	struct rte_eth_dev *eth_dev;
 
-	if (nb_ports == RTE_MAX_ETHPORTS) {
+	port_id = rte_eth_dev_allocate_new_port();
+	if (port_id == RTE_MAX_ETHPORTS) {
 		PMD_DEBUG_TRACE("Reached maximum number of Ethernet ports\n");
 		return NULL;
 	}
@@ -226,10 +251,12 @@ rte_eth_dev_allocate(const char *name)
 		return NULL;
 	}
 
-	eth_dev = &rte_eth_devices[nb_ports];
-	eth_dev->data = &rte_eth_dev_data[nb_ports];
+	eth_dev = &rte_eth_devices[port_id];
+	eth_dev->data = &rte_eth_dev_data[port_id];
 	snprintf(eth_dev->data->name, sizeof(eth_dev->data->name), "%s", name);
-	eth_dev->data->port_id = nb_ports++;
+	eth_dev->data->port_id = port_id;
+	eth_dev->attached = DEV_CONNECTED;
+	nb_ports++;
 	return eth_dev;
 }
 
@@ -283,6 +310,7 @@ rte_eth_dev_init(struct rte_pci_driver *pci_drv,
 			(unsigned) pci_dev->id.device_id);
 	if (rte_eal_process_type() == RTE_PROC_PRIMARY)
 		rte_free(eth_dev->data->dev_private);
+	eth_dev->attached = DEV_DISCONNECTED;
 	nb_ports--;
 	return diag;
 }
@@ -308,10 +336,22 @@ rte_eth_driver_register(struct eth_driver *eth_drv)
 	rte_eal_pci_register(&eth_drv->pci_drv);
 }
 
+static int
+rte_eth_dev_validate_port(uint8_t port_id)
+{
+	if (port_id >= RTE_MAX_ETHPORTS)
+		return DEV_INVALID;
+
+	if (rte_eth_devices[port_id].attached == DEV_CONNECTED)
+		return DEV_VALID;
+	else
+		return DEV_INVALID;
+}
+
 int
 rte_eth_dev_socket_id(uint8_t port_id)
 {
-	if (port_id >= nb_ports)
+	if (rte_eth_dev_validate_port(port_id) == DEV_INVALID)
 		return -1;
 	return rte_eth_devices[port_id].pci_dev->numa_node;
 }
@@ -369,7 +409,7 @@ rte_eth_dev_rx_queue_start(uint8_t port_id, uint16_t rx_queue_id)
 	 * in a multi-process setup*/
 	PROC_PRIMARY_OR_ERR_RET(-E_RTE_SECONDARY);
 
-	if (port_id >= nb_ports) {
+	if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
 		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
 		return -EINVAL;
 	}
@@ -395,7 +435,7 @@ rte_eth_dev_rx_queue_stop(uint8_t port_id, uint16_t rx_queue_id)
 	 * in a multi-process setup*/
 	PROC_PRIMARY_OR_ERR_RET(-E_RTE_SECONDARY);
 
-	if (port_id >= nb_ports) {
+	if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
 		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
 		return -EINVAL;
 	}
@@ -421,7 +461,7 @@ rte_eth_dev_tx_queue_start(uint8_t port_id, uint16_t tx_queue_id)
 	 * in a multi-process setup*/
 	PROC_PRIMARY_OR_ERR_RET(-E_RTE_SECONDARY);
 
-	if (port_id >= nb_ports) {
+	if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
 		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
 		return -EINVAL;
 	}
@@ -447,7 +487,7 @@ rte_eth_dev_tx_queue_stop(uint8_t port_id, uint16_t tx_queue_id)
 	 * in a multi-process setup*/
 	PROC_PRIMARY_OR_ERR_RET(-E_RTE_SECONDARY);
 
-	if (port_id >= nb_ports) {
+	if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
 		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
 		return -EINVAL;
 	}
@@ -703,7 +743,7 @@ rte_eth_dev_configure(uint8_t port_id, uint16_t nb_rx_q, uint16_t nb_tx_q,
 	 * in a multi-process setup*/
 	PROC_PRIMARY_OR_ERR_RET(-E_RTE_SECONDARY);
 
-	if (port_id >= nb_ports || port_id >= RTE_MAX_ETHPORTS) {
+	if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
 		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
 		return (-EINVAL);
 	}
@@ -888,7 +928,7 @@ rte_eth_dev_start(uint8_t port_id)
 	 * in a multi-process setup*/
 	PROC_PRIMARY_OR_ERR_RET(-E_RTE_SECONDARY);
 
-	if (port_id >= nb_ports) {
+	if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
 		PMD_DEBUG_TRACE("Invalid port_id=%" PRIu8 "\n", port_id);
 		return (-EINVAL);
 	}
@@ -923,7 +963,7 @@ rte_eth_dev_stop(uint8_t port_id)
 	 * in a multi-process setup*/
 	PROC_PRIMARY_OR_RET();
 
-	if (port_id >= nb_ports) {
+	if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
 		PMD_DEBUG_TRACE("Invalid port_id=%" PRIu8 "\n", port_id);
 		return;
 	}
@@ -951,7 +991,7 @@ rte_eth_dev_set_link_up(uint8_t port_id)
 	 * in a multi-process setup*/
 	PROC_PRIMARY_OR_ERR_RET(-E_RTE_SECONDARY);
 
-	if (port_id >= nb_ports) {
+	if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
 		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
 		return -EINVAL;
 	}
@@ -970,7 +1010,7 @@ rte_eth_dev_set_link_down(uint8_t port_id)
 	 * in a multi-process setup*/
 	PROC_PRIMARY_OR_ERR_RET(-E_RTE_SECONDARY);
 
-	if (port_id >= nb_ports) {
+	if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
 		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
 		return -EINVAL;
 	}
@@ -989,7 +1029,7 @@ rte_eth_dev_close(uint8_t port_id)
 	 * in a multi-process setup*/
 	PROC_PRIMARY_OR_RET();
 
-	if (port_id >= nb_ports) {
+	if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
 		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
 		return;
 	}
@@ -1017,7 +1057,7 @@ rte_eth_rx_queue_setup(uint8_t port_id, uint16_t rx_queue_id,
 	 * in a multi-process setup*/
 	PROC_PRIMARY_OR_ERR_RET(-E_RTE_SECONDARY);
 
-	if (port_id >= nb_ports) {
+	if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
 		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
 		return (-EINVAL);
 	}
@@ -1090,7 +1130,7 @@ rte_eth_tx_queue_setup(uint8_t port_id, uint16_t tx_queue_id,
 	 * in a multi-process setup*/
 	PROC_PRIMARY_OR_ERR_RET(-E_RTE_SECONDARY);
 
-	if (port_id >= RTE_MAX_ETHPORTS || port_id >= nb_ports) {
+	if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
 		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
 		return (-EINVAL);
 	}
@@ -1123,7 +1163,7 @@ rte_eth_promiscuous_enable(uint8_t port_id)
 {
 	struct rte_eth_dev *dev;
 
-	if (port_id >= nb_ports) {
+	if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
 		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
 		return;
 	}
@@ -1139,7 +1179,7 @@ rte_eth_promiscuous_disable(uint8_t port_id)
 {
 	struct rte_eth_dev *dev;
 
-	if (port_id >= nb_ports) {
+	if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
 		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
 		return;
 	}
@@ -1155,7 +1195,7 @@ rte_eth_promiscuous_get(uint8_t port_id)
 {
 	struct rte_eth_dev *dev;
 
-	if (port_id >= nb_ports) {
+	if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
 		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
 		return -1;
 	}
@@ -1169,7 +1209,7 @@ rte_eth_allmulticast_enable(uint8_t port_id)
 {
 	struct rte_eth_dev *dev;
 
-	if (port_id >= nb_ports) {
+	if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
 		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
 		return;
 	}
@@ -1185,7 +1225,7 @@ rte_eth_allmulticast_disable(uint8_t port_id)
 {
 	struct rte_eth_dev *dev;
 
-	if (port_id >= nb_ports) {
+	if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
 		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
 		return;
 	}
@@ -1201,7 +1241,7 @@ rte_eth_allmulticast_get(uint8_t port_id)
 {
 	struct rte_eth_dev *dev;
 
-	if (port_id >= nb_ports) {
+	if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
 		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
 		return -1;
 	}
@@ -1229,7 +1269,7 @@ rte_eth_link_get(uint8_t port_id, struct rte_eth_link *eth_link)
 {
 	struct rte_eth_dev *dev;
 
-	if (port_id >= nb_ports) {
+	if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
 		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
 		return;
 	}
@@ -1249,7 +1289,7 @@ rte_eth_link_get_nowait(uint8_t port_id, struct rte_eth_link *eth_link)
 {
 	struct rte_eth_dev *dev;
 
-	if (port_id >= nb_ports) {
+	if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
 		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
 		return;
 	}
@@ -1269,7 +1309,7 @@ rte_eth_stats_get(uint8_t port_id, struct rte_eth_stats *stats)
 {
 	struct rte_eth_dev *dev;
 
-	if (port_id >= nb_ports) {
+	if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
 		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
 		return;
 	}
@@ -1286,7 +1326,7 @@ rte_eth_stats_reset(uint8_t port_id)
 {
 	struct rte_eth_dev *dev;
 
-	if (port_id >= nb_ports) {
+	if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
 		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
 		return;
 	}
@@ -1307,7 +1347,7 @@ rte_eth_xstats_get(uint8_t port_id, struct rte_eth_xstats *xstats,
 	uint64_t val;
 	char *stats_ptr;
 
-	if (port_id >= nb_ports) {
+	if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
 		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
 		return -1;
 	}
@@ -1376,7 +1416,7 @@ rte_eth_xstats_reset(uint8_t port_id)
 {
 	struct rte_eth_dev *dev;
 
-	if (port_id >= nb_ports) {
+	if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
 		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
 		return;
 	}
@@ -1398,7 +1438,7 @@ set_queue_stats_mapping(uint8_t port_id, uint16_t queue_id, uint8_t stat_idx,
 {
 	struct rte_eth_dev *dev;
 
-	if (port_id >= nb_ports) {
+	if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
 		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
 		return -ENODEV;
 	}
@@ -1433,7 +1473,7 @@ rte_eth_dev_info_get(uint8_t port_id, struct rte_eth_dev_info *dev_info)
 {
 	struct rte_eth_dev *dev;
 
-	if (port_id >= nb_ports) {
+	if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
 		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
 		return;
 	}
@@ -1453,7 +1493,7 @@ rte_eth_macaddr_get(uint8_t port_id, struct ether_addr *mac_addr)
 {
 	struct rte_eth_dev *dev;
 
-	if (port_id >= nb_ports) {
+	if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
 		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
 		return;
 	}
@@ -1467,7 +1507,7 @@ rte_eth_dev_get_mtu(uint8_t port_id, uint16_t *mtu)
 {
 	struct rte_eth_dev *dev;
 
-	if (port_id >= nb_ports) {
+	if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
 		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
 		return (-ENODEV);
 	}
@@ -1483,7 +1523,7 @@ rte_eth_dev_set_mtu(uint8_t port_id, uint16_t mtu)
 	int ret;
 	struct rte_eth_dev *dev;
 
-	if (port_id >= nb_ports) {
+	if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
 		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
 		return (-ENODEV);
 	}
@@ -1503,7 +1543,7 @@ rte_eth_dev_vlan_filter(uint8_t port_id, uint16_t vlan_id, int on)
 {
 	struct rte_eth_dev *dev;
 
-	if (port_id >= nb_ports) {
+	if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
 		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
 		return (-ENODEV);
 	}
@@ -1528,7 +1568,7 @@ rte_eth_dev_set_vlan_strip_on_queue(uint8_t port_id, uint16_t rx_queue_id, int o
 {
 	struct rte_eth_dev *dev;
 
-	if (port_id >= nb_ports) {
+	if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
 		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
 		return (-ENODEV);
 	}
@@ -1550,7 +1590,7 @@ rte_eth_dev_set_vlan_ether_type(uint8_t port_id, uint16_t tpid)
 {
 	struct rte_eth_dev *dev;
 
-	if (port_id >= nb_ports) {
+	if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
 		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
 		return (-ENODEV);
 	}
@@ -1570,7 +1610,7 @@ rte_eth_dev_set_vlan_offload(uint8_t port_id, int offload_mask)
 	int mask = 0;
 	int cur, org = 0;
 
-	if (port_id >= nb_ports) {
+	if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
 		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
 		return (-ENODEV);
 	}
@@ -1615,7 +1655,7 @@ rte_eth_dev_get_vlan_offload(uint8_t port_id)
 	struct rte_eth_dev *dev;
 	int ret = 0;
 
-	if (port_id >= nb_ports) {
+	if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
 		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
 		return (-ENODEV);
 	}
@@ -1639,7 +1679,7 @@ rte_eth_dev_set_vlan_pvid(uint8_t port_id, uint16_t pvid, int on)
 {
 	struct rte_eth_dev *dev;
 
-	if (port_id >= nb_ports) {
+	if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
 		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
 		return (-ENODEV);
 	}
@@ -1657,7 +1697,7 @@ rte_eth_dev_fdir_add_signature_filter(uint8_t port_id,
 {
 	struct rte_eth_dev *dev;
 
-	if (port_id >= nb_ports) {
+	if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
 		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
 		return (-ENODEV);
 	}
@@ -1691,7 +1731,7 @@ rte_eth_dev_fdir_update_signature_filter(uint8_t port_id,
 {
 	struct rte_eth_dev *dev;
 
-	if (port_id >= nb_ports) {
+	if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
 		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
 		return (-ENODEV);
 	}
@@ -1725,7 +1765,7 @@ rte_eth_dev_fdir_remove_signature_filter(uint8_t port_id,
 {
 	struct rte_eth_dev *dev;
 
-	if (port_id >= nb_ports) {
+	if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
 		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
 		return (-ENODEV);
 	}
@@ -1756,7 +1796,7 @@ rte_eth_dev_fdir_get_infos(uint8_t port_id, struct rte_eth_fdir *fdir)
 {
 	struct rte_eth_dev *dev;
 
-	if (port_id >= nb_ports) {
+	if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
 		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
 		return (-ENODEV);
 	}
@@ -1781,7 +1821,7 @@ rte_eth_dev_fdir_add_perfect_filter(uint8_t port_id,
 {
 	struct rte_eth_dev *dev;
 
-	if (port_id >= nb_ports) {
+	if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
 		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
 		return (-ENODEV);
 	}
@@ -1821,7 +1861,7 @@ rte_eth_dev_fdir_update_perfect_filter(uint8_t port_id,
 {
 	struct rte_eth_dev *dev;
 
-	if (port_id >= nb_ports) {
+	if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
 		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
 		return (-ENODEV);
 	}
@@ -1859,7 +1899,7 @@ rte_eth_dev_fdir_remove_perfect_filter(uint8_t port_id,
 {
 	struct rte_eth_dev *dev;
 
-	if (port_id >= nb_ports) {
+	if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
 		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
 		return (-ENODEV);
 	}
@@ -1895,7 +1935,7 @@ rte_eth_dev_fdir_set_masks(uint8_t port_id, struct rte_fdir_masks *fdir_mask)
 {
 	struct rte_eth_dev *dev;
 
-	if (port_id >= nb_ports) {
+	if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
 		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
 		return (-ENODEV);
 	}
@@ -1915,7 +1955,7 @@ rte_eth_dev_flow_ctrl_get(uint8_t port_id, struct rte_eth_fc_conf *fc_conf)
 {
 	struct rte_eth_dev *dev;
 
-	if (port_id >= nb_ports) {
+	if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
 		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
 		return (-ENODEV);
 	}
@@ -1931,7 +1971,7 @@ rte_eth_dev_flow_ctrl_set(uint8_t port_id, struct rte_eth_fc_conf *fc_conf)
 {
 	struct rte_eth_dev *dev;
 
-	if (port_id >= nb_ports) {
+	if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
 		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
 		return (-ENODEV);
 	}
@@ -1951,7 +1991,7 @@ rte_eth_dev_priority_flow_ctrl_set(uint8_t port_id, struct rte_eth_pfc_conf *pfc
 {
 	struct rte_eth_dev *dev;
 
-	if (port_id >= nb_ports) {
+	if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
 		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
 		return (-ENODEV);
 	}
@@ -2030,7 +2070,7 @@ rte_eth_dev_rss_reta_update(uint8_t port_id,
 	struct rte_eth_dev *dev;
 	int ret;
 
-	if (port_id >= nb_ports) {
+	if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
 		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
 		return -ENODEV;
 	}
@@ -2081,7 +2121,7 @@ rte_eth_dev_rss_hash_update(uint8_t port_id, struct rte_eth_rss_conf *rss_conf)
 	struct rte_eth_dev *dev;
 	uint16_t rss_hash_protos;
 
-	if (port_id >= nb_ports) {
+	if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
 		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
 		return (-ENODEV);
 	}
@@ -2103,7 +2143,7 @@ rte_eth_dev_rss_hash_conf_get(uint8_t port_id,
 {
 	struct rte_eth_dev *dev;
 
-	if (port_id >= nb_ports) {
+	if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
 		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
 		return (-ENODEV);
 	}
@@ -2118,7 +2158,7 @@ rte_eth_dev_udp_tunnel_add(uint8_t port_id,
 {
 	struct rte_eth_dev *dev;
 
-	if (port_id >= nb_ports) {
+	if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
 		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
 		return -ENODEV;
 	}
@@ -2144,7 +2184,7 @@ rte_eth_dev_udp_tunnel_delete(uint8_t port_id,
 {
 	struct rte_eth_dev *dev;
 
-	if (port_id >= nb_ports) {
+	if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
 		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
 		return -ENODEV;
 	}
@@ -2169,7 +2209,7 @@ rte_eth_led_on(uint8_t port_id)
 {
 	struct rte_eth_dev *dev;
 
-	if (port_id >= nb_ports) {
+	if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
 		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
 		return (-ENODEV);
 	}
@@ -2184,7 +2224,7 @@ rte_eth_led_off(uint8_t port_id)
 {
 	struct rte_eth_dev *dev;
 
-	if (port_id >= nb_ports) {
+	if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
 		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
 		return (-ENODEV);
 	}
@@ -2224,7 +2264,7 @@ rte_eth_dev_mac_addr_add(uint8_t port_id, struct ether_addr *addr,
 	int index;
 	uint64_t pool_mask;
 
-	if (port_id >= nb_ports) {
+	if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
 		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
 		return (-ENODEV);
 	}
@@ -2275,7 +2315,7 @@ rte_eth_dev_mac_addr_remove(uint8_t port_id, struct ether_addr *addr)
 	struct rte_eth_dev *dev;
 	int index;
 
-	if (port_id >= nb_ports) {
+	if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
 		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
 		return (-ENODEV);
 	}
@@ -2309,7 +2349,7 @@ rte_eth_dev_set_vf_rxmode(uint8_t port_id,  uint16_t vf,
 	struct rte_eth_dev *dev;
 	struct rte_eth_dev_info dev_info;
 
-	if (port_id >= nb_ports) {
+	if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
 		PMD_DEBUG_TRACE("set VF RX mode:Invalid port_id=%d\n",
 				port_id);
 		return (-ENODEV);
@@ -2364,7 +2404,7 @@ rte_eth_dev_uc_hash_table_set(uint8_t port_id, struct ether_addr *addr,
 	int ret;
 	struct rte_eth_dev *dev;
 
-	if (port_id >= nb_ports) {
+	if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
 		PMD_DEBUG_TRACE("unicast hash setting:Invalid port_id=%d\n",
 			port_id);
 		return (-ENODEV);
@@ -2417,7 +2457,7 @@ rte_eth_dev_uc_all_hash_table_set(uint8_t port_id, uint8_t on)
 {
 	struct rte_eth_dev *dev;
 
-	if (port_id >= nb_ports) {
+	if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
 		PMD_DEBUG_TRACE("unicast hash setting:Invalid port_id=%d\n",
 			port_id);
 		return (-ENODEV);
@@ -2436,7 +2476,7 @@ rte_eth_dev_set_vf_rx(uint8_t port_id,uint16_t vf, uint8_t on)
 	struct rte_eth_dev *dev;
 	struct rte_eth_dev_info dev_info;
 
-	if (port_id >= nb_ports) {
+	if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
 		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
 		return (-ENODEV);
 	}
@@ -2462,7 +2502,7 @@ rte_eth_dev_set_vf_tx(uint8_t port_id,uint16_t vf, uint8_t on)
 	struct rte_eth_dev *dev;
 	struct rte_eth_dev_info dev_info;
 
-	if (port_id >= nb_ports) {
+	if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
 		PMD_DEBUG_TRACE("set pool tx:Invalid port_id=%d\n", port_id);
 		return (-ENODEV);
 	}
@@ -2487,7 +2527,7 @@ rte_eth_dev_set_vf_vlan_filter(uint8_t port_id, uint16_t vlan_id,
 {
 	struct rte_eth_dev *dev;
 
-	if (port_id >= nb_ports) {
+	if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
 		PMD_DEBUG_TRACE("VF VLAN filter:invalid port id=%d\n",
 				port_id);
 		return (-ENODEV);
@@ -2518,7 +2558,7 @@ int rte_eth_set_queue_rate_limit(uint8_t port_id, uint16_t queue_idx,
 	struct rte_eth_dev_info dev_info;
 	struct rte_eth_link link;
 
-	if (port_id >= nb_ports) {
+	if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
 		PMD_DEBUG_TRACE("set queue rate limit:invalid port id=%d\n",
 				port_id);
 		return -ENODEV;
@@ -2555,7 +2595,7 @@ int rte_eth_set_vf_rate_limit(uint8_t port_id, uint16_t vf, uint16_t tx_rate,
 	if (q_msk == 0)
 		return 0;
 
-	if (port_id >= nb_ports) {
+	if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
 		PMD_DEBUG_TRACE("set VF rate limit:invalid port id=%d\n",
 				port_id);
 		return -ENODEV;
@@ -2589,7 +2629,7 @@ rte_eth_mirror_rule_set(uint8_t port_id,
 {
 	struct rte_eth_dev *dev = &rte_eth_devices[port_id];
 
-	if (port_id >= nb_ports) {
+	if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
 		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
 		return (-ENODEV);
 	}
@@ -2630,7 +2670,7 @@ rte_eth_mirror_rule_reset(uint8_t port_id, uint8_t rule_id)
 {
 	struct rte_eth_dev *dev = &rte_eth_devices[port_id];
 
-	if (port_id >= nb_ports) {
+	if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
 		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
 		return (-ENODEV);
 	}
@@ -2655,7 +2695,7 @@ rte_eth_rx_burst(uint8_t port_id, uint16_t queue_id,
 {
 	struct rte_eth_dev *dev;
 
-	if (port_id >= nb_ports) {
+	if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
 		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
 		return 0;
 	}
@@ -2675,7 +2715,7 @@ rte_eth_tx_burst(uint8_t port_id, uint16_t queue_id,
 {
 	struct rte_eth_dev *dev;
 
-	if (port_id >= nb_ports) {
+	if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
 		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
 		return 0;
 	}
@@ -2695,7 +2735,7 @@ rte_eth_rx_queue_count(uint8_t port_id, uint16_t queue_id)
 {
 	struct rte_eth_dev *dev;
 
-	if (port_id >= nb_ports) {
+	if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
 		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
 		return 0;
 	}
@@ -2709,7 +2749,7 @@ rte_eth_rx_descriptor_done(uint8_t port_id, uint16_t queue_id, uint16_t offset)
 {
 	struct rte_eth_dev *dev;
 
-	if (port_id >= nb_ports) {
+	if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
 		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
 		return (-ENODEV);
 	}
@@ -2730,7 +2770,7 @@ rte_eth_dev_callback_register(uint8_t port_id,
 
 	if (!cb_fn)
 		return (-EINVAL);
-	if (port_id >= nb_ports) {
+	if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
 		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
 		return (-EINVAL);
 	}
@@ -2770,7 +2810,7 @@ rte_eth_dev_callback_unregister(uint8_t port_id,
 
 	if (!cb_fn)
 		return (-EINVAL);
-	if (port_id >= nb_ports) {
+	if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
 		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
 		return (-EINVAL);
 	}
@@ -2830,7 +2870,7 @@ int rte_eth_dev_bypass_init(uint8_t port_id)
 {
 	struct rte_eth_dev *dev;
 
-	if (port_id >= nb_ports) {
+	if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
 		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
 		return (-ENODEV);
 	}
@@ -2850,7 +2890,7 @@ rte_eth_dev_bypass_state_show(uint8_t port_id, uint32_t *state)
 {
 	struct rte_eth_dev *dev;
 
-	if (port_id >= nb_ports) {
+	if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
 		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
 		return (-ENODEV);
 	}
@@ -2869,7 +2909,7 @@ rte_eth_dev_bypass_state_set(uint8_t port_id, uint32_t *new_state)
 {
 	struct rte_eth_dev *dev;
 
-	if (port_id >= nb_ports) {
+	if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
 		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
 		return (-ENODEV);
 	}
@@ -2889,7 +2929,7 @@ rte_eth_dev_bypass_event_show(uint8_t port_id, uint32_t event, uint32_t *state)
 {
 	struct rte_eth_dev *dev;
 
-	if (port_id >= nb_ports) {
+	if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
 		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
 		return (-ENODEV);
 	}
@@ -2909,7 +2949,7 @@ rte_eth_dev_bypass_event_store(uint8_t port_id, uint32_t event, uint32_t state)
 {
 	struct rte_eth_dev *dev;
 
-	if (port_id >= nb_ports) {
+	if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
 		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
 		return (-ENODEV);
 	}
@@ -2929,7 +2969,7 @@ rte_eth_dev_wd_timeout_store(uint8_t port_id, uint32_t timeout)
 {
 	struct rte_eth_dev *dev;
 
-	if (port_id >= nb_ports) {
+	if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
 		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
 		return (-ENODEV);
 	}
@@ -2949,7 +2989,7 @@ rte_eth_dev_bypass_ver_show(uint8_t port_id, uint32_t *ver)
 {
 	struct rte_eth_dev *dev;
 
-	if (port_id >= nb_ports) {
+	if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
 		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
 		return (-ENODEV);
 	}
@@ -2969,7 +3009,7 @@ rte_eth_dev_bypass_wd_timeout_show(uint8_t port_id, uint32_t *wd_timeout)
 {
 	struct rte_eth_dev *dev;
 
-	if (port_id >= nb_ports) {
+	if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
 		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
 		return (-ENODEV);
 	}
@@ -2989,7 +3029,7 @@ rte_eth_dev_bypass_wd_reset(uint8_t port_id)
 {
 	struct rte_eth_dev *dev;
 
-	if (port_id >= nb_ports) {
+	if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
 		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
 		return (-ENODEV);
 	}
@@ -3011,7 +3051,7 @@ rte_eth_dev_add_syn_filter(uint8_t port_id,
 {
 	struct rte_eth_dev *dev;
 
-	if (port_id >= nb_ports) {
+	if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
 		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
 		return -ENODEV;
 	}
@@ -3026,7 +3066,7 @@ rte_eth_dev_remove_syn_filter(uint8_t port_id)
 {
 	struct rte_eth_dev *dev;
 
-	if (port_id >= nb_ports) {
+	if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
 		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
 		return -ENODEV;
 	}
@@ -3045,7 +3085,7 @@ rte_eth_dev_get_syn_filter(uint8_t port_id,
 	if (filter == NULL || rx_queue == NULL)
 		return -EINVAL;
 
-	if (port_id >= nb_ports) {
+	if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
 		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
 		return -ENODEV;
 	}
@@ -3061,7 +3101,7 @@ rte_eth_dev_add_ethertype_filter(uint8_t port_id, uint16_t index,
 {
 	struct rte_eth_dev *dev;
 
-	if (port_id >= nb_ports) {
+	if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
 		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
 		return -ENODEV;
 	}
@@ -3082,7 +3122,7 @@ rte_eth_dev_remove_ethertype_filter(uint8_t port_id,  uint16_t index)
 {
 	struct rte_eth_dev *dev;
 
-	if (port_id >= nb_ports) {
+	if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
 		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
 		return -ENODEV;
 	}
@@ -3101,7 +3141,7 @@ rte_eth_dev_get_ethertype_filter(uint8_t port_id, uint16_t index,
 	if (filter == NULL || rx_queue == NULL)
 		return -EINVAL;
 
-	if (port_id >= nb_ports) {
+	if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
 		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
 		return -ENODEV;
 	}
@@ -3118,7 +3158,7 @@ rte_eth_dev_add_2tuple_filter(uint8_t port_id, uint16_t index,
 {
 	struct rte_eth_dev *dev;
 
-	if (port_id >= nb_ports) {
+	if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
 		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
 		return -ENODEV;
 	}
@@ -3140,7 +3180,7 @@ rte_eth_dev_remove_2tuple_filter(uint8_t port_id, uint16_t index)
 {
 	struct rte_eth_dev *dev;
 
-	if (port_id >= nb_ports) {
+	if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
 		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
 		return -ENODEV;
 	}
@@ -3159,7 +3199,7 @@ rte_eth_dev_get_2tuple_filter(uint8_t port_id, uint16_t index,
 	if (filter == NULL || rx_queue == NULL)
 		return -EINVAL;
 
-	if (port_id >= nb_ports) {
+	if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
 		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
 		return -ENODEV;
 	}
@@ -3175,7 +3215,7 @@ rte_eth_dev_add_5tuple_filter(uint8_t port_id, uint16_t index,
 {
 	struct rte_eth_dev *dev;
 
-	if (port_id >= nb_ports) {
+	if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
 		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
 		return -ENODEV;
 	}
@@ -3198,7 +3238,7 @@ rte_eth_dev_remove_5tuple_filter(uint8_t port_id, uint16_t index)
 {
 	struct rte_eth_dev *dev;
 
-	if (port_id >= nb_ports) {
+	if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
 		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
 		return -ENODEV;
 	}
@@ -3217,7 +3257,7 @@ rte_eth_dev_get_5tuple_filter(uint8_t port_id, uint16_t index,
 	if (filter == NULL || rx_queue == NULL)
 		return -EINVAL;
 
-	if (port_id >= nb_ports) {
+	if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
 		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
 		return -ENODEV;
 	}
@@ -3234,7 +3274,7 @@ rte_eth_dev_add_flex_filter(uint8_t port_id, uint16_t index,
 {
 	struct rte_eth_dev *dev;
 
-	if (port_id >= nb_ports) {
+	if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
 		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
 		return -ENODEV;
 	}
@@ -3249,7 +3289,7 @@ rte_eth_dev_remove_flex_filter(uint8_t port_id, uint16_t index)
 {
 	struct rte_eth_dev *dev;
 
-	if (port_id >= nb_ports) {
+	if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
 		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
 		return -ENODEV;
 	}
@@ -3268,7 +3308,7 @@ rte_eth_dev_get_flex_filter(uint8_t port_id, uint16_t index,
 	if (filter == NULL || rx_queue == NULL)
 		return -EINVAL;
 
-	if (port_id >= nb_ports) {
+	if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
 		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
 		return -ENODEV;
 	}
@@ -3284,7 +3324,7 @@ rte_eth_dev_filter_supported(uint8_t port_id, enum rte_filter_type filter_type)
 {
 	struct rte_eth_dev *dev;
 
-	if (port_id >= nb_ports) {
+	if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
 		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
 		return -ENODEV;
 	}
@@ -3301,7 +3341,7 @@ rte_eth_dev_filter_ctrl(uint8_t port_id, enum rte_filter_type filter_type,
 {
 	struct rte_eth_dev *dev;
 
-	if (port_id >= nb_ports) {
+	if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
 		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
 		return -ENODEV;
 	}
diff --git a/lib/librte_ether/rte_ethdev.h b/lib/librte_ether/rte_ethdev.h
index ce0528f..616a44a 100644
--- a/lib/librte_ether/rte_ethdev.h
+++ b/lib/librte_ether/rte_ethdev.h
@@ -1565,6 +1565,7 @@ struct rte_eth_dev {
 	struct eth_dev_ops *dev_ops;    /**< Functions exported by PMD */
 	struct rte_pci_device *pci_dev; /**< PCI info. supplied by probing */
 	struct rte_eth_dev_cb_list callbacks; /**< User application callbacks */
+	uint8_t attached; /**< Flag indicating the port is attached */
 };
 
 struct rte_eth_dev_sriov {
@@ -1630,6 +1631,10 @@ extern struct rte_eth_dev rte_eth_devices[];
  * initialized by the [matching] Ethernet driver during the PCI probing phase.
  * All devices whose port identifier is in the range
  * [0,  rte_eth_dev_count() - 1] can be operated on by network applications.
+ * immediately after invoking rte_eal_init().
+ * If the application unplugs a port using hotplug function, The enabled port
+ * numbers may be noncontiguous. In the case, the applications need to manage
+ * enabled port by themselves.
  *
  * @return
  *   - The total number of usable Ethernet devices.
-- 
1.9.1

^ permalink raw reply	[flat|nested] 362+ messages in thread

* [dpdk-dev] [PATCH v4 02/11] eal/pci: Consolidate pci address comparison APIs
  2015-01-19 10:40 ` [dpdk-dev] [PATCH v4 00/11] Port Hotplug Framework Tetsuya Mukawa
  2015-01-19 10:40   ` [dpdk-dev] [PATCH v4 01/11] eal/pci, ethdev: Remove assumption that port will not be detached Tetsuya Mukawa
@ 2015-01-19 10:40   ` Tetsuya Mukawa
  2015-01-19 10:40   ` [dpdk-dev] [PATCH v4 03/11] ethdev: Add rte_eth_dev_free to free specified device Tetsuya Mukawa
                     ` (10 subsequent siblings)
  12 siblings, 0 replies; 362+ messages in thread
From: Tetsuya Mukawa @ 2015-01-19 10:40 UTC (permalink / raw)
  To: dev

This patch replaces pci_addr_comparison() and memcmp() of pci addresses by
eal_compare_pci_addr().

v4:
- Fix calculation method of eal_compare_pci_addr().
- Add parameter checking.

Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp>
---
 lib/librte_eal/bsdapp/eal/eal_pci.c       | 16 +--------------
 lib/librte_eal/common/eal_common_pci.c    |  2 +-
 lib/librte_eal/common/include/rte_pci.h   | 34 +++++++++++++++++++++++++++++++
 lib/librte_eal/linuxapp/eal/eal_pci.c     | 23 ++++++---------------
 lib/librte_eal/linuxapp/eal/eal_pci_uio.c |  2 +-
 5 files changed, 43 insertions(+), 34 deletions(-)

diff --git a/lib/librte_eal/bsdapp/eal/eal_pci.c b/lib/librte_eal/bsdapp/eal/eal_pci.c
index 74ecce7..7eda513 100644
--- a/lib/librte_eal/bsdapp/eal/eal_pci.c
+++ b/lib/librte_eal/bsdapp/eal/eal_pci.c
@@ -270,20 +270,6 @@ pci_uio_map_resource(struct rte_pci_device *dev)
 	return (0);
 }
 
-/* Compare two PCI device addresses. */
-static int
-pci_addr_comparison(struct rte_pci_addr *addr, struct rte_pci_addr *addr2)
-{
-	uint64_t dev_addr = (addr->domain << 24) + (addr->bus << 16) + (addr->devid << 8) + addr->function;
-	uint64_t dev_addr2 = (addr2->domain << 24) + (addr2->bus << 16) + (addr2->devid << 8) + addr2->function;
-
-	if (dev_addr > dev_addr2)
-		return 1;
-	else
-		return 0;
-}
-
-
 /* Scan one pci sysfs entry, and fill the devices list from it. */
 static int
 pci_scan_one(int dev_pci_fd, struct pci_conf *conf)
@@ -358,7 +344,7 @@ pci_scan_one(int dev_pci_fd, struct pci_conf *conf)
 		struct rte_pci_device *dev2 = NULL;
 
 		TAILQ_FOREACH(dev2, &pci_device_list, next) {
-			if (pci_addr_comparison(&dev->addr, &dev2->addr))
+			if (eal_compare_pci_addr(&dev->addr, &dev2->addr))
 				continue;
 			else {
 				TAILQ_INSERT_BEFORE(dev2, dev, next);
diff --git a/lib/librte_eal/common/eal_common_pci.c b/lib/librte_eal/common/eal_common_pci.c
index f3c7f71..a89f5c3 100644
--- a/lib/librte_eal/common/eal_common_pci.c
+++ b/lib/librte_eal/common/eal_common_pci.c
@@ -93,7 +93,7 @@ static struct rte_devargs *pci_devargs_lookup(struct rte_pci_device *dev)
 		if (devargs->type != RTE_DEVTYPE_BLACKLISTED_PCI &&
 			devargs->type != RTE_DEVTYPE_WHITELISTED_PCI)
 			continue;
-		if (!memcmp(&dev->addr, &devargs->pci.addr, sizeof(dev->addr)))
+		if (!eal_compare_pci_addr(&dev->addr, &devargs->pci.addr))
 			return devargs;
 	}
 	return NULL;
diff --git a/lib/librte_eal/common/include/rte_pci.h b/lib/librte_eal/common/include/rte_pci.h
index dd1da28..128b2ce 100644
--- a/lib/librte_eal/common/include/rte_pci.h
+++ b/lib/librte_eal/common/include/rte_pci.h
@@ -261,6 +261,40 @@ eal_parse_pci_DomBDF(const char *input, struct rte_pci_addr *dev_addr)
 }
 #undef GET_PCIADDR_FIELD
 
+/* Compare two PCI device addresses. */
+/**
+ * Utility function to compare two PCI device addresses.
+ *
+ * @param addr
+ *	The PCI Bus-Device-Function address to compare
+ * @param addr2
+ *	The PCI Bus-Device-Function address to compare
+ * @return
+ *	0 on equal PCI address.
+ *	Positive on addr is greater than addr2.
+ *	Negative on addr is less than addr2, or error.
+ */
+static inline int
+eal_compare_pci_addr(struct rte_pci_addr *addr, struct rte_pci_addr *addr2)
+{
+	uint64_t dev_addr, dev_addr2;
+
+	if ((addr == NULL) || (addr2 == NULL))
+		return -1;
+
+	dev_addr = (addr->domain << 24) | (addr->bus << 16) |
+				(addr->devid << 8) | addr->function;
+	dev_addr2 = (addr2->domain << 24) | (addr2->bus << 16) |
+				(addr2->devid << 8) | addr2->function;
+
+	if (dev_addr > dev_addr2)
+		return 1;
+	else if (dev_addr < dev_addr2)
+		return -1;
+	else
+		return 0;
+}
+
 /**
  * Probe the PCI bus for registered drivers.
  *
diff --git a/lib/librte_eal/linuxapp/eal/eal_pci.c b/lib/librte_eal/linuxapp/eal/eal_pci.c
index b5f5410..3d2d93c 100644
--- a/lib/librte_eal/linuxapp/eal/eal_pci.c
+++ b/lib/librte_eal/linuxapp/eal/eal_pci.c
@@ -200,20 +200,6 @@ error:
 	return -1;
 }
 
-/* Compare two PCI device addresses. */
-static int
-pci_addr_comparison(struct rte_pci_addr *addr, struct rte_pci_addr *addr2)
-{
-	uint64_t dev_addr = (addr->domain << 24) + (addr->bus << 16) + (addr->devid << 8) + addr->function;
-	uint64_t dev_addr2 = (addr2->domain << 24) + (addr2->bus << 16) + (addr2->devid << 8) + addr2->function;
-
-	if (dev_addr > dev_addr2)
-		return 1;
-	else
-		return 0;
-}
-
-
 /* Scan one pci sysfs entry, and fill the devices list from it. */
 static int
 pci_scan_one(const char *dirname, uint16_t domain, uint8_t bus,
@@ -304,14 +290,17 @@ pci_scan_one(const char *dirname, uint16_t domain, uint8_t bus,
 	}
 	else {
 		struct rte_pci_device *dev2 = NULL;
+		int ret;
 
 		TAILQ_FOREACH(dev2, &pci_device_list, next) {
-			if (pci_addr_comparison(&dev->addr, &dev2->addr))
+			ret = eal_compare_pci_addr(&dev->addr, &dev2->addr);
+			if (ret > 0)
 				continue;
-			else {
+			else if (ret < 0) {
 				TAILQ_INSERT_BEFORE(dev2, dev, next);
 				return 0;
-			}
+			} else	/* already registered */
+				return 0;
 		}
 		TAILQ_INSERT_TAIL(&pci_device_list, dev, next);
 	}
diff --git a/lib/librte_eal/linuxapp/eal/eal_pci_uio.c b/lib/librte_eal/linuxapp/eal/eal_pci_uio.c
index e53f06b..1da3507 100644
--- a/lib/librte_eal/linuxapp/eal/eal_pci_uio.c
+++ b/lib/librte_eal/linuxapp/eal/eal_pci_uio.c
@@ -123,7 +123,7 @@ pci_uio_map_secondary(struct rte_pci_device *dev)
 	TAILQ_FOREACH(uio_res, pci_res_list, next) {
 
 		/* skip this element if it doesn't match our PCI address */
-		if (memcmp(&uio_res->pci_addr, &dev->addr, sizeof(dev->addr)))
+		if (eal_compare_pci_addr(&uio_res->pci_addr, &dev->addr))
 			continue;
 
 		for (i = 0; i != uio_res->nb_maps; i++) {
-- 
1.9.1

^ permalink raw reply	[flat|nested] 362+ messages in thread

* [dpdk-dev] [PATCH v4 03/11] ethdev: Add rte_eth_dev_free to free specified device
  2015-01-19 10:40 ` [dpdk-dev] [PATCH v4 00/11] Port Hotplug Framework Tetsuya Mukawa
  2015-01-19 10:40   ` [dpdk-dev] [PATCH v4 01/11] eal/pci, ethdev: Remove assumption that port will not be detached Tetsuya Mukawa
  2015-01-19 10:40   ` [dpdk-dev] [PATCH v4 02/11] eal/pci: Consolidate pci address comparison APIs Tetsuya Mukawa
@ 2015-01-19 10:40   ` Tetsuya Mukawa
  2015-01-19 10:40   ` [dpdk-dev] [PATCH v4 04/11] eal, ethdev: Add a function and function pointers to close ether device Tetsuya Mukawa
                     ` (9 subsequent siblings)
  12 siblings, 0 replies; 362+ messages in thread
From: Tetsuya Mukawa @ 2015-01-19 10:40 UTC (permalink / raw)
  To: dev

This patch adds rte_eth_dev_free(). The function is used for changing a
attached status of the device that has specified name.

v4:
- Add paramerter checking.

Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp>
---
 lib/librte_ether/rte_ethdev.c | 20 ++++++++++++++++++++
 lib/librte_ether/rte_ethdev.h | 11 +++++++++++
 2 files changed, 31 insertions(+)

diff --git a/lib/librte_ether/rte_ethdev.c b/lib/librte_ether/rte_ethdev.c
index 46cabaf..16f22c0 100644
--- a/lib/librte_ether/rte_ethdev.c
+++ b/lib/librte_ether/rte_ethdev.c
@@ -260,6 +260,26 @@ rte_eth_dev_allocate(const char *name)
 	return eth_dev;
 }
 
+struct rte_eth_dev *
+rte_eth_dev_free(const char *name)
+{
+	struct rte_eth_dev *eth_dev;
+
+	if (name == NULL)
+		return NULL;
+
+	eth_dev = rte_eth_dev_allocated(name);
+	if (eth_dev == NULL) {
+		PMD_DEBUG_TRACE("Ethernet Device with name %s doesn't exist!\n",
+				name);
+		return NULL;
+	}
+
+	eth_dev->attached = 0;
+	nb_ports--;
+	return eth_dev;
+}
+
 static int
 rte_eth_dev_init(struct rte_pci_driver *pci_drv,
 		 struct rte_pci_device *pci_dev)
diff --git a/lib/librte_ether/rte_ethdev.h b/lib/librte_ether/rte_ethdev.h
index 616a44a..fe64c78 100644
--- a/lib/librte_ether/rte_ethdev.h
+++ b/lib/librte_ether/rte_ethdev.h
@@ -1653,6 +1653,17 @@ extern uint8_t rte_eth_dev_count(void);
  */
 struct rte_eth_dev *rte_eth_dev_allocate(const char *name);
 
+/**
+ * Function for internal use by dummy drivers primarily, e.g. ring-based
+ * driver.
+ * Free the specified ethdev and returns the pointer to that slot.
+ *
+ * @param	name	Unique identifier name for each Ethernet device
+ * @return
+ *   - Slot in the rte_dev_devices array for the freed device;
+ */
+struct rte_eth_dev *rte_eth_dev_free(const char *name);
+
 struct eth_driver;
 /**
  * @internal
-- 
1.9.1

^ permalink raw reply	[flat|nested] 362+ messages in thread

* [dpdk-dev] [PATCH v4 04/11] eal, ethdev: Add a function and function pointers to close ether device
  2015-01-19 10:40 ` [dpdk-dev] [PATCH v4 00/11] Port Hotplug Framework Tetsuya Mukawa
                     ` (2 preceding siblings ...)
  2015-01-19 10:40   ` [dpdk-dev] [PATCH v4 03/11] ethdev: Add rte_eth_dev_free to free specified device Tetsuya Mukawa
@ 2015-01-19 10:40   ` Tetsuya Mukawa
  2015-01-19 10:40   ` [dpdk-dev] [PATCH v4 05/11] ethdev: Add functions that will be used by port hotplug functions Tetsuya Mukawa
                     ` (8 subsequent siblings)
  12 siblings, 0 replies; 362+ messages in thread
From: Tetsuya Mukawa @ 2015-01-19 10:40 UTC (permalink / raw)
  To: dev

The patch adds function pointer to rte_pci_driver and eth_driver
structure. These function pointers are used when ports are detached.
Also the patch adds rte_eth_dev_uninit(). So far, it's not called
by anywhere, but it will be called when port hotplug function is
implemented.

v4:
- Add paramerter checking.
- Change function names.

Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp>
---
 lib/librte_eal/common/include/rte_pci.h |  7 ++++++
 lib/librte_ether/rte_ethdev.c           | 40 +++++++++++++++++++++++++++++++++
 lib/librte_ether/rte_ethdev.h           | 24 ++++++++++++++++++++
 3 files changed, 71 insertions(+)

diff --git a/lib/librte_eal/common/include/rte_pci.h b/lib/librte_eal/common/include/rte_pci.h
index 128b2ce..fe57cb6 100644
--- a/lib/librte_eal/common/include/rte_pci.h
+++ b/lib/librte_eal/common/include/rte_pci.h
@@ -181,12 +181,19 @@ struct rte_pci_driver;
 typedef int (pci_devinit_t)(struct rte_pci_driver *, struct rte_pci_device *);
 
 /**
+ * Uninitialisation function for the driver called during hotplugging.
+ */
+typedef int (pci_devuninit_t)(
+		struct rte_pci_driver *, struct rte_pci_device *);
+
+/**
  * A structure describing a PCI driver.
  */
 struct rte_pci_driver {
 	TAILQ_ENTRY(rte_pci_driver) next;       /**< Next in list. */
 	const char *name;                       /**< Driver name. */
 	pci_devinit_t *devinit;                 /**< Device init. function. */
+	pci_devuninit_t *devuninit;             /**< Device uninit function. */
 	struct rte_pci_id *id_table;            /**< ID table, NULL terminated. */
 	uint32_t drv_flags;                     /**< Flags contolling handling of device. */
 };
diff --git a/lib/librte_ether/rte_ethdev.c b/lib/librte_ether/rte_ethdev.c
index 16f22c0..e5145b7 100644
--- a/lib/librte_ether/rte_ethdev.c
+++ b/lib/librte_ether/rte_ethdev.c
@@ -335,6 +335,45 @@ rte_eth_dev_init(struct rte_pci_driver *pci_drv,
 	return diag;
 }
 
+static int
+rte_eth_dev_uninit(struct rte_pci_driver *pci_drv,
+		 struct rte_pci_device *pci_dev)
+{
+	struct eth_driver *eth_drv;
+	struct rte_eth_dev *eth_dev;
+	char ethdev_name[RTE_ETH_NAME_MAX_LEN];
+
+	if ((pci_drv == NULL) || (pci_dev == NULL))
+		return -EINVAL;
+
+	/* Create unique Ethernet device name using PCI address */
+	snprintf(ethdev_name, RTE_ETH_NAME_MAX_LEN, "%d:%d.%d",
+			pci_dev->addr.bus, pci_dev->addr.devid,
+			pci_dev->addr.function);
+
+	eth_dev = rte_eth_dev_free(ethdev_name);
+	if (eth_dev == NULL)
+		return -ENODEV;
+
+	eth_drv = (struct eth_driver *)pci_drv;
+
+	/* Invoke PMD device uninit function */
+	if (*eth_drv->eth_dev_uninit)
+		(*eth_drv->eth_dev_uninit)(eth_drv, eth_dev);
+
+	/* init user callbacks */
+	TAILQ_INIT(&(eth_dev->callbacks));
+
+	if (rte_eal_process_type() == RTE_PROC_PRIMARY)
+		rte_free(eth_dev->data->dev_private);
+
+	eth_dev->pci_dev = NULL;
+	eth_dev->driver = NULL;
+	eth_dev->data = NULL;
+
+	return 0;
+}
+
 /**
  * Register an Ethernet [Poll Mode] driver.
  *
@@ -353,6 +392,7 @@ void
 rte_eth_driver_register(struct eth_driver *eth_drv)
 {
 	eth_drv->pci_drv.devinit = rte_eth_dev_init;
+	eth_drv->pci_drv.devuninit = rte_eth_dev_uninit;
 	rte_eal_pci_register(&eth_drv->pci_drv);
 }
 
diff --git a/lib/librte_ether/rte_ethdev.h b/lib/librte_ether/rte_ethdev.h
index fe64c78..2a8ff26 100644
--- a/lib/librte_ether/rte_ethdev.h
+++ b/lib/librte_ether/rte_ethdev.h
@@ -1701,6 +1701,27 @@ typedef int (*eth_dev_init_t)(struct eth_driver  *eth_drv,
 
 /**
  * @internal
+ * Finalization function of an Ethernet driver invoked for each matching
+ * Ethernet PCI device detected during the PCI closing phase.
+ *
+ * @param eth_drv
+ *   The pointer to the [matching] Ethernet driver structure supplied by
+ *   the PMD when it registered itself.
+ * @param eth_dev
+ *   The *eth_dev* pointer is the address of the *rte_eth_dev* structure
+ *   associated with the matching device and which have been [automatically]
+ *   allocated in the *rte_eth_devices* array.
+ * @return
+ *   - 0: Success, the device is properly finalized by the driver.
+ *        In particular, the driver MUST free the *dev_ops* pointer
+ *        of the *eth_dev* structure.
+ *   - <0: Error code of the device initialization failure.
+ */
+typedef int (*eth_dev_uninit_t)(struct eth_driver  *eth_drv,
+				  struct rte_eth_dev *eth_dev);
+
+/**
+ * @internal
  * The structure associated with a PMD Ethernet driver.
  *
  * Each Ethernet driver acts as a PCI driver and is represented by a generic
@@ -1710,11 +1731,14 @@ typedef int (*eth_dev_init_t)(struct eth_driver  *eth_drv,
  *
  * - The *eth_dev_init* function invoked for each matching PCI device.
  *
+ * - The *eth_dev_uninit* function invoked for each matching PCI device.
+ *
  * - The size of the private data to allocate for each matching device.
  */
 struct eth_driver {
 	struct rte_pci_driver pci_drv;    /**< The PMD is also a PCI driver. */
 	eth_dev_init_t eth_dev_init;      /**< Device init function. */
+	eth_dev_uninit_t eth_dev_uninit;  /**< Device uninit function. */
 	unsigned int dev_private_size;    /**< Size of device private data. */
 };
 
-- 
1.9.1

^ permalink raw reply	[flat|nested] 362+ messages in thread

* [dpdk-dev] [PATCH v4 05/11] ethdev: Add functions that will be used by port hotplug functions
  2015-01-19 10:40 ` [dpdk-dev] [PATCH v4 00/11] Port Hotplug Framework Tetsuya Mukawa
                     ` (3 preceding siblings ...)
  2015-01-19 10:40   ` [dpdk-dev] [PATCH v4 04/11] eal, ethdev: Add a function and function pointers to close ether device Tetsuya Mukawa
@ 2015-01-19 10:40   ` Tetsuya Mukawa
  2015-01-21  2:40     ` Qiu, Michael
  2015-01-19 10:40   ` [dpdk-dev] [PATCH v4 06/11] eal/linux/pci: Add functions for unmapping igb_uio resources Tetsuya Mukawa
                     ` (7 subsequent siblings)
  12 siblings, 1 reply; 362+ messages in thread
From: Tetsuya Mukawa @ 2015-01-19 10:40 UTC (permalink / raw)
  To: dev

The patch adds following functions.

- rte_eth_dev_save()
  The function is used for saving current rte_eth_dev structures.
- rte_eth_dev_get_changed_port()
  The function receives the rte_eth_dev structures, then compare
  these with current values to know which port is actually
  attached or detached.
- rte_eth_dev_get_addr_by_port()
  The function returns a pci address of a ethdev specified by port
  identifier.
- rte_eth_dev_get_port_by_addr()
  The function returns a port identifier of a ethdev specified by
  pci address.
- rte_eth_dev_get_name_by_port()
  The function returns a unique identifier name of a ethdev
  specified by port identifier.
- Add rte_eth_dev_check_detachable()
  The function returns whether a PMD supports detach function.

Also the patch changes scope of rte_eth_dev_allocated() to global.
This function will be called by virtual PMDs to support port hotplug.
So change scope of the function to global.

v4:
- Add paramerter checking.
v3:
- Fix if-condition bug while comparing pci addresses.
- Add error checking codes.
Reported-by: Mark Enright <menrigh@brocade.com>

Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp>
---
 lib/librte_ether/rte_ethdev.c | 102 +++++++++++++++++++++++++++++++++++++++++-
 lib/librte_ether/rte_ethdev.h |  80 +++++++++++++++++++++++++++++++++
 2 files changed, 181 insertions(+), 1 deletion(-)

diff --git a/lib/librte_ether/rte_ethdev.c b/lib/librte_ether/rte_ethdev.c
index e5145b7..e572ef4 100644
--- a/lib/librte_ether/rte_ethdev.c
+++ b/lib/librte_ether/rte_ethdev.c
@@ -206,7 +206,7 @@ rte_eth_dev_data_alloc(void)
 				RTE_MAX_ETHPORTS * sizeof(*rte_eth_dev_data));
 }
 
-static struct rte_eth_dev *
+struct rte_eth_dev *
 rte_eth_dev_allocated(const char *name)
 {
 	unsigned i;
@@ -422,6 +422,106 @@ rte_eth_dev_count(void)
 	return (nb_ports);
 }
 
+void
+rte_eth_dev_save(struct rte_eth_dev *devs)
+{
+	if (devs == NULL)
+		return;
+
+	/* save current rte_eth_devices */
+	memcpy(devs, rte_eth_devices,
+			sizeof(struct rte_eth_dev) * RTE_MAX_ETHPORTS);
+}
+
+int
+rte_eth_dev_get_changed_port(struct rte_eth_dev *devs, uint8_t *port_id)
+{
+	if ((devs == NULL) || (port_id == NULL))
+		return -1;
+
+	/* check which port was attached or detached */
+	for (*port_id = 0; *port_id < RTE_MAX_ETHPORTS; (*port_id)++, devs++) {
+		if (rte_eth_devices[*port_id].attached ^ devs->attached)
+			return 0;
+	}
+	return 1;
+}
+
+int
+rte_eth_dev_get_addr_by_port(uint8_t port_id, struct rte_pci_addr *addr)
+{
+	if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
+		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
+		return -EINVAL;
+	}
+
+	if (addr == NULL) {
+		PMD_DEBUG_TRACE("Null pointer is specified\n");
+		return -EINVAL;
+	}
+
+	*addr = rte_eth_devices[port_id].pci_dev->addr;
+	return 0;
+}
+
+int
+rte_eth_dev_get_port_by_addr(struct rte_pci_addr *addr, uint8_t *port_id)
+{
+	struct rte_pci_addr *tmp;
+
+	if ((addr == NULL) || (port_id == NULL)) {
+		PMD_DEBUG_TRACE("Null pointer is specified\n");
+		return -1;
+	}
+
+	for (*port_id = 0; *port_id < RTE_MAX_ETHPORTS; (*port_id)++) {
+		if (!rte_eth_devices[*port_id].attached)
+			continue;
+		if (!rte_eth_devices[*port_id].pci_dev)
+			continue;
+		tmp = &rte_eth_devices[*port_id].pci_dev->addr;
+		if (eal_compare_pci_addr(tmp, addr) == 0)
+			return 0;
+	}
+	return -1;
+}
+
+int
+rte_eth_dev_get_name_by_port(uint8_t port_id, char *name)
+{
+	char *tmp;
+
+	if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
+		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
+		return -EINVAL;
+	}
+
+	if (name == NULL) {
+		PMD_DEBUG_TRACE("Null pointer is specified\n");
+		return -EINVAL;
+	}
+
+	/* shouldn't check 'rte_eth_devices[i].data',
+	 * because it might be overwritten by VDEV PMD */
+	tmp = rte_eth_dev_data[port_id].name;
+	strncpy(name, tmp, strlen(tmp) + 1);
+	return 0;
+}
+
+int
+rte_eth_dev_check_detachable(uint8_t port_id)
+{
+	uint32_t drv_flags;
+
+	if (port_id >= RTE_MAX_ETHPORTS) {
+		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
+		return -EINVAL;
+	}
+
+	drv_flags = rte_eth_devices[port_id].driver->pci_drv.drv_flags;
+	return !(drv_flags & RTE_PCI_DRV_DETACHABLE);
+}
+
 static int
 rte_eth_dev_rx_queue_config(struct rte_eth_dev *dev, uint16_t nb_queues)
 {
diff --git a/lib/librte_ether/rte_ethdev.h b/lib/librte_ether/rte_ethdev.h
index 2a8ff26..c0b98dc 100644
--- a/lib/librte_ether/rte_ethdev.h
+++ b/lib/librte_ether/rte_ethdev.h
@@ -1642,6 +1642,86 @@ extern struct rte_eth_dev rte_eth_devices[];
 extern uint8_t rte_eth_dev_count(void);
 
 /**
+ * Function for internal use by port hotplug functions.
+ * Copies current ethdev structures to the specified pointer.
+ *
+ * @param	devs	The pointer to the ethdev structures
+ */
+extern void rte_eth_dev_save(struct rte_eth_dev *devs);
+
+/**
+ * Function for internal use by port hotplug functions.
+ * Compare the specified ethdev structures with currents. Then
+ * if there is a port which status is changed, fill the specified pointer
+ * with the port id of that port.
+ * @param	devs	The pointer to the ethdev structures
+ * @param	port_id	The pointer to the port id
+ * @return
+ *   - 0 on success, negative on error
+ */
+extern int rte_eth_dev_get_changed_port(
+		struct rte_eth_dev *devs, uint8_t *port_id);
+
+/**
+ * Function for internal use by port hotplug functions.
+ * Returns a pci address of a ethdev specified by port identifier.
+ * @param	port_id
+ *   The port identifier of the Ethernet device
+ * @param	addr
+ *   The pointer to the pci address
+ * @return
+ *   - 0 on success, negative on error
+ */
+extern int rte_eth_dev_get_addr_by_port(
+		uint8_t port_id, struct rte_pci_addr *addr);
+
+/**
+ * Function for internal use by port hotplug functions.
+ * Returns a port identifier of a ethdev specified by pci address.
+ * @param	addr
+ *   The pointer to the pci address of the Ethernet device.
+ * @param	port_id
+ *   The pointer to the port identifier
+ * @return
+ *   - 0 on success, negative on error
+ */
+extern int rte_eth_dev_get_port_by_addr(
+		struct rte_pci_addr *addr, uint8_t *port_id);
+
+/**
+ * Function for internal use by port hotplug functions.
+ * Returns a unique identifier name of a ethdev specified by port identifier.
+ * @param	port_id
+ *   The port identifier.
+ * @param	name
+ *  The pointer to the Unique identifier name for each Ethernet device
+ * @return
+ *   - 0 on success, negative on error
+ */
+extern int rte_eth_dev_get_name_by_port(uint8_t port_id, char *name);
+
+/**
+ * Function for internal use by port hotplug functions.
+ * Check whether or not, a PMD that is handling the ethdev specified by port
+ * identifier can support detach function.
+ * @param	port_id
+ *   The port identifier
+ * @return
+ *   - 0 on supporting detach function, negative on not supporting
+ */
+extern int rte_eth_dev_check_detachable(uint8_t port_id);
+
+/**
+ * Function for internal use by port hotplug functions.
+ * Returns a ethdev slot specified by the unique identifier name.
+ * @param	name
+ *  The pointer to the Unique identifier name for each Ethernet device
+ * @return
+ *   - The pointer to the ethdev slot, on success. NULL on error
+ */
+extern struct rte_eth_dev *rte_eth_dev_allocated(const char *name);
+
+/**
  * Function for internal use by dummy drivers primarily, e.g. ring-based
  * driver.
  * Allocates a new ethdev slot for an ethernet device and returns the pointer
-- 
1.9.1

^ permalink raw reply	[flat|nested] 362+ messages in thread

* [dpdk-dev] [PATCH v4 06/11] eal/linux/pci: Add functions for unmapping igb_uio resources
  2015-01-19 10:40 ` [dpdk-dev] [PATCH v4 00/11] Port Hotplug Framework Tetsuya Mukawa
                     ` (4 preceding siblings ...)
  2015-01-19 10:40   ` [dpdk-dev] [PATCH v4 05/11] ethdev: Add functions that will be used by port hotplug functions Tetsuya Mukawa
@ 2015-01-19 10:40   ` Tetsuya Mukawa
  2015-01-20  9:23     ` Qiu, Michael
  2015-01-19 10:40   ` [dpdk-dev] [PATCH v4 07/11] eal/pci: Add a function to remove the entry of devargs list Tetsuya Mukawa
                     ` (6 subsequent siblings)
  12 siblings, 1 reply; 362+ messages in thread
From: Tetsuya Mukawa @ 2015-01-19 10:40 UTC (permalink / raw)
  To: dev

The patch adds functions for unmapping igb_uio resources. The patch is only
for Linux and igb_uio environment. VFIO and BSD are not supported.

v4:
- Add paramerter checking.
- Add header file to determine if hotplug can be enabled.

Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp>
---
 lib/librte_eal/common/Makefile                  |  1 +
 lib/librte_eal/common/include/rte_dev_hotplug.h | 44 +++++++++++++++++
 lib/librte_eal/linuxapp/eal/eal_pci.c           | 38 +++++++++++++++
 lib/librte_eal/linuxapp/eal/eal_pci_init.h      |  8 +++
 lib/librte_eal/linuxapp/eal/eal_pci_uio.c       | 65 +++++++++++++++++++++++++
 5 files changed, 156 insertions(+)
 create mode 100644 lib/librte_eal/common/include/rte_dev_hotplug.h

diff --git a/lib/librte_eal/common/Makefile b/lib/librte_eal/common/Makefile
index 52c1a5f..db7cc93 100644
--- a/lib/librte_eal/common/Makefile
+++ b/lib/librte_eal/common/Makefile
@@ -41,6 +41,7 @@ INC += rte_eal_memconfig.h rte_malloc_heap.h
 INC += rte_hexdump.h rte_devargs.h rte_dev.h
 INC += rte_common_vect.h
 INC += rte_pci_dev_feature_defs.h rte_pci_dev_features.h
+INC += rte_dev_hotplug.h
 
 ifeq ($(CONFIG_RTE_INSECURE_FUNCTION_WARNING),y)
 INC += rte_warnings.h
diff --git a/lib/librte_eal/common/include/rte_dev_hotplug.h b/lib/librte_eal/common/include/rte_dev_hotplug.h
new file mode 100644
index 0000000..b333e0f
--- /dev/null
+++ b/lib/librte_eal/common/include/rte_dev_hotplug.h
@@ -0,0 +1,44 @@
+/*-
+ *   BSD LICENSE
+ *
+ *   Copyright(c) 2015 IGEL Co.,LTd.
+ *   All rights reserved.
+ *
+ *   Redistribution and use in source and binary forms, with or without
+ *   modification, are permitted provided that the following conditions
+ *   are met:
+ *
+ *     * Redistributions of source code must retain the above copyright
+ *       notice, this list of conditions and the following disclaimer.
+ *     * Redistributions in binary form must reproduce the above copyright
+ *       notice, this list of conditions and the following disclaimer in
+ *       the documentation and/or other materials provided with the
+ *       distribution.
+ *     * Neither the name of IGEL Co.,Ltd. nor the names of its
+ *       contributors may be used to endorse or promote products derived
+ *       from this software without specific prior written permission.
+ *
+ *   THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+ *   "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+ *   LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+ *   A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+ *   OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ *   SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+ *   LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+ *   DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+ *   THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ *   (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ *   OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#ifndef _RTE_DEV_HOTPLUG_H_
+#define _RTE_DEV_HOTPLUG_H_
+
+/*
+ * determine if hotplug can be enabled on the system
+ */
+#if defined(RTE_LIBRTE_EAL_HOTPLUG) && defined(RTE_LIBRTE_EAL_LINUXAPP)
+#define ENABLE_HOTPLUG
+#endif /* RTE_LIBRTE_EAL_HOTPLUG & RTE_LIBRTE_EAL_LINUXAPP */
+
+#endif /* _RTE_DEV_HOTPLUG_H_ */
diff --git a/lib/librte_eal/linuxapp/eal/eal_pci.c b/lib/librte_eal/linuxapp/eal/eal_pci.c
index 3d2d93c..52c464c 100644
--- a/lib/librte_eal/linuxapp/eal/eal_pci.c
+++ b/lib/librte_eal/linuxapp/eal/eal_pci.c
@@ -137,6 +137,25 @@ pci_map_resource(void *requested_addr, int fd, off_t offset, size_t size)
 	return mapaddr;
 }
 
+#ifdef ENABLE_HOTPLUG
+/* unmap a particular resource */
+void
+pci_unmap_resource(void *requested_addr, size_t size)
+{
+	if (requested_addr == NULL)
+		return;
+
+	/* Unmap the PCI memory resource of device */
+	if (munmap(requested_addr, size)) {
+		RTE_LOG(ERR, EAL, "%s(): cannot munmap(%p, 0x%lx): %s\n",
+			__func__, requested_addr, (unsigned long)size,
+			strerror(errno));
+	} else
+		RTE_LOG(DEBUG, EAL, "  PCI memory mapped at %p\n",
+				requested_addr);
+}
+#endif /* ENABLE_HOTPLUG */
+
 /* parse the "resource" sysfs file */
 #define IORESOURCE_MEM  0x00000200
 
@@ -510,6 +529,25 @@ pci_map_device(struct rte_pci_device *dev)
 	return 0;
 }
 
+#ifdef ENABLE_HOTPLUG
+static void
+pci_unmap_device(struct rte_pci_device *dev)
+{
+	if (dev == NULL)
+		return;
+
+	/* try unmapping the NIC resources using VFIO if it exists */
+#ifdef VFIO_PRESENT
+	if (pci_vfio_is_enabled()) {
+		RTE_LOG(ERR, EAL, "%s() doesn't support vfio yet.\n",
+				__func__);
+		return;
+	}
+#endif
+	pci_uio_unmap_resource(dev);
+}
+#endif /* ENABLE_HOTPLUG */
+
 /*
  * If vendor/device ID match, call the devinit() function of the
  * driver.
diff --git a/lib/librte_eal/linuxapp/eal/eal_pci_init.h b/lib/librte_eal/linuxapp/eal/eal_pci_init.h
index 1070eb8..5152a0b 100644
--- a/lib/librte_eal/linuxapp/eal/eal_pci_init.h
+++ b/lib/librte_eal/linuxapp/eal/eal_pci_init.h
@@ -34,6 +34,7 @@
 #ifndef EAL_PCI_INIT_H_
 #define EAL_PCI_INIT_H_
 
+#include <rte_dev_hotplug.h>
 #include "eal_vfio.h"
 
 struct pci_map {
@@ -71,6 +72,13 @@ void *pci_map_resource(void *requested_addr, int fd, off_t offset,
 /* map IGB_UIO resource prototype */
 int pci_uio_map_resource(struct rte_pci_device *dev);
 
+#ifdef ENABLE_HOTPLUG
+void pci_unmap_resource(void *requested_addr, size_t size);
+
+/* unmap IGB_UIO resource prototype */
+void pci_uio_unmap_resource(struct rte_pci_device *dev);
+#endif /* ENABLE_HOTPLUG */
+
 #ifdef VFIO_PRESENT
 
 #define VFIO_MAX_GROUPS 64
diff --git a/lib/librte_eal/linuxapp/eal/eal_pci_uio.c b/lib/librte_eal/linuxapp/eal/eal_pci_uio.c
index 1da3507..81830d1 100644
--- a/lib/librte_eal/linuxapp/eal/eal_pci_uio.c
+++ b/lib/librte_eal/linuxapp/eal/eal_pci_uio.c
@@ -404,6 +404,71 @@ pci_uio_map_resource(struct rte_pci_device *dev)
 	return 0;
 }
 
+#ifdef ENABLE_HOTPLUG
+static void
+pci_uio_unmap(struct mapped_pci_resource *uio_res)
+{
+	int i;
+
+	if (uio_res == NULL)
+		return;
+
+	for (i = 0; i != uio_res->nb_maps; i++)
+		pci_unmap_resource(uio_res->maps[i].addr,
+				(size_t)uio_res->maps[i].size);
+}
+
+static struct mapped_pci_resource *
+pci_uio_find_resource(struct rte_pci_device *dev)
+{
+	struct mapped_pci_resource *uio_res;
+
+	if (dev == NULL)
+		return NULL;
+
+	TAILQ_FOREACH(uio_res, pci_res_list, next) {
+
+		/* skip this element if it doesn't match our PCI address */
+		if (!eal_compare_pci_addr(&uio_res->pci_addr, &dev->addr))
+			return uio_res;
+	}
+	return NULL;
+}
+
+/* unmap the PCI resource of a PCI device in virtual memory */
+void
+pci_uio_unmap_resource(struct rte_pci_device *dev)
+{
+	struct mapped_pci_resource *uio_res;
+
+	if (dev == NULL)
+		return;
+
+	/* find an entry for the device */
+	uio_res = pci_uio_find_resource(dev);
+	if (uio_res == NULL)
+		return;
+
+	/* secondary processes - just free maps */
+	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
+		return pci_uio_unmap(uio_res);
+
+	TAILQ_REMOVE(pci_res_list, uio_res, next);
+
+	/* unmap all resources */
+	pci_uio_unmap(uio_res);
+
+	/* free uio resource */
+	rte_free(uio_res);
+
+	/* close fd if in primary process */
+	close(dev->intr_handle.fd);
+
+	dev->intr_handle.fd = -1;
+	dev->intr_handle.type = RTE_INTR_HANDLE_UNKNOWN;
+}
+#endif /* ENABLE_HOTPLUG */
+
 /*
  * parse a sysfs file containing one integer value
  * different to the eal version, as it needs to work with 64-bit values
-- 
1.9.1

^ permalink raw reply	[flat|nested] 362+ messages in thread

* [dpdk-dev] [PATCH v4 07/11] eal/pci: Add a function to remove the entry of devargs list
  2015-01-19 10:40 ` [dpdk-dev] [PATCH v4 00/11] Port Hotplug Framework Tetsuya Mukawa
                     ` (5 preceding siblings ...)
  2015-01-19 10:40   ` [dpdk-dev] [PATCH v4 06/11] eal/linux/pci: Add functions for unmapping igb_uio resources Tetsuya Mukawa
@ 2015-01-19 10:40   ` Tetsuya Mukawa
  2015-01-21  2:55     ` Qiu, Michael
  2015-01-19 10:40   ` [dpdk-dev] [PATCH v4 08/11] eal/pci: Cleanup pci driver initialization code Tetsuya Mukawa
                     ` (5 subsequent siblings)
  12 siblings, 1 reply; 362+ messages in thread
From: Tetsuya Mukawa @ 2015-01-19 10:40 UTC (permalink / raw)
  To: dev

The function removes the specified devargs entry from devargs_list.
Also the patch adds sanity checking to rte_eal_devargs_add().

v4:
- Fix sanity check code

Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp>
---
 lib/librte_eal/common/eal_common_devargs.c  | 57 +++++++++++++++++++++++++++++
 lib/librte_eal/common/include/rte_devargs.h | 18 +++++++++
 2 files changed, 75 insertions(+)

diff --git a/lib/librte_eal/common/eal_common_devargs.c b/lib/librte_eal/common/eal_common_devargs.c
index 4c7d11a..a360a85 100644
--- a/lib/librte_eal/common/eal_common_devargs.c
+++ b/lib/librte_eal/common/eal_common_devargs.c
@@ -44,6 +44,35 @@
 struct rte_devargs_list devargs_list =
 	TAILQ_HEAD_INITIALIZER(devargs_list);
 
+
+/* find a entry specified by pci address or device name */
+static struct rte_devargs *
+rte_eal_devargs_find(enum rte_devtype devtype, void *args)
+{
+	struct rte_devargs *devargs;
+
+	if (args == NULL)
+		return NULL;
+
+	TAILQ_FOREACH(devargs, &devargs_list, next) {
+		switch (devtype) {
+		case RTE_DEVTYPE_WHITELISTED_PCI:
+		case RTE_DEVTYPE_BLACKLISTED_PCI:
+			if (eal_compare_pci_addr(&devargs->pci.addr, args) == 0)
+				goto found;
+			break;
+		case RTE_DEVTYPE_VIRTUAL:
+			if (memcmp(&devargs->virtual.drv_name, args,
+			    strlen((char *)args)) == 0)
+				goto found;
+			break;
+		}
+	}
+	return NULL;
+found:
+	return devargs;
+}
+
 /* store a whitelist parameter for later parsing */
 int
 rte_eal_devargs_add(enum rte_devtype devtype, const char *devargs_str)
@@ -87,6 +116,12 @@ rte_eal_devargs_add(enum rte_devtype devtype, const char *devargs_str)
 			free(devargs);
 			return -1;
 		}
+		/* make sure there is no same entry */
+		if (rte_eal_devargs_find(devtype, &devargs->pci.addr)) {
+			RTE_LOG(ERR, EAL,
+				"device already registered: <%s>\n", buf);
+			return -1;
+		}
 		break;
 	case RTE_DEVTYPE_VIRTUAL:
 		/* save driver name */
@@ -98,6 +133,12 @@ rte_eal_devargs_add(enum rte_devtype devtype, const char *devargs_str)
 			free(devargs);
 			return -1;
 		}
+		/* make sure there is no same entry */
+		if (rte_eal_devargs_find(devtype, &devargs->virtual.drv_name)) {
+			RTE_LOG(ERR, EAL,
+				"device already registered: <%s>\n", buf);
+			return -1;
+		}
 		break;
 	}
 
@@ -105,6 +146,22 @@ rte_eal_devargs_add(enum rte_devtype devtype, const char *devargs_str)
 	return 0;
 }
 
+/* remove it from the devargs_list */
+void
+rte_eal_devargs_remove(enum rte_devtype devtype, void *args)
+{
+	struct rte_devargs *devargs;
+
+	if (args == NULL)
+		return;
+
+	devargs = rte_eal_devargs_find(devtype, args);
+	if (devargs == NULL)
+		return;
+
+	TAILQ_REMOVE(&devargs_list, devargs, next);
+}
+
 /* count the number of devices of a specified type */
 unsigned int
 rte_eal_devargs_type_count(enum rte_devtype devtype)
diff --git a/lib/librte_eal/common/include/rte_devargs.h b/lib/librte_eal/common/include/rte_devargs.h
index 9f9c98f..1066efd 100644
--- a/lib/librte_eal/common/include/rte_devargs.h
+++ b/lib/librte_eal/common/include/rte_devargs.h
@@ -123,6 +123,24 @@ extern struct rte_devargs_list devargs_list;
 int rte_eal_devargs_add(enum rte_devtype devtype, const char *devargs_str);
 
 /**
+ * Remove a device from the user device list
+ *
+ * For PCI devices, the format of arguments string is "PCI_ADDR". It shouldn't
+ * involves parameters for the device. Example: "08:00.1".
+ *
+ * For virtual devices, the format of arguments string is "DRIVER_NAME*". It
+ * shouldn't involves parameters for the device. Example: "eth_ring". The
+ * validity of the driver name is not checked by this function, it is done
+ * when closing the drivers.
+ *
+ * @param devtype
+ *   The type of the device.
+ * @param name
+ *   The name of the device.
+ */
+void rte_eal_devargs_remove(enum rte_devtype devtype, void *args);
+
+/**
  * Count the number of user devices of a specified type
  *
  * @param devtype
-- 
1.9.1

^ permalink raw reply	[flat|nested] 362+ messages in thread

* [dpdk-dev] [PATCH v4 08/11] eal/pci: Cleanup pci driver initialization code
  2015-01-19 10:40 ` [dpdk-dev] [PATCH v4 00/11] Port Hotplug Framework Tetsuya Mukawa
                     ` (6 preceding siblings ...)
  2015-01-19 10:40   ` [dpdk-dev] [PATCH v4 07/11] eal/pci: Add a function to remove the entry of devargs list Tetsuya Mukawa
@ 2015-01-19 10:40   ` Tetsuya Mukawa
  2015-01-30  5:42     ` [dpdk-dev] [PATCH v5 00/13] Port Hotplug Framework Tetsuya Mukawa
  2015-02-01  4:01     ` [dpdk-dev] [PATCH v6 00/13] Port Hotplug Framework Tetsuya Mukawa
  2015-01-19 10:40   ` [dpdk-dev] [PATCH v4 09/11] ethdev: Add one dev_type paramerter to rte_eth_dev_allocate Tetsuya Mukawa
                     ` (4 subsequent siblings)
  12 siblings, 2 replies; 362+ messages in thread
From: Tetsuya Mukawa @ 2015-01-19 10:40 UTC (permalink / raw)
  To: dev

- Add rte_eal_pci_close_one_dirver()
  The function is used for closing the specified driver and device.
- Add pci_invoke_all_drivers()
  The function is based on pci_probe_all_drivers. But it can not only
  probe but also close drivers.
- Add pci_close_all_drivers()
  The function tries to find a driver for the specified device, and
  then close the driver.
- Add rte_eal_pci_probe_one() and rte_eal_pci_close_one()
  The functions are used for probe and close a device.
  First the function tries to find a device that has the specfied
  PCI address. Then, probe or close the device.

v4:
- Fix paramerter checking.
- Fix indent of 'if' statement.

Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp>
---
 lib/librte_eal/common/eal_common_pci.c  | 90 +++++++++++++++++++++++++++++----
 lib/librte_eal/common/eal_private.h     | 25 +++++++++
 lib/librte_eal/common/include/rte_pci.h | 33 ++++++++++++
 lib/librte_eal/linuxapp/eal/eal_pci.c   | 69 +++++++++++++++++++++++++
 4 files changed, 207 insertions(+), 10 deletions(-)

diff --git a/lib/librte_eal/common/eal_common_pci.c b/lib/librte_eal/common/eal_common_pci.c
index a89f5c3..7c9b8c5 100644
--- a/lib/librte_eal/common/eal_common_pci.c
+++ b/lib/librte_eal/common/eal_common_pci.c
@@ -99,19 +99,27 @@ static struct rte_devargs *pci_devargs_lookup(struct rte_pci_device *dev)
 	return NULL;
 }
 
-/*
- * If vendor/device ID match, call the devinit() function of all
- * registered driver for the given device. Return -1 if initialization
- * failed, return 1 if no driver is found for this device.
- */
 static int
-pci_probe_all_drivers(struct rte_pci_device *dev)
+pci_invoke_all_drivers(struct rte_pci_device *dev,
+		enum rte_eal_invoke_type type)
 {
 	struct rte_pci_driver *dr = NULL;
-	int rc;
+	int rc = 0;
+
+	if ((dev == NULL) || (type >= RTE_EAL_INVOKE_TYPE_MAX))
+		return -1;
 
 	TAILQ_FOREACH(dr, &pci_driver_list, next) {
-		rc = rte_eal_pci_probe_one_driver(dr, dev);
+		switch (type) {
+		case RTE_EAL_INVOKE_TYPE_PROBE:
+			rc = rte_eal_pci_probe_one_driver(dr, dev);
+			break;
+		case RTE_EAL_INVOKE_TYPE_CLOSE:
+			rc = rte_eal_pci_close_one_driver(dr, dev);
+			break;
+		default:
+			return -1;
+		}
 		if (rc < 0)
 			/* negative value is an error */
 			return -1;
@@ -123,6 +131,66 @@ pci_probe_all_drivers(struct rte_pci_device *dev)
 	return 1;
 }
 
+#ifdef ENABLE_HOTPLUG
+static int
+rte_eal_pci_invoke_one(struct rte_pci_addr *addr,
+		enum rte_eal_invoke_type type)
+{
+	struct rte_pci_device *dev = NULL;
+	int ret = 0;
+
+	if ((addr == NULL) || (type >= RTE_EAL_INVOKE_TYPE_MAX))
+		return -1;
+
+	TAILQ_FOREACH(dev, &pci_device_list, next) {
+		if (eal_compare_pci_addr(&dev->addr, addr))
+			continue;
+
+		ret = pci_invoke_all_drivers(dev, type);
+		if (ret < 0)
+			goto invoke_err_return;
+
+		if (type == RTE_EAL_INVOKE_TYPE_CLOSE)
+			goto remove_dev;
+
+		return 0;
+	}
+
+	return -1;
+
+invoke_err_return:
+	RTE_LOG(WARNING, EAL, "Requested device " PCI_PRI_FMT
+			" cannot be used\n", dev->addr.domain, dev->addr.bus,
+			dev->addr.devid, dev->addr.function);
+	return -1;
+
+remove_dev:
+	TAILQ_REMOVE(&pci_device_list, dev, next);
+	return 0;
+}
+
+
+/*
+ * Find the pci device specified by pci address, then invoke probe function of
+ * the driver of the devive.
+ */
+int
+rte_eal_pci_probe_one(struct rte_pci_addr *addr)
+{
+	return rte_eal_pci_invoke_one(addr, RTE_EAL_INVOKE_TYPE_PROBE);
+}
+
+/*
+ * Find the pci device specified by pci address, then invoke close function of
+ * the driver of the devive.
+ */
+int
+rte_eal_pci_close_one(struct rte_pci_addr *addr)
+{
+	return rte_eal_pci_invoke_one(addr, RTE_EAL_INVOKE_TYPE_CLOSE);
+}
+#endif /* ENABLE_HOTPLUG */
+
 /*
  * Scan the content of the PCI bus, and call the devinit() function for
  * all registered drivers that have a matching entry in its id_table
@@ -148,10 +216,12 @@ rte_eal_pci_probe(void)
 
 		/* probe all or only whitelisted devices */
 		if (probe_all)
-			ret = pci_probe_all_drivers(dev);
+			ret = pci_invoke_all_drivers(dev,
+					RTE_EAL_INVOKE_TYPE_PROBE);
 		else if (devargs != NULL &&
 			devargs->type == RTE_DEVTYPE_WHITELISTED_PCI)
-			ret = pci_probe_all_drivers(dev);
+			ret = pci_invoke_all_drivers(dev,
+					RTE_EAL_INVOKE_TYPE_PROBE);
 		if (ret < 0)
 			rte_exit(EXIT_FAILURE, "Requested device " PCI_PRI_FMT
 				 " cannot be used\n", dev->addr.domain, dev->addr.bus,
diff --git a/lib/librte_eal/common/eal_private.h b/lib/librte_eal/common/eal_private.h
index 159cd66..a97c5d8 100644
--- a/lib/librte_eal/common/eal_private.h
+++ b/lib/librte_eal/common/eal_private.h
@@ -154,6 +154,16 @@ struct rte_pci_driver;
 struct rte_pci_device;
 
 /**
+ * The invoke type.
+ */
+enum rte_eal_invoke_type {
+	RTE_EAL_INVOKE_TYPE_UNKNOWN,/**< unknown */
+	RTE_EAL_INVOKE_TYPE_PROBE,  /**< invoke probe function */
+	RTE_EAL_INVOKE_TYPE_CLOSE,  /**< invoke close function */
+	RTE_EAL_INVOKE_TYPE_MAX     /**< max value of this enum */
+};
+
+/**
  * Mmap memory for single PCI device
  *
  * This function is private to EAL.
@@ -165,6 +175,21 @@ int rte_eal_pci_probe_one_driver(struct rte_pci_driver *dr,
 		struct rte_pci_device *dev);
 
 /**
+ * Munmap memory for single PCI device
+ *
+ * This function is private to EAL.
+ *
+ * @param	dr
+ *  The pointer to the pci driver structure
+ * @param	dev
+ *  The pointer to the pci device structure
+ * @return
+ *   0 on success, negative on error
+ */
+int rte_eal_pci_close_one_driver(struct rte_pci_driver *dr,
+		struct rte_pci_device *dev);
+
+/**
  * Init tail queues for non-EAL library structures. This is to allow
  * the rings, mempools, etc. lists to be shared among multiple processes
  *
diff --git a/lib/librte_eal/common/include/rte_pci.h b/lib/librte_eal/common/include/rte_pci.h
index fe57cb6..28462f5 100644
--- a/lib/librte_eal/common/include/rte_pci.h
+++ b/lib/librte_eal/common/include/rte_pci.h
@@ -82,6 +82,7 @@ extern "C" {
 #include <inttypes.h>
 
 #include <rte_interrupts.h>
+#include <rte_dev_hotplug.h>
 
 TAILQ_HEAD(pci_device_list, rte_pci_device); /**< PCI devices in D-linked Q. */
 TAILQ_HEAD(pci_driver_list, rte_pci_driver); /**< PCI drivers in D-linked Q. */
@@ -315,6 +316,38 @@ eal_compare_pci_addr(struct rte_pci_addr *addr, struct rte_pci_addr *addr2)
  */
 int rte_eal_pci_probe(void);
 
+#ifdef ENABLE_HOTPLUG
+/**
+ * Probe the single PCI device.
+ *
+ * Scan the content of the PCI bus, and find the pci device specified by pci
+ * address, then call the probe() function for registered driver that has a
+ * matching entry in its id_table for discovered device.
+ *
+ * @param addr
+ *	The PCI Bus-Device-Function address to probe or close.
+ * @return
+ *   - 0 on success.
+ *   - Negative on error.
+ */
+int rte_eal_pci_probe_one(struct rte_pci_addr *addr);
+
+/**
+ * Close the single PCI device.
+ *
+ * Scan the content of the PCI bus, and find the pci device specified by pci
+ * address, then call the close() function for registered driver that has a
+ * matching entry in its id_table for discovered device.
+ *
+ * @param addr
+ *	The PCI Bus-Device-Function address to probe or close.
+ * @return
+ *   - 0 on success.
+ *   - Negative on error.
+ */
+int rte_eal_pci_close_one(struct rte_pci_addr *addr);
+#endif /* ENABLE_HOTPLUG */
+
 /**
  * Dump the content of the PCI bus.
  *
diff --git a/lib/librte_eal/linuxapp/eal/eal_pci.c b/lib/librte_eal/linuxapp/eal/eal_pci.c
index 52c464c..a23cc59 100644
--- a/lib/librte_eal/linuxapp/eal/eal_pci.c
+++ b/lib/librte_eal/linuxapp/eal/eal_pci.c
@@ -619,6 +619,75 @@ rte_eal_pci_probe_one_driver(struct rte_pci_driver *dr, struct rte_pci_device *d
 	return 1;
 }
 
+#ifdef ENABLE_HOTPLUG
+/*
+ * If vendor/device ID match, call the devuninit() function of the
+ * driver.
+ */
+int
+rte_eal_pci_close_one_driver(struct rte_pci_driver *dr,
+		struct rte_pci_device *dev)
+{
+	struct rte_pci_id *id_table;
+
+	if ((dr == NULL) || (dev == NULL))
+		return -EINVAL;
+
+	for (id_table = dr->id_table ; id_table->vendor_id != 0; id_table++) {
+
+		/* check if device's identifiers match the driver's ones */
+		if (id_table->vendor_id != dev->id.vendor_id &&
+		    id_table->vendor_id != PCI_ANY_ID)
+			continue;
+		if (id_table->device_id != dev->id.device_id &&
+		    id_table->device_id != PCI_ANY_ID)
+			continue;
+		if (id_table->subsystem_vendor_id !=
+		    dev->id.subsystem_vendor_id &&
+		    id_table->subsystem_vendor_id != PCI_ANY_ID)
+			continue;
+		if (id_table->subsystem_device_id !=
+		    dev->id.subsystem_device_id &&
+		    id_table->subsystem_device_id != PCI_ANY_ID)
+			continue;
+
+		struct rte_pci_addr *loc = &dev->addr;
+
+		RTE_LOG(DEBUG, EAL,
+				"PCI device "PCI_PRI_FMT" on NUMA socket %i\n",
+				loc->domain, loc->bus, loc->devid,
+				loc->function, dev->numa_node);
+
+		RTE_LOG(DEBUG, EAL, "  remove driver: %x:%x %s\n",
+				dev->id.vendor_id, dev->id.device_id,
+				dr->name);
+
+		/* call the driver devuninit() function */
+		if (dr->devuninit && (dr->devuninit(dr, dev) < 0))
+			return -1;	/* negative value is an error */
+
+		/* clear driver structure */
+		dev->driver = NULL;
+
+		if (dr->drv_flags & RTE_PCI_DRV_NEED_MAPPING)
+			/* unmap resources for devices that use igb_uio */
+			pci_unmap_device(dev);
+
+		return 0;
+	}
+	/* return positive value if driver is not found */
+	return 1;
+}
+#else /* ENABLE_HOTPLUG */
+int
+rte_eal_pci_close_one_driver(struct rte_pci_driver *dr __rte_unused,
+		struct rte_pci_device *dev __rte_unused)
+{
+	RTE_LOG(ERR, EAL, "Hotplug support isn't enabled\n");
+	return -1;
+}
+#endif /* ENABLE_HOTPLUG */
+
 /* Init the PCI EAL subsystem */
 int
 rte_eal_pci_init(void)
-- 
1.9.1

^ permalink raw reply	[flat|nested] 362+ messages in thread

* [dpdk-dev] [PATCH v4 09/11] ethdev: Add one dev_type paramerter to rte_eth_dev_allocate
  2015-01-19 10:40 ` [dpdk-dev] [PATCH v4 00/11] Port Hotplug Framework Tetsuya Mukawa
                     ` (7 preceding siblings ...)
  2015-01-19 10:40   ` [dpdk-dev] [PATCH v4 08/11] eal/pci: Cleanup pci driver initialization code Tetsuya Mukawa
@ 2015-01-19 10:40   ` Tetsuya Mukawa
  2015-01-19 10:40   ` [dpdk-dev] [PATCH v4 10/11] eal/pci: Add rte_eal_dev_attach/detach() functions Tetsuya Mukawa
                     ` (3 subsequent siblings)
  12 siblings, 0 replies; 362+ messages in thread
From: Tetsuya Mukawa @ 2015-01-19 10:40 UTC (permalink / raw)
  To: dev

This new parameter is needed to keep device type like physical or virtual.
Port detaching processes are different between physical and virtual.
This paramerter lets detaching function know a device type of the port.

v4:
- Fix comments of rte_eth_dev_type.

Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp>
---
 app/test/virtual_pmd.c                       |  2 +-
 lib/librte_ether/rte_ethdev.c                | 14 ++++++++++++--
 lib/librte_ether/rte_ethdev.h                | 25 ++++++++++++++++++++++++-
 lib/librte_pmd_af_packet/rte_eth_af_packet.c |  2 +-
 lib/librte_pmd_bond/rte_eth_bond_api.c       |  2 +-
 lib/librte_pmd_pcap/rte_eth_pcap.c           |  2 +-
 lib/librte_pmd_ring/rte_eth_ring.c           |  2 +-
 lib/librte_pmd_xenvirt/rte_eth_xenvirt.c     |  2 +-
 8 files changed, 42 insertions(+), 9 deletions(-)

diff --git a/app/test/virtual_pmd.c b/app/test/virtual_pmd.c
index 9fac95d..8d3a5ff 100644
--- a/app/test/virtual_pmd.c
+++ b/app/test/virtual_pmd.c
@@ -556,7 +556,7 @@ virtual_ethdev_create(const char *name, struct ether_addr *mac_addr,
 		goto err;
 
 	/* reserve an ethdev entry */
-	eth_dev = rte_eth_dev_allocate(name);
+	eth_dev = rte_eth_dev_allocate(name, RTE_ETH_DEV_PHYSICAL);
 	if (eth_dev == NULL)
 		goto err;
 
diff --git a/lib/librte_ether/rte_ethdev.c b/lib/librte_ether/rte_ethdev.c
index e572ef4..d9bf24c 100644
--- a/lib/librte_ether/rte_ethdev.c
+++ b/lib/librte_ether/rte_ethdev.c
@@ -232,7 +232,7 @@ rte_eth_dev_allocate_new_port(void)
 }
 
 struct rte_eth_dev *
-rte_eth_dev_allocate(const char *name)
+rte_eth_dev_allocate(const char *name, enum rte_eth_dev_type type)
 {
 	uint8_t port_id;
 	struct rte_eth_dev *eth_dev;
@@ -256,6 +256,7 @@ rte_eth_dev_allocate(const char *name)
 	snprintf(eth_dev->data->name, sizeof(eth_dev->data->name), "%s", name);
 	eth_dev->data->port_id = port_id;
 	eth_dev->attached = DEV_CONNECTED;
+	eth_dev->dev_type = type;
 	nb_ports++;
 	return eth_dev;
 }
@@ -276,6 +277,7 @@ rte_eth_dev_free(const char *name)
 	}
 
 	eth_dev->attached = 0;
+	eth_dev->dev_type = RTE_ETH_DEV_UNKNOWN;
 	nb_ports--;
 	return eth_dev;
 }
@@ -296,7 +298,7 @@ rte_eth_dev_init(struct rte_pci_driver *pci_drv,
 	snprintf(ethdev_name, RTE_ETH_NAME_MAX_LEN, "%d:%d.%d",
 			pci_dev->addr.bus, pci_dev->addr.devid, pci_dev->addr.function);
 
-	eth_dev = rte_eth_dev_allocate(ethdev_name);
+	eth_dev = rte_eth_dev_allocate(ethdev_name, RTE_ETH_DEV_PHYSICAL);
 	if (eth_dev == NULL)
 		return -ENOMEM;
 
@@ -422,6 +424,14 @@ rte_eth_dev_count(void)
 	return (nb_ports);
 }
 
+enum rte_eth_dev_type
+rte_eth_dev_get_device_type(uint8_t port_id)
+{
+	if (rte_eth_dev_validate_port(port_id) == DEV_INVALID)
+		return -1;
+	return rte_eth_devices[port_id].dev_type;
+}
+
 void
 rte_eth_dev_save(struct rte_eth_dev *devs)
 {
diff --git a/lib/librte_ether/rte_ethdev.h b/lib/librte_ether/rte_ethdev.h
index c0b98dc..8b0bad6 100644
--- a/lib/librte_ether/rte_ethdev.h
+++ b/lib/librte_ether/rte_ethdev.h
@@ -1548,6 +1548,17 @@ struct eth_dev_ops {
 };
 
 /**
+ * The eth device type
+ */
+enum rte_eth_dev_type {
+	RTE_ETH_DEV_UNKNOWN,	/**< unknown device type */
+	RTE_ETH_DEV_PHYSICAL,
+		/**< Physical function and Virtual function devices of NIC */
+	RTE_ETH_DEV_VIRTUAL,	/**< non hardware device */
+	RTE_ETH_DEV_MAX		/**< max value of this enum */
+};
+
+/**
  * @internal
  * The generic data structure associated with each ethernet device.
  *
@@ -1566,6 +1577,7 @@ struct rte_eth_dev {
 	struct rte_pci_device *pci_dev; /**< PCI info. supplied by probing */
 	struct rte_eth_dev_cb_list callbacks; /**< User application callbacks */
 	uint8_t attached; /**< Flag indicating the port is attached */
+	enum rte_eth_dev_type dev_type; /**< Flag indicating the device type */
 };
 
 struct rte_eth_dev_sriov {
@@ -1643,6 +1655,15 @@ extern uint8_t rte_eth_dev_count(void);
 
 /**
  * Function for internal use by port hotplug functions.
+ * Get the device type to know whether the device is physical or virtual.
+ * @param	port_id	The pointer to the port id
+ * @return
+ *   - Device type.
+ */
+extern enum rte_eth_dev_type rte_eth_dev_get_device_type(uint8_t port_id);
+
+/**
+ * Function for internal use by port hotplug functions.
  * Copies current ethdev structures to the specified pointer.
  *
  * @param	devs	The pointer to the ethdev structures
@@ -1728,10 +1749,12 @@ extern struct rte_eth_dev *rte_eth_dev_allocated(const char *name);
  * to that slot for the driver to use.
  *
  * @param	name	Unique identifier name for each Ethernet device
+ * @param	type	Device type of this Ethernet device
  * @return
  *   - Slot in the rte_dev_devices array for a new device;
  */
-struct rte_eth_dev *rte_eth_dev_allocate(const char *name);
+struct rte_eth_dev *rte_eth_dev_allocate(const char *name,
+		enum rte_eth_dev_type type);
 
 /**
  * Function for internal use by dummy drivers primarily, e.g. ring-based
diff --git a/lib/librte_pmd_af_packet/rte_eth_af_packet.c b/lib/librte_pmd_af_packet/rte_eth_af_packet.c
index 755780a..ff44ba7 100644
--- a/lib/librte_pmd_af_packet/rte_eth_af_packet.c
+++ b/lib/librte_pmd_af_packet/rte_eth_af_packet.c
@@ -651,7 +651,7 @@ rte_pmd_init_internals(const char *name,
 	}
 
 	/* reserve an ethdev entry */
-	*eth_dev = rte_eth_dev_allocate(name);
+	*eth_dev = rte_eth_dev_allocate(name, RTE_ETH_DEV_VIRTUAL);
 	if (*eth_dev == NULL)
 		goto error;
 
diff --git a/lib/librte_pmd_bond/rte_eth_bond_api.c b/lib/librte_pmd_bond/rte_eth_bond_api.c
index 4ab3267..7a6a5f7 100644
--- a/lib/librte_pmd_bond/rte_eth_bond_api.c
+++ b/lib/librte_pmd_bond/rte_eth_bond_api.c
@@ -235,7 +235,7 @@ rte_eth_bond_create(const char *name, uint8_t mode, uint8_t socket_id)
 	}
 
 	/* reserve an ethdev entry */
-	eth_dev = rte_eth_dev_allocate(name);
+	eth_dev = rte_eth_dev_allocate(name, RTE_ETH_DEV_VIRTUAL);
 	if (eth_dev == NULL) {
 		RTE_BOND_LOG(ERR, "Unable to allocate rte_eth_dev");
 		goto err;
diff --git a/lib/librte_pmd_pcap/rte_eth_pcap.c b/lib/librte_pmd_pcap/rte_eth_pcap.c
index f12d1e7..5da92a3 100644
--- a/lib/librte_pmd_pcap/rte_eth_pcap.c
+++ b/lib/librte_pmd_pcap/rte_eth_pcap.c
@@ -710,7 +710,7 @@ rte_pmd_init_internals(const char *name, const unsigned nb_rx_queues,
 		goto error;
 
 	/* reserve an ethdev entry */
-	*eth_dev = rte_eth_dev_allocate(name);
+	*eth_dev = rte_eth_dev_allocate(name, RTE_ETH_DEV_VIRTUAL);
 	if (*eth_dev == NULL)
 		goto error;
 
diff --git a/lib/librte_pmd_ring/rte_eth_ring.c b/lib/librte_pmd_ring/rte_eth_ring.c
index 4f1b6ed..d901053 100644
--- a/lib/librte_pmd_ring/rte_eth_ring.c
+++ b/lib/librte_pmd_ring/rte_eth_ring.c
@@ -256,7 +256,7 @@ rte_eth_from_rings(const char *name, struct rte_ring *const rx_queues[],
 		goto error;
 
 	/* reserve an ethdev entry */
-	eth_dev = rte_eth_dev_allocate(name);
+	eth_dev = rte_eth_dev_allocate(name, RTE_ETH_DEV_VIRTUAL);
 	if (eth_dev == NULL)
 		goto error;
 
diff --git a/lib/librte_pmd_xenvirt/rte_eth_xenvirt.c b/lib/librte_pmd_xenvirt/rte_eth_xenvirt.c
index 04e30c9..bc403d6 100644
--- a/lib/librte_pmd_xenvirt/rte_eth_xenvirt.c
+++ b/lib/librte_pmd_xenvirt/rte_eth_xenvirt.c
@@ -648,7 +648,7 @@ eth_dev_xenvirt_create(const char *name, const char *params,
 		goto err;
 
 	/* reserve an ethdev entry */
-	eth_dev = rte_eth_dev_allocate(name);
+	eth_dev = rte_eth_dev_allocate(name, RTE_ETH_DEV_VIRTUAL);
 	if (eth_dev == NULL)
 		goto err;
 
-- 
1.9.1

^ permalink raw reply	[flat|nested] 362+ messages in thread

* [dpdk-dev] [PATCH v4 10/11] eal/pci: Add rte_eal_dev_attach/detach() functions
  2015-01-19 10:40 ` [dpdk-dev] [PATCH v4 00/11] Port Hotplug Framework Tetsuya Mukawa
                     ` (8 preceding siblings ...)
  2015-01-19 10:40   ` [dpdk-dev] [PATCH v4 09/11] ethdev: Add one dev_type paramerter to rte_eth_dev_allocate Tetsuya Mukawa
@ 2015-01-19 10:40   ` Tetsuya Mukawa
  2015-01-21  3:49     ` Qiu, Michael
  2015-01-19 10:40   ` [dpdk-dev] [PATCH v4 11/11] eal: Enable port hotplug framework in Linux Tetsuya Mukawa
                     ` (2 subsequent siblings)
  12 siblings, 1 reply; 362+ messages in thread
From: Tetsuya Mukawa @ 2015-01-19 10:40 UTC (permalink / raw)
  To: dev

These functions are used for attaching or detaching a port.
When rte_eal_dev_attach() is called, the function tries to realize the
device name as pci address. If this is done successfully,
rte_eal_dev_attach() will attach physical device port. If not, attaches
virtual devive port.
When rte_eal_dev_detach() is called, the function gets the device type
of this port to know whether the port is came from physical or virtual.
And then specific detaching function will be called.

v4:
- Fix comment.
- Add error checking.
- Fix indent of 'if' statement.
- Change function name.

Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp>
---
 lib/librte_eal/common/eal_common_dev.c  | 273 ++++++++++++++++++++++++++++++++
 lib/librte_eal/common/eal_private.h     |  11 ++
 lib/librte_eal/common/include/rte_dev.h |  33 ++++
 lib/librte_eal/linuxapp/eal/Makefile    |   1 +
 lib/librte_eal/linuxapp/eal/eal_pci.c   |   6 +-
 5 files changed, 321 insertions(+), 3 deletions(-)

diff --git a/lib/librte_eal/common/eal_common_dev.c b/lib/librte_eal/common/eal_common_dev.c
index eae5656..828bd70 100644
--- a/lib/librte_eal/common/eal_common_dev.c
+++ b/lib/librte_eal/common/eal_common_dev.c
@@ -32,10 +32,13 @@
  *   OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
  */
 
+#include <stdio.h>
+#include <limits.h>
 #include <string.h>
 #include <inttypes.h>
 #include <sys/queue.h>
 
+#include <rte_ethdev.h>
 #include <rte_dev.h>
 #include <rte_devargs.h>
 #include <rte_debug.h>
@@ -107,3 +110,273 @@ rte_eal_dev_init(void)
 	}
 	return 0;
 }
+
+/* So far, DPDK hotplug function only supports linux */
+#ifdef ENABLE_HOTPLUG
+static void
+rte_eal_dev_invoke(struct rte_driver *driver,
+		struct rte_devargs *devargs, enum rte_eal_invoke_type type)
+{
+	if ((driver == NULL) || (devargs == NULL))
+		return;
+
+	switch (type) {
+	case RTE_EAL_INVOKE_TYPE_PROBE:
+		driver->init(devargs->virtual.drv_name, devargs->args);
+		break;
+	case RTE_EAL_INVOKE_TYPE_CLOSE:
+		driver->uninit(devargs->virtual.drv_name, devargs->args);
+		break;
+	default:
+		break;
+	}
+}
+
+static int
+rte_eal_dev_find_and_invoke(const char *name, int type)
+{
+	struct rte_devargs *devargs;
+	struct rte_driver *driver;
+
+	if (name == NULL)
+		return -EINVAL;
+
+	/* call the init function for each virtual device */
+	TAILQ_FOREACH(devargs, &devargs_list, next) {
+
+		if (devargs->type != RTE_DEVTYPE_VIRTUAL)
+			continue;
+
+		if (strncmp(name, devargs->virtual.drv_name, strlen(name)))
+			continue;
+
+		TAILQ_FOREACH(driver, &dev_driver_list, next) {
+			if (driver->type != PMD_VDEV)
+				continue;
+
+			/* search a driver prefix in virtual device name */
+			if (!strncmp(driver->name, devargs->virtual.drv_name,
+			    strlen(driver->name))) {
+				rte_eal_dev_invoke(driver, devargs, type);
+				break;
+			}
+		}
+
+		if (driver == NULL) {
+			RTE_LOG(WARNING, EAL, "no driver found for %s\n",
+				  devargs->virtual.drv_name);
+		}
+		return 0;
+	}
+	return 1;
+}
+
+/* attach the new physical device, then store port_id of the device */
+static int
+rte_eal_dev_attach_pdev(struct rte_pci_addr *addr, uint8_t *port_id)
+{
+	uint8_t new_port_id;
+	struct rte_eth_dev devs[RTE_MAX_ETHPORTS];
+
+	if ((addr == NULL) || (port_id == NULL))
+		goto err;
+
+	/* save current port status */
+	rte_eth_dev_save(devs);
+	/* re-construct pci_device_list */
+	if (rte_eal_pci_scan())
+		goto err;
+	/* invoke probe func of the driver can handle the new device */
+	if (rte_eal_pci_probe_one(addr))
+		goto err;
+	/* get port_id enabled by above procedures */
+	if (rte_eth_dev_get_changed_port(devs, &new_port_id))
+		goto err;
+
+	*port_id = new_port_id;
+	return 0;
+err:
+	RTE_LOG(ERR, EAL, "Drver, cannot attach the device\n");
+	return -1;
+}
+
+/* detach the new physical device, then store pci_addr of the device */
+static int
+rte_eal_dev_detach_pdev(uint8_t port_id, struct rte_pci_addr *addr)
+{
+	struct rte_pci_addr freed_addr;
+	struct rte_pci_addr vp;
+
+	if (addr == NULL)
+		goto err;
+
+	/* check whether the driver supports detach feature, or not */
+	if (rte_eth_dev_check_detachable(port_id))
+		goto err;
+
+	/* get pci address by port id */
+	if (rte_eth_dev_get_addr_by_port(port_id, &freed_addr))
+		goto err;
+
+	/* Zerod pci addr means the port comes from virtual device */
+	vp.domain = vp.bus = vp.devid = vp.function = 0;
+	if (eal_compare_pci_addr(&vp, &freed_addr) == 0)
+		goto err;
+
+	/* invoke close func of the driver,
+	 * also remove the device from pci_device_list */
+	if (rte_eal_pci_close_one(&freed_addr))
+		goto err;
+
+	*addr = freed_addr;
+	return 0;
+err:
+	RTE_LOG(ERR, EAL, "Drver, cannot detach the device\n");
+	return -1;
+}
+
+static void
+get_vdev_name(char *vdevargs)
+{
+	char *sep;
+
+	if (vdevargs == NULL)
+		return;
+
+	/* set the first ',' to '\0' to split name and arguments */
+	sep = strchr(vdevargs, ',');
+	if (sep != NULL)
+		sep[0] = '\0';
+}
+
+/* attach the new virtual device, then store port_id of the device */
+static int
+rte_eal_dev_attach_vdev(const char *vdevargs, uint8_t *port_id)
+{
+	char *args;
+	uint8_t new_port_id;
+	struct rte_eth_dev devs[RTE_MAX_ETHPORTS];
+
+	if ((vdevargs == NULL) || (port_id == NULL))
+		goto err0;
+
+	args = strdup(vdevargs);
+	if (args == NULL)
+		goto err0;
+
+	/* save current port status */
+	rte_eth_dev_save(devs);
+	/* add the vdevargs to devargs_list */
+	if (rte_eal_devargs_add(RTE_DEVTYPE_VIRTUAL, args))
+		goto err1;
+	/* parse vdevargs, then retrieve device name */
+	get_vdev_name(args);
+	/* walk around dev_driver_list to find the driver of the device,
+	 * then invoke probe function o the driver */
+	if (rte_eal_dev_find_and_invoke(args, RTE_EAL_INVOKE_TYPE_PROBE))
+		goto err2;
+	/* get port_id enabled by above procedures */
+	if (rte_eth_dev_get_changed_port(devs, &new_port_id))
+		goto err2;
+
+	free(args);
+	*port_id = new_port_id;
+	return 0;
+err2:
+	rte_eal_devargs_remove(RTE_DEVTYPE_VIRTUAL, args);
+err1:
+	free(args);
+err0:
+	RTE_LOG(ERR, EAL, "Drver, cannot detach the device\n");
+	return -1;
+}
+
+/* detach the new virtual device, then store the name of the device */
+static int
+rte_eal_dev_detach_vdev(uint8_t port_id, char *vdevname)
+{
+	char name[RTE_ETH_NAME_MAX_LEN];
+
+	if (vdevname == NULL)
+		goto err;
+
+	/* check whether the driver supports detach feature, or not */
+	if (rte_eth_dev_check_detachable(port_id))
+		goto err;
+
+	/* get device name by port id */
+	if (rte_eth_dev_get_name_by_port(port_id, name))
+		goto err;
+	/* walk around dev_driver_list to find the driver of the device,
+	 * then invoke close function o the driver */
+	if (rte_eal_dev_find_and_invoke(name, RTE_EAL_INVOKE_TYPE_CLOSE))
+		goto err;
+	/* remove the vdevname from devargs_list */
+	rte_eal_devargs_remove(RTE_DEVTYPE_VIRTUAL, name);
+
+	strncpy(vdevname, name, sizeof(name));
+	return 0;
+err:
+	RTE_LOG(ERR, EAL, "Drver, cannot detach the device\n");
+	return -1;
+}
+
+/* attach the new device, then store port_id of the device */
+int
+rte_eal_dev_attach(const char *devargs, uint8_t *port_id)
+{
+	struct rte_pci_addr addr;
+
+	if ((devargs == NULL) || (port_id == NULL))
+		return -EINVAL;
+
+	if (eal_parse_pci_DomBDF(devargs, &addr) == 0)
+		return rte_eal_dev_attach_pdev(&addr, port_id);
+	else
+		return rte_eal_dev_attach_vdev(devargs, port_id);
+}
+
+/* detach the device, then store the name of the device */
+int
+rte_eal_dev_detach(uint8_t port_id, char *name)
+{
+	struct rte_pci_addr addr;
+	int ret;
+
+	if (name == NULL)
+		return -EINVAL;
+
+	if (rte_eth_dev_get_device_type(port_id) == RTE_ETH_DEV_PHYSICAL) {
+		ret = rte_eth_dev_get_addr_by_port(port_id, &addr);
+		if (ret < 0)
+			return ret;
+
+		ret = rte_eal_dev_detach_pdev(port_id, &addr);
+		if (ret == 0)
+			snprintf(name, RTE_ETH_NAME_MAX_LEN,
+				"%04x.%02x.%02x.%d",
+				addr.domain, addr.bus,
+				addr.devid, addr.function);
+
+		return ret;
+	} else
+		return rte_eal_dev_detach_vdev(port_id, name);
+}
+#else /* ENABLE_HOTPLUG */
+int
+rte_eal_dev_attach(const char *devargs __rte_unused,
+			uint8_t *port_id __rte_unused)
+{
+	RTE_LOG(ERR, EAL, "Hotplug support isn't enabled\n");
+	return -1;
+}
+
+/* detach the device, then store the name of the device */
+int
+rte_eal_dev_detach(uint8_t port_id __rte_unused,
+			char *name __rte_unused)
+{
+	RTE_LOG(ERR, EAL, "Hotplug support isn't enabled\n");
+	return -1;
+}
+#endif /* ENABLE_HOTPLUG */
diff --git a/lib/librte_eal/common/eal_private.h b/lib/librte_eal/common/eal_private.h
index a97c5d8..453a1eb 100644
--- a/lib/librte_eal/common/eal_private.h
+++ b/lib/librte_eal/common/eal_private.h
@@ -164,6 +164,17 @@ enum rte_eal_invoke_type {
 };
 
 /**
+ * Scan the content of the PCI bus, and the devices in the devices
+ * list
+ *
+ * This function is private to EAL.
+ *
+ * @return
+ *  0 on success, negative on error
+ */
+int rte_eal_pci_scan(void);
+
+/**
  * Mmap memory for single PCI device
  *
  * This function is private to EAL.
diff --git a/lib/librte_eal/common/include/rte_dev.h b/lib/librte_eal/common/include/rte_dev.h
index f7e3a10..e63dd1c 100644
--- a/lib/librte_eal/common/include/rte_dev.h
+++ b/lib/librte_eal/common/include/rte_dev.h
@@ -47,6 +47,7 @@ extern "C" {
 #endif
 
 #include <sys/queue.h>
+#include <rte_pci.h>
 
 /** Double linked list of device drivers. */
 TAILQ_HEAD(rte_driver_list, rte_driver);
@@ -57,6 +58,11 @@ TAILQ_HEAD(rte_driver_list, rte_driver);
 typedef int (rte_dev_init_t)(const char *name, const char *args);
 
 /**
+ * Uninitilization function called for each device driver once.
+ */
+typedef int (rte_dev_uninit_t)(const char *name, const char *args);
+
+/**
  * Driver type enumeration
  */
 enum pmd_type {
@@ -72,6 +78,7 @@ struct rte_driver {
 	enum pmd_type type;		   /**< PMD Driver type */
 	const char *name;                   /**< Driver name. */
 	rte_dev_init_t *init;              /**< Device init. function. */
+	rte_dev_uninit_t *uninit;          /**< Device uninit. function. */
 };
 
 /**
@@ -93,6 +100,32 @@ void rte_eal_driver_register(struct rte_driver *driver);
 void rte_eal_driver_unregister(struct rte_driver *driver);
 
 /**
+ * Attach a new device.
+ *
+ * @param devargs
+ *   A pointer to a strings array describing the new device
+ *   to be attached. The strings should be a pci address like
+ *   '0000:01:00.0' or virtual device name like 'eth_pcap0'.
+ * @param port_id
+ *  A pointer to a port identifier actually attached.
+ * @return
+ *  0 on success and port_id is filled, negative on error
+ */
+int rte_eal_dev_attach(const char *devargs, uint8_t *port_id);
+
+/**
+ * Detach a device.
+ *
+ * @param port_id
+ *   The port identifier of the device to detach.
+ * @param addr
+ *  A pointer to a device name actually detached.
+ * @return
+ *  0 on success and devname is filled, negative on error
+ */
+int rte_eal_dev_detach(uint8_t port_id, char *devname);
+
+/**
  * Initalize all the registered drivers in this process
  */
 int rte_eal_dev_init(void);
diff --git a/lib/librte_eal/linuxapp/eal/Makefile b/lib/librte_eal/linuxapp/eal/Makefile
index 72ecf3a..0ec83b5 100644
--- a/lib/librte_eal/linuxapp/eal/Makefile
+++ b/lib/librte_eal/linuxapp/eal/Makefile
@@ -41,6 +41,7 @@ CFLAGS += -I$(RTE_SDK)/lib/librte_eal/common/include
 CFLAGS += -I$(RTE_SDK)/lib/librte_ring
 CFLAGS += -I$(RTE_SDK)/lib/librte_mempool
 CFLAGS += -I$(RTE_SDK)/lib/librte_malloc
+CFLAGS += -I$(RTE_SDK)/lib/librte_mbuf
 CFLAGS += -I$(RTE_SDK)/lib/librte_ether
 CFLAGS += -I$(RTE_SDK)/lib/librte_ivshmem
 CFLAGS += -I$(RTE_SDK)/lib/librte_pmd_ring
diff --git a/lib/librte_eal/linuxapp/eal/eal_pci.c b/lib/librte_eal/linuxapp/eal/eal_pci.c
index a23cc59..8e7e650 100644
--- a/lib/librte_eal/linuxapp/eal/eal_pci.c
+++ b/lib/librte_eal/linuxapp/eal/eal_pci.c
@@ -378,8 +378,8 @@ error:
  * Scan the content of the PCI bus, and the devices in the devices
  * list
  */
-static int
-pci_scan(void)
+int
+rte_eal_pci_scan(void)
 {
 	struct dirent *e;
 	DIR *dir;
@@ -701,7 +701,7 @@ rte_eal_pci_init(void)
 	if (internal_config.no_pci)
 		return 0;
 
-	if (pci_scan() < 0) {
+	if (rte_eal_pci_scan() < 0) {
 		RTE_LOG(ERR, EAL, "%s(): Cannot scan PCI bus\n", __func__);
 		return -1;
 	}
-- 
1.9.1

^ permalink raw reply	[flat|nested] 362+ messages in thread

* [dpdk-dev] [PATCH v4 11/11] eal: Enable port hotplug framework in Linux
  2015-01-19 10:40 ` [dpdk-dev] [PATCH v4 00/11] Port Hotplug Framework Tetsuya Mukawa
                     ` (9 preceding siblings ...)
  2015-01-19 10:40   ` [dpdk-dev] [PATCH v4 10/11] eal/pci: Add rte_eal_dev_attach/detach() functions Tetsuya Mukawa
@ 2015-01-19 10:40   ` Tetsuya Mukawa
  2015-01-19 13:15   ` [dpdk-dev] [PATCH v4 00/11] Port Hotplug Framework Qiu, Michael
  2015-01-27  3:00   ` Qiu, Michael
  12 siblings, 0 replies; 362+ messages in thread
From: Tetsuya Mukawa @ 2015-01-19 10:40 UTC (permalink / raw)
  To: dev

The patch enables CONFIG_RTE_LIBRTE_EAL_HOTPLUG in Linux configuration.

Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp>
---
 config/common_linuxapp | 5 +++++
 1 file changed, 5 insertions(+)

diff --git a/config/common_linuxapp b/config/common_linuxapp
index 2f9643b..27d05be 100644
--- a/config/common_linuxapp
+++ b/config/common_linuxapp
@@ -114,6 +114,11 @@ CONFIG_RTE_PCI_MAX_READ_REQUEST_SIZE=0
 CONFIG_RTE_LIBRTE_EAL_LINUXAPP=y
 
 #
+# Compile Environment Abstraction Layer to support hotplug
+#
+CONFIG_RTE_LIBRTE_EAL_HOTPLUG=y
+
+#
 # Compile Environment Abstraction Layer to support Vmware TSC map
 #
 CONFIG_RTE_LIBRTE_EAL_VMWARE_TSC_MAP_SUPPORT=y
-- 
1.9.1

^ permalink raw reply	[flat|nested] 362+ messages in thread

* [dpdk-dev] [PATCH v4] librte_pmd_pcap: Add port hotplug support
       [not found] <1418106629-22227-2-git-send-email-mukawa@igel.co.j>
  2015-01-19 10:40 ` [dpdk-dev] [PATCH v4 00/11] Port Hotplug Framework Tetsuya Mukawa
@ 2015-01-19 10:41 ` Tetsuya Mukawa
  2015-01-19 10:42 ` [dpdk-dev] [PATCH v4] testpmd: " Tetsuya Mukawa
  2 siblings, 0 replies; 362+ messages in thread
From: Tetsuya Mukawa @ 2015-01-19 10:41 UTC (permalink / raw)
  To: dev

This patch adds finalization code to free resources allocated by the
PMD.

v4:
 - Change function name.

Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp>
---
 lib/librte_pmd_pcap/rte_eth_pcap.c | 40 ++++++++++++++++++++++++++++++++++++++
 1 file changed, 40 insertions(+)

diff --git a/lib/librte_pmd_pcap/rte_eth_pcap.c b/lib/librte_pmd_pcap/rte_eth_pcap.c
index 5da92a3..13a68de 100644
--- a/lib/librte_pmd_pcap/rte_eth_pcap.c
+++ b/lib/librte_pmd_pcap/rte_eth_pcap.c
@@ -499,6 +499,13 @@ static struct eth_dev_ops ops = {
 		.stats_reset = eth_stats_reset,
 };
 
+static struct eth_driver rte_pcap_pmd = {
+	.pci_drv = {
+		.name = "rte_pcap_pmd",
+		.drv_flags = RTE_PCI_DRV_DETACHABLE,
+	},
+};
+
 /*
  * Function handler that opens the pcap file for reading a stores a
  * reference of it for use it later on.
@@ -714,6 +721,10 @@ rte_pmd_init_internals(const char *name, const unsigned nb_rx_queues,
 	if (*eth_dev == NULL)
 		goto error;
 
+	/* check length of device name */
+	if ((strlen((*eth_dev)->data->name) + 1) > sizeof(data->name))
+		goto error;
+
 	/* now put it all together
 	 * - store queue data in internals,
 	 * - store numa_node info in pci_driver
@@ -739,10 +750,13 @@ rte_pmd_init_internals(const char *name, const unsigned nb_rx_queues,
 	data->nb_tx_queues = (uint16_t)nb_tx_queues;
 	data->dev_link = pmd_link;
 	data->mac_addrs = &eth_addr;
+	strncpy(data->name,
+		(*eth_dev)->data->name, strlen((*eth_dev)->data->name));
 
 	(*eth_dev)->data = data;
 	(*eth_dev)->dev_ops = &ops;
 	(*eth_dev)->pci_dev = pci_dev;
+	(*eth_dev)->driver = &rte_pcap_pmd;
 
 	return 0;
 
@@ -927,10 +941,36 @@ rte_pmd_pcap_devinit(const char *name, const char *params)
 
 }
 
+static int
+rte_pmd_pcap_devuninit(const char *name, const char *params __rte_unused)
+{
+	struct rte_eth_dev *eth_dev = NULL;
+
+	RTE_LOG(INFO, PMD, "Closing pcap ethdev on numa socket %u\n",
+			rte_socket_id());
+
+	if (name == NULL)
+		return -1;
+
+	/* reserve an ethdev entry */
+	eth_dev = rte_eth_dev_allocated(name);
+	if (eth_dev == NULL)
+		return -1;
+
+	rte_free(eth_dev->data->dev_private);
+	rte_free(eth_dev->data);
+	rte_free(eth_dev->pci_dev);
+
+	rte_eth_dev_free(name);
+
+	return 0;
+}
+
 static struct rte_driver pmd_pcap_drv = {
 	.name = "eth_pcap",
 	.type = PMD_VDEV,
 	.init = rte_pmd_pcap_devinit,
+	.uninit = rte_pmd_pcap_devuninit,
 };
 
 PMD_REGISTER_DRIVER(pmd_pcap_drv);
-- 
1.9.1

^ permalink raw reply	[flat|nested] 362+ messages in thread

* [dpdk-dev] [PATCH v4] testpmd: Add port hotplug support
       [not found] <1418106629-22227-2-git-send-email-mukawa@igel.co.j>
  2015-01-19 10:40 ` [dpdk-dev] [PATCH v4 00/11] Port Hotplug Framework Tetsuya Mukawa
  2015-01-19 10:41 ` [dpdk-dev] [PATCH v4] librte_pmd_pcap: Add port hotplug support Tetsuya Mukawa
@ 2015-01-19 10:42 ` Tetsuya Mukawa
  2 siblings, 0 replies; 362+ messages in thread
From: Tetsuya Mukawa @ 2015-01-19 10:42 UTC (permalink / raw)
  To: dev

The patch introduces following commands.
- port attach [ident]
- port detach [port_id]
 - attach: attaching a port
 - detach: detaching a port
 - ident: pci address of physical device.
          Or device name and paramerters of virtual device.
         (ex. 0000:02:00.0, eth_pcap0,iface=eth0)
 - port_id: port identifier

v4:
 - Fix strings of command help.

Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp>
---
 app/test-pmd/cmdline.c    | 133 +++++++++++++++++++++++++------
 app/test-pmd/config.c     | 116 +++++++++++++++------------
 app/test-pmd/parameters.c |  22 +++--
 app/test-pmd/testpmd.c    | 199 +++++++++++++++++++++++++++++++++++-----------
 app/test-pmd/testpmd.h    |  18 ++++-
 5 files changed, 358 insertions(+), 130 deletions(-)

diff --git a/app/test-pmd/cmdline.c b/app/test-pmd/cmdline.c
index 4618b92..3557dff 100644
--- a/app/test-pmd/cmdline.c
+++ b/app/test-pmd/cmdline.c
@@ -572,6 +572,12 @@ static void cmd_help_long_parsed(void *parsed_result,
 			"port close (port_id|all)\n"
 			"    Close all ports or port_id.\n\n"
 
+			"port attach (ident)\n"
+			"    Attach physical or virtual dev by pci address or virtual device name\n\n"
+
+			"port detach (port_id)\n"
+			"    Detach physical or virtual dev by port_id\n\n"
+
 			"port config (port_id|all)"
 			" speed (10|100|1000|10000|40000|auto)"
 			" duplex (half|full|auto)\n"
@@ -853,6 +859,89 @@ cmdline_parse_inst_t cmd_operate_specific_port = {
 	},
 };
 
+/* *** attach a specificied port *** */
+struct cmd_operate_attach_port_result {
+	cmdline_fixed_string_t port;
+	cmdline_fixed_string_t keyword;
+	cmdline_fixed_string_t identifier;
+};
+
+static void cmd_operate_attach_port_parsed(void *parsed_result,
+				__attribute__((unused)) struct cmdline *cl,
+				__attribute__((unused)) void *data)
+{
+	struct cmd_operate_attach_port_result *res = parsed_result;
+
+	if (!strcmp(res->keyword, "attach"))
+		attach_port(res->identifier);
+	else
+		printf("Unknown parameter\n");
+}
+
+cmdline_parse_token_string_t cmd_operate_attach_port_port =
+	TOKEN_STRING_INITIALIZER(struct cmd_operate_attach_port_result,
+			port, "port");
+cmdline_parse_token_string_t cmd_operate_attach_port_keyword =
+	TOKEN_STRING_INITIALIZER(struct cmd_operate_attach_port_result,
+			keyword, "attach");
+cmdline_parse_token_string_t cmd_operate_attach_port_identifier =
+	TOKEN_STRING_INITIALIZER(struct cmd_operate_attach_port_result,
+			identifier, NULL);
+
+cmdline_parse_inst_t cmd_operate_attach_port = {
+	.f = cmd_operate_attach_port_parsed,
+	.data = NULL,
+	.help_str = "port attach identifier, "
+		"identifier: pci address or virtual dev name",
+	.tokens = {
+		(void *)&cmd_operate_attach_port_port,
+		(void *)&cmd_operate_attach_port_keyword,
+		(void *)&cmd_operate_attach_port_identifier,
+		NULL,
+	},
+};
+
+/* *** detach a specificied port *** */
+struct cmd_operate_detach_port_result {
+	cmdline_fixed_string_t port;
+	cmdline_fixed_string_t keyword;
+	uint8_t port_id;
+};
+
+static void cmd_operate_detach_port_parsed(void *parsed_result,
+				__attribute__((unused)) struct cmdline *cl,
+				__attribute__((unused)) void *data)
+{
+	struct cmd_operate_detach_port_result *res = parsed_result;
+
+	if (!strcmp(res->keyword, "detach"))
+		detach_port(res->port_id);
+	else
+		printf("Unknown parameter\n");
+}
+
+cmdline_parse_token_string_t cmd_operate_detach_port_port =
+	TOKEN_STRING_INITIALIZER(struct cmd_operate_detach_port_result,
+			port, "port");
+cmdline_parse_token_string_t cmd_operate_detach_port_keyword =
+	TOKEN_STRING_INITIALIZER(struct cmd_operate_detach_port_result,
+			keyword, "detach");
+cmdline_parse_token_num_t cmd_operate_detach_port_port_id =
+	TOKEN_NUM_INITIALIZER(struct cmd_operate_detach_port_result,
+			port_id, UINT8);
+
+cmdline_parse_inst_t cmd_operate_detach_port = {
+	.f = cmd_operate_detach_port_parsed,
+	.data = NULL,
+	.help_str = "port detach port_id",
+	.tokens = {
+		(void *)&cmd_operate_detach_port_port,
+		(void *)&cmd_operate_detach_port_keyword,
+		(void *)&cmd_operate_detach_port_port_id,
+		NULL,
+	},
+};
+
 /* *** configure speed for all ports *** */
 struct cmd_config_speed_all {
 	cmdline_fixed_string_t port;
@@ -907,7 +996,7 @@ cmd_config_speed_all_parsed(void *parsed_result,
 		return;
 	}
 
-	for (pid = 0; pid < nb_ports; pid++) {
+	FOREACH_PORT(pid, ports) {
 		ports[pid].dev_conf.link_speed = link_speed;
 		ports[pid].dev_conf.link_duplex = link_duplex;
 	}
@@ -975,10 +1064,8 @@ cmd_config_speed_specific_parsed(void *parsed_result,
 		return;
 	}
 
-	if (res->id >= nb_ports) {
-		printf("Port id %d must be less than %d\n", res->id, nb_ports);
+	if (port_id_is_invalid(res->id, ENABLED_WARN))
 		return;
-	}
 
 	if (!strcmp(res->value1, "10"))
 		link_speed = ETH_LINK_SPEED_10;
@@ -1538,7 +1625,7 @@ cmd_config_rxtx_queue_parsed(void *parsed_result,
 		return;
 	}
 
-	if (port_id_is_invalid(res->portid))
+	if (port_id_is_invalid(res->portid, ENABLED_WARN))
 		return;
 
 	if (port_is_started(res->portid) != 1) {
@@ -2881,7 +2968,7 @@ cmd_tx_cksum_parsed(void *parsed_result,
 	uint16_t ol_flags, mask = 0;
 	struct rte_eth_dev_info dev_info;
 
-	if (port_id_is_invalid(res->port_id)) {
+	if (port_id_is_invalid(res->port_id, ENABLED_WARN)) {
 		printf("invalid port %d\n", res->port_id);
 		return;
 	}
@@ -3008,7 +3095,7 @@ cmd_tso_set_parsed(void *parsed_result,
 	struct cmd_tso_set_result *res = parsed_result;
 	struct rte_eth_dev_info dev_info;
 
-	if (port_id_is_invalid(res->port_id))
+	if (port_id_is_invalid(res->port_id, ENABLED_WARN))
 		return;
 
 	if (!strcmp(res->mode, "set"))
@@ -3984,10 +4071,8 @@ static void cmd_set_bond_mac_addr_parsed(void *parsed_result,
 	struct cmd_set_bond_mac_addr_result *res = parsed_result;
 	int ret;
 
-	if (res->port_num >= nb_ports) {
-		printf("Port id %d must be less than %d\n", res->port_num, nb_ports);
+	if (port_id_is_invalid(res->port_num, ENABLED_WARN))
 		return;
-	}
 
 	ret = rte_eth_bond_mac_address_set(res->port_num, &res->address);
 
@@ -4224,7 +4309,7 @@ static void cmd_set_promisc_mode_parsed(void *parsed_result,
 
 	/* all ports */
 	if (allports) {
-		for (i = 0; i < nb_ports; i++) {
+		FOREACH_PORT(i, ports) {
 			if (enable)
 				rte_eth_promiscuous_enable(i);
 			else
@@ -4304,7 +4389,7 @@ static void cmd_set_allmulti_mode_parsed(void *parsed_result,
 
 	/* all ports */
 	if (allports) {
-		for (i = 0; i < nb_ports; i++) {
+		FOREACH_PORT(i, ports) {
 			if (enable)
 				rte_eth_allmulticast_enable(i);
 			else
@@ -5489,25 +5574,25 @@ static void cmd_showportall_parsed(void *parsed_result,
 	struct cmd_showportall_result *res = parsed_result;
 	if (!strcmp(res->show, "clear")) {
 		if (!strcmp(res->what, "stats"))
-			for (i = 0; i < nb_ports; i++)
+			FOREACH_PORT(i, ports)
 				nic_stats_clear(i);
 		else if (!strcmp(res->what, "xstats"))
-			for (i = 0; i < nb_ports; i++)
+			FOREACH_PORT(i, ports)
 				nic_xstats_clear(i);
 	} else if (!strcmp(res->what, "info"))
-		for (i = 0; i < nb_ports; i++)
+		FOREACH_PORT(i, ports)
 			port_infos_display(i);
 	else if (!strcmp(res->what, "stats"))
-		for (i = 0; i < nb_ports; i++)
+		FOREACH_PORT(i, ports)
 			nic_stats_display(i);
 	else if (!strcmp(res->what, "xstats"))
-		for (i = 0; i < nb_ports; i++)
+		FOREACH_PORT(i, ports)
 			nic_xstats_display(i);
 	else if (!strcmp(res->what, "fdir"))
-		for (i = 0; i < nb_ports; i++)
+		FOREACH_PORT(i, ports)
 			fdir_get_infos(i);
 	else if (!strcmp(res->what, "stat_qmap"))
-		for (i = 0; i < nb_ports; i++)
+		FOREACH_PORT(i, ports)
 			nic_stats_mapping_display(i);
 }
 
@@ -8783,6 +8868,8 @@ cmdline_parse_ctx_t main_ctx[] = {
 	(cmdline_parse_inst_t *)&cmd_set_qmap,
 	(cmdline_parse_inst_t *)&cmd_operate_port,
 	(cmdline_parse_inst_t *)&cmd_operate_specific_port,
+	(cmdline_parse_inst_t *)&cmd_operate_attach_port,
+	(cmdline_parse_inst_t *)&cmd_operate_detach_port,
 	(cmdline_parse_inst_t *)&cmd_config_speed_all,
 	(cmdline_parse_inst_t *)&cmd_config_speed_specific,
 	(cmdline_parse_inst_t *)&cmd_config_rx_tx,
@@ -8859,7 +8946,7 @@ prompt(void)
 static void
 cmd_reconfig_device_queue(portid_t id, uint8_t dev, uint8_t queue)
 {
-	if (id < nb_ports) {
+	if (!port_id_is_invalid(id, DISABLED_WARN)) {
 		/* check if need_reconfig has been set to 1 */
 		if (ports[id].need_reconfig == 0)
 			ports[id].need_reconfig = dev;
@@ -8869,7 +8956,7 @@ cmd_reconfig_device_queue(portid_t id, uint8_t dev, uint8_t queue)
 	} else {
 		portid_t pid;
 
-		for (pid = 0; pid < nb_ports; pid++) {
+		FOREACH_PORT(pid, ports) {
 			/* check if need_reconfig has been set to 1 */
 			if (ports[pid].need_reconfig == 0)
 				ports[pid].need_reconfig = dev;
@@ -8887,10 +8974,8 @@ bypass_is_supported(portid_t port_id)
 	struct rte_port   *port;
 	struct rte_pci_id *pci_id;
 
-	if (port_id >= nb_ports) {
-		printf("\tPort id must be less than %d.\n", nb_ports);
+	if (port_id_is_invalid(port_id, ENABLED_WARN))
 		return 0;
-	}
 
 	/* Get the device id. */
 	port    = &ports[port_id];
diff --git a/app/test-pmd/config.c b/app/test-pmd/config.c
index 97b6525..2662d7c 100644
--- a/app/test-pmd/config.c
+++ b/app/test-pmd/config.c
@@ -124,11 +124,15 @@ nic_stats_display(portid_t port_id)
 	struct rte_eth_stats stats;
 	struct rte_port *port = &ports[port_id];
 	uint8_t i;
+	portid_t pid;
 
 	static const char *nic_stats_border = "########################";
 
-	if (port_id >= nb_ports) {
-		printf("Invalid port, range is [0, %d]\n", nb_ports - 1);
+	if (port_id_is_invalid(port_id, ENABLED_WARN)) {
+		printf("Valid port range is [0");
+		FOREACH_PORT(pid, ports)
+			printf(", %d", pid);
+		printf("]\n");
 		return;
 	}
 	rte_eth_stats_get(port_id, &stats);
@@ -201,8 +205,13 @@ nic_stats_display(portid_t port_id)
 void
 nic_stats_clear(portid_t port_id)
 {
-	if (port_id >= nb_ports) {
-		printf("Invalid port, range is [0, %d]\n", nb_ports - 1);
+	portid_t pid;
+
+	if (port_id_is_invalid(port_id, ENABLED_WARN)) {
+		printf("Valid port range is [0");
+		FOREACH_PORT(pid, ports)
+			printf(", %d", pid);
+		printf("]\n");
 		return;
 	}
 	rte_eth_stats_reset(port_id);
@@ -249,11 +258,15 @@ nic_stats_mapping_display(portid_t port_id)
 {
 	struct rte_port *port = &ports[port_id];
 	uint16_t i;
+	portid_t pid;
 
 	static const char *nic_stats_mapping_border = "########################";
 
-	if (port_id >= nb_ports) {
-		printf("Invalid port, range is [0, %d]\n", nb_ports - 1);
+	if (port_id_is_invalid(port_id, ENABLED_WARN)) {
+		printf("Valid port range is [0");
+		FOREACH_PORT(pid, ports)
+			printf(", %d", pid);
+		printf("]\n");
 		return;
 	}
 
@@ -302,9 +315,13 @@ port_infos_display(portid_t port_id)
 	int vlan_offload;
 	struct rte_mempool * mp;
 	static const char *info_border = "*********************";
+	portid_t pid;
 
-	if (port_id >= nb_ports) {
-		printf("Invalid port, range is [0, %d]\n", nb_ports - 1);
+	if (port_id_is_invalid(port_id, ENABLED_WARN)) {
+		printf("Valid port range is [0");
+		FOREACH_PORT(pid, ports)
+			printf(", %d", pid);
+		printf("]\n");
 		return;
 	}
 	port = &ports[port_id];
@@ -362,11 +379,14 @@ port_infos_display(portid_t port_id)
 }
 
 int
-port_id_is_invalid(portid_t port_id)
+port_id_is_invalid(portid_t port_id, enum print_warning warning)
 {
-	if (port_id < nb_ports)
+	if (ports[port_id].enabled)
 		return 0;
-	printf("Invalid port %d (must be < nb_ports=%d)\n", port_id, nb_ports);
+
+	if (warning == ENABLED_WARN)
+		printf("Invalid port %d\n", port_id);
+
 	return 1;
 }
 
@@ -425,7 +445,7 @@ port_reg_bit_display(portid_t port_id, uint32_t reg_off, uint8_t bit_x)
 	uint32_t reg_v;
 
 
-	if (port_id_is_invalid(port_id))
+	if (port_id_is_invalid(port_id, ENABLED_WARN))
 		return;
 	if (port_reg_off_is_invalid(port_id, reg_off))
 		return;
@@ -444,7 +464,7 @@ port_reg_bit_field_display(portid_t port_id, uint32_t reg_off,
 	uint8_t  l_bit;
 	uint8_t  h_bit;
 
-	if (port_id_is_invalid(port_id))
+	if (port_id_is_invalid(port_id, ENABLED_WARN))
 		return;
 	if (port_reg_off_is_invalid(port_id, reg_off))
 		return;
@@ -471,7 +491,7 @@ port_reg_display(portid_t port_id, uint32_t reg_off)
 {
 	uint32_t reg_v;
 
-	if (port_id_is_invalid(port_id))
+	if (port_id_is_invalid(port_id, ENABLED_WARN))
 		return;
 	if (port_reg_off_is_invalid(port_id, reg_off))
 		return;
@@ -485,7 +505,7 @@ port_reg_bit_set(portid_t port_id, uint32_t reg_off, uint8_t bit_pos,
 {
 	uint32_t reg_v;
 
-	if (port_id_is_invalid(port_id))
+	if (port_id_is_invalid(port_id, ENABLED_WARN))
 		return;
 	if (port_reg_off_is_invalid(port_id, reg_off))
 		return;
@@ -513,7 +533,7 @@ port_reg_bit_field_set(portid_t port_id, uint32_t reg_off,
 	uint8_t  l_bit;
 	uint8_t  h_bit;
 
-	if (port_id_is_invalid(port_id))
+	if (port_id_is_invalid(port_id, ENABLED_WARN))
 		return;
 	if (port_reg_off_is_invalid(port_id, reg_off))
 		return;
@@ -547,7 +567,7 @@ port_reg_bit_field_set(portid_t port_id, uint32_t reg_off,
 void
 port_reg_set(portid_t port_id, uint32_t reg_off, uint32_t reg_v)
 {
-	if (port_id_is_invalid(port_id))
+	if (port_id_is_invalid(port_id, ENABLED_WARN))
 		return;
 	if (port_reg_off_is_invalid(port_id, reg_off))
 		return;
@@ -560,7 +580,7 @@ port_mtu_set(portid_t port_id, uint16_t mtu)
 {
 	int diag;
 
-	if (port_id_is_invalid(port_id))
+	if (port_id_is_invalid(port_id, ENABLED_WARN))
 		return;
 	diag = rte_eth_dev_set_mtu(port_id, mtu);
 	if (diag == 0)
@@ -723,7 +743,7 @@ rx_ring_desc_display(portid_t port_id, queueid_t rxq_id, uint16_t rxd_id)
 {
 	const struct rte_memzone *rx_mz;
 
-	if (port_id_is_invalid(port_id))
+	if (port_id_is_invalid(port_id, ENABLED_WARN))
 		return;
 	if (rx_queue_id_is_invalid(rxq_id))
 		return;
@@ -740,7 +760,7 @@ tx_ring_desc_display(portid_t port_id, queueid_t txq_id, uint16_t txd_id)
 {
 	const struct rte_memzone *tx_mz;
 
-	if (port_id_is_invalid(port_id))
+	if (port_id_is_invalid(port_id, ENABLED_WARN))
 		return;
 	if (tx_queue_id_is_invalid(txq_id))
 		return;
@@ -796,7 +816,7 @@ port_rss_reta_info(portid_t port_id,
 	uint16_t i, idx, shift;
 	int ret;
 
-	if (port_id_is_invalid(port_id))
+	if (port_id_is_invalid(port_id, ENABLED_WARN))
 		return;
 
 	ret = rte_eth_dev_rss_reta_query(port_id, reta_conf, nb_entries);
@@ -828,7 +848,7 @@ port_rss_hash_conf_show(portid_t port_id, int show_rss_key)
 	uint8_t i;
 	int diag;
 
-	if (port_id_is_invalid(port_id))
+	if (port_id_is_invalid(port_id, ENABLED_WARN))
 		return;
 	/* Get RSS hash key if asked to display it */
 	rss_conf.rss_key = (show_rss_key) ? rss_key : NULL;
@@ -1406,12 +1426,8 @@ set_fwd_ports_list(unsigned int *portlist, unsigned int nb_pt)
  again:
 	for (i = 0; i < nb_pt; i++) {
 		port_id = (portid_t) portlist[i];
-		if (port_id >= nb_ports) {
-			printf("Invalid port id %u >= %u\n",
-			       (unsigned int) port_id,
-			       (unsigned int) nb_ports);
+		if (port_id_is_invalid(port_id, ENABLED_WARN))
 			return;
-		}
 		if (record_now)
 			fwd_ports_ids[i] = port_id;
 	}
@@ -1569,7 +1585,7 @@ vlan_extend_set(portid_t port_id, int on)
 	int diag;
 	int vlan_offload;
 
-	if (port_id_is_invalid(port_id))
+	if (port_id_is_invalid(port_id, ENABLED_WARN))
 		return;
 
 	vlan_offload = rte_eth_dev_get_vlan_offload(port_id);
@@ -1591,7 +1607,7 @@ rx_vlan_strip_set(portid_t port_id, int on)
 	int diag;
 	int vlan_offload;
 
-	if (port_id_is_invalid(port_id))
+	if (port_id_is_invalid(port_id, ENABLED_WARN))
 		return;
 
 	vlan_offload = rte_eth_dev_get_vlan_offload(port_id);
@@ -1612,7 +1628,7 @@ rx_vlan_strip_set_on_queue(portid_t port_id, uint16_t queue_id, int on)
 {
 	int diag;
 
-	if (port_id_is_invalid(port_id))
+	if (port_id_is_invalid(port_id, ENABLED_WARN))
 		return;
 
 	diag = rte_eth_dev_set_vlan_strip_on_queue(port_id, queue_id, on);
@@ -1627,7 +1643,7 @@ rx_vlan_filter_set(portid_t port_id, int on)
 	int diag;
 	int vlan_offload;
 
-	if (port_id_is_invalid(port_id))
+	if (port_id_is_invalid(port_id, ENABLED_WARN))
 		return;
 
 	vlan_offload = rte_eth_dev_get_vlan_offload(port_id);
@@ -1648,7 +1664,7 @@ rx_vft_set(portid_t port_id, uint16_t vlan_id, int on)
 {
 	int diag;
 
-	if (port_id_is_invalid(port_id))
+	if (port_id_is_invalid(port_id, ENABLED_WARN))
 		return;
 	if (vlan_id_is_invalid(vlan_id))
 		return;
@@ -1665,7 +1681,7 @@ rx_vlan_all_filter_set(portid_t port_id, int on)
 {
 	uint16_t vlan_id;
 
-	if (port_id_is_invalid(port_id))
+	if (port_id_is_invalid(port_id, ENABLED_WARN))
 		return;
 	for (vlan_id = 0; vlan_id < 4096; vlan_id++)
 		rx_vft_set(port_id, vlan_id, on);
@@ -1675,7 +1691,7 @@ void
 vlan_tpid_set(portid_t port_id, uint16_t tp_id)
 {
 	int diag;
-	if (port_id_is_invalid(port_id))
+	if (port_id_is_invalid(port_id, ENABLED_WARN))
 		return;
 
 	diag = rte_eth_dev_set_vlan_ether_type(port_id, tp_id);
@@ -1690,7 +1706,7 @@ vlan_tpid_set(portid_t port_id, uint16_t tp_id)
 void
 tx_vlan_set(portid_t port_id, uint16_t vlan_id)
 {
-	if (port_id_is_invalid(port_id))
+	if (port_id_is_invalid(port_id, ENABLED_WARN))
 		return;
 	if (vlan_id_is_invalid(vlan_id))
 		return;
@@ -1701,7 +1717,7 @@ tx_vlan_set(portid_t port_id, uint16_t vlan_id)
 void
 tx_vlan_reset(portid_t port_id)
 {
-	if (port_id_is_invalid(port_id))
+	if (port_id_is_invalid(port_id, ENABLED_WARN))
 		return;
 	ports[port_id].tx_ol_flags &= ~TESTPMD_TX_OFFLOAD_INSERT_VLAN;
 }
@@ -1709,7 +1725,7 @@ tx_vlan_reset(portid_t port_id)
 void
 tx_vlan_pvid_set(portid_t port_id, uint16_t vlan_id, int on)
 {
-	if (port_id_is_invalid(port_id))
+	if (port_id_is_invalid(port_id, ENABLED_WARN))
 		return;
 
 	rte_eth_dev_set_vlan_pvid(port_id, vlan_id, on);
@@ -1721,7 +1737,7 @@ set_qmap(portid_t port_id, uint8_t is_rx, uint16_t queue_id, uint8_t map_value)
 	uint16_t i;
 	uint8_t existing_mapping_found = 0;
 
-	if (port_id_is_invalid(port_id))
+	if (port_id_is_invalid(port_id, ENABLED_WARN))
 		return;
 
 	if (is_rx ? (rx_queue_id_is_invalid(queue_id)) : (tx_queue_id_is_invalid(queue_id)))
@@ -1773,7 +1789,7 @@ fdir_add_signature_filter(portid_t port_id, uint8_t queue_id,
 {
 	int diag;
 
-	if (port_id_is_invalid(port_id))
+	if (port_id_is_invalid(port_id, ENABLED_WARN))
 		return;
 
 	diag = rte_eth_dev_fdir_add_signature_filter(port_id, fdir_filter,
@@ -1791,7 +1807,7 @@ fdir_update_signature_filter(portid_t port_id, uint8_t queue_id,
 {
 	int diag;
 
-	if (port_id_is_invalid(port_id))
+	if (port_id_is_invalid(port_id, ENABLED_WARN))
 		return;
 
 	diag = rte_eth_dev_fdir_update_signature_filter(port_id, fdir_filter,
@@ -1809,7 +1825,7 @@ fdir_remove_signature_filter(portid_t port_id,
 {
 	int diag;
 
-	if (port_id_is_invalid(port_id))
+	if (port_id_is_invalid(port_id, ENABLED_WARN))
 		return;
 
 	diag = rte_eth_dev_fdir_remove_signature_filter(port_id, fdir_filter);
@@ -1881,7 +1897,7 @@ fdir_get_infos(portid_t port_id)
 
 	static const char *fdir_stats_border = "########################";
 
-	if (port_id_is_invalid(port_id))
+	if (port_id_is_invalid(port_id, ENABLED_WARN))
 		return;
 	ret = rte_eth_dev_filter_supported(port_id, RTE_ETH_FILTER_FDIR);
 	if (ret < 0) {
@@ -1955,7 +1971,7 @@ fdir_add_perfect_filter(portid_t port_id, uint16_t soft_id, uint8_t queue_id,
 {
 	int diag;
 
-	if (port_id_is_invalid(port_id))
+	if (port_id_is_invalid(port_id, ENABLED_WARN))
 		return;
 
 	diag = rte_eth_dev_fdir_add_perfect_filter(port_id, fdir_filter,
@@ -1973,7 +1989,7 @@ fdir_update_perfect_filter(portid_t port_id, uint16_t soft_id, uint8_t queue_id,
 {
 	int diag;
 
-	if (port_id_is_invalid(port_id))
+	if (port_id_is_invalid(port_id, ENABLED_WARN))
 		return;
 
 	diag = rte_eth_dev_fdir_update_perfect_filter(port_id, fdir_filter,
@@ -1991,7 +2007,7 @@ fdir_remove_perfect_filter(portid_t port_id, uint16_t soft_id,
 {
 	int diag;
 
-	if (port_id_is_invalid(port_id))
+	if (port_id_is_invalid(port_id, ENABLED_WARN))
 		return;
 
 	diag = rte_eth_dev_fdir_remove_perfect_filter(port_id, fdir_filter,
@@ -2008,7 +2024,7 @@ fdir_set_masks(portid_t port_id, struct rte_fdir_masks *fdir_masks)
 {
 	int diag;
 
-	if (port_id_is_invalid(port_id))
+	if (port_id_is_invalid(port_id, ENABLED_WARN))
 		return;
 
 	diag = rte_eth_dev_fdir_set_masks(port_id, fdir_masks);
@@ -2085,7 +2101,7 @@ set_vf_traffic(portid_t port_id, uint8_t is_rx, uint16_t vf, uint8_t on)
 {
 	int diag;
 
-	if (port_id_is_invalid(port_id))
+	if (port_id_is_invalid(port_id, ENABLED_WARN))
 		return;
 	if (is_rx)
 		diag = rte_eth_dev_set_vf_rx(port_id,vf,on);
@@ -2107,7 +2123,7 @@ set_vf_rx_vlan(portid_t port_id, uint16_t vlan_id, uint64_t vf_mask, uint8_t on)
 {
 	int diag;
 
-	if (port_id_is_invalid(port_id))
+	if (port_id_is_invalid(port_id, ENABLED_WARN))
 		return;
 	if (vlan_id_is_invalid(vlan_id))
 		return;
@@ -2124,7 +2140,7 @@ set_queue_rate_limit(portid_t port_id, uint16_t queue_idx, uint16_t rate)
 	int diag;
 	struct rte_eth_link link;
 
-	if (port_id_is_invalid(port_id))
+	if (port_id_is_invalid(port_id, ENABLED_WARN))
 		return 1;
 	rte_eth_link_get_nowait(port_id, &link);
 	if (rate > link.link_speed) {
@@ -2149,7 +2165,7 @@ set_vf_rate_limit(portid_t port_id, uint16_t vf, uint16_t rate, uint64_t q_msk)
 	if (q_msk == 0)
 		return 0;
 
-	if (port_id_is_invalid(port_id))
+	if (port_id_is_invalid(port_id, ENABLED_WARN))
 		return 1;
 	rte_eth_link_get_nowait(port_id, &link);
 	if (rate > link.link_speed) {
diff --git a/app/test-pmd/parameters.c b/app/test-pmd/parameters.c
index adf3203..6f2af18 100644
--- a/app/test-pmd/parameters.c
+++ b/app/test-pmd/parameters.c
@@ -376,6 +376,7 @@ parse_portnuma_config(const char *q_arg)
 	};
 	unsigned long int_fld[_NUM_FLD];
 	char *str_fld[_NUM_FLD];
+	portid_t pid;
 
 	/* reset from value set at definition */
 	while ((p = strchr(p0,'(')) != NULL) {
@@ -397,8 +398,11 @@ parse_portnuma_config(const char *q_arg)
 				return -1;
 		}
 		port_id = (uint8_t)int_fld[FLD_PORT];
-		if (port_id >= nb_ports) {
-			printf("Invalid port, range is [0, %d]\n", nb_ports - 1);
+		if (port_id_is_invalid(port_id, ENABLED_WARN)) {
+			printf("Valid port range is [0");
+			FOREACH_PORT(pid, ports)
+				printf(", %d", pid);
+			printf("]\n");
 			return -1;
 		}
 		socket_id = (uint8_t)int_fld[FLD_SOCKET];
@@ -429,6 +433,7 @@ parse_ringnuma_config(const char *q_arg)
 	};
 	unsigned long int_fld[_NUM_FLD];
 	char *str_fld[_NUM_FLD];
+	portid_t pid;
 	#define RX_RING_ONLY 0x1
 	#define TX_RING_ONLY 0x2
 	#define RXTX_RING    0x3
@@ -453,8 +458,11 @@ parse_ringnuma_config(const char *q_arg)
 				return -1;
 		}
 		port_id = (uint8_t)int_fld[FLD_PORT];
-		if (port_id >= nb_ports) {
-			printf("Invalid port, range is [0, %d]\n", nb_ports - 1);
+		if (port_id_is_invalid(port_id, ENABLED_WARN)) {
+			printf("Valid port range is [0");
+			FOREACH_PORT(pid, ports)
+				printf(", %d", pid);
+			printf("]\n");
 			return -1;
 		}
 		socket_id = (uint8_t)int_fld[FLD_SOCKET];
@@ -626,12 +634,12 @@ launch_args_parse(int argc, char** argv)
 #endif
 			if (!strcmp(lgopts[opt_idx].name, "nb-ports")) {
 				n = atoi(optarg);
-				if (n > 0 && n <= nb_ports)
+				if (n > 0 &&
+				    !port_id_is_invalid(n, DISABLED_WARN))
 					nb_fwd_ports = (uint8_t) n;
 				else
 					rte_exit(EXIT_FAILURE,
-						 "nb-ports should be > 0 and <= %d\n",
-						 nb_ports);
+						 "Invalid port %d\n", n);
 			}
 			if (!strcmp(lgopts[opt_idx].name, "nb-cores")) {
 				n = atoi(optarg);
diff --git a/app/test-pmd/testpmd.c b/app/test-pmd/testpmd.c
index 773b8af..c18c1a9 100644
--- a/app/test-pmd/testpmd.c
+++ b/app/test-pmd/testpmd.c
@@ -71,6 +71,7 @@
 #include <rte_pci.h>
 #include <rte_ether.h>
 #include <rte_ethdev.h>
+#include <rte_dev.h>
 #include <rte_string_fns.h>
 #ifdef RTE_LIBRTE_PMD_XENVIRT
 #include <rte_eth_xenvirt.h>
@@ -315,7 +316,7 @@ uint16_t nb_rx_queue_stats_mappings = 0;
 
 /* Forward function declarations */
 static void map_port_queue_stats_mapping_registers(uint8_t pi, struct rte_port *port);
-static void check_all_ports_link_status(uint8_t port_num, uint32_t port_mask);
+static void check_all_ports_link_status(uint32_t port_mask);
 
 /*
  * Check if all the ports are started.
@@ -324,6 +325,20 @@ static void check_all_ports_link_status(uint8_t port_num, uint32_t port_mask);
 static int all_ports_started(void);
 
 /*
+ * Find next enabled port
+ */
+portid_t
+find_next_port(portid_t p, struct rte_port *ports, int size)
+{
+	if (ports == NULL)
+		rte_exit(-EINVAL, "failed to find a next port id\n");
+
+	while ((ports[p].enabled == 0) && (p < size))
+		p++;
+	return p;
+}
+
+/*
  * Setup default configuration.
  */
 static void
@@ -552,7 +567,8 @@ init_config(void)
 				+ RTE_TEST_TX_DESC_MAX + MAX_PKT_BURST;
 
 		if (!numa_support)
-			nb_mbuf_per_pool = (nb_mbuf_per_pool * nb_ports);
+			nb_mbuf_per_pool =
+				(nb_mbuf_per_pool * RTE_MAX_ETHPORTS);
 	}
 
 	if (!numa_support) {
@@ -565,14 +581,19 @@ init_config(void)
 
 	/* Configuration of Ethernet ports. */
 	ports = rte_zmalloc("testpmd: ports",
-			    sizeof(struct rte_port) * nb_ports,
+			    sizeof(struct rte_port) * RTE_MAX_ETHPORTS,
 			    RTE_CACHE_LINE_SIZE);
 	if (ports == NULL) {
-		rte_exit(EXIT_FAILURE, "rte_zmalloc(%d struct rte_port) "
-							"failed\n", nb_ports);
+		rte_exit(EXIT_FAILURE,
+				"rte_zmalloc(%d struct rte_port) failed\n",
+				RTE_MAX_ETHPORTS);
 	}
 
-	for (pid = 0; pid < nb_ports; pid++) {
+	/* enabled allocated ports */
+	for (pid = 0; pid < nb_ports; pid++)
+		ports[pid].enabled = 1;
+
+	FOREACH_PORT(pid, ports) {
 		port = &ports[pid];
 		rte_eth_dev_info_get(pid, &port->dev_info);
 
@@ -602,8 +623,7 @@ init_config(void)
 			nb_mbuf_per_pool = nb_mbuf_per_pool/nb_ports;
 
 		for (i = 0; i < MAX_SOCKET; i++) {
-			nb_mbuf = (nb_mbuf_per_pool *
-						port_per_socket[i]);
+			nb_mbuf = (nb_mbuf_per_pool * RTE_MAX_ETHPORTS);
 			if (nb_mbuf)
 				mbuf_pool_create(mbuf_data_size,
 						nb_mbuf,i);
@@ -635,14 +655,6 @@ reconfig(portid_t new_port_id, unsigned socket_id)
 	struct rte_port *port;
 
 	/* Reconfiguration of Ethernet ports. */
-	ports = rte_realloc(ports,
-			    sizeof(struct rte_port) * nb_ports,
-			    RTE_CACHE_LINE_SIZE);
-	if (ports == NULL) {
-		rte_exit(EXIT_FAILURE, "rte_realloc(%d struct rte_port) failed\n",
-				nb_ports);
-	}
-
 	port = &ports[new_port_id];
 	rte_eth_dev_info_get(new_port_id, &port->dev_info);
 
@@ -663,7 +675,7 @@ init_fwd_streams(void)
 	streamid_t sm_id, nb_fwd_streams_new;
 
 	/* set socket id according to numa or not */
-	for (pid = 0; pid < nb_ports; pid++) {
+	FOREACH_PORT(pid, ports) {
 		port = &ports[pid];
 		if (nb_rxq > port->dev_info.max_rx_queues) {
 			printf("Fail: nb_rxq(%d) is greater than "
@@ -1264,7 +1276,7 @@ all_ports_started(void)
 	portid_t pi;
 	struct rte_port *port;
 
-	for (pi = 0; pi < nb_ports; pi++) {
+	FOREACH_PORT(pi, ports) {
 		port = &ports[pi];
 		/* Check if there is a port which is not started */
 		if (port->port_status != RTE_PORT_STARTED)
@@ -1276,6 +1288,45 @@ all_ports_started(void)
 }
 
 int
+all_ports_stopped(void)
+{
+	portid_t pi;
+	struct rte_port *port;
+
+	FOREACH_PORT(pi, ports) {
+		port = &ports[pi];
+		if (port->port_status != RTE_PORT_STOPPED)
+			return 0;
+	}
+
+	return 1;
+}
+
+int
+port_is_started(portid_t port_id)
+{
+	if (port_id_is_invalid(port_id, ENABLED_WARN))
+		return 0;
+
+	if (ports[port_id].port_status != RTE_PORT_STARTED)
+		return 0;
+
+	return 1;
+}
+
+static int
+port_is_closed(portid_t port_id)
+{
+	if (port_id_is_invalid(port_id, ENABLED_WARN))
+		return 0;
+
+	if (ports[port_id].port_status != RTE_PORT_CLOSED)
+		return 0;
+
+	return 1;
+}
+
+int
 start_port(portid_t pid)
 {
 	int diag, need_check_link_status = 0;
@@ -1296,8 +1347,8 @@ start_port(portid_t pid)
 
 	if(dcb_config)
 		dcb_test = 1;
-	for (pi = 0; pi < nb_ports; pi++) {
-		if (pid < nb_ports && pid != pi)
+	FOREACH_PORT(pi, ports) {
+		if (!port_id_is_invalid(pid, DISABLED_WARN) && pid != pi)
 			continue;
 
 		port = &ports[pi];
@@ -1421,7 +1472,7 @@ start_port(portid_t pid)
 	}
 
 	if (need_check_link_status && !no_link_check)
-		check_all_ports_link_status(nb_ports, RTE_PORT_ALL);
+		check_all_ports_link_status(RTE_PORT_ALL);
 	else
 		printf("Please stop the ports first\n");
 
@@ -1446,8 +1497,8 @@ stop_port(portid_t pid)
 	}
 	printf("Stopping ports...\n");
 
-	for (pi = 0; pi < nb_ports; pi++) {
-		if (pid < nb_ports && pid != pi)
+	FOREACH_PORT(pi, ports) {
+		if (!port_id_is_invalid(pid, DISABLED_WARN) && pid != pi)
 			continue;
 
 		port = &ports[pi];
@@ -1463,7 +1514,7 @@ stop_port(portid_t pid)
 		need_check_link_status = 1;
 	}
 	if (need_check_link_status && !no_link_check)
-		check_all_ports_link_status(nb_ports, RTE_PORT_ALL);
+		check_all_ports_link_status(RTE_PORT_ALL);
 
 	printf("Done\n");
 }
@@ -1481,8 +1532,8 @@ close_port(portid_t pid)
 
 	printf("Closing ports...\n");
 
-	for (pi = 0; pi < nb_ports; pi++) {
-		if (pid < nb_ports && pid != pi)
+	FOREACH_PORT(pi, ports) {
+		if (!port_id_is_invalid(pid, DISABLED_WARN) && pid != pi)
 			continue;
 
 		port = &ports[pi];
@@ -1502,31 +1553,83 @@ close_port(portid_t pid)
 	printf("Done\n");
 }
 
-int
-all_ports_stopped(void)
+void
+attach_port(char *identifier)
 {
-	portid_t pi;
-	struct rte_port *port;
+	portid_t i, j, pi = 0;
 
-	for (pi = 0; pi < nb_ports; pi++) {
-		port = &ports[pi];
-		if (port->port_status != RTE_PORT_STOPPED)
-			return 0;
+	printf("Attaching a new port...\n");
+
+	if (identifier == NULL) {
+		printf("Invalid parameters are speficied\n");
+		return;
 	}
 
-	return 1;
+	if (test_done == 0) {
+		printf("Please stop forwarding first\n");
+		return;
+	}
+
+	if (rte_eal_dev_attach(identifier, &pi))
+		return;
+
+	ports[pi].enabled = 1;
+	reconfig(pi, rte_eth_dev_socket_id(pi));
+	rte_eth_promiscuous_enable(pi);
+
+	nb_ports = rte_eth_dev_count();
+
+	/* set_default_fwd_ports_config(); */
+	bzero(fwd_ports_ids, sizeof(fwd_ports_ids));
+	i = 0;
+	FOREACH_PORT(j, ports) {
+		fwd_ports_ids[i] = j;
+		i++;
+	}
+	nb_cfg_ports = nb_ports;
+	nb_fwd_ports++;
+
+	ports[pi].port_status = RTE_PORT_STOPPED;
+
+	printf("Port %d is attached. Now total ports is %d\n", pi, nb_ports);
+	printf("Done\n");
 }
 
-int
-port_is_started(portid_t port_id)
+void
+detach_port(uint8_t port_id)
 {
-	if (port_id_is_invalid(port_id))
-		return -1;
+	portid_t i, pi = 0;
+	char name[RTE_ETH_NAME_MAX_LEN];
 
-	if (ports[port_id].port_status != RTE_PORT_STARTED)
-		return 0;
+	printf("Detaching a port...\n");
 
-	return 1;
+	if (!port_is_closed(port_id)) {
+		printf("Please close port first\n");
+		return;
+	}
+
+	rte_eth_promiscuous_disable(port_id);
+
+	if (rte_eal_dev_detach(port_id, name))
+		return;
+
+	ports[port_id].enabled = 0;
+	nb_ports = rte_eth_dev_count();
+
+	/* set_default_fwd_ports_config(); */
+	bzero(fwd_ports_ids, sizeof(fwd_ports_ids));
+	i = 0;
+	FOREACH_PORT(pi, ports) {
+		fwd_ports_ids[i] = pi;
+		i++;
+	}
+	nb_cfg_ports = nb_ports;
+	nb_fwd_ports--;
+
+	printf("Port '%s' is detached. Now total ports is %d\n",
+			name, nb_ports);
+	printf("Done\n");
+	return;
 }
 
 void
@@ -1534,7 +1637,7 @@ pmd_test_exit(void)
 {
 	portid_t pt_id;
 
-	for (pt_id = 0; pt_id < nb_ports; pt_id++) {
+	FOREACH_PORT(pt_id, ports) {
 		printf("Stopping port %d...", pt_id);
 		fflush(stdout);
 		rte_eth_dev_close(pt_id);
@@ -1553,7 +1656,7 @@ struct pmd_test_command {
 
 /* Check the link status of all ports in up to 9s, and print them finally */
 static void
-check_all_ports_link_status(uint8_t port_num, uint32_t port_mask)
+check_all_ports_link_status(uint32_t port_mask)
 {
 #define CHECK_INTERVAL 100 /* 100ms */
 #define MAX_CHECK_TIME 90 /* 9s (90 * 100ms) in total */
@@ -1564,7 +1667,7 @@ check_all_ports_link_status(uint8_t port_num, uint32_t port_mask)
 	fflush(stdout);
 	for (count = 0; count <= MAX_CHECK_TIME; count++) {
 		all_ports_up = 1;
-		for (portid = 0; portid < port_num; portid++) {
+		FOREACH_PORT(portid, ports) {
 			if ((port_mask & (1 << portid)) == 0)
 				continue;
 			memset(&link, 0, sizeof(link));
@@ -1688,7 +1791,7 @@ init_port_config(void)
 	portid_t pid;
 	struct rte_port *port;
 
-	for (pid = 0; pid < nb_ports; pid++) {
+	FOREACH_PORT(pid, ports) {
 		port = &ports[pid];
 		port->dev_conf.rxmode = rx_mode;
 		port->dev_conf.fdir_conf = fdir_conf;
@@ -1877,7 +1980,7 @@ main(int argc, char** argv)
 
 	nb_ports = (portid_t) rte_eth_dev_count();
 	if (nb_ports == 0)
-		rte_exit(EXIT_FAILURE, "No probed ethernet device\n");
+		RTE_LOG(WARNING, EAL, "No probed ethernet devices\n");
 
 	set_def_fwd_config();
 	if (nb_lcores == 0)
@@ -1899,7 +2002,7 @@ main(int argc, char** argv)
 		rte_exit(EXIT_FAILURE, "Start ports failed\n");
 
 	/* set all ports to promiscuous mode by default */
-	for (port_id = 0; port_id < nb_ports; port_id++)
+	FOREACH_PORT(port_id, ports)
 		rte_eth_promiscuous_enable(port_id);
 
 #ifdef RTE_LIBRTE_CMDLINE
diff --git a/app/test-pmd/testpmd.h b/app/test-pmd/testpmd.h
index 8f5e6c7..109c670 100644
--- a/app/test-pmd/testpmd.h
+++ b/app/test-pmd/testpmd.h
@@ -134,6 +134,7 @@ struct fwd_stream {
  * The data structure associated with each port.
  */
 struct rte_port {
+	uint8_t                 enabled;    /**< Port enabled or not */
 	struct rte_eth_dev_info dev_info;   /**< PCI info + driver name */
 	struct rte_eth_conf     dev_conf;   /**< Port configuration. */
 	struct ether_addr       eth_addr;   /**< Port ethernet address */
@@ -159,6 +160,14 @@ struct rte_port {
 	struct rte_eth_txconf   tx_conf;    /**< tx configuration */
 };
 
+extern portid_t __rte_unused
+find_next_port(portid_t p, struct rte_port *ports, int size);
+
+#define FOREACH_PORT(p, ports) \
+	for (p = find_next_port(0, ports, RTE_MAX_ETHPORTS); \
+	    p < RTE_MAX_ETHPORTS; \
+	    p = find_next_port(p + 1, ports, RTE_MAX_ETHPORTS))
+
 /**
  * The data structure associated with each forwarding logical core.
  * The logical cores are internally numbered by a core index from 0 to
@@ -515,6 +524,8 @@ int init_port_dcb_config(portid_t pid,struct dcb_config *dcb_conf);
 int start_port(portid_t pid);
 void stop_port(portid_t pid);
 void close_port(portid_t pid);
+void attach_port(char *identifier);
+void detach_port(uint8_t port_id);
 int all_ports_stopped(void);
 int port_is_started(portid_t port_id);
 void pmd_test_exit(void);
@@ -558,10 +569,15 @@ void get_ethertype_filter(uint8_t port_id, uint16_t index);
 void get_2tuple_filter(uint8_t port_id, uint16_t index);
 void get_5tuple_filter(uint8_t port_id, uint16_t index);
 void get_flex_filter(uint8_t port_id, uint16_t index);
-int port_id_is_invalid(portid_t port_id);
 int rx_queue_id_is_invalid(queueid_t rxq_id);
 int tx_queue_id_is_invalid(queueid_t txq_id);
 
+enum print_warning {
+	ENABLED_WARN = 0,
+	DISABLED_WARN
+};
+int port_id_is_invalid(portid_t port_id, enum print_warning warning);
+
 /*
  * Work-around of a compilation error with ICC on invocations of the
  * rte_be_to_cpu_16() function.
-- 
1.9.1

^ permalink raw reply	[flat|nested] 362+ messages in thread

* Re: [dpdk-dev] [PATCH v4 00/11] Port Hotplug Framework
  2015-01-19 10:40 ` [dpdk-dev] [PATCH v4 00/11] Port Hotplug Framework Tetsuya Mukawa
                     ` (10 preceding siblings ...)
  2015-01-19 10:40   ` [dpdk-dev] [PATCH v4 11/11] eal: Enable port hotplug framework in Linux Tetsuya Mukawa
@ 2015-01-19 13:15   ` Qiu, Michael
  2015-01-27  3:00   ` Qiu, Michael
  12 siblings, 0 replies; 362+ messages in thread
From: Qiu, Michael @ 2015-01-19 13:15 UTC (permalink / raw)
  To: Tetsuya Mukawa, dev

Hi, Tetsuya

What I miss? I haven't got the first patch, just start with "PATCH 2/11".

Does this only happens with my mail client?

Thanks,
Michael

On 2015/1/19 18:42, Tetsuya Mukawa wrote:
> This patch series adds a dynamic port hotplug framework to DPDK.
> With the patches, DPDK apps can attach or detach ports at runtime.
>
> The basic concept of the port hotplug is like followings.
> - DPDK apps must have responsibility to manage ports.
>   DPDK apps only know which ports are attached or detached at the moment.
>   The port hotplug framework is implemented to allow DPDK apps to manage ports.
>   For example, when DPDK apps call port attach function, attached port number
>   will be returned. Also DPDK apps can detach port by port number.
> - Kernel support is needed for attaching or detaching physical device ports.
>   To attach new device, the device will be recognized by kernel at first and
>   controlled by kernel driver. Then user can bind the device to igb_uio
>   by 'dpdk_nic_bind.py'. Finally, DPDK apps can call the port hotplug
>   functions to attach ports.
>   For detaching, steps are vice versa.
> - Before detach ports, ports must be stopped and closed.
>   DPDK application must call rte_eth_dev_stop() and rte_eth_dev_close() before
>   detaching ports. These function will call finalization codes of PMDs.
>   But so far, no PMD frees all resources allocated by initialization.
>   It means PMDs are needed to be fixed to support the port hotplug.
>   'RTE_PCI_DRV_DETACHABLE' is a new flag indicating a PMD supports detaching.
>   Without this flag, detaching will be failed.
> - Mustn't affect legacy DPDK apps.
>   No DPDK EAL behavior is changed, if the port hotplug functions are't called.
>   So all legacy DPDK apps can still work without modifications.
>
> And a few limitations.
> - The port hotplug functions are not thread safe.
>   DPDK apps should handle it.
> - Only support Linux and igb_uio so far.
>   BSD and VFIO is not supported. I will send VFIO patches at least, but I don't
>   have a plan to submit BSD patch so far.
>
>
> Here is port hotplug APIs.
> -------------------------------------------------------------------------------
> /**
>  * Attach a new device.
>  *
>  * @param devargs
>  *   A pointer to a strings array describing the new device
>  *   to be attached. The strings should be a pci address like
>  *   '0000:01:00.0' or virtual device name like 'eth_pcap0'.
>  * @param port_id
>  *  A pointer to a port identifier actually attached.
>  * @return
>  *  0 on success and port_id is filled, negative on error
>  */
> int rte_eal_dev_attach(const char *devargs, uint8_t *port_id);
>
> /**
>  * Detach a device.
>  *
>  * @param port_id
>  *   The port identifier of the device to detach.
>  * @param addr
>  *  A pointer to a device name actually detached.
>  * @return
>  *  0 on success and devname is filled, negative on error
>  */
> int rte_eal_dev_detach(uint8_t port_id, char *devname);
> -------------------------------------------------------------------------------
>
> This patch series are for DPDK EAL. To use port hotplug function by DPDK apps,
> each PMD should be fixed to support 'RTE_PCI_DRV_DETACHABLE' flag. Please check
> a patch for pcap PMD.
>
> Also please check testpmd patch. It will show you how to fix your legacy
> applications to support port hotplug feature.
>
>
> PATCH v4 changes
>  - Merge patches to review easier.
>  - Fix indent of 'if' statement.
>  - Fix calculation method of eal_compare_pci_addr().
>  - Fix header file declaration.
>  - Add header file to determine if hotplug can be enabled.
>    (Thanks to Qiu, Michael)
>  - Use braces with 'for' loop.
>  - Add paramerter checking.
>  - Fix sanity check code
>  - Fix comments of rte_eth_dev_type.
>  - Change function names.
>    (Thanks to Iremonger, Bernard)
>
> PATCH v3 changes:
>  - Fix enum definition used in rte_ethdev.c.
>    (Thanks to Zhang, Helin)
>
> PATCH v2 changes:
>  - Replace rte_eal_dev_attach_pdev(), rte_eal_dev_detach_pdev,
>    rte_eal_dev_attach_vdev() and rte_eal_dev_detach_vdev() to
>    rte_eal_dev_attach() and rte_eal_dev_detach().
>  - Add parameter values checking.
>  - Refashion a few functions.
>    (Thanks to Iremonger, Bernard)
>
> PATCH v1 Changes:
>  - Fix error checking code of librte_eth APIs.
>  - Fix issue that port from pcap PMD cannot be detached correctly.
>  - Fix issue that testpmd could hang after forwarding, if attaching and detaching
>    is repeatedly.
>  - Fix if-condition of rte_eth_dev_get_port_by_addr().
>    (Thanks to Mark Enright)
>
> RFC PATCH v2 Changes:
> - remove 'rte_eth_dev_validate_port()', and cleanup codes.
>
>
> Tetsuya Mukawa (11):
>   eal/pci,ethdev: Remove assumption that port will not be detached
>   eal/pci: Consolidate pci address comparison APIs
>   ethdev: Add rte_eth_dev_free to free specified device
>   eal,ethdev: Add a function and function pointers to close ether device
>   ethdev: Add functions that will be used by port hotplug functions
>   eal/linux/pci: Add functions for unmapping igb_uio resources
>   eal/pci: Add a function to remove the entry of devargs list
>   eal/pci: Cleanup pci driver initialization code
>   ethdev: Add one dev_type paramerter to rte_eth_dev_allocate
>   eal/pci: Add rte_eal_dev_attach/detach() functions
>   eal: Enable port hotplug framework in Linux
>
>  app/test/virtual_pmd.c                          |   2 +-
>  config/common_linuxapp                          |   5 +
>  lib/librte_eal/bsdapp/eal/eal_pci.c             |  16 +-
>  lib/librte_eal/common/Makefile                  |   1 +
>  lib/librte_eal/common/eal_common_dev.c          | 273 +++++++++++++++
>  lib/librte_eal/common/eal_common_devargs.c      |  57 ++++
>  lib/librte_eal/common/eal_common_pci.c          |  92 ++++-
>  lib/librte_eal/common/eal_private.h             |  36 ++
>  lib/librte_eal/common/include/rte_dev.h         |  33 ++
>  lib/librte_eal/common/include/rte_dev_hotplug.h |  44 +++
>  lib/librte_eal/common/include/rte_devargs.h     |  18 +
>  lib/librte_eal/common/include/rte_pci.h         |  76 +++++
>  lib/librte_eal/linuxapp/eal/Makefile            |   1 +
>  lib/librte_eal/linuxapp/eal/eal_pci.c           | 136 ++++++--
>  lib/librte_eal/linuxapp/eal/eal_pci_init.h      |   8 +
>  lib/librte_eal/linuxapp/eal/eal_pci_uio.c       |  67 +++-
>  lib/librte_ether/rte_ethdev.c                   | 424 ++++++++++++++++++------
>  lib/librte_ether/rte_ethdev.h                   | 145 +++++++-
>  lib/librte_pmd_af_packet/rte_eth_af_packet.c    |   2 +-
>  lib/librte_pmd_bond/rte_eth_bond_api.c          |   2 +-
>  lib/librte_pmd_pcap/rte_eth_pcap.c              |   2 +-
>  lib/librte_pmd_ring/rte_eth_ring.c              |   2 +-
>  lib/librte_pmd_xenvirt/rte_eth_xenvirt.c        |   2 +-
>  23 files changed, 1283 insertions(+), 161 deletions(-)
>  create mode 100644 lib/librte_eal/common/include/rte_dev_hotplug.h
>


^ permalink raw reply	[flat|nested] 362+ messages in thread

* Re: [dpdk-dev] FW:  [PATCH v4 01/11] eal/pci, ethdev: Remove assumption that port will not be detached
       [not found]     ` <8CEF83825BEC744B83065625E567D7C2049D7C19@IRSMSX108.ger.corp.intel.com>
@ 2015-01-19 14:24       ` Qiu, Michael
  2015-01-20  2:02         ` Tetsuya Mukawa
  0 siblings, 1 reply; 362+ messages in thread
From: Qiu, Michael @ 2015-01-19 14:24 UTC (permalink / raw)
  To: Tetsuya Mukawa, dev

Hi, Tetsuya

You see lots of places have below three lines:

	if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
 		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
 		return -EINVAL;
 	}

They are all the same(only few has the print log different), so can we improve it?

See below:

+static int
+rte_eth_dev_validate_port(uint8_t port_id, bool trace) {
+	if (port_id >= RTE_MAX_ETHPORTS ||
+	    rte_eth_devices[port_id].attached != DEV_CONNECTED) {
+		if (trace)
+			PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
+		return DEV_INVALID;
+	}
+	else
+		return DEV_VALID;
+}

For normal case we just call this function use
rte_eth_dev_validate_port(port_id, 1)
here 1 could be a enmu value(Thus trace should be defined as int).
After call, we didn't need to add PMD_DEBUG_TRACE() any more.
 
For few cases like:
 		PMD_DEBUG_TRACE("set VF rate limit:invalid port id=%d\n",
 				port_id);

We can call the function with secondary param set to "0", and add the trace log after the function called, just as before.

I think after this enhancement, the code seems more clean and efficiency。


Thanks,
Michael

>> -----Original Message-----
>> From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Tetsuya Mukawa
>> Sent: Monday, January 19, 2015 10:40 AM
>> To: dev@dpdk.org
>> Subject: [dpdk-dev] [PATCH v4 01/11] eal/pci, ethdev: Remove assumption that port will not be
>> detached
>>
>> To remove assumption, do like followings.
>>
>> This patch adds "RTE_PCI_DRV_DETACHABLE" to drv_flags of rte_pci_driver structure. The flags
>> indicates the driver can detach devices at runtime.
>> Also remove assumption that port will not be detached.
>>
>> To remove the assumption.
>> - Add 'attached' member to rte_eth_dev structure.
>>   This member is used for indicating the port is attached, or not.
>> - Add rte_eth_dev_allocate_new_port().
>>   This function is used for allocating new port.
>>
>> v4:
>> - Use braces with 'for' loop.
>> - Fix indent of 'if' statement.
>>
>> Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp>
>> ---
>>  lib/librte_eal/common/include/rte_pci.h |   2 +
>>  lib/librte_ether/rte_ethdev.c           | 248 ++++++++++++++++++--------------
>>  lib/librte_ether/rte_ethdev.h           |   5 +
>>  3 files changed, 151 insertions(+), 104 deletions(-)
>>
>> diff --git a/lib/librte_eal/common/include/rte_pci.h b/lib/librte_eal/common/include/rte_pci.h
>> index 66ed793..dd1da28 100644
>> --- a/lib/librte_eal/common/include/rte_pci.h
>> +++ b/lib/librte_eal/common/include/rte_pci.h
>> @@ -199,6 +199,8 @@ struct rte_pci_driver {  #define RTE_PCI_DRV_FORCE_UNBIND 0x0004
>>  /** Device driver supports link state interrupt */
>>  #define RTE_PCI_DRV_INTR_LSC	0x0008
>> +/** Device driver supports detaching capability */
>> +#define RTE_PCI_DRV_DETACHABLE	0x0010
>>
>>  /**< Internal use only - Macro used by pci addr parsing functions **/
>>  #define GET_PCIADDR_FIELD(in, fd, lim, dlm)                   \
>> diff --git a/lib/librte_ether/rte_ethdev.c b/lib/librte_ether/rte_ethdev.c index 077d430..46cabaf
>> 100644
>> --- a/lib/librte_ether/rte_ethdev.c
>> +++ b/lib/librte_ether/rte_ethdev.c
>> @@ -175,6 +175,16 @@ enum {
>>  	STAT_QMAP_RX
>>  };
>>
>> +enum {
>> +	DEV_INVALID = 0,
>> +	DEV_VALID,
>> +};
>> +
>> +enum {
>> +	DEV_DISCONNECTED = 0,
>> +	DEV_CONNECTED
>> +};
>> +
>>  static inline void
>>  rte_eth_dev_data_alloc(void)
>>  {
>> @@ -201,19 +211,34 @@ rte_eth_dev_allocated(const char *name)  {
>>  	unsigned i;
>>
>> -	for (i = 0; i < nb_ports; i++) {
>> -		if (strcmp(rte_eth_devices[i].data->name, name) == 0)
>> +	for (i = 0; i < RTE_MAX_ETHPORTS; i++) {
>> +		if ((rte_eth_devices[i].attached == DEV_CONNECTED) &&
>> +		    strcmp(rte_eth_devices[i].data->name, name) == 0)
>>  			return &rte_eth_devices[i];
>>  	}
>>  	return NULL;
>>  }
>>
>> +static uint8_t
>> +rte_eth_dev_allocate_new_port(void)
>> +{
>> +	unsigned i;
>> +
>> +	for (i = 0; i < RTE_MAX_ETHPORTS; i++) {
>> +		if (rte_eth_devices[i].attached == DEV_DISCONNECTED)
>> +			return i;
>> +	}
>> +	return RTE_MAX_ETHPORTS;
>> +}
>> +
>>  struct rte_eth_dev *
>>  rte_eth_dev_allocate(const char *name)
>>  {
>> +	uint8_t port_id;
>>  	struct rte_eth_dev *eth_dev;
>>
>> -	if (nb_ports == RTE_MAX_ETHPORTS) {
>> +	port_id = rte_eth_dev_allocate_new_port();
>> +	if (port_id == RTE_MAX_ETHPORTS) {
>>  		PMD_DEBUG_TRACE("Reached maximum number of Ethernet ports\n");
>>  		return NULL;
>>  	}
>> @@ -226,10 +251,12 @@ rte_eth_dev_allocate(const char *name)
>>  		return NULL;
>>  	}
>>
>> -	eth_dev = &rte_eth_devices[nb_ports];
>> -	eth_dev->data = &rte_eth_dev_data[nb_ports];
>> +	eth_dev = &rte_eth_devices[port_id];
>> +	eth_dev->data = &rte_eth_dev_data[port_id];
>>  	snprintf(eth_dev->data->name, sizeof(eth_dev->data->name), "%s", name);
>> -	eth_dev->data->port_id = nb_ports++;
>> +	eth_dev->data->port_id = port_id;
>> +	eth_dev->attached = DEV_CONNECTED;
>> +	nb_ports++;
>>  	return eth_dev;
>>  }
>>
>> @@ -283,6 +310,7 @@ rte_eth_dev_init(struct rte_pci_driver *pci_drv,
>>  			(unsigned) pci_dev->id.device_id);
>>  	if (rte_eal_process_type() == RTE_PROC_PRIMARY)
>>  		rte_free(eth_dev->data->dev_private);
>> +	eth_dev->attached = DEV_DISCONNECTED;
>>  	nb_ports--;
>>  	return diag;
>>  }
>> @@ -308,10 +336,22 @@ rte_eth_driver_register(struct eth_driver *eth_drv)
>>  	rte_eal_pci_register(&eth_drv->pci_drv);
>>  }
>>
>> +static int
>> +rte_eth_dev_validate_port(uint8_t port_id) {
>> +	if (port_id >= RTE_MAX_ETHPORTS)
>> +		return DEV_INVALID;
>> +
>> +	if (rte_eth_devices[port_id].attached == DEV_CONNECTED)
>> +		return DEV_VALID;
>> +	else
>> +		return DEV_INVALID;
>> +}
>> +
>>  int
>>  rte_eth_dev_socket_id(uint8_t port_id)
>>  {
>> -	if (port_id >= nb_ports)
>> +	if (rte_eth_dev_validate_port(port_id) == DEV_INVALID)
>>  		return -1;
>>  	return rte_eth_devices[port_id].pci_dev->numa_node;
>>  }
>> @@ -369,7 +409,7 @@ rte_eth_dev_rx_queue_start(uint8_t port_id, uint16_t rx_queue_id)
>>  	 * in a multi-process setup*/
>>  	PROC_PRIMARY_OR_ERR_RET(-E_RTE_SECONDARY);
>>
>> -	if (port_id >= nb_ports) {
>> +	if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
>>  		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
>>  		return -EINVAL;
>>  	}
>> @@ -395,7 +435,7 @@ rte_eth_dev_rx_queue_stop(uint8_t port_id, uint16_t rx_queue_id)
>>  	 * in a multi-process setup*/
>>  	PROC_PRIMARY_OR_ERR_RET(-E_RTE_SECONDARY);
>>
>> -	if (port_id >= nb_ports) {
>> +	if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
>>  		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
>>  		return -EINVAL;
>>  	}
>> @@ -421,7 +461,7 @@ rte_eth_dev_tx_queue_start(uint8_t port_id, uint16_t tx_queue_id)
>>  	 * in a multi-process setup*/
>>  	PROC_PRIMARY_OR_ERR_RET(-E_RTE_SECONDARY);
>>
>> -	if (port_id >= nb_ports) {
>> +	if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
>>  		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
>>  		return -EINVAL;
>>  	}
>> @@ -447,7 +487,7 @@ rte_eth_dev_tx_queue_stop(uint8_t port_id, uint16_t tx_queue_id)
>>  	 * in a multi-process setup*/
>>  	PROC_PRIMARY_OR_ERR_RET(-E_RTE_SECONDARY);
>>
>> -	if (port_id >= nb_ports) {
>> +	if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
>>  		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
>>  		return -EINVAL;
>>  	}
>> @@ -703,7 +743,7 @@ rte_eth_dev_configure(uint8_t port_id, uint16_t nb_rx_q, uint16_t nb_tx_q,
>>  	 * in a multi-process setup*/
>>  	PROC_PRIMARY_OR_ERR_RET(-E_RTE_SECONDARY);
>>
>> -	if (port_id >= nb_ports || port_id >= RTE_MAX_ETHPORTS) {
>> +	if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
>>  		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
>>  		return (-EINVAL);
>>  	}
>> @@ -888,7 +928,7 @@ rte_eth_dev_start(uint8_t port_id)
>>  	 * in a multi-process setup*/
>>  	PROC_PRIMARY_OR_ERR_RET(-E_RTE_SECONDARY);
>>
>> -	if (port_id >= nb_ports) {
>> +	if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
>>  		PMD_DEBUG_TRACE("Invalid port_id=%" PRIu8 "\n", port_id);
>>  		return (-EINVAL);
>>  	}
>> @@ -923,7 +963,7 @@ rte_eth_dev_stop(uint8_t port_id)
>>  	 * in a multi-process setup*/
>>  	PROC_PRIMARY_OR_RET();
>>
>> -	if (port_id >= nb_ports) {
>> +	if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
>>  		PMD_DEBUG_TRACE("Invalid port_id=%" PRIu8 "\n", port_id);
>>  		return;
>>  	}
>> @@ -951,7 +991,7 @@ rte_eth_dev_set_link_up(uint8_t port_id)
>>  	 * in a multi-process setup*/
>>  	PROC_PRIMARY_OR_ERR_RET(-E_RTE_SECONDARY);
>>
>> -	if (port_id >= nb_ports) {
>> +	if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
>>  		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
>>  		return -EINVAL;
>>  	}
>> @@ -970,7 +1010,7 @@ rte_eth_dev_set_link_down(uint8_t port_id)
>>  	 * in a multi-process setup*/
>>  	PROC_PRIMARY_OR_ERR_RET(-E_RTE_SECONDARY);
>>
>> -	if (port_id >= nb_ports) {
>> +	if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
>>  		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
>>  		return -EINVAL;
>>  	}
>> @@ -989,7 +1029,7 @@ rte_eth_dev_close(uint8_t port_id)
>>  	 * in a multi-process setup*/
>>  	PROC_PRIMARY_OR_RET();
>>
>> -	if (port_id >= nb_ports) {
>> +	if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
>>  		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
>>  		return;
>>  	}
>> @@ -1017,7 +1057,7 @@ rte_eth_rx_queue_setup(uint8_t port_id, uint16_t rx_queue_id,
>>  	 * in a multi-process setup*/
>>  	PROC_PRIMARY_OR_ERR_RET(-E_RTE_SECONDARY);
>>
>> -	if (port_id >= nb_ports) {
>> +	if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
>>  		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
>>  		return (-EINVAL);
>>  	}
>> @@ -1090,7 +1130,7 @@ rte_eth_tx_queue_setup(uint8_t port_id, uint16_t tx_queue_id,
>>  	 * in a multi-process setup*/
>>  	PROC_PRIMARY_OR_ERR_RET(-E_RTE_SECONDARY);
>>
>> -	if (port_id >= RTE_MAX_ETHPORTS || port_id >= nb_ports) {
>> +	if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
>>  		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
>>  		return (-EINVAL);
>>  	}
>> @@ -1123,7 +1163,7 @@ rte_eth_promiscuous_enable(uint8_t port_id)  {
>>  	struct rte_eth_dev *dev;
>>
>> -	if (port_id >= nb_ports) {
>> +	if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
>>  		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
>>  		return;
>>  	}
>> @@ -1139,7 +1179,7 @@ rte_eth_promiscuous_disable(uint8_t port_id)  {
>>  	struct rte_eth_dev *dev;
>>
>> -	if (port_id >= nb_ports) {
>> +	if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
>>  		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
>>  		return;
>>  	}
>> @@ -1155,7 +1195,7 @@ rte_eth_promiscuous_get(uint8_t port_id)  {
>>  	struct rte_eth_dev *dev;
>>
>> -	if (port_id >= nb_ports) {
>> +	if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
>>  		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
>>  		return -1;
>>  	}
>> @@ -1169,7 +1209,7 @@ rte_eth_allmulticast_enable(uint8_t port_id)  {
>>  	struct rte_eth_dev *dev;
>>
>> -	if (port_id >= nb_ports) {
>> +	if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
>>  		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
>>  		return;
>>  	}
>> @@ -1185,7 +1225,7 @@ rte_eth_allmulticast_disable(uint8_t port_id)  {
>>  	struct rte_eth_dev *dev;
>>
>> -	if (port_id >= nb_ports) {
>> +	if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
>>  		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
>>  		return;
>>  	}
>> @@ -1201,7 +1241,7 @@ rte_eth_allmulticast_get(uint8_t port_id)  {
>>  	struct rte_eth_dev *dev;
>>
>> -	if (port_id >= nb_ports) {
>> +	if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
>>  		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
>>  		return -1;
>>  	}
>> @@ -1229,7 +1269,7 @@ rte_eth_link_get(uint8_t port_id, struct rte_eth_link *eth_link)  {
>>  	struct rte_eth_dev *dev;
>>
>> -	if (port_id >= nb_ports) {
>> +	if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
>>  		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
>>  		return;
>>  	}
>> @@ -1249,7 +1289,7 @@ rte_eth_link_get_nowait(uint8_t port_id, struct rte_eth_link *eth_link)  {
>>  	struct rte_eth_dev *dev;
>>
>> -	if (port_id >= nb_ports) {
>> +	if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
>>  		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
>>  		return;
>>  	}
>> @@ -1269,7 +1309,7 @@ rte_eth_stats_get(uint8_t port_id, struct rte_eth_stats *stats)  {
>>  	struct rte_eth_dev *dev;
>>
>> -	if (port_id >= nb_ports) {
>> +	if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
>>  		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
>>  		return;
>>  	}
>> @@ -1286,7 +1326,7 @@ rte_eth_stats_reset(uint8_t port_id)  {
>>  	struct rte_eth_dev *dev;
>>
>> -	if (port_id >= nb_ports) {
>> +	if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
>>  		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
>>  		return;
>>  	}
>> @@ -1307,7 +1347,7 @@ rte_eth_xstats_get(uint8_t port_id, struct rte_eth_xstats *xstats,
>>  	uint64_t val;
>>  	char *stats_ptr;
>>
>> -	if (port_id >= nb_ports) {
>> +	if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
>>  		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
>>  		return -1;
>>  	}
>> @@ -1376,7 +1416,7 @@ rte_eth_xstats_reset(uint8_t port_id)  {
>>  	struct rte_eth_dev *dev;
>>
>> -	if (port_id >= nb_ports) {
>> +	if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
>>  		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
>>  		return;
>>  	}
>> @@ -1398,7 +1438,7 @@ set_queue_stats_mapping(uint8_t port_id, uint16_t queue_id, uint8_t
>> stat_idx,  {
>>  	struct rte_eth_dev *dev;
>>
>> -	if (port_id >= nb_ports) {
>> +	if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
>>  		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
>>  		return -ENODEV;
>>  	}
>> @@ -1433,7 +1473,7 @@ rte_eth_dev_info_get(uint8_t port_id, struct rte_eth_dev_info *dev_info)
>> {
>>  	struct rte_eth_dev *dev;
>>
>> -	if (port_id >= nb_ports) {
>> +	if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
>>  		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
>>  		return;
>>  	}
>> @@ -1453,7 +1493,7 @@ rte_eth_macaddr_get(uint8_t port_id, struct ether_addr *mac_addr)  {
>>  	struct rte_eth_dev *dev;
>>
>> -	if (port_id >= nb_ports) {
>> +	if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
>>  		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
>>  		return;
>>  	}
>> @@ -1467,7 +1507,7 @@ rte_eth_dev_get_mtu(uint8_t port_id, uint16_t *mtu)  {
>>  	struct rte_eth_dev *dev;
>>
>> -	if (port_id >= nb_ports) {
>> +	if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
>>  		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
>>  		return (-ENODEV);
>>  	}
>> @@ -1483,7 +1523,7 @@ rte_eth_dev_set_mtu(uint8_t port_id, uint16_t mtu)
>>  	int ret;
>>  	struct rte_eth_dev *dev;
>>
>> -	if (port_id >= nb_ports) {
>> +	if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
>>  		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
>>  		return (-ENODEV);
>>  	}
>> @@ -1503,7 +1543,7 @@ rte_eth_dev_vlan_filter(uint8_t port_id, uint16_t vlan_id, int on)  {
>>  	struct rte_eth_dev *dev;
>>
>> -	if (port_id >= nb_ports) {
>> +	if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
>>  		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
>>  		return (-ENODEV);
>>  	}
>> @@ -1528,7 +1568,7 @@ rte_eth_dev_set_vlan_strip_on_queue(uint8_t port_id, uint16_t
>> rx_queue_id, int o  {
>>  	struct rte_eth_dev *dev;
>>
>> -	if (port_id >= nb_ports) {
>> +	if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
>>  		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
>>  		return (-ENODEV);
>>  	}
>> @@ -1550,7 +1590,7 @@ rte_eth_dev_set_vlan_ether_type(uint8_t port_id, uint16_t tpid)  {
>>  	struct rte_eth_dev *dev;
>>
>> -	if (port_id >= nb_ports) {
>> +	if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
>>  		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
>>  		return (-ENODEV);
>>  	}
>> @@ -1570,7 +1610,7 @@ rte_eth_dev_set_vlan_offload(uint8_t port_id, int offload_mask)
>>  	int mask = 0;
>>  	int cur, org = 0;
>>
>> -	if (port_id >= nb_ports) {
>> +	if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
>>  		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
>>  		return (-ENODEV);
>>  	}
>> @@ -1615,7 +1655,7 @@ rte_eth_dev_get_vlan_offload(uint8_t port_id)
>>  	struct rte_eth_dev *dev;
>>  	int ret = 0;
>>
>> -	if (port_id >= nb_ports) {
>> +	if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
>>  		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
>>  		return (-ENODEV);
>>  	}
>> @@ -1639,7 +1679,7 @@ rte_eth_dev_set_vlan_pvid(uint8_t port_id, uint16_t pvid, int on)  {
>>  	struct rte_eth_dev *dev;
>>
>> -	if (port_id >= nb_ports) {
>> +	if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
>>  		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
>>  		return (-ENODEV);
>>  	}
>> @@ -1657,7 +1697,7 @@ rte_eth_dev_fdir_add_signature_filter(uint8_t port_id,  {
>>  	struct rte_eth_dev *dev;
>>
>> -	if (port_id >= nb_ports) {
>> +	if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
>>  		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
>>  		return (-ENODEV);
>>  	}
>> @@ -1691,7 +1731,7 @@ rte_eth_dev_fdir_update_signature_filter(uint8_t port_id,  {
>>  	struct rte_eth_dev *dev;
>>
>> -	if (port_id >= nb_ports) {
>> +	if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
>>  		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
>>  		return (-ENODEV);
>>  	}
>> @@ -1725,7 +1765,7 @@ rte_eth_dev_fdir_remove_signature_filter(uint8_t port_id,  {
>>  	struct rte_eth_dev *dev;
>>
>> -	if (port_id >= nb_ports) {
>> +	if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
>>  		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
>>  		return (-ENODEV);
>>  	}
>> @@ -1756,7 +1796,7 @@ rte_eth_dev_fdir_get_infos(uint8_t port_id, struct rte_eth_fdir *fdir)  {
>>  	struct rte_eth_dev *dev;
>>
>> -	if (port_id >= nb_ports) {
>> +	if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
>>  		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
>>  		return (-ENODEV);
>>  	}
>> @@ -1781,7 +1821,7 @@ rte_eth_dev_fdir_add_perfect_filter(uint8_t port_id,  {
>>  	struct rte_eth_dev *dev;
>>
>> -	if (port_id >= nb_ports) {
>> +	if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
>>  		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
>>  		return (-ENODEV);
>>  	}
>> @@ -1821,7 +1861,7 @@ rte_eth_dev_fdir_update_perfect_filter(uint8_t port_id,  {
>>  	struct rte_eth_dev *dev;
>>
>> -	if (port_id >= nb_ports) {
>> +	if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
>>  		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
>>  		return (-ENODEV);
>>  	}
>> @@ -1859,7 +1899,7 @@ rte_eth_dev_fdir_remove_perfect_filter(uint8_t port_id,  {
>>  	struct rte_eth_dev *dev;
>>
>> -	if (port_id >= nb_ports) {
>> +	if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
>>  		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
>>  		return (-ENODEV);
>>  	}
>> @@ -1895,7 +1935,7 @@ rte_eth_dev_fdir_set_masks(uint8_t port_id, struct rte_fdir_masks
>> *fdir_mask)  {
>>  	struct rte_eth_dev *dev;
>>
>> -	if (port_id >= nb_ports) {
>> +	if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
>>  		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
>>  		return (-ENODEV);
>>  	}
>> @@ -1915,7 +1955,7 @@ rte_eth_dev_flow_ctrl_get(uint8_t port_id, struct rte_eth_fc_conf
>> *fc_conf)  {
>>  	struct rte_eth_dev *dev;
>>
>> -	if (port_id >= nb_ports) {
>> +	if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
>>  		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
>>  		return (-ENODEV);
>>  	}
>> @@ -1931,7 +1971,7 @@ rte_eth_dev_flow_ctrl_set(uint8_t port_id, struct rte_eth_fc_conf
>> *fc_conf)  {
>>  	struct rte_eth_dev *dev;
>>
>> -	if (port_id >= nb_ports) {
>> +	if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
>>  		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
>>  		return (-ENODEV);
>>  	}
>> @@ -1951,7 +1991,7 @@ rte_eth_dev_priority_flow_ctrl_set(uint8_t port_id, struct
>> rte_eth_pfc_conf *pfc  {
>>  	struct rte_eth_dev *dev;
>>
>> -	if (port_id >= nb_ports) {
>> +	if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
>>  		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
>>  		return (-ENODEV);
>>  	}
>> @@ -2030,7 +2070,7 @@ rte_eth_dev_rss_reta_update(uint8_t port_id,
>>  	struct rte_eth_dev *dev;
>>  	int ret;
>>
>> -	if (port_id >= nb_ports) {
>> +	if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
>>  		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
>>  		return -ENODEV;
>>  	}
>> @@ -2081,7 +2121,7 @@ rte_eth_dev_rss_hash_update(uint8_t port_id, struct rte_eth_rss_conf
>> *rss_conf)
>>  	struct rte_eth_dev *dev;
>>  	uint16_t rss_hash_protos;
>>
>> -	if (port_id >= nb_ports) {
>> +	if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
>>  		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
>>  		return (-ENODEV);
>>  	}
>> @@ -2103,7 +2143,7 @@ rte_eth_dev_rss_hash_conf_get(uint8_t port_id,  {
>>  	struct rte_eth_dev *dev;
>>
>> -	if (port_id >= nb_ports) {
>> +	if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
>>  		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
>>  		return (-ENODEV);
>>  	}
>> @@ -2118,7 +2158,7 @@ rte_eth_dev_udp_tunnel_add(uint8_t port_id,  {
>>  	struct rte_eth_dev *dev;
>>
>> -	if (port_id >= nb_ports) {
>> +	if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
>>  		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
>>  		return -ENODEV;
>>  	}
>> @@ -2144,7 +2184,7 @@ rte_eth_dev_udp_tunnel_delete(uint8_t port_id,  {
>>  	struct rte_eth_dev *dev;
>>
>> -	if (port_id >= nb_ports) {
>> +	if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
>>  		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
>>  		return -ENODEV;
>>  	}
>> @@ -2169,7 +2209,7 @@ rte_eth_led_on(uint8_t port_id)  {
>>  	struct rte_eth_dev *dev;
>>
>> -	if (port_id >= nb_ports) {
>> +	if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
>>  		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
>>  		return (-ENODEV);
>>  	}
>> @@ -2184,7 +2224,7 @@ rte_eth_led_off(uint8_t port_id)  {
>>  	struct rte_eth_dev *dev;
>>
>> -	if (port_id >= nb_ports) {
>> +	if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
>>  		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
>>  		return (-ENODEV);
>>  	}
>> @@ -2224,7 +2264,7 @@ rte_eth_dev_mac_addr_add(uint8_t port_id, struct ether_addr *addr,
>>  	int index;
>>  	uint64_t pool_mask;
>>
>> -	if (port_id >= nb_ports) {
>> +	if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
>>  		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
>>  		return (-ENODEV);
>>  	}
>> @@ -2275,7 +2315,7 @@ rte_eth_dev_mac_addr_remove(uint8_t port_id, struct ether_addr *addr)
>>  	struct rte_eth_dev *dev;
>>  	int index;
>>
>> -	if (port_id >= nb_ports) {
>> +	if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
>>  		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
>>  		return (-ENODEV);
>>  	}
>> @@ -2309,7 +2349,7 @@ rte_eth_dev_set_vf_rxmode(uint8_t port_id,  uint16_t vf,
>>  	struct rte_eth_dev *dev;
>>  	struct rte_eth_dev_info dev_info;
>>
>> -	if (port_id >= nb_ports) {
>> +	if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
>>  		PMD_DEBUG_TRACE("set VF RX mode:Invalid port_id=%d\n",
>>  				port_id);
>>  		return (-ENODEV);
>> @@ -2364,7 +2404,7 @@ rte_eth_dev_uc_hash_table_set(uint8_t port_id, struct ether_addr *addr,
>>  	int ret;
>>  	struct rte_eth_dev *dev;
>>
>> -	if (port_id >= nb_ports) {
>> +	if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
>>  		PMD_DEBUG_TRACE("unicast hash setting:Invalid port_id=%d\n",
>>  			port_id);
>>  		return (-ENODEV);
>> @@ -2417,7 +2457,7 @@ rte_eth_dev_uc_all_hash_table_set(uint8_t port_id, uint8_t on)  {
>>  	struct rte_eth_dev *dev;
>>
>> -	if (port_id >= nb_ports) {
>> +	if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
>>  		PMD_DEBUG_TRACE("unicast hash setting:Invalid port_id=%d\n",
>>  			port_id);
>>  		return (-ENODEV);
>> @@ -2436,7 +2476,7 @@ rte_eth_dev_set_vf_rx(uint8_t port_id,uint16_t vf, uint8_t on)
>>  	struct rte_eth_dev *dev;
>>  	struct rte_eth_dev_info dev_info;
>>
>> -	if (port_id >= nb_ports) {
>> +	if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
>>  		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
>>  		return (-ENODEV);
>>  	}
>> @@ -2462,7 +2502,7 @@ rte_eth_dev_set_vf_tx(uint8_t port_id,uint16_t vf, uint8_t on)
>>  	struct rte_eth_dev *dev;
>>  	struct rte_eth_dev_info dev_info;
>>
>> -	if (port_id >= nb_ports) {
>> +	if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
>>  		PMD_DEBUG_TRACE("set pool tx:Invalid port_id=%d\n", port_id);
>>  		return (-ENODEV);
>>  	}
>> @@ -2487,7 +2527,7 @@ rte_eth_dev_set_vf_vlan_filter(uint8_t port_id, uint16_t vlan_id,  {
>>  	struct rte_eth_dev *dev;
>>
>> -	if (port_id >= nb_ports) {
>> +	if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
>>  		PMD_DEBUG_TRACE("VF VLAN filter:invalid port id=%d\n",
>>  				port_id);
>>  		return (-ENODEV);
>> @@ -2518,7 +2558,7 @@ int rte_eth_set_queue_rate_limit(uint8_t port_id, uint16_t queue_idx,
>>  	struct rte_eth_dev_info dev_info;
>>  	struct rte_eth_link link;
>>
>> -	if (port_id >= nb_ports) {
>> +	if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
>>  		PMD_DEBUG_TRACE("set queue rate limit:invalid port id=%d\n",
>>  				port_id);
>>  		return -ENODEV;
>> @@ -2555,7 +2595,7 @@ int rte_eth_set_vf_rate_limit(uint8_t port_id, uint16_t vf, uint16_t tx_rate,
>>  	if (q_msk == 0)
>>  		return 0;
>>
>> -	if (port_id >= nb_ports) {
>> +	if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
>>  		PMD_DEBUG_TRACE("set VF rate limit:invalid port id=%d\n",
>>  				port_id);
>>  		return -ENODEV;
>> @@ -2589,7 +2629,7 @@ rte_eth_mirror_rule_set(uint8_t port_id,  {
>>  	struct rte_eth_dev *dev = &rte_eth_devices[port_id];
>>
>> -	if (port_id >= nb_ports) {
>> +	if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
>>  		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
>>  		return (-ENODEV);
>>  	}
>> @@ -2630,7 +2670,7 @@ rte_eth_mirror_rule_reset(uint8_t port_id, uint8_t rule_id)  {
>>  	struct rte_eth_dev *dev = &rte_eth_devices[port_id];
>>
>> -	if (port_id >= nb_ports) {
>> +	if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
>>  		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
>>  		return (-ENODEV);
>>  	}
>> @@ -2655,7 +2695,7 @@ rte_eth_rx_burst(uint8_t port_id, uint16_t queue_id,  {
>>  	struct rte_eth_dev *dev;
>>
>> -	if (port_id >= nb_ports) {
>> +	if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
>>  		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
>>  		return 0;
>>  	}
>> @@ -2675,7 +2715,7 @@ rte_eth_tx_burst(uint8_t port_id, uint16_t queue_id,  {
>>  	struct rte_eth_dev *dev;
>>
>> -	if (port_id >= nb_ports) {
>> +	if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
>>  		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
>>  		return 0;
>>  	}
>> @@ -2695,7 +2735,7 @@ rte_eth_rx_queue_count(uint8_t port_id, uint16_t queue_id)  {
>>  	struct rte_eth_dev *dev;
>>
>> -	if (port_id >= nb_ports) {
>> +	if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
>>  		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
>>  		return 0;
>>  	}
>> @@ -2709,7 +2749,7 @@ rte_eth_rx_descriptor_done(uint8_t port_id, uint16_t queue_id, uint16_t
>> offset)  {
>>  	struct rte_eth_dev *dev;
>>
>> -	if (port_id >= nb_ports) {
>> +	if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
>>  		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
>>  		return (-ENODEV);
>>  	}
>> @@ -2730,7 +2770,7 @@ rte_eth_dev_callback_register(uint8_t port_id,
>>
>>  	if (!cb_fn)
>>  		return (-EINVAL);
>> -	if (port_id >= nb_ports) {
>> +	if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
>>  		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
>>  		return (-EINVAL);
>>  	}
>> @@ -2770,7 +2810,7 @@ rte_eth_dev_callback_unregister(uint8_t port_id,
>>
>>  	if (!cb_fn)
>>  		return (-EINVAL);
>> -	if (port_id >= nb_ports) {
>> +	if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
>>  		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
>>  		return (-EINVAL);
>>  	}
>> @@ -2830,7 +2870,7 @@ int rte_eth_dev_bypass_init(uint8_t port_id)  {
>>  	struct rte_eth_dev *dev;
>>
>> -	if (port_id >= nb_ports) {
>> +	if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
>>  		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
>>  		return (-ENODEV);
>>  	}
>> @@ -2850,7 +2890,7 @@ rte_eth_dev_bypass_state_show(uint8_t port_id, uint32_t *state)  {
>>  	struct rte_eth_dev *dev;
>>
>> -	if (port_id >= nb_ports) {
>> +	if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
>>  		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
>>  		return (-ENODEV);
>>  	}
>> @@ -2869,7 +2909,7 @@ rte_eth_dev_bypass_state_set(uint8_t port_id, uint32_t *new_state)  {
>>  	struct rte_eth_dev *dev;
>>
>> -	if (port_id >= nb_ports) {
>> +	if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
>>  		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
>>  		return (-ENODEV);
>>  	}
>> @@ -2889,7 +2929,7 @@ rte_eth_dev_bypass_event_show(uint8_t port_id, uint32_t event, uint32_t
>> *state)  {
>>  	struct rte_eth_dev *dev;
>>
>> -	if (port_id >= nb_ports) {
>> +	if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
>>  		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
>>  		return (-ENODEV);
>>  	}
>> @@ -2909,7 +2949,7 @@ rte_eth_dev_bypass_event_store(uint8_t port_id, uint32_t event, uint32_t
>> state)  {
>>  	struct rte_eth_dev *dev;
>>
>> -	if (port_id >= nb_ports) {
>> +	if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
>>  		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
>>  		return (-ENODEV);
>>  	}
>> @@ -2929,7 +2969,7 @@ rte_eth_dev_wd_timeout_store(uint8_t port_id, uint32_t timeout)  {
>>  	struct rte_eth_dev *dev;
>>
>> -	if (port_id >= nb_ports) {
>> +	if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
>>  		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
>>  		return (-ENODEV);
>>  	}
>> @@ -2949,7 +2989,7 @@ rte_eth_dev_bypass_ver_show(uint8_t port_id, uint32_t *ver)  {
>>  	struct rte_eth_dev *dev;
>>
>> -	if (port_id >= nb_ports) {
>> +	if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
>>  		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
>>  		return (-ENODEV);
>>  	}
>> @@ -2969,7 +3009,7 @@ rte_eth_dev_bypass_wd_timeout_show(uint8_t port_id, uint32_t
>> *wd_timeout)  {
>>  	struct rte_eth_dev *dev;
>>
>> -	if (port_id >= nb_ports) {
>> +	if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
>>  		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
>>  		return (-ENODEV);
>>  	}
>> @@ -2989,7 +3029,7 @@ rte_eth_dev_bypass_wd_reset(uint8_t port_id)  {
>>  	struct rte_eth_dev *dev;
>>
>> -	if (port_id >= nb_ports) {
>> +	if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
>>  		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
>>  		return (-ENODEV);
>>  	}
>> @@ -3011,7 +3051,7 @@ rte_eth_dev_add_syn_filter(uint8_t port_id,  {
>>  	struct rte_eth_dev *dev;
>>
>> -	if (port_id >= nb_ports) {
>> +	if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
>>  		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
>>  		return -ENODEV;
>>  	}
>> @@ -3026,7 +3066,7 @@ rte_eth_dev_remove_syn_filter(uint8_t port_id)  {
>>  	struct rte_eth_dev *dev;
>>
>> -	if (port_id >= nb_ports) {
>> +	if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
>>  		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
>>  		return -ENODEV;
>>  	}
>> @@ -3045,7 +3085,7 @@ rte_eth_dev_get_syn_filter(uint8_t port_id,
>>  	if (filter == NULL || rx_queue == NULL)
>>  		return -EINVAL;
>>
>> -	if (port_id >= nb_ports) {
>> +	if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
>>  		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
>>  		return -ENODEV;
>>  	}
>> @@ -3061,7 +3101,7 @@ rte_eth_dev_add_ethertype_filter(uint8_t port_id, uint16_t index,  {
>>  	struct rte_eth_dev *dev;
>>
>> -	if (port_id >= nb_ports) {
>> +	if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
>>  		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
>>  		return -ENODEV;
>>  	}
>> @@ -3082,7 +3122,7 @@ rte_eth_dev_remove_ethertype_filter(uint8_t port_id,  uint16_t index)  {
>>  	struct rte_eth_dev *dev;
>>
>> -	if (port_id >= nb_ports) {
>> +	if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
>>  		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
>>  		return -ENODEV;
>>  	}
>> @@ -3101,7 +3141,7 @@ rte_eth_dev_get_ethertype_filter(uint8_t port_id, uint16_t index,
>>  	if (filter == NULL || rx_queue == NULL)
>>  		return -EINVAL;
>>
>> -	if (port_id >= nb_ports) {
>> +	if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
>>  		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
>>  		return -ENODEV;
>>  	}
>> @@ -3118,7 +3158,7 @@ rte_eth_dev_add_2tuple_filter(uint8_t port_id, uint16_t index,  {
>>  	struct rte_eth_dev *dev;
>>
>> -	if (port_id >= nb_ports) {
>> +	if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
>>  		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
>>  		return -ENODEV;
>>  	}
>> @@ -3140,7 +3180,7 @@ rte_eth_dev_remove_2tuple_filter(uint8_t port_id, uint16_t index)  {
>>  	struct rte_eth_dev *dev;
>>
>> -	if (port_id >= nb_ports) {
>> +	if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
>>  		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
>>  		return -ENODEV;
>>  	}
>> @@ -3159,7 +3199,7 @@ rte_eth_dev_get_2tuple_filter(uint8_t port_id, uint16_t index,
>>  	if (filter == NULL || rx_queue == NULL)
>>  		return -EINVAL;
>>
>> -	if (port_id >= nb_ports) {
>> +	if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
>>  		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
>>  		return -ENODEV;
>>  	}
>> @@ -3175,7 +3215,7 @@ rte_eth_dev_add_5tuple_filter(uint8_t port_id, uint16_t index,  {
>>  	struct rte_eth_dev *dev;
>>
>> -	if (port_id >= nb_ports) {
>> +	if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
>>  		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
>>  		return -ENODEV;
>>  	}
>> @@ -3198,7 +3238,7 @@ rte_eth_dev_remove_5tuple_filter(uint8_t port_id, uint16_t index)  {
>>  	struct rte_eth_dev *dev;
>>
>> -	if (port_id >= nb_ports) {
>> +	if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
>>  		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
>>  		return -ENODEV;
>>  	}
>> @@ -3217,7 +3257,7 @@ rte_eth_dev_get_5tuple_filter(uint8_t port_id, uint16_t index,
>>  	if (filter == NULL || rx_queue == NULL)
>>  		return -EINVAL;
>>
>> -	if (port_id >= nb_ports) {
>> +	if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
>>  		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
>>  		return -ENODEV;
>>  	}
>> @@ -3234,7 +3274,7 @@ rte_eth_dev_add_flex_filter(uint8_t port_id, uint16_t index,  {
>>  	struct rte_eth_dev *dev;
>>
>> -	if (port_id >= nb_ports) {
>> +	if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
>>  		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
>>  		return -ENODEV;
>>  	}
>> @@ -3249,7 +3289,7 @@ rte_eth_dev_remove_flex_filter(uint8_t port_id, uint16_t index)  {
>>  	struct rte_eth_dev *dev;
>>
>> -	if (port_id >= nb_ports) {
>> +	if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
>>  		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
>>  		return -ENODEV;
>>  	}
>> @@ -3268,7 +3308,7 @@ rte_eth_dev_get_flex_filter(uint8_t port_id, uint16_t index,
>>  	if (filter == NULL || rx_queue == NULL)
>>  		return -EINVAL;
>>
>> -	if (port_id >= nb_ports) {
>> +	if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
>>  		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
>>  		return -ENODEV;
>>  	}
>> @@ -3284,7 +3324,7 @@ rte_eth_dev_filter_supported(uint8_t port_id, enum rte_filter_type
>> filter_type)  {
>>  	struct rte_eth_dev *dev;
>>
>> -	if (port_id >= nb_ports) {
>> +	if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
>>  		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
>>  		return -ENODEV;
>>  	}
>> @@ -3301,7 +3341,7 @@ rte_eth_dev_filter_ctrl(uint8_t port_id, enum rte_filter_type filter_type,  {
>>  	struct rte_eth_dev *dev;
>>
>> -	if (port_id >= nb_ports) {
>> +	if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
>>  		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
>>  		return -ENODEV;
>>  	}
>> diff --git a/lib/librte_ether/rte_ethdev.h b/lib/librte_ether/rte_ethdev.h index ce0528f..616a44a
>> 100644
>> --- a/lib/librte_ether/rte_ethdev.h
>> +++ b/lib/librte_ether/rte_ethdev.h
>> @@ -1565,6 +1565,7 @@ struct rte_eth_dev {
>>  	struct eth_dev_ops *dev_ops;    /**< Functions exported by PMD */
>>  	struct rte_pci_device *pci_dev; /**< PCI info. supplied by probing */
>>  	struct rte_eth_dev_cb_list callbacks; /**< User application callbacks */
>> +	uint8_t attached; /**< Flag indicating the port is attached */
>>  };
>>
>>  struct rte_eth_dev_sriov {
>> @@ -1630,6 +1631,10 @@ extern struct rte_eth_dev rte_eth_devices[];
>>   * initialized by the [matching] Ethernet driver during the PCI probing phase.
>>   * All devices whose port identifier is in the range
>>   * [0,  rte_eth_dev_count() - 1] can be operated on by network applications.
>> + * immediately after invoking rte_eal_init().
>> + * If the application unplugs a port using hotplug function, The
>> + enabled port
>> + * numbers may be noncontiguous. In the case, the applications need to
>> + manage
>> + * enabled port by themselves.
>>   *
>>   * @return
>>   *   - The total number of usable Ethernet devices.
>> --
>> 1.9.1
>


^ permalink raw reply	[flat|nested] 362+ messages in thread

* Re: [dpdk-dev] FW:  [PATCH v4 01/11] eal/pci, ethdev: Remove assumption that port will not be detached
  2015-01-19 14:24       ` [dpdk-dev] FW: " Qiu, Michael
@ 2015-01-20  2:02         ` Tetsuya Mukawa
  0 siblings, 0 replies; 362+ messages in thread
From: Tetsuya Mukawa @ 2015-01-20  2:02 UTC (permalink / raw)
  To: Qiu, Michael, dev

Hi Michael,

On 2015/01/19 23:24, Qiu, Michael wrote:
> Hi, Tetsuya
>
> You see lots of places have below three lines:
>
> 	if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
>  		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
>  		return -EINVAL;
>  	}
>
> They are all the same(only few has the print log different), so can we improve it?
>
> See below:
>
> +static int
> +rte_eth_dev_validate_port(uint8_t port_id, bool trace) {
> +	if (port_id >= RTE_MAX_ETHPORTS ||
> +	    rte_eth_devices[port_id].attached != DEV_CONNECTED) {
> +		if (trace)
> +			PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
> +		return DEV_INVALID;
> +	}
> +	else
> +		return DEV_VALID;
> +}
>
> For normal case we just call this function use
> rte_eth_dev_validate_port(port_id, 1)
> here 1 could be a enmu value(Thus trace should be defined as int).
> After call, we didn't need to add PMD_DEBUG_TRACE() any more.
>  
> For few cases like:
>  		PMD_DEBUG_TRACE("set VF rate limit:invalid port id=%d\n",
>  				port_id);
>
> We can call the function with secondary param set to "0", and add the trace log after the function called, just as before.
>
> I think after this enhancement, the code seems more clean and efficiency。

I appreciate your comment.
Sounds nice. I changes like above.

Thanks,
Tetsuya


>
> Thanks,
> Michael
>
>>> -----Original Message-----
>>> From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Tetsuya Mukawa
>>> Sent: Monday, January 19, 2015 10:40 AM
>>> To: dev@dpdk.org
>>> Subject: [dpdk-dev] [PATCH v4 01/11] eal/pci, ethdev: Remove assumption that port will not be
>>> detached
>>>
>>> To remove assumption, do like followings.
>>>
>>> This patch adds "RTE_PCI_DRV_DETACHABLE" to drv_flags of rte_pci_driver structure. The flags
>>> indicates the driver can detach devices at runtime.
>>> Also remove assumption that port will not be detached.
>>>
>>> To remove the assumption.
>>> - Add 'attached' member to rte_eth_dev structure.
>>>   This member is used for indicating the port is attached, or not.
>>> - Add rte_eth_dev_allocate_new_port().
>>>   This function is used for allocating new port.
>>>
>>> v4:
>>> - Use braces with 'for' loop.
>>> - Fix indent of 'if' statement.
>>>
>>> Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp>
>>> ---
>>>  lib/librte_eal/common/include/rte_pci.h |   2 +
>>>  lib/librte_ether/rte_ethdev.c           | 248 ++++++++++++++++++--------------
>>>  lib/librte_ether/rte_ethdev.h           |   5 +
>>>  3 files changed, 151 insertions(+), 104 deletions(-)
>>>
>>> diff --git a/lib/librte_eal/common/include/rte_pci.h b/lib/librte_eal/common/include/rte_pci.h
>>> index 66ed793..dd1da28 100644
>>> --- a/lib/librte_eal/common/include/rte_pci.h
>>> +++ b/lib/librte_eal/common/include/rte_pci.h
>>> @@ -199,6 +199,8 @@ struct rte_pci_driver {  #define RTE_PCI_DRV_FORCE_UNBIND 0x0004
>>>  /** Device driver supports link state interrupt */
>>>  #define RTE_PCI_DRV_INTR_LSC	0x0008
>>> +/** Device driver supports detaching capability */
>>> +#define RTE_PCI_DRV_DETACHABLE	0x0010
>>>
>>>  /**< Internal use only - Macro used by pci addr parsing functions **/
>>>  #define GET_PCIADDR_FIELD(in, fd, lim, dlm)                   \
>>> diff --git a/lib/librte_ether/rte_ethdev.c b/lib/librte_ether/rte_ethdev.c index 077d430..46cabaf
>>> 100644
>>> --- a/lib/librte_ether/rte_ethdev.c
>>> +++ b/lib/librte_ether/rte_ethdev.c
>>> @@ -175,6 +175,16 @@ enum {
>>>  	STAT_QMAP_RX
>>>  };
>>>
>>> +enum {
>>> +	DEV_INVALID = 0,
>>> +	DEV_VALID,
>>> +};
>>> +
>>> +enum {
>>> +	DEV_DISCONNECTED = 0,
>>> +	DEV_CONNECTED
>>> +};
>>> +
>>>  static inline void
>>>  rte_eth_dev_data_alloc(void)
>>>  {
>>> @@ -201,19 +211,34 @@ rte_eth_dev_allocated(const char *name)  {
>>>  	unsigned i;
>>>
>>> -	for (i = 0; i < nb_ports; i++) {
>>> -		if (strcmp(rte_eth_devices[i].data->name, name) == 0)
>>> +	for (i = 0; i < RTE_MAX_ETHPORTS; i++) {
>>> +		if ((rte_eth_devices[i].attached == DEV_CONNECTED) &&
>>> +		    strcmp(rte_eth_devices[i].data->name, name) == 0)
>>>  			return &rte_eth_devices[i];
>>>  	}
>>>  	return NULL;
>>>  }
>>>
>>> +static uint8_t
>>> +rte_eth_dev_allocate_new_port(void)
>>> +{
>>> +	unsigned i;
>>> +
>>> +	for (i = 0; i < RTE_MAX_ETHPORTS; i++) {
>>> +		if (rte_eth_devices[i].attached == DEV_DISCONNECTED)
>>> +			return i;
>>> +	}
>>> +	return RTE_MAX_ETHPORTS;
>>> +}
>>> +
>>>  struct rte_eth_dev *
>>>  rte_eth_dev_allocate(const char *name)
>>>  {
>>> +	uint8_t port_id;
>>>  	struct rte_eth_dev *eth_dev;
>>>
>>> -	if (nb_ports == RTE_MAX_ETHPORTS) {
>>> +	port_id = rte_eth_dev_allocate_new_port();
>>> +	if (port_id == RTE_MAX_ETHPORTS) {
>>>  		PMD_DEBUG_TRACE("Reached maximum number of Ethernet ports\n");
>>>  		return NULL;
>>>  	}
>>> @@ -226,10 +251,12 @@ rte_eth_dev_allocate(const char *name)
>>>  		return NULL;
>>>  	}
>>>
>>> -	eth_dev = &rte_eth_devices[nb_ports];
>>> -	eth_dev->data = &rte_eth_dev_data[nb_ports];
>>> +	eth_dev = &rte_eth_devices[port_id];
>>> +	eth_dev->data = &rte_eth_dev_data[port_id];
>>>  	snprintf(eth_dev->data->name, sizeof(eth_dev->data->name), "%s", name);
>>> -	eth_dev->data->port_id = nb_ports++;
>>> +	eth_dev->data->port_id = port_id;
>>> +	eth_dev->attached = DEV_CONNECTED;
>>> +	nb_ports++;
>>>  	return eth_dev;
>>>  }
>>>
>>> @@ -283,6 +310,7 @@ rte_eth_dev_init(struct rte_pci_driver *pci_drv,
>>>  			(unsigned) pci_dev->id.device_id);
>>>  	if (rte_eal_process_type() == RTE_PROC_PRIMARY)
>>>  		rte_free(eth_dev->data->dev_private);
>>> +	eth_dev->attached = DEV_DISCONNECTED;
>>>  	nb_ports--;
>>>  	return diag;
>>>  }
>>> @@ -308,10 +336,22 @@ rte_eth_driver_register(struct eth_driver *eth_drv)
>>>  	rte_eal_pci_register(&eth_drv->pci_drv);
>>>  }
>>>
>>> +static int
>>> +rte_eth_dev_validate_port(uint8_t port_id) {
>>> +	if (port_id >= RTE_MAX_ETHPORTS)
>>> +		return DEV_INVALID;
>>> +
>>> +	if (rte_eth_devices[port_id].attached == DEV_CONNECTED)
>>> +		return DEV_VALID;
>>> +	else
>>> +		return DEV_INVALID;
>>> +}
>>> +
>>>  int
>>>  rte_eth_dev_socket_id(uint8_t port_id)
>>>  {
>>> -	if (port_id >= nb_ports)
>>> +	if (rte_eth_dev_validate_port(port_id) == DEV_INVALID)
>>>  		return -1;
>>>  	return rte_eth_devices[port_id].pci_dev->numa_node;
>>>  }
>>> @@ -369,7 +409,7 @@ rte_eth_dev_rx_queue_start(uint8_t port_id, uint16_t rx_queue_id)
>>>  	 * in a multi-process setup*/
>>>  	PROC_PRIMARY_OR_ERR_RET(-E_RTE_SECONDARY);
>>>
>>> -	if (port_id >= nb_ports) {
>>> +	if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
>>>  		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
>>>  		return -EINVAL;
>>>  	}
>>> @@ -395,7 +435,7 @@ rte_eth_dev_rx_queue_stop(uint8_t port_id, uint16_t rx_queue_id)
>>>  	 * in a multi-process setup*/
>>>  	PROC_PRIMARY_OR_ERR_RET(-E_RTE_SECONDARY);
>>>
>>> -	if (port_id >= nb_ports) {
>>> +	if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
>>>  		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
>>>  		return -EINVAL;
>>>  	}
>>> @@ -421,7 +461,7 @@ rte_eth_dev_tx_queue_start(uint8_t port_id, uint16_t tx_queue_id)
>>>  	 * in a multi-process setup*/
>>>  	PROC_PRIMARY_OR_ERR_RET(-E_RTE_SECONDARY);
>>>
>>> -	if (port_id >= nb_ports) {
>>> +	if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
>>>  		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
>>>  		return -EINVAL;
>>>  	}
>>> @@ -447,7 +487,7 @@ rte_eth_dev_tx_queue_stop(uint8_t port_id, uint16_t tx_queue_id)
>>>  	 * in a multi-process setup*/
>>>  	PROC_PRIMARY_OR_ERR_RET(-E_RTE_SECONDARY);
>>>
>>> -	if (port_id >= nb_ports) {
>>> +	if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
>>>  		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
>>>  		return -EINVAL;
>>>  	}
>>> @@ -703,7 +743,7 @@ rte_eth_dev_configure(uint8_t port_id, uint16_t nb_rx_q, uint16_t nb_tx_q,
>>>  	 * in a multi-process setup*/
>>>  	PROC_PRIMARY_OR_ERR_RET(-E_RTE_SECONDARY);
>>>
>>> -	if (port_id >= nb_ports || port_id >= RTE_MAX_ETHPORTS) {
>>> +	if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
>>>  		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
>>>  		return (-EINVAL);
>>>  	}
>>> @@ -888,7 +928,7 @@ rte_eth_dev_start(uint8_t port_id)
>>>  	 * in a multi-process setup*/
>>>  	PROC_PRIMARY_OR_ERR_RET(-E_RTE_SECONDARY);
>>>
>>> -	if (port_id >= nb_ports) {
>>> +	if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
>>>  		PMD_DEBUG_TRACE("Invalid port_id=%" PRIu8 "\n", port_id);
>>>  		return (-EINVAL);
>>>  	}
>>> @@ -923,7 +963,7 @@ rte_eth_dev_stop(uint8_t port_id)
>>>  	 * in a multi-process setup*/
>>>  	PROC_PRIMARY_OR_RET();
>>>
>>> -	if (port_id >= nb_ports) {
>>> +	if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
>>>  		PMD_DEBUG_TRACE("Invalid port_id=%" PRIu8 "\n", port_id);
>>>  		return;
>>>  	}
>>> @@ -951,7 +991,7 @@ rte_eth_dev_set_link_up(uint8_t port_id)
>>>  	 * in a multi-process setup*/
>>>  	PROC_PRIMARY_OR_ERR_RET(-E_RTE_SECONDARY);
>>>
>>> -	if (port_id >= nb_ports) {
>>> +	if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
>>>  		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
>>>  		return -EINVAL;
>>>  	}
>>> @@ -970,7 +1010,7 @@ rte_eth_dev_set_link_down(uint8_t port_id)
>>>  	 * in a multi-process setup*/
>>>  	PROC_PRIMARY_OR_ERR_RET(-E_RTE_SECONDARY);
>>>
>>> -	if (port_id >= nb_ports) {
>>> +	if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
>>>  		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
>>>  		return -EINVAL;
>>>  	}
>>> @@ -989,7 +1029,7 @@ rte_eth_dev_close(uint8_t port_id)
>>>  	 * in a multi-process setup*/
>>>  	PROC_PRIMARY_OR_RET();
>>>
>>> -	if (port_id >= nb_ports) {
>>> +	if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
>>>  		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
>>>  		return;
>>>  	}
>>> @@ -1017,7 +1057,7 @@ rte_eth_rx_queue_setup(uint8_t port_id, uint16_t rx_queue_id,
>>>  	 * in a multi-process setup*/
>>>  	PROC_PRIMARY_OR_ERR_RET(-E_RTE_SECONDARY);
>>>
>>> -	if (port_id >= nb_ports) {
>>> +	if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
>>>  		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
>>>  		return (-EINVAL);
>>>  	}
>>> @@ -1090,7 +1130,7 @@ rte_eth_tx_queue_setup(uint8_t port_id, uint16_t tx_queue_id,
>>>  	 * in a multi-process setup*/
>>>  	PROC_PRIMARY_OR_ERR_RET(-E_RTE_SECONDARY);
>>>
>>> -	if (port_id >= RTE_MAX_ETHPORTS || port_id >= nb_ports) {
>>> +	if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
>>>  		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
>>>  		return (-EINVAL);
>>>  	}
>>> @@ -1123,7 +1163,7 @@ rte_eth_promiscuous_enable(uint8_t port_id)  {
>>>  	struct rte_eth_dev *dev;
>>>
>>> -	if (port_id >= nb_ports) {
>>> +	if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
>>>  		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
>>>  		return;
>>>  	}
>>> @@ -1139,7 +1179,7 @@ rte_eth_promiscuous_disable(uint8_t port_id)  {
>>>  	struct rte_eth_dev *dev;
>>>
>>> -	if (port_id >= nb_ports) {
>>> +	if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
>>>  		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
>>>  		return;
>>>  	}
>>> @@ -1155,7 +1195,7 @@ rte_eth_promiscuous_get(uint8_t port_id)  {
>>>  	struct rte_eth_dev *dev;
>>>
>>> -	if (port_id >= nb_ports) {
>>> +	if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
>>>  		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
>>>  		return -1;
>>>  	}
>>> @@ -1169,7 +1209,7 @@ rte_eth_allmulticast_enable(uint8_t port_id)  {
>>>  	struct rte_eth_dev *dev;
>>>
>>> -	if (port_id >= nb_ports) {
>>> +	if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
>>>  		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
>>>  		return;
>>>  	}
>>> @@ -1185,7 +1225,7 @@ rte_eth_allmulticast_disable(uint8_t port_id)  {
>>>  	struct rte_eth_dev *dev;
>>>
>>> -	if (port_id >= nb_ports) {
>>> +	if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
>>>  		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
>>>  		return;
>>>  	}
>>> @@ -1201,7 +1241,7 @@ rte_eth_allmulticast_get(uint8_t port_id)  {
>>>  	struct rte_eth_dev *dev;
>>>
>>> -	if (port_id >= nb_ports) {
>>> +	if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
>>>  		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
>>>  		return -1;
>>>  	}
>>> @@ -1229,7 +1269,7 @@ rte_eth_link_get(uint8_t port_id, struct rte_eth_link *eth_link)  {
>>>  	struct rte_eth_dev *dev;
>>>
>>> -	if (port_id >= nb_ports) {
>>> +	if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
>>>  		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
>>>  		return;
>>>  	}
>>> @@ -1249,7 +1289,7 @@ rte_eth_link_get_nowait(uint8_t port_id, struct rte_eth_link *eth_link)  {
>>>  	struct rte_eth_dev *dev;
>>>
>>> -	if (port_id >= nb_ports) {
>>> +	if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
>>>  		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
>>>  		return;
>>>  	}
>>> @@ -1269,7 +1309,7 @@ rte_eth_stats_get(uint8_t port_id, struct rte_eth_stats *stats)  {
>>>  	struct rte_eth_dev *dev;
>>>
>>> -	if (port_id >= nb_ports) {
>>> +	if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
>>>  		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
>>>  		return;
>>>  	}
>>> @@ -1286,7 +1326,7 @@ rte_eth_stats_reset(uint8_t port_id)  {
>>>  	struct rte_eth_dev *dev;
>>>
>>> -	if (port_id >= nb_ports) {
>>> +	if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
>>>  		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
>>>  		return;
>>>  	}
>>> @@ -1307,7 +1347,7 @@ rte_eth_xstats_get(uint8_t port_id, struct rte_eth_xstats *xstats,
>>>  	uint64_t val;
>>>  	char *stats_ptr;
>>>
>>> -	if (port_id >= nb_ports) {
>>> +	if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
>>>  		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
>>>  		return -1;
>>>  	}
>>> @@ -1376,7 +1416,7 @@ rte_eth_xstats_reset(uint8_t port_id)  {
>>>  	struct rte_eth_dev *dev;
>>>
>>> -	if (port_id >= nb_ports) {
>>> +	if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
>>>  		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
>>>  		return;
>>>  	}
>>> @@ -1398,7 +1438,7 @@ set_queue_stats_mapping(uint8_t port_id, uint16_t queue_id, uint8_t
>>> stat_idx,  {
>>>  	struct rte_eth_dev *dev;
>>>
>>> -	if (port_id >= nb_ports) {
>>> +	if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
>>>  		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
>>>  		return -ENODEV;
>>>  	}
>>> @@ -1433,7 +1473,7 @@ rte_eth_dev_info_get(uint8_t port_id, struct rte_eth_dev_info *dev_info)
>>> {
>>>  	struct rte_eth_dev *dev;
>>>
>>> -	if (port_id >= nb_ports) {
>>> +	if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
>>>  		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
>>>  		return;
>>>  	}
>>> @@ -1453,7 +1493,7 @@ rte_eth_macaddr_get(uint8_t port_id, struct ether_addr *mac_addr)  {
>>>  	struct rte_eth_dev *dev;
>>>
>>> -	if (port_id >= nb_ports) {
>>> +	if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
>>>  		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
>>>  		return;
>>>  	}
>>> @@ -1467,7 +1507,7 @@ rte_eth_dev_get_mtu(uint8_t port_id, uint16_t *mtu)  {
>>>  	struct rte_eth_dev *dev;
>>>
>>> -	if (port_id >= nb_ports) {
>>> +	if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
>>>  		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
>>>  		return (-ENODEV);
>>>  	}
>>> @@ -1483,7 +1523,7 @@ rte_eth_dev_set_mtu(uint8_t port_id, uint16_t mtu)
>>>  	int ret;
>>>  	struct rte_eth_dev *dev;
>>>
>>> -	if (port_id >= nb_ports) {
>>> +	if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
>>>  		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
>>>  		return (-ENODEV);
>>>  	}
>>> @@ -1503,7 +1543,7 @@ rte_eth_dev_vlan_filter(uint8_t port_id, uint16_t vlan_id, int on)  {
>>>  	struct rte_eth_dev *dev;
>>>
>>> -	if (port_id >= nb_ports) {
>>> +	if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
>>>  		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
>>>  		return (-ENODEV);
>>>  	}
>>> @@ -1528,7 +1568,7 @@ rte_eth_dev_set_vlan_strip_on_queue(uint8_t port_id, uint16_t
>>> rx_queue_id, int o  {
>>>  	struct rte_eth_dev *dev;
>>>
>>> -	if (port_id >= nb_ports) {
>>> +	if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
>>>  		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
>>>  		return (-ENODEV);
>>>  	}
>>> @@ -1550,7 +1590,7 @@ rte_eth_dev_set_vlan_ether_type(uint8_t port_id, uint16_t tpid)  {
>>>  	struct rte_eth_dev *dev;
>>>
>>> -	if (port_id >= nb_ports) {
>>> +	if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
>>>  		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
>>>  		return (-ENODEV);
>>>  	}
>>> @@ -1570,7 +1610,7 @@ rte_eth_dev_set_vlan_offload(uint8_t port_id, int offload_mask)
>>>  	int mask = 0;
>>>  	int cur, org = 0;
>>>
>>> -	if (port_id >= nb_ports) {
>>> +	if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
>>>  		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
>>>  		return (-ENODEV);
>>>  	}
>>> @@ -1615,7 +1655,7 @@ rte_eth_dev_get_vlan_offload(uint8_t port_id)
>>>  	struct rte_eth_dev *dev;
>>>  	int ret = 0;
>>>
>>> -	if (port_id >= nb_ports) {
>>> +	if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
>>>  		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
>>>  		return (-ENODEV);
>>>  	}
>>> @@ -1639,7 +1679,7 @@ rte_eth_dev_set_vlan_pvid(uint8_t port_id, uint16_t pvid, int on)  {
>>>  	struct rte_eth_dev *dev;
>>>
>>> -	if (port_id >= nb_ports) {
>>> +	if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
>>>  		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
>>>  		return (-ENODEV);
>>>  	}
>>> @@ -1657,7 +1697,7 @@ rte_eth_dev_fdir_add_signature_filter(uint8_t port_id,  {
>>>  	struct rte_eth_dev *dev;
>>>
>>> -	if (port_id >= nb_ports) {
>>> +	if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
>>>  		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
>>>  		return (-ENODEV);
>>>  	}
>>> @@ -1691,7 +1731,7 @@ rte_eth_dev_fdir_update_signature_filter(uint8_t port_id,  {
>>>  	struct rte_eth_dev *dev;
>>>
>>> -	if (port_id >= nb_ports) {
>>> +	if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
>>>  		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
>>>  		return (-ENODEV);
>>>  	}
>>> @@ -1725,7 +1765,7 @@ rte_eth_dev_fdir_remove_signature_filter(uint8_t port_id,  {
>>>  	struct rte_eth_dev *dev;
>>>
>>> -	if (port_id >= nb_ports) {
>>> +	if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
>>>  		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
>>>  		return (-ENODEV);
>>>  	}
>>> @@ -1756,7 +1796,7 @@ rte_eth_dev_fdir_get_infos(uint8_t port_id, struct rte_eth_fdir *fdir)  {
>>>  	struct rte_eth_dev *dev;
>>>
>>> -	if (port_id >= nb_ports) {
>>> +	if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
>>>  		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
>>>  		return (-ENODEV);
>>>  	}
>>> @@ -1781,7 +1821,7 @@ rte_eth_dev_fdir_add_perfect_filter(uint8_t port_id,  {
>>>  	struct rte_eth_dev *dev;
>>>
>>> -	if (port_id >= nb_ports) {
>>> +	if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
>>>  		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
>>>  		return (-ENODEV);
>>>  	}
>>> @@ -1821,7 +1861,7 @@ rte_eth_dev_fdir_update_perfect_filter(uint8_t port_id,  {
>>>  	struct rte_eth_dev *dev;
>>>
>>> -	if (port_id >= nb_ports) {
>>> +	if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
>>>  		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
>>>  		return (-ENODEV);
>>>  	}
>>> @@ -1859,7 +1899,7 @@ rte_eth_dev_fdir_remove_perfect_filter(uint8_t port_id,  {
>>>  	struct rte_eth_dev *dev;
>>>
>>> -	if (port_id >= nb_ports) {
>>> +	if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
>>>  		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
>>>  		return (-ENODEV);
>>>  	}
>>> @@ -1895,7 +1935,7 @@ rte_eth_dev_fdir_set_masks(uint8_t port_id, struct rte_fdir_masks
>>> *fdir_mask)  {
>>>  	struct rte_eth_dev *dev;
>>>
>>> -	if (port_id >= nb_ports) {
>>> +	if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
>>>  		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
>>>  		return (-ENODEV);
>>>  	}
>>> @@ -1915,7 +1955,7 @@ rte_eth_dev_flow_ctrl_get(uint8_t port_id, struct rte_eth_fc_conf
>>> *fc_conf)  {
>>>  	struct rte_eth_dev *dev;
>>>
>>> -	if (port_id >= nb_ports) {
>>> +	if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
>>>  		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
>>>  		return (-ENODEV);
>>>  	}
>>> @@ -1931,7 +1971,7 @@ rte_eth_dev_flow_ctrl_set(uint8_t port_id, struct rte_eth_fc_conf
>>> *fc_conf)  {
>>>  	struct rte_eth_dev *dev;
>>>
>>> -	if (port_id >= nb_ports) {
>>> +	if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
>>>  		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
>>>  		return (-ENODEV);
>>>  	}
>>> @@ -1951,7 +1991,7 @@ rte_eth_dev_priority_flow_ctrl_set(uint8_t port_id, struct
>>> rte_eth_pfc_conf *pfc  {
>>>  	struct rte_eth_dev *dev;
>>>
>>> -	if (port_id >= nb_ports) {
>>> +	if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
>>>  		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
>>>  		return (-ENODEV);
>>>  	}
>>> @@ -2030,7 +2070,7 @@ rte_eth_dev_rss_reta_update(uint8_t port_id,
>>>  	struct rte_eth_dev *dev;
>>>  	int ret;
>>>
>>> -	if (port_id >= nb_ports) {
>>> +	if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
>>>  		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
>>>  		return -ENODEV;
>>>  	}
>>> @@ -2081,7 +2121,7 @@ rte_eth_dev_rss_hash_update(uint8_t port_id, struct rte_eth_rss_conf
>>> *rss_conf)
>>>  	struct rte_eth_dev *dev;
>>>  	uint16_t rss_hash_protos;
>>>
>>> -	if (port_id >= nb_ports) {
>>> +	if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
>>>  		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
>>>  		return (-ENODEV);
>>>  	}
>>> @@ -2103,7 +2143,7 @@ rte_eth_dev_rss_hash_conf_get(uint8_t port_id,  {
>>>  	struct rte_eth_dev *dev;
>>>
>>> -	if (port_id >= nb_ports) {
>>> +	if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
>>>  		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
>>>  		return (-ENODEV);
>>>  	}
>>> @@ -2118,7 +2158,7 @@ rte_eth_dev_udp_tunnel_add(uint8_t port_id,  {
>>>  	struct rte_eth_dev *dev;
>>>
>>> -	if (port_id >= nb_ports) {
>>> +	if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
>>>  		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
>>>  		return -ENODEV;
>>>  	}
>>> @@ -2144,7 +2184,7 @@ rte_eth_dev_udp_tunnel_delete(uint8_t port_id,  {
>>>  	struct rte_eth_dev *dev;
>>>
>>> -	if (port_id >= nb_ports) {
>>> +	if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
>>>  		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
>>>  		return -ENODEV;
>>>  	}
>>> @@ -2169,7 +2209,7 @@ rte_eth_led_on(uint8_t port_id)  {
>>>  	struct rte_eth_dev *dev;
>>>
>>> -	if (port_id >= nb_ports) {
>>> +	if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
>>>  		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
>>>  		return (-ENODEV);
>>>  	}
>>> @@ -2184,7 +2224,7 @@ rte_eth_led_off(uint8_t port_id)  {
>>>  	struct rte_eth_dev *dev;
>>>
>>> -	if (port_id >= nb_ports) {
>>> +	if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
>>>  		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
>>>  		return (-ENODEV);
>>>  	}
>>> @@ -2224,7 +2264,7 @@ rte_eth_dev_mac_addr_add(uint8_t port_id, struct ether_addr *addr,
>>>  	int index;
>>>  	uint64_t pool_mask;
>>>
>>> -	if (port_id >= nb_ports) {
>>> +	if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
>>>  		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
>>>  		return (-ENODEV);
>>>  	}
>>> @@ -2275,7 +2315,7 @@ rte_eth_dev_mac_addr_remove(uint8_t port_id, struct ether_addr *addr)
>>>  	struct rte_eth_dev *dev;
>>>  	int index;
>>>
>>> -	if (port_id >= nb_ports) {
>>> +	if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
>>>  		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
>>>  		return (-ENODEV);
>>>  	}
>>> @@ -2309,7 +2349,7 @@ rte_eth_dev_set_vf_rxmode(uint8_t port_id,  uint16_t vf,
>>>  	struct rte_eth_dev *dev;
>>>  	struct rte_eth_dev_info dev_info;
>>>
>>> -	if (port_id >= nb_ports) {
>>> +	if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
>>>  		PMD_DEBUG_TRACE("set VF RX mode:Invalid port_id=%d\n",
>>>  				port_id);
>>>  		return (-ENODEV);
>>> @@ -2364,7 +2404,7 @@ rte_eth_dev_uc_hash_table_set(uint8_t port_id, struct ether_addr *addr,
>>>  	int ret;
>>>  	struct rte_eth_dev *dev;
>>>
>>> -	if (port_id >= nb_ports) {
>>> +	if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
>>>  		PMD_DEBUG_TRACE("unicast hash setting:Invalid port_id=%d\n",
>>>  			port_id);
>>>  		return (-ENODEV);
>>> @@ -2417,7 +2457,7 @@ rte_eth_dev_uc_all_hash_table_set(uint8_t port_id, uint8_t on)  {
>>>  	struct rte_eth_dev *dev;
>>>
>>> -	if (port_id >= nb_ports) {
>>> +	if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
>>>  		PMD_DEBUG_TRACE("unicast hash setting:Invalid port_id=%d\n",
>>>  			port_id);
>>>  		return (-ENODEV);
>>> @@ -2436,7 +2476,7 @@ rte_eth_dev_set_vf_rx(uint8_t port_id,uint16_t vf, uint8_t on)
>>>  	struct rte_eth_dev *dev;
>>>  	struct rte_eth_dev_info dev_info;
>>>
>>> -	if (port_id >= nb_ports) {
>>> +	if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
>>>  		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
>>>  		return (-ENODEV);
>>>  	}
>>> @@ -2462,7 +2502,7 @@ rte_eth_dev_set_vf_tx(uint8_t port_id,uint16_t vf, uint8_t on)
>>>  	struct rte_eth_dev *dev;
>>>  	struct rte_eth_dev_info dev_info;
>>>
>>> -	if (port_id >= nb_ports) {
>>> +	if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
>>>  		PMD_DEBUG_TRACE("set pool tx:Invalid port_id=%d\n", port_id);
>>>  		return (-ENODEV);
>>>  	}
>>> @@ -2487,7 +2527,7 @@ rte_eth_dev_set_vf_vlan_filter(uint8_t port_id, uint16_t vlan_id,  {
>>>  	struct rte_eth_dev *dev;
>>>
>>> -	if (port_id >= nb_ports) {
>>> +	if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
>>>  		PMD_DEBUG_TRACE("VF VLAN filter:invalid port id=%d\n",
>>>  				port_id);
>>>  		return (-ENODEV);
>>> @@ -2518,7 +2558,7 @@ int rte_eth_set_queue_rate_limit(uint8_t port_id, uint16_t queue_idx,
>>>  	struct rte_eth_dev_info dev_info;
>>>  	struct rte_eth_link link;
>>>
>>> -	if (port_id >= nb_ports) {
>>> +	if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
>>>  		PMD_DEBUG_TRACE("set queue rate limit:invalid port id=%d\n",
>>>  				port_id);
>>>  		return -ENODEV;
>>> @@ -2555,7 +2595,7 @@ int rte_eth_set_vf_rate_limit(uint8_t port_id, uint16_t vf, uint16_t tx_rate,
>>>  	if (q_msk == 0)
>>>  		return 0;
>>>
>>> -	if (port_id >= nb_ports) {
>>> +	if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
>>>  		PMD_DEBUG_TRACE("set VF rate limit:invalid port id=%d\n",
>>>  				port_id);
>>>  		return -ENODEV;
>>> @@ -2589,7 +2629,7 @@ rte_eth_mirror_rule_set(uint8_t port_id,  {
>>>  	struct rte_eth_dev *dev = &rte_eth_devices[port_id];
>>>
>>> -	if (port_id >= nb_ports) {
>>> +	if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
>>>  		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
>>>  		return (-ENODEV);
>>>  	}
>>> @@ -2630,7 +2670,7 @@ rte_eth_mirror_rule_reset(uint8_t port_id, uint8_t rule_id)  {
>>>  	struct rte_eth_dev *dev = &rte_eth_devices[port_id];
>>>
>>> -	if (port_id >= nb_ports) {
>>> +	if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
>>>  		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
>>>  		return (-ENODEV);
>>>  	}
>>> @@ -2655,7 +2695,7 @@ rte_eth_rx_burst(uint8_t port_id, uint16_t queue_id,  {
>>>  	struct rte_eth_dev *dev;
>>>
>>> -	if (port_id >= nb_ports) {
>>> +	if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
>>>  		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
>>>  		return 0;
>>>  	}
>>> @@ -2675,7 +2715,7 @@ rte_eth_tx_burst(uint8_t port_id, uint16_t queue_id,  {
>>>  	struct rte_eth_dev *dev;
>>>
>>> -	if (port_id >= nb_ports) {
>>> +	if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
>>>  		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
>>>  		return 0;
>>>  	}
>>> @@ -2695,7 +2735,7 @@ rte_eth_rx_queue_count(uint8_t port_id, uint16_t queue_id)  {
>>>  	struct rte_eth_dev *dev;
>>>
>>> -	if (port_id >= nb_ports) {
>>> +	if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
>>>  		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
>>>  		return 0;
>>>  	}
>>> @@ -2709,7 +2749,7 @@ rte_eth_rx_descriptor_done(uint8_t port_id, uint16_t queue_id, uint16_t
>>> offset)  {
>>>  	struct rte_eth_dev *dev;
>>>
>>> -	if (port_id >= nb_ports) {
>>> +	if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
>>>  		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
>>>  		return (-ENODEV);
>>>  	}
>>> @@ -2730,7 +2770,7 @@ rte_eth_dev_callback_register(uint8_t port_id,
>>>
>>>  	if (!cb_fn)
>>>  		return (-EINVAL);
>>> -	if (port_id >= nb_ports) {
>>> +	if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
>>>  		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
>>>  		return (-EINVAL);
>>>  	}
>>> @@ -2770,7 +2810,7 @@ rte_eth_dev_callback_unregister(uint8_t port_id,
>>>
>>>  	if (!cb_fn)
>>>  		return (-EINVAL);
>>> -	if (port_id >= nb_ports) {
>>> +	if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
>>>  		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
>>>  		return (-EINVAL);
>>>  	}
>>> @@ -2830,7 +2870,7 @@ int rte_eth_dev_bypass_init(uint8_t port_id)  {
>>>  	struct rte_eth_dev *dev;
>>>
>>> -	if (port_id >= nb_ports) {
>>> +	if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
>>>  		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
>>>  		return (-ENODEV);
>>>  	}
>>> @@ -2850,7 +2890,7 @@ rte_eth_dev_bypass_state_show(uint8_t port_id, uint32_t *state)  {
>>>  	struct rte_eth_dev *dev;
>>>
>>> -	if (port_id >= nb_ports) {
>>> +	if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
>>>  		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
>>>  		return (-ENODEV);
>>>  	}
>>> @@ -2869,7 +2909,7 @@ rte_eth_dev_bypass_state_set(uint8_t port_id, uint32_t *new_state)  {
>>>  	struct rte_eth_dev *dev;
>>>
>>> -	if (port_id >= nb_ports) {
>>> +	if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
>>>  		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
>>>  		return (-ENODEV);
>>>  	}
>>> @@ -2889,7 +2929,7 @@ rte_eth_dev_bypass_event_show(uint8_t port_id, uint32_t event, uint32_t
>>> *state)  {
>>>  	struct rte_eth_dev *dev;
>>>
>>> -	if (port_id >= nb_ports) {
>>> +	if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
>>>  		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
>>>  		return (-ENODEV);
>>>  	}
>>> @@ -2909,7 +2949,7 @@ rte_eth_dev_bypass_event_store(uint8_t port_id, uint32_t event, uint32_t
>>> state)  {
>>>  	struct rte_eth_dev *dev;
>>>
>>> -	if (port_id >= nb_ports) {
>>> +	if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
>>>  		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
>>>  		return (-ENODEV);
>>>  	}
>>> @@ -2929,7 +2969,7 @@ rte_eth_dev_wd_timeout_store(uint8_t port_id, uint32_t timeout)  {
>>>  	struct rte_eth_dev *dev;
>>>
>>> -	if (port_id >= nb_ports) {
>>> +	if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
>>>  		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
>>>  		return (-ENODEV);
>>>  	}
>>> @@ -2949,7 +2989,7 @@ rte_eth_dev_bypass_ver_show(uint8_t port_id, uint32_t *ver)  {
>>>  	struct rte_eth_dev *dev;
>>>
>>> -	if (port_id >= nb_ports) {
>>> +	if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
>>>  		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
>>>  		return (-ENODEV);
>>>  	}
>>> @@ -2969,7 +3009,7 @@ rte_eth_dev_bypass_wd_timeout_show(uint8_t port_id, uint32_t
>>> *wd_timeout)  {
>>>  	struct rte_eth_dev *dev;
>>>
>>> -	if (port_id >= nb_ports) {
>>> +	if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
>>>  		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
>>>  		return (-ENODEV);
>>>  	}
>>> @@ -2989,7 +3029,7 @@ rte_eth_dev_bypass_wd_reset(uint8_t port_id)  {
>>>  	struct rte_eth_dev *dev;
>>>
>>> -	if (port_id >= nb_ports) {
>>> +	if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
>>>  		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
>>>  		return (-ENODEV);
>>>  	}
>>> @@ -3011,7 +3051,7 @@ rte_eth_dev_add_syn_filter(uint8_t port_id,  {
>>>  	struct rte_eth_dev *dev;
>>>
>>> -	if (port_id >= nb_ports) {
>>> +	if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
>>>  		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
>>>  		return -ENODEV;
>>>  	}
>>> @@ -3026,7 +3066,7 @@ rte_eth_dev_remove_syn_filter(uint8_t port_id)  {
>>>  	struct rte_eth_dev *dev;
>>>
>>> -	if (port_id >= nb_ports) {
>>> +	if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
>>>  		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
>>>  		return -ENODEV;
>>>  	}
>>> @@ -3045,7 +3085,7 @@ rte_eth_dev_get_syn_filter(uint8_t port_id,
>>>  	if (filter == NULL || rx_queue == NULL)
>>>  		return -EINVAL;
>>>
>>> -	if (port_id >= nb_ports) {
>>> +	if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
>>>  		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
>>>  		return -ENODEV;
>>>  	}
>>> @@ -3061,7 +3101,7 @@ rte_eth_dev_add_ethertype_filter(uint8_t port_id, uint16_t index,  {
>>>  	struct rte_eth_dev *dev;
>>>
>>> -	if (port_id >= nb_ports) {
>>> +	if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
>>>  		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
>>>  		return -ENODEV;
>>>  	}
>>> @@ -3082,7 +3122,7 @@ rte_eth_dev_remove_ethertype_filter(uint8_t port_id,  uint16_t index)  {
>>>  	struct rte_eth_dev *dev;
>>>
>>> -	if (port_id >= nb_ports) {
>>> +	if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
>>>  		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
>>>  		return -ENODEV;
>>>  	}
>>> @@ -3101,7 +3141,7 @@ rte_eth_dev_get_ethertype_filter(uint8_t port_id, uint16_t index,
>>>  	if (filter == NULL || rx_queue == NULL)
>>>  		return -EINVAL;
>>>
>>> -	if (port_id >= nb_ports) {
>>> +	if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
>>>  		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
>>>  		return -ENODEV;
>>>  	}
>>> @@ -3118,7 +3158,7 @@ rte_eth_dev_add_2tuple_filter(uint8_t port_id, uint16_t index,  {
>>>  	struct rte_eth_dev *dev;
>>>
>>> -	if (port_id >= nb_ports) {
>>> +	if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
>>>  		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
>>>  		return -ENODEV;
>>>  	}
>>> @@ -3140,7 +3180,7 @@ rte_eth_dev_remove_2tuple_filter(uint8_t port_id, uint16_t index)  {
>>>  	struct rte_eth_dev *dev;
>>>
>>> -	if (port_id >= nb_ports) {
>>> +	if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
>>>  		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
>>>  		return -ENODEV;
>>>  	}
>>> @@ -3159,7 +3199,7 @@ rte_eth_dev_get_2tuple_filter(uint8_t port_id, uint16_t index,
>>>  	if (filter == NULL || rx_queue == NULL)
>>>  		return -EINVAL;
>>>
>>> -	if (port_id >= nb_ports) {
>>> +	if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
>>>  		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
>>>  		return -ENODEV;
>>>  	}
>>> @@ -3175,7 +3215,7 @@ rte_eth_dev_add_5tuple_filter(uint8_t port_id, uint16_t index,  {
>>>  	struct rte_eth_dev *dev;
>>>
>>> -	if (port_id >= nb_ports) {
>>> +	if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
>>>  		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
>>>  		return -ENODEV;
>>>  	}
>>> @@ -3198,7 +3238,7 @@ rte_eth_dev_remove_5tuple_filter(uint8_t port_id, uint16_t index)  {
>>>  	struct rte_eth_dev *dev;
>>>
>>> -	if (port_id >= nb_ports) {
>>> +	if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
>>>  		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
>>>  		return -ENODEV;
>>>  	}
>>> @@ -3217,7 +3257,7 @@ rte_eth_dev_get_5tuple_filter(uint8_t port_id, uint16_t index,
>>>  	if (filter == NULL || rx_queue == NULL)
>>>  		return -EINVAL;
>>>
>>> -	if (port_id >= nb_ports) {
>>> +	if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
>>>  		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
>>>  		return -ENODEV;
>>>  	}
>>> @@ -3234,7 +3274,7 @@ rte_eth_dev_add_flex_filter(uint8_t port_id, uint16_t index,  {
>>>  	struct rte_eth_dev *dev;
>>>
>>> -	if (port_id >= nb_ports) {
>>> +	if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
>>>  		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
>>>  		return -ENODEV;
>>>  	}
>>> @@ -3249,7 +3289,7 @@ rte_eth_dev_remove_flex_filter(uint8_t port_id, uint16_t index)  {
>>>  	struct rte_eth_dev *dev;
>>>
>>> -	if (port_id >= nb_ports) {
>>> +	if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
>>>  		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
>>>  		return -ENODEV;
>>>  	}
>>> @@ -3268,7 +3308,7 @@ rte_eth_dev_get_flex_filter(uint8_t port_id, uint16_t index,
>>>  	if (filter == NULL || rx_queue == NULL)
>>>  		return -EINVAL;
>>>
>>> -	if (port_id >= nb_ports) {
>>> +	if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
>>>  		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
>>>  		return -ENODEV;
>>>  	}
>>> @@ -3284,7 +3324,7 @@ rte_eth_dev_filter_supported(uint8_t port_id, enum rte_filter_type
>>> filter_type)  {
>>>  	struct rte_eth_dev *dev;
>>>
>>> -	if (port_id >= nb_ports) {
>>> +	if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
>>>  		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
>>>  		return -ENODEV;
>>>  	}
>>> @@ -3301,7 +3341,7 @@ rte_eth_dev_filter_ctrl(uint8_t port_id, enum rte_filter_type filter_type,  {
>>>  	struct rte_eth_dev *dev;
>>>
>>> -	if (port_id >= nb_ports) {
>>> +	if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
>>>  		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
>>>  		return -ENODEV;
>>>  	}
>>> diff --git a/lib/librte_ether/rte_ethdev.h b/lib/librte_ether/rte_ethdev.h index ce0528f..616a44a
>>> 100644
>>> --- a/lib/librte_ether/rte_ethdev.h
>>> +++ b/lib/librte_ether/rte_ethdev.h
>>> @@ -1565,6 +1565,7 @@ struct rte_eth_dev {
>>>  	struct eth_dev_ops *dev_ops;    /**< Functions exported by PMD */
>>>  	struct rte_pci_device *pci_dev; /**< PCI info. supplied by probing */
>>>  	struct rte_eth_dev_cb_list callbacks; /**< User application callbacks */
>>> +	uint8_t attached; /**< Flag indicating the port is attached */
>>>  };
>>>
>>>  struct rte_eth_dev_sriov {
>>> @@ -1630,6 +1631,10 @@ extern struct rte_eth_dev rte_eth_devices[];
>>>   * initialized by the [matching] Ethernet driver during the PCI probing phase.
>>>   * All devices whose port identifier is in the range
>>>   * [0,  rte_eth_dev_count() - 1] can be operated on by network applications.
>>> + * immediately after invoking rte_eal_init().
>>> + * If the application unplugs a port using hotplug function, The
>>> + enabled port
>>> + * numbers may be noncontiguous. In the case, the applications need to
>>> + manage
>>> + * enabled port by themselves.
>>>   *
>>>   * @return
>>>   *   - The total number of usable Ethernet devices.
>>> --
>>> 1.9.1

^ permalink raw reply	[flat|nested] 362+ messages in thread

* Re: [dpdk-dev] [PATCH v4 06/11] eal/linux/pci: Add functions for unmapping igb_uio resources
  2015-01-19 10:40   ` [dpdk-dev] [PATCH v4 06/11] eal/linux/pci: Add functions for unmapping igb_uio resources Tetsuya Mukawa
@ 2015-01-20  9:23     ` Qiu, Michael
  2015-01-21  0:17       ` Tetsuya Mukawa
  2015-01-21 10:01       ` Tetsuya Mukawa
  0 siblings, 2 replies; 362+ messages in thread
From: Qiu, Michael @ 2015-01-20  9:23 UTC (permalink / raw)
  To: Tetsuya Mukawa, dev

On 1/19/2015 6:42 PM, Tetsuya Mukawa wrote:
> The patch adds functions for unmapping igb_uio resources. The patch is only
> for Linux and igb_uio environment. VFIO and BSD are not supported.
>
> v4:
> - Add paramerter checking.
> - Add header file to determine if hotplug can be enabled.
>
> Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp>
> ---
>  lib/librte_eal/common/Makefile                  |  1 +
>  lib/librte_eal/common/include/rte_dev_hotplug.h | 44 +++++++++++++++++
>  lib/librte_eal/linuxapp/eal/eal_pci.c           | 38 +++++++++++++++
>  lib/librte_eal/linuxapp/eal/eal_pci_init.h      |  8 +++
>  lib/librte_eal/linuxapp/eal/eal_pci_uio.c       | 65 +++++++++++++++++++++++++
>  5 files changed, 156 insertions(+)
>  create mode 100644 lib/librte_eal/common/include/rte_dev_hotplug.h
>
> diff --git a/lib/librte_eal/common/Makefile b/lib/librte_eal/common/Makefile
> index 52c1a5f..db7cc93 100644
> --- a/lib/librte_eal/common/Makefile
> +++ b/lib/librte_eal/common/Makefile
> @@ -41,6 +41,7 @@ INC += rte_eal_memconfig.h rte_malloc_heap.h
>  INC += rte_hexdump.h rte_devargs.h rte_dev.h
>  INC += rte_common_vect.h
>  INC += rte_pci_dev_feature_defs.h rte_pci_dev_features.h
> +INC += rte_dev_hotplug.h
>  
>  ifeq ($(CONFIG_RTE_INSECURE_FUNCTION_WARNING),y)
>  INC += rte_warnings.h
> diff --git a/lib/librte_eal/common/include/rte_dev_hotplug.h b/lib/librte_eal/common/include/rte_dev_hotplug.h
> new file mode 100644
> index 0000000..b333e0f
> --- /dev/null
> +++ b/lib/librte_eal/common/include/rte_dev_hotplug.h
> @@ -0,0 +1,44 @@
> +/*-
> + *   BSD LICENSE
> + *
> + *   Copyright(c) 2015 IGEL Co.,LTd.
> + *   All rights reserved.
> + *
> + *   Redistribution and use in source and binary forms, with or without
> + *   modification, are permitted provided that the following conditions
> + *   are met:
> + *
> + *     * Redistributions of source code must retain the above copyright
> + *       notice, this list of conditions and the following disclaimer.
> + *     * Redistributions in binary form must reproduce the above copyright
> + *       notice, this list of conditions and the following disclaimer in
> + *       the documentation and/or other materials provided with the
> + *       distribution.
> + *     * Neither the name of IGEL Co.,Ltd. nor the names of its
> + *       contributors may be used to endorse or promote products derived
> + *       from this software without specific prior written permission.
> + *
> + *   THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
> + *   "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
> + *   LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
> + *   A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
> + *   OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
> + *   SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
> + *   LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
> + *   DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
> + *   THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
> + *   (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
> + *   OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
> + */
> +
> +#ifndef _RTE_DEV_HOTPLUG_H_
> +#define _RTE_DEV_HOTPLUG_H_
> +
> +/*
> + * determine if hotplug can be enabled on the system
> + */
> +#if defined(RTE_LIBRTE_EAL_HOTPLUG) && defined(RTE_LIBRTE_EAL_LINUXAPP)

As you said, VFIO should not work with it, so does it need to add the
vfio check here?

Thanks,
Michael
> +#define ENABLE_HOTPLUG
> +#endif /* RTE_LIBRTE_EAL_HOTPLUG & RTE_LIBRTE_EAL_LINUXAPP */
> +
> +#endif /* _RTE_DEV_HOTPLUG_H_ */
> diff --git a/lib/librte_eal/linuxapp/eal/eal_pci.c b/lib/librte_eal/linuxapp/eal/eal_pci.c
> index 3d2d93c..52c464c 100644
> --- a/lib/librte_eal/linuxapp/eal/eal_pci.c
> +++ b/lib/librte_eal/linuxapp/eal/eal_pci.c
> @@ -137,6 +137,25 @@ pci_map_resource(void *requested_addr, int fd, off_t offset, size_t size)
>  	return mapaddr;
>  }
>  
> +#ifdef ENABLE_HOTPLUG
> +/* unmap a particular resource */
> +void
> +pci_unmap_resource(void *requested_addr, size_t size)
> +{
> +	if (requested_addr == NULL)
> +		return;
> +
> +	/* Unmap the PCI memory resource of device */
> +	if (munmap(requested_addr, size)) {
> +		RTE_LOG(ERR, EAL, "%s(): cannot munmap(%p, 0x%lx): %s\n",
> +			__func__, requested_addr, (unsigned long)size,
> +			strerror(errno));
> +	} else
> +		RTE_LOG(DEBUG, EAL, "  PCI memory mapped at %p\n",
> +				requested_addr);
> +}
> +#endif /* ENABLE_HOTPLUG */
> +
>  /* parse the "resource" sysfs file */
>  #define IORESOURCE_MEM  0x00000200
>  
> @@ -510,6 +529,25 @@ pci_map_device(struct rte_pci_device *dev)
>  	return 0;
>  }
>  
> +#ifdef ENABLE_HOTPLUG
> +static void
> +pci_unmap_device(struct rte_pci_device *dev)
> +{
> +	if (dev == NULL)
> +		return;
> +
> +	/* try unmapping the NIC resources using VFIO if it exists */
> +#ifdef VFIO_PRESENT
> +	if (pci_vfio_is_enabled()) {
> +		RTE_LOG(ERR, EAL, "%s() doesn't support vfio yet.\n",
> +				__func__);
> +		return;
> +	}
> +#endif
> +	pci_uio_unmap_resource(dev);
> +}
> +#endif /* ENABLE_HOTPLUG */
> +
>  /*
>   * If vendor/device ID match, call the devinit() function of the
>   * driver.
> diff --git a/lib/librte_eal/linuxapp/eal/eal_pci_init.h b/lib/librte_eal/linuxapp/eal/eal_pci_init.h
> index 1070eb8..5152a0b 100644
> --- a/lib/librte_eal/linuxapp/eal/eal_pci_init.h
> +++ b/lib/librte_eal/linuxapp/eal/eal_pci_init.h
> @@ -34,6 +34,7 @@
>  #ifndef EAL_PCI_INIT_H_
>  #define EAL_PCI_INIT_H_
>  
> +#include <rte_dev_hotplug.h>
>  #include "eal_vfio.h"
>  
>  struct pci_map {
> @@ -71,6 +72,13 @@ void *pci_map_resource(void *requested_addr, int fd, off_t offset,
>  /* map IGB_UIO resource prototype */
>  int pci_uio_map_resource(struct rte_pci_device *dev);
>  
> +#ifdef ENABLE_HOTPLUG
> +void pci_unmap_resource(void *requested_addr, size_t size);
> +
> +/* unmap IGB_UIO resource prototype */
> +void pci_uio_unmap_resource(struct rte_pci_device *dev);
> +#endif /* ENABLE_HOTPLUG */
> +
>  #ifdef VFIO_PRESENT
>  
>  #define VFIO_MAX_GROUPS 64
> diff --git a/lib/librte_eal/linuxapp/eal/eal_pci_uio.c b/lib/librte_eal/linuxapp/eal/eal_pci_uio.c
> index 1da3507..81830d1 100644
> --- a/lib/librte_eal/linuxapp/eal/eal_pci_uio.c
> +++ b/lib/librte_eal/linuxapp/eal/eal_pci_uio.c
> @@ -404,6 +404,71 @@ pci_uio_map_resource(struct rte_pci_device *dev)
>  	return 0;
>  }
>  
> +#ifdef ENABLE_HOTPLUG
> +static void
> +pci_uio_unmap(struct mapped_pci_resource *uio_res)
> +{
> +	int i;
> +
> +	if (uio_res == NULL)
> +		return;
> +
> +	for (i = 0; i != uio_res->nb_maps; i++)
> +		pci_unmap_resource(uio_res->maps[i].addr,
> +				(size_t)uio_res->maps[i].size);
> +}
> +
> +static struct mapped_pci_resource *
> +pci_uio_find_resource(struct rte_pci_device *dev)
> +{
> +	struct mapped_pci_resource *uio_res;
> +
> +	if (dev == NULL)
> +		return NULL;
> +
> +	TAILQ_FOREACH(uio_res, pci_res_list, next) {
> +
> +		/* skip this element if it doesn't match our PCI address */
> +		if (!eal_compare_pci_addr(&uio_res->pci_addr, &dev->addr))
> +			return uio_res;
> +	}
> +	return NULL;
> +}
> +
> +/* unmap the PCI resource of a PCI device in virtual memory */
> +void
> +pci_uio_unmap_resource(struct rte_pci_device *dev)
> +{
> +	struct mapped_pci_resource *uio_res;
> +
> +	if (dev == NULL)
> +		return;
> +
> +	/* find an entry for the device */
> +	uio_res = pci_uio_find_resource(dev);
> +	if (uio_res == NULL)
> +		return;
> +
> +	/* secondary processes - just free maps */
> +	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
> +		return pci_uio_unmap(uio_res);
> +
> +	TAILQ_REMOVE(pci_res_list, uio_res, next);
> +
> +	/* unmap all resources */
> +	pci_uio_unmap(uio_res);
> +
> +	/* free uio resource */
> +	rte_free(uio_res);
> +
> +	/* close fd if in primary process */
> +	close(dev->intr_handle.fd);
> +
> +	dev->intr_handle.fd = -1;
> +	dev->intr_handle.type = RTE_INTR_HANDLE_UNKNOWN;
> +}
> +#endif /* ENABLE_HOTPLUG */
> +
>  /*
>   * parse a sysfs file containing one integer value
>   * different to the eal version, as it needs to work with 64-bit values


^ permalink raw reply	[flat|nested] 362+ messages in thread

* Re: [dpdk-dev] [PATCH v4 06/11] eal/linux/pci: Add functions for unmapping igb_uio resources
  2015-01-20  9:23     ` Qiu, Michael
@ 2015-01-21  0:17       ` Tetsuya Mukawa
  2015-01-21 10:01       ` Tetsuya Mukawa
  1 sibling, 0 replies; 362+ messages in thread
From: Tetsuya Mukawa @ 2015-01-21  0:17 UTC (permalink / raw)
  To: Qiu, Michael, dev

Hi Michael,

On 2015/01/20 18:23, Qiu, Michael wrote:
> On 1/19/2015 6:42 PM, Tetsuya Mukawa wrote:
>> The patch adds functions for unmapping igb_uio resources. The patch is only
>> for Linux and igb_uio environment. VFIO and BSD are not supported.
>>
>> v4:
>> - Add paramerter checking.
>> - Add header file to determine if hotplug can be enabled.
>>
>> Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp>
>> ---
>>  lib/librte_eal/common/Makefile                  |  1 +
>>  lib/librte_eal/common/include/rte_dev_hotplug.h | 44 +++++++++++++++++
>>  lib/librte_eal/linuxapp/eal/eal_pci.c           | 38 +++++++++++++++
>>  lib/librte_eal/linuxapp/eal/eal_pci_init.h      |  8 +++
>>  lib/librte_eal/linuxapp/eal/eal_pci_uio.c       | 65 +++++++++++++++++++++++++
>>  5 files changed, 156 insertions(+)
>>  create mode 100644 lib/librte_eal/common/include/rte_dev_hotplug.h
>>
>> diff --git a/lib/librte_eal/common/Makefile b/lib/librte_eal/common/Makefile
>> index 52c1a5f..db7cc93 100644
>> --- a/lib/librte_eal/common/Makefile
>> +++ b/lib/librte_eal/common/Makefile
>> @@ -41,6 +41,7 @@ INC += rte_eal_memconfig.h rte_malloc_heap.h
>>  INC += rte_hexdump.h rte_devargs.h rte_dev.h
>>  INC += rte_common_vect.h
>>  INC += rte_pci_dev_feature_defs.h rte_pci_dev_features.h
>> +INC += rte_dev_hotplug.h
>>  
>>  ifeq ($(CONFIG_RTE_INSECURE_FUNCTION_WARNING),y)
>>  INC += rte_warnings.h
>> diff --git a/lib/librte_eal/common/include/rte_dev_hotplug.h b/lib/librte_eal/common/include/rte_dev_hotplug.h
>> new file mode 100644
>> index 0000000..b333e0f
>> --- /dev/null
>> +++ b/lib/librte_eal/common/include/rte_dev_hotplug.h
>> @@ -0,0 +1,44 @@
>> +/*-
>> + *   BSD LICENSE
>> + *
>> + *   Copyright(c) 2015 IGEL Co.,LTd.
>> + *   All rights reserved.
>> + *
>> + *   Redistribution and use in source and binary forms, with or without
>> + *   modification, are permitted provided that the following conditions
>> + *   are met:
>> + *
>> + *     * Redistributions of source code must retain the above copyright
>> + *       notice, this list of conditions and the following disclaimer.
>> + *     * Redistributions in binary form must reproduce the above copyright
>> + *       notice, this list of conditions and the following disclaimer in
>> + *       the documentation and/or other materials provided with the
>> + *       distribution.
>> + *     * Neither the name of IGEL Co.,Ltd. nor the names of its
>> + *       contributors may be used to endorse or promote products derived
>> + *       from this software without specific prior written permission.
>> + *
>> + *   THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
>> + *   "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
>> + *   LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
>> + *   A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
>> + *   OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
>> + *   SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
>> + *   LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
>> + *   DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
>> + *   THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
>> + *   (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
>> + *   OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
>> + */
>> +
>> +#ifndef _RTE_DEV_HOTPLUG_H_
>> +#define _RTE_DEV_HOTPLUG_H_
>> +
>> +/*
>> + * determine if hotplug can be enabled on the system
>> + */
>> +#if defined(RTE_LIBRTE_EAL_HOTPLUG) && defined(RTE_LIBRTE_EAL_LINUXAPP)
> As you said, VFIO should not work with it, so does it need to add the
> vfio check here?
I appreciate your comment.
Yes, it should be. I will fix it in next version.

Thanks,
Tetsuya

> Thanks,
> Michael
>> +#define ENABLE_HOTPLUG
>> +#endif /* RTE_LIBRTE_EAL_HOTPLUG & RTE_LIBRTE_EAL_LINUXAPP */
>> +
>> +#endif /* _RTE_DEV_HOTPLUG_H_ */
>> diff --git a/lib/librte_eal/linuxapp/eal/eal_pci.c b/lib/librte_eal/linuxapp/eal/eal_pci.c
>> index 3d2d93c..52c464c 100644
>> --- a/lib/librte_eal/linuxapp/eal/eal_pci.c
>> +++ b/lib/librte_eal/linuxapp/eal/eal_pci.c
>> @@ -137,6 +137,25 @@ pci_map_resource(void *requested_addr, int fd, off_t offset, size_t size)
>>  	return mapaddr;
>>  }
>>  
>> +#ifdef ENABLE_HOTPLUG
>> +/* unmap a particular resource */
>> +void
>> +pci_unmap_resource(void *requested_addr, size_t size)
>> +{
>> +	if (requested_addr == NULL)
>> +		return;
>> +
>> +	/* Unmap the PCI memory resource of device */
>> +	if (munmap(requested_addr, size)) {
>> +		RTE_LOG(ERR, EAL, "%s(): cannot munmap(%p, 0x%lx): %s\n",
>> +			__func__, requested_addr, (unsigned long)size,
>> +			strerror(errno));
>> +	} else
>> +		RTE_LOG(DEBUG, EAL, "  PCI memory mapped at %p\n",
>> +				requested_addr);
>> +}
>> +#endif /* ENABLE_HOTPLUG */
>> +
>>  /* parse the "resource" sysfs file */
>>  #define IORESOURCE_MEM  0x00000200
>>  
>> @@ -510,6 +529,25 @@ pci_map_device(struct rte_pci_device *dev)
>>  	return 0;
>>  }
>>  
>> +#ifdef ENABLE_HOTPLUG
>> +static void
>> +pci_unmap_device(struct rte_pci_device *dev)
>> +{
>> +	if (dev == NULL)
>> +		return;
>> +
>> +	/* try unmapping the NIC resources using VFIO if it exists */
>> +#ifdef VFIO_PRESENT
>> +	if (pci_vfio_is_enabled()) {
>> +		RTE_LOG(ERR, EAL, "%s() doesn't support vfio yet.\n",
>> +				__func__);
>> +		return;
>> +	}
>> +#endif
>> +	pci_uio_unmap_resource(dev);
>> +}
>> +#endif /* ENABLE_HOTPLUG */
>> +
>>  /*
>>   * If vendor/device ID match, call the devinit() function of the
>>   * driver.
>> diff --git a/lib/librte_eal/linuxapp/eal/eal_pci_init.h b/lib/librte_eal/linuxapp/eal/eal_pci_init.h
>> index 1070eb8..5152a0b 100644
>> --- a/lib/librte_eal/linuxapp/eal/eal_pci_init.h
>> +++ b/lib/librte_eal/linuxapp/eal/eal_pci_init.h
>> @@ -34,6 +34,7 @@
>>  #ifndef EAL_PCI_INIT_H_
>>  #define EAL_PCI_INIT_H_
>>  
>> +#include <rte_dev_hotplug.h>
>>  #include "eal_vfio.h"
>>  
>>  struct pci_map {
>> @@ -71,6 +72,13 @@ void *pci_map_resource(void *requested_addr, int fd, off_t offset,
>>  /* map IGB_UIO resource prototype */
>>  int pci_uio_map_resource(struct rte_pci_device *dev);
>>  
>> +#ifdef ENABLE_HOTPLUG
>> +void pci_unmap_resource(void *requested_addr, size_t size);
>> +
>> +/* unmap IGB_UIO resource prototype */
>> +void pci_uio_unmap_resource(struct rte_pci_device *dev);
>> +#endif /* ENABLE_HOTPLUG */
>> +
>>  #ifdef VFIO_PRESENT
>>  
>>  #define VFIO_MAX_GROUPS 64
>> diff --git a/lib/librte_eal/linuxapp/eal/eal_pci_uio.c b/lib/librte_eal/linuxapp/eal/eal_pci_uio.c
>> index 1da3507..81830d1 100644
>> --- a/lib/librte_eal/linuxapp/eal/eal_pci_uio.c
>> +++ b/lib/librte_eal/linuxapp/eal/eal_pci_uio.c
>> @@ -404,6 +404,71 @@ pci_uio_map_resource(struct rte_pci_device *dev)
>>  	return 0;
>>  }
>>  
>> +#ifdef ENABLE_HOTPLUG
>> +static void
>> +pci_uio_unmap(struct mapped_pci_resource *uio_res)
>> +{
>> +	int i;
>> +
>> +	if (uio_res == NULL)
>> +		return;
>> +
>> +	for (i = 0; i != uio_res->nb_maps; i++)
>> +		pci_unmap_resource(uio_res->maps[i].addr,
>> +				(size_t)uio_res->maps[i].size);
>> +}
>> +
>> +static struct mapped_pci_resource *
>> +pci_uio_find_resource(struct rte_pci_device *dev)
>> +{
>> +	struct mapped_pci_resource *uio_res;
>> +
>> +	if (dev == NULL)
>> +		return NULL;
>> +
>> +	TAILQ_FOREACH(uio_res, pci_res_list, next) {
>> +
>> +		/* skip this element if it doesn't match our PCI address */
>> +		if (!eal_compare_pci_addr(&uio_res->pci_addr, &dev->addr))
>> +			return uio_res;
>> +	}
>> +	return NULL;
>> +}
>> +
>> +/* unmap the PCI resource of a PCI device in virtual memory */
>> +void
>> +pci_uio_unmap_resource(struct rte_pci_device *dev)
>> +{
>> +	struct mapped_pci_resource *uio_res;
>> +
>> +	if (dev == NULL)
>> +		return;
>> +
>> +	/* find an entry for the device */
>> +	uio_res = pci_uio_find_resource(dev);
>> +	if (uio_res == NULL)
>> +		return;
>> +
>> +	/* secondary processes - just free maps */
>> +	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
>> +		return pci_uio_unmap(uio_res);
>> +
>> +	TAILQ_REMOVE(pci_res_list, uio_res, next);
>> +
>> +	/* unmap all resources */
>> +	pci_uio_unmap(uio_res);
>> +
>> +	/* free uio resource */
>> +	rte_free(uio_res);
>> +
>> +	/* close fd if in primary process */
>> +	close(dev->intr_handle.fd);
>> +
>> +	dev->intr_handle.fd = -1;
>> +	dev->intr_handle.type = RTE_INTR_HANDLE_UNKNOWN;
>> +}
>> +#endif /* ENABLE_HOTPLUG */
>> +
>>  /*
>>   * parse a sysfs file containing one integer value
>>   * different to the eal version, as it needs to work with 64-bit values

^ permalink raw reply	[flat|nested] 362+ messages in thread

* Re: [dpdk-dev] [PATCH v4 05/11] ethdev: Add functions that will be used by port hotplug functions
  2015-01-19 10:40   ` [dpdk-dev] [PATCH v4 05/11] ethdev: Add functions that will be used by port hotplug functions Tetsuya Mukawa
@ 2015-01-21  2:40     ` Qiu, Michael
  2015-01-21  9:02       ` Tetsuya Mukawa
  0 siblings, 1 reply; 362+ messages in thread
From: Qiu, Michael @ 2015-01-21  2:40 UTC (permalink / raw)
  To: Tetsuya Mukawa, dev

On 1/19/2015 6:42 PM, Tetsuya Mukawa wrote:
> The patch adds following functions.
>
> - rte_eth_dev_save()
>   The function is used for saving current rte_eth_dev structures.
> - rte_eth_dev_get_changed_port()
>   The function receives the rte_eth_dev structures, then compare
>   these with current values to know which port is actually
>   attached or detached.
> - rte_eth_dev_get_addr_by_port()
>   The function returns a pci address of a ethdev specified by port
>   identifier.
> - rte_eth_dev_get_port_by_addr()
>   The function returns a port identifier of a ethdev specified by
>   pci address.
> - rte_eth_dev_get_name_by_port()
>   The function returns a unique identifier name of a ethdev
>   specified by port identifier.
> - Add rte_eth_dev_check_detachable()
>   The function returns whether a PMD supports detach function.
>
> Also the patch changes scope of rte_eth_dev_allocated() to global.
> This function will be called by virtual PMDs to support port hotplug.
> So change scope of the function to global.
>
> v4:
> - Add paramerter checking.
> v3:
> - Fix if-condition bug while comparing pci addresses.
> - Add error checking codes.
> Reported-by: Mark Enright <menrigh@brocade.com>
>
> Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp>
> ---
>  lib/librte_ether/rte_ethdev.c | 102 +++++++++++++++++++++++++++++++++++++++++-
>  lib/librte_ether/rte_ethdev.h |  80 +++++++++++++++++++++++++++++++++
>  2 files changed, 181 insertions(+), 1 deletion(-)
>
> diff --git a/lib/librte_ether/rte_ethdev.c b/lib/librte_ether/rte_ethdev.c
> index e5145b7..e572ef4 100644
> --- a/lib/librte_ether/rte_ethdev.c
> +++ b/lib/librte_ether/rte_ethdev.c
> @@ -206,7 +206,7 @@ rte_eth_dev_data_alloc(void)
>  				RTE_MAX_ETHPORTS * sizeof(*rte_eth_dev_data));
>  }
>  
> -static struct rte_eth_dev *
> +struct rte_eth_dev *
>  rte_eth_dev_allocated(const char *name)
>  {
>  	unsigned i;
> @@ -422,6 +422,106 @@ rte_eth_dev_count(void)
>  	return (nb_ports);
>  }
>  
> +void
> +rte_eth_dev_save(struct rte_eth_dev *devs)
> +{
> +	if (devs == NULL)
> +		return;
> +
> +	/* save current rte_eth_devices */
> +	memcpy(devs, rte_eth_devices,
> +			sizeof(struct rte_eth_dev) * RTE_MAX_ETHPORTS);
> +}
> +
> +int
> +rte_eth_dev_get_changed_port(struct rte_eth_dev *devs, uint8_t *port_id)
> +{
> +	if ((devs == NULL) || (port_id == NULL))
> +		return -1;
> +
> +	/* check which port was attached or detached */
> +	for (*port_id = 0; *port_id < RTE_MAX_ETHPORTS; (*port_id)++, devs++) {
> +		if (rte_eth_devices[*port_id].attached ^ devs->attached)
> +			return 0;
> +	}
> +	return 1;
> +}
> +
> +int
> +rte_eth_dev_get_addr_by_port(uint8_t port_id, struct rte_pci_addr *addr)
> +{
> +	if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
> +		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
> +		return -EINVAL;
> +	}
> +
> +	if (addr == NULL) {
> +		PMD_DEBUG_TRACE("Null pointer is specified\n");
> +		return -EINVAL;
> +	}
> +
> +	*addr = rte_eth_devices[port_id].pci_dev->addr;
> +	return 0;
> +}
> +
> +int
> +rte_eth_dev_get_port_by_addr(struct rte_pci_addr *addr, uint8_t *port_id)
> +{
> +	struct rte_pci_addr *tmp;
> +
> +	if ((addr == NULL) || (port_id == NULL)) {
> +		PMD_DEBUG_TRACE("Null pointer is specified\n");
> +		return -1;

Is it better to replace "-1" to "-EINVAL" ?

> +	}
> +
> +	for (*port_id = 0; *port_id < RTE_MAX_ETHPORTS; (*port_id)++) {
> +		if (!rte_eth_devices[*port_id].attached)
> +			continue;
> +		if (!rte_eth_devices[*port_id].pci_dev)
> +			continue;
> +		tmp = &rte_eth_devices[*port_id].pci_dev->addr;
> +		if (eal_compare_pci_addr(tmp, addr) == 0)
> +			return 0;
> +	}
> +	return -1;
> +}
> +
> +int
> +rte_eth_dev_get_name_by_port(uint8_t port_id, char *name)
> +{
> +	char *tmp;
> +
> +	if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
> +		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
> +		return -EINVAL;
> +	}
> +
> +	if (name == NULL) {
> +		PMD_DEBUG_TRACE("Null pointer is specified\n");
> +		return -EINVAL;
> +	}
> +
> +	/* shouldn't check 'rte_eth_devices[i].data',
> +	 * because it might be overwritten by VDEV PMD */
> +	tmp = rte_eth_dev_data[port_id].name;
> +	strncpy(name, tmp, strlen(tmp) + 1);
> +	return 0;
> +}
> +
> +int
> +rte_eth_dev_check_detachable(uint8_t port_id)
> +{
> +	uint32_t drv_flags;
> +
> +	if (port_id >= RTE_MAX_ETHPORTS) {
> +		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
> +		return -EINVAL;
> +	}
> +
> +	drv_flags = rte_eth_devices[port_id].driver->pci_drv.drv_flags;
> +	return !(drv_flags & RTE_PCI_DRV_DETACHABLE);
> +}
> +
>  static int
>  rte_eth_dev_rx_queue_config(struct rte_eth_dev *dev, uint16_t nb_queues)
>  {
> diff --git a/lib/librte_ether/rte_ethdev.h b/lib/librte_ether/rte_ethdev.h
> index 2a8ff26..c0b98dc 100644
> --- a/lib/librte_ether/rte_ethdev.h
> +++ b/lib/librte_ether/rte_ethdev.h
> @@ -1642,6 +1642,86 @@ extern struct rte_eth_dev rte_eth_devices[];
>  extern uint8_t rte_eth_dev_count(void);
>  
>  /**
> + * Function for internal use by port hotplug functions.
> + * Copies current ethdev structures to the specified pointer.
> + *
> + * @param	devs	The pointer to the ethdev structures
> + */
> +extern void rte_eth_dev_save(struct rte_eth_dev *devs);
> +
> +/**
> + * Function for internal use by port hotplug functions.
> + * Compare the specified ethdev structures with currents. Then
> + * if there is a port which status is changed, fill the specified pointer
> + * with the port id of that port.
> + * @param	devs	The pointer to the ethdev structures
> + * @param	port_id	The pointer to the port id
> + * @return
> + *   - 0 on success, negative on error
> + */
> +extern int rte_eth_dev_get_changed_port(
> +		struct rte_eth_dev *devs, uint8_t *port_id);
> +
> +/**
> + * Function for internal use by port hotplug functions.
> + * Returns a pci address of a ethdev specified by port identifier.
> + * @param	port_id
> + *   The port identifier of the Ethernet device
> + * @param	addr
> + *   The pointer to the pci address
> + * @return
> + *   - 0 on success, negative on error
> + */
> +extern int rte_eth_dev_get_addr_by_port(
> +		uint8_t port_id, struct rte_pci_addr *addr);
> +
> +/**
> + * Function for internal use by port hotplug functions.
> + * Returns a port identifier of a ethdev specified by pci address.
> + * @param	addr
> + *   The pointer to the pci address of the Ethernet device.
> + * @param	port_id
> + *   The pointer to the port identifier
> + * @return
> + *   - 0 on success, negative on error
> + */
> +extern int rte_eth_dev_get_port_by_addr(
> +		struct rte_pci_addr *addr, uint8_t *port_id);
> +
> +/**
> + * Function for internal use by port hotplug functions.
> + * Returns a unique identifier name of a ethdev specified by port identifier.
> + * @param	port_id
> + *   The port identifier.
> + * @param	name
> + *  The pointer to the Unique identifier name for each Ethernet device
> + * @return
> + *   - 0 on success, negative on error
> + */
> +extern int rte_eth_dev_get_name_by_port(uint8_t port_id, char *name);
> +
> +/**
> + * Function for internal use by port hotplug functions.
> + * Check whether or not, a PMD that is handling the ethdev specified by port
> + * identifier can support detach function.
> + * @param	port_id
> + *   The port identifier
> + * @return
> + *   - 0 on supporting detach function, negative on not supporting
> + */
> +extern int rte_eth_dev_check_detachable(uint8_t port_id);
> +
> +/**
> + * Function for internal use by port hotplug functions.
> + * Returns a ethdev slot specified by the unique identifier name.
> + * @param	name
> + *  The pointer to the Unique identifier name for each Ethernet device
> + * @return
> + *   - The pointer to the ethdev slot, on success. NULL on error
> + */
> +extern struct rte_eth_dev *rte_eth_dev_allocated(const char *name);
> +
> +/**
>   * Function for internal use by dummy drivers primarily, e.g. ring-based
>   * driver.
>   * Allocates a new ethdev slot for an ethernet device and returns the pointer


^ permalink raw reply	[flat|nested] 362+ messages in thread

* Re: [dpdk-dev] [PATCH v4 07/11] eal/pci: Add a function to remove the entry of devargs list
  2015-01-19 10:40   ` [dpdk-dev] [PATCH v4 07/11] eal/pci: Add a function to remove the entry of devargs list Tetsuya Mukawa
@ 2015-01-21  2:55     ` Qiu, Michael
  2015-01-21  6:33       ` Tetsuya Mukawa
  0 siblings, 1 reply; 362+ messages in thread
From: Qiu, Michael @ 2015-01-21  2:55 UTC (permalink / raw)
  To: Tetsuya Mukawa, dev

On 1/19/2015 6:42 PM, Tetsuya Mukawa wrote:
> The function removes the specified devargs entry from devargs_list.
> Also the patch adds sanity checking to rte_eal_devargs_add().
>
> v4:
> - Fix sanity check code
>
> Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp>
> ---
>  lib/librte_eal/common/eal_common_devargs.c  | 57 +++++++++++++++++++++++++++++
>  lib/librte_eal/common/include/rte_devargs.h | 18 +++++++++
>  2 files changed, 75 insertions(+)
>
> diff --git a/lib/librte_eal/common/eal_common_devargs.c b/lib/librte_eal/common/eal_common_devargs.c
> index 4c7d11a..a360a85 100644
> --- a/lib/librte_eal/common/eal_common_devargs.c
> +++ b/lib/librte_eal/common/eal_common_devargs.c
> @@ -44,6 +44,35 @@
>  struct rte_devargs_list devargs_list =
>  	TAILQ_HEAD_INITIALIZER(devargs_list);
>  
> +
> +/* find a entry specified by pci address or device name */
> +static struct rte_devargs *
> +rte_eal_devargs_find(enum rte_devtype devtype, void *args)
> +{
> +	struct rte_devargs *devargs;
> +
> +	if (args == NULL)
> +		return NULL;
> +
> +	TAILQ_FOREACH(devargs, &devargs_list, next) {
> +		switch (devtype) {
> +		case RTE_DEVTYPE_WHITELISTED_PCI:
> +		case RTE_DEVTYPE_BLACKLISTED_PCI:
> +			if (eal_compare_pci_addr(&devargs->pci.addr, args) == 0)
> +				goto found;
> +			break;
> +		case RTE_DEVTYPE_VIRTUAL:
> +			if (memcmp(&devargs->virtual.drv_name, args,
> +			    strlen((char *)args)) == 0)
> +				goto found;
> +			break;
> +		}
> +	}
> +	return NULL;
> +found:
> +	return devargs;
> +}
> +
>  /* store a whitelist parameter for later parsing */
>  int
>  rte_eal_devargs_add(enum rte_devtype devtype, const char *devargs_str)
> @@ -87,6 +116,12 @@ rte_eal_devargs_add(enum rte_devtype devtype, const char *devargs_str)
>  			free(devargs);
>  			return -1;
>  		}
> +		/* make sure there is no same entry */
> +		if (rte_eal_devargs_find(devtype, &devargs->pci.addr)) {
> +			RTE_LOG(ERR, EAL,
> +				"device already registered: <%s>\n", buf);
> +			return -1;
> +		}
>  		break;
>  	case RTE_DEVTYPE_VIRTUAL:
>  		/* save driver name */
> @@ -98,6 +133,12 @@ rte_eal_devargs_add(enum rte_devtype devtype, const char *devargs_str)
>  			free(devargs);
>  			return -1;
>  		}
> +		/* make sure there is no same entry */
> +		if (rte_eal_devargs_find(devtype, &devargs->virtual.drv_name)) {
> +			RTE_LOG(ERR, EAL,
> +				"device already registered: <%s>\n", buf);
> +			return -1;
> +		}
>  		break;
>  	}
>  
> @@ -105,6 +146,22 @@ rte_eal_devargs_add(enum rte_devtype devtype, const char *devargs_str)
>  	return 0;
>  }
>  
> +/* remove it from the devargs_list */
> +void
> +rte_eal_devargs_remove(enum rte_devtype devtype, void *args)
> +{
> +	struct rte_devargs *devargs;
> +
> +	if (args == NULL)
> +		return;
> +
> +	devargs = rte_eal_devargs_find(devtype, args);
> +	if (devargs == NULL)

If  devargs == NULL, means not found, does it reasonable to ignore? Some
error happens I think, at least you should print out some logs.

Thanks,
Michael
> +		return;
> +
> +	TAILQ_REMOVE(&devargs_list, devargs, next);
> +}
> +
>  /* count the number of devices of a specified type */
>  unsigned int
>  rte_eal_devargs_type_count(enum rte_devtype devtype)
> diff --git a/lib/librte_eal/common/include/rte_devargs.h b/lib/librte_eal/common/include/rte_devargs.h
> index 9f9c98f..1066efd 100644
> --- a/lib/librte_eal/common/include/rte_devargs.h
> +++ b/lib/librte_eal/common/include/rte_devargs.h
> @@ -123,6 +123,24 @@ extern struct rte_devargs_list devargs_list;
>  int rte_eal_devargs_add(enum rte_devtype devtype, const char *devargs_str);
>  
>  /**
> + * Remove a device from the user device list
> + *
> + * For PCI devices, the format of arguments string is "PCI_ADDR". It shouldn't
> + * involves parameters for the device. Example: "08:00.1".
> + *
> + * For virtual devices, the format of arguments string is "DRIVER_NAME*". It
> + * shouldn't involves parameters for the device. Example: "eth_ring". The
> + * validity of the driver name is not checked by this function, it is done
> + * when closing the drivers.
> + *
> + * @param devtype
> + *   The type of the device.
> + * @param name
> + *   The name of the device.
> + */
> +void rte_eal_devargs_remove(enum rte_devtype devtype, void *args);
> +
> +/**
>   * Count the number of user devices of a specified type
>   *
>   * @param devtype


^ permalink raw reply	[flat|nested] 362+ messages in thread

* Re: [dpdk-dev] [PATCH v4 10/11] eal/pci: Add rte_eal_dev_attach/detach() functions
  2015-01-19 10:40   ` [dpdk-dev] [PATCH v4 10/11] eal/pci: Add rte_eal_dev_attach/detach() functions Tetsuya Mukawa
@ 2015-01-21  3:49     ` Qiu, Michael
  2015-01-21  6:34       ` Tetsuya Mukawa
  0 siblings, 1 reply; 362+ messages in thread
From: Qiu, Michael @ 2015-01-21  3:49 UTC (permalink / raw)
  To: Tetsuya Mukawa, dev

On 1/19/2015 6:43 PM, Tetsuya Mukawa wrote:
> These functions are used for attaching or detaching a port.
> When rte_eal_dev_attach() is called, the function tries to realize the
> device name as pci address. If this is done successfully,
> rte_eal_dev_attach() will attach physical device port. If not, attaches
> virtual devive port.
> When rte_eal_dev_detach() is called, the function gets the device type
> of this port to know whether the port is came from physical or virtual.
> And then specific detaching function will be called.
>
> v4:
> - Fix comment.
> - Add error checking.
> - Fix indent of 'if' statement.
> - Change function name.
>
> Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp>
> ---
>  lib/librte_eal/common/eal_common_dev.c  | 273 ++++++++++++++++++++++++++++++++
>  lib/librte_eal/common/eal_private.h     |  11 ++
>  lib/librte_eal/common/include/rte_dev.h |  33 ++++
>  lib/librte_eal/linuxapp/eal/Makefile    |   1 +
>  lib/librte_eal/linuxapp/eal/eal_pci.c   |   6 +-
>  5 files changed, 321 insertions(+), 3 deletions(-)
>
> diff --git a/lib/librte_eal/common/eal_common_dev.c b/lib/librte_eal/common/eal_common_dev.c
> index eae5656..828bd70 100644
> --- a/lib/librte_eal/common/eal_common_dev.c
> +++ b/lib/librte_eal/common/eal_common_dev.c
> @@ -32,10 +32,13 @@
>   *   OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
>   */
>  
> +#include <stdio.h>
> +#include <limits.h>
>  #include <string.h>
>  #include <inttypes.h>
>  #include <sys/queue.h>
>  
> +#include <rte_ethdev.h>
>  #include <rte_dev.h>
>  #include <rte_devargs.h>
>  #include <rte_debug.h>
> @@ -107,3 +110,273 @@ rte_eal_dev_init(void)
>  	}
>  	return 0;
>  }
> +
> +/* So far, DPDK hotplug function only supports linux */
> +#ifdef ENABLE_HOTPLUG
> +static void
> +rte_eal_dev_invoke(struct rte_driver *driver,
> +		struct rte_devargs *devargs, enum rte_eal_invoke_type type)
> +{
> +	if ((driver == NULL) || (devargs == NULL))
> +		return;
> +
> +	switch (type) {
> +	case RTE_EAL_INVOKE_TYPE_PROBE:
> +		driver->init(devargs->virtual.drv_name, devargs->args);
> +		break;
> +	case RTE_EAL_INVOKE_TYPE_CLOSE:
> +		driver->uninit(devargs->virtual.drv_name, devargs->args);
> +		break;
> +	default:
> +		break;
> +	}
> +}
> +
> +static int
> +rte_eal_dev_find_and_invoke(const char *name, int type)

This function is totally for vdev, so I would like it shows in name,
like *rte_eal_vdev_find_and_invoke*

> +{
> +	struct rte_devargs *devargs;
> +	struct rte_driver *driver;
> +
> +	if (name == NULL)
> +		return -EINVAL;
> +
> +	/* call the init function for each virtual device */
> +	TAILQ_FOREACH(devargs, &devargs_list, next) {
> +
> +		if (devargs->type != RTE_DEVTYPE_VIRTUAL)
> +			continue;
> +
> +		if (strncmp(name, devargs->virtual.drv_name, strlen(name)))
> +			continue;
> +
> +		TAILQ_FOREACH(driver, &dev_driver_list, next) {
> +			if (driver->type != PMD_VDEV)
> +				continue;
> +
> +			/* search a driver prefix in virtual device name */
> +			if (!strncmp(driver->name, devargs->virtual.drv_name,
> +			    strlen(driver->name))) {
> +				rte_eal_dev_invoke(driver, devargs, type);
> +				break;
> +			}
> +		}
> +
> +		if (driver == NULL) {
> +			RTE_LOG(WARNING, EAL, "no driver found for %s\n",
> +				  devargs->virtual.drv_name);
> +		}
> +		return 0;
> +	}
> +	return 1;
> +}
> +
> +/* attach the new physical device, then store port_id of the device */
> +static int
> +rte_eal_dev_attach_pdev(struct rte_pci_addr *addr, uint8_t *port_id)
> +{
> +	uint8_t new_port_id;
> +	struct rte_eth_dev devs[RTE_MAX_ETHPORTS];
> +
> +	if ((addr == NULL) || (port_id == NULL))
> +		goto err;
> +
> +	/* save current port status */
> +	rte_eth_dev_save(devs);
> +	/* re-construct pci_device_list */
> +	if (rte_eal_pci_scan())
> +		goto err;
> +	/* invoke probe func of the driver can handle the new device */
> +	if (rte_eal_pci_probe_one(addr))
> +		goto err;
> +	/* get port_id enabled by above procedures */
> +	if (rte_eth_dev_get_changed_port(devs, &new_port_id))
> +		goto err;
> +
> +	*port_id = new_port_id;
> +	return 0;
> +err:
> +	RTE_LOG(ERR, EAL, "Drver, cannot attach the device\n");
> +	return -1;
> +}
> +
> +/* detach the new physical device, then store pci_addr of the device */
> +static int
> +rte_eal_dev_detach_pdev(uint8_t port_id, struct rte_pci_addr *addr)
> +{
> +	struct rte_pci_addr freed_addr;
> +	struct rte_pci_addr vp;
> +
> +	if (addr == NULL)
> +		goto err;
> +
> +	/* check whether the driver supports detach feature, or not */
> +	if (rte_eth_dev_check_detachable(port_id))
> +		goto err;
> +
> +	/* get pci address by port id */
> +	if (rte_eth_dev_get_addr_by_port(port_id, &freed_addr))
> +		goto err;
> +
> +	/* Zerod pci addr means the port comes from virtual device */
> +	vp.domain = vp.bus = vp.devid = vp.function = 0;
> +	if (eal_compare_pci_addr(&vp, &freed_addr) == 0)
> +		goto err;
> +
> +	/* invoke close func of the driver,
> +	 * also remove the device from pci_device_list */
> +	if (rte_eal_pci_close_one(&freed_addr))
> +		goto err;
> +
> +	*addr = freed_addr;
> +	return 0;
> +err:
> +	RTE_LOG(ERR, EAL, "Drver, cannot detach the device\n");
> +	return -1;
> +}
> +
> +static void
> +get_vdev_name(char *vdevargs)
> +{
> +	char *sep;
> +
> +	if (vdevargs == NULL)
> +		return;
> +
> +	/* set the first ',' to '\0' to split name and arguments */
> +	sep = strchr(vdevargs, ',');
> +	if (sep != NULL)
> +		sep[0] = '\0';
> +}
> +
> +/* attach the new virtual device, then store port_id of the device */
> +static int
> +rte_eal_dev_attach_vdev(const char *vdevargs, uint8_t *port_id)
> +{
> +	char *args;
> +	uint8_t new_port_id;
> +	struct rte_eth_dev devs[RTE_MAX_ETHPORTS];
> +
> +	if ((vdevargs == NULL) || (port_id == NULL))
> +		goto err0;
> +
> +	args = strdup(vdevargs);
> +	if (args == NULL)
> +		goto err0;
> +
> +	/* save current port status */
> +	rte_eth_dev_save(devs);
> +	/* add the vdevargs to devargs_list */
> +	if (rte_eal_devargs_add(RTE_DEVTYPE_VIRTUAL, args))
> +		goto err1;
> +	/* parse vdevargs, then retrieve device name */
> +	get_vdev_name(args);
> +	/* walk around dev_driver_list to find the driver of the device,
> +	 * then invoke probe function o the driver */
> +	if (rte_eal_dev_find_and_invoke(args, RTE_EAL_INVOKE_TYPE_PROBE))
> +		goto err2;
> +	/* get port_id enabled by above procedures */
> +	if (rte_eth_dev_get_changed_port(devs, &new_port_id))
> +		goto err2;
> +
> +	free(args);
> +	*port_id = new_port_id;
> +	return 0;
> +err2:
> +	rte_eal_devargs_remove(RTE_DEVTYPE_VIRTUAL, args);
> +err1:
> +	free(args);
> +err0:
> +	RTE_LOG(ERR, EAL, "Drver, cannot detach the device\n");
> +	return -1;
> +}
> +
> +/* detach the new virtual device, then store the name of the device */
> +static int
> +rte_eal_dev_detach_vdev(uint8_t port_id, char *vdevname)
> +{
> +	char name[RTE_ETH_NAME_MAX_LEN];
> +
> +	if (vdevname == NULL)
> +		goto err;
> +
> +	/* check whether the driver supports detach feature, or not */
> +	if (rte_eth_dev_check_detachable(port_id))
> +		goto err;
> +
> +	/* get device name by port id */
> +	if (rte_eth_dev_get_name_by_port(port_id, name))
> +		goto err;
> +	/* walk around dev_driver_list to find the driver of the device,
> +	 * then invoke close function o the driver */
> +	if (rte_eal_dev_find_and_invoke(name, RTE_EAL_INVOKE_TYPE_CLOSE))
> +		goto err;
> +	/* remove the vdevname from devargs_list */
> +	rte_eal_devargs_remove(RTE_DEVTYPE_VIRTUAL, name);
> +
> +	strncpy(vdevname, name, sizeof(name));
> +	return 0;
> +err:
> +	RTE_LOG(ERR, EAL, "Drver, cannot detach the device\n");
> +	return -1;
> +}
> +
> +/* attach the new device, then store port_id of the device */
> +int
> +rte_eal_dev_attach(const char *devargs, uint8_t *port_id)
> +{
> +	struct rte_pci_addr addr;
> +
> +	if ((devargs == NULL) || (port_id == NULL))
> +		return -EINVAL;
> +
> +	if (eal_parse_pci_DomBDF(devargs, &addr) == 0)
> +		return rte_eal_dev_attach_pdev(&addr, port_id);
> +	else
> +		return rte_eal_dev_attach_vdev(devargs, port_id);
> +}
> +
> +/* detach the device, then store the name of the device */
> +int
> +rte_eal_dev_detach(uint8_t port_id, char *name)
> +{
> +	struct rte_pci_addr addr;
> +	int ret;
> +
> +	if (name == NULL)
> +		return -EINVAL;
> +
> +	if (rte_eth_dev_get_device_type(port_id) == RTE_ETH_DEV_PHYSICAL) {
> +		ret = rte_eth_dev_get_addr_by_port(port_id, &addr);
> +		if (ret < 0)
> +			return ret;
> +
> +		ret = rte_eal_dev_detach_pdev(port_id, &addr);
> +		if (ret == 0)
> +			snprintf(name, RTE_ETH_NAME_MAX_LEN,
> +				"%04x.%02x.%02x.%d",
> +				addr.domain, addr.bus,
> +				addr.devid, addr.function);
> +
> +		return ret;
> +	} else
> +		return rte_eal_dev_detach_vdev(port_id, name);
> +}
> +#else /* ENABLE_HOTPLUG */
> +int
> +rte_eal_dev_attach(const char *devargs __rte_unused,
> +			uint8_t *port_id __rte_unused)
> +{
> +	RTE_LOG(ERR, EAL, "Hotplug support isn't enabled\n");
> +	return -1;
> +}
> +
> +/* detach the device, then store the name of the device */
> +int
> +rte_eal_dev_detach(uint8_t port_id __rte_unused,
> +			char *name __rte_unused)
> +{
> +	RTE_LOG(ERR, EAL, "Hotplug support isn't enabled\n");
> +	return -1;
> +}
> +#endif /* ENABLE_HOTPLUG */
> diff --git a/lib/librte_eal/common/eal_private.h b/lib/librte_eal/common/eal_private.h
> index a97c5d8..453a1eb 100644
> --- a/lib/librte_eal/common/eal_private.h
> +++ b/lib/librte_eal/common/eal_private.h
> @@ -164,6 +164,17 @@ enum rte_eal_invoke_type {
>  };
>  
>  /**
> + * Scan the content of the PCI bus, and the devices in the devices
> + * list
> + *
> + * This function is private to EAL.
> + *
> + * @return
> + *  0 on success, negative on error
> + */
> +int rte_eal_pci_scan(void);
> +
> +/**
>   * Mmap memory for single PCI device
>   *
>   * This function is private to EAL.
> diff --git a/lib/librte_eal/common/include/rte_dev.h b/lib/librte_eal/common/include/rte_dev.h
> index f7e3a10..e63dd1c 100644
> --- a/lib/librte_eal/common/include/rte_dev.h
> +++ b/lib/librte_eal/common/include/rte_dev.h
> @@ -47,6 +47,7 @@ extern "C" {
>  #endif
>  
>  #include <sys/queue.h>
> +#include <rte_pci.h>
>  
>  /** Double linked list of device drivers. */
>  TAILQ_HEAD(rte_driver_list, rte_driver);
> @@ -57,6 +58,11 @@ TAILQ_HEAD(rte_driver_list, rte_driver);
>  typedef int (rte_dev_init_t)(const char *name, const char *args);
>  
>  /**
> + * Uninitilization function called for each device driver once.
> + */
> +typedef int (rte_dev_uninit_t)(const char *name, const char *args);
> +
> +/**
>   * Driver type enumeration
>   */
>  enum pmd_type {
> @@ -72,6 +78,7 @@ struct rte_driver {
>  	enum pmd_type type;		   /**< PMD Driver type */
>  	const char *name;                   /**< Driver name. */
>  	rte_dev_init_t *init;              /**< Device init. function. */
> +	rte_dev_uninit_t *uninit;          /**< Device uninit. function. */
>  };
>  
>  /**
> @@ -93,6 +100,32 @@ void rte_eal_driver_register(struct rte_driver *driver);
>  void rte_eal_driver_unregister(struct rte_driver *driver);
>  
>  /**
> + * Attach a new device.
> + *
> + * @param devargs
> + *   A pointer to a strings array describing the new device
> + *   to be attached. The strings should be a pci address like
> + *   '0000:01:00.0' or virtual device name like 'eth_pcap0'.
> + * @param port_id
> + *  A pointer to a port identifier actually attached.
> + * @return
> + *  0 on success and port_id is filled, negative on error
> + */
> +int rte_eal_dev_attach(const char *devargs, uint8_t *port_id);
> +
> +/**
> + * Detach a device.
> + *
> + * @param port_id
> + *   The port identifier of the device to detach.
> + * @param addr
> + *  A pointer to a device name actually detached.
> + * @return
> + *  0 on success and devname is filled, negative on error
> + */
> +int rte_eal_dev_detach(uint8_t port_id, char *devname);
> +
> +/**
>   * Initalize all the registered drivers in this process
>   */
>  int rte_eal_dev_init(void);
> diff --git a/lib/librte_eal/linuxapp/eal/Makefile b/lib/librte_eal/linuxapp/eal/Makefile
> index 72ecf3a..0ec83b5 100644
> --- a/lib/librte_eal/linuxapp/eal/Makefile
> +++ b/lib/librte_eal/linuxapp/eal/Makefile
> @@ -41,6 +41,7 @@ CFLAGS += -I$(RTE_SDK)/lib/librte_eal/common/include
>  CFLAGS += -I$(RTE_SDK)/lib/librte_ring
>  CFLAGS += -I$(RTE_SDK)/lib/librte_mempool
>  CFLAGS += -I$(RTE_SDK)/lib/librte_malloc
> +CFLAGS += -I$(RTE_SDK)/lib/librte_mbuf
>  CFLAGS += -I$(RTE_SDK)/lib/librte_ether
>  CFLAGS += -I$(RTE_SDK)/lib/librte_ivshmem
>  CFLAGS += -I$(RTE_SDK)/lib/librte_pmd_ring
> diff --git a/lib/librte_eal/linuxapp/eal/eal_pci.c b/lib/librte_eal/linuxapp/eal/eal_pci.c
> index a23cc59..8e7e650 100644
> --- a/lib/librte_eal/linuxapp/eal/eal_pci.c
> +++ b/lib/librte_eal/linuxapp/eal/eal_pci.c
> @@ -378,8 +378,8 @@ error:
>   * Scan the content of the PCI bus, and the devices in the devices
>   * list
>   */
> -static int
> -pci_scan(void)
> +int
> +rte_eal_pci_scan(void)
>  {
>  	struct dirent *e;
>  	DIR *dir;
> @@ -701,7 +701,7 @@ rte_eal_pci_init(void)
>  	if (internal_config.no_pci)
>  		return 0;
>  
> -	if (pci_scan() < 0) {
> +	if (rte_eal_pci_scan() < 0) {
>  		RTE_LOG(ERR, EAL, "%s(): Cannot scan PCI bus\n", __func__);
>  		return -1;
>  	}


^ permalink raw reply	[flat|nested] 362+ messages in thread

* Re: [dpdk-dev] [PATCH v4 07/11] eal/pci: Add a function to remove the entry of devargs list
  2015-01-21  2:55     ` Qiu, Michael
@ 2015-01-21  6:33       ` Tetsuya Mukawa
  0 siblings, 0 replies; 362+ messages in thread
From: Tetsuya Mukawa @ 2015-01-21  6:33 UTC (permalink / raw)
  To: Qiu, Michael; +Cc: dev

Hi Michael,

On 2015/01/21 11:55, Qiu, Michael wrote:
> On 1/19/2015 6:42 PM, Tetsuya Mukawa wrote:
>> The function removes the specified devargs entry from devargs_list.
>> Also the patch adds sanity checking to rte_eal_devargs_add().
>>
>> v4:
>> - Fix sanity check code
>>
>> Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp>
>> ---
>>  lib/librte_eal/common/eal_common_devargs.c  | 57 +++++++++++++++++++++++++++++
>>  lib/librte_eal/common/include/rte_devargs.h | 18 +++++++++
>>  2 files changed, 75 insertions(+)
>>
>> diff --git a/lib/librte_eal/common/eal_common_devargs.c b/lib/librte_eal/common/eal_common_devargs.c
>> index 4c7d11a..a360a85 100644
>> --- a/lib/librte_eal/common/eal_common_devargs.c
>> +++ b/lib/librte_eal/common/eal_common_devargs.c
>> @@ -44,6 +44,35 @@
>>  struct rte_devargs_list devargs_list =
>>  	TAILQ_HEAD_INITIALIZER(devargs_list);
>>  
>> +
>> +/* find a entry specified by pci address or device name */
>> +static struct rte_devargs *
>> +rte_eal_devargs_find(enum rte_devtype devtype, void *args)
>> +{
>> +	struct rte_devargs *devargs;
>> +
>> +	if (args == NULL)
>> +		return NULL;
>> +
>> +	TAILQ_FOREACH(devargs, &devargs_list, next) {
>> +		switch (devtype) {
>> +		case RTE_DEVTYPE_WHITELISTED_PCI:
>> +		case RTE_DEVTYPE_BLACKLISTED_PCI:
>> +			if (eal_compare_pci_addr(&devargs->pci.addr, args) == 0)
>> +				goto found;
>> +			break;
>> +		case RTE_DEVTYPE_VIRTUAL:
>> +			if (memcmp(&devargs->virtual.drv_name, args,
>> +			    strlen((char *)args)) == 0)
>> +				goto found;
>> +			break;
>> +		}
>> +	}
>> +	return NULL;
>> +found:
>> +	return devargs;
>> +}
>> +
>>  /* store a whitelist parameter for later parsing */
>>  int
>>  rte_eal_devargs_add(enum rte_devtype devtype, const char *devargs_str)
>> @@ -87,6 +116,12 @@ rte_eal_devargs_add(enum rte_devtype devtype, const char *devargs_str)
>>  			free(devargs);
>>  			return -1;
>>  		}
>> +		/* make sure there is no same entry */
>> +		if (rte_eal_devargs_find(devtype, &devargs->pci.addr)) {
>> +			RTE_LOG(ERR, EAL,
>> +				"device already registered: <%s>\n", buf);
>> +			return -1;
>> +		}
>>  		break;
>>  	case RTE_DEVTYPE_VIRTUAL:
>>  		/* save driver name */
>> @@ -98,6 +133,12 @@ rte_eal_devargs_add(enum rte_devtype devtype, const char *devargs_str)
>>  			free(devargs);
>>  			return -1;
>>  		}
>> +		/* make sure there is no same entry */
>> +		if (rte_eal_devargs_find(devtype, &devargs->virtual.drv_name)) {
>> +			RTE_LOG(ERR, EAL,
>> +				"device already registered: <%s>\n", buf);
>> +			return -1;
>> +		}
>>  		break;
>>  	}
>>  
>> @@ -105,6 +146,22 @@ rte_eal_devargs_add(enum rte_devtype devtype, const char *devargs_str)
>>  	return 0;
>>  }
>>  
>> +/* remove it from the devargs_list */
>> +void
>> +rte_eal_devargs_remove(enum rte_devtype devtype, void *args)
>> +{
>> +	struct rte_devargs *devargs;
>> +
>> +	if (args == NULL)
>> +		return;
>> +
>> +	devargs = rte_eal_devargs_find(devtype, args);
>> +	if (devargs == NULL)
> If  devargs == NULL, means not found, does it reasonable to ignore? Some
> error happens I think, at least you should print out some logs.

I appreciate your comment.
I agree with you. At least error message should be displayed.
I will change function definition to return a error code and fix caller
of the function.
Also I will add error message.

Thanks,
Tetsuya

> Thanks,
> Michael
>> +		return;
>> +
>> +	TAILQ_REMOVE(&devargs_list, devargs, next);
>> +}
>> +
>>  /* count the number of devices of a specified type */
>>  unsigned int
>>  rte_eal_devargs_type_count(enum rte_devtype devtype)
>> diff --git a/lib/librte_eal/common/include/rte_devargs.h b/lib/librte_eal/common/include/rte_devargs.h
>> index 9f9c98f..1066efd 100644
>> --- a/lib/librte_eal/common/include/rte_devargs.h
>> +++ b/lib/librte_eal/common/include/rte_devargs.h
>> @@ -123,6 +123,24 @@ extern struct rte_devargs_list devargs_list;
>>  int rte_eal_devargs_add(enum rte_devtype devtype, const char *devargs_str);
>>  
>>  /**
>> + * Remove a device from the user device list
>> + *
>> + * For PCI devices, the format of arguments string is "PCI_ADDR". It shouldn't
>> + * involves parameters for the device. Example: "08:00.1".
>> + *
>> + * For virtual devices, the format of arguments string is "DRIVER_NAME*". It
>> + * shouldn't involves parameters for the device. Example: "eth_ring". The
>> + * validity of the driver name is not checked by this function, it is done
>> + * when closing the drivers.
>> + *
>> + * @param devtype
>> + *   The type of the device.
>> + * @param name
>> + *   The name of the device.
>> + */
>> +void rte_eal_devargs_remove(enum rte_devtype devtype, void *args);
>> +
>> +/**
>>   * Count the number of user devices of a specified type
>>   *
>>   * @param devtype

^ permalink raw reply	[flat|nested] 362+ messages in thread

* Re: [dpdk-dev] [PATCH v4 10/11] eal/pci: Add rte_eal_dev_attach/detach() functions
  2015-01-21  3:49     ` Qiu, Michael
@ 2015-01-21  6:34       ` Tetsuya Mukawa
  0 siblings, 0 replies; 362+ messages in thread
From: Tetsuya Mukawa @ 2015-01-21  6:34 UTC (permalink / raw)
  To: Qiu, Michael; +Cc: dev

Hi Michael,

On 2015/01/21 12:49, Qiu, Michael wrote:
> On 1/19/2015 6:43 PM, Tetsuya Mukawa wrote:
>> These functions are used for attaching or detaching a port.
>> When rte_eal_dev_attach() is called, the function tries to realize the
>> device name as pci address. If this is done successfully,
>> rte_eal_dev_attach() will attach physical device port. If not, attaches
>> virtual devive port.
>> When rte_eal_dev_detach() is called, the function gets the device type
>> of this port to know whether the port is came from physical or virtual.
>> And then specific detaching function will be called.
>>
>> v4:
>> - Fix comment.
>> - Add error checking.
>> - Fix indent of 'if' statement.
>> - Change function name.
>>
>> Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp>
>> ---
>>  lib/librte_eal/common/eal_common_dev.c  | 273 ++++++++++++++++++++++++++++++++
>>  lib/librte_eal/common/eal_private.h     |  11 ++
>>  lib/librte_eal/common/include/rte_dev.h |  33 ++++
>>  lib/librte_eal/linuxapp/eal/Makefile    |   1 +
>>  lib/librte_eal/linuxapp/eal/eal_pci.c   |   6 +-
>>  5 files changed, 321 insertions(+), 3 deletions(-)
>>
>> diff --git a/lib/librte_eal/common/eal_common_dev.c b/lib/librte_eal/common/eal_common_dev.c
>> index eae5656..828bd70 100644
>> --- a/lib/librte_eal/common/eal_common_dev.c
>> +++ b/lib/librte_eal/common/eal_common_dev.c
>> @@ -32,10 +32,13 @@
>>   *   OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
>>   */
>>  
>> +#include <stdio.h>
>> +#include <limits.h>
>>  #include <string.h>
>>  #include <inttypes.h>
>>  #include <sys/queue.h>
>>  
>> +#include <rte_ethdev.h>
>>  #include <rte_dev.h>
>>  #include <rte_devargs.h>
>>  #include <rte_debug.h>
>> @@ -107,3 +110,273 @@ rte_eal_dev_init(void)
>>  	}
>>  	return 0;
>>  }
>> +
>> +/* So far, DPDK hotplug function only supports linux */
>> +#ifdef ENABLE_HOTPLUG
>> +static void
>> +rte_eal_dev_invoke(struct rte_driver *driver,
>> +		struct rte_devargs *devargs, enum rte_eal_invoke_type type)
>> +{
>> +	if ((driver == NULL) || (devargs == NULL))
>> +		return;
>> +
>> +	switch (type) {
>> +	case RTE_EAL_INVOKE_TYPE_PROBE:
>> +		driver->init(devargs->virtual.drv_name, devargs->args);
>> +		break;
>> +	case RTE_EAL_INVOKE_TYPE_CLOSE:
>> +		driver->uninit(devargs->virtual.drv_name, devargs->args);
>> +		break;
>> +	default:
>> +		break;
>> +	}
>> +}
>> +
>> +static int
>> +rte_eal_dev_find_and_invoke(const char *name, int type)
> This function is totally for vdev, so I would like it shows in name,
> like *rte_eal_vdev_find_and_invoke*

Sure, I will change like above. I appreciate your suggestion.

Thanks,
Tetsuya

>> +{
>> +	struct rte_devargs *devargs;
>> +	struct rte_driver *driver;
>> +
>> +	if (name == NULL)
>> +		return -EINVAL;
>> +
>> +	/* call the init function for each virtual device */
>> +	TAILQ_FOREACH(devargs, &devargs_list, next) {
>> +
>> +		if (devargs->type != RTE_DEVTYPE_VIRTUAL)
>> +			continue;
>> +
>> +		if (strncmp(name, devargs->virtual.drv_name, strlen(name)))
>> +			continue;
>> +
>> +		TAILQ_FOREACH(driver, &dev_driver_list, next) {
>> +			if (driver->type != PMD_VDEV)
>> +				continue;
>> +
>> +			/* search a driver prefix in virtual device name */
>> +			if (!strncmp(driver->name, devargs->virtual.drv_name,
>> +			    strlen(driver->name))) {
>> +				rte_eal_dev_invoke(driver, devargs, type);
>> +				break;
>> +			}
>> +		}
>> +
>> +		if (driver == NULL) {
>> +			RTE_LOG(WARNING, EAL, "no driver found for %s\n",
>> +				  devargs->virtual.drv_name);
>> +		}
>> +		return 0;
>> +	}
>> +	return 1;
>> +}
>> +
>> +/* attach the new physical device, then store port_id of the device */
>> +static int
>> +rte_eal_dev_attach_pdev(struct rte_pci_addr *addr, uint8_t *port_id)
>> +{
>> +	uint8_t new_port_id;
>> +	struct rte_eth_dev devs[RTE_MAX_ETHPORTS];
>> +
>> +	if ((addr == NULL) || (port_id == NULL))
>> +		goto err;
>> +
>> +	/* save current port status */
>> +	rte_eth_dev_save(devs);
>> +	/* re-construct pci_device_list */
>> +	if (rte_eal_pci_scan())
>> +		goto err;
>> +	/* invoke probe func of the driver can handle the new device */
>> +	if (rte_eal_pci_probe_one(addr))
>> +		goto err;
>> +	/* get port_id enabled by above procedures */
>> +	if (rte_eth_dev_get_changed_port(devs, &new_port_id))
>> +		goto err;
>> +
>> +	*port_id = new_port_id;
>> +	return 0;
>> +err:
>> +	RTE_LOG(ERR, EAL, "Drver, cannot attach the device\n");
>> +	return -1;
>> +}
>> +
>> +/* detach the new physical device, then store pci_addr of the device */
>> +static int
>> +rte_eal_dev_detach_pdev(uint8_t port_id, struct rte_pci_addr *addr)
>> +{
>> +	struct rte_pci_addr freed_addr;
>> +	struct rte_pci_addr vp;
>> +
>> +	if (addr == NULL)
>> +		goto err;
>> +
>> +	/* check whether the driver supports detach feature, or not */
>> +	if (rte_eth_dev_check_detachable(port_id))
>> +		goto err;
>> +
>> +	/* get pci address by port id */
>> +	if (rte_eth_dev_get_addr_by_port(port_id, &freed_addr))
>> +		goto err;
>> +
>> +	/* Zerod pci addr means the port comes from virtual device */
>> +	vp.domain = vp.bus = vp.devid = vp.function = 0;
>> +	if (eal_compare_pci_addr(&vp, &freed_addr) == 0)
>> +		goto err;
>> +
>> +	/* invoke close func of the driver,
>> +	 * also remove the device from pci_device_list */
>> +	if (rte_eal_pci_close_one(&freed_addr))
>> +		goto err;
>> +
>> +	*addr = freed_addr;
>> +	return 0;
>> +err:
>> +	RTE_LOG(ERR, EAL, "Drver, cannot detach the device\n");
>> +	return -1;
>> +}
>> +
>> +static void
>> +get_vdev_name(char *vdevargs)
>> +{
>> +	char *sep;
>> +
>> +	if (vdevargs == NULL)
>> +		return;
>> +
>> +	/* set the first ',' to '\0' to split name and arguments */
>> +	sep = strchr(vdevargs, ',');
>> +	if (sep != NULL)
>> +		sep[0] = '\0';
>> +}
>> +
>> +/* attach the new virtual device, then store port_id of the device */
>> +static int
>> +rte_eal_dev_attach_vdev(const char *vdevargs, uint8_t *port_id)
>> +{
>> +	char *args;
>> +	uint8_t new_port_id;
>> +	struct rte_eth_dev devs[RTE_MAX_ETHPORTS];
>> +
>> +	if ((vdevargs == NULL) || (port_id == NULL))
>> +		goto err0;
>> +
>> +	args = strdup(vdevargs);
>> +	if (args == NULL)
>> +		goto err0;
>> +
>> +	/* save current port status */
>> +	rte_eth_dev_save(devs);
>> +	/* add the vdevargs to devargs_list */
>> +	if (rte_eal_devargs_add(RTE_DEVTYPE_VIRTUAL, args))
>> +		goto err1;
>> +	/* parse vdevargs, then retrieve device name */
>> +	get_vdev_name(args);
>> +	/* walk around dev_driver_list to find the driver of the device,
>> +	 * then invoke probe function o the driver */
>> +	if (rte_eal_dev_find_and_invoke(args, RTE_EAL_INVOKE_TYPE_PROBE))
>> +		goto err2;
>> +	/* get port_id enabled by above procedures */
>> +	if (rte_eth_dev_get_changed_port(devs, &new_port_id))
>> +		goto err2;
>> +
>> +	free(args);
>> +	*port_id = new_port_id;
>> +	return 0;
>> +err2:
>> +	rte_eal_devargs_remove(RTE_DEVTYPE_VIRTUAL, args);
>> +err1:
>> +	free(args);
>> +err0:
>> +	RTE_LOG(ERR, EAL, "Drver, cannot detach the device\n");
>> +	return -1;
>> +}
>> +
>> +/* detach the new virtual device, then store the name of the device */
>> +static int
>> +rte_eal_dev_detach_vdev(uint8_t port_id, char *vdevname)
>> +{
>> +	char name[RTE_ETH_NAME_MAX_LEN];
>> +
>> +	if (vdevname == NULL)
>> +		goto err;
>> +
>> +	/* check whether the driver supports detach feature, or not */
>> +	if (rte_eth_dev_check_detachable(port_id))
>> +		goto err;
>> +
>> +	/* get device name by port id */
>> +	if (rte_eth_dev_get_name_by_port(port_id, name))
>> +		goto err;
>> +	/* walk around dev_driver_list to find the driver of the device,
>> +	 * then invoke close function o the driver */
>> +	if (rte_eal_dev_find_and_invoke(name, RTE_EAL_INVOKE_TYPE_CLOSE))
>> +		goto err;
>> +	/* remove the vdevname from devargs_list */
>> +	rte_eal_devargs_remove(RTE_DEVTYPE_VIRTUAL, name);
>> +
>> +	strncpy(vdevname, name, sizeof(name));
>> +	return 0;
>> +err:
>> +	RTE_LOG(ERR, EAL, "Drver, cannot detach the device\n");
>> +	return -1;
>> +}
>> +
>> +/* attach the new device, then store port_id of the device */
>> +int
>> +rte_eal_dev_attach(const char *devargs, uint8_t *port_id)
>> +{
>> +	struct rte_pci_addr addr;
>> +
>> +	if ((devargs == NULL) || (port_id == NULL))
>> +		return -EINVAL;
>> +
>> +	if (eal_parse_pci_DomBDF(devargs, &addr) == 0)
>> +		return rte_eal_dev_attach_pdev(&addr, port_id);
>> +	else
>> +		return rte_eal_dev_attach_vdev(devargs, port_id);
>> +}
>> +
>> +/* detach the device, then store the name of the device */
>> +int
>> +rte_eal_dev_detach(uint8_t port_id, char *name)
>> +{
>> +	struct rte_pci_addr addr;
>> +	int ret;
>> +
>> +	if (name == NULL)
>> +		return -EINVAL;
>> +
>> +	if (rte_eth_dev_get_device_type(port_id) == RTE_ETH_DEV_PHYSICAL) {
>> +		ret = rte_eth_dev_get_addr_by_port(port_id, &addr);
>> +		if (ret < 0)
>> +			return ret;
>> +
>> +		ret = rte_eal_dev_detach_pdev(port_id, &addr);
>> +		if (ret == 0)
>> +			snprintf(name, RTE_ETH_NAME_MAX_LEN,
>> +				"%04x.%02x.%02x.%d",
>> +				addr.domain, addr.bus,
>> +				addr.devid, addr.function);
>> +
>> +		return ret;
>> +	} else
>> +		return rte_eal_dev_detach_vdev(port_id, name);
>> +}
>> +#else /* ENABLE_HOTPLUG */
>> +int
>> +rte_eal_dev_attach(const char *devargs __rte_unused,
>> +			uint8_t *port_id __rte_unused)
>> +{
>> +	RTE_LOG(ERR, EAL, "Hotplug support isn't enabled\n");
>> +	return -1;
>> +}
>> +
>> +/* detach the device, then store the name of the device */
>> +int
>> +rte_eal_dev_detach(uint8_t port_id __rte_unused,
>> +			char *name __rte_unused)
>> +{
>> +	RTE_LOG(ERR, EAL, "Hotplug support isn't enabled\n");
>> +	return -1;
>> +}
>> +#endif /* ENABLE_HOTPLUG */
>> diff --git a/lib/librte_eal/common/eal_private.h b/lib/librte_eal/common/eal_private.h
>> index a97c5d8..453a1eb 100644
>> --- a/lib/librte_eal/common/eal_private.h
>> +++ b/lib/librte_eal/common/eal_private.h
>> @@ -164,6 +164,17 @@ enum rte_eal_invoke_type {
>>  };
>>  
>>  /**
>> + * Scan the content of the PCI bus, and the devices in the devices
>> + * list
>> + *
>> + * This function is private to EAL.
>> + *
>> + * @return
>> + *  0 on success, negative on error
>> + */
>> +int rte_eal_pci_scan(void);
>> +
>> +/**
>>   * Mmap memory for single PCI device
>>   *
>>   * This function is private to EAL.
>> diff --git a/lib/librte_eal/common/include/rte_dev.h b/lib/librte_eal/common/include/rte_dev.h
>> index f7e3a10..e63dd1c 100644
>> --- a/lib/librte_eal/common/include/rte_dev.h
>> +++ b/lib/librte_eal/common/include/rte_dev.h
>> @@ -47,6 +47,7 @@ extern "C" {
>>  #endif
>>  
>>  #include <sys/queue.h>
>> +#include <rte_pci.h>
>>  
>>  /** Double linked list of device drivers. */
>>  TAILQ_HEAD(rte_driver_list, rte_driver);
>> @@ -57,6 +58,11 @@ TAILQ_HEAD(rte_driver_list, rte_driver);
>>  typedef int (rte_dev_init_t)(const char *name, const char *args);
>>  
>>  /**
>> + * Uninitilization function called for each device driver once.
>> + */
>> +typedef int (rte_dev_uninit_t)(const char *name, const char *args);
>> +
>> +/**
>>   * Driver type enumeration
>>   */
>>  enum pmd_type {
>> @@ -72,6 +78,7 @@ struct rte_driver {
>>  	enum pmd_type type;		   /**< PMD Driver type */
>>  	const char *name;                   /**< Driver name. */
>>  	rte_dev_init_t *init;              /**< Device init. function. */
>> +	rte_dev_uninit_t *uninit;          /**< Device uninit. function. */
>>  };
>>  
>>  /**
>> @@ -93,6 +100,32 @@ void rte_eal_driver_register(struct rte_driver *driver);
>>  void rte_eal_driver_unregister(struct rte_driver *driver);
>>  
>>  /**
>> + * Attach a new device.
>> + *
>> + * @param devargs
>> + *   A pointer to a strings array describing the new device
>> + *   to be attached. The strings should be a pci address like
>> + *   '0000:01:00.0' or virtual device name like 'eth_pcap0'.
>> + * @param port_id
>> + *  A pointer to a port identifier actually attached.
>> + * @return
>> + *  0 on success and port_id is filled, negative on error
>> + */
>> +int rte_eal_dev_attach(const char *devargs, uint8_t *port_id);
>> +
>> +/**
>> + * Detach a device.
>> + *
>> + * @param port_id
>> + *   The port identifier of the device to detach.
>> + * @param addr
>> + *  A pointer to a device name actually detached.
>> + * @return
>> + *  0 on success and devname is filled, negative on error
>> + */
>> +int rte_eal_dev_detach(uint8_t port_id, char *devname);
>> +
>> +/**
>>   * Initalize all the registered drivers in this process
>>   */
>>  int rte_eal_dev_init(void);
>> diff --git a/lib/librte_eal/linuxapp/eal/Makefile b/lib/librte_eal/linuxapp/eal/Makefile
>> index 72ecf3a..0ec83b5 100644
>> --- a/lib/librte_eal/linuxapp/eal/Makefile
>> +++ b/lib/librte_eal/linuxapp/eal/Makefile
>> @@ -41,6 +41,7 @@ CFLAGS += -I$(RTE_SDK)/lib/librte_eal/common/include
>>  CFLAGS += -I$(RTE_SDK)/lib/librte_ring
>>  CFLAGS += -I$(RTE_SDK)/lib/librte_mempool
>>  CFLAGS += -I$(RTE_SDK)/lib/librte_malloc
>> +CFLAGS += -I$(RTE_SDK)/lib/librte_mbuf
>>  CFLAGS += -I$(RTE_SDK)/lib/librte_ether
>>  CFLAGS += -I$(RTE_SDK)/lib/librte_ivshmem
>>  CFLAGS += -I$(RTE_SDK)/lib/librte_pmd_ring
>> diff --git a/lib/librte_eal/linuxapp/eal/eal_pci.c b/lib/librte_eal/linuxapp/eal/eal_pci.c
>> index a23cc59..8e7e650 100644
>> --- a/lib/librte_eal/linuxapp/eal/eal_pci.c
>> +++ b/lib/librte_eal/linuxapp/eal/eal_pci.c
>> @@ -378,8 +378,8 @@ error:
>>   * Scan the content of the PCI bus, and the devices in the devices
>>   * list
>>   */
>> -static int
>> -pci_scan(void)
>> +int
>> +rte_eal_pci_scan(void)
>>  {
>>  	struct dirent *e;
>>  	DIR *dir;
>> @@ -701,7 +701,7 @@ rte_eal_pci_init(void)
>>  	if (internal_config.no_pci)
>>  		return 0;
>>  
>> -	if (pci_scan() < 0) {
>> +	if (rte_eal_pci_scan() < 0) {
>>  		RTE_LOG(ERR, EAL, "%s(): Cannot scan PCI bus\n", __func__);
>>  		return -1;
>>  	}

^ permalink raw reply	[flat|nested] 362+ messages in thread

* Re: [dpdk-dev] [PATCH v4 05/11] ethdev: Add functions that will be used by port hotplug functions
  2015-01-21  2:40     ` Qiu, Michael
@ 2015-01-21  9:02       ` Tetsuya Mukawa
  0 siblings, 0 replies; 362+ messages in thread
From: Tetsuya Mukawa @ 2015-01-21  9:02 UTC (permalink / raw)
  To: Qiu, Michael; +Cc: dev

Hi Michael,

On 2015/01/21 11:40, Qiu, Michael wrote:
> On 1/19/2015 6:42 PM, Tetsuya Mukawa wrote:
>> The patch adds following functions.
>>
>> - rte_eth_dev_save()
>>   The function is used for saving current rte_eth_dev structures.
>> - rte_eth_dev_get_changed_port()
>>   The function receives the rte_eth_dev structures, then compare
>>   these with current values to know which port is actually
>>   attached or detached.
>> - rte_eth_dev_get_addr_by_port()
>>   The function returns a pci address of a ethdev specified by port
>>   identifier.
>> - rte_eth_dev_get_port_by_addr()
>>   The function returns a port identifier of a ethdev specified by
>>   pci address.
>> - rte_eth_dev_get_name_by_port()
>>   The function returns a unique identifier name of a ethdev
>>   specified by port identifier.
>> - Add rte_eth_dev_check_detachable()
>>   The function returns whether a PMD supports detach function.
>>
>> Also the patch changes scope of rte_eth_dev_allocated() to global.
>> This function will be called by virtual PMDs to support port hotplug.
>> So change scope of the function to global.
>>
>> v4:
>> - Add paramerter checking.
>> v3:
>> - Fix if-condition bug while comparing pci addresses.
>> - Add error checking codes.
>> Reported-by: Mark Enright <menrigh@brocade.com>
>>
>> Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp>
>> ---
>>  lib/librte_ether/rte_ethdev.c | 102 +++++++++++++++++++++++++++++++++++++++++-
>>  lib/librte_ether/rte_ethdev.h |  80 +++++++++++++++++++++++++++++++++
>>  2 files changed, 181 insertions(+), 1 deletion(-)
>>
>> diff --git a/lib/librte_ether/rte_ethdev.c b/lib/librte_ether/rte_ethdev.c
>> index e5145b7..e572ef4 100644
>> --- a/lib/librte_ether/rte_ethdev.c
>> +++ b/lib/librte_ether/rte_ethdev.c
>> @@ -206,7 +206,7 @@ rte_eth_dev_data_alloc(void)
>>  				RTE_MAX_ETHPORTS * sizeof(*rte_eth_dev_data));
>>  }
>>  
>> -static struct rte_eth_dev *
>> +struct rte_eth_dev *
>>  rte_eth_dev_allocated(const char *name)
>>  {
>>  	unsigned i;
>> @@ -422,6 +422,106 @@ rte_eth_dev_count(void)
>>  	return (nb_ports);
>>  }
>>  
>> +void
>> +rte_eth_dev_save(struct rte_eth_dev *devs)
>> +{
>> +	if (devs == NULL)
>> +		return;
>> +
>> +	/* save current rte_eth_devices */
>> +	memcpy(devs, rte_eth_devices,
>> +			sizeof(struct rte_eth_dev) * RTE_MAX_ETHPORTS);
>> +}
>> +
>> +int
>> +rte_eth_dev_get_changed_port(struct rte_eth_dev *devs, uint8_t *port_id)
>> +{
>> +	if ((devs == NULL) || (port_id == NULL))
>> +		return -1;
>> +
>> +	/* check which port was attached or detached */
>> +	for (*port_id = 0; *port_id < RTE_MAX_ETHPORTS; (*port_id)++, devs++) {
>> +		if (rte_eth_devices[*port_id].attached ^ devs->attached)
>> +			return 0;
>> +	}
>> +	return 1;
>> +}
>> +
>> +int
>> +rte_eth_dev_get_addr_by_port(uint8_t port_id, struct rte_pci_addr *addr)
>> +{
>> +	if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
>> +		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
>> +		return -EINVAL;
>> +	}
>> +
>> +	if (addr == NULL) {
>> +		PMD_DEBUG_TRACE("Null pointer is specified\n");
>> +		return -EINVAL;
>> +	}
>> +
>> +	*addr = rte_eth_devices[port_id].pci_dev->addr;
>> +	return 0;
>> +}
>> +
>> +int
>> +rte_eth_dev_get_port_by_addr(struct rte_pci_addr *addr, uint8_t *port_id)
>> +{
>> +	struct rte_pci_addr *tmp;
>> +
>> +	if ((addr == NULL) || (port_id == NULL)) {
>> +		PMD_DEBUG_TRACE("Null pointer is specified\n");
>> +		return -1;
> Is it better to replace "-1" to "-EINVAL" ?

Thanks. I will do like above.

Regards,
Tetsuya

>
>> +	}
>> +
>> +	for (*port_id = 0; *port_id < RTE_MAX_ETHPORTS; (*port_id)++) {
>> +		if (!rte_eth_devices[*port_id].attached)
>> +			continue;
>> +		if (!rte_eth_devices[*port_id].pci_dev)
>> +			continue;
>> +		tmp = &rte_eth_devices[*port_id].pci_dev->addr;
>> +		if (eal_compare_pci_addr(tmp, addr) == 0)
>> +			return 0;
>> +	}
>> +	return -1;
>> +}
>> +
>> +int
>> +rte_eth_dev_get_name_by_port(uint8_t port_id, char *name)
>> +{
>> +	char *tmp;
>> +
>> +	if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
>> +		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
>> +		return -EINVAL;
>> +	}
>> +
>> +	if (name == NULL) {
>> +		PMD_DEBUG_TRACE("Null pointer is specified\n");
>> +		return -EINVAL;
>> +	}
>> +
>> +	/* shouldn't check 'rte_eth_devices[i].data',
>> +	 * because it might be overwritten by VDEV PMD */
>> +	tmp = rte_eth_dev_data[port_id].name;
>> +	strncpy(name, tmp, strlen(tmp) + 1);
>> +	return 0;
>> +}
>> +
>> +int
>> +rte_eth_dev_check_detachable(uint8_t port_id)
>> +{
>> +	uint32_t drv_flags;
>> +
>> +	if (port_id >= RTE_MAX_ETHPORTS) {
>> +		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
>> +		return -EINVAL;
>> +	}
>> +
>> +	drv_flags = rte_eth_devices[port_id].driver->pci_drv.drv_flags;
>> +	return !(drv_flags & RTE_PCI_DRV_DETACHABLE);
>> +}
>> +
>>  static int
>>  rte_eth_dev_rx_queue_config(struct rte_eth_dev *dev, uint16_t nb_queues)
>>  {
>> diff --git a/lib/librte_ether/rte_ethdev.h b/lib/librte_ether/rte_ethdev.h
>> index 2a8ff26..c0b98dc 100644
>> --- a/lib/librte_ether/rte_ethdev.h
>> +++ b/lib/librte_ether/rte_ethdev.h
>> @@ -1642,6 +1642,86 @@ extern struct rte_eth_dev rte_eth_devices[];
>>  extern uint8_t rte_eth_dev_count(void);
>>  
>>  /**
>> + * Function for internal use by port hotplug functions.
>> + * Copies current ethdev structures to the specified pointer.
>> + *
>> + * @param	devs	The pointer to the ethdev structures
>> + */
>> +extern void rte_eth_dev_save(struct rte_eth_dev *devs);
>> +
>> +/**
>> + * Function for internal use by port hotplug functions.
>> + * Compare the specified ethdev structures with currents. Then
>> + * if there is a port which status is changed, fill the specified pointer
>> + * with the port id of that port.
>> + * @param	devs	The pointer to the ethdev structures
>> + * @param	port_id	The pointer to the port id
>> + * @return
>> + *   - 0 on success, negative on error
>> + */
>> +extern int rte_eth_dev_get_changed_port(
>> +		struct rte_eth_dev *devs, uint8_t *port_id);
>> +
>> +/**
>> + * Function for internal use by port hotplug functions.
>> + * Returns a pci address of a ethdev specified by port identifier.
>> + * @param	port_id
>> + *   The port identifier of the Ethernet device
>> + * @param	addr
>> + *   The pointer to the pci address
>> + * @return
>> + *   - 0 on success, negative on error
>> + */
>> +extern int rte_eth_dev_get_addr_by_port(
>> +		uint8_t port_id, struct rte_pci_addr *addr);
>> +
>> +/**
>> + * Function for internal use by port hotplug functions.
>> + * Returns a port identifier of a ethdev specified by pci address.
>> + * @param	addr
>> + *   The pointer to the pci address of the Ethernet device.
>> + * @param	port_id
>> + *   The pointer to the port identifier
>> + * @return
>> + *   - 0 on success, negative on error
>> + */
>> +extern int rte_eth_dev_get_port_by_addr(
>> +		struct rte_pci_addr *addr, uint8_t *port_id);
>> +
>> +/**
>> + * Function for internal use by port hotplug functions.
>> + * Returns a unique identifier name of a ethdev specified by port identifier.
>> + * @param	port_id
>> + *   The port identifier.
>> + * @param	name
>> + *  The pointer to the Unique identifier name for each Ethernet device
>> + * @return
>> + *   - 0 on success, negative on error
>> + */
>> +extern int rte_eth_dev_get_name_by_port(uint8_t port_id, char *name);
>> +
>> +/**
>> + * Function for internal use by port hotplug functions.
>> + * Check whether or not, a PMD that is handling the ethdev specified by port
>> + * identifier can support detach function.
>> + * @param	port_id
>> + *   The port identifier
>> + * @return
>> + *   - 0 on supporting detach function, negative on not supporting
>> + */
>> +extern int rte_eth_dev_check_detachable(uint8_t port_id);
>> +
>> +/**
>> + * Function for internal use by port hotplug functions.
>> + * Returns a ethdev slot specified by the unique identifier name.
>> + * @param	name
>> + *  The pointer to the Unique identifier name for each Ethernet device
>> + * @return
>> + *   - The pointer to the ethdev slot, on success. NULL on error
>> + */
>> +extern struct rte_eth_dev *rte_eth_dev_allocated(const char *name);
>> +
>> +/**
>>   * Function for internal use by dummy drivers primarily, e.g. ring-based
>>   * driver.
>>   * Allocates a new ethdev slot for an ethernet device and returns the pointer

^ permalink raw reply	[flat|nested] 362+ messages in thread

* Re: [dpdk-dev] [PATCH v4 06/11] eal/linux/pci: Add functions for unmapping igb_uio resources
  2015-01-20  9:23     ` Qiu, Michael
  2015-01-21  0:17       ` Tetsuya Mukawa
@ 2015-01-21 10:01       ` Tetsuya Mukawa
  2015-01-22  8:12         ` Qiu, Michael
  1 sibling, 1 reply; 362+ messages in thread
From: Tetsuya Mukawa @ 2015-01-21 10:01 UTC (permalink / raw)
  To: Qiu, Michael; +Cc: dev

Hi Michael,

On 2015/01/20 18:23, Qiu, Michael wrote:
> On 1/19/2015 6:42 PM, Tetsuya Mukawa wrote:
>> The patch adds functions for unmapping igb_uio resources. The patch is only
>> for Linux and igb_uio environment. VFIO and BSD are not supported.
>>
>> v4:
>> - Add paramerter checking.
>> - Add header file to determine if hotplug can be enabled.
>>
>> Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp>
>> ---
>>  lib/librte_eal/common/Makefile                  |  1 +
>>  lib/librte_eal/common/include/rte_dev_hotplug.h | 44 +++++++++++++++++
>>  lib/librte_eal/linuxapp/eal/eal_pci.c           | 38 +++++++++++++++
>>  lib/librte_eal/linuxapp/eal/eal_pci_init.h      |  8 +++
>>  lib/librte_eal/linuxapp/eal/eal_pci_uio.c       | 65 +++++++++++++++++++++++++
>>  5 files changed, 156 insertions(+)
>>  create mode 100644 lib/librte_eal/common/include/rte_dev_hotplug.h
>>
>> diff --git a/lib/librte_eal/common/Makefile b/lib/librte_eal/common/Makefile
>> index 52c1a5f..db7cc93 100644
>> --- a/lib/librte_eal/common/Makefile
>> +++ b/lib/librte_eal/common/Makefile
>> @@ -41,6 +41,7 @@ INC += rte_eal_memconfig.h rte_malloc_heap.h
>>  INC += rte_hexdump.h rte_devargs.h rte_dev.h
>>  INC += rte_common_vect.h
>>  INC += rte_pci_dev_feature_defs.h rte_pci_dev_features.h
>> +INC += rte_dev_hotplug.h
>>  
>>  ifeq ($(CONFIG_RTE_INSECURE_FUNCTION_WARNING),y)
>>  INC += rte_warnings.h
>> diff --git a/lib/librte_eal/common/include/rte_dev_hotplug.h b/lib/librte_eal/common/include/rte_dev_hotplug.h
>> new file mode 100644
>> index 0000000..b333e0f
>> --- /dev/null
>> +++ b/lib/librte_eal/common/include/rte_dev_hotplug.h
>> @@ -0,0 +1,44 @@
>> +/*-
>> + *   BSD LICENSE
>> + *
>> + *   Copyright(c) 2015 IGEL Co.,LTd.
>> + *   All rights reserved.
>> + *
>> + *   Redistribution and use in source and binary forms, with or without
>> + *   modification, are permitted provided that the following conditions
>> + *   are met:
>> + *
>> + *     * Redistributions of source code must retain the above copyright
>> + *       notice, this list of conditions and the following disclaimer.
>> + *     * Redistributions in binary form must reproduce the above copyright
>> + *       notice, this list of conditions and the following disclaimer in
>> + *       the documentation and/or other materials provided with the
>> + *       distribution.
>> + *     * Neither the name of IGEL Co.,Ltd. nor the names of its
>> + *       contributors may be used to endorse or promote products derived
>> + *       from this software without specific prior written permission.
>> + *
>> + *   THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
>> + *   "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
>> + *   LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
>> + *   A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
>> + *   OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
>> + *   SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
>> + *   LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
>> + *   DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
>> + *   THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
>> + *   (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
>> + *   OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
>> + */
>> +
>> +#ifndef _RTE_DEV_HOTPLUG_H_
>> +#define _RTE_DEV_HOTPLUG_H_
>> +
>> +/*
>> + * determine if hotplug can be enabled on the system
>> + */
>> +#if defined(RTE_LIBRTE_EAL_HOTPLUG) && defined(RTE_LIBRTE_EAL_LINUXAPP)
> As you said, VFIO should not work with it, so does it need to add the
> vfio check here?

Could I have a advice of you?
First I guess it's the best to include "eal_vfio.h" here, and add
checking of VFIO_PRESENT macro.
But it seems I cannot reach "eal_vfio.h" from this file.

My second option is just checking RTE_EAL_VFIO macro.
But according to "eal_vfio.h", if kernel is under 3.6.0, VFIO_PRESENT
will not be defined even when RTE_EAL_VFIO is enabled.
So I guess simply macro checking will not work correctly.
 
Anyway, here are my implementation choices so far.

1. Like "eal_vfio.h", check kernel version in "rte_dev_hotplug.h".
In this case, if "eal_vfio.h" is changed, "rte_edv_hotplug.h" may need
to be changed also.

2. Merge "eal_vfio.h" and "rte_dev_hotplug.h" definitions, and define
these in new rte header like "rte_settings.h".

Can I have advice about it?

Thanks,
Tetsuya

>
> Thanks,
> Michael
>> +#define ENABLE_HOTPLUG
>> +#endif /* RTE_LIBRTE_EAL_HOTPLUG & RTE_LIBRTE_EAL_LINUXAPP */
>> +
>> +#endif /* _RTE_DEV_HOTPLUG_H_ */
>> diff --git a/lib/librte_eal/linuxapp/eal/eal_pci.c b/lib/librte_eal/linuxapp/eal/eal_pci.c
>> index 3d2d93c..52c464c 100644
>> --- a/lib/librte_eal/linuxapp/eal/eal_pci.c
>> +++ b/lib/librte_eal/linuxapp/eal/eal_pci.c
>> @@ -137,6 +137,25 @@ pci_map_resource(void *requested_addr, int fd, off_t offset, size_t size)
>>  	return mapaddr;
>>  }
>>  
>> +#ifdef ENABLE_HOTPLUG
>> +/* unmap a particular resource */
>> +void
>> +pci_unmap_resource(void *requested_addr, size_t size)
>> +{
>> +	if (requested_addr == NULL)
>> +		return;
>> +
>> +	/* Unmap the PCI memory resource of device */
>> +	if (munmap(requested_addr, size)) {
>> +		RTE_LOG(ERR, EAL, "%s(): cannot munmap(%p, 0x%lx): %s\n",
>> +			__func__, requested_addr, (unsigned long)size,
>> +			strerror(errno));
>> +	} else
>> +		RTE_LOG(DEBUG, EAL, "  PCI memory mapped at %p\n",
>> +				requested_addr);
>> +}
>> +#endif /* ENABLE_HOTPLUG */
>> +
>>  /* parse the "resource" sysfs file */
>>  #define IORESOURCE_MEM  0x00000200
>>  
>> @@ -510,6 +529,25 @@ pci_map_device(struct rte_pci_device *dev)
>>  	return 0;
>>  }
>>  
>> +#ifdef ENABLE_HOTPLUG
>> +static void
>> +pci_unmap_device(struct rte_pci_device *dev)
>> +{
>> +	if (dev == NULL)
>> +		return;
>> +
>> +	/* try unmapping the NIC resources using VFIO if it exists */
>> +#ifdef VFIO_PRESENT
>> +	if (pci_vfio_is_enabled()) {
>> +		RTE_LOG(ERR, EAL, "%s() doesn't support vfio yet.\n",
>> +				__func__);
>> +		return;
>> +	}
>> +#endif
>> +	pci_uio_unmap_resource(dev);
>> +}
>> +#endif /* ENABLE_HOTPLUG */
>> +
>>  /*
>>   * If vendor/device ID match, call the devinit() function of the
>>   * driver.
>> diff --git a/lib/librte_eal/linuxapp/eal/eal_pci_init.h b/lib/librte_eal/linuxapp/eal/eal_pci_init.h
>> index 1070eb8..5152a0b 100644
>> --- a/lib/librte_eal/linuxapp/eal/eal_pci_init.h
>> +++ b/lib/librte_eal/linuxapp/eal/eal_pci_init.h
>> @@ -34,6 +34,7 @@
>>  #ifndef EAL_PCI_INIT_H_
>>  #define EAL_PCI_INIT_H_
>>  
>> +#include <rte_dev_hotplug.h>
>>  #include "eal_vfio.h"
>>  
>>  struct pci_map {
>> @@ -71,6 +72,13 @@ void *pci_map_resource(void *requested_addr, int fd, off_t offset,
>>  /* map IGB_UIO resource prototype */
>>  int pci_uio_map_resource(struct rte_pci_device *dev);
>>  
>> +#ifdef ENABLE_HOTPLUG
>> +void pci_unmap_resource(void *requested_addr, size_t size);
>> +
>> +/* unmap IGB_UIO resource prototype */
>> +void pci_uio_unmap_resource(struct rte_pci_device *dev);
>> +#endif /* ENABLE_HOTPLUG */
>> +
>>  #ifdef VFIO_PRESENT
>>  
>>  #define VFIO_MAX_GROUPS 64
>> diff --git a/lib/librte_eal/linuxapp/eal/eal_pci_uio.c b/lib/librte_eal/linuxapp/eal/eal_pci_uio.c
>> index 1da3507..81830d1 100644
>> --- a/lib/librte_eal/linuxapp/eal/eal_pci_uio.c
>> +++ b/lib/librte_eal/linuxapp/eal/eal_pci_uio.c
>> @@ -404,6 +404,71 @@ pci_uio_map_resource(struct rte_pci_device *dev)
>>  	return 0;
>>  }
>>  
>> +#ifdef ENABLE_HOTPLUG
>> +static void
>> +pci_uio_unmap(struct mapped_pci_resource *uio_res)
>> +{
>> +	int i;
>> +
>> +	if (uio_res == NULL)
>> +		return;
>> +
>> +	for (i = 0; i != uio_res->nb_maps; i++)
>> +		pci_unmap_resource(uio_res->maps[i].addr,
>> +				(size_t)uio_res->maps[i].size);
>> +}
>> +
>> +static struct mapped_pci_resource *
>> +pci_uio_find_resource(struct rte_pci_device *dev)
>> +{
>> +	struct mapped_pci_resource *uio_res;
>> +
>> +	if (dev == NULL)
>> +		return NULL;
>> +
>> +	TAILQ_FOREACH(uio_res, pci_res_list, next) {
>> +
>> +		/* skip this element if it doesn't match our PCI address */
>> +		if (!eal_compare_pci_addr(&uio_res->pci_addr, &dev->addr))
>> +			return uio_res;
>> +	}
>> +	return NULL;
>> +}
>> +
>> +/* unmap the PCI resource of a PCI device in virtual memory */
>> +void
>> +pci_uio_unmap_resource(struct rte_pci_device *dev)
>> +{
>> +	struct mapped_pci_resource *uio_res;
>> +
>> +	if (dev == NULL)
>> +		return;
>> +
>> +	/* find an entry for the device */
>> +	uio_res = pci_uio_find_resource(dev);
>> +	if (uio_res == NULL)
>> +		return;
>> +
>> +	/* secondary processes - just free maps */
>> +	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
>> +		return pci_uio_unmap(uio_res);
>> +
>> +	TAILQ_REMOVE(pci_res_list, uio_res, next);
>> +
>> +	/* unmap all resources */
>> +	pci_uio_unmap(uio_res);
>> +
>> +	/* free uio resource */
>> +	rte_free(uio_res);
>> +
>> +	/* close fd if in primary process */
>> +	close(dev->intr_handle.fd);
>> +
>> +	dev->intr_handle.fd = -1;
>> +	dev->intr_handle.type = RTE_INTR_HANDLE_UNKNOWN;
>> +}
>> +#endif /* ENABLE_HOTPLUG */
>> +
>>  /*
>>   * parse a sysfs file containing one integer value
>>   * different to the eal version, as it needs to work with 64-bit values

^ permalink raw reply	[flat|nested] 362+ messages in thread

* Re: [dpdk-dev] [PATCH v4 06/11] eal/linux/pci: Add functions for unmapping igb_uio resources
  2015-01-21 10:01       ` Tetsuya Mukawa
@ 2015-01-22  8:12         ` Qiu, Michael
  2015-01-22 10:15           ` Tetsuya Mukawa
  0 siblings, 1 reply; 362+ messages in thread
From: Qiu, Michael @ 2015-01-22  8:12 UTC (permalink / raw)
  To: Tetsuya Mukawa; +Cc: dev

On 1/21/2015 6:01 PM, Tetsuya Mukawa wrote:
> Hi Michael,
>
> On 2015/01/20 18:23, Qiu, Michael wrote:
>> On 1/19/2015 6:42 PM, Tetsuya Mukawa wrote:
>>> The patch adds functions for unmapping igb_uio resources. The patch is only
>>> for Linux and igb_uio environment. VFIO and BSD are not supported.
>>>
>>> v4:
>>> - Add paramerter checking.
>>> - Add header file to determine if hotplug can be enabled.
>>>
>>> Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp>
>>> ---
>>>  lib/librte_eal/common/Makefile                  |  1 +
>>>  lib/librte_eal/common/include/rte_dev_hotplug.h | 44 +++++++++++++++++
>>>  lib/librte_eal/linuxapp/eal/eal_pci.c           | 38 +++++++++++++++
>>>  lib/librte_eal/linuxapp/eal/eal_pci_init.h      |  8 +++
>>>  lib/librte_eal/linuxapp/eal/eal_pci_uio.c       | 65 +++++++++++++++++++++++++
>>>  5 files changed, 156 insertions(+)
>>>  create mode 100644 lib/librte_eal/common/include/rte_dev_hotplug.h
>>>
>>> diff --git a/lib/librte_eal/common/Makefile b/lib/librte_eal/common/Makefile
>>> index 52c1a5f..db7cc93 100644
>>> --- a/lib/librte_eal/common/Makefile
>>> +++ b/lib/librte_eal/common/Makefile
>>> @@ -41,6 +41,7 @@ INC += rte_eal_memconfig.h rte_malloc_heap.h
>>>  INC += rte_hexdump.h rte_devargs.h rte_dev.h
>>>  INC += rte_common_vect.h
>>>  INC += rte_pci_dev_feature_defs.h rte_pci_dev_features.h
>>> +INC += rte_dev_hotplug.h
>>>  
>>>  ifeq ($(CONFIG_RTE_INSECURE_FUNCTION_WARNING),y)
>>>  INC += rte_warnings.h
>>> diff --git a/lib/librte_eal/common/include/rte_dev_hotplug.h b/lib/librte_eal/common/include/rte_dev_hotplug.h
>>> new file mode 100644
>>> index 0000000..b333e0f
>>> --- /dev/null
>>> +++ b/lib/librte_eal/common/include/rte_dev_hotplug.h
>>> @@ -0,0 +1,44 @@
>>> +/*-
>>> + *   BSD LICENSE
>>> + *
>>> + *   Copyright(c) 2015 IGEL Co.,LTd.
>>> + *   All rights reserved.
>>> + *
>>> + *   Redistribution and use in source and binary forms, with or without
>>> + *   modification, are permitted provided that the following conditions
>>> + *   are met:
>>> + *
>>> + *     * Redistributions of source code must retain the above copyright
>>> + *       notice, this list of conditions and the following disclaimer.
>>> + *     * Redistributions in binary form must reproduce the above copyright
>>> + *       notice, this list of conditions and the following disclaimer in
>>> + *       the documentation and/or other materials provided with the
>>> + *       distribution.
>>> + *     * Neither the name of IGEL Co.,Ltd. nor the names of its
>>> + *       contributors may be used to endorse or promote products derived
>>> + *       from this software without specific prior written permission.
>>> + *
>>> + *   THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
>>> + *   "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
>>> + *   LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
>>> + *   A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
>>> + *   OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
>>> + *   SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
>>> + *   LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
>>> + *   DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
>>> + *   THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
>>> + *   (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
>>> + *   OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
>>> + */
>>> +
>>> +#ifndef _RTE_DEV_HOTPLUG_H_
>>> +#define _RTE_DEV_HOTPLUG_H_
>>> +
>>> +/*
>>> + * determine if hotplug can be enabled on the system
>>> + */
>>> +#if defined(RTE_LIBRTE_EAL_HOTPLUG) && defined(RTE_LIBRTE_EAL_LINUXAPP)
>> As you said, VFIO should not work with it, so does it need to add the
>> vfio check here?
> Could I have a advice of you?
> First I guess it's the best to include "eal_vfio.h" here, and add
> checking of VFIO_PRESENT macro.


I have a question, will your hotplug  feature support freebsd ?

If not, how about to put it in  "lib/librte_eal/linuxapp/eal/" ? Also 
include attach or detach affairs.

> But it seems I cannot reach "eal_vfio.h" from this file.

Yes, you can't :)

> My second option is just checking RTE_EAL_VFIO macro.
> But according to "eal_vfio.h", if kernel is under 3.6.0, VFIO_PRESENT

Actually,  in my opinion, whatever vfio or uio, only need be care in
runtime.

DPDK to check vfio only to add support  for vfio, but this does not
means the device will use vfio,

So even if VFIO_PRESENT is defined, and vfio is enabled, but the device
is bind to igb_uio, then your hotplug still  need work, but if it bind
to vfio, will not, am I right?

If yes, I'm not sure if your hotplug has this ability, but it is
reasonable, I think.

> will not be defined even when RTE_EAL_VFIO is enabled.
> So I guess simply macro checking will not work correctly.
>  
> Anyway, here are my implementation choices so far.
>
> 1. Like "eal_vfio.h", check kernel version in "rte_dev_hotplug.h".
> In this case, if "eal_vfio.h" is changed, "rte_edv_hotplug.h" may need
> to be changed also.
>
> 2. Merge "eal_vfio.h" and "rte_dev_hotplug.h" definitions, and define
> these in new rte header like "rte_settings.h".
>
> Can I have advice about it?

As I said, VFIO enable or not is not related for your hotplug, only the
devices managed by VFIO will affect your hotplug.

If you agree, I think need discuss the details of it.

Thanks,
Michael
> Thanks,
> Tetsuya
>
>> Thanks,
>> Michael
>>> +#define ENABLE_HOTPLUG
>>> +#endif /* RTE_LIBRTE_EAL_HOTPLUG & RTE_LIBRTE_EAL_LINUXAPP */
>>> +
>>> +#endif /* _RTE_DEV_HOTPLUG_H_ */
>>> diff --git a/lib/librte_eal/linuxapp/eal/eal_pci.c b/lib/librte_eal/linuxapp/eal/eal_pci.c
>>> index 3d2d93c..52c464c 100644
>>> --- a/lib/librte_eal/linuxapp/eal/eal_pci.c
>>> +++ b/lib/librte_eal/linuxapp/eal/eal_pci.c
>>> @@ -137,6 +137,25 @@ pci_map_resource(void *requested_addr, int fd, off_t offset, size_t size)
>>>  	return mapaddr;
>>>  }
>>>  
>>> +#ifdef ENABLE_HOTPLUG
>>> +/* unmap a particular resource */
>>> +void
>>> +pci_unmap_resource(void *requested_addr, size_t size)
>>> +{
>>> +	if (requested_addr == NULL)
>>> +		return;
>>> +
>>> +	/* Unmap the PCI memory resource of device */
>>> +	if (munmap(requested_addr, size)) {
>>> +		RTE_LOG(ERR, EAL, "%s(): cannot munmap(%p, 0x%lx): %s\n",
>>> +			__func__, requested_addr, (unsigned long)size,
>>> +			strerror(errno));
>>> +	} else
>>> +		RTE_LOG(DEBUG, EAL, "  PCI memory mapped at %p\n",
>>> +				requested_addr);
>>> +}
>>> +#endif /* ENABLE_HOTPLUG */
>>> +
>>>  /* parse the "resource" sysfs file */
>>>  #define IORESOURCE_MEM  0x00000200
>>>  
>>> @@ -510,6 +529,25 @@ pci_map_device(struct rte_pci_device *dev)
>>>  	return 0;
>>>  }
>>>  
>>> +#ifdef ENABLE_HOTPLUG
>>> +static void
>>> +pci_unmap_device(struct rte_pci_device *dev)
>>> +{
>>> +	if (dev == NULL)
>>> +		return;
>>> +
>>> +	/* try unmapping the NIC resources using VFIO if it exists */
>>> +#ifdef VFIO_PRESENT
>>> +	if (pci_vfio_is_enabled()) {
>>> +		RTE_LOG(ERR, EAL, "%s() doesn't support vfio yet.\n",
>>> +				__func__);
>>> +		return;
>>> +	}
>>> +#endif
>>> +	pci_uio_unmap_resource(dev);
>>> +}
>>> +#endif /* ENABLE_HOTPLUG */
>>> +
>>>  /*
>>>   * If vendor/device ID match, call the devinit() function of the
>>>   * driver.
>>> diff --git a/lib/librte_eal/linuxapp/eal/eal_pci_init.h b/lib/librte_eal/linuxapp/eal/eal_pci_init.h
>>> index 1070eb8..5152a0b 100644
>>> --- a/lib/librte_eal/linuxapp/eal/eal_pci_init.h
>>> +++ b/lib/librte_eal/linuxapp/eal/eal_pci_init.h
>>> @@ -34,6 +34,7 @@
>>>  #ifndef EAL_PCI_INIT_H_
>>>  #define EAL_PCI_INIT_H_
>>>  
>>> +#include <rte_dev_hotplug.h>
>>>  #include "eal_vfio.h"
>>>  
>>>  struct pci_map {
>>> @@ -71,6 +72,13 @@ void *pci_map_resource(void *requested_addr, int fd, off_t offset,
>>>  /* map IGB_UIO resource prototype */
>>>  int pci_uio_map_resource(struct rte_pci_device *dev);
>>>  
>>> +#ifdef ENABLE_HOTPLUG
>>> +void pci_unmap_resource(void *requested_addr, size_t size);
>>> +
>>> +/* unmap IGB_UIO resource prototype */
>>> +void pci_uio_unmap_resource(struct rte_pci_device *dev);
>>> +#endif /* ENABLE_HOTPLUG */
>>> +
>>>  #ifdef VFIO_PRESENT
>>>  
>>>  #define VFIO_MAX_GROUPS 64
>>> diff --git a/lib/librte_eal/linuxapp/eal/eal_pci_uio.c b/lib/librte_eal/linuxapp/eal/eal_pci_uio.c
>>> index 1da3507..81830d1 100644
>>> --- a/lib/librte_eal/linuxapp/eal/eal_pci_uio.c
>>> +++ b/lib/librte_eal/linuxapp/eal/eal_pci_uio.c
>>> @@ -404,6 +404,71 @@ pci_uio_map_resource(struct rte_pci_device *dev)
>>>  	return 0;
>>>  }
>>>  
>>> +#ifdef ENABLE_HOTPLUG
>>> +static void
>>> +pci_uio_unmap(struct mapped_pci_resource *uio_res)
>>> +{
>>> +	int i;
>>> +
>>> +	if (uio_res == NULL)
>>> +		return;
>>> +
>>> +	for (i = 0; i != uio_res->nb_maps; i++)
>>> +		pci_unmap_resource(uio_res->maps[i].addr,
>>> +				(size_t)uio_res->maps[i].size);
>>> +}
>>> +
>>> +static struct mapped_pci_resource *
>>> +pci_uio_find_resource(struct rte_pci_device *dev)
>>> +{
>>> +	struct mapped_pci_resource *uio_res;
>>> +
>>> +	if (dev == NULL)
>>> +		return NULL;
>>> +
>>> +	TAILQ_FOREACH(uio_res, pci_res_list, next) {
>>> +
>>> +		/* skip this element if it doesn't match our PCI address */
>>> +		if (!eal_compare_pci_addr(&uio_res->pci_addr, &dev->addr))
>>> +			return uio_res;
>>> +	}
>>> +	return NULL;
>>> +}
>>> +
>>> +/* unmap the PCI resource of a PCI device in virtual memory */
>>> +void
>>> +pci_uio_unmap_resource(struct rte_pci_device *dev)
>>> +{
>>> +	struct mapped_pci_resource *uio_res;
>>> +
>>> +	if (dev == NULL)
>>> +		return;
>>> +
>>> +	/* find an entry for the device */
>>> +	uio_res = pci_uio_find_resource(dev);
>>> +	if (uio_res == NULL)
>>> +		return;
>>> +
>>> +	/* secondary processes - just free maps */
>>> +	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
>>> +		return pci_uio_unmap(uio_res);
>>> +
>>> +	TAILQ_REMOVE(pci_res_list, uio_res, next);
>>> +
>>> +	/* unmap all resources */
>>> +	pci_uio_unmap(uio_res);
>>> +
>>> +	/* free uio resource */
>>> +	rte_free(uio_res);
>>> +
>>> +	/* close fd if in primary process */
>>> +	close(dev->intr_handle.fd);
>>> +
>>> +	dev->intr_handle.fd = -1;
>>> +	dev->intr_handle.type = RTE_INTR_HANDLE_UNKNOWN;
>>> +}
>>> +#endif /* ENABLE_HOTPLUG */
>>> +
>>>  /*
>>>   * parse a sysfs file containing one integer value
>>>   * different to the eal version, as it needs to work with 64-bit values
>
>


^ permalink raw reply	[flat|nested] 362+ messages in thread

* Re: [dpdk-dev] [PATCH v4 06/11] eal/linux/pci: Add functions for unmapping igb_uio resources
  2015-01-22  8:12         ` Qiu, Michael
@ 2015-01-22 10:15           ` Tetsuya Mukawa
  2015-01-23  2:54             ` Qiu, Michael
  0 siblings, 1 reply; 362+ messages in thread
From: Tetsuya Mukawa @ 2015-01-22 10:15 UTC (permalink / raw)
  To: Qiu, Michael; +Cc: dev

Hi Michael,

On 2015/01/22 17:12, Qiu, Michael wrote:
> On 1/21/2015 6:01 PM, Tetsuya Mukawa wrote:
>> Hi Michael,
>>
>> On 2015/01/20 18:23, Qiu, Michael wrote:
>>> On 1/19/2015 6:42 PM, Tetsuya Mukawa wrote:
>>>> The patch adds functions for unmapping igb_uio resources. The patch is only
>>>> for Linux and igb_uio environment. VFIO and BSD are not supported.
>>>>
>>>> v4:
>>>> - Add paramerter checking.
>>>> - Add header file to determine if hotplug can be enabled.
>>>>
>>>> Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp>
>>>> ---
>>>>  lib/librte_eal/common/Makefile                  |  1 +
>>>>  lib/librte_eal/common/include/rte_dev_hotplug.h | 44 +++++++++++++++++
>>>>  lib/librte_eal/linuxapp/eal/eal_pci.c           | 38 +++++++++++++++
>>>>  lib/librte_eal/linuxapp/eal/eal_pci_init.h      |  8 +++
>>>>  lib/librte_eal/linuxapp/eal/eal_pci_uio.c       | 65 +++++++++++++++++++++++++
>>>>  5 files changed, 156 insertions(+)
>>>>  create mode 100644 lib/librte_eal/common/include/rte_dev_hotplug.h
>>>>
>>>> diff --git a/lib/librte_eal/common/Makefile b/lib/librte_eal/common/Makefile
>>>> index 52c1a5f..db7cc93 100644
>>>> --- a/lib/librte_eal/common/Makefile
>>>> +++ b/lib/librte_eal/common/Makefile
>>>> @@ -41,6 +41,7 @@ INC += rte_eal_memconfig.h rte_malloc_heap.h
>>>>  INC += rte_hexdump.h rte_devargs.h rte_dev.h
>>>>  INC += rte_common_vect.h
>>>>  INC += rte_pci_dev_feature_defs.h rte_pci_dev_features.h
>>>> +INC += rte_dev_hotplug.h
>>>>  
>>>>  ifeq ($(CONFIG_RTE_INSECURE_FUNCTION_WARNING),y)
>>>>  INC += rte_warnings.h
>>>> diff --git a/lib/librte_eal/common/include/rte_dev_hotplug.h b/lib/librte_eal/common/include/rte_dev_hotplug.h
>>>> new file mode 100644
>>>> index 0000000..b333e0f
>>>> --- /dev/null
>>>> +++ b/lib/librte_eal/common/include/rte_dev_hotplug.h
>>>> @@ -0,0 +1,44 @@
>>>> +/*-
>>>> + *   BSD LICENSE
>>>> + *
>>>> + *   Copyright(c) 2015 IGEL Co.,LTd.
>>>> + *   All rights reserved.
>>>> + *
>>>> + *   Redistribution and use in source and binary forms, with or without
>>>> + *   modification, are permitted provided that the following conditions
>>>> + *   are met:
>>>> + *
>>>> + *     * Redistributions of source code must retain the above copyright
>>>> + *       notice, this list of conditions and the following disclaimer.
>>>> + *     * Redistributions in binary form must reproduce the above copyright
>>>> + *       notice, this list of conditions and the following disclaimer in
>>>> + *       the documentation and/or other materials provided with the
>>>> + *       distribution.
>>>> + *     * Neither the name of IGEL Co.,Ltd. nor the names of its
>>>> + *       contributors may be used to endorse or promote products derived
>>>> + *       from this software without specific prior written permission.
>>>> + *
>>>> + *   THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
>>>> + *   "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
>>>> + *   LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
>>>> + *   A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
>>>> + *   OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
>>>> + *   SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
>>>> + *   LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
>>>> + *   DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
>>>> + *   THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
>>>> + *   (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
>>>> + *   OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
>>>> + */
>>>> +
>>>> +#ifndef _RTE_DEV_HOTPLUG_H_
>>>> +#define _RTE_DEV_HOTPLUG_H_
>>>> +
>>>> +/*
>>>> + * determine if hotplug can be enabled on the system
>>>> + */
>>>> +#if defined(RTE_LIBRTE_EAL_HOTPLUG) && defined(RTE_LIBRTE_EAL_LINUXAPP)
>>> As you said, VFIO should not work with it, so does it need to add the
>>> vfio check here?
>> Could I have a advice of you?
>> First I guess it's the best to include "eal_vfio.h" here, and add
>> checking of VFIO_PRESENT macro.
>
> I have a question, will your hotplug  feature support freebsd ?
>
> If not, how about to put it in  "lib/librte_eal/linuxapp/eal/" ? Also 
> include attach or detach affairs.

I appreciate your comments.

So far, FreeBSD doesn't support PCI hotplug. So I didn't implement code
for FreeBSD.
But in the future, I want to implement it when FreeBSD supports it.
Also my hotplug implementation depends on legacy code already
implemented in common layer.
Anyway, for me it's nice to implement the feature in common layer.

>> But it seems I cannot reach "eal_vfio.h" from this file.
> Yes, you can't :)
>
>> My second option is just checking RTE_EAL_VFIO macro.
>> But according to "eal_vfio.h", if kernel is under 3.6.0, VFIO_PRESENT
> Actually,  in my opinion, whatever vfio or uio, only need be care in
> runtime.
>
> DPDK to check vfio only to add support  for vfio, but this does not
> means the device will use vfio,
>
> So even if VFIO_PRESENT is defined, and vfio is enabled, but the device
> is bind to igb_uio, then your hotplug still  need work, but if it bind
> to vfio, will not, am I right?
>
> If yes, I'm not sure if your hotplug has this ability, but it is
> reasonable, I think.

I agree with your concept. But I guess it's not only related with
hotplug function.
The hotplug implementation depends on legacy functions that is for
probing device.
To implement above concept will change not only hotplug behavior but
also legacy device probing.

Conceptually I agree with such a functionality, but legacy probing
function doesn't have such a feature so far.
So I guess it's nice to separate this feature from hotplug patches.
Realistically, the hotplug patches are big, and it's a bit hard to add
and manage one more feature.
If it's ok to separate the patch, it's helpful to manage patches.

>> will not be defined even when RTE_EAL_VFIO is enabled.
>> So I guess simply macro checking will not work correctly.
>>  
>> Anyway, here are my implementation choices so far.
>>
>> 1. Like "eal_vfio.h", check kernel version in "rte_dev_hotplug.h".
>> In this case, if "eal_vfio.h" is changed, "rte_edv_hotplug.h" may need
>> to be changed also.
>>
>> 2. Merge "eal_vfio.h" and "rte_dev_hotplug.h" definitions, and define
>> these in new rte header like "rte_settings.h".
>>
>> Can I have advice about it?
> As I said, VFIO enable or not is not related for your hotplug, only the
> devices managed by VFIO will affect your hotplug.
>
> If you agree, I think need discuss the details of it.

Yes, I agree with your concept.
And if you agree, I will implement it separately.

To discuss how to handle VFIO and igb_uio devices in parallel, I guess
we need to
think about generic uio driver for pci devices.
I guess before applying uio generic patch, this kind of discussion will
be needed.
I hope igb_uio (and VFIO?) be obsolete after applying uio generic.
In the case, we don't need to think it.

BTW, back to 'rte_dev_hotplug.h' discussion of hotplug patch.
If VFIO_PRESENT isn't checked here, attaching port will be successful
even if the device is under VFIO.
(Because we already has legacy code for probing VFIO device. The hotplug
function just re-use it.)
But we cannot detach the device, because there is no code for closing
VFIO device.
There is no crash when the VFIO device is detached, but some error
messages will be displayed.
So I guess this behavior is like your description.
How about it?

Thanks,
Tetsuya

> Thanks,
> Michael
>> Thanks,
>> Tetsuya
>>
>>> Thanks,
>>> Michael
>>>> +#define ENABLE_HOTPLUG
>>>> +#endif /* RTE_LIBRTE_EAL_HOTPLUG & RTE_LIBRTE_EAL_LINUXAPP */
>>>> +
>>>> +#endif /* _RTE_DEV_HOTPLUG_H_ */
>>>> diff --git a/lib/librte_eal/linuxapp/eal/eal_pci.c b/lib/librte_eal/linuxapp/eal/eal_pci.c
>>>> index 3d2d93c..52c464c 100644
>>>> --- a/lib/librte_eal/linuxapp/eal/eal_pci.c
>>>> +++ b/lib/librte_eal/linuxapp/eal/eal_pci.c
>>>> @@ -137,6 +137,25 @@ pci_map_resource(void *requested_addr, int fd, off_t offset, size_t size)
>>>>  	return mapaddr;
>>>>  }
>>>>  
>>>> +#ifdef ENABLE_HOTPLUG
>>>> +/* unmap a particular resource */
>>>> +void
>>>> +pci_unmap_resource(void *requested_addr, size_t size)
>>>> +{
>>>> +	if (requested_addr == NULL)
>>>> +		return;
>>>> +
>>>> +	/* Unmap the PCI memory resource of device */
>>>> +	if (munmap(requested_addr, size)) {
>>>> +		RTE_LOG(ERR, EAL, "%s(): cannot munmap(%p, 0x%lx): %s\n",
>>>> +			__func__, requested_addr, (unsigned long)size,
>>>> +			strerror(errno));
>>>> +	} else
>>>> +		RTE_LOG(DEBUG, EAL, "  PCI memory mapped at %p\n",
>>>> +				requested_addr);
>>>> +}
>>>> +#endif /* ENABLE_HOTPLUG */
>>>> +
>>>>  /* parse the "resource" sysfs file */
>>>>  #define IORESOURCE_MEM  0x00000200
>>>>  
>>>> @@ -510,6 +529,25 @@ pci_map_device(struct rte_pci_device *dev)
>>>>  	return 0;
>>>>  }
>>>>  
>>>> +#ifdef ENABLE_HOTPLUG
>>>> +static void
>>>> +pci_unmap_device(struct rte_pci_device *dev)
>>>> +{
>>>> +	if (dev == NULL)
>>>> +		return;
>>>> +
>>>> +	/* try unmapping the NIC resources using VFIO if it exists */
>>>> +#ifdef VFIO_PRESENT
>>>> +	if (pci_vfio_is_enabled()) {
>>>> +		RTE_LOG(ERR, EAL, "%s() doesn't support vfio yet.\n",
>>>> +				__func__);
>>>> +		return;
>>>> +	}
>>>> +#endif
>>>> +	pci_uio_unmap_resource(dev);
>>>> +}
>>>> +#endif /* ENABLE_HOTPLUG */
>>>> +
>>>>  /*
>>>>   * If vendor/device ID match, call the devinit() function of the
>>>>   * driver.
>>>> diff --git a/lib/librte_eal/linuxapp/eal/eal_pci_init.h b/lib/librte_eal/linuxapp/eal/eal_pci_init.h
>>>> index 1070eb8..5152a0b 100644
>>>> --- a/lib/librte_eal/linuxapp/eal/eal_pci_init.h
>>>> +++ b/lib/librte_eal/linuxapp/eal/eal_pci_init.h
>>>> @@ -34,6 +34,7 @@
>>>>  #ifndef EAL_PCI_INIT_H_
>>>>  #define EAL_PCI_INIT_H_
>>>>  
>>>> +#include <rte_dev_hotplug.h>
>>>>  #include "eal_vfio.h"
>>>>  
>>>>  struct pci_map {
>>>> @@ -71,6 +72,13 @@ void *pci_map_resource(void *requested_addr, int fd, off_t offset,
>>>>  /* map IGB_UIO resource prototype */
>>>>  int pci_uio_map_resource(struct rte_pci_device *dev);
>>>>  
>>>> +#ifdef ENABLE_HOTPLUG
>>>> +void pci_unmap_resource(void *requested_addr, size_t size);
>>>> +
>>>> +/* unmap IGB_UIO resource prototype */
>>>> +void pci_uio_unmap_resource(struct rte_pci_device *dev);
>>>> +#endif /* ENABLE_HOTPLUG */
>>>> +
>>>>  #ifdef VFIO_PRESENT
>>>>  
>>>>  #define VFIO_MAX_GROUPS 64
>>>> diff --git a/lib/librte_eal/linuxapp/eal/eal_pci_uio.c b/lib/librte_eal/linuxapp/eal/eal_pci_uio.c
>>>> index 1da3507..81830d1 100644
>>>> --- a/lib/librte_eal/linuxapp/eal/eal_pci_uio.c
>>>> +++ b/lib/librte_eal/linuxapp/eal/eal_pci_uio.c
>>>> @@ -404,6 +404,71 @@ pci_uio_map_resource(struct rte_pci_device *dev)
>>>>  	return 0;
>>>>  }
>>>>  
>>>> +#ifdef ENABLE_HOTPLUG
>>>> +static void
>>>> +pci_uio_unmap(struct mapped_pci_resource *uio_res)
>>>> +{
>>>> +	int i;
>>>> +
>>>> +	if (uio_res == NULL)
>>>> +		return;
>>>> +
>>>> +	for (i = 0; i != uio_res->nb_maps; i++)
>>>> +		pci_unmap_resource(uio_res->maps[i].addr,
>>>> +				(size_t)uio_res->maps[i].size);
>>>> +}
>>>> +
>>>> +static struct mapped_pci_resource *
>>>> +pci_uio_find_resource(struct rte_pci_device *dev)
>>>> +{
>>>> +	struct mapped_pci_resource *uio_res;
>>>> +
>>>> +	if (dev == NULL)
>>>> +		return NULL;
>>>> +
>>>> +	TAILQ_FOREACH(uio_res, pci_res_list, next) {
>>>> +
>>>> +		/* skip this element if it doesn't match our PCI address */
>>>> +		if (!eal_compare_pci_addr(&uio_res->pci_addr, &dev->addr))
>>>> +			return uio_res;
>>>> +	}
>>>> +	return NULL;
>>>> +}
>>>> +
>>>> +/* unmap the PCI resource of a PCI device in virtual memory */
>>>> +void
>>>> +pci_uio_unmap_resource(struct rte_pci_device *dev)
>>>> +{
>>>> +	struct mapped_pci_resource *uio_res;
>>>> +
>>>> +	if (dev == NULL)
>>>> +		return;
>>>> +
>>>> +	/* find an entry for the device */
>>>> +	uio_res = pci_uio_find_resource(dev);
>>>> +	if (uio_res == NULL)
>>>> +		return;
>>>> +
>>>> +	/* secondary processes - just free maps */
>>>> +	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
>>>> +		return pci_uio_unmap(uio_res);
>>>> +
>>>> +	TAILQ_REMOVE(pci_res_list, uio_res, next);
>>>> +
>>>> +	/* unmap all resources */
>>>> +	pci_uio_unmap(uio_res);
>>>> +
>>>> +	/* free uio resource */
>>>> +	rte_free(uio_res);
>>>> +
>>>> +	/* close fd if in primary process */
>>>> +	close(dev->intr_handle.fd);
>>>> +
>>>> +	dev->intr_handle.fd = -1;
>>>> +	dev->intr_handle.type = RTE_INTR_HANDLE_UNKNOWN;
>>>> +}
>>>> +#endif /* ENABLE_HOTPLUG */
>>>> +
>>>>  /*
>>>>   * parse a sysfs file containing one integer value
>>>>   * different to the eal version, as it needs to work with 64-bit values
>>

^ permalink raw reply	[flat|nested] 362+ messages in thread

* Re: [dpdk-dev] [PATCH v4 06/11] eal/linux/pci: Add functions for unmapping igb_uio resources
  2015-01-22 10:15           ` Tetsuya Mukawa
@ 2015-01-23  2:54             ` Qiu, Michael
  2015-01-23  3:20               ` Tetsuya Mukawa
  0 siblings, 1 reply; 362+ messages in thread
From: Qiu, Michael @ 2015-01-23  2:54 UTC (permalink / raw)
  To: Tetsuya Mukawa; +Cc: dev

On 1/22/2015 6:16 PM, Tetsuya Mukawa wrote:
> Hi Michael,
>
> On 2015/01/22 17:12, Qiu, Michael wrote:
>> On 1/21/2015 6:01 PM, Tetsuya Mukawa wrote:
>>> Hi Michael,
>>>
>>> On 2015/01/20 18:23, Qiu, Michael wrote:
>>>> On 1/19/2015 6:42 PM, Tetsuya Mukawa wrote:
>>>>> The patch adds functions for unmapping igb_uio resources. The patch is only
>>>>> for Linux and igb_uio environment. VFIO and BSD are not supported.
>>>>>
>>>>> v4:
>>>>> - Add paramerter checking.
>>>>> - Add header file to determine if hotplug can be enabled.
>>>>>
>>>>> Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp>
>>>>> ---
>>>>>  lib/librte_eal/common/Makefile                  |  1 +
>>>>>  lib/librte_eal/common/include/rte_dev_hotplug.h | 44 +++++++++++++++++
>>>>>  lib/librte_eal/linuxapp/eal/eal_pci.c           | 38 +++++++++++++++
>>>>>  lib/librte_eal/linuxapp/eal/eal_pci_init.h      |  8 +++
>>>>>  lib/librte_eal/linuxapp/eal/eal_pci_uio.c       | 65 +++++++++++++++++++++++++
>>>>>  5 files changed, 156 insertions(+)
>>>>>  create mode 100644 lib/librte_eal/common/include/rte_dev_hotplug.h
>>>>>
>>>>> diff --git a/lib/librte_eal/common/Makefile b/lib/librte_eal/common/Makefile
>>>>> index 52c1a5f..db7cc93 100644
>>>>> --- a/lib/librte_eal/common/Makefile
>>>>> +++ b/lib/librte_eal/common/Makefile
>>>>> @@ -41,6 +41,7 @@ INC += rte_eal_memconfig.h rte_malloc_heap.h
>>>>>  INC += rte_hexdump.h rte_devargs.h rte_dev.h
>>>>>  INC += rte_common_vect.h
>>>>>  INC += rte_pci_dev_feature_defs.h rte_pci_dev_features.h
>>>>> +INC += rte_dev_hotplug.h
>>>>>  
>>>>>  ifeq ($(CONFIG_RTE_INSECURE_FUNCTION_WARNING),y)
>>>>>  INC += rte_warnings.h
>>>>> diff --git a/lib/librte_eal/common/include/rte_dev_hotplug.h b/lib/librte_eal/common/include/rte_dev_hotplug.h
>>>>> new file mode 100644
>>>>> index 0000000..b333e0f
>>>>> --- /dev/null
>>>>> +++ b/lib/librte_eal/common/include/rte_dev_hotplug.h
>>>>> @@ -0,0 +1,44 @@
>>>>> +/*-
>>>>> + *   BSD LICENSE
>>>>> + *
>>>>> + *   Copyright(c) 2015 IGEL Co.,LTd.
>>>>> + *   All rights reserved.
>>>>> + *
>>>>> + *   Redistribution and use in source and binary forms, with or without
>>>>> + *   modification, are permitted provided that the following conditions
>>>>> + *   are met:
>>>>> + *
>>>>> + *     * Redistributions of source code must retain the above copyright
>>>>> + *       notice, this list of conditions and the following disclaimer.
>>>>> + *     * Redistributions in binary form must reproduce the above copyright
>>>>> + *       notice, this list of conditions and the following disclaimer in
>>>>> + *       the documentation and/or other materials provided with the
>>>>> + *       distribution.
>>>>> + *     * Neither the name of IGEL Co.,Ltd. nor the names of its
>>>>> + *       contributors may be used to endorse or promote products derived
>>>>> + *       from this software without specific prior written permission.
>>>>> + *
>>>>> + *   THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
>>>>> + *   "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
>>>>> + *   LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
>>>>> + *   A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
>>>>> + *   OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
>>>>> + *   SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
>>>>> + *   LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
>>>>> + *   DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
>>>>> + *   THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
>>>>> + *   (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
>>>>> + *   OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
>>>>> + */
>>>>> +
>>>>> +#ifndef _RTE_DEV_HOTPLUG_H_
>>>>> +#define _RTE_DEV_HOTPLUG_H_
>>>>> +
>>>>> +/*
>>>>> + * determine if hotplug can be enabled on the system
>>>>> + */
>>>>> +#if defined(RTE_LIBRTE_EAL_HOTPLUG) && defined(RTE_LIBRTE_EAL_LINUXAPP)
>>>> As you said, VFIO should not work with it, so does it need to add the
>>>> vfio check here?
>>> Could I have a advice of you?
>>> First I guess it's the best to include "eal_vfio.h" here, and add
>>> checking of VFIO_PRESENT macro.
>> I have a question, will your hotplug  feature support freebsd ?
>>
>> If not, how about to put it in  "lib/librte_eal/linuxapp/eal/" ? Also 
>> include attach or detach affairs.
> I appreciate your comments.
>
> So far, FreeBSD doesn't support PCI hotplug. So I didn't implement code
> for FreeBSD.
> But in the future, I want to implement it when FreeBSD supports it.
> Also my hotplug implementation depends on legacy code already
> implemented in common layer.
> Anyway, for me it's nice to implement the feature in common layer.
>
>>> But it seems I cannot reach "eal_vfio.h" from this file.
>> Yes, you can't :)
>>
>>> My second option is just checking RTE_EAL_VFIO macro.
>>> But according to "eal_vfio.h", if kernel is under 3.6.0, VFIO_PRESENT
>> Actually,  in my opinion, whatever vfio or uio, only need be care in
>> runtime.
>>
>> DPDK to check vfio only to add support  for vfio, but this does not
>> means the device will use vfio,
>>
>> So even if VFIO_PRESENT is defined, and vfio is enabled, but the device
>> is bind to igb_uio, then your hotplug still  need work, but if it bind
>> to vfio, will not, am I right?
>>
>> If yes, I'm not sure if your hotplug has this ability, but it is
>> reasonable, I think.
> I agree with your concept. But I guess it's not only related with
> hotplug function.
> The hotplug implementation depends on legacy functions that is for
> probing device.
> To implement above concept will change not only hotplug behavior but
> also legacy device probing.
>
> Conceptually I agree with such a functionality, but legacy probing
> function doesn't have such a feature so far.
> So I guess it's nice to separate this feature from hotplug patches.
> Realistically, the hotplug patches are big, and it's a bit hard to add
> and manage one more feature.
> If it's ok to separate the patch, it's helpful to manage patches.
>
>>> will not be defined even when RTE_EAL_VFIO is enabled.
>>> So I guess simply macro checking will not work correctly.
>>>  
>>> Anyway, here are my implementation choices so far.
>>>
>>> 1. Like "eal_vfio.h", check kernel version in "rte_dev_hotplug.h".
>>> In this case, if "eal_vfio.h" is changed, "rte_edv_hotplug.h" may need
>>> to be changed also.
>>>
>>> 2. Merge "eal_vfio.h" and "rte_dev_hotplug.h" definitions, and define
>>> these in new rte header like "rte_settings.h".
>>>
>>> Can I have advice about it?
>> As I said, VFIO enable or not is not related for your hotplug, only the
>> devices managed by VFIO will affect your hotplug.
>>
>> If you agree, I think need discuss the details of it.
> Yes, I agree with your concept.
> And if you agree, I will implement it separately.
>
> To discuss how to handle VFIO and igb_uio devices in parallel, I guess
> we need to
> think about generic uio driver for pci devices.
> I guess before applying uio generic patch, this kind of discussion will
> be needed.
> I hope igb_uio (and VFIO?) be obsolete after applying uio generic.

No, igb_uio is similar to uio generic, but VFIO is another totally
different way of UIO.

So applying uio generic has no affect for VFIO

> In the case, we don't need to think it.
>
> BTW, back to 'rte_dev_hotplug.h' discussion of hotplug patch.
> If VFIO_PRESENT isn't checked here, attaching port will be successful
> even if the device is under VFIO.
> (Because we already has legacy code for probing VFIO device. The hotplug
> function just re-use it.)
> But we cannot detach the device, because there is no code for closing
> VFIO device.
> There is no crash when the VFIO device is detached, but some error
> messages will be displayed.
> So I guess this behavior is like your description.
> How about it?

Actually, what I'm considering is to add one flag like "int pt_mod" in
"struct rte_pci_device", and give the value in pci scan stage, then it
will reduce lots of work in EAL of VFIO checking, like PCI memory
mapping stage, hotplug and so on.
For hotplug, it can check the flag at runtime. And when hotplug supports
VFIO, you can simply ignore this flag.

Do you think it is valuable to do so?

If yes I think I will make a patch for that, and send to you, you can
post your patch set with that, or if you want implement it yourself, let
me know.

Thanks,
Michael
> Thanks,
> Tetsuya
>
>> Thanks,
>> Michael
>>> Thanks,
>>> Tetsuya
>>>
>>>> Thanks,
>>>> Michael
>>>>> +#define ENABLE_HOTPLUG
>>>>> +#endif /* RTE_LIBRTE_EAL_HOTPLUG & RTE_LIBRTE_EAL_LINUXAPP */
>>>>> +
>>>>> +#endif /* _RTE_DEV_HOTPLUG_H_ */
>>>>> diff --git a/lib/librte_eal/linuxapp/eal/eal_pci.c b/lib/librte_eal/linuxapp/eal/eal_pci.c
>>>>> index 3d2d93c..52c464c 100644
>>>>> --- a/lib/librte_eal/linuxapp/eal/eal_pci.c
>>>>> +++ b/lib/librte_eal/linuxapp/eal/eal_pci.c
>>>>> @@ -137,6 +137,25 @@ pci_map_resource(void *requested_addr, int fd, off_t offset, size_t size)
>>>>>  	return mapaddr;
>>>>>  }
>>>>>  
>>>>> +#ifdef ENABLE_HOTPLUG
>>>>> +/* unmap a particular resource */
>>>>> +void
>>>>> +pci_unmap_resource(void *requested_addr, size_t size)
>>>>> +{
>>>>> +	if (requested_addr == NULL)
>>>>> +		return;
>>>>> +
>>>>> +	/* Unmap the PCI memory resource of device */
>>>>> +	if (munmap(requested_addr, size)) {
>>>>> +		RTE_LOG(ERR, EAL, "%s(): cannot munmap(%p, 0x%lx): %s\n",
>>>>> +			__func__, requested_addr, (unsigned long)size,
>>>>> +			strerror(errno));
>>>>> +	} else
>>>>> +		RTE_LOG(DEBUG, EAL, "  PCI memory mapped at %p\n",
>>>>> +				requested_addr);
>>>>> +}
>>>>> +#endif /* ENABLE_HOTPLUG */
>>>>> +
>>>>>  /* parse the "resource" sysfs file */
>>>>>  #define IORESOURCE_MEM  0x00000200
>>>>>  
>>>>> @@ -510,6 +529,25 @@ pci_map_device(struct rte_pci_device *dev)
>>>>>  	return 0;
>>>>>  }
>>>>>  
>>>>> +#ifdef ENABLE_HOTPLUG
>>>>> +static void
>>>>> +pci_unmap_device(struct rte_pci_device *dev)
>>>>> +{
>>>>> +	if (dev == NULL)
>>>>> +		return;
>>>>> +
>>>>> +	/* try unmapping the NIC resources using VFIO if it exists */
>>>>> +#ifdef VFIO_PRESENT
>>>>> +	if (pci_vfio_is_enabled()) {
>>>>> +		RTE_LOG(ERR, EAL, "%s() doesn't support vfio yet.\n",
>>>>> +				__func__);
>>>>> +		return;
>>>>> +	}
>>>>> +#endif
>>>>> +	pci_uio_unmap_resource(dev);
>>>>> +}
>>>>> +#endif /* ENABLE_HOTPLUG */
>>>>> +
>>>>>  /*
>>>>>   * If vendor/device ID match, call the devinit() function of the
>>>>>   * driver.
>>>>> diff --git a/lib/librte_eal/linuxapp/eal/eal_pci_init.h b/lib/librte_eal/linuxapp/eal/eal_pci_init.h
>>>>> index 1070eb8..5152a0b 100644
>>>>> --- a/lib/librte_eal/linuxapp/eal/eal_pci_init.h
>>>>> +++ b/lib/librte_eal/linuxapp/eal/eal_pci_init.h
>>>>> @@ -34,6 +34,7 @@
>>>>>  #ifndef EAL_PCI_INIT_H_
>>>>>  #define EAL_PCI_INIT_H_
>>>>>  
>>>>> +#include <rte_dev_hotplug.h>
>>>>>  #include "eal_vfio.h"
>>>>>  
>>>>>  struct pci_map {
>>>>> @@ -71,6 +72,13 @@ void *pci_map_resource(void *requested_addr, int fd, off_t offset,
>>>>>  /* map IGB_UIO resource prototype */
>>>>>  int pci_uio_map_resource(struct rte_pci_device *dev);
>>>>>  
>>>>> +#ifdef ENABLE_HOTPLUG
>>>>> +void pci_unmap_resource(void *requested_addr, size_t size);
>>>>> +
>>>>> +/* unmap IGB_UIO resource prototype */
>>>>> +void pci_uio_unmap_resource(struct rte_pci_device *dev);
>>>>> +#endif /* ENABLE_HOTPLUG */
>>>>> +
>>>>>  #ifdef VFIO_PRESENT
>>>>>  
>>>>>  #define VFIO_MAX_GROUPS 64
>>>>> diff --git a/lib/librte_eal/linuxapp/eal/eal_pci_uio.c b/lib/librte_eal/linuxapp/eal/eal_pci_uio.c
>>>>> index 1da3507..81830d1 100644
>>>>> --- a/lib/librte_eal/linuxapp/eal/eal_pci_uio.c
>>>>> +++ b/lib/librte_eal/linuxapp/eal/eal_pci_uio.c
>>>>> @@ -404,6 +404,71 @@ pci_uio_map_resource(struct rte_pci_device *dev)
>>>>>  	return 0;
>>>>>  }
>>>>>  
>>>>> +#ifdef ENABLE_HOTPLUG
>>>>> +static void
>>>>> +pci_uio_unmap(struct mapped_pci_resource *uio_res)
>>>>> +{
>>>>> +	int i;
>>>>> +
>>>>> +	if (uio_res == NULL)
>>>>> +		return;
>>>>> +
>>>>> +	for (i = 0; i != uio_res->nb_maps; i++)
>>>>> +		pci_unmap_resource(uio_res->maps[i].addr,
>>>>> +				(size_t)uio_res->maps[i].size);
>>>>> +}
>>>>> +
>>>>> +static struct mapped_pci_resource *
>>>>> +pci_uio_find_resource(struct rte_pci_device *dev)
>>>>> +{
>>>>> +	struct mapped_pci_resource *uio_res;
>>>>> +
>>>>> +	if (dev == NULL)
>>>>> +		return NULL;
>>>>> +
>>>>> +	TAILQ_FOREACH(uio_res, pci_res_list, next) {
>>>>> +
>>>>> +		/* skip this element if it doesn't match our PCI address */
>>>>> +		if (!eal_compare_pci_addr(&uio_res->pci_addr, &dev->addr))
>>>>> +			return uio_res;
>>>>> +	}
>>>>> +	return NULL;
>>>>> +}
>>>>> +
>>>>> +/* unmap the PCI resource of a PCI device in virtual memory */
>>>>> +void
>>>>> +pci_uio_unmap_resource(struct rte_pci_device *dev)
>>>>> +{
>>>>> +	struct mapped_pci_resource *uio_res;
>>>>> +
>>>>> +	if (dev == NULL)
>>>>> +		return;
>>>>> +
>>>>> +	/* find an entry for the device */
>>>>> +	uio_res = pci_uio_find_resource(dev);
>>>>> +	if (uio_res == NULL)
>>>>> +		return;
>>>>> +
>>>>> +	/* secondary processes - just free maps */
>>>>> +	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
>>>>> +		return pci_uio_unmap(uio_res);
>>>>> +
>>>>> +	TAILQ_REMOVE(pci_res_list, uio_res, next);
>>>>> +
>>>>> +	/* unmap all resources */
>>>>> +	pci_uio_unmap(uio_res);
>>>>> +
>>>>> +	/* free uio resource */
>>>>> +	rte_free(uio_res);
>>>>> +
>>>>> +	/* close fd if in primary process */
>>>>> +	close(dev->intr_handle.fd);
>>>>> +
>>>>> +	dev->intr_handle.fd = -1;
>>>>> +	dev->intr_handle.type = RTE_INTR_HANDLE_UNKNOWN;
>>>>> +}
>>>>> +#endif /* ENABLE_HOTPLUG */
>>>>> +
>>>>>  /*
>>>>>   * parse a sysfs file containing one integer value
>>>>>   * different to the eal version, as it needs to work with 64-bit values
>
>


^ permalink raw reply	[flat|nested] 362+ messages in thread

* Re: [dpdk-dev] [PATCH v4 06/11] eal/linux/pci: Add functions for unmapping igb_uio resources
  2015-01-23  2:54             ` Qiu, Michael
@ 2015-01-23  3:20               ` Tetsuya Mukawa
  0 siblings, 0 replies; 362+ messages in thread
From: Tetsuya Mukawa @ 2015-01-23  3:20 UTC (permalink / raw)
  To: Qiu, Michael; +Cc: dev

Hi Michael,

On 2015/01/23 11:54, Qiu, Michael wrote:
> On 1/22/2015 6:16 PM, Tetsuya Mukawa wrote:
>> Hi Michael,
>>
>> On 2015/01/22 17:12, Qiu, Michael wrote:
>>> On 1/21/2015 6:01 PM, Tetsuya Mukawa wrote:
>>>> Hi Michael,
>>>>
>>>> On 2015/01/20 18:23, Qiu, Michael wrote:
>>>>> On 1/19/2015 6:42 PM, Tetsuya Mukawa wrote:
>>>>>> The patch adds functions for unmapping igb_uio resources. The patch is only
>>>>>> for Linux and igb_uio environment. VFIO and BSD are not supported.
>>>>>>
>>>>>> v4:
>>>>>> - Add paramerter checking.
>>>>>> - Add header file to determine if hotplug can be enabled.
>>>>>>
>>>>>> Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp>
>>>>>> ---
>>>>>>  lib/librte_eal/common/Makefile                  |  1 +
>>>>>>  lib/librte_eal/common/include/rte_dev_hotplug.h | 44 +++++++++++++++++
>>>>>>  lib/librte_eal/linuxapp/eal/eal_pci.c           | 38 +++++++++++++++
>>>>>>  lib/librte_eal/linuxapp/eal/eal_pci_init.h      |  8 +++
>>>>>>  lib/librte_eal/linuxapp/eal/eal_pci_uio.c       | 65 +++++++++++++++++++++++++
>>>>>>  5 files changed, 156 insertions(+)
>>>>>>  create mode 100644 lib/librte_eal/common/include/rte_dev_hotplug.h
>>>>>>
>>>>>> diff --git a/lib/librte_eal/common/Makefile b/lib/librte_eal/common/Makefile
>>>>>> index 52c1a5f..db7cc93 100644
>>>>>> --- a/lib/librte_eal/common/Makefile
>>>>>> +++ b/lib/librte_eal/common/Makefile
>>>>>> @@ -41,6 +41,7 @@ INC += rte_eal_memconfig.h rte_malloc_heap.h
>>>>>>  INC += rte_hexdump.h rte_devargs.h rte_dev.h
>>>>>>  INC += rte_common_vect.h
>>>>>>  INC += rte_pci_dev_feature_defs.h rte_pci_dev_features.h
>>>>>> +INC += rte_dev_hotplug.h
>>>>>>  
>>>>>>  ifeq ($(CONFIG_RTE_INSECURE_FUNCTION_WARNING),y)
>>>>>>  INC += rte_warnings.h
>>>>>> diff --git a/lib/librte_eal/common/include/rte_dev_hotplug.h b/lib/librte_eal/common/include/rte_dev_hotplug.h
>>>>>> new file mode 100644
>>>>>> index 0000000..b333e0f
>>>>>> --- /dev/null
>>>>>> +++ b/lib/librte_eal/common/include/rte_dev_hotplug.h
>>>>>> @@ -0,0 +1,44 @@
>>>>>> +/*-
>>>>>> + *   BSD LICENSE
>>>>>> + *
>>>>>> + *   Copyright(c) 2015 IGEL Co.,LTd.
>>>>>> + *   All rights reserved.
>>>>>> + *
>>>>>> + *   Redistribution and use in source and binary forms, with or without
>>>>>> + *   modification, are permitted provided that the following conditions
>>>>>> + *   are met:
>>>>>> + *
>>>>>> + *     * Redistributions of source code must retain the above copyright
>>>>>> + *       notice, this list of conditions and the following disclaimer.
>>>>>> + *     * Redistributions in binary form must reproduce the above copyright
>>>>>> + *       notice, this list of conditions and the following disclaimer in
>>>>>> + *       the documentation and/or other materials provided with the
>>>>>> + *       distribution.
>>>>>> + *     * Neither the name of IGEL Co.,Ltd. nor the names of its
>>>>>> + *       contributors may be used to endorse or promote products derived
>>>>>> + *       from this software without specific prior written permission.
>>>>>> + *
>>>>>> + *   THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
>>>>>> + *   "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
>>>>>> + *   LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
>>>>>> + *   A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
>>>>>> + *   OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
>>>>>> + *   SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
>>>>>> + *   LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
>>>>>> + *   DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
>>>>>> + *   THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
>>>>>> + *   (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
>>>>>> + *   OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
>>>>>> + */
>>>>>> +
>>>>>> +#ifndef _RTE_DEV_HOTPLUG_H_
>>>>>> +#define _RTE_DEV_HOTPLUG_H_
>>>>>> +
>>>>>> +/*
>>>>>> + * determine if hotplug can be enabled on the system
>>>>>> + */
>>>>>> +#if defined(RTE_LIBRTE_EAL_HOTPLUG) && defined(RTE_LIBRTE_EAL_LINUXAPP)
>>>>> As you said, VFIO should not work with it, so does it need to add the
>>>>> vfio check here?
>>>> Could I have a advice of you?
>>>> First I guess it's the best to include "eal_vfio.h" here, and add
>>>> checking of VFIO_PRESENT macro.
>>> I have a question, will your hotplug  feature support freebsd ?
>>>
>>> If not, how about to put it in  "lib/librte_eal/linuxapp/eal/" ? Also 
>>> include attach or detach affairs.
>> I appreciate your comments.
>>
>> So far, FreeBSD doesn't support PCI hotplug. So I didn't implement code
>> for FreeBSD.
>> But in the future, I want to implement it when FreeBSD supports it.
>> Also my hotplug implementation depends on legacy code already
>> implemented in common layer.
>> Anyway, for me it's nice to implement the feature in common layer.
>>
>>>> But it seems I cannot reach "eal_vfio.h" from this file.
>>> Yes, you can't :)
>>>
>>>> My second option is just checking RTE_EAL_VFIO macro.
>>>> But according to "eal_vfio.h", if kernel is under 3.6.0, VFIO_PRESENT
>>> Actually,  in my opinion, whatever vfio or uio, only need be care in
>>> runtime.
>>>
>>> DPDK to check vfio only to add support  for vfio, but this does not
>>> means the device will use vfio,
>>>
>>> So even if VFIO_PRESENT is defined, and vfio is enabled, but the device
>>> is bind to igb_uio, then your hotplug still  need work, but if it bind
>>> to vfio, will not, am I right?
>>>
>>> If yes, I'm not sure if your hotplug has this ability, but it is
>>> reasonable, I think.
>> I agree with your concept. But I guess it's not only related with
>> hotplug function.
>> The hotplug implementation depends on legacy functions that is for
>> probing device.
>> To implement above concept will change not only hotplug behavior but
>> also legacy device probing.
>>
>> Conceptually I agree with such a functionality, but legacy probing
>> function doesn't have such a feature so far.
>> So I guess it's nice to separate this feature from hotplug patches.
>> Realistically, the hotplug patches are big, and it's a bit hard to add
>> and manage one more feature.
>> If it's ok to separate the patch, it's helpful to manage patches.
>>
>>>> will not be defined even when RTE_EAL_VFIO is enabled.
>>>> So I guess simply macro checking will not work correctly.
>>>>  
>>>> Anyway, here are my implementation choices so far.
>>>>
>>>> 1. Like "eal_vfio.h", check kernel version in "rte_dev_hotplug.h".
>>>> In this case, if "eal_vfio.h" is changed, "rte_edv_hotplug.h" may need
>>>> to be changed also.
>>>>
>>>> 2. Merge "eal_vfio.h" and "rte_dev_hotplug.h" definitions, and define
>>>> these in new rte header like "rte_settings.h".
>>>>
>>>> Can I have advice about it?
>>> As I said, VFIO enable or not is not related for your hotplug, only the
>>> devices managed by VFIO will affect your hotplug.
>>>
>>> If you agree, I think need discuss the details of it.
>> Yes, I agree with your concept.
>> And if you agree, I will implement it separately.
>>
>> To discuss how to handle VFIO and igb_uio devices in parallel, I guess
>> we need to
>> think about generic uio driver for pci devices.
>> I guess before applying uio generic patch, this kind of discussion will
>> be needed.
>> I hope igb_uio (and VFIO?) be obsolete after applying uio generic.
> No, igb_uio is similar to uio generic, but VFIO is another totally
> different way of UIO.
>
> So applying uio generic has no affect for VFIO

I appreciate your comments.
OK, I understand the relation of VFIO and uio generic.

>
>> In the case, we don't need to think it.
>>
>> BTW, back to 'rte_dev_hotplug.h' discussion of hotplug patch.
>> If VFIO_PRESENT isn't checked here, attaching port will be successful
>> even if the device is under VFIO.
>> (Because we already has legacy code for probing VFIO device. The hotplug
>> function just re-use it.)
>> But we cannot detach the device, because there is no code for closing
>> VFIO device.
>> There is no crash when the VFIO device is detached, but some error
>> messages will be displayed.
>> So I guess this behavior is like your description.
>> How about it?
> Actually, what I'm considering is to add one flag like "int pt_mod" in
> "struct rte_pci_device", and give the value in pci scan stage, then it
> will reduce lots of work in EAL of VFIO checking, like PCI memory
> mapping stage, hotplug and so on.
> For hotplug, it can check the flag at runtime. And when hotplug supports
> VFIO, you can simply ignore this flag.
>
> Do you think it is valuable to do so?
>
> If yes I think I will make a patch for that, and send to you, you can
> post your patch set with that, or if you want implement it yourself, let
> me know.

I agree with you. And thank you so much.
Could you please send a patch? I will put it on the top of my patches.
(If you have favorite where you want to put it on, could you please let
me know?)

Thanks,
Tetsuya

> Thanks,
> Michael
>> Thanks,
>> Tetsuya
>>
>>> Thanks,
>>> Michael
>>>> Thanks,
>>>> Tetsuya
>>>>
>>>>> Thanks,
>>>>> Michael
>>>>>> +#define ENABLE_HOTPLUG
>>>>>> +#endif /* RTE_LIBRTE_EAL_HOTPLUG & RTE_LIBRTE_EAL_LINUXAPP */
>>>>>> +
>>>>>> +#endif /* _RTE_DEV_HOTPLUG_H_ */
>>>>>> diff --git a/lib/librte_eal/linuxapp/eal/eal_pci.c b/lib/librte_eal/linuxapp/eal/eal_pci.c
>>>>>> index 3d2d93c..52c464c 100644
>>>>>> --- a/lib/librte_eal/linuxapp/eal/eal_pci.c
>>>>>> +++ b/lib/librte_eal/linuxapp/eal/eal_pci.c
>>>>>> @@ -137,6 +137,25 @@ pci_map_resource(void *requested_addr, int fd, off_t offset, size_t size)
>>>>>>  	return mapaddr;
>>>>>>  }
>>>>>>  
>>>>>> +#ifdef ENABLE_HOTPLUG
>>>>>> +/* unmap a particular resource */
>>>>>> +void
>>>>>> +pci_unmap_resource(void *requested_addr, size_t size)
>>>>>> +{
>>>>>> +	if (requested_addr == NULL)
>>>>>> +		return;
>>>>>> +
>>>>>> +	/* Unmap the PCI memory resource of device */
>>>>>> +	if (munmap(requested_addr, size)) {
>>>>>> +		RTE_LOG(ERR, EAL, "%s(): cannot munmap(%p, 0x%lx): %s\n",
>>>>>> +			__func__, requested_addr, (unsigned long)size,
>>>>>> +			strerror(errno));
>>>>>> +	} else
>>>>>> +		RTE_LOG(DEBUG, EAL, "  PCI memory mapped at %p\n",
>>>>>> +				requested_addr);
>>>>>> +}
>>>>>> +#endif /* ENABLE_HOTPLUG */
>>>>>> +
>>>>>>  /* parse the "resource" sysfs file */
>>>>>>  #define IORESOURCE_MEM  0x00000200
>>>>>>  
>>>>>> @@ -510,6 +529,25 @@ pci_map_device(struct rte_pci_device *dev)
>>>>>>  	return 0;
>>>>>>  }
>>>>>>  
>>>>>> +#ifdef ENABLE_HOTPLUG
>>>>>> +static void
>>>>>> +pci_unmap_device(struct rte_pci_device *dev)
>>>>>> +{
>>>>>> +	if (dev == NULL)
>>>>>> +		return;
>>>>>> +
>>>>>> +	/* try unmapping the NIC resources using VFIO if it exists */
>>>>>> +#ifdef VFIO_PRESENT
>>>>>> +	if (pci_vfio_is_enabled()) {
>>>>>> +		RTE_LOG(ERR, EAL, "%s() doesn't support vfio yet.\n",
>>>>>> +				__func__);
>>>>>> +		return;
>>>>>> +	}
>>>>>> +#endif
>>>>>> +	pci_uio_unmap_resource(dev);
>>>>>> +}
>>>>>> +#endif /* ENABLE_HOTPLUG */
>>>>>> +
>>>>>>  /*
>>>>>>   * If vendor/device ID match, call the devinit() function of the
>>>>>>   * driver.
>>>>>> diff --git a/lib/librte_eal/linuxapp/eal/eal_pci_init.h b/lib/librte_eal/linuxapp/eal/eal_pci_init.h
>>>>>> index 1070eb8..5152a0b 100644
>>>>>> --- a/lib/librte_eal/linuxapp/eal/eal_pci_init.h
>>>>>> +++ b/lib/librte_eal/linuxapp/eal/eal_pci_init.h
>>>>>> @@ -34,6 +34,7 @@
>>>>>>  #ifndef EAL_PCI_INIT_H_
>>>>>>  #define EAL_PCI_INIT_H_
>>>>>>  
>>>>>> +#include <rte_dev_hotplug.h>
>>>>>>  #include "eal_vfio.h"
>>>>>>  
>>>>>>  struct pci_map {
>>>>>> @@ -71,6 +72,13 @@ void *pci_map_resource(void *requested_addr, int fd, off_t offset,
>>>>>>  /* map IGB_UIO resource prototype */
>>>>>>  int pci_uio_map_resource(struct rte_pci_device *dev);
>>>>>>  
>>>>>> +#ifdef ENABLE_HOTPLUG
>>>>>> +void pci_unmap_resource(void *requested_addr, size_t size);
>>>>>> +
>>>>>> +/* unmap IGB_UIO resource prototype */
>>>>>> +void pci_uio_unmap_resource(struct rte_pci_device *dev);
>>>>>> +#endif /* ENABLE_HOTPLUG */
>>>>>> +
>>>>>>  #ifdef VFIO_PRESENT
>>>>>>  
>>>>>>  #define VFIO_MAX_GROUPS 64
>>>>>> diff --git a/lib/librte_eal/linuxapp/eal/eal_pci_uio.c b/lib/librte_eal/linuxapp/eal/eal_pci_uio.c
>>>>>> index 1da3507..81830d1 100644
>>>>>> --- a/lib/librte_eal/linuxapp/eal/eal_pci_uio.c
>>>>>> +++ b/lib/librte_eal/linuxapp/eal/eal_pci_uio.c
>>>>>> @@ -404,6 +404,71 @@ pci_uio_map_resource(struct rte_pci_device *dev)
>>>>>>  	return 0;
>>>>>>  }
>>>>>>  
>>>>>> +#ifdef ENABLE_HOTPLUG
>>>>>> +static void
>>>>>> +pci_uio_unmap(struct mapped_pci_resource *uio_res)
>>>>>> +{
>>>>>> +	int i;
>>>>>> +
>>>>>> +	if (uio_res == NULL)
>>>>>> +		return;
>>>>>> +
>>>>>> +	for (i = 0; i != uio_res->nb_maps; i++)
>>>>>> +		pci_unmap_resource(uio_res->maps[i].addr,
>>>>>> +				(size_t)uio_res->maps[i].size);
>>>>>> +}
>>>>>> +
>>>>>> +static struct mapped_pci_resource *
>>>>>> +pci_uio_find_resource(struct rte_pci_device *dev)
>>>>>> +{
>>>>>> +	struct mapped_pci_resource *uio_res;
>>>>>> +
>>>>>> +	if (dev == NULL)
>>>>>> +		return NULL;
>>>>>> +
>>>>>> +	TAILQ_FOREACH(uio_res, pci_res_list, next) {
>>>>>> +
>>>>>> +		/* skip this element if it doesn't match our PCI address */
>>>>>> +		if (!eal_compare_pci_addr(&uio_res->pci_addr, &dev->addr))
>>>>>> +			return uio_res;
>>>>>> +	}
>>>>>> +	return NULL;
>>>>>> +}
>>>>>> +
>>>>>> +/* unmap the PCI resource of a PCI device in virtual memory */
>>>>>> +void
>>>>>> +pci_uio_unmap_resource(struct rte_pci_device *dev)
>>>>>> +{
>>>>>> +	struct mapped_pci_resource *uio_res;
>>>>>> +
>>>>>> +	if (dev == NULL)
>>>>>> +		return;
>>>>>> +
>>>>>> +	/* find an entry for the device */
>>>>>> +	uio_res = pci_uio_find_resource(dev);
>>>>>> +	if (uio_res == NULL)
>>>>>> +		return;
>>>>>> +
>>>>>> +	/* secondary processes - just free maps */
>>>>>> +	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
>>>>>> +		return pci_uio_unmap(uio_res);
>>>>>> +
>>>>>> +	TAILQ_REMOVE(pci_res_list, uio_res, next);
>>>>>> +
>>>>>> +	/* unmap all resources */
>>>>>> +	pci_uio_unmap(uio_res);
>>>>>> +
>>>>>> +	/* free uio resource */
>>>>>> +	rte_free(uio_res);
>>>>>> +
>>>>>> +	/* close fd if in primary process */
>>>>>> +	close(dev->intr_handle.fd);
>>>>>> +
>>>>>> +	dev->intr_handle.fd = -1;
>>>>>> +	dev->intr_handle.type = RTE_INTR_HANDLE_UNKNOWN;
>>>>>> +}
>>>>>> +#endif /* ENABLE_HOTPLUG */
>>>>>> +
>>>>>>  /*
>>>>>>   * parse a sysfs file containing one integer value
>>>>>>   * different to the eal version, as it needs to work with 64-bit values
>>

^ permalink raw reply	[flat|nested] 362+ messages in thread

* Re: [dpdk-dev] [PATCH v4 00/11] Port Hotplug Framework
  2015-01-19 10:40 ` [dpdk-dev] [PATCH v4 00/11] Port Hotplug Framework Tetsuya Mukawa
                     ` (11 preceding siblings ...)
  2015-01-19 13:15   ` [dpdk-dev] [PATCH v4 00/11] Port Hotplug Framework Qiu, Michael
@ 2015-01-27  3:00   ` Qiu, Michael
  2015-01-27  5:01     ` Tetsuya Mukawa
  12 siblings, 1 reply; 362+ messages in thread
From: Qiu, Michael @ 2015-01-27  3:00 UTC (permalink / raw)
  To: Tetsuya Mukawa, dev

On 1/19/2015 6:42 PM, Tetsuya Mukawa wrote:
> This patch series adds a dynamic port hotplug framework to DPDK.
> With the patches, DPDK apps can attach or detach ports at runtime.
>
> The basic concept of the port hotplug is like followings.
> - DPDK apps must have responsibility to manage ports.
>   DPDK apps only know which ports are attached or detached at the moment.
>   The port hotplug framework is implemented to allow DPDK apps to manage ports.
>   For example, when DPDK apps call port attach function, attached port number
>   will be returned. Also DPDK apps can detach port by port number.
> - Kernel support is needed for attaching or detaching physical device ports.
>   To attach new device, the device will be recognized by kernel at first and
>   controlled by kernel driver. Then user can bind the device to igb_uio

Here does it really need native kernel driver here? As it will be
controlled by igb_uio.
I think even if the device has no kernel driver is also OK.

Also I have finished initial patch of passthrough driver flag in
"struct rte_pci_device"

I will send to you after I do some basic test on that, then I will send to
you, and you can give some comments on that.


Thanks,
Michael

>   by 'dpdk_nic_bind.py'. Finally, DPDK apps can call the port hotplug
>   functions to attach ports.
>   For detaching, steps are vice versa.
> - Before detach ports, ports must be stopped and closed.
>   DPDK application must call rte_eth_dev_stop() and rte_eth_dev_close() before
>   detaching ports. These function will call finalization codes of PMDs.
>   But so far, no PMD frees all resources allocated by initialization.
>   It means PMDs are needed to be fixed to support the port hotplug.
>   'RTE_PCI_DRV_DETACHABLE' is a new flag indicating a PMD supports detaching.
>   Without this flag, detaching will be failed.
> - Mustn't affect legacy DPDK apps.
>   No DPDK EAL behavior is changed, if the port hotplug functions are't called.
>   So all legacy DPDK apps can still work without modifications.
>
> And a few limitations.
> - The port hotplug functions are not thread safe.
>   DPDK apps should handle it.
> - Only support Linux and igb_uio so far.
>   BSD and VFIO is not supported. I will send VFIO patches at least, but I don't
>   have a plan to submit BSD patch so far.
>
>
> Here is port hotplug APIs.
> -------------------------------------------------------------------------------
> /**
>  * Attach a new device.
>  *
>  * @param devargs
>  *   A pointer to a strings array describing the new device
>  *   to be attached. The strings should be a pci address like
>  *   '0000:01:00.0' or virtual device name like 'eth_pcap0'.
>  * @param port_id
>  *  A pointer to a port identifier actually attached.
>  * @return
>  *  0 on success and port_id is filled, negative on error
>  */
> int rte_eal_dev_attach(const char *devargs, uint8_t *port_id);
>
> /**
>  * Detach a device.
>  *
>  * @param port_id
>  *   The port identifier of the device to detach.
>  * @param addr
>  *  A pointer to a device name actually detached.
>  * @return
>  *  0 on success and devname is filled, negative on error
>  */
> int rte_eal_dev_detach(uint8_t port_id, char *devname);
> -------------------------------------------------------------------------------
>
> This patch series are for DPDK EAL. To use port hotplug function by DPDK apps,
> each PMD should be fixed to support 'RTE_PCI_DRV_DETACHABLE' flag. Please check
> a patch for pcap PMD.
>
> Also please check testpmd patch. It will show you how to fix your legacy
> applications to support port hotplug feature.
>
>
> PATCH v4 changes
>  - Merge patches to review easier.
>  - Fix indent of 'if' statement.
>  - Fix calculation method of eal_compare_pci_addr().
>  - Fix header file declaration.
>  - Add header file to determine if hotplug can be enabled.
>    (Thanks to Qiu, Michael)
>  - Use braces with 'for' loop.
>  - Add paramerter checking.
>  - Fix sanity check code
>  - Fix comments of rte_eth_dev_type.
>  - Change function names.
>    (Thanks to Iremonger, Bernard)
>
> PATCH v3 changes:
>  - Fix enum definition used in rte_ethdev.c.
>    (Thanks to Zhang, Helin)
>
> PATCH v2 changes:
>  - Replace rte_eal_dev_attach_pdev(), rte_eal_dev_detach_pdev,
>    rte_eal_dev_attach_vdev() and rte_eal_dev_detach_vdev() to
>    rte_eal_dev_attach() and rte_eal_dev_detach().
>  - Add parameter values checking.
>  - Refashion a few functions.
>    (Thanks to Iremonger, Bernard)
>
> PATCH v1 Changes:
>  - Fix error checking code of librte_eth APIs.
>  - Fix issue that port from pcap PMD cannot be detached correctly.
>  - Fix issue that testpmd could hang after forwarding, if attaching and detaching
>    is repeatedly.
>  - Fix if-condition of rte_eth_dev_get_port_by_addr().
>    (Thanks to Mark Enright)
>
> RFC PATCH v2 Changes:
> - remove 'rte_eth_dev_validate_port()', and cleanup codes.
>
>
> Tetsuya Mukawa (11):
>   eal/pci,ethdev: Remove assumption that port will not be detached
>   eal/pci: Consolidate pci address comparison APIs
>   ethdev: Add rte_eth_dev_free to free specified device
>   eal,ethdev: Add a function and function pointers to close ether device
>   ethdev: Add functions that will be used by port hotplug functions
>   eal/linux/pci: Add functions for unmapping igb_uio resources
>   eal/pci: Add a function to remove the entry of devargs list
>   eal/pci: Cleanup pci driver initialization code
>   ethdev: Add one dev_type paramerter to rte_eth_dev_allocate
>   eal/pci: Add rte_eal_dev_attach/detach() functions
>   eal: Enable port hotplug framework in Linux
>
>  app/test/virtual_pmd.c                          |   2 +-
>  config/common_linuxapp                          |   5 +
>  lib/librte_eal/bsdapp/eal/eal_pci.c             |  16 +-
>  lib/librte_eal/common/Makefile                  |   1 +
>  lib/librte_eal/common/eal_common_dev.c          | 273 +++++++++++++++
>  lib/librte_eal/common/eal_common_devargs.c      |  57 ++++
>  lib/librte_eal/common/eal_common_pci.c          |  92 ++++-
>  lib/librte_eal/common/eal_private.h             |  36 ++
>  lib/librte_eal/common/include/rte_dev.h         |  33 ++
>  lib/librte_eal/common/include/rte_dev_hotplug.h |  44 +++
>  lib/librte_eal/common/include/rte_devargs.h     |  18 +
>  lib/librte_eal/common/include/rte_pci.h         |  76 +++++
>  lib/librte_eal/linuxapp/eal/Makefile            |   1 +
>  lib/librte_eal/linuxapp/eal/eal_pci.c           | 136 ++++++--
>  lib/librte_eal/linuxapp/eal/eal_pci_init.h      |   8 +
>  lib/librte_eal/linuxapp/eal/eal_pci_uio.c       |  67 +++-
>  lib/librte_ether/rte_ethdev.c                   | 424 ++++++++++++++++++------
>  lib/librte_ether/rte_ethdev.h                   | 145 +++++++-
>  lib/librte_pmd_af_packet/rte_eth_af_packet.c    |   2 +-
>  lib/librte_pmd_bond/rte_eth_bond_api.c          |   2 +-
>  lib/librte_pmd_pcap/rte_eth_pcap.c              |   2 +-
>  lib/librte_pmd_ring/rte_eth_ring.c              |   2 +-
>  lib/librte_pmd_xenvirt/rte_eth_xenvirt.c        |   2 +-
>  23 files changed, 1283 insertions(+), 161 deletions(-)
>  create mode 100644 lib/librte_eal/common/include/rte_dev_hotplug.h
>


^ permalink raw reply	[flat|nested] 362+ messages in thread

* Re: [dpdk-dev] [PATCH v4 00/11] Port Hotplug Framework
  2015-01-27  3:00   ` Qiu, Michael
@ 2015-01-27  5:01     ` Tetsuya Mukawa
  2015-01-27  5:50       ` Qiu, Michael
  0 siblings, 1 reply; 362+ messages in thread
From: Tetsuya Mukawa @ 2015-01-27  5:01 UTC (permalink / raw)
  To: Qiu, Michael, dev

On 2015/01/27 12:00, Qiu, Michael wrote:
> On 1/19/2015 6:42 PM, Tetsuya Mukawa wrote:
>> This patch series adds a dynamic port hotplug framework to DPDK.
>> With the patches, DPDK apps can attach or detach ports at runtime.
>>
>> The basic concept of the port hotplug is like followings.
>> - DPDK apps must have responsibility to manage ports.
>>   DPDK apps only know which ports are attached or detached at the moment.
>>   The port hotplug framework is implemented to allow DPDK apps to manage ports.
>>   For example, when DPDK apps call port attach function, attached port number
>>   will be returned. Also DPDK apps can detach port by port number.
>> - Kernel support is needed for attaching or detaching physical device ports.
>>   To attach new device, the device will be recognized by kernel at first and
>>   controlled by kernel driver. Then user can bind the device to igb_uio
> Here does it really need native kernel driver here? As it will be
> controlled by igb_uio.
> I think even if the device has no kernel driver is also OK.

Thanks for correcting. Yes, it should be.
How about following.

- Kernel support is needed for attaching or detaching physical device ports.
  To attach a new device, the device will be recognized by kernel PCI
hotplug feature at first.
  Then user can bind the device to igb_uio.

>
> Also I have finished initial patch of passthrough driver flag in
> "struct rte_pci_device"
>
> I will send to you after I do some basic test on that, then I will send to
> you, and you can give some comments on that.

I appreciate for your implementing.

Thanks,
Tetsuya


>
> Thanks,
> Michael
>
>>   by 'dpdk_nic_bind.py'. Finally, DPDK apps can call the port hotplug
>>   functions to attach ports.
>>   For detaching, steps are vice versa.
>> - Before detach ports, ports must be stopped and closed.
>>   DPDK application must call rte_eth_dev_stop() and rte_eth_dev_close() before
>>   detaching ports. These function will call finalization codes of PMDs.
>>   But so far, no PMD frees all resources allocated by initialization.
>>   It means PMDs are needed to be fixed to support the port hotplug.
>>   'RTE_PCI_DRV_DETACHABLE' is a new flag indicating a PMD supports detaching.
>>   Without this flag, detaching will be failed.
>> - Mustn't affect legacy DPDK apps.
>>   No DPDK EAL behavior is changed, if the port hotplug functions are't called.
>>   So all legacy DPDK apps can still work without modifications.
>>
>> And a few limitations.
>> - The port hotplug functions are not thread safe.
>>   DPDK apps should handle it.
>> - Only support Linux and igb_uio so far.
>>   BSD and VFIO is not supported. I will send VFIO patches at least, but I don't
>>   have a plan to submit BSD patch so far.
>>
>>
>> Here is port hotplug APIs.
>> -------------------------------------------------------------------------------
>> /**
>>  * Attach a new device.
>>  *
>>  * @param devargs
>>  *   A pointer to a strings array describing the new device
>>  *   to be attached. The strings should be a pci address like
>>  *   '0000:01:00.0' or virtual device name like 'eth_pcap0'.
>>  * @param port_id
>>  *  A pointer to a port identifier actually attached.
>>  * @return
>>  *  0 on success and port_id is filled, negative on error
>>  */
>> int rte_eal_dev_attach(const char *devargs, uint8_t *port_id);
>>
>> /**
>>  * Detach a device.
>>  *
>>  * @param port_id
>>  *   The port identifier of the device to detach.
>>  * @param addr
>>  *  A pointer to a device name actually detached.
>>  * @return
>>  *  0 on success and devname is filled, negative on error
>>  */
>> int rte_eal_dev_detach(uint8_t port_id, char *devname);
>> -------------------------------------------------------------------------------
>>
>> This patch series are for DPDK EAL. To use port hotplug function by DPDK apps,
>> each PMD should be fixed to support 'RTE_PCI_DRV_DETACHABLE' flag. Please check
>> a patch for pcap PMD.
>>
>> Also please check testpmd patch. It will show you how to fix your legacy
>> applications to support port hotplug feature.
>>
>>
>> PATCH v4 changes
>>  - Merge patches to review easier.
>>  - Fix indent of 'if' statement.
>>  - Fix calculation method of eal_compare_pci_addr().
>>  - Fix header file declaration.
>>  - Add header file to determine if hotplug can be enabled.
>>    (Thanks to Qiu, Michael)
>>  - Use braces with 'for' loop.
>>  - Add paramerter checking.
>>  - Fix sanity check code
>>  - Fix comments of rte_eth_dev_type.
>>  - Change function names.
>>    (Thanks to Iremonger, Bernard)
>>
>> PATCH v3 changes:
>>  - Fix enum definition used in rte_ethdev.c.
>>    (Thanks to Zhang, Helin)
>>
>> PATCH v2 changes:
>>  - Replace rte_eal_dev_attach_pdev(), rte_eal_dev_detach_pdev,
>>    rte_eal_dev_attach_vdev() and rte_eal_dev_detach_vdev() to
>>    rte_eal_dev_attach() and rte_eal_dev_detach().
>>  - Add parameter values checking.
>>  - Refashion a few functions.
>>    (Thanks to Iremonger, Bernard)
>>
>> PATCH v1 Changes:
>>  - Fix error checking code of librte_eth APIs.
>>  - Fix issue that port from pcap PMD cannot be detached correctly.
>>  - Fix issue that testpmd could hang after forwarding, if attaching and detaching
>>    is repeatedly.
>>  - Fix if-condition of rte_eth_dev_get_port_by_addr().
>>    (Thanks to Mark Enright)
>>
>> RFC PATCH v2 Changes:
>> - remove 'rte_eth_dev_validate_port()', and cleanup codes.
>>
>>
>> Tetsuya Mukawa (11):
>>   eal/pci,ethdev: Remove assumption that port will not be detached
>>   eal/pci: Consolidate pci address comparison APIs
>>   ethdev: Add rte_eth_dev_free to free specified device
>>   eal,ethdev: Add a function and function pointers to close ether device
>>   ethdev: Add functions that will be used by port hotplug functions
>>   eal/linux/pci: Add functions for unmapping igb_uio resources
>>   eal/pci: Add a function to remove the entry of devargs list
>>   eal/pci: Cleanup pci driver initialization code
>>   ethdev: Add one dev_type paramerter to rte_eth_dev_allocate
>>   eal/pci: Add rte_eal_dev_attach/detach() functions
>>   eal: Enable port hotplug framework in Linux
>>
>>  app/test/virtual_pmd.c                          |   2 +-
>>  config/common_linuxapp                          |   5 +
>>  lib/librte_eal/bsdapp/eal/eal_pci.c             |  16 +-
>>  lib/librte_eal/common/Makefile                  |   1 +
>>  lib/librte_eal/common/eal_common_dev.c          | 273 +++++++++++++++
>>  lib/librte_eal/common/eal_common_devargs.c      |  57 ++++
>>  lib/librte_eal/common/eal_common_pci.c          |  92 ++++-
>>  lib/librte_eal/common/eal_private.h             |  36 ++
>>  lib/librte_eal/common/include/rte_dev.h         |  33 ++
>>  lib/librte_eal/common/include/rte_dev_hotplug.h |  44 +++
>>  lib/librte_eal/common/include/rte_devargs.h     |  18 +
>>  lib/librte_eal/common/include/rte_pci.h         |  76 +++++
>>  lib/librte_eal/linuxapp/eal/Makefile            |   1 +
>>  lib/librte_eal/linuxapp/eal/eal_pci.c           | 136 ++++++--
>>  lib/librte_eal/linuxapp/eal/eal_pci_init.h      |   8 +
>>  lib/librte_eal/linuxapp/eal/eal_pci_uio.c       |  67 +++-
>>  lib/librte_ether/rte_ethdev.c                   | 424 ++++++++++++++++++------
>>  lib/librte_ether/rte_ethdev.h                   | 145 +++++++-
>>  lib/librte_pmd_af_packet/rte_eth_af_packet.c    |   2 +-
>>  lib/librte_pmd_bond/rte_eth_bond_api.c          |   2 +-
>>  lib/librte_pmd_pcap/rte_eth_pcap.c              |   2 +-
>>  lib/librte_pmd_ring/rte_eth_ring.c              |   2 +-
>>  lib/librte_pmd_xenvirt/rte_eth_xenvirt.c        |   2 +-
>>  23 files changed, 1283 insertions(+), 161 deletions(-)
>>  create mode 100644 lib/librte_eal/common/include/rte_dev_hotplug.h
>>

^ permalink raw reply	[flat|nested] 362+ messages in thread

* Re: [dpdk-dev] [PATCH v4 00/11] Port Hotplug Framework
  2015-01-27  5:01     ` Tetsuya Mukawa
@ 2015-01-27  5:50       ` Qiu, Michael
  2015-01-27  7:24         ` Tetsuya Mukawa
  0 siblings, 1 reply; 362+ messages in thread
From: Qiu, Michael @ 2015-01-27  5:50 UTC (permalink / raw)
  To: Tetsuya Mukawa, dev

On 1/27/2015 1:02 PM, Tetsuya Mukawa wrote:
> On 2015/01/27 12:00, Qiu, Michael wrote:
>> On 1/19/2015 6:42 PM, Tetsuya Mukawa wrote:
>>> This patch series adds a dynamic port hotplug framework to DPDK.
>>> With the patches, DPDK apps can attach or detach ports at runtime.
>>>
>>> The basic concept of the port hotplug is like followings.
>>> - DPDK apps must have responsibility to manage ports.
>>>   DPDK apps only know which ports are attached or detached at the moment.
>>>   The port hotplug framework is implemented to allow DPDK apps to manage ports.
>>>   For example, when DPDK apps call port attach function, attached port number
>>>   will be returned. Also DPDK apps can detach port by port number.
>>> - Kernel support is needed for attaching or detaching physical device ports.
>>>   To attach new device, the device will be recognized by kernel at first and
>>>   controlled by kernel driver. Then user can bind the device to igb_uio
>> Here does it really need native kernel driver here? As it will be
>> controlled by igb_uio.
>> I think even if the device has no kernel driver is also OK.
> Thanks for correcting. Yes, it should be.
> How about following.
>
> - Kernel support is needed for attaching or detaching physical device ports.
>   To attach a new device, the device will be recognized by kernel PCI
> hotplug feature at first.

No, here should not explain as "kernel PCI hotplug feature" which is
stand for removing or adding a PCI device from system level, it is
devices related not driver.

What about:

- Kernel support is needed for attaching or detaching physical device
ports. To attach a new physical device port, the device will be
recognized by userspace directly I/O framework in kernel at first.

Thanks,
Michael
>   Then user can bind the device to igb_uio.
>
>> Also I have finished initial patch of passthrough driver flag in
>> "struct rte_pci_device"
>>
>> I will send to you after I do some basic test on that, then I will send to
>> you, and you can give some comments on that.
> I appreciate for your implementing.
>
> Thanks,
> Tetsuya
>
>
>> Thanks,
>> Michael
>>
>>>   by 'dpdk_nic_bind.py'. Finally, DPDK apps can call the port hotplug
>>>   functions to attach ports.
>>>   For detaching, steps are vice versa.
>>> - Before detach ports, ports must be stopped and closed.
>>>   DPDK application must call rte_eth_dev_stop() and rte_eth_dev_close() before
>>>   detaching ports. These function will call finalization codes of PMDs.
>>>   But so far, no PMD frees all resources allocated by initialization.
>>>   It means PMDs are needed to be fixed to support the port hotplug.
>>>   'RTE_PCI_DRV_DETACHABLE' is a new flag indicating a PMD supports detaching.
>>>   Without this flag, detaching will be failed.
>>> - Mustn't affect legacy DPDK apps.
>>>   No DPDK EAL behavior is changed, if the port hotplug functions are't called.
>>>   So all legacy DPDK apps can still work without modifications.
>>>
>>> And a few limitations.
>>> - The port hotplug functions are not thread safe.
>>>   DPDK apps should handle it.
>>> - Only support Linux and igb_uio so far.
>>>   BSD and VFIO is not supported. I will send VFIO patches at least, but I don't
>>>   have a plan to submit BSD patch so far.
>>>
>>>
>>> Here is port hotplug APIs.
>>> -------------------------------------------------------------------------------
>>> /**
>>>  * Attach a new device.
>>>  *
>>>  * @param devargs
>>>  *   A pointer to a strings array describing the new device
>>>  *   to be attached. The strings should be a pci address like
>>>  *   '0000:01:00.0' or virtual device name like 'eth_pcap0'.
>>>  * @param port_id
>>>  *  A pointer to a port identifier actually attached.
>>>  * @return
>>>  *  0 on success and port_id is filled, negative on error
>>>  */
>>> int rte_eal_dev_attach(const char *devargs, uint8_t *port_id);
>>>
>>> /**
>>>  * Detach a device.
>>>  *
>>>  * @param port_id
>>>  *   The port identifier of the device to detach.
>>>  * @param addr
>>>  *  A pointer to a device name actually detached.
>>>  * @return
>>>  *  0 on success and devname is filled, negative on error
>>>  */
>>> int rte_eal_dev_detach(uint8_t port_id, char *devname);
>>> -------------------------------------------------------------------------------
>>>
>>> This patch series are for DPDK EAL. To use port hotplug function by DPDK apps,
>>> each PMD should be fixed to support 'RTE_PCI_DRV_DETACHABLE' flag. Please check
>>> a patch for pcap PMD.
>>>
>>> Also please check testpmd patch. It will show you how to fix your legacy
>>> applications to support port hotplug feature.
>>>
>>>
>>> PATCH v4 changes
>>>  - Merge patches to review easier.
>>>  - Fix indent of 'if' statement.
>>>  - Fix calculation method of eal_compare_pci_addr().
>>>  - Fix header file declaration.
>>>  - Add header file to determine if hotplug can be enabled.
>>>    (Thanks to Qiu, Michael)
>>>  - Use braces with 'for' loop.
>>>  - Add paramerter checking.
>>>  - Fix sanity check code
>>>  - Fix comments of rte_eth_dev_type.
>>>  - Change function names.
>>>    (Thanks to Iremonger, Bernard)
>>>
>>> PATCH v3 changes:
>>>  - Fix enum definition used in rte_ethdev.c.
>>>    (Thanks to Zhang, Helin)
>>>
>>> PATCH v2 changes:
>>>  - Replace rte_eal_dev_attach_pdev(), rte_eal_dev_detach_pdev,
>>>    rte_eal_dev_attach_vdev() and rte_eal_dev_detach_vdev() to
>>>    rte_eal_dev_attach() and rte_eal_dev_detach().
>>>  - Add parameter values checking.
>>>  - Refashion a few functions.
>>>    (Thanks to Iremonger, Bernard)
>>>
>>> PATCH v1 Changes:
>>>  - Fix error checking code of librte_eth APIs.
>>>  - Fix issue that port from pcap PMD cannot be detached correctly.
>>>  - Fix issue that testpmd could hang after forwarding, if attaching and detaching
>>>    is repeatedly.
>>>  - Fix if-condition of rte_eth_dev_get_port_by_addr().
>>>    (Thanks to Mark Enright)
>>>
>>> RFC PATCH v2 Changes:
>>> - remove 'rte_eth_dev_validate_port()', and cleanup codes.
>>>
>>>
>>> Tetsuya Mukawa (11):
>>>   eal/pci,ethdev: Remove assumption that port will not be detached
>>>   eal/pci: Consolidate pci address comparison APIs
>>>   ethdev: Add rte_eth_dev_free to free specified device
>>>   eal,ethdev: Add a function and function pointers to close ether device
>>>   ethdev: Add functions that will be used by port hotplug functions
>>>   eal/linux/pci: Add functions for unmapping igb_uio resources
>>>   eal/pci: Add a function to remove the entry of devargs list
>>>   eal/pci: Cleanup pci driver initialization code
>>>   ethdev: Add one dev_type paramerter to rte_eth_dev_allocate
>>>   eal/pci: Add rte_eal_dev_attach/detach() functions
>>>   eal: Enable port hotplug framework in Linux
>>>
>>>  app/test/virtual_pmd.c                          |   2 +-
>>>  config/common_linuxapp                          |   5 +
>>>  lib/librte_eal/bsdapp/eal/eal_pci.c             |  16 +-
>>>  lib/librte_eal/common/Makefile                  |   1 +
>>>  lib/librte_eal/common/eal_common_dev.c          | 273 +++++++++++++++
>>>  lib/librte_eal/common/eal_common_devargs.c      |  57 ++++
>>>  lib/librte_eal/common/eal_common_pci.c          |  92 ++++-
>>>  lib/librte_eal/common/eal_private.h             |  36 ++
>>>  lib/librte_eal/common/include/rte_dev.h         |  33 ++
>>>  lib/librte_eal/common/include/rte_dev_hotplug.h |  44 +++
>>>  lib/librte_eal/common/include/rte_devargs.h     |  18 +
>>>  lib/librte_eal/common/include/rte_pci.h         |  76 +++++
>>>  lib/librte_eal/linuxapp/eal/Makefile            |   1 +
>>>  lib/librte_eal/linuxapp/eal/eal_pci.c           | 136 ++++++--
>>>  lib/librte_eal/linuxapp/eal/eal_pci_init.h      |   8 +
>>>  lib/librte_eal/linuxapp/eal/eal_pci_uio.c       |  67 +++-
>>>  lib/librte_ether/rte_ethdev.c                   | 424 ++++++++++++++++++------
>>>  lib/librte_ether/rte_ethdev.h                   | 145 +++++++-
>>>  lib/librte_pmd_af_packet/rte_eth_af_packet.c    |   2 +-
>>>  lib/librte_pmd_bond/rte_eth_bond_api.c          |   2 +-
>>>  lib/librte_pmd_pcap/rte_eth_pcap.c              |   2 +-
>>>  lib/librte_pmd_ring/rte_eth_ring.c              |   2 +-
>>>  lib/librte_pmd_xenvirt/rte_eth_xenvirt.c        |   2 +-
>>>  23 files changed, 1283 insertions(+), 161 deletions(-)
>>>  create mode 100644 lib/librte_eal/common/include/rte_dev_hotplug.h
>>>
>
>


^ permalink raw reply	[flat|nested] 362+ messages in thread

* Re: [dpdk-dev] [PATCH v4 00/11] Port Hotplug Framework
  2015-01-27  5:50       ` Qiu, Michael
@ 2015-01-27  7:24         ` Tetsuya Mukawa
  0 siblings, 0 replies; 362+ messages in thread
From: Tetsuya Mukawa @ 2015-01-27  7:24 UTC (permalink / raw)
  To: Qiu, Michael, dev

On 2015/01/27 14:50, Qiu, Michael wrote:
> On 1/27/2015 1:02 PM, Tetsuya Mukawa wrote:
>> On 2015/01/27 12:00, Qiu, Michael wrote:
>>> On 1/19/2015 6:42 PM, Tetsuya Mukawa wrote:
>>>> This patch series adds a dynamic port hotplug framework to DPDK.
>>>> With the patches, DPDK apps can attach or detach ports at runtime.
>>>>
>>>> The basic concept of the port hotplug is like followings.
>>>> - DPDK apps must have responsibility to manage ports.
>>>>   DPDK apps only know which ports are attached or detached at the moment.
>>>>   The port hotplug framework is implemented to allow DPDK apps to manage ports.
>>>>   For example, when DPDK apps call port attach function, attached port number
>>>>   will be returned. Also DPDK apps can detach port by port number.
>>>> - Kernel support is needed for attaching or detaching physical device ports.
>>>>   To attach new device, the device will be recognized by kernel at first and
>>>>   controlled by kernel driver. Then user can bind the device to igb_uio
>>> Here does it really need native kernel driver here? As it will be
>>> controlled by igb_uio.
>>> I think even if the device has no kernel driver is also OK.
>> Thanks for correcting. Yes, it should be.
>> How about following.
>>
>> - Kernel support is needed for attaching or detaching physical device ports.
>>   To attach a new device, the device will be recognized by kernel PCI
>> hotplug feature at first.
> No, here should not explain as "kernel PCI hotplug feature" which is
> stand for removing or adding a PCI device from system level, it is
> devices related not driver.
>
> What about:
>
> - Kernel support is needed for attaching or detaching physical device
> ports. To attach a new physical device port, the device will be
> recognized by userspace directly I/O framework in kernel at first.

Thanks. I will replace like above.

Thanks,
Tetsuya

> Thanks,
> Michael
>>   Then user can bind the device to igb_uio.
>>
>>> Also I have finished initial patch of passthrough driver flag in
>>> "struct rte_pci_device"
>>>
>>> I will send to you after I do some basic test on that, then I will send to
>>> you, and you can give some comments on that.
>> I appreciate for your implementing.
>>
>> Thanks,
>> Tetsuya
>>
>>
>>> Thanks,
>>> Michael
>>>
>>>>   by 'dpdk_nic_bind.py'. Finally, DPDK apps can call the port hotplug
>>>>   functions to attach ports.
>>>>   For detaching, steps are vice versa.
>>>> - Before detach ports, ports must be stopped and closed.
>>>>   DPDK application must call rte_eth_dev_stop() and rte_eth_dev_close() before
>>>>   detaching ports. These function will call finalization codes of PMDs.
>>>>   But so far, no PMD frees all resources allocated by initialization.
>>>>   It means PMDs are needed to be fixed to support the port hotplug.
>>>>   'RTE_PCI_DRV_DETACHABLE' is a new flag indicating a PMD supports detaching.
>>>>   Without this flag, detaching will be failed.
>>>> - Mustn't affect legacy DPDK apps.
>>>>   No DPDK EAL behavior is changed, if the port hotplug functions are't called.
>>>>   So all legacy DPDK apps can still work without modifications.
>>>>
>>>> And a few limitations.
>>>> - The port hotplug functions are not thread safe.
>>>>   DPDK apps should handle it.
>>>> - Only support Linux and igb_uio so far.
>>>>   BSD and VFIO is not supported. I will send VFIO patches at least, but I don't
>>>>   have a plan to submit BSD patch so far.
>>>>
>>>>
>>>> Here is port hotplug APIs.
>>>> -------------------------------------------------------------------------------
>>>> /**
>>>>  * Attach a new device.
>>>>  *
>>>>  * @param devargs
>>>>  *   A pointer to a strings array describing the new device
>>>>  *   to be attached. The strings should be a pci address like
>>>>  *   '0000:01:00.0' or virtual device name like 'eth_pcap0'.
>>>>  * @param port_id
>>>>  *  A pointer to a port identifier actually attached.
>>>>  * @return
>>>>  *  0 on success and port_id is filled, negative on error
>>>>  */
>>>> int rte_eal_dev_attach(const char *devargs, uint8_t *port_id);
>>>>
>>>> /**
>>>>  * Detach a device.
>>>>  *
>>>>  * @param port_id
>>>>  *   The port identifier of the device to detach.
>>>>  * @param addr
>>>>  *  A pointer to a device name actually detached.
>>>>  * @return
>>>>  *  0 on success and devname is filled, negative on error
>>>>  */
>>>> int rte_eal_dev_detach(uint8_t port_id, char *devname);
>>>> -------------------------------------------------------------------------------
>>>>
>>>> This patch series are for DPDK EAL. To use port hotplug function by DPDK apps,
>>>> each PMD should be fixed to support 'RTE_PCI_DRV_DETACHABLE' flag. Please check
>>>> a patch for pcap PMD.
>>>>
>>>> Also please check testpmd patch. It will show you how to fix your legacy
>>>> applications to support port hotplug feature.
>>>>
>>>>
>>>> PATCH v4 changes
>>>>  - Merge patches to review easier.
>>>>  - Fix indent of 'if' statement.
>>>>  - Fix calculation method of eal_compare_pci_addr().
>>>>  - Fix header file declaration.
>>>>  - Add header file to determine if hotplug can be enabled.
>>>>    (Thanks to Qiu, Michael)
>>>>  - Use braces with 'for' loop.
>>>>  - Add paramerter checking.
>>>>  - Fix sanity check code
>>>>  - Fix comments of rte_eth_dev_type.
>>>>  - Change function names.
>>>>    (Thanks to Iremonger, Bernard)
>>>>
>>>> PATCH v3 changes:
>>>>  - Fix enum definition used in rte_ethdev.c.
>>>>    (Thanks to Zhang, Helin)
>>>>
>>>> PATCH v2 changes:
>>>>  - Replace rte_eal_dev_attach_pdev(), rte_eal_dev_detach_pdev,
>>>>    rte_eal_dev_attach_vdev() and rte_eal_dev_detach_vdev() to
>>>>    rte_eal_dev_attach() and rte_eal_dev_detach().
>>>>  - Add parameter values checking.
>>>>  - Refashion a few functions.
>>>>    (Thanks to Iremonger, Bernard)
>>>>
>>>> PATCH v1 Changes:
>>>>  - Fix error checking code of librte_eth APIs.
>>>>  - Fix issue that port from pcap PMD cannot be detached correctly.
>>>>  - Fix issue that testpmd could hang after forwarding, if attaching and detaching
>>>>    is repeatedly.
>>>>  - Fix if-condition of rte_eth_dev_get_port_by_addr().
>>>>    (Thanks to Mark Enright)
>>>>
>>>> RFC PATCH v2 Changes:
>>>> - remove 'rte_eth_dev_validate_port()', and cleanup codes.
>>>>
>>>>
>>>> Tetsuya Mukawa (11):
>>>>   eal/pci,ethdev: Remove assumption that port will not be detached
>>>>   eal/pci: Consolidate pci address comparison APIs
>>>>   ethdev: Add rte_eth_dev_free to free specified device
>>>>   eal,ethdev: Add a function and function pointers to close ether device
>>>>   ethdev: Add functions that will be used by port hotplug functions
>>>>   eal/linux/pci: Add functions for unmapping igb_uio resources
>>>>   eal/pci: Add a function to remove the entry of devargs list
>>>>   eal/pci: Cleanup pci driver initialization code
>>>>   ethdev: Add one dev_type paramerter to rte_eth_dev_allocate
>>>>   eal/pci: Add rte_eal_dev_attach/detach() functions
>>>>   eal: Enable port hotplug framework in Linux
>>>>
>>>>  app/test/virtual_pmd.c                          |   2 +-
>>>>  config/common_linuxapp                          |   5 +
>>>>  lib/librte_eal/bsdapp/eal/eal_pci.c             |  16 +-
>>>>  lib/librte_eal/common/Makefile                  |   1 +
>>>>  lib/librte_eal/common/eal_common_dev.c          | 273 +++++++++++++++
>>>>  lib/librte_eal/common/eal_common_devargs.c      |  57 ++++
>>>>  lib/librte_eal/common/eal_common_pci.c          |  92 ++++-
>>>>  lib/librte_eal/common/eal_private.h             |  36 ++
>>>>  lib/librte_eal/common/include/rte_dev.h         |  33 ++
>>>>  lib/librte_eal/common/include/rte_dev_hotplug.h |  44 +++
>>>>  lib/librte_eal/common/include/rte_devargs.h     |  18 +
>>>>  lib/librte_eal/common/include/rte_pci.h         |  76 +++++
>>>>  lib/librte_eal/linuxapp/eal/Makefile            |   1 +
>>>>  lib/librte_eal/linuxapp/eal/eal_pci.c           | 136 ++++++--
>>>>  lib/librte_eal/linuxapp/eal/eal_pci_init.h      |   8 +
>>>>  lib/librte_eal/linuxapp/eal/eal_pci_uio.c       |  67 +++-
>>>>  lib/librte_ether/rte_ethdev.c                   | 424 ++++++++++++++++++------
>>>>  lib/librte_ether/rte_ethdev.h                   | 145 +++++++-
>>>>  lib/librte_pmd_af_packet/rte_eth_af_packet.c    |   2 +-
>>>>  lib/librte_pmd_bond/rte_eth_bond_api.c          |   2 +-
>>>>  lib/librte_pmd_pcap/rte_eth_pcap.c              |   2 +-
>>>>  lib/librte_pmd_ring/rte_eth_ring.c              |   2 +-
>>>>  lib/librte_pmd_xenvirt/rte_eth_xenvirt.c        |   2 +-
>>>>  23 files changed, 1283 insertions(+), 161 deletions(-)
>>>>  create mode 100644 lib/librte_eal/common/include/rte_dev_hotplug.h
>>>>
>>

^ permalink raw reply	[flat|nested] 362+ messages in thread

* [dpdk-dev] [PATCH v5 00/13] Port Hotplug Framework
  2015-01-19 10:40   ` [dpdk-dev] [PATCH v4 08/11] eal/pci: Cleanup pci driver initialization code Tetsuya Mukawa
@ 2015-01-30  5:42     ` Tetsuya Mukawa
  2015-01-30  5:42       ` [dpdk-dev] [PATCH v5 01/13] eal_pci: Add flag to hold kernel driver type Tetsuya Mukawa
                         ` (14 more replies)
  2015-02-01  4:01     ` [dpdk-dev] [PATCH v6 00/13] Port Hotplug Framework Tetsuya Mukawa
  1 sibling, 15 replies; 362+ messages in thread
From: Tetsuya Mukawa @ 2015-01-30  5:42 UTC (permalink / raw)
  To: dev

This patch series adds a dynamic port hotplug framework to DPDK.
With the patches, DPDK apps can attach or detach ports at runtime.

The basic concept of the port hotplug is like followings.
- DPDK apps must have responsibility to manage ports.
  DPDK apps only know which ports are attached or detached at the moment.
  The port hotplug framework is implemented to allow DPDK apps to manage ports.
  For example, when DPDK apps call port attach function, attached port number
  will be returned. Also DPDK apps can detach port by port number.
- Kernel support is needed for attaching or detaching physical device ports.
  To attach a new physical device port, the device will be recognized by
  userspace directly I/O framework in kernel at first. Then DPDK apps can
  call the port hotplug functions to attach ports.
  For detaching, steps are vice versa.
- Before detach ports, ports must be stopped and closed.
  DPDK application must call rte_eth_dev_stop() and rte_eth_dev_close() before
  detaching ports. These function will call finalization codes of PMDs.
  But so far, no PMD frees all resources allocated by initialization.
  It means PMDs are needed to be fixed to support the port hotplug.
  'RTE_PCI_DRV_DETACHABLE' is a new flag indicating a PMD supports detaching.
  Without this flag, detaching will be failed.
- Mustn't affect legacy DPDK apps.
  No DPDK EAL behavior is changed, if the port hotplug functions are't called.
  So all legacy DPDK apps can still work without modifications.

And a few limitations.
- The port hotplug functions are not thread safe.
  DPDK apps should handle it.
- Only support Linux and igb_uio so far.
  BSD and VFIO is not supported. I will send VFIO patches at least, but I don't
  have a plan to submit BSD patch so far.


Here is port hotplug APIs.
-------------------------------------------------------------------------------
/**
 * Attach a new device.
 *
 * @param devargs
 *   A pointer to a strings array describing the new device
 *   to be attached. The strings should be a pci address like
 *   '0000:01:00.0' or virtual device name like 'eth_pcap0'.
 * @param port_id
 *  A pointer to a port identifier actually attached.
 * @return
 *  0 on success and port_id is filled, negative on error
 */
int rte_eal_dev_attach(const char *devargs, uint8_t *port_id);

/**
 * Detach a device.
 *
 * @param port_id
 *   The port identifier of the device to detach.
 * @param addr
 *  A pointer to a device name actually detached.
 * @return
 *  0 on success and devname is filled, negative on error
 */
int rte_eal_dev_detach(uint8_t port_id, char *devname);
-------------------------------------------------------------------------------

This patch series are for DPDK EAL. To use port hotplug function by DPDK apps,
each PMD should be fixed to support 'RTE_PCI_DRV_DETACHABLE' flag. Please check
a patch for pcap PMD.

Also please check testpmd patch. It will show you how to fix your legacy
applications to support port hotplug feature.

PATCH v5 changes
 - Add runtime check passthrough driver type, like vfio-pci, igb_uio
   and uio_pci_generic.
   This was done by Qiu, Michael. Thanks a lot.
 - Change function names like below.
   - rte_eal_dev_find_and_invoke() to rte_eal_vdev_find_and_invoke().
   - rte_eal_dev_invoke() to rte_eal_vdev_invoke().
 - Add code to handle a return value of rte_eal_devargs_remove().
 - Fix pci address format in rte_eal_dev_detach().
 - Remove RTE_EAL_INVOKE_TYPE_UNKNOWN, because it's unused.
 - Change function definition of rte_eal_devargs_remove().
 - Fix pci_unmap_device() to check pt_driver.
 - Fix return value of below functions.
   - rte_eth_dev_get_changed_port().
   - rte_eth_dev_get_port_by_addr().
 - Change paramters of rte_eth_dev_validate_port() to cleanup code.
 - Fix pci_scan_one to handle pt_driver correctly.
   (Thanks to Qiu, Michael for above sugesstions)

PATCH v4 changes
 - Merge patches to review easier.
 - Fix indent of 'if' statement.
 - Fix calculation method of eal_compare_pci_addr().
 - Fix header file declaration.
 - Add header file to determine if hotplug can be enabled.
   (Thanks to Qiu, Michael)
 - Use braces with 'for' loop.
 - Add paramerter checking.
 - Fix sanity check code
 - Fix comments of rte_eth_dev_type.
 - Change function names.
   (Thanks to Iremonger, Bernard)

PATCH v3 changes:
 - Fix enum definition used in rte_ethdev.c.
   (Thanks to Zhang, Helin)

PATCH v2 changes:
 - Replace rte_eal_dev_attach_pdev(), rte_eal_dev_detach_pdev,
   rte_eal_dev_attach_vdev() and rte_eal_dev_detach_vdev() to
   rte_eal_dev_attach() and rte_eal_dev_detach().
 - Add parameter values checking.
 - Refashion a few functions.
   (Thanks to Iremonger, Bernard)

PATCH v1 Changes:
 - Fix error checking code of librte_eth APIs.
 - Fix issue that port from pcap PMD cannot be detached correctly.
 - Fix issue that testpmd could hang after forwarding, if attaching and detaching
   is repeatedly.
 - Fix if-condition of rte_eth_dev_get_port_by_addr().
   (Thanks to Mark Enright)

RFC PATCH v2 Changes:
- remove 'rte_eth_dev_validate_port()', and cleanup codes.


Michael Qiu (2):
  eal_pci: Add flag to hold kernel driver type
  eal_pci: pci memory map work with driver type

Tetsuya Mukawa (11):
  eal/pci,ethdev: Remove assumption that port will not be detached
  eal/pci: Consolidate pci address comparison APIs
  ethdev: Add rte_eth_dev_free to free specified device
  eal,ethdev: Add a function and function pointers to close ether device
  ethdev: Add functions that will be used by port hotplug functions
  eal/linux/pci: Add functions for unmapping igb_uio resources
  eal/pci: Add a function to remove the entry of devargs list
  eal/pci: Cleanup pci driver initialization code
  ethdev: Add one dev_type paramerter to rte_eth_dev_allocate
  eal/pci: Add rte_eal_dev_attach/detach() functions
  eal: Enable port hotplug framework in Linux

 app/test/virtual_pmd.c                          |   2 +-
 config/common_linuxapp                          |   5 +
 lib/librte_eal/bsdapp/eal/eal_pci.c             |  25 +-
 lib/librte_eal/common/Makefile                  |   1 +
 lib/librte_eal/common/eal_common_dev.c          | 274 +++++++++++
 lib/librte_eal/common/eal_common_devargs.c      |  60 +++
 lib/librte_eal/common/eal_common_pci.c          |  92 +++-
 lib/librte_eal/common/eal_private.h             |  35 ++
 lib/librte_eal/common/include/rte_dev.h         |  33 ++
 lib/librte_eal/common/include/rte_dev_hotplug.h |  44 ++
 lib/librte_eal/common/include/rte_devargs.h     |  21 +
 lib/librte_eal/common/include/rte_pci.h         |  84 ++++
 lib/librte_eal/linuxapp/eal/Makefile            |   1 +
 lib/librte_eal/linuxapp/eal/eal_pci.c           | 227 +++++++--
 lib/librte_eal/linuxapp/eal/eal_pci_init.h      |   8 +
 lib/librte_eal/linuxapp/eal/eal_pci_uio.c       |  67 ++-
 lib/librte_ether/rte_ethdev.c                   | 626 +++++++++++++-----------
 lib/librte_ether/rte_ethdev.h                   | 145 +++++-
 lib/librte_pmd_af_packet/rte_eth_af_packet.c    |   2 +-
 lib/librte_pmd_bond/rte_eth_bond_api.c          |   2 +-
 lib/librte_pmd_pcap/rte_eth_pcap.c              |   2 +-
 lib/librte_pmd_ring/rte_eth_ring.c              |   2 +-
 lib/librte_pmd_xenvirt/rte_eth_xenvirt.c        |   2 +-
 23 files changed, 1413 insertions(+), 347 deletions(-)
 create mode 100644 lib/librte_eal/common/include/rte_dev_hotplug.h

-- 
1.9.1

^ permalink raw reply	[flat|nested] 362+ messages in thread

* [dpdk-dev] [PATCH v5 01/13] eal_pci: Add flag to hold kernel driver type
  2015-01-30  5:42     ` [dpdk-dev] [PATCH v5 00/13] Port Hotplug Framework Tetsuya Mukawa
@ 2015-01-30  5:42       ` Tetsuya Mukawa
  2015-01-30  5:42       ` [dpdk-dev] [PATCH v5 02/13] eal_pci: pci memory map work with " Tetsuya Mukawa
                         ` (13 subsequent siblings)
  14 siblings, 0 replies; 362+ messages in thread
From: Tetsuya Mukawa @ 2015-01-30  5:42 UTC (permalink / raw)
  To: dev

From: Michael Qiu <michael.qiu@intel.com>

Currently, dpdk has no ability to know which type of driver(
vfio-pci/igb_uio/uio_pci_generic) the device used. It only can
check whether vfio is enabled or not staticly.

It really useful to have the flag, becasue different type need to
handle differently in runtime. For example, pci memory map,
pot hotplug, and so on.

This patch add a flag field for pci device to solve above issue.

Signed-off-by: Michael Qiu <michael.qiu@intel.com>
Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp>
---
 lib/librte_eal/common/include/rte_pci.h |  8 +++++
 lib/librte_eal/linuxapp/eal/eal_pci.c   | 53 +++++++++++++++++++++++++++++++--
 2 files changed, 59 insertions(+), 2 deletions(-)

diff --git a/lib/librte_eal/common/include/rte_pci.h b/lib/librte_eal/common/include/rte_pci.h
index 66ed793..7b48b55 100644
--- a/lib/librte_eal/common/include/rte_pci.h
+++ b/lib/librte_eal/common/include/rte_pci.h
@@ -139,6 +139,13 @@ struct rte_pci_addr {
 
 struct rte_devargs;
 
+enum rte_pt_driver {
+	RTE_PT_UNKNOWN		= 0,
+	RTE_PT_IGB_UIO		= 1,
+	RTE_PT_VFIO		= 2,
+	RTE_PT_UIO_GENERIC	= 3,
+};
+
 /**
  * A structure describing a PCI device.
  */
@@ -152,6 +159,7 @@ struct rte_pci_device {
 	uint16_t max_vfs;                       /**< sriov enable if not zero */
 	int numa_node;                          /**< NUMA node connection */
 	struct rte_devargs *devargs;            /**< Device user arguments */
+	enum rte_pt_driver pt_driver;		/**< Driver of passthrough */
 };
 
 /** Any PCI device identifier (vendor, device, ...) */
diff --git a/lib/librte_eal/linuxapp/eal/eal_pci.c b/lib/librte_eal/linuxapp/eal/eal_pci.c
index b5f5410..bd3f77d 100644
--- a/lib/librte_eal/linuxapp/eal/eal_pci.c
+++ b/lib/librte_eal/linuxapp/eal/eal_pci.c
@@ -97,6 +97,35 @@ error:
 	return -1;
 }
 
+static int
+pci_get_kernel_driver_by_path(const char *filename, char *dri_name)
+{
+	int count;
+	char path[PATH_MAX];
+	char *name;
+
+	if (!filename || !dri_name)
+		return -1;
+
+	count = readlink(filename, path, PATH_MAX);
+	if (count >= PATH_MAX)
+		return -1;
+
+	/* For device does not have a driver */
+	if (count < 0)
+		return 1;
+
+	path[count] = '\0';
+
+	name = strrchr(path, '/');
+	if (name) {
+		strncpy(dri_name, name + 1, strlen(name + 1) + 1);
+		return 0;
+	}
+
+	return -1;
+}
+
 void *
 pci_find_max_end_va(void)
 {
@@ -222,11 +251,12 @@ pci_scan_one(const char *dirname, uint16_t domain, uint8_t bus,
 	char filename[PATH_MAX];
 	unsigned long tmp;
 	struct rte_pci_device *dev;
+	char driver[PATH_MAX];
+	int ret;
 
 	dev = malloc(sizeof(*dev));
-	if (dev == NULL) {
+	if (dev == NULL)
 		return -1;
-	}
 
 	memset(dev, 0, sizeof(*dev));
 	dev->addr.domain = domain;
@@ -298,6 +328,25 @@ pci_scan_one(const char *dirname, uint16_t domain, uint8_t bus,
 		return -1;
 	}
 
+	/* parse driver */
+	snprintf(filename, sizeof(filename), "%s/driver", dirname);
+	ret = pci_get_kernel_driver_by_path(filename, driver);
+	if (!ret) {
+		if (!strcmp(driver, "vfio-pci"))
+			dev->pt_driver = RTE_PT_VFIO;
+		else if (!strcmp(driver, "igb_uio"))
+			dev->pt_driver = RTE_PT_IGB_UIO;
+		else if (!strcmp(driver, "uio_pci_generic"))
+			dev->pt_driver = RTE_PT_UIO_GENERIC;
+		else
+			dev->pt_driver = RTE_PT_UNKNOWN;
+	} else if (ret < 0) {
+		RTE_LOG(ERR, EAL, "Fail to get kernel driver\n");
+		free(dev);
+		return -1;
+	} else
+		dev->pt_driver = RTE_PT_UNKNOWN;
+
 	/* device is valid, add in list (sorted) */
 	if (TAILQ_EMPTY(&pci_device_list)) {
 		TAILQ_INSERT_TAIL(&pci_device_list, dev, next);
-- 
1.9.1

^ permalink raw reply	[flat|nested] 362+ messages in thread

* [dpdk-dev] [PATCH v5 02/13] eal_pci: pci memory map work with driver type
  2015-01-30  5:42     ` [dpdk-dev] [PATCH v5 00/13] Port Hotplug Framework Tetsuya Mukawa
  2015-01-30  5:42       ` [dpdk-dev] [PATCH v5 01/13] eal_pci: Add flag to hold kernel driver type Tetsuya Mukawa
@ 2015-01-30  5:42       ` Tetsuya Mukawa
  2015-01-30  5:42       ` [dpdk-dev] [PATCH v5 03/13] eal/pci, ethdev: Remove assumption that port will not be detached Tetsuya Mukawa
                         ` (12 subsequent siblings)
  14 siblings, 0 replies; 362+ messages in thread
From: Tetsuya Mukawa @ 2015-01-30  5:42 UTC (permalink / raw)
  To: dev

From: Michael Qiu <michael.qiu@intel.com>

With the driver type flag in struct rte_pci_dev, we do not need
to always  map uio devices with vfio related function when
vfio enabled.

Signed-off-by: Michael Qiu <michael.qiu@intel.com>
Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp>
---
 lib/librte_eal/linuxapp/eal/eal_pci.c | 30 +++++++++++++++++-------------
 1 file changed, 17 insertions(+), 13 deletions(-)

diff --git a/lib/librte_eal/linuxapp/eal/eal_pci.c b/lib/librte_eal/linuxapp/eal/eal_pci.c
index bd3f77d..c0ca5a5 100644
--- a/lib/librte_eal/linuxapp/eal/eal_pci.c
+++ b/lib/librte_eal/linuxapp/eal/eal_pci.c
@@ -549,25 +549,29 @@ pci_config_space_set(struct rte_pci_device *dev)
 static int
 pci_map_device(struct rte_pci_device *dev)
 {
-	int ret, mapped = 0;
+	int ret = -1;
 
 	/* try mapping the NIC resources using VFIO if it exists */
+	switch (dev->pt_driver) {
+	case RTE_PT_VFIO:
 #ifdef VFIO_PRESENT
-	if (pci_vfio_is_enabled()) {
-		ret = pci_vfio_map_resource(dev);
-		if (ret == 0)
-			mapped = 1;
-		else if (ret < 0)
-			return ret;
-	}
+		if (pci_vfio_is_enabled())
+			ret = pci_vfio_map_resource(dev);
 #endif
-	/* map resources for devices that use igb_uio */
-	if (!mapped) {
+		break;
+	case RTE_PT_IGB_UIO:
+	case RTE_PT_UIO_GENERIC:
+		/* map resources for devices that use uio */
 		ret = pci_uio_map_resource(dev);
-		if (ret != 0)
-			return ret;
+		break;
+	default:
+		RTE_LOG(DEBUG, EAL, "  Not managed by known pt driver,"
+			" skipped\n");
+		ret = 1;
+		break;
 	}
-	return 0;
+
+	return ret;
 }
 
 /*
-- 
1.9.1

^ permalink raw reply	[flat|nested] 362+ messages in thread

* [dpdk-dev] [PATCH v5 03/13] eal/pci, ethdev: Remove assumption that port will not be detached
  2015-01-30  5:42     ` [dpdk-dev] [PATCH v5 00/13] Port Hotplug Framework Tetsuya Mukawa
  2015-01-30  5:42       ` [dpdk-dev] [PATCH v5 01/13] eal_pci: Add flag to hold kernel driver type Tetsuya Mukawa
  2015-01-30  5:42       ` [dpdk-dev] [PATCH v5 02/13] eal_pci: pci memory map work with " Tetsuya Mukawa
@ 2015-01-30  5:42       ` Tetsuya Mukawa
  2015-01-30  5:42       ` [dpdk-dev] [PATCH v5 04/13] eal/pci: Consolidate pci address comparison APIs Tetsuya Mukawa
                         ` (11 subsequent siblings)
  14 siblings, 0 replies; 362+ messages in thread
From: Tetsuya Mukawa @ 2015-01-30  5:42 UTC (permalink / raw)
  To: dev

To remove assumption, do like followings.

This patch adds "RTE_PCI_DRV_DETACHABLE" to drv_flags of rte_pci_driver
structure. The flags indicates the driver can detach devices at runtime.
Also remove assumption that port will not be detached.

To remove the assumption.
- Add 'attached' member to rte_eth_dev structure.
  This member is used for indicating the port is attached, or not.
- Add rte_eth_dev_allocate_new_port().
  This function is used for allocating new port.

v5:
- Change paramters of rte_eth_dev_validate_port() to cleanup code.
v4:
- Use braces with 'for' loop.
- Fix indent of 'if' statement.

Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp>
---
 lib/librte_eal/common/include/rte_pci.h |   2 +
 lib/librte_ether/rte_ethdev.c           | 454 +++++++++++++-------------------
 lib/librte_ether/rte_ethdev.h           |   5 +
 3 files changed, 186 insertions(+), 275 deletions(-)

diff --git a/lib/librte_eal/common/include/rte_pci.h b/lib/librte_eal/common/include/rte_pci.h
index 7b48b55..7f2d699 100644
--- a/lib/librte_eal/common/include/rte_pci.h
+++ b/lib/librte_eal/common/include/rte_pci.h
@@ -207,6 +207,8 @@ struct rte_pci_driver {
 #define RTE_PCI_DRV_FORCE_UNBIND 0x0004
 /** Device driver supports link state interrupt */
 #define RTE_PCI_DRV_INTR_LSC	0x0008
+/** Device driver supports detaching capability */
+#define RTE_PCI_DRV_DETACHABLE	0x0010
 
 /**< Internal use only - Macro used by pci addr parsing functions **/
 #define GET_PCIADDR_FIELD(in, fd, lim, dlm)                   \
diff --git a/lib/librte_ether/rte_ethdev.c b/lib/librte_ether/rte_ethdev.c
index ea3a1fb..d70854f 100644
--- a/lib/librte_ether/rte_ethdev.c
+++ b/lib/librte_ether/rte_ethdev.c
@@ -175,6 +175,16 @@ enum {
 	STAT_QMAP_RX
 };
 
+enum {
+	DEV_INVALID = 0,
+	DEV_VALID,
+};
+
+enum {
+	DEV_DISCONNECTED = 0,
+	DEV_CONNECTED
+};
+
 static inline void
 rte_eth_dev_data_alloc(void)
 {
@@ -201,19 +211,34 @@ rte_eth_dev_allocated(const char *name)
 {
 	unsigned i;
 
-	for (i = 0; i < nb_ports; i++) {
-		if (strcmp(rte_eth_devices[i].data->name, name) == 0)
+	for (i = 0; i < RTE_MAX_ETHPORTS; i++) {
+		if ((rte_eth_devices[i].attached == DEV_CONNECTED) &&
+		    strcmp(rte_eth_devices[i].data->name, name) == 0)
 			return &rte_eth_devices[i];
 	}
 	return NULL;
 }
 
+static uint8_t
+rte_eth_dev_allocate_new_port(void)
+{
+	unsigned i;
+
+	for (i = 0; i < RTE_MAX_ETHPORTS; i++) {
+		if (rte_eth_devices[i].attached == DEV_DISCONNECTED)
+			return i;
+	}
+	return RTE_MAX_ETHPORTS;
+}
+
 struct rte_eth_dev *
 rte_eth_dev_allocate(const char *name)
 {
+	uint8_t port_id;
 	struct rte_eth_dev *eth_dev;
 
-	if (nb_ports == RTE_MAX_ETHPORTS) {
+	port_id = rte_eth_dev_allocate_new_port();
+	if (port_id == RTE_MAX_ETHPORTS) {
 		PMD_DEBUG_TRACE("Reached maximum number of Ethernet ports\n");
 		return NULL;
 	}
@@ -226,10 +251,12 @@ rte_eth_dev_allocate(const char *name)
 		return NULL;
 	}
 
-	eth_dev = &rte_eth_devices[nb_ports];
-	eth_dev->data = &rte_eth_dev_data[nb_ports];
+	eth_dev = &rte_eth_devices[port_id];
+	eth_dev->data = &rte_eth_dev_data[port_id];
 	snprintf(eth_dev->data->name, sizeof(eth_dev->data->name), "%s", name);
-	eth_dev->data->port_id = nb_ports++;
+	eth_dev->data->port_id = port_id;
+	eth_dev->attached = DEV_CONNECTED;
+	nb_ports++;
 	return eth_dev;
 }
 
@@ -283,6 +310,7 @@ rte_eth_dev_init(struct rte_pci_driver *pci_drv,
 			(unsigned) pci_dev->id.device_id);
 	if (rte_eal_process_type() == RTE_PROC_PRIMARY)
 		rte_free(eth_dev->data->dev_private);
+	eth_dev->attached = DEV_DISCONNECTED;
 	nb_ports--;
 	return diag;
 }
@@ -308,10 +336,28 @@ rte_eth_driver_register(struct eth_driver *eth_drv)
 	rte_eal_pci_register(&eth_drv->pci_drv);
 }
 
+enum {
+	NONE_TRACE = 0,
+	TRACE
+};
+
+static int
+rte_eth_dev_validate_port(uint8_t port_id, int trace)
+{
+	if (port_id >= RTE_MAX_ETHPORTS ||
+	    rte_eth_devices[port_id].attached != DEV_CONNECTED) {
+		if (trace) {
+			PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
+		}
+		return DEV_INVALID;
+	} else
+		return DEV_VALID;
+}
+
 int
 rte_eth_dev_socket_id(uint8_t port_id)
 {
-	if (port_id >= nb_ports)
+	if (rte_eth_dev_validate_port(port_id, NONE_TRACE) == DEV_INVALID)
 		return -1;
 	return rte_eth_devices[port_id].pci_dev->numa_node;
 }
@@ -369,10 +415,8 @@ rte_eth_dev_rx_queue_start(uint8_t port_id, uint16_t rx_queue_id)
 	 * in a multi-process setup*/
 	PROC_PRIMARY_OR_ERR_RET(-E_RTE_SECONDARY);
 
-	if (port_id >= nb_ports) {
-		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
+	if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID)
 		return -EINVAL;
-	}
 
 	dev = &rte_eth_devices[port_id];
 	if (rx_queue_id >= dev->data->nb_rx_queues) {
@@ -395,10 +439,8 @@ rte_eth_dev_rx_queue_stop(uint8_t port_id, uint16_t rx_queue_id)
 	 * in a multi-process setup*/
 	PROC_PRIMARY_OR_ERR_RET(-E_RTE_SECONDARY);
 
-	if (port_id >= nb_ports) {
-		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
+	if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID)
 		return -EINVAL;
-	}
 
 	dev = &rte_eth_devices[port_id];
 	if (rx_queue_id >= dev->data->nb_rx_queues) {
@@ -421,10 +463,8 @@ rte_eth_dev_tx_queue_start(uint8_t port_id, uint16_t tx_queue_id)
 	 * in a multi-process setup*/
 	PROC_PRIMARY_OR_ERR_RET(-E_RTE_SECONDARY);
 
-	if (port_id >= nb_ports) {
-		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
+	if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID)
 		return -EINVAL;
-	}
 
 	dev = &rte_eth_devices[port_id];
 	if (tx_queue_id >= dev->data->nb_tx_queues) {
@@ -447,10 +487,8 @@ rte_eth_dev_tx_queue_stop(uint8_t port_id, uint16_t tx_queue_id)
 	 * in a multi-process setup*/
 	PROC_PRIMARY_OR_ERR_RET(-E_RTE_SECONDARY);
 
-	if (port_id >= nb_ports) {
-		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
+	if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID)
 		return -EINVAL;
-	}
 
 	dev = &rte_eth_devices[port_id];
 	if (tx_queue_id >= dev->data->nb_tx_queues) {
@@ -703,10 +741,9 @@ rte_eth_dev_configure(uint8_t port_id, uint16_t nb_rx_q, uint16_t nb_tx_q,
 	 * in a multi-process setup*/
 	PROC_PRIMARY_OR_ERR_RET(-E_RTE_SECONDARY);
 
-	if (port_id >= nb_ports || port_id >= RTE_MAX_ETHPORTS) {
-		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
+	if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID)
 		return (-EINVAL);
-	}
+
 	dev = &rte_eth_devices[port_id];
 
 	FUNC_PTR_OR_ERR_RET(*dev->dev_ops->dev_infos_get, -ENOTSUP);
@@ -888,10 +925,9 @@ rte_eth_dev_start(uint8_t port_id)
 	 * in a multi-process setup*/
 	PROC_PRIMARY_OR_ERR_RET(-E_RTE_SECONDARY);
 
-	if (port_id >= nb_ports) {
-		PMD_DEBUG_TRACE("Invalid port_id=%" PRIu8 "\n", port_id);
+	if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID)
 		return (-EINVAL);
-	}
+
 	dev = &rte_eth_devices[port_id];
 
 	FUNC_PTR_OR_ERR_RET(*dev->dev_ops->dev_start, -ENOTSUP);
@@ -923,10 +959,9 @@ rte_eth_dev_stop(uint8_t port_id)
 	 * in a multi-process setup*/
 	PROC_PRIMARY_OR_RET();
 
-	if (port_id >= nb_ports) {
-		PMD_DEBUG_TRACE("Invalid port_id=%" PRIu8 "\n", port_id);
+	if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID)
 		return;
-	}
+
 	dev = &rte_eth_devices[port_id];
 
 	FUNC_PTR_OR_RET(*dev->dev_ops->dev_stop);
@@ -951,10 +986,9 @@ rte_eth_dev_set_link_up(uint8_t port_id)
 	 * in a multi-process setup*/
 	PROC_PRIMARY_OR_ERR_RET(-E_RTE_SECONDARY);
 
-	if (port_id >= nb_ports) {
-		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
+	if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID)
 		return -EINVAL;
-	}
+
 	dev = &rte_eth_devices[port_id];
 
 	FUNC_PTR_OR_ERR_RET(*dev->dev_ops->dev_set_link_up, -ENOTSUP);
@@ -970,10 +1004,9 @@ rte_eth_dev_set_link_down(uint8_t port_id)
 	 * in a multi-process setup*/
 	PROC_PRIMARY_OR_ERR_RET(-E_RTE_SECONDARY);
 
-	if (port_id >= nb_ports) {
-		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
+	if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID)
 		return -EINVAL;
-	}
+
 	dev = &rte_eth_devices[port_id];
 
 	FUNC_PTR_OR_ERR_RET(*dev->dev_ops->dev_set_link_down, -ENOTSUP);
@@ -989,10 +1022,8 @@ rte_eth_dev_close(uint8_t port_id)
 	 * in a multi-process setup*/
 	PROC_PRIMARY_OR_RET();
 
-	if (port_id >= nb_ports) {
-		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
+	if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID)
 		return;
-	}
 
 	dev = &rte_eth_devices[port_id];
 
@@ -1017,10 +1048,9 @@ rte_eth_rx_queue_setup(uint8_t port_id, uint16_t rx_queue_id,
 	 * in a multi-process setup*/
 	PROC_PRIMARY_OR_ERR_RET(-E_RTE_SECONDARY);
 
-	if (port_id >= nb_ports) {
-		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
+	if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID)
 		return (-EINVAL);
-	}
+
 	dev = &rte_eth_devices[port_id];
 	if (rx_queue_id >= dev->data->nb_rx_queues) {
 		PMD_DEBUG_TRACE("Invalid RX queue_id=%d\n", rx_queue_id);
@@ -1090,10 +1120,9 @@ rte_eth_tx_queue_setup(uint8_t port_id, uint16_t tx_queue_id,
 	 * in a multi-process setup*/
 	PROC_PRIMARY_OR_ERR_RET(-E_RTE_SECONDARY);
 
-	if (port_id >= RTE_MAX_ETHPORTS || port_id >= nb_ports) {
-		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
+	if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID)
 		return (-EINVAL);
-	}
+
 	dev = &rte_eth_devices[port_id];
 	if (tx_queue_id >= dev->data->nb_tx_queues) {
 		PMD_DEBUG_TRACE("Invalid TX queue_id=%d\n", tx_queue_id);
@@ -1123,10 +1152,9 @@ rte_eth_promiscuous_enable(uint8_t port_id)
 {
 	struct rte_eth_dev *dev;
 
-	if (port_id >= nb_ports) {
-		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
+	if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID)
 		return;
-	}
+
 	dev = &rte_eth_devices[port_id];
 
 	FUNC_PTR_OR_RET(*dev->dev_ops->promiscuous_enable);
@@ -1139,10 +1167,9 @@ rte_eth_promiscuous_disable(uint8_t port_id)
 {
 	struct rte_eth_dev *dev;
 
-	if (port_id >= nb_ports) {
-		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
+	if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID)
 		return;
-	}
+
 	dev = &rte_eth_devices[port_id];
 
 	FUNC_PTR_OR_RET(*dev->dev_ops->promiscuous_disable);
@@ -1155,10 +1182,8 @@ rte_eth_promiscuous_get(uint8_t port_id)
 {
 	struct rte_eth_dev *dev;
 
-	if (port_id >= nb_ports) {
-		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
+	if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID)
 		return -1;
-	}
 
 	dev = &rte_eth_devices[port_id];
 	return dev->data->promiscuous;
@@ -1169,10 +1194,9 @@ rte_eth_allmulticast_enable(uint8_t port_id)
 {
 	struct rte_eth_dev *dev;
 
-	if (port_id >= nb_ports) {
-		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
+	if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID)
 		return;
-	}
+
 	dev = &rte_eth_devices[port_id];
 
 	FUNC_PTR_OR_RET(*dev->dev_ops->allmulticast_enable);
@@ -1185,10 +1209,9 @@ rte_eth_allmulticast_disable(uint8_t port_id)
 {
 	struct rte_eth_dev *dev;
 
-	if (port_id >= nb_ports) {
-		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
+	if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID)
 		return;
-	}
+
 	dev = &rte_eth_devices[port_id];
 
 	FUNC_PTR_OR_RET(*dev->dev_ops->allmulticast_disable);
@@ -1201,10 +1224,8 @@ rte_eth_allmulticast_get(uint8_t port_id)
 {
 	struct rte_eth_dev *dev;
 
-	if (port_id >= nb_ports) {
-		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
+	if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID)
 		return -1;
-	}
 
 	dev = &rte_eth_devices[port_id];
 	return dev->data->all_multicast;
@@ -1229,10 +1250,9 @@ rte_eth_link_get(uint8_t port_id, struct rte_eth_link *eth_link)
 {
 	struct rte_eth_dev *dev;
 
-	if (port_id >= nb_ports) {
-		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
+	if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID)
 		return;
-	}
+
 	dev = &rte_eth_devices[port_id];
 
 	if (dev->data->dev_conf.intr_conf.lsc != 0)
@@ -1249,10 +1269,9 @@ rte_eth_link_get_nowait(uint8_t port_id, struct rte_eth_link *eth_link)
 {
 	struct rte_eth_dev *dev;
 
-	if (port_id >= nb_ports) {
-		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
+	if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID)
 		return;
-	}
+
 	dev = &rte_eth_devices[port_id];
 
 	if (dev->data->dev_conf.intr_conf.lsc != 0)
@@ -1269,10 +1288,9 @@ rte_eth_stats_get(uint8_t port_id, struct rte_eth_stats *stats)
 {
 	struct rte_eth_dev *dev;
 
-	if (port_id >= nb_ports) {
-		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
+	if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID)
 		return;
-	}
+
 	dev = &rte_eth_devices[port_id];
 	memset(stats, 0, sizeof(*stats));
 
@@ -1286,10 +1304,9 @@ rte_eth_stats_reset(uint8_t port_id)
 {
 	struct rte_eth_dev *dev;
 
-	if (port_id >= nb_ports) {
-		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
+	if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID)
 		return;
-	}
+
 	dev = &rte_eth_devices[port_id];
 
 	FUNC_PTR_OR_RET(*dev->dev_ops->stats_reset);
@@ -1307,10 +1324,9 @@ rte_eth_xstats_get(uint8_t port_id, struct rte_eth_xstats *xstats,
 	uint64_t val;
 	char *stats_ptr;
 
-	if (port_id >= nb_ports) {
-		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
+	if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID)
 		return -1;
-	}
+
 	dev = &rte_eth_devices[port_id];
 
 	/* implemented by the driver */
@@ -1376,10 +1392,9 @@ rte_eth_xstats_reset(uint8_t port_id)
 {
 	struct rte_eth_dev *dev;
 
-	if (port_id >= nb_ports) {
-		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
+	if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID)
 		return;
-	}
+
 	dev = &rte_eth_devices[port_id];
 
 	/* implemented by the driver */
@@ -1398,10 +1413,9 @@ set_queue_stats_mapping(uint8_t port_id, uint16_t queue_id, uint8_t stat_idx,
 {
 	struct rte_eth_dev *dev;
 
-	if (port_id >= nb_ports) {
-		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
+	if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID)
 		return -ENODEV;
-	}
+
 	dev = &rte_eth_devices[port_id];
 
 	FUNC_PTR_OR_ERR_RET(*dev->dev_ops->queue_stats_mapping_set, -ENOTSUP);
@@ -1433,10 +1447,9 @@ rte_eth_dev_info_get(uint8_t port_id, struct rte_eth_dev_info *dev_info)
 {
 	struct rte_eth_dev *dev;
 
-	if (port_id >= nb_ports) {
-		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
+	if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID)
 		return;
-	}
+
 	dev = &rte_eth_devices[port_id];
 
 	memset(dev_info, 0, sizeof(struct rte_eth_dev_info));
@@ -1453,10 +1466,9 @@ rte_eth_macaddr_get(uint8_t port_id, struct ether_addr *mac_addr)
 {
 	struct rte_eth_dev *dev;
 
-	if (port_id >= nb_ports) {
-		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
+	if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID)
 		return;
-	}
+
 	dev = &rte_eth_devices[port_id];
 	ether_addr_copy(&dev->data->mac_addrs[0], mac_addr);
 }
@@ -1467,10 +1479,8 @@ rte_eth_dev_get_mtu(uint8_t port_id, uint16_t *mtu)
 {
 	struct rte_eth_dev *dev;
 
-	if (port_id >= nb_ports) {
-		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
+	if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID)
 		return (-ENODEV);
-	}
 
 	dev = &rte_eth_devices[port_id];
 	*mtu = dev->data->mtu;
@@ -1483,10 +1493,8 @@ rte_eth_dev_set_mtu(uint8_t port_id, uint16_t mtu)
 	int ret;
 	struct rte_eth_dev *dev;
 
-	if (port_id >= nb_ports) {
-		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
+	if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID)
 		return (-ENODEV);
-	}
 
 	dev = &rte_eth_devices[port_id];
 	FUNC_PTR_OR_ERR_RET(*dev->dev_ops->mtu_set, -ENOTSUP);
@@ -1503,10 +1511,9 @@ rte_eth_dev_vlan_filter(uint8_t port_id, uint16_t vlan_id, int on)
 {
 	struct rte_eth_dev *dev;
 
-	if (port_id >= nb_ports) {
-		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
+	if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID)
 		return (-ENODEV);
-	}
+
 	dev = &rte_eth_devices[port_id];
 	if (! (dev->data->dev_conf.rxmode.hw_vlan_filter)) {
 		PMD_DEBUG_TRACE("port %d: vlan-filtering disabled\n", port_id);
@@ -1528,10 +1535,8 @@ rte_eth_dev_set_vlan_strip_on_queue(uint8_t port_id, uint16_t rx_queue_id, int o
 {
 	struct rte_eth_dev *dev;
 
-	if (port_id >= nb_ports) {
-		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
+	if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID)
 		return (-ENODEV);
-	}
 
 	dev = &rte_eth_devices[port_id];
 	if (rx_queue_id >= dev->data->nb_rx_queues) {
@@ -1550,10 +1555,8 @@ rte_eth_dev_set_vlan_ether_type(uint8_t port_id, uint16_t tpid)
 {
 	struct rte_eth_dev *dev;
 
-	if (port_id >= nb_ports) {
-		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
+	if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID)
 		return (-ENODEV);
-	}
 
 	dev = &rte_eth_devices[port_id];
 	FUNC_PTR_OR_ERR_RET(*dev->dev_ops->vlan_tpid_set, -ENOTSUP);
@@ -1570,10 +1573,8 @@ rte_eth_dev_set_vlan_offload(uint8_t port_id, int offload_mask)
 	int mask = 0;
 	int cur, org = 0;
 
-	if (port_id >= nb_ports) {
-		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
+	if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID)
 		return (-ENODEV);
-	}
 
 	dev = &rte_eth_devices[port_id];
 
@@ -1615,10 +1616,8 @@ rte_eth_dev_get_vlan_offload(uint8_t port_id)
 	struct rte_eth_dev *dev;
 	int ret = 0;
 
-	if (port_id >= nb_ports) {
-		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
+	if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID)
 		return (-ENODEV);
-	}
 
 	dev = &rte_eth_devices[port_id];
 
@@ -1639,10 +1638,9 @@ rte_eth_dev_set_vlan_pvid(uint8_t port_id, uint16_t pvid, int on)
 {
 	struct rte_eth_dev *dev;
 
-	if (port_id >= nb_ports) {
-		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
+	if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID)
 		return (-ENODEV);
-	}
+
 	dev = &rte_eth_devices[port_id];
 	FUNC_PTR_OR_ERR_RET(*dev->dev_ops->vlan_pvid_set, -ENOTSUP);
 	(*dev->dev_ops->vlan_pvid_set)(dev, pvid, on);
@@ -1657,10 +1655,8 @@ rte_eth_dev_fdir_add_signature_filter(uint8_t port_id,
 {
 	struct rte_eth_dev *dev;
 
-	if (port_id >= nb_ports) {
-		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
+	if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID)
 		return (-ENODEV);
-	}
 
 	dev = &rte_eth_devices[port_id];
 
@@ -1691,10 +1687,8 @@ rte_eth_dev_fdir_update_signature_filter(uint8_t port_id,
 {
 	struct rte_eth_dev *dev;
 
-	if (port_id >= nb_ports) {
-		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
+	if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID)
 		return (-ENODEV);
-	}
 
 	dev = &rte_eth_devices[port_id];
 
@@ -1725,10 +1719,8 @@ rte_eth_dev_fdir_remove_signature_filter(uint8_t port_id,
 {
 	struct rte_eth_dev *dev;
 
-	if (port_id >= nb_ports) {
-		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
+	if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID)
 		return (-ENODEV);
-	}
 
 	dev = &rte_eth_devices[port_id];
 
@@ -1756,10 +1748,8 @@ rte_eth_dev_fdir_get_infos(uint8_t port_id, struct rte_eth_fdir *fdir)
 {
 	struct rte_eth_dev *dev;
 
-	if (port_id >= nb_ports) {
-		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
+	if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID)
 		return (-ENODEV);
-	}
 
 	dev = &rte_eth_devices[port_id];
 	if (! (dev->data->dev_conf.fdir_conf.mode)) {
@@ -1781,10 +1771,8 @@ rte_eth_dev_fdir_add_perfect_filter(uint8_t port_id,
 {
 	struct rte_eth_dev *dev;
 
-	if (port_id >= nb_ports) {
-		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
+	if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID)
 		return (-ENODEV);
-	}
 
 	dev = &rte_eth_devices[port_id];
 
@@ -1821,10 +1809,8 @@ rte_eth_dev_fdir_update_perfect_filter(uint8_t port_id,
 {
 	struct rte_eth_dev *dev;
 
-	if (port_id >= nb_ports) {
-		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
+	if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID)
 		return (-ENODEV);
-	}
 
 	dev = &rte_eth_devices[port_id];
 
@@ -1859,10 +1845,8 @@ rte_eth_dev_fdir_remove_perfect_filter(uint8_t port_id,
 {
 	struct rte_eth_dev *dev;
 
-	if (port_id >= nb_ports) {
-		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
+	if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID)
 		return (-ENODEV);
-	}
 
 	dev = &rte_eth_devices[port_id];
 
@@ -1895,10 +1879,8 @@ rte_eth_dev_fdir_set_masks(uint8_t port_id, struct rte_fdir_masks *fdir_mask)
 {
 	struct rte_eth_dev *dev;
 
-	if (port_id >= nb_ports) {
-		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
+	if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID)
 		return (-ENODEV);
-	}
 
 	dev = &rte_eth_devices[port_id];
 	if (! (dev->data->dev_conf.fdir_conf.mode)) {
@@ -1915,10 +1897,8 @@ rte_eth_dev_flow_ctrl_get(uint8_t port_id, struct rte_eth_fc_conf *fc_conf)
 {
 	struct rte_eth_dev *dev;
 
-	if (port_id >= nb_ports) {
-		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
+	if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID)
 		return (-ENODEV);
-	}
 
 	dev = &rte_eth_devices[port_id];
 	FUNC_PTR_OR_ERR_RET(*dev->dev_ops->flow_ctrl_get, -ENOTSUP);
@@ -1931,10 +1911,8 @@ rte_eth_dev_flow_ctrl_set(uint8_t port_id, struct rte_eth_fc_conf *fc_conf)
 {
 	struct rte_eth_dev *dev;
 
-	if (port_id >= nb_ports) {
-		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
+	if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID)
 		return (-ENODEV);
-	}
 
 	if ((fc_conf->send_xon != 0) && (fc_conf->send_xon != 1)) {
 		PMD_DEBUG_TRACE("Invalid send_xon, only 0/1 allowed\n");
@@ -1951,10 +1929,8 @@ rte_eth_dev_priority_flow_ctrl_set(uint8_t port_id, struct rte_eth_pfc_conf *pfc
 {
 	struct rte_eth_dev *dev;
 
-	if (port_id >= nb_ports) {
-		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
+	if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID)
 		return (-ENODEV);
-	}
 
 	if (pfc_conf->priority > (ETH_DCB_NUM_USER_PRIORITIES - 1)) {
 		PMD_DEBUG_TRACE("Invalid priority, only 0-7 allowed\n");
@@ -2030,10 +2006,8 @@ rte_eth_dev_rss_reta_update(uint8_t port_id,
 	struct rte_eth_dev *dev;
 	int ret;
 
-	if (port_id >= nb_ports) {
-		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
+	if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID)
 		return -ENODEV;
-	}
 
 	/* Check mask bits */
 	ret = rte_eth_check_reta_mask(reta_conf, reta_size);
@@ -2081,10 +2055,9 @@ rte_eth_dev_rss_hash_update(uint8_t port_id, struct rte_eth_rss_conf *rss_conf)
 	struct rte_eth_dev *dev;
 	uint16_t rss_hash_protos;
 
-	if (port_id >= nb_ports) {
-		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
+	if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID)
 		return (-ENODEV);
-	}
+
 	rss_hash_protos = rss_conf->rss_hf;
 	if ((rss_hash_protos != 0) &&
 	    ((rss_hash_protos & ETH_RSS_PROTO_MASK) == 0)) {
@@ -2103,10 +2076,9 @@ rte_eth_dev_rss_hash_conf_get(uint8_t port_id,
 {
 	struct rte_eth_dev *dev;
 
-	if (port_id >= nb_ports) {
-		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
+	if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID)
 		return (-ENODEV);
-	}
+
 	dev = &rte_eth_devices[port_id];
 	FUNC_PTR_OR_ERR_RET(*dev->dev_ops->rss_hash_conf_get, -ENOTSUP);
 	return (*dev->dev_ops->rss_hash_conf_get)(dev, rss_conf);
@@ -2118,10 +2090,8 @@ rte_eth_dev_udp_tunnel_add(uint8_t port_id,
 {
 	struct rte_eth_dev *dev;
 
-	if (port_id >= nb_ports) {
-		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
+	if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID)
 		return -ENODEV;
-	}
 
 	if (udp_tunnel == NULL) {
 		PMD_DEBUG_TRACE("Invalid udp_tunnel parameter\n");
@@ -2144,10 +2114,9 @@ rte_eth_dev_udp_tunnel_delete(uint8_t port_id,
 {
 	struct rte_eth_dev *dev;
 
-	if (port_id >= nb_ports) {
-		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
+	if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID)
 		return -ENODEV;
-	}
+
 	dev = &rte_eth_devices[port_id];
 
 	if (udp_tunnel == NULL) {
@@ -2169,10 +2138,8 @@ rte_eth_led_on(uint8_t port_id)
 {
 	struct rte_eth_dev *dev;
 
-	if (port_id >= nb_ports) {
-		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
+	if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID)
 		return (-ENODEV);
-	}
 
 	dev = &rte_eth_devices[port_id];
 	FUNC_PTR_OR_ERR_RET(*dev->dev_ops->dev_led_on, -ENOTSUP);
@@ -2184,10 +2151,8 @@ rte_eth_led_off(uint8_t port_id)
 {
 	struct rte_eth_dev *dev;
 
-	if (port_id >= nb_ports) {
-		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
+	if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID)
 		return (-ENODEV);
-	}
 
 	dev = &rte_eth_devices[port_id];
 	FUNC_PTR_OR_ERR_RET(*dev->dev_ops->dev_led_off, -ENOTSUP);
@@ -2224,10 +2189,9 @@ rte_eth_dev_mac_addr_add(uint8_t port_id, struct ether_addr *addr,
 	int index;
 	uint64_t pool_mask;
 
-	if (port_id >= nb_ports) {
-		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
+	if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID)
 		return (-ENODEV);
-	}
+
 	dev = &rte_eth_devices[port_id];
 	FUNC_PTR_OR_ERR_RET(*dev->dev_ops->mac_addr_add, -ENOTSUP);
 
@@ -2275,10 +2239,9 @@ rte_eth_dev_mac_addr_remove(uint8_t port_id, struct ether_addr *addr)
 	struct rte_eth_dev *dev;
 	int index;
 
-	if (port_id >= nb_ports) {
-		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
+	if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID)
 		return (-ENODEV);
-	}
+
 	dev = &rte_eth_devices[port_id];
 	FUNC_PTR_OR_ERR_RET(*dev->dev_ops->mac_addr_remove, -ENOTSUP);
 
@@ -2309,7 +2272,7 @@ rte_eth_dev_set_vf_rxmode(uint8_t port_id,  uint16_t vf,
 	struct rte_eth_dev *dev;
 	struct rte_eth_dev_info dev_info;
 
-	if (port_id >= nb_ports) {
+	if (rte_eth_dev_validate_port(port_id, NONE_TRACE) == DEV_INVALID) {
 		PMD_DEBUG_TRACE("set VF RX mode:Invalid port_id=%d\n",
 				port_id);
 		return (-ENODEV);
@@ -2364,7 +2327,7 @@ rte_eth_dev_uc_hash_table_set(uint8_t port_id, struct ether_addr *addr,
 	int ret;
 	struct rte_eth_dev *dev;
 
-	if (port_id >= nb_ports) {
+	if (rte_eth_dev_validate_port(port_id, NONE_TRACE) == DEV_INVALID) {
 		PMD_DEBUG_TRACE("unicast hash setting:Invalid port_id=%d\n",
 			port_id);
 		return (-ENODEV);
@@ -2417,7 +2380,7 @@ rte_eth_dev_uc_all_hash_table_set(uint8_t port_id, uint8_t on)
 {
 	struct rte_eth_dev *dev;
 
-	if (port_id >= nb_ports) {
+	if (rte_eth_dev_validate_port(port_id, NONE_TRACE) == DEV_INVALID) {
 		PMD_DEBUG_TRACE("unicast hash setting:Invalid port_id=%d\n",
 			port_id);
 		return (-ENODEV);
@@ -2436,10 +2399,8 @@ rte_eth_dev_set_vf_rx(uint8_t port_id,uint16_t vf, uint8_t on)
 	struct rte_eth_dev *dev;
 	struct rte_eth_dev_info dev_info;
 
-	if (port_id >= nb_ports) {
-		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
+	if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID)
 		return (-ENODEV);
-	}
 
 	dev = &rte_eth_devices[port_id];
 	rte_eth_dev_info_get(port_id, &dev_info);
@@ -2462,7 +2423,7 @@ rte_eth_dev_set_vf_tx(uint8_t port_id,uint16_t vf, uint8_t on)
 	struct rte_eth_dev *dev;
 	struct rte_eth_dev_info dev_info;
 
-	if (port_id >= nb_ports) {
+	if (rte_eth_dev_validate_port(port_id, NONE_TRACE) == DEV_INVALID) {
 		PMD_DEBUG_TRACE("set pool tx:Invalid port_id=%d\n", port_id);
 		return (-ENODEV);
 	}
@@ -2487,7 +2448,7 @@ rte_eth_dev_set_vf_vlan_filter(uint8_t port_id, uint16_t vlan_id,
 {
 	struct rte_eth_dev *dev;
 
-	if (port_id >= nb_ports) {
+	if (rte_eth_dev_validate_port(port_id, NONE_TRACE) == DEV_INVALID) {
 		PMD_DEBUG_TRACE("VF VLAN filter:invalid port id=%d\n",
 				port_id);
 		return (-ENODEV);
@@ -2518,7 +2479,7 @@ int rte_eth_set_queue_rate_limit(uint8_t port_id, uint16_t queue_idx,
 	struct rte_eth_dev_info dev_info;
 	struct rte_eth_link link;
 
-	if (port_id >= nb_ports) {
+	if (rte_eth_dev_validate_port(port_id, NONE_TRACE) == DEV_INVALID) {
 		PMD_DEBUG_TRACE("set queue rate limit:invalid port id=%d\n",
 				port_id);
 		return -ENODEV;
@@ -2555,7 +2516,7 @@ int rte_eth_set_vf_rate_limit(uint8_t port_id, uint16_t vf, uint16_t tx_rate,
 	if (q_msk == 0)
 		return 0;
 
-	if (port_id >= nb_ports) {
+	if (rte_eth_dev_validate_port(port_id, NONE_TRACE) == DEV_INVALID) {
 		PMD_DEBUG_TRACE("set VF rate limit:invalid port id=%d\n",
 				port_id);
 		return -ENODEV;
@@ -2589,10 +2550,8 @@ rte_eth_mirror_rule_set(uint8_t port_id,
 {
 	struct rte_eth_dev *dev = &rte_eth_devices[port_id];
 
-	if (port_id >= nb_ports) {
-		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
+	if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID)
 		return (-ENODEV);
-	}
 
 	if (mirror_conf->rule_type_mask == 0) {
 		PMD_DEBUG_TRACE("mirror rule type can not be 0.\n");
@@ -2630,10 +2589,8 @@ rte_eth_mirror_rule_reset(uint8_t port_id, uint8_t rule_id)
 {
 	struct rte_eth_dev *dev = &rte_eth_devices[port_id];
 
-	if (port_id >= nb_ports) {
-		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
+	if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID)
 		return (-ENODEV);
-	}
 
 	if(rule_id >= ETH_VMDQ_NUM_MIRROR_RULE)
 	{
@@ -2655,10 +2612,9 @@ rte_eth_rx_burst(uint8_t port_id, uint16_t queue_id,
 {
 	struct rte_eth_dev *dev;
 
-	if (port_id >= nb_ports) {
-		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
+	if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID)
 		return 0;
-	}
+
 	dev = &rte_eth_devices[port_id];
 	FUNC_PTR_OR_ERR_RET(*dev->rx_pkt_burst, 0);
 	if (queue_id >= dev->data->nb_rx_queues) {
@@ -2675,10 +2631,9 @@ rte_eth_tx_burst(uint8_t port_id, uint16_t queue_id,
 {
 	struct rte_eth_dev *dev;
 
-	if (port_id >= nb_ports) {
-		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
+	if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID)
 		return 0;
-	}
+
 	dev = &rte_eth_devices[port_id];
 
 	FUNC_PTR_OR_ERR_RET(*dev->tx_pkt_burst, 0);
@@ -2695,10 +2650,9 @@ rte_eth_rx_queue_count(uint8_t port_id, uint16_t queue_id)
 {
 	struct rte_eth_dev *dev;
 
-	if (port_id >= nb_ports) {
-		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
+	if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID)
 		return 0;
-	}
+
 	dev = &rte_eth_devices[port_id];
 	FUNC_PTR_OR_ERR_RET(*dev->dev_ops->rx_queue_count, 0);
 	return (*dev->dev_ops->rx_queue_count)(dev, queue_id);
@@ -2709,10 +2663,9 @@ rte_eth_rx_descriptor_done(uint8_t port_id, uint16_t queue_id, uint16_t offset)
 {
 	struct rte_eth_dev *dev;
 
-	if (port_id >= nb_ports) {
-		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
+	if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID)
 		return (-ENODEV);
-	}
+
 	dev = &rte_eth_devices[port_id];
 	FUNC_PTR_OR_ERR_RET(*dev->dev_ops->rx_descriptor_done, -ENOTSUP);
 	return (*dev->dev_ops->rx_descriptor_done)( \
@@ -2730,10 +2683,8 @@ rte_eth_dev_callback_register(uint8_t port_id,
 
 	if (!cb_fn)
 		return (-EINVAL);
-	if (port_id >= nb_ports) {
-		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
+	if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID)
 		return (-EINVAL);
-	}
 
 	dev = &rte_eth_devices[port_id];
 	rte_spinlock_lock(&rte_eth_dev_cb_lock);
@@ -2770,10 +2721,8 @@ rte_eth_dev_callback_unregister(uint8_t port_id,
 
 	if (!cb_fn)
 		return (-EINVAL);
-	if (port_id >= nb_ports) {
-		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
+	if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID)
 		return (-EINVAL);
-	}
 
 	dev = &rte_eth_devices[port_id];
 	rte_spinlock_lock(&rte_eth_dev_cb_lock);
@@ -2830,10 +2779,8 @@ int rte_eth_dev_bypass_init(uint8_t port_id)
 {
 	struct rte_eth_dev *dev;
 
-	if (port_id >= nb_ports) {
-		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
+	if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID)
 		return (-ENODEV);
-	}
 
 	if ((dev= &rte_eth_devices[port_id]) == NULL) {
 		PMD_DEBUG_TRACE("Invalid port device\n");
@@ -2850,10 +2797,8 @@ rte_eth_dev_bypass_state_show(uint8_t port_id, uint32_t *state)
 {
 	struct rte_eth_dev *dev;
 
-	if (port_id >= nb_ports) {
-		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
+	if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID)
 		return (-ENODEV);
-	}
 
 	if ((dev= &rte_eth_devices[port_id]) == NULL) {
 		PMD_DEBUG_TRACE("Invalid port device\n");
@@ -2869,10 +2814,8 @@ rte_eth_dev_bypass_state_set(uint8_t port_id, uint32_t *new_state)
 {
 	struct rte_eth_dev *dev;
 
-	if (port_id >= nb_ports) {
-		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
+	if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID)
 		return (-ENODEV);
-	}
 
 	if ((dev= &rte_eth_devices[port_id]) == NULL) {
 		PMD_DEBUG_TRACE("Invalid port device\n");
@@ -2889,10 +2832,8 @@ rte_eth_dev_bypass_event_show(uint8_t port_id, uint32_t event, uint32_t *state)
 {
 	struct rte_eth_dev *dev;
 
-	if (port_id >= nb_ports) {
-		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
+	if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID)
 		return (-ENODEV);
-	}
 
 	if ((dev= &rte_eth_devices[port_id]) == NULL) {
 		PMD_DEBUG_TRACE("Invalid port device\n");
@@ -2909,10 +2850,8 @@ rte_eth_dev_bypass_event_store(uint8_t port_id, uint32_t event, uint32_t state)
 {
 	struct rte_eth_dev *dev;
 
-	if (port_id >= nb_ports) {
-		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
+	if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID)
 		return (-ENODEV);
-	}
 
 	if ((dev= &rte_eth_devices[port_id]) == NULL) {
 		PMD_DEBUG_TRACE("Invalid port device\n");
@@ -2929,10 +2868,8 @@ rte_eth_dev_wd_timeout_store(uint8_t port_id, uint32_t timeout)
 {
 	struct rte_eth_dev *dev;
 
-	if (port_id >= nb_ports) {
-		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
+	if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID)
 		return (-ENODEV);
-	}
 
 	if ((dev= &rte_eth_devices[port_id]) == NULL) {
 		PMD_DEBUG_TRACE("Invalid port device\n");
@@ -2949,10 +2886,8 @@ rte_eth_dev_bypass_ver_show(uint8_t port_id, uint32_t *ver)
 {
 	struct rte_eth_dev *dev;
 
-	if (port_id >= nb_ports) {
-		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
+	if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID)
 		return (-ENODEV);
-	}
 
 	if ((dev= &rte_eth_devices[port_id]) == NULL) {
 		PMD_DEBUG_TRACE("Invalid port device\n");
@@ -2969,10 +2904,8 @@ rte_eth_dev_bypass_wd_timeout_show(uint8_t port_id, uint32_t *wd_timeout)
 {
 	struct rte_eth_dev *dev;
 
-	if (port_id >= nb_ports) {
-		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
+	if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID)
 		return (-ENODEV);
-	}
 
 	if ((dev= &rte_eth_devices[port_id]) == NULL) {
 		PMD_DEBUG_TRACE("Invalid port device\n");
@@ -2989,10 +2922,8 @@ rte_eth_dev_bypass_wd_reset(uint8_t port_id)
 {
 	struct rte_eth_dev *dev;
 
-	if (port_id >= nb_ports) {
-		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
+	if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID)
 		return (-ENODEV);
-	}
 
 	if ((dev= &rte_eth_devices[port_id]) == NULL) {
 		PMD_DEBUG_TRACE("Invalid port device\n");
@@ -3011,10 +2942,8 @@ rte_eth_dev_add_syn_filter(uint8_t port_id,
 {
 	struct rte_eth_dev *dev;
 
-	if (port_id >= nb_ports) {
-		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
+	if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID)
 		return -ENODEV;
-	}
 
 	dev = &rte_eth_devices[port_id];
 	FUNC_PTR_OR_ERR_RET(*dev->dev_ops->add_syn_filter, -ENOTSUP);
@@ -3026,10 +2955,8 @@ rte_eth_dev_remove_syn_filter(uint8_t port_id)
 {
 	struct rte_eth_dev *dev;
 
-	if (port_id >= nb_ports) {
-		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
+	if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID)
 		return -ENODEV;
-	}
 
 	dev = &rte_eth_devices[port_id];
 	FUNC_PTR_OR_ERR_RET(*dev->dev_ops->remove_syn_filter, -ENOTSUP);
@@ -3045,10 +2972,8 @@ rte_eth_dev_get_syn_filter(uint8_t port_id,
 	if (filter == NULL || rx_queue == NULL)
 		return -EINVAL;
 
-	if (port_id >= nb_ports) {
-		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
+	if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID)
 		return -ENODEV;
-	}
 
 	dev = &rte_eth_devices[port_id];
 	FUNC_PTR_OR_ERR_RET(*dev->dev_ops->get_syn_filter, -ENOTSUP);
@@ -3061,10 +2986,9 @@ rte_eth_dev_add_2tuple_filter(uint8_t port_id, uint16_t index,
 {
 	struct rte_eth_dev *dev;
 
-	if (port_id >= nb_ports) {
-		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
+	if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID)
 		return -ENODEV;
-	}
+
 	if (filter->protocol != IPPROTO_TCP &&
 		filter->tcp_flags != 0){
 		PMD_DEBUG_TRACE("tcp flags is 0x%x, but the protocol value"
@@ -3083,10 +3007,8 @@ rte_eth_dev_remove_2tuple_filter(uint8_t port_id, uint16_t index)
 {
 	struct rte_eth_dev *dev;
 
-	if (port_id >= nb_ports) {
-		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
+	if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID)
 		return -ENODEV;
-	}
 
 	dev = &rte_eth_devices[port_id];
 	FUNC_PTR_OR_ERR_RET(*dev->dev_ops->remove_2tuple_filter, -ENOTSUP);
@@ -3102,10 +3024,8 @@ rte_eth_dev_get_2tuple_filter(uint8_t port_id, uint16_t index,
 	if (filter == NULL || rx_queue == NULL)
 		return -EINVAL;
 
-	if (port_id >= nb_ports) {
-		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
+	if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID)
 		return -ENODEV;
-	}
 
 	dev = &rte_eth_devices[port_id];
 	FUNC_PTR_OR_ERR_RET(*dev->dev_ops->get_2tuple_filter, -ENOTSUP);
@@ -3118,10 +3038,8 @@ rte_eth_dev_add_5tuple_filter(uint8_t port_id, uint16_t index,
 {
 	struct rte_eth_dev *dev;
 
-	if (port_id >= nb_ports) {
-		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
+	if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID)
 		return -ENODEV;
-	}
 
 	if (filter->protocol != IPPROTO_TCP &&
 		filter->tcp_flags != 0){
@@ -3141,10 +3059,8 @@ rte_eth_dev_remove_5tuple_filter(uint8_t port_id, uint16_t index)
 {
 	struct rte_eth_dev *dev;
 
-	if (port_id >= nb_ports) {
-		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
+	if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID)
 		return -ENODEV;
-	}
 
 	dev = &rte_eth_devices[port_id];
 	FUNC_PTR_OR_ERR_RET(*dev->dev_ops->remove_5tuple_filter, -ENOTSUP);
@@ -3160,10 +3076,8 @@ rte_eth_dev_get_5tuple_filter(uint8_t port_id, uint16_t index,
 	if (filter == NULL || rx_queue == NULL)
 		return -EINVAL;
 
-	if (port_id >= nb_ports) {
-		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
+	if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID)
 		return -ENODEV;
-	}
 
 	dev = &rte_eth_devices[port_id];
 	FUNC_PTR_OR_ERR_RET(*dev->dev_ops->get_5tuple_filter, -ENOTSUP);
@@ -3177,10 +3091,8 @@ rte_eth_dev_add_flex_filter(uint8_t port_id, uint16_t index,
 {
 	struct rte_eth_dev *dev;
 
-	if (port_id >= nb_ports) {
-		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
+	if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID)
 		return -ENODEV;
-	}
 
 	dev = &rte_eth_devices[port_id];
 	FUNC_PTR_OR_ERR_RET(*dev->dev_ops->add_flex_filter, -ENOTSUP);
@@ -3192,10 +3104,8 @@ rte_eth_dev_remove_flex_filter(uint8_t port_id, uint16_t index)
 {
 	struct rte_eth_dev *dev;
 
-	if (port_id >= nb_ports) {
-		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
+	if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID)
 		return -ENODEV;
-	}
 
 	dev = &rte_eth_devices[port_id];
 	FUNC_PTR_OR_ERR_RET(*dev->dev_ops->remove_flex_filter, -ENOTSUP);
@@ -3211,10 +3121,8 @@ rte_eth_dev_get_flex_filter(uint8_t port_id, uint16_t index,
 	if (filter == NULL || rx_queue == NULL)
 		return -EINVAL;
 
-	if (port_id >= nb_ports) {
-		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
+	if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID)
 		return -ENODEV;
-	}
 
 	dev = &rte_eth_devices[port_id];
 	FUNC_PTR_OR_ERR_RET(*dev->dev_ops->get_flex_filter, -ENOTSUP);
@@ -3227,10 +3135,8 @@ rte_eth_dev_filter_supported(uint8_t port_id, enum rte_filter_type filter_type)
 {
 	struct rte_eth_dev *dev;
 
-	if (port_id >= nb_ports) {
-		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
+	if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID)
 		return -ENODEV;
-	}
 
 	dev = &rte_eth_devices[port_id];
 	FUNC_PTR_OR_ERR_RET(*dev->dev_ops->filter_ctrl, -ENOTSUP);
@@ -3244,10 +3150,8 @@ rte_eth_dev_filter_ctrl(uint8_t port_id, enum rte_filter_type filter_type,
 {
 	struct rte_eth_dev *dev;
 
-	if (port_id >= nb_ports) {
-		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
+	if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID)
 		return -ENODEV;
-	}
 
 	dev = &rte_eth_devices[port_id];
 	FUNC_PTR_OR_ERR_RET(*dev->dev_ops->filter_ctrl, -ENOTSUP);
diff --git a/lib/librte_ether/rte_ethdev.h b/lib/librte_ether/rte_ethdev.h
index 1200c1c..ca101f5 100644
--- a/lib/librte_ether/rte_ethdev.h
+++ b/lib/librte_ether/rte_ethdev.h
@@ -1539,6 +1539,7 @@ struct rte_eth_dev {
 	struct eth_dev_ops *dev_ops;    /**< Functions exported by PMD */
 	struct rte_pci_device *pci_dev; /**< PCI info. supplied by probing */
 	struct rte_eth_dev_cb_list callbacks; /**< User application callbacks */
+	uint8_t attached; /**< Flag indicating the port is attached */
 };
 
 struct rte_eth_dev_sriov {
@@ -1604,6 +1605,10 @@ extern struct rte_eth_dev rte_eth_devices[];
  * initialized by the [matching] Ethernet driver during the PCI probing phase.
  * All devices whose port identifier is in the range
  * [0,  rte_eth_dev_count() - 1] can be operated on by network applications.
+ * immediately after invoking rte_eal_init().
+ * If the application unplugs a port using hotplug function, The enabled port
+ * numbers may be noncontiguous. In the case, the applications need to manage
+ * enabled port by themselves.
  *
  * @return
  *   - The total number of usable Ethernet devices.
-- 
1.9.1

^ permalink raw reply	[flat|nested] 362+ messages in thread

* [dpdk-dev] [PATCH v5 04/13] eal/pci: Consolidate pci address comparison APIs
  2015-01-30  5:42     ` [dpdk-dev] [PATCH v5 00/13] Port Hotplug Framework Tetsuya Mukawa
                         ` (2 preceding siblings ...)
  2015-01-30  5:42       ` [dpdk-dev] [PATCH v5 03/13] eal/pci, ethdev: Remove assumption that port will not be detached Tetsuya Mukawa
@ 2015-01-30  5:42       ` Tetsuya Mukawa
  2015-01-30  5:42       ` [dpdk-dev] [PATCH v5 05/13] ethdev: Add rte_eth_dev_free to free specified device Tetsuya Mukawa
                         ` (10 subsequent siblings)
  14 siblings, 0 replies; 362+ messages in thread
From: Tetsuya Mukawa @ 2015-01-30  5:42 UTC (permalink / raw)
  To: dev

This patch replaces pci_addr_comparison() and memcmp() of pci addresses by
eal_compare_pci_addr().

v5:
- Fix pci_scan_one to handle pt_driver correctly.
v4:
- Fix calculation method of eal_compare_pci_addr().
- Add parameter checking.

Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp>
---
 lib/librte_eal/bsdapp/eal/eal_pci.c       | 25 ++++++++---------------
 lib/librte_eal/common/eal_common_pci.c    |  2 +-
 lib/librte_eal/common/include/rte_pci.h   | 34 +++++++++++++++++++++++++++++++
 lib/librte_eal/linuxapp/eal/eal_pci.c     | 25 ++++++++---------------
 lib/librte_eal/linuxapp/eal/eal_pci_uio.c |  2 +-
 5 files changed, 54 insertions(+), 34 deletions(-)

diff --git a/lib/librte_eal/bsdapp/eal/eal_pci.c b/lib/librte_eal/bsdapp/eal/eal_pci.c
index 74ecce7..c844d58 100644
--- a/lib/librte_eal/bsdapp/eal/eal_pci.c
+++ b/lib/librte_eal/bsdapp/eal/eal_pci.c
@@ -270,20 +270,6 @@ pci_uio_map_resource(struct rte_pci_device *dev)
 	return (0);
 }
 
-/* Compare two PCI device addresses. */
-static int
-pci_addr_comparison(struct rte_pci_addr *addr, struct rte_pci_addr *addr2)
-{
-	uint64_t dev_addr = (addr->domain << 24) + (addr->bus << 16) + (addr->devid << 8) + addr->function;
-	uint64_t dev_addr2 = (addr2->domain << 24) + (addr2->bus << 16) + (addr2->devid << 8) + addr2->function;
-
-	if (dev_addr > dev_addr2)
-		return 1;
-	else
-		return 0;
-}
-
-
 /* Scan one pci sysfs entry, and fill the devices list from it. */
 static int
 pci_scan_one(int dev_pci_fd, struct pci_conf *conf)
@@ -356,13 +342,20 @@ pci_scan_one(int dev_pci_fd, struct pci_conf *conf)
 	}
 	else {
 		struct rte_pci_device *dev2 = NULL;
+		int ret;
 
 		TAILQ_FOREACH(dev2, &pci_device_list, next) {
-			if (pci_addr_comparison(&dev->addr, &dev2->addr))
+			ret = eal_compare_pci_addr(&dev->addr, &dev2->addr);
+			if (ret > 0)
 				continue;
-			else {
+			else if (ret < 0) {
 				TAILQ_INSERT_BEFORE(dev2, dev, next);
 				return 0;
+			} else { /* already registered */
+				/* update pt_driver */
+				dev2->pt_driver = dev->pt_driver;
+				free(dev);
+				return 0;
 			}
 		}
 		TAILQ_INSERT_TAIL(&pci_device_list, dev, next);
diff --git a/lib/librte_eal/common/eal_common_pci.c b/lib/librte_eal/common/eal_common_pci.c
index f3c7f71..a89f5c3 100644
--- a/lib/librte_eal/common/eal_common_pci.c
+++ b/lib/librte_eal/common/eal_common_pci.c
@@ -93,7 +93,7 @@ static struct rte_devargs *pci_devargs_lookup(struct rte_pci_device *dev)
 		if (devargs->type != RTE_DEVTYPE_BLACKLISTED_PCI &&
 			devargs->type != RTE_DEVTYPE_WHITELISTED_PCI)
 			continue;
-		if (!memcmp(&dev->addr, &devargs->pci.addr, sizeof(dev->addr)))
+		if (!eal_compare_pci_addr(&dev->addr, &devargs->pci.addr))
 			return devargs;
 	}
 	return NULL;
diff --git a/lib/librte_eal/common/include/rte_pci.h b/lib/librte_eal/common/include/rte_pci.h
index 7f2d699..4814cd7 100644
--- a/lib/librte_eal/common/include/rte_pci.h
+++ b/lib/librte_eal/common/include/rte_pci.h
@@ -269,6 +269,40 @@ eal_parse_pci_DomBDF(const char *input, struct rte_pci_addr *dev_addr)
 }
 #undef GET_PCIADDR_FIELD
 
+/* Compare two PCI device addresses. */
+/**
+ * Utility function to compare two PCI device addresses.
+ *
+ * @param addr
+ *	The PCI Bus-Device-Function address to compare
+ * @param addr2
+ *	The PCI Bus-Device-Function address to compare
+ * @return
+ *	0 on equal PCI address.
+ *	Positive on addr is greater than addr2.
+ *	Negative on addr is less than addr2, or error.
+ */
+static inline int
+eal_compare_pci_addr(struct rte_pci_addr *addr, struct rte_pci_addr *addr2)
+{
+	uint64_t dev_addr, dev_addr2;
+
+	if ((addr == NULL) || (addr2 == NULL))
+		return -1;
+
+	dev_addr = (addr->domain << 24) | (addr->bus << 16) |
+				(addr->devid << 8) | addr->function;
+	dev_addr2 = (addr2->domain << 24) | (addr2->bus << 16) |
+				(addr2->devid << 8) | addr2->function;
+
+	if (dev_addr > dev_addr2)
+		return 1;
+	else if (dev_addr < dev_addr2)
+		return -1;
+	else
+		return 0;
+}
+
 /**
  * Probe the PCI bus for registered drivers.
  *
diff --git a/lib/librte_eal/linuxapp/eal/eal_pci.c b/lib/librte_eal/linuxapp/eal/eal_pci.c
index c0ca5a5..d847102 100644
--- a/lib/librte_eal/linuxapp/eal/eal_pci.c
+++ b/lib/librte_eal/linuxapp/eal/eal_pci.c
@@ -229,20 +229,6 @@ error:
 	return -1;
 }
 
-/* Compare two PCI device addresses. */
-static int
-pci_addr_comparison(struct rte_pci_addr *addr, struct rte_pci_addr *addr2)
-{
-	uint64_t dev_addr = (addr->domain << 24) + (addr->bus << 16) + (addr->devid << 8) + addr->function;
-	uint64_t dev_addr2 = (addr2->domain << 24) + (addr2->bus << 16) + (addr2->devid << 8) + addr2->function;
-
-	if (dev_addr > dev_addr2)
-		return 1;
-	else
-		return 0;
-}
-
-
 /* Scan one pci sysfs entry, and fill the devices list from it. */
 static int
 pci_scan_one(const char *dirname, uint16_t domain, uint8_t bus,
@@ -353,13 +339,20 @@ pci_scan_one(const char *dirname, uint16_t domain, uint8_t bus,
 	}
 	else {
 		struct rte_pci_device *dev2 = NULL;
+		int ret;
 
 		TAILQ_FOREACH(dev2, &pci_device_list, next) {
-			if (pci_addr_comparison(&dev->addr, &dev2->addr))
+			ret = eal_compare_pci_addr(&dev->addr, &dev2->addr);
+			if (ret > 0)
 				continue;
-			else {
+			else if (ret < 0) {
 				TAILQ_INSERT_BEFORE(dev2, dev, next);
 				return 0;
+			} else { /* already registered */
+				/* update pt_driver */
+				dev2->pt_driver = dev->pt_driver;
+				free(dev);
+				return 0;
 			}
 		}
 		TAILQ_INSERT_TAIL(&pci_device_list, dev, next);
diff --git a/lib/librte_eal/linuxapp/eal/eal_pci_uio.c b/lib/librte_eal/linuxapp/eal/eal_pci_uio.c
index e53f06b..1da3507 100644
--- a/lib/librte_eal/linuxapp/eal/eal_pci_uio.c
+++ b/lib/librte_eal/linuxapp/eal/eal_pci_uio.c
@@ -123,7 +123,7 @@ pci_uio_map_secondary(struct rte_pci_device *dev)
 	TAILQ_FOREACH(uio_res, pci_res_list, next) {
 
 		/* skip this element if it doesn't match our PCI address */
-		if (memcmp(&uio_res->pci_addr, &dev->addr, sizeof(dev->addr)))
+		if (eal_compare_pci_addr(&uio_res->pci_addr, &dev->addr))
 			continue;
 
 		for (i = 0; i != uio_res->nb_maps; i++) {
-- 
1.9.1

^ permalink raw reply	[flat|nested] 362+ messages in thread

* [dpdk-dev] [PATCH v5 05/13] ethdev: Add rte_eth_dev_free to free specified device
  2015-01-30  5:42     ` [dpdk-dev] [PATCH v5 00/13] Port Hotplug Framework Tetsuya Mukawa
                         ` (3 preceding siblings ...)
  2015-01-30  5:42       ` [dpdk-dev] [PATCH v5 04/13] eal/pci: Consolidate pci address comparison APIs Tetsuya Mukawa
@ 2015-01-30  5:42       ` Tetsuya Mukawa
  2015-01-30  5:42       ` [dpdk-dev] [PATCH v5 06/13] eal, ethdev: Add a function and function pointers to close ether device Tetsuya Mukawa
                         ` (9 subsequent siblings)
  14 siblings, 0 replies; 362+ messages in thread
From: Tetsuya Mukawa @ 2015-01-30  5:42 UTC (permalink / raw)
  To: dev

This patch adds rte_eth_dev_free(). The function is used for changing a
attached status of the device that has specified name.

v4:
- Add paramerter checking.

Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp>
---
 lib/librte_ether/rte_ethdev.c | 20 ++++++++++++++++++++
 lib/librte_ether/rte_ethdev.h | 11 +++++++++++
 2 files changed, 31 insertions(+)

diff --git a/lib/librte_ether/rte_ethdev.c b/lib/librte_ether/rte_ethdev.c
index d70854f..0f3094f 100644
--- a/lib/librte_ether/rte_ethdev.c
+++ b/lib/librte_ether/rte_ethdev.c
@@ -260,6 +260,26 @@ rte_eth_dev_allocate(const char *name)
 	return eth_dev;
 }
 
+struct rte_eth_dev *
+rte_eth_dev_free(const char *name)
+{
+	struct rte_eth_dev *eth_dev;
+
+	if (name == NULL)
+		return NULL;
+
+	eth_dev = rte_eth_dev_allocated(name);
+	if (eth_dev == NULL) {
+		PMD_DEBUG_TRACE("Ethernet Device with name %s doesn't exist!\n",
+				name);
+		return NULL;
+	}
+
+	eth_dev->attached = 0;
+	nb_ports--;
+	return eth_dev;
+}
+
 static int
 rte_eth_dev_init(struct rte_pci_driver *pci_drv,
 		 struct rte_pci_device *pci_dev)
diff --git a/lib/librte_ether/rte_ethdev.h b/lib/librte_ether/rte_ethdev.h
index ca101f5..6add058 100644
--- a/lib/librte_ether/rte_ethdev.h
+++ b/lib/librte_ether/rte_ethdev.h
@@ -1627,6 +1627,17 @@ extern uint8_t rte_eth_dev_count(void);
  */
 struct rte_eth_dev *rte_eth_dev_allocate(const char *name);
 
+/**
+ * Function for internal use by dummy drivers primarily, e.g. ring-based
+ * driver.
+ * Free the specified ethdev and returns the pointer to that slot.
+ *
+ * @param	name	Unique identifier name for each Ethernet device
+ * @return
+ *   - Slot in the rte_dev_devices array for the freed device;
+ */
+struct rte_eth_dev *rte_eth_dev_free(const char *name);
+
 struct eth_driver;
 /**
  * @internal
-- 
1.9.1

^ permalink raw reply	[flat|nested] 362+ messages in thread

* [dpdk-dev] [PATCH v5 06/13] eal, ethdev: Add a function and function pointers to close ether device
  2015-01-30  5:42     ` [dpdk-dev] [PATCH v5 00/13] Port Hotplug Framework Tetsuya Mukawa
                         ` (4 preceding siblings ...)
  2015-01-30  5:42       ` [dpdk-dev] [PATCH v5 05/13] ethdev: Add rte_eth_dev_free to free specified device Tetsuya Mukawa
@ 2015-01-30  5:42       ` Tetsuya Mukawa
  2015-01-30  5:42       ` [dpdk-dev] [PATCH v5 07/13] ethdev: Add functions that will be used by port hotplug functions Tetsuya Mukawa
                         ` (8 subsequent siblings)
  14 siblings, 0 replies; 362+ messages in thread
From: Tetsuya Mukawa @ 2015-01-30  5:42 UTC (permalink / raw)
  To: dev

The patch adds function pointer to rte_pci_driver and eth_driver
structure. These function pointers are used when ports are detached.
Also the patch adds rte_eth_dev_uninit(). So far, it's not called
by anywhere, but it will be called when port hotplug function is
implemented.

v4:
- Add paramerter checking.
- Change function names.

Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp>
---
 lib/librte_eal/common/include/rte_pci.h |  7 ++++++
 lib/librte_ether/rte_ethdev.c           | 40 +++++++++++++++++++++++++++++++++
 lib/librte_ether/rte_ethdev.h           | 24 ++++++++++++++++++++
 3 files changed, 71 insertions(+)

diff --git a/lib/librte_eal/common/include/rte_pci.h b/lib/librte_eal/common/include/rte_pci.h
index 4814cd7..87ca4cf 100644
--- a/lib/librte_eal/common/include/rte_pci.h
+++ b/lib/librte_eal/common/include/rte_pci.h
@@ -189,12 +189,19 @@ struct rte_pci_driver;
 typedef int (pci_devinit_t)(struct rte_pci_driver *, struct rte_pci_device *);
 
 /**
+ * Uninitialisation function for the driver called during hotplugging.
+ */
+typedef int (pci_devuninit_t)(
+		struct rte_pci_driver *, struct rte_pci_device *);
+
+/**
  * A structure describing a PCI driver.
  */
 struct rte_pci_driver {
 	TAILQ_ENTRY(rte_pci_driver) next;       /**< Next in list. */
 	const char *name;                       /**< Driver name. */
 	pci_devinit_t *devinit;                 /**< Device init. function. */
+	pci_devuninit_t *devuninit;             /**< Device uninit function. */
 	struct rte_pci_id *id_table;            /**< ID table, NULL terminated. */
 	uint32_t drv_flags;                     /**< Flags contolling handling of device. */
 };
diff --git a/lib/librte_ether/rte_ethdev.c b/lib/librte_ether/rte_ethdev.c
index 0f3094f..fd19140 100644
--- a/lib/librte_ether/rte_ethdev.c
+++ b/lib/librte_ether/rte_ethdev.c
@@ -335,6 +335,45 @@ rte_eth_dev_init(struct rte_pci_driver *pci_drv,
 	return diag;
 }
 
+static int
+rte_eth_dev_uninit(struct rte_pci_driver *pci_drv,
+		 struct rte_pci_device *pci_dev)
+{
+	struct eth_driver *eth_drv;
+	struct rte_eth_dev *eth_dev;
+	char ethdev_name[RTE_ETH_NAME_MAX_LEN];
+
+	if ((pci_drv == NULL) || (pci_dev == NULL))
+		return -EINVAL;
+
+	/* Create unique Ethernet device name using PCI address */
+	snprintf(ethdev_name, RTE_ETH_NAME_MAX_LEN, "%d:%d.%d",
+			pci_dev->addr.bus, pci_dev->addr.devid,
+			pci_dev->addr.function);
+
+	eth_dev = rte_eth_dev_free(ethdev_name);
+	if (eth_dev == NULL)
+		return -ENODEV;
+
+	eth_drv = (struct eth_driver *)pci_drv;
+
+	/* Invoke PMD device uninit function */
+	if (*eth_drv->eth_dev_uninit)
+		(*eth_drv->eth_dev_uninit)(eth_drv, eth_dev);
+
+	/* init user callbacks */
+	TAILQ_INIT(&(eth_dev->callbacks));
+
+	if (rte_eal_process_type() == RTE_PROC_PRIMARY)
+		rte_free(eth_dev->data->dev_private);
+
+	eth_dev->pci_dev = NULL;
+	eth_dev->driver = NULL;
+	eth_dev->data = NULL;
+
+	return 0;
+}
+
 /**
  * Register an Ethernet [Poll Mode] driver.
  *
@@ -353,6 +392,7 @@ void
 rte_eth_driver_register(struct eth_driver *eth_drv)
 {
 	eth_drv->pci_drv.devinit = rte_eth_dev_init;
+	eth_drv->pci_drv.devuninit = rte_eth_dev_uninit;
 	rte_eal_pci_register(&eth_drv->pci_drv);
 }
 
diff --git a/lib/librte_ether/rte_ethdev.h b/lib/librte_ether/rte_ethdev.h
index 6add058..0b4c27c 100644
--- a/lib/librte_ether/rte_ethdev.h
+++ b/lib/librte_ether/rte_ethdev.h
@@ -1675,6 +1675,27 @@ typedef int (*eth_dev_init_t)(struct eth_driver  *eth_drv,
 
 /**
  * @internal
+ * Finalization function of an Ethernet driver invoked for each matching
+ * Ethernet PCI device detected during the PCI closing phase.
+ *
+ * @param eth_drv
+ *   The pointer to the [matching] Ethernet driver structure supplied by
+ *   the PMD when it registered itself.
+ * @param eth_dev
+ *   The *eth_dev* pointer is the address of the *rte_eth_dev* structure
+ *   associated with the matching device and which have been [automatically]
+ *   allocated in the *rte_eth_devices* array.
+ * @return
+ *   - 0: Success, the device is properly finalized by the driver.
+ *        In particular, the driver MUST free the *dev_ops* pointer
+ *        of the *eth_dev* structure.
+ *   - <0: Error code of the device initialization failure.
+ */
+typedef int (*eth_dev_uninit_t)(struct eth_driver  *eth_drv,
+				  struct rte_eth_dev *eth_dev);
+
+/**
+ * @internal
  * The structure associated with a PMD Ethernet driver.
  *
  * Each Ethernet driver acts as a PCI driver and is represented by a generic
@@ -1684,11 +1705,14 @@ typedef int (*eth_dev_init_t)(struct eth_driver  *eth_drv,
  *
  * - The *eth_dev_init* function invoked for each matching PCI device.
  *
+ * - The *eth_dev_uninit* function invoked for each matching PCI device.
+ *
  * - The size of the private data to allocate for each matching device.
  */
 struct eth_driver {
 	struct rte_pci_driver pci_drv;    /**< The PMD is also a PCI driver. */
 	eth_dev_init_t eth_dev_init;      /**< Device init function. */
+	eth_dev_uninit_t eth_dev_uninit;  /**< Device uninit function. */
 	unsigned int dev_private_size;    /**< Size of device private data. */
 };
 
-- 
1.9.1

^ permalink raw reply	[flat|nested] 362+ messages in thread

* [dpdk-dev] [PATCH v5 07/13] ethdev: Add functions that will be used by port hotplug functions
  2015-01-30  5:42     ` [dpdk-dev] [PATCH v5 00/13] Port Hotplug Framework Tetsuya Mukawa
                         ` (5 preceding siblings ...)
  2015-01-30  5:42       ` [dpdk-dev] [PATCH v5 06/13] eal, ethdev: Add a function and function pointers to close ether device Tetsuya Mukawa
@ 2015-01-30  5:42       ` Tetsuya Mukawa
  2015-01-30  5:42       ` [dpdk-dev] [PATCH v5 08/13] eal/linux/pci: Add functions for unmapping igb_uio resources Tetsuya Mukawa
                         ` (7 subsequent siblings)
  14 siblings, 0 replies; 362+ messages in thread
From: Tetsuya Mukawa @ 2015-01-30  5:42 UTC (permalink / raw)
  To: dev

The patch adds following functions.

- rte_eth_dev_save()
  The function is used for saving current rte_eth_dev structures.
- rte_eth_dev_get_changed_port()
  The function receives the rte_eth_dev structures, then compare
  these with current values to know which port is actually
  attached or detached.
- rte_eth_dev_get_addr_by_port()
  The function returns a pci address of a ethdev specified by port
  identifier.
- rte_eth_dev_get_port_by_addr()
  The function returns a port identifier of a ethdev specified by
  pci address.
- rte_eth_dev_get_name_by_port()
  The function returns a unique identifier name of a ethdev
  specified by port identifier.
- Add rte_eth_dev_check_detachable()
  The function returns whether a PMD supports detach function.

Also the patch changes scope of rte_eth_dev_allocated() to global.
This function will be called by virtual PMDs to support port hotplug.
So change scope of the function to global.

v5:
- Fix return value of below functions.
  rte_eth_dev_get_changed_port().
  rte_eth_dev_get_port_by_addr().
v4:
- Add paramerter checking.
v3:
- Fix if-condition bug while comparing pci addresses.
- Add error checking codes.
Reported-by: Mark Enright <menrigh@brocade.com>

Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp>
---
 lib/librte_ether/rte_ethdev.c | 98 ++++++++++++++++++++++++++++++++++++++++++-
 lib/librte_ether/rte_ethdev.h | 80 +++++++++++++++++++++++++++++++++++
 2 files changed, 177 insertions(+), 1 deletion(-)

diff --git a/lib/librte_ether/rte_ethdev.c b/lib/librte_ether/rte_ethdev.c
index fd19140..2acafc7 100644
--- a/lib/librte_ether/rte_ethdev.c
+++ b/lib/librte_ether/rte_ethdev.c
@@ -206,7 +206,7 @@ rte_eth_dev_data_alloc(void)
 				RTE_MAX_ETHPORTS * sizeof(*rte_eth_dev_data));
 }
 
-static struct rte_eth_dev *
+struct rte_eth_dev *
 rte_eth_dev_allocated(const char *name)
 {
 	unsigned i;
@@ -428,6 +428,102 @@ rte_eth_dev_count(void)
 	return (nb_ports);
 }
 
+void
+rte_eth_dev_save(struct rte_eth_dev *devs)
+{
+	if (devs == NULL)
+		return;
+
+	/* save current rte_eth_devices */
+	memcpy(devs, rte_eth_devices,
+			sizeof(struct rte_eth_dev) * RTE_MAX_ETHPORTS);
+}
+
+int
+rte_eth_dev_get_changed_port(struct rte_eth_dev *devs, uint8_t *port_id)
+{
+	if ((devs == NULL) || (port_id == NULL))
+		return -EINVAL;
+
+	/* check which port was attached or detached */
+	for (*port_id = 0; *port_id < RTE_MAX_ETHPORTS; (*port_id)++, devs++) {
+		if (rte_eth_devices[*port_id].attached ^ devs->attached)
+			return 0;
+	}
+	return -ENODEV;
+}
+
+int
+rte_eth_dev_get_addr_by_port(uint8_t port_id, struct rte_pci_addr *addr)
+{
+	if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID)
+		return -EINVAL;
+
+	if (addr == NULL) {
+		PMD_DEBUG_TRACE("Null pointer is specified\n");
+		return -EINVAL;
+	}
+
+	*addr = rte_eth_devices[port_id].pci_dev->addr;
+	return 0;
+}
+
+int
+rte_eth_dev_get_port_by_addr(struct rte_pci_addr *addr, uint8_t *port_id)
+{
+	struct rte_pci_addr *tmp;
+
+	if ((addr == NULL) || (port_id == NULL)) {
+		PMD_DEBUG_TRACE("Null pointer is specified\n");
+		return -EINVAL;
+	}
+
+	for (*port_id = 0; *port_id < RTE_MAX_ETHPORTS; (*port_id)++) {
+		if (!rte_eth_devices[*port_id].attached)
+			continue;
+		if (!rte_eth_devices[*port_id].pci_dev)
+			continue;
+		tmp = &rte_eth_devices[*port_id].pci_dev->addr;
+		if (eal_compare_pci_addr(tmp, addr) == 0)
+			return 0;
+	}
+	return -ENODEV;
+}
+
+int
+rte_eth_dev_get_name_by_port(uint8_t port_id, char *name)
+{
+	char *tmp;
+
+	if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID)
+		return -EINVAL;
+
+	if (name == NULL) {
+		PMD_DEBUG_TRACE("Null pointer is specified\n");
+		return -EINVAL;
+	}
+
+	/* shouldn't check 'rte_eth_devices[i].data',
+	 * because it might be overwritten by VDEV PMD */
+	tmp = rte_eth_dev_data[port_id].name;
+	strncpy(name, tmp, strlen(tmp) + 1);
+	return 0;
+}
+
+int
+rte_eth_dev_check_detachable(uint8_t port_id)
+{
+	uint32_t drv_flags;
+
+	if (port_id >= RTE_MAX_ETHPORTS) {
+		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
+		return -EINVAL;
+	}
+
+	drv_flags = rte_eth_devices[port_id].driver->pci_drv.drv_flags;
+	return !(drv_flags & RTE_PCI_DRV_DETACHABLE);
+}
+
 static int
 rte_eth_dev_rx_queue_config(struct rte_eth_dev *dev, uint16_t nb_queues)
 {
diff --git a/lib/librte_ether/rte_ethdev.h b/lib/librte_ether/rte_ethdev.h
index 0b4c27c..ebc48b0 100644
--- a/lib/librte_ether/rte_ethdev.h
+++ b/lib/librte_ether/rte_ethdev.h
@@ -1616,6 +1616,86 @@ extern struct rte_eth_dev rte_eth_devices[];
 extern uint8_t rte_eth_dev_count(void);
 
 /**
+ * Function for internal use by port hotplug functions.
+ * Copies current ethdev structures to the specified pointer.
+ *
+ * @param	devs	The pointer to the ethdev structures
+ */
+extern void rte_eth_dev_save(struct rte_eth_dev *devs);
+
+/**
+ * Function for internal use by port hotplug functions.
+ * Compare the specified ethdev structures with currents. Then
+ * if there is a port which status is changed, fill the specified pointer
+ * with the port id of that port.
+ * @param	devs	The pointer to the ethdev structures
+ * @param	port_id	The pointer to the port id
+ * @return
+ *   - 0 on success, negative on error
+ */
+extern int rte_eth_dev_get_changed_port(
+		struct rte_eth_dev *devs, uint8_t *port_id);
+
+/**
+ * Function for internal use by port hotplug functions.
+ * Returns a pci address of a ethdev specified by port identifier.
+ * @param	port_id
+ *   The port identifier of the Ethernet device
+ * @param	addr
+ *   The pointer to the pci address
+ * @return
+ *   - 0 on success, negative on error
+ */
+extern int rte_eth_dev_get_addr_by_port(
+		uint8_t port_id, struct rte_pci_addr *addr);
+
+/**
+ * Function for internal use by port hotplug functions.
+ * Returns a port identifier of a ethdev specified by pci address.
+ * @param	addr
+ *   The pointer to the pci address of the Ethernet device.
+ * @param	port_id
+ *   The pointer to the port identifier
+ * @return
+ *   - 0 on success, negative on error
+ */
+extern int rte_eth_dev_get_port_by_addr(
+		struct rte_pci_addr *addr, uint8_t *port_id);
+
+/**
+ * Function for internal use by port hotplug functions.
+ * Returns a unique identifier name of a ethdev specified by port identifier.
+ * @param	port_id
+ *   The port identifier.
+ * @param	name
+ *  The pointer to the Unique identifier name for each Ethernet device
+ * @return
+ *   - 0 on success, negative on error
+ */
+extern int rte_eth_dev_get_name_by_port(uint8_t port_id, char *name);
+
+/**
+ * Function for internal use by port hotplug functions.
+ * Check whether or not, a PMD that is handling the ethdev specified by port
+ * identifier can support detach function.
+ * @param	port_id
+ *   The port identifier
+ * @return
+ *   - 0 on supporting detach function, negative on not supporting
+ */
+extern int rte_eth_dev_check_detachable(uint8_t port_id);
+
+/**
+ * Function for internal use by port hotplug functions.
+ * Returns a ethdev slot specified by the unique identifier name.
+ * @param	name
+ *  The pointer to the Unique identifier name for each Ethernet device
+ * @return
+ *   - The pointer to the ethdev slot, on success. NULL on error
+ */
+extern struct rte_eth_dev *rte_eth_dev_allocated(const char *name);
+
+/**
  * Function for internal use by dummy drivers primarily, e.g. ring-based
  * driver.
  * Allocates a new ethdev slot for an ethernet device and returns the pointer
-- 
1.9.1

^ permalink raw reply	[flat|nested] 362+ messages in thread

* [dpdk-dev] [PATCH v5 08/13] eal/linux/pci: Add functions for unmapping igb_uio resources
  2015-01-30  5:42     ` [dpdk-dev] [PATCH v5 00/13] Port Hotplug Framework Tetsuya Mukawa
                         ` (6 preceding siblings ...)
  2015-01-30  5:42       ` [dpdk-dev] [PATCH v5 07/13] ethdev: Add functions that will be used by port hotplug functions Tetsuya Mukawa
@ 2015-01-30  5:42       ` Tetsuya Mukawa
  2015-01-30  5:42       ` [dpdk-dev] [PATCH v5 09/13] eal/pci: Add a function to remove the entry of devargs list Tetsuya Mukawa
                         ` (6 subsequent siblings)
  14 siblings, 0 replies; 362+ messages in thread
From: Tetsuya Mukawa @ 2015-01-30  5:42 UTC (permalink / raw)
  To: dev

The patch adds functions for unmapping igb_uio resources. The patch is only
for Linux and igb_uio environment. VFIO and BSD are not supported.

v5:
- Fix pci_unmap_device() to check pt_driver.
v4:
- Add paramerter checking.
- Add header file to determine if hotplug can be enabled.

Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp>
---
 lib/librte_eal/common/Makefile                  |  1 +
 lib/librte_eal/common/include/rte_dev_hotplug.h | 44 +++++++++++++++++
 lib/librte_eal/linuxapp/eal/eal_pci.c           | 44 +++++++++++++++++
 lib/librte_eal/linuxapp/eal/eal_pci_init.h      |  8 +++
 lib/librte_eal/linuxapp/eal/eal_pci_uio.c       | 65 +++++++++++++++++++++++++
 5 files changed, 162 insertions(+)
 create mode 100644 lib/librte_eal/common/include/rte_dev_hotplug.h

diff --git a/lib/librte_eal/common/Makefile b/lib/librte_eal/common/Makefile
index 52c1a5f..db7cc93 100644
--- a/lib/librte_eal/common/Makefile
+++ b/lib/librte_eal/common/Makefile
@@ -41,6 +41,7 @@ INC += rte_eal_memconfig.h rte_malloc_heap.h
 INC += rte_hexdump.h rte_devargs.h rte_dev.h
 INC += rte_common_vect.h
 INC += rte_pci_dev_feature_defs.h rte_pci_dev_features.h
+INC += rte_dev_hotplug.h
 
 ifeq ($(CONFIG_RTE_INSECURE_FUNCTION_WARNING),y)
 INC += rte_warnings.h
diff --git a/lib/librte_eal/common/include/rte_dev_hotplug.h b/lib/librte_eal/common/include/rte_dev_hotplug.h
new file mode 100644
index 0000000..b333e0f
--- /dev/null
+++ b/lib/librte_eal/common/include/rte_dev_hotplug.h
@@ -0,0 +1,44 @@
+/*-
+ *   BSD LICENSE
+ *
+ *   Copyright(c) 2015 IGEL Co.,LTd.
+ *   All rights reserved.
+ *
+ *   Redistribution and use in source and binary forms, with or without
+ *   modification, are permitted provided that the following conditions
+ *   are met:
+ *
+ *     * Redistributions of source code must retain the above copyright
+ *       notice, this list of conditions and the following disclaimer.
+ *     * Redistributions in binary form must reproduce the above copyright
+ *       notice, this list of conditions and the following disclaimer in
+ *       the documentation and/or other materials provided with the
+ *       distribution.
+ *     * Neither the name of IGEL Co.,Ltd. nor the names of its
+ *       contributors may be used to endorse or promote products derived
+ *       from this software without specific prior written permission.
+ *
+ *   THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+ *   "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+ *   LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+ *   A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+ *   OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ *   SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+ *   LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+ *   DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+ *   THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ *   (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ *   OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#ifndef _RTE_DEV_HOTPLUG_H_
+#define _RTE_DEV_HOTPLUG_H_
+
+/*
+ * determine if hotplug can be enabled on the system
+ */
+#if defined(RTE_LIBRTE_EAL_HOTPLUG) && defined(RTE_LIBRTE_EAL_LINUXAPP)
+#define ENABLE_HOTPLUG
+#endif /* RTE_LIBRTE_EAL_HOTPLUG & RTE_LIBRTE_EAL_LINUXAPP */
+
+#endif /* _RTE_DEV_HOTPLUG_H_ */
diff --git a/lib/librte_eal/linuxapp/eal/eal_pci.c b/lib/librte_eal/linuxapp/eal/eal_pci.c
index d847102..c3b7917 100644
--- a/lib/librte_eal/linuxapp/eal/eal_pci.c
+++ b/lib/librte_eal/linuxapp/eal/eal_pci.c
@@ -166,6 +166,25 @@ pci_map_resource(void *requested_addr, int fd, off_t offset, size_t size)
 	return mapaddr;
 }
 
+#ifdef ENABLE_HOTPLUG
+/* unmap a particular resource */
+void
+pci_unmap_resource(void *requested_addr, size_t size)
+{
+	if (requested_addr == NULL)
+		return;
+
+	/* Unmap the PCI memory resource of device */
+	if (munmap(requested_addr, size)) {
+		RTE_LOG(ERR, EAL, "%s(): cannot munmap(%p, 0x%lx): %s\n",
+			__func__, requested_addr, (unsigned long)size,
+			strerror(errno));
+	} else
+		RTE_LOG(DEBUG, EAL, "  PCI memory mapped at %p\n",
+				requested_addr);
+}
+#endif /* ENABLE_HOTPLUG */
+
 /* parse the "resource" sysfs file */
 #define IORESOURCE_MEM  0x00000200
 
@@ -567,6 +586,31 @@ pci_map_device(struct rte_pci_device *dev)
 	return ret;
 }
 
+#ifdef ENABLE_HOTPLUG
+static void
+pci_unmap_device(struct rte_pci_device *dev)
+{
+	if (dev == NULL)
+		return;
+
+	/* try unmapping the NIC resources using VFIO if it exists */
+	switch (dev->pt_driver) {
+	case RTE_PT_VFIO:
+		RTE_LOG(ERR, EAL, "Hotplug doesn't support vfio yet\n");
+		break;
+	case RTE_PT_IGB_UIO:
+	case RTE_PT_UIO_GENERIC:
+		/* unmap resources for devices that use uio */
+		pci_uio_unmap_resource(dev);
+		break;
+	default:
+		RTE_LOG(DEBUG, EAL, "  Not managed by known pt driver,"
+			" skipped\n");
+		break;
+	}
+}
+#endif /* ENABLE_HOTPLUG */
+
 /*
  * If vendor/device ID match, call the devinit() function of the
  * driver.
diff --git a/lib/librte_eal/linuxapp/eal/eal_pci_init.h b/lib/librte_eal/linuxapp/eal/eal_pci_init.h
index 1070eb8..5152a0b 100644
--- a/lib/librte_eal/linuxapp/eal/eal_pci_init.h
+++ b/lib/librte_eal/linuxapp/eal/eal_pci_init.h
@@ -34,6 +34,7 @@
 #ifndef EAL_PCI_INIT_H_
 #define EAL_PCI_INIT_H_
 
+#include <rte_dev_hotplug.h>
 #include "eal_vfio.h"
 
 struct pci_map {
@@ -71,6 +72,13 @@ void *pci_map_resource(void *requested_addr, int fd, off_t offset,
 /* map IGB_UIO resource prototype */
 int pci_uio_map_resource(struct rte_pci_device *dev);
 
+#ifdef ENABLE_HOTPLUG
+void pci_unmap_resource(void *requested_addr, size_t size);
+
+/* unmap IGB_UIO resource prototype */
+void pci_uio_unmap_resource(struct rte_pci_device *dev);
+#endif /* ENABLE_HOTPLUG */
+
 #ifdef VFIO_PRESENT
 
 #define VFIO_MAX_GROUPS 64
diff --git a/lib/librte_eal/linuxapp/eal/eal_pci_uio.c b/lib/librte_eal/linuxapp/eal/eal_pci_uio.c
index 1da3507..81830d1 100644
--- a/lib/librte_eal/linuxapp/eal/eal_pci_uio.c
+++ b/lib/librte_eal/linuxapp/eal/eal_pci_uio.c
@@ -404,6 +404,71 @@ pci_uio_map_resource(struct rte_pci_device *dev)
 	return 0;
 }
 
+#ifdef ENABLE_HOTPLUG
+static void
+pci_uio_unmap(struct mapped_pci_resource *uio_res)
+{
+	int i;
+
+	if (uio_res == NULL)
+		return;
+
+	for (i = 0; i != uio_res->nb_maps; i++)
+		pci_unmap_resource(uio_res->maps[i].addr,
+				(size_t)uio_res->maps[i].size);
+}
+
+static struct mapped_pci_resource *
+pci_uio_find_resource(struct rte_pci_device *dev)
+{
+	struct mapped_pci_resource *uio_res;
+
+	if (dev == NULL)
+		return NULL;
+
+	TAILQ_FOREACH(uio_res, pci_res_list, next) {
+
+		/* skip this element if it doesn't match our PCI address */
+		if (!eal_compare_pci_addr(&uio_res->pci_addr, &dev->addr))
+			return uio_res;
+	}
+	return NULL;
+}
+
+/* unmap the PCI resource of a PCI device in virtual memory */
+void
+pci_uio_unmap_resource(struct rte_pci_device *dev)
+{
+	struct mapped_pci_resource *uio_res;
+
+	if (dev == NULL)
+		return;
+
+	/* find an entry for the device */
+	uio_res = pci_uio_find_resource(dev);
+	if (uio_res == NULL)
+		return;
+
+	/* secondary processes - just free maps */
+	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
+		return pci_uio_unmap(uio_res);
+
+	TAILQ_REMOVE(pci_res_list, uio_res, next);
+
+	/* unmap all resources */
+	pci_uio_unmap(uio_res);
+
+	/* free uio resource */
+	rte_free(uio_res);
+
+	/* close fd if in primary process */
+	close(dev->intr_handle.fd);
+
+	dev->intr_handle.fd = -1;
+	dev->intr_handle.type = RTE_INTR_HANDLE_UNKNOWN;
+}
+#endif /* ENABLE_HOTPLUG */
+
 /*
  * parse a sysfs file containing one integer value
  * different to the eal version, as it needs to work with 64-bit values
-- 
1.9.1

^ permalink raw reply	[flat|nested] 362+ messages in thread

* [dpdk-dev] [PATCH v5 09/13] eal/pci: Add a function to remove the entry of devargs list
  2015-01-30  5:42     ` [dpdk-dev] [PATCH v5 00/13] Port Hotplug Framework Tetsuya Mukawa
                         ` (7 preceding siblings ...)
  2015-01-30  5:42       ` [dpdk-dev] [PATCH v5 08/13] eal/linux/pci: Add functions for unmapping igb_uio resources Tetsuya Mukawa
@ 2015-01-30  5:42       ` Tetsuya Mukawa
  2015-01-30  5:42       ` [dpdk-dev] [PATCH v5 10/13] eal/pci: Cleanup pci driver initialization code Tetsuya Mukawa
                         ` (5 subsequent siblings)
  14 siblings, 0 replies; 362+ messages in thread
From: Tetsuya Mukawa @ 2015-01-30  5:42 UTC (permalink / raw)
  To: dev

The function removes the specified devargs entry from devargs_list.
Also the patch adds sanity checking to rte_eal_devargs_add().

v5:
- Change function definition of rte_eal_devargs_remove().
v4:
- Fix sanity check code.

Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp>
---
 lib/librte_eal/common/eal_common_devargs.c  | 60 +++++++++++++++++++++++++++++
 lib/librte_eal/common/include/rte_devargs.h | 21 ++++++++++
 2 files changed, 81 insertions(+)

diff --git a/lib/librte_eal/common/eal_common_devargs.c b/lib/librte_eal/common/eal_common_devargs.c
index 4c7d11a..5b1ac8e 100644
--- a/lib/librte_eal/common/eal_common_devargs.c
+++ b/lib/librte_eal/common/eal_common_devargs.c
@@ -44,6 +44,35 @@
 struct rte_devargs_list devargs_list =
 	TAILQ_HEAD_INITIALIZER(devargs_list);
 
+
+/* find a entry specified by pci address or device name */
+static struct rte_devargs *
+rte_eal_devargs_find(enum rte_devtype devtype, void *args)
+{
+	struct rte_devargs *devargs;
+
+	if (args == NULL)
+		return NULL;
+
+	TAILQ_FOREACH(devargs, &devargs_list, next) {
+		switch (devtype) {
+		case RTE_DEVTYPE_WHITELISTED_PCI:
+		case RTE_DEVTYPE_BLACKLISTED_PCI:
+			if (eal_compare_pci_addr(&devargs->pci.addr, args) == 0)
+				goto found;
+			break;
+		case RTE_DEVTYPE_VIRTUAL:
+			if (memcmp(&devargs->virtual.drv_name, args,
+			    strlen((char *)args)) == 0)
+				goto found;
+			break;
+		}
+	}
+	return NULL;
+found:
+	return devargs;
+}
+
 /* store a whitelist parameter for later parsing */
 int
 rte_eal_devargs_add(enum rte_devtype devtype, const char *devargs_str)
@@ -87,6 +116,12 @@ rte_eal_devargs_add(enum rte_devtype devtype, const char *devargs_str)
 			free(devargs);
 			return -1;
 		}
+		/* make sure there is no same entry */
+		if (rte_eal_devargs_find(devtype, &devargs->pci.addr)) {
+			RTE_LOG(ERR, EAL,
+				"device already registered: <%s>\n", buf);
+			return -1;
+		}
 		break;
 	case RTE_DEVTYPE_VIRTUAL:
 		/* save driver name */
@@ -98,6 +133,12 @@ rte_eal_devargs_add(enum rte_devtype devtype, const char *devargs_str)
 			free(devargs);
 			return -1;
 		}
+		/* make sure there is no same entry */
+		if (rte_eal_devargs_find(devtype, &devargs->virtual.drv_name)) {
+			RTE_LOG(ERR, EAL,
+				"device already registered: <%s>\n", buf);
+			return -1;
+		}
 		break;
 	}
 
@@ -105,6 +146,25 @@ rte_eal_devargs_add(enum rte_devtype devtype, const char *devargs_str)
 	return 0;
 }
 
+/* remove it from the devargs_list */
+int
+rte_eal_devargs_remove(enum rte_devtype devtype, void *args)
+{
+	struct rte_devargs *devargs;
+
+	if (args == NULL)
+		return -EINVAL;
+
+	devargs = rte_eal_devargs_find(devtype, args);
+	if (devargs == NULL) {
+		RTE_LOG(ERR, EAL, "device not found\n");
+		return -ENODEV;
+	}
+
+	TAILQ_REMOVE(&devargs_list, devargs, next);
+	return 0;
+}
+
 /* count the number of devices of a specified type */
 unsigned int
 rte_eal_devargs_type_count(enum rte_devtype devtype)
diff --git a/lib/librte_eal/common/include/rte_devargs.h b/lib/librte_eal/common/include/rte_devargs.h
index 9f9c98f..b5ad4b3 100644
--- a/lib/librte_eal/common/include/rte_devargs.h
+++ b/lib/librte_eal/common/include/rte_devargs.h
@@ -123,6 +123,27 @@ extern struct rte_devargs_list devargs_list;
 int rte_eal_devargs_add(enum rte_devtype devtype, const char *devargs_str);
 
 /**
+ * Remove a device from the user device list
+ *
+ * For PCI devices, the format of arguments string is "PCI_ADDR". It shouldn't
+ * involves parameters for the device. Example: "08:00.1".
+ *
+ * For virtual devices, the format of arguments string is "DRIVER_NAME*". It
+ * shouldn't involves parameters for the device. Example: "eth_ring". The
+ * validity of the driver name is not checked by this function, it is done
+ * when closing the drivers.
+ *
+ * @param devtype
+ *   The type of the device.
+ * @param name
+ *   The name of the device.
+ *
+ * @return
+ *   - 0 on success, negative on error
+ */
+int rte_eal_devargs_remove(enum rte_devtype devtype, void *args);
+
+/**
  * Count the number of user devices of a specified type
  *
  * @param devtype
-- 
1.9.1

^ permalink raw reply	[flat|nested] 362+ messages in thread

* [dpdk-dev] [PATCH v5 10/13] eal/pci: Cleanup pci driver initialization code
  2015-01-30  5:42     ` [dpdk-dev] [PATCH v5 00/13] Port Hotplug Framework Tetsuya Mukawa
                         ` (8 preceding siblings ...)
  2015-01-30  5:42       ` [dpdk-dev] [PATCH v5 09/13] eal/pci: Add a function to remove the entry of devargs list Tetsuya Mukawa
@ 2015-01-30  5:42       ` Tetsuya Mukawa
  2015-01-30  5:42       ` [dpdk-dev] [PATCH v5 11/13] ethdev: Add one dev_type paramerter to rte_eth_dev_allocate Tetsuya Mukawa
                         ` (4 subsequent siblings)
  14 siblings, 0 replies; 362+ messages in thread
From: Tetsuya Mukawa @ 2015-01-30  5:42 UTC (permalink / raw)
  To: dev

- Add rte_eal_pci_close_one_dirver()
  The function is used for closing the specified driver and device.
- Add pci_invoke_all_drivers()
  The function is based on pci_probe_all_drivers. But it can not only
  probe but also close drivers.
- Add pci_close_all_drivers()
  The function tries to find a driver for the specified device, and
  then close the driver.
- Add rte_eal_pci_probe_one() and rte_eal_pci_close_one()
  The functions are used for probe and close a device.
  First the function tries to find a device that has the specfied
  PCI address. Then, probe or close the device.

v5:
- Remove RTE_EAL_INVOKE_TYPE_UNKNOWN, because it's unused.
v4:
- Fix paramerter checking.
- Fix indent of 'if' statement.

Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp>
---
 lib/librte_eal/common/eal_common_pci.c  | 90 +++++++++++++++++++++++++++++----
 lib/librte_eal/common/eal_private.h     | 24 +++++++++
 lib/librte_eal/common/include/rte_pci.h | 33 ++++++++++++
 lib/librte_eal/linuxapp/eal/eal_pci.c   | 69 +++++++++++++++++++++++++
 4 files changed, 206 insertions(+), 10 deletions(-)

diff --git a/lib/librte_eal/common/eal_common_pci.c b/lib/librte_eal/common/eal_common_pci.c
index a89f5c3..7c9b8c5 100644
--- a/lib/librte_eal/common/eal_common_pci.c
+++ b/lib/librte_eal/common/eal_common_pci.c
@@ -99,19 +99,27 @@ static struct rte_devargs *pci_devargs_lookup(struct rte_pci_device *dev)
 	return NULL;
 }
 
-/*
- * If vendor/device ID match, call the devinit() function of all
- * registered driver for the given device. Return -1 if initialization
- * failed, return 1 if no driver is found for this device.
- */
 static int
-pci_probe_all_drivers(struct rte_pci_device *dev)
+pci_invoke_all_drivers(struct rte_pci_device *dev,
+		enum rte_eal_invoke_type type)
 {
 	struct rte_pci_driver *dr = NULL;
-	int rc;
+	int rc = 0;
+
+	if ((dev == NULL) || (type >= RTE_EAL_INVOKE_TYPE_MAX))
+		return -1;
 
 	TAILQ_FOREACH(dr, &pci_driver_list, next) {
-		rc = rte_eal_pci_probe_one_driver(dr, dev);
+		switch (type) {
+		case RTE_EAL_INVOKE_TYPE_PROBE:
+			rc = rte_eal_pci_probe_one_driver(dr, dev);
+			break;
+		case RTE_EAL_INVOKE_TYPE_CLOSE:
+			rc = rte_eal_pci_close_one_driver(dr, dev);
+			break;
+		default:
+			return -1;
+		}
 		if (rc < 0)
 			/* negative value is an error */
 			return -1;
@@ -123,6 +131,66 @@ pci_probe_all_drivers(struct rte_pci_device *dev)
 	return 1;
 }
 
+#ifdef ENABLE_HOTPLUG
+static int
+rte_eal_pci_invoke_one(struct rte_pci_addr *addr,
+		enum rte_eal_invoke_type type)
+{
+	struct rte_pci_device *dev = NULL;
+	int ret = 0;
+
+	if ((addr == NULL) || (type >= RTE_EAL_INVOKE_TYPE_MAX))
+		return -1;
+
+	TAILQ_FOREACH(dev, &pci_device_list, next) {
+		if (eal_compare_pci_addr(&dev->addr, addr))
+			continue;
+
+		ret = pci_invoke_all_drivers(dev, type);
+		if (ret < 0)
+			goto invoke_err_return;
+
+		if (type == RTE_EAL_INVOKE_TYPE_CLOSE)
+			goto remove_dev;
+
+		return 0;
+	}
+
+	return -1;
+
+invoke_err_return:
+	RTE_LOG(WARNING, EAL, "Requested device " PCI_PRI_FMT
+			" cannot be used\n", dev->addr.domain, dev->addr.bus,
+			dev->addr.devid, dev->addr.function);
+	return -1;
+
+remove_dev:
+	TAILQ_REMOVE(&pci_device_list, dev, next);
+	return 0;
+}
+
+
+/*
+ * Find the pci device specified by pci address, then invoke probe function of
+ * the driver of the devive.
+ */
+int
+rte_eal_pci_probe_one(struct rte_pci_addr *addr)
+{
+	return rte_eal_pci_invoke_one(addr, RTE_EAL_INVOKE_TYPE_PROBE);
+}
+
+/*
+ * Find the pci device specified by pci address, then invoke close function of
+ * the driver of the devive.
+ */
+int
+rte_eal_pci_close_one(struct rte_pci_addr *addr)
+{
+	return rte_eal_pci_invoke_one(addr, RTE_EAL_INVOKE_TYPE_CLOSE);
+}
+#endif /* ENABLE_HOTPLUG */
+
 /*
  * Scan the content of the PCI bus, and call the devinit() function for
  * all registered drivers that have a matching entry in its id_table
@@ -148,10 +216,12 @@ rte_eal_pci_probe(void)
 
 		/* probe all or only whitelisted devices */
 		if (probe_all)
-			ret = pci_probe_all_drivers(dev);
+			ret = pci_invoke_all_drivers(dev,
+					RTE_EAL_INVOKE_TYPE_PROBE);
 		else if (devargs != NULL &&
 			devargs->type == RTE_DEVTYPE_WHITELISTED_PCI)
-			ret = pci_probe_all_drivers(dev);
+			ret = pci_invoke_all_drivers(dev,
+					RTE_EAL_INVOKE_TYPE_PROBE);
 		if (ret < 0)
 			rte_exit(EXIT_FAILURE, "Requested device " PCI_PRI_FMT
 				 " cannot be used\n", dev->addr.domain, dev->addr.bus,
diff --git a/lib/librte_eal/common/eal_private.h b/lib/librte_eal/common/eal_private.h
index 159cd66..1a362ab 100644
--- a/lib/librte_eal/common/eal_private.h
+++ b/lib/librte_eal/common/eal_private.h
@@ -154,6 +154,15 @@ struct rte_pci_driver;
 struct rte_pci_device;
 
 /**
+ * The invoke type.
+ */
+enum rte_eal_invoke_type {
+	RTE_EAL_INVOKE_TYPE_PROBE,  /**< invoke probe function */
+	RTE_EAL_INVOKE_TYPE_CLOSE,  /**< invoke close function */
+	RTE_EAL_INVOKE_TYPE_MAX     /**< max value of this enum */
+};
+
+/**
  * Mmap memory for single PCI device
  *
  * This function is private to EAL.
@@ -165,6 +174,21 @@ int rte_eal_pci_probe_one_driver(struct rte_pci_driver *dr,
 		struct rte_pci_device *dev);
 
 /**
+ * Munmap memory for single PCI device
+ *
+ * This function is private to EAL.
+ *
+ * @param	dr
+ *  The pointer to the pci driver structure
+ * @param	dev
+ *  The pointer to the pci device structure
+ * @return
+ *   0 on success, negative on error
+ */
+int rte_eal_pci_close_one_driver(struct rte_pci_driver *dr,
+		struct rte_pci_device *dev);
+
+/**
  * Init tail queues for non-EAL library structures. This is to allow
  * the rings, mempools, etc. lists to be shared among multiple processes
  *
diff --git a/lib/librte_eal/common/include/rte_pci.h b/lib/librte_eal/common/include/rte_pci.h
index 87ca4cf..a111066 100644
--- a/lib/librte_eal/common/include/rte_pci.h
+++ b/lib/librte_eal/common/include/rte_pci.h
@@ -82,6 +82,7 @@ extern "C" {
 #include <inttypes.h>
 
 #include <rte_interrupts.h>
+#include <rte_dev_hotplug.h>
 
 TAILQ_HEAD(pci_device_list, rte_pci_device); /**< PCI devices in D-linked Q. */
 TAILQ_HEAD(pci_driver_list, rte_pci_driver); /**< PCI drivers in D-linked Q. */
@@ -323,6 +324,38 @@ eal_compare_pci_addr(struct rte_pci_addr *addr, struct rte_pci_addr *addr2)
  */
 int rte_eal_pci_probe(void);
 
+#ifdef ENABLE_HOTPLUG
+/**
+ * Probe the single PCI device.
+ *
+ * Scan the content of the PCI bus, and find the pci device specified by pci
+ * address, then call the probe() function for registered driver that has a
+ * matching entry in its id_table for discovered device.
+ *
+ * @param addr
+ *	The PCI Bus-Device-Function address to probe or close.
+ * @return
+ *   - 0 on success.
+ *   - Negative on error.
+ */
+int rte_eal_pci_probe_one(struct rte_pci_addr *addr);
+
+/**
+ * Close the single PCI device.
+ *
+ * Scan the content of the PCI bus, and find the pci device specified by pci
+ * address, then call the close() function for registered driver that has a
+ * matching entry in its id_table for discovered device.
+ *
+ * @param addr
+ *	The PCI Bus-Device-Function address to probe or close.
+ * @return
+ *   - 0 on success.
+ *   - Negative on error.
+ */
+int rte_eal_pci_close_one(struct rte_pci_addr *addr);
+#endif /* ENABLE_HOTPLUG */
+
 /**
  * Dump the content of the PCI bus.
  *
diff --git a/lib/librte_eal/linuxapp/eal/eal_pci.c b/lib/librte_eal/linuxapp/eal/eal_pci.c
index c3b7917..831422e 100644
--- a/lib/librte_eal/linuxapp/eal/eal_pci.c
+++ b/lib/librte_eal/linuxapp/eal/eal_pci.c
@@ -682,6 +682,75 @@ rte_eal_pci_probe_one_driver(struct rte_pci_driver *dr, struct rte_pci_device *d
 	return 1;
 }
 
+#ifdef ENABLE_HOTPLUG
+/*
+ * If vendor/device ID match, call the devuninit() function of the
+ * driver.
+ */
+int
+rte_eal_pci_close_one_driver(struct rte_pci_driver *dr,
+		struct rte_pci_device *dev)
+{
+	struct rte_pci_id *id_table;
+
+	if ((dr == NULL) || (dev == NULL))
+		return -EINVAL;
+
+	for (id_table = dr->id_table ; id_table->vendor_id != 0; id_table++) {
+
+		/* check if device's identifiers match the driver's ones */
+		if (id_table->vendor_id != dev->id.vendor_id &&
+		    id_table->vendor_id != PCI_ANY_ID)
+			continue;
+		if (id_table->device_id != dev->id.device_id &&
+		    id_table->device_id != PCI_ANY_ID)
+			continue;
+		if (id_table->subsystem_vendor_id !=
+		    dev->id.subsystem_vendor_id &&
+		    id_table->subsystem_vendor_id != PCI_ANY_ID)
+			continue;
+		if (id_table->subsystem_device_id !=
+		    dev->id.subsystem_device_id &&
+		    id_table->subsystem_device_id != PCI_ANY_ID)
+			continue;
+
+		struct rte_pci_addr *loc = &dev->addr;
+
+		RTE_LOG(DEBUG, EAL,
+				"PCI device "PCI_PRI_FMT" on NUMA socket %i\n",
+				loc->domain, loc->bus, loc->devid,
+				loc->function, dev->numa_node);
+
+		RTE_LOG(DEBUG, EAL, "  remove driver: %x:%x %s\n",
+				dev->id.vendor_id, dev->id.device_id,
+				dr->name);
+
+		/* call the driver devuninit() function */
+		if (dr->devuninit && (dr->devuninit(dr, dev) < 0))
+			return -1;	/* negative value is an error */
+
+		/* clear driver structure */
+		dev->driver = NULL;
+
+		if (dr->drv_flags & RTE_PCI_DRV_NEED_MAPPING)
+			/* unmap resources for devices that use igb_uio */
+			pci_unmap_device(dev);
+
+		return 0;
+	}
+	/* return positive value if driver is not found */
+	return 1;
+}
+#else /* ENABLE_HOTPLUG */
+int
+rte_eal_pci_close_one_driver(struct rte_pci_driver *dr __rte_unused,
+		struct rte_pci_device *dev __rte_unused)
+{
+	RTE_LOG(ERR, EAL, "Hotplug support isn't enabled\n");
+	return -1;
+}
+#endif /* ENABLE_HOTPLUG */
+
 /* Init the PCI EAL subsystem */
 int
 rte_eal_pci_init(void)
-- 
1.9.1

^ permalink raw reply	[flat|nested] 362+ messages in thread

* [dpdk-dev] [PATCH v5 11/13] ethdev: Add one dev_type paramerter to rte_eth_dev_allocate
  2015-01-30  5:42     ` [dpdk-dev] [PATCH v5 00/13] Port Hotplug Framework Tetsuya Mukawa
                         ` (9 preceding siblings ...)
  2015-01-30  5:42       ` [dpdk-dev] [PATCH v5 10/13] eal/pci: Cleanup pci driver initialization code Tetsuya Mukawa
@ 2015-01-30  5:42       ` Tetsuya Mukawa
  2015-01-30  5:42       ` [dpdk-dev] [PATCH v5 12/13] eal/pci: Add rte_eal_dev_attach/detach() functions Tetsuya Mukawa
                         ` (3 subsequent siblings)
  14 siblings, 0 replies; 362+ messages in thread
From: Tetsuya Mukawa @ 2015-01-30  5:42 UTC (permalink / raw)
  To: dev

This new parameter is needed to keep device type like physical or virtual.
Port detaching processes are different between physical and virtual.
This paramerter lets detaching function know a device type of the port.

v4:
- Fix comments of rte_eth_dev_type.

Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp>
---
 app/test/virtual_pmd.c                       |  2 +-
 lib/librte_ether/rte_ethdev.c                | 14 ++++++++++++--
 lib/librte_ether/rte_ethdev.h                | 25 ++++++++++++++++++++++++-
 lib/librte_pmd_af_packet/rte_eth_af_packet.c |  2 +-
 lib/librte_pmd_bond/rte_eth_bond_api.c       |  2 +-
 lib/librte_pmd_pcap/rte_eth_pcap.c           |  2 +-
 lib/librte_pmd_ring/rte_eth_ring.c           |  2 +-
 lib/librte_pmd_xenvirt/rte_eth_xenvirt.c     |  2 +-
 8 files changed, 42 insertions(+), 9 deletions(-)

diff --git a/app/test/virtual_pmd.c b/app/test/virtual_pmd.c
index 9fac95d..8d3a5ff 100644
--- a/app/test/virtual_pmd.c
+++ b/app/test/virtual_pmd.c
@@ -556,7 +556,7 @@ virtual_ethdev_create(const char *name, struct ether_addr *mac_addr,
 		goto err;
 
 	/* reserve an ethdev entry */
-	eth_dev = rte_eth_dev_allocate(name);
+	eth_dev = rte_eth_dev_allocate(name, RTE_ETH_DEV_PHYSICAL);
 	if (eth_dev == NULL)
 		goto err;
 
diff --git a/lib/librte_ether/rte_ethdev.c b/lib/librte_ether/rte_ethdev.c
index 2acafc7..0c7fbb1 100644
--- a/lib/librte_ether/rte_ethdev.c
+++ b/lib/librte_ether/rte_ethdev.c
@@ -232,7 +232,7 @@ rte_eth_dev_allocate_new_port(void)
 }
 
 struct rte_eth_dev *
-rte_eth_dev_allocate(const char *name)
+rte_eth_dev_allocate(const char *name, enum rte_eth_dev_type type)
 {
 	uint8_t port_id;
 	struct rte_eth_dev *eth_dev;
@@ -256,6 +256,7 @@ rte_eth_dev_allocate(const char *name)
 	snprintf(eth_dev->data->name, sizeof(eth_dev->data->name), "%s", name);
 	eth_dev->data->port_id = port_id;
 	eth_dev->attached = DEV_CONNECTED;
+	eth_dev->dev_type = type;
 	nb_ports++;
 	return eth_dev;
 }
@@ -276,6 +277,7 @@ rte_eth_dev_free(const char *name)
 	}
 
 	eth_dev->attached = 0;
+	eth_dev->dev_type = RTE_ETH_DEV_UNKNOWN;
 	nb_ports--;
 	return eth_dev;
 }
@@ -296,7 +298,7 @@ rte_eth_dev_init(struct rte_pci_driver *pci_drv,
 	snprintf(ethdev_name, RTE_ETH_NAME_MAX_LEN, "%d:%d.%d",
 			pci_dev->addr.bus, pci_dev->addr.devid, pci_dev->addr.function);
 
-	eth_dev = rte_eth_dev_allocate(ethdev_name);
+	eth_dev = rte_eth_dev_allocate(ethdev_name, RTE_ETH_DEV_PHYSICAL);
 	if (eth_dev == NULL)
 		return -ENOMEM;
 
@@ -428,6 +430,14 @@ rte_eth_dev_count(void)
 	return (nb_ports);
 }
 
+enum rte_eth_dev_type
+rte_eth_dev_get_device_type(uint8_t port_id)
+{
+	if (rte_eth_dev_validate_port(port_id, NONE_TRACE) == DEV_INVALID)
+		return -1;
+	return rte_eth_devices[port_id].dev_type;
+}
+
 void
 rte_eth_dev_save(struct rte_eth_dev *devs)
 {
diff --git a/lib/librte_ether/rte_ethdev.h b/lib/librte_ether/rte_ethdev.h
index ebc48b0..83a4000 100644
--- a/lib/librte_ether/rte_ethdev.h
+++ b/lib/librte_ether/rte_ethdev.h
@@ -1522,6 +1522,17 @@ struct eth_dev_ops {
 };
 
 /**
+ * The eth device type
+ */
+enum rte_eth_dev_type {
+	RTE_ETH_DEV_UNKNOWN,	/**< unknown device type */
+	RTE_ETH_DEV_PHYSICAL,
+		/**< Physical function and Virtual function devices of NIC */
+	RTE_ETH_DEV_VIRTUAL,	/**< non hardware device */
+	RTE_ETH_DEV_MAX		/**< max value of this enum */
+};
+
+/**
  * @internal
  * The generic data structure associated with each ethernet device.
  *
@@ -1540,6 +1551,7 @@ struct rte_eth_dev {
 	struct rte_pci_device *pci_dev; /**< PCI info. supplied by probing */
 	struct rte_eth_dev_cb_list callbacks; /**< User application callbacks */
 	uint8_t attached; /**< Flag indicating the port is attached */
+	enum rte_eth_dev_type dev_type; /**< Flag indicating the device type */
 };
 
 struct rte_eth_dev_sriov {
@@ -1617,6 +1629,15 @@ extern uint8_t rte_eth_dev_count(void);
 
 /**
  * Function for internal use by port hotplug functions.
+ * Get the device type to know whether the device is physical or virtual.
+ * @param	port_id	The pointer to the port id
+ * @return
+ *   - Device type.
+ */
+extern enum rte_eth_dev_type rte_eth_dev_get_device_type(uint8_t port_id);
+
+/**
+ * Function for internal use by port hotplug functions.
  * Copies current ethdev structures to the specified pointer.
  *
  * @param	devs	The pointer to the ethdev structures
@@ -1702,10 +1723,12 @@ extern struct rte_eth_dev *rte_eth_dev_allocated(const char *name);
  * to that slot for the driver to use.
  *
  * @param	name	Unique identifier name for each Ethernet device
+ * @param	type	Device type of this Ethernet device
  * @return
  *   - Slot in the rte_dev_devices array for a new device;
  */
-struct rte_eth_dev *rte_eth_dev_allocate(const char *name);
+struct rte_eth_dev *rte_eth_dev_allocate(const char *name,
+		enum rte_eth_dev_type type);
 
 /**
  * Function for internal use by dummy drivers primarily, e.g. ring-based
diff --git a/lib/librte_pmd_af_packet/rte_eth_af_packet.c b/lib/librte_pmd_af_packet/rte_eth_af_packet.c
index 1ffe1cd..80e9bdf 100644
--- a/lib/librte_pmd_af_packet/rte_eth_af_packet.c
+++ b/lib/librte_pmd_af_packet/rte_eth_af_packet.c
@@ -649,7 +649,7 @@ rte_pmd_init_internals(const char *name,
 	}
 
 	/* reserve an ethdev entry */
-	*eth_dev = rte_eth_dev_allocate(name);
+	*eth_dev = rte_eth_dev_allocate(name, RTE_ETH_DEV_VIRTUAL);
 	if (*eth_dev == NULL)
 		goto error;
 
diff --git a/lib/librte_pmd_bond/rte_eth_bond_api.c b/lib/librte_pmd_bond/rte_eth_bond_api.c
index 4ab3267..7a6a5f7 100644
--- a/lib/librte_pmd_bond/rte_eth_bond_api.c
+++ b/lib/librte_pmd_bond/rte_eth_bond_api.c
@@ -235,7 +235,7 @@ rte_eth_bond_create(const char *name, uint8_t mode, uint8_t socket_id)
 	}
 
 	/* reserve an ethdev entry */
-	eth_dev = rte_eth_dev_allocate(name);
+	eth_dev = rte_eth_dev_allocate(name, RTE_ETH_DEV_VIRTUAL);
 	if (eth_dev == NULL) {
 		RTE_BOND_LOG(ERR, "Unable to allocate rte_eth_dev");
 		goto err;
diff --git a/lib/librte_pmd_pcap/rte_eth_pcap.c b/lib/librte_pmd_pcap/rte_eth_pcap.c
index d299288..af7fae8 100644
--- a/lib/librte_pmd_pcap/rte_eth_pcap.c
+++ b/lib/librte_pmd_pcap/rte_eth_pcap.c
@@ -709,7 +709,7 @@ rte_pmd_init_internals(const char *name, const unsigned nb_rx_queues,
 		goto error;
 
 	/* reserve an ethdev entry */
-	*eth_dev = rte_eth_dev_allocate(name);
+	*eth_dev = rte_eth_dev_allocate(name, RTE_ETH_DEV_VIRTUAL);
 	if (*eth_dev == NULL)
 		goto error;
 
diff --git a/lib/librte_pmd_ring/rte_eth_ring.c b/lib/librte_pmd_ring/rte_eth_ring.c
index f685f08..5876057 100644
--- a/lib/librte_pmd_ring/rte_eth_ring.c
+++ b/lib/librte_pmd_ring/rte_eth_ring.c
@@ -255,7 +255,7 @@ rte_eth_from_rings(const char *name, struct rte_ring *const rx_queues[],
 		goto error;
 
 	/* reserve an ethdev entry */
-	eth_dev = rte_eth_dev_allocate(name);
+	eth_dev = rte_eth_dev_allocate(name, RTE_ETH_DEV_VIRTUAL);
 	if (eth_dev == NULL)
 		goto error;
 
diff --git a/lib/librte_pmd_xenvirt/rte_eth_xenvirt.c b/lib/librte_pmd_xenvirt/rte_eth_xenvirt.c
index 04e30c9..bc403d6 100644
--- a/lib/librte_pmd_xenvirt/rte_eth_xenvirt.c
+++ b/lib/librte_pmd_xenvirt/rte_eth_xenvirt.c
@@ -648,7 +648,7 @@ eth_dev_xenvirt_create(const char *name, const char *params,
 		goto err;
 
 	/* reserve an ethdev entry */
-	eth_dev = rte_eth_dev_allocate(name);
+	eth_dev = rte_eth_dev_allocate(name, RTE_ETH_DEV_VIRTUAL);
 	if (eth_dev == NULL)
 		goto err;
 
-- 
1.9.1

^ permalink raw reply	[flat|nested] 362+ messages in thread

* [dpdk-dev] [PATCH v5 12/13] eal/pci: Add rte_eal_dev_attach/detach() functions
  2015-01-30  5:42     ` [dpdk-dev] [PATCH v5 00/13] Port Hotplug Framework Tetsuya Mukawa
                         ` (10 preceding siblings ...)
  2015-01-30  5:42       ` [dpdk-dev] [PATCH v5 11/13] ethdev: Add one dev_type paramerter to rte_eth_dev_allocate Tetsuya Mukawa
@ 2015-01-30  5:42       ` Tetsuya Mukawa
  2015-01-30  5:42       ` [dpdk-dev] [PATCH v5 13/13] eal: Enable port hotplug framework in Linux Tetsuya Mukawa
                         ` (2 subsequent siblings)
  14 siblings, 0 replies; 362+ messages in thread
From: Tetsuya Mukawa @ 2015-01-30  5:42 UTC (permalink / raw)
  To: dev

These functions are used for attaching or detaching a port.
When rte_eal_dev_attach() is called, the function tries to realize the
device name as pci address. If this is done successfully,
rte_eal_dev_attach() will attach physical device port. If not, attaches
virtual devive port.
When rte_eal_dev_detach() is called, the function gets the device type
of this port to know whether the port is came from physical or virtual.
And then specific detaching function will be called.

v5:
- Change function names like below.
  rte_eal_dev_find_and_invoke() to rte_eal_vdev_find_and_invoke().
  rte_eal_dev_invoke() to rte_eal_vdev_invoke().
- Add code to handle a return value of rte_eal_devargs_remove().
- Fix pci address format in rte_eal_dev_detach().
v4:
- Fix comment.
- Add error checking.
- Fix indent of 'if' statement.
- Change function name.

Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp>
---
 lib/librte_eal/common/eal_common_dev.c  | 274 ++++++++++++++++++++++++++++++++
 lib/librte_eal/common/eal_private.h     |  11 ++
 lib/librte_eal/common/include/rte_dev.h |  33 ++++
 lib/librte_eal/linuxapp/eal/Makefile    |   1 +
 lib/librte_eal/linuxapp/eal/eal_pci.c   |   6 +-
 5 files changed, 322 insertions(+), 3 deletions(-)

diff --git a/lib/librte_eal/common/eal_common_dev.c b/lib/librte_eal/common/eal_common_dev.c
index eae5656..e3a3f54 100644
--- a/lib/librte_eal/common/eal_common_dev.c
+++ b/lib/librte_eal/common/eal_common_dev.c
@@ -32,10 +32,13 @@
  *   OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
  */
 
+#include <stdio.h>
+#include <limits.h>
 #include <string.h>
 #include <inttypes.h>
 #include <sys/queue.h>
 
+#include <rte_ethdev.h>
 #include <rte_dev.h>
 #include <rte_devargs.h>
 #include <rte_debug.h>
@@ -107,3 +110,274 @@ rte_eal_dev_init(void)
 	}
 	return 0;
 }
+
+/* So far, DPDK hotplug function only supports linux */
+#ifdef ENABLE_HOTPLUG
+static void
+rte_eal_vdev_invoke(struct rte_driver *driver,
+		struct rte_devargs *devargs, enum rte_eal_invoke_type type)
+{
+	if ((driver == NULL) || (devargs == NULL))
+		return;
+
+	switch (type) {
+	case RTE_EAL_INVOKE_TYPE_PROBE:
+		driver->init(devargs->virtual.drv_name, devargs->args);
+		break;
+	case RTE_EAL_INVOKE_TYPE_CLOSE:
+		driver->uninit(devargs->virtual.drv_name, devargs->args);
+		break;
+	default:
+		break;
+	}
+}
+
+static int
+rte_eal_vdev_find_and_invoke(const char *name, int type)
+{
+	struct rte_devargs *devargs;
+	struct rte_driver *driver;
+
+	if (name == NULL)
+		return -EINVAL;
+
+	/* call the init function for each virtual device */
+	TAILQ_FOREACH(devargs, &devargs_list, next) {
+
+		if (devargs->type != RTE_DEVTYPE_VIRTUAL)
+			continue;
+
+		if (strncmp(name, devargs->virtual.drv_name, strlen(name)))
+			continue;
+
+		TAILQ_FOREACH(driver, &dev_driver_list, next) {
+			if (driver->type != PMD_VDEV)
+				continue;
+
+			/* search a driver prefix in virtual device name */
+			if (!strncmp(driver->name, devargs->virtual.drv_name,
+			    strlen(driver->name))) {
+				rte_eal_vdev_invoke(driver, devargs, type);
+				break;
+			}
+		}
+
+		if (driver == NULL) {
+			RTE_LOG(WARNING, EAL, "no driver found for %s\n",
+				  devargs->virtual.drv_name);
+		}
+		return 0;
+	}
+	return 1;
+}
+
+/* attach the new physical device, then store port_id of the device */
+static int
+rte_eal_dev_attach_pdev(struct rte_pci_addr *addr, uint8_t *port_id)
+{
+	uint8_t new_port_id;
+	struct rte_eth_dev devs[RTE_MAX_ETHPORTS];
+
+	if ((addr == NULL) || (port_id == NULL))
+		goto err;
+
+	/* save current port status */
+	rte_eth_dev_save(devs);
+	/* re-construct pci_device_list */
+	if (rte_eal_pci_scan())
+		goto err;
+	/* invoke probe func of the driver can handle the new device */
+	if (rte_eal_pci_probe_one(addr))
+		goto err;
+	/* get port_id enabled by above procedures */
+	if (rte_eth_dev_get_changed_port(devs, &new_port_id))
+		goto err;
+
+	*port_id = new_port_id;
+	return 0;
+err:
+	RTE_LOG(ERR, EAL, "Drver, cannot attach the device\n");
+	return -1;
+}
+
+/* detach the new physical device, then store pci_addr of the device */
+static int
+rte_eal_dev_detach_pdev(uint8_t port_id, struct rte_pci_addr *addr)
+{
+	struct rte_pci_addr freed_addr;
+	struct rte_pci_addr vp;
+
+	if (addr == NULL)
+		goto err;
+
+	/* check whether the driver supports detach feature, or not */
+	if (rte_eth_dev_check_detachable(port_id))
+		goto err;
+
+	/* get pci address by port id */
+	if (rte_eth_dev_get_addr_by_port(port_id, &freed_addr))
+		goto err;
+
+	/* Zerod pci addr means the port comes from virtual device */
+	vp.domain = vp.bus = vp.devid = vp.function = 0;
+	if (eal_compare_pci_addr(&vp, &freed_addr) == 0)
+		goto err;
+
+	/* invoke close func of the driver,
+	 * also remove the device from pci_device_list */
+	if (rte_eal_pci_close_one(&freed_addr))
+		goto err;
+
+	*addr = freed_addr;
+	return 0;
+err:
+	RTE_LOG(ERR, EAL, "Drver, cannot detach the device\n");
+	return -1;
+}
+
+static void
+get_vdev_name(char *vdevargs)
+{
+	char *sep;
+
+	if (vdevargs == NULL)
+		return;
+
+	/* set the first ',' to '\0' to split name and arguments */
+	sep = strchr(vdevargs, ',');
+	if (sep != NULL)
+		sep[0] = '\0';
+}
+
+/* attach the new virtual device, then store port_id of the device */
+static int
+rte_eal_dev_attach_vdev(const char *vdevargs, uint8_t *port_id)
+{
+	char *args;
+	uint8_t new_port_id;
+	struct rte_eth_dev devs[RTE_MAX_ETHPORTS];
+
+	if ((vdevargs == NULL) || (port_id == NULL))
+		goto err0;
+
+	args = strdup(vdevargs);
+	if (args == NULL)
+		goto err0;
+
+	/* save current port status */
+	rte_eth_dev_save(devs);
+	/* add the vdevargs to devargs_list */
+	if (rte_eal_devargs_add(RTE_DEVTYPE_VIRTUAL, args))
+		goto err1;
+	/* parse vdevargs, then retrieve device name */
+	get_vdev_name(args);
+	/* walk around dev_driver_list to find the driver of the device,
+	 * then invoke probe function o the driver */
+	if (rte_eal_vdev_find_and_invoke(args, RTE_EAL_INVOKE_TYPE_PROBE))
+		goto err2;
+	/* get port_id enabled by above procedures */
+	if (rte_eth_dev_get_changed_port(devs, &new_port_id))
+		goto err2;
+
+	free(args);
+	*port_id = new_port_id;
+	return 0;
+err2:
+	rte_eal_devargs_remove(RTE_DEVTYPE_VIRTUAL, args);
+err1:
+	free(args);
+err0:
+	RTE_LOG(ERR, EAL, "Drver, cannot detach the device\n");
+	return -1;
+}
+
+/* detach the new virtual device, then store the name of the device */
+static int
+rte_eal_dev_detach_vdev(uint8_t port_id, char *vdevname)
+{
+	char name[RTE_ETH_NAME_MAX_LEN];
+
+	if (vdevname == NULL)
+		goto err;
+
+	/* check whether the driver supports detach feature, or not */
+	if (rte_eth_dev_check_detachable(port_id))
+		goto err;
+
+	/* get device name by port id */
+	if (rte_eth_dev_get_name_by_port(port_id, name))
+		goto err;
+	/* walk around dev_driver_list to find the driver of the device,
+	 * then invoke close function o the driver */
+	if (rte_eal_vdev_find_and_invoke(name, RTE_EAL_INVOKE_TYPE_CLOSE))
+		goto err;
+	/* remove the vdevname from devargs_list */
+	if (rte_eal_devargs_remove(RTE_DEVTYPE_VIRTUAL, name))
+		goto err;
+
+	strncpy(vdevname, name, sizeof(name));
+	return 0;
+err:
+	RTE_LOG(ERR, EAL, "Drver, cannot detach the device\n");
+	return -1;
+}
+
+/* attach the new device, then store port_id of the device */
+int
+rte_eal_dev_attach(const char *devargs, uint8_t *port_id)
+{
+	struct rte_pci_addr addr;
+
+	if ((devargs == NULL) || (port_id == NULL))
+		return -EINVAL;
+
+	if (eal_parse_pci_DomBDF(devargs, &addr) == 0)
+		return rte_eal_dev_attach_pdev(&addr, port_id);
+	else
+		return rte_eal_dev_attach_vdev(devargs, port_id);
+}
+
+/* detach the device, then store the name of the device */
+int
+rte_eal_dev_detach(uint8_t port_id, char *name)
+{
+	struct rte_pci_addr addr;
+	int ret;
+
+	if (name == NULL)
+		return -EINVAL;
+
+	if (rte_eth_dev_get_device_type(port_id) == RTE_ETH_DEV_PHYSICAL) {
+		ret = rte_eth_dev_get_addr_by_port(port_id, &addr);
+		if (ret < 0)
+			return ret;
+
+		ret = rte_eal_dev_detach_pdev(port_id, &addr);
+		if (ret == 0)
+			snprintf(name, RTE_ETH_NAME_MAX_LEN,
+				"%04x:%02x:%02x.%d",
+				addr.domain, addr.bus,
+				addr.devid, addr.function);
+
+		return ret;
+	} else
+		return rte_eal_dev_detach_vdev(port_id, name);
+}
+#else /* ENABLE_HOTPLUG */
+int
+rte_eal_dev_attach(const char *devargs __rte_unused,
+			uint8_t *port_id __rte_unused)
+{
+	RTE_LOG(ERR, EAL, "Hotplug support isn't enabled\n");
+	return -1;
+}
+
+/* detach the device, then store the name of the device */
+int
+rte_eal_dev_detach(uint8_t port_id __rte_unused,
+			char *name __rte_unused)
+{
+	RTE_LOG(ERR, EAL, "Hotplug support isn't enabled\n");
+	return -1;
+}
+#endif /* ENABLE_HOTPLUG */
diff --git a/lib/librte_eal/common/eal_private.h b/lib/librte_eal/common/eal_private.h
index 1a362ab..8168a7a 100644
--- a/lib/librte_eal/common/eal_private.h
+++ b/lib/librte_eal/common/eal_private.h
@@ -163,6 +163,17 @@ enum rte_eal_invoke_type {
 };
 
 /**
+ * Scan the content of the PCI bus, and the devices in the devices
+ * list
+ *
+ * This function is private to EAL.
+ *
+ * @return
+ *  0 on success, negative on error
+ */
+int rte_eal_pci_scan(void);
+
+/**
  * Mmap memory for single PCI device
  *
  * This function is private to EAL.
diff --git a/lib/librte_eal/common/include/rte_dev.h b/lib/librte_eal/common/include/rte_dev.h
index f7e3a10..e63dd1c 100644
--- a/lib/librte_eal/common/include/rte_dev.h
+++ b/lib/librte_eal/common/include/rte_dev.h
@@ -47,6 +47,7 @@ extern "C" {
 #endif
 
 #include <sys/queue.h>
+#include <rte_pci.h>
 
 /** Double linked list of device drivers. */
 TAILQ_HEAD(rte_driver_list, rte_driver);
@@ -57,6 +58,11 @@ TAILQ_HEAD(rte_driver_list, rte_driver);
 typedef int (rte_dev_init_t)(const char *name, const char *args);
 
 /**
+ * Uninitilization function called for each device driver once.
+ */
+typedef int (rte_dev_uninit_t)(const char *name, const char *args);
+
+/**
  * Driver type enumeration
  */
 enum pmd_type {
@@ -72,6 +78,7 @@ struct rte_driver {
 	enum pmd_type type;		   /**< PMD Driver type */
 	const char *name;                   /**< Driver name. */
 	rte_dev_init_t *init;              /**< Device init. function. */
+	rte_dev_uninit_t *uninit;          /**< Device uninit. function. */
 };
 
 /**
@@ -93,6 +100,32 @@ void rte_eal_driver_register(struct rte_driver *driver);
 void rte_eal_driver_unregister(struct rte_driver *driver);
 
 /**
+ * Attach a new device.
+ *
+ * @param devargs
+ *   A pointer to a strings array describing the new device
+ *   to be attached. The strings should be a pci address like
+ *   '0000:01:00.0' or virtual device name like 'eth_pcap0'.
+ * @param port_id
+ *  A pointer to a port identifier actually attached.
+ * @return
+ *  0 on success and port_id is filled, negative on error
+ */
+int rte_eal_dev_attach(const char *devargs, uint8_t *port_id);
+
+/**
+ * Detach a device.
+ *
+ * @param port_id
+ *   The port identifier of the device to detach.
+ * @param addr
+ *  A pointer to a device name actually detached.
+ * @return
+ *  0 on success and devname is filled, negative on error
+ */
+int rte_eal_dev_detach(uint8_t port_id, char *devname);
+
+/**
  * Initalize all the registered drivers in this process
  */
 int rte_eal_dev_init(void);
diff --git a/lib/librte_eal/linuxapp/eal/Makefile b/lib/librte_eal/linuxapp/eal/Makefile
index 72ecf3a..0ec83b5 100644
--- a/lib/librte_eal/linuxapp/eal/Makefile
+++ b/lib/librte_eal/linuxapp/eal/Makefile
@@ -41,6 +41,7 @@ CFLAGS += -I$(RTE_SDK)/lib/librte_eal/common/include
 CFLAGS += -I$(RTE_SDK)/lib/librte_ring
 CFLAGS += -I$(RTE_SDK)/lib/librte_mempool
 CFLAGS += -I$(RTE_SDK)/lib/librte_malloc
+CFLAGS += -I$(RTE_SDK)/lib/librte_mbuf
 CFLAGS += -I$(RTE_SDK)/lib/librte_ether
 CFLAGS += -I$(RTE_SDK)/lib/librte_ivshmem
 CFLAGS += -I$(RTE_SDK)/lib/librte_pmd_ring
diff --git a/lib/librte_eal/linuxapp/eal/eal_pci.c b/lib/librte_eal/linuxapp/eal/eal_pci.c
index 831422e..1f43688 100644
--- a/lib/librte_eal/linuxapp/eal/eal_pci.c
+++ b/lib/librte_eal/linuxapp/eal/eal_pci.c
@@ -431,8 +431,8 @@ error:
  * Scan the content of the PCI bus, and the devices in the devices
  * list
  */
-static int
-pci_scan(void)
+int
+rte_eal_pci_scan(void)
 {
 	struct dirent *e;
 	DIR *dir;
@@ -764,7 +764,7 @@ rte_eal_pci_init(void)
 	if (internal_config.no_pci)
 		return 0;
 
-	if (pci_scan() < 0) {
+	if (rte_eal_pci_scan() < 0) {
 		RTE_LOG(ERR, EAL, "%s(): Cannot scan PCI bus\n", __func__);
 		return -1;
 	}
-- 
1.9.1

^ permalink raw reply	[flat|nested] 362+ messages in thread

* [dpdk-dev] [PATCH v5 13/13] eal: Enable port hotplug framework in Linux
  2015-01-30  5:42     ` [dpdk-dev] [PATCH v5 00/13] Port Hotplug Framework Tetsuya Mukawa
                         ` (11 preceding siblings ...)
  2015-01-30  5:42       ` [dpdk-dev] [PATCH v5 12/13] eal/pci: Add rte_eal_dev_attach/detach() functions Tetsuya Mukawa
@ 2015-01-30  5:42       ` Tetsuya Mukawa
  2015-01-30  5:42       ` [dpdk-dev] [PATCH v5] librte_pmd_pcap: Add port hotplug support Tetsuya Mukawa
  2015-01-30  5:42       ` [dpdk-dev] [PATCH v5] testpmd: " Tetsuya Mukawa
  14 siblings, 0 replies; 362+ messages in thread
From: Tetsuya Mukawa @ 2015-01-30  5:42 UTC (permalink / raw)
  To: dev

The patch enables CONFIG_RTE_LIBRTE_EAL_HOTPLUG in Linux configuration.

Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp>
---
 config/common_linuxapp | 5 +++++
 1 file changed, 5 insertions(+)

diff --git a/config/common_linuxapp b/config/common_linuxapp
index 2f9643b..27d05be 100644
--- a/config/common_linuxapp
+++ b/config/common_linuxapp
@@ -114,6 +114,11 @@ CONFIG_RTE_PCI_MAX_READ_REQUEST_SIZE=0
 CONFIG_RTE_LIBRTE_EAL_LINUXAPP=y
 
 #
+# Compile Environment Abstraction Layer to support hotplug
+#
+CONFIG_RTE_LIBRTE_EAL_HOTPLUG=y
+
+#
 # Compile Environment Abstraction Layer to support Vmware TSC map
 #
 CONFIG_RTE_LIBRTE_EAL_VMWARE_TSC_MAP_SUPPORT=y
-- 
1.9.1

^ permalink raw reply	[flat|nested] 362+ messages in thread

* [dpdk-dev] [PATCH v5] librte_pmd_pcap: Add port hotplug support
  2015-01-30  5:42     ` [dpdk-dev] [PATCH v5 00/13] Port Hotplug Framework Tetsuya Mukawa
                         ` (12 preceding siblings ...)
  2015-01-30  5:42       ` [dpdk-dev] [PATCH v5 13/13] eal: Enable port hotplug framework in Linux Tetsuya Mukawa
@ 2015-01-30  5:42       ` Tetsuya Mukawa
  2015-01-30  5:42       ` [dpdk-dev] [PATCH v5] testpmd: " Tetsuya Mukawa
  14 siblings, 0 replies; 362+ messages in thread
From: Tetsuya Mukawa @ 2015-01-30  5:42 UTC (permalink / raw)
  To: dev

This patch adds finalization code to free resources allocated by the
PMD.

v4:
 - Change function name.

Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp>
---
 lib/librte_pmd_pcap/rte_eth_pcap.c | 40 ++++++++++++++++++++++++++++++++++++++
 1 file changed, 40 insertions(+)

diff --git a/lib/librte_pmd_pcap/rte_eth_pcap.c b/lib/librte_pmd_pcap/rte_eth_pcap.c
index af7fae8..9263eab 100644
--- a/lib/librte_pmd_pcap/rte_eth_pcap.c
+++ b/lib/librte_pmd_pcap/rte_eth_pcap.c
@@ -498,6 +498,13 @@ static struct eth_dev_ops ops = {
 		.stats_reset = eth_stats_reset,
 };
 
+static struct eth_driver rte_pcap_pmd = {
+	.pci_drv = {
+		.name = "rte_pcap_pmd",
+		.drv_flags = RTE_PCI_DRV_DETACHABLE,
+	},
+};
+
 /*
  * Function handler that opens the pcap file for reading a stores a
  * reference of it for use it later on.
@@ -713,6 +720,10 @@ rte_pmd_init_internals(const char *name, const unsigned nb_rx_queues,
 	if (*eth_dev == NULL)
 		goto error;
 
+	/* check length of device name */
+	if ((strlen((*eth_dev)->data->name) + 1) > sizeof(data->name))
+		goto error;
+
 	/* now put it all together
 	 * - store queue data in internals,
 	 * - store numa_node info in pci_driver
@@ -739,10 +750,13 @@ rte_pmd_init_internals(const char *name, const unsigned nb_rx_queues,
 	data->nb_tx_queues = (uint16_t)nb_tx_queues;
 	data->dev_link = pmd_link;
 	data->mac_addrs = &eth_addr;
+	strncpy(data->name,
+		(*eth_dev)->data->name, strlen((*eth_dev)->data->name));
 
 	(*eth_dev)->data = data;
 	(*eth_dev)->dev_ops = &ops;
 	(*eth_dev)->pci_dev = pci_dev;
+	(*eth_dev)->driver = &rte_pcap_pmd;
 
 	return 0;
 
@@ -927,10 +941,36 @@ rte_pmd_pcap_devinit(const char *name, const char *params)
 
 }
 
+static int
+rte_pmd_pcap_devuninit(const char *name, const char *params __rte_unused)
+{
+	struct rte_eth_dev *eth_dev = NULL;
+
+	RTE_LOG(INFO, PMD, "Closing pcap ethdev on numa socket %u\n",
+			rte_socket_id());
+
+	if (name == NULL)
+		return -1;
+
+	/* reserve an ethdev entry */
+	eth_dev = rte_eth_dev_allocated(name);
+	if (eth_dev == NULL)
+		return -1;
+
+	rte_free(eth_dev->data->dev_private);
+	rte_free(eth_dev->data);
+	rte_free(eth_dev->pci_dev);
+
+	rte_eth_dev_free(name);
+
+	return 0;
+}
+
 static struct rte_driver pmd_pcap_drv = {
 	.name = "eth_pcap",
 	.type = PMD_VDEV,
 	.init = rte_pmd_pcap_devinit,
+	.uninit = rte_pmd_pcap_devuninit,
 };
 
 PMD_REGISTER_DRIVER(pmd_pcap_drv);
-- 
1.9.1

^ permalink raw reply	[flat|nested] 362+ messages in thread

* [dpdk-dev] [PATCH v5] testpmd: Add port hotplug support
  2015-01-30  5:42     ` [dpdk-dev] [PATCH v5 00/13] Port Hotplug Framework Tetsuya Mukawa
                         ` (13 preceding siblings ...)
  2015-01-30  5:42       ` [dpdk-dev] [PATCH v5] librte_pmd_pcap: Add port hotplug support Tetsuya Mukawa
@ 2015-01-30  5:42       ` Tetsuya Mukawa
  14 siblings, 0 replies; 362+ messages in thread
From: Tetsuya Mukawa @ 2015-01-30  5:42 UTC (permalink / raw)
  To: dev

The patch introduces following commands.
- port attach [ident]
- port detach [port_id]
 - attach: attaching a port
 - detach: detaching a port
 - ident: pci address of physical device.
          Or device name and paramerters of virtual device.
         (ex. 0000:02:00.0, eth_pcap0,iface=eth0)
 - port_id: port identifier

v5:
- Add testpmd documentation.
  (Thanks to Iremonger, Bernard)
v4:
 - Fix strings of command help.

Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp>
---
 app/test-pmd/cmdline.c                      | 133 +++++++++++++++----
 app/test-pmd/config.c                       | 116 +++++++++-------
 app/test-pmd/parameters.c                   |  22 ++-
 app/test-pmd/testpmd.c                      | 199 +++++++++++++++++++++-------
 app/test-pmd/testpmd.h                      |  18 ++-
 doc/guides/testpmd_app_ug/testpmd_funcs.rst |  57 ++++++++
 6 files changed, 415 insertions(+), 130 deletions(-)

diff --git a/app/test-pmd/cmdline.c b/app/test-pmd/cmdline.c
index 4beb404..2f813d8 100644
--- a/app/test-pmd/cmdline.c
+++ b/app/test-pmd/cmdline.c
@@ -572,6 +572,12 @@ static void cmd_help_long_parsed(void *parsed_result,
 			"port close (port_id|all)\n"
 			"    Close all ports or port_id.\n\n"
 
+			"port attach (ident)\n"
+			"    Attach physical or virtual dev by pci address or virtual device name\n\n"
+
+			"port detach (port_id)\n"
+			"    Detach physical or virtual dev by port_id\n\n"
+
 			"port config (port_id|all)"
 			" speed (10|100|1000|10000|40000|auto)"
 			" duplex (half|full|auto)\n"
@@ -848,6 +854,89 @@ cmdline_parse_inst_t cmd_operate_specific_port = {
 	},
 };
 
+/* *** attach a specificied port *** */
+struct cmd_operate_attach_port_result {
+	cmdline_fixed_string_t port;
+	cmdline_fixed_string_t keyword;
+	cmdline_fixed_string_t identifier;
+};
+
+static void cmd_operate_attach_port_parsed(void *parsed_result,
+				__attribute__((unused)) struct cmdline *cl,
+				__attribute__((unused)) void *data)
+{
+	struct cmd_operate_attach_port_result *res = parsed_result;
+
+	if (!strcmp(res->keyword, "attach"))
+		attach_port(res->identifier);
+	else
+		printf("Unknown parameter\n");
+}
+
+cmdline_parse_token_string_t cmd_operate_attach_port_port =
+	TOKEN_STRING_INITIALIZER(struct cmd_operate_attach_port_result,
+			port, "port");
+cmdline_parse_token_string_t cmd_operate_attach_port_keyword =
+	TOKEN_STRING_INITIALIZER(struct cmd_operate_attach_port_result,
+			keyword, "attach");
+cmdline_parse_token_string_t cmd_operate_attach_port_identifier =
+	TOKEN_STRING_INITIALIZER(struct cmd_operate_attach_port_result,
+			identifier, NULL);
+
+cmdline_parse_inst_t cmd_operate_attach_port = {
+	.f = cmd_operate_attach_port_parsed,
+	.data = NULL,
+	.help_str = "port attach identifier, "
+		"identifier: pci address or virtual dev name",
+	.tokens = {
+		(void *)&cmd_operate_attach_port_port,
+		(void *)&cmd_operate_attach_port_keyword,
+		(void *)&cmd_operate_attach_port_identifier,
+		NULL,
+	},
+};
+
+/* *** detach a specificied port *** */
+struct cmd_operate_detach_port_result {
+	cmdline_fixed_string_t port;
+	cmdline_fixed_string_t keyword;
+	uint8_t port_id;
+};
+
+static void cmd_operate_detach_port_parsed(void *parsed_result,
+				__attribute__((unused)) struct cmdline *cl,
+				__attribute__((unused)) void *data)
+{
+	struct cmd_operate_detach_port_result *res = parsed_result;
+
+	if (!strcmp(res->keyword, "detach"))
+		detach_port(res->port_id);
+	else
+		printf("Unknown parameter\n");
+}
+
+cmdline_parse_token_string_t cmd_operate_detach_port_port =
+	TOKEN_STRING_INITIALIZER(struct cmd_operate_detach_port_result,
+			port, "port");
+cmdline_parse_token_string_t cmd_operate_detach_port_keyword =
+	TOKEN_STRING_INITIALIZER(struct cmd_operate_detach_port_result,
+			keyword, "detach");
+cmdline_parse_token_num_t cmd_operate_detach_port_port_id =
+	TOKEN_NUM_INITIALIZER(struct cmd_operate_detach_port_result,
+			port_id, UINT8);
+
+cmdline_parse_inst_t cmd_operate_detach_port = {
+	.f = cmd_operate_detach_port_parsed,
+	.data = NULL,
+	.help_str = "port detach port_id",
+	.tokens = {
+		(void *)&cmd_operate_detach_port_port,
+		(void *)&cmd_operate_detach_port_keyword,
+		(void *)&cmd_operate_detach_port_port_id,
+		NULL,
+	},
+};
+
 /* *** configure speed for all ports *** */
 struct cmd_config_speed_all {
 	cmdline_fixed_string_t port;
@@ -902,7 +991,7 @@ cmd_config_speed_all_parsed(void *parsed_result,
 		return;
 	}
 
-	for (pid = 0; pid < nb_ports; pid++) {
+	FOREACH_PORT(pid, ports) {
 		ports[pid].dev_conf.link_speed = link_speed;
 		ports[pid].dev_conf.link_duplex = link_duplex;
 	}
@@ -970,10 +1059,8 @@ cmd_config_speed_specific_parsed(void *parsed_result,
 		return;
 	}
 
-	if (res->id >= nb_ports) {
-		printf("Port id %d must be less than %d\n", res->id, nb_ports);
+	if (port_id_is_invalid(res->id, ENABLED_WARN))
 		return;
-	}
 
 	if (!strcmp(res->value1, "10"))
 		link_speed = ETH_LINK_SPEED_10;
@@ -1533,7 +1620,7 @@ cmd_config_rxtx_queue_parsed(void *parsed_result,
 		return;
 	}
 
-	if (port_id_is_invalid(res->portid))
+	if (port_id_is_invalid(res->portid, ENABLED_WARN))
 		return;
 
 	if (port_is_started(res->portid) != 1) {
@@ -2876,7 +2963,7 @@ cmd_tx_cksum_parsed(void *parsed_result,
 	uint16_t ol_flags, mask = 0;
 	struct rte_eth_dev_info dev_info;
 
-	if (port_id_is_invalid(res->port_id)) {
+	if (port_id_is_invalid(res->port_id, ENABLED_WARN)) {
 		printf("invalid port %d\n", res->port_id);
 		return;
 	}
@@ -3003,7 +3090,7 @@ cmd_tso_set_parsed(void *parsed_result,
 	struct cmd_tso_set_result *res = parsed_result;
 	struct rte_eth_dev_info dev_info;
 
-	if (port_id_is_invalid(res->port_id))
+	if (port_id_is_invalid(res->port_id, ENABLED_WARN))
 		return;
 
 	if (!strcmp(res->mode, "set"))
@@ -3979,10 +4066,8 @@ static void cmd_set_bond_mac_addr_parsed(void *parsed_result,
 	struct cmd_set_bond_mac_addr_result *res = parsed_result;
 	int ret;
 
-	if (res->port_num >= nb_ports) {
-		printf("Port id %d must be less than %d\n", res->port_num, nb_ports);
+	if (port_id_is_invalid(res->port_num, ENABLED_WARN))
 		return;
-	}
 
 	ret = rte_eth_bond_mac_address_set(res->port_num, &res->address);
 
@@ -4219,7 +4304,7 @@ static void cmd_set_promisc_mode_parsed(void *parsed_result,
 
 	/* all ports */
 	if (allports) {
-		for (i = 0; i < nb_ports; i++) {
+		FOREACH_PORT(i, ports) {
 			if (enable)
 				rte_eth_promiscuous_enable(i);
 			else
@@ -4299,7 +4384,7 @@ static void cmd_set_allmulti_mode_parsed(void *parsed_result,
 
 	/* all ports */
 	if (allports) {
-		for (i = 0; i < nb_ports; i++) {
+		FOREACH_PORT(i, ports) {
 			if (enable)
 				rte_eth_allmulticast_enable(i);
 			else
@@ -5484,25 +5569,25 @@ static void cmd_showportall_parsed(void *parsed_result,
 	struct cmd_showportall_result *res = parsed_result;
 	if (!strcmp(res->show, "clear")) {
 		if (!strcmp(res->what, "stats"))
-			for (i = 0; i < nb_ports; i++)
+			FOREACH_PORT(i, ports)
 				nic_stats_clear(i);
 		else if (!strcmp(res->what, "xstats"))
-			for (i = 0; i < nb_ports; i++)
+			FOREACH_PORT(i, ports)
 				nic_xstats_clear(i);
 	} else if (!strcmp(res->what, "info"))
-		for (i = 0; i < nb_ports; i++)
+		FOREACH_PORT(i, ports)
 			port_infos_display(i);
 	else if (!strcmp(res->what, "stats"))
-		for (i = 0; i < nb_ports; i++)
+		FOREACH_PORT(i, ports)
 			nic_stats_display(i);
 	else if (!strcmp(res->what, "xstats"))
-		for (i = 0; i < nb_ports; i++)
+		FOREACH_PORT(i, ports)
 			nic_xstats_display(i);
 	else if (!strcmp(res->what, "fdir"))
-		for (i = 0; i < nb_ports; i++)
+		FOREACH_PORT(i, ports)
 			fdir_get_infos(i);
 	else if (!strcmp(res->what, "stat_qmap"))
-		for (i = 0; i < nb_ports; i++)
+		FOREACH_PORT(i, ports)
 			nic_stats_mapping_display(i);
 }
 
@@ -8756,6 +8841,8 @@ cmdline_parse_ctx_t main_ctx[] = {
 	(cmdline_parse_inst_t *)&cmd_set_qmap,
 	(cmdline_parse_inst_t *)&cmd_operate_port,
 	(cmdline_parse_inst_t *)&cmd_operate_specific_port,
+	(cmdline_parse_inst_t *)&cmd_operate_attach_port,
+	(cmdline_parse_inst_t *)&cmd_operate_detach_port,
 	(cmdline_parse_inst_t *)&cmd_config_speed_all,
 	(cmdline_parse_inst_t *)&cmd_config_speed_specific,
 	(cmdline_parse_inst_t *)&cmd_config_rx_tx,
@@ -8830,7 +8917,7 @@ prompt(void)
 static void
 cmd_reconfig_device_queue(portid_t id, uint8_t dev, uint8_t queue)
 {
-	if (id < nb_ports) {
+	if (!port_id_is_invalid(id, DISABLED_WARN)) {
 		/* check if need_reconfig has been set to 1 */
 		if (ports[id].need_reconfig == 0)
 			ports[id].need_reconfig = dev;
@@ -8840,7 +8927,7 @@ cmd_reconfig_device_queue(portid_t id, uint8_t dev, uint8_t queue)
 	} else {
 		portid_t pid;
 
-		for (pid = 0; pid < nb_ports; pid++) {
+		FOREACH_PORT(pid, ports) {
 			/* check if need_reconfig has been set to 1 */
 			if (ports[pid].need_reconfig == 0)
 				ports[pid].need_reconfig = dev;
@@ -8858,10 +8945,8 @@ bypass_is_supported(portid_t port_id)
 	struct rte_port   *port;
 	struct rte_pci_id *pci_id;
 
-	if (port_id >= nb_ports) {
-		printf("\tPort id must be less than %d.\n", nb_ports);
+	if (port_id_is_invalid(port_id, ENABLED_WARN))
 		return 0;
-	}
 
 	/* Get the device id. */
 	port    = &ports[port_id];
diff --git a/app/test-pmd/config.c b/app/test-pmd/config.c
index c40f819..32d8f9a 100644
--- a/app/test-pmd/config.c
+++ b/app/test-pmd/config.c
@@ -124,11 +124,15 @@ nic_stats_display(portid_t port_id)
 	struct rte_eth_stats stats;
 	struct rte_port *port = &ports[port_id];
 	uint8_t i;
+	portid_t pid;
 
 	static const char *nic_stats_border = "########################";
 
-	if (port_id >= nb_ports) {
-		printf("Invalid port, range is [0, %d]\n", nb_ports - 1);
+	if (port_id_is_invalid(port_id, ENABLED_WARN)) {
+		printf("Valid port range is [0");
+		FOREACH_PORT(pid, ports)
+			printf(", %d", pid);
+		printf("]\n");
 		return;
 	}
 	rte_eth_stats_get(port_id, &stats);
@@ -201,8 +205,13 @@ nic_stats_display(portid_t port_id)
 void
 nic_stats_clear(portid_t port_id)
 {
-	if (port_id >= nb_ports) {
-		printf("Invalid port, range is [0, %d]\n", nb_ports - 1);
+	portid_t pid;
+
+	if (port_id_is_invalid(port_id, ENABLED_WARN)) {
+		printf("Valid port range is [0");
+		FOREACH_PORT(pid, ports)
+			printf(", %d", pid);
+		printf("]\n");
 		return;
 	}
 	rte_eth_stats_reset(port_id);
@@ -249,11 +258,15 @@ nic_stats_mapping_display(portid_t port_id)
 {
 	struct rte_port *port = &ports[port_id];
 	uint16_t i;
+	portid_t pid;
 
 	static const char *nic_stats_mapping_border = "########################";
 
-	if (port_id >= nb_ports) {
-		printf("Invalid port, range is [0, %d]\n", nb_ports - 1);
+	if (port_id_is_invalid(port_id, ENABLED_WARN)) {
+		printf("Valid port range is [0");
+		FOREACH_PORT(pid, ports)
+			printf(", %d", pid);
+		printf("]\n");
 		return;
 	}
 
@@ -302,9 +315,13 @@ port_infos_display(portid_t port_id)
 	int vlan_offload;
 	struct rte_mempool * mp;
 	static const char *info_border = "*********************";
+	portid_t pid;
 
-	if (port_id >= nb_ports) {
-		printf("Invalid port, range is [0, %d]\n", nb_ports - 1);
+	if (port_id_is_invalid(port_id, ENABLED_WARN)) {
+		printf("Valid port range is [0");
+		FOREACH_PORT(pid, ports)
+			printf(", %d", pid);
+		printf("]\n");
 		return;
 	}
 	port = &ports[port_id];
@@ -362,11 +379,14 @@ port_infos_display(portid_t port_id)
 }
 
 int
-port_id_is_invalid(portid_t port_id)
+port_id_is_invalid(portid_t port_id, enum print_warning warning)
 {
-	if (port_id < nb_ports)
+	if (ports[port_id].enabled)
 		return 0;
-	printf("Invalid port %d (must be < nb_ports=%d)\n", port_id, nb_ports);
+
+	if (warning == ENABLED_WARN)
+		printf("Invalid port %d\n", port_id);
+
 	return 1;
 }
 
@@ -425,7 +445,7 @@ port_reg_bit_display(portid_t port_id, uint32_t reg_off, uint8_t bit_x)
 	uint32_t reg_v;
 
 
-	if (port_id_is_invalid(port_id))
+	if (port_id_is_invalid(port_id, ENABLED_WARN))
 		return;
 	if (port_reg_off_is_invalid(port_id, reg_off))
 		return;
@@ -444,7 +464,7 @@ port_reg_bit_field_display(portid_t port_id, uint32_t reg_off,
 	uint8_t  l_bit;
 	uint8_t  h_bit;
 
-	if (port_id_is_invalid(port_id))
+	if (port_id_is_invalid(port_id, ENABLED_WARN))
 		return;
 	if (port_reg_off_is_invalid(port_id, reg_off))
 		return;
@@ -471,7 +491,7 @@ port_reg_display(portid_t port_id, uint32_t reg_off)
 {
 	uint32_t reg_v;
 
-	if (port_id_is_invalid(port_id))
+	if (port_id_is_invalid(port_id, ENABLED_WARN))
 		return;
 	if (port_reg_off_is_invalid(port_id, reg_off))
 		return;
@@ -485,7 +505,7 @@ port_reg_bit_set(portid_t port_id, uint32_t reg_off, uint8_t bit_pos,
 {
 	uint32_t reg_v;
 
-	if (port_id_is_invalid(port_id))
+	if (port_id_is_invalid(port_id, ENABLED_WARN))
 		return;
 	if (port_reg_off_is_invalid(port_id, reg_off))
 		return;
@@ -513,7 +533,7 @@ port_reg_bit_field_set(portid_t port_id, uint32_t reg_off,
 	uint8_t  l_bit;
 	uint8_t  h_bit;
 
-	if (port_id_is_invalid(port_id))
+	if (port_id_is_invalid(port_id, ENABLED_WARN))
 		return;
 	if (port_reg_off_is_invalid(port_id, reg_off))
 		return;
@@ -547,7 +567,7 @@ port_reg_bit_field_set(portid_t port_id, uint32_t reg_off,
 void
 port_reg_set(portid_t port_id, uint32_t reg_off, uint32_t reg_v)
 {
-	if (port_id_is_invalid(port_id))
+	if (port_id_is_invalid(port_id, ENABLED_WARN))
 		return;
 	if (port_reg_off_is_invalid(port_id, reg_off))
 		return;
@@ -560,7 +580,7 @@ port_mtu_set(portid_t port_id, uint16_t mtu)
 {
 	int diag;
 
-	if (port_id_is_invalid(port_id))
+	if (port_id_is_invalid(port_id, ENABLED_WARN))
 		return;
 	diag = rte_eth_dev_set_mtu(port_id, mtu);
 	if (diag == 0)
@@ -723,7 +743,7 @@ rx_ring_desc_display(portid_t port_id, queueid_t rxq_id, uint16_t rxd_id)
 {
 	const struct rte_memzone *rx_mz;
 
-	if (port_id_is_invalid(port_id))
+	if (port_id_is_invalid(port_id, ENABLED_WARN))
 		return;
 	if (rx_queue_id_is_invalid(rxq_id))
 		return;
@@ -740,7 +760,7 @@ tx_ring_desc_display(portid_t port_id, queueid_t txq_id, uint16_t txd_id)
 {
 	const struct rte_memzone *tx_mz;
 
-	if (port_id_is_invalid(port_id))
+	if (port_id_is_invalid(port_id, ENABLED_WARN))
 		return;
 	if (tx_queue_id_is_invalid(txq_id))
 		return;
@@ -796,7 +816,7 @@ port_rss_reta_info(portid_t port_id,
 	uint16_t i, idx, shift;
 	int ret;
 
-	if (port_id_is_invalid(port_id))
+	if (port_id_is_invalid(port_id, ENABLED_WARN))
 		return;
 
 	ret = rte_eth_dev_rss_reta_query(port_id, reta_conf, nb_entries);
@@ -828,7 +848,7 @@ port_rss_hash_conf_show(portid_t port_id, int show_rss_key)
 	uint8_t i;
 	int diag;
 
-	if (port_id_is_invalid(port_id))
+	if (port_id_is_invalid(port_id, ENABLED_WARN))
 		return;
 	/* Get RSS hash key if asked to display it */
 	rss_conf.rss_key = (show_rss_key) ? rss_key : NULL;
@@ -1406,12 +1426,8 @@ set_fwd_ports_list(unsigned int *portlist, unsigned int nb_pt)
  again:
 	for (i = 0; i < nb_pt; i++) {
 		port_id = (portid_t) portlist[i];
-		if (port_id >= nb_ports) {
-			printf("Invalid port id %u >= %u\n",
-			       (unsigned int) port_id,
-			       (unsigned int) nb_ports);
+		if (port_id_is_invalid(port_id, ENABLED_WARN))
 			return;
-		}
 		if (record_now)
 			fwd_ports_ids[i] = port_id;
 	}
@@ -1569,7 +1585,7 @@ vlan_extend_set(portid_t port_id, int on)
 	int diag;
 	int vlan_offload;
 
-	if (port_id_is_invalid(port_id))
+	if (port_id_is_invalid(port_id, ENABLED_WARN))
 		return;
 
 	vlan_offload = rte_eth_dev_get_vlan_offload(port_id);
@@ -1591,7 +1607,7 @@ rx_vlan_strip_set(portid_t port_id, int on)
 	int diag;
 	int vlan_offload;
 
-	if (port_id_is_invalid(port_id))
+	if (port_id_is_invalid(port_id, ENABLED_WARN))
 		return;
 
 	vlan_offload = rte_eth_dev_get_vlan_offload(port_id);
@@ -1612,7 +1628,7 @@ rx_vlan_strip_set_on_queue(portid_t port_id, uint16_t queue_id, int on)
 {
 	int diag;
 
-	if (port_id_is_invalid(port_id))
+	if (port_id_is_invalid(port_id, ENABLED_WARN))
 		return;
 
 	diag = rte_eth_dev_set_vlan_strip_on_queue(port_id, queue_id, on);
@@ -1627,7 +1643,7 @@ rx_vlan_filter_set(portid_t port_id, int on)
 	int diag;
 	int vlan_offload;
 
-	if (port_id_is_invalid(port_id))
+	if (port_id_is_invalid(port_id, ENABLED_WARN))
 		return;
 
 	vlan_offload = rte_eth_dev_get_vlan_offload(port_id);
@@ -1648,7 +1664,7 @@ rx_vft_set(portid_t port_id, uint16_t vlan_id, int on)
 {
 	int diag;
 
-	if (port_id_is_invalid(port_id))
+	if (port_id_is_invalid(port_id, ENABLED_WARN))
 		return;
 	if (vlan_id_is_invalid(vlan_id))
 		return;
@@ -1665,7 +1681,7 @@ rx_vlan_all_filter_set(portid_t port_id, int on)
 {
 	uint16_t vlan_id;
 
-	if (port_id_is_invalid(port_id))
+	if (port_id_is_invalid(port_id, ENABLED_WARN))
 		return;
 	for (vlan_id = 0; vlan_id < 4096; vlan_id++)
 		rx_vft_set(port_id, vlan_id, on);
@@ -1675,7 +1691,7 @@ void
 vlan_tpid_set(portid_t port_id, uint16_t tp_id)
 {
 	int diag;
-	if (port_id_is_invalid(port_id))
+	if (port_id_is_invalid(port_id, ENABLED_WARN))
 		return;
 
 	diag = rte_eth_dev_set_vlan_ether_type(port_id, tp_id);
@@ -1690,7 +1706,7 @@ vlan_tpid_set(portid_t port_id, uint16_t tp_id)
 void
 tx_vlan_set(portid_t port_id, uint16_t vlan_id)
 {
-	if (port_id_is_invalid(port_id))
+	if (port_id_is_invalid(port_id, ENABLED_WARN))
 		return;
 	if (vlan_id_is_invalid(vlan_id))
 		return;
@@ -1701,7 +1717,7 @@ tx_vlan_set(portid_t port_id, uint16_t vlan_id)
 void
 tx_vlan_reset(portid_t port_id)
 {
-	if (port_id_is_invalid(port_id))
+	if (port_id_is_invalid(port_id, ENABLED_WARN))
 		return;
 	ports[port_id].tx_ol_flags &= ~TESTPMD_TX_OFFLOAD_INSERT_VLAN;
 }
@@ -1709,7 +1725,7 @@ tx_vlan_reset(portid_t port_id)
 void
 tx_vlan_pvid_set(portid_t port_id, uint16_t vlan_id, int on)
 {
-	if (port_id_is_invalid(port_id))
+	if (port_id_is_invalid(port_id, ENABLED_WARN))
 		return;
 
 	rte_eth_dev_set_vlan_pvid(port_id, vlan_id, on);
@@ -1721,7 +1737,7 @@ set_qmap(portid_t port_id, uint8_t is_rx, uint16_t queue_id, uint8_t map_value)
 	uint16_t i;
 	uint8_t existing_mapping_found = 0;
 
-	if (port_id_is_invalid(port_id))
+	if (port_id_is_invalid(port_id, ENABLED_WARN))
 		return;
 
 	if (is_rx ? (rx_queue_id_is_invalid(queue_id)) : (tx_queue_id_is_invalid(queue_id)))
@@ -1773,7 +1789,7 @@ fdir_add_signature_filter(portid_t port_id, uint8_t queue_id,
 {
 	int diag;
 
-	if (port_id_is_invalid(port_id))
+	if (port_id_is_invalid(port_id, ENABLED_WARN))
 		return;
 
 	diag = rte_eth_dev_fdir_add_signature_filter(port_id, fdir_filter,
@@ -1791,7 +1807,7 @@ fdir_update_signature_filter(portid_t port_id, uint8_t queue_id,
 {
 	int diag;
 
-	if (port_id_is_invalid(port_id))
+	if (port_id_is_invalid(port_id, ENABLED_WARN))
 		return;
 
 	diag = rte_eth_dev_fdir_update_signature_filter(port_id, fdir_filter,
@@ -1809,7 +1825,7 @@ fdir_remove_signature_filter(portid_t port_id,
 {
 	int diag;
 
-	if (port_id_is_invalid(port_id))
+	if (port_id_is_invalid(port_id, ENABLED_WARN))
 		return;
 
 	diag = rte_eth_dev_fdir_remove_signature_filter(port_id, fdir_filter);
@@ -1881,7 +1897,7 @@ fdir_get_infos(portid_t port_id)
 
 	static const char *fdir_stats_border = "########################";
 
-	if (port_id_is_invalid(port_id))
+	if (port_id_is_invalid(port_id, ENABLED_WARN))
 		return;
 	ret = rte_eth_dev_filter_supported(port_id, RTE_ETH_FILTER_FDIR);
 	if (ret < 0) {
@@ -1955,7 +1971,7 @@ fdir_add_perfect_filter(portid_t port_id, uint16_t soft_id, uint8_t queue_id,
 {
 	int diag;
 
-	if (port_id_is_invalid(port_id))
+	if (port_id_is_invalid(port_id, ENABLED_WARN))
 		return;
 
 	diag = rte_eth_dev_fdir_add_perfect_filter(port_id, fdir_filter,
@@ -1973,7 +1989,7 @@ fdir_update_perfect_filter(portid_t port_id, uint16_t soft_id, uint8_t queue_id,
 {
 	int diag;
 
-	if (port_id_is_invalid(port_id))
+	if (port_id_is_invalid(port_id, ENABLED_WARN))
 		return;
 
 	diag = rte_eth_dev_fdir_update_perfect_filter(port_id, fdir_filter,
@@ -1991,7 +2007,7 @@ fdir_remove_perfect_filter(portid_t port_id, uint16_t soft_id,
 {
 	int diag;
 
-	if (port_id_is_invalid(port_id))
+	if (port_id_is_invalid(port_id, ENABLED_WARN))
 		return;
 
 	diag = rte_eth_dev_fdir_remove_perfect_filter(port_id, fdir_filter,
@@ -2008,7 +2024,7 @@ fdir_set_masks(portid_t port_id, struct rte_fdir_masks *fdir_masks)
 {
 	int diag;
 
-	if (port_id_is_invalid(port_id))
+	if (port_id_is_invalid(port_id, ENABLED_WARN))
 		return;
 
 	diag = rte_eth_dev_fdir_set_masks(port_id, fdir_masks);
@@ -2085,7 +2101,7 @@ set_vf_traffic(portid_t port_id, uint8_t is_rx, uint16_t vf, uint8_t on)
 {
 	int diag;
 
-	if (port_id_is_invalid(port_id))
+	if (port_id_is_invalid(port_id, ENABLED_WARN))
 		return;
 	if (is_rx)
 		diag = rte_eth_dev_set_vf_rx(port_id,vf,on);
@@ -2107,7 +2123,7 @@ set_vf_rx_vlan(portid_t port_id, uint16_t vlan_id, uint64_t vf_mask, uint8_t on)
 {
 	int diag;
 
-	if (port_id_is_invalid(port_id))
+	if (port_id_is_invalid(port_id, ENABLED_WARN))
 		return;
 	if (vlan_id_is_invalid(vlan_id))
 		return;
@@ -2124,7 +2140,7 @@ set_queue_rate_limit(portid_t port_id, uint16_t queue_idx, uint16_t rate)
 	int diag;
 	struct rte_eth_link link;
 
-	if (port_id_is_invalid(port_id))
+	if (port_id_is_invalid(port_id, ENABLED_WARN))
 		return 1;
 	rte_eth_link_get_nowait(port_id, &link);
 	if (rate > link.link_speed) {
@@ -2149,7 +2165,7 @@ set_vf_rate_limit(portid_t port_id, uint16_t vf, uint16_t rate, uint64_t q_msk)
 	if (q_msk == 0)
 		return 0;
 
-	if (port_id_is_invalid(port_id))
+	if (port_id_is_invalid(port_id, ENABLED_WARN))
 		return 1;
 	rte_eth_link_get_nowait(port_id, &link);
 	if (rate > link.link_speed) {
diff --git a/app/test-pmd/parameters.c b/app/test-pmd/parameters.c
index adf3203..6f2af18 100644
--- a/app/test-pmd/parameters.c
+++ b/app/test-pmd/parameters.c
@@ -376,6 +376,7 @@ parse_portnuma_config(const char *q_arg)
 	};
 	unsigned long int_fld[_NUM_FLD];
 	char *str_fld[_NUM_FLD];
+	portid_t pid;
 
 	/* reset from value set at definition */
 	while ((p = strchr(p0,'(')) != NULL) {
@@ -397,8 +398,11 @@ parse_portnuma_config(const char *q_arg)
 				return -1;
 		}
 		port_id = (uint8_t)int_fld[FLD_PORT];
-		if (port_id >= nb_ports) {
-			printf("Invalid port, range is [0, %d]\n", nb_ports - 1);
+		if (port_id_is_invalid(port_id, ENABLED_WARN)) {
+			printf("Valid port range is [0");
+			FOREACH_PORT(pid, ports)
+				printf(", %d", pid);
+			printf("]\n");
 			return -1;
 		}
 		socket_id = (uint8_t)int_fld[FLD_SOCKET];
@@ -429,6 +433,7 @@ parse_ringnuma_config(const char *q_arg)
 	};
 	unsigned long int_fld[_NUM_FLD];
 	char *str_fld[_NUM_FLD];
+	portid_t pid;
 	#define RX_RING_ONLY 0x1
 	#define TX_RING_ONLY 0x2
 	#define RXTX_RING    0x3
@@ -453,8 +458,11 @@ parse_ringnuma_config(const char *q_arg)
 				return -1;
 		}
 		port_id = (uint8_t)int_fld[FLD_PORT];
-		if (port_id >= nb_ports) {
-			printf("Invalid port, range is [0, %d]\n", nb_ports - 1);
+		if (port_id_is_invalid(port_id, ENABLED_WARN)) {
+			printf("Valid port range is [0");
+			FOREACH_PORT(pid, ports)
+				printf(", %d", pid);
+			printf("]\n");
 			return -1;
 		}
 		socket_id = (uint8_t)int_fld[FLD_SOCKET];
@@ -626,12 +634,12 @@ launch_args_parse(int argc, char** argv)
 #endif
 			if (!strcmp(lgopts[opt_idx].name, "nb-ports")) {
 				n = atoi(optarg);
-				if (n > 0 && n <= nb_ports)
+				if (n > 0 &&
+				    !port_id_is_invalid(n, DISABLED_WARN))
 					nb_fwd_ports = (uint8_t) n;
 				else
 					rte_exit(EXIT_FAILURE,
-						 "nb-ports should be > 0 and <= %d\n",
-						 nb_ports);
+						 "Invalid port %d\n", n);
 			}
 			if (!strcmp(lgopts[opt_idx].name, "nb-cores")) {
 				n = atoi(optarg);
diff --git a/app/test-pmd/testpmd.c b/app/test-pmd/testpmd.c
index 773b8af..c18c1a9 100644
--- a/app/test-pmd/testpmd.c
+++ b/app/test-pmd/testpmd.c
@@ -71,6 +71,7 @@
 #include <rte_pci.h>
 #include <rte_ether.h>
 #include <rte_ethdev.h>
+#include <rte_dev.h>
 #include <rte_string_fns.h>
 #ifdef RTE_LIBRTE_PMD_XENVIRT
 #include <rte_eth_xenvirt.h>
@@ -315,7 +316,7 @@ uint16_t nb_rx_queue_stats_mappings = 0;
 
 /* Forward function declarations */
 static void map_port_queue_stats_mapping_registers(uint8_t pi, struct rte_port *port);
-static void check_all_ports_link_status(uint8_t port_num, uint32_t port_mask);
+static void check_all_ports_link_status(uint32_t port_mask);
 
 /*
  * Check if all the ports are started.
@@ -324,6 +325,20 @@ static void check_all_ports_link_status(uint8_t port_num, uint32_t port_mask);
 static int all_ports_started(void);
 
 /*
+ * Find next enabled port
+ */
+portid_t
+find_next_port(portid_t p, struct rte_port *ports, int size)
+{
+	if (ports == NULL)
+		rte_exit(-EINVAL, "failed to find a next port id\n");
+
+	while ((ports[p].enabled == 0) && (p < size))
+		p++;
+	return p;
+}
+
+/*
  * Setup default configuration.
  */
 static void
@@ -552,7 +567,8 @@ init_config(void)
 				+ RTE_TEST_TX_DESC_MAX + MAX_PKT_BURST;
 
 		if (!numa_support)
-			nb_mbuf_per_pool = (nb_mbuf_per_pool * nb_ports);
+			nb_mbuf_per_pool =
+				(nb_mbuf_per_pool * RTE_MAX_ETHPORTS);
 	}
 
 	if (!numa_support) {
@@ -565,14 +581,19 @@ init_config(void)
 
 	/* Configuration of Ethernet ports. */
 	ports = rte_zmalloc("testpmd: ports",
-			    sizeof(struct rte_port) * nb_ports,
+			    sizeof(struct rte_port) * RTE_MAX_ETHPORTS,
 			    RTE_CACHE_LINE_SIZE);
 	if (ports == NULL) {
-		rte_exit(EXIT_FAILURE, "rte_zmalloc(%d struct rte_port) "
-							"failed\n", nb_ports);
+		rte_exit(EXIT_FAILURE,
+				"rte_zmalloc(%d struct rte_port) failed\n",
+				RTE_MAX_ETHPORTS);
 	}
 
-	for (pid = 0; pid < nb_ports; pid++) {
+	/* enabled allocated ports */
+	for (pid = 0; pid < nb_ports; pid++)
+		ports[pid].enabled = 1;
+
+	FOREACH_PORT(pid, ports) {
 		port = &ports[pid];
 		rte_eth_dev_info_get(pid, &port->dev_info);
 
@@ -602,8 +623,7 @@ init_config(void)
 			nb_mbuf_per_pool = nb_mbuf_per_pool/nb_ports;
 
 		for (i = 0; i < MAX_SOCKET; i++) {
-			nb_mbuf = (nb_mbuf_per_pool *
-						port_per_socket[i]);
+			nb_mbuf = (nb_mbuf_per_pool * RTE_MAX_ETHPORTS);
 			if (nb_mbuf)
 				mbuf_pool_create(mbuf_data_size,
 						nb_mbuf,i);
@@ -635,14 +655,6 @@ reconfig(portid_t new_port_id, unsigned socket_id)
 	struct rte_port *port;
 
 	/* Reconfiguration of Ethernet ports. */
-	ports = rte_realloc(ports,
-			    sizeof(struct rte_port) * nb_ports,
-			    RTE_CACHE_LINE_SIZE);
-	if (ports == NULL) {
-		rte_exit(EXIT_FAILURE, "rte_realloc(%d struct rte_port) failed\n",
-				nb_ports);
-	}
-
 	port = &ports[new_port_id];
 	rte_eth_dev_info_get(new_port_id, &port->dev_info);
 
@@ -663,7 +675,7 @@ init_fwd_streams(void)
 	streamid_t sm_id, nb_fwd_streams_new;
 
 	/* set socket id according to numa or not */
-	for (pid = 0; pid < nb_ports; pid++) {
+	FOREACH_PORT(pid, ports) {
 		port = &ports[pid];
 		if (nb_rxq > port->dev_info.max_rx_queues) {
 			printf("Fail: nb_rxq(%d) is greater than "
@@ -1264,7 +1276,7 @@ all_ports_started(void)
 	portid_t pi;
 	struct rte_port *port;
 
-	for (pi = 0; pi < nb_ports; pi++) {
+	FOREACH_PORT(pi, ports) {
 		port = &ports[pi];
 		/* Check if there is a port which is not started */
 		if (port->port_status != RTE_PORT_STARTED)
@@ -1276,6 +1288,45 @@ all_ports_started(void)
 }
 
 int
+all_ports_stopped(void)
+{
+	portid_t pi;
+	struct rte_port *port;
+
+	FOREACH_PORT(pi, ports) {
+		port = &ports[pi];
+		if (port->port_status != RTE_PORT_STOPPED)
+			return 0;
+	}
+
+	return 1;
+}
+
+int
+port_is_started(portid_t port_id)
+{
+	if (port_id_is_invalid(port_id, ENABLED_WARN))
+		return 0;
+
+	if (ports[port_id].port_status != RTE_PORT_STARTED)
+		return 0;
+
+	return 1;
+}
+
+static int
+port_is_closed(portid_t port_id)
+{
+	if (port_id_is_invalid(port_id, ENABLED_WARN))
+		return 0;
+
+	if (ports[port_id].port_status != RTE_PORT_CLOSED)
+		return 0;
+
+	return 1;
+}
+
+int
 start_port(portid_t pid)
 {
 	int diag, need_check_link_status = 0;
@@ -1296,8 +1347,8 @@ start_port(portid_t pid)
 
 	if(dcb_config)
 		dcb_test = 1;
-	for (pi = 0; pi < nb_ports; pi++) {
-		if (pid < nb_ports && pid != pi)
+	FOREACH_PORT(pi, ports) {
+		if (!port_id_is_invalid(pid, DISABLED_WARN) && pid != pi)
 			continue;
 
 		port = &ports[pi];
@@ -1421,7 +1472,7 @@ start_port(portid_t pid)
 	}
 
 	if (need_check_link_status && !no_link_check)
-		check_all_ports_link_status(nb_ports, RTE_PORT_ALL);
+		check_all_ports_link_status(RTE_PORT_ALL);
 	else
 		printf("Please stop the ports first\n");
 
@@ -1446,8 +1497,8 @@ stop_port(portid_t pid)
 	}
 	printf("Stopping ports...\n");
 
-	for (pi = 0; pi < nb_ports; pi++) {
-		if (pid < nb_ports && pid != pi)
+	FOREACH_PORT(pi, ports) {
+		if (!port_id_is_invalid(pid, DISABLED_WARN) && pid != pi)
 			continue;
 
 		port = &ports[pi];
@@ -1463,7 +1514,7 @@ stop_port(portid_t pid)
 		need_check_link_status = 1;
 	}
 	if (need_check_link_status && !no_link_check)
-		check_all_ports_link_status(nb_ports, RTE_PORT_ALL);
+		check_all_ports_link_status(RTE_PORT_ALL);
 
 	printf("Done\n");
 }
@@ -1481,8 +1532,8 @@ close_port(portid_t pid)
 
 	printf("Closing ports...\n");
 
-	for (pi = 0; pi < nb_ports; pi++) {
-		if (pid < nb_ports && pid != pi)
+	FOREACH_PORT(pi, ports) {
+		if (!port_id_is_invalid(pid, DISABLED_WARN) && pid != pi)
 			continue;
 
 		port = &ports[pi];
@@ -1502,31 +1553,83 @@ close_port(portid_t pid)
 	printf("Done\n");
 }
 
-int
-all_ports_stopped(void)
+void
+attach_port(char *identifier)
 {
-	portid_t pi;
-	struct rte_port *port;
+	portid_t i, j, pi = 0;
 
-	for (pi = 0; pi < nb_ports; pi++) {
-		port = &ports[pi];
-		if (port->port_status != RTE_PORT_STOPPED)
-			return 0;
+	printf("Attaching a new port...\n");
+
+	if (identifier == NULL) {
+		printf("Invalid parameters are speficied\n");
+		return;
 	}
 
-	return 1;
+	if (test_done == 0) {
+		printf("Please stop forwarding first\n");
+		return;
+	}
+
+	if (rte_eal_dev_attach(identifier, &pi))
+		return;
+
+	ports[pi].enabled = 1;
+	reconfig(pi, rte_eth_dev_socket_id(pi));
+	rte_eth_promiscuous_enable(pi);
+
+	nb_ports = rte_eth_dev_count();
+
+	/* set_default_fwd_ports_config(); */
+	bzero(fwd_ports_ids, sizeof(fwd_ports_ids));
+	i = 0;
+	FOREACH_PORT(j, ports) {
+		fwd_ports_ids[i] = j;
+		i++;
+	}
+	nb_cfg_ports = nb_ports;
+	nb_fwd_ports++;
+
+	ports[pi].port_status = RTE_PORT_STOPPED;
+
+	printf("Port %d is attached. Now total ports is %d\n", pi, nb_ports);
+	printf("Done\n");
 }
 
-int
-port_is_started(portid_t port_id)
+void
+detach_port(uint8_t port_id)
 {
-	if (port_id_is_invalid(port_id))
-		return -1;
+	portid_t i, pi = 0;
+	char name[RTE_ETH_NAME_MAX_LEN];
 
-	if (ports[port_id].port_status != RTE_PORT_STARTED)
-		return 0;
+	printf("Detaching a port...\n");
 
-	return 1;
+	if (!port_is_closed(port_id)) {
+		printf("Please close port first\n");
+		return;
+	}
+
+	rte_eth_promiscuous_disable(port_id);
+
+	if (rte_eal_dev_detach(port_id, name))
+		return;
+
+	ports[port_id].enabled = 0;
+	nb_ports = rte_eth_dev_count();
+
+	/* set_default_fwd_ports_config(); */
+	bzero(fwd_ports_ids, sizeof(fwd_ports_ids));
+	i = 0;
+	FOREACH_PORT(pi, ports) {
+		fwd_ports_ids[i] = pi;
+		i++;
+	}
+	nb_cfg_ports = nb_ports;
+	nb_fwd_ports--;
+
+	printf("Port '%s' is detached. Now total ports is %d\n",
+			name, nb_ports);
+	printf("Done\n");
+	return;
 }
 
 void
@@ -1534,7 +1637,7 @@ pmd_test_exit(void)
 {
 	portid_t pt_id;
 
-	for (pt_id = 0; pt_id < nb_ports; pt_id++) {
+	FOREACH_PORT(pt_id, ports) {
 		printf("Stopping port %d...", pt_id);
 		fflush(stdout);
 		rte_eth_dev_close(pt_id);
@@ -1553,7 +1656,7 @@ struct pmd_test_command {
 
 /* Check the link status of all ports in up to 9s, and print them finally */
 static void
-check_all_ports_link_status(uint8_t port_num, uint32_t port_mask)
+check_all_ports_link_status(uint32_t port_mask)
 {
 #define CHECK_INTERVAL 100 /* 100ms */
 #define MAX_CHECK_TIME 90 /* 9s (90 * 100ms) in total */
@@ -1564,7 +1667,7 @@ check_all_ports_link_status(uint8_t port_num, uint32_t port_mask)
 	fflush(stdout);
 	for (count = 0; count <= MAX_CHECK_TIME; count++) {
 		all_ports_up = 1;
-		for (portid = 0; portid < port_num; portid++) {
+		FOREACH_PORT(portid, ports) {
 			if ((port_mask & (1 << portid)) == 0)
 				continue;
 			memset(&link, 0, sizeof(link));
@@ -1688,7 +1791,7 @@ init_port_config(void)
 	portid_t pid;
 	struct rte_port *port;
 
-	for (pid = 0; pid < nb_ports; pid++) {
+	FOREACH_PORT(pid, ports) {
 		port = &ports[pid];
 		port->dev_conf.rxmode = rx_mode;
 		port->dev_conf.fdir_conf = fdir_conf;
@@ -1877,7 +1980,7 @@ main(int argc, char** argv)
 
 	nb_ports = (portid_t) rte_eth_dev_count();
 	if (nb_ports == 0)
-		rte_exit(EXIT_FAILURE, "No probed ethernet device\n");
+		RTE_LOG(WARNING, EAL, "No probed ethernet devices\n");
 
 	set_def_fwd_config();
 	if (nb_lcores == 0)
@@ -1899,7 +2002,7 @@ main(int argc, char** argv)
 		rte_exit(EXIT_FAILURE, "Start ports failed\n");
 
 	/* set all ports to promiscuous mode by default */
-	for (port_id = 0; port_id < nb_ports; port_id++)
+	FOREACH_PORT(port_id, ports)
 		rte_eth_promiscuous_enable(port_id);
 
 #ifdef RTE_LIBRTE_CMDLINE
diff --git a/app/test-pmd/testpmd.h b/app/test-pmd/testpmd.h
index 8f5e6c7..109c670 100644
--- a/app/test-pmd/testpmd.h
+++ b/app/test-pmd/testpmd.h
@@ -134,6 +134,7 @@ struct fwd_stream {
  * The data structure associated with each port.
  */
 struct rte_port {
+	uint8_t                 enabled;    /**< Port enabled or not */
 	struct rte_eth_dev_info dev_info;   /**< PCI info + driver name */
 	struct rte_eth_conf     dev_conf;   /**< Port configuration. */
 	struct ether_addr       eth_addr;   /**< Port ethernet address */
@@ -159,6 +160,14 @@ struct rte_port {
 	struct rte_eth_txconf   tx_conf;    /**< tx configuration */
 };
 
+extern portid_t __rte_unused
+find_next_port(portid_t p, struct rte_port *ports, int size);
+
+#define FOREACH_PORT(p, ports) \
+	for (p = find_next_port(0, ports, RTE_MAX_ETHPORTS); \
+	    p < RTE_MAX_ETHPORTS; \
+	    p = find_next_port(p + 1, ports, RTE_MAX_ETHPORTS))
+
 /**
  * The data structure associated with each forwarding logical core.
  * The logical cores are internally numbered by a core index from 0 to
@@ -515,6 +524,8 @@ int init_port_dcb_config(portid_t pid,struct dcb_config *dcb_conf);
 int start_port(portid_t pid);
 void stop_port(portid_t pid);
 void close_port(portid_t pid);
+void attach_port(char *identifier);
+void detach_port(uint8_t port_id);
 int all_ports_stopped(void);
 int port_is_started(portid_t port_id);
 void pmd_test_exit(void);
@@ -558,10 +569,15 @@ void get_ethertype_filter(uint8_t port_id, uint16_t index);
 void get_2tuple_filter(uint8_t port_id, uint16_t index);
 void get_5tuple_filter(uint8_t port_id, uint16_t index);
 void get_flex_filter(uint8_t port_id, uint16_t index);
-int port_id_is_invalid(portid_t port_id);
 int rx_queue_id_is_invalid(queueid_t rxq_id);
 int tx_queue_id_is_invalid(queueid_t txq_id);
 
+enum print_warning {
+	ENABLED_WARN = 0,
+	DISABLED_WARN
+};
+int port_id_is_invalid(portid_t port_id, enum print_warning warning);
+
 /*
  * Work-around of a compilation error with ICC on invocations of the
  * rte_be_to_cpu_16() function.
diff --git a/doc/guides/testpmd_app_ug/testpmd_funcs.rst b/doc/guides/testpmd_app_ug/testpmd_funcs.rst
index 218835a..1cacbcf 100644
--- a/doc/guides/testpmd_app_ug/testpmd_funcs.rst
+++ b/doc/guides/testpmd_app_ug/testpmd_funcs.rst
@@ -808,6 +808,63 @@ The following sections show functions for configuring ports.
 
     Port configuration changes only become active when forwarding is started/restarted.
 
+port attach
+~~~~~~~~~~~
+
+Attach a port specified by pci address or virtual device args.
+
+To attach a new pci device, the device should be recognized by kernel first.
+Then it should be moved under DPDK management.
+Finally the port can be attached to testpmd.
+On the other hand, to attach a port created by virtual device, above steps are not needed.
+
+port attach (identifier)
+
+For example, to attach a port that pci address is 0000:02:00.0.
+
+.. code-block:: console
+
+    testpmd> port attach 0000:02:00.0
+    Attaching a new port...
+    ... snip ...
+    Port 0 is attached. Now total ports is 1
+    Done
+
+For example, to attach a port created by pcap PMD.
+
+.. code-block:: console
+
+    testpmd> port attach eth_pcap0,iface=eth0
+    Attaching a new port...
+    ... snip ...
+    Port 0 is attached. Now total ports is 1
+    Done
+
+In this case, identifier is "eth_pcap0,iface=eth0".
+This identifier format is the same as "--vdev" format of DPDK applications.
+
+port detach
+~~~~~~~~~~~
+
+Detach a specific port.
+
+Before detaching a port, the port should be closed.
+Also to remove a pci device completely from the system, first detach the port from testpmd.
+Then the device should be moved under kernel management.
+Finally the device can be remove using kernel pci hotplug functionality.
+On the other hand, to remove a port created by virtual device, above steps are not needed.
+
+port detach (port_id)
+
+For example, to detach a port 0.
+
+.. code-block:: console
+
+    testpmd> port detach 0
+    Detaching a port...
+    ... snip ...
+    Done
+
 port start
 ~~~~~~~~~~
 
-- 
1.9.1

^ permalink raw reply	[flat|nested] 362+ messages in thread

* [dpdk-dev] [PATCH v6 00/13] Port Hotplug Framework
  2015-01-19 10:40   ` [dpdk-dev] [PATCH v4 08/11] eal/pci: Cleanup pci driver initialization code Tetsuya Mukawa
  2015-01-30  5:42     ` [dpdk-dev] [PATCH v5 00/13] Port Hotplug Framework Tetsuya Mukawa
@ 2015-02-01  4:01     ` Tetsuya Mukawa
  2015-02-01  4:01       ` [dpdk-dev] [PATCH v6 01/13] eal_pci: Add flag to hold kernel driver type Tetsuya Mukawa
                         ` (16 more replies)
  1 sibling, 17 replies; 362+ messages in thread
From: Tetsuya Mukawa @ 2015-02-01  4:01 UTC (permalink / raw)
  To: dev

This patch series adds a dynamic port hotplug framework to DPDK.
With the patches, DPDK apps can attach or detach ports at runtime.

The basic concept of the port hotplug is like followings.
- DPDK apps must have responsibility to manage ports.
  DPDK apps only know which ports are attached or detached at the moment.
  The port hotplug framework is implemented to allow DPDK apps to manage ports.
  For example, when DPDK apps call port attach function, attached port number
  will be returned. Also DPDK apps can detach port by port number.
- Kernel support is needed for attaching or detaching physical device ports.
  To attach a new physical device port, the device will be recognized by
  userspace directly I/O framework in kernel at first. Then DPDK apps can
  call the port hotplug functions to attach ports.
  For detaching, steps are vice versa.
- Before detach ports, ports must be stopped and closed.
  DPDK application must call rte_eth_dev_stop() and rte_eth_dev_close() before
  detaching ports. These function will call finalization codes of PMDs.
  But so far, no PMD frees all resources allocated by initialization.
  It means PMDs are needed to be fixed to support the port hotplug.
  'RTE_PCI_DRV_DETACHABLE' is a new flag indicating a PMD supports detaching.
  Without this flag, detaching will be failed.
- Mustn't affect legacy DPDK apps.
  No DPDK EAL behavior is changed, if the port hotplug functions are't called.
  So all legacy DPDK apps can still work without modifications.

And a few limitations.
- The port hotplug functions are not thread safe.
  DPDK apps should handle it.
- Only support Linux and igb_uio so far.
  BSD and VFIO is not supported. I will send VFIO patches at least, but I don't
  have a plan to submit BSD patch so far.


Here is port hotplug APIs.
-------------------------------------------------------------------------------
/**
 * Attach a new device.
 *
 * @param devargs
 *   A pointer to a strings array describing the new device
 *   to be attached. The strings should be a pci address like
 *   '0000:01:00.0' or virtual device name like 'eth_pcap0'.
 * @param port_id
 *  A pointer to a port identifier actually attached.
 * @return
 *  0 on success and port_id is filled, negative on error
 */
int rte_eal_dev_attach(const char *devargs, uint8_t *port_id);

/**
 * Detach a device.
 *
 * @param port_id
 *   The port identifier of the device to detach.
 * @param addr
 *  A pointer to a device name actually detached.
 * @return
 *  0 on success and devname is filled, negative on error
 */
int rte_eal_dev_detach(uint8_t port_id, char *devname);
-------------------------------------------------------------------------------

This patch series are for DPDK EAL. To use port hotplug function by DPDK apps,
each PMD should be fixed to support 'RTE_PCI_DRV_DETACHABLE' flag. Please check
a patch for pcap PMD.

Also please check testpmd patch. It will show you how to fix your legacy
applications to support port hotplug feature.

PATCH v6 changes
 - Fix rte_eth_dev_uninit() to handle a return value of uninit
   function of PMD. To do this, below changes also be applied.
   - Fix a paramter of rte_eth_dev_free().
   - Use rte_eth_dev structure as the paramter of rte_eth_dev_free().

PATCH v5 changes
 - Add runtime check passthrough driver type, like vfio-pci, igb_uio
   and uio_pci_generic.
   This was done by Qiu, Michael. Thanks a lot.
 - Change function names like below.
   - rte_eal_dev_find_and_invoke() to rte_eal_vdev_find_and_invoke().
   - rte_eal_dev_invoke() to rte_eal_vdev_invoke().
 - Add code to handle a return value of rte_eal_devargs_remove().
 - Fix pci address format in rte_eal_dev_detach().
 - Remove RTE_EAL_INVOKE_TYPE_UNKNOWN, because it's unused.
 - Change function definition of rte_eal_devargs_remove().
 - Fix pci_unmap_device() to check pt_driver.
 - Fix return value of below functions.
   - rte_eth_dev_get_changed_port().
   - rte_eth_dev_get_port_by_addr().
 - Change paramters of rte_eth_dev_validate_port() to cleanup code.
 - Fix pci_scan_one to handle pt_driver correctly.
   (Thanks to Qiu, Michael for above sugesstions)

PATCH v4 changes
 - Merge patches to review easier.
 - Fix indent of 'if' statement.
 - Fix calculation method of eal_compare_pci_addr().
 - Fix header file declaration.
 - Add header file to determine if hotplug can be enabled.
   (Thanks to Qiu, Michael)
 - Use braces with 'for' loop.
 - Add paramerter checking.
 - Fix sanity check code
 - Fix comments of rte_eth_dev_type.
 - Change function names.
   (Thanks to Iremonger, Bernard)

PATCH v3 changes:
 - Fix enum definition used in rte_ethdev.c.
   (Thanks to Zhang, Helin)

PATCH v2 changes:
 - Replace rte_eal_dev_attach_pdev(), rte_eal_dev_detach_pdev,
   rte_eal_dev_attach_vdev() and rte_eal_dev_detach_vdev() to
   rte_eal_dev_attach() and rte_eal_dev_detach().
 - Add parameter values checking.
 - Refashion a few functions.
   (Thanks to Iremonger, Bernard)

PATCH v1 Changes:
 - Fix error checking code of librte_eth APIs.
 - Fix issue that port from pcap PMD cannot be detached correctly.
 - Fix issue that testpmd could hang after forwarding, if attaching and detaching
   is repeatedly.
 - Fix if-condition of rte_eth_dev_get_port_by_addr().
   (Thanks to Mark Enright)

RFC PATCH v2 Changes:
- remove 'rte_eth_dev_validate_port()', and cleanup codes.


Michael Qiu (2):
  eal_pci: Add flag to hold kernel driver type
  eal_pci: pci memory map work with driver type

Tetsuya Mukawa (11):
  eal/pci,ethdev: Remove assumption that port will not be detached
  eal/pci: Consolidate pci address comparison APIs
  ethdev: Add rte_eth_dev_free to free specified device
  eal,ethdev: Add a function and function pointers to close ether device
  ethdev: Add functions that will be used by port hotplug functions
  eal/linux/pci: Add functions for unmapping igb_uio resources
  eal/pci: Add a function to remove the entry of devargs list
  eal/pci: Cleanup pci driver initialization code
  ethdev: Add one dev_type paramerter to rte_eth_dev_allocate
  eal/pci: Add rte_eal_dev_attach/detach() functions
  eal: Enable port hotplug framework in Linux

 app/test/virtual_pmd.c                          |   2 +-
 config/common_linuxapp                          |   5 +
 lib/librte_eal/bsdapp/eal/eal_pci.c             |  25 +-
 lib/librte_eal/common/Makefile                  |   1 +
 lib/librte_eal/common/eal_common_dev.c          | 274 +++++++++++
 lib/librte_eal/common/eal_common_devargs.c      |  60 +++
 lib/librte_eal/common/eal_common_pci.c          |  92 +++-
 lib/librte_eal/common/eal_private.h             |  35 ++
 lib/librte_eal/common/include/rte_dev.h         |  33 ++
 lib/librte_eal/common/include/rte_dev_hotplug.h |  44 ++
 lib/librte_eal/common/include/rte_devargs.h     |  21 +
 lib/librte_eal/common/include/rte_pci.h         |  84 ++++
 lib/librte_eal/linuxapp/eal/Makefile            |   1 +
 lib/librte_eal/linuxapp/eal/eal_pci.c           | 227 +++++++--
 lib/librte_eal/linuxapp/eal/eal_pci_init.h      |   8 +
 lib/librte_eal/linuxapp/eal/eal_pci_uio.c       |  67 ++-
 lib/librte_ether/rte_ethdev.c                   | 624 +++++++++++++-----------
 lib/librte_ether/rte_ethdev.h                   | 148 +++++-
 lib/librte_pmd_af_packet/rte_eth_af_packet.c    |   2 +-
 lib/librte_pmd_bond/rte_eth_bond_api.c          |   2 +-
 lib/librte_pmd_pcap/rte_eth_pcap.c              |   2 +-
 lib/librte_pmd_ring/rte_eth_ring.c              |   2 +-
 lib/librte_pmd_xenvirt/rte_eth_xenvirt.c        |   2 +-
 23 files changed, 1414 insertions(+), 347 deletions(-)
 create mode 100644 lib/librte_eal/common/include/rte_dev_hotplug.h

-- 
1.9.1

^ permalink raw reply	[flat|nested] 362+ messages in thread

* [dpdk-dev] [PATCH v6 01/13] eal_pci: Add flag to hold kernel driver type
  2015-02-01  4:01     ` [dpdk-dev] [PATCH v6 00/13] Port Hotplug Framework Tetsuya Mukawa
@ 2015-02-01  4:01       ` Tetsuya Mukawa
  2015-02-01  4:01       ` [dpdk-dev] [PATCH v6 02/13] eal_pci: pci memory map work with " Tetsuya Mukawa
                         ` (15 subsequent siblings)
  16 siblings, 0 replies; 362+ messages in thread
From: Tetsuya Mukawa @ 2015-02-01  4:01 UTC (permalink / raw)
  To: dev

From: Michael Qiu <michael.qiu@intel.com>

Currently, dpdk has no ability to know which type of driver(
vfio-pci/igb_uio/uio_pci_generic) the device used. It only can
check whether vfio is enabled or not staticly.

It really useful to have the flag, becasue different type need to
handle differently in runtime. For example, pci memory map,
pot hotplug, and so on.

This patch add a flag field for pci device to solve above issue.

Signed-off-by: Michael Qiu <michael.qiu@intel.com>
Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp>
---
 lib/librte_eal/common/include/rte_pci.h |  8 +++++
 lib/librte_eal/linuxapp/eal/eal_pci.c   | 53 +++++++++++++++++++++++++++++++--
 2 files changed, 59 insertions(+), 2 deletions(-)

diff --git a/lib/librte_eal/common/include/rte_pci.h b/lib/librte_eal/common/include/rte_pci.h
index 66ed793..7b48b55 100644
--- a/lib/librte_eal/common/include/rte_pci.h
+++ b/lib/librte_eal/common/include/rte_pci.h
@@ -139,6 +139,13 @@ struct rte_pci_addr {
 
 struct rte_devargs;
 
+enum rte_pt_driver {
+	RTE_PT_UNKNOWN		= 0,
+	RTE_PT_IGB_UIO		= 1,
+	RTE_PT_VFIO		= 2,
+	RTE_PT_UIO_GENERIC	= 3,
+};
+
 /**
  * A structure describing a PCI device.
  */
@@ -152,6 +159,7 @@ struct rte_pci_device {
 	uint16_t max_vfs;                       /**< sriov enable if not zero */
 	int numa_node;                          /**< NUMA node connection */
 	struct rte_devargs *devargs;            /**< Device user arguments */
+	enum rte_pt_driver pt_driver;		/**< Driver of passthrough */
 };
 
 /** Any PCI device identifier (vendor, device, ...) */
diff --git a/lib/librte_eal/linuxapp/eal/eal_pci.c b/lib/librte_eal/linuxapp/eal/eal_pci.c
index b5f5410..bd3f77d 100644
--- a/lib/librte_eal/linuxapp/eal/eal_pci.c
+++ b/lib/librte_eal/linuxapp/eal/eal_pci.c
@@ -97,6 +97,35 @@ error:
 	return -1;
 }
 
+static int
+pci_get_kernel_driver_by_path(const char *filename, char *dri_name)
+{
+	int count;
+	char path[PATH_MAX];
+	char *name;
+
+	if (!filename || !dri_name)
+		return -1;
+
+	count = readlink(filename, path, PATH_MAX);
+	if (count >= PATH_MAX)
+		return -1;
+
+	/* For device does not have a driver */
+	if (count < 0)
+		return 1;
+
+	path[count] = '\0';
+
+	name = strrchr(path, '/');
+	if (name) {
+		strncpy(dri_name, name + 1, strlen(name + 1) + 1);
+		return 0;
+	}
+
+	return -1;
+}
+
 void *
 pci_find_max_end_va(void)
 {
@@ -222,11 +251,12 @@ pci_scan_one(const char *dirname, uint16_t domain, uint8_t bus,
 	char filename[PATH_MAX];
 	unsigned long tmp;
 	struct rte_pci_device *dev;
+	char driver[PATH_MAX];
+	int ret;
 
 	dev = malloc(sizeof(*dev));
-	if (dev == NULL) {
+	if (dev == NULL)
 		return -1;
-	}
 
 	memset(dev, 0, sizeof(*dev));
 	dev->addr.domain = domain;
@@ -298,6 +328,25 @@ pci_scan_one(const char *dirname, uint16_t domain, uint8_t bus,
 		return -1;
 	}
 
+	/* parse driver */
+	snprintf(filename, sizeof(filename), "%s/driver", dirname);
+	ret = pci_get_kernel_driver_by_path(filename, driver);
+	if (!ret) {
+		if (!strcmp(driver, "vfio-pci"))
+			dev->pt_driver = RTE_PT_VFIO;
+		else if (!strcmp(driver, "igb_uio"))
+			dev->pt_driver = RTE_PT_IGB_UIO;
+		else if (!strcmp(driver, "uio_pci_generic"))
+			dev->pt_driver = RTE_PT_UIO_GENERIC;
+		else
+			dev->pt_driver = RTE_PT_UNKNOWN;
+	} else if (ret < 0) {
+		RTE_LOG(ERR, EAL, "Fail to get kernel driver\n");
+		free(dev);
+		return -1;
+	} else
+		dev->pt_driver = RTE_PT_UNKNOWN;
+
 	/* device is valid, add in list (sorted) */
 	if (TAILQ_EMPTY(&pci_device_list)) {
 		TAILQ_INSERT_TAIL(&pci_device_list, dev, next);
-- 
1.9.1

^ permalink raw reply	[flat|nested] 362+ messages in thread

* [dpdk-dev] [PATCH v6 02/13] eal_pci: pci memory map work with driver type
  2015-02-01  4:01     ` [dpdk-dev] [PATCH v6 00/13] Port Hotplug Framework Tetsuya Mukawa
  2015-02-01  4:01       ` [dpdk-dev] [PATCH v6 01/13] eal_pci: Add flag to hold kernel driver type Tetsuya Mukawa
@ 2015-02-01  4:01       ` Tetsuya Mukawa
  2015-02-01  4:01       ` [dpdk-dev] [PATCH v6 03/13] eal/pci, ethdev: Remove assumption that port will not be detached Tetsuya Mukawa
                         ` (14 subsequent siblings)
  16 siblings, 0 replies; 362+ messages in thread
From: Tetsuya Mukawa @ 2015-02-01  4:01 UTC (permalink / raw)
  To: dev

From: Michael Qiu <michael.qiu@intel.com>

With the driver type flag in struct rte_pci_dev, we do not need
to always  map uio devices with vfio related function when
vfio enabled.

Signed-off-by: Michael Qiu <michael.qiu@intel.com>
Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp>
---
 lib/librte_eal/linuxapp/eal/eal_pci.c | 30 +++++++++++++++++-------------
 1 file changed, 17 insertions(+), 13 deletions(-)

diff --git a/lib/librte_eal/linuxapp/eal/eal_pci.c b/lib/librte_eal/linuxapp/eal/eal_pci.c
index bd3f77d..c0ca5a5 100644
--- a/lib/librte_eal/linuxapp/eal/eal_pci.c
+++ b/lib/librte_eal/linuxapp/eal/eal_pci.c
@@ -549,25 +549,29 @@ pci_config_space_set(struct rte_pci_device *dev)
 static int
 pci_map_device(struct rte_pci_device *dev)
 {
-	int ret, mapped = 0;
+	int ret = -1;
 
 	/* try mapping the NIC resources using VFIO if it exists */
+	switch (dev->pt_driver) {
+	case RTE_PT_VFIO:
 #ifdef VFIO_PRESENT
-	if (pci_vfio_is_enabled()) {
-		ret = pci_vfio_map_resource(dev);
-		if (ret == 0)
-			mapped = 1;
-		else if (ret < 0)
-			return ret;
-	}
+		if (pci_vfio_is_enabled())
+			ret = pci_vfio_map_resource(dev);
 #endif
-	/* map resources for devices that use igb_uio */
-	if (!mapped) {
+		break;
+	case RTE_PT_IGB_UIO:
+	case RTE_PT_UIO_GENERIC:
+		/* map resources for devices that use uio */
 		ret = pci_uio_map_resource(dev);
-		if (ret != 0)
-			return ret;
+		break;
+	default:
+		RTE_LOG(DEBUG, EAL, "  Not managed by known pt driver,"
+			" skipped\n");
+		ret = 1;
+		break;
 	}
-	return 0;
+
+	return ret;
 }
 
 /*
-- 
1.9.1

^ permalink raw reply	[flat|nested] 362+ messages in thread

* [dpdk-dev] [PATCH v6 03/13] eal/pci, ethdev: Remove assumption that port will not be detached
  2015-02-01  4:01     ` [dpdk-dev] [PATCH v6 00/13] Port Hotplug Framework Tetsuya Mukawa
  2015-02-01  4:01       ` [dpdk-dev] [PATCH v6 01/13] eal_pci: Add flag to hold kernel driver type Tetsuya Mukawa
  2015-02-01  4:01       ` [dpdk-dev] [PATCH v6 02/13] eal_pci: pci memory map work with " Tetsuya Mukawa
@ 2015-02-01  4:01       ` Tetsuya Mukawa
  2015-02-09  8:30         ` [dpdk-dev] [PATCH v7 00/14] Port Hotplug Framework Tetsuya Mukawa
  2015-02-01  4:01       ` [dpdk-dev] [PATCH v6 04/13] eal/pci: Consolidate pci address comparison APIs Tetsuya Mukawa
                         ` (13 subsequent siblings)
  16 siblings, 1 reply; 362+ messages in thread
From: Tetsuya Mukawa @ 2015-02-01  4:01 UTC (permalink / raw)
  To: dev

To remove assumption, do like followings.

This patch adds "RTE_PCI_DRV_DETACHABLE" to drv_flags of rte_pci_driver
structure. The flags indicates the driver can detach devices at runtime.
Also remove assumption that port will not be detached.

To remove the assumption.
- Add 'attached' member to rte_eth_dev structure.
  This member is used for indicating the port is attached, or not.
- Add rte_eth_dev_allocate_new_port().
  This function is used for allocating new port.

v5:
- Change paramters of rte_eth_dev_validate_port() to cleanup code.
v4:
- Use braces with 'for' loop.
- Fix indent of 'if' statement.

Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp>
---
 lib/librte_eal/common/include/rte_pci.h |   2 +
 lib/librte_ether/rte_ethdev.c           | 454 +++++++++++++-------------------
 lib/librte_ether/rte_ethdev.h           |   5 +
 3 files changed, 186 insertions(+), 275 deletions(-)

diff --git a/lib/librte_eal/common/include/rte_pci.h b/lib/librte_eal/common/include/rte_pci.h
index 7b48b55..7f2d699 100644
--- a/lib/librte_eal/common/include/rte_pci.h
+++ b/lib/librte_eal/common/include/rte_pci.h
@@ -207,6 +207,8 @@ struct rte_pci_driver {
 #define RTE_PCI_DRV_FORCE_UNBIND 0x0004
 /** Device driver supports link state interrupt */
 #define RTE_PCI_DRV_INTR_LSC	0x0008
+/** Device driver supports detaching capability */
+#define RTE_PCI_DRV_DETACHABLE	0x0010
 
 /**< Internal use only - Macro used by pci addr parsing functions **/
 #define GET_PCIADDR_FIELD(in, fd, lim, dlm)                   \
diff --git a/lib/librte_ether/rte_ethdev.c b/lib/librte_ether/rte_ethdev.c
index ea3a1fb..d70854f 100644
--- a/lib/librte_ether/rte_ethdev.c
+++ b/lib/librte_ether/rte_ethdev.c
@@ -175,6 +175,16 @@ enum {
 	STAT_QMAP_RX
 };
 
+enum {
+	DEV_INVALID = 0,
+	DEV_VALID,
+};
+
+enum {
+	DEV_DISCONNECTED = 0,
+	DEV_CONNECTED
+};
+
 static inline void
 rte_eth_dev_data_alloc(void)
 {
@@ -201,19 +211,34 @@ rte_eth_dev_allocated(const char *name)
 {
 	unsigned i;
 
-	for (i = 0; i < nb_ports; i++) {
-		if (strcmp(rte_eth_devices[i].data->name, name) == 0)
+	for (i = 0; i < RTE_MAX_ETHPORTS; i++) {
+		if ((rte_eth_devices[i].attached == DEV_CONNECTED) &&
+		    strcmp(rte_eth_devices[i].data->name, name) == 0)
 			return &rte_eth_devices[i];
 	}
 	return NULL;
 }
 
+static uint8_t
+rte_eth_dev_allocate_new_port(void)
+{
+	unsigned i;
+
+	for (i = 0; i < RTE_MAX_ETHPORTS; i++) {
+		if (rte_eth_devices[i].attached == DEV_DISCONNECTED)
+			return i;
+	}
+	return RTE_MAX_ETHPORTS;
+}
+
 struct rte_eth_dev *
 rte_eth_dev_allocate(const char *name)
 {
+	uint8_t port_id;
 	struct rte_eth_dev *eth_dev;
 
-	if (nb_ports == RTE_MAX_ETHPORTS) {
+	port_id = rte_eth_dev_allocate_new_port();
+	if (port_id == RTE_MAX_ETHPORTS) {
 		PMD_DEBUG_TRACE("Reached maximum number of Ethernet ports\n");
 		return NULL;
 	}
@@ -226,10 +251,12 @@ rte_eth_dev_allocate(const char *name)
 		return NULL;
 	}
 
-	eth_dev = &rte_eth_devices[nb_ports];
-	eth_dev->data = &rte_eth_dev_data[nb_ports];
+	eth_dev = &rte_eth_devices[port_id];
+	eth_dev->data = &rte_eth_dev_data[port_id];
 	snprintf(eth_dev->data->name, sizeof(eth_dev->data->name), "%s", name);
-	eth_dev->data->port_id = nb_ports++;
+	eth_dev->data->port_id = port_id;
+	eth_dev->attached = DEV_CONNECTED;
+	nb_ports++;
 	return eth_dev;
 }
 
@@ -283,6 +310,7 @@ rte_eth_dev_init(struct rte_pci_driver *pci_drv,
 			(unsigned) pci_dev->id.device_id);
 	if (rte_eal_process_type() == RTE_PROC_PRIMARY)
 		rte_free(eth_dev->data->dev_private);
+	eth_dev->attached = DEV_DISCONNECTED;
 	nb_ports--;
 	return diag;
 }
@@ -308,10 +336,28 @@ rte_eth_driver_register(struct eth_driver *eth_drv)
 	rte_eal_pci_register(&eth_drv->pci_drv);
 }
 
+enum {
+	NONE_TRACE = 0,
+	TRACE
+};
+
+static int
+rte_eth_dev_validate_port(uint8_t port_id, int trace)
+{
+	if (port_id >= RTE_MAX_ETHPORTS ||
+	    rte_eth_devices[port_id].attached != DEV_CONNECTED) {
+		if (trace) {
+			PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
+		}
+		return DEV_INVALID;
+	} else
+		return DEV_VALID;
+}
+
 int
 rte_eth_dev_socket_id(uint8_t port_id)
 {
-	if (port_id >= nb_ports)
+	if (rte_eth_dev_validate_port(port_id, NONE_TRACE) == DEV_INVALID)
 		return -1;
 	return rte_eth_devices[port_id].pci_dev->numa_node;
 }
@@ -369,10 +415,8 @@ rte_eth_dev_rx_queue_start(uint8_t port_id, uint16_t rx_queue_id)
 	 * in a multi-process setup*/
 	PROC_PRIMARY_OR_ERR_RET(-E_RTE_SECONDARY);
 
-	if (port_id >= nb_ports) {
-		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
+	if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID)
 		return -EINVAL;
-	}
 
 	dev = &rte_eth_devices[port_id];
 	if (rx_queue_id >= dev->data->nb_rx_queues) {
@@ -395,10 +439,8 @@ rte_eth_dev_rx_queue_stop(uint8_t port_id, uint16_t rx_queue_id)
 	 * in a multi-process setup*/
 	PROC_PRIMARY_OR_ERR_RET(-E_RTE_SECONDARY);
 
-	if (port_id >= nb_ports) {
-		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
+	if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID)
 		return -EINVAL;
-	}
 
 	dev = &rte_eth_devices[port_id];
 	if (rx_queue_id >= dev->data->nb_rx_queues) {
@@ -421,10 +463,8 @@ rte_eth_dev_tx_queue_start(uint8_t port_id, uint16_t tx_queue_id)
 	 * in a multi-process setup*/
 	PROC_PRIMARY_OR_ERR_RET(-E_RTE_SECONDARY);
 
-	if (port_id >= nb_ports) {
-		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
+	if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID)
 		return -EINVAL;
-	}
 
 	dev = &rte_eth_devices[port_id];
 	if (tx_queue_id >= dev->data->nb_tx_queues) {
@@ -447,10 +487,8 @@ rte_eth_dev_tx_queue_stop(uint8_t port_id, uint16_t tx_queue_id)
 	 * in a multi-process setup*/
 	PROC_PRIMARY_OR_ERR_RET(-E_RTE_SECONDARY);
 
-	if (port_id >= nb_ports) {
-		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
+	if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID)
 		return -EINVAL;
-	}
 
 	dev = &rte_eth_devices[port_id];
 	if (tx_queue_id >= dev->data->nb_tx_queues) {
@@ -703,10 +741,9 @@ rte_eth_dev_configure(uint8_t port_id, uint16_t nb_rx_q, uint16_t nb_tx_q,
 	 * in a multi-process setup*/
 	PROC_PRIMARY_OR_ERR_RET(-E_RTE_SECONDARY);
 
-	if (port_id >= nb_ports || port_id >= RTE_MAX_ETHPORTS) {
-		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
+	if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID)
 		return (-EINVAL);
-	}
+
 	dev = &rte_eth_devices[port_id];
 
 	FUNC_PTR_OR_ERR_RET(*dev->dev_ops->dev_infos_get, -ENOTSUP);
@@ -888,10 +925,9 @@ rte_eth_dev_start(uint8_t port_id)
 	 * in a multi-process setup*/
 	PROC_PRIMARY_OR_ERR_RET(-E_RTE_SECONDARY);
 
-	if (port_id >= nb_ports) {
-		PMD_DEBUG_TRACE("Invalid port_id=%" PRIu8 "\n", port_id);
+	if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID)
 		return (-EINVAL);
-	}
+
 	dev = &rte_eth_devices[port_id];
 
 	FUNC_PTR_OR_ERR_RET(*dev->dev_ops->dev_start, -ENOTSUP);
@@ -923,10 +959,9 @@ rte_eth_dev_stop(uint8_t port_id)
 	 * in a multi-process setup*/
 	PROC_PRIMARY_OR_RET();
 
-	if (port_id >= nb_ports) {
-		PMD_DEBUG_TRACE("Invalid port_id=%" PRIu8 "\n", port_id);
+	if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID)
 		return;
-	}
+
 	dev = &rte_eth_devices[port_id];
 
 	FUNC_PTR_OR_RET(*dev->dev_ops->dev_stop);
@@ -951,10 +986,9 @@ rte_eth_dev_set_link_up(uint8_t port_id)
 	 * in a multi-process setup*/
 	PROC_PRIMARY_OR_ERR_RET(-E_RTE_SECONDARY);
 
-	if (port_id >= nb_ports) {
-		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
+	if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID)
 		return -EINVAL;
-	}
+
 	dev = &rte_eth_devices[port_id];
 
 	FUNC_PTR_OR_ERR_RET(*dev->dev_ops->dev_set_link_up, -ENOTSUP);
@@ -970,10 +1004,9 @@ rte_eth_dev_set_link_down(uint8_t port_id)
 	 * in a multi-process setup*/
 	PROC_PRIMARY_OR_ERR_RET(-E_RTE_SECONDARY);
 
-	if (port_id >= nb_ports) {
-		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
+	if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID)
 		return -EINVAL;
-	}
+
 	dev = &rte_eth_devices[port_id];
 
 	FUNC_PTR_OR_ERR_RET(*dev->dev_ops->dev_set_link_down, -ENOTSUP);
@@ -989,10 +1022,8 @@ rte_eth_dev_close(uint8_t port_id)
 	 * in a multi-process setup*/
 	PROC_PRIMARY_OR_RET();
 
-	if (port_id >= nb_ports) {
-		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
+	if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID)
 		return;
-	}
 
 	dev = &rte_eth_devices[port_id];
 
@@ -1017,10 +1048,9 @@ rte_eth_rx_queue_setup(uint8_t port_id, uint16_t rx_queue_id,
 	 * in a multi-process setup*/
 	PROC_PRIMARY_OR_ERR_RET(-E_RTE_SECONDARY);
 
-	if (port_id >= nb_ports) {
-		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
+	if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID)
 		return (-EINVAL);
-	}
+
 	dev = &rte_eth_devices[port_id];
 	if (rx_queue_id >= dev->data->nb_rx_queues) {
 		PMD_DEBUG_TRACE("Invalid RX queue_id=%d\n", rx_queue_id);
@@ -1090,10 +1120,9 @@ rte_eth_tx_queue_setup(uint8_t port_id, uint16_t tx_queue_id,
 	 * in a multi-process setup*/
 	PROC_PRIMARY_OR_ERR_RET(-E_RTE_SECONDARY);
 
-	if (port_id >= RTE_MAX_ETHPORTS || port_id >= nb_ports) {
-		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
+	if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID)
 		return (-EINVAL);
-	}
+
 	dev = &rte_eth_devices[port_id];
 	if (tx_queue_id >= dev->data->nb_tx_queues) {
 		PMD_DEBUG_TRACE("Invalid TX queue_id=%d\n", tx_queue_id);
@@ -1123,10 +1152,9 @@ rte_eth_promiscuous_enable(uint8_t port_id)
 {
 	struct rte_eth_dev *dev;
 
-	if (port_id >= nb_ports) {
-		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
+	if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID)
 		return;
-	}
+
 	dev = &rte_eth_devices[port_id];
 
 	FUNC_PTR_OR_RET(*dev->dev_ops->promiscuous_enable);
@@ -1139,10 +1167,9 @@ rte_eth_promiscuous_disable(uint8_t port_id)
 {
 	struct rte_eth_dev *dev;
 
-	if (port_id >= nb_ports) {
-		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
+	if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID)
 		return;
-	}
+
 	dev = &rte_eth_devices[port_id];
 
 	FUNC_PTR_OR_RET(*dev->dev_ops->promiscuous_disable);
@@ -1155,10 +1182,8 @@ rte_eth_promiscuous_get(uint8_t port_id)
 {
 	struct rte_eth_dev *dev;
 
-	if (port_id >= nb_ports) {
-		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
+	if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID)
 		return -1;
-	}
 
 	dev = &rte_eth_devices[port_id];
 	return dev->data->promiscuous;
@@ -1169,10 +1194,9 @@ rte_eth_allmulticast_enable(uint8_t port_id)
 {
 	struct rte_eth_dev *dev;
 
-	if (port_id >= nb_ports) {
-		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
+	if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID)
 		return;
-	}
+
 	dev = &rte_eth_devices[port_id];
 
 	FUNC_PTR_OR_RET(*dev->dev_ops->allmulticast_enable);
@@ -1185,10 +1209,9 @@ rte_eth_allmulticast_disable(uint8_t port_id)
 {
 	struct rte_eth_dev *dev;
 
-	if (port_id >= nb_ports) {
-		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
+	if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID)
 		return;
-	}
+
 	dev = &rte_eth_devices[port_id];
 
 	FUNC_PTR_OR_RET(*dev->dev_ops->allmulticast_disable);
@@ -1201,10 +1224,8 @@ rte_eth_allmulticast_get(uint8_t port_id)
 {
 	struct rte_eth_dev *dev;
 
-	if (port_id >= nb_ports) {
-		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
+	if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID)
 		return -1;
-	}
 
 	dev = &rte_eth_devices[port_id];
 	return dev->data->all_multicast;
@@ -1229,10 +1250,9 @@ rte_eth_link_get(uint8_t port_id, struct rte_eth_link *eth_link)
 {
 	struct rte_eth_dev *dev;
 
-	if (port_id >= nb_ports) {
-		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
+	if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID)
 		return;
-	}
+
 	dev = &rte_eth_devices[port_id];
 
 	if (dev->data->dev_conf.intr_conf.lsc != 0)
@@ -1249,10 +1269,9 @@ rte_eth_link_get_nowait(uint8_t port_id, struct rte_eth_link *eth_link)
 {
 	struct rte_eth_dev *dev;
 
-	if (port_id >= nb_ports) {
-		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
+	if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID)
 		return;
-	}
+
 	dev = &rte_eth_devices[port_id];
 
 	if (dev->data->dev_conf.intr_conf.lsc != 0)
@@ -1269,10 +1288,9 @@ rte_eth_stats_get(uint8_t port_id, struct rte_eth_stats *stats)
 {
 	struct rte_eth_dev *dev;
 
-	if (port_id >= nb_ports) {
-		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
+	if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID)
 		return;
-	}
+
 	dev = &rte_eth_devices[port_id];
 	memset(stats, 0, sizeof(*stats));
 
@@ -1286,10 +1304,9 @@ rte_eth_stats_reset(uint8_t port_id)
 {
 	struct rte_eth_dev *dev;
 
-	if (port_id >= nb_ports) {
-		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
+	if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID)
 		return;
-	}
+
 	dev = &rte_eth_devices[port_id];
 
 	FUNC_PTR_OR_RET(*dev->dev_ops->stats_reset);
@@ -1307,10 +1324,9 @@ rte_eth_xstats_get(uint8_t port_id, struct rte_eth_xstats *xstats,
 	uint64_t val;
 	char *stats_ptr;
 
-	if (port_id >= nb_ports) {
-		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
+	if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID)
 		return -1;
-	}
+
 	dev = &rte_eth_devices[port_id];
 
 	/* implemented by the driver */
@@ -1376,10 +1392,9 @@ rte_eth_xstats_reset(uint8_t port_id)
 {
 	struct rte_eth_dev *dev;
 
-	if (port_id >= nb_ports) {
-		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
+	if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID)
 		return;
-	}
+
 	dev = &rte_eth_devices[port_id];
 
 	/* implemented by the driver */
@@ -1398,10 +1413,9 @@ set_queue_stats_mapping(uint8_t port_id, uint16_t queue_id, uint8_t stat_idx,
 {
 	struct rte_eth_dev *dev;
 
-	if (port_id >= nb_ports) {
-		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
+	if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID)
 		return -ENODEV;
-	}
+
 	dev = &rte_eth_devices[port_id];
 
 	FUNC_PTR_OR_ERR_RET(*dev->dev_ops->queue_stats_mapping_set, -ENOTSUP);
@@ -1433,10 +1447,9 @@ rte_eth_dev_info_get(uint8_t port_id, struct rte_eth_dev_info *dev_info)
 {
 	struct rte_eth_dev *dev;
 
-	if (port_id >= nb_ports) {
-		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
+	if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID)
 		return;
-	}
+
 	dev = &rte_eth_devices[port_id];
 
 	memset(dev_info, 0, sizeof(struct rte_eth_dev_info));
@@ -1453,10 +1466,9 @@ rte_eth_macaddr_get(uint8_t port_id, struct ether_addr *mac_addr)
 {
 	struct rte_eth_dev *dev;
 
-	if (port_id >= nb_ports) {
-		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
+	if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID)
 		return;
-	}
+
 	dev = &rte_eth_devices[port_id];
 	ether_addr_copy(&dev->data->mac_addrs[0], mac_addr);
 }
@@ -1467,10 +1479,8 @@ rte_eth_dev_get_mtu(uint8_t port_id, uint16_t *mtu)
 {
 	struct rte_eth_dev *dev;
 
-	if (port_id >= nb_ports) {
-		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
+	if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID)
 		return (-ENODEV);
-	}
 
 	dev = &rte_eth_devices[port_id];
 	*mtu = dev->data->mtu;
@@ -1483,10 +1493,8 @@ rte_eth_dev_set_mtu(uint8_t port_id, uint16_t mtu)
 	int ret;
 	struct rte_eth_dev *dev;
 
-	if (port_id >= nb_ports) {
-		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
+	if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID)
 		return (-ENODEV);
-	}
 
 	dev = &rte_eth_devices[port_id];
 	FUNC_PTR_OR_ERR_RET(*dev->dev_ops->mtu_set, -ENOTSUP);
@@ -1503,10 +1511,9 @@ rte_eth_dev_vlan_filter(uint8_t port_id, uint16_t vlan_id, int on)
 {
 	struct rte_eth_dev *dev;
 
-	if (port_id >= nb_ports) {
-		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
+	if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID)
 		return (-ENODEV);
-	}
+
 	dev = &rte_eth_devices[port_id];
 	if (! (dev->data->dev_conf.rxmode.hw_vlan_filter)) {
 		PMD_DEBUG_TRACE("port %d: vlan-filtering disabled\n", port_id);
@@ -1528,10 +1535,8 @@ rte_eth_dev_set_vlan_strip_on_queue(uint8_t port_id, uint16_t rx_queue_id, int o
 {
 	struct rte_eth_dev *dev;
 
-	if (port_id >= nb_ports) {
-		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
+	if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID)
 		return (-ENODEV);
-	}
 
 	dev = &rte_eth_devices[port_id];
 	if (rx_queue_id >= dev->data->nb_rx_queues) {
@@ -1550,10 +1555,8 @@ rte_eth_dev_set_vlan_ether_type(uint8_t port_id, uint16_t tpid)
 {
 	struct rte_eth_dev *dev;
 
-	if (port_id >= nb_ports) {
-		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
+	if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID)
 		return (-ENODEV);
-	}
 
 	dev = &rte_eth_devices[port_id];
 	FUNC_PTR_OR_ERR_RET(*dev->dev_ops->vlan_tpid_set, -ENOTSUP);
@@ -1570,10 +1573,8 @@ rte_eth_dev_set_vlan_offload(uint8_t port_id, int offload_mask)
 	int mask = 0;
 	int cur, org = 0;
 
-	if (port_id >= nb_ports) {
-		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
+	if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID)
 		return (-ENODEV);
-	}
 
 	dev = &rte_eth_devices[port_id];
 
@@ -1615,10 +1616,8 @@ rte_eth_dev_get_vlan_offload(uint8_t port_id)
 	struct rte_eth_dev *dev;
 	int ret = 0;
 
-	if (port_id >= nb_ports) {
-		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
+	if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID)
 		return (-ENODEV);
-	}
 
 	dev = &rte_eth_devices[port_id];
 
@@ -1639,10 +1638,9 @@ rte_eth_dev_set_vlan_pvid(uint8_t port_id, uint16_t pvid, int on)
 {
 	struct rte_eth_dev *dev;
 
-	if (port_id >= nb_ports) {
-		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
+	if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID)
 		return (-ENODEV);
-	}
+
 	dev = &rte_eth_devices[port_id];
 	FUNC_PTR_OR_ERR_RET(*dev->dev_ops->vlan_pvid_set, -ENOTSUP);
 	(*dev->dev_ops->vlan_pvid_set)(dev, pvid, on);
@@ -1657,10 +1655,8 @@ rte_eth_dev_fdir_add_signature_filter(uint8_t port_id,
 {
 	struct rte_eth_dev *dev;
 
-	if (port_id >= nb_ports) {
-		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
+	if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID)
 		return (-ENODEV);
-	}
 
 	dev = &rte_eth_devices[port_id];
 
@@ -1691,10 +1687,8 @@ rte_eth_dev_fdir_update_signature_filter(uint8_t port_id,
 {
 	struct rte_eth_dev *dev;
 
-	if (port_id >= nb_ports) {
-		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
+	if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID)
 		return (-ENODEV);
-	}
 
 	dev = &rte_eth_devices[port_id];
 
@@ -1725,10 +1719,8 @@ rte_eth_dev_fdir_remove_signature_filter(uint8_t port_id,
 {
 	struct rte_eth_dev *dev;
 
-	if (port_id >= nb_ports) {
-		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
+	if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID)
 		return (-ENODEV);
-	}
 
 	dev = &rte_eth_devices[port_id];
 
@@ -1756,10 +1748,8 @@ rte_eth_dev_fdir_get_infos(uint8_t port_id, struct rte_eth_fdir *fdir)
 {
 	struct rte_eth_dev *dev;
 
-	if (port_id >= nb_ports) {
-		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
+	if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID)
 		return (-ENODEV);
-	}
 
 	dev = &rte_eth_devices[port_id];
 	if (! (dev->data->dev_conf.fdir_conf.mode)) {
@@ -1781,10 +1771,8 @@ rte_eth_dev_fdir_add_perfect_filter(uint8_t port_id,
 {
 	struct rte_eth_dev *dev;
 
-	if (port_id >= nb_ports) {
-		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
+	if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID)
 		return (-ENODEV);
-	}
 
 	dev = &rte_eth_devices[port_id];
 
@@ -1821,10 +1809,8 @@ rte_eth_dev_fdir_update_perfect_filter(uint8_t port_id,
 {
 	struct rte_eth_dev *dev;
 
-	if (port_id >= nb_ports) {
-		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
+	if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID)
 		return (-ENODEV);
-	}
 
 	dev = &rte_eth_devices[port_id];
 
@@ -1859,10 +1845,8 @@ rte_eth_dev_fdir_remove_perfect_filter(uint8_t port_id,
 {
 	struct rte_eth_dev *dev;
 
-	if (port_id >= nb_ports) {
-		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
+	if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID)
 		return (-ENODEV);
-	}
 
 	dev = &rte_eth_devices[port_id];
 
@@ -1895,10 +1879,8 @@ rte_eth_dev_fdir_set_masks(uint8_t port_id, struct rte_fdir_masks *fdir_mask)
 {
 	struct rte_eth_dev *dev;
 
-	if (port_id >= nb_ports) {
-		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
+	if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID)
 		return (-ENODEV);
-	}
 
 	dev = &rte_eth_devices[port_id];
 	if (! (dev->data->dev_conf.fdir_conf.mode)) {
@@ -1915,10 +1897,8 @@ rte_eth_dev_flow_ctrl_get(uint8_t port_id, struct rte_eth_fc_conf *fc_conf)
 {
 	struct rte_eth_dev *dev;
 
-	if (port_id >= nb_ports) {
-		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
+	if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID)
 		return (-ENODEV);
-	}
 
 	dev = &rte_eth_devices[port_id];
 	FUNC_PTR_OR_ERR_RET(*dev->dev_ops->flow_ctrl_get, -ENOTSUP);
@@ -1931,10 +1911,8 @@ rte_eth_dev_flow_ctrl_set(uint8_t port_id, struct rte_eth_fc_conf *fc_conf)
 {
 	struct rte_eth_dev *dev;
 
-	if (port_id >= nb_ports) {
-		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
+	if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID)
 		return (-ENODEV);
-	}
 
 	if ((fc_conf->send_xon != 0) && (fc_conf->send_xon != 1)) {
 		PMD_DEBUG_TRACE("Invalid send_xon, only 0/1 allowed\n");
@@ -1951,10 +1929,8 @@ rte_eth_dev_priority_flow_ctrl_set(uint8_t port_id, struct rte_eth_pfc_conf *pfc
 {
 	struct rte_eth_dev *dev;
 
-	if (port_id >= nb_ports) {
-		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
+	if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID)
 		return (-ENODEV);
-	}
 
 	if (pfc_conf->priority > (ETH_DCB_NUM_USER_PRIORITIES - 1)) {
 		PMD_DEBUG_TRACE("Invalid priority, only 0-7 allowed\n");
@@ -2030,10 +2006,8 @@ rte_eth_dev_rss_reta_update(uint8_t port_id,
 	struct rte_eth_dev *dev;
 	int ret;
 
-	if (port_id >= nb_ports) {
-		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
+	if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID)
 		return -ENODEV;
-	}
 
 	/* Check mask bits */
 	ret = rte_eth_check_reta_mask(reta_conf, reta_size);
@@ -2081,10 +2055,9 @@ rte_eth_dev_rss_hash_update(uint8_t port_id, struct rte_eth_rss_conf *rss_conf)
 	struct rte_eth_dev *dev;
 	uint16_t rss_hash_protos;
 
-	if (port_id >= nb_ports) {
-		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
+	if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID)
 		return (-ENODEV);
-	}
+
 	rss_hash_protos = rss_conf->rss_hf;
 	if ((rss_hash_protos != 0) &&
 	    ((rss_hash_protos & ETH_RSS_PROTO_MASK) == 0)) {
@@ -2103,10 +2076,9 @@ rte_eth_dev_rss_hash_conf_get(uint8_t port_id,
 {
 	struct rte_eth_dev *dev;
 
-	if (port_id >= nb_ports) {
-		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
+	if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID)
 		return (-ENODEV);
-	}
+
 	dev = &rte_eth_devices[port_id];
 	FUNC_PTR_OR_ERR_RET(*dev->dev_ops->rss_hash_conf_get, -ENOTSUP);
 	return (*dev->dev_ops->rss_hash_conf_get)(dev, rss_conf);
@@ -2118,10 +2090,8 @@ rte_eth_dev_udp_tunnel_add(uint8_t port_id,
 {
 	struct rte_eth_dev *dev;
 
-	if (port_id >= nb_ports) {
-		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
+	if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID)
 		return -ENODEV;
-	}
 
 	if (udp_tunnel == NULL) {
 		PMD_DEBUG_TRACE("Invalid udp_tunnel parameter\n");
@@ -2144,10 +2114,9 @@ rte_eth_dev_udp_tunnel_delete(uint8_t port_id,
 {
 	struct rte_eth_dev *dev;
 
-	if (port_id >= nb_ports) {
-		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
+	if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID)
 		return -ENODEV;
-	}
+
 	dev = &rte_eth_devices[port_id];
 
 	if (udp_tunnel == NULL) {
@@ -2169,10 +2138,8 @@ rte_eth_led_on(uint8_t port_id)
 {
 	struct rte_eth_dev *dev;
 
-	if (port_id >= nb_ports) {
-		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
+	if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID)
 		return (-ENODEV);
-	}
 
 	dev = &rte_eth_devices[port_id];
 	FUNC_PTR_OR_ERR_RET(*dev->dev_ops->dev_led_on, -ENOTSUP);
@@ -2184,10 +2151,8 @@ rte_eth_led_off(uint8_t port_id)
 {
 	struct rte_eth_dev *dev;
 
-	if (port_id >= nb_ports) {
-		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
+	if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID)
 		return (-ENODEV);
-	}
 
 	dev = &rte_eth_devices[port_id];
 	FUNC_PTR_OR_ERR_RET(*dev->dev_ops->dev_led_off, -ENOTSUP);
@@ -2224,10 +2189,9 @@ rte_eth_dev_mac_addr_add(uint8_t port_id, struct ether_addr *addr,
 	int index;
 	uint64_t pool_mask;
 
-	if (port_id >= nb_ports) {
-		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
+	if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID)
 		return (-ENODEV);
-	}
+
 	dev = &rte_eth_devices[port_id];
 	FUNC_PTR_OR_ERR_RET(*dev->dev_ops->mac_addr_add, -ENOTSUP);
 
@@ -2275,10 +2239,9 @@ rte_eth_dev_mac_addr_remove(uint8_t port_id, struct ether_addr *addr)
 	struct rte_eth_dev *dev;
 	int index;
 
-	if (port_id >= nb_ports) {
-		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
+	if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID)
 		return (-ENODEV);
-	}
+
 	dev = &rte_eth_devices[port_id];
 	FUNC_PTR_OR_ERR_RET(*dev->dev_ops->mac_addr_remove, -ENOTSUP);
 
@@ -2309,7 +2272,7 @@ rte_eth_dev_set_vf_rxmode(uint8_t port_id,  uint16_t vf,
 	struct rte_eth_dev *dev;
 	struct rte_eth_dev_info dev_info;
 
-	if (port_id >= nb_ports) {
+	if (rte_eth_dev_validate_port(port_id, NONE_TRACE) == DEV_INVALID) {
 		PMD_DEBUG_TRACE("set VF RX mode:Invalid port_id=%d\n",
 				port_id);
 		return (-ENODEV);
@@ -2364,7 +2327,7 @@ rte_eth_dev_uc_hash_table_set(uint8_t port_id, struct ether_addr *addr,
 	int ret;
 	struct rte_eth_dev *dev;
 
-	if (port_id >= nb_ports) {
+	if (rte_eth_dev_validate_port(port_id, NONE_TRACE) == DEV_INVALID) {
 		PMD_DEBUG_TRACE("unicast hash setting:Invalid port_id=%d\n",
 			port_id);
 		return (-ENODEV);
@@ -2417,7 +2380,7 @@ rte_eth_dev_uc_all_hash_table_set(uint8_t port_id, uint8_t on)
 {
 	struct rte_eth_dev *dev;
 
-	if (port_id >= nb_ports) {
+	if (rte_eth_dev_validate_port(port_id, NONE_TRACE) == DEV_INVALID) {
 		PMD_DEBUG_TRACE("unicast hash setting:Invalid port_id=%d\n",
 			port_id);
 		return (-ENODEV);
@@ -2436,10 +2399,8 @@ rte_eth_dev_set_vf_rx(uint8_t port_id,uint16_t vf, uint8_t on)
 	struct rte_eth_dev *dev;
 	struct rte_eth_dev_info dev_info;
 
-	if (port_id >= nb_ports) {
-		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
+	if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID)
 		return (-ENODEV);
-	}
 
 	dev = &rte_eth_devices[port_id];
 	rte_eth_dev_info_get(port_id, &dev_info);
@@ -2462,7 +2423,7 @@ rte_eth_dev_set_vf_tx(uint8_t port_id,uint16_t vf, uint8_t on)
 	struct rte_eth_dev *dev;
 	struct rte_eth_dev_info dev_info;
 
-	if (port_id >= nb_ports) {
+	if (rte_eth_dev_validate_port(port_id, NONE_TRACE) == DEV_INVALID) {
 		PMD_DEBUG_TRACE("set pool tx:Invalid port_id=%d\n", port_id);
 		return (-ENODEV);
 	}
@@ -2487,7 +2448,7 @@ rte_eth_dev_set_vf_vlan_filter(uint8_t port_id, uint16_t vlan_id,
 {
 	struct rte_eth_dev *dev;
 
-	if (port_id >= nb_ports) {
+	if (rte_eth_dev_validate_port(port_id, NONE_TRACE) == DEV_INVALID) {
 		PMD_DEBUG_TRACE("VF VLAN filter:invalid port id=%d\n",
 				port_id);
 		return (-ENODEV);
@@ -2518,7 +2479,7 @@ int rte_eth_set_queue_rate_limit(uint8_t port_id, uint16_t queue_idx,
 	struct rte_eth_dev_info dev_info;
 	struct rte_eth_link link;
 
-	if (port_id >= nb_ports) {
+	if (rte_eth_dev_validate_port(port_id, NONE_TRACE) == DEV_INVALID) {
 		PMD_DEBUG_TRACE("set queue rate limit:invalid port id=%d\n",
 				port_id);
 		return -ENODEV;
@@ -2555,7 +2516,7 @@ int rte_eth_set_vf_rate_limit(uint8_t port_id, uint16_t vf, uint16_t tx_rate,
 	if (q_msk == 0)
 		return 0;
 
-	if (port_id >= nb_ports) {
+	if (rte_eth_dev_validate_port(port_id, NONE_TRACE) == DEV_INVALID) {
 		PMD_DEBUG_TRACE("set VF rate limit:invalid port id=%d\n",
 				port_id);
 		return -ENODEV;
@@ -2589,10 +2550,8 @@ rte_eth_mirror_rule_set(uint8_t port_id,
 {
 	struct rte_eth_dev *dev = &rte_eth_devices[port_id];
 
-	if (port_id >= nb_ports) {
-		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
+	if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID)
 		return (-ENODEV);
-	}
 
 	if (mirror_conf->rule_type_mask == 0) {
 		PMD_DEBUG_TRACE("mirror rule type can not be 0.\n");
@@ -2630,10 +2589,8 @@ rte_eth_mirror_rule_reset(uint8_t port_id, uint8_t rule_id)
 {
 	struct rte_eth_dev *dev = &rte_eth_devices[port_id];
 
-	if (port_id >= nb_ports) {
-		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
+	if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID)
 		return (-ENODEV);
-	}
 
 	if(rule_id >= ETH_VMDQ_NUM_MIRROR_RULE)
 	{
@@ -2655,10 +2612,9 @@ rte_eth_rx_burst(uint8_t port_id, uint16_t queue_id,
 {
 	struct rte_eth_dev *dev;
 
-	if (port_id >= nb_ports) {
-		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
+	if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID)
 		return 0;
-	}
+
 	dev = &rte_eth_devices[port_id];
 	FUNC_PTR_OR_ERR_RET(*dev->rx_pkt_burst, 0);
 	if (queue_id >= dev->data->nb_rx_queues) {
@@ -2675,10 +2631,9 @@ rte_eth_tx_burst(uint8_t port_id, uint16_t queue_id,
 {
 	struct rte_eth_dev *dev;
 
-	if (port_id >= nb_ports) {
-		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
+	if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID)
 		return 0;
-	}
+
 	dev = &rte_eth_devices[port_id];
 
 	FUNC_PTR_OR_ERR_RET(*dev->tx_pkt_burst, 0);
@@ -2695,10 +2650,9 @@ rte_eth_rx_queue_count(uint8_t port_id, uint16_t queue_id)
 {
 	struct rte_eth_dev *dev;
 
-	if (port_id >= nb_ports) {
-		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
+	if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID)
 		return 0;
-	}
+
 	dev = &rte_eth_devices[port_id];
 	FUNC_PTR_OR_ERR_RET(*dev->dev_ops->rx_queue_count, 0);
 	return (*dev->dev_ops->rx_queue_count)(dev, queue_id);
@@ -2709,10 +2663,9 @@ rte_eth_rx_descriptor_done(uint8_t port_id, uint16_t queue_id, uint16_t offset)
 {
 	struct rte_eth_dev *dev;
 
-	if (port_id >= nb_ports) {
-		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
+	if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID)
 		return (-ENODEV);
-	}
+
 	dev = &rte_eth_devices[port_id];
 	FUNC_PTR_OR_ERR_RET(*dev->dev_ops->rx_descriptor_done, -ENOTSUP);
 	return (*dev->dev_ops->rx_descriptor_done)( \
@@ -2730,10 +2683,8 @@ rte_eth_dev_callback_register(uint8_t port_id,
 
 	if (!cb_fn)
 		return (-EINVAL);
-	if (port_id >= nb_ports) {
-		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
+	if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID)
 		return (-EINVAL);
-	}
 
 	dev = &rte_eth_devices[port_id];
 	rte_spinlock_lock(&rte_eth_dev_cb_lock);
@@ -2770,10 +2721,8 @@ rte_eth_dev_callback_unregister(uint8_t port_id,
 
 	if (!cb_fn)
 		return (-EINVAL);
-	if (port_id >= nb_ports) {
-		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
+	if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID)
 		return (-EINVAL);
-	}
 
 	dev = &rte_eth_devices[port_id];
 	rte_spinlock_lock(&rte_eth_dev_cb_lock);
@@ -2830,10 +2779,8 @@ int rte_eth_dev_bypass_init(uint8_t port_id)
 {
 	struct rte_eth_dev *dev;
 
-	if (port_id >= nb_ports) {
-		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
+	if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID)
 		return (-ENODEV);
-	}
 
 	if ((dev= &rte_eth_devices[port_id]) == NULL) {
 		PMD_DEBUG_TRACE("Invalid port device\n");
@@ -2850,10 +2797,8 @@ rte_eth_dev_bypass_state_show(uint8_t port_id, uint32_t *state)
 {
 	struct rte_eth_dev *dev;
 
-	if (port_id >= nb_ports) {
-		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
+	if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID)
 		return (-ENODEV);
-	}
 
 	if ((dev= &rte_eth_devices[port_id]) == NULL) {
 		PMD_DEBUG_TRACE("Invalid port device\n");
@@ -2869,10 +2814,8 @@ rte_eth_dev_bypass_state_set(uint8_t port_id, uint32_t *new_state)
 {
 	struct rte_eth_dev *dev;
 
-	if (port_id >= nb_ports) {
-		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
+	if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID)
 		return (-ENODEV);
-	}
 
 	if ((dev= &rte_eth_devices[port_id]) == NULL) {
 		PMD_DEBUG_TRACE("Invalid port device\n");
@@ -2889,10 +2832,8 @@ rte_eth_dev_bypass_event_show(uint8_t port_id, uint32_t event, uint32_t *state)
 {
 	struct rte_eth_dev *dev;
 
-	if (port_id >= nb_ports) {
-		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
+	if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID)
 		return (-ENODEV);
-	}
 
 	if ((dev= &rte_eth_devices[port_id]) == NULL) {
 		PMD_DEBUG_TRACE("Invalid port device\n");
@@ -2909,10 +2850,8 @@ rte_eth_dev_bypass_event_store(uint8_t port_id, uint32_t event, uint32_t state)
 {
 	struct rte_eth_dev *dev;
 
-	if (port_id >= nb_ports) {
-		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
+	if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID)
 		return (-ENODEV);
-	}
 
 	if ((dev= &rte_eth_devices[port_id]) == NULL) {
 		PMD_DEBUG_TRACE("Invalid port device\n");
@@ -2929,10 +2868,8 @@ rte_eth_dev_wd_timeout_store(uint8_t port_id, uint32_t timeout)
 {
 	struct rte_eth_dev *dev;
 
-	if (port_id >= nb_ports) {
-		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
+	if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID)
 		return (-ENODEV);
-	}
 
 	if ((dev= &rte_eth_devices[port_id]) == NULL) {
 		PMD_DEBUG_TRACE("Invalid port device\n");
@@ -2949,10 +2886,8 @@ rte_eth_dev_bypass_ver_show(uint8_t port_id, uint32_t *ver)
 {
 	struct rte_eth_dev *dev;
 
-	if (port_id >= nb_ports) {
-		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
+	if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID)
 		return (-ENODEV);
-	}
 
 	if ((dev= &rte_eth_devices[port_id]) == NULL) {
 		PMD_DEBUG_TRACE("Invalid port device\n");
@@ -2969,10 +2904,8 @@ rte_eth_dev_bypass_wd_timeout_show(uint8_t port_id, uint32_t *wd_timeout)
 {
 	struct rte_eth_dev *dev;
 
-	if (port_id >= nb_ports) {
-		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
+	if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID)
 		return (-ENODEV);
-	}
 
 	if ((dev= &rte_eth_devices[port_id]) == NULL) {
 		PMD_DEBUG_TRACE("Invalid port device\n");
@@ -2989,10 +2922,8 @@ rte_eth_dev_bypass_wd_reset(uint8_t port_id)
 {
 	struct rte_eth_dev *dev;
 
-	if (port_id >= nb_ports) {
-		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
+	if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID)
 		return (-ENODEV);
-	}
 
 	if ((dev= &rte_eth_devices[port_id]) == NULL) {
 		PMD_DEBUG_TRACE("Invalid port device\n");
@@ -3011,10 +2942,8 @@ rte_eth_dev_add_syn_filter(uint8_t port_id,
 {
 	struct rte_eth_dev *dev;
 
-	if (port_id >= nb_ports) {
-		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
+	if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID)
 		return -ENODEV;
-	}
 
 	dev = &rte_eth_devices[port_id];
 	FUNC_PTR_OR_ERR_RET(*dev->dev_ops->add_syn_filter, -ENOTSUP);
@@ -3026,10 +2955,8 @@ rte_eth_dev_remove_syn_filter(uint8_t port_id)
 {
 	struct rte_eth_dev *dev;
 
-	if (port_id >= nb_ports) {
-		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
+	if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID)
 		return -ENODEV;
-	}
 
 	dev = &rte_eth_devices[port_id];
 	FUNC_PTR_OR_ERR_RET(*dev->dev_ops->remove_syn_filter, -ENOTSUP);
@@ -3045,10 +2972,8 @@ rte_eth_dev_get_syn_filter(uint8_t port_id,
 	if (filter == NULL || rx_queue == NULL)
 		return -EINVAL;
 
-	if (port_id >= nb_ports) {
-		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
+	if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID)
 		return -ENODEV;
-	}
 
 	dev = &rte_eth_devices[port_id];
 	FUNC_PTR_OR_ERR_RET(*dev->dev_ops->get_syn_filter, -ENOTSUP);
@@ -3061,10 +2986,9 @@ rte_eth_dev_add_2tuple_filter(uint8_t port_id, uint16_t index,
 {
 	struct rte_eth_dev *dev;
 
-	if (port_id >= nb_ports) {
-		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
+	if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID)
 		return -ENODEV;
-	}
+
 	if (filter->protocol != IPPROTO_TCP &&
 		filter->tcp_flags != 0){
 		PMD_DEBUG_TRACE("tcp flags is 0x%x, but the protocol value"
@@ -3083,10 +3007,8 @@ rte_eth_dev_remove_2tuple_filter(uint8_t port_id, uint16_t index)
 {
 	struct rte_eth_dev *dev;
 
-	if (port_id >= nb_ports) {
-		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
+	if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID)
 		return -ENODEV;
-	}
 
 	dev = &rte_eth_devices[port_id];
 	FUNC_PTR_OR_ERR_RET(*dev->dev_ops->remove_2tuple_filter, -ENOTSUP);
@@ -3102,10 +3024,8 @@ rte_eth_dev_get_2tuple_filter(uint8_t port_id, uint16_t index,
 	if (filter == NULL || rx_queue == NULL)
 		return -EINVAL;
 
-	if (port_id >= nb_ports) {
-		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
+	if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID)
 		return -ENODEV;
-	}
 
 	dev = &rte_eth_devices[port_id];
 	FUNC_PTR_OR_ERR_RET(*dev->dev_ops->get_2tuple_filter, -ENOTSUP);
@@ -3118,10 +3038,8 @@ rte_eth_dev_add_5tuple_filter(uint8_t port_id, uint16_t index,
 {
 	struct rte_eth_dev *dev;
 
-	if (port_id >= nb_ports) {
-		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
+	if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID)
 		return -ENODEV;
-	}
 
 	if (filter->protocol != IPPROTO_TCP &&
 		filter->tcp_flags != 0){
@@ -3141,10 +3059,8 @@ rte_eth_dev_remove_5tuple_filter(uint8_t port_id, uint16_t index)
 {
 	struct rte_eth_dev *dev;
 
-	if (port_id >= nb_ports) {
-		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
+	if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID)
 		return -ENODEV;
-	}
 
 	dev = &rte_eth_devices[port_id];
 	FUNC_PTR_OR_ERR_RET(*dev->dev_ops->remove_5tuple_filter, -ENOTSUP);
@@ -3160,10 +3076,8 @@ rte_eth_dev_get_5tuple_filter(uint8_t port_id, uint16_t index,
 	if (filter == NULL || rx_queue == NULL)
 		return -EINVAL;
 
-	if (port_id >= nb_ports) {
-		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
+	if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID)
 		return -ENODEV;
-	}
 
 	dev = &rte_eth_devices[port_id];
 	FUNC_PTR_OR_ERR_RET(*dev->dev_ops->get_5tuple_filter, -ENOTSUP);
@@ -3177,10 +3091,8 @@ rte_eth_dev_add_flex_filter(uint8_t port_id, uint16_t index,
 {
 	struct rte_eth_dev *dev;
 
-	if (port_id >= nb_ports) {
-		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
+	if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID)
 		return -ENODEV;
-	}
 
 	dev = &rte_eth_devices[port_id];
 	FUNC_PTR_OR_ERR_RET(*dev->dev_ops->add_flex_filter, -ENOTSUP);
@@ -3192,10 +3104,8 @@ rte_eth_dev_remove_flex_filter(uint8_t port_id, uint16_t index)
 {
 	struct rte_eth_dev *dev;
 
-	if (port_id >= nb_ports) {
-		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
+	if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID)
 		return -ENODEV;
-	}
 
 	dev = &rte_eth_devices[port_id];
 	FUNC_PTR_OR_ERR_RET(*dev->dev_ops->remove_flex_filter, -ENOTSUP);
@@ -3211,10 +3121,8 @@ rte_eth_dev_get_flex_filter(uint8_t port_id, uint16_t index,
 	if (filter == NULL || rx_queue == NULL)
 		return -EINVAL;
 
-	if (port_id >= nb_ports) {
-		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
+	if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID)
 		return -ENODEV;
-	}
 
 	dev = &rte_eth_devices[port_id];
 	FUNC_PTR_OR_ERR_RET(*dev->dev_ops->get_flex_filter, -ENOTSUP);
@@ -3227,10 +3135,8 @@ rte_eth_dev_filter_supported(uint8_t port_id, enum rte_filter_type filter_type)
 {
 	struct rte_eth_dev *dev;
 
-	if (port_id >= nb_ports) {
-		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
+	if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID)
 		return -ENODEV;
-	}
 
 	dev = &rte_eth_devices[port_id];
 	FUNC_PTR_OR_ERR_RET(*dev->dev_ops->filter_ctrl, -ENOTSUP);
@@ -3244,10 +3150,8 @@ rte_eth_dev_filter_ctrl(uint8_t port_id, enum rte_filter_type filter_type,
 {
 	struct rte_eth_dev *dev;
 
-	if (port_id >= nb_ports) {
-		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
+	if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID)
 		return -ENODEV;
-	}
 
 	dev = &rte_eth_devices[port_id];
 	FUNC_PTR_OR_ERR_RET(*dev->dev_ops->filter_ctrl, -ENOTSUP);
diff --git a/lib/librte_ether/rte_ethdev.h b/lib/librte_ether/rte_ethdev.h
index 1200c1c..ca101f5 100644
--- a/lib/librte_ether/rte_ethdev.h
+++ b/lib/librte_ether/rte_ethdev.h
@@ -1539,6 +1539,7 @@ struct rte_eth_dev {
 	struct eth_dev_ops *dev_ops;    /**< Functions exported by PMD */
 	struct rte_pci_device *pci_dev; /**< PCI info. supplied by probing */
 	struct rte_eth_dev_cb_list callbacks; /**< User application callbacks */
+	uint8_t attached; /**< Flag indicating the port is attached */
 };
 
 struct rte_eth_dev_sriov {
@@ -1604,6 +1605,10 @@ extern struct rte_eth_dev rte_eth_devices[];
  * initialized by the [matching] Ethernet driver during the PCI probing phase.
  * All devices whose port identifier is in the range
  * [0,  rte_eth_dev_count() - 1] can be operated on by network applications.
+ * immediately after invoking rte_eal_init().
+ * If the application unplugs a port using hotplug function, The enabled port
+ * numbers may be noncontiguous. In the case, the applications need to manage
+ * enabled port by themselves.
  *
  * @return
  *   - The total number of usable Ethernet devices.
-- 
1.9.1

^ permalink raw reply	[flat|nested] 362+ messages in thread

* [dpdk-dev] [PATCH v6 04/13] eal/pci: Consolidate pci address comparison APIs
  2015-02-01  4:01     ` [dpdk-dev] [PATCH v6 00/13] Port Hotplug Framework Tetsuya Mukawa
                         ` (2 preceding siblings ...)
  2015-02-01  4:01       ` [dpdk-dev] [PATCH v6 03/13] eal/pci, ethdev: Remove assumption that port will not be detached Tetsuya Mukawa
@ 2015-02-01  4:01       ` Tetsuya Mukawa
  2015-02-01  4:01       ` [dpdk-dev] [PATCH v6 05/13] ethdev: Add rte_eth_dev_free to free specified device Tetsuya Mukawa
                         ` (12 subsequent siblings)
  16 siblings, 0 replies; 362+ messages in thread
From: Tetsuya Mukawa @ 2015-02-01  4:01 UTC (permalink / raw)
  To: dev

This patch replaces pci_addr_comparison() and memcmp() of pci addresses by
eal_compare_pci_addr().

v5:
- Fix pci_scan_one to handle pt_driver correctly.
v4:
- Fix calculation method of eal_compare_pci_addr().
- Add parameter checking.

Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp>
---
 lib/librte_eal/bsdapp/eal/eal_pci.c       | 25 ++++++++---------------
 lib/librte_eal/common/eal_common_pci.c    |  2 +-
 lib/librte_eal/common/include/rte_pci.h   | 34 +++++++++++++++++++++++++++++++
 lib/librte_eal/linuxapp/eal/eal_pci.c     | 25 ++++++++---------------
 lib/librte_eal/linuxapp/eal/eal_pci_uio.c |  2 +-
 5 files changed, 54 insertions(+), 34 deletions(-)

diff --git a/lib/librte_eal/bsdapp/eal/eal_pci.c b/lib/librte_eal/bsdapp/eal/eal_pci.c
index 74ecce7..c844d58 100644
--- a/lib/librte_eal/bsdapp/eal/eal_pci.c
+++ b/lib/librte_eal/bsdapp/eal/eal_pci.c
@@ -270,20 +270,6 @@ pci_uio_map_resource(struct rte_pci_device *dev)
 	return (0);
 }
 
-/* Compare two PCI device addresses. */
-static int
-pci_addr_comparison(struct rte_pci_addr *addr, struct rte_pci_addr *addr2)
-{
-	uint64_t dev_addr = (addr->domain << 24) + (addr->bus << 16) + (addr->devid << 8) + addr->function;
-	uint64_t dev_addr2 = (addr2->domain << 24) + (addr2->bus << 16) + (addr2->devid << 8) + addr2->function;
-
-	if (dev_addr > dev_addr2)
-		return 1;
-	else
-		return 0;
-}
-
-
 /* Scan one pci sysfs entry, and fill the devices list from it. */
 static int
 pci_scan_one(int dev_pci_fd, struct pci_conf *conf)
@@ -356,13 +342,20 @@ pci_scan_one(int dev_pci_fd, struct pci_conf *conf)
 	}
 	else {
 		struct rte_pci_device *dev2 = NULL;
+		int ret;
 
 		TAILQ_FOREACH(dev2, &pci_device_list, next) {
-			if (pci_addr_comparison(&dev->addr, &dev2->addr))
+			ret = eal_compare_pci_addr(&dev->addr, &dev2->addr);
+			if (ret > 0)
 				continue;
-			else {
+			else if (ret < 0) {
 				TAILQ_INSERT_BEFORE(dev2, dev, next);
 				return 0;
+			} else { /* already registered */
+				/* update pt_driver */
+				dev2->pt_driver = dev->pt_driver;
+				free(dev);
+				return 0;
 			}
 		}
 		TAILQ_INSERT_TAIL(&pci_device_list, dev, next);
diff --git a/lib/librte_eal/common/eal_common_pci.c b/lib/librte_eal/common/eal_common_pci.c
index f3c7f71..a89f5c3 100644
--- a/lib/librte_eal/common/eal_common_pci.c
+++ b/lib/librte_eal/common/eal_common_pci.c
@@ -93,7 +93,7 @@ static struct rte_devargs *pci_devargs_lookup(struct rte_pci_device *dev)
 		if (devargs->type != RTE_DEVTYPE_BLACKLISTED_PCI &&
 			devargs->type != RTE_DEVTYPE_WHITELISTED_PCI)
 			continue;
-		if (!memcmp(&dev->addr, &devargs->pci.addr, sizeof(dev->addr)))
+		if (!eal_compare_pci_addr(&dev->addr, &devargs->pci.addr))
 			return devargs;
 	}
 	return NULL;
diff --git a/lib/librte_eal/common/include/rte_pci.h b/lib/librte_eal/common/include/rte_pci.h
index 7f2d699..4814cd7 100644
--- a/lib/librte_eal/common/include/rte_pci.h
+++ b/lib/librte_eal/common/include/rte_pci.h
@@ -269,6 +269,40 @@ eal_parse_pci_DomBDF(const char *input, struct rte_pci_addr *dev_addr)
 }
 #undef GET_PCIADDR_FIELD
 
+/* Compare two PCI device addresses. */
+/**
+ * Utility function to compare two PCI device addresses.
+ *
+ * @param addr
+ *	The PCI Bus-Device-Function address to compare
+ * @param addr2
+ *	The PCI Bus-Device-Function address to compare
+ * @return
+ *	0 on equal PCI address.
+ *	Positive on addr is greater than addr2.
+ *	Negative on addr is less than addr2, or error.
+ */
+static inline int
+eal_compare_pci_addr(struct rte_pci_addr *addr, struct rte_pci_addr *addr2)
+{
+	uint64_t dev_addr, dev_addr2;
+
+	if ((addr == NULL) || (addr2 == NULL))
+		return -1;
+
+	dev_addr = (addr->domain << 24) | (addr->bus << 16) |
+				(addr->devid << 8) | addr->function;
+	dev_addr2 = (addr2->domain << 24) | (addr2->bus << 16) |
+				(addr2->devid << 8) | addr2->function;
+
+	if (dev_addr > dev_addr2)
+		return 1;
+	else if (dev_addr < dev_addr2)
+		return -1;
+	else
+		return 0;
+}
+
 /**
  * Probe the PCI bus for registered drivers.
  *
diff --git a/lib/librte_eal/linuxapp/eal/eal_pci.c b/lib/librte_eal/linuxapp/eal/eal_pci.c
index c0ca5a5..d847102 100644
--- a/lib/librte_eal/linuxapp/eal/eal_pci.c
+++ b/lib/librte_eal/linuxapp/eal/eal_pci.c
@@ -229,20 +229,6 @@ error:
 	return -1;
 }
 
-/* Compare two PCI device addresses. */
-static int
-pci_addr_comparison(struct rte_pci_addr *addr, struct rte_pci_addr *addr2)
-{
-	uint64_t dev_addr = (addr->domain << 24) + (addr->bus << 16) + (addr->devid << 8) + addr->function;
-	uint64_t dev_addr2 = (addr2->domain << 24) + (addr2->bus << 16) + (addr2->devid << 8) + addr2->function;
-
-	if (dev_addr > dev_addr2)
-		return 1;
-	else
-		return 0;
-}
-
-
 /* Scan one pci sysfs entry, and fill the devices list from it. */
 static int
 pci_scan_one(const char *dirname, uint16_t domain, uint8_t bus,
@@ -353,13 +339,20 @@ pci_scan_one(const char *dirname, uint16_t domain, uint8_t bus,
 	}
 	else {
 		struct rte_pci_device *dev2 = NULL;
+		int ret;
 
 		TAILQ_FOREACH(dev2, &pci_device_list, next) {
-			if (pci_addr_comparison(&dev->addr, &dev2->addr))
+			ret = eal_compare_pci_addr(&dev->addr, &dev2->addr);
+			if (ret > 0)
 				continue;
-			else {
+			else if (ret < 0) {
 				TAILQ_INSERT_BEFORE(dev2, dev, next);
 				return 0;
+			} else { /* already registered */
+				/* update pt_driver */
+				dev2->pt_driver = dev->pt_driver;
+				free(dev);
+				return 0;
 			}
 		}
 		TAILQ_INSERT_TAIL(&pci_device_list, dev, next);
diff --git a/lib/librte_eal/linuxapp/eal/eal_pci_uio.c b/lib/librte_eal/linuxapp/eal/eal_pci_uio.c
index e53f06b..1da3507 100644
--- a/lib/librte_eal/linuxapp/eal/eal_pci_uio.c
+++ b/lib/librte_eal/linuxapp/eal/eal_pci_uio.c
@@ -123,7 +123,7 @@ pci_uio_map_secondary(struct rte_pci_device *dev)
 	TAILQ_FOREACH(uio_res, pci_res_list, next) {
 
 		/* skip this element if it doesn't match our PCI address */
-		if (memcmp(&uio_res->pci_addr, &dev->addr, sizeof(dev->addr)))
+		if (eal_compare_pci_addr(&uio_res->pci_addr, &dev->addr))
 			continue;
 
 		for (i = 0; i != uio_res->nb_maps; i++) {
-- 
1.9.1

^ permalink raw reply	[flat|nested] 362+ messages in thread

* [dpdk-dev] [PATCH v6 05/13] ethdev: Add rte_eth_dev_free to free specified device
  2015-02-01  4:01     ` [dpdk-dev] [PATCH v6 00/13] Port Hotplug Framework Tetsuya Mukawa
                         ` (3 preceding siblings ...)
  2015-02-01  4:01       ` [dpdk-dev] [PATCH v6 04/13] eal/pci: Consolidate pci address comparison APIs Tetsuya Mukawa
@ 2015-02-01  4:01       ` Tetsuya Mukawa
  2015-02-01  4:01       ` [dpdk-dev] [PATCH v6 06/13] eal, ethdev: Add a function and function pointers to close ether device Tetsuya Mukawa
                         ` (11 subsequent siblings)
  16 siblings, 0 replies; 362+ messages in thread
From: Tetsuya Mukawa @ 2015-02-01  4:01 UTC (permalink / raw)
  To: dev

This patch adds rte_eth_dev_free(). The function is used for changing a
attached status of the device that has specified name.

v6:
- Use rte_eth_dev structure as the paramter of rte_eth_dev_free().
v4:
- Add paramerter checking.

Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp>
---
 lib/librte_ether/rte_ethdev.c | 11 +++++++++++
 lib/librte_ether/rte_ethdev.h | 14 ++++++++++++++
 2 files changed, 25 insertions(+)

diff --git a/lib/librte_ether/rte_ethdev.c b/lib/librte_ether/rte_ethdev.c
index d70854f..b58bab3 100644
--- a/lib/librte_ether/rte_ethdev.c
+++ b/lib/librte_ether/rte_ethdev.c
@@ -260,6 +260,17 @@ rte_eth_dev_allocate(const char *name)
 	return eth_dev;
 }
 
+int
+rte_eth_dev_free(struct rte_eth_dev *eth_dev)
+{
+	if (eth_dev == NULL)
+		return -EINVAL;
+
+	eth_dev->attached = 0;
+	nb_ports--;
+	return 0;
+}
+
 static int
 rte_eth_dev_init(struct rte_pci_driver *pci_drv,
 		 struct rte_pci_device *pci_dev)
diff --git a/lib/librte_ether/rte_ethdev.h b/lib/librte_ether/rte_ethdev.h
index ca101f5..fbe7ac1 100644
--- a/lib/librte_ether/rte_ethdev.h
+++ b/lib/librte_ether/rte_ethdev.h
@@ -1627,6 +1627,20 @@ extern uint8_t rte_eth_dev_count(void);
  */
 struct rte_eth_dev *rte_eth_dev_allocate(const char *name);
 
+/**
+ * Function for internal use by dummy drivers primarily, e.g. ring-based
+ * driver.
+ * Free the specified ethdev.
+ *
+ * @param eth_dev
+ * The *eth_dev* pointer is the address of the *rte_eth_dev* structure
+ *   associated with the matching device and which have been [automatically]
+ *   allocated in the *rte_eth_devices* array.
+ * @return
+ *   - 0 on success, negative on error
+ */
+int rte_eth_dev_free(struct rte_eth_dev *eth_dev);
+
 struct eth_driver;
 /**
  * @internal
-- 
1.9.1

^ permalink raw reply	[flat|nested] 362+ messages in thread

* [dpdk-dev] [PATCH v6 06/13] eal, ethdev: Add a function and function pointers to close ether device
  2015-02-01  4:01     ` [dpdk-dev] [PATCH v6 00/13] Port Hotplug Framework Tetsuya Mukawa
                         ` (4 preceding siblings ...)
  2015-02-01  4:01       ` [dpdk-dev] [PATCH v6 05/13] ethdev: Add rte_eth_dev_free to free specified device Tetsuya Mukawa
@ 2015-02-01  4:01       ` Tetsuya Mukawa
  2015-02-01  4:01       ` [dpdk-dev] [PATCH v6 07/13] ethdev: Add functions that will be used by port hotplug functions Tetsuya Mukawa
                         ` (10 subsequent siblings)
  16 siblings, 0 replies; 362+ messages in thread
From: Tetsuya Mukawa @ 2015-02-01  4:01 UTC (permalink / raw)
  To: dev

The patch adds function pointer to rte_pci_driver and eth_driver
structure. These function pointers are used when ports are detached.
Also the patch adds rte_eth_dev_uninit(). So far, it's not called
by anywhere, but it will be called when port hotplug function is
implemented.

v6:
- Fix rte_eth_dev_uninit() to handle a return value of uninit
  function of PMD.
v4:
- Add paramerter checking.
- Change function names.

Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp>
---
 lib/librte_eal/common/include/rte_pci.h |  7 +++++
 lib/librte_ether/rte_ethdev.c           | 47 +++++++++++++++++++++++++++++++++
 lib/librte_ether/rte_ethdev.h           | 24 +++++++++++++++++
 3 files changed, 78 insertions(+)

diff --git a/lib/librte_eal/common/include/rte_pci.h b/lib/librte_eal/common/include/rte_pci.h
index 4814cd7..87ca4cf 100644
--- a/lib/librte_eal/common/include/rte_pci.h
+++ b/lib/librte_eal/common/include/rte_pci.h
@@ -189,12 +189,19 @@ struct rte_pci_driver;
 typedef int (pci_devinit_t)(struct rte_pci_driver *, struct rte_pci_device *);
 
 /**
+ * Uninitialisation function for the driver called during hotplugging.
+ */
+typedef int (pci_devuninit_t)(
+		struct rte_pci_driver *, struct rte_pci_device *);
+
+/**
  * A structure describing a PCI driver.
  */
 struct rte_pci_driver {
 	TAILQ_ENTRY(rte_pci_driver) next;       /**< Next in list. */
 	const char *name;                       /**< Driver name. */
 	pci_devinit_t *devinit;                 /**< Device init. function. */
+	pci_devuninit_t *devuninit;             /**< Device uninit function. */
 	struct rte_pci_id *id_table;            /**< ID table, NULL terminated. */
 	uint32_t drv_flags;                     /**< Flags contolling handling of device. */
 };
diff --git a/lib/librte_ether/rte_ethdev.c b/lib/librte_ether/rte_ethdev.c
index b58bab3..7bed901 100644
--- a/lib/librte_ether/rte_ethdev.c
+++ b/lib/librte_ether/rte_ethdev.c
@@ -326,6 +326,52 @@ rte_eth_dev_init(struct rte_pci_driver *pci_drv,
 	return diag;
 }
 
+static int
+rte_eth_dev_uninit(struct rte_pci_driver *pci_drv,
+		 struct rte_pci_device *pci_dev)
+{
+	struct eth_driver *eth_drv;
+	struct rte_eth_dev *eth_dev;
+	char ethdev_name[RTE_ETH_NAME_MAX_LEN];
+	int ret;
+
+	if ((pci_drv == NULL) || (pci_dev == NULL))
+		return -EINVAL;
+
+	/* Create unique Ethernet device name using PCI address */
+	snprintf(ethdev_name, RTE_ETH_NAME_MAX_LEN, "%d:%d.%d",
+			pci_dev->addr.bus, pci_dev->addr.devid,
+			pci_dev->addr.function);
+
+	eth_dev = rte_eth_dev_allocated(ethdev_name);
+	if (eth_dev == NULL)
+		return -ENODEV;
+
+	eth_drv = (struct eth_driver *)pci_drv;
+
+	/* Invoke PMD device uninit function */
+	if (*eth_drv->eth_dev_uninit) {
+		ret = (*eth_drv->eth_dev_uninit)(eth_drv, eth_dev);
+		if (ret)
+			return ret;
+	}
+
+	/* free ether device */
+	rte_eth_dev_free(eth_dev);
+
+	/* init user callbacks */
+	TAILQ_INIT(&(eth_dev->callbacks));
+
+	if (rte_eal_process_type() == RTE_PROC_PRIMARY)
+		rte_free(eth_dev->data->dev_private);
+
+	eth_dev->pci_dev = NULL;
+	eth_dev->driver = NULL;
+	eth_dev->data = NULL;
+
+	return 0;
+}
+
 /**
  * Register an Ethernet [Poll Mode] driver.
  *
@@ -344,6 +390,7 @@ void
 rte_eth_driver_register(struct eth_driver *eth_drv)
 {
 	eth_drv->pci_drv.devinit = rte_eth_dev_init;
+	eth_drv->pci_drv.devuninit = rte_eth_dev_uninit;
 	rte_eal_pci_register(&eth_drv->pci_drv);
 }
 
diff --git a/lib/librte_ether/rte_ethdev.h b/lib/librte_ether/rte_ethdev.h
index fbe7ac1..91d9e86 100644
--- a/lib/librte_ether/rte_ethdev.h
+++ b/lib/librte_ether/rte_ethdev.h
@@ -1678,6 +1678,27 @@ typedef int (*eth_dev_init_t)(struct eth_driver  *eth_drv,
 
 /**
  * @internal
+ * Finalization function of an Ethernet driver invoked for each matching
+ * Ethernet PCI device detected during the PCI closing phase.
+ *
+ * @param eth_drv
+ *   The pointer to the [matching] Ethernet driver structure supplied by
+ *   the PMD when it registered itself.
+ * @param eth_dev
+ *   The *eth_dev* pointer is the address of the *rte_eth_dev* structure
+ *   associated with the matching device and which have been [automatically]
+ *   allocated in the *rte_eth_devices* array.
+ * @return
+ *   - 0: Success, the device is properly finalized by the driver.
+ *        In particular, the driver MUST free the *dev_ops* pointer
+ *        of the *eth_dev* structure.
+ *   - <0: Error code of the device initialization failure.
+ */
+typedef int (*eth_dev_uninit_t)(struct eth_driver  *eth_drv,
+				  struct rte_eth_dev *eth_dev);
+
+/**
+ * @internal
  * The structure associated with a PMD Ethernet driver.
  *
  * Each Ethernet driver acts as a PCI driver and is represented by a generic
@@ -1687,11 +1708,14 @@ typedef int (*eth_dev_init_t)(struct eth_driver  *eth_drv,
  *
  * - The *eth_dev_init* function invoked for each matching PCI device.
  *
+ * - The *eth_dev_uninit* function invoked for each matching PCI device.
+ *
  * - The size of the private data to allocate for each matching device.
  */
 struct eth_driver {
 	struct rte_pci_driver pci_drv;    /**< The PMD is also a PCI driver. */
 	eth_dev_init_t eth_dev_init;      /**< Device init function. */
+	eth_dev_uninit_t eth_dev_uninit;  /**< Device uninit function. */
 	unsigned int dev_private_size;    /**< Size of device private data. */
 };
 
-- 
1.9.1

^ permalink raw reply	[flat|nested] 362+ messages in thread

* [dpdk-dev] [PATCH v6 07/13] ethdev: Add functions that will be used by port hotplug functions
  2015-02-01  4:01     ` [dpdk-dev] [PATCH v6 00/13] Port Hotplug Framework Tetsuya Mukawa
                         ` (5 preceding siblings ...)
  2015-02-01  4:01       ` [dpdk-dev] [PATCH v6 06/13] eal, ethdev: Add a function and function pointers to close ether device Tetsuya Mukawa
@ 2015-02-01  4:01       ` Tetsuya Mukawa
  2015-02-03  2:37         ` Qiu, Michael
  2015-02-01  4:01       ` [dpdk-dev] [PATCH v6 08/13] eal/linux/pci: Add functions for unmapping igb_uio resources Tetsuya Mukawa
                         ` (9 subsequent siblings)
  16 siblings, 1 reply; 362+ messages in thread
From: Tetsuya Mukawa @ 2015-02-01  4:01 UTC (permalink / raw)
  To: dev

The patch adds following functions.

- rte_eth_dev_save()
  The function is used for saving current rte_eth_dev structures.
- rte_eth_dev_get_changed_port()
  The function receives the rte_eth_dev structures, then compare
  these with current values to know which port is actually
  attached or detached.
- rte_eth_dev_get_addr_by_port()
  The function returns a pci address of a ethdev specified by port
  identifier.
- rte_eth_dev_get_port_by_addr()
  The function returns a port identifier of a ethdev specified by
  pci address.
- rte_eth_dev_get_name_by_port()
  The function returns a unique identifier name of a ethdev
  specified by port identifier.
- Add rte_eth_dev_check_detachable()
  The function returns whether a PMD supports detach function.

Also the patch changes scope of rte_eth_dev_allocated() to global.
This function will be called by virtual PMDs to support port hotplug.
So change scope of the function to global.

v5:
- Fix return value of below functions.
  rte_eth_dev_get_changed_port().
  rte_eth_dev_get_port_by_addr().
v4:
- Add paramerter checking.
v3:
- Fix if-condition bug while comparing pci addresses.
- Add error checking codes.
Reported-by: Mark Enright <menrigh@brocade.com>

Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp>
---
 lib/librte_ether/rte_ethdev.c | 98 ++++++++++++++++++++++++++++++++++++++++++-
 lib/librte_ether/rte_ethdev.h | 80 +++++++++++++++++++++++++++++++++++
 2 files changed, 177 insertions(+), 1 deletion(-)

diff --git a/lib/librte_ether/rte_ethdev.c b/lib/librte_ether/rte_ethdev.c
index 7bed901..5aded10 100644
--- a/lib/librte_ether/rte_ethdev.c
+++ b/lib/librte_ether/rte_ethdev.c
@@ -206,7 +206,7 @@ rte_eth_dev_data_alloc(void)
 				RTE_MAX_ETHPORTS * sizeof(*rte_eth_dev_data));
 }
 
-static struct rte_eth_dev *
+struct rte_eth_dev *
 rte_eth_dev_allocated(const char *name)
 {
 	unsigned i;
@@ -426,6 +426,102 @@ rte_eth_dev_count(void)
 	return (nb_ports);
 }
 
+void
+rte_eth_dev_save(struct rte_eth_dev *devs)
+{
+	if (devs == NULL)
+		return;
+
+	/* save current rte_eth_devices */
+	memcpy(devs, rte_eth_devices,
+			sizeof(struct rte_eth_dev) * RTE_MAX_ETHPORTS);
+}
+
+int
+rte_eth_dev_get_changed_port(struct rte_eth_dev *devs, uint8_t *port_id)
+{
+	if ((devs == NULL) || (port_id == NULL))
+		return -EINVAL;
+
+	/* check which port was attached or detached */
+	for (*port_id = 0; *port_id < RTE_MAX_ETHPORTS; (*port_id)++, devs++) {
+		if (rte_eth_devices[*port_id].attached ^ devs->attached)
+			return 0;
+	}
+	return -ENODEV;
+}
+
+int
+rte_eth_dev_get_addr_by_port(uint8_t port_id, struct rte_pci_addr *addr)
+{
+	if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID)
+		return -EINVAL;
+
+	if (addr == NULL) {
+		PMD_DEBUG_TRACE("Null pointer is specified\n");
+		return -EINVAL;
+	}
+
+	*addr = rte_eth_devices[port_id].pci_dev->addr;
+	return 0;
+}
+
+int
+rte_eth_dev_get_port_by_addr(struct rte_pci_addr *addr, uint8_t *port_id)
+{
+	struct rte_pci_addr *tmp;
+
+	if ((addr == NULL) || (port_id == NULL)) {
+		PMD_DEBUG_TRACE("Null pointer is specified\n");
+		return -EINVAL;
+	}
+
+	for (*port_id = 0; *port_id < RTE_MAX_ETHPORTS; (*port_id)++) {
+		if (!rte_eth_devices[*port_id].attached)
+			continue;
+		if (!rte_eth_devices[*port_id].pci_dev)
+			continue;
+		tmp = &rte_eth_devices[*port_id].pci_dev->addr;
+		if (eal_compare_pci_addr(tmp, addr) == 0)
+			return 0;
+	}
+	return -ENODEV;
+}
+
+int
+rte_eth_dev_get_name_by_port(uint8_t port_id, char *name)
+{
+	char *tmp;
+
+	if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID)
+		return -EINVAL;
+
+	if (name == NULL) {
+		PMD_DEBUG_TRACE("Null pointer is specified\n");
+		return -EINVAL;
+	}
+
+	/* shouldn't check 'rte_eth_devices[i].data',
+	 * because it might be overwritten by VDEV PMD */
+	tmp = rte_eth_dev_data[port_id].name;
+	strncpy(name, tmp, strlen(tmp) + 1);
+	return 0;
+}
+
+int
+rte_eth_dev_check_detachable(uint8_t port_id)
+{
+	uint32_t drv_flags;
+
+	if (port_id >= RTE_MAX_ETHPORTS) {
+		PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
+		return -EINVAL;
+	}
+
+	drv_flags = rte_eth_devices[port_id].driver->pci_drv.drv_flags;
+	return !(drv_flags & RTE_PCI_DRV_DETACHABLE);
+}
+
 static int
 rte_eth_dev_rx_queue_config(struct rte_eth_dev *dev, uint16_t nb_queues)
 {
diff --git a/lib/librte_ether/rte_ethdev.h b/lib/librte_ether/rte_ethdev.h
index 91d9e86..9919968 100644
--- a/lib/librte_ether/rte_ethdev.h
+++ b/lib/librte_ether/rte_ethdev.h
@@ -1616,6 +1616,86 @@ extern struct rte_eth_dev rte_eth_devices[];
 extern uint8_t rte_eth_dev_count(void);
 
 /**
+ * Function for internal use by port hotplug functions.
+ * Copies current ethdev structures to the specified pointer.
+ *
+ * @param	devs	The pointer to the ethdev structures
+ */
+extern void rte_eth_dev_save(struct rte_eth_dev *devs);
+
+/**
+ * Function for internal use by port hotplug functions.
+ * Compare the specified ethdev structures with currents. Then
+ * if there is a port which status is changed, fill the specified pointer
+ * with the port id of that port.
+ * @param	devs	The pointer to the ethdev structures
+ * @param	port_id	The pointer to the port id
+ * @return
+ *   - 0 on success, negative on error
+ */
+extern int rte_eth_dev_get_changed_port(
+		struct rte_eth_dev *devs, uint8_t *port_id);
+
+/**
+ * Function for internal use by port hotplug functions.
+ * Returns a pci address of a ethdev specified by port identifier.
+ * @param	port_id
+ *   The port identifier of the Ethernet device
+ * @param	addr
+ *   The pointer to the pci address
+ * @return
+ *   - 0 on success, negative on error
+ */
+extern int rte_eth_dev_get_addr_by_port(
+		uint8_t port_id, struct rte_pci_addr *addr);
+
+/**
+ * Function for internal use by port hotplug functions.
+ * Returns a port identifier of a ethdev specified by pci address.
+ * @param	addr
+ *   The pointer to the pci address of the Ethernet device.
+ * @param	port_id
+ *   The pointer to the port identifier
+ * @return
+ *   - 0 on success, negative on error
+ */
+extern int rte_eth_dev_get_port_by_addr(
+		struct rte_pci_addr *addr, uint8_t *port_id);
+
+/**
+ * Function for internal use by port hotplug functions.
+ * Returns a unique identifier name of a ethdev specified by port identifier.
+ * @param	port_id
+ *   The port identifier.
+ * @param	name
+ *  The pointer to the Unique identifier name for each Ethernet device
+ * @return
+ *   - 0 on success, negative on error
+ */
+extern int rte_eth_dev_get_name_by_port(uint8_t port_id, char *name);
+
+/**
+ * Function for internal use by port hotplug functions.
+ * Check whether or not, a PMD that is handling the ethdev specified by port
+ * identifier can support detach function.
+ * @param	port_id
+ *   The port identifier
+ * @return
+ *   - 0 on supporting detach function, negative on not supporting
+ */
+extern int rte_eth_dev_check_detachable(uint8_t port_id);
+
+/**
+ * Function for internal use by port hotplug functions.
+ * Returns a ethdev slot specified by the unique identifier name.
+ * @param	name
+ *  The pointer to the Unique identifier name for each Ethernet device
+ * @return
+ *   - The pointer to the ethdev slot, on success. NULL on error
+ */
+extern struct rte_eth_dev *rte_eth_dev_allocated(const char *name);
+
+/**
  * Function for internal use by dummy drivers primarily, e.g. ring-based
  * driver.
  * Allocates a new ethdev slot for an ethernet device and returns the pointer
-- 
1.9.1

^ permalink raw reply	[flat|nested] 362+ messages in thread

* [dpdk-dev] [PATCH v6 08/13] eal/linux/pci: Add functions for unmapping igb_uio resources
  2015-02-01  4:01     ` [dpdk-dev] [PATCH v6 00/13] Port Hotplug Framework Tetsuya Mukawa
                         ` (6 preceding siblings ...)
  2015-02-01  4:01       ` [dpdk-dev] [PATCH v6 07/13] ethdev: Add functions that will be used by port hotplug functions Tetsuya Mukawa
@ 2015-02-01  4:01       ` Tetsuya Mukawa
  2015-02-01  4:01       ` [dpdk-dev] [PATCH v6 09/13] eal/pci: Add a function to remove the entry of devargs list Tetsuya Mukawa
                         ` (8 subsequent siblings)
  16 siblings, 0 replies; 362+ messages in thread
From: Tetsuya Mukawa @ 2015-02-01  4:01 UTC (permalink / raw)
  To: dev

The patch adds functions for unmapping igb_uio resources. The patch is only
for Linux and igb_uio environment. VFIO and BSD are not supported.

v5:
- Fix pci_unmap_device() to check pt_driver.
v4:
- Add paramerter checking.
- Add header file to determine if hotplug can be enabled.

Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp>
---
 lib/librte_eal/common/Makefile                  |  1 +
 lib/librte_eal/common/include/rte_dev_hotplug.h | 44 +++++++++++++++++
 lib/librte_eal/linuxapp/eal/eal_pci.c           | 44 +++++++++++++++++
 lib/librte_eal/linuxapp/eal/eal_pci_init.h      |  8 +++
 lib/librte_eal/linuxapp/eal/eal_pci_uio.c       | 65 +++++++++++++++++++++++++
 5 files changed, 162 insertions(+)
 create mode 100644 lib/librte_eal/common/include/rte_dev_hotplug.h

diff --git a/lib/librte_eal/common/Makefile b/lib/librte_eal/common/Makefile
index 52c1a5f..db7cc93 100644
--- a/lib/librte_eal/common/Makefile
+++ b/lib/librte_eal/common/Makefile
@@ -41,6 +41,7 @@ INC += rte_eal_memconfig.h rte_malloc_heap.h
 INC += rte_hexdump.h rte_devargs.h rte_dev.h
 INC += rte_common_vect.h
 INC += rte_pci_dev_feature_defs.h rte_pci_dev_features.h
+INC += rte_dev_hotplug.h
 
 ifeq ($(CONFIG_RTE_INSECURE_FUNCTION_WARNING),y)
 INC += rte_warnings.h
diff --git a/lib/librte_eal/common/include/rte_dev_hotplug.h b/lib/librte_eal/common/include/rte_dev_hotplug.h
new file mode 100644
index 0000000..b333e0f
--- /dev/null
+++ b/lib/librte_eal/common/include/rte_dev_hotplug.h
@@ -0,0 +1,44 @@
+/*-
+ *   BSD LICENSE
+ *
+ *   Copyright(c) 2015 IGEL Co.,LTd.
+ *   All rights reserved.
+ *
+ *   Redistribution and use in source and binary forms, with or without
+ *   modification, are permitted provided that the following conditions
+ *   are met:
+ *
+ *     * Redistributions of source code must retain the above copyright
+ *       notice, this list of conditions and the following disclaimer.
+ *     * Redistributions in binary form must reproduce the above copyright
+ *       notice, this list of conditions and the following disclaimer in
+ *       the documentation and/or other materials provided with the
+ *       distribution.
+ *     * Neither the name of IGEL Co.,Ltd. nor the names of its
+ *       contributors may be used to endorse or promote products derived
+ *       from this software without specific prior written permission.
+ *
+ *   THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+ *   "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+ *   LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+ *   A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+ *   OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ *   SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+ *   LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+ *   DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+ *   THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ *   (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ *   OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#ifndef _RTE_DEV_HOTPLUG_H_
+#define _RTE_DEV_HOTPLUG_H_
+
+/*
+ * determine if hotplug can be enabled on the system
+ */
+#if defined(RTE_LIBRTE_EAL_HOTPLUG) && defined(RTE_LIBRTE_EAL_LINUXAPP)
+#define ENABLE_HOTPLUG
+#endif /* RTE_LIBRTE_EAL_HOTPLUG & RTE_LIBRTE_EAL_LINUXAPP */
+
+#endif /* _RTE_DEV_HOTPLUG_H_ */
diff --git a/lib/librte_eal/linuxapp/eal/eal_pci.c b/lib/librte_eal/linuxapp/eal/eal_pci.c
index d847102..c3b7917 100644
--- a/lib/librte_eal/linuxapp/eal/eal_pci.c
+++ b/lib/librte_eal/linuxapp/eal/eal_pci.c
@@ -166,6 +166,25 @@ pci_map_resource(void *requested_addr, int fd, off_t offset, size_t size)
 	return mapaddr;
 }
 
+#ifdef ENABLE_HOTPLUG
+/* unmap a particular resource */
+void
+pci_unmap_resource(void *requested_addr, size_t size)
+{
+	if (requested_addr == NULL)
+		return;
+
+	/* Unmap the PCI memory resource of device */
+	if (munmap(requested_addr, size)) {
+		RTE_LOG(ERR, EAL, "%s(): cannot munmap(%p, 0x%lx): %s\n",
+			__func__, requested_addr, (unsigned long)size,
+			strerror(errno));
+	} else
+		RTE_LOG(DEBUG, EAL, "  PCI memory mapped at %p\n",
+				requested_addr);
+}
+#endif /* ENABLE_HOTPLUG */
+
 /* parse the "resource" sysfs file */
 #define IORESOURCE_MEM  0x00000200
 
@@ -567,6 +586,31 @@ pci_map_device(struct rte_pci_device *dev)
 	return ret;
 }
 
+#ifdef ENABLE_HOTPLUG
+static void
+pci_unmap_device(struct rte_pci_device *dev)
+{
+	if (dev == NULL)
+		return;
+
+	/* try unmapping the NIC resources using VFIO if it exists */
+	switch (dev->pt_driver) {
+	case RTE_PT_VFIO:
+		RTE_LOG(ERR, EAL, "Hotplug doesn't support vfio yet\n");
+		break;
+	case RTE_PT_IGB_UIO:
+	case RTE_PT_UIO_GENERIC:
+		/* unmap resources for devices that use uio */
+		pci_uio_unmap_resource(dev);
+		break;
+	default:
+		RTE_LOG(DEBUG, EAL, "  Not managed by known pt driver,"
+			" skipped\n");
+		break;
+	}
+}
+#endif /* ENABLE_HOTPLUG */
+
 /*
  * If vendor/device ID match, call the devinit() function of the
  * driver.
diff --git a/lib/librte_eal/linuxapp/eal/eal_pci_init.h b/lib/librte_eal/linuxapp/eal/eal_pci_init.h
index 1070eb8..5152a0b 100644
--- a/lib/librte_eal/linuxapp/eal/eal_pci_init.h
+++ b/lib/librte_eal/linuxapp/eal/eal_pci_init.h
@@ -34,6 +34,7 @@
 #ifndef EAL_PCI_INIT_H_
 #define EAL_PCI_INIT_H_
 
+#include <rte_dev_hotplug.h>
 #include "eal_vfio.h"
 
 struct pci_map {
@@ -71,6 +72,13 @@ void *pci_map_resource(void *requested_addr, int fd, off_t offset,
 /* map IGB_UIO resource prototype */
 int pci_uio_map_resource(struct rte_pci_device *dev);
 
+#ifdef ENABLE_HOTPLUG
+void pci_unmap_resource(void *requested_addr, size_t size);
+
+/* unmap IGB_UIO resource prototype */
+void pci_uio_unmap_resource(struct rte_pci_device *dev);
+#endif /* ENABLE_HOTPLUG */
+
 #ifdef VFIO_PRESENT
 
 #define VFIO_MAX_GROUPS 64
diff --git a/lib/librte_eal/linuxapp/eal/eal_pci_uio.c b/lib/librte_eal/linuxapp/eal/eal_pci_uio.c
index 1da3507..81830d1 100644
--- a/lib/librte_eal/linuxapp/eal/eal_pci_uio.c
+++ b/lib/librte_eal/linuxapp/eal/eal_pci_uio.c
@@ -404,6 +404,71 @@ pci_uio_map_resource(struct rte_pci_device *dev)
 	return 0;
 }
 
+#ifdef ENABLE_HOTPLUG
+static void
+pci_uio_unmap(struct mapped_pci_resource *uio_res)
+{
+	int i;
+
+	if (uio_res == NULL)
+		return;
+
+	for (i = 0; i != uio_res->nb_maps; i++)
+		pci_unmap_resource(uio_res->maps[i].addr,
+				(size_t)uio_res->maps[i].size);
+}
+
+static struct mapped_pci_resource *
+pci_uio_find_resource(struct rte_pci_device *dev)
+{
+	struct mapped_pci_resource *uio_res;
+
+	if (dev == NULL)
+		return NULL;
+
+	TAILQ_FOREACH(uio_res, pci_res_list, next) {
+
+		/* skip this element if it doesn't match our PCI address */
+		if (!eal_compare_pci_addr(&uio_res->pci_addr, &dev->addr))
+			return uio_res;
+	}
+	return NULL;
+}
+
+/* unmap the PCI resource of a PCI device in virtual memory */
+void
+pci_uio_unmap_resource(struct rte_pci_device *dev)
+{
+	struct mapped_pci_resource *uio_res;
+
+	if (dev == NULL)
+		return;
+
+	/* find an entry for the device */
+	uio_res = pci_uio_find_resource(dev);
+	if (uio_res == NULL)
+		return;
+
+	/* secondary processes - just free maps */
+	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
+		return pci_uio_unmap(uio_res);
+
+	TAILQ_REMOVE(pci_res_list, uio_res, next);
+
+	/* unmap all resources */
+	pci_uio_unmap(uio_res);
+
+	/* free uio resource */
+	rte_free(uio_res);
+
+	/* close fd if in primary process */
+	close(dev->intr_handle.fd);
+
+	dev->intr_handle.fd = -1;
+	dev->intr_handle.type = RTE_INTR_HANDLE_UNKNOWN;
+}
+#endif /* ENABLE_HOTPLUG */
+
 /*
  * parse a sysfs file containing one integer value
  * different to the eal version, as it needs to work with 64-bit values
-- 
1.9.1

^ permalink raw reply	[flat|nested] 362+ messages in thread

* [dpdk-dev] [PATCH v6 09/13] eal/pci: Add a function to remove the entry of devargs list
  2015-02-01  4:01     ` [dpdk-dev] [PATCH v6 00/13] Port Hotplug Framework Tetsuya Mukawa
                         ` (7 preceding siblings ...)
  2015-02-01  4:01       ` [dpdk-dev] [PATCH v6 08/13] eal/linux/pci: Add functions for unmapping igb_uio resources Tetsuya Mukawa
@ 2015-02-01  4:01       ` Tetsuya Mukawa
  2015-02-01  4:01       ` [dpdk-dev] [PATCH v6 10/13] eal/pci: Cleanup pci driver initialization code Tetsuya Mukawa
                         ` (7 subsequent siblings)
  16 siblings, 0 replies; 362+ messages in thread
From: Tetsuya Mukawa @ 2015-02-01  4:01 UTC (permalink / raw)
  To: dev

The function removes the specified devargs entry from devargs_list.
Also the patch adds sanity checking to rte_eal_devargs_add().

v5:
- Change function definition of rte_eal_devargs_remove().
v4:
- Fix sanity check code.

Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp>
---
 lib/librte_eal/common/eal_common_devargs.c  | 60 +++++++++++++++++++++++++++++
 lib/librte_eal/common/include/rte_devargs.h | 21 ++++++++++
 2 files changed, 81 insertions(+)

diff --git a/lib/librte_eal/common/eal_common_devargs.c b/lib/librte_eal/common/eal_common_devargs.c
index 4c7d11a..5b1ac8e 100644
--- a/lib/librte_eal/common/eal_common_devargs.c
+++ b/lib/librte_eal/common/eal_common_devargs.c
@@ -44,6 +44,35 @@
 struct rte_devargs_list devargs_list =
 	TAILQ_HEAD_INITIALIZER(devargs_list);
 
+
+/* find a entry specified by pci address or device name */
+static struct rte_devargs *
+rte_eal_devargs_find(enum rte_devtype devtype, void *args)
+{
+	struct rte_devargs *devargs;
+
+	if (args == NULL)
+		return NULL;
+
+	TAILQ_FOREACH(devargs, &devargs_list, next) {
+		switch (devtype) {
+		case RTE_DEVTYPE_WHITELISTED_PCI:
+		case RTE_DEVTYPE_BLACKLISTED_PCI:
+			if (eal_compare_pci_addr(&devargs->pci.addr, args) == 0)
+				goto found;
+			break;
+		case RTE_DEVTYPE_VIRTUAL:
+			if (memcmp(&devargs->virtual.drv_name, args,
+			    strlen((char *)args)) == 0)
+				goto found;
+			break;
+		}
+	}
+	return NULL;
+found:
+	return devargs;
+}
+
 /* store a whitelist parameter for later parsing */
 int
 rte_eal_devargs_add(enum rte_devtype devtype, const char *devargs_str)
@@ -87,6 +116,12 @@ rte_eal_devargs_add(enum rte_devtype devtype, const char *devargs_str)
 			free(devargs);
 			return -1;
 		}
+		/* make sure there is no same entry */
+		if (rte_eal_devargs_find(devtype, &devargs->pci.addr)) {
+			RTE_LOG(ERR, EAL,
+				"device already registered: <%s>\n", buf);
+			return -1;
+		}
 		break;
 	case RTE_DEVTYPE_VIRTUAL:
 		/* save driver name */
@@ -98,6 +133,12 @@ rte_eal_devargs_add(enum rte_devtype devtype, const char *devargs_str)
 			free(devargs);
 			return -1;
 		}
+		/* make sure there is no same entry */
+		if (rte_eal_devargs_find(devtype, &devargs->virtual.drv_name)) {
+			RTE_LOG(ERR, EAL,
+				"device already registered: <%s>\n", buf);
+			return -1;
+		}
 		break;
 	}
 
@@ -105,6 +146,25 @@ rte_eal_devargs_add(enum rte_devtype devtype, const char *devargs_str)
 	return 0;
 }
 
+/* remove it from the devargs_list */
+int
+rte_eal_devargs_remove(enum rte_devtype devtype, void *args)
+{
+	struct rte_devargs *devargs;
+
+	if (args == NULL)
+		return -EINVAL;
+
+	devargs = rte_eal_devargs_find(devtype, args);
+	if (devargs == NULL) {
+		RTE_LOG(ERR, EAL, "device not found\n");
+		return -ENODEV;
+	}
+
+	TAILQ_REMOVE(&devargs_list, devargs, next);
+	return 0;
+}
+
 /* count the number of devices of a specified type */
 unsigned int
 rte_eal_devargs_type_count(enum rte_devtype devtype)
diff --git a/lib/librte_eal/common/include/rte_devargs.h b/lib/librte_eal/common/include/rte_devargs.h
index 9f9c98f..b5ad4b3 100644
--- a/lib/librte_eal/common/include/rte_devargs.h
+++ b/lib/librte_eal/common/include/rte_devargs.h
@@ -123,6 +123,27 @@ extern struct rte_devargs_list devargs_list;
 int rte_eal_devargs_add(enum rte_devtype devtype, const char *devargs_str);
 
 /**
+ * Remove a device from the user device list
+ *
+ * For PCI devices, the format of arguments string is "PCI_ADDR". It shouldn't
+ * involves parameters for the device. Example: "08:00.1".
+ *
+ * For virtual devices, the format of arguments string is "DRIVER_NAME*". It
+ * shouldn't involves parameters for the device. Example: "eth_ring". The
+ * validity of the driver name is not checked by this function, it is done
+ * when closing the drivers.
+ *
+ * @param devtype
+ *   The type of the device.
+ * @param name
+ *   The name of the device.
+ *
+ * @return
+ *   - 0 on success, negative on error
+ */
+int rte_eal_devargs_remove(enum rte_devtype devtype, void *args);
+
+/**
  * Count the number of user devices of a specified type
  *
  * @param devtype
-- 
1.9.1

^ permalink raw reply	[flat|nested] 362+ messages in thread

* [dpdk-dev] [PATCH v6 10/13] eal/pci: Cleanup pci driver initialization code
  2015-02-01  4:01     ` [dpdk-dev] [PATCH v6 00/13] Port Hotplug Framework Tetsuya Mukawa
                         ` (8 preceding siblings ...)
  2015-02-01  4:01       ` [dpdk-dev] [PATCH v6 09/13] eal/pci: Add a function to remove the entry of devargs list Tetsuya Mukawa
@ 2015-02-01  4:01       ` Tetsuya Mukawa
  2015-02-03  2:35         ` Qiu, Michael
  2015-02-01  4:01       ` [dpdk-dev] [PATCH v6 11/13] ethdev: Add one dev_type paramerter to rte_eth_dev_allocate Tetsuya Mukawa
                         ` (6 subsequent siblings)
  16 siblings, 1 reply; 362+ messages in thread
From: Tetsuya Mukawa @ 2015-02-01  4:01 UTC (permalink / raw)
  To: dev

- Add rte_eal_pci_close_one_dirver()
  The function is used for closing the specified driver and device.
- Add pci_invoke_all_drivers()
  The function is based on pci_probe_all_drivers. But it can not only
  probe but also close drivers.
- Add pci_close_all_drivers()
  The function tries to find a driver for the specified device, and
  then close the driver.
- Add rte_eal_pci_probe_one() and rte_eal_pci_close_one()
  The functions are used for probe and close a device.
  First the function tries to find a device that has the specfied
  PCI address. Then, probe or close the device.

v5:
- Remove RTE_EAL_INVOKE_TYPE_UNKNOWN, because it's unused.
v4:
- Fix paramerter checking.
- Fix indent of 'if' statement.

Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp>
---
 lib/librte_eal/common/eal_common_pci.c  | 90 +++++++++++++++++++++++++++++----
 lib/librte_eal/common/eal_private.h     | 24 +++++++++
 lib/librte_eal/common/include/rte_pci.h | 33 ++++++++++++
 lib/librte_eal/linuxapp/eal/eal_pci.c   | 69 +++++++++++++++++++++++++
 4 files changed, 206 insertions(+), 10 deletions(-)

diff --git a/lib/librte_eal/common/eal_common_pci.c b/lib/librte_eal/common/eal_common_pci.c
index a89f5c3..7c9b8c5 100644
--- a/lib/librte_eal/common/eal_common_pci.c
+++ b/lib/librte_eal/common/eal_common_pci.c
@@ -99,19 +99,27 @@ static struct rte_devargs *pci_devargs_lookup(struct rte_pci_device *dev)
 	return NULL;
 }
 
-/*
- * If vendor/device ID match, call the devinit() function of all
- * registered driver for the given device. Return -1 if initialization
- * failed, return 1 if no driver is found for this device.
- */
 static int
-pci_probe_all_drivers(struct rte_pci_device *dev)
+pci_invoke_all_drivers(struct rte_pci_device *dev,
+		enum rte_eal_invoke_type type)
 {
 	struct rte_pci_driver *dr = NULL;
-	int rc;
+	int rc = 0;
+
+	if ((dev == NULL) || (type >= RTE_EAL_INVOKE_TYPE_MAX))
+		return -1;
 
 	TAILQ_FOREACH(dr, &pci_driver_list, next) {
-		rc = rte_eal_pci_probe_one_driver(dr, dev);
+		switch (type) {
+		case RTE_EAL_INVOKE_TYPE_PROBE:
+			rc = rte_eal_pci_probe_one_driver(dr, dev);
+			break;
+		case RTE_EAL_INVOKE_TYPE_CLOSE:
+			rc = rte_eal_pci_close_one_driver(dr, dev);
+			break;
+		default:
+			return -1;
+		}
 		if (rc < 0)
 			/* negative value is an error */
 			return -1;
@@ -123,6 +131,66 @@ pci_probe_all_drivers(struct rte_pci_device *dev)
 	return 1;
 }
 
+#ifdef ENABLE_HOTPLUG
+static int
+rte_eal_pci_invoke_one(struct rte_pci_addr *addr,
+		enum rte_eal_invoke_type type)
+{
+	struct rte_pci_device *dev = NULL;
+	int ret = 0;
+
+	if ((addr == NULL) || (type >= RTE_EAL_INVOKE_TYPE_MAX))
+		return -1;
+
+	TAILQ_FOREACH(dev, &pci_device_list, next) {
+		if (eal_compare_pci_addr(&dev->addr, addr))
+			continue;
+
+		ret = pci_invoke_all_drivers(dev, type);
+		if (ret < 0)
+			goto invoke_err_return;
+
+		if (type == RTE_EAL_INVOKE_TYPE_CLOSE)
+			goto remove_dev;
+
+		return 0;
+	}
+
+	return -1;
+
+invoke_err_return:
+	RTE_LOG(WARNING, EAL, "Requested device " PCI_PRI_FMT
+			" cannot be used\n", dev->addr.domain, dev->addr.bus,
+			dev->addr.devid, dev->addr.function);
+	return -1;
+
+remove_dev:
+	TAILQ_REMOVE(&pci_device_list, dev, next);
+	return 0;
+}
+
+
+/*
+ * Find the pci device specified by pci address, then invoke probe function of
+ * the driver of the devive.
+ */
+int
+rte_eal_pci_probe_one(struct rte_pci_addr *addr)
+{
+	return rte_eal_pci_invoke_one(addr, RTE_EAL_INVOKE_TYPE_PROBE);
+}
+
+/*
+ * Find the pci device specified by pci address, then invoke close function of
+ * the driver of the devive.
+ */
+int
+rte_eal_pci_close_one(struct rte_pci_addr *addr)
+{
+	return rte_eal_pci_invoke_one(addr, RTE_EAL_INVOKE_TYPE_CLOSE);
+}
+#endif /* ENABLE_HOTPLUG */
+
 /*
  * Scan the content of the PCI bus, and call the devinit() function for
  * all registered drivers that have a matching entry in its id_table
@@ -148,10 +216,12 @@ rte_eal_pci_probe(void)
 
 		/* probe all or only whitelisted devices */
 		if (probe_all)
-			ret = pci_probe_all_drivers(dev);
+			ret = pci_invoke_all_drivers(dev,
+					RTE_EAL_INVOKE_TYPE_PROBE);
 		else if (devargs != NULL &&
 			devargs->type == RTE_DEVTYPE_WHITELISTED_PCI)
-			ret = pci_probe_all_drivers(dev);
+			ret = pci_invoke_all_drivers(dev,
+					RTE_EAL_INVOKE_TYPE_PROBE);
 		if (ret < 0)
 			rte_exit(EXIT_FAILURE, "Requested device " PCI_PRI_FMT
 				 " cannot be used\n", dev->addr.domain, dev->addr.bus,
diff --git a/lib/librte_eal/common/eal_private.h b/lib/librte_eal/common/eal_private.h
index 159cd66..1a362ab 100644
--- a/lib/librte_eal/common/eal_private.h
+++ b/lib/librte_eal/common/eal_private.h
@@ -154,6 +154,15 @@ struct rte_pci_driver;
 struct rte_pci_device;
 
 /**
+ * The invoke type.
+ */
+enum rte_eal_invoke_type {
+	RTE_EAL_INVOKE_TYPE_PROBE,  /**< invoke probe function */
+	RTE_EAL_INVOKE_TYPE_CLOSE,  /**< invoke close function */
+	RTE_EAL_INVOKE_TYPE_MAX     /**< max value of this enum */
+};
+
+/**
  * Mmap memory for single PCI device
  *
  * This function is private to EAL.
@@ -165,6 +174,21 @@ int rte_eal_pci_probe_one_driver(struct rte_pci_driver *dr,
 		struct rte_pci_device *dev);
 
 /**
+ * Munmap memory for single PCI device
+ *
+ * This function is private to EAL.
+ *
+ * @param	dr
+ *  The pointer to the pci driver structure
+ * @param	dev
+ *  The pointer to the pci device structure
+ * @return
+ *   0 on success, negative on error
+ */
+int rte_eal_pci_close_one_driver(struct rte_pci_driver *dr,
+		struct rte_pci_device *dev);
+
+/**
  * Init tail queues for non-EAL library structures. This is to allow
  * the rings, mempools, etc. lists to be shared among multiple processes
  *
diff --git a/lib/librte_eal/common/include/rte_pci.h b/lib/librte_eal/common/include/rte_pci.h
index 87ca4cf..a111066 100644
--- a/lib/librte_eal/common/include/rte_pci.h
+++ b/lib/librte_eal/common/include/rte_pci.h
@@ -82,6 +82,7 @@ extern "C" {
 #include <inttypes.h>
 
 #include <rte_interrupts.h>
+#include <rte_dev_hotplug.h>
 
 TAILQ_HEAD(pci_device_list, rte_pci_device); /**< PCI devices in D-linked Q. */
 TAILQ_HEAD(pci_driver_list, rte_pci_driver); /**< PCI drivers in D-linked Q. */
@@ -323,6 +324,38 @@ eal_compare_pci_addr(struct rte_pci_addr *addr, struct rte_pci_addr *addr2)
  */
 int rte_eal_pci_probe(void);
 
+#ifdef ENABLE_HOTPLUG
+/**
+ * Probe the single PCI device.
+ *
+ * Scan the content of the PCI bus, and find the pci device specified by pci
+ * address, then call the probe() function for registered driver that has a
+ * matching entry in its id_table for discovered device.
+ *
+ * @param addr
+ *	The PCI Bus-Device-Function address to probe or close.
+ * @return
+ *   - 0 on success.
+ *   - Negative on error.
+ */
+int rte_eal_pci_probe_one(struct rte_pci_addr *addr);
+
+/**
+ * Close the single PCI device.
+ *
+ * Scan the content of the PCI bus, and find the pci device specified by pci
+ * address, then call the close() function for registered driver that has a
+ * matching entry in its id_table for discovered device.
+ *
+ * @param addr
+ *	The PCI Bus-Device-Function address to probe or close.
+ * @return
+ *   - 0 on success.
+ *   - Negative on error.
+ */
+int rte_eal_pci_close_one(struct rte_pci_addr *addr);
+#endif /* ENABLE_HOTPLUG */
+
 /**
  * Dump the content of the PCI bus.
  *
diff --git a/lib/librte_eal/linuxapp/eal/eal_pci.c b/lib/librte_eal/linuxapp/eal/eal_pci.c
index c3b7917..831422e 100644
--- a/lib/librte_eal/linuxapp/eal/eal_pci.c
+++ b/lib/librte_eal/linuxapp/eal/eal_pci.c
@@ -682,6 +682,75 @@ rte_eal_pci_probe_one_driver(struct rte_pci_driver *dr, struct rte_pci_device *d
 	return 1;
 }
 
+#ifdef ENABLE_HOTPLUG
+/*
+ * If vendor/device ID match, call the devuninit() function of the
+ * driver.
+ */
+int
+rte_eal_pci_close_one_driver(struct rte_pci_driver *dr,
+		struct rte_pci_device *dev)
+{
+	struct rte_pci_id *id_table;
+
+	if ((dr == NULL) || (dev == NULL))
+		return -EINVAL;
+
+	for (id_table = dr->id_table ; id_table->vendor_id != 0; id_table++) {
+
+		/* check if device's identifiers match the driver's ones */
+		if (id_table->vendor_id != dev->id.vendor_id &&
+		    id_table->vendor_id != PCI_ANY_ID)
+			continue;
+		if (id_table->device_id != dev->id.device_id &&
+		    id_table->device_id != PCI_ANY_ID)
+			continue;
+		if (id_table->subsystem_vendor_id !=
+		    dev->id.subsystem_vendor_id &&
+		    id_table->subsystem_vendor_id != PCI_ANY_ID)
+			continue;
+		if (id_table->subsystem_device_id !=
+		    dev->id.subsystem_device_id &&
+		    id_table->subsystem_device_id != PCI_ANY_ID)
+			continue;
+
+		struct rte_pci_addr *loc = &dev->addr;
+
+		RTE_LOG(DEBUG, EAL,
+				"PCI device "PCI_PRI_FMT" on NUMA socket %i\n",
+				loc->domain, loc->bus, loc->devid,
+				loc->function, dev->numa_node);
+
+		RTE_LOG(DEBUG, EAL, "  remove driver: %x:%x %s\n",
+				dev->id.vendor_id, dev->id.device_id,
+				dr->name);
+
+		/* call the driver devuninit() function */
+		if (dr->devuninit && (dr->devuninit(dr, dev) < 0))
+			return -1;	/* negative value is an error */
+
+		/* clear driver structure */
+		dev->driver = NULL;
+
+		if (dr->drv_flags & RTE_PCI_DRV_NEED_MAPPING)
+			/* unmap resources for devices that use igb_uio */
+			pci_unmap_device(dev);
+
+		return 0;
+	}
+	/* return positive value if driver is not found */
+	return 1;
+}
+#else /* ENABLE_HOTPLUG */
+int
+rte_eal_pci_close_one_driver(struct rte_pci_driver *dr __rte_unused,
+		struct rte_pci_device *dev __rte_unused)
+{
+	RTE_LOG(ERR, EAL, "Hotplug support isn't enabled\n");
+	return -1;
+}
+#endif /* ENABLE_HOTPLUG */
+
 /* Init the PCI EAL subsystem */
 int
 rte_eal_pci_init(void)
-- 
1.9.1

^ permalink raw reply	[flat|nested] 362+ messages in thread

* [dpdk-dev] [PATCH v6 11/13] ethdev: Add one dev_type paramerter to rte_eth_dev_allocate
  2015-02-01  4:01     ` [dpdk-dev] [PATCH v6 00/13] Port Hotplug Framework Tetsuya Mukawa
                         ` (9 preceding siblings ...)
  2015-02-01  4:01       ` [dpdk-dev] [PATCH v6 10/13] eal/pci: Cleanup pci driver initialization code Tetsuya Mukawa
@ 2015-02-01  4:01       ` Tetsuya Mukawa
  2015-02-01  4:01       ` [dpdk-dev] [PATCH v6 12/13] eal/pci: Add rte_eal_dev_attach/detach() functions Tetsuya Mukawa
                         ` (5 subsequent siblings)
  16 siblings, 0 replies; 362+ messages in thread
From: Tetsuya Mukawa @ 2015-02-01  4:01 UTC (permalink / raw)
  To: dev

This new parameter is needed to keep device type like physical or virtual.
Port detaching processes are different between physical and virtual.
This paramerter lets detaching function know a device type of the port.

v4:
- Fix comments of rte_eth_dev_type.

Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp>
---
 app/test/virtual_pmd.c                       |  2 +-
 lib/librte_ether/rte_ethdev.c                | 14 ++++++++++++--
 lib/librte_ether/rte_ethdev.h                | 25 ++++++++++++++++++++++++-
 lib/librte_pmd_af_packet/rte_eth_af_packet.c |  2 +-
 lib/librte_pmd_bond/rte_eth_bond_api.c       |  2 +-
 lib/librte_pmd_pcap/rte_eth_pcap.c           |  2 +-
 lib/librte_pmd_ring/rte_eth_ring.c           |  2 +-
 lib/librte_pmd_xenvirt/rte_eth_xenvirt.c     |  2 +-
 8 files changed, 42 insertions(+), 9 deletions(-)

diff --git a/app/test/virtual_pmd.c b/app/test/virtual_pmd.c
index 9fac95d..8d3a5ff 100644
--- a/app/test/virtual_pmd.c
+++ b/app/test/virtual_pmd.c
@@ -556,7 +556,7 @@ virtual_ethdev_create(const char *name, struct ether_addr *mac_addr,
 		goto err;
 
 	/* reserve an ethdev entry */
-	eth_dev = rte_eth_dev_allocate(name);
+	eth_dev = rte_eth_dev_allocate(name, RTE_ETH_DEV_PHYSICAL);
 	if (eth_dev == NULL)
 		goto err;
 
diff --git a/lib/librte_ether/rte_ethdev.c b/lib/librte_ether/rte_ethdev.c
index 5aded10..b54b9ab 100644
--- a/lib/librte_ether/rte_ethdev.c
+++ b/lib/librte_ether/rte_ethdev.c
@@ -232,7 +232,7 @@ rte_eth_dev_allocate_new_port(void)
 }
 
 struct rte_eth_dev *
-rte_eth_dev_allocate(const char *name)
+rte_eth_dev_allocate(const char *name, enum rte_eth_dev_type type)
 {
 	uint8_t port_id;
 	struct rte_eth_dev *eth_dev;
@@ -256,6 +256,7 @@ rte_eth_dev_allocate(const char *name)
 	snprintf(eth_dev->data->name, sizeof(eth_dev->data->name), "%s", name);
 	eth_dev->data->port_id = port_id;
 	eth_dev->attached = DEV_CONNECTED;
+	eth_dev->dev_type = type;
 	nb_ports++;
 	return eth_dev;
 }
@@ -267,6 +268,7 @@ rte_eth_dev_free(struct rte_eth_dev *eth_dev)
 		return -EINVAL;
 
 	eth_dev->attached = 0;
+	eth_dev->dev_type = RTE_ETH_DEV_UNKNOWN;
 	nb_ports--;
 	return 0;
 }
@@ -287,7 +289,7 @@ rte_eth_dev_init(struct rte_pci_driver *pci_drv,
 	snprintf(ethdev_name, RTE_ETH_NAME_MAX_LEN, "%d:%d.%d",
 			pci_dev->addr.bus, pci_dev->addr.devid, pci_dev->addr.function);
 
-	eth_dev = rte_eth_dev_allocate(ethdev_name);
+	eth_dev = rte_eth_dev_allocate(ethdev_name, RTE_ETH_DEV_PHYSICAL);
 	if (eth_dev == NULL)
 		return -ENOMEM;
 
@@ -426,6 +428,14 @@ rte_eth_dev_count(void)
 	return (nb_ports);
 }
 
+enum rte_eth_dev_type
+rte_eth_dev_get_device_type(uint8_t port_id)
+{
+	if (rte_eth_dev_validate_port(port_id, NONE_TRACE) == DEV_INVALID)
+		return -1;
+	return rte_eth_devices[port_id].dev_type;
+}
+
 void
 rte_eth_dev_save(struct rte_eth_dev *devs)
 {
diff --git a/lib/librte_ether/rte_ethdev.h b/lib/librte_ether/rte_ethdev.h
index 9919968..00a6218 100644
--- a/lib/librte_ether/rte_ethdev.h
+++ b/lib/librte_ether/rte_ethdev.h
@@ -1522,6 +1522,17 @@ struct eth_dev_ops {
 };
 
 /**
+ * The eth device type
+ */
+enum rte_eth_dev_type {
+	RTE_ETH_DEV_UNKNOWN,	/**< unknown device type */
+	RTE_ETH_DEV_PHYSICAL,
+		/**< Physical function and Virtual function devices of NIC */
+	RTE_ETH_DEV_VIRTUAL,	/**< non hardware device */
+	RTE_ETH_DEV_MAX		/**< max value of this enum */
+};
+
+/**
  * @internal
  * The generic data structure associated with each ethernet device.
  *
@@ -1540,6 +1551,7 @@ struct rte_eth_dev {
 	struct rte_pci_device *pci_dev; /**< PCI info. supplied by probing */
 	struct rte_eth_dev_cb_list callbacks; /**< User application callbacks */
 	uint8_t attached; /**< Flag indicating the port is attached */
+	enum rte_eth_dev_type dev_type; /**< Flag indicating the device type */
 };
 
 struct rte_eth_dev_sriov {
@@ -1617,6 +1629,15 @@ extern uint8_t rte_eth_dev_count(void);
 
 /**
  * Function for internal use by port hotplug functions.
+ * Get the device type to know whether the device is physical or virtual.
+ * @param	port_id	The pointer to the port id
+ * @return
+ *   - Device type.
+ */
+extern enum rte_eth_dev_type rte_eth_dev_get_device_type(uint8_t port_id);
+
+/**
+ * Function for internal use by port hotplug functions.
  * Copies current ethdev structures to the specified pointer.
  *
  * @param	devs	The pointer to the ethdev structures
@@ -1702,10 +1723,12 @@ extern struct rte_eth_dev *rte_eth_dev_allocated(const char *name);
  * to that slot for the driver to use.
  *
  * @param	name	Unique identifier name for each Ethernet device
+ * @param	type	Device type of this Ethernet device
  * @return
  *   - Slot in the rte_dev_devices array for a new device;
  */
-struct rte_eth_dev *rte_eth_dev_allocate(const char *name);
+struct rte_eth_dev *rte_eth_dev_allocate(const char *name,
+		enum rte_eth_dev_type type);
 
 /**
  * Function for internal use by dummy drivers primarily, e.g. ring-based
diff --git a/lib/librte_pmd_af_packet/rte_eth_af_packet.c b/lib/librte_pmd_af_packet/rte_eth_af_packet.c
index 1ffe1cd..80e9bdf 100644
--- a/lib/librte_pmd_af_packet/rte_eth_af_packet.c
+++ b/lib/librte_pmd_af_packet/rte_eth_af_packet.c
@@ -649,7 +649,7 @@ rte_pmd_init_internals(const char *name,
 	}
 
 	/* reserve an ethdev entry */
-	*eth_dev = rte_eth_dev_allocate(name);
+	*eth_dev = rte_eth_dev_allocate(name, RTE_ETH_DEV_VIRTUAL);
 	if (*eth_dev == NULL)
 		goto error;
 
diff --git a/lib/librte_pmd_bond/rte_eth_bond_api.c b/lib/librte_pmd_bond/rte_eth_bond_api.c
index 4ab3267..7a6a5f7 100644
--- a/lib/librte_pmd_bond/rte_eth_bond_api.c
+++ b/lib/librte_pmd_bond/rte_eth_bond_api.c
@@ -235,7 +235,7 @@ rte_eth_bond_create(const char *name, uint8_t mode, uint8_t socket_id)
 	}
 
 	/* reserve an ethdev entry */
-	eth_dev = rte_eth_dev_allocate(name);
+	eth_dev = rte_eth_dev_allocate(name, RTE_ETH_DEV_VIRTUAL);
 	if (eth_dev == NULL) {
 		RTE_BOND_LOG(ERR, "Unable to allocate rte_eth_dev");
 		goto err;
diff --git a/lib/librte_pmd_pcap/rte_eth_pcap.c b/lib/librte_pmd_pcap/rte_eth_pcap.c
index d299288..af7fae8 100644
--- a/lib/librte_pmd_pcap/rte_eth_pcap.c
+++ b/lib/librte_pmd_pcap/rte_eth_pcap.c
@@ -709,7 +709,7 @@ rte_pmd_init_internals(const char *name, const unsigned nb_rx_queues,
 		goto error;
 
 	/* reserve an ethdev entry */
-	*eth_dev = rte_eth_dev_allocate(name);
+	*eth_dev = rte_eth_dev_allocate(name, RTE_ETH_DEV_VIRTUAL);
 	if (*eth_dev == NULL)
 		goto error;
 
diff --git a/lib/librte_pmd_ring/rte_eth_ring.c b/lib/librte_pmd_ring/rte_eth_ring.c
index f685f08..5876057 100644
--- a/lib/librte_pmd_ring/rte_eth_ring.c
+++ b/lib/librte_pmd_ring/rte_eth_ring.c
@@ -255,7 +255,7 @@ rte_eth_from_rings(const char *name, struct rte_ring *const rx_queues[],
 		goto error;
 
 	/* reserve an ethdev entry */
-	eth_dev = rte_eth_dev_allocate(name);
+	eth_dev = rte_eth_dev_allocate(name, RTE_ETH_DEV_VIRTUAL);
 	if (eth_dev == NULL)
 		goto error;
 
diff --git a/lib/librte_pmd_xenvirt/rte_eth_xenvirt.c b/lib/librte_pmd_xenvirt/rte_eth_xenvirt.c
index 04e30c9..bc403d6 100644
--- a/lib/librte_pmd_xenvirt/rte_eth_xenvirt.c
+++ b/lib/librte_pmd_xenvirt/rte_eth_xenvirt.c
@@ -648,7 +648,7 @@ eth_dev_xenvirt_create(const char *name, const char *params,
 		goto err;
 
 	/* reserve an ethdev entry */
-	eth_dev = rte_eth_dev_allocate(name);
+	eth_dev = rte_eth_dev_allocate(name, RTE_ETH_DEV_VIRTUAL);
 	if (eth_dev == NULL)
 		goto err;
 
-- 
1.9.1

^ permalink raw reply	[flat|nested] 362+ messages in thread

* [dpdk-dev] [PATCH v6 12/13] eal/pci: Add rte_eal_dev_attach/detach() functions
  2015-02-01  4:01     ` [dpdk-dev] [PATCH v6 00/13] Port Hotplug Framework Tetsuya Mukawa
                         ` (10 preceding siblings ...)
  2015-02-01  4:01       ` [dpdk-dev] [PATCH v6 11/13] ethdev: Add one dev_type paramerter to rte_eth_dev_allocate Tetsuya Mukawa
@ 2015-02-01  4:01       ` Tetsuya Mukawa
  2015-02-02  5:42         ` Qiu, Michael
  2015-02-01  4:02       ` [dpdk-dev] [PATCH v6 13/13] eal: Enable port hotplug framework in Linux Tetsuya Mukawa
                         ` (4 subsequent siblings)
  16 siblings, 1 reply; 362+ messages in thread
From: Tetsuya Mukawa @ 2015-02-01  4:01 UTC (permalink / raw)
  To: dev

These functions are used for attaching or detaching a port.
When rte_eal_dev_attach() is called, the function tries to realize the
device name as pci address. If this is done successfully,
rte_eal_dev_attach() will attach physical device port. If not, attaches
virtual devive port.
When rte_eal_dev_detach() is called, the function gets the device type
of this port to know whether the port is came from physical or virtual.
And then specific detaching function will be called.

v5:
- Change function names like below.
  rte_eal_dev_find_and_invoke() to rte_eal_vdev_find_and_invoke().
  rte_eal_dev_invoke() to rte_eal_vdev_invoke().
- Add code to handle a return value of rte_eal_devargs_remove().
- Fix pci address format in rte_eal_dev_detach().
v4:
- Fix comment.
- Add error checking.
- Fix indent of 'if' statement.
- Change function name.

Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp>
---
 lib/librte_eal/common/eal_common_dev.c  | 274 ++++++++++++++++++++++++++++++++
 lib/librte_eal/common/eal_private.h     |  11 ++
 lib/librte_eal/common/include/rte_dev.h |  33 ++++
 lib/librte_eal/linuxapp/eal/Makefile    |   1 +
 lib/librte_eal/linuxapp/eal/eal_pci.c   |   6 +-
 5 files changed, 322 insertions(+), 3 deletions(-)

diff --git a/lib/librte_eal/common/eal_common_dev.c b/lib/librte_eal/common/eal_common_dev.c
index eae5656..e3a3f54 100644
--- a/lib/librte_eal/common/eal_common_dev.c
+++ b/lib/librte_eal/common/eal_common_dev.c
@@ -32,10 +32,13 @@
  *   OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
  */
 
+#include <stdio.h>
+#include <limits.h>
 #include <string.h>
 #include <inttypes.h>
 #include <sys/queue.h>
 
+#include <rte_ethdev.h>
 #include <rte_dev.h>
 #include <rte_devargs.h>
 #include <rte_debug.h>
@@ -107,3 +110,274 @@ rte_eal_dev_init(void)
 	}
 	return 0;
 }
+
+/* So far, DPDK hotplug function only supports linux */
+#ifdef ENABLE_HOTPLUG
+static void
+rte_eal_vdev_invoke(struct rte_driver *driver,
+		struct rte_devargs *devargs, enum rte_eal_invoke_type type)
+{
+	if ((driver == NULL) || (devargs == NULL))
+		return;
+
+	switch (type) {
+	case RTE_EAL_INVOKE_TYPE_PROBE:
+		driver->init(devargs->virtual.drv_name, devargs->args);
+		break;
+	case RTE_EAL_INVOKE_TYPE_CLOSE:
+		driver->uninit(devargs->virtual.drv_name, devargs->args);
+		break;
+	default:
+		break;
+	}
+}
+
+static int
+rte_eal_vdev_find_and_invoke(const char *name, int type)
+{
+	struct rte_devargs *devargs;
+	struct rte_driver *driver;
+
+	if (name == NULL)
+		return -EINVAL;
+
+	/* call the init function for each virtual device */
+	TAILQ_FOREACH(devargs, &devargs_list, next) {
+
+		if (devargs->type != RTE_DEVTYPE_VIRTUAL)
+			continue;
+
+		if (strncmp(name, devargs->virtual.drv_name, strlen(name)))
+			continue;
+
+		TAILQ_FOREACH(driver, &dev_driver_list, next) {
+			if (driver->type != PMD_VDEV)
+				continue;
+
+			/* search a driver prefix in virtual device name */
+			if (!strncmp(driver->name, devargs->virtual.drv_name,
+			    strlen(driver->name))) {
+				rte_eal_vdev_invoke(driver, devargs, type);
+				break;
+			}
+		}
+
+		if (driver == NULL) {
+			RTE_LOG(WARNING, EAL, "no driver found for %s\n",
+				  devargs->virtual.drv_name);
+		}
+		return 0;
+	}
+	return 1;
+}
+
+/* attach the new physical device, then store port_id of the device */
+static int
+rte_eal_dev_attach_pdev(struct rte_pci_addr *addr, uint8_t *port_id)
+{
+	uint8_t new_port_id;
+	struct rte_eth_dev devs[RTE_MAX_ETHPORTS];
+
+	if ((addr == NULL) || (port_id == NULL))
+		goto err;
+
+	/* save current port status */
+	rte_eth_dev_save(devs);
+	/* re-construct pci_device_list */
+	if (rte_eal_pci_scan())
+		goto err;
+	/* invoke probe func of the driver can handle the new device */
+	if (rte_eal_pci_probe_one(addr))
+		goto err;
+	/* get port_id enabled by above procedures */
+	if (rte_eth_dev_get_changed_port(devs, &new_port_id))
+		goto err;
+
+	*port_id = new_port_id;
+	return 0;
+err:
+	RTE_LOG(ERR, EAL, "Drver, cannot attach the device\n");
+	return -1;
+}
+
+/* detach the new physical device, then store pci_addr of the device */
+static int
+rte_eal_dev_detach_pdev(uint8_t port_id, struct rte_pci_addr *addr)
+{
+	struct rte_pci_addr freed_addr;
+	struct rte_pci_addr vp;
+
+	if (addr == NULL)
+		goto err;
+
+	/* check whether the driver supports detach feature, or not */
+	if (rte_eth_dev_check_detachable(port_id))
+		goto err;
+
+	/* get pci address by port id */
+	if (rte_eth_dev_get_addr_by_port(port_id, &freed_addr))
+		goto err;
+
+	/* Zerod pci addr means the port comes from virtual device */
+	vp.domain = vp.bus = vp.devid = vp.function = 0;
+	if (eal_compare_pci_addr(&vp, &freed_addr) == 0)
+		goto err;
+
+	/* invoke close func of the driver,
+	 * also remove the device from pci_device_list */
+	if (rte_eal_pci_close_one(&freed_addr))
+		goto err;
+
+	*addr = freed_addr;
+	return 0;
+err:
+	RTE_LOG(ERR, EAL, "Drver, cannot detach the device\n");
+	return -1;
+}
+
+static void
+get_vdev_name(char *vdevargs)
+{
+	char *sep;
+
+	if (vdevargs == NULL)
+		return;
+
+	/* set the first ',' to '\0' to split name and arguments */
+	sep = strchr(vdevargs, ',');
+	if (sep != NULL)
+		sep[0] = '\0';
+}
+
+/* attach the new virtual device, then store port_id of the device */
+static int
+rte_eal_dev_attach_vdev(const char *vdevargs, uint8_t *port_id)
+{
+	char *args;
+	uint8_t new_port_id;
+	struct rte_eth_dev devs[RTE_MAX_ETHPORTS];
+
+	if ((vdevargs == NULL) || (port_id == NULL))
+		goto err0;
+
+	args = strdup(vdevargs);
+	if (args == NULL)
+		goto err0;
+
+	/* save current port status */
+	rte_eth_dev_save(devs);
+	/* add the vdevargs to devargs_list */
+	if (rte_eal_devargs_add(RTE_DEVTYPE_VIRTUAL, args))
+		goto err1;
+	/* parse vdevargs, then retrieve device name */
+	get_vdev_name(args);
+	/* walk around dev_driver_list to find the driver of the device,
+	 * then invoke probe function o the driver */
+	if (rte_eal_vdev_find_and_invoke(args, RTE_EAL_INVOKE_TYPE_PROBE))
+		goto err2;
+	/* get port_id enabled by above procedures */
+	if (rte_eth_dev_get_changed_port(devs, &new_port_id))
+		goto err2;
+
+	free(args);
+	*port_id = new_port_id;
+	return 0;
+err2:
+	rte_eal_devargs_remove(RTE_DEVTYPE_VIRTUAL, args);
+err1:
+	free(args);
+err0:
+	RTE_LOG(ERR, EAL, "Drver, cannot detach the device\n");
+	return -1;
+}
+
+/* detach the new virtual device, then store the name of the device */
+static int
+rte_eal_dev_detach_vdev(uint8_t port_id, char *vdevname)
+{
+	char name[RTE_ETH_NAME_MAX_LEN];
+
+	if (vdevname == NULL)
+		goto err;
+
+	/* check whether the driver supports detach feature, or not */
+	if (rte_eth_dev_check_detachable(port_id))
+		goto err;
+
+	/* get device name by port id */
+	if (rte_eth_dev_get_name_by_port(port_id, name))
+		goto err;
+	/* walk around dev_driver_list to find the driver of the device,
+	 * then invoke close function o the driver */
+	if (rte_eal_vdev_find_and_invoke(name, RTE_EAL_INVOKE_TYPE_CLOSE))
+		goto err;
+	/* remove the vdevname from devargs_list */
+	if (rte_eal_devargs_remove(RTE_DEVTYPE_VIRTUAL, name))
+		goto err;
+
+	strncpy(vdevname, name, sizeof(name));
+	return 0;
+err:
+	RTE_LOG(ERR, EAL, "Drver, cannot detach the device\n");
+	return -1;
+}
+
+/* attach the new device, then store port_id of the device */
+int
+rte_eal_dev_attach(const char *devargs, uint8_t *port_id)
+{
+	struct rte_pci_addr addr;
+
+	if ((devargs == NULL) || (port_id == NULL))
+		return -EINVAL;
+
+	if (eal_parse_pci_DomBDF(devargs, &addr) == 0)
+		return rte_eal_dev_attach_pdev(&addr, port_id);
+	else
+		return rte_eal_dev_attach_vdev(devargs, port_id);
+}
+
+/* detach the device, then store the name of the device */
+int
+rte_eal_dev_detach(uint8_t port_id, char *name)
+{
+	struct rte_pci_addr addr;
+	int ret;
+
+	if (name == NULL)
+		return -EINVAL;
+
+	if (rte_eth_dev_get_device_type(port_id) == RTE_ETH_DEV_PHYSICAL) {
+		ret = rte_eth_dev_get_addr_by_port(port_id, &addr);
+		if (ret < 0)
+			return ret;
+
+		ret = rte_eal_dev_detach_pdev(port_id, &addr);
+		if (ret == 0)
+			snprintf(name, RTE_ETH_NAME_MAX_LEN,
+				"%04x:%02x:%02x.%d",
+				addr.domain, addr.bus,
+				addr.devid, addr.function);
+
+		return ret;
+	} else
+		return rte_eal_dev_detach_vdev(port_id, name);
+}
+#else /* ENABLE_HOTPLUG */
+int
+rte_eal_dev_attach(const char *devargs __rte_unused,
+			uint8_t *port_id __rte_unused)
+{
+	RTE_LOG(ERR, EAL, "Hotplug support isn't enabled\n");
+	return -1;
+}
+
+/* detach the device, then store the name of the device */
+int
+rte_eal_dev_detach(uint8_t port_id __rte_unused,
+			char *name __rte_unused)
+{
+	RTE_LOG(ERR, EAL, "Hotplug support isn't enabled\n");
+	return -1;
+}
+#endif /* ENABLE_HOTPLUG */
diff --git a/lib/librte_eal/common/eal_private.h b/lib/librte_eal/common/eal_private.h
index 1a362ab..8168a7a 100644
--- a/lib/librte_eal/common/eal_private.h
+++ b/lib/librte_eal/common/eal_private.h
@@ -163,6 +163,17 @@ enum rte_eal_invoke_type {
 };
 
 /**
+ * Scan the content of the PCI bus, and the devices in the devices
+ * list
+ *
+ * This function is private to EAL.
+ *
+ * @return
+ *  0 on success, negative on error
+ */
+int rte_eal_pci_scan(void);
+
+/**
  * Mmap memory for single PCI device
  *
  * This function is private to EAL.
diff --git a/lib/librte_eal/common/include/rte_dev.h b/lib/librte_eal/common/include/rte_dev.h
index f7e3a10..e63dd1c 100644
--- a/lib/librte_eal/common/include/rte_dev.h
+++ b/lib/librte_eal/common/include/rte_dev.h
@@ -47,6 +47,7 @@ extern "C" {
 #endif
 
 #include <sys/queue.h>
+#include <rte_pci.h>
 
 /** Double linked list of device drivers. */
 TAILQ_HEAD(rte_driver_list, rte_driver);
@@ -57,6 +58,11 @@ TAILQ_HEAD(rte_driver_list, rte_driver);
 typedef int (rte_dev_init_t)(const char *name, const char *args);
 
 /**
+ * Uninitilization function called for each device driver once.
+ */
+typedef int (rte_dev_uninit_t)(const char *name, const char *args);
+
+/**
  * Driver type enumeration
  */
 enum pmd_type {
@@ -72,6 +78,7 @@ struct rte_driver {
 	enum pmd_type type;		   /**< PMD Driver type */
 	const char *name;                   /**< Driver name. */
 	rte_dev_init_t *init;              /**< Device init. function. */
+	rte_dev_uninit_t *uninit;          /**< Device uninit. function. */
 };
 
 /**
@@ -93,6 +100,32 @@ void rte_eal_driver_register(struct rte_driver *driver);
 void rte_eal_driver_unregister(struct rte_driver *driver);
 
 /**
+ * Attach a new device.
+ *
+ * @param devargs
+ *   A pointer to a strings array describing the new device
+ *   to be attached. The strings should be a pci address like
+ *   '0000:01:00.0' or virtual device name like 'eth_pcap0'.
+ * @param port_id
+ *  A pointer to a port identifier actually attached.
+ * @return
+ *  0 on success and port_id is filled, negative on error
+ */
+int rte_eal_dev_attach(const char *devargs, uint8_t *port_id);
+
+/**
+ * Detach a device.
+ *
+ * @param port_id
+ *   The port identifier of the device to detach.
+ * @param addr
+ *  A pointer to a device name actually detached.
+ * @return
+ *  0 on success and devname is filled, negative on error
+ */
+int rte_eal_dev_detach(uint8_t port_id, char *devname);
+
+/**
  * Initalize all the registered drivers in this process
  */
 int rte_eal_dev_init(void);
diff --git a/lib/librte_eal/linuxapp/eal/Makefile b/lib/librte_eal/linuxapp/eal/Makefile
index 72ecf3a..0ec83b5 100644
--- a/lib/librte_eal/linuxapp/eal/Makefile
+++ b/lib/librte_eal/linuxapp/eal/Makefile
@@ -41,6 +41,7 @@ CFLAGS += -I$(RTE_SDK)/lib/librte_eal/common/include
 CFLAGS += -I$(RTE_SDK)/lib/librte_ring
 CFLAGS += -I$(RTE_SDK)/lib/librte_mempool
 CFLAGS += -I$(RTE_SDK)/lib/librte_malloc
+CFLAGS += -I$(RTE_SDK)/lib/librte_mbuf
 CFLAGS += -I$(RTE_SDK)/lib/librte_ether
 CFLAGS += -I$(RTE_SDK)/lib/librte_ivshmem
 CFLAGS += -I$(RTE_SDK)/lib/librte_pmd_ring
diff --git a/lib/librte_eal/linuxapp/eal/eal_pci.c b/lib/librte_eal/linuxapp/eal/eal_pci.c
index 831422e..1f43688 100644
--- a/lib/librte_eal/linuxapp/eal/eal_pci.c
+++ b/lib/librte_eal/linuxapp/eal/eal_pci.c
@@ -431,8 +431,8 @@ error:
  * Scan the content of the PCI bus, and the devices in the devices
  * list
  */
-static int
-pci_scan(void)
+int
+rte_eal_pci_scan(void)
 {
 	struct dirent *e;
 	DIR *dir;
@@ -764,7 +764,7 @@ rte_eal_pci_init(void)
 	if (internal_config.no_pci)
 		return 0;
 
-	if (pci_scan() < 0) {
+	if (rte_eal_pci_scan() < 0) {
 		RTE_LOG(ERR, EAL, "%s(): Cannot scan PCI bus\n", __func__);
 		return -1;
 	}
-- 
1.9.1

^ permalink raw reply	[flat|nested] 362+ messages in thread

* [dpdk-dev] [PATCH v6 13/13] eal: Enable port hotplug framework in Linux
  2015-02-01  4:01     ` [dpdk-dev] [PATCH v6 00/13] Port Hotplug Framework Tetsuya Mukawa
                         ` (11 preceding siblings ...)
  2015-02-01  4:01       ` [dpdk-dev] [PATCH v6 12/13] eal/pci: Add rte_eal_dev_attach/detach() functions Tetsuya Mukawa
@ 2015-02-01  4:02       ` Tetsuya Mukawa
  2015-02-01  4:02       ` [dpdk-dev] [PATCH v6] librte_pmd_pcap: Add port hotplug support Tetsuya Mukawa
                         ` (3 subsequent siblings)
  16 siblings, 0 replies; 362+ messages in thread
From: Tetsuya Mukawa @ 2015-02-01  4:02 UTC (permalink / raw)
  To: dev

The patch enables CONFIG_RTE_LIBRTE_EAL_HOTPLUG in Linux configuration.

Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp>
---
 config/common_linuxapp | 5 +++++
 1 file changed, 5 insertions(+)

diff --git a/config/common_linuxapp b/config/common_linuxapp
index 2f9643b..27d05be 100644
--- a/config/common_linuxapp
+++ b/config/common_linuxapp
@@ -114,6 +114,11 @@ CONFIG_RTE_PCI_MAX_READ_REQUEST_SIZE=0
 CONFIG_RTE_LIBRTE_EAL_LINUXAPP=y
 
 #
+# Compile Environment Abstraction Layer to support hotplug
+#
+CONFIG_RTE_LIBRTE_EAL_HOTPLUG=y
+
+#
 # Compile Environment Abstraction Layer to support Vmware TSC map
 #
 CONFIG_RTE_LIBRTE_EAL_VMWARE_TSC_MAP_SUPPORT=y
-- 
1.9.1

^ permalink raw reply	[flat|nested] 362+ messages in thread

* [dpdk-dev] [PATCH v6] librte_pmd_pcap: Add port hotplug support
  2015-02-01  4:01     ` [dpdk-dev] [PATCH v6 00/13] Port Hotplug Framework Tetsuya Mukawa
                         ` (12 preceding siblings ...)
  2015-02-01  4:02       ` [dpdk-dev] [PATCH v6 13/13] eal: Enable port hotplug framework in Linux Tetsuya Mukawa
@ 2015-02-01  4:02       ` Tetsuya Mukawa
  2015-02-01  4:02       ` [dpdk-dev] [PATCH v6] testpmd: " Tetsuya Mukawa
                         ` (2 subsequent siblings)
  16 siblings, 0 replies; 362+ messages in thread
From: Tetsuya Mukawa @ 2015-02-01  4:02 UTC (permalink / raw)
  To: dev

This patch adds finalization code to free resources allocated by the
PMD.

v6:
 - Fix a paramter of rte_eth_dev_free().
v4:
 - Change function name.

Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp>
---
 lib/librte_pmd_pcap/rte_eth_pcap.c | 40 ++++++++++++++++++++++++++++++++++++++
 1 file changed, 40 insertions(+)

diff --git a/lib/librte_pmd_pcap/rte_eth_pcap.c b/lib/librte_pmd_pcap/rte_eth_pcap.c
index af7fae8..5f88efd 100644
--- a/lib/librte_pmd_pcap/rte_eth_pcap.c
+++ b/lib/librte_pmd_pcap/rte_eth_pcap.c
@@ -498,6 +498,13 @@ static struct eth_dev_ops ops = {
 		.stats_reset = eth_stats_reset,
 };
 
+static struct eth_driver rte_pcap_pmd = {
+	.pci_drv = {
+		.name = "rte_pcap_pmd",
+		.drv_flags = RTE_PCI_DRV_DETACHABLE,
+	},
+};
+
 /*
  * Function handler that opens the pcap file for reading a stores a
  * reference of it for use it later on.
@@ -713,6 +720,10 @@ rte_pmd_init_internals(const char *name, const unsigned nb_rx_queues,
 	if (*eth_dev == NULL)
 		goto error;
 
+	/* check length of device name */
+	if ((strlen((*eth_dev)->data->name) + 1) > sizeof(data->name))
+		goto error;
+
 	/* now put it all together
 	 * - store queue data in internals,
 	 * - store numa_node info in pci_driver
@@ -739,10 +750,13 @@ rte_pmd_init_internals(const char *name, const unsigned nb_rx_queues,
 	data->nb_tx_queues = (uint16_t)nb_tx_queues;
 	data->dev_link = pmd_link;
 	data->mac_addrs = &eth_addr;
+	strncpy(data->name,
+		(*eth_dev)->data->name, strlen((*eth_dev)->data->name));
 
 	(*eth_dev)->data = data;
 	(*eth_dev)->dev_ops = &ops;
 	(*eth_dev)->pci_dev = pci_dev;
+	(*eth_dev)->driver = &rte_pcap_pmd;
 
 	return 0;
 
@@ -927,10 +941,36 @@ rte_pmd_pcap_devinit(const char *name, const char *params)
 
 }
 
+static int
+rte_pmd_pcap_devuninit(const char *name, const char *params __rte_unused)
+{
+	struct rte_eth_dev *eth_dev = NULL;
+
+	RTE_LOG(INFO, PMD, "Closing pcap ethdev on numa socket %u\n",
+			rte_socket_id());
+
+	if (name == NULL)
+		return -1;
+
+	/* reserve an ethdev entry */
+	eth_dev = rte_eth_dev_allocated(name);
+	if (eth_dev == NULL)
+		return -1;
+
+	rte_free(eth_dev->data->dev_private);
+	rte_free(eth_dev->data);
+	rte_free(eth_dev->pci_dev);
+
+	rte_eth_dev_free(eth_dev);
+
+	return 0;
+}
+
 static struct rte_driver pmd_pcap_drv = {
 	.name = "eth_pcap",
 	.type = PMD_VDEV,
 	.init = rte_pmd_pcap_devinit,
+	.uninit = rte_pmd_pcap_devuninit,
 };
 
 PMD_REGISTER_DRIVER(pmd_pcap_drv);
-- 
1.9.1

^ permalink raw reply	[flat|nested] 362+ messages in thread

* [dpdk-dev] [PATCH v6] testpmd: Add port hotplug support
  2015-02-01  4:01     ` [dpdk-dev] [PATCH v6 00/13] Port Hotplug Framework Tetsuya Mukawa
                         ` (13 preceding siblings ...)
  2015-02-01  4:02       ` [dpdk-dev] [PATCH v6] librte_pmd_pcap: Add port hotplug support Tetsuya Mukawa
@ 2015-02-01  4:02       ` Tetsuya Mukawa
       [not found]         ` <8CEF83825BEC744B83065625E567D7C2049DCAD2@IRSMSX108.ger.corp.intel.com>
                           ` (2 more replies)
  2015-02-03 13:03       ` [dpdk-dev] [PATCH v6 00/13] Port Hotplug Framework Iremonger, Bernard
  2015-02-03 18:35       ` Iremonger, Bernard
  16 siblings, 3 replies; 362+ messages in thread
From: Tetsuya Mukawa @ 2015-02-01  4:02 UTC (permalink / raw)
  To: dev

The patch introduces following commands.
- port attach [ident]
- port detach [port_id]
 - attach: attaching a port
 - detach: detaching a port
 - ident: pci address of physical device.
          Or device name and paramerters of virtual device.
         (ex. 0000:02:00.0, eth_pcap0,iface=eth0)
 - port_id: port identifier

v5:
- Add testpmd documentation.
  (Thanks to Iremonger, Bernard)
v4:
 - Fix strings of command help.

Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp>
---
 app/test-pmd/cmdline.c                      | 133 +++++++++++++++----
 app/test-pmd/config.c                       | 116 +++++++++-------
 app/test-pmd/parameters.c                   |  22 ++-
 app/test-pmd/testpmd.c                      | 199 +++++++++++++++++++++-------
 app/test-pmd/testpmd.h                      |  18 ++-
 doc/guides/testpmd_app_ug/testpmd_funcs.rst |  57 ++++++++
 6 files changed, 415 insertions(+), 130 deletions(-)

diff --git a/app/test-pmd/cmdline.c b/app/test-pmd/cmdline.c
index 4beb404..2f813d8 100644
--- a/app/test-pmd/cmdline.c
+++ b/app/test-pmd/cmdline.c
@@ -572,6 +572,12 @@ static void cmd_help_long_parsed(void *parsed_result,
 			"port close (port_id|all)\n"
 			"    Close all ports or port_id.\n\n"
 
+			"port attach (ident)\n"
+			"    Attach physical or virtual dev by pci address or virtual device name\n\n"
+
+			"port detach (port_id)\n"
+			"    Detach physical or virtual dev by port_id\n\n"
+
 			"port config (port_id|all)"
 			" speed (10|100|1000|10000|40000|auto)"
 			" duplex (half|full|auto)\n"
@@ -848,6 +854,89 @@ cmdline_parse_inst_t cmd_operate_specific_port = {
 	},
 };
 
+/* *** attach a specificied port *** */
+struct cmd_operate_attach_port_result {
+	cmdline_fixed_string_t port;
+	cmdline_fixed_string_t keyword;
+	cmdline_fixed_string_t identifier;
+};
+
+static void cmd_operate_attach_port_parsed(void *parsed_result,
+				__attribute__((unused)) struct cmdline *cl,
+				__attribute__((unused)) void *data)
+{
+	struct cmd_operate_attach_port_result *res = parsed_result;
+
+	if (!strcmp(res->keyword, "attach"))
+		attach_port(res->identifier);
+	else
+		printf("Unknown parameter\n");
+}
+
+cmdline_parse_token_string_t cmd_operate_attach_port_port =
+	TOKEN_STRING_INITIALIZER(struct cmd_operate_attach_port_result,
+			port, "port");
+cmdline_parse_token_string_t cmd_operate_attach_port_keyword =
+	TOKEN_STRING_INITIALIZER(struct cmd_operate_attach_port_result,
+			keyword, "attach");
+cmdline_parse_token_string_t cmd_operate_attach_port_identifier =
+	TOKEN_STRING_INITIALIZER(struct cmd_operate_attach_port_result,
+			identifier, NULL);
+
+cmdline_parse_inst_t cmd_operate_attach_port = {
+	.f = cmd_operate_attach_port_parsed,
+	.data = NULL,
+	.help_str = "port attach identifier, "
+		"identifier: pci address or virtual dev name",
+	.tokens = {
+		(void *)&cmd_operate_attach_port_port,
+		(void *)&cmd_operate_attach_port_keyword,
+		(void *)&cmd_operate_attach_port_identifier,
+		NULL,
+	},
+};
+
+/* *** detach a specificied port *** */
+struct cmd_operate_detach_port_result {
+	cmdline_fixed_string_t port;
+	cmdline_fixed_string_t keyword;
+	uint8_t port_id;
+};
+
+static void cmd_operate_detach_port_parsed(void *parsed_result,
+				__attribute__((unused)) struct cmdline *cl,
+				__attribute__((unused)) void *data)
+{
+	struct cmd_operate_detach_port_result *res = parsed_result;
+
+	if (!strcmp(res->keyword, "detach"))
+		detach_port(res->port_id);
+	else
+		printf("Unknown parameter\n");
+}
+
+cmdline_parse_token_string_t cmd_operate_detach_port_port =
+	TOKEN_STRING_INITIALIZER(struct cmd_operate_detach_port_result,
+			port, "port");
+cmdline_parse_token_string_t cmd_operate_detach_port_keyword =
+	TOKEN_STRING_INITIALIZER(struct cmd_operate_detach_port_result,
+			keyword, "detach");
+cmdline_parse_token_num_t cmd_operate_detach_port_port_id =
+	TOKEN_NUM_INITIALIZER(struct cmd_operate_detach_port_result,
+			port_id, UINT8);
+
+cmdline_parse_inst_t cmd_operate_detach_port = {
+	.f = cmd_operate_detach_port_parsed,
+	.data = NULL,
+	.help_str = "port detach port_id",
+	.tokens = {
+		(void *)&cmd_operate_detach_port_port,
+		(void *)&cmd_operate_detach_port_keyword,
+		(void *)&cmd_operate_detach_port_port_id,
+		NULL,
+	},
+};
+
 /* *** configure speed for all ports *** */
 struct cmd_config_speed_all {
 	cmdline_fixed_string_t port;
@@ -902,7 +991,7 @@ cmd_config_speed_all_parsed(void *parsed_result,
 		return;
 	}
 
-	for (pid = 0; pid < nb_ports; pid++) {
+	FOREACH_PORT(pid, ports) {
 		ports[pid].dev_conf.link_speed = link_speed;
 		ports[pid].dev_conf.link_duplex = link_duplex;
 	}
@@ -970,10 +1059,8 @@ cmd_config_speed_specific_parsed(void *parsed_result,
 		return;
 	}
 
-	if (res->id >= nb_ports) {
-		printf("Port id %d must be less than %d\n", res->id, nb_ports);
+	if (port_id_is_invalid(res->id, ENABLED_WARN))
 		return;
-	}
 
 	if (!strcmp(res->value1, "10"))
 		link_speed = ETH_LINK_SPEED_10;
@@ -1533,7 +1620,7 @@ cmd_config_rxtx_queue_parsed(void *parsed_result,
 		return;
 	}
 
-	if (port_id_is_invalid(res->portid))
+	if (port_id_is_invalid(res->portid, ENABLED_WARN))
 		return;
 
 	if (port_is_started(res->portid) != 1) {
@@ -2876,7 +2963,7 @@ cmd_tx_cksum_parsed(void *parsed_result,
 	uint16_t ol_flags, mask = 0;
 	struct rte_eth_dev_info dev_info;
 
-	if (port_id_is_invalid(res->port_id)) {
+	if (port_id_is_invalid(res->port_id, ENABLED_WARN)) {
 		printf("invalid port %d\n", res->port_id);
 		return;
 	}
@@ -3003,7 +3090,7 @@ cmd_tso_set_parsed(void *parsed_result,
 	struct cmd_tso_set_result *res = parsed_result;
 	struct rte_eth_dev_info dev_info;
 
-	if (port_id_is_invalid(res->port_id))
+	if (port_id_is_invalid(res->port_id, ENABLED_WARN))
 		return;
 
 	if (!strcmp(res->mode, "set"))
@@ -3979,10 +4066,8 @@ static void cmd_set_bond_mac_addr_parsed(void *parsed_result,
 	struct cmd_set_bond_mac_addr_result *res = parsed_result;
 	int ret;
 
-	if (res->port_num >= nb_ports) {
-		printf("Port id %d must be less than %d\n", res->port_num, nb_ports);
+	if (port_id_is_invalid(res->port_num, ENABLED_WARN))
 		return;
-	}
 
 	ret = rte_eth_bond_mac_address_set(res->port_num, &res->address);
 
@@ -4219,7 +4304,7 @@ static void cmd_set_promisc_mode_parsed(void *parsed_result,
 
 	/* all ports */
 	if (allports) {
-		for (i = 0; i < nb_ports; i++) {
+		FOREACH_PORT(i, ports) {
 			if (enable)
 				rte_eth_promiscuous_enable(i);
 			else
@@ -4299,7 +4384,7 @@ static void cmd_set_allmulti_mode_parsed(void *parsed_result,
 
 	/* all ports */
 	if (allports) {
-		for (i = 0; i < nb_ports; i++) {
+		FOREACH_PORT(i, ports) {
 			if (enable)
 				rte_eth_allmulticast_enable(i);
 			else
@@ -5484,25 +5569,25 @@ static void cmd_showportall_parsed(void *parsed_result,
 	struct cmd_showportall_result *res = parsed_result;
 	if (!strcmp(res->show, "clear")) {
 		if (!strcmp(res->what, "stats"))
-			for (i = 0; i < nb_ports; i++)
+			FOREACH_PORT(i, ports)
 				nic_stats_clear(i);
 		else if (!strcmp(res->what, "xstats"))
-			for (i = 0; i < nb_ports; i++)
+			FOREACH_PORT(i, ports)
 				nic_xstats_clear(i);
 	} else if (!strcmp(res->what, "info"))
-		for (i = 0; i < nb_ports; i++)
+		FOREACH_PORT(i, ports)
 			port_infos_display(i);
 	else if (!strcmp(res->what, "stats"))
-		for (i = 0; i < nb_ports; i++)
+		FOREACH_PORT(i, ports)
 			nic_stats_display(i);
 	else if (!strcmp(res->what, "xstats"))
-		for (i = 0; i < nb_ports; i++)
+		FOREACH_PORT(i, ports)
 			nic_xstats_display(i);
 	else if (!strcmp(res->what, "fdir"))
-		for (i = 0; i < nb_ports; i++)
+		FOREACH_PORT(i, ports)
 			fdir_get_infos(i);
 	else if (!strcmp(res->what, "stat_qmap"))
-		for (i = 0; i < nb_ports; i++)
+		FOREACH_PORT(i, ports)
 			nic_stats_mapping_display(i);
 }
 
@@ -8756,6 +8841,8 @@ cmdline_parse_ctx_t main_ctx[] = {
 	(cmdline_parse_inst_t *)&cmd_set_qmap,
 	(cmdline_parse_inst_t *)&cmd_operate_port,
 	(cmdline_parse_inst_t *)&cmd_operate_specific_port,
+	(cmdline_parse_inst_t *)&cmd_operate_attach_port,
+	(cmdline_parse_inst_t *)&cmd_operate_detach_port,
 	(cmdline_parse_inst_t *)&cmd_config_speed_all,
 	(cmdline_parse_inst_t *)&cmd_config_speed_specific,
 	(cmdline_parse_inst_t *)&cmd_config_rx_tx,
@@ -8830,7 +8917,7 @@ prompt(void)
 static void
 cmd_reconfig_device_queue(portid_t id, uint8_t dev, uint8_t queue)
 {
-	if (id < nb_ports) {
+	if (!port_id_is_invalid(id, DISABLED_WARN)) {
 		/* check if need_reconfig has been set to 1 */
 		if (ports[id].need_reconfig == 0)
 			ports[id].need_reconfig = dev;
@@ -8840,7 +8927,7 @@ cmd_reconfig_device_queue(portid_t id, uint8_t dev, uint8_t queue)
 	} else {
 		portid_t pid;
 
-		for (pid = 0; pid < nb_ports; pid++) {
+		FOREACH_PORT(pid, ports) {
 			/* check if need_reconfig has been set to 1 */
 			if (ports[pid].need_reconfig == 0)
 				ports[pid].need_reconfig = dev;
@@ -8858,10 +8945,8 @@ bypass_is_supported(portid_t port_id)
 	struct rte_port   *port;
 	struct rte_pci_id *pci_id;
 
-	if (port_id >= nb_ports) {
-		printf("\tPort id must be less than %d.\n", nb_ports);
+	if (port_id_is_invalid(port_id, ENABLED_WARN))
 		return 0;
-	}
 
 	/* Get the device id. */
 	port    = &ports[port_id];
diff --git a/app/test-pmd/config.c b/app/test-pmd/config.c
index c40f819..32d8f9a 100644
--- a/app/test-pmd/config.c
+++ b/app/test-pmd/config.c
@@ -124,11 +124,15 @@ nic_stats_display(portid_t port_id)
 	struct rte_eth_stats stats;
 	struct rte_port *port = &ports[port_id];
 	uint8_t i;
+	portid_t pid;
 
 	static const char *nic_stats_border = "########################";
 
-	if (port_id >= nb_ports) {
-		printf("Invalid port, range is [0, %d]\n", nb_ports - 1);
+	if (port_id_is_invalid(port_id, ENABLED_WARN)) {
+		printf("Valid port range is [0");
+		FOREACH_PORT(pid, ports)
+			printf(", %d", pid);
+		printf("]\n");
 		return;
 	}
 	rte_eth_stats_get(port_id, &stats);
@@ -201,8 +205,13 @@ nic_stats_display(portid_t port_id)
 void
 nic_stats_clear(portid_t port_id)
 {
-	if (port_id >= nb_ports) {
-		printf("Invalid port, range is [0, %d]\n", nb_ports - 1);
+	portid_t pid;
+
+	if (port_id_is_invalid(port_id, ENABLED_WARN)) {
+		printf("Valid port range is [0");
+		FOREACH_PORT(pid, ports)
+			printf(", %d", pid);
+		printf("]\n");
 		return;
 	}
 	rte_eth_stats_reset(port_id);
@@ -249,11 +258,15 @@ nic_stats_mapping_display(portid_t port_id)
 {
 	struct rte_port *port = &ports[port_id];
 	uint16_t i;
+	portid_t pid;
 
 	static const char *nic_stats_mapping_border = "########################";
 
-	if (port_id >= nb_ports) {
-		printf("Invalid port, range is [0, %d]\n", nb_ports - 1);
+	if (port_id_is_invalid(port_id, ENABLED_WARN)) {
+		printf("Valid port range is [0");
+		FOREACH_PORT(pid, ports)
+			printf(", %d", pid);
+		printf("]\n");
 		return;
 	}
 
@@ -302,9 +315,13 @@ port_infos_display(portid_t port_id)
 	int vlan_offload;
 	struct rte_mempool * mp;
 	static const char *info_border = "*********************";
+	portid_t pid;
 
-	if (port_id >= nb_ports) {
-		printf("Invalid port, range is [0, %d]\n", nb_ports - 1);
+	if (port_id_is_invalid(port_id, ENABLED_WARN)) {
+		printf("Valid port range is [0");
+		FOREACH_PORT(pid, ports)
+			printf(", %d", pid);
+		printf("]\n");
 		return;
 	}
 	port = &ports[port_id];
@@ -362,11 +379,14 @@ port_infos_display(portid_t port_id)
 }
 
 int
-port_id_is_invalid(portid_t port_id)
+port_id_is_invalid(portid_t port_id, enum print_warning warning)
 {
-	if (port_id < nb_ports)
+	if (ports[port_id].enabled)
 		return 0;
-	printf("Invalid port %d (must be < nb_ports=%d)\n", port_id, nb_ports);
+
+	if (warning == ENABLED_WARN)
+		printf("Invalid port %d\n", port_id);
+
 	return 1;
 }
 
@@ -425,7 +445,7 @@ port_reg_bit_display(portid_t port_id, uint32_t reg_off, uint8_t bit_x)
 	uint32_t reg_v;
 
 
-	if (port_id_is_invalid(port_id))
+	if (port_id_is_invalid(port_id, ENABLED_WARN))
 		return;
 	if (port_reg_off_is_invalid(port_id, reg_off))
 		return;
@@ -444,7 +464,7 @@ port_reg_bit_field_display(portid_t port_id, uint32_t reg_off,
 	uint8_t  l_bit;
 	uint8_t  h_bit;
 
-	if (port_id_is_invalid(port_id))
+	if (port_id_is_invalid(port_id, ENABLED_WARN))
 		return;
 	if (port_reg_off_is_invalid(port_id, reg_off))
 		return;
@@ -471,7 +491,7 @@ port_reg_display(portid_t port_id, uint32_t reg_off)
 {
 	uint32_t reg_v;
 
-	if (port_id_is_invalid(port_id))
+	if (port_id_is_invalid(port_id, ENABLED_WARN))
 		return;
 	if (port_reg_off_is_invalid(port_id, reg_off))
 		return;
@@ -485,7 +505,7 @@ port_reg_bit_set(portid_t port_id, uint32_t reg_off, uint8_t bit_pos,
 {
 	uint32_t reg_v;
 
-	if (port_id_is_invalid(port_id))
+	if (port_id_is_invalid(port_id, ENABLED_WARN))
 		return;
 	if (port_reg_off_is_invalid(port_id, reg_off))
 		return;
@@ -513,7 +533,7 @@ port_reg_bit_field_set(portid_t port_id, uint32_t reg_off,
 	uint8_t  l_bit;
 	uint8_t  h_bit;
 
-	if (port_id_is_invalid(port_id))
+	if (port_id_is_invalid(port_id, ENABLED_WARN))
 		return;
 	if (port_reg_off_is_invalid(port_id, reg_off))
 		return;
@@ -547,7 +567,7 @@ port_reg_bit_field_set(portid_t port_id, uint32_t reg_off,
 void
 port_reg_set(portid_t port_id, uint32_t reg_off, uint32_t reg_v)
 {
-	if (port_id_is_invalid(port_id))
+	if (port_id_is_invalid(port_id, ENABLED_WARN))
 		return;
 	if (port_reg_off_is_invalid(port_id, reg_off))
 		return;
@@ -560,7 +580,7 @@ port_mtu_set(portid_t port_id, uint16_t mtu)
 {
 	int diag;
 
-	if (port_id_is_invalid(port_id))
+	if (port_id_is_invalid(port_id, ENABLED_WARN))
 		return;
 	diag = rte_eth_dev_set_mtu(port_id, mtu);
 	if (diag == 0)
@@ -723,7 +743,7 @@ rx_ring_desc_display(portid_t port_id, queueid_t rxq_id, uint16_t rxd_id)
 {
 	const struct rte_memzone *rx_mz;
 
-	if (port_id_is_invalid(port_id))
+	if (port_id_is_invalid(port_id, ENABLED_WARN))
 		return;
 	if (rx_queue_id_is_invalid(rxq_id))
 		return;
@@ -740,7 +760,7 @@ tx_ring_desc_display(portid_t port_id, queueid_t txq_id, uint16_t txd_id)
 {
 	const struct rte_memzone *tx_mz;
 
-	if (port_id_is_invalid(port_id))
+	if (port_id_is_invalid(port_id, ENABLED_WARN))
 		return;
 	if (tx_queue_id_is_invalid(txq_id))
 		return;
@@ -796,7 +816,7 @@ port_rss_reta_info(portid_t port_id,
 	uint16_t i, idx, shift;
 	int ret;
 
-	if (port_id_is_invalid(port_id))
+	if (port_id_is_invalid(port_id, ENABLED_WARN))
 		return;
 
 	ret = rte_eth_dev_rss_reta_query(port_id, reta_conf, nb_entries);
@@ -828,7 +848,7 @@ port_rss_hash_conf_show(portid_t port_id, int show_rss_key)
 	uint8_t i;
 	int diag;
 
-	if (port_id_is_invalid(port_id))
+	if (port_id_is_invalid(port_id, ENABLED_WARN))
 		return;
 	/* Get RSS hash key if asked to display it */
 	rss_conf.rss_key = (show_rss_key) ? rss_key : NULL;
@@ -1406,12 +1426,8 @@ set_fwd_ports_list(unsigned int *portlist, unsigned int nb_pt)
  again:
 	for (i = 0; i < nb_pt; i++) {
 		port_id = (portid_t) portlist[i];
-		if (port_id >= nb_ports) {
-			printf("Invalid port id %u >= %u\n",
-			       (unsigned int) port_id,
-			       (unsigned int) nb_ports);
+		if (port_id_is_invalid(port_id, ENABLED_WARN))
 			return;
-		}
 		if (record_now)
 			fwd_ports_ids[i] = port_id;
 	}
@@ -1569,7 +1585,7 @@ vlan_extend_set(portid_t port_id, int on)
 	int diag;
 	int vlan_offload;
 
-	if (port_id_is_invalid(port_id))
+	if (port_id_is_invalid(port_id, ENABLED_WARN))
 		return;
 
 	vlan_offload = rte_eth_dev_get_vlan_offload(port_id);
@@ -1591,7 +1607,7 @@ rx_vlan_strip_set(portid_t port_id, int on)
 	int diag;
 	int vlan_offload;
 
-	if (port_id_is_invalid(port_id))
+	if (port_id_is_invalid(port_id, ENABLED_WARN))
 		return;
 
 	vlan_offload = rte_eth_dev_get_vlan_offload(port_id);
@@ -1612,7 +1628,7 @@ rx_vlan_strip_set_on_queue(portid_t port_id, uint16_t queue_id, int on)
 {
 	int diag;
 
-	if (port_id_is_invalid(port_id))
+	if (port_id_is_invalid(port_id, ENABLED_WARN))
 		return;
 
 	diag = rte_eth_dev_set_vlan_strip_on_queue(port_id, queue_id, on);
@@ -1627,7 +1643,7 @@ rx_vlan_filter_set(portid_t port_id, int on)
 	int diag;
 	int vlan_offload;
 
-	if (port_id_is_invalid(port_id))
+	if (port_id_is_invalid(port_id, ENABLED_WARN))
 		return;
 
 	vlan_offload = rte_eth_dev_get_vlan_offload(port_id);
@@ -1648,7 +1664,7 @@ rx_vft_set(portid_t port_id, uint16_t vlan_id, int on)
 {
 	int diag;
 
-	if (port_id_is_invalid(port_id))
+	if (port_id_is_invalid(port_id, ENABLED_WARN))
 		return;
 	if (vlan_id_is_invalid(vlan_id))
 		return;
@@ -1665,7 +1681,7 @@ rx_vlan_all_filter_set(portid_t port_id, int on)
 {
 	uint16_t vlan_id;
 
-	if (port_id_is_invalid(port_id))
+	if (port_id_is_invalid(port_id, ENABLED_WARN))
 		return;
 	for (vlan_id = 0; vlan_id < 4096; vlan_id++)
 		rx_vft_set(port_id, vlan_id, on);
@@ -1675,7 +1691,7 @@ void
 vlan_tpid_set(portid_t port_id, uint16_t tp_id)
 {
 	int diag;
-	if (port_id_is_invalid(port_id))
+	if (port_id_is_invalid(port_id, ENABLED_WARN))
 		return;
 
 	diag = rte_eth_dev_set_vlan_ether_type(port_id, tp_id);
@@ -1690,7 +1706,7 @@ vlan_tpid_set(portid_t port_id, uint16_t tp_id)
 void
 tx_vlan_set(portid_t port_id, uint16_t vlan_id)
 {
-	if (port_id_is_invalid(port_id))
+	if (port_id_is_invalid(port_id, ENABLED_WARN))
 		return;
 	if (vlan_id_is_invalid(vlan_id))
 		return;
@@ -1701,7 +1717,7 @@ tx_vlan_set(portid_t port_id, uint16_t vlan_id)
 void
 tx_vlan_reset(portid_t port_id)
 {
-	if (port_id_is_invalid(port_id))
+	if (port_id_is_invalid(port_id, ENABLED_WARN))
 		return;
 	ports[port_id].tx_ol_flags &= ~TESTPMD_TX_OFFLOAD_INSERT_VLAN;
 }
@@ -1709,7 +1725,7 @@ tx_vlan_reset(portid_t port_id)
 void
 tx_vlan_pvid_set(portid_t port_id, uint16_t vlan_id, int on)
 {
-	if (port_id_is_invalid(port_id))
+	if (port_id_is_invalid(port_id, ENABLED_WARN))
 		return;
 
 	rte_eth_dev_set_vlan_pvid(port_id, vlan_id, on);
@@ -1721,7 +1737,7 @@ set_qmap(portid_t port_id, uint8_t is_rx, uint16_t queue_id, uint8_t map_value)
 	uint16_t i;
 	uint8_t existing_mapping_found = 0;
 
-	if (port_id_is_invalid(port_id))
+	if (port_id_is_invalid(port_id, ENABLED_WARN))
 		return;
 
 	if (is_rx ? (rx_queue_id_is_invalid(queue_id)) : (tx_queue_id_is_invalid(queue_id)))
@@ -1773,7 +1789,7 @@ fdir_add_signature_filter(portid_t port_id, uint8_t queue_id,
 {
 	int diag;
 
-	if (port_id_is_invalid(port_id))
+	if (port_id_is_invalid(port_id, ENABLED_WARN))
 		return;
 
 	diag = rte_eth_dev_fdir_add_signature_filter(port_id, fdir_filter,
@@ -1791,7 +1807,7 @@ fdir_update_signature_filter(portid_t port_id, uint8_t queue_id,
 {
 	int diag;
 
-	if (port_id_is_invalid(port_id))
+	if (port_id_is_invalid(port_id, ENABLED_WARN))
 		return;
 
 	diag = rte_eth_dev_fdir_update_signature_filter(port_id, fdir_filter,
@@ -1809,7 +1825,7 @@ fdir_remove_signature_filter(portid_t port_id,
 {
 	int diag;
 
-	if (port_id_is_invalid(port_id))
+	if (port_id_is_invalid(port_id, ENABLED_WARN))
 		return;
 
 	diag = rte_eth_dev_fdir_remove_signature_filter(port_id, fdir_filter);
@@ -1881,7 +1897,7 @@ fdir_get_infos(portid_t port_id)
 
 	static const char *fdir_stats_border = "########################";
 
-	if (port_id_is_invalid(port_id))
+	if (port_id_is_invalid(port_id, ENABLED_WARN))
 		return;
 	ret = rte_eth_dev_filter_supported(port_id, RTE_ETH_FILTER_FDIR);
 	if (ret < 0) {
@@ -1955,7 +1971,7 @@ fdir_add_perfect_filter(portid_t port_id, uint16_t soft_id, uint8_t queue_id,
 {
 	int diag;
 
-	if (port_id_is_invalid(port_id))
+	if (port_id_is_invalid(port_id, ENABLED_WARN))
 		return;
 
 	diag = rte_eth_dev_fdir_add_perfect_filter(port_id, fdir_filter,
@@ -1973,7 +1989,7 @@ fdir_update_perfect_filter(portid_t port_id, uint16_t soft_id, uint8_t queue_id,
 {
 	int diag;
 
-	if (port_id_is_invalid(port_id))
+	if (port_id_is_invalid(port_id, ENABLED_WARN))
 		return;
 
 	diag = rte_eth_dev_fdir_update_perfect_filter(port_id, fdir_filter,
@@ -1991,7 +2007,7 @@ fdir_remove_perfect_filter(portid_t port_id, uint16_t soft_id,
 {
 	int diag;
 
-	if (port_id_is_invalid(port_id))
+	if (port_id_is_invalid(port_id, ENABLED_WARN))
 		return;
 
 	diag = rte_eth_dev_fdir_remove_perfect_filter(port_id, fdir_filter,
@@ -2008,7 +2024,7 @@ fdir_set_masks(portid_t port_id, struct rte_fdir_masks *fdir_masks)
 {
 	int diag;
 
-	if (port_id_is_invalid(port_id))
+	if (port_id_is_invalid(port_id, ENABLED_WARN))
 		return;
 
 	diag = rte_eth_dev_fdir_set_masks(port_id, fdir_masks);
@@ -2085,7 +2101,7 @@ set_vf_traffic(portid_t port_id, uint8_t is_rx, uint16_t vf, uint8_t on)
 {
 	int diag;
 
-	if (port_id_is_invalid(port_id))
+	if (port_id_is_invalid(port_id, ENABLED_WARN))
 		return;
 	if (is_rx)
 		diag = rte_eth_dev_set_vf_rx(port_id,vf,on);
@@ -2107,7 +2123,7 @@ set_vf_rx_vlan(portid_t port_id, uint16_t vlan_id, uint64_t vf_mask, uint8_t on)
 {
 	int diag;
 
-	if (port_id_is_invalid(port_id))
+	if (port_id_is_invalid(port_id, ENABLED_WARN))
 		return;
 	if (vlan_id_is_invalid(vlan_id))
 		return;
@@ -2124,7 +2140,7 @@ set_queue_rate_limit(portid_t port_id, uint16_t queue_idx, uint16_t rate)
 	int diag;
 	struct rte_eth_link link;
 
-	if (port_id_is_invalid(port_id))
+	if (port_id_is_invalid(port_id, ENABLED_WARN))
 		return 1;
 	rte_eth_link_get_nowait(port_id, &link);
 	if (rate > link.link_speed) {
@@ -2149,7 +2165,7 @@ set_vf_rate_limit(portid_t port_id, uint16_t vf, uint16_t rate, uint64_t q_msk)
 	if (q_msk == 0)
 		return 0;
 
-	if (port_id_is_invalid(port_id))
+	if (port_id_is_invalid(port_id, ENABLED_WARN))
 		return 1;
 	rte_eth_link_get_nowait(port_id, &link);
 	if (rate > link.link_speed) {
diff --git a/app/test-pmd/parameters.c b/app/test-pmd/parameters.c
index adf3203..6f2af18 100644
--- a/app/test-pmd/parameters.c
+++ b/app/test-pmd/parameters.c
@@ -376,6 +376,7 @@ parse_portnuma_config(const char *q_arg)
 	};
 	unsigned long int_fld[_NUM_FLD];
 	char *str_fld[_NUM_FLD];
+	portid_t pid;
 
 	/* reset from value set at definition */
 	while ((p = strchr(p0,'(')) != NULL) {
@@ -397,8 +398,11 @@ parse_portnuma_config(const char *q_arg)
 				return -1;
 		}
 		port_id = (uint8_t)int_fld[FLD_PORT];
-		if (port_id >= nb_ports) {
-			printf("Invalid port, range is [0, %d]\n", nb_ports - 1);
+		if (port_id_is_invalid(port_id, ENABLED_WARN)) {
+			printf("Valid port range is [0");
+			FOREACH_PORT(pid, ports)
+				printf(", %d", pid);
+			printf("]\n");
 			return -1;
 		}
 		socket_id = (uint8_t)int_fld[FLD_SOCKET];
@@ -429,6 +433,7 @@ parse_ringnuma_config(const char *q_arg)
 	};
 	unsigned long int_fld[_NUM_FLD];
 	char *str_fld[_NUM_FLD];
+	portid_t pid;
 	#define RX_RING_ONLY 0x1
 	#define TX_RING_ONLY 0x2
 	#define RXTX_RING    0x3
@@ -453,8 +458,11 @@ parse_ringnuma_config(const char *q_arg)
 				return -1;
 		}
 		port_id = (uint8_t)int_fld[FLD_PORT];
-		if (port_id >= nb_ports) {
-			printf("Invalid port, range is [0, %d]\n", nb_ports - 1);
+		if (port_id_is_invalid(port_id, ENABLED_WARN)) {
+			printf("Valid port range is [0");
+			FOREACH_PORT(pid, ports)
+				printf(", %d", pid);
+			printf("]\n");
 			return -1;
 		}
 		socket_id = (uint8_t)int_fld[FLD_SOCKET];
@@ -626,12 +634,12 @@ launch_args_parse(int argc, char** argv)
 #endif
 			if (!strcmp(lgopts[opt_idx].name, "nb-ports")) {
 				n = atoi(optarg);
-				if (n > 0 && n <= nb_ports)
+				if (n > 0 &&
+				    !port_id_is_invalid(n, DISABLED_WARN))
 					nb_fwd_ports = (uint8_t) n;
 				else
 					rte_exit(EXIT_FAILURE,
-						 "nb-ports should be > 0 and <= %d\n",
-						 nb_ports);
+						 "Invalid port %d\n", n);
 			}
 			if (!strcmp(lgopts[opt_idx].name, "nb-cores")) {
 				n = atoi(optarg);
diff --git a/app/test-pmd/testpmd.c b/app/test-pmd/testpmd.c
index 773b8af..c18c1a9 100644
--- a/app/test-pmd/testpmd.c
+++ b/app/test-pmd/testpmd.c
@@ -71,6 +71,7 @@
 #include <rte_pci.h>
 #include <rte_ether.h>
 #include <rte_ethdev.h>
+#include <rte_dev.h>
 #include <rte_string_fns.h>
 #ifdef RTE_LIBRTE_PMD_XENVIRT
 #include <rte_eth_xenvirt.h>
@@ -315,7 +316,7 @@ uint16_t nb_rx_queue_stats_mappings = 0;
 
 /* Forward function declarations */
 static void map_port_queue_stats_mapping_registers(uint8_t pi, struct rte_port *port);
-static void check_all_ports_link_status(uint8_t port_num, uint32_t port_mask);
+static void check_all_ports_link_status(uint32_t port_mask);
 
 /*
  * Check if all the ports are started.
@@ -324,6 +325,20 @@ static void check_all_ports_link_status(uint8_t port_num, uint32_t port_mask);
 static int all_ports_started(void);
 
 /*
+ * Find next enabled port
+ */
+portid_t
+find_next_port(portid_t p, struct rte_port *ports, int size)
+{
+	if (ports == NULL)
+		rte_exit(-EINVAL, "failed to find a next port id\n");
+
+	while ((ports[p].enabled == 0) && (p < size))
+		p++;
+	return p;
+}
+
+/*
  * Setup default configuration.
  */
 static void
@@ -552,7 +567,8 @@ init_config(void)
 				+ RTE_TEST_TX_DESC_MAX + MAX_PKT_BURST;
 
 		if (!numa_support)
-			nb_mbuf_per_pool = (nb_mbuf_per_pool * nb_ports);
+			nb_mbuf_per_pool =
+				(nb_mbuf_per_pool * RTE_MAX_ETHPORTS);
 	}
 
 	if (!numa_support) {
@@ -565,14 +581,19 @@ init_config(void)
 
 	/* Configuration of Ethernet ports. */
 	ports = rte_zmalloc("testpmd: ports",
-			    sizeof(struct rte_port) * nb_ports,
+			    sizeof(struct rte_port) * RTE_MAX_ETHPORTS,
 			    RTE_CACHE_LINE_SIZE);
 	if (ports == NULL) {
-		rte_exit(EXIT_FAILURE, "rte_zmalloc(%d struct rte_port) "
-							"failed\n", nb_ports);
+		rte_exit(EXIT_FAILURE,
+				"rte_zmalloc(%d struct rte_port) failed\n",
+				RTE_MAX_ETHPORTS);
 	}
 
-	for (pid = 0; pid < nb_ports; pid++) {
+	/* enabled allocated ports */
+	for (pid = 0; pid < nb_ports; pid++)
+		ports[pid].enabled = 1;
+
+	FOREACH_PORT(pid, ports) {
 		port = &ports[pid];
 		rte_eth_dev_info_get(pid, &port->dev_info);
 
@@ -602,8 +623,7 @@ init_config(void)
 			nb_mbuf_per_pool = nb_mbuf_per_pool/nb_ports;
 
 		for (i = 0; i < MAX_SOCKET; i++) {
-			nb_mbuf = (nb_mbuf_per_pool *
-						port_per_socket[i]);
+			nb_mbuf = (nb_mbuf_per_pool * RTE_MAX_ETHPORTS);
 			if (nb_mbuf)
 				mbuf_pool_create(mbuf_data_size,
 						nb_mbuf,i);
@@ -635,14 +655,6 @@ reconfig(portid_t new_port_id, unsigned socket_id)
 	struct rte_port *port;
 
 	/* Reconfiguration of Ethernet ports. */
-	ports = rte_realloc(ports,
-			    sizeof(struct rte_port) * nb_ports,
-			    RTE_CACHE_LINE_SIZE);
-	if (ports == NULL) {
-		rte_exit(EXIT_FAILURE, "rte_realloc(%d struct rte_port) failed\n",
-				nb_ports);
-	}
-
 	port = &ports[new_port_id];
 	rte_eth_dev_info_get(new_port_id, &port->dev_info);
 
@@ -663,7 +675,7 @@ init_fwd_streams(void)
 	streamid_t sm_id, nb_fwd_streams_new;
 
 	/* set socket id according to numa or not */
-	for (pid = 0; pid < nb_ports; pid++) {
+	FOREACH_PORT(pid, ports) {
 		port = &ports[pid];
 		if (nb_rxq > port->dev_info.max_rx_queues) {
 			printf("Fail: nb_rxq(%d) is greater than "
@@ -1264,7 +1276,7 @@ all_ports_started(void)
 	portid_t pi;
 	struct rte_port *port;
 
-	for (pi = 0; pi < nb_ports; pi++) {
+	FOREACH_PORT(pi, ports) {
 		port = &ports[pi];
 		/* Check if there is a port which is not started */
 		if (port->port_status != RTE_PORT_STARTED)
@@ -1276,6 +1288,45 @@ all_ports_started(void)
 }
 
 int
+all_ports_stopped(void)
+{
+	portid_t pi;
+	struct rte_port *port;
+
+	FOREACH_PORT(pi, ports) {
+		port = &ports[pi];
+		if (port->port_status != RTE_PORT_STOPPED)
+			return 0;
+	}
+
+	return 1;
+}
+
+int
+port_is_started(portid_t port_id)
+{
+	if (port_id_is_invalid(port_id, ENABLED_WARN))
+		return 0;
+
+	if (ports[port_id].port_status != RTE_PORT_STARTED)
+		return 0;
+
+	return 1;
+}
+
+static int
+port_is_closed(portid_t port_id)
+{
+	if (port_id_is_invalid(port_id, ENABLED_WARN))
+		return 0;
+
+	if (ports[port_id].port_status != RTE_PORT_CLOSED)
+		return 0;
+
+	return 1;
+}
+
+int
 start_port(portid_t pid)
 {
 	int diag, need_check_link_status = 0;
@@ -1296,8 +1347,8 @@ start_port(portid_t pid)
 
 	if(dcb_config)
 		dcb_test = 1;
-	for (pi = 0; pi < nb_ports; pi++) {
-		if (pid < nb_ports && pid != pi)
+	FOREACH_PORT(pi, ports) {
+		if (!port_id_is_invalid(pid, DISABLED_WARN) && pid != pi)
 			continue;
 
 		port = &ports[pi];
@@ -1421,7 +1472,7 @@ start_port(portid_t pid)
 	}
 
 	if (need_check_link_status && !no_link_check)
-		check_all_ports_link_status(nb_ports, RTE_PORT_ALL);
+		check_all_ports_link_status(RTE_PORT_ALL);
 	else
 		printf("Please stop the ports first\n");
 
@@ -1446,8 +1497,8 @@ stop_port(portid_t pid)
 	}
 	printf("Stopping ports...\n");
 
-	for (pi = 0; pi < nb_ports; pi++) {
-		if (pid < nb_ports && pid != pi)
+	FOREACH_PORT(pi, ports) {
+		if (!port_id_is_invalid(pid, DISABLED_WARN) && pid != pi)
 			continue;
 
 		port = &ports[pi];
@@ -1463,7 +1514,7 @@ stop_port(portid_t pid)
 		need_check_link_status = 1;
 	}
 	if (need_check_link_status && !no_link_check)
-		check_all_ports_link_status(nb_ports, RTE_PORT_ALL);
+		check_all_ports_link_status(RTE_PORT_ALL);
 
 	printf("Done\n");
 }
@@ -1481,8 +1532,8 @@ close_port(portid_t pid)
 
 	printf("Closing ports...\n");
 
-	for (pi = 0; pi < nb_ports; pi++) {
-		if (pid < nb_ports && pid != pi)
+	FOREACH_PORT(pi, ports) {
+		if (!port_id_is_invalid(pid, DISABLED_WARN) && pid != pi)
 			continue;
 
 		port = &ports[pi];
@@ -1502,31 +1553,83 @@ close_port(portid_t pid)
 	printf("Done\n");
 }
 
-int
-all_ports_stopped(void)
+void
+attach_port(char *identifier)
 {
-	portid_t pi;
-	struct rte_port *port;
+	portid_t i, j, pi = 0;
 
-	for (pi = 0; pi < nb_ports; pi++) {
-		port = &ports[pi];
-		if (port->port_status != RTE_PORT_STOPPED)
-			return 0;
+	printf("Attaching a new port...\n");
+
+	if (identifier == NULL) {
+		printf("Invalid parameters are speficied\n");
+		return;
 	}
 
-	return 1;
+	if (test_done == 0) {
+		printf("Please stop forwarding first\n");
+		return;
+	}
+
+	if (rte_eal_dev_attach(identifier, &pi))
+		return;
+
+	ports[pi].enabled = 1;
+	reconfig(pi, rte_eth_dev_socket_id(pi));
+	rte_eth_promiscuous_enable(pi);
+
+	nb_ports = rte_eth_dev_count();
+
+	/* set_default_fwd_ports_config(); */
+	bzero(fwd_ports_ids, sizeof(fwd_ports_ids));
+	i = 0;
+	FOREACH_PORT(j, ports) {
+		fwd_ports_ids[i] = j;
+		i++;
+	}
+	nb_cfg_ports = nb_ports;
+	nb_fwd_ports++;
+
+	ports[pi].port_status = RTE_PORT_STOPPED;
+
+	printf("Port %d is attached. Now total ports is %d\n", pi, nb_ports);
+	printf("Done\n");
 }
 
-int
-port_is_started(portid_t port_id)
+void
+detach_port(uint8_t port_id)
 {
-	if (port_id_is_invalid(port_id))
-		return -1;
+	portid_t i, pi = 0;
+	char name[RTE_ETH_NAME_MAX_LEN];
 
-	if (ports[port_id].port_status != RTE_PORT_STARTED)
-		return 0;
+	printf("Detaching a port...\n");
 
-	return 1;
+	if (!port_is_closed(port_id)) {
+		printf("Please close port first\n");
+		return;
+	}
+
+	rte_eth_promiscuous_disable(port_id);
+
+	if (rte_eal_dev_detach(port_id, name))
+		return;
+
+	ports[port_id].enabled = 0;
+	nb_ports = rte_eth_dev_count();
+
+	/* set_default_fwd_ports_config(); */
+	bzero(fwd_ports_ids, sizeof(fwd_ports_ids));
+	i = 0;
+	FOREACH_PORT(pi, ports) {
+		fwd_ports_ids[i] = pi;
+		i++;
+	}
+	nb_cfg_ports = nb_ports;
+	nb_fwd_ports--;
+
+	printf("Port '%s' is detached. Now total ports is %d\n",
+			name, nb_ports);
+	printf("Done\n");
+	return;
 }
 
 void
@@ -1534,7 +1637,7 @@ pmd_test_exit(void)
 {
 	portid_t pt_id;
 
-	for (pt_id = 0; pt_id < nb_ports; pt_id++) {
+	FOREACH_PORT(pt_id, ports) {
 		printf("Stopping port %d...", pt_id);
 		fflush(stdout);
 		rte_eth_dev_close(pt_id);
@@ -1553,7 +1656,7 @@ struct pmd_test_command {
 
 /* Check the link status of all ports in up to 9s, and print them finally */
 static void
-check_all_ports_link_status(uint8_t port_num, uint32_t port_mask)
+check_all_ports_link_status(uint32_t port_mask)
 {
 #define CHECK_INTERVAL 100 /* 100ms */
 #define MAX_CHECK_TIME 90 /* 9s (90 * 100ms) in total */
@@ -1564,7 +1667,7 @@ check_all_ports_link_status(uint8_t port_num, uint32_t port_mask)
 	fflush(stdout);
 	for (count = 0; count <= MAX_CHECK_TIME; count++) {
 		all_ports_up = 1;
-		for (portid = 0; portid < port_num; portid++) {
+		FOREACH_PORT(portid, ports) {
 			if ((port_mask & (1 << portid)) == 0)
 				continue;
 			memset(&link, 0, sizeof(link));
@@ -1688,7 +1791,7 @@ init_port_config(void)
 	portid_t pid;
 	struct rte_port *port;
 
-	for (pid = 0; pid < nb_ports; pid++) {
+	FOREACH_PORT(pid, ports) {
 		port = &ports[pid];
 		port->dev_conf.rxmode = rx_mode;
 		port->dev_conf.fdir_conf = fdir_conf;
@@ -1877,7 +1980,7 @@ main(int argc, char** argv)
 
 	nb_ports = (portid_t) rte_eth_dev_count();
 	if (nb_ports == 0)
-		rte_exit(EXIT_FAILURE, "No probed ethernet device\n");
+		RTE_LOG(WARNING, EAL, "No probed ethernet devices\n");
 
 	set_def_fwd_config();
 	if (nb_lcores == 0)
@@ -1899,7 +2002,7 @@ main(int argc, char** argv)
 		rte_exit(EXIT_FAILURE, "Start ports failed\n");
 
 	/* set all ports to promiscuous mode by default */
-	for (port_id = 0; port_id < nb_ports; port_id++)
+	FOREACH_PORT(port_id, ports)
 		rte_eth_promiscuous_enable(port_id);
 
 #ifdef RTE_LIBRTE_CMDLINE
diff --git a/app/test-pmd/testpmd.h b/app/test-pmd/testpmd.h
index 8f5e6c7..109c670 100644
--- a/app/test-pmd/testpmd.h
+++ b/app/test-pmd/testpmd.h
@@ -134,6 +134,7 @@ struct fwd_stream {
  * The data structure associated with each port.
  */
 struct rte_port {
+	uint8_t                 enabled;    /**< Port enabled or not */
 	struct rte_eth_dev_info dev_info;   /**< PCI info + driver name */
 	struct rte_eth_conf     dev_conf;   /**< Port configuration. */
 	struct ether_addr       eth_addr;   /**< Port ethernet address */
@@ -159,6 +160,14 @@ struct rte_port {
 	struct rte_eth_txconf   tx_conf;    /**< tx configuration */
 };
 
+extern portid_t __rte_unused
+find_next_port(portid_t p, struct rte_port *ports, int size);
+
+#define FOREACH_PORT(p, ports) \
+	for (p = find_next_port(0, ports, RTE_MAX_ETHPORTS); \
+	    p < RTE_MAX_ETHPORTS; \
+	    p = find_next_port(p + 1, ports, RTE_MAX_ETHPORTS))
+
 /**
  * The data structure associated with each forwarding logical core.
  * The logical cores are internally numbered by a core index from 0 to
@@ -515,6 +524,8 @@ int init_port_dcb_config(portid_t pid,struct dcb_config *dcb_conf);
 int start_port(portid_t pid);
 void stop_port(portid_t pid);
 void close_port(portid_t pid);
+void attach_port(char *identifier);
+void detach_port(uint8_t port_id);
 int all_ports_stopped(void);
 int port_is_started(portid_t port_id);
 void pmd_test_exit(void);
@@ -558,10 +569,15 @@ void get_ethertype_filter(uint8_t port_id, uint16_t index);
 void get_2tuple_filter(uint8_t port_id, uint16_t index);
 void get_5tuple_filter(uint8_t port_id, uint16_t index);
 void get_flex_filter(uint8_t port_id, uint16_t index);
-int port_id_is_invalid(portid_t port_id);
 int rx_queue_id_is_invalid(queueid_t rxq_id);
 int tx_queue_id_is_invalid(queueid_t txq_id);
 
+enum print_warning {
+	ENABLED_WARN = 0,
+	DISABLED_WARN
+};
+int port_id_is_invalid(portid_t port_id, enum print_warning warning);
+
 /*
  * Work-around of a compilation error with ICC on invocations of the
  * rte_be_to_cpu_16() function.
diff --git a/doc/guides/testpmd_app_ug/testpmd_funcs.rst b/doc/guides/testpmd_app_ug/testpmd_funcs.rst
index 218835a..1cacbcf 100644
--- a/doc/guides/testpmd_app_ug/testpmd_funcs.rst
+++ b/doc/guides/testpmd_app_ug/testpmd_funcs.rst
@@ -808,6 +808,63 @@ The following sections show functions for configuring ports.
 
     Port configuration changes only become active when forwarding is started/restarted.
 
+port attach
+~~~~~~~~~~~
+
+Attach a port specified by pci address or virtual device args.
+
+To attach a new pci device, the device should be recognized by kernel first.
+Then it should be moved under DPDK management.
+Finally the port can be attached to testpmd.
+On the other hand, to attach a port created by virtual device, above steps are not needed.
+
+port attach (identifier)
+
+For example, to attach a port that pci address is 0000:02:00.0.
+
+.. code-block:: console
+
+    testpmd> port attach 0000:02:00.0
+    Attaching a new port...
+    ... snip ...
+    Port 0 is attached. Now total ports is 1
+    Done
+
+For example, to attach a port created by pcap PMD.
+
+.. code-block:: console
+
+    testpmd> port attach eth_pcap0,iface=eth0
+    Attaching a new port...
+    ... snip ...
+    Port 0 is attached. Now total ports is 1
+    Done
+
+In this case, identifier is "eth_pcap0,iface=eth0".
+This identifier format is the same as "--vdev" format of DPDK applications.
+
+port detach
+~~~~~~~~~~~
+
+Detach a specific port.
+
+Before detaching a port, the port should be closed.
+Also to remove a pci device completely from the system, first detach the port from testpmd.
+Then the device should be moved under kernel management.
+Finally the device can be remove using kernel pci hotplug functionality.
+On the other hand, to remove a port created by virtual device, above steps are not needed.
+
+port detach (port_id)
+
+For example, to detach a port 0.
+
+.. code-block:: console
+
+    testpmd> port detach 0
+    Detaching a port...
+    ... snip ...
+    Done
+
 port start
 ~~~~~~~~~~
 
-- 
1.9.1

^ permalink raw reply	[flat|nested] 362+ messages in thread

* Re: [dpdk-dev] [PATCH v6 12/13] eal/pci: Add rte_eal_dev_attach/detach() functions
  2015-02-01  4:01       ` [dpdk-dev] [PATCH v6 12/13] eal/pci: Add rte_eal_dev_attach/detach() functions Tetsuya Mukawa
@ 2015-02-02  5:42         ` Qiu, Michael
  2015-02-02  6:22           ` Qiu, Michael
  0 siblings, 1 reply; 362+ messages in thread
From: Qiu, Michael @ 2015-02-02  5:42 UTC (permalink / raw)
  To: Tetsuya Mukawa, dev

On 2/1/2015 12:02 PM, Tetsuya Mukawa wrote:
> These functions are used for attaching or detaching a port.
> When rte_eal_dev_attach() is called, the function tries to realize the
> device name as pci address. If this is done successfully,
> rte_eal_dev_attach() will attach physical device port. If not, attaches
> virtual devive port.
> When rte_eal_dev_detach() is called, the function gets the device type
> of this port to know whether the port is came from physical or virtual.
> And then specific detaching function will be called.
>
> v5:
> - Change function names like below.
>   rte_eal_dev_find_and_invoke() to rte_eal_vdev_find_and_invoke().
>   rte_eal_dev_invoke() to rte_eal_vdev_invoke().
> - Add code to handle a return value of rte_eal_devargs_remove().
> - Fix pci address format in rte_eal_dev_detach().
> v4:
> - Fix comment.
> - Add error checking.
> - Fix indent of 'if' statement.
> - Change function name.
>
> Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp>
> ---
>  lib/librte_eal/common/eal_common_dev.c  | 274 ++++++++++++++++++++++++++++++++
>  lib/librte_eal/common/eal_private.h     |  11 ++
>  lib/librte_eal/common/include/rte_dev.h |  33 ++++
>  lib/librte_eal/linuxapp/eal/Makefile    |   1 +
>  lib/librte_eal/linuxapp/eal/eal_pci.c   |   6 +-
>  5 files changed, 322 insertions(+), 3 deletions(-)
>
> diff --git a/lib/librte_eal/common/eal_common_dev.c b/lib/librte_eal/common/eal_common_dev.c
> index eae5656..e3a3f54 100644
> --- a/lib/librte_eal/common/eal_common_dev.c
> +++ b/lib/librte_eal/common/eal_common_dev.c
> @@ -32,10 +32,13 @@
>   *   OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
>   */
>  
> +#include <stdio.h>
> +#include <limits.h>
>  #include <string.h>
>  #include <inttypes.h>
>  #include <sys/queue.h>
>  
> +#include <rte_ethdev.h>
>  #include <rte_dev.h>
>  #include <rte_devargs.h>
>  #include <rte_debug.h>
> @@ -107,3 +110,274 @@ rte_eal_dev_init(void)
>  	}
>  	return 0;
>  }
> +
> +/* So far, DPDK hotplug function only supports linux */
> +#ifdef ENABLE_HOTPLUG
> +static void
> +rte_eal_vdev_invoke(struct rte_driver *driver,
> +		struct rte_devargs *devargs, enum rte_eal_invoke_type type)
> +{
> +	if ((driver == NULL) || (devargs == NULL))
> +		return;
> +
> +	switch (type) {
> +	case RTE_EAL_INVOKE_TYPE_PROBE:
> +		driver->init(devargs->virtual.drv_name, devargs->args);
> +		break;
> +	case RTE_EAL_INVOKE_TYPE_CLOSE:
> +		driver->uninit(devargs->virtual.drv_name, devargs->args);
> +		break;
> +	default:
> +		break;
> +	}
> +}
> +
> +static int
> +rte_eal_vdev_find_and_invoke(const char *name, int type)
> +{
> +	struct rte_devargs *devargs;
> +	struct rte_driver *driver;
> +
> +	if (name == NULL)
> +		return -EINVAL;
> +
> +	/* call the init function for each virtual device */
> +	TAILQ_FOREACH(devargs, &devargs_list, next) {
> +
> +		if (devargs->type != RTE_DEVTYPE_VIRTUAL)
> +			continue;
> +
> +		if (strncmp(name, devargs->virtual.drv_name, strlen(name)))
> +			continue;
> +
> +		TAILQ_FOREACH(driver, &dev_driver_list, next) {
> +			if (driver->type != PMD_VDEV)
> +				continue;
> +
> +			/* search a driver prefix in virtual device name */
> +			if (!strncmp(driver->name, devargs->virtual.drv_name,
> +			    strlen(driver->name))) {
> +				rte_eal_vdev_invoke(driver, devargs, type);
> +				break;
> +			}
> +		}
> +
> +		if (driver == NULL) {
> +			RTE_LOG(WARNING, EAL, "no driver found for %s\n",
> +				  devargs->virtual.drv_name);
> +		}
> +		return 0;
> +	}
> +	return 1;
> +}
> +
> +/* attach the new physical device, then store port_id of the device */
> +static int
> +rte_eal_dev_attach_pdev(struct rte_pci_addr *addr, uint8_t *port_id)
> +{
> +	uint8_t new_port_id;
> +	struct rte_eth_dev devs[RTE_MAX_ETHPORTS];
> +
> +	if ((addr == NULL) || (port_id == NULL))
> +		goto err;
> +
> +	/* save current port status */
> +	rte_eth_dev_save(devs);
> +	/* re-construct pci_device_list */
> +	if (rte_eal_pci_scan())
> +		goto err;
> +	/* invoke probe func of the driver can handle the new device */
> +	if (rte_eal_pci_probe_one(addr))
> +		goto err;
> +	/* get port_id enabled by above procedures */
> +	if (rte_eth_dev_get_changed_port(devs, &new_port_id))
> +		goto err;
> +
> +	*port_id = new_port_id;
> +	return 0;
> +err:
> +	RTE_LOG(ERR, EAL, "Drver, cannot attach the device\n");

Sorry, what does "Drver" means?

My English is bad, also I haven't gotten this work in google

Thanks,
Michael
> +	return -1;
> +}
> +
> +/* detach the new physical device, then store pci_addr of the device */
> +static int
> +rte_eal_dev_detach_pdev(uint8_t port_id, struct rte_pci_addr *addr)
> +{
> +	struct rte_pci_addr freed_addr;
> +	struct rte_pci_addr vp;
> +
> +	if (addr == NULL)
> +		goto err;
> +
> +	/* check whether the driver supports detach feature, or not */
> +	if (rte_eth_dev_check_detachable(port_id))
> +		goto err;
> +
> +	/* get pci address by port id */
> +	if (rte_eth_dev_get_addr_by_port(port_id, &freed_addr))
> +		goto err;
> +
> +	/* Zerod pci addr means the port comes from virtual device */
> +	vp.domain = vp.bus = vp.devid = vp.function = 0;
> +	if (eal_compare_pci_addr(&vp, &freed_addr) == 0)
> +		goto err;
> +
> +	/* invoke close func of the driver,
> +	 * also remove the device from pci_device_list */
> +	if (rte_eal_pci_close_one(&freed_addr))
> +		goto err;
> +
> +	*addr = freed_addr;
> +	return 0;
> +err:
> +	RTE_LOG(ERR, EAL, "Drver, cannot detach the device\n");

Here, "Drver"
> +	return -1;
> +}
> +
> +static void
> +get_vdev_name(char *vdevargs)
> +{
> +	char *sep;
> +
> +	if (vdevargs == NULL)
> +		return;
> +
> +	/* set the first ',' to '\0' to split name and arguments */
> +	sep = strchr(vdevargs, ',');
> +	if (sep != NULL)
> +		sep[0] = '\0';
> +}
> +
> +/* attach the new virtual device, then store port_id of the device */
> +static int
> +rte_eal_dev_attach_vdev(const char *vdevargs, uint8_t *port_id)
> +{
> +	char *args;
> +	uint8_t new_port_id;
> +	struct rte_eth_dev devs[RTE_MAX_ETHPORTS];
> +
> +	if ((vdevargs == NULL) || (port_id == NULL))
> +		goto err0;
> +
> +	args = strdup(vdevargs);
> +	if (args == NULL)
> +		goto err0;
> +
> +	/* save current port status */
> +	rte_eth_dev_save(devs);
> +	/* add the vdevargs to devargs_list */
> +	if (rte_eal_devargs_add(RTE_DEVTYPE_VIRTUAL, args))
> +		goto err1;
> +	/* parse vdevargs, then retrieve device name */
> +	get_vdev_name(args);
> +	/* walk around dev_driver_list to find the driver of the device,
> +	 * then invoke probe function o the driver */
> +	if (rte_eal_vdev_find_and_invoke(args, RTE_EAL_INVOKE_TYPE_PROBE))
> +		goto err2;
> +	/* get port_id enabled by above procedures */
> +	if (rte_eth_dev_get_changed_port(devs, &new_port_id))
> +		goto err2;
> +
> +	free(args);
> +	*port_id = new_port_id;
> +	return 0;
> +err2:
> +	rte_eal_devargs_remove(RTE_DEVTYPE_VIRTUAL, args);
> +err1:
> +	free(args);
> +err0:
> +	RTE_LOG(ERR, EAL, "Drver, cannot detach the device\n");

Here also "Drver",


Thanks,
Michael
> +	return -1;
> +}
> +
> +/* detach the new virtual device, then store the name of the device */
> +static int
> +rte_eal_dev_detach_vdev(uint8_t port_id, char *vdevname)
> +{
> +	char name[RTE_ETH_NAME_MAX_LEN];
> +
> +	if (vdevname == NULL)
> +		goto err;
> +
> +	/* check whether the driver supports detach feature, or not */
> +	if (rte_eth_dev_check_detachable(port_id))
> +		goto err;
> +
> +	/* get device name by port id */
> +	if (rte_eth_dev_get_name_by_port(port_id, name))
> +		goto err;
> +	/* walk around dev_driver_list to find the driver of the device,
> +	 * then invoke close function o the driver */
> +	if (rte_eal_vdev_find_and_invoke(name, RTE_EAL_INVOKE_TYPE_CLOSE))
> +		goto err;
> +	/* remove the vdevname from devargs_list */
> +	if (rte_eal_devargs_remove(RTE_DEVTYPE_VIRTUAL, name))
> +		goto err;
> +
> +	strncpy(vdevname, name, sizeof(name));
> +	return 0;
> +err:
> +	RTE_LOG(ERR, EAL, "Drver, cannot detach the device\n");
> +	return -1;
> +}
> +
> +/* attach the new device, then store port_id of the device */
> +int
> +rte_eal_dev_attach(const char *devargs, uint8_t *port_id)
> +{
> +	struct rte_pci_addr addr;
> +
> +	if ((devargs == NULL) || (port_id == NULL))
> +		return -EINVAL;
> +
> +	if (eal_parse_pci_DomBDF(devargs, &addr) == 0)
> +		return rte_eal_dev_attach_pdev(&addr, port_id);
> +	else
> +		return rte_eal_dev_attach_vdev(devargs, port_id);
> +}
> +
> +/* detach the device, then store the name of the device */
> +int
> +rte_eal_dev_detach(uint8_t port_id, char *name)
> +{
> +	struct rte_pci_addr addr;
> +	int ret;
> +
> +	if (name == NULL)
> +		return -EINVAL;
> +
> +	if (rte_eth_dev_get_device_type(port_id) == RTE_ETH_DEV_PHYSICAL) {
> +		ret = rte_eth_dev_get_addr_by_port(port_id, &addr);
> +		if (ret < 0)
> +			return ret;
> +
> +		ret = rte_eal_dev_detach_pdev(port_id, &addr);
> +		if (ret == 0)
> +			snprintf(name, RTE_ETH_NAME_MAX_LEN,
> +				"%04x:%02x:%02x.%d",
> +				addr.domain, addr.bus,
> +				addr.devid, addr.function);
> +
> +		return ret;
> +	} else
> +		return rte_eal_dev_detach_vdev(port_id, name);
> +}
> +#else /* ENABLE_HOTPLUG */
> +int
> +rte_eal_dev_attach(const char *devargs __rte_unused,
> +			uint8_t *port_id __rte_unused)
> +{
> +	RTE_LOG(ERR, EAL, "Hotplug support isn't enabled\n");
> +	return -1;
> +}
> +
> +/* detach the device, then store the name of the device */
> +int
> +rte_eal_dev_detach(uint8_t port_id __rte_unused,
> +			char *name __rte_unused)
> +{
> +	RTE_LOG(ERR, EAL, "Hotplug support isn't enabled\n");
> +	return -1;
> +}
> +#endif /* ENABLE_HOTPLUG */
> diff --git a/lib/librte_eal/common/eal_private.h b/lib/librte_eal/common/eal_private.h
> index 1a362ab..8168a7a 100644
> --- a/lib/librte_eal/common/eal_private.h
> +++ b/lib/librte_eal/common/eal_private.h
> @@ -163,6 +163,17 @@ enum rte_eal_invoke_type {
>  };
>  
>  /**
> + * Scan the content of the PCI bus, and the devices in the devices
> + * list
> + *
> + * This function is private to EAL.
> + *
> + * @return
> + *  0 on success, negative on error
> + */
> +int rte_eal_pci_scan(void);
> +
> +/**
>   * Mmap memory for single PCI device
>   *
>   * This function is private to EAL.
> diff --git a/lib/librte_eal/common/include/rte_dev.h b/lib/librte_eal/common/include/rte_dev.h
> index f7e3a10..e63dd1c 100644
> --- a/lib/librte_eal/common/include/rte_dev.h
> +++ b/lib/librte_eal/common/include/rte_dev.h
> @@ -47,6 +47,7 @@ extern "C" {
>  #endif
>  
>  #include <sys/queue.h>
> +#include <rte_pci.h>
>  
>  /** Double linked list of device drivers. */
>  TAILQ_HEAD(rte_driver_list, rte_driver);
> @@ -57,6 +58,11 @@ TAILQ_HEAD(rte_driver_list, rte_driver);
>  typedef int (rte_dev_init_t)(const char *name, const char *args);
>  
>  /**
> + * Uninitilization function called for each device driver once.
> + */
> +typedef int (rte_dev_uninit_t)(const char *name, const char *args);
> +
> +/**
>   * Driver type enumeration
>   */
>  enum pmd_type {
> @@ -72,6 +78,7 @@ struct rte_driver {
>  	enum pmd_type type;		   /**< PMD Driver type */
>  	const char *name;                   /**< Driver name. */
>  	rte_dev_init_t *init;              /**< Device init. function. */
> +	rte_dev_uninit_t *uninit;          /**< Device uninit. function. */
>  };
>  
>  /**
> @@ -93,6 +100,32 @@ void rte_eal_driver_register(struct rte_driver *driver);
>  void rte_eal_driver_unregister(struct rte_driver *driver);
>  
>  /**
> + * Attach a new device.
> + *
> + * @param devargs
> + *   A pointer to a strings array describing the new device
> + *   to be attached. The strings should be a pci address like
> + *   '0000:01:00.0' or virtual device name like 'eth_pcap0'.
> + * @param port_id
> + *  A pointer to a port identifier actually attached.
> + * @return
> + *  0 on success and port_id is filled, negative on error
> + */
> +int rte_eal_dev_attach(const char *devargs, uint8_t *port_id);
> +
> +/**
> + * Detach a device.
> + *
> + * @param port_id
> + *   The port identifier of the device to detach.
> + * @param addr
> + *  A pointer to a device name actually detached.
> + * @return
> + *  0 on success and devname is filled, negative on error
> + */
> +int rte_eal_dev_detach(uint8_t port_id, char *devname);
> +
> +/**
>   * Initalize all the registered drivers in this process
>   */
>  int rte_eal_dev_init(void);
> diff --git a/lib/librte_eal/linuxapp/eal/Makefile b/lib/librte_eal/linuxapp/eal/Makefile
> index 72ecf3a..0ec83b5 100644
> --- a/lib/librte_eal/linuxapp/eal/Makefile
> +++ b/lib/librte_eal/linuxapp/eal/Makefile
> @@ -41,6 +41,7 @@ CFLAGS += -I$(RTE_SDK)/lib/librte_eal/common/include
>  CFLAGS += -I$(RTE_SDK)/lib/librte_ring
>  CFLAGS += -I$(RTE_SDK)/lib/librte_mempool
>  CFLAGS += -I$(RTE_SDK)/lib/librte_malloc
> +CFLAGS += -I$(RTE_SDK)/lib/librte_mbuf
>  CFLAGS += -I$(RTE_SDK)/lib/librte_ether
>  CFLAGS += -I$(RTE_SDK)/lib/librte_ivshmem
>  CFLAGS += -I$(RTE_SDK)/lib/librte_pmd_ring
> diff --git a/lib/librte_eal/linuxapp/eal/eal_pci.c b/lib/librte_eal/linuxapp/eal/eal_pci.c
> index 831422e..1f43688 100644
> --- a/lib/librte_eal/linuxapp/eal/eal_pci.c
> +++ b/lib/librte_eal/linuxapp/eal/eal_pci.c
> @@ -431,8 +431,8 @@ error:
>   * Scan the content of the PCI bus, and the devices in the devices
>   * list
>   */
> -static int
> -pci_scan(void)
> +int
> +rte_eal_pci_scan(void)
>  {
>  	struct dirent *e;
>  	DIR *dir;
> @@ -764,7 +764,7 @@ rte_eal_pci_init(void)
>  	if (internal_config.no_pci)
>  		return 0;
>  
> -	if (pci_scan() < 0) {
> +	if (rte_eal_pci_scan() < 0) {
>  		RTE_LOG(ERR, EAL, "%s(): Cannot scan PCI bus\n", __func__);
>  		return -1;
>  	}


^ permalink raw reply	[flat|nested] 362+ messages in thread

* Re: [dpdk-dev] [PATCH v6 12/13] eal/pci: Add rte_eal_dev_attach/detach() functions
  2015-02-02  5:42         ` Qiu, Michael
@ 2015-02-02  6:22           ` Qiu, Michael
  2015-02-03  1:28             ` Tetsuya Mukawa
  0 siblings, 1 reply; 362+ messages in thread
From: Qiu, Michael @ 2015-02-02  6:22 UTC (permalink / raw)
  To: Tetsuya Mukawa, dev

On 2/2/2015 1:43 PM, Qiu, Michael wrote:
> On 2/1/2015 12:02 PM, Tetsuya Mukawa wrote:
>> These functions are used for attaching or detaching a port.
>> When rte_eal_dev_attach() is called, the function tries to realize the
>> device name as pci address. If this is done successfully,
>> rte_eal_dev_attach() will attach physical device port. If not, attaches
>> virtual devive port.
>> When rte_eal_dev_detach() is called, the function gets the device type
>> of this port to know whether the port is came from physical or virtual.
>> And then specific detaching function will be called.
>>
>> v5:
>> - Change function names like below.
>>   rte_eal_dev_find_and_invoke() to rte_eal_vdev_find_and_invoke().
>>   rte_eal_dev_invoke() to rte_eal_vdev_invoke().
>> - Add code to handle a return value of rte_eal_devargs_remove().
>> - Fix pci address format in rte_eal_dev_detach().
>> v4:
>> - Fix comment.
>> - Add error checking.
>> - Fix indent of 'if' statement.
>> - Change function name.
>>

[...]

>> +/* attach the new virtual device, then store port_id of the device */
>> +static int
>> +rte_eal_dev_attach_vdev(const char *vdevargs, uint8_t *port_id)
>> +{
>> +	char *args;
>> +	uint8_t new_port_id;
>> +	struct rte_eth_dev devs[RTE_MAX_ETHPORTS];
>> +
>> +	if ((vdevargs == NULL) || (port_id == NULL))
>> +		goto err0;
>> +
>> +	args = strdup(vdevargs);
>> +	if (args == NULL)
>> +		goto err0;
>> +
>> +	/* save current port status */
>> +	rte_eth_dev_save(devs);
>> +	/* add the vdevargs to devargs_list */
>> +	if (rte_eal_devargs_add(RTE_DEVTYPE_VIRTUAL, args))
>> +		goto err1;
>> +	/* parse vdevargs, then retrieve device name */
>> +	get_vdev_name(args);
>> +	/* walk around dev_driver_list to find the driver of the device,
>> +	 * then invoke probe function o the driver */
>> +	if (rte_eal_vdev_find_and_invoke(args, RTE_EAL_INVOKE_TYPE_PROBE))
>> +		goto err2;
>> +	/* get port_id enabled by above procedures */
>> +	if (rte_eth_dev_get_changed_port(devs, &new_port_id))
>> +		goto err2;
>> +
>> +	free(args);
>> +	*port_id = new_port_id;
>> +	return 0;
>> +err2:
>> +	rte_eal_devargs_remove(RTE_DEVTYPE_VIRTUAL, args);
>> +err1:
>> +	free(args);
>> +err0:
>> +	RTE_LOG(ERR, EAL, "Drver, cannot detach the device\n");

Here "cannot detach the device\n" should be "cannot attach the device" I
think.

> Here also "Drver",
>
>
> Thanks,
> Michael
>> +	return -1;
>> +}
>> +
>> +/* detach the new virtual device, then store the name of the device */
>> +static int
>> +rte_eal_dev_detach_vdev(uint8_t port_id, char *vdevname)
>> +{
>> +	char name[RTE_ETH_NAME_MAX_LEN];
>> +
>> +	if (vdevname == NULL)
>> +		goto err;
>> +
>> +	/* check whether the driver supports detach feature, or not */
>> +	if (rte_eth_dev_check_detachable(port_id))
>> +		goto err;
>> +
>> +	/* get device name by port id */
>> +	if (rte_eth_dev_get_name_by_port(port_id, name))
>> +		goto err;
>> +	/* walk around dev_driver_list to find the driver of the device,
>> +	 * then invoke close function o the driver */
>> +	if (rte_eal_vdev_find_and_invoke(name, RTE_EAL_INVOKE_TYPE_CLOSE))
>> +		goto err;
>> +	/* remove the vdevname from devargs_list */
>> +	if (rte_eal_devargs_remove(RTE_DEVTYPE_VIRTUAL, name))
>> +		goto err;
>> +
>> +	strncpy(vdevname, name, sizeof(name));
>> +	return 0;
>> +err:
>> +	RTE_LOG(ERR, EAL, "Drver, cannot detach the device\n");
>> +	return -1;
>> +}
>> +
>> +/* attach the new device, then store port_id of the device */
>> +int
>> +rte_eal_dev_attach(const char *devargs, uint8_t *port_id)
>> +{
>> +	struct rte_pci_addr addr;
>> +
>> +	if ((devargs == NULL) || (port_id == NULL))
>> +		return -EINVAL;
>> +
>> +	if (eal_parse_pci_DomBDF(devargs, &addr) == 0)
>> +		return rte_eal_dev_attach_pdev(&addr, port_id);
>> +	else
>> +		return rte_eal_dev_attach_vdev(devargs, port_id);
>> +}
>> +
>> +/* detach the device, then store the name of the device */
>> +int
>> +rte_eal_dev_detach(uint8_t port_id, char *name)
>> +{
>> +	struct rte_pci_addr addr;
>> +	int ret;
>> +
>> +	if (name == NULL)
>> +		return -EINVAL;
>> +
>> +	if (rte_eth_dev_get_device_type(port_id) == RTE_ETH_DEV_PHYSICAL) {
>> +		ret = rte_eth_dev_get_addr_by_port(port_id, &addr);
>> +		if (ret < 0)
>> +			return ret;
>> +
>> +		ret = rte_eal_dev_detach_pdev(port_id, &addr);
>> +		if (ret == 0)
>> +			snprintf(name, RTE_ETH_NAME_MAX_LEN,
>> +				"%04x:%02x:%02x.%d",
>> +				addr.domain, addr.bus,
>> +				addr.devid, addr.function);
>> +
>> +		return ret;
>> +	} else
>> +		return rte_eal_dev_detach_vdev(port_id, name);
>> +}
>> +#else /* ENABLE_HOTPLUG */
>> +int
>> +rte_eal_dev_attach(const char *devargs __rte_unused,
>> +			uint8_t *port_id __rte_unused)
>> +{
>> +	RTE_LOG(ERR, EAL, "Hotplug support isn't enabled\n");
>> +	return -1;
>> +}
>> +
>> +/* detach the device, then store the name of the device */
>> +int
>> +rte_eal_dev_detach(uint8_t port_id __rte_unused,
>> +			char *name __rte_unused)
>> +{
>> +	RTE_LOG(ERR, EAL, "Hotplug support isn't enabled\n");
>> +	return -1;
>> +}
>> +#endif /* ENABLE_HOTPLUG */
>> diff --git a/lib/librte_eal/common/eal_private.h b/lib/librte_eal/common/eal_private.h
>> index 1a362ab..8168a7a 100644
>> --- a/lib/librte_eal/common/eal_private.h
>> +++ b/lib/librte_eal/common/eal_private.h
>> @@ -163,6 +163,17 @@ enum rte_eal_invoke_type {
>>  };
>>  
>>  /**
>> + * Scan the content of the PCI bus, and the devices in the devices
>> + * list
>> + *
>> + * This function is private to EAL.
>> + *
>> + * @return
>> + *  0 on success, negative on error
>> + */
>> +int rte_eal_pci_scan(void);
>> +
>> +/**
>>   * Mmap memory for single PCI device
>>   *
>>   * This function is private to EAL.
>> diff --git a/lib/librte_eal/common/include/rte_dev.h b/lib/librte_eal/common/include/rte_dev.h
>> index f7e3a10..e63dd1c 100644
>> --- a/lib/librte_eal/common/include/rte_dev.h
>> +++ b/lib/librte_eal/common/include/rte_dev.h
>> @@ -47,6 +47,7 @@ extern "C" {
>>  #endif
>>  
>>  #include <sys/queue.h>
>> +#include <rte_pci.h>
>>  
>>  /** Double linked list of device drivers. */
>>  TAILQ_HEAD(rte_driver_list, rte_driver);
>> @@ -57,6 +58,11 @@ TAILQ_HEAD(rte_driver_list, rte_driver);
>>  typedef int (rte_dev_init_t)(const char *name, const char *args);
>>  
>>  /**
>> + * Uninitilization function called for each device driver once.
>> + */
>> +typedef int (rte_dev_uninit_t)(const char *name, const char *args);
>> +
>> +/**
>>   * Driver type enumeration
>>   */
>>  enum pmd_type {
>> @@ -72,6 +78,7 @@ struct rte_driver {
>>  	enum pmd_type type;		   /**< PMD Driver type */
>>  	const char *name;                   /**< Driver name. */
>>  	rte_dev_init_t *init;              /**< Device init. function. */
>> +	rte_dev_uninit_t *uninit;          /**< Device uninit. function. */
>>  };
>>  
>>  /**
>> @@ -93,6 +100,32 @@ void rte_eal_driver_register(struct rte_driver *driver);
>>  void rte_eal_driver_unregister(struct rte_driver *driver);
>>  
>>  /**
>> + * Attach a new device.
>> + *
>> + * @param devargs
>> + *   A pointer to a strings array describing the new device
>> + *   to be attached. The strings should be a pci address like
>> + *   '0000:01:00.0' or virtual device name like 'eth_pcap0'.
>> + * @param port_id
>> + *  A pointer to a port identifier actually attached.
>> + * @return
>> + *  0 on success and port_id is filled, negative on error
>> + */
>> +int rte_eal_dev_attach(const char *devargs, uint8_t *port_id);
>> +
>> +/**
>> + * Detach a device.
>> + *
>> + * @param port_id
>> + *   The port identifier of the device to detach.
>> + * @param addr
>> + *  A pointer to a device name actually detached.
>> + * @return
>> + *  0 on success and devname is filled, negative on error
>> + */
>> +int rte_eal_dev_detach(uint8_t port_id, char *devname);
>> +
>> +/**
>>   * Initalize all the registered drivers in this process
>>   */
>>  int rte_eal_dev_init(void);
>> diff --git a/lib/librte_eal/linuxapp/eal/Makefile b/lib/librte_eal/linuxapp/eal/Makefile
>> index 72ecf3a..0ec83b5 100644
>> --- a/lib/librte_eal/linuxapp/eal/Makefile
>> +++ b/lib/librte_eal/linuxapp/eal/Makefile
>> @@ -41,6 +41,7 @@ CFLAGS += -I$(RTE_SDK)/lib/librte_eal/common/include
>>  CFLAGS += -I$(RTE_SDK)/lib/librte_ring
>>  CFLAGS += -I$(RTE_SDK)/lib/librte_mempool
>>  CFLAGS += -I$(RTE_SDK)/lib/librte_malloc
>> +CFLAGS += -I$(RTE_SDK)/lib/librte_mbuf
>>  CFLAGS += -I$(RTE_SDK)/lib/librte_ether
>>  CFLAGS += -I$(RTE_SDK)/lib/librte_ivshmem
>>  CFLAGS += -I$(RTE_SDK)/lib/librte_pmd_ring
>> diff --git a/lib/librte_eal/linuxapp/eal/eal_pci.c b/lib/librte_eal/linuxapp/eal/eal_pci.c
>> index 831422e..1f43688 100644
>> --- a/lib/librte_eal/linuxapp/eal/eal_pci.c
>> +++ b/lib/librte_eal/linuxapp/eal/eal_pci.c
>> @@ -431,8 +431,8 @@ error:
>>   * Scan the content of the PCI bus, and the devices in the devices
>>   * list
>>   */
>> -static int
>> -pci_scan(void)
>> +int
>> +rte_eal_pci_scan(void)
>>  {
>>  	struct dirent *e;
>>  	DIR *dir;
>> @@ -764,7 +764,7 @@ rte_eal_pci_init(void)
>>  	if (internal_config.no_pci)
>>  		return 0;
>>  
>> -	if (pci_scan() < 0) {
>> +	if (rte_eal_pci_scan() < 0) {
>>  		RTE_LOG(ERR, EAL, "%s(): Cannot scan PCI bus\n", __func__);
>>  		return -1;
>>  	}
>


^ permalink raw reply	[flat|nested] 362+ messages in thread

* Re: [dpdk-dev] [PATCH v6 12/13] eal/pci: Add rte_eal_dev_attach/detach() functions
  2015-02-02  6:22           ` Qiu, Michael
@ 2015-02-03  1:28             ` Tetsuya Mukawa
  0 siblings, 0 replies; 362+ messages in thread
From: Tetsuya Mukawa @ 2015-02-03  1:28 UTC (permalink / raw)
  To: Qiu, Michael, dev

On 2015/02/02 15:22, Qiu, Michael wrote:
> On 2/2/2015 1:43 PM, Qiu, Michael wrote:
>> On 2/1/2015 12:02 PM, Tetsuya Mukawa wrote:
>>> These functions are used for attaching or detaching a port.
>>> When rte_eal_dev_attach() is called, the function tries to realize the
>>> device name as pci address. If this is done successfully,
>>> rte_eal_dev_attach() will attach physical device port. If not, attaches
>>> virtual devive port.
>>> When rte_eal_dev_detach() is called, the function gets the device type
>>> of this port to know whether the port is came from physical or virtual.
>>> And then specific detaching function will be called.
>>>
>>> v5:
>>> - Change function names like below.
>>>   rte_eal_dev_find_and_invoke() to rte_eal_vdev_find_and_invoke().
>>>   rte_eal_dev_invoke() to rte_eal_vdev_invoke().
>>> - Add code to handle a return value of rte_eal_devargs_remove().
>>> - Fix pci address format in rte_eal_dev_detach().
>>> v4:
>>> - Fix comment.
>>> - Add error checking.
>>> - Fix indent of 'if' statement.
>>> - Change function name.
>>>
> [...]
>
>>> +/* attach the new virtual device, then store port_id of the device */
>>> +static int
>>> +rte_eal_dev_attach_vdev(const char *vdevargs, uint8_t *port_id)
>>> +{
>>> +	char *args;
>>> +	uint8_t new_port_id;
>>> +	struct rte_eth_dev devs[RTE_MAX_ETHPORTS];
>>> +
>>> +	if ((vdevargs == NULL) || (port_id == NULL))
>>> +		goto err0;
>>> +
>>> +	args = strdup(vdevargs);
>>> +	if (args == NULL)
>>> +		goto err0;
>>> +
>>> +	/* save current port status */
>>> +	rte_eth_dev_save(devs);
>>> +	/* add the vdevargs to devargs_list */
>>> +	if (rte_eal_devargs_add(RTE_DEVTYPE_VIRTUAL, args))
>>> +		goto err1;
>>> +	/* parse vdevargs, then retrieve device name */
>>> +	get_vdev_name(args);
>>> +	/* walk around dev_driver_list to find the driver of the device,
>>> +	 * then invoke probe function o the driver */
>>> +	if (rte_eal_vdev_find_and_invoke(args, RTE_EAL_INVOKE_TYPE_PROBE))
>>> +		goto err2;
>>> +	/* get port_id enabled by above procedures */
>>> +	if (rte_eth_dev_get_changed_port(devs, &new_port_id))
>>> +		goto err2;
>>> +
>>> +	free(args);
>>> +	*port_id = new_port_id;
>>> +	return 0;
>>> +err2:
>>> +	rte_eal_devargs_remove(RTE_DEVTYPE_VIRTUAL, args);
>>> +err1:
>>> +	free(args);
>>> +err0:
>>> +	RTE_LOG(ERR, EAL, "Drver, cannot detach the device\n");
> Here "cannot detach the device\n" should be "cannot attach the device" I
> think.

Hi Michael,

Thanks, I will fix above error message.
Also I will fix my "Drver" typos.

Tetsuya

>> Here also "Drver",
>>
>>
>> Thanks,
>> Michael
>>> +	return -1;
>>> +}
>>> +
>>> +/* detach the new virtual device, then store the name of the device */
>>> +static int
>>> +rte_eal_dev_detach_vdev(uint8_t port_id, char *vdevname)
>>> +{
>>> +	char name[RTE_ETH_NAME_MAX_LEN];
>>> +
>>> +	if (vdevname == NULL)
>>> +		goto err;
>>> +
>>> +	/* check whether the driver supports detach feature, or not */
>>> +	if (rte_eth_dev_check_detachable(port_id))
>>> +		goto err;
>>> +
>>> +	/* get device name by port id */
>>> +	if (rte_eth_dev_get_name_by_port(port_id, name))
>>> +		goto err;
>>> +	/* walk around dev_driver_list to find the driver of the device,
>>> +	 * then invoke close function o the driver */
>>> +	if (rte_eal_vdev_find_and_invoke(name, RTE_EAL_INVOKE_TYPE_CLOSE))
>>> +		goto err;
>>> +	/* remove the vdevname from devargs_list */
>>> +	if (rte_eal_devargs_remove(RTE_DEVTYPE_VIRTUAL, name))
>>> +		goto err;
>>> +
>>> +	strncpy(vdevname, name, sizeof(name));
>>> +	return 0;
>>> +err:
>>> +	RTE_LOG(ERR, EAL, "Drver, cannot detach the device\n");
>>> +	return -1;
>>> +}
>>> +
>>> +/* attach the new device, then store port_id of the device */
>>> +int
>>> +rte_eal_dev_attach(const char *devargs, uint8_t *port_id)
>>> +{
>>> +	struct rte_pci_addr addr;
>>> +
>>> +	if ((devargs == NULL) || (port_id == NULL))
>>> +		return -EINVAL;
>>> +
>>> +	if (eal_parse_pci_DomBDF(devargs, &addr) == 0)
>>> +		return rte_eal_dev_attach_pdev(&addr, port_id);
>>> +	else
>>> +		return rte_eal_dev_attach_vdev(devargs, port_id);
>>> +}
>>> +
>>> +/* detach the device, then store the name of the device */
>>> +int
>>> +rte_eal_dev_detach(uint8_t port_id, char *name)
>>> +{
>>> +	struct rte_pci_addr addr;
>>> +	int ret;
>>> +
>>> +	if (name == NULL)
>>> +		return -EINVAL;
>>> +
>>> +	if (rte_eth_dev_get_device_type(port_id) == RTE_ETH_DEV_PHYSICAL) {
>>> +		ret = rte_eth_dev_get_addr_by_port(port_id, &addr);
>>> +		if (ret < 0)
>>> +			return ret;
>>> +
>>> +		ret = rte_eal_dev_detach_pdev(port_id, &addr);
>>> +		if (ret == 0)
>>> +			snprintf(name, RTE_ETH_NAME_MAX_LEN,
>>> +				"%04x:%02x:%02x.%d",
>>> +				addr.domain, addr.bus,
>>> +				addr.devid, addr.function);
>>> +
>>> +		return ret;
>>> +	} else
>>> +		return rte_eal_dev_detach_vdev(port_id, name);
>>> +}
>>> +#else /* ENABLE_HOTPLUG */
>>> +int
>>> +rte_eal_dev_attach(const char *devargs __rte_unused,
>>> +			uint8_t *port_id __rte_unused)
>>> +{
>>> +	RTE_LOG(ERR, EAL, "Hotplug support isn't enabled\n");
>>> +	return -1;
>>> +}
>>> +
>>> +/* detach the device, then store the name of the device */
>>> +int
>>> +rte_eal_dev_detach(uint8_t port_id __rte_unused,
>>> +			char *name __rte_unused)
>>> +{
>>> +	RTE_LOG(ERR, EAL, "Hotplug support isn't enabled\n");
>>> +	return -1;
>>> +}
>>> +#endif /* ENABLE_HOTPLUG */
>>> diff --git a/lib/librte_eal/common/eal_private.h b/lib/librte_eal/common/eal_private.h
>>> index 1a362ab..8168a7a 100644
>>> --- a/lib/librte_eal/common/eal_private.h
>>> +++ b/lib/librte_eal/common/eal_private.h
>>> @@ -163,6 +163,17 @@ enum rte_eal_invoke_type {
>>>  };
>>>  
>>>  /**
>>> + * Scan the content of the PCI bus, and the devices in the devices
>>> + * list
>>> + *
>>> + * This function is private to EAL.
>>> + *
>>> + * @return
>>> + *  0 on success, negative on error
>>> + */
>>> +int rte_eal_pci_scan(void);
>>> +
>>> +/**
>>>   * Mmap memory for single PCI device
>>>   *
>>>   * This function is private to EAL.
>>> diff --git a/lib/librte_eal/common/include/rte_dev.h b/lib/librte_eal/common/include/rte_dev.h
>>> index f7e3a10..e63dd1c 100644
>>> --- a/lib/librte_eal/common/include/rte_dev.h
>>> +++ b/lib/librte_eal/common/include/rte_dev.h
>>> @@ -47,6 +47,7 @@ extern "C" {
>>>  #endif
>>>  
>>>  #include <sys/queue.h>
>>> +#include <rte_pci.h>