This patch series adds a dynamic port hotplug framework to DPDK. With the patches, DPDK apps can attach or detach ports at runtime. The basic concept of the port hotplug is like followings. - DPDK apps must have responsibility to manage ports. DPDK apps only know which ports are attached or detached at the moment. The port hotplug framework is implemented to allow DPDK apps to manage ports. For example, when DPDK apps call port attach function, attached port number will be returned. Also DPDK apps can detach port by port number. - Kernel support is needed for attaching or detaching physical device ports. To attach new device, the device will be recognized by kernel at first and controlled by kernel driver. Then user can bind the device to igb_uio by 'dpdk_nic_bind.py'. Finally, DPDK apps can call the port hotplug functions to attach ports. For detaching, steps are vice versa. - Before detach ports, ports must be stopped and closed. DPDK application must call rte_eth_dev_stop() and rte_eth_dev_close() before detaching ports. These function will call finalization codes of PMDs. But so far, no PMD frees all resources allocated by initialization. It means PMDs are needed to be fixed to support the port hotplug. 'RTE_PCI_DRV_DETACHABLE' is a new flag indicating a PMD supports detaching. Without this flag, detaching will be failed. - Mustn't affect legacy DPDK apps. No DPDK EAL behavior is changed, if the port hotplug functions are't called. So all legacy DPDK apps can still work without modifications. And a few limitations. - The port hotplug functions are not thread safe. DPDK apps should handle it. - Only support Linux and igb_uio so far. BSD and VFIO is not supported. I will send VFIO patches at least, but I don't have a plan to submit BSD patch so far. Here is port hotplug APIs. ------------------------------------------------------------------------------- /** * Attach a new device. * * @param devargs * A pointer to a strings array describing the new device * to be attached. The strings should be a pci address like * '0000:01:00.0' or virtual device name like 'eth_pcap0'. * @param port_id * A pointer to a port identifier actually attached. * @return * 0 on success and port_id is filled, negative on error */ int rte_eal_dev_attach(const char *devargs, uint8_t *port_id); /** * Detach a device. * * @param port_id * The port identifier of the device to detach. * @param addr * A pointer to a device name actually detached. * @return * 0 on success and devname is filled, negative on error */ int rte_eal_dev_detach(uint8_t port_id, char *devname); ------------------------------------------------------------------------------- This patch series are for DPDK EAL. To use port hotplug function by DPDK apps, each PMD should be fixed to support 'RTE_PCI_DRV_DETACHABLE' flag. Please check a patch for pcap PMD. Also please check testpmd patch. It will show you how to fix your legacy applications to support port hotplug feature. PATCH v4 changes - Merge patches to review easier. - Fix indent of 'if' statement. - Fix calculation method of eal_compare_pci_addr(). - Fix header file declaration. - Add header file to determine if hotplug can be enabled. (Thanks to Qiu, Michael) - Use braces with 'for' loop. - Add paramerter checking. - Fix sanity check code - Fix comments of rte_eth_dev_type. - Change function names. (Thanks to Iremonger, Bernard) PATCH v3 changes: - Fix enum definition used in rte_ethdev.c. (Thanks to Zhang, Helin) PATCH v2 changes: - Replace rte_eal_dev_attach_pdev(), rte_eal_dev_detach_pdev, rte_eal_dev_attach_vdev() and rte_eal_dev_detach_vdev() to rte_eal_dev_attach() and rte_eal_dev_detach(). - Add parameter values checking. - Refashion a few functions. (Thanks to Iremonger, Bernard) PATCH v1 Changes: - Fix error checking code of librte_eth APIs. - Fix issue that port from pcap PMD cannot be detached correctly. - Fix issue that testpmd could hang after forwarding, if attaching and detaching is repeatedly. - Fix if-condition of rte_eth_dev_get_port_by_addr(). (Thanks to Mark Enright) RFC PATCH v2 Changes: - remove 'rte_eth_dev_validate_port()', and cleanup codes. Tetsuya Mukawa (11): eal/pci,ethdev: Remove assumption that port will not be detached eal/pci: Consolidate pci address comparison APIs ethdev: Add rte_eth_dev_free to free specified device eal,ethdev: Add a function and function pointers to close ether device ethdev: Add functions that will be used by port hotplug functions eal/linux/pci: Add functions for unmapping igb_uio resources eal/pci: Add a function to remove the entry of devargs list eal/pci: Cleanup pci driver initialization code ethdev: Add one dev_type paramerter to rte_eth_dev_allocate eal/pci: Add rte_eal_dev_attach/detach() functions eal: Enable port hotplug framework in Linux app/test/virtual_pmd.c | 2 +- config/common_linuxapp | 5 + lib/librte_eal/bsdapp/eal/eal_pci.c | 16 +- lib/librte_eal/common/Makefile | 1 + lib/librte_eal/common/eal_common_dev.c | 273 +++++++++++++++ lib/librte_eal/common/eal_common_devargs.c | 57 ++++ lib/librte_eal/common/eal_common_pci.c | 92 ++++- lib/librte_eal/common/eal_private.h | 36 ++ lib/librte_eal/common/include/rte_dev.h | 33 ++ lib/librte_eal/common/include/rte_dev_hotplug.h | 44 +++ lib/librte_eal/common/include/rte_devargs.h | 18 + lib/librte_eal/common/include/rte_pci.h | 76 +++++ lib/librte_eal/linuxapp/eal/Makefile | 1 + lib/librte_eal/linuxapp/eal/eal_pci.c | 136 ++++++-- lib/librte_eal/linuxapp/eal/eal_pci_init.h | 8 + lib/librte_eal/linuxapp/eal/eal_pci_uio.c | 67 +++- lib/librte_ether/rte_ethdev.c | 424 ++++++++++++++++++------ lib/librte_ether/rte_ethdev.h | 145 +++++++- lib/librte_pmd_af_packet/rte_eth_af_packet.c | 2 +- lib/librte_pmd_bond/rte_eth_bond_api.c | 2 +- lib/librte_pmd_pcap/rte_eth_pcap.c | 2 +- lib/librte_pmd_ring/rte_eth_ring.c | 2 +- lib/librte_pmd_xenvirt/rte_eth_xenvirt.c | 2 +- 23 files changed, 1283 insertions(+), 161 deletions(-) create mode 100644 lib/librte_eal/common/include/rte_dev_hotplug.h -- 1.9.1
To remove assumption, do like followings. This patch adds "RTE_PCI_DRV_DETACHABLE" to drv_flags of rte_pci_driver structure. The flags indicates the driver can detach devices at runtime. Also remove assumption that port will not be detached. To remove the assumption. - Add 'attached' member to rte_eth_dev structure. This member is used for indicating the port is attached, or not. - Add rte_eth_dev_allocate_new_port(). This function is used for allocating new port. v4: - Use braces with 'for' loop. - Fix indent of 'if' statement. Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp> --- lib/librte_eal/common/include/rte_pci.h | 2 + lib/librte_ether/rte_ethdev.c | 248 ++++++++++++++++++-------------- lib/librte_ether/rte_ethdev.h | 5 + 3 files changed, 151 insertions(+), 104 deletions(-) diff --git a/lib/librte_eal/common/include/rte_pci.h b/lib/librte_eal/common/include/rte_pci.h index 66ed793..dd1da28 100644 --- a/lib/librte_eal/common/include/rte_pci.h +++ b/lib/librte_eal/common/include/rte_pci.h @@ -199,6 +199,8 @@ struct rte_pci_driver { #define RTE_PCI_DRV_FORCE_UNBIND 0x0004 /** Device driver supports link state interrupt */ #define RTE_PCI_DRV_INTR_LSC 0x0008 +/** Device driver supports detaching capability */ +#define RTE_PCI_DRV_DETACHABLE 0x0010 /**< Internal use only - Macro used by pci addr parsing functions **/ #define GET_PCIADDR_FIELD(in, fd, lim, dlm) \ diff --git a/lib/librte_ether/rte_ethdev.c b/lib/librte_ether/rte_ethdev.c index 077d430..46cabaf 100644 --- a/lib/librte_ether/rte_ethdev.c +++ b/lib/librte_ether/rte_ethdev.c @@ -175,6 +175,16 @@ enum { STAT_QMAP_RX }; +enum { + DEV_INVALID = 0, + DEV_VALID, +}; + +enum { + DEV_DISCONNECTED = 0, + DEV_CONNECTED +}; + static inline void rte_eth_dev_data_alloc(void) { @@ -201,19 +211,34 @@ rte_eth_dev_allocated(const char *name) { unsigned i; - for (i = 0; i < nb_ports; i++) { - if (strcmp(rte_eth_devices[i].data->name, name) == 0) + for (i = 0; i < RTE_MAX_ETHPORTS; i++) { + if ((rte_eth_devices[i].attached == DEV_CONNECTED) && + strcmp(rte_eth_devices[i].data->name, name) == 0) return &rte_eth_devices[i]; } return NULL; } +static uint8_t +rte_eth_dev_allocate_new_port(void) +{ + unsigned i; + + for (i = 0; i < RTE_MAX_ETHPORTS; i++) { + if (rte_eth_devices[i].attached == DEV_DISCONNECTED) + return i; + } + return RTE_MAX_ETHPORTS; +} + struct rte_eth_dev * rte_eth_dev_allocate(const char *name) { + uint8_t port_id; struct rte_eth_dev *eth_dev; - if (nb_ports == RTE_MAX_ETHPORTS) { + port_id = rte_eth_dev_allocate_new_port(); + if (port_id == RTE_MAX_ETHPORTS) { PMD_DEBUG_TRACE("Reached maximum number of Ethernet ports\n"); return NULL; } @@ -226,10 +251,12 @@ rte_eth_dev_allocate(const char *name) return NULL; } - eth_dev = &rte_eth_devices[nb_ports]; - eth_dev->data = &rte_eth_dev_data[nb_ports]; + eth_dev = &rte_eth_devices[port_id]; + eth_dev->data = &rte_eth_dev_data[port_id]; snprintf(eth_dev->data->name, sizeof(eth_dev->data->name), "%s", name); - eth_dev->data->port_id = nb_ports++; + eth_dev->data->port_id = port_id; + eth_dev->attached = DEV_CONNECTED; + nb_ports++; return eth_dev; } @@ -283,6 +310,7 @@ rte_eth_dev_init(struct rte_pci_driver *pci_drv, (unsigned) pci_dev->id.device_id); if (rte_eal_process_type() == RTE_PROC_PRIMARY) rte_free(eth_dev->data->dev_private); + eth_dev->attached = DEV_DISCONNECTED; nb_ports--; return diag; } @@ -308,10 +336,22 @@ rte_eth_driver_register(struct eth_driver *eth_drv) rte_eal_pci_register(ð_drv->pci_drv); } +static int +rte_eth_dev_validate_port(uint8_t port_id) +{ + if (port_id >= RTE_MAX_ETHPORTS) + return DEV_INVALID; + + if (rte_eth_devices[port_id].attached == DEV_CONNECTED) + return DEV_VALID; + else + return DEV_INVALID; +} + int rte_eth_dev_socket_id(uint8_t port_id) { - if (port_id >= nb_ports) + if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) return -1; return rte_eth_devices[port_id].pci_dev->numa_node; } @@ -369,7 +409,7 @@ rte_eth_dev_rx_queue_start(uint8_t port_id, uint16_t rx_queue_id) * in a multi-process setup*/ PROC_PRIMARY_OR_ERR_RET(-E_RTE_SECONDARY); - if (port_id >= nb_ports) { + if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return -EINVAL; } @@ -395,7 +435,7 @@ rte_eth_dev_rx_queue_stop(uint8_t port_id, uint16_t rx_queue_id) * in a multi-process setup*/ PROC_PRIMARY_OR_ERR_RET(-E_RTE_SECONDARY); - if (port_id >= nb_ports) { + if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return -EINVAL; } @@ -421,7 +461,7 @@ rte_eth_dev_tx_queue_start(uint8_t port_id, uint16_t tx_queue_id) * in a multi-process setup*/ PROC_PRIMARY_OR_ERR_RET(-E_RTE_SECONDARY); - if (port_id >= nb_ports) { + if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return -EINVAL; } @@ -447,7 +487,7 @@ rte_eth_dev_tx_queue_stop(uint8_t port_id, uint16_t tx_queue_id) * in a multi-process setup*/ PROC_PRIMARY_OR_ERR_RET(-E_RTE_SECONDARY); - if (port_id >= nb_ports) { + if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return -EINVAL; } @@ -703,7 +743,7 @@ rte_eth_dev_configure(uint8_t port_id, uint16_t nb_rx_q, uint16_t nb_tx_q, * in a multi-process setup*/ PROC_PRIMARY_OR_ERR_RET(-E_RTE_SECONDARY); - if (port_id >= nb_ports || port_id >= RTE_MAX_ETHPORTS) { + if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-EINVAL); } @@ -888,7 +928,7 @@ rte_eth_dev_start(uint8_t port_id) * in a multi-process setup*/ PROC_PRIMARY_OR_ERR_RET(-E_RTE_SECONDARY); - if (port_id >= nb_ports) { + if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) { PMD_DEBUG_TRACE("Invalid port_id=%" PRIu8 "\n", port_id); return (-EINVAL); } @@ -923,7 +963,7 @@ rte_eth_dev_stop(uint8_t port_id) * in a multi-process setup*/ PROC_PRIMARY_OR_RET(); - if (port_id >= nb_ports) { + if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) { PMD_DEBUG_TRACE("Invalid port_id=%" PRIu8 "\n", port_id); return; } @@ -951,7 +991,7 @@ rte_eth_dev_set_link_up(uint8_t port_id) * in a multi-process setup*/ PROC_PRIMARY_OR_ERR_RET(-E_RTE_SECONDARY); - if (port_id >= nb_ports) { + if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return -EINVAL; } @@ -970,7 +1010,7 @@ rte_eth_dev_set_link_down(uint8_t port_id) * in a multi-process setup*/ PROC_PRIMARY_OR_ERR_RET(-E_RTE_SECONDARY); - if (port_id >= nb_ports) { + if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return -EINVAL; } @@ -989,7 +1029,7 @@ rte_eth_dev_close(uint8_t port_id) * in a multi-process setup*/ PROC_PRIMARY_OR_RET(); - if (port_id >= nb_ports) { + if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return; } @@ -1017,7 +1057,7 @@ rte_eth_rx_queue_setup(uint8_t port_id, uint16_t rx_queue_id, * in a multi-process setup*/ PROC_PRIMARY_OR_ERR_RET(-E_RTE_SECONDARY); - if (port_id >= nb_ports) { + if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-EINVAL); } @@ -1090,7 +1130,7 @@ rte_eth_tx_queue_setup(uint8_t port_id, uint16_t tx_queue_id, * in a multi-process setup*/ PROC_PRIMARY_OR_ERR_RET(-E_RTE_SECONDARY); - if (port_id >= RTE_MAX_ETHPORTS || port_id >= nb_ports) { + if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-EINVAL); } @@ -1123,7 +1163,7 @@ rte_eth_promiscuous_enable(uint8_t port_id) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return; } @@ -1139,7 +1179,7 @@ rte_eth_promiscuous_disable(uint8_t port_id) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return; } @@ -1155,7 +1195,7 @@ rte_eth_promiscuous_get(uint8_t port_id) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return -1; } @@ -1169,7 +1209,7 @@ rte_eth_allmulticast_enable(uint8_t port_id) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return; } @@ -1185,7 +1225,7 @@ rte_eth_allmulticast_disable(uint8_t port_id) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return; } @@ -1201,7 +1241,7 @@ rte_eth_allmulticast_get(uint8_t port_id) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return -1; } @@ -1229,7 +1269,7 @@ rte_eth_link_get(uint8_t port_id, struct rte_eth_link *eth_link) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return; } @@ -1249,7 +1289,7 @@ rte_eth_link_get_nowait(uint8_t port_id, struct rte_eth_link *eth_link) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return; } @@ -1269,7 +1309,7 @@ rte_eth_stats_get(uint8_t port_id, struct rte_eth_stats *stats) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return; } @@ -1286,7 +1326,7 @@ rte_eth_stats_reset(uint8_t port_id) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return; } @@ -1307,7 +1347,7 @@ rte_eth_xstats_get(uint8_t port_id, struct rte_eth_xstats *xstats, uint64_t val; char *stats_ptr; - if (port_id >= nb_ports) { + if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return -1; } @@ -1376,7 +1416,7 @@ rte_eth_xstats_reset(uint8_t port_id) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return; } @@ -1398,7 +1438,7 @@ set_queue_stats_mapping(uint8_t port_id, uint16_t queue_id, uint8_t stat_idx, { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return -ENODEV; } @@ -1433,7 +1473,7 @@ rte_eth_dev_info_get(uint8_t port_id, struct rte_eth_dev_info *dev_info) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return; } @@ -1453,7 +1493,7 @@ rte_eth_macaddr_get(uint8_t port_id, struct ether_addr *mac_addr) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return; } @@ -1467,7 +1507,7 @@ rte_eth_dev_get_mtu(uint8_t port_id, uint16_t *mtu) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-ENODEV); } @@ -1483,7 +1523,7 @@ rte_eth_dev_set_mtu(uint8_t port_id, uint16_t mtu) int ret; struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-ENODEV); } @@ -1503,7 +1543,7 @@ rte_eth_dev_vlan_filter(uint8_t port_id, uint16_t vlan_id, int on) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-ENODEV); } @@ -1528,7 +1568,7 @@ rte_eth_dev_set_vlan_strip_on_queue(uint8_t port_id, uint16_t rx_queue_id, int o { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-ENODEV); } @@ -1550,7 +1590,7 @@ rte_eth_dev_set_vlan_ether_type(uint8_t port_id, uint16_t tpid) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-ENODEV); } @@ -1570,7 +1610,7 @@ rte_eth_dev_set_vlan_offload(uint8_t port_id, int offload_mask) int mask = 0; int cur, org = 0; - if (port_id >= nb_ports) { + if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-ENODEV); } @@ -1615,7 +1655,7 @@ rte_eth_dev_get_vlan_offload(uint8_t port_id) struct rte_eth_dev *dev; int ret = 0; - if (port_id >= nb_ports) { + if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-ENODEV); } @@ -1639,7 +1679,7 @@ rte_eth_dev_set_vlan_pvid(uint8_t port_id, uint16_t pvid, int on) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-ENODEV); } @@ -1657,7 +1697,7 @@ rte_eth_dev_fdir_add_signature_filter(uint8_t port_id, { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-ENODEV); } @@ -1691,7 +1731,7 @@ rte_eth_dev_fdir_update_signature_filter(uint8_t port_id, { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-ENODEV); } @@ -1725,7 +1765,7 @@ rte_eth_dev_fdir_remove_signature_filter(uint8_t port_id, { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-ENODEV); } @@ -1756,7 +1796,7 @@ rte_eth_dev_fdir_get_infos(uint8_t port_id, struct rte_eth_fdir *fdir) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-ENODEV); } @@ -1781,7 +1821,7 @@ rte_eth_dev_fdir_add_perfect_filter(uint8_t port_id, { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-ENODEV); } @@ -1821,7 +1861,7 @@ rte_eth_dev_fdir_update_perfect_filter(uint8_t port_id, { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-ENODEV); } @@ -1859,7 +1899,7 @@ rte_eth_dev_fdir_remove_perfect_filter(uint8_t port_id, { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-ENODEV); } @@ -1895,7 +1935,7 @@ rte_eth_dev_fdir_set_masks(uint8_t port_id, struct rte_fdir_masks *fdir_mask) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-ENODEV); } @@ -1915,7 +1955,7 @@ rte_eth_dev_flow_ctrl_get(uint8_t port_id, struct rte_eth_fc_conf *fc_conf) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-ENODEV); } @@ -1931,7 +1971,7 @@ rte_eth_dev_flow_ctrl_set(uint8_t port_id, struct rte_eth_fc_conf *fc_conf) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-ENODEV); } @@ -1951,7 +1991,7 @@ rte_eth_dev_priority_flow_ctrl_set(uint8_t port_id, struct rte_eth_pfc_conf *pfc { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-ENODEV); } @@ -2030,7 +2070,7 @@ rte_eth_dev_rss_reta_update(uint8_t port_id, struct rte_eth_dev *dev; int ret; - if (port_id >= nb_ports) { + if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return -ENODEV; } @@ -2081,7 +2121,7 @@ rte_eth_dev_rss_hash_update(uint8_t port_id, struct rte_eth_rss_conf *rss_conf) struct rte_eth_dev *dev; uint16_t rss_hash_protos; - if (port_id >= nb_ports) { + if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-ENODEV); } @@ -2103,7 +2143,7 @@ rte_eth_dev_rss_hash_conf_get(uint8_t port_id, { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-ENODEV); } @@ -2118,7 +2158,7 @@ rte_eth_dev_udp_tunnel_add(uint8_t port_id, { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return -ENODEV; } @@ -2144,7 +2184,7 @@ rte_eth_dev_udp_tunnel_delete(uint8_t port_id, { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return -ENODEV; } @@ -2169,7 +2209,7 @@ rte_eth_led_on(uint8_t port_id) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-ENODEV); } @@ -2184,7 +2224,7 @@ rte_eth_led_off(uint8_t port_id) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-ENODEV); } @@ -2224,7 +2264,7 @@ rte_eth_dev_mac_addr_add(uint8_t port_id, struct ether_addr *addr, int index; uint64_t pool_mask; - if (port_id >= nb_ports) { + if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-ENODEV); } @@ -2275,7 +2315,7 @@ rte_eth_dev_mac_addr_remove(uint8_t port_id, struct ether_addr *addr) struct rte_eth_dev *dev; int index; - if (port_id >= nb_ports) { + if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-ENODEV); } @@ -2309,7 +2349,7 @@ rte_eth_dev_set_vf_rxmode(uint8_t port_id, uint16_t vf, struct rte_eth_dev *dev; struct rte_eth_dev_info dev_info; - if (port_id >= nb_ports) { + if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) { PMD_DEBUG_TRACE("set VF RX mode:Invalid port_id=%d\n", port_id); return (-ENODEV); @@ -2364,7 +2404,7 @@ rte_eth_dev_uc_hash_table_set(uint8_t port_id, struct ether_addr *addr, int ret; struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) { PMD_DEBUG_TRACE("unicast hash setting:Invalid port_id=%d\n", port_id); return (-ENODEV); @@ -2417,7 +2457,7 @@ rte_eth_dev_uc_all_hash_table_set(uint8_t port_id, uint8_t on) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) { PMD_DEBUG_TRACE("unicast hash setting:Invalid port_id=%d\n", port_id); return (-ENODEV); @@ -2436,7 +2476,7 @@ rte_eth_dev_set_vf_rx(uint8_t port_id,uint16_t vf, uint8_t on) struct rte_eth_dev *dev; struct rte_eth_dev_info dev_info; - if (port_id >= nb_ports) { + if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-ENODEV); } @@ -2462,7 +2502,7 @@ rte_eth_dev_set_vf_tx(uint8_t port_id,uint16_t vf, uint8_t on) struct rte_eth_dev *dev; struct rte_eth_dev_info dev_info; - if (port_id >= nb_ports) { + if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) { PMD_DEBUG_TRACE("set pool tx:Invalid port_id=%d\n", port_id); return (-ENODEV); } @@ -2487,7 +2527,7 @@ rte_eth_dev_set_vf_vlan_filter(uint8_t port_id, uint16_t vlan_id, { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) { PMD_DEBUG_TRACE("VF VLAN filter:invalid port id=%d\n", port_id); return (-ENODEV); @@ -2518,7 +2558,7 @@ int rte_eth_set_queue_rate_limit(uint8_t port_id, uint16_t queue_idx, struct rte_eth_dev_info dev_info; struct rte_eth_link link; - if (port_id >= nb_ports) { + if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) { PMD_DEBUG_TRACE("set queue rate limit:invalid port id=%d\n", port_id); return -ENODEV; @@ -2555,7 +2595,7 @@ int rte_eth_set_vf_rate_limit(uint8_t port_id, uint16_t vf, uint16_t tx_rate, if (q_msk == 0) return 0; - if (port_id >= nb_ports) { + if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) { PMD_DEBUG_TRACE("set VF rate limit:invalid port id=%d\n", port_id); return -ENODEV; @@ -2589,7 +2629,7 @@ rte_eth_mirror_rule_set(uint8_t port_id, { struct rte_eth_dev *dev = &rte_eth_devices[port_id]; - if (port_id >= nb_ports) { + if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-ENODEV); } @@ -2630,7 +2670,7 @@ rte_eth_mirror_rule_reset(uint8_t port_id, uint8_t rule_id) { struct rte_eth_dev *dev = &rte_eth_devices[port_id]; - if (port_id >= nb_ports) { + if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-ENODEV); } @@ -2655,7 +2695,7 @@ rte_eth_rx_burst(uint8_t port_id, uint16_t queue_id, { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return 0; } @@ -2675,7 +2715,7 @@ rte_eth_tx_burst(uint8_t port_id, uint16_t queue_id, { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return 0; } @@ -2695,7 +2735,7 @@ rte_eth_rx_queue_count(uint8_t port_id, uint16_t queue_id) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return 0; } @@ -2709,7 +2749,7 @@ rte_eth_rx_descriptor_done(uint8_t port_id, uint16_t queue_id, uint16_t offset) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-ENODEV); } @@ -2730,7 +2770,7 @@ rte_eth_dev_callback_register(uint8_t port_id, if (!cb_fn) return (-EINVAL); - if (port_id >= nb_ports) { + if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-EINVAL); } @@ -2770,7 +2810,7 @@ rte_eth_dev_callback_unregister(uint8_t port_id, if (!cb_fn) return (-EINVAL); - if (port_id >= nb_ports) { + if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-EINVAL); } @@ -2830,7 +2870,7 @@ int rte_eth_dev_bypass_init(uint8_t port_id) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-ENODEV); } @@ -2850,7 +2890,7 @@ rte_eth_dev_bypass_state_show(uint8_t port_id, uint32_t *state) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-ENODEV); } @@ -2869,7 +2909,7 @@ rte_eth_dev_bypass_state_set(uint8_t port_id, uint32_t *new_state) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-ENODEV); } @@ -2889,7 +2929,7 @@ rte_eth_dev_bypass_event_show(uint8_t port_id, uint32_t event, uint32_t *state) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-ENODEV); } @@ -2909,7 +2949,7 @@ rte_eth_dev_bypass_event_store(uint8_t port_id, uint32_t event, uint32_t state) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-ENODEV); } @@ -2929,7 +2969,7 @@ rte_eth_dev_wd_timeout_store(uint8_t port_id, uint32_t timeout) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-ENODEV); } @@ -2949,7 +2989,7 @@ rte_eth_dev_bypass_ver_show(uint8_t port_id, uint32_t *ver) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-ENODEV); } @@ -2969,7 +3009,7 @@ rte_eth_dev_bypass_wd_timeout_show(uint8_t port_id, uint32_t *wd_timeout) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-ENODEV); } @@ -2989,7 +3029,7 @@ rte_eth_dev_bypass_wd_reset(uint8_t port_id) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-ENODEV); } @@ -3011,7 +3051,7 @@ rte_eth_dev_add_syn_filter(uint8_t port_id, { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return -ENODEV; } @@ -3026,7 +3066,7 @@ rte_eth_dev_remove_syn_filter(uint8_t port_id) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return -ENODEV; } @@ -3045,7 +3085,7 @@ rte_eth_dev_get_syn_filter(uint8_t port_id, if (filter == NULL || rx_queue == NULL) return -EINVAL; - if (port_id >= nb_ports) { + if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return -ENODEV; } @@ -3061,7 +3101,7 @@ rte_eth_dev_add_ethertype_filter(uint8_t port_id, uint16_t index, { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return -ENODEV; } @@ -3082,7 +3122,7 @@ rte_eth_dev_remove_ethertype_filter(uint8_t port_id, uint16_t index) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return -ENODEV; } @@ -3101,7 +3141,7 @@ rte_eth_dev_get_ethertype_filter(uint8_t port_id, uint16_t index, if (filter == NULL || rx_queue == NULL) return -EINVAL; - if (port_id >= nb_ports) { + if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return -ENODEV; } @@ -3118,7 +3158,7 @@ rte_eth_dev_add_2tuple_filter(uint8_t port_id, uint16_t index, { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return -ENODEV; } @@ -3140,7 +3180,7 @@ rte_eth_dev_remove_2tuple_filter(uint8_t port_id, uint16_t index) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return -ENODEV; } @@ -3159,7 +3199,7 @@ rte_eth_dev_get_2tuple_filter(uint8_t port_id, uint16_t index, if (filter == NULL || rx_queue == NULL) return -EINVAL; - if (port_id >= nb_ports) { + if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return -ENODEV; } @@ -3175,7 +3215,7 @@ rte_eth_dev_add_5tuple_filter(uint8_t port_id, uint16_t index, { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return -ENODEV; } @@ -3198,7 +3238,7 @@ rte_eth_dev_remove_5tuple_filter(uint8_t port_id, uint16_t index) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return -ENODEV; } @@ -3217,7 +3257,7 @@ rte_eth_dev_get_5tuple_filter(uint8_t port_id, uint16_t index, if (filter == NULL || rx_queue == NULL) return -EINVAL; - if (port_id >= nb_ports) { + if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return -ENODEV; } @@ -3234,7 +3274,7 @@ rte_eth_dev_add_flex_filter(uint8_t port_id, uint16_t index, { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return -ENODEV; } @@ -3249,7 +3289,7 @@ rte_eth_dev_remove_flex_filter(uint8_t port_id, uint16_t index) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return -ENODEV; } @@ -3268,7 +3308,7 @@ rte_eth_dev_get_flex_filter(uint8_t port_id, uint16_t index, if (filter == NULL || rx_queue == NULL) return -EINVAL; - if (port_id >= nb_ports) { + if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return -ENODEV; } @@ -3284,7 +3324,7 @@ rte_eth_dev_filter_supported(uint8_t port_id, enum rte_filter_type filter_type) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return -ENODEV; } @@ -3301,7 +3341,7 @@ rte_eth_dev_filter_ctrl(uint8_t port_id, enum rte_filter_type filter_type, { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return -ENODEV; } diff --git a/lib/librte_ether/rte_ethdev.h b/lib/librte_ether/rte_ethdev.h index ce0528f..616a44a 100644 --- a/lib/librte_ether/rte_ethdev.h +++ b/lib/librte_ether/rte_ethdev.h @@ -1565,6 +1565,7 @@ struct rte_eth_dev { struct eth_dev_ops *dev_ops; /**< Functions exported by PMD */ struct rte_pci_device *pci_dev; /**< PCI info. supplied by probing */ struct rte_eth_dev_cb_list callbacks; /**< User application callbacks */ + uint8_t attached; /**< Flag indicating the port is attached */ }; struct rte_eth_dev_sriov { @@ -1630,6 +1631,10 @@ extern struct rte_eth_dev rte_eth_devices[]; * initialized by the [matching] Ethernet driver during the PCI probing phase. * All devices whose port identifier is in the range * [0, rte_eth_dev_count() - 1] can be operated on by network applications. + * immediately after invoking rte_eal_init(). + * If the application unplugs a port using hotplug function, The enabled port + * numbers may be noncontiguous. In the case, the applications need to manage + * enabled port by themselves. * * @return * - The total number of usable Ethernet devices. -- 1.9.1
This patch replaces pci_addr_comparison() and memcmp() of pci addresses by eal_compare_pci_addr(). v4: - Fix calculation method of eal_compare_pci_addr(). - Add parameter checking. Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp> --- lib/librte_eal/bsdapp/eal/eal_pci.c | 16 +-------------- lib/librte_eal/common/eal_common_pci.c | 2 +- lib/librte_eal/common/include/rte_pci.h | 34 +++++++++++++++++++++++++++++++ lib/librte_eal/linuxapp/eal/eal_pci.c | 23 ++++++--------------- lib/librte_eal/linuxapp/eal/eal_pci_uio.c | 2 +- 5 files changed, 43 insertions(+), 34 deletions(-) diff --git a/lib/librte_eal/bsdapp/eal/eal_pci.c b/lib/librte_eal/bsdapp/eal/eal_pci.c index 74ecce7..7eda513 100644 --- a/lib/librte_eal/bsdapp/eal/eal_pci.c +++ b/lib/librte_eal/bsdapp/eal/eal_pci.c @@ -270,20 +270,6 @@ pci_uio_map_resource(struct rte_pci_device *dev) return (0); } -/* Compare two PCI device addresses. */ -static int -pci_addr_comparison(struct rte_pci_addr *addr, struct rte_pci_addr *addr2) -{ - uint64_t dev_addr = (addr->domain << 24) + (addr->bus << 16) + (addr->devid << 8) + addr->function; - uint64_t dev_addr2 = (addr2->domain << 24) + (addr2->bus << 16) + (addr2->devid << 8) + addr2->function; - - if (dev_addr > dev_addr2) - return 1; - else - return 0; -} - - /* Scan one pci sysfs entry, and fill the devices list from it. */ static int pci_scan_one(int dev_pci_fd, struct pci_conf *conf) @@ -358,7 +344,7 @@ pci_scan_one(int dev_pci_fd, struct pci_conf *conf) struct rte_pci_device *dev2 = NULL; TAILQ_FOREACH(dev2, &pci_device_list, next) { - if (pci_addr_comparison(&dev->addr, &dev2->addr)) + if (eal_compare_pci_addr(&dev->addr, &dev2->addr)) continue; else { TAILQ_INSERT_BEFORE(dev2, dev, next); diff --git a/lib/librte_eal/common/eal_common_pci.c b/lib/librte_eal/common/eal_common_pci.c index f3c7f71..a89f5c3 100644 --- a/lib/librte_eal/common/eal_common_pci.c +++ b/lib/librte_eal/common/eal_common_pci.c @@ -93,7 +93,7 @@ static struct rte_devargs *pci_devargs_lookup(struct rte_pci_device *dev) if (devargs->type != RTE_DEVTYPE_BLACKLISTED_PCI && devargs->type != RTE_DEVTYPE_WHITELISTED_PCI) continue; - if (!memcmp(&dev->addr, &devargs->pci.addr, sizeof(dev->addr))) + if (!eal_compare_pci_addr(&dev->addr, &devargs->pci.addr)) return devargs; } return NULL; diff --git a/lib/librte_eal/common/include/rte_pci.h b/lib/librte_eal/common/include/rte_pci.h index dd1da28..128b2ce 100644 --- a/lib/librte_eal/common/include/rte_pci.h +++ b/lib/librte_eal/common/include/rte_pci.h @@ -261,6 +261,40 @@ eal_parse_pci_DomBDF(const char *input, struct rte_pci_addr *dev_addr) } #undef GET_PCIADDR_FIELD +/* Compare two PCI device addresses. */ +/** + * Utility function to compare two PCI device addresses. + * + * @param addr + * The PCI Bus-Device-Function address to compare + * @param addr2 + * The PCI Bus-Device-Function address to compare + * @return + * 0 on equal PCI address. + * Positive on addr is greater than addr2. + * Negative on addr is less than addr2, or error. + */ +static inline int +eal_compare_pci_addr(struct rte_pci_addr *addr, struct rte_pci_addr *addr2) +{ + uint64_t dev_addr, dev_addr2; + + if ((addr == NULL) || (addr2 == NULL)) + return -1; + + dev_addr = (addr->domain << 24) | (addr->bus << 16) | + (addr->devid << 8) | addr->function; + dev_addr2 = (addr2->domain << 24) | (addr2->bus << 16) | + (addr2->devid << 8) | addr2->function; + + if (dev_addr > dev_addr2) + return 1; + else if (dev_addr < dev_addr2) + return -1; + else + return 0; +} + /** * Probe the PCI bus for registered drivers. * diff --git a/lib/librte_eal/linuxapp/eal/eal_pci.c b/lib/librte_eal/linuxapp/eal/eal_pci.c index b5f5410..3d2d93c 100644 --- a/lib/librte_eal/linuxapp/eal/eal_pci.c +++ b/lib/librte_eal/linuxapp/eal/eal_pci.c @@ -200,20 +200,6 @@ error: return -1; } -/* Compare two PCI device addresses. */ -static int -pci_addr_comparison(struct rte_pci_addr *addr, struct rte_pci_addr *addr2) -{ - uint64_t dev_addr = (addr->domain << 24) + (addr->bus << 16) + (addr->devid << 8) + addr->function; - uint64_t dev_addr2 = (addr2->domain << 24) + (addr2->bus << 16) + (addr2->devid << 8) + addr2->function; - - if (dev_addr > dev_addr2) - return 1; - else - return 0; -} - - /* Scan one pci sysfs entry, and fill the devices list from it. */ static int pci_scan_one(const char *dirname, uint16_t domain, uint8_t bus, @@ -304,14 +290,17 @@ pci_scan_one(const char *dirname, uint16_t domain, uint8_t bus, } else { struct rte_pci_device *dev2 = NULL; + int ret; TAILQ_FOREACH(dev2, &pci_device_list, next) { - if (pci_addr_comparison(&dev->addr, &dev2->addr)) + ret = eal_compare_pci_addr(&dev->addr, &dev2->addr); + if (ret > 0) continue; - else { + else if (ret < 0) { TAILQ_INSERT_BEFORE(dev2, dev, next); return 0; - } + } else /* already registered */ + return 0; } TAILQ_INSERT_TAIL(&pci_device_list, dev, next); } diff --git a/lib/librte_eal/linuxapp/eal/eal_pci_uio.c b/lib/librte_eal/linuxapp/eal/eal_pci_uio.c index e53f06b..1da3507 100644 --- a/lib/librte_eal/linuxapp/eal/eal_pci_uio.c +++ b/lib/librte_eal/linuxapp/eal/eal_pci_uio.c @@ -123,7 +123,7 @@ pci_uio_map_secondary(struct rte_pci_device *dev) TAILQ_FOREACH(uio_res, pci_res_list, next) { /* skip this element if it doesn't match our PCI address */ - if (memcmp(&uio_res->pci_addr, &dev->addr, sizeof(dev->addr))) + if (eal_compare_pci_addr(&uio_res->pci_addr, &dev->addr)) continue; for (i = 0; i != uio_res->nb_maps; i++) { -- 1.9.1
This patch adds rte_eth_dev_free(). The function is used for changing a attached status of the device that has specified name. v4: - Add paramerter checking. Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp> --- lib/librte_ether/rte_ethdev.c | 20 ++++++++++++++++++++ lib/librte_ether/rte_ethdev.h | 11 +++++++++++ 2 files changed, 31 insertions(+) diff --git a/lib/librte_ether/rte_ethdev.c b/lib/librte_ether/rte_ethdev.c index 46cabaf..16f22c0 100644 --- a/lib/librte_ether/rte_ethdev.c +++ b/lib/librte_ether/rte_ethdev.c @@ -260,6 +260,26 @@ rte_eth_dev_allocate(const char *name) return eth_dev; } +struct rte_eth_dev * +rte_eth_dev_free(const char *name) +{ + struct rte_eth_dev *eth_dev; + + if (name == NULL) + return NULL; + + eth_dev = rte_eth_dev_allocated(name); + if (eth_dev == NULL) { + PMD_DEBUG_TRACE("Ethernet Device with name %s doesn't exist!\n", + name); + return NULL; + } + + eth_dev->attached = 0; + nb_ports--; + return eth_dev; +} + static int rte_eth_dev_init(struct rte_pci_driver *pci_drv, struct rte_pci_device *pci_dev) diff --git a/lib/librte_ether/rte_ethdev.h b/lib/librte_ether/rte_ethdev.h index 616a44a..fe64c78 100644 --- a/lib/librte_ether/rte_ethdev.h +++ b/lib/librte_ether/rte_ethdev.h @@ -1653,6 +1653,17 @@ extern uint8_t rte_eth_dev_count(void); */ struct rte_eth_dev *rte_eth_dev_allocate(const char *name); +/** + * Function for internal use by dummy drivers primarily, e.g. ring-based + * driver. + * Free the specified ethdev and returns the pointer to that slot. + * + * @param name Unique identifier name for each Ethernet device + * @return + * - Slot in the rte_dev_devices array for the freed device; + */ +struct rte_eth_dev *rte_eth_dev_free(const char *name); + struct eth_driver; /** * @internal -- 1.9.1
The patch adds function pointer to rte_pci_driver and eth_driver structure. These function pointers are used when ports are detached. Also the patch adds rte_eth_dev_uninit(). So far, it's not called by anywhere, but it will be called when port hotplug function is implemented. v4: - Add paramerter checking. - Change function names. Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp> --- lib/librte_eal/common/include/rte_pci.h | 7 ++++++ lib/librte_ether/rte_ethdev.c | 40 +++++++++++++++++++++++++++++++++ lib/librte_ether/rte_ethdev.h | 24 ++++++++++++++++++++ 3 files changed, 71 insertions(+) diff --git a/lib/librte_eal/common/include/rte_pci.h b/lib/librte_eal/common/include/rte_pci.h index 128b2ce..fe57cb6 100644 --- a/lib/librte_eal/common/include/rte_pci.h +++ b/lib/librte_eal/common/include/rte_pci.h @@ -181,12 +181,19 @@ struct rte_pci_driver; typedef int (pci_devinit_t)(struct rte_pci_driver *, struct rte_pci_device *); /** + * Uninitialisation function for the driver called during hotplugging. + */ +typedef int (pci_devuninit_t)( + struct rte_pci_driver *, struct rte_pci_device *); + +/** * A structure describing a PCI driver. */ struct rte_pci_driver { TAILQ_ENTRY(rte_pci_driver) next; /**< Next in list. */ const char *name; /**< Driver name. */ pci_devinit_t *devinit; /**< Device init. function. */ + pci_devuninit_t *devuninit; /**< Device uninit function. */ struct rte_pci_id *id_table; /**< ID table, NULL terminated. */ uint32_t drv_flags; /**< Flags contolling handling of device. */ }; diff --git a/lib/librte_ether/rte_ethdev.c b/lib/librte_ether/rte_ethdev.c index 16f22c0..e5145b7 100644 --- a/lib/librte_ether/rte_ethdev.c +++ b/lib/librte_ether/rte_ethdev.c @@ -335,6 +335,45 @@ rte_eth_dev_init(struct rte_pci_driver *pci_drv, return diag; } +static int +rte_eth_dev_uninit(struct rte_pci_driver *pci_drv, + struct rte_pci_device *pci_dev) +{ + struct eth_driver *eth_drv; + struct rte_eth_dev *eth_dev; + char ethdev_name[RTE_ETH_NAME_MAX_LEN]; + + if ((pci_drv == NULL) || (pci_dev == NULL)) + return -EINVAL; + + /* Create unique Ethernet device name using PCI address */ + snprintf(ethdev_name, RTE_ETH_NAME_MAX_LEN, "%d:%d.%d", + pci_dev->addr.bus, pci_dev->addr.devid, + pci_dev->addr.function); + + eth_dev = rte_eth_dev_free(ethdev_name); + if (eth_dev == NULL) + return -ENODEV; + + eth_drv = (struct eth_driver *)pci_drv; + + /* Invoke PMD device uninit function */ + if (*eth_drv->eth_dev_uninit) + (*eth_drv->eth_dev_uninit)(eth_drv, eth_dev); + + /* init user callbacks */ + TAILQ_INIT(&(eth_dev->callbacks)); + + if (rte_eal_process_type() == RTE_PROC_PRIMARY) + rte_free(eth_dev->data->dev_private); + + eth_dev->pci_dev = NULL; + eth_dev->driver = NULL; + eth_dev->data = NULL; + + return 0; +} + /** * Register an Ethernet [Poll Mode] driver. * @@ -353,6 +392,7 @@ void rte_eth_driver_register(struct eth_driver *eth_drv) { eth_drv->pci_drv.devinit = rte_eth_dev_init; + eth_drv->pci_drv.devuninit = rte_eth_dev_uninit; rte_eal_pci_register(ð_drv->pci_drv); } diff --git a/lib/librte_ether/rte_ethdev.h b/lib/librte_ether/rte_ethdev.h index fe64c78..2a8ff26 100644 --- a/lib/librte_ether/rte_ethdev.h +++ b/lib/librte_ether/rte_ethdev.h @@ -1701,6 +1701,27 @@ typedef int (*eth_dev_init_t)(struct eth_driver *eth_drv, /** * @internal + * Finalization function of an Ethernet driver invoked for each matching + * Ethernet PCI device detected during the PCI closing phase. + * + * @param eth_drv + * The pointer to the [matching] Ethernet driver structure supplied by + * the PMD when it registered itself. + * @param eth_dev + * The *eth_dev* pointer is the address of the *rte_eth_dev* structure + * associated with the matching device and which have been [automatically] + * allocated in the *rte_eth_devices* array. + * @return + * - 0: Success, the device is properly finalized by the driver. + * In particular, the driver MUST free the *dev_ops* pointer + * of the *eth_dev* structure. + * - <0: Error code of the device initialization failure. + */ +typedef int (*eth_dev_uninit_t)(struct eth_driver *eth_drv, + struct rte_eth_dev *eth_dev); + +/** + * @internal * The structure associated with a PMD Ethernet driver. * * Each Ethernet driver acts as a PCI driver and is represented by a generic @@ -1710,11 +1731,14 @@ typedef int (*eth_dev_init_t)(struct eth_driver *eth_drv, * * - The *eth_dev_init* function invoked for each matching PCI device. * + * - The *eth_dev_uninit* function invoked for each matching PCI device. + * * - The size of the private data to allocate for each matching device. */ struct eth_driver { struct rte_pci_driver pci_drv; /**< The PMD is also a PCI driver. */ eth_dev_init_t eth_dev_init; /**< Device init function. */ + eth_dev_uninit_t eth_dev_uninit; /**< Device uninit function. */ unsigned int dev_private_size; /**< Size of device private data. */ }; -- 1.9.1
The patch adds following functions. - rte_eth_dev_save() The function is used for saving current rte_eth_dev structures. - rte_eth_dev_get_changed_port() The function receives the rte_eth_dev structures, then compare these with current values to know which port is actually attached or detached. - rte_eth_dev_get_addr_by_port() The function returns a pci address of a ethdev specified by port identifier. - rte_eth_dev_get_port_by_addr() The function returns a port identifier of a ethdev specified by pci address. - rte_eth_dev_get_name_by_port() The function returns a unique identifier name of a ethdev specified by port identifier. - Add rte_eth_dev_check_detachable() The function returns whether a PMD supports detach function. Also the patch changes scope of rte_eth_dev_allocated() to global. This function will be called by virtual PMDs to support port hotplug. So change scope of the function to global. v4: - Add paramerter checking. v3: - Fix if-condition bug while comparing pci addresses. - Add error checking codes. Reported-by: Mark Enright <menrigh@brocade.com> Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp> --- lib/librte_ether/rte_ethdev.c | 102 +++++++++++++++++++++++++++++++++++++++++- lib/librte_ether/rte_ethdev.h | 80 +++++++++++++++++++++++++++++++++ 2 files changed, 181 insertions(+), 1 deletion(-) diff --git a/lib/librte_ether/rte_ethdev.c b/lib/librte_ether/rte_ethdev.c index e5145b7..e572ef4 100644 --- a/lib/librte_ether/rte_ethdev.c +++ b/lib/librte_ether/rte_ethdev.c @@ -206,7 +206,7 @@ rte_eth_dev_data_alloc(void) RTE_MAX_ETHPORTS * sizeof(*rte_eth_dev_data)); } -static struct rte_eth_dev * +struct rte_eth_dev * rte_eth_dev_allocated(const char *name) { unsigned i; @@ -422,6 +422,106 @@ rte_eth_dev_count(void) return (nb_ports); } +void +rte_eth_dev_save(struct rte_eth_dev *devs) +{ + if (devs == NULL) + return; + + /* save current rte_eth_devices */ + memcpy(devs, rte_eth_devices, + sizeof(struct rte_eth_dev) * RTE_MAX_ETHPORTS); +} + +int +rte_eth_dev_get_changed_port(struct rte_eth_dev *devs, uint8_t *port_id) +{ + if ((devs == NULL) || (port_id == NULL)) + return -1; + + /* check which port was attached or detached */ + for (*port_id = 0; *port_id < RTE_MAX_ETHPORTS; (*port_id)++, devs++) { + if (rte_eth_devices[*port_id].attached ^ devs->attached) + return 0; + } + return 1; +} + +int +rte_eth_dev_get_addr_by_port(uint8_t port_id, struct rte_pci_addr *addr) +{ + if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) { + PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + return -EINVAL; + } + + if (addr == NULL) { + PMD_DEBUG_TRACE("Null pointer is specified\n"); + return -EINVAL; + } + + *addr = rte_eth_devices[port_id].pci_dev->addr; + return 0; +} + +int +rte_eth_dev_get_port_by_addr(struct rte_pci_addr *addr, uint8_t *port_id) +{ + struct rte_pci_addr *tmp; + + if ((addr == NULL) || (port_id == NULL)) { + PMD_DEBUG_TRACE("Null pointer is specified\n"); + return -1; + } + + for (*port_id = 0; *port_id < RTE_MAX_ETHPORTS; (*port_id)++) { + if (!rte_eth_devices[*port_id].attached) + continue; + if (!rte_eth_devices[*port_id].pci_dev) + continue; + tmp = &rte_eth_devices[*port_id].pci_dev->addr; + if (eal_compare_pci_addr(tmp, addr) == 0) + return 0; + } + return -1; +} + +int +rte_eth_dev_get_name_by_port(uint8_t port_id, char *name) +{ + char *tmp; + + if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) { + PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + return -EINVAL; + } + + if (name == NULL) { + PMD_DEBUG_TRACE("Null pointer is specified\n"); + return -EINVAL; + } + + /* shouldn't check 'rte_eth_devices[i].data', + * because it might be overwritten by VDEV PMD */ + tmp = rte_eth_dev_data[port_id].name; + strncpy(name, tmp, strlen(tmp) + 1); + return 0; +} + +int +rte_eth_dev_check_detachable(uint8_t port_id) +{ + uint32_t drv_flags; + + if (port_id >= RTE_MAX_ETHPORTS) { + PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + return -EINVAL; + } + + drv_flags = rte_eth_devices[port_id].driver->pci_drv.drv_flags; + return !(drv_flags & RTE_PCI_DRV_DETACHABLE); +} + static int rte_eth_dev_rx_queue_config(struct rte_eth_dev *dev, uint16_t nb_queues) { diff --git a/lib/librte_ether/rte_ethdev.h b/lib/librte_ether/rte_ethdev.h index 2a8ff26..c0b98dc 100644 --- a/lib/librte_ether/rte_ethdev.h +++ b/lib/librte_ether/rte_ethdev.h @@ -1642,6 +1642,86 @@ extern struct rte_eth_dev rte_eth_devices[]; extern uint8_t rte_eth_dev_count(void); /** + * Function for internal use by port hotplug functions. + * Copies current ethdev structures to the specified pointer. + * + * @param devs The pointer to the ethdev structures + */ +extern void rte_eth_dev_save(struct rte_eth_dev *devs); + +/** + * Function for internal use by port hotplug functions. + * Compare the specified ethdev structures with currents. Then + * if there is a port which status is changed, fill the specified pointer + * with the port id of that port. + * @param devs The pointer to the ethdev structures + * @param port_id The pointer to the port id + * @return + * - 0 on success, negative on error + */ +extern int rte_eth_dev_get_changed_port( + struct rte_eth_dev *devs, uint8_t *port_id); + +/** + * Function for internal use by port hotplug functions. + * Returns a pci address of a ethdev specified by port identifier. + * @param port_id + * The port identifier of the Ethernet device + * @param addr + * The pointer to the pci address + * @return + * - 0 on success, negative on error + */ +extern int rte_eth_dev_get_addr_by_port( + uint8_t port_id, struct rte_pci_addr *addr); + +/** + * Function for internal use by port hotplug functions. + * Returns a port identifier of a ethdev specified by pci address. + * @param addr + * The pointer to the pci address of the Ethernet device. + * @param port_id + * The pointer to the port identifier + * @return + * - 0 on success, negative on error + */ +extern int rte_eth_dev_get_port_by_addr( + struct rte_pci_addr *addr, uint8_t *port_id); + +/** + * Function for internal use by port hotplug functions. + * Returns a unique identifier name of a ethdev specified by port identifier. + * @param port_id + * The port identifier. + * @param name + * The pointer to the Unique identifier name for each Ethernet device + * @return + * - 0 on success, negative on error + */ +extern int rte_eth_dev_get_name_by_port(uint8_t port_id, char *name); + +/** + * Function for internal use by port hotplug functions. + * Check whether or not, a PMD that is handling the ethdev specified by port + * identifier can support detach function. + * @param port_id + * The port identifier + * @return + * - 0 on supporting detach function, negative on not supporting + */ +extern int rte_eth_dev_check_detachable(uint8_t port_id); + +/** + * Function for internal use by port hotplug functions. + * Returns a ethdev slot specified by the unique identifier name. + * @param name + * The pointer to the Unique identifier name for each Ethernet device + * @return + * - The pointer to the ethdev slot, on success. NULL on error + */ +extern struct rte_eth_dev *rte_eth_dev_allocated(const char *name); + +/** * Function for internal use by dummy drivers primarily, e.g. ring-based * driver. * Allocates a new ethdev slot for an ethernet device and returns the pointer -- 1.9.1
The patch adds functions for unmapping igb_uio resources. The patch is only for Linux and igb_uio environment. VFIO and BSD are not supported. v4: - Add paramerter checking. - Add header file to determine if hotplug can be enabled. Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp> --- lib/librte_eal/common/Makefile | 1 + lib/librte_eal/common/include/rte_dev_hotplug.h | 44 +++++++++++++++++ lib/librte_eal/linuxapp/eal/eal_pci.c | 38 +++++++++++++++ lib/librte_eal/linuxapp/eal/eal_pci_init.h | 8 +++ lib/librte_eal/linuxapp/eal/eal_pci_uio.c | 65 +++++++++++++++++++++++++ 5 files changed, 156 insertions(+) create mode 100644 lib/librte_eal/common/include/rte_dev_hotplug.h diff --git a/lib/librte_eal/common/Makefile b/lib/librte_eal/common/Makefile index 52c1a5f..db7cc93 100644 --- a/lib/librte_eal/common/Makefile +++ b/lib/librte_eal/common/Makefile @@ -41,6 +41,7 @@ INC += rte_eal_memconfig.h rte_malloc_heap.h INC += rte_hexdump.h rte_devargs.h rte_dev.h INC += rte_common_vect.h INC += rte_pci_dev_feature_defs.h rte_pci_dev_features.h +INC += rte_dev_hotplug.h ifeq ($(CONFIG_RTE_INSECURE_FUNCTION_WARNING),y) INC += rte_warnings.h diff --git a/lib/librte_eal/common/include/rte_dev_hotplug.h b/lib/librte_eal/common/include/rte_dev_hotplug.h new file mode 100644 index 0000000..b333e0f --- /dev/null +++ b/lib/librte_eal/common/include/rte_dev_hotplug.h @@ -0,0 +1,44 @@ +/*- + * BSD LICENSE + * + * Copyright(c) 2015 IGEL Co.,LTd. + * All rights reserved. + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * + * * Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * * Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in + * the documentation and/or other materials provided with the + * distribution. + * * Neither the name of IGEL Co.,Ltd. nor the names of its + * contributors may be used to endorse or promote products derived + * from this software without specific prior written permission. + * + * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS + * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT + * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR + * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT + * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, + * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT + * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, + * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY + * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT + * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE + * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + */ + +#ifndef _RTE_DEV_HOTPLUG_H_ +#define _RTE_DEV_HOTPLUG_H_ + +/* + * determine if hotplug can be enabled on the system + */ +#if defined(RTE_LIBRTE_EAL_HOTPLUG) && defined(RTE_LIBRTE_EAL_LINUXAPP) +#define ENABLE_HOTPLUG +#endif /* RTE_LIBRTE_EAL_HOTPLUG & RTE_LIBRTE_EAL_LINUXAPP */ + +#endif /* _RTE_DEV_HOTPLUG_H_ */ diff --git a/lib/librte_eal/linuxapp/eal/eal_pci.c b/lib/librte_eal/linuxapp/eal/eal_pci.c index 3d2d93c..52c464c 100644 --- a/lib/librte_eal/linuxapp/eal/eal_pci.c +++ b/lib/librte_eal/linuxapp/eal/eal_pci.c @@ -137,6 +137,25 @@ pci_map_resource(void *requested_addr, int fd, off_t offset, size_t size) return mapaddr; } +#ifdef ENABLE_HOTPLUG +/* unmap a particular resource */ +void +pci_unmap_resource(void *requested_addr, size_t size) +{ + if (requested_addr == NULL) + return; + + /* Unmap the PCI memory resource of device */ + if (munmap(requested_addr, size)) { + RTE_LOG(ERR, EAL, "%s(): cannot munmap(%p, 0x%lx): %s\n", + __func__, requested_addr, (unsigned long)size, + strerror(errno)); + } else + RTE_LOG(DEBUG, EAL, " PCI memory mapped at %p\n", + requested_addr); +} +#endif /* ENABLE_HOTPLUG */ + /* parse the "resource" sysfs file */ #define IORESOURCE_MEM 0x00000200 @@ -510,6 +529,25 @@ pci_map_device(struct rte_pci_device *dev) return 0; } +#ifdef ENABLE_HOTPLUG +static void +pci_unmap_device(struct rte_pci_device *dev) +{ + if (dev == NULL) + return; + + /* try unmapping the NIC resources using VFIO if it exists */ +#ifdef VFIO_PRESENT + if (pci_vfio_is_enabled()) { + RTE_LOG(ERR, EAL, "%s() doesn't support vfio yet.\n", + __func__); + return; + } +#endif + pci_uio_unmap_resource(dev); +} +#endif /* ENABLE_HOTPLUG */ + /* * If vendor/device ID match, call the devinit() function of the * driver. diff --git a/lib/librte_eal/linuxapp/eal/eal_pci_init.h b/lib/librte_eal/linuxapp/eal/eal_pci_init.h index 1070eb8..5152a0b 100644 --- a/lib/librte_eal/linuxapp/eal/eal_pci_init.h +++ b/lib/librte_eal/linuxapp/eal/eal_pci_init.h @@ -34,6 +34,7 @@ #ifndef EAL_PCI_INIT_H_ #define EAL_PCI_INIT_H_ +#include <rte_dev_hotplug.h> #include "eal_vfio.h" struct pci_map { @@ -71,6 +72,13 @@ void *pci_map_resource(void *requested_addr, int fd, off_t offset, /* map IGB_UIO resource prototype */ int pci_uio_map_resource(struct rte_pci_device *dev); +#ifdef ENABLE_HOTPLUG +void pci_unmap_resource(void *requested_addr, size_t size); + +/* unmap IGB_UIO resource prototype */ +void pci_uio_unmap_resource(struct rte_pci_device *dev); +#endif /* ENABLE_HOTPLUG */ + #ifdef VFIO_PRESENT #define VFIO_MAX_GROUPS 64 diff --git a/lib/librte_eal/linuxapp/eal/eal_pci_uio.c b/lib/librte_eal/linuxapp/eal/eal_pci_uio.c index 1da3507..81830d1 100644 --- a/lib/librte_eal/linuxapp/eal/eal_pci_uio.c +++ b/lib/librte_eal/linuxapp/eal/eal_pci_uio.c @@ -404,6 +404,71 @@ pci_uio_map_resource(struct rte_pci_device *dev) return 0; } +#ifdef ENABLE_HOTPLUG +static void +pci_uio_unmap(struct mapped_pci_resource *uio_res) +{ + int i; + + if (uio_res == NULL) + return; + + for (i = 0; i != uio_res->nb_maps; i++) + pci_unmap_resource(uio_res->maps[i].addr, + (size_t)uio_res->maps[i].size); +} + +static struct mapped_pci_resource * +pci_uio_find_resource(struct rte_pci_device *dev) +{ + struct mapped_pci_resource *uio_res; + + if (dev == NULL) + return NULL; + + TAILQ_FOREACH(uio_res, pci_res_list, next) { + + /* skip this element if it doesn't match our PCI address */ + if (!eal_compare_pci_addr(&uio_res->pci_addr, &dev->addr)) + return uio_res; + } + return NULL; +} + +/* unmap the PCI resource of a PCI device in virtual memory */ +void +pci_uio_unmap_resource(struct rte_pci_device *dev) +{ + struct mapped_pci_resource *uio_res; + + if (dev == NULL) + return; + + /* find an entry for the device */ + uio_res = pci_uio_find_resource(dev); + if (uio_res == NULL) + return; + + /* secondary processes - just free maps */ + if (rte_eal_process_type() != RTE_PROC_PRIMARY) + return pci_uio_unmap(uio_res); + + TAILQ_REMOVE(pci_res_list, uio_res, next); + + /* unmap all resources */ + pci_uio_unmap(uio_res); + + /* free uio resource */ + rte_free(uio_res); + + /* close fd if in primary process */ + close(dev->intr_handle.fd); + + dev->intr_handle.fd = -1; + dev->intr_handle.type = RTE_INTR_HANDLE_UNKNOWN; +} +#endif /* ENABLE_HOTPLUG */ + /* * parse a sysfs file containing one integer value * different to the eal version, as it needs to work with 64-bit values -- 1.9.1
The function removes the specified devargs entry from devargs_list. Also the patch adds sanity checking to rte_eal_devargs_add(). v4: - Fix sanity check code Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp> --- lib/librte_eal/common/eal_common_devargs.c | 57 +++++++++++++++++++++++++++++ lib/librte_eal/common/include/rte_devargs.h | 18 +++++++++ 2 files changed, 75 insertions(+) diff --git a/lib/librte_eal/common/eal_common_devargs.c b/lib/librte_eal/common/eal_common_devargs.c index 4c7d11a..a360a85 100644 --- a/lib/librte_eal/common/eal_common_devargs.c +++ b/lib/librte_eal/common/eal_common_devargs.c @@ -44,6 +44,35 @@ struct rte_devargs_list devargs_list = TAILQ_HEAD_INITIALIZER(devargs_list); + +/* find a entry specified by pci address or device name */ +static struct rte_devargs * +rte_eal_devargs_find(enum rte_devtype devtype, void *args) +{ + struct rte_devargs *devargs; + + if (args == NULL) + return NULL; + + TAILQ_FOREACH(devargs, &devargs_list, next) { + switch (devtype) { + case RTE_DEVTYPE_WHITELISTED_PCI: + case RTE_DEVTYPE_BLACKLISTED_PCI: + if (eal_compare_pci_addr(&devargs->pci.addr, args) == 0) + goto found; + break; + case RTE_DEVTYPE_VIRTUAL: + if (memcmp(&devargs->virtual.drv_name, args, + strlen((char *)args)) == 0) + goto found; + break; + } + } + return NULL; +found: + return devargs; +} + /* store a whitelist parameter for later parsing */ int rte_eal_devargs_add(enum rte_devtype devtype, const char *devargs_str) @@ -87,6 +116,12 @@ rte_eal_devargs_add(enum rte_devtype devtype, const char *devargs_str) free(devargs); return -1; } + /* make sure there is no same entry */ + if (rte_eal_devargs_find(devtype, &devargs->pci.addr)) { + RTE_LOG(ERR, EAL, + "device already registered: <%s>\n", buf); + return -1; + } break; case RTE_DEVTYPE_VIRTUAL: /* save driver name */ @@ -98,6 +133,12 @@ rte_eal_devargs_add(enum rte_devtype devtype, const char *devargs_str) free(devargs); return -1; } + /* make sure there is no same entry */ + if (rte_eal_devargs_find(devtype, &devargs->virtual.drv_name)) { + RTE_LOG(ERR, EAL, + "device already registered: <%s>\n", buf); + return -1; + } break; } @@ -105,6 +146,22 @@ rte_eal_devargs_add(enum rte_devtype devtype, const char *devargs_str) return 0; } +/* remove it from the devargs_list */ +void +rte_eal_devargs_remove(enum rte_devtype devtype, void *args) +{ + struct rte_devargs *devargs; + + if (args == NULL) + return; + + devargs = rte_eal_devargs_find(devtype, args); + if (devargs == NULL) + return; + + TAILQ_REMOVE(&devargs_list, devargs, next); +} + /* count the number of devices of a specified type */ unsigned int rte_eal_devargs_type_count(enum rte_devtype devtype) diff --git a/lib/librte_eal/common/include/rte_devargs.h b/lib/librte_eal/common/include/rte_devargs.h index 9f9c98f..1066efd 100644 --- a/lib/librte_eal/common/include/rte_devargs.h +++ b/lib/librte_eal/common/include/rte_devargs.h @@ -123,6 +123,24 @@ extern struct rte_devargs_list devargs_list; int rte_eal_devargs_add(enum rte_devtype devtype, const char *devargs_str); /** + * Remove a device from the user device list + * + * For PCI devices, the format of arguments string is "PCI_ADDR". It shouldn't + * involves parameters for the device. Example: "08:00.1". + * + * For virtual devices, the format of arguments string is "DRIVER_NAME*". It + * shouldn't involves parameters for the device. Example: "eth_ring". The + * validity of the driver name is not checked by this function, it is done + * when closing the drivers. + * + * @param devtype + * The type of the device. + * @param name + * The name of the device. + */ +void rte_eal_devargs_remove(enum rte_devtype devtype, void *args); + +/** * Count the number of user devices of a specified type * * @param devtype -- 1.9.1
- Add rte_eal_pci_close_one_dirver() The function is used for closing the specified driver and device. - Add pci_invoke_all_drivers() The function is based on pci_probe_all_drivers. But it can not only probe but also close drivers. - Add pci_close_all_drivers() The function tries to find a driver for the specified device, and then close the driver. - Add rte_eal_pci_probe_one() and rte_eal_pci_close_one() The functions are used for probe and close a device. First the function tries to find a device that has the specfied PCI address. Then, probe or close the device. v4: - Fix paramerter checking. - Fix indent of 'if' statement. Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp> --- lib/librte_eal/common/eal_common_pci.c | 90 +++++++++++++++++++++++++++++---- lib/librte_eal/common/eal_private.h | 25 +++++++++ lib/librte_eal/common/include/rte_pci.h | 33 ++++++++++++ lib/librte_eal/linuxapp/eal/eal_pci.c | 69 +++++++++++++++++++++++++ 4 files changed, 207 insertions(+), 10 deletions(-) diff --git a/lib/librte_eal/common/eal_common_pci.c b/lib/librte_eal/common/eal_common_pci.c index a89f5c3..7c9b8c5 100644 --- a/lib/librte_eal/common/eal_common_pci.c +++ b/lib/librte_eal/common/eal_common_pci.c @@ -99,19 +99,27 @@ static struct rte_devargs *pci_devargs_lookup(struct rte_pci_device *dev) return NULL; } -/* - * If vendor/device ID match, call the devinit() function of all - * registered driver for the given device. Return -1 if initialization - * failed, return 1 if no driver is found for this device. - */ static int -pci_probe_all_drivers(struct rte_pci_device *dev) +pci_invoke_all_drivers(struct rte_pci_device *dev, + enum rte_eal_invoke_type type) { struct rte_pci_driver *dr = NULL; - int rc; + int rc = 0; + + if ((dev == NULL) || (type >= RTE_EAL_INVOKE_TYPE_MAX)) + return -1; TAILQ_FOREACH(dr, &pci_driver_list, next) { - rc = rte_eal_pci_probe_one_driver(dr, dev); + switch (type) { + case RTE_EAL_INVOKE_TYPE_PROBE: + rc = rte_eal_pci_probe_one_driver(dr, dev); + break; + case RTE_EAL_INVOKE_TYPE_CLOSE: + rc = rte_eal_pci_close_one_driver(dr, dev); + break; + default: + return -1; + } if (rc < 0) /* negative value is an error */ return -1; @@ -123,6 +131,66 @@ pci_probe_all_drivers(struct rte_pci_device *dev) return 1; } +#ifdef ENABLE_HOTPLUG +static int +rte_eal_pci_invoke_one(struct rte_pci_addr *addr, + enum rte_eal_invoke_type type) +{ + struct rte_pci_device *dev = NULL; + int ret = 0; + + if ((addr == NULL) || (type >= RTE_EAL_INVOKE_TYPE_MAX)) + return -1; + + TAILQ_FOREACH(dev, &pci_device_list, next) { + if (eal_compare_pci_addr(&dev->addr, addr)) + continue; + + ret = pci_invoke_all_drivers(dev, type); + if (ret < 0) + goto invoke_err_return; + + if (type == RTE_EAL_INVOKE_TYPE_CLOSE) + goto remove_dev; + + return 0; + } + + return -1; + +invoke_err_return: + RTE_LOG(WARNING, EAL, "Requested device " PCI_PRI_FMT + " cannot be used\n", dev->addr.domain, dev->addr.bus, + dev->addr.devid, dev->addr.function); + return -1; + +remove_dev: + TAILQ_REMOVE(&pci_device_list, dev, next); + return 0; +} + + +/* + * Find the pci device specified by pci address, then invoke probe function of + * the driver of the devive. + */ +int +rte_eal_pci_probe_one(struct rte_pci_addr *addr) +{ + return rte_eal_pci_invoke_one(addr, RTE_EAL_INVOKE_TYPE_PROBE); +} + +/* + * Find the pci device specified by pci address, then invoke close function of + * the driver of the devive. + */ +int +rte_eal_pci_close_one(struct rte_pci_addr *addr) +{ + return rte_eal_pci_invoke_one(addr, RTE_EAL_INVOKE_TYPE_CLOSE); +} +#endif /* ENABLE_HOTPLUG */ + /* * Scan the content of the PCI bus, and call the devinit() function for * all registered drivers that have a matching entry in its id_table @@ -148,10 +216,12 @@ rte_eal_pci_probe(void) /* probe all or only whitelisted devices */ if (probe_all) - ret = pci_probe_all_drivers(dev); + ret = pci_invoke_all_drivers(dev, + RTE_EAL_INVOKE_TYPE_PROBE); else if (devargs != NULL && devargs->type == RTE_DEVTYPE_WHITELISTED_PCI) - ret = pci_probe_all_drivers(dev); + ret = pci_invoke_all_drivers(dev, + RTE_EAL_INVOKE_TYPE_PROBE); if (ret < 0) rte_exit(EXIT_FAILURE, "Requested device " PCI_PRI_FMT " cannot be used\n", dev->addr.domain, dev->addr.bus, diff --git a/lib/librte_eal/common/eal_private.h b/lib/librte_eal/common/eal_private.h index 159cd66..a97c5d8 100644 --- a/lib/librte_eal/common/eal_private.h +++ b/lib/librte_eal/common/eal_private.h @@ -154,6 +154,16 @@ struct rte_pci_driver; struct rte_pci_device; /** + * The invoke type. + */ +enum rte_eal_invoke_type { + RTE_EAL_INVOKE_TYPE_UNKNOWN,/**< unknown */ + RTE_EAL_INVOKE_TYPE_PROBE, /**< invoke probe function */ + RTE_EAL_INVOKE_TYPE_CLOSE, /**< invoke close function */ + RTE_EAL_INVOKE_TYPE_MAX /**< max value of this enum */ +}; + +/** * Mmap memory for single PCI device * * This function is private to EAL. @@ -165,6 +175,21 @@ int rte_eal_pci_probe_one_driver(struct rte_pci_driver *dr, struct rte_pci_device *dev); /** + * Munmap memory for single PCI device + * + * This function is private to EAL. + * + * @param dr + * The pointer to the pci driver structure + * @param dev + * The pointer to the pci device structure + * @return + * 0 on success, negative on error + */ +int rte_eal_pci_close_one_driver(struct rte_pci_driver *dr, + struct rte_pci_device *dev); + +/** * Init tail queues for non-EAL library structures. This is to allow * the rings, mempools, etc. lists to be shared among multiple processes * diff --git a/lib/librte_eal/common/include/rte_pci.h b/lib/librte_eal/common/include/rte_pci.h index fe57cb6..28462f5 100644 --- a/lib/librte_eal/common/include/rte_pci.h +++ b/lib/librte_eal/common/include/rte_pci.h @@ -82,6 +82,7 @@ extern "C" { #include <inttypes.h> #include <rte_interrupts.h> +#include <rte_dev_hotplug.h> TAILQ_HEAD(pci_device_list, rte_pci_device); /**< PCI devices in D-linked Q. */ TAILQ_HEAD(pci_driver_list, rte_pci_driver); /**< PCI drivers in D-linked Q. */ @@ -315,6 +316,38 @@ eal_compare_pci_addr(struct rte_pci_addr *addr, struct rte_pci_addr *addr2) */ int rte_eal_pci_probe(void); +#ifdef ENABLE_HOTPLUG +/** + * Probe the single PCI device. + * + * Scan the content of the PCI bus, and find the pci device specified by pci + * address, then call the probe() function for registered driver that has a + * matching entry in its id_table for discovered device. + * + * @param addr + * The PCI Bus-Device-Function address to probe or close. + * @return + * - 0 on success. + * - Negative on error. + */ +int rte_eal_pci_probe_one(struct rte_pci_addr *addr); + +/** + * Close the single PCI device. + * + * Scan the content of the PCI bus, and find the pci device specified by pci + * address, then call the close() function for registered driver that has a + * matching entry in its id_table for discovered device. + * + * @param addr + * The PCI Bus-Device-Function address to probe or close. + * @return + * - 0 on success. + * - Negative on error. + */ +int rte_eal_pci_close_one(struct rte_pci_addr *addr); +#endif /* ENABLE_HOTPLUG */ + /** * Dump the content of the PCI bus. * diff --git a/lib/librte_eal/linuxapp/eal/eal_pci.c b/lib/librte_eal/linuxapp/eal/eal_pci.c index 52c464c..a23cc59 100644 --- a/lib/librte_eal/linuxapp/eal/eal_pci.c +++ b/lib/librte_eal/linuxapp/eal/eal_pci.c @@ -619,6 +619,75 @@ rte_eal_pci_probe_one_driver(struct rte_pci_driver *dr, struct rte_pci_device *d return 1; } +#ifdef ENABLE_HOTPLUG +/* + * If vendor/device ID match, call the devuninit() function of the + * driver. + */ +int +rte_eal_pci_close_one_driver(struct rte_pci_driver *dr, + struct rte_pci_device *dev) +{ + struct rte_pci_id *id_table; + + if ((dr == NULL) || (dev == NULL)) + return -EINVAL; + + for (id_table = dr->id_table ; id_table->vendor_id != 0; id_table++) { + + /* check if device's identifiers match the driver's ones */ + if (id_table->vendor_id != dev->id.vendor_id && + id_table->vendor_id != PCI_ANY_ID) + continue; + if (id_table->device_id != dev->id.device_id && + id_table->device_id != PCI_ANY_ID) + continue; + if (id_table->subsystem_vendor_id != + dev->id.subsystem_vendor_id && + id_table->subsystem_vendor_id != PCI_ANY_ID) + continue; + if (id_table->subsystem_device_id != + dev->id.subsystem_device_id && + id_table->subsystem_device_id != PCI_ANY_ID) + continue; + + struct rte_pci_addr *loc = &dev->addr; + + RTE_LOG(DEBUG, EAL, + "PCI device "PCI_PRI_FMT" on NUMA socket %i\n", + loc->domain, loc->bus, loc->devid, + loc->function, dev->numa_node); + + RTE_LOG(DEBUG, EAL, " remove driver: %x:%x %s\n", + dev->id.vendor_id, dev->id.device_id, + dr->name); + + /* call the driver devuninit() function */ + if (dr->devuninit && (dr->devuninit(dr, dev) < 0)) + return -1; /* negative value is an error */ + + /* clear driver structure */ + dev->driver = NULL; + + if (dr->drv_flags & RTE_PCI_DRV_NEED_MAPPING) + /* unmap resources for devices that use igb_uio */ + pci_unmap_device(dev); + + return 0; + } + /* return positive value if driver is not found */ + return 1; +} +#else /* ENABLE_HOTPLUG */ +int +rte_eal_pci_close_one_driver(struct rte_pci_driver *dr __rte_unused, + struct rte_pci_device *dev __rte_unused) +{ + RTE_LOG(ERR, EAL, "Hotplug support isn't enabled\n"); + return -1; +} +#endif /* ENABLE_HOTPLUG */ + /* Init the PCI EAL subsystem */ int rte_eal_pci_init(void) -- 1.9.1
This new parameter is needed to keep device type like physical or virtual. Port detaching processes are different between physical and virtual. This paramerter lets detaching function know a device type of the port. v4: - Fix comments of rte_eth_dev_type. Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp> --- app/test/virtual_pmd.c | 2 +- lib/librte_ether/rte_ethdev.c | 14 ++++++++++++-- lib/librte_ether/rte_ethdev.h | 25 ++++++++++++++++++++++++- lib/librte_pmd_af_packet/rte_eth_af_packet.c | 2 +- lib/librte_pmd_bond/rte_eth_bond_api.c | 2 +- lib/librte_pmd_pcap/rte_eth_pcap.c | 2 +- lib/librte_pmd_ring/rte_eth_ring.c | 2 +- lib/librte_pmd_xenvirt/rte_eth_xenvirt.c | 2 +- 8 files changed, 42 insertions(+), 9 deletions(-) diff --git a/app/test/virtual_pmd.c b/app/test/virtual_pmd.c index 9fac95d..8d3a5ff 100644 --- a/app/test/virtual_pmd.c +++ b/app/test/virtual_pmd.c @@ -556,7 +556,7 @@ virtual_ethdev_create(const char *name, struct ether_addr *mac_addr, goto err; /* reserve an ethdev entry */ - eth_dev = rte_eth_dev_allocate(name); + eth_dev = rte_eth_dev_allocate(name, RTE_ETH_DEV_PHYSICAL); if (eth_dev == NULL) goto err; diff --git a/lib/librte_ether/rte_ethdev.c b/lib/librte_ether/rte_ethdev.c index e572ef4..d9bf24c 100644 --- a/lib/librte_ether/rte_ethdev.c +++ b/lib/librte_ether/rte_ethdev.c @@ -232,7 +232,7 @@ rte_eth_dev_allocate_new_port(void) } struct rte_eth_dev * -rte_eth_dev_allocate(const char *name) +rte_eth_dev_allocate(const char *name, enum rte_eth_dev_type type) { uint8_t port_id; struct rte_eth_dev *eth_dev; @@ -256,6 +256,7 @@ rte_eth_dev_allocate(const char *name) snprintf(eth_dev->data->name, sizeof(eth_dev->data->name), "%s", name); eth_dev->data->port_id = port_id; eth_dev->attached = DEV_CONNECTED; + eth_dev->dev_type = type; nb_ports++; return eth_dev; } @@ -276,6 +277,7 @@ rte_eth_dev_free(const char *name) } eth_dev->attached = 0; + eth_dev->dev_type = RTE_ETH_DEV_UNKNOWN; nb_ports--; return eth_dev; } @@ -296,7 +298,7 @@ rte_eth_dev_init(struct rte_pci_driver *pci_drv, snprintf(ethdev_name, RTE_ETH_NAME_MAX_LEN, "%d:%d.%d", pci_dev->addr.bus, pci_dev->addr.devid, pci_dev->addr.function); - eth_dev = rte_eth_dev_allocate(ethdev_name); + eth_dev = rte_eth_dev_allocate(ethdev_name, RTE_ETH_DEV_PHYSICAL); if (eth_dev == NULL) return -ENOMEM; @@ -422,6 +424,14 @@ rte_eth_dev_count(void) return (nb_ports); } +enum rte_eth_dev_type +rte_eth_dev_get_device_type(uint8_t port_id) +{ + if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) + return -1; + return rte_eth_devices[port_id].dev_type; +} + void rte_eth_dev_save(struct rte_eth_dev *devs) { diff --git a/lib/librte_ether/rte_ethdev.h b/lib/librte_ether/rte_ethdev.h index c0b98dc..8b0bad6 100644 --- a/lib/librte_ether/rte_ethdev.h +++ b/lib/librte_ether/rte_ethdev.h @@ -1548,6 +1548,17 @@ struct eth_dev_ops { }; /** + * The eth device type + */ +enum rte_eth_dev_type { + RTE_ETH_DEV_UNKNOWN, /**< unknown device type */ + RTE_ETH_DEV_PHYSICAL, + /**< Physical function and Virtual function devices of NIC */ + RTE_ETH_DEV_VIRTUAL, /**< non hardware device */ + RTE_ETH_DEV_MAX /**< max value of this enum */ +}; + +/** * @internal * The generic data structure associated with each ethernet device. * @@ -1566,6 +1577,7 @@ struct rte_eth_dev { struct rte_pci_device *pci_dev; /**< PCI info. supplied by probing */ struct rte_eth_dev_cb_list callbacks; /**< User application callbacks */ uint8_t attached; /**< Flag indicating the port is attached */ + enum rte_eth_dev_type dev_type; /**< Flag indicating the device type */ }; struct rte_eth_dev_sriov { @@ -1643,6 +1655,15 @@ extern uint8_t rte_eth_dev_count(void); /** * Function for internal use by port hotplug functions. + * Get the device type to know whether the device is physical or virtual. + * @param port_id The pointer to the port id + * @return + * - Device type. + */ +extern enum rte_eth_dev_type rte_eth_dev_get_device_type(uint8_t port_id); + +/** + * Function for internal use by port hotplug functions. * Copies current ethdev structures to the specified pointer. * * @param devs The pointer to the ethdev structures @@ -1728,10 +1749,12 @@ extern struct rte_eth_dev *rte_eth_dev_allocated(const char *name); * to that slot for the driver to use. * * @param name Unique identifier name for each Ethernet device + * @param type Device type of this Ethernet device * @return * - Slot in the rte_dev_devices array for a new device; */ -struct rte_eth_dev *rte_eth_dev_allocate(const char *name); +struct rte_eth_dev *rte_eth_dev_allocate(const char *name, + enum rte_eth_dev_type type); /** * Function for internal use by dummy drivers primarily, e.g. ring-based diff --git a/lib/librte_pmd_af_packet/rte_eth_af_packet.c b/lib/librte_pmd_af_packet/rte_eth_af_packet.c index 755780a..ff44ba7 100644 --- a/lib/librte_pmd_af_packet/rte_eth_af_packet.c +++ b/lib/librte_pmd_af_packet/rte_eth_af_packet.c @@ -651,7 +651,7 @@ rte_pmd_init_internals(const char *name, } /* reserve an ethdev entry */ - *eth_dev = rte_eth_dev_allocate(name); + *eth_dev = rte_eth_dev_allocate(name, RTE_ETH_DEV_VIRTUAL); if (*eth_dev == NULL) goto error; diff --git a/lib/librte_pmd_bond/rte_eth_bond_api.c b/lib/librte_pmd_bond/rte_eth_bond_api.c index 4ab3267..7a6a5f7 100644 --- a/lib/librte_pmd_bond/rte_eth_bond_api.c +++ b/lib/librte_pmd_bond/rte_eth_bond_api.c @@ -235,7 +235,7 @@ rte_eth_bond_create(const char *name, uint8_t mode, uint8_t socket_id) } /* reserve an ethdev entry */ - eth_dev = rte_eth_dev_allocate(name); + eth_dev = rte_eth_dev_allocate(name, RTE_ETH_DEV_VIRTUAL); if (eth_dev == NULL) { RTE_BOND_LOG(ERR, "Unable to allocate rte_eth_dev"); goto err; diff --git a/lib/librte_pmd_pcap/rte_eth_pcap.c b/lib/librte_pmd_pcap/rte_eth_pcap.c index f12d1e7..5da92a3 100644 --- a/lib/librte_pmd_pcap/rte_eth_pcap.c +++ b/lib/librte_pmd_pcap/rte_eth_pcap.c @@ -710,7 +710,7 @@ rte_pmd_init_internals(const char *name, const unsigned nb_rx_queues, goto error; /* reserve an ethdev entry */ - *eth_dev = rte_eth_dev_allocate(name); + *eth_dev = rte_eth_dev_allocate(name, RTE_ETH_DEV_VIRTUAL); if (*eth_dev == NULL) goto error; diff --git a/lib/librte_pmd_ring/rte_eth_ring.c b/lib/librte_pmd_ring/rte_eth_ring.c index 4f1b6ed..d901053 100644 --- a/lib/librte_pmd_ring/rte_eth_ring.c +++ b/lib/librte_pmd_ring/rte_eth_ring.c @@ -256,7 +256,7 @@ rte_eth_from_rings(const char *name, struct rte_ring *const rx_queues[], goto error; /* reserve an ethdev entry */ - eth_dev = rte_eth_dev_allocate(name); + eth_dev = rte_eth_dev_allocate(name, RTE_ETH_DEV_VIRTUAL); if (eth_dev == NULL) goto error; diff --git a/lib/librte_pmd_xenvirt/rte_eth_xenvirt.c b/lib/librte_pmd_xenvirt/rte_eth_xenvirt.c index 04e30c9..bc403d6 100644 --- a/lib/librte_pmd_xenvirt/rte_eth_xenvirt.c +++ b/lib/librte_pmd_xenvirt/rte_eth_xenvirt.c @@ -648,7 +648,7 @@ eth_dev_xenvirt_create(const char *name, const char *params, goto err; /* reserve an ethdev entry */ - eth_dev = rte_eth_dev_allocate(name); + eth_dev = rte_eth_dev_allocate(name, RTE_ETH_DEV_VIRTUAL); if (eth_dev == NULL) goto err; -- 1.9.1
These functions are used for attaching or detaching a port. When rte_eal_dev_attach() is called, the function tries to realize the device name as pci address. If this is done successfully, rte_eal_dev_attach() will attach physical device port. If not, attaches virtual devive port. When rte_eal_dev_detach() is called, the function gets the device type of this port to know whether the port is came from physical or virtual. And then specific detaching function will be called. v4: - Fix comment. - Add error checking. - Fix indent of 'if' statement. - Change function name. Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp> --- lib/librte_eal/common/eal_common_dev.c | 273 ++++++++++++++++++++++++++++++++ lib/librte_eal/common/eal_private.h | 11 ++ lib/librte_eal/common/include/rte_dev.h | 33 ++++ lib/librte_eal/linuxapp/eal/Makefile | 1 + lib/librte_eal/linuxapp/eal/eal_pci.c | 6 +- 5 files changed, 321 insertions(+), 3 deletions(-) diff --git a/lib/librte_eal/common/eal_common_dev.c b/lib/librte_eal/common/eal_common_dev.c index eae5656..828bd70 100644 --- a/lib/librte_eal/common/eal_common_dev.c +++ b/lib/librte_eal/common/eal_common_dev.c @@ -32,10 +32,13 @@ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ +#include <stdio.h> +#include <limits.h> #include <string.h> #include <inttypes.h> #include <sys/queue.h> +#include <rte_ethdev.h> #include <rte_dev.h> #include <rte_devargs.h> #include <rte_debug.h> @@ -107,3 +110,273 @@ rte_eal_dev_init(void) } return 0; } + +/* So far, DPDK hotplug function only supports linux */ +#ifdef ENABLE_HOTPLUG +static void +rte_eal_dev_invoke(struct rte_driver *driver, + struct rte_devargs *devargs, enum rte_eal_invoke_type type) +{ + if ((driver == NULL) || (devargs == NULL)) + return; + + switch (type) { + case RTE_EAL_INVOKE_TYPE_PROBE: + driver->init(devargs->virtual.drv_name, devargs->args); + break; + case RTE_EAL_INVOKE_TYPE_CLOSE: + driver->uninit(devargs->virtual.drv_name, devargs->args); + break; + default: + break; + } +} + +static int +rte_eal_dev_find_and_invoke(const char *name, int type) +{ + struct rte_devargs *devargs; + struct rte_driver *driver; + + if (name == NULL) + return -EINVAL; + + /* call the init function for each virtual device */ + TAILQ_FOREACH(devargs, &devargs_list, next) { + + if (devargs->type != RTE_DEVTYPE_VIRTUAL) + continue; + + if (strncmp(name, devargs->virtual.drv_name, strlen(name))) + continue; + + TAILQ_FOREACH(driver, &dev_driver_list, next) { + if (driver->type != PMD_VDEV) + continue; + + /* search a driver prefix in virtual device name */ + if (!strncmp(driver->name, devargs->virtual.drv_name, + strlen(driver->name))) { + rte_eal_dev_invoke(driver, devargs, type); + break; + } + } + + if (driver == NULL) { + RTE_LOG(WARNING, EAL, "no driver found for %s\n", + devargs->virtual.drv_name); + } + return 0; + } + return 1; +} + +/* attach the new physical device, then store port_id of the device */ +static int +rte_eal_dev_attach_pdev(struct rte_pci_addr *addr, uint8_t *port_id) +{ + uint8_t new_port_id; + struct rte_eth_dev devs[RTE_MAX_ETHPORTS]; + + if ((addr == NULL) || (port_id == NULL)) + goto err; + + /* save current port status */ + rte_eth_dev_save(devs); + /* re-construct pci_device_list */ + if (rte_eal_pci_scan()) + goto err; + /* invoke probe func of the driver can handle the new device */ + if (rte_eal_pci_probe_one(addr)) + goto err; + /* get port_id enabled by above procedures */ + if (rte_eth_dev_get_changed_port(devs, &new_port_id)) + goto err; + + *port_id = new_port_id; + return 0; +err: + RTE_LOG(ERR, EAL, "Drver, cannot attach the device\n"); + return -1; +} + +/* detach the new physical device, then store pci_addr of the device */ +static int +rte_eal_dev_detach_pdev(uint8_t port_id, struct rte_pci_addr *addr) +{ + struct rte_pci_addr freed_addr; + struct rte_pci_addr vp; + + if (addr == NULL) + goto err; + + /* check whether the driver supports detach feature, or not */ + if (rte_eth_dev_check_detachable(port_id)) + goto err; + + /* get pci address by port id */ + if (rte_eth_dev_get_addr_by_port(port_id, &freed_addr)) + goto err; + + /* Zerod pci addr means the port comes from virtual device */ + vp.domain = vp.bus = vp.devid = vp.function = 0; + if (eal_compare_pci_addr(&vp, &freed_addr) == 0) + goto err; + + /* invoke close func of the driver, + * also remove the device from pci_device_list */ + if (rte_eal_pci_close_one(&freed_addr)) + goto err; + + *addr = freed_addr; + return 0; +err: + RTE_LOG(ERR, EAL, "Drver, cannot detach the device\n"); + return -1; +} + +static void +get_vdev_name(char *vdevargs) +{ + char *sep; + + if (vdevargs == NULL) + return; + + /* set the first ',' to '\0' to split name and arguments */ + sep = strchr(vdevargs, ','); + if (sep != NULL) + sep[0] = '\0'; +} + +/* attach the new virtual device, then store port_id of the device */ +static int +rte_eal_dev_attach_vdev(const char *vdevargs, uint8_t *port_id) +{ + char *args; + uint8_t new_port_id; + struct rte_eth_dev devs[RTE_MAX_ETHPORTS]; + + if ((vdevargs == NULL) || (port_id == NULL)) + goto err0; + + args = strdup(vdevargs); + if (args == NULL) + goto err0; + + /* save current port status */ + rte_eth_dev_save(devs); + /* add the vdevargs to devargs_list */ + if (rte_eal_devargs_add(RTE_DEVTYPE_VIRTUAL, args)) + goto err1; + /* parse vdevargs, then retrieve device name */ + get_vdev_name(args); + /* walk around dev_driver_list to find the driver of the device, + * then invoke probe function o the driver */ + if (rte_eal_dev_find_and_invoke(args, RTE_EAL_INVOKE_TYPE_PROBE)) + goto err2; + /* get port_id enabled by above procedures */ + if (rte_eth_dev_get_changed_port(devs, &new_port_id)) + goto err2; + + free(args); + *port_id = new_port_id; + return 0; +err2: + rte_eal_devargs_remove(RTE_DEVTYPE_VIRTUAL, args); +err1: + free(args); +err0: + RTE_LOG(ERR, EAL, "Drver, cannot detach the device\n"); + return -1; +} + +/* detach the new virtual device, then store the name of the device */ +static int +rte_eal_dev_detach_vdev(uint8_t port_id, char *vdevname) +{ + char name[RTE_ETH_NAME_MAX_LEN]; + + if (vdevname == NULL) + goto err; + + /* check whether the driver supports detach feature, or not */ + if (rte_eth_dev_check_detachable(port_id)) + goto err; + + /* get device name by port id */ + if (rte_eth_dev_get_name_by_port(port_id, name)) + goto err; + /* walk around dev_driver_list to find the driver of the device, + * then invoke close function o the driver */ + if (rte_eal_dev_find_and_invoke(name, RTE_EAL_INVOKE_TYPE_CLOSE)) + goto err; + /* remove the vdevname from devargs_list */ + rte_eal_devargs_remove(RTE_DEVTYPE_VIRTUAL, name); + + strncpy(vdevname, name, sizeof(name)); + return 0; +err: + RTE_LOG(ERR, EAL, "Drver, cannot detach the device\n"); + return -1; +} + +/* attach the new device, then store port_id of the device */ +int +rte_eal_dev_attach(const char *devargs, uint8_t *port_id) +{ + struct rte_pci_addr addr; + + if ((devargs == NULL) || (port_id == NULL)) + return -EINVAL; + + if (eal_parse_pci_DomBDF(devargs, &addr) == 0) + return rte_eal_dev_attach_pdev(&addr, port_id); + else + return rte_eal_dev_attach_vdev(devargs, port_id); +} + +/* detach the device, then store the name of the device */ +int +rte_eal_dev_detach(uint8_t port_id, char *name) +{ + struct rte_pci_addr addr; + int ret; + + if (name == NULL) + return -EINVAL; + + if (rte_eth_dev_get_device_type(port_id) == RTE_ETH_DEV_PHYSICAL) { + ret = rte_eth_dev_get_addr_by_port(port_id, &addr); + if (ret < 0) + return ret; + + ret = rte_eal_dev_detach_pdev(port_id, &addr); + if (ret == 0) + snprintf(name, RTE_ETH_NAME_MAX_LEN, + "%04x.%02x.%02x.%d", + addr.domain, addr.bus, + addr.devid, addr.function); + + return ret; + } else + return rte_eal_dev_detach_vdev(port_id, name); +} +#else /* ENABLE_HOTPLUG */ +int +rte_eal_dev_attach(const char *devargs __rte_unused, + uint8_t *port_id __rte_unused) +{ + RTE_LOG(ERR, EAL, "Hotplug support isn't enabled\n"); + return -1; +} + +/* detach the device, then store the name of the device */ +int +rte_eal_dev_detach(uint8_t port_id __rte_unused, + char *name __rte_unused) +{ + RTE_LOG(ERR, EAL, "Hotplug support isn't enabled\n"); + return -1; +} +#endif /* ENABLE_HOTPLUG */ diff --git a/lib/librte_eal/common/eal_private.h b/lib/librte_eal/common/eal_private.h index a97c5d8..453a1eb 100644 --- a/lib/librte_eal/common/eal_private.h +++ b/lib/librte_eal/common/eal_private.h @@ -164,6 +164,17 @@ enum rte_eal_invoke_type { }; /** + * Scan the content of the PCI bus, and the devices in the devices + * list + * + * This function is private to EAL. + * + * @return + * 0 on success, negative on error + */ +int rte_eal_pci_scan(void); + +/** * Mmap memory for single PCI device * * This function is private to EAL. diff --git a/lib/librte_eal/common/include/rte_dev.h b/lib/librte_eal/common/include/rte_dev.h index f7e3a10..e63dd1c 100644 --- a/lib/librte_eal/common/include/rte_dev.h +++ b/lib/librte_eal/common/include/rte_dev.h @@ -47,6 +47,7 @@ extern "C" { #endif #include <sys/queue.h> +#include <rte_pci.h> /** Double linked list of device drivers. */ TAILQ_HEAD(rte_driver_list, rte_driver); @@ -57,6 +58,11 @@ TAILQ_HEAD(rte_driver_list, rte_driver); typedef int (rte_dev_init_t)(const char *name, const char *args); /** + * Uninitilization function called for each device driver once. + */ +typedef int (rte_dev_uninit_t)(const char *name, const char *args); + +/** * Driver type enumeration */ enum pmd_type { @@ -72,6 +78,7 @@ struct rte_driver { enum pmd_type type; /**< PMD Driver type */ const char *name; /**< Driver name. */ rte_dev_init_t *init; /**< Device init. function. */ + rte_dev_uninit_t *uninit; /**< Device uninit. function. */ }; /** @@ -93,6 +100,32 @@ void rte_eal_driver_register(struct rte_driver *driver); void rte_eal_driver_unregister(struct rte_driver *driver); /** + * Attach a new device. + * + * @param devargs + * A pointer to a strings array describing the new device + * to be attached. The strings should be a pci address like + * '0000:01:00.0' or virtual device name like 'eth_pcap0'. + * @param port_id + * A pointer to a port identifier actually attached. + * @return + * 0 on success and port_id is filled, negative on error + */ +int rte_eal_dev_attach(const char *devargs, uint8_t *port_id); + +/** + * Detach a device. + * + * @param port_id + * The port identifier of the device to detach. + * @param addr + * A pointer to a device name actually detached. + * @return + * 0 on success and devname is filled, negative on error + */ +int rte_eal_dev_detach(uint8_t port_id, char *devname); + +/** * Initalize all the registered drivers in this process */ int rte_eal_dev_init(void); diff --git a/lib/librte_eal/linuxapp/eal/Makefile b/lib/librte_eal/linuxapp/eal/Makefile index 72ecf3a..0ec83b5 100644 --- a/lib/librte_eal/linuxapp/eal/Makefile +++ b/lib/librte_eal/linuxapp/eal/Makefile @@ -41,6 +41,7 @@ CFLAGS += -I$(RTE_SDK)/lib/librte_eal/common/include CFLAGS += -I$(RTE_SDK)/lib/librte_ring CFLAGS += -I$(RTE_SDK)/lib/librte_mempool CFLAGS += -I$(RTE_SDK)/lib/librte_malloc +CFLAGS += -I$(RTE_SDK)/lib/librte_mbuf CFLAGS += -I$(RTE_SDK)/lib/librte_ether CFLAGS += -I$(RTE_SDK)/lib/librte_ivshmem CFLAGS += -I$(RTE_SDK)/lib/librte_pmd_ring diff --git a/lib/librte_eal/linuxapp/eal/eal_pci.c b/lib/librte_eal/linuxapp/eal/eal_pci.c index a23cc59..8e7e650 100644 --- a/lib/librte_eal/linuxapp/eal/eal_pci.c +++ b/lib/librte_eal/linuxapp/eal/eal_pci.c @@ -378,8 +378,8 @@ error: * Scan the content of the PCI bus, and the devices in the devices * list */ -static int -pci_scan(void) +int +rte_eal_pci_scan(void) { struct dirent *e; DIR *dir; @@ -701,7 +701,7 @@ rte_eal_pci_init(void) if (internal_config.no_pci) return 0; - if (pci_scan() < 0) { + if (rte_eal_pci_scan() < 0) { RTE_LOG(ERR, EAL, "%s(): Cannot scan PCI bus\n", __func__); return -1; } -- 1.9.1
The patch enables CONFIG_RTE_LIBRTE_EAL_HOTPLUG in Linux configuration. Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp> --- config/common_linuxapp | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/config/common_linuxapp b/config/common_linuxapp index 2f9643b..27d05be 100644 --- a/config/common_linuxapp +++ b/config/common_linuxapp @@ -114,6 +114,11 @@ CONFIG_RTE_PCI_MAX_READ_REQUEST_SIZE=0 CONFIG_RTE_LIBRTE_EAL_LINUXAPP=y # +# Compile Environment Abstraction Layer to support hotplug +# +CONFIG_RTE_LIBRTE_EAL_HOTPLUG=y + +# # Compile Environment Abstraction Layer to support Vmware TSC map # CONFIG_RTE_LIBRTE_EAL_VMWARE_TSC_MAP_SUPPORT=y -- 1.9.1
This patch adds finalization code to free resources allocated by the PMD. v4: - Change function name. Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp> --- lib/librte_pmd_pcap/rte_eth_pcap.c | 40 ++++++++++++++++++++++++++++++++++++++ 1 file changed, 40 insertions(+) diff --git a/lib/librte_pmd_pcap/rte_eth_pcap.c b/lib/librte_pmd_pcap/rte_eth_pcap.c index 5da92a3..13a68de 100644 --- a/lib/librte_pmd_pcap/rte_eth_pcap.c +++ b/lib/librte_pmd_pcap/rte_eth_pcap.c @@ -499,6 +499,13 @@ static struct eth_dev_ops ops = { .stats_reset = eth_stats_reset, }; +static struct eth_driver rte_pcap_pmd = { + .pci_drv = { + .name = "rte_pcap_pmd", + .drv_flags = RTE_PCI_DRV_DETACHABLE, + }, +}; + /* * Function handler that opens the pcap file for reading a stores a * reference of it for use it later on. @@ -714,6 +721,10 @@ rte_pmd_init_internals(const char *name, const unsigned nb_rx_queues, if (*eth_dev == NULL) goto error; + /* check length of device name */ + if ((strlen((*eth_dev)->data->name) + 1) > sizeof(data->name)) + goto error; + /* now put it all together * - store queue data in internals, * - store numa_node info in pci_driver @@ -739,10 +750,13 @@ rte_pmd_init_internals(const char *name, const unsigned nb_rx_queues, data->nb_tx_queues = (uint16_t)nb_tx_queues; data->dev_link = pmd_link; data->mac_addrs = ð_addr; + strncpy(data->name, + (*eth_dev)->data->name, strlen((*eth_dev)->data->name)); (*eth_dev)->data = data; (*eth_dev)->dev_ops = &ops; (*eth_dev)->pci_dev = pci_dev; + (*eth_dev)->driver = &rte_pcap_pmd; return 0; @@ -927,10 +941,36 @@ rte_pmd_pcap_devinit(const char *name, const char *params) } +static int +rte_pmd_pcap_devuninit(const char *name, const char *params __rte_unused) +{ + struct rte_eth_dev *eth_dev = NULL; + + RTE_LOG(INFO, PMD, "Closing pcap ethdev on numa socket %u\n", + rte_socket_id()); + + if (name == NULL) + return -1; + + /* reserve an ethdev entry */ + eth_dev = rte_eth_dev_allocated(name); + if (eth_dev == NULL) + return -1; + + rte_free(eth_dev->data->dev_private); + rte_free(eth_dev->data); + rte_free(eth_dev->pci_dev); + + rte_eth_dev_free(name); + + return 0; +} + static struct rte_driver pmd_pcap_drv = { .name = "eth_pcap", .type = PMD_VDEV, .init = rte_pmd_pcap_devinit, + .uninit = rte_pmd_pcap_devuninit, }; PMD_REGISTER_DRIVER(pmd_pcap_drv); -- 1.9.1
The patch introduces following commands. - port attach [ident] - port detach [port_id] - attach: attaching a port - detach: detaching a port - ident: pci address of physical device. Or device name and paramerters of virtual device. (ex. 0000:02:00.0, eth_pcap0,iface=eth0) - port_id: port identifier v4: - Fix strings of command help. Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp> --- app/test-pmd/cmdline.c | 133 +++++++++++++++++++++++++------ app/test-pmd/config.c | 116 +++++++++++++++------------ app/test-pmd/parameters.c | 22 +++-- app/test-pmd/testpmd.c | 199 +++++++++++++++++++++++++++++++++++----------- app/test-pmd/testpmd.h | 18 ++++- 5 files changed, 358 insertions(+), 130 deletions(-) diff --git a/app/test-pmd/cmdline.c b/app/test-pmd/cmdline.c index 4618b92..3557dff 100644 --- a/app/test-pmd/cmdline.c +++ b/app/test-pmd/cmdline.c @@ -572,6 +572,12 @@ static void cmd_help_long_parsed(void *parsed_result, "port close (port_id|all)\n" " Close all ports or port_id.\n\n" + "port attach (ident)\n" + " Attach physical or virtual dev by pci address or virtual device name\n\n" + + "port detach (port_id)\n" + " Detach physical or virtual dev by port_id\n\n" + "port config (port_id|all)" " speed (10|100|1000|10000|40000|auto)" " duplex (half|full|auto)\n" @@ -853,6 +859,89 @@ cmdline_parse_inst_t cmd_operate_specific_port = { }, }; +/* *** attach a specificied port *** */ +struct cmd_operate_attach_port_result { + cmdline_fixed_string_t port; + cmdline_fixed_string_t keyword; + cmdline_fixed_string_t identifier; +}; + +static void cmd_operate_attach_port_parsed(void *parsed_result, + __attribute__((unused)) struct cmdline *cl, + __attribute__((unused)) void *data) +{ + struct cmd_operate_attach_port_result *res = parsed_result; + + if (!strcmp(res->keyword, "attach")) + attach_port(res->identifier); + else + printf("Unknown parameter\n"); +} + +cmdline_parse_token_string_t cmd_operate_attach_port_port = + TOKEN_STRING_INITIALIZER(struct cmd_operate_attach_port_result, + port, "port"); +cmdline_parse_token_string_t cmd_operate_attach_port_keyword = + TOKEN_STRING_INITIALIZER(struct cmd_operate_attach_port_result, + keyword, "attach"); +cmdline_parse_token_string_t cmd_operate_attach_port_identifier = + TOKEN_STRING_INITIALIZER(struct cmd_operate_attach_port_result, + identifier, NULL); + +cmdline_parse_inst_t cmd_operate_attach_port = { + .f = cmd_operate_attach_port_parsed, + .data = NULL, + .help_str = "port attach identifier, " + "identifier: pci address or virtual dev name", + .tokens = { + (void *)&cmd_operate_attach_port_port, + (void *)&cmd_operate_attach_port_keyword, + (void *)&cmd_operate_attach_port_identifier, + NULL, + }, +}; + +/* *** detach a specificied port *** */ +struct cmd_operate_detach_port_result { + cmdline_fixed_string_t port; + cmdline_fixed_string_t keyword; + uint8_t port_id; +}; + +static void cmd_operate_detach_port_parsed(void *parsed_result, + __attribute__((unused)) struct cmdline *cl, + __attribute__((unused)) void *data) +{ + struct cmd_operate_detach_port_result *res = parsed_result; + + if (!strcmp(res->keyword, "detach")) + detach_port(res->port_id); + else + printf("Unknown parameter\n"); +} + +cmdline_parse_token_string_t cmd_operate_detach_port_port = + TOKEN_STRING_INITIALIZER(struct cmd_operate_detach_port_result, + port, "port"); +cmdline_parse_token_string_t cmd_operate_detach_port_keyword = + TOKEN_STRING_INITIALIZER(struct cmd_operate_detach_port_result, + keyword, "detach"); +cmdline_parse_token_num_t cmd_operate_detach_port_port_id = + TOKEN_NUM_INITIALIZER(struct cmd_operate_detach_port_result, + port_id, UINT8); + +cmdline_parse_inst_t cmd_operate_detach_port = { + .f = cmd_operate_detach_port_parsed, + .data = NULL, + .help_str = "port detach port_id", + .tokens = { + (void *)&cmd_operate_detach_port_port, + (void *)&cmd_operate_detach_port_keyword, + (void *)&cmd_operate_detach_port_port_id, + NULL, + }, +}; + /* *** configure speed for all ports *** */ struct cmd_config_speed_all { cmdline_fixed_string_t port; @@ -907,7 +996,7 @@ cmd_config_speed_all_parsed(void *parsed_result, return; } - for (pid = 0; pid < nb_ports; pid++) { + FOREACH_PORT(pid, ports) { ports[pid].dev_conf.link_speed = link_speed; ports[pid].dev_conf.link_duplex = link_duplex; } @@ -975,10 +1064,8 @@ cmd_config_speed_specific_parsed(void *parsed_result, return; } - if (res->id >= nb_ports) { - printf("Port id %d must be less than %d\n", res->id, nb_ports); + if (port_id_is_invalid(res->id, ENABLED_WARN)) return; - } if (!strcmp(res->value1, "10")) link_speed = ETH_LINK_SPEED_10; @@ -1538,7 +1625,7 @@ cmd_config_rxtx_queue_parsed(void *parsed_result, return; } - if (port_id_is_invalid(res->portid)) + if (port_id_is_invalid(res->portid, ENABLED_WARN)) return; if (port_is_started(res->portid) != 1) { @@ -2881,7 +2968,7 @@ cmd_tx_cksum_parsed(void *parsed_result, uint16_t ol_flags, mask = 0; struct rte_eth_dev_info dev_info; - if (port_id_is_invalid(res->port_id)) { + if (port_id_is_invalid(res->port_id, ENABLED_WARN)) { printf("invalid port %d\n", res->port_id); return; } @@ -3008,7 +3095,7 @@ cmd_tso_set_parsed(void *parsed_result, struct cmd_tso_set_result *res = parsed_result; struct rte_eth_dev_info dev_info; - if (port_id_is_invalid(res->port_id)) + if (port_id_is_invalid(res->port_id, ENABLED_WARN)) return; if (!strcmp(res->mode, "set")) @@ -3984,10 +4071,8 @@ static void cmd_set_bond_mac_addr_parsed(void *parsed_result, struct cmd_set_bond_mac_addr_result *res = parsed_result; int ret; - if (res->port_num >= nb_ports) { - printf("Port id %d must be less than %d\n", res->port_num, nb_ports); + if (port_id_is_invalid(res->port_num, ENABLED_WARN)) return; - } ret = rte_eth_bond_mac_address_set(res->port_num, &res->address); @@ -4224,7 +4309,7 @@ static void cmd_set_promisc_mode_parsed(void *parsed_result, /* all ports */ if (allports) { - for (i = 0; i < nb_ports; i++) { + FOREACH_PORT(i, ports) { if (enable) rte_eth_promiscuous_enable(i); else @@ -4304,7 +4389,7 @@ static void cmd_set_allmulti_mode_parsed(void *parsed_result, /* all ports */ if (allports) { - for (i = 0; i < nb_ports; i++) { + FOREACH_PORT(i, ports) { if (enable) rte_eth_allmulticast_enable(i); else @@ -5489,25 +5574,25 @@ static void cmd_showportall_parsed(void *parsed_result, struct cmd_showportall_result *res = parsed_result; if (!strcmp(res->show, "clear")) { if (!strcmp(res->what, "stats")) - for (i = 0; i < nb_ports; i++) + FOREACH_PORT(i, ports) nic_stats_clear(i); else if (!strcmp(res->what, "xstats")) - for (i = 0; i < nb_ports; i++) + FOREACH_PORT(i, ports) nic_xstats_clear(i); } else if (!strcmp(res->what, "info")) - for (i = 0; i < nb_ports; i++) + FOREACH_PORT(i, ports) port_infos_display(i); else if (!strcmp(res->what, "stats")) - for (i = 0; i < nb_ports; i++) + FOREACH_PORT(i, ports) nic_stats_display(i); else if (!strcmp(res->what, "xstats")) - for (i = 0; i < nb_ports; i++) + FOREACH_PORT(i, ports) nic_xstats_display(i); else if (!strcmp(res->what, "fdir")) - for (i = 0; i < nb_ports; i++) + FOREACH_PORT(i, ports) fdir_get_infos(i); else if (!strcmp(res->what, "stat_qmap")) - for (i = 0; i < nb_ports; i++) + FOREACH_PORT(i, ports) nic_stats_mapping_display(i); } @@ -8783,6 +8868,8 @@ cmdline_parse_ctx_t main_ctx[] = { (cmdline_parse_inst_t *)&cmd_set_qmap, (cmdline_parse_inst_t *)&cmd_operate_port, (cmdline_parse_inst_t *)&cmd_operate_specific_port, + (cmdline_parse_inst_t *)&cmd_operate_attach_port, + (cmdline_parse_inst_t *)&cmd_operate_detach_port, (cmdline_parse_inst_t *)&cmd_config_speed_all, (cmdline_parse_inst_t *)&cmd_config_speed_specific, (cmdline_parse_inst_t *)&cmd_config_rx_tx, @@ -8859,7 +8946,7 @@ prompt(void) static void cmd_reconfig_device_queue(portid_t id, uint8_t dev, uint8_t queue) { - if (id < nb_ports) { + if (!port_id_is_invalid(id, DISABLED_WARN)) { /* check if need_reconfig has been set to 1 */ if (ports[id].need_reconfig == 0) ports[id].need_reconfig = dev; @@ -8869,7 +8956,7 @@ cmd_reconfig_device_queue(portid_t id, uint8_t dev, uint8_t queue) } else { portid_t pid; - for (pid = 0; pid < nb_ports; pid++) { + FOREACH_PORT(pid, ports) { /* check if need_reconfig has been set to 1 */ if (ports[pid].need_reconfig == 0) ports[pid].need_reconfig = dev; @@ -8887,10 +8974,8 @@ bypass_is_supported(portid_t port_id) struct rte_port *port; struct rte_pci_id *pci_id; - if (port_id >= nb_ports) { - printf("\tPort id must be less than %d.\n", nb_ports); + if (port_id_is_invalid(port_id, ENABLED_WARN)) return 0; - } /* Get the device id. */ port = &ports[port_id]; diff --git a/app/test-pmd/config.c b/app/test-pmd/config.c index 97b6525..2662d7c 100644 --- a/app/test-pmd/config.c +++ b/app/test-pmd/config.c @@ -124,11 +124,15 @@ nic_stats_display(portid_t port_id) struct rte_eth_stats stats; struct rte_port *port = &ports[port_id]; uint8_t i; + portid_t pid; static const char *nic_stats_border = "########################"; - if (port_id >= nb_ports) { - printf("Invalid port, range is [0, %d]\n", nb_ports - 1); + if (port_id_is_invalid(port_id, ENABLED_WARN)) { + printf("Valid port range is [0"); + FOREACH_PORT(pid, ports) + printf(", %d", pid); + printf("]\n"); return; } rte_eth_stats_get(port_id, &stats); @@ -201,8 +205,13 @@ nic_stats_display(portid_t port_id) void nic_stats_clear(portid_t port_id) { - if (port_id >= nb_ports) { - printf("Invalid port, range is [0, %d]\n", nb_ports - 1); + portid_t pid; + + if (port_id_is_invalid(port_id, ENABLED_WARN)) { + printf("Valid port range is [0"); + FOREACH_PORT(pid, ports) + printf(", %d", pid); + printf("]\n"); return; } rte_eth_stats_reset(port_id); @@ -249,11 +258,15 @@ nic_stats_mapping_display(portid_t port_id) { struct rte_port *port = &ports[port_id]; uint16_t i; + portid_t pid; static const char *nic_stats_mapping_border = "########################"; - if (port_id >= nb_ports) { - printf("Invalid port, range is [0, %d]\n", nb_ports - 1); + if (port_id_is_invalid(port_id, ENABLED_WARN)) { + printf("Valid port range is [0"); + FOREACH_PORT(pid, ports) + printf(", %d", pid); + printf("]\n"); return; } @@ -302,9 +315,13 @@ port_infos_display(portid_t port_id) int vlan_offload; struct rte_mempool * mp; static const char *info_border = "*********************"; + portid_t pid; - if (port_id >= nb_ports) { - printf("Invalid port, range is [0, %d]\n", nb_ports - 1); + if (port_id_is_invalid(port_id, ENABLED_WARN)) { + printf("Valid port range is [0"); + FOREACH_PORT(pid, ports) + printf(", %d", pid); + printf("]\n"); return; } port = &ports[port_id]; @@ -362,11 +379,14 @@ port_infos_display(portid_t port_id) } int -port_id_is_invalid(portid_t port_id) +port_id_is_invalid(portid_t port_id, enum print_warning warning) { - if (port_id < nb_ports) + if (ports[port_id].enabled) return 0; - printf("Invalid port %d (must be < nb_ports=%d)\n", port_id, nb_ports); + + if (warning == ENABLED_WARN) + printf("Invalid port %d\n", port_id); + return 1; } @@ -425,7 +445,7 @@ port_reg_bit_display(portid_t port_id, uint32_t reg_off, uint8_t bit_x) uint32_t reg_v; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; if (port_reg_off_is_invalid(port_id, reg_off)) return; @@ -444,7 +464,7 @@ port_reg_bit_field_display(portid_t port_id, uint32_t reg_off, uint8_t l_bit; uint8_t h_bit; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; if (port_reg_off_is_invalid(port_id, reg_off)) return; @@ -471,7 +491,7 @@ port_reg_display(portid_t port_id, uint32_t reg_off) { uint32_t reg_v; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; if (port_reg_off_is_invalid(port_id, reg_off)) return; @@ -485,7 +505,7 @@ port_reg_bit_set(portid_t port_id, uint32_t reg_off, uint8_t bit_pos, { uint32_t reg_v; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; if (port_reg_off_is_invalid(port_id, reg_off)) return; @@ -513,7 +533,7 @@ port_reg_bit_field_set(portid_t port_id, uint32_t reg_off, uint8_t l_bit; uint8_t h_bit; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; if (port_reg_off_is_invalid(port_id, reg_off)) return; @@ -547,7 +567,7 @@ port_reg_bit_field_set(portid_t port_id, uint32_t reg_off, void port_reg_set(portid_t port_id, uint32_t reg_off, uint32_t reg_v) { - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; if (port_reg_off_is_invalid(port_id, reg_off)) return; @@ -560,7 +580,7 @@ port_mtu_set(portid_t port_id, uint16_t mtu) { int diag; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; diag = rte_eth_dev_set_mtu(port_id, mtu); if (diag == 0) @@ -723,7 +743,7 @@ rx_ring_desc_display(portid_t port_id, queueid_t rxq_id, uint16_t rxd_id) { const struct rte_memzone *rx_mz; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; if (rx_queue_id_is_invalid(rxq_id)) return; @@ -740,7 +760,7 @@ tx_ring_desc_display(portid_t port_id, queueid_t txq_id, uint16_t txd_id) { const struct rte_memzone *tx_mz; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; if (tx_queue_id_is_invalid(txq_id)) return; @@ -796,7 +816,7 @@ port_rss_reta_info(portid_t port_id, uint16_t i, idx, shift; int ret; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; ret = rte_eth_dev_rss_reta_query(port_id, reta_conf, nb_entries); @@ -828,7 +848,7 @@ port_rss_hash_conf_show(portid_t port_id, int show_rss_key) uint8_t i; int diag; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; /* Get RSS hash key if asked to display it */ rss_conf.rss_key = (show_rss_key) ? rss_key : NULL; @@ -1406,12 +1426,8 @@ set_fwd_ports_list(unsigned int *portlist, unsigned int nb_pt) again: for (i = 0; i < nb_pt; i++) { port_id = (portid_t) portlist[i]; - if (port_id >= nb_ports) { - printf("Invalid port id %u >= %u\n", - (unsigned int) port_id, - (unsigned int) nb_ports); + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; - } if (record_now) fwd_ports_ids[i] = port_id; } @@ -1569,7 +1585,7 @@ vlan_extend_set(portid_t port_id, int on) int diag; int vlan_offload; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; vlan_offload = rte_eth_dev_get_vlan_offload(port_id); @@ -1591,7 +1607,7 @@ rx_vlan_strip_set(portid_t port_id, int on) int diag; int vlan_offload; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; vlan_offload = rte_eth_dev_get_vlan_offload(port_id); @@ -1612,7 +1628,7 @@ rx_vlan_strip_set_on_queue(portid_t port_id, uint16_t queue_id, int on) { int diag; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; diag = rte_eth_dev_set_vlan_strip_on_queue(port_id, queue_id, on); @@ -1627,7 +1643,7 @@ rx_vlan_filter_set(portid_t port_id, int on) int diag; int vlan_offload; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; vlan_offload = rte_eth_dev_get_vlan_offload(port_id); @@ -1648,7 +1664,7 @@ rx_vft_set(portid_t port_id, uint16_t vlan_id, int on) { int diag; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; if (vlan_id_is_invalid(vlan_id)) return; @@ -1665,7 +1681,7 @@ rx_vlan_all_filter_set(portid_t port_id, int on) { uint16_t vlan_id; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; for (vlan_id = 0; vlan_id < 4096; vlan_id++) rx_vft_set(port_id, vlan_id, on); @@ -1675,7 +1691,7 @@ void vlan_tpid_set(portid_t port_id, uint16_t tp_id) { int diag; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; diag = rte_eth_dev_set_vlan_ether_type(port_id, tp_id); @@ -1690,7 +1706,7 @@ vlan_tpid_set(portid_t port_id, uint16_t tp_id) void tx_vlan_set(portid_t port_id, uint16_t vlan_id) { - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; if (vlan_id_is_invalid(vlan_id)) return; @@ -1701,7 +1717,7 @@ tx_vlan_set(portid_t port_id, uint16_t vlan_id) void tx_vlan_reset(portid_t port_id) { - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; ports[port_id].tx_ol_flags &= ~TESTPMD_TX_OFFLOAD_INSERT_VLAN; } @@ -1709,7 +1725,7 @@ tx_vlan_reset(portid_t port_id) void tx_vlan_pvid_set(portid_t port_id, uint16_t vlan_id, int on) { - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; rte_eth_dev_set_vlan_pvid(port_id, vlan_id, on); @@ -1721,7 +1737,7 @@ set_qmap(portid_t port_id, uint8_t is_rx, uint16_t queue_id, uint8_t map_value) uint16_t i; uint8_t existing_mapping_found = 0; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; if (is_rx ? (rx_queue_id_is_invalid(queue_id)) : (tx_queue_id_is_invalid(queue_id))) @@ -1773,7 +1789,7 @@ fdir_add_signature_filter(portid_t port_id, uint8_t queue_id, { int diag; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; diag = rte_eth_dev_fdir_add_signature_filter(port_id, fdir_filter, @@ -1791,7 +1807,7 @@ fdir_update_signature_filter(portid_t port_id, uint8_t queue_id, { int diag; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; diag = rte_eth_dev_fdir_update_signature_filter(port_id, fdir_filter, @@ -1809,7 +1825,7 @@ fdir_remove_signature_filter(portid_t port_id, { int diag; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; diag = rte_eth_dev_fdir_remove_signature_filter(port_id, fdir_filter); @@ -1881,7 +1897,7 @@ fdir_get_infos(portid_t port_id) static const char *fdir_stats_border = "########################"; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; ret = rte_eth_dev_filter_supported(port_id, RTE_ETH_FILTER_FDIR); if (ret < 0) { @@ -1955,7 +1971,7 @@ fdir_add_perfect_filter(portid_t port_id, uint16_t soft_id, uint8_t queue_id, { int diag; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; diag = rte_eth_dev_fdir_add_perfect_filter(port_id, fdir_filter, @@ -1973,7 +1989,7 @@ fdir_update_perfect_filter(portid_t port_id, uint16_t soft_id, uint8_t queue_id, { int diag; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; diag = rte_eth_dev_fdir_update_perfect_filter(port_id, fdir_filter, @@ -1991,7 +2007,7 @@ fdir_remove_perfect_filter(portid_t port_id, uint16_t soft_id, { int diag; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; diag = rte_eth_dev_fdir_remove_perfect_filter(port_id, fdir_filter, @@ -2008,7 +2024,7 @@ fdir_set_masks(portid_t port_id, struct rte_fdir_masks *fdir_masks) { int diag; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; diag = rte_eth_dev_fdir_set_masks(port_id, fdir_masks); @@ -2085,7 +2101,7 @@ set_vf_traffic(portid_t port_id, uint8_t is_rx, uint16_t vf, uint8_t on) { int diag; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; if (is_rx) diag = rte_eth_dev_set_vf_rx(port_id,vf,on); @@ -2107,7 +2123,7 @@ set_vf_rx_vlan(portid_t port_id, uint16_t vlan_id, uint64_t vf_mask, uint8_t on) { int diag; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; if (vlan_id_is_invalid(vlan_id)) return; @@ -2124,7 +2140,7 @@ set_queue_rate_limit(portid_t port_id, uint16_t queue_idx, uint16_t rate) int diag; struct rte_eth_link link; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return 1; rte_eth_link_get_nowait(port_id, &link); if (rate > link.link_speed) { @@ -2149,7 +2165,7 @@ set_vf_rate_limit(portid_t port_id, uint16_t vf, uint16_t rate, uint64_t q_msk) if (q_msk == 0) return 0; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return 1; rte_eth_link_get_nowait(port_id, &link); if (rate > link.link_speed) { diff --git a/app/test-pmd/parameters.c b/app/test-pmd/parameters.c index adf3203..6f2af18 100644 --- a/app/test-pmd/parameters.c +++ b/app/test-pmd/parameters.c @@ -376,6 +376,7 @@ parse_portnuma_config(const char *q_arg) }; unsigned long int_fld[_NUM_FLD]; char *str_fld[_NUM_FLD]; + portid_t pid; /* reset from value set at definition */ while ((p = strchr(p0,'(')) != NULL) { @@ -397,8 +398,11 @@ parse_portnuma_config(const char *q_arg) return -1; } port_id = (uint8_t)int_fld[FLD_PORT]; - if (port_id >= nb_ports) { - printf("Invalid port, range is [0, %d]\n", nb_ports - 1); + if (port_id_is_invalid(port_id, ENABLED_WARN)) { + printf("Valid port range is [0"); + FOREACH_PORT(pid, ports) + printf(", %d", pid); + printf("]\n"); return -1; } socket_id = (uint8_t)int_fld[FLD_SOCKET]; @@ -429,6 +433,7 @@ parse_ringnuma_config(const char *q_arg) }; unsigned long int_fld[_NUM_FLD]; char *str_fld[_NUM_FLD]; + portid_t pid; #define RX_RING_ONLY 0x1 #define TX_RING_ONLY 0x2 #define RXTX_RING 0x3 @@ -453,8 +458,11 @@ parse_ringnuma_config(const char *q_arg) return -1; } port_id = (uint8_t)int_fld[FLD_PORT]; - if (port_id >= nb_ports) { - printf("Invalid port, range is [0, %d]\n", nb_ports - 1); + if (port_id_is_invalid(port_id, ENABLED_WARN)) { + printf("Valid port range is [0"); + FOREACH_PORT(pid, ports) + printf(", %d", pid); + printf("]\n"); return -1; } socket_id = (uint8_t)int_fld[FLD_SOCKET]; @@ -626,12 +634,12 @@ launch_args_parse(int argc, char** argv) #endif if (!strcmp(lgopts[opt_idx].name, "nb-ports")) { n = atoi(optarg); - if (n > 0 && n <= nb_ports) + if (n > 0 && + !port_id_is_invalid(n, DISABLED_WARN)) nb_fwd_ports = (uint8_t) n; else rte_exit(EXIT_FAILURE, - "nb-ports should be > 0 and <= %d\n", - nb_ports); + "Invalid port %d\n", n); } if (!strcmp(lgopts[opt_idx].name, "nb-cores")) { n = atoi(optarg); diff --git a/app/test-pmd/testpmd.c b/app/test-pmd/testpmd.c index 773b8af..c18c1a9 100644 --- a/app/test-pmd/testpmd.c +++ b/app/test-pmd/testpmd.c @@ -71,6 +71,7 @@ #include <rte_pci.h> #include <rte_ether.h> #include <rte_ethdev.h> +#include <rte_dev.h> #include <rte_string_fns.h> #ifdef RTE_LIBRTE_PMD_XENVIRT #include <rte_eth_xenvirt.h> @@ -315,7 +316,7 @@ uint16_t nb_rx_queue_stats_mappings = 0; /* Forward function declarations */ static void map_port_queue_stats_mapping_registers(uint8_t pi, struct rte_port *port); -static void check_all_ports_link_status(uint8_t port_num, uint32_t port_mask); +static void check_all_ports_link_status(uint32_t port_mask); /* * Check if all the ports are started. @@ -324,6 +325,20 @@ static void check_all_ports_link_status(uint8_t port_num, uint32_t port_mask); static int all_ports_started(void); /* + * Find next enabled port + */ +portid_t +find_next_port(portid_t p, struct rte_port *ports, int size) +{ + if (ports == NULL) + rte_exit(-EINVAL, "failed to find a next port id\n"); + + while ((ports[p].enabled == 0) && (p < size)) + p++; + return p; +} + +/* * Setup default configuration. */ static void @@ -552,7 +567,8 @@ init_config(void) + RTE_TEST_TX_DESC_MAX + MAX_PKT_BURST; if (!numa_support) - nb_mbuf_per_pool = (nb_mbuf_per_pool * nb_ports); + nb_mbuf_per_pool = + (nb_mbuf_per_pool * RTE_MAX_ETHPORTS); } if (!numa_support) { @@ -565,14 +581,19 @@ init_config(void) /* Configuration of Ethernet ports. */ ports = rte_zmalloc("testpmd: ports", - sizeof(struct rte_port) * nb_ports, + sizeof(struct rte_port) * RTE_MAX_ETHPORTS, RTE_CACHE_LINE_SIZE); if (ports == NULL) { - rte_exit(EXIT_FAILURE, "rte_zmalloc(%d struct rte_port) " - "failed\n", nb_ports); + rte_exit(EXIT_FAILURE, + "rte_zmalloc(%d struct rte_port) failed\n", + RTE_MAX_ETHPORTS); } - for (pid = 0; pid < nb_ports; pid++) { + /* enabled allocated ports */ + for (pid = 0; pid < nb_ports; pid++) + ports[pid].enabled = 1; + + FOREACH_PORT(pid, ports) { port = &ports[pid]; rte_eth_dev_info_get(pid, &port->dev_info); @@ -602,8 +623,7 @@ init_config(void) nb_mbuf_per_pool = nb_mbuf_per_pool/nb_ports; for (i = 0; i < MAX_SOCKET; i++) { - nb_mbuf = (nb_mbuf_per_pool * - port_per_socket[i]); + nb_mbuf = (nb_mbuf_per_pool * RTE_MAX_ETHPORTS); if (nb_mbuf) mbuf_pool_create(mbuf_data_size, nb_mbuf,i); @@ -635,14 +655,6 @@ reconfig(portid_t new_port_id, unsigned socket_id) struct rte_port *port; /* Reconfiguration of Ethernet ports. */ - ports = rte_realloc(ports, - sizeof(struct rte_port) * nb_ports, - RTE_CACHE_LINE_SIZE); - if (ports == NULL) { - rte_exit(EXIT_FAILURE, "rte_realloc(%d struct rte_port) failed\n", - nb_ports); - } - port = &ports[new_port_id]; rte_eth_dev_info_get(new_port_id, &port->dev_info); @@ -663,7 +675,7 @@ init_fwd_streams(void) streamid_t sm_id, nb_fwd_streams_new; /* set socket id according to numa or not */ - for (pid = 0; pid < nb_ports; pid++) { + FOREACH_PORT(pid, ports) { port = &ports[pid]; if (nb_rxq > port->dev_info.max_rx_queues) { printf("Fail: nb_rxq(%d) is greater than " @@ -1264,7 +1276,7 @@ all_ports_started(void) portid_t pi; struct rte_port *port; - for (pi = 0; pi < nb_ports; pi++) { + FOREACH_PORT(pi, ports) { port = &ports[pi]; /* Check if there is a port which is not started */ if (port->port_status != RTE_PORT_STARTED) @@ -1276,6 +1288,45 @@ all_ports_started(void) } int +all_ports_stopped(void) +{ + portid_t pi; + struct rte_port *port; + + FOREACH_PORT(pi, ports) { + port = &ports[pi]; + if (port->port_status != RTE_PORT_STOPPED) + return 0; + } + + return 1; +} + +int +port_is_started(portid_t port_id) +{ + if (port_id_is_invalid(port_id, ENABLED_WARN)) + return 0; + + if (ports[port_id].port_status != RTE_PORT_STARTED) + return 0; + + return 1; +} + +static int +port_is_closed(portid_t port_id) +{ + if (port_id_is_invalid(port_id, ENABLED_WARN)) + return 0; + + if (ports[port_id].port_status != RTE_PORT_CLOSED) + return 0; + + return 1; +} + +int start_port(portid_t pid) { int diag, need_check_link_status = 0; @@ -1296,8 +1347,8 @@ start_port(portid_t pid) if(dcb_config) dcb_test = 1; - for (pi = 0; pi < nb_ports; pi++) { - if (pid < nb_ports && pid != pi) + FOREACH_PORT(pi, ports) { + if (!port_id_is_invalid(pid, DISABLED_WARN) && pid != pi) continue; port = &ports[pi]; @@ -1421,7 +1472,7 @@ start_port(portid_t pid) } if (need_check_link_status && !no_link_check) - check_all_ports_link_status(nb_ports, RTE_PORT_ALL); + check_all_ports_link_status(RTE_PORT_ALL); else printf("Please stop the ports first\n"); @@ -1446,8 +1497,8 @@ stop_port(portid_t pid) } printf("Stopping ports...\n"); - for (pi = 0; pi < nb_ports; pi++) { - if (pid < nb_ports && pid != pi) + FOREACH_PORT(pi, ports) { + if (!port_id_is_invalid(pid, DISABLED_WARN) && pid != pi) continue; port = &ports[pi]; @@ -1463,7 +1514,7 @@ stop_port(portid_t pid) need_check_link_status = 1; } if (need_check_link_status && !no_link_check) - check_all_ports_link_status(nb_ports, RTE_PORT_ALL); + check_all_ports_link_status(RTE_PORT_ALL); printf("Done\n"); } @@ -1481,8 +1532,8 @@ close_port(portid_t pid) printf("Closing ports...\n"); - for (pi = 0; pi < nb_ports; pi++) { - if (pid < nb_ports && pid != pi) + FOREACH_PORT(pi, ports) { + if (!port_id_is_invalid(pid, DISABLED_WARN) && pid != pi) continue; port = &ports[pi]; @@ -1502,31 +1553,83 @@ close_port(portid_t pid) printf("Done\n"); } -int -all_ports_stopped(void) +void +attach_port(char *identifier) { - portid_t pi; - struct rte_port *port; + portid_t i, j, pi = 0; - for (pi = 0; pi < nb_ports; pi++) { - port = &ports[pi]; - if (port->port_status != RTE_PORT_STOPPED) - return 0; + printf("Attaching a new port...\n"); + + if (identifier == NULL) { + printf("Invalid parameters are speficied\n"); + return; } - return 1; + if (test_done == 0) { + printf("Please stop forwarding first\n"); + return; + } + + if (rte_eal_dev_attach(identifier, &pi)) + return; + + ports[pi].enabled = 1; + reconfig(pi, rte_eth_dev_socket_id(pi)); + rte_eth_promiscuous_enable(pi); + + nb_ports = rte_eth_dev_count(); + + /* set_default_fwd_ports_config(); */ + bzero(fwd_ports_ids, sizeof(fwd_ports_ids)); + i = 0; + FOREACH_PORT(j, ports) { + fwd_ports_ids[i] = j; + i++; + } + nb_cfg_ports = nb_ports; + nb_fwd_ports++; + + ports[pi].port_status = RTE_PORT_STOPPED; + + printf("Port %d is attached. Now total ports is %d\n", pi, nb_ports); + printf("Done\n"); } -int -port_is_started(portid_t port_id) +void +detach_port(uint8_t port_id) { - if (port_id_is_invalid(port_id)) - return -1; + portid_t i, pi = 0; + char name[RTE_ETH_NAME_MAX_LEN]; - if (ports[port_id].port_status != RTE_PORT_STARTED) - return 0; + printf("Detaching a port...\n"); - return 1; + if (!port_is_closed(port_id)) { + printf("Please close port first\n"); + return; + } + + rte_eth_promiscuous_disable(port_id); + + if (rte_eal_dev_detach(port_id, name)) + return; + + ports[port_id].enabled = 0; + nb_ports = rte_eth_dev_count(); + + /* set_default_fwd_ports_config(); */ + bzero(fwd_ports_ids, sizeof(fwd_ports_ids)); + i = 0; + FOREACH_PORT(pi, ports) { + fwd_ports_ids[i] = pi; + i++; + } + nb_cfg_ports = nb_ports; + nb_fwd_ports--; + + printf("Port '%s' is detached. Now total ports is %d\n", + name, nb_ports); + printf("Done\n"); + return; } void @@ -1534,7 +1637,7 @@ pmd_test_exit(void) { portid_t pt_id; - for (pt_id = 0; pt_id < nb_ports; pt_id++) { + FOREACH_PORT(pt_id, ports) { printf("Stopping port %d...", pt_id); fflush(stdout); rte_eth_dev_close(pt_id); @@ -1553,7 +1656,7 @@ struct pmd_test_command { /* Check the link status of all ports in up to 9s, and print them finally */ static void -check_all_ports_link_status(uint8_t port_num, uint32_t port_mask) +check_all_ports_link_status(uint32_t port_mask) { #define CHECK_INTERVAL 100 /* 100ms */ #define MAX_CHECK_TIME 90 /* 9s (90 * 100ms) in total */ @@ -1564,7 +1667,7 @@ check_all_ports_link_status(uint8_t port_num, uint32_t port_mask) fflush(stdout); for (count = 0; count <= MAX_CHECK_TIME; count++) { all_ports_up = 1; - for (portid = 0; portid < port_num; portid++) { + FOREACH_PORT(portid, ports) { if ((port_mask & (1 << portid)) == 0) continue; memset(&link, 0, sizeof(link)); @@ -1688,7 +1791,7 @@ init_port_config(void) portid_t pid; struct rte_port *port; - for (pid = 0; pid < nb_ports; pid++) { + FOREACH_PORT(pid, ports) { port = &ports[pid]; port->dev_conf.rxmode = rx_mode; port->dev_conf.fdir_conf = fdir_conf; @@ -1877,7 +1980,7 @@ main(int argc, char** argv) nb_ports = (portid_t) rte_eth_dev_count(); if (nb_ports == 0) - rte_exit(EXIT_FAILURE, "No probed ethernet device\n"); + RTE_LOG(WARNING, EAL, "No probed ethernet devices\n"); set_def_fwd_config(); if (nb_lcores == 0) @@ -1899,7 +2002,7 @@ main(int argc, char** argv) rte_exit(EXIT_FAILURE, "Start ports failed\n"); /* set all ports to promiscuous mode by default */ - for (port_id = 0; port_id < nb_ports; port_id++) + FOREACH_PORT(port_id, ports) rte_eth_promiscuous_enable(port_id); #ifdef RTE_LIBRTE_CMDLINE diff --git a/app/test-pmd/testpmd.h b/app/test-pmd/testpmd.h index 8f5e6c7..109c670 100644 --- a/app/test-pmd/testpmd.h +++ b/app/test-pmd/testpmd.h @@ -134,6 +134,7 @@ struct fwd_stream { * The data structure associated with each port. */ struct rte_port { + uint8_t enabled; /**< Port enabled or not */ struct rte_eth_dev_info dev_info; /**< PCI info + driver name */ struct rte_eth_conf dev_conf; /**< Port configuration. */ struct ether_addr eth_addr; /**< Port ethernet address */ @@ -159,6 +160,14 @@ struct rte_port { struct rte_eth_txconf tx_conf; /**< tx configuration */ }; +extern portid_t __rte_unused +find_next_port(portid_t p, struct rte_port *ports, int size); + +#define FOREACH_PORT(p, ports) \ + for (p = find_next_port(0, ports, RTE_MAX_ETHPORTS); \ + p < RTE_MAX_ETHPORTS; \ + p = find_next_port(p + 1, ports, RTE_MAX_ETHPORTS)) + /** * The data structure associated with each forwarding logical core. * The logical cores are internally numbered by a core index from 0 to @@ -515,6 +524,8 @@ int init_port_dcb_config(portid_t pid,struct dcb_config *dcb_conf); int start_port(portid_t pid); void stop_port(portid_t pid); void close_port(portid_t pid); +void attach_port(char *identifier); +void detach_port(uint8_t port_id); int all_ports_stopped(void); int port_is_started(portid_t port_id); void pmd_test_exit(void); @@ -558,10 +569,15 @@ void get_ethertype_filter(uint8_t port_id, uint16_t index); void get_2tuple_filter(uint8_t port_id, uint16_t index); void get_5tuple_filter(uint8_t port_id, uint16_t index); void get_flex_filter(uint8_t port_id, uint16_t index); -int port_id_is_invalid(portid_t port_id); int rx_queue_id_is_invalid(queueid_t rxq_id); int tx_queue_id_is_invalid(queueid_t txq_id); +enum print_warning { + ENABLED_WARN = 0, + DISABLED_WARN +}; +int port_id_is_invalid(portid_t port_id, enum print_warning warning); + /* * Work-around of a compilation error with ICC on invocations of the * rte_be_to_cpu_16() function. -- 1.9.1
Hi, Tetsuya
What I miss? I haven't got the first patch, just start with "PATCH 2/11".
Does this only happens with my mail client?
Thanks,
Michael
On 2015/1/19 18:42, Tetsuya Mukawa wrote:
> This patch series adds a dynamic port hotplug framework to DPDK.
> With the patches, DPDK apps can attach or detach ports at runtime.
>
> The basic concept of the port hotplug is like followings.
> - DPDK apps must have responsibility to manage ports.
> DPDK apps only know which ports are attached or detached at the moment.
> The port hotplug framework is implemented to allow DPDK apps to manage ports.
> For example, when DPDK apps call port attach function, attached port number
> will be returned. Also DPDK apps can detach port by port number.
> - Kernel support is needed for attaching or detaching physical device ports.
> To attach new device, the device will be recognized by kernel at first and
> controlled by kernel driver. Then user can bind the device to igb_uio
> by 'dpdk_nic_bind.py'. Finally, DPDK apps can call the port hotplug
> functions to attach ports.
> For detaching, steps are vice versa.
> - Before detach ports, ports must be stopped and closed.
> DPDK application must call rte_eth_dev_stop() and rte_eth_dev_close() before
> detaching ports. These function will call finalization codes of PMDs.
> But so far, no PMD frees all resources allocated by initialization.
> It means PMDs are needed to be fixed to support the port hotplug.
> 'RTE_PCI_DRV_DETACHABLE' is a new flag indicating a PMD supports detaching.
> Without this flag, detaching will be failed.
> - Mustn't affect legacy DPDK apps.
> No DPDK EAL behavior is changed, if the port hotplug functions are't called.
> So all legacy DPDK apps can still work without modifications.
>
> And a few limitations.
> - The port hotplug functions are not thread safe.
> DPDK apps should handle it.
> - Only support Linux and igb_uio so far.
> BSD and VFIO is not supported. I will send VFIO patches at least, but I don't
> have a plan to submit BSD patch so far.
>
>
> Here is port hotplug APIs.
> -------------------------------------------------------------------------------
> /**
> * Attach a new device.
> *
> * @param devargs
> * A pointer to a strings array describing the new device
> * to be attached. The strings should be a pci address like
> * '0000:01:00.0' or virtual device name like 'eth_pcap0'.
> * @param port_id
> * A pointer to a port identifier actually attached.
> * @return
> * 0 on success and port_id is filled, negative on error
> */
> int rte_eal_dev_attach(const char *devargs, uint8_t *port_id);
>
> /**
> * Detach a device.
> *
> * @param port_id
> * The port identifier of the device to detach.
> * @param addr
> * A pointer to a device name actually detached.
> * @return
> * 0 on success and devname is filled, negative on error
> */
> int rte_eal_dev_detach(uint8_t port_id, char *devname);
> -------------------------------------------------------------------------------
>
> This patch series are for DPDK EAL. To use port hotplug function by DPDK apps,
> each PMD should be fixed to support 'RTE_PCI_DRV_DETACHABLE' flag. Please check
> a patch for pcap PMD.
>
> Also please check testpmd patch. It will show you how to fix your legacy
> applications to support port hotplug feature.
>
>
> PATCH v4 changes
> - Merge patches to review easier.
> - Fix indent of 'if' statement.
> - Fix calculation method of eal_compare_pci_addr().
> - Fix header file declaration.
> - Add header file to determine if hotplug can be enabled.
> (Thanks to Qiu, Michael)
> - Use braces with 'for' loop.
> - Add paramerter checking.
> - Fix sanity check code
> - Fix comments of rte_eth_dev_type.
> - Change function names.
> (Thanks to Iremonger, Bernard)
>
> PATCH v3 changes:
> - Fix enum definition used in rte_ethdev.c.
> (Thanks to Zhang, Helin)
>
> PATCH v2 changes:
> - Replace rte_eal_dev_attach_pdev(), rte_eal_dev_detach_pdev,
> rte_eal_dev_attach_vdev() and rte_eal_dev_detach_vdev() to
> rte_eal_dev_attach() and rte_eal_dev_detach().
> - Add parameter values checking.
> - Refashion a few functions.
> (Thanks to Iremonger, Bernard)
>
> PATCH v1 Changes:
> - Fix error checking code of librte_eth APIs.
> - Fix issue that port from pcap PMD cannot be detached correctly.
> - Fix issue that testpmd could hang after forwarding, if attaching and detaching
> is repeatedly.
> - Fix if-condition of rte_eth_dev_get_port_by_addr().
> (Thanks to Mark Enright)
>
> RFC PATCH v2 Changes:
> - remove 'rte_eth_dev_validate_port()', and cleanup codes.
>
>
> Tetsuya Mukawa (11):
> eal/pci,ethdev: Remove assumption that port will not be detached
> eal/pci: Consolidate pci address comparison APIs
> ethdev: Add rte_eth_dev_free to free specified device
> eal,ethdev: Add a function and function pointers to close ether device
> ethdev: Add functions that will be used by port hotplug functions
> eal/linux/pci: Add functions for unmapping igb_uio resources
> eal/pci: Add a function to remove the entry of devargs list
> eal/pci: Cleanup pci driver initialization code
> ethdev: Add one dev_type paramerter to rte_eth_dev_allocate
> eal/pci: Add rte_eal_dev_attach/detach() functions
> eal: Enable port hotplug framework in Linux
>
> app/test/virtual_pmd.c | 2 +-
> config/common_linuxapp | 5 +
> lib/librte_eal/bsdapp/eal/eal_pci.c | 16 +-
> lib/librte_eal/common/Makefile | 1 +
> lib/librte_eal/common/eal_common_dev.c | 273 +++++++++++++++
> lib/librte_eal/common/eal_common_devargs.c | 57 ++++
> lib/librte_eal/common/eal_common_pci.c | 92 ++++-
> lib/librte_eal/common/eal_private.h | 36 ++
> lib/librte_eal/common/include/rte_dev.h | 33 ++
> lib/librte_eal/common/include/rte_dev_hotplug.h | 44 +++
> lib/librte_eal/common/include/rte_devargs.h | 18 +
> lib/librte_eal/common/include/rte_pci.h | 76 +++++
> lib/librte_eal/linuxapp/eal/Makefile | 1 +
> lib/librte_eal/linuxapp/eal/eal_pci.c | 136 ++++++--
> lib/librte_eal/linuxapp/eal/eal_pci_init.h | 8 +
> lib/librte_eal/linuxapp/eal/eal_pci_uio.c | 67 +++-
> lib/librte_ether/rte_ethdev.c | 424 ++++++++++++++++++------
> lib/librte_ether/rte_ethdev.h | 145 +++++++-
> lib/librte_pmd_af_packet/rte_eth_af_packet.c | 2 +-
> lib/librte_pmd_bond/rte_eth_bond_api.c | 2 +-
> lib/librte_pmd_pcap/rte_eth_pcap.c | 2 +-
> lib/librte_pmd_ring/rte_eth_ring.c | 2 +-
> lib/librte_pmd_xenvirt/rte_eth_xenvirt.c | 2 +-
> 23 files changed, 1283 insertions(+), 161 deletions(-)
> create mode 100644 lib/librte_eal/common/include/rte_dev_hotplug.h
>
Hi, Tetsuya
You see lots of places have below three lines:
if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
return -EINVAL;
}
They are all the same(only few has the print log different), so can we improve it?
See below:
+static int
+rte_eth_dev_validate_port(uint8_t port_id, bool trace) {
+ if (port_id >= RTE_MAX_ETHPORTS ||
+ rte_eth_devices[port_id].attached != DEV_CONNECTED) {
+ if (trace)
+ PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
+ return DEV_INVALID;
+ }
+ else
+ return DEV_VALID;
+}
For normal case we just call this function use
rte_eth_dev_validate_port(port_id, 1)
here 1 could be a enmu value(Thus trace should be defined as int).
After call, we didn't need to add PMD_DEBUG_TRACE() any more.
For few cases like:
PMD_DEBUG_TRACE("set VF rate limit:invalid port id=%d\n",
port_id);
We can call the function with secondary param set to "0", and add the trace log after the function called, just as before.
I think after this enhancement, the code seems more clean and efficiency。
Thanks,
Michael
>> -----Original Message-----
>> From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Tetsuya Mukawa
>> Sent: Monday, January 19, 2015 10:40 AM
>> To: dev@dpdk.org
>> Subject: [dpdk-dev] [PATCH v4 01/11] eal/pci, ethdev: Remove assumption that port will not be
>> detached
>>
>> To remove assumption, do like followings.
>>
>> This patch adds "RTE_PCI_DRV_DETACHABLE" to drv_flags of rte_pci_driver structure. The flags
>> indicates the driver can detach devices at runtime.
>> Also remove assumption that port will not be detached.
>>
>> To remove the assumption.
>> - Add 'attached' member to rte_eth_dev structure.
>> This member is used for indicating the port is attached, or not.
>> - Add rte_eth_dev_allocate_new_port().
>> This function is used for allocating new port.
>>
>> v4:
>> - Use braces with 'for' loop.
>> - Fix indent of 'if' statement.
>>
>> Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp>
>> ---
>> lib/librte_eal/common/include/rte_pci.h | 2 +
>> lib/librte_ether/rte_ethdev.c | 248 ++++++++++++++++++--------------
>> lib/librte_ether/rte_ethdev.h | 5 +
>> 3 files changed, 151 insertions(+), 104 deletions(-)
>>
>> diff --git a/lib/librte_eal/common/include/rte_pci.h b/lib/librte_eal/common/include/rte_pci.h
>> index 66ed793..dd1da28 100644
>> --- a/lib/librte_eal/common/include/rte_pci.h
>> +++ b/lib/librte_eal/common/include/rte_pci.h
>> @@ -199,6 +199,8 @@ struct rte_pci_driver { #define RTE_PCI_DRV_FORCE_UNBIND 0x0004
>> /** Device driver supports link state interrupt */
>> #define RTE_PCI_DRV_INTR_LSC 0x0008
>> +/** Device driver supports detaching capability */
>> +#define RTE_PCI_DRV_DETACHABLE 0x0010
>>
>> /**< Internal use only - Macro used by pci addr parsing functions **/
>> #define GET_PCIADDR_FIELD(in, fd, lim, dlm) \
>> diff --git a/lib/librte_ether/rte_ethdev.c b/lib/librte_ether/rte_ethdev.c index 077d430..46cabaf
>> 100644
>> --- a/lib/librte_ether/rte_ethdev.c
>> +++ b/lib/librte_ether/rte_ethdev.c
>> @@ -175,6 +175,16 @@ enum {
>> STAT_QMAP_RX
>> };
>>
>> +enum {
>> + DEV_INVALID = 0,
>> + DEV_VALID,
>> +};
>> +
>> +enum {
>> + DEV_DISCONNECTED = 0,
>> + DEV_CONNECTED
>> +};
>> +
>> static inline void
>> rte_eth_dev_data_alloc(void)
>> {
>> @@ -201,19 +211,34 @@ rte_eth_dev_allocated(const char *name) {
>> unsigned i;
>>
>> - for (i = 0; i < nb_ports; i++) {
>> - if (strcmp(rte_eth_devices[i].data->name, name) == 0)
>> + for (i = 0; i < RTE_MAX_ETHPORTS; i++) {
>> + if ((rte_eth_devices[i].attached == DEV_CONNECTED) &&
>> + strcmp(rte_eth_devices[i].data->name, name) == 0)
>> return &rte_eth_devices[i];
>> }
>> return NULL;
>> }
>>
>> +static uint8_t
>> +rte_eth_dev_allocate_new_port(void)
>> +{
>> + unsigned i;
>> +
>> + for (i = 0; i < RTE_MAX_ETHPORTS; i++) {
>> + if (rte_eth_devices[i].attached == DEV_DISCONNECTED)
>> + return i;
>> + }
>> + return RTE_MAX_ETHPORTS;
>> +}
>> +
>> struct rte_eth_dev *
>> rte_eth_dev_allocate(const char *name)
>> {
>> + uint8_t port_id;
>> struct rte_eth_dev *eth_dev;
>>
>> - if (nb_ports == RTE_MAX_ETHPORTS) {
>> + port_id = rte_eth_dev_allocate_new_port();
>> + if (port_id == RTE_MAX_ETHPORTS) {
>> PMD_DEBUG_TRACE("Reached maximum number of Ethernet ports\n");
>> return NULL;
>> }
>> @@ -226,10 +251,12 @@ rte_eth_dev_allocate(const char *name)
>> return NULL;
>> }
>>
>> - eth_dev = &rte_eth_devices[nb_ports];
>> - eth_dev->data = &rte_eth_dev_data[nb_ports];
>> + eth_dev = &rte_eth_devices[port_id];
>> + eth_dev->data = &rte_eth_dev_data[port_id];
>> snprintf(eth_dev->data->name, sizeof(eth_dev->data->name), "%s", name);
>> - eth_dev->data->port_id = nb_ports++;
>> + eth_dev->data->port_id = port_id;
>> + eth_dev->attached = DEV_CONNECTED;
>> + nb_ports++;
>> return eth_dev;
>> }
>>
>> @@ -283,6 +310,7 @@ rte_eth_dev_init(struct rte_pci_driver *pci_drv,
>> (unsigned) pci_dev->id.device_id);
>> if (rte_eal_process_type() == RTE_PROC_PRIMARY)
>> rte_free(eth_dev->data->dev_private);
>> + eth_dev->attached = DEV_DISCONNECTED;
>> nb_ports--;
>> return diag;
>> }
>> @@ -308,10 +336,22 @@ rte_eth_driver_register(struct eth_driver *eth_drv)
>> rte_eal_pci_register(ð_drv->pci_drv);
>> }
>>
>> +static int
>> +rte_eth_dev_validate_port(uint8_t port_id) {
>> + if (port_id >= RTE_MAX_ETHPORTS)
>> + return DEV_INVALID;
>> +
>> + if (rte_eth_devices[port_id].attached == DEV_CONNECTED)
>> + return DEV_VALID;
>> + else
>> + return DEV_INVALID;
>> +}
>> +
>> int
>> rte_eth_dev_socket_id(uint8_t port_id)
>> {
>> - if (port_id >= nb_ports)
>> + if (rte_eth_dev_validate_port(port_id) == DEV_INVALID)
>> return -1;
>> return rte_eth_devices[port_id].pci_dev->numa_node;
>> }
>> @@ -369,7 +409,7 @@ rte_eth_dev_rx_queue_start(uint8_t port_id, uint16_t rx_queue_id)
>> * in a multi-process setup*/
>> PROC_PRIMARY_OR_ERR_RET(-E_RTE_SECONDARY);
>>
>> - if (port_id >= nb_ports) {
>> + if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
>> PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
>> return -EINVAL;
>> }
>> @@ -395,7 +435,7 @@ rte_eth_dev_rx_queue_stop(uint8_t port_id, uint16_t rx_queue_id)
>> * in a multi-process setup*/
>> PROC_PRIMARY_OR_ERR_RET(-E_RTE_SECONDARY);
>>
>> - if (port_id >= nb_ports) {
>> + if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
>> PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
>> return -EINVAL;
>> }
>> @@ -421,7 +461,7 @@ rte_eth_dev_tx_queue_start(uint8_t port_id, uint16_t tx_queue_id)
>> * in a multi-process setup*/
>> PROC_PRIMARY_OR_ERR_RET(-E_RTE_SECONDARY);
>>
>> - if (port_id >= nb_ports) {
>> + if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
>> PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
>> return -EINVAL;
>> }
>> @@ -447,7 +487,7 @@ rte_eth_dev_tx_queue_stop(uint8_t port_id, uint16_t tx_queue_id)
>> * in a multi-process setup*/
>> PROC_PRIMARY_OR_ERR_RET(-E_RTE_SECONDARY);
>>
>> - if (port_id >= nb_ports) {
>> + if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
>> PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
>> return -EINVAL;
>> }
>> @@ -703,7 +743,7 @@ rte_eth_dev_configure(uint8_t port_id, uint16_t nb_rx_q, uint16_t nb_tx_q,
>> * in a multi-process setup*/
>> PROC_PRIMARY_OR_ERR_RET(-E_RTE_SECONDARY);
>>
>> - if (port_id >= nb_ports || port_id >= RTE_MAX_ETHPORTS) {
>> + if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
>> PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
>> return (-EINVAL);
>> }
>> @@ -888,7 +928,7 @@ rte_eth_dev_start(uint8_t port_id)
>> * in a multi-process setup*/
>> PROC_PRIMARY_OR_ERR_RET(-E_RTE_SECONDARY);
>>
>> - if (port_id >= nb_ports) {
>> + if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
>> PMD_DEBUG_TRACE("Invalid port_id=%" PRIu8 "\n", port_id);
>> return (-EINVAL);
>> }
>> @@ -923,7 +963,7 @@ rte_eth_dev_stop(uint8_t port_id)
>> * in a multi-process setup*/
>> PROC_PRIMARY_OR_RET();
>>
>> - if (port_id >= nb_ports) {
>> + if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
>> PMD_DEBUG_TRACE("Invalid port_id=%" PRIu8 "\n", port_id);
>> return;
>> }
>> @@ -951,7 +991,7 @@ rte_eth_dev_set_link_up(uint8_t port_id)
>> * in a multi-process setup*/
>> PROC_PRIMARY_OR_ERR_RET(-E_RTE_SECONDARY);
>>
>> - if (port_id >= nb_ports) {
>> + if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
>> PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
>> return -EINVAL;
>> }
>> @@ -970,7 +1010,7 @@ rte_eth_dev_set_link_down(uint8_t port_id)
>> * in a multi-process setup*/
>> PROC_PRIMARY_OR_ERR_RET(-E_RTE_SECONDARY);
>>
>> - if (port_id >= nb_ports) {
>> + if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
>> PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
>> return -EINVAL;
>> }
>> @@ -989,7 +1029,7 @@ rte_eth_dev_close(uint8_t port_id)
>> * in a multi-process setup*/
>> PROC_PRIMARY_OR_RET();
>>
>> - if (port_id >= nb_ports) {
>> + if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
>> PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
>> return;
>> }
>> @@ -1017,7 +1057,7 @@ rte_eth_rx_queue_setup(uint8_t port_id, uint16_t rx_queue_id,
>> * in a multi-process setup*/
>> PROC_PRIMARY_OR_ERR_RET(-E_RTE_SECONDARY);
>>
>> - if (port_id >= nb_ports) {
>> + if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
>> PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
>> return (-EINVAL);
>> }
>> @@ -1090,7 +1130,7 @@ rte_eth_tx_queue_setup(uint8_t port_id, uint16_t tx_queue_id,
>> * in a multi-process setup*/
>> PROC_PRIMARY_OR_ERR_RET(-E_RTE_SECONDARY);
>>
>> - if (port_id >= RTE_MAX_ETHPORTS || port_id >= nb_ports) {
>> + if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
>> PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
>> return (-EINVAL);
>> }
>> @@ -1123,7 +1163,7 @@ rte_eth_promiscuous_enable(uint8_t port_id) {
>> struct rte_eth_dev *dev;
>>
>> - if (port_id >= nb_ports) {
>> + if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
>> PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
>> return;
>> }
>> @@ -1139,7 +1179,7 @@ rte_eth_promiscuous_disable(uint8_t port_id) {
>> struct rte_eth_dev *dev;
>>
>> - if (port_id >= nb_ports) {
>> + if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
>> PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
>> return;
>> }
>> @@ -1155,7 +1195,7 @@ rte_eth_promiscuous_get(uint8_t port_id) {
>> struct rte_eth_dev *dev;
>>
>> - if (port_id >= nb_ports) {
>> + if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
>> PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
>> return -1;
>> }
>> @@ -1169,7 +1209,7 @@ rte_eth_allmulticast_enable(uint8_t port_id) {
>> struct rte_eth_dev *dev;
>>
>> - if (port_id >= nb_ports) {
>> + if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
>> PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
>> return;
>> }
>> @@ -1185,7 +1225,7 @@ rte_eth_allmulticast_disable(uint8_t port_id) {
>> struct rte_eth_dev *dev;
>>
>> - if (port_id >= nb_ports) {
>> + if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
>> PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
>> return;
>> }
>> @@ -1201,7 +1241,7 @@ rte_eth_allmulticast_get(uint8_t port_id) {
>> struct rte_eth_dev *dev;
>>
>> - if (port_id >= nb_ports) {
>> + if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
>> PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
>> return -1;
>> }
>> @@ -1229,7 +1269,7 @@ rte_eth_link_get(uint8_t port_id, struct rte_eth_link *eth_link) {
>> struct rte_eth_dev *dev;
>>
>> - if (port_id >= nb_ports) {
>> + if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
>> PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
>> return;
>> }
>> @@ -1249,7 +1289,7 @@ rte_eth_link_get_nowait(uint8_t port_id, struct rte_eth_link *eth_link) {
>> struct rte_eth_dev *dev;
>>
>> - if (port_id >= nb_ports) {
>> + if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
>> PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
>> return;
>> }
>> @@ -1269,7 +1309,7 @@ rte_eth_stats_get(uint8_t port_id, struct rte_eth_stats *stats) {
>> struct rte_eth_dev *dev;
>>
>> - if (port_id >= nb_ports) {
>> + if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
>> PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
>> return;
>> }
>> @@ -1286,7 +1326,7 @@ rte_eth_stats_reset(uint8_t port_id) {
>> struct rte_eth_dev *dev;
>>
>> - if (port_id >= nb_ports) {
>> + if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
>> PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
>> return;
>> }
>> @@ -1307,7 +1347,7 @@ rte_eth_xstats_get(uint8_t port_id, struct rte_eth_xstats *xstats,
>> uint64_t val;
>> char *stats_ptr;
>>
>> - if (port_id >= nb_ports) {
>> + if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
>> PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
>> return -1;
>> }
>> @@ -1376,7 +1416,7 @@ rte_eth_xstats_reset(uint8_t port_id) {
>> struct rte_eth_dev *dev;
>>
>> - if (port_id >= nb_ports) {
>> + if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
>> PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
>> return;
>> }
>> @@ -1398,7 +1438,7 @@ set_queue_stats_mapping(uint8_t port_id, uint16_t queue_id, uint8_t
>> stat_idx, {
>> struct rte_eth_dev *dev;
>>
>> - if (port_id >= nb_ports) {
>> + if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
>> PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
>> return -ENODEV;
>> }
>> @@ -1433,7 +1473,7 @@ rte_eth_dev_info_get(uint8_t port_id, struct rte_eth_dev_info *dev_info)
>> {
>> struct rte_eth_dev *dev;
>>
>> - if (port_id >= nb_ports) {
>> + if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
>> PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
>> return;
>> }
>> @@ -1453,7 +1493,7 @@ rte_eth_macaddr_get(uint8_t port_id, struct ether_addr *mac_addr) {
>> struct rte_eth_dev *dev;
>>
>> - if (port_id >= nb_ports) {
>> + if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
>> PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
>> return;
>> }
>> @@ -1467,7 +1507,7 @@ rte_eth_dev_get_mtu(uint8_t port_id, uint16_t *mtu) {
>> struct rte_eth_dev *dev;
>>
>> - if (port_id >= nb_ports) {
>> + if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
>> PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
>> return (-ENODEV);
>> }
>> @@ -1483,7 +1523,7 @@ rte_eth_dev_set_mtu(uint8_t port_id, uint16_t mtu)
>> int ret;
>> struct rte_eth_dev *dev;
>>
>> - if (port_id >= nb_ports) {
>> + if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
>> PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
>> return (-ENODEV);
>> }
>> @@ -1503,7 +1543,7 @@ rte_eth_dev_vlan_filter(uint8_t port_id, uint16_t vlan_id, int on) {
>> struct rte_eth_dev *dev;
>>
>> - if (port_id >= nb_ports) {
>> + if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
>> PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
>> return (-ENODEV);
>> }
>> @@ -1528,7 +1568,7 @@ rte_eth_dev_set_vlan_strip_on_queue(uint8_t port_id, uint16_t
>> rx_queue_id, int o {
>> struct rte_eth_dev *dev;
>>
>> - if (port_id >= nb_ports) {
>> + if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
>> PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
>> return (-ENODEV);
>> }
>> @@ -1550,7 +1590,7 @@ rte_eth_dev_set_vlan_ether_type(uint8_t port_id, uint16_t tpid) {
>> struct rte_eth_dev *dev;
>>
>> - if (port_id >= nb_ports) {
>> + if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
>> PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
>> return (-ENODEV);
>> }
>> @@ -1570,7 +1610,7 @@ rte_eth_dev_set_vlan_offload(uint8_t port_id, int offload_mask)
>> int mask = 0;
>> int cur, org = 0;
>>
>> - if (port_id >= nb_ports) {
>> + if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
>> PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
>> return (-ENODEV);
>> }
>> @@ -1615,7 +1655,7 @@ rte_eth_dev_get_vlan_offload(uint8_t port_id)
>> struct rte_eth_dev *dev;
>> int ret = 0;
>>
>> - if (port_id >= nb_ports) {
>> + if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
>> PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
>> return (-ENODEV);
>> }
>> @@ -1639,7 +1679,7 @@ rte_eth_dev_set_vlan_pvid(uint8_t port_id, uint16_t pvid, int on) {
>> struct rte_eth_dev *dev;
>>
>> - if (port_id >= nb_ports) {
>> + if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
>> PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
>> return (-ENODEV);
>> }
>> @@ -1657,7 +1697,7 @@ rte_eth_dev_fdir_add_signature_filter(uint8_t port_id, {
>> struct rte_eth_dev *dev;
>>
>> - if (port_id >= nb_ports) {
>> + if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
>> PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
>> return (-ENODEV);
>> }
>> @@ -1691,7 +1731,7 @@ rte_eth_dev_fdir_update_signature_filter(uint8_t port_id, {
>> struct rte_eth_dev *dev;
>>
>> - if (port_id >= nb_ports) {
>> + if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
>> PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
>> return (-ENODEV);
>> }
>> @@ -1725,7 +1765,7 @@ rte_eth_dev_fdir_remove_signature_filter(uint8_t port_id, {
>> struct rte_eth_dev *dev;
>>
>> - if (port_id >= nb_ports) {
>> + if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
>> PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
>> return (-ENODEV);
>> }
>> @@ -1756,7 +1796,7 @@ rte_eth_dev_fdir_get_infos(uint8_t port_id, struct rte_eth_fdir *fdir) {
>> struct rte_eth_dev *dev;
>>
>> - if (port_id >= nb_ports) {
>> + if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
>> PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
>> return (-ENODEV);
>> }
>> @@ -1781,7 +1821,7 @@ rte_eth_dev_fdir_add_perfect_filter(uint8_t port_id, {
>> struct rte_eth_dev *dev;
>>
>> - if (port_id >= nb_ports) {
>> + if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
>> PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
>> return (-ENODEV);
>> }
>> @@ -1821,7 +1861,7 @@ rte_eth_dev_fdir_update_perfect_filter(uint8_t port_id, {
>> struct rte_eth_dev *dev;
>>
>> - if (port_id >= nb_ports) {
>> + if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
>> PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
>> return (-ENODEV);
>> }
>> @@ -1859,7 +1899,7 @@ rte_eth_dev_fdir_remove_perfect_filter(uint8_t port_id, {
>> struct rte_eth_dev *dev;
>>
>> - if (port_id >= nb_ports) {
>> + if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
>> PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
>> return (-ENODEV);
>> }
>> @@ -1895,7 +1935,7 @@ rte_eth_dev_fdir_set_masks(uint8_t port_id, struct rte_fdir_masks
>> *fdir_mask) {
>> struct rte_eth_dev *dev;
>>
>> - if (port_id >= nb_ports) {
>> + if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
>> PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
>> return (-ENODEV);
>> }
>> @@ -1915,7 +1955,7 @@ rte_eth_dev_flow_ctrl_get(uint8_t port_id, struct rte_eth_fc_conf
>> *fc_conf) {
>> struct rte_eth_dev *dev;
>>
>> - if (port_id >= nb_ports) {
>> + if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
>> PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
>> return (-ENODEV);
>> }
>> @@ -1931,7 +1971,7 @@ rte_eth_dev_flow_ctrl_set(uint8_t port_id, struct rte_eth_fc_conf
>> *fc_conf) {
>> struct rte_eth_dev *dev;
>>
>> - if (port_id >= nb_ports) {
>> + if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
>> PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
>> return (-ENODEV);
>> }
>> @@ -1951,7 +1991,7 @@ rte_eth_dev_priority_flow_ctrl_set(uint8_t port_id, struct
>> rte_eth_pfc_conf *pfc {
>> struct rte_eth_dev *dev;
>>
>> - if (port_id >= nb_ports) {
>> + if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
>> PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
>> return (-ENODEV);
>> }
>> @@ -2030,7 +2070,7 @@ rte_eth_dev_rss_reta_update(uint8_t port_id,
>> struct rte_eth_dev *dev;
>> int ret;
>>
>> - if (port_id >= nb_ports) {
>> + if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
>> PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
>> return -ENODEV;
>> }
>> @@ -2081,7 +2121,7 @@ rte_eth_dev_rss_hash_update(uint8_t port_id, struct rte_eth_rss_conf
>> *rss_conf)
>> struct rte_eth_dev *dev;
>> uint16_t rss_hash_protos;
>>
>> - if (port_id >= nb_ports) {
>> + if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
>> PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
>> return (-ENODEV);
>> }
>> @@ -2103,7 +2143,7 @@ rte_eth_dev_rss_hash_conf_get(uint8_t port_id, {
>> struct rte_eth_dev *dev;
>>
>> - if (port_id >= nb_ports) {
>> + if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
>> PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
>> return (-ENODEV);
>> }
>> @@ -2118,7 +2158,7 @@ rte_eth_dev_udp_tunnel_add(uint8_t port_id, {
>> struct rte_eth_dev *dev;
>>
>> - if (port_id >= nb_ports) {
>> + if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
>> PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
>> return -ENODEV;
>> }
>> @@ -2144,7 +2184,7 @@ rte_eth_dev_udp_tunnel_delete(uint8_t port_id, {
>> struct rte_eth_dev *dev;
>>
>> - if (port_id >= nb_ports) {
>> + if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
>> PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
>> return -ENODEV;
>> }
>> @@ -2169,7 +2209,7 @@ rte_eth_led_on(uint8_t port_id) {
>> struct rte_eth_dev *dev;
>>
>> - if (port_id >= nb_ports) {
>> + if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
>> PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
>> return (-ENODEV);
>> }
>> @@ -2184,7 +2224,7 @@ rte_eth_led_off(uint8_t port_id) {
>> struct rte_eth_dev *dev;
>>
>> - if (port_id >= nb_ports) {
>> + if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
>> PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
>> return (-ENODEV);
>> }
>> @@ -2224,7 +2264,7 @@ rte_eth_dev_mac_addr_add(uint8_t port_id, struct ether_addr *addr,
>> int index;
>> uint64_t pool_mask;
>>
>> - if (port_id >= nb_ports) {
>> + if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
>> PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
>> return (-ENODEV);
>> }
>> @@ -2275,7 +2315,7 @@ rte_eth_dev_mac_addr_remove(uint8_t port_id, struct ether_addr *addr)
>> struct rte_eth_dev *dev;
>> int index;
>>
>> - if (port_id >= nb_ports) {
>> + if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
>> PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
>> return (-ENODEV);
>> }
>> @@ -2309,7 +2349,7 @@ rte_eth_dev_set_vf_rxmode(uint8_t port_id, uint16_t vf,
>> struct rte_eth_dev *dev;
>> struct rte_eth_dev_info dev_info;
>>
>> - if (port_id >= nb_ports) {
>> + if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
>> PMD_DEBUG_TRACE("set VF RX mode:Invalid port_id=%d\n",
>> port_id);
>> return (-ENODEV);
>> @@ -2364,7 +2404,7 @@ rte_eth_dev_uc_hash_table_set(uint8_t port_id, struct ether_addr *addr,
>> int ret;
>> struct rte_eth_dev *dev;
>>
>> - if (port_id >= nb_ports) {
>> + if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
>> PMD_DEBUG_TRACE("unicast hash setting:Invalid port_id=%d\n",
>> port_id);
>> return (-ENODEV);
>> @@ -2417,7 +2457,7 @@ rte_eth_dev_uc_all_hash_table_set(uint8_t port_id, uint8_t on) {
>> struct rte_eth_dev *dev;
>>
>> - if (port_id >= nb_ports) {
>> + if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
>> PMD_DEBUG_TRACE("unicast hash setting:Invalid port_id=%d\n",
>> port_id);
>> return (-ENODEV);
>> @@ -2436,7 +2476,7 @@ rte_eth_dev_set_vf_rx(uint8_t port_id,uint16_t vf, uint8_t on)
>> struct rte_eth_dev *dev;
>> struct rte_eth_dev_info dev_info;
>>
>> - if (port_id >= nb_ports) {
>> + if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
>> PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
>> return (-ENODEV);
>> }
>> @@ -2462,7 +2502,7 @@ rte_eth_dev_set_vf_tx(uint8_t port_id,uint16_t vf, uint8_t on)
>> struct rte_eth_dev *dev;
>> struct rte_eth_dev_info dev_info;
>>
>> - if (port_id >= nb_ports) {
>> + if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
>> PMD_DEBUG_TRACE("set pool tx:Invalid port_id=%d\n", port_id);
>> return (-ENODEV);
>> }
>> @@ -2487,7 +2527,7 @@ rte_eth_dev_set_vf_vlan_filter(uint8_t port_id, uint16_t vlan_id, {
>> struct rte_eth_dev *dev;
>>
>> - if (port_id >= nb_ports) {
>> + if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
>> PMD_DEBUG_TRACE("VF VLAN filter:invalid port id=%d\n",
>> port_id);
>> return (-ENODEV);
>> @@ -2518,7 +2558,7 @@ int rte_eth_set_queue_rate_limit(uint8_t port_id, uint16_t queue_idx,
>> struct rte_eth_dev_info dev_info;
>> struct rte_eth_link link;
>>
>> - if (port_id >= nb_ports) {
>> + if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
>> PMD_DEBUG_TRACE("set queue rate limit:invalid port id=%d\n",
>> port_id);
>> return -ENODEV;
>> @@ -2555,7 +2595,7 @@ int rte_eth_set_vf_rate_limit(uint8_t port_id, uint16_t vf, uint16_t tx_rate,
>> if (q_msk == 0)
>> return 0;
>>
>> - if (port_id >= nb_ports) {
>> + if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
>> PMD_DEBUG_TRACE("set VF rate limit:invalid port id=%d\n",
>> port_id);
>> return -ENODEV;
>> @@ -2589,7 +2629,7 @@ rte_eth_mirror_rule_set(uint8_t port_id, {
>> struct rte_eth_dev *dev = &rte_eth_devices[port_id];
>>
>> - if (port_id >= nb_ports) {
>> + if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
>> PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
>> return (-ENODEV);
>> }
>> @@ -2630,7 +2670,7 @@ rte_eth_mirror_rule_reset(uint8_t port_id, uint8_t rule_id) {
>> struct rte_eth_dev *dev = &rte_eth_devices[port_id];
>>
>> - if (port_id >= nb_ports) {
>> + if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
>> PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
>> return (-ENODEV);
>> }
>> @@ -2655,7 +2695,7 @@ rte_eth_rx_burst(uint8_t port_id, uint16_t queue_id, {
>> struct rte_eth_dev *dev;
>>
>> - if (port_id >= nb_ports) {
>> + if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
>> PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
>> return 0;
>> }
>> @@ -2675,7 +2715,7 @@ rte_eth_tx_burst(uint8_t port_id, uint16_t queue_id, {
>> struct rte_eth_dev *dev;
>>
>> - if (port_id >= nb_ports) {
>> + if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
>> PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
>> return 0;
>> }
>> @@ -2695,7 +2735,7 @@ rte_eth_rx_queue_count(uint8_t port_id, uint16_t queue_id) {
>> struct rte_eth_dev *dev;
>>
>> - if (port_id >= nb_ports) {
>> + if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
>> PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
>> return 0;
>> }
>> @@ -2709,7 +2749,7 @@ rte_eth_rx_descriptor_done(uint8_t port_id, uint16_t queue_id, uint16_t
>> offset) {
>> struct rte_eth_dev *dev;
>>
>> - if (port_id >= nb_ports) {
>> + if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
>> PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
>> return (-ENODEV);
>> }
>> @@ -2730,7 +2770,7 @@ rte_eth_dev_callback_register(uint8_t port_id,
>>
>> if (!cb_fn)
>> return (-EINVAL);
>> - if (port_id >= nb_ports) {
>> + if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
>> PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
>> return (-EINVAL);
>> }
>> @@ -2770,7 +2810,7 @@ rte_eth_dev_callback_unregister(uint8_t port_id,
>>
>> if (!cb_fn)
>> return (-EINVAL);
>> - if (port_id >= nb_ports) {
>> + if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
>> PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
>> return (-EINVAL);
>> }
>> @@ -2830,7 +2870,7 @@ int rte_eth_dev_bypass_init(uint8_t port_id) {
>> struct rte_eth_dev *dev;
>>
>> - if (port_id >= nb_ports) {
>> + if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
>> PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
>> return (-ENODEV);
>> }
>> @@ -2850,7 +2890,7 @@ rte_eth_dev_bypass_state_show(uint8_t port_id, uint32_t *state) {
>> struct rte_eth_dev *dev;
>>
>> - if (port_id >= nb_ports) {
>> + if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
>> PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
>> return (-ENODEV);
>> }
>> @@ -2869,7 +2909,7 @@ rte_eth_dev_bypass_state_set(uint8_t port_id, uint32_t *new_state) {
>> struct rte_eth_dev *dev;
>>
>> - if (port_id >= nb_ports) {
>> + if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
>> PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
>> return (-ENODEV);
>> }
>> @@ -2889,7 +2929,7 @@ rte_eth_dev_bypass_event_show(uint8_t port_id, uint32_t event, uint32_t
>> *state) {
>> struct rte_eth_dev *dev;
>>
>> - if (port_id >= nb_ports) {
>> + if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
>> PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
>> return (-ENODEV);
>> }
>> @@ -2909,7 +2949,7 @@ rte_eth_dev_bypass_event_store(uint8_t port_id, uint32_t event, uint32_t
>> state) {
>> struct rte_eth_dev *dev;
>>
>> - if (port_id >= nb_ports) {
>> + if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
>> PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
>> return (-ENODEV);
>> }
>> @@ -2929,7 +2969,7 @@ rte_eth_dev_wd_timeout_store(uint8_t port_id, uint32_t timeout) {
>> struct rte_eth_dev *dev;
>>
>> - if (port_id >= nb_ports) {
>> + if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
>> PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
>> return (-ENODEV);
>> }
>> @@ -2949,7 +2989,7 @@ rte_eth_dev_bypass_ver_show(uint8_t port_id, uint32_t *ver) {
>> struct rte_eth_dev *dev;
>>
>> - if (port_id >= nb_ports) {
>> + if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
>> PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
>> return (-ENODEV);
>> }
>> @@ -2969,7 +3009,7 @@ rte_eth_dev_bypass_wd_timeout_show(uint8_t port_id, uint32_t
>> *wd_timeout) {
>> struct rte_eth_dev *dev;
>>
>> - if (port_id >= nb_ports) {
>> + if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
>> PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
>> return (-ENODEV);
>> }
>> @@ -2989,7 +3029,7 @@ rte_eth_dev_bypass_wd_reset(uint8_t port_id) {
>> struct rte_eth_dev *dev;
>>
>> - if (port_id >= nb_ports) {
>> + if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
>> PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
>> return (-ENODEV);
>> }
>> @@ -3011,7 +3051,7 @@ rte_eth_dev_add_syn_filter(uint8_t port_id, {
>> struct rte_eth_dev *dev;
>>
>> - if (port_id >= nb_ports) {
>> + if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
>> PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
>> return -ENODEV;
>> }
>> @@ -3026,7 +3066,7 @@ rte_eth_dev_remove_syn_filter(uint8_t port_id) {
>> struct rte_eth_dev *dev;
>>
>> - if (port_id >= nb_ports) {
>> + if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
>> PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
>> return -ENODEV;
>> }
>> @@ -3045,7 +3085,7 @@ rte_eth_dev_get_syn_filter(uint8_t port_id,
>> if (filter == NULL || rx_queue == NULL)
>> return -EINVAL;
>>
>> - if (port_id >= nb_ports) {
>> + if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
>> PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
>> return -ENODEV;
>> }
>> @@ -3061,7 +3101,7 @@ rte_eth_dev_add_ethertype_filter(uint8_t port_id, uint16_t index, {
>> struct rte_eth_dev *dev;
>>
>> - if (port_id >= nb_ports) {
>> + if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
>> PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
>> return -ENODEV;
>> }
>> @@ -3082,7 +3122,7 @@ rte_eth_dev_remove_ethertype_filter(uint8_t port_id, uint16_t index) {
>> struct rte_eth_dev *dev;
>>
>> - if (port_id >= nb_ports) {
>> + if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
>> PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
>> return -ENODEV;
>> }
>> @@ -3101,7 +3141,7 @@ rte_eth_dev_get_ethertype_filter(uint8_t port_id, uint16_t index,
>> if (filter == NULL || rx_queue == NULL)
>> return -EINVAL;
>>
>> - if (port_id >= nb_ports) {
>> + if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
>> PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
>> return -ENODEV;
>> }
>> @@ -3118,7 +3158,7 @@ rte_eth_dev_add_2tuple_filter(uint8_t port_id, uint16_t index, {
>> struct rte_eth_dev *dev;
>>
>> - if (port_id >= nb_ports) {
>> + if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
>> PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
>> return -ENODEV;
>> }
>> @@ -3140,7 +3180,7 @@ rte_eth_dev_remove_2tuple_filter(uint8_t port_id, uint16_t index) {
>> struct rte_eth_dev *dev;
>>
>> - if (port_id >= nb_ports) {
>> + if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
>> PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
>> return -ENODEV;
>> }
>> @@ -3159,7 +3199,7 @@ rte_eth_dev_get_2tuple_filter(uint8_t port_id, uint16_t index,
>> if (filter == NULL || rx_queue == NULL)
>> return -EINVAL;
>>
>> - if (port_id >= nb_ports) {
>> + if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
>> PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
>> return -ENODEV;
>> }
>> @@ -3175,7 +3215,7 @@ rte_eth_dev_add_5tuple_filter(uint8_t port_id, uint16_t index, {
>> struct rte_eth_dev *dev;
>>
>> - if (port_id >= nb_ports) {
>> + if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
>> PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
>> return -ENODEV;
>> }
>> @@ -3198,7 +3238,7 @@ rte_eth_dev_remove_5tuple_filter(uint8_t port_id, uint16_t index) {
>> struct rte_eth_dev *dev;
>>
>> - if (port_id >= nb_ports) {
>> + if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
>> PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
>> return -ENODEV;
>> }
>> @@ -3217,7 +3257,7 @@ rte_eth_dev_get_5tuple_filter(uint8_t port_id, uint16_t index,
>> if (filter == NULL || rx_queue == NULL)
>> return -EINVAL;
>>
>> - if (port_id >= nb_ports) {
>> + if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
>> PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
>> return -ENODEV;
>> }
>> @@ -3234,7 +3274,7 @@ rte_eth_dev_add_flex_filter(uint8_t port_id, uint16_t index, {
>> struct rte_eth_dev *dev;
>>
>> - if (port_id >= nb_ports) {
>> + if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
>> PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
>> return -ENODEV;
>> }
>> @@ -3249,7 +3289,7 @@ rte_eth_dev_remove_flex_filter(uint8_t port_id, uint16_t index) {
>> struct rte_eth_dev *dev;
>>
>> - if (port_id >= nb_ports) {
>> + if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
>> PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
>> return -ENODEV;
>> }
>> @@ -3268,7 +3308,7 @@ rte_eth_dev_get_flex_filter(uint8_t port_id, uint16_t index,
>> if (filter == NULL || rx_queue == NULL)
>> return -EINVAL;
>>
>> - if (port_id >= nb_ports) {
>> + if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
>> PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
>> return -ENODEV;
>> }
>> @@ -3284,7 +3324,7 @@ rte_eth_dev_filter_supported(uint8_t port_id, enum rte_filter_type
>> filter_type) {
>> struct rte_eth_dev *dev;
>>
>> - if (port_id >= nb_ports) {
>> + if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
>> PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
>> return -ENODEV;
>> }
>> @@ -3301,7 +3341,7 @@ rte_eth_dev_filter_ctrl(uint8_t port_id, enum rte_filter_type filter_type, {
>> struct rte_eth_dev *dev;
>>
>> - if (port_id >= nb_ports) {
>> + if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) {
>> PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
>> return -ENODEV;
>> }
>> diff --git a/lib/librte_ether/rte_ethdev.h b/lib/librte_ether/rte_ethdev.h index ce0528f..616a44a
>> 100644
>> --- a/lib/librte_ether/rte_ethdev.h
>> +++ b/lib/librte_ether/rte_ethdev.h
>> @@ -1565,6 +1565,7 @@ struct rte_eth_dev {
>> struct eth_dev_ops *dev_ops; /**< Functions exported by PMD */
>> struct rte_pci_device *pci_dev; /**< PCI info. supplied by probing */
>> struct rte_eth_dev_cb_list callbacks; /**< User application callbacks */
>> + uint8_t attached; /**< Flag indicating the port is attached */
>> };
>>
>> struct rte_eth_dev_sriov {
>> @@ -1630,6 +1631,10 @@ extern struct rte_eth_dev rte_eth_devices[];
>> * initialized by the [matching] Ethernet driver during the PCI probing phase.
>> * All devices whose port identifier is in the range
>> * [0, rte_eth_dev_count() - 1] can be operated on by network applications.
>> + * immediately after invoking rte_eal_init().
>> + * If the application unplugs a port using hotplug function, The
>> + enabled port
>> + * numbers may be noncontiguous. In the case, the applications need to
>> + manage
>> + * enabled port by themselves.
>> *
>> * @return
>> * - The total number of usable Ethernet devices.
>> --
>> 1.9.1
>
Hi Michael, On 2015/01/19 23:24, Qiu, Michael wrote: > Hi, Tetsuya > > You see lots of places have below three lines: > > if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) { > PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); > return -EINVAL; > } > > They are all the same(only few has the print log different), so can we improve it? > > See below: > > +static int > +rte_eth_dev_validate_port(uint8_t port_id, bool trace) { > + if (port_id >= RTE_MAX_ETHPORTS || > + rte_eth_devices[port_id].attached != DEV_CONNECTED) { > + if (trace) > + PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); > + return DEV_INVALID; > + } > + else > + return DEV_VALID; > +} > > For normal case we just call this function use > rte_eth_dev_validate_port(port_id, 1) > here 1 could be a enmu value(Thus trace should be defined as int). > After call, we didn't need to add PMD_DEBUG_TRACE() any more. > > For few cases like: > PMD_DEBUG_TRACE("set VF rate limit:invalid port id=%d\n", > port_id); > > We can call the function with secondary param set to "0", and add the trace log after the function called, just as before. > > I think after this enhancement, the code seems more clean and efficiency。 I appreciate your comment. Sounds nice. I changes like above. Thanks, Tetsuya > > Thanks, > Michael > >>> -----Original Message----- >>> From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Tetsuya Mukawa >>> Sent: Monday, January 19, 2015 10:40 AM >>> To: dev@dpdk.org >>> Subject: [dpdk-dev] [PATCH v4 01/11] eal/pci, ethdev: Remove assumption that port will not be >>> detached >>> >>> To remove assumption, do like followings. >>> >>> This patch adds "RTE_PCI_DRV_DETACHABLE" to drv_flags of rte_pci_driver structure. The flags >>> indicates the driver can detach devices at runtime. >>> Also remove assumption that port will not be detached. >>> >>> To remove the assumption. >>> - Add 'attached' member to rte_eth_dev structure. >>> This member is used for indicating the port is attached, or not. >>> - Add rte_eth_dev_allocate_new_port(). >>> This function is used for allocating new port. >>> >>> v4: >>> - Use braces with 'for' loop. >>> - Fix indent of 'if' statement. >>> >>> Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp> >>> --- >>> lib/librte_eal/common/include/rte_pci.h | 2 + >>> lib/librte_ether/rte_ethdev.c | 248 ++++++++++++++++++-------------- >>> lib/librte_ether/rte_ethdev.h | 5 + >>> 3 files changed, 151 insertions(+), 104 deletions(-) >>> >>> diff --git a/lib/librte_eal/common/include/rte_pci.h b/lib/librte_eal/common/include/rte_pci.h >>> index 66ed793..dd1da28 100644 >>> --- a/lib/librte_eal/common/include/rte_pci.h >>> +++ b/lib/librte_eal/common/include/rte_pci.h >>> @@ -199,6 +199,8 @@ struct rte_pci_driver { #define RTE_PCI_DRV_FORCE_UNBIND 0x0004 >>> /** Device driver supports link state interrupt */ >>> #define RTE_PCI_DRV_INTR_LSC 0x0008 >>> +/** Device driver supports detaching capability */ >>> +#define RTE_PCI_DRV_DETACHABLE 0x0010 >>> >>> /**< Internal use only - Macro used by pci addr parsing functions **/ >>> #define GET_PCIADDR_FIELD(in, fd, lim, dlm) \ >>> diff --git a/lib/librte_ether/rte_ethdev.c b/lib/librte_ether/rte_ethdev.c index 077d430..46cabaf >>> 100644 >>> --- a/lib/librte_ether/rte_ethdev.c >>> +++ b/lib/librte_ether/rte_ethdev.c >>> @@ -175,6 +175,16 @@ enum { >>> STAT_QMAP_RX >>> }; >>> >>> +enum { >>> + DEV_INVALID = 0, >>> + DEV_VALID, >>> +}; >>> + >>> +enum { >>> + DEV_DISCONNECTED = 0, >>> + DEV_CONNECTED >>> +}; >>> + >>> static inline void >>> rte_eth_dev_data_alloc(void) >>> { >>> @@ -201,19 +211,34 @@ rte_eth_dev_allocated(const char *name) { >>> unsigned i; >>> >>> - for (i = 0; i < nb_ports; i++) { >>> - if (strcmp(rte_eth_devices[i].data->name, name) == 0) >>> + for (i = 0; i < RTE_MAX_ETHPORTS; i++) { >>> + if ((rte_eth_devices[i].attached == DEV_CONNECTED) && >>> + strcmp(rte_eth_devices[i].data->name, name) == 0) >>> return &rte_eth_devices[i]; >>> } >>> return NULL; >>> } >>> >>> +static uint8_t >>> +rte_eth_dev_allocate_new_port(void) >>> +{ >>> + unsigned i; >>> + >>> + for (i = 0; i < RTE_MAX_ETHPORTS; i++) { >>> + if (rte_eth_devices[i].attached == DEV_DISCONNECTED) >>> + return i; >>> + } >>> + return RTE_MAX_ETHPORTS; >>> +} >>> + >>> struct rte_eth_dev * >>> rte_eth_dev_allocate(const char *name) >>> { >>> + uint8_t port_id; >>> struct rte_eth_dev *eth_dev; >>> >>> - if (nb_ports == RTE_MAX_ETHPORTS) { >>> + port_id = rte_eth_dev_allocate_new_port(); >>> + if (port_id == RTE_MAX_ETHPORTS) { >>> PMD_DEBUG_TRACE("Reached maximum number of Ethernet ports\n"); >>> return NULL; >>> } >>> @@ -226,10 +251,12 @@ rte_eth_dev_allocate(const char *name) >>> return NULL; >>> } >>> >>> - eth_dev = &rte_eth_devices[nb_ports]; >>> - eth_dev->data = &rte_eth_dev_data[nb_ports]; >>> + eth_dev = &rte_eth_devices[port_id]; >>> + eth_dev->data = &rte_eth_dev_data[port_id]; >>> snprintf(eth_dev->data->name, sizeof(eth_dev->data->name), "%s", name); >>> - eth_dev->data->port_id = nb_ports++; >>> + eth_dev->data->port_id = port_id; >>> + eth_dev->attached = DEV_CONNECTED; >>> + nb_ports++; >>> return eth_dev; >>> } >>> >>> @@ -283,6 +310,7 @@ rte_eth_dev_init(struct rte_pci_driver *pci_drv, >>> (unsigned) pci_dev->id.device_id); >>> if (rte_eal_process_type() == RTE_PROC_PRIMARY) >>> rte_free(eth_dev->data->dev_private); >>> + eth_dev->attached = DEV_DISCONNECTED; >>> nb_ports--; >>> return diag; >>> } >>> @@ -308,10 +336,22 @@ rte_eth_driver_register(struct eth_driver *eth_drv) >>> rte_eal_pci_register(ð_drv->pci_drv); >>> } >>> >>> +static int >>> +rte_eth_dev_validate_port(uint8_t port_id) { >>> + if (port_id >= RTE_MAX_ETHPORTS) >>> + return DEV_INVALID; >>> + >>> + if (rte_eth_devices[port_id].attached == DEV_CONNECTED) >>> + return DEV_VALID; >>> + else >>> + return DEV_INVALID; >>> +} >>> + >>> int >>> rte_eth_dev_socket_id(uint8_t port_id) >>> { >>> - if (port_id >= nb_ports) >>> + if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) >>> return -1; >>> return rte_eth_devices[port_id].pci_dev->numa_node; >>> } >>> @@ -369,7 +409,7 @@ rte_eth_dev_rx_queue_start(uint8_t port_id, uint16_t rx_queue_id) >>> * in a multi-process setup*/ >>> PROC_PRIMARY_OR_ERR_RET(-E_RTE_SECONDARY); >>> >>> - if (port_id >= nb_ports) { >>> + if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) { >>> PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); >>> return -EINVAL; >>> } >>> @@ -395,7 +435,7 @@ rte_eth_dev_rx_queue_stop(uint8_t port_id, uint16_t rx_queue_id) >>> * in a multi-process setup*/ >>> PROC_PRIMARY_OR_ERR_RET(-E_RTE_SECONDARY); >>> >>> - if (port_id >= nb_ports) { >>> + if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) { >>> PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); >>> return -EINVAL; >>> } >>> @@ -421,7 +461,7 @@ rte_eth_dev_tx_queue_start(uint8_t port_id, uint16_t tx_queue_id) >>> * in a multi-process setup*/ >>> PROC_PRIMARY_OR_ERR_RET(-E_RTE_SECONDARY); >>> >>> - if (port_id >= nb_ports) { >>> + if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) { >>> PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); >>> return -EINVAL; >>> } >>> @@ -447,7 +487,7 @@ rte_eth_dev_tx_queue_stop(uint8_t port_id, uint16_t tx_queue_id) >>> * in a multi-process setup*/ >>> PROC_PRIMARY_OR_ERR_RET(-E_RTE_SECONDARY); >>> >>> - if (port_id >= nb_ports) { >>> + if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) { >>> PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); >>> return -EINVAL; >>> } >>> @@ -703,7 +743,7 @@ rte_eth_dev_configure(uint8_t port_id, uint16_t nb_rx_q, uint16_t nb_tx_q, >>> * in a multi-process setup*/ >>> PROC_PRIMARY_OR_ERR_RET(-E_RTE_SECONDARY); >>> >>> - if (port_id >= nb_ports || port_id >= RTE_MAX_ETHPORTS) { >>> + if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) { >>> PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); >>> return (-EINVAL); >>> } >>> @@ -888,7 +928,7 @@ rte_eth_dev_start(uint8_t port_id) >>> * in a multi-process setup*/ >>> PROC_PRIMARY_OR_ERR_RET(-E_RTE_SECONDARY); >>> >>> - if (port_id >= nb_ports) { >>> + if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) { >>> PMD_DEBUG_TRACE("Invalid port_id=%" PRIu8 "\n", port_id); >>> return (-EINVAL); >>> } >>> @@ -923,7 +963,7 @@ rte_eth_dev_stop(uint8_t port_id) >>> * in a multi-process setup*/ >>> PROC_PRIMARY_OR_RET(); >>> >>> - if (port_id >= nb_ports) { >>> + if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) { >>> PMD_DEBUG_TRACE("Invalid port_id=%" PRIu8 "\n", port_id); >>> return; >>> } >>> @@ -951,7 +991,7 @@ rte_eth_dev_set_link_up(uint8_t port_id) >>> * in a multi-process setup*/ >>> PROC_PRIMARY_OR_ERR_RET(-E_RTE_SECONDARY); >>> >>> - if (port_id >= nb_ports) { >>> + if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) { >>> PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); >>> return -EINVAL; >>> } >>> @@ -970,7 +1010,7 @@ rte_eth_dev_set_link_down(uint8_t port_id) >>> * in a multi-process setup*/ >>> PROC_PRIMARY_OR_ERR_RET(-E_RTE_SECONDARY); >>> >>> - if (port_id >= nb_ports) { >>> + if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) { >>> PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); >>> return -EINVAL; >>> } >>> @@ -989,7 +1029,7 @@ rte_eth_dev_close(uint8_t port_id) >>> * in a multi-process setup*/ >>> PROC_PRIMARY_OR_RET(); >>> >>> - if (port_id >= nb_ports) { >>> + if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) { >>> PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); >>> return; >>> } >>> @@ -1017,7 +1057,7 @@ rte_eth_rx_queue_setup(uint8_t port_id, uint16_t rx_queue_id, >>> * in a multi-process setup*/ >>> PROC_PRIMARY_OR_ERR_RET(-E_RTE_SECONDARY); >>> >>> - if (port_id >= nb_ports) { >>> + if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) { >>> PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); >>> return (-EINVAL); >>> } >>> @@ -1090,7 +1130,7 @@ rte_eth_tx_queue_setup(uint8_t port_id, uint16_t tx_queue_id, >>> * in a multi-process setup*/ >>> PROC_PRIMARY_OR_ERR_RET(-E_RTE_SECONDARY); >>> >>> - if (port_id >= RTE_MAX_ETHPORTS || port_id >= nb_ports) { >>> + if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) { >>> PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); >>> return (-EINVAL); >>> } >>> @@ -1123,7 +1163,7 @@ rte_eth_promiscuous_enable(uint8_t port_id) { >>> struct rte_eth_dev *dev; >>> >>> - if (port_id >= nb_ports) { >>> + if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) { >>> PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); >>> return; >>> } >>> @@ -1139,7 +1179,7 @@ rte_eth_promiscuous_disable(uint8_t port_id) { >>> struct rte_eth_dev *dev; >>> >>> - if (port_id >= nb_ports) { >>> + if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) { >>> PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); >>> return; >>> } >>> @@ -1155,7 +1195,7 @@ rte_eth_promiscuous_get(uint8_t port_id) { >>> struct rte_eth_dev *dev; >>> >>> - if (port_id >= nb_ports) { >>> + if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) { >>> PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); >>> return -1; >>> } >>> @@ -1169,7 +1209,7 @@ rte_eth_allmulticast_enable(uint8_t port_id) { >>> struct rte_eth_dev *dev; >>> >>> - if (port_id >= nb_ports) { >>> + if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) { >>> PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); >>> return; >>> } >>> @@ -1185,7 +1225,7 @@ rte_eth_allmulticast_disable(uint8_t port_id) { >>> struct rte_eth_dev *dev; >>> >>> - if (port_id >= nb_ports) { >>> + if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) { >>> PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); >>> return; >>> } >>> @@ -1201,7 +1241,7 @@ rte_eth_allmulticast_get(uint8_t port_id) { >>> struct rte_eth_dev *dev; >>> >>> - if (port_id >= nb_ports) { >>> + if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) { >>> PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); >>> return -1; >>> } >>> @@ -1229,7 +1269,7 @@ rte_eth_link_get(uint8_t port_id, struct rte_eth_link *eth_link) { >>> struct rte_eth_dev *dev; >>> >>> - if (port_id >= nb_ports) { >>> + if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) { >>> PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); >>> return; >>> } >>> @@ -1249,7 +1289,7 @@ rte_eth_link_get_nowait(uint8_t port_id, struct rte_eth_link *eth_link) { >>> struct rte_eth_dev *dev; >>> >>> - if (port_id >= nb_ports) { >>> + if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) { >>> PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); >>> return; >>> } >>> @@ -1269,7 +1309,7 @@ rte_eth_stats_get(uint8_t port_id, struct rte_eth_stats *stats) { >>> struct rte_eth_dev *dev; >>> >>> - if (port_id >= nb_ports) { >>> + if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) { >>> PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); >>> return; >>> } >>> @@ -1286,7 +1326,7 @@ rte_eth_stats_reset(uint8_t port_id) { >>> struct rte_eth_dev *dev; >>> >>> - if (port_id >= nb_ports) { >>> + if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) { >>> PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); >>> return; >>> } >>> @@ -1307,7 +1347,7 @@ rte_eth_xstats_get(uint8_t port_id, struct rte_eth_xstats *xstats, >>> uint64_t val; >>> char *stats_ptr; >>> >>> - if (port_id >= nb_ports) { >>> + if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) { >>> PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); >>> return -1; >>> } >>> @@ -1376,7 +1416,7 @@ rte_eth_xstats_reset(uint8_t port_id) { >>> struct rte_eth_dev *dev; >>> >>> - if (port_id >= nb_ports) { >>> + if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) { >>> PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); >>> return; >>> } >>> @@ -1398,7 +1438,7 @@ set_queue_stats_mapping(uint8_t port_id, uint16_t queue_id, uint8_t >>> stat_idx, { >>> struct rte_eth_dev *dev; >>> >>> - if (port_id >= nb_ports) { >>> + if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) { >>> PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); >>> return -ENODEV; >>> } >>> @@ -1433,7 +1473,7 @@ rte_eth_dev_info_get(uint8_t port_id, struct rte_eth_dev_info *dev_info) >>> { >>> struct rte_eth_dev *dev; >>> >>> - if (port_id >= nb_ports) { >>> + if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) { >>> PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); >>> return; >>> } >>> @@ -1453,7 +1493,7 @@ rte_eth_macaddr_get(uint8_t port_id, struct ether_addr *mac_addr) { >>> struct rte_eth_dev *dev; >>> >>> - if (port_id >= nb_ports) { >>> + if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) { >>> PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); >>> return; >>> } >>> @@ -1467,7 +1507,7 @@ rte_eth_dev_get_mtu(uint8_t port_id, uint16_t *mtu) { >>> struct rte_eth_dev *dev; >>> >>> - if (port_id >= nb_ports) { >>> + if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) { >>> PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); >>> return (-ENODEV); >>> } >>> @@ -1483,7 +1523,7 @@ rte_eth_dev_set_mtu(uint8_t port_id, uint16_t mtu) >>> int ret; >>> struct rte_eth_dev *dev; >>> >>> - if (port_id >= nb_ports) { >>> + if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) { >>> PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); >>> return (-ENODEV); >>> } >>> @@ -1503,7 +1543,7 @@ rte_eth_dev_vlan_filter(uint8_t port_id, uint16_t vlan_id, int on) { >>> struct rte_eth_dev *dev; >>> >>> - if (port_id >= nb_ports) { >>> + if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) { >>> PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); >>> return (-ENODEV); >>> } >>> @@ -1528,7 +1568,7 @@ rte_eth_dev_set_vlan_strip_on_queue(uint8_t port_id, uint16_t >>> rx_queue_id, int o { >>> struct rte_eth_dev *dev; >>> >>> - if (port_id >= nb_ports) { >>> + if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) { >>> PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); >>> return (-ENODEV); >>> } >>> @@ -1550,7 +1590,7 @@ rte_eth_dev_set_vlan_ether_type(uint8_t port_id, uint16_t tpid) { >>> struct rte_eth_dev *dev; >>> >>> - if (port_id >= nb_ports) { >>> + if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) { >>> PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); >>> return (-ENODEV); >>> } >>> @@ -1570,7 +1610,7 @@ rte_eth_dev_set_vlan_offload(uint8_t port_id, int offload_mask) >>> int mask = 0; >>> int cur, org = 0; >>> >>> - if (port_id >= nb_ports) { >>> + if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) { >>> PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); >>> return (-ENODEV); >>> } >>> @@ -1615,7 +1655,7 @@ rte_eth_dev_get_vlan_offload(uint8_t port_id) >>> struct rte_eth_dev *dev; >>> int ret = 0; >>> >>> - if (port_id >= nb_ports) { >>> + if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) { >>> PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); >>> return (-ENODEV); >>> } >>> @@ -1639,7 +1679,7 @@ rte_eth_dev_set_vlan_pvid(uint8_t port_id, uint16_t pvid, int on) { >>> struct rte_eth_dev *dev; >>> >>> - if (port_id >= nb_ports) { >>> + if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) { >>> PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); >>> return (-ENODEV); >>> } >>> @@ -1657,7 +1697,7 @@ rte_eth_dev_fdir_add_signature_filter(uint8_t port_id, { >>> struct rte_eth_dev *dev; >>> >>> - if (port_id >= nb_ports) { >>> + if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) { >>> PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); >>> return (-ENODEV); >>> } >>> @@ -1691,7 +1731,7 @@ rte_eth_dev_fdir_update_signature_filter(uint8_t port_id, { >>> struct rte_eth_dev *dev; >>> >>> - if (port_id >= nb_ports) { >>> + if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) { >>> PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); >>> return (-ENODEV); >>> } >>> @@ -1725,7 +1765,7 @@ rte_eth_dev_fdir_remove_signature_filter(uint8_t port_id, { >>> struct rte_eth_dev *dev; >>> >>> - if (port_id >= nb_ports) { >>> + if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) { >>> PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); >>> return (-ENODEV); >>> } >>> @@ -1756,7 +1796,7 @@ rte_eth_dev_fdir_get_infos(uint8_t port_id, struct rte_eth_fdir *fdir) { >>> struct rte_eth_dev *dev; >>> >>> - if (port_id >= nb_ports) { >>> + if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) { >>> PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); >>> return (-ENODEV); >>> } >>> @@ -1781,7 +1821,7 @@ rte_eth_dev_fdir_add_perfect_filter(uint8_t port_id, { >>> struct rte_eth_dev *dev; >>> >>> - if (port_id >= nb_ports) { >>> + if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) { >>> PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); >>> return (-ENODEV); >>> } >>> @@ -1821,7 +1861,7 @@ rte_eth_dev_fdir_update_perfect_filter(uint8_t port_id, { >>> struct rte_eth_dev *dev; >>> >>> - if (port_id >= nb_ports) { >>> + if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) { >>> PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); >>> return (-ENODEV); >>> } >>> @@ -1859,7 +1899,7 @@ rte_eth_dev_fdir_remove_perfect_filter(uint8_t port_id, { >>> struct rte_eth_dev *dev; >>> >>> - if (port_id >= nb_ports) { >>> + if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) { >>> PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); >>> return (-ENODEV); >>> } >>> @@ -1895,7 +1935,7 @@ rte_eth_dev_fdir_set_masks(uint8_t port_id, struct rte_fdir_masks >>> *fdir_mask) { >>> struct rte_eth_dev *dev; >>> >>> - if (port_id >= nb_ports) { >>> + if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) { >>> PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); >>> return (-ENODEV); >>> } >>> @@ -1915,7 +1955,7 @@ rte_eth_dev_flow_ctrl_get(uint8_t port_id, struct rte_eth_fc_conf >>> *fc_conf) { >>> struct rte_eth_dev *dev; >>> >>> - if (port_id >= nb_ports) { >>> + if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) { >>> PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); >>> return (-ENODEV); >>> } >>> @@ -1931,7 +1971,7 @@ rte_eth_dev_flow_ctrl_set(uint8_t port_id, struct rte_eth_fc_conf >>> *fc_conf) { >>> struct rte_eth_dev *dev; >>> >>> - if (port_id >= nb_ports) { >>> + if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) { >>> PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); >>> return (-ENODEV); >>> } >>> @@ -1951,7 +1991,7 @@ rte_eth_dev_priority_flow_ctrl_set(uint8_t port_id, struct >>> rte_eth_pfc_conf *pfc { >>> struct rte_eth_dev *dev; >>> >>> - if (port_id >= nb_ports) { >>> + if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) { >>> PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); >>> return (-ENODEV); >>> } >>> @@ -2030,7 +2070,7 @@ rte_eth_dev_rss_reta_update(uint8_t port_id, >>> struct rte_eth_dev *dev; >>> int ret; >>> >>> - if (port_id >= nb_ports) { >>> + if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) { >>> PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); >>> return -ENODEV; >>> } >>> @@ -2081,7 +2121,7 @@ rte_eth_dev_rss_hash_update(uint8_t port_id, struct rte_eth_rss_conf >>> *rss_conf) >>> struct rte_eth_dev *dev; >>> uint16_t rss_hash_protos; >>> >>> - if (port_id >= nb_ports) { >>> + if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) { >>> PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); >>> return (-ENODEV); >>> } >>> @@ -2103,7 +2143,7 @@ rte_eth_dev_rss_hash_conf_get(uint8_t port_id, { >>> struct rte_eth_dev *dev; >>> >>> - if (port_id >= nb_ports) { >>> + if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) { >>> PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); >>> return (-ENODEV); >>> } >>> @@ -2118,7 +2158,7 @@ rte_eth_dev_udp_tunnel_add(uint8_t port_id, { >>> struct rte_eth_dev *dev; >>> >>> - if (port_id >= nb_ports) { >>> + if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) { >>> PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); >>> return -ENODEV; >>> } >>> @@ -2144,7 +2184,7 @@ rte_eth_dev_udp_tunnel_delete(uint8_t port_id, { >>> struct rte_eth_dev *dev; >>> >>> - if (port_id >= nb_ports) { >>> + if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) { >>> PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); >>> return -ENODEV; >>> } >>> @@ -2169,7 +2209,7 @@ rte_eth_led_on(uint8_t port_id) { >>> struct rte_eth_dev *dev; >>> >>> - if (port_id >= nb_ports) { >>> + if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) { >>> PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); >>> return (-ENODEV); >>> } >>> @@ -2184,7 +2224,7 @@ rte_eth_led_off(uint8_t port_id) { >>> struct rte_eth_dev *dev; >>> >>> - if (port_id >= nb_ports) { >>> + if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) { >>> PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); >>> return (-ENODEV); >>> } >>> @@ -2224,7 +2264,7 @@ rte_eth_dev_mac_addr_add(uint8_t port_id, struct ether_addr *addr, >>> int index; >>> uint64_t pool_mask; >>> >>> - if (port_id >= nb_ports) { >>> + if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) { >>> PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); >>> return (-ENODEV); >>> } >>> @@ -2275,7 +2315,7 @@ rte_eth_dev_mac_addr_remove(uint8_t port_id, struct ether_addr *addr) >>> struct rte_eth_dev *dev; >>> int index; >>> >>> - if (port_id >= nb_ports) { >>> + if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) { >>> PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); >>> return (-ENODEV); >>> } >>> @@ -2309,7 +2349,7 @@ rte_eth_dev_set_vf_rxmode(uint8_t port_id, uint16_t vf, >>> struct rte_eth_dev *dev; >>> struct rte_eth_dev_info dev_info; >>> >>> - if (port_id >= nb_ports) { >>> + if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) { >>> PMD_DEBUG_TRACE("set VF RX mode:Invalid port_id=%d\n", >>> port_id); >>> return (-ENODEV); >>> @@ -2364,7 +2404,7 @@ rte_eth_dev_uc_hash_table_set(uint8_t port_id, struct ether_addr *addr, >>> int ret; >>> struct rte_eth_dev *dev; >>> >>> - if (port_id >= nb_ports) { >>> + if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) { >>> PMD_DEBUG_TRACE("unicast hash setting:Invalid port_id=%d\n", >>> port_id); >>> return (-ENODEV); >>> @@ -2417,7 +2457,7 @@ rte_eth_dev_uc_all_hash_table_set(uint8_t port_id, uint8_t on) { >>> struct rte_eth_dev *dev; >>> >>> - if (port_id >= nb_ports) { >>> + if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) { >>> PMD_DEBUG_TRACE("unicast hash setting:Invalid port_id=%d\n", >>> port_id); >>> return (-ENODEV); >>> @@ -2436,7 +2476,7 @@ rte_eth_dev_set_vf_rx(uint8_t port_id,uint16_t vf, uint8_t on) >>> struct rte_eth_dev *dev; >>> struct rte_eth_dev_info dev_info; >>> >>> - if (port_id >= nb_ports) { >>> + if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) { >>> PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); >>> return (-ENODEV); >>> } >>> @@ -2462,7 +2502,7 @@ rte_eth_dev_set_vf_tx(uint8_t port_id,uint16_t vf, uint8_t on) >>> struct rte_eth_dev *dev; >>> struct rte_eth_dev_info dev_info; >>> >>> - if (port_id >= nb_ports) { >>> + if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) { >>> PMD_DEBUG_TRACE("set pool tx:Invalid port_id=%d\n", port_id); >>> return (-ENODEV); >>> } >>> @@ -2487,7 +2527,7 @@ rte_eth_dev_set_vf_vlan_filter(uint8_t port_id, uint16_t vlan_id, { >>> struct rte_eth_dev *dev; >>> >>> - if (port_id >= nb_ports) { >>> + if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) { >>> PMD_DEBUG_TRACE("VF VLAN filter:invalid port id=%d\n", >>> port_id); >>> return (-ENODEV); >>> @@ -2518,7 +2558,7 @@ int rte_eth_set_queue_rate_limit(uint8_t port_id, uint16_t queue_idx, >>> struct rte_eth_dev_info dev_info; >>> struct rte_eth_link link; >>> >>> - if (port_id >= nb_ports) { >>> + if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) { >>> PMD_DEBUG_TRACE("set queue rate limit:invalid port id=%d\n", >>> port_id); >>> return -ENODEV; >>> @@ -2555,7 +2595,7 @@ int rte_eth_set_vf_rate_limit(uint8_t port_id, uint16_t vf, uint16_t tx_rate, >>> if (q_msk == 0) >>> return 0; >>> >>> - if (port_id >= nb_ports) { >>> + if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) { >>> PMD_DEBUG_TRACE("set VF rate limit:invalid port id=%d\n", >>> port_id); >>> return -ENODEV; >>> @@ -2589,7 +2629,7 @@ rte_eth_mirror_rule_set(uint8_t port_id, { >>> struct rte_eth_dev *dev = &rte_eth_devices[port_id]; >>> >>> - if (port_id >= nb_ports) { >>> + if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) { >>> PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); >>> return (-ENODEV); >>> } >>> @@ -2630,7 +2670,7 @@ rte_eth_mirror_rule_reset(uint8_t port_id, uint8_t rule_id) { >>> struct rte_eth_dev *dev = &rte_eth_devices[port_id]; >>> >>> - if (port_id >= nb_ports) { >>> + if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) { >>> PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); >>> return (-ENODEV); >>> } >>> @@ -2655,7 +2695,7 @@ rte_eth_rx_burst(uint8_t port_id, uint16_t queue_id, { >>> struct rte_eth_dev *dev; >>> >>> - if (port_id >= nb_ports) { >>> + if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) { >>> PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); >>> return 0; >>> } >>> @@ -2675,7 +2715,7 @@ rte_eth_tx_burst(uint8_t port_id, uint16_t queue_id, { >>> struct rte_eth_dev *dev; >>> >>> - if (port_id >= nb_ports) { >>> + if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) { >>> PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); >>> return 0; >>> } >>> @@ -2695,7 +2735,7 @@ rte_eth_rx_queue_count(uint8_t port_id, uint16_t queue_id) { >>> struct rte_eth_dev *dev; >>> >>> - if (port_id >= nb_ports) { >>> + if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) { >>> PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); >>> return 0; >>> } >>> @@ -2709,7 +2749,7 @@ rte_eth_rx_descriptor_done(uint8_t port_id, uint16_t queue_id, uint16_t >>> offset) { >>> struct rte_eth_dev *dev; >>> >>> - if (port_id >= nb_ports) { >>> + if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) { >>> PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); >>> return (-ENODEV); >>> } >>> @@ -2730,7 +2770,7 @@ rte_eth_dev_callback_register(uint8_t port_id, >>> >>> if (!cb_fn) >>> return (-EINVAL); >>> - if (port_id >= nb_ports) { >>> + if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) { >>> PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); >>> return (-EINVAL); >>> } >>> @@ -2770,7 +2810,7 @@ rte_eth_dev_callback_unregister(uint8_t port_id, >>> >>> if (!cb_fn) >>> return (-EINVAL); >>> - if (port_id >= nb_ports) { >>> + if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) { >>> PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); >>> return (-EINVAL); >>> } >>> @@ -2830,7 +2870,7 @@ int rte_eth_dev_bypass_init(uint8_t port_id) { >>> struct rte_eth_dev *dev; >>> >>> - if (port_id >= nb_ports) { >>> + if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) { >>> PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); >>> return (-ENODEV); >>> } >>> @@ -2850,7 +2890,7 @@ rte_eth_dev_bypass_state_show(uint8_t port_id, uint32_t *state) { >>> struct rte_eth_dev *dev; >>> >>> - if (port_id >= nb_ports) { >>> + if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) { >>> PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); >>> return (-ENODEV); >>> } >>> @@ -2869,7 +2909,7 @@ rte_eth_dev_bypass_state_set(uint8_t port_id, uint32_t *new_state) { >>> struct rte_eth_dev *dev; >>> >>> - if (port_id >= nb_ports) { >>> + if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) { >>> PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); >>> return (-ENODEV); >>> } >>> @@ -2889,7 +2929,7 @@ rte_eth_dev_bypass_event_show(uint8_t port_id, uint32_t event, uint32_t >>> *state) { >>> struct rte_eth_dev *dev; >>> >>> - if (port_id >= nb_ports) { >>> + if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) { >>> PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); >>> return (-ENODEV); >>> } >>> @@ -2909,7 +2949,7 @@ rte_eth_dev_bypass_event_store(uint8_t port_id, uint32_t event, uint32_t >>> state) { >>> struct rte_eth_dev *dev; >>> >>> - if (port_id >= nb_ports) { >>> + if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) { >>> PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); >>> return (-ENODEV); >>> } >>> @@ -2929,7 +2969,7 @@ rte_eth_dev_wd_timeout_store(uint8_t port_id, uint32_t timeout) { >>> struct rte_eth_dev *dev; >>> >>> - if (port_id >= nb_ports) { >>> + if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) { >>> PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); >>> return (-ENODEV); >>> } >>> @@ -2949,7 +2989,7 @@ rte_eth_dev_bypass_ver_show(uint8_t port_id, uint32_t *ver) { >>> struct rte_eth_dev *dev; >>> >>> - if (port_id >= nb_ports) { >>> + if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) { >>> PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); >>> return (-ENODEV); >>> } >>> @@ -2969,7 +3009,7 @@ rte_eth_dev_bypass_wd_timeout_show(uint8_t port_id, uint32_t >>> *wd_timeout) { >>> struct rte_eth_dev *dev; >>> >>> - if (port_id >= nb_ports) { >>> + if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) { >>> PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); >>> return (-ENODEV); >>> } >>> @@ -2989,7 +3029,7 @@ rte_eth_dev_bypass_wd_reset(uint8_t port_id) { >>> struct rte_eth_dev *dev; >>> >>> - if (port_id >= nb_ports) { >>> + if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) { >>> PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); >>> return (-ENODEV); >>> } >>> @@ -3011,7 +3051,7 @@ rte_eth_dev_add_syn_filter(uint8_t port_id, { >>> struct rte_eth_dev *dev; >>> >>> - if (port_id >= nb_ports) { >>> + if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) { >>> PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); >>> return -ENODEV; >>> } >>> @@ -3026,7 +3066,7 @@ rte_eth_dev_remove_syn_filter(uint8_t port_id) { >>> struct rte_eth_dev *dev; >>> >>> - if (port_id >= nb_ports) { >>> + if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) { >>> PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); >>> return -ENODEV; >>> } >>> @@ -3045,7 +3085,7 @@ rte_eth_dev_get_syn_filter(uint8_t port_id, >>> if (filter == NULL || rx_queue == NULL) >>> return -EINVAL; >>> >>> - if (port_id >= nb_ports) { >>> + if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) { >>> PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); >>> return -ENODEV; >>> } >>> @@ -3061,7 +3101,7 @@ rte_eth_dev_add_ethertype_filter(uint8_t port_id, uint16_t index, { >>> struct rte_eth_dev *dev; >>> >>> - if (port_id >= nb_ports) { >>> + if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) { >>> PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); >>> return -ENODEV; >>> } >>> @@ -3082,7 +3122,7 @@ rte_eth_dev_remove_ethertype_filter(uint8_t port_id, uint16_t index) { >>> struct rte_eth_dev *dev; >>> >>> - if (port_id >= nb_ports) { >>> + if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) { >>> PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); >>> return -ENODEV; >>> } >>> @@ -3101,7 +3141,7 @@ rte_eth_dev_get_ethertype_filter(uint8_t port_id, uint16_t index, >>> if (filter == NULL || rx_queue == NULL) >>> return -EINVAL; >>> >>> - if (port_id >= nb_ports) { >>> + if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) { >>> PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); >>> return -ENODEV; >>> } >>> @@ -3118,7 +3158,7 @@ rte_eth_dev_add_2tuple_filter(uint8_t port_id, uint16_t index, { >>> struct rte_eth_dev *dev; >>> >>> - if (port_id >= nb_ports) { >>> + if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) { >>> PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); >>> return -ENODEV; >>> } >>> @@ -3140,7 +3180,7 @@ rte_eth_dev_remove_2tuple_filter(uint8_t port_id, uint16_t index) { >>> struct rte_eth_dev *dev; >>> >>> - if (port_id >= nb_ports) { >>> + if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) { >>> PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); >>> return -ENODEV; >>> } >>> @@ -3159,7 +3199,7 @@ rte_eth_dev_get_2tuple_filter(uint8_t port_id, uint16_t index, >>> if (filter == NULL || rx_queue == NULL) >>> return -EINVAL; >>> >>> - if (port_id >= nb_ports) { >>> + if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) { >>> PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); >>> return -ENODEV; >>> } >>> @@ -3175,7 +3215,7 @@ rte_eth_dev_add_5tuple_filter(uint8_t port_id, uint16_t index, { >>> struct rte_eth_dev *dev; >>> >>> - if (port_id >= nb_ports) { >>> + if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) { >>> PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); >>> return -ENODEV; >>> } >>> @@ -3198,7 +3238,7 @@ rte_eth_dev_remove_5tuple_filter(uint8_t port_id, uint16_t index) { >>> struct rte_eth_dev *dev; >>> >>> - if (port_id >= nb_ports) { >>> + if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) { >>> PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); >>> return -ENODEV; >>> } >>> @@ -3217,7 +3257,7 @@ rte_eth_dev_get_5tuple_filter(uint8_t port_id, uint16_t index, >>> if (filter == NULL || rx_queue == NULL) >>> return -EINVAL; >>> >>> - if (port_id >= nb_ports) { >>> + if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) { >>> PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); >>> return -ENODEV; >>> } >>> @@ -3234,7 +3274,7 @@ rte_eth_dev_add_flex_filter(uint8_t port_id, uint16_t index, { >>> struct rte_eth_dev *dev; >>> >>> - if (port_id >= nb_ports) { >>> + if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) { >>> PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); >>> return -ENODEV; >>> } >>> @@ -3249,7 +3289,7 @@ rte_eth_dev_remove_flex_filter(uint8_t port_id, uint16_t index) { >>> struct rte_eth_dev *dev; >>> >>> - if (port_id >= nb_ports) { >>> + if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) { >>> PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); >>> return -ENODEV; >>> } >>> @@ -3268,7 +3308,7 @@ rte_eth_dev_get_flex_filter(uint8_t port_id, uint16_t index, >>> if (filter == NULL || rx_queue == NULL) >>> return -EINVAL; >>> >>> - if (port_id >= nb_ports) { >>> + if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) { >>> PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); >>> return -ENODEV; >>> } >>> @@ -3284,7 +3324,7 @@ rte_eth_dev_filter_supported(uint8_t port_id, enum rte_filter_type >>> filter_type) { >>> struct rte_eth_dev *dev; >>> >>> - if (port_id >= nb_ports) { >>> + if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) { >>> PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); >>> return -ENODEV; >>> } >>> @@ -3301,7 +3341,7 @@ rte_eth_dev_filter_ctrl(uint8_t port_id, enum rte_filter_type filter_type, { >>> struct rte_eth_dev *dev; >>> >>> - if (port_id >= nb_ports) { >>> + if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) { >>> PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); >>> return -ENODEV; >>> } >>> diff --git a/lib/librte_ether/rte_ethdev.h b/lib/librte_ether/rte_ethdev.h index ce0528f..616a44a >>> 100644 >>> --- a/lib/librte_ether/rte_ethdev.h >>> +++ b/lib/librte_ether/rte_ethdev.h >>> @@ -1565,6 +1565,7 @@ struct rte_eth_dev { >>> struct eth_dev_ops *dev_ops; /**< Functions exported by PMD */ >>> struct rte_pci_device *pci_dev; /**< PCI info. supplied by probing */ >>> struct rte_eth_dev_cb_list callbacks; /**< User application callbacks */ >>> + uint8_t attached; /**< Flag indicating the port is attached */ >>> }; >>> >>> struct rte_eth_dev_sriov { >>> @@ -1630,6 +1631,10 @@ extern struct rte_eth_dev rte_eth_devices[]; >>> * initialized by the [matching] Ethernet driver during the PCI probing phase. >>> * All devices whose port identifier is in the range >>> * [0, rte_eth_dev_count() - 1] can be operated on by network applications. >>> + * immediately after invoking rte_eal_init(). >>> + * If the application unplugs a port using hotplug function, The >>> + enabled port >>> + * numbers may be noncontiguous. In the case, the applications need to >>> + manage >>> + * enabled port by themselves. >>> * >>> * @return >>> * - The total number of usable Ethernet devices. >>> -- >>> 1.9.1
On 1/19/2015 6:42 PM, Tetsuya Mukawa wrote: > The patch adds functions for unmapping igb_uio resources. The patch is only > for Linux and igb_uio environment. VFIO and BSD are not supported. > > v4: > - Add paramerter checking. > - Add header file to determine if hotplug can be enabled. > > Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp> > --- > lib/librte_eal/common/Makefile | 1 + > lib/librte_eal/common/include/rte_dev_hotplug.h | 44 +++++++++++++++++ > lib/librte_eal/linuxapp/eal/eal_pci.c | 38 +++++++++++++++ > lib/librte_eal/linuxapp/eal/eal_pci_init.h | 8 +++ > lib/librte_eal/linuxapp/eal/eal_pci_uio.c | 65 +++++++++++++++++++++++++ > 5 files changed, 156 insertions(+) > create mode 100644 lib/librte_eal/common/include/rte_dev_hotplug.h > > diff --git a/lib/librte_eal/common/Makefile b/lib/librte_eal/common/Makefile > index 52c1a5f..db7cc93 100644 > --- a/lib/librte_eal/common/Makefile > +++ b/lib/librte_eal/common/Makefile > @@ -41,6 +41,7 @@ INC += rte_eal_memconfig.h rte_malloc_heap.h > INC += rte_hexdump.h rte_devargs.h rte_dev.h > INC += rte_common_vect.h > INC += rte_pci_dev_feature_defs.h rte_pci_dev_features.h > +INC += rte_dev_hotplug.h > > ifeq ($(CONFIG_RTE_INSECURE_FUNCTION_WARNING),y) > INC += rte_warnings.h > diff --git a/lib/librte_eal/common/include/rte_dev_hotplug.h b/lib/librte_eal/common/include/rte_dev_hotplug.h > new file mode 100644 > index 0000000..b333e0f > --- /dev/null > +++ b/lib/librte_eal/common/include/rte_dev_hotplug.h > @@ -0,0 +1,44 @@ > +/*- > + * BSD LICENSE > + * > + * Copyright(c) 2015 IGEL Co.,LTd. > + * All rights reserved. > + * > + * Redistribution and use in source and binary forms, with or without > + * modification, are permitted provided that the following conditions > + * are met: > + * > + * * Redistributions of source code must retain the above copyright > + * notice, this list of conditions and the following disclaimer. > + * * Redistributions in binary form must reproduce the above copyright > + * notice, this list of conditions and the following disclaimer in > + * the documentation and/or other materials provided with the > + * distribution. > + * * Neither the name of IGEL Co.,Ltd. nor the names of its > + * contributors may be used to endorse or promote products derived > + * from this software without specific prior written permission. > + * > + * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS > + * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT > + * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR > + * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT > + * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, > + * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT > + * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, > + * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY > + * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT > + * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE > + * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. > + */ > + > +#ifndef _RTE_DEV_HOTPLUG_H_ > +#define _RTE_DEV_HOTPLUG_H_ > + > +/* > + * determine if hotplug can be enabled on the system > + */ > +#if defined(RTE_LIBRTE_EAL_HOTPLUG) && defined(RTE_LIBRTE_EAL_LINUXAPP) As you said, VFIO should not work with it, so does it need to add the vfio check here? Thanks, Michael > +#define ENABLE_HOTPLUG > +#endif /* RTE_LIBRTE_EAL_HOTPLUG & RTE_LIBRTE_EAL_LINUXAPP */ > + > +#endif /* _RTE_DEV_HOTPLUG_H_ */ > diff --git a/lib/librte_eal/linuxapp/eal/eal_pci.c b/lib/librte_eal/linuxapp/eal/eal_pci.c > index 3d2d93c..52c464c 100644 > --- a/lib/librte_eal/linuxapp/eal/eal_pci.c > +++ b/lib/librte_eal/linuxapp/eal/eal_pci.c > @@ -137,6 +137,25 @@ pci_map_resource(void *requested_addr, int fd, off_t offset, size_t size) > return mapaddr; > } > > +#ifdef ENABLE_HOTPLUG > +/* unmap a particular resource */ > +void > +pci_unmap_resource(void *requested_addr, size_t size) > +{ > + if (requested_addr == NULL) > + return; > + > + /* Unmap the PCI memory resource of device */ > + if (munmap(requested_addr, size)) { > + RTE_LOG(ERR, EAL, "%s(): cannot munmap(%p, 0x%lx): %s\n", > + __func__, requested_addr, (unsigned long)size, > + strerror(errno)); > + } else > + RTE_LOG(DEBUG, EAL, " PCI memory mapped at %p\n", > + requested_addr); > +} > +#endif /* ENABLE_HOTPLUG */ > + > /* parse the "resource" sysfs file */ > #define IORESOURCE_MEM 0x00000200 > > @@ -510,6 +529,25 @@ pci_map_device(struct rte_pci_device *dev) > return 0; > } > > +#ifdef ENABLE_HOTPLUG > +static void > +pci_unmap_device(struct rte_pci_device *dev) > +{ > + if (dev == NULL) > + return; > + > + /* try unmapping the NIC resources using VFIO if it exists */ > +#ifdef VFIO_PRESENT > + if (pci_vfio_is_enabled()) { > + RTE_LOG(ERR, EAL, "%s() doesn't support vfio yet.\n", > + __func__); > + return; > + } > +#endif > + pci_uio_unmap_resource(dev); > +} > +#endif /* ENABLE_HOTPLUG */ > + > /* > * If vendor/device ID match, call the devinit() function of the > * driver. > diff --git a/lib/librte_eal/linuxapp/eal/eal_pci_init.h b/lib/librte_eal/linuxapp/eal/eal_pci_init.h > index 1070eb8..5152a0b 100644 > --- a/lib/librte_eal/linuxapp/eal/eal_pci_init.h > +++ b/lib/librte_eal/linuxapp/eal/eal_pci_init.h > @@ -34,6 +34,7 @@ > #ifndef EAL_PCI_INIT_H_ > #define EAL_PCI_INIT_H_ > > +#include <rte_dev_hotplug.h> > #include "eal_vfio.h" > > struct pci_map { > @@ -71,6 +72,13 @@ void *pci_map_resource(void *requested_addr, int fd, off_t offset, > /* map IGB_UIO resource prototype */ > int pci_uio_map_resource(struct rte_pci_device *dev); > > +#ifdef ENABLE_HOTPLUG > +void pci_unmap_resource(void *requested_addr, size_t size); > + > +/* unmap IGB_UIO resource prototype */ > +void pci_uio_unmap_resource(struct rte_pci_device *dev); > +#endif /* ENABLE_HOTPLUG */ > + > #ifdef VFIO_PRESENT > > #define VFIO_MAX_GROUPS 64 > diff --git a/lib/librte_eal/linuxapp/eal/eal_pci_uio.c b/lib/librte_eal/linuxapp/eal/eal_pci_uio.c > index 1da3507..81830d1 100644 > --- a/lib/librte_eal/linuxapp/eal/eal_pci_uio.c > +++ b/lib/librte_eal/linuxapp/eal/eal_pci_uio.c > @@ -404,6 +404,71 @@ pci_uio_map_resource(struct rte_pci_device *dev) > return 0; > } > > +#ifdef ENABLE_HOTPLUG > +static void > +pci_uio_unmap(struct mapped_pci_resource *uio_res) > +{ > + int i; > + > + if (uio_res == NULL) > + return; > + > + for (i = 0; i != uio_res->nb_maps; i++) > + pci_unmap_resource(uio_res->maps[i].addr, > + (size_t)uio_res->maps[i].size); > +} > + > +static struct mapped_pci_resource * > +pci_uio_find_resource(struct rte_pci_device *dev) > +{ > + struct mapped_pci_resource *uio_res; > + > + if (dev == NULL) > + return NULL; > + > + TAILQ_FOREACH(uio_res, pci_res_list, next) { > + > + /* skip this element if it doesn't match our PCI address */ > + if (!eal_compare_pci_addr(&uio_res->pci_addr, &dev->addr)) > + return uio_res; > + } > + return NULL; > +} > + > +/* unmap the PCI resource of a PCI device in virtual memory */ > +void > +pci_uio_unmap_resource(struct rte_pci_device *dev) > +{ > + struct mapped_pci_resource *uio_res; > + > + if (dev == NULL) > + return; > + > + /* find an entry for the device */ > + uio_res = pci_uio_find_resource(dev); > + if (uio_res == NULL) > + return; > + > + /* secondary processes - just free maps */ > + if (rte_eal_process_type() != RTE_PROC_PRIMARY) > + return pci_uio_unmap(uio_res); > + > + TAILQ_REMOVE(pci_res_list, uio_res, next); > + > + /* unmap all resources */ > + pci_uio_unmap(uio_res); > + > + /* free uio resource */ > + rte_free(uio_res); > + > + /* close fd if in primary process */ > + close(dev->intr_handle.fd); > + > + dev->intr_handle.fd = -1; > + dev->intr_handle.type = RTE_INTR_HANDLE_UNKNOWN; > +} > +#endif /* ENABLE_HOTPLUG */ > + > /* > * parse a sysfs file containing one integer value > * different to the eal version, as it needs to work with 64-bit values
Hi Michael, On 2015/01/20 18:23, Qiu, Michael wrote: > On 1/19/2015 6:42 PM, Tetsuya Mukawa wrote: >> The patch adds functions for unmapping igb_uio resources. The patch is only >> for Linux and igb_uio environment. VFIO and BSD are not supported. >> >> v4: >> - Add paramerter checking. >> - Add header file to determine if hotplug can be enabled. >> >> Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp> >> --- >> lib/librte_eal/common/Makefile | 1 + >> lib/librte_eal/common/include/rte_dev_hotplug.h | 44 +++++++++++++++++ >> lib/librte_eal/linuxapp/eal/eal_pci.c | 38 +++++++++++++++ >> lib/librte_eal/linuxapp/eal/eal_pci_init.h | 8 +++ >> lib/librte_eal/linuxapp/eal/eal_pci_uio.c | 65 +++++++++++++++++++++++++ >> 5 files changed, 156 insertions(+) >> create mode 100644 lib/librte_eal/common/include/rte_dev_hotplug.h >> >> diff --git a/lib/librte_eal/common/Makefile b/lib/librte_eal/common/Makefile >> index 52c1a5f..db7cc93 100644 >> --- a/lib/librte_eal/common/Makefile >> +++ b/lib/librte_eal/common/Makefile >> @@ -41,6 +41,7 @@ INC += rte_eal_memconfig.h rte_malloc_heap.h >> INC += rte_hexdump.h rte_devargs.h rte_dev.h >> INC += rte_common_vect.h >> INC += rte_pci_dev_feature_defs.h rte_pci_dev_features.h >> +INC += rte_dev_hotplug.h >> >> ifeq ($(CONFIG_RTE_INSECURE_FUNCTION_WARNING),y) >> INC += rte_warnings.h >> diff --git a/lib/librte_eal/common/include/rte_dev_hotplug.h b/lib/librte_eal/common/include/rte_dev_hotplug.h >> new file mode 100644 >> index 0000000..b333e0f >> --- /dev/null >> +++ b/lib/librte_eal/common/include/rte_dev_hotplug.h >> @@ -0,0 +1,44 @@ >> +/*- >> + * BSD LICENSE >> + * >> + * Copyright(c) 2015 IGEL Co.,LTd. >> + * All rights reserved. >> + * >> + * Redistribution and use in source and binary forms, with or without >> + * modification, are permitted provided that the following conditions >> + * are met: >> + * >> + * * Redistributions of source code must retain the above copyright >> + * notice, this list of conditions and the following disclaimer. >> + * * Redistributions in binary form must reproduce the above copyright >> + * notice, this list of conditions and the following disclaimer in >> + * the documentation and/or other materials provided with the >> + * distribution. >> + * * Neither the name of IGEL Co.,Ltd. nor the names of its >> + * contributors may be used to endorse or promote products derived >> + * from this software without specific prior written permission. >> + * >> + * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS >> + * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT >> + * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR >> + * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT >> + * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, >> + * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT >> + * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, >> + * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY >> + * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT >> + * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE >> + * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. >> + */ >> + >> +#ifndef _RTE_DEV_HOTPLUG_H_ >> +#define _RTE_DEV_HOTPLUG_H_ >> + >> +/* >> + * determine if hotplug can be enabled on the system >> + */ >> +#if defined(RTE_LIBRTE_EAL_HOTPLUG) && defined(RTE_LIBRTE_EAL_LINUXAPP) > As you said, VFIO should not work with it, so does it need to add the > vfio check here? I appreciate your comment. Yes, it should be. I will fix it in next version. Thanks, Tetsuya > Thanks, > Michael >> +#define ENABLE_HOTPLUG >> +#endif /* RTE_LIBRTE_EAL_HOTPLUG & RTE_LIBRTE_EAL_LINUXAPP */ >> + >> +#endif /* _RTE_DEV_HOTPLUG_H_ */ >> diff --git a/lib/librte_eal/linuxapp/eal/eal_pci.c b/lib/librte_eal/linuxapp/eal/eal_pci.c >> index 3d2d93c..52c464c 100644 >> --- a/lib/librte_eal/linuxapp/eal/eal_pci.c >> +++ b/lib/librte_eal/linuxapp/eal/eal_pci.c >> @@ -137,6 +137,25 @@ pci_map_resource(void *requested_addr, int fd, off_t offset, size_t size) >> return mapaddr; >> } >> >> +#ifdef ENABLE_HOTPLUG >> +/* unmap a particular resource */ >> +void >> +pci_unmap_resource(void *requested_addr, size_t size) >> +{ >> + if (requested_addr == NULL) >> + return; >> + >> + /* Unmap the PCI memory resource of device */ >> + if (munmap(requested_addr, size)) { >> + RTE_LOG(ERR, EAL, "%s(): cannot munmap(%p, 0x%lx): %s\n", >> + __func__, requested_addr, (unsigned long)size, >> + strerror(errno)); >> + } else >> + RTE_LOG(DEBUG, EAL, " PCI memory mapped at %p\n", >> + requested_addr); >> +} >> +#endif /* ENABLE_HOTPLUG */ >> + >> /* parse the "resource" sysfs file */ >> #define IORESOURCE_MEM 0x00000200 >> >> @@ -510,6 +529,25 @@ pci_map_device(struct rte_pci_device *dev) >> return 0; >> } >> >> +#ifdef ENABLE_HOTPLUG >> +static void >> +pci_unmap_device(struct rte_pci_device *dev) >> +{ >> + if (dev == NULL) >> + return; >> + >> + /* try unmapping the NIC resources using VFIO if it exists */ >> +#ifdef VFIO_PRESENT >> + if (pci_vfio_is_enabled()) { >> + RTE_LOG(ERR, EAL, "%s() doesn't support vfio yet.\n", >> + __func__); >> + return; >> + } >> +#endif >> + pci_uio_unmap_resource(dev); >> +} >> +#endif /* ENABLE_HOTPLUG */ >> + >> /* >> * If vendor/device ID match, call the devinit() function of the >> * driver. >> diff --git a/lib/librte_eal/linuxapp/eal/eal_pci_init.h b/lib/librte_eal/linuxapp/eal/eal_pci_init.h >> index 1070eb8..5152a0b 100644 >> --- a/lib/librte_eal/linuxapp/eal/eal_pci_init.h >> +++ b/lib/librte_eal/linuxapp/eal/eal_pci_init.h >> @@ -34,6 +34,7 @@ >> #ifndef EAL_PCI_INIT_H_ >> #define EAL_PCI_INIT_H_ >> >> +#include <rte_dev_hotplug.h> >> #include "eal_vfio.h" >> >> struct pci_map { >> @@ -71,6 +72,13 @@ void *pci_map_resource(void *requested_addr, int fd, off_t offset, >> /* map IGB_UIO resource prototype */ >> int pci_uio_map_resource(struct rte_pci_device *dev); >> >> +#ifdef ENABLE_HOTPLUG >> +void pci_unmap_resource(void *requested_addr, size_t size); >> + >> +/* unmap IGB_UIO resource prototype */ >> +void pci_uio_unmap_resource(struct rte_pci_device *dev); >> +#endif /* ENABLE_HOTPLUG */ >> + >> #ifdef VFIO_PRESENT >> >> #define VFIO_MAX_GROUPS 64 >> diff --git a/lib/librte_eal/linuxapp/eal/eal_pci_uio.c b/lib/librte_eal/linuxapp/eal/eal_pci_uio.c >> index 1da3507..81830d1 100644 >> --- a/lib/librte_eal/linuxapp/eal/eal_pci_uio.c >> +++ b/lib/librte_eal/linuxapp/eal/eal_pci_uio.c >> @@ -404,6 +404,71 @@ pci_uio_map_resource(struct rte_pci_device *dev) >> return 0; >> } >> >> +#ifdef ENABLE_HOTPLUG >> +static void >> +pci_uio_unmap(struct mapped_pci_resource *uio_res) >> +{ >> + int i; >> + >> + if (uio_res == NULL) >> + return; >> + >> + for (i = 0; i != uio_res->nb_maps; i++) >> + pci_unmap_resource(uio_res->maps[i].addr, >> + (size_t)uio_res->maps[i].size); >> +} >> + >> +static struct mapped_pci_resource * >> +pci_uio_find_resource(struct rte_pci_device *dev) >> +{ >> + struct mapped_pci_resource *uio_res; >> + >> + if (dev == NULL) >> + return NULL; >> + >> + TAILQ_FOREACH(uio_res, pci_res_list, next) { >> + >> + /* skip this element if it doesn't match our PCI address */ >> + if (!eal_compare_pci_addr(&uio_res->pci_addr, &dev->addr)) >> + return uio_res; >> + } >> + return NULL; >> +} >> + >> +/* unmap the PCI resource of a PCI device in virtual memory */ >> +void >> +pci_uio_unmap_resource(struct rte_pci_device *dev) >> +{ >> + struct mapped_pci_resource *uio_res; >> + >> + if (dev == NULL) >> + return; >> + >> + /* find an entry for the device */ >> + uio_res = pci_uio_find_resource(dev); >> + if (uio_res == NULL) >> + return; >> + >> + /* secondary processes - just free maps */ >> + if (rte_eal_process_type() != RTE_PROC_PRIMARY) >> + return pci_uio_unmap(uio_res); >> + >> + TAILQ_REMOVE(pci_res_list, uio_res, next); >> + >> + /* unmap all resources */ >> + pci_uio_unmap(uio_res); >> + >> + /* free uio resource */ >> + rte_free(uio_res); >> + >> + /* close fd if in primary process */ >> + close(dev->intr_handle.fd); >> + >> + dev->intr_handle.fd = -1; >> + dev->intr_handle.type = RTE_INTR_HANDLE_UNKNOWN; >> +} >> +#endif /* ENABLE_HOTPLUG */ >> + >> /* >> * parse a sysfs file containing one integer value >> * different to the eal version, as it needs to work with 64-bit values
On 1/19/2015 6:42 PM, Tetsuya Mukawa wrote: > The patch adds following functions. > > - rte_eth_dev_save() > The function is used for saving current rte_eth_dev structures. > - rte_eth_dev_get_changed_port() > The function receives the rte_eth_dev structures, then compare > these with current values to know which port is actually > attached or detached. > - rte_eth_dev_get_addr_by_port() > The function returns a pci address of a ethdev specified by port > identifier. > - rte_eth_dev_get_port_by_addr() > The function returns a port identifier of a ethdev specified by > pci address. > - rte_eth_dev_get_name_by_port() > The function returns a unique identifier name of a ethdev > specified by port identifier. > - Add rte_eth_dev_check_detachable() > The function returns whether a PMD supports detach function. > > Also the patch changes scope of rte_eth_dev_allocated() to global. > This function will be called by virtual PMDs to support port hotplug. > So change scope of the function to global. > > v4: > - Add paramerter checking. > v3: > - Fix if-condition bug while comparing pci addresses. > - Add error checking codes. > Reported-by: Mark Enright <menrigh@brocade.com> > > Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp> > --- > lib/librte_ether/rte_ethdev.c | 102 +++++++++++++++++++++++++++++++++++++++++- > lib/librte_ether/rte_ethdev.h | 80 +++++++++++++++++++++++++++++++++ > 2 files changed, 181 insertions(+), 1 deletion(-) > > diff --git a/lib/librte_ether/rte_ethdev.c b/lib/librte_ether/rte_ethdev.c > index e5145b7..e572ef4 100644 > --- a/lib/librte_ether/rte_ethdev.c > +++ b/lib/librte_ether/rte_ethdev.c > @@ -206,7 +206,7 @@ rte_eth_dev_data_alloc(void) > RTE_MAX_ETHPORTS * sizeof(*rte_eth_dev_data)); > } > > -static struct rte_eth_dev * > +struct rte_eth_dev * > rte_eth_dev_allocated(const char *name) > { > unsigned i; > @@ -422,6 +422,106 @@ rte_eth_dev_count(void) > return (nb_ports); > } > > +void > +rte_eth_dev_save(struct rte_eth_dev *devs) > +{ > + if (devs == NULL) > + return; > + > + /* save current rte_eth_devices */ > + memcpy(devs, rte_eth_devices, > + sizeof(struct rte_eth_dev) * RTE_MAX_ETHPORTS); > +} > + > +int > +rte_eth_dev_get_changed_port(struct rte_eth_dev *devs, uint8_t *port_id) > +{ > + if ((devs == NULL) || (port_id == NULL)) > + return -1; > + > + /* check which port was attached or detached */ > + for (*port_id = 0; *port_id < RTE_MAX_ETHPORTS; (*port_id)++, devs++) { > + if (rte_eth_devices[*port_id].attached ^ devs->attached) > + return 0; > + } > + return 1; > +} > + > +int > +rte_eth_dev_get_addr_by_port(uint8_t port_id, struct rte_pci_addr *addr) > +{ > + if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) { > + PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); > + return -EINVAL; > + } > + > + if (addr == NULL) { > + PMD_DEBUG_TRACE("Null pointer is specified\n"); > + return -EINVAL; > + } > + > + *addr = rte_eth_devices[port_id].pci_dev->addr; > + return 0; > +} > + > +int > +rte_eth_dev_get_port_by_addr(struct rte_pci_addr *addr, uint8_t *port_id) > +{ > + struct rte_pci_addr *tmp; > + > + if ((addr == NULL) || (port_id == NULL)) { > + PMD_DEBUG_TRACE("Null pointer is specified\n"); > + return -1; Is it better to replace "-1" to "-EINVAL" ? > + } > + > + for (*port_id = 0; *port_id < RTE_MAX_ETHPORTS; (*port_id)++) { > + if (!rte_eth_devices[*port_id].attached) > + continue; > + if (!rte_eth_devices[*port_id].pci_dev) > + continue; > + tmp = &rte_eth_devices[*port_id].pci_dev->addr; > + if (eal_compare_pci_addr(tmp, addr) == 0) > + return 0; > + } > + return -1; > +} > + > +int > +rte_eth_dev_get_name_by_port(uint8_t port_id, char *name) > +{ > + char *tmp; > + > + if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) { > + PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); > + return -EINVAL; > + } > + > + if (name == NULL) { > + PMD_DEBUG_TRACE("Null pointer is specified\n"); > + return -EINVAL; > + } > + > + /* shouldn't check 'rte_eth_devices[i].data', > + * because it might be overwritten by VDEV PMD */ > + tmp = rte_eth_dev_data[port_id].name; > + strncpy(name, tmp, strlen(tmp) + 1); > + return 0; > +} > + > +int > +rte_eth_dev_check_detachable(uint8_t port_id) > +{ > + uint32_t drv_flags; > + > + if (port_id >= RTE_MAX_ETHPORTS) { > + PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); > + return -EINVAL; > + } > + > + drv_flags = rte_eth_devices[port_id].driver->pci_drv.drv_flags; > + return !(drv_flags & RTE_PCI_DRV_DETACHABLE); > +} > + > static int > rte_eth_dev_rx_queue_config(struct rte_eth_dev *dev, uint16_t nb_queues) > { > diff --git a/lib/librte_ether/rte_ethdev.h b/lib/librte_ether/rte_ethdev.h > index 2a8ff26..c0b98dc 100644 > --- a/lib/librte_ether/rte_ethdev.h > +++ b/lib/librte_ether/rte_ethdev.h > @@ -1642,6 +1642,86 @@ extern struct rte_eth_dev rte_eth_devices[]; > extern uint8_t rte_eth_dev_count(void); > > /** > + * Function for internal use by port hotplug functions. > + * Copies current ethdev structures to the specified pointer. > + * > + * @param devs The pointer to the ethdev structures > + */ > +extern void rte_eth_dev_save(struct rte_eth_dev *devs); > + > +/** > + * Function for internal use by port hotplug functions. > + * Compare the specified ethdev structures with currents. Then > + * if there is a port which status is changed, fill the specified pointer > + * with the port id of that port. > + * @param devs The pointer to the ethdev structures > + * @param port_id The pointer to the port id > + * @return > + * - 0 on success, negative on error > + */ > +extern int rte_eth_dev_get_changed_port( > + struct rte_eth_dev *devs, uint8_t *port_id); > + > +/** > + * Function for internal use by port hotplug functions. > + * Returns a pci address of a ethdev specified by port identifier. > + * @param port_id > + * The port identifier of the Ethernet device > + * @param addr > + * The pointer to the pci address > + * @return > + * - 0 on success, negative on error > + */ > +extern int rte_eth_dev_get_addr_by_port( > + uint8_t port_id, struct rte_pci_addr *addr); > + > +/** > + * Function for internal use by port hotplug functions. > + * Returns a port identifier of a ethdev specified by pci address. > + * @param addr > + * The pointer to the pci address of the Ethernet device. > + * @param port_id > + * The pointer to the port identifier > + * @return > + * - 0 on success, negative on error > + */ > +extern int rte_eth_dev_get_port_by_addr( > + struct rte_pci_addr *addr, uint8_t *port_id); > + > +/** > + * Function for internal use by port hotplug functions. > + * Returns a unique identifier name of a ethdev specified by port identifier. > + * @param port_id > + * The port identifier. > + * @param name > + * The pointer to the Unique identifier name for each Ethernet device > + * @return > + * - 0 on success, negative on error > + */ > +extern int rte_eth_dev_get_name_by_port(uint8_t port_id, char *name); > + > +/** > + * Function for internal use by port hotplug functions. > + * Check whether or not, a PMD that is handling the ethdev specified by port > + * identifier can support detach function. > + * @param port_id > + * The port identifier > + * @return > + * - 0 on supporting detach function, negative on not supporting > + */ > +extern int rte_eth_dev_check_detachable(uint8_t port_id); > + > +/** > + * Function for internal use by port hotplug functions. > + * Returns a ethdev slot specified by the unique identifier name. > + * @param name > + * The pointer to the Unique identifier name for each Ethernet device > + * @return > + * - The pointer to the ethdev slot, on success. NULL on error > + */ > +extern struct rte_eth_dev *rte_eth_dev_allocated(const char *name); > + > +/** > * Function for internal use by dummy drivers primarily, e.g. ring-based > * driver. > * Allocates a new ethdev slot for an ethernet device and returns the pointer
On 1/19/2015 6:42 PM, Tetsuya Mukawa wrote: > The function removes the specified devargs entry from devargs_list. > Also the patch adds sanity checking to rte_eal_devargs_add(). > > v4: > - Fix sanity check code > > Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp> > --- > lib/librte_eal/common/eal_common_devargs.c | 57 +++++++++++++++++++++++++++++ > lib/librte_eal/common/include/rte_devargs.h | 18 +++++++++ > 2 files changed, 75 insertions(+) > > diff --git a/lib/librte_eal/common/eal_common_devargs.c b/lib/librte_eal/common/eal_common_devargs.c > index 4c7d11a..a360a85 100644 > --- a/lib/librte_eal/common/eal_common_devargs.c > +++ b/lib/librte_eal/common/eal_common_devargs.c > @@ -44,6 +44,35 @@ > struct rte_devargs_list devargs_list = > TAILQ_HEAD_INITIALIZER(devargs_list); > > + > +/* find a entry specified by pci address or device name */ > +static struct rte_devargs * > +rte_eal_devargs_find(enum rte_devtype devtype, void *args) > +{ > + struct rte_devargs *devargs; > + > + if (args == NULL) > + return NULL; > + > + TAILQ_FOREACH(devargs, &devargs_list, next) { > + switch (devtype) { > + case RTE_DEVTYPE_WHITELISTED_PCI: > + case RTE_DEVTYPE_BLACKLISTED_PCI: > + if (eal_compare_pci_addr(&devargs->pci.addr, args) == 0) > + goto found; > + break; > + case RTE_DEVTYPE_VIRTUAL: > + if (memcmp(&devargs->virtual.drv_name, args, > + strlen((char *)args)) == 0) > + goto found; > + break; > + } > + } > + return NULL; > +found: > + return devargs; > +} > + > /* store a whitelist parameter for later parsing */ > int > rte_eal_devargs_add(enum rte_devtype devtype, const char *devargs_str) > @@ -87,6 +116,12 @@ rte_eal_devargs_add(enum rte_devtype devtype, const char *devargs_str) > free(devargs); > return -1; > } > + /* make sure there is no same entry */ > + if (rte_eal_devargs_find(devtype, &devargs->pci.addr)) { > + RTE_LOG(ERR, EAL, > + "device already registered: <%s>\n", buf); > + return -1; > + } > break; > case RTE_DEVTYPE_VIRTUAL: > /* save driver name */ > @@ -98,6 +133,12 @@ rte_eal_devargs_add(enum rte_devtype devtype, const char *devargs_str) > free(devargs); > return -1; > } > + /* make sure there is no same entry */ > + if (rte_eal_devargs_find(devtype, &devargs->virtual.drv_name)) { > + RTE_LOG(ERR, EAL, > + "device already registered: <%s>\n", buf); > + return -1; > + } > break; > } > > @@ -105,6 +146,22 @@ rte_eal_devargs_add(enum rte_devtype devtype, const char *devargs_str) > return 0; > } > > +/* remove it from the devargs_list */ > +void > +rte_eal_devargs_remove(enum rte_devtype devtype, void *args) > +{ > + struct rte_devargs *devargs; > + > + if (args == NULL) > + return; > + > + devargs = rte_eal_devargs_find(devtype, args); > + if (devargs == NULL) If devargs == NULL, means not found, does it reasonable to ignore? Some error happens I think, at least you should print out some logs. Thanks, Michael > + return; > + > + TAILQ_REMOVE(&devargs_list, devargs, next); > +} > + > /* count the number of devices of a specified type */ > unsigned int > rte_eal_devargs_type_count(enum rte_devtype devtype) > diff --git a/lib/librte_eal/common/include/rte_devargs.h b/lib/librte_eal/common/include/rte_devargs.h > index 9f9c98f..1066efd 100644 > --- a/lib/librte_eal/common/include/rte_devargs.h > +++ b/lib/librte_eal/common/include/rte_devargs.h > @@ -123,6 +123,24 @@ extern struct rte_devargs_list devargs_list; > int rte_eal_devargs_add(enum rte_devtype devtype, const char *devargs_str); > > /** > + * Remove a device from the user device list > + * > + * For PCI devices, the format of arguments string is "PCI_ADDR". It shouldn't > + * involves parameters for the device. Example: "08:00.1". > + * > + * For virtual devices, the format of arguments string is "DRIVER_NAME*". It > + * shouldn't involves parameters for the device. Example: "eth_ring". The > + * validity of the driver name is not checked by this function, it is done > + * when closing the drivers. > + * > + * @param devtype > + * The type of the device. > + * @param name > + * The name of the device. > + */ > +void rte_eal_devargs_remove(enum rte_devtype devtype, void *args); > + > +/** > * Count the number of user devices of a specified type > * > * @param devtype
On 1/19/2015 6:43 PM, Tetsuya Mukawa wrote: > These functions are used for attaching or detaching a port. > When rte_eal_dev_attach() is called, the function tries to realize the > device name as pci address. If this is done successfully, > rte_eal_dev_attach() will attach physical device port. If not, attaches > virtual devive port. > When rte_eal_dev_detach() is called, the function gets the device type > of this port to know whether the port is came from physical or virtual. > And then specific detaching function will be called. > > v4: > - Fix comment. > - Add error checking. > - Fix indent of 'if' statement. > - Change function name. > > Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp> > --- > lib/librte_eal/common/eal_common_dev.c | 273 ++++++++++++++++++++++++++++++++ > lib/librte_eal/common/eal_private.h | 11 ++ > lib/librte_eal/common/include/rte_dev.h | 33 ++++ > lib/librte_eal/linuxapp/eal/Makefile | 1 + > lib/librte_eal/linuxapp/eal/eal_pci.c | 6 +- > 5 files changed, 321 insertions(+), 3 deletions(-) > > diff --git a/lib/librte_eal/common/eal_common_dev.c b/lib/librte_eal/common/eal_common_dev.c > index eae5656..828bd70 100644 > --- a/lib/librte_eal/common/eal_common_dev.c > +++ b/lib/librte_eal/common/eal_common_dev.c > @@ -32,10 +32,13 @@ > * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. > */ > > +#include <stdio.h> > +#include <limits.h> > #include <string.h> > #include <inttypes.h> > #include <sys/queue.h> > > +#include <rte_ethdev.h> > #include <rte_dev.h> > #include <rte_devargs.h> > #include <rte_debug.h> > @@ -107,3 +110,273 @@ rte_eal_dev_init(void) > } > return 0; > } > + > +/* So far, DPDK hotplug function only supports linux */ > +#ifdef ENABLE_HOTPLUG > +static void > +rte_eal_dev_invoke(struct rte_driver *driver, > + struct rte_devargs *devargs, enum rte_eal_invoke_type type) > +{ > + if ((driver == NULL) || (devargs == NULL)) > + return; > + > + switch (type) { > + case RTE_EAL_INVOKE_TYPE_PROBE: > + driver->init(devargs->virtual.drv_name, devargs->args); > + break; > + case RTE_EAL_INVOKE_TYPE_CLOSE: > + driver->uninit(devargs->virtual.drv_name, devargs->args); > + break; > + default: > + break; > + } > +} > + > +static int > +rte_eal_dev_find_and_invoke(const char *name, int type) This function is totally for vdev, so I would like it shows in name, like *rte_eal_vdev_find_and_invoke* > +{ > + struct rte_devargs *devargs; > + struct rte_driver *driver; > + > + if (name == NULL) > + return -EINVAL; > + > + /* call the init function for each virtual device */ > + TAILQ_FOREACH(devargs, &devargs_list, next) { > + > + if (devargs->type != RTE_DEVTYPE_VIRTUAL) > + continue; > + > + if (strncmp(name, devargs->virtual.drv_name, strlen(name))) > + continue; > + > + TAILQ_FOREACH(driver, &dev_driver_list, next) { > + if (driver->type != PMD_VDEV) > + continue; > + > + /* search a driver prefix in virtual device name */ > + if (!strncmp(driver->name, devargs->virtual.drv_name, > + strlen(driver->name))) { > + rte_eal_dev_invoke(driver, devargs, type); > + break; > + } > + } > + > + if (driver == NULL) { > + RTE_LOG(WARNING, EAL, "no driver found for %s\n", > + devargs->virtual.drv_name); > + } > + return 0; > + } > + return 1; > +} > + > +/* attach the new physical device, then store port_id of the device */ > +static int > +rte_eal_dev_attach_pdev(struct rte_pci_addr *addr, uint8_t *port_id) > +{ > + uint8_t new_port_id; > + struct rte_eth_dev devs[RTE_MAX_ETHPORTS]; > + > + if ((addr == NULL) || (port_id == NULL)) > + goto err; > + > + /* save current port status */ > + rte_eth_dev_save(devs); > + /* re-construct pci_device_list */ > + if (rte_eal_pci_scan()) > + goto err; > + /* invoke probe func of the driver can handle the new device */ > + if (rte_eal_pci_probe_one(addr)) > + goto err; > + /* get port_id enabled by above procedures */ > + if (rte_eth_dev_get_changed_port(devs, &new_port_id)) > + goto err; > + > + *port_id = new_port_id; > + return 0; > +err: > + RTE_LOG(ERR, EAL, "Drver, cannot attach the device\n"); > + return -1; > +} > + > +/* detach the new physical device, then store pci_addr of the device */ > +static int > +rte_eal_dev_detach_pdev(uint8_t port_id, struct rte_pci_addr *addr) > +{ > + struct rte_pci_addr freed_addr; > + struct rte_pci_addr vp; > + > + if (addr == NULL) > + goto err; > + > + /* check whether the driver supports detach feature, or not */ > + if (rte_eth_dev_check_detachable(port_id)) > + goto err; > + > + /* get pci address by port id */ > + if (rte_eth_dev_get_addr_by_port(port_id, &freed_addr)) > + goto err; > + > + /* Zerod pci addr means the port comes from virtual device */ > + vp.domain = vp.bus = vp.devid = vp.function = 0; > + if (eal_compare_pci_addr(&vp, &freed_addr) == 0) > + goto err; > + > + /* invoke close func of the driver, > + * also remove the device from pci_device_list */ > + if (rte_eal_pci_close_one(&freed_addr)) > + goto err; > + > + *addr = freed_addr; > + return 0; > +err: > + RTE_LOG(ERR, EAL, "Drver, cannot detach the device\n"); > + return -1; > +} > + > +static void > +get_vdev_name(char *vdevargs) > +{ > + char *sep; > + > + if (vdevargs == NULL) > + return; > + > + /* set the first ',' to '\0' to split name and arguments */ > + sep = strchr(vdevargs, ','); > + if (sep != NULL) > + sep[0] = '\0'; > +} > + > +/* attach the new virtual device, then store port_id of the device */ > +static int > +rte_eal_dev_attach_vdev(const char *vdevargs, uint8_t *port_id) > +{ > + char *args; > + uint8_t new_port_id; > + struct rte_eth_dev devs[RTE_MAX_ETHPORTS]; > + > + if ((vdevargs == NULL) || (port_id == NULL)) > + goto err0; > + > + args = strdup(vdevargs); > + if (args == NULL) > + goto err0; > + > + /* save current port status */ > + rte_eth_dev_save(devs); > + /* add the vdevargs to devargs_list */ > + if (rte_eal_devargs_add(RTE_DEVTYPE_VIRTUAL, args)) > + goto err1; > + /* parse vdevargs, then retrieve device name */ > + get_vdev_name(args); > + /* walk around dev_driver_list to find the driver of the device, > + * then invoke probe function o the driver */ > + if (rte_eal_dev_find_and_invoke(args, RTE_EAL_INVOKE_TYPE_PROBE)) > + goto err2; > + /* get port_id enabled by above procedures */ > + if (rte_eth_dev_get_changed_port(devs, &new_port_id)) > + goto err2; > + > + free(args); > + *port_id = new_port_id; > + return 0; > +err2: > + rte_eal_devargs_remove(RTE_DEVTYPE_VIRTUAL, args); > +err1: > + free(args); > +err0: > + RTE_LOG(ERR, EAL, "Drver, cannot detach the device\n"); > + return -1; > +} > + > +/* detach the new virtual device, then store the name of the device */ > +static int > +rte_eal_dev_detach_vdev(uint8_t port_id, char *vdevname) > +{ > + char name[RTE_ETH_NAME_MAX_LEN]; > + > + if (vdevname == NULL) > + goto err; > + > + /* check whether the driver supports detach feature, or not */ > + if (rte_eth_dev_check_detachable(port_id)) > + goto err; > + > + /* get device name by port id */ > + if (rte_eth_dev_get_name_by_port(port_id, name)) > + goto err; > + /* walk around dev_driver_list to find the driver of the device, > + * then invoke close function o the driver */ > + if (rte_eal_dev_find_and_invoke(name, RTE_EAL_INVOKE_TYPE_CLOSE)) > + goto err; > + /* remove the vdevname from devargs_list */ > + rte_eal_devargs_remove(RTE_DEVTYPE_VIRTUAL, name); > + > + strncpy(vdevname, name, sizeof(name)); > + return 0; > +err: > + RTE_LOG(ERR, EAL, "Drver, cannot detach the device\n"); > + return -1; > +} > + > +/* attach the new device, then store port_id of the device */ > +int > +rte_eal_dev_attach(const char *devargs, uint8_t *port_id) > +{ > + struct rte_pci_addr addr; > + > + if ((devargs == NULL) || (port_id == NULL)) > + return -EINVAL; > + > + if (eal_parse_pci_DomBDF(devargs, &addr) == 0) > + return rte_eal_dev_attach_pdev(&addr, port_id); > + else > + return rte_eal_dev_attach_vdev(devargs, port_id); > +} > + > +/* detach the device, then store the name of the device */ > +int > +rte_eal_dev_detach(uint8_t port_id, char *name) > +{ > + struct rte_pci_addr addr; > + int ret; > + > + if (name == NULL) > + return -EINVAL; > + > + if (rte_eth_dev_get_device_type(port_id) == RTE_ETH_DEV_PHYSICAL) { > + ret = rte_eth_dev_get_addr_by_port(port_id, &addr); > + if (ret < 0) > + return ret; > + > + ret = rte_eal_dev_detach_pdev(port_id, &addr); > + if (ret == 0) > + snprintf(name, RTE_ETH_NAME_MAX_LEN, > + "%04x.%02x.%02x.%d", > + addr.domain, addr.bus, > + addr.devid, addr.function); > + > + return ret; > + } else > + return rte_eal_dev_detach_vdev(port_id, name); > +} > +#else /* ENABLE_HOTPLUG */ > +int > +rte_eal_dev_attach(const char *devargs __rte_unused, > + uint8_t *port_id __rte_unused) > +{ > + RTE_LOG(ERR, EAL, "Hotplug support isn't enabled\n"); > + return -1; > +} > + > +/* detach the device, then store the name of the device */ > +int > +rte_eal_dev_detach(uint8_t port_id __rte_unused, > + char *name __rte_unused) > +{ > + RTE_LOG(ERR, EAL, "Hotplug support isn't enabled\n"); > + return -1; > +} > +#endif /* ENABLE_HOTPLUG */ > diff --git a/lib/librte_eal/common/eal_private.h b/lib/librte_eal/common/eal_private.h > index a97c5d8..453a1eb 100644 > --- a/lib/librte_eal/common/eal_private.h > +++ b/lib/librte_eal/common/eal_private.h > @@ -164,6 +164,17 @@ enum rte_eal_invoke_type { > }; > > /** > + * Scan the content of the PCI bus, and the devices in the devices > + * list > + * > + * This function is private to EAL. > + * > + * @return > + * 0 on success, negative on error > + */ > +int rte_eal_pci_scan(void); > + > +/** > * Mmap memory for single PCI device > * > * This function is private to EAL. > diff --git a/lib/librte_eal/common/include/rte_dev.h b/lib/librte_eal/common/include/rte_dev.h > index f7e3a10..e63dd1c 100644 > --- a/lib/librte_eal/common/include/rte_dev.h > +++ b/lib/librte_eal/common/include/rte_dev.h > @@ -47,6 +47,7 @@ extern "C" { > #endif > > #include <sys/queue.h> > +#include <rte_pci.h> > > /** Double linked list of device drivers. */ > TAILQ_HEAD(rte_driver_list, rte_driver); > @@ -57,6 +58,11 @@ TAILQ_HEAD(rte_driver_list, rte_driver); > typedef int (rte_dev_init_t)(const char *name, const char *args); > > /** > + * Uninitilization function called for each device driver once. > + */ > +typedef int (rte_dev_uninit_t)(const char *name, const char *args); > + > +/** > * Driver type enumeration > */ > enum pmd_type { > @@ -72,6 +78,7 @@ struct rte_driver { > enum pmd_type type; /**< PMD Driver type */ > const char *name; /**< Driver name. */ > rte_dev_init_t *init; /**< Device init. function. */ > + rte_dev_uninit_t *uninit; /**< Device uninit. function. */ > }; > > /** > @@ -93,6 +100,32 @@ void rte_eal_driver_register(struct rte_driver *driver); > void rte_eal_driver_unregister(struct rte_driver *driver); > > /** > + * Attach a new device. > + * > + * @param devargs > + * A pointer to a strings array describing the new device > + * to be attached. The strings should be a pci address like > + * '0000:01:00.0' or virtual device name like 'eth_pcap0'. > + * @param port_id > + * A pointer to a port identifier actually attached. > + * @return > + * 0 on success and port_id is filled, negative on error > + */ > +int rte_eal_dev_attach(const char *devargs, uint8_t *port_id); > + > +/** > + * Detach a device. > + * > + * @param port_id > + * The port identifier of the device to detach. > + * @param addr > + * A pointer to a device name actually detached. > + * @return > + * 0 on success and devname is filled, negative on error > + */ > +int rte_eal_dev_detach(uint8_t port_id, char *devname); > + > +/** > * Initalize all the registered drivers in this process > */ > int rte_eal_dev_init(void); > diff --git a/lib/librte_eal/linuxapp/eal/Makefile b/lib/librte_eal/linuxapp/eal/Makefile > index 72ecf3a..0ec83b5 100644 > --- a/lib/librte_eal/linuxapp/eal/Makefile > +++ b/lib/librte_eal/linuxapp/eal/Makefile > @@ -41,6 +41,7 @@ CFLAGS += -I$(RTE_SDK)/lib/librte_eal/common/include > CFLAGS += -I$(RTE_SDK)/lib/librte_ring > CFLAGS += -I$(RTE_SDK)/lib/librte_mempool > CFLAGS += -I$(RTE_SDK)/lib/librte_malloc > +CFLAGS += -I$(RTE_SDK)/lib/librte_mbuf > CFLAGS += -I$(RTE_SDK)/lib/librte_ether > CFLAGS += -I$(RTE_SDK)/lib/librte_ivshmem > CFLAGS += -I$(RTE_SDK)/lib/librte_pmd_ring > diff --git a/lib/librte_eal/linuxapp/eal/eal_pci.c b/lib/librte_eal/linuxapp/eal/eal_pci.c > index a23cc59..8e7e650 100644 > --- a/lib/librte_eal/linuxapp/eal/eal_pci.c > +++ b/lib/librte_eal/linuxapp/eal/eal_pci.c > @@ -378,8 +378,8 @@ error: > * Scan the content of the PCI bus, and the devices in the devices > * list > */ > -static int > -pci_scan(void) > +int > +rte_eal_pci_scan(void) > { > struct dirent *e; > DIR *dir; > @@ -701,7 +701,7 @@ rte_eal_pci_init(void) > if (internal_config.no_pci) > return 0; > > - if (pci_scan() < 0) { > + if (rte_eal_pci_scan() < 0) { > RTE_LOG(ERR, EAL, "%s(): Cannot scan PCI bus\n", __func__); > return -1; > }
Hi Michael, On 2015/01/21 11:55, Qiu, Michael wrote: > On 1/19/2015 6:42 PM, Tetsuya Mukawa wrote: >> The function removes the specified devargs entry from devargs_list. >> Also the patch adds sanity checking to rte_eal_devargs_add(). >> >> v4: >> - Fix sanity check code >> >> Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp> >> --- >> lib/librte_eal/common/eal_common_devargs.c | 57 +++++++++++++++++++++++++++++ >> lib/librte_eal/common/include/rte_devargs.h | 18 +++++++++ >> 2 files changed, 75 insertions(+) >> >> diff --git a/lib/librte_eal/common/eal_common_devargs.c b/lib/librte_eal/common/eal_common_devargs.c >> index 4c7d11a..a360a85 100644 >> --- a/lib/librte_eal/common/eal_common_devargs.c >> +++ b/lib/librte_eal/common/eal_common_devargs.c >> @@ -44,6 +44,35 @@ >> struct rte_devargs_list devargs_list = >> TAILQ_HEAD_INITIALIZER(devargs_list); >> >> + >> +/* find a entry specified by pci address or device name */ >> +static struct rte_devargs * >> +rte_eal_devargs_find(enum rte_devtype devtype, void *args) >> +{ >> + struct rte_devargs *devargs; >> + >> + if (args == NULL) >> + return NULL; >> + >> + TAILQ_FOREACH(devargs, &devargs_list, next) { >> + switch (devtype) { >> + case RTE_DEVTYPE_WHITELISTED_PCI: >> + case RTE_DEVTYPE_BLACKLISTED_PCI: >> + if (eal_compare_pci_addr(&devargs->pci.addr, args) == 0) >> + goto found; >> + break; >> + case RTE_DEVTYPE_VIRTUAL: >> + if (memcmp(&devargs->virtual.drv_name, args, >> + strlen((char *)args)) == 0) >> + goto found; >> + break; >> + } >> + } >> + return NULL; >> +found: >> + return devargs; >> +} >> + >> /* store a whitelist parameter for later parsing */ >> int >> rte_eal_devargs_add(enum rte_devtype devtype, const char *devargs_str) >> @@ -87,6 +116,12 @@ rte_eal_devargs_add(enum rte_devtype devtype, const char *devargs_str) >> free(devargs); >> return -1; >> } >> + /* make sure there is no same entry */ >> + if (rte_eal_devargs_find(devtype, &devargs->pci.addr)) { >> + RTE_LOG(ERR, EAL, >> + "device already registered: <%s>\n", buf); >> + return -1; >> + } >> break; >> case RTE_DEVTYPE_VIRTUAL: >> /* save driver name */ >> @@ -98,6 +133,12 @@ rte_eal_devargs_add(enum rte_devtype devtype, const char *devargs_str) >> free(devargs); >> return -1; >> } >> + /* make sure there is no same entry */ >> + if (rte_eal_devargs_find(devtype, &devargs->virtual.drv_name)) { >> + RTE_LOG(ERR, EAL, >> + "device already registered: <%s>\n", buf); >> + return -1; >> + } >> break; >> } >> >> @@ -105,6 +146,22 @@ rte_eal_devargs_add(enum rte_devtype devtype, const char *devargs_str) >> return 0; >> } >> >> +/* remove it from the devargs_list */ >> +void >> +rte_eal_devargs_remove(enum rte_devtype devtype, void *args) >> +{ >> + struct rte_devargs *devargs; >> + >> + if (args == NULL) >> + return; >> + >> + devargs = rte_eal_devargs_find(devtype, args); >> + if (devargs == NULL) > If devargs == NULL, means not found, does it reasonable to ignore? Some > error happens I think, at least you should print out some logs. I appreciate your comment. I agree with you. At least error message should be displayed. I will change function definition to return a error code and fix caller of the function. Also I will add error message. Thanks, Tetsuya > Thanks, > Michael >> + return; >> + >> + TAILQ_REMOVE(&devargs_list, devargs, next); >> +} >> + >> /* count the number of devices of a specified type */ >> unsigned int >> rte_eal_devargs_type_count(enum rte_devtype devtype) >> diff --git a/lib/librte_eal/common/include/rte_devargs.h b/lib/librte_eal/common/include/rte_devargs.h >> index 9f9c98f..1066efd 100644 >> --- a/lib/librte_eal/common/include/rte_devargs.h >> +++ b/lib/librte_eal/common/include/rte_devargs.h >> @@ -123,6 +123,24 @@ extern struct rte_devargs_list devargs_list; >> int rte_eal_devargs_add(enum rte_devtype devtype, const char *devargs_str); >> >> /** >> + * Remove a device from the user device list >> + * >> + * For PCI devices, the format of arguments string is "PCI_ADDR". It shouldn't >> + * involves parameters for the device. Example: "08:00.1". >> + * >> + * For virtual devices, the format of arguments string is "DRIVER_NAME*". It >> + * shouldn't involves parameters for the device. Example: "eth_ring". The >> + * validity of the driver name is not checked by this function, it is done >> + * when closing the drivers. >> + * >> + * @param devtype >> + * The type of the device. >> + * @param name >> + * The name of the device. >> + */ >> +void rte_eal_devargs_remove(enum rte_devtype devtype, void *args); >> + >> +/** >> * Count the number of user devices of a specified type >> * >> * @param devtype
Hi Michael, On 2015/01/21 12:49, Qiu, Michael wrote: > On 1/19/2015 6:43 PM, Tetsuya Mukawa wrote: >> These functions are used for attaching or detaching a port. >> When rte_eal_dev_attach() is called, the function tries to realize the >> device name as pci address. If this is done successfully, >> rte_eal_dev_attach() will attach physical device port. If not, attaches >> virtual devive port. >> When rte_eal_dev_detach() is called, the function gets the device type >> of this port to know whether the port is came from physical or virtual. >> And then specific detaching function will be called. >> >> v4: >> - Fix comment. >> - Add error checking. >> - Fix indent of 'if' statement. >> - Change function name. >> >> Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp> >> --- >> lib/librte_eal/common/eal_common_dev.c | 273 ++++++++++++++++++++++++++++++++ >> lib/librte_eal/common/eal_private.h | 11 ++ >> lib/librte_eal/common/include/rte_dev.h | 33 ++++ >> lib/librte_eal/linuxapp/eal/Makefile | 1 + >> lib/librte_eal/linuxapp/eal/eal_pci.c | 6 +- >> 5 files changed, 321 insertions(+), 3 deletions(-) >> >> diff --git a/lib/librte_eal/common/eal_common_dev.c b/lib/librte_eal/common/eal_common_dev.c >> index eae5656..828bd70 100644 >> --- a/lib/librte_eal/common/eal_common_dev.c >> +++ b/lib/librte_eal/common/eal_common_dev.c >> @@ -32,10 +32,13 @@ >> * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. >> */ >> >> +#include <stdio.h> >> +#include <limits.h> >> #include <string.h> >> #include <inttypes.h> >> #include <sys/queue.h> >> >> +#include <rte_ethdev.h> >> #include <rte_dev.h> >> #include <rte_devargs.h> >> #include <rte_debug.h> >> @@ -107,3 +110,273 @@ rte_eal_dev_init(void) >> } >> return 0; >> } >> + >> +/* So far, DPDK hotplug function only supports linux */ >> +#ifdef ENABLE_HOTPLUG >> +static void >> +rte_eal_dev_invoke(struct rte_driver *driver, >> + struct rte_devargs *devargs, enum rte_eal_invoke_type type) >> +{ >> + if ((driver == NULL) || (devargs == NULL)) >> + return; >> + >> + switch (type) { >> + case RTE_EAL_INVOKE_TYPE_PROBE: >> + driver->init(devargs->virtual.drv_name, devargs->args); >> + break; >> + case RTE_EAL_INVOKE_TYPE_CLOSE: >> + driver->uninit(devargs->virtual.drv_name, devargs->args); >> + break; >> + default: >> + break; >> + } >> +} >> + >> +static int >> +rte_eal_dev_find_and_invoke(const char *name, int type) > This function is totally for vdev, so I would like it shows in name, > like *rte_eal_vdev_find_and_invoke* Sure, I will change like above. I appreciate your suggestion. Thanks, Tetsuya >> +{ >> + struct rte_devargs *devargs; >> + struct rte_driver *driver; >> + >> + if (name == NULL) >> + return -EINVAL; >> + >> + /* call the init function for each virtual device */ >> + TAILQ_FOREACH(devargs, &devargs_list, next) { >> + >> + if (devargs->type != RTE_DEVTYPE_VIRTUAL) >> + continue; >> + >> + if (strncmp(name, devargs->virtual.drv_name, strlen(name))) >> + continue; >> + >> + TAILQ_FOREACH(driver, &dev_driver_list, next) { >> + if (driver->type != PMD_VDEV) >> + continue; >> + >> + /* search a driver prefix in virtual device name */ >> + if (!strncmp(driver->name, devargs->virtual.drv_name, >> + strlen(driver->name))) { >> + rte_eal_dev_invoke(driver, devargs, type); >> + break; >> + } >> + } >> + >> + if (driver == NULL) { >> + RTE_LOG(WARNING, EAL, "no driver found for %s\n", >> + devargs->virtual.drv_name); >> + } >> + return 0; >> + } >> + return 1; >> +} >> + >> +/* attach the new physical device, then store port_id of the device */ >> +static int >> +rte_eal_dev_attach_pdev(struct rte_pci_addr *addr, uint8_t *port_id) >> +{ >> + uint8_t new_port_id; >> + struct rte_eth_dev devs[RTE_MAX_ETHPORTS]; >> + >> + if ((addr == NULL) || (port_id == NULL)) >> + goto err; >> + >> + /* save current port status */ >> + rte_eth_dev_save(devs); >> + /* re-construct pci_device_list */ >> + if (rte_eal_pci_scan()) >> + goto err; >> + /* invoke probe func of the driver can handle the new device */ >> + if (rte_eal_pci_probe_one(addr)) >> + goto err; >> + /* get port_id enabled by above procedures */ >> + if (rte_eth_dev_get_changed_port(devs, &new_port_id)) >> + goto err; >> + >> + *port_id = new_port_id; >> + return 0; >> +err: >> + RTE_LOG(ERR, EAL, "Drver, cannot attach the device\n"); >> + return -1; >> +} >> + >> +/* detach the new physical device, then store pci_addr of the device */ >> +static int >> +rte_eal_dev_detach_pdev(uint8_t port_id, struct rte_pci_addr *addr) >> +{ >> + struct rte_pci_addr freed_addr; >> + struct rte_pci_addr vp; >> + >> + if (addr == NULL) >> + goto err; >> + >> + /* check whether the driver supports detach feature, or not */ >> + if (rte_eth_dev_check_detachable(port_id)) >> + goto err; >> + >> + /* get pci address by port id */ >> + if (rte_eth_dev_get_addr_by_port(port_id, &freed_addr)) >> + goto err; >> + >> + /* Zerod pci addr means the port comes from virtual device */ >> + vp.domain = vp.bus = vp.devid = vp.function = 0; >> + if (eal_compare_pci_addr(&vp, &freed_addr) == 0) >> + goto err; >> + >> + /* invoke close func of the driver, >> + * also remove the device from pci_device_list */ >> + if (rte_eal_pci_close_one(&freed_addr)) >> + goto err; >> + >> + *addr = freed_addr; >> + return 0; >> +err: >> + RTE_LOG(ERR, EAL, "Drver, cannot detach the device\n"); >> + return -1; >> +} >> + >> +static void >> +get_vdev_name(char *vdevargs) >> +{ >> + char *sep; >> + >> + if (vdevargs == NULL) >> + return; >> + >> + /* set the first ',' to '\0' to split name and arguments */ >> + sep = strchr(vdevargs, ','); >> + if (sep != NULL) >> + sep[0] = '\0'; >> +} >> + >> +/* attach the new virtual device, then store port_id of the device */ >> +static int >> +rte_eal_dev_attach_vdev(const char *vdevargs, uint8_t *port_id) >> +{ >> + char *args; >> + uint8_t new_port_id; >> + struct rte_eth_dev devs[RTE_MAX_ETHPORTS]; >> + >> + if ((vdevargs == NULL) || (port_id == NULL)) >> + goto err0; >> + >> + args = strdup(vdevargs); >> + if (args == NULL) >> + goto err0; >> + >> + /* save current port status */ >> + rte_eth_dev_save(devs); >> + /* add the vdevargs to devargs_list */ >> + if (rte_eal_devargs_add(RTE_DEVTYPE_VIRTUAL, args)) >> + goto err1; >> + /* parse vdevargs, then retrieve device name */ >> + get_vdev_name(args); >> + /* walk around dev_driver_list to find the driver of the device, >> + * then invoke probe function o the driver */ >> + if (rte_eal_dev_find_and_invoke(args, RTE_EAL_INVOKE_TYPE_PROBE)) >> + goto err2; >> + /* get port_id enabled by above procedures */ >> + if (rte_eth_dev_get_changed_port(devs, &new_port_id)) >> + goto err2; >> + >> + free(args); >> + *port_id = new_port_id; >> + return 0; >> +err2: >> + rte_eal_devargs_remove(RTE_DEVTYPE_VIRTUAL, args); >> +err1: >> + free(args); >> +err0: >> + RTE_LOG(ERR, EAL, "Drver, cannot detach the device\n"); >> + return -1; >> +} >> + >> +/* detach the new virtual device, then store the name of the device */ >> +static int >> +rte_eal_dev_detach_vdev(uint8_t port_id, char *vdevname) >> +{ >> + char name[RTE_ETH_NAME_MAX_LEN]; >> + >> + if (vdevname == NULL) >> + goto err; >> + >> + /* check whether the driver supports detach feature, or not */ >> + if (rte_eth_dev_check_detachable(port_id)) >> + goto err; >> + >> + /* get device name by port id */ >> + if (rte_eth_dev_get_name_by_port(port_id, name)) >> + goto err; >> + /* walk around dev_driver_list to find the driver of the device, >> + * then invoke close function o the driver */ >> + if (rte_eal_dev_find_and_invoke(name, RTE_EAL_INVOKE_TYPE_CLOSE)) >> + goto err; >> + /* remove the vdevname from devargs_list */ >> + rte_eal_devargs_remove(RTE_DEVTYPE_VIRTUAL, name); >> + >> + strncpy(vdevname, name, sizeof(name)); >> + return 0; >> +err: >> + RTE_LOG(ERR, EAL, "Drver, cannot detach the device\n"); >> + return -1; >> +} >> + >> +/* attach the new device, then store port_id of the device */ >> +int >> +rte_eal_dev_attach(const char *devargs, uint8_t *port_id) >> +{ >> + struct rte_pci_addr addr; >> + >> + if ((devargs == NULL) || (port_id == NULL)) >> + return -EINVAL; >> + >> + if (eal_parse_pci_DomBDF(devargs, &addr) == 0) >> + return rte_eal_dev_attach_pdev(&addr, port_id); >> + else >> + return rte_eal_dev_attach_vdev(devargs, port_id); >> +} >> + >> +/* detach the device, then store the name of the device */ >> +int >> +rte_eal_dev_detach(uint8_t port_id, char *name) >> +{ >> + struct rte_pci_addr addr; >> + int ret; >> + >> + if (name == NULL) >> + return -EINVAL; >> + >> + if (rte_eth_dev_get_device_type(port_id) == RTE_ETH_DEV_PHYSICAL) { >> + ret = rte_eth_dev_get_addr_by_port(port_id, &addr); >> + if (ret < 0) >> + return ret; >> + >> + ret = rte_eal_dev_detach_pdev(port_id, &addr); >> + if (ret == 0) >> + snprintf(name, RTE_ETH_NAME_MAX_LEN, >> + "%04x.%02x.%02x.%d", >> + addr.domain, addr.bus, >> + addr.devid, addr.function); >> + >> + return ret; >> + } else >> + return rte_eal_dev_detach_vdev(port_id, name); >> +} >> +#else /* ENABLE_HOTPLUG */ >> +int >> +rte_eal_dev_attach(const char *devargs __rte_unused, >> + uint8_t *port_id __rte_unused) >> +{ >> + RTE_LOG(ERR, EAL, "Hotplug support isn't enabled\n"); >> + return -1; >> +} >> + >> +/* detach the device, then store the name of the device */ >> +int >> +rte_eal_dev_detach(uint8_t port_id __rte_unused, >> + char *name __rte_unused) >> +{ >> + RTE_LOG(ERR, EAL, "Hotplug support isn't enabled\n"); >> + return -1; >> +} >> +#endif /* ENABLE_HOTPLUG */ >> diff --git a/lib/librte_eal/common/eal_private.h b/lib/librte_eal/common/eal_private.h >> index a97c5d8..453a1eb 100644 >> --- a/lib/librte_eal/common/eal_private.h >> +++ b/lib/librte_eal/common/eal_private.h >> @@ -164,6 +164,17 @@ enum rte_eal_invoke_type { >> }; >> >> /** >> + * Scan the content of the PCI bus, and the devices in the devices >> + * list >> + * >> + * This function is private to EAL. >> + * >> + * @return >> + * 0 on success, negative on error >> + */ >> +int rte_eal_pci_scan(void); >> + >> +/** >> * Mmap memory for single PCI device >> * >> * This function is private to EAL. >> diff --git a/lib/librte_eal/common/include/rte_dev.h b/lib/librte_eal/common/include/rte_dev.h >> index f7e3a10..e63dd1c 100644 >> --- a/lib/librte_eal/common/include/rte_dev.h >> +++ b/lib/librte_eal/common/include/rte_dev.h >> @@ -47,6 +47,7 @@ extern "C" { >> #endif >> >> #include <sys/queue.h> >> +#include <rte_pci.h> >> >> /** Double linked list of device drivers. */ >> TAILQ_HEAD(rte_driver_list, rte_driver); >> @@ -57,6 +58,11 @@ TAILQ_HEAD(rte_driver_list, rte_driver); >> typedef int (rte_dev_init_t)(const char *name, const char *args); >> >> /** >> + * Uninitilization function called for each device driver once. >> + */ >> +typedef int (rte_dev_uninit_t)(const char *name, const char *args); >> + >> +/** >> * Driver type enumeration >> */ >> enum pmd_type { >> @@ -72,6 +78,7 @@ struct rte_driver { >> enum pmd_type type; /**< PMD Driver type */ >> const char *name; /**< Driver name. */ >> rte_dev_init_t *init; /**< Device init. function. */ >> + rte_dev_uninit_t *uninit; /**< Device uninit. function. */ >> }; >> >> /** >> @@ -93,6 +100,32 @@ void rte_eal_driver_register(struct rte_driver *driver); >> void rte_eal_driver_unregister(struct rte_driver *driver); >> >> /** >> + * Attach a new device. >> + * >> + * @param devargs >> + * A pointer to a strings array describing the new device >> + * to be attached. The strings should be a pci address like >> + * '0000:01:00.0' or virtual device name like 'eth_pcap0'. >> + * @param port_id >> + * A pointer to a port identifier actually attached. >> + * @return >> + * 0 on success and port_id is filled, negative on error >> + */ >> +int rte_eal_dev_attach(const char *devargs, uint8_t *port_id); >> + >> +/** >> + * Detach a device. >> + * >> + * @param port_id >> + * The port identifier of the device to detach. >> + * @param addr >> + * A pointer to a device name actually detached. >> + * @return >> + * 0 on success and devname is filled, negative on error >> + */ >> +int rte_eal_dev_detach(uint8_t port_id, char *devname); >> + >> +/** >> * Initalize all the registered drivers in this process >> */ >> int rte_eal_dev_init(void); >> diff --git a/lib/librte_eal/linuxapp/eal/Makefile b/lib/librte_eal/linuxapp/eal/Makefile >> index 72ecf3a..0ec83b5 100644 >> --- a/lib/librte_eal/linuxapp/eal/Makefile >> +++ b/lib/librte_eal/linuxapp/eal/Makefile >> @@ -41,6 +41,7 @@ CFLAGS += -I$(RTE_SDK)/lib/librte_eal/common/include >> CFLAGS += -I$(RTE_SDK)/lib/librte_ring >> CFLAGS += -I$(RTE_SDK)/lib/librte_mempool >> CFLAGS += -I$(RTE_SDK)/lib/librte_malloc >> +CFLAGS += -I$(RTE_SDK)/lib/librte_mbuf >> CFLAGS += -I$(RTE_SDK)/lib/librte_ether >> CFLAGS += -I$(RTE_SDK)/lib/librte_ivshmem >> CFLAGS += -I$(RTE_SDK)/lib/librte_pmd_ring >> diff --git a/lib/librte_eal/linuxapp/eal/eal_pci.c b/lib/librte_eal/linuxapp/eal/eal_pci.c >> index a23cc59..8e7e650 100644 >> --- a/lib/librte_eal/linuxapp/eal/eal_pci.c >> +++ b/lib/librte_eal/linuxapp/eal/eal_pci.c >> @@ -378,8 +378,8 @@ error: >> * Scan the content of the PCI bus, and the devices in the devices >> * list >> */ >> -static int >> -pci_scan(void) >> +int >> +rte_eal_pci_scan(void) >> { >> struct dirent *e; >> DIR *dir; >> @@ -701,7 +701,7 @@ rte_eal_pci_init(void) >> if (internal_config.no_pci) >> return 0; >> >> - if (pci_scan() < 0) { >> + if (rte_eal_pci_scan() < 0) { >> RTE_LOG(ERR, EAL, "%s(): Cannot scan PCI bus\n", __func__); >> return -1; >> }
Hi Michael, On 2015/01/21 11:40, Qiu, Michael wrote: > On 1/19/2015 6:42 PM, Tetsuya Mukawa wrote: >> The patch adds following functions. >> >> - rte_eth_dev_save() >> The function is used for saving current rte_eth_dev structures. >> - rte_eth_dev_get_changed_port() >> The function receives the rte_eth_dev structures, then compare >> these with current values to know which port is actually >> attached or detached. >> - rte_eth_dev_get_addr_by_port() >> The function returns a pci address of a ethdev specified by port >> identifier. >> - rte_eth_dev_get_port_by_addr() >> The function returns a port identifier of a ethdev specified by >> pci address. >> - rte_eth_dev_get_name_by_port() >> The function returns a unique identifier name of a ethdev >> specified by port identifier. >> - Add rte_eth_dev_check_detachable() >> The function returns whether a PMD supports detach function. >> >> Also the patch changes scope of rte_eth_dev_allocated() to global. >> This function will be called by virtual PMDs to support port hotplug. >> So change scope of the function to global. >> >> v4: >> - Add paramerter checking. >> v3: >> - Fix if-condition bug while comparing pci addresses. >> - Add error checking codes. >> Reported-by: Mark Enright <menrigh@brocade.com> >> >> Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp> >> --- >> lib/librte_ether/rte_ethdev.c | 102 +++++++++++++++++++++++++++++++++++++++++- >> lib/librte_ether/rte_ethdev.h | 80 +++++++++++++++++++++++++++++++++ >> 2 files changed, 181 insertions(+), 1 deletion(-) >> >> diff --git a/lib/librte_ether/rte_ethdev.c b/lib/librte_ether/rte_ethdev.c >> index e5145b7..e572ef4 100644 >> --- a/lib/librte_ether/rte_ethdev.c >> +++ b/lib/librte_ether/rte_ethdev.c >> @@ -206,7 +206,7 @@ rte_eth_dev_data_alloc(void) >> RTE_MAX_ETHPORTS * sizeof(*rte_eth_dev_data)); >> } >> >> -static struct rte_eth_dev * >> +struct rte_eth_dev * >> rte_eth_dev_allocated(const char *name) >> { >> unsigned i; >> @@ -422,6 +422,106 @@ rte_eth_dev_count(void) >> return (nb_ports); >> } >> >> +void >> +rte_eth_dev_save(struct rte_eth_dev *devs) >> +{ >> + if (devs == NULL) >> + return; >> + >> + /* save current rte_eth_devices */ >> + memcpy(devs, rte_eth_devices, >> + sizeof(struct rte_eth_dev) * RTE_MAX_ETHPORTS); >> +} >> + >> +int >> +rte_eth_dev_get_changed_port(struct rte_eth_dev *devs, uint8_t *port_id) >> +{ >> + if ((devs == NULL) || (port_id == NULL)) >> + return -1; >> + >> + /* check which port was attached or detached */ >> + for (*port_id = 0; *port_id < RTE_MAX_ETHPORTS; (*port_id)++, devs++) { >> + if (rte_eth_devices[*port_id].attached ^ devs->attached) >> + return 0; >> + } >> + return 1; >> +} >> + >> +int >> +rte_eth_dev_get_addr_by_port(uint8_t port_id, struct rte_pci_addr *addr) >> +{ >> + if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) { >> + PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); >> + return -EINVAL; >> + } >> + >> + if (addr == NULL) { >> + PMD_DEBUG_TRACE("Null pointer is specified\n"); >> + return -EINVAL; >> + } >> + >> + *addr = rte_eth_devices[port_id].pci_dev->addr; >> + return 0; >> +} >> + >> +int >> +rte_eth_dev_get_port_by_addr(struct rte_pci_addr *addr, uint8_t *port_id) >> +{ >> + struct rte_pci_addr *tmp; >> + >> + if ((addr == NULL) || (port_id == NULL)) { >> + PMD_DEBUG_TRACE("Null pointer is specified\n"); >> + return -1; > Is it better to replace "-1" to "-EINVAL" ? Thanks. I will do like above. Regards, Tetsuya > >> + } >> + >> + for (*port_id = 0; *port_id < RTE_MAX_ETHPORTS; (*port_id)++) { >> + if (!rte_eth_devices[*port_id].attached) >> + continue; >> + if (!rte_eth_devices[*port_id].pci_dev) >> + continue; >> + tmp = &rte_eth_devices[*port_id].pci_dev->addr; >> + if (eal_compare_pci_addr(tmp, addr) == 0) >> + return 0; >> + } >> + return -1; >> +} >> + >> +int >> +rte_eth_dev_get_name_by_port(uint8_t port_id, char *name) >> +{ >> + char *tmp; >> + >> + if (rte_eth_dev_validate_port(port_id) == DEV_INVALID) { >> + PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); >> + return -EINVAL; >> + } >> + >> + if (name == NULL) { >> + PMD_DEBUG_TRACE("Null pointer is specified\n"); >> + return -EINVAL; >> + } >> + >> + /* shouldn't check 'rte_eth_devices[i].data', >> + * because it might be overwritten by VDEV PMD */ >> + tmp = rte_eth_dev_data[port_id].name; >> + strncpy(name, tmp, strlen(tmp) + 1); >> + return 0; >> +} >> + >> +int >> +rte_eth_dev_check_detachable(uint8_t port_id) >> +{ >> + uint32_t drv_flags; >> + >> + if (port_id >= RTE_MAX_ETHPORTS) { >> + PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); >> + return -EINVAL; >> + } >> + >> + drv_flags = rte_eth_devices[port_id].driver->pci_drv.drv_flags; >> + return !(drv_flags & RTE_PCI_DRV_DETACHABLE); >> +} >> + >> static int >> rte_eth_dev_rx_queue_config(struct rte_eth_dev *dev, uint16_t nb_queues) >> { >> diff --git a/lib/librte_ether/rte_ethdev.h b/lib/librte_ether/rte_ethdev.h >> index 2a8ff26..c0b98dc 100644 >> --- a/lib/librte_ether/rte_ethdev.h >> +++ b/lib/librte_ether/rte_ethdev.h >> @@ -1642,6 +1642,86 @@ extern struct rte_eth_dev rte_eth_devices[]; >> extern uint8_t rte_eth_dev_count(void); >> >> /** >> + * Function for internal use by port hotplug functions. >> + * Copies current ethdev structures to the specified pointer. >> + * >> + * @param devs The pointer to the ethdev structures >> + */ >> +extern void rte_eth_dev_save(struct rte_eth_dev *devs); >> + >> +/** >> + * Function for internal use by port hotplug functions. >> + * Compare the specified ethdev structures with currents. Then >> + * if there is a port which status is changed, fill the specified pointer >> + * with the port id of that port. >> + * @param devs The pointer to the ethdev structures >> + * @param port_id The pointer to the port id >> + * @return >> + * - 0 on success, negative on error >> + */ >> +extern int rte_eth_dev_get_changed_port( >> + struct rte_eth_dev *devs, uint8_t *port_id); >> + >> +/** >> + * Function for internal use by port hotplug functions. >> + * Returns a pci address of a ethdev specified by port identifier. >> + * @param port_id >> + * The port identifier of the Ethernet device >> + * @param addr >> + * The pointer to the pci address >> + * @return >> + * - 0 on success, negative on error >> + */ >> +extern int rte_eth_dev_get_addr_by_port( >> + uint8_t port_id, struct rte_pci_addr *addr); >> + >> +/** >> + * Function for internal use by port hotplug functions. >> + * Returns a port identifier of a ethdev specified by pci address. >> + * @param addr >> + * The pointer to the pci address of the Ethernet device. >> + * @param port_id >> + * The pointer to the port identifier >> + * @return >> + * - 0 on success, negative on error >> + */ >> +extern int rte_eth_dev_get_port_by_addr( >> + struct rte_pci_addr *addr, uint8_t *port_id); >> + >> +/** >> + * Function for internal use by port hotplug functions. >> + * Returns a unique identifier name of a ethdev specified by port identifier. >> + * @param port_id >> + * The port identifier. >> + * @param name >> + * The pointer to the Unique identifier name for each Ethernet device >> + * @return >> + * - 0 on success, negative on error >> + */ >> +extern int rte_eth_dev_get_name_by_port(uint8_t port_id, char *name); >> + >> +/** >> + * Function for internal use by port hotplug functions. >> + * Check whether or not, a PMD that is handling the ethdev specified by port >> + * identifier can support detach function. >> + * @param port_id >> + * The port identifier >> + * @return >> + * - 0 on supporting detach function, negative on not supporting >> + */ >> +extern int rte_eth_dev_check_detachable(uint8_t port_id); >> + >> +/** >> + * Function for internal use by port hotplug functions. >> + * Returns a ethdev slot specified by the unique identifier name. >> + * @param name >> + * The pointer to the Unique identifier name for each Ethernet device >> + * @return >> + * - The pointer to the ethdev slot, on success. NULL on error >> + */ >> +extern struct rte_eth_dev *rte_eth_dev_allocated(const char *name); >> + >> +/** >> * Function for internal use by dummy drivers primarily, e.g. ring-based >> * driver. >> * Allocates a new ethdev slot for an ethernet device and returns the pointer
Hi Michael, On 2015/01/20 18:23, Qiu, Michael wrote: > On 1/19/2015 6:42 PM, Tetsuya Mukawa wrote: >> The patch adds functions for unmapping igb_uio resources. The patch is only >> for Linux and igb_uio environment. VFIO and BSD are not supported. >> >> v4: >> - Add paramerter checking. >> - Add header file to determine if hotplug can be enabled. >> >> Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp> >> --- >> lib/librte_eal/common/Makefile | 1 + >> lib/librte_eal/common/include/rte_dev_hotplug.h | 44 +++++++++++++++++ >> lib/librte_eal/linuxapp/eal/eal_pci.c | 38 +++++++++++++++ >> lib/librte_eal/linuxapp/eal/eal_pci_init.h | 8 +++ >> lib/librte_eal/linuxapp/eal/eal_pci_uio.c | 65 +++++++++++++++++++++++++ >> 5 files changed, 156 insertions(+) >> create mode 100644 lib/librte_eal/common/include/rte_dev_hotplug.h >> >> diff --git a/lib/librte_eal/common/Makefile b/lib/librte_eal/common/Makefile >> index 52c1a5f..db7cc93 100644 >> --- a/lib/librte_eal/common/Makefile >> +++ b/lib/librte_eal/common/Makefile >> @@ -41,6 +41,7 @@ INC += rte_eal_memconfig.h rte_malloc_heap.h >> INC += rte_hexdump.h rte_devargs.h rte_dev.h >> INC += rte_common_vect.h >> INC += rte_pci_dev_feature_defs.h rte_pci_dev_features.h >> +INC += rte_dev_hotplug.h >> >> ifeq ($(CONFIG_RTE_INSECURE_FUNCTION_WARNING),y) >> INC += rte_warnings.h >> diff --git a/lib/librte_eal/common/include/rte_dev_hotplug.h b/lib/librte_eal/common/include/rte_dev_hotplug.h >> new file mode 100644 >> index 0000000..b333e0f >> --- /dev/null >> +++ b/lib/librte_eal/common/include/rte_dev_hotplug.h >> @@ -0,0 +1,44 @@ >> +/*- >> + * BSD LICENSE >> + * >> + * Copyright(c) 2015 IGEL Co.,LTd. >> + * All rights reserved. >> + * >> + * Redistribution and use in source and binary forms, with or without >> + * modification, are permitted provided that the following conditions >> + * are met: >> + * >> + * * Redistributions of source code must retain the above copyright >> + * notice, this list of conditions and the following disclaimer. >> + * * Redistributions in binary form must reproduce the above copyright >> + * notice, this list of conditions and the following disclaimer in >> + * the documentation and/or other materials provided with the >> + * distribution. >> + * * Neither the name of IGEL Co.,Ltd. nor the names of its >> + * contributors may be used to endorse or promote products derived >> + * from this software without specific prior written permission. >> + * >> + * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS >> + * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT >> + * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR >> + * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT >> + * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, >> + * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT >> + * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, >> + * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY >> + * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT >> + * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE >> + * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. >> + */ >> + >> +#ifndef _RTE_DEV_HOTPLUG_H_ >> +#define _RTE_DEV_HOTPLUG_H_ >> + >> +/* >> + * determine if hotplug can be enabled on the system >> + */ >> +#if defined(RTE_LIBRTE_EAL_HOTPLUG) && defined(RTE_LIBRTE_EAL_LINUXAPP) > As you said, VFIO should not work with it, so does it need to add the > vfio check here? Could I have a advice of you? First I guess it's the best to include "eal_vfio.h" here, and add checking of VFIO_PRESENT macro. But it seems I cannot reach "eal_vfio.h" from this file. My second option is just checking RTE_EAL_VFIO macro. But according to "eal_vfio.h", if kernel is under 3.6.0, VFIO_PRESENT will not be defined even when RTE_EAL_VFIO is enabled. So I guess simply macro checking will not work correctly. Anyway, here are my implementation choices so far. 1. Like "eal_vfio.h", check kernel version in "rte_dev_hotplug.h". In this case, if "eal_vfio.h" is changed, "rte_edv_hotplug.h" may need to be changed also. 2. Merge "eal_vfio.h" and "rte_dev_hotplug.h" definitions, and define these in new rte header like "rte_settings.h". Can I have advice about it? Thanks, Tetsuya > > Thanks, > Michael >> +#define ENABLE_HOTPLUG >> +#endif /* RTE_LIBRTE_EAL_HOTPLUG & RTE_LIBRTE_EAL_LINUXAPP */ >> + >> +#endif /* _RTE_DEV_HOTPLUG_H_ */ >> diff --git a/lib/librte_eal/linuxapp/eal/eal_pci.c b/lib/librte_eal/linuxapp/eal/eal_pci.c >> index 3d2d93c..52c464c 100644 >> --- a/lib/librte_eal/linuxapp/eal/eal_pci.c >> +++ b/lib/librte_eal/linuxapp/eal/eal_pci.c >> @@ -137,6 +137,25 @@ pci_map_resource(void *requested_addr, int fd, off_t offset, size_t size) >> return mapaddr; >> } >> >> +#ifdef ENABLE_HOTPLUG >> +/* unmap a particular resource */ >> +void >> +pci_unmap_resource(void *requested_addr, size_t size) >> +{ >> + if (requested_addr == NULL) >> + return; >> + >> + /* Unmap the PCI memory resource of device */ >> + if (munmap(requested_addr, size)) { >> + RTE_LOG(ERR, EAL, "%s(): cannot munmap(%p, 0x%lx): %s\n", >> + __func__, requested_addr, (unsigned long)size, >> + strerror(errno)); >> + } else >> + RTE_LOG(DEBUG, EAL, " PCI memory mapped at %p\n", >> + requested_addr); >> +} >> +#endif /* ENABLE_HOTPLUG */ >> + >> /* parse the "resource" sysfs file */ >> #define IORESOURCE_MEM 0x00000200 >> >> @@ -510,6 +529,25 @@ pci_map_device(struct rte_pci_device *dev) >> return 0; >> } >> >> +#ifdef ENABLE_HOTPLUG >> +static void >> +pci_unmap_device(struct rte_pci_device *dev) >> +{ >> + if (dev == NULL) >> + return; >> + >> + /* try unmapping the NIC resources using VFIO if it exists */ >> +#ifdef VFIO_PRESENT >> + if (pci_vfio_is_enabled()) { >> + RTE_LOG(ERR, EAL, "%s() doesn't support vfio yet.\n", >> + __func__); >> + return; >> + } >> +#endif >> + pci_uio_unmap_resource(dev); >> +} >> +#endif /* ENABLE_HOTPLUG */ >> + >> /* >> * If vendor/device ID match, call the devinit() function of the >> * driver. >> diff --git a/lib/librte_eal/linuxapp/eal/eal_pci_init.h b/lib/librte_eal/linuxapp/eal/eal_pci_init.h >> index 1070eb8..5152a0b 100644 >> --- a/lib/librte_eal/linuxapp/eal/eal_pci_init.h >> +++ b/lib/librte_eal/linuxapp/eal/eal_pci_init.h >> @@ -34,6 +34,7 @@ >> #ifndef EAL_PCI_INIT_H_ >> #define EAL_PCI_INIT_H_ >> >> +#include <rte_dev_hotplug.h> >> #include "eal_vfio.h" >> >> struct pci_map { >> @@ -71,6 +72,13 @@ void *pci_map_resource(void *requested_addr, int fd, off_t offset, >> /* map IGB_UIO resource prototype */ >> int pci_uio_map_resource(struct rte_pci_device *dev); >> >> +#ifdef ENABLE_HOTPLUG >> +void pci_unmap_resource(void *requested_addr, size_t size); >> + >> +/* unmap IGB_UIO resource prototype */ >> +void pci_uio_unmap_resource(struct rte_pci_device *dev); >> +#endif /* ENABLE_HOTPLUG */ >> + >> #ifdef VFIO_PRESENT >> >> #define VFIO_MAX_GROUPS 64 >> diff --git a/lib/librte_eal/linuxapp/eal/eal_pci_uio.c b/lib/librte_eal/linuxapp/eal/eal_pci_uio.c >> index 1da3507..81830d1 100644 >> --- a/lib/librte_eal/linuxapp/eal/eal_pci_uio.c >> +++ b/lib/librte_eal/linuxapp/eal/eal_pci_uio.c >> @@ -404,6 +404,71 @@ pci_uio_map_resource(struct rte_pci_device *dev) >> return 0; >> } >> >> +#ifdef ENABLE_HOTPLUG >> +static void >> +pci_uio_unmap(struct mapped_pci_resource *uio_res) >> +{ >> + int i; >> + >> + if (uio_res == NULL) >> + return; >> + >> + for (i = 0; i != uio_res->nb_maps; i++) >> + pci_unmap_resource(uio_res->maps[i].addr, >> + (size_t)uio_res->maps[i].size); >> +} >> + >> +static struct mapped_pci_resource * >> +pci_uio_find_resource(struct rte_pci_device *dev) >> +{ >> + struct mapped_pci_resource *uio_res; >> + >> + if (dev == NULL) >> + return NULL; >> + >> + TAILQ_FOREACH(uio_res, pci_res_list, next) { >> + >> + /* skip this element if it doesn't match our PCI address */ >> + if (!eal_compare_pci_addr(&uio_res->pci_addr, &dev->addr)) >> + return uio_res; >> + } >> + return NULL; >> +} >> + >> +/* unmap the PCI resource of a PCI device in virtual memory */ >> +void >> +pci_uio_unmap_resource(struct rte_pci_device *dev) >> +{ >> + struct mapped_pci_resource *uio_res; >> + >> + if (dev == NULL) >> + return; >> + >> + /* find an entry for the device */ >> + uio_res = pci_uio_find_resource(dev); >> + if (uio_res == NULL) >> + return; >> + >> + /* secondary processes - just free maps */ >> + if (rte_eal_process_type() != RTE_PROC_PRIMARY) >> + return pci_uio_unmap(uio_res); >> + >> + TAILQ_REMOVE(pci_res_list, uio_res, next); >> + >> + /* unmap all resources */ >> + pci_uio_unmap(uio_res); >> + >> + /* free uio resource */ >> + rte_free(uio_res); >> + >> + /* close fd if in primary process */ >> + close(dev->intr_handle.fd); >> + >> + dev->intr_handle.fd = -1; >> + dev->intr_handle.type = RTE_INTR_HANDLE_UNKNOWN; >> +} >> +#endif /* ENABLE_HOTPLUG */ >> + >> /* >> * parse a sysfs file containing one integer value >> * different to the eal version, as it needs to work with 64-bit values
On 1/21/2015 6:01 PM, Tetsuya Mukawa wrote: > Hi Michael, > > On 2015/01/20 18:23, Qiu, Michael wrote: >> On 1/19/2015 6:42 PM, Tetsuya Mukawa wrote: >>> The patch adds functions for unmapping igb_uio resources. The patch is only >>> for Linux and igb_uio environment. VFIO and BSD are not supported. >>> >>> v4: >>> - Add paramerter checking. >>> - Add header file to determine if hotplug can be enabled. >>> >>> Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp> >>> --- >>> lib/librte_eal/common/Makefile | 1 + >>> lib/librte_eal/common/include/rte_dev_hotplug.h | 44 +++++++++++++++++ >>> lib/librte_eal/linuxapp/eal/eal_pci.c | 38 +++++++++++++++ >>> lib/librte_eal/linuxapp/eal/eal_pci_init.h | 8 +++ >>> lib/librte_eal/linuxapp/eal/eal_pci_uio.c | 65 +++++++++++++++++++++++++ >>> 5 files changed, 156 insertions(+) >>> create mode 100644 lib/librte_eal/common/include/rte_dev_hotplug.h >>> >>> diff --git a/lib/librte_eal/common/Makefile b/lib/librte_eal/common/Makefile >>> index 52c1a5f..db7cc93 100644 >>> --- a/lib/librte_eal/common/Makefile >>> +++ b/lib/librte_eal/common/Makefile >>> @@ -41,6 +41,7 @@ INC += rte_eal_memconfig.h rte_malloc_heap.h >>> INC += rte_hexdump.h rte_devargs.h rte_dev.h >>> INC += rte_common_vect.h >>> INC += rte_pci_dev_feature_defs.h rte_pci_dev_features.h >>> +INC += rte_dev_hotplug.h >>> >>> ifeq ($(CONFIG_RTE_INSECURE_FUNCTION_WARNING),y) >>> INC += rte_warnings.h >>> diff --git a/lib/librte_eal/common/include/rte_dev_hotplug.h b/lib/librte_eal/common/include/rte_dev_hotplug.h >>> new file mode 100644 >>> index 0000000..b333e0f >>> --- /dev/null >>> +++ b/lib/librte_eal/common/include/rte_dev_hotplug.h >>> @@ -0,0 +1,44 @@ >>> +/*- >>> + * BSD LICENSE >>> + * >>> + * Copyright(c) 2015 IGEL Co.,LTd. >>> + * All rights reserved. >>> + * >>> + * Redistribution and use in source and binary forms, with or without >>> + * modification, are permitted provided that the following conditions >>> + * are met: >>> + * >>> + * * Redistributions of source code must retain the above copyright >>> + * notice, this list of conditions and the following disclaimer. >>> + * * Redistributions in binary form must reproduce the above copyright >>> + * notice, this list of conditions and the following disclaimer in >>> + * the documentation and/or other materials provided with the >>> + * distribution. >>> + * * Neither the name of IGEL Co.,Ltd. nor the names of its >>> + * contributors may be used to endorse or promote products derived >>> + * from this software without specific prior written permission. >>> + * >>> + * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS >>> + * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT >>> + * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR >>> + * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT >>> + * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, >>> + * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT >>> + * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, >>> + * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY >>> + * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT >>> + * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE >>> + * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. >>> + */ >>> + >>> +#ifndef _RTE_DEV_HOTPLUG_H_ >>> +#define _RTE_DEV_HOTPLUG_H_ >>> + >>> +/* >>> + * determine if hotplug can be enabled on the system >>> + */ >>> +#if defined(RTE_LIBRTE_EAL_HOTPLUG) && defined(RTE_LIBRTE_EAL_LINUXAPP) >> As you said, VFIO should not work with it, so does it need to add the >> vfio check here? > Could I have a advice of you? > First I guess it's the best to include "eal_vfio.h" here, and add > checking of VFIO_PRESENT macro. I have a question, will your hotplug feature support freebsd ? If not, how about to put it in "lib/librte_eal/linuxapp/eal/" ? Also include attach or detach affairs. > But it seems I cannot reach "eal_vfio.h" from this file. Yes, you can't :) > My second option is just checking RTE_EAL_VFIO macro. > But according to "eal_vfio.h", if kernel is under 3.6.0, VFIO_PRESENT Actually, in my opinion, whatever vfio or uio, only need be care in runtime. DPDK to check vfio only to add support for vfio, but this does not means the device will use vfio, So even if VFIO_PRESENT is defined, and vfio is enabled, but the device is bind to igb_uio, then your hotplug still need work, but if it bind to vfio, will not, am I right? If yes, I'm not sure if your hotplug has this ability, but it is reasonable, I think. > will not be defined even when RTE_EAL_VFIO is enabled. > So I guess simply macro checking will not work correctly. > > Anyway, here are my implementation choices so far. > > 1. Like "eal_vfio.h", check kernel version in "rte_dev_hotplug.h". > In this case, if "eal_vfio.h" is changed, "rte_edv_hotplug.h" may need > to be changed also. > > 2. Merge "eal_vfio.h" and "rte_dev_hotplug.h" definitions, and define > these in new rte header like "rte_settings.h". > > Can I have advice about it? As I said, VFIO enable or not is not related for your hotplug, only the devices managed by VFIO will affect your hotplug. If you agree, I think need discuss the details of it. Thanks, Michael > Thanks, > Tetsuya > >> Thanks, >> Michael >>> +#define ENABLE_HOTPLUG >>> +#endif /* RTE_LIBRTE_EAL_HOTPLUG & RTE_LIBRTE_EAL_LINUXAPP */ >>> + >>> +#endif /* _RTE_DEV_HOTPLUG_H_ */ >>> diff --git a/lib/librte_eal/linuxapp/eal/eal_pci.c b/lib/librte_eal/linuxapp/eal/eal_pci.c >>> index 3d2d93c..52c464c 100644 >>> --- a/lib/librte_eal/linuxapp/eal/eal_pci.c >>> +++ b/lib/librte_eal/linuxapp/eal/eal_pci.c >>> @@ -137,6 +137,25 @@ pci_map_resource(void *requested_addr, int fd, off_t offset, size_t size) >>> return mapaddr; >>> } >>> >>> +#ifdef ENABLE_HOTPLUG >>> +/* unmap a particular resource */ >>> +void >>> +pci_unmap_resource(void *requested_addr, size_t size) >>> +{ >>> + if (requested_addr == NULL) >>> + return; >>> + >>> + /* Unmap the PCI memory resource of device */ >>> + if (munmap(requested_addr, size)) { >>> + RTE_LOG(ERR, EAL, "%s(): cannot munmap(%p, 0x%lx): %s\n", >>> + __func__, requested_addr, (unsigned long)size, >>> + strerror(errno)); >>> + } else >>> + RTE_LOG(DEBUG, EAL, " PCI memory mapped at %p\n", >>> + requested_addr); >>> +} >>> +#endif /* ENABLE_HOTPLUG */ >>> + >>> /* parse the "resource" sysfs file */ >>> #define IORESOURCE_MEM 0x00000200 >>> >>> @@ -510,6 +529,25 @@ pci_map_device(struct rte_pci_device *dev) >>> return 0; >>> } >>> >>> +#ifdef ENABLE_HOTPLUG >>> +static void >>> +pci_unmap_device(struct rte_pci_device *dev) >>> +{ >>> + if (dev == NULL) >>> + return; >>> + >>> + /* try unmapping the NIC resources using VFIO if it exists */ >>> +#ifdef VFIO_PRESENT >>> + if (pci_vfio_is_enabled()) { >>> + RTE_LOG(ERR, EAL, "%s() doesn't support vfio yet.\n", >>> + __func__); >>> + return; >>> + } >>> +#endif >>> + pci_uio_unmap_resource(dev); >>> +} >>> +#endif /* ENABLE_HOTPLUG */ >>> + >>> /* >>> * If vendor/device ID match, call the devinit() function of the >>> * driver. >>> diff --git a/lib/librte_eal/linuxapp/eal/eal_pci_init.h b/lib/librte_eal/linuxapp/eal/eal_pci_init.h >>> index 1070eb8..5152a0b 100644 >>> --- a/lib/librte_eal/linuxapp/eal/eal_pci_init.h >>> +++ b/lib/librte_eal/linuxapp/eal/eal_pci_init.h >>> @@ -34,6 +34,7 @@ >>> #ifndef EAL_PCI_INIT_H_ >>> #define EAL_PCI_INIT_H_ >>> >>> +#include <rte_dev_hotplug.h> >>> #include "eal_vfio.h" >>> >>> struct pci_map { >>> @@ -71,6 +72,13 @@ void *pci_map_resource(void *requested_addr, int fd, off_t offset, >>> /* map IGB_UIO resource prototype */ >>> int pci_uio_map_resource(struct rte_pci_device *dev); >>> >>> +#ifdef ENABLE_HOTPLUG >>> +void pci_unmap_resource(void *requested_addr, size_t size); >>> + >>> +/* unmap IGB_UIO resource prototype */ >>> +void pci_uio_unmap_resource(struct rte_pci_device *dev); >>> +#endif /* ENABLE_HOTPLUG */ >>> + >>> #ifdef VFIO_PRESENT >>> >>> #define VFIO_MAX_GROUPS 64 >>> diff --git a/lib/librte_eal/linuxapp/eal/eal_pci_uio.c b/lib/librte_eal/linuxapp/eal/eal_pci_uio.c >>> index 1da3507..81830d1 100644 >>> --- a/lib/librte_eal/linuxapp/eal/eal_pci_uio.c >>> +++ b/lib/librte_eal/linuxapp/eal/eal_pci_uio.c >>> @@ -404,6 +404,71 @@ pci_uio_map_resource(struct rte_pci_device *dev) >>> return 0; >>> } >>> >>> +#ifdef ENABLE_HOTPLUG >>> +static void >>> +pci_uio_unmap(struct mapped_pci_resource *uio_res) >>> +{ >>> + int i; >>> + >>> + if (uio_res == NULL) >>> + return; >>> + >>> + for (i = 0; i != uio_res->nb_maps; i++) >>> + pci_unmap_resource(uio_res->maps[i].addr, >>> + (size_t)uio_res->maps[i].size); >>> +} >>> + >>> +static struct mapped_pci_resource * >>> +pci_uio_find_resource(struct rte_pci_device *dev) >>> +{ >>> + struct mapped_pci_resource *uio_res; >>> + >>> + if (dev == NULL) >>> + return NULL; >>> + >>> + TAILQ_FOREACH(uio_res, pci_res_list, next) { >>> + >>> + /* skip this element if it doesn't match our PCI address */ >>> + if (!eal_compare_pci_addr(&uio_res->pci_addr, &dev->addr)) >>> + return uio_res; >>> + } >>> + return NULL; >>> +} >>> + >>> +/* unmap the PCI resource of a PCI device in virtual memory */ >>> +void >>> +pci_uio_unmap_resource(struct rte_pci_device *dev) >>> +{ >>> + struct mapped_pci_resource *uio_res; >>> + >>> + if (dev == NULL) >>> + return; >>> + >>> + /* find an entry for the device */ >>> + uio_res = pci_uio_find_resource(dev); >>> + if (uio_res == NULL) >>> + return; >>> + >>> + /* secondary processes - just free maps */ >>> + if (rte_eal_process_type() != RTE_PROC_PRIMARY) >>> + return pci_uio_unmap(uio_res); >>> + >>> + TAILQ_REMOVE(pci_res_list, uio_res, next); >>> + >>> + /* unmap all resources */ >>> + pci_uio_unmap(uio_res); >>> + >>> + /* free uio resource */ >>> + rte_free(uio_res); >>> + >>> + /* close fd if in primary process */ >>> + close(dev->intr_handle.fd); >>> + >>> + dev->intr_handle.fd = -1; >>> + dev->intr_handle.type = RTE_INTR_HANDLE_UNKNOWN; >>> +} >>> +#endif /* ENABLE_HOTPLUG */ >>> + >>> /* >>> * parse a sysfs file containing one integer value >>> * different to the eal version, as it needs to work with 64-bit values > >
Hi Michael, On 2015/01/22 17:12, Qiu, Michael wrote: > On 1/21/2015 6:01 PM, Tetsuya Mukawa wrote: >> Hi Michael, >> >> On 2015/01/20 18:23, Qiu, Michael wrote: >>> On 1/19/2015 6:42 PM, Tetsuya Mukawa wrote: >>>> The patch adds functions for unmapping igb_uio resources. The patch is only >>>> for Linux and igb_uio environment. VFIO and BSD are not supported. >>>> >>>> v4: >>>> - Add paramerter checking. >>>> - Add header file to determine if hotplug can be enabled. >>>> >>>> Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp> >>>> --- >>>> lib/librte_eal/common/Makefile | 1 + >>>> lib/librte_eal/common/include/rte_dev_hotplug.h | 44 +++++++++++++++++ >>>> lib/librte_eal/linuxapp/eal/eal_pci.c | 38 +++++++++++++++ >>>> lib/librte_eal/linuxapp/eal/eal_pci_init.h | 8 +++ >>>> lib/librte_eal/linuxapp/eal/eal_pci_uio.c | 65 +++++++++++++++++++++++++ >>>> 5 files changed, 156 insertions(+) >>>> create mode 100644 lib/librte_eal/common/include/rte_dev_hotplug.h >>>> >>>> diff --git a/lib/librte_eal/common/Makefile b/lib/librte_eal/common/Makefile >>>> index 52c1a5f..db7cc93 100644 >>>> --- a/lib/librte_eal/common/Makefile >>>> +++ b/lib/librte_eal/common/Makefile >>>> @@ -41,6 +41,7 @@ INC += rte_eal_memconfig.h rte_malloc_heap.h >>>> INC += rte_hexdump.h rte_devargs.h rte_dev.h >>>> INC += rte_common_vect.h >>>> INC += rte_pci_dev_feature_defs.h rte_pci_dev_features.h >>>> +INC += rte_dev_hotplug.h >>>> >>>> ifeq ($(CONFIG_RTE_INSECURE_FUNCTION_WARNING),y) >>>> INC += rte_warnings.h >>>> diff --git a/lib/librte_eal/common/include/rte_dev_hotplug.h b/lib/librte_eal/common/include/rte_dev_hotplug.h >>>> new file mode 100644 >>>> index 0000000..b333e0f >>>> --- /dev/null >>>> +++ b/lib/librte_eal/common/include/rte_dev_hotplug.h >>>> @@ -0,0 +1,44 @@ >>>> +/*- >>>> + * BSD LICENSE >>>> + * >>>> + * Copyright(c) 2015 IGEL Co.,LTd. >>>> + * All rights reserved. >>>> + * >>>> + * Redistribution and use in source and binary forms, with or without >>>> + * modification, are permitted provided that the following conditions >>>> + * are met: >>>> + * >>>> + * * Redistributions of source code must retain the above copyright >>>> + * notice, this list of conditions and the following disclaimer. >>>> + * * Redistributions in binary form must reproduce the above copyright >>>> + * notice, this list of conditions and the following disclaimer in >>>> + * the documentation and/or other materials provided with the >>>> + * distribution. >>>> + * * Neither the name of IGEL Co.,Ltd. nor the names of its >>>> + * contributors may be used to endorse or promote products derived >>>> + * from this software without specific prior written permission. >>>> + * >>>> + * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS >>>> + * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT >>>> + * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR >>>> + * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT >>>> + * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, >>>> + * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT >>>> + * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, >>>> + * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY >>>> + * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT >>>> + * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE >>>> + * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. >>>> + */ >>>> + >>>> +#ifndef _RTE_DEV_HOTPLUG_H_ >>>> +#define _RTE_DEV_HOTPLUG_H_ >>>> + >>>> +/* >>>> + * determine if hotplug can be enabled on the system >>>> + */ >>>> +#if defined(RTE_LIBRTE_EAL_HOTPLUG) && defined(RTE_LIBRTE_EAL_LINUXAPP) >>> As you said, VFIO should not work with it, so does it need to add the >>> vfio check here? >> Could I have a advice of you? >> First I guess it's the best to include "eal_vfio.h" here, and add >> checking of VFIO_PRESENT macro. > > I have a question, will your hotplug feature support freebsd ? > > If not, how about to put it in "lib/librte_eal/linuxapp/eal/" ? Also > include attach or detach affairs. I appreciate your comments. So far, FreeBSD doesn't support PCI hotplug. So I didn't implement code for FreeBSD. But in the future, I want to implement it when FreeBSD supports it. Also my hotplug implementation depends on legacy code already implemented in common layer. Anyway, for me it's nice to implement the feature in common layer. >> But it seems I cannot reach "eal_vfio.h" from this file. > Yes, you can't :) > >> My second option is just checking RTE_EAL_VFIO macro. >> But according to "eal_vfio.h", if kernel is under 3.6.0, VFIO_PRESENT > Actually, in my opinion, whatever vfio or uio, only need be care in > runtime. > > DPDK to check vfio only to add support for vfio, but this does not > means the device will use vfio, > > So even if VFIO_PRESENT is defined, and vfio is enabled, but the device > is bind to igb_uio, then your hotplug still need work, but if it bind > to vfio, will not, am I right? > > If yes, I'm not sure if your hotplug has this ability, but it is > reasonable, I think. I agree with your concept. But I guess it's not only related with hotplug function. The hotplug implementation depends on legacy functions that is for probing device. To implement above concept will change not only hotplug behavior but also legacy device probing. Conceptually I agree with such a functionality, but legacy probing function doesn't have such a feature so far. So I guess it's nice to separate this feature from hotplug patches. Realistically, the hotplug patches are big, and it's a bit hard to add and manage one more feature. If it's ok to separate the patch, it's helpful to manage patches. >> will not be defined even when RTE_EAL_VFIO is enabled. >> So I guess simply macro checking will not work correctly. >> >> Anyway, here are my implementation choices so far. >> >> 1. Like "eal_vfio.h", check kernel version in "rte_dev_hotplug.h". >> In this case, if "eal_vfio.h" is changed, "rte_edv_hotplug.h" may need >> to be changed also. >> >> 2. Merge "eal_vfio.h" and "rte_dev_hotplug.h" definitions, and define >> these in new rte header like "rte_settings.h". >> >> Can I have advice about it? > As I said, VFIO enable or not is not related for your hotplug, only the > devices managed by VFIO will affect your hotplug. > > If you agree, I think need discuss the details of it. Yes, I agree with your concept. And if you agree, I will implement it separately. To discuss how to handle VFIO and igb_uio devices in parallel, I guess we need to think about generic uio driver for pci devices. I guess before applying uio generic patch, this kind of discussion will be needed. I hope igb_uio (and VFIO?) be obsolete after applying uio generic. In the case, we don't need to think it. BTW, back to 'rte_dev_hotplug.h' discussion of hotplug patch. If VFIO_PRESENT isn't checked here, attaching port will be successful even if the device is under VFIO. (Because we already has legacy code for probing VFIO device. The hotplug function just re-use it.) But we cannot detach the device, because there is no code for closing VFIO device. There is no crash when the VFIO device is detached, but some error messages will be displayed. So I guess this behavior is like your description. How about it? Thanks, Tetsuya > Thanks, > Michael >> Thanks, >> Tetsuya >> >>> Thanks, >>> Michael >>>> +#define ENABLE_HOTPLUG >>>> +#endif /* RTE_LIBRTE_EAL_HOTPLUG & RTE_LIBRTE_EAL_LINUXAPP */ >>>> + >>>> +#endif /* _RTE_DEV_HOTPLUG_H_ */ >>>> diff --git a/lib/librte_eal/linuxapp/eal/eal_pci.c b/lib/librte_eal/linuxapp/eal/eal_pci.c >>>> index 3d2d93c..52c464c 100644 >>>> --- a/lib/librte_eal/linuxapp/eal/eal_pci.c >>>> +++ b/lib/librte_eal/linuxapp/eal/eal_pci.c >>>> @@ -137,6 +137,25 @@ pci_map_resource(void *requested_addr, int fd, off_t offset, size_t size) >>>> return mapaddr; >>>> } >>>> >>>> +#ifdef ENABLE_HOTPLUG >>>> +/* unmap a particular resource */ >>>> +void >>>> +pci_unmap_resource(void *requested_addr, size_t size) >>>> +{ >>>> + if (requested_addr == NULL) >>>> + return; >>>> + >>>> + /* Unmap the PCI memory resource of device */ >>>> + if (munmap(requested_addr, size)) { >>>> + RTE_LOG(ERR, EAL, "%s(): cannot munmap(%p, 0x%lx): %s\n", >>>> + __func__, requested_addr, (unsigned long)size, >>>> + strerror(errno)); >>>> + } else >>>> + RTE_LOG(DEBUG, EAL, " PCI memory mapped at %p\n", >>>> + requested_addr); >>>> +} >>>> +#endif /* ENABLE_HOTPLUG */ >>>> + >>>> /* parse the "resource" sysfs file */ >>>> #define IORESOURCE_MEM 0x00000200 >>>> >>>> @@ -510,6 +529,25 @@ pci_map_device(struct rte_pci_device *dev) >>>> return 0; >>>> } >>>> >>>> +#ifdef ENABLE_HOTPLUG >>>> +static void >>>> +pci_unmap_device(struct rte_pci_device *dev) >>>> +{ >>>> + if (dev == NULL) >>>> + return; >>>> + >>>> + /* try unmapping the NIC resources using VFIO if it exists */ >>>> +#ifdef VFIO_PRESENT >>>> + if (pci_vfio_is_enabled()) { >>>> + RTE_LOG(ERR, EAL, "%s() doesn't support vfio yet.\n", >>>> + __func__); >>>> + return; >>>> + } >>>> +#endif >>>> + pci_uio_unmap_resource(dev); >>>> +} >>>> +#endif /* ENABLE_HOTPLUG */ >>>> + >>>> /* >>>> * If vendor/device ID match, call the devinit() function of the >>>> * driver. >>>> diff --git a/lib/librte_eal/linuxapp/eal/eal_pci_init.h b/lib/librte_eal/linuxapp/eal/eal_pci_init.h >>>> index 1070eb8..5152a0b 100644 >>>> --- a/lib/librte_eal/linuxapp/eal/eal_pci_init.h >>>> +++ b/lib/librte_eal/linuxapp/eal/eal_pci_init.h >>>> @@ -34,6 +34,7 @@ >>>> #ifndef EAL_PCI_INIT_H_ >>>> #define EAL_PCI_INIT_H_ >>>> >>>> +#include <rte_dev_hotplug.h> >>>> #include "eal_vfio.h" >>>> >>>> struct pci_map { >>>> @@ -71,6 +72,13 @@ void *pci_map_resource(void *requested_addr, int fd, off_t offset, >>>> /* map IGB_UIO resource prototype */ >>>> int pci_uio_map_resource(struct rte_pci_device *dev); >>>> >>>> +#ifdef ENABLE_HOTPLUG >>>> +void pci_unmap_resource(void *requested_addr, size_t size); >>>> + >>>> +/* unmap IGB_UIO resource prototype */ >>>> +void pci_uio_unmap_resource(struct rte_pci_device *dev); >>>> +#endif /* ENABLE_HOTPLUG */ >>>> + >>>> #ifdef VFIO_PRESENT >>>> >>>> #define VFIO_MAX_GROUPS 64 >>>> diff --git a/lib/librte_eal/linuxapp/eal/eal_pci_uio.c b/lib/librte_eal/linuxapp/eal/eal_pci_uio.c >>>> index 1da3507..81830d1 100644 >>>> --- a/lib/librte_eal/linuxapp/eal/eal_pci_uio.c >>>> +++ b/lib/librte_eal/linuxapp/eal/eal_pci_uio.c >>>> @@ -404,6 +404,71 @@ pci_uio_map_resource(struct rte_pci_device *dev) >>>> return 0; >>>> } >>>> >>>> +#ifdef ENABLE_HOTPLUG >>>> +static void >>>> +pci_uio_unmap(struct mapped_pci_resource *uio_res) >>>> +{ >>>> + int i; >>>> + >>>> + if (uio_res == NULL) >>>> + return; >>>> + >>>> + for (i = 0; i != uio_res->nb_maps; i++) >>>> + pci_unmap_resource(uio_res->maps[i].addr, >>>> + (size_t)uio_res->maps[i].size); >>>> +} >>>> + >>>> +static struct mapped_pci_resource * >>>> +pci_uio_find_resource(struct rte_pci_device *dev) >>>> +{ >>>> + struct mapped_pci_resource *uio_res; >>>> + >>>> + if (dev == NULL) >>>> + return NULL; >>>> + >>>> + TAILQ_FOREACH(uio_res, pci_res_list, next) { >>>> + >>>> + /* skip this element if it doesn't match our PCI address */ >>>> + if (!eal_compare_pci_addr(&uio_res->pci_addr, &dev->addr)) >>>> + return uio_res; >>>> + } >>>> + return NULL; >>>> +} >>>> + >>>> +/* unmap the PCI resource of a PCI device in virtual memory */ >>>> +void >>>> +pci_uio_unmap_resource(struct rte_pci_device *dev) >>>> +{ >>>> + struct mapped_pci_resource *uio_res; >>>> + >>>> + if (dev == NULL) >>>> + return; >>>> + >>>> + /* find an entry for the device */ >>>> + uio_res = pci_uio_find_resource(dev); >>>> + if (uio_res == NULL) >>>> + return; >>>> + >>>> + /* secondary processes - just free maps */ >>>> + if (rte_eal_process_type() != RTE_PROC_PRIMARY) >>>> + return pci_uio_unmap(uio_res); >>>> + >>>> + TAILQ_REMOVE(pci_res_list, uio_res, next); >>>> + >>>> + /* unmap all resources */ >>>> + pci_uio_unmap(uio_res); >>>> + >>>> + /* free uio resource */ >>>> + rte_free(uio_res); >>>> + >>>> + /* close fd if in primary process */ >>>> + close(dev->intr_handle.fd); >>>> + >>>> + dev->intr_handle.fd = -1; >>>> + dev->intr_handle.type = RTE_INTR_HANDLE_UNKNOWN; >>>> +} >>>> +#endif /* ENABLE_HOTPLUG */ >>>> + >>>> /* >>>> * parse a sysfs file containing one integer value >>>> * different to the eal version, as it needs to work with 64-bit values >>
On 1/22/2015 6:16 PM, Tetsuya Mukawa wrote: > Hi Michael, > > On 2015/01/22 17:12, Qiu, Michael wrote: >> On 1/21/2015 6:01 PM, Tetsuya Mukawa wrote: >>> Hi Michael, >>> >>> On 2015/01/20 18:23, Qiu, Michael wrote: >>>> On 1/19/2015 6:42 PM, Tetsuya Mukawa wrote: >>>>> The patch adds functions for unmapping igb_uio resources. The patch is only >>>>> for Linux and igb_uio environment. VFIO and BSD are not supported. >>>>> >>>>> v4: >>>>> - Add paramerter checking. >>>>> - Add header file to determine if hotplug can be enabled. >>>>> >>>>> Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp> >>>>> --- >>>>> lib/librte_eal/common/Makefile | 1 + >>>>> lib/librte_eal/common/include/rte_dev_hotplug.h | 44 +++++++++++++++++ >>>>> lib/librte_eal/linuxapp/eal/eal_pci.c | 38 +++++++++++++++ >>>>> lib/librte_eal/linuxapp/eal/eal_pci_init.h | 8 +++ >>>>> lib/librte_eal/linuxapp/eal/eal_pci_uio.c | 65 +++++++++++++++++++++++++ >>>>> 5 files changed, 156 insertions(+) >>>>> create mode 100644 lib/librte_eal/common/include/rte_dev_hotplug.h >>>>> >>>>> diff --git a/lib/librte_eal/common/Makefile b/lib/librte_eal/common/Makefile >>>>> index 52c1a5f..db7cc93 100644 >>>>> --- a/lib/librte_eal/common/Makefile >>>>> +++ b/lib/librte_eal/common/Makefile >>>>> @@ -41,6 +41,7 @@ INC += rte_eal_memconfig.h rte_malloc_heap.h >>>>> INC += rte_hexdump.h rte_devargs.h rte_dev.h >>>>> INC += rte_common_vect.h >>>>> INC += rte_pci_dev_feature_defs.h rte_pci_dev_features.h >>>>> +INC += rte_dev_hotplug.h >>>>> >>>>> ifeq ($(CONFIG_RTE_INSECURE_FUNCTION_WARNING),y) >>>>> INC += rte_warnings.h >>>>> diff --git a/lib/librte_eal/common/include/rte_dev_hotplug.h b/lib/librte_eal/common/include/rte_dev_hotplug.h >>>>> new file mode 100644 >>>>> index 0000000..b333e0f >>>>> --- /dev/null >>>>> +++ b/lib/librte_eal/common/include/rte_dev_hotplug.h >>>>> @@ -0,0 +1,44 @@ >>>>> +/*- >>>>> + * BSD LICENSE >>>>> + * >>>>> + * Copyright(c) 2015 IGEL Co.,LTd. >>>>> + * All rights reserved. >>>>> + * >>>>> + * Redistribution and use in source and binary forms, with or without >>>>> + * modification, are permitted provided that the following conditions >>>>> + * are met: >>>>> + * >>>>> + * * Redistributions of source code must retain the above copyright >>>>> + * notice, this list of conditions and the following disclaimer. >>>>> + * * Redistributions in binary form must reproduce the above copyright >>>>> + * notice, this list of conditions and the following disclaimer in >>>>> + * the documentation and/or other materials provided with the >>>>> + * distribution. >>>>> + * * Neither the name of IGEL Co.,Ltd. nor the names of its >>>>> + * contributors may be used to endorse or promote products derived >>>>> + * from this software without specific prior written permission. >>>>> + * >>>>> + * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS >>>>> + * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT >>>>> + * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR >>>>> + * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT >>>>> + * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, >>>>> + * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT >>>>> + * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, >>>>> + * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY >>>>> + * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT >>>>> + * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE >>>>> + * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. >>>>> + */ >>>>> + >>>>> +#ifndef _RTE_DEV_HOTPLUG_H_ >>>>> +#define _RTE_DEV_HOTPLUG_H_ >>>>> + >>>>> +/* >>>>> + * determine if hotplug can be enabled on the system >>>>> + */ >>>>> +#if defined(RTE_LIBRTE_EAL_HOTPLUG) && defined(RTE_LIBRTE_EAL_LINUXAPP) >>>> As you said, VFIO should not work with it, so does it need to add the >>>> vfio check here? >>> Could I have a advice of you? >>> First I guess it's the best to include "eal_vfio.h" here, and add >>> checking of VFIO_PRESENT macro. >> I have a question, will your hotplug feature support freebsd ? >> >> If not, how about to put it in "lib/librte_eal/linuxapp/eal/" ? Also >> include attach or detach affairs. > I appreciate your comments. > > So far, FreeBSD doesn't support PCI hotplug. So I didn't implement code > for FreeBSD. > But in the future, I want to implement it when FreeBSD supports it. > Also my hotplug implementation depends on legacy code already > implemented in common layer. > Anyway, for me it's nice to implement the feature in common layer. > >>> But it seems I cannot reach "eal_vfio.h" from this file. >> Yes, you can't :) >> >>> My second option is just checking RTE_EAL_VFIO macro. >>> But according to "eal_vfio.h", if kernel is under 3.6.0, VFIO_PRESENT >> Actually, in my opinion, whatever vfio or uio, only need be care in >> runtime. >> >> DPDK to check vfio only to add support for vfio, but this does not >> means the device will use vfio, >> >> So even if VFIO_PRESENT is defined, and vfio is enabled, but the device >> is bind to igb_uio, then your hotplug still need work, but if it bind >> to vfio, will not, am I right? >> >> If yes, I'm not sure if your hotplug has this ability, but it is >> reasonable, I think. > I agree with your concept. But I guess it's not only related with > hotplug function. > The hotplug implementation depends on legacy functions that is for > probing device. > To implement above concept will change not only hotplug behavior but > also legacy device probing. > > Conceptually I agree with such a functionality, but legacy probing > function doesn't have such a feature so far. > So I guess it's nice to separate this feature from hotplug patches. > Realistically, the hotplug patches are big, and it's a bit hard to add > and manage one more feature. > If it's ok to separate the patch, it's helpful to manage patches. > >>> will not be defined even when RTE_EAL_VFIO is enabled. >>> So I guess simply macro checking will not work correctly. >>> >>> Anyway, here are my implementation choices so far. >>> >>> 1. Like "eal_vfio.h", check kernel version in "rte_dev_hotplug.h". >>> In this case, if "eal_vfio.h" is changed, "rte_edv_hotplug.h" may need >>> to be changed also. >>> >>> 2. Merge "eal_vfio.h" and "rte_dev_hotplug.h" definitions, and define >>> these in new rte header like "rte_settings.h". >>> >>> Can I have advice about it? >> As I said, VFIO enable or not is not related for your hotplug, only the >> devices managed by VFIO will affect your hotplug. >> >> If you agree, I think need discuss the details of it. > Yes, I agree with your concept. > And if you agree, I will implement it separately. > > To discuss how to handle VFIO and igb_uio devices in parallel, I guess > we need to > think about generic uio driver for pci devices. > I guess before applying uio generic patch, this kind of discussion will > be needed. > I hope igb_uio (and VFIO?) be obsolete after applying uio generic. No, igb_uio is similar to uio generic, but VFIO is another totally different way of UIO. So applying uio generic has no affect for VFIO > In the case, we don't need to think it. > > BTW, back to 'rte_dev_hotplug.h' discussion of hotplug patch. > If VFIO_PRESENT isn't checked here, attaching port will be successful > even if the device is under VFIO. > (Because we already has legacy code for probing VFIO device. The hotplug > function just re-use it.) > But we cannot detach the device, because there is no code for closing > VFIO device. > There is no crash when the VFIO device is detached, but some error > messages will be displayed. > So I guess this behavior is like your description. > How about it? Actually, what I'm considering is to add one flag like "int pt_mod" in "struct rte_pci_device", and give the value in pci scan stage, then it will reduce lots of work in EAL of VFIO checking, like PCI memory mapping stage, hotplug and so on. For hotplug, it can check the flag at runtime. And when hotplug supports VFIO, you can simply ignore this flag. Do you think it is valuable to do so? If yes I think I will make a patch for that, and send to you, you can post your patch set with that, or if you want implement it yourself, let me know. Thanks, Michael > Thanks, > Tetsuya > >> Thanks, >> Michael >>> Thanks, >>> Tetsuya >>> >>>> Thanks, >>>> Michael >>>>> +#define ENABLE_HOTPLUG >>>>> +#endif /* RTE_LIBRTE_EAL_HOTPLUG & RTE_LIBRTE_EAL_LINUXAPP */ >>>>> + >>>>> +#endif /* _RTE_DEV_HOTPLUG_H_ */ >>>>> diff --git a/lib/librte_eal/linuxapp/eal/eal_pci.c b/lib/librte_eal/linuxapp/eal/eal_pci.c >>>>> index 3d2d93c..52c464c 100644 >>>>> --- a/lib/librte_eal/linuxapp/eal/eal_pci.c >>>>> +++ b/lib/librte_eal/linuxapp/eal/eal_pci.c >>>>> @@ -137,6 +137,25 @@ pci_map_resource(void *requested_addr, int fd, off_t offset, size_t size) >>>>> return mapaddr; >>>>> } >>>>> >>>>> +#ifdef ENABLE_HOTPLUG >>>>> +/* unmap a particular resource */ >>>>> +void >>>>> +pci_unmap_resource(void *requested_addr, size_t size) >>>>> +{ >>>>> + if (requested_addr == NULL) >>>>> + return; >>>>> + >>>>> + /* Unmap the PCI memory resource of device */ >>>>> + if (munmap(requested_addr, size)) { >>>>> + RTE_LOG(ERR, EAL, "%s(): cannot munmap(%p, 0x%lx): %s\n", >>>>> + __func__, requested_addr, (unsigned long)size, >>>>> + strerror(errno)); >>>>> + } else >>>>> + RTE_LOG(DEBUG, EAL, " PCI memory mapped at %p\n", >>>>> + requested_addr); >>>>> +} >>>>> +#endif /* ENABLE_HOTPLUG */ >>>>> + >>>>> /* parse the "resource" sysfs file */ >>>>> #define IORESOURCE_MEM 0x00000200 >>>>> >>>>> @@ -510,6 +529,25 @@ pci_map_device(struct rte_pci_device *dev) >>>>> return 0; >>>>> } >>>>> >>>>> +#ifdef ENABLE_HOTPLUG >>>>> +static void >>>>> +pci_unmap_device(struct rte_pci_device *dev) >>>>> +{ >>>>> + if (dev == NULL) >>>>> + return; >>>>> + >>>>> + /* try unmapping the NIC resources using VFIO if it exists */ >>>>> +#ifdef VFIO_PRESENT >>>>> + if (pci_vfio_is_enabled()) { >>>>> + RTE_LOG(ERR, EAL, "%s() doesn't support vfio yet.\n", >>>>> + __func__); >>>>> + return; >>>>> + } >>>>> +#endif >>>>> + pci_uio_unmap_resource(dev); >>>>> +} >>>>> +#endif /* ENABLE_HOTPLUG */ >>>>> + >>>>> /* >>>>> * If vendor/device ID match, call the devinit() function of the >>>>> * driver. >>>>> diff --git a/lib/librte_eal/linuxapp/eal/eal_pci_init.h b/lib/librte_eal/linuxapp/eal/eal_pci_init.h >>>>> index 1070eb8..5152a0b 100644 >>>>> --- a/lib/librte_eal/linuxapp/eal/eal_pci_init.h >>>>> +++ b/lib/librte_eal/linuxapp/eal/eal_pci_init.h >>>>> @@ -34,6 +34,7 @@ >>>>> #ifndef EAL_PCI_INIT_H_ >>>>> #define EAL_PCI_INIT_H_ >>>>> >>>>> +#include <rte_dev_hotplug.h> >>>>> #include "eal_vfio.h" >>>>> >>>>> struct pci_map { >>>>> @@ -71,6 +72,13 @@ void *pci_map_resource(void *requested_addr, int fd, off_t offset, >>>>> /* map IGB_UIO resource prototype */ >>>>> int pci_uio_map_resource(struct rte_pci_device *dev); >>>>> >>>>> +#ifdef ENABLE_HOTPLUG >>>>> +void pci_unmap_resource(void *requested_addr, size_t size); >>>>> + >>>>> +/* unmap IGB_UIO resource prototype */ >>>>> +void pci_uio_unmap_resource(struct rte_pci_device *dev); >>>>> +#endif /* ENABLE_HOTPLUG */ >>>>> + >>>>> #ifdef VFIO_PRESENT >>>>> >>>>> #define VFIO_MAX_GROUPS 64 >>>>> diff --git a/lib/librte_eal/linuxapp/eal/eal_pci_uio.c b/lib/librte_eal/linuxapp/eal/eal_pci_uio.c >>>>> index 1da3507..81830d1 100644 >>>>> --- a/lib/librte_eal/linuxapp/eal/eal_pci_uio.c >>>>> +++ b/lib/librte_eal/linuxapp/eal/eal_pci_uio.c >>>>> @@ -404,6 +404,71 @@ pci_uio_map_resource(struct rte_pci_device *dev) >>>>> return 0; >>>>> } >>>>> >>>>> +#ifdef ENABLE_HOTPLUG >>>>> +static void >>>>> +pci_uio_unmap(struct mapped_pci_resource *uio_res) >>>>> +{ >>>>> + int i; >>>>> + >>>>> + if (uio_res == NULL) >>>>> + return; >>>>> + >>>>> + for (i = 0; i != uio_res->nb_maps; i++) >>>>> + pci_unmap_resource(uio_res->maps[i].addr, >>>>> + (size_t)uio_res->maps[i].size); >>>>> +} >>>>> + >>>>> +static struct mapped_pci_resource * >>>>> +pci_uio_find_resource(struct rte_pci_device *dev) >>>>> +{ >>>>> + struct mapped_pci_resource *uio_res; >>>>> + >>>>> + if (dev == NULL) >>>>> + return NULL; >>>>> + >>>>> + TAILQ_FOREACH(uio_res, pci_res_list, next) { >>>>> + >>>>> + /* skip this element if it doesn't match our PCI address */ >>>>> + if (!eal_compare_pci_addr(&uio_res->pci_addr, &dev->addr)) >>>>> + return uio_res; >>>>> + } >>>>> + return NULL; >>>>> +} >>>>> + >>>>> +/* unmap the PCI resource of a PCI device in virtual memory */ >>>>> +void >>>>> +pci_uio_unmap_resource(struct rte_pci_device *dev) >>>>> +{ >>>>> + struct mapped_pci_resource *uio_res; >>>>> + >>>>> + if (dev == NULL) >>>>> + return; >>>>> + >>>>> + /* find an entry for the device */ >>>>> + uio_res = pci_uio_find_resource(dev); >>>>> + if (uio_res == NULL) >>>>> + return; >>>>> + >>>>> + /* secondary processes - just free maps */ >>>>> + if (rte_eal_process_type() != RTE_PROC_PRIMARY) >>>>> + return pci_uio_unmap(uio_res); >>>>> + >>>>> + TAILQ_REMOVE(pci_res_list, uio_res, next); >>>>> + >>>>> + /* unmap all resources */ >>>>> + pci_uio_unmap(uio_res); >>>>> + >>>>> + /* free uio resource */ >>>>> + rte_free(uio_res); >>>>> + >>>>> + /* close fd if in primary process */ >>>>> + close(dev->intr_handle.fd); >>>>> + >>>>> + dev->intr_handle.fd = -1; >>>>> + dev->intr_handle.type = RTE_INTR_HANDLE_UNKNOWN; >>>>> +} >>>>> +#endif /* ENABLE_HOTPLUG */ >>>>> + >>>>> /* >>>>> * parse a sysfs file containing one integer value >>>>> * different to the eal version, as it needs to work with 64-bit values > >
Hi Michael, On 2015/01/23 11:54, Qiu, Michael wrote: > On 1/22/2015 6:16 PM, Tetsuya Mukawa wrote: >> Hi Michael, >> >> On 2015/01/22 17:12, Qiu, Michael wrote: >>> On 1/21/2015 6:01 PM, Tetsuya Mukawa wrote: >>>> Hi Michael, >>>> >>>> On 2015/01/20 18:23, Qiu, Michael wrote: >>>>> On 1/19/2015 6:42 PM, Tetsuya Mukawa wrote: >>>>>> The patch adds functions for unmapping igb_uio resources. The patch is only >>>>>> for Linux and igb_uio environment. VFIO and BSD are not supported. >>>>>> >>>>>> v4: >>>>>> - Add paramerter checking. >>>>>> - Add header file to determine if hotplug can be enabled. >>>>>> >>>>>> Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp> >>>>>> --- >>>>>> lib/librte_eal/common/Makefile | 1 + >>>>>> lib/librte_eal/common/include/rte_dev_hotplug.h | 44 +++++++++++++++++ >>>>>> lib/librte_eal/linuxapp/eal/eal_pci.c | 38 +++++++++++++++ >>>>>> lib/librte_eal/linuxapp/eal/eal_pci_init.h | 8 +++ >>>>>> lib/librte_eal/linuxapp/eal/eal_pci_uio.c | 65 +++++++++++++++++++++++++ >>>>>> 5 files changed, 156 insertions(+) >>>>>> create mode 100644 lib/librte_eal/common/include/rte_dev_hotplug.h >>>>>> >>>>>> diff --git a/lib/librte_eal/common/Makefile b/lib/librte_eal/common/Makefile >>>>>> index 52c1a5f..db7cc93 100644 >>>>>> --- a/lib/librte_eal/common/Makefile >>>>>> +++ b/lib/librte_eal/common/Makefile >>>>>> @@ -41,6 +41,7 @@ INC += rte_eal_memconfig.h rte_malloc_heap.h >>>>>> INC += rte_hexdump.h rte_devargs.h rte_dev.h >>>>>> INC += rte_common_vect.h >>>>>> INC += rte_pci_dev_feature_defs.h rte_pci_dev_features.h >>>>>> +INC += rte_dev_hotplug.h >>>>>> >>>>>> ifeq ($(CONFIG_RTE_INSECURE_FUNCTION_WARNING),y) >>>>>> INC += rte_warnings.h >>>>>> diff --git a/lib/librte_eal/common/include/rte_dev_hotplug.h b/lib/librte_eal/common/include/rte_dev_hotplug.h >>>>>> new file mode 100644 >>>>>> index 0000000..b333e0f >>>>>> --- /dev/null >>>>>> +++ b/lib/librte_eal/common/include/rte_dev_hotplug.h >>>>>> @@ -0,0 +1,44 @@ >>>>>> +/*- >>>>>> + * BSD LICENSE >>>>>> + * >>>>>> + * Copyright(c) 2015 IGEL Co.,LTd. >>>>>> + * All rights reserved. >>>>>> + * >>>>>> + * Redistribution and use in source and binary forms, with or without >>>>>> + * modification, are permitted provided that the following conditions >>>>>> + * are met: >>>>>> + * >>>>>> + * * Redistributions of source code must retain the above copyright >>>>>> + * notice, this list of conditions and the following disclaimer. >>>>>> + * * Redistributions in binary form must reproduce the above copyright >>>>>> + * notice, this list of conditions and the following disclaimer in >>>>>> + * the documentation and/or other materials provided with the >>>>>> + * distribution. >>>>>> + * * Neither the name of IGEL Co.,Ltd. nor the names of its >>>>>> + * contributors may be used to endorse or promote products derived >>>>>> + * from this software without specific prior written permission. >>>>>> + * >>>>>> + * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS >>>>>> + * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT >>>>>> + * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR >>>>>> + * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT >>>>>> + * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, >>>>>> + * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT >>>>>> + * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, >>>>>> + * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY >>>>>> + * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT >>>>>> + * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE >>>>>> + * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. >>>>>> + */ >>>>>> + >>>>>> +#ifndef _RTE_DEV_HOTPLUG_H_ >>>>>> +#define _RTE_DEV_HOTPLUG_H_ >>>>>> + >>>>>> +/* >>>>>> + * determine if hotplug can be enabled on the system >>>>>> + */ >>>>>> +#if defined(RTE_LIBRTE_EAL_HOTPLUG) && defined(RTE_LIBRTE_EAL_LINUXAPP) >>>>> As you said, VFIO should not work with it, so does it need to add the >>>>> vfio check here? >>>> Could I have a advice of you? >>>> First I guess it's the best to include "eal_vfio.h" here, and add >>>> checking of VFIO_PRESENT macro. >>> I have a question, will your hotplug feature support freebsd ? >>> >>> If not, how about to put it in "lib/librte_eal/linuxapp/eal/" ? Also >>> include attach or detach affairs. >> I appreciate your comments. >> >> So far, FreeBSD doesn't support PCI hotplug. So I didn't implement code >> for FreeBSD. >> But in the future, I want to implement it when FreeBSD supports it. >> Also my hotplug implementation depends on legacy code already >> implemented in common layer. >> Anyway, for me it's nice to implement the feature in common layer. >> >>>> But it seems I cannot reach "eal_vfio.h" from this file. >>> Yes, you can't :) >>> >>>> My second option is just checking RTE_EAL_VFIO macro. >>>> But according to "eal_vfio.h", if kernel is under 3.6.0, VFIO_PRESENT >>> Actually, in my opinion, whatever vfio or uio, only need be care in >>> runtime. >>> >>> DPDK to check vfio only to add support for vfio, but this does not >>> means the device will use vfio, >>> >>> So even if VFIO_PRESENT is defined, and vfio is enabled, but the device >>> is bind to igb_uio, then your hotplug still need work, but if it bind >>> to vfio, will not, am I right? >>> >>> If yes, I'm not sure if your hotplug has this ability, but it is >>> reasonable, I think. >> I agree with your concept. But I guess it's not only related with >> hotplug function. >> The hotplug implementation depends on legacy functions that is for >> probing device. >> To implement above concept will change not only hotplug behavior but >> also legacy device probing. >> >> Conceptually I agree with such a functionality, but legacy probing >> function doesn't have such a feature so far. >> So I guess it's nice to separate this feature from hotplug patches. >> Realistically, the hotplug patches are big, and it's a bit hard to add >> and manage one more feature. >> If it's ok to separate the patch, it's helpful to manage patches. >> >>>> will not be defined even when RTE_EAL_VFIO is enabled. >>>> So I guess simply macro checking will not work correctly. >>>> >>>> Anyway, here are my implementation choices so far. >>>> >>>> 1. Like "eal_vfio.h", check kernel version in "rte_dev_hotplug.h". >>>> In this case, if "eal_vfio.h" is changed, "rte_edv_hotplug.h" may need >>>> to be changed also. >>>> >>>> 2. Merge "eal_vfio.h" and "rte_dev_hotplug.h" definitions, and define >>>> these in new rte header like "rte_settings.h". >>>> >>>> Can I have advice about it? >>> As I said, VFIO enable or not is not related for your hotplug, only the >>> devices managed by VFIO will affect your hotplug. >>> >>> If you agree, I think need discuss the details of it. >> Yes, I agree with your concept. >> And if you agree, I will implement it separately. >> >> To discuss how to handle VFIO and igb_uio devices in parallel, I guess >> we need to >> think about generic uio driver for pci devices. >> I guess before applying uio generic patch, this kind of discussion will >> be needed. >> I hope igb_uio (and VFIO?) be obsolete after applying uio generic. > No, igb_uio is similar to uio generic, but VFIO is another totally > different way of UIO. > > So applying uio generic has no affect for VFIO I appreciate your comments. OK, I understand the relation of VFIO and uio generic. > >> In the case, we don't need to think it. >> >> BTW, back to 'rte_dev_hotplug.h' discussion of hotplug patch. >> If VFIO_PRESENT isn't checked here, attaching port will be successful >> even if the device is under VFIO. >> (Because we already has legacy code for probing VFIO device. The hotplug >> function just re-use it.) >> But we cannot detach the device, because there is no code for closing >> VFIO device. >> There is no crash when the VFIO device is detached, but some error >> messages will be displayed. >> So I guess this behavior is like your description. >> How about it? > Actually, what I'm considering is to add one flag like "int pt_mod" in > "struct rte_pci_device", and give the value in pci scan stage, then it > will reduce lots of work in EAL of VFIO checking, like PCI memory > mapping stage, hotplug and so on. > For hotplug, it can check the flag at runtime. And when hotplug supports > VFIO, you can simply ignore this flag. > > Do you think it is valuable to do so? > > If yes I think I will make a patch for that, and send to you, you can > post your patch set with that, or if you want implement it yourself, let > me know. I agree with you. And thank you so much. Could you please send a patch? I will put it on the top of my patches. (If you have favorite where you want to put it on, could you please let me know?) Thanks, Tetsuya > Thanks, > Michael >> Thanks, >> Tetsuya >> >>> Thanks, >>> Michael >>>> Thanks, >>>> Tetsuya >>>> >>>>> Thanks, >>>>> Michael >>>>>> +#define ENABLE_HOTPLUG >>>>>> +#endif /* RTE_LIBRTE_EAL_HOTPLUG & RTE_LIBRTE_EAL_LINUXAPP */ >>>>>> + >>>>>> +#endif /* _RTE_DEV_HOTPLUG_H_ */ >>>>>> diff --git a/lib/librte_eal/linuxapp/eal/eal_pci.c b/lib/librte_eal/linuxapp/eal/eal_pci.c >>>>>> index 3d2d93c..52c464c 100644 >>>>>> --- a/lib/librte_eal/linuxapp/eal/eal_pci.c >>>>>> +++ b/lib/librte_eal/linuxapp/eal/eal_pci.c >>>>>> @@ -137,6 +137,25 @@ pci_map_resource(void *requested_addr, int fd, off_t offset, size_t size) >>>>>> return mapaddr; >>>>>> } >>>>>> >>>>>> +#ifdef ENABLE_HOTPLUG >>>>>> +/* unmap a particular resource */ >>>>>> +void >>>>>> +pci_unmap_resource(void *requested_addr, size_t size) >>>>>> +{ >>>>>> + if (requested_addr == NULL) >>>>>> + return; >>>>>> + >>>>>> + /* Unmap the PCI memory resource of device */ >>>>>> + if (munmap(requested_addr, size)) { >>>>>> + RTE_LOG(ERR, EAL, "%s(): cannot munmap(%p, 0x%lx): %s\n", >>>>>> + __func__, requested_addr, (unsigned long)size, >>>>>> + strerror(errno)); >>>>>> + } else >>>>>> + RTE_LOG(DEBUG, EAL, " PCI memory mapped at %p\n", >>>>>> + requested_addr); >>>>>> +} >>>>>> +#endif /* ENABLE_HOTPLUG */ >>>>>> + >>>>>> /* parse the "resource" sysfs file */ >>>>>> #define IORESOURCE_MEM 0x00000200 >>>>>> >>>>>> @@ -510,6 +529,25 @@ pci_map_device(struct rte_pci_device *dev) >>>>>> return 0; >>>>>> } >>>>>> >>>>>> +#ifdef ENABLE_HOTPLUG >>>>>> +static void >>>>>> +pci_unmap_device(struct rte_pci_device *dev) >>>>>> +{ >>>>>> + if (dev == NULL) >>>>>> + return; >>>>>> + >>>>>> + /* try unmapping the NIC resources using VFIO if it exists */ >>>>>> +#ifdef VFIO_PRESENT >>>>>> + if (pci_vfio_is_enabled()) { >>>>>> + RTE_LOG(ERR, EAL, "%s() doesn't support vfio yet.\n", >>>>>> + __func__); >>>>>> + return; >>>>>> + } >>>>>> +#endif >>>>>> + pci_uio_unmap_resource(dev); >>>>>> +} >>>>>> +#endif /* ENABLE_HOTPLUG */ >>>>>> + >>>>>> /* >>>>>> * If vendor/device ID match, call the devinit() function of the >>>>>> * driver. >>>>>> diff --git a/lib/librte_eal/linuxapp/eal/eal_pci_init.h b/lib/librte_eal/linuxapp/eal/eal_pci_init.h >>>>>> index 1070eb8..5152a0b 100644 >>>>>> --- a/lib/librte_eal/linuxapp/eal/eal_pci_init.h >>>>>> +++ b/lib/librte_eal/linuxapp/eal/eal_pci_init.h >>>>>> @@ -34,6 +34,7 @@ >>>>>> #ifndef EAL_PCI_INIT_H_ >>>>>> #define EAL_PCI_INIT_H_ >>>>>> >>>>>> +#include <rte_dev_hotplug.h> >>>>>> #include "eal_vfio.h" >>>>>> >>>>>> struct pci_map { >>>>>> @@ -71,6 +72,13 @@ void *pci_map_resource(void *requested_addr, int fd, off_t offset, >>>>>> /* map IGB_UIO resource prototype */ >>>>>> int pci_uio_map_resource(struct rte_pci_device *dev); >>>>>> >>>>>> +#ifdef ENABLE_HOTPLUG >>>>>> +void pci_unmap_resource(void *requested_addr, size_t size); >>>>>> + >>>>>> +/* unmap IGB_UIO resource prototype */ >>>>>> +void pci_uio_unmap_resource(struct rte_pci_device *dev); >>>>>> +#endif /* ENABLE_HOTPLUG */ >>>>>> + >>>>>> #ifdef VFIO_PRESENT >>>>>> >>>>>> #define VFIO_MAX_GROUPS 64 >>>>>> diff --git a/lib/librte_eal/linuxapp/eal/eal_pci_uio.c b/lib/librte_eal/linuxapp/eal/eal_pci_uio.c >>>>>> index 1da3507..81830d1 100644 >>>>>> --- a/lib/librte_eal/linuxapp/eal/eal_pci_uio.c >>>>>> +++ b/lib/librte_eal/linuxapp/eal/eal_pci_uio.c >>>>>> @@ -404,6 +404,71 @@ pci_uio_map_resource(struct rte_pci_device *dev) >>>>>> return 0; >>>>>> } >>>>>> >>>>>> +#ifdef ENABLE_HOTPLUG >>>>>> +static void >>>>>> +pci_uio_unmap(struct mapped_pci_resource *uio_res) >>>>>> +{ >>>>>> + int i; >>>>>> + >>>>>> + if (uio_res == NULL) >>>>>> + return; >>>>>> + >>>>>> + for (i = 0; i != uio_res->nb_maps; i++) >>>>>> + pci_unmap_resource(uio_res->maps[i].addr, >>>>>> + (size_t)uio_res->maps[i].size); >>>>>> +} >>>>>> + >>>>>> +static struct mapped_pci_resource * >>>>>> +pci_uio_find_resource(struct rte_pci_device *dev) >>>>>> +{ >>>>>> + struct mapped_pci_resource *uio_res; >>>>>> + >>>>>> + if (dev == NULL) >>>>>> + return NULL; >>>>>> + >>>>>> + TAILQ_FOREACH(uio_res, pci_res_list, next) { >>>>>> + >>>>>> + /* skip this element if it doesn't match our PCI address */ >>>>>> + if (!eal_compare_pci_addr(&uio_res->pci_addr, &dev->addr)) >>>>>> + return uio_res; >>>>>> + } >>>>>> + return NULL; >>>>>> +} >>>>>> + >>>>>> +/* unmap the PCI resource of a PCI device in virtual memory */ >>>>>> +void >>>>>> +pci_uio_unmap_resource(struct rte_pci_device *dev) >>>>>> +{ >>>>>> + struct mapped_pci_resource *uio_res; >>>>>> + >>>>>> + if (dev == NULL) >>>>>> + return; >>>>>> + >>>>>> + /* find an entry for the device */ >>>>>> + uio_res = pci_uio_find_resource(dev); >>>>>> + if (uio_res == NULL) >>>>>> + return; >>>>>> + >>>>>> + /* secondary processes - just free maps */ >>>>>> + if (rte_eal_process_type() != RTE_PROC_PRIMARY) >>>>>> + return pci_uio_unmap(uio_res); >>>>>> + >>>>>> + TAILQ_REMOVE(pci_res_list, uio_res, next); >>>>>> + >>>>>> + /* unmap all resources */ >>>>>> + pci_uio_unmap(uio_res); >>>>>> + >>>>>> + /* free uio resource */ >>>>>> + rte_free(uio_res); >>>>>> + >>>>>> + /* close fd if in primary process */ >>>>>> + close(dev->intr_handle.fd); >>>>>> + >>>>>> + dev->intr_handle.fd = -1; >>>>>> + dev->intr_handle.type = RTE_INTR_HANDLE_UNKNOWN; >>>>>> +} >>>>>> +#endif /* ENABLE_HOTPLUG */ >>>>>> + >>>>>> /* >>>>>> * parse a sysfs file containing one integer value >>>>>> * different to the eal version, as it needs to work with 64-bit values >>
On 1/19/2015 6:42 PM, Tetsuya Mukawa wrote: > This patch series adds a dynamic port hotplug framework to DPDK. > With the patches, DPDK apps can attach or detach ports at runtime. > > The basic concept of the port hotplug is like followings. > - DPDK apps must have responsibility to manage ports. > DPDK apps only know which ports are attached or detached at the moment. > The port hotplug framework is implemented to allow DPDK apps to manage ports. > For example, when DPDK apps call port attach function, attached port number > will be returned. Also DPDK apps can detach port by port number. > - Kernel support is needed for attaching or detaching physical device ports. > To attach new device, the device will be recognized by kernel at first and > controlled by kernel driver. Then user can bind the device to igb_uio Here does it really need native kernel driver here? As it will be controlled by igb_uio. I think even if the device has no kernel driver is also OK. Also I have finished initial patch of passthrough driver flag in "struct rte_pci_device" I will send to you after I do some basic test on that, then I will send to you, and you can give some comments on that. Thanks, Michael > by 'dpdk_nic_bind.py'. Finally, DPDK apps can call the port hotplug > functions to attach ports. > For detaching, steps are vice versa. > - Before detach ports, ports must be stopped and closed. > DPDK application must call rte_eth_dev_stop() and rte_eth_dev_close() before > detaching ports. These function will call finalization codes of PMDs. > But so far, no PMD frees all resources allocated by initialization. > It means PMDs are needed to be fixed to support the port hotplug. > 'RTE_PCI_DRV_DETACHABLE' is a new flag indicating a PMD supports detaching. > Without this flag, detaching will be failed. > - Mustn't affect legacy DPDK apps. > No DPDK EAL behavior is changed, if the port hotplug functions are't called. > So all legacy DPDK apps can still work without modifications. > > And a few limitations. > - The port hotplug functions are not thread safe. > DPDK apps should handle it. > - Only support Linux and igb_uio so far. > BSD and VFIO is not supported. I will send VFIO patches at least, but I don't > have a plan to submit BSD patch so far. > > > Here is port hotplug APIs. > ------------------------------------------------------------------------------- > /** > * Attach a new device. > * > * @param devargs > * A pointer to a strings array describing the new device > * to be attached. The strings should be a pci address like > * '0000:01:00.0' or virtual device name like 'eth_pcap0'. > * @param port_id > * A pointer to a port identifier actually attached. > * @return > * 0 on success and port_id is filled, negative on error > */ > int rte_eal_dev_attach(const char *devargs, uint8_t *port_id); > > /** > * Detach a device. > * > * @param port_id > * The port identifier of the device to detach. > * @param addr > * A pointer to a device name actually detached. > * @return > * 0 on success and devname is filled, negative on error > */ > int rte_eal_dev_detach(uint8_t port_id, char *devname); > ------------------------------------------------------------------------------- > > This patch series are for DPDK EAL. To use port hotplug function by DPDK apps, > each PMD should be fixed to support 'RTE_PCI_DRV_DETACHABLE' flag. Please check > a patch for pcap PMD. > > Also please check testpmd patch. It will show you how to fix your legacy > applications to support port hotplug feature. > > > PATCH v4 changes > - Merge patches to review easier. > - Fix indent of 'if' statement. > - Fix calculation method of eal_compare_pci_addr(). > - Fix header file declaration. > - Add header file to determine if hotplug can be enabled. > (Thanks to Qiu, Michael) > - Use braces with 'for' loop. > - Add paramerter checking. > - Fix sanity check code > - Fix comments of rte_eth_dev_type. > - Change function names. > (Thanks to Iremonger, Bernard) > > PATCH v3 changes: > - Fix enum definition used in rte_ethdev.c. > (Thanks to Zhang, Helin) > > PATCH v2 changes: > - Replace rte_eal_dev_attach_pdev(), rte_eal_dev_detach_pdev, > rte_eal_dev_attach_vdev() and rte_eal_dev_detach_vdev() to > rte_eal_dev_attach() and rte_eal_dev_detach(). > - Add parameter values checking. > - Refashion a few functions. > (Thanks to Iremonger, Bernard) > > PATCH v1 Changes: > - Fix error checking code of librte_eth APIs. > - Fix issue that port from pcap PMD cannot be detached correctly. > - Fix issue that testpmd could hang after forwarding, if attaching and detaching > is repeatedly. > - Fix if-condition of rte_eth_dev_get_port_by_addr(). > (Thanks to Mark Enright) > > RFC PATCH v2 Changes: > - remove 'rte_eth_dev_validate_port()', and cleanup codes. > > > Tetsuya Mukawa (11): > eal/pci,ethdev: Remove assumption that port will not be detached > eal/pci: Consolidate pci address comparison APIs > ethdev: Add rte_eth_dev_free to free specified device > eal,ethdev: Add a function and function pointers to close ether device > ethdev: Add functions that will be used by port hotplug functions > eal/linux/pci: Add functions for unmapping igb_uio resources > eal/pci: Add a function to remove the entry of devargs list > eal/pci: Cleanup pci driver initialization code > ethdev: Add one dev_type paramerter to rte_eth_dev_allocate > eal/pci: Add rte_eal_dev_attach/detach() functions > eal: Enable port hotplug framework in Linux > > app/test/virtual_pmd.c | 2 +- > config/common_linuxapp | 5 + > lib/librte_eal/bsdapp/eal/eal_pci.c | 16 +- > lib/librte_eal/common/Makefile | 1 + > lib/librte_eal/common/eal_common_dev.c | 273 +++++++++++++++ > lib/librte_eal/common/eal_common_devargs.c | 57 ++++ > lib/librte_eal/common/eal_common_pci.c | 92 ++++- > lib/librte_eal/common/eal_private.h | 36 ++ > lib/librte_eal/common/include/rte_dev.h | 33 ++ > lib/librte_eal/common/include/rte_dev_hotplug.h | 44 +++ > lib/librte_eal/common/include/rte_devargs.h | 18 + > lib/librte_eal/common/include/rte_pci.h | 76 +++++ > lib/librte_eal/linuxapp/eal/Makefile | 1 + > lib/librte_eal/linuxapp/eal/eal_pci.c | 136 ++++++-- > lib/librte_eal/linuxapp/eal/eal_pci_init.h | 8 + > lib/librte_eal/linuxapp/eal/eal_pci_uio.c | 67 +++- > lib/librte_ether/rte_ethdev.c | 424 ++++++++++++++++++------ > lib/librte_ether/rte_ethdev.h | 145 +++++++- > lib/librte_pmd_af_packet/rte_eth_af_packet.c | 2 +- > lib/librte_pmd_bond/rte_eth_bond_api.c | 2 +- > lib/librte_pmd_pcap/rte_eth_pcap.c | 2 +- > lib/librte_pmd_ring/rte_eth_ring.c | 2 +- > lib/librte_pmd_xenvirt/rte_eth_xenvirt.c | 2 +- > 23 files changed, 1283 insertions(+), 161 deletions(-) > create mode 100644 lib/librte_eal/common/include/rte_dev_hotplug.h >
On 2015/01/27 12:00, Qiu, Michael wrote: > On 1/19/2015 6:42 PM, Tetsuya Mukawa wrote: >> This patch series adds a dynamic port hotplug framework to DPDK. >> With the patches, DPDK apps can attach or detach ports at runtime. >> >> The basic concept of the port hotplug is like followings. >> - DPDK apps must have responsibility to manage ports. >> DPDK apps only know which ports are attached or detached at the moment. >> The port hotplug framework is implemented to allow DPDK apps to manage ports. >> For example, when DPDK apps call port attach function, attached port number >> will be returned. Also DPDK apps can detach port by port number. >> - Kernel support is needed for attaching or detaching physical device ports. >> To attach new device, the device will be recognized by kernel at first and >> controlled by kernel driver. Then user can bind the device to igb_uio > Here does it really need native kernel driver here? As it will be > controlled by igb_uio. > I think even if the device has no kernel driver is also OK. Thanks for correcting. Yes, it should be. How about following. - Kernel support is needed for attaching or detaching physical device ports. To attach a new device, the device will be recognized by kernel PCI hotplug feature at first. Then user can bind the device to igb_uio. > > Also I have finished initial patch of passthrough driver flag in > "struct rte_pci_device" > > I will send to you after I do some basic test on that, then I will send to > you, and you can give some comments on that. I appreciate for your implementing. Thanks, Tetsuya > > Thanks, > Michael > >> by 'dpdk_nic_bind.py'. Finally, DPDK apps can call the port hotplug >> functions to attach ports. >> For detaching, steps are vice versa. >> - Before detach ports, ports must be stopped and closed. >> DPDK application must call rte_eth_dev_stop() and rte_eth_dev_close() before >> detaching ports. These function will call finalization codes of PMDs. >> But so far, no PMD frees all resources allocated by initialization. >> It means PMDs are needed to be fixed to support the port hotplug. >> 'RTE_PCI_DRV_DETACHABLE' is a new flag indicating a PMD supports detaching. >> Without this flag, detaching will be failed. >> - Mustn't affect legacy DPDK apps. >> No DPDK EAL behavior is changed, if the port hotplug functions are't called. >> So all legacy DPDK apps can still work without modifications. >> >> And a few limitations. >> - The port hotplug functions are not thread safe. >> DPDK apps should handle it. >> - Only support Linux and igb_uio so far. >> BSD and VFIO is not supported. I will send VFIO patches at least, but I don't >> have a plan to submit BSD patch so far. >> >> >> Here is port hotplug APIs. >> ------------------------------------------------------------------------------- >> /** >> * Attach a new device. >> * >> * @param devargs >> * A pointer to a strings array describing the new device >> * to be attached. The strings should be a pci address like >> * '0000:01:00.0' or virtual device name like 'eth_pcap0'. >> * @param port_id >> * A pointer to a port identifier actually attached. >> * @return >> * 0 on success and port_id is filled, negative on error >> */ >> int rte_eal_dev_attach(const char *devargs, uint8_t *port_id); >> >> /** >> * Detach a device. >> * >> * @param port_id >> * The port identifier of the device to detach. >> * @param addr >> * A pointer to a device name actually detached. >> * @return >> * 0 on success and devname is filled, negative on error >> */ >> int rte_eal_dev_detach(uint8_t port_id, char *devname); >> ------------------------------------------------------------------------------- >> >> This patch series are for DPDK EAL. To use port hotplug function by DPDK apps, >> each PMD should be fixed to support 'RTE_PCI_DRV_DETACHABLE' flag. Please check >> a patch for pcap PMD. >> >> Also please check testpmd patch. It will show you how to fix your legacy >> applications to support port hotplug feature. >> >> >> PATCH v4 changes >> - Merge patches to review easier. >> - Fix indent of 'if' statement. >> - Fix calculation method of eal_compare_pci_addr(). >> - Fix header file declaration. >> - Add header file to determine if hotplug can be enabled. >> (Thanks to Qiu, Michael) >> - Use braces with 'for' loop. >> - Add paramerter checking. >> - Fix sanity check code >> - Fix comments of rte_eth_dev_type. >> - Change function names. >> (Thanks to Iremonger, Bernard) >> >> PATCH v3 changes: >> - Fix enum definition used in rte_ethdev.c. >> (Thanks to Zhang, Helin) >> >> PATCH v2 changes: >> - Replace rte_eal_dev_attach_pdev(), rte_eal_dev_detach_pdev, >> rte_eal_dev_attach_vdev() and rte_eal_dev_detach_vdev() to >> rte_eal_dev_attach() and rte_eal_dev_detach(). >> - Add parameter values checking. >> - Refashion a few functions. >> (Thanks to Iremonger, Bernard) >> >> PATCH v1 Changes: >> - Fix error checking code of librte_eth APIs. >> - Fix issue that port from pcap PMD cannot be detached correctly. >> - Fix issue that testpmd could hang after forwarding, if attaching and detaching >> is repeatedly. >> - Fix if-condition of rte_eth_dev_get_port_by_addr(). >> (Thanks to Mark Enright) >> >> RFC PATCH v2 Changes: >> - remove 'rte_eth_dev_validate_port()', and cleanup codes. >> >> >> Tetsuya Mukawa (11): >> eal/pci,ethdev: Remove assumption that port will not be detached >> eal/pci: Consolidate pci address comparison APIs >> ethdev: Add rte_eth_dev_free to free specified device >> eal,ethdev: Add a function and function pointers to close ether device >> ethdev: Add functions that will be used by port hotplug functions >> eal/linux/pci: Add functions for unmapping igb_uio resources >> eal/pci: Add a function to remove the entry of devargs list >> eal/pci: Cleanup pci driver initialization code >> ethdev: Add one dev_type paramerter to rte_eth_dev_allocate >> eal/pci: Add rte_eal_dev_attach/detach() functions >> eal: Enable port hotplug framework in Linux >> >> app/test/virtual_pmd.c | 2 +- >> config/common_linuxapp | 5 + >> lib/librte_eal/bsdapp/eal/eal_pci.c | 16 +- >> lib/librte_eal/common/Makefile | 1 + >> lib/librte_eal/common/eal_common_dev.c | 273 +++++++++++++++ >> lib/librte_eal/common/eal_common_devargs.c | 57 ++++ >> lib/librte_eal/common/eal_common_pci.c | 92 ++++- >> lib/librte_eal/common/eal_private.h | 36 ++ >> lib/librte_eal/common/include/rte_dev.h | 33 ++ >> lib/librte_eal/common/include/rte_dev_hotplug.h | 44 +++ >> lib/librte_eal/common/include/rte_devargs.h | 18 + >> lib/librte_eal/common/include/rte_pci.h | 76 +++++ >> lib/librte_eal/linuxapp/eal/Makefile | 1 + >> lib/librte_eal/linuxapp/eal/eal_pci.c | 136 ++++++-- >> lib/librte_eal/linuxapp/eal/eal_pci_init.h | 8 + >> lib/librte_eal/linuxapp/eal/eal_pci_uio.c | 67 +++- >> lib/librte_ether/rte_ethdev.c | 424 ++++++++++++++++++------ >> lib/librte_ether/rte_ethdev.h | 145 +++++++- >> lib/librte_pmd_af_packet/rte_eth_af_packet.c | 2 +- >> lib/librte_pmd_bond/rte_eth_bond_api.c | 2 +- >> lib/librte_pmd_pcap/rte_eth_pcap.c | 2 +- >> lib/librte_pmd_ring/rte_eth_ring.c | 2 +- >> lib/librte_pmd_xenvirt/rte_eth_xenvirt.c | 2 +- >> 23 files changed, 1283 insertions(+), 161 deletions(-) >> create mode 100644 lib/librte_eal/common/include/rte_dev_hotplug.h >>
On 1/27/2015 1:02 PM, Tetsuya Mukawa wrote: > On 2015/01/27 12:00, Qiu, Michael wrote: >> On 1/19/2015 6:42 PM, Tetsuya Mukawa wrote: >>> This patch series adds a dynamic port hotplug framework to DPDK. >>> With the patches, DPDK apps can attach or detach ports at runtime. >>> >>> The basic concept of the port hotplug is like followings. >>> - DPDK apps must have responsibility to manage ports. >>> DPDK apps only know which ports are attached or detached at the moment. >>> The port hotplug framework is implemented to allow DPDK apps to manage ports. >>> For example, when DPDK apps call port attach function, attached port number >>> will be returned. Also DPDK apps can detach port by port number. >>> - Kernel support is needed for attaching or detaching physical device ports. >>> To attach new device, the device will be recognized by kernel at first and >>> controlled by kernel driver. Then user can bind the device to igb_uio >> Here does it really need native kernel driver here? As it will be >> controlled by igb_uio. >> I think even if the device has no kernel driver is also OK. > Thanks for correcting. Yes, it should be. > How about following. > > - Kernel support is needed for attaching or detaching physical device ports. > To attach a new device, the device will be recognized by kernel PCI > hotplug feature at first. No, here should not explain as "kernel PCI hotplug feature" which is stand for removing or adding a PCI device from system level, it is devices related not driver. What about: - Kernel support is needed for attaching or detaching physical device ports. To attach a new physical device port, the device will be recognized by userspace directly I/O framework in kernel at first. Thanks, Michael > Then user can bind the device to igb_uio. > >> Also I have finished initial patch of passthrough driver flag in >> "struct rte_pci_device" >> >> I will send to you after I do some basic test on that, then I will send to >> you, and you can give some comments on that. > I appreciate for your implementing. > > Thanks, > Tetsuya > > >> Thanks, >> Michael >> >>> by 'dpdk_nic_bind.py'. Finally, DPDK apps can call the port hotplug >>> functions to attach ports. >>> For detaching, steps are vice versa. >>> - Before detach ports, ports must be stopped and closed. >>> DPDK application must call rte_eth_dev_stop() and rte_eth_dev_close() before >>> detaching ports. These function will call finalization codes of PMDs. >>> But so far, no PMD frees all resources allocated by initialization. >>> It means PMDs are needed to be fixed to support the port hotplug. >>> 'RTE_PCI_DRV_DETACHABLE' is a new flag indicating a PMD supports detaching. >>> Without this flag, detaching will be failed. >>> - Mustn't affect legacy DPDK apps. >>> No DPDK EAL behavior is changed, if the port hotplug functions are't called. >>> So all legacy DPDK apps can still work without modifications. >>> >>> And a few limitations. >>> - The port hotplug functions are not thread safe. >>> DPDK apps should handle it. >>> - Only support Linux and igb_uio so far. >>> BSD and VFIO is not supported. I will send VFIO patches at least, but I don't >>> have a plan to submit BSD patch so far. >>> >>> >>> Here is port hotplug APIs. >>> ------------------------------------------------------------------------------- >>> /** >>> * Attach a new device. >>> * >>> * @param devargs >>> * A pointer to a strings array describing the new device >>> * to be attached. The strings should be a pci address like >>> * '0000:01:00.0' or virtual device name like 'eth_pcap0'. >>> * @param port_id >>> * A pointer to a port identifier actually attached. >>> * @return >>> * 0 on success and port_id is filled, negative on error >>> */ >>> int rte_eal_dev_attach(const char *devargs, uint8_t *port_id); >>> >>> /** >>> * Detach a device. >>> * >>> * @param port_id >>> * The port identifier of the device to detach. >>> * @param addr >>> * A pointer to a device name actually detached. >>> * @return >>> * 0 on success and devname is filled, negative on error >>> */ >>> int rte_eal_dev_detach(uint8_t port_id, char *devname); >>> ------------------------------------------------------------------------------- >>> >>> This patch series are for DPDK EAL. To use port hotplug function by DPDK apps, >>> each PMD should be fixed to support 'RTE_PCI_DRV_DETACHABLE' flag. Please check >>> a patch for pcap PMD. >>> >>> Also please check testpmd patch. It will show you how to fix your legacy >>> applications to support port hotplug feature. >>> >>> >>> PATCH v4 changes >>> - Merge patches to review easier. >>> - Fix indent of 'if' statement. >>> - Fix calculation method of eal_compare_pci_addr(). >>> - Fix header file declaration. >>> - Add header file to determine if hotplug can be enabled. >>> (Thanks to Qiu, Michael) >>> - Use braces with 'for' loop. >>> - Add paramerter checking. >>> - Fix sanity check code >>> - Fix comments of rte_eth_dev_type. >>> - Change function names. >>> (Thanks to Iremonger, Bernard) >>> >>> PATCH v3 changes: >>> - Fix enum definition used in rte_ethdev.c. >>> (Thanks to Zhang, Helin) >>> >>> PATCH v2 changes: >>> - Replace rte_eal_dev_attach_pdev(), rte_eal_dev_detach_pdev, >>> rte_eal_dev_attach_vdev() and rte_eal_dev_detach_vdev() to >>> rte_eal_dev_attach() and rte_eal_dev_detach(). >>> - Add parameter values checking. >>> - Refashion a few functions. >>> (Thanks to Iremonger, Bernard) >>> >>> PATCH v1 Changes: >>> - Fix error checking code of librte_eth APIs. >>> - Fix issue that port from pcap PMD cannot be detached correctly. >>> - Fix issue that testpmd could hang after forwarding, if attaching and detaching >>> is repeatedly. >>> - Fix if-condition of rte_eth_dev_get_port_by_addr(). >>> (Thanks to Mark Enright) >>> >>> RFC PATCH v2 Changes: >>> - remove 'rte_eth_dev_validate_port()', and cleanup codes. >>> >>> >>> Tetsuya Mukawa (11): >>> eal/pci,ethdev: Remove assumption that port will not be detached >>> eal/pci: Consolidate pci address comparison APIs >>> ethdev: Add rte_eth_dev_free to free specified device >>> eal,ethdev: Add a function and function pointers to close ether device >>> ethdev: Add functions that will be used by port hotplug functions >>> eal/linux/pci: Add functions for unmapping igb_uio resources >>> eal/pci: Add a function to remove the entry of devargs list >>> eal/pci: Cleanup pci driver initialization code >>> ethdev: Add one dev_type paramerter to rte_eth_dev_allocate >>> eal/pci: Add rte_eal_dev_attach/detach() functions >>> eal: Enable port hotplug framework in Linux >>> >>> app/test/virtual_pmd.c | 2 +- >>> config/common_linuxapp | 5 + >>> lib/librte_eal/bsdapp/eal/eal_pci.c | 16 +- >>> lib/librte_eal/common/Makefile | 1 + >>> lib/librte_eal/common/eal_common_dev.c | 273 +++++++++++++++ >>> lib/librte_eal/common/eal_common_devargs.c | 57 ++++ >>> lib/librte_eal/common/eal_common_pci.c | 92 ++++- >>> lib/librte_eal/common/eal_private.h | 36 ++ >>> lib/librte_eal/common/include/rte_dev.h | 33 ++ >>> lib/librte_eal/common/include/rte_dev_hotplug.h | 44 +++ >>> lib/librte_eal/common/include/rte_devargs.h | 18 + >>> lib/librte_eal/common/include/rte_pci.h | 76 +++++ >>> lib/librte_eal/linuxapp/eal/Makefile | 1 + >>> lib/librte_eal/linuxapp/eal/eal_pci.c | 136 ++++++-- >>> lib/librte_eal/linuxapp/eal/eal_pci_init.h | 8 + >>> lib/librte_eal/linuxapp/eal/eal_pci_uio.c | 67 +++- >>> lib/librte_ether/rte_ethdev.c | 424 ++++++++++++++++++------ >>> lib/librte_ether/rte_ethdev.h | 145 +++++++- >>> lib/librte_pmd_af_packet/rte_eth_af_packet.c | 2 +- >>> lib/librte_pmd_bond/rte_eth_bond_api.c | 2 +- >>> lib/librte_pmd_pcap/rte_eth_pcap.c | 2 +- >>> lib/librte_pmd_ring/rte_eth_ring.c | 2 +- >>> lib/librte_pmd_xenvirt/rte_eth_xenvirt.c | 2 +- >>> 23 files changed, 1283 insertions(+), 161 deletions(-) >>> create mode 100644 lib/librte_eal/common/include/rte_dev_hotplug.h >>> > >
On 2015/01/27 14:50, Qiu, Michael wrote: > On 1/27/2015 1:02 PM, Tetsuya Mukawa wrote: >> On 2015/01/27 12:00, Qiu, Michael wrote: >>> On 1/19/2015 6:42 PM, Tetsuya Mukawa wrote: >>>> This patch series adds a dynamic port hotplug framework to DPDK. >>>> With the patches, DPDK apps can attach or detach ports at runtime. >>>> >>>> The basic concept of the port hotplug is like followings. >>>> - DPDK apps must have responsibility to manage ports. >>>> DPDK apps only know which ports are attached or detached at the moment. >>>> The port hotplug framework is implemented to allow DPDK apps to manage ports. >>>> For example, when DPDK apps call port attach function, attached port number >>>> will be returned. Also DPDK apps can detach port by port number. >>>> - Kernel support is needed for attaching or detaching physical device ports. >>>> To attach new device, the device will be recognized by kernel at first and >>>> controlled by kernel driver. Then user can bind the device to igb_uio >>> Here does it really need native kernel driver here? As it will be >>> controlled by igb_uio. >>> I think even if the device has no kernel driver is also OK. >> Thanks for correcting. Yes, it should be. >> How about following. >> >> - Kernel support is needed for attaching or detaching physical device ports. >> To attach a new device, the device will be recognized by kernel PCI >> hotplug feature at first. > No, here should not explain as "kernel PCI hotplug feature" which is > stand for removing or adding a PCI device from system level, it is > devices related not driver. > > What about: > > - Kernel support is needed for attaching or detaching physical device > ports. To attach a new physical device port, the device will be > recognized by userspace directly I/O framework in kernel at first. Thanks. I will replace like above. Thanks, Tetsuya > Thanks, > Michael >> Then user can bind the device to igb_uio. >> >>> Also I have finished initial patch of passthrough driver flag in >>> "struct rte_pci_device" >>> >>> I will send to you after I do some basic test on that, then I will send to >>> you, and you can give some comments on that. >> I appreciate for your implementing. >> >> Thanks, >> Tetsuya >> >> >>> Thanks, >>> Michael >>> >>>> by 'dpdk_nic_bind.py'. Finally, DPDK apps can call the port hotplug >>>> functions to attach ports. >>>> For detaching, steps are vice versa. >>>> - Before detach ports, ports must be stopped and closed. >>>> DPDK application must call rte_eth_dev_stop() and rte_eth_dev_close() before >>>> detaching ports. These function will call finalization codes of PMDs. >>>> But so far, no PMD frees all resources allocated by initialization. >>>> It means PMDs are needed to be fixed to support the port hotplug. >>>> 'RTE_PCI_DRV_DETACHABLE' is a new flag indicating a PMD supports detaching. >>>> Without this flag, detaching will be failed. >>>> - Mustn't affect legacy DPDK apps. >>>> No DPDK EAL behavior is changed, if the port hotplug functions are't called. >>>> So all legacy DPDK apps can still work without modifications. >>>> >>>> And a few limitations. >>>> - The port hotplug functions are not thread safe. >>>> DPDK apps should handle it. >>>> - Only support Linux and igb_uio so far. >>>> BSD and VFIO is not supported. I will send VFIO patches at least, but I don't >>>> have a plan to submit BSD patch so far. >>>> >>>> >>>> Here is port hotplug APIs. >>>> ------------------------------------------------------------------------------- >>>> /** >>>> * Attach a new device. >>>> * >>>> * @param devargs >>>> * A pointer to a strings array describing the new device >>>> * to be attached. The strings should be a pci address like >>>> * '0000:01:00.0' or virtual device name like 'eth_pcap0'. >>>> * @param port_id >>>> * A pointer to a port identifier actually attached. >>>> * @return >>>> * 0 on success and port_id is filled, negative on error >>>> */ >>>> int rte_eal_dev_attach(const char *devargs, uint8_t *port_id); >>>> >>>> /** >>>> * Detach a device. >>>> * >>>> * @param port_id >>>> * The port identifier of the device to detach. >>>> * @param addr >>>> * A pointer to a device name actually detached. >>>> * @return >>>> * 0 on success and devname is filled, negative on error >>>> */ >>>> int rte_eal_dev_detach(uint8_t port_id, char *devname); >>>> ------------------------------------------------------------------------------- >>>> >>>> This patch series are for DPDK EAL. To use port hotplug function by DPDK apps, >>>> each PMD should be fixed to support 'RTE_PCI_DRV_DETACHABLE' flag. Please check >>>> a patch for pcap PMD. >>>> >>>> Also please check testpmd patch. It will show you how to fix your legacy >>>> applications to support port hotplug feature. >>>> >>>> >>>> PATCH v4 changes >>>> - Merge patches to review easier. >>>> - Fix indent of 'if' statement. >>>> - Fix calculation method of eal_compare_pci_addr(). >>>> - Fix header file declaration. >>>> - Add header file to determine if hotplug can be enabled. >>>> (Thanks to Qiu, Michael) >>>> - Use braces with 'for' loop. >>>> - Add paramerter checking. >>>> - Fix sanity check code >>>> - Fix comments of rte_eth_dev_type. >>>> - Change function names. >>>> (Thanks to Iremonger, Bernard) >>>> >>>> PATCH v3 changes: >>>> - Fix enum definition used in rte_ethdev.c. >>>> (Thanks to Zhang, Helin) >>>> >>>> PATCH v2 changes: >>>> - Replace rte_eal_dev_attach_pdev(), rte_eal_dev_detach_pdev, >>>> rte_eal_dev_attach_vdev() and rte_eal_dev_detach_vdev() to >>>> rte_eal_dev_attach() and rte_eal_dev_detach(). >>>> - Add parameter values checking. >>>> - Refashion a few functions. >>>> (Thanks to Iremonger, Bernard) >>>> >>>> PATCH v1 Changes: >>>> - Fix error checking code of librte_eth APIs. >>>> - Fix issue that port from pcap PMD cannot be detached correctly. >>>> - Fix issue that testpmd could hang after forwarding, if attaching and detaching >>>> is repeatedly. >>>> - Fix if-condition of rte_eth_dev_get_port_by_addr(). >>>> (Thanks to Mark Enright) >>>> >>>> RFC PATCH v2 Changes: >>>> - remove 'rte_eth_dev_validate_port()', and cleanup codes. >>>> >>>> >>>> Tetsuya Mukawa (11): >>>> eal/pci,ethdev: Remove assumption that port will not be detached >>>> eal/pci: Consolidate pci address comparison APIs >>>> ethdev: Add rte_eth_dev_free to free specified device >>>> eal,ethdev: Add a function and function pointers to close ether device >>>> ethdev: Add functions that will be used by port hotplug functions >>>> eal/linux/pci: Add functions for unmapping igb_uio resources >>>> eal/pci: Add a function to remove the entry of devargs list >>>> eal/pci: Cleanup pci driver initialization code >>>> ethdev: Add one dev_type paramerter to rte_eth_dev_allocate >>>> eal/pci: Add rte_eal_dev_attach/detach() functions >>>> eal: Enable port hotplug framework in Linux >>>> >>>> app/test/virtual_pmd.c | 2 +- >>>> config/common_linuxapp | 5 + >>>> lib/librte_eal/bsdapp/eal/eal_pci.c | 16 +- >>>> lib/librte_eal/common/Makefile | 1 + >>>> lib/librte_eal/common/eal_common_dev.c | 273 +++++++++++++++ >>>> lib/librte_eal/common/eal_common_devargs.c | 57 ++++ >>>> lib/librte_eal/common/eal_common_pci.c | 92 ++++- >>>> lib/librte_eal/common/eal_private.h | 36 ++ >>>> lib/librte_eal/common/include/rte_dev.h | 33 ++ >>>> lib/librte_eal/common/include/rte_dev_hotplug.h | 44 +++ >>>> lib/librte_eal/common/include/rte_devargs.h | 18 + >>>> lib/librte_eal/common/include/rte_pci.h | 76 +++++ >>>> lib/librte_eal/linuxapp/eal/Makefile | 1 + >>>> lib/librte_eal/linuxapp/eal/eal_pci.c | 136 ++++++-- >>>> lib/librte_eal/linuxapp/eal/eal_pci_init.h | 8 + >>>> lib/librte_eal/linuxapp/eal/eal_pci_uio.c | 67 +++- >>>> lib/librte_ether/rte_ethdev.c | 424 ++++++++++++++++++------ >>>> lib/librte_ether/rte_ethdev.h | 145 +++++++- >>>> lib/librte_pmd_af_packet/rte_eth_af_packet.c | 2 +- >>>> lib/librte_pmd_bond/rte_eth_bond_api.c | 2 +- >>>> lib/librte_pmd_pcap/rte_eth_pcap.c | 2 +- >>>> lib/librte_pmd_ring/rte_eth_ring.c | 2 +- >>>> lib/librte_pmd_xenvirt/rte_eth_xenvirt.c | 2 +- >>>> 23 files changed, 1283 insertions(+), 161 deletions(-) >>>> create mode 100644 lib/librte_eal/common/include/rte_dev_hotplug.h >>>> >>
This patch series adds a dynamic port hotplug framework to DPDK. With the patches, DPDK apps can attach or detach ports at runtime. The basic concept of the port hotplug is like followings. - DPDK apps must have responsibility to manage ports. DPDK apps only know which ports are attached or detached at the moment. The port hotplug framework is implemented to allow DPDK apps to manage ports. For example, when DPDK apps call port attach function, attached port number will be returned. Also DPDK apps can detach port by port number. - Kernel support is needed for attaching or detaching physical device ports. To attach a new physical device port, the device will be recognized by userspace directly I/O framework in kernel at first. Then DPDK apps can call the port hotplug functions to attach ports. For detaching, steps are vice versa. - Before detach ports, ports must be stopped and closed. DPDK application must call rte_eth_dev_stop() and rte_eth_dev_close() before detaching ports. These function will call finalization codes of PMDs. But so far, no PMD frees all resources allocated by initialization. It means PMDs are needed to be fixed to support the port hotplug. 'RTE_PCI_DRV_DETACHABLE' is a new flag indicating a PMD supports detaching. Without this flag, detaching will be failed. - Mustn't affect legacy DPDK apps. No DPDK EAL behavior is changed, if the port hotplug functions are't called. So all legacy DPDK apps can still work without modifications. And a few limitations. - The port hotplug functions are not thread safe. DPDK apps should handle it. - Only support Linux and igb_uio so far. BSD and VFIO is not supported. I will send VFIO patches at least, but I don't have a plan to submit BSD patch so far. Here is port hotplug APIs. ------------------------------------------------------------------------------- /** * Attach a new device. * * @param devargs * A pointer to a strings array describing the new device * to be attached. The strings should be a pci address like * '0000:01:00.0' or virtual device name like 'eth_pcap0'. * @param port_id * A pointer to a port identifier actually attached. * @return * 0 on success and port_id is filled, negative on error */ int rte_eal_dev_attach(const char *devargs, uint8_t *port_id); /** * Detach a device. * * @param port_id * The port identifier of the device to detach. * @param addr * A pointer to a device name actually detached. * @return * 0 on success and devname is filled, negative on error */ int rte_eal_dev_detach(uint8_t port_id, char *devname); ------------------------------------------------------------------------------- This patch series are for DPDK EAL. To use port hotplug function by DPDK apps, each PMD should be fixed to support 'RTE_PCI_DRV_DETACHABLE' flag. Please check a patch for pcap PMD. Also please check testpmd patch. It will show you how to fix your legacy applications to support port hotplug feature. PATCH v5 changes - Add runtime check passthrough driver type, like vfio-pci, igb_uio and uio_pci_generic. This was done by Qiu, Michael. Thanks a lot. - Change function names like below. - rte_eal_dev_find_and_invoke() to rte_eal_vdev_find_and_invoke(). - rte_eal_dev_invoke() to rte_eal_vdev_invoke(). - Add code to handle a return value of rte_eal_devargs_remove(). - Fix pci address format in rte_eal_dev_detach(). - Remove RTE_EAL_INVOKE_TYPE_UNKNOWN, because it's unused. - Change function definition of rte_eal_devargs_remove(). - Fix pci_unmap_device() to check pt_driver. - Fix return value of below functions. - rte_eth_dev_get_changed_port(). - rte_eth_dev_get_port_by_addr(). - Change paramters of rte_eth_dev_validate_port() to cleanup code. - Fix pci_scan_one to handle pt_driver correctly. (Thanks to Qiu, Michael for above sugesstions) PATCH v4 changes - Merge patches to review easier. - Fix indent of 'if' statement. - Fix calculation method of eal_compare_pci_addr(). - Fix header file declaration. - Add header file to determine if hotplug can be enabled. (Thanks to Qiu, Michael) - Use braces with 'for' loop. - Add paramerter checking. - Fix sanity check code - Fix comments of rte_eth_dev_type. - Change function names. (Thanks to Iremonger, Bernard) PATCH v3 changes: - Fix enum definition used in rte_ethdev.c. (Thanks to Zhang, Helin) PATCH v2 changes: - Replace rte_eal_dev_attach_pdev(), rte_eal_dev_detach_pdev, rte_eal_dev_attach_vdev() and rte_eal_dev_detach_vdev() to rte_eal_dev_attach() and rte_eal_dev_detach(). - Add parameter values checking. - Refashion a few functions. (Thanks to Iremonger, Bernard) PATCH v1 Changes: - Fix error checking code of librte_eth APIs. - Fix issue that port from pcap PMD cannot be detached correctly. - Fix issue that testpmd could hang after forwarding, if attaching and detaching is repeatedly. - Fix if-condition of rte_eth_dev_get_port_by_addr(). (Thanks to Mark Enright) RFC PATCH v2 Changes: - remove 'rte_eth_dev_validate_port()', and cleanup codes. Michael Qiu (2): eal_pci: Add flag to hold kernel driver type eal_pci: pci memory map work with driver type Tetsuya Mukawa (11): eal/pci,ethdev: Remove assumption that port will not be detached eal/pci: Consolidate pci address comparison APIs ethdev: Add rte_eth_dev_free to free specified device eal,ethdev: Add a function and function pointers to close ether device ethdev: Add functions that will be used by port hotplug functions eal/linux/pci: Add functions for unmapping igb_uio resources eal/pci: Add a function to remove the entry of devargs list eal/pci: Cleanup pci driver initialization code ethdev: Add one dev_type paramerter to rte_eth_dev_allocate eal/pci: Add rte_eal_dev_attach/detach() functions eal: Enable port hotplug framework in Linux app/test/virtual_pmd.c | 2 +- config/common_linuxapp | 5 + lib/librte_eal/bsdapp/eal/eal_pci.c | 25 +- lib/librte_eal/common/Makefile | 1 + lib/librte_eal/common/eal_common_dev.c | 274 +++++++++++ lib/librte_eal/common/eal_common_devargs.c | 60 +++ lib/librte_eal/common/eal_common_pci.c | 92 +++- lib/librte_eal/common/eal_private.h | 35 ++ lib/librte_eal/common/include/rte_dev.h | 33 ++ lib/librte_eal/common/include/rte_dev_hotplug.h | 44 ++ lib/librte_eal/common/include/rte_devargs.h | 21 + lib/librte_eal/common/include/rte_pci.h | 84 ++++ lib/librte_eal/linuxapp/eal/Makefile | 1 + lib/librte_eal/linuxapp/eal/eal_pci.c | 227 +++++++-- lib/librte_eal/linuxapp/eal/eal_pci_init.h | 8 + lib/librte_eal/linuxapp/eal/eal_pci_uio.c | 67 ++- lib/librte_ether/rte_ethdev.c | 626 +++++++++++++----------- lib/librte_ether/rte_ethdev.h | 145 +++++- lib/librte_pmd_af_packet/rte_eth_af_packet.c | 2 +- lib/librte_pmd_bond/rte_eth_bond_api.c | 2 +- lib/librte_pmd_pcap/rte_eth_pcap.c | 2 +- lib/librte_pmd_ring/rte_eth_ring.c | 2 +- lib/librte_pmd_xenvirt/rte_eth_xenvirt.c | 2 +- 23 files changed, 1413 insertions(+), 347 deletions(-) create mode 100644 lib/librte_eal/common/include/rte_dev_hotplug.h -- 1.9.1
From: Michael Qiu <michael.qiu@intel.com> Currently, dpdk has no ability to know which type of driver( vfio-pci/igb_uio/uio_pci_generic) the device used. It only can check whether vfio is enabled or not staticly. It really useful to have the flag, becasue different type need to handle differently in runtime. For example, pci memory map, pot hotplug, and so on. This patch add a flag field for pci device to solve above issue. Signed-off-by: Michael Qiu <michael.qiu@intel.com> Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp> --- lib/librte_eal/common/include/rte_pci.h | 8 +++++ lib/librte_eal/linuxapp/eal/eal_pci.c | 53 +++++++++++++++++++++++++++++++-- 2 files changed, 59 insertions(+), 2 deletions(-) diff --git a/lib/librte_eal/common/include/rte_pci.h b/lib/librte_eal/common/include/rte_pci.h index 66ed793..7b48b55 100644 --- a/lib/librte_eal/common/include/rte_pci.h +++ b/lib/librte_eal/common/include/rte_pci.h @@ -139,6 +139,13 @@ struct rte_pci_addr { struct rte_devargs; +enum rte_pt_driver { + RTE_PT_UNKNOWN = 0, + RTE_PT_IGB_UIO = 1, + RTE_PT_VFIO = 2, + RTE_PT_UIO_GENERIC = 3, +}; + /** * A structure describing a PCI device. */ @@ -152,6 +159,7 @@ struct rte_pci_device { uint16_t max_vfs; /**< sriov enable if not zero */ int numa_node; /**< NUMA node connection */ struct rte_devargs *devargs; /**< Device user arguments */ + enum rte_pt_driver pt_driver; /**< Driver of passthrough */ }; /** Any PCI device identifier (vendor, device, ...) */ diff --git a/lib/librte_eal/linuxapp/eal/eal_pci.c b/lib/librte_eal/linuxapp/eal/eal_pci.c index b5f5410..bd3f77d 100644 --- a/lib/librte_eal/linuxapp/eal/eal_pci.c +++ b/lib/librte_eal/linuxapp/eal/eal_pci.c @@ -97,6 +97,35 @@ error: return -1; } +static int +pci_get_kernel_driver_by_path(const char *filename, char *dri_name) +{ + int count; + char path[PATH_MAX]; + char *name; + + if (!filename || !dri_name) + return -1; + + count = readlink(filename, path, PATH_MAX); + if (count >= PATH_MAX) + return -1; + + /* For device does not have a driver */ + if (count < 0) + return 1; + + path[count] = '\0'; + + name = strrchr(path, '/'); + if (name) { + strncpy(dri_name, name + 1, strlen(name + 1) + 1); + return 0; + } + + return -1; +} + void * pci_find_max_end_va(void) { @@ -222,11 +251,12 @@ pci_scan_one(const char *dirname, uint16_t domain, uint8_t bus, char filename[PATH_MAX]; unsigned long tmp; struct rte_pci_device *dev; + char driver[PATH_MAX]; + int ret; dev = malloc(sizeof(*dev)); - if (dev == NULL) { + if (dev == NULL) return -1; - } memset(dev, 0, sizeof(*dev)); dev->addr.domain = domain; @@ -298,6 +328,25 @@ pci_scan_one(const char *dirname, uint16_t domain, uint8_t bus, return -1; } + /* parse driver */ + snprintf(filename, sizeof(filename), "%s/driver", dirname); + ret = pci_get_kernel_driver_by_path(filename, driver); + if (!ret) { + if (!strcmp(driver, "vfio-pci")) + dev->pt_driver = RTE_PT_VFIO; + else if (!strcmp(driver, "igb_uio")) + dev->pt_driver = RTE_PT_IGB_UIO; + else if (!strcmp(driver, "uio_pci_generic")) + dev->pt_driver = RTE_PT_UIO_GENERIC; + else + dev->pt_driver = RTE_PT_UNKNOWN; + } else if (ret < 0) { + RTE_LOG(ERR, EAL, "Fail to get kernel driver\n"); + free(dev); + return -1; + } else + dev->pt_driver = RTE_PT_UNKNOWN; + /* device is valid, add in list (sorted) */ if (TAILQ_EMPTY(&pci_device_list)) { TAILQ_INSERT_TAIL(&pci_device_list, dev, next); -- 1.9.1
From: Michael Qiu <michael.qiu@intel.com> With the driver type flag in struct rte_pci_dev, we do not need to always map uio devices with vfio related function when vfio enabled. Signed-off-by: Michael Qiu <michael.qiu@intel.com> Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp> --- lib/librte_eal/linuxapp/eal/eal_pci.c | 30 +++++++++++++++++------------- 1 file changed, 17 insertions(+), 13 deletions(-) diff --git a/lib/librte_eal/linuxapp/eal/eal_pci.c b/lib/librte_eal/linuxapp/eal/eal_pci.c index bd3f77d..c0ca5a5 100644 --- a/lib/librte_eal/linuxapp/eal/eal_pci.c +++ b/lib/librte_eal/linuxapp/eal/eal_pci.c @@ -549,25 +549,29 @@ pci_config_space_set(struct rte_pci_device *dev) static int pci_map_device(struct rte_pci_device *dev) { - int ret, mapped = 0; + int ret = -1; /* try mapping the NIC resources using VFIO if it exists */ + switch (dev->pt_driver) { + case RTE_PT_VFIO: #ifdef VFIO_PRESENT - if (pci_vfio_is_enabled()) { - ret = pci_vfio_map_resource(dev); - if (ret == 0) - mapped = 1; - else if (ret < 0) - return ret; - } + if (pci_vfio_is_enabled()) + ret = pci_vfio_map_resource(dev); #endif - /* map resources for devices that use igb_uio */ - if (!mapped) { + break; + case RTE_PT_IGB_UIO: + case RTE_PT_UIO_GENERIC: + /* map resources for devices that use uio */ ret = pci_uio_map_resource(dev); - if (ret != 0) - return ret; + break; + default: + RTE_LOG(DEBUG, EAL, " Not managed by known pt driver," + " skipped\n"); + ret = 1; + break; } - return 0; + + return ret; } /* -- 1.9.1
To remove assumption, do like followings. This patch adds "RTE_PCI_DRV_DETACHABLE" to drv_flags of rte_pci_driver structure. The flags indicates the driver can detach devices at runtime. Also remove assumption that port will not be detached. To remove the assumption. - Add 'attached' member to rte_eth_dev structure. This member is used for indicating the port is attached, or not. - Add rte_eth_dev_allocate_new_port(). This function is used for allocating new port. v5: - Change paramters of rte_eth_dev_validate_port() to cleanup code. v4: - Use braces with 'for' loop. - Fix indent of 'if' statement. Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp> --- lib/librte_eal/common/include/rte_pci.h | 2 + lib/librte_ether/rte_ethdev.c | 454 +++++++++++++------------------- lib/librte_ether/rte_ethdev.h | 5 + 3 files changed, 186 insertions(+), 275 deletions(-) diff --git a/lib/librte_eal/common/include/rte_pci.h b/lib/librte_eal/common/include/rte_pci.h index 7b48b55..7f2d699 100644 --- a/lib/librte_eal/common/include/rte_pci.h +++ b/lib/librte_eal/common/include/rte_pci.h @@ -207,6 +207,8 @@ struct rte_pci_driver { #define RTE_PCI_DRV_FORCE_UNBIND 0x0004 /** Device driver supports link state interrupt */ #define RTE_PCI_DRV_INTR_LSC 0x0008 +/** Device driver supports detaching capability */ +#define RTE_PCI_DRV_DETACHABLE 0x0010 /**< Internal use only - Macro used by pci addr parsing functions **/ #define GET_PCIADDR_FIELD(in, fd, lim, dlm) \ diff --git a/lib/librte_ether/rte_ethdev.c b/lib/librte_ether/rte_ethdev.c index ea3a1fb..d70854f 100644 --- a/lib/librte_ether/rte_ethdev.c +++ b/lib/librte_ether/rte_ethdev.c @@ -175,6 +175,16 @@ enum { STAT_QMAP_RX }; +enum { + DEV_INVALID = 0, + DEV_VALID, +}; + +enum { + DEV_DISCONNECTED = 0, + DEV_CONNECTED +}; + static inline void rte_eth_dev_data_alloc(void) { @@ -201,19 +211,34 @@ rte_eth_dev_allocated(const char *name) { unsigned i; - for (i = 0; i < nb_ports; i++) { - if (strcmp(rte_eth_devices[i].data->name, name) == 0) + for (i = 0; i < RTE_MAX_ETHPORTS; i++) { + if ((rte_eth_devices[i].attached == DEV_CONNECTED) && + strcmp(rte_eth_devices[i].data->name, name) == 0) return &rte_eth_devices[i]; } return NULL; } +static uint8_t +rte_eth_dev_allocate_new_port(void) +{ + unsigned i; + + for (i = 0; i < RTE_MAX_ETHPORTS; i++) { + if (rte_eth_devices[i].attached == DEV_DISCONNECTED) + return i; + } + return RTE_MAX_ETHPORTS; +} + struct rte_eth_dev * rte_eth_dev_allocate(const char *name) { + uint8_t port_id; struct rte_eth_dev *eth_dev; - if (nb_ports == RTE_MAX_ETHPORTS) { + port_id = rte_eth_dev_allocate_new_port(); + if (port_id == RTE_MAX_ETHPORTS) { PMD_DEBUG_TRACE("Reached maximum number of Ethernet ports\n"); return NULL; } @@ -226,10 +251,12 @@ rte_eth_dev_allocate(const char *name) return NULL; } - eth_dev = &rte_eth_devices[nb_ports]; - eth_dev->data = &rte_eth_dev_data[nb_ports]; + eth_dev = &rte_eth_devices[port_id]; + eth_dev->data = &rte_eth_dev_data[port_id]; snprintf(eth_dev->data->name, sizeof(eth_dev->data->name), "%s", name); - eth_dev->data->port_id = nb_ports++; + eth_dev->data->port_id = port_id; + eth_dev->attached = DEV_CONNECTED; + nb_ports++; return eth_dev; } @@ -283,6 +310,7 @@ rte_eth_dev_init(struct rte_pci_driver *pci_drv, (unsigned) pci_dev->id.device_id); if (rte_eal_process_type() == RTE_PROC_PRIMARY) rte_free(eth_dev->data->dev_private); + eth_dev->attached = DEV_DISCONNECTED; nb_ports--; return diag; } @@ -308,10 +336,28 @@ rte_eth_driver_register(struct eth_driver *eth_drv) rte_eal_pci_register(ð_drv->pci_drv); } +enum { + NONE_TRACE = 0, + TRACE +}; + +static int +rte_eth_dev_validate_port(uint8_t port_id, int trace) +{ + if (port_id >= RTE_MAX_ETHPORTS || + rte_eth_devices[port_id].attached != DEV_CONNECTED) { + if (trace) { + PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + } + return DEV_INVALID; + } else + return DEV_VALID; +} + int rte_eth_dev_socket_id(uint8_t port_id) { - if (port_id >= nb_ports) + if (rte_eth_dev_validate_port(port_id, NONE_TRACE) == DEV_INVALID) return -1; return rte_eth_devices[port_id].pci_dev->numa_node; } @@ -369,10 +415,8 @@ rte_eth_dev_rx_queue_start(uint8_t port_id, uint16_t rx_queue_id) * in a multi-process setup*/ PROC_PRIMARY_OR_ERR_RET(-E_RTE_SECONDARY); - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return -EINVAL; - } dev = &rte_eth_devices[port_id]; if (rx_queue_id >= dev->data->nb_rx_queues) { @@ -395,10 +439,8 @@ rte_eth_dev_rx_queue_stop(uint8_t port_id, uint16_t rx_queue_id) * in a multi-process setup*/ PROC_PRIMARY_OR_ERR_RET(-E_RTE_SECONDARY); - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return -EINVAL; - } dev = &rte_eth_devices[port_id]; if (rx_queue_id >= dev->data->nb_rx_queues) { @@ -421,10 +463,8 @@ rte_eth_dev_tx_queue_start(uint8_t port_id, uint16_t tx_queue_id) * in a multi-process setup*/ PROC_PRIMARY_OR_ERR_RET(-E_RTE_SECONDARY); - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return -EINVAL; - } dev = &rte_eth_devices[port_id]; if (tx_queue_id >= dev->data->nb_tx_queues) { @@ -447,10 +487,8 @@ rte_eth_dev_tx_queue_stop(uint8_t port_id, uint16_t tx_queue_id) * in a multi-process setup*/ PROC_PRIMARY_OR_ERR_RET(-E_RTE_SECONDARY); - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return -EINVAL; - } dev = &rte_eth_devices[port_id]; if (tx_queue_id >= dev->data->nb_tx_queues) { @@ -703,10 +741,9 @@ rte_eth_dev_configure(uint8_t port_id, uint16_t nb_rx_q, uint16_t nb_tx_q, * in a multi-process setup*/ PROC_PRIMARY_OR_ERR_RET(-E_RTE_SECONDARY); - if (port_id >= nb_ports || port_id >= RTE_MAX_ETHPORTS) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return (-EINVAL); - } + dev = &rte_eth_devices[port_id]; FUNC_PTR_OR_ERR_RET(*dev->dev_ops->dev_infos_get, -ENOTSUP); @@ -888,10 +925,9 @@ rte_eth_dev_start(uint8_t port_id) * in a multi-process setup*/ PROC_PRIMARY_OR_ERR_RET(-E_RTE_SECONDARY); - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%" PRIu8 "\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return (-EINVAL); - } + dev = &rte_eth_devices[port_id]; FUNC_PTR_OR_ERR_RET(*dev->dev_ops->dev_start, -ENOTSUP); @@ -923,10 +959,9 @@ rte_eth_dev_stop(uint8_t port_id) * in a multi-process setup*/ PROC_PRIMARY_OR_RET(); - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%" PRIu8 "\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return; - } + dev = &rte_eth_devices[port_id]; FUNC_PTR_OR_RET(*dev->dev_ops->dev_stop); @@ -951,10 +986,9 @@ rte_eth_dev_set_link_up(uint8_t port_id) * in a multi-process setup*/ PROC_PRIMARY_OR_ERR_RET(-E_RTE_SECONDARY); - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return -EINVAL; - } + dev = &rte_eth_devices[port_id]; FUNC_PTR_OR_ERR_RET(*dev->dev_ops->dev_set_link_up, -ENOTSUP); @@ -970,10 +1004,9 @@ rte_eth_dev_set_link_down(uint8_t port_id) * in a multi-process setup*/ PROC_PRIMARY_OR_ERR_RET(-E_RTE_SECONDARY); - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return -EINVAL; - } + dev = &rte_eth_devices[port_id]; FUNC_PTR_OR_ERR_RET(*dev->dev_ops->dev_set_link_down, -ENOTSUP); @@ -989,10 +1022,8 @@ rte_eth_dev_close(uint8_t port_id) * in a multi-process setup*/ PROC_PRIMARY_OR_RET(); - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return; - } dev = &rte_eth_devices[port_id]; @@ -1017,10 +1048,9 @@ rte_eth_rx_queue_setup(uint8_t port_id, uint16_t rx_queue_id, * in a multi-process setup*/ PROC_PRIMARY_OR_ERR_RET(-E_RTE_SECONDARY); - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return (-EINVAL); - } + dev = &rte_eth_devices[port_id]; if (rx_queue_id >= dev->data->nb_rx_queues) { PMD_DEBUG_TRACE("Invalid RX queue_id=%d\n", rx_queue_id); @@ -1090,10 +1120,9 @@ rte_eth_tx_queue_setup(uint8_t port_id, uint16_t tx_queue_id, * in a multi-process setup*/ PROC_PRIMARY_OR_ERR_RET(-E_RTE_SECONDARY); - if (port_id >= RTE_MAX_ETHPORTS || port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return (-EINVAL); - } + dev = &rte_eth_devices[port_id]; if (tx_queue_id >= dev->data->nb_tx_queues) { PMD_DEBUG_TRACE("Invalid TX queue_id=%d\n", tx_queue_id); @@ -1123,10 +1152,9 @@ rte_eth_promiscuous_enable(uint8_t port_id) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return; - } + dev = &rte_eth_devices[port_id]; FUNC_PTR_OR_RET(*dev->dev_ops->promiscuous_enable); @@ -1139,10 +1167,9 @@ rte_eth_promiscuous_disable(uint8_t port_id) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return; - } + dev = &rte_eth_devices[port_id]; FUNC_PTR_OR_RET(*dev->dev_ops->promiscuous_disable); @@ -1155,10 +1182,8 @@ rte_eth_promiscuous_get(uint8_t port_id) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return -1; - } dev = &rte_eth_devices[port_id]; return dev->data->promiscuous; @@ -1169,10 +1194,9 @@ rte_eth_allmulticast_enable(uint8_t port_id) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return; - } + dev = &rte_eth_devices[port_id]; FUNC_PTR_OR_RET(*dev->dev_ops->allmulticast_enable); @@ -1185,10 +1209,9 @@ rte_eth_allmulticast_disable(uint8_t port_id) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return; - } + dev = &rte_eth_devices[port_id]; FUNC_PTR_OR_RET(*dev->dev_ops->allmulticast_disable); @@ -1201,10 +1224,8 @@ rte_eth_allmulticast_get(uint8_t port_id) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return -1; - } dev = &rte_eth_devices[port_id]; return dev->data->all_multicast; @@ -1229,10 +1250,9 @@ rte_eth_link_get(uint8_t port_id, struct rte_eth_link *eth_link) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return; - } + dev = &rte_eth_devices[port_id]; if (dev->data->dev_conf.intr_conf.lsc != 0) @@ -1249,10 +1269,9 @@ rte_eth_link_get_nowait(uint8_t port_id, struct rte_eth_link *eth_link) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return; - } + dev = &rte_eth_devices[port_id]; if (dev->data->dev_conf.intr_conf.lsc != 0) @@ -1269,10 +1288,9 @@ rte_eth_stats_get(uint8_t port_id, struct rte_eth_stats *stats) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return; - } + dev = &rte_eth_devices[port_id]; memset(stats, 0, sizeof(*stats)); @@ -1286,10 +1304,9 @@ rte_eth_stats_reset(uint8_t port_id) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return; - } + dev = &rte_eth_devices[port_id]; FUNC_PTR_OR_RET(*dev->dev_ops->stats_reset); @@ -1307,10 +1324,9 @@ rte_eth_xstats_get(uint8_t port_id, struct rte_eth_xstats *xstats, uint64_t val; char *stats_ptr; - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return -1; - } + dev = &rte_eth_devices[port_id]; /* implemented by the driver */ @@ -1376,10 +1392,9 @@ rte_eth_xstats_reset(uint8_t port_id) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return; - } + dev = &rte_eth_devices[port_id]; /* implemented by the driver */ @@ -1398,10 +1413,9 @@ set_queue_stats_mapping(uint8_t port_id, uint16_t queue_id, uint8_t stat_idx, { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return -ENODEV; - } + dev = &rte_eth_devices[port_id]; FUNC_PTR_OR_ERR_RET(*dev->dev_ops->queue_stats_mapping_set, -ENOTSUP); @@ -1433,10 +1447,9 @@ rte_eth_dev_info_get(uint8_t port_id, struct rte_eth_dev_info *dev_info) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return; - } + dev = &rte_eth_devices[port_id]; memset(dev_info, 0, sizeof(struct rte_eth_dev_info)); @@ -1453,10 +1466,9 @@ rte_eth_macaddr_get(uint8_t port_id, struct ether_addr *mac_addr) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return; - } + dev = &rte_eth_devices[port_id]; ether_addr_copy(&dev->data->mac_addrs[0], mac_addr); } @@ -1467,10 +1479,8 @@ rte_eth_dev_get_mtu(uint8_t port_id, uint16_t *mtu) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return (-ENODEV); - } dev = &rte_eth_devices[port_id]; *mtu = dev->data->mtu; @@ -1483,10 +1493,8 @@ rte_eth_dev_set_mtu(uint8_t port_id, uint16_t mtu) int ret; struct rte_eth_dev *dev; - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return (-ENODEV); - } dev = &rte_eth_devices[port_id]; FUNC_PTR_OR_ERR_RET(*dev->dev_ops->mtu_set, -ENOTSUP); @@ -1503,10 +1511,9 @@ rte_eth_dev_vlan_filter(uint8_t port_id, uint16_t vlan_id, int on) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return (-ENODEV); - } + dev = &rte_eth_devices[port_id]; if (! (dev->data->dev_conf.rxmode.hw_vlan_filter)) { PMD_DEBUG_TRACE("port %d: vlan-filtering disabled\n", port_id); @@ -1528,10 +1535,8 @@ rte_eth_dev_set_vlan_strip_on_queue(uint8_t port_id, uint16_t rx_queue_id, int o { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return (-ENODEV); - } dev = &rte_eth_devices[port_id]; if (rx_queue_id >= dev->data->nb_rx_queues) { @@ -1550,10 +1555,8 @@ rte_eth_dev_set_vlan_ether_type(uint8_t port_id, uint16_t tpid) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return (-ENODEV); - } dev = &rte_eth_devices[port_id]; FUNC_PTR_OR_ERR_RET(*dev->dev_ops->vlan_tpid_set, -ENOTSUP); @@ -1570,10 +1573,8 @@ rte_eth_dev_set_vlan_offload(uint8_t port_id, int offload_mask) int mask = 0; int cur, org = 0; - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return (-ENODEV); - } dev = &rte_eth_devices[port_id]; @@ -1615,10 +1616,8 @@ rte_eth_dev_get_vlan_offload(uint8_t port_id) struct rte_eth_dev *dev; int ret = 0; - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return (-ENODEV); - } dev = &rte_eth_devices[port_id]; @@ -1639,10 +1638,9 @@ rte_eth_dev_set_vlan_pvid(uint8_t port_id, uint16_t pvid, int on) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return (-ENODEV); - } + dev = &rte_eth_devices[port_id]; FUNC_PTR_OR_ERR_RET(*dev->dev_ops->vlan_pvid_set, -ENOTSUP); (*dev->dev_ops->vlan_pvid_set)(dev, pvid, on); @@ -1657,10 +1655,8 @@ rte_eth_dev_fdir_add_signature_filter(uint8_t port_id, { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return (-ENODEV); - } dev = &rte_eth_devices[port_id]; @@ -1691,10 +1687,8 @@ rte_eth_dev_fdir_update_signature_filter(uint8_t port_id, { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return (-ENODEV); - } dev = &rte_eth_devices[port_id]; @@ -1725,10 +1719,8 @@ rte_eth_dev_fdir_remove_signature_filter(uint8_t port_id, { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return (-ENODEV); - } dev = &rte_eth_devices[port_id]; @@ -1756,10 +1748,8 @@ rte_eth_dev_fdir_get_infos(uint8_t port_id, struct rte_eth_fdir *fdir) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return (-ENODEV); - } dev = &rte_eth_devices[port_id]; if (! (dev->data->dev_conf.fdir_conf.mode)) { @@ -1781,10 +1771,8 @@ rte_eth_dev_fdir_add_perfect_filter(uint8_t port_id, { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return (-ENODEV); - } dev = &rte_eth_devices[port_id]; @@ -1821,10 +1809,8 @@ rte_eth_dev_fdir_update_perfect_filter(uint8_t port_id, { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return (-ENODEV); - } dev = &rte_eth_devices[port_id]; @@ -1859,10 +1845,8 @@ rte_eth_dev_fdir_remove_perfect_filter(uint8_t port_id, { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return (-ENODEV); - } dev = &rte_eth_devices[port_id]; @@ -1895,10 +1879,8 @@ rte_eth_dev_fdir_set_masks(uint8_t port_id, struct rte_fdir_masks *fdir_mask) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return (-ENODEV); - } dev = &rte_eth_devices[port_id]; if (! (dev->data->dev_conf.fdir_conf.mode)) { @@ -1915,10 +1897,8 @@ rte_eth_dev_flow_ctrl_get(uint8_t port_id, struct rte_eth_fc_conf *fc_conf) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return (-ENODEV); - } dev = &rte_eth_devices[port_id]; FUNC_PTR_OR_ERR_RET(*dev->dev_ops->flow_ctrl_get, -ENOTSUP); @@ -1931,10 +1911,8 @@ rte_eth_dev_flow_ctrl_set(uint8_t port_id, struct rte_eth_fc_conf *fc_conf) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return (-ENODEV); - } if ((fc_conf->send_xon != 0) && (fc_conf->send_xon != 1)) { PMD_DEBUG_TRACE("Invalid send_xon, only 0/1 allowed\n"); @@ -1951,10 +1929,8 @@ rte_eth_dev_priority_flow_ctrl_set(uint8_t port_id, struct rte_eth_pfc_conf *pfc { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return (-ENODEV); - } if (pfc_conf->priority > (ETH_DCB_NUM_USER_PRIORITIES - 1)) { PMD_DEBUG_TRACE("Invalid priority, only 0-7 allowed\n"); @@ -2030,10 +2006,8 @@ rte_eth_dev_rss_reta_update(uint8_t port_id, struct rte_eth_dev *dev; int ret; - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return -ENODEV; - } /* Check mask bits */ ret = rte_eth_check_reta_mask(reta_conf, reta_size); @@ -2081,10 +2055,9 @@ rte_eth_dev_rss_hash_update(uint8_t port_id, struct rte_eth_rss_conf *rss_conf) struct rte_eth_dev *dev; uint16_t rss_hash_protos; - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return (-ENODEV); - } + rss_hash_protos = rss_conf->rss_hf; if ((rss_hash_protos != 0) && ((rss_hash_protos & ETH_RSS_PROTO_MASK) == 0)) { @@ -2103,10 +2076,9 @@ rte_eth_dev_rss_hash_conf_get(uint8_t port_id, { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return (-ENODEV); - } + dev = &rte_eth_devices[port_id]; FUNC_PTR_OR_ERR_RET(*dev->dev_ops->rss_hash_conf_get, -ENOTSUP); return (*dev->dev_ops->rss_hash_conf_get)(dev, rss_conf); @@ -2118,10 +2090,8 @@ rte_eth_dev_udp_tunnel_add(uint8_t port_id, { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return -ENODEV; - } if (udp_tunnel == NULL) { PMD_DEBUG_TRACE("Invalid udp_tunnel parameter\n"); @@ -2144,10 +2114,9 @@ rte_eth_dev_udp_tunnel_delete(uint8_t port_id, { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return -ENODEV; - } + dev = &rte_eth_devices[port_id]; if (udp_tunnel == NULL) { @@ -2169,10 +2138,8 @@ rte_eth_led_on(uint8_t port_id) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return (-ENODEV); - } dev = &rte_eth_devices[port_id]; FUNC_PTR_OR_ERR_RET(*dev->dev_ops->dev_led_on, -ENOTSUP); @@ -2184,10 +2151,8 @@ rte_eth_led_off(uint8_t port_id) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return (-ENODEV); - } dev = &rte_eth_devices[port_id]; FUNC_PTR_OR_ERR_RET(*dev->dev_ops->dev_led_off, -ENOTSUP); @@ -2224,10 +2189,9 @@ rte_eth_dev_mac_addr_add(uint8_t port_id, struct ether_addr *addr, int index; uint64_t pool_mask; - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return (-ENODEV); - } + dev = &rte_eth_devices[port_id]; FUNC_PTR_OR_ERR_RET(*dev->dev_ops->mac_addr_add, -ENOTSUP); @@ -2275,10 +2239,9 @@ rte_eth_dev_mac_addr_remove(uint8_t port_id, struct ether_addr *addr) struct rte_eth_dev *dev; int index; - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return (-ENODEV); - } + dev = &rte_eth_devices[port_id]; FUNC_PTR_OR_ERR_RET(*dev->dev_ops->mac_addr_remove, -ENOTSUP); @@ -2309,7 +2272,7 @@ rte_eth_dev_set_vf_rxmode(uint8_t port_id, uint16_t vf, struct rte_eth_dev *dev; struct rte_eth_dev_info dev_info; - if (port_id >= nb_ports) { + if (rte_eth_dev_validate_port(port_id, NONE_TRACE) == DEV_INVALID) { PMD_DEBUG_TRACE("set VF RX mode:Invalid port_id=%d\n", port_id); return (-ENODEV); @@ -2364,7 +2327,7 @@ rte_eth_dev_uc_hash_table_set(uint8_t port_id, struct ether_addr *addr, int ret; struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (rte_eth_dev_validate_port(port_id, NONE_TRACE) == DEV_INVALID) { PMD_DEBUG_TRACE("unicast hash setting:Invalid port_id=%d\n", port_id); return (-ENODEV); @@ -2417,7 +2380,7 @@ rte_eth_dev_uc_all_hash_table_set(uint8_t port_id, uint8_t on) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (rte_eth_dev_validate_port(port_id, NONE_TRACE) == DEV_INVALID) { PMD_DEBUG_TRACE("unicast hash setting:Invalid port_id=%d\n", port_id); return (-ENODEV); @@ -2436,10 +2399,8 @@ rte_eth_dev_set_vf_rx(uint8_t port_id,uint16_t vf, uint8_t on) struct rte_eth_dev *dev; struct rte_eth_dev_info dev_info; - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return (-ENODEV); - } dev = &rte_eth_devices[port_id]; rte_eth_dev_info_get(port_id, &dev_info); @@ -2462,7 +2423,7 @@ rte_eth_dev_set_vf_tx(uint8_t port_id,uint16_t vf, uint8_t on) struct rte_eth_dev *dev; struct rte_eth_dev_info dev_info; - if (port_id >= nb_ports) { + if (rte_eth_dev_validate_port(port_id, NONE_TRACE) == DEV_INVALID) { PMD_DEBUG_TRACE("set pool tx:Invalid port_id=%d\n", port_id); return (-ENODEV); } @@ -2487,7 +2448,7 @@ rte_eth_dev_set_vf_vlan_filter(uint8_t port_id, uint16_t vlan_id, { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (rte_eth_dev_validate_port(port_id, NONE_TRACE) == DEV_INVALID) { PMD_DEBUG_TRACE("VF VLAN filter:invalid port id=%d\n", port_id); return (-ENODEV); @@ -2518,7 +2479,7 @@ int rte_eth_set_queue_rate_limit(uint8_t port_id, uint16_t queue_idx, struct rte_eth_dev_info dev_info; struct rte_eth_link link; - if (port_id >= nb_ports) { + if (rte_eth_dev_validate_port(port_id, NONE_TRACE) == DEV_INVALID) { PMD_DEBUG_TRACE("set queue rate limit:invalid port id=%d\n", port_id); return -ENODEV; @@ -2555,7 +2516,7 @@ int rte_eth_set_vf_rate_limit(uint8_t port_id, uint16_t vf, uint16_t tx_rate, if (q_msk == 0) return 0; - if (port_id >= nb_ports) { + if (rte_eth_dev_validate_port(port_id, NONE_TRACE) == DEV_INVALID) { PMD_DEBUG_TRACE("set VF rate limit:invalid port id=%d\n", port_id); return -ENODEV; @@ -2589,10 +2550,8 @@ rte_eth_mirror_rule_set(uint8_t port_id, { struct rte_eth_dev *dev = &rte_eth_devices[port_id]; - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return (-ENODEV); - } if (mirror_conf->rule_type_mask == 0) { PMD_DEBUG_TRACE("mirror rule type can not be 0.\n"); @@ -2630,10 +2589,8 @@ rte_eth_mirror_rule_reset(uint8_t port_id, uint8_t rule_id) { struct rte_eth_dev *dev = &rte_eth_devices[port_id]; - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return (-ENODEV); - } if(rule_id >= ETH_VMDQ_NUM_MIRROR_RULE) { @@ -2655,10 +2612,9 @@ rte_eth_rx_burst(uint8_t port_id, uint16_t queue_id, { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return 0; - } + dev = &rte_eth_devices[port_id]; FUNC_PTR_OR_ERR_RET(*dev->rx_pkt_burst, 0); if (queue_id >= dev->data->nb_rx_queues) { @@ -2675,10 +2631,9 @@ rte_eth_tx_burst(uint8_t port_id, uint16_t queue_id, { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return 0; - } + dev = &rte_eth_devices[port_id]; FUNC_PTR_OR_ERR_RET(*dev->tx_pkt_burst, 0); @@ -2695,10 +2650,9 @@ rte_eth_rx_queue_count(uint8_t port_id, uint16_t queue_id) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return 0; - } + dev = &rte_eth_devices[port_id]; FUNC_PTR_OR_ERR_RET(*dev->dev_ops->rx_queue_count, 0); return (*dev->dev_ops->rx_queue_count)(dev, queue_id); @@ -2709,10 +2663,9 @@ rte_eth_rx_descriptor_done(uint8_t port_id, uint16_t queue_id, uint16_t offset) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return (-ENODEV); - } + dev = &rte_eth_devices[port_id]; FUNC_PTR_OR_ERR_RET(*dev->dev_ops->rx_descriptor_done, -ENOTSUP); return (*dev->dev_ops->rx_descriptor_done)( \ @@ -2730,10 +2683,8 @@ rte_eth_dev_callback_register(uint8_t port_id, if (!cb_fn) return (-EINVAL); - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return (-EINVAL); - } dev = &rte_eth_devices[port_id]; rte_spinlock_lock(&rte_eth_dev_cb_lock); @@ -2770,10 +2721,8 @@ rte_eth_dev_callback_unregister(uint8_t port_id, if (!cb_fn) return (-EINVAL); - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return (-EINVAL); - } dev = &rte_eth_devices[port_id]; rte_spinlock_lock(&rte_eth_dev_cb_lock); @@ -2830,10 +2779,8 @@ int rte_eth_dev_bypass_init(uint8_t port_id) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return (-ENODEV); - } if ((dev= &rte_eth_devices[port_id]) == NULL) { PMD_DEBUG_TRACE("Invalid port device\n"); @@ -2850,10 +2797,8 @@ rte_eth_dev_bypass_state_show(uint8_t port_id, uint32_t *state) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return (-ENODEV); - } if ((dev= &rte_eth_devices[port_id]) == NULL) { PMD_DEBUG_TRACE("Invalid port device\n"); @@ -2869,10 +2814,8 @@ rte_eth_dev_bypass_state_set(uint8_t port_id, uint32_t *new_state) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return (-ENODEV); - } if ((dev= &rte_eth_devices[port_id]) == NULL) { PMD_DEBUG_TRACE("Invalid port device\n"); @@ -2889,10 +2832,8 @@ rte_eth_dev_bypass_event_show(uint8_t port_id, uint32_t event, uint32_t *state) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return (-ENODEV); - } if ((dev= &rte_eth_devices[port_id]) == NULL) { PMD_DEBUG_TRACE("Invalid port device\n"); @@ -2909,10 +2850,8 @@ rte_eth_dev_bypass_event_store(uint8_t port_id, uint32_t event, uint32_t state) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return (-ENODEV); - } if ((dev= &rte_eth_devices[port_id]) == NULL) { PMD_DEBUG_TRACE("Invalid port device\n"); @@ -2929,10 +2868,8 @@ rte_eth_dev_wd_timeout_store(uint8_t port_id, uint32_t timeout) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return (-ENODEV); - } if ((dev= &rte_eth_devices[port_id]) == NULL) { PMD_DEBUG_TRACE("Invalid port device\n"); @@ -2949,10 +2886,8 @@ rte_eth_dev_bypass_ver_show(uint8_t port_id, uint32_t *ver) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return (-ENODEV); - } if ((dev= &rte_eth_devices[port_id]) == NULL) { PMD_DEBUG_TRACE("Invalid port device\n"); @@ -2969,10 +2904,8 @@ rte_eth_dev_bypass_wd_timeout_show(uint8_t port_id, uint32_t *wd_timeout) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return (-ENODEV); - } if ((dev= &rte_eth_devices[port_id]) == NULL) { PMD_DEBUG_TRACE("Invalid port device\n"); @@ -2989,10 +2922,8 @@ rte_eth_dev_bypass_wd_reset(uint8_t port_id) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return (-ENODEV); - } if ((dev= &rte_eth_devices[port_id]) == NULL) { PMD_DEBUG_TRACE("Invalid port device\n"); @@ -3011,10 +2942,8 @@ rte_eth_dev_add_syn_filter(uint8_t port_id, { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return -ENODEV; - } dev = &rte_eth_devices[port_id]; FUNC_PTR_OR_ERR_RET(*dev->dev_ops->add_syn_filter, -ENOTSUP); @@ -3026,10 +2955,8 @@ rte_eth_dev_remove_syn_filter(uint8_t port_id) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return -ENODEV; - } dev = &rte_eth_devices[port_id]; FUNC_PTR_OR_ERR_RET(*dev->dev_ops->remove_syn_filter, -ENOTSUP); @@ -3045,10 +2972,8 @@ rte_eth_dev_get_syn_filter(uint8_t port_id, if (filter == NULL || rx_queue == NULL) return -EINVAL; - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return -ENODEV; - } dev = &rte_eth_devices[port_id]; FUNC_PTR_OR_ERR_RET(*dev->dev_ops->get_syn_filter, -ENOTSUP); @@ -3061,10 +2986,9 @@ rte_eth_dev_add_2tuple_filter(uint8_t port_id, uint16_t index, { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return -ENODEV; - } + if (filter->protocol != IPPROTO_TCP && filter->tcp_flags != 0){ PMD_DEBUG_TRACE("tcp flags is 0x%x, but the protocol value" @@ -3083,10 +3007,8 @@ rte_eth_dev_remove_2tuple_filter(uint8_t port_id, uint16_t index) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return -ENODEV; - } dev = &rte_eth_devices[port_id]; FUNC_PTR_OR_ERR_RET(*dev->dev_ops->remove_2tuple_filter, -ENOTSUP); @@ -3102,10 +3024,8 @@ rte_eth_dev_get_2tuple_filter(uint8_t port_id, uint16_t index, if (filter == NULL || rx_queue == NULL) return -EINVAL; - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return -ENODEV; - } dev = &rte_eth_devices[port_id]; FUNC_PTR_OR_ERR_RET(*dev->dev_ops->get_2tuple_filter, -ENOTSUP); @@ -3118,10 +3038,8 @@ rte_eth_dev_add_5tuple_filter(uint8_t port_id, uint16_t index, { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return -ENODEV; - } if (filter->protocol != IPPROTO_TCP && filter->tcp_flags != 0){ @@ -3141,10 +3059,8 @@ rte_eth_dev_remove_5tuple_filter(uint8_t port_id, uint16_t index) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return -ENODEV; - } dev = &rte_eth_devices[port_id]; FUNC_PTR_OR_ERR_RET(*dev->dev_ops->remove_5tuple_filter, -ENOTSUP); @@ -3160,10 +3076,8 @@ rte_eth_dev_get_5tuple_filter(uint8_t port_id, uint16_t index, if (filter == NULL || rx_queue == NULL) return -EINVAL; - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return -ENODEV; - } dev = &rte_eth_devices[port_id]; FUNC_PTR_OR_ERR_RET(*dev->dev_ops->get_5tuple_filter, -ENOTSUP); @@ -3177,10 +3091,8 @@ rte_eth_dev_add_flex_filter(uint8_t port_id, uint16_t index, { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return -ENODEV; - } dev = &rte_eth_devices[port_id]; FUNC_PTR_OR_ERR_RET(*dev->dev_ops->add_flex_filter, -ENOTSUP); @@ -3192,10 +3104,8 @@ rte_eth_dev_remove_flex_filter(uint8_t port_id, uint16_t index) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return -ENODEV; - } dev = &rte_eth_devices[port_id]; FUNC_PTR_OR_ERR_RET(*dev->dev_ops->remove_flex_filter, -ENOTSUP); @@ -3211,10 +3121,8 @@ rte_eth_dev_get_flex_filter(uint8_t port_id, uint16_t index, if (filter == NULL || rx_queue == NULL) return -EINVAL; - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return -ENODEV; - } dev = &rte_eth_devices[port_id]; FUNC_PTR_OR_ERR_RET(*dev->dev_ops->get_flex_filter, -ENOTSUP); @@ -3227,10 +3135,8 @@ rte_eth_dev_filter_supported(uint8_t port_id, enum rte_filter_type filter_type) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return -ENODEV; - } dev = &rte_eth_devices[port_id]; FUNC_PTR_OR_ERR_RET(*dev->dev_ops->filter_ctrl, -ENOTSUP); @@ -3244,10 +3150,8 @@ rte_eth_dev_filter_ctrl(uint8_t port_id, enum rte_filter_type filter_type, { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return -ENODEV; - } dev = &rte_eth_devices[port_id]; FUNC_PTR_OR_ERR_RET(*dev->dev_ops->filter_ctrl, -ENOTSUP); diff --git a/lib/librte_ether/rte_ethdev.h b/lib/librte_ether/rte_ethdev.h index 1200c1c..ca101f5 100644 --- a/lib/librte_ether/rte_ethdev.h +++ b/lib/librte_ether/rte_ethdev.h @@ -1539,6 +1539,7 @@ struct rte_eth_dev { struct eth_dev_ops *dev_ops; /**< Functions exported by PMD */ struct rte_pci_device *pci_dev; /**< PCI info. supplied by probing */ struct rte_eth_dev_cb_list callbacks; /**< User application callbacks */ + uint8_t attached; /**< Flag indicating the port is attached */ }; struct rte_eth_dev_sriov { @@ -1604,6 +1605,10 @@ extern struct rte_eth_dev rte_eth_devices[]; * initialized by the [matching] Ethernet driver during the PCI probing phase. * All devices whose port identifier is in the range * [0, rte_eth_dev_count() - 1] can be operated on by network applications. + * immediately after invoking rte_eal_init(). + * If the application unplugs a port using hotplug function, The enabled port + * numbers may be noncontiguous. In the case, the applications need to manage + * enabled port by themselves. * * @return * - The total number of usable Ethernet devices. -- 1.9.1
This patch replaces pci_addr_comparison() and memcmp() of pci addresses by eal_compare_pci_addr(). v5: - Fix pci_scan_one to handle pt_driver correctly. v4: - Fix calculation method of eal_compare_pci_addr(). - Add parameter checking. Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp> --- lib/librte_eal/bsdapp/eal/eal_pci.c | 25 ++++++++--------------- lib/librte_eal/common/eal_common_pci.c | 2 +- lib/librte_eal/common/include/rte_pci.h | 34 +++++++++++++++++++++++++++++++ lib/librte_eal/linuxapp/eal/eal_pci.c | 25 ++++++++--------------- lib/librte_eal/linuxapp/eal/eal_pci_uio.c | 2 +- 5 files changed, 54 insertions(+), 34 deletions(-) diff --git a/lib/librte_eal/bsdapp/eal/eal_pci.c b/lib/librte_eal/bsdapp/eal/eal_pci.c index 74ecce7..c844d58 100644 --- a/lib/librte_eal/bsdapp/eal/eal_pci.c +++ b/lib/librte_eal/bsdapp/eal/eal_pci.c @@ -270,20 +270,6 @@ pci_uio_map_resource(struct rte_pci_device *dev) return (0); } -/* Compare two PCI device addresses. */ -static int -pci_addr_comparison(struct rte_pci_addr *addr, struct rte_pci_addr *addr2) -{ - uint64_t dev_addr = (addr->domain << 24) + (addr->bus << 16) + (addr->devid << 8) + addr->function; - uint64_t dev_addr2 = (addr2->domain << 24) + (addr2->bus << 16) + (addr2->devid << 8) + addr2->function; - - if (dev_addr > dev_addr2) - return 1; - else - return 0; -} - - /* Scan one pci sysfs entry, and fill the devices list from it. */ static int pci_scan_one(int dev_pci_fd, struct pci_conf *conf) @@ -356,13 +342,20 @@ pci_scan_one(int dev_pci_fd, struct pci_conf *conf) } else { struct rte_pci_device *dev2 = NULL; + int ret; TAILQ_FOREACH(dev2, &pci_device_list, next) { - if (pci_addr_comparison(&dev->addr, &dev2->addr)) + ret = eal_compare_pci_addr(&dev->addr, &dev2->addr); + if (ret > 0) continue; - else { + else if (ret < 0) { TAILQ_INSERT_BEFORE(dev2, dev, next); return 0; + } else { /* already registered */ + /* update pt_driver */ + dev2->pt_driver = dev->pt_driver; + free(dev); + return 0; } } TAILQ_INSERT_TAIL(&pci_device_list, dev, next); diff --git a/lib/librte_eal/common/eal_common_pci.c b/lib/librte_eal/common/eal_common_pci.c index f3c7f71..a89f5c3 100644 --- a/lib/librte_eal/common/eal_common_pci.c +++ b/lib/librte_eal/common/eal_common_pci.c @@ -93,7 +93,7 @@ static struct rte_devargs *pci_devargs_lookup(struct rte_pci_device *dev) if (devargs->type != RTE_DEVTYPE_BLACKLISTED_PCI && devargs->type != RTE_DEVTYPE_WHITELISTED_PCI) continue; - if (!memcmp(&dev->addr, &devargs->pci.addr, sizeof(dev->addr))) + if (!eal_compare_pci_addr(&dev->addr, &devargs->pci.addr)) return devargs; } return NULL; diff --git a/lib/librte_eal/common/include/rte_pci.h b/lib/librte_eal/common/include/rte_pci.h index 7f2d699..4814cd7 100644 --- a/lib/librte_eal/common/include/rte_pci.h +++ b/lib/librte_eal/common/include/rte_pci.h @@ -269,6 +269,40 @@ eal_parse_pci_DomBDF(const char *input, struct rte_pci_addr *dev_addr) } #undef GET_PCIADDR_FIELD +/* Compare two PCI device addresses. */ +/** + * Utility function to compare two PCI device addresses. + * + * @param addr + * The PCI Bus-Device-Function address to compare + * @param addr2 + * The PCI Bus-Device-Function address to compare + * @return + * 0 on equal PCI address. + * Positive on addr is greater than addr2. + * Negative on addr is less than addr2, or error. + */ +static inline int +eal_compare_pci_addr(struct rte_pci_addr *addr, struct rte_pci_addr *addr2) +{ + uint64_t dev_addr, dev_addr2; + + if ((addr == NULL) || (addr2 == NULL)) + return -1; + + dev_addr = (addr->domain << 24) | (addr->bus << 16) | + (addr->devid << 8) | addr->function; + dev_addr2 = (addr2->domain << 24) | (addr2->bus << 16) | + (addr2->devid << 8) | addr2->function; + + if (dev_addr > dev_addr2) + return 1; + else if (dev_addr < dev_addr2) + return -1; + else + return 0; +} + /** * Probe the PCI bus for registered drivers. * diff --git a/lib/librte_eal/linuxapp/eal/eal_pci.c b/lib/librte_eal/linuxapp/eal/eal_pci.c index c0ca5a5..d847102 100644 --- a/lib/librte_eal/linuxapp/eal/eal_pci.c +++ b/lib/librte_eal/linuxapp/eal/eal_pci.c @@ -229,20 +229,6 @@ error: return -1; } -/* Compare two PCI device addresses. */ -static int -pci_addr_comparison(struct rte_pci_addr *addr, struct rte_pci_addr *addr2) -{ - uint64_t dev_addr = (addr->domain << 24) + (addr->bus << 16) + (addr->devid << 8) + addr->function; - uint64_t dev_addr2 = (addr2->domain << 24) + (addr2->bus << 16) + (addr2->devid << 8) + addr2->function; - - if (dev_addr > dev_addr2) - return 1; - else - return 0; -} - - /* Scan one pci sysfs entry, and fill the devices list from it. */ static int pci_scan_one(const char *dirname, uint16_t domain, uint8_t bus, @@ -353,13 +339,20 @@ pci_scan_one(const char *dirname, uint16_t domain, uint8_t bus, } else { struct rte_pci_device *dev2 = NULL; + int ret; TAILQ_FOREACH(dev2, &pci_device_list, next) { - if (pci_addr_comparison(&dev->addr, &dev2->addr)) + ret = eal_compare_pci_addr(&dev->addr, &dev2->addr); + if (ret > 0) continue; - else { + else if (ret < 0) { TAILQ_INSERT_BEFORE(dev2, dev, next); return 0; + } else { /* already registered */ + /* update pt_driver */ + dev2->pt_driver = dev->pt_driver; + free(dev); + return 0; } } TAILQ_INSERT_TAIL(&pci_device_list, dev, next); diff --git a/lib/librte_eal/linuxapp/eal/eal_pci_uio.c b/lib/librte_eal/linuxapp/eal/eal_pci_uio.c index e53f06b..1da3507 100644 --- a/lib/librte_eal/linuxapp/eal/eal_pci_uio.c +++ b/lib/librte_eal/linuxapp/eal/eal_pci_uio.c @@ -123,7 +123,7 @@ pci_uio_map_secondary(struct rte_pci_device *dev) TAILQ_FOREACH(uio_res, pci_res_list, next) { /* skip this element if it doesn't match our PCI address */ - if (memcmp(&uio_res->pci_addr, &dev->addr, sizeof(dev->addr))) + if (eal_compare_pci_addr(&uio_res->pci_addr, &dev->addr)) continue; for (i = 0; i != uio_res->nb_maps; i++) { -- 1.9.1
This patch adds rte_eth_dev_free(). The function is used for changing a attached status of the device that has specified name. v4: - Add paramerter checking. Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp> --- lib/librte_ether/rte_ethdev.c | 20 ++++++++++++++++++++ lib/librte_ether/rte_ethdev.h | 11 +++++++++++ 2 files changed, 31 insertions(+) diff --git a/lib/librte_ether/rte_ethdev.c b/lib/librte_ether/rte_ethdev.c index d70854f..0f3094f 100644 --- a/lib/librte_ether/rte_ethdev.c +++ b/lib/librte_ether/rte_ethdev.c @@ -260,6 +260,26 @@ rte_eth_dev_allocate(const char *name) return eth_dev; } +struct rte_eth_dev * +rte_eth_dev_free(const char *name) +{ + struct rte_eth_dev *eth_dev; + + if (name == NULL) + return NULL; + + eth_dev = rte_eth_dev_allocated(name); + if (eth_dev == NULL) { + PMD_DEBUG_TRACE("Ethernet Device with name %s doesn't exist!\n", + name); + return NULL; + } + + eth_dev->attached = 0; + nb_ports--; + return eth_dev; +} + static int rte_eth_dev_init(struct rte_pci_driver *pci_drv, struct rte_pci_device *pci_dev) diff --git a/lib/librte_ether/rte_ethdev.h b/lib/librte_ether/rte_ethdev.h index ca101f5..6add058 100644 --- a/lib/librte_ether/rte_ethdev.h +++ b/lib/librte_ether/rte_ethdev.h @@ -1627,6 +1627,17 @@ extern uint8_t rte_eth_dev_count(void); */ struct rte_eth_dev *rte_eth_dev_allocate(const char *name); +/** + * Function for internal use by dummy drivers primarily, e.g. ring-based + * driver. + * Free the specified ethdev and returns the pointer to that slot. + * + * @param name Unique identifier name for each Ethernet device + * @return + * - Slot in the rte_dev_devices array for the freed device; + */ +struct rte_eth_dev *rte_eth_dev_free(const char *name); + struct eth_driver; /** * @internal -- 1.9.1
The patch adds function pointer to rte_pci_driver and eth_driver structure. These function pointers are used when ports are detached. Also the patch adds rte_eth_dev_uninit(). So far, it's not called by anywhere, but it will be called when port hotplug function is implemented. v4: - Add paramerter checking. - Change function names. Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp> --- lib/librte_eal/common/include/rte_pci.h | 7 ++++++ lib/librte_ether/rte_ethdev.c | 40 +++++++++++++++++++++++++++++++++ lib/librte_ether/rte_ethdev.h | 24 ++++++++++++++++++++ 3 files changed, 71 insertions(+) diff --git a/lib/librte_eal/common/include/rte_pci.h b/lib/librte_eal/common/include/rte_pci.h index 4814cd7..87ca4cf 100644 --- a/lib/librte_eal/common/include/rte_pci.h +++ b/lib/librte_eal/common/include/rte_pci.h @@ -189,12 +189,19 @@ struct rte_pci_driver; typedef int (pci_devinit_t)(struct rte_pci_driver *, struct rte_pci_device *); /** + * Uninitialisation function for the driver called during hotplugging. + */ +typedef int (pci_devuninit_t)( + struct rte_pci_driver *, struct rte_pci_device *); + +/** * A structure describing a PCI driver. */ struct rte_pci_driver { TAILQ_ENTRY(rte_pci_driver) next; /**< Next in list. */ const char *name; /**< Driver name. */ pci_devinit_t *devinit; /**< Device init. function. */ + pci_devuninit_t *devuninit; /**< Device uninit function. */ struct rte_pci_id *id_table; /**< ID table, NULL terminated. */ uint32_t drv_flags; /**< Flags contolling handling of device. */ }; diff --git a/lib/librte_ether/rte_ethdev.c b/lib/librte_ether/rte_ethdev.c index 0f3094f..fd19140 100644 --- a/lib/librte_ether/rte_ethdev.c +++ b/lib/librte_ether/rte_ethdev.c @@ -335,6 +335,45 @@ rte_eth_dev_init(struct rte_pci_driver *pci_drv, return diag; } +static int +rte_eth_dev_uninit(struct rte_pci_driver *pci_drv, + struct rte_pci_device *pci_dev) +{ + struct eth_driver *eth_drv; + struct rte_eth_dev *eth_dev; + char ethdev_name[RTE_ETH_NAME_MAX_LEN]; + + if ((pci_drv == NULL) || (pci_dev == NULL)) + return -EINVAL; + + /* Create unique Ethernet device name using PCI address */ + snprintf(ethdev_name, RTE_ETH_NAME_MAX_LEN, "%d:%d.%d", + pci_dev->addr.bus, pci_dev->addr.devid, + pci_dev->addr.function); + + eth_dev = rte_eth_dev_free(ethdev_name); + if (eth_dev == NULL) + return -ENODEV; + + eth_drv = (struct eth_driver *)pci_drv; + + /* Invoke PMD device uninit function */ + if (*eth_drv->eth_dev_uninit) + (*eth_drv->eth_dev_uninit)(eth_drv, eth_dev); + + /* init user callbacks */ + TAILQ_INIT(&(eth_dev->callbacks)); + + if (rte_eal_process_type() == RTE_PROC_PRIMARY) + rte_free(eth_dev->data->dev_private); + + eth_dev->pci_dev = NULL; + eth_dev->driver = NULL; + eth_dev->data = NULL; + + return 0; +} + /** * Register an Ethernet [Poll Mode] driver. * @@ -353,6 +392,7 @@ void rte_eth_driver_register(struct eth_driver *eth_drv) { eth_drv->pci_drv.devinit = rte_eth_dev_init; + eth_drv->pci_drv.devuninit = rte_eth_dev_uninit; rte_eal_pci_register(ð_drv->pci_drv); } diff --git a/lib/librte_ether/rte_ethdev.h b/lib/librte_ether/rte_ethdev.h index 6add058..0b4c27c 100644 --- a/lib/librte_ether/rte_ethdev.h +++ b/lib/librte_ether/rte_ethdev.h @@ -1675,6 +1675,27 @@ typedef int (*eth_dev_init_t)(struct eth_driver *eth_drv, /** * @internal + * Finalization function of an Ethernet driver invoked for each matching + * Ethernet PCI device detected during the PCI closing phase. + * + * @param eth_drv + * The pointer to the [matching] Ethernet driver structure supplied by + * the PMD when it registered itself. + * @param eth_dev + * The *eth_dev* pointer is the address of the *rte_eth_dev* structure + * associated with the matching device and which have been [automatically] + * allocated in the *rte_eth_devices* array. + * @return + * - 0: Success, the device is properly finalized by the driver. + * In particular, the driver MUST free the *dev_ops* pointer + * of the *eth_dev* structure. + * - <0: Error code of the device initialization failure. + */ +typedef int (*eth_dev_uninit_t)(struct eth_driver *eth_drv, + struct rte_eth_dev *eth_dev); + +/** + * @internal * The structure associated with a PMD Ethernet driver. * * Each Ethernet driver acts as a PCI driver and is represented by a generic @@ -1684,11 +1705,14 @@ typedef int (*eth_dev_init_t)(struct eth_driver *eth_drv, * * - The *eth_dev_init* function invoked for each matching PCI device. * + * - The *eth_dev_uninit* function invoked for each matching PCI device. + * * - The size of the private data to allocate for each matching device. */ struct eth_driver { struct rte_pci_driver pci_drv; /**< The PMD is also a PCI driver. */ eth_dev_init_t eth_dev_init; /**< Device init function. */ + eth_dev_uninit_t eth_dev_uninit; /**< Device uninit function. */ unsigned int dev_private_size; /**< Size of device private data. */ }; -- 1.9.1
The patch adds following functions. - rte_eth_dev_save() The function is used for saving current rte_eth_dev structures. - rte_eth_dev_get_changed_port() The function receives the rte_eth_dev structures, then compare these with current values to know which port is actually attached or detached. - rte_eth_dev_get_addr_by_port() The function returns a pci address of a ethdev specified by port identifier. - rte_eth_dev_get_port_by_addr() The function returns a port identifier of a ethdev specified by pci address. - rte_eth_dev_get_name_by_port() The function returns a unique identifier name of a ethdev specified by port identifier. - Add rte_eth_dev_check_detachable() The function returns whether a PMD supports detach function. Also the patch changes scope of rte_eth_dev_allocated() to global. This function will be called by virtual PMDs to support port hotplug. So change scope of the function to global. v5: - Fix return value of below functions. rte_eth_dev_get_changed_port(). rte_eth_dev_get_port_by_addr(). v4: - Add paramerter checking. v3: - Fix if-condition bug while comparing pci addresses. - Add error checking codes. Reported-by: Mark Enright <menrigh@brocade.com> Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp> --- lib/librte_ether/rte_ethdev.c | 98 ++++++++++++++++++++++++++++++++++++++++++- lib/librte_ether/rte_ethdev.h | 80 +++++++++++++++++++++++++++++++++++ 2 files changed, 177 insertions(+), 1 deletion(-) diff --git a/lib/librte_ether/rte_ethdev.c b/lib/librte_ether/rte_ethdev.c index fd19140..2acafc7 100644 --- a/lib/librte_ether/rte_ethdev.c +++ b/lib/librte_ether/rte_ethdev.c @@ -206,7 +206,7 @@ rte_eth_dev_data_alloc(void) RTE_MAX_ETHPORTS * sizeof(*rte_eth_dev_data)); } -static struct rte_eth_dev * +struct rte_eth_dev * rte_eth_dev_allocated(const char *name) { unsigned i; @@ -428,6 +428,102 @@ rte_eth_dev_count(void) return (nb_ports); } +void +rte_eth_dev_save(struct rte_eth_dev *devs) +{ + if (devs == NULL) + return; + + /* save current rte_eth_devices */ + memcpy(devs, rte_eth_devices, + sizeof(struct rte_eth_dev) * RTE_MAX_ETHPORTS); +} + +int +rte_eth_dev_get_changed_port(struct rte_eth_dev *devs, uint8_t *port_id) +{ + if ((devs == NULL) || (port_id == NULL)) + return -EINVAL; + + /* check which port was attached or detached */ + for (*port_id = 0; *port_id < RTE_MAX_ETHPORTS; (*port_id)++, devs++) { + if (rte_eth_devices[*port_id].attached ^ devs->attached) + return 0; + } + return -ENODEV; +} + +int +rte_eth_dev_get_addr_by_port(uint8_t port_id, struct rte_pci_addr *addr) +{ + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) + return -EINVAL; + + if (addr == NULL) { + PMD_DEBUG_TRACE("Null pointer is specified\n"); + return -EINVAL; + } + + *addr = rte_eth_devices[port_id].pci_dev->addr; + return 0; +} + +int +rte_eth_dev_get_port_by_addr(struct rte_pci_addr *addr, uint8_t *port_id) +{ + struct rte_pci_addr *tmp; + + if ((addr == NULL) || (port_id == NULL)) { + PMD_DEBUG_TRACE("Null pointer is specified\n"); + return -EINVAL; + } + + for (*port_id = 0; *port_id < RTE_MAX_ETHPORTS; (*port_id)++) { + if (!rte_eth_devices[*port_id].attached) + continue; + if (!rte_eth_devices[*port_id].pci_dev) + continue; + tmp = &rte_eth_devices[*port_id].pci_dev->addr; + if (eal_compare_pci_addr(tmp, addr) == 0) + return 0; + } + return -ENODEV; +} + +int +rte_eth_dev_get_name_by_port(uint8_t port_id, char *name) +{ + char *tmp; + + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) + return -EINVAL; + + if (name == NULL) { + PMD_DEBUG_TRACE("Null pointer is specified\n"); + return -EINVAL; + } + + /* shouldn't check 'rte_eth_devices[i].data', + * because it might be overwritten by VDEV PMD */ + tmp = rte_eth_dev_data[port_id].name; + strncpy(name, tmp, strlen(tmp) + 1); + return 0; +} + +int +rte_eth_dev_check_detachable(uint8_t port_id) +{ + uint32_t drv_flags; + + if (port_id >= RTE_MAX_ETHPORTS) { + PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + return -EINVAL; + } + + drv_flags = rte_eth_devices[port_id].driver->pci_drv.drv_flags; + return !(drv_flags & RTE_PCI_DRV_DETACHABLE); +} + static int rte_eth_dev_rx_queue_config(struct rte_eth_dev *dev, uint16_t nb_queues) { diff --git a/lib/librte_ether/rte_ethdev.h b/lib/librte_ether/rte_ethdev.h index 0b4c27c..ebc48b0 100644 --- a/lib/librte_ether/rte_ethdev.h +++ b/lib/librte_ether/rte_ethdev.h @@ -1616,6 +1616,86 @@ extern struct rte_eth_dev rte_eth_devices[]; extern uint8_t rte_eth_dev_count(void); /** + * Function for internal use by port hotplug functions. + * Copies current ethdev structures to the specified pointer. + * + * @param devs The pointer to the ethdev structures + */ +extern void rte_eth_dev_save(struct rte_eth_dev *devs); + +/** + * Function for internal use by port hotplug functions. + * Compare the specified ethdev structures with currents. Then + * if there is a port which status is changed, fill the specified pointer + * with the port id of that port. + * @param devs The pointer to the ethdev structures + * @param port_id The pointer to the port id + * @return + * - 0 on success, negative on error + */ +extern int rte_eth_dev_get_changed_port( + struct rte_eth_dev *devs, uint8_t *port_id); + +/** + * Function for internal use by port hotplug functions. + * Returns a pci address of a ethdev specified by port identifier. + * @param port_id + * The port identifier of the Ethernet device + * @param addr + * The pointer to the pci address + * @return + * - 0 on success, negative on error + */ +extern int rte_eth_dev_get_addr_by_port( + uint8_t port_id, struct rte_pci_addr *addr); + +/** + * Function for internal use by port hotplug functions. + * Returns a port identifier of a ethdev specified by pci address. + * @param addr + * The pointer to the pci address of the Ethernet device. + * @param port_id + * The pointer to the port identifier + * @return + * - 0 on success, negative on error + */ +extern int rte_eth_dev_get_port_by_addr( + struct rte_pci_addr *addr, uint8_t *port_id); + +/** + * Function for internal use by port hotplug functions. + * Returns a unique identifier name of a ethdev specified by port identifier. + * @param port_id + * The port identifier. + * @param name + * The pointer to the Unique identifier name for each Ethernet device + * @return + * - 0 on success, negative on error + */ +extern int rte_eth_dev_get_name_by_port(uint8_t port_id, char *name); + +/** + * Function for internal use by port hotplug functions. + * Check whether or not, a PMD that is handling the ethdev specified by port + * identifier can support detach function. + * @param port_id + * The port identifier + * @return + * - 0 on supporting detach function, negative on not supporting + */ +extern int rte_eth_dev_check_detachable(uint8_t port_id); + +/** + * Function for internal use by port hotplug functions. + * Returns a ethdev slot specified by the unique identifier name. + * @param name + * The pointer to the Unique identifier name for each Ethernet device + * @return + * - The pointer to the ethdev slot, on success. NULL on error + */ +extern struct rte_eth_dev *rte_eth_dev_allocated(const char *name); + +/** * Function for internal use by dummy drivers primarily, e.g. ring-based * driver. * Allocates a new ethdev slot for an ethernet device and returns the pointer -- 1.9.1
The patch adds functions for unmapping igb_uio resources. The patch is only for Linux and igb_uio environment. VFIO and BSD are not supported. v5: - Fix pci_unmap_device() to check pt_driver. v4: - Add paramerter checking. - Add header file to determine if hotplug can be enabled. Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp> --- lib/librte_eal/common/Makefile | 1 + lib/librte_eal/common/include/rte_dev_hotplug.h | 44 +++++++++++++++++ lib/librte_eal/linuxapp/eal/eal_pci.c | 44 +++++++++++++++++ lib/librte_eal/linuxapp/eal/eal_pci_init.h | 8 +++ lib/librte_eal/linuxapp/eal/eal_pci_uio.c | 65 +++++++++++++++++++++++++ 5 files changed, 162 insertions(+) create mode 100644 lib/librte_eal/common/include/rte_dev_hotplug.h diff --git a/lib/librte_eal/common/Makefile b/lib/librte_eal/common/Makefile index 52c1a5f..db7cc93 100644 --- a/lib/librte_eal/common/Makefile +++ b/lib/librte_eal/common/Makefile @@ -41,6 +41,7 @@ INC += rte_eal_memconfig.h rte_malloc_heap.h INC += rte_hexdump.h rte_devargs.h rte_dev.h INC += rte_common_vect.h INC += rte_pci_dev_feature_defs.h rte_pci_dev_features.h +INC += rte_dev_hotplug.h ifeq ($(CONFIG_RTE_INSECURE_FUNCTION_WARNING),y) INC += rte_warnings.h diff --git a/lib/librte_eal/common/include/rte_dev_hotplug.h b/lib/librte_eal/common/include/rte_dev_hotplug.h new file mode 100644 index 0000000..b333e0f --- /dev/null +++ b/lib/librte_eal/common/include/rte_dev_hotplug.h @@ -0,0 +1,44 @@ +/*- + * BSD LICENSE + * + * Copyright(c) 2015 IGEL Co.,LTd. + * All rights reserved. + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * + * * Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * * Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in + * the documentation and/or other materials provided with the + * distribution. + * * Neither the name of IGEL Co.,Ltd. nor the names of its + * contributors may be used to endorse or promote products derived + * from this software without specific prior written permission. + * + * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS + * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT + * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR + * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT + * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, + * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT + * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, + * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY + * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT + * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE + * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + */ + +#ifndef _RTE_DEV_HOTPLUG_H_ +#define _RTE_DEV_HOTPLUG_H_ + +/* + * determine if hotplug can be enabled on the system + */ +#if defined(RTE_LIBRTE_EAL_HOTPLUG) && defined(RTE_LIBRTE_EAL_LINUXAPP) +#define ENABLE_HOTPLUG +#endif /* RTE_LIBRTE_EAL_HOTPLUG & RTE_LIBRTE_EAL_LINUXAPP */ + +#endif /* _RTE_DEV_HOTPLUG_H_ */ diff --git a/lib/librte_eal/linuxapp/eal/eal_pci.c b/lib/librte_eal/linuxapp/eal/eal_pci.c index d847102..c3b7917 100644 --- a/lib/librte_eal/linuxapp/eal/eal_pci.c +++ b/lib/librte_eal/linuxapp/eal/eal_pci.c @@ -166,6 +166,25 @@ pci_map_resource(void *requested_addr, int fd, off_t offset, size_t size) return mapaddr; } +#ifdef ENABLE_HOTPLUG +/* unmap a particular resource */ +void +pci_unmap_resource(void *requested_addr, size_t size) +{ + if (requested_addr == NULL) + return; + + /* Unmap the PCI memory resource of device */ + if (munmap(requested_addr, size)) { + RTE_LOG(ERR, EAL, "%s(): cannot munmap(%p, 0x%lx): %s\n", + __func__, requested_addr, (unsigned long)size, + strerror(errno)); + } else + RTE_LOG(DEBUG, EAL, " PCI memory mapped at %p\n", + requested_addr); +} +#endif /* ENABLE_HOTPLUG */ + /* parse the "resource" sysfs file */ #define IORESOURCE_MEM 0x00000200 @@ -567,6 +586,31 @@ pci_map_device(struct rte_pci_device *dev) return ret; } +#ifdef ENABLE_HOTPLUG +static void +pci_unmap_device(struct rte_pci_device *dev) +{ + if (dev == NULL) + return; + + /* try unmapping the NIC resources using VFIO if it exists */ + switch (dev->pt_driver) { + case RTE_PT_VFIO: + RTE_LOG(ERR, EAL, "Hotplug doesn't support vfio yet\n"); + break; + case RTE_PT_IGB_UIO: + case RTE_PT_UIO_GENERIC: + /* unmap resources for devices that use uio */ + pci_uio_unmap_resource(dev); + break; + default: + RTE_LOG(DEBUG, EAL, " Not managed by known pt driver," + " skipped\n"); + break; + } +} +#endif /* ENABLE_HOTPLUG */ + /* * If vendor/device ID match, call the devinit() function of the * driver. diff --git a/lib/librte_eal/linuxapp/eal/eal_pci_init.h b/lib/librte_eal/linuxapp/eal/eal_pci_init.h index 1070eb8..5152a0b 100644 --- a/lib/librte_eal/linuxapp/eal/eal_pci_init.h +++ b/lib/librte_eal/linuxapp/eal/eal_pci_init.h @@ -34,6 +34,7 @@ #ifndef EAL_PCI_INIT_H_ #define EAL_PCI_INIT_H_ +#include <rte_dev_hotplug.h> #include "eal_vfio.h" struct pci_map { @@ -71,6 +72,13 @@ void *pci_map_resource(void *requested_addr, int fd, off_t offset, /* map IGB_UIO resource prototype */ int pci_uio_map_resource(struct rte_pci_device *dev); +#ifdef ENABLE_HOTPLUG +void pci_unmap_resource(void *requested_addr, size_t size); + +/* unmap IGB_UIO resource prototype */ +void pci_uio_unmap_resource(struct rte_pci_device *dev); +#endif /* ENABLE_HOTPLUG */ + #ifdef VFIO_PRESENT #define VFIO_MAX_GROUPS 64 diff --git a/lib/librte_eal/linuxapp/eal/eal_pci_uio.c b/lib/librte_eal/linuxapp/eal/eal_pci_uio.c index 1da3507..81830d1 100644 --- a/lib/librte_eal/linuxapp/eal/eal_pci_uio.c +++ b/lib/librte_eal/linuxapp/eal/eal_pci_uio.c @@ -404,6 +404,71 @@ pci_uio_map_resource(struct rte_pci_device *dev) return 0; } +#ifdef ENABLE_HOTPLUG +static void +pci_uio_unmap(struct mapped_pci_resource *uio_res) +{ + int i; + + if (uio_res == NULL) + return; + + for (i = 0; i != uio_res->nb_maps; i++) + pci_unmap_resource(uio_res->maps[i].addr, + (size_t)uio_res->maps[i].size); +} + +static struct mapped_pci_resource * +pci_uio_find_resource(struct rte_pci_device *dev) +{ + struct mapped_pci_resource *uio_res; + + if (dev == NULL) + return NULL; + + TAILQ_FOREACH(uio_res, pci_res_list, next) { + + /* skip this element if it doesn't match our PCI address */ + if (!eal_compare_pci_addr(&uio_res->pci_addr, &dev->addr)) + return uio_res; + } + return NULL; +} + +/* unmap the PCI resource of a PCI device in virtual memory */ +void +pci_uio_unmap_resource(struct rte_pci_device *dev) +{ + struct mapped_pci_resource *uio_res; + + if (dev == NULL) + return; + + /* find an entry for the device */ + uio_res = pci_uio_find_resource(dev); + if (uio_res == NULL) + return; + + /* secondary processes - just free maps */ + if (rte_eal_process_type() != RTE_PROC_PRIMARY) + return pci_uio_unmap(uio_res); + + TAILQ_REMOVE(pci_res_list, uio_res, next); + + /* unmap all resources */ + pci_uio_unmap(uio_res); + + /* free uio resource */ + rte_free(uio_res); + + /* close fd if in primary process */ + close(dev->intr_handle.fd); + + dev->intr_handle.fd = -1; + dev->intr_handle.type = RTE_INTR_HANDLE_UNKNOWN; +} +#endif /* ENABLE_HOTPLUG */ + /* * parse a sysfs file containing one integer value * different to the eal version, as it needs to work with 64-bit values -- 1.9.1
The function removes the specified devargs entry from devargs_list. Also the patch adds sanity checking to rte_eal_devargs_add(). v5: - Change function definition of rte_eal_devargs_remove(). v4: - Fix sanity check code. Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp> --- lib/librte_eal/common/eal_common_devargs.c | 60 +++++++++++++++++++++++++++++ lib/librte_eal/common/include/rte_devargs.h | 21 ++++++++++ 2 files changed, 81 insertions(+) diff --git a/lib/librte_eal/common/eal_common_devargs.c b/lib/librte_eal/common/eal_common_devargs.c index 4c7d11a..5b1ac8e 100644 --- a/lib/librte_eal/common/eal_common_devargs.c +++ b/lib/librte_eal/common/eal_common_devargs.c @@ -44,6 +44,35 @@ struct rte_devargs_list devargs_list = TAILQ_HEAD_INITIALIZER(devargs_list); + +/* find a entry specified by pci address or device name */ +static struct rte_devargs * +rte_eal_devargs_find(enum rte_devtype devtype, void *args) +{ + struct rte_devargs *devargs; + + if (args == NULL) + return NULL; + + TAILQ_FOREACH(devargs, &devargs_list, next) { + switch (devtype) { + case RTE_DEVTYPE_WHITELISTED_PCI: + case RTE_DEVTYPE_BLACKLISTED_PCI: + if (eal_compare_pci_addr(&devargs->pci.addr, args) == 0) + goto found; + break; + case RTE_DEVTYPE_VIRTUAL: + if (memcmp(&devargs->virtual.drv_name, args, + strlen((char *)args)) == 0) + goto found; + break; + } + } + return NULL; +found: + return devargs; +} + /* store a whitelist parameter for later parsing */ int rte_eal_devargs_add(enum rte_devtype devtype, const char *devargs_str) @@ -87,6 +116,12 @@ rte_eal_devargs_add(enum rte_devtype devtype, const char *devargs_str) free(devargs); return -1; } + /* make sure there is no same entry */ + if (rte_eal_devargs_find(devtype, &devargs->pci.addr)) { + RTE_LOG(ERR, EAL, + "device already registered: <%s>\n", buf); + return -1; + } break; case RTE_DEVTYPE_VIRTUAL: /* save driver name */ @@ -98,6 +133,12 @@ rte_eal_devargs_add(enum rte_devtype devtype, const char *devargs_str) free(devargs); return -1; } + /* make sure there is no same entry */ + if (rte_eal_devargs_find(devtype, &devargs->virtual.drv_name)) { + RTE_LOG(ERR, EAL, + "device already registered: <%s>\n", buf); + return -1; + } break; } @@ -105,6 +146,25 @@ rte_eal_devargs_add(enum rte_devtype devtype, const char *devargs_str) return 0; } +/* remove it from the devargs_list */ +int +rte_eal_devargs_remove(enum rte_devtype devtype, void *args) +{ + struct rte_devargs *devargs; + + if (args == NULL) + return -EINVAL; + + devargs = rte_eal_devargs_find(devtype, args); + if (devargs == NULL) { + RTE_LOG(ERR, EAL, "device not found\n"); + return -ENODEV; + } + + TAILQ_REMOVE(&devargs_list, devargs, next); + return 0; +} + /* count the number of devices of a specified type */ unsigned int rte_eal_devargs_type_count(enum rte_devtype devtype) diff --git a/lib/librte_eal/common/include/rte_devargs.h b/lib/librte_eal/common/include/rte_devargs.h index 9f9c98f..b5ad4b3 100644 --- a/lib/librte_eal/common/include/rte_devargs.h +++ b/lib/librte_eal/common/include/rte_devargs.h @@ -123,6 +123,27 @@ extern struct rte_devargs_list devargs_list; int rte_eal_devargs_add(enum rte_devtype devtype, const char *devargs_str); /** + * Remove a device from the user device list + * + * For PCI devices, the format of arguments string is "PCI_ADDR". It shouldn't + * involves parameters for the device. Example: "08:00.1". + * + * For virtual devices, the format of arguments string is "DRIVER_NAME*". It + * shouldn't involves parameters for the device. Example: "eth_ring". The + * validity of the driver name is not checked by this function, it is done + * when closing the drivers. + * + * @param devtype + * The type of the device. + * @param name + * The name of the device. + * + * @return + * - 0 on success, negative on error + */ +int rte_eal_devargs_remove(enum rte_devtype devtype, void *args); + +/** * Count the number of user devices of a specified type * * @param devtype -- 1.9.1
- Add rte_eal_pci_close_one_dirver() The function is used for closing the specified driver and device. - Add pci_invoke_all_drivers() The function is based on pci_probe_all_drivers. But it can not only probe but also close drivers. - Add pci_close_all_drivers() The function tries to find a driver for the specified device, and then close the driver. - Add rte_eal_pci_probe_one() and rte_eal_pci_close_one() The functions are used for probe and close a device. First the function tries to find a device that has the specfied PCI address. Then, probe or close the device. v5: - Remove RTE_EAL_INVOKE_TYPE_UNKNOWN, because it's unused. v4: - Fix paramerter checking. - Fix indent of 'if' statement. Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp> --- lib/librte_eal/common/eal_common_pci.c | 90 +++++++++++++++++++++++++++++---- lib/librte_eal/common/eal_private.h | 24 +++++++++ lib/librte_eal/common/include/rte_pci.h | 33 ++++++++++++ lib/librte_eal/linuxapp/eal/eal_pci.c | 69 +++++++++++++++++++++++++ 4 files changed, 206 insertions(+), 10 deletions(-) diff --git a/lib/librte_eal/common/eal_common_pci.c b/lib/librte_eal/common/eal_common_pci.c index a89f5c3..7c9b8c5 100644 --- a/lib/librte_eal/common/eal_common_pci.c +++ b/lib/librte_eal/common/eal_common_pci.c @@ -99,19 +99,27 @@ static struct rte_devargs *pci_devargs_lookup(struct rte_pci_device *dev) return NULL; } -/* - * If vendor/device ID match, call the devinit() function of all - * registered driver for the given device. Return -1 if initialization - * failed, return 1 if no driver is found for this device. - */ static int -pci_probe_all_drivers(struct rte_pci_device *dev) +pci_invoke_all_drivers(struct rte_pci_device *dev, + enum rte_eal_invoke_type type) { struct rte_pci_driver *dr = NULL; - int rc; + int rc = 0; + + if ((dev == NULL) || (type >= RTE_EAL_INVOKE_TYPE_MAX)) + return -1; TAILQ_FOREACH(dr, &pci_driver_list, next) { - rc = rte_eal_pci_probe_one_driver(dr, dev); + switch (type) { + case RTE_EAL_INVOKE_TYPE_PROBE: + rc = rte_eal_pci_probe_one_driver(dr, dev); + break; + case RTE_EAL_INVOKE_TYPE_CLOSE: + rc = rte_eal_pci_close_one_driver(dr, dev); + break; + default: + return -1; + } if (rc < 0) /* negative value is an error */ return -1; @@ -123,6 +131,66 @@ pci_probe_all_drivers(struct rte_pci_device *dev) return 1; } +#ifdef ENABLE_HOTPLUG +static int +rte_eal_pci_invoke_one(struct rte_pci_addr *addr, + enum rte_eal_invoke_type type) +{ + struct rte_pci_device *dev = NULL; + int ret = 0; + + if ((addr == NULL) || (type >= RTE_EAL_INVOKE_TYPE_MAX)) + return -1; + + TAILQ_FOREACH(dev, &pci_device_list, next) { + if (eal_compare_pci_addr(&dev->addr, addr)) + continue; + + ret = pci_invoke_all_drivers(dev, type); + if (ret < 0) + goto invoke_err_return; + + if (type == RTE_EAL_INVOKE_TYPE_CLOSE) + goto remove_dev; + + return 0; + } + + return -1; + +invoke_err_return: + RTE_LOG(WARNING, EAL, "Requested device " PCI_PRI_FMT + " cannot be used\n", dev->addr.domain, dev->addr.bus, + dev->addr.devid, dev->addr.function); + return -1; + +remove_dev: + TAILQ_REMOVE(&pci_device_list, dev, next); + return 0; +} + + +/* + * Find the pci device specified by pci address, then invoke probe function of + * the driver of the devive. + */ +int +rte_eal_pci_probe_one(struct rte_pci_addr *addr) +{ + return rte_eal_pci_invoke_one(addr, RTE_EAL_INVOKE_TYPE_PROBE); +} + +/* + * Find the pci device specified by pci address, then invoke close function of + * the driver of the devive. + */ +int +rte_eal_pci_close_one(struct rte_pci_addr *addr) +{ + return rte_eal_pci_invoke_one(addr, RTE_EAL_INVOKE_TYPE_CLOSE); +} +#endif /* ENABLE_HOTPLUG */ + /* * Scan the content of the PCI bus, and call the devinit() function for * all registered drivers that have a matching entry in its id_table @@ -148,10 +216,12 @@ rte_eal_pci_probe(void) /* probe all or only whitelisted devices */ if (probe_all) - ret = pci_probe_all_drivers(dev); + ret = pci_invoke_all_drivers(dev, + RTE_EAL_INVOKE_TYPE_PROBE); else if (devargs != NULL && devargs->type == RTE_DEVTYPE_WHITELISTED_PCI) - ret = pci_probe_all_drivers(dev); + ret = pci_invoke_all_drivers(dev, + RTE_EAL_INVOKE_TYPE_PROBE); if (ret < 0) rte_exit(EXIT_FAILURE, "Requested device " PCI_PRI_FMT " cannot be used\n", dev->addr.domain, dev->addr.bus, diff --git a/lib/librte_eal/common/eal_private.h b/lib/librte_eal/common/eal_private.h index 159cd66..1a362ab 100644 --- a/lib/librte_eal/common/eal_private.h +++ b/lib/librte_eal/common/eal_private.h @@ -154,6 +154,15 @@ struct rte_pci_driver; struct rte_pci_device; /** + * The invoke type. + */ +enum rte_eal_invoke_type { + RTE_EAL_INVOKE_TYPE_PROBE, /**< invoke probe function */ + RTE_EAL_INVOKE_TYPE_CLOSE, /**< invoke close function */ + RTE_EAL_INVOKE_TYPE_MAX /**< max value of this enum */ +}; + +/** * Mmap memory for single PCI device * * This function is private to EAL. @@ -165,6 +174,21 @@ int rte_eal_pci_probe_one_driver(struct rte_pci_driver *dr, struct rte_pci_device *dev); /** + * Munmap memory for single PCI device + * + * This function is private to EAL. + * + * @param dr + * The pointer to the pci driver structure + * @param dev + * The pointer to the pci device structure + * @return + * 0 on success, negative on error + */ +int rte_eal_pci_close_one_driver(struct rte_pci_driver *dr, + struct rte_pci_device *dev); + +/** * Init tail queues for non-EAL library structures. This is to allow * the rings, mempools, etc. lists to be shared among multiple processes * diff --git a/lib/librte_eal/common/include/rte_pci.h b/lib/librte_eal/common/include/rte_pci.h index 87ca4cf..a111066 100644 --- a/lib/librte_eal/common/include/rte_pci.h +++ b/lib/librte_eal/common/include/rte_pci.h @@ -82,6 +82,7 @@ extern "C" { #include <inttypes.h> #include <rte_interrupts.h> +#include <rte_dev_hotplug.h> TAILQ_HEAD(pci_device_list, rte_pci_device); /**< PCI devices in D-linked Q. */ TAILQ_HEAD(pci_driver_list, rte_pci_driver); /**< PCI drivers in D-linked Q. */ @@ -323,6 +324,38 @@ eal_compare_pci_addr(struct rte_pci_addr *addr, struct rte_pci_addr *addr2) */ int rte_eal_pci_probe(void); +#ifdef ENABLE_HOTPLUG +/** + * Probe the single PCI device. + * + * Scan the content of the PCI bus, and find the pci device specified by pci + * address, then call the probe() function for registered driver that has a + * matching entry in its id_table for discovered device. + * + * @param addr + * The PCI Bus-Device-Function address to probe or close. + * @return + * - 0 on success. + * - Negative on error. + */ +int rte_eal_pci_probe_one(struct rte_pci_addr *addr); + +/** + * Close the single PCI device. + * + * Scan the content of the PCI bus, and find the pci device specified by pci + * address, then call the close() function for registered driver that has a + * matching entry in its id_table for discovered device. + * + * @param addr + * The PCI Bus-Device-Function address to probe or close. + * @return + * - 0 on success. + * - Negative on error. + */ +int rte_eal_pci_close_one(struct rte_pci_addr *addr); +#endif /* ENABLE_HOTPLUG */ + /** * Dump the content of the PCI bus. * diff --git a/lib/librte_eal/linuxapp/eal/eal_pci.c b/lib/librte_eal/linuxapp/eal/eal_pci.c index c3b7917..831422e 100644 --- a/lib/librte_eal/linuxapp/eal/eal_pci.c +++ b/lib/librte_eal/linuxapp/eal/eal_pci.c @@ -682,6 +682,75 @@ rte_eal_pci_probe_one_driver(struct rte_pci_driver *dr, struct rte_pci_device *d return 1; } +#ifdef ENABLE_HOTPLUG +/* + * If vendor/device ID match, call the devuninit() function of the + * driver. + */ +int +rte_eal_pci_close_one_driver(struct rte_pci_driver *dr, + struct rte_pci_device *dev) +{ + struct rte_pci_id *id_table; + + if ((dr == NULL) || (dev == NULL)) + return -EINVAL; + + for (id_table = dr->id_table ; id_table->vendor_id != 0; id_table++) { + + /* check if device's identifiers match the driver's ones */ + if (id_table->vendor_id != dev->id.vendor_id && + id_table->vendor_id != PCI_ANY_ID) + continue; + if (id_table->device_id != dev->id.device_id && + id_table->device_id != PCI_ANY_ID) + continue; + if (id_table->subsystem_vendor_id != + dev->id.subsystem_vendor_id && + id_table->subsystem_vendor_id != PCI_ANY_ID) + continue; + if (id_table->subsystem_device_id != + dev->id.subsystem_device_id && + id_table->subsystem_device_id != PCI_ANY_ID) + continue; + + struct rte_pci_addr *loc = &dev->addr; + + RTE_LOG(DEBUG, EAL, + "PCI device "PCI_PRI_FMT" on NUMA socket %i\n", + loc->domain, loc->bus, loc->devid, + loc->function, dev->numa_node); + + RTE_LOG(DEBUG, EAL, " remove driver: %x:%x %s\n", + dev->id.vendor_id, dev->id.device_id, + dr->name); + + /* call the driver devuninit() function */ + if (dr->devuninit && (dr->devuninit(dr, dev) < 0)) + return -1; /* negative value is an error */ + + /* clear driver structure */ + dev->driver = NULL; + + if (dr->drv_flags & RTE_PCI_DRV_NEED_MAPPING) + /* unmap resources for devices that use igb_uio */ + pci_unmap_device(dev); + + return 0; + } + /* return positive value if driver is not found */ + return 1; +} +#else /* ENABLE_HOTPLUG */ +int +rte_eal_pci_close_one_driver(struct rte_pci_driver *dr __rte_unused, + struct rte_pci_device *dev __rte_unused) +{ + RTE_LOG(ERR, EAL, "Hotplug support isn't enabled\n"); + return -1; +} +#endif /* ENABLE_HOTPLUG */ + /* Init the PCI EAL subsystem */ int rte_eal_pci_init(void) -- 1.9.1
This new parameter is needed to keep device type like physical or virtual. Port detaching processes are different between physical and virtual. This paramerter lets detaching function know a device type of the port. v4: - Fix comments of rte_eth_dev_type. Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp> --- app/test/virtual_pmd.c | 2 +- lib/librte_ether/rte_ethdev.c | 14 ++++++++++++-- lib/librte_ether/rte_ethdev.h | 25 ++++++++++++++++++++++++- lib/librte_pmd_af_packet/rte_eth_af_packet.c | 2 +- lib/librte_pmd_bond/rte_eth_bond_api.c | 2 +- lib/librte_pmd_pcap/rte_eth_pcap.c | 2 +- lib/librte_pmd_ring/rte_eth_ring.c | 2 +- lib/librte_pmd_xenvirt/rte_eth_xenvirt.c | 2 +- 8 files changed, 42 insertions(+), 9 deletions(-) diff --git a/app/test/virtual_pmd.c b/app/test/virtual_pmd.c index 9fac95d..8d3a5ff 100644 --- a/app/test/virtual_pmd.c +++ b/app/test/virtual_pmd.c @@ -556,7 +556,7 @@ virtual_ethdev_create(const char *name, struct ether_addr *mac_addr, goto err; /* reserve an ethdev entry */ - eth_dev = rte_eth_dev_allocate(name); + eth_dev = rte_eth_dev_allocate(name, RTE_ETH_DEV_PHYSICAL); if (eth_dev == NULL) goto err; diff --git a/lib/librte_ether/rte_ethdev.c b/lib/librte_ether/rte_ethdev.c index 2acafc7..0c7fbb1 100644 --- a/lib/librte_ether/rte_ethdev.c +++ b/lib/librte_ether/rte_ethdev.c @@ -232,7 +232,7 @@ rte_eth_dev_allocate_new_port(void) } struct rte_eth_dev * -rte_eth_dev_allocate(const char *name) +rte_eth_dev_allocate(const char *name, enum rte_eth_dev_type type) { uint8_t port_id; struct rte_eth_dev *eth_dev; @@ -256,6 +256,7 @@ rte_eth_dev_allocate(const char *name) snprintf(eth_dev->data->name, sizeof(eth_dev->data->name), "%s", name); eth_dev->data->port_id = port_id; eth_dev->attached = DEV_CONNECTED; + eth_dev->dev_type = type; nb_ports++; return eth_dev; } @@ -276,6 +277,7 @@ rte_eth_dev_free(const char *name) } eth_dev->attached = 0; + eth_dev->dev_type = RTE_ETH_DEV_UNKNOWN; nb_ports--; return eth_dev; } @@ -296,7 +298,7 @@ rte_eth_dev_init(struct rte_pci_driver *pci_drv, snprintf(ethdev_name, RTE_ETH_NAME_MAX_LEN, "%d:%d.%d", pci_dev->addr.bus, pci_dev->addr.devid, pci_dev->addr.function); - eth_dev = rte_eth_dev_allocate(ethdev_name); + eth_dev = rte_eth_dev_allocate(ethdev_name, RTE_ETH_DEV_PHYSICAL); if (eth_dev == NULL) return -ENOMEM; @@ -428,6 +430,14 @@ rte_eth_dev_count(void) return (nb_ports); } +enum rte_eth_dev_type +rte_eth_dev_get_device_type(uint8_t port_id) +{ + if (rte_eth_dev_validate_port(port_id, NONE_TRACE) == DEV_INVALID) + return -1; + return rte_eth_devices[port_id].dev_type; +} + void rte_eth_dev_save(struct rte_eth_dev *devs) { diff --git a/lib/librte_ether/rte_ethdev.h b/lib/librte_ether/rte_ethdev.h index ebc48b0..83a4000 100644 --- a/lib/librte_ether/rte_ethdev.h +++ b/lib/librte_ether/rte_ethdev.h @@ -1522,6 +1522,17 @@ struct eth_dev_ops { }; /** + * The eth device type + */ +enum rte_eth_dev_type { + RTE_ETH_DEV_UNKNOWN, /**< unknown device type */ + RTE_ETH_DEV_PHYSICAL, + /**< Physical function and Virtual function devices of NIC */ + RTE_ETH_DEV_VIRTUAL, /**< non hardware device */ + RTE_ETH_DEV_MAX /**< max value of this enum */ +}; + +/** * @internal * The generic data structure associated with each ethernet device. * @@ -1540,6 +1551,7 @@ struct rte_eth_dev { struct rte_pci_device *pci_dev; /**< PCI info. supplied by probing */ struct rte_eth_dev_cb_list callbacks; /**< User application callbacks */ uint8_t attached; /**< Flag indicating the port is attached */ + enum rte_eth_dev_type dev_type; /**< Flag indicating the device type */ }; struct rte_eth_dev_sriov { @@ -1617,6 +1629,15 @@ extern uint8_t rte_eth_dev_count(void); /** * Function for internal use by port hotplug functions. + * Get the device type to know whether the device is physical or virtual. + * @param port_id The pointer to the port id + * @return + * - Device type. + */ +extern enum rte_eth_dev_type rte_eth_dev_get_device_type(uint8_t port_id); + +/** + * Function for internal use by port hotplug functions. * Copies current ethdev structures to the specified pointer. * * @param devs The pointer to the ethdev structures @@ -1702,10 +1723,12 @@ extern struct rte_eth_dev *rte_eth_dev_allocated(const char *name); * to that slot for the driver to use. * * @param name Unique identifier name for each Ethernet device + * @param type Device type of this Ethernet device * @return * - Slot in the rte_dev_devices array for a new device; */ -struct rte_eth_dev *rte_eth_dev_allocate(const char *name); +struct rte_eth_dev *rte_eth_dev_allocate(const char *name, + enum rte_eth_dev_type type); /** * Function for internal use by dummy drivers primarily, e.g. ring-based diff --git a/lib/librte_pmd_af_packet/rte_eth_af_packet.c b/lib/librte_pmd_af_packet/rte_eth_af_packet.c index 1ffe1cd..80e9bdf 100644 --- a/lib/librte_pmd_af_packet/rte_eth_af_packet.c +++ b/lib/librte_pmd_af_packet/rte_eth_af_packet.c @@ -649,7 +649,7 @@ rte_pmd_init_internals(const char *name, } /* reserve an ethdev entry */ - *eth_dev = rte_eth_dev_allocate(name); + *eth_dev = rte_eth_dev_allocate(name, RTE_ETH_DEV_VIRTUAL); if (*eth_dev == NULL) goto error; diff --git a/lib/librte_pmd_bond/rte_eth_bond_api.c b/lib/librte_pmd_bond/rte_eth_bond_api.c index 4ab3267..7a6a5f7 100644 --- a/lib/librte_pmd_bond/rte_eth_bond_api.c +++ b/lib/librte_pmd_bond/rte_eth_bond_api.c @@ -235,7 +235,7 @@ rte_eth_bond_create(const char *name, uint8_t mode, uint8_t socket_id) } /* reserve an ethdev entry */ - eth_dev = rte_eth_dev_allocate(name); + eth_dev = rte_eth_dev_allocate(name, RTE_ETH_DEV_VIRTUAL); if (eth_dev == NULL) { RTE_BOND_LOG(ERR, "Unable to allocate rte_eth_dev"); goto err; diff --git a/lib/librte_pmd_pcap/rte_eth_pcap.c b/lib/librte_pmd_pcap/rte_eth_pcap.c index d299288..af7fae8 100644 --- a/lib/librte_pmd_pcap/rte_eth_pcap.c +++ b/lib/librte_pmd_pcap/rte_eth_pcap.c @@ -709,7 +709,7 @@ rte_pmd_init_internals(const char *name, const unsigned nb_rx_queues, goto error; /* reserve an ethdev entry */ - *eth_dev = rte_eth_dev_allocate(name); + *eth_dev = rte_eth_dev_allocate(name, RTE_ETH_DEV_VIRTUAL); if (*eth_dev == NULL) goto error; diff --git a/lib/librte_pmd_ring/rte_eth_ring.c b/lib/librte_pmd_ring/rte_eth_ring.c index f685f08..5876057 100644 --- a/lib/librte_pmd_ring/rte_eth_ring.c +++ b/lib/librte_pmd_ring/rte_eth_ring.c @@ -255,7 +255,7 @@ rte_eth_from_rings(const char *name, struct rte_ring *const rx_queues[], goto error; /* reserve an ethdev entry */ - eth_dev = rte_eth_dev_allocate(name); + eth_dev = rte_eth_dev_allocate(name, RTE_ETH_DEV_VIRTUAL); if (eth_dev == NULL) goto error; diff --git a/lib/librte_pmd_xenvirt/rte_eth_xenvirt.c b/lib/librte_pmd_xenvirt/rte_eth_xenvirt.c index 04e30c9..bc403d6 100644 --- a/lib/librte_pmd_xenvirt/rte_eth_xenvirt.c +++ b/lib/librte_pmd_xenvirt/rte_eth_xenvirt.c @@ -648,7 +648,7 @@ eth_dev_xenvirt_create(const char *name, const char *params, goto err; /* reserve an ethdev entry */ - eth_dev = rte_eth_dev_allocate(name); + eth_dev = rte_eth_dev_allocate(name, RTE_ETH_DEV_VIRTUAL); if (eth_dev == NULL) goto err; -- 1.9.1
These functions are used for attaching or detaching a port. When rte_eal_dev_attach() is called, the function tries to realize the device name as pci address. If this is done successfully, rte_eal_dev_attach() will attach physical device port. If not, attaches virtual devive port. When rte_eal_dev_detach() is called, the function gets the device type of this port to know whether the port is came from physical or virtual. And then specific detaching function will be called. v5: - Change function names like below. rte_eal_dev_find_and_invoke() to rte_eal_vdev_find_and_invoke(). rte_eal_dev_invoke() to rte_eal_vdev_invoke(). - Add code to handle a return value of rte_eal_devargs_remove(). - Fix pci address format in rte_eal_dev_detach(). v4: - Fix comment. - Add error checking. - Fix indent of 'if' statement. - Change function name. Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp> --- lib/librte_eal/common/eal_common_dev.c | 274 ++++++++++++++++++++++++++++++++ lib/librte_eal/common/eal_private.h | 11 ++ lib/librte_eal/common/include/rte_dev.h | 33 ++++ lib/librte_eal/linuxapp/eal/Makefile | 1 + lib/librte_eal/linuxapp/eal/eal_pci.c | 6 +- 5 files changed, 322 insertions(+), 3 deletions(-) diff --git a/lib/librte_eal/common/eal_common_dev.c b/lib/librte_eal/common/eal_common_dev.c index eae5656..e3a3f54 100644 --- a/lib/librte_eal/common/eal_common_dev.c +++ b/lib/librte_eal/common/eal_common_dev.c @@ -32,10 +32,13 @@ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ +#include <stdio.h> +#include <limits.h> #include <string.h> #include <inttypes.h> #include <sys/queue.h> +#include <rte_ethdev.h> #include <rte_dev.h> #include <rte_devargs.h> #include <rte_debug.h> @@ -107,3 +110,274 @@ rte_eal_dev_init(void) } return 0; } + +/* So far, DPDK hotplug function only supports linux */ +#ifdef ENABLE_HOTPLUG +static void +rte_eal_vdev_invoke(struct rte_driver *driver, + struct rte_devargs *devargs, enum rte_eal_invoke_type type) +{ + if ((driver == NULL) || (devargs == NULL)) + return; + + switch (type) { + case RTE_EAL_INVOKE_TYPE_PROBE: + driver->init(devargs->virtual.drv_name, devargs->args); + break; + case RTE_EAL_INVOKE_TYPE_CLOSE: + driver->uninit(devargs->virtual.drv_name, devargs->args); + break; + default: + break; + } +} + +static int +rte_eal_vdev_find_and_invoke(const char *name, int type) +{ + struct rte_devargs *devargs; + struct rte_driver *driver; + + if (name == NULL) + return -EINVAL; + + /* call the init function for each virtual device */ + TAILQ_FOREACH(devargs, &devargs_list, next) { + + if (devargs->type != RTE_DEVTYPE_VIRTUAL) + continue; + + if (strncmp(name, devargs->virtual.drv_name, strlen(name))) + continue; + + TAILQ_FOREACH(driver, &dev_driver_list, next) { + if (driver->type != PMD_VDEV) + continue; + + /* search a driver prefix in virtual device name */ + if (!strncmp(driver->name, devargs->virtual.drv_name, + strlen(driver->name))) { + rte_eal_vdev_invoke(driver, devargs, type); + break; + } + } + + if (driver == NULL) { + RTE_LOG(WARNING, EAL, "no driver found for %s\n", + devargs->virtual.drv_name); + } + return 0; + } + return 1; +} + +/* attach the new physical device, then store port_id of the device */ +static int +rte_eal_dev_attach_pdev(struct rte_pci_addr *addr, uint8_t *port_id) +{ + uint8_t new_port_id; + struct rte_eth_dev devs[RTE_MAX_ETHPORTS]; + + if ((addr == NULL) || (port_id == NULL)) + goto err; + + /* save current port status */ + rte_eth_dev_save(devs); + /* re-construct pci_device_list */ + if (rte_eal_pci_scan()) + goto err; + /* invoke probe func of the driver can handle the new device */ + if (rte_eal_pci_probe_one(addr)) + goto err; + /* get port_id enabled by above procedures */ + if (rte_eth_dev_get_changed_port(devs, &new_port_id)) + goto err; + + *port_id = new_port_id; + return 0; +err: + RTE_LOG(ERR, EAL, "Drver, cannot attach the device\n"); + return -1; +} + +/* detach the new physical device, then store pci_addr of the device */ +static int +rte_eal_dev_detach_pdev(uint8_t port_id, struct rte_pci_addr *addr) +{ + struct rte_pci_addr freed_addr; + struct rte_pci_addr vp; + + if (addr == NULL) + goto err; + + /* check whether the driver supports detach feature, or not */ + if (rte_eth_dev_check_detachable(port_id)) + goto err; + + /* get pci address by port id */ + if (rte_eth_dev_get_addr_by_port(port_id, &freed_addr)) + goto err; + + /* Zerod pci addr means the port comes from virtual device */ + vp.domain = vp.bus = vp.devid = vp.function = 0; + if (eal_compare_pci_addr(&vp, &freed_addr) == 0) + goto err; + + /* invoke close func of the driver, + * also remove the device from pci_device_list */ + if (rte_eal_pci_close_one(&freed_addr)) + goto err; + + *addr = freed_addr; + return 0; +err: + RTE_LOG(ERR, EAL, "Drver, cannot detach the device\n"); + return -1; +} + +static void +get_vdev_name(char *vdevargs) +{ + char *sep; + + if (vdevargs == NULL) + return; + + /* set the first ',' to '\0' to split name and arguments */ + sep = strchr(vdevargs, ','); + if (sep != NULL) + sep[0] = '\0'; +} + +/* attach the new virtual device, then store port_id of the device */ +static int +rte_eal_dev_attach_vdev(const char *vdevargs, uint8_t *port_id) +{ + char *args; + uint8_t new_port_id; + struct rte_eth_dev devs[RTE_MAX_ETHPORTS]; + + if ((vdevargs == NULL) || (port_id == NULL)) + goto err0; + + args = strdup(vdevargs); + if (args == NULL) + goto err0; + + /* save current port status */ + rte_eth_dev_save(devs); + /* add the vdevargs to devargs_list */ + if (rte_eal_devargs_add(RTE_DEVTYPE_VIRTUAL, args)) + goto err1; + /* parse vdevargs, then retrieve device name */ + get_vdev_name(args); + /* walk around dev_driver_list to find the driver of the device, + * then invoke probe function o the driver */ + if (rte_eal_vdev_find_and_invoke(args, RTE_EAL_INVOKE_TYPE_PROBE)) + goto err2; + /* get port_id enabled by above procedures */ + if (rte_eth_dev_get_changed_port(devs, &new_port_id)) + goto err2; + + free(args); + *port_id = new_port_id; + return 0; +err2: + rte_eal_devargs_remove(RTE_DEVTYPE_VIRTUAL, args); +err1: + free(args); +err0: + RTE_LOG(ERR, EAL, "Drver, cannot detach the device\n"); + return -1; +} + +/* detach the new virtual device, then store the name of the device */ +static int +rte_eal_dev_detach_vdev(uint8_t port_id, char *vdevname) +{ + char name[RTE_ETH_NAME_MAX_LEN]; + + if (vdevname == NULL) + goto err; + + /* check whether the driver supports detach feature, or not */ + if (rte_eth_dev_check_detachable(port_id)) + goto err; + + /* get device name by port id */ + if (rte_eth_dev_get_name_by_port(port_id, name)) + goto err; + /* walk around dev_driver_list to find the driver of the device, + * then invoke close function o the driver */ + if (rte_eal_vdev_find_and_invoke(name, RTE_EAL_INVOKE_TYPE_CLOSE)) + goto err; + /* remove the vdevname from devargs_list */ + if (rte_eal_devargs_remove(RTE_DEVTYPE_VIRTUAL, name)) + goto err; + + strncpy(vdevname, name, sizeof(name)); + return 0; +err: + RTE_LOG(ERR, EAL, "Drver, cannot detach the device\n"); + return -1; +} + +/* attach the new device, then store port_id of the device */ +int +rte_eal_dev_attach(const char *devargs, uint8_t *port_id) +{ + struct rte_pci_addr addr; + + if ((devargs == NULL) || (port_id == NULL)) + return -EINVAL; + + if (eal_parse_pci_DomBDF(devargs, &addr) == 0) + return rte_eal_dev_attach_pdev(&addr, port_id); + else + return rte_eal_dev_attach_vdev(devargs, port_id); +} + +/* detach the device, then store the name of the device */ +int +rte_eal_dev_detach(uint8_t port_id, char *name) +{ + struct rte_pci_addr addr; + int ret; + + if (name == NULL) + return -EINVAL; + + if (rte_eth_dev_get_device_type(port_id) == RTE_ETH_DEV_PHYSICAL) { + ret = rte_eth_dev_get_addr_by_port(port_id, &addr); + if (ret < 0) + return ret; + + ret = rte_eal_dev_detach_pdev(port_id, &addr); + if (ret == 0) + snprintf(name, RTE_ETH_NAME_MAX_LEN, + "%04x:%02x:%02x.%d", + addr.domain, addr.bus, + addr.devid, addr.function); + + return ret; + } else + return rte_eal_dev_detach_vdev(port_id, name); +} +#else /* ENABLE_HOTPLUG */ +int +rte_eal_dev_attach(const char *devargs __rte_unused, + uint8_t *port_id __rte_unused) +{ + RTE_LOG(ERR, EAL, "Hotplug support isn't enabled\n"); + return -1; +} + +/* detach the device, then store the name of the device */ +int +rte_eal_dev_detach(uint8_t port_id __rte_unused, + char *name __rte_unused) +{ + RTE_LOG(ERR, EAL, "Hotplug support isn't enabled\n"); + return -1; +} +#endif /* ENABLE_HOTPLUG */ diff --git a/lib/librte_eal/common/eal_private.h b/lib/librte_eal/common/eal_private.h index 1a362ab..8168a7a 100644 --- a/lib/librte_eal/common/eal_private.h +++ b/lib/librte_eal/common/eal_private.h @@ -163,6 +163,17 @@ enum rte_eal_invoke_type { }; /** + * Scan the content of the PCI bus, and the devices in the devices + * list + * + * This function is private to EAL. + * + * @return + * 0 on success, negative on error + */ +int rte_eal_pci_scan(void); + +/** * Mmap memory for single PCI device * * This function is private to EAL. diff --git a/lib/librte_eal/common/include/rte_dev.h b/lib/librte_eal/common/include/rte_dev.h index f7e3a10..e63dd1c 100644 --- a/lib/librte_eal/common/include/rte_dev.h +++ b/lib/librte_eal/common/include/rte_dev.h @@ -47,6 +47,7 @@ extern "C" { #endif #include <sys/queue.h> +#include <rte_pci.h> /** Double linked list of device drivers. */ TAILQ_HEAD(rte_driver_list, rte_driver); @@ -57,6 +58,11 @@ TAILQ_HEAD(rte_driver_list, rte_driver); typedef int (rte_dev_init_t)(const char *name, const char *args); /** + * Uninitilization function called for each device driver once. + */ +typedef int (rte_dev_uninit_t)(const char *name, const char *args); + +/** * Driver type enumeration */ enum pmd_type { @@ -72,6 +78,7 @@ struct rte_driver { enum pmd_type type; /**< PMD Driver type */ const char *name; /**< Driver name. */ rte_dev_init_t *init; /**< Device init. function. */ + rte_dev_uninit_t *uninit; /**< Device uninit. function. */ }; /** @@ -93,6 +100,32 @@ void rte_eal_driver_register(struct rte_driver *driver); void rte_eal_driver_unregister(struct rte_driver *driver); /** + * Attach a new device. + * + * @param devargs + * A pointer to a strings array describing the new device + * to be attached. The strings should be a pci address like + * '0000:01:00.0' or virtual device name like 'eth_pcap0'. + * @param port_id + * A pointer to a port identifier actually attached. + * @return + * 0 on success and port_id is filled, negative on error + */ +int rte_eal_dev_attach(const char *devargs, uint8_t *port_id); + +/** + * Detach a device. + * + * @param port_id + * The port identifier of the device to detach. + * @param addr + * A pointer to a device name actually detached. + * @return + * 0 on success and devname is filled, negative on error + */ +int rte_eal_dev_detach(uint8_t port_id, char *devname); + +/** * Initalize all the registered drivers in this process */ int rte_eal_dev_init(void); diff --git a/lib/librte_eal/linuxapp/eal/Makefile b/lib/librte_eal/linuxapp/eal/Makefile index 72ecf3a..0ec83b5 100644 --- a/lib/librte_eal/linuxapp/eal/Makefile +++ b/lib/librte_eal/linuxapp/eal/Makefile @@ -41,6 +41,7 @@ CFLAGS += -I$(RTE_SDK)/lib/librte_eal/common/include CFLAGS += -I$(RTE_SDK)/lib/librte_ring CFLAGS += -I$(RTE_SDK)/lib/librte_mempool CFLAGS += -I$(RTE_SDK)/lib/librte_malloc +CFLAGS += -I$(RTE_SDK)/lib/librte_mbuf CFLAGS += -I$(RTE_SDK)/lib/librte_ether CFLAGS += -I$(RTE_SDK)/lib/librte_ivshmem CFLAGS += -I$(RTE_SDK)/lib/librte_pmd_ring diff --git a/lib/librte_eal/linuxapp/eal/eal_pci.c b/lib/librte_eal/linuxapp/eal/eal_pci.c index 831422e..1f43688 100644 --- a/lib/librte_eal/linuxapp/eal/eal_pci.c +++ b/lib/librte_eal/linuxapp/eal/eal_pci.c @@ -431,8 +431,8 @@ error: * Scan the content of the PCI bus, and the devices in the devices * list */ -static int -pci_scan(void) +int +rte_eal_pci_scan(void) { struct dirent *e; DIR *dir; @@ -764,7 +764,7 @@ rte_eal_pci_init(void) if (internal_config.no_pci) return 0; - if (pci_scan() < 0) { + if (rte_eal_pci_scan() < 0) { RTE_LOG(ERR, EAL, "%s(): Cannot scan PCI bus\n", __func__); return -1; } -- 1.9.1
The patch enables CONFIG_RTE_LIBRTE_EAL_HOTPLUG in Linux configuration. Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp> --- config/common_linuxapp | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/config/common_linuxapp b/config/common_linuxapp index 2f9643b..27d05be 100644 --- a/config/common_linuxapp +++ b/config/common_linuxapp @@ -114,6 +114,11 @@ CONFIG_RTE_PCI_MAX_READ_REQUEST_SIZE=0 CONFIG_RTE_LIBRTE_EAL_LINUXAPP=y # +# Compile Environment Abstraction Layer to support hotplug +# +CONFIG_RTE_LIBRTE_EAL_HOTPLUG=y + +# # Compile Environment Abstraction Layer to support Vmware TSC map # CONFIG_RTE_LIBRTE_EAL_VMWARE_TSC_MAP_SUPPORT=y -- 1.9.1
This patch adds finalization code to free resources allocated by the PMD. v4: - Change function name. Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp> --- lib/librte_pmd_pcap/rte_eth_pcap.c | 40 ++++++++++++++++++++++++++++++++++++++ 1 file changed, 40 insertions(+) diff --git a/lib/librte_pmd_pcap/rte_eth_pcap.c b/lib/librte_pmd_pcap/rte_eth_pcap.c index af7fae8..9263eab 100644 --- a/lib/librte_pmd_pcap/rte_eth_pcap.c +++ b/lib/librte_pmd_pcap/rte_eth_pcap.c @@ -498,6 +498,13 @@ static struct eth_dev_ops ops = { .stats_reset = eth_stats_reset, }; +static struct eth_driver rte_pcap_pmd = { + .pci_drv = { + .name = "rte_pcap_pmd", + .drv_flags = RTE_PCI_DRV_DETACHABLE, + }, +}; + /* * Function handler that opens the pcap file for reading a stores a * reference of it for use it later on. @@ -713,6 +720,10 @@ rte_pmd_init_internals(const char *name, const unsigned nb_rx_queues, if (*eth_dev == NULL) goto error; + /* check length of device name */ + if ((strlen((*eth_dev)->data->name) + 1) > sizeof(data->name)) + goto error; + /* now put it all together * - store queue data in internals, * - store numa_node info in pci_driver @@ -739,10 +750,13 @@ rte_pmd_init_internals(const char *name, const unsigned nb_rx_queues, data->nb_tx_queues = (uint16_t)nb_tx_queues; data->dev_link = pmd_link; data->mac_addrs = ð_addr; + strncpy(data->name, + (*eth_dev)->data->name, strlen((*eth_dev)->data->name)); (*eth_dev)->data = data; (*eth_dev)->dev_ops = &ops; (*eth_dev)->pci_dev = pci_dev; + (*eth_dev)->driver = &rte_pcap_pmd; return 0; @@ -927,10 +941,36 @@ rte_pmd_pcap_devinit(const char *name, const char *params) } +static int +rte_pmd_pcap_devuninit(const char *name, const char *params __rte_unused) +{ + struct rte_eth_dev *eth_dev = NULL; + + RTE_LOG(INFO, PMD, "Closing pcap ethdev on numa socket %u\n", + rte_socket_id()); + + if (name == NULL) + return -1; + + /* reserve an ethdev entry */ + eth_dev = rte_eth_dev_allocated(name); + if (eth_dev == NULL) + return -1; + + rte_free(eth_dev->data->dev_private); + rte_free(eth_dev->data); + rte_free(eth_dev->pci_dev); + + rte_eth_dev_free(name); + + return 0; +} + static struct rte_driver pmd_pcap_drv = { .name = "eth_pcap", .type = PMD_VDEV, .init = rte_pmd_pcap_devinit, + .uninit = rte_pmd_pcap_devuninit, }; PMD_REGISTER_DRIVER(pmd_pcap_drv); -- 1.9.1
The patch introduces following commands. - port attach [ident] - port detach [port_id] - attach: attaching a port - detach: detaching a port - ident: pci address of physical device. Or device name and paramerters of virtual device. (ex. 0000:02:00.0, eth_pcap0,iface=eth0) - port_id: port identifier v5: - Add testpmd documentation. (Thanks to Iremonger, Bernard) v4: - Fix strings of command help. Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp> --- app/test-pmd/cmdline.c | 133 +++++++++++++++---- app/test-pmd/config.c | 116 +++++++++------- app/test-pmd/parameters.c | 22 ++- app/test-pmd/testpmd.c | 199 +++++++++++++++++++++------- app/test-pmd/testpmd.h | 18 ++- doc/guides/testpmd_app_ug/testpmd_funcs.rst | 57 ++++++++ 6 files changed, 415 insertions(+), 130 deletions(-) diff --git a/app/test-pmd/cmdline.c b/app/test-pmd/cmdline.c index 4beb404..2f813d8 100644 --- a/app/test-pmd/cmdline.c +++ b/app/test-pmd/cmdline.c @@ -572,6 +572,12 @@ static void cmd_help_long_parsed(void *parsed_result, "port close (port_id|all)\n" " Close all ports or port_id.\n\n" + "port attach (ident)\n" + " Attach physical or virtual dev by pci address or virtual device name\n\n" + + "port detach (port_id)\n" + " Detach physical or virtual dev by port_id\n\n" + "port config (port_id|all)" " speed (10|100|1000|10000|40000|auto)" " duplex (half|full|auto)\n" @@ -848,6 +854,89 @@ cmdline_parse_inst_t cmd_operate_specific_port = { }, }; +/* *** attach a specificied port *** */ +struct cmd_operate_attach_port_result { + cmdline_fixed_string_t port; + cmdline_fixed_string_t keyword; + cmdline_fixed_string_t identifier; +}; + +static void cmd_operate_attach_port_parsed(void *parsed_result, + __attribute__((unused)) struct cmdline *cl, + __attribute__((unused)) void *data) +{ + struct cmd_operate_attach_port_result *res = parsed_result; + + if (!strcmp(res->keyword, "attach")) + attach_port(res->identifier); + else + printf("Unknown parameter\n"); +} + +cmdline_parse_token_string_t cmd_operate_attach_port_port = + TOKEN_STRING_INITIALIZER(struct cmd_operate_attach_port_result, + port, "port"); +cmdline_parse_token_string_t cmd_operate_attach_port_keyword = + TOKEN_STRING_INITIALIZER(struct cmd_operate_attach_port_result, + keyword, "attach"); +cmdline_parse_token_string_t cmd_operate_attach_port_identifier = + TOKEN_STRING_INITIALIZER(struct cmd_operate_attach_port_result, + identifier, NULL); + +cmdline_parse_inst_t cmd_operate_attach_port = { + .f = cmd_operate_attach_port_parsed, + .data = NULL, + .help_str = "port attach identifier, " + "identifier: pci address or virtual dev name", + .tokens = { + (void *)&cmd_operate_attach_port_port, + (void *)&cmd_operate_attach_port_keyword, + (void *)&cmd_operate_attach_port_identifier, + NULL, + }, +}; + +/* *** detach a specificied port *** */ +struct cmd_operate_detach_port_result { + cmdline_fixed_string_t port; + cmdline_fixed_string_t keyword; + uint8_t port_id; +}; + +static void cmd_operate_detach_port_parsed(void *parsed_result, + __attribute__((unused)) struct cmdline *cl, + __attribute__((unused)) void *data) +{ + struct cmd_operate_detach_port_result *res = parsed_result; + + if (!strcmp(res->keyword, "detach")) + detach_port(res->port_id); + else + printf("Unknown parameter\n"); +} + +cmdline_parse_token_string_t cmd_operate_detach_port_port = + TOKEN_STRING_INITIALIZER(struct cmd_operate_detach_port_result, + port, "port"); +cmdline_parse_token_string_t cmd_operate_detach_port_keyword = + TOKEN_STRING_INITIALIZER(struct cmd_operate_detach_port_result, + keyword, "detach"); +cmdline_parse_token_num_t cmd_operate_detach_port_port_id = + TOKEN_NUM_INITIALIZER(struct cmd_operate_detach_port_result, + port_id, UINT8); + +cmdline_parse_inst_t cmd_operate_detach_port = { + .f = cmd_operate_detach_port_parsed, + .data = NULL, + .help_str = "port detach port_id", + .tokens = { + (void *)&cmd_operate_detach_port_port, + (void *)&cmd_operate_detach_port_keyword, + (void *)&cmd_operate_detach_port_port_id, + NULL, + }, +}; + /* *** configure speed for all ports *** */ struct cmd_config_speed_all { cmdline_fixed_string_t port; @@ -902,7 +991,7 @@ cmd_config_speed_all_parsed(void *parsed_result, return; } - for (pid = 0; pid < nb_ports; pid++) { + FOREACH_PORT(pid, ports) { ports[pid].dev_conf.link_speed = link_speed; ports[pid].dev_conf.link_duplex = link_duplex; } @@ -970,10 +1059,8 @@ cmd_config_speed_specific_parsed(void *parsed_result, return; } - if (res->id >= nb_ports) { - printf("Port id %d must be less than %d\n", res->id, nb_ports); + if (port_id_is_invalid(res->id, ENABLED_WARN)) return; - } if (!strcmp(res->value1, "10")) link_speed = ETH_LINK_SPEED_10; @@ -1533,7 +1620,7 @@ cmd_config_rxtx_queue_parsed(void *parsed_result, return; } - if (port_id_is_invalid(res->portid)) + if (port_id_is_invalid(res->portid, ENABLED_WARN)) return; if (port_is_started(res->portid) != 1) { @@ -2876,7 +2963,7 @@ cmd_tx_cksum_parsed(void *parsed_result, uint16_t ol_flags, mask = 0; struct rte_eth_dev_info dev_info; - if (port_id_is_invalid(res->port_id)) { + if (port_id_is_invalid(res->port_id, ENABLED_WARN)) { printf("invalid port %d\n", res->port_id); return; } @@ -3003,7 +3090,7 @@ cmd_tso_set_parsed(void *parsed_result, struct cmd_tso_set_result *res = parsed_result; struct rte_eth_dev_info dev_info; - if (port_id_is_invalid(res->port_id)) + if (port_id_is_invalid(res->port_id, ENABLED_WARN)) return; if (!strcmp(res->mode, "set")) @@ -3979,10 +4066,8 @@ static void cmd_set_bond_mac_addr_parsed(void *parsed_result, struct cmd_set_bond_mac_addr_result *res = parsed_result; int ret; - if (res->port_num >= nb_ports) { - printf("Port id %d must be less than %d\n", res->port_num, nb_ports); + if (port_id_is_invalid(res->port_num, ENABLED_WARN)) return; - } ret = rte_eth_bond_mac_address_set(res->port_num, &res->address); @@ -4219,7 +4304,7 @@ static void cmd_set_promisc_mode_parsed(void *parsed_result, /* all ports */ if (allports) { - for (i = 0; i < nb_ports; i++) { + FOREACH_PORT(i, ports) { if (enable) rte_eth_promiscuous_enable(i); else @@ -4299,7 +4384,7 @@ static void cmd_set_allmulti_mode_parsed(void *parsed_result, /* all ports */ if (allports) { - for (i = 0; i < nb_ports; i++) { + FOREACH_PORT(i, ports) { if (enable) rte_eth_allmulticast_enable(i); else @@ -5484,25 +5569,25 @@ static void cmd_showportall_parsed(void *parsed_result, struct cmd_showportall_result *res = parsed_result; if (!strcmp(res->show, "clear")) { if (!strcmp(res->what, "stats")) - for (i = 0; i < nb_ports; i++) + FOREACH_PORT(i, ports) nic_stats_clear(i); else if (!strcmp(res->what, "xstats")) - for (i = 0; i < nb_ports; i++) + FOREACH_PORT(i, ports) nic_xstats_clear(i); } else if (!strcmp(res->what, "info")) - for (i = 0; i < nb_ports; i++) + FOREACH_PORT(i, ports) port_infos_display(i); else if (!strcmp(res->what, "stats")) - for (i = 0; i < nb_ports; i++) + FOREACH_PORT(i, ports) nic_stats_display(i); else if (!strcmp(res->what, "xstats")) - for (i = 0; i < nb_ports; i++) + FOREACH_PORT(i, ports) nic_xstats_display(i); else if (!strcmp(res->what, "fdir")) - for (i = 0; i < nb_ports; i++) + FOREACH_PORT(i, ports) fdir_get_infos(i); else if (!strcmp(res->what, "stat_qmap")) - for (i = 0; i < nb_ports; i++) + FOREACH_PORT(i, ports) nic_stats_mapping_display(i); } @@ -8756,6 +8841,8 @@ cmdline_parse_ctx_t main_ctx[] = { (cmdline_parse_inst_t *)&cmd_set_qmap, (cmdline_parse_inst_t *)&cmd_operate_port, (cmdline_parse_inst_t *)&cmd_operate_specific_port, + (cmdline_parse_inst_t *)&cmd_operate_attach_port, + (cmdline_parse_inst_t *)&cmd_operate_detach_port, (cmdline_parse_inst_t *)&cmd_config_speed_all, (cmdline_parse_inst_t *)&cmd_config_speed_specific, (cmdline_parse_inst_t *)&cmd_config_rx_tx, @@ -8830,7 +8917,7 @@ prompt(void) static void cmd_reconfig_device_queue(portid_t id, uint8_t dev, uint8_t queue) { - if (id < nb_ports) { + if (!port_id_is_invalid(id, DISABLED_WARN)) { /* check if need_reconfig has been set to 1 */ if (ports[id].need_reconfig == 0) ports[id].need_reconfig = dev; @@ -8840,7 +8927,7 @@ cmd_reconfig_device_queue(portid_t id, uint8_t dev, uint8_t queue) } else { portid_t pid; - for (pid = 0; pid < nb_ports; pid++) { + FOREACH_PORT(pid, ports) { /* check if need_reconfig has been set to 1 */ if (ports[pid].need_reconfig == 0) ports[pid].need_reconfig = dev; @@ -8858,10 +8945,8 @@ bypass_is_supported(portid_t port_id) struct rte_port *port; struct rte_pci_id *pci_id; - if (port_id >= nb_ports) { - printf("\tPort id must be less than %d.\n", nb_ports); + if (port_id_is_invalid(port_id, ENABLED_WARN)) return 0; - } /* Get the device id. */ port = &ports[port_id]; diff --git a/app/test-pmd/config.c b/app/test-pmd/config.c index c40f819..32d8f9a 100644 --- a/app/test-pmd/config.c +++ b/app/test-pmd/config.c @@ -124,11 +124,15 @@ nic_stats_display(portid_t port_id) struct rte_eth_stats stats; struct rte_port *port = &ports[port_id]; uint8_t i; + portid_t pid; static const char *nic_stats_border = "########################"; - if (port_id >= nb_ports) { - printf("Invalid port, range is [0, %d]\n", nb_ports - 1); + if (port_id_is_invalid(port_id, ENABLED_WARN)) { + printf("Valid port range is [0"); + FOREACH_PORT(pid, ports) + printf(", %d", pid); + printf("]\n"); return; } rte_eth_stats_get(port_id, &stats); @@ -201,8 +205,13 @@ nic_stats_display(portid_t port_id) void nic_stats_clear(portid_t port_id) { - if (port_id >= nb_ports) { - printf("Invalid port, range is [0, %d]\n", nb_ports - 1); + portid_t pid; + + if (port_id_is_invalid(port_id, ENABLED_WARN)) { + printf("Valid port range is [0"); + FOREACH_PORT(pid, ports) + printf(", %d", pid); + printf("]\n"); return; } rte_eth_stats_reset(port_id); @@ -249,11 +258,15 @@ nic_stats_mapping_display(portid_t port_id) { struct rte_port *port = &ports[port_id]; uint16_t i; + portid_t pid; static const char *nic_stats_mapping_border = "########################"; - if (port_id >= nb_ports) { - printf("Invalid port, range is [0, %d]\n", nb_ports - 1); + if (port_id_is_invalid(port_id, ENABLED_WARN)) { + printf("Valid port range is [0"); + FOREACH_PORT(pid, ports) + printf(", %d", pid); + printf("]\n"); return; } @@ -302,9 +315,13 @@ port_infos_display(portid_t port_id) int vlan_offload; struct rte_mempool * mp; static const char *info_border = "*********************"; + portid_t pid; - if (port_id >= nb_ports) { - printf("Invalid port, range is [0, %d]\n", nb_ports - 1); + if (port_id_is_invalid(port_id, ENABLED_WARN)) { + printf("Valid port range is [0"); + FOREACH_PORT(pid, ports) + printf(", %d", pid); + printf("]\n"); return; } port = &ports[port_id]; @@ -362,11 +379,14 @@ port_infos_display(portid_t port_id) } int -port_id_is_invalid(portid_t port_id) +port_id_is_invalid(portid_t port_id, enum print_warning warning) { - if (port_id < nb_ports) + if (ports[port_id].enabled) return 0; - printf("Invalid port %d (must be < nb_ports=%d)\n", port_id, nb_ports); + + if (warning == ENABLED_WARN) + printf("Invalid port %d\n", port_id); + return 1; } @@ -425,7 +445,7 @@ port_reg_bit_display(portid_t port_id, uint32_t reg_off, uint8_t bit_x) uint32_t reg_v; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; if (port_reg_off_is_invalid(port_id, reg_off)) return; @@ -444,7 +464,7 @@ port_reg_bit_field_display(portid_t port_id, uint32_t reg_off, uint8_t l_bit; uint8_t h_bit; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; if (port_reg_off_is_invalid(port_id, reg_off)) return; @@ -471,7 +491,7 @@ port_reg_display(portid_t port_id, uint32_t reg_off) { uint32_t reg_v; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; if (port_reg_off_is_invalid(port_id, reg_off)) return; @@ -485,7 +505,7 @@ port_reg_bit_set(portid_t port_id, uint32_t reg_off, uint8_t bit_pos, { uint32_t reg_v; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; if (port_reg_off_is_invalid(port_id, reg_off)) return; @@ -513,7 +533,7 @@ port_reg_bit_field_set(portid_t port_id, uint32_t reg_off, uint8_t l_bit; uint8_t h_bit; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; if (port_reg_off_is_invalid(port_id, reg_off)) return; @@ -547,7 +567,7 @@ port_reg_bit_field_set(portid_t port_id, uint32_t reg_off, void port_reg_set(portid_t port_id, uint32_t reg_off, uint32_t reg_v) { - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; if (port_reg_off_is_invalid(port_id, reg_off)) return; @@ -560,7 +580,7 @@ port_mtu_set(portid_t port_id, uint16_t mtu) { int diag; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; diag = rte_eth_dev_set_mtu(port_id, mtu); if (diag == 0) @@ -723,7 +743,7 @@ rx_ring_desc_display(portid_t port_id, queueid_t rxq_id, uint16_t rxd_id) { const struct rte_memzone *rx_mz; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; if (rx_queue_id_is_invalid(rxq_id)) return; @@ -740,7 +760,7 @@ tx_ring_desc_display(portid_t port_id, queueid_t txq_id, uint16_t txd_id) { const struct rte_memzone *tx_mz; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; if (tx_queue_id_is_invalid(txq_id)) return; @@ -796,7 +816,7 @@ port_rss_reta_info(portid_t port_id, uint16_t i, idx, shift; int ret; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; ret = rte_eth_dev_rss_reta_query(port_id, reta_conf, nb_entries); @@ -828,7 +848,7 @@ port_rss_hash_conf_show(portid_t port_id, int show_rss_key) uint8_t i; int diag; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; /* Get RSS hash key if asked to display it */ rss_conf.rss_key = (show_rss_key) ? rss_key : NULL; @@ -1406,12 +1426,8 @@ set_fwd_ports_list(unsigned int *portlist, unsigned int nb_pt) again: for (i = 0; i < nb_pt; i++) { port_id = (portid_t) portlist[i]; - if (port_id >= nb_ports) { - printf("Invalid port id %u >= %u\n", - (unsigned int) port_id, - (unsigned int) nb_ports); + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; - } if (record_now) fwd_ports_ids[i] = port_id; } @@ -1569,7 +1585,7 @@ vlan_extend_set(portid_t port_id, int on) int diag; int vlan_offload; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; vlan_offload = rte_eth_dev_get_vlan_offload(port_id); @@ -1591,7 +1607,7 @@ rx_vlan_strip_set(portid_t port_id, int on) int diag; int vlan_offload; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; vlan_offload = rte_eth_dev_get_vlan_offload(port_id); @@ -1612,7 +1628,7 @@ rx_vlan_strip_set_on_queue(portid_t port_id, uint16_t queue_id, int on) { int diag; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; diag = rte_eth_dev_set_vlan_strip_on_queue(port_id, queue_id, on); @@ -1627,7 +1643,7 @@ rx_vlan_filter_set(portid_t port_id, int on) int diag; int vlan_offload; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; vlan_offload = rte_eth_dev_get_vlan_offload(port_id); @@ -1648,7 +1664,7 @@ rx_vft_set(portid_t port_id, uint16_t vlan_id, int on) { int diag; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; if (vlan_id_is_invalid(vlan_id)) return; @@ -1665,7 +1681,7 @@ rx_vlan_all_filter_set(portid_t port_id, int on) { uint16_t vlan_id; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; for (vlan_id = 0; vlan_id < 4096; vlan_id++) rx_vft_set(port_id, vlan_id, on); @@ -1675,7 +1691,7 @@ void vlan_tpid_set(portid_t port_id, uint16_t tp_id) { int diag; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; diag = rte_eth_dev_set_vlan_ether_type(port_id, tp_id); @@ -1690,7 +1706,7 @@ vlan_tpid_set(portid_t port_id, uint16_t tp_id) void tx_vlan_set(portid_t port_id, uint16_t vlan_id) { - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; if (vlan_id_is_invalid(vlan_id)) return; @@ -1701,7 +1717,7 @@ tx_vlan_set(portid_t port_id, uint16_t vlan_id) void tx_vlan_reset(portid_t port_id) { - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; ports[port_id].tx_ol_flags &= ~TESTPMD_TX_OFFLOAD_INSERT_VLAN; } @@ -1709,7 +1725,7 @@ tx_vlan_reset(portid_t port_id) void tx_vlan_pvid_set(portid_t port_id, uint16_t vlan_id, int on) { - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; rte_eth_dev_set_vlan_pvid(port_id, vlan_id, on); @@ -1721,7 +1737,7 @@ set_qmap(portid_t port_id, uint8_t is_rx, uint16_t queue_id, uint8_t map_value) uint16_t i; uint8_t existing_mapping_found = 0; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; if (is_rx ? (rx_queue_id_is_invalid(queue_id)) : (tx_queue_id_is_invalid(queue_id))) @@ -1773,7 +1789,7 @@ fdir_add_signature_filter(portid_t port_id, uint8_t queue_id, { int diag; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; diag = rte_eth_dev_fdir_add_signature_filter(port_id, fdir_filter, @@ -1791,7 +1807,7 @@ fdir_update_signature_filter(portid_t port_id, uint8_t queue_id, { int diag; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; diag = rte_eth_dev_fdir_update_signature_filter(port_id, fdir_filter, @@ -1809,7 +1825,7 @@ fdir_remove_signature_filter(portid_t port_id, { int diag; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; diag = rte_eth_dev_fdir_remove_signature_filter(port_id, fdir_filter); @@ -1881,7 +1897,7 @@ fdir_get_infos(portid_t port_id) static const char *fdir_stats_border = "########################"; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; ret = rte_eth_dev_filter_supported(port_id, RTE_ETH_FILTER_FDIR); if (ret < 0) { @@ -1955,7 +1971,7 @@ fdir_add_perfect_filter(portid_t port_id, uint16_t soft_id, uint8_t queue_id, { int diag; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; diag = rte_eth_dev_fdir_add_perfect_filter(port_id, fdir_filter, @@ -1973,7 +1989,7 @@ fdir_update_perfect_filter(portid_t port_id, uint16_t soft_id, uint8_t queue_id, { int diag; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; diag = rte_eth_dev_fdir_update_perfect_filter(port_id, fdir_filter, @@ -1991,7 +2007,7 @@ fdir_remove_perfect_filter(portid_t port_id, uint16_t soft_id, { int diag; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; diag = rte_eth_dev_fdir_remove_perfect_filter(port_id, fdir_filter, @@ -2008,7 +2024,7 @@ fdir_set_masks(portid_t port_id, struct rte_fdir_masks *fdir_masks) { int diag; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; diag = rte_eth_dev_fdir_set_masks(port_id, fdir_masks); @@ -2085,7 +2101,7 @@ set_vf_traffic(portid_t port_id, uint8_t is_rx, uint16_t vf, uint8_t on) { int diag; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; if (is_rx) diag = rte_eth_dev_set_vf_rx(port_id,vf,on); @@ -2107,7 +2123,7 @@ set_vf_rx_vlan(portid_t port_id, uint16_t vlan_id, uint64_t vf_mask, uint8_t on) { int diag; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; if (vlan_id_is_invalid(vlan_id)) return; @@ -2124,7 +2140,7 @@ set_queue_rate_limit(portid_t port_id, uint16_t queue_idx, uint16_t rate) int diag; struct rte_eth_link link; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return 1; rte_eth_link_get_nowait(port_id, &link); if (rate > link.link_speed) { @@ -2149,7 +2165,7 @@ set_vf_rate_limit(portid_t port_id, uint16_t vf, uint16_t rate, uint64_t q_msk) if (q_msk == 0) return 0; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return 1; rte_eth_link_get_nowait(port_id, &link); if (rate > link.link_speed) { diff --git a/app/test-pmd/parameters.c b/app/test-pmd/parameters.c index adf3203..6f2af18 100644 --- a/app/test-pmd/parameters.c +++ b/app/test-pmd/parameters.c @@ -376,6 +376,7 @@ parse_portnuma_config(const char *q_arg) }; unsigned long int_fld[_NUM_FLD]; char *str_fld[_NUM_FLD]; + portid_t pid; /* reset from value set at definition */ while ((p = strchr(p0,'(')) != NULL) { @@ -397,8 +398,11 @@ parse_portnuma_config(const char *q_arg) return -1; } port_id = (uint8_t)int_fld[FLD_PORT]; - if (port_id >= nb_ports) { - printf("Invalid port, range is [0, %d]\n", nb_ports - 1); + if (port_id_is_invalid(port_id, ENABLED_WARN)) { + printf("Valid port range is [0"); + FOREACH_PORT(pid, ports) + printf(", %d", pid); + printf("]\n"); return -1; } socket_id = (uint8_t)int_fld[FLD_SOCKET]; @@ -429,6 +433,7 @@ parse_ringnuma_config(const char *q_arg) }; unsigned long int_fld[_NUM_FLD]; char *str_fld[_NUM_FLD]; + portid_t pid; #define RX_RING_ONLY 0x1 #define TX_RING_ONLY 0x2 #define RXTX_RING 0x3 @@ -453,8 +458,11 @@ parse_ringnuma_config(const char *q_arg) return -1; } port_id = (uint8_t)int_fld[FLD_PORT]; - if (port_id >= nb_ports) { - printf("Invalid port, range is [0, %d]\n", nb_ports - 1); + if (port_id_is_invalid(port_id, ENABLED_WARN)) { + printf("Valid port range is [0"); + FOREACH_PORT(pid, ports) + printf(", %d", pid); + printf("]\n"); return -1; } socket_id = (uint8_t)int_fld[FLD_SOCKET]; @@ -626,12 +634,12 @@ launch_args_parse(int argc, char** argv) #endif if (!strcmp(lgopts[opt_idx].name, "nb-ports")) { n = atoi(optarg); - if (n > 0 && n <= nb_ports) + if (n > 0 && + !port_id_is_invalid(n, DISABLED_WARN)) nb_fwd_ports = (uint8_t) n; else rte_exit(EXIT_FAILURE, - "nb-ports should be > 0 and <= %d\n", - nb_ports); + "Invalid port %d\n", n); } if (!strcmp(lgopts[opt_idx].name, "nb-cores")) { n = atoi(optarg); diff --git a/app/test-pmd/testpmd.c b/app/test-pmd/testpmd.c index 773b8af..c18c1a9 100644 --- a/app/test-pmd/testpmd.c +++ b/app/test-pmd/testpmd.c @@ -71,6 +71,7 @@ #include <rte_pci.h> #include <rte_ether.h> #include <rte_ethdev.h> +#include <rte_dev.h> #include <rte_string_fns.h> #ifdef RTE_LIBRTE_PMD_XENVIRT #include <rte_eth_xenvirt.h> @@ -315,7 +316,7 @@ uint16_t nb_rx_queue_stats_mappings = 0; /* Forward function declarations */ static void map_port_queue_stats_mapping_registers(uint8_t pi, struct rte_port *port); -static void check_all_ports_link_status(uint8_t port_num, uint32_t port_mask); +static void check_all_ports_link_status(uint32_t port_mask); /* * Check if all the ports are started. @@ -324,6 +325,20 @@ static void check_all_ports_link_status(uint8_t port_num, uint32_t port_mask); static int all_ports_started(void); /* + * Find next enabled port + */ +portid_t +find_next_port(portid_t p, struct rte_port *ports, int size) +{ + if (ports == NULL) + rte_exit(-EINVAL, "failed to find a next port id\n"); + + while ((ports[p].enabled == 0) && (p < size)) + p++; + return p; +} + +/* * Setup default configuration. */ static void @@ -552,7 +567,8 @@ init_config(void) + RTE_TEST_TX_DESC_MAX + MAX_PKT_BURST; if (!numa_support) - nb_mbuf_per_pool = (nb_mbuf_per_pool * nb_ports); + nb_mbuf_per_pool = + (nb_mbuf_per_pool * RTE_MAX_ETHPORTS); } if (!numa_support) { @@ -565,14 +581,19 @@ init_config(void) /* Configuration of Ethernet ports. */ ports = rte_zmalloc("testpmd: ports", - sizeof(struct rte_port) * nb_ports, + sizeof(struct rte_port) * RTE_MAX_ETHPORTS, RTE_CACHE_LINE_SIZE); if (ports == NULL) { - rte_exit(EXIT_FAILURE, "rte_zmalloc(%d struct rte_port) " - "failed\n", nb_ports); + rte_exit(EXIT_FAILURE, + "rte_zmalloc(%d struct rte_port) failed\n", + RTE_MAX_ETHPORTS); } - for (pid = 0; pid < nb_ports; pid++) { + /* enabled allocated ports */ + for (pid = 0; pid < nb_ports; pid++) + ports[pid].enabled = 1; + + FOREACH_PORT(pid, ports) { port = &ports[pid]; rte_eth_dev_info_get(pid, &port->dev_info); @@ -602,8 +623,7 @@ init_config(void) nb_mbuf_per_pool = nb_mbuf_per_pool/nb_ports; for (i = 0; i < MAX_SOCKET; i++) { - nb_mbuf = (nb_mbuf_per_pool * - port_per_socket[i]); + nb_mbuf = (nb_mbuf_per_pool * RTE_MAX_ETHPORTS); if (nb_mbuf) mbuf_pool_create(mbuf_data_size, nb_mbuf,i); @@ -635,14 +655,6 @@ reconfig(portid_t new_port_id, unsigned socket_id) struct rte_port *port; /* Reconfiguration of Ethernet ports. */ - ports = rte_realloc(ports, - sizeof(struct rte_port) * nb_ports, - RTE_CACHE_LINE_SIZE); - if (ports == NULL) { - rte_exit(EXIT_FAILURE, "rte_realloc(%d struct rte_port) failed\n", - nb_ports); - } - port = &ports[new_port_id]; rte_eth_dev_info_get(new_port_id, &port->dev_info); @@ -663,7 +675,7 @@ init_fwd_streams(void) streamid_t sm_id, nb_fwd_streams_new; /* set socket id according to numa or not */ - for (pid = 0; pid < nb_ports; pid++) { + FOREACH_PORT(pid, ports) { port = &ports[pid]; if (nb_rxq > port->dev_info.max_rx_queues) { printf("Fail: nb_rxq(%d) is greater than " @@ -1264,7 +1276,7 @@ all_ports_started(void) portid_t pi; struct rte_port *port; - for (pi = 0; pi < nb_ports; pi++) { + FOREACH_PORT(pi, ports) { port = &ports[pi]; /* Check if there is a port which is not started */ if (port->port_status != RTE_PORT_STARTED) @@ -1276,6 +1288,45 @@ all_ports_started(void) } int +all_ports_stopped(void) +{ + portid_t pi; + struct rte_port *port; + + FOREACH_PORT(pi, ports) { + port = &ports[pi]; + if (port->port_status != RTE_PORT_STOPPED) + return 0; + } + + return 1; +} + +int +port_is_started(portid_t port_id) +{ + if (port_id_is_invalid(port_id, ENABLED_WARN)) + return 0; + + if (ports[port_id].port_status != RTE_PORT_STARTED) + return 0; + + return 1; +} + +static int +port_is_closed(portid_t port_id) +{ + if (port_id_is_invalid(port_id, ENABLED_WARN)) + return 0; + + if (ports[port_id].port_status != RTE_PORT_CLOSED) + return 0; + + return 1; +} + +int start_port(portid_t pid) { int diag, need_check_link_status = 0; @@ -1296,8 +1347,8 @@ start_port(portid_t pid) if(dcb_config) dcb_test = 1; - for (pi = 0; pi < nb_ports; pi++) { - if (pid < nb_ports && pid != pi) + FOREACH_PORT(pi, ports) { + if (!port_id_is_invalid(pid, DISABLED_WARN) && pid != pi) continue; port = &ports[pi]; @@ -1421,7 +1472,7 @@ start_port(portid_t pid) } if (need_check_link_status && !no_link_check) - check_all_ports_link_status(nb_ports, RTE_PORT_ALL); + check_all_ports_link_status(RTE_PORT_ALL); else printf("Please stop the ports first\n"); @@ -1446,8 +1497,8 @@ stop_port(portid_t pid) } printf("Stopping ports...\n"); - for (pi = 0; pi < nb_ports; pi++) { - if (pid < nb_ports && pid != pi) + FOREACH_PORT(pi, ports) { + if (!port_id_is_invalid(pid, DISABLED_WARN) && pid != pi) continue; port = &ports[pi]; @@ -1463,7 +1514,7 @@ stop_port(portid_t pid) need_check_link_status = 1; } if (need_check_link_status && !no_link_check) - check_all_ports_link_status(nb_ports, RTE_PORT_ALL); + check_all_ports_link_status(RTE_PORT_ALL); printf("Done\n"); } @@ -1481,8 +1532,8 @@ close_port(portid_t pid) printf("Closing ports...\n"); - for (pi = 0; pi < nb_ports; pi++) { - if (pid < nb_ports && pid != pi) + FOREACH_PORT(pi, ports) { + if (!port_id_is_invalid(pid, DISABLED_WARN) && pid != pi) continue; port = &ports[pi]; @@ -1502,31 +1553,83 @@ close_port(portid_t pid) printf("Done\n"); } -int -all_ports_stopped(void) +void +attach_port(char *identifier) { - portid_t pi; - struct rte_port *port; + portid_t i, j, pi = 0; - for (pi = 0; pi < nb_ports; pi++) { - port = &ports[pi]; - if (port->port_status != RTE_PORT_STOPPED) - return 0; + printf("Attaching a new port...\n"); + + if (identifier == NULL) { + printf("Invalid parameters are speficied\n"); + return; } - return 1; + if (test_done == 0) { + printf("Please stop forwarding first\n"); + return; + } + + if (rte_eal_dev_attach(identifier, &pi)) + return; + + ports[pi].enabled = 1; + reconfig(pi, rte_eth_dev_socket_id(pi)); + rte_eth_promiscuous_enable(pi); + + nb_ports = rte_eth_dev_count(); + + /* set_default_fwd_ports_config(); */ + bzero(fwd_ports_ids, sizeof(fwd_ports_ids)); + i = 0; + FOREACH_PORT(j, ports) { + fwd_ports_ids[i] = j; + i++; + } + nb_cfg_ports = nb_ports; + nb_fwd_ports++; + + ports[pi].port_status = RTE_PORT_STOPPED; + + printf("Port %d is attached. Now total ports is %d\n", pi, nb_ports); + printf("Done\n"); } -int -port_is_started(portid_t port_id) +void +detach_port(uint8_t port_id) { - if (port_id_is_invalid(port_id)) - return -1; + portid_t i, pi = 0; + char name[RTE_ETH_NAME_MAX_LEN]; - if (ports[port_id].port_status != RTE_PORT_STARTED) - return 0; + printf("Detaching a port...\n"); - return 1; + if (!port_is_closed(port_id)) { + printf("Please close port first\n"); + return; + } + + rte_eth_promiscuous_disable(port_id); + + if (rte_eal_dev_detach(port_id, name)) + return; + + ports[port_id].enabled = 0; + nb_ports = rte_eth_dev_count(); + + /* set_default_fwd_ports_config(); */ + bzero(fwd_ports_ids, sizeof(fwd_ports_ids)); + i = 0; + FOREACH_PORT(pi, ports) { + fwd_ports_ids[i] = pi; + i++; + } + nb_cfg_ports = nb_ports; + nb_fwd_ports--; + + printf("Port '%s' is detached. Now total ports is %d\n", + name, nb_ports); + printf("Done\n"); + return; } void @@ -1534,7 +1637,7 @@ pmd_test_exit(void) { portid_t pt_id; - for (pt_id = 0; pt_id < nb_ports; pt_id++) { + FOREACH_PORT(pt_id, ports) { printf("Stopping port %d...", pt_id); fflush(stdout); rte_eth_dev_close(pt_id); @@ -1553,7 +1656,7 @@ struct pmd_test_command { /* Check the link status of all ports in up to 9s, and print them finally */ static void -check_all_ports_link_status(uint8_t port_num, uint32_t port_mask) +check_all_ports_link_status(uint32_t port_mask) { #define CHECK_INTERVAL 100 /* 100ms */ #define MAX_CHECK_TIME 90 /* 9s (90 * 100ms) in total */ @@ -1564,7 +1667,7 @@ check_all_ports_link_status(uint8_t port_num, uint32_t port_mask) fflush(stdout); for (count = 0; count <= MAX_CHECK_TIME; count++) { all_ports_up = 1; - for (portid = 0; portid < port_num; portid++) { + FOREACH_PORT(portid, ports) { if ((port_mask & (1 << portid)) == 0) continue; memset(&link, 0, sizeof(link)); @@ -1688,7 +1791,7 @@ init_port_config(void) portid_t pid; struct rte_port *port; - for (pid = 0; pid < nb_ports; pid++) { + FOREACH_PORT(pid, ports) { port = &ports[pid]; port->dev_conf.rxmode = rx_mode; port->dev_conf.fdir_conf = fdir_conf; @@ -1877,7 +1980,7 @@ main(int argc, char** argv) nb_ports = (portid_t) rte_eth_dev_count(); if (nb_ports == 0) - rte_exit(EXIT_FAILURE, "No probed ethernet device\n"); + RTE_LOG(WARNING, EAL, "No probed ethernet devices\n"); set_def_fwd_config(); if (nb_lcores == 0) @@ -1899,7 +2002,7 @@ main(int argc, char** argv) rte_exit(EXIT_FAILURE, "Start ports failed\n"); /* set all ports to promiscuous mode by default */ - for (port_id = 0; port_id < nb_ports; port_id++) + FOREACH_PORT(port_id, ports) rte_eth_promiscuous_enable(port_id); #ifdef RTE_LIBRTE_CMDLINE diff --git a/app/test-pmd/testpmd.h b/app/test-pmd/testpmd.h index 8f5e6c7..109c670 100644 --- a/app/test-pmd/testpmd.h +++ b/app/test-pmd/testpmd.h @@ -134,6 +134,7 @@ struct fwd_stream { * The data structure associated with each port. */ struct rte_port { + uint8_t enabled; /**< Port enabled or not */ struct rte_eth_dev_info dev_info; /**< PCI info + driver name */ struct rte_eth_conf dev_conf; /**< Port configuration. */ struct ether_addr eth_addr; /**< Port ethernet address */ @@ -159,6 +160,14 @@ struct rte_port { struct rte_eth_txconf tx_conf; /**< tx configuration */ }; +extern portid_t __rte_unused +find_next_port(portid_t p, struct rte_port *ports, int size); + +#define FOREACH_PORT(p, ports) \ + for (p = find_next_port(0, ports, RTE_MAX_ETHPORTS); \ + p < RTE_MAX_ETHPORTS; \ + p = find_next_port(p + 1, ports, RTE_MAX_ETHPORTS)) + /** * The data structure associated with each forwarding logical core. * The logical cores are internally numbered by a core index from 0 to @@ -515,6 +524,8 @@ int init_port_dcb_config(portid_t pid,struct dcb_config *dcb_conf); int start_port(portid_t pid); void stop_port(portid_t pid); void close_port(portid_t pid); +void attach_port(char *identifier); +void detach_port(uint8_t port_id); int all_ports_stopped(void); int port_is_started(portid_t port_id); void pmd_test_exit(void); @@ -558,10 +569,15 @@ void get_ethertype_filter(uint8_t port_id, uint16_t index); void get_2tuple_filter(uint8_t port_id, uint16_t index); void get_5tuple_filter(uint8_t port_id, uint16_t index); void get_flex_filter(uint8_t port_id, uint16_t index); -int port_id_is_invalid(portid_t port_id); int rx_queue_id_is_invalid(queueid_t rxq_id); int tx_queue_id_is_invalid(queueid_t txq_id); +enum print_warning { + ENABLED_WARN = 0, + DISABLED_WARN +}; +int port_id_is_invalid(portid_t port_id, enum print_warning warning); + /* * Work-around of a compilation error with ICC on invocations of the * rte_be_to_cpu_16() function. diff --git a/doc/guides/testpmd_app_ug/testpmd_funcs.rst b/doc/guides/testpmd_app_ug/testpmd_funcs.rst index 218835a..1cacbcf 100644 --- a/doc/guides/testpmd_app_ug/testpmd_funcs.rst +++ b/doc/guides/testpmd_app_ug/testpmd_funcs.rst @@ -808,6 +808,63 @@ The following sections show functions for configuring ports. Port configuration changes only become active when forwarding is started/restarted. +port attach +~~~~~~~~~~~ + +Attach a port specified by pci address or virtual device args. + +To attach a new pci device, the device should be recognized by kernel first. +Then it should be moved under DPDK management. +Finally the port can be attached to testpmd. +On the other hand, to attach a port created by virtual device, above steps are not needed. + +port attach (identifier) + +For example, to attach a port that pci address is 0000:02:00.0. + +.. code-block:: console + + testpmd> port attach 0000:02:00.0 + Attaching a new port... + ... snip ... + Port 0 is attached. Now total ports is 1 + Done + +For example, to attach a port created by pcap PMD. + +.. code-block:: console + + testpmd> port attach eth_pcap0,iface=eth0 + Attaching a new port... + ... snip ... + Port 0 is attached. Now total ports is 1 + Done + +In this case, identifier is "eth_pcap0,iface=eth0". +This identifier format is the same as "--vdev" format of DPDK applications. + +port detach +~~~~~~~~~~~ + +Detach a specific port. + +Before detaching a port, the port should be closed. +Also to remove a pci device completely from the system, first detach the port from testpmd. +Then the device should be moved under kernel management. +Finally the device can be remove using kernel pci hotplug functionality. +On the other hand, to remove a port created by virtual device, above steps are not needed. + +port detach (port_id) + +For example, to detach a port 0. + +.. code-block:: console + + testpmd> port detach 0 + Detaching a port... + ... snip ... + Done + port start ~~~~~~~~~~ -- 1.9.1
This patch series adds a dynamic port hotplug framework to DPDK. With the patches, DPDK apps can attach or detach ports at runtime. The basic concept of the port hotplug is like followings. - DPDK apps must have responsibility to manage ports. DPDK apps only know which ports are attached or detached at the moment. The port hotplug framework is implemented to allow DPDK apps to manage ports. For example, when DPDK apps call port attach function, attached port number will be returned. Also DPDK apps can detach port by port number. - Kernel support is needed for attaching or detaching physical device ports. To attach a new physical device port, the device will be recognized by userspace directly I/O framework in kernel at first. Then DPDK apps can call the port hotplug functions to attach ports. For detaching, steps are vice versa. - Before detach ports, ports must be stopped and closed. DPDK application must call rte_eth_dev_stop() and rte_eth_dev_close() before detaching ports. These function will call finalization codes of PMDs. But so far, no PMD frees all resources allocated by initialization. It means PMDs are needed to be fixed to support the port hotplug. 'RTE_PCI_DRV_DETACHABLE' is a new flag indicating a PMD supports detaching. Without this flag, detaching will be failed. - Mustn't affect legacy DPDK apps. No DPDK EAL behavior is changed, if the port hotplug functions are't called. So all legacy DPDK apps can still work without modifications. And a few limitations. - The port hotplug functions are not thread safe. DPDK apps should handle it. - Only support Linux and igb_uio so far. BSD and VFIO is not supported. I will send VFIO patches at least, but I don't have a plan to submit BSD patch so far. Here is port hotplug APIs. ------------------------------------------------------------------------------- /** * Attach a new device. * * @param devargs * A pointer to a strings array describing the new device * to be attached. The strings should be a pci address like * '0000:01:00.0' or virtual device name like 'eth_pcap0'. * @param port_id * A pointer to a port identifier actually attached. * @return * 0 on success and port_id is filled, negative on error */ int rte_eal_dev_attach(const char *devargs, uint8_t *port_id); /** * Detach a device. * * @param port_id * The port identifier of the device to detach. * @param addr * A pointer to a device name actually detached. * @return * 0 on success and devname is filled, negative on error */ int rte_eal_dev_detach(uint8_t port_id, char *devname); ------------------------------------------------------------------------------- This patch series are for DPDK EAL. To use port hotplug function by DPDK apps, each PMD should be fixed to support 'RTE_PCI_DRV_DETACHABLE' flag. Please check a patch for pcap PMD. Also please check testpmd patch. It will show you how to fix your legacy applications to support port hotplug feature. PATCH v6 changes - Fix rte_eth_dev_uninit() to handle a return value of uninit function of PMD. To do this, below changes also be applied. - Fix a paramter of rte_eth_dev_free(). - Use rte_eth_dev structure as the paramter of rte_eth_dev_free(). PATCH v5 changes - Add runtime check passthrough driver type, like vfio-pci, igb_uio and uio_pci_generic. This was done by Qiu, Michael. Thanks a lot. - Change function names like below. - rte_eal_dev_find_and_invoke() to rte_eal_vdev_find_and_invoke(). - rte_eal_dev_invoke() to rte_eal_vdev_invoke(). - Add code to handle a return value of rte_eal_devargs_remove(). - Fix pci address format in rte_eal_dev_detach(). - Remove RTE_EAL_INVOKE_TYPE_UNKNOWN, because it's unused. - Change function definition of rte_eal_devargs_remove(). - Fix pci_unmap_device() to check pt_driver. - Fix return value of below functions. - rte_eth_dev_get_changed_port(). - rte_eth_dev_get_port_by_addr(). - Change paramters of rte_eth_dev_validate_port() to cleanup code. - Fix pci_scan_one to handle pt_driver correctly. (Thanks to Qiu, Michael for above sugesstions) PATCH v4 changes - Merge patches to review easier. - Fix indent of 'if' statement. - Fix calculation method of eal_compare_pci_addr(). - Fix header file declaration. - Add header file to determine if hotplug can be enabled. (Thanks to Qiu, Michael) - Use braces with 'for' loop. - Add paramerter checking. - Fix sanity check code - Fix comments of rte_eth_dev_type. - Change function names. (Thanks to Iremonger, Bernard) PATCH v3 changes: - Fix enum definition used in rte_ethdev.c. (Thanks to Zhang, Helin) PATCH v2 changes: - Replace rte_eal_dev_attach_pdev(), rte_eal_dev_detach_pdev, rte_eal_dev_attach_vdev() and rte_eal_dev_detach_vdev() to rte_eal_dev_attach() and rte_eal_dev_detach(). - Add parameter values checking. - Refashion a few functions. (Thanks to Iremonger, Bernard) PATCH v1 Changes: - Fix error checking code of librte_eth APIs. - Fix issue that port from pcap PMD cannot be detached correctly. - Fix issue that testpmd could hang after forwarding, if attaching and detaching is repeatedly. - Fix if-condition of rte_eth_dev_get_port_by_addr(). (Thanks to Mark Enright) RFC PATCH v2 Changes: - remove 'rte_eth_dev_validate_port()', and cleanup codes. Michael Qiu (2): eal_pci: Add flag to hold kernel driver type eal_pci: pci memory map work with driver type Tetsuya Mukawa (11): eal/pci,ethdev: Remove assumption that port will not be detached eal/pci: Consolidate pci address comparison APIs ethdev: Add rte_eth_dev_free to free specified device eal,ethdev: Add a function and function pointers to close ether device ethdev: Add functions that will be used by port hotplug functions eal/linux/pci: Add functions for unmapping igb_uio resources eal/pci: Add a function to remove the entry of devargs list eal/pci: Cleanup pci driver initialization code ethdev: Add one dev_type paramerter to rte_eth_dev_allocate eal/pci: Add rte_eal_dev_attach/detach() functions eal: Enable port hotplug framework in Linux app/test/virtual_pmd.c | 2 +- config/common_linuxapp | 5 + lib/librte_eal/bsdapp/eal/eal_pci.c | 25 +- lib/librte_eal/common/Makefile | 1 + lib/librte_eal/common/eal_common_dev.c | 274 +++++++++++ lib/librte_eal/common/eal_common_devargs.c | 60 +++ lib/librte_eal/common/eal_common_pci.c | 92 +++- lib/librte_eal/common/eal_private.h | 35 ++ lib/librte_eal/common/include/rte_dev.h | 33 ++ lib/librte_eal/common/include/rte_dev_hotplug.h | 44 ++ lib/librte_eal/common/include/rte_devargs.h | 21 + lib/librte_eal/common/include/rte_pci.h | 84 ++++ lib/librte_eal/linuxapp/eal/Makefile | 1 + lib/librte_eal/linuxapp/eal/eal_pci.c | 227 +++++++-- lib/librte_eal/linuxapp/eal/eal_pci_init.h | 8 + lib/librte_eal/linuxapp/eal/eal_pci_uio.c | 67 ++- lib/librte_ether/rte_ethdev.c | 624 +++++++++++++----------- lib/librte_ether/rte_ethdev.h | 148 +++++- lib/librte_pmd_af_packet/rte_eth_af_packet.c | 2 +- lib/librte_pmd_bond/rte_eth_bond_api.c | 2 +- lib/librte_pmd_pcap/rte_eth_pcap.c | 2 +- lib/librte_pmd_ring/rte_eth_ring.c | 2 +- lib/librte_pmd_xenvirt/rte_eth_xenvirt.c | 2 +- 23 files changed, 1414 insertions(+), 347 deletions(-) create mode 100644 lib/librte_eal/common/include/rte_dev_hotplug.h -- 1.9.1
From: Michael Qiu <michael.qiu@intel.com> Currently, dpdk has no ability to know which type of driver( vfio-pci/igb_uio/uio_pci_generic) the device used. It only can check whether vfio is enabled or not staticly. It really useful to have the flag, becasue different type need to handle differently in runtime. For example, pci memory map, pot hotplug, and so on. This patch add a flag field for pci device to solve above issue. Signed-off-by: Michael Qiu <michael.qiu@intel.com> Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp> --- lib/librte_eal/common/include/rte_pci.h | 8 +++++ lib/librte_eal/linuxapp/eal/eal_pci.c | 53 +++++++++++++++++++++++++++++++-- 2 files changed, 59 insertions(+), 2 deletions(-) diff --git a/lib/librte_eal/common/include/rte_pci.h b/lib/librte_eal/common/include/rte_pci.h index 66ed793..7b48b55 100644 --- a/lib/librte_eal/common/include/rte_pci.h +++ b/lib/librte_eal/common/include/rte_pci.h @@ -139,6 +139,13 @@ struct rte_pci_addr { struct rte_devargs; +enum rte_pt_driver { + RTE_PT_UNKNOWN = 0, + RTE_PT_IGB_UIO = 1, + RTE_PT_VFIO = 2, + RTE_PT_UIO_GENERIC = 3, +}; + /** * A structure describing a PCI device. */ @@ -152,6 +159,7 @@ struct rte_pci_device { uint16_t max_vfs; /**< sriov enable if not zero */ int numa_node; /**< NUMA node connection */ struct rte_devargs *devargs; /**< Device user arguments */ + enum rte_pt_driver pt_driver; /**< Driver of passthrough */ }; /** Any PCI device identifier (vendor, device, ...) */ diff --git a/lib/librte_eal/linuxapp/eal/eal_pci.c b/lib/librte_eal/linuxapp/eal/eal_pci.c index b5f5410..bd3f77d 100644 --- a/lib/librte_eal/linuxapp/eal/eal_pci.c +++ b/lib/librte_eal/linuxapp/eal/eal_pci.c @@ -97,6 +97,35 @@ error: return -1; } +static int +pci_get_kernel_driver_by_path(const char *filename, char *dri_name) +{ + int count; + char path[PATH_MAX]; + char *name; + + if (!filename || !dri_name) + return -1; + + count = readlink(filename, path, PATH_MAX); + if (count >= PATH_MAX) + return -1; + + /* For device does not have a driver */ + if (count < 0) + return 1; + + path[count] = '\0'; + + name = strrchr(path, '/'); + if (name) { + strncpy(dri_name, name + 1, strlen(name + 1) + 1); + return 0; + } + + return -1; +} + void * pci_find_max_end_va(void) { @@ -222,11 +251,12 @@ pci_scan_one(const char *dirname, uint16_t domain, uint8_t bus, char filename[PATH_MAX]; unsigned long tmp; struct rte_pci_device *dev; + char driver[PATH_MAX]; + int ret; dev = malloc(sizeof(*dev)); - if (dev == NULL) { + if (dev == NULL) return -1; - } memset(dev, 0, sizeof(*dev)); dev->addr.domain = domain; @@ -298,6 +328,25 @@ pci_scan_one(const char *dirname, uint16_t domain, uint8_t bus, return -1; } + /* parse driver */ + snprintf(filename, sizeof(filename), "%s/driver", dirname); + ret = pci_get_kernel_driver_by_path(filename, driver); + if (!ret) { + if (!strcmp(driver, "vfio-pci")) + dev->pt_driver = RTE_PT_VFIO; + else if (!strcmp(driver, "igb_uio")) + dev->pt_driver = RTE_PT_IGB_UIO; + else if (!strcmp(driver, "uio_pci_generic")) + dev->pt_driver = RTE_PT_UIO_GENERIC; + else + dev->pt_driver = RTE_PT_UNKNOWN; + } else if (ret < 0) { + RTE_LOG(ERR, EAL, "Fail to get kernel driver\n"); + free(dev); + return -1; + } else + dev->pt_driver = RTE_PT_UNKNOWN; + /* device is valid, add in list (sorted) */ if (TAILQ_EMPTY(&pci_device_list)) { TAILQ_INSERT_TAIL(&pci_device_list, dev, next); -- 1.9.1
From: Michael Qiu <michael.qiu@intel.com> With the driver type flag in struct rte_pci_dev, we do not need to always map uio devices with vfio related function when vfio enabled. Signed-off-by: Michael Qiu <michael.qiu@intel.com> Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp> --- lib/librte_eal/linuxapp/eal/eal_pci.c | 30 +++++++++++++++++------------- 1 file changed, 17 insertions(+), 13 deletions(-) diff --git a/lib/librte_eal/linuxapp/eal/eal_pci.c b/lib/librte_eal/linuxapp/eal/eal_pci.c index bd3f77d..c0ca5a5 100644 --- a/lib/librte_eal/linuxapp/eal/eal_pci.c +++ b/lib/librte_eal/linuxapp/eal/eal_pci.c @@ -549,25 +549,29 @@ pci_config_space_set(struct rte_pci_device *dev) static int pci_map_device(struct rte_pci_device *dev) { - int ret, mapped = 0; + int ret = -1; /* try mapping the NIC resources using VFIO if it exists */ + switch (dev->pt_driver) { + case RTE_PT_VFIO: #ifdef VFIO_PRESENT - if (pci_vfio_is_enabled()) { - ret = pci_vfio_map_resource(dev); - if (ret == 0) - mapped = 1; - else if (ret < 0) - return ret; - } + if (pci_vfio_is_enabled()) + ret = pci_vfio_map_resource(dev); #endif - /* map resources for devices that use igb_uio */ - if (!mapped) { + break; + case RTE_PT_IGB_UIO: + case RTE_PT_UIO_GENERIC: + /* map resources for devices that use uio */ ret = pci_uio_map_resource(dev); - if (ret != 0) - return ret; + break; + default: + RTE_LOG(DEBUG, EAL, " Not managed by known pt driver," + " skipped\n"); + ret = 1; + break; } - return 0; + + return ret; } /* -- 1.9.1
To remove assumption, do like followings. This patch adds "RTE_PCI_DRV_DETACHABLE" to drv_flags of rte_pci_driver structure. The flags indicates the driver can detach devices at runtime. Also remove assumption that port will not be detached. To remove the assumption. - Add 'attached' member to rte_eth_dev structure. This member is used for indicating the port is attached, or not. - Add rte_eth_dev_allocate_new_port(). This function is used for allocating new port. v5: - Change paramters of rte_eth_dev_validate_port() to cleanup code. v4: - Use braces with 'for' loop. - Fix indent of 'if' statement. Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp> --- lib/librte_eal/common/include/rte_pci.h | 2 + lib/librte_ether/rte_ethdev.c | 454 +++++++++++++------------------- lib/librte_ether/rte_ethdev.h | 5 + 3 files changed, 186 insertions(+), 275 deletions(-) diff --git a/lib/librte_eal/common/include/rte_pci.h b/lib/librte_eal/common/include/rte_pci.h index 7b48b55..7f2d699 100644 --- a/lib/librte_eal/common/include/rte_pci.h +++ b/lib/librte_eal/common/include/rte_pci.h @@ -207,6 +207,8 @@ struct rte_pci_driver { #define RTE_PCI_DRV_FORCE_UNBIND 0x0004 /** Device driver supports link state interrupt */ #define RTE_PCI_DRV_INTR_LSC 0x0008 +/** Device driver supports detaching capability */ +#define RTE_PCI_DRV_DETACHABLE 0x0010 /**< Internal use only - Macro used by pci addr parsing functions **/ #define GET_PCIADDR_FIELD(in, fd, lim, dlm) \ diff --git a/lib/librte_ether/rte_ethdev.c b/lib/librte_ether/rte_ethdev.c index ea3a1fb..d70854f 100644 --- a/lib/librte_ether/rte_ethdev.c +++ b/lib/librte_ether/rte_ethdev.c @@ -175,6 +175,16 @@ enum { STAT_QMAP_RX }; +enum { + DEV_INVALID = 0, + DEV_VALID, +}; + +enum { + DEV_DISCONNECTED = 0, + DEV_CONNECTED +}; + static inline void rte_eth_dev_data_alloc(void) { @@ -201,19 +211,34 @@ rte_eth_dev_allocated(const char *name) { unsigned i; - for (i = 0; i < nb_ports; i++) { - if (strcmp(rte_eth_devices[i].data->name, name) == 0) + for (i = 0; i < RTE_MAX_ETHPORTS; i++) { + if ((rte_eth_devices[i].attached == DEV_CONNECTED) && + strcmp(rte_eth_devices[i].data->name, name) == 0) return &rte_eth_devices[i]; } return NULL; } +static uint8_t +rte_eth_dev_allocate_new_port(void) +{ + unsigned i; + + for (i = 0; i < RTE_MAX_ETHPORTS; i++) { + if (rte_eth_devices[i].attached == DEV_DISCONNECTED) + return i; + } + return RTE_MAX_ETHPORTS; +} + struct rte_eth_dev * rte_eth_dev_allocate(const char *name) { + uint8_t port_id; struct rte_eth_dev *eth_dev; - if (nb_ports == RTE_MAX_ETHPORTS) { + port_id = rte_eth_dev_allocate_new_port(); + if (port_id == RTE_MAX_ETHPORTS) { PMD_DEBUG_TRACE("Reached maximum number of Ethernet ports\n"); return NULL; } @@ -226,10 +251,12 @@ rte_eth_dev_allocate(const char *name) return NULL; } - eth_dev = &rte_eth_devices[nb_ports]; - eth_dev->data = &rte_eth_dev_data[nb_ports]; + eth_dev = &rte_eth_devices[port_id]; + eth_dev->data = &rte_eth_dev_data[port_id]; snprintf(eth_dev->data->name, sizeof(eth_dev->data->name), "%s", name); - eth_dev->data->port_id = nb_ports++; + eth_dev->data->port_id = port_id; + eth_dev->attached = DEV_CONNECTED; + nb_ports++; return eth_dev; } @@ -283,6 +310,7 @@ rte_eth_dev_init(struct rte_pci_driver *pci_drv, (unsigned) pci_dev->id.device_id); if (rte_eal_process_type() == RTE_PROC_PRIMARY) rte_free(eth_dev->data->dev_private); + eth_dev->attached = DEV_DISCONNECTED; nb_ports--; return diag; } @@ -308,10 +336,28 @@ rte_eth_driver_register(struct eth_driver *eth_drv) rte_eal_pci_register(ð_drv->pci_drv); } +enum { + NONE_TRACE = 0, + TRACE +}; + +static int +rte_eth_dev_validate_port(uint8_t port_id, int trace) +{ + if (port_id >= RTE_MAX_ETHPORTS || + rte_eth_devices[port_id].attached != DEV_CONNECTED) { + if (trace) { + PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + } + return DEV_INVALID; + } else + return DEV_VALID; +} + int rte_eth_dev_socket_id(uint8_t port_id) { - if (port_id >= nb_ports) + if (rte_eth_dev_validate_port(port_id, NONE_TRACE) == DEV_INVALID) return -1; return rte_eth_devices[port_id].pci_dev->numa_node; } @@ -369,10 +415,8 @@ rte_eth_dev_rx_queue_start(uint8_t port_id, uint16_t rx_queue_id) * in a multi-process setup*/ PROC_PRIMARY_OR_ERR_RET(-E_RTE_SECONDARY); - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return -EINVAL; - } dev = &rte_eth_devices[port_id]; if (rx_queue_id >= dev->data->nb_rx_queues) { @@ -395,10 +439,8 @@ rte_eth_dev_rx_queue_stop(uint8_t port_id, uint16_t rx_queue_id) * in a multi-process setup*/ PROC_PRIMARY_OR_ERR_RET(-E_RTE_SECONDARY); - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return -EINVAL; - } dev = &rte_eth_devices[port_id]; if (rx_queue_id >= dev->data->nb_rx_queues) { @@ -421,10 +463,8 @@ rte_eth_dev_tx_queue_start(uint8_t port_id, uint16_t tx_queue_id) * in a multi-process setup*/ PROC_PRIMARY_OR_ERR_RET(-E_RTE_SECONDARY); - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return -EINVAL; - } dev = &rte_eth_devices[port_id]; if (tx_queue_id >= dev->data->nb_tx_queues) { @@ -447,10 +487,8 @@ rte_eth_dev_tx_queue_stop(uint8_t port_id, uint16_t tx_queue_id) * in a multi-process setup*/ PROC_PRIMARY_OR_ERR_RET(-E_RTE_SECONDARY); - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return -EINVAL; - } dev = &rte_eth_devices[port_id]; if (tx_queue_id >= dev->data->nb_tx_queues) { @@ -703,10 +741,9 @@ rte_eth_dev_configure(uint8_t port_id, uint16_t nb_rx_q, uint16_t nb_tx_q, * in a multi-process setup*/ PROC_PRIMARY_OR_ERR_RET(-E_RTE_SECONDARY); - if (port_id >= nb_ports || port_id >= RTE_MAX_ETHPORTS) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return (-EINVAL); - } + dev = &rte_eth_devices[port_id]; FUNC_PTR_OR_ERR_RET(*dev->dev_ops->dev_infos_get, -ENOTSUP); @@ -888,10 +925,9 @@ rte_eth_dev_start(uint8_t port_id) * in a multi-process setup*/ PROC_PRIMARY_OR_ERR_RET(-E_RTE_SECONDARY); - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%" PRIu8 "\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return (-EINVAL); - } + dev = &rte_eth_devices[port_id]; FUNC_PTR_OR_ERR_RET(*dev->dev_ops->dev_start, -ENOTSUP); @@ -923,10 +959,9 @@ rte_eth_dev_stop(uint8_t port_id) * in a multi-process setup*/ PROC_PRIMARY_OR_RET(); - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%" PRIu8 "\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return; - } + dev = &rte_eth_devices[port_id]; FUNC_PTR_OR_RET(*dev->dev_ops->dev_stop); @@ -951,10 +986,9 @@ rte_eth_dev_set_link_up(uint8_t port_id) * in a multi-process setup*/ PROC_PRIMARY_OR_ERR_RET(-E_RTE_SECONDARY); - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return -EINVAL; - } + dev = &rte_eth_devices[port_id]; FUNC_PTR_OR_ERR_RET(*dev->dev_ops->dev_set_link_up, -ENOTSUP); @@ -970,10 +1004,9 @@ rte_eth_dev_set_link_down(uint8_t port_id) * in a multi-process setup*/ PROC_PRIMARY_OR_ERR_RET(-E_RTE_SECONDARY); - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return -EINVAL; - } + dev = &rte_eth_devices[port_id]; FUNC_PTR_OR_ERR_RET(*dev->dev_ops->dev_set_link_down, -ENOTSUP); @@ -989,10 +1022,8 @@ rte_eth_dev_close(uint8_t port_id) * in a multi-process setup*/ PROC_PRIMARY_OR_RET(); - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return; - } dev = &rte_eth_devices[port_id]; @@ -1017,10 +1048,9 @@ rte_eth_rx_queue_setup(uint8_t port_id, uint16_t rx_queue_id, * in a multi-process setup*/ PROC_PRIMARY_OR_ERR_RET(-E_RTE_SECONDARY); - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return (-EINVAL); - } + dev = &rte_eth_devices[port_id]; if (rx_queue_id >= dev->data->nb_rx_queues) { PMD_DEBUG_TRACE("Invalid RX queue_id=%d\n", rx_queue_id); @@ -1090,10 +1120,9 @@ rte_eth_tx_queue_setup(uint8_t port_id, uint16_t tx_queue_id, * in a multi-process setup*/ PROC_PRIMARY_OR_ERR_RET(-E_RTE_SECONDARY); - if (port_id >= RTE_MAX_ETHPORTS || port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return (-EINVAL); - } + dev = &rte_eth_devices[port_id]; if (tx_queue_id >= dev->data->nb_tx_queues) { PMD_DEBUG_TRACE("Invalid TX queue_id=%d\n", tx_queue_id); @@ -1123,10 +1152,9 @@ rte_eth_promiscuous_enable(uint8_t port_id) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return; - } + dev = &rte_eth_devices[port_id]; FUNC_PTR_OR_RET(*dev->dev_ops->promiscuous_enable); @@ -1139,10 +1167,9 @@ rte_eth_promiscuous_disable(uint8_t port_id) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return; - } + dev = &rte_eth_devices[port_id]; FUNC_PTR_OR_RET(*dev->dev_ops->promiscuous_disable); @@ -1155,10 +1182,8 @@ rte_eth_promiscuous_get(uint8_t port_id) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return -1; - } dev = &rte_eth_devices[port_id]; return dev->data->promiscuous; @@ -1169,10 +1194,9 @@ rte_eth_allmulticast_enable(uint8_t port_id) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return; - } + dev = &rte_eth_devices[port_id]; FUNC_PTR_OR_RET(*dev->dev_ops->allmulticast_enable); @@ -1185,10 +1209,9 @@ rte_eth_allmulticast_disable(uint8_t port_id) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return; - } + dev = &rte_eth_devices[port_id]; FUNC_PTR_OR_RET(*dev->dev_ops->allmulticast_disable); @@ -1201,10 +1224,8 @@ rte_eth_allmulticast_get(uint8_t port_id) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return -1; - } dev = &rte_eth_devices[port_id]; return dev->data->all_multicast; @@ -1229,10 +1250,9 @@ rte_eth_link_get(uint8_t port_id, struct rte_eth_link *eth_link) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return; - } + dev = &rte_eth_devices[port_id]; if (dev->data->dev_conf.intr_conf.lsc != 0) @@ -1249,10 +1269,9 @@ rte_eth_link_get_nowait(uint8_t port_id, struct rte_eth_link *eth_link) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return; - } + dev = &rte_eth_devices[port_id]; if (dev->data->dev_conf.intr_conf.lsc != 0) @@ -1269,10 +1288,9 @@ rte_eth_stats_get(uint8_t port_id, struct rte_eth_stats *stats) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return; - } + dev = &rte_eth_devices[port_id]; memset(stats, 0, sizeof(*stats)); @@ -1286,10 +1304,9 @@ rte_eth_stats_reset(uint8_t port_id) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return; - } + dev = &rte_eth_devices[port_id]; FUNC_PTR_OR_RET(*dev->dev_ops->stats_reset); @@ -1307,10 +1324,9 @@ rte_eth_xstats_get(uint8_t port_id, struct rte_eth_xstats *xstats, uint64_t val; char *stats_ptr; - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return -1; - } + dev = &rte_eth_devices[port_id]; /* implemented by the driver */ @@ -1376,10 +1392,9 @@ rte_eth_xstats_reset(uint8_t port_id) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return; - } + dev = &rte_eth_devices[port_id]; /* implemented by the driver */ @@ -1398,10 +1413,9 @@ set_queue_stats_mapping(uint8_t port_id, uint16_t queue_id, uint8_t stat_idx, { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return -ENODEV; - } + dev = &rte_eth_devices[port_id]; FUNC_PTR_OR_ERR_RET(*dev->dev_ops->queue_stats_mapping_set, -ENOTSUP); @@ -1433,10 +1447,9 @@ rte_eth_dev_info_get(uint8_t port_id, struct rte_eth_dev_info *dev_info) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return; - } + dev = &rte_eth_devices[port_id]; memset(dev_info, 0, sizeof(struct rte_eth_dev_info)); @@ -1453,10 +1466,9 @@ rte_eth_macaddr_get(uint8_t port_id, struct ether_addr *mac_addr) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return; - } + dev = &rte_eth_devices[port_id]; ether_addr_copy(&dev->data->mac_addrs[0], mac_addr); } @@ -1467,10 +1479,8 @@ rte_eth_dev_get_mtu(uint8_t port_id, uint16_t *mtu) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return (-ENODEV); - } dev = &rte_eth_devices[port_id]; *mtu = dev->data->mtu; @@ -1483,10 +1493,8 @@ rte_eth_dev_set_mtu(uint8_t port_id, uint16_t mtu) int ret; struct rte_eth_dev *dev; - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return (-ENODEV); - } dev = &rte_eth_devices[port_id]; FUNC_PTR_OR_ERR_RET(*dev->dev_ops->mtu_set, -ENOTSUP); @@ -1503,10 +1511,9 @@ rte_eth_dev_vlan_filter(uint8_t port_id, uint16_t vlan_id, int on) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return (-ENODEV); - } + dev = &rte_eth_devices[port_id]; if (! (dev->data->dev_conf.rxmode.hw_vlan_filter)) { PMD_DEBUG_TRACE("port %d: vlan-filtering disabled\n", port_id); @@ -1528,10 +1535,8 @@ rte_eth_dev_set_vlan_strip_on_queue(uint8_t port_id, uint16_t rx_queue_id, int o { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return (-ENODEV); - } dev = &rte_eth_devices[port_id]; if (rx_queue_id >= dev->data->nb_rx_queues) { @@ -1550,10 +1555,8 @@ rte_eth_dev_set_vlan_ether_type(uint8_t port_id, uint16_t tpid) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return (-ENODEV); - } dev = &rte_eth_devices[port_id]; FUNC_PTR_OR_ERR_RET(*dev->dev_ops->vlan_tpid_set, -ENOTSUP); @@ -1570,10 +1573,8 @@ rte_eth_dev_set_vlan_offload(uint8_t port_id, int offload_mask) int mask = 0; int cur, org = 0; - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return (-ENODEV); - } dev = &rte_eth_devices[port_id]; @@ -1615,10 +1616,8 @@ rte_eth_dev_get_vlan_offload(uint8_t port_id) struct rte_eth_dev *dev; int ret = 0; - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return (-ENODEV); - } dev = &rte_eth_devices[port_id]; @@ -1639,10 +1638,9 @@ rte_eth_dev_set_vlan_pvid(uint8_t port_id, uint16_t pvid, int on) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return (-ENODEV); - } + dev = &rte_eth_devices[port_id]; FUNC_PTR_OR_ERR_RET(*dev->dev_ops->vlan_pvid_set, -ENOTSUP); (*dev->dev_ops->vlan_pvid_set)(dev, pvid, on); @@ -1657,10 +1655,8 @@ rte_eth_dev_fdir_add_signature_filter(uint8_t port_id, { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return (-ENODEV); - } dev = &rte_eth_devices[port_id]; @@ -1691,10 +1687,8 @@ rte_eth_dev_fdir_update_signature_filter(uint8_t port_id, { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return (-ENODEV); - } dev = &rte_eth_devices[port_id]; @@ -1725,10 +1719,8 @@ rte_eth_dev_fdir_remove_signature_filter(uint8_t port_id, { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return (-ENODEV); - } dev = &rte_eth_devices[port_id]; @@ -1756,10 +1748,8 @@ rte_eth_dev_fdir_get_infos(uint8_t port_id, struct rte_eth_fdir *fdir) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return (-ENODEV); - } dev = &rte_eth_devices[port_id]; if (! (dev->data->dev_conf.fdir_conf.mode)) { @@ -1781,10 +1771,8 @@ rte_eth_dev_fdir_add_perfect_filter(uint8_t port_id, { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return (-ENODEV); - } dev = &rte_eth_devices[port_id]; @@ -1821,10 +1809,8 @@ rte_eth_dev_fdir_update_perfect_filter(uint8_t port_id, { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return (-ENODEV); - } dev = &rte_eth_devices[port_id]; @@ -1859,10 +1845,8 @@ rte_eth_dev_fdir_remove_perfect_filter(uint8_t port_id, { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return (-ENODEV); - } dev = &rte_eth_devices[port_id]; @@ -1895,10 +1879,8 @@ rte_eth_dev_fdir_set_masks(uint8_t port_id, struct rte_fdir_masks *fdir_mask) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return (-ENODEV); - } dev = &rte_eth_devices[port_id]; if (! (dev->data->dev_conf.fdir_conf.mode)) { @@ -1915,10 +1897,8 @@ rte_eth_dev_flow_ctrl_get(uint8_t port_id, struct rte_eth_fc_conf *fc_conf) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return (-ENODEV); - } dev = &rte_eth_devices[port_id]; FUNC_PTR_OR_ERR_RET(*dev->dev_ops->flow_ctrl_get, -ENOTSUP); @@ -1931,10 +1911,8 @@ rte_eth_dev_flow_ctrl_set(uint8_t port_id, struct rte_eth_fc_conf *fc_conf) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return (-ENODEV); - } if ((fc_conf->send_xon != 0) && (fc_conf->send_xon != 1)) { PMD_DEBUG_TRACE("Invalid send_xon, only 0/1 allowed\n"); @@ -1951,10 +1929,8 @@ rte_eth_dev_priority_flow_ctrl_set(uint8_t port_id, struct rte_eth_pfc_conf *pfc { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return (-ENODEV); - } if (pfc_conf->priority > (ETH_DCB_NUM_USER_PRIORITIES - 1)) { PMD_DEBUG_TRACE("Invalid priority, only 0-7 allowed\n"); @@ -2030,10 +2006,8 @@ rte_eth_dev_rss_reta_update(uint8_t port_id, struct rte_eth_dev *dev; int ret; - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return -ENODEV; - } /* Check mask bits */ ret = rte_eth_check_reta_mask(reta_conf, reta_size); @@ -2081,10 +2055,9 @@ rte_eth_dev_rss_hash_update(uint8_t port_id, struct rte_eth_rss_conf *rss_conf) struct rte_eth_dev *dev; uint16_t rss_hash_protos; - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return (-ENODEV); - } + rss_hash_protos = rss_conf->rss_hf; if ((rss_hash_protos != 0) && ((rss_hash_protos & ETH_RSS_PROTO_MASK) == 0)) { @@ -2103,10 +2076,9 @@ rte_eth_dev_rss_hash_conf_get(uint8_t port_id, { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return (-ENODEV); - } + dev = &rte_eth_devices[port_id]; FUNC_PTR_OR_ERR_RET(*dev->dev_ops->rss_hash_conf_get, -ENOTSUP); return (*dev->dev_ops->rss_hash_conf_get)(dev, rss_conf); @@ -2118,10 +2090,8 @@ rte_eth_dev_udp_tunnel_add(uint8_t port_id, { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return -ENODEV; - } if (udp_tunnel == NULL) { PMD_DEBUG_TRACE("Invalid udp_tunnel parameter\n"); @@ -2144,10 +2114,9 @@ rte_eth_dev_udp_tunnel_delete(uint8_t port_id, { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return -ENODEV; - } + dev = &rte_eth_devices[port_id]; if (udp_tunnel == NULL) { @@ -2169,10 +2138,8 @@ rte_eth_led_on(uint8_t port_id) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return (-ENODEV); - } dev = &rte_eth_devices[port_id]; FUNC_PTR_OR_ERR_RET(*dev->dev_ops->dev_led_on, -ENOTSUP); @@ -2184,10 +2151,8 @@ rte_eth_led_off(uint8_t port_id) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return (-ENODEV); - } dev = &rte_eth_devices[port_id]; FUNC_PTR_OR_ERR_RET(*dev->dev_ops->dev_led_off, -ENOTSUP); @@ -2224,10 +2189,9 @@ rte_eth_dev_mac_addr_add(uint8_t port_id, struct ether_addr *addr, int index; uint64_t pool_mask; - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return (-ENODEV); - } + dev = &rte_eth_devices[port_id]; FUNC_PTR_OR_ERR_RET(*dev->dev_ops->mac_addr_add, -ENOTSUP); @@ -2275,10 +2239,9 @@ rte_eth_dev_mac_addr_remove(uint8_t port_id, struct ether_addr *addr) struct rte_eth_dev *dev; int index; - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return (-ENODEV); - } + dev = &rte_eth_devices[port_id]; FUNC_PTR_OR_ERR_RET(*dev->dev_ops->mac_addr_remove, -ENOTSUP); @@ -2309,7 +2272,7 @@ rte_eth_dev_set_vf_rxmode(uint8_t port_id, uint16_t vf, struct rte_eth_dev *dev; struct rte_eth_dev_info dev_info; - if (port_id >= nb_ports) { + if (rte_eth_dev_validate_port(port_id, NONE_TRACE) == DEV_INVALID) { PMD_DEBUG_TRACE("set VF RX mode:Invalid port_id=%d\n", port_id); return (-ENODEV); @@ -2364,7 +2327,7 @@ rte_eth_dev_uc_hash_table_set(uint8_t port_id, struct ether_addr *addr, int ret; struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (rte_eth_dev_validate_port(port_id, NONE_TRACE) == DEV_INVALID) { PMD_DEBUG_TRACE("unicast hash setting:Invalid port_id=%d\n", port_id); return (-ENODEV); @@ -2417,7 +2380,7 @@ rte_eth_dev_uc_all_hash_table_set(uint8_t port_id, uint8_t on) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (rte_eth_dev_validate_port(port_id, NONE_TRACE) == DEV_INVALID) { PMD_DEBUG_TRACE("unicast hash setting:Invalid port_id=%d\n", port_id); return (-ENODEV); @@ -2436,10 +2399,8 @@ rte_eth_dev_set_vf_rx(uint8_t port_id,uint16_t vf, uint8_t on) struct rte_eth_dev *dev; struct rte_eth_dev_info dev_info; - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return (-ENODEV); - } dev = &rte_eth_devices[port_id]; rte_eth_dev_info_get(port_id, &dev_info); @@ -2462,7 +2423,7 @@ rte_eth_dev_set_vf_tx(uint8_t port_id,uint16_t vf, uint8_t on) struct rte_eth_dev *dev; struct rte_eth_dev_info dev_info; - if (port_id >= nb_ports) { + if (rte_eth_dev_validate_port(port_id, NONE_TRACE) == DEV_INVALID) { PMD_DEBUG_TRACE("set pool tx:Invalid port_id=%d\n", port_id); return (-ENODEV); } @@ -2487,7 +2448,7 @@ rte_eth_dev_set_vf_vlan_filter(uint8_t port_id, uint16_t vlan_id, { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (rte_eth_dev_validate_port(port_id, NONE_TRACE) == DEV_INVALID) { PMD_DEBUG_TRACE("VF VLAN filter:invalid port id=%d\n", port_id); return (-ENODEV); @@ -2518,7 +2479,7 @@ int rte_eth_set_queue_rate_limit(uint8_t port_id, uint16_t queue_idx, struct rte_eth_dev_info dev_info; struct rte_eth_link link; - if (port_id >= nb_ports) { + if (rte_eth_dev_validate_port(port_id, NONE_TRACE) == DEV_INVALID) { PMD_DEBUG_TRACE("set queue rate limit:invalid port id=%d\n", port_id); return -ENODEV; @@ -2555,7 +2516,7 @@ int rte_eth_set_vf_rate_limit(uint8_t port_id, uint16_t vf, uint16_t tx_rate, if (q_msk == 0) return 0; - if (port_id >= nb_ports) { + if (rte_eth_dev_validate_port(port_id, NONE_TRACE) == DEV_INVALID) { PMD_DEBUG_TRACE("set VF rate limit:invalid port id=%d\n", port_id); return -ENODEV; @@ -2589,10 +2550,8 @@ rte_eth_mirror_rule_set(uint8_t port_id, { struct rte_eth_dev *dev = &rte_eth_devices[port_id]; - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return (-ENODEV); - } if (mirror_conf->rule_type_mask == 0) { PMD_DEBUG_TRACE("mirror rule type can not be 0.\n"); @@ -2630,10 +2589,8 @@ rte_eth_mirror_rule_reset(uint8_t port_id, uint8_t rule_id) { struct rte_eth_dev *dev = &rte_eth_devices[port_id]; - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return (-ENODEV); - } if(rule_id >= ETH_VMDQ_NUM_MIRROR_RULE) { @@ -2655,10 +2612,9 @@ rte_eth_rx_burst(uint8_t port_id, uint16_t queue_id, { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return 0; - } + dev = &rte_eth_devices[port_id]; FUNC_PTR_OR_ERR_RET(*dev->rx_pkt_burst, 0); if (queue_id >= dev->data->nb_rx_queues) { @@ -2675,10 +2631,9 @@ rte_eth_tx_burst(uint8_t port_id, uint16_t queue_id, { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return 0; - } + dev = &rte_eth_devices[port_id]; FUNC_PTR_OR_ERR_RET(*dev->tx_pkt_burst, 0); @@ -2695,10 +2650,9 @@ rte_eth_rx_queue_count(uint8_t port_id, uint16_t queue_id) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return 0; - } + dev = &rte_eth_devices[port_id]; FUNC_PTR_OR_ERR_RET(*dev->dev_ops->rx_queue_count, 0); return (*dev->dev_ops->rx_queue_count)(dev, queue_id); @@ -2709,10 +2663,9 @@ rte_eth_rx_descriptor_done(uint8_t port_id, uint16_t queue_id, uint16_t offset) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return (-ENODEV); - } + dev = &rte_eth_devices[port_id]; FUNC_PTR_OR_ERR_RET(*dev->dev_ops->rx_descriptor_done, -ENOTSUP); return (*dev->dev_ops->rx_descriptor_done)( \ @@ -2730,10 +2683,8 @@ rte_eth_dev_callback_register(uint8_t port_id, if (!cb_fn) return (-EINVAL); - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return (-EINVAL); - } dev = &rte_eth_devices[port_id]; rte_spinlock_lock(&rte_eth_dev_cb_lock); @@ -2770,10 +2721,8 @@ rte_eth_dev_callback_unregister(uint8_t port_id, if (!cb_fn) return (-EINVAL); - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return (-EINVAL); - } dev = &rte_eth_devices[port_id]; rte_spinlock_lock(&rte_eth_dev_cb_lock); @@ -2830,10 +2779,8 @@ int rte_eth_dev_bypass_init(uint8_t port_id) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return (-ENODEV); - } if ((dev= &rte_eth_devices[port_id]) == NULL) { PMD_DEBUG_TRACE("Invalid port device\n"); @@ -2850,10 +2797,8 @@ rte_eth_dev_bypass_state_show(uint8_t port_id, uint32_t *state) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return (-ENODEV); - } if ((dev= &rte_eth_devices[port_id]) == NULL) { PMD_DEBUG_TRACE("Invalid port device\n"); @@ -2869,10 +2814,8 @@ rte_eth_dev_bypass_state_set(uint8_t port_id, uint32_t *new_state) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return (-ENODEV); - } if ((dev= &rte_eth_devices[port_id]) == NULL) { PMD_DEBUG_TRACE("Invalid port device\n"); @@ -2889,10 +2832,8 @@ rte_eth_dev_bypass_event_show(uint8_t port_id, uint32_t event, uint32_t *state) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return (-ENODEV); - } if ((dev= &rte_eth_devices[port_id]) == NULL) { PMD_DEBUG_TRACE("Invalid port device\n"); @@ -2909,10 +2850,8 @@ rte_eth_dev_bypass_event_store(uint8_t port_id, uint32_t event, uint32_t state) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return (-ENODEV); - } if ((dev= &rte_eth_devices[port_id]) == NULL) { PMD_DEBUG_TRACE("Invalid port device\n"); @@ -2929,10 +2868,8 @@ rte_eth_dev_wd_timeout_store(uint8_t port_id, uint32_t timeout) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return (-ENODEV); - } if ((dev= &rte_eth_devices[port_id]) == NULL) { PMD_DEBUG_TRACE("Invalid port device\n"); @@ -2949,10 +2886,8 @@ rte_eth_dev_bypass_ver_show(uint8_t port_id, uint32_t *ver) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return (-ENODEV); - } if ((dev= &rte_eth_devices[port_id]) == NULL) { PMD_DEBUG_TRACE("Invalid port device\n"); @@ -2969,10 +2904,8 @@ rte_eth_dev_bypass_wd_timeout_show(uint8_t port_id, uint32_t *wd_timeout) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return (-ENODEV); - } if ((dev= &rte_eth_devices[port_id]) == NULL) { PMD_DEBUG_TRACE("Invalid port device\n"); @@ -2989,10 +2922,8 @@ rte_eth_dev_bypass_wd_reset(uint8_t port_id) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return (-ENODEV); - } if ((dev= &rte_eth_devices[port_id]) == NULL) { PMD_DEBUG_TRACE("Invalid port device\n"); @@ -3011,10 +2942,8 @@ rte_eth_dev_add_syn_filter(uint8_t port_id, { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return -ENODEV; - } dev = &rte_eth_devices[port_id]; FUNC_PTR_OR_ERR_RET(*dev->dev_ops->add_syn_filter, -ENOTSUP); @@ -3026,10 +2955,8 @@ rte_eth_dev_remove_syn_filter(uint8_t port_id) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return -ENODEV; - } dev = &rte_eth_devices[port_id]; FUNC_PTR_OR_ERR_RET(*dev->dev_ops->remove_syn_filter, -ENOTSUP); @@ -3045,10 +2972,8 @@ rte_eth_dev_get_syn_filter(uint8_t port_id, if (filter == NULL || rx_queue == NULL) return -EINVAL; - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return -ENODEV; - } dev = &rte_eth_devices[port_id]; FUNC_PTR_OR_ERR_RET(*dev->dev_ops->get_syn_filter, -ENOTSUP); @@ -3061,10 +2986,9 @@ rte_eth_dev_add_2tuple_filter(uint8_t port_id, uint16_t index, { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return -ENODEV; - } + if (filter->protocol != IPPROTO_TCP && filter->tcp_flags != 0){ PMD_DEBUG_TRACE("tcp flags is 0x%x, but the protocol value" @@ -3083,10 +3007,8 @@ rte_eth_dev_remove_2tuple_filter(uint8_t port_id, uint16_t index) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return -ENODEV; - } dev = &rte_eth_devices[port_id]; FUNC_PTR_OR_ERR_RET(*dev->dev_ops->remove_2tuple_filter, -ENOTSUP); @@ -3102,10 +3024,8 @@ rte_eth_dev_get_2tuple_filter(uint8_t port_id, uint16_t index, if (filter == NULL || rx_queue == NULL) return -EINVAL; - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return -ENODEV; - } dev = &rte_eth_devices[port_id]; FUNC_PTR_OR_ERR_RET(*dev->dev_ops->get_2tuple_filter, -ENOTSUP); @@ -3118,10 +3038,8 @@ rte_eth_dev_add_5tuple_filter(uint8_t port_id, uint16_t index, { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return -ENODEV; - } if (filter->protocol != IPPROTO_TCP && filter->tcp_flags != 0){ @@ -3141,10 +3059,8 @@ rte_eth_dev_remove_5tuple_filter(uint8_t port_id, uint16_t index) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return -ENODEV; - } dev = &rte_eth_devices[port_id]; FUNC_PTR_OR_ERR_RET(*dev->dev_ops->remove_5tuple_filter, -ENOTSUP); @@ -3160,10 +3076,8 @@ rte_eth_dev_get_5tuple_filter(uint8_t port_id, uint16_t index, if (filter == NULL || rx_queue == NULL) return -EINVAL; - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return -ENODEV; - } dev = &rte_eth_devices[port_id]; FUNC_PTR_OR_ERR_RET(*dev->dev_ops->get_5tuple_filter, -ENOTSUP); @@ -3177,10 +3091,8 @@ rte_eth_dev_add_flex_filter(uint8_t port_id, uint16_t index, { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return -ENODEV; - } dev = &rte_eth_devices[port_id]; FUNC_PTR_OR_ERR_RET(*dev->dev_ops->add_flex_filter, -ENOTSUP); @@ -3192,10 +3104,8 @@ rte_eth_dev_remove_flex_filter(uint8_t port_id, uint16_t index) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return -ENODEV; - } dev = &rte_eth_devices[port_id]; FUNC_PTR_OR_ERR_RET(*dev->dev_ops->remove_flex_filter, -ENOTSUP); @@ -3211,10 +3121,8 @@ rte_eth_dev_get_flex_filter(uint8_t port_id, uint16_t index, if (filter == NULL || rx_queue == NULL) return -EINVAL; - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return -ENODEV; - } dev = &rte_eth_devices[port_id]; FUNC_PTR_OR_ERR_RET(*dev->dev_ops->get_flex_filter, -ENOTSUP); @@ -3227,10 +3135,8 @@ rte_eth_dev_filter_supported(uint8_t port_id, enum rte_filter_type filter_type) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return -ENODEV; - } dev = &rte_eth_devices[port_id]; FUNC_PTR_OR_ERR_RET(*dev->dev_ops->filter_ctrl, -ENOTSUP); @@ -3244,10 +3150,8 @@ rte_eth_dev_filter_ctrl(uint8_t port_id, enum rte_filter_type filter_type, { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return -ENODEV; - } dev = &rte_eth_devices[port_id]; FUNC_PTR_OR_ERR_RET(*dev->dev_ops->filter_ctrl, -ENOTSUP); diff --git a/lib/librte_ether/rte_ethdev.h b/lib/librte_ether/rte_ethdev.h index 1200c1c..ca101f5 100644 --- a/lib/librte_ether/rte_ethdev.h +++ b/lib/librte_ether/rte_ethdev.h @@ -1539,6 +1539,7 @@ struct rte_eth_dev { struct eth_dev_ops *dev_ops; /**< Functions exported by PMD */ struct rte_pci_device *pci_dev; /**< PCI info. supplied by probing */ struct rte_eth_dev_cb_list callbacks; /**< User application callbacks */ + uint8_t attached; /**< Flag indicating the port is attached */ }; struct rte_eth_dev_sriov { @@ -1604,6 +1605,10 @@ extern struct rte_eth_dev rte_eth_devices[]; * initialized by the [matching] Ethernet driver during the PCI probing phase. * All devices whose port identifier is in the range * [0, rte_eth_dev_count() - 1] can be operated on by network applications. + * immediately after invoking rte_eal_init(). + * If the application unplugs a port using hotplug function, The enabled port + * numbers may be noncontiguous. In the case, the applications need to manage + * enabled port by themselves. * * @return * - The total number of usable Ethernet devices. -- 1.9.1
This patch replaces pci_addr_comparison() and memcmp() of pci addresses by eal_compare_pci_addr(). v5: - Fix pci_scan_one to handle pt_driver correctly. v4: - Fix calculation method of eal_compare_pci_addr(). - Add parameter checking. Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp> --- lib/librte_eal/bsdapp/eal/eal_pci.c | 25 ++++++++--------------- lib/librte_eal/common/eal_common_pci.c | 2 +- lib/librte_eal/common/include/rte_pci.h | 34 +++++++++++++++++++++++++++++++ lib/librte_eal/linuxapp/eal/eal_pci.c | 25 ++++++++--------------- lib/librte_eal/linuxapp/eal/eal_pci_uio.c | 2 +- 5 files changed, 54 insertions(+), 34 deletions(-) diff --git a/lib/librte_eal/bsdapp/eal/eal_pci.c b/lib/librte_eal/bsdapp/eal/eal_pci.c index 74ecce7..c844d58 100644 --- a/lib/librte_eal/bsdapp/eal/eal_pci.c +++ b/lib/librte_eal/bsdapp/eal/eal_pci.c @@ -270,20 +270,6 @@ pci_uio_map_resource(struct rte_pci_device *dev) return (0); } -/* Compare two PCI device addresses. */ -static int -pci_addr_comparison(struct rte_pci_addr *addr, struct rte_pci_addr *addr2) -{ - uint64_t dev_addr = (addr->domain << 24) + (addr->bus << 16) + (addr->devid << 8) + addr->function; - uint64_t dev_addr2 = (addr2->domain << 24) + (addr2->bus << 16) + (addr2->devid << 8) + addr2->function; - - if (dev_addr > dev_addr2) - return 1; - else - return 0; -} - - /* Scan one pci sysfs entry, and fill the devices list from it. */ static int pci_scan_one(int dev_pci_fd, struct pci_conf *conf) @@ -356,13 +342,20 @@ pci_scan_one(int dev_pci_fd, struct pci_conf *conf) } else { struct rte_pci_device *dev2 = NULL; + int ret; TAILQ_FOREACH(dev2, &pci_device_list, next) { - if (pci_addr_comparison(&dev->addr, &dev2->addr)) + ret = eal_compare_pci_addr(&dev->addr, &dev2->addr); + if (ret > 0) continue; - else { + else if (ret < 0) { TAILQ_INSERT_BEFORE(dev2, dev, next); return 0; + } else { /* already registered */ + /* update pt_driver */ + dev2->pt_driver = dev->pt_driver; + free(dev); + return 0; } } TAILQ_INSERT_TAIL(&pci_device_list, dev, next); diff --git a/lib/librte_eal/common/eal_common_pci.c b/lib/librte_eal/common/eal_common_pci.c index f3c7f71..a89f5c3 100644 --- a/lib/librte_eal/common/eal_common_pci.c +++ b/lib/librte_eal/common/eal_common_pci.c @@ -93,7 +93,7 @@ static struct rte_devargs *pci_devargs_lookup(struct rte_pci_device *dev) if (devargs->type != RTE_DEVTYPE_BLACKLISTED_PCI && devargs->type != RTE_DEVTYPE_WHITELISTED_PCI) continue; - if (!memcmp(&dev->addr, &devargs->pci.addr, sizeof(dev->addr))) + if (!eal_compare_pci_addr(&dev->addr, &devargs->pci.addr)) return devargs; } return NULL; diff --git a/lib/librte_eal/common/include/rte_pci.h b/lib/librte_eal/common/include/rte_pci.h index 7f2d699..4814cd7 100644 --- a/lib/librte_eal/common/include/rte_pci.h +++ b/lib/librte_eal/common/include/rte_pci.h @@ -269,6 +269,40 @@ eal_parse_pci_DomBDF(const char *input, struct rte_pci_addr *dev_addr) } #undef GET_PCIADDR_FIELD +/* Compare two PCI device addresses. */ +/** + * Utility function to compare two PCI device addresses. + * + * @param addr + * The PCI Bus-Device-Function address to compare + * @param addr2 + * The PCI Bus-Device-Function address to compare + * @return + * 0 on equal PCI address. + * Positive on addr is greater than addr2. + * Negative on addr is less than addr2, or error. + */ +static inline int +eal_compare_pci_addr(struct rte_pci_addr *addr, struct rte_pci_addr *addr2) +{ + uint64_t dev_addr, dev_addr2; + + if ((addr == NULL) || (addr2 == NULL)) + return -1; + + dev_addr = (addr->domain << 24) | (addr->bus << 16) | + (addr->devid << 8) | addr->function; + dev_addr2 = (addr2->domain << 24) | (addr2->bus << 16) | + (addr2->devid << 8) | addr2->function; + + if (dev_addr > dev_addr2) + return 1; + else if (dev_addr < dev_addr2) + return -1; + else + return 0; +} + /** * Probe the PCI bus for registered drivers. * diff --git a/lib/librte_eal/linuxapp/eal/eal_pci.c b/lib/librte_eal/linuxapp/eal/eal_pci.c index c0ca5a5..d847102 100644 --- a/lib/librte_eal/linuxapp/eal/eal_pci.c +++ b/lib/librte_eal/linuxapp/eal/eal_pci.c @@ -229,20 +229,6 @@ error: return -1; } -/* Compare two PCI device addresses. */ -static int -pci_addr_comparison(struct rte_pci_addr *addr, struct rte_pci_addr *addr2) -{ - uint64_t dev_addr = (addr->domain << 24) + (addr->bus << 16) + (addr->devid << 8) + addr->function; - uint64_t dev_addr2 = (addr2->domain << 24) + (addr2->bus << 16) + (addr2->devid << 8) + addr2->function; - - if (dev_addr > dev_addr2) - return 1; - else - return 0; -} - - /* Scan one pci sysfs entry, and fill the devices list from it. */ static int pci_scan_one(const char *dirname, uint16_t domain, uint8_t bus, @@ -353,13 +339,20 @@ pci_scan_one(const char *dirname, uint16_t domain, uint8_t bus, } else { struct rte_pci_device *dev2 = NULL; + int ret; TAILQ_FOREACH(dev2, &pci_device_list, next) { - if (pci_addr_comparison(&dev->addr, &dev2->addr)) + ret = eal_compare_pci_addr(&dev->addr, &dev2->addr); + if (ret > 0) continue; - else { + else if (ret < 0) { TAILQ_INSERT_BEFORE(dev2, dev, next); return 0; + } else { /* already registered */ + /* update pt_driver */ + dev2->pt_driver = dev->pt_driver; + free(dev); + return 0; } } TAILQ_INSERT_TAIL(&pci_device_list, dev, next); diff --git a/lib/librte_eal/linuxapp/eal/eal_pci_uio.c b/lib/librte_eal/linuxapp/eal/eal_pci_uio.c index e53f06b..1da3507 100644 --- a/lib/librte_eal/linuxapp/eal/eal_pci_uio.c +++ b/lib/librte_eal/linuxapp/eal/eal_pci_uio.c @@ -123,7 +123,7 @@ pci_uio_map_secondary(struct rte_pci_device *dev) TAILQ_FOREACH(uio_res, pci_res_list, next) { /* skip this element if it doesn't match our PCI address */ - if (memcmp(&uio_res->pci_addr, &dev->addr, sizeof(dev->addr))) + if (eal_compare_pci_addr(&uio_res->pci_addr, &dev->addr)) continue; for (i = 0; i != uio_res->nb_maps; i++) { -- 1.9.1
This patch adds rte_eth_dev_free(). The function is used for changing a attached status of the device that has specified name. v6: - Use rte_eth_dev structure as the paramter of rte_eth_dev_free(). v4: - Add paramerter checking. Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp> --- lib/librte_ether/rte_ethdev.c | 11 +++++++++++ lib/librte_ether/rte_ethdev.h | 14 ++++++++++++++ 2 files changed, 25 insertions(+) diff --git a/lib/librte_ether/rte_ethdev.c b/lib/librte_ether/rte_ethdev.c index d70854f..b58bab3 100644 --- a/lib/librte_ether/rte_ethdev.c +++ b/lib/librte_ether/rte_ethdev.c @@ -260,6 +260,17 @@ rte_eth_dev_allocate(const char *name) return eth_dev; } +int +rte_eth_dev_free(struct rte_eth_dev *eth_dev) +{ + if (eth_dev == NULL) + return -EINVAL; + + eth_dev->attached = 0; + nb_ports--; + return 0; +} + static int rte_eth_dev_init(struct rte_pci_driver *pci_drv, struct rte_pci_device *pci_dev) diff --git a/lib/librte_ether/rte_ethdev.h b/lib/librte_ether/rte_ethdev.h index ca101f5..fbe7ac1 100644 --- a/lib/librte_ether/rte_ethdev.h +++ b/lib/librte_ether/rte_ethdev.h @@ -1627,6 +1627,20 @@ extern uint8_t rte_eth_dev_count(void); */ struct rte_eth_dev *rte_eth_dev_allocate(const char *name); +/** + * Function for internal use by dummy drivers primarily, e.g. ring-based + * driver. + * Free the specified ethdev. + * + * @param eth_dev + * The *eth_dev* pointer is the address of the *rte_eth_dev* structure + * associated with the matching device and which have been [automatically] + * allocated in the *rte_eth_devices* array. + * @return + * - 0 on success, negative on error + */ +int rte_eth_dev_free(struct rte_eth_dev *eth_dev); + struct eth_driver; /** * @internal -- 1.9.1
The patch adds function pointer to rte_pci_driver and eth_driver structure. These function pointers are used when ports are detached. Also the patch adds rte_eth_dev_uninit(). So far, it's not called by anywhere, but it will be called when port hotplug function is implemented. v6: - Fix rte_eth_dev_uninit() to handle a return value of uninit function of PMD. v4: - Add paramerter checking. - Change function names. Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp> --- lib/librte_eal/common/include/rte_pci.h | 7 +++++ lib/librte_ether/rte_ethdev.c | 47 +++++++++++++++++++++++++++++++++ lib/librte_ether/rte_ethdev.h | 24 +++++++++++++++++ 3 files changed, 78 insertions(+) diff --git a/lib/librte_eal/common/include/rte_pci.h b/lib/librte_eal/common/include/rte_pci.h index 4814cd7..87ca4cf 100644 --- a/lib/librte_eal/common/include/rte_pci.h +++ b/lib/librte_eal/common/include/rte_pci.h @@ -189,12 +189,19 @@ struct rte_pci_driver; typedef int (pci_devinit_t)(struct rte_pci_driver *, struct rte_pci_device *); /** + * Uninitialisation function for the driver called during hotplugging. + */ +typedef int (pci_devuninit_t)( + struct rte_pci_driver *, struct rte_pci_device *); + +/** * A structure describing a PCI driver. */ struct rte_pci_driver { TAILQ_ENTRY(rte_pci_driver) next; /**< Next in list. */ const char *name; /**< Driver name. */ pci_devinit_t *devinit; /**< Device init. function. */ + pci_devuninit_t *devuninit; /**< Device uninit function. */ struct rte_pci_id *id_table; /**< ID table, NULL terminated. */ uint32_t drv_flags; /**< Flags contolling handling of device. */ }; diff --git a/lib/librte_ether/rte_ethdev.c b/lib/librte_ether/rte_ethdev.c index b58bab3..7bed901 100644 --- a/lib/librte_ether/rte_ethdev.c +++ b/lib/librte_ether/rte_ethdev.c @@ -326,6 +326,52 @@ rte_eth_dev_init(struct rte_pci_driver *pci_drv, return diag; } +static int +rte_eth_dev_uninit(struct rte_pci_driver *pci_drv, + struct rte_pci_device *pci_dev) +{ + struct eth_driver *eth_drv; + struct rte_eth_dev *eth_dev; + char ethdev_name[RTE_ETH_NAME_MAX_LEN]; + int ret; + + if ((pci_drv == NULL) || (pci_dev == NULL)) + return -EINVAL; + + /* Create unique Ethernet device name using PCI address */ + snprintf(ethdev_name, RTE_ETH_NAME_MAX_LEN, "%d:%d.%d", + pci_dev->addr.bus, pci_dev->addr.devid, + pci_dev->addr.function); + + eth_dev = rte_eth_dev_allocated(ethdev_name); + if (eth_dev == NULL) + return -ENODEV; + + eth_drv = (struct eth_driver *)pci_drv; + + /* Invoke PMD device uninit function */ + if (*eth_drv->eth_dev_uninit) { + ret = (*eth_drv->eth_dev_uninit)(eth_drv, eth_dev); + if (ret) + return ret; + } + + /* free ether device */ + rte_eth_dev_free(eth_dev); + + /* init user callbacks */ + TAILQ_INIT(&(eth_dev->callbacks)); + + if (rte_eal_process_type() == RTE_PROC_PRIMARY) + rte_free(eth_dev->data->dev_private); + + eth_dev->pci_dev = NULL; + eth_dev->driver = NULL; + eth_dev->data = NULL; + + return 0; +} + /** * Register an Ethernet [Poll Mode] driver. * @@ -344,6 +390,7 @@ void rte_eth_driver_register(struct eth_driver *eth_drv) { eth_drv->pci_drv.devinit = rte_eth_dev_init; + eth_drv->pci_drv.devuninit = rte_eth_dev_uninit; rte_eal_pci_register(ð_drv->pci_drv); } diff --git a/lib/librte_ether/rte_ethdev.h b/lib/librte_ether/rte_ethdev.h index fbe7ac1..91d9e86 100644 --- a/lib/librte_ether/rte_ethdev.h +++ b/lib/librte_ether/rte_ethdev.h @@ -1678,6 +1678,27 @@ typedef int (*eth_dev_init_t)(struct eth_driver *eth_drv, /** * @internal + * Finalization function of an Ethernet driver invoked for each matching + * Ethernet PCI device detected during the PCI closing phase. + * + * @param eth_drv + * The pointer to the [matching] Ethernet driver structure supplied by + * the PMD when it registered itself. + * @param eth_dev + * The *eth_dev* pointer is the address of the *rte_eth_dev* structure + * associated with the matching device and which have been [automatically] + * allocated in the *rte_eth_devices* array. + * @return + * - 0: Success, the device is properly finalized by the driver. + * In particular, the driver MUST free the *dev_ops* pointer + * of the *eth_dev* structure. + * - <0: Error code of the device initialization failure. + */ +typedef int (*eth_dev_uninit_t)(struct eth_driver *eth_drv, + struct rte_eth_dev *eth_dev); + +/** + * @internal * The structure associated with a PMD Ethernet driver. * * Each Ethernet driver acts as a PCI driver and is represented by a generic @@ -1687,11 +1708,14 @@ typedef int (*eth_dev_init_t)(struct eth_driver *eth_drv, * * - The *eth_dev_init* function invoked for each matching PCI device. * + * - The *eth_dev_uninit* function invoked for each matching PCI device. + * * - The size of the private data to allocate for each matching device. */ struct eth_driver { struct rte_pci_driver pci_drv; /**< The PMD is also a PCI driver. */ eth_dev_init_t eth_dev_init; /**< Device init function. */ + eth_dev_uninit_t eth_dev_uninit; /**< Device uninit function. */ unsigned int dev_private_size; /**< Size of device private data. */ }; -- 1.9.1
The patch adds following functions. - rte_eth_dev_save() The function is used for saving current rte_eth_dev structures. - rte_eth_dev_get_changed_port() The function receives the rte_eth_dev structures, then compare these with current values to know which port is actually attached or detached. - rte_eth_dev_get_addr_by_port() The function returns a pci address of a ethdev specified by port identifier. - rte_eth_dev_get_port_by_addr() The function returns a port identifier of a ethdev specified by pci address. - rte_eth_dev_get_name_by_port() The function returns a unique identifier name of a ethdev specified by port identifier. - Add rte_eth_dev_check_detachable() The function returns whether a PMD supports detach function. Also the patch changes scope of rte_eth_dev_allocated() to global. This function will be called by virtual PMDs to support port hotplug. So change scope of the function to global. v5: - Fix return value of below functions. rte_eth_dev_get_changed_port(). rte_eth_dev_get_port_by_addr(). v4: - Add paramerter checking. v3: - Fix if-condition bug while comparing pci addresses. - Add error checking codes. Reported-by: Mark Enright <menrigh@brocade.com> Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp> --- lib/librte_ether/rte_ethdev.c | 98 ++++++++++++++++++++++++++++++++++++++++++- lib/librte_ether/rte_ethdev.h | 80 +++++++++++++++++++++++++++++++++++ 2 files changed, 177 insertions(+), 1 deletion(-) diff --git a/lib/librte_ether/rte_ethdev.c b/lib/librte_ether/rte_ethdev.c index 7bed901..5aded10 100644 --- a/lib/librte_ether/rte_ethdev.c +++ b/lib/librte_ether/rte_ethdev.c @@ -206,7 +206,7 @@ rte_eth_dev_data_alloc(void) RTE_MAX_ETHPORTS * sizeof(*rte_eth_dev_data)); } -static struct rte_eth_dev * +struct rte_eth_dev * rte_eth_dev_allocated(const char *name) { unsigned i; @@ -426,6 +426,102 @@ rte_eth_dev_count(void) return (nb_ports); } +void +rte_eth_dev_save(struct rte_eth_dev *devs) +{ + if (devs == NULL) + return; + + /* save current rte_eth_devices */ + memcpy(devs, rte_eth_devices, + sizeof(struct rte_eth_dev) * RTE_MAX_ETHPORTS); +} + +int +rte_eth_dev_get_changed_port(struct rte_eth_dev *devs, uint8_t *port_id) +{ + if ((devs == NULL) || (port_id == NULL)) + return -EINVAL; + + /* check which port was attached or detached */ + for (*port_id = 0; *port_id < RTE_MAX_ETHPORTS; (*port_id)++, devs++) { + if (rte_eth_devices[*port_id].attached ^ devs->attached) + return 0; + } + return -ENODEV; +} + +int +rte_eth_dev_get_addr_by_port(uint8_t port_id, struct rte_pci_addr *addr) +{ + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) + return -EINVAL; + + if (addr == NULL) { + PMD_DEBUG_TRACE("Null pointer is specified\n"); + return -EINVAL; + } + + *addr = rte_eth_devices[port_id].pci_dev->addr; + return 0; +} + +int +rte_eth_dev_get_port_by_addr(struct rte_pci_addr *addr, uint8_t *port_id) +{ + struct rte_pci_addr *tmp; + + if ((addr == NULL) || (port_id == NULL)) { + PMD_DEBUG_TRACE("Null pointer is specified\n"); + return -EINVAL; + } + + for (*port_id = 0; *port_id < RTE_MAX_ETHPORTS; (*port_id)++) { + if (!rte_eth_devices[*port_id].attached) + continue; + if (!rte_eth_devices[*port_id].pci_dev) + continue; + tmp = &rte_eth_devices[*port_id].pci_dev->addr; + if (eal_compare_pci_addr(tmp, addr) == 0) + return 0; + } + return -ENODEV; +} + +int +rte_eth_dev_get_name_by_port(uint8_t port_id, char *name) +{ + char *tmp; + + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) + return -EINVAL; + + if (name == NULL) { + PMD_DEBUG_TRACE("Null pointer is specified\n"); + return -EINVAL; + } + + /* shouldn't check 'rte_eth_devices[i].data', + * because it might be overwritten by VDEV PMD */ + tmp = rte_eth_dev_data[port_id].name; + strncpy(name, tmp, strlen(tmp) + 1); + return 0; +} + +int +rte_eth_dev_check_detachable(uint8_t port_id) +{ + uint32_t drv_flags; + + if (port_id >= RTE_MAX_ETHPORTS) { + PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + return -EINVAL; + } + + drv_flags = rte_eth_devices[port_id].driver->pci_drv.drv_flags; + return !(drv_flags & RTE_PCI_DRV_DETACHABLE); +} + static int rte_eth_dev_rx_queue_config(struct rte_eth_dev *dev, uint16_t nb_queues) { diff --git a/lib/librte_ether/rte_ethdev.h b/lib/librte_ether/rte_ethdev.h index 91d9e86..9919968 100644 --- a/lib/librte_ether/rte_ethdev.h +++ b/lib/librte_ether/rte_ethdev.h @@ -1616,6 +1616,86 @@ extern struct rte_eth_dev rte_eth_devices[]; extern uint8_t rte_eth_dev_count(void); /** + * Function for internal use by port hotplug functions. + * Copies current ethdev structures to the specified pointer. + * + * @param devs The pointer to the ethdev structures + */ +extern void rte_eth_dev_save(struct rte_eth_dev *devs); + +/** + * Function for internal use by port hotplug functions. + * Compare the specified ethdev structures with currents. Then + * if there is a port which status is changed, fill the specified pointer + * with the port id of that port. + * @param devs The pointer to the ethdev structures + * @param port_id The pointer to the port id + * @return + * - 0 on success, negative on error + */ +extern int rte_eth_dev_get_changed_port( + struct rte_eth_dev *devs, uint8_t *port_id); + +/** + * Function for internal use by port hotplug functions. + * Returns a pci address of a ethdev specified by port identifier. + * @param port_id + * The port identifier of the Ethernet device + * @param addr + * The pointer to the pci address + * @return + * - 0 on success, negative on error + */ +extern int rte_eth_dev_get_addr_by_port( + uint8_t port_id, struct rte_pci_addr *addr); + +/** + * Function for internal use by port hotplug functions. + * Returns a port identifier of a ethdev specified by pci address. + * @param addr + * The pointer to the pci address of the Ethernet device. + * @param port_id + * The pointer to the port identifier + * @return + * - 0 on success, negative on error + */ +extern int rte_eth_dev_get_port_by_addr( + struct rte_pci_addr *addr, uint8_t *port_id); + +/** + * Function for internal use by port hotplug functions. + * Returns a unique identifier name of a ethdev specified by port identifier. + * @param port_id + * The port identifier. + * @param name + * The pointer to the Unique identifier name for each Ethernet device + * @return + * - 0 on success, negative on error + */ +extern int rte_eth_dev_get_name_by_port(uint8_t port_id, char *name); + +/** + * Function for internal use by port hotplug functions. + * Check whether or not, a PMD that is handling the ethdev specified by port + * identifier can support detach function. + * @param port_id + * The port identifier + * @return + * - 0 on supporting detach function, negative on not supporting + */ +extern int rte_eth_dev_check_detachable(uint8_t port_id); + +/** + * Function for internal use by port hotplug functions. + * Returns a ethdev slot specified by the unique identifier name. + * @param name + * The pointer to the Unique identifier name for each Ethernet device + * @return + * - The pointer to the ethdev slot, on success. NULL on error + */ +extern struct rte_eth_dev *rte_eth_dev_allocated(const char *name); + +/** * Function for internal use by dummy drivers primarily, e.g. ring-based * driver. * Allocates a new ethdev slot for an ethernet device and returns the pointer -- 1.9.1
The patch adds functions for unmapping igb_uio resources. The patch is only for Linux and igb_uio environment. VFIO and BSD are not supported. v5: - Fix pci_unmap_device() to check pt_driver. v4: - Add paramerter checking. - Add header file to determine if hotplug can be enabled. Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp> --- lib/librte_eal/common/Makefile | 1 + lib/librte_eal/common/include/rte_dev_hotplug.h | 44 +++++++++++++++++ lib/librte_eal/linuxapp/eal/eal_pci.c | 44 +++++++++++++++++ lib/librte_eal/linuxapp/eal/eal_pci_init.h | 8 +++ lib/librte_eal/linuxapp/eal/eal_pci_uio.c | 65 +++++++++++++++++++++++++ 5 files changed, 162 insertions(+) create mode 100644 lib/librte_eal/common/include/rte_dev_hotplug.h diff --git a/lib/librte_eal/common/Makefile b/lib/librte_eal/common/Makefile index 52c1a5f..db7cc93 100644 --- a/lib/librte_eal/common/Makefile +++ b/lib/librte_eal/common/Makefile @@ -41,6 +41,7 @@ INC += rte_eal_memconfig.h rte_malloc_heap.h INC += rte_hexdump.h rte_devargs.h rte_dev.h INC += rte_common_vect.h INC += rte_pci_dev_feature_defs.h rte_pci_dev_features.h +INC += rte_dev_hotplug.h ifeq ($(CONFIG_RTE_INSECURE_FUNCTION_WARNING),y) INC += rte_warnings.h diff --git a/lib/librte_eal/common/include/rte_dev_hotplug.h b/lib/librte_eal/common/include/rte_dev_hotplug.h new file mode 100644 index 0000000..b333e0f --- /dev/null +++ b/lib/librte_eal/common/include/rte_dev_hotplug.h @@ -0,0 +1,44 @@ +/*- + * BSD LICENSE + * + * Copyright(c) 2015 IGEL Co.,LTd. + * All rights reserved. + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * + * * Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * * Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in + * the documentation and/or other materials provided with the + * distribution. + * * Neither the name of IGEL Co.,Ltd. nor the names of its + * contributors may be used to endorse or promote products derived + * from this software without specific prior written permission. + * + * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS + * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT + * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR + * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT + * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, + * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT + * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, + * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY + * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT + * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE + * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + */ + +#ifndef _RTE_DEV_HOTPLUG_H_ +#define _RTE_DEV_HOTPLUG_H_ + +/* + * determine if hotplug can be enabled on the system + */ +#if defined(RTE_LIBRTE_EAL_HOTPLUG) && defined(RTE_LIBRTE_EAL_LINUXAPP) +#define ENABLE_HOTPLUG +#endif /* RTE_LIBRTE_EAL_HOTPLUG & RTE_LIBRTE_EAL_LINUXAPP */ + +#endif /* _RTE_DEV_HOTPLUG_H_ */ diff --git a/lib/librte_eal/linuxapp/eal/eal_pci.c b/lib/librte_eal/linuxapp/eal/eal_pci.c index d847102..c3b7917 100644 --- a/lib/librte_eal/linuxapp/eal/eal_pci.c +++ b/lib/librte_eal/linuxapp/eal/eal_pci.c @@ -166,6 +166,25 @@ pci_map_resource(void *requested_addr, int fd, off_t offset, size_t size) return mapaddr; } +#ifdef ENABLE_HOTPLUG +/* unmap a particular resource */ +void +pci_unmap_resource(void *requested_addr, size_t size) +{ + if (requested_addr == NULL) + return; + + /* Unmap the PCI memory resource of device */ + if (munmap(requested_addr, size)) { + RTE_LOG(ERR, EAL, "%s(): cannot munmap(%p, 0x%lx): %s\n", + __func__, requested_addr, (unsigned long)size, + strerror(errno)); + } else + RTE_LOG(DEBUG, EAL, " PCI memory mapped at %p\n", + requested_addr); +} +#endif /* ENABLE_HOTPLUG */ + /* parse the "resource" sysfs file */ #define IORESOURCE_MEM 0x00000200 @@ -567,6 +586,31 @@ pci_map_device(struct rte_pci_device *dev) return ret; } +#ifdef ENABLE_HOTPLUG +static void +pci_unmap_device(struct rte_pci_device *dev) +{ + if (dev == NULL) + return; + + /* try unmapping the NIC resources using VFIO if it exists */ + switch (dev->pt_driver) { + case RTE_PT_VFIO: + RTE_LOG(ERR, EAL, "Hotplug doesn't support vfio yet\n"); + break; + case RTE_PT_IGB_UIO: + case RTE_PT_UIO_GENERIC: + /* unmap resources for devices that use uio */ + pci_uio_unmap_resource(dev); + break; + default: + RTE_LOG(DEBUG, EAL, " Not managed by known pt driver," + " skipped\n"); + break; + } +} +#endif /* ENABLE_HOTPLUG */ + /* * If vendor/device ID match, call the devinit() function of the * driver. diff --git a/lib/librte_eal/linuxapp/eal/eal_pci_init.h b/lib/librte_eal/linuxapp/eal/eal_pci_init.h index 1070eb8..5152a0b 100644 --- a/lib/librte_eal/linuxapp/eal/eal_pci_init.h +++ b/lib/librte_eal/linuxapp/eal/eal_pci_init.h @@ -34,6 +34,7 @@ #ifndef EAL_PCI_INIT_H_ #define EAL_PCI_INIT_H_ +#include <rte_dev_hotplug.h> #include "eal_vfio.h" struct pci_map { @@ -71,6 +72,13 @@ void *pci_map_resource(void *requested_addr, int fd, off_t offset, /* map IGB_UIO resource prototype */ int pci_uio_map_resource(struct rte_pci_device *dev); +#ifdef ENABLE_HOTPLUG +void pci_unmap_resource(void *requested_addr, size_t size); + +/* unmap IGB_UIO resource prototype */ +void pci_uio_unmap_resource(struct rte_pci_device *dev); +#endif /* ENABLE_HOTPLUG */ + #ifdef VFIO_PRESENT #define VFIO_MAX_GROUPS 64 diff --git a/lib/librte_eal/linuxapp/eal/eal_pci_uio.c b/lib/librte_eal/linuxapp/eal/eal_pci_uio.c index 1da3507..81830d1 100644 --- a/lib/librte_eal/linuxapp/eal/eal_pci_uio.c +++ b/lib/librte_eal/linuxapp/eal/eal_pci_uio.c @@ -404,6 +404,71 @@ pci_uio_map_resource(struct rte_pci_device *dev) return 0; } +#ifdef ENABLE_HOTPLUG +static void +pci_uio_unmap(struct mapped_pci_resource *uio_res) +{ + int i; + + if (uio_res == NULL) + return; + + for (i = 0; i != uio_res->nb_maps; i++) + pci_unmap_resource(uio_res->maps[i].addr, + (size_t)uio_res->maps[i].size); +} + +static struct mapped_pci_resource * +pci_uio_find_resource(struct rte_pci_device *dev) +{ + struct mapped_pci_resource *uio_res; + + if (dev == NULL) + return NULL; + + TAILQ_FOREACH(uio_res, pci_res_list, next) { + + /* skip this element if it doesn't match our PCI address */ + if (!eal_compare_pci_addr(&uio_res->pci_addr, &dev->addr)) + return uio_res; + } + return NULL; +} + +/* unmap the PCI resource of a PCI device in virtual memory */ +void +pci_uio_unmap_resource(struct rte_pci_device *dev) +{ + struct mapped_pci_resource *uio_res; + + if (dev == NULL) + return; + + /* find an entry for the device */ + uio_res = pci_uio_find_resource(dev); + if (uio_res == NULL) + return; + + /* secondary processes - just free maps */ + if (rte_eal_process_type() != RTE_PROC_PRIMARY) + return pci_uio_unmap(uio_res); + + TAILQ_REMOVE(pci_res_list, uio_res, next); + + /* unmap all resources */ + pci_uio_unmap(uio_res); + + /* free uio resource */ + rte_free(uio_res); + + /* close fd if in primary process */ + close(dev->intr_handle.fd); + + dev->intr_handle.fd = -1; + dev->intr_handle.type = RTE_INTR_HANDLE_UNKNOWN; +} +#endif /* ENABLE_HOTPLUG */ + /* * parse a sysfs file containing one integer value * different to the eal version, as it needs to work with 64-bit values -- 1.9.1
The function removes the specified devargs entry from devargs_list. Also the patch adds sanity checking to rte_eal_devargs_add(). v5: - Change function definition of rte_eal_devargs_remove(). v4: - Fix sanity check code. Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp> --- lib/librte_eal/common/eal_common_devargs.c | 60 +++++++++++++++++++++++++++++ lib/librte_eal/common/include/rte_devargs.h | 21 ++++++++++ 2 files changed, 81 insertions(+) diff --git a/lib/librte_eal/common/eal_common_devargs.c b/lib/librte_eal/common/eal_common_devargs.c index 4c7d11a..5b1ac8e 100644 --- a/lib/librte_eal/common/eal_common_devargs.c +++ b/lib/librte_eal/common/eal_common_devargs.c @@ -44,6 +44,35 @@ struct rte_devargs_list devargs_list = TAILQ_HEAD_INITIALIZER(devargs_list); + +/* find a entry specified by pci address or device name */ +static struct rte_devargs * +rte_eal_devargs_find(enum rte_devtype devtype, void *args) +{ + struct rte_devargs *devargs; + + if (args == NULL) + return NULL; + + TAILQ_FOREACH(devargs, &devargs_list, next) { + switch (devtype) { + case RTE_DEVTYPE_WHITELISTED_PCI: + case RTE_DEVTYPE_BLACKLISTED_PCI: + if (eal_compare_pci_addr(&devargs->pci.addr, args) == 0) + goto found; + break; + case RTE_DEVTYPE_VIRTUAL: + if (memcmp(&devargs->virtual.drv_name, args, + strlen((char *)args)) == 0) + goto found; + break; + } + } + return NULL; +found: + return devargs; +} + /* store a whitelist parameter for later parsing */ int rte_eal_devargs_add(enum rte_devtype devtype, const char *devargs_str) @@ -87,6 +116,12 @@ rte_eal_devargs_add(enum rte_devtype devtype, const char *devargs_str) free(devargs); return -1; } + /* make sure there is no same entry */ + if (rte_eal_devargs_find(devtype, &devargs->pci.addr)) { + RTE_LOG(ERR, EAL, + "device already registered: <%s>\n", buf); + return -1; + } break; case RTE_DEVTYPE_VIRTUAL: /* save driver name */ @@ -98,6 +133,12 @@ rte_eal_devargs_add(enum rte_devtype devtype, const char *devargs_str) free(devargs); return -1; } + /* make sure there is no same entry */ + if (rte_eal_devargs_find(devtype, &devargs->virtual.drv_name)) { + RTE_LOG(ERR, EAL, + "device already registered: <%s>\n", buf); + return -1; + } break; } @@ -105,6 +146,25 @@ rte_eal_devargs_add(enum rte_devtype devtype, const char *devargs_str) return 0; } +/* remove it from the devargs_list */ +int +rte_eal_devargs_remove(enum rte_devtype devtype, void *args) +{ + struct rte_devargs *devargs; + + if (args == NULL) + return -EINVAL; + + devargs = rte_eal_devargs_find(devtype, args); + if (devargs == NULL) { + RTE_LOG(ERR, EAL, "device not found\n"); + return -ENODEV; + } + + TAILQ_REMOVE(&devargs_list, devargs, next); + return 0; +} + /* count the number of devices of a specified type */ unsigned int rte_eal_devargs_type_count(enum rte_devtype devtype) diff --git a/lib/librte_eal/common/include/rte_devargs.h b/lib/librte_eal/common/include/rte_devargs.h index 9f9c98f..b5ad4b3 100644 --- a/lib/librte_eal/common/include/rte_devargs.h +++ b/lib/librte_eal/common/include/rte_devargs.h @@ -123,6 +123,27 @@ extern struct rte_devargs_list devargs_list; int rte_eal_devargs_add(enum rte_devtype devtype, const char *devargs_str); /** + * Remove a device from the user device list + * + * For PCI devices, the format of arguments string is "PCI_ADDR". It shouldn't + * involves parameters for the device. Example: "08:00.1". + * + * For virtual devices, the format of arguments string is "DRIVER_NAME*". It + * shouldn't involves parameters for the device. Example: "eth_ring". The + * validity of the driver name is not checked by this function, it is done + * when closing the drivers. + * + * @param devtype + * The type of the device. + * @param name + * The name of the device. + * + * @return + * - 0 on success, negative on error + */ +int rte_eal_devargs_remove(enum rte_devtype devtype, void *args); + +/** * Count the number of user devices of a specified type * * @param devtype -- 1.9.1
- Add rte_eal_pci_close_one_dirver() The function is used for closing the specified driver and device. - Add pci_invoke_all_drivers() The function is based on pci_probe_all_drivers. But it can not only probe but also close drivers. - Add pci_close_all_drivers() The function tries to find a driver for the specified device, and then close the driver. - Add rte_eal_pci_probe_one() and rte_eal_pci_close_one() The functions are used for probe and close a device. First the function tries to find a device that has the specfied PCI address. Then, probe or close the device. v5: - Remove RTE_EAL_INVOKE_TYPE_UNKNOWN, because it's unused. v4: - Fix paramerter checking. - Fix indent of 'if' statement. Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp> --- lib/librte_eal/common/eal_common_pci.c | 90 +++++++++++++++++++++++++++++---- lib/librte_eal/common/eal_private.h | 24 +++++++++ lib/librte_eal/common/include/rte_pci.h | 33 ++++++++++++ lib/librte_eal/linuxapp/eal/eal_pci.c | 69 +++++++++++++++++++++++++ 4 files changed, 206 insertions(+), 10 deletions(-) diff --git a/lib/librte_eal/common/eal_common_pci.c b/lib/librte_eal/common/eal_common_pci.c index a89f5c3..7c9b8c5 100644 --- a/lib/librte_eal/common/eal_common_pci.c +++ b/lib/librte_eal/common/eal_common_pci.c @@ -99,19 +99,27 @@ static struct rte_devargs *pci_devargs_lookup(struct rte_pci_device *dev) return NULL; } -/* - * If vendor/device ID match, call the devinit() function of all - * registered driver for the given device. Return -1 if initialization - * failed, return 1 if no driver is found for this device. - */ static int -pci_probe_all_drivers(struct rte_pci_device *dev) +pci_invoke_all_drivers(struct rte_pci_device *dev, + enum rte_eal_invoke_type type) { struct rte_pci_driver *dr = NULL; - int rc; + int rc = 0; + + if ((dev == NULL) || (type >= RTE_EAL_INVOKE_TYPE_MAX)) + return -1; TAILQ_FOREACH(dr, &pci_driver_list, next) { - rc = rte_eal_pci_probe_one_driver(dr, dev); + switch (type) { + case RTE_EAL_INVOKE_TYPE_PROBE: + rc = rte_eal_pci_probe_one_driver(dr, dev); + break; + case RTE_EAL_INVOKE_TYPE_CLOSE: + rc = rte_eal_pci_close_one_driver(dr, dev); + break; + default: + return -1; + } if (rc < 0) /* negative value is an error */ return -1; @@ -123,6 +131,66 @@ pci_probe_all_drivers(struct rte_pci_device *dev) return 1; } +#ifdef ENABLE_HOTPLUG +static int +rte_eal_pci_invoke_one(struct rte_pci_addr *addr, + enum rte_eal_invoke_type type) +{ + struct rte_pci_device *dev = NULL; + int ret = 0; + + if ((addr == NULL) || (type >= RTE_EAL_INVOKE_TYPE_MAX)) + return -1; + + TAILQ_FOREACH(dev, &pci_device_list, next) { + if (eal_compare_pci_addr(&dev->addr, addr)) + continue; + + ret = pci_invoke_all_drivers(dev, type); + if (ret < 0) + goto invoke_err_return; + + if (type == RTE_EAL_INVOKE_TYPE_CLOSE) + goto remove_dev; + + return 0; + } + + return -1; + +invoke_err_return: + RTE_LOG(WARNING, EAL, "Requested device " PCI_PRI_FMT + " cannot be used\n", dev->addr.domain, dev->addr.bus, + dev->addr.devid, dev->addr.function); + return -1; + +remove_dev: + TAILQ_REMOVE(&pci_device_list, dev, next); + return 0; +} + + +/* + * Find the pci device specified by pci address, then invoke probe function of + * the driver of the devive. + */ +int +rte_eal_pci_probe_one(struct rte_pci_addr *addr) +{ + return rte_eal_pci_invoke_one(addr, RTE_EAL_INVOKE_TYPE_PROBE); +} + +/* + * Find the pci device specified by pci address, then invoke close function of + * the driver of the devive. + */ +int +rte_eal_pci_close_one(struct rte_pci_addr *addr) +{ + return rte_eal_pci_invoke_one(addr, RTE_EAL_INVOKE_TYPE_CLOSE); +} +#endif /* ENABLE_HOTPLUG */ + /* * Scan the content of the PCI bus, and call the devinit() function for * all registered drivers that have a matching entry in its id_table @@ -148,10 +216,12 @@ rte_eal_pci_probe(void) /* probe all or only whitelisted devices */ if (probe_all) - ret = pci_probe_all_drivers(dev); + ret = pci_invoke_all_drivers(dev, + RTE_EAL_INVOKE_TYPE_PROBE); else if (devargs != NULL && devargs->type == RTE_DEVTYPE_WHITELISTED_PCI) - ret = pci_probe_all_drivers(dev); + ret = pci_invoke_all_drivers(dev, + RTE_EAL_INVOKE_TYPE_PROBE); if (ret < 0) rte_exit(EXIT_FAILURE, "Requested device " PCI_PRI_FMT " cannot be used\n", dev->addr.domain, dev->addr.bus, diff --git a/lib/librte_eal/common/eal_private.h b/lib/librte_eal/common/eal_private.h index 159cd66..1a362ab 100644 --- a/lib/librte_eal/common/eal_private.h +++ b/lib/librte_eal/common/eal_private.h @@ -154,6 +154,15 @@ struct rte_pci_driver; struct rte_pci_device; /** + * The invoke type. + */ +enum rte_eal_invoke_type { + RTE_EAL_INVOKE_TYPE_PROBE, /**< invoke probe function */ + RTE_EAL_INVOKE_TYPE_CLOSE, /**< invoke close function */ + RTE_EAL_INVOKE_TYPE_MAX /**< max value of this enum */ +}; + +/** * Mmap memory for single PCI device * * This function is private to EAL. @@ -165,6 +174,21 @@ int rte_eal_pci_probe_one_driver(struct rte_pci_driver *dr, struct rte_pci_device *dev); /** + * Munmap memory for single PCI device + * + * This function is private to EAL. + * + * @param dr + * The pointer to the pci driver structure + * @param dev + * The pointer to the pci device structure + * @return + * 0 on success, negative on error + */ +int rte_eal_pci_close_one_driver(struct rte_pci_driver *dr, + struct rte_pci_device *dev); + +/** * Init tail queues for non-EAL library structures. This is to allow * the rings, mempools, etc. lists to be shared among multiple processes * diff --git a/lib/librte_eal/common/include/rte_pci.h b/lib/librte_eal/common/include/rte_pci.h index 87ca4cf..a111066 100644 --- a/lib/librte_eal/common/include/rte_pci.h +++ b/lib/librte_eal/common/include/rte_pci.h @@ -82,6 +82,7 @@ extern "C" { #include <inttypes.h> #include <rte_interrupts.h> +#include <rte_dev_hotplug.h> TAILQ_HEAD(pci_device_list, rte_pci_device); /**< PCI devices in D-linked Q. */ TAILQ_HEAD(pci_driver_list, rte_pci_driver); /**< PCI drivers in D-linked Q. */ @@ -323,6 +324,38 @@ eal_compare_pci_addr(struct rte_pci_addr *addr, struct rte_pci_addr *addr2) */ int rte_eal_pci_probe(void); +#ifdef ENABLE_HOTPLUG +/** + * Probe the single PCI device. + * + * Scan the content of the PCI bus, and find the pci device specified by pci + * address, then call the probe() function for registered driver that has a + * matching entry in its id_table for discovered device. + * + * @param addr + * The PCI Bus-Device-Function address to probe or close. + * @return + * - 0 on success. + * - Negative on error. + */ +int rte_eal_pci_probe_one(struct rte_pci_addr *addr); + +/** + * Close the single PCI device. + * + * Scan the content of the PCI bus, and find the pci device specified by pci + * address, then call the close() function for registered driver that has a + * matching entry in its id_table for discovered device. + * + * @param addr + * The PCI Bus-Device-Function address to probe or close. + * @return + * - 0 on success. + * - Negative on error. + */ +int rte_eal_pci_close_one(struct rte_pci_addr *addr); +#endif /* ENABLE_HOTPLUG */ + /** * Dump the content of the PCI bus. * diff --git a/lib/librte_eal/linuxapp/eal/eal_pci.c b/lib/librte_eal/linuxapp/eal/eal_pci.c index c3b7917..831422e 100644 --- a/lib/librte_eal/linuxapp/eal/eal_pci.c +++ b/lib/librte_eal/linuxapp/eal/eal_pci.c @@ -682,6 +682,75 @@ rte_eal_pci_probe_one_driver(struct rte_pci_driver *dr, struct rte_pci_device *d return 1; } +#ifdef ENABLE_HOTPLUG +/* + * If vendor/device ID match, call the devuninit() function of the + * driver. + */ +int +rte_eal_pci_close_one_driver(struct rte_pci_driver *dr, + struct rte_pci_device *dev) +{ + struct rte_pci_id *id_table; + + if ((dr == NULL) || (dev == NULL)) + return -EINVAL; + + for (id_table = dr->id_table ; id_table->vendor_id != 0; id_table++) { + + /* check if device's identifiers match the driver's ones */ + if (id_table->vendor_id != dev->id.vendor_id && + id_table->vendor_id != PCI_ANY_ID) + continue; + if (id_table->device_id != dev->id.device_id && + id_table->device_id != PCI_ANY_ID) + continue; + if (id_table->subsystem_vendor_id != + dev->id.subsystem_vendor_id && + id_table->subsystem_vendor_id != PCI_ANY_ID) + continue; + if (id_table->subsystem_device_id != + dev->id.subsystem_device_id && + id_table->subsystem_device_id != PCI_ANY_ID) + continue; + + struct rte_pci_addr *loc = &dev->addr; + + RTE_LOG(DEBUG, EAL, + "PCI device "PCI_PRI_FMT" on NUMA socket %i\n", + loc->domain, loc->bus, loc->devid, + loc->function, dev->numa_node); + + RTE_LOG(DEBUG, EAL, " remove driver: %x:%x %s\n", + dev->id.vendor_id, dev->id.device_id, + dr->name); + + /* call the driver devuninit() function */ + if (dr->devuninit && (dr->devuninit(dr, dev) < 0)) + return -1; /* negative value is an error */ + + /* clear driver structure */ + dev->driver = NULL; + + if (dr->drv_flags & RTE_PCI_DRV_NEED_MAPPING) + /* unmap resources for devices that use igb_uio */ + pci_unmap_device(dev); + + return 0; + } + /* return positive value if driver is not found */ + return 1; +} +#else /* ENABLE_HOTPLUG */ +int +rte_eal_pci_close_one_driver(struct rte_pci_driver *dr __rte_unused, + struct rte_pci_device *dev __rte_unused) +{ + RTE_LOG(ERR, EAL, "Hotplug support isn't enabled\n"); + return -1; +} +#endif /* ENABLE_HOTPLUG */ + /* Init the PCI EAL subsystem */ int rte_eal_pci_init(void) -- 1.9.1
This new parameter is needed to keep device type like physical or virtual. Port detaching processes are different between physical and virtual. This paramerter lets detaching function know a device type of the port. v4: - Fix comments of rte_eth_dev_type. Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp> --- app/test/virtual_pmd.c | 2 +- lib/librte_ether/rte_ethdev.c | 14 ++++++++++++-- lib/librte_ether/rte_ethdev.h | 25 ++++++++++++++++++++++++- lib/librte_pmd_af_packet/rte_eth_af_packet.c | 2 +- lib/librte_pmd_bond/rte_eth_bond_api.c | 2 +- lib/librte_pmd_pcap/rte_eth_pcap.c | 2 +- lib/librte_pmd_ring/rte_eth_ring.c | 2 +- lib/librte_pmd_xenvirt/rte_eth_xenvirt.c | 2 +- 8 files changed, 42 insertions(+), 9 deletions(-) diff --git a/app/test/virtual_pmd.c b/app/test/virtual_pmd.c index 9fac95d..8d3a5ff 100644 --- a/app/test/virtual_pmd.c +++ b/app/test/virtual_pmd.c @@ -556,7 +556,7 @@ virtual_ethdev_create(const char *name, struct ether_addr *mac_addr, goto err; /* reserve an ethdev entry */ - eth_dev = rte_eth_dev_allocate(name); + eth_dev = rte_eth_dev_allocate(name, RTE_ETH_DEV_PHYSICAL); if (eth_dev == NULL) goto err; diff --git a/lib/librte_ether/rte_ethdev.c b/lib/librte_ether/rte_ethdev.c index 5aded10..b54b9ab 100644 --- a/lib/librte_ether/rte_ethdev.c +++ b/lib/librte_ether/rte_ethdev.c @@ -232,7 +232,7 @@ rte_eth_dev_allocate_new_port(void) } struct rte_eth_dev * -rte_eth_dev_allocate(const char *name) +rte_eth_dev_allocate(const char *name, enum rte_eth_dev_type type) { uint8_t port_id; struct rte_eth_dev *eth_dev; @@ -256,6 +256,7 @@ rte_eth_dev_allocate(const char *name) snprintf(eth_dev->data->name, sizeof(eth_dev->data->name), "%s", name); eth_dev->data->port_id = port_id; eth_dev->attached = DEV_CONNECTED; + eth_dev->dev_type = type; nb_ports++; return eth_dev; } @@ -267,6 +268,7 @@ rte_eth_dev_free(struct rte_eth_dev *eth_dev) return -EINVAL; eth_dev->attached = 0; + eth_dev->dev_type = RTE_ETH_DEV_UNKNOWN; nb_ports--; return 0; } @@ -287,7 +289,7 @@ rte_eth_dev_init(struct rte_pci_driver *pci_drv, snprintf(ethdev_name, RTE_ETH_NAME_MAX_LEN, "%d:%d.%d", pci_dev->addr.bus, pci_dev->addr.devid, pci_dev->addr.function); - eth_dev = rte_eth_dev_allocate(ethdev_name); + eth_dev = rte_eth_dev_allocate(ethdev_name, RTE_ETH_DEV_PHYSICAL); if (eth_dev == NULL) return -ENOMEM; @@ -426,6 +428,14 @@ rte_eth_dev_count(void) return (nb_ports); } +enum rte_eth_dev_type +rte_eth_dev_get_device_type(uint8_t port_id) +{ + if (rte_eth_dev_validate_port(port_id, NONE_TRACE) == DEV_INVALID) + return -1; + return rte_eth_devices[port_id].dev_type; +} + void rte_eth_dev_save(struct rte_eth_dev *devs) { diff --git a/lib/librte_ether/rte_ethdev.h b/lib/librte_ether/rte_ethdev.h index 9919968..00a6218 100644 --- a/lib/librte_ether/rte_ethdev.h +++ b/lib/librte_ether/rte_ethdev.h @@ -1522,6 +1522,17 @@ struct eth_dev_ops { }; /** + * The eth device type + */ +enum rte_eth_dev_type { + RTE_ETH_DEV_UNKNOWN, /**< unknown device type */ + RTE_ETH_DEV_PHYSICAL, + /**< Physical function and Virtual function devices of NIC */ + RTE_ETH_DEV_VIRTUAL, /**< non hardware device */ + RTE_ETH_DEV_MAX /**< max value of this enum */ +}; + +/** * @internal * The generic data structure associated with each ethernet device. * @@ -1540,6 +1551,7 @@ struct rte_eth_dev { struct rte_pci_device *pci_dev; /**< PCI info. supplied by probing */ struct rte_eth_dev_cb_list callbacks; /**< User application callbacks */ uint8_t attached; /**< Flag indicating the port is attached */ + enum rte_eth_dev_type dev_type; /**< Flag indicating the device type */ }; struct rte_eth_dev_sriov { @@ -1617,6 +1629,15 @@ extern uint8_t rte_eth_dev_count(void); /** * Function for internal use by port hotplug functions. + * Get the device type to know whether the device is physical or virtual. + * @param port_id The pointer to the port id + * @return + * - Device type. + */ +extern enum rte_eth_dev_type rte_eth_dev_get_device_type(uint8_t port_id); + +/** + * Function for internal use by port hotplug functions. * Copies current ethdev structures to the specified pointer. * * @param devs The pointer to the ethdev structures @@ -1702,10 +1723,12 @@ extern struct rte_eth_dev *rte_eth_dev_allocated(const char *name); * to that slot for the driver to use. * * @param name Unique identifier name for each Ethernet device + * @param type Device type of this Ethernet device * @return * - Slot in the rte_dev_devices array for a new device; */ -struct rte_eth_dev *rte_eth_dev_allocate(const char *name); +struct rte_eth_dev *rte_eth_dev_allocate(const char *name, + enum rte_eth_dev_type type); /** * Function for internal use by dummy drivers primarily, e.g. ring-based diff --git a/lib/librte_pmd_af_packet/rte_eth_af_packet.c b/lib/librte_pmd_af_packet/rte_eth_af_packet.c index 1ffe1cd..80e9bdf 100644 --- a/lib/librte_pmd_af_packet/rte_eth_af_packet.c +++ b/lib/librte_pmd_af_packet/rte_eth_af_packet.c @@ -649,7 +649,7 @@ rte_pmd_init_internals(const char *name, } /* reserve an ethdev entry */ - *eth_dev = rte_eth_dev_allocate(name); + *eth_dev = rte_eth_dev_allocate(name, RTE_ETH_DEV_VIRTUAL); if (*eth_dev == NULL) goto error; diff --git a/lib/librte_pmd_bond/rte_eth_bond_api.c b/lib/librte_pmd_bond/rte_eth_bond_api.c index 4ab3267..7a6a5f7 100644 --- a/lib/librte_pmd_bond/rte_eth_bond_api.c +++ b/lib/librte_pmd_bond/rte_eth_bond_api.c @@ -235,7 +235,7 @@ rte_eth_bond_create(const char *name, uint8_t mode, uint8_t socket_id) } /* reserve an ethdev entry */ - eth_dev = rte_eth_dev_allocate(name); + eth_dev = rte_eth_dev_allocate(name, RTE_ETH_DEV_VIRTUAL); if (eth_dev == NULL) { RTE_BOND_LOG(ERR, "Unable to allocate rte_eth_dev"); goto err; diff --git a/lib/librte_pmd_pcap/rte_eth_pcap.c b/lib/librte_pmd_pcap/rte_eth_pcap.c index d299288..af7fae8 100644 --- a/lib/librte_pmd_pcap/rte_eth_pcap.c +++ b/lib/librte_pmd_pcap/rte_eth_pcap.c @@ -709,7 +709,7 @@ rte_pmd_init_internals(const char *name, const unsigned nb_rx_queues, goto error; /* reserve an ethdev entry */ - *eth_dev = rte_eth_dev_allocate(name); + *eth_dev = rte_eth_dev_allocate(name, RTE_ETH_DEV_VIRTUAL); if (*eth_dev == NULL) goto error; diff --git a/lib/librte_pmd_ring/rte_eth_ring.c b/lib/librte_pmd_ring/rte_eth_ring.c index f685f08..5876057 100644 --- a/lib/librte_pmd_ring/rte_eth_ring.c +++ b/lib/librte_pmd_ring/rte_eth_ring.c @@ -255,7 +255,7 @@ rte_eth_from_rings(const char *name, struct rte_ring *const rx_queues[], goto error; /* reserve an ethdev entry */ - eth_dev = rte_eth_dev_allocate(name); + eth_dev = rte_eth_dev_allocate(name, RTE_ETH_DEV_VIRTUAL); if (eth_dev == NULL) goto error; diff --git a/lib/librte_pmd_xenvirt/rte_eth_xenvirt.c b/lib/librte_pmd_xenvirt/rte_eth_xenvirt.c index 04e30c9..bc403d6 100644 --- a/lib/librte_pmd_xenvirt/rte_eth_xenvirt.c +++ b/lib/librte_pmd_xenvirt/rte_eth_xenvirt.c @@ -648,7 +648,7 @@ eth_dev_xenvirt_create(const char *name, const char *params, goto err; /* reserve an ethdev entry */ - eth_dev = rte_eth_dev_allocate(name); + eth_dev = rte_eth_dev_allocate(name, RTE_ETH_DEV_VIRTUAL); if (eth_dev == NULL) goto err; -- 1.9.1
These functions are used for attaching or detaching a port. When rte_eal_dev_attach() is called, the function tries to realize the device name as pci address. If this is done successfully, rte_eal_dev_attach() will attach physical device port. If not, attaches virtual devive port. When rte_eal_dev_detach() is called, the function gets the device type of this port to know whether the port is came from physical or virtual. And then specific detaching function will be called. v5: - Change function names like below. rte_eal_dev_find_and_invoke() to rte_eal_vdev_find_and_invoke(). rte_eal_dev_invoke() to rte_eal_vdev_invoke(). - Add code to handle a return value of rte_eal_devargs_remove(). - Fix pci address format in rte_eal_dev_detach(). v4: - Fix comment. - Add error checking. - Fix indent of 'if' statement. - Change function name. Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp> --- lib/librte_eal/common/eal_common_dev.c | 274 ++++++++++++++++++++++++++++++++ lib/librte_eal/common/eal_private.h | 11 ++ lib/librte_eal/common/include/rte_dev.h | 33 ++++ lib/librte_eal/linuxapp/eal/Makefile | 1 + lib/librte_eal/linuxapp/eal/eal_pci.c | 6 +- 5 files changed, 322 insertions(+), 3 deletions(-) diff --git a/lib/librte_eal/common/eal_common_dev.c b/lib/librte_eal/common/eal_common_dev.c index eae5656..e3a3f54 100644 --- a/lib/librte_eal/common/eal_common_dev.c +++ b/lib/librte_eal/common/eal_common_dev.c @@ -32,10 +32,13 @@ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ +#include <stdio.h> +#include <limits.h> #include <string.h> #include <inttypes.h> #include <sys/queue.h> +#include <rte_ethdev.h> #include <rte_dev.h> #include <rte_devargs.h> #include <rte_debug.h> @@ -107,3 +110,274 @@ rte_eal_dev_init(void) } return 0; } + +/* So far, DPDK hotplug function only supports linux */ +#ifdef ENABLE_HOTPLUG +static void +rte_eal_vdev_invoke(struct rte_driver *driver, + struct rte_devargs *devargs, enum rte_eal_invoke_type type) +{ + if ((driver == NULL) || (devargs == NULL)) + return; + + switch (type) { + case RTE_EAL_INVOKE_TYPE_PROBE: + driver->init(devargs->virtual.drv_name, devargs->args); + break; + case RTE_EAL_INVOKE_TYPE_CLOSE: + driver->uninit(devargs->virtual.drv_name, devargs->args); + break; + default: + break; + } +} + +static int +rte_eal_vdev_find_and_invoke(const char *name, int type) +{ + struct rte_devargs *devargs; + struct rte_driver *driver; + + if (name == NULL) + return -EINVAL; + + /* call the init function for each virtual device */ + TAILQ_FOREACH(devargs, &devargs_list, next) { + + if (devargs->type != RTE_DEVTYPE_VIRTUAL) + continue; + + if (strncmp(name, devargs->virtual.drv_name, strlen(name))) + continue; + + TAILQ_FOREACH(driver, &dev_driver_list, next) { + if (driver->type != PMD_VDEV) + continue; + + /* search a driver prefix in virtual device name */ + if (!strncmp(driver->name, devargs->virtual.drv_name, + strlen(driver->name))) { + rte_eal_vdev_invoke(driver, devargs, type); + break; + } + } + + if (driver == NULL) { + RTE_LOG(WARNING, EAL, "no driver found for %s\n", + devargs->virtual.drv_name); + } + return 0; + } + return 1; +} + +/* attach the new physical device, then store port_id of the device */ +static int +rte_eal_dev_attach_pdev(struct rte_pci_addr *addr, uint8_t *port_id) +{ + uint8_t new_port_id; + struct rte_eth_dev devs[RTE_MAX_ETHPORTS]; + + if ((addr == NULL) || (port_id == NULL)) + goto err; + + /* save current port status */ + rte_eth_dev_save(devs); + /* re-construct pci_device_list */ + if (rte_eal_pci_scan()) + goto err; + /* invoke probe func of the driver can handle the new device */ + if (rte_eal_pci_probe_one(addr)) + goto err; + /* get port_id enabled by above procedures */ + if (rte_eth_dev_get_changed_port(devs, &new_port_id)) + goto err; + + *port_id = new_port_id; + return 0; +err: + RTE_LOG(ERR, EAL, "Drver, cannot attach the device\n"); + return -1; +} + +/* detach the new physical device, then store pci_addr of the device */ +static int +rte_eal_dev_detach_pdev(uint8_t port_id, struct rte_pci_addr *addr) +{ + struct rte_pci_addr freed_addr; + struct rte_pci_addr vp; + + if (addr == NULL) + goto err; + + /* check whether the driver supports detach feature, or not */ + if (rte_eth_dev_check_detachable(port_id)) + goto err; + + /* get pci address by port id */ + if (rte_eth_dev_get_addr_by_port(port_id, &freed_addr)) + goto err; + + /* Zerod pci addr means the port comes from virtual device */ + vp.domain = vp.bus = vp.devid = vp.function = 0; + if (eal_compare_pci_addr(&vp, &freed_addr) == 0) + goto err; + + /* invoke close func of the driver, + * also remove the device from pci_device_list */ + if (rte_eal_pci_close_one(&freed_addr)) + goto err; + + *addr = freed_addr; + return 0; +err: + RTE_LOG(ERR, EAL, "Drver, cannot detach the device\n"); + return -1; +} + +static void +get_vdev_name(char *vdevargs) +{ + char *sep; + + if (vdevargs == NULL) + return; + + /* set the first ',' to '\0' to split name and arguments */ + sep = strchr(vdevargs, ','); + if (sep != NULL) + sep[0] = '\0'; +} + +/* attach the new virtual device, then store port_id of the device */ +static int +rte_eal_dev_attach_vdev(const char *vdevargs, uint8_t *port_id) +{ + char *args; + uint8_t new_port_id; + struct rte_eth_dev devs[RTE_MAX_ETHPORTS]; + + if ((vdevargs == NULL) || (port_id == NULL)) + goto err0; + + args = strdup(vdevargs); + if (args == NULL) + goto err0; + + /* save current port status */ + rte_eth_dev_save(devs); + /* add the vdevargs to devargs_list */ + if (rte_eal_devargs_add(RTE_DEVTYPE_VIRTUAL, args)) + goto err1; + /* parse vdevargs, then retrieve device name */ + get_vdev_name(args); + /* walk around dev_driver_list to find the driver of the device, + * then invoke probe function o the driver */ + if (rte_eal_vdev_find_and_invoke(args, RTE_EAL_INVOKE_TYPE_PROBE)) + goto err2; + /* get port_id enabled by above procedures */ + if (rte_eth_dev_get_changed_port(devs, &new_port_id)) + goto err2; + + free(args); + *port_id = new_port_id; + return 0; +err2: + rte_eal_devargs_remove(RTE_DEVTYPE_VIRTUAL, args); +err1: + free(args); +err0: + RTE_LOG(ERR, EAL, "Drver, cannot detach the device\n"); + return -1; +} + +/* detach the new virtual device, then store the name of the device */ +static int +rte_eal_dev_detach_vdev(uint8_t port_id, char *vdevname) +{ + char name[RTE_ETH_NAME_MAX_LEN]; + + if (vdevname == NULL) + goto err; + + /* check whether the driver supports detach feature, or not */ + if (rte_eth_dev_check_detachable(port_id)) + goto err; + + /* get device name by port id */ + if (rte_eth_dev_get_name_by_port(port_id, name)) + goto err; + /* walk around dev_driver_list to find the driver of the device, + * then invoke close function o the driver */ + if (rte_eal_vdev_find_and_invoke(name, RTE_EAL_INVOKE_TYPE_CLOSE)) + goto err; + /* remove the vdevname from devargs_list */ + if (rte_eal_devargs_remove(RTE_DEVTYPE_VIRTUAL, name)) + goto err; + + strncpy(vdevname, name, sizeof(name)); + return 0; +err: + RTE_LOG(ERR, EAL, "Drver, cannot detach the device\n"); + return -1; +} + +/* attach the new device, then store port_id of the device */ +int +rte_eal_dev_attach(const char *devargs, uint8_t *port_id) +{ + struct rte_pci_addr addr; + + if ((devargs == NULL) || (port_id == NULL)) + return -EINVAL; + + if (eal_parse_pci_DomBDF(devargs, &addr) == 0) + return rte_eal_dev_attach_pdev(&addr, port_id); + else + return rte_eal_dev_attach_vdev(devargs, port_id); +} + +/* detach the device, then store the name of the device */ +int +rte_eal_dev_detach(uint8_t port_id, char *name) +{ + struct rte_pci_addr addr; + int ret; + + if (name == NULL) + return -EINVAL; + + if (rte_eth_dev_get_device_type(port_id) == RTE_ETH_DEV_PHYSICAL) { + ret = rte_eth_dev_get_addr_by_port(port_id, &addr); + if (ret < 0) + return ret; + + ret = rte_eal_dev_detach_pdev(port_id, &addr); + if (ret == 0) + snprintf(name, RTE_ETH_NAME_MAX_LEN, + "%04x:%02x:%02x.%d", + addr.domain, addr.bus, + addr.devid, addr.function); + + return ret; + } else + return rte_eal_dev_detach_vdev(port_id, name); +} +#else /* ENABLE_HOTPLUG */ +int +rte_eal_dev_attach(const char *devargs __rte_unused, + uint8_t *port_id __rte_unused) +{ + RTE_LOG(ERR, EAL, "Hotplug support isn't enabled\n"); + return -1; +} + +/* detach the device, then store the name of the device */ +int +rte_eal_dev_detach(uint8_t port_id __rte_unused, + char *name __rte_unused) +{ + RTE_LOG(ERR, EAL, "Hotplug support isn't enabled\n"); + return -1; +} +#endif /* ENABLE_HOTPLUG */ diff --git a/lib/librte_eal/common/eal_private.h b/lib/librte_eal/common/eal_private.h index 1a362ab..8168a7a 100644 --- a/lib/librte_eal/common/eal_private.h +++ b/lib/librte_eal/common/eal_private.h @@ -163,6 +163,17 @@ enum rte_eal_invoke_type { }; /** + * Scan the content of the PCI bus, and the devices in the devices + * list + * + * This function is private to EAL. + * + * @return + * 0 on success, negative on error + */ +int rte_eal_pci_scan(void); + +/** * Mmap memory for single PCI device * * This function is private to EAL. diff --git a/lib/librte_eal/common/include/rte_dev.h b/lib/librte_eal/common/include/rte_dev.h index f7e3a10..e63dd1c 100644 --- a/lib/librte_eal/common/include/rte_dev.h +++ b/lib/librte_eal/common/include/rte_dev.h @@ -47,6 +47,7 @@ extern "C" { #endif #include <sys/queue.h> +#include <rte_pci.h> /** Double linked list of device drivers. */ TAILQ_HEAD(rte_driver_list, rte_driver); @@ -57,6 +58,11 @@ TAILQ_HEAD(rte_driver_list, rte_driver); typedef int (rte_dev_init_t)(const char *name, const char *args); /** + * Uninitilization function called for each device driver once. + */ +typedef int (rte_dev_uninit_t)(const char *name, const char *args); + +/** * Driver type enumeration */ enum pmd_type { @@ -72,6 +78,7 @@ struct rte_driver { enum pmd_type type; /**< PMD Driver type */ const char *name; /**< Driver name. */ rte_dev_init_t *init; /**< Device init. function. */ + rte_dev_uninit_t *uninit; /**< Device uninit. function. */ }; /** @@ -93,6 +100,32 @@ void rte_eal_driver_register(struct rte_driver *driver); void rte_eal_driver_unregister(struct rte_driver *driver); /** + * Attach a new device. + * + * @param devargs + * A pointer to a strings array describing the new device + * to be attached. The strings should be a pci address like + * '0000:01:00.0' or virtual device name like 'eth_pcap0'. + * @param port_id + * A pointer to a port identifier actually attached. + * @return + * 0 on success and port_id is filled, negative on error + */ +int rte_eal_dev_attach(const char *devargs, uint8_t *port_id); + +/** + * Detach a device. + * + * @param port_id + * The port identifier of the device to detach. + * @param addr + * A pointer to a device name actually detached. + * @return + * 0 on success and devname is filled, negative on error + */ +int rte_eal_dev_detach(uint8_t port_id, char *devname); + +/** * Initalize all the registered drivers in this process */ int rte_eal_dev_init(void); diff --git a/lib/librte_eal/linuxapp/eal/Makefile b/lib/librte_eal/linuxapp/eal/Makefile index 72ecf3a..0ec83b5 100644 --- a/lib/librte_eal/linuxapp/eal/Makefile +++ b/lib/librte_eal/linuxapp/eal/Makefile @@ -41,6 +41,7 @@ CFLAGS += -I$(RTE_SDK)/lib/librte_eal/common/include CFLAGS += -I$(RTE_SDK)/lib/librte_ring CFLAGS += -I$(RTE_SDK)/lib/librte_mempool CFLAGS += -I$(RTE_SDK)/lib/librte_malloc +CFLAGS += -I$(RTE_SDK)/lib/librte_mbuf CFLAGS += -I$(RTE_SDK)/lib/librte_ether CFLAGS += -I$(RTE_SDK)/lib/librte_ivshmem CFLAGS += -I$(RTE_SDK)/lib/librte_pmd_ring diff --git a/lib/librte_eal/linuxapp/eal/eal_pci.c b/lib/librte_eal/linuxapp/eal/eal_pci.c index 831422e..1f43688 100644 --- a/lib/librte_eal/linuxapp/eal/eal_pci.c +++ b/lib/librte_eal/linuxapp/eal/eal_pci.c @@ -431,8 +431,8 @@ error: * Scan the content of the PCI bus, and the devices in the devices * list */ -static int -pci_scan(void) +int +rte_eal_pci_scan(void) { struct dirent *e; DIR *dir; @@ -764,7 +764,7 @@ rte_eal_pci_init(void) if (internal_config.no_pci) return 0; - if (pci_scan() < 0) { + if (rte_eal_pci_scan() < 0) { RTE_LOG(ERR, EAL, "%s(): Cannot scan PCI bus\n", __func__); return -1; } -- 1.9.1
The patch enables CONFIG_RTE_LIBRTE_EAL_HOTPLUG in Linux configuration. Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp> --- config/common_linuxapp | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/config/common_linuxapp b/config/common_linuxapp index 2f9643b..27d05be 100644 --- a/config/common_linuxapp +++ b/config/common_linuxapp @@ -114,6 +114,11 @@ CONFIG_RTE_PCI_MAX_READ_REQUEST_SIZE=0 CONFIG_RTE_LIBRTE_EAL_LINUXAPP=y # +# Compile Environment Abstraction Layer to support hotplug +# +CONFIG_RTE_LIBRTE_EAL_HOTPLUG=y + +# # Compile Environment Abstraction Layer to support Vmware TSC map # CONFIG_RTE_LIBRTE_EAL_VMWARE_TSC_MAP_SUPPORT=y -- 1.9.1
This patch adds finalization code to free resources allocated by the PMD. v6: - Fix a paramter of rte_eth_dev_free(). v4: - Change function name. Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp> --- lib/librte_pmd_pcap/rte_eth_pcap.c | 40 ++++++++++++++++++++++++++++++++++++++ 1 file changed, 40 insertions(+) diff --git a/lib/librte_pmd_pcap/rte_eth_pcap.c b/lib/librte_pmd_pcap/rte_eth_pcap.c index af7fae8..5f88efd 100644 --- a/lib/librte_pmd_pcap/rte_eth_pcap.c +++ b/lib/librte_pmd_pcap/rte_eth_pcap.c @@ -498,6 +498,13 @@ static struct eth_dev_ops ops = { .stats_reset = eth_stats_reset, }; +static struct eth_driver rte_pcap_pmd = { + .pci_drv = { + .name = "rte_pcap_pmd", + .drv_flags = RTE_PCI_DRV_DETACHABLE, + }, +}; + /* * Function handler that opens the pcap file for reading a stores a * reference of it for use it later on. @@ -713,6 +720,10 @@ rte_pmd_init_internals(const char *name, const unsigned nb_rx_queues, if (*eth_dev == NULL) goto error; + /* check length of device name */ + if ((strlen((*eth_dev)->data->name) + 1) > sizeof(data->name)) + goto error; + /* now put it all together * - store queue data in internals, * - store numa_node info in pci_driver @@ -739,10 +750,13 @@ rte_pmd_init_internals(const char *name, const unsigned nb_rx_queues, data->nb_tx_queues = (uint16_t)nb_tx_queues; data->dev_link = pmd_link; data->mac_addrs = ð_addr; + strncpy(data->name, + (*eth_dev)->data->name, strlen((*eth_dev)->data->name)); (*eth_dev)->data = data; (*eth_dev)->dev_ops = &ops; (*eth_dev)->pci_dev = pci_dev; + (*eth_dev)->driver = &rte_pcap_pmd; return 0; @@ -927,10 +941,36 @@ rte_pmd_pcap_devinit(const char *name, const char *params) } +static int +rte_pmd_pcap_devuninit(const char *name, const char *params __rte_unused) +{ + struct rte_eth_dev *eth_dev = NULL; + + RTE_LOG(INFO, PMD, "Closing pcap ethdev on numa socket %u\n", + rte_socket_id()); + + if (name == NULL) + return -1; + + /* reserve an ethdev entry */ + eth_dev = rte_eth_dev_allocated(name); + if (eth_dev == NULL) + return -1; + + rte_free(eth_dev->data->dev_private); + rte_free(eth_dev->data); + rte_free(eth_dev->pci_dev); + + rte_eth_dev_free(eth_dev); + + return 0; +} + static struct rte_driver pmd_pcap_drv = { .name = "eth_pcap", .type = PMD_VDEV, .init = rte_pmd_pcap_devinit, + .uninit = rte_pmd_pcap_devuninit, }; PMD_REGISTER_DRIVER(pmd_pcap_drv); -- 1.9.1
The patch introduces following commands. - port attach [ident] - port detach [port_id] - attach: attaching a port - detach: detaching a port - ident: pci address of physical device. Or device name and paramerters of virtual device. (ex. 0000:02:00.0, eth_pcap0,iface=eth0) - port_id: port identifier v5: - Add testpmd documentation. (Thanks to Iremonger, Bernard) v4: - Fix strings of command help. Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp> --- app/test-pmd/cmdline.c | 133 +++++++++++++++---- app/test-pmd/config.c | 116 +++++++++------- app/test-pmd/parameters.c | 22 ++- app/test-pmd/testpmd.c | 199 +++++++++++++++++++++------- app/test-pmd/testpmd.h | 18 ++- doc/guides/testpmd_app_ug/testpmd_funcs.rst | 57 ++++++++ 6 files changed, 415 insertions(+), 130 deletions(-) diff --git a/app/test-pmd/cmdline.c b/app/test-pmd/cmdline.c index 4beb404..2f813d8 100644 --- a/app/test-pmd/cmdline.c +++ b/app/test-pmd/cmdline.c @@ -572,6 +572,12 @@ static void cmd_help_long_parsed(void *parsed_result, "port close (port_id|all)\n" " Close all ports or port_id.\n\n" + "port attach (ident)\n" + " Attach physical or virtual dev by pci address or virtual device name\n\n" + + "port detach (port_id)\n" + " Detach physical or virtual dev by port_id\n\n" + "port config (port_id|all)" " speed (10|100|1000|10000|40000|auto)" " duplex (half|full|auto)\n" @@ -848,6 +854,89 @@ cmdline_parse_inst_t cmd_operate_specific_port = { }, }; +/* *** attach a specificied port *** */ +struct cmd_operate_attach_port_result { + cmdline_fixed_string_t port; + cmdline_fixed_string_t keyword; + cmdline_fixed_string_t identifier; +}; + +static void cmd_operate_attach_port_parsed(void *parsed_result, + __attribute__((unused)) struct cmdline *cl, + __attribute__((unused)) void *data) +{ + struct cmd_operate_attach_port_result *res = parsed_result; + + if (!strcmp(res->keyword, "attach")) + attach_port(res->identifier); + else + printf("Unknown parameter\n"); +} + +cmdline_parse_token_string_t cmd_operate_attach_port_port = + TOKEN_STRING_INITIALIZER(struct cmd_operate_attach_port_result, + port, "port"); +cmdline_parse_token_string_t cmd_operate_attach_port_keyword = + TOKEN_STRING_INITIALIZER(struct cmd_operate_attach_port_result, + keyword, "attach"); +cmdline_parse_token_string_t cmd_operate_attach_port_identifier = + TOKEN_STRING_INITIALIZER(struct cmd_operate_attach_port_result, + identifier, NULL); + +cmdline_parse_inst_t cmd_operate_attach_port = { + .f = cmd_operate_attach_port_parsed, + .data = NULL, + .help_str = "port attach identifier, " + "identifier: pci address or virtual dev name", + .tokens = { + (void *)&cmd_operate_attach_port_port, + (void *)&cmd_operate_attach_port_keyword, + (void *)&cmd_operate_attach_port_identifier, + NULL, + }, +}; + +/* *** detach a specificied port *** */ +struct cmd_operate_detach_port_result { + cmdline_fixed_string_t port; + cmdline_fixed_string_t keyword; + uint8_t port_id; +}; + +static void cmd_operate_detach_port_parsed(void *parsed_result, + __attribute__((unused)) struct cmdline *cl, + __attribute__((unused)) void *data) +{ + struct cmd_operate_detach_port_result *res = parsed_result; + + if (!strcmp(res->keyword, "detach")) + detach_port(res->port_id); + else + printf("Unknown parameter\n"); +} + +cmdline_parse_token_string_t cmd_operate_detach_port_port = + TOKEN_STRING_INITIALIZER(struct cmd_operate_detach_port_result, + port, "port"); +cmdline_parse_token_string_t cmd_operate_detach_port_keyword = + TOKEN_STRING_INITIALIZER(struct cmd_operate_detach_port_result, + keyword, "detach"); +cmdline_parse_token_num_t cmd_operate_detach_port_port_id = + TOKEN_NUM_INITIALIZER(struct cmd_operate_detach_port_result, + port_id, UINT8); + +cmdline_parse_inst_t cmd_operate_detach_port = { + .f = cmd_operate_detach_port_parsed, + .data = NULL, + .help_str = "port detach port_id", + .tokens = { + (void *)&cmd_operate_detach_port_port, + (void *)&cmd_operate_detach_port_keyword, + (void *)&cmd_operate_detach_port_port_id, + NULL, + }, +}; + /* *** configure speed for all ports *** */ struct cmd_config_speed_all { cmdline_fixed_string_t port; @@ -902,7 +991,7 @@ cmd_config_speed_all_parsed(void *parsed_result, return; } - for (pid = 0; pid < nb_ports; pid++) { + FOREACH_PORT(pid, ports) { ports[pid].dev_conf.link_speed = link_speed; ports[pid].dev_conf.link_duplex = link_duplex; } @@ -970,10 +1059,8 @@ cmd_config_speed_specific_parsed(void *parsed_result, return; } - if (res->id >= nb_ports) { - printf("Port id %d must be less than %d\n", res->id, nb_ports); + if (port_id_is_invalid(res->id, ENABLED_WARN)) return; - } if (!strcmp(res->value1, "10")) link_speed = ETH_LINK_SPEED_10; @@ -1533,7 +1620,7 @@ cmd_config_rxtx_queue_parsed(void *parsed_result, return; } - if (port_id_is_invalid(res->portid)) + if (port_id_is_invalid(res->portid, ENABLED_WARN)) return; if (port_is_started(res->portid) != 1) { @@ -2876,7 +2963,7 @@ cmd_tx_cksum_parsed(void *parsed_result, uint16_t ol_flags, mask = 0; struct rte_eth_dev_info dev_info; - if (port_id_is_invalid(res->port_id)) { + if (port_id_is_invalid(res->port_id, ENABLED_WARN)) { printf("invalid port %d\n", res->port_id); return; } @@ -3003,7 +3090,7 @@ cmd_tso_set_parsed(void *parsed_result, struct cmd_tso_set_result *res = parsed_result; struct rte_eth_dev_info dev_info; - if (port_id_is_invalid(res->port_id)) + if (port_id_is_invalid(res->port_id, ENABLED_WARN)) return; if (!strcmp(res->mode, "set")) @@ -3979,10 +4066,8 @@ static void cmd_set_bond_mac_addr_parsed(void *parsed_result, struct cmd_set_bond_mac_addr_result *res = parsed_result; int ret; - if (res->port_num >= nb_ports) { - printf("Port id %d must be less than %d\n", res->port_num, nb_ports); + if (port_id_is_invalid(res->port_num, ENABLED_WARN)) return; - } ret = rte_eth_bond_mac_address_set(res->port_num, &res->address); @@ -4219,7 +4304,7 @@ static void cmd_set_promisc_mode_parsed(void *parsed_result, /* all ports */ if (allports) { - for (i = 0; i < nb_ports; i++) { + FOREACH_PORT(i, ports) { if (enable) rte_eth_promiscuous_enable(i); else @@ -4299,7 +4384,7 @@ static void cmd_set_allmulti_mode_parsed(void *parsed_result, /* all ports */ if (allports) { - for (i = 0; i < nb_ports; i++) { + FOREACH_PORT(i, ports) { if (enable) rte_eth_allmulticast_enable(i); else @@ -5484,25 +5569,25 @@ static void cmd_showportall_parsed(void *parsed_result, struct cmd_showportall_result *res = parsed_result; if (!strcmp(res->show, "clear")) { if (!strcmp(res->what, "stats")) - for (i = 0; i < nb_ports; i++) + FOREACH_PORT(i, ports) nic_stats_clear(i); else if (!strcmp(res->what, "xstats")) - for (i = 0; i < nb_ports; i++) + FOREACH_PORT(i, ports) nic_xstats_clear(i); } else if (!strcmp(res->what, "info")) - for (i = 0; i < nb_ports; i++) + FOREACH_PORT(i, ports) port_infos_display(i); else if (!strcmp(res->what, "stats")) - for (i = 0; i < nb_ports; i++) + FOREACH_PORT(i, ports) nic_stats_display(i); else if (!strcmp(res->what, "xstats")) - for (i = 0; i < nb_ports; i++) + FOREACH_PORT(i, ports) nic_xstats_display(i); else if (!strcmp(res->what, "fdir")) - for (i = 0; i < nb_ports; i++) + FOREACH_PORT(i, ports) fdir_get_infos(i); else if (!strcmp(res->what, "stat_qmap")) - for (i = 0; i < nb_ports; i++) + FOREACH_PORT(i, ports) nic_stats_mapping_display(i); } @@ -8756,6 +8841,8 @@ cmdline_parse_ctx_t main_ctx[] = { (cmdline_parse_inst_t *)&cmd_set_qmap, (cmdline_parse_inst_t *)&cmd_operate_port, (cmdline_parse_inst_t *)&cmd_operate_specific_port, + (cmdline_parse_inst_t *)&cmd_operate_attach_port, + (cmdline_parse_inst_t *)&cmd_operate_detach_port, (cmdline_parse_inst_t *)&cmd_config_speed_all, (cmdline_parse_inst_t *)&cmd_config_speed_specific, (cmdline_parse_inst_t *)&cmd_config_rx_tx, @@ -8830,7 +8917,7 @@ prompt(void) static void cmd_reconfig_device_queue(portid_t id, uint8_t dev, uint8_t queue) { - if (id < nb_ports) { + if (!port_id_is_invalid(id, DISABLED_WARN)) { /* check if need_reconfig has been set to 1 */ if (ports[id].need_reconfig == 0) ports[id].need_reconfig = dev; @@ -8840,7 +8927,7 @@ cmd_reconfig_device_queue(portid_t id, uint8_t dev, uint8_t queue) } else { portid_t pid; - for (pid = 0; pid < nb_ports; pid++) { + FOREACH_PORT(pid, ports) { /* check if need_reconfig has been set to 1 */ if (ports[pid].need_reconfig == 0) ports[pid].need_reconfig = dev; @@ -8858,10 +8945,8 @@ bypass_is_supported(portid_t port_id) struct rte_port *port; struct rte_pci_id *pci_id; - if (port_id >= nb_ports) { - printf("\tPort id must be less than %d.\n", nb_ports); + if (port_id_is_invalid(port_id, ENABLED_WARN)) return 0; - } /* Get the device id. */ port = &ports[port_id]; diff --git a/app/test-pmd/config.c b/app/test-pmd/config.c index c40f819..32d8f9a 100644 --- a/app/test-pmd/config.c +++ b/app/test-pmd/config.c @@ -124,11 +124,15 @@ nic_stats_display(portid_t port_id) struct rte_eth_stats stats; struct rte_port *port = &ports[port_id]; uint8_t i; + portid_t pid; static const char *nic_stats_border = "########################"; - if (port_id >= nb_ports) { - printf("Invalid port, range is [0, %d]\n", nb_ports - 1); + if (port_id_is_invalid(port_id, ENABLED_WARN)) { + printf("Valid port range is [0"); + FOREACH_PORT(pid, ports) + printf(", %d", pid); + printf("]\n"); return; } rte_eth_stats_get(port_id, &stats); @@ -201,8 +205,13 @@ nic_stats_display(portid_t port_id) void nic_stats_clear(portid_t port_id) { - if (port_id >= nb_ports) { - printf("Invalid port, range is [0, %d]\n", nb_ports - 1); + portid_t pid; + + if (port_id_is_invalid(port_id, ENABLED_WARN)) { + printf("Valid port range is [0"); + FOREACH_PORT(pid, ports) + printf(", %d", pid); + printf("]\n"); return; } rte_eth_stats_reset(port_id); @@ -249,11 +258,15 @@ nic_stats_mapping_display(portid_t port_id) { struct rte_port *port = &ports[port_id]; uint16_t i; + portid_t pid; static const char *nic_stats_mapping_border = "########################"; - if (port_id >= nb_ports) { - printf("Invalid port, range is [0, %d]\n", nb_ports - 1); + if (port_id_is_invalid(port_id, ENABLED_WARN)) { + printf("Valid port range is [0"); + FOREACH_PORT(pid, ports) + printf(", %d", pid); + printf("]\n"); return; } @@ -302,9 +315,13 @@ port_infos_display(portid_t port_id) int vlan_offload; struct rte_mempool * mp; static const char *info_border = "*********************"; + portid_t pid; - if (port_id >= nb_ports) { - printf("Invalid port, range is [0, %d]\n", nb_ports - 1); + if (port_id_is_invalid(port_id, ENABLED_WARN)) { + printf("Valid port range is [0"); + FOREACH_PORT(pid, ports) + printf(", %d", pid); + printf("]\n"); return; } port = &ports[port_id]; @@ -362,11 +379,14 @@ port_infos_display(portid_t port_id) } int -port_id_is_invalid(portid_t port_id) +port_id_is_invalid(portid_t port_id, enum print_warning warning) { - if (port_id < nb_ports) + if (ports[port_id].enabled) return 0; - printf("Invalid port %d (must be < nb_ports=%d)\n", port_id, nb_ports); + + if (warning == ENABLED_WARN) + printf("Invalid port %d\n", port_id); + return 1; } @@ -425,7 +445,7 @@ port_reg_bit_display(portid_t port_id, uint32_t reg_off, uint8_t bit_x) uint32_t reg_v; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; if (port_reg_off_is_invalid(port_id, reg_off)) return; @@ -444,7 +464,7 @@ port_reg_bit_field_display(portid_t port_id, uint32_t reg_off, uint8_t l_bit; uint8_t h_bit; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; if (port_reg_off_is_invalid(port_id, reg_off)) return; @@ -471,7 +491,7 @@ port_reg_display(portid_t port_id, uint32_t reg_off) { uint32_t reg_v; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; if (port_reg_off_is_invalid(port_id, reg_off)) return; @@ -485,7 +505,7 @@ port_reg_bit_set(portid_t port_id, uint32_t reg_off, uint8_t bit_pos, { uint32_t reg_v; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; if (port_reg_off_is_invalid(port_id, reg_off)) return; @@ -513,7 +533,7 @@ port_reg_bit_field_set(portid_t port_id, uint32_t reg_off, uint8_t l_bit; uint8_t h_bit; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; if (port_reg_off_is_invalid(port_id, reg_off)) return; @@ -547,7 +567,7 @@ port_reg_bit_field_set(portid_t port_id, uint32_t reg_off, void port_reg_set(portid_t port_id, uint32_t reg_off, uint32_t reg_v) { - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; if (port_reg_off_is_invalid(port_id, reg_off)) return; @@ -560,7 +580,7 @@ port_mtu_set(portid_t port_id, uint16_t mtu) { int diag; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; diag = rte_eth_dev_set_mtu(port_id, mtu); if (diag == 0) @@ -723,7 +743,7 @@ rx_ring_desc_display(portid_t port_id, queueid_t rxq_id, uint16_t rxd_id) { const struct rte_memzone *rx_mz; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; if (rx_queue_id_is_invalid(rxq_id)) return; @@ -740,7 +760,7 @@ tx_ring_desc_display(portid_t port_id, queueid_t txq_id, uint16_t txd_id) { const struct rte_memzone *tx_mz; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; if (tx_queue_id_is_invalid(txq_id)) return; @@ -796,7 +816,7 @@ port_rss_reta_info(portid_t port_id, uint16_t i, idx, shift; int ret; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; ret = rte_eth_dev_rss_reta_query(port_id, reta_conf, nb_entries); @@ -828,7 +848,7 @@ port_rss_hash_conf_show(portid_t port_id, int show_rss_key) uint8_t i; int diag; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; /* Get RSS hash key if asked to display it */ rss_conf.rss_key = (show_rss_key) ? rss_key : NULL; @@ -1406,12 +1426,8 @@ set_fwd_ports_list(unsigned int *portlist, unsigned int nb_pt) again: for (i = 0; i < nb_pt; i++) { port_id = (portid_t) portlist[i]; - if (port_id >= nb_ports) { - printf("Invalid port id %u >= %u\n", - (unsigned int) port_id, - (unsigned int) nb_ports); + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; - } if (record_now) fwd_ports_ids[i] = port_id; } @@ -1569,7 +1585,7 @@ vlan_extend_set(portid_t port_id, int on) int diag; int vlan_offload; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; vlan_offload = rte_eth_dev_get_vlan_offload(port_id); @@ -1591,7 +1607,7 @@ rx_vlan_strip_set(portid_t port_id, int on) int diag; int vlan_offload; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; vlan_offload = rte_eth_dev_get_vlan_offload(port_id); @@ -1612,7 +1628,7 @@ rx_vlan_strip_set_on_queue(portid_t port_id, uint16_t queue_id, int on) { int diag; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; diag = rte_eth_dev_set_vlan_strip_on_queue(port_id, queue_id, on); @@ -1627,7 +1643,7 @@ rx_vlan_filter_set(portid_t port_id, int on) int diag; int vlan_offload; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; vlan_offload = rte_eth_dev_get_vlan_offload(port_id); @@ -1648,7 +1664,7 @@ rx_vft_set(portid_t port_id, uint16_t vlan_id, int on) { int diag; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; if (vlan_id_is_invalid(vlan_id)) return; @@ -1665,7 +1681,7 @@ rx_vlan_all_filter_set(portid_t port_id, int on) { uint16_t vlan_id; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; for (vlan_id = 0; vlan_id < 4096; vlan_id++) rx_vft_set(port_id, vlan_id, on); @@ -1675,7 +1691,7 @@ void vlan_tpid_set(portid_t port_id, uint16_t tp_id) { int diag; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; diag = rte_eth_dev_set_vlan_ether_type(port_id, tp_id); @@ -1690,7 +1706,7 @@ vlan_tpid_set(portid_t port_id, uint16_t tp_id) void tx_vlan_set(portid_t port_id, uint16_t vlan_id) { - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; if (vlan_id_is_invalid(vlan_id)) return; @@ -1701,7 +1717,7 @@ tx_vlan_set(portid_t port_id, uint16_t vlan_id) void tx_vlan_reset(portid_t port_id) { - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; ports[port_id].tx_ol_flags &= ~TESTPMD_TX_OFFLOAD_INSERT_VLAN; } @@ -1709,7 +1725,7 @@ tx_vlan_reset(portid_t port_id) void tx_vlan_pvid_set(portid_t port_id, uint16_t vlan_id, int on) { - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; rte_eth_dev_set_vlan_pvid(port_id, vlan_id, on); @@ -1721,7 +1737,7 @@ set_qmap(portid_t port_id, uint8_t is_rx, uint16_t queue_id, uint8_t map_value) uint16_t i; uint8_t existing_mapping_found = 0; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; if (is_rx ? (rx_queue_id_is_invalid(queue_id)) : (tx_queue_id_is_invalid(queue_id))) @@ -1773,7 +1789,7 @@ fdir_add_signature_filter(portid_t port_id, uint8_t queue_id, { int diag; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; diag = rte_eth_dev_fdir_add_signature_filter(port_id, fdir_filter, @@ -1791,7 +1807,7 @@ fdir_update_signature_filter(portid_t port_id, uint8_t queue_id, { int diag; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; diag = rte_eth_dev_fdir_update_signature_filter(port_id, fdir_filter, @@ -1809,7 +1825,7 @@ fdir_remove_signature_filter(portid_t port_id, { int diag; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; diag = rte_eth_dev_fdir_remove_signature_filter(port_id, fdir_filter); @@ -1881,7 +1897,7 @@ fdir_get_infos(portid_t port_id) static const char *fdir_stats_border = "########################"; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; ret = rte_eth_dev_filter_supported(port_id, RTE_ETH_FILTER_FDIR); if (ret < 0) { @@ -1955,7 +1971,7 @@ fdir_add_perfect_filter(portid_t port_id, uint16_t soft_id, uint8_t queue_id, { int diag; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; diag = rte_eth_dev_fdir_add_perfect_filter(port_id, fdir_filter, @@ -1973,7 +1989,7 @@ fdir_update_perfect_filter(portid_t port_id, uint16_t soft_id, uint8_t queue_id, { int diag; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; diag = rte_eth_dev_fdir_update_perfect_filter(port_id, fdir_filter, @@ -1991,7 +2007,7 @@ fdir_remove_perfect_filter(portid_t port_id, uint16_t soft_id, { int diag; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; diag = rte_eth_dev_fdir_remove_perfect_filter(port_id, fdir_filter, @@ -2008,7 +2024,7 @@ fdir_set_masks(portid_t port_id, struct rte_fdir_masks *fdir_masks) { int diag; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; diag = rte_eth_dev_fdir_set_masks(port_id, fdir_masks); @@ -2085,7 +2101,7 @@ set_vf_traffic(portid_t port_id, uint8_t is_rx, uint16_t vf, uint8_t on) { int diag; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; if (is_rx) diag = rte_eth_dev_set_vf_rx(port_id,vf,on); @@ -2107,7 +2123,7 @@ set_vf_rx_vlan(portid_t port_id, uint16_t vlan_id, uint64_t vf_mask, uint8_t on) { int diag; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; if (vlan_id_is_invalid(vlan_id)) return; @@ -2124,7 +2140,7 @@ set_queue_rate_limit(portid_t port_id, uint16_t queue_idx, uint16_t rate) int diag; struct rte_eth_link link; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return 1; rte_eth_link_get_nowait(port_id, &link); if (rate > link.link_speed) { @@ -2149,7 +2165,7 @@ set_vf_rate_limit(portid_t port_id, uint16_t vf, uint16_t rate, uint64_t q_msk) if (q_msk == 0) return 0; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return 1; rte_eth_link_get_nowait(port_id, &link); if (rate > link.link_speed) { diff --git a/app/test-pmd/parameters.c b/app/test-pmd/parameters.c index adf3203..6f2af18 100644 --- a/app/test-pmd/parameters.c +++ b/app/test-pmd/parameters.c @@ -376,6 +376,7 @@ parse_portnuma_config(const char *q_arg) }; unsigned long int_fld[_NUM_FLD]; char *str_fld[_NUM_FLD]; + portid_t pid; /* reset from value set at definition */ while ((p = strchr(p0,'(')) != NULL) { @@ -397,8 +398,11 @@ parse_portnuma_config(const char *q_arg) return -1; } port_id = (uint8_t)int_fld[FLD_PORT]; - if (port_id >= nb_ports) { - printf("Invalid port, range is [0, %d]\n", nb_ports - 1); + if (port_id_is_invalid(port_id, ENABLED_WARN)) { + printf("Valid port range is [0"); + FOREACH_PORT(pid, ports) + printf(", %d", pid); + printf("]\n"); return -1; } socket_id = (uint8_t)int_fld[FLD_SOCKET]; @@ -429,6 +433,7 @@ parse_ringnuma_config(const char *q_arg) }; unsigned long int_fld[_NUM_FLD]; char *str_fld[_NUM_FLD]; + portid_t pid; #define RX_RING_ONLY 0x1 #define TX_RING_ONLY 0x2 #define RXTX_RING 0x3 @@ -453,8 +458,11 @@ parse_ringnuma_config(const char *q_arg) return -1; } port_id = (uint8_t)int_fld[FLD_PORT]; - if (port_id >= nb_ports) { - printf("Invalid port, range is [0, %d]\n", nb_ports - 1); + if (port_id_is_invalid(port_id, ENABLED_WARN)) { + printf("Valid port range is [0"); + FOREACH_PORT(pid, ports) + printf(", %d", pid); + printf("]\n"); return -1; } socket_id = (uint8_t)int_fld[FLD_SOCKET]; @@ -626,12 +634,12 @@ launch_args_parse(int argc, char** argv) #endif if (!strcmp(lgopts[opt_idx].name, "nb-ports")) { n = atoi(optarg); - if (n > 0 && n <= nb_ports) + if (n > 0 && + !port_id_is_invalid(n, DISABLED_WARN)) nb_fwd_ports = (uint8_t) n; else rte_exit(EXIT_FAILURE, - "nb-ports should be > 0 and <= %d\n", - nb_ports); + "Invalid port %d\n", n); } if (!strcmp(lgopts[opt_idx].name, "nb-cores")) { n = atoi(optarg); diff --git a/app/test-pmd/testpmd.c b/app/test-pmd/testpmd.c index 773b8af..c18c1a9 100644 --- a/app/test-pmd/testpmd.c +++ b/app/test-pmd/testpmd.c @@ -71,6 +71,7 @@ #include <rte_pci.h> #include <rte_ether.h> #include <rte_ethdev.h> +#include <rte_dev.h> #include <rte_string_fns.h> #ifdef RTE_LIBRTE_PMD_XENVIRT #include <rte_eth_xenvirt.h> @@ -315,7 +316,7 @@ uint16_t nb_rx_queue_stats_mappings = 0; /* Forward function declarations */ static void map_port_queue_stats_mapping_registers(uint8_t pi, struct rte_port *port); -static void check_all_ports_link_status(uint8_t port_num, uint32_t port_mask); +static void check_all_ports_link_status(uint32_t port_mask); /* * Check if all the ports are started. @@ -324,6 +325,20 @@ static void check_all_ports_link_status(uint8_t port_num, uint32_t port_mask); static int all_ports_started(void); /* + * Find next enabled port + */ +portid_t +find_next_port(portid_t p, struct rte_port *ports, int size) +{ + if (ports == NULL) + rte_exit(-EINVAL, "failed to find a next port id\n"); + + while ((ports[p].enabled == 0) && (p < size)) + p++; + return p; +} + +/* * Setup default configuration. */ static void @@ -552,7 +567,8 @@ init_config(void) + RTE_TEST_TX_DESC_MAX + MAX_PKT_BURST; if (!numa_support) - nb_mbuf_per_pool = (nb_mbuf_per_pool * nb_ports); + nb_mbuf_per_pool = + (nb_mbuf_per_pool * RTE_MAX_ETHPORTS); } if (!numa_support) { @@ -565,14 +581,19 @@ init_config(void) /* Configuration of Ethernet ports. */ ports = rte_zmalloc("testpmd: ports", - sizeof(struct rte_port) * nb_ports, + sizeof(struct rte_port) * RTE_MAX_ETHPORTS, RTE_CACHE_LINE_SIZE); if (ports == NULL) { - rte_exit(EXIT_FAILURE, "rte_zmalloc(%d struct rte_port) " - "failed\n", nb_ports); + rte_exit(EXIT_FAILURE, + "rte_zmalloc(%d struct rte_port) failed\n", + RTE_MAX_ETHPORTS); } - for (pid = 0; pid < nb_ports; pid++) { + /* enabled allocated ports */ + for (pid = 0; pid < nb_ports; pid++) + ports[pid].enabled = 1; + + FOREACH_PORT(pid, ports) { port = &ports[pid]; rte_eth_dev_info_get(pid, &port->dev_info); @@ -602,8 +623,7 @@ init_config(void) nb_mbuf_per_pool = nb_mbuf_per_pool/nb_ports; for (i = 0; i < MAX_SOCKET; i++) { - nb_mbuf = (nb_mbuf_per_pool * - port_per_socket[i]); + nb_mbuf = (nb_mbuf_per_pool * RTE_MAX_ETHPORTS); if (nb_mbuf) mbuf_pool_create(mbuf_data_size, nb_mbuf,i); @@ -635,14 +655,6 @@ reconfig(portid_t new_port_id, unsigned socket_id) struct rte_port *port; /* Reconfiguration of Ethernet ports. */ - ports = rte_realloc(ports, - sizeof(struct rte_port) * nb_ports, - RTE_CACHE_LINE_SIZE); - if (ports == NULL) { - rte_exit(EXIT_FAILURE, "rte_realloc(%d struct rte_port) failed\n", - nb_ports); - } - port = &ports[new_port_id]; rte_eth_dev_info_get(new_port_id, &port->dev_info); @@ -663,7 +675,7 @@ init_fwd_streams(void) streamid_t sm_id, nb_fwd_streams_new; /* set socket id according to numa or not */ - for (pid = 0; pid < nb_ports; pid++) { + FOREACH_PORT(pid, ports) { port = &ports[pid]; if (nb_rxq > port->dev_info.max_rx_queues) { printf("Fail: nb_rxq(%d) is greater than " @@ -1264,7 +1276,7 @@ all_ports_started(void) portid_t pi; struct rte_port *port; - for (pi = 0; pi < nb_ports; pi++) { + FOREACH_PORT(pi, ports) { port = &ports[pi]; /* Check if there is a port which is not started */ if (port->port_status != RTE_PORT_STARTED) @@ -1276,6 +1288,45 @@ all_ports_started(void) } int +all_ports_stopped(void) +{ + portid_t pi; + struct rte_port *port; + + FOREACH_PORT(pi, ports) { + port = &ports[pi]; + if (port->port_status != RTE_PORT_STOPPED) + return 0; + } + + return 1; +} + +int +port_is_started(portid_t port_id) +{ + if (port_id_is_invalid(port_id, ENABLED_WARN)) + return 0; + + if (ports[port_id].port_status != RTE_PORT_STARTED) + return 0; + + return 1; +} + +static int +port_is_closed(portid_t port_id) +{ + if (port_id_is_invalid(port_id, ENABLED_WARN)) + return 0; + + if (ports[port_id].port_status != RTE_PORT_CLOSED) + return 0; + + return 1; +} + +int start_port(portid_t pid) { int diag, need_check_link_status = 0; @@ -1296,8 +1347,8 @@ start_port(portid_t pid) if(dcb_config) dcb_test = 1; - for (pi = 0; pi < nb_ports; pi++) { - if (pid < nb_ports && pid != pi) + FOREACH_PORT(pi, ports) { + if (!port_id_is_invalid(pid, DISABLED_WARN) && pid != pi) continue; port = &ports[pi]; @@ -1421,7 +1472,7 @@ start_port(portid_t pid) } if (need_check_link_status && !no_link_check) - check_all_ports_link_status(nb_ports, RTE_PORT_ALL); + check_all_ports_link_status(RTE_PORT_ALL); else printf("Please stop the ports first\n"); @@ -1446,8 +1497,8 @@ stop_port(portid_t pid) } printf("Stopping ports...\n"); - for (pi = 0; pi < nb_ports; pi++) { - if (pid < nb_ports && pid != pi) + FOREACH_PORT(pi, ports) { + if (!port_id_is_invalid(pid, DISABLED_WARN) && pid != pi) continue; port = &ports[pi]; @@ -1463,7 +1514,7 @@ stop_port(portid_t pid) need_check_link_status = 1; } if (need_check_link_status && !no_link_check) - check_all_ports_link_status(nb_ports, RTE_PORT_ALL); + check_all_ports_link_status(RTE_PORT_ALL); printf("Done\n"); } @@ -1481,8 +1532,8 @@ close_port(portid_t pid) printf("Closing ports...\n"); - for (pi = 0; pi < nb_ports; pi++) { - if (pid < nb_ports && pid != pi) + FOREACH_PORT(pi, ports) { + if (!port_id_is_invalid(pid, DISABLED_WARN) && pid != pi) continue; port = &ports[pi]; @@ -1502,31 +1553,83 @@ close_port(portid_t pid) printf("Done\n"); } -int -all_ports_stopped(void) +void +attach_port(char *identifier) { - portid_t pi; - struct rte_port *port; + portid_t i, j, pi = 0; - for (pi = 0; pi < nb_ports; pi++) { - port = &ports[pi]; - if (port->port_status != RTE_PORT_STOPPED) - return 0; + printf("Attaching a new port...\n"); + + if (identifier == NULL) { + printf("Invalid parameters are speficied\n"); + return; } - return 1; + if (test_done == 0) { + printf("Please stop forwarding first\n"); + return; + } + + if (rte_eal_dev_attach(identifier, &pi)) + return; + + ports[pi].enabled = 1; + reconfig(pi, rte_eth_dev_socket_id(pi)); + rte_eth_promiscuous_enable(pi); + + nb_ports = rte_eth_dev_count(); + + /* set_default_fwd_ports_config(); */ + bzero(fwd_ports_ids, sizeof(fwd_ports_ids)); + i = 0; + FOREACH_PORT(j, ports) { + fwd_ports_ids[i] = j; + i++; + } + nb_cfg_ports = nb_ports; + nb_fwd_ports++; + + ports[pi].port_status = RTE_PORT_STOPPED; + + printf("Port %d is attached. Now total ports is %d\n", pi, nb_ports); + printf("Done\n"); } -int -port_is_started(portid_t port_id) +void +detach_port(uint8_t port_id) { - if (port_id_is_invalid(port_id)) - return -1; + portid_t i, pi = 0; + char name[RTE_ETH_NAME_MAX_LEN]; - if (ports[port_id].port_status != RTE_PORT_STARTED) - return 0; + printf("Detaching a port...\n"); - return 1; + if (!port_is_closed(port_id)) { + printf("Please close port first\n"); + return; + } + + rte_eth_promiscuous_disable(port_id); + + if (rte_eal_dev_detach(port_id, name)) + return; + + ports[port_id].enabled = 0; + nb_ports = rte_eth_dev_count(); + + /* set_default_fwd_ports_config(); */ + bzero(fwd_ports_ids, sizeof(fwd_ports_ids)); + i = 0; + FOREACH_PORT(pi, ports) { + fwd_ports_ids[i] = pi; + i++; + } + nb_cfg_ports = nb_ports; + nb_fwd_ports--; + + printf("Port '%s' is detached. Now total ports is %d\n", + name, nb_ports); + printf("Done\n"); + return; } void @@ -1534,7 +1637,7 @@ pmd_test_exit(void) { portid_t pt_id; - for (pt_id = 0; pt_id < nb_ports; pt_id++) { + FOREACH_PORT(pt_id, ports) { printf("Stopping port %d...", pt_id); fflush(stdout); rte_eth_dev_close(pt_id); @@ -1553,7 +1656,7 @@ struct pmd_test_command { /* Check the link status of all ports in up to 9s, and print them finally */ static void -check_all_ports_link_status(uint8_t port_num, uint32_t port_mask) +check_all_ports_link_status(uint32_t port_mask) { #define CHECK_INTERVAL 100 /* 100ms */ #define MAX_CHECK_TIME 90 /* 9s (90 * 100ms) in total */ @@ -1564,7 +1667,7 @@ check_all_ports_link_status(uint8_t port_num, uint32_t port_mask) fflush(stdout); for (count = 0; count <= MAX_CHECK_TIME; count++) { all_ports_up = 1; - for (portid = 0; portid < port_num; portid++) { + FOREACH_PORT(portid, ports) { if ((port_mask & (1 << portid)) == 0) continue; memset(&link, 0, sizeof(link)); @@ -1688,7 +1791,7 @@ init_port_config(void) portid_t pid; struct rte_port *port; - for (pid = 0; pid < nb_ports; pid++) { + FOREACH_PORT(pid, ports) { port = &ports[pid]; port->dev_conf.rxmode = rx_mode; port->dev_conf.fdir_conf = fdir_conf; @@ -1877,7 +1980,7 @@ main(int argc, char** argv) nb_ports = (portid_t) rte_eth_dev_count(); if (nb_ports == 0) - rte_exit(EXIT_FAILURE, "No probed ethernet device\n"); + RTE_LOG(WARNING, EAL, "No probed ethernet devices\n"); set_def_fwd_config(); if (nb_lcores == 0) @@ -1899,7 +2002,7 @@ main(int argc, char** argv) rte_exit(EXIT_FAILURE, "Start ports failed\n"); /* set all ports to promiscuous mode by default */ - for (port_id = 0; port_id < nb_ports; port_id++) + FOREACH_PORT(port_id, ports) rte_eth_promiscuous_enable(port_id); #ifdef RTE_LIBRTE_CMDLINE diff --git a/app/test-pmd/testpmd.h b/app/test-pmd/testpmd.h index 8f5e6c7..109c670 100644 --- a/app/test-pmd/testpmd.h +++ b/app/test-pmd/testpmd.h @@ -134,6 +134,7 @@ struct fwd_stream { * The data structure associated with each port. */ struct rte_port { + uint8_t enabled; /**< Port enabled or not */ struct rte_eth_dev_info dev_info; /**< PCI info + driver name */ struct rte_eth_conf dev_conf; /**< Port configuration. */ struct ether_addr eth_addr; /**< Port ethernet address */ @@ -159,6 +160,14 @@ struct rte_port { struct rte_eth_txconf tx_conf; /**< tx configuration */ }; +extern portid_t __rte_unused +find_next_port(portid_t p, struct rte_port *ports, int size); + +#define FOREACH_PORT(p, ports) \ + for (p = find_next_port(0, ports, RTE_MAX_ETHPORTS); \ + p < RTE_MAX_ETHPORTS; \ + p = find_next_port(p + 1, ports, RTE_MAX_ETHPORTS)) + /** * The data structure associated with each forwarding logical core. * The logical cores are internally numbered by a core index from 0 to @@ -515,6 +524,8 @@ int init_port_dcb_config(portid_t pid,struct dcb_config *dcb_conf); int start_port(portid_t pid); void stop_port(portid_t pid); void close_port(portid_t pid); +void attach_port(char *identifier); +void detach_port(uint8_t port_id); int all_ports_stopped(void); int port_is_started(portid_t port_id); void pmd_test_exit(void); @@ -558,10 +569,15 @@ void get_ethertype_filter(uint8_t port_id, uint16_t index); void get_2tuple_filter(uint8_t port_id, uint16_t index); void get_5tuple_filter(uint8_t port_id, uint16_t index); void get_flex_filter(uint8_t port_id, uint16_t index); -int port_id_is_invalid(portid_t port_id); int rx_queue_id_is_invalid(queueid_t rxq_id); int tx_queue_id_is_invalid(queueid_t txq_id); +enum print_warning { + ENABLED_WARN = 0, + DISABLED_WARN +}; +int port_id_is_invalid(portid_t port_id, enum print_warning warning); + /* * Work-around of a compilation error with ICC on invocations of the * rte_be_to_cpu_16() function. diff --git a/doc/guides/testpmd_app_ug/testpmd_funcs.rst b/doc/guides/testpmd_app_ug/testpmd_funcs.rst index 218835a..1cacbcf 100644 --- a/doc/guides/testpmd_app_ug/testpmd_funcs.rst +++ b/doc/guides/testpmd_app_ug/testpmd_funcs.rst @@ -808,6 +808,63 @@ The following sections show functions for configuring ports. Port configuration changes only become active when forwarding is started/restarted. +port attach +~~~~~~~~~~~ + +Attach a port specified by pci address or virtual device args. + +To attach a new pci device, the device should be recognized by kernel first. +Then it should be moved under DPDK management. +Finally the port can be attached to testpmd. +On the other hand, to attach a port created by virtual device, above steps are not needed. + +port attach (identifier) + +For example, to attach a port that pci address is 0000:02:00.0. + +.. code-block:: console + + testpmd> port attach 0000:02:00.0 + Attaching a new port... + ... snip ... + Port 0 is attached. Now total ports is 1 + Done + +For example, to attach a port created by pcap PMD. + +.. code-block:: console + + testpmd> port attach eth_pcap0,iface=eth0 + Attaching a new port... + ... snip ... + Port 0 is attached. Now total ports is 1 + Done + +In this case, identifier is "eth_pcap0,iface=eth0". +This identifier format is the same as "--vdev" format of DPDK applications. + +port detach +~~~~~~~~~~~ + +Detach a specific port. + +Before detaching a port, the port should be closed. +Also to remove a pci device completely from the system, first detach the port from testpmd. +Then the device should be moved under kernel management. +Finally the device can be remove using kernel pci hotplug functionality. +On the other hand, to remove a port created by virtual device, above steps are not needed. + +port detach (port_id) + +For example, to detach a port 0. + +.. code-block:: console + + testpmd> port detach 0 + Detaching a port... + ... snip ... + Done + port start ~~~~~~~~~~ -- 1.9.1
On 2/1/2015 12:02 PM, Tetsuya Mukawa wrote: > These functions are used for attaching or detaching a port. > When rte_eal_dev_attach() is called, the function tries to realize the > device name as pci address. If this is done successfully, > rte_eal_dev_attach() will attach physical device port. If not, attaches > virtual devive port. > When rte_eal_dev_detach() is called, the function gets the device type > of this port to know whether the port is came from physical or virtual. > And then specific detaching function will be called. > > v5: > - Change function names like below. > rte_eal_dev_find_and_invoke() to rte_eal_vdev_find_and_invoke(). > rte_eal_dev_invoke() to rte_eal_vdev_invoke(). > - Add code to handle a return value of rte_eal_devargs_remove(). > - Fix pci address format in rte_eal_dev_detach(). > v4: > - Fix comment. > - Add error checking. > - Fix indent of 'if' statement. > - Change function name. > > Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp> > --- > lib/librte_eal/common/eal_common_dev.c | 274 ++++++++++++++++++++++++++++++++ > lib/librte_eal/common/eal_private.h | 11 ++ > lib/librte_eal/common/include/rte_dev.h | 33 ++++ > lib/librte_eal/linuxapp/eal/Makefile | 1 + > lib/librte_eal/linuxapp/eal/eal_pci.c | 6 +- > 5 files changed, 322 insertions(+), 3 deletions(-) > > diff --git a/lib/librte_eal/common/eal_common_dev.c b/lib/librte_eal/common/eal_common_dev.c > index eae5656..e3a3f54 100644 > --- a/lib/librte_eal/common/eal_common_dev.c > +++ b/lib/librte_eal/common/eal_common_dev.c > @@ -32,10 +32,13 @@ > * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. > */ > > +#include <stdio.h> > +#include <limits.h> > #include <string.h> > #include <inttypes.h> > #include <sys/queue.h> > > +#include <rte_ethdev.h> > #include <rte_dev.h> > #include <rte_devargs.h> > #include <rte_debug.h> > @@ -107,3 +110,274 @@ rte_eal_dev_init(void) > } > return 0; > } > + > +/* So far, DPDK hotplug function only supports linux */ > +#ifdef ENABLE_HOTPLUG > +static void > +rte_eal_vdev_invoke(struct rte_driver *driver, > + struct rte_devargs *devargs, enum rte_eal_invoke_type type) > +{ > + if ((driver == NULL) || (devargs == NULL)) > + return; > + > + switch (type) { > + case RTE_EAL_INVOKE_TYPE_PROBE: > + driver->init(devargs->virtual.drv_name, devargs->args); > + break; > + case RTE_EAL_INVOKE_TYPE_CLOSE: > + driver->uninit(devargs->virtual.drv_name, devargs->args); > + break; > + default: > + break; > + } > +} > + > +static int > +rte_eal_vdev_find_and_invoke(const char *name, int type) > +{ > + struct rte_devargs *devargs; > + struct rte_driver *driver; > + > + if (name == NULL) > + return -EINVAL; > + > + /* call the init function for each virtual device */ > + TAILQ_FOREACH(devargs, &devargs_list, next) { > + > + if (devargs->type != RTE_DEVTYPE_VIRTUAL) > + continue; > + > + if (strncmp(name, devargs->virtual.drv_name, strlen(name))) > + continue; > + > + TAILQ_FOREACH(driver, &dev_driver_list, next) { > + if (driver->type != PMD_VDEV) > + continue; > + > + /* search a driver prefix in virtual device name */ > + if (!strncmp(driver->name, devargs->virtual.drv_name, > + strlen(driver->name))) { > + rte_eal_vdev_invoke(driver, devargs, type); > + break; > + } > + } > + > + if (driver == NULL) { > + RTE_LOG(WARNING, EAL, "no driver found for %s\n", > + devargs->virtual.drv_name); > + } > + return 0; > + } > + return 1; > +} > + > +/* attach the new physical device, then store port_id of the device */ > +static int > +rte_eal_dev_attach_pdev(struct rte_pci_addr *addr, uint8_t *port_id) > +{ > + uint8_t new_port_id; > + struct rte_eth_dev devs[RTE_MAX_ETHPORTS]; > + > + if ((addr == NULL) || (port_id == NULL)) > + goto err; > + > + /* save current port status */ > + rte_eth_dev_save(devs); > + /* re-construct pci_device_list */ > + if (rte_eal_pci_scan()) > + goto err; > + /* invoke probe func of the driver can handle the new device */ > + if (rte_eal_pci_probe_one(addr)) > + goto err; > + /* get port_id enabled by above procedures */ > + if (rte_eth_dev_get_changed_port(devs, &new_port_id)) > + goto err; > + > + *port_id = new_port_id; > + return 0; > +err: > + RTE_LOG(ERR, EAL, "Drver, cannot attach the device\n"); Sorry, what does "Drver" means? My English is bad, also I haven't gotten this work in google Thanks, Michael > + return -1; > +} > + > +/* detach the new physical device, then store pci_addr of the device */ > +static int > +rte_eal_dev_detach_pdev(uint8_t port_id, struct rte_pci_addr *addr) > +{ > + struct rte_pci_addr freed_addr; > + struct rte_pci_addr vp; > + > + if (addr == NULL) > + goto err; > + > + /* check whether the driver supports detach feature, or not */ > + if (rte_eth_dev_check_detachable(port_id)) > + goto err; > + > + /* get pci address by port id */ > + if (rte_eth_dev_get_addr_by_port(port_id, &freed_addr)) > + goto err; > + > + /* Zerod pci addr means the port comes from virtual device */ > + vp.domain = vp.bus = vp.devid = vp.function = 0; > + if (eal_compare_pci_addr(&vp, &freed_addr) == 0) > + goto err; > + > + /* invoke close func of the driver, > + * also remove the device from pci_device_list */ > + if (rte_eal_pci_close_one(&freed_addr)) > + goto err; > + > + *addr = freed_addr; > + return 0; > +err: > + RTE_LOG(ERR, EAL, "Drver, cannot detach the device\n"); Here, "Drver" > + return -1; > +} > + > +static void > +get_vdev_name(char *vdevargs) > +{ > + char *sep; > + > + if (vdevargs == NULL) > + return; > + > + /* set the first ',' to '\0' to split name and arguments */ > + sep = strchr(vdevargs, ','); > + if (sep != NULL) > + sep[0] = '\0'; > +} > + > +/* attach the new virtual device, then store port_id of the device */ > +static int > +rte_eal_dev_attach_vdev(const char *vdevargs, uint8_t *port_id) > +{ > + char *args; > + uint8_t new_port_id; > + struct rte_eth_dev devs[RTE_MAX_ETHPORTS]; > + > + if ((vdevargs == NULL) || (port_id == NULL)) > + goto err0; > + > + args = strdup(vdevargs); > + if (args == NULL) > + goto err0; > + > + /* save current port status */ > + rte_eth_dev_save(devs); > + /* add the vdevargs to devargs_list */ > + if (rte_eal_devargs_add(RTE_DEVTYPE_VIRTUAL, args)) > + goto err1; > + /* parse vdevargs, then retrieve device name */ > + get_vdev_name(args); > + /* walk around dev_driver_list to find the driver of the device, > + * then invoke probe function o the driver */ > + if (rte_eal_vdev_find_and_invoke(args, RTE_EAL_INVOKE_TYPE_PROBE)) > + goto err2; > + /* get port_id enabled by above procedures */ > + if (rte_eth_dev_get_changed_port(devs, &new_port_id)) > + goto err2; > + > + free(args); > + *port_id = new_port_id; > + return 0; > +err2: > + rte_eal_devargs_remove(RTE_DEVTYPE_VIRTUAL, args); > +err1: > + free(args); > +err0: > + RTE_LOG(ERR, EAL, "Drver, cannot detach the device\n"); Here also "Drver", Thanks, Michael > + return -1; > +} > + > +/* detach the new virtual device, then store the name of the device */ > +static int > +rte_eal_dev_detach_vdev(uint8_t port_id, char *vdevname) > +{ > + char name[RTE_ETH_NAME_MAX_LEN]; > + > + if (vdevname == NULL) > + goto err; > + > + /* check whether the driver supports detach feature, or not */ > + if (rte_eth_dev_check_detachable(port_id)) > + goto err; > + > + /* get device name by port id */ > + if (rte_eth_dev_get_name_by_port(port_id, name)) > + goto err; > + /* walk around dev_driver_list to find the driver of the device, > + * then invoke close function o the driver */ > + if (rte_eal_vdev_find_and_invoke(name, RTE_EAL_INVOKE_TYPE_CLOSE)) > + goto err; > + /* remove the vdevname from devargs_list */ > + if (rte_eal_devargs_remove(RTE_DEVTYPE_VIRTUAL, name)) > + goto err; > + > + strncpy(vdevname, name, sizeof(name)); > + return 0; > +err: > + RTE_LOG(ERR, EAL, "Drver, cannot detach the device\n"); > + return -1; > +} > + > +/* attach the new device, then store port_id of the device */ > +int > +rte_eal_dev_attach(const char *devargs, uint8_t *port_id) > +{ > + struct rte_pci_addr addr; > + > + if ((devargs == NULL) || (port_id == NULL)) > + return -EINVAL; > + > + if (eal_parse_pci_DomBDF(devargs, &addr) == 0) > + return rte_eal_dev_attach_pdev(&addr, port_id); > + else > + return rte_eal_dev_attach_vdev(devargs, port_id); > +} > + > +/* detach the device, then store the name of the device */ > +int > +rte_eal_dev_detach(uint8_t port_id, char *name) > +{ > + struct rte_pci_addr addr; > + int ret; > + > + if (name == NULL) > + return -EINVAL; > + > + if (rte_eth_dev_get_device_type(port_id) == RTE_ETH_DEV_PHYSICAL) { > + ret = rte_eth_dev_get_addr_by_port(port_id, &addr); > + if (ret < 0) > + return ret; > + > + ret = rte_eal_dev_detach_pdev(port_id, &addr); > + if (ret == 0) > + snprintf(name, RTE_ETH_NAME_MAX_LEN, > + "%04x:%02x:%02x.%d", > + addr.domain, addr.bus, > + addr.devid, addr.function); > + > + return ret; > + } else > + return rte_eal_dev_detach_vdev(port_id, name); > +} > +#else /* ENABLE_HOTPLUG */ > +int > +rte_eal_dev_attach(const char *devargs __rte_unused, > + uint8_t *port_id __rte_unused) > +{ > + RTE_LOG(ERR, EAL, "Hotplug support isn't enabled\n"); > + return -1; > +} > + > +/* detach the device, then store the name of the device */ > +int > +rte_eal_dev_detach(uint8_t port_id __rte_unused, > + char *name __rte_unused) > +{ > + RTE_LOG(ERR, EAL, "Hotplug support isn't enabled\n"); > + return -1; > +} > +#endif /* ENABLE_HOTPLUG */ > diff --git a/lib/librte_eal/common/eal_private.h b/lib/librte_eal/common/eal_private.h > index 1a362ab..8168a7a 100644 > --- a/lib/librte_eal/common/eal_private.h > +++ b/lib/librte_eal/common/eal_private.h > @@ -163,6 +163,17 @@ enum rte_eal_invoke_type { > }; > > /** > + * Scan the content of the PCI bus, and the devices in the devices > + * list > + * > + * This function is private to EAL. > + * > + * @return > + * 0 on success, negative on error > + */ > +int rte_eal_pci_scan(void); > + > +/** > * Mmap memory for single PCI device > * > * This function is private to EAL. > diff --git a/lib/librte_eal/common/include/rte_dev.h b/lib/librte_eal/common/include/rte_dev.h > index f7e3a10..e63dd1c 100644 > --- a/lib/librte_eal/common/include/rte_dev.h > +++ b/lib/librte_eal/common/include/rte_dev.h > @@ -47,6 +47,7 @@ extern "C" { > #endif > > #include <sys/queue.h> > +#include <rte_pci.h> > > /** Double linked list of device drivers. */ > TAILQ_HEAD(rte_driver_list, rte_driver); > @@ -57,6 +58,11 @@ TAILQ_HEAD(rte_driver_list, rte_driver); > typedef int (rte_dev_init_t)(const char *name, const char *args); > > /** > + * Uninitilization function called for each device driver once. > + */ > +typedef int (rte_dev_uninit_t)(const char *name, const char *args); > + > +/** > * Driver type enumeration > */ > enum pmd_type { > @@ -72,6 +78,7 @@ struct rte_driver { > enum pmd_type type; /**< PMD Driver type */ > const char *name; /**< Driver name. */ > rte_dev_init_t *init; /**< Device init. function. */ > + rte_dev_uninit_t *uninit; /**< Device uninit. function. */ > }; > > /** > @@ -93,6 +100,32 @@ void rte_eal_driver_register(struct rte_driver *driver); > void rte_eal_driver_unregister(struct rte_driver *driver); > > /** > + * Attach a new device. > + * > + * @param devargs > + * A pointer to a strings array describing the new device > + * to be attached. The strings should be a pci address like > + * '0000:01:00.0' or virtual device name like 'eth_pcap0'. > + * @param port_id > + * A pointer to a port identifier actually attached. > + * @return > + * 0 on success and port_id is filled, negative on error > + */ > +int rte_eal_dev_attach(const char *devargs, uint8_t *port_id); > + > +/** > + * Detach a device. > + * > + * @param port_id > + * The port identifier of the device to detach. > + * @param addr > + * A pointer to a device name actually detached. > + * @return > + * 0 on success and devname is filled, negative on error > + */ > +int rte_eal_dev_detach(uint8_t port_id, char *devname); > + > +/** > * Initalize all the registered drivers in this process > */ > int rte_eal_dev_init(void); > diff --git a/lib/librte_eal/linuxapp/eal/Makefile b/lib/librte_eal/linuxapp/eal/Makefile > index 72ecf3a..0ec83b5 100644 > --- a/lib/librte_eal/linuxapp/eal/Makefile > +++ b/lib/librte_eal/linuxapp/eal/Makefile > @@ -41,6 +41,7 @@ CFLAGS += -I$(RTE_SDK)/lib/librte_eal/common/include > CFLAGS += -I$(RTE_SDK)/lib/librte_ring > CFLAGS += -I$(RTE_SDK)/lib/librte_mempool > CFLAGS += -I$(RTE_SDK)/lib/librte_malloc > +CFLAGS += -I$(RTE_SDK)/lib/librte_mbuf > CFLAGS += -I$(RTE_SDK)/lib/librte_ether > CFLAGS += -I$(RTE_SDK)/lib/librte_ivshmem > CFLAGS += -I$(RTE_SDK)/lib/librte_pmd_ring > diff --git a/lib/librte_eal/linuxapp/eal/eal_pci.c b/lib/librte_eal/linuxapp/eal/eal_pci.c > index 831422e..1f43688 100644 > --- a/lib/librte_eal/linuxapp/eal/eal_pci.c > +++ b/lib/librte_eal/linuxapp/eal/eal_pci.c > @@ -431,8 +431,8 @@ error: > * Scan the content of the PCI bus, and the devices in the devices > * list > */ > -static int > -pci_scan(void) > +int > +rte_eal_pci_scan(void) > { > struct dirent *e; > DIR *dir; > @@ -764,7 +764,7 @@ rte_eal_pci_init(void) > if (internal_config.no_pci) > return 0; > > - if (pci_scan() < 0) { > + if (rte_eal_pci_scan() < 0) { > RTE_LOG(ERR, EAL, "%s(): Cannot scan PCI bus\n", __func__); > return -1; > }
On 2/2/2015 1:43 PM, Qiu, Michael wrote: > On 2/1/2015 12:02 PM, Tetsuya Mukawa wrote: >> These functions are used for attaching or detaching a port. >> When rte_eal_dev_attach() is called, the function tries to realize the >> device name as pci address. If this is done successfully, >> rte_eal_dev_attach() will attach physical device port. If not, attaches >> virtual devive port. >> When rte_eal_dev_detach() is called, the function gets the device type >> of this port to know whether the port is came from physical or virtual. >> And then specific detaching function will be called. >> >> v5: >> - Change function names like below. >> rte_eal_dev_find_and_invoke() to rte_eal_vdev_find_and_invoke(). >> rte_eal_dev_invoke() to rte_eal_vdev_invoke(). >> - Add code to handle a return value of rte_eal_devargs_remove(). >> - Fix pci address format in rte_eal_dev_detach(). >> v4: >> - Fix comment. >> - Add error checking. >> - Fix indent of 'if' statement. >> - Change function name. >> [...] >> +/* attach the new virtual device, then store port_id of the device */ >> +static int >> +rte_eal_dev_attach_vdev(const char *vdevargs, uint8_t *port_id) >> +{ >> + char *args; >> + uint8_t new_port_id; >> + struct rte_eth_dev devs[RTE_MAX_ETHPORTS]; >> + >> + if ((vdevargs == NULL) || (port_id == NULL)) >> + goto err0; >> + >> + args = strdup(vdevargs); >> + if (args == NULL) >> + goto err0; >> + >> + /* save current port status */ >> + rte_eth_dev_save(devs); >> + /* add the vdevargs to devargs_list */ >> + if (rte_eal_devargs_add(RTE_DEVTYPE_VIRTUAL, args)) >> + goto err1; >> + /* parse vdevargs, then retrieve device name */ >> + get_vdev_name(args); >> + /* walk around dev_driver_list to find the driver of the device, >> + * then invoke probe function o the driver */ >> + if (rte_eal_vdev_find_and_invoke(args, RTE_EAL_INVOKE_TYPE_PROBE)) >> + goto err2; >> + /* get port_id enabled by above procedures */ >> + if (rte_eth_dev_get_changed_port(devs, &new_port_id)) >> + goto err2; >> + >> + free(args); >> + *port_id = new_port_id; >> + return 0; >> +err2: >> + rte_eal_devargs_remove(RTE_DEVTYPE_VIRTUAL, args); >> +err1: >> + free(args); >> +err0: >> + RTE_LOG(ERR, EAL, "Drver, cannot detach the device\n"); Here "cannot detach the device\n" should be "cannot attach the device" I think. > Here also "Drver", > > > Thanks, > Michael >> + return -1; >> +} >> + >> +/* detach the new virtual device, then store the name of the device */ >> +static int >> +rte_eal_dev_detach_vdev(uint8_t port_id, char *vdevname) >> +{ >> + char name[RTE_ETH_NAME_MAX_LEN]; >> + >> + if (vdevname == NULL) >> + goto err; >> + >> + /* check whether the driver supports detach feature, or not */ >> + if (rte_eth_dev_check_detachable(port_id)) >> + goto err; >> + >> + /* get device name by port id */ >> + if (rte_eth_dev_get_name_by_port(port_id, name)) >> + goto err; >> + /* walk around dev_driver_list to find the driver of the device, >> + * then invoke close function o the driver */ >> + if (rte_eal_vdev_find_and_invoke(name, RTE_EAL_INVOKE_TYPE_CLOSE)) >> + goto err; >> + /* remove the vdevname from devargs_list */ >> + if (rte_eal_devargs_remove(RTE_DEVTYPE_VIRTUAL, name)) >> + goto err; >> + >> + strncpy(vdevname, name, sizeof(name)); >> + return 0; >> +err: >> + RTE_LOG(ERR, EAL, "Drver, cannot detach the device\n"); >> + return -1; >> +} >> + >> +/* attach the new device, then store port_id of the device */ >> +int >> +rte_eal_dev_attach(const char *devargs, uint8_t *port_id) >> +{ >> + struct rte_pci_addr addr; >> + >> + if ((devargs == NULL) || (port_id == NULL)) >> + return -EINVAL; >> + >> + if (eal_parse_pci_DomBDF(devargs, &addr) == 0) >> + return rte_eal_dev_attach_pdev(&addr, port_id); >> + else >> + return rte_eal_dev_attach_vdev(devargs, port_id); >> +} >> + >> +/* detach the device, then store the name of the device */ >> +int >> +rte_eal_dev_detach(uint8_t port_id, char *name) >> +{ >> + struct rte_pci_addr addr; >> + int ret; >> + >> + if (name == NULL) >> + return -EINVAL; >> + >> + if (rte_eth_dev_get_device_type(port_id) == RTE_ETH_DEV_PHYSICAL) { >> + ret = rte_eth_dev_get_addr_by_port(port_id, &addr); >> + if (ret < 0) >> + return ret; >> + >> + ret = rte_eal_dev_detach_pdev(port_id, &addr); >> + if (ret == 0) >> + snprintf(name, RTE_ETH_NAME_MAX_LEN, >> + "%04x:%02x:%02x.%d", >> + addr.domain, addr.bus, >> + addr.devid, addr.function); >> + >> + return ret; >> + } else >> + return rte_eal_dev_detach_vdev(port_id, name); >> +} >> +#else /* ENABLE_HOTPLUG */ >> +int >> +rte_eal_dev_attach(const char *devargs __rte_unused, >> + uint8_t *port_id __rte_unused) >> +{ >> + RTE_LOG(ERR, EAL, "Hotplug support isn't enabled\n"); >> + return -1; >> +} >> + >> +/* detach the device, then store the name of the device */ >> +int >> +rte_eal_dev_detach(uint8_t port_id __rte_unused, >> + char *name __rte_unused) >> +{ >> + RTE_LOG(ERR, EAL, "Hotplug support isn't enabled\n"); >> + return -1; >> +} >> +#endif /* ENABLE_HOTPLUG */ >> diff --git a/lib/librte_eal/common/eal_private.h b/lib/librte_eal/common/eal_private.h >> index 1a362ab..8168a7a 100644 >> --- a/lib/librte_eal/common/eal_private.h >> +++ b/lib/librte_eal/common/eal_private.h >> @@ -163,6 +163,17 @@ enum rte_eal_invoke_type { >> }; >> >> /** >> + * Scan the content of the PCI bus, and the devices in the devices >> + * list >> + * >> + * This function is private to EAL. >> + * >> + * @return >> + * 0 on success, negative on error >> + */ >> +int rte_eal_pci_scan(void); >> + >> +/** >> * Mmap memory for single PCI device >> * >> * This function is private to EAL. >> diff --git a/lib/librte_eal/common/include/rte_dev.h b/lib/librte_eal/common/include/rte_dev.h >> index f7e3a10..e63dd1c 100644 >> --- a/lib/librte_eal/common/include/rte_dev.h >> +++ b/lib/librte_eal/common/include/rte_dev.h >> @@ -47,6 +47,7 @@ extern "C" { >> #endif >> >> #include <sys/queue.h> >> +#include <rte_pci.h> >> >> /** Double linked list of device drivers. */ >> TAILQ_HEAD(rte_driver_list, rte_driver); >> @@ -57,6 +58,11 @@ TAILQ_HEAD(rte_driver_list, rte_driver); >> typedef int (rte_dev_init_t)(const char *name, const char *args); >> >> /** >> + * Uninitilization function called for each device driver once. >> + */ >> +typedef int (rte_dev_uninit_t)(const char *name, const char *args); >> + >> +/** >> * Driver type enumeration >> */ >> enum pmd_type { >> @@ -72,6 +78,7 @@ struct rte_driver { >> enum pmd_type type; /**< PMD Driver type */ >> const char *name; /**< Driver name. */ >> rte_dev_init_t *init; /**< Device init. function. */ >> + rte_dev_uninit_t *uninit; /**< Device uninit. function. */ >> }; >> >> /** >> @@ -93,6 +100,32 @@ void rte_eal_driver_register(struct rte_driver *driver); >> void rte_eal_driver_unregister(struct rte_driver *driver); >> >> /** >> + * Attach a new device. >> + * >> + * @param devargs >> + * A pointer to a strings array describing the new device >> + * to be attached. The strings should be a pci address like >> + * '0000:01:00.0' or virtual device name like 'eth_pcap0'. >> + * @param port_id >> + * A pointer to a port identifier actually attached. >> + * @return >> + * 0 on success and port_id is filled, negative on error >> + */ >> +int rte_eal_dev_attach(const char *devargs, uint8_t *port_id); >> + >> +/** >> + * Detach a device. >> + * >> + * @param port_id >> + * The port identifier of the device to detach. >> + * @param addr >> + * A pointer to a device name actually detached. >> + * @return >> + * 0 on success and devname is filled, negative on error >> + */ >> +int rte_eal_dev_detach(uint8_t port_id, char *devname); >> + >> +/** >> * Initalize all the registered drivers in this process >> */ >> int rte_eal_dev_init(void); >> diff --git a/lib/librte_eal/linuxapp/eal/Makefile b/lib/librte_eal/linuxapp/eal/Makefile >> index 72ecf3a..0ec83b5 100644 >> --- a/lib/librte_eal/linuxapp/eal/Makefile >> +++ b/lib/librte_eal/linuxapp/eal/Makefile >> @@ -41,6 +41,7 @@ CFLAGS += -I$(RTE_SDK)/lib/librte_eal/common/include >> CFLAGS += -I$(RTE_SDK)/lib/librte_ring >> CFLAGS += -I$(RTE_SDK)/lib/librte_mempool >> CFLAGS += -I$(RTE_SDK)/lib/librte_malloc >> +CFLAGS += -I$(RTE_SDK)/lib/librte_mbuf >> CFLAGS += -I$(RTE_SDK)/lib/librte_ether >> CFLAGS += -I$(RTE_SDK)/lib/librte_ivshmem >> CFLAGS += -I$(RTE_SDK)/lib/librte_pmd_ring >> diff --git a/lib/librte_eal/linuxapp/eal/eal_pci.c b/lib/librte_eal/linuxapp/eal/eal_pci.c >> index 831422e..1f43688 100644 >> --- a/lib/librte_eal/linuxapp/eal/eal_pci.c >> +++ b/lib/librte_eal/linuxapp/eal/eal_pci.c >> @@ -431,8 +431,8 @@ error: >> * Scan the content of the PCI bus, and the devices in the devices >> * list >> */ >> -static int >> -pci_scan(void) >> +int >> +rte_eal_pci_scan(void) >> { >> struct dirent *e; >> DIR *dir; >> @@ -764,7 +764,7 @@ rte_eal_pci_init(void) >> if (internal_config.no_pci) >> return 0; >> >> - if (pci_scan() < 0) { >> + if (rte_eal_pci_scan() < 0) { >> RTE_LOG(ERR, EAL, "%s(): Cannot scan PCI bus\n", __func__); >> return -1; >> } >
On 2015/02/02 15:22, Qiu, Michael wrote: > On 2/2/2015 1:43 PM, Qiu, Michael wrote: >> On 2/1/2015 12:02 PM, Tetsuya Mukawa wrote: >>> These functions are used for attaching or detaching a port. >>> When rte_eal_dev_attach() is called, the function tries to realize the >>> device name as pci address. If this is done successfully, >>> rte_eal_dev_attach() will attach physical device port. If not, attaches >>> virtual devive port. >>> When rte_eal_dev_detach() is called, the function gets the device type >>> of this port to know whether the port is came from physical or virtual. >>> And then specific detaching function will be called. >>> >>> v5: >>> - Change function names like below. >>> rte_eal_dev_find_and_invoke() to rte_eal_vdev_find_and_invoke(). >>> rte_eal_dev_invoke() to rte_eal_vdev_invoke(). >>> - Add code to handle a return value of rte_eal_devargs_remove(). >>> - Fix pci address format in rte_eal_dev_detach(). >>> v4: >>> - Fix comment. >>> - Add error checking. >>> - Fix indent of 'if' statement. >>> - Change function name. >>> > [...] > >>> +/* attach the new virtual device, then store port_id of the device */ >>> +static int >>> +rte_eal_dev_attach_vdev(const char *vdevargs, uint8_t *port_id) >>> +{ >>> + char *args; >>> + uint8_t new_port_id; >>> + struct rte_eth_dev devs[RTE_MAX_ETHPORTS]; >>> + >>> + if ((vdevargs == NULL) || (port_id == NULL)) >>> + goto err0; >>> + >>> + args = strdup(vdevargs); >>> + if (args == NULL) >>> + goto err0; >>> + >>> + /* save current port status */ >>> + rte_eth_dev_save(devs); >>> + /* add the vdevargs to devargs_list */ >>> + if (rte_eal_devargs_add(RTE_DEVTYPE_VIRTUAL, args)) >>> + goto err1; >>> + /* parse vdevargs, then retrieve device name */ >>> + get_vdev_name(args); >>> + /* walk around dev_driver_list to find the driver of the device, >>> + * then invoke probe function o the driver */ >>> + if (rte_eal_vdev_find_and_invoke(args, RTE_EAL_INVOKE_TYPE_PROBE)) >>> + goto err2; >>> + /* get port_id enabled by above procedures */ >>> + if (rte_eth_dev_get_changed_port(devs, &new_port_id)) >>> + goto err2; >>> + >>> + free(args); >>> + *port_id = new_port_id; >>> + return 0; >>> +err2: >>> + rte_eal_devargs_remove(RTE_DEVTYPE_VIRTUAL, args); >>> +err1: >>> + free(args); >>> +err0: >>> + RTE_LOG(ERR, EAL, "Drver, cannot detach the device\n"); > Here "cannot detach the device\n" should be "cannot attach the device" I > think. Hi Michael, Thanks, I will fix above error message. Also I will fix my "Drver" typos. Tetsuya >> Here also "Drver", >> >> >> Thanks, >> Michael >>> + return -1; >>> +} >>> + >>> +/* detach the new virtual device, then store the name of the device */ >>> +static int >>> +rte_eal_dev_detach_vdev(uint8_t port_id, char *vdevname) >>> +{ >>> + char name[RTE_ETH_NAME_MAX_LEN]; >>> + >>> + if (vdevname == NULL) >>> + goto err; >>> + >>> + /* check whether the driver supports detach feature, or not */ >>> + if (rte_eth_dev_check_detachable(port_id)) >>> + goto err; >>> + >>> + /* get device name by port id */ >>> + if (rte_eth_dev_get_name_by_port(port_id, name)) >>> + goto err; >>> + /* walk around dev_driver_list to find the driver of the device, >>> + * then invoke close function o the driver */ >>> + if (rte_eal_vdev_find_and_invoke(name, RTE_EAL_INVOKE_TYPE_CLOSE)) >>> + goto err; >>> + /* remove the vdevname from devargs_list */ >>> + if (rte_eal_devargs_remove(RTE_DEVTYPE_VIRTUAL, name)) >>> + goto err; >>> + >>> + strncpy(vdevname, name, sizeof(name)); >>> + return 0; >>> +err: >>> + RTE_LOG(ERR, EAL, "Drver, cannot detach the device\n"); >>> + return -1; >>> +} >>> + >>> +/* attach the new device, then store port_id of the device */ >>> +int >>> +rte_eal_dev_attach(const char *devargs, uint8_t *port_id) >>> +{ >>> + struct rte_pci_addr addr; >>> + >>> + if ((devargs == NULL) || (port_id == NULL)) >>> + return -EINVAL; >>> + >>> + if (eal_parse_pci_DomBDF(devargs, &addr) == 0) >>> + return rte_eal_dev_attach_pdev(&addr, port_id); >>> + else >>> + return rte_eal_dev_attach_vdev(devargs, port_id); >>> +} >>> + >>> +/* detach the device, then store the name of the device */ >>> +int >>> +rte_eal_dev_detach(uint8_t port_id, char *name) >>> +{ >>> + struct rte_pci_addr addr; >>> + int ret; >>> + >>> + if (name == NULL) >>> + return -EINVAL; >>> + >>> + if (rte_eth_dev_get_device_type(port_id) == RTE_ETH_DEV_PHYSICAL) { >>> + ret = rte_eth_dev_get_addr_by_port(port_id, &addr); >>> + if (ret < 0) >>> + return ret; >>> + >>> + ret = rte_eal_dev_detach_pdev(port_id, &addr); >>> + if (ret == 0) >>> + snprintf(name, RTE_ETH_NAME_MAX_LEN, >>> + "%04x:%02x:%02x.%d", >>> + addr.domain, addr.bus, >>> + addr.devid, addr.function); >>> + >>> + return ret; >>> + } else >>> + return rte_eal_dev_detach_vdev(port_id, name); >>> +} >>> +#else /* ENABLE_HOTPLUG */ >>> +int >>> +rte_eal_dev_attach(const char *devargs __rte_unused, >>> + uint8_t *port_id __rte_unused) >>> +{ >>> + RTE_LOG(ERR, EAL, "Hotplug support isn't enabled\n"); >>> + return -1; >>> +} >>> + >>> +/* detach the device, then store the name of the device */ >>> +int >>> +rte_eal_dev_detach(uint8_t port_id __rte_unused, >>> + char *name __rte_unused) >>> +{ >>> + RTE_LOG(ERR, EAL, "Hotplug support isn't enabled\n"); >>> + return -1; >>> +} >>> +#endif /* ENABLE_HOTPLUG */ >>> diff --git a/lib/librte_eal/common/eal_private.h b/lib/librte_eal/common/eal_private.h >>> index 1a362ab..8168a7a 100644 >>> --- a/lib/librte_eal/common/eal_private.h >>> +++ b/lib/librte_eal/common/eal_private.h >>> @@ -163,6 +163,17 @@ enum rte_eal_invoke_type { >>> }; >>> >>> /** >>> + * Scan the content of the PCI bus, and the devices in the devices >>> + * list >>> + * >>> + * This function is private to EAL. >>> + * >>> + * @return >>> + * 0 on success, negative on error >>> + */ >>> +int rte_eal_pci_scan(void); >>> + >>> +/** >>> * Mmap memory for single PCI device >>> * >>> * This function is private to EAL. >>> diff --git a/lib/librte_eal/common/include/rte_dev.h b/lib/librte_eal/common/include/rte_dev.h >>> index f7e3a10..e63dd1c 100644 >>> --- a/lib/librte_eal/common/include/rte_dev.h >>> +++ b/lib/librte_eal/common/include/rte_dev.h >>> @@ -47,6 +47,7 @@ extern "C" { >>> #endif >>> >>> #include <sys/queue.h> >>> +#include <rte_pci.h> >>> >>> /** Double linked list of device drivers. */ >>> TAILQ_HEAD(rte_driver_list, rte_driver); >>> @@ -57,6 +58,11 @@ TAILQ_HEAD(rte_driver_list, rte_driver); >>> typedef int (rte_dev_init_t)(const char *name, const char *args); >>> >>> /** >>> + * Uninitilization function called for each device driver once. >>> + */ >>> +typedef int (rte_dev_uninit_t)(const char *name, const char *args); >>> + >>> +/** >>> * Driver type enumeration >>> */ >>> enum pmd_type { >>> @@ -72,6 +78,7 @@ struct rte_driver { >>> enum pmd_type type; /**< PMD Driver type */ >>> const char *name; /**< Driver name. */ >>> rte_dev_init_t *init; /**< Device init. function. */ >>> + rte_dev_uninit_t *uninit; /**< Device uninit. function. */ >>> }; >>> >>> /** >>> @@ -93,6 +100,32 @@ void rte_eal_driver_register(struct rte_driver *driver); >>> void rte_eal_driver_unregister(struct rte_driver *driver); >>> >>> /** >>> + * Attach a new device. >>> + * >>> + * @param devargs >>> + * A pointer to a strings array describing the new device >>> + * to be attached. The strings should be a pci address like >>> + * '0000:01:00.0' or virtual device name like 'eth_pcap0'. >>> + * @param port_id >>> + * A pointer to a port identifier actually attached. >>> + * @return >>> + * 0 on success and port_id is filled, negative on error >>> + */ >>> +int rte_eal_dev_attach(const char *devargs, uint8_t *port_id); >>> + >>> +/** >>> + * Detach a device. >>> + * >>> + * @param port_id >>> + * The port identifier of the device to detach. >>> + * @param addr >>> + * A pointer to a device name actually detached. >>> + * @return >>> + * 0 on success and devname is filled, negative on error >>> + */ >>> +int rte_eal_dev_detach(uint8_t port_id, char *devname); >>> + >>> +/** >>> * Initalize all the registered drivers in this process >>> */ >>> int rte_eal_dev_init(void); >>> diff --git a/lib/librte_eal/linuxapp/eal/Makefile b/lib/librte_eal/linuxapp/eal/Makefile >>> index 72ecf3a..0ec83b5 100644 >>> --- a/lib/librte_eal/linuxapp/eal/Makefile >>> +++ b/lib/librte_eal/linuxapp/eal/Makefile >>> @@ -41,6 +41,7 @@ CFLAGS += -I$(RTE_SDK)/lib/librte_eal/common/include >>> CFLAGS += -I$(RTE_SDK)/lib/librte_ring >>> CFLAGS += -I$(RTE_SDK)/lib/librte_mempool >>> CFLAGS += -I$(RTE_SDK)/lib/librte_malloc >>> +CFLAGS += -I$(RTE_SDK)/lib/librte_mbuf >>> CFLAGS += -I$(RTE_SDK)/lib/librte_ether >>> CFLAGS += -I$(RTE_SDK)/lib/librte_ivshmem >>> CFLAGS += -I$(RTE_SDK)/lib/librte_pmd_ring >>> diff --git a/lib/librte_eal/linuxapp/eal/eal_pci.c b/lib/librte_eal/linuxapp/eal/eal_pci.c >>> index 831422e..1f43688 100644 >>> --- a/lib/librte_eal/linuxapp/eal/eal_pci.c >>> +++ b/lib/librte_eal/linuxapp/eal/eal_pci.c >>> @@ -431,8 +431,8 @@ error: >>> * Scan the content of the PCI bus, and the devices in the devices >>> * list >>> */ >>> -static int >>> -pci_scan(void) >>> +int >>> +rte_eal_pci_scan(void) >>> { >>> struct dirent *e; >>> DIR *dir; >>> @@ -764,7 +764,7 @@ rte_eal_pci_init(void) >>> if (internal_config.no_pci) >>> return 0; >>> >>> - if (pci_scan() < 0) { >>> + if (rte_eal_pci_scan() < 0) { >>> RTE_LOG(ERR, EAL, "%s(): Cannot scan PCI bus\n", __func__); >>> return -1; >>> }
On 2015/02/02 20:33, Iremonger, Bernard wrote >> /* >> * Work-around of a compilation error with ICC on invocations of the >> * rte_be_to_cpu_16() function. >> diff --git a/doc/guides/testpmd_app_ug/testpmd_funcs.rst >> b/doc/guides/testpmd_app_ug/testpmd_funcs.rst >> index 218835a..1cacbcf 100644 >> --- a/doc/guides/testpmd_app_ug/testpmd_funcs.rst >> +++ b/doc/guides/testpmd_app_ug/testpmd_funcs.rst > Hi Tetsuya, > > The doc changes should be in separate commit using the "doc: explaination" commit line. > >> @@ -808,6 +808,63 @@ The following sections show functions for configuring ports. >> >> Port configuration changes only become active when forwarding is started/restarted. >> >> +port attach >> +~~~~~~~~~~~ >> + >> +Attach a port specified by pci address or virtual device args. >> + >> +To attach a new pci device, the device should be recognized by kernel first. >> +Then it should be moved under DPDK management. >> +Finally the port can be attached to testpmd. >> +On the other hand, to attach a port created by virtual device, above steps are not needed. >> + >> +port attach (identifier) >> + >> +For example, to attach a port that pci address is 0000:02:00.0. > Reword " port that pci address is " to "port whose pci address is" Hi Bernard, Thanks, I will fix below comments like your suggestion. Tetsuya >> + >> +.. code-block:: console >> + >> + testpmd> port attach 0000:02:00.0 >> + Attaching a new port... >> + ... snip ... >> + Port 0 is attached. Now total ports is 1 >> + Done >> + >> +For example, to attach a port created by pcap PMD. >> + >> +.. code-block:: console >> + >> + testpmd> port attach eth_pcap0,iface=eth0 >> + Attaching a new port... >> + ... snip ... >> + Port 0 is attached. Now total ports is 1 >> + Done >> + >> +In this case, identifier is "eth_pcap0,iface=eth0". >> +This identifier format is the same as "--vdev" format of DPDK applications. >> + >> +port detach >> +~~~~~~~~~~~ >> + >> +Detach a specific port. >> + >> +Before detaching a port, the port should be closed. >> +Also to remove a pci device completely from the system, first detach the port from testpmd. >> +Then the device should be moved under kernel management. >> +Finally the device can be remove using kernel pci hotplug functionality. > Reword "remove" to "removed" > >> +On the other hand, to remove a port created by virtual device, above steps are not needed. > Reword " created by virtual device" to "created by a virtual device" > >> + >> +port detach (port_id) >> + >> +For example, to detach a port 0. >> + >> +.. code-block:: console >> + >> + testpmd> port detach 0 >> + Detaching a port... >> + ... snip ... >> + Done >> + >> port start >> ~~~~~~~~~~ >> >> -- >> 1.9.1 > Regards, > > Bernard. >
On 2015/02/02 20:57, Thomas Monjalon wrote:
> 2015-02-02 11:33, Iremonger, Bernard:
>> From: Tetsuya Mukawa [mailto:mukawa@igel.co.jp]
>>> --- a/doc/guides/testpmd_app_ug/testpmd_funcs.rst
>>> +++ b/doc/guides/testpmd_app_ug/testpmd_funcs.rst
>> Hi Tetsuya,
>>
>> The doc changes should be in separate commit using the "doc: explaination" commit line.
> I agree that new docs should be in a separate commit.
> Though when updating some code (like here), it's a good idea to update the doc
> in the same commit. Then the change is atomic.
>
> If you want to figure which patches are updating the doc, it should be possible
> to have a filter on "+++ b/doc/".
>
Hi Thomas,
I appreciate your comment. I will follow this guideline.
Thanks,
Tetsuya
On 2/1/2015 12:02 PM, Tetsuya Mukawa wrote: > - Add rte_eal_pci_close_one_dirver() > The function is used for closing the specified driver and device. > - Add pci_invoke_all_drivers() > The function is based on pci_probe_all_drivers. But it can not only > probe but also close drivers. > - Add pci_close_all_drivers() > The function tries to find a driver for the specified device, and > then close the driver. > - Add rte_eal_pci_probe_one() and rte_eal_pci_close_one() > The functions are used for probe and close a device. > First the function tries to find a device that has the specfied > PCI address. Then, probe or close the device. > > v5: > - Remove RTE_EAL_INVOKE_TYPE_UNKNOWN, because it's unused. > v4: > - Fix paramerter checking. > - Fix indent of 'if' statement. > > Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp> > --- > lib/librte_eal/common/eal_common_pci.c | 90 +++++++++++++++++++++++++++++---- > lib/librte_eal/common/eal_private.h | 24 +++++++++ > lib/librte_eal/common/include/rte_pci.h | 33 ++++++++++++ > lib/librte_eal/linuxapp/eal/eal_pci.c | 69 +++++++++++++++++++++++++ > 4 files changed, 206 insertions(+), 10 deletions(-) > > diff --git a/lib/librte_eal/common/eal_common_pci.c b/lib/librte_eal/common/eal_common_pci.c > index a89f5c3..7c9b8c5 100644 > --- a/lib/librte_eal/common/eal_common_pci.c > +++ b/lib/librte_eal/common/eal_common_pci.c > @@ -99,19 +99,27 @@ static struct rte_devargs *pci_devargs_lookup(struct rte_pci_device *dev) > return NULL; > } > > -/* > - * If vendor/device ID match, call the devinit() function of all > - * registered driver for the given device. Return -1 if initialization > - * failed, return 1 if no driver is found for this device. > - */ > static int > -pci_probe_all_drivers(struct rte_pci_device *dev) > +pci_invoke_all_drivers(struct rte_pci_device *dev, > + enum rte_eal_invoke_type type) > { > struct rte_pci_driver *dr = NULL; > - int rc; > + int rc = 0; > + > + if ((dev == NULL) || (type >= RTE_EAL_INVOKE_TYPE_MAX)) > + return -1; > > TAILQ_FOREACH(dr, &pci_driver_list, next) { > - rc = rte_eal_pci_probe_one_driver(dr, dev); > + switch (type) { > + case RTE_EAL_INVOKE_TYPE_PROBE: > + rc = rte_eal_pci_probe_one_driver(dr, dev); > + break; > + case RTE_EAL_INVOKE_TYPE_CLOSE: > + rc = rte_eal_pci_close_one_driver(dr, dev); > + break; > + default: > + return -1; > + } > if (rc < 0) > /* negative value is an error */ > return -1; > @@ -123,6 +131,66 @@ pci_probe_all_drivers(struct rte_pci_device *dev) > return 1; > } > > +#ifdef ENABLE_HOTPLUG > +static int > +rte_eal_pci_invoke_one(struct rte_pci_addr *addr, > + enum rte_eal_invoke_type type) > +{ > + struct rte_pci_device *dev = NULL; > + int ret = 0; > + > + if ((addr == NULL) || (type >= RTE_EAL_INVOKE_TYPE_MAX)) > + return -1; > + > + TAILQ_FOREACH(dev, &pci_device_list, next) { > + if (eal_compare_pci_addr(&dev->addr, addr)) > + continue; > + > + ret = pci_invoke_all_drivers(dev, type); > + if (ret < 0) > + goto invoke_err_return; > + > + if (type == RTE_EAL_INVOKE_TYPE_CLOSE) > + goto remove_dev; > + > + return 0; > + } > + > + return -1; > + > +invoke_err_return: > + RTE_LOG(WARNING, EAL, "Requested device " PCI_PRI_FMT > + " cannot be used\n", dev->addr.domain, dev->addr.bus, > + dev->addr.devid, dev->addr.function); > + return -1; > + > +remove_dev: > + TAILQ_REMOVE(&pci_device_list, dev, next); > + return 0; > +} > + > + > +/* > + * Find the pci device specified by pci address, then invoke probe function of > + * the driver of the devive. > + */ > +int > +rte_eal_pci_probe_one(struct rte_pci_addr *addr) > +{ > + return rte_eal_pci_invoke_one(addr, RTE_EAL_INVOKE_TYPE_PROBE); > +} > + > +/* > + * Find the pci device specified by pci address, then invoke close function of > + * the driver of the devive. > + */ > +int > +rte_eal_pci_close_one(struct rte_pci_addr *addr) > +{ > + return rte_eal_pci_invoke_one(addr, RTE_EAL_INVOKE_TYPE_CLOSE); > +} > +#endif /* ENABLE_HOTPLUG */ > + > /* > * Scan the content of the PCI bus, and call the devinit() function for > * all registered drivers that have a matching entry in its id_table > @@ -148,10 +216,12 @@ rte_eal_pci_probe(void) > > /* probe all or only whitelisted devices */ > if (probe_all) > - ret = pci_probe_all_drivers(dev); > + ret = pci_invoke_all_drivers(dev, > + RTE_EAL_INVOKE_TYPE_PROBE); > else if (devargs != NULL && > devargs->type == RTE_DEVTYPE_WHITELISTED_PCI) > - ret = pci_probe_all_drivers(dev); > + ret = pci_invoke_all_drivers(dev, > + RTE_EAL_INVOKE_TYPE_PROBE); > if (ret < 0) > rte_exit(EXIT_FAILURE, "Requested device " PCI_PRI_FMT > " cannot be used\n", dev->addr.domain, dev->addr.bus, > diff --git a/lib/librte_eal/common/eal_private.h b/lib/librte_eal/common/eal_private.h > index 159cd66..1a362ab 100644 > --- a/lib/librte_eal/common/eal_private.h > +++ b/lib/librte_eal/common/eal_private.h > @@ -154,6 +154,15 @@ struct rte_pci_driver; > struct rte_pci_device; > > /** > + * The invoke type. > + */ > +enum rte_eal_invoke_type { > + RTE_EAL_INVOKE_TYPE_PROBE, /**< invoke probe function */ > + RTE_EAL_INVOKE_TYPE_CLOSE, /**< invoke close function */ > + RTE_EAL_INVOKE_TYPE_MAX /**< max value of this enum */ > +}; > + > +/** > * Mmap memory for single PCI device > * > * This function is private to EAL. > @@ -165,6 +174,21 @@ int rte_eal_pci_probe_one_driver(struct rte_pci_driver *dr, > struct rte_pci_device *dev); > > /** > + * Munmap memory for single PCI device > + * > + * This function is private to EAL. > + * > + * @param dr > + * The pointer to the pci driver structure > + * @param dev > + * The pointer to the pci device structure > + * @return > + * 0 on success, negative on error > + */ > +int rte_eal_pci_close_one_driver(struct rte_pci_driver *dr, > + struct rte_pci_device *dev); > + > +/** > * Init tail queues for non-EAL library structures. This is to allow > * the rings, mempools, etc. lists to be shared among multiple processes > * > diff --git a/lib/librte_eal/common/include/rte_pci.h b/lib/librte_eal/common/include/rte_pci.h > index 87ca4cf..a111066 100644 > --- a/lib/librte_eal/common/include/rte_pci.h > +++ b/lib/librte_eal/common/include/rte_pci.h > @@ -82,6 +82,7 @@ extern "C" { > #include <inttypes.h> > > #include <rte_interrupts.h> > +#include <rte_dev_hotplug.h> > > TAILQ_HEAD(pci_device_list, rte_pci_device); /**< PCI devices in D-linked Q. */ > TAILQ_HEAD(pci_driver_list, rte_pci_driver); /**< PCI drivers in D-linked Q. */ > @@ -323,6 +324,38 @@ eal_compare_pci_addr(struct rte_pci_addr *addr, struct rte_pci_addr *addr2) > */ > int rte_eal_pci_probe(void); > > +#ifdef ENABLE_HOTPLUG > +/** > + * Probe the single PCI device. > + * > + * Scan the content of the PCI bus, and find the pci device specified by pci > + * address, then call the probe() function for registered driver that has a > + * matching entry in its id_table for discovered device. > + * > + * @param addr > + * The PCI Bus-Device-Function address to probe or close. > + * @return > + * - 0 on success. > + * - Negative on error. > + */ > +int rte_eal_pci_probe_one(struct rte_pci_addr *addr); > + > +/** > + * Close the single PCI device. > + * > + * Scan the content of the PCI bus, and find the pci device specified by pci > + * address, then call the close() function for registered driver that has a > + * matching entry in its id_table for discovered device. > + * > + * @param addr > + * The PCI Bus-Device-Function address to probe or close. > + * @return > + * - 0 on success. > + * - Negative on error. > + */ > +int rte_eal_pci_close_one(struct rte_pci_addr *addr); > +#endif /* ENABLE_HOTPLUG */ > + > /** > * Dump the content of the PCI bus. > * > diff --git a/lib/librte_eal/linuxapp/eal/eal_pci.c b/lib/librte_eal/linuxapp/eal/eal_pci.c > index c3b7917..831422e 100644 > --- a/lib/librte_eal/linuxapp/eal/eal_pci.c > +++ b/lib/librte_eal/linuxapp/eal/eal_pci.c > @@ -682,6 +682,75 @@ rte_eal_pci_probe_one_driver(struct rte_pci_driver *dr, struct rte_pci_device *d > return 1; > } > > +#ifdef ENABLE_HOTPLUG > +/* > + * If vendor/device ID match, call the devuninit() function of the > + * driver. > + */ > +int > +rte_eal_pci_close_one_driver(struct rte_pci_driver *dr, > + struct rte_pci_device *dev) > +{ > + struct rte_pci_id *id_table; > + > + if ((dr == NULL) || (dev == NULL)) > + return -EINVAL; > + > + for (id_table = dr->id_table ; id_table->vendor_id != 0; id_table++) { > + > + /* check if device's identifiers match the driver's ones */ > + if (id_table->vendor_id != dev->id.vendor_id && > + id_table->vendor_id != PCI_ANY_ID) > + continue; > + if (id_table->device_id != dev->id.device_id && > + id_table->device_id != PCI_ANY_ID) > + continue; > + if (id_table->subsystem_vendor_id != > + dev->id.subsystem_vendor_id && > + id_table->subsystem_vendor_id != PCI_ANY_ID) > + continue; > + if (id_table->subsystem_device_id != > + dev->id.subsystem_device_id && > + id_table->subsystem_device_id != PCI_ANY_ID) > + continue; > + > + struct rte_pci_addr *loc = &dev->addr; > + > + RTE_LOG(DEBUG, EAL, > + "PCI device "PCI_PRI_FMT" on NUMA socket %i\n", > + loc->domain, loc->bus, loc->devid, > + loc->function, dev->numa_node); > + > + RTE_LOG(DEBUG, EAL, " remove driver: %x:%x %s\n", > + dev->id.vendor_id, dev->id.device_id, > + dr->name); > + > + /* call the driver devuninit() function */ > + if (dr->devuninit && (dr->devuninit(dr, dev) < 0)) > + return -1; /* negative value is an error */ > + > + /* clear driver structure */ > + dev->driver = NULL; > + > + if (dr->drv_flags & RTE_PCI_DRV_NEED_MAPPING) > + /* unmap resources for devices that use igb_uio */ > + pci_unmap_device(dev); Hi, Tetsuya I have one question, as the code shows, in pci_unmap_device(), will check pt_driver. But assume that, we are now try to detach a vfio device, after print out a error message of unsupported, the does this port workable? I think this port will unworkable, am I right? But actually, we should keep it workable. My suggestion is to add a check in rte_eth_dev_check_detachable() for pci_device port. Thanks Michael > + > + return 0; > + } > + /* return positive value if driver is not found */ > + return 1; > +} > +#else /* ENABLE_HOTPLUG */ > +int > +rte_eal_pci_close_one_driver(struct rte_pci_driver *dr __rte_unused, > + struct rte_pci_device *dev __rte_unused) > +{ > + RTE_LOG(ERR, EAL, "Hotplug support isn't enabled\n"); > + return -1; > +} > +#endif /* ENABLE_HOTPLUG */ > + > /* Init the PCI EAL subsystem */ > int > rte_eal_pci_init(void)
On 2/1/2015 12:02 PM, Tetsuya Mukawa wrote: > The patch adds following functions. > > - rte_eth_dev_save() > The function is used for saving current rte_eth_dev structures. > - rte_eth_dev_get_changed_port() > The function receives the rte_eth_dev structures, then compare > these with current values to know which port is actually > attached or detached. > - rte_eth_dev_get_addr_by_port() > The function returns a pci address of a ethdev specified by port > identifier. > - rte_eth_dev_get_port_by_addr() > The function returns a port identifier of a ethdev specified by > pci address. > - rte_eth_dev_get_name_by_port() > The function returns a unique identifier name of a ethdev > specified by port identifier. > - Add rte_eth_dev_check_detachable() > The function returns whether a PMD supports detach function. > > Also the patch changes scope of rte_eth_dev_allocated() to global. > This function will be called by virtual PMDs to support port hotplug. > So change scope of the function to global. > > v5: > - Fix return value of below functions. > rte_eth_dev_get_changed_port(). > rte_eth_dev_get_port_by_addr(). > v4: > - Add paramerter checking. > v3: > - Fix if-condition bug while comparing pci addresses. > - Add error checking codes. > Reported-by: Mark Enright <menrigh@brocade.com> > > Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp> > --- > lib/librte_ether/rte_ethdev.c | 98 ++++++++++++++++++++++++++++++++++++++++++- > lib/librte_ether/rte_ethdev.h | 80 +++++++++++++++++++++++++++++++++++ > 2 files changed, 177 insertions(+), 1 deletion(-) > > diff --git a/lib/librte_ether/rte_ethdev.c b/lib/librte_ether/rte_ethdev.c > index 7bed901..5aded10 100644 > --- a/lib/librte_ether/rte_ethdev.c > +++ b/lib/librte_ether/rte_ethdev.c > @@ -206,7 +206,7 @@ rte_eth_dev_data_alloc(void) > RTE_MAX_ETHPORTS * sizeof(*rte_eth_dev_data)); > } > > -static struct rte_eth_dev * > +struct rte_eth_dev * > rte_eth_dev_allocated(const char *name) > { > unsigned i; > @@ -426,6 +426,102 @@ rte_eth_dev_count(void) > return (nb_ports); > } > > +void > +rte_eth_dev_save(struct rte_eth_dev *devs) > +{ > + if (devs == NULL) > + return; > + > + /* save current rte_eth_devices */ > + memcpy(devs, rte_eth_devices, > + sizeof(struct rte_eth_dev) * RTE_MAX_ETHPORTS); > +} > + > +int > +rte_eth_dev_get_changed_port(struct rte_eth_dev *devs, uint8_t *port_id) > +{ > + if ((devs == NULL) || (port_id == NULL)) > + return -EINVAL; > + > + /* check which port was attached or detached */ > + for (*port_id = 0; *port_id < RTE_MAX_ETHPORTS; (*port_id)++, devs++) { > + if (rte_eth_devices[*port_id].attached ^ devs->attached) > + return 0; > + } > + return -ENODEV; > +} > + > +int > +rte_eth_dev_get_addr_by_port(uint8_t port_id, struct rte_pci_addr *addr) > +{ > + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) > + return -EINVAL; > + > + if (addr == NULL) { > + PMD_DEBUG_TRACE("Null pointer is specified\n"); > + return -EINVAL; > + } > + > + *addr = rte_eth_devices[port_id].pci_dev->addr; > + return 0; > +} > + > +int > +rte_eth_dev_get_port_by_addr(struct rte_pci_addr *addr, uint8_t *port_id) > +{ > + struct rte_pci_addr *tmp; > + > + if ((addr == NULL) || (port_id == NULL)) { > + PMD_DEBUG_TRACE("Null pointer is specified\n"); > + return -EINVAL; > + } > + > + for (*port_id = 0; *port_id < RTE_MAX_ETHPORTS; (*port_id)++) { > + if (!rte_eth_devices[*port_id].attached) > + continue; > + if (!rte_eth_devices[*port_id].pci_dev) > + continue; > + tmp = &rte_eth_devices[*port_id].pci_dev->addr; > + if (eal_compare_pci_addr(tmp, addr) == 0) > + return 0; > + } > + return -ENODEV; > +} > + > +int > +rte_eth_dev_get_name_by_port(uint8_t port_id, char *name) > +{ > + char *tmp; > + > + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) > + return -EINVAL; > + > + if (name == NULL) { > + PMD_DEBUG_TRACE("Null pointer is specified\n"); > + return -EINVAL; > + } > + > + /* shouldn't check 'rte_eth_devices[i].data', > + * because it might be overwritten by VDEV PMD */ > + tmp = rte_eth_dev_data[port_id].name; > + strncpy(name, tmp, strlen(tmp) + 1); > + return 0; > +} > + > +int > +rte_eth_dev_check_detachable(uint8_t port_id) > +{ > + uint32_t drv_flags; > + > + if (port_id >= RTE_MAX_ETHPORTS) { > + PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); > + return -EINVAL; > + } > + > + drv_flags = rte_eth_devices[port_id].driver->pci_drv.drv_flags; Here should be better to add pt_driver for pci_dev type port. Thanks, Michael > + return !(drv_flags & RTE_PCI_DRV_DETACHABLE); > +} > + > static int > rte_eth_dev_rx_queue_config(struct rte_eth_dev *dev, uint16_t nb_queues) > { > diff --git a/lib/librte_ether/rte_ethdev.h b/lib/librte_ether/rte_ethdev.h > index 91d9e86..9919968 100644 > --- a/lib/librte_ether/rte_ethdev.h > +++ b/lib/librte_ether/rte_ethdev.h > @@ -1616,6 +1616,86 @@ extern struct rte_eth_dev rte_eth_devices[]; > extern uint8_t rte_eth_dev_count(void); > > /** > + * Function for internal use by port hotplug functions. > + * Copies current ethdev structures to the specified pointer. > + * > + * @param devs The pointer to the ethdev structures > + */ > +extern void rte_eth_dev_save(struct rte_eth_dev *devs); > + > +/** > + * Function for internal use by port hotplug functions. > + * Compare the specified ethdev structures with currents. Then > + * if there is a port which status is changed, fill the specified pointer > + * with the port id of that port. > + * @param devs The pointer to the ethdev structures > + * @param port_id The pointer to the port id > + * @return > + * - 0 on success, negative on error > + */ > +extern int rte_eth_dev_get_changed_port( > + struct rte_eth_dev *devs, uint8_t *port_id); > + > +/** > + * Function for internal use by port hotplug functions. > + * Returns a pci address of a ethdev specified by port identifier. > + * @param port_id > + * The port identifier of the Ethernet device > + * @param addr > + * The pointer to the pci address > + * @return > + * - 0 on success, negative on error > + */ > +extern int rte_eth_dev_get_addr_by_port( > + uint8_t port_id, struct rte_pci_addr *addr); > + > +/** > + * Function for internal use by port hotplug functions. > + * Returns a port identifier of a ethdev specified by pci address. > + * @param addr > + * The pointer to the pci address of the Ethernet device. > + * @param port_id > + * The pointer to the port identifier > + * @return > + * - 0 on success, negative on error > + */ > +extern int rte_eth_dev_get_port_by_addr( > + struct rte_pci_addr *addr, uint8_t *port_id); > + > +/** > + * Function for internal use by port hotplug functions. > + * Returns a unique identifier name of a ethdev specified by port identifier. > + * @param port_id > + * The port identifier. > + * @param name > + * The pointer to the Unique identifier name for each Ethernet device > + * @return > + * - 0 on success, negative on error > + */ > +extern int rte_eth_dev_get_name_by_port(uint8_t port_id, char *name); > + > +/** > + * Function for internal use by port hotplug functions. > + * Check whether or not, a PMD that is handling the ethdev specified by port > + * identifier can support detach function. > + * @param port_id > + * The port identifier > + * @return > + * - 0 on supporting detach function, negative on not supporting > + */ > +extern int rte_eth_dev_check_detachable(uint8_t port_id); > + > +/** > + * Function for internal use by port hotplug functions. > + * Returns a ethdev slot specified by the unique identifier name. > + * @param name > + * The pointer to the Unique identifier name for each Ethernet device > + * @return > + * - The pointer to the ethdev slot, on success. NULL on error > + */ > +extern struct rte_eth_dev *rte_eth_dev_allocated(const char *name); > + > +/** > * Function for internal use by dummy drivers primarily, e.g. ring-based > * driver. > * Allocates a new ethdev slot for an ethernet device and returns the pointer
On 2015/02/03 11:35, Qiu, Michael wrote: > On 2/1/2015 12:02 PM, Tetsuya Mukawa wrote: >> - Add rte_eal_pci_close_one_dirver() >> The function is used for closing the specified driver and device. >> - Add pci_invoke_all_drivers() >> The function is based on pci_probe_all_drivers. But it can not only >> probe but also close drivers. >> - Add pci_close_all_drivers() >> The function tries to find a driver for the specified device, and >> then close the driver. >> - Add rte_eal_pci_probe_one() and rte_eal_pci_close_one() >> The functions are used for probe and close a device. >> First the function tries to find a device that has the specfied >> PCI address. Then, probe or close the device. >> >> v5: >> - Remove RTE_EAL_INVOKE_TYPE_UNKNOWN, because it's unused. >> v4: >> - Fix paramerter checking. >> - Fix indent of 'if' statement. >> >> Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp> >> --- >> lib/librte_eal/common/eal_common_pci.c | 90 +++++++++++++++++++++++++++++---- >> lib/librte_eal/common/eal_private.h | 24 +++++++++ >> lib/librte_eal/common/include/rte_pci.h | 33 ++++++++++++ >> lib/librte_eal/linuxapp/eal/eal_pci.c | 69 +++++++++++++++++++++++++ >> 4 files changed, 206 insertions(+), 10 deletions(-) >> >> diff --git a/lib/librte_eal/common/eal_common_pci.c b/lib/librte_eal/common/eal_common_pci.c >> index a89f5c3..7c9b8c5 100644 >> --- a/lib/librte_eal/common/eal_common_pci.c >> +++ b/lib/librte_eal/common/eal_common_pci.c >> @@ -99,19 +99,27 @@ static struct rte_devargs *pci_devargs_lookup(struct rte_pci_device *dev) >> return NULL; >> } >> >> -/* >> - * If vendor/device ID match, call the devinit() function of all >> - * registered driver for the given device. Return -1 if initialization >> - * failed, return 1 if no driver is found for this device. >> - */ >> static int >> -pci_probe_all_drivers(struct rte_pci_device *dev) >> +pci_invoke_all_drivers(struct rte_pci_device *dev, >> + enum rte_eal_invoke_type type) >> { >> struct rte_pci_driver *dr = NULL; >> - int rc; >> + int rc = 0; >> + >> + if ((dev == NULL) || (type >= RTE_EAL_INVOKE_TYPE_MAX)) >> + return -1; >> >> TAILQ_FOREACH(dr, &pci_driver_list, next) { >> - rc = rte_eal_pci_probe_one_driver(dr, dev); >> + switch (type) { >> + case RTE_EAL_INVOKE_TYPE_PROBE: >> + rc = rte_eal_pci_probe_one_driver(dr, dev); >> + break; >> + case RTE_EAL_INVOKE_TYPE_CLOSE: >> + rc = rte_eal_pci_close_one_driver(dr, dev); >> + break; >> + default: >> + return -1; >> + } >> if (rc < 0) >> /* negative value is an error */ >> return -1; >> @@ -123,6 +131,66 @@ pci_probe_all_drivers(struct rte_pci_device *dev) >> return 1; >> } >> >> +#ifdef ENABLE_HOTPLUG >> +static int >> +rte_eal_pci_invoke_one(struct rte_pci_addr *addr, >> + enum rte_eal_invoke_type type) >> +{ >> + struct rte_pci_device *dev = NULL; >> + int ret = 0; >> + >> + if ((addr == NULL) || (type >= RTE_EAL_INVOKE_TYPE_MAX)) >> + return -1; >> + >> + TAILQ_FOREACH(dev, &pci_device_list, next) { >> + if (eal_compare_pci_addr(&dev->addr, addr)) >> + continue; >> + >> + ret = pci_invoke_all_drivers(dev, type); >> + if (ret < 0) >> + goto invoke_err_return; >> + >> + if (type == RTE_EAL_INVOKE_TYPE_CLOSE) >> + goto remove_dev; >> + >> + return 0; >> + } >> + >> + return -1; >> + >> +invoke_err_return: >> + RTE_LOG(WARNING, EAL, "Requested device " PCI_PRI_FMT >> + " cannot be used\n", dev->addr.domain, dev->addr.bus, >> + dev->addr.devid, dev->addr.function); >> + return -1; >> + >> +remove_dev: >> + TAILQ_REMOVE(&pci_device_list, dev, next); >> + return 0; >> +} >> + >> + >> +/* >> + * Find the pci device specified by pci address, then invoke probe function of >> + * the driver of the devive. >> + */ >> +int >> +rte_eal_pci_probe_one(struct rte_pci_addr *addr) >> +{ >> + return rte_eal_pci_invoke_one(addr, RTE_EAL_INVOKE_TYPE_PROBE); >> +} >> + >> +/* >> + * Find the pci device specified by pci address, then invoke close function of >> + * the driver of the devive. >> + */ >> +int >> +rte_eal_pci_close_one(struct rte_pci_addr *addr) >> +{ >> + return rte_eal_pci_invoke_one(addr, RTE_EAL_INVOKE_TYPE_CLOSE); >> +} >> +#endif /* ENABLE_HOTPLUG */ >> + >> /* >> * Scan the content of the PCI bus, and call the devinit() function for >> * all registered drivers that have a matching entry in its id_table >> @@ -148,10 +216,12 @@ rte_eal_pci_probe(void) >> >> /* probe all or only whitelisted devices */ >> if (probe_all) >> - ret = pci_probe_all_drivers(dev); >> + ret = pci_invoke_all_drivers(dev, >> + RTE_EAL_INVOKE_TYPE_PROBE); >> else if (devargs != NULL && >> devargs->type == RTE_DEVTYPE_WHITELISTED_PCI) >> - ret = pci_probe_all_drivers(dev); >> + ret = pci_invoke_all_drivers(dev, >> + RTE_EAL_INVOKE_TYPE_PROBE); >> if (ret < 0) >> rte_exit(EXIT_FAILURE, "Requested device " PCI_PRI_FMT >> " cannot be used\n", dev->addr.domain, dev->addr.bus, >> diff --git a/lib/librte_eal/common/eal_private.h b/lib/librte_eal/common/eal_private.h >> index 159cd66..1a362ab 100644 >> --- a/lib/librte_eal/common/eal_private.h >> +++ b/lib/librte_eal/common/eal_private.h >> @@ -154,6 +154,15 @@ struct rte_pci_driver; >> struct rte_pci_device; >> >> /** >> + * The invoke type. >> + */ >> +enum rte_eal_invoke_type { >> + RTE_EAL_INVOKE_TYPE_PROBE, /**< invoke probe function */ >> + RTE_EAL_INVOKE_TYPE_CLOSE, /**< invoke close function */ >> + RTE_EAL_INVOKE_TYPE_MAX /**< max value of this enum */ >> +}; >> + >> +/** >> * Mmap memory for single PCI device >> * >> * This function is private to EAL. >> @@ -165,6 +174,21 @@ int rte_eal_pci_probe_one_driver(struct rte_pci_driver *dr, >> struct rte_pci_device *dev); >> >> /** >> + * Munmap memory for single PCI device >> + * >> + * This function is private to EAL. >> + * >> + * @param dr >> + * The pointer to the pci driver structure >> + * @param dev >> + * The pointer to the pci device structure >> + * @return >> + * 0 on success, negative on error >> + */ >> +int rte_eal_pci_close_one_driver(struct rte_pci_driver *dr, >> + struct rte_pci_device *dev); >> + >> +/** >> * Init tail queues for non-EAL library structures. This is to allow >> * the rings, mempools, etc. lists to be shared among multiple processes >> * >> diff --git a/lib/librte_eal/common/include/rte_pci.h b/lib/librte_eal/common/include/rte_pci.h >> index 87ca4cf..a111066 100644 >> --- a/lib/librte_eal/common/include/rte_pci.h >> +++ b/lib/librte_eal/common/include/rte_pci.h >> @@ -82,6 +82,7 @@ extern "C" { >> #include <inttypes.h> >> >> #include <rte_interrupts.h> >> +#include <rte_dev_hotplug.h> >> >> TAILQ_HEAD(pci_device_list, rte_pci_device); /**< PCI devices in D-linked Q. */ >> TAILQ_HEAD(pci_driver_list, rte_pci_driver); /**< PCI drivers in D-linked Q. */ >> @@ -323,6 +324,38 @@ eal_compare_pci_addr(struct rte_pci_addr *addr, struct rte_pci_addr *addr2) >> */ >> int rte_eal_pci_probe(void); >> >> +#ifdef ENABLE_HOTPLUG >> +/** >> + * Probe the single PCI device. >> + * >> + * Scan the content of the PCI bus, and find the pci device specified by pci >> + * address, then call the probe() function for registered driver that has a >> + * matching entry in its id_table for discovered device. >> + * >> + * @param addr >> + * The PCI Bus-Device-Function address to probe or close. >> + * @return >> + * - 0 on success. >> + * - Negative on error. >> + */ >> +int rte_eal_pci_probe_one(struct rte_pci_addr *addr); >> + >> +/** >> + * Close the single PCI device. >> + * >> + * Scan the content of the PCI bus, and find the pci device specified by pci >> + * address, then call the close() function for registered driver that has a >> + * matching entry in its id_table for discovered device. >> + * >> + * @param addr >> + * The PCI Bus-Device-Function address to probe or close. >> + * @return >> + * - 0 on success. >> + * - Negative on error. >> + */ >> +int rte_eal_pci_close_one(struct rte_pci_addr *addr); >> +#endif /* ENABLE_HOTPLUG */ >> + >> /** >> * Dump the content of the PCI bus. >> * >> diff --git a/lib/librte_eal/linuxapp/eal/eal_pci.c b/lib/librte_eal/linuxapp/eal/eal_pci.c >> index c3b7917..831422e 100644 >> --- a/lib/librte_eal/linuxapp/eal/eal_pci.c >> +++ b/lib/librte_eal/linuxapp/eal/eal_pci.c >> @@ -682,6 +682,75 @@ rte_eal_pci_probe_one_driver(struct rte_pci_driver *dr, struct rte_pci_device *d >> return 1; >> } >> >> +#ifdef ENABLE_HOTPLUG >> +/* >> + * If vendor/device ID match, call the devuninit() function of the >> + * driver. >> + */ >> +int >> +rte_eal_pci_close_one_driver(struct rte_pci_driver *dr, >> + struct rte_pci_device *dev) >> +{ >> + struct rte_pci_id *id_table; >> + >> + if ((dr == NULL) || (dev == NULL)) >> + return -EINVAL; >> + >> + for (id_table = dr->id_table ; id_table->vendor_id != 0; id_table++) { >> + >> + /* check if device's identifiers match the driver's ones */ >> + if (id_table->vendor_id != dev->id.vendor_id && >> + id_table->vendor_id != PCI_ANY_ID) >> + continue; >> + if (id_table->device_id != dev->id.device_id && >> + id_table->device_id != PCI_ANY_ID) >> + continue; >> + if (id_table->subsystem_vendor_id != >> + dev->id.subsystem_vendor_id && >> + id_table->subsystem_vendor_id != PCI_ANY_ID) >> + continue; >> + if (id_table->subsystem_device_id != >> + dev->id.subsystem_device_id && >> + id_table->subsystem_device_id != PCI_ANY_ID) >> + continue; >> + >> + struct rte_pci_addr *loc = &dev->addr; >> + >> + RTE_LOG(DEBUG, EAL, >> + "PCI device "PCI_PRI_FMT" on NUMA socket %i\n", >> + loc->domain, loc->bus, loc->devid, >> + loc->function, dev->numa_node); >> + >> + RTE_LOG(DEBUG, EAL, " remove driver: %x:%x %s\n", >> + dev->id.vendor_id, dev->id.device_id, >> + dr->name); >> + >> + /* call the driver devuninit() function */ >> + if (dr->devuninit && (dr->devuninit(dr, dev) < 0)) >> + return -1; /* negative value is an error */ >> + >> + /* clear driver structure */ >> + dev->driver = NULL; >> + >> + if (dr->drv_flags & RTE_PCI_DRV_NEED_MAPPING) >> + /* unmap resources for devices that use igb_uio */ >> + pci_unmap_device(dev); > Hi, Tetsuya > > I have one question, as the code shows, in pci_unmap_device(), will > check pt_driver. > > But assume that, we are now try to detach a vfio device, after print out > a error message of unsupported, the does this port workable? > > I think this port will unworkable, am I right? > > But actually, we should keep it workable. > > My suggestion is to add a check in rte_eth_dev_check_detachable() for > pci_device port. Hi Michael, I appreciate your comment. In the function called "rte_eal_dev_detach_pdev()", "rte_eth_dev_check_detachable()" has been already checked. But in the future, someone may want to reuse "rte_eal_pci_close_one_driver()". So I will add the checking like your suggestion. Thanks, Tetsuya > > Thanks > Michael > >> + >> + return 0; >> + } >> + /* return positive value if driver is not found */ >> + return 1; >> +} >> +#else /* ENABLE_HOTPLUG */ >> +int >> +rte_eal_pci_close_one_driver(struct rte_pci_driver *dr __rte_unused, >> + struct rte_pci_device *dev __rte_unused) >> +{ >> + RTE_LOG(ERR, EAL, "Hotplug support isn't enabled\n"); >> + return -1; >> +} >> +#endif /* ENABLE_HOTPLUG */ >> + >> /* Init the PCI EAL subsystem */ >> int >> rte_eal_pci_init(void)
On 2015/02/03 11:37, Qiu, Michael wrote: > On 2/1/2015 12:02 PM, Tetsuya Mukawa wrote: >> The patch adds following functions. >> >> - rte_eth_dev_save() >> The function is used for saving current rte_eth_dev structures. >> - rte_eth_dev_get_changed_port() >> The function receives the rte_eth_dev structures, then compare >> these with current values to know which port is actually >> attached or detached. >> - rte_eth_dev_get_addr_by_port() >> The function returns a pci address of a ethdev specified by port >> identifier. >> - rte_eth_dev_get_port_by_addr() >> The function returns a port identifier of a ethdev specified by >> pci address. >> - rte_eth_dev_get_name_by_port() >> The function returns a unique identifier name of a ethdev >> specified by port identifier. >> - Add rte_eth_dev_check_detachable() >> The function returns whether a PMD supports detach function. >> >> Also the patch changes scope of rte_eth_dev_allocated() to global. >> This function will be called by virtual PMDs to support port hotplug. >> So change scope of the function to global. >> >> v5: >> - Fix return value of below functions. >> rte_eth_dev_get_changed_port(). >> rte_eth_dev_get_port_by_addr(). >> v4: >> - Add paramerter checking. >> v3: >> - Fix if-condition bug while comparing pci addresses. >> - Add error checking codes. >> Reported-by: Mark Enright <menrigh@brocade.com> >> >> Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp> >> --- >> lib/librte_ether/rte_ethdev.c | 98 ++++++++++++++++++++++++++++++++++++++++++- >> lib/librte_ether/rte_ethdev.h | 80 +++++++++++++++++++++++++++++++++++ >> 2 files changed, 177 insertions(+), 1 deletion(-) >> >> diff --git a/lib/librte_ether/rte_ethdev.c b/lib/librte_ether/rte_ethdev.c >> index 7bed901..5aded10 100644 >> --- a/lib/librte_ether/rte_ethdev.c >> +++ b/lib/librte_ether/rte_ethdev.c >> @@ -206,7 +206,7 @@ rte_eth_dev_data_alloc(void) >> RTE_MAX_ETHPORTS * sizeof(*rte_eth_dev_data)); >> } >> >> -static struct rte_eth_dev * >> +struct rte_eth_dev * >> rte_eth_dev_allocated(const char *name) >> { >> unsigned i; >> @@ -426,6 +426,102 @@ rte_eth_dev_count(void) >> return (nb_ports); >> } >> >> +void >> +rte_eth_dev_save(struct rte_eth_dev *devs) >> +{ >> + if (devs == NULL) >> + return; >> + >> + /* save current rte_eth_devices */ >> + memcpy(devs, rte_eth_devices, >> + sizeof(struct rte_eth_dev) * RTE_MAX_ETHPORTS); >> +} >> + >> +int >> +rte_eth_dev_get_changed_port(struct rte_eth_dev *devs, uint8_t *port_id) >> +{ >> + if ((devs == NULL) || (port_id == NULL)) >> + return -EINVAL; >> + >> + /* check which port was attached or detached */ >> + for (*port_id = 0; *port_id < RTE_MAX_ETHPORTS; (*port_id)++, devs++) { >> + if (rte_eth_devices[*port_id].attached ^ devs->attached) >> + return 0; >> + } >> + return -ENODEV; >> +} >> + >> +int >> +rte_eth_dev_get_addr_by_port(uint8_t port_id, struct rte_pci_addr *addr) >> +{ >> + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) >> + return -EINVAL; >> + >> + if (addr == NULL) { >> + PMD_DEBUG_TRACE("Null pointer is specified\n"); >> + return -EINVAL; >> + } >> + >> + *addr = rte_eth_devices[port_id].pci_dev->addr; >> + return 0; >> +} >> + >> +int >> +rte_eth_dev_get_port_by_addr(struct rte_pci_addr *addr, uint8_t *port_id) >> +{ >> + struct rte_pci_addr *tmp; >> + >> + if ((addr == NULL) || (port_id == NULL)) { >> + PMD_DEBUG_TRACE("Null pointer is specified\n"); >> + return -EINVAL; >> + } >> + >> + for (*port_id = 0; *port_id < RTE_MAX_ETHPORTS; (*port_id)++) { >> + if (!rte_eth_devices[*port_id].attached) >> + continue; >> + if (!rte_eth_devices[*port_id].pci_dev) >> + continue; >> + tmp = &rte_eth_devices[*port_id].pci_dev->addr; >> + if (eal_compare_pci_addr(tmp, addr) == 0) >> + return 0; >> + } >> + return -ENODEV; >> +} >> + >> +int >> +rte_eth_dev_get_name_by_port(uint8_t port_id, char *name) >> +{ >> + char *tmp; >> + >> + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) >> + return -EINVAL; >> + >> + if (name == NULL) { >> + PMD_DEBUG_TRACE("Null pointer is specified\n"); >> + return -EINVAL; >> + } >> + >> + /* shouldn't check 'rte_eth_devices[i].data', >> + * because it might be overwritten by VDEV PMD */ >> + tmp = rte_eth_dev_data[port_id].name; >> + strncpy(name, tmp, strlen(tmp) + 1); >> + return 0; >> +} >> + >> +int >> +rte_eth_dev_check_detachable(uint8_t port_id) >> +{ >> + uint32_t drv_flags; >> + >> + if (port_id >= RTE_MAX_ETHPORTS) { >> + PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); >> + return -EINVAL; >> + } >> + >> + drv_flags = rte_eth_devices[port_id].driver->pci_drv.drv_flags; > Here should be better to add pt_driver for pci_dev type port. Sure, I will add it. Tetsuya > Thanks, > Michael >> + return !(drv_flags & RTE_PCI_DRV_DETACHABLE); >> +} >> + >> static int >> rte_eth_dev_rx_queue_config(struct rte_eth_dev *dev, uint16_t nb_queues) >> { >> diff --git a/lib/librte_ether/rte_ethdev.h b/lib/librte_ether/rte_ethdev.h >> index 91d9e86..9919968 100644 >> --- a/lib/librte_ether/rte_ethdev.h >> +++ b/lib/librte_ether/rte_ethdev.h >> @@ -1616,6 +1616,86 @@ extern struct rte_eth_dev rte_eth_devices[]; >> extern uint8_t rte_eth_dev_count(void); >> >> /** >> + * Function for internal use by port hotplug functions. >> + * Copies current ethdev structures to the specified pointer. >> + * >> + * @param devs The pointer to the ethdev structures >> + */ >> +extern void rte_eth_dev_save(struct rte_eth_dev *devs); >> + >> +/** >> + * Function for internal use by port hotplug functions. >> + * Compare the specified ethdev structures with currents. Then >> + * if there is a port which status is changed, fill the specified pointer >> + * with the port id of that port. >> + * @param devs The pointer to the ethdev structures >> + * @param port_id The pointer to the port id >> + * @return >> + * - 0 on success, negative on error >> + */ >> +extern int rte_eth_dev_get_changed_port( >> + struct rte_eth_dev *devs, uint8_t *port_id); >> + >> +/** >> + * Function for internal use by port hotplug functions. >> + * Returns a pci address of a ethdev specified by port identifier. >> + * @param port_id >> + * The port identifier of the Ethernet device >> + * @param addr >> + * The pointer to the pci address >> + * @return >> + * - 0 on success, negative on error >> + */ >> +extern int rte_eth_dev_get_addr_by_port( >> + uint8_t port_id, struct rte_pci_addr *addr); >> + >> +/** >> + * Function for internal use by port hotplug functions. >> + * Returns a port identifier of a ethdev specified by pci address. >> + * @param addr >> + * The pointer to the pci address of the Ethernet device. >> + * @param port_id >> + * The pointer to the port identifier >> + * @return >> + * - 0 on success, negative on error >> + */ >> +extern int rte_eth_dev_get_port_by_addr( >> + struct rte_pci_addr *addr, uint8_t *port_id); >> + >> +/** >> + * Function for internal use by port hotplug functions. >> + * Returns a unique identifier name of a ethdev specified by port identifier. >> + * @param port_id >> + * The port identifier. >> + * @param name >> + * The pointer to the Unique identifier name for each Ethernet device >> + * @return >> + * - 0 on success, negative on error >> + */ >> +extern int rte_eth_dev_get_name_by_port(uint8_t port_id, char *name); >> + >> +/** >> + * Function for internal use by port hotplug functions. >> + * Check whether or not, a PMD that is handling the ethdev specified by port >> + * identifier can support detach function. >> + * @param port_id >> + * The port identifier >> + * @return >> + * - 0 on supporting detach function, negative on not supporting >> + */ >> +extern int rte_eth_dev_check_detachable(uint8_t port_id); >> + >> +/** >> + * Function for internal use by port hotplug functions. >> + * Returns a ethdev slot specified by the unique identifier name. >> + * @param name >> + * The pointer to the Unique identifier name for each Ethernet device >> + * @return >> + * - The pointer to the ethdev slot, on success. NULL on error >> + */ >> +extern struct rte_eth_dev *rte_eth_dev_allocated(const char *name); >> + >> +/** >> * Function for internal use by dummy drivers primarily, e.g. ring-based >> * driver. >> * Allocates a new ethdev slot for an ethernet device and returns the pointer
On 2/3/2015 12:07 PM, Tetsuya Mukawa wrote: > On 2015/02/03 11:35, Qiu, Michael wrote: >> On 2/1/2015 12:02 PM, Tetsuya Mukawa wrote: >>> - Add rte_eal_pci_close_one_dirver() >>> The function is used for closing the specified driver and device. >>> - Add pci_invoke_all_drivers() [...] >>> >>> +#ifdef ENABLE_HOTPLUG >>> +/* >>> + * If vendor/device ID match, call the devuninit() function of the >>> + * driver. >>> + */ >>> +int >>> +rte_eal_pci_close_one_driver(struct rte_pci_driver *dr, >>> + struct rte_pci_device *dev) >>> +{ >>> + struct rte_pci_id *id_table; >>> + >>> + if ((dr == NULL) || (dev == NULL)) >>> + return -EINVAL; >>> + >>> + for (id_table = dr->id_table ; id_table->vendor_id != 0; id_table++) { >>> + >>> + /* check if device's identifiers match the driver's ones */ >>> + if (id_table->vendor_id != dev->id.vendor_id && >>> + id_table->vendor_id != PCI_ANY_ID) >>> + continue; >>> + if (id_table->device_id != dev->id.device_id && >>> + id_table->device_id != PCI_ANY_ID) >>> + continue; >>> + if (id_table->subsystem_vendor_id != >>> + dev->id.subsystem_vendor_id && >>> + id_table->subsystem_vendor_id != PCI_ANY_ID) >>> + continue; >>> + if (id_table->subsystem_device_id != >>> + dev->id.subsystem_device_id && >>> + id_table->subsystem_device_id != PCI_ANY_ID) >>> + continue; >>> + >>> + struct rte_pci_addr *loc = &dev->addr; >>> + >>> + RTE_LOG(DEBUG, EAL, >>> + "PCI device "PCI_PRI_FMT" on NUMA socket %i\n", >>> + loc->domain, loc->bus, loc->devid, >>> + loc->function, dev->numa_node); >>> + >>> + RTE_LOG(DEBUG, EAL, " remove driver: %x:%x %s\n", >>> + dev->id.vendor_id, dev->id.device_id, >>> + dr->name); >>> + >>> + /* call the driver devuninit() function */ >>> + if (dr->devuninit && (dr->devuninit(dr, dev) < 0)) >>> + return -1; /* negative value is an error */ >>> + >>> + /* clear driver structure */ >>> + dev->driver = NULL; >>> + >>> + if (dr->drv_flags & RTE_PCI_DRV_NEED_MAPPING) >>> + /* unmap resources for devices that use igb_uio */ >>> + pci_unmap_device(dev); >> Hi, Tetsuya >> >> I have one question, as the code shows, in pci_unmap_device(), will >> check pt_driver. >> >> But assume that, we are now try to detach a vfio device, after print out >> a error message of unsupported, the does this port workable? >> >> I think this port will unworkable, am I right? >> >> But actually, we should keep it workable. >> >> My suggestion is to add a check in rte_eth_dev_check_detachable() for >> pci_device port. > Hi Michael, > > I appreciate your comment. > In the function called "rte_eal_dev_detach_pdev()", > "rte_eth_dev_check_detachable()" has been already checked. What I mean is check the pt_driver for pci_dev in rte_eth_dev_check_detachable(), so that hotplug framework will not affect vfio devices, just as I reply in another mail. Current logic will affect vfio devices if try to detach( Not do the really test, just the logic shows), am I right? Thanks, Michael > But in the future, someone may want to reuse > "rte_eal_pci_close_one_driver()". > So I will add the checking like your suggestion. > > Thanks, > Tetsuya > >> Thanks >> Michael >> >>> + >>> + return 0; >>> + } >>> + /* return positive value if driver is not found */ >>> + return 1; >>> +} >>> +#else /* ENABLE_HOTPLUG */ >>> +int >>> +rte_eal_pci_close_one_driver(struct rte_pci_driver *dr __rte_unused, >>> + struct rte_pci_device *dev __rte_unused) >>> +{ >>> + RTE_LOG(ERR, EAL, "Hotplug support isn't enabled\n"); >>> + return -1; >>> +} >>> +#endif /* ENABLE_HOTPLUG */ >>> + >>> /* Init the PCI EAL subsystem */ >>> int >>> rte_eal_pci_init(void) > >
On 2/1/2015 12:02 PM, Tetsuya Mukawa wrote: > The patch introduces following commands. > - port attach [ident] > - port detach [port_id] > - attach: attaching a port > - detach: detaching a port > - ident: pci address of physical device. > Or device name and paramerters of virtual device. > (ex. 0000:02:00.0, eth_pcap0,iface=eth0) > - port_id: port identifier > > v5: > - Add testpmd documentation. > (Thanks to Iremonger, Bernard) > v4: > - Fix strings of command help. > > Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp> > --- > app/test-pmd/cmdline.c | 133 +++++++++++++++---- > app/test-pmd/config.c | 116 +++++++++------- > app/test-pmd/parameters.c | 22 ++- > app/test-pmd/testpmd.c | 199 +++++++++++++++++++++------- > app/test-pmd/testpmd.h | 18 ++- > doc/guides/testpmd_app_ug/testpmd_funcs.rst | 57 ++++++++ > 6 files changed, 415 insertions(+), 130 deletions(-) > > diff --git a/app/test-pmd/cmdline.c b/app/test-pmd/cmdline.c > index 4beb404..2f813d8 100644 > --- a/app/test-pmd/cmdline.c > +++ b/app/test-pmd/cmdline.c > @@ -572,6 +572,12 @@ static void cmd_help_long_parsed(void *parsed_result, > "port close (port_id|all)\n" > " Close all ports or port_id.\n\n" > > + "port attach (ident)\n" > + " Attach physical or virtual dev by pci address or virtual device name\n\n" > + > + "port detach (port_id)\n" > + " Detach physical or virtual dev by port_id\n\n" > + > "port config (port_id|all)" > " speed (10|100|1000|10000|40000|auto)" > " duplex (half|full|auto)\n" > @@ -848,6 +854,89 @@ cmdline_parse_inst_t cmd_operate_specific_port = { > }, > }; > > +/* *** attach a specificied port *** */ > +struct cmd_operate_attach_port_result { > + cmdline_fixed_string_t port; > + cmdline_fixed_string_t keyword; > + cmdline_fixed_string_t identifier; > +}; > + > +static void cmd_operate_attach_port_parsed(void *parsed_result, > + __attribute__((unused)) struct cmdline *cl, > + __attribute__((unused)) void *data) > +{ > + struct cmd_operate_attach_port_result *res = parsed_result; > + > + if (!strcmp(res->keyword, "attach")) > + attach_port(res->identifier); > + else > + printf("Unknown parameter\n"); > +} > + > +cmdline_parse_token_string_t cmd_operate_attach_port_port = > + TOKEN_STRING_INITIALIZER(struct cmd_operate_attach_port_result, > + port, "port"); > +cmdline_parse_token_string_t cmd_operate_attach_port_keyword = > + TOKEN_STRING_INITIALIZER(struct cmd_operate_attach_port_result, > + keyword, "attach"); > +cmdline_parse_token_string_t cmd_operate_attach_port_identifier = > + TOKEN_STRING_INITIALIZER(struct cmd_operate_attach_port_result, > + identifier, NULL); > + > +cmdline_parse_inst_t cmd_operate_attach_port = { > + .f = cmd_operate_attach_port_parsed, > + .data = NULL, > + .help_str = "port attach identifier, " > + "identifier: pci address or virtual dev name", > + .tokens = { > + (void *)&cmd_operate_attach_port_port, > + (void *)&cmd_operate_attach_port_keyword, > + (void *)&cmd_operate_attach_port_identifier, > + NULL, > + }, > +}; > + > +/* *** detach a specificied port *** */ > +struct cmd_operate_detach_port_result { > + cmdline_fixed_string_t port; > + cmdline_fixed_string_t keyword; > + uint8_t port_id; > +}; > + > +static void cmd_operate_detach_port_parsed(void *parsed_result, > + __attribute__((unused)) struct cmdline *cl, > + __attribute__((unused)) void *data) > +{ > + struct cmd_operate_detach_port_result *res = parsed_result; > + > + if (!strcmp(res->keyword, "detach")) > + detach_port(res->port_id); > + else > + printf("Unknown parameter\n"); > +} > + > +cmdline_parse_token_string_t cmd_operate_detach_port_port = > + TOKEN_STRING_INITIALIZER(struct cmd_operate_detach_port_result, > + port, "port"); > +cmdline_parse_token_string_t cmd_operate_detach_port_keyword = > + TOKEN_STRING_INITIALIZER(struct cmd_operate_detach_port_result, > + keyword, "detach"); > +cmdline_parse_token_num_t cmd_operate_detach_port_port_id = > + TOKEN_NUM_INITIALIZER(struct cmd_operate_detach_port_result, > + port_id, UINT8); > + > +cmdline_parse_inst_t cmd_operate_detach_port = { > + .f = cmd_operate_detach_port_parsed, > + .data = NULL, > + .help_str = "port detach port_id", > + .tokens = { > + (void *)&cmd_operate_detach_port_port, > + (void *)&cmd_operate_detach_port_keyword, > + (void *)&cmd_operate_detach_port_port_id, > + NULL, > + }, > +}; > + > /* *** configure speed for all ports *** */ > struct cmd_config_speed_all { > cmdline_fixed_string_t port; > @@ -902,7 +991,7 @@ cmd_config_speed_all_parsed(void *parsed_result, > return; > } > > - for (pid = 0; pid < nb_ports; pid++) { > + FOREACH_PORT(pid, ports) { > ports[pid].dev_conf.link_speed = link_speed; > ports[pid].dev_conf.link_duplex = link_duplex; > } > @@ -970,10 +1059,8 @@ cmd_config_speed_specific_parsed(void *parsed_result, > return; > } > > - if (res->id >= nb_ports) { > - printf("Port id %d must be less than %d\n", res->id, nb_ports); > + if (port_id_is_invalid(res->id, ENABLED_WARN)) > return; > - } > > if (!strcmp(res->value1, "10")) > link_speed = ETH_LINK_SPEED_10; > @@ -1533,7 +1620,7 @@ cmd_config_rxtx_queue_parsed(void *parsed_result, > return; > } > > - if (port_id_is_invalid(res->portid)) > + if (port_id_is_invalid(res->portid, ENABLED_WARN)) > return; > > if (port_is_started(res->portid) != 1) { > @@ -2876,7 +2963,7 @@ cmd_tx_cksum_parsed(void *parsed_result, > uint16_t ol_flags, mask = 0; > struct rte_eth_dev_info dev_info; > > - if (port_id_is_invalid(res->port_id)) { > + if (port_id_is_invalid(res->port_id, ENABLED_WARN)) { > printf("invalid port %d\n", res->port_id); > return; > } > @@ -3003,7 +3090,7 @@ cmd_tso_set_parsed(void *parsed_result, > struct cmd_tso_set_result *res = parsed_result; > struct rte_eth_dev_info dev_info; > > - if (port_id_is_invalid(res->port_id)) > + if (port_id_is_invalid(res->port_id, ENABLED_WARN)) > return; > > if (!strcmp(res->mode, "set")) > @@ -3979,10 +4066,8 @@ static void cmd_set_bond_mac_addr_parsed(void *parsed_result, > struct cmd_set_bond_mac_addr_result *res = parsed_result; > int ret; > > - if (res->port_num >= nb_ports) { > - printf("Port id %d must be less than %d\n", res->port_num, nb_ports); > + if (port_id_is_invalid(res->port_num, ENABLED_WARN)) > return; > - } > > ret = rte_eth_bond_mac_address_set(res->port_num, &res->address); > > @@ -4219,7 +4304,7 @@ static void cmd_set_promisc_mode_parsed(void *parsed_result, > > /* all ports */ > if (allports) { > - for (i = 0; i < nb_ports; i++) { > + FOREACH_PORT(i, ports) { > if (enable) > rte_eth_promiscuous_enable(i); > else > @@ -4299,7 +4384,7 @@ static void cmd_set_allmulti_mode_parsed(void *parsed_result, > > /* all ports */ > if (allports) { > - for (i = 0; i < nb_ports; i++) { > + FOREACH_PORT(i, ports) { > if (enable) > rte_eth_allmulticast_enable(i); > else > @@ -5484,25 +5569,25 @@ static void cmd_showportall_parsed(void *parsed_result, > struct cmd_showportall_result *res = parsed_result; > if (!strcmp(res->show, "clear")) { > if (!strcmp(res->what, "stats")) > - for (i = 0; i < nb_ports; i++) > + FOREACH_PORT(i, ports) > nic_stats_clear(i); > else if (!strcmp(res->what, "xstats")) > - for (i = 0; i < nb_ports; i++) > + FOREACH_PORT(i, ports) > nic_xstats_clear(i); > } else if (!strcmp(res->what, "info")) > - for (i = 0; i < nb_ports; i++) > + FOREACH_PORT(i, ports) > port_infos_display(i); > else if (!strcmp(res->what, "stats")) > - for (i = 0; i < nb_ports; i++) > + FOREACH_PORT(i, ports) > nic_stats_display(i); > else if (!strcmp(res->what, "xstats")) > - for (i = 0; i < nb_ports; i++) > + FOREACH_PORT(i, ports) > nic_xstats_display(i); > else if (!strcmp(res->what, "fdir")) > - for (i = 0; i < nb_ports; i++) > + FOREACH_PORT(i, ports) > fdir_get_infos(i); > else if (!strcmp(res->what, "stat_qmap")) > - for (i = 0; i < nb_ports; i++) > + FOREACH_PORT(i, ports) > nic_stats_mapping_display(i); > } > > @@ -8756,6 +8841,8 @@ cmdline_parse_ctx_t main_ctx[] = { > (cmdline_parse_inst_t *)&cmd_set_qmap, > (cmdline_parse_inst_t *)&cmd_operate_port, > (cmdline_parse_inst_t *)&cmd_operate_specific_port, > + (cmdline_parse_inst_t *)&cmd_operate_attach_port, > + (cmdline_parse_inst_t *)&cmd_operate_detach_port, > (cmdline_parse_inst_t *)&cmd_config_speed_all, > (cmdline_parse_inst_t *)&cmd_config_speed_specific, > (cmdline_parse_inst_t *)&cmd_config_rx_tx, > @@ -8830,7 +8917,7 @@ prompt(void) > static void > cmd_reconfig_device_queue(portid_t id, uint8_t dev, uint8_t queue) > { > - if (id < nb_ports) { > + if (!port_id_is_invalid(id, DISABLED_WARN)) { > /* check if need_reconfig has been set to 1 */ > if (ports[id].need_reconfig == 0) > ports[id].need_reconfig = dev; > @@ -8840,7 +8927,7 @@ cmd_reconfig_device_queue(portid_t id, uint8_t dev, uint8_t queue) > } else { > portid_t pid; > > - for (pid = 0; pid < nb_ports; pid++) { > + FOREACH_PORT(pid, ports) { > /* check if need_reconfig has been set to 1 */ > if (ports[pid].need_reconfig == 0) > ports[pid].need_reconfig = dev; > @@ -8858,10 +8945,8 @@ bypass_is_supported(portid_t port_id) > struct rte_port *port; > struct rte_pci_id *pci_id; > > - if (port_id >= nb_ports) { > - printf("\tPort id must be less than %d.\n", nb_ports); > + if (port_id_is_invalid(port_id, ENABLED_WARN)) > return 0; > - } > > /* Get the device id. */ > port = &ports[port_id]; > diff --git a/app/test-pmd/config.c b/app/test-pmd/config.c > index c40f819..32d8f9a 100644 > --- a/app/test-pmd/config.c > +++ b/app/test-pmd/config.c > @@ -124,11 +124,15 @@ nic_stats_display(portid_t port_id) > struct rte_eth_stats stats; > struct rte_port *port = &ports[port_id]; > uint8_t i; > + portid_t pid; > > static const char *nic_stats_border = "########################"; > > - if (port_id >= nb_ports) { > - printf("Invalid port, range is [0, %d]\n", nb_ports - 1); > + if (port_id_is_invalid(port_id, ENABLED_WARN)) { > + printf("Valid port range is [0"); > + FOREACH_PORT(pid, ports) > + printf(", %d", pid); > + printf("]\n"); > return; > } > rte_eth_stats_get(port_id, &stats); > @@ -201,8 +205,13 @@ nic_stats_display(portid_t port_id) > void > nic_stats_clear(portid_t port_id) > { > - if (port_id >= nb_ports) { > - printf("Invalid port, range is [0, %d]\n", nb_ports - 1); > + portid_t pid; > + > + if (port_id_is_invalid(port_id, ENABLED_WARN)) { > + printf("Valid port range is [0"); > + FOREACH_PORT(pid, ports) > + printf(", %d", pid); > + printf("]\n"); > return; > } > rte_eth_stats_reset(port_id); > @@ -249,11 +258,15 @@ nic_stats_mapping_display(portid_t port_id) > { > struct rte_port *port = &ports[port_id]; > uint16_t i; > + portid_t pid; > > static const char *nic_stats_mapping_border = "########################"; > > - if (port_id >= nb_ports) { > - printf("Invalid port, range is [0, %d]\n", nb_ports - 1); > + if (port_id_is_invalid(port_id, ENABLED_WARN)) { > + printf("Valid port range is [0"); > + FOREACH_PORT(pid, ports) > + printf(", %d", pid); > + printf("]\n"); > return; > } > > @@ -302,9 +315,13 @@ port_infos_display(portid_t port_id) > int vlan_offload; > struct rte_mempool * mp; > static const char *info_border = "*********************"; > + portid_t pid; > > - if (port_id >= nb_ports) { > - printf("Invalid port, range is [0, %d]\n", nb_ports - 1); > + if (port_id_is_invalid(port_id, ENABLED_WARN)) { > + printf("Valid port range is [0"); > + FOREACH_PORT(pid, ports) > + printf(", %d", pid); > + printf("]\n"); > return; > } > port = &ports[port_id]; > @@ -362,11 +379,14 @@ port_infos_display(portid_t port_id) > } > > int > -port_id_is_invalid(portid_t port_id) > +port_id_is_invalid(portid_t port_id, enum print_warning warning) > { > - if (port_id < nb_ports) > + if (ports[port_id].enabled) > return 0; > - printf("Invalid port %d (must be < nb_ports=%d)\n", port_id, nb_ports); > + > + if (warning == ENABLED_WARN) > + printf("Invalid port %d\n", port_id); > + > return 1; > } > > @@ -425,7 +445,7 @@ port_reg_bit_display(portid_t port_id, uint32_t reg_off, uint8_t bit_x) > uint32_t reg_v; > > > - if (port_id_is_invalid(port_id)) > + if (port_id_is_invalid(port_id, ENABLED_WARN)) > return; > if (port_reg_off_is_invalid(port_id, reg_off)) > return; > @@ -444,7 +464,7 @@ port_reg_bit_field_display(portid_t port_id, uint32_t reg_off, > uint8_t l_bit; > uint8_t h_bit; > > - if (port_id_is_invalid(port_id)) > + if (port_id_is_invalid(port_id, ENABLED_WARN)) > return; > if (port_reg_off_is_invalid(port_id, reg_off)) > return; > @@ -471,7 +491,7 @@ port_reg_display(portid_t port_id, uint32_t reg_off) > { > uint32_t reg_v; > > - if (port_id_is_invalid(port_id)) > + if (port_id_is_invalid(port_id, ENABLED_WARN)) > return; > if (port_reg_off_is_invalid(port_id, reg_off)) > return; > @@ -485,7 +505,7 @@ port_reg_bit_set(portid_t port_id, uint32_t reg_off, uint8_t bit_pos, > { > uint32_t reg_v; > > - if (port_id_is_invalid(port_id)) > + if (port_id_is_invalid(port_id, ENABLED_WARN)) > return; > if (port_reg_off_is_invalid(port_id, reg_off)) > return; > @@ -513,7 +533,7 @@ port_reg_bit_field_set(portid_t port_id, uint32_t reg_off, > uint8_t l_bit; > uint8_t h_bit; > > - if (port_id_is_invalid(port_id)) > + if (port_id_is_invalid(port_id, ENABLED_WARN)) > return; > if (port_reg_off_is_invalid(port_id, reg_off)) > return; > @@ -547,7 +567,7 @@ port_reg_bit_field_set(portid_t port_id, uint32_t reg_off, > void > port_reg_set(portid_t port_id, uint32_t reg_off, uint32_t reg_v) > { > - if (port_id_is_invalid(port_id)) > + if (port_id_is_invalid(port_id, ENABLED_WARN)) > return; > if (port_reg_off_is_invalid(port_id, reg_off)) > return; > @@ -560,7 +580,7 @@ port_mtu_set(portid_t port_id, uint16_t mtu) > { > int diag; > > - if (port_id_is_invalid(port_id)) > + if (port_id_is_invalid(port_id, ENABLED_WARN)) > return; > diag = rte_eth_dev_set_mtu(port_id, mtu); > if (diag == 0) > @@ -723,7 +743,7 @@ rx_ring_desc_display(portid_t port_id, queueid_t rxq_id, uint16_t rxd_id) > { > const struct rte_memzone *rx_mz; > > - if (port_id_is_invalid(port_id)) > + if (port_id_is_invalid(port_id, ENABLED_WARN)) > return; > if (rx_queue_id_is_invalid(rxq_id)) > return; > @@ -740,7 +760,7 @@ tx_ring_desc_display(portid_t port_id, queueid_t txq_id, uint16_t txd_id) > { > const struct rte_memzone *tx_mz; > > - if (port_id_is_invalid(port_id)) > + if (port_id_is_invalid(port_id, ENABLED_WARN)) > return; > if (tx_queue_id_is_invalid(txq_id)) > return; > @@ -796,7 +816,7 @@ port_rss_reta_info(portid_t port_id, > uint16_t i, idx, shift; > int ret; > > - if (port_id_is_invalid(port_id)) > + if (port_id_is_invalid(port_id, ENABLED_WARN)) > return; > > ret = rte_eth_dev_rss_reta_query(port_id, reta_conf, nb_entries); > @@ -828,7 +848,7 @@ port_rss_hash_conf_show(portid_t port_id, int show_rss_key) > uint8_t i; > int diag; > > - if (port_id_is_invalid(port_id)) > + if (port_id_is_invalid(port_id, ENABLED_WARN)) > return; > /* Get RSS hash key if asked to display it */ > rss_conf.rss_key = (show_rss_key) ? rss_key : NULL; > @@ -1406,12 +1426,8 @@ set_fwd_ports_list(unsigned int *portlist, unsigned int nb_pt) > again: > for (i = 0; i < nb_pt; i++) { > port_id = (portid_t) portlist[i]; > - if (port_id >= nb_ports) { > - printf("Invalid port id %u >= %u\n", > - (unsigned int) port_id, > - (unsigned int) nb_ports); > + if (port_id_is_invalid(port_id, ENABLED_WARN)) > return; > - } > if (record_now) > fwd_ports_ids[i] = port_id; > } > @@ -1569,7 +1585,7 @@ vlan_extend_set(portid_t port_id, int on) > int diag; > int vlan_offload; > > - if (port_id_is_invalid(port_id)) > + if (port_id_is_invalid(port_id, ENABLED_WARN)) > return; > > vlan_offload = rte_eth_dev_get_vlan_offload(port_id); > @@ -1591,7 +1607,7 @@ rx_vlan_strip_set(portid_t port_id, int on) > int diag; > int vlan_offload; > > - if (port_id_is_invalid(port_id)) > + if (port_id_is_invalid(port_id, ENABLED_WARN)) > return; > > vlan_offload = rte_eth_dev_get_vlan_offload(port_id); > @@ -1612,7 +1628,7 @@ rx_vlan_strip_set_on_queue(portid_t port_id, uint16_t queue_id, int on) > { > int diag; > > - if (port_id_is_invalid(port_id)) > + if (port_id_is_invalid(port_id, ENABLED_WARN)) > return; > > diag = rte_eth_dev_set_vlan_strip_on_queue(port_id, queue_id, on); > @@ -1627,7 +1643,7 @@ rx_vlan_filter_set(portid_t port_id, int on) > int diag; > int vlan_offload; > > - if (port_id_is_invalid(port_id)) > + if (port_id_is_invalid(port_id, ENABLED_WARN)) > return; > > vlan_offload = rte_eth_dev_get_vlan_offload(port_id); > @@ -1648,7 +1664,7 @@ rx_vft_set(portid_t port_id, uint16_t vlan_id, int on) > { > int diag; > > - if (port_id_is_invalid(port_id)) > + if (port_id_is_invalid(port_id, ENABLED_WARN)) > return; > if (vlan_id_is_invalid(vlan_id)) > return; > @@ -1665,7 +1681,7 @@ rx_vlan_all_filter_set(portid_t port_id, int on) > { > uint16_t vlan_id; > > - if (port_id_is_invalid(port_id)) > + if (port_id_is_invalid(port_id, ENABLED_WARN)) > return; > for (vlan_id = 0; vlan_id < 4096; vlan_id++) > rx_vft_set(port_id, vlan_id, on); > @@ -1675,7 +1691,7 @@ void > vlan_tpid_set(portid_t port_id, uint16_t tp_id) > { > int diag; > - if (port_id_is_invalid(port_id)) > + if (port_id_is_invalid(port_id, ENABLED_WARN)) > return; > > diag = rte_eth_dev_set_vlan_ether_type(port_id, tp_id); > @@ -1690,7 +1706,7 @@ vlan_tpid_set(portid_t port_id, uint16_t tp_id) > void > tx_vlan_set(portid_t port_id, uint16_t vlan_id) > { > - if (port_id_is_invalid(port_id)) > + if (port_id_is_invalid(port_id, ENABLED_WARN)) > return; > if (vlan_id_is_invalid(vlan_id)) > return; > @@ -1701,7 +1717,7 @@ tx_vlan_set(portid_t port_id, uint16_t vlan_id) > void > tx_vlan_reset(portid_t port_id) > { > - if (port_id_is_invalid(port_id)) > + if (port_id_is_invalid(port_id, ENABLED_WARN)) > return; > ports[port_id].tx_ol_flags &= ~TESTPMD_TX_OFFLOAD_INSERT_VLAN; > } > @@ -1709,7 +1725,7 @@ tx_vlan_reset(portid_t port_id) > void > tx_vlan_pvid_set(portid_t port_id, uint16_t vlan_id, int on) > { > - if (port_id_is_invalid(port_id)) > + if (port_id_is_invalid(port_id, ENABLED_WARN)) > return; > > rte_eth_dev_set_vlan_pvid(port_id, vlan_id, on); > @@ -1721,7 +1737,7 @@ set_qmap(portid_t port_id, uint8_t is_rx, uint16_t queue_id, uint8_t map_value) > uint16_t i; > uint8_t existing_mapping_found = 0; > > - if (port_id_is_invalid(port_id)) > + if (port_id_is_invalid(port_id, ENABLED_WARN)) > return; > > if (is_rx ? (rx_queue_id_is_invalid(queue_id)) : (tx_queue_id_is_invalid(queue_id))) > @@ -1773,7 +1789,7 @@ fdir_add_signature_filter(portid_t port_id, uint8_t queue_id, > { > int diag; > > - if (port_id_is_invalid(port_id)) > + if (port_id_is_invalid(port_id, ENABLED_WARN)) > return; > > diag = rte_eth_dev_fdir_add_signature_filter(port_id, fdir_filter, > @@ -1791,7 +1807,7 @@ fdir_update_signature_filter(portid_t port_id, uint8_t queue_id, > { > int diag; > > - if (port_id_is_invalid(port_id)) > + if (port_id_is_invalid(port_id, ENABLED_WARN)) > return; > > diag = rte_eth_dev_fdir_update_signature_filter(port_id, fdir_filter, > @@ -1809,7 +1825,7 @@ fdir_remove_signature_filter(portid_t port_id, > { > int diag; > > - if (port_id_is_invalid(port_id)) > + if (port_id_is_invalid(port_id, ENABLED_WARN)) > return; > > diag = rte_eth_dev_fdir_remove_signature_filter(port_id, fdir_filter); > @@ -1881,7 +1897,7 @@ fdir_get_infos(portid_t port_id) > > static const char *fdir_stats_border = "########################"; > > - if (port_id_is_invalid(port_id)) > + if (port_id_is_invalid(port_id, ENABLED_WARN)) > return; > ret = rte_eth_dev_filter_supported(port_id, RTE_ETH_FILTER_FDIR); > if (ret < 0) { > @@ -1955,7 +1971,7 @@ fdir_add_perfect_filter(portid_t port_id, uint16_t soft_id, uint8_t queue_id, > { > int diag; > > - if (port_id_is_invalid(port_id)) > + if (port_id_is_invalid(port_id, ENABLED_WARN)) > return; > > diag = rte_eth_dev_fdir_add_perfect_filter(port_id, fdir_filter, > @@ -1973,7 +1989,7 @@ fdir_update_perfect_filter(portid_t port_id, uint16_t soft_id, uint8_t queue_id, > { > int diag; > > - if (port_id_is_invalid(port_id)) > + if (port_id_is_invalid(port_id, ENABLED_WARN)) > return; > > diag = rte_eth_dev_fdir_update_perfect_filter(port_id, fdir_filter, > @@ -1991,7 +2007,7 @@ fdir_remove_perfect_filter(portid_t port_id, uint16_t soft_id, > { > int diag; > > - if (port_id_is_invalid(port_id)) > + if (port_id_is_invalid(port_id, ENABLED_WARN)) > return; > > diag = rte_eth_dev_fdir_remove_perfect_filter(port_id, fdir_filter, > @@ -2008,7 +2024,7 @@ fdir_set_masks(portid_t port_id, struct rte_fdir_masks *fdir_masks) > { > int diag; > > - if (port_id_is_invalid(port_id)) > + if (port_id_is_invalid(port_id, ENABLED_WARN)) > return; > > diag = rte_eth_dev_fdir_set_masks(port_id, fdir_masks); > @@ -2085,7 +2101,7 @@ set_vf_traffic(portid_t port_id, uint8_t is_rx, uint16_t vf, uint8_t on) > { > int diag; > > - if (port_id_is_invalid(port_id)) > + if (port_id_is_invalid(port_id, ENABLED_WARN)) > return; > if (is_rx) > diag = rte_eth_dev_set_vf_rx(port_id,vf,on); > @@ -2107,7 +2123,7 @@ set_vf_rx_vlan(portid_t port_id, uint16_t vlan_id, uint64_t vf_mask, uint8_t on) > { > int diag; > > - if (port_id_is_invalid(port_id)) > + if (port_id_is_invalid(port_id, ENABLED_WARN)) > return; > if (vlan_id_is_invalid(vlan_id)) > return; > @@ -2124,7 +2140,7 @@ set_queue_rate_limit(portid_t port_id, uint16_t queue_idx, uint16_t rate) > int diag; > struct rte_eth_link link; > > - if (port_id_is_invalid(port_id)) > + if (port_id_is_invalid(port_id, ENABLED_WARN)) > return 1; > rte_eth_link_get_nowait(port_id, &link); > if (rate > link.link_speed) { > @@ -2149,7 +2165,7 @@ set_vf_rate_limit(portid_t port_id, uint16_t vf, uint16_t rate, uint64_t q_msk) > if (q_msk == 0) > return 0; > > - if (port_id_is_invalid(port_id)) > + if (port_id_is_invalid(port_id, ENABLED_WARN)) > return 1; > rte_eth_link_get_nowait(port_id, &link); > if (rate > link.link_speed) { > diff --git a/app/test-pmd/parameters.c b/app/test-pmd/parameters.c > index adf3203..6f2af18 100644 > --- a/app/test-pmd/parameters.c > +++ b/app/test-pmd/parameters.c > @@ -376,6 +376,7 @@ parse_portnuma_config(const char *q_arg) > }; > unsigned long int_fld[_NUM_FLD]; > char *str_fld[_NUM_FLD]; > + portid_t pid; > > /* reset from value set at definition */ > while ((p = strchr(p0,'(')) != NULL) { > @@ -397,8 +398,11 @@ parse_portnuma_config(const char *q_arg) > return -1; > } > port_id = (uint8_t)int_fld[FLD_PORT]; > - if (port_id >= nb_ports) { > - printf("Invalid port, range is [0, %d]\n", nb_ports - 1); > + if (port_id_is_invalid(port_id, ENABLED_WARN)) { > + printf("Valid port range is [0"); > + FOREACH_PORT(pid, ports) > + printf(", %d", pid); > + printf("]\n"); > return -1; > } > socket_id = (uint8_t)int_fld[FLD_SOCKET]; > @@ -429,6 +433,7 @@ parse_ringnuma_config(const char *q_arg) > }; > unsigned long int_fld[_NUM_FLD]; > char *str_fld[_NUM_FLD]; > + portid_t pid; > #define RX_RING_ONLY 0x1 > #define TX_RING_ONLY 0x2 > #define RXTX_RING 0x3 > @@ -453,8 +458,11 @@ parse_ringnuma_config(const char *q_arg) > return -1; > } > port_id = (uint8_t)int_fld[FLD_PORT]; > - if (port_id >= nb_ports) { > - printf("Invalid port, range is [0, %d]\n", nb_ports - 1); > + if (port_id_is_invalid(port_id, ENABLED_WARN)) { > + printf("Valid port range is [0"); > + FOREACH_PORT(pid, ports) > + printf(", %d", pid); > + printf("]\n"); > return -1; > } > socket_id = (uint8_t)int_fld[FLD_SOCKET]; > @@ -626,12 +634,12 @@ launch_args_parse(int argc, char** argv) > #endif > if (!strcmp(lgopts[opt_idx].name, "nb-ports")) { > n = atoi(optarg); > - if (n > 0 && n <= nb_ports) > + if (n > 0 && > + !port_id_is_invalid(n, DISABLED_WARN)) > nb_fwd_ports = (uint8_t) n; > else > rte_exit(EXIT_FAILURE, > - "nb-ports should be > 0 and <= %d\n", > - nb_ports); > + "Invalid port %d\n", n); > } > if (!strcmp(lgopts[opt_idx].name, "nb-cores")) { > n = atoi(optarg); > diff --git a/app/test-pmd/testpmd.c b/app/test-pmd/testpmd.c > index 773b8af..c18c1a9 100644 > --- a/app/test-pmd/testpmd.c > +++ b/app/test-pmd/testpmd.c > @@ -71,6 +71,7 @@ > #include <rte_pci.h> > #include <rte_ether.h> > #include <rte_ethdev.h> > +#include <rte_dev.h> > #include <rte_string_fns.h> > #ifdef RTE_LIBRTE_PMD_XENVIRT > #include <rte_eth_xenvirt.h> > @@ -315,7 +316,7 @@ uint16_t nb_rx_queue_stats_mappings = 0; > > /* Forward function declarations */ > static void map_port_queue_stats_mapping_registers(uint8_t pi, struct rte_port *port); > -static void check_all_ports_link_status(uint8_t port_num, uint32_t port_mask); > +static void check_all_ports_link_status(uint32_t port_mask); > > /* > * Check if all the ports are started. > @@ -324,6 +325,20 @@ static void check_all_ports_link_status(uint8_t port_num, uint32_t port_mask); > static int all_ports_started(void); > > /* > + * Find next enabled port > + */ > +portid_t > +find_next_port(portid_t p, struct rte_port *ports, int size) > +{ > + if (ports == NULL) > + rte_exit(-EINVAL, "failed to find a next port id\n"); > + > + while ((ports[p].enabled == 0) && (p < size)) > + p++; > + return p; > +} > + > +/* > * Setup default configuration. > */ > static void > @@ -552,7 +567,8 @@ init_config(void) > + RTE_TEST_TX_DESC_MAX + MAX_PKT_BURST; > > if (!numa_support) > - nb_mbuf_per_pool = (nb_mbuf_per_pool * nb_ports); > + nb_mbuf_per_pool = > + (nb_mbuf_per_pool * RTE_MAX_ETHPORTS); > } > > if (!numa_support) { > @@ -565,14 +581,19 @@ init_config(void) > > /* Configuration of Ethernet ports. */ > ports = rte_zmalloc("testpmd: ports", > - sizeof(struct rte_port) * nb_ports, > + sizeof(struct rte_port) * RTE_MAX_ETHPORTS, > RTE_CACHE_LINE_SIZE); > if (ports == NULL) { > - rte_exit(EXIT_FAILURE, "rte_zmalloc(%d struct rte_port) " > - "failed\n", nb_ports); > + rte_exit(EXIT_FAILURE, > + "rte_zmalloc(%d struct rte_port) failed\n", > + RTE_MAX_ETHPORTS); > } > > - for (pid = 0; pid < nb_ports; pid++) { > + /* enabled allocated ports */ > + for (pid = 0; pid < nb_ports; pid++) > + ports[pid].enabled = 1; > + > + FOREACH_PORT(pid, ports) { > port = &ports[pid]; > rte_eth_dev_info_get(pid, &port->dev_info); > > @@ -602,8 +623,7 @@ init_config(void) > nb_mbuf_per_pool = nb_mbuf_per_pool/nb_ports; > > for (i = 0; i < MAX_SOCKET; i++) { > - nb_mbuf = (nb_mbuf_per_pool * > - port_per_socket[i]); > + nb_mbuf = (nb_mbuf_per_pool * RTE_MAX_ETHPORTS); > if (nb_mbuf) > mbuf_pool_create(mbuf_data_size, > nb_mbuf,i); > @@ -635,14 +655,6 @@ reconfig(portid_t new_port_id, unsigned socket_id) > struct rte_port *port; > > /* Reconfiguration of Ethernet ports. */ > - ports = rte_realloc(ports, > - sizeof(struct rte_port) * nb_ports, > - RTE_CACHE_LINE_SIZE); > - if (ports == NULL) { > - rte_exit(EXIT_FAILURE, "rte_realloc(%d struct rte_port) failed\n", > - nb_ports); > - } > - > port = &ports[new_port_id]; > rte_eth_dev_info_get(new_port_id, &port->dev_info); > > @@ -663,7 +675,7 @@ init_fwd_streams(void) > streamid_t sm_id, nb_fwd_streams_new; > > /* set socket id according to numa or not */ > - for (pid = 0; pid < nb_ports; pid++) { > + FOREACH_PORT(pid, ports) { > port = &ports[pid]; > if (nb_rxq > port->dev_info.max_rx_queues) { > printf("Fail: nb_rxq(%d) is greater than " > @@ -1264,7 +1276,7 @@ all_ports_started(void) > portid_t pi; > struct rte_port *port; > > - for (pi = 0; pi < nb_ports; pi++) { > + FOREACH_PORT(pi, ports) { > port = &ports[pi]; > /* Check if there is a port which is not started */ > if (port->port_status != RTE_PORT_STARTED) > @@ -1276,6 +1288,45 @@ all_ports_started(void) > } > > int > +all_ports_stopped(void) > +{ > + portid_t pi; > + struct rte_port *port; > + > + FOREACH_PORT(pi, ports) { > + port = &ports[pi]; > + if (port->port_status != RTE_PORT_STOPPED) > + return 0; > + } > + > + return 1; > +} > + > +int > +port_is_started(portid_t port_id) > +{ > + if (port_id_is_invalid(port_id, ENABLED_WARN)) > + return 0; > + > + if (ports[port_id].port_status != RTE_PORT_STARTED) > + return 0; > + > + return 1; > +} > + > +static int > +port_is_closed(portid_t port_id) > +{ > + if (port_id_is_invalid(port_id, ENABLED_WARN)) > + return 0; > + > + if (ports[port_id].port_status != RTE_PORT_CLOSED) > + return 0; > + > + return 1; > +} > + > +int > start_port(portid_t pid) > { > int diag, need_check_link_status = 0; > @@ -1296,8 +1347,8 @@ start_port(portid_t pid) > > if(dcb_config) > dcb_test = 1; > - for (pi = 0; pi < nb_ports; pi++) { > - if (pid < nb_ports && pid != pi) > + FOREACH_PORT(pi, ports) { > + if (!port_id_is_invalid(pid, DISABLED_WARN) && pid != pi) Here may it be: if (!port_id_is_invalid(pid, DISABLED_WARN) && (pid != pi || pid == RET_PORT_ALL)) Otherwise no port will be start by default. Thanks, Michael > continue; > > port = &ports[pi]; > @@ -1421,7 +1472,7 @@ start_port(portid_t pid) > } > > if (need_check_link_status && !no_link_check) > - check_all_ports_link_status(nb_ports, RTE_PORT_ALL); > + check_all_ports_link_status(RTE_PORT_ALL); > else > printf("Please stop the ports first\n"); > > @@ -1446,8 +1497,8 @@ stop_port(portid_t pid) > } > printf("Stopping ports...\n"); > > - for (pi = 0; pi < nb_ports; pi++) { > - if (pid < nb_ports && pid != pi) > + FOREACH_PORT(pi, ports) { > + if (!port_id_is_invalid(pid, DISABLED_WARN) && pid != pi) > continue; > > port = &ports[pi]; > @@ -1463,7 +1514,7 @@ stop_port(portid_t pid) > need_check_link_status = 1; > } > if (need_check_link_status && !no_link_check) > - check_all_ports_link_status(nb_ports, RTE_PORT_ALL); > + check_all_ports_link_status(RTE_PORT_ALL); > > printf("Done\n"); > } > @@ -1481,8 +1532,8 @@ close_port(portid_t pid) > > printf("Closing ports...\n"); > > - for (pi = 0; pi < nb_ports; pi++) { > - if (pid < nb_ports && pid != pi) > + FOREACH_PORT(pi, ports) { > + if (!port_id_is_invalid(pid, DISABLED_WARN) && pid != pi) > continue; > > port = &ports[pi]; > @@ -1502,31 +1553,83 @@ close_port(portid_t pid) > printf("Done\n"); > } > > -int > -all_ports_stopped(void) > +void > +attach_port(char *identifier) > { > - portid_t pi; > - struct rte_port *port; > + portid_t i, j, pi = 0; > > - for (pi = 0; pi < nb_ports; pi++) { > - port = &ports[pi]; > - if (port->port_status != RTE_PORT_STOPPED) > - return 0; > + printf("Attaching a new port...\n"); > + > + if (identifier == NULL) { > + printf("Invalid parameters are speficied\n"); > + return; > } > > - return 1; > + if (test_done == 0) { > + printf("Please stop forwarding first\n"); > + return; > + } > + > + if (rte_eal_dev_attach(identifier, &pi)) > + return; > + > + ports[pi].enabled = 1; > + reconfig(pi, rte_eth_dev_socket_id(pi)); > + rte_eth_promiscuous_enable(pi); > + > + nb_ports = rte_eth_dev_count(); > + > + /* set_default_fwd_ports_config(); */ > + bzero(fwd_ports_ids, sizeof(fwd_ports_ids)); > + i = 0; > + FOREACH_PORT(j, ports) { > + fwd_ports_ids[i] = j; > + i++; > + } > + nb_cfg_ports = nb_ports; > + nb_fwd_ports++; > + > + ports[pi].port_status = RTE_PORT_STOPPED; > + > + printf("Port %d is attached. Now total ports is %d\n", pi, nb_ports); > + printf("Done\n"); > } > > -int > -port_is_started(portid_t port_id) > +void > +detach_port(uint8_t port_id) > { > - if (port_id_is_invalid(port_id)) > - return -1; > + portid_t i, pi = 0; > + char name[RTE_ETH_NAME_MAX_LEN]; > > - if (ports[port_id].port_status != RTE_PORT_STARTED) > - return 0; > + printf("Detaching a port...\n"); > > - return 1; > + if (!port_is_closed(port_id)) { > + printf("Please close port first\n"); > + return; > + } > + > + rte_eth_promiscuous_disable(port_id); > + > + if (rte_eal_dev_detach(port_id, name)) > + return; > + > + ports[port_id].enabled = 0; > + nb_ports = rte_eth_dev_count(); > + > + /* set_default_fwd_ports_config(); */ > + bzero(fwd_ports_ids, sizeof(fwd_ports_ids)); > + i = 0; > + FOREACH_PORT(pi, ports) { > + fwd_ports_ids[i] = pi; > + i++; > + } > + nb_cfg_ports = nb_ports; > + nb_fwd_ports--; > + > + printf("Port '%s' is detached. Now total ports is %d\n", > + name, nb_ports); > + printf("Done\n"); > + return; > } > > void > @@ -1534,7 +1637,7 @@ pmd_test_exit(void) > { > portid_t pt_id; > > - for (pt_id = 0; pt_id < nb_ports; pt_id++) { > + FOREACH_PORT(pt_id, ports) { > printf("Stopping port %d...", pt_id); > fflush(stdout); > rte_eth_dev_close(pt_id); > @@ -1553,7 +1656,7 @@ struct pmd_test_command { > > /* Check the link status of all ports in up to 9s, and print them finally */ > static void > -check_all_ports_link_status(uint8_t port_num, uint32_t port_mask) > +check_all_ports_link_status(uint32_t port_mask) > { > #define CHECK_INTERVAL 100 /* 100ms */ > #define MAX_CHECK_TIME 90 /* 9s (90 * 100ms) in total */ > @@ -1564,7 +1667,7 @@ check_all_ports_link_status(uint8_t port_num, uint32_t port_mask) > fflush(stdout); > for (count = 0; count <= MAX_CHECK_TIME; count++) { > all_ports_up = 1; > - for (portid = 0; portid < port_num; portid++) { > + FOREACH_PORT(portid, ports) { > if ((port_mask & (1 << portid)) == 0) > continue; > memset(&link, 0, sizeof(link)); > @@ -1688,7 +1791,7 @@ init_port_config(void) > portid_t pid; > struct rte_port *port; > > - for (pid = 0; pid < nb_ports; pid++) { > + FOREACH_PORT(pid, ports) { > port = &ports[pid]; > port->dev_conf.rxmode = rx_mode; > port->dev_conf.fdir_conf = fdir_conf; > @@ -1877,7 +1980,7 @@ main(int argc, char** argv) > > nb_ports = (portid_t) rte_eth_dev_count(); > if (nb_ports == 0) > - rte_exit(EXIT_FAILURE, "No probed ethernet device\n"); > + RTE_LOG(WARNING, EAL, "No probed ethernet devices\n"); > > set_def_fwd_config(); > if (nb_lcores == 0) > @@ -1899,7 +2002,7 @@ main(int argc, char** argv) > rte_exit(EXIT_FAILURE, "Start ports failed\n"); > > /* set all ports to promiscuous mode by default */ > - for (port_id = 0; port_id < nb_ports; port_id++) > + FOREACH_PORT(port_id, ports) > rte_eth_promiscuous_enable(port_id); > > #ifdef RTE_LIBRTE_CMDLINE > diff --git a/app/test-pmd/testpmd.h b/app/test-pmd/testpmd.h > index 8f5e6c7..109c670 100644 > --- a/app/test-pmd/testpmd.h > +++ b/app/test-pmd/testpmd.h > @@ -134,6 +134,7 @@ struct fwd_stream { > * The data structure associated with each port. > */ > struct rte_port { > + uint8_t enabled; /**< Port enabled or not */ > struct rte_eth_dev_info dev_info; /**< PCI info + driver name */ > struct rte_eth_conf dev_conf; /**< Port configuration. */ > struct ether_addr eth_addr; /**< Port ethernet address */ > @@ -159,6 +160,14 @@ struct rte_port { > struct rte_eth_txconf tx_conf; /**< tx configuration */ > }; > > +extern portid_t __rte_unused > +find_next_port(portid_t p, struct rte_port *ports, int size); > + > +#define FOREACH_PORT(p, ports) \ > + for (p = find_next_port(0, ports, RTE_MAX_ETHPORTS); \ > + p < RTE_MAX_ETHPORTS; \ > + p = find_next_port(p + 1, ports, RTE_MAX_ETHPORTS)) > + > /** > * The data structure associated with each forwarding logical core. > * The logical cores are internally numbered by a core index from 0 to > @@ -515,6 +524,8 @@ int init_port_dcb_config(portid_t pid,struct dcb_config *dcb_conf); > int start_port(portid_t pid); > void stop_port(portid_t pid); > void close_port(portid_t pid); > +void attach_port(char *identifier); > +void detach_port(uint8_t port_id); > int all_ports_stopped(void); > int port_is_started(portid_t port_id); > void pmd_test_exit(void); > @@ -558,10 +569,15 @@ void get_ethertype_filter(uint8_t port_id, uint16_t index); > void get_2tuple_filter(uint8_t port_id, uint16_t index); > void get_5tuple_filter(uint8_t port_id, uint16_t index); > void get_flex_filter(uint8_t port_id, uint16_t index); > -int port_id_is_invalid(portid_t port_id); > int rx_queue_id_is_invalid(queueid_t rxq_id); > int tx_queue_id_is_invalid(queueid_t txq_id); > > +enum print_warning { > + ENABLED_WARN = 0, > + DISABLED_WARN > +}; > +int port_id_is_invalid(portid_t port_id, enum print_warning warning); > + > /* > * Work-around of a compilation error with ICC on invocations of the > * rte_be_to_cpu_16() function. > diff --git a/doc/guides/testpmd_app_ug/testpmd_funcs.rst b/doc/guides/testpmd_app_ug/testpmd_funcs.rst > index 218835a..1cacbcf 100644 > --- a/doc/guides/testpmd_app_ug/testpmd_funcs.rst > +++ b/doc/guides/testpmd_app_ug/testpmd_funcs.rst > @@ -808,6 +808,63 @@ The following sections show functions for configuring ports. > > Port configuration changes only become active when forwarding is started/restarted. > > +port attach > +~~~~~~~~~~~ > + > +Attach a port specified by pci address or virtual device args. > + > +To attach a new pci device, the device should be recognized by kernel first. > +Then it should be moved under DPDK management. > +Finally the port can be attached to testpmd. > +On the other hand, to attach a port created by virtual device, above steps are not needed. > + > +port attach (identifier) > + > +For example, to attach a port that pci address is 0000:02:00.0. > + > +.. code-block:: console > + > + testpmd> port attach 0000:02:00.0 > + Attaching a new port... > + ... snip ... > + Port 0 is attached. Now total ports is 1 > + Done > + > +For example, to attach a port created by pcap PMD. > + > +.. code-block:: console > + > + testpmd> port attach eth_pcap0,iface=eth0 > + Attaching a new port... > + ... snip ... > + Port 0 is attached. Now total ports is 1 > + Done > + > +In this case, identifier is "eth_pcap0,iface=eth0". > +This identifier format is the same as "--vdev" format of DPDK applications. > + > +port detach > +~~~~~~~~~~~ > + > +Detach a specific port. > + > +Before detaching a port, the port should be closed. > +Also to remove a pci device completely from the system, first detach the port from testpmd. > +Then the device should be moved under kernel management. > +Finally the device can be remove using kernel pci hotplug functionality. > +On the other hand, to remove a port created by virtual device, above steps are not needed. > + > +port detach (port_id) > + > +For example, to detach a port 0. > + > +.. code-block:: console > + > + testpmd> port detach 0 > + Detaching a port... > + ... snip ... > + Done > + > port start > ~~~~~~~~~~ >
On 2/1/2015 12:02 PM, Tetsuya Mukawa wrote: > The patch introduces following commands. > - port attach [ident] > - port detach [port_id] > - attach: attaching a port > - detach: detaching a port > - ident: pci address of physical device. > Or device name and paramerters of virtual device. > (ex. 0000:02:00.0, eth_pcap0,iface=eth0) > - port_id: port identifier > > v5: > - Add testpmd documentation. > (Thanks to Iremonger, Bernard) > v4: > - Fix strings of command help. > > Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp> > --- > app/test-pmd/cmdline.c | 133 +++++++++++++++---- > app/test-pmd/config.c | 116 +++++++++------- > app/test-pmd/parameters.c | 22 ++- > app/test-pmd/testpmd.c | 199 +++++++++++++++++++++------- > app/test-pmd/testpmd.h | 18 ++- > doc/guides/testpmd_app_ug/testpmd_funcs.rst | 57 ++++++++ > 6 files changed, 415 insertions(+), 130 deletions(-) > > diff --git a/app/test-pmd/cmdline.c b/app/test-pmd/cmdline.c > index 4beb404..2f813d8 100644 > --- a/app/test-pmd/cmdline.c > +++ b/app/test-pmd/cmdline.c > @@ -572,6 +572,12 @@ static void cmd_help_long_parsed(void *parsed_result, > "port close (port_id|all)\n" > " Close all ports or port_id.\n\n" > > + "port attach (ident)\n" > + " Attach physical or virtual dev by pci address or virtual device name\n\n" > + > + "port detach (port_id)\n" > + " Detach physical or virtual dev by port_id\n\n" > + > "port config (port_id|all)" > " speed (10|100|1000|10000|40000|auto)" > " duplex (half|full|auto)\n" > @@ -848,6 +854,89 @@ cmdline_parse_inst_t cmd_operate_specific_port = { > }, > }; > > +/* *** attach a specificied port *** */ > +struct cmd_operate_attach_port_result { > + cmdline_fixed_string_t port; > + cmdline_fixed_string_t keyword; > + cmdline_fixed_string_t identifier; > +}; > + > +static void cmd_operate_attach_port_parsed(void *parsed_result, > + __attribute__((unused)) struct cmdline *cl, > + __attribute__((unused)) void *data) > +{ > + struct cmd_operate_attach_port_result *res = parsed_result; > + > + if (!strcmp(res->keyword, "attach")) > + attach_port(res->identifier); > + else > + printf("Unknown parameter\n"); > +} > + > +cmdline_parse_token_string_t cmd_operate_attach_port_port = > + TOKEN_STRING_INITIALIZER(struct cmd_operate_attach_port_result, > + port, "port"); > +cmdline_parse_token_string_t cmd_operate_attach_port_keyword = > + TOKEN_STRING_INITIALIZER(struct cmd_operate_attach_port_result, > + keyword, "attach"); > +cmdline_parse_token_string_t cmd_operate_attach_port_identifier = > + TOKEN_STRING_INITIALIZER(struct cmd_operate_attach_port_result, > + identifier, NULL); > + > +cmdline_parse_inst_t cmd_operate_attach_port = { > + .f = cmd_operate_attach_port_parsed, > + .data = NULL, > + .help_str = "port attach identifier, " > + "identifier: pci address or virtual dev name", > + .tokens = { > + (void *)&cmd_operate_attach_port_port, > + (void *)&cmd_operate_attach_port_keyword, > + (void *)&cmd_operate_attach_port_identifier, > + NULL, > + }, > +}; > + > +/* *** detach a specificied port *** */ > +struct cmd_operate_detach_port_result { > + cmdline_fixed_string_t port; > + cmdline_fixed_string_t keyword; > + uint8_t port_id; > +}; > + > +static void cmd_operate_detach_port_parsed(void *parsed_result, > + __attribute__((unused)) struct cmdline *cl, > + __attribute__((unused)) void *data) > +{ > + struct cmd_operate_detach_port_result *res = parsed_result; > + > + if (!strcmp(res->keyword, "detach")) > + detach_port(res->port_id); > + else > + printf("Unknown parameter\n"); > +} > + > +cmdline_parse_token_string_t cmd_operate_detach_port_port = > + TOKEN_STRING_INITIALIZER(struct cmd_operate_detach_port_result, > + port, "port"); > +cmdline_parse_token_string_t cmd_operate_detach_port_keyword = > + TOKEN_STRING_INITIALIZER(struct cmd_operate_detach_port_result, > + keyword, "detach"); > +cmdline_parse_token_num_t cmd_operate_detach_port_port_id = > + TOKEN_NUM_INITIALIZER(struct cmd_operate_detach_port_result, > + port_id, UINT8); > + > +cmdline_parse_inst_t cmd_operate_detach_port = { > + .f = cmd_operate_detach_port_parsed, > + .data = NULL, > + .help_str = "port detach port_id", > + .tokens = { > + (void *)&cmd_operate_detach_port_port, > + (void *)&cmd_operate_detach_port_keyword, > + (void *)&cmd_operate_detach_port_port_id, > + NULL, > + }, > +}; > + > /* *** configure speed for all ports *** */ > struct cmd_config_speed_all { > cmdline_fixed_string_t port; > @@ -902,7 +991,7 @@ cmd_config_speed_all_parsed(void *parsed_result, > return; > } > > - for (pid = 0; pid < nb_ports; pid++) { > + FOREACH_PORT(pid, ports) { > ports[pid].dev_conf.link_speed = link_speed; > ports[pid].dev_conf.link_duplex = link_duplex; > } > @@ -970,10 +1059,8 @@ cmd_config_speed_specific_parsed(void *parsed_result, > return; > } > > - if (res->id >= nb_ports) { > - printf("Port id %d must be less than %d\n", res->id, nb_ports); > + if (port_id_is_invalid(res->id, ENABLED_WARN)) > return; > - } > > if (!strcmp(res->value1, "10")) > link_speed = ETH_LINK_SPEED_10; > @@ -1533,7 +1620,7 @@ cmd_config_rxtx_queue_parsed(void *parsed_result, > return; > } > > - if (port_id_is_invalid(res->portid)) > + if (port_id_is_invalid(res->portid, ENABLED_WARN)) > return; > > if (port_is_started(res->portid) != 1) { > @@ -2876,7 +2963,7 @@ cmd_tx_cksum_parsed(void *parsed_result, > uint16_t ol_flags, mask = 0; > struct rte_eth_dev_info dev_info; > > - if (port_id_is_invalid(res->port_id)) { > + if (port_id_is_invalid(res->port_id, ENABLED_WARN)) { > printf("invalid port %d\n", res->port_id); > return; > } > @@ -3003,7 +3090,7 @@ cmd_tso_set_parsed(void *parsed_result, > struct cmd_tso_set_result *res = parsed_result; > struct rte_eth_dev_info dev_info; > > - if (port_id_is_invalid(res->port_id)) > + if (port_id_is_invalid(res->port_id, ENABLED_WARN)) > return; > > if (!strcmp(res->mode, "set")) > @@ -3979,10 +4066,8 @@ static void cmd_set_bond_mac_addr_parsed(void *parsed_result, > struct cmd_set_bond_mac_addr_result *res = parsed_result; > int ret; > > - if (res->port_num >= nb_ports) { > - printf("Port id %d must be less than %d\n", res->port_num, nb_ports); > + if (port_id_is_invalid(res->port_num, ENABLED_WARN)) > return; > - } > > ret = rte_eth_bond_mac_address_set(res->port_num, &res->address); > > @@ -4219,7 +4304,7 @@ static void cmd_set_promisc_mode_parsed(void *parsed_result, > > /* all ports */ > if (allports) { > - for (i = 0; i < nb_ports; i++) { > + FOREACH_PORT(i, ports) { > if (enable) > rte_eth_promiscuous_enable(i); > else > @@ -4299,7 +4384,7 @@ static void cmd_set_allmulti_mode_parsed(void *parsed_result, > > /* all ports */ > if (allports) { > - for (i = 0; i < nb_ports; i++) { > + FOREACH_PORT(i, ports) { > if (enable) > rte_eth_allmulticast_enable(i); > else > @@ -5484,25 +5569,25 @@ static void cmd_showportall_parsed(void *parsed_result, > struct cmd_showportall_result *res = parsed_result; > if (!strcmp(res->show, "clear")) { > if (!strcmp(res->what, "stats")) > - for (i = 0; i < nb_ports; i++) > + FOREACH_PORT(i, ports) > nic_stats_clear(i); > else if (!strcmp(res->what, "xstats")) > - for (i = 0; i < nb_ports; i++) > + FOREACH_PORT(i, ports) > nic_xstats_clear(i); > } else if (!strcmp(res->what, "info")) > - for (i = 0; i < nb_ports; i++) > + FOREACH_PORT(i, ports) > port_infos_display(i); > else if (!strcmp(res->what, "stats")) > - for (i = 0; i < nb_ports; i++) > + FOREACH_PORT(i, ports) > nic_stats_display(i); > else if (!strcmp(res->what, "xstats")) > - for (i = 0; i < nb_ports; i++) > + FOREACH_PORT(i, ports) > nic_xstats_display(i); > else if (!strcmp(res->what, "fdir")) > - for (i = 0; i < nb_ports; i++) > + FOREACH_PORT(i, ports) > fdir_get_infos(i); > else if (!strcmp(res->what, "stat_qmap")) > - for (i = 0; i < nb_ports; i++) > + FOREACH_PORT(i, ports) > nic_stats_mapping_display(i); > } > > @@ -8756,6 +8841,8 @@ cmdline_parse_ctx_t main_ctx[] = { > (cmdline_parse_inst_t *)&cmd_set_qmap, > (cmdline_parse_inst_t *)&cmd_operate_port, > (cmdline_parse_inst_t *)&cmd_operate_specific_port, > + (cmdline_parse_inst_t *)&cmd_operate_attach_port, > + (cmdline_parse_inst_t *)&cmd_operate_detach_port, > (cmdline_parse_inst_t *)&cmd_config_speed_all, > (cmdline_parse_inst_t *)&cmd_config_speed_specific, > (cmdline_parse_inst_t *)&cmd_config_rx_tx, > @@ -8830,7 +8917,7 @@ prompt(void) > static void > cmd_reconfig_device_queue(portid_t id, uint8_t dev, uint8_t queue) > { > - if (id < nb_ports) { > + if (!port_id_is_invalid(id, DISABLED_WARN)) { > /* check if need_reconfig has been set to 1 */ > if (ports[id].need_reconfig == 0) > ports[id].need_reconfig = dev; > @@ -8840,7 +8927,7 @@ cmd_reconfig_device_queue(portid_t id, uint8_t dev, uint8_t queue) > } else { > portid_t pid; > > - for (pid = 0; pid < nb_ports; pid++) { > + FOREACH_PORT(pid, ports) { > /* check if need_reconfig has been set to 1 */ > if (ports[pid].need_reconfig == 0) > ports[pid].need_reconfig = dev; > @@ -8858,10 +8945,8 @@ bypass_is_supported(portid_t port_id) > struct rte_port *port; > struct rte_pci_id *pci_id; > > - if (port_id >= nb_ports) { > - printf("\tPort id must be less than %d.\n", nb_ports); > + if (port_id_is_invalid(port_id, ENABLED_WARN)) > return 0; > - } > > /* Get the device id. */ > port = &ports[port_id]; > diff --git a/app/test-pmd/config.c b/app/test-pmd/config.c > index c40f819..32d8f9a 100644 > --- a/app/test-pmd/config.c > +++ b/app/test-pmd/config.c > @@ -124,11 +124,15 @@ nic_stats_display(portid_t port_id) > struct rte_eth_stats stats; > struct rte_port *port = &ports[port_id]; > uint8_t i; > + portid_t pid; > > static const char *nic_stats_border = "########################"; > > - if (port_id >= nb_ports) { > - printf("Invalid port, range is [0, %d]\n", nb_ports - 1); > + if (port_id_is_invalid(port_id, ENABLED_WARN)) { > + printf("Valid port range is [0"); > + FOREACH_PORT(pid, ports) > + printf(", %d", pid); > + printf("]\n"); > return; > } > rte_eth_stats_get(port_id, &stats); > @@ -201,8 +205,13 @@ nic_stats_display(portid_t port_id) > void > nic_stats_clear(portid_t port_id) > { > - if (port_id >= nb_ports) { > - printf("Invalid port, range is [0, %d]\n", nb_ports - 1); > + portid_t pid; > + > + if (port_id_is_invalid(port_id, ENABLED_WARN)) { > + printf("Valid port range is [0"); > + FOREACH_PORT(pid, ports) > + printf(", %d", pid); > + printf("]\n"); > return; > } > rte_eth_stats_reset(port_id); > @@ -249,11 +258,15 @@ nic_stats_mapping_display(portid_t port_id) > { > struct rte_port *port = &ports[port_id]; > uint16_t i; > + portid_t pid; > > static const char *nic_stats_mapping_border = "########################"; > > - if (port_id >= nb_ports) { > - printf("Invalid port, range is [0, %d]\n", nb_ports - 1); > + if (port_id_is_invalid(port_id, ENABLED_WARN)) { > + printf("Valid port range is [0"); > + FOREACH_PORT(pid, ports) > + printf(", %d", pid); > + printf("]\n"); > return; > } > > @@ -302,9 +315,13 @@ port_infos_display(portid_t port_id) > int vlan_offload; > struct rte_mempool * mp; > static const char *info_border = "*********************"; > + portid_t pid; > > - if (port_id >= nb_ports) { > - printf("Invalid port, range is [0, %d]\n", nb_ports - 1); > + if (port_id_is_invalid(port_id, ENABLED_WARN)) { > + printf("Valid port range is [0"); > + FOREACH_PORT(pid, ports) > + printf(", %d", pid); > + printf("]\n"); > return; > } > port = &ports[port_id]; > @@ -362,11 +379,14 @@ port_infos_display(portid_t port_id) > } > > int > -port_id_is_invalid(portid_t port_id) > +port_id_is_invalid(portid_t port_id, enum print_warning warning) > { > - if (port_id < nb_ports) > + if (ports[port_id].enabled) Here maybe care about overflow, it could be passed a value of RTE_PORT_ALL, which is 255. Thanks, Michael > return 0; > - printf("Invalid port %d (must be < nb_ports=%d)\n", port_id, nb_ports); > + > + if (warning == ENABLED_WARN) > + printf("Invalid port %d\n", port_id); > + > return 1; > } > > @@ -425,7 +445,7 @@ port_reg_bit_display(portid_t port_id, uint32_t reg_off, uint8_t bit_x) > uint32_t reg_v; > > > - if (port_id_is_invalid(port_id)) > + if (port_id_is_invalid(port_id, ENABLED_WARN)) > return; > if (port_reg_off_is_invalid(port_id, reg_off)) > return; > @@ -444,7 +464,7 @@ port_reg_bit_field_display(portid_t port_id, uint32_t reg_off, > uint8_t l_bit; > uint8_t h_bit; > > - if (port_id_is_invalid(port_id)) > + if (port_id_is_invalid(port_id, ENABLED_WARN)) > return; > if (port_reg_off_is_invalid(port_id, reg_off)) > return; > @@ -471,7 +491,7 @@ port_reg_display(portid_t port_id, uint32_t reg_off) > { > uint32_t reg_v; > > - if (port_id_is_invalid(port_id)) > + if (port_id_is_invalid(port_id, ENABLED_WARN)) > return; > if (port_reg_off_is_invalid(port_id, reg_off)) > return; > @@ -485,7 +505,7 @@ port_reg_bit_set(portid_t port_id, uint32_t reg_off, uint8_t bit_pos, > { > uint32_t reg_v; > > - if (port_id_is_invalid(port_id)) > + if (port_id_is_invalid(port_id, ENABLED_WARN)) > return; > if (port_reg_off_is_invalid(port_id, reg_off)) > return; > @@ -513,7 +533,7 @@ port_reg_bit_field_set(portid_t port_id, uint32_t reg_off, > uint8_t l_bit; > uint8_t h_bit; > > - if (port_id_is_invalid(port_id)) > + if (port_id_is_invalid(port_id, ENABLED_WARN)) > return; > if (port_reg_off_is_invalid(port_id, reg_off)) > return; > @@ -547,7 +567,7 @@ port_reg_bit_field_set(portid_t port_id, uint32_t reg_off, > void > port_reg_set(portid_t port_id, uint32_t reg_off, uint32_t reg_v) > { > - if (port_id_is_invalid(port_id)) > + if (port_id_is_invalid(port_id, ENABLED_WARN)) > return; > if (port_reg_off_is_invalid(port_id, reg_off)) > return; > @@ -560,7 +580,7 @@ port_mtu_set(portid_t port_id, uint16_t mtu) > { > int diag; > > - if (port_id_is_invalid(port_id)) > + if (port_id_is_invalid(port_id, ENABLED_WARN)) > return; > diag = rte_eth_dev_set_mtu(port_id, mtu); > if (diag == 0) > @@ -723,7 +743,7 @@ rx_ring_desc_display(portid_t port_id, queueid_t rxq_id, uint16_t rxd_id) > { > const struct rte_memzone *rx_mz; > > - if (port_id_is_invalid(port_id)) > + if (port_id_is_invalid(port_id, ENABLED_WARN)) > return; > if (rx_queue_id_is_invalid(rxq_id)) > return; > @@ -740,7 +760,7 @@ tx_ring_desc_display(portid_t port_id, queueid_t txq_id, uint16_t txd_id) > { > const struct rte_memzone *tx_mz; > > - if (port_id_is_invalid(port_id)) > + if (port_id_is_invalid(port_id, ENABLED_WARN)) > return; > if (tx_queue_id_is_invalid(txq_id)) > return; > @@ -796,7 +816,7 @@ port_rss_reta_info(portid_t port_id, > uint16_t i, idx, shift; > int ret; > > - if (port_id_is_invalid(port_id)) > + if (port_id_is_invalid(port_id, ENABLED_WARN)) > return; > > ret = rte_eth_dev_rss_reta_query(port_id, reta_conf, nb_entries); > @@ -828,7 +848,7 @@ port_rss_hash_conf_show(portid_t port_id, int show_rss_key) > uint8_t i; > int diag; > > - if (port_id_is_invalid(port_id)) > + if (port_id_is_invalid(port_id, ENABLED_WARN)) > return; > /* Get RSS hash key if asked to display it */ > rss_conf.rss_key = (show_rss_key) ? rss_key : NULL; > @@ -1406,12 +1426,8 @@ set_fwd_ports_list(unsigned int *portlist, unsigned int nb_pt) > again: > for (i = 0; i < nb_pt; i++) { > port_id = (portid_t) portlist[i]; > - if (port_id >= nb_ports) { > - printf("Invalid port id %u >= %u\n", > - (unsigned int) port_id, > - (unsigned int) nb_ports); > + if (port_id_is_invalid(port_id, ENABLED_WARN)) > return; > - } > if (record_now) > fwd_ports_ids[i] = port_id; > } > @@ -1569,7 +1585,7 @@ vlan_extend_set(portid_t port_id, int on) > int diag; > int vlan_offload; > > - if (port_id_is_invalid(port_id)) > + if (port_id_is_invalid(port_id, ENABLED_WARN)) > return; > > vlan_offload = rte_eth_dev_get_vlan_offload(port_id); > @@ -1591,7 +1607,7 @@ rx_vlan_strip_set(portid_t port_id, int on) > int diag; > int vlan_offload; > > - if (port_id_is_invalid(port_id)) > + if (port_id_is_invalid(port_id, ENABLED_WARN)) > return; > > vlan_offload = rte_eth_dev_get_vlan_offload(port_id); > @@ -1612,7 +1628,7 @@ rx_vlan_strip_set_on_queue(portid_t port_id, uint16_t queue_id, int on) > { > int diag; > > - if (port_id_is_invalid(port_id)) > + if (port_id_is_invalid(port_id, ENABLED_WARN)) > return; > > diag = rte_eth_dev_set_vlan_strip_on_queue(port_id, queue_id, on); > @@ -1627,7 +1643,7 @@ rx_vlan_filter_set(portid_t port_id, int on) > int diag; > int vlan_offload; > > - if (port_id_is_invalid(port_id)) > + if (port_id_is_invalid(port_id, ENABLED_WARN)) > return; > > vlan_offload = rte_eth_dev_get_vlan_offload(port_id); > @@ -1648,7 +1664,7 @@ rx_vft_set(portid_t port_id, uint16_t vlan_id, int on) > { > int diag; > > - if (port_id_is_invalid(port_id)) > + if (port_id_is_invalid(port_id, ENABLED_WARN)) > return; > if (vlan_id_is_invalid(vlan_id)) > return; > @@ -1665,7 +1681,7 @@ rx_vlan_all_filter_set(portid_t port_id, int on) > { > uint16_t vlan_id; > > - if (port_id_is_invalid(port_id)) > + if (port_id_is_invalid(port_id, ENABLED_WARN)) > return; > for (vlan_id = 0; vlan_id < 4096; vlan_id++) > rx_vft_set(port_id, vlan_id, on); > @@ -1675,7 +1691,7 @@ void > vlan_tpid_set(portid_t port_id, uint16_t tp_id) > { > int diag; > - if (port_id_is_invalid(port_id)) > + if (port_id_is_invalid(port_id, ENABLED_WARN)) > return; > > diag = rte_eth_dev_set_vlan_ether_type(port_id, tp_id); > @@ -1690,7 +1706,7 @@ vlan_tpid_set(portid_t port_id, uint16_t tp_id) > void > tx_vlan_set(portid_t port_id, uint16_t vlan_id) > { > - if (port_id_is_invalid(port_id)) > + if (port_id_is_invalid(port_id, ENABLED_WARN)) > return; > if (vlan_id_is_invalid(vlan_id)) > return; > @@ -1701,7 +1717,7 @@ tx_vlan_set(portid_t port_id, uint16_t vlan_id) > void > tx_vlan_reset(portid_t port_id) > { > - if (port_id_is_invalid(port_id)) > + if (port_id_is_invalid(port_id, ENABLED_WARN)) > return; > ports[port_id].tx_ol_flags &= ~TESTPMD_TX_OFFLOAD_INSERT_VLAN; > } > @@ -1709,7 +1725,7 @@ tx_vlan_reset(portid_t port_id) > void > tx_vlan_pvid_set(portid_t port_id, uint16_t vlan_id, int on) > { > - if (port_id_is_invalid(port_id)) > + if (port_id_is_invalid(port_id, ENABLED_WARN)) > return; > > rte_eth_dev_set_vlan_pvid(port_id, vlan_id, on); > @@ -1721,7 +1737,7 @@ set_qmap(portid_t port_id, uint8_t is_rx, uint16_t queue_id, uint8_t map_value) > uint16_t i; > uint8_t existing_mapping_found = 0; > > - if (port_id_is_invalid(port_id)) > + if (port_id_is_invalid(port_id, ENABLED_WARN)) > return; > > if (is_rx ? (rx_queue_id_is_invalid(queue_id)) : (tx_queue_id_is_invalid(queue_id))) > @@ -1773,7 +1789,7 @@ fdir_add_signature_filter(portid_t port_id, uint8_t queue_id, > { > int diag; > > - if (port_id_is_invalid(port_id)) > + if (port_id_is_invalid(port_id, ENABLED_WARN)) > return; > > diag = rte_eth_dev_fdir_add_signature_filter(port_id, fdir_filter, > @@ -1791,7 +1807,7 @@ fdir_update_signature_filter(portid_t port_id, uint8_t queue_id, > { > int diag; > > - if (port_id_is_invalid(port_id)) > + if (port_id_is_invalid(port_id, ENABLED_WARN)) > return; > > diag = rte_eth_dev_fdir_update_signature_filter(port_id, fdir_filter, > @@ -1809,7 +1825,7 @@ fdir_remove_signature_filter(portid_t port_id, > { > int diag; > > - if (port_id_is_invalid(port_id)) > + if (port_id_is_invalid(port_id, ENABLED_WARN)) > return; > > diag = rte_eth_dev_fdir_remove_signature_filter(port_id, fdir_filter); > @@ -1881,7 +1897,7 @@ fdir_get_infos(portid_t port_id) > > static const char *fdir_stats_border = "########################"; > > - if (port_id_is_invalid(port_id)) > + if (port_id_is_invalid(port_id, ENABLED_WARN)) > return; > ret = rte_eth_dev_filter_supported(port_id, RTE_ETH_FILTER_FDIR); > if (ret < 0) { > @@ -1955,7 +1971,7 @@ fdir_add_perfect_filter(portid_t port_id, uint16_t soft_id, uint8_t queue_id, > { > int diag; > > - if (port_id_is_invalid(port_id)) > + if (port_id_is_invalid(port_id, ENABLED_WARN)) > return; > > diag = rte_eth_dev_fdir_add_perfect_filter(port_id, fdir_filter, > @@ -1973,7 +1989,7 @@ fdir_update_perfect_filter(portid_t port_id, uint16_t soft_id, uint8_t queue_id, > { > int diag; > > - if (port_id_is_invalid(port_id)) > + if (port_id_is_invalid(port_id, ENABLED_WARN)) > return; > > diag = rte_eth_dev_fdir_update_perfect_filter(port_id, fdir_filter, > @@ -1991,7 +2007,7 @@ fdir_remove_perfect_filter(portid_t port_id, uint16_t soft_id, > { > int diag; > > - if (port_id_is_invalid(port_id)) > + if (port_id_is_invalid(port_id, ENABLED_WARN)) > return; > > diag = rte_eth_dev_fdir_remove_perfect_filter(port_id, fdir_filter, > @@ -2008,7 +2024,7 @@ fdir_set_masks(portid_t port_id, struct rte_fdir_masks *fdir_masks) > { > int diag; > > - if (port_id_is_invalid(port_id)) > + if (port_id_is_invalid(port_id, ENABLED_WARN)) > return; > > diag = rte_eth_dev_fdir_set_masks(port_id, fdir_masks); > @@ -2085,7 +2101,7 @@ set_vf_traffic(portid_t port_id, uint8_t is_rx, uint16_t vf, uint8_t on) > { > int diag; > > - if (port_id_is_invalid(port_id)) > + if (port_id_is_invalid(port_id, ENABLED_WARN)) > return; > if (is_rx) > diag = rte_eth_dev_set_vf_rx(port_id,vf,on); > @@ -2107,7 +2123,7 @@ set_vf_rx_vlan(portid_t port_id, uint16_t vlan_id, uint64_t vf_mask, uint8_t on) > { > int diag; > > - if (port_id_is_invalid(port_id)) > + if (port_id_is_invalid(port_id, ENABLED_WARN)) > return; > if (vlan_id_is_invalid(vlan_id)) > return; > @@ -2124,7 +2140,7 @@ set_queue_rate_limit(portid_t port_id, uint16_t queue_idx, uint16_t rate) > int diag; > struct rte_eth_link link; > > - if (port_id_is_invalid(port_id)) > + if (port_id_is_invalid(port_id, ENABLED_WARN)) > return 1; > rte_eth_link_get_nowait(port_id, &link); > if (rate > link.link_speed) { > @@ -2149,7 +2165,7 @@ set_vf_rate_limit(portid_t port_id, uint16_t vf, uint16_t rate, uint64_t q_msk) > if (q_msk == 0) > return 0; > > - if (port_id_is_invalid(port_id)) > + if (port_id_is_invalid(port_id, ENABLED_WARN)) > return 1; > rte_eth_link_get_nowait(port_id, &link); > if (rate > link.link_speed) { > diff --git a/app/test-pmd/parameters.c b/app/test-pmd/parameters.c > index adf3203..6f2af18 100644 > --- a/app/test-pmd/parameters.c > +++ b/app/test-pmd/parameters.c > @@ -376,6 +376,7 @@ parse_portnuma_config(const char *q_arg) > }; > unsigned long int_fld[_NUM_FLD]; > char *str_fld[_NUM_FLD]; > + portid_t pid; > > /* reset from value set at definition */ > while ((p = strchr(p0,'(')) != NULL) { > @@ -397,8 +398,11 @@ parse_portnuma_config(const char *q_arg) > return -1; > } > port_id = (uint8_t)int_fld[FLD_PORT]; > - if (port_id >= nb_ports) { > - printf("Invalid port, range is [0, %d]\n", nb_ports - 1); > + if (port_id_is_invalid(port_id, ENABLED_WARN)) { > + printf("Valid port range is [0"); > + FOREACH_PORT(pid, ports) > + printf(", %d", pid); > + printf("]\n"); > return -1; > } > socket_id = (uint8_t)int_fld[FLD_SOCKET]; > @@ -429,6 +433,7 @@ parse_ringnuma_config(const char *q_arg) > }; > unsigned long int_fld[_NUM_FLD]; > char *str_fld[_NUM_FLD]; > + portid_t pid; > #define RX_RING_ONLY 0x1 > #define TX_RING_ONLY 0x2 > #define RXTX_RING 0x3 > @@ -453,8 +458,11 @@ parse_ringnuma_config(const char *q_arg) > return -1; > } > port_id = (uint8_t)int_fld[FLD_PORT]; > - if (port_id >= nb_ports) { > - printf("Invalid port, range is [0, %d]\n", nb_ports - 1); > + if (port_id_is_invalid(port_id, ENABLED_WARN)) { > + printf("Valid port range is [0"); > + FOREACH_PORT(pid, ports) > + printf(", %d", pid); > + printf("]\n"); > return -1; > } > socket_id = (uint8_t)int_fld[FLD_SOCKET]; > @@ -626,12 +634,12 @@ launch_args_parse(int argc, char** argv) > #endif > if (!strcmp(lgopts[opt_idx].name, "nb-ports")) { > n = atoi(optarg); > - if (n > 0 && n <= nb_ports) > + if (n > 0 && > + !port_id_is_invalid(n, DISABLED_WARN)) > nb_fwd_ports = (uint8_t) n; > else > rte_exit(EXIT_FAILURE, > - "nb-ports should be > 0 and <= %d\n", > - nb_ports); > + "Invalid port %d\n", n); > } > if (!strcmp(lgopts[opt_idx].name, "nb-cores")) { > n = atoi(optarg); > diff --git a/app/test-pmd/testpmd.c b/app/test-pmd/testpmd.c > index 773b8af..c18c1a9 100644 > --- a/app/test-pmd/testpmd.c > +++ b/app/test-pmd/testpmd.c > @@ -71,6 +71,7 @@ > #include <rte_pci.h> > #include <rte_ether.h> > #include <rte_ethdev.h> > +#include <rte_dev.h> > #include <rte_string_fns.h> > #ifdef RTE_LIBRTE_PMD_XENVIRT > #include <rte_eth_xenvirt.h> > @@ -315,7 +316,7 @@ uint16_t nb_rx_queue_stats_mappings = 0; > > /* Forward function declarations */ > static void map_port_queue_stats_mapping_registers(uint8_t pi, struct rte_port *port); > -static void check_all_ports_link_status(uint8_t port_num, uint32_t port_mask); > +static void check_all_ports_link_status(uint32_t port_mask); > > /* > * Check if all the ports are started. > @@ -324,6 +325,20 @@ static void check_all_ports_link_status(uint8_t port_num, uint32_t port_mask); > static int all_ports_started(void); > > /* > + * Find next enabled port > + */ > +portid_t > +find_next_port(portid_t p, struct rte_port *ports, int size) > +{ > + if (ports == NULL) > + rte_exit(-EINVAL, "failed to find a next port id\n"); > + > + while ((ports[p].enabled == 0) && (p < size)) > + p++; > + return p; > +} > + > +/* > * Setup default configuration. > */ > static void > @@ -552,7 +567,8 @@ init_config(void) > + RTE_TEST_TX_DESC_MAX + MAX_PKT_BURST; > > if (!numa_support) > - nb_mbuf_per_pool = (nb_mbuf_per_pool * nb_ports); > + nb_mbuf_per_pool = > + (nb_mbuf_per_pool * RTE_MAX_ETHPORTS); > } > > if (!numa_support) { > @@ -565,14 +581,19 @@ init_config(void) > > /* Configuration of Ethernet ports. */ > ports = rte_zmalloc("testpmd: ports", > - sizeof(struct rte_port) * nb_ports, > + sizeof(struct rte_port) * RTE_MAX_ETHPORTS, > RTE_CACHE_LINE_SIZE); > if (ports == NULL) { > - rte_exit(EXIT_FAILURE, "rte_zmalloc(%d struct rte_port) " > - "failed\n", nb_ports); > + rte_exit(EXIT_FAILURE, > + "rte_zmalloc(%d struct rte_port) failed\n", > + RTE_MAX_ETHPORTS); > } > > - for (pid = 0; pid < nb_ports; pid++) { > + /* enabled allocated ports */ > + for (pid = 0; pid < nb_ports; pid++) > + ports[pid].enabled = 1; > + > + FOREACH_PORT(pid, ports) { > port = &ports[pid]; > rte_eth_dev_info_get(pid, &port->dev_info); > > @@ -602,8 +623,7 @@ init_config(void) > nb_mbuf_per_pool = nb_mbuf_per_pool/nb_ports; > > for (i = 0; i < MAX_SOCKET; i++) { > - nb_mbuf = (nb_mbuf_per_pool * > - port_per_socket[i]); > + nb_mbuf = (nb_mbuf_per_pool * RTE_MAX_ETHPORTS); > if (nb_mbuf) > mbuf_pool_create(mbuf_data_size, > nb_mbuf,i); > @@ -635,14 +655,6 @@ reconfig(portid_t new_port_id, unsigned socket_id) > struct rte_port *port; > > /* Reconfiguration of Ethernet ports. */ > - ports = rte_realloc(ports, > - sizeof(struct rte_port) * nb_ports, > - RTE_CACHE_LINE_SIZE); > - if (ports == NULL) { > - rte_exit(EXIT_FAILURE, "rte_realloc(%d struct rte_port) failed\n", > - nb_ports); > - } > - > port = &ports[new_port_id]; > rte_eth_dev_info_get(new_port_id, &port->dev_info); > > @@ -663,7 +675,7 @@ init_fwd_streams(void) > streamid_t sm_id, nb_fwd_streams_new; > > /* set socket id according to numa or not */ > - for (pid = 0; pid < nb_ports; pid++) { > + FOREACH_PORT(pid, ports) { > port = &ports[pid]; > if (nb_rxq > port->dev_info.max_rx_queues) { > printf("Fail: nb_rxq(%d) is greater than " > @@ -1264,7 +1276,7 @@ all_ports_started(void) > portid_t pi; > struct rte_port *port; > > - for (pi = 0; pi < nb_ports; pi++) { > + FOREACH_PORT(pi, ports) { > port = &ports[pi]; > /* Check if there is a port which is not started */ > if (port->port_status != RTE_PORT_STARTED) > @@ -1276,6 +1288,45 @@ all_ports_started(void) > } > > int > +all_ports_stopped(void) > +{ > + portid_t pi; > + struct rte_port *port; > + > + FOREACH_PORT(pi, ports) { > + port = &ports[pi]; > + if (port->port_status != RTE_PORT_STOPPED) > + return 0; > + } > + > + return 1; > +} > + > +int > +port_is_started(portid_t port_id) > +{ > + if (port_id_is_invalid(port_id, ENABLED_WARN)) > + return 0; > + > + if (ports[port_id].port_status != RTE_PORT_STARTED) > + return 0; > + > + return 1; > +} > + > +static int > +port_is_closed(portid_t port_id) > +{ > + if (port_id_is_invalid(port_id, ENABLED_WARN)) > + return 0; > + > + if (ports[port_id].port_status != RTE_PORT_CLOSED) > + return 0; > + > + return 1; > +} > + > +int > start_port(portid_t pid) > { > int diag, need_check_link_status = 0; > @@ -1296,8 +1347,8 @@ start_port(portid_t pid) > > if(dcb_config) > dcb_test = 1; > - for (pi = 0; pi < nb_ports; pi++) { > - if (pid < nb_ports && pid != pi) > + FOREACH_PORT(pi, ports) { > + if (!port_id_is_invalid(pid, DISABLED_WARN) && pid != pi) > continue; > > port = &ports[pi]; > @@ -1421,7 +1472,7 @@ start_port(portid_t pid) > } > > if (need_check_link_status && !no_link_check) > - check_all_ports_link_status(nb_ports, RTE_PORT_ALL); > + check_all_ports_link_status(RTE_PORT_ALL); > else > printf("Please stop the ports first\n"); > > @@ -1446,8 +1497,8 @@ stop_port(portid_t pid) > } > printf("Stopping ports...\n"); > > - for (pi = 0; pi < nb_ports; pi++) { > - if (pid < nb_ports && pid != pi) > + FOREACH_PORT(pi, ports) { > + if (!port_id_is_invalid(pid, DISABLED_WARN) && pid != pi) > continue; > > port = &ports[pi]; > @@ -1463,7 +1514,7 @@ stop_port(portid_t pid) > need_check_link_status = 1; > } > if (need_check_link_status && !no_link_check) > - check_all_ports_link_status(nb_ports, RTE_PORT_ALL); > + check_all_ports_link_status(RTE_PORT_ALL); > > printf("Done\n"); > } > @@ -1481,8 +1532,8 @@ close_port(portid_t pid) > > printf("Closing ports...\n"); > > - for (pi = 0; pi < nb_ports; pi++) { > - if (pid < nb_ports && pid != pi) > + FOREACH_PORT(pi, ports) { > + if (!port_id_is_invalid(pid, DISABLED_WARN) && pid != pi) > continue; > > port = &ports[pi]; > @@ -1502,31 +1553,83 @@ close_port(portid_t pid) > printf("Done\n"); > } > > -int > -all_ports_stopped(void) > +void > +attach_port(char *identifier) > { > - portid_t pi; > - struct rte_port *port; > + portid_t i, j, pi = 0; > > - for (pi = 0; pi < nb_ports; pi++) { > - port = &ports[pi]; > - if (port->port_status != RTE_PORT_STOPPED) > - return 0; > + printf("Attaching a new port...\n"); > + > + if (identifier == NULL) { > + printf("Invalid parameters are speficied\n"); > + return; > } > > - return 1; > + if (test_done == 0) { > + printf("Please stop forwarding first\n"); > + return; > + } > + > + if (rte_eal_dev_attach(identifier, &pi)) > + return; > + > + ports[pi].enabled = 1; > + reconfig(pi, rte_eth_dev_socket_id(pi)); > + rte_eth_promiscuous_enable(pi); > + > + nb_ports = rte_eth_dev_count(); > + > + /* set_default_fwd_ports_config(); */ > + bzero(fwd_ports_ids, sizeof(fwd_ports_ids)); > + i = 0; > + FOREACH_PORT(j, ports) { > + fwd_ports_ids[i] = j; > + i++; > + } > + nb_cfg_ports = nb_ports; > + nb_fwd_ports++; > + > + ports[pi].port_status = RTE_PORT_STOPPED; > + > + printf("Port %d is attached. Now total ports is %d\n", pi, nb_ports); > + printf("Done\n"); > } > > -int > -port_is_started(portid_t port_id) > +void > +detach_port(uint8_t port_id) > { > - if (port_id_is_invalid(port_id)) > - return -1; > + portid_t i, pi = 0; > + char name[RTE_ETH_NAME_MAX_LEN]; > > - if (ports[port_id].port_status != RTE_PORT_STARTED) > - return 0; > + printf("Detaching a port...\n"); > > - return 1; > + if (!port_is_closed(port_id)) { > + printf("Please close port first\n"); > + return; > + } > + > + rte_eth_promiscuous_disable(port_id); > + > + if (rte_eal_dev_detach(port_id, name)) > + return; > + > + ports[port_id].enabled = 0; > + nb_ports = rte_eth_dev_count(); > + > + /* set_default_fwd_ports_config(); */ > + bzero(fwd_ports_ids, sizeof(fwd_ports_ids)); > + i = 0; > + FOREACH_PORT(pi, ports) { > + fwd_ports_ids[i] = pi; > + i++; > + } > + nb_cfg_ports = nb_ports; > + nb_fwd_ports--; > + > + printf("Port '%s' is detached. Now total ports is %d\n", > + name, nb_ports); > + printf("Done\n"); > + return; > } > > void > @@ -1534,7 +1637,7 @@ pmd_test_exit(void) > { > portid_t pt_id; > > - for (pt_id = 0; pt_id < nb_ports; pt_id++) { > + FOREACH_PORT(pt_id, ports) { > printf("Stopping port %d...", pt_id); > fflush(stdout); > rte_eth_dev_close(pt_id); > @@ -1553,7 +1656,7 @@ struct pmd_test_command { > > /* Check the link status of all ports in up to 9s, and print them finally */ > static void > -check_all_ports_link_status(uint8_t port_num, uint32_t port_mask) > +check_all_ports_link_status(uint32_t port_mask) > { > #define CHECK_INTERVAL 100 /* 100ms */ > #define MAX_CHECK_TIME 90 /* 9s (90 * 100ms) in total */ > @@ -1564,7 +1667,7 @@ check_all_ports_link_status(uint8_t port_num, uint32_t port_mask) > fflush(stdout); > for (count = 0; count <= MAX_CHECK_TIME; count++) { > all_ports_up = 1; > - for (portid = 0; portid < port_num; portid++) { > + FOREACH_PORT(portid, ports) { > if ((port_mask & (1 << portid)) == 0) > continue; > memset(&link, 0, sizeof(link)); > @@ -1688,7 +1791,7 @@ init_port_config(void) > portid_t pid; > struct rte_port *port; > > - for (pid = 0; pid < nb_ports; pid++) { > + FOREACH_PORT(pid, ports) { > port = &ports[pid]; > port->dev_conf.rxmode = rx_mode; > port->dev_conf.fdir_conf = fdir_conf; > @@ -1877,7 +1980,7 @@ main(int argc, char** argv) > > nb_ports = (portid_t) rte_eth_dev_count(); > if (nb_ports == 0) > - rte_exit(EXIT_FAILURE, "No probed ethernet device\n"); > + RTE_LOG(WARNING, EAL, "No probed ethernet devices\n"); > > set_def_fwd_config(); > if (nb_lcores == 0) > @@ -1899,7 +2002,7 @@ main(int argc, char** argv) > rte_exit(EXIT_FAILURE, "Start ports failed\n"); > > /* set all ports to promiscuous mode by default */ > - for (port_id = 0; port_id < nb_ports; port_id++) > + FOREACH_PORT(port_id, ports) > rte_eth_promiscuous_enable(port_id); > > #ifdef RTE_LIBRTE_CMDLINE > diff --git a/app/test-pmd/testpmd.h b/app/test-pmd/testpmd.h > index 8f5e6c7..109c670 100644 > --- a/app/test-pmd/testpmd.h > +++ b/app/test-pmd/testpmd.h > @@ -134,6 +134,7 @@ struct fwd_stream { > * The data structure associated with each port. > */ > struct rte_port { > + uint8_t enabled; /**< Port enabled or not */ > struct rte_eth_dev_info dev_info; /**< PCI info + driver name */ > struct rte_eth_conf dev_conf; /**< Port configuration. */ > struct ether_addr eth_addr; /**< Port ethernet address */ > @@ -159,6 +160,14 @@ struct rte_port { > struct rte_eth_txconf tx_conf; /**< tx configuration */ > }; > > +extern portid_t __rte_unused > +find_next_port(portid_t p, struct rte_port *ports, int size); > + > +#define FOREACH_PORT(p, ports) \ > + for (p = find_next_port(0, ports, RTE_MAX_ETHPORTS); \ > + p < RTE_MAX_ETHPORTS; \ > + p = find_next_port(p + 1, ports, RTE_MAX_ETHPORTS)) > + > /** > * The data structure associated with each forwarding logical core. > * The logical cores are internally numbered by a core index from 0 to > @@ -515,6 +524,8 @@ int init_port_dcb_config(portid_t pid,struct dcb_config *dcb_conf); > int start_port(portid_t pid); > void stop_port(portid_t pid); > void close_port(portid_t pid); > +void attach_port(char *identifier); > +void detach_port(uint8_t port_id); > int all_ports_stopped(void); > int port_is_started(portid_t port_id); > void pmd_test_exit(void); > @@ -558,10 +569,15 @@ void get_ethertype_filter(uint8_t port_id, uint16_t index); > void get_2tuple_filter(uint8_t port_id, uint16_t index); > void get_5tuple_filter(uint8_t port_id, uint16_t index); > void get_flex_filter(uint8_t port_id, uint16_t index); > -int port_id_is_invalid(portid_t port_id); > int rx_queue_id_is_invalid(queueid_t rxq_id); > int tx_queue_id_is_invalid(queueid_t txq_id); > > +enum print_warning { > + ENABLED_WARN = 0, > + DISABLED_WARN > +}; > +int port_id_is_invalid(portid_t port_id, enum print_warning warning); > + > /* > * Work-around of a compilation error with ICC on invocations of the > * rte_be_to_cpu_16() function. > diff --git a/doc/guides/testpmd_app_ug/testpmd_funcs.rst b/doc/guides/testpmd_app_ug/testpmd_funcs.rst > index 218835a..1cacbcf 100644 > --- a/doc/guides/testpmd_app_ug/testpmd_funcs.rst > +++ b/doc/guides/testpmd_app_ug/testpmd_funcs.rst > @@ -808,6 +808,63 @@ The following sections show functions for configuring ports. > > Port configuration changes only become active when forwarding is started/restarted. > > +port attach > +~~~~~~~~~~~ > + > +Attach a port specified by pci address or virtual device args. > + > +To attach a new pci device, the device should be recognized by kernel first. > +Then it should be moved under DPDK management. > +Finally the port can be attached to testpmd. > +On the other hand, to attach a port created by virtual device, above steps are not needed. > + > +port attach (identifier) > + > +For example, to attach a port that pci address is 0000:02:00.0. > + > +.. code-block:: console > + > + testpmd> port attach 0000:02:00.0 > + Attaching a new port... > + ... snip ... > + Port 0 is attached. Now total ports is 1 > + Done > + > +For example, to attach a port created by pcap PMD. > + > +.. code-block:: console > + > + testpmd> port attach eth_pcap0,iface=eth0 > + Attaching a new port... > + ... snip ... > + Port 0 is attached. Now total ports is 1 > + Done > + > +In this case, identifier is "eth_pcap0,iface=eth0". > +This identifier format is the same as "--vdev" format of DPDK applications. > + > +port detach > +~~~~~~~~~~~ > + > +Detach a specific port. > + > +Before detaching a port, the port should be closed. > +Also to remove a pci device completely from the system, first detach the port from testpmd. > +Then the device should be moved under kernel management. > +Finally the device can be remove using kernel pci hotplug functionality. > +On the other hand, to remove a port created by virtual device, above steps are not needed. > + > +port detach (port_id) > + > +For example, to detach a port 0. > + > +.. code-block:: console > + > + testpmd> port detach 0 > + Detaching a port... > + ... snip ... > + Done > + > port start > ~~~~~~~~~~ >
On 2015/02/03 14:05, Qiu, Michael wrote: > On 2/3/2015 12:07 PM, Tetsuya Mukawa wrote: >> On 2015/02/03 11:35, Qiu, Michael wrote: >>> On 2/1/2015 12:02 PM, Tetsuya Mukawa wrote: >>>> - Add rte_eal_pci_close_one_dirver() >>>> The function is used for closing the specified driver and device. >>>> - Add pci_invoke_all_drivers() > [...] >>>> >>>> +#ifdef ENABLE_HOTPLUG >>>> +/* >>>> + * If vendor/device ID match, call the devuninit() function of the >>>> + * driver. >>>> + */ >>>> +int >>>> +rte_eal_pci_close_one_driver(struct rte_pci_driver *dr, >>>> + struct rte_pci_device *dev) >>>> +{ >>>> + struct rte_pci_id *id_table; >>>> + >>>> + if ((dr == NULL) || (dev == NULL)) >>>> + return -EINVAL; >>>> + >>>> + for (id_table = dr->id_table ; id_table->vendor_id != 0; id_table++) { >>>> + >>>> + /* check if device's identifiers match the driver's ones */ >>>> + if (id_table->vendor_id != dev->id.vendor_id && >>>> + id_table->vendor_id != PCI_ANY_ID) >>>> + continue; >>>> + if (id_table->device_id != dev->id.device_id && >>>> + id_table->device_id != PCI_ANY_ID) >>>> + continue; >>>> + if (id_table->subsystem_vendor_id != >>>> + dev->id.subsystem_vendor_id && >>>> + id_table->subsystem_vendor_id != PCI_ANY_ID) >>>> + continue; >>>> + if (id_table->subsystem_device_id != >>>> + dev->id.subsystem_device_id && >>>> + id_table->subsystem_device_id != PCI_ANY_ID) >>>> + continue; >>>> + >>>> + struct rte_pci_addr *loc = &dev->addr; >>>> + >>>> + RTE_LOG(DEBUG, EAL, >>>> + "PCI device "PCI_PRI_FMT" on NUMA socket %i\n", >>>> + loc->domain, loc->bus, loc->devid, >>>> + loc->function, dev->numa_node); >>>> + >>>> + RTE_LOG(DEBUG, EAL, " remove driver: %x:%x %s\n", >>>> + dev->id.vendor_id, dev->id.device_id, >>>> + dr->name); >>>> + >>>> + /* call the driver devuninit() function */ >>>> + if (dr->devuninit && (dr->devuninit(dr, dev) < 0)) >>>> + return -1; /* negative value is an error */ >>>> + >>>> + /* clear driver structure */ >>>> + dev->driver = NULL; >>>> + >>>> + if (dr->drv_flags & RTE_PCI_DRV_NEED_MAPPING) >>>> + /* unmap resources for devices that use igb_uio */ >>>> + pci_unmap_device(dev); >>> Hi, Tetsuya >>> >>> I have one question, as the code shows, in pci_unmap_device(), will >>> check pt_driver. >>> >>> But assume that, we are now try to detach a vfio device, after print out >>> a error message of unsupported, the does this port workable? >>> >>> I think this port will unworkable, am I right? >>> >>> But actually, we should keep it workable. >>> >>> My suggestion is to add a check in rte_eth_dev_check_detachable() for >>> pci_device port. >> Hi Michael, >> >> I appreciate your comment. >> In the function called "rte_eal_dev_detach_pdev()", >> "rte_eth_dev_check_detachable()" has been already checked. > What I mean is check the pt_driver for pci_dev in > rte_eth_dev_check_detachable(), so that hotplug framework will not > affect vfio devices, just as I reply in another mail. > > Current logic will affect vfio devices if try to detach( Not do the > really test, just the logic shows), am I right? Thanks, I've got your point. Yes, you are right. I will fix it. Tetsuya > Thanks, > Michael > >> But in the future, someone may want to reuse >> "rte_eal_pci_close_one_driver()". >> So I will add the checking like your suggestion. >> >> Thanks, >> Tetsuya >> >>> Thanks >>> Michael >>> >>>> + >>>> + return 0; >>>> + } >>>> + /* return positive value if driver is not found */ >>>> + return 1; >>>> +} >>>> +#else /* ENABLE_HOTPLUG */ >>>> +int >>>> +rte_eal_pci_close_one_driver(struct rte_pci_driver *dr __rte_unused, >>>> + struct rte_pci_device *dev __rte_unused) >>>> +{ >>>> + RTE_LOG(ERR, EAL, "Hotplug support isn't enabled\n"); >>>> + return -1; >>>> +} >>>> +#endif /* ENABLE_HOTPLUG */ >>>> + >>>> /* Init the PCI EAL subsystem */ >>>> int >>>> rte_eal_pci_init(void) >>
On 2/3/2015 2:16 PM, Qiu, Michael wrote: > On 2/1/2015 12:02 PM, Tetsuya Mukawa wrote: >> The patch introduces following commands. >> - port attach [ident] >> - port detach [port_id] >> - attach: attaching a port >> - detach: detaching a port >> - ident: pci address of physical device. >> Or device name and paramerters of virtual device. >> (ex. 0000:02:00.0, eth_pcap0,iface=eth0) >> - port_id: port identifier >> >> v5: >> - Add testpmd documentation. >> (Thanks to Iremonger, Bernard) >> v4: >> - Fix strings of command help. >> >> Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp> [...] >> +static int >> +port_is_closed(portid_t port_id) >> +{ >> + if (port_id_is_invalid(port_id, ENABLED_WARN)) >> + return 0; >> + >> + if (ports[port_id].port_status != RTE_PORT_CLOSED) >> + return 0; >> + >> + return 1; >> +} >> + >> +int >> start_port(portid_t pid) >> { >> int diag, need_check_link_status = 0; >> @@ -1296,8 +1347,8 @@ start_port(portid_t pid) >> >> if(dcb_config) >> dcb_test = 1; >> - for (pi = 0; pi < nb_ports; pi++) { >> - if (pid < nb_ports && pid != pi) >> + FOREACH_PORT(pi, ports) { >> + if (!port_id_is_invalid(pid, DISABLED_WARN) && pid != pi) > Here may it be: > > if (!port_id_is_invalid(pid, DISABLED_WARN) && (pid != pi || pid == RET_PORT_ALL)) Sorry, should be: if (!port_id_is_invalid(pid, DISABLED_WARN) && pid != pi && pid != (portid_t)RET_PORT_ALL) Otherwise, should check for "RET_PORT_ALL" in function port_id_is_invalid() Thanks, Michael > Otherwise no port will be start by default. > > > Thanks, > Michael > >> continue; >> >> port = &ports[pi]; >> @@ -1421,7 +1472,7 @@ start_port(portid_t pid) >> } >> >> if (need_check_link_status && !no_link_check) >> - check_all_ports_link_status(nb_ports, RTE_PORT_ALL); >> + check_all_ports_link_status(RTE_PORT_ALL); >> else >> printf("Please stop the ports first\n"); >> >> @@ -1446,8 +1497,8 @@ stop_port(portid_t pid) >> } >> printf("Stopping ports...\n"); >> >> - for (pi = 0; pi < nb_ports; pi++) { >> - if (pid < nb_ports && pid != pi) >> + FOREACH_PORT(pi, ports) { >> + if (!port_id_is_invalid(pid, DISABLED_WARN) && pid != pi) >> continue; >> >> port = &ports[pi]; >> @@ -1463,7 +1514,7 @@ stop_port(portid_t pid) >> need_check_link_status = 1; >> } >> if (need_check_link_status && !no_link_check) >> - check_all_ports_link_status(nb_ports, RTE_PORT_ALL); >> + check_all_ports_link_status(RTE_PORT_ALL); >> >> printf("Done\n"); >> } >> @@ -1481,8 +1532,8 @@ close_port(portid_t pid) >> >> printf("Closing ports...\n"); >> >> - for (pi = 0; pi < nb_ports; pi++) { >> - if (pid < nb_ports && pid != pi) >> + FOREACH_PORT(pi, ports) { >> + if (!port_id_is_invalid(pid, DISABLED_WARN) && pid != pi) >> continue; >> >> port = &ports[pi]; >> @@ -1502,31 +1553,83 @@ close_port(portid_t pid) >> printf("Done\n"); >> } >> >> -int >> -all_ports_stopped(void) >> +void >> +attach_port(char *identifier) >> { >> - portid_t pi; >> - struct rte_port *port; >> + portid_t i, j, pi = 0; >> >> - for (pi = 0; pi < nb_ports; pi++) { >> - port = &ports[pi]; >> - if (port->port_status != RTE_PORT_STOPPED) >> - return 0; >> + printf("Attaching a new port...\n"); >> + >> + if (identifier == NULL) { >> + printf("Invalid parameters are speficied\n"); >> + return; >> } >> >> - return 1; >> + if (test_done == 0) { >> + printf("Please stop forwarding first\n"); >> + return; >> + } >> + >> + if (rte_eal_dev_attach(identifier, &pi)) >> + return; >> + >> + ports[pi].enabled = 1; >> + reconfig(pi, rte_eth_dev_socket_id(pi)); >> + rte_eth_promiscuous_enable(pi); >> + >> + nb_ports = rte_eth_dev_count(); >> + >> + /* set_default_fwd_ports_config(); */ >> + bzero(fwd_ports_ids, sizeof(fwd_ports_ids)); >> + i = 0; >> + FOREACH_PORT(j, ports) { >> + fwd_ports_ids[i] = j; >> + i++; >> + } >> + nb_cfg_ports = nb_ports; >> + nb_fwd_ports++; >> + >> + ports[pi].port_status = RTE_PORT_STOPPED; >> + >> + printf("Port %d is attached. Now total ports is %d\n", pi, nb_ports); >> + printf("Done\n"); >> } >> >> -int >> -port_is_started(portid_t port_id) >> +void >> +detach_port(uint8_t port_id) >> { >> - if (port_id_is_invalid(port_id)) >> - return -1; >> + portid_t i, pi = 0; >> + char name[RTE_ETH_NAME_MAX_LEN]; >> >> - if (ports[port_id].port_status != RTE_PORT_STARTED) >> - return 0; >> + printf("Detaching a port...\n"); >> >> - return 1; >> + if (!port_is_closed(port_id)) { >> + printf("Please close port first\n"); >> + return; >> + } >> + >> + rte_eth_promiscuous_disable(port_id); >> + >> + if (rte_eal_dev_detach(port_id, name)) >> + return; >> + >> + ports[port_id].enabled = 0; >> + nb_ports = rte_eth_dev_count(); >> + >> + /* set_default_fwd_ports_config(); */ >> + bzero(fwd_ports_ids, sizeof(fwd_ports_ids)); >> + i = 0; >> + FOREACH_PORT(pi, ports) { >> + fwd_ports_ids[i] = pi; >> + i++; >> + } >> + nb_cfg_ports = nb_ports; >> + nb_fwd_ports--; >> + >> + printf("Port '%s' is detached. Now total ports is %d\n", >> + name, nb_ports); >> + printf("Done\n"); >> + return; >> } >> >> void >> @@ -1534,7 +1637,7 @@ pmd_test_exit(void) >> { >> portid_t pt_id; >> >> - for (pt_id = 0; pt_id < nb_ports; pt_id++) { >> + FOREACH_PORT(pt_id, ports) { >> printf("Stopping port %d...", pt_id); >> fflush(stdout); >> rte_eth_dev_close(pt_id); >> @@ -1553,7 +1656,7 @@ struct pmd_test_command { >> >> /* Check the link status of all ports in up to 9s, and print them finally */ >> static void >> -check_all_ports_link_status(uint8_t port_num, uint32_t port_mask) >> +check_all_ports_link_status(uint32_t port_mask) >> { >> #define CHECK_INTERVAL 100 /* 100ms */ >> #define MAX_CHECK_TIME 90 /* 9s (90 * 100ms) in total */ >> @@ -1564,7 +1667,7 @@ check_all_ports_link_status(uint8_t port_num, uint32_t port_mask) >> fflush(stdout); >> for (count = 0; count <= MAX_CHECK_TIME; count++) { >> all_ports_up = 1; >> - for (portid = 0; portid < port_num; portid++) { >> + FOREACH_PORT(portid, ports) { >> if ((port_mask & (1 << portid)) == 0) >> continue; >> memset(&link, 0, sizeof(link)); >> @@ -1688,7 +1791,7 @@ init_port_config(void) >> portid_t pid; >> struct rte_port *port; >> >> - for (pid = 0; pid < nb_ports; pid++) { >> + FOREACH_PORT(pid, ports) { >> port = &ports[pid]; >> port->dev_conf.rxmode = rx_mode; >> port->dev_conf.fdir_conf = fdir_conf; >> @@ -1877,7 +1980,7 @@ main(int argc, char** argv) >> >> nb_ports = (portid_t) rte_eth_dev_count(); >> if (nb_ports == 0) >> - rte_exit(EXIT_FAILURE, "No probed ethernet device\n"); >> + RTE_LOG(WARNING, EAL, "No probed ethernet devices\n"); >> >> set_def_fwd_config(); >> if (nb_lcores == 0) >> @@ -1899,7 +2002,7 @@ main(int argc, char** argv) >> rte_exit(EXIT_FAILURE, "Start ports failed\n"); >> >> /* set all ports to promiscuous mode by default */ >> - for (port_id = 0; port_id < nb_ports; port_id++) >> + FOREACH_PORT(port_id, ports) >> rte_eth_promiscuous_enable(port_id); >> >> #ifdef RTE_LIBRTE_CMDLINE >> diff --git a/app/test-pmd/testpmd.h b/app/test-pmd/testpmd.h >> index 8f5e6c7..109c670 100644 >> --- a/app/test-pmd/testpmd.h >> +++ b/app/test-pmd/testpmd.h >> @@ -134,6 +134,7 @@ struct fwd_stream { >> * The data structure associated with each port. >> */ >> struct rte_port { >> + uint8_t enabled; /**< Port enabled or not */ >> struct rte_eth_dev_info dev_info; /**< PCI info + driver name */ >> struct rte_eth_conf dev_conf; /**< Port configuration. */ >> struct ether_addr eth_addr; /**< Port ethernet address */ >> @@ -159,6 +160,14 @@ struct rte_port { >> struct rte_eth_txconf tx_conf; /**< tx configuration */ >> }; >> >> +extern portid_t __rte_unused >> +find_next_port(portid_t p, struct rte_port *ports, int size); >> + >> +#define FOREACH_PORT(p, ports) \ >> + for (p = find_next_port(0, ports, RTE_MAX_ETHPORTS); \ >> + p < RTE_MAX_ETHPORTS; \ >> + p = find_next_port(p + 1, ports, RTE_MAX_ETHPORTS)) >> + >> /** >> * The data structure associated with each forwarding logical core. >> * The logical cores are internally numbered by a core index from 0 to >> @@ -515,6 +524,8 @@ int init_port_dcb_config(portid_t pid,struct dcb_config *dcb_conf); >> int start_port(portid_t pid); >> void stop_port(portid_t pid); >> void close_port(portid_t pid); >> +void attach_port(char *identifier); >> +void detach_port(uint8_t port_id); >> int all_ports_stopped(void); >> int port_is_started(portid_t port_id); >> void pmd_test_exit(void); >> @@ -558,10 +569,15 @@ void get_ethertype_filter(uint8_t port_id, uint16_t index); >> void get_2tuple_filter(uint8_t port_id, uint16_t index); >> void get_5tuple_filter(uint8_t port_id, uint16_t index); >> void get_flex_filter(uint8_t port_id, uint16_t index); >> -int port_id_is_invalid(portid_t port_id); >> int rx_queue_id_is_invalid(queueid_t rxq_id); >> int tx_queue_id_is_invalid(queueid_t txq_id); >> >> +enum print_warning { >> + ENABLED_WARN = 0, >> + DISABLED_WARN >> +}; >> +int port_id_is_invalid(portid_t port_id, enum print_warning warning); >> + >> /* >> * Work-around of a compilation error with ICC on invocations of the >> * rte_be_to_cpu_16() function. >> diff --git a/doc/guides/testpmd_app_ug/testpmd_funcs.rst b/doc/guides/testpmd_app_ug/testpmd_funcs.rst >> index 218835a..1cacbcf 100644 >> --- a/doc/guides/testpmd_app_ug/testpmd_funcs.rst >> +++ b/doc/guides/testpmd_app_ug/testpmd_funcs.rst >> @@ -808,6 +808,63 @@ The following sections show functions for configuring ports. >> >> Port configuration changes only become active when forwarding is started/restarted. >> >> +port attach >> +~~~~~~~~~~~ >> + >> +Attach a port specified by pci address or virtual device args. >> + >> +To attach a new pci device, the device should be recognized by kernel first. >> +Then it should be moved under DPDK management. >> +Finally the port can be attached to testpmd. >> +On the other hand, to attach a port created by virtual device, above steps are not needed. >> + >> +port attach (identifier) >> + >> +For example, to attach a port that pci address is 0000:02:00.0. >> + >> +.. code-block:: console >> + >> + testpmd> port attach 0000:02:00.0 >> + Attaching a new port... >> + ... snip ... >> + Port 0 is attached. Now total ports is 1 >> + Done >> + >> +For example, to attach a port created by pcap PMD. >> + >> +.. code-block:: console >> + >> + testpmd> port attach eth_pcap0,iface=eth0 >> + Attaching a new port... >> + ... snip ... >> + Port 0 is attached. Now total ports is 1 >> + Done >> + >> +In this case, identifier is "eth_pcap0,iface=eth0". >> +This identifier format is the same as "--vdev" format of DPDK applications. >> + >> +port detach >> +~~~~~~~~~~~ >> + >> +Detach a specific port. >> + >> +Before detaching a port, the port should be closed. >> +Also to remove a pci device completely from the system, first detach the port from testpmd. >> +Then the device should be moved under kernel management. >> +Finally the device can be remove using kernel pci hotplug functionality. >> +On the other hand, to remove a port created by virtual device, above steps are not needed. >> + >> +port detach (port_id) >> + >> +For example, to detach a port 0. >> + >> +.. code-block:: console >> + >> + testpmd> port detach 0 >> + Detaching a port... >> + ... snip ... >> + Done >> + >> port start >> ~~~~~~~~~~ >> >
> >> +.. code-block:: console > >> + > >> + testpmd> port attach 0000:02:00.0 > >> + Attaching a new port... > >> + ... snip ... > >> + Port 0 is attached. Now total ports is 1 > >> + Done > >> +port detach > >> +~~~~~~~~~~~ > >> + > >> +Detach a specific port. > >> + > >> +Before detaching a port, the port should be closed. > >> +Also to remove a pci device completely from the system, first detach the port from testpmd. > >> +Then the device should be moved under kernel management. > >> +Finally the device can be remove using kernel pci hotplug functionality. Hi Tetsuya, Reword "remove" to "removed" > >> +On the other hand, to remove a port created by virtual device, above steps are not needed. Reword " created by virtual device" to "created by a virtual device" > > > >> + > >> +port detach (port_id) > >> + > >> +For example, to detach a port 0. > >> + > >> +.. code-block:: console > >> + > >> + testpmd> port detach 0 > >> + Detaching a port... > >> + ... snip ... > >> + Done > >> + > >> port start > >> ~~~~~~~~~~ > >> > >> -- > >> 1.9.1 > > Regards, > > > > Bernard. > >
On 2015/02/03 15:59, Qiu, Michael wrote: > On 2/1/2015 12:02 PM, Tetsuya Mukawa wrote: >> The patch introduces following commands. >> - port attach [ident] >> - port detach [port_id] >> - attach: attaching a port >> - detach: detaching a port >> - ident: pci address of physical device. >> Or device name and paramerters of virtual device. >> (ex. 0000:02:00.0, eth_pcap0,iface=eth0) >> - port_id: port identifier >> >> v5: >> - Add testpmd documentation. >> (Thanks to Iremonger, Bernard) >> v4: >> - Fix strings of command help. >> >> Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp> >> --- >> app/test-pmd/cmdline.c | 133 +++++++++++++++---- >> app/test-pmd/config.c | 116 +++++++++------- >> app/test-pmd/parameters.c | 22 ++- >> app/test-pmd/testpmd.c | 199 +++++++++++++++++++++------- >> app/test-pmd/testpmd.h | 18 ++- >> doc/guides/testpmd_app_ug/testpmd_funcs.rst | 57 ++++++++ >> 6 files changed, 415 insertions(+), 130 deletions(-) >> >> diff --git a/app/test-pmd/cmdline.c b/app/test-pmd/cmdline.c >> index 4beb404..2f813d8 100644 >> --- a/app/test-pmd/cmdline.c >> +++ b/app/test-pmd/cmdline.c >> @@ -572,6 +572,12 @@ static void cmd_help_long_parsed(void *parsed_result, >> "port close (port_id|all)\n" >> " Close all ports or port_id.\n\n" >> >> + "port attach (ident)\n" >> + " Attach physical or virtual dev by pci address or virtual device name\n\n" >> + >> + "port detach (port_id)\n" >> + " Detach physical or virtual dev by port_id\n\n" >> + >> "port config (port_id|all)" >> " speed (10|100|1000|10000|40000|auto)" >> " duplex (half|full|auto)\n" >> @@ -848,6 +854,89 @@ cmdline_parse_inst_t cmd_operate_specific_port = { >> }, >> }; >> >> +/* *** attach a specificied port *** */ >> +struct cmd_operate_attach_port_result { >> + cmdline_fixed_string_t port; >> + cmdline_fixed_string_t keyword; >> + cmdline_fixed_string_t identifier; >> +}; >> + >> +static void cmd_operate_attach_port_parsed(void *parsed_result, >> + __attribute__((unused)) struct cmdline *cl, >> + __attribute__((unused)) void *data) >> +{ >> + struct cmd_operate_attach_port_result *res = parsed_result; >> + >> + if (!strcmp(res->keyword, "attach")) >> + attach_port(res->identifier); >> + else >> + printf("Unknown parameter\n"); >> +} >> + >> +cmdline_parse_token_string_t cmd_operate_attach_port_port = >> + TOKEN_STRING_INITIALIZER(struct cmd_operate_attach_port_result, >> + port, "port"); >> +cmdline_parse_token_string_t cmd_operate_attach_port_keyword = >> + TOKEN_STRING_INITIALIZER(struct cmd_operate_attach_port_result, >> + keyword, "attach"); >> +cmdline_parse_token_string_t cmd_operate_attach_port_identifier = >> + TOKEN_STRING_INITIALIZER(struct cmd_operate_attach_port_result, >> + identifier, NULL); >> + >> +cmdline_parse_inst_t cmd_operate_attach_port = { >> + .f = cmd_operate_attach_port_parsed, >> + .data = NULL, >> + .help_str = "port attach identifier, " >> + "identifier: pci address or virtual dev name", >> + .tokens = { >> + (void *)&cmd_operate_attach_port_port, >> + (void *)&cmd_operate_attach_port_keyword, >> + (void *)&cmd_operate_attach_port_identifier, >> + NULL, >> + }, >> +}; >> + >> +/* *** detach a specificied port *** */ >> +struct cmd_operate_detach_port_result { >> + cmdline_fixed_string_t port; >> + cmdline_fixed_string_t keyword; >> + uint8_t port_id; >> +}; >> + >> +static void cmd_operate_detach_port_parsed(void *parsed_result, >> + __attribute__((unused)) struct cmdline *cl, >> + __attribute__((unused)) void *data) >> +{ >> + struct cmd_operate_detach_port_result *res = parsed_result; >> + >> + if (!strcmp(res->keyword, "detach")) >> + detach_port(res->port_id); >> + else >> + printf("Unknown parameter\n"); >> +} >> + >> +cmdline_parse_token_string_t cmd_operate_detach_port_port = >> + TOKEN_STRING_INITIALIZER(struct cmd_operate_detach_port_result, >> + port, "port"); >> +cmdline_parse_token_string_t cmd_operate_detach_port_keyword = >> + TOKEN_STRING_INITIALIZER(struct cmd_operate_detach_port_result, >> + keyword, "detach"); >> +cmdline_parse_token_num_t cmd_operate_detach_port_port_id = >> + TOKEN_NUM_INITIALIZER(struct cmd_operate_detach_port_result, >> + port_id, UINT8); >> + >> +cmdline_parse_inst_t cmd_operate_detach_port = { >> + .f = cmd_operate_detach_port_parsed, >> + .data = NULL, >> + .help_str = "port detach port_id", >> + .tokens = { >> + (void *)&cmd_operate_detach_port_port, >> + (void *)&cmd_operate_detach_port_keyword, >> + (void *)&cmd_operate_detach_port_port_id, >> + NULL, >> + }, >> +}; >> + >> /* *** configure speed for all ports *** */ >> struct cmd_config_speed_all { >> cmdline_fixed_string_t port; >> @@ -902,7 +991,7 @@ cmd_config_speed_all_parsed(void *parsed_result, >> return; >> } >> >> - for (pid = 0; pid < nb_ports; pid++) { >> + FOREACH_PORT(pid, ports) { >> ports[pid].dev_conf.link_speed = link_speed; >> ports[pid].dev_conf.link_duplex = link_duplex; >> } >> @@ -970,10 +1059,8 @@ cmd_config_speed_specific_parsed(void *parsed_result, >> return; >> } >> >> - if (res->id >= nb_ports) { >> - printf("Port id %d must be less than %d\n", res->id, nb_ports); >> + if (port_id_is_invalid(res->id, ENABLED_WARN)) >> return; >> - } >> >> if (!strcmp(res->value1, "10")) >> link_speed = ETH_LINK_SPEED_10; >> @@ -1533,7 +1620,7 @@ cmd_config_rxtx_queue_parsed(void *parsed_result, >> return; >> } >> >> - if (port_id_is_invalid(res->portid)) >> + if (port_id_is_invalid(res->portid, ENABLED_WARN)) >> return; >> >> if (port_is_started(res->portid) != 1) { >> @@ -2876,7 +2963,7 @@ cmd_tx_cksum_parsed(void *parsed_result, >> uint16_t ol_flags, mask = 0; >> struct rte_eth_dev_info dev_info; >> >> - if (port_id_is_invalid(res->port_id)) { >> + if (port_id_is_invalid(res->port_id, ENABLED_WARN)) { >> printf("invalid port %d\n", res->port_id); >> return; >> } >> @@ -3003,7 +3090,7 @@ cmd_tso_set_parsed(void *parsed_result, >> struct cmd_tso_set_result *res = parsed_result; >> struct rte_eth_dev_info dev_info; >> >> - if (port_id_is_invalid(res->port_id)) >> + if (port_id_is_invalid(res->port_id, ENABLED_WARN)) >> return; >> >> if (!strcmp(res->mode, "set")) >> @@ -3979,10 +4066,8 @@ static void cmd_set_bond_mac_addr_parsed(void *parsed_result, >> struct cmd_set_bond_mac_addr_result *res = parsed_result; >> int ret; >> >> - if (res->port_num >= nb_ports) { >> - printf("Port id %d must be less than %d\n", res->port_num, nb_ports); >> + if (port_id_is_invalid(res->port_num, ENABLED_WARN)) >> return; >> - } >> >> ret = rte_eth_bond_mac_address_set(res->port_num, &res->address); >> >> @@ -4219,7 +4304,7 @@ static void cmd_set_promisc_mode_parsed(void *parsed_result, >> >> /* all ports */ >> if (allports) { >> - for (i = 0; i < nb_ports; i++) { >> + FOREACH_PORT(i, ports) { >> if (enable) >> rte_eth_promiscuous_enable(i); >> else >> @@ -4299,7 +4384,7 @@ static void cmd_set_allmulti_mode_parsed(void *parsed_result, >> >> /* all ports */ >> if (allports) { >> - for (i = 0; i < nb_ports; i++) { >> + FOREACH_PORT(i, ports) { >> if (enable) >> rte_eth_allmulticast_enable(i); >> else >> @@ -5484,25 +5569,25 @@ static void cmd_showportall_parsed(void *parsed_result, >> struct cmd_showportall_result *res = parsed_result; >> if (!strcmp(res->show, "clear")) { >> if (!strcmp(res->what, "stats")) >> - for (i = 0; i < nb_ports; i++) >> + FOREACH_PORT(i, ports) >> nic_stats_clear(i); >> else if (!strcmp(res->what, "xstats")) >> - for (i = 0; i < nb_ports; i++) >> + FOREACH_PORT(i, ports) >> nic_xstats_clear(i); >> } else if (!strcmp(res->what, "info")) >> - for (i = 0; i < nb_ports; i++) >> + FOREACH_PORT(i, ports) >> port_infos_display(i); >> else if (!strcmp(res->what, "stats")) >> - for (i = 0; i < nb_ports; i++) >> + FOREACH_PORT(i, ports) >> nic_stats_display(i); >> else if (!strcmp(res->what, "xstats")) >> - for (i = 0; i < nb_ports; i++) >> + FOREACH_PORT(i, ports) >> nic_xstats_display(i); >> else if (!strcmp(res->what, "fdir")) >> - for (i = 0; i < nb_ports; i++) >> + FOREACH_PORT(i, ports) >> fdir_get_infos(i); >> else if (!strcmp(res->what, "stat_qmap")) >> - for (i = 0; i < nb_ports; i++) >> + FOREACH_PORT(i, ports) >> nic_stats_mapping_display(i); >> } >> >> @@ -8756,6 +8841,8 @@ cmdline_parse_ctx_t main_ctx[] = { >> (cmdline_parse_inst_t *)&cmd_set_qmap, >> (cmdline_parse_inst_t *)&cmd_operate_port, >> (cmdline_parse_inst_t *)&cmd_operate_specific_port, >> + (cmdline_parse_inst_t *)&cmd_operate_attach_port, >> + (cmdline_parse_inst_t *)&cmd_operate_detach_port, >> (cmdline_parse_inst_t *)&cmd_config_speed_all, >> (cmdline_parse_inst_t *)&cmd_config_speed_specific, >> (cmdline_parse_inst_t *)&cmd_config_rx_tx, >> @@ -8830,7 +8917,7 @@ prompt(void) >> static void >> cmd_reconfig_device_queue(portid_t id, uint8_t dev, uint8_t queue) >> { >> - if (id < nb_ports) { >> + if (!port_id_is_invalid(id, DISABLED_WARN)) { >> /* check if need_reconfig has been set to 1 */ >> if (ports[id].need_reconfig == 0) >> ports[id].need_reconfig = dev; >> @@ -8840,7 +8927,7 @@ cmd_reconfig_device_queue(portid_t id, uint8_t dev, uint8_t queue) >> } else { >> portid_t pid; >> >> - for (pid = 0; pid < nb_ports; pid++) { >> + FOREACH_PORT(pid, ports) { >> /* check if need_reconfig has been set to 1 */ >> if (ports[pid].need_reconfig == 0) >> ports[pid].need_reconfig = dev; >> @@ -8858,10 +8945,8 @@ bypass_is_supported(portid_t port_id) >> struct rte_port *port; >> struct rte_pci_id *pci_id; >> >> - if (port_id >= nb_ports) { >> - printf("\tPort id must be less than %d.\n", nb_ports); >> + if (port_id_is_invalid(port_id, ENABLED_WARN)) >> return 0; >> - } >> >> /* Get the device id. */ >> port = &ports[port_id]; >> diff --git a/app/test-pmd/config.c b/app/test-pmd/config.c >> index c40f819..32d8f9a 100644 >> --- a/app/test-pmd/config.c >> +++ b/app/test-pmd/config.c >> @@ -124,11 +124,15 @@ nic_stats_display(portid_t port_id) >> struct rte_eth_stats stats; >> struct rte_port *port = &ports[port_id]; >> uint8_t i; >> + portid_t pid; >> >> static const char *nic_stats_border = "########################"; >> >> - if (port_id >= nb_ports) { >> - printf("Invalid port, range is [0, %d]\n", nb_ports - 1); >> + if (port_id_is_invalid(port_id, ENABLED_WARN)) { >> + printf("Valid port range is [0"); >> + FOREACH_PORT(pid, ports) >> + printf(", %d", pid); >> + printf("]\n"); >> return; >> } >> rte_eth_stats_get(port_id, &stats); >> @@ -201,8 +205,13 @@ nic_stats_display(portid_t port_id) >> void >> nic_stats_clear(portid_t port_id) >> { >> - if (port_id >= nb_ports) { >> - printf("Invalid port, range is [0, %d]\n", nb_ports - 1); >> + portid_t pid; >> + >> + if (port_id_is_invalid(port_id, ENABLED_WARN)) { >> + printf("Valid port range is [0"); >> + FOREACH_PORT(pid, ports) >> + printf(", %d", pid); >> + printf("]\n"); >> return; >> } >> rte_eth_stats_reset(port_id); >> @@ -249,11 +258,15 @@ nic_stats_mapping_display(portid_t port_id) >> { >> struct rte_port *port = &ports[port_id]; >> uint16_t i; >> + portid_t pid; >> >> static const char *nic_stats_mapping_border = "########################"; >> >> - if (port_id >= nb_ports) { >> - printf("Invalid port, range is [0, %d]\n", nb_ports - 1); >> + if (port_id_is_invalid(port_id, ENABLED_WARN)) { >> + printf("Valid port range is [0"); >> + FOREACH_PORT(pid, ports) >> + printf(", %d", pid); >> + printf("]\n"); >> return; >> } >> >> @@ -302,9 +315,13 @@ port_infos_display(portid_t port_id) >> int vlan_offload; >> struct rte_mempool * mp; >> static const char *info_border = "*********************"; >> + portid_t pid; >> >> - if (port_id >= nb_ports) { >> - printf("Invalid port, range is [0, %d]\n", nb_ports - 1); >> + if (port_id_is_invalid(port_id, ENABLED_WARN)) { >> + printf("Valid port range is [0"); >> + FOREACH_PORT(pid, ports) >> + printf(", %d", pid); >> + printf("]\n"); >> return; >> } >> port = &ports[port_id]; >> @@ -362,11 +379,14 @@ port_infos_display(portid_t port_id) >> } >> >> int >> -port_id_is_invalid(portid_t port_id) >> +port_id_is_invalid(portid_t port_id, enum print_warning warning) >> { >> - if (port_id < nb_ports) >> + if (ports[port_id].enabled) > Here maybe care about overflow, it could be passed a value of > RTE_PORT_ALL, which is 255. Thanks, I will fix like above. Tetsuya > Thanks, > Michael >> return 0; >> - printf("Invalid port %d (must be < nb_ports=%d)\n", port_id, nb_ports); >> + >> + if (warning == ENABLED_WARN) >> + printf("Invalid port %d\n", port_id); >> + >> return 1; >> } >> >> @@ -425,7 +445,7 @@ port_reg_bit_display(portid_t port_id, uint32_t reg_off, uint8_t bit_x) >> uint32_t reg_v; >> >> >> - if (port_id_is_invalid(port_id)) >> + if (port_id_is_invalid(port_id, ENABLED_WARN)) >> return; >> if (port_reg_off_is_invalid(port_id, reg_off)) >> return; >> @@ -444,7 +464,7 @@ port_reg_bit_field_display(portid_t port_id, uint32_t reg_off, >> uint8_t l_bit; >> uint8_t h_bit; >> >> - if (port_id_is_invalid(port_id)) >> + if (port_id_is_invalid(port_id, ENABLED_WARN)) >> return; >> if (port_reg_off_is_invalid(port_id, reg_off)) >> return; >> @@ -471,7 +491,7 @@ port_reg_display(portid_t port_id, uint32_t reg_off) >> { >> uint32_t reg_v; >> >> - if (port_id_is_invalid(port_id)) >> + if (port_id_is_invalid(port_id, ENABLED_WARN)) >> return; >> if (port_reg_off_is_invalid(port_id, reg_off)) >> return; >> @@ -485,7 +505,7 @@ port_reg_bit_set(portid_t port_id, uint32_t reg_off, uint8_t bit_pos, >> { >> uint32_t reg_v; >> >> - if (port_id_is_invalid(port_id)) >> + if (port_id_is_invalid(port_id, ENABLED_WARN)) >> return; >> if (port_reg_off_is_invalid(port_id, reg_off)) >> return; >> @@ -513,7 +533,7 @@ port_reg_bit_field_set(portid_t port_id, uint32_t reg_off, >> uint8_t l_bit; >> uint8_t h_bit; >> >> - if (port_id_is_invalid(port_id)) >> + if (port_id_is_invalid(port_id, ENABLED_WARN)) >> return; >> if (port_reg_off_is_invalid(port_id, reg_off)) >> return; >> @@ -547,7 +567,7 @@ port_reg_bit_field_set(portid_t port_id, uint32_t reg_off, >> void >> port_reg_set(portid_t port_id, uint32_t reg_off, uint32_t reg_v) >> { >> - if (port_id_is_invalid(port_id)) >> + if (port_id_is_invalid(port_id, ENABLED_WARN)) >> return; >> if (port_reg_off_is_invalid(port_id, reg_off)) >> return; >> @@ -560,7 +580,7 @@ port_mtu_set(portid_t port_id, uint16_t mtu) >> { >> int diag; >> >> - if (port_id_is_invalid(port_id)) >> + if (port_id_is_invalid(port_id, ENABLED_WARN)) >> return; >> diag = rte_eth_dev_set_mtu(port_id, mtu); >> if (diag == 0) >> @@ -723,7 +743,7 @@ rx_ring_desc_display(portid_t port_id, queueid_t rxq_id, uint16_t rxd_id) >> { >> const struct rte_memzone *rx_mz; >> >> - if (port_id_is_invalid(port_id)) >> + if (port_id_is_invalid(port_id, ENABLED_WARN)) >> return; >> if (rx_queue_id_is_invalid(rxq_id)) >> return; >> @@ -740,7 +760,7 @@ tx_ring_desc_display(portid_t port_id, queueid_t txq_id, uint16_t txd_id) >> { >> const struct rte_memzone *tx_mz; >> >> - if (port_id_is_invalid(port_id)) >> + if (port_id_is_invalid(port_id, ENABLED_WARN)) >> return; >> if (tx_queue_id_is_invalid(txq_id)) >> return; >> @@ -796,7 +816,7 @@ port_rss_reta_info(portid_t port_id, >> uint16_t i, idx, shift; >> int ret; >> >> - if (port_id_is_invalid(port_id)) >> + if (port_id_is_invalid(port_id, ENABLED_WARN)) >> return; >> >> ret = rte_eth_dev_rss_reta_query(port_id, reta_conf, nb_entries); >> @@ -828,7 +848,7 @@ port_rss_hash_conf_show(portid_t port_id, int show_rss_key) >> uint8_t i; >> int diag; >> >> - if (port_id_is_invalid(port_id)) >> + if (port_id_is_invalid(port_id, ENABLED_WARN)) >> return; >> /* Get RSS hash key if asked to display it */ >> rss_conf.rss_key = (show_rss_key) ? rss_key : NULL; >> @@ -1406,12 +1426,8 @@ set_fwd_ports_list(unsigned int *portlist, unsigned int nb_pt) >> again: >> for (i = 0; i < nb_pt; i++) { >> port_id = (portid_t) portlist[i]; >> - if (port_id >= nb_ports) { >> - printf("Invalid port id %u >= %u\n", >> - (unsigned int) port_id, >> - (unsigned int) nb_ports); >> + if (port_id_is_invalid(port_id, ENABLED_WARN)) >> return; >> - } >> if (record_now) >> fwd_ports_ids[i] = port_id; >> } >> @@ -1569,7 +1585,7 @@ vlan_extend_set(portid_t port_id, int on) >> int diag; >> int vlan_offload; >> >> - if (port_id_is_invalid(port_id)) >> + if (port_id_is_invalid(port_id, ENABLED_WARN)) >> return; >> >> vlan_offload = rte_eth_dev_get_vlan_offload(port_id); >> @@ -1591,7 +1607,7 @@ rx_vlan_strip_set(portid_t port_id, int on) >> int diag; >> int vlan_offload; >> >> - if (port_id_is_invalid(port_id)) >> + if (port_id_is_invalid(port_id, ENABLED_WARN)) >> return; >> >> vlan_offload = rte_eth_dev_get_vlan_offload(port_id); >> @@ -1612,7 +1628,7 @@ rx_vlan_strip_set_on_queue(portid_t port_id, uint16_t queue_id, int on) >> { >> int diag; >> >> - if (port_id_is_invalid(port_id)) >> + if (port_id_is_invalid(port_id, ENABLED_WARN)) >> return; >> >> diag = rte_eth_dev_set_vlan_strip_on_queue(port_id, queue_id, on); >> @@ -1627,7 +1643,7 @@ rx_vlan_filter_set(portid_t port_id, int on) >> int diag; >> int vlan_offload; >> >> - if (port_id_is_invalid(port_id)) >> + if (port_id_is_invalid(port_id, ENABLED_WARN)) >> return; >> >> vlan_offload = rte_eth_dev_get_vlan_offload(port_id); >> @@ -1648,7 +1664,7 @@ rx_vft_set(portid_t port_id, uint16_t vlan_id, int on) >> { >> int diag; >> >> - if (port_id_is_invalid(port_id)) >> + if (port_id_is_invalid(port_id, ENABLED_WARN)) >> return; >> if (vlan_id_is_invalid(vlan_id)) >> return; >> @@ -1665,7 +1681,7 @@ rx_vlan_all_filter_set(portid_t port_id, int on) >> { >> uint16_t vlan_id; >> >> - if (port_id_is_invalid(port_id)) >> + if (port_id_is_invalid(port_id, ENABLED_WARN)) >> return; >> for (vlan_id = 0; vlan_id < 4096; vlan_id++) >> rx_vft_set(port_id, vlan_id, on); >> @@ -1675,7 +1691,7 @@ void >> vlan_tpid_set(portid_t port_id, uint16_t tp_id) >> { >> int diag; >> - if (port_id_is_invalid(port_id)) >> + if (port_id_is_invalid(port_id, ENABLED_WARN)) >> return; >> >> diag = rte_eth_dev_set_vlan_ether_type(port_id, tp_id); >> @@ -1690,7 +1706,7 @@ vlan_tpid_set(portid_t port_id, uint16_t tp_id) >> void >> tx_vlan_set(portid_t port_id, uint16_t vlan_id) >> { >> - if (port_id_is_invalid(port_id)) >> + if (port_id_is_invalid(port_id, ENABLED_WARN)) >> return; >> if (vlan_id_is_invalid(vlan_id)) >> return; >> @@ -1701,7 +1717,7 @@ tx_vlan_set(portid_t port_id, uint16_t vlan_id) >> void >> tx_vlan_reset(portid_t port_id) >> { >> - if (port_id_is_invalid(port_id)) >> + if (port_id_is_invalid(port_id, ENABLED_WARN)) >> return; >> ports[port_id].tx_ol_flags &= ~TESTPMD_TX_OFFLOAD_INSERT_VLAN; >> } >> @@ -1709,7 +1725,7 @@ tx_vlan_reset(portid_t port_id) >> void >> tx_vlan_pvid_set(portid_t port_id, uint16_t vlan_id, int on) >> { >> - if (port_id_is_invalid(port_id)) >> + if (port_id_is_invalid(port_id, ENABLED_WARN)) >> return; >> >> rte_eth_dev_set_vlan_pvid(port_id, vlan_id, on); >> @@ -1721,7 +1737,7 @@ set_qmap(portid_t port_id, uint8_t is_rx, uint16_t queue_id, uint8_t map_value) >> uint16_t i; >> uint8_t existing_mapping_found = 0; >> >> - if (port_id_is_invalid(port_id)) >> + if (port_id_is_invalid(port_id, ENABLED_WARN)) >> return; >> >> if (is_rx ? (rx_queue_id_is_invalid(queue_id)) : (tx_queue_id_is_invalid(queue_id))) >> @@ -1773,7 +1789,7 @@ fdir_add_signature_filter(portid_t port_id, uint8_t queue_id, >> { >> int diag; >> >> - if (port_id_is_invalid(port_id)) >> + if (port_id_is_invalid(port_id, ENABLED_WARN)) >> return; >> >> diag = rte_eth_dev_fdir_add_signature_filter(port_id, fdir_filter, >> @@ -1791,7 +1807,7 @@ fdir_update_signature_filter(portid_t port_id, uint8_t queue_id, >> { >> int diag; >> >> - if (port_id_is_invalid(port_id)) >> + if (port_id_is_invalid(port_id, ENABLED_WARN)) >> return; >> >> diag = rte_eth_dev_fdir_update_signature_filter(port_id, fdir_filter, >> @@ -1809,7 +1825,7 @@ fdir_remove_signature_filter(portid_t port_id, >> { >> int diag; >> >> - if (port_id_is_invalid(port_id)) >> + if (port_id_is_invalid(port_id, ENABLED_WARN)) >> return; >> >> diag = rte_eth_dev_fdir_remove_signature_filter(port_id, fdir_filter); >> @@ -1881,7 +1897,7 @@ fdir_get_infos(portid_t port_id) >> >> static const char *fdir_stats_border = "########################"; >> >> - if (port_id_is_invalid(port_id)) >> + if (port_id_is_invalid(port_id, ENABLED_WARN)) >> return; >> ret = rte_eth_dev_filter_supported(port_id, RTE_ETH_FILTER_FDIR); >> if (ret < 0) { >> @@ -1955,7 +1971,7 @@ fdir_add_perfect_filter(portid_t port_id, uint16_t soft_id, uint8_t queue_id, >> { >> int diag; >> >> - if (port_id_is_invalid(port_id)) >> + if (port_id_is_invalid(port_id, ENABLED_WARN)) >> return; >> >> diag = rte_eth_dev_fdir_add_perfect_filter(port_id, fdir_filter, >> @@ -1973,7 +1989,7 @@ fdir_update_perfect_filter(portid_t port_id, uint16_t soft_id, uint8_t queue_id, >> { >> int diag; >> >> - if (port_id_is_invalid(port_id)) >> + if (port_id_is_invalid(port_id, ENABLED_WARN)) >> return; >> >> diag = rte_eth_dev_fdir_update_perfect_filter(port_id, fdir_filter, >> @@ -1991,7 +2007,7 @@ fdir_remove_perfect_filter(portid_t port_id, uint16_t soft_id, >> { >> int diag; >> >> - if (port_id_is_invalid(port_id)) >> + if (port_id_is_invalid(port_id, ENABLED_WARN)) >> return; >> >> diag = rte_eth_dev_fdir_remove_perfect_filter(port_id, fdir_filter, >> @@ -2008,7 +2024,7 @@ fdir_set_masks(portid_t port_id, struct rte_fdir_masks *fdir_masks) >> { >> int diag; >> >> - if (port_id_is_invalid(port_id)) >> + if (port_id_is_invalid(port_id, ENABLED_WARN)) >> return; >> >> diag = rte_eth_dev_fdir_set_masks(port_id, fdir_masks); >> @@ -2085,7 +2101,7 @@ set_vf_traffic(portid_t port_id, uint8_t is_rx, uint16_t vf, uint8_t on) >> { >> int diag; >> >> - if (port_id_is_invalid(port_id)) >> + if (port_id_is_invalid(port_id, ENABLED_WARN)) >> return; >> if (is_rx) >> diag = rte_eth_dev_set_vf_rx(port_id,vf,on); >> @@ -2107,7 +2123,7 @@ set_vf_rx_vlan(portid_t port_id, uint16_t vlan_id, uint64_t vf_mask, uint8_t on) >> { >> int diag; >> >> - if (port_id_is_invalid(port_id)) >> + if (port_id_is_invalid(port_id, ENABLED_WARN)) >> return; >> if (vlan_id_is_invalid(vlan_id)) >> return; >> @@ -2124,7 +2140,7 @@ set_queue_rate_limit(portid_t port_id, uint16_t queue_idx, uint16_t rate) >> int diag; >> struct rte_eth_link link; >> >> - if (port_id_is_invalid(port_id)) >> + if (port_id_is_invalid(port_id, ENABLED_WARN)) >> return 1; >> rte_eth_link_get_nowait(port_id, &link); >> if (rate > link.link_speed) { >> @@ -2149,7 +2165,7 @@ set_vf_rate_limit(portid_t port_id, uint16_t vf, uint16_t rate, uint64_t q_msk) >> if (q_msk == 0) >> return 0; >> >> - if (port_id_is_invalid(port_id)) >> + if (port_id_is_invalid(port_id, ENABLED_WARN)) >> return 1; >> rte_eth_link_get_nowait(port_id, &link); >> if (rate > link.link_speed) { >> diff --git a/app/test-pmd/parameters.c b/app/test-pmd/parameters.c >> index adf3203..6f2af18 100644 >> --- a/app/test-pmd/parameters.c >> +++ b/app/test-pmd/parameters.c >> @@ -376,6 +376,7 @@ parse_portnuma_config(const char *q_arg) >> }; >> unsigned long int_fld[_NUM_FLD]; >> char *str_fld[_NUM_FLD]; >> + portid_t pid; >> >> /* reset from value set at definition */ >> while ((p = strchr(p0,'(')) != NULL) { >> @@ -397,8 +398,11 @@ parse_portnuma_config(const char *q_arg) >> return -1; >> } >> port_id = (uint8_t)int_fld[FLD_PORT]; >> - if (port_id >= nb_ports) { >> - printf("Invalid port, range is [0, %d]\n", nb_ports - 1); >> + if (port_id_is_invalid(port_id, ENABLED_WARN)) { >> + printf("Valid port range is [0"); >> + FOREACH_PORT(pid, ports) >> + printf(", %d", pid); >> + printf("]\n"); >> return -1; >> } >> socket_id = (uint8_t)int_fld[FLD_SOCKET]; >> @@ -429,6 +433,7 @@ parse_ringnuma_config(const char *q_arg) >> }; >> unsigned long int_fld[_NUM_FLD]; >> char *str_fld[_NUM_FLD]; >> + portid_t pid; >> #define RX_RING_ONLY 0x1 >> #define TX_RING_ONLY 0x2 >> #define RXTX_RING 0x3 >> @@ -453,8 +458,11 @@ parse_ringnuma_config(const char *q_arg) >> return -1; >> } >> port_id = (uint8_t)int_fld[FLD_PORT]; >> - if (port_id >= nb_ports) { >> - printf("Invalid port, range is [0, %d]\n", nb_ports - 1); >> + if (port_id_is_invalid(port_id, ENABLED_WARN)) { >> + printf("Valid port range is [0"); >> + FOREACH_PORT(pid, ports) >> + printf(", %d", pid); >> + printf("]\n"); >> return -1; >> } >> socket_id = (uint8_t)int_fld[FLD_SOCKET]; >> @@ -626,12 +634,12 @@ launch_args_parse(int argc, char** argv) >> #endif >> if (!strcmp(lgopts[opt_idx].name, "nb-ports")) { >> n = atoi(optarg); >> - if (n > 0 && n <= nb_ports) >> + if (n > 0 && >> + !port_id_is_invalid(n, DISABLED_WARN)) >> nb_fwd_ports = (uint8_t) n; >> else >> rte_exit(EXIT_FAILURE, >> - "nb-ports should be > 0 and <= %d\n", >> - nb_ports); >> + "Invalid port %d\n", n); >> } >> if (!strcmp(lgopts[opt_idx].name, "nb-cores")) { >> n = atoi(optarg); >> diff --git a/app/test-pmd/testpmd.c b/app/test-pmd/testpmd.c >> index 773b8af..c18c1a9 100644 >> --- a/app/test-pmd/testpmd.c >> +++ b/app/test-pmd/testpmd.c >> @@ -71,6 +71,7 @@ >> #include <rte_pci.h> >> #include <rte_ether.h> >> #include <rte_ethdev.h> >> +#include <rte_dev.h> >> #include <rte_string_fns.h> >> #ifdef RTE_LIBRTE_PMD_XENVIRT >> #include <rte_eth_xenvirt.h> >> @@ -315,7 +316,7 @@ uint16_t nb_rx_queue_stats_mappings = 0; >> >> /* Forward function declarations */ >> static void map_port_queue_stats_mapping_registers(uint8_t pi, struct rte_port *port); >> -static void check_all_ports_link_status(uint8_t port_num, uint32_t port_mask); >> +static void check_all_ports_link_status(uint32_t port_mask); >> >> /* >> * Check if all the ports are started. >> @@ -324,6 +325,20 @@ static void check_all_ports_link_status(uint8_t port_num, uint32_t port_mask); >> static int all_ports_started(void); >> >> /* >> + * Find next enabled port >> + */ >> +portid_t >> +find_next_port(portid_t p, struct rte_port *ports, int size) >> +{ >> + if (ports == NULL) >> + rte_exit(-EINVAL, "failed to find a next port id\n"); >> + >> + while ((ports[p].enabled == 0) && (p < size)) >> + p++; >> + return p; >> +} >> + >> +/* >> * Setup default configuration. >> */ >> static void >> @@ -552,7 +567,8 @@ init_config(void) >> + RTE_TEST_TX_DESC_MAX + MAX_PKT_BURST; >> >> if (!numa_support) >> - nb_mbuf_per_pool = (nb_mbuf_per_pool * nb_ports); >> + nb_mbuf_per_pool = >> + (nb_mbuf_per_pool * RTE_MAX_ETHPORTS); >> } >> >> if (!numa_support) { >> @@ -565,14 +581,19 @@ init_config(void) >> >> /* Configuration of Ethernet ports. */ >> ports = rte_zmalloc("testpmd: ports", >> - sizeof(struct rte_port) * nb_ports, >> + sizeof(struct rte_port) * RTE_MAX_ETHPORTS, >> RTE_CACHE_LINE_SIZE); >> if (ports == NULL) { >> - rte_exit(EXIT_FAILURE, "rte_zmalloc(%d struct rte_port) " >> - "failed\n", nb_ports); >> + rte_exit(EXIT_FAILURE, >> + "rte_zmalloc(%d struct rte_port) failed\n", >> + RTE_MAX_ETHPORTS); >> } >> >> - for (pid = 0; pid < nb_ports; pid++) { >> + /* enabled allocated ports */ >> + for (pid = 0; pid < nb_ports; pid++) >> + ports[pid].enabled = 1; >> + >> + FOREACH_PORT(pid, ports) { >> port = &ports[pid]; >> rte_eth_dev_info_get(pid, &port->dev_info); >> >> @@ -602,8 +623,7 @@ init_config(void) >> nb_mbuf_per_pool = nb_mbuf_per_pool/nb_ports; >> >> for (i = 0; i < MAX_SOCKET; i++) { >> - nb_mbuf = (nb_mbuf_per_pool * >> - port_per_socket[i]); >> + nb_mbuf = (nb_mbuf_per_pool * RTE_MAX_ETHPORTS); >> if (nb_mbuf) >> mbuf_pool_create(mbuf_data_size, >> nb_mbuf,i); >> @@ -635,14 +655,6 @@ reconfig(portid_t new_port_id, unsigned socket_id) >> struct rte_port *port; >> >> /* Reconfiguration of Ethernet ports. */ >> - ports = rte_realloc(ports, >> - sizeof(struct rte_port) * nb_ports, >> - RTE_CACHE_LINE_SIZE); >> - if (ports == NULL) { >> - rte_exit(EXIT_FAILURE, "rte_realloc(%d struct rte_port) failed\n", >> - nb_ports); >> - } >> - >> port = &ports[new_port_id]; >> rte_eth_dev_info_get(new_port_id, &port->dev_info); >> >> @@ -663,7 +675,7 @@ init_fwd_streams(void) >> streamid_t sm_id, nb_fwd_streams_new; >> >> /* set socket id according to numa or not */ >> - for (pid = 0; pid < nb_ports; pid++) { >> + FOREACH_PORT(pid, ports) { >> port = &ports[pid]; >> if (nb_rxq > port->dev_info.max_rx_queues) { >> printf("Fail: nb_rxq(%d) is greater than " >> @@ -1264,7 +1276,7 @@ all_ports_started(void) >> portid_t pi; >> struct rte_port *port; >> >> - for (pi = 0; pi < nb_ports; pi++) { >> + FOREACH_PORT(pi, ports) { >> port = &ports[pi]; >> /* Check if there is a port which is not started */ >> if (port->port_status != RTE_PORT_STARTED) >> @@ -1276,6 +1288,45 @@ all_ports_started(void) >> } >> >> int >> +all_ports_stopped(void) >> +{ >> + portid_t pi; >> + struct rte_port *port; >> + >> + FOREACH_PORT(pi, ports) { >> + port = &ports[pi]; >> + if (port->port_status != RTE_PORT_STOPPED) >> + return 0; >> + } >> + >> + return 1; >> +} >> + >> +int >> +port_is_started(portid_t port_id) >> +{ >> + if (port_id_is_invalid(port_id, ENABLED_WARN)) >> + return 0; >> + >> + if (ports[port_id].port_status != RTE_PORT_STARTED) >> + return 0; >> + >> + return 1; >> +} >> + >> +static int >> +port_is_closed(portid_t port_id) >> +{ >> + if (port_id_is_invalid(port_id, ENABLED_WARN)) >> + return 0; >> + >> + if (ports[port_id].port_status != RTE_PORT_CLOSED) >> + return 0; >> + >> + return 1; >> +} >> + >> +int >> start_port(portid_t pid) >> { >> int diag, need_check_link_status = 0; >> @@ -1296,8 +1347,8 @@ start_port(portid_t pid) >> >> if(dcb_config) >> dcb_test = 1; >> - for (pi = 0; pi < nb_ports; pi++) { >> - if (pid < nb_ports && pid != pi) >> + FOREACH_PORT(pi, ports) { >> + if (!port_id_is_invalid(pid, DISABLED_WARN) && pid != pi) >> continue; >> >> port = &ports[pi]; >> @@ -1421,7 +1472,7 @@ start_port(portid_t pid) >> } >> >> if (need_check_link_status && !no_link_check) >> - check_all_ports_link_status(nb_ports, RTE_PORT_ALL); >> + check_all_ports_link_status(RTE_PORT_ALL); >> else >> printf("Please stop the ports first\n"); >> >> @@ -1446,8 +1497,8 @@ stop_port(portid_t pid) >> } >> printf("Stopping ports...\n"); >> >> - for (pi = 0; pi < nb_ports; pi++) { >> - if (pid < nb_ports && pid != pi) >> + FOREACH_PORT(pi, ports) { >> + if (!port_id_is_invalid(pid, DISABLED_WARN) && pid != pi) >> continue; >> >> port = &ports[pi]; >> @@ -1463,7 +1514,7 @@ stop_port(portid_t pid) >> need_check_link_status = 1; >> } >> if (need_check_link_status && !no_link_check) >> - check_all_ports_link_status(nb_ports, RTE_PORT_ALL); >> + check_all_ports_link_status(RTE_PORT_ALL); >> >> printf("Done\n"); >> } >> @@ -1481,8 +1532,8 @@ close_port(portid_t pid) >> >> printf("Closing ports...\n"); >> >> - for (pi = 0; pi < nb_ports; pi++) { >> - if (pid < nb_ports && pid != pi) >> + FOREACH_PORT(pi, ports) { >> + if (!port_id_is_invalid(pid, DISABLED_WARN) && pid != pi) >> continue; >> >> port = &ports[pi]; >> @@ -1502,31 +1553,83 @@ close_port(portid_t pid) >> printf("Done\n"); >> } >> >> -int >> -all_ports_stopped(void) >> +void >> +attach_port(char *identifier) >> { >> - portid_t pi; >> - struct rte_port *port; >> + portid_t i, j, pi = 0; >> >> - for (pi = 0; pi < nb_ports; pi++) { >> - port = &ports[pi]; >> - if (port->port_status != RTE_PORT_STOPPED) >> - return 0; >> + printf("Attaching a new port...\n"); >> + >> + if (identifier == NULL) { >> + printf("Invalid parameters are speficied\n"); >> + return; >> } >> >> - return 1; >> + if (test_done == 0) { >> + printf("Please stop forwarding first\n"); >> + return; >> + } >> + >> + if (rte_eal_dev_attach(identifier, &pi)) >> + return; >> + >> + ports[pi].enabled = 1; >> + reconfig(pi, rte_eth_dev_socket_id(pi)); >> + rte_eth_promiscuous_enable(pi); >> + >> + nb_ports = rte_eth_dev_count(); >> + >> + /* set_default_fwd_ports_config(); */ >> + bzero(fwd_ports_ids, sizeof(fwd_ports_ids)); >> + i = 0; >> + FOREACH_PORT(j, ports) { >> + fwd_ports_ids[i] = j; >> + i++; >> + } >> + nb_cfg_ports = nb_ports; >> + nb_fwd_ports++; >> + >> + ports[pi].port_status = RTE_PORT_STOPPED; >> + >> + printf("Port %d is attached. Now total ports is %d\n", pi, nb_ports); >> + printf("Done\n"); >> } >> >> -int >> -port_is_started(portid_t port_id) >> +void >> +detach_port(uint8_t port_id) >> { >> - if (port_id_is_invalid(port_id)) >> - return -1; >> + portid_t i, pi = 0; >> + char name[RTE_ETH_NAME_MAX_LEN]; >> >> - if (ports[port_id].port_status != RTE_PORT_STARTED) >> - return 0; >> + printf("Detaching a port...\n"); >> >> - return 1; >> + if (!port_is_closed(port_id)) { >> + printf("Please close port first\n"); >> + return; >> + } >> + >> + rte_eth_promiscuous_disable(port_id); >> + >> + if (rte_eal_dev_detach(port_id, name)) >> + return; >> + >> + ports[port_id].enabled = 0; >> + nb_ports = rte_eth_dev_count(); >> + >> + /* set_default_fwd_ports_config(); */ >> + bzero(fwd_ports_ids, sizeof(fwd_ports_ids)); >> + i = 0; >> + FOREACH_PORT(pi, ports) { >> + fwd_ports_ids[i] = pi; >> + i++; >> + } >> + nb_cfg_ports = nb_ports; >> + nb_fwd_ports--; >> + >> + printf("Port '%s' is detached. Now total ports is %d\n", >> + name, nb_ports); >> + printf("Done\n"); >> + return; >> } >> >> void >> @@ -1534,7 +1637,7 @@ pmd_test_exit(void) >> { >> portid_t pt_id; >> >> - for (pt_id = 0; pt_id < nb_ports; pt_id++) { >> + FOREACH_PORT(pt_id, ports) { >> printf("Stopping port %d...", pt_id); >> fflush(stdout); >> rte_eth_dev_close(pt_id); >> @@ -1553,7 +1656,7 @@ struct pmd_test_command { >> >> /* Check the link status of all ports in up to 9s, and print them finally */ >> static void >> -check_all_ports_link_status(uint8_t port_num, uint32_t port_mask) >> +check_all_ports_link_status(uint32_t port_mask) >> { >> #define CHECK_INTERVAL 100 /* 100ms */ >> #define MAX_CHECK_TIME 90 /* 9s (90 * 100ms) in total */ >> @@ -1564,7 +1667,7 @@ check_all_ports_link_status(uint8_t port_num, uint32_t port_mask) >> fflush(stdout); >> for (count = 0; count <= MAX_CHECK_TIME; count++) { >> all_ports_up = 1; >> - for (portid = 0; portid < port_num; portid++) { >> + FOREACH_PORT(portid, ports) { >> if ((port_mask & (1 << portid)) == 0) >> continue; >> memset(&link, 0, sizeof(link)); >> @@ -1688,7 +1791,7 @@ init_port_config(void) >> portid_t pid; >> struct rte_port *port; >> >> - for (pid = 0; pid < nb_ports; pid++) { >> + FOREACH_PORT(pid, ports) { >> port = &ports[pid]; >> port->dev_conf.rxmode = rx_mode; >> port->dev_conf.fdir_conf = fdir_conf; >> @@ -1877,7 +1980,7 @@ main(int argc, char** argv) >> >> nb_ports = (portid_t) rte_eth_dev_count(); >> if (nb_ports == 0) >> - rte_exit(EXIT_FAILURE, "No probed ethernet device\n"); >> + RTE_LOG(WARNING, EAL, "No probed ethernet devices\n"); >> >> set_def_fwd_config(); >> if (nb_lcores == 0) >> @@ -1899,7 +2002,7 @@ main(int argc, char** argv) >> rte_exit(EXIT_FAILURE, "Start ports failed\n"); >> >> /* set all ports to promiscuous mode by default */ >> - for (port_id = 0; port_id < nb_ports; port_id++) >> + FOREACH_PORT(port_id, ports) >> rte_eth_promiscuous_enable(port_id); >> >> #ifdef RTE_LIBRTE_CMDLINE >> diff --git a/app/test-pmd/testpmd.h b/app/test-pmd/testpmd.h >> index 8f5e6c7..109c670 100644 >> --- a/app/test-pmd/testpmd.h >> +++ b/app/test-pmd/testpmd.h >> @@ -134,6 +134,7 @@ struct fwd_stream { >> * The data structure associated with each port. >> */ >> struct rte_port { >> + uint8_t enabled; /**< Port enabled or not */ >> struct rte_eth_dev_info dev_info; /**< PCI info + driver name */ >> struct rte_eth_conf dev_conf; /**< Port configuration. */ >> struct ether_addr eth_addr; /**< Port ethernet address */ >> @@ -159,6 +160,14 @@ struct rte_port { >> struct rte_eth_txconf tx_conf; /**< tx configuration */ >> }; >> >> +extern portid_t __rte_unused >> +find_next_port(portid_t p, struct rte_port *ports, int size); >> + >> +#define FOREACH_PORT(p, ports) \ >> + for (p = find_next_port(0, ports, RTE_MAX_ETHPORTS); \ >> + p < RTE_MAX_ETHPORTS; \ >> + p = find_next_port(p + 1, ports, RTE_MAX_ETHPORTS)) >> + >> /** >> * The data structure associated with each forwarding logical core. >> * The logical cores are internally numbered by a core index from 0 to >> @@ -515,6 +524,8 @@ int init_port_dcb_config(portid_t pid,struct dcb_config *dcb_conf); >> int start_port(portid_t pid); >> void stop_port(portid_t pid); >> void close_port(portid_t pid); >> +void attach_port(char *identifier); >> +void detach_port(uint8_t port_id); >> int all_ports_stopped(void); >> int port_is_started(portid_t port_id); >> void pmd_test_exit(void); >> @@ -558,10 +569,15 @@ void get_ethertype_filter(uint8_t port_id, uint16_t index); >> void get_2tuple_filter(uint8_t port_id, uint16_t index); >> void get_5tuple_filter(uint8_t port_id, uint16_t index); >> void get_flex_filter(uint8_t port_id, uint16_t index); >> -int port_id_is_invalid(portid_t port_id); >> int rx_queue_id_is_invalid(queueid_t rxq_id); >> int tx_queue_id_is_invalid(queueid_t txq_id); >> >> +enum print_warning { >> + ENABLED_WARN = 0, >> + DISABLED_WARN >> +}; >> +int port_id_is_invalid(portid_t port_id, enum print_warning warning); >> + >> /* >> * Work-around of a compilation error with ICC on invocations of the >> * rte_be_to_cpu_16() function. >> diff --git a/doc/guides/testpmd_app_ug/testpmd_funcs.rst b/doc/guides/testpmd_app_ug/testpmd_funcs.rst >> index 218835a..1cacbcf 100644 >> --- a/doc/guides/testpmd_app_ug/testpmd_funcs.rst >> +++ b/doc/guides/testpmd_app_ug/testpmd_funcs.rst >> @@ -808,6 +808,63 @@ The following sections show functions for configuring ports. >> >> Port configuration changes only become active when forwarding is started/restarted. >> >> +port attach >> +~~~~~~~~~~~ >> + >> +Attach a port specified by pci address or virtual device args. >> + >> +To attach a new pci device, the device should be recognized by kernel first. >> +Then it should be moved under DPDK management. >> +Finally the port can be attached to testpmd. >> +On the other hand, to attach a port created by virtual device, above steps are not needed. >> + >> +port attach (identifier) >> + >> +For example, to attach a port that pci address is 0000:02:00.0. >> + >> +.. code-block:: console >> + >> + testpmd> port attach 0000:02:00.0 >> + Attaching a new port... >> + ... snip ... >> + Port 0 is attached. Now total ports is 1 >> + Done >> + >> +For example, to attach a port created by pcap PMD. >> + >> +.. code-block:: console >> + >> + testpmd> port attach eth_pcap0,iface=eth0 >> + Attaching a new port... >> + ... snip ... >> + Port 0 is attached. Now total ports is 1 >> + Done >> + >> +In this case, identifier is "eth_pcap0,iface=eth0". >> +This identifier format is the same as "--vdev" format of DPDK applications. >> + >> +port detach >> +~~~~~~~~~~~ >> + >> +Detach a specific port. >> + >> +Before detaching a port, the port should be closed. >> +Also to remove a pci device completely from the system, first detach the port from testpmd. >> +Then the device should be moved under kernel management. >> +Finally the device can be remove using kernel pci hotplug functionality. >> +On the other hand, to remove a port created by virtual device, above steps are not needed. >> + >> +port detach (port_id) >> + >> +For example, to detach a port 0. >> + >> +.. code-block:: console >> + >> + testpmd> port detach 0 >> + Detaching a port... >> + ... snip ... >> + Done >> + >> port start >> ~~~~~~~~~~ >>
On 2015/02/03 18:14, Qiu, Michael wrote: > On 2/3/2015 2:16 PM, Qiu, Michael wrote: >> On 2/1/2015 12:02 PM, Tetsuya Mukawa wrote: >>> The patch introduces following commands. >>> - port attach [ident] >>> - port detach [port_id] >>> - attach: attaching a port >>> - detach: detaching a port >>> - ident: pci address of physical device. >>> Or device name and paramerters of virtual device. >>> (ex. 0000:02:00.0, eth_pcap0,iface=eth0) >>> - port_id: port identifier >>> >>> v5: >>> - Add testpmd documentation. >>> (Thanks to Iremonger, Bernard) >>> v4: >>> - Fix strings of command help. >>> >>> Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp> > [...] > >>> +static int >>> +port_is_closed(portid_t port_id) >>> +{ >>> + if (port_id_is_invalid(port_id, ENABLED_WARN)) >>> + return 0; >>> + >>> + if (ports[port_id].port_status != RTE_PORT_CLOSED) >>> + return 0; >>> + >>> + return 1; >>> +} >>> + >>> +int >>> start_port(portid_t pid) >>> { >>> int diag, need_check_link_status = 0; >>> @@ -1296,8 +1347,8 @@ start_port(portid_t pid) >>> >>> if(dcb_config) >>> dcb_test = 1; >>> - for (pi = 0; pi < nb_ports; pi++) { >>> - if (pid < nb_ports && pid != pi) >>> + FOREACH_PORT(pi, ports) { >>> + if (!port_id_is_invalid(pid, DISABLED_WARN) && pid != pi) >> Here may it be: >> >> if (!port_id_is_invalid(pid, DISABLED_WARN) && (pid != pi || pid == RET_PORT_ALL)) > Sorry, should be: > > if (!port_id_is_invalid(pid, DISABLED_WARN) && pid != pi && pid != (portid_t)RET_PORT_ALL) > > Otherwise, should check for "RET_PORT_ALL" in function port_id_is_invalid() Thanks for comment. I've found 2 issues. (I guess the original code has same issue.) One is that "port_id_is_invalid" should receives "pi" instead of "pid". The other is if statement is wrong as you said. I guess following statement will be good. if (port_id_is_invalid(pi, DISABLED_WARN) || (pid != pi && pid != (portid_t)RTE_PORT_ALL)) How about it? Thanks, Tetsuya > Thanks, > Michael > >> Otherwise no port will be start by default. >> >> >> Thanks, >> Michael >> >>> continue; >>> >>> port = &ports[pi]; >>> @@ -1421,7 +1472,7 @@ start_port(portid_t pid) >>> } >>> >>> if (need_check_link_status && !no_link_check) >>> - check_all_ports_link_status(nb_ports, RTE_PORT_ALL); >>> + check_all_ports_link_status(RTE_PORT_ALL); >>> else >>> printf("Please stop the ports first\n"); >>> >>> @@ -1446,8 +1497,8 @@ stop_port(portid_t pid) >>> } >>> printf("Stopping ports...\n"); >>> >>> - for (pi = 0; pi < nb_ports; pi++) { >>> - if (pid < nb_ports && pid != pi) >>> + FOREACH_PORT(pi, ports) { >>> + if (!port_id_is_invalid(pid, DISABLED_WARN) && pid != pi) >>> continue; >>> >>> port = &ports[pi]; >>> @@ -1463,7 +1514,7 @@ stop_port(portid_t pid) >>> need_check_link_status = 1; >>> } >>> if (need_check_link_status && !no_link_check) >>> - check_all_ports_link_status(nb_ports, RTE_PORT_ALL); >>> + check_all_ports_link_status(RTE_PORT_ALL); >>> >>> printf("Done\n"); >>> } >>> @@ -1481,8 +1532,8 @@ close_port(portid_t pid) >>> >>> printf("Closing ports...\n"); >>> >>> - for (pi = 0; pi < nb_ports; pi++) { >>> - if (pid < nb_ports && pid != pi) >>> + FOREACH_PORT(pi, ports) { >>> + if (!port_id_is_invalid(pid, DISABLED_WARN) && pid != pi) >>> continue; >>> >>> port = &ports[pi]; >>> @@ -1502,31 +1553,83 @@ close_port(portid_t pid) >>> printf("Done\n"); >>> } >>> >>> -int >>> -all_ports_stopped(void) >>> +void >>> +attach_port(char *identifier) >>> { >>> - portid_t pi; >>> - struct rte_port *port; >>> + portid_t i, j, pi = 0; >>> >>> - for (pi = 0; pi < nb_ports; pi++) { >>> - port = &ports[pi]; >>> - if (port->port_status != RTE_PORT_STOPPED) >>> - return 0; >>> + printf("Attaching a new port...\n"); >>> + >>> + if (identifier == NULL) { >>> + printf("Invalid parameters are speficied\n"); >>> + return; >>> } >>> >>> - return 1; >>> + if (test_done == 0) { >>> + printf("Please stop forwarding first\n"); >>> + return; >>> + } >>> + >>> + if (rte_eal_dev_attach(identifier, &pi)) >>> + return; >>> + >>> + ports[pi].enabled = 1; >>> + reconfig(pi, rte_eth_dev_socket_id(pi)); >>> + rte_eth_promiscuous_enable(pi); >>> + >>> + nb_ports = rte_eth_dev_count(); >>> + >>> + /* set_default_fwd_ports_config(); */ >>> + bzero(fwd_ports_ids, sizeof(fwd_ports_ids)); >>> + i = 0; >>> + FOREACH_PORT(j, ports) { >>> + fwd_ports_ids[i] = j; >>> + i++; >>> + } >>> + nb_cfg_ports = nb_ports; >>> + nb_fwd_ports++; >>> + >>> + ports[pi].port_status = RTE_PORT_STOPPED; >>> + >>> + printf("Port %d is attached. Now total ports is %d\n", pi, nb_ports); >>> + printf("Done\n"); >>> } >>> >>> -int >>> -port_is_started(portid_t port_id) >>> +void >>> +detach_port(uint8_t port_id) >>> { >>> - if (port_id_is_invalid(port_id)) >>> - return -1; >>> + portid_t i, pi = 0; >>> + char name[RTE_ETH_NAME_MAX_LEN]; >>> >>> - if (ports[port_id].port_status != RTE_PORT_STARTED) >>> - return 0; >>> + printf("Detaching a port...\n"); >>> >>> - return 1; >>> + if (!port_is_closed(port_id)) { >>> + printf("Please close port first\n"); >>> + return; >>> + } >>> + >>> + rte_eth_promiscuous_disable(port_id); >>> + >>> + if (rte_eal_dev_detach(port_id, name)) >>> + return; >>> + >>> + ports[port_id].enabled = 0; >>> + nb_ports = rte_eth_dev_count(); >>> + >>> + /* set_default_fwd_ports_config(); */ >>> + bzero(fwd_ports_ids, sizeof(fwd_ports_ids)); >>> + i = 0; >>> + FOREACH_PORT(pi, ports) { >>> + fwd_ports_ids[i] = pi; >>> + i++; >>> + } >>> + nb_cfg_ports = nb_ports; >>> + nb_fwd_ports--; >>> + >>> + printf("Port '%s' is detached. Now total ports is %d\n", >>> + name, nb_ports); >>> + printf("Done\n"); >>> + return; >>> } >>> >>> void >>> @@ -1534,7 +1637,7 @@ pmd_test_exit(void) >>> { >>> portid_t pt_id; >>> >>> - for (pt_id = 0; pt_id < nb_ports; pt_id++) { >>> + FOREACH_PORT(pt_id, ports) { >>> printf("Stopping port %d...", pt_id); >>> fflush(stdout); >>> rte_eth_dev_close(pt_id); >>> @@ -1553,7 +1656,7 @@ struct pmd_test_command { >>> >>> /* Check the link status of all ports in up to 9s, and print them finally */ >>> static void >>> -check_all_ports_link_status(uint8_t port_num, uint32_t port_mask) >>> +check_all_ports_link_status(uint32_t port_mask) >>> { >>> #define CHECK_INTERVAL 100 /* 100ms */ >>> #define MAX_CHECK_TIME 90 /* 9s (90 * 100ms) in total */ >>> @@ -1564,7 +1667,7 @@ check_all_ports_link_status(uint8_t port_num, uint32_t port_mask) >>> fflush(stdout); >>> for (count = 0; count <= MAX_CHECK_TIME; count++) { >>> all_ports_up = 1; >>> - for (portid = 0; portid < port_num; portid++) { >>> + FOREACH_PORT(portid, ports) { >>> if ((port_mask & (1 << portid)) == 0) >>> continue; >>> memset(&link, 0, sizeof(link)); >>> @@ -1688,7 +1791,7 @@ init_port_config(void) >>> portid_t pid; >>> struct rte_port *port; >>> >>> - for (pid = 0; pid < nb_ports; pid++) { >>> + FOREACH_PORT(pid, ports) { >>> port = &ports[pid]; >>> port->dev_conf.rxmode = rx_mode; >>> port->dev_conf.fdir_conf = fdir_conf; >>> @@ -1877,7 +1980,7 @@ main(int argc, char** argv) >>> >>> nb_ports = (portid_t) rte_eth_dev_count(); >>> if (nb_ports == 0) >>> - rte_exit(EXIT_FAILURE, "No probed ethernet device\n"); >>> + RTE_LOG(WARNING, EAL, "No probed ethernet devices\n"); >>> >>> set_def_fwd_config(); >>> if (nb_lcores == 0) >>> @@ -1899,7 +2002,7 @@ main(int argc, char** argv) >>> rte_exit(EXIT_FAILURE, "Start ports failed\n"); >>> >>> /* set all ports to promiscuous mode by default */ >>> - for (port_id = 0; port_id < nb_ports; port_id++) >>> + FOREACH_PORT(port_id, ports) >>> rte_eth_promiscuous_enable(port_id); >>> >>> #ifdef RTE_LIBRTE_CMDLINE >>> diff --git a/app/test-pmd/testpmd.h b/app/test-pmd/testpmd.h >>> index 8f5e6c7..109c670 100644 >>> --- a/app/test-pmd/testpmd.h >>> +++ b/app/test-pmd/testpmd.h >>> @@ -134,6 +134,7 @@ struct fwd_stream { >>> * The data structure associated with each port. >>> */ >>> struct rte_port { >>> + uint8_t enabled; /**< Port enabled or not */ >>> struct rte_eth_dev_info dev_info; /**< PCI info + driver name */ >>> struct rte_eth_conf dev_conf; /**< Port configuration. */ >>> struct ether_addr eth_addr; /**< Port ethernet address */ >>> @@ -159,6 +160,14 @@ struct rte_port { >>> struct rte_eth_txconf tx_conf; /**< tx configuration */ >>> }; >>> >>> +extern portid_t __rte_unused >>> +find_next_port(portid_t p, struct rte_port *ports, int size); >>> + >>> +#define FOREACH_PORT(p, ports) \ >>> + for (p = find_next_port(0, ports, RTE_MAX_ETHPORTS); \ >>> + p < RTE_MAX_ETHPORTS; \ >>> + p = find_next_port(p + 1, ports, RTE_MAX_ETHPORTS)) >>> + >>> /** >>> * The data structure associated with each forwarding logical core. >>> * The logical cores are internally numbered by a core index from 0 to >>> @@ -515,6 +524,8 @@ int init_port_dcb_config(portid_t pid,struct dcb_config *dcb_conf); >>> int start_port(portid_t pid); >>> void stop_port(portid_t pid); >>> void close_port(portid_t pid); >>> +void attach_port(char *identifier); >>> +void detach_port(uint8_t port_id); >>> int all_ports_stopped(void); >>> int port_is_started(portid_t port_id); >>> void pmd_test_exit(void); >>> @@ -558,10 +569,15 @@ void get_ethertype_filter(uint8_t port_id, uint16_t index); >>> void get_2tuple_filter(uint8_t port_id, uint16_t index); >>> void get_5tuple_filter(uint8_t port_id, uint16_t index); >>> void get_flex_filter(uint8_t port_id, uint16_t index); >>> -int port_id_is_invalid(portid_t port_id); >>> int rx_queue_id_is_invalid(queueid_t rxq_id); >>> int tx_queue_id_is_invalid(queueid_t txq_id); >>> >>> +enum print_warning { >>> + ENABLED_WARN = 0, >>> + DISABLED_WARN >>> +}; >>> +int port_id_is_invalid(portid_t port_id, enum print_warning warning); >>> + >>> /* >>> * Work-around of a compilation error with ICC on invocations of the >>> * rte_be_to_cpu_16() function. >>> diff --git a/doc/guides/testpmd_app_ug/testpmd_funcs.rst b/doc/guides/testpmd_app_ug/testpmd_funcs.rst >>> index 218835a..1cacbcf 100644 >>> --- a/doc/guides/testpmd_app_ug/testpmd_funcs.rst >>> +++ b/doc/guides/testpmd_app_ug/testpmd_funcs.rst >>> @@ -808,6 +808,63 @@ The following sections show functions for configuring ports. >>> >>> Port configuration changes only become active when forwarding is started/restarted. >>> >>> +port attach >>> +~~~~~~~~~~~ >>> + >>> +Attach a port specified by pci address or virtual device args. >>> + >>> +To attach a new pci device, the device should be recognized by kernel first. >>> +Then it should be moved under DPDK management. >>> +Finally the port can be attached to testpmd. >>> +On the other hand, to attach a port created by virtual device, above steps are not needed. >>> + >>> +port attach (identifier) >>> + >>> +For example, to attach a port that pci address is 0000:02:00.0. >>> + >>> +.. code-block:: console >>> + >>> + testpmd> port attach 0000:02:00.0 >>> + Attaching a new port... >>> + ... snip ... >>> + Port 0 is attached. Now total ports is 1 >>> + Done >>> + >>> +For example, to attach a port created by pcap PMD. >>> + >>> +.. code-block:: console >>> + >>> + testpmd> port attach eth_pcap0,iface=eth0 >>> + Attaching a new port... >>> + ... snip ... >>> + Port 0 is attached. Now total ports is 1 >>> + Done >>> + >>> +In this case, identifier is "eth_pcap0,iface=eth0". >>> +This identifier format is the same as "--vdev" format of DPDK applications. >>> + >>> +port detach >>> +~~~~~~~~~~~ >>> + >>> +Detach a specific port. >>> + >>> +Before detaching a port, the port should be closed. >>> +Also to remove a pci device completely from the system, first detach the port from testpmd. >>> +Then the device should be moved under kernel management. >>> +Finally the device can be remove using kernel pci hotplug functionality. >>> +On the other hand, to remove a port created by virtual device, above steps are not needed. >>> + >>> +port detach (port_id) >>> + >>> +For example, to detach a port 0. >>> + >>> +.. code-block:: console >>> + >>> + testpmd> port detach 0 >>> + Detaching a port... >>> + ... snip ... >>> + Done >>> + >>> port start >>> ~~~~~~~~~~ >>>
Hi Bernard,
I appreciate your checking.
I will fix like below.
Tetsuya
On 2015/02/03 19:03, Iremonger, Bernard wrote:
>>>> +.. code-block:: console
>>>> +
>>>> + testpmd> port attach 0000:02:00.0
>>>> + Attaching a new port...
>>>> + ... snip ...
>>>> + Port 0 is attached. Now total ports is 1
>>>> + Done
>>>> +port detach
>>>> +~~~~~~~~~~~
>>>> +
>>>> +Detach a specific port.
>>>> +
>>>> +Before detaching a port, the port should be closed.
>>>> +Also to remove a pci device completely from the system, first detach the port from testpmd.
>>>> +Then the device should be moved under kernel management.
>>>> +Finally the device can be remove using kernel pci hotplug functionality.
> Hi Tetsuya,
> Reword "remove" to "removed"
>
>>>> +On the other hand, to remove a port created by virtual device, above steps are not needed.
> Reword " created by virtual device" to "created by a virtual device"
>
>>>> +
>>>> +port detach (port_id)
>>>> +
>>>> +For example, to detach a port 0.
>>>> +
>>>> +.. code-block:: console
>>>> +
>>>> + testpmd> port detach 0
>>>> + Detaching a port...
>>>> + ... snip ...
>>>> + Done
>>>> +
>>>> port start
>>>> ~~~~~~~~~~
>>>>
>>>> --
>>>> 1.9.1
>>> Regards,
>>>
>>> Bernard.
>>>
> -----Original Message-----
> From: Tetsuya Mukawa [mailto:mukawa@igel.co.jp]
> Sent: Sunday, February 1, 2015 4:02 AM
> To: dev@dpdk.org
> Cc: Qiu, Michael; Iremonger, Bernard; Tetsuya Mukawa
> Subject: [PATCH v6 00/13] Port Hotplug Framework
>
> This patch series adds a dynamic port hotplug framework to DPDK.
> With the patches, DPDK apps can attach or detach ports at runtime.
>
> The basic concept of the port hotplug is like followings.
> - DPDK apps must have responsibility to manage ports.
> DPDK apps only know which ports are attached or detached at the moment.
> The port hotplug framework is implemented to allow DPDK apps to manage ports.
> For example, when DPDK apps call port attach function, attached port number
> will be returned. Also DPDK apps can detach port by port number.
> - Kernel support is needed for attaching or detaching physical device ports.
> To attach a new physical device port, the device will be recognized by
> userspace directly I/O framework in kernel at first. Then DPDK apps can
> call the port hotplug functions to attach ports.
> For detaching, steps are vice versa.
> - Before detach ports, ports must be stopped and closed.
> DPDK application must call rte_eth_dev_stop() and rte_eth_dev_close() before
> detaching ports. These function will call finalization codes of PMDs.
> But so far, no PMD frees all resources allocated by initialization.
> It means PMDs are needed to be fixed to support the port hotplug.
> 'RTE_PCI_DRV_DETACHABLE' is a new flag indicating a PMD supports detaching.
> Without this flag, detaching will be failed.
> - Mustn't affect legacy DPDK apps.
> No DPDK EAL behavior is changed, if the port hotplug functions are't called.
> So all legacy DPDK apps can still work without modifications.
>
> And a few limitations.
> - The port hotplug functions are not thread safe.
> DPDK apps should handle it.
> - Only support Linux and igb_uio so far.
> BSD and VFIO is not supported. I will send VFIO patches at least, but I don't
> have a plan to submit BSD patch so far.
>
>
> Here is port hotplug APIs.
> -------------------------------------------------------------------------------
> /**
> * Attach a new device.
> *
> * @param devargs
> * A pointer to a strings array describing the new device
> * to be attached. The strings should be a pci address like
> * '0000:01:00.0' or virtual device name like 'eth_pcap0'.
> * @param port_id
> * A pointer to a port identifier actually attached.
> * @return
> * 0 on success and port_id is filled, negative on error */ int rte_eal_dev_attach(const char *devargs,
> uint8_t *port_id);
>
> /**
> * Detach a device.
> *
> * @param port_id
> * The port identifier of the device to detach.
> * @param addr
> * A pointer to a device name actually detached.
> * @return
> * 0 on success and devname is filled, negative on error */ int rte_eal_dev_detach(uint8_t port_id,
> char *devname);
> -------------------------------------------------------------------------------
>
> This patch series are for DPDK EAL. To use port hotplug function by DPDK apps, each PMD should be
> fixed to support 'RTE_PCI_DRV_DETACHABLE' flag. Please check a patch for pcap PMD.
>
> Also please check testpmd patch. It will show you how to fix your legacy applications to support port
> hotplug feature.
>
> PATCH v6 changes
> - Fix rte_eth_dev_uninit() to handle a return value of uninit
> function of PMD. To do this, below changes also be applied.
> - Fix a paramter of rte_eth_dev_free().
> - Use rte_eth_dev structure as the paramter of rte_eth_dev_free().
>
> PATCH v5 changes
> - Add runtime check passthrough driver type, like vfio-pci, igb_uio
> and uio_pci_generic.
> This was done by Qiu, Michael. Thanks a lot.
> - Change function names like below.
> - rte_eal_dev_find_and_invoke() to rte_eal_vdev_find_and_invoke().
> - rte_eal_dev_invoke() to rte_eal_vdev_invoke().
> - Add code to handle a return value of rte_eal_devargs_remove().
> - Fix pci address format in rte_eal_dev_detach().
> - Remove RTE_EAL_INVOKE_TYPE_UNKNOWN, because it's unused.
> - Change function definition of rte_eal_devargs_remove().
> - Fix pci_unmap_device() to check pt_driver.
> - Fix return value of below functions.
> - rte_eth_dev_get_changed_port().
> - rte_eth_dev_get_port_by_addr().
> - Change paramters of rte_eth_dev_validate_port() to cleanup code.
> - Fix pci_scan_one to handle pt_driver correctly.
> (Thanks to Qiu, Michael for above sugesstions)
>
> PATCH v4 changes
> - Merge patches to review easier.
> - Fix indent of 'if' statement.
> - Fix calculation method of eal_compare_pci_addr().
> - Fix header file declaration.
> - Add header file to determine if hotplug can be enabled.
> (Thanks to Qiu, Michael)
> - Use braces with 'for' loop.
> - Add paramerter checking.
> - Fix sanity check code
> - Fix comments of rte_eth_dev_type.
> - Change function names.
> (Thanks to Iremonger, Bernard)
>
> PATCH v3 changes:
> - Fix enum definition used in rte_ethdev.c.
> (Thanks to Zhang, Helin)
>
> PATCH v2 changes:
> - Replace rte_eal_dev_attach_pdev(), rte_eal_dev_detach_pdev,
> rte_eal_dev_attach_vdev() and rte_eal_dev_detach_vdev() to
> rte_eal_dev_attach() and rte_eal_dev_detach().
> - Add parameter values checking.
> - Refashion a few functions.
> (Thanks to Iremonger, Bernard)
>
> PATCH v1 Changes:
> - Fix error checking code of librte_eth APIs.
> - Fix issue that port from pcap PMD cannot be detached correctly.
> - Fix issue that testpmd could hang after forwarding, if attaching and detaching
> is repeatedly.
> - Fix if-condition of rte_eth_dev_get_port_by_addr().
> (Thanks to Mark Enright)
>
> RFC PATCH v2 Changes:
> - remove 'rte_eth_dev_validate_port()', and cleanup codes.
>
>
> Michael Qiu (2):
> eal_pci: Add flag to hold kernel driver type
> eal_pci: pci memory map work with driver type
>
> Tetsuya Mukawa (11):
> eal/pci,ethdev: Remove assumption that port will not be detached
> eal/pci: Consolidate pci address comparison APIs
> ethdev: Add rte_eth_dev_free to free specified device
> eal,ethdev: Add a function and function pointers to close ether device
> ethdev: Add functions that will be used by port hotplug functions
> eal/linux/pci: Add functions for unmapping igb_uio resources
> eal/pci: Add a function to remove the entry of devargs list
> eal/pci: Cleanup pci driver initialization code
> ethdev: Add one dev_type paramerter to rte_eth_dev_allocate
> eal/pci: Add rte_eal_dev_attach/detach() functions
> eal: Enable port hotplug framework in Linux
>
> app/test/virtual_pmd.c | 2 +-
> config/common_linuxapp | 5 +
> lib/librte_eal/bsdapp/eal/eal_pci.c | 25 +-
> lib/librte_eal/common/Makefile | 1 +
> lib/librte_eal/common/eal_common_dev.c | 274 +++++++++++
> lib/librte_eal/common/eal_common_devargs.c | 60 +++
> lib/librte_eal/common/eal_common_pci.c | 92 +++-
> lib/librte_eal/common/eal_private.h | 35 ++
> lib/librte_eal/common/include/rte_dev.h | 33 ++
> lib/librte_eal/common/include/rte_dev_hotplug.h | 44 ++
> lib/librte_eal/common/include/rte_devargs.h | 21 +
> lib/librte_eal/common/include/rte_pci.h | 84 ++++
> lib/librte_eal/linuxapp/eal/Makefile | 1 +
> lib/librte_eal/linuxapp/eal/eal_pci.c | 227 +++++++--
> lib/librte_eal/linuxapp/eal/eal_pci_init.h | 8 +
> lib/librte_eal/linuxapp/eal/eal_pci_uio.c | 67 ++-
> lib/librte_ether/rte_ethdev.c | 624 +++++++++++++-----------
> lib/librte_ether/rte_ethdev.h | 148 +++++-
> lib/librte_pmd_af_packet/rte_eth_af_packet.c | 2 +-
> lib/librte_pmd_bond/rte_eth_bond_api.c | 2 +-
> lib/librte_pmd_pcap/rte_eth_pcap.c | 2 +-
> lib/librte_pmd_ring/rte_eth_ring.c | 2 +-
> lib/librte_pmd_xenvirt/rte_eth_xenvirt.c | 2 +-
> 23 files changed, 1414 insertions(+), 347 deletions(-) create mode 100644
> lib/librte_eal/common/include/rte_dev_hotplug.h
>
> --
> 1.9.1
Hi Tetsuya,
I have a few comments as follows:
Lib/librte_ether/rte_ethdev.c
Line 400 : NO_TRACE would be clearer than NONE_TRACE.
Line 440: function rte_eth_dev_save() should pass in a size parameter which should be used in the memcpy function.
Lib/librte_eal/common/include/ret_dev_hotplug.h
Is this file really necessary as it contains only one macro?
Could it be merged into Lib/librte_eal/common/include/ret_dev.h ?
Regards,
Bernard.
> -----Original Message-----
> From: Tetsuya Mukawa [mailto:mukawa@igel.co.jp]
> Sent: Sunday, February 1, 2015 4:02 AM
> To: dev@dpdk.org
> Cc: Qiu, Michael; Iremonger, Bernard; Tetsuya Mukawa
> Subject: [PATCH v6 00/13] Port Hotplug Framework
>
> This patch series adds a dynamic port hotplug framework to DPDK.
> With the patches, DPDK apps can attach or detach ports at runtime.
>
> The basic concept of the port hotplug is like followings.
> - DPDK apps must have responsibility to manage ports.
> DPDK apps only know which ports are attached or detached at the moment.
> The port hotplug framework is implemented to allow DPDK apps to manage ports.
> For example, when DPDK apps call port attach function, attached port number
> will be returned. Also DPDK apps can detach port by port number.
> - Kernel support is needed for attaching or detaching physical device ports.
> To attach a new physical device port, the device will be recognized by
> userspace directly I/O framework in kernel at first. Then DPDK apps can
> call the port hotplug functions to attach ports.
> For detaching, steps are vice versa.
> - Before detach ports, ports must be stopped and closed.
> DPDK application must call rte_eth_dev_stop() and rte_eth_dev_close() before
> detaching ports. These function will call finalization codes of PMDs.
> But so far, no PMD frees all resources allocated by initialization.
> It means PMDs are needed to be fixed to support the port hotplug.
> 'RTE_PCI_DRV_DETACHABLE' is a new flag indicating a PMD supports detaching.
> Without this flag, detaching will be failed.
> - Mustn't affect legacy DPDK apps.
> No DPDK EAL behavior is changed, if the port hotplug functions are't called.
> So all legacy DPDK apps can still work without modifications.
>
> And a few limitations.
> - The port hotplug functions are not thread safe.
> DPDK apps should handle it.
> - Only support Linux and igb_uio so far.
> BSD and VFIO is not supported. I will send VFIO patches at least, but I don't
> have a plan to submit BSD patch so far.
>
>
> Here is port hotplug APIs.
> -------------------------------------------------------------------------------
> /**
> * Attach a new device.
> *
> * @param devargs
> * A pointer to a strings array describing the new device
> * to be attached. The strings should be a pci address like
> * '0000:01:00.0' or virtual device name like 'eth_pcap0'.
> * @param port_id
> * A pointer to a port identifier actually attached.
> * @return
> * 0 on success and port_id is filled, negative on error */ int rte_eal_dev_attach(const char *devargs,
> uint8_t *port_id);
>
> /**
> * Detach a device.
> *
> * @param port_id
> * The port identifier of the device to detach.
> * @param addr
> * A pointer to a device name actually detached.
> * @return
> * 0 on success and devname is filled, negative on error */ int rte_eal_dev_detach(uint8_t port_id,
> char *devname);
> -------------------------------------------------------------------------------
>
> This patch series are for DPDK EAL. To use port hotplug function by DPDK apps, each PMD should be
> fixed to support 'RTE_PCI_DRV_DETACHABLE' flag. Please check a patch for pcap PMD.
>
> Also please check testpmd patch. It will show you how to fix your legacy applications to support port
> hotplug feature.
>
> PATCH v6 changes
> - Fix rte_eth_dev_uninit() to handle a return value of uninit
> function of PMD. To do this, below changes also be applied.
> - Fix a paramter of rte_eth_dev_free().
> - Use rte_eth_dev structure as the paramter of rte_eth_dev_free().
>
> PATCH v5 changes
> - Add runtime check passthrough driver type, like vfio-pci, igb_uio
> and uio_pci_generic.
> This was done by Qiu, Michael. Thanks a lot.
> - Change function names like below.
> - rte_eal_dev_find_and_invoke() to rte_eal_vdev_find_and_invoke().
> - rte_eal_dev_invoke() to rte_eal_vdev_invoke().
> - Add code to handle a return value of rte_eal_devargs_remove().
> - Fix pci address format in rte_eal_dev_detach().
> - Remove RTE_EAL_INVOKE_TYPE_UNKNOWN, because it's unused.
> - Change function definition of rte_eal_devargs_remove().
> - Fix pci_unmap_device() to check pt_driver.
> - Fix return value of below functions.
> - rte_eth_dev_get_changed_port().
> - rte_eth_dev_get_port_by_addr().
> - Change paramters of rte_eth_dev_validate_port() to cleanup code.
> - Fix pci_scan_one to handle pt_driver correctly.
> (Thanks to Qiu, Michael for above sugesstions)
>
> PATCH v4 changes
> - Merge patches to review easier.
> - Fix indent of 'if' statement.
> - Fix calculation method of eal_compare_pci_addr().
> - Fix header file declaration.
> - Add header file to determine if hotplug can be enabled.
> (Thanks to Qiu, Michael)
> - Use braces with 'for' loop.
> - Add paramerter checking.
> - Fix sanity check code
> - Fix comments of rte_eth_dev_type.
> - Change function names.
> (Thanks to Iremonger, Bernard)
>
> PATCH v3 changes:
> - Fix enum definition used in rte_ethdev.c.
> (Thanks to Zhang, Helin)
>
> PATCH v2 changes:
> - Replace rte_eal_dev_attach_pdev(), rte_eal_dev_detach_pdev,
> rte_eal_dev_attach_vdev() and rte_eal_dev_detach_vdev() to
> rte_eal_dev_attach() and rte_eal_dev_detach().
> - Add parameter values checking.
> - Refashion a few functions.
> (Thanks to Iremonger, Bernard)
>
> PATCH v1 Changes:
> - Fix error checking code of librte_eth APIs.
> - Fix issue that port from pcap PMD cannot be detached correctly.
> - Fix issue that testpmd could hang after forwarding, if attaching and detaching
> is repeatedly.
> - Fix if-condition of rte_eth_dev_get_port_by_addr().
> (Thanks to Mark Enright)
>
> RFC PATCH v2 Changes:
> - remove 'rte_eth_dev_validate_port()', and cleanup codes.
>
>
> Michael Qiu (2):
> eal_pci: Add flag to hold kernel driver type
> eal_pci: pci memory map work with driver type
>
> Tetsuya Mukawa (11):
> eal/pci,ethdev: Remove assumption that port will not be detached
> eal/pci: Consolidate pci address comparison APIs
> ethdev: Add rte_eth_dev_free to free specified device
> eal,ethdev: Add a function and function pointers to close ether device
> ethdev: Add functions that will be used by port hotplug functions
> eal/linux/pci: Add functions for unmapping igb_uio resources
> eal/pci: Add a function to remove the entry of devargs list
> eal/pci: Cleanup pci driver initialization code
> ethdev: Add one dev_type paramerter to rte_eth_dev_allocate
> eal/pci: Add rte_eal_dev_attach/detach() functions
> eal: Enable port hotplug framework in Linux
>
> app/test/virtual_pmd.c | 2 +-
> config/common_linuxapp | 5 +
> lib/librte_eal/bsdapp/eal/eal_pci.c | 25 +-
> lib/librte_eal/common/Makefile | 1 +
> lib/librte_eal/common/eal_common_dev.c | 274 +++++++++++
> lib/librte_eal/common/eal_common_devargs.c | 60 +++
> lib/librte_eal/common/eal_common_pci.c | 92 +++-
> lib/librte_eal/common/eal_private.h | 35 ++
> lib/librte_eal/common/include/rte_dev.h | 33 ++
> lib/librte_eal/common/include/rte_dev_hotplug.h | 44 ++
> lib/librte_eal/common/include/rte_devargs.h | 21 +
> lib/librte_eal/common/include/rte_pci.h | 84 ++++
> lib/librte_eal/linuxapp/eal/Makefile | 1 +
> lib/librte_eal/linuxapp/eal/eal_pci.c | 227 +++++++--
> lib/librte_eal/linuxapp/eal/eal_pci_init.h | 8 +
> lib/librte_eal/linuxapp/eal/eal_pci_uio.c | 67 ++-
> lib/librte_ether/rte_ethdev.c | 624 +++++++++++++-----------
> lib/librte_ether/rte_ethdev.h | 148 +++++-
> lib/librte_pmd_af_packet/rte_eth_af_packet.c | 2 +-
> lib/librte_pmd_bond/rte_eth_bond_api.c | 2 +-
> lib/librte_pmd_pcap/rte_eth_pcap.c | 2 +-
> lib/librte_pmd_ring/rte_eth_ring.c | 2 +-
> lib/librte_pmd_xenvirt/rte_eth_xenvirt.c | 2 +-
> 23 files changed, 1414 insertions(+), 347 deletions(-) create mode 100644
> lib/librte_eal/common/include/rte_dev_hotplug.h
>
> --
> 1.9.1
Hi Tetsuya,
In lib/librte_eal/linuxapp/eal/eal_pci.c
Line 183: should "mapped" be "unmapped" ?
Regards,
Bernard.
On 2/3/2015 6:30 PM, Tetsuya Mukawa wrote: > On 2015/02/03 18:14, Qiu, Michael wrote: >> On 2/3/2015 2:16 PM, Qiu, Michael wrote: >>> On 2/1/2015 12:02 PM, Tetsuya Mukawa wrote: >>>> The patch introduces following commands. >>>> - port attach [ident] >>>> - port detach [port_id] >>>> - attach: attaching a port >>>> - detach: detaching a port >>>> - ident: pci address of physical device. >>>> Or device name and paramerters of virtual device. >>>> (ex. 0000:02:00.0, eth_pcap0,iface=eth0) >>>> - port_id: port identifier >>>> >>>> v5: >>>> - Add testpmd documentation. >>>> (Thanks to Iremonger, Bernard) >>>> v4: >>>> - Fix strings of command help. >>>> >>>> Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp> >> [...] >> >>>> +static int >>>> +port_is_closed(portid_t port_id) >>>> +{ >>>> + if (port_id_is_invalid(port_id, ENABLED_WARN)) >>>> + return 0; >>>> + >>>> + if (ports[port_id].port_status != RTE_PORT_CLOSED) >>>> + return 0; >>>> + >>>> + return 1; >>>> +} >>>> + >>>> +int >>>> start_port(portid_t pid) >>>> { >>>> int diag, need_check_link_status = 0; >>>> @@ -1296,8 +1347,8 @@ start_port(portid_t pid) >>>> >>>> if(dcb_config) >>>> dcb_test = 1; >>>> - for (pi = 0; pi < nb_ports; pi++) { >>>> - if (pid < nb_ports && pid != pi) >>>> + FOREACH_PORT(pi, ports) { >>>> + if (!port_id_is_invalid(pid, DISABLED_WARN) && pid != pi) >>> Here may it be: >>> >>> if (!port_id_is_invalid(pid, DISABLED_WARN) && (pid != pi || pid == RET_PORT_ALL)) >> Sorry, should be: >> >> if (!port_id_is_invalid(pid, DISABLED_WARN) && pid != pi && pid != (portid_t)RET_PORT_ALL) >> >> Otherwise, should check for "RET_PORT_ALL" in function port_id_is_invalid() > Thanks for comment. I've found 2 issues. > (I guess the original code has same issue.) Original code may not have this issue, cause RET_PORT_ALL is always greater than nb_ports, so "continue" will not exec. The logic may be right, but it is a little hard to understand. > One is that "port_id_is_invalid" should receives "pi" instead of "pid". > The other is if statement is wrong as you said. > > I guess following statement will be good. > > if (port_id_is_invalid(pi, DISABLED_WARN) || (pid != pi && pid != > (portid_t)RTE_PORT_ALL)) Actually, "port_id_is_invalid(pi, DISABLED_WARN)" could be removed as "FOREACH_PORT" will find a valid port. So it could be: if (pid != pi && pid != (portid_t)RTE_PORT_ALL) What do you think? Thanks, Michael > How about it? > > Thanks, > Tetsuya > > >> Thanks, >> Michael >> >>> Otherwise no port will be start by default. >>> >>> >>> Thanks, >>> Michael >>> >>>> continue; >>>> >>>> port = &ports[pi]; >>>> @@ -1421,7 +1472,7 @@ start_port(portid_t pid) >>>> } >>>> >>>> if (need_check_link_status && !no_link_check) >>>> - check_all_ports_link_status(nb_ports, RTE_PORT_ALL); >>>> + check_all_ports_link_status(RTE_PORT_ALL); >>>> else >>>> printf("Please stop the ports first\n"); >>>> >>>> @@ -1446,8 +1497,8 @@ stop_port(portid_t pid) >>>> } >>>> printf("Stopping ports...\n"); >>>> >>>> - for (pi = 0; pi < nb_ports; pi++) { >>>> - if (pid < nb_ports && pid != pi) >>>> + FOREACH_PORT(pi, ports) { >>>> + if (!port_id_is_invalid(pid, DISABLED_WARN) && pid != pi) >>>> continue; >>>> >>>> port = &ports[pi]; >>>> @@ -1463,7 +1514,7 @@ stop_port(portid_t pid) >>>> need_check_link_status = 1; >>>> } >>>> if (need_check_link_status && !no_link_check) >>>> - check_all_ports_link_status(nb_ports, RTE_PORT_ALL); >>>> + check_all_ports_link_status(RTE_PORT_ALL); >>>> >>>> printf("Done\n"); >>>> } >>>> @@ -1481,8 +1532,8 @@ close_port(portid_t pid) >>>> >>>> printf("Closing ports...\n"); >>>> >>>> - for (pi = 0; pi < nb_ports; pi++) { >>>> - if (pid < nb_ports && pid != pi) >>>> + FOREACH_PORT(pi, ports) { >>>> + if (!port_id_is_invalid(pid, DISABLED_WARN) && pid != pi) >>>> continue; >>>> >>>> port = &ports[pi]; >>>> @@ -1502,31 +1553,83 @@ close_port(portid_t pid) >>>> printf("Done\n"); >>>> } >>>> >>>> -int >>>> -all_ports_stopped(void) >>>> +void >>>> +attach_port(char *identifier) >>>> { >>>> - portid_t pi; >>>> - struct rte_port *port; >>>> + portid_t i, j, pi = 0; >>>> >>>> - for (pi = 0; pi < nb_ports; pi++) { >>>> - port = &ports[pi]; >>>> - if (port->port_status != RTE_PORT_STOPPED) >>>> - return 0; >>>> + printf("Attaching a new port...\n"); >>>> + >>>> + if (identifier == NULL) { >>>> + printf("Invalid parameters are speficied\n"); >>>> + return; >>>> } >>>> >>>> - return 1; >>>> + if (test_done == 0) { >>>> + printf("Please stop forwarding first\n"); >>>> + return; >>>> + } >>>> + >>>> + if (rte_eal_dev_attach(identifier, &pi)) >>>> + return; >>>> + >>>> + ports[pi].enabled = 1; >>>> + reconfig(pi, rte_eth_dev_socket_id(pi)); >>>> + rte_eth_promiscuous_enable(pi); >>>> + >>>> + nb_ports = rte_eth_dev_count(); >>>> + >>>> + /* set_default_fwd_ports_config(); */ >>>> + bzero(fwd_ports_ids, sizeof(fwd_ports_ids)); >>>> + i = 0; >>>> + FOREACH_PORT(j, ports) { >>>> + fwd_ports_ids[i] = j; >>>> + i++; >>>> + } >>>> + nb_cfg_ports = nb_ports; >>>> + nb_fwd_ports++; >>>> + >>>> + ports[pi].port_status = RTE_PORT_STOPPED; >>>> + >>>> + printf("Port %d is attached. Now total ports is %d\n", pi, nb_ports); >>>> + printf("Done\n"); >>>> } >>>> >>>> -int >>>> -port_is_started(portid_t port_id) >>>> +void >>>> +detach_port(uint8_t port_id) >>>> { >>>> - if (port_id_is_invalid(port_id)) >>>> - return -1; >>>> + portid_t i, pi = 0; >>>> + char name[RTE_ETH_NAME_MAX_LEN]; >>>> >>>> - if (ports[port_id].port_status != RTE_PORT_STARTED) >>>> - return 0; >>>> + printf("Detaching a port...\n"); >>>> >>>> - return 1; >>>> + if (!port_is_closed(port_id)) { >>>> + printf("Please close port first\n"); >>>> + return; >>>> + } >>>> + >>>> + rte_eth_promiscuous_disable(port_id); >>>> + >>>> + if (rte_eal_dev_detach(port_id, name)) >>>> + return; >>>> + >>>> + ports[port_id].enabled = 0; >>>> + nb_ports = rte_eth_dev_count(); >>>> + >>>> + /* set_default_fwd_ports_config(); */ >>>> + bzero(fwd_ports_ids, sizeof(fwd_ports_ids)); >>>> + i = 0; >>>> + FOREACH_PORT(pi, ports) { >>>> + fwd_ports_ids[i] = pi; >>>> + i++; >>>> + } >>>> + nb_cfg_ports = nb_ports; >>>> + nb_fwd_ports--; >>>> + >>>> + printf("Port '%s' is detached. Now total ports is %d\n", >>>> + name, nb_ports); >>>> + printf("Done\n"); >>>> + return; >>>> } >>>> >>>> void >>>> @@ -1534,7 +1637,7 @@ pmd_test_exit(void) >>>> { >>>> portid_t pt_id; >>>> >>>> - for (pt_id = 0; pt_id < nb_ports; pt_id++) { >>>> + FOREACH_PORT(pt_id, ports) { >>>> printf("Stopping port %d...", pt_id); >>>> fflush(stdout); >>>> rte_eth_dev_close(pt_id); >>>> @@ -1553,7 +1656,7 @@ struct pmd_test_command { >>>> >>>> /* Check the link status of all ports in up to 9s, and print them finally */ >>>> static void >>>> -check_all_ports_link_status(uint8_t port_num, uint32_t port_mask) >>>> +check_all_ports_link_status(uint32_t port_mask) >>>> { >>>> #define CHECK_INTERVAL 100 /* 100ms */ >>>> #define MAX_CHECK_TIME 90 /* 9s (90 * 100ms) in total */ >>>> @@ -1564,7 +1667,7 @@ check_all_ports_link_status(uint8_t port_num, uint32_t port_mask) >>>> fflush(stdout); >>>> for (count = 0; count <= MAX_CHECK_TIME; count++) { >>>> all_ports_up = 1; >>>> - for (portid = 0; portid < port_num; portid++) { >>>> + FOREACH_PORT(portid, ports) { >>>> if ((port_mask & (1 << portid)) == 0) >>>> continue; >>>> memset(&link, 0, sizeof(link)); >>>> @@ -1688,7 +1791,7 @@ init_port_config(void) >>>> portid_t pid; >>>> struct rte_port *port; >>>> >>>> - for (pid = 0; pid < nb_ports; pid++) { >>>> + FOREACH_PORT(pid, ports) { >>>> port = &ports[pid]; >>>> port->dev_conf.rxmode = rx_mode; >>>> port->dev_conf.fdir_conf = fdir_conf; >>>> @@ -1877,7 +1980,7 @@ main(int argc, char** argv) >>>> >>>> nb_ports = (portid_t) rte_eth_dev_count(); >>>> if (nb_ports == 0) >>>> - rte_exit(EXIT_FAILURE, "No probed ethernet device\n"); >>>> + RTE_LOG(WARNING, EAL, "No probed ethernet devices\n"); >>>> >>>> set_def_fwd_config(); >>>> if (nb_lcores == 0) >>>> @@ -1899,7 +2002,7 @@ main(int argc, char** argv) >>>> rte_exit(EXIT_FAILURE, "Start ports failed\n"); >>>> >>>> /* set all ports to promiscuous mode by default */ >>>> - for (port_id = 0; port_id < nb_ports; port_id++) >>>> + FOREACH_PORT(port_id, ports) >>>> rte_eth_promiscuous_enable(port_id); >>>> >>>> #ifdef RTE_LIBRTE_CMDLINE >>>> diff --git a/app/test-pmd/testpmd.h b/app/test-pmd/testpmd.h >>>> index 8f5e6c7..109c670 100644 >>>> --- a/app/test-pmd/testpmd.h >>>> +++ b/app/test-pmd/testpmd.h >>>> @@ -134,6 +134,7 @@ struct fwd_stream { >>>> * The data structure associated with each port. >>>> */ >>>> struct rte_port { >>>> + uint8_t enabled; /**< Port enabled or not */ >>>> struct rte_eth_dev_info dev_info; /**< PCI info + driver name */ >>>> struct rte_eth_conf dev_conf; /**< Port configuration. */ >>>> struct ether_addr eth_addr; /**< Port ethernet address */ >>>> @@ -159,6 +160,14 @@ struct rte_port { >>>> struct rte_eth_txconf tx_conf; /**< tx configuration */ >>>> }; >>>> >>>> +extern portid_t __rte_unused >>>> +find_next_port(portid_t p, struct rte_port *ports, int size); >>>> + >>>> +#define FOREACH_PORT(p, ports) \ >>>> + for (p = find_next_port(0, ports, RTE_MAX_ETHPORTS); \ >>>> + p < RTE_MAX_ETHPORTS; \ >>>> + p = find_next_port(p + 1, ports, RTE_MAX_ETHPORTS)) >>>> + >>>> /** >>>> * The data structure associated with each forwarding logical core. >>>> * The logical cores are internally numbered by a core index from 0 to >>>> @@ -515,6 +524,8 @@ int init_port_dcb_config(portid_t pid,struct dcb_config *dcb_conf); >>>> int start_port(portid_t pid); >>>> void stop_port(portid_t pid); >>>> void close_port(portid_t pid); >>>> +void attach_port(char *identifier); >>>> +void detach_port(uint8_t port_id); >>>> int all_ports_stopped(void); >>>> int port_is_started(portid_t port_id); >>>> void pmd_test_exit(void); >>>> @@ -558,10 +569,15 @@ void get_ethertype_filter(uint8_t port_id, uint16_t index); >>>> void get_2tuple_filter(uint8_t port_id, uint16_t index); >>>> void get_5tuple_filter(uint8_t port_id, uint16_t index); >>>> void get_flex_filter(uint8_t port_id, uint16_t index); >>>> -int port_id_is_invalid(portid_t port_id); >>>> int rx_queue_id_is_invalid(queueid_t rxq_id); >>>> int tx_queue_id_is_invalid(queueid_t txq_id); >>>> >>>> +enum print_warning { >>>> + ENABLED_WARN = 0, >>>> + DISABLED_WARN >>>> +}; >>>> +int port_id_is_invalid(portid_t port_id, enum print_warning warning); >>>> + >>>> /* >>>> * Work-around of a compilation error with ICC on invocations of the >>>> * rte_be_to_cpu_16() function. >>>> diff --git a/doc/guides/testpmd_app_ug/testpmd_funcs.rst b/doc/guides/testpmd_app_ug/testpmd_funcs.rst >>>> index 218835a..1cacbcf 100644 >>>> --- a/doc/guides/testpmd_app_ug/testpmd_funcs.rst >>>> +++ b/doc/guides/testpmd_app_ug/testpmd_funcs.rst >>>> @@ -808,6 +808,63 @@ The following sections show functions for configuring ports. >>>> >>>> Port configuration changes only become active when forwarding is started/restarted. >>>> >>>> +port attach >>>> +~~~~~~~~~~~ >>>> + >>>> +Attach a port specified by pci address or virtual device args. >>>> + >>>> +To attach a new pci device, the device should be recognized by kernel first. >>>> +Then it should be moved under DPDK management. >>>> +Finally the port can be attached to testpmd. >>>> +On the other hand, to attach a port created by virtual device, above steps are not needed. >>>> + >>>> +port attach (identifier) >>>> + >>>> +For example, to attach a port that pci address is 0000:02:00.0. >>>> + >>>> +.. code-block:: console >>>> + >>>> + testpmd> port attach 0000:02:00.0 >>>> + Attaching a new port... >>>> + ... snip ... >>>> + Port 0 is attached. Now total ports is 1 >>>> + Done >>>> + >>>> +For example, to attach a port created by pcap PMD. >>>> + >>>> +.. code-block:: console >>>> + >>>> + testpmd> port attach eth_pcap0,iface=eth0 >>>> + Attaching a new port... >>>> + ... snip ... >>>> + Port 0 is attached. Now total ports is 1 >>>> + Done >>>> + >>>> +In this case, identifier is "eth_pcap0,iface=eth0". >>>> +This identifier format is the same as "--vdev" format of DPDK applications. >>>> + >>>> +port detach >>>> +~~~~~~~~~~~ >>>> + >>>> +Detach a specific port. >>>> + >>>> +Before detaching a port, the port should be closed. >>>> +Also to remove a pci device completely from the system, first detach the port from testpmd. >>>> +Then the device should be moved under kernel management. >>>> +Finally the device can be remove using kernel pci hotplug functionality. >>>> +On the other hand, to remove a port created by virtual device, above steps are not needed. >>>> + >>>> +port detach (port_id) >>>> + >>>> +For example, to detach a port 0. >>>> + >>>> +.. code-block:: console >>>> + >>>> + testpmd> port detach 0 >>>> + Detaching a port... >>>> + ... snip ... >>>> + Done >>>> + >>>> port start >>>> ~~~~~~~~~~ >>>> > >
On 2015/02/03 22:03, Iremonger, Bernard wrote: >> -----Original Message----- >> From: Tetsuya Mukawa [mailto:mukawa@igel.co.jp] >> Sent: Sunday, February 1, 2015 4:02 AM >> To: dev@dpdk.org >> Cc: Qiu, Michael; Iremonger, Bernard; Tetsuya Mukawa >> Subject: [PATCH v6 00/13] Port Hotplug Framework >> >> This patch series adds a dynamic port hotplug framework to DPDK. >> With the patches, DPDK apps can attach or detach ports at runtime. >> >> The basic concept of the port hotplug is like followings. >> - DPDK apps must have responsibility to manage ports. >> DPDK apps only know which ports are attached or detached at the moment. >> The port hotplug framework is implemented to allow DPDK apps to manage ports. >> For example, when DPDK apps call port attach function, attached port number >> will be returned. Also DPDK apps can detach port by port number. >> - Kernel support is needed for attaching or detaching physical device ports. >> To attach a new physical device port, the device will be recognized by >> userspace directly I/O framework in kernel at first. Then DPDK apps can >> call the port hotplug functions to attach ports. >> For detaching, steps are vice versa. >> - Before detach ports, ports must be stopped and closed. >> DPDK application must call rte_eth_dev_stop() and rte_eth_dev_close() before >> detaching ports. These function will call finalization codes of PMDs. >> But so far, no PMD frees all resources allocated by initialization. >> It means PMDs are needed to be fixed to support the port hotplug. >> 'RTE_PCI_DRV_DETACHABLE' is a new flag indicating a PMD supports detaching. >> Without this flag, detaching will be failed. >> - Mustn't affect legacy DPDK apps. >> No DPDK EAL behavior is changed, if the port hotplug functions are't called. >> So all legacy DPDK apps can still work without modifications. >> >> And a few limitations. >> - The port hotplug functions are not thread safe. >> DPDK apps should handle it. >> - Only support Linux and igb_uio so far. >> BSD and VFIO is not supported. I will send VFIO patches at least, but I don't >> have a plan to submit BSD patch so far. >> >> >> Here is port hotplug APIs. >> ------------------------------------------------------------------------------- >> /** >> * Attach a new device. >> * >> * @param devargs >> * A pointer to a strings array describing the new device >> * to be attached. The strings should be a pci address like >> * '0000:01:00.0' or virtual device name like 'eth_pcap0'. >> * @param port_id >> * A pointer to a port identifier actually attached. >> * @return >> * 0 on success and port_id is filled, negative on error */ int rte_eal_dev_attach(const char *devargs, >> uint8_t *port_id); >> >> /** >> * Detach a device. >> * >> * @param port_id >> * The port identifier of the device to detach. >> * @param addr >> * A pointer to a device name actually detached. >> * @return >> * 0 on success and devname is filled, negative on error */ int rte_eal_dev_detach(uint8_t port_id, >> char *devname); >> ------------------------------------------------------------------------------- >> >> This patch series are for DPDK EAL. To use port hotplug function by DPDK apps, each PMD should be >> fixed to support 'RTE_PCI_DRV_DETACHABLE' flag. Please check a patch for pcap PMD. >> >> Also please check testpmd patch. It will show you how to fix your legacy applications to support port >> hotplug feature. >> >> PATCH v6 changes >> - Fix rte_eth_dev_uninit() to handle a return value of uninit >> function of PMD. To do this, below changes also be applied. >> - Fix a paramter of rte_eth_dev_free(). >> - Use rte_eth_dev structure as the paramter of rte_eth_dev_free(). >> >> PATCH v5 changes >> - Add runtime check passthrough driver type, like vfio-pci, igb_uio >> and uio_pci_generic. >> This was done by Qiu, Michael. Thanks a lot. >> - Change function names like below. >> - rte_eal_dev_find_and_invoke() to rte_eal_vdev_find_and_invoke(). >> - rte_eal_dev_invoke() to rte_eal_vdev_invoke(). >> - Add code to handle a return value of rte_eal_devargs_remove(). >> - Fix pci address format in rte_eal_dev_detach(). >> - Remove RTE_EAL_INVOKE_TYPE_UNKNOWN, because it's unused. >> - Change function definition of rte_eal_devargs_remove(). >> - Fix pci_unmap_device() to check pt_driver. >> - Fix return value of below functions. >> - rte_eth_dev_get_changed_port(). >> - rte_eth_dev_get_port_by_addr(). >> - Change paramters of rte_eth_dev_validate_port() to cleanup code. >> - Fix pci_scan_one to handle pt_driver correctly. >> (Thanks to Qiu, Michael for above sugesstions) >> >> PATCH v4 changes >> - Merge patches to review easier. >> - Fix indent of 'if' statement. >> - Fix calculation method of eal_compare_pci_addr(). >> - Fix header file declaration. >> - Add header file to determine if hotplug can be enabled. >> (Thanks to Qiu, Michael) >> - Use braces with 'for' loop. >> - Add paramerter checking. >> - Fix sanity check code >> - Fix comments of rte_eth_dev_type. >> - Change function names. >> (Thanks to Iremonger, Bernard) >> >> PATCH v3 changes: >> - Fix enum definition used in rte_ethdev.c. >> (Thanks to Zhang, Helin) >> >> PATCH v2 changes: >> - Replace rte_eal_dev_attach_pdev(), rte_eal_dev_detach_pdev, >> rte_eal_dev_attach_vdev() and rte_eal_dev_detach_vdev() to >> rte_eal_dev_attach() and rte_eal_dev_detach(). >> - Add parameter values checking. >> - Refashion a few functions. >> (Thanks to Iremonger, Bernard) >> >> PATCH v1 Changes: >> - Fix error checking code of librte_eth APIs. >> - Fix issue that port from pcap PMD cannot be detached correctly. >> - Fix issue that testpmd could hang after forwarding, if attaching and detaching >> is repeatedly. >> - Fix if-condition of rte_eth_dev_get_port_by_addr(). >> (Thanks to Mark Enright) >> >> RFC PATCH v2 Changes: >> - remove 'rte_eth_dev_validate_port()', and cleanup codes. >> >> >> Michael Qiu (2): >> eal_pci: Add flag to hold kernel driver type >> eal_pci: pci memory map work with driver type >> >> Tetsuya Mukawa (11): >> eal/pci,ethdev: Remove assumption that port will not be detached >> eal/pci: Consolidate pci address comparison APIs >> ethdev: Add rte_eth_dev_free to free specified device >> eal,ethdev: Add a function and function pointers to close ether device >> ethdev: Add functions that will be used by port hotplug functions >> eal/linux/pci: Add functions for unmapping igb_uio resources >> eal/pci: Add a function to remove the entry of devargs list >> eal/pci: Cleanup pci driver initialization code >> ethdev: Add one dev_type paramerter to rte_eth_dev_allocate >> eal/pci: Add rte_eal_dev_attach/detach() functions >> eal: Enable port hotplug framework in Linux >> >> app/test/virtual_pmd.c | 2 +- >> config/common_linuxapp | 5 + >> lib/librte_eal/bsdapp/eal/eal_pci.c | 25 +- >> lib/librte_eal/common/Makefile | 1 + >> lib/librte_eal/common/eal_common_dev.c | 274 +++++++++++ >> lib/librte_eal/common/eal_common_devargs.c | 60 +++ >> lib/librte_eal/common/eal_common_pci.c | 92 +++- >> lib/librte_eal/common/eal_private.h | 35 ++ >> lib/librte_eal/common/include/rte_dev.h | 33 ++ >> lib/librte_eal/common/include/rte_dev_hotplug.h | 44 ++ >> lib/librte_eal/common/include/rte_devargs.h | 21 + >> lib/librte_eal/common/include/rte_pci.h | 84 ++++ >> lib/librte_eal/linuxapp/eal/Makefile | 1 + >> lib/librte_eal/linuxapp/eal/eal_pci.c | 227 +++++++-- >> lib/librte_eal/linuxapp/eal/eal_pci_init.h | 8 + >> lib/librte_eal/linuxapp/eal/eal_pci_uio.c | 67 ++- >> lib/librte_ether/rte_ethdev.c | 624 +++++++++++++----------- >> lib/librte_ether/rte_ethdev.h | 148 +++++- >> lib/librte_pmd_af_packet/rte_eth_af_packet.c | 2 +- >> lib/librte_pmd_bond/rte_eth_bond_api.c | 2 +- >> lib/librte_pmd_pcap/rte_eth_pcap.c | 2 +- >> lib/librte_pmd_ring/rte_eth_ring.c | 2 +- >> lib/librte_pmd_xenvirt/rte_eth_xenvirt.c | 2 +- >> 23 files changed, 1414 insertions(+), 347 deletions(-) create mode 100644 >> lib/librte_eal/common/include/rte_dev_hotplug.h >> >> -- >> 1.9.1 > Hi Tetsuya, > > I have a few comments as follows: > > Lib/librte_ether/rte_ethdev.c > Line 400 : NO_TRACE would be clearer than NONE_TRACE. > Line 440: function rte_eth_dev_save() should pass in a size parameter which should be used in the memcpy function. Hi Bernard, I appreciate your comment, I will fix like above. > Lib/librte_eal/common/include/ret_dev_hotplug.h > Is this file really necessary as it contains only one macro? > Could it be merged into Lib/librte_eal/common/include/ret_dev.h ? I guess it's nice to separate. As you said, rte_dev_hotplug.h has only one macro. This macro is used for determining whether user enables hotplug function, or not. And it is included by below headers. ./lib/librte_eal/common/include/rte_pci.h ./lib/librte_eal/linuxapp/eal/eal_pci_init.h Above 2 headers doesn't need any rte_dev.h definitions without hotplug macro. Also when someone reads DPDK codes, they can easily understand 'pci' and 'dev' layer is conceptually separated. So I guess it's nice to separate hotplug definition from rte_dev.h. Thanks, Tetsuya > Regards, > > Bernard. > > >
On 2015/02/04 3:35, Iremonger, Bernard wrote: > >> -----Original Message----- >> From: Tetsuya Mukawa [mailto:mukawa@igel.co.jp] >> Sent: Sunday, February 1, 2015 4:02 AM >> To: dev@dpdk.org >> Cc: Qiu, Michael; Iremonger, Bernard; Tetsuya Mukawa >> Subject: [PATCH v6 00/13] Port Hotplug Framework >> >> This patch series adds a dynamic port hotplug framework to DPDK. >> With the patches, DPDK apps can attach or detach ports at runtime. >> >> The basic concept of the port hotplug is like followings. >> - DPDK apps must have responsibility to manage ports. >> DPDK apps only know which ports are attached or detached at the moment. >> The port hotplug framework is implemented to allow DPDK apps to manage ports. >> For example, when DPDK apps call port attach function, attached port number >> will be returned. Also DPDK apps can detach port by port number. >> - Kernel support is needed for attaching or detaching physical device ports. >> To attach a new physical device port, the device will be recognized by >> userspace directly I/O framework in kernel at first. Then DPDK apps can >> call the port hotplug functions to attach ports. >> For detaching, steps are vice versa. >> - Before detach ports, ports must be stopped and closed. >> DPDK application must call rte_eth_dev_stop() and rte_eth_dev_close() before >> detaching ports. These function will call finalization codes of PMDs. >> But so far, no PMD frees all resources allocated by initialization. >> It means PMDs are needed to be fixed to support the port hotplug. >> 'RTE_PCI_DRV_DETACHABLE' is a new flag indicating a PMD supports detaching. >> Without this flag, detaching will be failed. >> - Mustn't affect legacy DPDK apps. >> No DPDK EAL behavior is changed, if the port hotplug functions are't called. >> So all legacy DPDK apps can still work without modifications. >> >> And a few limitations. >> - The port hotplug functions are not thread safe. >> DPDK apps should handle it. >> - Only support Linux and igb_uio so far. >> BSD and VFIO is not supported. I will send VFIO patches at least, but I don't >> have a plan to submit BSD patch so far. >> >> >> Here is port hotplug APIs. >> ------------------------------------------------------------------------------- >> /** >> * Attach a new device. >> * >> * @param devargs >> * A pointer to a strings array describing the new device >> * to be attached. The strings should be a pci address like >> * '0000:01:00.0' or virtual device name like 'eth_pcap0'. >> * @param port_id >> * A pointer to a port identifier actually attached. >> * @return >> * 0 on success and port_id is filled, negative on error */ int rte_eal_dev_attach(const char *devargs, >> uint8_t *port_id); >> >> /** >> * Detach a device. >> * >> * @param port_id >> * The port identifier of the device to detach. >> * @param addr >> * A pointer to a device name actually detached. >> * @return >> * 0 on success and devname is filled, negative on error */ int rte_eal_dev_detach(uint8_t port_id, >> char *devname); >> ------------------------------------------------------------------------------- >> >> This patch series are for DPDK EAL. To use port hotplug function by DPDK apps, each PMD should be >> fixed to support 'RTE_PCI_DRV_DETACHABLE' flag. Please check a patch for pcap PMD. >> >> Also please check testpmd patch. It will show you how to fix your legacy applications to support port >> hotplug feature. >> >> PATCH v6 changes >> - Fix rte_eth_dev_uninit() to handle a return value of uninit >> function of PMD. To do this, below changes also be applied. >> - Fix a paramter of rte_eth_dev_free(). >> - Use rte_eth_dev structure as the paramter of rte_eth_dev_free(). >> >> PATCH v5 changes >> - Add runtime check passthrough driver type, like vfio-pci, igb_uio >> and uio_pci_generic. >> This was done by Qiu, Michael. Thanks a lot. >> - Change function names like below. >> - rte_eal_dev_find_and_invoke() to rte_eal_vdev_find_and_invoke(). >> - rte_eal_dev_invoke() to rte_eal_vdev_invoke(). >> - Add code to handle a return value of rte_eal_devargs_remove(). >> - Fix pci address format in rte_eal_dev_detach(). >> - Remove RTE_EAL_INVOKE_TYPE_UNKNOWN, because it's unused. >> - Change function definition of rte_eal_devargs_remove(). >> - Fix pci_unmap_device() to check pt_driver. >> - Fix return value of below functions. >> - rte_eth_dev_get_changed_port(). >> - rte_eth_dev_get_port_by_addr(). >> - Change paramters of rte_eth_dev_validate_port() to cleanup code. >> - Fix pci_scan_one to handle pt_driver correctly. >> (Thanks to Qiu, Michael for above sugesstions) >> >> PATCH v4 changes >> - Merge patches to review easier. >> - Fix indent of 'if' statement. >> - Fix calculation method of eal_compare_pci_addr(). >> - Fix header file declaration. >> - Add header file to determine if hotplug can be enabled. >> (Thanks to Qiu, Michael) >> - Use braces with 'for' loop. >> - Add paramerter checking. >> - Fix sanity check code >> - Fix comments of rte_eth_dev_type. >> - Change function names. >> (Thanks to Iremonger, Bernard) >> >> PATCH v3 changes: >> - Fix enum definition used in rte_ethdev.c. >> (Thanks to Zhang, Helin) >> >> PATCH v2 changes: >> - Replace rte_eal_dev_attach_pdev(), rte_eal_dev_detach_pdev, >> rte_eal_dev_attach_vdev() and rte_eal_dev_detach_vdev() to >> rte_eal_dev_attach() and rte_eal_dev_detach(). >> - Add parameter values checking. >> - Refashion a few functions. >> (Thanks to Iremonger, Bernard) >> >> PATCH v1 Changes: >> - Fix error checking code of librte_eth APIs. >> - Fix issue that port from pcap PMD cannot be detached correctly. >> - Fix issue that testpmd could hang after forwarding, if attaching and detaching >> is repeatedly. >> - Fix if-condition of rte_eth_dev_get_port_by_addr(). >> (Thanks to Mark Enright) >> >> RFC PATCH v2 Changes: >> - remove 'rte_eth_dev_validate_port()', and cleanup codes. >> >> >> Michael Qiu (2): >> eal_pci: Add flag to hold kernel driver type >> eal_pci: pci memory map work with driver type >> >> Tetsuya Mukawa (11): >> eal/pci,ethdev: Remove assumption that port will not be detached >> eal/pci: Consolidate pci address comparison APIs >> ethdev: Add rte_eth_dev_free to free specified device >> eal,ethdev: Add a function and function pointers to close ether device >> ethdev: Add functions that will be used by port hotplug functions >> eal/linux/pci: Add functions for unmapping igb_uio resources >> eal/pci: Add a function to remove the entry of devargs list >> eal/pci: Cleanup pci driver initialization code >> ethdev: Add one dev_type paramerter to rte_eth_dev_allocate >> eal/pci: Add rte_eal_dev_attach/detach() functions >> eal: Enable port hotplug framework in Linux >> >> app/test/virtual_pmd.c | 2 +- >> config/common_linuxapp | 5 + >> lib/librte_eal/bsdapp/eal/eal_pci.c | 25 +- >> lib/librte_eal/common/Makefile | 1 + >> lib/librte_eal/common/eal_common_dev.c | 274 +++++++++++ >> lib/librte_eal/common/eal_common_devargs.c | 60 +++ >> lib/librte_eal/common/eal_common_pci.c | 92 +++- >> lib/librte_eal/common/eal_private.h | 35 ++ >> lib/librte_eal/common/include/rte_dev.h | 33 ++ >> lib/librte_eal/common/include/rte_dev_hotplug.h | 44 ++ >> lib/librte_eal/common/include/rte_devargs.h | 21 + >> lib/librte_eal/common/include/rte_pci.h | 84 ++++ >> lib/librte_eal/linuxapp/eal/Makefile | 1 + >> lib/librte_eal/linuxapp/eal/eal_pci.c | 227 +++++++-- >> lib/librte_eal/linuxapp/eal/eal_pci_init.h | 8 + >> lib/librte_eal/linuxapp/eal/eal_pci_uio.c | 67 ++- >> lib/librte_ether/rte_ethdev.c | 624 +++++++++++++----------- >> lib/librte_ether/rte_ethdev.h | 148 +++++- >> lib/librte_pmd_af_packet/rte_eth_af_packet.c | 2 +- >> lib/librte_pmd_bond/rte_eth_bond_api.c | 2 +- >> lib/librte_pmd_pcap/rte_eth_pcap.c | 2 +- >> lib/librte_pmd_ring/rte_eth_ring.c | 2 +- >> lib/librte_pmd_xenvirt/rte_eth_xenvirt.c | 2 +- >> 23 files changed, 1414 insertions(+), 347 deletions(-) create mode 100644 >> lib/librte_eal/common/include/rte_dev_hotplug.h >> >> -- >> 1.9.1 > Hi Tetsuya, > > In lib/librte_eal/linuxapp/eal/eal_pci.c > Line 183: should "mapped" be "unmapped" ? Hi Bernard, Thanks, this error message should be "unmapped". I will fix it. Thanks, Tetsuya > Regards, > > Bernard. > > >
On 2015/02/04 10:44, Qiu, Michael wrote: > On 2/3/2015 6:30 PM, Tetsuya Mukawa wrote: >> On 2015/02/03 18:14, Qiu, Michael wrote: >>> On 2/3/2015 2:16 PM, Qiu, Michael wrote: >>>> On 2/1/2015 12:02 PM, Tetsuya Mukawa wrote: >>>>> The patch introduces following commands. >>>>> - port attach [ident] >>>>> - port detach [port_id] >>>>> - attach: attaching a port >>>>> - detach: detaching a port >>>>> - ident: pci address of physical device. >>>>> Or device name and paramerters of virtual device. >>>>> (ex. 0000:02:00.0, eth_pcap0,iface=eth0) >>>>> - port_id: port identifier >>>>> >>>>> v5: >>>>> - Add testpmd documentation. >>>>> (Thanks to Iremonger, Bernard) >>>>> v4: >>>>> - Fix strings of command help. >>>>> >>>>> Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp> >>> [...] >>> >>>>> +static int >>>>> +port_is_closed(portid_t port_id) >>>>> +{ >>>>> + if (port_id_is_invalid(port_id, ENABLED_WARN)) >>>>> + return 0; >>>>> + >>>>> + if (ports[port_id].port_status != RTE_PORT_CLOSED) >>>>> + return 0; >>>>> + >>>>> + return 1; >>>>> +} >>>>> + >>>>> +int >>>>> start_port(portid_t pid) >>>>> { >>>>> int diag, need_check_link_status = 0; >>>>> @@ -1296,8 +1347,8 @@ start_port(portid_t pid) >>>>> >>>>> if(dcb_config) >>>>> dcb_test = 1; >>>>> - for (pi = 0; pi < nb_ports; pi++) { >>>>> - if (pid < nb_ports && pid != pi) >>>>> + FOREACH_PORT(pi, ports) { >>>>> + if (!port_id_is_invalid(pid, DISABLED_WARN) && pid != pi) >>>> Here may it be: >>>> >>>> if (!port_id_is_invalid(pid, DISABLED_WARN) && (pid != pi || pid == RET_PORT_ALL)) >>> Sorry, should be: >>> >>> if (!port_id_is_invalid(pid, DISABLED_WARN) && pid != pi && pid != (portid_t)RET_PORT_ALL) >>> >>> Otherwise, should check for "RET_PORT_ALL" in function port_id_is_invalid() >> Thanks for comment. I've found 2 issues. >> (I guess the original code has same issue.) > Original code may not have this issue, cause RET_PORT_ALL is always > greater than nb_ports, so "continue" will not exec. The logic may be > right, but it is a little hard to understand. > >> One is that "port_id_is_invalid" should receives "pi" instead of "pid". >> The other is if statement is wrong as you said. >> >> I guess following statement will be good. >> >> if (port_id_is_invalid(pi, DISABLED_WARN) || (pid != pi && pid != >> (portid_t)RTE_PORT_ALL)) > Actually, "port_id_is_invalid(pi, DISABLED_WARN)" could be removed as > "FOREACH_PORT" will find a valid port. Good point! > So it could be: > > if (pid != pi && pid != (portid_t)RTE_PORT_ALL) > > What do you think? I agree with you. I will change like above. Thanks, Tetsuya > Thanks, > Michael >> How about it? >> >> Thanks, >> Tetsuya >> >> >>> Thanks, >>> Michael >>> >>>> Otherwise no port will be start by default. >>>> >>>> >>>> Thanks, >>>> Michael >>>> >>>>> continue; >>>>> >>>>> port = &ports[pi]; >>>>> @@ -1421,7 +1472,7 @@ start_port(portid_t pid) >>>>> } >>>>> >>>>> if (need_check_link_status && !no_link_check) >>>>> - check_all_ports_link_status(nb_ports, RTE_PORT_ALL); >>>>> + check_all_ports_link_status(RTE_PORT_ALL); >>>>> else >>>>> printf("Please stop the ports first\n"); >>>>> >>>>> @@ -1446,8 +1497,8 @@ stop_port(portid_t pid) >>>>> } >>>>> printf("Stopping ports...\n"); >>>>> >>>>> - for (pi = 0; pi < nb_ports; pi++) { >>>>> - if (pid < nb_ports && pid != pi) >>>>> + FOREACH_PORT(pi, ports) { >>>>> + if (!port_id_is_invalid(pid, DISABLED_WARN) && pid != pi) >>>>> continue; >>>>> >>>>> port = &ports[pi]; >>>>> @@ -1463,7 +1514,7 @@ stop_port(portid_t pid) >>>>> need_check_link_status = 1; >>>>> } >>>>> if (need_check_link_status && !no_link_check) >>>>> - check_all_ports_link_status(nb_ports, RTE_PORT_ALL); >>>>> + check_all_ports_link_status(RTE_PORT_ALL); >>>>> >>>>> printf("Done\n"); >>>>> } >>>>> @@ -1481,8 +1532,8 @@ close_port(portid_t pid) >>>>> >>>>> printf("Closing ports...\n"); >>>>> >>>>> - for (pi = 0; pi < nb_ports; pi++) { >>>>> - if (pid < nb_ports && pid != pi) >>>>> + FOREACH_PORT(pi, ports) { >>>>> + if (!port_id_is_invalid(pid, DISABLED_WARN) && pid != pi) >>>>> continue; >>>>> >>>>> port = &ports[pi]; >>>>> @@ -1502,31 +1553,83 @@ close_port(portid_t pid) >>>>> printf("Done\n"); >>>>> } >>>>> >>>>> -int >>>>> -all_ports_stopped(void) >>>>> +void >>>>> +attach_port(char *identifier) >>>>> { >>>>> - portid_t pi; >>>>> - struct rte_port *port; >>>>> + portid_t i, j, pi = 0; >>>>> >>>>> - for (pi = 0; pi < nb_ports; pi++) { >>>>> - port = &ports[pi]; >>>>> - if (port->port_status != RTE_PORT_STOPPED) >>>>> - return 0; >>>>> + printf("Attaching a new port...\n"); >>>>> + >>>>> + if (identifier == NULL) { >>>>> + printf("Invalid parameters are speficied\n"); >>>>> + return; >>>>> } >>>>> >>>>> - return 1; >>>>> + if (test_done == 0) { >>>>> + printf("Please stop forwarding first\n"); >>>>> + return; >>>>> + } >>>>> + >>>>> + if (rte_eal_dev_attach(identifier, &pi)) >>>>> + return; >>>>> + >>>>> + ports[pi].enabled = 1; >>>>> + reconfig(pi, rte_eth_dev_socket_id(pi)); >>>>> + rte_eth_promiscuous_enable(pi); >>>>> + >>>>> + nb_ports = rte_eth_dev_count(); >>>>> + >>>>> + /* set_default_fwd_ports_config(); */ >>>>> + bzero(fwd_ports_ids, sizeof(fwd_ports_ids)); >>>>> + i = 0; >>>>> + FOREACH_PORT(j, ports) { >>>>> + fwd_ports_ids[i] = j; >>>>> + i++; >>>>> + } >>>>> + nb_cfg_ports = nb_ports; >>>>> + nb_fwd_ports++; >>>>> + >>>>> + ports[pi].port_status = RTE_PORT_STOPPED; >>>>> + >>>>> + printf("Port %d is attached. Now total ports is %d\n", pi, nb_ports); >>>>> + printf("Done\n"); >>>>> } >>>>> >>>>> -int >>>>> -port_is_started(portid_t port_id) >>>>> +void >>>>> +detach_port(uint8_t port_id) >>>>> { >>>>> - if (port_id_is_invalid(port_id)) >>>>> - return -1; >>>>> + portid_t i, pi = 0; >>>>> + char name[RTE_ETH_NAME_MAX_LEN]; >>>>> >>>>> - if (ports[port_id].port_status != RTE_PORT_STARTED) >>>>> - return 0; >>>>> + printf("Detaching a port...\n"); >>>>> >>>>> - return 1; >>>>> + if (!port_is_closed(port_id)) { >>>>> + printf("Please close port first\n"); >>>>> + return; >>>>> + } >>>>> + >>>>> + rte_eth_promiscuous_disable(port_id); >>>>> + >>>>> + if (rte_eal_dev_detach(port_id, name)) >>>>> + return; >>>>> + >>>>> + ports[port_id].enabled = 0; >>>>> + nb_ports = rte_eth_dev_count(); >>>>> + >>>>> + /* set_default_fwd_ports_config(); */ >>>>> + bzero(fwd_ports_ids, sizeof(fwd_ports_ids)); >>>>> + i = 0; >>>>> + FOREACH_PORT(pi, ports) { >>>>> + fwd_ports_ids[i] = pi; >>>>> + i++; >>>>> + } >>>>> + nb_cfg_ports = nb_ports; >>>>> + nb_fwd_ports--; >>>>> + >>>>> + printf("Port '%s' is detached. Now total ports is %d\n", >>>>> + name, nb_ports); >>>>> + printf("Done\n"); >>>>> + return; >>>>> } >>>>> >>>>> void >>>>> @@ -1534,7 +1637,7 @@ pmd_test_exit(void) >>>>> { >>>>> portid_t pt_id; >>>>> >>>>> - for (pt_id = 0; pt_id < nb_ports; pt_id++) { >>>>> + FOREACH_PORT(pt_id, ports) { >>>>> printf("Stopping port %d...", pt_id); >>>>> fflush(stdout); >>>>> rte_eth_dev_close(pt_id); >>>>> @@ -1553,7 +1656,7 @@ struct pmd_test_command { >>>>> >>>>> /* Check the link status of all ports in up to 9s, and print them finally */ >>>>> static void >>>>> -check_all_ports_link_status(uint8_t port_num, uint32_t port_mask) >>>>> +check_all_ports_link_status(uint32_t port_mask) >>>>> { >>>>> #define CHECK_INTERVAL 100 /* 100ms */ >>>>> #define MAX_CHECK_TIME 90 /* 9s (90 * 100ms) in total */ >>>>> @@ -1564,7 +1667,7 @@ check_all_ports_link_status(uint8_t port_num, uint32_t port_mask) >>>>> fflush(stdout); >>>>> for (count = 0; count <= MAX_CHECK_TIME; count++) { >>>>> all_ports_up = 1; >>>>> - for (portid = 0; portid < port_num; portid++) { >>>>> + FOREACH_PORT(portid, ports) { >>>>> if ((port_mask & (1 << portid)) == 0) >>>>> continue; >>>>> memset(&link, 0, sizeof(link)); >>>>> @@ -1688,7 +1791,7 @@ init_port_config(void) >>>>> portid_t pid; >>>>> struct rte_port *port; >>>>> >>>>> - for (pid = 0; pid < nb_ports; pid++) { >>>>> + FOREACH_PORT(pid, ports) { >>>>> port = &ports[pid]; >>>>> port->dev_conf.rxmode = rx_mode; >>>>> port->dev_conf.fdir_conf = fdir_conf; >>>>> @@ -1877,7 +1980,7 @@ main(int argc, char** argv) >>>>> >>>>> nb_ports = (portid_t) rte_eth_dev_count(); >>>>> if (nb_ports == 0) >>>>> - rte_exit(EXIT_FAILURE, "No probed ethernet device\n"); >>>>> + RTE_LOG(WARNING, EAL, "No probed ethernet devices\n"); >>>>> >>>>> set_def_fwd_config(); >>>>> if (nb_lcores == 0) >>>>> @@ -1899,7 +2002,7 @@ main(int argc, char** argv) >>>>> rte_exit(EXIT_FAILURE, "Start ports failed\n"); >>>>> >>>>> /* set all ports to promiscuous mode by default */ >>>>> - for (port_id = 0; port_id < nb_ports; port_id++) >>>>> + FOREACH_PORT(port_id, ports) >>>>> rte_eth_promiscuous_enable(port_id); >>>>> >>>>> #ifdef RTE_LIBRTE_CMDLINE >>>>> diff --git a/app/test-pmd/testpmd.h b/app/test-pmd/testpmd.h >>>>> index 8f5e6c7..109c670 100644 >>>>> --- a/app/test-pmd/testpmd.h >>>>> +++ b/app/test-pmd/testpmd.h >>>>> @@ -134,6 +134,7 @@ struct fwd_stream { >>>>> * The data structure associated with each port. >>>>> */ >>>>> struct rte_port { >>>>> + uint8_t enabled; /**< Port enabled or not */ >>>>> struct rte_eth_dev_info dev_info; /**< PCI info + driver name */ >>>>> struct rte_eth_conf dev_conf; /**< Port configuration. */ >>>>> struct ether_addr eth_addr; /**< Port ethernet address */ >>>>> @@ -159,6 +160,14 @@ struct rte_port { >>>>> struct rte_eth_txconf tx_conf; /**< tx configuration */ >>>>> }; >>>>> >>>>> +extern portid_t __rte_unused >>>>> +find_next_port(portid_t p, struct rte_port *ports, int size); >>>>> + >>>>> +#define FOREACH_PORT(p, ports) \ >>>>> + for (p = find_next_port(0, ports, RTE_MAX_ETHPORTS); \ >>>>> + p < RTE_MAX_ETHPORTS; \ >>>>> + p = find_next_port(p + 1, ports, RTE_MAX_ETHPORTS)) >>>>> + >>>>> /** >>>>> * The data structure associated with each forwarding logical core. >>>>> * The logical cores are internally numbered by a core index from 0 to >>>>> @@ -515,6 +524,8 @@ int init_port_dcb_config(portid_t pid,struct dcb_config *dcb_conf); >>>>> int start_port(portid_t pid); >>>>> void stop_port(portid_t pid); >>>>> void close_port(portid_t pid); >>>>> +void attach_port(char *identifier); >>>>> +void detach_port(uint8_t port_id); >>>>> int all_ports_stopped(void); >>>>> int port_is_started(portid_t port_id); >>>>> void pmd_test_exit(void); >>>>> @@ -558,10 +569,15 @@ void get_ethertype_filter(uint8_t port_id, uint16_t index); >>>>> void get_2tuple_filter(uint8_t port_id, uint16_t index); >>>>> void get_5tuple_filter(uint8_t port_id, uint16_t index); >>>>> void get_flex_filter(uint8_t port_id, uint16_t index); >>>>> -int port_id_is_invalid(portid_t port_id); >>>>> int rx_queue_id_is_invalid(queueid_t rxq_id); >>>>> int tx_queue_id_is_invalid(queueid_t txq_id); >>>>> >>>>> +enum print_warning { >>>>> + ENABLED_WARN = 0, >>>>> + DISABLED_WARN >>>>> +}; >>>>> +int port_id_is_invalid(portid_t port_id, enum print_warning warning); >>>>> + >>>>> /* >>>>> * Work-around of a compilation error with ICC on invocations of the >>>>> * rte_be_to_cpu_16() function. >>>>> diff --git a/doc/guides/testpmd_app_ug/testpmd_funcs.rst b/doc/guides/testpmd_app_ug/testpmd_funcs.rst >>>>> index 218835a..1cacbcf 100644 >>>>> --- a/doc/guides/testpmd_app_ug/testpmd_funcs.rst >>>>> +++ b/doc/guides/testpmd_app_ug/testpmd_funcs.rst >>>>> @@ -808,6 +808,63 @@ The following sections show functions for configuring ports. >>>>> >>>>> Port configuration changes only become active when forwarding is started/restarted. >>>>> >>>>> +port attach >>>>> +~~~~~~~~~~~ >>>>> + >>>>> +Attach a port specified by pci address or virtual device args. >>>>> + >>>>> +To attach a new pci device, the device should be recognized by kernel first. >>>>> +Then it should be moved under DPDK management. >>>>> +Finally the port can be attached to testpmd. >>>>> +On the other hand, to attach a port created by virtual device, above steps are not needed. >>>>> + >>>>> +port attach (identifier) >>>>> + >>>>> +For example, to attach a port that pci address is 0000:02:00.0. >>>>> + >>>>> +.. code-block:: console >>>>> + >>>>> + testpmd> port attach 0000:02:00.0 >>>>> + Attaching a new port... >>>>> + ... snip ... >>>>> + Port 0 is attached. Now total ports is 1 >>>>> + Done >>>>> + >>>>> +For example, to attach a port created by pcap PMD. >>>>> + >>>>> +.. code-block:: console >>>>> + >>>>> + testpmd> port attach eth_pcap0,iface=eth0 >>>>> + Attaching a new port... >>>>> + ... snip ... >>>>> + Port 0 is attached. Now total ports is 1 >>>>> + Done >>>>> + >>>>> +In this case, identifier is "eth_pcap0,iface=eth0". >>>>> +This identifier format is the same as "--vdev" format of DPDK applications. >>>>> + >>>>> +port detach >>>>> +~~~~~~~~~~~ >>>>> + >>>>> +Detach a specific port. >>>>> + >>>>> +Before detaching a port, the port should be closed. >>>>> +Also to remove a pci device completely from the system, first detach the port from testpmd. >>>>> +Then the device should be moved under kernel management. >>>>> +Finally the device can be remove using kernel pci hotplug functionality. >>>>> +On the other hand, to remove a port created by virtual device, above steps are not needed. >>>>> + >>>>> +port detach (port_id) >>>>> + >>>>> +For example, to detach a port 0. >>>>> + >>>>> +.. code-block:: console >>>>> + >>>>> + testpmd> port detach 0 >>>>> + Detaching a port... >>>>> + ... snip ... >>>>> + Done >>>>> + >>>>> port start >>>>> ~~~~~~~~~~ >>>>> >>
This patch series adds a dynamic port hotplug framework to DPDK. With the patches, DPDK apps can attach or detach ports at runtime. The basic concept of the port hotplug is like followings. - DPDK apps must have responsibility to manage ports. DPDK apps only know which ports are attached or detached at the moment. The port hotplug framework is implemented to allow DPDK apps to manage ports. For example, when DPDK apps call port attach function, attached port number will be returned. Also, DPDK apps can detach port by port number. - Kernel support is needed for attaching or detaching physical device ports. To attach a new physical device port, the device will be recognized by userspace directly I/O framework in kernel at first. Then DPDK apps can call the port hotplug functions to attach ports. For detaching, steps are vice versa. - Before detach ports, ports must be stopped and closed. DPDK application must call rte_eth_dev_stop() and rte_eth_dev_close() before detaching ports. These function will call finalization codes of PMDs. But so far, no PMD frees all resources allocated by initialization. It means PMDs are needed to be fixed to support the port hotplug. 'RTE_PCI_DRV_DETACHABLE' is a new flag indicating a PMD supports detaching. Without this flag, detaching will be failed. - Mustn't affect legacy DPDK apps. No DPDK EAL behavior is changed, if the port hotplug functions are't called. So all legacy DPDK apps can still work without modifications. And a few limitations. - The port hotplug functions are not thread safe. DPDK apps should handle it. - Only support Linux and igb_uio so far. BSD and VFIO is not supported. I will send VFIO patches at least, but I don't have a plan to submit BSD patch so far. Here is port hotplug APIs. ------------------------------------------------------------------------------- /** * Attach a new device. * * @param devargs * A pointer to a strings array describing the new device * to be attached. The strings should be a pci address like * '0000:01:00.0' or virtual device name like 'eth_pcap0'. * @param port_id * A pointer to a port identifier actually attached. * @return * 0 on success and port_id is filled, negative on error */ int rte_eal_dev_attach(const char *devargs, uint8_t *port_id); /** * Detach a device. * * @param port_id * The port identifier of the device to detach. * @param addr * A pointer to a device name actually detached. * @return * 0 on success and devname is filled, negative on error */ int rte_eal_dev_detach(uint8_t port_id, char *devname); ------------------------------------------------------------------------------- This patch series are for DPDK EAL. To use port hotplug function by DPDK apps, each PMD should be fixed to support 'RTE_PCI_DRV_DETACHABLE' flag. Please check a patch for pcap PMD. Also, please check testpmd patch. It will show you how to fix your legacy applications to support port hotplug feature. PATCH v7 changes - Add a new section to programmer's guide. (Thanks to Iremonger, Bernard) - Fix port checking implementation of star_port(). - Fix typo of warning messages. - Add pt_driver checking to rte_eth_dev_check_detachable(). (Thanks to Qiu, Michael) PATCH v6 changes - Fix rte_eth_dev_uninit() to handle a return value of uninit function of PMD. To do this, below changes also be applied. - Fix a parameter of rte_eth_dev_free(). - Use rte_eth_dev structure as the paramter of rte_eth_dev_free(). PATCH v5 changes - Add runtime check passthrough driver type, like vfio-pci, igb_uio and uio_pci_generic. This was done by Qiu, Michael. Thanks a lot. - Change function names like below. - rte_eal_dev_find_and_invoke() to rte_eal_vdev_find_and_invoke(). - rte_eal_dev_invoke() to rte_eal_vdev_invoke(). - Add code to handle a return value of rte_eal_devargs_remove(). - Fix pci address format in rte_eal_dev_detach(). - Remove RTE_EAL_INVOKE_TYPE_UNKNOWN, because it's unused. - Change function definition of rte_eal_devargs_remove(). - Fix pci_unmap_device() to check pt_driver. - Fix return value of below functions. - rte_eth_dev_get_changed_port(). - rte_eth_dev_get_port_by_addr(). - Change paramters of rte_eth_dev_validate_port() to cleanup code. - Fix pci_scan_one to handle pt_driver correctly. (Thanks to Qiu, Michael for above suggestions) PATCH v4 changes - Merge patches to review easier. - Fix indent of 'if' statement. - Fix calculation method of eal_compare_pci_addr(). - Fix header file declaration. - Add header file to determine if hotplug can be enabled. (Thanks to Qiu, Michael) - Use braces with 'for' loop. - Add parameter checking. - Fix sanity check code - Fix comments of rte_eth_dev_type. - Change function names. (Thanks to Iremonger, Bernard) PATCH v3 changes: - Fix enum definition used in rte_ethdev.c. (Thanks to Zhang, Helin) PATCH v2 changes: - Replace rte_eal_dev_attach_pdev(), rte_eal_dev_detach_pdev, rte_eal_dev_attach_vdev() and rte_eal_dev_detach_vdev() to rte_eal_dev_attach() and rte_eal_dev_detach(). - Add parameter values checking. - Refashion a few functions. (Thanks to Iremonger, Bernard) PATCH v1 Changes: - Fix error checking code of librte_eth APIs. - Fix issue that port from pcap PMD cannot be detached correctly. - Fix issue that testpmd could hang after forwarding, if attaching and detaching is repeatedly. - Fix if-condition of rte_eth_dev_get_port_by_addr(). (Thanks to Mark Enright) RFC PATCH v2 Changes: - remove 'rte_eth_dev_validate_port()', and cleanup codes. Michael Qiu (2): eal_pci: Add flag to hold kernel driver type eal_pci: pci memory map work with driver type Tetsuya Mukawa (12): eal/pci,ethdev: Remove assumption that port will not be detached eal/pci: Consolidate pci address comparison APIs ethdev: Add rte_eth_dev_free to free specified device eal,ethdev: Add a function and function pointers to close ether device ethdev: Add functions that will be used by port hotplug functions eal/linux/pci: Add functions for unmapping igb_uio resources eal/pci: Add a function to remove the entry of devargs list eal/pci: Cleanup pci driver initialization code ethdev: Add one dev_type parameter to rte_eth_dev_allocate eal/pci: Add rte_eal_dev_attach/detach() functions eal: Enable port hotplug framework in Linux doc: Add port hotplug framework section to programmers guide app/test/virtual_pmd.c | 2 +- config/common_linuxapp | 5 + doc/guides/prog_guide/index.rst | 1 + doc/guides/prog_guide/port_hotplug_framework.rst | 110 ++++ lib/librte_eal/bsdapp/eal/eal_pci.c | 25 +- lib/librte_eal/common/Makefile | 1 + lib/librte_eal/common/eal_common_dev.c | 274 ++++++++++ lib/librte_eal/common/eal_common_devargs.c | 60 +++ lib/librte_eal/common/eal_common_pci.c | 92 +++- lib/librte_eal/common/eal_private.h | 35 ++ lib/librte_eal/common/include/rte_dev.h | 33 ++ lib/librte_eal/common/include/rte_dev_hotplug.h | 44 ++ lib/librte_eal/common/include/rte_devargs.h | 21 + lib/librte_eal/common/include/rte_pci.h | 84 +++ lib/librte_eal/linuxapp/eal/Makefile | 1 + lib/librte_eal/linuxapp/eal/eal_pci.c | 227 ++++++-- lib/librte_eal/linuxapp/eal/eal_pci_init.h | 8 + lib/librte_eal/linuxapp/eal/eal_pci_uio.c | 67 ++- lib/librte_ether/rte_ethdev.c | 635 +++++++++++++---------- lib/librte_ether/rte_ethdev.h | 148 +++++- lib/librte_pmd_af_packet/rte_eth_af_packet.c | 2 +- lib/librte_pmd_bond/rte_eth_bond_api.c | 2 +- lib/librte_pmd_pcap/rte_eth_pcap.c | 2 +- lib/librte_pmd_ring/rte_eth_ring.c | 2 +- lib/librte_pmd_xenvirt/rte_eth_xenvirt.c | 2 +- 25 files changed, 1536 insertions(+), 347 deletions(-) create mode 100644 doc/guides/prog_guide/port_hotplug_framework.rst create mode 100644 lib/librte_eal/common/include/rte_dev_hotplug.h -- 1.9.1
From: Michael Qiu <michael.qiu@intel.com> Currently, dpdk has no ability to know which type of driver( vfio-pci/igb_uio/uio_pci_generic) the device used. It only can check whether vfio is enabled or not staticly. It really useful to have the flag, becasue different type need to handle differently in runtime. For example, pci memory map, pot hotplug, and so on. This patch add a flag field for pci device to solve above issue. Signed-off-by: Michael Qiu <michael.qiu@intel.com> Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp> --- lib/librte_eal/common/include/rte_pci.h | 8 +++++ lib/librte_eal/linuxapp/eal/eal_pci.c | 53 +++++++++++++++++++++++++++++++-- 2 files changed, 59 insertions(+), 2 deletions(-) diff --git a/lib/librte_eal/common/include/rte_pci.h b/lib/librte_eal/common/include/rte_pci.h index 66ed793..7b48b55 100644 --- a/lib/librte_eal/common/include/rte_pci.h +++ b/lib/librte_eal/common/include/rte_pci.h @@ -139,6 +139,13 @@ struct rte_pci_addr { struct rte_devargs; +enum rte_pt_driver { + RTE_PT_UNKNOWN = 0, + RTE_PT_IGB_UIO = 1, + RTE_PT_VFIO = 2, + RTE_PT_UIO_GENERIC = 3, +}; + /** * A structure describing a PCI device. */ @@ -152,6 +159,7 @@ struct rte_pci_device { uint16_t max_vfs; /**< sriov enable if not zero */ int numa_node; /**< NUMA node connection */ struct rte_devargs *devargs; /**< Device user arguments */ + enum rte_pt_driver pt_driver; /**< Driver of passthrough */ }; /** Any PCI device identifier (vendor, device, ...) */ diff --git a/lib/librte_eal/linuxapp/eal/eal_pci.c b/lib/librte_eal/linuxapp/eal/eal_pci.c index b5f5410..bd3f77d 100644 --- a/lib/librte_eal/linuxapp/eal/eal_pci.c +++ b/lib/librte_eal/linuxapp/eal/eal_pci.c @@ -97,6 +97,35 @@ error: return -1; } +static int +pci_get_kernel_driver_by_path(const char *filename, char *dri_name) +{ + int count; + char path[PATH_MAX]; + char *name; + + if (!filename || !dri_name) + return -1; + + count = readlink(filename, path, PATH_MAX); + if (count >= PATH_MAX) + return -1; + + /* For device does not have a driver */ + if (count < 0) + return 1; + + path[count] = '\0'; + + name = strrchr(path, '/'); + if (name) { + strncpy(dri_name, name + 1, strlen(name + 1) + 1); + return 0; + } + + return -1; +} + void * pci_find_max_end_va(void) { @@ -222,11 +251,12 @@ pci_scan_one(const char *dirname, uint16_t domain, uint8_t bus, char filename[PATH_MAX]; unsigned long tmp; struct rte_pci_device *dev; + char driver[PATH_MAX]; + int ret; dev = malloc(sizeof(*dev)); - if (dev == NULL) { + if (dev == NULL) return -1; - } memset(dev, 0, sizeof(*dev)); dev->addr.domain = domain; @@ -298,6 +328,25 @@ pci_scan_one(const char *dirname, uint16_t domain, uint8_t bus, return -1; } + /* parse driver */ + snprintf(filename, sizeof(filename), "%s/driver", dirname); + ret = pci_get_kernel_driver_by_path(filename, driver); + if (!ret) { + if (!strcmp(driver, "vfio-pci")) + dev->pt_driver = RTE_PT_VFIO; + else if (!strcmp(driver, "igb_uio")) + dev->pt_driver = RTE_PT_IGB_UIO; + else if (!strcmp(driver, "uio_pci_generic")) + dev->pt_driver = RTE_PT_UIO_GENERIC; + else + dev->pt_driver = RTE_PT_UNKNOWN; + } else if (ret < 0) { + RTE_LOG(ERR, EAL, "Fail to get kernel driver\n"); + free(dev); + return -1; + } else + dev->pt_driver = RTE_PT_UNKNOWN; + /* device is valid, add in list (sorted) */ if (TAILQ_EMPTY(&pci_device_list)) { TAILQ_INSERT_TAIL(&pci_device_list, dev, next); -- 1.9.1
From: Michael Qiu <michael.qiu@intel.com> With the driver type flag in struct rte_pci_dev, we do not need to always map uio devices with vfio related function when vfio enabled. Signed-off-by: Michael Qiu <michael.qiu@intel.com> Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp> --- lib/librte_eal/linuxapp/eal/eal_pci.c | 30 +++++++++++++++++------------- 1 file changed, 17 insertions(+), 13 deletions(-) diff --git a/lib/librte_eal/linuxapp/eal/eal_pci.c b/lib/librte_eal/linuxapp/eal/eal_pci.c index bd3f77d..c0ca5a5 100644 --- a/lib/librte_eal/linuxapp/eal/eal_pci.c +++ b/lib/librte_eal/linuxapp/eal/eal_pci.c @@ -549,25 +549,29 @@ pci_config_space_set(struct rte_pci_device *dev) static int pci_map_device(struct rte_pci_device *dev) { - int ret, mapped = 0; + int ret = -1; /* try mapping the NIC resources using VFIO if it exists */ + switch (dev->pt_driver) { + case RTE_PT_VFIO: #ifdef VFIO_PRESENT - if (pci_vfio_is_enabled()) { - ret = pci_vfio_map_resource(dev); - if (ret == 0) - mapped = 1; - else if (ret < 0) - return ret; - } + if (pci_vfio_is_enabled()) + ret = pci_vfio_map_resource(dev); #endif - /* map resources for devices that use igb_uio */ - if (!mapped) { + break; + case RTE_PT_IGB_UIO: + case RTE_PT_UIO_GENERIC: + /* map resources for devices that use uio */ ret = pci_uio_map_resource(dev); - if (ret != 0) - return ret; + break; + default: + RTE_LOG(DEBUG, EAL, " Not managed by known pt driver," + " skipped\n"); + ret = 1; + break; } - return 0; + + return ret; } /* -- 1.9.1
To remove assumption, do like followings. This patch adds "RTE_PCI_DRV_DETACHABLE" to drv_flags of rte_pci_driver structure. The flags indicate the driver can detach devices at runtime. Also, remove assumption that port will not be detached. To remove the assumption. - Add 'attached' member to rte_eth_dev structure. This member is used for indicating the port is attached, or not. - Add rte_eth_dev_allocate_new_port(). This function is used for allocating new port. v5: - Change parameters of rte_eth_dev_validate_port() to cleanup code. v4: - Use braces with 'for' loop. - Fix indent of 'if' statement. Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp> --- lib/librte_eal/common/include/rte_pci.h | 2 + lib/librte_ether/rte_ethdev.c | 454 +++++++++++++------------------- lib/librte_ether/rte_ethdev.h | 5 + 3 files changed, 186 insertions(+), 275 deletions(-) diff --git a/lib/librte_eal/common/include/rte_pci.h b/lib/librte_eal/common/include/rte_pci.h index 7b48b55..7f2d699 100644 --- a/lib/librte_eal/common/include/rte_pci.h +++ b/lib/librte_eal/common/include/rte_pci.h @@ -207,6 +207,8 @@ struct rte_pci_driver { #define RTE_PCI_DRV_FORCE_UNBIND 0x0004 /** Device driver supports link state interrupt */ #define RTE_PCI_DRV_INTR_LSC 0x0008 +/** Device driver supports detaching capability */ +#define RTE_PCI_DRV_DETACHABLE 0x0010 /**< Internal use only - Macro used by pci addr parsing functions **/ #define GET_PCIADDR_FIELD(in, fd, lim, dlm) \ diff --git a/lib/librte_ether/rte_ethdev.c b/lib/librte_ether/rte_ethdev.c index ea3a1fb..d70854f 100644 --- a/lib/librte_ether/rte_ethdev.c +++ b/lib/librte_ether/rte_ethdev.c @@ -175,6 +175,16 @@ enum { STAT_QMAP_RX }; +enum { + DEV_INVALID = 0, + DEV_VALID, +}; + +enum { + DEV_DISCONNECTED = 0, + DEV_CONNECTED +}; + static inline void rte_eth_dev_data_alloc(void) { @@ -201,19 +211,34 @@ rte_eth_dev_allocated(const char *name) { unsigned i; - for (i = 0; i < nb_ports; i++) { - if (strcmp(rte_eth_devices[i].data->name, name) == 0) + for (i = 0; i < RTE_MAX_ETHPORTS; i++) { + if ((rte_eth_devices[i].attached == DEV_CONNECTED) && + strcmp(rte_eth_devices[i].data->name, name) == 0) return &rte_eth_devices[i]; } return NULL; } +static uint8_t +rte_eth_dev_allocate_new_port(void) +{ + unsigned i; + + for (i = 0; i < RTE_MAX_ETHPORTS; i++) { + if (rte_eth_devices[i].attached == DEV_DISCONNECTED) + return i; + } + return RTE_MAX_ETHPORTS; +} + struct rte_eth_dev * rte_eth_dev_allocate(const char *name) { + uint8_t port_id; struct rte_eth_dev *eth_dev; - if (nb_ports == RTE_MAX_ETHPORTS) { + port_id = rte_eth_dev_allocate_new_port(); + if (port_id == RTE_MAX_ETHPORTS) { PMD_DEBUG_TRACE("Reached maximum number of Ethernet ports\n"); return NULL; } @@ -226,10 +251,12 @@ rte_eth_dev_allocate(const char *name) return NULL; } - eth_dev = &rte_eth_devices[nb_ports]; - eth_dev->data = &rte_eth_dev_data[nb_ports]; + eth_dev = &rte_eth_devices[port_id]; + eth_dev->data = &rte_eth_dev_data[port_id]; snprintf(eth_dev->data->name, sizeof(eth_dev->data->name), "%s", name); - eth_dev->data->port_id = nb_ports++; + eth_dev->data->port_id = port_id; + eth_dev->attached = DEV_CONNECTED; + nb_ports++; return eth_dev; } @@ -283,6 +310,7 @@ rte_eth_dev_init(struct rte_pci_driver *pci_drv, (unsigned) pci_dev->id.device_id); if (rte_eal_process_type() == RTE_PROC_PRIMARY) rte_free(eth_dev->data->dev_private); + eth_dev->attached = DEV_DISCONNECTED; nb_ports--; return diag; } @@ -308,10 +336,28 @@ rte_eth_driver_register(struct eth_driver *eth_drv) rte_eal_pci_register(ð_drv->pci_drv); } +enum { + NONE_TRACE = 0, + TRACE +}; + +static int +rte_eth_dev_validate_port(uint8_t port_id, int trace) +{ + if (port_id >= RTE_MAX_ETHPORTS || + rte_eth_devices[port_id].attached != DEV_CONNECTED) { + if (trace) { + PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + } + return DEV_INVALID; + } else + return DEV_VALID; +} + int rte_eth_dev_socket_id(uint8_t port_id) { - if (port_id >= nb_ports) + if (rte_eth_dev_validate_port(port_id, NONE_TRACE) == DEV_INVALID) return -1; return rte_eth_devices[port_id].pci_dev->numa_node; } @@ -369,10 +415,8 @@ rte_eth_dev_rx_queue_start(uint8_t port_id, uint16_t rx_queue_id) * in a multi-process setup*/ PROC_PRIMARY_OR_ERR_RET(-E_RTE_SECONDARY); - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return -EINVAL; - } dev = &rte_eth_devices[port_id]; if (rx_queue_id >= dev->data->nb_rx_queues) { @@ -395,10 +439,8 @@ rte_eth_dev_rx_queue_stop(uint8_t port_id, uint16_t rx_queue_id) * in a multi-process setup*/ PROC_PRIMARY_OR_ERR_RET(-E_RTE_SECONDARY); - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return -EINVAL; - } dev = &rte_eth_devices[port_id]; if (rx_queue_id >= dev->data->nb_rx_queues) { @@ -421,10 +463,8 @@ rte_eth_dev_tx_queue_start(uint8_t port_id, uint16_t tx_queue_id) * in a multi-process setup*/ PROC_PRIMARY_OR_ERR_RET(-E_RTE_SECONDARY); - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return -EINVAL; - } dev = &rte_eth_devices[port_id]; if (tx_queue_id >= dev->data->nb_tx_queues) { @@ -447,10 +487,8 @@ rte_eth_dev_tx_queue_stop(uint8_t port_id, uint16_t tx_queue_id) * in a multi-process setup*/ PROC_PRIMARY_OR_ERR_RET(-E_RTE_SECONDARY); - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return -EINVAL; - } dev = &rte_eth_devices[port_id]; if (tx_queue_id >= dev->data->nb_tx_queues) { @@ -703,10 +741,9 @@ rte_eth_dev_configure(uint8_t port_id, uint16_t nb_rx_q, uint16_t nb_tx_q, * in a multi-process setup*/ PROC_PRIMARY_OR_ERR_RET(-E_RTE_SECONDARY); - if (port_id >= nb_ports || port_id >= RTE_MAX_ETHPORTS) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return (-EINVAL); - } + dev = &rte_eth_devices[port_id]; FUNC_PTR_OR_ERR_RET(*dev->dev_ops->dev_infos_get, -ENOTSUP); @@ -888,10 +925,9 @@ rte_eth_dev_start(uint8_t port_id) * in a multi-process setup*/ PROC_PRIMARY_OR_ERR_RET(-E_RTE_SECONDARY); - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%" PRIu8 "\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return (-EINVAL); - } + dev = &rte_eth_devices[port_id]; FUNC_PTR_OR_ERR_RET(*dev->dev_ops->dev_start, -ENOTSUP); @@ -923,10 +959,9 @@ rte_eth_dev_stop(uint8_t port_id) * in a multi-process setup*/ PROC_PRIMARY_OR_RET(); - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%" PRIu8 "\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return; - } + dev = &rte_eth_devices[port_id]; FUNC_PTR_OR_RET(*dev->dev_ops->dev_stop); @@ -951,10 +986,9 @@ rte_eth_dev_set_link_up(uint8_t port_id) * in a multi-process setup*/ PROC_PRIMARY_OR_ERR_RET(-E_RTE_SECONDARY); - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return -EINVAL; - } + dev = &rte_eth_devices[port_id]; FUNC_PTR_OR_ERR_RET(*dev->dev_ops->dev_set_link_up, -ENOTSUP); @@ -970,10 +1004,9 @@ rte_eth_dev_set_link_down(uint8_t port_id) * in a multi-process setup*/ PROC_PRIMARY_OR_ERR_RET(-E_RTE_SECONDARY); - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return -EINVAL; - } + dev = &rte_eth_devices[port_id]; FUNC_PTR_OR_ERR_RET(*dev->dev_ops->dev_set_link_down, -ENOTSUP); @@ -989,10 +1022,8 @@ rte_eth_dev_close(uint8_t port_id) * in a multi-process setup*/ PROC_PRIMARY_OR_RET(); - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return; - } dev = &rte_eth_devices[port_id]; @@ -1017,10 +1048,9 @@ rte_eth_rx_queue_setup(uint8_t port_id, uint16_t rx_queue_id, * in a multi-process setup*/ PROC_PRIMARY_OR_ERR_RET(-E_RTE_SECONDARY); - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return (-EINVAL); - } + dev = &rte_eth_devices[port_id]; if (rx_queue_id >= dev->data->nb_rx_queues) { PMD_DEBUG_TRACE("Invalid RX queue_id=%d\n", rx_queue_id); @@ -1090,10 +1120,9 @@ rte_eth_tx_queue_setup(uint8_t port_id, uint16_t tx_queue_id, * in a multi-process setup*/ PROC_PRIMARY_OR_ERR_RET(-E_RTE_SECONDARY); - if (port_id >= RTE_MAX_ETHPORTS || port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return (-EINVAL); - } + dev = &rte_eth_devices[port_id]; if (tx_queue_id >= dev->data->nb_tx_queues) { PMD_DEBUG_TRACE("Invalid TX queue_id=%d\n", tx_queue_id); @@ -1123,10 +1152,9 @@ rte_eth_promiscuous_enable(uint8_t port_id) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return; - } + dev = &rte_eth_devices[port_id]; FUNC_PTR_OR_RET(*dev->dev_ops->promiscuous_enable); @@ -1139,10 +1167,9 @@ rte_eth_promiscuous_disable(uint8_t port_id) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return; - } + dev = &rte_eth_devices[port_id]; FUNC_PTR_OR_RET(*dev->dev_ops->promiscuous_disable); @@ -1155,10 +1182,8 @@ rte_eth_promiscuous_get(uint8_t port_id) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return -1; - } dev = &rte_eth_devices[port_id]; return dev->data->promiscuous; @@ -1169,10 +1194,9 @@ rte_eth_allmulticast_enable(uint8_t port_id) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return; - } + dev = &rte_eth_devices[port_id]; FUNC_PTR_OR_RET(*dev->dev_ops->allmulticast_enable); @@ -1185,10 +1209,9 @@ rte_eth_allmulticast_disable(uint8_t port_id) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return; - } + dev = &rte_eth_devices[port_id]; FUNC_PTR_OR_RET(*dev->dev_ops->allmulticast_disable); @@ -1201,10 +1224,8 @@ rte_eth_allmulticast_get(uint8_t port_id) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return -1; - } dev = &rte_eth_devices[port_id]; return dev->data->all_multicast; @@ -1229,10 +1250,9 @@ rte_eth_link_get(uint8_t port_id, struct rte_eth_link *eth_link) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return; - } + dev = &rte_eth_devices[port_id]; if (dev->data->dev_conf.intr_conf.lsc != 0) @@ -1249,10 +1269,9 @@ rte_eth_link_get_nowait(uint8_t port_id, struct rte_eth_link *eth_link) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return; - } + dev = &rte_eth_devices[port_id]; if (dev->data->dev_conf.intr_conf.lsc != 0) @@ -1269,10 +1288,9 @@ rte_eth_stats_get(uint8_t port_id, struct rte_eth_stats *stats) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return; - } + dev = &rte_eth_devices[port_id]; memset(stats, 0, sizeof(*stats)); @@ -1286,10 +1304,9 @@ rte_eth_stats_reset(uint8_t port_id) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return; - } + dev = &rte_eth_devices[port_id]; FUNC_PTR_OR_RET(*dev->dev_ops->stats_reset); @@ -1307,10 +1324,9 @@ rte_eth_xstats_get(uint8_t port_id, struct rte_eth_xstats *xstats, uint64_t val; char *stats_ptr; - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return -1; - } + dev = &rte_eth_devices[port_id]; /* implemented by the driver */ @@ -1376,10 +1392,9 @@ rte_eth_xstats_reset(uint8_t port_id) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return; - } + dev = &rte_eth_devices[port_id]; /* implemented by the driver */ @@ -1398,10 +1413,9 @@ set_queue_stats_mapping(uint8_t port_id, uint16_t queue_id, uint8_t stat_idx, { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return -ENODEV; - } + dev = &rte_eth_devices[port_id]; FUNC_PTR_OR_ERR_RET(*dev->dev_ops->queue_stats_mapping_set, -ENOTSUP); @@ -1433,10 +1447,9 @@ rte_eth_dev_info_get(uint8_t port_id, struct rte_eth_dev_info *dev_info) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return; - } + dev = &rte_eth_devices[port_id]; memset(dev_info, 0, sizeof(struct rte_eth_dev_info)); @@ -1453,10 +1466,9 @@ rte_eth_macaddr_get(uint8_t port_id, struct ether_addr *mac_addr) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return; - } + dev = &rte_eth_devices[port_id]; ether_addr_copy(&dev->data->mac_addrs[0], mac_addr); } @@ -1467,10 +1479,8 @@ rte_eth_dev_get_mtu(uint8_t port_id, uint16_t *mtu) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return (-ENODEV); - } dev = &rte_eth_devices[port_id]; *mtu = dev->data->mtu; @@ -1483,10 +1493,8 @@ rte_eth_dev_set_mtu(uint8_t port_id, uint16_t mtu) int ret; struct rte_eth_dev *dev; - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return (-ENODEV); - } dev = &rte_eth_devices[port_id]; FUNC_PTR_OR_ERR_RET(*dev->dev_ops->mtu_set, -ENOTSUP); @@ -1503,10 +1511,9 @@ rte_eth_dev_vlan_filter(uint8_t port_id, uint16_t vlan_id, int on) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return (-ENODEV); - } + dev = &rte_eth_devices[port_id]; if (! (dev->data->dev_conf.rxmode.hw_vlan_filter)) { PMD_DEBUG_TRACE("port %d: vlan-filtering disabled\n", port_id); @@ -1528,10 +1535,8 @@ rte_eth_dev_set_vlan_strip_on_queue(uint8_t port_id, uint16_t rx_queue_id, int o { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return (-ENODEV); - } dev = &rte_eth_devices[port_id]; if (rx_queue_id >= dev->data->nb_rx_queues) { @@ -1550,10 +1555,8 @@ rte_eth_dev_set_vlan_ether_type(uint8_t port_id, uint16_t tpid) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return (-ENODEV); - } dev = &rte_eth_devices[port_id]; FUNC_PTR_OR_ERR_RET(*dev->dev_ops->vlan_tpid_set, -ENOTSUP); @@ -1570,10 +1573,8 @@ rte_eth_dev_set_vlan_offload(uint8_t port_id, int offload_mask) int mask = 0; int cur, org = 0; - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return (-ENODEV); - } dev = &rte_eth_devices[port_id]; @@ -1615,10 +1616,8 @@ rte_eth_dev_get_vlan_offload(uint8_t port_id) struct rte_eth_dev *dev; int ret = 0; - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return (-ENODEV); - } dev = &rte_eth_devices[port_id]; @@ -1639,10 +1638,9 @@ rte_eth_dev_set_vlan_pvid(uint8_t port_id, uint16_t pvid, int on) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return (-ENODEV); - } + dev = &rte_eth_devices[port_id]; FUNC_PTR_OR_ERR_RET(*dev->dev_ops->vlan_pvid_set, -ENOTSUP); (*dev->dev_ops->vlan_pvid_set)(dev, pvid, on); @@ -1657,10 +1655,8 @@ rte_eth_dev_fdir_add_signature_filter(uint8_t port_id, { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return (-ENODEV); - } dev = &rte_eth_devices[port_id]; @@ -1691,10 +1687,8 @@ rte_eth_dev_fdir_update_signature_filter(uint8_t port_id, { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return (-ENODEV); - } dev = &rte_eth_devices[port_id]; @@ -1725,10 +1719,8 @@ rte_eth_dev_fdir_remove_signature_filter(uint8_t port_id, { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return (-ENODEV); - } dev = &rte_eth_devices[port_id]; @@ -1756,10 +1748,8 @@ rte_eth_dev_fdir_get_infos(uint8_t port_id, struct rte_eth_fdir *fdir) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return (-ENODEV); - } dev = &rte_eth_devices[port_id]; if (! (dev->data->dev_conf.fdir_conf.mode)) { @@ -1781,10 +1771,8 @@ rte_eth_dev_fdir_add_perfect_filter(uint8_t port_id, { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return (-ENODEV); - } dev = &rte_eth_devices[port_id]; @@ -1821,10 +1809,8 @@ rte_eth_dev_fdir_update_perfect_filter(uint8_t port_id, { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return (-ENODEV); - } dev = &rte_eth_devices[port_id]; @@ -1859,10 +1845,8 @@ rte_eth_dev_fdir_remove_perfect_filter(uint8_t port_id, { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return (-ENODEV); - } dev = &rte_eth_devices[port_id]; @@ -1895,10 +1879,8 @@ rte_eth_dev_fdir_set_masks(uint8_t port_id, struct rte_fdir_masks *fdir_mask) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return (-ENODEV); - } dev = &rte_eth_devices[port_id]; if (! (dev->data->dev_conf.fdir_conf.mode)) { @@ -1915,10 +1897,8 @@ rte_eth_dev_flow_ctrl_get(uint8_t port_id, struct rte_eth_fc_conf *fc_conf) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return (-ENODEV); - } dev = &rte_eth_devices[port_id]; FUNC_PTR_OR_ERR_RET(*dev->dev_ops->flow_ctrl_get, -ENOTSUP); @@ -1931,10 +1911,8 @@ rte_eth_dev_flow_ctrl_set(uint8_t port_id, struct rte_eth_fc_conf *fc_conf) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return (-ENODEV); - } if ((fc_conf->send_xon != 0) && (fc_conf->send_xon != 1)) { PMD_DEBUG_TRACE("Invalid send_xon, only 0/1 allowed\n"); @@ -1951,10 +1929,8 @@ rte_eth_dev_priority_flow_ctrl_set(uint8_t port_id, struct rte_eth_pfc_conf *pfc { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return (-ENODEV); - } if (pfc_conf->priority > (ETH_DCB_NUM_USER_PRIORITIES - 1)) { PMD_DEBUG_TRACE("Invalid priority, only 0-7 allowed\n"); @@ -2030,10 +2006,8 @@ rte_eth_dev_rss_reta_update(uint8_t port_id, struct rte_eth_dev *dev; int ret; - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return -ENODEV; - } /* Check mask bits */ ret = rte_eth_check_reta_mask(reta_conf, reta_size); @@ -2081,10 +2055,9 @@ rte_eth_dev_rss_hash_update(uint8_t port_id, struct rte_eth_rss_conf *rss_conf) struct rte_eth_dev *dev; uint16_t rss_hash_protos; - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return (-ENODEV); - } + rss_hash_protos = rss_conf->rss_hf; if ((rss_hash_protos != 0) && ((rss_hash_protos & ETH_RSS_PROTO_MASK) == 0)) { @@ -2103,10 +2076,9 @@ rte_eth_dev_rss_hash_conf_get(uint8_t port_id, { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return (-ENODEV); - } + dev = &rte_eth_devices[port_id]; FUNC_PTR_OR_ERR_RET(*dev->dev_ops->rss_hash_conf_get, -ENOTSUP); return (*dev->dev_ops->rss_hash_conf_get)(dev, rss_conf); @@ -2118,10 +2090,8 @@ rte_eth_dev_udp_tunnel_add(uint8_t port_id, { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return -ENODEV; - } if (udp_tunnel == NULL) { PMD_DEBUG_TRACE("Invalid udp_tunnel parameter\n"); @@ -2144,10 +2114,9 @@ rte_eth_dev_udp_tunnel_delete(uint8_t port_id, { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return -ENODEV; - } + dev = &rte_eth_devices[port_id]; if (udp_tunnel == NULL) { @@ -2169,10 +2138,8 @@ rte_eth_led_on(uint8_t port_id) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return (-ENODEV); - } dev = &rte_eth_devices[port_id]; FUNC_PTR_OR_ERR_RET(*dev->dev_ops->dev_led_on, -ENOTSUP); @@ -2184,10 +2151,8 @@ rte_eth_led_off(uint8_t port_id) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return (-ENODEV); - } dev = &rte_eth_devices[port_id]; FUNC_PTR_OR_ERR_RET(*dev->dev_ops->dev_led_off, -ENOTSUP); @@ -2224,10 +2189,9 @@ rte_eth_dev_mac_addr_add(uint8_t port_id, struct ether_addr *addr, int index; uint64_t pool_mask; - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return (-ENODEV); - } + dev = &rte_eth_devices[port_id]; FUNC_PTR_OR_ERR_RET(*dev->dev_ops->mac_addr_add, -ENOTSUP); @@ -2275,10 +2239,9 @@ rte_eth_dev_mac_addr_remove(uint8_t port_id, struct ether_addr *addr) struct rte_eth_dev *dev; int index; - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return (-ENODEV); - } + dev = &rte_eth_devices[port_id]; FUNC_PTR_OR_ERR_RET(*dev->dev_ops->mac_addr_remove, -ENOTSUP); @@ -2309,7 +2272,7 @@ rte_eth_dev_set_vf_rxmode(uint8_t port_id, uint16_t vf, struct rte_eth_dev *dev; struct rte_eth_dev_info dev_info; - if (port_id >= nb_ports) { + if (rte_eth_dev_validate_port(port_id, NONE_TRACE) == DEV_INVALID) { PMD_DEBUG_TRACE("set VF RX mode:Invalid port_id=%d\n", port_id); return (-ENODEV); @@ -2364,7 +2327,7 @@ rte_eth_dev_uc_hash_table_set(uint8_t port_id, struct ether_addr *addr, int ret; struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (rte_eth_dev_validate_port(port_id, NONE_TRACE) == DEV_INVALID) { PMD_DEBUG_TRACE("unicast hash setting:Invalid port_id=%d\n", port_id); return (-ENODEV); @@ -2417,7 +2380,7 @@ rte_eth_dev_uc_all_hash_table_set(uint8_t port_id, uint8_t on) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (rte_eth_dev_validate_port(port_id, NONE_TRACE) == DEV_INVALID) { PMD_DEBUG_TRACE("unicast hash setting:Invalid port_id=%d\n", port_id); return (-ENODEV); @@ -2436,10 +2399,8 @@ rte_eth_dev_set_vf_rx(uint8_t port_id,uint16_t vf, uint8_t on) struct rte_eth_dev *dev; struct rte_eth_dev_info dev_info; - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return (-ENODEV); - } dev = &rte_eth_devices[port_id]; rte_eth_dev_info_get(port_id, &dev_info); @@ -2462,7 +2423,7 @@ rte_eth_dev_set_vf_tx(uint8_t port_id,uint16_t vf, uint8_t on) struct rte_eth_dev *dev; struct rte_eth_dev_info dev_info; - if (port_id >= nb_ports) { + if (rte_eth_dev_validate_port(port_id, NONE_TRACE) == DEV_INVALID) { PMD_DEBUG_TRACE("set pool tx:Invalid port_id=%d\n", port_id); return (-ENODEV); } @@ -2487,7 +2448,7 @@ rte_eth_dev_set_vf_vlan_filter(uint8_t port_id, uint16_t vlan_id, { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (rte_eth_dev_validate_port(port_id, NONE_TRACE) == DEV_INVALID) { PMD_DEBUG_TRACE("VF VLAN filter:invalid port id=%d\n", port_id); return (-ENODEV); @@ -2518,7 +2479,7 @@ int rte_eth_set_queue_rate_limit(uint8_t port_id, uint16_t queue_idx, struct rte_eth_dev_info dev_info; struct rte_eth_link link; - if (port_id >= nb_ports) { + if (rte_eth_dev_validate_port(port_id, NONE_TRACE) == DEV_INVALID) { PMD_DEBUG_TRACE("set queue rate limit:invalid port id=%d\n", port_id); return -ENODEV; @@ -2555,7 +2516,7 @@ int rte_eth_set_vf_rate_limit(uint8_t port_id, uint16_t vf, uint16_t tx_rate, if (q_msk == 0) return 0; - if (port_id >= nb_ports) { + if (rte_eth_dev_validate_port(port_id, NONE_TRACE) == DEV_INVALID) { PMD_DEBUG_TRACE("set VF rate limit:invalid port id=%d\n", port_id); return -ENODEV; @@ -2589,10 +2550,8 @@ rte_eth_mirror_rule_set(uint8_t port_id, { struct rte_eth_dev *dev = &rte_eth_devices[port_id]; - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return (-ENODEV); - } if (mirror_conf->rule_type_mask == 0) { PMD_DEBUG_TRACE("mirror rule type can not be 0.\n"); @@ -2630,10 +2589,8 @@ rte_eth_mirror_rule_reset(uint8_t port_id, uint8_t rule_id) { struct rte_eth_dev *dev = &rte_eth_devices[port_id]; - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return (-ENODEV); - } if(rule_id >= ETH_VMDQ_NUM_MIRROR_RULE) { @@ -2655,10 +2612,9 @@ rte_eth_rx_burst(uint8_t port_id, uint16_t queue_id, { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return 0; - } + dev = &rte_eth_devices[port_id]; FUNC_PTR_OR_ERR_RET(*dev->rx_pkt_burst, 0); if (queue_id >= dev->data->nb_rx_queues) { @@ -2675,10 +2631,9 @@ rte_eth_tx_burst(uint8_t port_id, uint16_t queue_id, { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return 0; - } + dev = &rte_eth_devices[port_id]; FUNC_PTR_OR_ERR_RET(*dev->tx_pkt_burst, 0); @@ -2695,10 +2650,9 @@ rte_eth_rx_queue_count(uint8_t port_id, uint16_t queue_id) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return 0; - } + dev = &rte_eth_devices[port_id]; FUNC_PTR_OR_ERR_RET(*dev->dev_ops->rx_queue_count, 0); return (*dev->dev_ops->rx_queue_count)(dev, queue_id); @@ -2709,10 +2663,9 @@ rte_eth_rx_descriptor_done(uint8_t port_id, uint16_t queue_id, uint16_t offset) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return (-ENODEV); - } + dev = &rte_eth_devices[port_id]; FUNC_PTR_OR_ERR_RET(*dev->dev_ops->rx_descriptor_done, -ENOTSUP); return (*dev->dev_ops->rx_descriptor_done)( \ @@ -2730,10 +2683,8 @@ rte_eth_dev_callback_register(uint8_t port_id, if (!cb_fn) return (-EINVAL); - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return (-EINVAL); - } dev = &rte_eth_devices[port_id]; rte_spinlock_lock(&rte_eth_dev_cb_lock); @@ -2770,10 +2721,8 @@ rte_eth_dev_callback_unregister(uint8_t port_id, if (!cb_fn) return (-EINVAL); - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return (-EINVAL); - } dev = &rte_eth_devices[port_id]; rte_spinlock_lock(&rte_eth_dev_cb_lock); @@ -2830,10 +2779,8 @@ int rte_eth_dev_bypass_init(uint8_t port_id) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return (-ENODEV); - } if ((dev= &rte_eth_devices[port_id]) == NULL) { PMD_DEBUG_TRACE("Invalid port device\n"); @@ -2850,10 +2797,8 @@ rte_eth_dev_bypass_state_show(uint8_t port_id, uint32_t *state) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return (-ENODEV); - } if ((dev= &rte_eth_devices[port_id]) == NULL) { PMD_DEBUG_TRACE("Invalid port device\n"); @@ -2869,10 +2814,8 @@ rte_eth_dev_bypass_state_set(uint8_t port_id, uint32_t *new_state) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return (-ENODEV); - } if ((dev= &rte_eth_devices[port_id]) == NULL) { PMD_DEBUG_TRACE("Invalid port device\n"); @@ -2889,10 +2832,8 @@ rte_eth_dev_bypass_event_show(uint8_t port_id, uint32_t event, uint32_t *state) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return (-ENODEV); - } if ((dev= &rte_eth_devices[port_id]) == NULL) { PMD_DEBUG_TRACE("Invalid port device\n"); @@ -2909,10 +2850,8 @@ rte_eth_dev_bypass_event_store(uint8_t port_id, uint32_t event, uint32_t state) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return (-ENODEV); - } if ((dev= &rte_eth_devices[port_id]) == NULL) { PMD_DEBUG_TRACE("Invalid port device\n"); @@ -2929,10 +2868,8 @@ rte_eth_dev_wd_timeout_store(uint8_t port_id, uint32_t timeout) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return (-ENODEV); - } if ((dev= &rte_eth_devices[port_id]) == NULL) { PMD_DEBUG_TRACE("Invalid port device\n"); @@ -2949,10 +2886,8 @@ rte_eth_dev_bypass_ver_show(uint8_t port_id, uint32_t *ver) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return (-ENODEV); - } if ((dev= &rte_eth_devices[port_id]) == NULL) { PMD_DEBUG_TRACE("Invalid port device\n"); @@ -2969,10 +2904,8 @@ rte_eth_dev_bypass_wd_timeout_show(uint8_t port_id, uint32_t *wd_timeout) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return (-ENODEV); - } if ((dev= &rte_eth_devices[port_id]) == NULL) { PMD_DEBUG_TRACE("Invalid port device\n"); @@ -2989,10 +2922,8 @@ rte_eth_dev_bypass_wd_reset(uint8_t port_id) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return (-ENODEV); - } if ((dev= &rte_eth_devices[port_id]) == NULL) { PMD_DEBUG_TRACE("Invalid port device\n"); @@ -3011,10 +2942,8 @@ rte_eth_dev_add_syn_filter(uint8_t port_id, { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return -ENODEV; - } dev = &rte_eth_devices[port_id]; FUNC_PTR_OR_ERR_RET(*dev->dev_ops->add_syn_filter, -ENOTSUP); @@ -3026,10 +2955,8 @@ rte_eth_dev_remove_syn_filter(uint8_t port_id) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return -ENODEV; - } dev = &rte_eth_devices[port_id]; FUNC_PTR_OR_ERR_RET(*dev->dev_ops->remove_syn_filter, -ENOTSUP); @@ -3045,10 +2972,8 @@ rte_eth_dev_get_syn_filter(uint8_t port_id, if (filter == NULL || rx_queue == NULL) return -EINVAL; - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return -ENODEV; - } dev = &rte_eth_devices[port_id]; FUNC_PTR_OR_ERR_RET(*dev->dev_ops->get_syn_filter, -ENOTSUP); @@ -3061,10 +2986,9 @@ rte_eth_dev_add_2tuple_filter(uint8_t port_id, uint16_t index, { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return -ENODEV; - } + if (filter->protocol != IPPROTO_TCP && filter->tcp_flags != 0){ PMD_DEBUG_TRACE("tcp flags is 0x%x, but the protocol value" @@ -3083,10 +3007,8 @@ rte_eth_dev_remove_2tuple_filter(uint8_t port_id, uint16_t index) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return -ENODEV; - } dev = &rte_eth_devices[port_id]; FUNC_PTR_OR_ERR_RET(*dev->dev_ops->remove_2tuple_filter, -ENOTSUP); @@ -3102,10 +3024,8 @@ rte_eth_dev_get_2tuple_filter(uint8_t port_id, uint16_t index, if (filter == NULL || rx_queue == NULL) return -EINVAL; - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return -ENODEV; - } dev = &rte_eth_devices[port_id]; FUNC_PTR_OR_ERR_RET(*dev->dev_ops->get_2tuple_filter, -ENOTSUP); @@ -3118,10 +3038,8 @@ rte_eth_dev_add_5tuple_filter(uint8_t port_id, uint16_t index, { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return -ENODEV; - } if (filter->protocol != IPPROTO_TCP && filter->tcp_flags != 0){ @@ -3141,10 +3059,8 @@ rte_eth_dev_remove_5tuple_filter(uint8_t port_id, uint16_t index) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return -ENODEV; - } dev = &rte_eth_devices[port_id]; FUNC_PTR_OR_ERR_RET(*dev->dev_ops->remove_5tuple_filter, -ENOTSUP); @@ -3160,10 +3076,8 @@ rte_eth_dev_get_5tuple_filter(uint8_t port_id, uint16_t index, if (filter == NULL || rx_queue == NULL) return -EINVAL; - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return -ENODEV; - } dev = &rte_eth_devices[port_id]; FUNC_PTR_OR_ERR_RET(*dev->dev_ops->get_5tuple_filter, -ENOTSUP); @@ -3177,10 +3091,8 @@ rte_eth_dev_add_flex_filter(uint8_t port_id, uint16_t index, { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return -ENODEV; - } dev = &rte_eth_devices[port_id]; FUNC_PTR_OR_ERR_RET(*dev->dev_ops->add_flex_filter, -ENOTSUP); @@ -3192,10 +3104,8 @@ rte_eth_dev_remove_flex_filter(uint8_t port_id, uint16_t index) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return -ENODEV; - } dev = &rte_eth_devices[port_id]; FUNC_PTR_OR_ERR_RET(*dev->dev_ops->remove_flex_filter, -ENOTSUP); @@ -3211,10 +3121,8 @@ rte_eth_dev_get_flex_filter(uint8_t port_id, uint16_t index, if (filter == NULL || rx_queue == NULL) return -EINVAL; - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return -ENODEV; - } dev = &rte_eth_devices[port_id]; FUNC_PTR_OR_ERR_RET(*dev->dev_ops->get_flex_filter, -ENOTSUP); @@ -3227,10 +3135,8 @@ rte_eth_dev_filter_supported(uint8_t port_id, enum rte_filter_type filter_type) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return -ENODEV; - } dev = &rte_eth_devices[port_id]; FUNC_PTR_OR_ERR_RET(*dev->dev_ops->filter_ctrl, -ENOTSUP); @@ -3244,10 +3150,8 @@ rte_eth_dev_filter_ctrl(uint8_t port_id, enum rte_filter_type filter_type, { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return -ENODEV; - } dev = &rte_eth_devices[port_id]; FUNC_PTR_OR_ERR_RET(*dev->dev_ops->filter_ctrl, -ENOTSUP); diff --git a/lib/librte_ether/rte_ethdev.h b/lib/librte_ether/rte_ethdev.h index 1200c1c..ca101f5 100644 --- a/lib/librte_ether/rte_ethdev.h +++ b/lib/librte_ether/rte_ethdev.h @@ -1539,6 +1539,7 @@ struct rte_eth_dev { struct eth_dev_ops *dev_ops; /**< Functions exported by PMD */ struct rte_pci_device *pci_dev; /**< PCI info. supplied by probing */ struct rte_eth_dev_cb_list callbacks; /**< User application callbacks */ + uint8_t attached; /**< Flag indicating the port is attached */ }; struct rte_eth_dev_sriov { @@ -1604,6 +1605,10 @@ extern struct rte_eth_dev rte_eth_devices[]; * initialized by the [matching] Ethernet driver during the PCI probing phase. * All devices whose port identifier is in the range * [0, rte_eth_dev_count() - 1] can be operated on by network applications. + * immediately after invoking rte_eal_init(). + * If the application unplugs a port using hotplug function, The enabled port + * numbers may be noncontiguous. In the case, the applications need to manage + * enabled port by themselves. * * @return * - The total number of usable Ethernet devices. -- 1.9.1
This patch replaces pci_addr_comparison() and memcmp() of pci addresses by eal_compare_pci_addr(). v5: - Fix pci_scan_one to handle pt_driver correctly. v4: - Fix calculation method of eal_compare_pci_addr(). - Add parameter checking. Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp> --- lib/librte_eal/bsdapp/eal/eal_pci.c | 25 ++++++++--------------- lib/librte_eal/common/eal_common_pci.c | 2 +- lib/librte_eal/common/include/rte_pci.h | 34 +++++++++++++++++++++++++++++++ lib/librte_eal/linuxapp/eal/eal_pci.c | 25 ++++++++--------------- lib/librte_eal/linuxapp/eal/eal_pci_uio.c | 2 +- 5 files changed, 54 insertions(+), 34 deletions(-) diff --git a/lib/librte_eal/bsdapp/eal/eal_pci.c b/lib/librte_eal/bsdapp/eal/eal_pci.c index 74ecce7..c844d58 100644 --- a/lib/librte_eal/bsdapp/eal/eal_pci.c +++ b/lib/librte_eal/bsdapp/eal/eal_pci.c @@ -270,20 +270,6 @@ pci_uio_map_resource(struct rte_pci_device *dev) return (0); } -/* Compare two PCI device addresses. */ -static int -pci_addr_comparison(struct rte_pci_addr *addr, struct rte_pci_addr *addr2) -{ - uint64_t dev_addr = (addr->domain << 24) + (addr->bus << 16) + (addr->devid << 8) + addr->function; - uint64_t dev_addr2 = (addr2->domain << 24) + (addr2->bus << 16) + (addr2->devid << 8) + addr2->function; - - if (dev_addr > dev_addr2) - return 1; - else - return 0; -} - - /* Scan one pci sysfs entry, and fill the devices list from it. */ static int pci_scan_one(int dev_pci_fd, struct pci_conf *conf) @@ -356,13 +342,20 @@ pci_scan_one(int dev_pci_fd, struct pci_conf *conf) } else { struct rte_pci_device *dev2 = NULL; + int ret; TAILQ_FOREACH(dev2, &pci_device_list, next) { - if (pci_addr_comparison(&dev->addr, &dev2->addr)) + ret = eal_compare_pci_addr(&dev->addr, &dev2->addr); + if (ret > 0) continue; - else { + else if (ret < 0) { TAILQ_INSERT_BEFORE(dev2, dev, next); return 0; + } else { /* already registered */ + /* update pt_driver */ + dev2->pt_driver = dev->pt_driver; + free(dev); + return 0; } } TAILQ_INSERT_TAIL(&pci_device_list, dev, next); diff --git a/lib/librte_eal/common/eal_common_pci.c b/lib/librte_eal/common/eal_common_pci.c index f3c7f71..a89f5c3 100644 --- a/lib/librte_eal/common/eal_common_pci.c +++ b/lib/librte_eal/common/eal_common_pci.c @@ -93,7 +93,7 @@ static struct rte_devargs *pci_devargs_lookup(struct rte_pci_device *dev) if (devargs->type != RTE_DEVTYPE_BLACKLISTED_PCI && devargs->type != RTE_DEVTYPE_WHITELISTED_PCI) continue; - if (!memcmp(&dev->addr, &devargs->pci.addr, sizeof(dev->addr))) + if (!eal_compare_pci_addr(&dev->addr, &devargs->pci.addr)) return devargs; } return NULL; diff --git a/lib/librte_eal/common/include/rte_pci.h b/lib/librte_eal/common/include/rte_pci.h index 7f2d699..4814cd7 100644 --- a/lib/librte_eal/common/include/rte_pci.h +++ b/lib/librte_eal/common/include/rte_pci.h @@ -269,6 +269,40 @@ eal_parse_pci_DomBDF(const char *input, struct rte_pci_addr *dev_addr) } #undef GET_PCIADDR_FIELD +/* Compare two PCI device addresses. */ +/** + * Utility function to compare two PCI device addresses. + * + * @param addr + * The PCI Bus-Device-Function address to compare + * @param addr2 + * The PCI Bus-Device-Function address to compare + * @return + * 0 on equal PCI address. + * Positive on addr is greater than addr2. + * Negative on addr is less than addr2, or error. + */ +static inline int +eal_compare_pci_addr(struct rte_pci_addr *addr, struct rte_pci_addr *addr2) +{ + uint64_t dev_addr, dev_addr2; + + if ((addr == NULL) || (addr2 == NULL)) + return -1; + + dev_addr = (addr->domain << 24) | (addr->bus << 16) | + (addr->devid << 8) | addr->function; + dev_addr2 = (addr2->domain << 24) | (addr2->bus << 16) | + (addr2->devid << 8) | addr2->function; + + if (dev_addr > dev_addr2) + return 1; + else if (dev_addr < dev_addr2) + return -1; + else + return 0; +} + /** * Probe the PCI bus for registered drivers. * diff --git a/lib/librte_eal/linuxapp/eal/eal_pci.c b/lib/librte_eal/linuxapp/eal/eal_pci.c index c0ca5a5..d847102 100644 --- a/lib/librte_eal/linuxapp/eal/eal_pci.c +++ b/lib/librte_eal/linuxapp/eal/eal_pci.c @@ -229,20 +229,6 @@ error: return -1; } -/* Compare two PCI device addresses. */ -static int -pci_addr_comparison(struct rte_pci_addr *addr, struct rte_pci_addr *addr2) -{ - uint64_t dev_addr = (addr->domain << 24) + (addr->bus << 16) + (addr->devid << 8) + addr->function; - uint64_t dev_addr2 = (addr2->domain << 24) + (addr2->bus << 16) + (addr2->devid << 8) + addr2->function; - - if (dev_addr > dev_addr2) - return 1; - else - return 0; -} - - /* Scan one pci sysfs entry, and fill the devices list from it. */ static int pci_scan_one(const char *dirname, uint16_t domain, uint8_t bus, @@ -353,13 +339,20 @@ pci_scan_one(const char *dirname, uint16_t domain, uint8_t bus, } else { struct rte_pci_device *dev2 = NULL; + int ret; TAILQ_FOREACH(dev2, &pci_device_list, next) { - if (pci_addr_comparison(&dev->addr, &dev2->addr)) + ret = eal_compare_pci_addr(&dev->addr, &dev2->addr); + if (ret > 0) continue; - else { + else if (ret < 0) { TAILQ_INSERT_BEFORE(dev2, dev, next); return 0; + } else { /* already registered */ + /* update pt_driver */ + dev2->pt_driver = dev->pt_driver; + free(dev); + return 0; } } TAILQ_INSERT_TAIL(&pci_device_list, dev, next); diff --git a/lib/librte_eal/linuxapp/eal/eal_pci_uio.c b/lib/librte_eal/linuxapp/eal/eal_pci_uio.c index e53f06b..1da3507 100644 --- a/lib/librte_eal/linuxapp/eal/eal_pci_uio.c +++ b/lib/librte_eal/linuxapp/eal/eal_pci_uio.c @@ -123,7 +123,7 @@ pci_uio_map_secondary(struct rte_pci_device *dev) TAILQ_FOREACH(uio_res, pci_res_list, next) { /* skip this element if it doesn't match our PCI address */ - if (memcmp(&uio_res->pci_addr, &dev->addr, sizeof(dev->addr))) + if (eal_compare_pci_addr(&uio_res->pci_addr, &dev->addr)) continue; for (i = 0; i != uio_res->nb_maps; i++) { -- 1.9.1
This patch adds rte_eth_dev_free(). The function is used for changing an attached status of the device that has specified name. v6: - Use rte_eth_dev structure as the paramter of rte_eth_dev_free(). v4: - Add parameter checking. Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp> --- lib/librte_ether/rte_ethdev.c | 11 +++++++++++ lib/librte_ether/rte_ethdev.h | 14 ++++++++++++++ 2 files changed, 25 insertions(+) diff --git a/lib/librte_ether/rte_ethdev.c b/lib/librte_ether/rte_ethdev.c index d70854f..b58bab3 100644 --- a/lib/librte_ether/rte_ethdev.c +++ b/lib/librte_ether/rte_ethdev.c @@ -260,6 +260,17 @@ rte_eth_dev_allocate(const char *name) return eth_dev; } +int +rte_eth_dev_free(struct rte_eth_dev *eth_dev) +{ + if (eth_dev == NULL) + return -EINVAL; + + eth_dev->attached = 0; + nb_ports--; + return 0; +} + static int rte_eth_dev_init(struct rte_pci_driver *pci_drv, struct rte_pci_device *pci_dev) diff --git a/lib/librte_ether/rte_ethdev.h b/lib/librte_ether/rte_ethdev.h index ca101f5..fbe7ac1 100644 --- a/lib/librte_ether/rte_ethdev.h +++ b/lib/librte_ether/rte_ethdev.h @@ -1627,6 +1627,20 @@ extern uint8_t rte_eth_dev_count(void); */ struct rte_eth_dev *rte_eth_dev_allocate(const char *name); +/** + * Function for internal use by dummy drivers primarily, e.g. ring-based + * driver. + * Free the specified ethdev. + * + * @param eth_dev + * The *eth_dev* pointer is the address of the *rte_eth_dev* structure + * associated with the matching device and which have been [automatically] + * allocated in the *rte_eth_devices* array. + * @return + * - 0 on success, negative on error + */ +int rte_eth_dev_free(struct rte_eth_dev *eth_dev); + struct eth_driver; /** * @internal -- 1.9.1
The patch adds function pointer to rte_pci_driver and eth_driver structure. These function pointers are used when ports are detached. Also, the patch adds rte_eth_dev_uninit(). So far, it's not called by anywhere, but it will be called when port hotplug function is implemented. v6: - Fix rte_eth_dev_uninit() to handle a return value of uninit function of PMD. v4: - Add parameter checking. - Change function names. Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp> --- lib/librte_eal/common/include/rte_pci.h | 7 +++++ lib/librte_ether/rte_ethdev.c | 47 +++++++++++++++++++++++++++++++++ lib/librte_ether/rte_ethdev.h | 24 +++++++++++++++++ 3 files changed, 78 insertions(+) diff --git a/lib/librte_eal/common/include/rte_pci.h b/lib/librte_eal/common/include/rte_pci.h index 4814cd7..87ca4cf 100644 --- a/lib/librte_eal/common/include/rte_pci.h +++ b/lib/librte_eal/common/include/rte_pci.h @@ -189,12 +189,19 @@ struct rte_pci_driver; typedef int (pci_devinit_t)(struct rte_pci_driver *, struct rte_pci_device *); /** + * Uninitialisation function for the driver called during hotplugging. + */ +typedef int (pci_devuninit_t)( + struct rte_pci_driver *, struct rte_pci_device *); + +/** * A structure describing a PCI driver. */ struct rte_pci_driver { TAILQ_ENTRY(rte_pci_driver) next; /**< Next in list. */ const char *name; /**< Driver name. */ pci_devinit_t *devinit; /**< Device init. function. */ + pci_devuninit_t *devuninit; /**< Device uninit function. */ struct rte_pci_id *id_table; /**< ID table, NULL terminated. */ uint32_t drv_flags; /**< Flags contolling handling of device. */ }; diff --git a/lib/librte_ether/rte_ethdev.c b/lib/librte_ether/rte_ethdev.c index b58bab3..7bed901 100644 --- a/lib/librte_ether/rte_ethdev.c +++ b/lib/librte_ether/rte_ethdev.c @@ -326,6 +326,52 @@ rte_eth_dev_init(struct rte_pci_driver *pci_drv, return diag; } +static int +rte_eth_dev_uninit(struct rte_pci_driver *pci_drv, + struct rte_pci_device *pci_dev) +{ + struct eth_driver *eth_drv; + struct rte_eth_dev *eth_dev; + char ethdev_name[RTE_ETH_NAME_MAX_LEN]; + int ret; + + if ((pci_drv == NULL) || (pci_dev == NULL)) + return -EINVAL; + + /* Create unique Ethernet device name using PCI address */ + snprintf(ethdev_name, RTE_ETH_NAME_MAX_LEN, "%d:%d.%d", + pci_dev->addr.bus, pci_dev->addr.devid, + pci_dev->addr.function); + + eth_dev = rte_eth_dev_allocated(ethdev_name); + if (eth_dev == NULL) + return -ENODEV; + + eth_drv = (struct eth_driver *)pci_drv; + + /* Invoke PMD device uninit function */ + if (*eth_drv->eth_dev_uninit) { + ret = (*eth_drv->eth_dev_uninit)(eth_drv, eth_dev); + if (ret) + return ret; + } + + /* free ether device */ + rte_eth_dev_free(eth_dev); + + /* init user callbacks */ + TAILQ_INIT(&(eth_dev->callbacks)); + + if (rte_eal_process_type() == RTE_PROC_PRIMARY) + rte_free(eth_dev->data->dev_private); + + eth_dev->pci_dev = NULL; + eth_dev->driver = NULL; + eth_dev->data = NULL; + + return 0; +} + /** * Register an Ethernet [Poll Mode] driver. * @@ -344,6 +390,7 @@ void rte_eth_driver_register(struct eth_driver *eth_drv) { eth_drv->pci_drv.devinit = rte_eth_dev_init; + eth_drv->pci_drv.devuninit = rte_eth_dev_uninit; rte_eal_pci_register(ð_drv->pci_drv); } diff --git a/lib/librte_ether/rte_ethdev.h b/lib/librte_ether/rte_ethdev.h index fbe7ac1..91d9e86 100644 --- a/lib/librte_ether/rte_ethdev.h +++ b/lib/librte_ether/rte_ethdev.h @@ -1678,6 +1678,27 @@ typedef int (*eth_dev_init_t)(struct eth_driver *eth_drv, /** * @internal + * Finalization function of an Ethernet driver invoked for each matching + * Ethernet PCI device detected during the PCI closing phase. + * + * @param eth_drv + * The pointer to the [matching] Ethernet driver structure supplied by + * the PMD when it registered itself. + * @param eth_dev + * The *eth_dev* pointer is the address of the *rte_eth_dev* structure + * associated with the matching device and which have been [automatically] + * allocated in the *rte_eth_devices* array. + * @return + * - 0: Success, the device is properly finalized by the driver. + * In particular, the driver MUST free the *dev_ops* pointer + * of the *eth_dev* structure. + * - <0: Error code of the device initialization failure. + */ +typedef int (*eth_dev_uninit_t)(struct eth_driver *eth_drv, + struct rte_eth_dev *eth_dev); + +/** + * @internal * The structure associated with a PMD Ethernet driver. * * Each Ethernet driver acts as a PCI driver and is represented by a generic @@ -1687,11 +1708,14 @@ typedef int (*eth_dev_init_t)(struct eth_driver *eth_drv, * * - The *eth_dev_init* function invoked for each matching PCI device. * + * - The *eth_dev_uninit* function invoked for each matching PCI device. + * * - The size of the private data to allocate for each matching device. */ struct eth_driver { struct rte_pci_driver pci_drv; /**< The PMD is also a PCI driver. */ eth_dev_init_t eth_dev_init; /**< Device init function. */ + eth_dev_uninit_t eth_dev_uninit; /**< Device uninit function. */ unsigned int dev_private_size; /**< Size of device private data. */ }; -- 1.9.1
The patch adds following functions. - rte_eth_dev_save() The function is used for saving current rte_eth_dev structures. - rte_eth_dev_get_changed_port() The function receives the rte_eth_dev structures, then compare these with current values to know which port is actually attached or detached. - rte_eth_dev_get_addr_by_port() The function returns a pci address of an ethdev specified by port identifier. - rte_eth_dev_get_port_by_addr() The function returns a port identifier of an ethdev specified by pci address. - rte_eth_dev_get_name_by_port() The function returns a unique identifier name of an ethdev specified by port identifier. - Add rte_eth_dev_check_detachable() The function returns whether a PMD supports detach function. Also, the patch changes scope of rte_eth_dev_allocated() to global. This function will be called by virtual PMDs to support port hotplug. So change scope of the function to global. v7: - Add pt_driver checking to rte_eth_dev_check_detachable(). (Thanks to Qiu, Michael) v5: - Fix return value of below functions. rte_eth_dev_get_changed_port(). rte_eth_dev_get_port_by_addr(). v4: - Add parameter checking. v3: - Fix if-condition bug while comparing pci addresses. - Add error checking codes. Reported-by: Mark Enright <menrigh@brocade.com> Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp> --- lib/librte_ether/rte_ethdev.c | 109 +++++++++++++++++++++++++++++++++++++++++- lib/librte_ether/rte_ethdev.h | 80 +++++++++++++++++++++++++++++++ 2 files changed, 188 insertions(+), 1 deletion(-) diff --git a/lib/librte_ether/rte_ethdev.c b/lib/librte_ether/rte_ethdev.c index 7bed901..14a040a 100644 --- a/lib/librte_ether/rte_ethdev.c +++ b/lib/librte_ether/rte_ethdev.c @@ -206,7 +206,7 @@ rte_eth_dev_data_alloc(void) RTE_MAX_ETHPORTS * sizeof(*rte_eth_dev_data)); } -static struct rte_eth_dev * +struct rte_eth_dev * rte_eth_dev_allocated(const char *name) { unsigned i; @@ -426,6 +426,113 @@ rte_eth_dev_count(void) return (nb_ports); } +void +rte_eth_dev_save(struct rte_eth_dev *devs) +{ + if (devs == NULL) + return; + + /* save current rte_eth_devices */ + memcpy(devs, rte_eth_devices, + sizeof(struct rte_eth_dev) * RTE_MAX_ETHPORTS); +} + +int +rte_eth_dev_get_changed_port(struct rte_eth_dev *devs, uint8_t *port_id) +{ + if ((devs == NULL) || (port_id == NULL)) + return -EINVAL; + + /* check which port was attached or detached */ + for (*port_id = 0; *port_id < RTE_MAX_ETHPORTS; (*port_id)++, devs++) { + if (rte_eth_devices[*port_id].attached ^ devs->attached) + return 0; + } + return -ENODEV; +} + +int +rte_eth_dev_get_addr_by_port(uint8_t port_id, struct rte_pci_addr *addr) +{ + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) + return -EINVAL; + + if (addr == NULL) { + PMD_DEBUG_TRACE("Null pointer is specified\n"); + return -EINVAL; + } + + *addr = rte_eth_devices[port_id].pci_dev->addr; + return 0; +} + +int +rte_eth_dev_get_port_by_addr(struct rte_pci_addr *addr, uint8_t *port_id) +{ + struct rte_pci_addr *tmp; + + if ((addr == NULL) || (port_id == NULL)) { + PMD_DEBUG_TRACE("Null pointer is specified\n"); + return -EINVAL; + } + + for (*port_id = 0; *port_id < RTE_MAX_ETHPORTS; (*port_id)++) { + if (!rte_eth_devices[*port_id].attached) + continue; + if (!rte_eth_devices[*port_id].pci_dev) + continue; + tmp = &rte_eth_devices[*port_id].pci_dev->addr; + if (eal_compare_pci_addr(tmp, addr) == 0) + return 0; + } + return -ENODEV; +} + +int +rte_eth_dev_get_name_by_port(uint8_t port_id, char *name) +{ + char *tmp; + + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) + return -EINVAL; + + if (name == NULL) { + PMD_DEBUG_TRACE("Null pointer is specified\n"); + return -EINVAL; + } + + /* shouldn't check 'rte_eth_devices[i].data', + * because it might be overwritten by VDEV PMD */ + tmp = rte_eth_dev_data[port_id].name; + strncpy(name, tmp, strlen(tmp) + 1); + return 0; +} + +int +rte_eth_dev_check_detachable(uint8_t port_id) +{ + uint32_t drv_flags; + + if (port_id >= RTE_MAX_ETHPORTS) { + PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + return -EINVAL; + } + + if (rte_eth_devices[port_id].dev_type == RTE_ETH_DEV_PHYSICAL) { + switch (rte_eth_devices[port_id].pci_dev->pt_driver) { + case RTE_PT_IGB_UIO: + case RTE_PT_UIO_GENERIC: + break; + case RTE_PT_VFIO: + default: + return -ENOTSUP; + } + } + + drv_flags = rte_eth_devices[port_id].driver->pci_drv.drv_flags; + return !(drv_flags & RTE_PCI_DRV_DETACHABLE); +} + static int rte_eth_dev_rx_queue_config(struct rte_eth_dev *dev, uint16_t nb_queues) { diff --git a/lib/librte_ether/rte_ethdev.h b/lib/librte_ether/rte_ethdev.h index 91d9e86..9919968 100644 --- a/lib/librte_ether/rte_ethdev.h +++ b/lib/librte_ether/rte_ethdev.h @@ -1616,6 +1616,86 @@ extern struct rte_eth_dev rte_eth_devices[]; extern uint8_t rte_eth_dev_count(void); /** + * Function for internal use by port hotplug functions. + * Copies current ethdev structures to the specified pointer. + * + * @param devs The pointer to the ethdev structures + */ +extern void rte_eth_dev_save(struct rte_eth_dev *devs); + +/** + * Function for internal use by port hotplug functions. + * Compare the specified ethdev structures with currents. Then + * if there is a port which status is changed, fill the specified pointer + * with the port id of that port. + * @param devs The pointer to the ethdev structures + * @param port_id The pointer to the port id + * @return + * - 0 on success, negative on error + */ +extern int rte_eth_dev_get_changed_port( + struct rte_eth_dev *devs, uint8_t *port_id); + +/** + * Function for internal use by port hotplug functions. + * Returns a pci address of a ethdev specified by port identifier. + * @param port_id + * The port identifier of the Ethernet device + * @param addr + * The pointer to the pci address + * @return + * - 0 on success, negative on error + */ +extern int rte_eth_dev_get_addr_by_port( + uint8_t port_id, struct rte_pci_addr *addr); + +/** + * Function for internal use by port hotplug functions. + * Returns a port identifier of a ethdev specified by pci address. + * @param addr + * The pointer to the pci address of the Ethernet device. + * @param port_id + * The pointer to the port identifier + * @return + * - 0 on success, negative on error + */ +extern int rte_eth_dev_get_port_by_addr( + struct rte_pci_addr *addr, uint8_t *port_id); + +/** + * Function for internal use by port hotplug functions. + * Returns a unique identifier name of a ethdev specified by port identifier. + * @param port_id + * The port identifier. + * @param name + * The pointer to the Unique identifier name for each Ethernet device + * @return + * - 0 on success, negative on error + */ +extern int rte_eth_dev_get_name_by_port(uint8_t port_id, char *name); + +/** + * Function for internal use by port hotplug functions. + * Check whether or not, a PMD that is handling the ethdev specified by port + * identifier can support detach function. + * @param port_id + * The port identifier + * @return + * - 0 on supporting detach function, negative on not supporting + */ +extern int rte_eth_dev_check_detachable(uint8_t port_id); + +/** + * Function for internal use by port hotplug functions. + * Returns a ethdev slot specified by the unique identifier name. + * @param name + * The pointer to the Unique identifier name for each Ethernet device + * @return + * - The pointer to the ethdev slot, on success. NULL on error + */ +extern struct rte_eth_dev *rte_eth_dev_allocated(const char *name); + +/** * Function for internal use by dummy drivers primarily, e.g. ring-based * driver. * Allocates a new ethdev slot for an ethernet device and returns the pointer -- 1.9.1
The patch adds functions for unmapping igb_uio resources. The patch is only for Linux and igb_uio environment. VFIO and BSD are not supported. v5: - Fix pci_unmap_device() to check pt_driver. v4: - Add parameter checking. - Add header file to determine if hotplug can be enabled. Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp> --- lib/librte_eal/common/Makefile | 1 + lib/librte_eal/common/include/rte_dev_hotplug.h | 44 +++++++++++++++++ lib/librte_eal/linuxapp/eal/eal_pci.c | 44 +++++++++++++++++ lib/librte_eal/linuxapp/eal/eal_pci_init.h | 8 +++ lib/librte_eal/linuxapp/eal/eal_pci_uio.c | 65 +++++++++++++++++++++++++ 5 files changed, 162 insertions(+) create mode 100644 lib/librte_eal/common/include/rte_dev_hotplug.h diff --git a/lib/librte_eal/common/Makefile b/lib/librte_eal/common/Makefile index 52c1a5f..db7cc93 100644 --- a/lib/librte_eal/common/Makefile +++ b/lib/librte_eal/common/Makefile @@ -41,6 +41,7 @@ INC += rte_eal_memconfig.h rte_malloc_heap.h INC += rte_hexdump.h rte_devargs.h rte_dev.h INC += rte_common_vect.h INC += rte_pci_dev_feature_defs.h rte_pci_dev_features.h +INC += rte_dev_hotplug.h ifeq ($(CONFIG_RTE_INSECURE_FUNCTION_WARNING),y) INC += rte_warnings.h diff --git a/lib/librte_eal/common/include/rte_dev_hotplug.h b/lib/librte_eal/common/include/rte_dev_hotplug.h new file mode 100644 index 0000000..b333e0f --- /dev/null +++ b/lib/librte_eal/common/include/rte_dev_hotplug.h @@ -0,0 +1,44 @@ +/*- + * BSD LICENSE + * + * Copyright(c) 2015 IGEL Co.,LTd. + * All rights reserved. + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * + * * Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * * Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in + * the documentation and/or other materials provided with the + * distribution. + * * Neither the name of IGEL Co.,Ltd. nor the names of its + * contributors may be used to endorse or promote products derived + * from this software without specific prior written permission. + * + * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS + * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT + * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR + * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT + * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, + * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT + * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, + * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY + * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT + * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE + * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + */ + +#ifndef _RTE_DEV_HOTPLUG_H_ +#define _RTE_DEV_HOTPLUG_H_ + +/* + * determine if hotplug can be enabled on the system + */ +#if defined(RTE_LIBRTE_EAL_HOTPLUG) && defined(RTE_LIBRTE_EAL_LINUXAPP) +#define ENABLE_HOTPLUG +#endif /* RTE_LIBRTE_EAL_HOTPLUG & RTE_LIBRTE_EAL_LINUXAPP */ + +#endif /* _RTE_DEV_HOTPLUG_H_ */ diff --git a/lib/librte_eal/linuxapp/eal/eal_pci.c b/lib/librte_eal/linuxapp/eal/eal_pci.c index d847102..c3b7917 100644 --- a/lib/librte_eal/linuxapp/eal/eal_pci.c +++ b/lib/librte_eal/linuxapp/eal/eal_pci.c @@ -166,6 +166,25 @@ pci_map_resource(void *requested_addr, int fd, off_t offset, size_t size) return mapaddr; } +#ifdef ENABLE_HOTPLUG +/* unmap a particular resource */ +void +pci_unmap_resource(void *requested_addr, size_t size) +{ + if (requested_addr == NULL) + return; + + /* Unmap the PCI memory resource of device */ + if (munmap(requested_addr, size)) { + RTE_LOG(ERR, EAL, "%s(): cannot munmap(%p, 0x%lx): %s\n", + __func__, requested_addr, (unsigned long)size, + strerror(errno)); + } else + RTE_LOG(DEBUG, EAL, " PCI memory mapped at %p\n", + requested_addr); +} +#endif /* ENABLE_HOTPLUG */ + /* parse the "resource" sysfs file */ #define IORESOURCE_MEM 0x00000200 @@ -567,6 +586,31 @@ pci_map_device(struct rte_pci_device *dev) return ret; } +#ifdef ENABLE_HOTPLUG +static void +pci_unmap_device(struct rte_pci_device *dev) +{ + if (dev == NULL) + return; + + /* try unmapping the NIC resources using VFIO if it exists */ + switch (dev->pt_driver) { + case RTE_PT_VFIO: + RTE_LOG(ERR, EAL, "Hotplug doesn't support vfio yet\n"); + break; + case RTE_PT_IGB_UIO: + case RTE_PT_UIO_GENERIC: + /* unmap resources for devices that use uio */ + pci_uio_unmap_resource(dev); + break; + default: + RTE_LOG(DEBUG, EAL, " Not managed by known pt driver," + " skipped\n"); + break; + } +} +#endif /* ENABLE_HOTPLUG */ + /* * If vendor/device ID match, call the devinit() function of the * driver. diff --git a/lib/librte_eal/linuxapp/eal/eal_pci_init.h b/lib/librte_eal/linuxapp/eal/eal_pci_init.h index 1070eb8..5152a0b 100644 --- a/lib/librte_eal/linuxapp/eal/eal_pci_init.h +++ b/lib/librte_eal/linuxapp/eal/eal_pci_init.h @@ -34,6 +34,7 @@ #ifndef EAL_PCI_INIT_H_ #define EAL_PCI_INIT_H_ +#include <rte_dev_hotplug.h> #include "eal_vfio.h" struct pci_map { @@ -71,6 +72,13 @@ void *pci_map_resource(void *requested_addr, int fd, off_t offset, /* map IGB_UIO resource prototype */ int pci_uio_map_resource(struct rte_pci_device *dev); +#ifdef ENABLE_HOTPLUG +void pci_unmap_resource(void *requested_addr, size_t size); + +/* unmap IGB_UIO resource prototype */ +void pci_uio_unmap_resource(struct rte_pci_device *dev); +#endif /* ENABLE_HOTPLUG */ + #ifdef VFIO_PRESENT #define VFIO_MAX_GROUPS 64 diff --git a/lib/librte_eal/linuxapp/eal/eal_pci_uio.c b/lib/librte_eal/linuxapp/eal/eal_pci_uio.c index 1da3507..81830d1 100644 --- a/lib/librte_eal/linuxapp/eal/eal_pci_uio.c +++ b/lib/librte_eal/linuxapp/eal/eal_pci_uio.c @@ -404,6 +404,71 @@ pci_uio_map_resource(struct rte_pci_device *dev) return 0; } +#ifdef ENABLE_HOTPLUG +static void +pci_uio_unmap(struct mapped_pci_resource *uio_res) +{ + int i; + + if (uio_res == NULL) + return; + + for (i = 0; i != uio_res->nb_maps; i++) + pci_unmap_resource(uio_res->maps[i].addr, + (size_t)uio_res->maps[i].size); +} + +static struct mapped_pci_resource * +pci_uio_find_resource(struct rte_pci_device *dev) +{ + struct mapped_pci_resource *uio_res; + + if (dev == NULL) + return NULL; + + TAILQ_FOREACH(uio_res, pci_res_list, next) { + + /* skip this element if it doesn't match our PCI address */ + if (!eal_compare_pci_addr(&uio_res->pci_addr, &dev->addr)) + return uio_res; + } + return NULL; +} + +/* unmap the PCI resource of a PCI device in virtual memory */ +void +pci_uio_unmap_resource(struct rte_pci_device *dev) +{ + struct mapped_pci_resource *uio_res; + + if (dev == NULL) + return; + + /* find an entry for the device */ + uio_res = pci_uio_find_resource(dev); + if (uio_res == NULL) + return; + + /* secondary processes - just free maps */ + if (rte_eal_process_type() != RTE_PROC_PRIMARY) + return pci_uio_unmap(uio_res); + + TAILQ_REMOVE(pci_res_list, uio_res, next); + + /* unmap all resources */ + pci_uio_unmap(uio_res); + + /* free uio resource */ + rte_free(uio_res); + + /* close fd if in primary process */ + close(dev->intr_handle.fd); + + dev->intr_handle.fd = -1; + dev->intr_handle.type = RTE_INTR_HANDLE_UNKNOWN; +} +#endif /* ENABLE_HOTPLUG */ + /* * parse a sysfs file containing one integer value * different to the eal version, as it needs to work with 64-bit values -- 1.9.1
The function removes the specified devargs entry from devargs_list. Also, the patch adds sanity checking to rte_eal_devargs_add(). v5: - Change function definition of rte_eal_devargs_remove(). v4: - Fix sanity check code. Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp> --- lib/librte_eal/common/eal_common_devargs.c | 60 +++++++++++++++++++++++++++++ lib/librte_eal/common/include/rte_devargs.h | 21 ++++++++++ 2 files changed, 81 insertions(+) diff --git a/lib/librte_eal/common/eal_common_devargs.c b/lib/librte_eal/common/eal_common_devargs.c index 4c7d11a..5b1ac8e 100644 --- a/lib/librte_eal/common/eal_common_devargs.c +++ b/lib/librte_eal/common/eal_common_devargs.c @@ -44,6 +44,35 @@ struct rte_devargs_list devargs_list = TAILQ_HEAD_INITIALIZER(devargs_list); + +/* find a entry specified by pci address or device name */ +static struct rte_devargs * +rte_eal_devargs_find(enum rte_devtype devtype, void *args) +{ + struct rte_devargs *devargs; + + if (args == NULL) + return NULL; + + TAILQ_FOREACH(devargs, &devargs_list, next) { + switch (devtype) { + case RTE_DEVTYPE_WHITELISTED_PCI: + case RTE_DEVTYPE_BLACKLISTED_PCI: + if (eal_compare_pci_addr(&devargs->pci.addr, args) == 0) + goto found; + break; + case RTE_DEVTYPE_VIRTUAL: + if (memcmp(&devargs->virtual.drv_name, args, + strlen((char *)args)) == 0) + goto found; + break; + } + } + return NULL; +found: + return devargs; +} + /* store a whitelist parameter for later parsing */ int rte_eal_devargs_add(enum rte_devtype devtype, const char *devargs_str) @@ -87,6 +116,12 @@ rte_eal_devargs_add(enum rte_devtype devtype, const char *devargs_str) free(devargs); return -1; } + /* make sure there is no same entry */ + if (rte_eal_devargs_find(devtype, &devargs->pci.addr)) { + RTE_LOG(ERR, EAL, + "device already registered: <%s>\n", buf); + return -1; + } break; case RTE_DEVTYPE_VIRTUAL: /* save driver name */ @@ -98,6 +133,12 @@ rte_eal_devargs_add(enum rte_devtype devtype, const char *devargs_str) free(devargs); return -1; } + /* make sure there is no same entry */ + if (rte_eal_devargs_find(devtype, &devargs->virtual.drv_name)) { + RTE_LOG(ERR, EAL, + "device already registered: <%s>\n", buf); + return -1; + } break; } @@ -105,6 +146,25 @@ rte_eal_devargs_add(enum rte_devtype devtype, const char *devargs_str) return 0; } +/* remove it from the devargs_list */ +int +rte_eal_devargs_remove(enum rte_devtype devtype, void *args) +{ + struct rte_devargs *devargs; + + if (args == NULL) + return -EINVAL; + + devargs = rte_eal_devargs_find(devtype, args); + if (devargs == NULL) { + RTE_LOG(ERR, EAL, "device not found\n"); + return -ENODEV; + } + + TAILQ_REMOVE(&devargs_list, devargs, next); + return 0; +} + /* count the number of devices of a specified type */ unsigned int rte_eal_devargs_type_count(enum rte_devtype devtype) diff --git a/lib/librte_eal/common/include/rte_devargs.h b/lib/librte_eal/common/include/rte_devargs.h index 9f9c98f..6d9763b 100644 --- a/lib/librte_eal/common/include/rte_devargs.h +++ b/lib/librte_eal/common/include/rte_devargs.h @@ -123,6 +123,27 @@ extern struct rte_devargs_list devargs_list; int rte_eal_devargs_add(enum rte_devtype devtype, const char *devargs_str); /** + * Remove a device from the user device list + * + * For PCI devices, the format of arguments string is "PCI_ADDR". It shouldn't + * involve parameters for the device. Example: "08:00.1". + * + * For virtual devices, the format of arguments string is "DRIVER_NAME*". It + * shouldn't involve parameters for the device. Example: "eth_ring". The + * validity of the driver name is not checked by this function, it is done + * when closing the drivers. + * + * @param devtype + * The type of the device. + * @param name + * The name of the device. + * + * @return + * - 0 on success, negative on error + */ +int rte_eal_devargs_remove(enum rte_devtype devtype, void *args); + +/** * Count the number of user devices of a specified type * * @param devtype -- 1.9.1
- Add rte_eal_pci_close_one_dirver() The function is used for closing the specified driver and device. - Add pci_invoke_all_drivers() The function is based on pci_probe_all_drivers. But it can not only probe but also close drivers. - Add pci_close_all_drivers() The function tries to find a driver for the specified device, and then close the driver. - Add rte_eal_pci_probe_one() and rte_eal_pci_close_one() The functions are used for probe and close a device. First the function tries to find a device that has the specified PCI address. Then, probe or close the device. v5: - Remove RTE_EAL_INVOKE_TYPE_UNKNOWN, because it's unused. v4: - Fix parameter checking. - Fix indent of 'if' statement. Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp> --- lib/librte_eal/common/eal_common_pci.c | 90 +++++++++++++++++++++++++++++---- lib/librte_eal/common/eal_private.h | 24 +++++++++ lib/librte_eal/common/include/rte_pci.h | 33 ++++++++++++ lib/librte_eal/linuxapp/eal/eal_pci.c | 69 +++++++++++++++++++++++++ 4 files changed, 206 insertions(+), 10 deletions(-) diff --git a/lib/librte_eal/common/eal_common_pci.c b/lib/librte_eal/common/eal_common_pci.c index a89f5c3..7c9b8c5 100644 --- a/lib/librte_eal/common/eal_common_pci.c +++ b/lib/librte_eal/common/eal_common_pci.c @@ -99,19 +99,27 @@ static struct rte_devargs *pci_devargs_lookup(struct rte_pci_device *dev) return NULL; } -/* - * If vendor/device ID match, call the devinit() function of all - * registered driver for the given device. Return -1 if initialization - * failed, return 1 if no driver is found for this device. - */ static int -pci_probe_all_drivers(struct rte_pci_device *dev) +pci_invoke_all_drivers(struct rte_pci_device *dev, + enum rte_eal_invoke_type type) { struct rte_pci_driver *dr = NULL; - int rc; + int rc = 0; + + if ((dev == NULL) || (type >= RTE_EAL_INVOKE_TYPE_MAX)) + return -1; TAILQ_FOREACH(dr, &pci_driver_list, next) { - rc = rte_eal_pci_probe_one_driver(dr, dev); + switch (type) { + case RTE_EAL_INVOKE_TYPE_PROBE: + rc = rte_eal_pci_probe_one_driver(dr, dev); + break; + case RTE_EAL_INVOKE_TYPE_CLOSE: + rc = rte_eal_pci_close_one_driver(dr, dev); + break; + default: + return -1; + } if (rc < 0) /* negative value is an error */ return -1; @@ -123,6 +131,66 @@ pci_probe_all_drivers(struct rte_pci_device *dev) return 1; } +#ifdef ENABLE_HOTPLUG +static int +rte_eal_pci_invoke_one(struct rte_pci_addr *addr, + enum rte_eal_invoke_type type) +{ + struct rte_pci_device *dev = NULL; + int ret = 0; + + if ((addr == NULL) || (type >= RTE_EAL_INVOKE_TYPE_MAX)) + return -1; + + TAILQ_FOREACH(dev, &pci_device_list, next) { + if (eal_compare_pci_addr(&dev->addr, addr)) + continue; + + ret = pci_invoke_all_drivers(dev, type); + if (ret < 0) + goto invoke_err_return; + + if (type == RTE_EAL_INVOKE_TYPE_CLOSE) + goto remove_dev; + + return 0; + } + + return -1; + +invoke_err_return: + RTE_LOG(WARNING, EAL, "Requested device " PCI_PRI_FMT + " cannot be used\n", dev->addr.domain, dev->addr.bus, + dev->addr.devid, dev->addr.function); + return -1; + +remove_dev: + TAILQ_REMOVE(&pci_device_list, dev, next); + return 0; +} + + +/* + * Find the pci device specified by pci address, then invoke probe function of + * the driver of the devive. + */ +int +rte_eal_pci_probe_one(struct rte_pci_addr *addr) +{ + return rte_eal_pci_invoke_one(addr, RTE_EAL_INVOKE_TYPE_PROBE); +} + +/* + * Find the pci device specified by pci address, then invoke close function of + * the driver of the devive. + */ +int +rte_eal_pci_close_one(struct rte_pci_addr *addr) +{ + return rte_eal_pci_invoke_one(addr, RTE_EAL_INVOKE_TYPE_CLOSE); +} +#endif /* ENABLE_HOTPLUG */ + /* * Scan the content of the PCI bus, and call the devinit() function for * all registered drivers that have a matching entry in its id_table @@ -148,10 +216,12 @@ rte_eal_pci_probe(void) /* probe all or only whitelisted devices */ if (probe_all) - ret = pci_probe_all_drivers(dev); + ret = pci_invoke_all_drivers(dev, + RTE_EAL_INVOKE_TYPE_PROBE); else if (devargs != NULL && devargs->type == RTE_DEVTYPE_WHITELISTED_PCI) - ret = pci_probe_all_drivers(dev); + ret = pci_invoke_all_drivers(dev, + RTE_EAL_INVOKE_TYPE_PROBE); if (ret < 0) rte_exit(EXIT_FAILURE, "Requested device " PCI_PRI_FMT " cannot be used\n", dev->addr.domain, dev->addr.bus, diff --git a/lib/librte_eal/common/eal_private.h b/lib/librte_eal/common/eal_private.h index 159cd66..1a362ab 100644 --- a/lib/librte_eal/common/eal_private.h +++ b/lib/librte_eal/common/eal_private.h @@ -154,6 +154,15 @@ struct rte_pci_driver; struct rte_pci_device; /** + * The invoke type. + */ +enum rte_eal_invoke_type { + RTE_EAL_INVOKE_TYPE_PROBE, /**< invoke probe function */ + RTE_EAL_INVOKE_TYPE_CLOSE, /**< invoke close function */ + RTE_EAL_INVOKE_TYPE_MAX /**< max value of this enum */ +}; + +/** * Mmap memory for single PCI device * * This function is private to EAL. @@ -165,6 +174,21 @@ int rte_eal_pci_probe_one_driver(struct rte_pci_driver *dr, struct rte_pci_device *dev); /** + * Munmap memory for single PCI device + * + * This function is private to EAL. + * + * @param dr + * The pointer to the pci driver structure + * @param dev + * The pointer to the pci device structure + * @return + * 0 on success, negative on error + */ +int rte_eal_pci_close_one_driver(struct rte_pci_driver *dr, + struct rte_pci_device *dev); + +/** * Init tail queues for non-EAL library structures. This is to allow * the rings, mempools, etc. lists to be shared among multiple processes * diff --git a/lib/librte_eal/common/include/rte_pci.h b/lib/librte_eal/common/include/rte_pci.h index 87ca4cf..a111066 100644 --- a/lib/librte_eal/common/include/rte_pci.h +++ b/lib/librte_eal/common/include/rte_pci.h @@ -82,6 +82,7 @@ extern "C" { #include <inttypes.h> #include <rte_interrupts.h> +#include <rte_dev_hotplug.h> TAILQ_HEAD(pci_device_list, rte_pci_device); /**< PCI devices in D-linked Q. */ TAILQ_HEAD(pci_driver_list, rte_pci_driver); /**< PCI drivers in D-linked Q. */ @@ -323,6 +324,38 @@ eal_compare_pci_addr(struct rte_pci_addr *addr, struct rte_pci_addr *addr2) */ int rte_eal_pci_probe(void); +#ifdef ENABLE_HOTPLUG +/** + * Probe the single PCI device. + * + * Scan the content of the PCI bus, and find the pci device specified by pci + * address, then call the probe() function for registered driver that has a + * matching entry in its id_table for discovered device. + * + * @param addr + * The PCI Bus-Device-Function address to probe or close. + * @return + * - 0 on success. + * - Negative on error. + */ +int rte_eal_pci_probe_one(struct rte_pci_addr *addr); + +/** + * Close the single PCI device. + * + * Scan the content of the PCI bus, and find the pci device specified by pci + * address, then call the close() function for registered driver that has a + * matching entry in its id_table for discovered device. + * + * @param addr + * The PCI Bus-Device-Function address to probe or close. + * @return + * - 0 on success. + * - Negative on error. + */ +int rte_eal_pci_close_one(struct rte_pci_addr *addr); +#endif /* ENABLE_HOTPLUG */ + /** * Dump the content of the PCI bus. * diff --git a/lib/librte_eal/linuxapp/eal/eal_pci.c b/lib/librte_eal/linuxapp/eal/eal_pci.c index c3b7917..831422e 100644 --- a/lib/librte_eal/linuxapp/eal/eal_pci.c +++ b/lib/librte_eal/linuxapp/eal/eal_pci.c @@ -682,6 +682,75 @@ rte_eal_pci_probe_one_driver(struct rte_pci_driver *dr, struct rte_pci_device *d return 1; } +#ifdef ENABLE_HOTPLUG +/* + * If vendor/device ID match, call the devuninit() function of the + * driver. + */ +int +rte_eal_pci_close_one_driver(struct rte_pci_driver *dr, + struct rte_pci_device *dev) +{ + struct rte_pci_id *id_table; + + if ((dr == NULL) || (dev == NULL)) + return -EINVAL; + + for (id_table = dr->id_table ; id_table->vendor_id != 0; id_table++) { + + /* check if device's identifiers match the driver's ones */ + if (id_table->vendor_id != dev->id.vendor_id && + id_table->vendor_id != PCI_ANY_ID) + continue; + if (id_table->device_id != dev->id.device_id && + id_table->device_id != PCI_ANY_ID) + continue; + if (id_table->subsystem_vendor_id != + dev->id.subsystem_vendor_id && + id_table->subsystem_vendor_id != PCI_ANY_ID) + continue; + if (id_table->subsystem_device_id != + dev->id.subsystem_device_id && + id_table->subsystem_device_id != PCI_ANY_ID) + continue; + + struct rte_pci_addr *loc = &dev->addr; + + RTE_LOG(DEBUG, EAL, + "PCI device "PCI_PRI_FMT" on NUMA socket %i\n", + loc->domain, loc->bus, loc->devid, + loc->function, dev->numa_node); + + RTE_LOG(DEBUG, EAL, " remove driver: %x:%x %s\n", + dev->id.vendor_id, dev->id.device_id, + dr->name); + + /* call the driver devuninit() function */ + if (dr->devuninit && (dr->devuninit(dr, dev) < 0)) + return -1; /* negative value is an error */ + + /* clear driver structure */ + dev->driver = NULL; + + if (dr->drv_flags & RTE_PCI_DRV_NEED_MAPPING) + /* unmap resources for devices that use igb_uio */ + pci_unmap_device(dev); + + return 0; + } + /* return positive value if driver is not found */ + return 1; +} +#else /* ENABLE_HOTPLUG */ +int +rte_eal_pci_close_one_driver(struct rte_pci_driver *dr __rte_unused, + struct rte_pci_device *dev __rte_unused) +{ + RTE_LOG(ERR, EAL, "Hotplug support isn't enabled\n"); + return -1; +} +#endif /* ENABLE_HOTPLUG */ + /* Init the PCI EAL subsystem */ int rte_eal_pci_init(void) -- 1.9.1
This new parameter is needed to keep device type like physical or virtual. Port detaching processes are different between physical and virtual. This parameter lets detaching function know a device type of the port. v4: - Fix comments of rte_eth_dev_type. Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp> --- app/test/virtual_pmd.c | 2 +- lib/librte_ether/rte_ethdev.c | 14 ++++++++++++-- lib/librte_ether/rte_ethdev.h | 25 ++++++++++++++++++++++++- lib/librte_pmd_af_packet/rte_eth_af_packet.c | 2 +- lib/librte_pmd_bond/rte_eth_bond_api.c | 2 +- lib/librte_pmd_pcap/rte_eth_pcap.c | 2 +- lib/librte_pmd_ring/rte_eth_ring.c | 2 +- lib/librte_pmd_xenvirt/rte_eth_xenvirt.c | 2 +- 8 files changed, 42 insertions(+), 9 deletions(-) diff --git a/app/test/virtual_pmd.c b/app/test/virtual_pmd.c index 9fac95d..8d3a5ff 100644 --- a/app/test/virtual_pmd.c +++ b/app/test/virtual_pmd.c @@ -556,7 +556,7 @@ virtual_ethdev_create(const char *name, struct ether_addr *mac_addr, goto err; /* reserve an ethdev entry */ - eth_dev = rte_eth_dev_allocate(name); + eth_dev = rte_eth_dev_allocate(name, RTE_ETH_DEV_PHYSICAL); if (eth_dev == NULL) goto err; diff --git a/lib/librte_ether/rte_ethdev.c b/lib/librte_ether/rte_ethdev.c index 14a040a..704185d 100644 --- a/lib/librte_ether/rte_ethdev.c +++ b/lib/librte_ether/rte_ethdev.c @@ -232,7 +232,7 @@ rte_eth_dev_allocate_new_port(void) } struct rte_eth_dev * -rte_eth_dev_allocate(const char *name) +rte_eth_dev_allocate(const char *name, enum rte_eth_dev_type type) { uint8_t port_id; struct rte_eth_dev *eth_dev; @@ -256,6 +256,7 @@ rte_eth_dev_allocate(const char *name) snprintf(eth_dev->data->name, sizeof(eth_dev->data->name), "%s", name); eth_dev->data->port_id = port_id; eth_dev->attached = DEV_CONNECTED; + eth_dev->dev_type = type; nb_ports++; return eth_dev; } @@ -267,6 +268,7 @@ rte_eth_dev_free(struct rte_eth_dev *eth_dev) return -EINVAL; eth_dev->attached = 0; + eth_dev->dev_type = RTE_ETH_DEV_UNKNOWN; nb_ports--; return 0; } @@ -287,7 +289,7 @@ rte_eth_dev_init(struct rte_pci_driver *pci_drv, snprintf(ethdev_name, RTE_ETH_NAME_MAX_LEN, "%d:%d.%d", pci_dev->addr.bus, pci_dev->addr.devid, pci_dev->addr.function); - eth_dev = rte_eth_dev_allocate(ethdev_name); + eth_dev = rte_eth_dev_allocate(ethdev_name, RTE_ETH_DEV_PHYSICAL); if (eth_dev == NULL) return -ENOMEM; @@ -426,6 +428,14 @@ rte_eth_dev_count(void) return (nb_ports); } +enum rte_eth_dev_type +rte_eth_dev_get_device_type(uint8_t port_id) +{ + if (rte_eth_dev_validate_port(port_id, NONE_TRACE) == DEV_INVALID) + return -1; + return rte_eth_devices[port_id].dev_type; +} + void rte_eth_dev_save(struct rte_eth_dev *devs) { diff --git a/lib/librte_ether/rte_ethdev.h b/lib/librte_ether/rte_ethdev.h index 9919968..00a6218 100644 --- a/lib/librte_ether/rte_ethdev.h +++ b/lib/librte_ether/rte_ethdev.h @@ -1522,6 +1522,17 @@ struct eth_dev_ops { }; /** + * The eth device type + */ +enum rte_eth_dev_type { + RTE_ETH_DEV_UNKNOWN, /**< unknown device type */ + RTE_ETH_DEV_PHYSICAL, + /**< Physical function and Virtual function devices of NIC */ + RTE_ETH_DEV_VIRTUAL, /**< non hardware device */ + RTE_ETH_DEV_MAX /**< max value of this enum */ +}; + +/** * @internal * The generic data structure associated with each ethernet device. * @@ -1540,6 +1551,7 @@ struct rte_eth_dev { struct rte_pci_device *pci_dev; /**< PCI info. supplied by probing */ struct rte_eth_dev_cb_list callbacks; /**< User application callbacks */ uint8_t attached; /**< Flag indicating the port is attached */ + enum rte_eth_dev_type dev_type; /**< Flag indicating the device type */ }; struct rte_eth_dev_sriov { @@ -1617,6 +1629,15 @@ extern uint8_t rte_eth_dev_count(void); /** * Function for internal use by port hotplug functions. + * Get the device type to know whether the device is physical or virtual. + * @param port_id The pointer to the port id + * @return + * - Device type. + */ +extern enum rte_eth_dev_type rte_eth_dev_get_device_type(uint8_t port_id); + +/** + * Function for internal use by port hotplug functions. * Copies current ethdev structures to the specified pointer. * * @param devs The pointer to the ethdev structures @@ -1702,10 +1723,12 @@ extern struct rte_eth_dev *rte_eth_dev_allocated(const char *name); * to that slot for the driver to use. * * @param name Unique identifier name for each Ethernet device + * @param type Device type of this Ethernet device * @return * - Slot in the rte_dev_devices array for a new device; */ -struct rte_eth_dev *rte_eth_dev_allocate(const char *name); +struct rte_eth_dev *rte_eth_dev_allocate(const char *name, + enum rte_eth_dev_type type); /** * Function for internal use by dummy drivers primarily, e.g. ring-based diff --git a/lib/librte_pmd_af_packet/rte_eth_af_packet.c b/lib/librte_pmd_af_packet/rte_eth_af_packet.c index 1ffe1cd..80e9bdf 100644 --- a/lib/librte_pmd_af_packet/rte_eth_af_packet.c +++ b/lib/librte_pmd_af_packet/rte_eth_af_packet.c @@ -649,7 +649,7 @@ rte_pmd_init_internals(const char *name, } /* reserve an ethdev entry */ - *eth_dev = rte_eth_dev_allocate(name); + *eth_dev = rte_eth_dev_allocate(name, RTE_ETH_DEV_VIRTUAL); if (*eth_dev == NULL) goto error; diff --git a/lib/librte_pmd_bond/rte_eth_bond_api.c b/lib/librte_pmd_bond/rte_eth_bond_api.c index 4ab3267..7a6a5f7 100644 --- a/lib/librte_pmd_bond/rte_eth_bond_api.c +++ b/lib/librte_pmd_bond/rte_eth_bond_api.c @@ -235,7 +235,7 @@ rte_eth_bond_create(const char *name, uint8_t mode, uint8_t socket_id) } /* reserve an ethdev entry */ - eth_dev = rte_eth_dev_allocate(name); + eth_dev = rte_eth_dev_allocate(name, RTE_ETH_DEV_VIRTUAL); if (eth_dev == NULL) { RTE_BOND_LOG(ERR, "Unable to allocate rte_eth_dev"); goto err; diff --git a/lib/librte_pmd_pcap/rte_eth_pcap.c b/lib/librte_pmd_pcap/rte_eth_pcap.c index d299288..af7fae8 100644 --- a/lib/librte_pmd_pcap/rte_eth_pcap.c +++ b/lib/librte_pmd_pcap/rte_eth_pcap.c @@ -709,7 +709,7 @@ rte_pmd_init_internals(const char *name, const unsigned nb_rx_queues, goto error; /* reserve an ethdev entry */ - *eth_dev = rte_eth_dev_allocate(name); + *eth_dev = rte_eth_dev_allocate(name, RTE_ETH_DEV_VIRTUAL); if (*eth_dev == NULL) goto error; diff --git a/lib/librte_pmd_ring/rte_eth_ring.c b/lib/librte_pmd_ring/rte_eth_ring.c index d5b1686..c837fcd 100644 --- a/lib/librte_pmd_ring/rte_eth_ring.c +++ b/lib/librte_pmd_ring/rte_eth_ring.c @@ -255,7 +255,7 @@ rte_eth_from_rings(const char *name, struct rte_ring *const rx_queues[], goto error; /* reserve an ethdev entry */ - eth_dev = rte_eth_dev_allocate(name); + eth_dev = rte_eth_dev_allocate(name, RTE_ETH_DEV_VIRTUAL); if (eth_dev == NULL) goto error; diff --git a/lib/librte_pmd_xenvirt/rte_eth_xenvirt.c b/lib/librte_pmd_xenvirt/rte_eth_xenvirt.c index 04e30c9..bc403d6 100644 --- a/lib/librte_pmd_xenvirt/rte_eth_xenvirt.c +++ b/lib/librte_pmd_xenvirt/rte_eth_xenvirt.c @@ -648,7 +648,7 @@ eth_dev_xenvirt_create(const char *name, const char *params, goto err; /* reserve an ethdev entry */ - eth_dev = rte_eth_dev_allocate(name); + eth_dev = rte_eth_dev_allocate(name, RTE_ETH_DEV_VIRTUAL); if (eth_dev == NULL) goto err; -- 1.9.1
These functions are used for attaching or detaching a port. When rte_eal_dev_attach() is called, the function tries to realize the device name as pci address. If this is done successfully, rte_eal_dev_attach() will attach physical device port. If not, attaches virtual devive port. When rte_eal_dev_detach() is called, the function gets the device type of this port to know whether the port is come from physical or virtual. And then specific detaching function will be called. v7: - Fix typo of warning messages. (Thanks to Qiu, Michael) v5: - Change function names like below. rte_eal_dev_find_and_invoke() to rte_eal_vdev_find_and_invoke(). rte_eal_dev_invoke() to rte_eal_vdev_invoke(). - Add code to handle a return value of rte_eal_devargs_remove(). - Fix pci address format in rte_eal_dev_detach(). v4: - Fix comment. - Add error checking. - Fix indent of 'if' statement. - Change function name. Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp> --- lib/librte_eal/common/eal_common_dev.c | 274 ++++++++++++++++++++++++++++++++ lib/librte_eal/common/eal_private.h | 11 ++ lib/librte_eal/common/include/rte_dev.h | 33 ++++ lib/librte_eal/linuxapp/eal/Makefile | 1 + lib/librte_eal/linuxapp/eal/eal_pci.c | 6 +- 5 files changed, 322 insertions(+), 3 deletions(-) diff --git a/lib/librte_eal/common/eal_common_dev.c b/lib/librte_eal/common/eal_common_dev.c index eae5656..39407c0 100644 --- a/lib/librte_eal/common/eal_common_dev.c +++ b/lib/librte_eal/common/eal_common_dev.c @@ -32,10 +32,13 @@ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ +#include <stdio.h> +#include <limits.h> #include <string.h> #include <inttypes.h> #include <sys/queue.h> +#include <rte_ethdev.h> #include <rte_dev.h> #include <rte_devargs.h> #include <rte_debug.h> @@ -107,3 +110,274 @@ rte_eal_dev_init(void) } return 0; } + +/* So far, DPDK hotplug function only supports linux */ +#ifdef ENABLE_HOTPLUG +static void +rte_eal_vdev_invoke(struct rte_driver *driver, + struct rte_devargs *devargs, enum rte_eal_invoke_type type) +{ + if ((driver == NULL) || (devargs == NULL)) + return; + + switch (type) { + case RTE_EAL_INVOKE_TYPE_PROBE: + driver->init(devargs->virtual.drv_name, devargs->args); + break; + case RTE_EAL_INVOKE_TYPE_CLOSE: + driver->uninit(devargs->virtual.drv_name, devargs->args); + break; + default: + break; + } +} + +static int +rte_eal_vdev_find_and_invoke(const char *name, int type) +{ + struct rte_devargs *devargs; + struct rte_driver *driver; + + if (name == NULL) + return -EINVAL; + + /* call the init function for each virtual device */ + TAILQ_FOREACH(devargs, &devargs_list, next) { + + if (devargs->type != RTE_DEVTYPE_VIRTUAL) + continue; + + if (strncmp(name, devargs->virtual.drv_name, strlen(name))) + continue; + + TAILQ_FOREACH(driver, &dev_driver_list, next) { + if (driver->type != PMD_VDEV) + continue; + + /* search a driver prefix in virtual device name */ + if (!strncmp(driver->name, devargs->virtual.drv_name, + strlen(driver->name))) { + rte_eal_vdev_invoke(driver, devargs, type); + break; + } + } + + if (driver == NULL) { + RTE_LOG(WARNING, EAL, "no driver found for %s\n", + devargs->virtual.drv_name); + } + return 0; + } + return 1; +} + +/* attach the new physical device, then store port_id of the device */ +static int +rte_eal_dev_attach_pdev(struct rte_pci_addr *addr, uint8_t *port_id) +{ + uint8_t new_port_id; + struct rte_eth_dev devs[RTE_MAX_ETHPORTS]; + + if ((addr == NULL) || (port_id == NULL)) + goto err; + + /* save current port status */ + rte_eth_dev_save(devs); + /* re-construct pci_device_list */ + if (rte_eal_pci_scan()) + goto err; + /* invoke probe func of the driver can handle the new device */ + if (rte_eal_pci_probe_one(addr)) + goto err; + /* get port_id enabled by above procedures */ + if (rte_eth_dev_get_changed_port(devs, &new_port_id)) + goto err; + + *port_id = new_port_id; + return 0; +err: + RTE_LOG(ERR, EAL, "Driver, cannot attach the device\n"); + return -1; +} + +/* detach the new physical device, then store pci_addr of the device */ +static int +rte_eal_dev_detach_pdev(uint8_t port_id, struct rte_pci_addr *addr) +{ + struct rte_pci_addr freed_addr; + struct rte_pci_addr vp; + + if (addr == NULL) + goto err; + + /* check whether the driver supports detach feature, or not */ + if (rte_eth_dev_check_detachable(port_id)) + goto err; + + /* get pci address by port id */ + if (rte_eth_dev_get_addr_by_port(port_id, &freed_addr)) + goto err; + + /* Zerod pci addr means the port comes from virtual device */ + vp.domain = vp.bus = vp.devid = vp.function = 0; + if (eal_compare_pci_addr(&vp, &freed_addr) == 0) + goto err; + + /* invoke close func of the driver, + * also remove the device from pci_device_list */ + if (rte_eal_pci_close_one(&freed_addr)) + goto err; + + *addr = freed_addr; + return 0; +err: + RTE_LOG(ERR, EAL, "Driver, cannot detach the device\n"); + return -1; +} + +static void +get_vdev_name(char *vdevargs) +{ + char *sep; + + if (vdevargs == NULL) + return; + + /* set the first ',' to '\0' to split name and arguments */ + sep = strchr(vdevargs, ','); + if (sep != NULL) + sep[0] = '\0'; +} + +/* attach the new virtual device, then store port_id of the device */ +static int +rte_eal_dev_attach_vdev(const char *vdevargs, uint8_t *port_id) +{ + char *args; + uint8_t new_port_id; + struct rte_eth_dev devs[RTE_MAX_ETHPORTS]; + + if ((vdevargs == NULL) || (port_id == NULL)) + goto err0; + + args = strdup(vdevargs); + if (args == NULL) + goto err0; + + /* save current port status */ + rte_eth_dev_save(devs); + /* add the vdevargs to devargs_list */ + if (rte_eal_devargs_add(RTE_DEVTYPE_VIRTUAL, args)) + goto err1; + /* parse vdevargs, then retrieve device name */ + get_vdev_name(args); + /* walk around dev_driver_list to find the driver of the device, + * then invoke probe function o the driver */ + if (rte_eal_vdev_find_and_invoke(args, RTE_EAL_INVOKE_TYPE_PROBE)) + goto err2; + /* get port_id enabled by above procedures */ + if (rte_eth_dev_get_changed_port(devs, &new_port_id)) + goto err2; + + free(args); + *port_id = new_port_id; + return 0; +err2: + rte_eal_devargs_remove(RTE_DEVTYPE_VIRTUAL, args); +err1: + free(args); +err0: + RTE_LOG(ERR, EAL, "Driver, cannot attach the device\n"); + return -1; +} + +/* detach the new virtual device, then store the name of the device */ +static int +rte_eal_dev_detach_vdev(uint8_t port_id, char *vdevname) +{ + char name[RTE_ETH_NAME_MAX_LEN]; + + if (vdevname == NULL) + goto err; + + /* check whether the driver supports detach feature, or not */ + if (rte_eth_dev_check_detachable(port_id)) + goto err; + + /* get device name by port id */ + if (rte_eth_dev_get_name_by_port(port_id, name)) + goto err; + /* walk around dev_driver_list to find the driver of the device, + * then invoke close function o the driver */ + if (rte_eal_vdev_find_and_invoke(name, RTE_EAL_INVOKE_TYPE_CLOSE)) + goto err; + /* remove the vdevname from devargs_list */ + if (rte_eal_devargs_remove(RTE_DEVTYPE_VIRTUAL, name)) + goto err; + + strncpy(vdevname, name, sizeof(name)); + return 0; +err: + RTE_LOG(ERR, EAL, "Driver, cannot detach the device\n"); + return -1; +} + +/* attach the new device, then store port_id of the device */ +int +rte_eal_dev_attach(const char *devargs, uint8_t *port_id) +{ + struct rte_pci_addr addr; + + if ((devargs == NULL) || (port_id == NULL)) + return -EINVAL; + + if (eal_parse_pci_DomBDF(devargs, &addr) == 0) + return rte_eal_dev_attach_pdev(&addr, port_id); + else + return rte_eal_dev_attach_vdev(devargs, port_id); +} + +/* detach the device, then store the name of the device */ +int +rte_eal_dev_detach(uint8_t port_id, char *name) +{ + struct rte_pci_addr addr; + int ret; + + if (name == NULL) + return -EINVAL; + + if (rte_eth_dev_get_device_type(port_id) == RTE_ETH_DEV_PHYSICAL) { + ret = rte_eth_dev_get_addr_by_port(port_id, &addr); + if (ret < 0) + return ret; + + ret = rte_eal_dev_detach_pdev(port_id, &addr); + if (ret == 0) + snprintf(name, RTE_ETH_NAME_MAX_LEN, + "%04x:%02x:%02x.%d", + addr.domain, addr.bus, + addr.devid, addr.function); + + return ret; + } else + return rte_eal_dev_detach_vdev(port_id, name); +} +#else /* ENABLE_HOTPLUG */ +int +rte_eal_dev_attach(const char *devargs __rte_unused, + uint8_t *port_id __rte_unused) +{ + RTE_LOG(ERR, EAL, "Hotplug support isn't enabled\n"); + return -1; +} + +/* detach the device, then store the name of the device */ +int +rte_eal_dev_detach(uint8_t port_id __rte_unused, + char *name __rte_unused) +{ + RTE_LOG(ERR, EAL, "Hotplug support isn't enabled\n"); + return -1; +} +#endif /* ENABLE_HOTPLUG */ diff --git a/lib/librte_eal/common/eal_private.h b/lib/librte_eal/common/eal_private.h index 1a362ab..8168a7a 100644 --- a/lib/librte_eal/common/eal_private.h +++ b/lib/librte_eal/common/eal_private.h @@ -163,6 +163,17 @@ enum rte_eal_invoke_type { }; /** + * Scan the content of the PCI bus, and the devices in the devices + * list + * + * This function is private to EAL. + * + * @return + * 0 on success, negative on error + */ +int rte_eal_pci_scan(void); + +/** * Mmap memory for single PCI device * * This function is private to EAL. diff --git a/lib/librte_eal/common/include/rte_dev.h b/lib/librte_eal/common/include/rte_dev.h index f7e3a10..e63dd1c 100644 --- a/lib/librte_eal/common/include/rte_dev.h +++ b/lib/librte_eal/common/include/rte_dev.h @@ -47,6 +47,7 @@ extern "C" { #endif #include <sys/queue.h> +#include <rte_pci.h> /** Double linked list of device drivers. */ TAILQ_HEAD(rte_driver_list, rte_driver); @@ -57,6 +58,11 @@ TAILQ_HEAD(rte_driver_list, rte_driver); typedef int (rte_dev_init_t)(const char *name, const char *args); /** + * Uninitilization function called for each device driver once. + */ +typedef int (rte_dev_uninit_t)(const char *name, const char *args); + +/** * Driver type enumeration */ enum pmd_type { @@ -72,6 +78,7 @@ struct rte_driver { enum pmd_type type; /**< PMD Driver type */ const char *name; /**< Driver name. */ rte_dev_init_t *init; /**< Device init. function. */ + rte_dev_uninit_t *uninit; /**< Device uninit. function. */ }; /** @@ -93,6 +100,32 @@ void rte_eal_driver_register(struct rte_driver *driver); void rte_eal_driver_unregister(struct rte_driver *driver); /** + * Attach a new device. + * + * @param devargs + * A pointer to a strings array describing the new device + * to be attached. The strings should be a pci address like + * '0000:01:00.0' or virtual device name like 'eth_pcap0'. + * @param port_id + * A pointer to a port identifier actually attached. + * @return + * 0 on success and port_id is filled, negative on error + */ +int rte_eal_dev_attach(const char *devargs, uint8_t *port_id); + +/** + * Detach a device. + * + * @param port_id + * The port identifier of the device to detach. + * @param addr + * A pointer to a device name actually detached. + * @return + * 0 on success and devname is filled, negative on error + */ +int rte_eal_dev_detach(uint8_t port_id, char *devname); + +/** * Initalize all the registered drivers in this process */ int rte_eal_dev_init(void); diff --git a/lib/librte_eal/linuxapp/eal/Makefile b/lib/librte_eal/linuxapp/eal/Makefile index e117cec..b59b201 100644 --- a/lib/librte_eal/linuxapp/eal/Makefile +++ b/lib/librte_eal/linuxapp/eal/Makefile @@ -45,6 +45,7 @@ CFLAGS += -I$(RTE_SDK)/lib/librte_eal/common/include CFLAGS += -I$(RTE_SDK)/lib/librte_ring CFLAGS += -I$(RTE_SDK)/lib/librte_mempool CFLAGS += -I$(RTE_SDK)/lib/librte_malloc +CFLAGS += -I$(RTE_SDK)/lib/librte_mbuf CFLAGS += -I$(RTE_SDK)/lib/librte_ether CFLAGS += -I$(RTE_SDK)/lib/librte_ivshmem CFLAGS += -I$(RTE_SDK)/lib/librte_pmd_ring diff --git a/lib/librte_eal/linuxapp/eal/eal_pci.c b/lib/librte_eal/linuxapp/eal/eal_pci.c index 831422e..1f43688 100644 --- a/lib/librte_eal/linuxapp/eal/eal_pci.c +++ b/lib/librte_eal/linuxapp/eal/eal_pci.c @@ -431,8 +431,8 @@ error: * Scan the content of the PCI bus, and the devices in the devices * list */ -static int -pci_scan(void) +int +rte_eal_pci_scan(void) { struct dirent *e; DIR *dir; @@ -764,7 +764,7 @@ rte_eal_pci_init(void) if (internal_config.no_pci) return 0; - if (pci_scan() < 0) { + if (rte_eal_pci_scan() < 0) { RTE_LOG(ERR, EAL, "%s(): Cannot scan PCI bus\n", __func__); return -1; } -- 1.9.1
The patch enables CONFIG_RTE_LIBRTE_EAL_HOTPLUG in Linux configuration. Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp> --- config/common_linuxapp | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/config/common_linuxapp b/config/common_linuxapp index d428f84..81055f8 100644 --- a/config/common_linuxapp +++ b/config/common_linuxapp @@ -114,6 +114,11 @@ CONFIG_RTE_PCI_MAX_READ_REQUEST_SIZE=0 CONFIG_RTE_LIBRTE_EAL_LINUXAPP=y # +# Compile Environment Abstraction Layer to support hotplug +# +CONFIG_RTE_LIBRTE_EAL_HOTPLUG=y + +# # Compile Environment Abstraction Layer to support Vmware TSC map # CONFIG_RTE_LIBRTE_EAL_VMWARE_TSC_MAP_SUPPORT=y -- 1.9.1
This patch adds a new section for describing port hotplug framework. Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp> --- doc/guides/prog_guide/index.rst | 1 + doc/guides/prog_guide/port_hotplug_framework.rst | 110 +++++++++++++++++++++++ 2 files changed, 111 insertions(+) create mode 100644 doc/guides/prog_guide/port_hotplug_framework.rst diff --git a/doc/guides/prog_guide/index.rst b/doc/guides/prog_guide/index.rst index 8d86dd4..428b76b 100644 --- a/doc/guides/prog_guide/index.rst +++ b/doc/guides/prog_guide/index.rst @@ -70,6 +70,7 @@ Programmer's Guide packet_classif_access_ctrl packet_framework vhost_lib + port_hotplug_framework source_org dev_kit_build_system dev_kit_root_make_help diff --git a/doc/guides/prog_guide/port_hotplug_framework.rst b/doc/guides/prog_guide/port_hotplug_framework.rst new file mode 100644 index 0000000..355ae28 --- /dev/null +++ b/doc/guides/prog_guide/port_hotplug_framework.rst @@ -0,0 +1,110 @@ +.. BSD LICENSE + Copyright(c) 2015 IGEL Co.,Ltd. All rights reserved. + All rights reserved. + + Redistribution and use in source and binary forms, with or without + modification, are permitted provided that the following conditions + are met: + + * Redistributions of source code must retain the above copyright + notice, this list of conditions and the following disclaimer. + * Redistributions in binary form must reproduce the above copyright + notice, this list of conditions and the following disclaimer in + the documentation and/or other materials provided with the + distribution. + * Neither the name of IGEL Co.,Ltd. nor the names of its + contributors may be used to endorse or promote products derived + from this software without specific prior written permission. + + THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS + "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT + LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR + A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT + OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, + SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT + LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, + DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY + THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT + (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE + OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + +Port Hotplug Framework +====================== + +The Port Hotplug Framework provides DPDK applications with the ability to +attach and detach ports at runtime. Because the framework depends on PMD +implementation, the ports that PMDs cannot handle are out of scope of this +framework. Furthermore, after detaching a port from a DPDK application, the +framework doesn't provide a way for removing the devices from the system. +For the ports backed by a physical NIC, the kernel will need to support PCI +Hotplug feature. + +Overview +-------- + +The basic requirements of the Port Hotplug Framework are: + +* DPDK applications that use the Port Hotplug Framework must manage their + own ports. + + The Port Hotplug Framework is implemented to allow DPDK applications to + manage ports. For example, when DPDK applications call the port attach + function, the attached port number is returned. DPDK applications can + also detach the port by port number. + +* Kernel support is needed for attaching or detaching physical device + ports. + + To attach new physical device ports, the device will be recognized by + userspace driver I/O framework in kernel at first. Then DPDK + applications can call the Port Hotplug functions to attach the ports. + For detaching, steps are vice versa. + +* Before detaching, they must be stopped and closed. + + DPDK applications must call "rte_eth_dev_stop()" and + "rte_eth_dev_close()" APIs before detaching ports. These functions will + start finalization sequence of the PMDs. + +* The framework doesn't affect legacy DPDK applications behavior. + + If the Port Hotplug functions aren't called, all legacy DPDK apps can + still work without modifications. + +Port Hotplug API overview +------------------------- + +* Attaching a port + + "rte_eal_dev_attach()" API attaches a port to DPDK application, and + returns the attached port number. Before calling the API, the device + should be recognized by an userspace driver I/O framework. The API + receives a pci address like "0000:01:00.0" or a virtual device name + like "eth_pcap0,iface=eth0". In the case of virtual device name, the + format is the same as the general "--vdev" option of DPDK. + +* Detaching a port + + "rte_eal_dev_detach()" API detaches a port from DPDK application, and + returns a pci address of the detached device or a virtual device name + of the device. + +Reference +--------- + + "testpmd" supports the Port Hotplug Framework. + +Limitations +----------- + +* The Port Hotplug APIs are not thread safe. + +* The framework can only be enabled with Linux. BSD is not supported. + +* To detach a port, the port should be backed by a device that igb_uio + manages. VFIO is not supported. + +* Not all PMDs support detaching feature. + To know whether a PMD can support detaching, search for the + "RTE_PCI_DRV_DETACHABLE" flag in PMD implementation. If the flag is + defined in the PMD, detaching is supported. -- 1.9.1
This patch adds finalization code to free resources allocated by the PMD. v6: - Fix a paramter of rte_eth_dev_free(). v4: - Change function name. Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp> --- lib/librte_pmd_pcap/rte_eth_pcap.c | 40 ++++++++++++++++++++++++++++++++++++++ 1 file changed, 40 insertions(+) diff --git a/lib/librte_pmd_pcap/rte_eth_pcap.c b/lib/librte_pmd_pcap/rte_eth_pcap.c index af7fae8..5f88efd 100644 --- a/lib/librte_pmd_pcap/rte_eth_pcap.c +++ b/lib/librte_pmd_pcap/rte_eth_pcap.c @@ -498,6 +498,13 @@ static struct eth_dev_ops ops = { .stats_reset = eth_stats_reset, }; +static struct eth_driver rte_pcap_pmd = { + .pci_drv = { + .name = "rte_pcap_pmd", + .drv_flags = RTE_PCI_DRV_DETACHABLE, + }, +}; + /* * Function handler that opens the pcap file for reading a stores a * reference of it for use it later on. @@ -713,6 +720,10 @@ rte_pmd_init_internals(const char *name, const unsigned nb_rx_queues, if (*eth_dev == NULL) goto error; + /* check length of device name */ + if ((strlen((*eth_dev)->data->name) + 1) > sizeof(data->name)) + goto error; + /* now put it all together * - store queue data in internals, * - store numa_node info in pci_driver @@ -739,10 +750,13 @@ rte_pmd_init_internals(const char *name, const unsigned nb_rx_queues, data->nb_tx_queues = (uint16_t)nb_tx_queues; data->dev_link = pmd_link; data->mac_addrs = ð_addr; + strncpy(data->name, + (*eth_dev)->data->name, strlen((*eth_dev)->data->name)); (*eth_dev)->data = data; (*eth_dev)->dev_ops = &ops; (*eth_dev)->pci_dev = pci_dev; + (*eth_dev)->driver = &rte_pcap_pmd; return 0; @@ -927,10 +941,36 @@ rte_pmd_pcap_devinit(const char *name, const char *params) } +static int +rte_pmd_pcap_devuninit(const char *name, const char *params __rte_unused) +{ + struct rte_eth_dev *eth_dev = NULL; + + RTE_LOG(INFO, PMD, "Closing pcap ethdev on numa socket %u\n", + rte_socket_id()); + + if (name == NULL) + return -1; + + /* reserve an ethdev entry */ + eth_dev = rte_eth_dev_allocated(name); + if (eth_dev == NULL) + return -1; + + rte_free(eth_dev->data->dev_private); + rte_free(eth_dev->data); + rte_free(eth_dev->pci_dev); + + rte_eth_dev_free(eth_dev); + + return 0; +} + static struct rte_driver pmd_pcap_drv = { .name = "eth_pcap", .type = PMD_VDEV, .init = rte_pmd_pcap_devinit, + .uninit = rte_pmd_pcap_devuninit, }; PMD_REGISTER_DRIVER(pmd_pcap_drv); -- 1.9.1
The patch introduces following commands. - port attach [ident] - port detach [port_id] - attach: attaching a port - detach: detaching a port - ident: pci address of physical device. Or device name and parameters of virtual device. (ex. 0000:02:00.0, eth_pcap0,iface=eth0) - port_id: port identifier v7: - Fix doc. (Thanks to Iremonger, Bernard) - Fix port checking implementation of star_port(); (Thanks to Qiu, Michael) v5: - Add testpmd documentation. (Thanks to Iremonger, Bernard) v4: - Fix strings of command help. Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp> --- app/test-pmd/cmdline.c | 133 +++++++++++++++---- app/test-pmd/config.c | 116 +++++++++------- app/test-pmd/parameters.c | 22 ++- app/test-pmd/testpmd.c | 199 +++++++++++++++++++++------- app/test-pmd/testpmd.h | 18 ++- doc/guides/testpmd_app_ug/testpmd_funcs.rst | 57 ++++++++ 6 files changed, 415 insertions(+), 130 deletions(-) diff --git a/app/test-pmd/cmdline.c b/app/test-pmd/cmdline.c index 590e427..a4ca914 100644 --- a/app/test-pmd/cmdline.c +++ b/app/test-pmd/cmdline.c @@ -573,6 +573,12 @@ static void cmd_help_long_parsed(void *parsed_result, "port close (port_id|all)\n" " Close all ports or port_id.\n\n" + "port attach (ident)\n" + " Attach physical or virtual dev by pci address or virtual device name\n\n" + + "port detach (port_id)\n" + " Detach physical or virtual dev by port_id\n\n" + "port config (port_id|all)" " speed (10|100|1000|10000|40000|auto)" " duplex (half|full|auto)\n" @@ -864,6 +870,89 @@ cmdline_parse_inst_t cmd_operate_specific_port = { }, }; +/* *** attach a specified port *** */ +struct cmd_operate_attach_port_result { + cmdline_fixed_string_t port; + cmdline_fixed_string_t keyword; + cmdline_fixed_string_t identifier; +}; + +static void cmd_operate_attach_port_parsed(void *parsed_result, + __attribute__((unused)) struct cmdline *cl, + __attribute__((unused)) void *data) +{ + struct cmd_operate_attach_port_result *res = parsed_result; + + if (!strcmp(res->keyword, "attach")) + attach_port(res->identifier); + else + printf("Unknown parameter\n"); +} + +cmdline_parse_token_string_t cmd_operate_attach_port_port = + TOKEN_STRING_INITIALIZER(struct cmd_operate_attach_port_result, + port, "port"); +cmdline_parse_token_string_t cmd_operate_attach_port_keyword = + TOKEN_STRING_INITIALIZER(struct cmd_operate_attach_port_result, + keyword, "attach"); +cmdline_parse_token_string_t cmd_operate_attach_port_identifier = + TOKEN_STRING_INITIALIZER(struct cmd_operate_attach_port_result, + identifier, NULL); + +cmdline_parse_inst_t cmd_operate_attach_port = { + .f = cmd_operate_attach_port_parsed, + .data = NULL, + .help_str = "port attach identifier, " + "identifier: pci address or virtual dev name", + .tokens = { + (void *)&cmd_operate_attach_port_port, + (void *)&cmd_operate_attach_port_keyword, + (void *)&cmd_operate_attach_port_identifier, + NULL, + }, +}; + +/* *** detach a specified port *** */ +struct cmd_operate_detach_port_result { + cmdline_fixed_string_t port; + cmdline_fixed_string_t keyword; + uint8_t port_id; +}; + +static void cmd_operate_detach_port_parsed(void *parsed_result, + __attribute__((unused)) struct cmdline *cl, + __attribute__((unused)) void *data) +{ + struct cmd_operate_detach_port_result *res = parsed_result; + + if (!strcmp(res->keyword, "detach")) + detach_port(res->port_id); + else + printf("Unknown parameter\n"); +} + +cmdline_parse_token_string_t cmd_operate_detach_port_port = + TOKEN_STRING_INITIALIZER(struct cmd_operate_detach_port_result, + port, "port"); +cmdline_parse_token_string_t cmd_operate_detach_port_keyword = + TOKEN_STRING_INITIALIZER(struct cmd_operate_detach_port_result, + keyword, "detach"); +cmdline_parse_token_num_t cmd_operate_detach_port_port_id = + TOKEN_NUM_INITIALIZER(struct cmd_operate_detach_port_result, + port_id, UINT8); + +cmdline_parse_inst_t cmd_operate_detach_port = { + .f = cmd_operate_detach_port_parsed, + .data = NULL, + .help_str = "port detach port_id", + .tokens = { + (void *)&cmd_operate_detach_port_port, + (void *)&cmd_operate_detach_port_keyword, + (void *)&cmd_operate_detach_port_port_id, + NULL, + }, +}; + /* *** configure speed for all ports *** */ struct cmd_config_speed_all { cmdline_fixed_string_t port; @@ -918,7 +1007,7 @@ cmd_config_speed_all_parsed(void *parsed_result, return; } - for (pid = 0; pid < nb_ports; pid++) { + FOREACH_PORT(pid, ports) { ports[pid].dev_conf.link_speed = link_speed; ports[pid].dev_conf.link_duplex = link_duplex; } @@ -986,10 +1075,8 @@ cmd_config_speed_specific_parsed(void *parsed_result, return; } - if (res->id >= nb_ports) { - printf("Port id %d must be less than %d\n", res->id, nb_ports); + if (port_id_is_invalid(res->id, ENABLED_WARN)) return; - } if (!strcmp(res->value1, "10")) link_speed = ETH_LINK_SPEED_10; @@ -1549,7 +1636,7 @@ cmd_config_rxtx_queue_parsed(void *parsed_result, return; } - if (port_id_is_invalid(res->portid)) + if (port_id_is_invalid(res->portid, ENABLED_WARN)) return; if (port_is_started(res->portid) != 1) { @@ -2892,7 +2979,7 @@ cmd_tx_cksum_parsed(void *parsed_result, uint16_t ol_flags, mask = 0; struct rte_eth_dev_info dev_info; - if (port_id_is_invalid(res->port_id)) { + if (port_id_is_invalid(res->port_id, ENABLED_WARN)) { printf("invalid port %d\n", res->port_id); return; } @@ -3019,7 +3106,7 @@ cmd_tso_set_parsed(void *parsed_result, struct cmd_tso_set_result *res = parsed_result; struct rte_eth_dev_info dev_info; - if (port_id_is_invalid(res->port_id)) + if (port_id_is_invalid(res->port_id, ENABLED_WARN)) return; if (!strcmp(res->mode, "set")) @@ -3995,10 +4082,8 @@ static void cmd_set_bond_mac_addr_parsed(void *parsed_result, struct cmd_set_bond_mac_addr_result *res = parsed_result; int ret; - if (res->port_num >= nb_ports) { - printf("Port id %d must be less than %d\n", res->port_num, nb_ports); + if (port_id_is_invalid(res->port_num, ENABLED_WARN)) return; - } ret = rte_eth_bond_mac_address_set(res->port_num, &res->address); @@ -4235,7 +4320,7 @@ static void cmd_set_promisc_mode_parsed(void *parsed_result, /* all ports */ if (allports) { - for (i = 0; i < nb_ports; i++) { + FOREACH_PORT(i, ports) { if (enable) rte_eth_promiscuous_enable(i); else @@ -4315,7 +4400,7 @@ static void cmd_set_allmulti_mode_parsed(void *parsed_result, /* all ports */ if (allports) { - for (i = 0; i < nb_ports; i++) { + FOREACH_PORT(i, ports) { if (enable) rte_eth_allmulticast_enable(i); else @@ -5500,25 +5585,25 @@ static void cmd_showportall_parsed(void *parsed_result, struct cmd_showportall_result *res = parsed_result; if (!strcmp(res->show, "clear")) { if (!strcmp(res->what, "stats")) - for (i = 0; i < nb_ports; i++) + FOREACH_PORT(i, ports) nic_stats_clear(i); else if (!strcmp(res->what, "xstats")) - for (i = 0; i < nb_ports; i++) + FOREACH_PORT(i, ports) nic_xstats_clear(i); } else if (!strcmp(res->what, "info")) - for (i = 0; i < nb_ports; i++) + FOREACH_PORT(i, ports) port_infos_display(i); else if (!strcmp(res->what, "stats")) - for (i = 0; i < nb_ports; i++) + FOREACH_PORT(i, ports) nic_stats_display(i); else if (!strcmp(res->what, "xstats")) - for (i = 0; i < nb_ports; i++) + FOREACH_PORT(i, ports) nic_xstats_display(i); else if (!strcmp(res->what, "fdir")) - for (i = 0; i < nb_ports; i++) + FOREACH_PORT(i, ports) fdir_get_infos(i); else if (!strcmp(res->what, "stat_qmap")) - for (i = 0; i < nb_ports; i++) + FOREACH_PORT(i, ports) nic_stats_mapping_display(i); } @@ -9085,6 +9170,8 @@ cmdline_parse_ctx_t main_ctx[] = { (cmdline_parse_inst_t *)&cmd_set_qmap, (cmdline_parse_inst_t *)&cmd_operate_port, (cmdline_parse_inst_t *)&cmd_operate_specific_port, + (cmdline_parse_inst_t *)&cmd_operate_attach_port, + (cmdline_parse_inst_t *)&cmd_operate_detach_port, (cmdline_parse_inst_t *)&cmd_config_speed_all, (cmdline_parse_inst_t *)&cmd_config_speed_specific, (cmdline_parse_inst_t *)&cmd_config_rx_tx, @@ -9163,7 +9250,7 @@ prompt(void) static void cmd_reconfig_device_queue(portid_t id, uint8_t dev, uint8_t queue) { - if (id < nb_ports) { + if (!port_id_is_invalid(id, DISABLED_WARN)) { /* check if need_reconfig has been set to 1 */ if (ports[id].need_reconfig == 0) ports[id].need_reconfig = dev; @@ -9173,7 +9260,7 @@ cmd_reconfig_device_queue(portid_t id, uint8_t dev, uint8_t queue) } else { portid_t pid; - for (pid = 0; pid < nb_ports; pid++) { + FOREACH_PORT(pid, ports) { /* check if need_reconfig has been set to 1 */ if (ports[pid].need_reconfig == 0) ports[pid].need_reconfig = dev; @@ -9191,10 +9278,8 @@ bypass_is_supported(portid_t port_id) struct rte_port *port; struct rte_pci_id *pci_id; - if (port_id >= nb_ports) { - printf("\tPort id must be less than %d.\n", nb_ports); + if (port_id_is_invalid(port_id, ENABLED_WARN)) return 0; - } /* Get the device id. */ port = &ports[port_id]; diff --git a/app/test-pmd/config.c b/app/test-pmd/config.c index c40f819..32d8f9a 100644 --- a/app/test-pmd/config.c +++ b/app/test-pmd/config.c @@ -124,11 +124,15 @@ nic_stats_display(portid_t port_id) struct rte_eth_stats stats; struct rte_port *port = &ports[port_id]; uint8_t i; + portid_t pid; static const char *nic_stats_border = "########################"; - if (port_id >= nb_ports) { - printf("Invalid port, range is [0, %d]\n", nb_ports - 1); + if (port_id_is_invalid(port_id, ENABLED_WARN)) { + printf("Valid port range is [0"); + FOREACH_PORT(pid, ports) + printf(", %d", pid); + printf("]\n"); return; } rte_eth_stats_get(port_id, &stats); @@ -201,8 +205,13 @@ nic_stats_display(portid_t port_id) void nic_stats_clear(portid_t port_id) { - if (port_id >= nb_ports) { - printf("Invalid port, range is [0, %d]\n", nb_ports - 1); + portid_t pid; + + if (port_id_is_invalid(port_id, ENABLED_WARN)) { + printf("Valid port range is [0"); + FOREACH_PORT(pid, ports) + printf(", %d", pid); + printf("]\n"); return; } rte_eth_stats_reset(port_id); @@ -249,11 +258,15 @@ nic_stats_mapping_display(portid_t port_id) { struct rte_port *port = &ports[port_id]; uint16_t i; + portid_t pid; static const char *nic_stats_mapping_border = "########################"; - if (port_id >= nb_ports) { - printf("Invalid port, range is [0, %d]\n", nb_ports - 1); + if (port_id_is_invalid(port_id, ENABLED_WARN)) { + printf("Valid port range is [0"); + FOREACH_PORT(pid, ports) + printf(", %d", pid); + printf("]\n"); return; } @@ -302,9 +315,13 @@ port_infos_display(portid_t port_id) int vlan_offload; struct rte_mempool * mp; static const char *info_border = "*********************"; + portid_t pid; - if (port_id >= nb_ports) { - printf("Invalid port, range is [0, %d]\n", nb_ports - 1); + if (port_id_is_invalid(port_id, ENABLED_WARN)) { + printf("Valid port range is [0"); + FOREACH_PORT(pid, ports) + printf(", %d", pid); + printf("]\n"); return; } port = &ports[port_id]; @@ -362,11 +379,14 @@ port_infos_display(portid_t port_id) } int -port_id_is_invalid(portid_t port_id) +port_id_is_invalid(portid_t port_id, enum print_warning warning) { - if (port_id < nb_ports) + if (ports[port_id].enabled) return 0; - printf("Invalid port %d (must be < nb_ports=%d)\n", port_id, nb_ports); + + if (warning == ENABLED_WARN) + printf("Invalid port %d\n", port_id); + return 1; } @@ -425,7 +445,7 @@ port_reg_bit_display(portid_t port_id, uint32_t reg_off, uint8_t bit_x) uint32_t reg_v; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; if (port_reg_off_is_invalid(port_id, reg_off)) return; @@ -444,7 +464,7 @@ port_reg_bit_field_display(portid_t port_id, uint32_t reg_off, uint8_t l_bit; uint8_t h_bit; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; if (port_reg_off_is_invalid(port_id, reg_off)) return; @@ -471,7 +491,7 @@ port_reg_display(portid_t port_id, uint32_t reg_off) { uint32_t reg_v; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; if (port_reg_off_is_invalid(port_id, reg_off)) return; @@ -485,7 +505,7 @@ port_reg_bit_set(portid_t port_id, uint32_t reg_off, uint8_t bit_pos, { uint32_t reg_v; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; if (port_reg_off_is_invalid(port_id, reg_off)) return; @@ -513,7 +533,7 @@ port_reg_bit_field_set(portid_t port_id, uint32_t reg_off, uint8_t l_bit; uint8_t h_bit; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; if (port_reg_off_is_invalid(port_id, reg_off)) return; @@ -547,7 +567,7 @@ port_reg_bit_field_set(portid_t port_id, uint32_t reg_off, void port_reg_set(portid_t port_id, uint32_t reg_off, uint32_t reg_v) { - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; if (port_reg_off_is_invalid(port_id, reg_off)) return; @@ -560,7 +580,7 @@ port_mtu_set(portid_t port_id, uint16_t mtu) { int diag; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; diag = rte_eth_dev_set_mtu(port_id, mtu); if (diag == 0) @@ -723,7 +743,7 @@ rx_ring_desc_display(portid_t port_id, queueid_t rxq_id, uint16_t rxd_id) { const struct rte_memzone *rx_mz; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; if (rx_queue_id_is_invalid(rxq_id)) return; @@ -740,7 +760,7 @@ tx_ring_desc_display(portid_t port_id, queueid_t txq_id, uint16_t txd_id) { const struct rte_memzone *tx_mz; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; if (tx_queue_id_is_invalid(txq_id)) return; @@ -796,7 +816,7 @@ port_rss_reta_info(portid_t port_id, uint16_t i, idx, shift; int ret; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; ret = rte_eth_dev_rss_reta_query(port_id, reta_conf, nb_entries); @@ -828,7 +848,7 @@ port_rss_hash_conf_show(portid_t port_id, int show_rss_key) uint8_t i; int diag; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; /* Get RSS hash key if asked to display it */ rss_conf.rss_key = (show_rss_key) ? rss_key : NULL; @@ -1406,12 +1426,8 @@ set_fwd_ports_list(unsigned int *portlist, unsigned int nb_pt) again: for (i = 0; i < nb_pt; i++) { port_id = (portid_t) portlist[i]; - if (port_id >= nb_ports) { - printf("Invalid port id %u >= %u\n", - (unsigned int) port_id, - (unsigned int) nb_ports); + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; - } if (record_now) fwd_ports_ids[i] = port_id; } @@ -1569,7 +1585,7 @@ vlan_extend_set(portid_t port_id, int on) int diag; int vlan_offload; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; vlan_offload = rte_eth_dev_get_vlan_offload(port_id); @@ -1591,7 +1607,7 @@ rx_vlan_strip_set(portid_t port_id, int on) int diag; int vlan_offload; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; vlan_offload = rte_eth_dev_get_vlan_offload(port_id); @@ -1612,7 +1628,7 @@ rx_vlan_strip_set_on_queue(portid_t port_id, uint16_t queue_id, int on) { int diag; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; diag = rte_eth_dev_set_vlan_strip_on_queue(port_id, queue_id, on); @@ -1627,7 +1643,7 @@ rx_vlan_filter_set(portid_t port_id, int on) int diag; int vlan_offload; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; vlan_offload = rte_eth_dev_get_vlan_offload(port_id); @@ -1648,7 +1664,7 @@ rx_vft_set(portid_t port_id, uint16_t vlan_id, int on) { int diag; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; if (vlan_id_is_invalid(vlan_id)) return; @@ -1665,7 +1681,7 @@ rx_vlan_all_filter_set(portid_t port_id, int on) { uint16_t vlan_id; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; for (vlan_id = 0; vlan_id < 4096; vlan_id++) rx_vft_set(port_id, vlan_id, on); @@ -1675,7 +1691,7 @@ void vlan_tpid_set(portid_t port_id, uint16_t tp_id) { int diag; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; diag = rte_eth_dev_set_vlan_ether_type(port_id, tp_id); @@ -1690,7 +1706,7 @@ vlan_tpid_set(portid_t port_id, uint16_t tp_id) void tx_vlan_set(portid_t port_id, uint16_t vlan_id) { - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; if (vlan_id_is_invalid(vlan_id)) return; @@ -1701,7 +1717,7 @@ tx_vlan_set(portid_t port_id, uint16_t vlan_id) void tx_vlan_reset(portid_t port_id) { - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; ports[port_id].tx_ol_flags &= ~TESTPMD_TX_OFFLOAD_INSERT_VLAN; } @@ -1709,7 +1725,7 @@ tx_vlan_reset(portid_t port_id) void tx_vlan_pvid_set(portid_t port_id, uint16_t vlan_id, int on) { - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; rte_eth_dev_set_vlan_pvid(port_id, vlan_id, on); @@ -1721,7 +1737,7 @@ set_qmap(portid_t port_id, uint8_t is_rx, uint16_t queue_id, uint8_t map_value) uint16_t i; uint8_t existing_mapping_found = 0; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; if (is_rx ? (rx_queue_id_is_invalid(queue_id)) : (tx_queue_id_is_invalid(queue_id))) @@ -1773,7 +1789,7 @@ fdir_add_signature_filter(portid_t port_id, uint8_t queue_id, { int diag; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; diag = rte_eth_dev_fdir_add_signature_filter(port_id, fdir_filter, @@ -1791,7 +1807,7 @@ fdir_update_signature_filter(portid_t port_id, uint8_t queue_id, { int diag; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; diag = rte_eth_dev_fdir_update_signature_filter(port_id, fdir_filter, @@ -1809,7 +1825,7 @@ fdir_remove_signature_filter(portid_t port_id, { int diag; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; diag = rte_eth_dev_fdir_remove_signature_filter(port_id, fdir_filter); @@ -1881,7 +1897,7 @@ fdir_get_infos(portid_t port_id) static const char *fdir_stats_border = "########################"; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; ret = rte_eth_dev_filter_supported(port_id, RTE_ETH_FILTER_FDIR); if (ret < 0) { @@ -1955,7 +1971,7 @@ fdir_add_perfect_filter(portid_t port_id, uint16_t soft_id, uint8_t queue_id, { int diag; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; diag = rte_eth_dev_fdir_add_perfect_filter(port_id, fdir_filter, @@ -1973,7 +1989,7 @@ fdir_update_perfect_filter(portid_t port_id, uint16_t soft_id, uint8_t queue_id, { int diag; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; diag = rte_eth_dev_fdir_update_perfect_filter(port_id, fdir_filter, @@ -1991,7 +2007,7 @@ fdir_remove_perfect_filter(portid_t port_id, uint16_t soft_id, { int diag; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; diag = rte_eth_dev_fdir_remove_perfect_filter(port_id, fdir_filter, @@ -2008,7 +2024,7 @@ fdir_set_masks(portid_t port_id, struct rte_fdir_masks *fdir_masks) { int diag; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; diag = rte_eth_dev_fdir_set_masks(port_id, fdir_masks); @@ -2085,7 +2101,7 @@ set_vf_traffic(portid_t port_id, uint8_t is_rx, uint16_t vf, uint8_t on) { int diag; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; if (is_rx) diag = rte_eth_dev_set_vf_rx(port_id,vf,on); @@ -2107,7 +2123,7 @@ set_vf_rx_vlan(portid_t port_id, uint16_t vlan_id, uint64_t vf_mask, uint8_t on) { int diag; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; if (vlan_id_is_invalid(vlan_id)) return; @@ -2124,7 +2140,7 @@ set_queue_rate_limit(portid_t port_id, uint16_t queue_idx, uint16_t rate) int diag; struct rte_eth_link link; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return 1; rte_eth_link_get_nowait(port_id, &link); if (rate > link.link_speed) { @@ -2149,7 +2165,7 @@ set_vf_rate_limit(portid_t port_id, uint16_t vf, uint16_t rate, uint64_t q_msk) if (q_msk == 0) return 0; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return 1; rte_eth_link_get_nowait(port_id, &link); if (rate > link.link_speed) { diff --git a/app/test-pmd/parameters.c b/app/test-pmd/parameters.c index adf3203..6f2af18 100644 --- a/app/test-pmd/parameters.c +++ b/app/test-pmd/parameters.c @@ -376,6 +376,7 @@ parse_portnuma_config(const char *q_arg) }; unsigned long int_fld[_NUM_FLD]; char *str_fld[_NUM_FLD]; + portid_t pid; /* reset from value set at definition */ while ((p = strchr(p0,'(')) != NULL) { @@ -397,8 +398,11 @@ parse_portnuma_config(const char *q_arg) return -1; } port_id = (uint8_t)int_fld[FLD_PORT]; - if (port_id >= nb_ports) { - printf("Invalid port, range is [0, %d]\n", nb_ports - 1); + if (port_id_is_invalid(port_id, ENABLED_WARN)) { + printf("Valid port range is [0"); + FOREACH_PORT(pid, ports) + printf(", %d", pid); + printf("]\n"); return -1; } socket_id = (uint8_t)int_fld[FLD_SOCKET]; @@ -429,6 +433,7 @@ parse_ringnuma_config(const char *q_arg) }; unsigned long int_fld[_NUM_FLD]; char *str_fld[_NUM_FLD]; + portid_t pid; #define RX_RING_ONLY 0x1 #define TX_RING_ONLY 0x2 #define RXTX_RING 0x3 @@ -453,8 +458,11 @@ parse_ringnuma_config(const char *q_arg) return -1; } port_id = (uint8_t)int_fld[FLD_PORT]; - if (port_id >= nb_ports) { - printf("Invalid port, range is [0, %d]\n", nb_ports - 1); + if (port_id_is_invalid(port_id, ENABLED_WARN)) { + printf("Valid port range is [0"); + FOREACH_PORT(pid, ports) + printf(", %d", pid); + printf("]\n"); return -1; } socket_id = (uint8_t)int_fld[FLD_SOCKET]; @@ -626,12 +634,12 @@ launch_args_parse(int argc, char** argv) #endif if (!strcmp(lgopts[opt_idx].name, "nb-ports")) { n = atoi(optarg); - if (n > 0 && n <= nb_ports) + if (n > 0 && + !port_id_is_invalid(n, DISABLED_WARN)) nb_fwd_ports = (uint8_t) n; else rte_exit(EXIT_FAILURE, - "nb-ports should be > 0 and <= %d\n", - nb_ports); + "Invalid port %d\n", n); } if (!strcmp(lgopts[opt_idx].name, "nb-cores")) { n = atoi(optarg); diff --git a/app/test-pmd/testpmd.c b/app/test-pmd/testpmd.c index 773b8af..cf99a7d 100644 --- a/app/test-pmd/testpmd.c +++ b/app/test-pmd/testpmd.c @@ -71,6 +71,7 @@ #include <rte_pci.h> #include <rte_ether.h> #include <rte_ethdev.h> +#include <rte_dev.h> #include <rte_string_fns.h> #ifdef RTE_LIBRTE_PMD_XENVIRT #include <rte_eth_xenvirt.h> @@ -315,7 +316,7 @@ uint16_t nb_rx_queue_stats_mappings = 0; /* Forward function declarations */ static void map_port_queue_stats_mapping_registers(uint8_t pi, struct rte_port *port); -static void check_all_ports_link_status(uint8_t port_num, uint32_t port_mask); +static void check_all_ports_link_status(uint32_t port_mask); /* * Check if all the ports are started. @@ -324,6 +325,20 @@ static void check_all_ports_link_status(uint8_t port_num, uint32_t port_mask); static int all_ports_started(void); /* + * Find next enabled port + */ +portid_t +find_next_port(portid_t p, struct rte_port *ports, int size) +{ + if (ports == NULL) + rte_exit(-EINVAL, "failed to find a next port id\n"); + + while ((ports[p].enabled == 0) && (p < size)) + p++; + return p; +} + +/* * Setup default configuration. */ static void @@ -552,7 +567,8 @@ init_config(void) + RTE_TEST_TX_DESC_MAX + MAX_PKT_BURST; if (!numa_support) - nb_mbuf_per_pool = (nb_mbuf_per_pool * nb_ports); + nb_mbuf_per_pool = + (nb_mbuf_per_pool * RTE_MAX_ETHPORTS); } if (!numa_support) { @@ -565,14 +581,19 @@ init_config(void) /* Configuration of Ethernet ports. */ ports = rte_zmalloc("testpmd: ports", - sizeof(struct rte_port) * nb_ports, + sizeof(struct rte_port) * RTE_MAX_ETHPORTS, RTE_CACHE_LINE_SIZE); if (ports == NULL) { - rte_exit(EXIT_FAILURE, "rte_zmalloc(%d struct rte_port) " - "failed\n", nb_ports); + rte_exit(EXIT_FAILURE, + "rte_zmalloc(%d struct rte_port) failed\n", + RTE_MAX_ETHPORTS); } - for (pid = 0; pid < nb_ports; pid++) { + /* enabled allocated ports */ + for (pid = 0; pid < nb_ports; pid++) + ports[pid].enabled = 1; + + FOREACH_PORT(pid, ports) { port = &ports[pid]; rte_eth_dev_info_get(pid, &port->dev_info); @@ -602,8 +623,7 @@ init_config(void) nb_mbuf_per_pool = nb_mbuf_per_pool/nb_ports; for (i = 0; i < MAX_SOCKET; i++) { - nb_mbuf = (nb_mbuf_per_pool * - port_per_socket[i]); + nb_mbuf = (nb_mbuf_per_pool * RTE_MAX_ETHPORTS); if (nb_mbuf) mbuf_pool_create(mbuf_data_size, nb_mbuf,i); @@ -635,14 +655,6 @@ reconfig(portid_t new_port_id, unsigned socket_id) struct rte_port *port; /* Reconfiguration of Ethernet ports. */ - ports = rte_realloc(ports, - sizeof(struct rte_port) * nb_ports, - RTE_CACHE_LINE_SIZE); - if (ports == NULL) { - rte_exit(EXIT_FAILURE, "rte_realloc(%d struct rte_port) failed\n", - nb_ports); - } - port = &ports[new_port_id]; rte_eth_dev_info_get(new_port_id, &port->dev_info); @@ -663,7 +675,7 @@ init_fwd_streams(void) streamid_t sm_id, nb_fwd_streams_new; /* set socket id according to numa or not */ - for (pid = 0; pid < nb_ports; pid++) { + FOREACH_PORT(pid, ports) { port = &ports[pid]; if (nb_rxq > port->dev_info.max_rx_queues) { printf("Fail: nb_rxq(%d) is greater than " @@ -1264,7 +1276,7 @@ all_ports_started(void) portid_t pi; struct rte_port *port; - for (pi = 0; pi < nb_ports; pi++) { + FOREACH_PORT(pi, ports) { port = &ports[pi]; /* Check if there is a port which is not started */ if (port->port_status != RTE_PORT_STARTED) @@ -1276,6 +1288,45 @@ all_ports_started(void) } int +all_ports_stopped(void) +{ + portid_t pi; + struct rte_port *port; + + FOREACH_PORT(pi, ports) { + port = &ports[pi]; + if (port->port_status != RTE_PORT_STOPPED) + return 0; + } + + return 1; +} + +int +port_is_started(portid_t port_id) +{ + if (port_id_is_invalid(port_id, ENABLED_WARN)) + return 0; + + if (ports[port_id].port_status != RTE_PORT_STARTED) + return 0; + + return 1; +} + +static int +port_is_closed(portid_t port_id) +{ + if (port_id_is_invalid(port_id, ENABLED_WARN)) + return 0; + + if (ports[port_id].port_status != RTE_PORT_CLOSED) + return 0; + + return 1; +} + +int start_port(portid_t pid) { int diag, need_check_link_status = 0; @@ -1296,8 +1347,8 @@ start_port(portid_t pid) if(dcb_config) dcb_test = 1; - for (pi = 0; pi < nb_ports; pi++) { - if (pid < nb_ports && pid != pi) + FOREACH_PORT(pi, ports) { + if (pid != pi && pid != (portid_t)RTE_PORT_ALL) continue; port = &ports[pi]; @@ -1421,7 +1472,7 @@ start_port(portid_t pid) } if (need_check_link_status && !no_link_check) - check_all_ports_link_status(nb_ports, RTE_PORT_ALL); + check_all_ports_link_status(RTE_PORT_ALL); else printf("Please stop the ports first\n"); @@ -1446,8 +1497,8 @@ stop_port(portid_t pid) } printf("Stopping ports...\n"); - for (pi = 0; pi < nb_ports; pi++) { - if (pid < nb_ports && pid != pi) + FOREACH_PORT(pi, ports) { + if (!port_id_is_invalid(pid, DISABLED_WARN) && pid != pi) continue; port = &ports[pi]; @@ -1463,7 +1514,7 @@ stop_port(portid_t pid) need_check_link_status = 1; } if (need_check_link_status && !no_link_check) - check_all_ports_link_status(nb_ports, RTE_PORT_ALL); + check_all_ports_link_status(RTE_PORT_ALL); printf("Done\n"); } @@ -1481,8 +1532,8 @@ close_port(portid_t pid) printf("Closing ports...\n"); - for (pi = 0; pi < nb_ports; pi++) { - if (pid < nb_ports && pid != pi) + FOREACH_PORT(pi, ports) { + if (!port_id_is_invalid(pid, DISABLED_WARN) && pid != pi) continue; port = &ports[pi]; @@ -1502,31 +1553,83 @@ close_port(portid_t pid) printf("Done\n"); } -int -all_ports_stopped(void) +void +attach_port(char *identifier) { - portid_t pi; - struct rte_port *port; + portid_t i, j, pi = 0; - for (pi = 0; pi < nb_ports; pi++) { - port = &ports[pi]; - if (port->port_status != RTE_PORT_STOPPED) - return 0; + printf("Attaching a new port...\n"); + + if (identifier == NULL) { + printf("Invalid parameters are specified\n"); + return; } - return 1; + if (test_done == 0) { + printf("Please stop forwarding first\n"); + return; + } + + if (rte_eal_dev_attach(identifier, &pi)) + return; + + ports[pi].enabled = 1; + reconfig(pi, rte_eth_dev_socket_id(pi)); + rte_eth_promiscuous_enable(pi); + + nb_ports = rte_eth_dev_count(); + + /* set_default_fwd_ports_config(); */ + bzero(fwd_ports_ids, sizeof(fwd_ports_ids)); + i = 0; + FOREACH_PORT(j, ports) { + fwd_ports_ids[i] = j; + i++; + } + nb_cfg_ports = nb_ports; + nb_fwd_ports++; + + ports[pi].port_status = RTE_PORT_STOPPED; + + printf("Port %d is attached. Now total ports is %d\n", pi, nb_ports); + printf("Done\n"); } -int -port_is_started(portid_t port_id) +void +detach_port(uint8_t port_id) { - if (port_id_is_invalid(port_id)) - return -1; + portid_t i, pi = 0; + char name[RTE_ETH_NAME_MAX_LEN]; - if (ports[port_id].port_status != RTE_PORT_STARTED) - return 0; + printf("Detaching a port...\n"); - return 1; + if (!port_is_closed(port_id)) { + printf("Please close port first\n"); + return; + } + + rte_eth_promiscuous_disable(port_id); + + if (rte_eal_dev_detach(port_id, name)) + return; + + ports[port_id].enabled = 0; + nb_ports = rte_eth_dev_count(); + + /* set_default_fwd_ports_config(); */ + bzero(fwd_ports_ids, sizeof(fwd_ports_ids)); + i = 0; + FOREACH_PORT(pi, ports) { + fwd_ports_ids[i] = pi; + i++; + } + nb_cfg_ports = nb_ports; + nb_fwd_ports--; + + printf("Port '%s' is detached. Now total ports is %d\n", + name, nb_ports); + printf("Done\n"); + return; } void @@ -1534,7 +1637,7 @@ pmd_test_exit(void) { portid_t pt_id; - for (pt_id = 0; pt_id < nb_ports; pt_id++) { + FOREACH_PORT(pt_id, ports) { printf("Stopping port %d...", pt_id); fflush(stdout); rte_eth_dev_close(pt_id); @@ -1553,7 +1656,7 @@ struct pmd_test_command { /* Check the link status of all ports in up to 9s, and print them finally */ static void -check_all_ports_link_status(uint8_t port_num, uint32_t port_mask) +check_all_ports_link_status(uint32_t port_mask) { #define CHECK_INTERVAL 100 /* 100ms */ #define MAX_CHECK_TIME 90 /* 9s (90 * 100ms) in total */ @@ -1564,7 +1667,7 @@ check_all_ports_link_status(uint8_t port_num, uint32_t port_mask) fflush(stdout); for (count = 0; count <= MAX_CHECK_TIME; count++) { all_ports_up = 1; - for (portid = 0; portid < port_num; portid++) { + FOREACH_PORT(portid, ports) { if ((port_mask & (1 << portid)) == 0) continue; memset(&link, 0, sizeof(link)); @@ -1688,7 +1791,7 @@ init_port_config(void) portid_t pid; struct rte_port *port; - for (pid = 0; pid < nb_ports; pid++) { + FOREACH_PORT(pid, ports) { port = &ports[pid]; port->dev_conf.rxmode = rx_mode; port->dev_conf.fdir_conf = fdir_conf; @@ -1877,7 +1980,7 @@ main(int argc, char** argv) nb_ports = (portid_t) rte_eth_dev_count(); if (nb_ports == 0) - rte_exit(EXIT_FAILURE, "No probed ethernet device\n"); + RTE_LOG(WARNING, EAL, "No probed ethernet devices\n"); set_def_fwd_config(); if (nb_lcores == 0) @@ -1899,7 +2002,7 @@ main(int argc, char** argv) rte_exit(EXIT_FAILURE, "Start ports failed\n"); /* set all ports to promiscuous mode by default */ - for (port_id = 0; port_id < nb_ports; port_id++) + FOREACH_PORT(port_id, ports) rte_eth_promiscuous_enable(port_id); #ifdef RTE_LIBRTE_CMDLINE diff --git a/app/test-pmd/testpmd.h b/app/test-pmd/testpmd.h index 8f5e6c7..109c670 100644 --- a/app/test-pmd/testpmd.h +++ b/app/test-pmd/testpmd.h @@ -134,6 +134,7 @@ struct fwd_stream { * The data structure associated with each port. */ struct rte_port { + uint8_t enabled; /**< Port enabled or not */ struct rte_eth_dev_info dev_info; /**< PCI info + driver name */ struct rte_eth_conf dev_conf; /**< Port configuration. */ struct ether_addr eth_addr; /**< Port ethernet address */ @@ -159,6 +160,14 @@ struct rte_port { struct rte_eth_txconf tx_conf; /**< tx configuration */ }; +extern portid_t __rte_unused +find_next_port(portid_t p, struct rte_port *ports, int size); + +#define FOREACH_PORT(p, ports) \ + for (p = find_next_port(0, ports, RTE_MAX_ETHPORTS); \ + p < RTE_MAX_ETHPORTS; \ + p = find_next_port(p + 1, ports, RTE_MAX_ETHPORTS)) + /** * The data structure associated with each forwarding logical core. * The logical cores are internally numbered by a core index from 0 to @@ -515,6 +524,8 @@ int init_port_dcb_config(portid_t pid,struct dcb_config *dcb_conf); int start_port(portid_t pid); void stop_port(portid_t pid); void close_port(portid_t pid); +void attach_port(char *identifier); +void detach_port(uint8_t port_id); int all_ports_stopped(void); int port_is_started(portid_t port_id); void pmd_test_exit(void); @@ -558,10 +569,15 @@ void get_ethertype_filter(uint8_t port_id, uint16_t index); void get_2tuple_filter(uint8_t port_id, uint16_t index); void get_5tuple_filter(uint8_t port_id, uint16_t index); void get_flex_filter(uint8_t port_id, uint16_t index); -int port_id_is_invalid(portid_t port_id); int rx_queue_id_is_invalid(queueid_t rxq_id); int tx_queue_id_is_invalid(queueid_t txq_id); +enum print_warning { + ENABLED_WARN = 0, + DISABLED_WARN +}; +int port_id_is_invalid(portid_t port_id, enum print_warning warning); + /* * Work-around of a compilation error with ICC on invocations of the * rte_be_to_cpu_16() function. diff --git a/doc/guides/testpmd_app_ug/testpmd_funcs.rst b/doc/guides/testpmd_app_ug/testpmd_funcs.rst index 218835a..936f9e0 100644 --- a/doc/guides/testpmd_app_ug/testpmd_funcs.rst +++ b/doc/guides/testpmd_app_ug/testpmd_funcs.rst @@ -808,6 +808,63 @@ The following sections show functions for configuring ports. Port configuration changes only become active when forwarding is started/restarted. +port attach +~~~~~~~~~~~ + +Attach a port specified by pci address or virtual device args. + +To attach a new pci device, the device should be recognized by kernel first. +Then it should be moved under DPDK management. +Finally the port can be attached to testpmd. +On the other hand, to attach a port created by virtual device, above steps are not needed. + +port attach (identifier) + +For example, to attach a port whose pci address is 0000:02:00.0. + +.. code-block:: console + + testpmd> port attach 0000:02:00.0 + Attaching a new port... + ... snip ... + Port 0 is attached. Now total ports is 1 + Done + +For example, to attach a port created by pcap PMD. + +.. code-block:: console + + testpmd> port attach eth_pcap0,iface=eth0 + Attaching a new port... + ... snip ... + Port 0 is attached. Now total ports is 1 + Done + +In this case, identifier is "eth_pcap0,iface=eth0". +This identifier format is the same as "--vdev" format of DPDK applications. + +port detach +~~~~~~~~~~~ + +Detach a specific port. + +Before detaching a port, the port should be closed. +Also to remove a pci device completely from the system, first detach the port from testpmd. +Then the device should be moved under kernel management. +Finally the device can be removed using kernel pci hotplug functionality. +On the other hand, to remove a port created by a virtual device, above steps are not needed. + +port detach (port_id) + +For example, to detach a port 0. + +.. code-block:: console + + testpmd> port detach 0 + Detaching a port... + ... snip ... + Done + port start ~~~~~~~~~~ -- 1.9.1
On 2/9/2015 4:31 PM, Tetsuya Mukawa wrote: > To remove assumption, do like followings. [...] > struct rte_eth_dev_sriov { > @@ -1604,6 +1605,10 @@ extern struct rte_eth_dev rte_eth_devices[]; > * initialized by the [matching] Ethernet driver during the PCI probing phase. > * All devices whose port identifier is in the range > * [0, rte_eth_dev_count() - 1] can be operated on by network applications. > + * immediately after invoking rte_eal_init(). > + * If the application unplugs a port using hotplug function, The enabled port > + * numbers may be noncontiguous. In the case, the applications need to manage > + * enabled port by themselves. > * > * @return > * - The total number of usable Ethernet devices. Acked-by: Michael Qiu <michael.qiu@intel.com>
On 2/9/2015 4:31 PM, Tetsuya Mukawa wrote:
> This patch replaces pci_addr_comparison() and memcmp() of pci addresses by
> eal_compare_pci_addr().
>
> v5:
> - Fix pci_scan_one to handle pt_driver correctly.
> v4:
> - Fix calculation method of eal_compare_pci_addr().
> - Add parameter checking.
>
> Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp>
> ---
> lib/librte_eal/bsdapp/eal/eal_pci.c | 25 ++++++++---------------
> lib/librte_eal/common/eal_common_pci.c | 2 +-
> lib/librte_eal/common/include/rte_pci.h | 34 +++++++++++++++++++++++++++++++
> lib/librte_eal/linuxapp/eal/eal_pci.c | 25 ++++++++---------------
> lib/librte_eal/linuxapp/eal/eal_pci_uio.c | 2 +-
> 5 files changed, 54 insertions(+), 34 deletions(-)
>
> diff --git a/lib/librte_eal/bsdapp/eal/eal_pci.c b/lib/librte_eal/bsdapp/eal/eal_pci.c
> index 74ecce7..c844d58 100644
> --- a/lib/librte_eal/bsdapp/eal/eal_pci.c
> +++ b/lib/librte_eal/bsdapp/eal/eal_pci.c
> @@ -270,20 +270,6 @@ pci_uio_map_resource(struct rte_pci_device *dev)
> return (0);
> }
>
> -/* Compare two PCI device addresses. */
> -static int
> -pci_addr_comparison(struct rte_pci_addr *addr, struct rte_pci_addr *addr2)
> -{
> - uint64_t dev_addr = (addr->domain << 24) + (addr->bus << 16) + (addr->devid << 8) + addr->function;
> - uint64_t dev_addr2 = (addr2->domain << 24) + (addr2->bus << 16) + (addr2->devid << 8) + addr2->function;
> -
> - if (dev_addr > dev_addr2)
> - return 1;
> - else
> - return 0;
> -}
> -
> -
> /* Scan one pci sysfs entry, and fill the devices list from it. */
> static int
> pci_scan_one(int dev_pci_fd, struct pci_conf *conf)
> @@ -356,13 +342,20 @@ pci_scan_one(int dev_pci_fd, struct pci_conf *conf)
> }
> else {
> struct rte_pci_device *dev2 = NULL;
> + int ret;
>
> TAILQ_FOREACH(dev2, &pci_device_list, next) {
> - if (pci_addr_comparison(&dev->addr, &dev2->addr))
> + ret = eal_compare_pci_addr(&dev->addr, &dev2->addr);
> + if (ret > 0)
> continue;
> - else {
> + else if (ret < 0) {
> TAILQ_INSERT_BEFORE(dev2, dev, next);
> return 0;
> + } else { /* already registered */
> + /* update pt_driver */
> + dev2->pt_driver = dev->pt_driver;
> + free(dev);
> + return 0;
> }
> }
> TAILQ_INSERT_TAIL(&pci_device_list, dev, next);
> diff --git a/lib/librte_eal/common/eal_common_pci.c b/lib/librte_eal/common/eal_common_pci.c
> index f3c7f71..a89f5c3 100644
> --- a/lib/librte_eal/common/eal_common_pci.c
> +++ b/lib/librte_eal/common/eal_common_pci.c
> @@ -93,7 +93,7 @@ static struct rte_devargs *pci_devargs_lookup(struct rte_pci_device *dev)
> if (devargs->type != RTE_DEVTYPE_BLACKLISTED_PCI &&
> devargs->type != RTE_DEVTYPE_WHITELISTED_PCI)
> continue;
> - if (!memcmp(&dev->addr, &devargs->pci.addr, sizeof(dev->addr)))
> + if (!eal_compare_pci_addr(&dev->addr, &devargs->pci.addr))
> return devargs;
> }
> return NULL;
> diff --git a/lib/librte_eal/common/include/rte_pci.h b/lib/librte_eal/common/include/rte_pci.h
> index 7f2d699..4814cd7 100644
> --- a/lib/librte_eal/common/include/rte_pci.h
> +++ b/lib/librte_eal/common/include/rte_pci.h
> @@ -269,6 +269,40 @@ eal_parse_pci_DomBDF(const char *input, struct rte_pci_addr *dev_addr)
> }
> #undef GET_PCIADDR_FIELD
>
> +/* Compare two PCI device addresses. */
> +/**
> + * Utility function to compare two PCI device addresses.
> + *
> + * @param addr
> + * The PCI Bus-Device-Function address to compare
> + * @param addr2
> + * The PCI Bus-Device-Function address to compare
> + * @return
> + * 0 on equal PCI address.
> + * Positive on addr is greater than addr2.
> + * Negative on addr is less than addr2, or error.
> + */
> +static inline int
> +eal_compare_pci_addr(struct rte_pci_addr *addr, struct rte_pci_addr *addr2)
> +{
> + uint64_t dev_addr, dev_addr2;
> +
> + if ((addr == NULL) || (addr2 == NULL))
> + return -1;
> +
> + dev_addr = (addr->domain << 24) | (addr->bus << 16) |
> + (addr->devid << 8) | addr->function;
> + dev_addr2 = (addr2->domain << 24) | (addr2->bus << 16) |
> + (addr2->devid << 8) | addr2->function;
> +
> + if (dev_addr > dev_addr2)
> + return 1;
> + else if (dev_addr < dev_addr2)
> + return -1;
> + else
> + return 0;
> +}
> +
> /**
> * Probe the PCI bus for registered drivers.
> *
> diff --git a/lib/librte_eal/linuxapp/eal/eal_pci.c b/lib/librte_eal/linuxapp/eal/eal_pci.c
> index c0ca5a5..d847102 100644
> --- a/lib/librte_eal/linuxapp/eal/eal_pci.c
> +++ b/lib/librte_eal/linuxapp/eal/eal_pci.c
> @@ -229,20 +229,6 @@ error:
> return -1;
> }
>
> -/* Compare two PCI device addresses. */
> -static int
> -pci_addr_comparison(struct rte_pci_addr *addr, struct rte_pci_addr *addr2)
> -{
> - uint64_t dev_addr = (addr->domain << 24) + (addr->bus << 16) + (addr->devid << 8) + addr->function;
> - uint64_t dev_addr2 = (addr2->domain << 24) + (addr2->bus << 16) + (addr2->devid << 8) + addr2->function;
> -
> - if (dev_addr > dev_addr2)
> - return 1;
> - else
> - return 0;
> -}
> -
> -
> /* Scan one pci sysfs entry, and fill the devices list from it. */
> static int
> pci_scan_one(const char *dirname, uint16_t domain, uint8_t bus,
> @@ -353,13 +339,20 @@ pci_scan_one(const char *dirname, uint16_t domain, uint8_t bus,
> }
> else {
> struct rte_pci_device *dev2 = NULL;
> + int ret;
>
> TAILQ_FOREACH(dev2, &pci_device_list, next) {
> - if (pci_addr_comparison(&dev->addr, &dev2->addr))
> + ret = eal_compare_pci_addr(&dev->addr, &dev2->addr);
> + if (ret > 0)
> continue;
> - else {
> + else if (ret < 0) {
> TAILQ_INSERT_BEFORE(dev2, dev, next);
> return 0;
> + } else { /* already registered */
> + /* update pt_driver */
> + dev2->pt_driver = dev->pt_driver;
> + free(dev);
> + return 0;
> }
> }
> TAILQ_INSERT_TAIL(&pci_device_list, dev, next);
> diff --git a/lib/librte_eal/linuxapp/eal/eal_pci_uio.c b/lib/librte_eal/linuxapp/eal/eal_pci_uio.c
> index e53f06b..1da3507 100644
> --- a/lib/librte_eal/linuxapp/eal/eal_pci_uio.c
> +++ b/lib/librte_eal/linuxapp/eal/eal_pci_uio.c
> @@ -123,7 +123,7 @@ pci_uio_map_secondary(struct rte_pci_device *dev)
> TAILQ_FOREACH(uio_res, pci_res_list, next) {
>
> /* skip this element if it doesn't match our PCI address */
> - if (memcmp(&uio_res->pci_addr, &dev->addr, sizeof(dev->addr)))
> + if (eal_compare_pci_addr(&uio_res->pci_addr, &dev->addr))
> continue;
>
> for (i = 0; i != uio_res->nb_maps; i++) {
Acked-by: Michael Qiu <michael.qiu@intel.com>
> -----Original Message----- > From: Tetsuya Mukawa [mailto:mukawa@igel.co.jp] > Sent: Monday, February 9, 2015 8:31 AM > To: dev@dpdk.org > Cc: Iremonger, Bernard; Qiu, Michael; Tetsuya Mukawa > Subject: [PATCH v7 08/14] eal/linux/pci: Add functions for unmapping igb_uio resources > > The patch adds functions for unmapping igb_uio resources. The patch is only for Linux and igb_uio > environment. VFIO and BSD are not supported. > > v5: > - Fix pci_unmap_device() to check pt_driver. > v4: > - Add parameter checking. > - Add header file to determine if hotplug can be enabled. > > Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp> > --- > lib/librte_eal/common/Makefile | 1 + > lib/librte_eal/common/include/rte_dev_hotplug.h | 44 +++++++++++++++++ > lib/librte_eal/linuxapp/eal/eal_pci.c | 44 +++++++++++++++++ > lib/librte_eal/linuxapp/eal/eal_pci_init.h | 8 +++ > lib/librte_eal/linuxapp/eal/eal_pci_uio.c | 65 +++++++++++++++++++++++++ > 5 files changed, 162 insertions(+) > create mode 100644 lib/librte_eal/common/include/rte_dev_hotplug.h > > diff --git a/lib/librte_eal/common/Makefile b/lib/librte_eal/common/Makefile index 52c1a5f..db7cc93 > 100644 > --- a/lib/librte_eal/common/Makefile > +++ b/lib/librte_eal/common/Makefile > @@ -41,6 +41,7 @@ INC += rte_eal_memconfig.h rte_malloc_heap.h INC += rte_hexdump.h > rte_devargs.h rte_dev.h INC += rte_common_vect.h INC += rte_pci_dev_feature_defs.h > rte_pci_dev_features.h > +INC += rte_dev_hotplug.h > > ifeq ($(CONFIG_RTE_INSECURE_FUNCTION_WARNING),y) > INC += rte_warnings.h > diff --git a/lib/librte_eal/common/include/rte_dev_hotplug.h > b/lib/librte_eal/common/include/rte_dev_hotplug.h > new file mode 100644 > index 0000000..b333e0f > --- /dev/null > +++ b/lib/librte_eal/common/include/rte_dev_hotplug.h > @@ -0,0 +1,44 @@ > +/*- > + * BSD LICENSE > + * > + * Copyright(c) 2015 IGEL Co.,LTd. > + * All rights reserved. > + * > + * Redistribution and use in source and binary forms, with or without > + * modification, are permitted provided that the following conditions > + * are met: > + * > + * * Redistributions of source code must retain the above copyright > + * notice, this list of conditions and the following disclaimer. > + * * Redistributions in binary form must reproduce the above copyright > + * notice, this list of conditions and the following disclaimer in > + * the documentation and/or other materials provided with the > + * distribution. > + * * Neither the name of IGEL Co.,Ltd. nor the names of its > + * contributors may be used to endorse or promote products derived > + * from this software without specific prior written permission. > + * > + * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS > + * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT > + * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR > + * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT > + * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, > + * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT > + * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, > + * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY > + * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT > + * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE > + * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. > + */ > + > +#ifndef _RTE_DEV_HOTPLUG_H_ > +#define _RTE_DEV_HOTPLUG_H_ > + > +/* > + * determine if hotplug can be enabled on the system */ #if > +defined(RTE_LIBRTE_EAL_HOTPLUG) && defined(RTE_LIBRTE_EAL_LINUXAPP) > +#define ENABLE_HOTPLUG #endif /* RTE_LIBRTE_EAL_HOTPLUG & > +RTE_LIBRTE_EAL_LINUXAPP */ > + > +#endif /* _RTE_DEV_HOTPLUG_H_ */ > diff --git a/lib/librte_eal/linuxapp/eal/eal_pci.c b/lib/librte_eal/linuxapp/eal/eal_pci.c > index d847102..c3b7917 100644 > --- a/lib/librte_eal/linuxapp/eal/eal_pci.c > +++ b/lib/librte_eal/linuxapp/eal/eal_pci.c > @@ -166,6 +166,25 @@ pci_map_resource(void *requested_addr, int fd, off_t offset, size_t size) > return mapaddr; > } > > +#ifdef ENABLE_HOTPLUG > +/* unmap a particular resource */ > +void > +pci_unmap_resource(void *requested_addr, size_t size) { > + if (requested_addr == NULL) > + return; > + > + /* Unmap the PCI memory resource of device */ > + if (munmap(requested_addr, size)) { > + RTE_LOG(ERR, EAL, "%s(): cannot munmap(%p, 0x%lx): %s\n", > + __func__, requested_addr, (unsigned long)size, > + strerror(errno)); > + } else > + RTE_LOG(DEBUG, EAL, " PCI memory mapped at %p\n", Hi Tetsuya, " PCI memory mapped" should be "PCI memory unmapped" Regards, Bernard > + requested_addr); > +} > +#endif /* ENABLE_HOTPLUG */ > + > /* parse the "resource" sysfs file */ > #define IORESOURCE_MEM 0x00000200 > > @@ -567,6 +586,31 @@ pci_map_device(struct rte_pci_device *dev) > return ret; > } > > +#ifdef ENABLE_HOTPLUG > +static void > +pci_unmap_device(struct rte_pci_device *dev) { > + if (dev == NULL) > + return; > + > + /* try unmapping the NIC resources using VFIO if it exists */ > + switch (dev->pt_driver) { > + case RTE_PT_VFIO: > + RTE_LOG(ERR, EAL, "Hotplug doesn't support vfio yet\n"); > + break; > + case RTE_PT_IGB_UIO: > + case RTE_PT_UIO_GENERIC: > + /* unmap resources for devices that use uio */ > + pci_uio_unmap_resource(dev); > + break; > + default: > + RTE_LOG(DEBUG, EAL, " Not managed by known pt driver," > + " skipped\n"); > + break; > + } > +} > +#endif /* ENABLE_HOTPLUG */ > + > /* > * If vendor/device ID match, call the devinit() function of the > * driver. > diff --git a/lib/librte_eal/linuxapp/eal/eal_pci_init.h b/lib/librte_eal/linuxapp/eal/eal_pci_init.h > index 1070eb8..5152a0b 100644 > --- a/lib/librte_eal/linuxapp/eal/eal_pci_init.h > +++ b/lib/librte_eal/linuxapp/eal/eal_pci_init.h > @@ -34,6 +34,7 @@ > #ifndef EAL_PCI_INIT_H_ > #define EAL_PCI_INIT_H_ > > +#include <rte_dev_hotplug.h> > #include "eal_vfio.h" > > struct pci_map { > @@ -71,6 +72,13 @@ void *pci_map_resource(void *requested_addr, int fd, off_t offset, > /* map IGB_UIO resource prototype */ > int pci_uio_map_resource(struct rte_pci_device *dev); > > +#ifdef ENABLE_HOTPLUG > +void pci_unmap_resource(void *requested_addr, size_t size); > + > +/* unmap IGB_UIO resource prototype */ > +void pci_uio_unmap_resource(struct rte_pci_device *dev); #endif /* > +ENABLE_HOTPLUG */ > + > #ifdef VFIO_PRESENT > > #define VFIO_MAX_GROUPS 64 > diff --git a/lib/librte_eal/linuxapp/eal/eal_pci_uio.c b/lib/librte_eal/linuxapp/eal/eal_pci_uio.c > index 1da3507..81830d1 100644 > --- a/lib/librte_eal/linuxapp/eal/eal_pci_uio.c > +++ b/lib/librte_eal/linuxapp/eal/eal_pci_uio.c > @@ -404,6 +404,71 @@ pci_uio_map_resource(struct rte_pci_device *dev) > return 0; > } > > +#ifdef ENABLE_HOTPLUG > +static void > +pci_uio_unmap(struct mapped_pci_resource *uio_res) { > + int i; > + > + if (uio_res == NULL) > + return; > + > + for (i = 0; i != uio_res->nb_maps; i++) > + pci_unmap_resource(uio_res->maps[i].addr, > + (size_t)uio_res->maps[i].size); > +} > + > +static struct mapped_pci_resource * > +pci_uio_find_resource(struct rte_pci_device *dev) { > + struct mapped_pci_resource *uio_res; > + > + if (dev == NULL) > + return NULL; > + > + TAILQ_FOREACH(uio_res, pci_res_list, next) { > + > + /* skip this element if it doesn't match our PCI address */ > + if (!eal_compare_pci_addr(&uio_res->pci_addr, &dev->addr)) > + return uio_res; > + } > + return NULL; > +} > + > +/* unmap the PCI resource of a PCI device in virtual memory */ void > +pci_uio_unmap_resource(struct rte_pci_device *dev) { > + struct mapped_pci_resource *uio_res; > + > + if (dev == NULL) > + return; > + > + /* find an entry for the device */ > + uio_res = pci_uio_find_resource(dev); > + if (uio_res == NULL) > + return; > + > + /* secondary processes - just free maps */ > + if (rte_eal_process_type() != RTE_PROC_PRIMARY) > + return pci_uio_unmap(uio_res); > + > + TAILQ_REMOVE(pci_res_list, uio_res, next); > + > + /* unmap all resources */ > + pci_uio_unmap(uio_res); > + > + /* free uio resource */ > + rte_free(uio_res); > + > + /* close fd if in primary process */ > + close(dev->intr_handle.fd); > + > + dev->intr_handle.fd = -1; > + dev->intr_handle.type = RTE_INTR_HANDLE_UNKNOWN; } #endif /* > +ENABLE_HOTPLUG */ > + > /* > * parse a sysfs file containing one integer value > * different to the eal version, as it needs to work with 64-bit values > -- > 1.9.1
> -----Original Message----- > From: Tetsuya Mukawa [mailto:mukawa@igel.co.jp] > Sent: Monday, February 9, 2015 8:31 AM > To: dev@dpdk.org > Cc: Iremonger, Bernard; Qiu, Michael; Tetsuya Mukawa > Subject: [PATCH v7 11/14] ethdev: Add one dev_type parameter to rte_eth_dev_allocate > > This new parameter is needed to keep device type like physical or virtual. > Port detaching processes are different between physical and virtual. > This parameter lets detaching function know a device type of the port. > > v4: > - Fix comments of rte_eth_dev_type. > > Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp> > --- > app/test/virtual_pmd.c | 2 +- > lib/librte_ether/rte_ethdev.c | 14 ++++++++++++-- > lib/librte_ether/rte_ethdev.h | 25 ++++++++++++++++++++++++- > lib/librte_pmd_af_packet/rte_eth_af_packet.c | 2 +- > lib/librte_pmd_bond/rte_eth_bond_api.c | 2 +- > lib/librte_pmd_pcap/rte_eth_pcap.c | 2 +- > lib/librte_pmd_ring/rte_eth_ring.c | 2 +- > lib/librte_pmd_xenvirt/rte_eth_xenvirt.c | 2 +- > 8 files changed, 42 insertions(+), 9 deletions(-) > > diff --git a/app/test/virtual_pmd.c b/app/test/virtual_pmd.c index 9fac95d..8d3a5ff 100644 > --- a/app/test/virtual_pmd.c > +++ b/app/test/virtual_pmd.c > @@ -556,7 +556,7 @@ virtual_ethdev_create(const char *name, struct ether_addr *mac_addr, > goto err; > > /* reserve an ethdev entry */ > - eth_dev = rte_eth_dev_allocate(name); > + eth_dev = rte_eth_dev_allocate(name, RTE_ETH_DEV_PHYSICAL); > if (eth_dev == NULL) > goto err; > > diff --git a/lib/librte_ether/rte_ethdev.c b/lib/librte_ether/rte_ethdev.c index 14a040a..704185d > 100644 > --- a/lib/librte_ether/rte_ethdev.c > +++ b/lib/librte_ether/rte_ethdev.c > @@ -232,7 +232,7 @@ rte_eth_dev_allocate_new_port(void) > } > > struct rte_eth_dev * > -rte_eth_dev_allocate(const char *name) > +rte_eth_dev_allocate(const char *name, enum rte_eth_dev_type type) > { > uint8_t port_id; > struct rte_eth_dev *eth_dev; > @@ -256,6 +256,7 @@ rte_eth_dev_allocate(const char *name) > snprintf(eth_dev->data->name, sizeof(eth_dev->data->name), "%s", name); > eth_dev->data->port_id = port_id; > eth_dev->attached = DEV_CONNECTED; > + eth_dev->dev_type = type; > nb_ports++; > return eth_dev; > } > @@ -267,6 +268,7 @@ rte_eth_dev_free(struct rte_eth_dev *eth_dev) > return -EINVAL; > > eth_dev->attached = 0; > + eth_dev->dev_type = RTE_ETH_DEV_UNKNOWN; > nb_ports--; > return 0; > } > @@ -287,7 +289,7 @@ rte_eth_dev_init(struct rte_pci_driver *pci_drv, > snprintf(ethdev_name, RTE_ETH_NAME_MAX_LEN, "%d:%d.%d", > pci_dev->addr.bus, pci_dev->addr.devid, pci_dev->addr.function); > > - eth_dev = rte_eth_dev_allocate(ethdev_name); > + eth_dev = rte_eth_dev_allocate(ethdev_name, RTE_ETH_DEV_PHYSICAL); > if (eth_dev == NULL) > return -ENOMEM; > > @@ -426,6 +428,14 @@ rte_eth_dev_count(void) > return (nb_ports); > } > > +enum rte_eth_dev_type > +rte_eth_dev_get_device_type(uint8_t port_id) { > + if (rte_eth_dev_validate_port(port_id, NONE_TRACE) == DEV_INVALID) Hi Tetsuya, NO_TRACE would be better than NON_TRACE. > + return -1; > + return rte_eth_devices[port_id].dev_type; > +} > + > void > rte_eth_dev_save(struct rte_eth_dev *devs) { diff --git a/lib/librte_ether/rte_ethdev.h > b/lib/librte_ether/rte_ethdev.h index 9919968..00a6218 100644 > --- a/lib/librte_ether/rte_ethdev.h > +++ b/lib/librte_ether/rte_ethdev.h > @@ -1522,6 +1522,17 @@ struct eth_dev_ops { }; > > /** > + * The eth device type > + */ > +enum rte_eth_dev_type { > + RTE_ETH_DEV_UNKNOWN, /**< unknown device type */ > + RTE_ETH_DEV_PHYSICAL, > + /**< Physical function and Virtual function devices of NIC */ > + RTE_ETH_DEV_VIRTUAL, /**< non hardware device */ > + RTE_ETH_DEV_MAX /**< max value of this enum */ > +}; > + > +/** > * @internal > * The generic data structure associated with each ethernet device. > * > @@ -1540,6 +1551,7 @@ struct rte_eth_dev { > struct rte_pci_device *pci_dev; /**< PCI info. supplied by probing */ > struct rte_eth_dev_cb_list callbacks; /**< User application callbacks */ > uint8_t attached; /**< Flag indicating the port is attached */ > + enum rte_eth_dev_type dev_type; /**< Flag indicating the device type > +*/ > }; > > struct rte_eth_dev_sriov { > @@ -1617,6 +1629,15 @@ extern uint8_t rte_eth_dev_count(void); > > /** > * Function for internal use by port hotplug functions. > + * Get the device type to know whether the device is physical or virtual. > + * @param port_id The pointer to the port id > + * @return > + * - Device type. > + */ > +extern enum rte_eth_dev_type rte_eth_dev_get_device_type(uint8_t > +port_id); > + > +/** > + * Function for internal use by port hotplug functions. > * Copies current ethdev structures to the specified pointer. > * > * @param devs The pointer to the ethdev structures > @@ -1702,10 +1723,12 @@ extern struct rte_eth_dev *rte_eth_dev_allocated(const char *name); > * to that slot for the driver to use. > * > * @param name Unique identifier name for each Ethernet device > + * @param type Device type of this Ethernet device > * @return > * - Slot in the rte_dev_devices array for a new device; > */ > -struct rte_eth_dev *rte_eth_dev_allocate(const char *name); > +struct rte_eth_dev *rte_eth_dev_allocate(const char *name, > + enum rte_eth_dev_type type); > > /** > * Function for internal use by dummy drivers primarily, e.g. ring-based diff --git > a/lib/librte_pmd_af_packet/rte_eth_af_packet.c b/lib/librte_pmd_af_packet/rte_eth_af_packet.c > index 1ffe1cd..80e9bdf 100644 > --- a/lib/librte_pmd_af_packet/rte_eth_af_packet.c > +++ b/lib/librte_pmd_af_packet/rte_eth_af_packet.c > @@ -649,7 +649,7 @@ rte_pmd_init_internals(const char *name, > } > > /* reserve an ethdev entry */ > - *eth_dev = rte_eth_dev_allocate(name); > + *eth_dev = rte_eth_dev_allocate(name, RTE_ETH_DEV_VIRTUAL); > if (*eth_dev == NULL) > goto error; > > diff --git a/lib/librte_pmd_bond/rte_eth_bond_api.c b/lib/librte_pmd_bond/rte_eth_bond_api.c > index 4ab3267..7a6a5f7 100644 > --- a/lib/librte_pmd_bond/rte_eth_bond_api.c > +++ b/lib/librte_pmd_bond/rte_eth_bond_api.c > @@ -235,7 +235,7 @@ rte_eth_bond_create(const char *name, uint8_t mode, uint8_t socket_id) > } > > /* reserve an ethdev entry */ > - eth_dev = rte_eth_dev_allocate(name); > + eth_dev = rte_eth_dev_allocate(name, RTE_ETH_DEV_VIRTUAL); > if (eth_dev == NULL) { > RTE_BOND_LOG(ERR, "Unable to allocate rte_eth_dev"); > goto err; > diff --git a/lib/librte_pmd_pcap/rte_eth_pcap.c b/lib/librte_pmd_pcap/rte_eth_pcap.c > index d299288..af7fae8 100644 > --- a/lib/librte_pmd_pcap/rte_eth_pcap.c > +++ b/lib/librte_pmd_pcap/rte_eth_pcap.c > @@ -709,7 +709,7 @@ rte_pmd_init_internals(const char *name, const unsigned nb_rx_queues, > goto error; > > /* reserve an ethdev entry */ > - *eth_dev = rte_eth_dev_allocate(name); > + *eth_dev = rte_eth_dev_allocate(name, RTE_ETH_DEV_VIRTUAL); > if (*eth_dev == NULL) > goto error; > > diff --git a/lib/librte_pmd_ring/rte_eth_ring.c b/lib/librte_pmd_ring/rte_eth_ring.c > index d5b1686..c837fcd 100644 > --- a/lib/librte_pmd_ring/rte_eth_ring.c > +++ b/lib/librte_pmd_ring/rte_eth_ring.c > @@ -255,7 +255,7 @@ rte_eth_from_rings(const char *name, struct rte_ring *const rx_queues[], > goto error; > > /* reserve an ethdev entry */ > - eth_dev = rte_eth_dev_allocate(name); > + eth_dev = rte_eth_dev_allocate(name, RTE_ETH_DEV_VIRTUAL); > if (eth_dev == NULL) > goto error; > > diff --git a/lib/librte_pmd_xenvirt/rte_eth_xenvirt.c b/lib/librte_pmd_xenvirt/rte_eth_xenvirt.c > index 04e30c9..bc403d6 100644 > --- a/lib/librte_pmd_xenvirt/rte_eth_xenvirt.c > +++ b/lib/librte_pmd_xenvirt/rte_eth_xenvirt.c > @@ -648,7 +648,7 @@ eth_dev_xenvirt_create(const char *name, const char *params, > goto err; > > /* reserve an ethdev entry */ > - eth_dev = rte_eth_dev_allocate(name); > + eth_dev = rte_eth_dev_allocate(name, RTE_ETH_DEV_VIRTUAL); > if (eth_dev == NULL) > goto err; > > -- > 1.9.1 Regards, Bernard.
> -----Original Message----- > From: Tetsuya Mukawa [mailto:mukawa@igel.co.jp] > Sent: Monday, February 9, 2015 8:30 AM > To: dev@dpdk.org > Cc: Iremonger, Bernard; Qiu, Michael; Tetsuya Mukawa > Subject: [PATCH v7 03/14] eal/pci,ethdev: Remove assumption that port will not be detached > > To remove assumption, do like followings. > > This patch adds "RTE_PCI_DRV_DETACHABLE" to drv_flags of rte_pci_driver structure. The flags > indicate the driver can detach devices at runtime. > Also, remove assumption that port will not be detached. > > To remove the assumption. > - Add 'attached' member to rte_eth_dev structure. > This member is used for indicating the port is attached, or not. > - Add rte_eth_dev_allocate_new_port(). > This function is used for allocating new port. > > v5: > - Change parameters of rte_eth_dev_validate_port() to cleanup code. > v4: > - Use braces with 'for' loop. > - Fix indent of 'if' statement. > > Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp> > --- > lib/librte_eal/common/include/rte_pci.h | 2 + > lib/librte_ether/rte_ethdev.c | 454 +++++++++++++------------------- > lib/librte_ether/rte_ethdev.h | 5 + > 3 files changed, 186 insertions(+), 275 deletions(-) > > diff --git a/lib/librte_eal/common/include/rte_pci.h b/lib/librte_eal/common/include/rte_pci.h > index 7b48b55..7f2d699 100644 > --- a/lib/librte_eal/common/include/rte_pci.h > +++ b/lib/librte_eal/common/include/rte_pci.h > @@ -207,6 +207,8 @@ struct rte_pci_driver { #define RTE_PCI_DRV_FORCE_UNBIND 0x0004 > /** Device driver supports link state interrupt */ > #define RTE_PCI_DRV_INTR_LSC 0x0008 > +/** Device driver supports detaching capability */ > +#define RTE_PCI_DRV_DETACHABLE 0x0010 > > /**< Internal use only - Macro used by pci addr parsing functions **/ > #define GET_PCIADDR_FIELD(in, fd, lim, dlm) \ > diff --git a/lib/librte_ether/rte_ethdev.c b/lib/librte_ether/rte_ethdev.c index ea3a1fb..d70854f > 100644 > --- a/lib/librte_ether/rte_ethdev.c > +++ b/lib/librte_ether/rte_ethdev.c > @@ -175,6 +175,16 @@ enum { > STAT_QMAP_RX > }; > > +enum { > + DEV_INVALID = 0, > + DEV_VALID, > +}; > + > +enum { > + DEV_DISCONNECTED = 0, > + DEV_CONNECTED > +}; > + > static inline void > rte_eth_dev_data_alloc(void) > { > @@ -201,19 +211,34 @@ rte_eth_dev_allocated(const char *name) { > unsigned i; > > - for (i = 0; i < nb_ports; i++) { > - if (strcmp(rte_eth_devices[i].data->name, name) == 0) > + for (i = 0; i < RTE_MAX_ETHPORTS; i++) { > + if ((rte_eth_devices[i].attached == DEV_CONNECTED) && > + strcmp(rte_eth_devices[i].data->name, name) == 0) > return &rte_eth_devices[i]; > } > return NULL; > } > > +static uint8_t > +rte_eth_dev_allocate_new_port(void) > +{ > + unsigned i; > + > + for (i = 0; i < RTE_MAX_ETHPORTS; i++) { > + if (rte_eth_devices[i].attached == DEV_DISCONNECTED) > + return i; > + } > + return RTE_MAX_ETHPORTS; > +} > + > struct rte_eth_dev * > rte_eth_dev_allocate(const char *name) > { > + uint8_t port_id; > struct rte_eth_dev *eth_dev; > > - if (nb_ports == RTE_MAX_ETHPORTS) { > + port_id = rte_eth_dev_allocate_new_port(); > + if (port_id == RTE_MAX_ETHPORTS) { > PMD_DEBUG_TRACE("Reached maximum number of Ethernet ports\n"); > return NULL; > } > @@ -226,10 +251,12 @@ rte_eth_dev_allocate(const char *name) > return NULL; > } > > - eth_dev = &rte_eth_devices[nb_ports]; > - eth_dev->data = &rte_eth_dev_data[nb_ports]; > + eth_dev = &rte_eth_devices[port_id]; > + eth_dev->data = &rte_eth_dev_data[port_id]; > snprintf(eth_dev->data->name, sizeof(eth_dev->data->name), "%s", name); > - eth_dev->data->port_id = nb_ports++; > + eth_dev->data->port_id = port_id; > + eth_dev->attached = DEV_CONNECTED; > + nb_ports++; > return eth_dev; > } > > @@ -283,6 +310,7 @@ rte_eth_dev_init(struct rte_pci_driver *pci_drv, > (unsigned) pci_dev->id.device_id); > if (rte_eal_process_type() == RTE_PROC_PRIMARY) > rte_free(eth_dev->data->dev_private); > + eth_dev->attached = DEV_DISCONNECTED; > nb_ports--; > return diag; > } > @@ -308,10 +336,28 @@ rte_eth_driver_register(struct eth_driver *eth_drv) > rte_eal_pci_register(ð_drv->pci_drv); > } > > +enum { > + NONE_TRACE = 0, Hi Tetsuya, NO_TRACE would be clearer that NONE_TRACE Regards, Bernard. > + TRACE > +}; > + > +static int > +rte_eth_dev_validate_port(uint8_t port_id, int trace) { > + if (port_id >= RTE_MAX_ETHPORTS || > + rte_eth_devices[port_id].attached != DEV_CONNECTED) { > + if (trace) { > + PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); > + } > + return DEV_INVALID; > + } else > + return DEV_VALID; > +} > + > int > rte_eth_dev_socket_id(uint8_t port_id) > { > - if (port_id >= nb_ports) > + if (rte_eth_dev_validate_port(port_id, NONE_TRACE) == DEV_INVALID) > return -1; > return rte_eth_devices[port_id].pci_dev->numa_node; > } > @@ -369,10 +415,8 @@ rte_eth_dev_rx_queue_start(uint8_t port_id, uint16_t rx_queue_id) > * in a multi-process setup*/ > PROC_PRIMARY_OR_ERR_RET(-E_RTE_SECONDARY); > > - if (port_id >= nb_ports) { > - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); > + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) > return -EINVAL; > - } > > dev = &rte_eth_devices[port_id]; > if (rx_queue_id >= dev->data->nb_rx_queues) { @@ -395,10 +439,8 @@ > rte_eth_dev_rx_queue_stop(uint8_t port_id, uint16_t rx_queue_id) > * in a multi-process setup*/ > PROC_PRIMARY_OR_ERR_RET(-E_RTE_SECONDARY); > > - if (port_id >= nb_ports) { > - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); > + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) > return -EINVAL; > - } > > dev = &rte_eth_devices[port_id]; > if (rx_queue_id >= dev->data->nb_rx_queues) { @@ -421,10 +463,8 @@ > rte_eth_dev_tx_queue_start(uint8_t port_id, uint16_t tx_queue_id) > * in a multi-process setup*/ > PROC_PRIMARY_OR_ERR_RET(-E_RTE_SECONDARY); > > - if (port_id >= nb_ports) { > - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); > + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) > return -EINVAL; > - } > > dev = &rte_eth_devices[port_id]; > if (tx_queue_id >= dev->data->nb_tx_queues) { @@ -447,10 +487,8 @@ > rte_eth_dev_tx_queue_stop(uint8_t port_id, uint16_t tx_queue_id) > * in a multi-process setup*/ > PROC_PRIMARY_OR_ERR_RET(-E_RTE_SECONDARY); > > - if (port_id >= nb_ports) { > - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); > + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) > return -EINVAL; > - } > > dev = &rte_eth_devices[port_id]; > if (tx_queue_id >= dev->data->nb_tx_queues) { @@ -703,10 +741,9 @@ > rte_eth_dev_configure(uint8_t port_id, uint16_t nb_rx_q, uint16_t nb_tx_q, > * in a multi-process setup*/ > PROC_PRIMARY_OR_ERR_RET(-E_RTE_SECONDARY); > > - if (port_id >= nb_ports || port_id >= RTE_MAX_ETHPORTS) { > - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); > + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) > return (-EINVAL); > - } > + > dev = &rte_eth_devices[port_id]; > > FUNC_PTR_OR_ERR_RET(*dev->dev_ops->dev_infos_get, -ENOTSUP); @@ -888,10 +925,9 > @@ rte_eth_dev_start(uint8_t port_id) > * in a multi-process setup*/ > PROC_PRIMARY_OR_ERR_RET(-E_RTE_SECONDARY); > > - if (port_id >= nb_ports) { > - PMD_DEBUG_TRACE("Invalid port_id=%" PRIu8 "\n", port_id); > + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) > return (-EINVAL); > - } > + > dev = &rte_eth_devices[port_id]; > > FUNC_PTR_OR_ERR_RET(*dev->dev_ops->dev_start, -ENOTSUP); @@ -923,10 +959,9 @@ > rte_eth_dev_stop(uint8_t port_id) > * in a multi-process setup*/ > PROC_PRIMARY_OR_RET(); > > - if (port_id >= nb_ports) { > - PMD_DEBUG_TRACE("Invalid port_id=%" PRIu8 "\n", port_id); > + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) > return; > - } > + > dev = &rte_eth_devices[port_id]; > > FUNC_PTR_OR_RET(*dev->dev_ops->dev_stop); > @@ -951,10 +986,9 @@ rte_eth_dev_set_link_up(uint8_t port_id) > * in a multi-process setup*/ > PROC_PRIMARY_OR_ERR_RET(-E_RTE_SECONDARY); > > - if (port_id >= nb_ports) { > - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); > + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) > return -EINVAL; > - } > + > dev = &rte_eth_devices[port_id]; > > FUNC_PTR_OR_ERR_RET(*dev->dev_ops->dev_set_link_up, -ENOTSUP); @@ -970,10 > +1004,9 @@ rte_eth_dev_set_link_down(uint8_t port_id) > * in a multi-process setup*/ > PROC_PRIMARY_OR_ERR_RET(-E_RTE_SECONDARY); > > - if (port_id >= nb_ports) { > - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); > + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) > return -EINVAL; > - } > + > dev = &rte_eth_devices[port_id]; > > FUNC_PTR_OR_ERR_RET(*dev->dev_ops->dev_set_link_down, -ENOTSUP); @@ -989,10 > +1022,8 @@ rte_eth_dev_close(uint8_t port_id) > * in a multi-process setup*/ > PROC_PRIMARY_OR_RET(); > > - if (port_id >= nb_ports) { > - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); > + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) > return; > - } > > dev = &rte_eth_devices[port_id]; > > @@ -1017,10 +1048,9 @@ rte_eth_rx_queue_setup(uint8_t port_id, uint16_t rx_queue_id, > * in a multi-process setup*/ > PROC_PRIMARY_OR_ERR_RET(-E_RTE_SECONDARY); > > - if (port_id >= nb_ports) { > - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); > + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) > return (-EINVAL); > - } > + > dev = &rte_eth_devices[port_id]; > if (rx_queue_id >= dev->data->nb_rx_queues) { > PMD_DEBUG_TRACE("Invalid RX queue_id=%d\n", rx_queue_id); @@ -1090,10 > +1120,9 @@ rte_eth_tx_queue_setup(uint8_t port_id, uint16_t tx_queue_id, > * in a multi-process setup*/ > PROC_PRIMARY_OR_ERR_RET(-E_RTE_SECONDARY); > > - if (port_id >= RTE_MAX_ETHPORTS || port_id >= nb_ports) { > - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); > + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) > return (-EINVAL); > - } > + > dev = &rte_eth_devices[port_id]; > if (tx_queue_id >= dev->data->nb_tx_queues) { > PMD_DEBUG_TRACE("Invalid TX queue_id=%d\n", tx_queue_id); @@ -1123,10 > +1152,9 @@ rte_eth_promiscuous_enable(uint8_t port_id) { > struct rte_eth_dev *dev; > > - if (port_id >= nb_ports) { > - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); > + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) > return; > - } > + > dev = &rte_eth_devices[port_id]; > > FUNC_PTR_OR_RET(*dev->dev_ops->promiscuous_enable); > @@ -1139,10 +1167,9 @@ rte_eth_promiscuous_disable(uint8_t port_id) { > struct rte_eth_dev *dev; > > - if (port_id >= nb_ports) { > - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); > + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) > return; > - } > + > dev = &rte_eth_devices[port_id]; > > FUNC_PTR_OR_RET(*dev->dev_ops->promiscuous_disable); > @@ -1155,10 +1182,8 @@ rte_eth_promiscuous_get(uint8_t port_id) { > struct rte_eth_dev *dev; > > - if (port_id >= nb_ports) { > - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); > + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) > return -1; > - } > > dev = &rte_eth_devices[port_id]; > return dev->data->promiscuous; > @@ -1169,10 +1194,9 @@ rte_eth_allmulticast_enable(uint8_t port_id) { > struct rte_eth_dev *dev; > > - if (port_id >= nb_ports) { > - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); > + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) > return; > - } > + > dev = &rte_eth_devices[port_id]; > > FUNC_PTR_OR_RET(*dev->dev_ops->allmulticast_enable); > @@ -1185,10 +1209,9 @@ rte_eth_allmulticast_disable(uint8_t port_id) { > struct rte_eth_dev *dev; > > - if (port_id >= nb_ports) { > - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); > + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) > return; > - } > + > dev = &rte_eth_devices[port_id]; > > FUNC_PTR_OR_RET(*dev->dev_ops->allmulticast_disable); > @@ -1201,10 +1224,8 @@ rte_eth_allmulticast_get(uint8_t port_id) { > struct rte_eth_dev *dev; > > - if (port_id >= nb_ports) { > - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); > + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) > return -1; > - } > > dev = &rte_eth_devices[port_id]; > return dev->data->all_multicast; > @@ -1229,10 +1250,9 @@ rte_eth_link_get(uint8_t port_id, struct rte_eth_link *eth_link) { > struct rte_eth_dev *dev; > > - if (port_id >= nb_ports) { > - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); > + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) > return; > - } > + > dev = &rte_eth_devices[port_id]; > > if (dev->data->dev_conf.intr_conf.lsc != 0) @@ -1249,10 +1269,9 @@ > rte_eth_link_get_nowait(uint8_t port_id, struct rte_eth_link *eth_link) { > struct rte_eth_dev *dev; > > - if (port_id >= nb_ports) { > - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); > + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) > return; > - } > + > dev = &rte_eth_devices[port_id]; > > if (dev->data->dev_conf.intr_conf.lsc != 0) @@ -1269,10 +1288,9 @@ > rte_eth_stats_get(uint8_t port_id, struct rte_eth_stats *stats) { > struct rte_eth_dev *dev; > > - if (port_id >= nb_ports) { > - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); > + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) > return; > - } > + > dev = &rte_eth_devices[port_id]; > memset(stats, 0, sizeof(*stats)); > > @@ -1286,10 +1304,9 @@ rte_eth_stats_reset(uint8_t port_id) { > struct rte_eth_dev *dev; > > - if (port_id >= nb_ports) { > - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); > + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) > return; > - } > + > dev = &rte_eth_devices[port_id]; > > FUNC_PTR_OR_RET(*dev->dev_ops->stats_reset); > @@ -1307,10 +1324,9 @@ rte_eth_xstats_get(uint8_t port_id, struct rte_eth_xstats *xstats, > uint64_t val; > char *stats_ptr; > > - if (port_id >= nb_ports) { > - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); > + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) > return -1; > - } > + > dev = &rte_eth_devices[port_id]; > > /* implemented by the driver */ > @@ -1376,10 +1392,9 @@ rte_eth_xstats_reset(uint8_t port_id) { > struct rte_eth_dev *dev; > > - if (port_id >= nb_ports) { > - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); > + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) > return; > - } > + > dev = &rte_eth_devices[port_id]; > > /* implemented by the driver */ > @@ -1398,10 +1413,9 @@ set_queue_stats_mapping(uint8_t port_id, uint16_t queue_id, uint8_t > stat_idx, { > struct rte_eth_dev *dev; > > - if (port_id >= nb_ports) { > - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); > + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) > return -ENODEV; > - } > + > dev = &rte_eth_devices[port_id]; > > FUNC_PTR_OR_ERR_RET(*dev->dev_ops->queue_stats_mapping_set, -ENOTSUP); @@ - > 1433,10 +1447,9 @@ rte_eth_dev_info_get(uint8_t port_id, struct rte_eth_dev_info *dev_info) { > struct rte_eth_dev *dev; > > - if (port_id >= nb_ports) { > - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); > + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) > return; > - } > + > dev = &rte_eth_devices[port_id]; > > memset(dev_info, 0, sizeof(struct rte_eth_dev_info)); @@ -1453,10 +1466,9 @@ > rte_eth_macaddr_get(uint8_t port_id, struct ether_addr *mac_addr) { > struct rte_eth_dev *dev; > > - if (port_id >= nb_ports) { > - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); > + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) > return; > - } > + > dev = &rte_eth_devices[port_id]; > ether_addr_copy(&dev->data->mac_addrs[0], mac_addr); } @@ -1467,10 +1479,8 @@ > rte_eth_dev_get_mtu(uint8_t port_id, uint16_t *mtu) { > struct rte_eth_dev *dev; > > - if (port_id >= nb_ports) { > - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); > + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) > return (-ENODEV); > - } > > dev = &rte_eth_devices[port_id]; > *mtu = dev->data->mtu; > @@ -1483,10 +1493,8 @@ rte_eth_dev_set_mtu(uint8_t port_id, uint16_t mtu) > int ret; > struct rte_eth_dev *dev; > > - if (port_id >= nb_ports) { > - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); > + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) > return (-ENODEV); > - } > > dev = &rte_eth_devices[port_id]; > FUNC_PTR_OR_ERR_RET(*dev->dev_ops->mtu_set, -ENOTSUP); @@ -1503,10 +1511,9 @@ > rte_eth_dev_vlan_filter(uint8_t port_id, uint16_t vlan_id, int on) { > struct rte_eth_dev *dev; > > - if (port_id >= nb_ports) { > - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); > + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) > return (-ENODEV); > - } > + > dev = &rte_eth_devices[port_id]; > if (! (dev->data->dev_conf.rxmode.hw_vlan_filter)) { > PMD_DEBUG_TRACE("port %d: vlan-filtering disabled\n", port_id); @@ -1528,10 > +1535,8 @@ rte_eth_dev_set_vlan_strip_on_queue(uint8_t port_id, uint16_t rx_queue_id, int o { > struct rte_eth_dev *dev; > > - if (port_id >= nb_ports) { > - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); > + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) > return (-ENODEV); > - } > > dev = &rte_eth_devices[port_id]; > if (rx_queue_id >= dev->data->nb_rx_queues) { @@ -1550,10 +1555,8 @@ > rte_eth_dev_set_vlan_ether_type(uint8_t port_id, uint16_t tpid) { > struct rte_eth_dev *dev; > > - if (port_id >= nb_ports) { > - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); > + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) > return (-ENODEV); > - } > > dev = &rte_eth_devices[port_id]; > FUNC_PTR_OR_ERR_RET(*dev->dev_ops->vlan_tpid_set, -ENOTSUP); @@ -1570,10 +1573,8 > @@ rte_eth_dev_set_vlan_offload(uint8_t port_id, int offload_mask) > int mask = 0; > int cur, org = 0; > > - if (port_id >= nb_ports) { > - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); > + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) > return (-ENODEV); > - } > > dev = &rte_eth_devices[port_id]; > > @@ -1615,10 +1616,8 @@ rte_eth_dev_get_vlan_offload(uint8_t port_id) > struct rte_eth_dev *dev; > int ret = 0; > > - if (port_id >= nb_ports) { > - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); > + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) > return (-ENODEV); > - } > > dev = &rte_eth_devices[port_id]; > > @@ -1639,10 +1638,9 @@ rte_eth_dev_set_vlan_pvid(uint8_t port_id, uint16_t pvid, int on) { > struct rte_eth_dev *dev; > > - if (port_id >= nb_ports) { > - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); > + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) > return (-ENODEV); > - } > + > dev = &rte_eth_devices[port_id]; > FUNC_PTR_OR_ERR_RET(*dev->dev_ops->vlan_pvid_set, -ENOTSUP); > (*dev->dev_ops->vlan_pvid_set)(dev, pvid, on); @@ -1657,10 +1655,8 @@ > rte_eth_dev_fdir_add_signature_filter(uint8_t port_id, { > struct rte_eth_dev *dev; > > - if (port_id >= nb_ports) { > - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); > + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) > return (-ENODEV); > - } > > dev = &rte_eth_devices[port_id]; > > @@ -1691,10 +1687,8 @@ rte_eth_dev_fdir_update_signature_filter(uint8_t port_id, { > struct rte_eth_dev *dev; > > - if (port_id >= nb_ports) { > - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); > + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) > return (-ENODEV); > - } > > dev = &rte_eth_devices[port_id]; > > @@ -1725,10 +1719,8 @@ rte_eth_dev_fdir_remove_signature_filter(uint8_t port_id, { > struct rte_eth_dev *dev; > > - if (port_id >= nb_ports) { > - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); > + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) > return (-ENODEV); > - } > > dev = &rte_eth_devices[port_id]; > > @@ -1756,10 +1748,8 @@ rte_eth_dev_fdir_get_infos(uint8_t port_id, struct rte_eth_fdir *fdir) { > struct rte_eth_dev *dev; > > - if (port_id >= nb_ports) { > - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); > + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) > return (-ENODEV); > - } > > dev = &rte_eth_devices[port_id]; > if (! (dev->data->dev_conf.fdir_conf.mode)) { @@ -1781,10 +1771,8 @@ > rte_eth_dev_fdir_add_perfect_filter(uint8_t port_id, { > struct rte_eth_dev *dev; > > - if (port_id >= nb_ports) { > - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); > + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) > return (-ENODEV); > - } > > dev = &rte_eth_devices[port_id]; > > @@ -1821,10 +1809,8 @@ rte_eth_dev_fdir_update_perfect_filter(uint8_t port_id, { > struct rte_eth_dev *dev; > > - if (port_id >= nb_ports) { > - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); > + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) > return (-ENODEV); > - } > > dev = &rte_eth_devices[port_id]; > > @@ -1859,10 +1845,8 @@ rte_eth_dev_fdir_remove_perfect_filter(uint8_t port_id, { > struct rte_eth_dev *dev; > > - if (port_id >= nb_ports) { > - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); > + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) > return (-ENODEV); > - } > > dev = &rte_eth_devices[port_id]; > > @@ -1895,10 +1879,8 @@ rte_eth_dev_fdir_set_masks(uint8_t port_id, struct rte_fdir_masks > *fdir_mask) { > struct rte_eth_dev *dev; > > - if (port_id >= nb_ports) { > - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); > + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) > return (-ENODEV); > - } > > dev = &rte_eth_devices[port_id]; > if (! (dev->data->dev_conf.fdir_conf.mode)) { @@ -1915,10 +1897,8 @@ > rte_eth_dev_flow_ctrl_get(uint8_t port_id, struct rte_eth_fc_conf *fc_conf) { > struct rte_eth_dev *dev; > > - if (port_id >= nb_ports) { > - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); > + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) > return (-ENODEV); > - } > > dev = &rte_eth_devices[port_id]; > FUNC_PTR_OR_ERR_RET(*dev->dev_ops->flow_ctrl_get, -ENOTSUP); @@ -1931,10 +1911,8 > @@ rte_eth_dev_flow_ctrl_set(uint8_t port_id, struct rte_eth_fc_conf *fc_conf) { > struct rte_eth_dev *dev; > > - if (port_id >= nb_ports) { > - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); > + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) > return (-ENODEV); > - } > > if ((fc_conf->send_xon != 0) && (fc_conf->send_xon != 1)) { > PMD_DEBUG_TRACE("Invalid send_xon, only 0/1 allowed\n"); @@ -1951,10 +1929,8 > @@ rte_eth_dev_priority_flow_ctrl_set(uint8_t port_id, struct rte_eth_pfc_conf *pfc { > struct rte_eth_dev *dev; > > - if (port_id >= nb_ports) { > - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); > + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) > return (-ENODEV); > - } > > if (pfc_conf->priority > (ETH_DCB_NUM_USER_PRIORITIES - 1)) { > PMD_DEBUG_TRACE("Invalid priority, only 0-7 allowed\n"); @@ -2030,10 +2006,8 @@ > rte_eth_dev_rss_reta_update(uint8_t port_id, > struct rte_eth_dev *dev; > int ret; > > - if (port_id >= nb_ports) { > - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); > + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) > return -ENODEV; > - } > > /* Check mask bits */ > ret = rte_eth_check_reta_mask(reta_conf, reta_size); @@ -2081,10 +2055,9 @@ > rte_eth_dev_rss_hash_update(uint8_t port_id, struct rte_eth_rss_conf *rss_conf) > struct rte_eth_dev *dev; > uint16_t rss_hash_protos; > > - if (port_id >= nb_ports) { > - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); > + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) > return (-ENODEV); > - } > + > rss_hash_protos = rss_conf->rss_hf; > if ((rss_hash_protos != 0) && > ((rss_hash_protos & ETH_RSS_PROTO_MASK) == 0)) { @@ -2103,10 +2076,9 @@ > rte_eth_dev_rss_hash_conf_get(uint8_t port_id, { > struct rte_eth_dev *dev; > > - if (port_id >= nb_ports) { > - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); > + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) > return (-ENODEV); > - } > + > dev = &rte_eth_devices[port_id]; > FUNC_PTR_OR_ERR_RET(*dev->dev_ops->rss_hash_conf_get, -ENOTSUP); > return (*dev->dev_ops->rss_hash_conf_get)(dev, rss_conf); @@ -2118,10 +2090,8 @@ > rte_eth_dev_udp_tunnel_add(uint8_t port_id, { > struct rte_eth_dev *dev; > > - if (port_id >= nb_ports) { > - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); > + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) > return -ENODEV; > - } > > if (udp_tunnel == NULL) { > PMD_DEBUG_TRACE("Invalid udp_tunnel parameter\n"); @@ -2144,10 +2114,9 @@ > rte_eth_dev_udp_tunnel_delete(uint8_t port_id, { > struct rte_eth_dev *dev; > > - if (port_id >= nb_ports) { > - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); > + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) > return -ENODEV; > - } > + > dev = &rte_eth_devices[port_id]; > > if (udp_tunnel == NULL) { > @@ -2169,10 +2138,8 @@ rte_eth_led_on(uint8_t port_id) { > struct rte_eth_dev *dev; > > - if (port_id >= nb_ports) { > - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); > + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) > return (-ENODEV); > - } > > dev = &rte_eth_devices[port_id]; > FUNC_PTR_OR_ERR_RET(*dev->dev_ops->dev_led_on, -ENOTSUP); @@ -2184,10 +2151,8 > @@ rte_eth_led_off(uint8_t port_id) { > struct rte_eth_dev *dev; > > - if (port_id >= nb_ports) { > - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); > + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) > return (-ENODEV); > - } > > dev = &rte_eth_devices[port_id]; > FUNC_PTR_OR_ERR_RET(*dev->dev_ops->dev_led_off, -ENOTSUP); @@ -2224,10 +2189,9 > @@ rte_eth_dev_mac_addr_add(uint8_t port_id, struct ether_addr *addr, > int index; > uint64_t pool_mask; > > - if (port_id >= nb_ports) { > - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); > + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) > return (-ENODEV); > - } > + > dev = &rte_eth_devices[port_id]; > FUNC_PTR_OR_ERR_RET(*dev->dev_ops->mac_addr_add, -ENOTSUP); > > @@ -2275,10 +2239,9 @@ rte_eth_dev_mac_addr_remove(uint8_t port_id, struct ether_addr *addr) > struct rte_eth_dev *dev; > int index; > > - if (port_id >= nb_ports) { > - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); > + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) > return (-ENODEV); > - } > + > dev = &rte_eth_devices[port_id]; > FUNC_PTR_OR_ERR_RET(*dev->dev_ops->mac_addr_remove, -ENOTSUP); > > @@ -2309,7 +2272,7 @@ rte_eth_dev_set_vf_rxmode(uint8_t port_id, uint16_t vf, > struct rte_eth_dev *dev; > struct rte_eth_dev_info dev_info; > > - if (port_id >= nb_ports) { > + if (rte_eth_dev_validate_port(port_id, NONE_TRACE) == DEV_INVALID) { > PMD_DEBUG_TRACE("set VF RX mode:Invalid port_id=%d\n", > port_id); > return (-ENODEV); > @@ -2364,7 +2327,7 @@ rte_eth_dev_uc_hash_table_set(uint8_t port_id, struct ether_addr *addr, > int ret; > struct rte_eth_dev *dev; > > - if (port_id >= nb_ports) { > + if (rte_eth_dev_validate_port(port_id, NONE_TRACE) == DEV_INVALID) { > PMD_DEBUG_TRACE("unicast hash setting:Invalid port_id=%d\n", > port_id); > return (-ENODEV); > @@ -2417,7 +2380,7 @@ rte_eth_dev_uc_all_hash_table_set(uint8_t port_id, uint8_t on) { > struct rte_eth_dev *dev; > > - if (port_id >= nb_ports) { > + if (rte_eth_dev_validate_port(port_id, NONE_TRACE) == DEV_INVALID) { > PMD_DEBUG_TRACE("unicast hash setting:Invalid port_id=%d\n", > port_id); > return (-ENODEV); > @@ -2436,10 +2399,8 @@ rte_eth_dev_set_vf_rx(uint8_t port_id,uint16_t vf, uint8_t on) > struct rte_eth_dev *dev; > struct rte_eth_dev_info dev_info; > > - if (port_id >= nb_ports) { > - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); > + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) > return (-ENODEV); > - } > > dev = &rte_eth_devices[port_id]; > rte_eth_dev_info_get(port_id, &dev_info); @@ -2462,7 +2423,7 @@ > rte_eth_dev_set_vf_tx(uint8_t port_id,uint16_t vf, uint8_t on) > struct rte_eth_dev *dev; > struct rte_eth_dev_info dev_info; > > - if (port_id >= nb_ports) { > + if (rte_eth_dev_validate_port(port_id, NONE_TRACE) == DEV_INVALID) { > PMD_DEBUG_TRACE("set pool tx:Invalid port_id=%d\n", port_id); > return (-ENODEV); > } > @@ -2487,7 +2448,7 @@ rte_eth_dev_set_vf_vlan_filter(uint8_t port_id, uint16_t vlan_id, { > struct rte_eth_dev *dev; > > - if (port_id >= nb_ports) { > + if (rte_eth_dev_validate_port(port_id, NONE_TRACE) == DEV_INVALID) { > PMD_DEBUG_TRACE("VF VLAN filter:invalid port id=%d\n", > port_id); > return (-ENODEV); > @@ -2518,7 +2479,7 @@ int rte_eth_set_queue_rate_limit(uint8_t port_id, uint16_t queue_idx, > struct rte_eth_dev_info dev_info; > struct rte_eth_link link; > > - if (port_id >= nb_ports) { > + if (rte_eth_dev_validate_port(port_id, NONE_TRACE) == DEV_INVALID) { > PMD_DEBUG_TRACE("set queue rate limit:invalid port id=%d\n", > port_id); > return -ENODEV; > @@ -2555,7 +2516,7 @@ int rte_eth_set_vf_rate_limit(uint8_t port_id, uint16_t vf, uint16_t tx_rate, > if (q_msk == 0) > return 0; > > - if (port_id >= nb_ports) { > + if (rte_eth_dev_validate_port(port_id, NONE_TRACE) == DEV_INVALID) { > PMD_DEBUG_TRACE("set VF rate limit:invalid port id=%d\n", > port_id); > return -ENODEV; > @@ -2589,10 +2550,8 @@ rte_eth_mirror_rule_set(uint8_t port_id, { > struct rte_eth_dev *dev = &rte_eth_devices[port_id]; > > - if (port_id >= nb_ports) { > - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); > + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) > return (-ENODEV); > - } > > if (mirror_conf->rule_type_mask == 0) { > PMD_DEBUG_TRACE("mirror rule type can not be 0.\n"); @@ -2630,10 +2589,8 @@ > rte_eth_mirror_rule_reset(uint8_t port_id, uint8_t rule_id) { > struct rte_eth_dev *dev = &rte_eth_devices[port_id]; > > - if (port_id >= nb_ports) { > - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); > + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) > return (-ENODEV); > - } > > if(rule_id >= ETH_VMDQ_NUM_MIRROR_RULE) > { > @@ -2655,10 +2612,9 @@ rte_eth_rx_burst(uint8_t port_id, uint16_t queue_id, { > struct rte_eth_dev *dev; > > - if (port_id >= nb_ports) { > - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); > + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) > return 0; > - } > + > dev = &rte_eth_devices[port_id]; > FUNC_PTR_OR_ERR_RET(*dev->rx_pkt_burst, 0); > if (queue_id >= dev->data->nb_rx_queues) { @@ -2675,10 +2631,9 @@ > rte_eth_tx_burst(uint8_t port_id, uint16_t queue_id, { > struct rte_eth_dev *dev; > > - if (port_id >= nb_ports) { > - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); > + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) > return 0; > - } > + > dev = &rte_eth_devices[port_id]; > > FUNC_PTR_OR_ERR_RET(*dev->tx_pkt_burst, 0); @@ -2695,10 +2650,9 @@ > rte_eth_rx_queue_count(uint8_t port_id, uint16_t queue_id) { > struct rte_eth_dev *dev; > > - if (port_id >= nb_ports) { > - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); > + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) > return 0; > - } > + > dev = &rte_eth_devices[port_id]; > FUNC_PTR_OR_ERR_RET(*dev->dev_ops->rx_queue_count, 0); > return (*dev->dev_ops->rx_queue_count)(dev, queue_id); @@ -2709,10 +2663,9 @@ > rte_eth_rx_descriptor_done(uint8_t port_id, uint16_t queue_id, uint16_t offset) { > struct rte_eth_dev *dev; > > - if (port_id >= nb_ports) { > - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); > + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) > return (-ENODEV); > - } > + > dev = &rte_eth_devices[port_id]; > FUNC_PTR_OR_ERR_RET(*dev->dev_ops->rx_descriptor_done, -ENOTSUP); > return (*dev->dev_ops->rx_descriptor_done)( \ @@ -2730,10 +2683,8 @@ > rte_eth_dev_callback_register(uint8_t port_id, > > if (!cb_fn) > return (-EINVAL); > - if (port_id >= nb_ports) { > - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); > + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) > return (-EINVAL); > - } > > dev = &rte_eth_devices[port_id]; > rte_spinlock_lock(&rte_eth_dev_cb_lock); > @@ -2770,10 +2721,8 @@ rte_eth_dev_callback_unregister(uint8_t port_id, > > if (!cb_fn) > return (-EINVAL); > - if (port_id >= nb_ports) { > - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); > + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) > return (-EINVAL); > - } > > dev = &rte_eth_devices[port_id]; > rte_spinlock_lock(&rte_eth_dev_cb_lock); > @@ -2830,10 +2779,8 @@ int rte_eth_dev_bypass_init(uint8_t port_id) { > struct rte_eth_dev *dev; > > - if (port_id >= nb_ports) { > - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); > + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) > return (-ENODEV); > - } > > if ((dev= &rte_eth_devices[port_id]) == NULL) { > PMD_DEBUG_TRACE("Invalid port device\n"); @@ -2850,10 +2797,8 @@ > rte_eth_dev_bypass_state_show(uint8_t port_id, uint32_t *state) { > struct rte_eth_dev *dev; > > - if (port_id >= nb_ports) { > - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); > + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) > return (-ENODEV); > - } > > if ((dev= &rte_eth_devices[port_id]) == NULL) { > PMD_DEBUG_TRACE("Invalid port device\n"); @@ -2869,10 +2814,8 @@ > rte_eth_dev_bypass_state_set(uint8_t port_id, uint32_t *new_state) { > struct rte_eth_dev *dev; > > - if (port_id >= nb_ports) { > - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); > + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) > return (-ENODEV); > - } > > if ((dev= &rte_eth_devices[port_id]) == NULL) { > PMD_DEBUG_TRACE("Invalid port device\n"); @@ -2889,10 +2832,8 @@ > rte_eth_dev_bypass_event_show(uint8_t port_id, uint32_t event, uint32_t *state) { > struct rte_eth_dev *dev; > > - if (port_id >= nb_ports) { > - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); > + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) > return (-ENODEV); > - } > > if ((dev= &rte_eth_devices[port_id]) == NULL) { > PMD_DEBUG_TRACE("Invalid port device\n"); @@ -2909,10 +2850,8 @@ > rte_eth_dev_bypass_event_store(uint8_t port_id, uint32_t event, uint32_t state) { > struct rte_eth_dev *dev; > > - if (port_id >= nb_ports) { > - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); > + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) > return (-ENODEV); > - } > > if ((dev= &rte_eth_devices[port_id]) == NULL) { > PMD_DEBUG_TRACE("Invalid port device\n"); @@ -2929,10 +2868,8 @@ > rte_eth_dev_wd_timeout_store(uint8_t port_id, uint32_t timeout) { > struct rte_eth_dev *dev; > > - if (port_id >= nb_ports) { > - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); > + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) > return (-ENODEV); > - } > > if ((dev= &rte_eth_devices[port_id]) == NULL) { > PMD_DEBUG_TRACE("Invalid port device\n"); @@ -2949,10 +2886,8 @@ > rte_eth_dev_bypass_ver_show(uint8_t port_id, uint32_t *ver) { > struct rte_eth_dev *dev; > > - if (port_id >= nb_ports) { > - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); > + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) > return (-ENODEV); > - } > > if ((dev= &rte_eth_devices[port_id]) == NULL) { > PMD_DEBUG_TRACE("Invalid port device\n"); @@ -2969,10 +2904,8 @@ > rte_eth_dev_bypass_wd_timeout_show(uint8_t port_id, uint32_t *wd_timeout) { > struct rte_eth_dev *dev; > > - if (port_id >= nb_ports) { > - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); > + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) > return (-ENODEV); > - } > > if ((dev= &rte_eth_devices[port_id]) == NULL) { > PMD_DEBUG_TRACE("Invalid port device\n"); @@ -2989,10 +2922,8 @@ > rte_eth_dev_bypass_wd_reset(uint8_t port_id) { > struct rte_eth_dev *dev; > > - if (port_id >= nb_ports) { > - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); > + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) > return (-ENODEV); > - } > > if ((dev= &rte_eth_devices[port_id]) == NULL) { > PMD_DEBUG_TRACE("Invalid port device\n"); @@ -3011,10 +2942,8 @@ > rte_eth_dev_add_syn_filter(uint8_t port_id, { > struct rte_eth_dev *dev; > > - if (port_id >= nb_ports) { > - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); > + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) > return -ENODEV; > - } > > dev = &rte_eth_devices[port_id]; > FUNC_PTR_OR_ERR_RET(*dev->dev_ops->add_syn_filter, -ENOTSUP); @@ -3026,10 +2955,8 > @@ rte_eth_dev_remove_syn_filter(uint8_t port_id) { > struct rte_eth_dev *dev; > > - if (port_id >= nb_ports) { > - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); > + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) > return -ENODEV; > - } > > dev = &rte_eth_devices[port_id]; > FUNC_PTR_OR_ERR_RET(*dev->dev_ops->remove_syn_filter, -ENOTSUP); @@ -3045,10 > +2972,8 @@ rte_eth_dev_get_syn_filter(uint8_t port_id, > if (filter == NULL || rx_queue == NULL) > return -EINVAL; > > - if (port_id >= nb_ports) { > - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); > + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) > return -ENODEV; > - } > > dev = &rte_eth_devices[port_id]; > FUNC_PTR_OR_ERR_RET(*dev->dev_ops->get_syn_filter, -ENOTSUP); @@ -3061,10 +2986,9 > @@ rte_eth_dev_add_2tuple_filter(uint8_t port_id, uint16_t index, { > struct rte_eth_dev *dev; > > - if (port_id >= nb_ports) { > - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); > + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) > return -ENODEV; > - } > + > if (filter->protocol != IPPROTO_TCP && > filter->tcp_flags != 0){ > PMD_DEBUG_TRACE("tcp flags is 0x%x, but the protocol value" > @@ -3083,10 +3007,8 @@ rte_eth_dev_remove_2tuple_filter(uint8_t port_id, uint16_t index) { > struct rte_eth_dev *dev; > > - if (port_id >= nb_ports) { > - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); > + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) > return -ENODEV; > - } > > dev = &rte_eth_devices[port_id]; > FUNC_PTR_OR_ERR_RET(*dev->dev_ops->remove_2tuple_filter, -ENOTSUP); @@ -3102,10 > +3024,8 @@ rte_eth_dev_get_2tuple_filter(uint8_t port_id, uint16_t index, > if (filter == NULL || rx_queue == NULL) > return -EINVAL; > > - if (port_id >= nb_ports) { > - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); > + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) > return -ENODEV; > - } > > dev = &rte_eth_devices[port_id]; > FUNC_PTR_OR_ERR_RET(*dev->dev_ops->get_2tuple_filter, -ENOTSUP); @@ -3118,10 > +3038,8 @@ rte_eth_dev_add_5tuple_filter(uint8_t port_id, uint16_t index, { > struct rte_eth_dev *dev; > > - if (port_id >= nb_ports) { > - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); > + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) > return -ENODEV; > - } > > if (filter->protocol != IPPROTO_TCP && > filter->tcp_flags != 0){ > @@ -3141,10 +3059,8 @@ rte_eth_dev_remove_5tuple_filter(uint8_t port_id, uint16_t index) { > struct rte_eth_dev *dev; > > - if (port_id >= nb_ports) { > - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); > + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) > return -ENODEV; > - } > > dev = &rte_eth_devices[port_id]; > FUNC_PTR_OR_ERR_RET(*dev->dev_ops->remove_5tuple_filter, -ENOTSUP); @@ -3160,10 > +3076,8 @@ rte_eth_dev_get_5tuple_filter(uint8_t port_id, uint16_t index, > if (filter == NULL || rx_queue == NULL) > return -EINVAL; > > - if (port_id >= nb_ports) { > - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); > + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) > return -ENODEV; > - } > > dev = &rte_eth_devices[port_id]; > FUNC_PTR_OR_ERR_RET(*dev->dev_ops->get_5tuple_filter, -ENOTSUP); @@ -3177,10 > +3091,8 @@ rte_eth_dev_add_flex_filter(uint8_t port_id, uint16_t index, { > struct rte_eth_dev *dev; > > - if (port_id >= nb_ports) { > - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); > + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) > return -ENODEV; > - } > > dev = &rte_eth_devices[port_id]; > FUNC_PTR_OR_ERR_RET(*dev->dev_ops->add_flex_filter, -ENOTSUP); @@ -3192,10 +3104,8 > @@ rte_eth_dev_remove_flex_filter(uint8_t port_id, uint16_t index) { > struct rte_eth_dev *dev; > > - if (port_id >= nb_ports) { > - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); > + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) > return -ENODEV; > - } > > dev = &rte_eth_devices[port_id]; > FUNC_PTR_OR_ERR_RET(*dev->dev_ops->remove_flex_filter, -ENOTSUP); @@ -3211,10 > +3121,8 @@ rte_eth_dev_get_flex_filter(uint8_t port_id, uint16_t index, > if (filter == NULL || rx_queue == NULL) > return -EINVAL; > > - if (port_id >= nb_ports) { > - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); > + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) > return -ENODEV; > - } > > dev = &rte_eth_devices[port_id]; > FUNC_PTR_OR_ERR_RET(*dev->dev_ops->get_flex_filter, -ENOTSUP); @@ -3227,10 +3135,8 > @@ rte_eth_dev_filter_supported(uint8_t port_id, enum rte_filter_type filter_type) { > struct rte_eth_dev *dev; > > - if (port_id >= nb_ports) { > - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); > + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) > return -ENODEV; > - } > > dev = &rte_eth_devices[port_id]; > FUNC_PTR_OR_ERR_RET(*dev->dev_ops->filter_ctrl, -ENOTSUP); @@ -3244,10 +3150,8 @@ > rte_eth_dev_filter_ctrl(uint8_t port_id, enum rte_filter_type filter_type, { > struct rte_eth_dev *dev; > > - if (port_id >= nb_ports) { > - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); > + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) > return -ENODEV; > - } > > dev = &rte_eth_devices[port_id]; > FUNC_PTR_OR_ERR_RET(*dev->dev_ops->filter_ctrl, -ENOTSUP); diff --git > a/lib/librte_ether/rte_ethdev.h b/lib/librte_ether/rte_ethdev.h index 1200c1c..ca101f5 100644 > --- a/lib/librte_ether/rte_ethdev.h > +++ b/lib/librte_ether/rte_ethdev.h > @@ -1539,6 +1539,7 @@ struct rte_eth_dev { > struct eth_dev_ops *dev_ops; /**< Functions exported by PMD */ > struct rte_pci_device *pci_dev; /**< PCI info. supplied by probing */ > struct rte_eth_dev_cb_list callbacks; /**< User application callbacks */ > + uint8_t attached; /**< Flag indicating the port is attached */ > }; > > struct rte_eth_dev_sriov { > @@ -1604,6 +1605,10 @@ extern struct rte_eth_dev rte_eth_devices[]; > * initialized by the [matching] Ethernet driver during the PCI probing phase. > * All devices whose port identifier is in the range > * [0, rte_eth_dev_count() - 1] can be operated on by network applications. > + * immediately after invoking rte_eal_init(). > + * If the application unplugs a port using hotplug function, The > + enabled port > + * numbers may be noncontiguous. In the case, the applications need to > + manage > + * enabled port by themselves. > * > * @return > * - The total number of usable Ethernet devices. > -- > 1.9.1
> -----Original Message----- > From: Tetsuya Mukawa [mailto:mukawa@igel.co.jp] > Sent: Monday, February 9, 2015 8:31 AM > To: dev@dpdk.org > Cc: Iremonger, Bernard; Qiu, Michael; Tetsuya Mukawa > Subject: [PATCH v7 07/14] ethdev: Add functions that will be used by port hotplug functions > > The patch adds following functions. > > - rte_eth_dev_save() > The function is used for saving current rte_eth_dev structures. > - rte_eth_dev_get_changed_port() > The function receives the rte_eth_dev structures, then compare > these with current values to know which port is actually > attached or detached. > - rte_eth_dev_get_addr_by_port() > The function returns a pci address of an ethdev specified by port > identifier. > - rte_eth_dev_get_port_by_addr() > The function returns a port identifier of an ethdev specified by > pci address. > - rte_eth_dev_get_name_by_port() > The function returns a unique identifier name of an ethdev > specified by port identifier. > - Add rte_eth_dev_check_detachable() > The function returns whether a PMD supports detach function. > > Also, the patch changes scope of rte_eth_dev_allocated() to global. > This function will be called by virtual PMDs to support port hotplug. > So change scope of the function to global. > > v7: > - Add pt_driver checking to rte_eth_dev_check_detachable(). > (Thanks to Qiu, Michael) > v5: > - Fix return value of below functions. > rte_eth_dev_get_changed_port(). > rte_eth_dev_get_port_by_addr(). > v4: > - Add parameter checking. > v3: > - Fix if-condition bug while comparing pci addresses. > - Add error checking codes. > Reported-by: Mark Enright <menrigh@brocade.com> > > Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp> > --- > lib/librte_ether/rte_ethdev.c | 109 +++++++++++++++++++++++++++++++++++++++++- > lib/librte_ether/rte_ethdev.h | 80 +++++++++++++++++++++++++++++++ > 2 files changed, 188 insertions(+), 1 deletion(-) > > diff --git a/lib/librte_ether/rte_ethdev.c b/lib/librte_ether/rte_ethdev.c index 7bed901..14a040a > 100644 > --- a/lib/librte_ether/rte_ethdev.c > +++ b/lib/librte_ether/rte_ethdev.c > @@ -206,7 +206,7 @@ rte_eth_dev_data_alloc(void) > RTE_MAX_ETHPORTS * sizeof(*rte_eth_dev_data)); } > > -static struct rte_eth_dev * > +struct rte_eth_dev * > rte_eth_dev_allocated(const char *name) { > unsigned i; > @@ -426,6 +426,113 @@ rte_eth_dev_count(void) > return (nb_ports); > } > > +void > +rte_eth_dev_save(struct rte_eth_dev *devs) { > + if (devs == NULL) > + return; > + Hi Tetsuya, This function should probably have an input parameter for the size of the devs buffer. This input parameter should be used with the memcpy function to ensure that nothing is overwritten. Regards, Bernard. > + /* save current rte_eth_devices */ > + memcpy(devs, rte_eth_devices, > + sizeof(struct rte_eth_dev) * RTE_MAX_ETHPORTS); } > + > +int > +rte_eth_dev_get_changed_port(struct rte_eth_dev *devs, uint8_t > +*port_id) { > + if ((devs == NULL) || (port_id == NULL)) > + return -EINVAL; > + > + /* check which port was attached or detached */ > + for (*port_id = 0; *port_id < RTE_MAX_ETHPORTS; (*port_id)++, devs++) { > + if (rte_eth_devices[*port_id].attached ^ devs->attached) > + return 0; > + } > + return -ENODEV; > +} > + > +int > +rte_eth_dev_get_addr_by_port(uint8_t port_id, struct rte_pci_addr > +*addr) { > + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) > + return -EINVAL; > + > + if (addr == NULL) { > + PMD_DEBUG_TRACE("Null pointer is specified\n"); > + return -EINVAL; > + } > + > + *addr = rte_eth_devices[port_id].pci_dev->addr; > + return 0; > +} > + > +int > +rte_eth_dev_get_port_by_addr(struct rte_pci_addr *addr, uint8_t > +*port_id) { > + struct rte_pci_addr *tmp; > + > + if ((addr == NULL) || (port_id == NULL)) { > + PMD_DEBUG_TRACE("Null pointer is specified\n"); > + return -EINVAL; > + } > + > + for (*port_id = 0; *port_id < RTE_MAX_ETHPORTS; (*port_id)++) { > + if (!rte_eth_devices[*port_id].attached) > + continue; > + if (!rte_eth_devices[*port_id].pci_dev) > + continue; > + tmp = &rte_eth_devices[*port_id].pci_dev->addr; > + if (eal_compare_pci_addr(tmp, addr) == 0) > + return 0; > + } > + return -ENODEV; > +} > + > +int > +rte_eth_dev_get_name_by_port(uint8_t port_id, char *name) { > + char *tmp; > + > + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) > + return -EINVAL; > + > + if (name == NULL) { > + PMD_DEBUG_TRACE("Null pointer is specified\n"); > + return -EINVAL; > + } > + > + /* shouldn't check 'rte_eth_devices[i].data', > + * because it might be overwritten by VDEV PMD */ > + tmp = rte_eth_dev_data[port_id].name; > + strncpy(name, tmp, strlen(tmp) + 1); > + return 0; > +} > + > +int > +rte_eth_dev_check_detachable(uint8_t port_id) { > + uint32_t drv_flags; > + > + if (port_id >= RTE_MAX_ETHPORTS) { > + PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); > + return -EINVAL; > + } > + > + if (rte_eth_devices[port_id].dev_type == RTE_ETH_DEV_PHYSICAL) { > + switch (rte_eth_devices[port_id].pci_dev->pt_driver) { > + case RTE_PT_IGB_UIO: > + case RTE_PT_UIO_GENERIC: > + break; > + case RTE_PT_VFIO: > + default: > + return -ENOTSUP; > + } > + } > + > + drv_flags = rte_eth_devices[port_id].driver->pci_drv.drv_flags; > + return !(drv_flags & RTE_PCI_DRV_DETACHABLE); } > + > static int > rte_eth_dev_rx_queue_config(struct rte_eth_dev *dev, uint16_t nb_queues) { diff --git > a/lib/librte_ether/rte_ethdev.h b/lib/librte_ether/rte_ethdev.h index 91d9e86..9919968 100644 > --- a/lib/librte_ether/rte_ethdev.h > +++ b/lib/librte_ether/rte_ethdev.h > @@ -1616,6 +1616,86 @@ extern struct rte_eth_dev rte_eth_devices[]; extern uint8_t > rte_eth_dev_count(void); > > /** > + * Function for internal use by port hotplug functions. > + * Copies current ethdev structures to the specified pointer. > + * > + * @param devs The pointer to the ethdev structures > + */ > +extern void rte_eth_dev_save(struct rte_eth_dev *devs); > + > +/** > + * Function for internal use by port hotplug functions. > + * Compare the specified ethdev structures with currents. Then > + * if there is a port which status is changed, fill the specified > +pointer > + * with the port id of that port. > + * @param devs The pointer to the ethdev structures > + * @param port_id The pointer to the port id > + * @return > + * - 0 on success, negative on error > + */ > +extern int rte_eth_dev_get_changed_port( > + struct rte_eth_dev *devs, uint8_t *port_id); > + > +/** > + * Function for internal use by port hotplug functions. > + * Returns a pci address of a ethdev specified by port identifier. > + * @param port_id > + * The port identifier of the Ethernet device > + * @param addr > + * The pointer to the pci address > + * @return > + * - 0 on success, negative on error > + */ > +extern int rte_eth_dev_get_addr_by_port( > + uint8_t port_id, struct rte_pci_addr *addr); > + > +/** > + * Function for internal use by port hotplug functions. > + * Returns a port identifier of a ethdev specified by pci address. > + * @param addr > + * The pointer to the pci address of the Ethernet device. > + * @param port_id > + * The pointer to the port identifier > + * @return > + * - 0 on success, negative on error > + */ > +extern int rte_eth_dev_get_port_by_addr( > + struct rte_pci_addr *addr, uint8_t *port_id); > + > +/** > + * Function for internal use by port hotplug functions. > + * Returns a unique identifier name of a ethdev specified by port identifier. > + * @param port_id > + * The port identifier. > + * @param name > + * The pointer to the Unique identifier name for each Ethernet device > + * @return > + * - 0 on success, negative on error > + */ > +extern int rte_eth_dev_get_name_by_port(uint8_t port_id, char *name); > + > +/** > + * Function for internal use by port hotplug functions. > + * Check whether or not, a PMD that is handling the ethdev specified by > +port > + * identifier can support detach function. > + * @param port_id > + * The port identifier > + * @return > + * - 0 on supporting detach function, negative on not supporting > + */ > +extern int rte_eth_dev_check_detachable(uint8_t port_id); > + > +/** > + * Function for internal use by port hotplug functions. > + * Returns a ethdev slot specified by the unique identifier name. > + * @param name > + * The pointer to the Unique identifier name for each Ethernet device > + * @return > + * - The pointer to the ethdev slot, on success. NULL on error > + */ > +extern struct rte_eth_dev *rte_eth_dev_allocated(const char *name); > + > +/** > * Function for internal use by dummy drivers primarily, e.g. ring-based > * driver. > * Allocates a new ethdev slot for an ethernet device and returns the pointer > -- > 1.9.1
On 2015/02/10 0:34, Iremonger, Bernard wrote: > >> -----Original Message----- >> From: Tetsuya Mukawa [mailto:mukawa@igel.co.jp] >> Sent: Monday, February 9, 2015 8:31 AM >> To: dev@dpdk.org >> Cc: Iremonger, Bernard; Qiu, Michael; Tetsuya Mukawa >> Subject: [PATCH v7 07/14] ethdev: Add functions that will be used by port hotplug functions >> >> The patch adds following functions. >> >> - rte_eth_dev_save() >> The function is used for saving current rte_eth_dev structures. >> - rte_eth_dev_get_changed_port() >> The function receives the rte_eth_dev structures, then compare >> these with current values to know which port is actually >> attached or detached. >> - rte_eth_dev_get_addr_by_port() >> The function returns a pci address of an ethdev specified by port >> identifier. >> - rte_eth_dev_get_port_by_addr() >> The function returns a port identifier of an ethdev specified by >> pci address. >> - rte_eth_dev_get_name_by_port() >> The function returns a unique identifier name of an ethdev >> specified by port identifier. >> - Add rte_eth_dev_check_detachable() >> The function returns whether a PMD supports detach function. >> >> Also, the patch changes scope of rte_eth_dev_allocated() to global. >> This function will be called by virtual PMDs to support port hotplug. >> So change scope of the function to global. >> >> v7: >> - Add pt_driver checking to rte_eth_dev_check_detachable(). >> (Thanks to Qiu, Michael) >> v5: >> - Fix return value of below functions. >> rte_eth_dev_get_changed_port(). >> rte_eth_dev_get_port_by_addr(). >> v4: >> - Add parameter checking. >> v3: >> - Fix if-condition bug while comparing pci addresses. >> - Add error checking codes. >> Reported-by: Mark Enright <menrigh@brocade.com> >> >> Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp> >> --- >> lib/librte_ether/rte_ethdev.c | 109 +++++++++++++++++++++++++++++++++++++++++- >> lib/librte_ether/rte_ethdev.h | 80 +++++++++++++++++++++++++++++++ >> 2 files changed, 188 insertions(+), 1 deletion(-) >> >> diff --git a/lib/librte_ether/rte_ethdev.c b/lib/librte_ether/rte_ethdev.c index 7bed901..14a040a >> 100644 >> --- a/lib/librte_ether/rte_ethdev.c >> +++ b/lib/librte_ether/rte_ethdev.c >> @@ -206,7 +206,7 @@ rte_eth_dev_data_alloc(void) >> RTE_MAX_ETHPORTS * sizeof(*rte_eth_dev_data)); } >> >> -static struct rte_eth_dev * >> +struct rte_eth_dev * >> rte_eth_dev_allocated(const char *name) { >> unsigned i; >> @@ -426,6 +426,113 @@ rte_eth_dev_count(void) >> return (nb_ports); >> } >> >> +void >> +rte_eth_dev_save(struct rte_eth_dev *devs) { >> + if (devs == NULL) >> + return; >> + > Hi Tetsuya, > > This function should probably have an input parameter for the size of the devs buffer. > This input parameter should be used with the memcpy function to ensure that nothing is overwritten. Hi Bernard, I am sorry that I forgot to fix issues your suggested. (I checked the patchwork to submit v7 patches, and I forgot that cover letter wasn't involved in. So all suggestion to cover letter were not involved in v7 patches. I am sorry for that.) I will fix your all suggestions and submit again by this weekend. Regards, Tetsuya > Regards, > > Bernard. > >> + /* save current rte_eth_devices */ >> + memcpy(devs, rte_eth_devices, >> + sizeof(struct rte_eth_dev) * RTE_MAX_ETHPORTS); } >> + >> +int >> +rte_eth_dev_get_changed_port(struct rte_eth_dev *devs, uint8_t >> +*port_id) { >> + if ((devs == NULL) || (port_id == NULL)) >> + return -EINVAL; >> + >> + /* check which port was attached or detached */ >> + for (*port_id = 0; *port_id < RTE_MAX_ETHPORTS; (*port_id)++, devs++) { >> + if (rte_eth_devices[*port_id].attached ^ devs->attached) >> + return 0; >> + } >> + return -ENODEV; >> +} >> + >> +int >> +rte_eth_dev_get_addr_by_port(uint8_t port_id, struct rte_pci_addr >> +*addr) { >> + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) >> + return -EINVAL; >> + >> + if (addr == NULL) { >> + PMD_DEBUG_TRACE("Null pointer is specified\n"); >> + return -EINVAL; >> + } >> + >> + *addr = rte_eth_devices[port_id].pci_dev->addr; >> + return 0; >> +} >> + >> +int >> +rte_eth_dev_get_port_by_addr(struct rte_pci_addr *addr, uint8_t >> +*port_id) { >> + struct rte_pci_addr *tmp; >> + >> + if ((addr == NULL) || (port_id == NULL)) { >> + PMD_DEBUG_TRACE("Null pointer is specified\n"); >> + return -EINVAL; >> + } >> + >> + for (*port_id = 0; *port_id < RTE_MAX_ETHPORTS; (*port_id)++) { >> + if (!rte_eth_devices[*port_id].attached) >> + continue; >> + if (!rte_eth_devices[*port_id].pci_dev) >> + continue; >> + tmp = &rte_eth_devices[*port_id].pci_dev->addr; >> + if (eal_compare_pci_addr(tmp, addr) == 0) >> + return 0; >> + } >> + return -ENODEV; >> +} >> + >> +int >> +rte_eth_dev_get_name_by_port(uint8_t port_id, char *name) { >> + char *tmp; >> + >> + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) >> + return -EINVAL; >> + >> + if (name == NULL) { >> + PMD_DEBUG_TRACE("Null pointer is specified\n"); >> + return -EINVAL; >> + } >> + >> + /* shouldn't check 'rte_eth_devices[i].data', >> + * because it might be overwritten by VDEV PMD */ >> + tmp = rte_eth_dev_data[port_id].name; >> + strncpy(name, tmp, strlen(tmp) + 1); >> + return 0; >> +} >> + >> +int >> +rte_eth_dev_check_detachable(uint8_t port_id) { >> + uint32_t drv_flags; >> + >> + if (port_id >= RTE_MAX_ETHPORTS) { >> + PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); >> + return -EINVAL; >> + } >> + >> + if (rte_eth_devices[port_id].dev_type == RTE_ETH_DEV_PHYSICAL) { >> + switch (rte_eth_devices[port_id].pci_dev->pt_driver) { >> + case RTE_PT_IGB_UIO: >> + case RTE_PT_UIO_GENERIC: >> + break; >> + case RTE_PT_VFIO: >> + default: >> + return -ENOTSUP; >> + } >> + } >> + >> + drv_flags = rte_eth_devices[port_id].driver->pci_drv.drv_flags; >> + return !(drv_flags & RTE_PCI_DRV_DETACHABLE); } >> + >> static int >> rte_eth_dev_rx_queue_config(struct rte_eth_dev *dev, uint16_t nb_queues) { diff --git >> a/lib/librte_ether/rte_ethdev.h b/lib/librte_ether/rte_ethdev.h index 91d9e86..9919968 100644 >> --- a/lib/librte_ether/rte_ethdev.h >> +++ b/lib/librte_ether/rte_ethdev.h >> @@ -1616,6 +1616,86 @@ extern struct rte_eth_dev rte_eth_devices[]; extern uint8_t >> rte_eth_dev_count(void); >> >> /** >> + * Function for internal use by port hotplug functions. >> + * Copies current ethdev structures to the specified pointer. >> + * >> + * @param devs The pointer to the ethdev structures >> + */ >> +extern void rte_eth_dev_save(struct rte_eth_dev *devs); >> + >> +/** >> + * Function for internal use by port hotplug functions. >> + * Compare the specified ethdev structures with currents. Then >> + * if there is a port which status is changed, fill the specified >> +pointer >> + * with the port id of that port. >> + * @param devs The pointer to the ethdev structures >> + * @param port_id The pointer to the port id >> + * @return >> + * - 0 on success, negative on error >> + */ >> +extern int rte_eth_dev_get_changed_port( >> + struct rte_eth_dev *devs, uint8_t *port_id); >> + >> +/** >> + * Function for internal use by port hotplug functions. >> + * Returns a pci address of a ethdev specified by port identifier. >> + * @param port_id >> + * The port identifier of the Ethernet device >> + * @param addr >> + * The pointer to the pci address >> + * @return >> + * - 0 on success, negative on error >> + */ >> +extern int rte_eth_dev_get_addr_by_port( >> + uint8_t port_id, struct rte_pci_addr *addr); >> + >> +/** >> + * Function for internal use by port hotplug functions. >> + * Returns a port identifier of a ethdev specified by pci address. >> + * @param addr >> + * The pointer to the pci address of the Ethernet device. >> + * @param port_id >> + * The pointer to the port identifier >> + * @return >> + * - 0 on success, negative on error >> + */ >> +extern int rte_eth_dev_get_port_by_addr( >> + struct rte_pci_addr *addr, uint8_t *port_id); >> + >> +/** >> + * Function for internal use by port hotplug functions. >> + * Returns a unique identifier name of a ethdev specified by port identifier. >> + * @param port_id >> + * The port identifier. >> + * @param name >> + * The pointer to the Unique identifier name for each Ethernet device >> + * @return >> + * - 0 on success, negative on error >> + */ >> +extern int rte_eth_dev_get_name_by_port(uint8_t port_id, char *name); >> + >> +/** >> + * Function for internal use by port hotplug functions. >> + * Check whether or not, a PMD that is handling the ethdev specified by >> +port >> + * identifier can support detach function. >> + * @param port_id >> + * The port identifier >> + * @return >> + * - 0 on supporting detach function, negative on not supporting >> + */ >> +extern int rte_eth_dev_check_detachable(uint8_t port_id); >> + >> +/** >> + * Function for internal use by port hotplug functions. >> + * Returns a ethdev slot specified by the unique identifier name. >> + * @param name >> + * The pointer to the Unique identifier name for each Ethernet device >> + * @return >> + * - The pointer to the ethdev slot, on success. NULL on error >> + */ >> +extern struct rte_eth_dev *rte_eth_dev_allocated(const char *name); >> + >> +/** >> * Function for internal use by dummy drivers primarily, e.g. ring-based >> * driver. >> * Allocates a new ethdev slot for an ethernet device and returns the pointer >> -- >> 1.9.1
> -----Original Message----- > From: Qiu, Michael > Sent: Monday, February 9, 2015 1:10 PM > To: Tetsuya Mukawa; dev@dpdk.org > Cc: Iremonger, Bernard > Subject: Re: [PATCH v7 04/14] eal/pci: Consolidate pci address comparison APIs > > On 2/9/2015 4:31 PM, Tetsuya Mukawa wrote: > > This patch replaces pci_addr_comparison() and memcmp() of pci > > addresses by eal_compare_pci_addr(). > > > > v5: > > - Fix pci_scan_one to handle pt_driver correctly. > > v4: > > - Fix calculation method of eal_compare_pci_addr(). > > - Add parameter checking. > > > > Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp> > > --- > > lib/librte_eal/bsdapp/eal/eal_pci.c | 25 ++++++++--------------- > > lib/librte_eal/common/eal_common_pci.c | 2 +- > > lib/librte_eal/common/include/rte_pci.h | 34 +++++++++++++++++++++++++++++++ > > lib/librte_eal/linuxapp/eal/eal_pci.c | 25 ++++++++--------------- > > lib/librte_eal/linuxapp/eal/eal_pci_uio.c | 2 +- > > 5 files changed, 54 insertions(+), 34 deletions(-) > > > > diff --git a/lib/librte_eal/bsdapp/eal/eal_pci.c > > b/lib/librte_eal/bsdapp/eal/eal_pci.c > > index 74ecce7..c844d58 100644 > > --- a/lib/librte_eal/bsdapp/eal/eal_pci.c > > +++ b/lib/librte_eal/bsdapp/eal/eal_pci.c > > @@ -270,20 +270,6 @@ pci_uio_map_resource(struct rte_pci_device *dev) > > return (0); > > } > > > > -/* Compare two PCI device addresses. */ -static int > > -pci_addr_comparison(struct rte_pci_addr *addr, struct rte_pci_addr > > *addr2) -{ > > - uint64_t dev_addr = (addr->domain << 24) + (addr->bus << 16) + (addr->devid << 8) + addr- > >function; > > - uint64_t dev_addr2 = (addr2->domain << 24) + (addr2->bus << 16) + (addr2->devid << 8) + > addr2->function; > > - > > - if (dev_addr > dev_addr2) > > - return 1; > > - else > > - return 0; > > -} > > - > > - > > /* Scan one pci sysfs entry, and fill the devices list from it. */ > > static int pci_scan_one(int dev_pci_fd, struct pci_conf *conf) @@ > > -356,13 +342,20 @@ pci_scan_one(int dev_pci_fd, struct pci_conf *conf) > > } > > else { > > struct rte_pci_device *dev2 = NULL; > > + int ret; > > > > TAILQ_FOREACH(dev2, &pci_device_list, next) { > > - if (pci_addr_comparison(&dev->addr, &dev2->addr)) > > + ret = eal_compare_pci_addr(&dev->addr, &dev2->addr); > > + if (ret > 0) > > continue; > > - else { > > + else if (ret < 0) { > > TAILQ_INSERT_BEFORE(dev2, dev, next); > > return 0; > > + } else { /* already registered */ > > + /* update pt_driver */ > > + dev2->pt_driver = dev->pt_driver; > > + free(dev); > > + return 0; > > } > > } > > TAILQ_INSERT_TAIL(&pci_device_list, dev, next); diff --git > > a/lib/librte_eal/common/eal_common_pci.c > > b/lib/librte_eal/common/eal_common_pci.c > > index f3c7f71..a89f5c3 100644 > > --- a/lib/librte_eal/common/eal_common_pci.c > > +++ b/lib/librte_eal/common/eal_common_pci.c > > @@ -93,7 +93,7 @@ static struct rte_devargs *pci_devargs_lookup(struct rte_pci_device *dev) > > if (devargs->type != RTE_DEVTYPE_BLACKLISTED_PCI && > > devargs->type != RTE_DEVTYPE_WHITELISTED_PCI) > > continue; > > - if (!memcmp(&dev->addr, &devargs->pci.addr, sizeof(dev->addr))) > > + if (!eal_compare_pci_addr(&dev->addr, &devargs->pci.addr)) > > return devargs; > > } > > return NULL; > > diff --git a/lib/librte_eal/common/include/rte_pci.h > > b/lib/librte_eal/common/include/rte_pci.h > > index 7f2d699..4814cd7 100644 > > --- a/lib/librte_eal/common/include/rte_pci.h > > +++ b/lib/librte_eal/common/include/rte_pci.h > > @@ -269,6 +269,40 @@ eal_parse_pci_DomBDF(const char *input, struct > > rte_pci_addr *dev_addr) } #undef GET_PCIADDR_FIELD > > > > +/* Compare two PCI device addresses. */ > > +/** > > + * Utility function to compare two PCI device addresses. > > + * > > + * @param addr > > + * The PCI Bus-Device-Function address to compare > > + * @param addr2 > > + * The PCI Bus-Device-Function address to compare > > + * @return > > + * 0 on equal PCI address. > > + * Positive on addr is greater than addr2. > > + * Negative on addr is less than addr2, or error. > > + */ > > +static inline int > > +eal_compare_pci_addr(struct rte_pci_addr *addr, struct rte_pci_addr > > +*addr2) { > > + uint64_t dev_addr, dev_addr2; > > + > > + if ((addr == NULL) || (addr2 == NULL)) > > + return -1; > > + > > + dev_addr = (addr->domain << 24) | (addr->bus << 16) | > > + (addr->devid << 8) | addr->function; > > + dev_addr2 = (addr2->domain << 24) | (addr2->bus << 16) | > > + (addr2->devid << 8) | addr2->function; > > + > > + if (dev_addr > dev_addr2) > > + return 1; > > + else if (dev_addr < dev_addr2) > > + return -1; > > + else > > + return 0; > > +} > > + > > /** > > * Probe the PCI bus for registered drivers. > > * > > diff --git a/lib/librte_eal/linuxapp/eal/eal_pci.c > > b/lib/librte_eal/linuxapp/eal/eal_pci.c > > index c0ca5a5..d847102 100644 > > --- a/lib/librte_eal/linuxapp/eal/eal_pci.c > > +++ b/lib/librte_eal/linuxapp/eal/eal_pci.c > > @@ -229,20 +229,6 @@ error: > > return -1; > > } > > > > -/* Compare two PCI device addresses. */ -static int > > -pci_addr_comparison(struct rte_pci_addr *addr, struct rte_pci_addr > > *addr2) -{ > > - uint64_t dev_addr = (addr->domain << 24) + (addr->bus << 16) + (addr->devid << 8) + addr- > >function; > > - uint64_t dev_addr2 = (addr2->domain << 24) + (addr2->bus << 16) + (addr2->devid << 8) + > addr2->function; > > - > > - if (dev_addr > dev_addr2) > > - return 1; > > - else > > - return 0; > > -} > > - > > - > > /* Scan one pci sysfs entry, and fill the devices list from it. */ > > static int pci_scan_one(const char *dirname, uint16_t domain, uint8_t > > bus, @@ -353,13 +339,20 @@ pci_scan_one(const char *dirname, uint16_t > > domain, uint8_t bus, > > } > > else { > > struct rte_pci_device *dev2 = NULL; > > + int ret; > > > > TAILQ_FOREACH(dev2, &pci_device_list, next) { > > - if (pci_addr_comparison(&dev->addr, &dev2->addr)) > > + ret = eal_compare_pci_addr(&dev->addr, &dev2->addr); > > + if (ret > 0) > > continue; > > - else { > > + else if (ret < 0) { > > TAILQ_INSERT_BEFORE(dev2, dev, next); > > return 0; > > + } else { /* already registered */ > > + /* update pt_driver */ > > + dev2->pt_driver = dev->pt_driver; Hi Tetsuya, I am seeing a problem with the librte_pmd_ixgbe code where dev->max_vfs is being lost in some scenarios. The following line should be added here: dev2->max_vfs = dev->max_vfs; numa_mode should probably be updated too (although it is not causing a problem at present). dev2->numa_mode = dev->numa_mode; Regards, Bernard. > > + free(dev); > > + return 0; > > } > > } > > TAILQ_INSERT_TAIL(&pci_device_list, dev, next); diff --git > > a/lib/librte_eal/linuxapp/eal/eal_pci_uio.c > > b/lib/librte_eal/linuxapp/eal/eal_pci_uio.c > > index e53f06b..1da3507 100644 > > --- a/lib/librte_eal/linuxapp/eal/eal_pci_uio.c > > +++ b/lib/librte_eal/linuxapp/eal/eal_pci_uio.c > > @@ -123,7 +123,7 @@ pci_uio_map_secondary(struct rte_pci_device *dev) > > TAILQ_FOREACH(uio_res, pci_res_list, next) { > > > > /* skip this element if it doesn't match our PCI address */ > > - if (memcmp(&uio_res->pci_addr, &dev->addr, sizeof(dev->addr))) > > + if (eal_compare_pci_addr(&uio_res->pci_addr, &dev->addr)) > > continue; > > > > for (i = 0; i != uio_res->nb_maps; i++) { > Acked-by: Michael Qiu <michael.qiu@intel.com>
On 2015/02/11 0:08, Iremonger, Bernard wrote: >> -----Original Message----- >> From: Qiu, Michael >> Sent: Monday, February 9, 2015 1:10 PM >> To: Tetsuya Mukawa; dev@dpdk.org >> Cc: Iremonger, Bernard >> Subject: Re: [PATCH v7 04/14] eal/pci: Consolidate pci address comparison APIs >> >> On 2/9/2015 4:31 PM, Tetsuya Mukawa wrote: >>> This patch replaces pci_addr_comparison() and memcmp() of pci >>> addresses by eal_compare_pci_addr(). >>> >>> v5: >>> - Fix pci_scan_one to handle pt_driver correctly. >>> v4: >>> - Fix calculation method of eal_compare_pci_addr(). >>> - Add parameter checking. >>> >>> Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp> >>> --- >>> lib/librte_eal/bsdapp/eal/eal_pci.c | 25 ++++++++--------------- >>> lib/librte_eal/common/eal_common_pci.c | 2 +- >>> lib/librte_eal/common/include/rte_pci.h | 34 +++++++++++++++++++++++++++++++ >>> lib/librte_eal/linuxapp/eal/eal_pci.c | 25 ++++++++--------------- >>> lib/librte_eal/linuxapp/eal/eal_pci_uio.c | 2 +- >>> 5 files changed, 54 insertions(+), 34 deletions(-) >>> >>> diff --git a/lib/librte_eal/bsdapp/eal/eal_pci.c >>> b/lib/librte_eal/bsdapp/eal/eal_pci.c >>> index 74ecce7..c844d58 100644 >>> --- a/lib/librte_eal/bsdapp/eal/eal_pci.c >>> +++ b/lib/librte_eal/bsdapp/eal/eal_pci.c >>> @@ -270,20 +270,6 @@ pci_uio_map_resource(struct rte_pci_device *dev) >>> return (0); >>> } >>> >>> -/* Compare two PCI device addresses. */ -static int >>> -pci_addr_comparison(struct rte_pci_addr *addr, struct rte_pci_addr >>> *addr2) -{ >>> - uint64_t dev_addr = (addr->domain << 24) + (addr->bus << 16) + (addr->devid << 8) + addr- >>> function; >>> - uint64_t dev_addr2 = (addr2->domain << 24) + (addr2->bus << 16) + (addr2->devid << 8) + >> addr2->function; >>> - >>> - if (dev_addr > dev_addr2) >>> - return 1; >>> - else >>> - return 0; >>> -} >>> - >>> - >>> /* Scan one pci sysfs entry, and fill the devices list from it. */ >>> static int pci_scan_one(int dev_pci_fd, struct pci_conf *conf) @@ >>> -356,13 +342,20 @@ pci_scan_one(int dev_pci_fd, struct pci_conf *conf) >>> } >>> else { >>> struct rte_pci_device *dev2 = NULL; >>> + int ret; >>> >>> TAILQ_FOREACH(dev2, &pci_device_list, next) { >>> - if (pci_addr_comparison(&dev->addr, &dev2->addr)) >>> + ret = eal_compare_pci_addr(&dev->addr, &dev2->addr); >>> + if (ret > 0) >>> continue; >>> - else { >>> + else if (ret < 0) { >>> TAILQ_INSERT_BEFORE(dev2, dev, next); >>> return 0; >>> + } else { /* already registered */ >>> + /* update pt_driver */ >>> + dev2->pt_driver = dev->pt_driver; >>> + free(dev); >>> + return 0; >>> } >>> } >>> TAILQ_INSERT_TAIL(&pci_device_list, dev, next); diff --git >>> a/lib/librte_eal/common/eal_common_pci.c >>> b/lib/librte_eal/common/eal_common_pci.c >>> index f3c7f71..a89f5c3 100644 >>> --- a/lib/librte_eal/common/eal_common_pci.c >>> +++ b/lib/librte_eal/common/eal_common_pci.c >>> @@ -93,7 +93,7 @@ static struct rte_devargs *pci_devargs_lookup(struct rte_pci_device *dev) >>> if (devargs->type != RTE_DEVTYPE_BLACKLISTED_PCI && >>> devargs->type != RTE_DEVTYPE_WHITELISTED_PCI) >>> continue; >>> - if (!memcmp(&dev->addr, &devargs->pci.addr, sizeof(dev->addr))) >>> + if (!eal_compare_pci_addr(&dev->addr, &devargs->pci.addr)) >>> return devargs; >>> } >>> return NULL; >>> diff --git a/lib/librte_eal/common/include/rte_pci.h >>> b/lib/librte_eal/common/include/rte_pci.h >>> index 7f2d699..4814cd7 100644 >>> --- a/lib/librte_eal/common/include/rte_pci.h >>> +++ b/lib/librte_eal/common/include/rte_pci.h >>> @@ -269,6 +269,40 @@ eal_parse_pci_DomBDF(const char *input, struct >>> rte_pci_addr *dev_addr) } #undef GET_PCIADDR_FIELD >>> >>> +/* Compare two PCI device addresses. */ >>> +/** >>> + * Utility function to compare two PCI device addresses. >>> + * >>> + * @param addr >>> + * The PCI Bus-Device-Function address to compare >>> + * @param addr2 >>> + * The PCI Bus-Device-Function address to compare >>> + * @return >>> + * 0 on equal PCI address. >>> + * Positive on addr is greater than addr2. >>> + * Negative on addr is less than addr2, or error. >>> + */ >>> +static inline int >>> +eal_compare_pci_addr(struct rte_pci_addr *addr, struct rte_pci_addr >>> +*addr2) { >>> + uint64_t dev_addr, dev_addr2; >>> + >>> + if ((addr == NULL) || (addr2 == NULL)) >>> + return -1; >>> + >>> + dev_addr = (addr->domain << 24) | (addr->bus << 16) | >>> + (addr->devid << 8) | addr->function; >>> + dev_addr2 = (addr2->domain << 24) | (addr2->bus << 16) | >>> + (addr2->devid << 8) | addr2->function; >>> + >>> + if (dev_addr > dev_addr2) >>> + return 1; >>> + else if (dev_addr < dev_addr2) >>> + return -1; >>> + else >>> + return 0; >>> +} >>> + >>> /** >>> * Probe the PCI bus for registered drivers. >>> * >>> diff --git a/lib/librte_eal/linuxapp/eal/eal_pci.c >>> b/lib/librte_eal/linuxapp/eal/eal_pci.c >>> index c0ca5a5..d847102 100644 >>> --- a/lib/librte_eal/linuxapp/eal/eal_pci.c >>> +++ b/lib/librte_eal/linuxapp/eal/eal_pci.c >>> @@ -229,20 +229,6 @@ error: >>> return -1; >>> } >>> >>> -/* Compare two PCI device addresses. */ -static int >>> -pci_addr_comparison(struct rte_pci_addr *addr, struct rte_pci_addr >>> *addr2) -{ >>> - uint64_t dev_addr = (addr->domain << 24) + (addr->bus << 16) + (addr->devid << 8) + addr- >>> function; >>> - uint64_t dev_addr2 = (addr2->domain << 24) + (addr2->bus << 16) + (addr2->devid << 8) + >> addr2->function; >>> - >>> - if (dev_addr > dev_addr2) >>> - return 1; >>> - else >>> - return 0; >>> -} >>> - >>> - >>> /* Scan one pci sysfs entry, and fill the devices list from it. */ >>> static int pci_scan_one(const char *dirname, uint16_t domain, uint8_t >>> bus, @@ -353,13 +339,20 @@ pci_scan_one(const char *dirname, uint16_t >>> domain, uint8_t bus, >>> } >>> else { >>> struct rte_pci_device *dev2 = NULL; >>> + int ret; >>> >>> TAILQ_FOREACH(dev2, &pci_device_list, next) { >>> - if (pci_addr_comparison(&dev->addr, &dev2->addr)) >>> + ret = eal_compare_pci_addr(&dev->addr, &dev2->addr); >>> + if (ret > 0) >>> continue; >>> - else { >>> + else if (ret < 0) { >>> TAILQ_INSERT_BEFORE(dev2, dev, next); >>> return 0; >>> + } else { /* already registered */ >>> + /* update pt_driver */ >>> + dev2->pt_driver = dev->pt_driver; > Hi Tetsuya, > > I am seeing a problem with the librte_pmd_ixgbe code where dev->max_vfs is being lost in some scenarios. > The following line should be added here: > dev2->max_vfs = dev->max_vfs; > > numa_mode should probably be updated too (although it is not causing a problem at present). > dev2->numa_mode = dev->numa_mode; Hi Bernard, Thanks for reporting. I will add above 2 lines. Regards, Tetsuya > Regards, > > Bernard. > > > > >>> + free(dev); >>> + return 0; >>> } >>> } >>> TAILQ_INSERT_TAIL(&pci_device_list, dev, next); diff --git >>> a/lib/librte_eal/linuxapp/eal/eal_pci_uio.c >>> b/lib/librte_eal/linuxapp/eal/eal_pci_uio.c >>> index e53f06b..1da3507 100644 >>> --- a/lib/librte_eal/linuxapp/eal/eal_pci_uio.c >>> +++ b/lib/librte_eal/linuxapp/eal/eal_pci_uio.c >>> @@ -123,7 +123,7 @@ pci_uio_map_secondary(struct rte_pci_device *dev) >>> TAILQ_FOREACH(uio_res, pci_res_list, next) { >>> >>> /* skip this element if it doesn't match our PCI address */ >>> - if (memcmp(&uio_res->pci_addr, &dev->addr, sizeof(dev->addr))) >>> + if (eal_compare_pci_addr(&uio_res->pci_addr, &dev->addr)) >>> continue; >>> >>> for (i = 0; i != uio_res->nb_maps; i++) { >> Acked-by: Michael Qiu <michael.qiu@intel.com>
On 2/10/2015 11:11 PM, Iremonger, Bernard wrote: >> -----Original Message----- >> From: Qiu, Michael >> Sent: Monday, February 9, 2015 1:10 PM >> To: Tetsuya Mukawa; dev@dpdk.org >> Cc: Iremonger, Bernard >> Subject: Re: [PATCH v7 04/14] eal/pci: Consolidate pci address comparison APIs >> >> On 2/9/2015 4:31 PM, Tetsuya Mukawa wrote: >>> This patch replaces pci_addr_comparison() and memcmp() of pci >>> addresses by eal_compare_pci_addr(). >>> >>> v5: >>> - Fix pci_scan_one to handle pt_driver correctly. >>> v4: >>> - Fix calculation method of eal_compare_pci_addr(). >>> - Add parameter checking. >>> >>> Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp> >>> --- >>> lib/librte_eal/bsdapp/eal/eal_pci.c | 25 ++++++++--------------- >>> lib/librte_eal/common/eal_common_pci.c | 2 +- >>> lib/librte_eal/common/include/rte_pci.h | 34 +++++++++++++++++++++++++++++++ >>> lib/librte_eal/linuxapp/eal/eal_pci.c | 25 ++++++++--------------- >>> lib/librte_eal/linuxapp/eal/eal_pci_uio.c | 2 +- >>> 5 files changed, 54 insertions(+), 34 deletions(-) >>> >>> diff --git a/lib/librte_eal/bsdapp/eal/eal_pci.c >>> b/lib/librte_eal/bsdapp/eal/eal_pci.c >>> index 74ecce7..c844d58 100644 >>> --- a/lib/librte_eal/bsdapp/eal/eal_pci.c >>> +++ b/lib/librte_eal/bsdapp/eal/eal_pci.c >>> @@ -270,20 +270,6 @@ pci_uio_map_resource(struct rte_pci_device *dev) >>> return (0); >>> } >>> >>> -/* Compare two PCI device addresses. */ -static int >>> -pci_addr_comparison(struct rte_pci_addr *addr, struct rte_pci_addr >>> *addr2) -{ >>> - uint64_t dev_addr = (addr->domain << 24) + (addr->bus << 16) + (addr->devid << 8) + addr- >>> function; >>> - uint64_t dev_addr2 = (addr2->domain << 24) + (addr2->bus << 16) + (addr2->devid << 8) + >> addr2->function; >>> - >>> - if (dev_addr > dev_addr2) >>> - return 1; >>> - else >>> - return 0; >>> -} >>> - >>> - >>> /* Scan one pci sysfs entry, and fill the devices list from it. */ >>> static int pci_scan_one(int dev_pci_fd, struct pci_conf *conf) @@ >>> -356,13 +342,20 @@ pci_scan_one(int dev_pci_fd, struct pci_conf *conf) >>> } >>> else { >>> struct rte_pci_device *dev2 = NULL; >>> + int ret; >>> >>> TAILQ_FOREACH(dev2, &pci_device_list, next) { >>> - if (pci_addr_comparison(&dev->addr, &dev2->addr)) >>> + ret = eal_compare_pci_addr(&dev->addr, &dev2->addr); >>> + if (ret > 0) >>> continue; >>> - else { >>> + else if (ret < 0) { >>> TAILQ_INSERT_BEFORE(dev2, dev, next); >>> return 0; >>> + } else { /* already registered */ >>> + /* update pt_driver */ >>> + dev2->pt_driver = dev->pt_driver; >>> + free(dev); >>> + return 0; >>> } >>> } >>> TAILQ_INSERT_TAIL(&pci_device_list, dev, next); diff --git >>> a/lib/librte_eal/common/eal_common_pci.c >>> b/lib/librte_eal/common/eal_common_pci.c >>> index f3c7f71..a89f5c3 100644 >>> --- a/lib/librte_eal/common/eal_common_pci.c >>> +++ b/lib/librte_eal/common/eal_common_pci.c >>> @@ -93,7 +93,7 @@ static struct rte_devargs *pci_devargs_lookup(struct rte_pci_device *dev) >>> if (devargs->type != RTE_DEVTYPE_BLACKLISTED_PCI && >>> devargs->type != RTE_DEVTYPE_WHITELISTED_PCI) >>> continue; >>> - if (!memcmp(&dev->addr, &devargs->pci.addr, sizeof(dev->addr))) >>> + if (!eal_compare_pci_addr(&dev->addr, &devargs->pci.addr)) >>> return devargs; >>> } >>> return NULL; >>> diff --git a/lib/librte_eal/common/include/rte_pci.h >>> b/lib/librte_eal/common/include/rte_pci.h >>> index 7f2d699..4814cd7 100644 >>> --- a/lib/librte_eal/common/include/rte_pci.h >>> +++ b/lib/librte_eal/common/include/rte_pci.h >>> @@ -269,6 +269,40 @@ eal_parse_pci_DomBDF(const char *input, struct >>> rte_pci_addr *dev_addr) } #undef GET_PCIADDR_FIELD >>> >>> +/* Compare two PCI device addresses. */ >>> +/** >>> + * Utility function to compare two PCI device addresses. >>> + * >>> + * @param addr >>> + * The PCI Bus-Device-Function address to compare >>> + * @param addr2 >>> + * The PCI Bus-Device-Function address to compare >>> + * @return >>> + * 0 on equal PCI address. >>> + * Positive on addr is greater than addr2. >>> + * Negative on addr is less than addr2, or error. >>> + */ >>> +static inline int >>> +eal_compare_pci_addr(struct rte_pci_addr *addr, struct rte_pci_addr >>> +*addr2) { >>> + uint64_t dev_addr, dev_addr2; >>> + >>> + if ((addr == NULL) || (addr2 == NULL)) >>> + return -1; >>> + >>> + dev_addr = (addr->domain << 24) | (addr->bus << 16) | >>> + (addr->devid << 8) | addr->function; >>> + dev_addr2 = (addr2->domain << 24) | (addr2->bus << 16) | >>> + (addr2->devid << 8) | addr2->function; >>> + >>> + if (dev_addr > dev_addr2) >>> + return 1; >>> + else if (dev_addr < dev_addr2) >>> + return -1; >>> + else >>> + return 0; >>> +} >>> + >>> /** >>> * Probe the PCI bus for registered drivers. >>> * >>> diff --git a/lib/librte_eal/linuxapp/eal/eal_pci.c >>> b/lib/librte_eal/linuxapp/eal/eal_pci.c >>> index c0ca5a5..d847102 100644 >>> --- a/lib/librte_eal/linuxapp/eal/eal_pci.c >>> +++ b/lib/librte_eal/linuxapp/eal/eal_pci.c >>> @@ -229,20 +229,6 @@ error: >>> return -1; >>> } >>> >>> -/* Compare two PCI device addresses. */ -static int >>> -pci_addr_comparison(struct rte_pci_addr *addr, struct rte_pci_addr >>> *addr2) -{ >>> - uint64_t dev_addr = (addr->domain << 24) + (addr->bus << 16) + (addr->devid << 8) + addr- >>> function; >>> - uint64_t dev_addr2 = (addr2->domain << 24) + (addr2->bus << 16) + (addr2->devid << 8) + >> addr2->function; >>> - >>> - if (dev_addr > dev_addr2) >>> - return 1; >>> - else >>> - return 0; >>> -} >>> - >>> - >>> /* Scan one pci sysfs entry, and fill the devices list from it. */ >>> static int pci_scan_one(const char *dirname, uint16_t domain, uint8_t >>> bus, @@ -353,13 +339,20 @@ pci_scan_one(const char *dirname, uint16_t >>> domain, uint8_t bus, >>> } >>> else { >>> struct rte_pci_device *dev2 = NULL; >>> + int ret; >>> >>> TAILQ_FOREACH(dev2, &pci_device_list, next) { >>> - if (pci_addr_comparison(&dev->addr, &dev2->addr)) >>> + ret = eal_compare_pci_addr(&dev->addr, &dev2->addr); >>> + if (ret > 0) >>> continue; >>> - else { >>> + else if (ret < 0) { >>> TAILQ_INSERT_BEFORE(dev2, dev, next); >>> return 0; >>> + } else { /* already registered */ >>> + /* update pt_driver */ >>> + dev2->pt_driver = dev->pt_driver; > Hi Tetsuya, > > I am seeing a problem with the librte_pmd_ixgbe code where dev->max_vfs is being lost in some scenarios. > The following line should be added here: > dev2->max_vfs = dev->max_vfs; > > numa_mode should probably be updated too (although it is not causing a problem at present). > dev2->numa_mode = dev->numa_mode; I'm very curious, why those field miss? I haven't see any places clear this field. What is the root cause? Thanks, Michael > Regards, > > Bernard. > > > > >>> + free(dev); >>> + return 0; >>> } >>> } >>> TAILQ_INSERT_TAIL(&pci_device_list, dev, next); diff --git >>> a/lib/librte_eal/linuxapp/eal/eal_pci_uio.c >>> b/lib/librte_eal/linuxapp/eal/eal_pci_uio.c >>> index e53f06b..1da3507 100644 >>> --- a/lib/librte_eal/linuxapp/eal/eal_pci_uio.c >>> +++ b/lib/librte_eal/linuxapp/eal/eal_pci_uio.c >>> @@ -123,7 +123,7 @@ pci_uio_map_secondary(struct rte_pci_device *dev) >>> TAILQ_FOREACH(uio_res, pci_res_list, next) { >>> >>> /* skip this element if it doesn't match our PCI address */ >>> - if (memcmp(&uio_res->pci_addr, &dev->addr, sizeof(dev->addr))) >>> + if (eal_compare_pci_addr(&uio_res->pci_addr, &dev->addr)) >>> continue; >>> >>> for (i = 0; i != uio_res->nb_maps; i++) { >> Acked-by: Michael Qiu <michael.qiu@intel.com>
On 2015/02/11 12:27, Qiu, Michael wrote: > On 2/10/2015 11:11 PM, Iremonger, Bernard wrote: >>> -----Original Message----- >>> From: Qiu, Michael >>> Sent: Monday, February 9, 2015 1:10 PM >>> To: Tetsuya Mukawa; dev@dpdk.org >>> Cc: Iremonger, Bernard >>> Subject: Re: [PATCH v7 04/14] eal/pci: Consolidate pci address comparison APIs >>> >>> On 2/9/2015 4:31 PM, Tetsuya Mukawa wrote: >>>> This patch replaces pci_addr_comparison() and memcmp() of pci >>>> addresses by eal_compare_pci_addr(). >>>> >>>> v5: >>>> - Fix pci_scan_one to handle pt_driver correctly. >>>> v4: >>>> - Fix calculation method of eal_compare_pci_addr(). >>>> - Add parameter checking. >>>> >>>> Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp> >>>> --- >>>> lib/librte_eal/bsdapp/eal/eal_pci.c | 25 ++++++++--------------- >>>> lib/librte_eal/common/eal_common_pci.c | 2 +- >>>> lib/librte_eal/common/include/rte_pci.h | 34 +++++++++++++++++++++++++++++++ >>>> lib/librte_eal/linuxapp/eal/eal_pci.c | 25 ++++++++--------------- >>>> lib/librte_eal/linuxapp/eal/eal_pci_uio.c | 2 +- >>>> 5 files changed, 54 insertions(+), 34 deletions(-) >>>> >>>> diff --git a/lib/librte_eal/bsdapp/eal/eal_pci.c >>>> b/lib/librte_eal/bsdapp/eal/eal_pci.c >>>> index 74ecce7..c844d58 100644 >>>> --- a/lib/librte_eal/bsdapp/eal/eal_pci.c >>>> +++ b/lib/librte_eal/bsdapp/eal/eal_pci.c >>>> @@ -270,20 +270,6 @@ pci_uio_map_resource(struct rte_pci_device *dev) >>>> return (0); >>>> } >>>> >>>> -/* Compare two PCI device addresses. */ -static int >>>> -pci_addr_comparison(struct rte_pci_addr *addr, struct rte_pci_addr >>>> *addr2) -{ >>>> - uint64_t dev_addr = (addr->domain << 24) + (addr->bus << 16) + (addr->devid << 8) + addr- >>>> function; >>>> - uint64_t dev_addr2 = (addr2->domain << 24) + (addr2->bus << 16) + (addr2->devid << 8) + >>> addr2->function; >>>> - >>>> - if (dev_addr > dev_addr2) >>>> - return 1; >>>> - else >>>> - return 0; >>>> -} >>>> - >>>> - >>>> /* Scan one pci sysfs entry, and fill the devices list from it. */ >>>> static int pci_scan_one(int dev_pci_fd, struct pci_conf *conf) @@ >>>> -356,13 +342,20 @@ pci_scan_one(int dev_pci_fd, struct pci_conf *conf) >>>> } >>>> else { >>>> struct rte_pci_device *dev2 = NULL; >>>> + int ret; >>>> >>>> TAILQ_FOREACH(dev2, &pci_device_list, next) { >>>> - if (pci_addr_comparison(&dev->addr, &dev2->addr)) >>>> + ret = eal_compare_pci_addr(&dev->addr, &dev2->addr); >>>> + if (ret > 0) >>>> continue; >>>> - else { >>>> + else if (ret < 0) { >>>> TAILQ_INSERT_BEFORE(dev2, dev, next); >>>> return 0; >>>> + } else { /* already registered */ >>>> + /* update pt_driver */ >>>> + dev2->pt_driver = dev->pt_driver; >>>> + free(dev); >>>> + return 0; >>>> } >>>> } >>>> TAILQ_INSERT_TAIL(&pci_device_list, dev, next); diff --git >>>> a/lib/librte_eal/common/eal_common_pci.c >>>> b/lib/librte_eal/common/eal_common_pci.c >>>> index f3c7f71..a89f5c3 100644 >>>> --- a/lib/librte_eal/common/eal_common_pci.c >>>> +++ b/lib/librte_eal/common/eal_common_pci.c >>>> @@ -93,7 +93,7 @@ static struct rte_devargs *pci_devargs_lookup(struct rte_pci_device *dev) >>>> if (devargs->type != RTE_DEVTYPE_BLACKLISTED_PCI && >>>> devargs->type != RTE_DEVTYPE_WHITELISTED_PCI) >>>> continue; >>>> - if (!memcmp(&dev->addr, &devargs->pci.addr, sizeof(dev->addr))) >>>> + if (!eal_compare_pci_addr(&dev->addr, &devargs->pci.addr)) >>>> return devargs; >>>> } >>>> return NULL; >>>> diff --git a/lib/librte_eal/common/include/rte_pci.h >>>> b/lib/librte_eal/common/include/rte_pci.h >>>> index 7f2d699..4814cd7 100644 >>>> --- a/lib/librte_eal/common/include/rte_pci.h >>>> +++ b/lib/librte_eal/common/include/rte_pci.h >>>> @@ -269,6 +269,40 @@ eal_parse_pci_DomBDF(const char *input, struct >>>> rte_pci_addr *dev_addr) } #undef GET_PCIADDR_FIELD >>>> >>>> +/* Compare two PCI device addresses. */ >>>> +/** >>>> + * Utility function to compare two PCI device addresses. >>>> + * >>>> + * @param addr >>>> + * The PCI Bus-Device-Function address to compare >>>> + * @param addr2 >>>> + * The PCI Bus-Device-Function address to compare >>>> + * @return >>>> + * 0 on equal PCI address. >>>> + * Positive on addr is greater than addr2. >>>> + * Negative on addr is less than addr2, or error. >>>> + */ >>>> +static inline int >>>> +eal_compare_pci_addr(struct rte_pci_addr *addr, struct rte_pci_addr >>>> +*addr2) { >>>> + uint64_t dev_addr, dev_addr2; >>>> + >>>> + if ((addr == NULL) || (addr2 == NULL)) >>>> + return -1; >>>> + >>>> + dev_addr = (addr->domain << 24) | (addr->bus << 16) | >>>> + (addr->devid << 8) | addr->function; >>>> + dev_addr2 = (addr2->domain << 24) | (addr2->bus << 16) | >>>> + (addr2->devid << 8) | addr2->function; >>>> + >>>> + if (dev_addr > dev_addr2) >>>> + return 1; >>>> + else if (dev_addr < dev_addr2) >>>> + return -1; >>>> + else >>>> + return 0; >>>> +} >>>> + >>>> /** >>>> * Probe the PCI bus for registered drivers. >>>> * >>>> diff --git a/lib/librte_eal/linuxapp/eal/eal_pci.c >>>> b/lib/librte_eal/linuxapp/eal/eal_pci.c >>>> index c0ca5a5..d847102 100644 >>>> --- a/lib/librte_eal/linuxapp/eal/eal_pci.c >>>> +++ b/lib/librte_eal/linuxapp/eal/eal_pci.c >>>> @@ -229,20 +229,6 @@ error: >>>> return -1; >>>> } >>>> >>>> -/* Compare two PCI device addresses. */ -static int >>>> -pci_addr_comparison(struct rte_pci_addr *addr, struct rte_pci_addr >>>> *addr2) -{ >>>> - uint64_t dev_addr = (addr->domain << 24) + (addr->bus << 16) + (addr->devid << 8) + addr- >>>> function; >>>> - uint64_t dev_addr2 = (addr2->domain << 24) + (addr2->bus << 16) + (addr2->devid << 8) + >>> addr2->function; >>>> - >>>> - if (dev_addr > dev_addr2) >>>> - return 1; >>>> - else >>>> - return 0; >>>> -} >>>> - >>>> - >>>> /* Scan one pci sysfs entry, and fill the devices list from it. */ >>>> static int pci_scan_one(const char *dirname, uint16_t domain, uint8_t >>>> bus, @@ -353,13 +339,20 @@ pci_scan_one(const char *dirname, uint16_t >>>> domain, uint8_t bus, >>>> } >>>> else { >>>> struct rte_pci_device *dev2 = NULL; >>>> + int ret; >>>> >>>> TAILQ_FOREACH(dev2, &pci_device_list, next) { >>>> - if (pci_addr_comparison(&dev->addr, &dev2->addr)) >>>> + ret = eal_compare_pci_addr(&dev->addr, &dev2->addr); >>>> + if (ret > 0) >>>> continue; >>>> - else { >>>> + else if (ret < 0) { >>>> TAILQ_INSERT_BEFORE(dev2, dev, next); >>>> return 0; >>>> + } else { /* already registered */ >>>> + /* update pt_driver */ >>>> + dev2->pt_driver = dev->pt_driver; >> Hi Tetsuya, >> >> I am seeing a problem with the librte_pmd_ixgbe code where dev->max_vfs is being lost in some scenarios. >> The following line should be added here: >> dev2->max_vfs = dev->max_vfs; >> >> numa_mode should probably be updated too (although it is not causing a problem at present). >> dev2->numa_mode = dev->numa_mode; > I'm very curious, why those field miss? I haven't see any places clear > this field. > > What is the root cause? Hi Michael, Here is my guess. The above function creates pci device list. And current DPDK implementation assumes all devices needed to be managed are under igb_uio or vfio when above code is processed. To add hotplug function, we also need to think some devices will start to be managed under igb_uio or vfio after initializing pci device list. Anyway, I guess "max_vfs" value will be changed when igb_uio or vfio manages the device. Hi Bernard, Could you please check "max_vfs" and "num_node" values, then check the values again after the device is managed by igb_uio or vfio? In my environment, it seems max_vfs is created by igb_uio. But my NIC doesn't have VF, so behavior might be different in your environment. I guess "numa_node" should not be changed theoretically. If my guess is correct, how about replacing following values? - driver - max_vfs - resource - (numa_node) Except for above value, I guess other value shouldn't be changed even after the device is managed by igb_uio or vfio. Thanks, Tetsuya > Thanks, > Michael > >> Regards, >> >> Bernard. >> >> >> >> >>>> + free(dev); >>>> + return 0; >>>> } >>>> } >>>> TAILQ_INSERT_TAIL(&pci_device_list, dev, next); diff --git >>>> a/lib/librte_eal/linuxapp/eal/eal_pci_uio.c >>>> b/lib/librte_eal/linuxapp/eal/eal_pci_uio.c >>>> index e53f06b..1da3507 100644 >>>> --- a/lib/librte_eal/linuxapp/eal/eal_pci_uio.c >>>> +++ b/lib/librte_eal/linuxapp/eal/eal_pci_uio.c >>>> @@ -123,7 +123,7 @@ pci_uio_map_secondary(struct rte_pci_device *dev) >>>> TAILQ_FOREACH(uio_res, pci_res_list, next) { >>>> >>>> /* skip this element if it doesn't match our PCI address */ >>>> - if (memcmp(&uio_res->pci_addr, &dev->addr, sizeof(dev->addr))) >>>> + if (eal_compare_pci_addr(&uio_res->pci_addr, &dev->addr)) >>>> continue; >>>> >>>> for (i = 0; i != uio_res->nb_maps; i++) { >>> Acked-by: Michael Qiu <michael.qiu@intel.com>
On 2015/02/11 13:53, Tetsuya Mukawa wrote: > On 2015/02/11 12:27, Qiu, Michael wrote: >> On 2/10/2015 11:11 PM, Iremonger, Bernard wrote: >>>> -----Original Message----- >>>> From: Qiu, Michael >>>> Sent: Monday, February 9, 2015 1:10 PM >>>> To: Tetsuya Mukawa; dev@dpdk.org >>>> Cc: Iremonger, Bernard >>>> Subject: Re: [PATCH v7 04/14] eal/pci: Consolidate pci address comparison APIs >>>> >>>> On 2/9/2015 4:31 PM, Tetsuya Mukawa wrote: >>>>> This patch replaces pci_addr_comparison() and memcmp() of pci >>>>> addresses by eal_compare_pci_addr(). >>>>> >>>>> v5: >>>>> - Fix pci_scan_one to handle pt_driver correctly. >>>>> v4: >>>>> - Fix calculation method of eal_compare_pci_addr(). >>>>> - Add parameter checking. >>>>> >>>>> Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp> >>>>> --- >>>>> lib/librte_eal/bsdapp/eal/eal_pci.c | 25 ++++++++--------------- >>>>> lib/librte_eal/common/eal_common_pci.c | 2 +- >>>>> lib/librte_eal/common/include/rte_pci.h | 34 +++++++++++++++++++++++++++++++ >>>>> lib/librte_eal/linuxapp/eal/eal_pci.c | 25 ++++++++--------------- >>>>> lib/librte_eal/linuxapp/eal/eal_pci_uio.c | 2 +- >>>>> 5 files changed, 54 insertions(+), 34 deletions(-) >>>>> >>>>> diff --git a/lib/librte_eal/bsdapp/eal/eal_pci.c >>>>> b/lib/librte_eal/bsdapp/eal/eal_pci.c >>>>> index 74ecce7..c844d58 100644 >>>>> --- a/lib/librte_eal/bsdapp/eal/eal_pci.c >>>>> +++ b/lib/librte_eal/bsdapp/eal/eal_pci.c >>>>> @@ -270,20 +270,6 @@ pci_uio_map_resource(struct rte_pci_device *dev) >>>>> return (0); >>>>> } >>>>> >>>>> -/* Compare two PCI device addresses. */ -static int >>>>> -pci_addr_comparison(struct rte_pci_addr *addr, struct rte_pci_addr >>>>> *addr2) -{ >>>>> - uint64_t dev_addr = (addr->domain << 24) + (addr->bus << 16) + (addr->devid << 8) + addr- >>>>> function; >>>>> - uint64_t dev_addr2 = (addr2->domain << 24) + (addr2->bus << 16) + (addr2->devid << 8) + >>>> addr2->function; >>>>> - >>>>> - if (dev_addr > dev_addr2) >>>>> - return 1; >>>>> - else >>>>> - return 0; >>>>> -} >>>>> - >>>>> - >>>>> /* Scan one pci sysfs entry, and fill the devices list from it. */ >>>>> static int pci_scan_one(int dev_pci_fd, struct pci_conf *conf) @@ >>>>> -356,13 +342,20 @@ pci_scan_one(int dev_pci_fd, struct pci_conf *conf) >>>>> } >>>>> else { >>>>> struct rte_pci_device *dev2 = NULL; >>>>> + int ret; >>>>> >>>>> TAILQ_FOREACH(dev2, &pci_device_list, next) { >>>>> - if (pci_addr_comparison(&dev->addr, &dev2->addr)) >>>>> + ret = eal_compare_pci_addr(&dev->addr, &dev2->addr); >>>>> + if (ret > 0) >>>>> continue; >>>>> - else { >>>>> + else if (ret < 0) { >>>>> TAILQ_INSERT_BEFORE(dev2, dev, next); >>>>> return 0; >>>>> + } else { /* already registered */ >>>>> + /* update pt_driver */ >>>>> + dev2->pt_driver = dev->pt_driver; >>>>> + free(dev); >>>>> + return 0; >>>>> } >>>>> } >>>>> TAILQ_INSERT_TAIL(&pci_device_list, dev, next); diff --git >>>>> a/lib/librte_eal/common/eal_common_pci.c >>>>> b/lib/librte_eal/common/eal_common_pci.c >>>>> index f3c7f71..a89f5c3 100644 >>>>> --- a/lib/librte_eal/common/eal_common_pci.c >>>>> +++ b/lib/librte_eal/common/eal_common_pci.c >>>>> @@ -93,7 +93,7 @@ static struct rte_devargs *pci_devargs_lookup(struct rte_pci_device *dev) >>>>> if (devargs->type != RTE_DEVTYPE_BLACKLISTED_PCI && >>>>> devargs->type != RTE_DEVTYPE_WHITELISTED_PCI) >>>>> continue; >>>>> - if (!memcmp(&dev->addr, &devargs->pci.addr, sizeof(dev->addr))) >>>>> + if (!eal_compare_pci_addr(&dev->addr, &devargs->pci.addr)) >>>>> return devargs; >>>>> } >>>>> return NULL; >>>>> diff --git a/lib/librte_eal/common/include/rte_pci.h >>>>> b/lib/librte_eal/common/include/rte_pci.h >>>>> index 7f2d699..4814cd7 100644 >>>>> --- a/lib/librte_eal/common/include/rte_pci.h >>>>> +++ b/lib/librte_eal/common/include/rte_pci.h >>>>> @@ -269,6 +269,40 @@ eal_parse_pci_DomBDF(const char *input, struct >>>>> rte_pci_addr *dev_addr) } #undef GET_PCIADDR_FIELD >>>>> >>>>> +/* Compare two PCI device addresses. */ >>>>> +/** >>>>> + * Utility function to compare two PCI device addresses. >>>>> + * >>>>> + * @param addr >>>>> + * The PCI Bus-Device-Function address to compare >>>>> + * @param addr2 >>>>> + * The PCI Bus-Device-Function address to compare >>>>> + * @return >>>>> + * 0 on equal PCI address. >>>>> + * Positive on addr is greater than addr2. >>>>> + * Negative on addr is less than addr2, or error. >>>>> + */ >>>>> +static inline int >>>>> +eal_compare_pci_addr(struct rte_pci_addr *addr, struct rte_pci_addr >>>>> +*addr2) { >>>>> + uint64_t dev_addr, dev_addr2; >>>>> + >>>>> + if ((addr == NULL) || (addr2 == NULL)) >>>>> + return -1; >>>>> + >>>>> + dev_addr = (addr->domain << 24) | (addr->bus << 16) | >>>>> + (addr->devid << 8) | addr->function; >>>>> + dev_addr2 = (addr2->domain << 24) | (addr2->bus << 16) | >>>>> + (addr2->devid << 8) | addr2->function; >>>>> + >>>>> + if (dev_addr > dev_addr2) >>>>> + return 1; >>>>> + else if (dev_addr < dev_addr2) >>>>> + return -1; >>>>> + else >>>>> + return 0; >>>>> +} >>>>> + >>>>> /** >>>>> * Probe the PCI bus for registered drivers. >>>>> * >>>>> diff --git a/lib/librte_eal/linuxapp/eal/eal_pci.c >>>>> b/lib/librte_eal/linuxapp/eal/eal_pci.c >>>>> index c0ca5a5..d847102 100644 >>>>> --- a/lib/librte_eal/linuxapp/eal/eal_pci.c >>>>> +++ b/lib/librte_eal/linuxapp/eal/eal_pci.c >>>>> @@ -229,20 +229,6 @@ error: >>>>> return -1; >>>>> } >>>>> >>>>> -/* Compare two PCI device addresses. */ -static int >>>>> -pci_addr_comparison(struct rte_pci_addr *addr, struct rte_pci_addr >>>>> *addr2) -{ >>>>> - uint64_t dev_addr = (addr->domain << 24) + (addr->bus << 16) + (addr->devid << 8) + addr- >>>>> function; >>>>> - uint64_t dev_addr2 = (addr2->domain << 24) + (addr2->bus << 16) + (addr2->devid << 8) + >>>> addr2->function; >>>>> - >>>>> - if (dev_addr > dev_addr2) >>>>> - return 1; >>>>> - else >>>>> - return 0; >>>>> -} >>>>> - >>>>> - >>>>> /* Scan one pci sysfs entry, and fill the devices list from it. */ >>>>> static int pci_scan_one(const char *dirname, uint16_t domain, uint8_t >>>>> bus, @@ -353,13 +339,20 @@ pci_scan_one(const char *dirname, uint16_t >>>>> domain, uint8_t bus, >>>>> } >>>>> else { >>>>> struct rte_pci_device *dev2 = NULL; >>>>> + int ret; >>>>> >>>>> TAILQ_FOREACH(dev2, &pci_device_list, next) { >>>>> - if (pci_addr_comparison(&dev->addr, &dev2->addr)) >>>>> + ret = eal_compare_pci_addr(&dev->addr, &dev2->addr); >>>>> + if (ret > 0) >>>>> continue; >>>>> - else { >>>>> + else if (ret < 0) { >>>>> TAILQ_INSERT_BEFORE(dev2, dev, next); >>>>> return 0; >>>>> + } else { /* already registered */ >>>>> + /* update pt_driver */ >>>>> + dev2->pt_driver = dev->pt_driver; >>> Hi Tetsuya, >>> >>> I am seeing a problem with the librte_pmd_ixgbe code where dev->max_vfs is being lost in some scenarios. >>> The following line should be added here: >>> dev2->max_vfs = dev->max_vfs; >>> >>> numa_mode should probably be updated too (although it is not causing a problem at present). >>> dev2->numa_mode = dev->numa_mode; >> I'm very curious, why those field miss? I haven't see any places clear >> this field. >> >> What is the root cause? > Hi Michael, > > Here is my guess. > The above function creates pci device list. I am sorry. I forgot to add below information. "max_vfs" or "numa_node" value is came from sysfs when the above function is processed. > And current DPDK implementation assumes all devices needed to be managed > are under igb_uio or vfio when above code is processed. > To add hotplug function, we also need to think some devices will start > to be managed under igb_uio or vfio after initializing pci device list. > Anyway, I guess "max_vfs" value will be changed when igb_uio or vfio > manages the device. > > Hi Bernard, > > Could you please check "max_vfs" and "num_node" values, then check the > values again after the device is managed by igb_uio or vfio? > In my environment, it seems max_vfs is created by igb_uio. > But my NIC doesn't have VF, so behavior might be different in your > environment. > I guess "numa_node" should not be changed theoretically. > > If my guess is correct, how about replacing following values? > - driver > - max_vfs > - resource > - (numa_node) > Except for above value, I guess other value shouldn't be changed even > after the device is managed by igb_uio or vfio. > > Thanks, > Tetsuya > >> Thanks, >> Michael >> >>> Regards, >>> >>> Bernard. >>> >>> >>> >>> >>>>> + free(dev); >>>>> + return 0; >>>>> } >>>>> } >>>>> TAILQ_INSERT_TAIL(&pci_device_list, dev, next); diff --git >>>>> a/lib/librte_eal/linuxapp/eal/eal_pci_uio.c >>>>> b/lib/librte_eal/linuxapp/eal/eal_pci_uio.c >>>>> index e53f06b..1da3507 100644 >>>>> --- a/lib/librte_eal/linuxapp/eal/eal_pci_uio.c >>>>> +++ b/lib/librte_eal/linuxapp/eal/eal_pci_uio.c >>>>> @@ -123,7 +123,7 @@ pci_uio_map_secondary(struct rte_pci_device *dev) >>>>> TAILQ_FOREACH(uio_res, pci_res_list, next) { >>>>> >>>>> /* skip this element if it doesn't match our PCI address */ >>>>> - if (memcmp(&uio_res->pci_addr, &dev->addr, sizeof(dev->addr))) >>>>> + if (eal_compare_pci_addr(&uio_res->pci_addr, &dev->addr)) >>>>> continue; >>>>> >>>>> for (i = 0; i != uio_res->nb_maps; i++) { >>>> Acked-by: Michael Qiu <michael.qiu@intel.com> >
On 2/11/2015 12:57 PM, Tetsuya Mukawa wrote: > On 2015/02/11 13:53, Tetsuya Mukawa wrote: >> On 2015/02/11 12:27, Qiu, Michael wrote: >>> On 2/10/2015 11:11 PM, Iremonger, Bernard wrote: >>>>> -----Original Message----- >>>>> From: Qiu, Michael >>>>> Sent: Monday, February 9, 2015 1:10 PM >>>>> To: Tetsuya Mukawa; dev@dpdk.org >>>>> Cc: Iremonger, Bernard >>>>> Subject: Re: [PATCH v7 04/14] eal/pci: Consolidate pci address comparison APIs >>>>> >>>>> On 2/9/2015 4:31 PM, Tetsuya Mukawa wrote: >>>>>> This patch replaces pci_addr_comparison() and memcmp() of pci >>>>>> addresses by eal_compare_pci_addr(). >>>>>> >>>>>> v5: >>>>>> - Fix pci_scan_one to handle pt_driver correctly. >>>>>> v4: >>>>>> - Fix calculation method of eal_compare_pci_addr(). >>>>>> - Add parameter checking. >>>>>> >>>>>> Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp> >>>>>> --- >>>>>> lib/librte_eal/bsdapp/eal/eal_pci.c | 25 ++++++++--------------- >>>>>> lib/librte_eal/common/eal_common_pci.c | 2 +- >>>>>> lib/librte_eal/common/include/rte_pci.h | 34 +++++++++++++++++++++++++++++++ >>>>>> lib/librte_eal/linuxapp/eal/eal_pci.c | 25 ++++++++--------------- >>>>>> lib/librte_eal/linuxapp/eal/eal_pci_uio.c | 2 +- >>>>>> 5 files changed, 54 insertions(+), 34 deletions(-) >>>>>> >>>>>> diff --git a/lib/librte_eal/bsdapp/eal/eal_pci.c >>>>>> b/lib/librte_eal/bsdapp/eal/eal_pci.c >>>>>> index 74ecce7..c844d58 100644 >>>>>> --- a/lib/librte_eal/bsdapp/eal/eal_pci.c >>>>>> +++ b/lib/librte_eal/bsdapp/eal/eal_pci.c >>>>>> @@ -270,20 +270,6 @@ pci_uio_map_resource(struct rte_pci_device *dev) >>>>>> return (0); >>>>>> } >>>>>> >>>>>> -/* Compare two PCI device addresses. */ -static int >>>>>> -pci_addr_comparison(struct rte_pci_addr *addr, struct rte_pci_addr >>>>>> *addr2) -{ >>>>>> - uint64_t dev_addr = (addr->domain << 24) + (addr->bus << 16) + (addr->devid << 8) + addr- >>>>>> function; >>>>>> - uint64_t dev_addr2 = (addr2->domain << 24) + (addr2->bus << 16) + (addr2->devid << 8) + >>>>> addr2->function; >>>>>> - >>>>>> - if (dev_addr > dev_addr2) >>>>>> - return 1; >>>>>> - else >>>>>> - return 0; >>>>>> -} >>>>>> - >>>>>> - >>>>>> /* Scan one pci sysfs entry, and fill the devices list from it. */ >>>>>> static int pci_scan_one(int dev_pci_fd, struct pci_conf *conf) @@ >>>>>> -356,13 +342,20 @@ pci_scan_one(int dev_pci_fd, struct pci_conf *conf) >>>>>> } >>>>>> else { >>>>>> struct rte_pci_device *dev2 = NULL; >>>>>> + int ret; >>>>>> >>>>>> TAILQ_FOREACH(dev2, &pci_device_list, next) { >>>>>> - if (pci_addr_comparison(&dev->addr, &dev2->addr)) >>>>>> + ret = eal_compare_pci_addr(&dev->addr, &dev2->addr); >>>>>> + if (ret > 0) >>>>>> continue; >>>>>> - else { >>>>>> + else if (ret < 0) { >>>>>> TAILQ_INSERT_BEFORE(dev2, dev, next); >>>>>> return 0; >>>>>> + } else { /* already registered */ >>>>>> + /* update pt_driver */ >>>>>> + dev2->pt_driver = dev->pt_driver; >>>>>> + free(dev); >>>>>> + return 0; >>>>>> } >>>>>> } >>>>>> TAILQ_INSERT_TAIL(&pci_device_list, dev, next); diff --git >>>>>> a/lib/librte_eal/common/eal_common_pci.c >>>>>> b/lib/librte_eal/common/eal_common_pci.c >>>>>> index f3c7f71..a89f5c3 100644 >>>>>> --- a/lib/librte_eal/common/eal_common_pci.c >>>>>> +++ b/lib/librte_eal/common/eal_common_pci.c >>>>>> @@ -93,7 +93,7 @@ static struct rte_devargs *pci_devargs_lookup(struct rte_pci_device *dev) >>>>>> if (devargs->type != RTE_DEVTYPE_BLACKLISTED_PCI && >>>>>> devargs->type != RTE_DEVTYPE_WHITELISTED_PCI) >>>>>> continue; >>>>>> - if (!memcmp(&dev->addr, &devargs->pci.addr, sizeof(dev->addr))) >>>>>> + if (!eal_compare_pci_addr(&dev->addr, &devargs->pci.addr)) >>>>>> return devargs; >>>>>> } >>>>>> return NULL; >>>>>> diff --git a/lib/librte_eal/common/include/rte_pci.h >>>>>> b/lib/librte_eal/common/include/rte_pci.h >>>>>> index 7f2d699..4814cd7 100644 >>>>>> --- a/lib/librte_eal/common/include/rte_pci.h >>>>>> +++ b/lib/librte_eal/common/include/rte_pci.h >>>>>> @@ -269,6 +269,40 @@ eal_parse_pci_DomBDF(const char *input, struct >>>>>> rte_pci_addr *dev_addr) } #undef GET_PCIADDR_FIELD >>>>>> >>>>>> +/* Compare two PCI device addresses. */ >>>>>> +/** >>>>>> + * Utility function to compare two PCI device addresses. >>>>>> + * >>>>>> + * @param addr >>>>>> + * The PCI Bus-Device-Function address to compare >>>>>> + * @param addr2 >>>>>> + * The PCI Bus-Device-Function address to compare >>>>>> + * @return >>>>>> + * 0 on equal PCI address. >>>>>> + * Positive on addr is greater than addr2. >>>>>> + * Negative on addr is less than addr2, or error. >>>>>> + */ >>>>>> +static inline int >>>>>> +eal_compare_pci_addr(struct rte_pci_addr *addr, struct rte_pci_addr >>>>>> +*addr2) { >>>>>> + uint64_t dev_addr, dev_addr2; >>>>>> + >>>>>> + if ((addr == NULL) || (addr2 == NULL)) >>>>>> + return -1; >>>>>> + >>>>>> + dev_addr = (addr->domain << 24) | (addr->bus << 16) | >>>>>> + (addr->devid << 8) | addr->function; >>>>>> + dev_addr2 = (addr2->domain << 24) | (addr2->bus << 16) | >>>>>> + (addr2->devid << 8) | addr2->function; >>>>>> + >>>>>> + if (dev_addr > dev_addr2) >>>>>> + return 1; >>>>>> + else if (dev_addr < dev_addr2) >>>>>> + return -1; >>>>>> + else >>>>>> + return 0; >>>>>> +} >>>>>> + >>>>>> /** >>>>>> * Probe the PCI bus for registered drivers. >>>>>> * >>>>>> diff --git a/lib/librte_eal/linuxapp/eal/eal_pci.c >>>>>> b/lib/librte_eal/linuxapp/eal/eal_pci.c >>>>>> index c0ca5a5..d847102 100644 >>>>>> --- a/lib/librte_eal/linuxapp/eal/eal_pci.c >>>>>> +++ b/lib/librte_eal/linuxapp/eal/eal_pci.c >>>>>> @@ -229,20 +229,6 @@ error: >>>>>> return -1; >>>>>> } >>>>>> >>>>>> -/* Compare two PCI device addresses. */ -static int >>>>>> -pci_addr_comparison(struct rte_pci_addr *addr, struct rte_pci_addr >>>>>> *addr2) -{ >>>>>> - uint64_t dev_addr = (addr->domain << 24) + (addr->bus << 16) + (addr->devid << 8) + addr- >>>>>> function; >>>>>> - uint64_t dev_addr2 = (addr2->domain << 24) + (addr2->bus << 16) + (addr2->devid << 8) + >>>>> addr2->function; >>>>>> - >>>>>> - if (dev_addr > dev_addr2) >>>>>> - return 1; >>>>>> - else >>>>>> - return 0; >>>>>> -} >>>>>> - >>>>>> - >>>>>> /* Scan one pci sysfs entry, and fill the devices list from it. */ >>>>>> static int pci_scan_one(const char *dirname, uint16_t domain, uint8_t >>>>>> bus, @@ -353,13 +339,20 @@ pci_scan_one(const char *dirname, uint16_t >>>>>> domain, uint8_t bus, >>>>>> } >>>>>> else { >>>>>> struct rte_pci_device *dev2 = NULL; >>>>>> + int ret; >>>>>> >>>>>> TAILQ_FOREACH(dev2, &pci_device_list, next) { >>>>>> - if (pci_addr_comparison(&dev->addr, &dev2->addr)) >>>>>> + ret = eal_compare_pci_addr(&dev->addr, &dev2->addr); >>>>>> + if (ret > 0) >>>>>> continue; >>>>>> - else { >>>>>> + else if (ret < 0) { >>>>>> TAILQ_INSERT_BEFORE(dev2, dev, next); >>>>>> return 0; >>>>>> + } else { /* already registered */ >>>>>> + /* update pt_driver */ >>>>>> + dev2->pt_driver = dev->pt_driver; >>>> Hi Tetsuya, >>>> >>>> I am seeing a problem with the librte_pmd_ixgbe code where dev->max_vfs is being lost in some scenarios. >>>> The following line should be added here: >>>> dev2->max_vfs = dev->max_vfs; >>>> >>>> numa_mode should probably be updated too (although it is not causing a problem at present). >>>> dev2->numa_mode = dev->numa_mode; >>> I'm very curious, why those field miss? I haven't see any places clear >>> this field. >>> >>> What is the root cause? >> Hi Michael, >> >> Here is my guess. >> The above function creates pci device list. > I am sorry. I forgot to add below information. > > "max_vfs" or "numa_node" value is came from sysfs when the above > function is processed. Yes, but it has already been registered, why it missed? Thanks, Michael >> And current DPDK implementation assumes all devices needed to be managed >> are under igb_uio or vfio when above code is processed. >> To add hotplug function, we also need to think some devices will start >> to be managed under igb_uio or vfio after initializing pci device list. >> Anyway, I guess "max_vfs" value will be changed when igb_uio or vfio >> manages the device. >> >> Hi Bernard, >> >> Could you please check "max_vfs" and "num_node" values, then check the >> values again after the device is managed by igb_uio or vfio? >> In my environment, it seems max_vfs is created by igb_uio. >> But my NIC doesn't have VF, so behavior might be different in your >> environment. >> I guess "numa_node" should not be changed theoretically. >> >> If my guess is correct, how about replacing following values? >> - driver >> - max_vfs >> - resource >> - (numa_node) >> Except for above value, I guess other value shouldn't be changed even >> after the device is managed by igb_uio or vfio. >> >> Thanks, >> Tetsuya >> >>> Thanks, >>> Michael >>> >>>> Regards, >>>> >>>> Bernard. >>>> >>>> >>>> >>>> >>>>>> + free(dev); >>>>>> + return 0; >>>>>> } >>>>>> } >>>>>> TAILQ_INSERT_TAIL(&pci_device_list, dev, next); diff --git >>>>>> a/lib/librte_eal/linuxapp/eal/eal_pci_uio.c >>>>>> b/lib/librte_eal/linuxapp/eal/eal_pci_uio.c >>>>>> index e53f06b..1da3507 100644 >>>>>> --- a/lib/librte_eal/linuxapp/eal/eal_pci_uio.c >>>>>> +++ b/lib/librte_eal/linuxapp/eal/eal_pci_uio.c >>>>>> @@ -123,7 +123,7 @@ pci_uio_map_secondary(struct rte_pci_device *dev) >>>>>> TAILQ_FOREACH(uio_res, pci_res_list, next) { >>>>>> >>>>>> /* skip this element if it doesn't match our PCI address */ >>>>>> - if (memcmp(&uio_res->pci_addr, &dev->addr, sizeof(dev->addr))) >>>>>> + if (eal_compare_pci_addr(&uio_res->pci_addr, &dev->addr)) >>>>>> continue; >>>>>> >>>>>> for (i = 0; i != uio_res->nb_maps; i++) { >>>>> Acked-by: Michael Qiu <michael.qiu@intel.com> >
On 2015/02/11 15:29, Qiu, Michael wrote: > On 2/11/2015 12:57 PM, Tetsuya Mukawa wrote: >> On 2015/02/11 13:53, Tetsuya Mukawa wrote: >>> On 2015/02/11 12:27, Qiu, Michael wrote: >>>> On 2/10/2015 11:11 PM, Iremonger, Bernard wrote: >>>>>> -----Original Message----- >>>>>> From: Qiu, Michael >>>>>> Sent: Monday, February 9, 2015 1:10 PM >>>>>> To: Tetsuya Mukawa; dev@dpdk.org >>>>>> Cc: Iremonger, Bernard >>>>>> Subject: Re: [PATCH v7 04/14] eal/pci: Consolidate pci address comparison APIs >>>>>> >>>>>> On 2/9/2015 4:31 PM, Tetsuya Mukawa wrote: >>>>>>> This patch replaces pci_addr_comparison() and memcmp() of pci >>>>>>> addresses by eal_compare_pci_addr(). >>>>>>> >>>>>>> v5: >>>>>>> - Fix pci_scan_one to handle pt_driver correctly. >>>>>>> v4: >>>>>>> - Fix calculation method of eal_compare_pci_addr(). >>>>>>> - Add parameter checking. >>>>>>> >>>>>>> Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp> >>>>>>> --- >>>>>>> lib/librte_eal/bsdapp/eal/eal_pci.c | 25 ++++++++--------------- >>>>>>> lib/librte_eal/common/eal_common_pci.c | 2 +- >>>>>>> lib/librte_eal/common/include/rte_pci.h | 34 +++++++++++++++++++++++++++++++ >>>>>>> lib/librte_eal/linuxapp/eal/eal_pci.c | 25 ++++++++--------------- >>>>>>> lib/librte_eal/linuxapp/eal/eal_pci_uio.c | 2 +- >>>>>>> 5 files changed, 54 insertions(+), 34 deletions(-) >>>>>>> >>>>>>> diff --git a/lib/librte_eal/bsdapp/eal/eal_pci.c >>>>>>> b/lib/librte_eal/bsdapp/eal/eal_pci.c >>>>>>> index 74ecce7..c844d58 100644 >>>>>>> --- a/lib/librte_eal/bsdapp/eal/eal_pci.c >>>>>>> +++ b/lib/librte_eal/bsdapp/eal/eal_pci.c >>>>>>> @@ -270,20 +270,6 @@ pci_uio_map_resource(struct rte_pci_device *dev) >>>>>>> return (0); >>>>>>> } >>>>>>> >>>>>>> -/* Compare two PCI device addresses. */ -static int >>>>>>> -pci_addr_comparison(struct rte_pci_addr *addr, struct rte_pci_addr >>>>>>> *addr2) -{ >>>>>>> - uint64_t dev_addr = (addr->domain << 24) + (addr->bus << 16) + (addr->devid << 8) + addr- >>>>>>> function; >>>>>>> - uint64_t dev_addr2 = (addr2->domain << 24) + (addr2->bus << 16) + (addr2->devid << 8) + >>>>>> addr2->function; >>>>>>> - >>>>>>> - if (dev_addr > dev_addr2) >>>>>>> - return 1; >>>>>>> - else >>>>>>> - return 0; >>>>>>> -} >>>>>>> - >>>>>>> - >>>>>>> /* Scan one pci sysfs entry, and fill the devices list from it. */ >>>>>>> static int pci_scan_one(int dev_pci_fd, struct pci_conf *conf) @@ >>>>>>> -356,13 +342,20 @@ pci_scan_one(int dev_pci_fd, struct pci_conf *conf) >>>>>>> } >>>>>>> else { >>>>>>> struct rte_pci_device *dev2 = NULL; >>>>>>> + int ret; >>>>>>> >>>>>>> TAILQ_FOREACH(dev2, &pci_device_list, next) { >>>>>>> - if (pci_addr_comparison(&dev->addr, &dev2->addr)) >>>>>>> + ret = eal_compare_pci_addr(&dev->addr, &dev2->addr); >>>>>>> + if (ret > 0) >>>>>>> continue; >>>>>>> - else { >>>>>>> + else if (ret < 0) { >>>>>>> TAILQ_INSERT_BEFORE(dev2, dev, next); >>>>>>> return 0; >>>>>>> + } else { /* already registered */ >>>>>>> + /* update pt_driver */ >>>>>>> + dev2->pt_driver = dev->pt_driver; >>>>>>> + free(dev); >>>>>>> + return 0; >>>>>>> } >>>>>>> } >>>>>>> TAILQ_INSERT_TAIL(&pci_device_list, dev, next); diff --git >>>>>>> a/lib/librte_eal/common/eal_common_pci.c >>>>>>> b/lib/librte_eal/common/eal_common_pci.c >>>>>>> index f3c7f71..a89f5c3 100644 >>>>>>> --- a/lib/librte_eal/common/eal_common_pci.c >>>>>>> +++ b/lib/librte_eal/common/eal_common_pci.c >>>>>>> @@ -93,7 +93,7 @@ static struct rte_devargs *pci_devargs_lookup(struct rte_pci_device *dev) >>>>>>> if (devargs->type != RTE_DEVTYPE_BLACKLISTED_PCI && >>>>>>> devargs->type != RTE_DEVTYPE_WHITELISTED_PCI) >>>>>>> continue; >>>>>>> - if (!memcmp(&dev->addr, &devargs->pci.addr, sizeof(dev->addr))) >>>>>>> + if (!eal_compare_pci_addr(&dev->addr, &devargs->pci.addr)) >>>>>>> return devargs; >>>>>>> } >>>>>>> return NULL; >>>>>>> diff --git a/lib/librte_eal/common/include/rte_pci.h >>>>>>> b/lib/librte_eal/common/include/rte_pci.h >>>>>>> index 7f2d699..4814cd7 100644 >>>>>>> --- a/lib/librte_eal/common/include/rte_pci.h >>>>>>> +++ b/lib/librte_eal/common/include/rte_pci.h >>>>>>> @@ -269,6 +269,40 @@ eal_parse_pci_DomBDF(const char *input, struct >>>>>>> rte_pci_addr *dev_addr) } #undef GET_PCIADDR_FIELD >>>>>>> >>>>>>> +/* Compare two PCI device addresses. */ >>>>>>> +/** >>>>>>> + * Utility function to compare two PCI device addresses. >>>>>>> + * >>>>>>> + * @param addr >>>>>>> + * The PCI Bus-Device-Function address to compare >>>>>>> + * @param addr2 >>>>>>> + * The PCI Bus-Device-Function address to compare >>>>>>> + * @return >>>>>>> + * 0 on equal PCI address. >>>>>>> + * Positive on addr is greater than addr2. >>>>>>> + * Negative on addr is less than addr2, or error. >>>>>>> + */ >>>>>>> +static inline int >>>>>>> +eal_compare_pci_addr(struct rte_pci_addr *addr, struct rte_pci_addr >>>>>>> +*addr2) { >>>>>>> + uint64_t dev_addr, dev_addr2; >>>>>>> + >>>>>>> + if ((addr == NULL) || (addr2 == NULL)) >>>>>>> + return -1; >>>>>>> + >>>>>>> + dev_addr = (addr->domain << 24) | (addr->bus << 16) | >>>>>>> + (addr->devid << 8) | addr->function; >>>>>>> + dev_addr2 = (addr2->domain << 24) | (addr2->bus << 16) | >>>>>>> + (addr2->devid << 8) | addr2->function; >>>>>>> + >>>>>>> + if (dev_addr > dev_addr2) >>>>>>> + return 1; >>>>>>> + else if (dev_addr < dev_addr2) >>>>>>> + return -1; >>>>>>> + else >>>>>>> + return 0; >>>>>>> +} >>>>>>> + >>>>>>> /** >>>>>>> * Probe the PCI bus for registered drivers. >>>>>>> * >>>>>>> diff --git a/lib/librte_eal/linuxapp/eal/eal_pci.c >>>>>>> b/lib/librte_eal/linuxapp/eal/eal_pci.c >>>>>>> index c0ca5a5..d847102 100644 >>>>>>> --- a/lib/librte_eal/linuxapp/eal/eal_pci.c >>>>>>> +++ b/lib/librte_eal/linuxapp/eal/eal_pci.c >>>>>>> @@ -229,20 +229,6 @@ error: >>>>>>> return -1; >>>>>>> } >>>>>>> >>>>>>> -/* Compare two PCI device addresses. */ -static int >>>>>>> -pci_addr_comparison(struct rte_pci_addr *addr, struct rte_pci_addr >>>>>>> *addr2) -{ >>>>>>> - uint64_t dev_addr = (addr->domain << 24) + (addr->bus << 16) + (addr->devid << 8) + addr- >>>>>>> function; >>>>>>> - uint64_t dev_addr2 = (addr2->domain << 24) + (addr2->bus << 16) + (addr2->devid << 8) + >>>>>> addr2->function; >>>>>>> - >>>>>>> - if (dev_addr > dev_addr2) >>>>>>> - return 1; >>>>>>> - else >>>>>>> - return 0; >>>>>>> -} >>>>>>> - >>>>>>> - >>>>>>> /* Scan one pci sysfs entry, and fill the devices list from it. */ >>>>>>> static int pci_scan_one(const char *dirname, uint16_t domain, uint8_t >>>>>>> bus, @@ -353,13 +339,20 @@ pci_scan_one(const char *dirname, uint16_t >>>>>>> domain, uint8_t bus, >>>>>>> } >>>>>>> else { >>>>>>> struct rte_pci_device *dev2 = NULL; >>>>>>> + int ret; >>>>>>> >>>>>>> TAILQ_FOREACH(dev2, &pci_device_list, next) { >>>>>>> - if (pci_addr_comparison(&dev->addr, &dev2->addr)) >>>>>>> + ret = eal_compare_pci_addr(&dev->addr, &dev2->addr); >>>>>>> + if (ret > 0) >>>>>>> continue; >>>>>>> - else { >>>>>>> + else if (ret < 0) { >>>>>>> TAILQ_INSERT_BEFORE(dev2, dev, next); >>>>>>> return 0; >>>>>>> + } else { /* already registered */ >>>>>>> + /* update pt_driver */ >>>>>>> + dev2->pt_driver = dev->pt_driver; >>>>> Hi Tetsuya, >>>>> >>>>> I am seeing a problem with the librte_pmd_ixgbe code where dev->max_vfs is being lost in some scenarios. >>>>> The following line should be added here: >>>>> dev2->max_vfs = dev->max_vfs; >>>>> >>>>> numa_mode should probably be updated too (although it is not causing a problem at present). >>>>> dev2->numa_mode = dev->numa_mode; >>>> I'm very curious, why those field miss? I haven't see any places clear >>>> this field. >>>> >>>> What is the root cause? >>> Hi Michael, >>> >>> Here is my guess. >>> The above function creates pci device list. >> I am sorry. I forgot to add below information. >> >> "max_vfs" or "numa_node" value is came from sysfs when the above >> function is processed. > Yes, but it has already been registered, why it missed? Yes, it has been registered already, but probably should be updated. I guess sysfs value will be changed when igb_uio starts managing the device. ex) 1. Boot linux 2. start a dpdk application with no port. 3. pci device list is registered. - Here, "max_vfs" is came from sysfs. Or there is no such a entry. 4. igb_uio binds the device. 5. I guess max_vfs value of sysfs is changed. Or max_vfs entry is created. 6. The dpdk application calls hotplug function. - Here, I guess we need to update "max_vfs" value. Above is a just my assumption. It may be good to wait for Bernard's reply. Thanks, Tetsuya > Thanks, > Michael >>> And current DPDK implementation assumes all devices needed to be managed >>> are under igb_uio or vfio when above code is processed. >>> To add hotplug function, we also need to think some devices will start >>> to be managed under igb_uio or vfio after initializing pci device list. >>> Anyway, I guess "max_vfs" value will be changed when igb_uio or vfio >>> manages the device. >>> >>> Hi Bernard, >>> >>> Could you please check "max_vfs" and "num_node" values, then check the >>> values again after the device is managed by igb_uio or vfio? >>> In my environment, it seems max_vfs is created by igb_uio. >>> But my NIC doesn't have VF, so behavior might be different in your >>> environment. >>> I guess "numa_node" should not be changed theoretically. >>> >>> If my guess is correct, how about replacing following values? >>> - driver >>> - max_vfs >>> - resource >>> - (numa_node) >>> Except for above value, I guess other value shouldn't be changed even >>> after the device is managed by igb_uio or vfio. >>> >>> Thanks, >>> Tetsuya >>> >>>> Thanks, >>>> Michael >>>> >>>>> Regards, >>>>> >>>>> Bernard. >>>>> >>>>> >>>>> >>>>> >>>>>>> + free(dev); >>>>>>> + return 0; >>>>>>> } >>>>>>> } >>>>>>> TAILQ_INSERT_TAIL(&pci_device_list, dev, next); diff --git >>>>>>> a/lib/librte_eal/linuxapp/eal/eal_pci_uio.c >>>>>>> b/lib/librte_eal/linuxapp/eal/eal_pci_uio.c >>>>>>> index e53f06b..1da3507 100644 >>>>>>> --- a/lib/librte_eal/linuxapp/eal/eal_pci_uio.c >>>>>>> +++ b/lib/librte_eal/linuxapp/eal/eal_pci_uio.c >>>>>>> @@ -123,7 +123,7 @@ pci_uio_map_secondary(struct rte_pci_device *dev) >>>>>>> TAILQ_FOREACH(uio_res, pci_res_list, next) { >>>>>>> >>>>>>> /* skip this element if it doesn't match our PCI address */ >>>>>>> - if (memcmp(&uio_res->pci_addr, &dev->addr, sizeof(dev->addr))) >>>>>>> + if (eal_compare_pci_addr(&uio_res->pci_addr, &dev->addr)) >>>>>>> continue; >>>>>>> >>>>>>> for (i = 0; i != uio_res->nb_maps; i++) { >>>>>> Acked-by: Michael Qiu <michael.qiu@intel.com>
On 2/11/2015 4:14 PM, Tetsuya Mukawa wrote: > On 2015/02/11 15:29, Qiu, Michael wrote: >> On 2/11/2015 12:57 PM, Tetsuya Mukawa wrote: >>> On 2015/02/11 13:53, Tetsuya Mukawa wrote: >>>> On 2015/02/11 12:27, Qiu, Michael wrote: >>>>> On 2/10/2015 11:11 PM, Iremonger, Bernard wrote: >>>>>>> -----Original Message----- >>>>>>> From: Qiu, Michael >>>>>>> Sent: Monday, February 9, 2015 1:10 PM >>>>>>> To: Tetsuya Mukawa; dev@dpdk.org >>>>>>> Cc: Iremonger, Bernard >>>>>>> Subject: Re: [PATCH v7 04/14] eal/pci: Consolidate pci address comparison APIs >>>>>>> >>>>>>> On 2/9/2015 4:31 PM, Tetsuya Mukawa wrote: >>>>>>>> This patch replaces pci_addr_comparison() and memcmp() of pci >>>>>>>> addresses by eal_compare_pci_addr(). >>>>>>>> >>>>>>>> v5: >>>>>>>> - Fix pci_scan_one to handle pt_driver correctly. >>>>>>>> v4: >>>>>>>> - Fix calculation method of eal_compare_pci_addr(). >>>>>>>> - Add parameter checking. >>>>>>>> >>>>>>>> Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp> >>>>>>>> --- >>>>>>>> lib/librte_eal/bsdapp/eal/eal_pci.c | 25 ++++++++--------------- >>>>>>>> lib/librte_eal/common/eal_common_pci.c | 2 +- >>>>>>>> lib/librte_eal/common/include/rte_pci.h | 34 +++++++++++++++++++++++++++++++ >>>>>>>> lib/librte_eal/linuxapp/eal/eal_pci.c | 25 ++++++++--------------- >>>>>>>> lib/librte_eal/linuxapp/eal/eal_pci_uio.c | 2 +- >>>>>>>> 5 files changed, 54 insertions(+), 34 deletions(-) >>>>>>>> >>>>>>>> diff --git a/lib/librte_eal/bsdapp/eal/eal_pci.c >>>>>>>> b/lib/librte_eal/bsdapp/eal/eal_pci.c >>>>>>>> index 74ecce7..c844d58 100644 >>>>>>>> --- a/lib/librte_eal/bsdapp/eal/eal_pci.c >>>>>>>> +++ b/lib/librte_eal/bsdapp/eal/eal_pci.c >>>>>>>> @@ -270,20 +270,6 @@ pci_uio_map_resource(struct rte_pci_device *dev) >>>>>>>> return (0); >>>>>>>> } >>>>>>>> >>>>>>>> -/* Compare two PCI device addresses. */ -static int >>>>>>>> -pci_addr_comparison(struct rte_pci_addr *addr, struct rte_pci_addr >>>>>>>> *addr2) -{ >>>>>>>> - uint64_t dev_addr = (addr->domain << 24) + (addr->bus << 16) + (addr->devid << 8) + addr- >>>>>>>> function; >>>>>>>> - uint64_t dev_addr2 = (addr2->domain << 24) + (addr2->bus << 16) + (addr2->devid << 8) + >>>>>>> addr2->function; >>>>>>>> - >>>>>>>> - if (dev_addr > dev_addr2) >>>>>>>> - return 1; >>>>>>>> - else >>>>>>>> - return 0; >>>>>>>> -} >>>>>>>> - >>>>>>>> - >>>>>>>> /* Scan one pci sysfs entry, and fill the devices list from it. */ >>>>>>>> static int pci_scan_one(int dev_pci_fd, struct pci_conf *conf) @@ >>>>>>>> -356,13 +342,20 @@ pci_scan_one(int dev_pci_fd, struct pci_conf *conf) >>>>>>>> } >>>>>>>> else { >>>>>>>> struct rte_pci_device *dev2 = NULL; >>>>>>>> + int ret; >>>>>>>> >>>>>>>> TAILQ_FOREACH(dev2, &pci_device_list, next) { >>>>>>>> - if (pci_addr_comparison(&dev->addr, &dev2->addr)) >>>>>>>> + ret = eal_compare_pci_addr(&dev->addr, &dev2->addr); >>>>>>>> + if (ret > 0) >>>>>>>> continue; >>>>>>>> - else { >>>>>>>> + else if (ret < 0) { >>>>>>>> TAILQ_INSERT_BEFORE(dev2, dev, next); >>>>>>>> return 0; >>>>>>>> + } else { /* already registered */ >>>>>>>> + /* update pt_driver */ >>>>>>>> + dev2->pt_driver = dev->pt_driver; >>>>>>>> + free(dev); >>>>>>>> + return 0; >>>>>>>> } >>>>>>>> } >>>>>>>> TAILQ_INSERT_TAIL(&pci_device_list, dev, next); diff --git >>>>>>>> a/lib/librte_eal/common/eal_common_pci.c >>>>>>>> b/lib/librte_eal/common/eal_common_pci.c >>>>>>>> index f3c7f71..a89f5c3 100644 >>>>>>>> --- a/lib/librte_eal/common/eal_common_pci.c >>>>>>>> +++ b/lib/librte_eal/common/eal_common_pci.c >>>>>>>> @@ -93,7 +93,7 @@ static struct rte_devargs *pci_devargs_lookup(struct rte_pci_device *dev) >>>>>>>> if (devargs->type != RTE_DEVTYPE_BLACKLISTED_PCI && >>>>>>>> devargs->type != RTE_DEVTYPE_WHITELISTED_PCI) >>>>>>>> continue; >>>>>>>> - if (!memcmp(&dev->addr, &devargs->pci.addr, sizeof(dev->addr))) >>>>>>>> + if (!eal_compare_pci_addr(&dev->addr, &devargs->pci.addr)) >>>>>>>> return devargs; >>>>>>>> } >>>>>>>> return NULL; >>>>>>>> diff --git a/lib/librte_eal/common/include/rte_pci.h >>>>>>>> b/lib/librte_eal/common/include/rte_pci.h >>>>>>>> index 7f2d699..4814cd7 100644 >>>>>>>> --- a/lib/librte_eal/common/include/rte_pci.h >>>>>>>> +++ b/lib/librte_eal/common/include/rte_pci.h >>>>>>>> @@ -269,6 +269,40 @@ eal_parse_pci_DomBDF(const char *input, struct >>>>>>>> rte_pci_addr *dev_addr) } #undef GET_PCIADDR_FIELD >>>>>>>> >>>>>>>> +/* Compare two PCI device addresses. */ >>>>>>>> +/** >>>>>>>> + * Utility function to compare two PCI device addresses. >>>>>>>> + * >>>>>>>> + * @param addr >>>>>>>> + * The PCI Bus-Device-Function address to compare >>>>>>>> + * @param addr2 >>>>>>>> + * The PCI Bus-Device-Function address to compare >>>>>>>> + * @return >>>>>>>> + * 0 on equal PCI address. >>>>>>>> + * Positive on addr is greater than addr2. >>>>>>>> + * Negative on addr is less than addr2, or error. >>>>>>>> + */ >>>>>>>> +static inline int >>>>>>>> +eal_compare_pci_addr(struct rte_pci_addr *addr, struct rte_pci_addr >>>>>>>> +*addr2) { >>>>>>>> + uint64_t dev_addr, dev_addr2; >>>>>>>> + >>>>>>>> + if ((addr == NULL) || (addr2 == NULL)) >>>>>>>> + return -1; >>>>>>>> + >>>>>>>> + dev_addr = (addr->domain << 24) | (addr->bus << 16) | >>>>>>>> + (addr->devid << 8) | addr->function; >>>>>>>> + dev_addr2 = (addr2->domain << 24) | (addr2->bus << 16) | >>>>>>>> + (addr2->devid << 8) | addr2->function; >>>>>>>> + >>>>>>>> + if (dev_addr > dev_addr2) >>>>>>>> + return 1; >>>>>>>> + else if (dev_addr < dev_addr2) >>>>>>>> + return -1; >>>>>>>> + else >>>>>>>> + return 0; >>>>>>>> +} >>>>>>>> + >>>>>>>> /** >>>>>>>> * Probe the PCI bus for registered drivers. >>>>>>>> * >>>>>>>> diff --git a/lib/librte_eal/linuxapp/eal/eal_pci.c >>>>>>>> b/lib/librte_eal/linuxapp/eal/eal_pci.c >>>>>>>> index c0ca5a5..d847102 100644 >>>>>>>> --- a/lib/librte_eal/linuxapp/eal/eal_pci.c >>>>>>>> +++ b/lib/librte_eal/linuxapp/eal/eal_pci.c >>>>>>>> @@ -229,20 +229,6 @@ error: >>>>>>>> return -1; >>>>>>>> } >>>>>>>> >>>>>>>> -/* Compare two PCI device addresses. */ -static int >>>>>>>> -pci_addr_comparison(struct rte_pci_addr *addr, struct rte_pci_addr >>>>>>>> *addr2) -{ >>>>>>>> - uint64_t dev_addr = (addr->domain << 24) + (addr->bus << 16) + (addr->devid << 8) + addr- >>>>>>>> function; >>>>>>>> - uint64_t dev_addr2 = (addr2->domain << 24) + (addr2->bus << 16) + (addr2->devid << 8) + >>>>>>> addr2->function; >>>>>>>> - >>>>>>>> - if (dev_addr > dev_addr2) >>>>>>>> - return 1; >>>>>>>> - else >>>>>>>> - return 0; >>>>>>>> -} >>>>>>>> - >>>>>>>> - >>>>>>>> /* Scan one pci sysfs entry, and fill the devices list from it. */ >>>>>>>> static int pci_scan_one(const char *dirname, uint16_t domain, uint8_t >>>>>>>> bus, @@ -353,13 +339,20 @@ pci_scan_one(const char *dirname, uint16_t >>>>>>>> domain, uint8_t bus, >>>>>>>> } >>>>>>>> else { >>>>>>>> struct rte_pci_device *dev2 = NULL; >>>>>>>> + int ret; >>>>>>>> >>>>>>>> TAILQ_FOREACH(dev2, &pci_device_list, next) { >>>>>>>> - if (pci_addr_comparison(&dev->addr, &dev2->addr)) >>>>>>>> + ret = eal_compare_pci_addr(&dev->addr, &dev2->addr); >>>>>>>> + if (ret > 0) >>>>>>>> continue; >>>>>>>> - else { >>>>>>>> + else if (ret < 0) { >>>>>>>> TAILQ_INSERT_BEFORE(dev2, dev, next); >>>>>>>> return 0; >>>>>>>> + } else { /* already registered */ >>>>>>>> + /* update pt_driver */ >>>>>>>> + dev2->pt_driver = dev->pt_driver; >>>>>> Hi Tetsuya, >>>>>> >>>>>> I am seeing a problem with the librte_pmd_ixgbe code where dev->max_vfs is being lost in some scenarios. >>>>>> The following line should be added here: >>>>>> dev2->max_vfs = dev->max_vfs; >>>>>> >>>>>> numa_mode should probably be updated too (although it is not causing a problem at present). >>>>>> dev2->numa_mode = dev->numa_mode; >>>>> I'm very curious, why those field miss? I haven't see any places clear >>>>> this field. >>>>> >>>>> What is the root cause? >>>> Hi Michael, >>>> >>>> Here is my guess. >>>> The above function creates pci device list. >>> I am sorry. I forgot to add below information. >>> >>> "max_vfs" or "numa_node" value is came from sysfs when the above >>> function is processed. >> Yes, but it has already been registered, why it missed? > Yes, it has been registered already, but probably should be updated. > I guess sysfs value will be changed when igb_uio starts managing the device. > > ex) > 1. Boot linux > 2. start a dpdk application with no port. > 3. pci device list is registered. > - Here, "max_vfs" is came from sysfs. Or there is no such a entry. > 4. igb_uio binds the device. > 5. I guess max_vfs value of sysfs is changed. Or max_vfs entry is created. > 6. The dpdk application calls hotplug function. Yes, agree. But numa node can be changed? Bernard, does your issue occur after max_vfs changed in igb_uio? If not, I think must be figure out the reason. Thanks, Michael > - Here, I guess we need to update "max_vfs" value. > > Above is a just my assumption. > It may be good to wait for Bernard's reply. > > Thanks, > Tetsuya > >> Thanks, >> Michael >>>> And current DPDK implementation assumes all devices needed to be managed >>>> are under igb_uio or vfio when above code is processed. >>>> To add hotplug function, we also need to think some devices will start >>>> to be managed under igb_uio or vfio after initializing pci device list. >>>> Anyway, I guess "max_vfs" value will be changed when igb_uio or vfio >>>> manages the device. >>>> >>>> Hi Bernard, >>>> >>>> Could you please check "max_vfs" and "num_node" values, then check the >>>> values again after the device is managed by igb_uio or vfio? >>>> In my environment, it seems max_vfs is created by igb_uio. >>>> But my NIC doesn't have VF, so behavior might be different in your >>>> environment. >>>> I guess "numa_node" should not be changed theoretically. >>>> >>>> If my guess is correct, how about replacing following values? >>>> - driver >>>> - max_vfs >>>> - resource >>>> - (numa_node) >>>> Except for above value, I guess other value shouldn't be changed even >>>> after the device is managed by igb_uio or vfio. >>>> >>>> Thanks, >>>> Tetsuya >>>> >>>>> Thanks, >>>>> Michael >>>>> >>>>>> Regards, >>>>>> >>>>>> Bernard. >>>>>> >>>>>> >>>>>> >>>>>> >>>>>>>> + free(dev); >>>>>>>> + return 0; >>>>>>>> } >>>>>>>> } >>>>>>>> TAILQ_INSERT_TAIL(&pci_device_list, dev, next); diff --git >>>>>>>> a/lib/librte_eal/linuxapp/eal/eal_pci_uio.c >>>>>>>> b/lib/librte_eal/linuxapp/eal/eal_pci_uio.c >>>>>>>> index e53f06b..1da3507 100644 >>>>>>>> --- a/lib/librte_eal/linuxapp/eal/eal_pci_uio.c >>>>>>>> +++ b/lib/librte_eal/linuxapp/eal/eal_pci_uio.c >>>>>>>> @@ -123,7 +123,7 @@ pci_uio_map_secondary(struct rte_pci_device *dev) >>>>>>>> TAILQ_FOREACH(uio_res, pci_res_list, next) { >>>>>>>> >>>>>>>> /* skip this element if it doesn't match our PCI address */ >>>>>>>> - if (memcmp(&uio_res->pci_addr, &dev->addr, sizeof(dev->addr))) >>>>>>>> + if (eal_compare_pci_addr(&uio_res->pci_addr, &dev->addr)) >>>>>>>> continue; >>>>>>>> >>>>>>>> for (i = 0; i != uio_res->nb_maps; i++) { >>>>>>> Acked-by: Michael Qiu <michael.qiu@intel.com> > >
On 2015/02/11 21:13, Qiu, Michael wrote: > On 2/11/2015 4:14 PM, Tetsuya Mukawa wrote: >> On 2015/02/11 15:29, Qiu, Michael wrote: >>> On 2/11/2015 12:57 PM, Tetsuya Mukawa wrote: >>>> On 2015/02/11 13:53, Tetsuya Mukawa wrote: >>>>> On 2015/02/11 12:27, Qiu, Michael wrote: >>>>>> On 2/10/2015 11:11 PM, Iremonger, Bernard wrote: >>>>>>>> -----Original Message----- >>>>>>>> From: Qiu, Michael >>>>>>>> Sent: Monday, February 9, 2015 1:10 PM >>>>>>>> To: Tetsuya Mukawa; dev@dpdk.org >>>>>>>> Cc: Iremonger, Bernard >>>>>>>> Subject: Re: [PATCH v7 04/14] eal/pci: Consolidate pci address comparison APIs >>>>>>>> >>>>>>>> On 2/9/2015 4:31 PM, Tetsuya Mukawa wrote: >>>>>>>>> This patch replaces pci_addr_comparison() and memcmp() of pci >>>>>>>>> addresses by eal_compare_pci_addr(). >>>>>>>>> >>>>>>>>> v5: >>>>>>>>> - Fix pci_scan_one to handle pt_driver correctly. >>>>>>>>> v4: >>>>>>>>> - Fix calculation method of eal_compare_pci_addr(). >>>>>>>>> - Add parameter checking. >>>>>>>>> >>>>>>>>> Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp> >>>>>>>>> --- >>>>>>>>> lib/librte_eal/bsdapp/eal/eal_pci.c | 25 ++++++++--------------- >>>>>>>>> lib/librte_eal/common/eal_common_pci.c | 2 +- >>>>>>>>> lib/librte_eal/common/include/rte_pci.h | 34 +++++++++++++++++++++++++++++++ >>>>>>>>> lib/librte_eal/linuxapp/eal/eal_pci.c | 25 ++++++++--------------- >>>>>>>>> lib/librte_eal/linuxapp/eal/eal_pci_uio.c | 2 +- >>>>>>>>> 5 files changed, 54 insertions(+), 34 deletions(-) >>>>>>>>> >>>>>>>>> diff --git a/lib/librte_eal/bsdapp/eal/eal_pci.c >>>>>>>>> b/lib/librte_eal/bsdapp/eal/eal_pci.c >>>>>>>>> index 74ecce7..c844d58 100644 >>>>>>>>> --- a/lib/librte_eal/bsdapp/eal/eal_pci.c >>>>>>>>> +++ b/lib/librte_eal/bsdapp/eal/eal_pci.c >>>>>>>>> @@ -270,20 +270,6 @@ pci_uio_map_resource(struct rte_pci_device *dev) >>>>>>>>> return (0); >>>>>>>>> } >>>>>>>>> >>>>>>>>> -/* Compare two PCI device addresses. */ -static int >>>>>>>>> -pci_addr_comparison(struct rte_pci_addr *addr, struct rte_pci_addr >>>>>>>>> *addr2) -{ >>>>>>>>> - uint64_t dev_addr = (addr->domain << 24) + (addr->bus << 16) + (addr->devid << 8) + addr- >>>>>>>>> function; >>>>>>>>> - uint64_t dev_addr2 = (addr2->domain << 24) + (addr2->bus << 16) + (addr2->devid << 8) + >>>>>>>> addr2->function; >>>>>>>>> - >>>>>>>>> - if (dev_addr > dev_addr2) >>>>>>>>> - return 1; >>>>>>>>> - else >>>>>>>>> - return 0; >>>>>>>>> -} >>>>>>>>> - >>>>>>>>> - >>>>>>>>> /* Scan one pci sysfs entry, and fill the devices list from it. */ >>>>>>>>> static int pci_scan_one(int dev_pci_fd, struct pci_conf *conf) @@ >>>>>>>>> -356,13 +342,20 @@ pci_scan_one(int dev_pci_fd, struct pci_conf *conf) >>>>>>>>> } >>>>>>>>> else { >>>>>>>>> struct rte_pci_device *dev2 = NULL; >>>>>>>>> + int ret; >>>>>>>>> >>>>>>>>> TAILQ_FOREACH(dev2, &pci_device_list, next) { >>>>>>>>> - if (pci_addr_comparison(&dev->addr, &dev2->addr)) >>>>>>>>> + ret = eal_compare_pci_addr(&dev->addr, &dev2->addr); >>>>>>>>> + if (ret > 0) >>>>>>>>> continue; >>>>>>>>> - else { >>>>>>>>> + else if (ret < 0) { >>>>>>>>> TAILQ_INSERT_BEFORE(dev2, dev, next); >>>>>>>>> return 0; >>>>>>>>> + } else { /* already registered */ >>>>>>>>> + /* update pt_driver */ >>>>>>>>> + dev2->pt_driver = dev->pt_driver; >>>>>>>>> + free(dev); >>>>>>>>> + return 0; >>>>>>>>> } >>>>>>>>> } >>>>>>>>> TAILQ_INSERT_TAIL(&pci_device_list, dev, next); diff --git >>>>>>>>> a/lib/librte_eal/common/eal_common_pci.c >>>>>>>>> b/lib/librte_eal/common/eal_common_pci.c >>>>>>>>> index f3c7f71..a89f5c3 100644 >>>>>>>>> --- a/lib/librte_eal/common/eal_common_pci.c >>>>>>>>> +++ b/lib/librte_eal/common/eal_common_pci.c >>>>>>>>> @@ -93,7 +93,7 @@ static struct rte_devargs *pci_devargs_lookup(struct rte_pci_device *dev) >>>>>>>>> if (devargs->type != RTE_DEVTYPE_BLACKLISTED_PCI && >>>>>>>>> devargs->type != RTE_DEVTYPE_WHITELISTED_PCI) >>>>>>>>> continue; >>>>>>>>> - if (!memcmp(&dev->addr, &devargs->pci.addr, sizeof(dev->addr))) >>>>>>>>> + if (!eal_compare_pci_addr(&dev->addr, &devargs->pci.addr)) >>>>>>>>> return devargs; >>>>>>>>> } >>>>>>>>> return NULL; >>>>>>>>> diff --git a/lib/librte_eal/common/include/rte_pci.h >>>>>>>>> b/lib/librte_eal/common/include/rte_pci.h >>>>>>>>> index 7f2d699..4814cd7 100644 >>>>>>>>> --- a/lib/librte_eal/common/include/rte_pci.h >>>>>>>>> +++ b/lib/librte_eal/common/include/rte_pci.h >>>>>>>>> @@ -269,6 +269,40 @@ eal_parse_pci_DomBDF(const char *input, struct >>>>>>>>> rte_pci_addr *dev_addr) } #undef GET_PCIADDR_FIELD >>>>>>>>> >>>>>>>>> +/* Compare two PCI device addresses. */ >>>>>>>>> +/** >>>>>>>>> + * Utility function to compare two PCI device addresses. >>>>>>>>> + * >>>>>>>>> + * @param addr >>>>>>>>> + * The PCI Bus-Device-Function address to compare >>>>>>>>> + * @param addr2 >>>>>>>>> + * The PCI Bus-Device-Function address to compare >>>>>>>>> + * @return >>>>>>>>> + * 0 on equal PCI address. >>>>>>>>> + * Positive on addr is greater than addr2. >>>>>>>>> + * Negative on addr is less than addr2, or error. >>>>>>>>> + */ >>>>>>>>> +static inline int >>>>>>>>> +eal_compare_pci_addr(struct rte_pci_addr *addr, struct rte_pci_addr >>>>>>>>> +*addr2) { >>>>>>>>> + uint64_t dev_addr, dev_addr2; >>>>>>>>> + >>>>>>>>> + if ((addr == NULL) || (addr2 == NULL)) >>>>>>>>> + return -1; >>>>>>>>> + >>>>>>>>> + dev_addr = (addr->domain << 24) | (addr->bus << 16) | >>>>>>>>> + (addr->devid << 8) | addr->function; >>>>>>>>> + dev_addr2 = (addr2->domain << 24) | (addr2->bus << 16) | >>>>>>>>> + (addr2->devid << 8) | addr2->function; >>>>>>>>> + >>>>>>>>> + if (dev_addr > dev_addr2) >>>>>>>>> + return 1; >>>>>>>>> + else if (dev_addr < dev_addr2) >>>>>>>>> + return -1; >>>>>>>>> + else >>>>>>>>> + return 0; >>>>>>>>> +} >>>>>>>>> + >>>>>>>>> /** >>>>>>>>> * Probe the PCI bus for registered drivers. >>>>>>>>> * >>>>>>>>> diff --git a/lib/librte_eal/linuxapp/eal/eal_pci.c >>>>>>>>> b/lib/librte_eal/linuxapp/eal/eal_pci.c >>>>>>>>> index c0ca5a5..d847102 100644 >>>>>>>>> --- a/lib/librte_eal/linuxapp/eal/eal_pci.c >>>>>>>>> +++ b/lib/librte_eal/linuxapp/eal/eal_pci.c >>>>>>>>> @@ -229,20 +229,6 @@ error: >>>>>>>>> return -1; >>>>>>>>> } >>>>>>>>> >>>>>>>>> -/* Compare two PCI device addresses. */ -static int >>>>>>>>> -pci_addr_comparison(struct rte_pci_addr *addr, struct rte_pci_addr >>>>>>>>> *addr2) -{ >>>>>>>>> - uint64_t dev_addr = (addr->domain << 24) + (addr->bus << 16) + (addr->devid << 8) + addr- >>>>>>>>> function; >>>>>>>>> - uint64_t dev_addr2 = (addr2->domain << 24) + (addr2->bus << 16) + (addr2->devid << 8) + >>>>>>>> addr2->function; >>>>>>>>> - >>>>>>>>> - if (dev_addr > dev_addr2) >>>>>>>>> - return 1; >>>>>>>>> - else >>>>>>>>> - return 0; >>>>>>>>> -} >>>>>>>>> - >>>>>>>>> - >>>>>>>>> /* Scan one pci sysfs entry, and fill the devices list from it. */ >>>>>>>>> static int pci_scan_one(const char *dirname, uint16_t domain, uint8_t >>>>>>>>> bus, @@ -353,13 +339,20 @@ pci_scan_one(const char *dirname, uint16_t >>>>>>>>> domain, uint8_t bus, >>>>>>>>> } >>>>>>>>> else { >>>>>>>>> struct rte_pci_device *dev2 = NULL; >>>>>>>>> + int ret; >>>>>>>>> >>>>>>>>> TAILQ_FOREACH(dev2, &pci_device_list, next) { >>>>>>>>> - if (pci_addr_comparison(&dev->addr, &dev2->addr)) >>>>>>>>> + ret = eal_compare_pci_addr(&dev->addr, &dev2->addr); >>>>>>>>> + if (ret > 0) >>>>>>>>> continue; >>>>>>>>> - else { >>>>>>>>> + else if (ret < 0) { >>>>>>>>> TAILQ_INSERT_BEFORE(dev2, dev, next); >>>>>>>>> return 0; >>>>>>>>> + } else { /* already registered */ >>>>>>>>> + /* update pt_driver */ >>>>>>>>> + dev2->pt_driver = dev->pt_driver; >>>>>>> Hi Tetsuya, >>>>>>> >>>>>>> I am seeing a problem with the librte_pmd_ixgbe code where dev->max_vfs is being lost in some scenarios. >>>>>>> The following line should be added here: >>>>>>> dev2->max_vfs = dev->max_vfs; >>>>>>> >>>>>>> numa_mode should probably be updated too (although it is not causing a problem at present). >>>>>>> dev2->numa_mode = dev->numa_mode; >>>>>> I'm very curious, why those field miss? I haven't see any places clear >>>>>> this field. >>>>>> >>>>>> What is the root cause? >>>>> Hi Michael, >>>>> >>>>> Here is my guess. >>>>> The above function creates pci device list. >>>> I am sorry. I forgot to add below information. >>>> >>>> "max_vfs" or "numa_node" value is came from sysfs when the above >>>> function is processed. >>> Yes, but it has already been registered, why it missed? >> Yes, it has been registered already, but probably should be updated. >> I guess sysfs value will be changed when igb_uio starts managing the device. >> >> ex) >> 1. Boot linux >> 2. start a dpdk application with no port. >> 3. pci device list is registered. >> - Here, "max_vfs" is came from sysfs. Or there is no such a entry. >> 4. igb_uio binds the device. >> 5. I guess max_vfs value of sysfs is changed. Or max_vfs entry is created. >> 6. The dpdk application calls hotplug function. > Yes, agree. > > But numa node can be changed? Hi Michael, I may misunderstand meaning of numa_node. I thought it indicated which numa node was nearest from the pci device, so it could not be configurable. BTW, I will be out of office tomorrow. So, I will submit v8 patches next Monday. Thanks, Tetsuya > > Bernard, does your issue occur after max_vfs changed in igb_uio? > > If not, I think must be figure out the reason. > > Thanks, > Michael >> - Here, I guess we need to update "max_vfs" value. >> >> Above is a just my assumption. >> It may be good to wait for Bernard's reply. >> >> Thanks, >> Tetsuya >> >>> Thanks, >>> Michael >>>>> And current DPDK implementation assumes all devices needed to be managed >>>>> are under igb_uio or vfio when above code is processed. >>>>> To add hotplug function, we also need to think some devices will start >>>>> to be managed under igb_uio or vfio after initializing pci device list. >>>>> Anyway, I guess "max_vfs" value will be changed when igb_uio or vfio >>>>> manages the device. >>>>> >>>>> Hi Bernard, >>>>> >>>>> Could you please check "max_vfs" and "num_node" values, then check the >>>>> values again after the device is managed by igb_uio or vfio? >>>>> In my environment, it seems max_vfs is created by igb_uio. >>>>> But my NIC doesn't have VF, so behavior might be different in your >>>>> environment. >>>>> I guess "numa_node" should not be changed theoretically. >>>>> >>>>> If my guess is correct, how about replacing following values? >>>>> - driver >>>>> - max_vfs >>>>> - resource >>>>> - (numa_node) >>>>> Except for above value, I guess other value shouldn't be changed even >>>>> after the device is managed by igb_uio or vfio. >>>>> >>>>> Thanks, >>>>> Tetsuya >>>>> >>>>>> Thanks, >>>>>> Michael >>>>>> >>>>>>> Regards, >>>>>>> >>>>>>> Bernard. >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>>>> + free(dev); >>>>>>>>> + return 0; >>>>>>>>> } >>>>>>>>> } >>>>>>>>> TAILQ_INSERT_TAIL(&pci_device_list, dev, next); diff --git >>>>>>>>> a/lib/librte_eal/linuxapp/eal/eal_pci_uio.c >>>>>>>>> b/lib/librte_eal/linuxapp/eal/eal_pci_uio.c >>>>>>>>> index e53f06b..1da3507 100644 >>>>>>>>> --- a/lib/librte_eal/linuxapp/eal/eal_pci_uio.c >>>>>>>>> +++ b/lib/librte_eal/linuxapp/eal/eal_pci_uio.c >>>>>>>>> @@ -123,7 +123,7 @@ pci_uio_map_secondary(struct rte_pci_device *dev) >>>>>>>>> TAILQ_FOREACH(uio_res, pci_res_list, next) { >>>>>>>>> >>>>>>>>> /* skip this element if it doesn't match our PCI address */ >>>>>>>>> - if (memcmp(&uio_res->pci_addr, &dev->addr, sizeof(dev->addr))) >>>>>>>>> + if (eal_compare_pci_addr(&uio_res->pci_addr, &dev->addr)) >>>>>>>>> continue; >>>>>>>>> >>>>>>>>> for (i = 0; i != uio_res->nb_maps; i++) { >>>>>>>> Acked-by: Michael Qiu <michael.qiu@intel.com> >>
> >>>>>>>>> /* Scan one pci sysfs entry, and fill the devices list from > >>>>>>>>> it. */ static int pci_scan_one(const char *dirname, uint16_t > >>>>>>>>> domain, uint8_t bus, @@ -353,13 +339,20 @@ pci_scan_one(const > >>>>>>>>> char *dirname, uint16_t domain, uint8_t bus, > >>>>>>>>> } > >>>>>>>>> else { > >>>>>>>>> struct rte_pci_device *dev2 = NULL; > >>>>>>>>> + int ret; > >>>>>>>>> > >>>>>>>>> TAILQ_FOREACH(dev2, &pci_device_list, next) { > >>>>>>>>> - if (pci_addr_comparison(&dev->addr, &dev2->addr)) > >>>>>>>>> + ret = eal_compare_pci_addr(&dev->addr, &dev2->addr); > >>>>>>>>> + if (ret > 0) > >>>>>>>>> continue; > >>>>>>>>> - else { > >>>>>>>>> + else if (ret < 0) { > >>>>>>>>> TAILQ_INSERT_BEFORE(dev2, dev, next); > >>>>>>>>> return 0; > >>>>>>>>> + } else { /* already registered */ > >>>>>>>>> + /* update pt_driver */ > >>>>>>>>> + dev2->pt_driver = dev->pt_driver; > >>>>>>> Hi Tetsuya, > >>>>>>> > >>>>>>> I am seeing a problem with the librte_pmd_ixgbe code where dev->max_vfs is being lost in > some scenarios. > >>>>>>> The following line should be added here: > >>>>>>> dev2->max_vfs = dev->max_vfs; > >>>>>>> > >>>>>>> numa_mode should probably be updated too (although it is not causing a problem at > present). > >>>>>>> dev2->numa_mode = dev->numa_mode; Hi Tetsuya, Michael, The "already registered path" in the code is being executed. dev->numa_mode does not change so the above line is not needed. dev2->max_vfs has a value of 0 and dev->max_vfs has a value of 2 ( 2 VF's have been configured previously). dev2->mem_resource is different from dev->mem_resource so the following line needs to be added: memmove(dev2->mem_resource, dev->mem_resource, sizeof(dev->mem_resource)); More replies below. > >>>>>> I'm very curious, why those field miss? I haven't see any places > >>>>>> clear this field. > >>>>>> > >>>>>> What is the root cause? The max_vfs entry is created when igb_uio binds the device. dev2 is the device already in the pci_device_list so dev2->max_vfs is 0. dev is the new device being parsed, in this case dev->max_vfs is 2. So the dev2->max_vfs value needs to be updated. Similarly dev2-> pt_driver needs to be updated as its value is 0 (this is already done). Similarly dev2->mem_resource needs to be updated as it is different from dev->mem_resource. > >>>>> Hi Michael, > >>>>> > >>>>> Here is my guess. > >>>>> The above function creates pci device list. > >>>> I am sorry. I forgot to add below information. > >>>> > >>>> "max_vfs" or "numa_node" value is came from sysfs when the above > >>>> function is processed. > >>> Yes, but it has already been registered, why it missed? > >> Yes, it has been registered already, but probably should be updated. > >> I guess sysfs value will be changed when igb_uio starts managing the device. > >> > >> ex) > >> 1. Boot linux > >> 2. start a dpdk application with no port. > >> 3. pci device list is registered. > >> - Here, "max_vfs" is came from sysfs. Or there is no such a entry. > >> 4. igb_uio binds the device. > >> 5. I guess max_vfs value of sysfs is changed. Or max_vfs entry is created. > >> 6. The dpdk application calls hotplug function. > > Yes, agree. > > > > But numa node can be changed? > > Hi Michael, > > I may misunderstand meaning of numa_node. > I thought it indicated which numa node was nearest from the pci device, so it could not be > configurable. > BTW, I will be out of office tomorrow. So, I will submit v8 patches next Monday. > > Thanks, > Tetsuya > > > > > Bernard, does your issue occur after max_vfs changed in igb_uio? > > > > If not, I think must be figure out the reason. > > > > Thanks, > > Michael > >> - Here, I guess we need to update "max_vfs" value. > >> > >> Above is a just my assumption. > >> It may be good to wait for Bernard's reply. > >> > >> Thanks, > >> Tetsuya > >> > >>> Thanks, > >>> Michael > >>>>> And current DPDK implementation assumes all devices needed to be > >>>>> managed are under igb_uio or vfio when above code is processed. > >>>>> To add hotplug function, we also need to think some devices will > >>>>> start to be managed under igb_uio or vfio after initializing pci device list. > >>>>> Anyway, I guess "max_vfs" value will be changed when igb_uio or > >>>>> vfio manages the device. > >>>>> > >>>>> Hi Bernard, > >>>>> > >>>>> Could you please check "max_vfs" and "num_node" values, then check > >>>>> the values again after the device is managed by igb_uio or vfio? > >>>>> In my environment, it seems max_vfs is created by igb_uio. Yes, max_vfs is created by igb_uio > >>>>> But my NIC doesn't have VF, so behavior might be different in your > >>>>> environment. > >>>>> I guess "numa_node" should not be changed theoretically. > >>>>> > >>>>> If my guess is correct, how about replacing following values? > >>>>> - driver Agreed > >>>>> - max_vfs Agreed > >>>>> - resource Agreed > >>>>> - (numa_node) Not necessary, as it does not change. > >>>>> Except for above value, I guess other value shouldn't be changed > >>>>> even after the device is managed by igb_uio or vfio. > >>>>> > >>>>> Thanks, > >>>>> Tetsuya > >>>>> > >>>>>> Thanks, > >>>>>> Michael > >>>>>> The scenario I am using is as follows: Terminal1: rmmod ixgbe Terminal1: rmmod ixgbevf Terminal1: rmmod igb_uio Terminal2: ./testpmd -c f -n 1 -d ../../librte_pmd_null.so -- -i --no-flush-rx Terminal1: insmod ./igb_uio.ko Terminal3: ./dpdk_nic_bind.py -b igb_uio 0000:05:00.0 Terminal2: testpmd> port attach 0000:05:00.0 Regards, Bernard.
This patch series adds a dynamic port hotplug framework to DPDK. With the patches, DPDK apps can attach or detach ports at runtime. The basic concept of the port hotplug is like followings. - DPDK apps must have responsibility to manage ports. DPDK apps only know which ports are attached or detached at the moment. The port hotplug framework is implemented to allow DPDK apps to manage ports. For example, when DPDK apps call port attach function, attached port number will be returned. Also, DPDK apps can detach port by port number. - Kernel support is needed for attaching or detaching physical device ports. To attach a new physical device port, the device will be recognized by userspace directly I/O framework in kernel at first. Then DPDK apps can call the port hotplug functions to attach ports. For detaching, steps are vice versa. - Before detach ports, ports must be stopped and closed. DPDK application must call rte_eth_dev_stop() and rte_eth_dev_close() before detaching ports. These function will call finalization codes of PMDs. But so far, no PMD frees all resources allocated by initialization. It means PMDs are needed to be fixed to support the port hotplug. 'RTE_PCI_DRV_DETACHABLE' is a new flag indicating a PMD supports detaching. Without this flag, detaching will be failed. - Mustn't affect legacy DPDK apps. No DPDK EAL behavior is changed, if the port hotplug functions are't called. So all legacy DPDK apps can still work without modifications. And a few limitations. - The port hotplug functions are not thread safe. DPDK apps should handle it. - Only support Linux and igb_uio so far. BSD and VFIO is not supported. I will send VFIO patches at least, but I don't have a plan to submit BSD patch so far. Here is port hotplug APIs. ------------------------------------------------------------------------------- /** * Attach a new device. * * @param devargs * A pointer to a strings array describing the new device * to be attached. The strings should be a pci address like * '0000:01:00.0' or virtual device name like 'eth_pcap0'. * @param port_id * A pointer to a port identifier actually attached. * @return * 0 on success and port_id is filled, negative on error */ int rte_eal_dev_attach(const char *devargs, uint8_t *port_id); /** * Detach a device. * * @param port_id * The port identifier of the device to detach. * @param addr * A pointer to a device name actually detached. * @return * 0 on success and devname is filled, negative on error */ int rte_eal_dev_detach(uint8_t port_id, char *devname); ------------------------------------------------------------------------------- This patch series are for DPDK EAL. To use port hotplug function by DPDK apps, each PMD should be fixed to support 'RTE_PCI_DRV_DETACHABLE' flag. Please check a patch for pcap PMD. Also, please check testpmd patch. It will show you how to fix your legacy applications to support port hotplug feature. PATCH v8 changes - Fix Makefile and add version map file. - Add missing symbol in version map. - Fix pci_scan_one() to update sysfs values. (Thanks to Qiu, Michael and Iremonger, Bernard) - NONE_TRACE is replaced by NO_TRACE. - Fix typo. - Add size parameter to rte_eth_dev_save(). (Thanks to Iremonger, Bernard) PATCH v7 changes - Add a new section to programmer's guide. (Thanks to Iremonger, Bernard) - Fix port checking implementation of star_port(). - Fix typo of warning messages. - Add pt_driver checking to rte_eth_dev_check_detachable(). (Thanks to Qiu, Michael) PATCH v6 changes - Fix rte_eth_dev_uninit() to handle a return value of uninit function of PMD. To do this, below changes also be applied. - Fix a parameter of rte_eth_dev_free(). - Use rte_eth_dev structure as the paramter of rte_eth_dev_free(). PATCH v5 changes - Add runtime check passthrough driver type, like vfio-pci, igb_uio and uio_pci_generic. This was done by Qiu, Michael. Thanks a lot. - Change function names like below. - rte_eal_dev_find_and_invoke() to rte_eal_vdev_find_and_invoke(). - rte_eal_dev_invoke() to rte_eal_vdev_invoke(). - Add code to handle a return value of rte_eal_devargs_remove(). - Fix pci address format in rte_eal_dev_detach(). - Remove RTE_EAL_INVOKE_TYPE_UNKNOWN, because it's unused. - Change function definition of rte_eal_devargs_remove(). - Fix pci_unmap_device() to check pt_driver. - Fix return value of below functions. - rte_eth_dev_get_changed_port(). - rte_eth_dev_get_port_by_addr(). - Change paramters of rte_eth_dev_validate_port() to cleanup code. - Fix pci_scan_one to handle pt_driver correctly. (Thanks to Qiu, Michael for above suggestions) PATCH v4 changes - Merge patches to review easier. - Fix indent of 'if' statement. - Fix calculation method of eal_compare_pci_addr(). - Fix header file declaration. - Add header file to determine if hotplug can be enabled. (Thanks to Qiu, Michael) - Use braces with 'for' loop. - Add parameter checking. - Fix sanity check code - Fix comments of rte_eth_dev_type. - Change function names. (Thanks to Iremonger, Bernard) PATCH v3 changes: - Fix enum definition used in rte_ethdev.c. (Thanks to Zhang, Helin) PATCH v2 changes: - Replace rte_eal_dev_attach_pdev(), rte_eal_dev_detach_pdev, rte_eal_dev_attach_vdev() and rte_eal_dev_detach_vdev() to rte_eal_dev_attach() and rte_eal_dev_detach(). - Add parameter values checking. - Refashion a few functions. (Thanks to Iremonger, Bernard) PATCH v1 Changes: - Fix error checking code of librte_eth APIs. - Fix issue that port from pcap PMD cannot be detached correctly. - Fix issue that testpmd could hang after forwarding, if attaching and detaching is repeatedly. - Fix if-condition of rte_eth_dev_get_port_by_addr(). (Thanks to Mark Enright) RFC PATCH v2 Changes: - remove 'rte_eth_dev_validate_port()', and cleanup codes. Michael Qiu (2): eal_pci: Add flag to hold kernel driver type eal_pci: pci memory map work with driver type Tetsuya Mukawa (12): eal/pci,ethdev: Remove assumption that port will not be detached eal/pci: Consolidate pci address comparison APIs ethdev: Add rte_eth_dev_free to free specified device eal,ethdev: Add a function and function pointers to close ether device ethdev: Add functions that will be used by port hotplug functions eal/linux/pci: Add functions for unmapping igb_uio resources eal/pci: Add a function to remove the entry of devargs list eal/pci: Cleanup pci driver initialization code ethdev: Add one dev_type parameter to rte_eth_dev_allocate eal/pci: Add rte_eal_dev_attach/detach() functions eal: Enable port hotplug framework in Linux doc: Add port hotplug framework section to programmers guide app/test/virtual_pmd.c | 2 +- config/common_linuxapp | 5 + doc/guides/prog_guide/index.rst | 1 + doc/guides/prog_guide/port_hotplug_framework.rst | 110 ++++ lib/librte_eal/bsdapp/eal/eal_pci.c | 29 +- lib/librte_eal/common/Makefile | 1 + lib/librte_eal/common/eal_common_dev.c | 276 ++++++++++ lib/librte_eal/common/eal_common_devargs.c | 60 +++ lib/librte_eal/common/eal_common_pci.c | 92 +++- lib/librte_eal/common/eal_private.h | 35 ++ lib/librte_eal/common/include/rte_dev.h | 33 ++ lib/librte_eal/common/include/rte_dev_hotplug.h | 44 ++ lib/librte_eal/common/include/rte_devargs.h | 21 + lib/librte_eal/common/include/rte_pci.h | 84 +++ lib/librte_eal/linuxapp/eal/Makefile | 1 + lib/librte_eal/linuxapp/eal/eal_pci.c | 232 +++++++-- lib/librte_eal/linuxapp/eal/eal_pci_init.h | 8 + lib/librte_eal/linuxapp/eal/eal_pci_uio.c | 67 ++- lib/librte_eal/linuxapp/eal/rte_eal_version.map | 2 + lib/librte_ether/rte_ethdev.c | 636 +++++++++++++---------- lib/librte_ether/rte_ethdev.h | 151 +++++- lib/librte_ether/rte_ether_version.map | 7 + lib/librte_pmd_af_packet/rte_eth_af_packet.c | 2 +- lib/librte_pmd_bond/rte_eth_bond_api.c | 2 +- lib/librte_pmd_pcap/rte_eth_pcap.c | 2 +- lib/librte_pmd_ring/rte_eth_ring.c | 2 +- lib/librte_pmd_xenvirt/rte_eth_xenvirt.c | 2 +- 27 files changed, 1560 insertions(+), 347 deletions(-) create mode 100644 doc/guides/prog_guide/port_hotplug_framework.rst create mode 100644 lib/librte_eal/common/include/rte_dev_hotplug.h -- 1.9.1
From: Michael Qiu <michael.qiu@intel.com> Currently, dpdk has no ability to know which type of driver( vfio-pci/igb_uio/uio_pci_generic) the device used. It only can check whether vfio is enabled or not staticly. It really useful to have the flag, becasue different type need to handle differently in runtime. For example, pci memory map, pot hotplug, and so on. This patch add a flag field for pci device to solve above issue. Signed-off-by: Michael Qiu <michael.qiu@intel.com> Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp> --- lib/librte_eal/common/include/rte_pci.h | 8 +++++ lib/librte_eal/linuxapp/eal/eal_pci.c | 53 +++++++++++++++++++++++++++++++-- 2 files changed, 59 insertions(+), 2 deletions(-) diff --git a/lib/librte_eal/common/include/rte_pci.h b/lib/librte_eal/common/include/rte_pci.h index 66ed793..7b48b55 100644 --- a/lib/librte_eal/common/include/rte_pci.h +++ b/lib/librte_eal/common/include/rte_pci.h @@ -139,6 +139,13 @@ struct rte_pci_addr { struct rte_devargs; +enum rte_pt_driver { + RTE_PT_UNKNOWN = 0, + RTE_PT_IGB_UIO = 1, + RTE_PT_VFIO = 2, + RTE_PT_UIO_GENERIC = 3, +}; + /** * A structure describing a PCI device. */ @@ -152,6 +159,7 @@ struct rte_pci_device { uint16_t max_vfs; /**< sriov enable if not zero */ int numa_node; /**< NUMA node connection */ struct rte_devargs *devargs; /**< Device user arguments */ + enum rte_pt_driver pt_driver; /**< Driver of passthrough */ }; /** Any PCI device identifier (vendor, device, ...) */ diff --git a/lib/librte_eal/linuxapp/eal/eal_pci.c b/lib/librte_eal/linuxapp/eal/eal_pci.c index 15db9c4..e760452 100644 --- a/lib/librte_eal/linuxapp/eal/eal_pci.c +++ b/lib/librte_eal/linuxapp/eal/eal_pci.c @@ -97,6 +97,35 @@ error: return -1; } +static int +pci_get_kernel_driver_by_path(const char *filename, char *dri_name) +{ + int count; + char path[PATH_MAX]; + char *name; + + if (!filename || !dri_name) + return -1; + + count = readlink(filename, path, PATH_MAX); + if (count >= PATH_MAX) + return -1; + + /* For device does not have a driver */ + if (count < 0) + return 1; + + path[count] = '\0'; + + name = strrchr(path, '/'); + if (name) { + strncpy(dri_name, name + 1, strlen(name + 1) + 1); + return 0; + } + + return -1; +} + void * pci_find_max_end_va(void) { @@ -222,11 +251,12 @@ pci_scan_one(const char *dirname, uint16_t domain, uint8_t bus, char filename[PATH_MAX]; unsigned long tmp; struct rte_pci_device *dev; + char driver[PATH_MAX]; + int ret; dev = malloc(sizeof(*dev)); - if (dev == NULL) { + if (dev == NULL) return -1; - } memset(dev, 0, sizeof(*dev)); dev->addr.domain = domain; @@ -305,6 +335,25 @@ pci_scan_one(const char *dirname, uint16_t domain, uint8_t bus, return -1; } + /* parse driver */ + snprintf(filename, sizeof(filename), "%s/driver", dirname); + ret = pci_get_kernel_driver_by_path(filename, driver); + if (!ret) { + if (!strcmp(driver, "vfio-pci")) + dev->pt_driver = RTE_PT_VFIO; + else if (!strcmp(driver, "igb_uio")) + dev->pt_driver = RTE_PT_IGB_UIO; + else if (!strcmp(driver, "uio_pci_generic")) + dev->pt_driver = RTE_PT_UIO_GENERIC; + else + dev->pt_driver = RTE_PT_UNKNOWN; + } else if (ret < 0) { + RTE_LOG(ERR, EAL, "Fail to get kernel driver\n"); + free(dev); + return -1; + } else + dev->pt_driver = RTE_PT_UNKNOWN; + /* device is valid, add in list (sorted) */ if (TAILQ_EMPTY(&pci_device_list)) { TAILQ_INSERT_TAIL(&pci_device_list, dev, next); -- 1.9.1
From: Michael Qiu <michael.qiu@intel.com> With the driver type flag in struct rte_pci_dev, we do not need to always map uio devices with vfio related function when vfio enabled. Signed-off-by: Michael Qiu <michael.qiu@intel.com> Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp> --- lib/librte_eal/linuxapp/eal/eal_pci.c | 30 +++++++++++++++++------------- 1 file changed, 17 insertions(+), 13 deletions(-) diff --git a/lib/librte_eal/linuxapp/eal/eal_pci.c b/lib/librte_eal/linuxapp/eal/eal_pci.c index e760452..3c463b2 100644 --- a/lib/librte_eal/linuxapp/eal/eal_pci.c +++ b/lib/librte_eal/linuxapp/eal/eal_pci.c @@ -556,25 +556,29 @@ pci_config_space_set(struct rte_pci_device *dev) static int pci_map_device(struct rte_pci_device *dev) { - int ret, mapped = 0; + int ret = -1; /* try mapping the NIC resources using VFIO if it exists */ + switch (dev->pt_driver) { + case RTE_PT_VFIO: #ifdef VFIO_PRESENT - if (pci_vfio_is_enabled()) { - ret = pci_vfio_map_resource(dev); - if (ret == 0) - mapped = 1; - else if (ret < 0) - return ret; - } + if (pci_vfio_is_enabled()) + ret = pci_vfio_map_resource(dev); #endif - /* map resources for devices that use igb_uio */ - if (!mapped) { + break; + case RTE_PT_IGB_UIO: + case RTE_PT_UIO_GENERIC: + /* map resources for devices that use uio */ ret = pci_uio_map_resource(dev); - if (ret != 0) - return ret; + break; + default: + RTE_LOG(DEBUG, EAL, " Not managed by known pt driver," + " skipped\n"); + ret = 1; + break; } - return 0; + + return ret; } /* -- 1.9.1
To remove assumption, do like followings. This patch adds "RTE_PCI_DRV_DETACHABLE" to drv_flags of rte_pci_driver structure. The flags indicate the driver can detach devices at runtime. Also, remove assumption that port will not be detached. To remove the assumption. - Add 'attached' member to rte_eth_dev structure. This member is used for indicating the port is attached, or not. - Add rte_eth_dev_allocate_new_port(). This function is used for allocating new port. v8: - NONE_TRACE is changed to NO_TRACE. (Thanks to Iremonger, Bernard) v5: - Change parameters of rte_eth_dev_validate_port() to cleanup code. v4: - Use braces with 'for' loop. - Fix indent of 'if' statement. Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp> --- lib/librte_eal/common/include/rte_pci.h | 2 + lib/librte_ether/rte_ethdev.c | 454 +++++++++++++------------------- lib/librte_ether/rte_ethdev.h | 5 + 3 files changed, 186 insertions(+), 275 deletions(-) diff --git a/lib/librte_eal/common/include/rte_pci.h b/lib/librte_eal/common/include/rte_pci.h index 7b48b55..7f2d699 100644 --- a/lib/librte_eal/common/include/rte_pci.h +++ b/lib/librte_eal/common/include/rte_pci.h @@ -207,6 +207,8 @@ struct rte_pci_driver { #define RTE_PCI_DRV_FORCE_UNBIND 0x0004 /** Device driver supports link state interrupt */ #define RTE_PCI_DRV_INTR_LSC 0x0008 +/** Device driver supports detaching capability */ +#define RTE_PCI_DRV_DETACHABLE 0x0010 /**< Internal use only - Macro used by pci addr parsing functions **/ #define GET_PCIADDR_FIELD(in, fd, lim, dlm) \ diff --git a/lib/librte_ether/rte_ethdev.c b/lib/librte_ether/rte_ethdev.c index ea3a1fb..a79fa5b 100644 --- a/lib/librte_ether/rte_ethdev.c +++ b/lib/librte_ether/rte_ethdev.c @@ -175,6 +175,16 @@ enum { STAT_QMAP_RX }; +enum { + DEV_INVALID = 0, + DEV_VALID, +}; + +enum { + DEV_DISCONNECTED = 0, + DEV_CONNECTED +}; + static inline void rte_eth_dev_data_alloc(void) { @@ -201,19 +211,34 @@ rte_eth_dev_allocated(const char *name) { unsigned i; - for (i = 0; i < nb_ports; i++) { - if (strcmp(rte_eth_devices[i].data->name, name) == 0) + for (i = 0; i < RTE_MAX_ETHPORTS; i++) { + if ((rte_eth_devices[i].attached == DEV_CONNECTED) && + strcmp(rte_eth_devices[i].data->name, name) == 0) return &rte_eth_devices[i]; } return NULL; } +static uint8_t +rte_eth_dev_allocate_new_port(void) +{ + unsigned i; + + for (i = 0; i < RTE_MAX_ETHPORTS; i++) { + if (rte_eth_devices[i].attached == DEV_DISCONNECTED) + return i; + } + return RTE_MAX_ETHPORTS; +} + struct rte_eth_dev * rte_eth_dev_allocate(const char *name) { + uint8_t port_id; struct rte_eth_dev *eth_dev; - if (nb_ports == RTE_MAX_ETHPORTS) { + port_id = rte_eth_dev_allocate_new_port(); + if (port_id == RTE_MAX_ETHPORTS) { PMD_DEBUG_TRACE("Reached maximum number of Ethernet ports\n"); return NULL; } @@ -226,10 +251,12 @@ rte_eth_dev_allocate(const char *name) return NULL; } - eth_dev = &rte_eth_devices[nb_ports]; - eth_dev->data = &rte_eth_dev_data[nb_ports]; + eth_dev = &rte_eth_devices[port_id]; + eth_dev->data = &rte_eth_dev_data[port_id]; snprintf(eth_dev->data->name, sizeof(eth_dev->data->name), "%s", name); - eth_dev->data->port_id = nb_ports++; + eth_dev->data->port_id = port_id; + eth_dev->attached = DEV_CONNECTED; + nb_ports++; return eth_dev; } @@ -283,6 +310,7 @@ rte_eth_dev_init(struct rte_pci_driver *pci_drv, (unsigned) pci_dev->id.device_id); if (rte_eal_process_type() == RTE_PROC_PRIMARY) rte_free(eth_dev->data->dev_private); + eth_dev->attached = DEV_DISCONNECTED; nb_ports--; return diag; } @@ -308,10 +336,28 @@ rte_eth_driver_register(struct eth_driver *eth_drv) rte_eal_pci_register(ð_drv->pci_drv); } +enum { + NO_TRACE = 0, + TRACE +}; + +static int +rte_eth_dev_validate_port(uint8_t port_id, int trace) +{ + if (port_id >= RTE_MAX_ETHPORTS || + rte_eth_devices[port_id].attached != DEV_CONNECTED) { + if (trace) { + PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + } + return DEV_INVALID; + } else + return DEV_VALID; +} + int rte_eth_dev_socket_id(uint8_t port_id) { - if (port_id >= nb_ports) + if (rte_eth_dev_validate_port(port_id, NO_TRACE) == DEV_INVALID) return -1; return rte_eth_devices[port_id].pci_dev->numa_node; } @@ -369,10 +415,8 @@ rte_eth_dev_rx_queue_start(uint8_t port_id, uint16_t rx_queue_id) * in a multi-process setup*/ PROC_PRIMARY_OR_ERR_RET(-E_RTE_SECONDARY); - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return -EINVAL; - } dev = &rte_eth_devices[port_id]; if (rx_queue_id >= dev->data->nb_rx_queues) { @@ -395,10 +439,8 @@ rte_eth_dev_rx_queue_stop(uint8_t port_id, uint16_t rx_queue_id) * in a multi-process setup*/ PROC_PRIMARY_OR_ERR_RET(-E_RTE_SECONDARY); - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return -EINVAL; - } dev = &rte_eth_devices[port_id]; if (rx_queue_id >= dev->data->nb_rx_queues) { @@ -421,10 +463,8 @@ rte_eth_dev_tx_queue_start(uint8_t port_id, uint16_t tx_queue_id) * in a multi-process setup*/ PROC_PRIMARY_OR_ERR_RET(-E_RTE_SECONDARY); - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return -EINVAL; - } dev = &rte_eth_devices[port_id]; if (tx_queue_id >= dev->data->nb_tx_queues) { @@ -447,10 +487,8 @@ rte_eth_dev_tx_queue_stop(uint8_t port_id, uint16_t tx_queue_id) * in a multi-process setup*/ PROC_PRIMARY_OR_ERR_RET(-E_RTE_SECONDARY); - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return -EINVAL; - } dev = &rte_eth_devices[port_id]; if (tx_queue_id >= dev->data->nb_tx_queues) { @@ -703,10 +741,9 @@ rte_eth_dev_configure(uint8_t port_id, uint16_t nb_rx_q, uint16_t nb_tx_q, * in a multi-process setup*/ PROC_PRIMARY_OR_ERR_RET(-E_RTE_SECONDARY); - if (port_id >= nb_ports || port_id >= RTE_MAX_ETHPORTS) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return (-EINVAL); - } + dev = &rte_eth_devices[port_id]; FUNC_PTR_OR_ERR_RET(*dev->dev_ops->dev_infos_get, -ENOTSUP); @@ -888,10 +925,9 @@ rte_eth_dev_start(uint8_t port_id) * in a multi-process setup*/ PROC_PRIMARY_OR_ERR_RET(-E_RTE_SECONDARY); - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%" PRIu8 "\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return (-EINVAL); - } + dev = &rte_eth_devices[port_id]; FUNC_PTR_OR_ERR_RET(*dev->dev_ops->dev_start, -ENOTSUP); @@ -923,10 +959,9 @@ rte_eth_dev_stop(uint8_t port_id) * in a multi-process setup*/ PROC_PRIMARY_OR_RET(); - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%" PRIu8 "\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return; - } + dev = &rte_eth_devices[port_id]; FUNC_PTR_OR_RET(*dev->dev_ops->dev_stop); @@ -951,10 +986,9 @@ rte_eth_dev_set_link_up(uint8_t port_id) * in a multi-process setup*/ PROC_PRIMARY_OR_ERR_RET(-E_RTE_SECONDARY); - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return -EINVAL; - } + dev = &rte_eth_devices[port_id]; FUNC_PTR_OR_ERR_RET(*dev->dev_ops->dev_set_link_up, -ENOTSUP); @@ -970,10 +1004,9 @@ rte_eth_dev_set_link_down(uint8_t port_id) * in a multi-process setup*/ PROC_PRIMARY_OR_ERR_RET(-E_RTE_SECONDARY); - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return -EINVAL; - } + dev = &rte_eth_devices[port_id]; FUNC_PTR_OR_ERR_RET(*dev->dev_ops->dev_set_link_down, -ENOTSUP); @@ -989,10 +1022,8 @@ rte_eth_dev_close(uint8_t port_id) * in a multi-process setup*/ PROC_PRIMARY_OR_RET(); - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return; - } dev = &rte_eth_devices[port_id]; @@ -1017,10 +1048,9 @@ rte_eth_rx_queue_setup(uint8_t port_id, uint16_t rx_queue_id, * in a multi-process setup*/ PROC_PRIMARY_OR_ERR_RET(-E_RTE_SECONDARY); - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return (-EINVAL); - } + dev = &rte_eth_devices[port_id]; if (rx_queue_id >= dev->data->nb_rx_queues) { PMD_DEBUG_TRACE("Invalid RX queue_id=%d\n", rx_queue_id); @@ -1090,10 +1120,9 @@ rte_eth_tx_queue_setup(uint8_t port_id, uint16_t tx_queue_id, * in a multi-process setup*/ PROC_PRIMARY_OR_ERR_RET(-E_RTE_SECONDARY); - if (port_id >= RTE_MAX_ETHPORTS || port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return (-EINVAL); - } + dev = &rte_eth_devices[port_id]; if (tx_queue_id >= dev->data->nb_tx_queues) { PMD_DEBUG_TRACE("Invalid TX queue_id=%d\n", tx_queue_id); @@ -1123,10 +1152,9 @@ rte_eth_promiscuous_enable(uint8_t port_id) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return; - } + dev = &rte_eth_devices[port_id]; FUNC_PTR_OR_RET(*dev->dev_ops->promiscuous_enable); @@ -1139,10 +1167,9 @@ rte_eth_promiscuous_disable(uint8_t port_id) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return; - } + dev = &rte_eth_devices[port_id]; FUNC_PTR_OR_RET(*dev->dev_ops->promiscuous_disable); @@ -1155,10 +1182,8 @@ rte_eth_promiscuous_get(uint8_t port_id) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return -1; - } dev = &rte_eth_devices[port_id]; return dev->data->promiscuous; @@ -1169,10 +1194,9 @@ rte_eth_allmulticast_enable(uint8_t port_id) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return; - } + dev = &rte_eth_devices[port_id]; FUNC_PTR_OR_RET(*dev->dev_ops->allmulticast_enable); @@ -1185,10 +1209,9 @@ rte_eth_allmulticast_disable(uint8_t port_id) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return; - } + dev = &rte_eth_devices[port_id]; FUNC_PTR_OR_RET(*dev->dev_ops->allmulticast_disable); @@ -1201,10 +1224,8 @@ rte_eth_allmulticast_get(uint8_t port_id) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return -1; - } dev = &rte_eth_devices[port_id]; return dev->data->all_multicast; @@ -1229,10 +1250,9 @@ rte_eth_link_get(uint8_t port_id, struct rte_eth_link *eth_link) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return; - } + dev = &rte_eth_devices[port_id]; if (dev->data->dev_conf.intr_conf.lsc != 0) @@ -1249,10 +1269,9 @@ rte_eth_link_get_nowait(uint8_t port_id, struct rte_eth_link *eth_link) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return; - } + dev = &rte_eth_devices[port_id]; if (dev->data->dev_conf.intr_conf.lsc != 0) @@ -1269,10 +1288,9 @@ rte_eth_stats_get(uint8_t port_id, struct rte_eth_stats *stats) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return; - } + dev = &rte_eth_devices[port_id]; memset(stats, 0, sizeof(*stats)); @@ -1286,10 +1304,9 @@ rte_eth_stats_reset(uint8_t port_id) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return; - } + dev = &rte_eth_devices[port_id]; FUNC_PTR_OR_RET(*dev->dev_ops->stats_reset); @@ -1307,10 +1324,9 @@ rte_eth_xstats_get(uint8_t port_id, struct rte_eth_xstats *xstats, uint64_t val; char *stats_ptr; - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return -1; - } + dev = &rte_eth_devices[port_id]; /* implemented by the driver */ @@ -1376,10 +1392,9 @@ rte_eth_xstats_reset(uint8_t port_id) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return; - } + dev = &rte_eth_devices[port_id]; /* implemented by the driver */ @@ -1398,10 +1413,9 @@ set_queue_stats_mapping(uint8_t port_id, uint16_t queue_id, uint8_t stat_idx, { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return -ENODEV; - } + dev = &rte_eth_devices[port_id]; FUNC_PTR_OR_ERR_RET(*dev->dev_ops->queue_stats_mapping_set, -ENOTSUP); @@ -1433,10 +1447,9 @@ rte_eth_dev_info_get(uint8_t port_id, struct rte_eth_dev_info *dev_info) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return; - } + dev = &rte_eth_devices[port_id]; memset(dev_info, 0, sizeof(struct rte_eth_dev_info)); @@ -1453,10 +1466,9 @@ rte_eth_macaddr_get(uint8_t port_id, struct ether_addr *mac_addr) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return; - } + dev = &rte_eth_devices[port_id]; ether_addr_copy(&dev->data->mac_addrs[0], mac_addr); } @@ -1467,10 +1479,8 @@ rte_eth_dev_get_mtu(uint8_t port_id, uint16_t *mtu) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return (-ENODEV); - } dev = &rte_eth_devices[port_id]; *mtu = dev->data->mtu; @@ -1483,10 +1493,8 @@ rte_eth_dev_set_mtu(uint8_t port_id, uint16_t mtu) int ret; struct rte_eth_dev *dev; - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return (-ENODEV); - } dev = &rte_eth_devices[port_id]; FUNC_PTR_OR_ERR_RET(*dev->dev_ops->mtu_set, -ENOTSUP); @@ -1503,10 +1511,9 @@ rte_eth_dev_vlan_filter(uint8_t port_id, uint16_t vlan_id, int on) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return (-ENODEV); - } + dev = &rte_eth_devices[port_id]; if (! (dev->data->dev_conf.rxmode.hw_vlan_filter)) { PMD_DEBUG_TRACE("port %d: vlan-filtering disabled\n", port_id); @@ -1528,10 +1535,8 @@ rte_eth_dev_set_vlan_strip_on_queue(uint8_t port_id, uint16_t rx_queue_id, int o { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return (-ENODEV); - } dev = &rte_eth_devices[port_id]; if (rx_queue_id >= dev->data->nb_rx_queues) { @@ -1550,10 +1555,8 @@ rte_eth_dev_set_vlan_ether_type(uint8_t port_id, uint16_t tpid) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return (-ENODEV); - } dev = &rte_eth_devices[port_id]; FUNC_PTR_OR_ERR_RET(*dev->dev_ops->vlan_tpid_set, -ENOTSUP); @@ -1570,10 +1573,8 @@ rte_eth_dev_set_vlan_offload(uint8_t port_id, int offload_mask) int mask = 0; int cur, org = 0; - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return (-ENODEV); - } dev = &rte_eth_devices[port_id]; @@ -1615,10 +1616,8 @@ rte_eth_dev_get_vlan_offload(uint8_t port_id) struct rte_eth_dev *dev; int ret = 0; - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return (-ENODEV); - } dev = &rte_eth_devices[port_id]; @@ -1639,10 +1638,9 @@ rte_eth_dev_set_vlan_pvid(uint8_t port_id, uint16_t pvid, int on) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return (-ENODEV); - } + dev = &rte_eth_devices[port_id]; FUNC_PTR_OR_ERR_RET(*dev->dev_ops->vlan_pvid_set, -ENOTSUP); (*dev->dev_ops->vlan_pvid_set)(dev, pvid, on); @@ -1657,10 +1655,8 @@ rte_eth_dev_fdir_add_signature_filter(uint8_t port_id, { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return (-ENODEV); - } dev = &rte_eth_devices[port_id]; @@ -1691,10 +1687,8 @@ rte_eth_dev_fdir_update_signature_filter(uint8_t port_id, { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return (-ENODEV); - } dev = &rte_eth_devices[port_id]; @@ -1725,10 +1719,8 @@ rte_eth_dev_fdir_remove_signature_filter(uint8_t port_id, { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return (-ENODEV); - } dev = &rte_eth_devices[port_id]; @@ -1756,10 +1748,8 @@ rte_eth_dev_fdir_get_infos(uint8_t port_id, struct rte_eth_fdir *fdir) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return (-ENODEV); - } dev = &rte_eth_devices[port_id]; if (! (dev->data->dev_conf.fdir_conf.mode)) { @@ -1781,10 +1771,8 @@ rte_eth_dev_fdir_add_perfect_filter(uint8_t port_id, { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return (-ENODEV); - } dev = &rte_eth_devices[port_id]; @@ -1821,10 +1809,8 @@ rte_eth_dev_fdir_update_perfect_filter(uint8_t port_id, { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return (-ENODEV); - } dev = &rte_eth_devices[port_id]; @@ -1859,10 +1845,8 @@ rte_eth_dev_fdir_remove_perfect_filter(uint8_t port_id, { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return (-ENODEV); - } dev = &rte_eth_devices[port_id]; @@ -1895,10 +1879,8 @@ rte_eth_dev_fdir_set_masks(uint8_t port_id, struct rte_fdir_masks *fdir_mask) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return (-ENODEV); - } dev = &rte_eth_devices[port_id]; if (! (dev->data->dev_conf.fdir_conf.mode)) { @@ -1915,10 +1897,8 @@ rte_eth_dev_flow_ctrl_get(uint8_t port_id, struct rte_eth_fc_conf *fc_conf) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return (-ENODEV); - } dev = &rte_eth_devices[port_id]; FUNC_PTR_OR_ERR_RET(*dev->dev_ops->flow_ctrl_get, -ENOTSUP); @@ -1931,10 +1911,8 @@ rte_eth_dev_flow_ctrl_set(uint8_t port_id, struct rte_eth_fc_conf *fc_conf) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return (-ENODEV); - } if ((fc_conf->send_xon != 0) && (fc_conf->send_xon != 1)) { PMD_DEBUG_TRACE("Invalid send_xon, only 0/1 allowed\n"); @@ -1951,10 +1929,8 @@ rte_eth_dev_priority_flow_ctrl_set(uint8_t port_id, struct rte_eth_pfc_conf *pfc { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return (-ENODEV); - } if (pfc_conf->priority > (ETH_DCB_NUM_USER_PRIORITIES - 1)) { PMD_DEBUG_TRACE("Invalid priority, only 0-7 allowed\n"); @@ -2030,10 +2006,8 @@ rte_eth_dev_rss_reta_update(uint8_t port_id, struct rte_eth_dev *dev; int ret; - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return -ENODEV; - } /* Check mask bits */ ret = rte_eth_check_reta_mask(reta_conf, reta_size); @@ -2081,10 +2055,9 @@ rte_eth_dev_rss_hash_update(uint8_t port_id, struct rte_eth_rss_conf *rss_conf) struct rte_eth_dev *dev; uint16_t rss_hash_protos; - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return (-ENODEV); - } + rss_hash_protos = rss_conf->rss_hf; if ((rss_hash_protos != 0) && ((rss_hash_protos & ETH_RSS_PROTO_MASK) == 0)) { @@ -2103,10 +2076,9 @@ rte_eth_dev_rss_hash_conf_get(uint8_t port_id, { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return (-ENODEV); - } + dev = &rte_eth_devices[port_id]; FUNC_PTR_OR_ERR_RET(*dev->dev_ops->rss_hash_conf_get, -ENOTSUP); return (*dev->dev_ops->rss_hash_conf_get)(dev, rss_conf); @@ -2118,10 +2090,8 @@ rte_eth_dev_udp_tunnel_add(uint8_t port_id, { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return -ENODEV; - } if (udp_tunnel == NULL) { PMD_DEBUG_TRACE("Invalid udp_tunnel parameter\n"); @@ -2144,10 +2114,9 @@ rte_eth_dev_udp_tunnel_delete(uint8_t port_id, { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return -ENODEV; - } + dev = &rte_eth_devices[port_id]; if (udp_tunnel == NULL) { @@ -2169,10 +2138,8 @@ rte_eth_led_on(uint8_t port_id) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return (-ENODEV); - } dev = &rte_eth_devices[port_id]; FUNC_PTR_OR_ERR_RET(*dev->dev_ops->dev_led_on, -ENOTSUP); @@ -2184,10 +2151,8 @@ rte_eth_led_off(uint8_t port_id) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return (-ENODEV); - } dev = &rte_eth_devices[port_id]; FUNC_PTR_OR_ERR_RET(*dev->dev_ops->dev_led_off, -ENOTSUP); @@ -2224,10 +2189,9 @@ rte_eth_dev_mac_addr_add(uint8_t port_id, struct ether_addr *addr, int index; uint64_t pool_mask; - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return (-ENODEV); - } + dev = &rte_eth_devices[port_id]; FUNC_PTR_OR_ERR_RET(*dev->dev_ops->mac_addr_add, -ENOTSUP); @@ -2275,10 +2239,9 @@ rte_eth_dev_mac_addr_remove(uint8_t port_id, struct ether_addr *addr) struct rte_eth_dev *dev; int index; - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return (-ENODEV); - } + dev = &rte_eth_devices[port_id]; FUNC_PTR_OR_ERR_RET(*dev->dev_ops->mac_addr_remove, -ENOTSUP); @@ -2309,7 +2272,7 @@ rte_eth_dev_set_vf_rxmode(uint8_t port_id, uint16_t vf, struct rte_eth_dev *dev; struct rte_eth_dev_info dev_info; - if (port_id >= nb_ports) { + if (rte_eth_dev_validate_port(port_id, NO_TRACE) == DEV_INVALID) { PMD_DEBUG_TRACE("set VF RX mode:Invalid port_id=%d\n", port_id); return (-ENODEV); @@ -2364,7 +2327,7 @@ rte_eth_dev_uc_hash_table_set(uint8_t port_id, struct ether_addr *addr, int ret; struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (rte_eth_dev_validate_port(port_id, NO_TRACE) == DEV_INVALID) { PMD_DEBUG_TRACE("unicast hash setting:Invalid port_id=%d\n", port_id); return (-ENODEV); @@ -2417,7 +2380,7 @@ rte_eth_dev_uc_all_hash_table_set(uint8_t port_id, uint8_t on) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (rte_eth_dev_validate_port(port_id, NO_TRACE) == DEV_INVALID) { PMD_DEBUG_TRACE("unicast hash setting:Invalid port_id=%d\n", port_id); return (-ENODEV); @@ -2436,10 +2399,8 @@ rte_eth_dev_set_vf_rx(uint8_t port_id,uint16_t vf, uint8_t on) struct rte_eth_dev *dev; struct rte_eth_dev_info dev_info; - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return (-ENODEV); - } dev = &rte_eth_devices[port_id]; rte_eth_dev_info_get(port_id, &dev_info); @@ -2462,7 +2423,7 @@ rte_eth_dev_set_vf_tx(uint8_t port_id,uint16_t vf, uint8_t on) struct rte_eth_dev *dev; struct rte_eth_dev_info dev_info; - if (port_id >= nb_ports) { + if (rte_eth_dev_validate_port(port_id, NO_TRACE) == DEV_INVALID) { PMD_DEBUG_TRACE("set pool tx:Invalid port_id=%d\n", port_id); return (-ENODEV); } @@ -2487,7 +2448,7 @@ rte_eth_dev_set_vf_vlan_filter(uint8_t port_id, uint16_t vlan_id, { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (rte_eth_dev_validate_port(port_id, NO_TRACE) == DEV_INVALID) { PMD_DEBUG_TRACE("VF VLAN filter:invalid port id=%d\n", port_id); return (-ENODEV); @@ -2518,7 +2479,7 @@ int rte_eth_set_queue_rate_limit(uint8_t port_id, uint16_t queue_idx, struct rte_eth_dev_info dev_info; struct rte_eth_link link; - if (port_id >= nb_ports) { + if (rte_eth_dev_validate_port(port_id, NO_TRACE) == DEV_INVALID) { PMD_DEBUG_TRACE("set queue rate limit:invalid port id=%d\n", port_id); return -ENODEV; @@ -2555,7 +2516,7 @@ int rte_eth_set_vf_rate_limit(uint8_t port_id, uint16_t vf, uint16_t tx_rate, if (q_msk == 0) return 0; - if (port_id >= nb_ports) { + if (rte_eth_dev_validate_port(port_id, NO_TRACE) == DEV_INVALID) { PMD_DEBUG_TRACE("set VF rate limit:invalid port id=%d\n", port_id); return -ENODEV; @@ -2589,10 +2550,8 @@ rte_eth_mirror_rule_set(uint8_t port_id, { struct rte_eth_dev *dev = &rte_eth_devices[port_id]; - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return (-ENODEV); - } if (mirror_conf->rule_type_mask == 0) { PMD_DEBUG_TRACE("mirror rule type can not be 0.\n"); @@ -2630,10 +2589,8 @@ rte_eth_mirror_rule_reset(uint8_t port_id, uint8_t rule_id) { struct rte_eth_dev *dev = &rte_eth_devices[port_id]; - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return (-ENODEV); - } if(rule_id >= ETH_VMDQ_NUM_MIRROR_RULE) { @@ -2655,10 +2612,9 @@ rte_eth_rx_burst(uint8_t port_id, uint16_t queue_id, { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return 0; - } + dev = &rte_eth_devices[port_id]; FUNC_PTR_OR_ERR_RET(*dev->rx_pkt_burst, 0); if (queue_id >= dev->data->nb_rx_queues) { @@ -2675,10 +2631,9 @@ rte_eth_tx_burst(uint8_t port_id, uint16_t queue_id, { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return 0; - } + dev = &rte_eth_devices[port_id]; FUNC_PTR_OR_ERR_RET(*dev->tx_pkt_burst, 0); @@ -2695,10 +2650,9 @@ rte_eth_rx_queue_count(uint8_t port_id, uint16_t queue_id) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return 0; - } + dev = &rte_eth_devices[port_id]; FUNC_PTR_OR_ERR_RET(*dev->dev_ops->rx_queue_count, 0); return (*dev->dev_ops->rx_queue_count)(dev, queue_id); @@ -2709,10 +2663,9 @@ rte_eth_rx_descriptor_done(uint8_t port_id, uint16_t queue_id, uint16_t offset) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return (-ENODEV); - } + dev = &rte_eth_devices[port_id]; FUNC_PTR_OR_ERR_RET(*dev->dev_ops->rx_descriptor_done, -ENOTSUP); return (*dev->dev_ops->rx_descriptor_done)( \ @@ -2730,10 +2683,8 @@ rte_eth_dev_callback_register(uint8_t port_id, if (!cb_fn) return (-EINVAL); - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return (-EINVAL); - } dev = &rte_eth_devices[port_id]; rte_spinlock_lock(&rte_eth_dev_cb_lock); @@ -2770,10 +2721,8 @@ rte_eth_dev_callback_unregister(uint8_t port_id, if (!cb_fn) return (-EINVAL); - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return (-EINVAL); - } dev = &rte_eth_devices[port_id]; rte_spinlock_lock(&rte_eth_dev_cb_lock); @@ -2830,10 +2779,8 @@ int rte_eth_dev_bypass_init(uint8_t port_id) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return (-ENODEV); - } if ((dev= &rte_eth_devices[port_id]) == NULL) { PMD_DEBUG_TRACE("Invalid port device\n"); @@ -2850,10 +2797,8 @@ rte_eth_dev_bypass_state_show(uint8_t port_id, uint32_t *state) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return (-ENODEV); - } if ((dev= &rte_eth_devices[port_id]) == NULL) { PMD_DEBUG_TRACE("Invalid port device\n"); @@ -2869,10 +2814,8 @@ rte_eth_dev_bypass_state_set(uint8_t port_id, uint32_t *new_state) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return (-ENODEV); - } if ((dev= &rte_eth_devices[port_id]) == NULL) { PMD_DEBUG_TRACE("Invalid port device\n"); @@ -2889,10 +2832,8 @@ rte_eth_dev_bypass_event_show(uint8_t port_id, uint32_t event, uint32_t *state) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return (-ENODEV); - } if ((dev= &rte_eth_devices[port_id]) == NULL) { PMD_DEBUG_TRACE("Invalid port device\n"); @@ -2909,10 +2850,8 @@ rte_eth_dev_bypass_event_store(uint8_t port_id, uint32_t event, uint32_t state) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return (-ENODEV); - } if ((dev= &rte_eth_devices[port_id]) == NULL) { PMD_DEBUG_TRACE("Invalid port device\n"); @@ -2929,10 +2868,8 @@ rte_eth_dev_wd_timeout_store(uint8_t port_id, uint32_t timeout) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return (-ENODEV); - } if ((dev= &rte_eth_devices[port_id]) == NULL) { PMD_DEBUG_TRACE("Invalid port device\n"); @@ -2949,10 +2886,8 @@ rte_eth_dev_bypass_ver_show(uint8_t port_id, uint32_t *ver) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return (-ENODEV); - } if ((dev= &rte_eth_devices[port_id]) == NULL) { PMD_DEBUG_TRACE("Invalid port device\n"); @@ -2969,10 +2904,8 @@ rte_eth_dev_bypass_wd_timeout_show(uint8_t port_id, uint32_t *wd_timeout) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return (-ENODEV); - } if ((dev= &rte_eth_devices[port_id]) == NULL) { PMD_DEBUG_TRACE("Invalid port device\n"); @@ -2989,10 +2922,8 @@ rte_eth_dev_bypass_wd_reset(uint8_t port_id) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return (-ENODEV); - } if ((dev= &rte_eth_devices[port_id]) == NULL) { PMD_DEBUG_TRACE("Invalid port device\n"); @@ -3011,10 +2942,8 @@ rte_eth_dev_add_syn_filter(uint8_t port_id, { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return -ENODEV; - } dev = &rte_eth_devices[port_id]; FUNC_PTR_OR_ERR_RET(*dev->dev_ops->add_syn_filter, -ENOTSUP); @@ -3026,10 +2955,8 @@ rte_eth_dev_remove_syn_filter(uint8_t port_id) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return -ENODEV; - } dev = &rte_eth_devices[port_id]; FUNC_PTR_OR_ERR_RET(*dev->dev_ops->remove_syn_filter, -ENOTSUP); @@ -3045,10 +2972,8 @@ rte_eth_dev_get_syn_filter(uint8_t port_id, if (filter == NULL || rx_queue == NULL) return -EINVAL; - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return -ENODEV; - } dev = &rte_eth_devices[port_id]; FUNC_PTR_OR_ERR_RET(*dev->dev_ops->get_syn_filter, -ENOTSUP); @@ -3061,10 +2986,9 @@ rte_eth_dev_add_2tuple_filter(uint8_t port_id, uint16_t index, { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return -ENODEV; - } + if (filter->protocol != IPPROTO_TCP && filter->tcp_flags != 0){ PMD_DEBUG_TRACE("tcp flags is 0x%x, but the protocol value" @@ -3083,10 +3007,8 @@ rte_eth_dev_remove_2tuple_filter(uint8_t port_id, uint16_t index) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return -ENODEV; - } dev = &rte_eth_devices[port_id]; FUNC_PTR_OR_ERR_RET(*dev->dev_ops->remove_2tuple_filter, -ENOTSUP); @@ -3102,10 +3024,8 @@ rte_eth_dev_get_2tuple_filter(uint8_t port_id, uint16_t index, if (filter == NULL || rx_queue == NULL) return -EINVAL; - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return -ENODEV; - } dev = &rte_eth_devices[port_id]; FUNC_PTR_OR_ERR_RET(*dev->dev_ops->get_2tuple_filter, -ENOTSUP); @@ -3118,10 +3038,8 @@ rte_eth_dev_add_5tuple_filter(uint8_t port_id, uint16_t index, { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return -ENODEV; - } if (filter->protocol != IPPROTO_TCP && filter->tcp_flags != 0){ @@ -3141,10 +3059,8 @@ rte_eth_dev_remove_5tuple_filter(uint8_t port_id, uint16_t index) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return -ENODEV; - } dev = &rte_eth_devices[port_id]; FUNC_PTR_OR_ERR_RET(*dev->dev_ops->remove_5tuple_filter, -ENOTSUP); @@ -3160,10 +3076,8 @@ rte_eth_dev_get_5tuple_filter(uint8_t port_id, uint16_t index, if (filter == NULL || rx_queue == NULL) return -EINVAL; - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return -ENODEV; - } dev = &rte_eth_devices[port_id]; FUNC_PTR_OR_ERR_RET(*dev->dev_ops->get_5tuple_filter, -ENOTSUP); @@ -3177,10 +3091,8 @@ rte_eth_dev_add_flex_filter(uint8_t port_id, uint16_t index, { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return -ENODEV; - } dev = &rte_eth_devices[port_id]; FUNC_PTR_OR_ERR_RET(*dev->dev_ops->add_flex_filter, -ENOTSUP); @@ -3192,10 +3104,8 @@ rte_eth_dev_remove_flex_filter(uint8_t port_id, uint16_t index) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return -ENODEV; - } dev = &rte_eth_devices[port_id]; FUNC_PTR_OR_ERR_RET(*dev->dev_ops->remove_flex_filter, -ENOTSUP); @@ -3211,10 +3121,8 @@ rte_eth_dev_get_flex_filter(uint8_t port_id, uint16_t index, if (filter == NULL || rx_queue == NULL) return -EINVAL; - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return -ENODEV; - } dev = &rte_eth_devices[port_id]; FUNC_PTR_OR_ERR_RET(*dev->dev_ops->get_flex_filter, -ENOTSUP); @@ -3227,10 +3135,8 @@ rte_eth_dev_filter_supported(uint8_t port_id, enum rte_filter_type filter_type) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return -ENODEV; - } dev = &rte_eth_devices[port_id]; FUNC_PTR_OR_ERR_RET(*dev->dev_ops->filter_ctrl, -ENOTSUP); @@ -3244,10 +3150,8 @@ rte_eth_dev_filter_ctrl(uint8_t port_id, enum rte_filter_type filter_type, { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) return -ENODEV; - } dev = &rte_eth_devices[port_id]; FUNC_PTR_OR_ERR_RET(*dev->dev_ops->filter_ctrl, -ENOTSUP); diff --git a/lib/librte_ether/rte_ethdev.h b/lib/librte_ether/rte_ethdev.h index 1200c1c..ca101f5 100644 --- a/lib/librte_ether/rte_ethdev.h +++ b/lib/librte_ether/rte_ethdev.h @@ -1539,6 +1539,7 @@ struct rte_eth_dev { struct eth_dev_ops *dev_ops; /**< Functions exported by PMD */ struct rte_pci_device *pci_dev; /**< PCI info. supplied by probing */ struct rte_eth_dev_cb_list callbacks; /**< User application callbacks */ + uint8_t attached; /**< Flag indicating the port is attached */ }; struct rte_eth_dev_sriov { @@ -1604,6 +1605,10 @@ extern struct rte_eth_dev rte_eth_devices[]; * initialized by the [matching] Ethernet driver during the PCI probing phase. * All devices whose port identifier is in the range * [0, rte_eth_dev_count() - 1] can be operated on by network applications. + * immediately after invoking rte_eal_init(). + * If the application unplugs a port using hotplug function, The enabled port + * numbers may be noncontiguous. In the case, the applications need to manage + * enabled port by themselves. * * @return * - The total number of usable Ethernet devices. -- 1.9.1
This patch replaces pci_addr_comparison() and memcmp() of pci addresses by eal_compare_pci_addr(). v8: - Fix pci_scan_one() to update sysfs values. (Thanks to Qiu, Michael and Iremonger, Bernard) v5: - Fix pci_scan_one to handle pt_driver correctly. v4: - Fix calculation method of eal_compare_pci_addr(). - Add parameter checking. Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp> --- lib/librte_eal/bsdapp/eal/eal_pci.c | 29 ++++++++++++-------------- lib/librte_eal/common/eal_common_pci.c | 2 +- lib/librte_eal/common/include/rte_pci.h | 34 +++++++++++++++++++++++++++++++ lib/librte_eal/linuxapp/eal/eal_pci.c | 30 +++++++++++++-------------- lib/librte_eal/linuxapp/eal/eal_pci_uio.c | 2 +- 5 files changed, 63 insertions(+), 34 deletions(-) diff --git a/lib/librte_eal/bsdapp/eal/eal_pci.c b/lib/librte_eal/bsdapp/eal/eal_pci.c index 74ecce7..7dbdccd 100644 --- a/lib/librte_eal/bsdapp/eal/eal_pci.c +++ b/lib/librte_eal/bsdapp/eal/eal_pci.c @@ -270,20 +270,6 @@ pci_uio_map_resource(struct rte_pci_device *dev) return (0); } -/* Compare two PCI device addresses. */ -static int -pci_addr_comparison(struct rte_pci_addr *addr, struct rte_pci_addr *addr2) -{ - uint64_t dev_addr = (addr->domain << 24) + (addr->bus << 16) + (addr->devid << 8) + addr->function; - uint64_t dev_addr2 = (addr2->domain << 24) + (addr2->bus << 16) + (addr2->devid << 8) + addr2->function; - - if (dev_addr > dev_addr2) - return 1; - else - return 0; -} - - /* Scan one pci sysfs entry, and fill the devices list from it. */ static int pci_scan_one(int dev_pci_fd, struct pci_conf *conf) @@ -356,13 +342,24 @@ pci_scan_one(int dev_pci_fd, struct pci_conf *conf) } else { struct rte_pci_device *dev2 = NULL; + int ret; TAILQ_FOREACH(dev2, &pci_device_list, next) { - if (pci_addr_comparison(&dev->addr, &dev2->addr)) + ret = eal_compare_pci_addr(&dev->addr, &dev2->addr); + if (ret > 0) continue; - else { + else if (ret < 0) { TAILQ_INSERT_BEFORE(dev2, dev, next); return 0; + } else { /* already registered */ + /* update pt_driver */ + dev2->pt_driver = dev->pt_driver; + dev2->max_vfs = dev->max_vfs; + memmove(dev2->mem_resource, + dev->mem_resource, + sizeof(dev->mem_resource)); + free(dev); + return 0; } } TAILQ_INSERT_TAIL(&pci_device_list, dev, next); diff --git a/lib/librte_eal/common/eal_common_pci.c b/lib/librte_eal/common/eal_common_pci.c index f3c7f71..a89f5c3 100644 --- a/lib/librte_eal/common/eal_common_pci.c +++ b/lib/librte_eal/common/eal_common_pci.c @@ -93,7 +93,7 @@ static struct rte_devargs *pci_devargs_lookup(struct rte_pci_device *dev) if (devargs->type != RTE_DEVTYPE_BLACKLISTED_PCI && devargs->type != RTE_DEVTYPE_WHITELISTED_PCI) continue; - if (!memcmp(&dev->addr, &devargs->pci.addr, sizeof(dev->addr))) + if (!eal_compare_pci_addr(&dev->addr, &devargs->pci.addr)) return devargs; } return NULL; diff --git a/lib/librte_eal/common/include/rte_pci.h b/lib/librte_eal/common/include/rte_pci.h index 7f2d699..4814cd7 100644 --- a/lib/librte_eal/common/include/rte_pci.h +++ b/lib/librte_eal/common/include/rte_pci.h @@ -269,6 +269,40 @@ eal_parse_pci_DomBDF(const char *input, struct rte_pci_addr *dev_addr) } #undef GET_PCIADDR_FIELD +/* Compare two PCI device addresses. */ +/** + * Utility function to compare two PCI device addresses. + * + * @param addr + * The PCI Bus-Device-Function address to compare + * @param addr2 + * The PCI Bus-Device-Function address to compare + * @return + * 0 on equal PCI address. + * Positive on addr is greater than addr2. + * Negative on addr is less than addr2, or error. + */ +static inline int +eal_compare_pci_addr(struct rte_pci_addr *addr, struct rte_pci_addr *addr2) +{ + uint64_t dev_addr, dev_addr2; + + if ((addr == NULL) || (addr2 == NULL)) + return -1; + + dev_addr = (addr->domain << 24) | (addr->bus << 16) | + (addr->devid << 8) | addr->function; + dev_addr2 = (addr2->domain << 24) | (addr2->bus << 16) | + (addr2->devid << 8) | addr2->function; + + if (dev_addr > dev_addr2) + return 1; + else if (dev_addr < dev_addr2) + return -1; + else + return 0; +} + /** * Probe the PCI bus for registered drivers. * diff --git a/lib/librte_eal/linuxapp/eal/eal_pci.c b/lib/librte_eal/linuxapp/eal/eal_pci.c index 3c463b2..2d5f6a6 100644 --- a/lib/librte_eal/linuxapp/eal/eal_pci.c +++ b/lib/librte_eal/linuxapp/eal/eal_pci.c @@ -41,6 +41,7 @@ #include <rte_eal_memconfig.h> #include <rte_malloc.h> #include <rte_devargs.h> +#include <rte_memcpy.h> #include "rte_pci_dev_ids.h" #include "eal_filesystem.h" @@ -229,20 +230,6 @@ error: return -1; } -/* Compare two PCI device addresses. */ -static int -pci_addr_comparison(struct rte_pci_addr *addr, struct rte_pci_addr *addr2) -{ - uint64_t dev_addr = (addr->domain << 24) + (addr->bus << 16) + (addr->devid << 8) + addr->function; - uint64_t dev_addr2 = (addr2->domain << 24) + (addr2->bus << 16) + (addr2->devid << 8) + addr2->function; - - if (dev_addr > dev_addr2) - return 1; - else - return 0; -} - - /* Scan one pci sysfs entry, and fill the devices list from it. */ static int pci_scan_one(const char *dirname, uint16_t domain, uint8_t bus, @@ -360,13 +347,24 @@ pci_scan_one(const char *dirname, uint16_t domain, uint8_t bus, } else { struct rte_pci_device *dev2 = NULL; + int ret; TAILQ_FOREACH(dev2, &pci_device_list, next) { - if (pci_addr_comparison(&dev->addr, &dev2->addr)) + ret = eal_compare_pci_addr(&dev->addr, &dev2->addr); + if (ret > 0) continue; - else { + else if (ret < 0) { TAILQ_INSERT_BEFORE(dev2, dev, next); return 0; + } else { /* already registered */ + /* update pt_driver */ + dev2->pt_driver = dev->pt_driver; + dev2->max_vfs = dev->max_vfs; + memmove(dev2->mem_resource, + dev->mem_resource, + sizeof(dev->mem_resource)); + free(dev); + return 0; } } TAILQ_INSERT_TAIL(&pci_device_list, dev, next); diff --git a/lib/librte_eal/linuxapp/eal/eal_pci_uio.c b/lib/librte_eal/linuxapp/eal/eal_pci_uio.c index e53f06b..1da3507 100644 --- a/lib/librte_eal/linuxapp/eal/eal_pci_uio.c +++ b/lib/librte_eal/linuxapp/eal/eal_pci_uio.c @@ -123,7 +123,7 @@ pci_uio_map_secondary(struct rte_pci_device *dev) TAILQ_FOREACH(uio_res, pci_res_list, next) { /* skip this element if it doesn't match our PCI address */ - if (memcmp(&uio_res->pci_addr, &dev->addr, sizeof(dev->addr))) + if (eal_compare_pci_addr(&uio_res->pci_addr, &dev->addr)) continue; for (i = 0; i != uio_res->nb_maps; i++) { -- 1.9.1
This patch adds rte_eth_dev_free(). The function is used for changing an attached status of the device that has specified name. v6: - Use rte_eth_dev structure as the paramter of rte_eth_dev_free(). v4: - Add parameter checking. Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp> --- lib/librte_ether/rte_ethdev.c | 11 +++++++++++ lib/librte_ether/rte_ethdev.h | 14 ++++++++++++++ 2 files changed, 25 insertions(+) diff --git a/lib/librte_ether/rte_ethdev.c b/lib/librte_ether/rte_ethdev.c index a79fa5b..2463d18 100644 --- a/lib/librte_ether/rte_ethdev.c +++ b/lib/librte_ether/rte_ethdev.c @@ -260,6 +260,17 @@ rte_eth_dev_allocate(const char *name) return eth_dev; } +int +rte_eth_dev_free(struct rte_eth_dev *eth_dev) +{ + if (eth_dev == NULL) + return -EINVAL; + + eth_dev->attached = 0; + nb_ports--; + return 0; +} + static int rte_eth_dev_init(struct rte_pci_driver *pci_drv, struct rte_pci_device *pci_dev) diff --git a/lib/librte_ether/rte_ethdev.h b/lib/librte_ether/rte_ethdev.h index ca101f5..fbe7ac1 100644 --- a/lib/librte_ether/rte_ethdev.h +++ b/lib/librte_ether/rte_ethdev.h @@ -1627,6 +1627,20 @@ extern uint8_t rte_eth_dev_count(void); */ struct rte_eth_dev *rte_eth_dev_allocate(const char *name); +/** + * Function for internal use by dummy drivers primarily, e.g. ring-based + * driver. + * Free the specified ethdev. + * + * @param eth_dev + * The *eth_dev* pointer is the address of the *rte_eth_dev* structure + * associated with the matching device and which have been [automatically] + * allocated in the *rte_eth_devices* array. + * @return + * - 0 on success, negative on error + */ +int rte_eth_dev_free(struct rte_eth_dev *eth_dev); + struct eth_driver; /** * @internal -- 1.9.1
The patch adds function pointer to rte_pci_driver and eth_driver structure. These function pointers are used when ports are detached. Also, the patch adds rte_eth_dev_uninit(). So far, it's not called by anywhere, but it will be called when port hotplug function is implemented. v6: - Fix rte_eth_dev_uninit() to handle a return value of uninit function of PMD. v4: - Add parameter checking. - Change function names. Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp> --- lib/librte_eal/common/include/rte_pci.h | 7 +++++ lib/librte_ether/rte_ethdev.c | 47 +++++++++++++++++++++++++++++++++ lib/librte_ether/rte_ethdev.h | 24 +++++++++++++++++ 3 files changed, 78 insertions(+) diff --git a/lib/librte_eal/common/include/rte_pci.h b/lib/librte_eal/common/include/rte_pci.h index 4814cd7..87ca4cf 100644 --- a/lib/librte_eal/common/include/rte_pci.h +++ b/lib/librte_eal/common/include/rte_pci.h @@ -189,12 +189,19 @@ struct rte_pci_driver; typedef int (pci_devinit_t)(struct rte_pci_driver *, struct rte_pci_device *); /** + * Uninitialisation function for the driver called during hotplugging. + */ +typedef int (pci_devuninit_t)( + struct rte_pci_driver *, struct rte_pci_device *); + +/** * A structure describing a PCI driver. */ struct rte_pci_driver { TAILQ_ENTRY(rte_pci_driver) next; /**< Next in list. */ const char *name; /**< Driver name. */ pci_devinit_t *devinit; /**< Device init. function. */ + pci_devuninit_t *devuninit; /**< Device uninit function. */ struct rte_pci_id *id_table; /**< ID table, NULL terminated. */ uint32_t drv_flags; /**< Flags contolling handling of device. */ }; diff --git a/lib/librte_ether/rte_ethdev.c b/lib/librte_ether/rte_ethdev.c index 2463d18..58d8072 100644 --- a/lib/librte_ether/rte_ethdev.c +++ b/lib/librte_ether/rte_ethdev.c @@ -326,6 +326,52 @@ rte_eth_dev_init(struct rte_pci_driver *pci_drv, return diag; } +static int +rte_eth_dev_uninit(struct rte_pci_driver *pci_drv, + struct rte_pci_device *pci_dev) +{ + struct eth_driver *eth_drv; + struct rte_eth_dev *eth_dev; + char ethdev_name[RTE_ETH_NAME_MAX_LEN]; + int ret; + + if ((pci_drv == NULL) || (pci_dev == NULL)) + return -EINVAL; + + /* Create unique Ethernet device name using PCI address */ + snprintf(ethdev_name, RTE_ETH_NAME_MAX_LEN, "%d:%d.%d", + pci_dev->addr.bus, pci_dev->addr.devid, + pci_dev->addr.function); + + eth_dev = rte_eth_dev_allocated(ethdev_name); + if (eth_dev == NULL) + return -ENODEV; + + eth_drv = (struct eth_driver *)pci_drv; + + /* Invoke PMD device uninit function */ + if (*eth_drv->eth_dev_uninit) { + ret = (*eth_drv->eth_dev_uninit)(eth_drv, eth_dev); + if (ret) + return ret; + } + + /* free ether device */ + rte_eth_dev_free(eth_dev); + + /* init user callbacks */ + TAILQ_INIT(&(eth_dev->callbacks)); + + if (rte_eal_process_type() == RTE_PROC_PRIMARY) + rte_free(eth_dev->data->dev_private); + + eth_dev->pci_dev = NULL; + eth_dev->driver = NULL; + eth_dev->data = NULL; + + return 0; +} + /** * Register an Ethernet [Poll Mode] driver. * @@ -344,6 +390,7 @@ void rte_eth_driver_register(struct eth_driver *eth_drv) { eth_drv->pci_drv.devinit = rte_eth_dev_init; + eth_drv->pci_drv.devuninit = rte_eth_dev_uninit; rte_eal_pci_register(ð_drv->pci_drv); } diff --git a/lib/librte_ether/rte_ethdev.h b/lib/librte_ether/rte_ethdev.h index fbe7ac1..91d9e86 100644 --- a/lib/librte_ether/rte_ethdev.h +++ b/lib/librte_ether/rte_ethdev.h @@ -1678,6 +1678,27 @@ typedef int (*eth_dev_init_t)(struct eth_driver *eth_drv, /** * @internal + * Finalization function of an Ethernet driver invoked for each matching + * Ethernet PCI device detected during the PCI closing phase. + * + * @param eth_drv + * The pointer to the [matching] Ethernet driver structure supplied by + * the PMD when it registered itself. + * @param eth_dev + * The *eth_dev* pointer is the address of the *rte_eth_dev* structure + * associated with the matching device and which have been [automatically] + * allocated in the *rte_eth_devices* array. + * @return + * - 0: Success, the device is properly finalized by the driver. + * In particular, the driver MUST free the *dev_ops* pointer + * of the *eth_dev* structure. + * - <0: Error code of the device initialization failure. + */ +typedef int (*eth_dev_uninit_t)(struct eth_driver *eth_drv, + struct rte_eth_dev *eth_dev); + +/** + * @internal * The structure associated with a PMD Ethernet driver. * * Each Ethernet driver acts as a PCI driver and is represented by a generic @@ -1687,11 +1708,14 @@ typedef int (*eth_dev_init_t)(struct eth_driver *eth_drv, * * - The *eth_dev_init* function invoked for each matching PCI device. * + * - The *eth_dev_uninit* function invoked for each matching PCI device. + * * - The size of the private data to allocate for each matching device. */ struct eth_driver { struct rte_pci_driver pci_drv; /**< The PMD is also a PCI driver. */ eth_dev_init_t eth_dev_init; /**< Device init function. */ + eth_dev_uninit_t eth_dev_uninit; /**< Device uninit function. */ unsigned int dev_private_size; /**< Size of device private data. */ }; -- 1.9.1
The patch adds following functions. - rte_eth_dev_save() The function is used for saving current rte_eth_dev structures. - rte_eth_dev_get_changed_port() The function receives the rte_eth_dev structures, then compare these with current values to know which port is actually attached or detached. - rte_eth_dev_get_addr_by_port() The function returns a pci address of an ethdev specified by port identifier. - rte_eth_dev_get_port_by_addr() The function returns a port identifier of an ethdev specified by pci address. - rte_eth_dev_get_name_by_port() The function returns a unique identifier name of an ethdev specified by port identifier. - Add rte_eth_dev_check_detachable() The function returns whether a PMD supports detach function. Also, the patch changes scope of rte_eth_dev_allocated() to global. This function will be called by virtual PMDs to support port hotplug. So change scope of the function to global. v8: - Add size parameter to rte_eth_dev_save(). - Add missing symbol in version map. (Thanks to Qiu, Michael and Iremonger, Bernard) v7: - Add pt_driver checking to rte_eth_dev_check_detachable(). (Thanks to Qiu, Michael) v5: - Fix return value of below functions. rte_eth_dev_get_changed_port(). rte_eth_dev_get_port_by_addr(). v4: - Add parameter checking. v3: - Fix if-condition bug while comparing pci addresses. - Add error checking codes. Reported-by: Mark Enright <menrigh@brocade.com> Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp> --- lib/librte_ether/rte_ethdev.c | 99 +++++++++++++++++++++++++++++++++- lib/librte_ether/rte_ethdev.h | 83 ++++++++++++++++++++++++++++ lib/librte_ether/rte_ether_version.map | 6 +++ 3 files changed, 187 insertions(+), 1 deletion(-) diff --git a/lib/librte_ether/rte_ethdev.c b/lib/librte_ether/rte_ethdev.c index 58d8072..3869a96 100644 --- a/lib/librte_ether/rte_ethdev.c +++ b/lib/librte_ether/rte_ethdev.c @@ -206,7 +206,7 @@ rte_eth_dev_data_alloc(void) RTE_MAX_ETHPORTS * sizeof(*rte_eth_dev_data)); } -static struct rte_eth_dev * +struct rte_eth_dev * rte_eth_dev_allocated(const char *name) { unsigned i; @@ -426,6 +426,103 @@ rte_eth_dev_count(void) return (nb_ports); } +int +rte_eth_dev_save(struct rte_eth_dev *devs, size_t size) +{ + if ((devs == NULL) || + (size != sizeof(struct rte_eth_dev) * RTE_MAX_ETHPORTS)) + return -EINVAL; + + /* save current rte_eth_devices */ + memcpy(devs, rte_eth_devices, size); + return 0; +} + +int +rte_eth_dev_get_changed_port(struct rte_eth_dev *devs, uint8_t *port_id) +{ + if ((devs == NULL) || (port_id == NULL)) + return -EINVAL; + + /* check which port was attached or detached */ + for (*port_id = 0; *port_id < RTE_MAX_ETHPORTS; (*port_id)++, devs++) { + if (rte_eth_devices[*port_id].attached ^ devs->attached) + return 0; + } + return -ENODEV; +} + +int +rte_eth_dev_get_addr_by_port(uint8_t port_id, struct rte_pci_addr *addr) +{ + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) + return -EINVAL; + + if (addr == NULL) { + PMD_DEBUG_TRACE("Null pointer is specified\n"); + return -EINVAL; + } + + *addr = rte_eth_devices[port_id].pci_dev->addr; + return 0; +} + +int +rte_eth_dev_get_port_by_addr(struct rte_pci_addr *addr, uint8_t *port_id) +{ + struct rte_pci_addr *tmp; + + if ((addr == NULL) || (port_id == NULL)) { + PMD_DEBUG_TRACE("Null pointer is specified\n"); + return -EINVAL; + } + + for (*port_id = 0; *port_id < RTE_MAX_ETHPORTS; (*port_id)++) { + if (!rte_eth_devices[*port_id].attached) + continue; + if (!rte_eth_devices[*port_id].pci_dev) + continue; + tmp = &rte_eth_devices[*port_id].pci_dev->addr; + if (eal_compare_pci_addr(tmp, addr) == 0) + return 0; + } + return -ENODEV; +} + +int +rte_eth_dev_get_name_by_port(uint8_t port_id, char *name) +{ + char *tmp; + + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) + return -EINVAL; + + if (name == NULL) { + PMD_DEBUG_TRACE("Null pointer is specified\n"); + return -EINVAL; + } + + /* shouldn't check 'rte_eth_devices[i].data', + * because it might be overwritten by VDEV PMD */ + tmp = rte_eth_dev_data[port_id].name; + strncpy(name, tmp, strlen(tmp) + 1); + return 0; +} + +int +rte_eth_dev_check_detachable(uint8_t port_id) +{ + uint32_t drv_flags; + + if (port_id >= RTE_MAX_ETHPORTS) { + PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + return -EINVAL; + } + + drv_flags = rte_eth_devices[port_id].driver->pci_drv.drv_flags; + return !(drv_flags & RTE_PCI_DRV_DETACHABLE); +} + static int rte_eth_dev_rx_queue_config(struct rte_eth_dev *dev, uint16_t nb_queues) { diff --git a/lib/librte_ether/rte_ethdev.h b/lib/librte_ether/rte_ethdev.h index 91d9e86..6651890 100644 --- a/lib/librte_ether/rte_ethdev.h +++ b/lib/librte_ether/rte_ethdev.h @@ -1616,6 +1616,89 @@ extern struct rte_eth_dev rte_eth_devices[]; extern uint8_t rte_eth_dev_count(void); /** + * Function for internal use by port hotplug functions. + * Copies current ethdev structures to the specified pointer. + * + * @param devs The pointer to the ethdev structures + * @param size The size of ethdev structures + * @return + * - 0 on success, negative on error + */ +extern int rte_eth_dev_save(struct rte_eth_dev *devs, size_t size); + +/** + * Function for internal use by port hotplug functions. + * Compare the specified ethdev structures with currents. Then + * if there is a port which status is changed, fill the specified pointer + * with the port id of that port. + * @param devs The pointer to the ethdev structures + * @param port_id The pointer to the port id + * @return + * - 0 on success, negative on error + */ +extern int rte_eth_dev_get_changed_port( + struct rte_eth_dev *devs, uint8_t *port_id); + +/** + * Function for internal use by port hotplug functions. + * Returns a pci address of a ethdev specified by port identifier. + * @param port_id + * The port identifier of the Ethernet device + * @param addr + * The pointer to the pci address + * @return + * - 0 on success, negative on error + */ +extern int rte_eth_dev_get_addr_by_port( + uint8_t port_id, struct rte_pci_addr *addr); + +/** + * Function for internal use by port hotplug functions. + * Returns a port identifier of a ethdev specified by pci address. + * @param addr + * The pointer to the pci address of the Ethernet device. + * @param port_id + * The pointer to the port identifier + * @return + * - 0 on success, negative on error + */ +extern int rte_eth_dev_get_port_by_addr( + struct rte_pci_addr *addr, uint8_t *port_id); + +/** + * Function for internal use by port hotplug functions. + * Returns a unique identifier name of a ethdev specified by port identifier. + * @param port_id + * The port identifier. + * @param name + * The pointer to the Unique identifier name for each Ethernet device + * @return + * - 0 on success, negative on error + */ +extern int rte_eth_dev_get_name_by_port(uint8_t port_id, char *name); + +/** + * Function for internal use by port hotplug functions. + * Check whether or not, a PMD that is handling the ethdev specified by port + * identifier can support detach function. + * @param port_id + * The port identifier + * @return + * - 0 on supporting detach function, negative on not supporting + */ +extern int rte_eth_dev_check_detachable(uint8_t port_id); + +/** + * Function for internal use by port hotplug functions. + * Returns a ethdev slot specified by the unique identifier name. + * @param name + * The pointer to the Unique identifier name for each Ethernet device + * @return + * - The pointer to the ethdev slot, on success. NULL on error + */ +extern struct rte_eth_dev *rte_eth_dev_allocated(const char *name); + +/** * Function for internal use by dummy drivers primarily, e.g. ring-based * driver. * Allocates a new ethdev slot for an ethernet device and returns the pointer diff --git a/lib/librte_ether/rte_ether_version.map b/lib/librte_ether/rte_ether_version.map index 7316530..fc5dc27 100644 --- a/lib/librte_ether/rte_ether_version.map +++ b/lib/librte_ether/rte_ether_version.map @@ -109,6 +109,12 @@ DPDK_2.0 { rte_eth_tx_queue_setup; rte_eth_xstats_get; rte_eth_xstats_reset; + rte_eth_dev_check_detachable; + rte_eth_dev_get_name_by_port; + rte_eth_dev_get_addr_by_port; + rte_eth_dev_get_port_by_addr; + rte_eth_dev_get_changed_port; + rte_eth_dev_save; local: *; }; -- 1.9.1
The patch adds functions for unmapping igb_uio resources. The patch is only for Linux and igb_uio environment. VFIO and BSD are not supported. v8: - Fix typo. (Thanks to Iremonger, Bernard) v5: - Fix pci_unmap_device() to check pt_driver. v4: - Add parameter checking. - Add header file to determine if hotplug can be enabled. Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp> --- lib/librte_eal/common/Makefile | 1 + lib/librte_eal/common/include/rte_dev_hotplug.h | 44 +++++++++++++++++ lib/librte_eal/linuxapp/eal/eal_pci.c | 44 +++++++++++++++++ lib/librte_eal/linuxapp/eal/eal_pci_init.h | 8 +++ lib/librte_eal/linuxapp/eal/eal_pci_uio.c | 65 +++++++++++++++++++++++++ 5 files changed, 162 insertions(+) create mode 100644 lib/librte_eal/common/include/rte_dev_hotplug.h diff --git a/lib/librte_eal/common/Makefile b/lib/librte_eal/common/Makefile index 52c1a5f..db7cc93 100644 --- a/lib/librte_eal/common/Makefile +++ b/lib/librte_eal/common/Makefile @@ -41,6 +41,7 @@ INC += rte_eal_memconfig.h rte_malloc_heap.h INC += rte_hexdump.h rte_devargs.h rte_dev.h INC += rte_common_vect.h INC += rte_pci_dev_feature_defs.h rte_pci_dev_features.h +INC += rte_dev_hotplug.h ifeq ($(CONFIG_RTE_INSECURE_FUNCTION_WARNING),y) INC += rte_warnings.h diff --git a/lib/librte_eal/common/include/rte_dev_hotplug.h b/lib/librte_eal/common/include/rte_dev_hotplug.h new file mode 100644 index 0000000..b333e0f --- /dev/null +++ b/lib/librte_eal/common/include/rte_dev_hotplug.h @@ -0,0 +1,44 @@ +/*- + * BSD LICENSE + * + * Copyright(c) 2015 IGEL Co.,LTd. + * All rights reserved. + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * + * * Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * * Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in + * the documentation and/or other materials provided with the + * distribution. + * * Neither the name of IGEL Co.,Ltd. nor the names of its + * contributors may be used to endorse or promote products derived + * from this software without specific prior written permission. + * + * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS + * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT + * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR + * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT + * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, + * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT + * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, + * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY + * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT + * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE + * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + */ + +#ifndef _RTE_DEV_HOTPLUG_H_ +#define _RTE_DEV_HOTPLUG_H_ + +/* + * determine if hotplug can be enabled on the system + */ +#if defined(RTE_LIBRTE_EAL_HOTPLUG) && defined(RTE_LIBRTE_EAL_LINUXAPP) +#define ENABLE_HOTPLUG +#endif /* RTE_LIBRTE_EAL_HOTPLUG & RTE_LIBRTE_EAL_LINUXAPP */ + +#endif /* _RTE_DEV_HOTPLUG_H_ */ diff --git a/lib/librte_eal/linuxapp/eal/eal_pci.c b/lib/librte_eal/linuxapp/eal/eal_pci.c index 2d5f6a6..72a1362 100644 --- a/lib/librte_eal/linuxapp/eal/eal_pci.c +++ b/lib/librte_eal/linuxapp/eal/eal_pci.c @@ -167,6 +167,25 @@ pci_map_resource(void *requested_addr, int fd, off_t offset, size_t size) return mapaddr; } +#ifdef ENABLE_HOTPLUG +/* unmap a particular resource */ +void +pci_unmap_resource(void *requested_addr, size_t size) +{ + if (requested_addr == NULL) + return; + + /* Unmap the PCI memory resource of device */ + if (munmap(requested_addr, size)) { + RTE_LOG(ERR, EAL, "%s(): cannot munmap(%p, 0x%lx): %s\n", + __func__, requested_addr, (unsigned long)size, + strerror(errno)); + } else + RTE_LOG(DEBUG, EAL, " PCI memory unmapped at %p\n", + requested_addr); +} +#endif /* ENABLE_HOTPLUG */ + /* parse the "resource" sysfs file */ #define IORESOURCE_MEM 0x00000200 @@ -579,6 +598,31 @@ pci_map_device(struct rte_pci_device *dev) return ret; } +#ifdef ENABLE_HOTPLUG +static void +pci_unmap_device(struct rte_pci_device *dev) +{ + if (dev == NULL) + return; + + /* try unmapping the NIC resources using VFIO if it exists */ + switch (dev->pt_driver) { + case RTE_PT_VFIO: + RTE_LOG(ERR, EAL, "Hotplug doesn't support vfio yet\n"); + break; + case RTE_PT_IGB_UIO: + case RTE_PT_UIO_GENERIC: + /* unmap resources for devices that use uio */ + pci_uio_unmap_resource(dev); + break; + default: + RTE_LOG(DEBUG, EAL, " Not managed by known pt driver," + " skipped\n"); + break; + } +} +#endif /* ENABLE_HOTPLUG */ + /* * If vendor/device ID match, call the devinit() function of the * driver. diff --git a/lib/librte_eal/linuxapp/eal/eal_pci_init.h b/lib/librte_eal/linuxapp/eal/eal_pci_init.h index 1070eb8..5152a0b 100644 --- a/lib/librte_eal/linuxapp/eal/eal_pci_init.h +++ b/lib/librte_eal/linuxapp/eal/eal_pci_init.h @@ -34,6 +34,7 @@ #ifndef EAL_PCI_INIT_H_ #define EAL_PCI_INIT_H_ +#include <rte_dev_hotplug.h> #include "eal_vfio.h" struct pci_map { @@ -71,6 +72,13 @@ void *pci_map_resource(void *requested_addr, int fd, off_t offset, /* map IGB_UIO resource prototype */ int pci_uio_map_resource(struct rte_pci_device *dev); +#ifdef ENABLE_HOTPLUG +void pci_unmap_resource(void *requested_addr, size_t size); + +/* unmap IGB_UIO resource prototype */ +void pci_uio_unmap_resource(struct rte_pci_device *dev); +#endif /* ENABLE_HOTPLUG */ + #ifdef VFIO_PRESENT #define VFIO_MAX_GROUPS 64 diff --git a/lib/librte_eal/linuxapp/eal/eal_pci_uio.c b/lib/librte_eal/linuxapp/eal/eal_pci_uio.c index 1da3507..81830d1 100644 --- a/lib/librte_eal/linuxapp/eal/eal_pci_uio.c +++ b/lib/librte_eal/linuxapp/eal/eal_pci_uio.c @@ -404,6 +404,71 @@ pci_uio_map_resource(struct rte_pci_device *dev) return 0; } +#ifdef ENABLE_HOTPLUG +static void +pci_uio_unmap(struct mapped_pci_resource *uio_res) +{ + int i; + + if (uio_res == NULL) + return; + + for (i = 0; i != uio_res->nb_maps; i++) + pci_unmap_resource(uio_res->maps[i].addr, + (size_t)uio_res->maps[i].size); +} + +static struct mapped_pci_resource * +pci_uio_find_resource(struct rte_pci_device *dev) +{ + struct mapped_pci_resource *uio_res; + + if (dev == NULL) + return NULL; + + TAILQ_FOREACH(uio_res, pci_res_list, next) { + + /* skip this element if it doesn't match our PCI address */ + if (!eal_compare_pci_addr(&uio_res->pci_addr, &dev->addr)) + return uio_res; + } + return NULL; +} + +/* unmap the PCI resource of a PCI device in virtual memory */ +void +pci_uio_unmap_resource(struct rte_pci_device *dev) +{ + struct mapped_pci_resource *uio_res; + + if (dev == NULL) + return; + + /* find an entry for the device */ + uio_res = pci_uio_find_resource(dev); + if (uio_res == NULL) + return; + + /* secondary processes - just free maps */ + if (rte_eal_process_type() != RTE_PROC_PRIMARY) + return pci_uio_unmap(uio_res); + + TAILQ_REMOVE(pci_res_list, uio_res, next); + + /* unmap all resources */ + pci_uio_unmap(uio_res); + + /* free uio resource */ + rte_free(uio_res); + + /* close fd if in primary process */ + close(dev->intr_handle.fd); + + dev->intr_handle.fd = -1; + dev->intr_handle.type = RTE_INTR_HANDLE_UNKNOWN; +} +#endif /* ENABLE_HOTPLUG */ + /* * parse a sysfs file containing one integer value * different to the eal version, as it needs to work with 64-bit values -- 1.9.1
The function removes the specified devargs entry from devargs_list. Also, the patch adds sanity checking to rte_eal_devargs_add(). v5: - Change function definition of rte_eal_devargs_remove(). v4: - Fix sanity check code. Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp> --- lib/librte_eal/common/eal_common_devargs.c | 60 +++++++++++++++++++++++++++++ lib/librte_eal/common/include/rte_devargs.h | 21 ++++++++++ 2 files changed, 81 insertions(+) diff --git a/lib/librte_eal/common/eal_common_devargs.c b/lib/librte_eal/common/eal_common_devargs.c index 4c7d11a..5b1ac8e 100644 --- a/lib/librte_eal/common/eal_common_devargs.c +++ b/lib/librte_eal/common/eal_common_devargs.c @@ -44,6 +44,35 @@ struct rte_devargs_list devargs_list = TAILQ_HEAD_INITIALIZER(devargs_list); + +/* find a entry specified by pci address or device name */ +static struct rte_devargs * +rte_eal_devargs_find(enum rte_devtype devtype, void *args) +{ + struct rte_devargs *devargs; + + if (args == NULL) + return NULL; + + TAILQ_FOREACH(devargs, &devargs_list, next) { + switch (devtype) { + case RTE_DEVTYPE_WHITELISTED_PCI: + case RTE_DEVTYPE_BLACKLISTED_PCI: + if (eal_compare_pci_addr(&devargs->pci.addr, args) == 0) + goto found; + break; + case RTE_DEVTYPE_VIRTUAL: + if (memcmp(&devargs->virtual.drv_name, args, + strlen((char *)args)) == 0) + goto found; + break; + } + } + return NULL; +found: + return devargs; +} + /* store a whitelist parameter for later parsing */ int rte_eal_devargs_add(enum rte_devtype devtype, const char *devargs_str) @@ -87,6 +116,12 @@ rte_eal_devargs_add(enum rte_devtype devtype, const char *devargs_str) free(devargs); return -1; } + /* make sure there is no same entry */ + if (rte_eal_devargs_find(devtype, &devargs->pci.addr)) { + RTE_LOG(ERR, EAL, + "device already registered: <%s>\n", buf); + return -1; + } break; case RTE_DEVTYPE_VIRTUAL: /* save driver name */ @@ -98,6 +133,12 @@ rte_eal_devargs_add(enum rte_devtype devtype, const char *devargs_str) free(devargs); return -1; } + /* make sure there is no same entry */ + if (rte_eal_devargs_find(devtype, &devargs->virtual.drv_name)) { + RTE_LOG(ERR, EAL, + "device already registered: <%s>\n", buf); + return -1; + } break; } @@ -105,6 +146,25 @@ rte_eal_devargs_add(enum rte_devtype devtype, const char *devargs_str) return 0; } +/* remove it from the devargs_list */ +int +rte_eal_devargs_remove(enum rte_devtype devtype, void *args) +{ + struct rte_devargs *devargs; + + if (args == NULL) + return -EINVAL; + + devargs = rte_eal_devargs_find(devtype, args); + if (devargs == NULL) { + RTE_LOG(ERR, EAL, "device not found\n"); + return -ENODEV; + } + + TAILQ_REMOVE(&devargs_list, devargs, next); + return 0; +} + /* count the number of devices of a specified type */ unsigned int rte_eal_devargs_type_count(enum rte_devtype devtype) diff --git a/lib/librte_eal/common/include/rte_devargs.h b/lib/librte_eal/common/include/rte_devargs.h index 9f9c98f..6d9763b 100644 --- a/lib/librte_eal/common/include/rte_devargs.h +++ b/lib/librte_eal/common/include/rte_devargs.h @@ -123,6 +123,27 @@ extern struct rte_devargs_list devargs_list; int rte_eal_devargs_add(enum rte_devtype devtype, const char *devargs_str); /** + * Remove a device from the user device list + * + * For PCI devices, the format of arguments string is "PCI_ADDR". It shouldn't + * involve parameters for the device. Example: "08:00.1". + * + * For virtual devices, the format of arguments string is "DRIVER_NAME*". It + * shouldn't involve parameters for the device. Example: "eth_ring". The + * validity of the driver name is not checked by this function, it is done + * when closing the drivers. + * + * @param devtype + * The type of the device. + * @param name + * The name of the device. + * + * @return + * - 0 on success, negative on error + */ +int rte_eal_devargs_remove(enum rte_devtype devtype, void *args); + +/** * Count the number of user devices of a specified type * * @param devtype -- 1.9.1
- Add rte_eal_pci_close_one_dirver() The function is used for closing the specified driver and device. - Add pci_invoke_all_drivers() The function is based on pci_probe_all_drivers. But it can not only probe but also close drivers. - Add pci_close_all_drivers() The function tries to find a driver for the specified device, and then close the driver. - Add rte_eal_pci_probe_one() and rte_eal_pci_close_one() The functions are used for probe and close a device. First the function tries to find a device that has the specified PCI address. Then, probe or close the device. v5: - Remove RTE_EAL_INVOKE_TYPE_UNKNOWN, because it's unused. v4: - Fix parameter checking. - Fix indent of 'if' statement. Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp> --- lib/librte_eal/common/eal_common_pci.c | 90 +++++++++++++++++++++++++++++---- lib/librte_eal/common/eal_private.h | 24 +++++++++ lib/librte_eal/common/include/rte_pci.h | 33 ++++++++++++ lib/librte_eal/linuxapp/eal/eal_pci.c | 69 +++++++++++++++++++++++++ 4 files changed, 206 insertions(+), 10 deletions(-) diff --git a/lib/librte_eal/common/eal_common_pci.c b/lib/librte_eal/common/eal_common_pci.c index a89f5c3..7c9b8c5 100644 --- a/lib/librte_eal/common/eal_common_pci.c +++ b/lib/librte_eal/common/eal_common_pci.c @@ -99,19 +99,27 @@ static struct rte_devargs *pci_devargs_lookup(struct rte_pci_device *dev) return NULL; } -/* - * If vendor/device ID match, call the devinit() function of all - * registered driver for the given device. Return -1 if initialization - * failed, return 1 if no driver is found for this device. - */ static int -pci_probe_all_drivers(struct rte_pci_device *dev) +pci_invoke_all_drivers(struct rte_pci_device *dev, + enum rte_eal_invoke_type type) { struct rte_pci_driver *dr = NULL; - int rc; + int rc = 0; + + if ((dev == NULL) || (type >= RTE_EAL_INVOKE_TYPE_MAX)) + return -1; TAILQ_FOREACH(dr, &pci_driver_list, next) { - rc = rte_eal_pci_probe_one_driver(dr, dev); + switch (type) { + case RTE_EAL_INVOKE_TYPE_PROBE: + rc = rte_eal_pci_probe_one_driver(dr, dev); + break; + case RTE_EAL_INVOKE_TYPE_CLOSE: + rc = rte_eal_pci_close_one_driver(dr, dev); + break; + default: + return -1; + } if (rc < 0) /* negative value is an error */ return -1; @@ -123,6 +131,66 @@ pci_probe_all_drivers(struct rte_pci_device *dev) return 1; } +#ifdef ENABLE_HOTPLUG +static int +rte_eal_pci_invoke_one(struct rte_pci_addr *addr, + enum rte_eal_invoke_type type) +{ + struct rte_pci_device *dev = NULL; + int ret = 0; + + if ((addr == NULL) || (type >= RTE_EAL_INVOKE_TYPE_MAX)) + return -1; + + TAILQ_FOREACH(dev, &pci_device_list, next) { + if (eal_compare_pci_addr(&dev->addr, addr)) + continue; + + ret = pci_invoke_all_drivers(dev, type); + if (ret < 0) + goto invoke_err_return; + + if (type == RTE_EAL_INVOKE_TYPE_CLOSE) + goto remove_dev; + + return 0; + } + + return -1; + +invoke_err_return: + RTE_LOG(WARNING, EAL, "Requested device " PCI_PRI_FMT + " cannot be used\n", dev->addr.domain, dev->addr.bus, + dev->addr.devid, dev->addr.function); + return -1; + +remove_dev: + TAILQ_REMOVE(&pci_device_list, dev, next); + return 0; +} + + +/* + * Find the pci device specified by pci address, then invoke probe function of + * the driver of the devive. + */ +int +rte_eal_pci_probe_one(struct rte_pci_addr *addr) +{ + return rte_eal_pci_invoke_one(addr, RTE_EAL_INVOKE_TYPE_PROBE); +} + +/* + * Find the pci device specified by pci address, then invoke close function of + * the driver of the devive. + */ +int +rte_eal_pci_close_one(struct rte_pci_addr *addr) +{ + return rte_eal_pci_invoke_one(addr, RTE_EAL_INVOKE_TYPE_CLOSE); +} +#endif /* ENABLE_HOTPLUG */ + /* * Scan the content of the PCI bus, and call the devinit() function for * all registered drivers that have a matching entry in its id_table @@ -148,10 +216,12 @@ rte_eal_pci_probe(void) /* probe all or only whitelisted devices */ if (probe_all) - ret = pci_probe_all_drivers(dev); + ret = pci_invoke_all_drivers(dev, + RTE_EAL_INVOKE_TYPE_PROBE); else if (devargs != NULL && devargs->type == RTE_DEVTYPE_WHITELISTED_PCI) - ret = pci_probe_all_drivers(dev); + ret = pci_invoke_all_drivers(dev, + RTE_EAL_INVOKE_TYPE_PROBE); if (ret < 0) rte_exit(EXIT_FAILURE, "Requested device " PCI_PRI_FMT " cannot be used\n", dev->addr.domain, dev->addr.bus, diff --git a/lib/librte_eal/common/eal_private.h b/lib/librte_eal/common/eal_private.h index 159cd66..1a362ab 100644 --- a/lib/librte_eal/common/eal_private.h +++ b/lib/librte_eal/common/eal_private.h @@ -154,6 +154,15 @@ struct rte_pci_driver; struct rte_pci_device; /** + * The invoke type. + */ +enum rte_eal_invoke_type { + RTE_EAL_INVOKE_TYPE_PROBE, /**< invoke probe function */ + RTE_EAL_INVOKE_TYPE_CLOSE, /**< invoke close function */ + RTE_EAL_INVOKE_TYPE_MAX /**< max value of this enum */ +}; + +/** * Mmap memory for single PCI device * * This function is private to EAL. @@ -165,6 +174,21 @@ int rte_eal_pci_probe_one_driver(struct rte_pci_driver *dr, struct rte_pci_device *dev); /** + * Munmap memory for single PCI device + * + * This function is private to EAL. + * + * @param dr + * The pointer to the pci driver structure + * @param dev + * The pointer to the pci device structure + * @return + * 0 on success, negative on error + */ +int rte_eal_pci_close_one_driver(struct rte_pci_driver *dr, + struct rte_pci_device *dev); + +/** * Init tail queues for non-EAL library structures. This is to allow * the rings, mempools, etc. lists to be shared among multiple processes * diff --git a/lib/librte_eal/common/include/rte_pci.h b/lib/librte_eal/common/include/rte_pci.h index 87ca4cf..a111066 100644 --- a/lib/librte_eal/common/include/rte_pci.h +++ b/lib/librte_eal/common/include/rte_pci.h @@ -82,6 +82,7 @@ extern "C" { #include <inttypes.h> #include <rte_interrupts.h> +#include <rte_dev_hotplug.h> TAILQ_HEAD(pci_device_list, rte_pci_device); /**< PCI devices in D-linked Q. */ TAILQ_HEAD(pci_driver_list, rte_pci_driver); /**< PCI drivers in D-linked Q. */ @@ -323,6 +324,38 @@ eal_compare_pci_addr(struct rte_pci_addr *addr, struct rte_pci_addr *addr2) */ int rte_eal_pci_probe(void); +#ifdef ENABLE_HOTPLUG +/** + * Probe the single PCI device. + * + * Scan the content of the PCI bus, and find the pci device specified by pci + * address, then call the probe() function for registered driver that has a + * matching entry in its id_table for discovered device. + * + * @param addr + * The PCI Bus-Device-Function address to probe or close. + * @return + * - 0 on success. + * - Negative on error. + */ +int rte_eal_pci_probe_one(struct rte_pci_addr *addr); + +/** + * Close the single PCI device. + * + * Scan the content of the PCI bus, and find the pci device specified by pci + * address, then call the close() function for registered driver that has a + * matching entry in its id_table for discovered device. + * + * @param addr + * The PCI Bus-Device-Function address to probe or close. + * @return + * - 0 on success. + * - Negative on error. + */ +int rte_eal_pci_close_one(struct rte_pci_addr *addr); +#endif /* ENABLE_HOTPLUG */ + /** * Dump the content of the PCI bus. * diff --git a/lib/librte_eal/linuxapp/eal/eal_pci.c b/lib/librte_eal/linuxapp/eal/eal_pci.c index 72a1362..0adb217 100644 --- a/lib/librte_eal/linuxapp/eal/eal_pci.c +++ b/lib/librte_eal/linuxapp/eal/eal_pci.c @@ -694,6 +694,75 @@ rte_eal_pci_probe_one_driver(struct rte_pci_driver *dr, struct rte_pci_device *d return 1; } +#ifdef ENABLE_HOTPLUG +/* + * If vendor/device ID match, call the devuninit() function of the + * driver. + */ +int +rte_eal_pci_close_one_driver(struct rte_pci_driver *dr, + struct rte_pci_device *dev) +{ + struct rte_pci_id *id_table; + + if ((dr == NULL) || (dev == NULL)) + return -EINVAL; + + for (id_table = dr->id_table ; id_table->vendor_id != 0; id_table++) { + + /* check if device's identifiers match the driver's ones */ + if (id_table->vendor_id != dev->id.vendor_id && + id_table->vendor_id != PCI_ANY_ID) + continue; + if (id_table->device_id != dev->id.device_id && + id_table->device_id != PCI_ANY_ID) + continue; + if (id_table->subsystem_vendor_id != + dev->id.subsystem_vendor_id && + id_table->subsystem_vendor_id != PCI_ANY_ID) + continue; + if (id_table->subsystem_device_id != + dev->id.subsystem_device_id && + id_table->subsystem_device_id != PCI_ANY_ID) + continue; + + struct rte_pci_addr *loc = &dev->addr; + + RTE_LOG(DEBUG, EAL, + "PCI device "PCI_PRI_FMT" on NUMA socket %i\n", + loc->domain, loc->bus, loc->devid, + loc->function, dev->numa_node); + + RTE_LOG(DEBUG, EAL, " remove driver: %x:%x %s\n", + dev->id.vendor_id, dev->id.device_id, + dr->name); + + /* call the driver devuninit() function */ + if (dr->devuninit && (dr->devuninit(dr, dev) < 0)) + return -1; /* negative value is an error */ + + /* clear driver structure */ + dev->driver = NULL; + + if (dr->drv_flags & RTE_PCI_DRV_NEED_MAPPING) + /* unmap resources for devices that use igb_uio */ + pci_unmap_device(dev); + + return 0; + } + /* return positive value if driver is not found */ + return 1; +} +#else /* ENABLE_HOTPLUG */ +int +rte_eal_pci_close_one_driver(struct rte_pci_driver *dr __rte_unused, + struct rte_pci_device *dev __rte_unused) +{ + RTE_LOG(ERR, EAL, "Hotplug support isn't enabled\n"); + return -1; +} +#endif /* ENABLE_HOTPLUG */ + /* Init the PCI EAL subsystem */ int rte_eal_pci_init(void) -- 1.9.1
This new parameter is needed to keep device type like physical or virtual. Port detaching processes are different between physical and virtual. This parameter lets detaching function know a device type of the port. v8: - NONE_TRACE is replaced by NO_TRACE. - Add missing symbol in version map. (Thanks to Qiu, Michael and Iremonger, Bernard) v4: - Fix comments of rte_eth_dev_type. Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp> --- app/test/virtual_pmd.c | 2 +- lib/librte_ether/rte_ethdev.c | 25 +++++++++++++++++++++++-- lib/librte_ether/rte_ethdev.h | 25 ++++++++++++++++++++++++- lib/librte_ether/rte_ether_version.map | 1 + lib/librte_pmd_af_packet/rte_eth_af_packet.c | 2 +- lib/librte_pmd_bond/rte_eth_bond_api.c | 2 +- lib/librte_pmd_pcap/rte_eth_pcap.c | 2 +- lib/librte_pmd_ring/rte_eth_ring.c | 2 +- lib/librte_pmd_xenvirt/rte_eth_xenvirt.c | 2 +- 9 files changed, 54 insertions(+), 9 deletions(-) diff --git a/app/test/virtual_pmd.c b/app/test/virtual_pmd.c index 9fac95d..8d3a5ff 100644 --- a/app/test/virtual_pmd.c +++ b/app/test/virtual_pmd.c @@ -556,7 +556,7 @@ virtual_ethdev_create(const char *name, struct ether_addr *mac_addr, goto err; /* reserve an ethdev entry */ - eth_dev = rte_eth_dev_allocate(name); + eth_dev = rte_eth_dev_allocate(name, RTE_ETH_DEV_PHYSICAL); if (eth_dev == NULL) goto err; diff --git a/lib/librte_ether/rte_ethdev.c b/lib/librte_ether/rte_ethdev.c index 3869a96..70c4589 100644 --- a/lib/librte_ether/rte_ethdev.c +++ b/lib/librte_ether/rte_ethdev.c @@ -232,7 +232,7 @@ rte_eth_dev_allocate_new_port(void) } struct rte_eth_dev * -rte_eth_dev_allocate(const char *name) +rte_eth_dev_allocate(const char *name, enum rte_eth_dev_type type) { uint8_t port_id; struct rte_eth_dev *eth_dev; @@ -256,6 +256,7 @@ rte_eth_dev_allocate(const char *name) snprintf(eth_dev->data->name, sizeof(eth_dev->data->name), "%s", name); eth_dev->data->port_id = port_id; eth_dev->attached = DEV_CONNECTED; + eth_dev->dev_type = type; nb_ports++; return eth_dev; } @@ -267,6 +268,7 @@ rte_eth_dev_free(struct rte_eth_dev *eth_dev) return -EINVAL; eth_dev->attached = 0; + eth_dev->dev_type = RTE_ETH_DEV_UNKNOWN; nb_ports--; return 0; } @@ -287,7 +289,7 @@ rte_eth_dev_init(struct rte_pci_driver *pci_drv, snprintf(ethdev_name, RTE_ETH_NAME_MAX_LEN, "%d:%d.%d", pci_dev->addr.bus, pci_dev->addr.devid, pci_dev->addr.function); - eth_dev = rte_eth_dev_allocate(ethdev_name); + eth_dev = rte_eth_dev_allocate(ethdev_name, RTE_ETH_DEV_PHYSICAL); if (eth_dev == NULL) return -ENOMEM; @@ -426,6 +428,14 @@ rte_eth_dev_count(void) return (nb_ports); } +enum rte_eth_dev_type +rte_eth_dev_get_device_type(uint8_t port_id) +{ + if (rte_eth_dev_validate_port(port_id, NO_TRACE) == DEV_INVALID) + return -1; + return rte_eth_devices[port_id].dev_type; +} + int rte_eth_dev_save(struct rte_eth_dev *devs, size_t size) { @@ -519,6 +529,17 @@ rte_eth_dev_check_detachable(uint8_t port_id) return -EINVAL; } + if (rte_eth_devices[port_id].dev_type == RTE_ETH_DEV_PHYSICAL) { + switch (rte_eth_devices[port_id].pci_dev->pt_driver) { + case RTE_PT_IGB_UIO: + case RTE_PT_UIO_GENERIC: + break; + case RTE_PT_VFIO: + default: + return -ENOTSUP; + } + } + drv_flags = rte_eth_devices[port_id].driver->pci_drv.drv_flags; return !(drv_flags & RTE_PCI_DRV_DETACHABLE); } diff --git a/lib/librte_ether/rte_ethdev.h b/lib/librte_ether/rte_ethdev.h index 6651890..caf0e8a 100644 --- a/lib/librte_ether/rte_ethdev.h +++ b/lib/librte_ether/rte_ethdev.h @@ -1522,6 +1522,17 @@ struct eth_dev_ops { }; /** + * The eth device type + */ +enum rte_eth_dev_type { + RTE_ETH_DEV_UNKNOWN, /**< unknown device type */ + RTE_ETH_DEV_PHYSICAL, + /**< Physical function and Virtual function devices of NIC */ + RTE_ETH_DEV_VIRTUAL, /**< non hardware device */ + RTE_ETH_DEV_MAX /**< max value of this enum */ +}; + +/** * @internal * The generic data structure associated with each ethernet device. * @@ -1540,6 +1551,7 @@ struct rte_eth_dev { struct rte_pci_device *pci_dev; /**< PCI info. supplied by probing */ struct rte_eth_dev_cb_list callbacks; /**< User application callbacks */ uint8_t attached; /**< Flag indicating the port is attached */ + enum rte_eth_dev_type dev_type; /**< Flag indicating the device type */ }; struct rte_eth_dev_sriov { @@ -1617,6 +1629,15 @@ extern uint8_t rte_eth_dev_count(void); /** * Function for internal use by port hotplug functions. + * Get the device type to know whether the device is physical or virtual. + * @param port_id The pointer to the port id + * @return + * - Device type. + */ +extern enum rte_eth_dev_type rte_eth_dev_get_device_type(uint8_t port_id); + +/** + * Function for internal use by port hotplug functions. * Copies current ethdev structures to the specified pointer. * * @param devs The pointer to the ethdev structures @@ -1705,10 +1726,12 @@ extern struct rte_eth_dev *rte_eth_dev_allocated(const char *name); * to that slot for the driver to use. * * @param name Unique identifier name for each Ethernet device + * @param type Device type of this Ethernet device * @return * - Slot in the rte_dev_devices array for a new device; */ -struct rte_eth_dev *rte_eth_dev_allocate(const char *name); +struct rte_eth_dev *rte_eth_dev_allocate(const char *name, + enum rte_eth_dev_type type); /** * Function for internal use by dummy drivers primarily, e.g. ring-based diff --git a/lib/librte_ether/rte_ether_version.map b/lib/librte_ether/rte_ether_version.map index fc5dc27..c145856 100644 --- a/lib/librte_ether/rte_ether_version.map +++ b/lib/librte_ether/rte_ether_version.map @@ -115,6 +115,7 @@ DPDK_2.0 { rte_eth_dev_get_port_by_addr; rte_eth_dev_get_changed_port; rte_eth_dev_save; + rte_eth_dev_get_device_type; local: *; }; diff --git a/lib/librte_pmd_af_packet/rte_eth_af_packet.c b/lib/librte_pmd_af_packet/rte_eth_af_packet.c index 1ffe1cd..80e9bdf 100644 --- a/lib/librte_pmd_af_packet/rte_eth_af_packet.c +++ b/lib/librte_pmd_af_packet/rte_eth_af_packet.c @@ -649,7 +649,7 @@ rte_pmd_init_internals(const char *name, } /* reserve an ethdev entry */ - *eth_dev = rte_eth_dev_allocate(name); + *eth_dev = rte_eth_dev_allocate(name, RTE_ETH_DEV_VIRTUAL); if (*eth_dev == NULL) goto error; diff --git a/lib/librte_pmd_bond/rte_eth_bond_api.c b/lib/librte_pmd_bond/rte_eth_bond_api.c index 4ab3267..7a6a5f7 100644 --- a/lib/librte_pmd_bond/rte_eth_bond_api.c +++ b/lib/librte_pmd_bond/rte_eth_bond_api.c @@ -235,7 +235,7 @@ rte_eth_bond_create(const char *name, uint8_t mode, uint8_t socket_id) } /* reserve an ethdev entry */ - eth_dev = rte_eth_dev_allocate(name); + eth_dev = rte_eth_dev_allocate(name, RTE_ETH_DEV_VIRTUAL); if (eth_dev == NULL) { RTE_BOND_LOG(ERR, "Unable to allocate rte_eth_dev"); goto err; diff --git a/lib/librte_pmd_pcap/rte_eth_pcap.c b/lib/librte_pmd_pcap/rte_eth_pcap.c index d299288..af7fae8 100644 --- a/lib/librte_pmd_pcap/rte_eth_pcap.c +++ b/lib/librte_pmd_pcap/rte_eth_pcap.c @@ -709,7 +709,7 @@ rte_pmd_init_internals(const char *name, const unsigned nb_rx_queues, goto error; /* reserve an ethdev entry */ - *eth_dev = rte_eth_dev_allocate(name); + *eth_dev = rte_eth_dev_allocate(name, RTE_ETH_DEV_VIRTUAL); if (*eth_dev == NULL) goto error; diff --git a/lib/librte_pmd_ring/rte_eth_ring.c b/lib/librte_pmd_ring/rte_eth_ring.c index d5b1686..c837fcd 100644 --- a/lib/librte_pmd_ring/rte_eth_ring.c +++ b/lib/librte_pmd_ring/rte_eth_ring.c @@ -255,7 +255,7 @@ rte_eth_from_rings(const char *name, struct rte_ring *const rx_queues[], goto error; /* reserve an ethdev entry */ - eth_dev = rte_eth_dev_allocate(name); + eth_dev = rte_eth_dev_allocate(name, RTE_ETH_DEV_VIRTUAL); if (eth_dev == NULL) goto error; diff --git a/lib/librte_pmd_xenvirt/rte_eth_xenvirt.c b/lib/librte_pmd_xenvirt/rte_eth_xenvirt.c index 04e30c9..bc403d6 100644 --- a/lib/librte_pmd_xenvirt/rte_eth_xenvirt.c +++ b/lib/librte_pmd_xenvirt/rte_eth_xenvirt.c @@ -648,7 +648,7 @@ eth_dev_xenvirt_create(const char *name, const char *params, goto err; /* reserve an ethdev entry */ - eth_dev = rte_eth_dev_allocate(name); + eth_dev = rte_eth_dev_allocate(name, RTE_ETH_DEV_VIRTUAL); if (eth_dev == NULL) goto err; -- 1.9.1
These functions are used for attaching or detaching a port. When rte_eal_dev_attach() is called, the function tries to realize the device name as pci address. If this is done successfully, rte_eal_dev_attach() will attach physical device port. If not, attaches virtual devive port. When rte_eal_dev_detach() is called, the function gets the device type of this port to know whether the port is come from physical or virtual. And then specific detaching function will be called. v8: - Add missing symbol in version map. (Thanks to Qiu, Michael and Iremonger, Bernard) v7: - Fix typo of warning messages. (Thanks to Qiu, Michael) v5: - Change function names like below. rte_eal_dev_find_and_invoke() to rte_eal_vdev_find_and_invoke(). rte_eal_dev_invoke() to rte_eal_vdev_invoke(). - Add code to handle a return value of rte_eal_devargs_remove(). - Fix pci address format in rte_eal_dev_detach(). v4: - Fix comment. - Add error checking. - Fix indent of 'if' statement. - Change function name. Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp> --- lib/librte_eal/common/eal_common_dev.c | 276 ++++++++++++++++++++++++ lib/librte_eal/common/eal_private.h | 11 + lib/librte_eal/common/include/rte_dev.h | 33 +++ lib/librte_eal/linuxapp/eal/Makefile | 1 + lib/librte_eal/linuxapp/eal/eal_pci.c | 6 +- lib/librte_eal/linuxapp/eal/rte_eal_version.map | 2 + 6 files changed, 326 insertions(+), 3 deletions(-) diff --git a/lib/librte_eal/common/eal_common_dev.c b/lib/librte_eal/common/eal_common_dev.c index eae5656..3d169a4 100644 --- a/lib/librte_eal/common/eal_common_dev.c +++ b/lib/librte_eal/common/eal_common_dev.c @@ -32,10 +32,13 @@ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ +#include <stdio.h> +#include <limits.h> #include <string.h> #include <inttypes.h> #include <sys/queue.h> +#include <rte_ethdev.h> #include <rte_dev.h> #include <rte_devargs.h> #include <rte_debug.h> @@ -107,3 +110,276 @@ rte_eal_dev_init(void) } return 0; } + +/* So far, DPDK hotplug function only supports linux */ +#ifdef ENABLE_HOTPLUG +static void +rte_eal_vdev_invoke(struct rte_driver *driver, + struct rte_devargs *devargs, enum rte_eal_invoke_type type) +{ + if ((driver == NULL) || (devargs == NULL)) + return; + + switch (type) { + case RTE_EAL_INVOKE_TYPE_PROBE: + driver->init(devargs->virtual.drv_name, devargs->args); + break; + case RTE_EAL_INVOKE_TYPE_CLOSE: + driver->uninit(devargs->virtual.drv_name, devargs->args); + break; + default: + break; + } +} + +static int +rte_eal_vdev_find_and_invoke(const char *name, int type) +{ + struct rte_devargs *devargs; + struct rte_driver *driver; + + if (name == NULL) + return -EINVAL; + + /* call the init function for each virtual device */ + TAILQ_FOREACH(devargs, &devargs_list, next) { + + if (devargs->type != RTE_DEVTYPE_VIRTUAL) + continue; + + if (strncmp(name, devargs->virtual.drv_name, strlen(name))) + continue; + + TAILQ_FOREACH(driver, &dev_driver_list, next) { + if (driver->type != PMD_VDEV) + continue; + + /* search a driver prefix in virtual device name */ + if (!strncmp(driver->name, devargs->virtual.drv_name, + strlen(driver->name))) { + rte_eal_vdev_invoke(driver, devargs, type); + break; + } + } + + if (driver == NULL) { + RTE_LOG(WARNING, EAL, "no driver found for %s\n", + devargs->virtual.drv_name); + } + return 0; + } + return 1; +} + +/* attach the new physical device, then store port_id of the device */ +static int +rte_eal_dev_attach_pdev(struct rte_pci_addr *addr, uint8_t *port_id) +{ + uint8_t new_port_id; + struct rte_eth_dev devs[RTE_MAX_ETHPORTS]; + + if ((addr == NULL) || (port_id == NULL)) + goto err; + + /* save current port status */ + if (rte_eth_dev_save(devs, sizeof(devs))) + goto err; + /* re-construct pci_device_list */ + if (rte_eal_pci_scan()) + goto err; + /* invoke probe func of the driver can handle the new device */ + if (rte_eal_pci_probe_one(addr)) + goto err; + /* get port_id enabled by above procedures */ + if (rte_eth_dev_get_changed_port(devs, &new_port_id)) + goto err; + + *port_id = new_port_id; + return 0; +err: + RTE_LOG(ERR, EAL, "Driver, cannot attach the device\n"); + return -1; +} + +/* detach the new physical device, then store pci_addr of the device */ +static int +rte_eal_dev_detach_pdev(uint8_t port_id, struct rte_pci_addr *addr) +{ + struct rte_pci_addr freed_addr; + struct rte_pci_addr vp; + + if (addr == NULL) + goto err; + + /* check whether the driver supports detach feature, or not */ + if (rte_eth_dev_check_detachable(port_id)) + goto err; + + /* get pci address by port id */ + if (rte_eth_dev_get_addr_by_port(port_id, &freed_addr)) + goto err; + + /* Zerod pci addr means the port comes from virtual device */ + vp.domain = vp.bus = vp.devid = vp.function = 0; + if (eal_compare_pci_addr(&vp, &freed_addr) == 0) + goto err; + + /* invoke close func of the driver, + * also remove the device from pci_device_list */ + if (rte_eal_pci_close_one(&freed_addr)) + goto err; + + *addr = freed_addr; + return 0; +err: + RTE_LOG(ERR, EAL, "Driver, cannot detach the device\n"); + return -1; +} + +static void +get_vdev_name(char *vdevargs) +{ + char *sep; + + if (vdevargs == NULL) + return; + + /* set the first ',' to '\0' to split name and arguments */ + sep = strchr(vdevargs, ','); + if (sep != NULL) + sep[0] = '\0'; +} + +/* attach the new virtual device, then store port_id of the device */ +static int +rte_eal_dev_attach_vdev(const char *vdevargs, uint8_t *port_id) +{ + char *args; + uint8_t new_port_id; + struct rte_eth_dev devs[RTE_MAX_ETHPORTS]; + + if ((vdevargs == NULL) || (port_id == NULL)) + goto err0; + + args = strdup(vdevargs); + if (args == NULL) + goto err0; + + /* save current port status */ + if (rte_eth_dev_save(devs, sizeof(devs))) + goto err1; + /* add the vdevargs to devargs_list */ + if (rte_eal_devargs_add(RTE_DEVTYPE_VIRTUAL, args)) + goto err1; + /* parse vdevargs, then retrieve device name */ + get_vdev_name(args); + /* walk around dev_driver_list to find the driver of the device, + * then invoke probe function o the driver */ + if (rte_eal_vdev_find_and_invoke(args, RTE_EAL_INVOKE_TYPE_PROBE)) + goto err2; + /* get port_id enabled by above procedures */ + if (rte_eth_dev_get_changed_port(devs, &new_port_id)) + goto err2; + + free(args); + *port_id = new_port_id; + return 0; +err2: + rte_eal_devargs_remove(RTE_DEVTYPE_VIRTUAL, args); +err1: + free(args); +err0: + RTE_LOG(ERR, EAL, "Driver, cannot attach the device\n"); + return -1; +} + +/* detach the new virtual device, then store the name of the device */ +static int +rte_eal_dev_detach_vdev(uint8_t port_id, char *vdevname) +{ + char name[RTE_ETH_NAME_MAX_LEN]; + + if (vdevname == NULL) + goto err; + + /* check whether the driver supports detach feature, or not */ + if (rte_eth_dev_check_detachable(port_id)) + goto err; + + /* get device name by port id */ + if (rte_eth_dev_get_name_by_port(port_id, name)) + goto err; + /* walk around dev_driver_list to find the driver of the device, + * then invoke close function o the driver */ + if (rte_eal_vdev_find_and_invoke(name, RTE_EAL_INVOKE_TYPE_CLOSE)) + goto err; + /* remove the vdevname from devargs_list */ + if (rte_eal_devargs_remove(RTE_DEVTYPE_VIRTUAL, name)) + goto err; + + strncpy(vdevname, name, sizeof(name)); + return 0; +err: + RTE_LOG(ERR, EAL, "Driver, cannot detach the device\n"); + return -1; +} + +/* attach the new device, then store port_id of the device */ +int +rte_eal_dev_attach(const char *devargs, uint8_t *port_id) +{ + struct rte_pci_addr addr; + + if ((devargs == NULL) || (port_id == NULL)) + return -EINVAL; + + if (eal_parse_pci_DomBDF(devargs, &addr) == 0) + return rte_eal_dev_attach_pdev(&addr, port_id); + else + return rte_eal_dev_attach_vdev(devargs, port_id); +} + +/* detach the device, then store the name of the device */ +int +rte_eal_dev_detach(uint8_t port_id, char *name) +{ + struct rte_pci_addr addr; + int ret; + + if (name == NULL) + return -EINVAL; + + if (rte_eth_dev_get_device_type(port_id) == RTE_ETH_DEV_PHYSICAL) { + ret = rte_eth_dev_get_addr_by_port(port_id, &addr); + if (ret < 0) + return ret; + + ret = rte_eal_dev_detach_pdev(port_id, &addr); + if (ret == 0) + snprintf(name, RTE_ETH_NAME_MAX_LEN, + "%04x:%02x:%02x.%d", + addr.domain, addr.bus, + addr.devid, addr.function); + + return ret; + } else + return rte_eal_dev_detach_vdev(port_id, name); +} +#else /* ENABLE_HOTPLUG */ +int +rte_eal_dev_attach(const char *devargs __rte_unused, + uint8_t *port_id __rte_unused) +{ + RTE_LOG(ERR, EAL, "Hotplug support isn't enabled\n"); + return -1; +} + +/* detach the device, then store the name of the device */ +int +rte_eal_dev_detach(uint8_t port_id __rte_unused, + char *name __rte_unused) +{ + RTE_LOG(ERR, EAL, "Hotplug support isn't enabled\n"); + return -1; +} +#endif /* ENABLE_HOTPLUG */ diff --git a/lib/librte_eal/common/eal_private.h b/lib/librte_eal/common/eal_private.h index 1a362ab..8168a7a 100644 --- a/lib/librte_eal/common/eal_private.h +++ b/lib/librte_eal/common/eal_private.h @@ -163,6 +163,17 @@ enum rte_eal_invoke_type { }; /** + * Scan the content of the PCI bus, and the devices in the devices + * list + * + * This function is private to EAL. + * + * @return + * 0 on success, negative on error + */ +int rte_eal_pci_scan(void); + +/** * Mmap memory for single PCI device * * This function is private to EAL. diff --git a/lib/librte_eal/common/include/rte_dev.h b/lib/librte_eal/common/include/rte_dev.h index f7e3a10..e63dd1c 100644 --- a/lib/librte_eal/common/include/rte_dev.h +++ b/lib/librte_eal/common/include/rte_dev.h @@ -47,6 +47,7 @@ extern "C" { #endif #include <sys/queue.h> +#include <rte_pci.h> /** Double linked list of device drivers. */ TAILQ_HEAD(rte_driver_list, rte_driver); @@ -57,6 +58,11 @@ TAILQ_HEAD(rte_driver_list, rte_driver); typedef int (rte_dev_init_t)(const char *name, const char *args); /** + * Uninitilization function called for each device driver once. + */ +typedef int (rte_dev_uninit_t)(const char *name, const char *args); + +/** * Driver type enumeration */ enum pmd_type { @@ -72,6 +78,7 @@ struct rte_driver { enum pmd_type type; /**< PMD Driver type */ const char *name; /**< Driver name. */ rte_dev_init_t *init; /**< Device init. function. */ + rte_dev_uninit_t *uninit; /**< Device uninit. function. */ }; /** @@ -93,6 +100,32 @@ void rte_eal_driver_register(struct rte_driver *driver); void rte_eal_driver_unregister(struct rte_driver *driver); /** + * Attach a new device. + * + * @param devargs + * A pointer to a strings array describing the new device + * to be attached. The strings should be a pci address like + * '0000:01:00.0' or virtual device name like 'eth_pcap0'. + * @param port_id + * A pointer to a port identifier actually attached. + * @return + * 0 on success and port_id is filled, negative on error + */ +int rte_eal_dev_attach(const char *devargs, uint8_t *port_id); + +/** + * Detach a device. + * + * @param port_id + * The port identifier of the device to detach. + * @param addr + * A pointer to a device name actually detached. + * @return + * 0 on success and devname is filled, negative on error + */ +int rte_eal_dev_detach(uint8_t port_id, char *devname); + +/** * Initalize all the registered drivers in this process */ int rte_eal_dev_init(void); diff --git a/lib/librte_eal/linuxapp/eal/Makefile b/lib/librte_eal/linuxapp/eal/Makefile index e117cec..b59b201 100644 --- a/lib/librte_eal/linuxapp/eal/Makefile +++ b/lib/librte_eal/linuxapp/eal/Makefile @@ -45,6 +45,7 @@ CFLAGS += -I$(RTE_SDK)/lib/librte_eal/common/include CFLAGS += -I$(RTE_SDK)/lib/librte_ring CFLAGS += -I$(RTE_SDK)/lib/librte_mempool CFLAGS += -I$(RTE_SDK)/lib/librte_malloc +CFLAGS += -I$(RTE_SDK)/lib/librte_mbuf CFLAGS += -I$(RTE_SDK)/lib/librte_ether CFLAGS += -I$(RTE_SDK)/lib/librte_ivshmem CFLAGS += -I$(RTE_SDK)/lib/librte_pmd_ring diff --git a/lib/librte_eal/linuxapp/eal/eal_pci.c b/lib/librte_eal/linuxapp/eal/eal_pci.c index 0adb217..e2dd034 100644 --- a/lib/librte_eal/linuxapp/eal/eal_pci.c +++ b/lib/librte_eal/linuxapp/eal/eal_pci.c @@ -443,8 +443,8 @@ error: * Scan the content of the PCI bus, and the devices in the devices * list */ -static int -pci_scan(void) +int +rte_eal_pci_scan(void) { struct dirent *e; DIR *dir; @@ -776,7 +776,7 @@ rte_eal_pci_init(void) if (internal_config.no_pci) return 0; - if (pci_scan() < 0) { + if (rte_eal_pci_scan() < 0) { RTE_LOG(ERR, EAL, "%s(): Cannot scan PCI bus\n", __func__); return -1; } diff --git a/lib/librte_eal/linuxapp/eal/rte_eal_version.map b/lib/librte_eal/linuxapp/eal/rte_eal_version.map index d36286e..ff46b3a 100644 --- a/lib/librte_eal/linuxapp/eal/rte_eal_version.map +++ b/lib/librte_eal/linuxapp/eal/rte_eal_version.map @@ -21,6 +21,8 @@ DPDK_2.0 { rte_eal_alarm_cancel; rte_eal_alarm_set; rte_eal_dev_init; + rte_eal_dev_attach; + rte_eal_dev_detach; rte_eal_devargs_add; rte_eal_devargs_dump; rte_eal_devargs_type_count; -- 1.9.1
The patch enables CONFIG_RTE_LIBRTE_EAL_HOTPLUG in Linux configuration. Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp> --- config/common_linuxapp | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/config/common_linuxapp b/config/common_linuxapp index d428f84..81055f8 100644 --- a/config/common_linuxapp +++ b/config/common_linuxapp @@ -114,6 +114,11 @@ CONFIG_RTE_PCI_MAX_READ_REQUEST_SIZE=0 CONFIG_RTE_LIBRTE_EAL_LINUXAPP=y # +# Compile Environment Abstraction Layer to support hotplug +# +CONFIG_RTE_LIBRTE_EAL_HOTPLUG=y + +# # Compile Environment Abstraction Layer to support Vmware TSC map # CONFIG_RTE_LIBRTE_EAL_VMWARE_TSC_MAP_SUPPORT=y -- 1.9.1
This patch adds a new section for describing port hotplug framework. Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp> --- doc/guides/prog_guide/index.rst | 1 + doc/guides/prog_guide/port_hotplug_framework.rst | 110 +++++++++++++++++++++++ 2 files changed, 111 insertions(+) create mode 100644 doc/guides/prog_guide/port_hotplug_framework.rst diff --git a/doc/guides/prog_guide/index.rst b/doc/guides/prog_guide/index.rst index 8d86dd4..428b76b 100644 --- a/doc/guides/prog_guide/index.rst +++ b/doc/guides/prog_guide/index.rst @@ -70,6 +70,7 @@ Programmer's Guide packet_classif_access_ctrl packet_framework vhost_lib + port_hotplug_framework source_org dev_kit_build_system dev_kit_root_make_help diff --git a/doc/guides/prog_guide/port_hotplug_framework.rst b/doc/guides/prog_guide/port_hotplug_framework.rst new file mode 100644 index 0000000..355ae28 --- /dev/null +++ b/doc/guides/prog_guide/port_hotplug_framework.rst @@ -0,0 +1,110 @@ +.. BSD LICENSE + Copyright(c) 2015 IGEL Co.,Ltd. All rights reserved. + All rights reserved. + + Redistribution and use in source and binary forms, with or without + modification, are permitted provided that the following conditions + are met: + + * Redistributions of source code must retain the above copyright + notice, this list of conditions and the following disclaimer. + * Redistributions in binary form must reproduce the above copyright + notice, this list of conditions and the following disclaimer in + the documentation and/or other materials provided with the + distribution. + * Neither the name of IGEL Co.,Ltd. nor the names of its + contributors may be used to endorse or promote products derived + from this software without specific prior written permission. + + THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS + "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT + LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR + A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT + OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, + SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT + LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, + DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY + THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT + (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE + OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + +Port Hotplug Framework +====================== + +The Port Hotplug Framework provides DPDK applications with the ability to +attach and detach ports at runtime. Because the framework depends on PMD +implementation, the ports that PMDs cannot handle are out of scope of this +framework. Furthermore, after detaching a port from a DPDK application, the +framework doesn't provide a way for removing the devices from the system. +For the ports backed by a physical NIC, the kernel will need to support PCI +Hotplug feature. + +Overview +-------- + +The basic requirements of the Port Hotplug Framework are: + +* DPDK applications that use the Port Hotplug Framework must manage their + own ports. + + The Port Hotplug Framework is implemented to allow DPDK applications to + manage ports. For example, when DPDK applications call the port attach + function, the attached port number is returned. DPDK applications can + also detach the port by port number. + +* Kernel support is needed for attaching or detaching physical device + ports. + + To attach new physical device ports, the device will be recognized by + userspace driver I/O framework in kernel at first. Then DPDK + applications can call the Port Hotplug functions to attach the ports. + For detaching, steps are vice versa. + +* Before detaching, they must be stopped and closed. + + DPDK applications must call "rte_eth_dev_stop()" and + "rte_eth_dev_close()" APIs before detaching ports. These functions will + start finalization sequence of the PMDs. + +* The framework doesn't affect legacy DPDK applications behavior. + + If the Port Hotplug functions aren't called, all legacy DPDK apps can + still work without modifications. + +Port Hotplug API overview +------------------------- + +* Attaching a port + + "rte_eal_dev_attach()" API attaches a port to DPDK application, and + returns the attached port number. Before calling the API, the device + should be recognized by an userspace driver I/O framework. The API + receives a pci address like "0000:01:00.0" or a virtual device name + like "eth_pcap0,iface=eth0". In the case of virtual device name, the + format is the same as the general "--vdev" option of DPDK. + +* Detaching a port + + "rte_eal_dev_detach()" API detaches a port from DPDK application, and + returns a pci address of the detached device or a virtual device name + of the device. + +Reference +--------- + + "testpmd" supports the Port Hotplug Framework. + +Limitations +----------- + +* The Port Hotplug APIs are not thread safe. + +* The framework can only be enabled with Linux. BSD is not supported. + +* To detach a port, the port should be backed by a device that igb_uio + manages. VFIO is not supported. + +* Not all PMDs support detaching feature. + To know whether a PMD can support detaching, search for the + "RTE_PCI_DRV_DETACHABLE" flag in PMD implementation. If the flag is + defined in the PMD, detaching is supported. -- 1.9.1
This patch adds finalization code to free resources allocated by the PMD. v6: - Fix a paramter of rte_eth_dev_free(). v4: - Change function name. Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp> --- lib/librte_pmd_pcap/rte_eth_pcap.c | 40 ++++++++++++++++++++++++++++++++++++++ 1 file changed, 40 insertions(+) diff --git a/lib/librte_pmd_pcap/rte_eth_pcap.c b/lib/librte_pmd_pcap/rte_eth_pcap.c index af7fae8..5f88efd 100644 --- a/lib/librte_pmd_pcap/rte_eth_pcap.c +++ b/lib/librte_pmd_pcap/rte_eth_pcap.c @@ -498,6 +498,13 @@ static struct eth_dev_ops ops = { .stats_reset = eth_stats_reset, }; +static struct eth_driver rte_pcap_pmd = { + .pci_drv = { + .name = "rte_pcap_pmd", + .drv_flags = RTE_PCI_DRV_DETACHABLE, + }, +}; + /* * Function handler that opens the pcap file for reading a stores a * reference of it for use it later on. @@ -713,6 +720,10 @@ rte_pmd_init_internals(const char *name, const unsigned nb_rx_queues, if (*eth_dev == NULL) goto error; + /* check length of device name */ + if ((strlen((*eth_dev)->data->name) + 1) > sizeof(data->name)) + goto error; + /* now put it all together * - store queue data in internals, * - store numa_node info in pci_driver @@ -739,10 +750,13 @@ rte_pmd_init_internals(const char *name, const unsigned nb_rx_queues, data->nb_tx_queues = (uint16_t)nb_tx_queues; data->dev_link = pmd_link; data->mac_addrs = ð_addr; + strncpy(data->name, + (*eth_dev)->data->name, strlen((*eth_dev)->data->name)); (*eth_dev)->data = data; (*eth_dev)->dev_ops = &ops; (*eth_dev)->pci_dev = pci_dev; + (*eth_dev)->driver = &rte_pcap_pmd; return 0; @@ -927,10 +941,36 @@ rte_pmd_pcap_devinit(const char *name, const char *params) } +static int +rte_pmd_pcap_devuninit(const char *name, const char *params __rte_unused) +{ + struct rte_eth_dev *eth_dev = NULL; + + RTE_LOG(INFO, PMD, "Closing pcap ethdev on numa socket %u\n", + rte_socket_id()); + + if (name == NULL) + return -1; + + /* reserve an ethdev entry */ + eth_dev = rte_eth_dev_allocated(name); + if (eth_dev == NULL) + return -1; + + rte_free(eth_dev->data->dev_private); + rte_free(eth_dev->data); + rte_free(eth_dev->pci_dev); + + rte_eth_dev_free(eth_dev); + + return 0; +} + static struct rte_driver pmd_pcap_drv = { .name = "eth_pcap", .type = PMD_VDEV, .init = rte_pmd_pcap_devinit, + .uninit = rte_pmd_pcap_devuninit, }; PMD_REGISTER_DRIVER(pmd_pcap_drv); -- 1.9.1
The patch introduces following commands. - port attach [ident] - port detach [port_id] - attach: attaching a port - detach: detaching a port - ident: pci address of physical device. Or device name and parameters of virtual device. (ex. 0000:02:00.0, eth_pcap0,iface=eth0) - port_id: port identifier v7: - Fix doc. (Thanks to Iremonger, Bernard) - Fix port checking implementation of star_port(); (Thanks to Qiu, Michael) v5: - Add testpmd documentation. (Thanks to Iremonger, Bernard) v4: - Fix strings of command help. Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp> --- app/test-pmd/cmdline.c | 133 +++++++++++++++---- app/test-pmd/config.c | 116 +++++++++------- app/test-pmd/parameters.c | 22 ++- app/test-pmd/testpmd.c | 199 +++++++++++++++++++++------- app/test-pmd/testpmd.h | 18 ++- doc/guides/testpmd_app_ug/testpmd_funcs.rst | 57 ++++++++ 6 files changed, 415 insertions(+), 130 deletions(-) diff --git a/app/test-pmd/cmdline.c b/app/test-pmd/cmdline.c index b2aab40..69cf522 100644 --- a/app/test-pmd/cmdline.c +++ b/app/test-pmd/cmdline.c @@ -573,6 +573,12 @@ static void cmd_help_long_parsed(void *parsed_result, "port close (port_id|all)\n" " Close all ports or port_id.\n\n" + "port attach (ident)\n" + " Attach physical or virtual dev by pci address or virtual device name\n\n" + + "port detach (port_id)\n" + " Detach physical or virtual dev by port_id\n\n" + "port config (port_id|all)" " speed (10|100|1000|10000|40000|auto)" " duplex (half|full|auto)\n" @@ -864,6 +870,89 @@ cmdline_parse_inst_t cmd_operate_specific_port = { }, }; +/* *** attach a specified port *** */ +struct cmd_operate_attach_port_result { + cmdline_fixed_string_t port; + cmdline_fixed_string_t keyword; + cmdline_fixed_string_t identifier; +}; + +static void cmd_operate_attach_port_parsed(void *parsed_result, + __attribute__((unused)) struct cmdline *cl, + __attribute__((unused)) void *data) +{ + struct cmd_operate_attach_port_result *res = parsed_result; + + if (!strcmp(res->keyword, "attach")) + attach_port(res->identifier); + else + printf("Unknown parameter\n"); +} + +cmdline_parse_token_string_t cmd_operate_attach_port_port = + TOKEN_STRING_INITIALIZER(struct cmd_operate_attach_port_result, + port, "port"); +cmdline_parse_token_string_t cmd_operate_attach_port_keyword = + TOKEN_STRING_INITIALIZER(struct cmd_operate_attach_port_result, + keyword, "attach"); +cmdline_parse_token_string_t cmd_operate_attach_port_identifier = + TOKEN_STRING_INITIALIZER(struct cmd_operate_attach_port_result, + identifier, NULL); + +cmdline_parse_inst_t cmd_operate_attach_port = { + .f = cmd_operate_attach_port_parsed, + .data = NULL, + .help_str = "port attach identifier, " + "identifier: pci address or virtual dev name", + .tokens = { + (void *)&cmd_operate_attach_port_port, + (void *)&cmd_operate_attach_port_keyword, + (void *)&cmd_operate_attach_port_identifier, + NULL, + }, +}; + +/* *** detach a specified port *** */ +struct cmd_operate_detach_port_result { + cmdline_fixed_string_t port; + cmdline_fixed_string_t keyword; + uint8_t port_id; +}; + +static void cmd_operate_detach_port_parsed(void *parsed_result, + __attribute__((unused)) struct cmdline *cl, + __attribute__((unused)) void *data) +{ + struct cmd_operate_detach_port_result *res = parsed_result; + + if (!strcmp(res->keyword, "detach")) + detach_port(res->port_id); + else + printf("Unknown parameter\n"); +} + +cmdline_parse_token_string_t cmd_operate_detach_port_port = + TOKEN_STRING_INITIALIZER(struct cmd_operate_detach_port_result, + port, "port"); +cmdline_parse_token_string_t cmd_operate_detach_port_keyword = + TOKEN_STRING_INITIALIZER(struct cmd_operate_detach_port_result, + keyword, "detach"); +cmdline_parse_token_num_t cmd_operate_detach_port_port_id = + TOKEN_NUM_INITIALIZER(struct cmd_operate_detach_port_result, + port_id, UINT8); + +cmdline_parse_inst_t cmd_operate_detach_port = { + .f = cmd_operate_detach_port_parsed, + .data = NULL, + .help_str = "port detach port_id", + .tokens = { + (void *)&cmd_operate_detach_port_port, + (void *)&cmd_operate_detach_port_keyword, + (void *)&cmd_operate_detach_port_port_id, + NULL, + }, +}; + /* *** configure speed for all ports *** */ struct cmd_config_speed_all { cmdline_fixed_string_t port; @@ -918,7 +1007,7 @@ cmd_config_speed_all_parsed(void *parsed_result, return; } - for (pid = 0; pid < nb_ports; pid++) { + FOREACH_PORT(pid, ports) { ports[pid].dev_conf.link_speed = link_speed; ports[pid].dev_conf.link_duplex = link_duplex; } @@ -986,10 +1075,8 @@ cmd_config_speed_specific_parsed(void *parsed_result, return; } - if (res->id >= nb_ports) { - printf("Port id %d must be less than %d\n", res->id, nb_ports); + if (port_id_is_invalid(res->id, ENABLED_WARN)) return; - } if (!strcmp(res->value1, "10")) link_speed = ETH_LINK_SPEED_10; @@ -1549,7 +1636,7 @@ cmd_config_rxtx_queue_parsed(void *parsed_result, return; } - if (port_id_is_invalid(res->portid)) + if (port_id_is_invalid(res->portid, ENABLED_WARN)) return; if (port_is_started(res->portid) != 1) { @@ -2901,7 +2988,7 @@ cmd_tx_cksum_parsed(void *parsed_result, uint16_t ol_flags, mask = 0; struct rte_eth_dev_info dev_info; - if (port_id_is_invalid(res->port_id)) { + if (port_id_is_invalid(res->port_id, ENABLED_WARN)) { printf("invalid port %d\n", res->port_id); return; } @@ -3028,7 +3115,7 @@ cmd_tso_set_parsed(void *parsed_result, struct cmd_tso_set_result *res = parsed_result; struct rte_eth_dev_info dev_info; - if (port_id_is_invalid(res->port_id)) + if (port_id_is_invalid(res->port_id, ENABLED_WARN)) return; if (!strcmp(res->mode, "set")) @@ -4004,10 +4091,8 @@ static void cmd_set_bond_mac_addr_parsed(void *parsed_result, struct cmd_set_bond_mac_addr_result *res = parsed_result; int ret; - if (res->port_num >= nb_ports) { - printf("Port id %d must be less than %d\n", res->port_num, nb_ports); + if (port_id_is_invalid(res->port_num, ENABLED_WARN)) return; - } ret = rte_eth_bond_mac_address_set(res->port_num, &res->address); @@ -4244,7 +4329,7 @@ static void cmd_set_promisc_mode_parsed(void *parsed_result, /* all ports */ if (allports) { - for (i = 0; i < nb_ports; i++) { + FOREACH_PORT(i, ports) { if (enable) rte_eth_promiscuous_enable(i); else @@ -4324,7 +4409,7 @@ static void cmd_set_allmulti_mode_parsed(void *parsed_result, /* all ports */ if (allports) { - for (i = 0; i < nb_ports; i++) { + FOREACH_PORT(i, ports) { if (enable) rte_eth_allmulticast_enable(i); else @@ -5509,25 +5594,25 @@ static void cmd_showportall_parsed(void *parsed_result, struct cmd_showportall_result *res = parsed_result; if (!strcmp(res->show, "clear")) { if (!strcmp(res->what, "stats")) - for (i = 0; i < nb_ports; i++) + FOREACH_PORT(i, ports) nic_stats_clear(i); else if (!strcmp(res->what, "xstats")) - for (i = 0; i < nb_ports; i++) + FOREACH_PORT(i, ports) nic_xstats_clear(i); } else if (!strcmp(res->what, "info")) - for (i = 0; i < nb_ports; i++) + FOREACH_PORT(i, ports) port_infos_display(i); else if (!strcmp(res->what, "stats")) - for (i = 0; i < nb_ports; i++) + FOREACH_PORT(i, ports) nic_stats_display(i); else if (!strcmp(res->what, "xstats")) - for (i = 0; i < nb_ports; i++) + FOREACH_PORT(i, ports) nic_xstats_display(i); else if (!strcmp(res->what, "fdir")) - for (i = 0; i < nb_ports; i++) + FOREACH_PORT(i, ports) fdir_get_infos(i); else if (!strcmp(res->what, "stat_qmap")) - for (i = 0; i < nb_ports; i++) + FOREACH_PORT(i, ports) nic_stats_mapping_display(i); } @@ -9094,6 +9179,8 @@ cmdline_parse_ctx_t main_ctx[] = { (cmdline_parse_inst_t *)&cmd_set_qmap, (cmdline_parse_inst_t *)&cmd_operate_port, (cmdline_parse_inst_t *)&cmd_operate_specific_port, + (cmdline_parse_inst_t *)&cmd_operate_attach_port, + (cmdline_parse_inst_t *)&cmd_operate_detach_port, (cmdline_parse_inst_t *)&cmd_config_speed_all, (cmdline_parse_inst_t *)&cmd_config_speed_specific, (cmdline_parse_inst_t *)&cmd_config_rx_tx, @@ -9172,7 +9259,7 @@ prompt(void) static void cmd_reconfig_device_queue(portid_t id, uint8_t dev, uint8_t queue) { - if (id < nb_ports) { + if (!port_id_is_invalid(id, DISABLED_WARN)) { /* check if need_reconfig has been set to 1 */ if (ports[id].need_reconfig == 0) ports[id].need_reconfig = dev; @@ -9182,7 +9269,7 @@ cmd_reconfig_device_queue(portid_t id, uint8_t dev, uint8_t queue) } else { portid_t pid; - for (pid = 0; pid < nb_ports; pid++) { + FOREACH_PORT(pid, ports) { /* check if need_reconfig has been set to 1 */ if (ports[pid].need_reconfig == 0) ports[pid].need_reconfig = dev; @@ -9200,10 +9287,8 @@ bypass_is_supported(portid_t port_id) struct rte_port *port; struct rte_pci_id *pci_id; - if (port_id >= nb_ports) { - printf("\tPort id must be less than %d.\n", nb_ports); + if (port_id_is_invalid(port_id, ENABLED_WARN)) return 0; - } /* Get the device id. */ port = &ports[port_id]; diff --git a/app/test-pmd/config.c b/app/test-pmd/config.c index 6bcd23c..718167c 100644 --- a/app/test-pmd/config.c +++ b/app/test-pmd/config.c @@ -124,11 +124,15 @@ nic_stats_display(portid_t port_id) struct rte_eth_stats stats; struct rte_port *port = &ports[port_id]; uint8_t i; + portid_t pid; static const char *nic_stats_border = "########################"; - if (port_id >= nb_ports) { - printf("Invalid port, range is [0, %d]\n", nb_ports - 1); + if (port_id_is_invalid(port_id, ENABLED_WARN)) { + printf("Valid port range is [0"); + FOREACH_PORT(pid, ports) + printf(", %d", pid); + printf("]\n"); return; } rte_eth_stats_get(port_id, &stats); @@ -201,8 +205,13 @@ nic_stats_display(portid_t port_id) void nic_stats_clear(portid_t port_id) { - if (port_id >= nb_ports) { - printf("Invalid port, range is [0, %d]\n", nb_ports - 1); + portid_t pid; + + if (port_id_is_invalid(port_id, ENABLED_WARN)) { + printf("Valid port range is [0"); + FOREACH_PORT(pid, ports) + printf(", %d", pid); + printf("]\n"); return; } rte_eth_stats_reset(port_id); @@ -249,11 +258,15 @@ nic_stats_mapping_display(portid_t port_id) { struct rte_port *port = &ports[port_id]; uint16_t i; + portid_t pid; static const char *nic_stats_mapping_border = "########################"; - if (port_id >= nb_ports) { - printf("Invalid port, range is [0, %d]\n", nb_ports - 1); + if (port_id_is_invalid(port_id, ENABLED_WARN)) { + printf("Valid port range is [0"); + FOREACH_PORT(pid, ports) + printf(", %d", pid); + printf("]\n"); return; } @@ -302,9 +315,13 @@ port_infos_display(portid_t port_id) int vlan_offload; struct rte_mempool * mp; static const char *info_border = "*********************"; + portid_t pid; - if (port_id >= nb_ports) { - printf("Invalid port, range is [0, %d]\n", nb_ports - 1); + if (port_id_is_invalid(port_id, ENABLED_WARN)) { + printf("Valid port range is [0"); + FOREACH_PORT(pid, ports) + printf(", %d", pid); + printf("]\n"); return; } port = &ports[port_id]; @@ -362,11 +379,14 @@ port_infos_display(portid_t port_id) } int -port_id_is_invalid(portid_t port_id) +port_id_is_invalid(portid_t port_id, enum print_warning warning) { - if (port_id < nb_ports) + if (ports[port_id].enabled) return 0; - printf("Invalid port %d (must be < nb_ports=%d)\n", port_id, nb_ports); + + if (warning == ENABLED_WARN) + printf("Invalid port %d\n", port_id); + return 1; } @@ -425,7 +445,7 @@ port_reg_bit_display(portid_t port_id, uint32_t reg_off, uint8_t bit_x) uint32_t reg_v; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; if (port_reg_off_is_invalid(port_id, reg_off)) return; @@ -444,7 +464,7 @@ port_reg_bit_field_display(portid_t port_id, uint32_t reg_off, uint8_t l_bit; uint8_t h_bit; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; if (port_reg_off_is_invalid(port_id, reg_off)) return; @@ -471,7 +491,7 @@ port_reg_display(portid_t port_id, uint32_t reg_off) { uint32_t reg_v; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; if (port_reg_off_is_invalid(port_id, reg_off)) return; @@ -485,7 +505,7 @@ port_reg_bit_set(portid_t port_id, uint32_t reg_off, uint8_t bit_pos, { uint32_t reg_v; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; if (port_reg_off_is_invalid(port_id, reg_off)) return; @@ -513,7 +533,7 @@ port_reg_bit_field_set(portid_t port_id, uint32_t reg_off, uint8_t l_bit; uint8_t h_bit; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; if (port_reg_off_is_invalid(port_id, reg_off)) return; @@ -547,7 +567,7 @@ port_reg_bit_field_set(portid_t port_id, uint32_t reg_off, void port_reg_set(portid_t port_id, uint32_t reg_off, uint32_t reg_v) { - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; if (port_reg_off_is_invalid(port_id, reg_off)) return; @@ -560,7 +580,7 @@ port_mtu_set(portid_t port_id, uint16_t mtu) { int diag; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; diag = rte_eth_dev_set_mtu(port_id, mtu); if (diag == 0) @@ -723,7 +743,7 @@ rx_ring_desc_display(portid_t port_id, queueid_t rxq_id, uint16_t rxd_id) { const struct rte_memzone *rx_mz; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; if (rx_queue_id_is_invalid(rxq_id)) return; @@ -740,7 +760,7 @@ tx_ring_desc_display(portid_t port_id, queueid_t txq_id, uint16_t txd_id) { const struct rte_memzone *tx_mz; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; if (tx_queue_id_is_invalid(txq_id)) return; @@ -801,7 +821,7 @@ port_rss_reta_info(portid_t port_id, uint16_t i, idx, shift; int ret; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; ret = rte_eth_dev_rss_reta_query(port_id, reta_conf, nb_entries); @@ -833,7 +853,7 @@ port_rss_hash_conf_show(portid_t port_id, int show_rss_key) uint8_t i; int diag; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; /* Get RSS hash key if asked to display it */ rss_conf.rss_key = (show_rss_key) ? rss_key : NULL; @@ -1411,12 +1431,8 @@ set_fwd_ports_list(unsigned int *portlist, unsigned int nb_pt) again: for (i = 0; i < nb_pt; i++) { port_id = (portid_t) portlist[i]; - if (port_id >= nb_ports) { - printf("Invalid port id %u >= %u\n", - (unsigned int) port_id, - (unsigned int) nb_ports); + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; - } if (record_now) fwd_ports_ids[i] = port_id; } @@ -1574,7 +1590,7 @@ vlan_extend_set(portid_t port_id, int on) int diag; int vlan_offload; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; vlan_offload = rte_eth_dev_get_vlan_offload(port_id); @@ -1596,7 +1612,7 @@ rx_vlan_strip_set(portid_t port_id, int on) int diag; int vlan_offload; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; vlan_offload = rte_eth_dev_get_vlan_offload(port_id); @@ -1617,7 +1633,7 @@ rx_vlan_strip_set_on_queue(portid_t port_id, uint16_t queue_id, int on) { int diag; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; diag = rte_eth_dev_set_vlan_strip_on_queue(port_id, queue_id, on); @@ -1632,7 +1648,7 @@ rx_vlan_filter_set(portid_t port_id, int on) int diag; int vlan_offload; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; vlan_offload = rte_eth_dev_get_vlan_offload(port_id); @@ -1653,7 +1669,7 @@ rx_vft_set(portid_t port_id, uint16_t vlan_id, int on) { int diag; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; if (vlan_id_is_invalid(vlan_id)) return; @@ -1670,7 +1686,7 @@ rx_vlan_all_filter_set(portid_t port_id, int on) { uint16_t vlan_id; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; for (vlan_id = 0; vlan_id < 4096; vlan_id++) rx_vft_set(port_id, vlan_id, on); @@ -1680,7 +1696,7 @@ void vlan_tpid_set(portid_t port_id, uint16_t tp_id) { int diag; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; diag = rte_eth_dev_set_vlan_ether_type(port_id, tp_id); @@ -1695,7 +1711,7 @@ vlan_tpid_set(portid_t port_id, uint16_t tp_id) void tx_vlan_set(portid_t port_id, uint16_t vlan_id) { - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; if (vlan_id_is_invalid(vlan_id)) return; @@ -1706,7 +1722,7 @@ tx_vlan_set(portid_t port_id, uint16_t vlan_id) void tx_vlan_reset(portid_t port_id) { - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; ports[port_id].tx_ol_flags &= ~TESTPMD_TX_OFFLOAD_INSERT_VLAN; } @@ -1714,7 +1730,7 @@ tx_vlan_reset(portid_t port_id) void tx_vlan_pvid_set(portid_t port_id, uint16_t vlan_id, int on) { - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; rte_eth_dev_set_vlan_pvid(port_id, vlan_id, on); @@ -1726,7 +1742,7 @@ set_qmap(portid_t port_id, uint8_t is_rx, uint16_t queue_id, uint8_t map_value) uint16_t i; uint8_t existing_mapping_found = 0; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; if (is_rx ? (rx_queue_id_is_invalid(queue_id)) : (tx_queue_id_is_invalid(queue_id))) @@ -1778,7 +1794,7 @@ fdir_add_signature_filter(portid_t port_id, uint8_t queue_id, { int diag; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; diag = rte_eth_dev_fdir_add_signature_filter(port_id, fdir_filter, @@ -1796,7 +1812,7 @@ fdir_update_signature_filter(portid_t port_id, uint8_t queue_id, { int diag; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; diag = rte_eth_dev_fdir_update_signature_filter(port_id, fdir_filter, @@ -1814,7 +1830,7 @@ fdir_remove_signature_filter(portid_t port_id, { int diag; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; diag = rte_eth_dev_fdir_remove_signature_filter(port_id, fdir_filter); @@ -1886,7 +1902,7 @@ fdir_get_infos(portid_t port_id) static const char *fdir_stats_border = "########################"; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; ret = rte_eth_dev_filter_supported(port_id, RTE_ETH_FILTER_FDIR); if (ret < 0) { @@ -1960,7 +1976,7 @@ fdir_add_perfect_filter(portid_t port_id, uint16_t soft_id, uint8_t queue_id, { int diag; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; diag = rte_eth_dev_fdir_add_perfect_filter(port_id, fdir_filter, @@ -1978,7 +1994,7 @@ fdir_update_perfect_filter(portid_t port_id, uint16_t soft_id, uint8_t queue_id, { int diag; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; diag = rte_eth_dev_fdir_update_perfect_filter(port_id, fdir_filter, @@ -1996,7 +2012,7 @@ fdir_remove_perfect_filter(portid_t port_id, uint16_t soft_id, { int diag; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; diag = rte_eth_dev_fdir_remove_perfect_filter(port_id, fdir_filter, @@ -2013,7 +2029,7 @@ fdir_set_masks(portid_t port_id, struct rte_fdir_masks *fdir_masks) { int diag; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; diag = rte_eth_dev_fdir_set_masks(port_id, fdir_masks); @@ -2090,7 +2106,7 @@ set_vf_traffic(portid_t port_id, uint8_t is_rx, uint16_t vf, uint8_t on) { int diag; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; if (is_rx) diag = rte_eth_dev_set_vf_rx(port_id,vf,on); @@ -2112,7 +2128,7 @@ set_vf_rx_vlan(portid_t port_id, uint16_t vlan_id, uint64_t vf_mask, uint8_t on) { int diag; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; if (vlan_id_is_invalid(vlan_id)) return; @@ -2129,7 +2145,7 @@ set_queue_rate_limit(portid_t port_id, uint16_t queue_idx, uint16_t rate) int diag; struct rte_eth_link link; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return 1; rte_eth_link_get_nowait(port_id, &link); if (rate > link.link_speed) { @@ -2154,7 +2170,7 @@ set_vf_rate_limit(portid_t port_id, uint16_t vf, uint16_t rate, uint64_t q_msk) if (q_msk == 0) return 0; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return 1; rte_eth_link_get_nowait(port_id, &link); if (rate > link.link_speed) { diff --git a/app/test-pmd/parameters.c b/app/test-pmd/parameters.c index 19fbf46..07740c4 100644 --- a/app/test-pmd/parameters.c +++ b/app/test-pmd/parameters.c @@ -370,6 +370,7 @@ parse_portnuma_config(const char *q_arg) }; unsigned long int_fld[_NUM_FLD]; char *str_fld[_NUM_FLD]; + portid_t pid; /* reset from value set at definition */ while ((p = strchr(p0,'(')) != NULL) { @@ -391,8 +392,11 @@ parse_portnuma_config(const char *q_arg) return -1; } port_id = (uint8_t)int_fld[FLD_PORT]; - if (port_id >= nb_ports) { - printf("Invalid port, range is [0, %d]\n", nb_ports - 1); + if (port_id_is_invalid(port_id, ENABLED_WARN)) { + printf("Valid port range is [0"); + FOREACH_PORT(pid, ports) + printf(", %d", pid); + printf("]\n"); return -1; } socket_id = (uint8_t)int_fld[FLD_SOCKET]; @@ -423,6 +427,7 @@ parse_ringnuma_config(const char *q_arg) }; unsigned long int_fld[_NUM_FLD]; char *str_fld[_NUM_FLD]; + portid_t pid; #define RX_RING_ONLY 0x1 #define TX_RING_ONLY 0x2 #define RXTX_RING 0x3 @@ -447,8 +452,11 @@ parse_ringnuma_config(const char *q_arg) return -1; } port_id = (uint8_t)int_fld[FLD_PORT]; - if (port_id >= nb_ports) { - printf("Invalid port, range is [0, %d]\n", nb_ports - 1); + if (port_id_is_invalid(port_id, ENABLED_WARN)) { + printf("Valid port range is [0"); + FOREACH_PORT(pid, ports) + printf(", %d", pid); + printf("]\n"); return -1; } socket_id = (uint8_t)int_fld[FLD_SOCKET]; @@ -620,12 +628,12 @@ launch_args_parse(int argc, char** argv) #endif if (!strcmp(lgopts[opt_idx].name, "nb-ports")) { n = atoi(optarg); - if (n > 0 && n <= nb_ports) + if (n > 0 && + !port_id_is_invalid(n, DISABLED_WARN)) nb_fwd_ports = (uint8_t) n; else rte_exit(EXIT_FAILURE, - "nb-ports should be > 0 and <= %d\n", - nb_ports); + "Invalid port %d\n", n); } if (!strcmp(lgopts[opt_idx].name, "nb-cores")) { n = atoi(optarg); diff --git a/app/test-pmd/testpmd.c b/app/test-pmd/testpmd.c index 3aebea6..48e726e 100644 --- a/app/test-pmd/testpmd.c +++ b/app/test-pmd/testpmd.c @@ -71,6 +71,7 @@ #include <rte_pci.h> #include <rte_ether.h> #include <rte_ethdev.h> +#include <rte_dev.h> #include <rte_string_fns.h> #ifdef RTE_LIBRTE_PMD_XENVIRT #include <rte_eth_xenvirt.h> @@ -303,7 +304,7 @@ uint16_t nb_rx_queue_stats_mappings = 0; /* Forward function declarations */ static void map_port_queue_stats_mapping_registers(uint8_t pi, struct rte_port *port); -static void check_all_ports_link_status(uint8_t port_num, uint32_t port_mask); +static void check_all_ports_link_status(uint32_t port_mask); /* * Check if all the ports are started. @@ -312,6 +313,20 @@ static void check_all_ports_link_status(uint8_t port_num, uint32_t port_mask); static int all_ports_started(void); /* + * Find next enabled port + */ +portid_t +find_next_port(portid_t p, struct rte_port *ports, int size) +{ + if (ports == NULL) + rte_exit(-EINVAL, "failed to find a next port id\n"); + + while ((ports[p].enabled == 0) && (p < size)) + p++; + return p; +} + +/* * Setup default configuration. */ static void @@ -540,7 +555,8 @@ init_config(void) + RTE_TEST_TX_DESC_MAX + MAX_PKT_BURST; if (!numa_support) - nb_mbuf_per_pool = (nb_mbuf_per_pool * nb_ports); + nb_mbuf_per_pool = + (nb_mbuf_per_pool * RTE_MAX_ETHPORTS); } if (!numa_support) { @@ -553,14 +569,19 @@ init_config(void) /* Configuration of Ethernet ports. */ ports = rte_zmalloc("testpmd: ports", - sizeof(struct rte_port) * nb_ports, + sizeof(struct rte_port) * RTE_MAX_ETHPORTS, RTE_CACHE_LINE_SIZE); if (ports == NULL) { - rte_exit(EXIT_FAILURE, "rte_zmalloc(%d struct rte_port) " - "failed\n", nb_ports); + rte_exit(EXIT_FAILURE, + "rte_zmalloc(%d struct rte_port) failed\n", + RTE_MAX_ETHPORTS); } - for (pid = 0; pid < nb_ports; pid++) { + /* enabled allocated ports */ + for (pid = 0; pid < nb_ports; pid++) + ports[pid].enabled = 1; + + FOREACH_PORT(pid, ports) { port = &ports[pid]; rte_eth_dev_info_get(pid, &port->dev_info); @@ -590,8 +611,7 @@ init_config(void) nb_mbuf_per_pool = nb_mbuf_per_pool/nb_ports; for (i = 0; i < MAX_SOCKET; i++) { - nb_mbuf = (nb_mbuf_per_pool * - port_per_socket[i]); + nb_mbuf = (nb_mbuf_per_pool * RTE_MAX_ETHPORTS); if (nb_mbuf) mbuf_pool_create(mbuf_data_size, nb_mbuf,i); @@ -623,14 +643,6 @@ reconfig(portid_t new_port_id, unsigned socket_id) struct rte_port *port; /* Reconfiguration of Ethernet ports. */ - ports = rte_realloc(ports, - sizeof(struct rte_port) * nb_ports, - RTE_CACHE_LINE_SIZE); - if (ports == NULL) { - rte_exit(EXIT_FAILURE, "rte_realloc(%d struct rte_port) failed\n", - nb_ports); - } - port = &ports[new_port_id]; rte_eth_dev_info_get(new_port_id, &port->dev_info); @@ -651,7 +663,7 @@ init_fwd_streams(void) streamid_t sm_id, nb_fwd_streams_new; /* set socket id according to numa or not */ - for (pid = 0; pid < nb_ports; pid++) { + FOREACH_PORT(pid, ports) { port = &ports[pid]; if (nb_rxq > port->dev_info.max_rx_queues) { printf("Fail: nb_rxq(%d) is greater than " @@ -1252,7 +1264,7 @@ all_ports_started(void) portid_t pi; struct rte_port *port; - for (pi = 0; pi < nb_ports; pi++) { + FOREACH_PORT(pi, ports) { port = &ports[pi]; /* Check if there is a port which is not started */ if (port->port_status != RTE_PORT_STARTED) @@ -1264,6 +1276,45 @@ all_ports_started(void) } int +all_ports_stopped(void) +{ + portid_t pi; + struct rte_port *port; + + FOREACH_PORT(pi, ports) { + port = &ports[pi]; + if (port->port_status != RTE_PORT_STOPPED) + return 0; + } + + return 1; +} + +int +port_is_started(portid_t port_id) +{ + if (port_id_is_invalid(port_id, ENABLED_WARN)) + return 0; + + if (ports[port_id].port_status != RTE_PORT_STARTED) + return 0; + + return 1; +} + +static int +port_is_closed(portid_t port_id) +{ + if (port_id_is_invalid(port_id, ENABLED_WARN)) + return 0; + + if (ports[port_id].port_status != RTE_PORT_CLOSED) + return 0; + + return 1; +} + +int start_port(portid_t pid) { int diag, need_check_link_status = 0; @@ -1284,8 +1335,8 @@ start_port(portid_t pid) if(dcb_config) dcb_test = 1; - for (pi = 0; pi < nb_ports; pi++) { - if (pid < nb_ports && pid != pi) + FOREACH_PORT(pi, ports) { + if (pid != pi && pid != (portid_t)RTE_PORT_ALL) continue; port = &ports[pi]; @@ -1409,7 +1460,7 @@ start_port(portid_t pid) } if (need_check_link_status && !no_link_check) - check_all_ports_link_status(nb_ports, RTE_PORT_ALL); + check_all_ports_link_status(RTE_PORT_ALL); else printf("Please stop the ports first\n"); @@ -1434,8 +1485,8 @@ stop_port(portid_t pid) } printf("Stopping ports...\n"); - for (pi = 0; pi < nb_ports; pi++) { - if (pid < nb_ports && pid != pi) + FOREACH_PORT(pi, ports) { + if (!port_id_is_invalid(pid, DISABLED_WARN) && pid != pi) continue; port = &ports[pi]; @@ -1451,7 +1502,7 @@ stop_port(portid_t pid) need_check_link_status = 1; } if (need_check_link_status && !no_link_check) - check_all_ports_link_status(nb_ports, RTE_PORT_ALL); + check_all_ports_link_status(RTE_PORT_ALL); printf("Done\n"); } @@ -1469,8 +1520,8 @@ close_port(portid_t pid) printf("Closing ports...\n"); - for (pi = 0; pi < nb_ports; pi++) { - if (pid < nb_ports && pid != pi) + FOREACH_PORT(pi, ports) { + if (!port_id_is_invalid(pid, DISABLED_WARN) && pid != pi) continue; port = &ports[pi]; @@ -1490,31 +1541,83 @@ close_port(portid_t pid) printf("Done\n"); } -int -all_ports_stopped(void) +void +attach_port(char *identifier) { - portid_t pi; - struct rte_port *port; + portid_t i, j, pi = 0; - for (pi = 0; pi < nb_ports; pi++) { - port = &ports[pi]; - if (port->port_status != RTE_PORT_STOPPED) - return 0; + printf("Attaching a new port...\n"); + + if (identifier == NULL) { + printf("Invalid parameters are specified\n"); + return; } - return 1; + if (test_done == 0) { + printf("Please stop forwarding first\n"); + return; + } + + if (rte_eal_dev_attach(identifier, &pi)) + return; + + ports[pi].enabled = 1; + reconfig(pi, rte_eth_dev_socket_id(pi)); + rte_eth_promiscuous_enable(pi); + + nb_ports = rte_eth_dev_count(); + + /* set_default_fwd_ports_config(); */ + bzero(fwd_ports_ids, sizeof(fwd_ports_ids)); + i = 0; + FOREACH_PORT(j, ports) { + fwd_ports_ids[i] = j; + i++; + } + nb_cfg_ports = nb_ports; + nb_fwd_ports++; + + ports[pi].port_status = RTE_PORT_STOPPED; + + printf("Port %d is attached. Now total ports is %d\n", pi, nb_ports); + printf("Done\n"); } -int -port_is_started(portid_t port_id) +void +detach_port(uint8_t port_id) { - if (port_id_is_invalid(port_id)) - return -1; + portid_t i, pi = 0; + char name[RTE_ETH_NAME_MAX_LEN]; - if (ports[port_id].port_status != RTE_PORT_STARTED) - return 0; + printf("Detaching a port...\n"); - return 1; + if (!port_is_closed(port_id)) { + printf("Please close port first\n"); + return; + } + + rte_eth_promiscuous_disable(port_id); + + if (rte_eal_dev_detach(port_id, name)) + return; + + ports[port_id].enabled = 0; + nb_ports = rte_eth_dev_count(); + + /* set_default_fwd_ports_config(); */ + bzero(fwd_ports_ids, sizeof(fwd_ports_ids)); + i = 0; + FOREACH_PORT(pi, ports) { + fwd_ports_ids[i] = pi; + i++; + } + nb_cfg_ports = nb_ports; + nb_fwd_ports--; + + printf("Port '%s' is detached. Now total ports is %d\n", + name, nb_ports); + printf("Done\n"); + return; } void @@ -1522,7 +1625,7 @@ pmd_test_exit(void) { portid_t pt_id; - for (pt_id = 0; pt_id < nb_ports; pt_id++) { + FOREACH_PORT(pt_id, ports) { printf("Stopping port %d...", pt_id); fflush(stdout); rte_eth_dev_close(pt_id); @@ -1541,7 +1644,7 @@ struct pmd_test_command { /* Check the link status of all ports in up to 9s, and print them finally */ static void -check_all_ports_link_status(uint8_t port_num, uint32_t port_mask) +check_all_ports_link_status(uint32_t port_mask) { #define CHECK_INTERVAL 100 /* 100ms */ #define MAX_CHECK_TIME 90 /* 9s (90 * 100ms) in total */ @@ -1552,7 +1655,7 @@ check_all_ports_link_status(uint8_t port_num, uint32_t port_mask) fflush(stdout); for (count = 0; count <= MAX_CHECK_TIME; count++) { all_ports_up = 1; - for (portid = 0; portid < port_num; portid++) { + FOREACH_PORT(portid, ports) { if ((port_mask & (1 << portid)) == 0) continue; memset(&link, 0, sizeof(link)); @@ -1717,7 +1820,7 @@ init_port_config(void) portid_t pid; struct rte_port *port; - for (pid = 0; pid < nb_ports; pid++) { + FOREACH_PORT(pid, ports) { port = &ports[pid]; port->dev_conf.rxmode = rx_mode; port->dev_conf.fdir_conf = fdir_conf; @@ -1896,7 +1999,7 @@ main(int argc, char** argv) nb_ports = (portid_t) rte_eth_dev_count(); if (nb_ports == 0) - rte_exit(EXIT_FAILURE, "No probed ethernet device\n"); + RTE_LOG(WARNING, EAL, "No probed ethernet devices\n"); set_def_fwd_config(); if (nb_lcores == 0) @@ -1918,7 +2021,7 @@ main(int argc, char** argv) rte_exit(EXIT_FAILURE, "Start ports failed\n"); /* set all ports to promiscuous mode by default */ - for (port_id = 0; port_id < nb_ports; port_id++) + FOREACH_PORT(port_id, ports) rte_eth_promiscuous_enable(port_id); #ifdef RTE_LIBRTE_CMDLINE diff --git a/app/test-pmd/testpmd.h b/app/test-pmd/testpmd.h index 6b3daf8..80ca345 100644 --- a/app/test-pmd/testpmd.h +++ b/app/test-pmd/testpmd.h @@ -134,6 +134,7 @@ struct fwd_stream { * The data structure associated with each port. */ struct rte_port { + uint8_t enabled; /**< Port enabled or not */ struct rte_eth_dev_info dev_info; /**< PCI info + driver name */ struct rte_eth_conf dev_conf; /**< Port configuration. */ struct ether_addr eth_addr; /**< Port ethernet address */ @@ -159,6 +160,14 @@ struct rte_port { struct rte_eth_txconf tx_conf; /**< tx configuration */ }; +extern portid_t __rte_unused +find_next_port(portid_t p, struct rte_port *ports, int size); + +#define FOREACH_PORT(p, ports) \ + for (p = find_next_port(0, ports, RTE_MAX_ETHPORTS); \ + p < RTE_MAX_ETHPORTS; \ + p = find_next_port(p + 1, ports, RTE_MAX_ETHPORTS)) + /** * The data structure associated with each forwarding logical core. * The logical cores are internally numbered by a core index from 0 to @@ -519,6 +528,8 @@ int init_port_dcb_config(portid_t pid,struct dcb_config *dcb_conf); int start_port(portid_t pid); void stop_port(portid_t pid); void close_port(portid_t pid); +void attach_port(char *identifier); +void detach_port(uint8_t port_id); int all_ports_stopped(void); int port_is_started(portid_t port_id); void pmd_test_exit(void); @@ -562,10 +573,15 @@ void get_ethertype_filter(uint8_t port_id, uint16_t index); void get_2tuple_filter(uint8_t port_id, uint16_t index); void get_5tuple_filter(uint8_t port_id, uint16_t index); void get_flex_filter(uint8_t port_id, uint16_t index); -int port_id_is_invalid(portid_t port_id); int rx_queue_id_is_invalid(queueid_t rxq_id); int tx_queue_id_is_invalid(queueid_t txq_id); +enum print_warning { + ENABLED_WARN = 0, + DISABLED_WARN +}; +int port_id_is_invalid(portid_t port_id, enum print_warning warning); + /* * Work-around of a compilation error with ICC on invocations of the * rte_be_to_cpu_16() function. diff --git a/doc/guides/testpmd_app_ug/testpmd_funcs.rst b/doc/guides/testpmd_app_ug/testpmd_funcs.rst index 218835a..936f9e0 100644 --- a/doc/guides/testpmd_app_ug/testpmd_funcs.rst +++ b/doc/guides/testpmd_app_ug/testpmd_funcs.rst @@ -808,6 +808,63 @@ The following sections show functions for configuring ports. Port configuration changes only become active when forwarding is started/restarted. +port attach +~~~~~~~~~~~ + +Attach a port specified by pci address or virtual device args. + +To attach a new pci device, the device should be recognized by kernel first. +Then it should be moved under DPDK management. +Finally the port can be attached to testpmd. +On the other hand, to attach a port created by virtual device, above steps are not needed. + +port attach (identifier) + +For example, to attach a port whose pci address is 0000:02:00.0. + +.. code-block:: console + + testpmd> port attach 0000:02:00.0 + Attaching a new port... + ... snip ... + Port 0 is attached. Now total ports is 1 + Done + +For example, to attach a port created by pcap PMD. + +.. code-block:: console + + testpmd> port attach eth_pcap0,iface=eth0 + Attaching a new port... + ... snip ... + Port 0 is attached. Now total ports is 1 + Done + +In this case, identifier is "eth_pcap0,iface=eth0". +This identifier format is the same as "--vdev" format of DPDK applications. + +port detach +~~~~~~~~~~~ + +Detach a specific port. + +Before detaching a port, the port should be closed. +Also to remove a pci device completely from the system, first detach the port from testpmd. +Then the device should be moved under kernel management. +Finally the device can be removed using kernel pci hotplug functionality. +On the other hand, to remove a port created by a virtual device, above steps are not needed. + +port detach (port_id) + +For example, to detach a port 0. + +.. code-block:: console + + testpmd> port detach 0 + Detaching a port... + ... snip ... + Done + port start ~~~~~~~~~~ -- 1.9.1
On 2/16/2015 12:15 PM, Tetsuya Mukawa wrote:
> This patch series adds a dynamic port hotplug framework to DPDK.
> With the patches, DPDK apps can attach or detach ports at runtime.
>
> The basic concept of the port hotplug is like followings.
> - DPDK apps must have responsibility to manage ports.
> DPDK apps only know which ports are attached or detached at the moment.
> The port hotplug framework is implemented to allow DPDK apps to manage ports.
> For example, when DPDK apps call port attach function, attached port number
> will be returned. Also, DPDK apps can detach port by port number.
> - Kernel support is needed for attaching or detaching physical device ports.
> To attach a new physical device port, the device will be recognized by
> userspace directly I/O framework in kernel at first. Then DPDK apps can
> call the port hotplug functions to attach ports.
> For detaching, steps are vice versa.
> - Before detach ports, ports must be stopped and closed.
> DPDK application must call rte_eth_dev_stop() and rte_eth_dev_close() before
> detaching ports. These function will call finalization codes of PMDs.
> But so far, no PMD frees all resources allocated by initialization.
> It means PMDs are needed to be fixed to support the port hotplug.
> 'RTE_PCI_DRV_DETACHABLE' is a new flag indicating a PMD supports detaching.
> Without this flag, detaching will be failed.
> - Mustn't affect legacy DPDK apps.
> No DPDK EAL behavior is changed, if the port hotplug functions are't called.
> So all legacy DPDK apps can still work without modifications.
>
> And a few limitations.
> - The port hotplug functions are not thread safe.
> DPDK apps should handle it.
> - Only support Linux and igb_uio so far.
> BSD and VFIO is not supported. I will send VFIO patches at least, but I don't
> have a plan to submit BSD patch so far.
>
>
> Here is port hotplug APIs.
> -------------------------------------------------------------------------------
> /**
> * Attach a new device.
> *
> * @param devargs
> * A pointer to a strings array describing the new device
> * to be attached. The strings should be a pci address like
> * '0000:01:00.0' or virtual device name like 'eth_pcap0'.
> * @param port_id
> * A pointer to a port identifier actually attached.
> * @return
> * 0 on success and port_id is filled, negative on error
> */
> int rte_eal_dev_attach(const char *devargs, uint8_t *port_id);
>
> /**
> * Detach a device.
> *
> * @param port_id
> * The port identifier of the device to detach.
> * @param addr
> * A pointer to a device name actually detached.
> * @return
> * 0 on success and devname is filled, negative on error
> */
> int rte_eal_dev_detach(uint8_t port_id, char *devname);
> -------------------------------------------------------------------------------
>
> This patch series are for DPDK EAL. To use port hotplug function by DPDK apps,
> each PMD should be fixed to support 'RTE_PCI_DRV_DETACHABLE' flag. Please check
> a patch for pcap PMD.
>
> Also, please check testpmd patch. It will show you how to fix your legacy
> applications to support port hotplug feature.
>
> PATCH v8 changes
> - Fix Makefile and add version map file.
> - Add missing symbol in version map.
> - Fix pci_scan_one() to update sysfs values.
> (Thanks to Qiu, Michael and Iremonger, Bernard)
> - NONE_TRACE is replaced by NO_TRACE.
> - Fix typo.
> - Add size parameter to rte_eth_dev_save().
> (Thanks to Iremonger, Bernard)
>
> PATCH v7 changes
> - Add a new section to programmer's guide.
> (Thanks to Iremonger, Bernard)
> - Fix port checking implementation of star_port().
> - Fix typo of warning messages.
> - Add pt_driver checking to rte_eth_dev_check_detachable().
> (Thanks to Qiu, Michael)
>
> PATCH v6 changes
> - Fix rte_eth_dev_uninit() to handle a return value of uninit
> function of PMD. To do this, below changes also be applied.
> - Fix a parameter of rte_eth_dev_free().
> - Use rte_eth_dev structure as the paramter of rte_eth_dev_free().
>
> PATCH v5 changes
> - Add runtime check passthrough driver type, like vfio-pci, igb_uio
> and uio_pci_generic.
> This was done by Qiu, Michael. Thanks a lot.
> - Change function names like below.
> - rte_eal_dev_find_and_invoke() to rte_eal_vdev_find_and_invoke().
> - rte_eal_dev_invoke() to rte_eal_vdev_invoke().
> - Add code to handle a return value of rte_eal_devargs_remove().
> - Fix pci address format in rte_eal_dev_detach().
> - Remove RTE_EAL_INVOKE_TYPE_UNKNOWN, because it's unused.
> - Change function definition of rte_eal_devargs_remove().
> - Fix pci_unmap_device() to check pt_driver.
> - Fix return value of below functions.
> - rte_eth_dev_get_changed_port().
> - rte_eth_dev_get_port_by_addr().
> - Change paramters of rte_eth_dev_validate_port() to cleanup code.
> - Fix pci_scan_one to handle pt_driver correctly.
> (Thanks to Qiu, Michael for above suggestions)
>
> PATCH v4 changes
> - Merge patches to review easier.
> - Fix indent of 'if' statement.
> - Fix calculation method of eal_compare_pci_addr().
> - Fix header file declaration.
> - Add header file to determine if hotplug can be enabled.
> (Thanks to Qiu, Michael)
> - Use braces with 'for' loop.
> - Add parameter checking.
> - Fix sanity check code
> - Fix comments of rte_eth_dev_type.
> - Change function names.
> (Thanks to Iremonger, Bernard)
>
> PATCH v3 changes:
> - Fix enum definition used in rte_ethdev.c.
> (Thanks to Zhang, Helin)
>
> PATCH v2 changes:
> - Replace rte_eal_dev_attach_pdev(), rte_eal_dev_detach_pdev,
> rte_eal_dev_attach_vdev() and rte_eal_dev_detach_vdev() to
> rte_eal_dev_attach() and rte_eal_dev_detach().
> - Add parameter values checking.
> - Refashion a few functions.
> (Thanks to Iremonger, Bernard)
>
> PATCH v1 Changes:
> - Fix error checking code of librte_eth APIs.
> - Fix issue that port from pcap PMD cannot be detached correctly.
> - Fix issue that testpmd could hang after forwarding, if attaching and detaching
> is repeatedly.
> - Fix if-condition of rte_eth_dev_get_port_by_addr().
> (Thanks to Mark Enright)
>
> RFC PATCH v2 Changes:
> - remove 'rte_eth_dev_validate_port()', and cleanup codes.
>
>
> Michael Qiu (2):
> eal_pci: Add flag to hold kernel driver type
> eal_pci: pci memory map work with driver type
>
> Tetsuya Mukawa (12):
> eal/pci,ethdev: Remove assumption that port will not be detached
> eal/pci: Consolidate pci address comparison APIs
> ethdev: Add rte_eth_dev_free to free specified device
> eal,ethdev: Add a function and function pointers to close ether device
> ethdev: Add functions that will be used by port hotplug functions
> eal/linux/pci: Add functions for unmapping igb_uio resources
> eal/pci: Add a function to remove the entry of devargs list
> eal/pci: Cleanup pci driver initialization code
> ethdev: Add one dev_type parameter to rte_eth_dev_allocate
> eal/pci: Add rte_eal_dev_attach/detach() functions
> eal: Enable port hotplug framework in Linux
> doc: Add port hotplug framework section to programmers guide
>
> app/test/virtual_pmd.c | 2 +-
> config/common_linuxapp | 5 +
> doc/guides/prog_guide/index.rst | 1 +
> doc/guides/prog_guide/port_hotplug_framework.rst | 110 ++++
> lib/librte_eal/bsdapp/eal/eal_pci.c | 29 +-
> lib/librte_eal/common/Makefile | 1 +
> lib/librte_eal/common/eal_common_dev.c | 276 ++++++++++
> lib/librte_eal/common/eal_common_devargs.c | 60 +++
> lib/librte_eal/common/eal_common_pci.c | 92 +++-
> lib/librte_eal/common/eal_private.h | 35 ++
> lib/librte_eal/common/include/rte_dev.h | 33 ++
> lib/librte_eal/common/include/rte_dev_hotplug.h | 44 ++
> lib/librte_eal/common/include/rte_devargs.h | 21 +
> lib/librte_eal/common/include/rte_pci.h | 84 +++
> lib/librte_eal/linuxapp/eal/Makefile | 1 +
> lib/librte_eal/linuxapp/eal/eal_pci.c | 232 +++++++--
> lib/librte_eal/linuxapp/eal/eal_pci_init.h | 8 +
> lib/librte_eal/linuxapp/eal/eal_pci_uio.c | 67 ++-
> lib/librte_eal/linuxapp/eal/rte_eal_version.map | 2 +
> lib/librte_ether/rte_ethdev.c | 636 +++++++++++++----------
> lib/librte_ether/rte_ethdev.h | 151 +++++-
> lib/librte_ether/rte_ether_version.map | 7 +
> lib/librte_pmd_af_packet/rte_eth_af_packet.c | 2 +-
> lib/librte_pmd_bond/rte_eth_bond_api.c | 2 +-
> lib/librte_pmd_pcap/rte_eth_pcap.c | 2 +-
> lib/librte_pmd_ring/rte_eth_ring.c | 2 +-
> lib/librte_pmd_xenvirt/rte_eth_xenvirt.c | 2 +-
> 27 files changed, 1560 insertions(+), 347 deletions(-)
> create mode 100644 doc/guides/prog_guide/port_hotplug_framework.rst
> create mode 100644 lib/librte_eal/common/include/rte_dev_hotplug.h
>
Acked-by: Michael Qiu <michael.qiu@intel.com>
On Mon, Feb 16, 2015 at 01:14:26PM +0900, Tetsuya Mukawa wrote:
> The patch adds following functions.
>
> - rte_eth_dev_save()
> The function is used for saving current rte_eth_dev structures.
> - rte_eth_dev_get_changed_port()
> The function receives the rte_eth_dev structures, then compare
> these with current values to know which port is actually
> attached or detached.
> - rte_eth_dev_get_addr_by_port()
> The function returns a pci address of an ethdev specified by port
> identifier.
> - rte_eth_dev_get_port_by_addr()
> The function returns a port identifier of an ethdev specified by
> pci address.
> - rte_eth_dev_get_name_by_port()
> The function returns a unique identifier name of an ethdev
> specified by port identifier.
> - Add rte_eth_dev_check_detachable()
> The function returns whether a PMD supports detach function.
>
> Also, the patch changes scope of rte_eth_dev_allocated() to global.
> This function will be called by virtual PMDs to support port hotplug.
> So change scope of the function to global.
>
> v8:
> - Add size parameter to rte_eth_dev_save().
> - Add missing symbol in version map.
> (Thanks to Qiu, Michael and Iremonger, Bernard)
> v7:
> - Add pt_driver checking to rte_eth_dev_check_detachable().
> (Thanks to Qiu, Michael)
> v5:
> - Fix return value of below functions.
> rte_eth_dev_get_changed_port().
> rte_eth_dev_get_port_by_addr().
> v4:
> - Add parameter checking.
> v3:
> - Fix if-condition bug while comparing pci addresses.
> - Add error checking codes.
> Reported-by: Mark Enright <menrigh@brocade.com>
>
> Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp>
> ---
> lib/librte_ether/rte_ethdev.c | 99 +++++++++++++++++++++++++++++++++-
> lib/librte_ether/rte_ethdev.h | 83 ++++++++++++++++++++++++++++
> lib/librte_ether/rte_ether_version.map | 6 +++
> 3 files changed, 187 insertions(+), 1 deletion(-)
>
> diff --git a/lib/librte_ether/rte_ethdev.c b/lib/librte_ether/rte_ethdev.c
> index 58d8072..3869a96 100644
> --- a/lib/librte_ether/rte_ethdev.c
> +++ b/lib/librte_ether/rte_ethdev.c
> @@ -206,7 +206,7 @@ rte_eth_dev_data_alloc(void)
> RTE_MAX_ETHPORTS * sizeof(*rte_eth_dev_data));
> }
>
> -static struct rte_eth_dev *
> +struct rte_eth_dev *
> rte_eth_dev_allocated(const char *name)
> {
> unsigned i;
> @@ -426,6 +426,103 @@ rte_eth_dev_count(void)
> return (nb_ports);
> }
>
> +int
> +rte_eth_dev_save(struct rte_eth_dev *devs, size_t size)
> +{
> + if ((devs == NULL) ||
> + (size != sizeof(struct rte_eth_dev) * RTE_MAX_ETHPORTS))
> + return -EINVAL;
> +
> + /* save current rte_eth_devices */
> + memcpy(devs, rte_eth_devices, size);
> + return 0;
> +}
> +
> +int
> +rte_eth_dev_get_changed_port(struct rte_eth_dev *devs, uint8_t *port_id)
> +{
> + if ((devs == NULL) || (port_id == NULL))
> + return -EINVAL;
> +
> + /* check which port was attached or detached */
> + for (*port_id = 0; *port_id < RTE_MAX_ETHPORTS; (*port_id)++, devs++) {
> + if (rte_eth_devices[*port_id].attached ^ devs->attached)
> + return 0;
> + }
> + return -ENODEV;
> +}
> +
> +int
> +rte_eth_dev_get_addr_by_port(uint8_t port_id, struct rte_pci_addr *addr)
> +{
> + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID)
> + return -EINVAL;
> +
> + if (addr == NULL) {
> + PMD_DEBUG_TRACE("Null pointer is specified\n");
> + return -EINVAL;
> + }
> +
> + *addr = rte_eth_devices[port_id].pci_dev->addr;
> + return 0;
> +}
> +
> +int
> +rte_eth_dev_get_port_by_addr(struct rte_pci_addr *addr, uint8_t *port_id)
> +{
> + struct rte_pci_addr *tmp;
> +
> + if ((addr == NULL) || (port_id == NULL)) {
> + PMD_DEBUG_TRACE("Null pointer is specified\n");
> + return -EINVAL;
> + }
> +
> + for (*port_id = 0; *port_id < RTE_MAX_ETHPORTS; (*port_id)++) {
> + if (!rte_eth_devices[*port_id].attached)
> + continue;
> + if (!rte_eth_devices[*port_id].pci_dev)
> + continue;
> + tmp = &rte_eth_devices[*port_id].pci_dev->addr;
> + if (eal_compare_pci_addr(tmp, addr) == 0)
> + return 0;
> + }
> + return -ENODEV;
> +}
> +
> +int
> +rte_eth_dev_get_name_by_port(uint8_t port_id, char *name)
> +{
> + char *tmp;
> +
> + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID)
> + return -EINVAL;
> +
> + if (name == NULL) {
> + PMD_DEBUG_TRACE("Null pointer is specified\n");
> + return -EINVAL;
> + }
> +
> + /* shouldn't check 'rte_eth_devices[i].data',
> + * because it might be overwritten by VDEV PMD */
> + tmp = rte_eth_dev_data[port_id].name;
> + strncpy(name, tmp, strlen(tmp) + 1);
> + return 0;
> +}
> +
> +int
> +rte_eth_dev_check_detachable(uint8_t port_id)
> +{
> + uint32_t drv_flags;
> +
> + if (port_id >= RTE_MAX_ETHPORTS) {
> + PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
> + return -EINVAL;
> + }
> +
> + drv_flags = rte_eth_devices[port_id].driver->pci_drv.drv_flags;
> + return !(drv_flags & RTE_PCI_DRV_DETACHABLE);
> +}
> +
> static int
> rte_eth_dev_rx_queue_config(struct rte_eth_dev *dev, uint16_t nb_queues)
> {
> diff --git a/lib/librte_ether/rte_ethdev.h b/lib/librte_ether/rte_ethdev.h
> index 91d9e86..6651890 100644
> --- a/lib/librte_ether/rte_ethdev.h
> +++ b/lib/librte_ether/rte_ethdev.h
> @@ -1616,6 +1616,89 @@ extern struct rte_eth_dev rte_eth_devices[];
> extern uint8_t rte_eth_dev_count(void);
>
> /**
> + * Function for internal use by port hotplug functions.
> + * Copies current ethdev structures to the specified pointer.
> + *
> + * @param devs The pointer to the ethdev structures
> + * @param size The size of ethdev structures
> + * @return
> + * - 0 on success, negative on error
> + */
> +extern int rte_eth_dev_save(struct rte_eth_dev *devs, size_t size);
> +
> +/**
> + * Function for internal use by port hotplug functions.
> + * Compare the specified ethdev structures with currents. Then
> + * if there is a port which status is changed, fill the specified pointer
> + * with the port id of that port.
> + * @param devs The pointer to the ethdev structures
> + * @param port_id The pointer to the port id
> + * @return
> + * - 0 on success, negative on error
> + */
> +extern int rte_eth_dev_get_changed_port(
> + struct rte_eth_dev *devs, uint8_t *port_id);
> +
> +/**
> + * Function for internal use by port hotplug functions.
> + * Returns a pci address of a ethdev specified by port identifier.
> + * @param port_id
> + * The port identifier of the Ethernet device
> + * @param addr
> + * The pointer to the pci address
> + * @return
> + * - 0 on success, negative on error
> + */
> +extern int rte_eth_dev_get_addr_by_port(
> + uint8_t port_id, struct rte_pci_addr *addr);
> +
> +/**
> + * Function for internal use by port hotplug functions.
> + * Returns a port identifier of a ethdev specified by pci address.
> + * @param addr
> + * The pointer to the pci address of the Ethernet device.
> + * @param port_id
> + * The pointer to the port identifier
> + * @return
> + * - 0 on success, negative on error
> + */
> +extern int rte_eth_dev_get_port_by_addr(
> + struct rte_pci_addr *addr, uint8_t *port_id);
> +
> +/**
> + * Function for internal use by port hotplug functions.
> + * Returns a unique identifier name of a ethdev specified by port identifier.
> + * @param port_id
> + * The port identifier.
> + * @param name
> + * The pointer to the Unique identifier name for each Ethernet device
> + * @return
> + * - 0 on success, negative on error
> + */
> +extern int rte_eth_dev_get_name_by_port(uint8_t port_id, char *name);
> +
> +/**
> + * Function for internal use by port hotplug functions.
> + * Check whether or not, a PMD that is handling the ethdev specified by port
> + * identifier can support detach function.
> + * @param port_id
> + * The port identifier
> + * @return
> + * - 0 on supporting detach function, negative on not supporting
> + */
> +extern int rte_eth_dev_check_detachable(uint8_t port_id);
> +
> +/**
> + * Function for internal use by port hotplug functions.
> + * Returns a ethdev slot specified by the unique identifier name.
> + * @param name
> + * The pointer to the Unique identifier name for each Ethernet device
> + * @return
> + * - The pointer to the ethdev slot, on success. NULL on error
> + */
> +extern struct rte_eth_dev *rte_eth_dev_allocated(const char *name);
> +
> +/**
> * Function for internal use by dummy drivers primarily, e.g. ring-based
> * driver.
> * Allocates a new ethdev slot for an ethernet device and returns the pointer
> diff --git a/lib/librte_ether/rte_ether_version.map b/lib/librte_ether/rte_ether_version.map
> index 7316530..fc5dc27 100644
> --- a/lib/librte_ether/rte_ether_version.map
> +++ b/lib/librte_ether/rte_ether_version.map
> @@ -109,6 +109,12 @@ DPDK_2.0 {
> rte_eth_tx_queue_setup;
> rte_eth_xstats_get;
> rte_eth_xstats_reset;
> + rte_eth_dev_check_detachable;
> + rte_eth_dev_get_name_by_port;
> + rte_eth_dev_get_addr_by_port;
> + rte_eth_dev_get_port_by_addr;
> + rte_eth_dev_get_changed_port;
> + rte_eth_dev_save;
rte_eth_dev_allocated needs to get added in here too
Neil
On Mon, Feb 16, 2015 at 01:14:27PM +0900, Tetsuya Mukawa wrote:
> The patch adds functions for unmapping igb_uio resources. The patch is only
> for Linux and igb_uio environment. VFIO and BSD are not supported.
>
> v8:
> - Fix typo.
> (Thanks to Iremonger, Bernard)
> v5:
> - Fix pci_unmap_device() to check pt_driver.
> v4:
> - Add parameter checking.
> - Add header file to determine if hotplug can be enabled.
>
> Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp>
> ---
> lib/librte_eal/common/Makefile | 1 +
> lib/librte_eal/common/include/rte_dev_hotplug.h | 44 +++++++++++++++++
> lib/librte_eal/linuxapp/eal/eal_pci.c | 44 +++++++++++++++++
> lib/librte_eal/linuxapp/eal/eal_pci_init.h | 8 +++
> lib/librte_eal/linuxapp/eal/eal_pci_uio.c | 65 +++++++++++++++++++++++++
> 5 files changed, 162 insertions(+)
> create mode 100644 lib/librte_eal/common/include/rte_dev_hotplug.h
>
> diff --git a/lib/librte_eal/common/Makefile b/lib/librte_eal/common/Makefile
> index 52c1a5f..db7cc93 100644
> --- a/lib/librte_eal/common/Makefile
> +++ b/lib/librte_eal/common/Makefile
> @@ -41,6 +41,7 @@ INC += rte_eal_memconfig.h rte_malloc_heap.h
> INC += rte_hexdump.h rte_devargs.h rte_dev.h
> INC += rte_common_vect.h
> INC += rte_pci_dev_feature_defs.h rte_pci_dev_features.h
> +INC += rte_dev_hotplug.h
>
> ifeq ($(CONFIG_RTE_INSECURE_FUNCTION_WARNING),y)
> INC += rte_warnings.h
> diff --git a/lib/librte_eal/common/include/rte_dev_hotplug.h b/lib/librte_eal/common/include/rte_dev_hotplug.h
> new file mode 100644
> index 0000000..b333e0f
> --- /dev/null
> +++ b/lib/librte_eal/common/include/rte_dev_hotplug.h
> @@ -0,0 +1,44 @@
> +/*-
> + * BSD LICENSE
> + *
> + * Copyright(c) 2015 IGEL Co.,LTd.
> + * All rights reserved.
> + *
> + * Redistribution and use in source and binary forms, with or without
> + * modification, are permitted provided that the following conditions
> + * are met:
> + *
> + * * Redistributions of source code must retain the above copyright
> + * notice, this list of conditions and the following disclaimer.
> + * * Redistributions in binary form must reproduce the above copyright
> + * notice, this list of conditions and the following disclaimer in
> + * the documentation and/or other materials provided with the
> + * distribution.
> + * * Neither the name of IGEL Co.,Ltd. nor the names of its
> + * contributors may be used to endorse or promote products derived
> + * from this software without specific prior written permission.
> + *
> + * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
> + * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
> + * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
> + * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
> + * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
> + * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
> + * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
> + * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
> + * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
> + * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
> + * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
> + */
> +
> +#ifndef _RTE_DEV_HOTPLUG_H_
> +#define _RTE_DEV_HOTPLUG_H_
> +
> +/*
> + * determine if hotplug can be enabled on the system
> + */
> +#if defined(RTE_LIBRTE_EAL_HOTPLUG) && defined(RTE_LIBRTE_EAL_LINUXAPP)
> +#define ENABLE_HOTPLUG
> +#endif /* RTE_LIBRTE_EAL_HOTPLUG & RTE_LIBRTE_EAL_LINUXAPP */
I'm not sure why you're doing this. Why not just test RTE_LIBRTE_EAL_HOTPLUG in
the various locations where your testing ENABLE_HOTPLUG? This seems like
indirection for the sake of indirection.
Neil
> -----Original Message-----
> From: Tetsuya Mukawa [mailto:mukawa@igel.co.jp]
> Sent: Monday, February 16, 2015 4:14 AM
> To: dev@dpdk.org
> Cc: Qiu, Michael; Iremonger, Bernard; Tetsuya Mukawa
> Subject: [PATCH v8 01/14] eal_pci: Add flag to hold kernel driver type
>
> From: Michael Qiu <michael.qiu@intel.com>
>
> Currently, dpdk has no ability to know which type of driver(
> vfio-pci/igb_uio/uio_pci_generic) the device used. It only can check whether vfio is enabled or not
> staticly.
>
> It really useful to have the flag, becasue different type need to handle differently in runtime. For
> example, pci memory map, pot hotplug, and so on.
>
> This patch add a flag field for pci device to solve above issue.
>
> Signed-off-by: Michael Qiu <michael.qiu@intel.com>
> Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp>
Acked-by: Bernard Iremonger <bernard.iremonger@intel.com>
> -----Original Message-----
> From: Tetsuya Mukawa [mailto:mukawa@igel.co.jp]
> Sent: Monday, February 16, 2015 4:14 AM
> To: dev@dpdk.org
> Cc: Qiu, Michael; Iremonger, Bernard; Tetsuya Mukawa
> Subject: [PATCH v8 02/14] eal_pci: pci memory map work with driver type
>
> From: Michael Qiu <michael.qiu@intel.com>
>
> With the driver type flag in struct rte_pci_dev, we do not need to always map uio devices with vfio
> related function when vfio enabled.
>
> Signed-off-by: Michael Qiu <michael.qiu@intel.com>
> Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp>
Acked-by: Bernard Iremonger <bernard.iremonger@intel.com>
> -----Original Message-----
> From: Tetsuya Mukawa [mailto:mukawa@igel.co.jp]
> Sent: Monday, February 16, 2015 4:14 AM
> To: dev@dpdk.org
> Cc: Qiu, Michael; Iremonger, Bernard; Tetsuya Mukawa
> Subject: [PATCH v8 03/14] eal/pci,ethdev: Remove assumption that port will not be detached
>
> To remove assumption, do like followings.
>
> This patch adds "RTE_PCI_DRV_DETACHABLE" to drv_flags of rte_pci_driver structure. The flags
> indicate the driver can detach devices at runtime.
> Also, remove assumption that port will not be detached.
>
> To remove the assumption.
> - Add 'attached' member to rte_eth_dev structure.
> This member is used for indicating the port is attached, or not.
> - Add rte_eth_dev_allocate_new_port().
> This function is used for allocating new port.
>
> v8:
> - NONE_TRACE is changed to NO_TRACE.
> (Thanks to Iremonger, Bernard)
> v5:
> - Change parameters of rte_eth_dev_validate_port() to cleanup code.
> v4:
> - Use braces with 'for' loop.
> - Fix indent of 'if' statement.
>
> Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp>
Acked-by: Bernard Iremonger <bernard.iremonger@intel.com>
> -----Original Message-----
> From: Tetsuya Mukawa [mailto:mukawa@igel.co.jp]
> Sent: Monday, February 16, 2015 4:14 AM
> To: dev@dpdk.org
> Cc: Qiu, Michael; Iremonger, Bernard; Tetsuya Mukawa
> Subject: [PATCH v8 04/14] eal/pci: Consolidate pci address comparison APIs
>
> This patch replaces pci_addr_comparison() and memcmp() of pci addresses by
> eal_compare_pci_addr().
>
> v8:
> - Fix pci_scan_one() to update sysfs values.
> (Thanks to Qiu, Michael and Iremonger, Bernard)
> v5:
> - Fix pci_scan_one to handle pt_driver correctly.
> v4:
> - Fix calculation method of eal_compare_pci_addr().
> - Add parameter checking.
>
> Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp>
Acked-by: Bernard Iremonger <bernard.iremonger@intel.com>
> -----Original Message-----
> From: Tetsuya Mukawa [mailto:mukawa@igel.co.jp]
> Sent: Monday, February 16, 2015 4:14 AM
> To: dev@dpdk.org
> Cc: Qiu, Michael; Iremonger, Bernard; Tetsuya Mukawa
> Subject: [PATCH v8 05/14] ethdev: Add rte_eth_dev_free to free specified device
>
> This patch adds rte_eth_dev_free(). The function is used for changing an attached status of the device
> that has specified name.
>
> v6:
> - Use rte_eth_dev structure as the paramter of rte_eth_dev_free().
> v4:
> - Add parameter checking.
>
> Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp>
Acked-by: Bernard Iremonger <bernard.iremonger@intel.com>
> -----Original Message-----
> From: Tetsuya Mukawa [mailto:mukawa@igel.co.jp]
> Sent: Monday, February 16, 2015 4:14 AM
> To: dev@dpdk.org
> Cc: Qiu, Michael; Iremonger, Bernard; Tetsuya Mukawa
> Subject: [PATCH v8 06/14] eal,ethdev: Add a function and function pointers to close ether device
>
> The patch adds function pointer to rte_pci_driver and eth_driver structure. These function pointers
> are used when ports are detached.
> Also, the patch adds rte_eth_dev_uninit(). So far, it's not called by anywhere, but it will be called
> when port hotplug function is implemented.
>
> v6:
> - Fix rte_eth_dev_uninit() to handle a return value of uninit
> function of PMD.
> v4:
> - Add parameter checking.
> - Change function names.
>
> Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp>
Acked-by: Bernard Iremonger <bernard.iremonger@intel.com>
> -----Original Message-----
> From: Tetsuya Mukawa [mailto:mukawa@igel.co.jp]
> Sent: Monday, February 16, 2015 4:14 AM
> To: dev@dpdk.org
> Cc: Qiu, Michael; Iremonger, Bernard; Tetsuya Mukawa
> Subject: [PATCH v8 07/14] ethdev: Add functions that will be used by port hotplug functions
>
> The patch adds following functions.
>
> - rte_eth_dev_save()
> The function is used for saving current rte_eth_dev structures.
> - rte_eth_dev_get_changed_port()
> The function receives the rte_eth_dev structures, then compare
> these with current values to know which port is actually
> attached or detached.
> - rte_eth_dev_get_addr_by_port()
> The function returns a pci address of an ethdev specified by port
> identifier.
> - rte_eth_dev_get_port_by_addr()
> The function returns a port identifier of an ethdev specified by
> pci address.
> - rte_eth_dev_get_name_by_port()
> The function returns a unique identifier name of an ethdev
> specified by port identifier.
> - Add rte_eth_dev_check_detachable()
> The function returns whether a PMD supports detach function.
>
> Also, the patch changes scope of rte_eth_dev_allocated() to global.
> This function will be called by virtual PMDs to support port hotplug.
> So change scope of the function to global.
>
> v8:
> - Add size parameter to rte_eth_dev_save().
> - Add missing symbol in version map.
> (Thanks to Qiu, Michael and Iremonger, Bernard)
> v7:
> - Add pt_driver checking to rte_eth_dev_check_detachable().
> (Thanks to Qiu, Michael)
> v5:
> - Fix return value of below functions.
> rte_eth_dev_get_changed_port().
> rte_eth_dev_get_port_by_addr().
> v4:
> - Add parameter checking.
> v3:
> - Fix if-condition bug while comparing pci addresses.
> - Add error checking codes.
> Reported-by: Mark Enright <menrigh@brocade.com>
>
> Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp>
Acked-by: Bernard Iremonger <bernard.iremonger@intel.com>
> -----Original Message-----
> From: Tetsuya Mukawa [mailto:mukawa@igel.co.jp]
> Sent: Monday, February 16, 2015 4:14 AM
> To: dev@dpdk.org
> Cc: Qiu, Michael; Iremonger, Bernard; Tetsuya Mukawa
> Subject: [PATCH v8 08/14] eal/linux/pci: Add functions for unmapping igb_uio resources
>
> The patch adds functions for unmapping igb_uio resources. The patch is only for Linux and igb_uio
> environment. VFIO and BSD are not supported.
>
> v8:
> - Fix typo.
> (Thanks to Iremonger, Bernard)
> v5:
> - Fix pci_unmap_device() to check pt_driver.
> v4:
> - Add parameter checking.
> - Add header file to determine if hotplug can be enabled.
>
> Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp>
Acked-by: Bernard Iremonger <bernard.iremonger@intel.com>
> -----Original Message-----
> From: Tetsuya Mukawa [mailto:mukawa@igel.co.jp]
> Sent: Monday, February 16, 2015 4:14 AM
> To: dev@dpdk.org
> Cc: Qiu, Michael; Iremonger, Bernard; Tetsuya Mukawa
> Subject: [PATCH v8 09/14] eal/pci: Add a function to remove the entry of devargs list
>
> The function removes the specified devargs entry from devargs_list.
> Also, the patch adds sanity checking to rte_eal_devargs_add().
>
> v5:
> - Change function definition of rte_eal_devargs_remove().
> v4:
> - Fix sanity check code.
>
> Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp>
Acked-by: Bernard Iremonger <bernard.iremonger@intel.com>
> -----Original Message-----
> From: Tetsuya Mukawa [mailto:mukawa@igel.co.jp]
> Sent: Monday, February 16, 2015 4:14 AM
> To: dev@dpdk.org
> Cc: Qiu, Michael; Iremonger, Bernard; Tetsuya Mukawa
> Subject: [PATCH v8 10/14] eal/pci: Cleanup pci driver initialization code
>
> - Add rte_eal_pci_close_one_dirver()
> The function is used for closing the specified driver and device.
> - Add pci_invoke_all_drivers()
> The function is based on pci_probe_all_drivers. But it can not only
> probe but also close drivers.
> - Add pci_close_all_drivers()
> The function tries to find a driver for the specified device, and
> then close the driver.
> - Add rte_eal_pci_probe_one() and rte_eal_pci_close_one()
> The functions are used for probe and close a device.
> First the function tries to find a device that has the specified
> PCI address. Then, probe or close the device.
>
> v5:
> - Remove RTE_EAL_INVOKE_TYPE_UNKNOWN, because it's unused.
> v4:
> - Fix parameter checking.
> - Fix indent of 'if' statement.
>
> Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp>
Acked-by: Bernard Iremonger <bernard.iremonger@intel.com>
> -----Original Message-----
> From: Tetsuya Mukawa [mailto:mukawa@igel.co.jp]
> Sent: Monday, February 16, 2015 4:15 AM
> To: dev@dpdk.org
> Cc: Qiu, Michael; Iremonger, Bernard; Tetsuya Mukawa
> Subject: [PATCH v8 11/14] ethdev: Add one dev_type parameter to rte_eth_dev_allocate
>
> This new parameter is needed to keep device type like physical or virtual.
> Port detaching processes are different between physical and virtual.
> This parameter lets detaching function know a device type of the port.
>
> v8:
> - NONE_TRACE is replaced by NO_TRACE.
> - Add missing symbol in version map.
> (Thanks to Qiu, Michael and Iremonger, Bernard)
> v4:
> - Fix comments of rte_eth_dev_type.
>
> Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp>
Acked-by: Bernard Iremonger <bernard.iremonger@intel.com>
> -----Original Message-----
> From: Tetsuya Mukawa [mailto:mukawa@igel.co.jp]
> Sent: Monday, February 16, 2015 4:15 AM
> To: dev@dpdk.org
> Cc: Qiu, Michael; Iremonger, Bernard; Tetsuya Mukawa
> Subject: [PATCH v8 12/14] eal/pci: Add rte_eal_dev_attach/detach() functions
>
> These functions are used for attaching or detaching a port.
> When rte_eal_dev_attach() is called, the function tries to realize the device name as pci address. If
> this is done successfully,
> rte_eal_dev_attach() will attach physical device port. If not, attaches virtual devive port.
> When rte_eal_dev_detach() is called, the function gets the device type of this port to know whether
> the port is come from physical or virtual.
> And then specific detaching function will be called.
>
> v8:
> - Add missing symbol in version map.
> (Thanks to Qiu, Michael and Iremonger, Bernard)
> v7:
> - Fix typo of warning messages.
> (Thanks to Qiu, Michael)
> v5:
> - Change function names like below.
> rte_eal_dev_find_and_invoke() to rte_eal_vdev_find_and_invoke().
> rte_eal_dev_invoke() to rte_eal_vdev_invoke().
> - Add code to handle a return value of rte_eal_devargs_remove().
> - Fix pci address format in rte_eal_dev_detach().
> v4:
> - Fix comment.
> - Add error checking.
> - Fix indent of 'if' statement.
> - Change function name.
>
> Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp>
Acked-by: Bernard Iremonger <bernard.iremonger@intel.com>
> -----Original Message-----
> From: Tetsuya Mukawa [mailto:mukawa@igel.co.jp]
> Sent: Monday, February 16, 2015 4:15 AM
> To: dev@dpdk.org
> Cc: Qiu, Michael; Iremonger, Bernard; Tetsuya Mukawa
> Subject: [PATCH v8 13/14] eal: Enable port hotplug framework in Linux
>
> The patch enables CONFIG_RTE_LIBRTE_EAL_HOTPLUG in Linux configuration.
>
> Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp>
Acked-by: Bernard Iremonger <bernard.iremonger@intel.com>
> -----Original Message-----
> From: Tetsuya Mukawa [mailto:mukawa@igel.co.jp]
> Sent: Monday, February 16, 2015 4:15 AM
> To: dev@dpdk.org
> Cc: Qiu, Michael; Iremonger, Bernard; Tetsuya Mukawa
> Subject: [PATCH v8 14/14] doc: Add port hotplug framework section to programmers guide
>
> This patch adds a new section for describing port hotplug framework.
>
> Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp>
Acked-by: Bernard Iremonger <bernard.iremonger@intel.com>
> -----Original Message-----
> From: Tetsuya Mukawa [mailto:mukawa@igel.co.jp]
> Sent: Monday, February 16, 2015 4:16 AM
> To: dev@dpdk.org
> Cc: Qiu, Michael; Iremonger, Bernard; Tetsuya Mukawa
> Subject: [PATCH v8] testpmd: Add port hotplug support
>
> The patch introduces following commands.
> - port attach [ident]
> - port detach [port_id]
> - attach: attaching a port
> - detach: detaching a port
> - ident: pci address of physical device.
> Or device name and parameters of virtual device.
> (ex. 0000:02:00.0, eth_pcap0,iface=eth0)
> - port_id: port identifier
>
> v7:
> - Fix doc.
> (Thanks to Iremonger, Bernard)
> - Fix port checking implementation of star_port();
> (Thanks to Qiu, Michael)
> v5:
> - Add testpmd documentation.
> (Thanks to Iremonger, Bernard)
> v4:
> - Fix strings of command help.
>
> Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp>
Acked-by: Bernard Iremonger <bernard.iremonger@intel.com>
> -----Original Message-----
> From: Tetsuya Mukawa [mailto:mukawa@igel.co.jp]
> Sent: Monday, February 16, 2015 4:15 AM
> To: dev@dpdk.org
> Cc: Qiu, Michael; Iremonger, Bernard; Tetsuya Mukawa
> Subject: [PATCH v8] librte_pmd_pcap: Add port hotplug support
>
> This patch adds finalization code to free resources allocated by the PMD.
>
> v6:
> - Fix a paramter of rte_eth_dev_free().
> v4:
> - Change function name.
>
> Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp>
Acked-by: Bernard Iremonger <bernard.iremonger@intel.com>
2015-02-16 13:14, Tetsuya Mukawa: > From: Michael Qiu <michael.qiu@intel.com> > > Currently, dpdk has no ability to know which type of driver( > vfio-pci/igb_uio/uio_pci_generic) the device used. It only can > check whether vfio is enabled or not staticly. > > It really useful to have the flag, becasue different type need to > handle differently in runtime. For example, pci memory map, > pot hotplug, and so on. > > This patch add a flag field for pci device to solve above issue. > > Signed-off-by: Michael Qiu <michael.qiu@intel.com> > Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp> > --- > lib/librte_eal/common/include/rte_pci.h | 8 +++++ > lib/librte_eal/linuxapp/eal/eal_pci.c | 53 +++++++++++++++++++++++++++++++-- > 2 files changed, 59 insertions(+), 2 deletions(-) > > diff --git a/lib/librte_eal/common/include/rte_pci.h b/lib/librte_eal/common/include/rte_pci.h > index 66ed793..7b48b55 100644 > --- a/lib/librte_eal/common/include/rte_pci.h > +++ b/lib/librte_eal/common/include/rte_pci.h > @@ -139,6 +139,13 @@ struct rte_pci_addr { > > struct rte_devargs; > > +enum rte_pt_driver { > + RTE_PT_UNKNOWN = 0, > + RTE_PT_IGB_UIO = 1, > + RTE_PT_VFIO = 2, > + RTE_PT_UIO_GENERIC = 3, > +}; What means PT? > + > /** > * A structure describing a PCI device. > */ > @@ -152,6 +159,7 @@ struct rte_pci_device { > uint16_t max_vfs; /**< sriov enable if not zero */ > int numa_node; /**< NUMA node connection */ > struct rte_devargs *devargs; /**< Device user arguments */ > + enum rte_pt_driver pt_driver; /**< Driver of passthrough */ I'm not sure passtrough is the right word for this kind of driver. What about "kernel driver"? > }; > > /** Any PCI device identifier (vendor, device, ...) */ > diff --git a/lib/librte_eal/linuxapp/eal/eal_pci.c b/lib/librte_eal/linuxapp/eal/eal_pci.c > index 15db9c4..e760452 100644 > --- a/lib/librte_eal/linuxapp/eal/eal_pci.c > +++ b/lib/librte_eal/linuxapp/eal/eal_pci.c > @@ -97,6 +97,35 @@ error: > return -1; > } > > +static int > +pci_get_kernel_driver_by_path(const char *filename, char *dri_name) > +{ > + int count; > + char path[PATH_MAX]; > + char *name; > + > + if (!filename || !dri_name) > + return -1; > + > + count = readlink(filename, path, PATH_MAX); > + if (count >= PATH_MAX) > + return -1; > + > + /* For device does not have a driver */ > + if (count < 0) > + return 1; > + > + path[count] = '\0'; > + > + name = strrchr(path, '/'); > + if (name) { > + strncpy(dri_name, name + 1, strlen(name + 1) + 1); > + return 0; > + } > + > + return -1; > +} > + > void * > pci_find_max_end_va(void) > { > @@ -222,11 +251,12 @@ pci_scan_one(const char *dirname, uint16_t domain, uint8_t bus, > char filename[PATH_MAX]; > unsigned long tmp; > struct rte_pci_device *dev; > + char driver[PATH_MAX]; > + int ret; > > dev = malloc(sizeof(*dev)); > - if (dev == NULL) { > + if (dev == NULL) > return -1; > - } > > memset(dev, 0, sizeof(*dev)); > dev->addr.domain = domain; > @@ -305,6 +335,25 @@ pci_scan_one(const char *dirname, uint16_t domain, uint8_t bus, > return -1; > } > > + /* parse driver */ > + snprintf(filename, sizeof(filename), "%s/driver", dirname); > + ret = pci_get_kernel_driver_by_path(filename, driver); > + if (!ret) { > + if (!strcmp(driver, "vfio-pci")) > + dev->pt_driver = RTE_PT_VFIO; > + else if (!strcmp(driver, "igb_uio")) > + dev->pt_driver = RTE_PT_IGB_UIO; > + else if (!strcmp(driver, "uio_pci_generic")) > + dev->pt_driver = RTE_PT_UIO_GENERIC; > + else > + dev->pt_driver = RTE_PT_UNKNOWN; > + } else if (ret < 0) { > + RTE_LOG(ERR, EAL, "Fail to get kernel driver\n"); > + free(dev); > + return -1; > + } else > + dev->pt_driver = RTE_PT_UNKNOWN; Why not considering a NONE value? Example: for virtio with port I/O. > + > /* device is valid, add in list (sorted) */ > if (TAILQ_EMPTY(&pci_device_list)) { > TAILQ_INSERT_TAIL(&pci_device_list, dev, next); >
2015-02-16 13:14, Tetsuya Mukawa: > To remove assumption, do like followings. > > This patch adds "RTE_PCI_DRV_DETACHABLE" to drv_flags of rte_pci_driver > structure. The flags indicate the driver can detach devices at runtime. > Also, remove assumption that port will not be detached. > > To remove the assumption. > - Add 'attached' member to rte_eth_dev structure. > This member is used for indicating the port is attached, or not. > - Add rte_eth_dev_allocate_new_port(). > This function is used for allocating new port. > > v8: > - NONE_TRACE is changed to NO_TRACE. > (Thanks to Iremonger, Bernard) > v5: > - Change parameters of rte_eth_dev_validate_port() to cleanup code. > v4: > - Use braces with 'for' loop. > - Fix indent of 'if' statement. > > Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp> > --- > lib/librte_eal/common/include/rte_pci.h | 2 + > lib/librte_ether/rte_ethdev.c | 454 +++++++++++++------------------- > lib/librte_ether/rte_ethdev.h | 5 + > 3 files changed, 186 insertions(+), 275 deletions(-) > > diff --git a/lib/librte_eal/common/include/rte_pci.h b/lib/librte_eal/common/include/rte_pci.h > index 7b48b55..7f2d699 100644 > --- a/lib/librte_eal/common/include/rte_pci.h > +++ b/lib/librte_eal/common/include/rte_pci.h > @@ -207,6 +207,8 @@ struct rte_pci_driver { > #define RTE_PCI_DRV_FORCE_UNBIND 0x0004 > /** Device driver supports link state interrupt */ > #define RTE_PCI_DRV_INTR_LSC 0x0008 > +/** Device driver supports detaching capability */ > +#define RTE_PCI_DRV_DETACHABLE 0x0010 > > /**< Internal use only - Macro used by pci addr parsing functions **/ > #define GET_PCIADDR_FIELD(in, fd, lim, dlm) \ > diff --git a/lib/librte_ether/rte_ethdev.c b/lib/librte_ether/rte_ethdev.c > index ea3a1fb..a79fa5b 100644 > --- a/lib/librte_ether/rte_ethdev.c > +++ b/lib/librte_ether/rte_ethdev.c > @@ -175,6 +175,16 @@ enum { > STAT_QMAP_RX > }; > > +enum { > + DEV_INVALID = 0, > + DEV_VALID, > +}; > + > +enum { > + DEV_DISCONNECTED = 0, > + DEV_CONNECTED > +}; The commit log explains what is an attached port but not what means valid or connected. These enums should have a comment to explain their usage. > + > static inline void > rte_eth_dev_data_alloc(void) > { > @@ -201,19 +211,34 @@ rte_eth_dev_allocated(const char *name) > { > unsigned i; > > - for (i = 0; i < nb_ports; i++) { > - if (strcmp(rte_eth_devices[i].data->name, name) == 0) > + for (i = 0; i < RTE_MAX_ETHPORTS; i++) { > + if ((rte_eth_devices[i].attached == DEV_CONNECTED) && > + strcmp(rte_eth_devices[i].data->name, name) == 0) > return &rte_eth_devices[i]; > } > return NULL; > } > > +static uint8_t > +rte_eth_dev_allocate_new_port(void) > +{ > + unsigned i; > + > + for (i = 0; i < RTE_MAX_ETHPORTS; i++) { > + if (rte_eth_devices[i].attached == DEV_DISCONNECTED) > + return i; > + } > + return RTE_MAX_ETHPORTS; > +} This function does not allocate a new port. It get the first free port id. Is uint8_t sill a good size for hotpluggable virtual device ids? > + > struct rte_eth_dev * > rte_eth_dev_allocate(const char *name) > { > + uint8_t port_id; > struct rte_eth_dev *eth_dev; > > - if (nb_ports == RTE_MAX_ETHPORTS) { > + port_id = rte_eth_dev_allocate_new_port(); > + if (port_id == RTE_MAX_ETHPORTS) { > PMD_DEBUG_TRACE("Reached maximum number of Ethernet ports\n"); > return NULL; > } > @@ -226,10 +251,12 @@ rte_eth_dev_allocate(const char *name) > return NULL; > } > > - eth_dev = &rte_eth_devices[nb_ports]; > - eth_dev->data = &rte_eth_dev_data[nb_ports]; > + eth_dev = &rte_eth_devices[port_id]; > + eth_dev->data = &rte_eth_dev_data[port_id]; > snprintf(eth_dev->data->name, sizeof(eth_dev->data->name), "%s", name); > - eth_dev->data->port_id = nb_ports++; > + eth_dev->data->port_id = port_id; > + eth_dev->attached = DEV_CONNECTED; > + nb_ports++; > return eth_dev; > } > > @@ -283,6 +310,7 @@ rte_eth_dev_init(struct rte_pci_driver *pci_drv, > (unsigned) pci_dev->id.device_id); > if (rte_eal_process_type() == RTE_PROC_PRIMARY) > rte_free(eth_dev->data->dev_private); > + eth_dev->attached = DEV_DISCONNECTED; > nb_ports--; > return diag; > } > @@ -308,10 +336,28 @@ rte_eth_driver_register(struct eth_driver *eth_drv) > rte_eal_pci_register(ð_drv->pci_drv); > } > > +enum { > + NO_TRACE = 0, > + TRACE > +}; What means this enum? > + > +static int > +rte_eth_dev_validate_port(uint8_t port_id, int trace) > +{ > + if (port_id >= RTE_MAX_ETHPORTS || > + rte_eth_devices[port_id].attached != DEV_CONNECTED) { > + if (trace) { > + PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); > + } It's not common in DPDK to have a parameter to toggle log. It should be enabled globally via the log level. > + return DEV_INVALID; It would be simpler to return 0 in a function called "is_valid_port". > + } else > + return DEV_VALID; So it could 1 (true) here. [...] > --- a/lib/librte_ether/rte_ethdev.h > +++ b/lib/librte_ether/rte_ethdev.h > @@ -1539,6 +1539,7 @@ struct rte_eth_dev { > struct eth_dev_ops *dev_ops; /**< Functions exported by PMD */ > struct rte_pci_device *pci_dev; /**< PCI info. supplied by probing */ > struct rte_eth_dev_cb_list callbacks; /**< User application callbacks */ > + uint8_t attached; /**< Flag indicating the port is attached */ > }; > > struct rte_eth_dev_sriov { > @@ -1604,6 +1605,10 @@ extern struct rte_eth_dev rte_eth_devices[]; > * initialized by the [matching] Ethernet driver during the PCI probing phase. > * All devices whose port identifier is in the range > * [0, rte_eth_dev_count() - 1] can be operated on by network applications. > + * immediately after invoking rte_eal_init(). > + * If the application unplugs a port using hotplug function, The enabled port > + * numbers may be noncontiguous. In the case, the applications need to manage > + * enabled port by themselves. > * > * @return > * - The total number of usable Ethernet devices. >
2015-02-16 13:14, Tetsuya Mukawa: > This patch replaces pci_addr_comparison() and memcmp() of pci addresses by > eal_compare_pci_addr(). > > v8: > - Fix pci_scan_one() to update sysfs values. > (Thanks to Qiu, Michael and Iremonger, Bernard) > v5: > - Fix pci_scan_one to handle pt_driver correctly. > v4: > - Fix calculation method of eal_compare_pci_addr(). > - Add parameter checking. > > Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp> > --- > lib/librte_eal/bsdapp/eal/eal_pci.c | 29 ++++++++++++-------------- > lib/librte_eal/common/eal_common_pci.c | 2 +- > lib/librte_eal/common/include/rte_pci.h | 34 +++++++++++++++++++++++++++++++ > lib/librte_eal/linuxapp/eal/eal_pci.c | 30 +++++++++++++-------------- > lib/librte_eal/linuxapp/eal/eal_pci_uio.c | 2 +- > 5 files changed, 63 insertions(+), 34 deletions(-) > > diff --git a/lib/librte_eal/bsdapp/eal/eal_pci.c b/lib/librte_eal/bsdapp/eal/eal_pci.c > index 74ecce7..7dbdccd 100644 > --- a/lib/librte_eal/bsdapp/eal/eal_pci.c > +++ b/lib/librte_eal/bsdapp/eal/eal_pci.c > @@ -270,20 +270,6 @@ pci_uio_map_resource(struct rte_pci_device *dev) > return (0); > } > > -/* Compare two PCI device addresses. */ > -static int > -pci_addr_comparison(struct rte_pci_addr *addr, struct rte_pci_addr *addr2) > -{ > - uint64_t dev_addr = (addr->domain << 24) + (addr->bus << 16) + (addr->devid << 8) + addr->function; > - uint64_t dev_addr2 = (addr2->domain << 24) + (addr2->bus << 16) + (addr2->devid << 8) + addr2->function; > - > - if (dev_addr > dev_addr2) > - return 1; > - else > - return 0; > -} > - > - > /* Scan one pci sysfs entry, and fill the devices list from it. */ > static int > pci_scan_one(int dev_pci_fd, struct pci_conf *conf) > @@ -356,13 +342,24 @@ pci_scan_one(int dev_pci_fd, struct pci_conf *conf) > } > else { > struct rte_pci_device *dev2 = NULL; > + int ret; > > TAILQ_FOREACH(dev2, &pci_device_list, next) { > - if (pci_addr_comparison(&dev->addr, &dev2->addr)) > + ret = eal_compare_pci_addr(&dev->addr, &dev2->addr); > + if (ret > 0) > continue; > - else { > + else if (ret < 0) { > TAILQ_INSERT_BEFORE(dev2, dev, next); > return 0; > + } else { /* already registered */ > + /* update pt_driver */ > + dev2->pt_driver = dev->pt_driver; > + dev2->max_vfs = dev->max_vfs; > + memmove(dev2->mem_resource, > + dev->mem_resource, > + sizeof(dev->mem_resource)); > + free(dev); > + return 0; Could you comment this "else part" please? [...] > --- a/lib/librte_eal/common/include/rte_pci.h > +++ b/lib/librte_eal/common/include/rte_pci.h > @@ -269,6 +269,40 @@ eal_parse_pci_DomBDF(const char *input, struct rte_pci_addr *dev_addr) > } > #undef GET_PCIADDR_FIELD > > +/* Compare two PCI device addresses. */ > +/** > + * Utility function to compare two PCI device addresses. > + * > + * @param addr > + * The PCI Bus-Device-Function address to compare > + * @param addr2 > + * The PCI Bus-Device-Function address to compare > + * @return > + * 0 on equal PCI address. > + * Positive on addr is greater than addr2. > + * Negative on addr is less than addr2, or error. > + */ > +static inline int > +eal_compare_pci_addr(struct rte_pci_addr *addr, struct rte_pci_addr *addr2) I think that this function should be prefixed with rte_ [...] > /* skip this element if it doesn't match our PCI address */ > - if (memcmp(&uio_res->pci_addr, &dev->addr, sizeof(dev->addr))) > + if (eal_compare_pci_addr(&uio_res->pci_addr, &dev->addr)) Why memcmp is not sufficient to compare PCI addresses? The only exception I see is endianness for natural sorting. > continue; > > for (i = 0; i != uio_res->nb_maps; i++) { >
2015-02-16 13:14, Tetsuya Mukawa: > This patch adds rte_eth_dev_free(). The function is used for changing an > attached status of the device that has specified name. > > v6: > - Use rte_eth_dev structure as the paramter of rte_eth_dev_free(). > v4: > - Add parameter checking. > > Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp> > --- > lib/librte_ether/rte_ethdev.c | 11 +++++++++++ > lib/librte_ether/rte_ethdev.h | 14 ++++++++++++++ > 2 files changed, 25 insertions(+) > > diff --git a/lib/librte_ether/rte_ethdev.c b/lib/librte_ether/rte_ethdev.c > index a79fa5b..2463d18 100644 > --- a/lib/librte_ether/rte_ethdev.c > +++ b/lib/librte_ether/rte_ethdev.c > @@ -260,6 +260,17 @@ rte_eth_dev_allocate(const char *name) > return eth_dev; > } > > +int > +rte_eth_dev_free(struct rte_eth_dev *eth_dev) > +{ > + if (eth_dev == NULL) > + return -EINVAL; > + > + eth_dev->attached = 0; > + nb_ports--; > + return 0; > +} This function is strange. I would imagine it calling the free (uninit) function of the driver. [...] > +/** > + * Function for internal use by dummy drivers primarily, e.g. ring-based > + * driver. > + * Free the specified ethdev. > + * > + * @param eth_dev > + * The *eth_dev* pointer is the address of the *rte_eth_dev* structure > + * associated with the matching device and which have been [automatically] > + * allocated in the *rte_eth_devices* array. > + * @return > + * - 0 on success, negative on error > + */ > +int rte_eth_dev_free(struct rte_eth_dev *eth_dev); > + > struct eth_driver; > /** > * @internal >
2015-02-16 13:14, Tetsuya Mukawa: > The patch adds function pointer to rte_pci_driver and eth_driver > structure. These function pointers are used when ports are detached. > Also, the patch adds rte_eth_dev_uninit(). So far, it's not called > by anywhere, but it will be called when port hotplug function is > implemented. > > v6: > - Fix rte_eth_dev_uninit() to handle a return value of uninit > function of PMD. > v4: > - Add parameter checking. > - Change function names. > > Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp> > --- > lib/librte_eal/common/include/rte_pci.h | 7 +++++ > lib/librte_ether/rte_ethdev.c | 47 +++++++++++++++++++++++++++++++++ > lib/librte_ether/rte_ethdev.h | 24 +++++++++++++++++ > 3 files changed, 78 insertions(+) > > diff --git a/lib/librte_eal/common/include/rte_pci.h b/lib/librte_eal/common/include/rte_pci.h > index 4814cd7..87ca4cf 100644 > --- a/lib/librte_eal/common/include/rte_pci.h > +++ b/lib/librte_eal/common/include/rte_pci.h > @@ -189,12 +189,19 @@ struct rte_pci_driver; > typedef int (pci_devinit_t)(struct rte_pci_driver *, struct rte_pci_device *); > > /** > + * Uninitialisation function for the driver called during hotplugging. > + */ > +typedef int (pci_devuninit_t)( > + struct rte_pci_driver *, struct rte_pci_device *); > + > +/** > * A structure describing a PCI driver. > */ > struct rte_pci_driver { > TAILQ_ENTRY(rte_pci_driver) next; /**< Next in list. */ > const char *name; /**< Driver name. */ > pci_devinit_t *devinit; /**< Device init. function. */ > + pci_devuninit_t *devuninit; /**< Device uninit function. */ > struct rte_pci_id *id_table; /**< ID table, NULL terminated. */ > uint32_t drv_flags; /**< Flags contolling handling of device. */ > }; > diff --git a/lib/librte_ether/rte_ethdev.c b/lib/librte_ether/rte_ethdev.c > index 2463d18..58d8072 100644 > --- a/lib/librte_ether/rte_ethdev.c > +++ b/lib/librte_ether/rte_ethdev.c > @@ -326,6 +326,52 @@ rte_eth_dev_init(struct rte_pci_driver *pci_drv, > return diag; > } > > +static int > +rte_eth_dev_uninit(struct rte_pci_driver *pci_drv, > + struct rte_pci_device *pci_dev) There's something strange here: this is an uninit of an ethdev but the parameter is a pci_dev. The driver parameter shouldn't be needed. > +{ > + struct eth_driver *eth_drv; > + struct rte_eth_dev *eth_dev; > + char ethdev_name[RTE_ETH_NAME_MAX_LEN]; > + int ret; > + > + if ((pci_drv == NULL) || (pci_dev == NULL)) > + return -EINVAL; > + > + /* Create unique Ethernet device name using PCI address */ > + snprintf(ethdev_name, RTE_ETH_NAME_MAX_LEN, "%d:%d.%d", > + pci_dev->addr.bus, pci_dev->addr.devid, > + pci_dev->addr.function); You should call a function common to init and uninit to generate the same unique name. > + > + eth_dev = rte_eth_dev_allocated(ethdev_name); > + if (eth_dev == NULL) > + return -ENODEV; > + > + eth_drv = (struct eth_driver *)pci_drv; > + > + /* Invoke PMD device uninit function */ > + if (*eth_drv->eth_dev_uninit) { > + ret = (*eth_drv->eth_dev_uninit)(eth_drv, eth_dev); > + if (ret) > + return ret; > + } > + > + /* free ether device */ > + rte_eth_dev_free(eth_dev); > + > + /* init user callbacks */ > + TAILQ_INIT(&(eth_dev->callbacks)); Please comment more why you are resetting callbacks. An init in an uninit function seems weird ;) > + > + if (rte_eal_process_type() == RTE_PROC_PRIMARY) > + rte_free(eth_dev->data->dev_private); > + > + eth_dev->pci_dev = NULL; > + eth_dev->driver = NULL; > + eth_dev->data = NULL; > + > + return 0; > +} > + > /** > * Register an Ethernet [Poll Mode] driver. > * > @@ -344,6 +390,7 @@ void > rte_eth_driver_register(struct eth_driver *eth_drv) > { > eth_drv->pci_drv.devinit = rte_eth_dev_init; > + eth_drv->pci_drv.devuninit = rte_eth_dev_uninit; > rte_eal_pci_register(ð_drv->pci_drv); > } > > diff --git a/lib/librte_ether/rte_ethdev.h b/lib/librte_ether/rte_ethdev.h > index fbe7ac1..91d9e86 100644 > --- a/lib/librte_ether/rte_ethdev.h > +++ b/lib/librte_ether/rte_ethdev.h > @@ -1678,6 +1678,27 @@ typedef int (*eth_dev_init_t)(struct eth_driver *eth_drv, > > /** > * @internal > + * Finalization function of an Ethernet driver invoked for each matching > + * Ethernet PCI device detected during the PCI closing phase. > + * > + * @param eth_drv > + * The pointer to the [matching] Ethernet driver structure supplied by > + * the PMD when it registered itself. > + * @param eth_dev > + * The *eth_dev* pointer is the address of the *rte_eth_dev* structure > + * associated with the matching device and which have been [automatically] > + * allocated in the *rte_eth_devices* array. > + * @return > + * - 0: Success, the device is properly finalized by the driver. > + * In particular, the driver MUST free the *dev_ops* pointer > + * of the *eth_dev* structure. > + * - <0: Error code of the device initialization failure. > + */ > +typedef int (*eth_dev_uninit_t)(struct eth_driver *eth_drv, > + struct rte_eth_dev *eth_dev); > + > +/** > + * @internal > * The structure associated with a PMD Ethernet driver. > * > * Each Ethernet driver acts as a PCI driver and is represented by a generic > @@ -1687,11 +1708,14 @@ typedef int (*eth_dev_init_t)(struct eth_driver *eth_drv, > * > * - The *eth_dev_init* function invoked for each matching PCI device. > * > + * - The *eth_dev_uninit* function invoked for each matching PCI device. > + * > * - The size of the private data to allocate for each matching device. > */ > struct eth_driver { > struct rte_pci_driver pci_drv; /**< The PMD is also a PCI driver. */ > eth_dev_init_t eth_dev_init; /**< Device init function. */ > + eth_dev_uninit_t eth_dev_uninit; /**< Device uninit function. */ > unsigned int dev_private_size; /**< Size of device private data. */ > }; > >
2015-02-16 13:14, Tetsuya Mukawa: > The patch adds following functions. > > - rte_eth_dev_save() > The function is used for saving current rte_eth_dev structures. > - rte_eth_dev_get_changed_port() > The function receives the rte_eth_dev structures, then compare > these with current values to know which port is actually > attached or detached. > - rte_eth_dev_get_addr_by_port() > The function returns a pci address of an ethdev specified by port > identifier. > - rte_eth_dev_get_port_by_addr() > The function returns a port identifier of an ethdev specified by > pci address. > - rte_eth_dev_get_name_by_port() > The function returns a unique identifier name of an ethdev > specified by port identifier. > - Add rte_eth_dev_check_detachable() > The function returns whether a PMD supports detach function. > > Also, the patch changes scope of rte_eth_dev_allocated() to global. > This function will be called by virtual PMDs to support port hotplug. > So change scope of the function to global. > > v8: > - Add size parameter to rte_eth_dev_save(). > - Add missing symbol in version map. > (Thanks to Qiu, Michael and Iremonger, Bernard) > v7: > - Add pt_driver checking to rte_eth_dev_check_detachable(). > (Thanks to Qiu, Michael) > v5: > - Fix return value of below functions. > rte_eth_dev_get_changed_port(). > rte_eth_dev_get_port_by_addr(). > v4: > - Add parameter checking. > v3: > - Fix if-condition bug while comparing pci addresses. > - Add error checking codes. > Reported-by: Mark Enright <menrigh@brocade.com> > > Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp> > --- > lib/librte_ether/rte_ethdev.c | 99 +++++++++++++++++++++++++++++++++- > lib/librte_ether/rte_ethdev.h | 83 ++++++++++++++++++++++++++++ > lib/librte_ether/rte_ether_version.map | 6 +++ > 3 files changed, 187 insertions(+), 1 deletion(-) > > diff --git a/lib/librte_ether/rte_ethdev.c b/lib/librte_ether/rte_ethdev.c > index 58d8072..3869a96 100644 > --- a/lib/librte_ether/rte_ethdev.c > +++ b/lib/librte_ether/rte_ethdev.c > @@ -206,7 +206,7 @@ rte_eth_dev_data_alloc(void) > RTE_MAX_ETHPORTS * sizeof(*rte_eth_dev_data)); > } > > -static struct rte_eth_dev * > +struct rte_eth_dev * > rte_eth_dev_allocated(const char *name) > { > unsigned i; > @@ -426,6 +426,103 @@ rte_eth_dev_count(void) > return (nb_ports); > } > > +int > +rte_eth_dev_save(struct rte_eth_dev *devs, size_t size) > +{ > + if ((devs == NULL) || > + (size != sizeof(struct rte_eth_dev) * RTE_MAX_ETHPORTS)) > + return -EINVAL; > + > + /* save current rte_eth_devices */ > + memcpy(devs, rte_eth_devices, size); > + return 0; > +} Why creating a function for a memcpy? > + > +int > +rte_eth_dev_get_changed_port(struct rte_eth_dev *devs, uint8_t *port_id) > +{ > + if ((devs == NULL) || (port_id == NULL)) > + return -EINVAL; > + > + /* check which port was attached or detached */ > + for (*port_id = 0; *port_id < RTE_MAX_ETHPORTS; (*port_id)++, devs++) { > + if (rte_eth_devices[*port_id].attached ^ devs->attached) > + return 0; > + } > + return -ENODEV; > +} It seems weird to require this function. Functions which are attaching a new port should return the port_id. > + > +int > +rte_eth_dev_get_addr_by_port(uint8_t port_id, struct rte_pci_addr *addr) > +{ > + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) > + return -EINVAL; > + > + if (addr == NULL) { > + PMD_DEBUG_TRACE("Null pointer is specified\n"); > + return -EINVAL; > + } > + > + *addr = rte_eth_devices[port_id].pci_dev->addr; > + return 0; > +} Again, I'm not sure this function is needed. > + > +int > +rte_eth_dev_get_port_by_addr(struct rte_pci_addr *addr, uint8_t *port_id) > +{ > + struct rte_pci_addr *tmp; > + > + if ((addr == NULL) || (port_id == NULL)) { > + PMD_DEBUG_TRACE("Null pointer is specified\n"); > + return -EINVAL; > + } > + > + for (*port_id = 0; *port_id < RTE_MAX_ETHPORTS; (*port_id)++) { > + if (!rte_eth_devices[*port_id].attached) > + continue; > + if (!rte_eth_devices[*port_id].pci_dev) > + continue; > + tmp = &rte_eth_devices[*port_id].pci_dev->addr; > + if (eal_compare_pci_addr(tmp, addr) == 0) > + return 0; > + } > + return -ENODEV; > +} > + > +int > +rte_eth_dev_get_name_by_port(uint8_t port_id, char *name) > +{ > + char *tmp; > + > + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) > + return -EINVAL; > + > + if (name == NULL) { > + PMD_DEBUG_TRACE("Null pointer is specified\n"); > + return -EINVAL; > + } > + > + /* shouldn't check 'rte_eth_devices[i].data', > + * because it might be overwritten by VDEV PMD */ I don't understand the comment. > + tmp = rte_eth_dev_data[port_id].name; > + strncpy(name, tmp, strlen(tmp) + 1); tmp seems useless. strncpy with strlen should be equivalent to strcpy. > + return 0; > +} > + > +int > +rte_eth_dev_check_detachable(uint8_t port_id) Why not "is_detachable" instead of "check_detachable"? > +{ > + uint32_t drv_flags; > + > + if (port_id >= RTE_MAX_ETHPORTS) { > + PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); > + return -EINVAL; > + } > + > + drv_flags = rte_eth_devices[port_id].driver->pci_drv.drv_flags; > + return !(drv_flags & RTE_PCI_DRV_DETACHABLE); > +} > + > static int > rte_eth_dev_rx_queue_config(struct rte_eth_dev *dev, uint16_t nb_queues) > { > diff --git a/lib/librte_ether/rte_ethdev.h b/lib/librte_ether/rte_ethdev.h > index 91d9e86..6651890 100644 > --- a/lib/librte_ether/rte_ethdev.h > +++ b/lib/librte_ether/rte_ethdev.h > @@ -1616,6 +1616,89 @@ extern struct rte_eth_dev rte_eth_devices[]; > extern uint8_t rte_eth_dev_count(void); > > /** > + * Function for internal use by port hotplug functions. > + * Copies current ethdev structures to the specified pointer. > + * > + * @param devs The pointer to the ethdev structures > + * @param size The size of ethdev structures > + * @return > + * - 0 on success, negative on error > + */ > +extern int rte_eth_dev_save(struct rte_eth_dev *devs, size_t size); > + > +/** > + * Function for internal use by port hotplug functions. > + * Compare the specified ethdev structures with currents. Then > + * if there is a port which status is changed, fill the specified pointer > + * with the port id of that port. > + * @param devs The pointer to the ethdev structures > + * @param port_id The pointer to the port id > + * @return > + * - 0 on success, negative on error > + */ > +extern int rte_eth_dev_get_changed_port( > + struct rte_eth_dev *devs, uint8_t *port_id); > + > +/** > + * Function for internal use by port hotplug functions. > + * Returns a pci address of a ethdev specified by port identifier. > + * @param port_id > + * The port identifier of the Ethernet device > + * @param addr > + * The pointer to the pci address > + * @return > + * - 0 on success, negative on error > + */ > +extern int rte_eth_dev_get_addr_by_port( > + uint8_t port_id, struct rte_pci_addr *addr); > + > +/** > + * Function for internal use by port hotplug functions. > + * Returns a port identifier of a ethdev specified by pci address. > + * @param addr > + * The pointer to the pci address of the Ethernet device. > + * @param port_id > + * The pointer to the port identifier > + * @return > + * - 0 on success, negative on error > + */ > +extern int rte_eth_dev_get_port_by_addr( > + struct rte_pci_addr *addr, uint8_t *port_id); > + > +/** > + * Function for internal use by port hotplug functions. > + * Returns a unique identifier name of a ethdev specified by port identifier. > + * @param port_id > + * The port identifier. > + * @param name > + * The pointer to the Unique identifier name for each Ethernet device > + * @return > + * - 0 on success, negative on error > + */ > +extern int rte_eth_dev_get_name_by_port(uint8_t port_id, char *name); > + > +/** > + * Function for internal use by port hotplug functions. > + * Check whether or not, a PMD that is handling the ethdev specified by port > + * identifier can support detach function. > + * @param port_id > + * The port identifier > + * @return > + * - 0 on supporting detach function, negative on not supporting > + */ > +extern int rte_eth_dev_check_detachable(uint8_t port_id); > + > +/** > + * Function for internal use by port hotplug functions. > + * Returns a ethdev slot specified by the unique identifier name. > + * @param name > + * The pointer to the Unique identifier name for each Ethernet device > + * @return > + * - The pointer to the ethdev slot, on success. NULL on error > + */ > +extern struct rte_eth_dev *rte_eth_dev_allocated(const char *name); > + > +/** > * Function for internal use by dummy drivers primarily, e.g. ring-based > * driver. > * Allocates a new ethdev slot for an ethernet device and returns the pointer > diff --git a/lib/librte_ether/rte_ether_version.map b/lib/librte_ether/rte_ether_version.map > index 7316530..fc5dc27 100644 > --- a/lib/librte_ether/rte_ether_version.map > +++ b/lib/librte_ether/rte_ether_version.map > @@ -109,6 +109,12 @@ DPDK_2.0 { > rte_eth_tx_queue_setup; > rte_eth_xstats_get; > rte_eth_xstats_reset; > + rte_eth_dev_check_detachable; > + rte_eth_dev_get_name_by_port; > + rte_eth_dev_get_addr_by_port; > + rte_eth_dev_get_port_by_addr; > + rte_eth_dev_get_changed_port; > + rte_eth_dev_save; > > local: *; > }; >
2015-02-16 13:14, Tetsuya Mukawa: > The patch adds functions for unmapping igb_uio resources. The patch is only > for Linux and igb_uio environment. VFIO and BSD are not supported. > > v8: > - Fix typo. > (Thanks to Iremonger, Bernard) > v5: > - Fix pci_unmap_device() to check pt_driver. > v4: > - Add parameter checking. > - Add header file to determine if hotplug can be enabled. > > Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp> > --- > lib/librte_eal/common/Makefile | 1 + > lib/librte_eal/common/include/rte_dev_hotplug.h | 44 +++++++++++++++++ > lib/librte_eal/linuxapp/eal/eal_pci.c | 44 +++++++++++++++++ > lib/librte_eal/linuxapp/eal/eal_pci_init.h | 8 +++ > lib/librte_eal/linuxapp/eal/eal_pci_uio.c | 65 +++++++++++++++++++++++++ > 5 files changed, 162 insertions(+) > create mode 100644 lib/librte_eal/common/include/rte_dev_hotplug.h > > diff --git a/lib/librte_eal/common/Makefile b/lib/librte_eal/common/Makefile > index 52c1a5f..db7cc93 100644 > --- a/lib/librte_eal/common/Makefile > +++ b/lib/librte_eal/common/Makefile > @@ -41,6 +41,7 @@ INC += rte_eal_memconfig.h rte_malloc_heap.h > INC += rte_hexdump.h rte_devargs.h rte_dev.h > INC += rte_common_vect.h > INC += rte_pci_dev_feature_defs.h rte_pci_dev_features.h > +INC += rte_dev_hotplug.h > > ifeq ($(CONFIG_RTE_INSECURE_FUNCTION_WARNING),y) > INC += rte_warnings.h > diff --git a/lib/librte_eal/common/include/rte_dev_hotplug.h b/lib/librte_eal/common/include/rte_dev_hotplug.h > new file mode 100644 > index 0000000..b333e0f > --- /dev/null > +++ b/lib/librte_eal/common/include/rte_dev_hotplug.h > @@ -0,0 +1,44 @@ > +/*- > + * BSD LICENSE > + * > + * Copyright(c) 2015 IGEL Co.,LTd. > + * All rights reserved. > + * > + * Redistribution and use in source and binary forms, with or without > + * modification, are permitted provided that the following conditions > + * are met: > + * > + * * Redistributions of source code must retain the above copyright > + * notice, this list of conditions and the following disclaimer. > + * * Redistributions in binary form must reproduce the above copyright > + * notice, this list of conditions and the following disclaimer in > + * the documentation and/or other materials provided with the > + * distribution. > + * * Neither the name of IGEL Co.,Ltd. nor the names of its > + * contributors may be used to endorse or promote products derived > + * from this software without specific prior written permission. > + * > + * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS > + * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT > + * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR > + * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT > + * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, > + * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT > + * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, > + * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY > + * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT > + * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE > + * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. > + */ > + > +#ifndef _RTE_DEV_HOTPLUG_H_ > +#define _RTE_DEV_HOTPLUG_H_ > + > +/* > + * determine if hotplug can be enabled on the system > + */ > +#if defined(RTE_LIBRTE_EAL_HOTPLUG) && defined(RTE_LIBRTE_EAL_LINUXAPP) > +#define ENABLE_HOTPLUG > +#endif /* RTE_LIBRTE_EAL_HOTPLUG & RTE_LIBRTE_EAL_LINUXAPP */ Please no. You just have to set the build option to no in the default BSD config file with a comment explaining it is not supported. > + > +#endif /* _RTE_DEV_HOTPLUG_H_ */ > diff --git a/lib/librte_eal/linuxapp/eal/eal_pci.c b/lib/librte_eal/linuxapp/eal/eal_pci.c > index 2d5f6a6..72a1362 100644 > --- a/lib/librte_eal/linuxapp/eal/eal_pci.c > +++ b/lib/librte_eal/linuxapp/eal/eal_pci.c > @@ -167,6 +167,25 @@ pci_map_resource(void *requested_addr, int fd, off_t offset, size_t size) > return mapaddr; > } > > +#ifdef ENABLE_HOTPLUG Please avoid using #ifdef if not really necessary. > +/* unmap a particular resource */ > +void > +pci_unmap_resource(void *requested_addr, size_t size) > +{ > + if (requested_addr == NULL) > + return; > + > + /* Unmap the PCI memory resource of device */ > + if (munmap(requested_addr, size)) { > + RTE_LOG(ERR, EAL, "%s(): cannot munmap(%p, 0x%lx): %s\n", > + __func__, requested_addr, (unsigned long)size, > + strerror(errno)); > + } else > + RTE_LOG(DEBUG, EAL, " PCI memory unmapped at %p\n", > + requested_addr); > +} > +#endif /* ENABLE_HOTPLUG */ > + > /* parse the "resource" sysfs file */ > #define IORESOURCE_MEM 0x00000200 [...]
2015-02-16 13:14, Tetsuya Mukawa: > - Add rte_eal_pci_close_one_dirver() > The function is used for closing the specified driver and device. > - Add pci_invoke_all_drivers() > The function is based on pci_probe_all_drivers. But it can not only > probe but also close drivers. > - Add pci_close_all_drivers() > The function tries to find a driver for the specified device, and > then close the driver. > - Add rte_eal_pci_probe_one() and rte_eal_pci_close_one() > The functions are used for probe and close a device. > First the function tries to find a device that has the specified > PCI address. Then, probe or close the device. > > v5: > - Remove RTE_EAL_INVOKE_TYPE_UNKNOWN, because it's unused. > v4: > - Fix parameter checking. > - Fix indent of 'if' statement. > > Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp> > --- > lib/librte_eal/common/eal_common_pci.c | 90 +++++++++++++++++++++++++++++---- > lib/librte_eal/common/eal_private.h | 24 +++++++++ > lib/librte_eal/common/include/rte_pci.h | 33 ++++++++++++ > lib/librte_eal/linuxapp/eal/eal_pci.c | 69 +++++++++++++++++++++++++ > 4 files changed, 206 insertions(+), 10 deletions(-) 206 insertions and 10 deletions: it cannot really be a cleanup ;) Maybe the title should be reworded. [...] > - rc = rte_eal_pci_probe_one_driver(dr, dev); > + switch (type) { > + case RTE_EAL_INVOKE_TYPE_PROBE: > + rc = rte_eal_pci_probe_one_driver(dr, dev); > + break; > + case RTE_EAL_INVOKE_TYPE_CLOSE: > + rc = rte_eal_pci_close_one_driver(dr, dev); > + break; Honestly, I don't like this kind of functions with a switch to toggle different actions. It makes code unclear.
2015-02-16 13:14, Tetsuya Mukawa: > This new parameter is needed to keep device type like physical or virtual. Actually types are "PCI" and "virtual". > Port detaching processes are different between physical and virtual. > This parameter lets detaching function know a device type of the port. I think the type should be guessed automatically thanks to register infos.
2015-02-16 13:14, Tetsuya Mukawa:
> The patch enables CONFIG_RTE_LIBRTE_EAL_HOTPLUG in Linux configuration.
This patch should be introduced earlier in the patchset.
2015-02-16 13:14, Tetsuya Mukawa: > These functions are used for attaching or detaching a port. > When rte_eal_dev_attach() is called, the function tries to realize the > device name as pci address. If this is done successfully, > rte_eal_dev_attach() will attach physical device port. If not, attaches > virtual devive port. > When rte_eal_dev_detach() is called, the function gets the device type > of this port to know whether the port is come from physical or virtual. > And then specific detaching function will be called. > > v8: > - Add missing symbol in version map. > (Thanks to Qiu, Michael and Iremonger, Bernard) > v7: > - Fix typo of warning messages. > (Thanks to Qiu, Michael) > v5: > - Change function names like below. > rte_eal_dev_find_and_invoke() to rte_eal_vdev_find_and_invoke(). > rte_eal_dev_invoke() to rte_eal_vdev_invoke(). > - Add code to handle a return value of rte_eal_devargs_remove(). > - Fix pci address format in rte_eal_dev_detach(). > v4: > - Fix comment. > - Add error checking. > - Fix indent of 'if' statement. > - Change function name. > > Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp> > --- > lib/librte_eal/common/eal_common_dev.c | 276 ++++++++++++++++++++++++ > lib/librte_eal/common/eal_private.h | 11 + > lib/librte_eal/common/include/rte_dev.h | 33 +++ > lib/librte_eal/linuxapp/eal/Makefile | 1 + > lib/librte_eal/linuxapp/eal/eal_pci.c | 6 +- > lib/librte_eal/linuxapp/eal/rte_eal_version.map | 2 + > 6 files changed, 326 insertions(+), 3 deletions(-) > > diff --git a/lib/librte_eal/common/eal_common_dev.c b/lib/librte_eal/common/eal_common_dev.c > index eae5656..3d169a4 100644 > --- a/lib/librte_eal/common/eal_common_dev.c > +++ b/lib/librte_eal/common/eal_common_dev.c > @@ -32,10 +32,13 @@ > * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. > */ > > +#include <stdio.h> > +#include <limits.h> > #include <string.h> > #include <inttypes.h> > #include <sys/queue.h> > > +#include <rte_ethdev.h> > #include <rte_dev.h> > #include <rte_devargs.h> > #include <rte_debug.h> > @@ -107,3 +110,276 @@ rte_eal_dev_init(void) > } > return 0; > } > + > +/* So far, DPDK hotplug function only supports linux */ This comment should be in the configuration. > +#ifdef ENABLE_HOTPLUG > +static void > +rte_eal_vdev_invoke(struct rte_driver *driver, > + struct rte_devargs *devargs, enum rte_eal_invoke_type type) > +{ > + if ((driver == NULL) || (devargs == NULL)) > + return; > + > + switch (type) { > + case RTE_EAL_INVOKE_TYPE_PROBE: > + driver->init(devargs->virtual.drv_name, devargs->args); > + break; > + case RTE_EAL_INVOKE_TYPE_CLOSE: > + driver->uninit(devargs->virtual.drv_name, devargs->args); > + break; > + default: > + break; > + } > +} It would be clearer to directly call init and uninit instead of using this "invoke" method. > + > +static int > +rte_eal_vdev_find_and_invoke(const char *name, int type) > +{ > + struct rte_devargs *devargs; > + struct rte_driver *driver; > + > + if (name == NULL) > + return -EINVAL; > + > + /* call the init function for each virtual device */ This comment is wrong. > + TAILQ_FOREACH(devargs, &devargs_list, next) { > + > + if (devargs->type != RTE_DEVTYPE_VIRTUAL) > + continue; > + > + if (strncmp(name, devargs->virtual.drv_name, strlen(name))) > + continue; > + > + TAILQ_FOREACH(driver, &dev_driver_list, next) { > + if (driver->type != PMD_VDEV) > + continue; > + > + /* search a driver prefix in virtual device name */ > + if (!strncmp(driver->name, devargs->virtual.drv_name, > + strlen(driver->name))) { Why not use strcmp? > + rte_eal_vdev_invoke(driver, devargs, type); > + break; > + } > + } > + > + if (driver == NULL) { > + RTE_LOG(WARNING, EAL, "no driver found for %s\n", > + devargs->virtual.drv_name); > + } > + return 0; > + } > + return 1; > +} > + > +/* attach the new physical device, then store port_id of the device */ > +static int > +rte_eal_dev_attach_pdev(struct rte_pci_addr *addr, uint8_t *port_id) > +{ > + uint8_t new_port_id; > + struct rte_eth_dev devs[RTE_MAX_ETHPORTS]; > + > + if ((addr == NULL) || (port_id == NULL)) > + goto err; > + > + /* save current port status */ > + if (rte_eth_dev_save(devs, sizeof(devs))) > + goto err; > + /* re-construct pci_device_list */ > + if (rte_eal_pci_scan()) > + goto err; > + /* invoke probe func of the driver can handle the new device */ > + if (rte_eal_pci_probe_one(addr)) > + goto err; You should get the port_id from the previous function instead of searching it. > + /* get port_id enabled by above procedures */ > + if (rte_eth_dev_get_changed_port(devs, &new_port_id)) > + goto err; > + > + *port_id = new_port_id; > + return 0; > +err: > + RTE_LOG(ERR, EAL, "Driver, cannot attach the device\n"); > + return -1; > +} [...] > +/* attach the new virtual device, then store port_id of the device */ > +static int > +rte_eal_dev_attach_vdev(const char *vdevargs, uint8_t *port_id) > +{ > + char *args; > + uint8_t new_port_id; > + struct rte_eth_dev devs[RTE_MAX_ETHPORTS]; > + > + if ((vdevargs == NULL) || (port_id == NULL)) > + goto err0; > + > + args = strdup(vdevargs); > + if (args == NULL) > + goto err0; > + > + /* save current port status */ > + if (rte_eth_dev_save(devs, sizeof(devs))) > + goto err1; > + /* add the vdevargs to devargs_list */ > + if (rte_eal_devargs_add(RTE_DEVTYPE_VIRTUAL, args)) > + goto err1; > + /* parse vdevargs, then retrieve device name */ > + get_vdev_name(args); > + /* walk around dev_driver_list to find the driver of the device, > + * then invoke probe function o the driver */ > + if (rte_eal_vdev_find_and_invoke(args, RTE_EAL_INVOKE_TYPE_PROBE)) > + goto err2; Again, you should get port_id from the attach procedure. > + /* get port_id enabled by above procedures */ > + if (rte_eth_dev_get_changed_port(devs, &new_port_id)) > + goto err2; [...] > /** > + * Uninitilization function called for each device driver once. > + */ > +typedef int (rte_dev_uninit_t)(const char *name, const char *args); Why do you need args for uninit?
On 2/17/2015 8:13 AM, Thomas Monjalon wrote: > 2015-02-16 13:14, Tetsuya Mukawa: >> From: Michael Qiu <michael.qiu@intel.com> >> >> Currently, dpdk has no ability to know which type of driver( >> vfio-pci/igb_uio/uio_pci_generic) the device used. It only can >> check whether vfio is enabled or not staticly. >> >> It really useful to have the flag, becasue different type need to >> handle differently in runtime. For example, pci memory map, >> pot hotplug, and so on. >> >> This patch add a flag field for pci device to solve above issue. >> >> Signed-off-by: Michael Qiu <michael.qiu@intel.com> >> Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp> >> --- >> lib/librte_eal/common/include/rte_pci.h | 8 +++++ >> lib/librte_eal/linuxapp/eal/eal_pci.c | 53 +++++++++++++++++++++++++++++++-- >> 2 files changed, 59 insertions(+), 2 deletions(-) >> >> diff --git a/lib/librte_eal/common/include/rte_pci.h b/lib/librte_eal/common/include/rte_pci.h >> index 66ed793..7b48b55 100644 >> --- a/lib/librte_eal/common/include/rte_pci.h >> +++ b/lib/librte_eal/common/include/rte_pci.h >> @@ -139,6 +139,13 @@ struct rte_pci_addr { >> >> struct rte_devargs; >> >> +enum rte_pt_driver { >> + RTE_PT_UNKNOWN = 0, >> + RTE_PT_IGB_UIO = 1, >> + RTE_PT_VFIO = 2, >> + RTE_PT_UIO_GENERIC = 3, >> +}; > What means PT? > >> + >> /** >> * A structure describing a PCI device. >> */ >> @@ -152,6 +159,7 @@ struct rte_pci_device { >> uint16_t max_vfs; /**< sriov enable if not zero */ >> int numa_node; /**< NUMA node connection */ >> struct rte_devargs *devargs; /**< Device user arguments */ >> + enum rte_pt_driver pt_driver; /**< Driver of passthrough */ > I'm not sure passtrough is the right word for this kind of driver. > What about "kernel driver"? We only care vfio and uio driver, here pass through means pass the devices from kernel to userspace If use "kernel driver"or "kn_driver", there will be some issue that we set to "UNKNOWN" when the devices use the native linux kernel driver. > >> }; >> >> /** Any PCI device identifier (vendor, device, ...) */ >> diff --git a/lib/librte_eal/linuxapp/eal/eal_pci.c b/lib/librte_eal/linuxapp/eal/eal_pci.c >> index 15db9c4..e760452 100644 >> --- a/lib/librte_eal/linuxapp/eal/eal_pci.c >> +++ b/lib/librte_eal/linuxapp/eal/eal_pci.c >> @@ -97,6 +97,35 @@ error: >> return -1; >> } >> >> +static int >> +pci_get_kernel_driver_by_path(const char *filename, char *dri_name) >> +{ >> + int count; >> + char path[PATH_MAX]; >> + char *name; >> + >> + if (!filename || !dri_name) >> + return -1; >> + >> + count = readlink(filename, path, PATH_MAX); >> + if (count >= PATH_MAX) >> + return -1; >> + >> + /* For device does not have a driver */ >> + if (count < 0) >> + return 1; >> + >> + path[count] = '\0'; >> + >> + name = strrchr(path, '/'); >> + if (name) { >> + strncpy(dri_name, name + 1, strlen(name + 1) + 1); >> + return 0; >> + } >> + >> + return -1; >> +} >> + >> void * >> pci_find_max_end_va(void) >> { >> @@ -222,11 +251,12 @@ pci_scan_one(const char *dirname, uint16_t domain, uint8_t bus, >> char filename[PATH_MAX]; >> unsigned long tmp; >> struct rte_pci_device *dev; >> + char driver[PATH_MAX]; >> + int ret; >> >> dev = malloc(sizeof(*dev)); >> - if (dev == NULL) { >> + if (dev == NULL) >> return -1; >> - } >> >> memset(dev, 0, sizeof(*dev)); >> dev->addr.domain = domain; >> @@ -305,6 +335,25 @@ pci_scan_one(const char *dirname, uint16_t domain, uint8_t bus, >> return -1; >> } >> >> + /* parse driver */ >> + snprintf(filename, sizeof(filename), "%s/driver", dirname); >> + ret = pci_get_kernel_driver_by_path(filename, driver); >> + if (!ret) { >> + if (!strcmp(driver, "vfio-pci")) >> + dev->pt_driver = RTE_PT_VFIO; >> + else if (!strcmp(driver, "igb_uio")) >> + dev->pt_driver = RTE_PT_IGB_UIO; >> + else if (!strcmp(driver, "uio_pci_generic")) >> + dev->pt_driver = RTE_PT_UIO_GENERIC; >> + else >> + dev->pt_driver = RTE_PT_UNKNOWN; >> + } else if (ret < 0) { >> + RTE_LOG(ERR, EAL, "Fail to get kernel driver\n"); >> + free(dev); >> + return -1; >> + } else >> + dev->pt_driver = RTE_PT_UNKNOWN; > Why not considering a NONE value? Example: for virtio with port I/O. Why NONE? Do you mean, leave it unset? RTE_PT_UNKNOWN equals to zero, when code runs here means the devices does not have a driver or not bind to any driver. For virtio with port I/O, as I know it will use the kernel driver virtio-pci. So it will not run here. Thanks, Michael > >> + >> /* device is valid, add in list (sorted) */ >> if (TAILQ_EMPTY(&pci_device_list)) { >> TAILQ_INSERT_TAIL(&pci_device_list, dev, next); >> > >
On 2015/02/16 21:45, Neil Horman wrote:
> On Mon, Feb 16, 2015 at 01:14:27PM +0900, Tetsuya Mukawa wrote:
>> The patch adds functions for unmapping igb_uio resources. The patch is only
>> for Linux and igb_uio environment. VFIO and BSD are not supported.
>>
>> v8:
>> - Fix typo.
>> (Thanks to Iremonger, Bernard)
>> v5:
>> - Fix pci_unmap_device() to check pt_driver.
>> v4:
>> - Add parameter checking.
>> - Add header file to determine if hotplug can be enabled.
>>
>> Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp>
>> ---
>> lib/librte_eal/common/Makefile | 1 +
>> lib/librte_eal/common/include/rte_dev_hotplug.h | 44 +++++++++++++++++
>> lib/librte_eal/linuxapp/eal/eal_pci.c | 44 +++++++++++++++++
>> lib/librte_eal/linuxapp/eal/eal_pci_init.h | 8 +++
>> lib/librte_eal/linuxapp/eal/eal_pci_uio.c | 65 +++++++++++++++++++++++++
>> 5 files changed, 162 insertions(+)
>> create mode 100644 lib/librte_eal/common/include/rte_dev_hotplug.h
>>
>> diff --git a/lib/librte_eal/common/Makefile b/lib/librte_eal/common/Makefile
>> index 52c1a5f..db7cc93 100644
>> --- a/lib/librte_eal/common/Makefile
>> +++ b/lib/librte_eal/common/Makefile
>> @@ -41,6 +41,7 @@ INC += rte_eal_memconfig.h rte_malloc_heap.h
>> INC += rte_hexdump.h rte_devargs.h rte_dev.h
>> INC += rte_common_vect.h
>> INC += rte_pci_dev_feature_defs.h rte_pci_dev_features.h
>> +INC += rte_dev_hotplug.h
>>
>> ifeq ($(CONFIG_RTE_INSECURE_FUNCTION_WARNING),y)
>> INC += rte_warnings.h
>> diff --git a/lib/librte_eal/common/include/rte_dev_hotplug.h b/lib/librte_eal/common/include/rte_dev_hotplug.h
>> new file mode 100644
>> index 0000000..b333e0f
>> --- /dev/null
>> +++ b/lib/librte_eal/common/include/rte_dev_hotplug.h
>> @@ -0,0 +1,44 @@
>> +/*-
>> + * BSD LICENSE
>> + *
>> + * Copyright(c) 2015 IGEL Co.,LTd.
>> + * All rights reserved.
>> + *
>> + * Redistribution and use in source and binary forms, with or without
>> + * modification, are permitted provided that the following conditions
>> + * are met:
>> + *
>> + * * Redistributions of source code must retain the above copyright
>> + * notice, this list of conditions and the following disclaimer.
>> + * * Redistributions in binary form must reproduce the above copyright
>> + * notice, this list of conditions and the following disclaimer in
>> + * the documentation and/or other materials provided with the
>> + * distribution.
>> + * * Neither the name of IGEL Co.,Ltd. nor the names of its
>> + * contributors may be used to endorse or promote products derived
>> + * from this software without specific prior written permission.
>> + *
>> + * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
>> + * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
>> + * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
>> + * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
>> + * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
>> + * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
>> + * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
>> + * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
>> + * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
>> + * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
>> + * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
>> + */
>> +
>> +#ifndef _RTE_DEV_HOTPLUG_H_
>> +#define _RTE_DEV_HOTPLUG_H_
>> +
>> +/*
>> + * determine if hotplug can be enabled on the system
>> + */
>> +#if defined(RTE_LIBRTE_EAL_HOTPLUG) && defined(RTE_LIBRTE_EAL_LINUXAPP)
>> +#define ENABLE_HOTPLUG
>> +#endif /* RTE_LIBRTE_EAL_HOTPLUG & RTE_LIBRTE_EAL_LINUXAPP */
> I'm not sure why you're doing this. Why not just test RTE_LIBRTE_EAL_HOTPLUG in
> the various locations where your testing ENABLE_HOTPLUG? This seems like
> indirection for the sake of indirection.
> Neil
>
Hi Neil,
I will define RTE_LIBRTE_EAL_HOTPLUG=y in only Linux configuration.
And use it other places like you said.
Thanks,
Tetsuya
On 2015/02/16 21:41, Neil Horman wrote:
> On Mon, Feb 16, 2015 at 01:14:26PM +0900, Tetsuya Mukawa wrote:
>> The patch adds following functions.
>>
>> - rte_eth_dev_save()
>> The function is used for saving current rte_eth_dev structures.
>> - rte_eth_dev_get_changed_port()
>> The function receives the rte_eth_dev structures, then compare
>> these with current values to know which port is actually
>> attached or detached.
>> - rte_eth_dev_get_addr_by_port()
>> The function returns a pci address of an ethdev specified by port
>> identifier.
>> - rte_eth_dev_get_port_by_addr()
>> The function returns a port identifier of an ethdev specified by
>> pci address.
>> - rte_eth_dev_get_name_by_port()
>> The function returns a unique identifier name of an ethdev
>> specified by port identifier.
>> - Add rte_eth_dev_check_detachable()
>> The function returns whether a PMD supports detach function.
>>
>> Also, the patch changes scope of rte_eth_dev_allocated() to global.
>> This function will be called by virtual PMDs to support port hotplug.
>> So change scope of the function to global.
>>
>> v8:
>> - Add size parameter to rte_eth_dev_save().
>> - Add missing symbol in version map.
>> (Thanks to Qiu, Michael and Iremonger, Bernard)
>> v7:
>> - Add pt_driver checking to rte_eth_dev_check_detachable().
>> (Thanks to Qiu, Michael)
>> v5:
>> - Fix return value of below functions.
>> rte_eth_dev_get_changed_port().
>> rte_eth_dev_get_port_by_addr().
>> v4:
>> - Add parameter checking.
>> v3:
>> - Fix if-condition bug while comparing pci addresses.
>> - Add error checking codes.
>> Reported-by: Mark Enright <menrigh@brocade.com>
>>
>> Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp>
>> ---
>> lib/librte_ether/rte_ethdev.c | 99 +++++++++++++++++++++++++++++++++-
>> lib/librte_ether/rte_ethdev.h | 83 ++++++++++++++++++++++++++++
>> lib/librte_ether/rte_ether_version.map | 6 +++
>> 3 files changed, 187 insertions(+), 1 deletion(-)
>>
>> diff --git a/lib/librte_ether/rte_ethdev.c b/lib/librte_ether/rte_ethdev.c
>> index 58d8072..3869a96 100644
>> --- a/lib/librte_ether/rte_ethdev.c
>> +++ b/lib/librte_ether/rte_ethdev.c
>> @@ -206,7 +206,7 @@ rte_eth_dev_data_alloc(void)
>> RTE_MAX_ETHPORTS * sizeof(*rte_eth_dev_data));
>> }
>>
>> -static struct rte_eth_dev *
>> +struct rte_eth_dev *
>> rte_eth_dev_allocated(const char *name)
>> {
>> unsigned i;
>> @@ -426,6 +426,103 @@ rte_eth_dev_count(void)
>> return (nb_ports);
>> }
>>
>> +int
>> +rte_eth_dev_save(struct rte_eth_dev *devs, size_t size)
>> +{
>> + if ((devs == NULL) ||
>> + (size != sizeof(struct rte_eth_dev) * RTE_MAX_ETHPORTS))
>> + return -EINVAL;
>> +
>> + /* save current rte_eth_devices */
>> + memcpy(devs, rte_eth_devices, size);
>> + return 0;
>> +}
>> +
>> +int
>> +rte_eth_dev_get_changed_port(struct rte_eth_dev *devs, uint8_t *port_id)
>> +{
>> + if ((devs == NULL) || (port_id == NULL))
>> + return -EINVAL;
>> +
>> + /* check which port was attached or detached */
>> + for (*port_id = 0; *port_id < RTE_MAX_ETHPORTS; (*port_id)++, devs++) {
>> + if (rte_eth_devices[*port_id].attached ^ devs->attached)
>> + return 0;
>> + }
>> + return -ENODEV;
>> +}
>> +
>> +int
>> +rte_eth_dev_get_addr_by_port(uint8_t port_id, struct rte_pci_addr *addr)
>> +{
>> + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID)
>> + return -EINVAL;
>> +
>> + if (addr == NULL) {
>> + PMD_DEBUG_TRACE("Null pointer is specified\n");
>> + return -EINVAL;
>> + }
>> +
>> + *addr = rte_eth_devices[port_id].pci_dev->addr;
>> + return 0;
>> +}
>> +
>> +int
>> +rte_eth_dev_get_port_by_addr(struct rte_pci_addr *addr, uint8_t *port_id)
>> +{
>> + struct rte_pci_addr *tmp;
>> +
>> + if ((addr == NULL) || (port_id == NULL)) {
>> + PMD_DEBUG_TRACE("Null pointer is specified\n");
>> + return -EINVAL;
>> + }
>> +
>> + for (*port_id = 0; *port_id < RTE_MAX_ETHPORTS; (*port_id)++) {
>> + if (!rte_eth_devices[*port_id].attached)
>> + continue;
>> + if (!rte_eth_devices[*port_id].pci_dev)
>> + continue;
>> + tmp = &rte_eth_devices[*port_id].pci_dev->addr;
>> + if (eal_compare_pci_addr(tmp, addr) == 0)
>> + return 0;
>> + }
>> + return -ENODEV;
>> +}
>> +
>> +int
>> +rte_eth_dev_get_name_by_port(uint8_t port_id, char *name)
>> +{
>> + char *tmp;
>> +
>> + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID)
>> + return -EINVAL;
>> +
>> + if (name == NULL) {
>> + PMD_DEBUG_TRACE("Null pointer is specified\n");
>> + return -EINVAL;
>> + }
>> +
>> + /* shouldn't check 'rte_eth_devices[i].data',
>> + * because it might be overwritten by VDEV PMD */
>> + tmp = rte_eth_dev_data[port_id].name;
>> + strncpy(name, tmp, strlen(tmp) + 1);
>> + return 0;
>> +}
>> +
>> +int
>> +rte_eth_dev_check_detachable(uint8_t port_id)
>> +{
>> + uint32_t drv_flags;
>> +
>> + if (port_id >= RTE_MAX_ETHPORTS) {
>> + PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
>> + return -EINVAL;
>> + }
>> +
>> + drv_flags = rte_eth_devices[port_id].driver->pci_drv.drv_flags;
>> + return !(drv_flags & RTE_PCI_DRV_DETACHABLE);
>> +}
>> +
>> static int
>> rte_eth_dev_rx_queue_config(struct rte_eth_dev *dev, uint16_t nb_queues)
>> {
>> diff --git a/lib/librte_ether/rte_ethdev.h b/lib/librte_ether/rte_ethdev.h
>> index 91d9e86..6651890 100644
>> --- a/lib/librte_ether/rte_ethdev.h
>> +++ b/lib/librte_ether/rte_ethdev.h
>> @@ -1616,6 +1616,89 @@ extern struct rte_eth_dev rte_eth_devices[];
>> extern uint8_t rte_eth_dev_count(void);
>>
>> /**
>> + * Function for internal use by port hotplug functions.
>> + * Copies current ethdev structures to the specified pointer.
>> + *
>> + * @param devs The pointer to the ethdev structures
>> + * @param size The size of ethdev structures
>> + * @return
>> + * - 0 on success, negative on error
>> + */
>> +extern int rte_eth_dev_save(struct rte_eth_dev *devs, size_t size);
>> +
>> +/**
>> + * Function for internal use by port hotplug functions.
>> + * Compare the specified ethdev structures with currents. Then
>> + * if there is a port which status is changed, fill the specified pointer
>> + * with the port id of that port.
>> + * @param devs The pointer to the ethdev structures
>> + * @param port_id The pointer to the port id
>> + * @return
>> + * - 0 on success, negative on error
>> + */
>> +extern int rte_eth_dev_get_changed_port(
>> + struct rte_eth_dev *devs, uint8_t *port_id);
>> +
>> +/**
>> + * Function for internal use by port hotplug functions.
>> + * Returns a pci address of a ethdev specified by port identifier.
>> + * @param port_id
>> + * The port identifier of the Ethernet device
>> + * @param addr
>> + * The pointer to the pci address
>> + * @return
>> + * - 0 on success, negative on error
>> + */
>> +extern int rte_eth_dev_get_addr_by_port(
>> + uint8_t port_id, struct rte_pci_addr *addr);
>> +
>> +/**
>> + * Function for internal use by port hotplug functions.
>> + * Returns a port identifier of a ethdev specified by pci address.
>> + * @param addr
>> + * The pointer to the pci address of the Ethernet device.
>> + * @param port_id
>> + * The pointer to the port identifier
>> + * @return
>> + * - 0 on success, negative on error
>> + */
>> +extern int rte_eth_dev_get_port_by_addr(
>> + struct rte_pci_addr *addr, uint8_t *port_id);
>> +
>> +/**
>> + * Function for internal use by port hotplug functions.
>> + * Returns a unique identifier name of a ethdev specified by port identifier.
>> + * @param port_id
>> + * The port identifier.
>> + * @param name
>> + * The pointer to the Unique identifier name for each Ethernet device
>> + * @return
>> + * - 0 on success, negative on error
>> + */
>> +extern int rte_eth_dev_get_name_by_port(uint8_t port_id, char *name);
>> +
>> +/**
>> + * Function for internal use by port hotplug functions.
>> + * Check whether or not, a PMD that is handling the ethdev specified by port
>> + * identifier can support detach function.
>> + * @param port_id
>> + * The port identifier
>> + * @return
>> + * - 0 on supporting detach function, negative on not supporting
>> + */
>> +extern int rte_eth_dev_check_detachable(uint8_t port_id);
>> +
>> +/**
>> + * Function for internal use by port hotplug functions.
>> + * Returns a ethdev slot specified by the unique identifier name.
>> + * @param name
>> + * The pointer to the Unique identifier name for each Ethernet device
>> + * @return
>> + * - The pointer to the ethdev slot, on success. NULL on error
>> + */
>> +extern struct rte_eth_dev *rte_eth_dev_allocated(const char *name);
>> +
>> +/**
>> * Function for internal use by dummy drivers primarily, e.g. ring-based
>> * driver.
>> * Allocates a new ethdev slot for an ethernet device and returns the pointer
>> diff --git a/lib/librte_ether/rte_ether_version.map b/lib/librte_ether/rte_ether_version.map
>> index 7316530..fc5dc27 100644
>> --- a/lib/librte_ether/rte_ether_version.map
>> +++ b/lib/librte_ether/rte_ether_version.map
>> @@ -109,6 +109,12 @@ DPDK_2.0 {
>> rte_eth_tx_queue_setup;
>> rte_eth_xstats_get;
>> rte_eth_xstats_reset;
>> + rte_eth_dev_check_detachable;
>> + rte_eth_dev_get_name_by_port;
>> + rte_eth_dev_get_addr_by_port;
>> + rte_eth_dev_get_port_by_addr;
>> + rte_eth_dev_get_changed_port;
>> + rte_eth_dev_save;
> rte_eth_dev_allocated needs to get added in here too
> Neil
>
Hi Neil,
I appreciate for your indication.
I will fix it.
Thanks,
Tetsuya
Hi Thomas, I appreciate for your all comments. On 2015/02/17 9:36, Thomas Monjalon wrote: > 2015-02-16 13:14, Tetsuya Mukawa: >> To remove assumption, do like followings. >> >> This patch adds "RTE_PCI_DRV_DETACHABLE" to drv_flags of rte_pci_driver >> structure. The flags indicate the driver can detach devices at runtime. >> Also, remove assumption that port will not be detached. >> >> To remove the assumption. >> - Add 'attached' member to rte_eth_dev structure. >> This member is used for indicating the port is attached, or not. >> - Add rte_eth_dev_allocate_new_port(). >> This function is used for allocating new port. >> >> v8: >> - NONE_TRACE is changed to NO_TRACE. >> (Thanks to Iremonger, Bernard) >> v5: >> - Change parameters of rte_eth_dev_validate_port() to cleanup code. >> v4: >> - Use braces with 'for' loop. >> - Fix indent of 'if' statement. >> >> Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp> >> --- >> lib/librte_eal/common/include/rte_pci.h | 2 + >> lib/librte_ether/rte_ethdev.c | 454 +++++++++++++------------------- >> lib/librte_ether/rte_ethdev.h | 5 + >> 3 files changed, 186 insertions(+), 275 deletions(-) >> >> diff --git a/lib/librte_eal/common/include/rte_pci.h b/lib/librte_eal/common/include/rte_pci.h >> index 7b48b55..7f2d699 100644 >> --- a/lib/librte_eal/common/include/rte_pci.h >> +++ b/lib/librte_eal/common/include/rte_pci.h >> @@ -207,6 +207,8 @@ struct rte_pci_driver { >> #define RTE_PCI_DRV_FORCE_UNBIND 0x0004 >> /** Device driver supports link state interrupt */ >> #define RTE_PCI_DRV_INTR_LSC 0x0008 >> +/** Device driver supports detaching capability */ >> +#define RTE_PCI_DRV_DETACHABLE 0x0010 >> >> /**< Internal use only - Macro used by pci addr parsing functions **/ >> #define GET_PCIADDR_FIELD(in, fd, lim, dlm) \ >> diff --git a/lib/librte_ether/rte_ethdev.c b/lib/librte_ether/rte_ethdev.c >> index ea3a1fb..a79fa5b 100644 >> --- a/lib/librte_ether/rte_ethdev.c >> +++ b/lib/librte_ether/rte_ethdev.c >> @@ -175,6 +175,16 @@ enum { >> STAT_QMAP_RX >> }; >> >> +enum { >> + DEV_INVALID = 0, >> + DEV_VALID, >> +}; >> + >> +enum { >> + DEV_DISCONNECTED = 0, >> + DEV_CONNECTED >> +}; > The commit log explains what is an attached port but not what means > valid or connected. > These enums should have a comment to explain their usage. Sure, I will add comment, and fix commit log. >> + >> static inline void >> rte_eth_dev_data_alloc(void) >> { >> @@ -201,19 +211,34 @@ rte_eth_dev_allocated(const char *name) >> { >> unsigned i; >> >> - for (i = 0; i < nb_ports; i++) { >> - if (strcmp(rte_eth_devices[i].data->name, name) == 0) >> + for (i = 0; i < RTE_MAX_ETHPORTS; i++) { >> + if ((rte_eth_devices[i].attached == DEV_CONNECTED) && >> + strcmp(rte_eth_devices[i].data->name, name) == 0) >> return &rte_eth_devices[i]; >> } >> return NULL; >> } >> >> +static uint8_t >> +rte_eth_dev_allocate_new_port(void) >> +{ >> + unsigned i; >> + >> + for (i = 0; i < RTE_MAX_ETHPORTS; i++) { >> + if (rte_eth_devices[i].attached == DEV_DISCONNECTED) >> + return i; >> + } >> + return RTE_MAX_ETHPORTS; >> +} > This function does not allocate a new port. > It get the first free port id. I will change the function name like below. rte_eth_dev_find_free_port() > Is uint8_t sill a good size for hotpluggable virtual device ids? I am not sure it's enough, but uint8_t is widely used in "rte_ethdev.c" as port id. If someone reports it doesn't enough, I guess it will be the time to write a patch to change all uint_8 in one patch. >> + >> struct rte_eth_dev * >> rte_eth_dev_allocate(const char *name) >> { >> + uint8_t port_id; >> struct rte_eth_dev *eth_dev; >> >> - if (nb_ports == RTE_MAX_ETHPORTS) { >> + port_id = rte_eth_dev_allocate_new_port(); >> + if (port_id == RTE_MAX_ETHPORTS) { >> PMD_DEBUG_TRACE("Reached maximum number of Ethernet ports\n"); >> return NULL; >> } >> @@ -226,10 +251,12 @@ rte_eth_dev_allocate(const char *name) >> return NULL; >> } >> >> - eth_dev = &rte_eth_devices[nb_ports]; >> - eth_dev->data = &rte_eth_dev_data[nb_ports]; >> + eth_dev = &rte_eth_devices[port_id]; >> + eth_dev->data = &rte_eth_dev_data[port_id]; >> snprintf(eth_dev->data->name, sizeof(eth_dev->data->name), "%s", name); >> - eth_dev->data->port_id = nb_ports++; >> + eth_dev->data->port_id = port_id; >> + eth_dev->attached = DEV_CONNECTED; >> + nb_ports++; >> return eth_dev; >> } >> >> @@ -283,6 +310,7 @@ rte_eth_dev_init(struct rte_pci_driver *pci_drv, >> (unsigned) pci_dev->id.device_id); >> if (rte_eal_process_type() == RTE_PROC_PRIMARY) >> rte_free(eth_dev->data->dev_private); >> + eth_dev->attached = DEV_DISCONNECTED; >> nb_ports--; >> return diag; >> } >> @@ -308,10 +336,28 @@ rte_eth_driver_register(struct eth_driver *eth_drv) >> rte_eal_pci_register(ð_drv->pci_drv); >> } >> >> +enum { >> + NO_TRACE = 0, >> + TRACE >> +}; > What means this enum? > >> + >> +static int >> +rte_eth_dev_validate_port(uint8_t port_id, int trace) >> +{ >> + if (port_id >= RTE_MAX_ETHPORTS || >> + rte_eth_devices[port_id].attached != DEV_CONNECTED) { >> + if (trace) { >> + PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); >> + } > It's not common in DPDK to have a parameter to toggle log. > It should be enabled globally via the log level. Sure, I will change it. >> + return DEV_INVALID; > It would be simpler to return 0 in a function called "is_valid_port". I will change the function name. >> + } else >> + return DEV_VALID; > So it could 1 (true) here. Sure, I will change like above, also delete DEV_INVALID and DEV_VALID definitions. > [...] >> --- a/lib/librte_ether/rte_ethdev.h >> +++ b/lib/librte_ether/rte_ethdev.h >> @@ -1539,6 +1539,7 @@ struct rte_eth_dev { >> struct eth_dev_ops *dev_ops; /**< Functions exported by PMD */ >> struct rte_pci_device *pci_dev; /**< PCI info. supplied by probing */ >> struct rte_eth_dev_cb_list callbacks; /**< User application callbacks */ >> + uint8_t attached; /**< Flag indicating the port is attached */ >> }; >> >> struct rte_eth_dev_sriov { >> @@ -1604,6 +1605,10 @@ extern struct rte_eth_dev rte_eth_devices[]; >> * initialized by the [matching] Ethernet driver during the PCI probing phase. >> * All devices whose port identifier is in the range >> * [0, rte_eth_dev_count() - 1] can be operated on by network applications. >> + * immediately after invoking rte_eal_init(). >> + * If the application unplugs a port using hotplug function, The enabled port >> + * numbers may be noncontiguous. In the case, the applications need to manage >> + * enabled port by themselves. >> * >> * @return >> * - The total number of usable Ethernet devices. >> >
On 2015/02/17 9:44, Thomas Monjalon wrote: > 2015-02-16 13:14, Tetsuya Mukawa: >> This patch replaces pci_addr_comparison() and memcmp() of pci addresses by >> eal_compare_pci_addr(). >> >> v8: >> - Fix pci_scan_one() to update sysfs values. >> (Thanks to Qiu, Michael and Iremonger, Bernard) >> v5: >> - Fix pci_scan_one to handle pt_driver correctly. >> v4: >> - Fix calculation method of eal_compare_pci_addr(). >> - Add parameter checking. >> >> Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp> >> --- >> lib/librte_eal/bsdapp/eal/eal_pci.c | 29 ++++++++++++-------------- >> lib/librte_eal/common/eal_common_pci.c | 2 +- >> lib/librte_eal/common/include/rte_pci.h | 34 +++++++++++++++++++++++++++++++ >> lib/librte_eal/linuxapp/eal/eal_pci.c | 30 +++++++++++++-------------- >> lib/librte_eal/linuxapp/eal/eal_pci_uio.c | 2 +- >> 5 files changed, 63 insertions(+), 34 deletions(-) >> >> diff --git a/lib/librte_eal/bsdapp/eal/eal_pci.c b/lib/librte_eal/bsdapp/eal/eal_pci.c >> index 74ecce7..7dbdccd 100644 >> --- a/lib/librte_eal/bsdapp/eal/eal_pci.c >> +++ b/lib/librte_eal/bsdapp/eal/eal_pci.c >> @@ -270,20 +270,6 @@ pci_uio_map_resource(struct rte_pci_device *dev) >> return (0); >> } >> >> -/* Compare two PCI device addresses. */ >> -static int >> -pci_addr_comparison(struct rte_pci_addr *addr, struct rte_pci_addr *addr2) >> -{ >> - uint64_t dev_addr = (addr->domain << 24) + (addr->bus << 16) + (addr->devid << 8) + addr->function; >> - uint64_t dev_addr2 = (addr2->domain << 24) + (addr2->bus << 16) + (addr2->devid << 8) + addr2->function; >> - >> - if (dev_addr > dev_addr2) >> - return 1; >> - else >> - return 0; >> -} >> - >> - >> /* Scan one pci sysfs entry, and fill the devices list from it. */ >> static int >> pci_scan_one(int dev_pci_fd, struct pci_conf *conf) >> @@ -356,13 +342,24 @@ pci_scan_one(int dev_pci_fd, struct pci_conf *conf) >> } >> else { >> struct rte_pci_device *dev2 = NULL; >> + int ret; >> >> TAILQ_FOREACH(dev2, &pci_device_list, next) { >> - if (pci_addr_comparison(&dev->addr, &dev2->addr)) >> + ret = eal_compare_pci_addr(&dev->addr, &dev2->addr); >> + if (ret > 0) >> continue; >> - else { >> + else if (ret < 0) { >> TAILQ_INSERT_BEFORE(dev2, dev, next); >> return 0; >> + } else { /* already registered */ >> + /* update pt_driver */ >> + dev2->pt_driver = dev->pt_driver; >> + dev2->max_vfs = dev->max_vfs; >> + memmove(dev2->mem_resource, >> + dev->mem_resource, >> + sizeof(dev->mem_resource)); >> + free(dev); >> + return 0; > Could you comment this "else part" please? PCI device list is allocated when rte_eal_init() is called. At the time, to fill pci device information, sysfs value is used. If sysfs values written by kernel device driver will not be changed by igb_uio, vfio or pci_uio_genereic, above code isn't needed. But actually above values are changed or added by them. Here is a step to cause issue. 1. Boot linux. 2. Start DPDK application without any physical NIC ports. - Here, some sysfs values are read, and store to pci device list. 3. igb_uio starts managing a port. - Here, some sysfs values are changed. 4. Add a NIC port to DPDK application using hotplug functions. - Here, we need to replace some values. > [...] >> --- a/lib/librte_eal/common/include/rte_pci.h >> +++ b/lib/librte_eal/common/include/rte_pci.h >> @@ -269,6 +269,40 @@ eal_parse_pci_DomBDF(const char *input, struct rte_pci_addr *dev_addr) >> } >> #undef GET_PCIADDR_FIELD >> >> +/* Compare two PCI device addresses. */ >> +/** >> + * Utility function to compare two PCI device addresses. >> + * >> + * @param addr >> + * The PCI Bus-Device-Function address to compare >> + * @param addr2 >> + * The PCI Bus-Device-Function address to compare >> + * @return >> + * 0 on equal PCI address. >> + * Positive on addr is greater than addr2. >> + * Negative on addr is less than addr2, or error. >> + */ >> +static inline int >> +eal_compare_pci_addr(struct rte_pci_addr *addr, struct rte_pci_addr *addr2) > I think that this function should be prefixed with rte_ Sure, I will. > [...] >> /* skip this element if it doesn't match our PCI address */ >> - if (memcmp(&uio_res->pci_addr, &dev->addr, sizeof(dev->addr))) >> + if (eal_compare_pci_addr(&uio_res->pci_addr, &dev->addr)) > Why memcmp is not sufficient to compare PCI addresses? > The only exception I see is endianness for natural sorting. Here is the definition. struct rte_pci_addr { uint16_t domain; /**< Device domain */ uint8_t bus; /**< Device bus */ uint8_t devid; /**< Device ID */ uint8_t function; /**< Device function. */ }; But, sizeof(struct rte_pci_addr) will be 6. If rte_pci_addr is allocated in a function without bzero, last 1 byte may have some value. As a result, memcmp may not work. To avoid such a case, I checked like above. >> continue; >> >> for (i = 0; i != uio_res->nb_maps; i++) { >>
On 2015/02/17 9:46, Thomas Monjalon wrote: > 2015-02-16 13:14, Tetsuya Mukawa: >> This patch adds rte_eth_dev_free(). The function is used for changing an >> attached status of the device that has specified name. >> >> v6: >> - Use rte_eth_dev structure as the paramter of rte_eth_dev_free(). >> v4: >> - Add parameter checking. >> >> Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp> >> --- >> lib/librte_ether/rte_ethdev.c | 11 +++++++++++ >> lib/librte_ether/rte_ethdev.h | 14 ++++++++++++++ >> 2 files changed, 25 insertions(+) >> >> diff --git a/lib/librte_ether/rte_ethdev.c b/lib/librte_ether/rte_ethdev.c >> index a79fa5b..2463d18 100644 >> --- a/lib/librte_ether/rte_ethdev.c >> +++ b/lib/librte_ether/rte_ethdev.c >> @@ -260,6 +260,17 @@ rte_eth_dev_allocate(const char *name) >> return eth_dev; >> } >> >> +int >> +rte_eth_dev_free(struct rte_eth_dev *eth_dev) >> +{ >> + if (eth_dev == NULL) >> + return -EINVAL; >> + >> + eth_dev->attached = 0; >> + nb_ports--; >> + return 0; >> +} > This function is strange. I would imagine it calling the free (uninit) function > of the driver. How about changing the name like below? rte_eth_dev_release_port() > [...] >> +/** >> + * Function for internal use by dummy drivers primarily, e.g. ring-based >> + * driver. >> + * Free the specified ethdev. >> + * >> + * @param eth_dev >> + * The *eth_dev* pointer is the address of the *rte_eth_dev* structure >> + * associated with the matching device and which have been [automatically] >> + * allocated in the *rte_eth_devices* array. >> + * @return >> + * - 0 on success, negative on error >> + */ >> +int rte_eth_dev_free(struct rte_eth_dev *eth_dev); >> + >> struct eth_driver; >> /** >> * @internal >> >
On 2015/02/17 9:56, Thomas Monjalon wrote: > 2015-02-16 13:14, Tetsuya Mukawa: >> The patch adds function pointer to rte_pci_driver and eth_driver >> structure. These function pointers are used when ports are detached. >> Also, the patch adds rte_eth_dev_uninit(). So far, it's not called >> by anywhere, but it will be called when port hotplug function is >> implemented. >> >> v6: >> - Fix rte_eth_dev_uninit() to handle a return value of uninit >> function of PMD. >> v4: >> - Add parameter checking. >> - Change function names. >> >> Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp> >> --- >> lib/librte_eal/common/include/rte_pci.h | 7 +++++ >> lib/librte_ether/rte_ethdev.c | 47 +++++++++++++++++++++++++++++++++ >> lib/librte_ether/rte_ethdev.h | 24 +++++++++++++++++ >> 3 files changed, 78 insertions(+) >> >> diff --git a/lib/librte_eal/common/include/rte_pci.h b/lib/librte_eal/common/include/rte_pci.h >> index 4814cd7..87ca4cf 100644 >> --- a/lib/librte_eal/common/include/rte_pci.h >> +++ b/lib/librte_eal/common/include/rte_pci.h >> @@ -189,12 +189,19 @@ struct rte_pci_driver; >> typedef int (pci_devinit_t)(struct rte_pci_driver *, struct rte_pci_device *); >> >> /** >> + * Uninitialisation function for the driver called during hotplugging. >> + */ >> +typedef int (pci_devuninit_t)( >> + struct rte_pci_driver *, struct rte_pci_device *); >> + >> +/** >> * A structure describing a PCI driver. >> */ >> struct rte_pci_driver { >> TAILQ_ENTRY(rte_pci_driver) next; /**< Next in list. */ >> const char *name; /**< Driver name. */ >> pci_devinit_t *devinit; /**< Device init. function. */ >> + pci_devuninit_t *devuninit; /**< Device uninit function. */ >> struct rte_pci_id *id_table; /**< ID table, NULL terminated. */ >> uint32_t drv_flags; /**< Flags contolling handling of device. */ >> }; >> diff --git a/lib/librte_ether/rte_ethdev.c b/lib/librte_ether/rte_ethdev.c >> index 2463d18..58d8072 100644 >> --- a/lib/librte_ether/rte_ethdev.c >> +++ b/lib/librte_ether/rte_ethdev.c >> @@ -326,6 +326,52 @@ rte_eth_dev_init(struct rte_pci_driver *pci_drv, >> return diag; >> } >> >> +static int >> +rte_eth_dev_uninit(struct rte_pci_driver *pci_drv, >> + struct rte_pci_device *pci_dev) > There's something strange here: this is an uninit of an ethdev but the parameter > is a pci_dev. rte_eth_dev_init and rte_eth_dev_uninit are called by PCI layer. I guess PCI layer cannot handle eth device or eth driver directly. I guess receiving pci dev in eth layer may be fair. But as you said, pci_drv can be removed. > The driver parameter shouldn't be needed. I will change the function like below. static int rte_eth_dev_uninit(struct rte_pci_device *pci_dev) >> +{ >> + struct eth_driver *eth_drv; >> + struct rte_eth_dev *eth_dev; >> + char ethdev_name[RTE_ETH_NAME_MAX_LEN]; >> + int ret; >> + >> + if ((pci_drv == NULL) || (pci_dev == NULL)) >> + return -EINVAL; >> + >> + /* Create unique Ethernet device name using PCI address */ >> + snprintf(ethdev_name, RTE_ETH_NAME_MAX_LEN, "%d:%d.%d", >> + pci_dev->addr.bus, pci_dev->addr.devid, >> + pci_dev->addr.function); > You should call a function common to init and uninit to generate > the same unique name. Sure, I will. >> + >> + eth_dev = rte_eth_dev_allocated(ethdev_name); >> + if (eth_dev == NULL) >> + return -ENODEV; >> + >> + eth_drv = (struct eth_driver *)pci_drv; >> + >> + /* Invoke PMD device uninit function */ >> + if (*eth_drv->eth_dev_uninit) { >> + ret = (*eth_drv->eth_dev_uninit)(eth_drv, eth_dev); >> + if (ret) >> + return ret; >> + } >> + >> + /* free ether device */ >> + rte_eth_dev_free(eth_dev); >> + >> + /* init user callbacks */ >> + TAILQ_INIT(&(eth_dev->callbacks)); > Please comment more why you are resetting callbacks. > An init in an uninit function seems weird ;) I agree. This code can be removed. (Actually callbacks will be initialized when device is initialized.) >> + >> + if (rte_eal_process_type() == RTE_PROC_PRIMARY) >> + rte_free(eth_dev->data->dev_private); >> + >> + eth_dev->pci_dev = NULL; >> + eth_dev->driver = NULL; >> + eth_dev->data = NULL; >> + >> + return 0; >> +} >> + >> /** >> * Register an Ethernet [Poll Mode] driver. >> * >> @@ -344,6 +390,7 @@ void >> rte_eth_driver_register(struct eth_driver *eth_drv) >> { >> eth_drv->pci_drv.devinit = rte_eth_dev_init; >> + eth_drv->pci_drv.devuninit = rte_eth_dev_uninit; >> rte_eal_pci_register(ð_drv->pci_drv); >> } >> >> diff --git a/lib/librte_ether/rte_ethdev.h b/lib/librte_ether/rte_ethdev.h >> index fbe7ac1..91d9e86 100644 >> --- a/lib/librte_ether/rte_ethdev.h >> +++ b/lib/librte_ether/rte_ethdev.h >> @@ -1678,6 +1678,27 @@ typedef int (*eth_dev_init_t)(struct eth_driver *eth_drv, >> >> /** >> * @internal >> + * Finalization function of an Ethernet driver invoked for each matching >> + * Ethernet PCI device detected during the PCI closing phase. >> + * >> + * @param eth_drv >> + * The pointer to the [matching] Ethernet driver structure supplied by >> + * the PMD when it registered itself. >> + * @param eth_dev >> + * The *eth_dev* pointer is the address of the *rte_eth_dev* structure >> + * associated with the matching device and which have been [automatically] >> + * allocated in the *rte_eth_devices* array. >> + * @return >> + * - 0: Success, the device is properly finalized by the driver. >> + * In particular, the driver MUST free the *dev_ops* pointer >> + * of the *eth_dev* structure. >> + * - <0: Error code of the device initialization failure. >> + */ >> +typedef int (*eth_dev_uninit_t)(struct eth_driver *eth_drv, >> + struct rte_eth_dev *eth_dev); >> + >> +/** >> + * @internal >> * The structure associated with a PMD Ethernet driver. >> * >> * Each Ethernet driver acts as a PCI driver and is represented by a generic >> @@ -1687,11 +1708,14 @@ typedef int (*eth_dev_init_t)(struct eth_driver *eth_drv, >> * >> * - The *eth_dev_init* function invoked for each matching PCI device. >> * >> + * - The *eth_dev_uninit* function invoked for each matching PCI device. >> + * >> * - The size of the private data to allocate for each matching device. >> */ >> struct eth_driver { >> struct rte_pci_driver pci_drv; /**< The PMD is also a PCI driver. */ >> eth_dev_init_t eth_dev_init; /**< Device init function. */ >> + eth_dev_uninit_t eth_dev_uninit; /**< Device uninit function. */ >> unsigned int dev_private_size; /**< Size of device private data. */ >> }; >> >> >
On 2015/02/17 10:11, Thomas Monjalon wrote: > 2015-02-16 13:14, Tetsuya Mukawa: >> The patch adds functions for unmapping igb_uio resources. The patch is only >> for Linux and igb_uio environment. VFIO and BSD are not supported. >> >> v8: >> - Fix typo. >> (Thanks to Iremonger, Bernard) >> v5: >> - Fix pci_unmap_device() to check pt_driver. >> v4: >> - Add parameter checking. >> - Add header file to determine if hotplug can be enabled. >> >> Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp> >> --- >> lib/librte_eal/common/Makefile | 1 + >> lib/librte_eal/common/include/rte_dev_hotplug.h | 44 +++++++++++++++++ >> lib/librte_eal/linuxapp/eal/eal_pci.c | 44 +++++++++++++++++ >> lib/librte_eal/linuxapp/eal/eal_pci_init.h | 8 +++ >> lib/librte_eal/linuxapp/eal/eal_pci_uio.c | 65 +++++++++++++++++++++++++ >> 5 files changed, 162 insertions(+) >> create mode 100644 lib/librte_eal/common/include/rte_dev_hotplug.h >> >> diff --git a/lib/librte_eal/common/Makefile b/lib/librte_eal/common/Makefile >> index 52c1a5f..db7cc93 100644 >> --- a/lib/librte_eal/common/Makefile >> +++ b/lib/librte_eal/common/Makefile >> @@ -41,6 +41,7 @@ INC += rte_eal_memconfig.h rte_malloc_heap.h >> INC += rte_hexdump.h rte_devargs.h rte_dev.h >> INC += rte_common_vect.h >> INC += rte_pci_dev_feature_defs.h rte_pci_dev_features.h >> +INC += rte_dev_hotplug.h >> >> ifeq ($(CONFIG_RTE_INSECURE_FUNCTION_WARNING),y) >> INC += rte_warnings.h >> diff --git a/lib/librte_eal/common/include/rte_dev_hotplug.h b/lib/librte_eal/common/include/rte_dev_hotplug.h >> new file mode 100644 >> index 0000000..b333e0f >> --- /dev/null >> +++ b/lib/librte_eal/common/include/rte_dev_hotplug.h >> @@ -0,0 +1,44 @@ >> +/*- >> + * BSD LICENSE >> + * >> + * Copyright(c) 2015 IGEL Co.,LTd. >> + * All rights reserved. >> + * >> + * Redistribution and use in source and binary forms, with or without >> + * modification, are permitted provided that the following conditions >> + * are met: >> + * >> + * * Redistributions of source code must retain the above copyright >> + * notice, this list of conditions and the following disclaimer. >> + * * Redistributions in binary form must reproduce the above copyright >> + * notice, this list of conditions and the following disclaimer in >> + * the documentation and/or other materials provided with the >> + * distribution. >> + * * Neither the name of IGEL Co.,Ltd. nor the names of its >> + * contributors may be used to endorse or promote products derived >> + * from this software without specific prior written permission. >> + * >> + * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS >> + * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT >> + * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR >> + * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT >> + * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, >> + * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT >> + * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, >> + * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY >> + * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT >> + * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE >> + * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. >> + */ >> + >> +#ifndef _RTE_DEV_HOTPLUG_H_ >> +#define _RTE_DEV_HOTPLUG_H_ >> + >> +/* >> + * determine if hotplug can be enabled on the system >> + */ >> +#if defined(RTE_LIBRTE_EAL_HOTPLUG) && defined(RTE_LIBRTE_EAL_LINUXAPP) >> +#define ENABLE_HOTPLUG >> +#endif /* RTE_LIBRTE_EAL_HOTPLUG & RTE_LIBRTE_EAL_LINUXAPP */ > Please no. > You just have to set the build option to no in the default BSD config file > with a comment explaining it is not supported. OK, I will change like you said. >> + >> +#endif /* _RTE_DEV_HOTPLUG_H_ */ >> diff --git a/lib/librte_eal/linuxapp/eal/eal_pci.c b/lib/librte_eal/linuxapp/eal/eal_pci.c >> index 2d5f6a6..72a1362 100644 >> --- a/lib/librte_eal/linuxapp/eal/eal_pci.c >> +++ b/lib/librte_eal/linuxapp/eal/eal_pci.c >> @@ -167,6 +167,25 @@ pci_map_resource(void *requested_addr, int fd, off_t offset, size_t size) >> return mapaddr; >> } >> >> +#ifdef ENABLE_HOTPLUG > Please avoid using #ifdef if not really necessary. I agree with you. In this case, only hotplug functions call pci_unmap_resource(). So this will be needed. >> +/* unmap a particular resource */ >> +void >> +pci_unmap_resource(void *requested_addr, size_t size) >> +{ >> + if (requested_addr == NULL) >> + return; >> + >> + /* Unmap the PCI memory resource of device */ >> + if (munmap(requested_addr, size)) { >> + RTE_LOG(ERR, EAL, "%s(): cannot munmap(%p, 0x%lx): %s\n", >> + __func__, requested_addr, (unsigned long)size, >> + strerror(errno)); >> + } else >> + RTE_LOG(DEBUG, EAL, " PCI memory unmapped at %p\n", >> + requested_addr); >> +} >> +#endif /* ENABLE_HOTPLUG */ >> + >> /* parse the "resource" sysfs file */ >> #define IORESOURCE_MEM 0x00000200 > [...]
On 2015/02/17 10:18, Thomas Monjalon wrote: > 2015-02-16 13:14, Tetsuya Mukawa: >> - Add rte_eal_pci_close_one_dirver() >> The function is used for closing the specified driver and device. >> - Add pci_invoke_all_drivers() >> The function is based on pci_probe_all_drivers. But it can not only >> probe but also close drivers. >> - Add pci_close_all_drivers() >> The function tries to find a driver for the specified device, and >> then close the driver. >> - Add rte_eal_pci_probe_one() and rte_eal_pci_close_one() >> The functions are used for probe and close a device. >> First the function tries to find a device that has the specified >> PCI address. Then, probe or close the device. >> >> v5: >> - Remove RTE_EAL_INVOKE_TYPE_UNKNOWN, because it's unused. >> v4: >> - Fix parameter checking. >> - Fix indent of 'if' statement. >> >> Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp> >> --- >> lib/librte_eal/common/eal_common_pci.c | 90 +++++++++++++++++++++++++++++---- >> lib/librte_eal/common/eal_private.h | 24 +++++++++ >> lib/librte_eal/common/include/rte_pci.h | 33 ++++++++++++ >> lib/librte_eal/linuxapp/eal/eal_pci.c | 69 +++++++++++++++++++++++++ >> 4 files changed, 206 insertions(+), 10 deletions(-) > 206 insertions and 10 deletions: it cannot really be a cleanup ;) > Maybe the title should be reworded. I will reword the tile. > > [...] >> - rc = rte_eal_pci_probe_one_driver(dr, dev); >> + switch (type) { >> + case RTE_EAL_INVOKE_TYPE_PROBE: >> + rc = rte_eal_pci_probe_one_driver(dr, dev); >> + break; >> + case RTE_EAL_INVOKE_TYPE_CLOSE: >> + rc = rte_eal_pci_close_one_driver(dr, dev); >> + break; > Honestly, I don't like this kind of functions with a switch to toggle > different actions. It makes code unclear. > Sure, I will remove such a toggle. Probably I will separate the function.
On 2015/02/17 10:24, Thomas Monjalon wrote: > 2015-02-16 13:14, Tetsuya Mukawa: >> This new parameter is needed to keep device type like physical or virtual. > Actually types are "PCI" and "virtual". OK, I will change above sentence to explain it more clearly. >> Port detaching processes are different between physical and virtual. >> This parameter lets detaching function know a device type of the port. > I think the type should be guessed automatically thanks to register infos. > It seems I forgot rewrite this comment, I will change it.
On 2015/02/17 10:25, Thomas Monjalon wrote:
> 2015-02-16 13:14, Tetsuya Mukawa:
>> The patch enables CONFIG_RTE_LIBRTE_EAL_HOTPLUG in Linux configuration.
> This patch should be introduced earlier in the patchset.
Sure, I will.
On 2015/02/17 10:04, Thomas Monjalon wrote: > 2015-02-16 13:14, Tetsuya Mukawa: >> The patch adds following functions. >> >> - rte_eth_dev_save() >> The function is used for saving current rte_eth_dev structures. >> - rte_eth_dev_get_changed_port() >> The function receives the rte_eth_dev structures, then compare >> these with current values to know which port is actually >> attached or detached. >> - rte_eth_dev_get_addr_by_port() >> The function returns a pci address of an ethdev specified by port >> identifier. >> - rte_eth_dev_get_port_by_addr() >> The function returns a port identifier of an ethdev specified by >> pci address. >> - rte_eth_dev_get_name_by_port() >> The function returns a unique identifier name of an ethdev >> specified by port identifier. >> - Add rte_eth_dev_check_detachable() >> The function returns whether a PMD supports detach function. >> >> Also, the patch changes scope of rte_eth_dev_allocated() to global. >> This function will be called by virtual PMDs to support port hotplug. >> So change scope of the function to global. >> >> v8: >> - Add size parameter to rte_eth_dev_save(). >> - Add missing symbol in version map. >> (Thanks to Qiu, Michael and Iremonger, Bernard) >> v7: >> - Add pt_driver checking to rte_eth_dev_check_detachable(). >> (Thanks to Qiu, Michael) >> v5: >> - Fix return value of below functions. >> rte_eth_dev_get_changed_port(). >> rte_eth_dev_get_port_by_addr(). >> v4: >> - Add parameter checking. >> v3: >> - Fix if-condition bug while comparing pci addresses. >> - Add error checking codes. >> Reported-by: Mark Enright <menrigh@brocade.com> >> >> Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp> >> --- >> lib/librte_ether/rte_ethdev.c | 99 +++++++++++++++++++++++++++++++++- >> lib/librte_ether/rte_ethdev.h | 83 ++++++++++++++++++++++++++++ >> lib/librte_ether/rte_ether_version.map | 6 +++ >> 3 files changed, 187 insertions(+), 1 deletion(-) >> >> diff --git a/lib/librte_ether/rte_ethdev.c b/lib/librte_ether/rte_ethdev.c >> index 58d8072..3869a96 100644 >> --- a/lib/librte_ether/rte_ethdev.c >> +++ b/lib/librte_ether/rte_ethdev.c >> @@ -206,7 +206,7 @@ rte_eth_dev_data_alloc(void) >> RTE_MAX_ETHPORTS * sizeof(*rte_eth_dev_data)); >> } >> >> -static struct rte_eth_dev * >> +struct rte_eth_dev * >> rte_eth_dev_allocated(const char *name) >> { >> unsigned i; >> @@ -426,6 +426,103 @@ rte_eth_dev_count(void) >> return (nb_ports); >> } >> >> +int >> +rte_eth_dev_save(struct rte_eth_dev *devs, size_t size) >> +{ >> + if ((devs == NULL) || >> + (size != sizeof(struct rte_eth_dev) * RTE_MAX_ETHPORTS)) >> + return -EINVAL; >> + >> + /* save current rte_eth_devices */ >> + memcpy(devs, rte_eth_devices, size); >> + return 0; >> +} > Why creating a function for a memcpy? When PCI layer initializes or un-initializes an eth device, a port id that is actually plugged cannot be know out of eth layer. But hotplug function needs to return the port id to DPDK application. This function is used like below. 1. Hotplug function calls this function to save port status. 2. Attach or detach. 3. Hotplug function calls rte_eth_dev_get_changed_port() to know port id actually plugged. Above steps are done in rte_dev_attach() and rte_dev_detach(). And these 2 functions are not thread safe, so DPDK application needs to handle it correctly. >> + >> +int >> +rte_eth_dev_get_changed_port(struct rte_eth_dev *devs, uint8_t *port_id) >> +{ >> + if ((devs == NULL) || (port_id == NULL)) >> + return -EINVAL; >> + >> + /* check which port was attached or detached */ >> + for (*port_id = 0; *port_id < RTE_MAX_ETHPORTS; (*port_id)++, devs++) { >> + if (rte_eth_devices[*port_id].attached ^ devs->attached) >> + return 0; >> + } >> + return -ENODEV; >> +} > It seems weird to require this function. > Functions which are attaching a new port should return the port_id. Please check above comment. >> + >> +int >> +rte_eth_dev_get_addr_by_port(uint8_t port_id, struct rte_pci_addr *addr) >> +{ >> + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) >> + return -EINVAL; >> + >> + if (addr == NULL) { >> + PMD_DEBUG_TRACE("Null pointer is specified\n"); >> + return -EINVAL; >> + } >> + >> + *addr = rte_eth_devices[port_id].pci_dev->addr; >> + return 0; >> +} > Again, I'm not sure this function is needed. PCI layer doesn't know relation between port id and pci address. Eth layer only knows it. So the function is needed. >> + >> +int >> +rte_eth_dev_get_port_by_addr(struct rte_pci_addr *addr, uint8_t *port_id) >> +{ >> + struct rte_pci_addr *tmp; >> + >> + if ((addr == NULL) || (port_id == NULL)) { >> + PMD_DEBUG_TRACE("Null pointer is specified\n"); >> + return -EINVAL; >> + } >> + >> + for (*port_id = 0; *port_id < RTE_MAX_ETHPORTS; (*port_id)++) { >> + if (!rte_eth_devices[*port_id].attached) >> + continue; >> + if (!rte_eth_devices[*port_id].pci_dev) >> + continue; >> + tmp = &rte_eth_devices[*port_id].pci_dev->addr; >> + if (eal_compare_pci_addr(tmp, addr) == 0) >> + return 0; >> + } >> + return -ENODEV; >> +} >> + >> +int >> +rte_eth_dev_get_name_by_port(uint8_t port_id, char *name) >> +{ >> + char *tmp; >> + >> + if (rte_eth_dev_validate_port(port_id, TRACE) == DEV_INVALID) >> + return -EINVAL; >> + >> + if (name == NULL) { >> + PMD_DEBUG_TRACE("Null pointer is specified\n"); >> + return -EINVAL; >> + } >> + >> + /* shouldn't check 'rte_eth_devices[i].data', >> + * because it might be overwritten by VDEV PMD */ > I don't understand the comment. Sorry for my English. rte_eth_devices[i].data is over written by some PMDs like pcap PMD. Please check following code. static int rte_pmd_init_internals() { struct rte_eth_dev_data *data = NULL; .....snip..... data = rte_zmalloc_socket(name, sizeof(*data), 0, numa_node); .....snip..... (*eth_dev)->data = data; .....snip..... } 'data' is over written like above, but name is kept. So use it for comparing. >> + tmp = rte_eth_dev_data[port_id].name; >> + strncpy(name, tmp, strlen(tmp) + 1); > tmp seems useless. > strncpy with strlen should be equivalent to strcpy. Sure, I will fix it. > >> + return 0; >> +} >> + >> +int >> +rte_eth_dev_check_detachable(uint8_t port_id) > Why not "is_detachable" instead of "check_detachable"? I will change it. >> +{ >> + uint32_t drv_flags; >> + >> + if (port_id >= RTE_MAX_ETHPORTS) { >> + PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); >> + return -EINVAL; >> + } >> + >> + drv_flags = rte_eth_devices[port_id].driver->pci_drv.drv_flags; >> + return !(drv_flags & RTE_PCI_DRV_DETACHABLE); >> +} >> + >> static int >> rte_eth_dev_rx_queue_config(struct rte_eth_dev *dev, uint16_t nb_queues) >> { >> diff --git a/lib/librte_ether/rte_ethdev.h b/lib/librte_ether/rte_ethdev.h >> index 91d9e86..6651890 100644 >> --- a/lib/librte_ether/rte_ethdev.h >> +++ b/lib/librte_ether/rte_ethdev.h >> @@ -1616,6 +1616,89 @@ extern struct rte_eth_dev rte_eth_devices[]; >> extern uint8_t rte_eth_dev_count(void); >> >> /** >> + * Function for internal use by port hotplug functions. >> + * Copies current ethdev structures to the specified pointer. >> + * >> + * @param devs The pointer to the ethdev structures >> + * @param size The size of ethdev structures >> + * @return >> + * - 0 on success, negative on error >> + */ >> +extern int rte_eth_dev_save(struct rte_eth_dev *devs, size_t size); >> + >> +/** >> + * Function for internal use by port hotplug functions. >> + * Compare the specified ethdev structures with currents. Then >> + * if there is a port which status is changed, fill the specified pointer >> + * with the port id of that port. >> + * @param devs The pointer to the ethdev structures >> + * @param port_id The pointer to the port id >> + * @return >> + * - 0 on success, negative on error >> + */ >> +extern int rte_eth_dev_get_changed_port( >> + struct rte_eth_dev *devs, uint8_t *port_id); >> + >> +/** >> + * Function for internal use by port hotplug functions. >> + * Returns a pci address of a ethdev specified by port identifier. >> + * @param port_id >> + * The port identifier of the Ethernet device >> + * @param addr >> + * The pointer to the pci address >> + * @return >> + * - 0 on success, negative on error >> + */ >> +extern int rte_eth_dev_get_addr_by_port( >> + uint8_t port_id, struct rte_pci_addr *addr); >> + >> +/** >> + * Function for internal use by port hotplug functions. >> + * Returns a port identifier of a ethdev specified by pci address. >> + * @param addr >> + * The pointer to the pci address of the Ethernet device. >> + * @param port_id >> + * The pointer to the port identifier >> + * @return >> + * - 0 on success, negative on error >> + */ >> +extern int rte_eth_dev_get_port_by_addr( >> + struct rte_pci_addr *addr, uint8_t *port_id); >> + >> +/** >> + * Function for internal use by port hotplug functions. >> + * Returns a unique identifier name of a ethdev specified by port identifier. >> + * @param port_id >> + * The port identifier. >> + * @param name >> + * The pointer to the Unique identifier name for each Ethernet device >> + * @return >> + * - 0 on success, negative on error >> + */ >> +extern int rte_eth_dev_get_name_by_port(uint8_t port_id, char *name); >> + >> +/** >> + * Function for internal use by port hotplug functions. >> + * Check whether or not, a PMD that is handling the ethdev specified by port >> + * identifier can support detach function. >> + * @param port_id >> + * The port identifier >> + * @return >> + * - 0 on supporting detach function, negative on not supporting >> + */ >> +extern int rte_eth_dev_check_detachable(uint8_t port_id); >> + >> +/** >> + * Function for internal use by port hotplug functions. >> + * Returns a ethdev slot specified by the unique identifier name. >> + * @param name >> + * The pointer to the Unique identifier name for each Ethernet device >> + * @return >> + * - The pointer to the ethdev slot, on success. NULL on error >> + */ >> +extern struct rte_eth_dev *rte_eth_dev_allocated(const char *name); >> + >> +/** >> * Function for internal use by dummy drivers primarily, e.g. ring-based >> * driver. >> * Allocates a new ethdev slot for an ethernet device and returns the pointer >> diff --git a/lib/librte_ether/rte_ether_version.map b/lib/librte_ether/rte_ether_version.map >> index 7316530..fc5dc27 100644 >> --- a/lib/librte_ether/rte_ether_version.map >> +++ b/lib/librte_ether/rte_ether_version.map >> @@ -109,6 +109,12 @@ DPDK_2.0 { >> rte_eth_tx_queue_setup; >> rte_eth_xstats_get; >> rte_eth_xstats_reset; >> + rte_eth_dev_check_detachable; >> + rte_eth_dev_get_name_by_port; >> + rte_eth_dev_get_addr_by_port; >> + rte_eth_dev_get_port_by_addr; >> + rte_eth_dev_get_changed_port; >> + rte_eth_dev_save; >> >> local: *; >> }; >> >
On 2015/02/17 10:48, Thomas Monjalon wrote: > 2015-02-16 13:14, Tetsuya Mukawa: >> These functions are used for attaching or detaching a port. >> When rte_eal_dev_attach() is called, the function tries to realize the >> device name as pci address. If this is done successfully, >> rte_eal_dev_attach() will attach physical device port. If not, attaches >> virtual devive port. >> When rte_eal_dev_detach() is called, the function gets the device type >> of this port to know whether the port is come from physical or virtual. >> And then specific detaching function will be called. >> >> v8: >> - Add missing symbol in version map. >> (Thanks to Qiu, Michael and Iremonger, Bernard) >> v7: >> - Fix typo of warning messages. >> (Thanks to Qiu, Michael) >> v5: >> - Change function names like below. >> rte_eal_dev_find_and_invoke() to rte_eal_vdev_find_and_invoke(). >> rte_eal_dev_invoke() to rte_eal_vdev_invoke(). >> - Add code to handle a return value of rte_eal_devargs_remove(). >> - Fix pci address format in rte_eal_dev_detach(). >> v4: >> - Fix comment. >> - Add error checking. >> - Fix indent of 'if' statement. >> - Change function name. >> >> Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp> >> --- >> lib/librte_eal/common/eal_common_dev.c | 276 ++++++++++++++++++++++++ >> lib/librte_eal/common/eal_private.h | 11 + >> lib/librte_eal/common/include/rte_dev.h | 33 +++ >> lib/librte_eal/linuxapp/eal/Makefile | 1 + >> lib/librte_eal/linuxapp/eal/eal_pci.c | 6 +- >> lib/librte_eal/linuxapp/eal/rte_eal_version.map | 2 + >> 6 files changed, 326 insertions(+), 3 deletions(-) >> >> diff --git a/lib/librte_eal/common/eal_common_dev.c b/lib/librte_eal/common/eal_common_dev.c >> index eae5656..3d169a4 100644 >> --- a/lib/librte_eal/common/eal_common_dev.c >> +++ b/lib/librte_eal/common/eal_common_dev.c >> @@ -32,10 +32,13 @@ >> * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. >> */ >> >> +#include <stdio.h> >> +#include <limits.h> >> #include <string.h> >> #include <inttypes.h> >> #include <sys/queue.h> >> >> +#include <rte_ethdev.h> >> #include <rte_dev.h> >> #include <rte_devargs.h> >> #include <rte_debug.h> >> @@ -107,3 +110,276 @@ rte_eal_dev_init(void) >> } >> return 0; >> } >> + >> +/* So far, DPDK hotplug function only supports linux */ > This comment should be in the configuration. Sure I will. > >> +#ifdef ENABLE_HOTPLUG >> +static void >> +rte_eal_vdev_invoke(struct rte_driver *driver, >> + struct rte_devargs *devargs, enum rte_eal_invoke_type type) >> +{ >> + if ((driver == NULL) || (devargs == NULL)) >> + return; >> + >> + switch (type) { >> + case RTE_EAL_INVOKE_TYPE_PROBE: >> + driver->init(devargs->virtual.drv_name, devargs->args); >> + break; >> + case RTE_EAL_INVOKE_TYPE_CLOSE: >> + driver->uninit(devargs->virtual.drv_name, devargs->args); >> + break; >> + default: >> + break; >> + } >> +} > It would be clearer to directly call init and uninit instead of using this > "invoke" method. Sure, I will change it. >> + >> +static int >> +rte_eal_vdev_find_and_invoke(const char *name, int type) >> +{ >> + struct rte_devargs *devargs; >> + struct rte_driver *driver; >> + >> + if (name == NULL) >> + return -EINVAL; >> + >> + /* call the init function for each virtual device */ > This comment is wrong. Thanks, I will fix it. >> + TAILQ_FOREACH(devargs, &devargs_list, next) { >> + >> + if (devargs->type != RTE_DEVTYPE_VIRTUAL) >> + continue; >> + >> + if (strncmp(name, devargs->virtual.drv_name, strlen(name))) >> + continue; >> + >> + TAILQ_FOREACH(driver, &dev_driver_list, next) { >> + if (driver->type != PMD_VDEV) >> + continue; >> + >> + /* search a driver prefix in virtual device name */ >> + if (!strncmp(driver->name, devargs->virtual.drv_name, >> + strlen(driver->name))) { > Why not use strcmp? I will replace it. >> + rte_eal_vdev_invoke(driver, devargs, type); >> + break; >> + } >> + } >> + >> + if (driver == NULL) { >> + RTE_LOG(WARNING, EAL, "no driver found for %s\n", >> + devargs->virtual.drv_name); >> + } >> + return 0; >> + } >> + return 1; >> +} >> + >> +/* attach the new physical device, then store port_id of the device */ >> +static int >> +rte_eal_dev_attach_pdev(struct rte_pci_addr *addr, uint8_t *port_id) >> +{ >> + uint8_t new_port_id; >> + struct rte_eth_dev devs[RTE_MAX_ETHPORTS]; >> + >> + if ((addr == NULL) || (port_id == NULL)) >> + goto err; >> + >> + /* save current port status */ >> + if (rte_eth_dev_save(devs, sizeof(devs))) >> + goto err; >> + /* re-construct pci_device_list */ >> + if (rte_eal_pci_scan()) >> + goto err; >> + /* invoke probe func of the driver can handle the new device */ >> + if (rte_eal_pci_probe_one(addr)) >> + goto err; > You should get the port_id from the previous function instead of searching it. I agree this will beautify this code, but actually to do like above changes current DPDK code much more, and it will not be clear, and not beautiful. Could I explain it more. Problem is initialization sequence of virtual device and physical device are completely different. (1) Attaching a physical device case - To return port id, pci_invoke_all_drivers() needs to return port id. - It means "devinit" of "struct rte_pci_driver" needs to return port_id. - "devinit" will be rte_eth_dev_init(). But if the device is virtual, this function is not implemented. (2) Attaching virtual device case - To return port id from rte_eal_pci_probe_one(), pci_invoke_all_drivers() needs to return port id. - It means "init" of "struct rte_driver" needs to return port_id. - "init" will be implemented in PMD. But this function has different usage in physical device and virtual device. - Especially, In the case of physical device, "init" doesn't allocate eth device, so cannot return port id. As a result, to remove rte_eth_dev_save() and rte_eth_dev_get_changed_port(), below different functions needs to return port id. - "devinit" of "struct rte_pci_driver". - "init" of "struct rte_driver". That is why I implement like above. >> + /* get port_id enabled by above procedures */ >> + if (rte_eth_dev_get_changed_port(devs, &new_port_id)) >> + goto err; >> + >> + *port_id = new_port_id; >> + return 0; >> +err: >> + RTE_LOG(ERR, EAL, "Driver, cannot attach the device\n"); >> + return -1; >> +} > [...] > >> +/* attach the new virtual device, then store port_id of the device */ >> +static int >> +rte_eal_dev_attach_vdev(const char *vdevargs, uint8_t *port_id) >> +{ >> + char *args; >> + uint8_t new_port_id; >> + struct rte_eth_dev devs[RTE_MAX_ETHPORTS]; >> + >> + if ((vdevargs == NULL) || (port_id == NULL)) >> + goto err0; >> + >> + args = strdup(vdevargs); >> + if (args == NULL) >> + goto err0; >> + >> + /* save current port status */ >> + if (rte_eth_dev_save(devs, sizeof(devs))) >> + goto err1; >> + /* add the vdevargs to devargs_list */ >> + if (rte_eal_devargs_add(RTE_DEVTYPE_VIRTUAL, args)) >> + goto err1; >> + /* parse vdevargs, then retrieve device name */ >> + get_vdev_name(args); >> + /* walk around dev_driver_list to find the driver of the device, >> + * then invoke probe function o the driver */ >> + if (rte_eal_vdev_find_and_invoke(args, RTE_EAL_INVOKE_TYPE_PROBE)) >> + goto err2; > Again, you should get port_id from the attach procedure. > >> + /* get port_id enabled by above procedures */ >> + if (rte_eth_dev_get_changed_port(devs, &new_port_id)) >> + goto err2; > [...] > >> /** >> + * Uninitilization function called for each device driver once. >> + */ >> +typedef int (rte_dev_uninit_t)(const char *name, const char *args); > Why do you need args for uninit? > I just added for the case that finalization code of PMD needs it. But, probably "args" parameter can be removed.
2015-02-17 17:51, Tetsuya Mukawa: > On 2015/02/17 10:48, Thomas Monjalon wrote: > > 2015-02-16 13:14, Tetsuya Mukawa: > >> +/* attach the new physical device, then store port_id of the device */ > >> +static int > >> +rte_eal_dev_attach_pdev(struct rte_pci_addr *addr, uint8_t *port_id) > >> +{ > >> + uint8_t new_port_id; > >> + struct rte_eth_dev devs[RTE_MAX_ETHPORTS]; > >> + > >> + if ((addr == NULL) || (port_id == NULL)) > >> + goto err; > >> + > >> + /* save current port status */ > >> + if (rte_eth_dev_save(devs, sizeof(devs))) > >> + goto err; > >> + /* re-construct pci_device_list */ > >> + if (rte_eal_pci_scan()) > >> + goto err; > >> + /* invoke probe func of the driver can handle the new device */ > >> + if (rte_eal_pci_probe_one(addr)) > >> + goto err; > > You should get the port_id from the previous function instead of searching it. > > I agree this will beautify this code, but actually to do like above > changes current DPDK code much more, and it will not be clear, and not > beautiful. > > Could I explain it more. > Problem is initialization sequence of virtual device and physical device > are completely different. > > (1) Attaching a physical device case > - To return port id, pci_invoke_all_drivers() needs to return port id. > - It means "devinit" of "struct rte_pci_driver" needs to return port_id. > - "devinit" will be rte_eth_dev_init(). But if the device is virtual, > this function is not implemented. > > (2) Attaching virtual device case > - To return port id from rte_eal_pci_probe_one(), > pci_invoke_all_drivers() needs to return port id. > - It means "init" of "struct rte_driver" needs to return port_id. > - "init" will be implemented in PMD. But this function has different > usage in physical device and virtual device. > - Especially, In the case of physical device, "init" doesn't allocate > eth device, so cannot return port id. > > As a result, to remove rte_eth_dev_save() and > rte_eth_dev_get_changed_port(), below different functions needs to > return port id. > - "devinit" of "struct rte_pci_driver". > - "init" of "struct rte_driver". Yes, exactly, I think you shouldn't hesitate to improve existing EAL code. And I also think we should try to remove differences between virtual and pci devices. > That is why I implement like above. > > >> + /* get port_id enabled by above procedures */ > >> + if (rte_eth_dev_get_changed_port(devs, &new_port_id)) > >> + goto err; > >> + > >> + *port_id = new_port_id; > >> + return 0; > >> +err: > >> + RTE_LOG(ERR, EAL, "Driver, cannot attach the device\n"); > >> + return -1; > >> +} > > [...] > > > >> +/* attach the new virtual device, then store port_id of the device */ > >> +static int > >> +rte_eal_dev_attach_vdev(const char *vdevargs, uint8_t *port_id) > >> +{ > >> + char *args; > >> + uint8_t new_port_id; > >> + struct rte_eth_dev devs[RTE_MAX_ETHPORTS]; > >> + > >> + if ((vdevargs == NULL) || (port_id == NULL)) > >> + goto err0; > >> + > >> + args = strdup(vdevargs); > >> + if (args == NULL) > >> + goto err0; > >> + > >> + /* save current port status */ > >> + if (rte_eth_dev_save(devs, sizeof(devs))) > >> + goto err1; > >> + /* add the vdevargs to devargs_list */ > >> + if (rte_eal_devargs_add(RTE_DEVTYPE_VIRTUAL, args)) > >> + goto err1; > >> + /* parse vdevargs, then retrieve device name */ > >> + get_vdev_name(args); > >> + /* walk around dev_driver_list to find the driver of the device, > >> + * then invoke probe function o the driver */ > >> + if (rte_eal_vdev_find_and_invoke(args, RTE_EAL_INVOKE_TYPE_PROBE)) > >> + goto err2; > > Again, you should get port_id from the attach procedure. > > > >> + /* get port_id enabled by above procedures */ > >> + if (rte_eth_dev_get_changed_port(devs, &new_port_id)) > >> + goto err2; > > [...] > > > >> /** > >> + * Uninitilization function called for each device driver once. > >> + */ > >> +typedef int (rte_dev_uninit_t)(const char *name, const char *args); > > Why do you need args for uninit? > > > > I just added for the case that finalization code of PMD needs it. > But, probably "args" parameter can be removed. Yes I think
On 2015/02/17 18:23, Thomas Monjalon wrote: > 2015-02-17 17:51, Tetsuya Mukawa: >> On 2015/02/17 10:48, Thomas Monjalon wrote: >>> 2015-02-16 13:14, Tetsuya Mukawa: >>>> +/* attach the new physical device, then store port_id of the device */ >>>> +static int >>>> +rte_eal_dev_attach_pdev(struct rte_pci_addr *addr, uint8_t *port_id) >>>> +{ >>>> + uint8_t new_port_id; >>>> + struct rte_eth_dev devs[RTE_MAX_ETHPORTS]; >>>> + >>>> + if ((addr == NULL) || (port_id == NULL)) >>>> + goto err; >>>> + >>>> + /* save current port status */ >>>> + if (rte_eth_dev_save(devs, sizeof(devs))) >>>> + goto err; >>>> + /* re-construct pci_device_list */ >>>> + if (rte_eal_pci_scan()) >>>> + goto err; >>>> + /* invoke probe func of the driver can handle the new device */ >>>> + if (rte_eal_pci_probe_one(addr)) >>>> + goto err; >>> You should get the port_id from the previous function instead of searching it. >> I agree this will beautify this code, but actually to do like above >> changes current DPDK code much more, and it will not be clear, and not >> beautiful. >> >> Could I explain it more. >> Problem is initialization sequence of virtual device and physical device >> are completely different. >> >> (1) Attaching a physical device case >> - To return port id, pci_invoke_all_drivers() needs to return port id. >> - It means "devinit" of "struct rte_pci_driver" needs to return port_id. >> - "devinit" will be rte_eth_dev_init(). But if the device is virtual, >> this function is not implemented. >> >> (2) Attaching virtual device case >> - To return port id from rte_eal_pci_probe_one(), >> pci_invoke_all_drivers() needs to return port id. >> - It means "init" of "struct rte_driver" needs to return port_id. >> - "init" will be implemented in PMD. But this function has different >> usage in physical device and virtual device. >> - Especially, In the case of physical device, "init" doesn't allocate >> eth device, so cannot return port id. >> >> As a result, to remove rte_eth_dev_save() and >> rte_eth_dev_get_changed_port(), below different functions needs to >> return port id. >> - "devinit" of "struct rte_pci_driver". >> - "init" of "struct rte_driver". > Yes, exactly, > I think you shouldn't hesitate to improve existing EAL code. > And I also think we should try to remove differences between virtual and > pci devices. I strongly agree with it. But I haven't investigated how to remove it so far. To be honest, I want to submit hotplug patches to DPDK-2.0. Is above functionality needed to merge the hotplug patches? I guess I will not be able to remove it by 23rd. Regards, Tetsuya >> That is why I implement like above. >> >>>> + /* get port_id enabled by above procedures */ >>>> + if (rte_eth_dev_get_changed_port(devs, &new_port_id)) >>>> + goto err; >>>> + >>>> + *port_id = new_port_id; >>>> + return 0; >>>> +err: >>>> + RTE_LOG(ERR, EAL, "Driver, cannot attach the device\n"); >>>> + return -1; >>>> +} >>> [...] >>> >>>> +/* attach the new virtual device, then store port_id of the device */ >>>> +static int >>>> +rte_eal_dev_attach_vdev(const char *vdevargs, uint8_t *port_id) >>>> +{ >>>> + char *args; >>>> + uint8_t new_port_id; >>>> + struct rte_eth_dev devs[RTE_MAX_ETHPORTS]; >>>> + >>>> + if ((vdevargs == NULL) || (port_id == NULL)) >>>> + goto err0; >>>> + >>>> + args = strdup(vdevargs); >>>> + if (args == NULL) >>>> + goto err0; >>>> + >>>> + /* save current port status */ >>>> + if (rte_eth_dev_save(devs, sizeof(devs))) >>>> + goto err1; >>>> + /* add the vdevargs to devargs_list */ >>>> + if (rte_eal_devargs_add(RTE_DEVTYPE_VIRTUAL, args)) >>>> + goto err1; >>>> + /* parse vdevargs, then retrieve device name */ >>>> + get_vdev_name(args); >>>> + /* walk around dev_driver_list to find the driver of the device, >>>> + * then invoke probe function o the driver */ >>>> + if (rte_eal_vdev_find_and_invoke(args, RTE_EAL_INVOKE_TYPE_PROBE)) >>>> + goto err2; >>> Again, you should get port_id from the attach procedure. >>> >>>> + /* get port_id enabled by above procedures */ >>>> + if (rte_eth_dev_get_changed_port(devs, &new_port_id)) >>>> + goto err2; >>> [...] >>> >>>> /** >>>> + * Uninitilization function called for each device driver once. >>>> + */ >>>> +typedef int (rte_dev_uninit_t)(const char *name, const char *args); >>> Why do you need args for uninit? >>> >> I just added for the case that finalization code of PMD needs it. >> But, probably "args" parameter can be removed. > Yes I think > >
Hi Tetsuya, On Tue, Feb 17, 2015 at 9:51 AM, Tetsuya Mukawa <mukawa@igel.co.jp> wrote: > > > >> + /* get port_id enabled by above procedures */ > >> + if (rte_eth_dev_get_changed_port(devs, &new_port_id)) > >> + goto err2; > > [...] > > > >> /** > >> + * Uninitilization function called for each device driver once. > >> + */ > >> +typedef int (rte_dev_uninit_t)(const char *name, const char *args); > > Why do you need args for uninit? > > > > I just added for the case that finalization code of PMD needs it. > But, probably "args" parameter can be removed. > > I think there are no needs to have any args in the uninit function: 1) You librte_pmd_null doesn't use it 2) You give exactly the same argument that was used by the init function. A driver should have already stored these parameters in an internal private structure at initialization. So it's not needed to give me back for uninit method. >From my understanding devargs_list is only needed at the init to store the arguments when we parse the command line. Then, at initialization, rte_eal_dev_init creates the devices from this list . By removing args from uninit function, you doesn't need to add and remove anymore devargs in devargs_list to (de)attach a new device. What do you think ? Maxime
2015-02-17 15:14, Tetsuya Mukawa:
> On 2015/02/17 9:36, Thomas Monjalon wrote:
> > 2015-02-16 13:14, Tetsuya Mukawa:
> > Is uint8_t sill a good size for hotpluggable virtual device ids?
>
> I am not sure it's enough, but uint8_t is widely used in "rte_ethdev.c"
> as port id.
> If someone reports it doesn't enough, I guess it will be the time to
> write a patch to change all uint_8 in one patch.
It's a big ABI breakage. So if we feel it's going to be required,
it's better to do it now in 2.0 release I think.
Any opinion?
2015-02-17 15:14, Tetsuya Mukawa: > On 2015/02/17 9:44, Thomas Monjalon wrote: > > 2015-02-16 13:14, Tetsuya Mukawa: > >> @@ -356,13 +342,24 @@ pci_scan_one(int dev_pci_fd, struct pci_conf *conf) > >> } > >> else { > >> struct rte_pci_device *dev2 = NULL; > >> + int ret; > >> > >> TAILQ_FOREACH(dev2, &pci_device_list, next) { > >> - if (pci_addr_comparison(&dev->addr, &dev2->addr)) > >> + ret = eal_compare_pci_addr(&dev->addr, &dev2->addr); > >> + if (ret > 0) > >> continue; > >> - else { > >> + else if (ret < 0) { > >> TAILQ_INSERT_BEFORE(dev2, dev, next); > >> return 0; > >> + } else { /* already registered */ > >> + /* update pt_driver */ > >> + dev2->pt_driver = dev->pt_driver; > >> + dev2->max_vfs = dev->max_vfs; > >> + memmove(dev2->mem_resource, > >> + dev->mem_resource, > >> + sizeof(dev->mem_resource)); > >> + free(dev); > >> + return 0; > > Could you comment this "else part" please? > > PCI device list is allocated when rte_eal_init() is called. At the time, > to fill pci device information, sysfs value is used. > If sysfs values written by kernel device driver will not be changed by > igb_uio, vfio or pci_uio_genereic, above code isn't needed. > But actually above values are changed or added by them. > > Here is a step to cause issue. > 1. Boot linux. > 2. Start DPDK application without any physical NIC ports. > - Here, some sysfs values are read, and store to pci device list. > 3. igb_uio starts managing a port. > - Here, some sysfs values are changed. > 4. Add a NIC port to DPDK application using hotplug functions. > - Here, we need to replace some values. I think that you are showing that something is wrongly designed in these EAL structures. I suggest to try cleaning this mess instead of workarounding. [...] > >> - if (memcmp(&uio_res->pci_addr, &dev->addr, sizeof(dev->addr))) > >> + if (eal_compare_pci_addr(&uio_res->pci_addr, &dev->addr)) > > Why memcmp is not sufficient to compare PCI addresses? > > The only exception I see is endianness for natural sorting. > > Here is the definition. > > struct rte_pci_addr { > uint16_t domain; /**< Device domain */ > uint8_t bus; /**< Device bus */ > uint8_t devid; /**< Device ID */ > uint8_t function; /**< Device function. */ > }; > > But, sizeof(struct rte_pci_addr) will be 6. > If rte_pci_addr is allocated in a function without bzero, last 1 byte > may have some value. > As a result, memcmp may not work. To avoid such a case, I checked like > above. OK thanks. That's the kind of information which is valuable in a commit log.
2015-02-17 15:15, Tetsuya Mukawa:
> On 2015/02/17 10:11, Thomas Monjalon wrote:
> > 2015-02-16 13:14, Tetsuya Mukawa:
> >> +#ifdef ENABLE_HOTPLUG
> > Please avoid using #ifdef if not really necessary.
>
> I agree with you.
> In this case, only hotplug functions call pci_unmap_resource().
> So this will be needed.
Why is it needed?
2015-02-17 19:26, Tetsuya Mukawa:
> On 2015/02/17 18:23, Thomas Monjalon wrote:
> > 2015-02-17 17:51, Tetsuya Mukawa:
> >> On 2015/02/17 10:48, Thomas Monjalon wrote:
> >>> 2015-02-16 13:14, Tetsuya Mukawa:
> >>>> +/* attach the new physical device, then store port_id of the device */
> >>>> +static int
> >>>> +rte_eal_dev_attach_pdev(struct rte_pci_addr *addr, uint8_t *port_id)
> >>>> +{
> >>>> + uint8_t new_port_id;
> >>>> + struct rte_eth_dev devs[RTE_MAX_ETHPORTS];
> >>>> +
> >>>> + if ((addr == NULL) || (port_id == NULL))
> >>>> + goto err;
> >>>> +
> >>>> + /* save current port status */
> >>>> + if (rte_eth_dev_save(devs, sizeof(devs)))
> >>>> + goto err;
> >>>> + /* re-construct pci_device_list */
> >>>> + if (rte_eal_pci_scan())
> >>>> + goto err;
> >>>> + /* invoke probe func of the driver can handle the new device */
> >>>> + if (rte_eal_pci_probe_one(addr))
> >>>> + goto err;
> >>> You should get the port_id from the previous function instead of searching it.
> >> I agree this will beautify this code, but actually to do like above
> >> changes current DPDK code much more, and it will not be clear, and not
> >> beautiful.
> >>
> >> Could I explain it more.
> >> Problem is initialization sequence of virtual device and physical device
> >> are completely different.
> >>
> >> (1) Attaching a physical device case
> >> - To return port id, pci_invoke_all_drivers() needs to return port id.
> >> - It means "devinit" of "struct rte_pci_driver" needs to return port_id.
> >> - "devinit" will be rte_eth_dev_init(). But if the device is virtual,
> >> this function is not implemented.
> >>
> >> (2) Attaching virtual device case
> >> - To return port id from rte_eal_pci_probe_one(),
> >> pci_invoke_all_drivers() needs to return port id.
> >> - It means "init" of "struct rte_driver" needs to return port_id.
> >> - "init" will be implemented in PMD. But this function has different
> >> usage in physical device and virtual device.
> >> - Especially, In the case of physical device, "init" doesn't allocate
> >> eth device, so cannot return port id.
> >>
> >> As a result, to remove rte_eth_dev_save() and
> >> rte_eth_dev_get_changed_port(), below different functions needs to
> >> return port id.
> >> - "devinit" of "struct rte_pci_driver".
> >> - "init" of "struct rte_driver".
> > Yes, exactly,
> > I think you shouldn't hesitate to improve existing EAL code.
> > And I also think we should try to remove differences between virtual and
> > pci devices.
>
> I strongly agree with it. But I haven't investigated how to remove it so
> far.
> To be honest, I want to submit hotplug patches to DPDK-2.0.
> Is above functionality needed to merge the hotplug patches?
> I guess I will not be able to remove it by 23rd.
Obviously, it would be better to have it in dpdk-2.0.0-rc1.
If not possible to fix it, would it be possible to work on other comments
and keep this cleanup for post-rc1 integration?
I feel this cleanup is important to get the right design but it wouldn't be
fair to block this (old) patchset for this reason.
On 2015/02/18 1:15, Maxime Leroy wrote: > Hi Tetsuya, > > On Tue, Feb 17, 2015 at 9:51 AM, Tetsuya Mukawa <mukawa@igel.co.jp> wrote: >> >>>> + /* get port_id enabled by above procedures */ >>>> + if (rte_eth_dev_get_changed_port(devs, &new_port_id)) >>>> + goto err2; >>> [...] >>> >>>> /** >>>> + * Uninitilization function called for each device driver once. >>>> + */ >>>> +typedef int (rte_dev_uninit_t)(const char *name, const char *args); >>> Why do you need args for uninit? >>> >> I just added for the case that finalization code of PMD needs it. >> But, probably "args" parameter can be removed. >> >> > I think there are no needs to have any args in the uninit function: > 1) You librte_pmd_null doesn't use it > 2) You give exactly the same argument that was used by the init > function. A driver should have already stored these parameters in an > internal private structure at initialization. So it's not needed to > give me back for uninit method. > > From my understanding devargs_list is only needed at the init to store > the arguments when we parse the command line. Then, at initialization, > rte_eal_dev_init creates the devices from this list . > > By removing args from uninit function, you doesn't need to add and > remove anymore devargs in devargs_list to (de)attach a new device. > > What do you think ? Hi Maxime, Yes, I agree. We can remove the argument. I will do it. Regards, Tetsuya > > Maxime
On 2015/02/18 9:31, Thomas Monjalon wrote:
> 2015-02-17 15:14, Tetsuya Mukawa:
>> On 2015/02/17 9:36, Thomas Monjalon wrote:
>>> 2015-02-16 13:14, Tetsuya Mukawa:
>>> Is uint8_t sill a good size for hotpluggable virtual device ids?
>> I am not sure it's enough, but uint8_t is widely used in "rte_ethdev.c"
>> as port id.
>> If someone reports it doesn't enough, I guess it will be the time to
>> write a patch to change all uint_8 in one patch.
> It's a big ABI breakage. So if we feel it's going to be required,
> it's better to do it now in 2.0 release I think.
>
> Any opinion?
>
Hi Thomas,
I agree with it.
I will add an one more patch to change uint8_t to uint16_t.
Thanks,
Tetsuya
On 2015/02/18 10:02, Thomas Monjalon wrote:
> 2015-02-17 15:14, Tetsuya Mukawa:
>> On 2015/02/17 9:44, Thomas Monjalon wrote:
>>> 2015-02-16 13:14, Tetsuya Mukawa:
>>>> @@ -356,13 +342,24 @@ pci_scan_one(int dev_pci_fd, struct pci_conf *conf)
>>>> }
>>>> else {
>>>> struct rte_pci_device *dev2 = NULL;
>>>> + int ret;
>>>>
>>>> TAILQ_FOREACH(dev2, &pci_device_list, next) {
>>>> - if (pci_addr_comparison(&dev->addr, &dev2->addr))
>>>> + ret = eal_compare_pci_addr(&dev->addr, &dev2->addr);
>>>> + if (ret > 0)
>>>> continue;
>>>> - else {
>>>> + else if (ret < 0) {
>>>> TAILQ_INSERT_BEFORE(dev2, dev, next);
>>>> return 0;
>>>> + } else { /* already registered */
>>>> + /* update pt_driver */
>>>> + dev2->pt_driver = dev->pt_driver;
>>>> + dev2->max_vfs = dev->max_vfs;
>>>> + memmove(dev2->mem_resource,
>>>> + dev->mem_resource,
>>>> + sizeof(dev->mem_resource));
>>>> + free(dev);
>>>> + return 0;
>>> Could you comment this "else part" please?
>> PCI device list is allocated when rte_eal_init() is called. At the time,
>> to fill pci device information, sysfs value is used.
>> If sysfs values written by kernel device driver will not be changed by
>> igb_uio, vfio or pci_uio_genereic, above code isn't needed.
>> But actually above values are changed or added by them.
>>
>> Here is a step to cause issue.
>> 1. Boot linux.
>> 2. Start DPDK application without any physical NIC ports.
>> - Here, some sysfs values are read, and store to pci device list.
>> 3. igb_uio starts managing a port.
>> - Here, some sysfs values are changed.
>> 4. Add a NIC port to DPDK application using hotplug functions.
>> - Here, we need to replace some values.
> I think that you are showing that something is wrongly designed in these
> EAL structures. I suggest to try cleaning this mess instead of workarounding.
>
> [...]
>>>> - if (memcmp(&uio_res->pci_addr, &dev->addr, sizeof(dev->addr)))
>>>> + if (eal_compare_pci_addr(&uio_res->pci_addr, &dev->addr))
>>> Why memcmp is not sufficient to compare PCI addresses?
>>> The only exception I see is endianness for natural sorting.
>> Here is the definition.
>>
>> struct rte_pci_addr {
>> uint16_t domain; /**< Device domain */
>> uint8_t bus; /**< Device bus */
>> uint8_t devid; /**< Device ID */
>> uint8_t function; /**< Device function. */
>> };
>>
>> But, sizeof(struct rte_pci_addr) will be 6.
>> If rte_pci_addr is allocated in a function without bzero, last 1 byte
>> may have some value.
>> As a result, memcmp may not work. To avoid such a case, I checked like
>> above.
> OK thanks. That's the kind of information which is valuable in a commit log.
>
Sure I will add it.
Thanks,
Tetsuya
On 2015/02/18 10:17, Thomas Monjalon wrote:
> 2015-02-17 19:26, Tetsuya Mukawa:
>> On 2015/02/17 18:23, Thomas Monjalon wrote:
>>> 2015-02-17 17:51, Tetsuya Mukawa:
>>>> On 2015/02/17 10:48, Thomas Monjalon wrote:
>>>>> 2015-02-16 13:14, Tetsuya Mukawa:
>>>>>> +/* attach the new physical device, then store port_id of the device */
>>>>>> +static int
>>>>>> +rte_eal_dev_attach_pdev(struct rte_pci_addr *addr, uint8_t *port_id)
>>>>>> +{
>>>>>> + uint8_t new_port_id;
>>>>>> + struct rte_eth_dev devs[RTE_MAX_ETHPORTS];
>>>>>> +
>>>>>> + if ((addr == NULL) || (port_id == NULL))
>>>>>> + goto err;
>>>>>> +
>>>>>> + /* save current port status */
>>>>>> + if (rte_eth_dev_save(devs, sizeof(devs)))
>>>>>> + goto err;
>>>>>> + /* re-construct pci_device_list */
>>>>>> + if (rte_eal_pci_scan())
>>>>>> + goto err;
>>>>>> + /* invoke probe func of the driver can handle the new device */
>>>>>> + if (rte_eal_pci_probe_one(addr))
>>>>>> + goto err;
>>>>> You should get the port_id from the previous function instead of searching it.
>>>> I agree this will beautify this code, but actually to do like above
>>>> changes current DPDK code much more, and it will not be clear, and not
>>>> beautiful.
>>>>
>>>> Could I explain it more.
>>>> Problem is initialization sequence of virtual device and physical device
>>>> are completely different.
>>>>
>>>> (1) Attaching a physical device case
>>>> - To return port id, pci_invoke_all_drivers() needs to return port id.
>>>> - It means "devinit" of "struct rte_pci_driver" needs to return port_id.
>>>> - "devinit" will be rte_eth_dev_init(). But if the device is virtual,
>>>> this function is not implemented.
>>>>
>>>> (2) Attaching virtual device case
>>>> - To return port id from rte_eal_pci_probe_one(),
>>>> pci_invoke_all_drivers() needs to return port id.
>>>> - It means "init" of "struct rte_driver" needs to return port_id.
>>>> - "init" will be implemented in PMD. But this function has different
>>>> usage in physical device and virtual device.
>>>> - Especially, In the case of physical device, "init" doesn't allocate
>>>> eth device, so cannot return port id.
>>>>
>>>> As a result, to remove rte_eth_dev_save() and
>>>> rte_eth_dev_get_changed_port(), below different functions needs to
>>>> return port id.
>>>> - "devinit" of "struct rte_pci_driver".
>>>> - "init" of "struct rte_driver".
>>> Yes, exactly,
>>> I think you shouldn't hesitate to improve existing EAL code.
>>> And I also think we should try to remove differences between virtual and
>>> pci devices.
>> I strongly agree with it. But I haven't investigated how to remove it so
>> far.
>> To be honest, I want to submit hotplug patches to DPDK-2.0.
>> Is above functionality needed to merge the hotplug patches?
>> I guess I will not be able to remove it by 23rd.
> Obviously, it would be better to have it in dpdk-2.0.0-rc1.
> If not possible to fix it, would it be possible to work on other comments
> and keep this cleanup for post-rc1 integration?
> I feel this cleanup is important to get the right design but it wouldn't be
> fair to block this (old) patchset for this reason.
>
I appreciate for your suggestion. I will keep working on it for post-rc1.
Thanks,
Tetsuya
[-- Attachment #1: Type: text/plain, Size: 1446 bytes --] On 2015/02/18 10:54, Tetsuya Mukawa wrote: > On 2015/02/18 9:31, Thomas Monjalon wrote: >> 2015-02-17 15:14, Tetsuya Mukawa: >>> On 2015/02/17 9:36, Thomas Monjalon wrote: >>>> 2015-02-16 13:14, Tetsuya Mukawa: >>>> Is uint8_t sill a good size for hotpluggable virtual device ids? >>> I am not sure it's enough, but uint8_t is widely used in "rte_ethdev.c" >>> as port id. >>> If someone reports it doesn't enough, I guess it will be the time to >>> write a patch to change all uint_8 in one patch. >> It's a big ABI breakage. So if we feel it's going to be required, >> it's better to do it now in 2.0 release I think. >> >> Any opinion? >> > Hi Thomas, > > I agree with it. > I will add an one more patch to change uint8_t to uint16_t. > > Thanks, > Tetsuya > Hi Thomas, Could I make sure. After changing uint8_t to uint16_t in "rte_ethdev.[ch]", must I also need to change other applications and libraries that call ethdev APIs? If so, I would not finish it by 23rd. I've counted how many lines call ethdev APIs that are related to port_id. Could you please check an attached file? It's over 1200 lines. Probably to fix one of caller, I will need to check how port_id is used, and fix more related lines. So probably thousands lines may need to be fixed. When is deadline for fixing this changing? Also, if you have a good idea to fix it easier, could you please let me know? Thanks, Tetsuya [-- Attachment #2: caller.txt --] [-- Type: text/plain, Size: 72070 bytes --] rte_eth_dev_configure app/test-pipeline/init.c 240 rte_eth_dev_configure app/test-pmd/testpmd.c 1304 rte_eth_dev_configure app/test/test_kni.c 523 rte_eth_dev_configure app/test/test_link_bonding.c 238 rte_eth_dev_configure app/test/test_link_bonding.c 240 rte_eth_dev_configure app/test/test_pmd_perf.c 751 rte_eth_dev_configure app/test/test_pmd_ring.c 67 rte_eth_dev_configure app/test/test_pmd_ring.c 73 rte_eth_dev_configure app/test/test_pmd_ring.c 77 rte_eth_dev_configure app/test/test_pmd_ring.c 81 rte_eth_dev_configure app/test/test_pmd_ring.c 256 rte_eth_dev_configure app/test/test_pmd_ring.c 257 rte_eth_dev_configure examples/distributor/main.c 125 rte_eth_dev_configure examples/dpdk_qat/main.c 726 rte_eth_dev_configure examples/exception_path/main.c 433 rte_eth_dev_configure examples/ip_fragmentation/main.c 890 rte_eth_dev_configure examples/ip_pipeline/init.c 486 rte_eth_dev_configure examples/ip_reassembly/main.c 1095 rte_eth_dev_configure examples/ipv4_multicast/main.c 755 rte_eth_dev_configure examples/kni/main.c 617 rte_eth_dev_configure examples/kni/main.c 725 rte_eth_dev_configure examples/l2fwd-ivshmem/host/host.c 745 rte_eth_dev_configure examples/l2fwd/main.c 650 rte_eth_dev_configure examples/l3fwd-acl/main.c 1991 rte_eth_dev_configure examples/l3fwd-power/main.c 1534 rte_eth_dev_configure examples/l3fwd-vf/main.c 1013 rte_eth_dev_configure examples/l3fwd/main.c 2457 rte_eth_dev_configure examples/link_status_interrupt/main.c 696 rte_eth_dev_configure examples/link_status_interrupt/main.c 702 rte_eth_dev_configure examples/load_balancer/init.c 446 rte_eth_dev_configure examples/multi_process/client_server_mp/mp_server/init.c 144 rte_eth_dev_configure examples/multi_process/l2fwd_fork/main.c 1121 rte_eth_dev_configure examples/multi_process/symmetric_mp/main.c 248 rte_eth_dev_configure examples/netmap_compat/lib/compat_netmap.c 706 rte_eth_dev_configure examples/qos_meter/main.c 370 rte_eth_dev_configure examples/qos_meter/main.c 386 rte_eth_dev_configure examples/qos_sched/init.c 129 rte_eth_dev_configure examples/quota_watermark/qw/init.c 82 rte_eth_dev_configure examples/skeleton/basicfwd.c 69 rte_eth_dev_configure examples/vhost/main.c 442 rte_eth_dev_configure examples/vhost_xen/main.c 306 rte_eth_dev_configure examples/vmdq/main.c 254 rte_eth_dev_configure examples/vmdq_dcb/main.c 177 rte_eth_dev_configure lib/librte_ether/rte_ethdev.c 728 rte_eth_dev_configure lib/librte_ether/rte_ethdev.h 91 rte_eth_dev_configure lib/librte_ether/rte_ethdev.h 102 rte_eth_dev_configure lib/librte_ether/rte_ethdev.h 1726 rte_eth_dev_configure lib/librte_ether/rte_ethdev.h 1744 rte_eth_dev_configure lib/librte_ether/rte_ethdev.h 1783 rte_eth_dev_configure lib/librte_ether/rte_ethdev.h 1844 rte_eth_dev_configure lib/librte_ether/rte_ethdev.h 1860 rte_eth_dev_configure lib/librte_ether/rte_ethdev.h 1877 rte_eth_dev_configure lib/librte_ether/rte_ethdev.h 1893 rte_eth_dev_configure lib/librte_ether/rte_ethdev.h 2112 rte_eth_dev_configure lib/librte_ether/rte_ethdev.h 2133 rte_eth_dev_configure lib/librte_ether/rte_ethdev.h 2225 rte_eth_dev_configure lib/librte_ether/rte_ethdev.h 2377 rte_eth_dev_configure lib/librte_ether/rte_ethdev.h 2492 rte_eth_dev_configure lib/librte_ether/rte_ethdev.h 2504 rte_eth_dev_configure lib/librte_pmd_bond/rte_eth_bond_pmd.c 950 rte_eth_rx_queue_setup app/test-pipeline/init.c 251 rte_eth_rx_queue_setup app/test-pmd/testpmd.c 1359 rte_eth_rx_queue_setup app/test-pmd/testpmd.c 1364 rte_eth_rx_queue_setup app/test/test_kni.c 529 rte_eth_rx_queue_setup app/test/test_link_bonding.c 243 rte_eth_rx_queue_setup app/test/test_link_bonding.c 246 rte_eth_rx_queue_setup app/test/test_pmd_perf.c 772 rte_eth_rx_queue_setup app/test/test_pmd_perf.c 776 rte_eth_rx_queue_setup app/test/test_pmd_ring.c 90 rte_eth_rx_queue_setup app/test/test_pmd_ring.c 99 rte_eth_rx_queue_setup app/test/test_pmd_ring.c 268 rte_eth_rx_queue_setup app/test/test_pmd_ring.c 269 rte_eth_rx_queue_setup examples/distributor/main.c 130 rte_eth_rx_queue_setup examples/dpdk_qat/main.c 768 rte_eth_rx_queue_setup examples/dpdk_qat/main.c 773 rte_eth_rx_queue_setup examples/exception_path/main.c 438 rte_eth_rx_queue_setup examples/ip_fragmentation/main.c 900 rte_eth_rx_queue_setup examples/ip_fragmentation/main.c 905 rte_eth_rx_queue_setup examples/ip_pipeline/init.c 496 rte_eth_rx_queue_setup examples/ip_reassembly/main.c 1105 rte_eth_rx_queue_setup examples/ip_reassembly/main.c 1110 rte_eth_rx_queue_setup examples/ipv4_multicast/main.c 769 rte_eth_rx_queue_setup examples/kni/main.c 622 rte_eth_rx_queue_setup examples/l2fwd-ivshmem/host/host.c 754 rte_eth_rx_queue_setup examples/l2fwd-ivshmem/host/host.c 759 rte_eth_rx_queue_setup examples/l2fwd/main.c 659 rte_eth_rx_queue_setup examples/l2fwd/main.c 664 rte_eth_rx_queue_setup examples/l3fwd-acl/main.c 2060 rte_eth_rx_queue_setup examples/l3fwd-acl/main.c 2065 rte_eth_rx_queue_setup examples/l3fwd-power/main.c 1616 rte_eth_rx_queue_setup examples/l3fwd-power/main.c 1621 rte_eth_rx_queue_setup examples/l3fwd-vf/main.c 1066 rte_eth_rx_queue_setup examples/l3fwd-vf/main.c 1070 rte_eth_rx_queue_setup examples/l3fwd/main.c 2530 rte_eth_rx_queue_setup examples/l3fwd/main.c 2535 rte_eth_rx_queue_setup examples/link_status_interrupt/main.c 714 rte_eth_rx_queue_setup examples/link_status_interrupt/main.c 719 rte_eth_rx_queue_setup examples/load_balancer/init.c 469 rte_eth_rx_queue_setup examples/multi_process/client_server_mp/mp_server/init.c 149 rte_eth_rx_queue_setup examples/multi_process/l2fwd_fork/main.c 1130 rte_eth_rx_queue_setup examples/multi_process/l2fwd_fork/main.c 1135 rte_eth_rx_queue_setup examples/multi_process/symmetric_mp/main.c 253 rte_eth_rx_queue_setup examples/netmap_compat/lib/compat_netmap.c 726 rte_eth_rx_queue_setup examples/qos_meter/main.c 374 rte_eth_rx_queue_setup examples/qos_meter/main.c 390 rte_eth_rx_queue_setup examples/qos_sched/init.c 136 rte_eth_rx_queue_setup examples/quota_watermark/qw/init.c 88 rte_eth_rx_queue_setup examples/skeleton/basicfwd.c 74 rte_eth_rx_queue_setup examples/vhost/main.c 448 rte_eth_rx_queue_setup examples/vhost_xen/main.c 315 rte_eth_rx_queue_setup examples/vmdq/main.c 262 rte_eth_rx_queue_setup examples/vmdq_dcb/main.c 182 rte_eth_rx_queue_setup lib/librte_ether/rte_ethdev.c 1043 rte_eth_rx_queue_setup lib/librte_ether/rte_ethdev.h 93 rte_eth_rx_queue_setup lib/librte_ether/rte_ethdev.h 103 rte_eth_rx_queue_setup lib/librte_ether/rte_ethdev.h 1770 rte_eth_rx_queue_setup lib/librte_pmd_bond/rte_eth_bond_pmd.c 964 rte_eth_rx_queue_setup lib/librte_pmd_bond/rte_eth_bond_pmd.c 970 rte_eth_tx_queue_setup app/test-pipeline/init.c 263 rte_eth_tx_queue_setup app/test-pmd/testpmd.c 1323 rte_eth_tx_queue_setup app/test-pmd/testpmd.c 1327 rte_eth_tx_queue_setup app/test/test_kni.c 535 rte_eth_tx_queue_setup app/test/test_link_bonding.c 249 rte_eth_tx_queue_setup app/test/test_link_bonding.c 251 rte_eth_tx_queue_setup app/test/test_pmd_perf.c 764 rte_eth_tx_queue_setup app/test/test_pmd_perf.c 768 rte_eth_tx_queue_setup app/test/test_pmd_ring.c 86 rte_eth_tx_queue_setup app/test/test_pmd_ring.c 95 rte_eth_tx_queue_setup app/test/test_pmd_ring.c 262 rte_eth_tx_queue_setup app/test/test_pmd_ring.c 263 rte_eth_tx_queue_setup examples/distributor/main.c 138 rte_eth_tx_queue_setup examples/dpdk_qat/main.c 742 rte_eth_tx_queue_setup examples/dpdk_qat/main.c 746 rte_eth_tx_queue_setup examples/exception_path/main.c 445 rte_eth_tx_queue_setup examples/ip_fragmentation/main.c 927 rte_eth_tx_queue_setup examples/ip_fragmentation/main.c 931 rte_eth_tx_queue_setup examples/ip_pipeline/init.c 508 rte_eth_tx_queue_setup examples/ip_reassembly/main.c 1134 rte_eth_tx_queue_setup examples/ip_reassembly/main.c 1137 rte_eth_tx_queue_setup examples/ipv4_multicast/main.c 774 rte_eth_tx_queue_setup examples/ipv4_multicast/main.c 789 rte_eth_tx_queue_setup examples/ipv4_multicast/main.c 792 rte_eth_tx_queue_setup examples/kni/main.c 628 rte_eth_tx_queue_setup examples/l2fwd-ivshmem/host/host.c 764 rte_eth_tx_queue_setup examples/l2fwd-ivshmem/host/host.c 768 rte_eth_tx_queue_setup examples/l2fwd/main.c 669 rte_eth_tx_queue_setup examples/l2fwd/main.c 673 rte_eth_tx_queue_setup examples/l3fwd-acl/main.c 2026 rte_eth_tx_queue_setup examples/l3fwd-acl/main.c 2030 rte_eth_tx_queue_setup examples/l3fwd-power/main.c 1568 rte_eth_tx_queue_setup examples/l3fwd-power/main.c 1572 rte_eth_tx_queue_setup examples/l3fwd-vf/main.c 1036 rte_eth_tx_queue_setup examples/l3fwd-vf/main.c 1039 rte_eth_tx_queue_setup examples/l3fwd/main.c 2498 rte_eth_tx_queue_setup examples/l3fwd/main.c 2501 rte_eth_tx_queue_setup examples/link_status_interrupt/main.c 724 rte_eth_tx_queue_setup examples/link_status_interrupt/main.c 728 rte_eth_tx_queue_setup examples/load_balancer/init.c 490 rte_eth_tx_queue_setup examples/multi_process/client_server_mp/mp_server/init.c 156 rte_eth_tx_queue_setup examples/multi_process/l2fwd_fork/main.c 1140 rte_eth_tx_queue_setup examples/multi_process/l2fwd_fork/main.c 1144 rte_eth_tx_queue_setup examples/multi_process/symmetric_mp/main.c 262 rte_eth_tx_queue_setup examples/netmap_compat/lib/compat_netmap.c 715 rte_eth_tx_queue_setup examples/qos_meter/main.c 380 rte_eth_tx_queue_setup examples/qos_meter/main.c 396 rte_eth_tx_queue_setup examples/qos_sched/init.c 139 rte_eth_tx_queue_setup examples/qos_sched/init.c 144 rte_eth_tx_queue_setup examples/qos_sched/init.c 147 rte_eth_tx_queue_setup examples/quota_watermark/qw/init.c 97 rte_eth_tx_queue_setup examples/skeleton/basicfwd.c 81 rte_eth_tx_queue_setup examples/vhost/main.c 456 rte_eth_tx_queue_setup examples/vhost_xen/main.c 322 rte_eth_tx_queue_setup examples/vmdq/main.c 273 rte_eth_tx_queue_setup examples/vmdq_dcb/main.c 191 rte_eth_tx_queue_setup lib/librte_ether/rte_ethdev.c 1121 rte_eth_tx_queue_setup lib/librte_ether/rte_ethdev.h 92 rte_eth_tx_queue_setup lib/librte_ether/rte_ethdev.h 102 rte_eth_tx_queue_setup lib/librte_ether/rte_ethdev.h 1818 rte_eth_tx_queue_setup lib/librte_pmd_bond/rte_eth_bond_pmd.c 980 rte_eth_tx_queue_setup lib/librte_pmd_bond/rte_eth_bond_pmd.c 986 rte_eth_dev_socket_id app/test-pipeline/init.c 255 rte_eth_dev_socket_id app/test-pipeline/init.c 267 rte_eth_dev_socket_id app/test-pmd/testpmd.c 571 rte_eth_dev_socket_id app/test-pmd/testpmd.c 672 rte_eth_dev_socket_id app/test/test_link_bonding.c 244 rte_eth_dev_socket_id app/test/test_link_bonding.c 250 rte_eth_dev_socket_id app/test/test_pmd_perf.c 735 rte_eth_dev_socket_id app/test/test_pmd_perf.c 745 rte_eth_dev_socket_id app/test/test_pmd_perf.c 826 rte_eth_dev_socket_id examples/distributor/main.c 131 rte_eth_dev_socket_id examples/distributor/main.c 139 rte_eth_dev_socket_id examples/distributor/main.c 212 rte_eth_dev_socket_id examples/distributor/main.c 213 rte_eth_dev_socket_id examples/distributor/main.c 312 rte_eth_dev_socket_id examples/distributor/main.c 313 rte_eth_dev_socket_id examples/exception_path/main.c 438 rte_eth_dev_socket_id examples/exception_path/main.c 445 rte_eth_dev_socket_id examples/ip_pipeline/init.c 500 rte_eth_dev_socket_id examples/ip_pipeline/init.c 512 rte_eth_dev_socket_id examples/ipv4_multicast/main.c 770 rte_eth_dev_socket_id examples/kni/main.c 623 rte_eth_dev_socket_id examples/kni/main.c 629 rte_eth_dev_socket_id examples/l2fwd-ivshmem/host/host.c 755 rte_eth_dev_socket_id examples/l2fwd-ivshmem/host/host.c 765 rte_eth_dev_socket_id examples/l2fwd/main.c 660 rte_eth_dev_socket_id examples/l2fwd/main.c 670 rte_eth_dev_socket_id examples/link_status_interrupt/main.c 715 rte_eth_dev_socket_id examples/link_status_interrupt/main.c 725 rte_eth_dev_socket_id examples/multi_process/client_server_mp/mp_server/init.c 150 rte_eth_dev_socket_id examples/multi_process/client_server_mp/mp_server/init.c 157 rte_eth_dev_socket_id examples/multi_process/l2fwd_fork/main.c 1131 rte_eth_dev_socket_id examples/multi_process/l2fwd_fork/main.c 1141 rte_eth_dev_socket_id examples/multi_process/symmetric_mp/main.c 254 rte_eth_dev_socket_id examples/multi_process/symmetric_mp/main.c 263 rte_eth_dev_socket_id examples/qos_meter/main.c 375 rte_eth_dev_socket_id examples/qos_meter/main.c 381 rte_eth_dev_socket_id examples/qos_meter/main.c 391 rte_eth_dev_socket_id examples/qos_meter/main.c 397 rte_eth_dev_socket_id examples/qos_sched/init.c 137 rte_eth_dev_socket_id examples/qos_sched/init.c 145 rte_eth_dev_socket_id examples/qos_sched/init.c 343 rte_eth_dev_socket_id examples/quota_watermark/qw/init.c 89 rte_eth_dev_socket_id examples/quota_watermark/qw/init.c 98 rte_eth_dev_socket_id examples/skeleton/basicfwd.c 75 rte_eth_dev_socket_id examples/skeleton/basicfwd.c 82 rte_eth_dev_socket_id examples/skeleton/basicfwd.c 115 rte_eth_dev_socket_id examples/skeleton/basicfwd.c 116 rte_eth_dev_socket_id examples/vhost/main.c 449 rte_eth_dev_socket_id examples/vhost/main.c 457 rte_eth_dev_socket_id examples/vhost_xen/main.c 316 rte_eth_dev_socket_id examples/vhost_xen/main.c 323 rte_eth_dev_socket_id examples/vmdq/main.c 263 rte_eth_dev_socket_id examples/vmdq/main.c 274 rte_eth_dev_socket_id examples/vmdq_dcb/main.c 183 rte_eth_dev_socket_id examples/vmdq_dcb/main.c 192 rte_eth_dev_socket_id lib/librte_ether/rte_ethdev.c 345 rte_eth_dev_socket_id lib/librte_ether/rte_ethdev.h 1832 rte_eth_dev_socket_id lib/librte_pmd_bond/rte_eth_bond_pmd.c 966 rte_eth_dev_socket_id lib/librte_pmd_bond/rte_eth_bond_pmd.c 982 rte_eth_dev_rx_queue_start app/test-pmd/cmdline.c 1671 rte_eth_dev_rx_queue_start examples/vhost/main.c 2684 rte_eth_dev_rx_queue_start lib/librte_ether/rte_ethdev.c 397 rte_eth_dev_rx_queue_start lib/librte_ether/rte_ethdev.h 1850 rte_eth_dev_rx_queue_stop app/test-pmd/cmdline.c 1673 rte_eth_dev_rx_queue_stop examples/vhost/main.c 2377 rte_eth_dev_rx_queue_stop lib/librte_ether/rte_ethdev.c 423 rte_eth_dev_rx_queue_stop lib/librte_ether/rte_ethdev.h 1866 rte_eth_dev_tx_queue_start app/test-pmd/cmdline.c 1675 rte_eth_dev_tx_queue_start examples/vhost/main.c 2670 rte_eth_dev_tx_queue_start lib/librte_ether/rte_ethdev.c 449 rte_eth_dev_tx_queue_start lib/librte_ether/rte_ethdev.h 1883 rte_eth_dev_tx_queue_stop app/test-pmd/cmdline.c 1677 rte_eth_dev_tx_queue_stop examples/vhost/main.c 2393 rte_eth_dev_tx_queue_stop examples/vhost/main.c 2693 rte_eth_dev_tx_queue_stop lib/librte_ether/rte_ethdev.c 475 rte_eth_dev_tx_queue_stop lib/librte_ether/rte_ethdev.h 1899 rte_eth_dev_start app/test-pipeline/init.c 274 rte_eth_dev_start app/test-pmd/testpmd.c 1386 rte_eth_dev_start app/test/test_kni.c 541 rte_eth_dev_start app/test/test_link_bonding.c 254 rte_eth_dev_start app/test/test_link_bonding.c 255 rte_eth_dev_start app/test/test_link_bonding.c 619 rte_eth_dev_start app/test/test_link_bonding.c 808 rte_eth_dev_start app/test/test_link_bonding.c 1008 rte_eth_dev_start app/test/test_link_bonding.c 1049 rte_eth_dev_start app/test/test_link_bonding.c 1140 rte_eth_dev_start app/test/test_link_bonding.c 1764 rte_eth_dev_start app/test/test_link_bonding.c 2362 rte_eth_dev_start app/test/test_link_bonding.c 3264 rte_eth_dev_start app/test/test_link_bonding.c 3843 rte_eth_dev_start app/test/test_link_bonding.c 4344 rte_eth_dev_start app/test/test_pmd_perf.c 781 rte_eth_dev_start app/test/test_pmd_perf.c 784 rte_eth_dev_start app/test/test_pmd_ring.c 105 rte_eth_dev_start app/test/test_pmd_ring.c 109 rte_eth_dev_start app/test/test_pmd_ring.c 113 rte_eth_dev_start app/test/test_pmd_ring.c 274 rte_eth_dev_start app/test/test_pmd_ring.c 275 rte_eth_dev_start examples/distributor/main.c 145 rte_eth_dev_start examples/dpdk_qat/main.c 785 rte_eth_dev_start examples/dpdk_qat/main.c 787 rte_eth_dev_start examples/exception_path/main.c 451 rte_eth_dev_start examples/ip_fragmentation/main.c 951 rte_eth_dev_start examples/ip_fragmentation/main.c 953 rte_eth_dev_start examples/ip_pipeline/init.c 519 rte_eth_dev_start examples/ip_reassembly/main.c 1156 rte_eth_dev_start examples/ip_reassembly/main.c 1158 rte_eth_dev_start examples/ipv4_multicast/main.c 801 rte_eth_dev_start examples/ipv4_multicast/main.c 803 rte_eth_dev_start examples/kni/main.c 634 rte_eth_dev_start examples/kni/main.c 732 rte_eth_dev_start examples/kni/main.c 757 rte_eth_dev_start examples/l2fwd-ivshmem/host/host.c 772 rte_eth_dev_start examples/l2fwd-ivshmem/host/host.c 774 rte_eth_dev_start examples/l2fwd/main.c 677 rte_eth_dev_start examples/l2fwd/main.c 679 rte_eth_dev_start examples/l3fwd-acl/main.c 2078 rte_eth_dev_start examples/l3fwd-acl/main.c 2081 rte_eth_dev_start examples/l3fwd-power/main.c 1634 rte_eth_dev_start examples/l3fwd-power/main.c 1636 rte_eth_dev_start examples/l3fwd-vf/main.c 1082 rte_eth_dev_start examples/l3fwd-vf/main.c 1084 rte_eth_dev_start examples/l3fwd/main.c 2548 rte_eth_dev_start examples/l3fwd/main.c 2550 rte_eth_dev_start examples/link_status_interrupt/main.c 732 rte_eth_dev_start examples/link_status_interrupt/main.c 734 rte_eth_dev_start examples/load_balancer/init.c 504 rte_eth_dev_start examples/multi_process/client_server_mp/mp_server/init.c 164 rte_eth_dev_start examples/multi_process/l2fwd_fork/main.c 450 rte_eth_dev_start examples/multi_process/l2fwd_fork/main.c 1148 rte_eth_dev_start examples/multi_process/l2fwd_fork/main.c 1150 rte_eth_dev_start examples/multi_process/symmetric_mp/main.c 271 rte_eth_dev_start examples/netmap_compat/lib/compat_netmap.c 371 rte_eth_dev_start examples/qos_meter/main.c 402 rte_eth_dev_start examples/qos_meter/main.c 406 rte_eth_dev_start examples/qos_sched/init.c 151 rte_eth_dev_start examples/quota_watermark/qw/init.c 111 rte_eth_dev_start examples/skeleton/basicfwd.c 87 rte_eth_dev_start examples/vhost/main.c 464 rte_eth_dev_start examples/vhost_xen/main.c 330 rte_eth_dev_start examples/vmdq/main.c 282 rte_eth_dev_start examples/vmdq_dcb/main.c 198 rte_eth_dev_start lib/librte_ether/rte_ethdev.c 916 rte_eth_dev_start lib/librte_ether/rte_ethdev.h 94 rte_eth_dev_start lib/librte_ether/rte_ethdev.h 104 rte_eth_dev_start lib/librte_ether/rte_ethdev.h 109 rte_eth_dev_start lib/librte_ether/rte_ethdev.h 121 rte_eth_dev_start lib/librte_ether/rte_ethdev.h 633 rte_eth_dev_start lib/librte_ether/rte_ethdev.h 654 rte_eth_dev_start lib/librte_ether/rte_ethdev.h 1918 rte_eth_dev_start lib/librte_ether/rte_ethdev.h 1922 rte_eth_dev_start lib/librte_pmd_bond/rte_eth_bond_pmd.c 993 rte_eth_dev_start lib/librte_pmd_bond/rte_eth_bond_pmd.c 995 rte_eth_dev_stop app/test-pmd/testpmd.c 1446 rte_eth_dev_stop app/test/test_kni.c 675 rte_eth_dev_stop app/test/test_link_bonding.c 666 rte_eth_dev_stop app/test/test_link_bonding.c 696 rte_eth_dev_stop app/test/test_link_bonding.c 806 rte_eth_dev_stop app/test/test_link_bonding.c 1048 rte_eth_dev_stop app/test/test_link_bonding.c 1079 rte_eth_dev_stop app/test/test_link_bonding.c 1762 rte_eth_dev_stop app/test/test_link_bonding.c 2360 rte_eth_dev_stop app/test/test_link_bonding.c 3262 rte_eth_dev_stop app/test/test_link_bonding.c 3841 rte_eth_dev_stop app/test/test_link_bonding.c 4342 rte_eth_dev_stop app/test/test_pmd_perf.c 829 rte_eth_dev_stop app/test/test_pmd_ring.c 400 rte_eth_dev_stop app/test/test_pmd_ring.c 401 rte_eth_dev_stop app/test/test_pmd_ring.c 436 rte_eth_dev_stop app/test/test_pmd_ring.c 437 rte_eth_dev_stop app/test/test_pmd_ring.c 438 rte_eth_dev_stop examples/kni/main.c 713 rte_eth_dev_stop examples/kni/main.c 756 rte_eth_dev_stop examples/kni/main.c 759 rte_eth_dev_stop examples/kni/main.c 839 rte_eth_dev_stop examples/multi_process/l2fwd_fork/main.c 440 rte_eth_dev_stop examples/netmap_compat/lib/compat_netmap.c 417 rte_eth_dev_stop examples/quota_watermark/qw/init.c 80 rte_eth_dev_stop lib/librte_ether/rte_ethdev.c 953 rte_eth_dev_stop lib/librte_ether/rte_ethdev.h 103 rte_eth_dev_stop lib/librte_ether/rte_ethdev.h 109 rte_eth_dev_stop lib/librte_ether/rte_ethdev.h 1927 rte_eth_dev_stop lib/librte_pmd_bond/rte_eth_bond_pmd.c 943 rte_eth_dev_set_link_up app/test-pmd/testpmd.c 1238 rte_eth_dev_set_link_up lib/librte_ether/rte_ethdev.c 982 rte_eth_dev_set_link_up lib/librte_ether/rte_ethdev.h 1942 rte_eth_dev_set_link_up lib/librte_ether/rte_ethdev.h 1948 rte_eth_dev_set_link_down app/test-pmd/testpmd.c 1245 rte_eth_dev_set_link_down lib/librte_ether/rte_ethdev.c 1002 rte_eth_dev_set_link_down lib/librte_ether/rte_ethdev.h 1953 rte_eth_dev_close app/test-pmd/testpmd.c 1483 rte_eth_dev_close app/test-pmd/testpmd.c 1528 rte_eth_dev_close app/test/test_link_bonding.c 4028 rte_eth_dev_close examples/l3fwd-vf/main.c 694 rte_eth_dev_close lib/librte_ether/rte_ethdev.c 1022 rte_eth_dev_close lib/librte_ether/rte_ethdev.h 124 rte_eth_dev_close lib/librte_ether/rte_ethdev.h 1961 rte_eth_promiscuous_enable app/test-pipeline/init.c 248 rte_eth_promiscuous_enable app/test-pmd/cmdline.c 4043 rte_eth_promiscuous_enable app/test-pmd/cmdline.c 4334 rte_eth_promiscuous_enable app/test-pmd/cmdline.c 4341 rte_eth_promiscuous_enable app/test-pmd/testpmd.c 1922 rte_eth_promiscuous_enable app/test/test_kni.c 546 rte_eth_promiscuous_enable app/test/test_link_bonding.c 1813 rte_eth_promiscuous_enable app/test/test_link_bonding.c 2261 rte_eth_promiscuous_enable app/test/test_link_bonding.c 3172 rte_eth_promiscuous_enable app/test/test_link_bonding.c 3767 rte_eth_promiscuous_enable app/test/test_link_bonding.c 4242 rte_eth_promiscuous_enable app/test/test_pmd_perf.c 788 rte_eth_promiscuous_enable examples/distributor/main.c 170 rte_eth_promiscuous_enable examples/dpdk_qat/main.c 808 rte_eth_promiscuous_enable examples/exception_path/main.c 455 rte_eth_promiscuous_enable examples/ip_fragmentation/main.c 956 rte_eth_promiscuous_enable examples/ip_pipeline/init.c 493 rte_eth_promiscuous_enable examples/ip_reassembly/main.c 1161 rte_eth_promiscuous_enable examples/kni/main.c 640 rte_eth_promiscuous_enable examples/l2fwd-ivshmem/host/host.c 779 rte_eth_promiscuous_enable examples/l2fwd/main.c 684 rte_eth_promiscuous_enable examples/l3fwd-acl/main.c 2091 rte_eth_promiscuous_enable examples/l3fwd-power/main.c 1646 rte_eth_promiscuous_enable examples/l3fwd/main.c 2560 rte_eth_promiscuous_enable examples/load_balancer/init.c 454 rte_eth_promiscuous_enable examples/multi_process/client_server_mp/mp_server/init.c 162 rte_eth_promiscuous_enable examples/multi_process/l2fwd_fork/main.c 1155 rte_eth_promiscuous_enable examples/multi_process/symmetric_mp/main.c 269 rte_eth_promiscuous_enable examples/netmap_compat/bridge/bridge.c 302 rte_eth_promiscuous_enable examples/qos_meter/main.c 410 rte_eth_promiscuous_enable examples/qos_meter/main.c 412 rte_eth_promiscuous_enable examples/qos_sched/init.c 168 rte_eth_promiscuous_enable examples/quota_watermark/qw/init.c 117 rte_eth_promiscuous_enable examples/skeleton/basicfwd.c 100 rte_eth_promiscuous_enable examples/vhost/main.c 471 rte_eth_promiscuous_enable lib/librte_ether/rte_ethdev.c 904 rte_eth_promiscuous_enable lib/librte_ether/rte_ethdev.c 1162 rte_eth_promiscuous_enable lib/librte_ether/rte_ethdev.h 1969 rte_eth_promiscuous_enable lib/librte_pmd_bond/rte_eth_bond_8023ad.c 873 rte_eth_promiscuous_enable lib/librte_pmd_bond/rte_eth_bond_pmd.c 1417 rte_eth_promiscuous_enable lib/librte_pmd_bond/rte_eth_bond_pmd.c 1426 rte_eth_promiscuous_disable app/test-pmd/cmdline.c 4336 rte_eth_promiscuous_disable app/test-pmd/cmdline.c 4343 rte_eth_promiscuous_disable app/test/test_link_bonding.c 1828 rte_eth_promiscuous_disable app/test/test_link_bonding.c 2282 rte_eth_promiscuous_disable app/test/test_link_bonding.c 3185 rte_eth_promiscuous_disable app/test/test_link_bonding.c 3781 rte_eth_promiscuous_disable app/test/test_link_bonding.c 4263 rte_eth_promiscuous_disable lib/librte_ether/rte_ethdev.c 906 rte_eth_promiscuous_disable lib/librte_ether/rte_ethdev.c 1179 rte_eth_promiscuous_disable lib/librte_ether/rte_ethdev.h 1977 rte_eth_promiscuous_disable lib/librte_pmd_bond/rte_eth_bond_pmd.c 1446 rte_eth_promiscuous_disable lib/librte_pmd_bond/rte_eth_bond_pmd.c 1455 rte_eth_promiscuous_get app/test-pmd/config.c 331 rte_eth_promiscuous_get app/test/test_link_bonding.c 1815 rte_eth_promiscuous_get app/test/test_link_bonding.c 1821 rte_eth_promiscuous_get app/test/test_link_bonding.c 1830 rte_eth_promiscuous_get app/test/test_link_bonding.c 1836 rte_eth_promiscuous_get app/test/test_link_bonding.c 2263 rte_eth_promiscuous_get app/test/test_link_bonding.c 2268 rte_eth_promiscuous_get app/test/test_link_bonding.c 2284 rte_eth_promiscuous_get app/test/test_link_bonding.c 2289 rte_eth_promiscuous_get app/test/test_link_bonding.c 3174 rte_eth_promiscuous_get app/test/test_link_bonding.c 3179 rte_eth_promiscuous_get app/test/test_link_bonding.c 3187 rte_eth_promiscuous_get app/test/test_link_bonding.c 3192 rte_eth_promiscuous_get app/test/test_link_bonding.c 3770 rte_eth_promiscuous_get app/test/test_link_bonding.c 3775 rte_eth_promiscuous_get app/test/test_link_bonding.c 3783 rte_eth_promiscuous_get app/test/test_link_bonding.c 3788 rte_eth_promiscuous_get app/test/test_link_bonding.c 4244 rte_eth_promiscuous_get app/test/test_link_bonding.c 4249 rte_eth_promiscuous_get app/test/test_link_bonding.c 4265 rte_eth_promiscuous_get app/test/test_link_bonding.c 4271 rte_eth_promiscuous_get lib/librte_ether/rte_ethdev.c 903 rte_eth_promiscuous_get lib/librte_ether/rte_ethdev.c 905 rte_eth_promiscuous_get lib/librte_ether/rte_ethdev.c 1196 rte_eth_promiscuous_get lib/librte_ether/rte_ethdev.h 1989 rte_eth_allmulticast_enable app/test-pmd/cmdline.c 4414 rte_eth_allmulticast_enable app/test-pmd/cmdline.c 4421 rte_eth_allmulticast_enable lib/librte_ether/rte_ethdev.c 910 rte_eth_allmulticast_enable lib/librte_ether/rte_ethdev.c 1210 rte_eth_allmulticast_enable lib/librte_ether/rte_ethdev.h 1997 rte_eth_allmulticast_disable app/test-pmd/cmdline.c 4416 rte_eth_allmulticast_disable app/test-pmd/cmdline.c 4423 rte_eth_allmulticast_disable lib/librte_ether/rte_ethdev.c 912 rte_eth_allmulticast_disable lib/librte_ether/rte_ethdev.c 1227 rte_eth_allmulticast_disable lib/librte_ether/rte_ethdev.h 2005 rte_eth_allmulticast_get app/test-pmd/config.c 333 rte_eth_allmulticast_get lib/librte_ether/rte_ethdev.c 909 rte_eth_allmulticast_get lib/librte_ether/rte_ethdev.c 911 rte_eth_allmulticast_get lib/librte_ether/rte_ethdev.c 1244 rte_eth_allmulticast_get lib/librte_ether/rte_ethdev.h 2017 rte_eth_link_get app/test-pipeline/init.c 212 rte_eth_link_get app/test-pmd/config.c 311 rte_eth_link_get app/test-pmd/config.c 2134 rte_eth_link_get app/test-pmd/config.c 2159 rte_eth_link_get app/test-pmd/testpmd.c 1559 rte_eth_link_get app/test/test_link_bonding.c 650 rte_eth_link_get app/test/test_link_bonding.c 668 rte_eth_link_get app/test/test_pmd_perf.c 179 rte_eth_link_get app/test/test_pmd_ring.c 118 rte_eth_link_get app/test/test_pmd_ring.c 119 rte_eth_link_get app/test/test_pmd_ring.c 120 rte_eth_link_get examples/distributor/main.c 150 rte_eth_link_get examples/distributor/main.c 153 rte_eth_link_get examples/dpdk_qat/main.c 793 rte_eth_link_get examples/exception_path/main.c 475 rte_eth_link_get examples/ip_fragmentation/main.c 623 rte_eth_link_get examples/ip_pipeline/init.c 458 rte_eth_link_get examples/ip_reassembly/main.c 752 rte_eth_link_get examples/ipv4_multicast/main.c 631 rte_eth_link_get examples/kni/main.c 660 rte_eth_link_get examples/l2fwd-ivshmem/host/host.c 361 rte_eth_link_get examples/l2fwd/main.c 501 rte_eth_link_get examples/l3fwd-acl/main.c 1890 rte_eth_link_get examples/l3fwd-power/main.c 1429 rte_eth_link_get examples/l3fwd/main.c 2360 rte_eth_link_get examples/link_status_interrupt/main.c 176 rte_eth_link_get examples/link_status_interrupt/main.c 528 rte_eth_link_get examples/link_status_interrupt/main.c 555 rte_eth_link_get examples/load_balancer/init.c 386 rte_eth_link_get examples/multi_process/client_server_mp/mp_server/init.c 220 rte_eth_link_get examples/multi_process/l2fwd_fork/main.c 924 rte_eth_link_get examples/multi_process/symmetric_mp/main.c 378 rte_eth_link_get examples/qos_sched/init.c 159 rte_eth_link_get examples/qos_sched/init.c 247 rte_eth_link_get lib/librte_ether/rte_ethdev.c 1272 rte_eth_link_get lib/librte_ether/rte_ethdev.c 1293 rte_eth_link_get lib/librte_ether/rte_ethdev.h 2030 rte_eth_link_get lib/librte_ether/rte_ethdev.h 2035 rte_eth_link_get lib/librte_ether/rte_ethdev.h 2043 rte_eth_link_get lib/librte_pmd_bond/rte_eth_bond_8023ad.c 758 rte_eth_link_get lib/librte_pmd_bond/rte_eth_bond_api.c 416 rte_eth_link_get lib/librte_pmd_bond/rte_eth_bond_pmd.c 425 rte_eth_link_get lib/librte_pmd_bond/rte_eth_bond_pmd.c 1511 rte_eth_link_get_nowait app/test-pipeline/init.c 212 rte_eth_link_get_nowait app/test-pmd/config.c 311 rte_eth_link_get_nowait app/test-pmd/config.c 2134 rte_eth_link_get_nowait app/test-pmd/config.c 2159 rte_eth_link_get_nowait app/test-pmd/testpmd.c 1559 rte_eth_link_get_nowait app/test/test_pmd_perf.c 179 rte_eth_link_get_nowait examples/distributor/main.c 150 rte_eth_link_get_nowait examples/distributor/main.c 153 rte_eth_link_get_nowait examples/exception_path/main.c 475 rte_eth_link_get_nowait examples/ip_fragmentation/main.c 623 rte_eth_link_get_nowait examples/ip_pipeline/init.c 458 rte_eth_link_get_nowait examples/ip_reassembly/main.c 752 rte_eth_link_get_nowait examples/ipv4_multicast/main.c 631 rte_eth_link_get_nowait examples/kni/main.c 660 rte_eth_link_get_nowait examples/l2fwd-ivshmem/host/host.c 361 rte_eth_link_get_nowait examples/l2fwd/main.c 501 rte_eth_link_get_nowait examples/l3fwd-acl/main.c 1890 rte_eth_link_get_nowait examples/l3fwd-power/main.c 1429 rte_eth_link_get_nowait examples/l3fwd/main.c 2360 rte_eth_link_get_nowait examples/link_status_interrupt/main.c 176 rte_eth_link_get_nowait examples/link_status_interrupt/main.c 528 rte_eth_link_get_nowait examples/link_status_interrupt/main.c 555 rte_eth_link_get_nowait examples/load_balancer/init.c 386 rte_eth_link_get_nowait examples/multi_process/client_server_mp/mp_server/init.c 220 rte_eth_link_get_nowait examples/multi_process/l2fwd_fork/main.c 924 rte_eth_link_get_nowait examples/multi_process/symmetric_mp/main.c 378 rte_eth_link_get_nowait lib/librte_ether/rte_ethdev.c 1293 rte_eth_link_get_nowait lib/librte_ether/rte_ethdev.h 2043 rte_eth_link_get_nowait lib/librte_pmd_bond/rte_eth_bond_api.c 416 rte_eth_link_get_nowait lib/librte_pmd_bond/rte_eth_bond_pmd.c 1511 rte_eth_stats_get app/test-pmd/config.c 134 rte_eth_stats_get app/test-pmd/testpmd.c 1052 rte_eth_stats_get app/test-pmd/testpmd.c 1182 rte_eth_stats_get app/test/test_link_bonding.c 1397 rte_eth_stats_get app/test/test_link_bonding.c 1405 rte_eth_stats_get app/test/test_link_bonding.c 1518 rte_eth_stats_get app/test/test_link_bonding.c 1532 rte_eth_stats_get app/test/test_link_bonding.c 1593 rte_eth_stats_get app/test/test_link_bonding.c 1604 rte_eth_stats_get app/test/test_link_bonding.c 1679 rte_eth_stats_get app/test/test_link_bonding.c 1687 rte_eth_stats_get app/test/test_link_bonding.c 1693 rte_eth_stats_get app/test/test_link_bonding.c 1699 rte_eth_stats_get app/test/test_link_bonding.c 1705 rte_eth_stats_get app/test/test_link_bonding.c 1917 rte_eth_stats_get app/test/test_link_bonding.c 1923 rte_eth_stats_get app/test/test_link_bonding.c 1928 rte_eth_stats_get app/test/test_link_bonding.c 1933 rte_eth_stats_get app/test/test_link_bonding.c 1938 rte_eth_stats_get app/test/test_link_bonding.c 1967 rte_eth_stats_get app/test/test_link_bonding.c 2119 rte_eth_stats_get app/test/test_link_bonding.c 2129 rte_eth_stats_get app/test/test_link_bonding.c 2199 rte_eth_stats_get app/test/test_link_bonding.c 2207 rte_eth_stats_get app/test/test_link_bonding.c 2222 rte_eth_stats_get app/test/test_link_bonding.c 2491 rte_eth_stats_get app/test/test_link_bonding.c 2496 rte_eth_stats_get app/test/test_link_bonding.c 2501 rte_eth_stats_get app/test/test_link_bonding.c 2506 rte_eth_stats_get app/test/test_link_bonding.c 2527 rte_eth_stats_get app/test/test_link_bonding.c 2532 rte_eth_stats_get app/test/test_link_bonding.c 2537 rte_eth_stats_get app/test/test_link_bonding.c 2542 rte_eth_stats_get app/test/test_link_bonding.c 2547 rte_eth_stats_get app/test/test_link_bonding.c 2671 rte_eth_stats_get app/test/test_link_bonding.c 2680 rte_eth_stats_get app/test/test_link_bonding.c 2686 rte_eth_stats_get app/test/test_link_bonding.c 2753 rte_eth_stats_get app/test/test_link_bonding.c 2760 rte_eth_stats_get app/test/test_link_bonding.c 2766 rte_eth_stats_get app/test/test_link_bonding.c 2866 rte_eth_stats_get app/test/test_link_bonding.c 2873 rte_eth_stats_get app/test/test_link_bonding.c 2879 rte_eth_stats_get app/test/test_link_bonding.c 3022 rte_eth_stats_get app/test/test_link_bonding.c 3036 rte_eth_stats_get app/test/test_link_bonding.c 3050 rte_eth_stats_get app/test/test_link_bonding.c 3115 rte_eth_stats_get app/test/test_link_bonding.c 3124 rte_eth_stats_get app/test/test_link_bonding.c 3130 rte_eth_stats_get app/test/test_link_bonding.c 3136 rte_eth_stats_get app/test/test_link_bonding.c 3142 rte_eth_stats_get app/test/test_link_bonding.c 3382 rte_eth_stats_get app/test/test_link_bonding.c 3388 rte_eth_stats_get app/test/test_link_bonding.c 3394 rte_eth_stats_get app/test/test_link_bonding.c 3418 rte_eth_stats_get app/test/test_link_bonding.c 3425 rte_eth_stats_get app/test/test_link_bonding.c 3455 rte_eth_stats_get app/test/test_link_bonding.c 3517 rte_eth_stats_get app/test/test_link_bonding.c 3526 rte_eth_stats_get app/test/test_link_bonding.c 3624 rte_eth_stats_get app/test/test_link_bonding.c 3635 rte_eth_stats_get app/test/test_link_bonding.c 3645 rte_eth_stats_get app/test/test_link_bonding.c 3710 rte_eth_stats_get app/test/test_link_bonding.c 3719 rte_eth_stats_get app/test/test_link_bonding.c 3725 rte_eth_stats_get app/test/test_link_bonding.c 3731 rte_eth_stats_get app/test/test_link_bonding.c 3737 rte_eth_stats_get app/test/test_link_bonding.c 3942 rte_eth_stats_get app/test/test_link_bonding.c 3948 rte_eth_stats_get app/test/test_link_bonding.c 3953 rte_eth_stats_get app/test/test_link_bonding.c 3958 rte_eth_stats_get app/test/test_link_bonding.c 3964 rte_eth_stats_get app/test/test_link_bonding.c 3986 rte_eth_stats_get app/test/test_link_bonding.c 4097 rte_eth_stats_get app/test/test_link_bonding.c 4110 rte_eth_stats_get app/test/test_link_bonding.c 4184 rte_eth_stats_get app/test/test_link_bonding.c 4192 rte_eth_stats_get app/test/test_link_bonding.c 4207 rte_eth_stats_get app/test/test_link_bonding.c 4475 rte_eth_stats_get app/test/test_link_bonding.c 4480 rte_eth_stats_get app/test/test_link_bonding.c 4485 rte_eth_stats_get app/test/test_link_bonding.c 4490 rte_eth_stats_get app/test/test_link_bonding.c 4515 rte_eth_stats_get app/test/test_pmd_perf.c 346 rte_eth_stats_get app/test/test_pmd_ring.c 165 rte_eth_stats_get app/test/test_pmd_ring.c 183 rte_eth_stats_get app/test/test_pmd_ring.c 204 rte_eth_stats_get app/test/test_pmd_ring.c 223 rte_eth_stats_get app/test/test_pmd_ring.c 234 rte_eth_stats_get app/test/test_pmd_ring.c 294 rte_eth_stats_get app/test/test_pmd_ring.c 295 rte_eth_stats_get app/test/test_pmd_ring.c 324 rte_eth_stats_get app/test/test_pmd_ring.c 325 rte_eth_stats_get app/test/test_pmd_ring.c 354 rte_eth_stats_get app/test/test_pmd_ring.c 355 rte_eth_stats_get app/test/test_pmd_ring.c 384 rte_eth_stats_get app/test/test_pmd_ring.c 385 rte_eth_stats_get examples/distributor/main.c 390 rte_eth_stats_get examples/load_balancer/runtime.c 213 rte_eth_stats_get examples/qos_sched/main.c 188 rte_eth_stats_get examples/qos_sched/main.c 197 rte_eth_stats_get lib/librte_ether/rte_ethdev.c 1314 rte_eth_stats_get lib/librte_ether/rte_ethdev.c 1380 rte_eth_stats_get lib/librte_ether/rte_ethdev.h 2061 rte_eth_stats_get lib/librte_pmd_bond/rte_eth_bond_pmd.c 454 rte_eth_stats_get lib/librte_pmd_bond/rte_eth_bond_pmd.c 1372 rte_eth_stats_reset app/test-pmd/config.c 208 rte_eth_stats_reset app/test/test_link_bonding.c 465 rte_eth_stats_reset app/test/test_link_bonding.c 697 rte_eth_stats_reset app/test/test_link_bonding.c 1169 rte_eth_stats_reset app/test/test_link_bonding.c 1619 rte_eth_stats_reset app/test/test_link_bonding.c 1622 rte_eth_stats_reset app/test/test_link_bonding.c 1910 rte_eth_stats_reset app/test/test_link_bonding.c 2239 rte_eth_stats_reset app/test/test_link_bonding.c 3929 rte_eth_stats_reset app/test/test_link_bonding.c 4220 rte_eth_stats_reset app/test/test_pmd_ring.c 201 rte_eth_stats_reset app/test/test_pmd_ring.c 231 rte_eth_stats_reset lib/librte_ether/rte_ethdev.c 1332 rte_eth_stats_reset lib/librte_ether/rte_ethdev.c 1442 rte_eth_stats_reset lib/librte_ether/rte_ethdev.h 2069 rte_eth_stats_reset lib/librte_pmd_bond/rte_eth_bond_pmd.c 1398 rte_eth_xstats_get app/test-pmd/config.c 220 rte_eth_xstats_get app/test-pmd/config.c 230 rte_eth_xstats_get lib/librte_ether/rte_ethdev.c 1349 rte_eth_xstats_get lib/librte_ether/rte_ethdev.h 2092 rte_eth_xstats_reset app/test-pmd/config.c 244 rte_eth_xstats_reset lib/librte_ether/rte_ethdev.c 1424 rte_eth_xstats_reset lib/librte_ether/rte_ethdev.h 2101 rte_eth_dev_set_tx_queue_stats_mapping app/test-pmd/testpmd.c 1605 rte_eth_dev_set_tx_queue_stats_mapping lib/librte_ether/rte_ethdev.c 1465 rte_eth_dev_set_tx_queue_stats_mapping lib/librte_ether/rte_ethdev.h 2120 rte_eth_dev_set_rx_queue_stats_mapping app/test-pmd/testpmd.c 1628 rte_eth_dev_set_rx_queue_stats_mapping lib/librte_ether/rte_ethdev.c 1474 rte_eth_dev_set_rx_queue_stats_mapping lib/librte_ether/rte_ethdev.h 2141 rte_eth_macaddr_get app/test-pmd/config.c 314 rte_eth_macaddr_get app/test-pmd/testpmd.c 1401 rte_eth_macaddr_get app/test-pmd/testpmd.c 1752 rte_eth_macaddr_get app/test-pmd/testpmd.c 1879 rte_eth_macaddr_get app/test/test_link_bonding.c 459 rte_eth_macaddr_get app/test/test_link_bonding.c 816 rte_eth_macaddr_get app/test/test_link_bonding.c 822 rte_eth_macaddr_get app/test/test_link_bonding.c 830 rte_eth_macaddr_get app/test/test_link_bonding.c 896 rte_eth_macaddr_get app/test/test_link_bonding.c 902 rte_eth_macaddr_get app/test/test_link_bonding.c 1017 rte_eth_macaddr_get app/test/test_link_bonding.c 1022 rte_eth_macaddr_get app/test/test_link_bonding.c 1028 rte_eth_macaddr_get app/test/test_link_bonding.c 1034 rte_eth_macaddr_get app/test/test_link_bonding.c 1053 rte_eth_macaddr_get app/test/test_link_bonding.c 1059 rte_eth_macaddr_get app/test/test_link_bonding.c 1065 rte_eth_macaddr_get app/test/test_link_bonding.c 1070 rte_eth_macaddr_get app/test/test_link_bonding.c 1096 rte_eth_macaddr_get app/test/test_link_bonding.c 1102 rte_eth_macaddr_get app/test/test_link_bonding.c 1108 rte_eth_macaddr_get app/test/test_link_bonding.c 1728 rte_eth_macaddr_get app/test/test_link_bonding.c 1729 rte_eth_macaddr_get app/test/test_link_bonding.c 1738 rte_eth_macaddr_get app/test/test_link_bonding.c 1752 rte_eth_macaddr_get app/test/test_link_bonding.c 1767 rte_eth_macaddr_get app/test/test_link_bonding.c 1774 rte_eth_macaddr_get app/test/test_link_bonding.c 1786 rte_eth_macaddr_get app/test/test_link_bonding.c 1793 rte_eth_macaddr_get app/test/test_link_bonding.c 2305 rte_eth_macaddr_get app/test/test_link_bonding.c 2306 rte_eth_macaddr_get app/test/test_link_bonding.c 2315 rte_eth_macaddr_get app/test/test_link_bonding.c 2321 rte_eth_macaddr_get app/test/test_link_bonding.c 2327 rte_eth_macaddr_get app/test/test_link_bonding.c 2339 rte_eth_macaddr_get app/test/test_link_bonding.c 2345 rte_eth_macaddr_get app/test/test_link_bonding.c 2351 rte_eth_macaddr_get app/test/test_link_bonding.c 2365 rte_eth_macaddr_get app/test/test_link_bonding.c 2371 rte_eth_macaddr_get app/test/test_link_bonding.c 2377 rte_eth_macaddr_get app/test/test_link_bonding.c 2388 rte_eth_macaddr_get app/test/test_link_bonding.c 2394 rte_eth_macaddr_get app/test/test_link_bonding.c 2400 rte_eth_macaddr_get app/test/test_link_bonding.c 3207 rte_eth_macaddr_get app/test/test_link_bonding.c 3208 rte_eth_macaddr_get app/test/test_link_bonding.c 3217 rte_eth_macaddr_get app/test/test_link_bonding.c 3223 rte_eth_macaddr_get app/test/test_link_bonding.c 3229 rte_eth_macaddr_get app/test/test_link_bonding.c 3241 rte_eth_macaddr_get app/test/test_link_bonding.c 3247 rte_eth_macaddr_get app/test/test_link_bonding.c 3253 rte_eth_macaddr_get app/test/test_link_bonding.c 3267 rte_eth_macaddr_get app/test/test_link_bonding.c 3273 rte_eth_macaddr_get app/test/test_link_bonding.c 3279 rte_eth_macaddr_get app/test/test_link_bonding.c 3290 rte_eth_macaddr_get app/test/test_link_bonding.c 3296 rte_eth_macaddr_get app/test/test_link_bonding.c 3302 rte_eth_macaddr_get app/test/test_link_bonding.c 3805 rte_eth_macaddr_get app/test/test_link_bonding.c 3806 rte_eth_macaddr_get app/test/test_link_bonding.c 3816 rte_eth_macaddr_get app/test/test_link_bonding.c 3830 rte_eth_macaddr_get app/test/test_link_bonding.c 3846 rte_eth_macaddr_get app/test/test_link_bonding.c 3853 rte_eth_macaddr_get app/test/test_link_bonding.c 3865 rte_eth_macaddr_get app/test/test_link_bonding.c 3873 rte_eth_macaddr_get app/test/test_link_bonding.c 4287 rte_eth_macaddr_get app/test/test_link_bonding.c 4288 rte_eth_macaddr_get app/test/test_link_bonding.c 4297 rte_eth_macaddr_get app/test/test_link_bonding.c 4303 rte_eth_macaddr_get app/test/test_link_bonding.c 4309 rte_eth_macaddr_get app/test/test_link_bonding.c 4321 rte_eth_macaddr_get app/test/test_link_bonding.c 4327 rte_eth_macaddr_get app/test/test_link_bonding.c 4333 rte_eth_macaddr_get app/test/test_link_bonding.c 4347 rte_eth_macaddr_get app/test/test_link_bonding.c 4353 rte_eth_macaddr_get app/test/test_link_bonding.c 4359 rte_eth_macaddr_get app/test/test_link_bonding.c 4371 rte_eth_macaddr_get app/test/test_link_bonding.c 4377 rte_eth_macaddr_get app/test/test_link_bonding.c 4383 rte_eth_macaddr_get app/test/test_pmd_perf.c 758 rte_eth_macaddr_get examples/distributor/main.c 162 rte_eth_macaddr_get examples/dpdk_qat/main.c 732 rte_eth_macaddr_get examples/ip_fragmentation/main.c 910 rte_eth_macaddr_get examples/ip_reassembly/main.c 1115 rte_eth_macaddr_get examples/ipv4_multicast/main.c 761 rte_eth_macaddr_get examples/l2fwd-ivshmem/host/host.c 750 rte_eth_macaddr_get examples/l2fwd-ivshmem/host/host.c 819 rte_eth_macaddr_get examples/l2fwd/main.c 655 rte_eth_macaddr_get examples/l3fwd-acl/main.c 1998 rte_eth_macaddr_get examples/l3fwd-power/main.c 1540 rte_eth_macaddr_get examples/l3fwd-vf/main.c 1018 rte_eth_macaddr_get examples/l3fwd/main.c 2463 rte_eth_macaddr_get examples/link_status_interrupt/main.c 709 rte_eth_macaddr_get examples/multi_process/client_server_mp/mp_server/main.c 104 rte_eth_macaddr_get examples/multi_process/l2fwd_fork/main.c 1126 rte_eth_macaddr_get examples/quota_watermark/qw/main.c 96 rte_eth_macaddr_get examples/skeleton/basicfwd.c 92 rte_eth_macaddr_get examples/vhost/main.c 473 rte_eth_macaddr_get examples/vhost_xen/main.c 334 rte_eth_macaddr_get examples/vmdq/main.c 288 rte_eth_macaddr_get examples/vmdq_dcb/main.c 203 rte_eth_macaddr_get lib/librte_ether/rte_ethdev.c 1504 rte_eth_macaddr_get lib/librte_ether/rte_ethdev.h 2154 rte_eth_macaddr_get lib/librte_pmd_bond/rte_eth_bond_8023ad.c 598 rte_eth_macaddr_get lib/librte_pmd_bond/rte_eth_bond_8023ad.c 759 rte_eth_macaddr_get lib/librte_pmd_bond/rte_eth_bond_8023ad.c 979 rte_eth_macaddr_get lib/librte_pmd_bond/rte_eth_bond_8023ad.c 1112 rte_eth_macaddr_get lib/librte_pmd_bond/rte_eth_bond_pmd.c 124 rte_eth_dev_info_get app/test-pmd/cmdline.c 1787 rte_eth_dev_info_get app/test-pmd/cmdline.c 1904 rte_eth_dev_info_get app/test-pmd/cmdline.c 3032 rte_eth_dev_info_get app/test-pmd/cmdline.c 3131 rte_eth_dev_info_get app/test-pmd/config.c 359 rte_eth_dev_info_get app/test-pmd/config.c 675 rte_eth_dev_info_get app/test-pmd/testpmd.c 565 rte_eth_dev_info_get app/test-pmd/testpmd.c 635 rte_eth_dev_info_get app/test/test_kni.c 389 rte_eth_dev_info_get app/test/test_kni.c 557 rte_eth_dev_info_get app/test/test_kni.c 586 rte_eth_dev_info_get examples/dpdk_qat/main.c 361 rte_eth_dev_info_get examples/dpdk_qat/main.c 370 rte_eth_dev_info_get examples/ip_fragmentation/main.c 924 rte_eth_dev_info_get examples/ip_reassembly/main.c 1130 rte_eth_dev_info_get examples/ipv4_multicast/main.c 786 rte_eth_dev_info_get examples/kni/main.c 804 rte_eth_dev_info_get examples/l2fwd-ivshmem/host/host.c 717 rte_eth_dev_info_get examples/l2fwd/main.c 603 rte_eth_dev_info_get examples/l3fwd-acl/main.c 2022 rte_eth_dev_info_get examples/l3fwd-power/main.c 1564 rte_eth_dev_info_get examples/l3fwd-vf/main.c 1032 rte_eth_dev_info_get examples/l3fwd/main.c 2494 rte_eth_dev_info_get examples/link_status_interrupt/main.c 652 rte_eth_dev_info_get examples/multi_process/l2fwd_fork/main.c 1064 rte_eth_dev_info_get examples/multi_process/symmetric_mp/main.c 245 rte_eth_dev_info_get examples/vhost/main.c 381 rte_eth_dev_info_get examples/vhost_xen/main.c 287 rte_eth_dev_info_get examples/vhost_xen/main.c 310 rte_eth_dev_info_get examples/vmdq/main.c 210 rte_eth_dev_info_get examples/vmdq/main.c 258 rte_eth_dev_info_get lib/librte_ether/rte_ethdev.c 877 rte_eth_dev_info_get lib/librte_ether/rte_ethdev.c 1083 rte_eth_dev_info_get lib/librte_ether/rte_ethdev.c 1152 rte_eth_dev_info_get lib/librte_ether/rte_ethdev.c 1483 rte_eth_dev_info_get lib/librte_ether/rte_ethdev.c 2268 rte_eth_dev_info_get lib/librte_ether/rte_ethdev.c 2381 rte_eth_dev_info_get lib/librte_ether/rte_ethdev.c 2409 rte_eth_dev_info_get lib/librte_ether/rte_ethdev.c 2507 rte_eth_dev_info_get lib/librte_ether/rte_ethdev.c 2533 rte_eth_dev_info_get lib/librte_ether/rte_ethdev.c 2590 rte_eth_dev_info_get lib/librte_ether/rte_ethdev.c 2627 rte_eth_dev_info_get lib/librte_ether/rte_ethdev.h 2165 rte_eth_dev_info_get lib/librte_ether/rte_ethdev.h 2941 rte_eth_dev_info_get lib/librte_ether/rte_ethdev.h 2960 rte_eth_dev_info_get lib/librte_kni/rte_kni.c 320 rte_eth_dev_info_get lib/librte_pmd_bond/rte_eth_bond_api.c 357 rte_eth_dev_get_mtu lib/librte_ether/rte_ethdev.c 1519 rte_eth_dev_get_mtu lib/librte_ether/rte_ethdev.h 2179 rte_eth_dev_set_mtu app/test-pmd/config.c 565 rte_eth_dev_set_mtu lib/librte_ether/rte_ethdev.c 1534 rte_eth_dev_set_mtu lib/librte_ether/rte_ethdev.h 2194 rte_eth_dev_vlan_filter app/test-pmd/config.c 1660 rte_eth_dev_vlan_filter app/test-pmd/config.c 1663 rte_eth_dev_vlan_filter lib/librte_ether/rte_ethdev.c 1555 rte_eth_dev_vlan_filter lib/librte_ether/rte_ethdev.h 2214 rte_eth_dev_set_vlan_strip_on_queue app/test-pmd/config.c 1623 rte_eth_dev_set_vlan_strip_on_queue examples/vhost/main.c 953 rte_eth_dev_set_vlan_strip_on_queue examples/vhost_xen/main.c 736 rte_eth_dev_set_vlan_strip_on_queue lib/librte_ether/rte_ethdev.c 1581 rte_eth_dev_set_vlan_strip_on_queue lib/librte_ether/rte_ethdev.h 2235 rte_eth_dev_set_vlan_ether_type app/test-pmd/config.c 1686 rte_eth_dev_set_vlan_ether_type lib/librte_ether/rte_ethdev.c 1604 rte_eth_dev_set_vlan_ether_type lib/librte_ether/rte_ethdev.h 2252 rte_eth_dev_set_vlan_offload app/test-pmd/config.c 1587 rte_eth_dev_set_vlan_offload app/test-pmd/config.c 1609 rte_eth_dev_set_vlan_offload app/test-pmd/config.c 1645 rte_eth_dev_set_vlan_offload lib/librte_ether/rte_ethdev.c 1621 rte_eth_dev_set_vlan_offload lib/librte_ether/rte_ethdev.h 2274 rte_eth_dev_get_vlan_offload app/test-pmd/config.c 339 rte_eth_dev_get_vlan_offload app/test-pmd/config.c 1580 rte_eth_dev_get_vlan_offload app/test-pmd/config.c 1602 rte_eth_dev_get_vlan_offload app/test-pmd/config.c 1638 rte_eth_dev_get_vlan_offload lib/librte_ether/rte_ethdev.c 1668 rte_eth_dev_get_vlan_offload lib/librte_ether/rte_ethdev.h 2288 rte_eth_dev_set_vlan_pvid app/test-pmd/config.c 1720 rte_eth_dev_set_vlan_pvid lib/librte_ether/rte_ethdev.c 1693 rte_eth_dev_set_vlan_pvid lib/librte_ether/rte_ethdev.h 2304 rte_eth_rx_burst app/test-pipeline/pipeline_hash.c 439 rte_eth_rx_burst app/test-pipeline/runtime.c 88 rte_eth_rx_burst app/test-pmd/csumonly.c 506 rte_eth_rx_burst app/test-pmd/flowgen.c 158 rte_eth_rx_burst app/test-pmd/icmpecho.c 313 rte_eth_rx_burst app/test-pmd/ieee1588fwd.c 540 rte_eth_rx_burst app/test-pmd/iofwd.c 97 rte_eth_rx_burst app/test-pmd/macfwd-retry.c 111 rte_eth_rx_burst app/test-pmd/macfwd.c 102 rte_eth_rx_burst app/test-pmd/macswap.c 102 rte_eth_rx_burst app/test-pmd/rxonly.c 110 rte_eth_rx_burst app/test-pmd/testpmd.c 922 rte_eth_rx_burst app/test/test_link_bonding.c 1587 rte_eth_rx_burst app/test/test_link_bonding.c 1672 rte_eth_rx_burst app/test/test_link_bonding.c 1961 rte_eth_rx_burst app/test/test_link_bonding.c 1964 rte_eth_rx_burst app/test/test_link_bonding.c 2193 rte_eth_rx_burst app/test/test_link_bonding.c 2195 rte_eth_rx_burst app/test/test_link_bonding.c 2522 rte_eth_rx_burst app/test/test_link_bonding.c 2524 rte_eth_rx_burst app/test/test_link_bonding.c 3109 rte_eth_rx_burst app/test/test_link_bonding.c 3451 rte_eth_rx_burst app/test/test_link_bonding.c 3704 rte_eth_rx_burst app/test/test_link_bonding.c 3980 rte_eth_rx_burst app/test/test_link_bonding.c 3982 rte_eth_rx_burst app/test/test_link_bonding.c 4177 rte_eth_rx_burst app/test/test_link_bonding.c 4180 rte_eth_rx_burst app/test/test_link_bonding.c 4507 rte_eth_rx_burst app/test/test_link_bonding.c 4509 rte_eth_rx_burst app/test/test_pmd_perf.c 394 rte_eth_rx_burst app/test/test_pmd_perf.c 433 rte_eth_rx_burst app/test/test_pmd_perf.c 470 rte_eth_rx_burst app/test/test_pmd_perf.c 548 rte_eth_rx_burst app/test/test_pmd_perf.c 611 rte_eth_rx_burst app/test/test_pmd_ring.c 142 rte_eth_rx_burst app/test/test_pmd_ring.c 178 rte_eth_rx_burst app/test/test_pmd_ring.c 218 rte_eth_rx_burst app/test/test_pmd_ring.c 289 rte_eth_rx_burst app/test/test_pmd_ring.c 319 rte_eth_rx_burst app/test/test_pmd_ring.c 349 rte_eth_rx_burst app/test/test_pmd_ring.c 379 rte_eth_rx_burst examples/distributor/main.c 230 rte_eth_rx_burst examples/dpdk_qat/main.c 193 rte_eth_rx_burst examples/exception_path/main.c 245 rte_eth_rx_burst examples/ip_fragmentation/main.c 466 rte_eth_rx_burst examples/ip_pipeline/pipeline_rx.c 305 rte_eth_rx_burst examples/ip_reassembly/main.c 511 rte_eth_rx_burst examples/ipv4_multicast/main.c 465 rte_eth_rx_burst examples/kni/main.c 257 rte_eth_rx_burst examples/l2fwd-ivshmem/host/host.c 610 rte_eth_rx_burst examples/l2fwd/main.c 334 rte_eth_rx_burst examples/l3fwd-acl/main.c 1454 rte_eth_rx_burst examples/l3fwd-power/main.c 849 rte_eth_rx_burst examples/l3fwd-vf/main.c 562 rte_eth_rx_burst examples/l3fwd/main.c 1470 rte_eth_rx_burst examples/link_status_interrupt/main.c 353 rte_eth_rx_burst examples/load_balancer/runtime.c 196 rte_eth_rx_burst examples/multi_process/client_server_mp/mp_server/main.c 288 rte_eth_rx_burst examples/multi_process/l2fwd_fork/main.c 718 rte_eth_rx_burst examples/multi_process/symmetric_mp/main.c 345 rte_eth_rx_burst examples/netmap_compat/lib/compat_netmap.c 471 rte_eth_rx_burst examples/qos_meter/main.c 217 rte_eth_rx_burst examples/qos_sched/app_thread.c 96 rte_eth_rx_burst examples/quota_watermark/qw/main.c 188 rte_eth_rx_burst examples/skeleton/basicfwd.c 127 rte_eth_rx_burst examples/vhost/main.c 981 rte_eth_rx_burst examples/vhost/main.c 988 rte_eth_rx_burst examples/vhost/main.c 1274 rte_eth_rx_burst examples/vhost/main.c 2114 rte_eth_rx_burst examples/vhost_xen/main.c 765 rte_eth_rx_burst examples/vhost_xen/main.c 772 rte_eth_rx_burst examples/vhost_xen/main.c 1065 rte_eth_rx_burst examples/vmdq/main.c 520 rte_eth_rx_burst examples/vmdq_dcb/main.c 354 rte_eth_rx_burst lib/librte_ether/rte_ethdev.c 2715 rte_eth_rx_burst lib/librte_ether/rte_ethdev.h 2312 rte_eth_rx_burst lib/librte_ether/rte_ethdev.h 2328 rte_eth_rx_burst lib/librte_ether/rte_ethdev.h 2332 rte_eth_rx_burst lib/librte_ether/rte_ethdev.h 2339 rte_eth_rx_burst lib/librte_ether/rte_ethdev.h 2354 rte_eth_rx_burst lib/librte_ether/rte_ethdev.h 2358 rte_eth_rx_burst lib/librte_ether/rte_ethdev.h 2367 rte_eth_rx_burst lib/librte_ether/rte_ethdev.h 2389 rte_eth_rx_burst lib/librte_ether/rte_ethdev.h 2393 rte_eth_rx_burst lib/librte_pmd_bond/rte_eth_bond_pmd.c 78 rte_eth_rx_burst lib/librte_pmd_bond/rte_eth_bond_pmd.c 100 rte_eth_rx_burst lib/librte_pmd_bond/rte_eth_bond_pmd.c 136 rte_eth_rx_burst lib/librte_port/rte_port_ethdev.c 83 rte_eth_rx_queue_count lib/librte_ether/rte_ethdev.c 2758 rte_eth_rx_queue_count lib/librte_ether/rte_ethdev.h 2414 rte_eth_rx_queue_count lib/librte_ether/rte_ethdev.h 2417 rte_eth_rx_descriptor_done examples/l3fwd-power/main.c 746 rte_eth_rx_descriptor_done examples/l3fwd-power/main.c 750 rte_eth_rx_descriptor_done examples/l3fwd-power/main.c 753 rte_eth_rx_descriptor_done lib/librte_ether/rte_ethdev.c 2773 rte_eth_rx_descriptor_done lib/librte_ether/rte_ethdev.h 2441 rte_eth_rx_descriptor_done lib/librte_ether/rte_ethdev.h 2446 rte_eth_tx_burst app/test-pipeline/runtime.c 166 rte_eth_tx_burst app/test-pmd/csumonly.c 676 rte_eth_tx_burst app/test-pmd/flowgen.c 219 rte_eth_tx_burst app/test-pmd/icmpecho.c 470 rte_eth_tx_burst app/test-pmd/ieee1588fwd.c 614 rte_eth_tx_burst app/test-pmd/iofwd.c 106 rte_eth_tx_burst app/test-pmd/macfwd-retry.c 128 rte_eth_tx_burst app/test-pmd/macfwd-retry.c 136 rte_eth_tx_burst app/test-pmd/macfwd.c 126 rte_eth_tx_burst app/test-pmd/macswap.c 128 rte_eth_tx_burst app/test-pmd/txonly.c 275 rte_eth_tx_burst app/test/test_link_bonding.c 1392 rte_eth_tx_burst app/test/test_link_bonding.c 1420 rte_eth_tx_burst app/test/test_link_bonding.c 1501 rte_eth_tx_burst app/test/test_link_bonding.c 1914 rte_eth_tx_burst app/test/test_link_bonding.c 1915 rte_eth_tx_burst app/test/test_link_bonding.c 2115 rte_eth_tx_burst app/test/test_link_bonding.c 2151 rte_eth_tx_burst app/test/test_link_bonding.c 2487 rte_eth_tx_burst app/test/test_link_bonding.c 2489 rte_eth_tx_burst app/test/test_link_bonding.c 2665 rte_eth_tx_burst app/test/test_link_bonding.c 2700 rte_eth_tx_burst app/test/test_link_bonding.c 2743 rte_eth_tx_burst app/test/test_link_bonding.c 2748 rte_eth_tx_burst app/test/test_link_bonding.c 2780 rte_eth_tx_burst app/test/test_link_bonding.c 2855 rte_eth_tx_burst app/test/test_link_bonding.c 2860 rte_eth_tx_burst app/test/test_link_bonding.c 2893 rte_eth_tx_burst app/test/test_link_bonding.c 2996 rte_eth_tx_burst app/test/test_link_bonding.c 3013 rte_eth_tx_burst app/test/test_link_bonding.c 3373 rte_eth_tx_burst app/test/test_link_bonding.c 3375 rte_eth_tx_burst app/test/test_link_bonding.c 3377 rte_eth_tx_burst app/test/test_link_bonding.c 3379 rte_eth_tx_burst app/test/test_link_bonding.c 3414 rte_eth_tx_burst app/test/test_link_bonding.c 3416 rte_eth_tx_burst app/test/test_link_bonding.c 3510 rte_eth_tx_burst app/test/test_link_bonding.c 3541 rte_eth_tx_burst app/test/test_link_bonding.c 3606 rte_eth_tx_burst app/test/test_link_bonding.c 3938 rte_eth_tx_burst app/test/test_link_bonding.c 3940 rte_eth_tx_burst app/test/test_link_bonding.c 4085 rte_eth_tx_burst app/test/test_link_bonding.c 4132 rte_eth_tx_burst app/test/test_link_bonding.c 4469 rte_eth_tx_burst app/test/test_link_bonding.c 4471 rte_eth_tx_burst app/test/test_pmd_perf.c 402 rte_eth_tx_burst app/test/test_pmd_perf.c 442 rte_eth_tx_burst app/test/test_pmd_perf.c 480 rte_eth_tx_burst app/test/test_pmd_perf.c 525 rte_eth_tx_burst app/test/test_pmd_perf.c 669 rte_eth_tx_burst app/test/test_pmd_ring.c 137 rte_eth_tx_burst app/test/test_pmd_ring.c 174 rte_eth_tx_burst app/test/test_pmd_ring.c 213 rte_eth_tx_burst app/test/test_pmd_ring.c 284 rte_eth_tx_burst app/test/test_pmd_ring.c 314 rte_eth_tx_burst app/test/test_pmd_ring.c 344 rte_eth_tx_burst app/test/test_pmd_ring.c 374 rte_eth_tx_burst examples/distributor/main.c 270 rte_eth_tx_burst examples/dpdk_qat/main.c 235 rte_eth_tx_burst examples/dpdk_qat/main.c 270 rte_eth_tx_burst examples/exception_path/main.c 291 rte_eth_tx_burst examples/ip_fragmentation/main.c 257 rte_eth_tx_burst examples/ip_pipeline/pipeline_tx.c 264 rte_eth_tx_burst examples/ip_reassembly/main.c 297 rte_eth_tx_burst examples/ipv4_multicast/main.c 212 rte_eth_tx_burst examples/kni/main.c 299 rte_eth_tx_burst examples/l2fwd-ivshmem/host/host.c 409 rte_eth_tx_burst examples/l2fwd/main.c 200 rte_eth_tx_burst examples/l3fwd-acl/main.c 1310 rte_eth_tx_burst examples/l3fwd-power/main.c 444 rte_eth_tx_burst examples/l3fwd-vf/main.c 319 rte_eth_tx_burst examples/l3fwd/main.c 508 rte_eth_tx_burst examples/l3fwd/main.c 556 rte_eth_tx_burst examples/link_status_interrupt/main.c 218 rte_eth_tx_burst examples/load_balancer/runtime.c 384 rte_eth_tx_burst examples/load_balancer/runtime.c 432 rte_eth_tx_burst examples/multi_process/client_server_mp/mp_client/client.c 181 rte_eth_tx_burst examples/multi_process/l2fwd_fork/main.c 596 rte_eth_tx_burst examples/multi_process/symmetric_mp/main.c 350 rte_eth_tx_burst examples/netmap_compat/lib/compat_netmap.c 561 rte_eth_tx_burst examples/qos_meter/main.c 201 rte_eth_tx_burst examples/qos_meter/main.c 234 rte_eth_tx_burst examples/qos_sched/app_thread.c 141 rte_eth_tx_burst examples/quota_watermark/qw/main.c 110 rte_eth_tx_burst examples/quota_watermark/qw/main.c 299 rte_eth_tx_burst examples/skeleton/basicfwd.c 131 rte_eth_tx_burst examples/vhost/main.c 1170 rte_eth_tx_burst examples/vhost/main.c 1232 rte_eth_tx_burst examples/vhost/main.c 1844 rte_eth_tx_burst examples/vhost/main.c 2050 rte_eth_tx_burst examples/vhost_xen/main.c 889 rte_eth_tx_burst examples/vhost_xen/main.c 1025 rte_eth_tx_burst examples/vmdq/main.c 531 rte_eth_tx_burst examples/vmdq_dcb/main.c 360 rte_eth_tx_burst lib/librte_ether/rte_ethdev.c 2736 rte_eth_tx_burst lib/librte_ether/rte_ethdev.h 2459 rte_eth_tx_burst lib/librte_ether/rte_ethdev.h 2464 rte_eth_tx_burst lib/librte_ether/rte_ethdev.h 2467 rte_eth_tx_burst lib/librte_ether/rte_ethdev.h 2478 rte_eth_tx_burst lib/librte_ether/rte_ethdev.h 2481 rte_eth_tx_burst lib/librte_ether/rte_ethdev.h 2486 rte_eth_tx_burst lib/librte_ether/rte_ethdev.h 2489 rte_eth_tx_burst lib/librte_ether/rte_ethdev.h 2494 rte_eth_tx_burst lib/librte_ether/rte_ethdev.h 2516 rte_eth_tx_burst lib/librte_ether/rte_ethdev.h 2520 rte_eth_tx_burst lib/librte_pmd_bond/rte_eth_bond_pmd.c 218 rte_eth_tx_burst lib/librte_pmd_bond/rte_eth_bond_pmd.c 251 rte_eth_tx_burst lib/librte_pmd_bond/rte_eth_bond_pmd.c 521 rte_eth_tx_burst lib/librte_pmd_bond/rte_eth_bond_pmd.c 572 rte_eth_tx_burst lib/librte_pmd_bond/rte_eth_bond_pmd.c 663 rte_eth_tx_burst lib/librte_pmd_bond/rte_eth_bond_pmd.c 718 rte_eth_tx_burst lib/librte_port/rte_port_ethdev.c 152 rte_eth_tx_burst lib/librte_port/rte_port_ethdev.c 232 rte_eth_dev_fdir_add_signature_filter app/test-pmd/config.c 1784 rte_eth_dev_fdir_add_signature_filter app/test-pmd/config.c 1789 rte_eth_dev_fdir_add_signature_filter app/test-pmd/config.c 1824 rte_eth_dev_fdir_add_signature_filter lib/librte_ether/rte_ethdev.c 1710 rte_eth_dev_fdir_add_signature_filter lib/librte_ether/rte_ethdev.h 2552 rte_eth_dev_fdir_update_signature_filter app/test-pmd/config.c 1802 rte_eth_dev_fdir_update_signature_filter app/test-pmd/config.c 1807 rte_eth_dev_fdir_update_signature_filter lib/librte_ether/rte_ethdev.c 1744 rte_eth_dev_fdir_update_signature_filter lib/librte_ether/rte_ethdev.h 2578 rte_eth_dev_fdir_remove_signature_filter app/test-pmd/config.c 1820 rte_eth_dev_fdir_remove_signature_filter lib/librte_ether/rte_ethdev.c 1779 rte_eth_dev_fdir_remove_signature_filter lib/librte_ether/rte_ethdev.h 2600 rte_eth_dev_fdir_get_infos app/test-pmd/config.c 1896 rte_eth_dev_fdir_get_infos lib/librte_ether/rte_ethdev.c 1811 rte_eth_dev_fdir_get_infos lib/librte_ether/rte_ethdev.h 2617 rte_eth_dev_fdir_add_perfect_filter app/test-pmd/config.c 1966 rte_eth_dev_fdir_add_perfect_filter app/test-pmd/config.c 1971 rte_eth_dev_fdir_add_perfect_filter lib/librte_ether/rte_ethdev.c 1833 rte_eth_dev_fdir_add_perfect_filter lib/librte_ether/rte_ethdev.h 2647 rte_eth_dev_fdir_update_perfect_filter app/test-pmd/config.c 1984 rte_eth_dev_fdir_update_perfect_filter app/test-pmd/config.c 1989 rte_eth_dev_fdir_update_perfect_filter app/test-pmd/config.c 2007 rte_eth_dev_fdir_update_perfect_filter lib/librte_ether/rte_ethdev.c 1873 rte_eth_dev_fdir_update_perfect_filter lib/librte_ether/rte_ethdev.h 2681 rte_eth_dev_fdir_remove_perfect_filter app/test-pmd/config.c 2002 rte_eth_dev_fdir_remove_perfect_filter lib/librte_ether/rte_ethdev.c 1912 rte_eth_dev_fdir_remove_perfect_filter lib/librte_ether/rte_ethdev.h 2707 rte_eth_dev_fdir_set_masks app/test-pmd/config.c 2019 rte_eth_dev_fdir_set_masks lib/librte_ether/rte_ethdev.c 1950 rte_eth_dev_fdir_set_masks lib/librte_ether/rte_ethdev.h 2743 rte_eth_dev_callback_register app/test/test_link_bonding.c 1236 rte_eth_dev_callback_register app/test/test_link_bonding.c 2036 rte_eth_dev_callback_register examples/link_status_interrupt/main.c 706 rte_eth_dev_callback_register lib/librte_ether/rte_ethdev.c 2790 rte_eth_dev_callback_register lib/librte_ether/rte_ethdev.h 2777 rte_eth_dev_callback_register lib/librte_pmd_bond/rte_eth_bond_api.c 410 rte_eth_dev_callback_unregister app/test/test_link_bonding.c 1303 rte_eth_dev_callback_unregister app/test/test_link_bonding.c 2064 rte_eth_dev_callback_unregister lib/librte_ether/rte_ethdev.c 2830 rte_eth_dev_callback_unregister lib/librte_ether/rte_ethdev.h 2798 rte_eth_dev_callback_unregister lib/librte_pmd_bond/rte_eth_bond_api.c 486 rte_eth_led_on lib/librte_ether/rte_ethdev.c 2228 rte_eth_led_on lib/librte_ether/rte_ethdev.h 2830 rte_eth_led_off lib/librte_ether/rte_ethdev.c 2243 rte_eth_led_off lib/librte_ether/rte_ethdev.h 2844 rte_eth_dev_flow_ctrl_get app/test-pmd/cmdline.c 5220 rte_eth_dev_flow_ctrl_get lib/librte_ether/rte_ethdev.c 1970 rte_eth_dev_flow_ctrl_get lib/librte_ether/rte_ethdev.h 2858 rte_eth_dev_flow_ctrl_set app/test-pmd/cmdline.c 5265 rte_eth_dev_flow_ctrl_set examples/quota_watermark/qw/init.c 105 rte_eth_dev_flow_ctrl_set lib/librte_ether/rte_ethdev.c 1986 rte_eth_dev_flow_ctrl_set lib/librte_ether/rte_ethdev.h 2875 rte_eth_dev_priority_flow_ctrl_set app/test-pmd/cmdline.c 5313 rte_eth_dev_priority_flow_ctrl_set lib/librte_ether/rte_ethdev.c 2006 rte_eth_dev_priority_flow_ctrl_set lib/librte_ether/rte_ethdev.h 2893 rte_eth_dev_mac_addr_add app/test-pmd/cmdline.c 6076 rte_eth_dev_mac_addr_add app/test-pmd/cmdline.c 6558 rte_eth_dev_mac_addr_add examples/vhost/main.c 946 rte_eth_dev_mac_addr_add examples/vhost_xen/main.c 728 rte_eth_dev_mac_addr_add examples/vmdq/main.c 314 rte_eth_dev_mac_addr_add lib/librte_ether/rte_ethdev.c 2280 rte_eth_dev_mac_addr_add lib/librte_ether/rte_ethdev.h 2914 rte_eth_dev_mac_addr_remove app/test-pmd/cmdline.c 6078 rte_eth_dev_mac_addr_remove examples/vhost/main.c 974 rte_eth_dev_mac_addr_remove examples/vhost_xen/main.c 758 rte_eth_dev_mac_addr_remove lib/librte_ether/rte_ethdev.c 2334 rte_eth_dev_mac_addr_remove lib/librte_ether/rte_ethdev.h 2930 rte_eth_dev_rss_reta_update app/test-pmd/cmdline.c 1809 rte_eth_dev_rss_reta_update lib/librte_ether/rte_ethdev.c 2083 rte_eth_dev_rss_reta_update lib/librte_ether/rte_ethdev.h 2947 rte_eth_dev_rss_reta_query app/test-pmd/config.c 807 rte_eth_dev_rss_reta_query lib/librte_ether/rte_ethdev.c 2113 rte_eth_dev_rss_reta_query lib/librte_ether/rte_ethdev.h 2966 rte_eth_dev_uc_hash_table_set app/test-pmd/cmdline.c 6188 rte_eth_dev_uc_hash_table_set lib/librte_ether/rte_ethdev.c 2422 rte_eth_dev_uc_hash_table_set lib/librte_ether/rte_ethdev.h 2988 rte_eth_dev_uc_all_hash_table_set app/test-pmd/cmdline.c 6250 rte_eth_dev_uc_all_hash_table_set lib/librte_ether/rte_ethdev.c 2478 rte_eth_dev_uc_all_hash_table_set lib/librte_ether/rte_ethdev.h 3008 rte_eth_dev_set_vf_rxmode app/test-pmd/cmdline.c 6491 rte_eth_dev_set_vf_rxmode lib/librte_ether/rte_ethdev.c 2367 rte_eth_dev_set_vf_rxmode lib/librte_ether/rte_ethdev.h 3032 rte_eth_dev_set_vf_tx app/test-pmd/config.c 2098 rte_eth_dev_set_vf_tx app/test-pmd/config.c 2105 rte_eth_dev_set_vf_tx lib/librte_ether/rte_ethdev.c 2521 rte_eth_dev_set_vf_tx lib/librte_ether/rte_ethdev.h 3052 rte_eth_dev_set_vf_rx app/test-pmd/cmdline.c 6491 rte_eth_dev_set_vf_rx app/test-pmd/config.c 2096 rte_eth_dev_set_vf_rx app/test-pmd/config.c 2102 rte_eth_dev_set_vf_rx lib/librte_ether/rte_ethdev.c 2367 rte_eth_dev_set_vf_rx lib/librte_ether/rte_ethdev.c 2495 rte_eth_dev_set_vf_rx lib/librte_ether/rte_ethdev.h 3032 rte_eth_dev_set_vf_rx lib/librte_ether/rte_ethdev.h 3071 rte_eth_dev_set_vf_vlan_filter app/test-pmd/config.c 2119 rte_eth_dev_set_vf_vlan_filter app/test-pmd/config.c 2122 rte_eth_dev_set_vf_vlan_filter lib/librte_ether/rte_ethdev.c 2547 rte_eth_dev_set_vf_vlan_filter lib/librte_ether/rte_ethdev.h 3093 rte_eth_mirror_rule_set app/test-pmd/cmdline.c 7094 rte_eth_mirror_rule_set app/test-pmd/cmdline.c 7097 rte_eth_mirror_rule_set app/test-pmd/cmdline.c 7182 rte_eth_mirror_rule_set app/test-pmd/cmdline.c 7185 rte_eth_mirror_rule_set lib/librte_ether/rte_ethdev.c 2648 rte_eth_mirror_rule_set lib/librte_ether/rte_ethdev.h 3118 rte_eth_mirror_rule_reset app/test-pmd/cmdline.c 7246 rte_eth_mirror_rule_reset lib/librte_ether/rte_ethdev.c 2691 rte_eth_mirror_rule_reset lib/librte_ether/rte_ethdev.h 3136 rte_eth_set_queue_rate_limit app/test-pmd/config.c 2140 rte_eth_set_queue_rate_limit app/test-pmd/config.c 2143 rte_eth_set_queue_rate_limit lib/librte_ether/rte_ethdev.c 2576 rte_eth_set_queue_rate_limit lib/librte_ether/rte_ethdev.h 3154 rte_eth_set_vf_rate_limit app/test-pmd/config.c 2165 rte_eth_set_vf_rate_limit app/test-pmd/config.c 2168 rte_eth_set_vf_rate_limit lib/librte_ether/rte_ethdev.c 2610 rte_eth_set_vf_rate_limit lib/librte_ether/rte_ethdev.h 3174 rte_eth_dev_bypass_init app/test-pmd/testpmd.c 1756 rte_eth_dev_bypass_init lib/librte_ether/rte_ethdev.c 2897 rte_eth_dev_bypass_init lib/librte_ether/rte_ethdev.h 3188 rte_eth_dev_bypass_state_show app/test-pmd/cmdline.c 3533 rte_eth_dev_bypass_state_show lib/librte_ether/rte_ethdev.c 2917 rte_eth_dev_bypass_state_show lib/librte_ether/rte_ethdev.h 3205 rte_eth_dev_bypass_state_set app/test-pmd/cmdline.c 3293 rte_eth_dev_bypass_state_set lib/librte_ether/rte_ethdev.c 2936 rte_eth_dev_bypass_state_set lib/librte_ether/rte_ethdev.h 3222 rte_eth_dev_wd_timeout_store app/test-pmd/cmdline.c 3379 rte_eth_dev_wd_timeout_store lib/librte_ether/rte_ethdev.c 2996 rte_eth_dev_wd_timeout_store lib/librte_ether/rte_ethdev.h 3294 rte_eth_dev_bypass_ver_show lib/librte_ether/rte_ethdev.c 3016 rte_eth_dev_bypass_ver_show lib/librte_ether/rte_ethdev.h 3308 rte_eth_dev_bypass_wd_timeout_show lib/librte_ether/rte_ethdev.c 3036 rte_eth_dev_bypass_wd_timeout_show lib/librte_ether/rte_ethdev.h 3330 rte_eth_dev_bypass_wd_reset lib/librte_ether/rte_ethdev.c 3056 rte_eth_dev_bypass_wd_reset lib/librte_ether/rte_ethdev.h 3342 rte_eth_dev_rss_hash_update app/test-pmd/cmdline.c 1494 rte_eth_dev_rss_hash_update app/test-pmd/config.c 898 rte_eth_dev_rss_hash_update lib/librte_ether/rte_ethdev.c 2136 rte_eth_dev_rss_hash_update lib/librte_ether/rte_ethdev.h 3357 rte_eth_dev_rss_hash_conf_get app/test-pmd/config.c 840 rte_eth_dev_rss_hash_conf_get app/test-pmd/config.c 895 rte_eth_dev_rss_hash_conf_get lib/librte_ether/rte_ethdev.c 2159 rte_eth_dev_rss_hash_conf_get lib/librte_ether/rte_ethdev.h 3374 rte_eth_dev_udp_tunnel_add app/test-pmd/cmdline.c 6977 rte_eth_dev_udp_tunnel_add lib/librte_ether/rte_ethdev.c 2175 rte_eth_dev_udp_tunnel_add lib/librte_ether/rte_ethdev.h 3392 rte_eth_dev_udp_tunnel_delete app/test-pmd/cmdline.c 6979 rte_eth_dev_udp_tunnel_delete lib/librte_ether/rte_ethdev.c 2201 rte_eth_dev_udp_tunnel_delete lib/librte_ether/rte_ethdev.h 3409 rte_eth_dev_add_syn_filter app/test-pmd/cmdline.c 7396 rte_eth_dev_add_syn_filter lib/librte_ether/rte_ethdev.c 3077 rte_eth_dev_add_syn_filter lib/librte_ether/rte_ethdev.h 3426 rte_eth_dev_remove_syn_filter app/test-pmd/cmdline.c 7399 rte_eth_dev_remove_syn_filter lib/librte_ether/rte_ethdev.c 3093 rte_eth_dev_remove_syn_filter lib/librte_ether/rte_ethdev.h 3439 rte_eth_dev_get_syn_filter app/test-pmd/config.c 2181 rte_eth_dev_get_syn_filter lib/librte_ether/rte_ethdev.c 3108 rte_eth_dev_get_syn_filter lib/librte_ether/rte_ethdev.h 3455 rte_eth_dev_add_2tuple_filter app/test-pmd/cmdline.c 7506 rte_eth_dev_add_2tuple_filter lib/librte_ether/rte_ethdev.c 3127 rte_eth_dev_add_2tuple_filter lib/librte_ether/rte_ethdev.h 3479 rte_eth_dev_remove_2tuple_filter app/test-pmd/cmdline.c 7509 rte_eth_dev_remove_2tuple_filter lib/librte_ether/rte_ethdev.c 3151 rte_eth_dev_remove_2tuple_filter lib/librte_ether/rte_ethdev.h 3495 rte_eth_dev_get_2tuple_filter app/test-pmd/config.c 2202 rte_eth_dev_get_2tuple_filter lib/librte_ether/rte_ethdev.c 3166 rte_eth_dev_get_2tuple_filter lib/librte_ether/rte_ethdev.h 3516 rte_eth_dev_add_5tuple_filter app/test-pmd/cmdline.c 7694 rte_eth_dev_add_5tuple_filter lib/librte_ether/rte_ethdev.c 3185 rte_eth_dev_add_5tuple_filter lib/librte_ether/rte_ethdev.h 3539 rte_eth_dev_remove_5tuple_filter app/test-pmd/cmdline.c 7697 rte_eth_dev_remove_5tuple_filter lib/librte_ether/rte_ethdev.c 3209 rte_eth_dev_remove_5tuple_filter lib/librte_ether/rte_ethdev.h 3555 rte_eth_dev_get_5tuple_filter app/test-pmd/config.c 2231 rte_eth_dev_get_5tuple_filter lib/librte_ether/rte_ethdev.c 3224 rte_eth_dev_get_5tuple_filter lib/librte_ether/rte_ethdev.h 3575 rte_eth_dev_add_flex_filter app/test-pmd/cmdline.c 7959 rte_eth_dev_add_flex_filter app/test-pmd/cmdline.c 7963 rte_eth_dev_add_flex_filter lib/librte_ether/rte_ethdev.c 3244 rte_eth_dev_add_flex_filter lib/librte_ether/rte_ethdev.h 3599 rte_eth_dev_remove_flex_filter app/test-pmd/cmdline.c 7967 rte_eth_dev_remove_flex_filter lib/librte_ether/rte_ethdev.c 3260 rte_eth_dev_remove_flex_filter lib/librte_ether/rte_ethdev.h 3615 rte_eth_dev_get_flex_filter app/test-pmd/config.c 2270 rte_eth_dev_get_flex_filter lib/librte_ether/rte_ethdev.c 3275 rte_eth_dev_get_flex_filter lib/librte_ether/rte_ethdev.h 3636 rte_eth_dev_filter_supported app/test-pmd/cmdline.c 8129 rte_eth_dev_filter_supported app/test-pmd/cmdline.c 8303 rte_eth_dev_filter_supported app/test-pmd/cmdline.c 8555 rte_eth_dev_filter_supported app/test-pmd/cmdline.c 8799 rte_eth_dev_filter_supported app/test-pmd/cmdline.c 8855 rte_eth_dev_filter_supported app/test-pmd/cmdline.c 8944 rte_eth_dev_filter_supported app/test-pmd/cmdline.c 9027 rte_eth_dev_filter_supported app/test-pmd/config.c 1891 rte_eth_dev_filter_supported lib/librte_ether/rte_ethdev.c 3295 rte_eth_dev_filter_supported lib/librte_ether/rte_ethdev.h 3652 rte_eth_dev_filter_ctrl app/test-pmd/cmdline.c 6333 rte_eth_dev_filter_ctrl app/test-pmd/cmdline.c 6338 rte_eth_dev_filter_ctrl app/test-pmd/cmdline.c 6879 rte_eth_dev_filter_ctrl app/test-pmd/cmdline.c 6884 rte_eth_dev_filter_ctrl app/test-pmd/cmdline.c 8149 rte_eth_dev_filter_ctrl app/test-pmd/cmdline.c 8154 rte_eth_dev_filter_ctrl app/test-pmd/cmdline.c 8383 rte_eth_dev_filter_ctrl app/test-pmd/cmdline.c 8386 rte_eth_dev_filter_ctrl app/test-pmd/cmdline.c 8562 rte_eth_dev_filter_ctrl app/test-pmd/cmdline.c 8808 rte_eth_dev_filter_ctrl app/test-pmd/cmdline.c 8866 rte_eth_dev_filter_ctrl app/test-pmd/cmdline.c 8953 rte_eth_dev_filter_ctrl app/test-pmd/cmdline.c 9052 rte_eth_dev_filter_ctrl app/test-pmd/config.c 1917 rte_eth_dev_filter_ctrl app/test-pmd/config.c 1920 rte_eth_dev_filter_ctrl lib/librte_ether/rte_ethdev.c 3311 rte_eth_dev_filter_ctrl lib/librte_ether/rte_ethdev.h 3673
> -----Original Message-----
> From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Tetsuya Mukawa
> Sent: Wednesday, February 18, 2015 6:10 AM
> To: Thomas Monjalon
> Cc: dev@dpdk.org; Neil Horman
> Subject: Re: [dpdk-dev] [PATCH v8 03/14] eal/pci, ethdev: Remove assumption that port will not be
> detached
>
> On 2015/02/18 10:54, Tetsuya Mukawa wrote:
> > On 2015/02/18 9:31, Thomas Monjalon wrote:
> >> 2015-02-17 15:14, Tetsuya Mukawa:
> >>> On 2015/02/17 9:36, Thomas Monjalon wrote:
> >>>> 2015-02-16 13:14, Tetsuya Mukawa:
> >>>> Is uint8_t sill a good size for hotpluggable virtual device ids?
> >>> I am not sure it's enough, but uint8_t is widely used in "rte_ethdev.c"
> >>> as port id.
> >>> If someone reports it doesn't enough, I guess it will be the time to
> >>> write a patch to change all uint_8 in one patch.
> >> It's a big ABI breakage. So if we feel it's going to be required,
> >> it's better to do it now in 2.0 release I think.
> >>
> >> Any opinion?
> >>
> > Hi Thomas,
> >
> > I agree with it.
> > I will add an one more patch to change uint8_t to uint16_t.
> >
> > Thanks,
> > Tetsuya
> >
>
> Hi Thomas,
>
> Could I make sure.
> After changing uint8_t to uint16_t in "rte_ethdev.[ch]", must I also need to change other applications
> and libraries that call ethdev APIs?
> If so, I would not finish it by 23rd.
>
> I've counted how many lines call ethdev APIs that are related to port_id.
> Could you please check an attached file?
> It's over 1200 lines. Probably to fix one of caller, I will need to check how port_id is used, and fix more
> related lines. So probably thousands lines may need to be fixed.
>
> When is deadline for fixing this changing?
> Also, if you have a good idea to fix it easier, could you please let me know?
>
> Thanks,
> Tetsuya
Hi Tetsuya, Thomas,
As uint8_t is already widely used for port_id, I don't think it should be changed in this patchset.
If it is to be changed to uint16_t it should be done as a separate task (in a new patchset).
Regards,
Bernard.
On 2015/02/18 10:09, Thomas Monjalon wrote:
> 2015-02-17 15:15, Tetsuya Mukawa:
>> On 2015/02/17 10:11, Thomas Monjalon wrote:
>>> 2015-02-16 13:14, Tetsuya Mukawa:
>>>> +#ifdef ENABLE_HOTPLUG
>>> Please avoid using #ifdef if not really necessary.
>> I agree with you.
>> In this case, only hotplug functions call pci_unmap_resource().
>> So this will be needed.
> Why is it needed?
It was my misunderstanding. Yes, we can remove it.
I will fix it.
Thanks,
Tetsuya
2015-02-18 15:10, Tetsuya Mukawa:
> On 2015/02/18 10:54, Tetsuya Mukawa wrote:
> > On 2015/02/18 9:31, Thomas Monjalon wrote:
> >> 2015-02-17 15:14, Tetsuya Mukawa:
> >>> On 2015/02/17 9:36, Thomas Monjalon wrote:
> >>>> 2015-02-16 13:14, Tetsuya Mukawa:
> >>>> Is uint8_t sill a good size for hotpluggable virtual device ids?
> >>> I am not sure it's enough, but uint8_t is widely used in "rte_ethdev.c"
> >>> as port id.
> >>> If someone reports it doesn't enough, I guess it will be the time to
> >>> write a patch to change all uint_8 in one patch.
> >> It's a big ABI breakage. So if we feel it's going to be required,
> >> it's better to do it now in 2.0 release I think.
> >>
> >> Any opinion?
> >>
> > Hi Thomas,
> >
> > I agree with it.
> > I will add an one more patch to change uint8_t to uint16_t.
> >
> > Thanks,
> > Tetsuya
> >
>
> Hi Thomas,
>
> Could I make sure.
> After changing uint8_t to uint16_t in "rte_ethdev.[ch]", must I also
> need to change other applications and libraries that call ethdev APIs?
> If so, I would not finish it by 23rd.
>
> I've counted how many lines call ethdev APIs that are related to port_id.
> Could you please check an attached file?
> It's over 1200 lines. Probably to fix one of caller, I will need to
> check how port_id is used, and fix more related lines. So probably
> thousands lines may need to be fixed.
>
> When is deadline for fixing this changing?
> Also, if you have a good idea to fix it easier, could you please let me
> know?
It was an open question.
If everybody is fine with 255 ports maximum, let's keep it as is.
On Wed, Feb 18, 2015 at 10:57:25AM +0100, Thomas Monjalon wrote:
> 2015-02-18 15:10, Tetsuya Mukawa:
> > On 2015/02/18 10:54, Tetsuya Mukawa wrote:
> > > On 2015/02/18 9:31, Thomas Monjalon wrote:
> > >> 2015-02-17 15:14, Tetsuya Mukawa:
> > >>> On 2015/02/17 9:36, Thomas Monjalon wrote:
> > >>>> 2015-02-16 13:14, Tetsuya Mukawa:
> > >>>> Is uint8_t sill a good size for hotpluggable virtual device ids?
> > >>> I am not sure it's enough, but uint8_t is widely used in "rte_ethdev.c"
> > >>> as port id.
> > >>> If someone reports it doesn't enough, I guess it will be the time to
> > >>> write a patch to change all uint_8 in one patch.
> > >> It's a big ABI breakage. So if we feel it's going to be required,
> > >> it's better to do it now in 2.0 release I think.
> > >>
> > >> Any opinion?
> > >>
> > > Hi Thomas,
> > >
> > > I agree with it.
> > > I will add an one more patch to change uint8_t to uint16_t.
> > >
> > > Thanks,
> > > Tetsuya
> > >
> >
> > Hi Thomas,
> >
> > Could I make sure.
> > After changing uint8_t to uint16_t in "rte_ethdev.[ch]", must I also
> > need to change other applications and libraries that call ethdev APIs?
> > If so, I would not finish it by 23rd.
> >
> > I've counted how many lines call ethdev APIs that are related to port_id.
> > Could you please check an attached file?
> > It's over 1200 lines. Probably to fix one of caller, I will need to
> > check how port_id is used, and fix more related lines. So probably
> > thousands lines may need to be fixed.
> >
> > When is deadline for fixing this changing?
> > Also, if you have a good idea to fix it easier, could you please let me
> > know?
>
> It was an open question.
> If everybody is fine with 255 ports maximum, let's keep it as is.
>
I think we are probably ok for now (and forseeable future) with 255 max.
However, if we did change it, I agree that in 2.0 is a very good time to do so.
Since we are expanding the field, rather than shrinking it, I don't see why we
can't just make the change at the ethdev level (and in libs API) in 2.0 and then in
later releases (e.g. 2.1) update the apps and examples to match. That way the
ABI stays the same from 2.0 onwards, and we don't have a huge amount of churn
changing it everywhere late in the 2.0 release cycle.
/Bruce
> -----Original Message----- > From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Tetsuya Mukawa > Sent: Wednesday, February 18, 2015 1:55 AM > To: Thomas Monjalon > Cc: dev@dpdk.org > Subject: Re: [dpdk-dev] [PATCH v8 04/14] eal/pci: Consolidate pci address comparison APIs > > On 2015/02/18 10:02, Thomas Monjalon wrote: > > 2015-02-17 15:14, Tetsuya Mukawa: > >> On 2015/02/17 9:44, Thomas Monjalon wrote: > >>> 2015-02-16 13:14, Tetsuya Mukawa: > >>>> @@ -356,13 +342,24 @@ pci_scan_one(int dev_pci_fd, struct pci_conf *conf) > >>>> } > >>>> else { > >>>> struct rte_pci_device *dev2 = NULL; > >>>> + int ret; > >>>> > >>>> TAILQ_FOREACH(dev2, &pci_device_list, next) { > >>>> - if (pci_addr_comparison(&dev->addr, &dev2->addr)) > >>>> + ret = eal_compare_pci_addr(&dev->addr, &dev2->addr); > >>>> + if (ret > 0) > >>>> continue; > >>>> - else { > >>>> + else if (ret < 0) { > >>>> TAILQ_INSERT_BEFORE(dev2, dev, next); > >>>> return 0; > >>>> + } else { /* already registered */ > >>>> + /* update pt_driver */ > >>>> + dev2->pt_driver = dev->pt_driver; > >>>> + dev2->max_vfs = dev->max_vfs; > >>>> + memmove(dev2->mem_resource, > >>>> + dev->mem_resource, > >>>> + sizeof(dev->mem_resource)); > >>>> + free(dev); > >>>> + return 0; > >>> Could you comment this "else part" please? > >> PCI device list is allocated when rte_eal_init() is called. At the > >> time, to fill pci device information, sysfs value is used. > >> If sysfs values written by kernel device driver will not be changed > >> by igb_uio, vfio or pci_uio_genereic, above code isn't needed. > >> But actually above values are changed or added by them. > >> > >> Here is a step to cause issue. > >> 1. Boot linux. > >> 2. Start DPDK application without any physical NIC ports. > >> - Here, some sysfs values are read, and store to pci device list. > >> 3. igb_uio starts managing a port. > >> - Here, some sysfs values are changed. > >> 4. Add a NIC port to DPDK application using hotplug functions. > >> - Here, we need to replace some values. > > I think that you are showing that something is wrongly designed in > > these EAL structures. I suggest to try cleaning this mess instead of workarounding. Hi Tetsuya, Thomas, I think that redesigning the EAL structures is beyond the scope of this patchset and should be undertaken as a separate task. I suspect there may be a problem in the original code when a device which was using a kernel driver is bound to igb_uio. The igb_uio driver adds /sys/bus/pci/devices/0000\:05\:00.0/max_vfs. Regards, Bernard. > > > > [...] > >>>> - if (memcmp(&uio_res->pci_addr, &dev->addr, sizeof(dev->addr))) > >>>> + if (eal_compare_pci_addr(&uio_res->pci_addr, &dev->addr)) > >>> Why memcmp is not sufficient to compare PCI addresses? > >>> The only exception I see is endianness for natural sorting. > >> Here is the definition. > >> > >> struct rte_pci_addr { > >> uint16_t domain; /**< Device domain */ > >> uint8_t bus; /**< Device bus */ > >> uint8_t devid; /**< Device ID */ > >> uint8_t function; /**< Device function. */ > >> }; > >> > >> But, sizeof(struct rte_pci_addr) will be 6. > >> If rte_pci_addr is allocated in a function without bzero, last 1 byte > >> may have some value. > >> As a result, memcmp may not work. To avoid such a case, I checked > >> like above. > > OK thanks. That's the kind of information which is valuable in a commit log. > > > > Sure I will add it. > > Thanks, > Tetsuya
Hi Bernard, 2015-02-18 10:26, Iremonger, Bernard: > From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Tetsuya Mukawa > > On 2015/02/18 10:02, Thomas Monjalon wrote: > > > 2015-02-17 15:14, Tetsuya Mukawa: > > >> On 2015/02/17 9:44, Thomas Monjalon wrote: > > >>> 2015-02-16 13:14, Tetsuya Mukawa: > > >>>> @@ -356,13 +342,24 @@ pci_scan_one(int dev_pci_fd, struct pci_conf *conf) > > >>>> } > > >>>> else { > > >>>> struct rte_pci_device *dev2 = NULL; > > >>>> + int ret; > > >>>> > > >>>> TAILQ_FOREACH(dev2, &pci_device_list, next) { > > >>>> - if (pci_addr_comparison(&dev->addr, &dev2->addr)) > > >>>> + ret = eal_compare_pci_addr(&dev->addr, &dev2->addr); > > >>>> + if (ret > 0) > > >>>> continue; > > >>>> - else { > > >>>> + else if (ret < 0) { > > >>>> TAILQ_INSERT_BEFORE(dev2, dev, next); > > >>>> return 0; > > >>>> + } else { /* already registered */ > > >>>> + /* update pt_driver */ > > >>>> + dev2->pt_driver = dev->pt_driver; > > >>>> + dev2->max_vfs = dev->max_vfs; > > >>>> + memmove(dev2->mem_resource, > > >>>> + dev->mem_resource, > > >>>> + sizeof(dev->mem_resource)); > > >>>> + free(dev); > > >>>> + return 0; > > >>> Could you comment this "else part" please? > > >> PCI device list is allocated when rte_eal_init() is called. At the > > >> time, to fill pci device information, sysfs value is used. > > >> If sysfs values written by kernel device driver will not be changed > > >> by igb_uio, vfio or pci_uio_genereic, above code isn't needed. > > >> But actually above values are changed or added by them. > > >> > > >> Here is a step to cause issue. > > >> 1. Boot linux. > > >> 2. Start DPDK application without any physical NIC ports. > > >> - Here, some sysfs values are read, and store to pci device list. > > >> 3. igb_uio starts managing a port. > > >> - Here, some sysfs values are changed. > > >> 4. Add a NIC port to DPDK application using hotplug functions. > > >> - Here, we need to replace some values. > > > > > > I think that you are showing that something is wrongly designed in > > > these EAL structures. I suggest to try cleaning this mess instead of workarounding. > > Hi Tetsuya, Thomas, > I think that redesigning the EAL structures is beyond the scope of this patchset and should be undertaken as a separate task. I strongly disagrees this opinion. We should never workaround design problems and add more complex/weird code. I think that this kind of consideration is the heart of some design problems we have to face today. Please let's stop adding some code which just works without thinking the whole design. > I suspect there may be a problem in the original code when a device which was using a kernel driver is bound to igb_uio. The igb_uio driver adds /sys/bus/pci/devices/0000\:05\:00.0/max_vfs.
On 2015/02/18 19:03, Bruce Richardson wrote: > On Wed, Feb 18, 2015 at 10:57:25AM +0100, Thomas Monjalon wrote: >> 2015-02-18 15:10, Tetsuya Mukawa: >>> On 2015/02/18 10:54, Tetsuya Mukawa wrote: >>>> On 2015/02/18 9:31, Thomas Monjalon wrote: >>>>> 2015-02-17 15:14, Tetsuya Mukawa: >>>>>> On 2015/02/17 9:36, Thomas Monjalon wrote: >>>>>>> 2015-02-16 13:14, Tetsuya Mukawa: >>>>>>> Is uint8_t sill a good size for hotpluggable virtual device ids? >>>>>> I am not sure it's enough, but uint8_t is widely used in "rte_ethdev.c" >>>>>> as port id. >>>>>> If someone reports it doesn't enough, I guess it will be the time to >>>>>> write a patch to change all uint_8 in one patch. >>>>> It's a big ABI breakage. So if we feel it's going to be required, >>>>> it's better to do it now in 2.0 release I think. >>>>> >>>>> Any opinion? >>>>> >>>> Hi Thomas, >>>> >>>> I agree with it. >>>> I will add an one more patch to change uint8_t to uint16_t. >>>> >>>> Thanks, >>>> Tetsuya >>>> >>> Hi Thomas, >>> >>> Could I make sure. >>> After changing uint8_t to uint16_t in "rte_ethdev.[ch]", must I also >>> need to change other applications and libraries that call ethdev APIs? >>> If so, I would not finish it by 23rd. >>> >>> I've counted how many lines call ethdev APIs that are related to port_id. >>> Could you please check an attached file? >>> It's over 1200 lines. Probably to fix one of caller, I will need to >>> check how port_id is used, and fix more related lines. So probably >>> thousands lines may need to be fixed. >>> >>> When is deadline for fixing this changing? >>> Also, if you have a good idea to fix it easier, could you please let me >>> know? >> It was an open question. >> If everybody is fine with 255 ports maximum, let's keep it as is. >> > I think we are probably ok for now (and forseeable future) with 255 max. > > However, if we did change it, I agree that in 2.0 is a very good time to do so. > Since we are expanding the field, rather than shrinking it, I don't see why we > can't just make the change at the ethdev level (and in libs API) in 2.0 and then in > later releases (e.g. 2.1) update the apps and examples to match. That way the > ABI stays the same from 2.0 onwards, and we don't have a huge amount of churn > changing it everywhere late in the 2.0 release cycle. Hi Bruce, Could you please check my RFC patch I will send soon? I wrote the patch like below. 1. Copy header file like below. $ cp lib/librte_ether/rte_ethdev.h lib/librte_ether/rte_ethdev_internal.h 2. Change "rte_ethdev.c" to include "rte_ethdev_internal.h" 3. Change type of port id in "rte_ethdev.c" and "rte_ethdev_internal.h". If the patch is OK, I wll send it with hotplug patches. Thanks, Tetsuya > /Bruce
> -----Original Message-----
> From: Thomas Monjalon [mailto:thomas.monjalon@6wind.com]
> Sent: Wednesday, February 18, 2015 10:33 AM
> To: Iremonger, Bernard
> Cc: Tetsuya Mukawa; dev@dpdk.org; ivan.boule@6wind.com
> Subject: Re: [dpdk-dev] [PATCH v8 04/14] eal/pci: Consolidate pci address comparison APIs
>
> Hi Bernard,
>
> 2015-02-18 10:26, Iremonger, Bernard:
> > From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Tetsuya Mukawa
> > > On 2015/02/18 10:02, Thomas Monjalon wrote:
> > > > 2015-02-17 15:14, Tetsuya Mukawa:
> > > >> On 2015/02/17 9:44, Thomas Monjalon wrote:
> > > >>> 2015-02-16 13:14, Tetsuya Mukawa:
> > > >>>> @@ -356,13 +342,24 @@ pci_scan_one(int dev_pci_fd, struct pci_conf *conf)
> > > >>>> }
> > > >>>> else {
> > > >>>> struct rte_pci_device *dev2 = NULL;
> > > >>>> + int ret;
> > > >>>>
> > > >>>> TAILQ_FOREACH(dev2, &pci_device_list, next) {
> > > >>>> - if (pci_addr_comparison(&dev->addr, &dev2->addr))
> > > >>>> + ret = eal_compare_pci_addr(&dev->addr, &dev2->addr);
> > > >>>> + if (ret > 0)
> > > >>>> continue;
> > > >>>> - else {
> > > >>>> + else if (ret < 0) {
> > > >>>> TAILQ_INSERT_BEFORE(dev2, dev, next);
> > > >>>> return 0;
> > > >>>> + } else { /* already registered */
> > > >>>> + /* update pt_driver */
> > > >>>> + dev2->pt_driver = dev->pt_driver;
> > > >>>> + dev2->max_vfs = dev->max_vfs;
> > > >>>> + memmove(dev2->mem_resource,
> > > >>>> + dev->mem_resource,
> > > >>>> + sizeof(dev->mem_resource));
> > > >>>> + free(dev);
> > > >>>> + return 0;
> > > >>> Could you comment this "else part" please?
> > > >> PCI device list is allocated when rte_eal_init() is called. At
> > > >> the time, to fill pci device information, sysfs value is used.
> > > >> If sysfs values written by kernel device driver will not be
> > > >> changed by igb_uio, vfio or pci_uio_genereic, above code isn't needed.
> > > >> But actually above values are changed or added by them.
> > > >>
> > > >> Here is a step to cause issue.
> > > >> 1. Boot linux.
> > > >> 2. Start DPDK application without any physical NIC ports.
> > > >> - Here, some sysfs values are read, and store to pci device list.
> > > >> 3. igb_uio starts managing a port.
> > > >> - Here, some sysfs values are changed.
> > > >> 4. Add a NIC port to DPDK application using hotplug functions.
> > > >> - Here, we need to replace some values.
> > > >
> > > > I think that you are showing that something is wrongly designed in
> > > > these EAL structures. I suggest to try cleaning this mess instead of workarounding.
> >
> > Hi Tetsuya, Thomas,
> > I think that redesigning the EAL structures is beyond the scope of this patchset and should be
> undertaken as a separate task.
>
> I strongly disagrees this opinion. We should never workaround design problems and add more
> complex/weird code.
> I think that this kind of consideration is the heart of some design problems we have to face today.
> Please let's stop adding some code which just works without thinking the whole design.
>
> > I suspect there may be a problem in the original code when a device which was using a kernel driver
> is bound to igb_uio. The igb_uio driver adds /sys/bus/pci/devices/0000\:05\:00.0/max_vfs.
Hi Tomas, Tetsuya,
In general, I agree that we should not workaround design problems.
In this case I don't think there is a problem with the rte_pci_device and pci_device_list structures.
The "already registered" device has been replaced. It would probably be cleaner to remove the "already registered" device from the list and then add the new device to the list rather than update the "already registered" device.
Regards,
Bernard.
On 2015/02/18 20:39, Iremonger, Bernard wrote: > >> -----Original Message----- >> From: Thomas Monjalon [mailto:thomas.monjalon@6wind.com] >> Sent: Wednesday, February 18, 2015 10:33 AM >> To: Iremonger, Bernard >> Cc: Tetsuya Mukawa; dev@dpdk.org; ivan.boule@6wind.com >> Subject: Re: [dpdk-dev] [PATCH v8 04/14] eal/pci: Consolidate pci address comparison APIs >> >> Hi Bernard, >> >> 2015-02-18 10:26, Iremonger, Bernard: >>> From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Tetsuya Mukawa >>>> On 2015/02/18 10:02, Thomas Monjalon wrote: >>>>> 2015-02-17 15:14, Tetsuya Mukawa: >>>>>> On 2015/02/17 9:44, Thomas Monjalon wrote: >>>>>>> 2015-02-16 13:14, Tetsuya Mukawa: >>>>>>>> @@ -356,13 +342,24 @@ pci_scan_one(int dev_pci_fd, struct pci_conf *conf) >>>>>>>> } >>>>>>>> else { >>>>>>>> struct rte_pci_device *dev2 = NULL; >>>>>>>> + int ret; >>>>>>>> >>>>>>>> TAILQ_FOREACH(dev2, &pci_device_list, next) { >>>>>>>> - if (pci_addr_comparison(&dev->addr, &dev2->addr)) >>>>>>>> + ret = eal_compare_pci_addr(&dev->addr, &dev2->addr); >>>>>>>> + if (ret > 0) >>>>>>>> continue; >>>>>>>> - else { >>>>>>>> + else if (ret < 0) { >>>>>>>> TAILQ_INSERT_BEFORE(dev2, dev, next); >>>>>>>> return 0; >>>>>>>> + } else { /* already registered */ >>>>>>>> + /* update pt_driver */ >>>>>>>> + dev2->pt_driver = dev->pt_driver; >>>>>>>> + dev2->max_vfs = dev->max_vfs; >>>>>>>> + memmove(dev2->mem_resource, >>>>>>>> + dev->mem_resource, >>>>>>>> + sizeof(dev->mem_resource)); >>>>>>>> + free(dev); >>>>>>>> + return 0; >>>>>>> Could you comment this "else part" please? >>>>>> PCI device list is allocated when rte_eal_init() is called. At >>>>>> the time, to fill pci device information, sysfs value is used. >>>>>> If sysfs values written by kernel device driver will not be >>>>>> changed by igb_uio, vfio or pci_uio_genereic, above code isn't needed. >>>>>> But actually above values are changed or added by them. >>>>>> >>>>>> Here is a step to cause issue. >>>>>> 1. Boot linux. >>>>>> 2. Start DPDK application without any physical NIC ports. >>>>>> - Here, some sysfs values are read, and store to pci device list. >>>>>> 3. igb_uio starts managing a port. >>>>>> - Here, some sysfs values are changed. >>>>>> 4. Add a NIC port to DPDK application using hotplug functions. >>>>>> - Here, we need to replace some values. >>>>> I think that you are showing that something is wrongly designed in >>>>> these EAL structures. I suggest to try cleaning this mess instead of workarounding. >>> Hi Tetsuya, Thomas, >>> I think that redesigning the EAL structures is beyond the scope of this patchset and should be >> undertaken as a separate task. >> >> I strongly disagrees this opinion. We should never workaround design problems and add more >> complex/weird code. >> I think that this kind of consideration is the heart of some design problems we have to face today. >> Please let's stop adding some code which just works without thinking the whole design. >> >>> I suspect there may be a problem in the original code when a device which was using a kernel driver >> is bound to igb_uio. The igb_uio driver adds /sys/bus/pci/devices/0000\:05\:00.0/max_vfs. > Hi Tomas, Tetsuya, > > In general, I agree that we should not workaround design problems. > In this case I don't think there is a problem with the rte_pci_device and pci_device_list structures. I agree with it. > The "already registered" device has been replaced. It would probably be cleaner to remove the "already registered" device from the list and then add the new device to the list rather than update the "already registered" device. > I guess "replacing" will not work, because rte_pci_device structure is also registered in rte_eth_dev structure. If we remove and free the pci device, I guess something goes wrong in ethdev library. Just removing is one more option, but it means there is a working pci device that is not registered in the pci_device_list. I guess it's weird. I still think updating may be correct behavior. The pci_device_list is used like below when rte_eal_init() is called. 1. When rte_eal_pci_init() is called, all pci devices are registered in the pci_device_list. 2. When rte_eal_dev_init() is called, dev_driver_list is traversed, and if a PCI device for a driver is found in the pci_device_list, init() of the driver is called. I guess it's not so strange design. But this design assumes pci_device_list is latest while matching a driver registered in dev_driver_list. Now, we have hotplug patch. I guess we should do same thing. Before matching, we should update the pci_device_list. Thanks, Tetsuya > Regards, > > Bernard. >
On Wed, Feb 18, 2015 at 07:58:06PM +0900, Tetsuya Mukawa wrote:
> On 2015/02/18 19:03, Bruce Richardson wrote:
> > On Wed, Feb 18, 2015 at 10:57:25AM +0100, Thomas Monjalon wrote:
> >> 2015-02-18 15:10, Tetsuya Mukawa:
> >>> On 2015/02/18 10:54, Tetsuya Mukawa wrote:
> >>>> On 2015/02/18 9:31, Thomas Monjalon wrote:
> >>>>> 2015-02-17 15:14, Tetsuya Mukawa:
> >>>>>> On 2015/02/17 9:36, Thomas Monjalon wrote:
> >>>>>>> 2015-02-16 13:14, Tetsuya Mukawa:
> >>>>>>> Is uint8_t sill a good size for hotpluggable virtual device ids?
> >>>>>> I am not sure it's enough, but uint8_t is widely used in "rte_ethdev.c"
> >>>>>> as port id.
> >>>>>> If someone reports it doesn't enough, I guess it will be the time to
> >>>>>> write a patch to change all uint_8 in one patch.
> >>>>> It's a big ABI breakage. So if we feel it's going to be required,
> >>>>> it's better to do it now in 2.0 release I think.
> >>>>>
> >>>>> Any opinion?
> >>>>>
> >>>> Hi Thomas,
> >>>>
> >>>> I agree with it.
> >>>> I will add an one more patch to change uint8_t to uint16_t.
> >>>>
> >>>> Thanks,
> >>>> Tetsuya
> >>>>
> >>> Hi Thomas,
> >>>
> >>> Could I make sure.
> >>> After changing uint8_t to uint16_t in "rte_ethdev.[ch]", must I also
> >>> need to change other applications and libraries that call ethdev APIs?
> >>> If so, I would not finish it by 23rd.
> >>>
> >>> I've counted how many lines call ethdev APIs that are related to port_id.
> >>> Could you please check an attached file?
> >>> It's over 1200 lines. Probably to fix one of caller, I will need to
> >>> check how port_id is used, and fix more related lines. So probably
> >>> thousands lines may need to be fixed.
> >>>
> >>> When is deadline for fixing this changing?
> >>> Also, if you have a good idea to fix it easier, could you please let me
> >>> know?
> >> It was an open question.
> >> If everybody is fine with 255 ports maximum, let's keep it as is.
> >>
> > I think we are probably ok for now (and forseeable future) with 255 max.
> >
> > However, if we did change it, I agree that in 2.0 is a very good time to do so.
> > Since we are expanding the field, rather than shrinking it, I don't see why we
> > can't just make the change at the ethdev level (and in libs API) in 2.0 and then in
> > later releases (e.g. 2.1) update the apps and examples to match. That way the
> > ABI stays the same from 2.0 onwards, and we don't have a huge amount of churn
> > changing it everywhere late in the 2.0 release cycle.
>
> Hi Bruce,
>
> Could you please check my RFC patch I will send soon?
> I wrote the patch like below.
>
> 1. Copy header file like below.
> $ cp lib/librte_ether/rte_ethdev.h lib/librte_ether/rte_ethdev_internal.h
> 2. Change "rte_ethdev.c" to include "rte_ethdev_internal.h"
> 3. Change type of port id in "rte_ethdev.c" and "rte_ethdev_internal.h".
>
> If the patch is OK, I wll send it with hotplug patches.
>
> Thanks,
> Tetsuya
>
>
Why the new ethdev internal file?
> -----Original Message-----
> From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Tetsuya Mukawa
> Sent: Wednesday, February 18, 2015 10:58 AM
> To: Richardson, Bruce; Thomas Monjalon
> Cc: dev@dpdk.org; Neil Horman
> Subject: Re: [dpdk-dev] [PATCH v8 03/14] eal/pci, ethdev: Remove assumption that port will not be
> detached
>
> On 2015/02/18 19:03, Bruce Richardson wrote:
> > On Wed, Feb 18, 2015 at 10:57:25AM +0100, Thomas Monjalon wrote:
> >> 2015-02-18 15:10, Tetsuya Mukawa:
> >>> On 2015/02/18 10:54, Tetsuya Mukawa wrote:
> >>>> On 2015/02/18 9:31, Thomas Monjalon wrote:
> >>>>> 2015-02-17 15:14, Tetsuya Mukawa:
> >>>>>> On 2015/02/17 9:36, Thomas Monjalon wrote:
> >>>>>>> 2015-02-16 13:14, Tetsuya Mukawa:
> >>>>>>> Is uint8_t sill a good size for hotpluggable virtual device ids?
> >>>>>> I am not sure it's enough, but uint8_t is widely used in "rte_ethdev.c"
> >>>>>> as port id.
> >>>>>> If someone reports it doesn't enough, I guess it will be the time
> >>>>>> to write a patch to change all uint_8 in one patch.
> >>>>> It's a big ABI breakage. So if we feel it's going to be required,
> >>>>> it's better to do it now in 2.0 release I think.
> >>>>>
> >>>>> Any opinion?
> >>>>>
> >>>> Hi Thomas,
> >>>>
> >>>> I agree with it.
> >>>> I will add an one more patch to change uint8_t to uint16_t.
> >>>>
> >>>> Thanks,
> >>>> Tetsuya
> >>>>
> >>> Hi Thomas,
> >>>
> >>> Could I make sure.
> >>> After changing uint8_t to uint16_t in "rte_ethdev.[ch]", must I also
> >>> need to change other applications and libraries that call ethdev APIs?
> >>> If so, I would not finish it by 23rd.
> >>>
> >>> I've counted how many lines call ethdev APIs that are related to port_id.
> >>> Could you please check an attached file?
> >>> It's over 1200 lines. Probably to fix one of caller, I will need to
> >>> check how port_id is used, and fix more related lines. So probably
> >>> thousands lines may need to be fixed.
> >>>
> >>> When is deadline for fixing this changing?
> >>> Also, if you have a good idea to fix it easier, could you please let
> >>> me know?
> >> It was an open question.
> >> If everybody is fine with 255 ports maximum, let's keep it as is.
> >>
> > I think we are probably ok for now (and forseeable future) with 255 max.
> >
> > However, if we did change it, I agree that in 2.0 is a very good time to do so.
> > Since we are expanding the field, rather than shrinking it, I don't
> > see why we can't just make the change at the ethdev level (and in libs
> > API) in 2.0 and then in later releases (e.g. 2.1) update the apps and
> > examples to match. That way the ABI stays the same from 2.0 onwards,
> > and we don't have a huge amount of churn changing it everywhere late in the 2.0 release cycle.
>
> Hi Bruce,
>
> Could you please check my RFC patch I will send soon?
> I wrote the patch like below.
>
> 1. Copy header file like below.
> $ cp lib/librte_ether/rte_ethdev.h lib/librte_ether/rte_ethdev_internal.h
> 2. Change "rte_ethdev.c" to include "rte_ethdev_internal.h"
> 3. Change type of port id in "rte_ethdev.c" and "rte_ethdev_internal.h".
>
> If the patch is OK, I wll send it with hotplug patches.
>
> Thanks,
> Tetsuya
>
>
> > /Bruce
>
Hi Tetsuya,
After this change there will be two header files with a lot of the same information.
lib/librte_ether/rte_ethdev.h
lib/librte_ether/rte_ethdev_internal.h
I don't think this is a good idea for maintenance in the future.
If 255 is ok for the foreseeable future, why change it now.
Regards,
Bernard.
On 2015/02/18 21:23, Bruce Richardson wrote:
> On Wed, Feb 18, 2015 at 07:58:06PM +0900, Tetsuya Mukawa wrote:
>> On 2015/02/18 19:03, Bruce Richardson wrote:
>>> On Wed, Feb 18, 2015 at 10:57:25AM +0100, Thomas Monjalon wrote:
>>>> 2015-02-18 15:10, Tetsuya Mukawa:
>>>>> On 2015/02/18 10:54, Tetsuya Mukawa wrote:
>>>>>> On 2015/02/18 9:31, Thomas Monjalon wrote:
>>>>>>> 2015-02-17 15:14, Tetsuya Mukawa:
>>>>>>>> On 2015/02/17 9:36, Thomas Monjalon wrote:
>>>>>>>>> 2015-02-16 13:14, Tetsuya Mukawa:
>>>>>>>>> Is uint8_t sill a good size for hotpluggable virtual device ids?
>>>>>>>> I am not sure it's enough, but uint8_t is widely used in "rte_ethdev.c"
>>>>>>>> as port id.
>>>>>>>> If someone reports it doesn't enough, I guess it will be the time to
>>>>>>>> write a patch to change all uint_8 in one patch.
>>>>>>> It's a big ABI breakage. So if we feel it's going to be required,
>>>>>>> it's better to do it now in 2.0 release I think.
>>>>>>>
>>>>>>> Any opinion?
>>>>>>>
>>>>>> Hi Thomas,
>>>>>>
>>>>>> I agree with it.
>>>>>> I will add an one more patch to change uint8_t to uint16_t.
>>>>>>
>>>>>> Thanks,
>>>>>> Tetsuya
>>>>>>
>>>>> Hi Thomas,
>>>>>
>>>>> Could I make sure.
>>>>> After changing uint8_t to uint16_t in "rte_ethdev.[ch]", must I also
>>>>> need to change other applications and libraries that call ethdev APIs?
>>>>> If so, I would not finish it by 23rd.
>>>>>
>>>>> I've counted how many lines call ethdev APIs that are related to port_id.
>>>>> Could you please check an attached file?
>>>>> It's over 1200 lines. Probably to fix one of caller, I will need to
>>>>> check how port_id is used, and fix more related lines. So probably
>>>>> thousands lines may need to be fixed.
>>>>>
>>>>> When is deadline for fixing this changing?
>>>>> Also, if you have a good idea to fix it easier, could you please let me
>>>>> know?
>>>> It was an open question.
>>>> If everybody is fine with 255 ports maximum, let's keep it as is.
>>>>
>>> I think we are probably ok for now (and forseeable future) with 255 max.
>>>
>>> However, if we did change it, I agree that in 2.0 is a very good time to do so.
>>> Since we are expanding the field, rather than shrinking it, I don't see why we
>>> can't just make the change at the ethdev level (and in libs API) in 2.0 and then in
>>> later releases (e.g. 2.1) update the apps and examples to match. That way the
>>> ABI stays the same from 2.0 onwards, and we don't have a huge amount of churn
>>> changing it everywhere late in the 2.0 release cycle.
>> Hi Bruce,
>>
>> Could you please check my RFC patch I will send soon?
>> I wrote the patch like below.
>>
>> 1. Copy header file like below.
>> $ cp lib/librte_ether/rte_ethdev.h lib/librte_ether/rte_ethdev_internal.h
>> 2. Change "rte_ethdev.c" to include "rte_ethdev_internal.h"
>> 3. Change type of port id in "rte_ethdev.c" and "rte_ethdev_internal.h".
>>
>> If the patch is OK, I wll send it with hotplug patches.
>>
>> Thanks,
>> Tetsuya
>>
>>
> Why the new ethdev internal file?
I guess some libraries that include "rte_ethdev.h". To compile these
libraries, I thought such a header was needed.
But, it seems it's not the time to change type of port_id.
I appreciate for your checking.
Tetsuya
On 2015/02/18 21:33, Iremonger, Bernard wrote: > >> -----Original Message----- >> From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Tetsuya Mukawa >> Sent: Wednesday, February 18, 2015 10:58 AM >> To: Richardson, Bruce; Thomas Monjalon >> Cc: dev@dpdk.org; Neil Horman >> Subject: Re: [dpdk-dev] [PATCH v8 03/14] eal/pci, ethdev: Remove assumption that port will not be >> detached >> >> On 2015/02/18 19:03, Bruce Richardson wrote: >>> On Wed, Feb 18, 2015 at 10:57:25AM +0100, Thomas Monjalon wrote: >>>> 2015-02-18 15:10, Tetsuya Mukawa: >>>>> On 2015/02/18 10:54, Tetsuya Mukawa wrote: >>>>>> On 2015/02/18 9:31, Thomas Monjalon wrote: >>>>>>> 2015-02-17 15:14, Tetsuya Mukawa: >>>>>>>> On 2015/02/17 9:36, Thomas Monjalon wrote: >>>>>>>>> 2015-02-16 13:14, Tetsuya Mukawa: >>>>>>>>> Is uint8_t sill a good size for hotpluggable virtual device ids? >>>>>>>> I am not sure it's enough, but uint8_t is widely used in "rte_ethdev.c" >>>>>>>> as port id. >>>>>>>> If someone reports it doesn't enough, I guess it will be the time >>>>>>>> to write a patch to change all uint_8 in one patch. >>>>>>> It's a big ABI breakage. So if we feel it's going to be required, >>>>>>> it's better to do it now in 2.0 release I think. >>>>>>> >>>>>>> Any opinion? >>>>>>> >>>>>> Hi Thomas, >>>>>> >>>>>> I agree with it. >>>>>> I will add an one more patch to change uint8_t to uint16_t. >>>>>> >>>>>> Thanks, >>>>>> Tetsuya >>>>>> >>>>> Hi Thomas, >>>>> >>>>> Could I make sure. >>>>> After changing uint8_t to uint16_t in "rte_ethdev.[ch]", must I also >>>>> need to change other applications and libraries that call ethdev APIs? >>>>> If so, I would not finish it by 23rd. >>>>> >>>>> I've counted how many lines call ethdev APIs that are related to port_id. >>>>> Could you please check an attached file? >>>>> It's over 1200 lines. Probably to fix one of caller, I will need to >>>>> check how port_id is used, and fix more related lines. So probably >>>>> thousands lines may need to be fixed. >>>>> >>>>> When is deadline for fixing this changing? >>>>> Also, if you have a good idea to fix it easier, could you please let >>>>> me know? >>>> It was an open question. >>>> If everybody is fine with 255 ports maximum, let's keep it as is. >>>> >>> I think we are probably ok for now (and forseeable future) with 255 max. >>> >>> However, if we did change it, I agree that in 2.0 is a very good time to do so. >>> Since we are expanding the field, rather than shrinking it, I don't >>> see why we can't just make the change at the ethdev level (and in libs >>> API) in 2.0 and then in later releases (e.g. 2.1) update the apps and >>> examples to match. That way the ABI stays the same from 2.0 onwards, >>> and we don't have a huge amount of churn changing it everywhere late in the 2.0 release cycle. >> Hi Bruce, >> >> Could you please check my RFC patch I will send soon? >> I wrote the patch like below. >> >> 1. Copy header file like below. >> $ cp lib/librte_ether/rte_ethdev.h lib/librte_ether/rte_ethdev_internal.h >> 2. Change "rte_ethdev.c" to include "rte_ethdev_internal.h" >> 3. Change type of port id in "rte_ethdev.c" and "rte_ethdev_internal.h". >> >> If the patch is OK, I wll send it with hotplug patches. >> >> Thanks, >> Tetsuya >> >> >>> /Bruce > Hi Tetsuya, > > After this change there will be two header files with a lot of the same information. > lib/librte_ether/rte_ethdev.h > lib/librte_ether/rte_ethdev_internal.h > I don't think this is a good idea for maintenance in the future. > If 255 is ok for the foreseeable future, why change it now. Hi Bernard, I appreciate for your checking. Agree, it will not be good to have almost same headers. Thanks, Tetsuya > Regards, > > Bernard. > >
This patch series adds a dynamic port hotplug framework to DPDK. With the patches, DPDK apps can attach or detach ports at runtime. The basic concept of the port hotplug is like followings. - DPDK apps must have responsibility to manage ports. DPDK apps only know which ports are attached or detached at the moment. The port hotplug framework is implemented to allow DPDK apps to manage ports. For example, when DPDK apps call port attach function, attached port number will be returned. Also, DPDK apps can detach port by port number. - Kernel support is needed for attaching or detaching physical device ports. To attach a new physical device port, the device will be recognized by userspace directly I/O framework in kernel at first. Then DPDK apps can call the port hotplug functions to attach ports. For detaching, steps are vice versa. - Before detach ports, ports must be stopped and closed. DPDK application must call rte_eth_dev_stop() and rte_eth_dev_close() before detaching ports. These function will call finalization codes of PMDs. But so far, no PMD frees all resources allocated by initialization. It means PMDs are needed to be fixed to support the port hotplug. 'RTE_PCI_DRV_DETACHABLE' is a new flag indicating a PMD supports detaching. Without this flag, detaching will be failed. - Mustn't affect legacy DPDK apps. No DPDK EAL behavior is changed, if the port hotplug functions are't called. So all legacy DPDK apps can still work without modifications. And a few limitations. - The port hotplug functions are not thread safe. DPDK apps should handle it. - Only support Linux and igb_uio so far. BSD and VFIO is not supported. I will send VFIO patches at least, but I don't have a plan to submit BSD patch so far. Here is port hotplug APIs. ------------------------------------------------------------------------------- /** * Attach a new device. * * @param devargs * A pointer to a strings array describing the new device * to be attached. The strings should be a pci address like * '0000:01:00.0' or virtual device name like 'eth_pcap0'. * @param port_id * A pointer to a port identifier actually attached. * @return * 0 on success and port_id is filled, negative on error */ int rte_eal_dev_attach(const char *devargs, uint8_t *port_id); /** * Detach a device. * * @param port_id * The port identifier of the device to detach. * @param addr * A pointer to a device name actually detached. * @return * 0 on success and devname is filled, negative on error */ int rte_eal_dev_detach(uint8_t port_id, char *devname); ------------------------------------------------------------------------------- This patch series are for DPDK EAL. To use port hotplug function by DPDK apps, each PMD should be fixed to support 'RTE_PCI_DRV_DETACHABLE' flag. Please check a patch for pcap PMD. Also, please check testpmd patch. It will show you how to fix your legacy applications to support port hotplug feature. PATCH v9 changes - Fix commit title. - Fix commit log. - Fix comments. - Define CONFIG_RTE_LIBRTE_EAL_HOTPLUG at the top of this patch series. - DEV_INVALID/VALID are removed. - DEV_DISCONNECTED is replaced by DEV_DETACHED. - DEV_CONNECTED is replaced by DEV_ATTACHED. - rte_eth_dev_allocate_new_port() is renamed to rte_eth_dev_find_free_port(). - rte_eth_dev_validate_port() is renamed to rte_eth_dev_is_valid_port(). - rte_eth_dev_is_valid_port() is changed not to handle log toggle. - eal_compare_pci_addr() is replaced by rte_eal_compare_pci_addr(). - rte_eth_dev_free() is replaced by rte_eth_dev_release_port(). - Add a function to create a unique device name. - Change parameter of pci_devuninit_t and rte_eth_dev_uninit. - Remove code that initiaize callback of ethdev from rte_eth_dev_uninit(). - Remove pci_unmap_device(). It will be implemented in later patch. - rte_eth_dev_check_detachable() is replaced by rte_eth_dev_is_detachable(). - strncpy() is replaced by strcpy(). - Implement pci_unmap_device() in this patch. - Remove "rte_dev_hotplug.h". - Remove needless "#ifdef". - Remove RTE_EAL_INVOKE_TYPE_PROBE/CLOSE. - RTE_ETH_DEV_PHYSICAL is replaced by RTE_ETH_DEV_PCI. - Use strcmp() instead of strncmp(). - Remove RTE_EAL_INVOKE_TYPE_PROBE/CLOSE. (Thanks to Thomas Monjalon) - Change definition of rte_dev_uninit_t. (Thanks to Thomas Monjalon and Maxime Leroy) - Add missing symbol in version map. (Thanks to Nail Horman) PATCH v8 changes - Fix Makefile and add version map file. - Add missing symbol in version map. - Fix pci_scan_one() to update sysfs values. (Thanks to Qiu, Michael and Iremonger, Bernard) - NONE_TRACE is replaced by NO_TRACE. - Fix typo. - Add size parameter to rte_eth_dev_save(). (Thanks to Iremonger, Bernard) PATCH v7 changes - Add a new section to programmer's guide. (Thanks to Iremonger, Bernard) - Fix port checking implementation of star_port(). - Fix typo of warning messages. - Add pt_driver checking to rte_eth_dev_check_detachable(). (Thanks to Qiu, Michael) PATCH v6 changes - Fix rte_eth_dev_uninit() to handle a return value of uninit function of PMD. To do this, below changes also be applied. - Fix a parameter of rte_eth_dev_free(). - Use rte_eth_dev structure as the paramter of rte_eth_dev_free(). PATCH v5 changes - Add runtime check passthrough driver type, like vfio-pci, igb_uio and uio_pci_generic. This was done by Qiu, Michael. Thanks a lot. - Change function names like below. - rte_eal_dev_find_and_invoke() to rte_eal_vdev_find_and_invoke(). - rte_eal_dev_invoke() to rte_eal_vdev_invoke(). - Add code to handle a return value of rte_eal_devargs_remove(). - Fix pci address format in rte_eal_dev_detach(). - Remove RTE_EAL_INVOKE_TYPE_UNKNOWN, because it's unused. - Change function definition of rte_eal_devargs_remove(). - Fix pci_unmap_device() to check pt_driver. - Fix return value of below functions. - rte_eth_dev_get_changed_port(). - rte_eth_dev_get_port_by_addr(). - Change paramters of rte_eth_dev_validate_port() to cleanup code. - Fix pci_scan_one to handle pt_driver correctly. (Thanks to Qiu, Michael for above suggestions) PATCH v4 changes - Merge patches to review easier. - Fix indent of 'if' statement. - Fix calculation method of eal_compare_pci_addr(). - Fix header file declaration. - Add header file to determine if hotplug can be enabled. (Thanks to Qiu, Michael) - Use braces with 'for' loop. - Add parameter checking. - Fix sanity check code - Fix comments of rte_eth_dev_type. - Change function names. (Thanks to Iremonger, Bernard) PATCH v3 changes: - Fix enum definition used in rte_ethdev.c. (Thanks to Zhang, Helin) PATCH v2 changes: - Replace rte_eal_dev_attach_pdev(), rte_eal_dev_detach_pdev, rte_eal_dev_attach_vdev() and rte_eal_dev_detach_vdev() to rte_eal_dev_attach() and rte_eal_dev_detach(). - Add parameter values checking. - Refashion a few functions. (Thanks to Iremonger, Bernard) PATCH v1 Changes: - Fix error checking code of librte_eth APIs. - Fix issue that port from pcap PMD cannot be detached correctly. - Fix issue that testpmd could hang after forwarding, if attaching and detaching is repeatedly. - Fix if-condition of rte_eth_dev_get_port_by_addr(). (Thanks to Mark Enright) RFC PATCH v2 Changes: - remove 'rte_eth_dev_validate_port()', and cleanup codes. Michael Qiu (2): eal_pci: Add flag to hold kernel driver type eal_pci: pci memory map work with driver type Tetsuya Mukawa (12): eal: Enable port Hotplug framework in Linux eal/pci,ethdev: Remove assumption that port will not be detached eal/pci: Consolidate pci address comparison APIs ethdev: Add rte_eth_dev_release_port to release specified port eal,ethdev: Add a function and function pointers to close ether device ethdev: Add functions that will be used by port hotplug functions eal/linux/pci: Add functions for unmapping igb_uio resources eal/pci: Add a function to remove the entry of devargs list eal/pci: Add probe and close functions of pci driver ethdev: Add one dev_type parameter to rte_eth_dev_allocate eal/pci: Add rte_eal_dev_attach/detach() functions doc: Add port hotplug framework section to programmers guide app/test/virtual_pmd.c | 2 +- config/common_bsdapp | 6 + config/common_linuxapp | 5 + doc/guides/prog_guide/index.rst | 1 + doc/guides/prog_guide/port_hotplug_framework.rst | 110 ++++++ lib/librte_eal/bsdapp/eal/eal_pci.c | 29 +- lib/librte_eal/common/eal_common_dev.c | 307 +++++++++++++++ lib/librte_eal/common/eal_common_devargs.c | 61 +++ lib/librte_eal/common/eal_common_pci.c | 100 ++++- lib/librte_eal/common/eal_private.h | 26 ++ lib/librte_eal/common/include/rte_dev.h | 33 ++ lib/librte_eal/common/include/rte_devargs.h | 21 + lib/librte_eal/common/include/rte_pci.h | 82 ++++ lib/librte_eal/linuxapp/eal/Makefile | 1 + lib/librte_eal/linuxapp/eal/eal_pci.c | 230 +++++++++-- lib/librte_eal/linuxapp/eal/eal_pci_init.h | 7 + lib/librte_eal/linuxapp/eal/eal_pci_uio.c | 67 +++- lib/librte_eal/linuxapp/eal/rte_eal_version.map | 2 + lib/librte_ether/rte_ethdev.c | 474 +++++++++++++++++------ lib/librte_ether/rte_ethdev.h | 149 ++++++- lib/librte_ether/rte_ether_version.map | 8 + lib/librte_pmd_af_packet/rte_eth_af_packet.c | 2 +- lib/librte_pmd_bond/rte_eth_bond_api.c | 2 +- lib/librte_pmd_pcap/rte_eth_pcap.c | 2 +- lib/librte_pmd_ring/rte_eth_ring.c | 2 +- lib/librte_pmd_xenvirt/rte_eth_xenvirt.c | 2 +- 26 files changed, 1563 insertions(+), 168 deletions(-) create mode 100644 doc/guides/prog_guide/port_hotplug_framework.rst -- 1.9.1
The patch adds CONFIG_RTE_LIBRTE_EAL_HOTPLUG in Linux and BSD configuration. So far, Hotplug functions only support linux. v9: - Move this patch at the top of this patch series. (Thanks to Thomas Monjalon) Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp> --- config/common_bsdapp | 6 ++++++ config/common_linuxapp | 5 +++++ 2 files changed, 11 insertions(+) diff --git a/config/common_bsdapp b/config/common_bsdapp index 8cfa4e6..d73cbba 100644 --- a/config/common_bsdapp +++ b/config/common_bsdapp @@ -116,6 +116,12 @@ CONFIG_RTE_LIBRTE_EAL_BSDAPP=y CONFIG_RTE_LIBRTE_EAL_LINUXAPP=n # +# Compile Environment Abstraction Layer to support hotplug +# So far, Hotplug functions only support linux +# +CONFIG_RTE_LIBRTE_EAL_HOTPLUG=n + +# # Compile Environment Abstraction Layer to support Vmware TSC map # CONFIG_RTE_LIBRTE_EAL_VMWARE_TSC_MAP_SUPPORT=y diff --git a/config/common_linuxapp b/config/common_linuxapp index db8332d..a677071 100644 --- a/config/common_linuxapp +++ b/config/common_linuxapp @@ -114,6 +114,11 @@ CONFIG_RTE_PCI_MAX_READ_REQUEST_SIZE=0 CONFIG_RTE_LIBRTE_EAL_LINUXAPP=y # +# Compile Environment Abstraction Layer to support hotplug +# +CONFIG_RTE_LIBRTE_EAL_HOTPLUG=y + +# # Compile Environment Abstraction Layer to support Vmware TSC map # CONFIG_RTE_LIBRTE_EAL_VMWARE_TSC_MAP_SUPPORT=y -- 1.9.1
From: Michael Qiu <michael.qiu@intel.com> Currently, dpdk has no ability to know which type of driver( vfio-pci/igb_uio/uio_pci_generic) the device used. It only can check whether vfio is enabled or not staticly. It really useful to have the flag, becasue different type need to handle differently in runtime. For example, pci memory map, pot hotplug, and so on. This patch add a flag field for pci device to solve above issue. Signed-off-by: Michael Qiu <michael.qiu@intel.com> Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp> --- lib/librte_eal/common/include/rte_pci.h | 8 +++++ lib/librte_eal/linuxapp/eal/eal_pci.c | 53 +++++++++++++++++++++++++++++++-- 2 files changed, 59 insertions(+), 2 deletions(-) diff --git a/lib/librte_eal/common/include/rte_pci.h b/lib/librte_eal/common/include/rte_pci.h index 66ed793..7b48b55 100644 --- a/lib/librte_eal/common/include/rte_pci.h +++ b/lib/librte_eal/common/include/rte_pci.h @@ -139,6 +139,13 @@ struct rte_pci_addr { struct rte_devargs; +enum rte_pt_driver { + RTE_PT_UNKNOWN = 0, + RTE_PT_IGB_UIO = 1, + RTE_PT_VFIO = 2, + RTE_PT_UIO_GENERIC = 3, +}; + /** * A structure describing a PCI device. */ @@ -152,6 +159,7 @@ struct rte_pci_device { uint16_t max_vfs; /**< sriov enable if not zero */ int numa_node; /**< NUMA node connection */ struct rte_devargs *devargs; /**< Device user arguments */ + enum rte_pt_driver pt_driver; /**< Driver of passthrough */ }; /** Any PCI device identifier (vendor, device, ...) */ diff --git a/lib/librte_eal/linuxapp/eal/eal_pci.c b/lib/librte_eal/linuxapp/eal/eal_pci.c index 15db9c4..e760452 100644 --- a/lib/librte_eal/linuxapp/eal/eal_pci.c +++ b/lib/librte_eal/linuxapp/eal/eal_pci.c @@ -97,6 +97,35 @@ error: return -1; } +static int +pci_get_kernel_driver_by_path(const char *filename, char *dri_name) +{ + int count; + char path[PATH_MAX]; + char *name; + + if (!filename || !dri_name) + return -1; + + count = readlink(filename, path, PATH_MAX); + if (count >= PATH_MAX) + return -1; + + /* For device does not have a driver */ + if (count < 0) + return 1; + + path[count] = '\0'; + + name = strrchr(path, '/'); + if (name) { + strncpy(dri_name, name + 1, strlen(name + 1) + 1); + return 0; + } + + return -1; +} + void * pci_find_max_end_va(void) { @@ -222,11 +251,12 @@ pci_scan_one(const char *dirname, uint16_t domain, uint8_t bus, char filename[PATH_MAX]; unsigned long tmp; struct rte_pci_device *dev; + char driver[PATH_MAX]; + int ret; dev = malloc(sizeof(*dev)); - if (dev == NULL) { + if (dev == NULL) return -1; - } memset(dev, 0, sizeof(*dev)); dev->addr.domain = domain; @@ -305,6 +335,25 @@ pci_scan_one(const char *dirname, uint16_t domain, uint8_t bus, return -1; } + /* parse driver */ + snprintf(filename, sizeof(filename), "%s/driver", dirname); + ret = pci_get_kernel_driver_by_path(filename, driver); + if (!ret) { + if (!strcmp(driver, "vfio-pci")) + dev->pt_driver = RTE_PT_VFIO; + else if (!strcmp(driver, "igb_uio")) + dev->pt_driver = RTE_PT_IGB_UIO; + else if (!strcmp(driver, "uio_pci_generic")) + dev->pt_driver = RTE_PT_UIO_GENERIC; + else + dev->pt_driver = RTE_PT_UNKNOWN; + } else if (ret < 0) { + RTE_LOG(ERR, EAL, "Fail to get kernel driver\n"); + free(dev); + return -1; + } else + dev->pt_driver = RTE_PT_UNKNOWN; + /* device is valid, add in list (sorted) */ if (TAILQ_EMPTY(&pci_device_list)) { TAILQ_INSERT_TAIL(&pci_device_list, dev, next); -- 1.9.1
From: Michael Qiu <michael.qiu@intel.com> With the driver type flag in struct rte_pci_dev, we do not need to always map uio devices with vfio related function when vfio enabled. Signed-off-by: Michael Qiu <michael.qiu@intel.com> Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp> --- lib/librte_eal/linuxapp/eal/eal_pci.c | 30 +++++++++++++++++------------- 1 file changed, 17 insertions(+), 13 deletions(-) diff --git a/lib/librte_eal/linuxapp/eal/eal_pci.c b/lib/librte_eal/linuxapp/eal/eal_pci.c index e760452..3c463b2 100644 --- a/lib/librte_eal/linuxapp/eal/eal_pci.c +++ b/lib/librte_eal/linuxapp/eal/eal_pci.c @@ -556,25 +556,29 @@ pci_config_space_set(struct rte_pci_device *dev) static int pci_map_device(struct rte_pci_device *dev) { - int ret, mapped = 0; + int ret = -1; /* try mapping the NIC resources using VFIO if it exists */ + switch (dev->pt_driver) { + case RTE_PT_VFIO: #ifdef VFIO_PRESENT - if (pci_vfio_is_enabled()) { - ret = pci_vfio_map_resource(dev); - if (ret == 0) - mapped = 1; - else if (ret < 0) - return ret; - } + if (pci_vfio_is_enabled()) + ret = pci_vfio_map_resource(dev); #endif - /* map resources for devices that use igb_uio */ - if (!mapped) { + break; + case RTE_PT_IGB_UIO: + case RTE_PT_UIO_GENERIC: + /* map resources for devices that use uio */ ret = pci_uio_map_resource(dev); - if (ret != 0) - return ret; + break; + default: + RTE_LOG(DEBUG, EAL, " Not managed by known pt driver," + " skipped\n"); + ret = 1; + break; } - return 0; + + return ret; } /* -- 1.9.1
To remove assumption, do like followings. This patch adds "RTE_PCI_DRV_DETACHABLE" to drv_flags of rte_pci_driver structure. The flags indicate the driver can detach devices at runtime. Also, remove assumption that port will not be detached. To remove the assumption. - Add 'attached' member to rte_eth_dev structure. This member is used for indicating the port is attached, or not. DEV_ATTACHED indicates a port is attached. DEV_DETACHED indicates a port is detached. - Add rte_eth_dev_allocate_new_port(). This function is used for allocating new port. v9: - DEV_INVALID/VALID are removed. - DEV_DISCONNECTED is replaced by DEV_DETACHED. - DEV_CONNECTED is replaced by DEV_ATTACHED. - rte_eth_dev_allocate_new_port() is renamed to rte_eth_dev_find_free_port(). - rte_eth_dev_validate_port() is renamed to rte_eth_dev_is_valid_port(). - rte_eth_dev_is_valid_port() is changed not to handle log toggle. - Fix commit log to describe DEV_ATACHED and DEV_DETACHED. (Thanks to Thomas Monjalon) v8: - NONE_TRACE is changed to NO_TRACE. (Thanks to Iremonger, Bernard) v5: - Change parameters of rte_eth_dev_validate_port() to cleanup code. v4: - Use braces with 'for' loop. - Fix indent of 'if' statement. Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp> --- lib/librte_eal/common/include/rte_pci.h | 2 + lib/librte_ether/rte_ethdev.c | 273 ++++++++++++++++++++------------ lib/librte_ether/rte_ethdev.h | 5 + 3 files changed, 177 insertions(+), 103 deletions(-) diff --git a/lib/librte_eal/common/include/rte_pci.h b/lib/librte_eal/common/include/rte_pci.h index 7b48b55..7f2d699 100644 --- a/lib/librte_eal/common/include/rte_pci.h +++ b/lib/librte_eal/common/include/rte_pci.h @@ -207,6 +207,8 @@ struct rte_pci_driver { #define RTE_PCI_DRV_FORCE_UNBIND 0x0004 /** Device driver supports link state interrupt */ #define RTE_PCI_DRV_INTR_LSC 0x0008 +/** Device driver supports detaching capability */ +#define RTE_PCI_DRV_DETACHABLE 0x0010 /**< Internal use only - Macro used by pci addr parsing functions **/ #define GET_PCIADDR_FIELD(in, fd, lim, dlm) \ diff --git a/lib/librte_ether/rte_ethdev.c b/lib/librte_ether/rte_ethdev.c index ea3a1fb..56797a5 100644 --- a/lib/librte_ether/rte_ethdev.c +++ b/lib/librte_ether/rte_ethdev.c @@ -175,6 +175,11 @@ enum { STAT_QMAP_RX }; +enum { + DEV_DETACHED = 0, + DEV_ATTACHED +}; + static inline void rte_eth_dev_data_alloc(void) { @@ -201,19 +206,34 @@ rte_eth_dev_allocated(const char *name) { unsigned i; - for (i = 0; i < nb_ports; i++) { - if (strcmp(rte_eth_devices[i].data->name, name) == 0) + for (i = 0; i < RTE_MAX_ETHPORTS; i++) { + if ((rte_eth_devices[i].attached == DEV_ATTACHED) && + strcmp(rte_eth_devices[i].data->name, name) == 0) return &rte_eth_devices[i]; } return NULL; } +static uint8_t +rte_eth_dev_find_free_port(void) +{ + unsigned i; + + for (i = 0; i < RTE_MAX_ETHPORTS; i++) { + if (rte_eth_devices[i].attached == DEV_DETACHED) + return i; + } + return RTE_MAX_ETHPORTS; +} + struct rte_eth_dev * rte_eth_dev_allocate(const char *name) { + uint8_t port_id; struct rte_eth_dev *eth_dev; - if (nb_ports == RTE_MAX_ETHPORTS) { + port_id = rte_eth_dev_find_free_port(); + if (port_id == RTE_MAX_ETHPORTS) { PMD_DEBUG_TRACE("Reached maximum number of Ethernet ports\n"); return NULL; } @@ -226,10 +246,12 @@ rte_eth_dev_allocate(const char *name) return NULL; } - eth_dev = &rte_eth_devices[nb_ports]; - eth_dev->data = &rte_eth_dev_data[nb_ports]; + eth_dev = &rte_eth_devices[port_id]; + eth_dev->data = &rte_eth_dev_data[port_id]; snprintf(eth_dev->data->name, sizeof(eth_dev->data->name), "%s", name); - eth_dev->data->port_id = nb_ports++; + eth_dev->data->port_id = port_id; + eth_dev->attached = DEV_ATTACHED; + nb_ports++; return eth_dev; } @@ -283,6 +305,7 @@ rte_eth_dev_init(struct rte_pci_driver *pci_drv, (unsigned) pci_dev->id.device_id); if (rte_eal_process_type() == RTE_PROC_PRIMARY) rte_free(eth_dev->data->dev_private); + eth_dev->attached = DEV_DETACHED; nb_ports--; return diag; } @@ -308,10 +331,20 @@ rte_eth_driver_register(struct eth_driver *eth_drv) rte_eal_pci_register(ð_drv->pci_drv); } +static int +rte_eth_dev_is_valid_port(uint8_t port_id) +{ + if (port_id >= RTE_MAX_ETHPORTS || + rte_eth_devices[port_id].attached != DEV_ATTACHED) + return 0; + else + return 1; +} + int rte_eth_dev_socket_id(uint8_t port_id) { - if (port_id >= nb_ports) + if (!rte_eth_dev_is_valid_port(port_id)) return -1; return rte_eth_devices[port_id].pci_dev->numa_node; } @@ -369,7 +402,7 @@ rte_eth_dev_rx_queue_start(uint8_t port_id, uint16_t rx_queue_id) * in a multi-process setup*/ PROC_PRIMARY_OR_ERR_RET(-E_RTE_SECONDARY); - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return -EINVAL; } @@ -395,7 +428,7 @@ rte_eth_dev_rx_queue_stop(uint8_t port_id, uint16_t rx_queue_id) * in a multi-process setup*/ PROC_PRIMARY_OR_ERR_RET(-E_RTE_SECONDARY); - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return -EINVAL; } @@ -421,7 +454,7 @@ rte_eth_dev_tx_queue_start(uint8_t port_id, uint16_t tx_queue_id) * in a multi-process setup*/ PROC_PRIMARY_OR_ERR_RET(-E_RTE_SECONDARY); - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return -EINVAL; } @@ -447,7 +480,7 @@ rte_eth_dev_tx_queue_stop(uint8_t port_id, uint16_t tx_queue_id) * in a multi-process setup*/ PROC_PRIMARY_OR_ERR_RET(-E_RTE_SECONDARY); - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return -EINVAL; } @@ -703,10 +736,11 @@ rte_eth_dev_configure(uint8_t port_id, uint16_t nb_rx_q, uint16_t nb_tx_q, * in a multi-process setup*/ PROC_PRIMARY_OR_ERR_RET(-E_RTE_SECONDARY); - if (port_id >= nb_ports || port_id >= RTE_MAX_ETHPORTS) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-EINVAL); } + dev = &rte_eth_devices[port_id]; FUNC_PTR_OR_ERR_RET(*dev->dev_ops->dev_infos_get, -ENOTSUP); @@ -888,10 +922,11 @@ rte_eth_dev_start(uint8_t port_id) * in a multi-process setup*/ PROC_PRIMARY_OR_ERR_RET(-E_RTE_SECONDARY); - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%" PRIu8 "\n", port_id); + if (!rte_eth_dev_is_valid_port(port_id)) { + PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-EINVAL); } + dev = &rte_eth_devices[port_id]; FUNC_PTR_OR_ERR_RET(*dev->dev_ops->dev_start, -ENOTSUP); @@ -923,10 +958,11 @@ rte_eth_dev_stop(uint8_t port_id) * in a multi-process setup*/ PROC_PRIMARY_OR_RET(); - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%" PRIu8 "\n", port_id); + if (!rte_eth_dev_is_valid_port(port_id)) { + PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return; } + dev = &rte_eth_devices[port_id]; FUNC_PTR_OR_RET(*dev->dev_ops->dev_stop); @@ -951,10 +987,11 @@ rte_eth_dev_set_link_up(uint8_t port_id) * in a multi-process setup*/ PROC_PRIMARY_OR_ERR_RET(-E_RTE_SECONDARY); - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return -EINVAL; } + dev = &rte_eth_devices[port_id]; FUNC_PTR_OR_ERR_RET(*dev->dev_ops->dev_set_link_up, -ENOTSUP); @@ -970,10 +1007,11 @@ rte_eth_dev_set_link_down(uint8_t port_id) * in a multi-process setup*/ PROC_PRIMARY_OR_ERR_RET(-E_RTE_SECONDARY); - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return -EINVAL; } + dev = &rte_eth_devices[port_id]; FUNC_PTR_OR_ERR_RET(*dev->dev_ops->dev_set_link_down, -ENOTSUP); @@ -989,7 +1027,7 @@ rte_eth_dev_close(uint8_t port_id) * in a multi-process setup*/ PROC_PRIMARY_OR_RET(); - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return; } @@ -1017,10 +1055,11 @@ rte_eth_rx_queue_setup(uint8_t port_id, uint16_t rx_queue_id, * in a multi-process setup*/ PROC_PRIMARY_OR_ERR_RET(-E_RTE_SECONDARY); - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-EINVAL); } + dev = &rte_eth_devices[port_id]; if (rx_queue_id >= dev->data->nb_rx_queues) { PMD_DEBUG_TRACE("Invalid RX queue_id=%d\n", rx_queue_id); @@ -1090,10 +1129,11 @@ rte_eth_tx_queue_setup(uint8_t port_id, uint16_t tx_queue_id, * in a multi-process setup*/ PROC_PRIMARY_OR_ERR_RET(-E_RTE_SECONDARY); - if (port_id >= RTE_MAX_ETHPORTS || port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-EINVAL); } + dev = &rte_eth_devices[port_id]; if (tx_queue_id >= dev->data->nb_tx_queues) { PMD_DEBUG_TRACE("Invalid TX queue_id=%d\n", tx_queue_id); @@ -1123,10 +1163,11 @@ rte_eth_promiscuous_enable(uint8_t port_id) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return; } + dev = &rte_eth_devices[port_id]; FUNC_PTR_OR_RET(*dev->dev_ops->promiscuous_enable); @@ -1139,10 +1180,11 @@ rte_eth_promiscuous_disable(uint8_t port_id) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return; } + dev = &rte_eth_devices[port_id]; FUNC_PTR_OR_RET(*dev->dev_ops->promiscuous_disable); @@ -1155,7 +1197,7 @@ rte_eth_promiscuous_get(uint8_t port_id) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return -1; } @@ -1169,10 +1211,11 @@ rte_eth_allmulticast_enable(uint8_t port_id) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return; } + dev = &rte_eth_devices[port_id]; FUNC_PTR_OR_RET(*dev->dev_ops->allmulticast_enable); @@ -1185,10 +1228,11 @@ rte_eth_allmulticast_disable(uint8_t port_id) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return; } + dev = &rte_eth_devices[port_id]; FUNC_PTR_OR_RET(*dev->dev_ops->allmulticast_disable); @@ -1201,7 +1245,7 @@ rte_eth_allmulticast_get(uint8_t port_id) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return -1; } @@ -1229,10 +1273,11 @@ rte_eth_link_get(uint8_t port_id, struct rte_eth_link *eth_link) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return; } + dev = &rte_eth_devices[port_id]; if (dev->data->dev_conf.intr_conf.lsc != 0) @@ -1249,10 +1294,11 @@ rte_eth_link_get_nowait(uint8_t port_id, struct rte_eth_link *eth_link) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return; } + dev = &rte_eth_devices[port_id]; if (dev->data->dev_conf.intr_conf.lsc != 0) @@ -1269,10 +1315,11 @@ rte_eth_stats_get(uint8_t port_id, struct rte_eth_stats *stats) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return; } + dev = &rte_eth_devices[port_id]; memset(stats, 0, sizeof(*stats)); @@ -1286,10 +1333,11 @@ rte_eth_stats_reset(uint8_t port_id) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return; } + dev = &rte_eth_devices[port_id]; FUNC_PTR_OR_RET(*dev->dev_ops->stats_reset); @@ -1307,10 +1355,11 @@ rte_eth_xstats_get(uint8_t port_id, struct rte_eth_xstats *xstats, uint64_t val; char *stats_ptr; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return -1; } + dev = &rte_eth_devices[port_id]; /* implemented by the driver */ @@ -1376,10 +1425,11 @@ rte_eth_xstats_reset(uint8_t port_id) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return; } + dev = &rte_eth_devices[port_id]; /* implemented by the driver */ @@ -1398,10 +1448,11 @@ set_queue_stats_mapping(uint8_t port_id, uint16_t queue_id, uint8_t stat_idx, { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return -ENODEV; } + dev = &rte_eth_devices[port_id]; FUNC_PTR_OR_ERR_RET(*dev->dev_ops->queue_stats_mapping_set, -ENOTSUP); @@ -1433,10 +1484,11 @@ rte_eth_dev_info_get(uint8_t port_id, struct rte_eth_dev_info *dev_info) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return; } + dev = &rte_eth_devices[port_id]; memset(dev_info, 0, sizeof(struct rte_eth_dev_info)); @@ -1453,10 +1505,11 @@ rte_eth_macaddr_get(uint8_t port_id, struct ether_addr *mac_addr) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return; } + dev = &rte_eth_devices[port_id]; ether_addr_copy(&dev->data->mac_addrs[0], mac_addr); } @@ -1467,7 +1520,7 @@ rte_eth_dev_get_mtu(uint8_t port_id, uint16_t *mtu) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-ENODEV); } @@ -1483,7 +1536,7 @@ rte_eth_dev_set_mtu(uint8_t port_id, uint16_t mtu) int ret; struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-ENODEV); } @@ -1503,10 +1556,11 @@ rte_eth_dev_vlan_filter(uint8_t port_id, uint16_t vlan_id, int on) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-ENODEV); } + dev = &rte_eth_devices[port_id]; if (! (dev->data->dev_conf.rxmode.hw_vlan_filter)) { PMD_DEBUG_TRACE("port %d: vlan-filtering disabled\n", port_id); @@ -1528,7 +1582,7 @@ rte_eth_dev_set_vlan_strip_on_queue(uint8_t port_id, uint16_t rx_queue_id, int o { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-ENODEV); } @@ -1550,7 +1604,7 @@ rte_eth_dev_set_vlan_ether_type(uint8_t port_id, uint16_t tpid) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-ENODEV); } @@ -1570,7 +1624,7 @@ rte_eth_dev_set_vlan_offload(uint8_t port_id, int offload_mask) int mask = 0; int cur, org = 0; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-ENODEV); } @@ -1615,7 +1669,7 @@ rte_eth_dev_get_vlan_offload(uint8_t port_id) struct rte_eth_dev *dev; int ret = 0; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-ENODEV); } @@ -1639,10 +1693,11 @@ rte_eth_dev_set_vlan_pvid(uint8_t port_id, uint16_t pvid, int on) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-ENODEV); } + dev = &rte_eth_devices[port_id]; FUNC_PTR_OR_ERR_RET(*dev->dev_ops->vlan_pvid_set, -ENOTSUP); (*dev->dev_ops->vlan_pvid_set)(dev, pvid, on); @@ -1657,7 +1712,7 @@ rte_eth_dev_fdir_add_signature_filter(uint8_t port_id, { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-ENODEV); } @@ -1691,7 +1746,7 @@ rte_eth_dev_fdir_update_signature_filter(uint8_t port_id, { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-ENODEV); } @@ -1725,7 +1780,7 @@ rte_eth_dev_fdir_remove_signature_filter(uint8_t port_id, { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-ENODEV); } @@ -1756,7 +1811,7 @@ rte_eth_dev_fdir_get_infos(uint8_t port_id, struct rte_eth_fdir *fdir) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-ENODEV); } @@ -1781,7 +1836,7 @@ rte_eth_dev_fdir_add_perfect_filter(uint8_t port_id, { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-ENODEV); } @@ -1821,7 +1876,7 @@ rte_eth_dev_fdir_update_perfect_filter(uint8_t port_id, { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-ENODEV); } @@ -1859,7 +1914,7 @@ rte_eth_dev_fdir_remove_perfect_filter(uint8_t port_id, { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-ENODEV); } @@ -1895,7 +1950,7 @@ rte_eth_dev_fdir_set_masks(uint8_t port_id, struct rte_fdir_masks *fdir_mask) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-ENODEV); } @@ -1915,7 +1970,7 @@ rte_eth_dev_flow_ctrl_get(uint8_t port_id, struct rte_eth_fc_conf *fc_conf) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-ENODEV); } @@ -1931,7 +1986,7 @@ rte_eth_dev_flow_ctrl_set(uint8_t port_id, struct rte_eth_fc_conf *fc_conf) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-ENODEV); } @@ -1951,7 +2006,7 @@ rte_eth_dev_priority_flow_ctrl_set(uint8_t port_id, struct rte_eth_pfc_conf *pfc { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-ENODEV); } @@ -2030,7 +2085,7 @@ rte_eth_dev_rss_reta_update(uint8_t port_id, struct rte_eth_dev *dev; int ret; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return -ENODEV; } @@ -2081,10 +2136,11 @@ rte_eth_dev_rss_hash_update(uint8_t port_id, struct rte_eth_rss_conf *rss_conf) struct rte_eth_dev *dev; uint16_t rss_hash_protos; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-ENODEV); } + rss_hash_protos = rss_conf->rss_hf; if ((rss_hash_protos != 0) && ((rss_hash_protos & ETH_RSS_PROTO_MASK) == 0)) { @@ -2103,10 +2159,11 @@ rte_eth_dev_rss_hash_conf_get(uint8_t port_id, { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-ENODEV); } + dev = &rte_eth_devices[port_id]; FUNC_PTR_OR_ERR_RET(*dev->dev_ops->rss_hash_conf_get, -ENOTSUP); return (*dev->dev_ops->rss_hash_conf_get)(dev, rss_conf); @@ -2118,7 +2175,7 @@ rte_eth_dev_udp_tunnel_add(uint8_t port_id, { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return -ENODEV; } @@ -2144,10 +2201,11 @@ rte_eth_dev_udp_tunnel_delete(uint8_t port_id, { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return -ENODEV; } + dev = &rte_eth_devices[port_id]; if (udp_tunnel == NULL) { @@ -2169,7 +2227,7 @@ rte_eth_led_on(uint8_t port_id) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-ENODEV); } @@ -2184,7 +2242,7 @@ rte_eth_led_off(uint8_t port_id) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-ENODEV); } @@ -2224,10 +2282,11 @@ rte_eth_dev_mac_addr_add(uint8_t port_id, struct ether_addr *addr, int index; uint64_t pool_mask; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-ENODEV); } + dev = &rte_eth_devices[port_id]; FUNC_PTR_OR_ERR_RET(*dev->dev_ops->mac_addr_add, -ENOTSUP); @@ -2275,10 +2334,11 @@ rte_eth_dev_mac_addr_remove(uint8_t port_id, struct ether_addr *addr) struct rte_eth_dev *dev; int index; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-ENODEV); } + dev = &rte_eth_devices[port_id]; FUNC_PTR_OR_ERR_RET(*dev->dev_ops->mac_addr_remove, -ENOTSUP); @@ -2309,7 +2369,7 @@ rte_eth_dev_set_vf_rxmode(uint8_t port_id, uint16_t vf, struct rte_eth_dev *dev; struct rte_eth_dev_info dev_info; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("set VF RX mode:Invalid port_id=%d\n", port_id); return (-ENODEV); @@ -2364,7 +2424,7 @@ rte_eth_dev_uc_hash_table_set(uint8_t port_id, struct ether_addr *addr, int ret; struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("unicast hash setting:Invalid port_id=%d\n", port_id); return (-ENODEV); @@ -2417,7 +2477,7 @@ rte_eth_dev_uc_all_hash_table_set(uint8_t port_id, uint8_t on) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("unicast hash setting:Invalid port_id=%d\n", port_id); return (-ENODEV); @@ -2436,7 +2496,7 @@ rte_eth_dev_set_vf_rx(uint8_t port_id,uint16_t vf, uint8_t on) struct rte_eth_dev *dev; struct rte_eth_dev_info dev_info; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-ENODEV); } @@ -2462,7 +2522,7 @@ rte_eth_dev_set_vf_tx(uint8_t port_id,uint16_t vf, uint8_t on) struct rte_eth_dev *dev; struct rte_eth_dev_info dev_info; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("set pool tx:Invalid port_id=%d\n", port_id); return (-ENODEV); } @@ -2487,7 +2547,7 @@ rte_eth_dev_set_vf_vlan_filter(uint8_t port_id, uint16_t vlan_id, { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("VF VLAN filter:invalid port id=%d\n", port_id); return (-ENODEV); @@ -2518,7 +2578,7 @@ int rte_eth_set_queue_rate_limit(uint8_t port_id, uint16_t queue_idx, struct rte_eth_dev_info dev_info; struct rte_eth_link link; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("set queue rate limit:invalid port id=%d\n", port_id); return -ENODEV; @@ -2555,7 +2615,7 @@ int rte_eth_set_vf_rate_limit(uint8_t port_id, uint16_t vf, uint16_t tx_rate, if (q_msk == 0) return 0; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("set VF rate limit:invalid port id=%d\n", port_id); return -ENODEV; @@ -2589,7 +2649,7 @@ rte_eth_mirror_rule_set(uint8_t port_id, { struct rte_eth_dev *dev = &rte_eth_devices[port_id]; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-ENODEV); } @@ -2630,7 +2690,7 @@ rte_eth_mirror_rule_reset(uint8_t port_id, uint8_t rule_id) { struct rte_eth_dev *dev = &rte_eth_devices[port_id]; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-ENODEV); } @@ -2655,10 +2715,11 @@ rte_eth_rx_burst(uint8_t port_id, uint16_t queue_id, { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return 0; } + dev = &rte_eth_devices[port_id]; FUNC_PTR_OR_ERR_RET(*dev->rx_pkt_burst, 0); if (queue_id >= dev->data->nb_rx_queues) { @@ -2675,10 +2736,11 @@ rte_eth_tx_burst(uint8_t port_id, uint16_t queue_id, { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return 0; } + dev = &rte_eth_devices[port_id]; FUNC_PTR_OR_ERR_RET(*dev->tx_pkt_burst, 0); @@ -2695,10 +2757,11 @@ rte_eth_rx_queue_count(uint8_t port_id, uint16_t queue_id) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return 0; } + dev = &rte_eth_devices[port_id]; FUNC_PTR_OR_ERR_RET(*dev->dev_ops->rx_queue_count, 0); return (*dev->dev_ops->rx_queue_count)(dev, queue_id); @@ -2709,10 +2772,11 @@ rte_eth_rx_descriptor_done(uint8_t port_id, uint16_t queue_id, uint16_t offset) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-ENODEV); } + dev = &rte_eth_devices[port_id]; FUNC_PTR_OR_ERR_RET(*dev->dev_ops->rx_descriptor_done, -ENOTSUP); return (*dev->dev_ops->rx_descriptor_done)( \ @@ -2730,7 +2794,8 @@ rte_eth_dev_callback_register(uint8_t port_id, if (!cb_fn) return (-EINVAL); - if (port_id >= nb_ports) { + + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-EINVAL); } @@ -2770,7 +2835,8 @@ rte_eth_dev_callback_unregister(uint8_t port_id, if (!cb_fn) return (-EINVAL); - if (port_id >= nb_ports) { + + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-EINVAL); } @@ -2830,7 +2896,7 @@ int rte_eth_dev_bypass_init(uint8_t port_id) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-ENODEV); } @@ -2850,7 +2916,7 @@ rte_eth_dev_bypass_state_show(uint8_t port_id, uint32_t *state) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-ENODEV); } @@ -2869,7 +2935,7 @@ rte_eth_dev_bypass_state_set(uint8_t port_id, uint32_t *new_state) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-ENODEV); } @@ -2889,7 +2955,7 @@ rte_eth_dev_bypass_event_show(uint8_t port_id, uint32_t event, uint32_t *state) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-ENODEV); } @@ -2909,7 +2975,7 @@ rte_eth_dev_bypass_event_store(uint8_t port_id, uint32_t event, uint32_t state) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-ENODEV); } @@ -2929,7 +2995,7 @@ rte_eth_dev_wd_timeout_store(uint8_t port_id, uint32_t timeout) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-ENODEV); } @@ -2949,7 +3015,7 @@ rte_eth_dev_bypass_ver_show(uint8_t port_id, uint32_t *ver) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-ENODEV); } @@ -2969,7 +3035,7 @@ rte_eth_dev_bypass_wd_timeout_show(uint8_t port_id, uint32_t *wd_timeout) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-ENODEV); } @@ -2989,7 +3055,7 @@ rte_eth_dev_bypass_wd_reset(uint8_t port_id) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-ENODEV); } @@ -3011,7 +3077,7 @@ rte_eth_dev_add_syn_filter(uint8_t port_id, { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return -ENODEV; } @@ -3026,7 +3092,7 @@ rte_eth_dev_remove_syn_filter(uint8_t port_id) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return -ENODEV; } @@ -3045,7 +3111,7 @@ rte_eth_dev_get_syn_filter(uint8_t port_id, if (filter == NULL || rx_queue == NULL) return -EINVAL; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return -ENODEV; } @@ -3061,10 +3127,11 @@ rte_eth_dev_add_2tuple_filter(uint8_t port_id, uint16_t index, { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return -ENODEV; } + if (filter->protocol != IPPROTO_TCP && filter->tcp_flags != 0){ PMD_DEBUG_TRACE("tcp flags is 0x%x, but the protocol value" @@ -3083,7 +3150,7 @@ rte_eth_dev_remove_2tuple_filter(uint8_t port_id, uint16_t index) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return -ENODEV; } @@ -3102,7 +3169,7 @@ rte_eth_dev_get_2tuple_filter(uint8_t port_id, uint16_t index, if (filter == NULL || rx_queue == NULL) return -EINVAL; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return -ENODEV; } @@ -3118,7 +3185,7 @@ rte_eth_dev_add_5tuple_filter(uint8_t port_id, uint16_t index, { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return -ENODEV; } @@ -3141,7 +3208,7 @@ rte_eth_dev_remove_5tuple_filter(uint8_t port_id, uint16_t index) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return -ENODEV; } @@ -3160,7 +3227,7 @@ rte_eth_dev_get_5tuple_filter(uint8_t port_id, uint16_t index, if (filter == NULL || rx_queue == NULL) return -EINVAL; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return -ENODEV; } @@ -3177,7 +3244,7 @@ rte_eth_dev_add_flex_filter(uint8_t port_id, uint16_t index, { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return -ENODEV; } @@ -3192,7 +3259,7 @@ rte_eth_dev_remove_flex_filter(uint8_t port_id, uint16_t index) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return -ENODEV; } @@ -3211,7 +3278,7 @@ rte_eth_dev_get_flex_filter(uint8_t port_id, uint16_t index, if (filter == NULL || rx_queue == NULL) return -EINVAL; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return -ENODEV; } @@ -3227,7 +3294,7 @@ rte_eth_dev_filter_supported(uint8_t port_id, enum rte_filter_type filter_type) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return -ENODEV; } @@ -3244,7 +3311,7 @@ rte_eth_dev_filter_ctrl(uint8_t port_id, enum rte_filter_type filter_type, { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return -ENODEV; } diff --git a/lib/librte_ether/rte_ethdev.h b/lib/librte_ether/rte_ethdev.h index 84160c3..768f372 100644 --- a/lib/librte_ether/rte_ethdev.h +++ b/lib/librte_ether/rte_ethdev.h @@ -1540,6 +1540,7 @@ struct rte_eth_dev { struct eth_dev_ops *dev_ops; /**< Functions exported by PMD */ struct rte_pci_device *pci_dev; /**< PCI info. supplied by probing */ struct rte_eth_dev_cb_list callbacks; /**< User application callbacks */ + uint8_t attached; /**< Flag indicating the port is attached */ }; struct rte_eth_dev_sriov { @@ -1605,6 +1606,10 @@ extern struct rte_eth_dev rte_eth_devices[]; * initialized by the [matching] Ethernet driver during the PCI probing phase. * All devices whose port identifier is in the range * [0, rte_eth_dev_count() - 1] can be operated on by network applications. + * immediately after invoking rte_eal_init(). + * If the application unplugs a port using hotplug function, The enabled port + * numbers may be noncontiguous. In the case, the applications need to manage + * enabled port by themselves. * * @return * - The total number of usable Ethernet devices. -- 1.9.1
This patch replaces pci_addr_comparison() and memcmp() of pci addresses by rte_eal_compare_pci_addr(). To compare PCI addresses, rte_eal_compare_pci_addr() doesn't use memcmp(). This is because sizeof(struct rte_pci_addr) returns 6, but actually this structure is like below. struct rte_pci_addr { uint16_t domain; /**< Device domain */ uint8_t bus; /**< Device bus */ uint8_t devid; /**< Device ID */ uint8_t function; /**< Device function. */ }; If the structure is dynamically allocated in a function without bzero, last 1 byte may have value. As a result, memcmp may not work. To avoid such a case, rte_eal_compare_pci_addr() compare following values. dev_addr = (addr->domain << 24) | (addr->bus << 16) | (addr->devid << 8) | addr->function; v9: - eal_compare_pci_addr() is replaced by rte_eal_compare_pci_addr(). - Fix commit log. (Thanks to Thomas Monjalon) v8: - Fix pci_scan_one() to update sysfs values. (Thanks to Qiu, Michael and Iremonger, Bernard) v5: - Fix pci_scan_one to handle pt_driver correctly. v4: - Fix calculation method of eal_compare_pci_addr(). - Add parameter checking. Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp> --- lib/librte_eal/bsdapp/eal/eal_pci.c | 29 ++++++++++++-------------- lib/librte_eal/common/eal_common_pci.c | 2 +- lib/librte_eal/common/include/rte_pci.h | 34 +++++++++++++++++++++++++++++++ lib/librte_eal/linuxapp/eal/eal_pci.c | 30 +++++++++++++-------------- lib/librte_eal/linuxapp/eal/eal_pci_uio.c | 2 +- 5 files changed, 63 insertions(+), 34 deletions(-) diff --git a/lib/librte_eal/bsdapp/eal/eal_pci.c b/lib/librte_eal/bsdapp/eal/eal_pci.c index 74ecce7..9193f80 100644 --- a/lib/librte_eal/bsdapp/eal/eal_pci.c +++ b/lib/librte_eal/bsdapp/eal/eal_pci.c @@ -270,20 +270,6 @@ pci_uio_map_resource(struct rte_pci_device *dev) return (0); } -/* Compare two PCI device addresses. */ -static int -pci_addr_comparison(struct rte_pci_addr *addr, struct rte_pci_addr *addr2) -{ - uint64_t dev_addr = (addr->domain << 24) + (addr->bus << 16) + (addr->devid << 8) + addr->function; - uint64_t dev_addr2 = (addr2->domain << 24) + (addr2->bus << 16) + (addr2->devid << 8) + addr2->function; - - if (dev_addr > dev_addr2) - return 1; - else - return 0; -} - - /* Scan one pci sysfs entry, and fill the devices list from it. */ static int pci_scan_one(int dev_pci_fd, struct pci_conf *conf) @@ -356,13 +342,24 @@ pci_scan_one(int dev_pci_fd, struct pci_conf *conf) } else { struct rte_pci_device *dev2 = NULL; + int ret; TAILQ_FOREACH(dev2, &pci_device_list, next) { - if (pci_addr_comparison(&dev->addr, &dev2->addr)) + ret = rte_eal_compare_pci_addr(&dev->addr, &dev2->addr); + if (ret > 0) continue; - else { + else if (ret < 0) { TAILQ_INSERT_BEFORE(dev2, dev, next); return 0; + } else { /* already registered */ + /* update pt_driver */ + dev2->pt_driver = dev->pt_driver; + dev2->max_vfs = dev->max_vfs; + memmove(dev2->mem_resource, + dev->mem_resource, + sizeof(dev->mem_resource)); + free(dev); + return 0; } } TAILQ_INSERT_TAIL(&pci_device_list, dev, next); diff --git a/lib/librte_eal/common/eal_common_pci.c b/lib/librte_eal/common/eal_common_pci.c index f3c7f71..bf2793f 100644 --- a/lib/librte_eal/common/eal_common_pci.c +++ b/lib/librte_eal/common/eal_common_pci.c @@ -93,7 +93,7 @@ static struct rte_devargs *pci_devargs_lookup(struct rte_pci_device *dev) if (devargs->type != RTE_DEVTYPE_BLACKLISTED_PCI && devargs->type != RTE_DEVTYPE_WHITELISTED_PCI) continue; - if (!memcmp(&dev->addr, &devargs->pci.addr, sizeof(dev->addr))) + if (!rte_eal_compare_pci_addr(&dev->addr, &devargs->pci.addr)) return devargs; } return NULL; diff --git a/lib/librte_eal/common/include/rte_pci.h b/lib/librte_eal/common/include/rte_pci.h index 7f2d699..c609ef3 100644 --- a/lib/librte_eal/common/include/rte_pci.h +++ b/lib/librte_eal/common/include/rte_pci.h @@ -269,6 +269,40 @@ eal_parse_pci_DomBDF(const char *input, struct rte_pci_addr *dev_addr) } #undef GET_PCIADDR_FIELD +/* Compare two PCI device addresses. */ +/** + * Utility function to compare two PCI device addresses. + * + * @param addr + * The PCI Bus-Device-Function address to compare + * @param addr2 + * The PCI Bus-Device-Function address to compare + * @return + * 0 on equal PCI address. + * Positive on addr is greater than addr2. + * Negative on addr is less than addr2, or error. + */ +static inline int +rte_eal_compare_pci_addr(struct rte_pci_addr *addr, struct rte_pci_addr *addr2) +{ + uint64_t dev_addr, dev_addr2; + + if ((addr == NULL) || (addr2 == NULL)) + return -1; + + dev_addr = (addr->domain << 24) | (addr->bus << 16) | + (addr->devid << 8) | addr->function; + dev_addr2 = (addr2->domain << 24) | (addr2->bus << 16) | + (addr2->devid << 8) | addr2->function; + + if (dev_addr > dev_addr2) + return 1; + else if (dev_addr < dev_addr2) + return -1; + else + return 0; +} + /** * Probe the PCI bus for registered drivers. * diff --git a/lib/librte_eal/linuxapp/eal/eal_pci.c b/lib/librte_eal/linuxapp/eal/eal_pci.c index 3c463b2..f593f2c 100644 --- a/lib/librte_eal/linuxapp/eal/eal_pci.c +++ b/lib/librte_eal/linuxapp/eal/eal_pci.c @@ -41,6 +41,7 @@ #include <rte_eal_memconfig.h> #include <rte_malloc.h> #include <rte_devargs.h> +#include <rte_memcpy.h> #include "rte_pci_dev_ids.h" #include "eal_filesystem.h" @@ -229,20 +230,6 @@ error: return -1; } -/* Compare two PCI device addresses. */ -static int -pci_addr_comparison(struct rte_pci_addr *addr, struct rte_pci_addr *addr2) -{ - uint64_t dev_addr = (addr->domain << 24) + (addr->bus << 16) + (addr->devid << 8) + addr->function; - uint64_t dev_addr2 = (addr2->domain << 24) + (addr2->bus << 16) + (addr2->devid << 8) + addr2->function; - - if (dev_addr > dev_addr2) - return 1; - else - return 0; -} - - /* Scan one pci sysfs entry, and fill the devices list from it. */ static int pci_scan_one(const char *dirname, uint16_t domain, uint8_t bus, @@ -360,13 +347,24 @@ pci_scan_one(const char *dirname, uint16_t domain, uint8_t bus, } else { struct rte_pci_device *dev2 = NULL; + int ret; TAILQ_FOREACH(dev2, &pci_device_list, next) { - if (pci_addr_comparison(&dev->addr, &dev2->addr)) + ret = rte_eal_compare_pci_addr(&dev->addr, &dev2->addr); + if (ret > 0) continue; - else { + else if (ret < 0) { TAILQ_INSERT_BEFORE(dev2, dev, next); return 0; + } else { /* already registered */ + /* update pt_driver */ + dev2->pt_driver = dev->pt_driver; + dev2->max_vfs = dev->max_vfs; + memmove(dev2->mem_resource, + dev->mem_resource, + sizeof(dev->mem_resource)); + free(dev); + return 0; } } TAILQ_INSERT_TAIL(&pci_device_list, dev, next); diff --git a/lib/librte_eal/linuxapp/eal/eal_pci_uio.c b/lib/librte_eal/linuxapp/eal/eal_pci_uio.c index e53f06b..d75eb59 100644 --- a/lib/librte_eal/linuxapp/eal/eal_pci_uio.c +++ b/lib/librte_eal/linuxapp/eal/eal_pci_uio.c @@ -123,7 +123,7 @@ pci_uio_map_secondary(struct rte_pci_device *dev) TAILQ_FOREACH(uio_res, pci_res_list, next) { /* skip this element if it doesn't match our PCI address */ - if (memcmp(&uio_res->pci_addr, &dev->addr, sizeof(dev->addr))) + if (rte_eal_compare_pci_addr(&uio_res->pci_addr, &dev->addr)) continue; for (i = 0; i != uio_res->nb_maps; i++) { -- 1.9.1
This patch adds rte_eth_dev_release_port(). The function is used for changing an attached status of the device that has specified name. v9: - rte_eth_dev_free() is replaced by rte_eth_dev_release_port(). (Thanks to Thomas Monjalon) v6: - Use rte_eth_dev structure as the paramter of rte_eth_dev_free(). v4: - Add parameter checking. Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp> --- lib/librte_ether/rte_ethdev.c | 11 +++++++++++ lib/librte_ether/rte_ethdev.h | 12 ++++++++++++ 2 files changed, 23 insertions(+) diff --git a/lib/librte_ether/rte_ethdev.c b/lib/librte_ether/rte_ethdev.c index 56797a5..be5aa18 100644 --- a/lib/librte_ether/rte_ethdev.c +++ b/lib/librte_ether/rte_ethdev.c @@ -255,6 +255,17 @@ rte_eth_dev_allocate(const char *name) return eth_dev; } +int +rte_eth_dev_release_port(struct rte_eth_dev *eth_dev) +{ + if (eth_dev == NULL) + return -EINVAL; + + eth_dev->attached = 0; + nb_ports--; + return 0; +} + static int rte_eth_dev_init(struct rte_pci_driver *pci_drv, struct rte_pci_device *pci_dev) diff --git a/lib/librte_ether/rte_ethdev.h b/lib/librte_ether/rte_ethdev.h index 768f372..28ecafd 100644 --- a/lib/librte_ether/rte_ethdev.h +++ b/lib/librte_ether/rte_ethdev.h @@ -1628,6 +1628,18 @@ extern uint8_t rte_eth_dev_count(void); */ struct rte_eth_dev *rte_eth_dev_allocate(const char *name); +/** + * Function for internal use by dummy drivers primarily, e.g. ring-based + * driver. + * Release the specified ethdev port. + * + * @param eth_dev + * The *eth_dev* pointer is the address of the *rte_eth_dev* structure. + * @return + * - 0 on success, negative on error + */ +int rte_eth_dev_release_port(struct rte_eth_dev *eth_dev); + struct eth_driver; /** * @internal -- 1.9.1
The patch adds function pointer to rte_pci_driver and eth_driver structure. These function pointers are used when ports are detached. Also, the patch adds rte_eth_dev_uninit(). So far, it's not called by anywhere, but it will be called when port hotplug function is implemented. v9: - Change parameter of pci_devuninit_t and rte_eth_dev_uninit. - Remove code that initiaize callback of ethdev from rte_eth_dev_uninit(). - Add a function to create a unique device name. (Thanks to Thomas Monjalon) v6: - Fix rte_eth_dev_uninit() to handle a return value of uninit function of PMD. v4: - Add parameter checking. - Change function names. Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp> --- lib/librte_eal/common/include/rte_pci.h | 6 ++++ lib/librte_ether/rte_ethdev.c | 62 +++++++++++++++++++++++++++++++-- lib/librte_ether/rte_ethdev.h | 24 +++++++++++++ 3 files changed, 90 insertions(+), 2 deletions(-) diff --git a/lib/librte_eal/common/include/rte_pci.h b/lib/librte_eal/common/include/rte_pci.h index c609ef3..376f66a 100644 --- a/lib/librte_eal/common/include/rte_pci.h +++ b/lib/librte_eal/common/include/rte_pci.h @@ -189,12 +189,18 @@ struct rte_pci_driver; typedef int (pci_devinit_t)(struct rte_pci_driver *, struct rte_pci_device *); /** + * Uninitialisation function for the driver called during hotplugging. + */ +typedef int (pci_devuninit_t)(struct rte_pci_device *); + +/** * A structure describing a PCI driver. */ struct rte_pci_driver { TAILQ_ENTRY(rte_pci_driver) next; /**< Next in list. */ const char *name; /**< Driver name. */ pci_devinit_t *devinit; /**< Device init. function. */ + pci_devuninit_t *devuninit; /**< Device uninit function. */ struct rte_pci_id *id_table; /**< ID table, NULL terminated. */ uint32_t drv_flags; /**< Flags contolling handling of device. */ }; diff --git a/lib/librte_ether/rte_ethdev.c b/lib/librte_ether/rte_ethdev.c index be5aa18..ef5d226 100644 --- a/lib/librte_ether/rte_ethdev.c +++ b/lib/librte_ether/rte_ethdev.c @@ -266,6 +266,24 @@ rte_eth_dev_release_port(struct rte_eth_dev *eth_dev) return 0; } +static inline int +rte_eth_dev_create_unique_device_name(char *name, + struct rte_pci_device *pci_dev) +{ + int ret; + + if ((name == NULL) || (pci_dev == NULL)) + return -EINVAL; + + ret = snprintf(name, RTE_ETH_NAME_MAX_LEN, "%d:%d.%d", + pci_dev->addr.bus, pci_dev->addr.devid, + pci_dev->addr.function); + if (ret < 0) + return ret; + + return 0; +} + static int rte_eth_dev_init(struct rte_pci_driver *pci_drv, struct rte_pci_device *pci_dev) @@ -279,8 +297,7 @@ rte_eth_dev_init(struct rte_pci_driver *pci_drv, eth_drv = (struct eth_driver *)pci_drv; /* Create unique Ethernet device name using PCI address */ - snprintf(ethdev_name, RTE_ETH_NAME_MAX_LEN, "%d:%d.%d", - pci_dev->addr.bus, pci_dev->addr.devid, pci_dev->addr.function); + rte_eth_dev_create_unique_device_name(ethdev_name, pci_dev); eth_dev = rte_eth_dev_allocate(ethdev_name); if (eth_dev == NULL) @@ -321,6 +338,46 @@ rte_eth_dev_init(struct rte_pci_driver *pci_drv, return diag; } +static int +rte_eth_dev_uninit(struct rte_pci_device *pci_dev) +{ + const struct eth_driver *eth_drv; + struct rte_eth_dev *eth_dev; + char ethdev_name[RTE_ETH_NAME_MAX_LEN]; + int ret; + + if (pci_dev == NULL) + return -EINVAL; + + /* Create unique Ethernet device name using PCI address */ + rte_eth_dev_create_unique_device_name(ethdev_name, pci_dev); + + eth_dev = rte_eth_dev_allocated(ethdev_name); + if (eth_dev == NULL) + return -ENODEV; + + eth_drv = (const struct eth_driver *)pci_dev->driver; + + /* Invoke PMD device uninit function */ + if (*eth_drv->eth_dev_uninit) { + ret = (*eth_drv->eth_dev_uninit)(eth_drv, eth_dev); + if (ret) + return ret; + } + + /* free ether device */ + rte_eth_dev_release_port(eth_dev); + + if (rte_eal_process_type() == RTE_PROC_PRIMARY) + rte_free(eth_dev->data->dev_private); + + eth_dev->pci_dev = NULL; + eth_dev->driver = NULL; + eth_dev->data = NULL; + + return 0; +} + /** * Register an Ethernet [Poll Mode] driver. * @@ -339,6 +396,7 @@ void rte_eth_driver_register(struct eth_driver *eth_drv) { eth_drv->pci_drv.devinit = rte_eth_dev_init; + eth_drv->pci_drv.devuninit = rte_eth_dev_uninit; rte_eal_pci_register(ð_drv->pci_drv); } diff --git a/lib/librte_ether/rte_ethdev.h b/lib/librte_ether/rte_ethdev.h index 28ecafd..f403780 100644 --- a/lib/librte_ether/rte_ethdev.h +++ b/lib/librte_ether/rte_ethdev.h @@ -1677,6 +1677,27 @@ typedef int (*eth_dev_init_t)(struct eth_driver *eth_drv, /** * @internal + * Finalization function of an Ethernet driver invoked for each matching + * Ethernet PCI device detected during the PCI closing phase. + * + * @param eth_drv + * The pointer to the [matching] Ethernet driver structure supplied by + * the PMD when it registered itself. + * @param eth_dev + * The *eth_dev* pointer is the address of the *rte_eth_dev* structure + * associated with the matching device and which have been [automatically] + * allocated in the *rte_eth_devices* array. + * @return + * - 0: Success, the device is properly finalized by the driver. + * In particular, the driver MUST free the *dev_ops* pointer + * of the *eth_dev* structure. + * - <0: Error code of the device initialization failure. + */ +typedef int (*eth_dev_uninit_t)(const struct eth_driver *eth_drv, + struct rte_eth_dev *eth_dev); + +/** + * @internal * The structure associated with a PMD Ethernet driver. * * Each Ethernet driver acts as a PCI driver and is represented by a generic @@ -1686,11 +1707,14 @@ typedef int (*eth_dev_init_t)(struct eth_driver *eth_drv, * * - The *eth_dev_init* function invoked for each matching PCI device. * + * - The *eth_dev_uninit* function invoked for each matching PCI device. + * * - The size of the private data to allocate for each matching device. */ struct eth_driver { struct rte_pci_driver pci_drv; /**< The PMD is also a PCI driver. */ eth_dev_init_t eth_dev_init; /**< Device init function. */ + eth_dev_uninit_t eth_dev_uninit; /**< Device uninit function. */ unsigned int dev_private_size; /**< Size of device private data. */ }; -- 1.9.1
The patch adds following functions. - rte_eth_dev_save() The function is used for saving current rte_eth_dev structures. - rte_eth_dev_get_changed_port() The function receives the rte_eth_dev structures, then compare these with current values to know which port is actually attached or detached. - rte_eth_dev_get_addr_by_port() The function returns a pci address of an ethdev specified by port identifier. - rte_eth_dev_get_port_by_addr() The function returns a port identifier of an ethdev specified by pci address. - rte_eth_dev_get_name_by_port() The function returns a unique identifier name of an ethdev specified by port identifier. - Add rte_eth_dev_is_detachable() The function returns whether a PMD supports detach function. Also, the patch changes scope of rte_eth_dev_allocated() to global. This function will be called by virtual PMDs to support port hotplug. So change scope of the function to global. v9: - rte_eth_dev_check_detachable() is replaced by rte_eth_dev_is_detachable(). - strncpy() is replaced by strcpy(). (Thanks to Thomas Monjalon) - Add missing symbol in version map. (Thanks to Nail Horman) v8: - Add size parameter to rte_eth_dev_save(). - Add missing symbol in version map. (Thanks to Qiu, Michael and Iremonger, Bernard) v7: - Add pt_driver checking to rte_eth_dev_check_detachable(). (Thanks to Qiu, Michael) v5: - Fix return value of below functions. rte_eth_dev_get_changed_port(). rte_eth_dev_get_port_by_addr(). v4: - Add parameter checking. v3: - Fix if-condition bug while comparing pci addresses. - Add error checking codes. Reported-by: Mark Enright <menrigh@brocade.com> Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp> --- lib/librte_ether/rte_ethdev.c | 103 ++++++++++++++++++++++++++++++++- lib/librte_ether/rte_ethdev.h | 83 ++++++++++++++++++++++++++ lib/librte_ether/rte_ether_version.map | 7 +++ 3 files changed, 192 insertions(+), 1 deletion(-) diff --git a/lib/librte_ether/rte_ethdev.c b/lib/librte_ether/rte_ethdev.c index ef5d226..3b64f3a 100644 --- a/lib/librte_ether/rte_ethdev.c +++ b/lib/librte_ether/rte_ethdev.c @@ -201,7 +201,7 @@ rte_eth_dev_data_alloc(void) RTE_MAX_ETHPORTS * sizeof(*rte_eth_dev_data)); } -static struct rte_eth_dev * +struct rte_eth_dev * rte_eth_dev_allocated(const char *name) { unsigned i; @@ -424,6 +424,107 @@ rte_eth_dev_count(void) return (nb_ports); } +int +rte_eth_dev_save(struct rte_eth_dev *devs, size_t size) +{ + if ((devs == NULL) || + (size != sizeof(struct rte_eth_dev) * RTE_MAX_ETHPORTS)) + return -EINVAL; + + /* save current rte_eth_devices */ + memcpy(devs, rte_eth_devices, size); + return 0; +} + +int +rte_eth_dev_get_changed_port(struct rte_eth_dev *devs, uint8_t *port_id) +{ + if ((devs == NULL) || (port_id == NULL)) + return -EINVAL; + + /* check which port was attached or detached */ + for (*port_id = 0; *port_id < RTE_MAX_ETHPORTS; (*port_id)++, devs++) { + if (rte_eth_devices[*port_id].attached ^ devs->attached) + return 0; + } + return -ENODEV; +} + +int +rte_eth_dev_get_addr_by_port(uint8_t port_id, struct rte_pci_addr *addr) +{ + if (!rte_eth_dev_is_valid_port(port_id)) { + PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + return -EINVAL; + } + + if (addr == NULL) { + PMD_DEBUG_TRACE("Null pointer is specified\n"); + return -EINVAL; + } + + *addr = rte_eth_devices[port_id].pci_dev->addr; + return 0; +} + +int +rte_eth_dev_get_port_by_addr(struct rte_pci_addr *addr, uint8_t *port_id) +{ + struct rte_pci_addr *tmp; + + if ((addr == NULL) || (port_id == NULL)) { + PMD_DEBUG_TRACE("Null pointer is specified\n"); + return -EINVAL; + } + + for (*port_id = 0; *port_id < RTE_MAX_ETHPORTS; (*port_id)++) { + if (!rte_eth_devices[*port_id].attached) + continue; + if (!rte_eth_devices[*port_id].pci_dev) + continue; + tmp = &rte_eth_devices[*port_id].pci_dev->addr; + if (rte_eal_compare_pci_addr(tmp, addr) == 0) + return 0; + } + return -ENODEV; +} + +int +rte_eth_dev_get_name_by_port(uint8_t port_id, char *name) +{ + char *tmp; + + if (!rte_eth_dev_is_valid_port(port_id)) { + PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + return -EINVAL; + } + + if (name == NULL) { + PMD_DEBUG_TRACE("Null pointer is specified\n"); + return -EINVAL; + } + + /* shouldn't check 'rte_eth_devices[i].data', + * because it might be overwritten by VDEV PMD */ + tmp = rte_eth_dev_data[port_id].name; + strcpy(name, tmp); + return 0; +} + +int +rte_eth_dev_is_detachable(uint8_t port_id) +{ + uint32_t drv_flags; + + if (port_id >= RTE_MAX_ETHPORTS) { + PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + return -EINVAL; + } + + drv_flags = rte_eth_devices[port_id].driver->pci_drv.drv_flags; + return !(drv_flags & RTE_PCI_DRV_DETACHABLE); +} + static int rte_eth_dev_rx_queue_config(struct rte_eth_dev *dev, uint16_t nb_queues) { diff --git a/lib/librte_ether/rte_ethdev.h b/lib/librte_ether/rte_ethdev.h index f403780..90b7f25 100644 --- a/lib/librte_ether/rte_ethdev.h +++ b/lib/librte_ether/rte_ethdev.h @@ -1617,6 +1617,89 @@ extern struct rte_eth_dev rte_eth_devices[]; extern uint8_t rte_eth_dev_count(void); /** + * Function for internal use by port hotplug functions. + * Copies current ethdev structures to the specified pointer. + * + * @param devs The pointer to the ethdev structures + * @param size The size of ethdev structures + * @return + * - 0 on success, negative on error + */ +extern int rte_eth_dev_save(struct rte_eth_dev *devs, size_t size); + +/** + * Function for internal use by port hotplug functions. + * Compare the specified ethdev structures with currents. Then + * if there is a port which status is changed, fill the specified pointer + * with the port id of that port. + * @param devs The pointer to the ethdev structures + * @param port_id The pointer to the port id + * @return + * - 0 on success, negative on error + */ +extern int rte_eth_dev_get_changed_port( + struct rte_eth_dev *devs, uint8_t *port_id); + +/** + * Function for internal use by port hotplug functions. + * Returns a pci address of a ethdev specified by port identifier. + * @param port_id + * The port identifier of the Ethernet device + * @param addr + * The pointer to the pci address + * @return + * - 0 on success, negative on error + */ +extern int rte_eth_dev_get_addr_by_port( + uint8_t port_id, struct rte_pci_addr *addr); + +/** + * Function for internal use by port hotplug functions. + * Returns a port identifier of a ethdev specified by pci address. + * @param addr + * The pointer to the pci address of the Ethernet device. + * @param port_id + * The pointer to the port identifier + * @return + * - 0 on success, negative on error + */ +extern int rte_eth_dev_get_port_by_addr( + struct rte_pci_addr *addr, uint8_t *port_id); + +/** + * Function for internal use by port hotplug functions. + * Returns a unique identifier name of a ethdev specified by port identifier. + * @param port_id + * The port identifier. + * @param name + * The pointer to the Unique identifier name for each Ethernet device + * @return + * - 0 on success, negative on error + */ +extern int rte_eth_dev_get_name_by_port(uint8_t port_id, char *name); + +/** + * Function for internal use by port hotplug functions. + * Check whether or not, a PMD that is handling the ethdev specified by port + * identifier can support detach function. + * @param port_id + * The port identifier + * @return + * - 0 on supporting detach function, negative on not supporting + */ +extern int rte_eth_dev_is_detachable(uint8_t port_id); + +/** + * Function for internal use by port hotplug functions. + * Returns a ethdev slot specified by the unique identifier name. + * @param name + * The pointer to the Unique identifier name for each Ethernet device + * @return + * - The pointer to the ethdev slot, on success. NULL on error + */ +extern struct rte_eth_dev *rte_eth_dev_allocated(const char *name); + +/** * Function for internal use by dummy drivers primarily, e.g. ring-based * driver. * Allocates a new ethdev slot for an ethernet device and returns the pointer diff --git a/lib/librte_ether/rte_ether_version.map b/lib/librte_ether/rte_ether_version.map index 7316530..1d039ed 100644 --- a/lib/librte_ether/rte_ether_version.map +++ b/lib/librte_ether/rte_ether_version.map @@ -109,6 +109,13 @@ DPDK_2.0 { rte_eth_tx_queue_setup; rte_eth_xstats_get; rte_eth_xstats_reset; + rte_eth_dev_allocated; + rte_eth_dev_is_detachable; + rte_eth_dev_get_name_by_port; + rte_eth_dev_get_addr_by_port; + rte_eth_dev_get_port_by_addr; + rte_eth_dev_get_changed_port; + rte_eth_dev_save; local: *; }; -- 1.9.1
The patch adds functions for unmapping igb_uio resources. The patch is only for Linux and igb_uio environment. VFIO and BSD are not supported. v9: - Remove "rte_dev_hotplug.h". - Remove needless "#ifdef". (Thanks to Thomas Monjalon and Neil Horman) - Remove pci_unmap_device(). It will be implemented in later patch. v8: - Fix typo. (Thanks to Iremonger, Bernard) v5: - Fix pci_unmap_device() to check pt_driver. v4: - Add parameter checking. - Add header file to determine if hotplug can be enabled. Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp> --- lib/librte_eal/linuxapp/eal/eal_pci.c | 17 ++++++++ lib/librte_eal/linuxapp/eal/eal_pci_init.h | 7 ++++ lib/librte_eal/linuxapp/eal/eal_pci_uio.c | 65 ++++++++++++++++++++++++++++++ 3 files changed, 89 insertions(+) diff --git a/lib/librte_eal/linuxapp/eal/eal_pci.c b/lib/librte_eal/linuxapp/eal/eal_pci.c index f593f2c..7349a60 100644 --- a/lib/librte_eal/linuxapp/eal/eal_pci.c +++ b/lib/librte_eal/linuxapp/eal/eal_pci.c @@ -167,6 +167,23 @@ pci_map_resource(void *requested_addr, int fd, off_t offset, size_t size) return mapaddr; } +/* unmap a particular resource */ +void +pci_unmap_resource(void *requested_addr, size_t size) +{ + if (requested_addr == NULL) + return; + + /* Unmap the PCI memory resource of device */ + if (munmap(requested_addr, size)) { + RTE_LOG(ERR, EAL, "%s(): cannot munmap(%p, 0x%lx): %s\n", + __func__, requested_addr, (unsigned long)size, + strerror(errno)); + } else + RTE_LOG(DEBUG, EAL, " PCI memory unmapped at %p\n", + requested_addr); +} + /* parse the "resource" sysfs file */ #define IORESOURCE_MEM 0x00000200 diff --git a/lib/librte_eal/linuxapp/eal/eal_pci_init.h b/lib/librte_eal/linuxapp/eal/eal_pci_init.h index 1070eb8..e2dd8a5 100644 --- a/lib/librte_eal/linuxapp/eal/eal_pci_init.h +++ b/lib/librte_eal/linuxapp/eal/eal_pci_init.h @@ -71,6 +71,13 @@ void *pci_map_resource(void *requested_addr, int fd, off_t offset, /* map IGB_UIO resource prototype */ int pci_uio_map_resource(struct rte_pci_device *dev); +void pci_unmap_resource(void *requested_addr, size_t size); + +#ifdef RTE_LIBRTE_EAL_HOTPLUG +/* unmap IGB_UIO resource prototype */ +void pci_uio_unmap_resource(struct rte_pci_device *dev); +#endif /* RTE_LIBRTE_EAL_HOTPLUG */ + #ifdef VFIO_PRESENT #define VFIO_MAX_GROUPS 64 diff --git a/lib/librte_eal/linuxapp/eal/eal_pci_uio.c b/lib/librte_eal/linuxapp/eal/eal_pci_uio.c index d75eb59..43f47dc 100644 --- a/lib/librte_eal/linuxapp/eal/eal_pci_uio.c +++ b/lib/librte_eal/linuxapp/eal/eal_pci_uio.c @@ -404,6 +404,71 @@ pci_uio_map_resource(struct rte_pci_device *dev) return 0; } +#ifdef RTE_LIBRTE_EAL_HOTPLUG +static void +pci_uio_unmap(struct mapped_pci_resource *uio_res) +{ + int i; + + if (uio_res == NULL) + return; + + for (i = 0; i != uio_res->nb_maps; i++) + pci_unmap_resource(uio_res->maps[i].addr, + (size_t)uio_res->maps[i].size); +} + +static struct mapped_pci_resource * +pci_uio_find_resource(struct rte_pci_device *dev) +{ + struct mapped_pci_resource *uio_res; + + if (dev == NULL) + return NULL; + + TAILQ_FOREACH(uio_res, pci_res_list, next) { + + /* skip this element if it doesn't match our PCI address */ + if (!rte_eal_compare_pci_addr(&uio_res->pci_addr, &dev->addr)) + return uio_res; + } + return NULL; +} + +/* unmap the PCI resource of a PCI device in virtual memory */ +void +pci_uio_unmap_resource(struct rte_pci_device *dev) +{ + struct mapped_pci_resource *uio_res; + + if (dev == NULL) + return; + + /* find an entry for the device */ + uio_res = pci_uio_find_resource(dev); + if (uio_res == NULL) + return; + + /* secondary processes - just free maps */ + if (rte_eal_process_type() != RTE_PROC_PRIMARY) + return pci_uio_unmap(uio_res); + + TAILQ_REMOVE(pci_res_list, uio_res, next); + + /* unmap all resources */ + pci_uio_unmap(uio_res); + + /* free uio resource */ + rte_free(uio_res); + + /* close fd if in primary process */ + close(dev->intr_handle.fd); + + dev->intr_handle.fd = -1; + dev->intr_handle.type = RTE_INTR_HANDLE_UNKNOWN; +} +#endif /* RTE_LIBRTE_EAL_HOTPLUG */ + /* * parse a sysfs file containing one integer value * different to the eal version, as it needs to work with 64-bit values -- 1.9.1
The function removes the specified devargs entry from devargs_list. Also, the patch adds sanity checking to rte_eal_devargs_add(). v5: - Change function definition of rte_eal_devargs_remove(). v4: - Fix sanity check code. Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp> --- lib/librte_eal/common/eal_common_devargs.c | 61 +++++++++++++++++++++++++++++ lib/librte_eal/common/include/rte_devargs.h | 21 ++++++++++ 2 files changed, 82 insertions(+) diff --git a/lib/librte_eal/common/eal_common_devargs.c b/lib/librte_eal/common/eal_common_devargs.c index 4c7d11a..d71faaa 100644 --- a/lib/librte_eal/common/eal_common_devargs.c +++ b/lib/librte_eal/common/eal_common_devargs.c @@ -44,6 +44,36 @@ struct rte_devargs_list devargs_list = TAILQ_HEAD_INITIALIZER(devargs_list); + +/* find a entry specified by pci address or device name */ +static struct rte_devargs * +rte_eal_devargs_find(enum rte_devtype devtype, void *args) +{ + struct rte_devargs *devargs; + + if (args == NULL) + return NULL; + + TAILQ_FOREACH(devargs, &devargs_list, next) { + switch (devtype) { + case RTE_DEVTYPE_WHITELISTED_PCI: + case RTE_DEVTYPE_BLACKLISTED_PCI: + if (rte_eal_compare_pci_addr( + &devargs->pci.addr, args) == 0) + goto found; + break; + case RTE_DEVTYPE_VIRTUAL: + if (memcmp(&devargs->virtual.drv_name, args, + strlen((char *)args)) == 0) + goto found; + break; + } + } + return NULL; +found: + return devargs; +} + /* store a whitelist parameter for later parsing */ int rte_eal_devargs_add(enum rte_devtype devtype, const char *devargs_str) @@ -87,6 +117,12 @@ rte_eal_devargs_add(enum rte_devtype devtype, const char *devargs_str) free(devargs); return -1; } + /* make sure there is no same entry */ + if (rte_eal_devargs_find(devtype, &devargs->pci.addr)) { + RTE_LOG(ERR, EAL, + "device already registered: <%s>\n", buf); + return -1; + } break; case RTE_DEVTYPE_VIRTUAL: /* save driver name */ @@ -98,6 +134,12 @@ rte_eal_devargs_add(enum rte_devtype devtype, const char *devargs_str) free(devargs); return -1; } + /* make sure there is no same entry */ + if (rte_eal_devargs_find(devtype, &devargs->virtual.drv_name)) { + RTE_LOG(ERR, EAL, + "device already registered: <%s>\n", buf); + return -1; + } break; } @@ -105,6 +147,25 @@ rte_eal_devargs_add(enum rte_devtype devtype, const char *devargs_str) return 0; } +/* remove it from the devargs_list */ +int +rte_eal_devargs_remove(enum rte_devtype devtype, void *args) +{ + struct rte_devargs *devargs; + + if (args == NULL) + return -EINVAL; + + devargs = rte_eal_devargs_find(devtype, args); + if (devargs == NULL) { + RTE_LOG(ERR, EAL, "device not found\n"); + return -ENODEV; + } + + TAILQ_REMOVE(&devargs_list, devargs, next); + return 0; +} + /* count the number of devices of a specified type */ unsigned int rte_eal_devargs_type_count(enum rte_devtype devtype) diff --git a/lib/librte_eal/common/include/rte_devargs.h b/lib/librte_eal/common/include/rte_devargs.h index 9f9c98f..6d9763b 100644 --- a/lib/librte_eal/common/include/rte_devargs.h +++ b/lib/librte_eal/common/include/rte_devargs.h @@ -123,6 +123,27 @@ extern struct rte_devargs_list devargs_list; int rte_eal_devargs_add(enum rte_devtype devtype, const char *devargs_str); /** + * Remove a device from the user device list + * + * For PCI devices, the format of arguments string is "PCI_ADDR". It shouldn't + * involve parameters for the device. Example: "08:00.1". + * + * For virtual devices, the format of arguments string is "DRIVER_NAME*". It + * shouldn't involve parameters for the device. Example: "eth_ring". The + * validity of the driver name is not checked by this function, it is done + * when closing the drivers. + * + * @param devtype + * The type of the device. + * @param name + * The name of the device. + * + * @return + * - 0 on success, negative on error + */ +int rte_eal_devargs_remove(enum rte_devtype devtype, void *args); + +/** * Count the number of user devices of a specified type * * @param devtype -- 1.9.1
- Add pci_close_all_drivers() The function tries to find a driver for the specified device, and then close the driver. - Add rte_eal_pci_probe_one() and rte_eal_pci_close_one() The functions are used for probe and close a device. First the function tries to find a device that has the specified PCI address. Then, probe or close the device. v9: - Fix commit title. - Remove RTE_EAL_INVOKE_TYPE_PROBE/CLOSE. (Thanks to Thomas Monjalon) - Implement pci_unmap_device() in this patch. v5: - Remove RTE_EAL_INVOKE_TYPE_UNKNOWN, because it's unused. v4: - Fix parameter checking. - Fix indent of 'if' statement. Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp> --- lib/librte_eal/common/eal_common_pci.c | 98 ++++++++++++++++++++++++++++++++- lib/librte_eal/common/eal_private.h | 15 +++++ lib/librte_eal/common/include/rte_pci.h | 32 +++++++++++ lib/librte_eal/linuxapp/eal/eal_pci.c | 94 +++++++++++++++++++++++++++++++ 4 files changed, 238 insertions(+), 1 deletion(-) diff --git a/lib/librte_eal/common/eal_common_pci.c b/lib/librte_eal/common/eal_common_pci.c index bf2793f..5b6b55d 100644 --- a/lib/librte_eal/common/eal_common_pci.c +++ b/lib/librte_eal/common/eal_common_pci.c @@ -108,7 +108,10 @@ static int pci_probe_all_drivers(struct rte_pci_device *dev) { struct rte_pci_driver *dr = NULL; - int rc; + int rc = 0; + + if (dev == NULL) + return -1; TAILQ_FOREACH(dr, &pci_driver_list, next) { rc = rte_eal_pci_probe_one_driver(dr, dev); @@ -123,6 +126,99 @@ pci_probe_all_drivers(struct rte_pci_device *dev) return 1; } +#ifdef RTE_LIBRTE_EAL_HOTPLUG +/* + * If vendor/device ID match, call the devuninit() function of all + * registered driver for the given device. Return -1 if initialization + * failed, return 1 if no driver is found for this device. + */ +static int +pci_close_all_drivers(struct rte_pci_device *dev) +{ + struct rte_pci_driver *dr = NULL; + int rc = 0; + + if (dev == NULL) + return -1; + + TAILQ_FOREACH(dr, &pci_driver_list, next) { + rc = rte_eal_pci_close_one_driver(dr, dev); + if (rc < 0) + /* negative value is an error */ + return -1; + if (rc > 0) + /* positive value means driver not found */ + continue; + return 0; + } + return 1; +} + +/* + * Find the pci device specified by pci address, then invoke probe function of + * the driver of the devive. + */ +int +rte_eal_pci_probe_one(struct rte_pci_addr *addr) +{ + struct rte_pci_device *dev = NULL; + int ret = 0; + + if (addr == NULL) + return -1; + + TAILQ_FOREACH(dev, &pci_device_list, next) { + if (rte_eal_compare_pci_addr(&dev->addr, addr)) + continue; + + ret = pci_probe_all_drivers(dev); + if (ret < 0) + goto err_return; + return 0; + } + return -1; + +err_return: + RTE_LOG(WARNING, EAL, "Requested device " PCI_PRI_FMT + " cannot be used\n", dev->addr.domain, dev->addr.bus, + dev->addr.devid, dev->addr.function); + return -1; +} + +/* + * Find the pci device specified by pci address, then invoke close function of + * the driver of the devive. + */ +int +rte_eal_pci_close_one(struct rte_pci_addr *addr) +{ + struct rte_pci_device *dev = NULL; + int ret = 0; + + if (addr == NULL) + return -1; + + TAILQ_FOREACH(dev, &pci_device_list, next) { + if (rte_eal_compare_pci_addr(&dev->addr, addr)) + continue; + + ret = pci_close_all_drivers(dev); + if (ret < 0) + goto err_return; + + TAILQ_REMOVE(&pci_device_list, dev, next); + return 0; + } + return -1; + +err_return: + RTE_LOG(WARNING, EAL, "Requested device " PCI_PRI_FMT + " cannot be used\n", dev->addr.domain, dev->addr.bus, + dev->addr.devid, dev->addr.function); + return -1; +} +#endif /* RTE_LIBRTE_EAL_HOTPLUG */ + /* * Scan the content of the PCI bus, and call the devinit() function for * all registered drivers that have a matching entry in its id_table diff --git a/lib/librte_eal/common/eal_private.h b/lib/librte_eal/common/eal_private.h index 159cd66..4acf5a0 100644 --- a/lib/librte_eal/common/eal_private.h +++ b/lib/librte_eal/common/eal_private.h @@ -165,6 +165,21 @@ int rte_eal_pci_probe_one_driver(struct rte_pci_driver *dr, struct rte_pci_device *dev); /** + * Munmap memory for single PCI device + * + * This function is private to EAL. + * + * @param dr + * The pointer to the pci driver structure + * @param dev + * The pointer to the pci device structure + * @return + * 0 on success, negative on error + */ +int rte_eal_pci_close_one_driver(struct rte_pci_driver *dr, + struct rte_pci_device *dev); + +/** * Init tail queues for non-EAL library structures. This is to allow * the rings, mempools, etc. lists to be shared among multiple processes * diff --git a/lib/librte_eal/common/include/rte_pci.h b/lib/librte_eal/common/include/rte_pci.h index 376f66a..4aed70a 100644 --- a/lib/librte_eal/common/include/rte_pci.h +++ b/lib/librte_eal/common/include/rte_pci.h @@ -322,6 +322,38 @@ rte_eal_compare_pci_addr(struct rte_pci_addr *addr, struct rte_pci_addr *addr2) */ int rte_eal_pci_probe(void); +#ifdef RTE_LIBRTE_EAL_HOTPLUG +/** + * Probe the single PCI device. + * + * Scan the content of the PCI bus, and find the pci device specified by pci + * address, then call the probe() function for registered driver that has a + * matching entry in its id_table for discovered device. + * + * @param addr + * The PCI Bus-Device-Function address to probe. + * @return + * - 0 on success. + * - Negative on error. + */ +int rte_eal_pci_probe_one(struct rte_pci_addr *addr); + +/** + * Close the single PCI device. + * + * Scan the content of the PCI bus, and find the pci device specified by pci + * address, then call the close() function for registered driver that has a + * matching entry in its id_table for discovered device. + * + * @param addr + * The PCI Bus-Device-Function address to close. + * @return + * - 0 on success. + * - Negative on error. + */ +int rte_eal_pci_close_one(struct rte_pci_addr *addr); +#endif /* RTE_LIBRTE_EAL_HOTPLUG */ + /** * Dump the content of the PCI bus. * diff --git a/lib/librte_eal/linuxapp/eal/eal_pci.c b/lib/librte_eal/linuxapp/eal/eal_pci.c index 7349a60..4bdf51b 100644 --- a/lib/librte_eal/linuxapp/eal/eal_pci.c +++ b/lib/librte_eal/linuxapp/eal/eal_pci.c @@ -596,6 +596,31 @@ pci_map_device(struct rte_pci_device *dev) return ret; } +#ifdef RTE_LIBRTE_EAL_HOTPLUG +static void +pci_unmap_device(struct rte_pci_device *dev) +{ + if (dev == NULL) + return; + + /* try unmapping the NIC resources using VFIO if it exists */ + switch (dev->pt_driver) { + case RTE_PT_VFIO: + RTE_LOG(ERR, EAL, "Hotplug doesn't support vfio yet\n"); + break; + case RTE_PT_IGB_UIO: + case RTE_PT_UIO_GENERIC: + /* unmap resources for devices that use uio */ + pci_uio_unmap_resource(dev); + break; + default: + RTE_LOG(DEBUG, EAL, " Not managed by known pt driver," + " skipped\n"); + break; + } +} +#endif /* RTE_LIBRTE_EAL_HOTPLUG */ + /* * If vendor/device ID match, call the devinit() function of the * driver. @@ -667,6 +692,75 @@ rte_eal_pci_probe_one_driver(struct rte_pci_driver *dr, struct rte_pci_device *d return 1; } +#ifdef RTE_LIBRTE_EAL_HOTPLUG +/* + * If vendor/device ID match, call the devuninit() function of the + * driver. + */ +int +rte_eal_pci_close_one_driver(struct rte_pci_driver *dr, + struct rte_pci_device *dev) +{ + struct rte_pci_id *id_table; + + if ((dr == NULL) || (dev == NULL)) + return -EINVAL; + + for (id_table = dr->id_table ; id_table->vendor_id != 0; id_table++) { + + /* check if device's identifiers match the driver's ones */ + if (id_table->vendor_id != dev->id.vendor_id && + id_table->vendor_id != PCI_ANY_ID) + continue; + if (id_table->device_id != dev->id.device_id && + id_table->device_id != PCI_ANY_ID) + continue; + if (id_table->subsystem_vendor_id != + dev->id.subsystem_vendor_id && + id_table->subsystem_vendor_id != PCI_ANY_ID) + continue; + if (id_table->subsystem_device_id != + dev->id.subsystem_device_id && + id_table->subsystem_device_id != PCI_ANY_ID) + continue; + + struct rte_pci_addr *loc = &dev->addr; + + RTE_LOG(DEBUG, EAL, + "PCI device "PCI_PRI_FMT" on NUMA socket %i\n", + loc->domain, loc->bus, loc->devid, + loc->function, dev->numa_node); + + RTE_LOG(DEBUG, EAL, " remove driver: %x:%x %s\n", + dev->id.vendor_id, dev->id.device_id, + dr->name); + + /* call the driver devuninit() function */ + if (dr->devuninit && (dr->devuninit(dev) < 0)) + return -1; /* negative value is an error */ + + /* clear driver structure */ + dev->driver = NULL; + + if (dr->drv_flags & RTE_PCI_DRV_NEED_MAPPING) + /* unmap resources for devices that use igb_uio */ + pci_unmap_device(dev); + + return 0; + } + /* return positive value if driver is not found */ + return 1; +} +#else /* RTE_LIBRTE_EAL_HOTPLUG */ +int +rte_eal_pci_close_one_driver(struct rte_pci_driver *dr __rte_unused, + struct rte_pci_device *dev __rte_unused) +{ + RTE_LOG(ERR, EAL, "Hotplug support isn't enabled\n"); + return -1; +} +#endif /* RTE_LIBRTE_EAL_HOTPLUG */ + /* Init the PCI EAL subsystem */ int rte_eal_pci_init(void) -- 1.9.1
This new parameter is needed to keep device type like PCI or virtual. Port detaching processes are different between PCI device and virtual device. RTE_ETH_DEV_PCI indicates device type is PCI. RTE_ETH_DEV_VIRTUAL indicates device is virtual. v9: - Fix commit log. - RTE_ETH_DEV_PHYSICAL is replaced by RTE_ETH_DEV_PCI. (Thanks to Thomas Monjalon) v8: - NONE_TRACE is replaced by NO_TRACE. - Add missing symbol in version map. (Thanks to Qiu, Michael and Iremonger, Bernard) v4: - Fix comments of rte_eth_dev_type. Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp> --- app/test/virtual_pmd.c | 2 +- lib/librte_ether/rte_ethdev.c | 25 +++++++++++++++++++++++-- lib/librte_ether/rte_ethdev.h | 25 ++++++++++++++++++++++++- lib/librte_ether/rte_ether_version.map | 1 + lib/librte_pmd_af_packet/rte_eth_af_packet.c | 2 +- lib/librte_pmd_bond/rte_eth_bond_api.c | 2 +- lib/librte_pmd_pcap/rte_eth_pcap.c | 2 +- lib/librte_pmd_ring/rte_eth_ring.c | 2 +- lib/librte_pmd_xenvirt/rte_eth_xenvirt.c | 2 +- 9 files changed, 54 insertions(+), 9 deletions(-) diff --git a/app/test/virtual_pmd.c b/app/test/virtual_pmd.c index 9fac95d..c02644a 100644 --- a/app/test/virtual_pmd.c +++ b/app/test/virtual_pmd.c @@ -556,7 +556,7 @@ virtual_ethdev_create(const char *name, struct ether_addr *mac_addr, goto err; /* reserve an ethdev entry */ - eth_dev = rte_eth_dev_allocate(name); + eth_dev = rte_eth_dev_allocate(name, RTE_ETH_DEV_PCI); if (eth_dev == NULL) goto err; diff --git a/lib/librte_ether/rte_ethdev.c b/lib/librte_ether/rte_ethdev.c index 3b64f3a..201c04a 100644 --- a/lib/librte_ether/rte_ethdev.c +++ b/lib/librte_ether/rte_ethdev.c @@ -227,7 +227,7 @@ rte_eth_dev_find_free_port(void) } struct rte_eth_dev * -rte_eth_dev_allocate(const char *name) +rte_eth_dev_allocate(const char *name, enum rte_eth_dev_type type) { uint8_t port_id; struct rte_eth_dev *eth_dev; @@ -251,6 +251,7 @@ rte_eth_dev_allocate(const char *name) snprintf(eth_dev->data->name, sizeof(eth_dev->data->name), "%s", name); eth_dev->data->port_id = port_id; eth_dev->attached = DEV_ATTACHED; + eth_dev->dev_type = type; nb_ports++; return eth_dev; } @@ -262,6 +263,7 @@ rte_eth_dev_release_port(struct rte_eth_dev *eth_dev) return -EINVAL; eth_dev->attached = 0; + eth_dev->dev_type = RTE_ETH_DEV_UNKNOWN; nb_ports--; return 0; } @@ -299,7 +301,7 @@ rte_eth_dev_init(struct rte_pci_driver *pci_drv, /* Create unique Ethernet device name using PCI address */ rte_eth_dev_create_unique_device_name(ethdev_name, pci_dev); - eth_dev = rte_eth_dev_allocate(ethdev_name); + eth_dev = rte_eth_dev_allocate(ethdev_name, RTE_ETH_DEV_PCI); if (eth_dev == NULL) return -ENOMEM; @@ -424,6 +426,14 @@ rte_eth_dev_count(void) return (nb_ports); } +enum rte_eth_dev_type +rte_eth_dev_get_device_type(uint8_t port_id) +{ + if (!rte_eth_dev_is_valid_port(port_id)) + return -1; + return rte_eth_devices[port_id].dev_type; +} + int rte_eth_dev_save(struct rte_eth_dev *devs, size_t size) { @@ -521,6 +531,17 @@ rte_eth_dev_is_detachable(uint8_t port_id) return -EINVAL; } + if (rte_eth_devices[port_id].dev_type == RTE_ETH_DEV_PCI) { + switch (rte_eth_devices[port_id].pci_dev->pt_driver) { + case RTE_PT_IGB_UIO: + case RTE_PT_UIO_GENERIC: + break; + case RTE_PT_VFIO: + default: + return -ENOTSUP; + } + } + drv_flags = rte_eth_devices[port_id].driver->pci_drv.drv_flags; return !(drv_flags & RTE_PCI_DRV_DETACHABLE); } diff --git a/lib/librte_ether/rte_ethdev.h b/lib/librte_ether/rte_ethdev.h index 90b7f25..4f66bb6 100644 --- a/lib/librte_ether/rte_ethdev.h +++ b/lib/librte_ether/rte_ethdev.h @@ -1523,6 +1523,17 @@ struct eth_dev_ops { }; /** + * The eth device type + */ +enum rte_eth_dev_type { + RTE_ETH_DEV_UNKNOWN, /**< unknown device type */ + RTE_ETH_DEV_PCI, + /**< Physical function and Virtual function of PCI devices */ + RTE_ETH_DEV_VIRTUAL, /**< non hardware device */ + RTE_ETH_DEV_MAX /**< max value of this enum */ +}; + +/** * @internal * The generic data structure associated with each ethernet device. * @@ -1541,6 +1552,7 @@ struct rte_eth_dev { struct rte_pci_device *pci_dev; /**< PCI info. supplied by probing */ struct rte_eth_dev_cb_list callbacks; /**< User application callbacks */ uint8_t attached; /**< Flag indicating the port is attached */ + enum rte_eth_dev_type dev_type; /**< Flag indicating the device type */ }; struct rte_eth_dev_sriov { @@ -1618,6 +1630,15 @@ extern uint8_t rte_eth_dev_count(void); /** * Function for internal use by port hotplug functions. + * Get the device type to know whether the device is physical or virtual. + * @param port_id The pointer to the port id + * @return + * - Device type. + */ +extern enum rte_eth_dev_type rte_eth_dev_get_device_type(uint8_t port_id); + +/** + * Function for internal use by port hotplug functions. * Copies current ethdev structures to the specified pointer. * * @param devs The pointer to the ethdev structures @@ -1706,10 +1727,12 @@ extern struct rte_eth_dev *rte_eth_dev_allocated(const char *name); * to that slot for the driver to use. * * @param name Unique identifier name for each Ethernet device + * @param type Device type of this Ethernet device * @return * - Slot in the rte_dev_devices array for a new device; */ -struct rte_eth_dev *rte_eth_dev_allocate(const char *name); +struct rte_eth_dev *rte_eth_dev_allocate(const char *name, + enum rte_eth_dev_type type); /** * Function for internal use by dummy drivers primarily, e.g. ring-based diff --git a/lib/librte_ether/rte_ether_version.map b/lib/librte_ether/rte_ether_version.map index 1d039ed..b78f5e3 100644 --- a/lib/librte_ether/rte_ether_version.map +++ b/lib/librte_ether/rte_ether_version.map @@ -116,6 +116,7 @@ DPDK_2.0 { rte_eth_dev_get_port_by_addr; rte_eth_dev_get_changed_port; rte_eth_dev_save; + rte_eth_dev_get_device_type; local: *; }; diff --git a/lib/librte_pmd_af_packet/rte_eth_af_packet.c b/lib/librte_pmd_af_packet/rte_eth_af_packet.c index 1ffe1cd..80e9bdf 100644 --- a/lib/librte_pmd_af_packet/rte_eth_af_packet.c +++ b/lib/librte_pmd_af_packet/rte_eth_af_packet.c @@ -649,7 +649,7 @@ rte_pmd_init_internals(const char *name, } /* reserve an ethdev entry */ - *eth_dev = rte_eth_dev_allocate(name); + *eth_dev = rte_eth_dev_allocate(name, RTE_ETH_DEV_VIRTUAL); if (*eth_dev == NULL) goto error; diff --git a/lib/librte_pmd_bond/rte_eth_bond_api.c b/lib/librte_pmd_bond/rte_eth_bond_api.c index 4ab3267..7a6a5f7 100644 --- a/lib/librte_pmd_bond/rte_eth_bond_api.c +++ b/lib/librte_pmd_bond/rte_eth_bond_api.c @@ -235,7 +235,7 @@ rte_eth_bond_create(const char *name, uint8_t mode, uint8_t socket_id) } /* reserve an ethdev entry */ - eth_dev = rte_eth_dev_allocate(name); + eth_dev = rte_eth_dev_allocate(name, RTE_ETH_DEV_VIRTUAL); if (eth_dev == NULL) { RTE_BOND_LOG(ERR, "Unable to allocate rte_eth_dev"); goto err; diff --git a/lib/librte_pmd_pcap/rte_eth_pcap.c b/lib/librte_pmd_pcap/rte_eth_pcap.c index d299288..af7fae8 100644 --- a/lib/librte_pmd_pcap/rte_eth_pcap.c +++ b/lib/librte_pmd_pcap/rte_eth_pcap.c @@ -709,7 +709,7 @@ rte_pmd_init_internals(const char *name, const unsigned nb_rx_queues, goto error; /* reserve an ethdev entry */ - *eth_dev = rte_eth_dev_allocate(name); + *eth_dev = rte_eth_dev_allocate(name, RTE_ETH_DEV_VIRTUAL); if (*eth_dev == NULL) goto error; diff --git a/lib/librte_pmd_ring/rte_eth_ring.c b/lib/librte_pmd_ring/rte_eth_ring.c index d5b1686..c837fcd 100644 --- a/lib/librte_pmd_ring/rte_eth_ring.c +++ b/lib/librte_pmd_ring/rte_eth_ring.c @@ -255,7 +255,7 @@ rte_eth_from_rings(const char *name, struct rte_ring *const rx_queues[], goto error; /* reserve an ethdev entry */ - eth_dev = rte_eth_dev_allocate(name); + eth_dev = rte_eth_dev_allocate(name, RTE_ETH_DEV_VIRTUAL); if (eth_dev == NULL) goto error; diff --git a/lib/librte_pmd_xenvirt/rte_eth_xenvirt.c b/lib/librte_pmd_xenvirt/rte_eth_xenvirt.c index 04e30c9..bc403d6 100644 --- a/lib/librte_pmd_xenvirt/rte_eth_xenvirt.c +++ b/lib/librte_pmd_xenvirt/rte_eth_xenvirt.c @@ -648,7 +648,7 @@ eth_dev_xenvirt_create(const char *name, const char *params, goto err; /* reserve an ethdev entry */ - eth_dev = rte_eth_dev_allocate(name); + eth_dev = rte_eth_dev_allocate(name, RTE_ETH_DEV_VIRTUAL); if (eth_dev == NULL) goto err; -- 1.9.1
These functions are used for attaching or detaching a port. When rte_eal_dev_attach() is called, the function tries to realize the device name as pci address. If this is done successfully, rte_eal_dev_attach() will attach physical device port. If not, attaches virtual devive port. When rte_eal_dev_detach() is called, the function gets the device type of this port to know whether the port is come from physical or virtual. And then specific detaching function will be called. v9: - Fix comments. - Use strcmp() instead of strncmp(). - Remove RTE_EAL_INVOKE_TYPE_PROBE/CLOSE. - Change definition of rte_dev_uninit_t. (Thanks to Thomas Monjalon and Maxime Leroy) v8: - Add missing symbol in version map. (Thanks to Qiu, Michael and Iremonger, Bernard) v7: - Fix typo of warning messages. (Thanks to Qiu, Michael) v5: - Change function names like below. rte_eal_dev_find_and_invoke() to rte_eal_vdev_find_and_invoke(). rte_eal_dev_invoke() to rte_eal_vdev_invoke(). - Add code to handle a return value of rte_eal_devargs_remove(). - Fix pci address format in rte_eal_dev_detach(). v4: - Fix comment. - Add error checking. - Fix indent of 'if' statement. - Change function name. Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp> --- lib/librte_eal/common/eal_common_dev.c | 307 ++++++++++++++++++++++++ lib/librte_eal/common/eal_private.h | 11 + lib/librte_eal/common/include/rte_dev.h | 33 +++ lib/librte_eal/linuxapp/eal/Makefile | 1 + lib/librte_eal/linuxapp/eal/eal_pci.c | 6 +- lib/librte_eal/linuxapp/eal/rte_eal_version.map | 2 + 6 files changed, 357 insertions(+), 3 deletions(-) diff --git a/lib/librte_eal/common/eal_common_dev.c b/lib/librte_eal/common/eal_common_dev.c index eae5656..4976bb9 100644 --- a/lib/librte_eal/common/eal_common_dev.c +++ b/lib/librte_eal/common/eal_common_dev.c @@ -32,10 +32,13 @@ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ +#include <stdio.h> +#include <limits.h> #include <string.h> #include <inttypes.h> #include <sys/queue.h> +#include <rte_ethdev.h> #include <rte_dev.h> #include <rte_devargs.h> #include <rte_debug.h> @@ -107,3 +110,307 @@ rte_eal_dev_init(void) } return 0; } + +/* So far, DPDK hotplug function only supports linux */ +#ifdef RTE_LIBRTE_EAL_HOTPLUG +static int +rte_eal_vdev_find_and_init(const char *name) +{ + struct rte_devargs *devargs; + struct rte_driver *driver; + + if (name == NULL) + return -EINVAL; + + /* find the specified device and call init function */ + TAILQ_FOREACH(devargs, &devargs_list, next) { + + if (devargs->type != RTE_DEVTYPE_VIRTUAL) + continue; + + if (strcmp(name, devargs->virtual.drv_name)) + continue; + + TAILQ_FOREACH(driver, &dev_driver_list, next) { + if (driver->type != PMD_VDEV) + continue; + + /* + * search a driver prefix in virtual device name. + * For example, if the driver is pcap PMD, driver->name + * will be "eth_pcap", but devargs->virtual.drv_name + * will be "eth_pcapN". So use strncmp to compare. + */ + if (!strncmp(driver->name, devargs->virtual.drv_name, + strlen(driver->name))) { + driver->init(devargs->virtual.drv_name, + devargs->args); + break; + } + } + + if (driver == NULL) { + RTE_LOG(WARNING, EAL, "no driver found for %s\n", + devargs->virtual.drv_name); + } + return 0; + } + return 1; +} + +static int +rte_eal_vdev_find_and_uninit(const char *name) +{ + struct rte_devargs *devargs; + struct rte_driver *driver; + + if (name == NULL) + return -EINVAL; + + /* find the specified device and call uninit function */ + TAILQ_FOREACH(devargs, &devargs_list, next) { + + if (devargs->type != RTE_DEVTYPE_VIRTUAL) + continue; + + if (strcmp(name, devargs->virtual.drv_name)) + continue; + + TAILQ_FOREACH(driver, &dev_driver_list, next) { + if (driver->type != PMD_VDEV) + continue; + + /* + * search a driver prefix in virtual device name. + * For example, if the driver is pcap PMD, driver->name + * will be "eth_pcap", but devargs->virtual.drv_name + * will be "eth_pcapN". So use strncmp to compare. + */ + if (!strncmp(driver->name, devargs->virtual.drv_name, + strlen(driver->name))) { + driver->uninit(devargs->virtual.drv_name); + break; + } + } + + if (driver == NULL) { + RTE_LOG(WARNING, EAL, "no driver found for %s\n", + devargs->virtual.drv_name); + } + return 0; + } + return 1; +} + +/* attach the new physical device, then store port_id of the device */ +static int +rte_eal_dev_attach_pdev(struct rte_pci_addr *addr, uint8_t *port_id) +{ + uint8_t new_port_id; + struct rte_eth_dev devs[RTE_MAX_ETHPORTS]; + + if ((addr == NULL) || (port_id == NULL)) + goto err; + + /* save current port status */ + if (rte_eth_dev_save(devs, sizeof(devs))) + goto err; + /* re-construct pci_device_list */ + if (rte_eal_pci_scan()) + goto err; + /* invoke probe func of the driver can handle the new device */ + if (rte_eal_pci_probe_one(addr)) + goto err; + /* get port_id enabled by above procedures */ + if (rte_eth_dev_get_changed_port(devs, &new_port_id)) + goto err; + + *port_id = new_port_id; + return 0; +err: + RTE_LOG(ERR, EAL, "Driver, cannot attach the device\n"); + return -1; +} + +/* detach the new physical device, then store pci_addr of the device */ +static int +rte_eal_dev_detach_pdev(uint8_t port_id, struct rte_pci_addr *addr) +{ + struct rte_pci_addr freed_addr; + struct rte_pci_addr vp; + + if (addr == NULL) + goto err; + + /* check whether the driver supports detach feature, or not */ + if (rte_eth_dev_is_detachable(port_id)) + goto err; + + /* get pci address by port id */ + if (rte_eth_dev_get_addr_by_port(port_id, &freed_addr)) + goto err; + + /* Zerod pci addr means the port comes from virtual device */ + vp.domain = vp.bus = vp.devid = vp.function = 0; + if (rte_eal_compare_pci_addr(&vp, &freed_addr) == 0) + goto err; + + /* invoke close func of the driver, + * also remove the device from pci_device_list */ + if (rte_eal_pci_close_one(&freed_addr)) + goto err; + + *addr = freed_addr; + return 0; +err: + RTE_LOG(ERR, EAL, "Driver, cannot detach the device\n"); + return -1; +} + +static void +get_vdev_name(char *vdevargs) +{ + char *sep; + + if (vdevargs == NULL) + return; + + /* set the first ',' to '\0' to split name and arguments */ + sep = strchr(vdevargs, ','); + if (sep != NULL) + sep[0] = '\0'; +} + +/* attach the new virtual device, then store port_id of the device */ +static int +rte_eal_dev_attach_vdev(const char *vdevargs, uint8_t *port_id) +{ + char *args; + uint8_t new_port_id; + struct rte_eth_dev devs[RTE_MAX_ETHPORTS]; + + if ((vdevargs == NULL) || (port_id == NULL)) + goto err0; + + args = strdup(vdevargs); + if (args == NULL) + goto err0; + + /* save current port status */ + if (rte_eth_dev_save(devs, sizeof(devs))) + goto err1; + /* add the vdevargs to devargs_list */ + if (rte_eal_devargs_add(RTE_DEVTYPE_VIRTUAL, args)) + goto err1; + /* parse vdevargs, then retrieve device name */ + get_vdev_name(args); + /* walk around dev_driver_list to find the driver of the device, + * then invoke probe function o the driver */ + if (rte_eal_vdev_find_and_init(args)) + goto err2; + /* get port_id enabled by above procedures */ + if (rte_eth_dev_get_changed_port(devs, &new_port_id)) + goto err2; + + free(args); + *port_id = new_port_id; + return 0; +err2: + rte_eal_devargs_remove(RTE_DEVTYPE_VIRTUAL, args); +err1: + free(args); +err0: + RTE_LOG(ERR, EAL, "Driver, cannot attach the device\n"); + return -1; +} + +/* detach the new virtual device, then store the name of the device */ +static int +rte_eal_dev_detach_vdev(uint8_t port_id, char *vdevname) +{ + char name[RTE_ETH_NAME_MAX_LEN]; + + if (vdevname == NULL) + goto err; + + /* check whether the driver supports detach feature, or not */ + if (rte_eth_dev_is_detachable(port_id)) + goto err; + + /* get device name by port id */ + if (rte_eth_dev_get_name_by_port(port_id, name)) + goto err; + /* walk around dev_driver_list to find the driver of the device, + * then invoke close function o the driver */ + if (rte_eal_vdev_find_and_uninit(name)) + goto err; + /* remove the vdevname from devargs_list */ + if (rte_eal_devargs_remove(RTE_DEVTYPE_VIRTUAL, name)) + goto err; + + strncpy(vdevname, name, sizeof(name)); + return 0; +err: + RTE_LOG(ERR, EAL, "Driver, cannot detach the device\n"); + return -1; +} + +/* attach the new device, then store port_id of the device */ +int +rte_eal_dev_attach(const char *devargs, uint8_t *port_id) +{ + struct rte_pci_addr addr; + + if ((devargs == NULL) || (port_id == NULL)) + return -EINVAL; + + if (eal_parse_pci_DomBDF(devargs, &addr) == 0) + return rte_eal_dev_attach_pdev(&addr, port_id); + else + return rte_eal_dev_attach_vdev(devargs, port_id); +} + +/* detach the device, then store the name of the device */ +int +rte_eal_dev_detach(uint8_t port_id, char *name) +{ + struct rte_pci_addr addr; + int ret; + + if (name == NULL) + return -EINVAL; + + if (rte_eth_dev_get_device_type(port_id) == RTE_ETH_DEV_PCI) { + ret = rte_eth_dev_get_addr_by_port(port_id, &addr); + if (ret < 0) + return ret; + + ret = rte_eal_dev_detach_pdev(port_id, &addr); + if (ret == 0) + snprintf(name, RTE_ETH_NAME_MAX_LEN, + "%04x:%02x:%02x.%d", + addr.domain, addr.bus, + addr.devid, addr.function); + + return ret; + } else + return rte_eal_dev_detach_vdev(port_id, name); +} +#else /* RTE_LIBRTE_EAL_HOTPLUG */ +int +rte_eal_dev_attach(const char *devargs __rte_unused, + uint8_t *port_id __rte_unused) +{ + RTE_LOG(ERR, EAL, "Hotplug support isn't enabled\n"); + return -1; +} + +/* detach the device, then store the name of the device */ +int +rte_eal_dev_detach(uint8_t port_id __rte_unused, + char *name __rte_unused) +{ + RTE_LOG(ERR, EAL, "Hotplug support isn't enabled\n"); + return -1; +} +#endif /* RTE_LIBRTE_EAL_HOTPLUG */ diff --git a/lib/librte_eal/common/eal_private.h b/lib/librte_eal/common/eal_private.h index 4acf5a0..98b286a 100644 --- a/lib/librte_eal/common/eal_private.h +++ b/lib/librte_eal/common/eal_private.h @@ -154,6 +154,17 @@ struct rte_pci_driver; struct rte_pci_device; /** + * Scan the content of the PCI bus, and the devices in the devices + * list + * + * This function is private to EAL. + * + * @return + * 0 on success, negative on error + */ +int rte_eal_pci_scan(void); + +/** * Mmap memory for single PCI device * * This function is private to EAL. diff --git a/lib/librte_eal/common/include/rte_dev.h b/lib/librte_eal/common/include/rte_dev.h index f7e3a10..a5ac770 100644 --- a/lib/librte_eal/common/include/rte_dev.h +++ b/lib/librte_eal/common/include/rte_dev.h @@ -47,6 +47,7 @@ extern "C" { #endif #include <sys/queue.h> +#include <rte_pci.h> /** Double linked list of device drivers. */ TAILQ_HEAD(rte_driver_list, rte_driver); @@ -57,6 +58,11 @@ TAILQ_HEAD(rte_driver_list, rte_driver); typedef int (rte_dev_init_t)(const char *name, const char *args); /** + * Uninitilization function called for each device driver once. + */ +typedef int (rte_dev_uninit_t)(const char *name); + +/** * Driver type enumeration */ enum pmd_type { @@ -72,6 +78,7 @@ struct rte_driver { enum pmd_type type; /**< PMD Driver type */ const char *name; /**< Driver name. */ rte_dev_init_t *init; /**< Device init. function. */ + rte_dev_uninit_t *uninit; /**< Device uninit. function. */ }; /** @@ -93,6 +100,32 @@ void rte_eal_driver_register(struct rte_driver *driver); void rte_eal_driver_unregister(struct rte_driver *driver); /** + * Attach a new device. + * + * @param devargs + * A pointer to a strings array describing the new device + * to be attached. The strings should be a pci address like + * '0000:01:00.0' or virtual device name like 'eth_pcap0'. + * @param port_id + * A pointer to a port identifier actually attached. + * @return + * 0 on success and port_id is filled, negative on error + */ +int rte_eal_dev_attach(const char *devargs, uint8_t *port_id); + +/** + * Detach a device. + * + * @param port_id + * The port identifier of the device to detach. + * @param addr + * A pointer to a device name actually detached. + * @return + * 0 on success and devname is filled, negative on error + */ +int rte_eal_dev_detach(uint8_t port_id, char *devname); + +/** * Initalize all the registered drivers in this process */ int rte_eal_dev_init(void); diff --git a/lib/librte_eal/linuxapp/eal/Makefile b/lib/librte_eal/linuxapp/eal/Makefile index e117cec..b59b201 100644 --- a/lib/librte_eal/linuxapp/eal/Makefile +++ b/lib/librte_eal/linuxapp/eal/Makefile @@ -45,6 +45,7 @@ CFLAGS += -I$(RTE_SDK)/lib/librte_eal/common/include CFLAGS += -I$(RTE_SDK)/lib/librte_ring CFLAGS += -I$(RTE_SDK)/lib/librte_mempool CFLAGS += -I$(RTE_SDK)/lib/librte_malloc +CFLAGS += -I$(RTE_SDK)/lib/librte_mbuf CFLAGS += -I$(RTE_SDK)/lib/librte_ether CFLAGS += -I$(RTE_SDK)/lib/librte_ivshmem CFLAGS += -I$(RTE_SDK)/lib/librte_pmd_ring diff --git a/lib/librte_eal/linuxapp/eal/eal_pci.c b/lib/librte_eal/linuxapp/eal/eal_pci.c index 4bdf51b..21d3e85 100644 --- a/lib/librte_eal/linuxapp/eal/eal_pci.c +++ b/lib/librte_eal/linuxapp/eal/eal_pci.c @@ -441,8 +441,8 @@ error: * Scan the content of the PCI bus, and the devices in the devices * list */ -static int -pci_scan(void) +int +rte_eal_pci_scan(void) { struct dirent *e; DIR *dir; @@ -774,7 +774,7 @@ rte_eal_pci_init(void) if (internal_config.no_pci) return 0; - if (pci_scan() < 0) { + if (rte_eal_pci_scan() < 0) { RTE_LOG(ERR, EAL, "%s(): Cannot scan PCI bus\n", __func__); return -1; } diff --git a/lib/librte_eal/linuxapp/eal/rte_eal_version.map b/lib/librte_eal/linuxapp/eal/rte_eal_version.map index d36286e..ff46b3a 100644 --- a/lib/librte_eal/linuxapp/eal/rte_eal_version.map +++ b/lib/librte_eal/linuxapp/eal/rte_eal_version.map @@ -21,6 +21,8 @@ DPDK_2.0 { rte_eal_alarm_cancel; rte_eal_alarm_set; rte_eal_dev_init; + rte_eal_dev_attach; + rte_eal_dev_detach; rte_eal_devargs_add; rte_eal_devargs_dump; rte_eal_devargs_type_count; -- 1.9.1
This patch adds a new section for describing port hotplug framework. Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp> --- doc/guides/prog_guide/index.rst | 1 + doc/guides/prog_guide/port_hotplug_framework.rst | 110 +++++++++++++++++++++++ 2 files changed, 111 insertions(+) create mode 100644 doc/guides/prog_guide/port_hotplug_framework.rst diff --git a/doc/guides/prog_guide/index.rst b/doc/guides/prog_guide/index.rst index 8d86dd4..428b76b 100644 --- a/doc/guides/prog_guide/index.rst +++ b/doc/guides/prog_guide/index.rst @@ -70,6 +70,7 @@ Programmer's Guide packet_classif_access_ctrl packet_framework vhost_lib + port_hotplug_framework source_org dev_kit_build_system dev_kit_root_make_help diff --git a/doc/guides/prog_guide/port_hotplug_framework.rst b/doc/guides/prog_guide/port_hotplug_framework.rst new file mode 100644 index 0000000..355ae28 --- /dev/null +++ b/doc/guides/prog_guide/port_hotplug_framework.rst @@ -0,0 +1,110 @@ +.. BSD LICENSE + Copyright(c) 2015 IGEL Co.,Ltd. All rights reserved. + All rights reserved. + + Redistribution and use in source and binary forms, with or without + modification, are permitted provided that the following conditions + are met: + + * Redistributions of source code must retain the above copyright + notice, this list of conditions and the following disclaimer. + * Redistributions in binary form must reproduce the above copyright + notice, this list of conditions and the following disclaimer in + the documentation and/or other materials provided with the + distribution. + * Neither the name of IGEL Co.,Ltd. nor the names of its + contributors may be used to endorse or promote products derived + from this software without specific prior written permission. + + THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS + "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT + LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR + A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT + OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, + SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT + LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, + DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY + THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT + (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE + OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + +Port Hotplug Framework +====================== + +The Port Hotplug Framework provides DPDK applications with the ability to +attach and detach ports at runtime. Because the framework depends on PMD +implementation, the ports that PMDs cannot handle are out of scope of this +framework. Furthermore, after detaching a port from a DPDK application, the +framework doesn't provide a way for removing the devices from the system. +For the ports backed by a physical NIC, the kernel will need to support PCI +Hotplug feature. + +Overview +-------- + +The basic requirements of the Port Hotplug Framework are: + +* DPDK applications that use the Port Hotplug Framework must manage their + own ports. + + The Port Hotplug Framework is implemented to allow DPDK applications to + manage ports. For example, when DPDK applications call the port attach + function, the attached port number is returned. DPDK applications can + also detach the port by port number. + +* Kernel support is needed for attaching or detaching physical device + ports. + + To attach new physical device ports, the device will be recognized by + userspace driver I/O framework in kernel at first. Then DPDK + applications can call the Port Hotplug functions to attach the ports. + For detaching, steps are vice versa. + +* Before detaching, they must be stopped and closed. + + DPDK applications must call "rte_eth_dev_stop()" and + "rte_eth_dev_close()" APIs before detaching ports. These functions will + start finalization sequence of the PMDs. + +* The framework doesn't affect legacy DPDK applications behavior. + + If the Port Hotplug functions aren't called, all legacy DPDK apps can + still work without modifications. + +Port Hotplug API overview +------------------------- + +* Attaching a port + + "rte_eal_dev_attach()" API attaches a port to DPDK application, and + returns the attached port number. Before calling the API, the device + should be recognized by an userspace driver I/O framework. The API + receives a pci address like "0000:01:00.0" or a virtual device name + like "eth_pcap0,iface=eth0". In the case of virtual device name, the + format is the same as the general "--vdev" option of DPDK. + +* Detaching a port + + "rte_eal_dev_detach()" API detaches a port from DPDK application, and + returns a pci address of the detached device or a virtual device name + of the device. + +Reference +--------- + + "testpmd" supports the Port Hotplug Framework. + +Limitations +----------- + +* The Port Hotplug APIs are not thread safe. + +* The framework can only be enabled with Linux. BSD is not supported. + +* To detach a port, the port should be backed by a device that igb_uio + manages. VFIO is not supported. + +* Not all PMDs support detaching feature. + To know whether a PMD can support detaching, search for the + "RTE_PCI_DRV_DETACHABLE" flag in PMD implementation. If the flag is + defined in the PMD, detaching is supported. -- 1.9.1
This patch adds finalization code to free resources allocated by the PMD. v6: - Fix a paramter of rte_eth_dev_free(). v4: - Change function name. Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp> --- lib/librte_pmd_pcap/rte_eth_pcap.c | 40 ++++++++++++++++++++++++++++++++++++++ 1 file changed, 40 insertions(+) diff --git a/lib/librte_pmd_pcap/rte_eth_pcap.c b/lib/librte_pmd_pcap/rte_eth_pcap.c index af7fae8..5e94930 100644 --- a/lib/librte_pmd_pcap/rte_eth_pcap.c +++ b/lib/librte_pmd_pcap/rte_eth_pcap.c @@ -498,6 +498,13 @@ static struct eth_dev_ops ops = { .stats_reset = eth_stats_reset, }; +static struct eth_driver rte_pcap_pmd = { + .pci_drv = { + .name = "rte_pcap_pmd", + .drv_flags = RTE_PCI_DRV_DETACHABLE, + }, +}; + /* * Function handler that opens the pcap file for reading a stores a * reference of it for use it later on. @@ -713,6 +720,10 @@ rte_pmd_init_internals(const char *name, const unsigned nb_rx_queues, if (*eth_dev == NULL) goto error; + /* check length of device name */ + if ((strlen((*eth_dev)->data->name) + 1) > sizeof(data->name)) + goto error; + /* now put it all together * - store queue data in internals, * - store numa_node info in pci_driver @@ -739,10 +750,13 @@ rte_pmd_init_internals(const char *name, const unsigned nb_rx_queues, data->nb_tx_queues = (uint16_t)nb_tx_queues; data->dev_link = pmd_link; data->mac_addrs = ð_addr; + strncpy(data->name, + (*eth_dev)->data->name, strlen((*eth_dev)->data->name)); (*eth_dev)->data = data; (*eth_dev)->dev_ops = &ops; (*eth_dev)->pci_dev = pci_dev; + (*eth_dev)->driver = &rte_pcap_pmd; return 0; @@ -927,10 +941,36 @@ rte_pmd_pcap_devinit(const char *name, const char *params) } +static int +rte_pmd_pcap_devuninit(const char *name) +{ + struct rte_eth_dev *eth_dev = NULL; + + RTE_LOG(INFO, PMD, "Closing pcap ethdev on numa socket %u\n", + rte_socket_id()); + + if (name == NULL) + return -1; + + /* reserve an ethdev entry */ + eth_dev = rte_eth_dev_allocated(name); + if (eth_dev == NULL) + return -1; + + rte_free(eth_dev->data->dev_private); + rte_free(eth_dev->data); + rte_free(eth_dev->pci_dev); + + rte_eth_dev_release_port(eth_dev); + + return 0; +} + static struct rte_driver pmd_pcap_drv = { .name = "eth_pcap", .type = PMD_VDEV, .init = rte_pmd_pcap_devinit, + .uninit = rte_pmd_pcap_devuninit, }; PMD_REGISTER_DRIVER(pmd_pcap_drv); -- 1.9.1
The patch introduces following commands. - port attach [ident] - port detach [port_id] - attach: attaching a port - detach: detaching a port - ident: pci address of physical device. Or device name and parameters of virtual device. (ex. 0000:02:00.0, eth_pcap0,iface=eth0) - port_id: port identifier v7: - Fix doc. (Thanks to Iremonger, Bernard) - Fix port checking implementation of star_port(); (Thanks to Qiu, Michael) v5: - Add testpmd documentation. (Thanks to Iremonger, Bernard) v4: - Fix strings of command help. Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp> --- app/test-pmd/cmdline.c | 137 +++++++++++++++---- app/test-pmd/config.c | 116 +++++++++------- app/test-pmd/parameters.c | 22 ++- app/test-pmd/testpmd.c | 199 +++++++++++++++++++++------- app/test-pmd/testpmd.h | 18 ++- doc/guides/testpmd_app_ug/testpmd_funcs.rst | 57 ++++++++ 6 files changed, 416 insertions(+), 133 deletions(-) diff --git a/app/test-pmd/cmdline.c b/app/test-pmd/cmdline.c index 4753bb4..fa6e3a6 100644 --- a/app/test-pmd/cmdline.c +++ b/app/test-pmd/cmdline.c @@ -579,6 +579,12 @@ static void cmd_help_long_parsed(void *parsed_result, "port close (port_id|all)\n" " Close all ports or port_id.\n\n" + "port attach (ident)\n" + " Attach physical or virtual dev by pci address or virtual device name\n\n" + + "port detach (port_id)\n" + " Detach physical or virtual dev by port_id\n\n" + "port config (port_id|all)" " speed (10|100|1000|10000|40000|auto)" " duplex (half|full|auto)\n" @@ -870,6 +876,89 @@ cmdline_parse_inst_t cmd_operate_specific_port = { }, }; +/* *** attach a specified port *** */ +struct cmd_operate_attach_port_result { + cmdline_fixed_string_t port; + cmdline_fixed_string_t keyword; + cmdline_fixed_string_t identifier; +}; + +static void cmd_operate_attach_port_parsed(void *parsed_result, + __attribute__((unused)) struct cmdline *cl, + __attribute__((unused)) void *data) +{ + struct cmd_operate_attach_port_result *res = parsed_result; + + if (!strcmp(res->keyword, "attach")) + attach_port(res->identifier); + else + printf("Unknown parameter\n"); +} + +cmdline_parse_token_string_t cmd_operate_attach_port_port = + TOKEN_STRING_INITIALIZER(struct cmd_operate_attach_port_result, + port, "port"); +cmdline_parse_token_string_t cmd_operate_attach_port_keyword = + TOKEN_STRING_INITIALIZER(struct cmd_operate_attach_port_result, + keyword, "attach"); +cmdline_parse_token_string_t cmd_operate_attach_port_identifier = + TOKEN_STRING_INITIALIZER(struct cmd_operate_attach_port_result, + identifier, NULL); + +cmdline_parse_inst_t cmd_operate_attach_port = { + .f = cmd_operate_attach_port_parsed, + .data = NULL, + .help_str = "port attach identifier, " + "identifier: pci address or virtual dev name", + .tokens = { + (void *)&cmd_operate_attach_port_port, + (void *)&cmd_operate_attach_port_keyword, + (void *)&cmd_operate_attach_port_identifier, + NULL, + }, +}; + +/* *** detach a specified port *** */ +struct cmd_operate_detach_port_result { + cmdline_fixed_string_t port; + cmdline_fixed_string_t keyword; + uint8_t port_id; +}; + +static void cmd_operate_detach_port_parsed(void *parsed_result, + __attribute__((unused)) struct cmdline *cl, + __attribute__((unused)) void *data) +{ + struct cmd_operate_detach_port_result *res = parsed_result; + + if (!strcmp(res->keyword, "detach")) + detach_port(res->port_id); + else + printf("Unknown parameter\n"); +} + +cmdline_parse_token_string_t cmd_operate_detach_port_port = + TOKEN_STRING_INITIALIZER(struct cmd_operate_detach_port_result, + port, "port"); +cmdline_parse_token_string_t cmd_operate_detach_port_keyword = + TOKEN_STRING_INITIALIZER(struct cmd_operate_detach_port_result, + keyword, "detach"); +cmdline_parse_token_num_t cmd_operate_detach_port_port_id = + TOKEN_NUM_INITIALIZER(struct cmd_operate_detach_port_result, + port_id, UINT8); + +cmdline_parse_inst_t cmd_operate_detach_port = { + .f = cmd_operate_detach_port_parsed, + .data = NULL, + .help_str = "port detach port_id", + .tokens = { + (void *)&cmd_operate_detach_port_port, + (void *)&cmd_operate_detach_port_keyword, + (void *)&cmd_operate_detach_port_port_id, + NULL, + }, +}; + /* *** configure speed for all ports *** */ struct cmd_config_speed_all { cmdline_fixed_string_t port; @@ -924,7 +1013,7 @@ cmd_config_speed_all_parsed(void *parsed_result, return; } - for (pid = 0; pid < nb_ports; pid++) { + FOREACH_PORT(pid, ports) { ports[pid].dev_conf.link_speed = link_speed; ports[pid].dev_conf.link_duplex = link_duplex; } @@ -992,10 +1081,8 @@ cmd_config_speed_specific_parsed(void *parsed_result, return; } - if (res->id >= nb_ports) { - printf("Port id %d must be less than %d\n", res->id, nb_ports); + if (port_id_is_invalid(res->id, ENABLED_WARN)) return; - } if (!strcmp(res->value1, "10")) link_speed = ETH_LINK_SPEED_10; @@ -1555,7 +1642,7 @@ cmd_config_rxtx_queue_parsed(void *parsed_result, return; } - if (port_id_is_invalid(res->portid)) + if (port_id_is_invalid(res->portid, ENABLED_WARN)) return; if (port_is_started(res->portid) != 1) { @@ -2955,7 +3042,7 @@ cmd_csum_parsed(void *parsed_result, int hw = 0; uint16_t mask = 0; - if (port_id_is_invalid(res->port_id)) { + if (port_id_is_invalid(res->port_id, ENABLED_WARN)) { printf("invalid port %d\n", res->port_id); return; } @@ -3047,10 +3134,8 @@ cmd_csum_tunnel_parsed(void *parsed_result, { struct cmd_csum_tunnel_result *res = parsed_result; - if (port_id_is_invalid(res->port_id)) { - printf("invalid port %d\n", res->port_id); + if (port_id_is_invalid(res->port_id, ENABLED_WARN)) return; - } if (!strcmp(res->onoff, "on")) ports[res->port_id].tx_ol_flags |= @@ -3105,7 +3190,7 @@ cmd_tso_set_parsed(void *parsed_result, struct cmd_tso_set_result *res = parsed_result; struct rte_eth_dev_info dev_info; - if (port_id_is_invalid(res->port_id)) + if (port_id_is_invalid(res->port_id, ENABLED_WARN)) return; if (!strcmp(res->mode, "set")) @@ -4081,10 +4166,8 @@ static void cmd_set_bond_mac_addr_parsed(void *parsed_result, struct cmd_set_bond_mac_addr_result *res = parsed_result; int ret; - if (res->port_num >= nb_ports) { - printf("Port id %d must be less than %d\n", res->port_num, nb_ports); + if (port_id_is_invalid(res->port_num, ENABLED_WARN)) return; - } ret = rte_eth_bond_mac_address_set(res->port_num, &res->address); @@ -4321,7 +4404,7 @@ static void cmd_set_promisc_mode_parsed(void *parsed_result, /* all ports */ if (allports) { - for (i = 0; i < nb_ports; i++) { + FOREACH_PORT(i, ports) { if (enable) rte_eth_promiscuous_enable(i); else @@ -4401,7 +4484,7 @@ static void cmd_set_allmulti_mode_parsed(void *parsed_result, /* all ports */ if (allports) { - for (i = 0; i < nb_ports; i++) { + FOREACH_PORT(i, ports) { if (enable) rte_eth_allmulticast_enable(i); else @@ -5586,25 +5669,25 @@ static void cmd_showportall_parsed(void *parsed_result, struct cmd_showportall_result *res = parsed_result; if (!strcmp(res->show, "clear")) { if (!strcmp(res->what, "stats")) - for (i = 0; i < nb_ports; i++) + FOREACH_PORT(i, ports) nic_stats_clear(i); else if (!strcmp(res->what, "xstats")) - for (i = 0; i < nb_ports; i++) + FOREACH_PORT(i, ports) nic_xstats_clear(i); } else if (!strcmp(res->what, "info")) - for (i = 0; i < nb_ports; i++) + FOREACH_PORT(i, ports) port_infos_display(i); else if (!strcmp(res->what, "stats")) - for (i = 0; i < nb_ports; i++) + FOREACH_PORT(i, ports) nic_stats_display(i); else if (!strcmp(res->what, "xstats")) - for (i = 0; i < nb_ports; i++) + FOREACH_PORT(i, ports) nic_xstats_display(i); else if (!strcmp(res->what, "fdir")) - for (i = 0; i < nb_ports; i++) + FOREACH_PORT(i, ports) fdir_get_infos(i); else if (!strcmp(res->what, "stat_qmap")) - for (i = 0; i < nb_ports; i++) + FOREACH_PORT(i, ports) nic_stats_mapping_display(i); } @@ -9172,6 +9255,8 @@ cmdline_parse_ctx_t main_ctx[] = { (cmdline_parse_inst_t *)&cmd_set_qmap, (cmdline_parse_inst_t *)&cmd_operate_port, (cmdline_parse_inst_t *)&cmd_operate_specific_port, + (cmdline_parse_inst_t *)&cmd_operate_attach_port, + (cmdline_parse_inst_t *)&cmd_operate_detach_port, (cmdline_parse_inst_t *)&cmd_config_speed_all, (cmdline_parse_inst_t *)&cmd_config_speed_specific, (cmdline_parse_inst_t *)&cmd_config_rx_tx, @@ -9250,7 +9335,7 @@ prompt(void) static void cmd_reconfig_device_queue(portid_t id, uint8_t dev, uint8_t queue) { - if (id < nb_ports) { + if (!port_id_is_invalid(id, DISABLED_WARN)) { /* check if need_reconfig has been set to 1 */ if (ports[id].need_reconfig == 0) ports[id].need_reconfig = dev; @@ -9260,7 +9345,7 @@ cmd_reconfig_device_queue(portid_t id, uint8_t dev, uint8_t queue) } else { portid_t pid; - for (pid = 0; pid < nb_ports; pid++) { + FOREACH_PORT(pid, ports) { /* check if need_reconfig has been set to 1 */ if (ports[pid].need_reconfig == 0) ports[pid].need_reconfig = dev; @@ -9278,10 +9363,8 @@ bypass_is_supported(portid_t port_id) struct rte_port *port; struct rte_pci_id *pci_id; - if (port_id >= nb_ports) { - printf("\tPort id must be less than %d.\n", nb_ports); + if (port_id_is_invalid(port_id, ENABLED_WARN)) return 0; - } /* Get the device id. */ port = &ports[port_id]; diff --git a/app/test-pmd/config.c b/app/test-pmd/config.c index 6bcd23c..718167c 100644 --- a/app/test-pmd/config.c +++ b/app/test-pmd/config.c @@ -124,11 +124,15 @@ nic_stats_display(portid_t port_id) struct rte_eth_stats stats; struct rte_port *port = &ports[port_id]; uint8_t i; + portid_t pid; static const char *nic_stats_border = "########################"; - if (port_id >= nb_ports) { - printf("Invalid port, range is [0, %d]\n", nb_ports - 1); + if (port_id_is_invalid(port_id, ENABLED_WARN)) { + printf("Valid port range is [0"); + FOREACH_PORT(pid, ports) + printf(", %d", pid); + printf("]\n"); return; } rte_eth_stats_get(port_id, &stats); @@ -201,8 +205,13 @@ nic_stats_display(portid_t port_id) void nic_stats_clear(portid_t port_id) { - if (port_id >= nb_ports) { - printf("Invalid port, range is [0, %d]\n", nb_ports - 1); + portid_t pid; + + if (port_id_is_invalid(port_id, ENABLED_WARN)) { + printf("Valid port range is [0"); + FOREACH_PORT(pid, ports) + printf(", %d", pid); + printf("]\n"); return; } rte_eth_stats_reset(port_id); @@ -249,11 +258,15 @@ nic_stats_mapping_display(portid_t port_id) { struct rte_port *port = &ports[port_id]; uint16_t i; + portid_t pid; static const char *nic_stats_mapping_border = "########################"; - if (port_id >= nb_ports) { - printf("Invalid port, range is [0, %d]\n", nb_ports - 1); + if (port_id_is_invalid(port_id, ENABLED_WARN)) { + printf("Valid port range is [0"); + FOREACH_PORT(pid, ports) + printf(", %d", pid); + printf("]\n"); return; } @@ -302,9 +315,13 @@ port_infos_display(portid_t port_id) int vlan_offload; struct rte_mempool * mp; static const char *info_border = "*********************"; + portid_t pid; - if (port_id >= nb_ports) { - printf("Invalid port, range is [0, %d]\n", nb_ports - 1); + if (port_id_is_invalid(port_id, ENABLED_WARN)) { + printf("Valid port range is [0"); + FOREACH_PORT(pid, ports) + printf(", %d", pid); + printf("]\n"); return; } port = &ports[port_id]; @@ -362,11 +379,14 @@ port_infos_display(portid_t port_id) } int -port_id_is_invalid(portid_t port_id) +port_id_is_invalid(portid_t port_id, enum print_warning warning) { - if (port_id < nb_ports) + if (ports[port_id].enabled) return 0; - printf("Invalid port %d (must be < nb_ports=%d)\n", port_id, nb_ports); + + if (warning == ENABLED_WARN) + printf("Invalid port %d\n", port_id); + return 1; } @@ -425,7 +445,7 @@ port_reg_bit_display(portid_t port_id, uint32_t reg_off, uint8_t bit_x) uint32_t reg_v; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; if (port_reg_off_is_invalid(port_id, reg_off)) return; @@ -444,7 +464,7 @@ port_reg_bit_field_display(portid_t port_id, uint32_t reg_off, uint8_t l_bit; uint8_t h_bit; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; if (port_reg_off_is_invalid(port_id, reg_off)) return; @@ -471,7 +491,7 @@ port_reg_display(portid_t port_id, uint32_t reg_off) { uint32_t reg_v; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; if (port_reg_off_is_invalid(port_id, reg_off)) return; @@ -485,7 +505,7 @@ port_reg_bit_set(portid_t port_id, uint32_t reg_off, uint8_t bit_pos, { uint32_t reg_v; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; if (port_reg_off_is_invalid(port_id, reg_off)) return; @@ -513,7 +533,7 @@ port_reg_bit_field_set(portid_t port_id, uint32_t reg_off, uint8_t l_bit; uint8_t h_bit; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; if (port_reg_off_is_invalid(port_id, reg_off)) return; @@ -547,7 +567,7 @@ port_reg_bit_field_set(portid_t port_id, uint32_t reg_off, void port_reg_set(portid_t port_id, uint32_t reg_off, uint32_t reg_v) { - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; if (port_reg_off_is_invalid(port_id, reg_off)) return; @@ -560,7 +580,7 @@ port_mtu_set(portid_t port_id, uint16_t mtu) { int diag; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; diag = rte_eth_dev_set_mtu(port_id, mtu); if (diag == 0) @@ -723,7 +743,7 @@ rx_ring_desc_display(portid_t port_id, queueid_t rxq_id, uint16_t rxd_id) { const struct rte_memzone *rx_mz; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; if (rx_queue_id_is_invalid(rxq_id)) return; @@ -740,7 +760,7 @@ tx_ring_desc_display(portid_t port_id, queueid_t txq_id, uint16_t txd_id) { const struct rte_memzone *tx_mz; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; if (tx_queue_id_is_invalid(txq_id)) return; @@ -801,7 +821,7 @@ port_rss_reta_info(portid_t port_id, uint16_t i, idx, shift; int ret; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; ret = rte_eth_dev_rss_reta_query(port_id, reta_conf, nb_entries); @@ -833,7 +853,7 @@ port_rss_hash_conf_show(portid_t port_id, int show_rss_key) uint8_t i; int diag; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; /* Get RSS hash key if asked to display it */ rss_conf.rss_key = (show_rss_key) ? rss_key : NULL; @@ -1411,12 +1431,8 @@ set_fwd_ports_list(unsigned int *portlist, unsigned int nb_pt) again: for (i = 0; i < nb_pt; i++) { port_id = (portid_t) portlist[i]; - if (port_id >= nb_ports) { - printf("Invalid port id %u >= %u\n", - (unsigned int) port_id, - (unsigned int) nb_ports); + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; - } if (record_now) fwd_ports_ids[i] = port_id; } @@ -1574,7 +1590,7 @@ vlan_extend_set(portid_t port_id, int on) int diag; int vlan_offload; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; vlan_offload = rte_eth_dev_get_vlan_offload(port_id); @@ -1596,7 +1612,7 @@ rx_vlan_strip_set(portid_t port_id, int on) int diag; int vlan_offload; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; vlan_offload = rte_eth_dev_get_vlan_offload(port_id); @@ -1617,7 +1633,7 @@ rx_vlan_strip_set_on_queue(portid_t port_id, uint16_t queue_id, int on) { int diag; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; diag = rte_eth_dev_set_vlan_strip_on_queue(port_id, queue_id, on); @@ -1632,7 +1648,7 @@ rx_vlan_filter_set(portid_t port_id, int on) int diag; int vlan_offload; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; vlan_offload = rte_eth_dev_get_vlan_offload(port_id); @@ -1653,7 +1669,7 @@ rx_vft_set(portid_t port_id, uint16_t vlan_id, int on) { int diag; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; if (vlan_id_is_invalid(vlan_id)) return; @@ -1670,7 +1686,7 @@ rx_vlan_all_filter_set(portid_t port_id, int on) { uint16_t vlan_id; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; for (vlan_id = 0; vlan_id < 4096; vlan_id++) rx_vft_set(port_id, vlan_id, on); @@ -1680,7 +1696,7 @@ void vlan_tpid_set(portid_t port_id, uint16_t tp_id) { int diag; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; diag = rte_eth_dev_set_vlan_ether_type(port_id, tp_id); @@ -1695,7 +1711,7 @@ vlan_tpid_set(portid_t port_id, uint16_t tp_id) void tx_vlan_set(portid_t port_id, uint16_t vlan_id) { - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; if (vlan_id_is_invalid(vlan_id)) return; @@ -1706,7 +1722,7 @@ tx_vlan_set(portid_t port_id, uint16_t vlan_id) void tx_vlan_reset(portid_t port_id) { - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; ports[port_id].tx_ol_flags &= ~TESTPMD_TX_OFFLOAD_INSERT_VLAN; } @@ -1714,7 +1730,7 @@ tx_vlan_reset(portid_t port_id) void tx_vlan_pvid_set(portid_t port_id, uint16_t vlan_id, int on) { - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; rte_eth_dev_set_vlan_pvid(port_id, vlan_id, on); @@ -1726,7 +1742,7 @@ set_qmap(portid_t port_id, uint8_t is_rx, uint16_t queue_id, uint8_t map_value) uint16_t i; uint8_t existing_mapping_found = 0; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; if (is_rx ? (rx_queue_id_is_invalid(queue_id)) : (tx_queue_id_is_invalid(queue_id))) @@ -1778,7 +1794,7 @@ fdir_add_signature_filter(portid_t port_id, uint8_t queue_id, { int diag; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; diag = rte_eth_dev_fdir_add_signature_filter(port_id, fdir_filter, @@ -1796,7 +1812,7 @@ fdir_update_signature_filter(portid_t port_id, uint8_t queue_id, { int diag; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; diag = rte_eth_dev_fdir_update_signature_filter(port_id, fdir_filter, @@ -1814,7 +1830,7 @@ fdir_remove_signature_filter(portid_t port_id, { int diag; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; diag = rte_eth_dev_fdir_remove_signature_filter(port_id, fdir_filter); @@ -1886,7 +1902,7 @@ fdir_get_infos(portid_t port_id) static const char *fdir_stats_border = "########################"; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; ret = rte_eth_dev_filter_supported(port_id, RTE_ETH_FILTER_FDIR); if (ret < 0) { @@ -1960,7 +1976,7 @@ fdir_add_perfect_filter(portid_t port_id, uint16_t soft_id, uint8_t queue_id, { int diag; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; diag = rte_eth_dev_fdir_add_perfect_filter(port_id, fdir_filter, @@ -1978,7 +1994,7 @@ fdir_update_perfect_filter(portid_t port_id, uint16_t soft_id, uint8_t queue_id, { int diag; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; diag = rte_eth_dev_fdir_update_perfect_filter(port_id, fdir_filter, @@ -1996,7 +2012,7 @@ fdir_remove_perfect_filter(portid_t port_id, uint16_t soft_id, { int diag; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; diag = rte_eth_dev_fdir_remove_perfect_filter(port_id, fdir_filter, @@ -2013,7 +2029,7 @@ fdir_set_masks(portid_t port_id, struct rte_fdir_masks *fdir_masks) { int diag; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; diag = rte_eth_dev_fdir_set_masks(port_id, fdir_masks); @@ -2090,7 +2106,7 @@ set_vf_traffic(portid_t port_id, uint8_t is_rx, uint16_t vf, uint8_t on) { int diag; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; if (is_rx) diag = rte_eth_dev_set_vf_rx(port_id,vf,on); @@ -2112,7 +2128,7 @@ set_vf_rx_vlan(portid_t port_id, uint16_t vlan_id, uint64_t vf_mask, uint8_t on) { int diag; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; if (vlan_id_is_invalid(vlan_id)) return; @@ -2129,7 +2145,7 @@ set_queue_rate_limit(portid_t port_id, uint16_t queue_idx, uint16_t rate) int diag; struct rte_eth_link link; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return 1; rte_eth_link_get_nowait(port_id, &link); if (rate > link.link_speed) { @@ -2154,7 +2170,7 @@ set_vf_rate_limit(portid_t port_id, uint16_t vf, uint16_t rate, uint64_t q_msk) if (q_msk == 0) return 0; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return 1; rte_eth_link_get_nowait(port_id, &link); if (rate > link.link_speed) { diff --git a/app/test-pmd/parameters.c b/app/test-pmd/parameters.c index 19fbf46..07740c4 100644 --- a/app/test-pmd/parameters.c +++ b/app/test-pmd/parameters.c @@ -370,6 +370,7 @@ parse_portnuma_config(const char *q_arg) }; unsigned long int_fld[_NUM_FLD]; char *str_fld[_NUM_FLD]; + portid_t pid; /* reset from value set at definition */ while ((p = strchr(p0,'(')) != NULL) { @@ -391,8 +392,11 @@ parse_portnuma_config(const char *q_arg) return -1; } port_id = (uint8_t)int_fld[FLD_PORT]; - if (port_id >= nb_ports) { - printf("Invalid port, range is [0, %d]\n", nb_ports - 1); + if (port_id_is_invalid(port_id, ENABLED_WARN)) { + printf("Valid port range is [0"); + FOREACH_PORT(pid, ports) + printf(", %d", pid); + printf("]\n"); return -1; } socket_id = (uint8_t)int_fld[FLD_SOCKET]; @@ -423,6 +427,7 @@ parse_ringnuma_config(const char *q_arg) }; unsigned long int_fld[_NUM_FLD]; char *str_fld[_NUM_FLD]; + portid_t pid; #define RX_RING_ONLY 0x1 #define TX_RING_ONLY 0x2 #define RXTX_RING 0x3 @@ -447,8 +452,11 @@ parse_ringnuma_config(const char *q_arg) return -1; } port_id = (uint8_t)int_fld[FLD_PORT]; - if (port_id >= nb_ports) { - printf("Invalid port, range is [0, %d]\n", nb_ports - 1); + if (port_id_is_invalid(port_id, ENABLED_WARN)) { + printf("Valid port range is [0"); + FOREACH_PORT(pid, ports) + printf(", %d", pid); + printf("]\n"); return -1; } socket_id = (uint8_t)int_fld[FLD_SOCKET]; @@ -620,12 +628,12 @@ launch_args_parse(int argc, char** argv) #endif if (!strcmp(lgopts[opt_idx].name, "nb-ports")) { n = atoi(optarg); - if (n > 0 && n <= nb_ports) + if (n > 0 && + !port_id_is_invalid(n, DISABLED_WARN)) nb_fwd_ports = (uint8_t) n; else rte_exit(EXIT_FAILURE, - "nb-ports should be > 0 and <= %d\n", - nb_ports); + "Invalid port %d\n", n); } if (!strcmp(lgopts[opt_idx].name, "nb-cores")) { n = atoi(optarg); diff --git a/app/test-pmd/testpmd.c b/app/test-pmd/testpmd.c index 3aebea6..48e726e 100644 --- a/app/test-pmd/testpmd.c +++ b/app/test-pmd/testpmd.c @@ -71,6 +71,7 @@ #include <rte_pci.h> #include <rte_ether.h> #include <rte_ethdev.h> +#include <rte_dev.h> #include <rte_string_fns.h> #ifdef RTE_LIBRTE_PMD_XENVIRT #include <rte_eth_xenvirt.h> @@ -303,7 +304,7 @@ uint16_t nb_rx_queue_stats_mappings = 0; /* Forward function declarations */ static void map_port_queue_stats_mapping_registers(uint8_t pi, struct rte_port *port); -static void check_all_ports_link_status(uint8_t port_num, uint32_t port_mask); +static void check_all_ports_link_status(uint32_t port_mask); /* * Check if all the ports are started. @@ -312,6 +313,20 @@ static void check_all_ports_link_status(uint8_t port_num, uint32_t port_mask); static int all_ports_started(void); /* + * Find next enabled port + */ +portid_t +find_next_port(portid_t p, struct rte_port *ports, int size) +{ + if (ports == NULL) + rte_exit(-EINVAL, "failed to find a next port id\n"); + + while ((ports[p].enabled == 0) && (p < size)) + p++; + return p; +} + +/* * Setup default configuration. */ static void @@ -540,7 +555,8 @@ init_config(void) + RTE_TEST_TX_DESC_MAX + MAX_PKT_BURST; if (!numa_support) - nb_mbuf_per_pool = (nb_mbuf_per_pool * nb_ports); + nb_mbuf_per_pool = + (nb_mbuf_per_pool * RTE_MAX_ETHPORTS); } if (!numa_support) { @@ -553,14 +569,19 @@ init_config(void) /* Configuration of Ethernet ports. */ ports = rte_zmalloc("testpmd: ports", - sizeof(struct rte_port) * nb_ports, + sizeof(struct rte_port) * RTE_MAX_ETHPORTS, RTE_CACHE_LINE_SIZE); if (ports == NULL) { - rte_exit(EXIT_FAILURE, "rte_zmalloc(%d struct rte_port) " - "failed\n", nb_ports); + rte_exit(EXIT_FAILURE, + "rte_zmalloc(%d struct rte_port) failed\n", + RTE_MAX_ETHPORTS); } - for (pid = 0; pid < nb_ports; pid++) { + /* enabled allocated ports */ + for (pid = 0; pid < nb_ports; pid++) + ports[pid].enabled = 1; + + FOREACH_PORT(pid, ports) { port = &ports[pid]; rte_eth_dev_info_get(pid, &port->dev_info); @@ -590,8 +611,7 @@ init_config(void) nb_mbuf_per_pool = nb_mbuf_per_pool/nb_ports; for (i = 0; i < MAX_SOCKET; i++) { - nb_mbuf = (nb_mbuf_per_pool * - port_per_socket[i]); + nb_mbuf = (nb_mbuf_per_pool * RTE_MAX_ETHPORTS); if (nb_mbuf) mbuf_pool_create(mbuf_data_size, nb_mbuf,i); @@ -623,14 +643,6 @@ reconfig(portid_t new_port_id, unsigned socket_id) struct rte_port *port; /* Reconfiguration of Ethernet ports. */ - ports = rte_realloc(ports, - sizeof(struct rte_port) * nb_ports, - RTE_CACHE_LINE_SIZE); - if (ports == NULL) { - rte_exit(EXIT_FAILURE, "rte_realloc(%d struct rte_port) failed\n", - nb_ports); - } - port = &ports[new_port_id]; rte_eth_dev_info_get(new_port_id, &port->dev_info); @@ -651,7 +663,7 @@ init_fwd_streams(void) streamid_t sm_id, nb_fwd_streams_new; /* set socket id according to numa or not */ - for (pid = 0; pid < nb_ports; pid++) { + FOREACH_PORT(pid, ports) { port = &ports[pid]; if (nb_rxq > port->dev_info.max_rx_queues) { printf("Fail: nb_rxq(%d) is greater than " @@ -1252,7 +1264,7 @@ all_ports_started(void) portid_t pi; struct rte_port *port; - for (pi = 0; pi < nb_ports; pi++) { + FOREACH_PORT(pi, ports) { port = &ports[pi]; /* Check if there is a port which is not started */ if (port->port_status != RTE_PORT_STARTED) @@ -1264,6 +1276,45 @@ all_ports_started(void) } int +all_ports_stopped(void) +{ + portid_t pi; + struct rte_port *port; + + FOREACH_PORT(pi, ports) { + port = &ports[pi]; + if (port->port_status != RTE_PORT_STOPPED) + return 0; + } + + return 1; +} + +int +port_is_started(portid_t port_id) +{ + if (port_id_is_invalid(port_id, ENABLED_WARN)) + return 0; + + if (ports[port_id].port_status != RTE_PORT_STARTED) + return 0; + + return 1; +} + +static int +port_is_closed(portid_t port_id) +{ + if (port_id_is_invalid(port_id, ENABLED_WARN)) + return 0; + + if (ports[port_id].port_status != RTE_PORT_CLOSED) + return 0; + + return 1; +} + +int start_port(portid_t pid) { int diag, need_check_link_status = 0; @@ -1284,8 +1335,8 @@ start_port(portid_t pid) if(dcb_config) dcb_test = 1; - for (pi = 0; pi < nb_ports; pi++) { - if (pid < nb_ports && pid != pi) + FOREACH_PORT(pi, ports) { + if (pid != pi && pid != (portid_t)RTE_PORT_ALL) continue; port = &ports[pi]; @@ -1409,7 +1460,7 @@ start_port(portid_t pid) } if (need_check_link_status && !no_link_check) - check_all_ports_link_status(nb_ports, RTE_PORT_ALL); + check_all_ports_link_status(RTE_PORT_ALL); else printf("Please stop the ports first\n"); @@ -1434,8 +1485,8 @@ stop_port(portid_t pid) } printf("Stopping ports...\n"); - for (pi = 0; pi < nb_ports; pi++) { - if (pid < nb_ports && pid != pi) + FOREACH_PORT(pi, ports) { + if (!port_id_is_invalid(pid, DISABLED_WARN) && pid != pi) continue; port = &ports[pi]; @@ -1451,7 +1502,7 @@ stop_port(portid_t pid) need_check_link_status = 1; } if (need_check_link_status && !no_link_check) - check_all_ports_link_status(nb_ports, RTE_PORT_ALL); + check_all_ports_link_status(RTE_PORT_ALL); printf("Done\n"); } @@ -1469,8 +1520,8 @@ close_port(portid_t pid) printf("Closing ports...\n"); - for (pi = 0; pi < nb_ports; pi++) { - if (pid < nb_ports && pid != pi) + FOREACH_PORT(pi, ports) { + if (!port_id_is_invalid(pid, DISABLED_WARN) && pid != pi) continue; port = &ports[pi]; @@ -1490,31 +1541,83 @@ close_port(portid_t pid) printf("Done\n"); } -int -all_ports_stopped(void) +void +attach_port(char *identifier) { - portid_t pi; - struct rte_port *port; + portid_t i, j, pi = 0; - for (pi = 0; pi < nb_ports; pi++) { - port = &ports[pi]; - if (port->port_status != RTE_PORT_STOPPED) - return 0; + printf("Attaching a new port...\n"); + + if (identifier == NULL) { + printf("Invalid parameters are specified\n"); + return; } - return 1; + if (test_done == 0) { + printf("Please stop forwarding first\n"); + return; + } + + if (rte_eal_dev_attach(identifier, &pi)) + return; + + ports[pi].enabled = 1; + reconfig(pi, rte_eth_dev_socket_id(pi)); + rte_eth_promiscuous_enable(pi); + + nb_ports = rte_eth_dev_count(); + + /* set_default_fwd_ports_config(); */ + bzero(fwd_ports_ids, sizeof(fwd_ports_ids)); + i = 0; + FOREACH_PORT(j, ports) { + fwd_ports_ids[i] = j; + i++; + } + nb_cfg_ports = nb_ports; + nb_fwd_ports++; + + ports[pi].port_status = RTE_PORT_STOPPED; + + printf("Port %d is attached. Now total ports is %d\n", pi, nb_ports); + printf("Done\n"); } -int -port_is_started(portid_t port_id) +void +detach_port(uint8_t port_id) { - if (port_id_is_invalid(port_id)) - return -1; + portid_t i, pi = 0; + char name[RTE_ETH_NAME_MAX_LEN]; - if (ports[port_id].port_status != RTE_PORT_STARTED) - return 0; + printf("Detaching a port...\n"); - return 1; + if (!port_is_closed(port_id)) { + printf("Please close port first\n"); + return; + } + + rte_eth_promiscuous_disable(port_id); + + if (rte_eal_dev_detach(port_id, name)) + return; + + ports[port_id].enabled = 0; + nb_ports = rte_eth_dev_count(); + + /* set_default_fwd_ports_config(); */ + bzero(fwd_ports_ids, sizeof(fwd_ports_ids)); + i = 0; + FOREACH_PORT(pi, ports) { + fwd_ports_ids[i] = pi; + i++; + } + nb_cfg_ports = nb_ports; + nb_fwd_ports--; + + printf("Port '%s' is detached. Now total ports is %d\n", + name, nb_ports); + printf("Done\n"); + return; } void @@ -1522,7 +1625,7 @@ pmd_test_exit(void) { portid_t pt_id; - for (pt_id = 0; pt_id < nb_ports; pt_id++) { + FOREACH_PORT(pt_id, ports) { printf("Stopping port %d...", pt_id); fflush(stdout); rte_eth_dev_close(pt_id); @@ -1541,7 +1644,7 @@ struct pmd_test_command { /* Check the link status of all ports in up to 9s, and print them finally */ static void -check_all_ports_link_status(uint8_t port_num, uint32_t port_mask) +check_all_ports_link_status(uint32_t port_mask) { #define CHECK_INTERVAL 100 /* 100ms */ #define MAX_CHECK_TIME 90 /* 9s (90 * 100ms) in total */ @@ -1552,7 +1655,7 @@ check_all_ports_link_status(uint8_t port_num, uint32_t port_mask) fflush(stdout); for (count = 0; count <= MAX_CHECK_TIME; count++) { all_ports_up = 1; - for (portid = 0; portid < port_num; portid++) { + FOREACH_PORT(portid, ports) { if ((port_mask & (1 << portid)) == 0) continue; memset(&link, 0, sizeof(link)); @@ -1717,7 +1820,7 @@ init_port_config(void) portid_t pid; struct rte_port *port; - for (pid = 0; pid < nb_ports; pid++) { + FOREACH_PORT(pid, ports) { port = &ports[pid]; port->dev_conf.rxmode = rx_mode; port->dev_conf.fdir_conf = fdir_conf; @@ -1896,7 +1999,7 @@ main(int argc, char** argv) nb_ports = (portid_t) rte_eth_dev_count(); if (nb_ports == 0) - rte_exit(EXIT_FAILURE, "No probed ethernet device\n"); + RTE_LOG(WARNING, EAL, "No probed ethernet devices\n"); set_def_fwd_config(); if (nb_lcores == 0) @@ -1918,7 +2021,7 @@ main(int argc, char** argv) rte_exit(EXIT_FAILURE, "Start ports failed\n"); /* set all ports to promiscuous mode by default */ - for (port_id = 0; port_id < nb_ports; port_id++) + FOREACH_PORT(port_id, ports) rte_eth_promiscuous_enable(port_id); #ifdef RTE_LIBRTE_CMDLINE diff --git a/app/test-pmd/testpmd.h b/app/test-pmd/testpmd.h index 581130b..29b6755 100644 --- a/app/test-pmd/testpmd.h +++ b/app/test-pmd/testpmd.h @@ -137,6 +137,7 @@ struct fwd_stream { * The data structure associated with each port. */ struct rte_port { + uint8_t enabled; /**< Port enabled or not */ struct rte_eth_dev_info dev_info; /**< PCI info + driver name */ struct rte_eth_conf dev_conf; /**< Port configuration. */ struct ether_addr eth_addr; /**< Port ethernet address */ @@ -162,6 +163,14 @@ struct rte_port { struct rte_eth_txconf tx_conf; /**< tx configuration */ }; +extern portid_t __rte_unused +find_next_port(portid_t p, struct rte_port *ports, int size); + +#define FOREACH_PORT(p, ports) \ + for (p = find_next_port(0, ports, RTE_MAX_ETHPORTS); \ + p < RTE_MAX_ETHPORTS; \ + p = find_next_port(p + 1, ports, RTE_MAX_ETHPORTS)) + /** * The data structure associated with each forwarding logical core. * The logical cores are internally numbered by a core index from 0 to @@ -522,6 +531,8 @@ int init_port_dcb_config(portid_t pid,struct dcb_config *dcb_conf); int start_port(portid_t pid); void stop_port(portid_t pid); void close_port(portid_t pid); +void attach_port(char *identifier); +void detach_port(uint8_t port_id); int all_ports_stopped(void); int port_is_started(portid_t port_id); void pmd_test_exit(void); @@ -565,10 +576,15 @@ void get_ethertype_filter(uint8_t port_id, uint16_t index); void get_2tuple_filter(uint8_t port_id, uint16_t index); void get_5tuple_filter(uint8_t port_id, uint16_t index); void get_flex_filter(uint8_t port_id, uint16_t index); -int port_id_is_invalid(portid_t port_id); int rx_queue_id_is_invalid(queueid_t rxq_id); int tx_queue_id_is_invalid(queueid_t txq_id); +enum print_warning { + ENABLED_WARN = 0, + DISABLED_WARN +}; +int port_id_is_invalid(portid_t port_id, enum print_warning warning); + /* * Work-around of a compilation error with ICC on invocations of the * rte_be_to_cpu_16() function. diff --git a/doc/guides/testpmd_app_ug/testpmd_funcs.rst b/doc/guides/testpmd_app_ug/testpmd_funcs.rst index 218835a..936f9e0 100644 --- a/doc/guides/testpmd_app_ug/testpmd_funcs.rst +++ b/doc/guides/testpmd_app_ug/testpmd_funcs.rst @@ -808,6 +808,63 @@ The following sections show functions for configuring ports. Port configuration changes only become active when forwarding is started/restarted. +port attach +~~~~~~~~~~~ + +Attach a port specified by pci address or virtual device args. + +To attach a new pci device, the device should be recognized by kernel first. +Then it should be moved under DPDK management. +Finally the port can be attached to testpmd. +On the other hand, to attach a port created by virtual device, above steps are not needed. + +port attach (identifier) + +For example, to attach a port whose pci address is 0000:02:00.0. + +.. code-block:: console + + testpmd> port attach 0000:02:00.0 + Attaching a new port... + ... snip ... + Port 0 is attached. Now total ports is 1 + Done + +For example, to attach a port created by pcap PMD. + +.. code-block:: console + + testpmd> port attach eth_pcap0,iface=eth0 + Attaching a new port... + ... snip ... + Port 0 is attached. Now total ports is 1 + Done + +In this case, identifier is "eth_pcap0,iface=eth0". +This identifier format is the same as "--vdev" format of DPDK applications. + +port detach +~~~~~~~~~~~ + +Detach a specific port. + +Before detaching a port, the port should be closed. +Also to remove a pci device completely from the system, first detach the port from testpmd. +Then the device should be moved under kernel management. +Finally the device can be removed using kernel pci hotplug functionality. +On the other hand, to remove a port created by a virtual device, above steps are not needed. + +port detach (port_id) + +For example, to detach a port 0. + +.. code-block:: console + + testpmd> port detach 0 + Detaching a port... + ... snip ... + Done + port start ~~~~~~~~~~ -- 1.9.1
> @@ -152,6 +159,7 @@ struct rte_pci_device { > uint16_t max_vfs; /**< sriov enable if not zero */ > int numa_node; /**< NUMA node connection */ > struct rte_devargs *devargs; /**< Device user arguments */ > + enum rte_pt_driver pt_driver; /**< Driver of passthrough */ [...] > +static int > +pci_get_kernel_driver_by_path(const char *filename, char *dri_name) I think "kernel driver" is a good name. Why not using this name in the pci_device struct to be more consistent? Thanks
2015-02-19 11:49, Tetsuya Mukawa:
> --- a/lib/librte_ether/rte_ether_version.map
> +++ b/lib/librte_ether/rte_ether_version.map
> @@ -109,6 +109,13 @@ DPDK_2.0 {
> rte_eth_tx_queue_setup;
> rte_eth_xstats_get;
> rte_eth_xstats_reset;
> + rte_eth_dev_allocated;
> + rte_eth_dev_is_detachable;
> + rte_eth_dev_get_name_by_port;
> + rte_eth_dev_get_addr_by_port;
> + rte_eth_dev_get_port_by_addr;
> + rte_eth_dev_get_changed_port;
> + rte_eth_dev_save;
In this file, alphabetical order is preferred.
2015-02-19 11:49, Tetsuya Mukawa: > +/* attach the new virtual device, then store port_id of the device */ > +static int > +rte_eal_dev_attach_vdev(const char *vdevargs, uint8_t *port_id) > +{ > + char *args; > + uint8_t new_port_id; > + struct rte_eth_dev devs[RTE_MAX_ETHPORTS]; > + > + if ((vdevargs == NULL) || (port_id == NULL)) > + goto err0; > + > + args = strdup(vdevargs); > + if (args == NULL) > + goto err0; > + > + /* save current port status */ > + if (rte_eth_dev_save(devs, sizeof(devs))) > + goto err1; > + /* add the vdevargs to devargs_list */ > + if (rte_eal_devargs_add(RTE_DEVTYPE_VIRTUAL, args)) > + goto err1; Could you explain why you store devargs in a list? > + /* parse vdevargs, then retrieve device name */ > + get_vdev_name(args); > + /* walk around dev_driver_list to find the driver of the device, > + * then invoke probe function o the driver */ > + if (rte_eal_vdev_find_and_init(args)) TODO: get port_id from init. > + goto err2; > + /* get port_id enabled by above procedures */ > + if (rte_eth_dev_get_changed_port(devs, &new_port_id)) > + goto err2; [...] > --- a/lib/librte_eal/common/include/rte_dev.h > +++ b/lib/librte_eal/common/include/rte_dev.h > @@ -47,6 +47,7 @@ extern "C" { > #endif > > #include <sys/queue.h> > +#include <rte_pci.h> > > /** Double linked list of device drivers. */ > TAILQ_HEAD(rte_driver_list, rte_driver); > @@ -57,6 +58,11 @@ TAILQ_HEAD(rte_driver_list, rte_driver); > typedef int (rte_dev_init_t)(const char *name, const char *args); > > /** > + * Uninitilization function called for each device driver once. > + */ > +typedef int (rte_dev_uninit_t)(const char *name); Why using name as parameter and not port_id? [...] > --- a/lib/librte_eal/linuxapp/eal/Makefile > +++ b/lib/librte_eal/linuxapp/eal/Makefile > @@ -45,6 +45,7 @@ CFLAGS += -I$(RTE_SDK)/lib/librte_eal/common/include > CFLAGS += -I$(RTE_SDK)/lib/librte_ring > CFLAGS += -I$(RTE_SDK)/lib/librte_mempool > CFLAGS += -I$(RTE_SDK)/lib/librte_malloc > +CFLAGS += -I$(RTE_SDK)/lib/librte_mbuf Why do you need mbuf? [...] > --- a/lib/librte_eal/linuxapp/eal/rte_eal_version.map > +++ b/lib/librte_eal/linuxapp/eal/rte_eal_version.map > @@ -21,6 +21,8 @@ DPDK_2.0 { > rte_eal_alarm_cancel; > rte_eal_alarm_set; > rte_eal_dev_init; > + rte_eal_dev_attach; > + rte_eal_dev_detach; Please keep alphabetical order.
On 2015/02/19 20:17, Thomas Monjalon wrote: >> @@ -152,6 +159,7 @@ struct rte_pci_device { >> uint16_t max_vfs; /**< sriov enable if not zero */ >> int numa_node; /**< NUMA node connection */ >> struct rte_devargs *devargs; /**< Device user arguments */ >> + enum rte_pt_driver pt_driver; /**< Driver of passthrough */ > [...] >> +static int >> +pci_get_kernel_driver_by_path(const char *filename, char *dri_name) > I think "kernel driver" is a good name. Why not using this name in the > pci_device struct to be more consistent? Hi Michael, Could you please let me know what do you think about it? Thanks Tetsuya > Thanks
On 2015/02/19 20:24, Thomas Monjalon wrote:
> 2015-02-19 11:49, Tetsuya Mukawa:
>> --- a/lib/librte_ether/rte_ether_version.map
>> +++ b/lib/librte_ether/rte_ether_version.map
>> @@ -109,6 +109,13 @@ DPDK_2.0 {
>> rte_eth_tx_queue_setup;
>> rte_eth_xstats_get;
>> rte_eth_xstats_reset;
>> + rte_eth_dev_allocated;
>> + rte_eth_dev_is_detachable;
>> + rte_eth_dev_get_name_by_port;
>> + rte_eth_dev_get_addr_by_port;
>> + rte_eth_dev_get_port_by_addr;
>> + rte_eth_dev_get_changed_port;
>> + rte_eth_dev_save;
> In this file, alphabetical order is preferred.
>
Hi Thomas,
Sure, I will change order.
Tetsuya
On 2015/02/19 21:10, Thomas Monjalon wrote: > 2015-02-19 11:49, Tetsuya Mukawa: >> +/* attach the new virtual device, then store port_id of the device */ >> +static int >> +rte_eal_dev_attach_vdev(const char *vdevargs, uint8_t *port_id) >> +{ >> + char *args; >> + uint8_t new_port_id; >> + struct rte_eth_dev devs[RTE_MAX_ETHPORTS]; >> + >> + if ((vdevargs == NULL) || (port_id == NULL)) >> + goto err0; >> + >> + args = strdup(vdevargs); >> + if (args == NULL) >> + goto err0; >> + >> + /* save current port status */ >> + if (rte_eth_dev_save(devs, sizeof(devs))) >> + goto err1; >> + /* add the vdevargs to devargs_list */ >> + if (rte_eal_devargs_add(RTE_DEVTYPE_VIRTUAL, args)) >> + goto err1; > Could you explain why you store devargs in a list? I try to do same behavior when rte_eal_init() is called. If only hotplug doesn't do this, someone may be confused when they try to realize devargs_list. >> + /* parse vdevargs, then retrieve device name */ >> + get_vdev_name(args); >> + /* walk around dev_driver_list to find the driver of the device, >> + * then invoke probe function o the driver */ >> + if (rte_eal_vdev_find_and_init(args)) > TODO: get port_id from init. Yes, I will. I also add comment about it. >> + goto err2; >> + /* get port_id enabled by above procedures */ >> + if (rte_eth_dev_get_changed_port(devs, &new_port_id)) >> + goto err2; > [...] >> --- a/lib/librte_eal/common/include/rte_dev.h >> +++ b/lib/librte_eal/common/include/rte_dev.h >> @@ -47,6 +47,7 @@ extern "C" { >> #endif >> >> #include <sys/queue.h> >> +#include <rte_pci.h> >> >> /** Double linked list of device drivers. */ >> TAILQ_HEAD(rte_driver_list, rte_driver); >> @@ -57,6 +58,11 @@ TAILQ_HEAD(rte_driver_list, rte_driver); >> typedef int (rte_dev_init_t)(const char *name, const char *args); >> >> /** >> + * Uninitilization function called for each device driver once. >> + */ >> +typedef int (rte_dev_uninit_t)(const char *name); > Why using name as parameter and not port_id? This function pointer will be implemented in PMDs. For example, in pcap PMD, rte_pmd_pcap_devuninit() is the function. static struct rte_driver pmd_pcap_drv = { .name = "eth_pcap", .type = PMD_VDEV, .init = rte_pmd_pcap_devinit, .uninit = rte_pmd_pcap_devuninit, }; "port_id" isn't needed in PMD. > > [...] >> --- a/lib/librte_eal/linuxapp/eal/Makefile >> +++ b/lib/librte_eal/linuxapp/eal/Makefile >> @@ -45,6 +45,7 @@ CFLAGS += -I$(RTE_SDK)/lib/librte_eal/common/include >> CFLAGS += -I$(RTE_SDK)/lib/librte_ring >> CFLAGS += -I$(RTE_SDK)/lib/librte_mempool >> CFLAGS += -I$(RTE_SDK)/lib/librte_malloc >> +CFLAGS += -I$(RTE_SDK)/lib/librte_mbuf > Why do you need mbuf? To include rte_ethdev.h in this code, rte_mbuf.h is also needed. > [...] >> --- a/lib/librte_eal/linuxapp/eal/rte_eal_version.map >> +++ b/lib/librte_eal/linuxapp/eal/rte_eal_version.map >> @@ -21,6 +21,8 @@ DPDK_2.0 { >> rte_eal_alarm_cancel; >> rte_eal_alarm_set; >> rte_eal_dev_init; >> + rte_eal_dev_attach; >> + rte_eal_dev_detach; > Please keep alphabetical order. > Sure, I will. Thanks, Tetsuya
On 2015/02/19 22:30, Tetsuya Mukawa wrote: > On 2015/02/19 21:10, Thomas Monjalon wrote: >> 2015-02-19 11:49, Tetsuya Mukawa: >>> +/* attach the new virtual device, then store port_id of the device */ >>> +static int >>> +rte_eal_dev_attach_vdev(const char *vdevargs, uint8_t *port_id) >>> +{ >>> + char *args; >>> + uint8_t new_port_id; >>> + struct rte_eth_dev devs[RTE_MAX_ETHPORTS]; >>> + >>> + if ((vdevargs == NULL) || (port_id == NULL)) >>> + goto err0; >>> + >>> + args = strdup(vdevargs); >>> + if (args == NULL) >>> + goto err0; >>> + >>> + /* save current port status */ >>> + if (rte_eth_dev_save(devs, sizeof(devs))) >>> + goto err1; >>> + /* add the vdevargs to devargs_list */ >>> + if (rte_eal_devargs_add(RTE_DEVTYPE_VIRTUAL, args)) >>> + goto err1; >> Could you explain why you store devargs in a list? > I try to do same behavior when rte_eal_init() is called. Sorry for lack of explanation. "vdevargs" of rte_eal_dev_attach_vdev() will be same format of "--vdev" option. And when rte_eal_init() is called, such a "--vdev" option value will be stored in devargs_list. So I try to same thing here. > If only hotplug doesn't do this, someone may be confused when they try > to realize devargs_list. > >>> + /* parse vdevargs, then retrieve device name */ >>> + get_vdev_name(args); >>> + /* walk around dev_driver_list to find the driver of the device, >>> + * then invoke probe function o the driver */ >>> + if (rte_eal_vdev_find_and_init(args)) >> TODO: get port_id from init. > Yes, I will. > I also add comment about it. > >>> + goto err2; >>> + /* get port_id enabled by above procedures */ >>> + if (rte_eth_dev_get_changed_port(devs, &new_port_id)) >>> + goto err2; >> [...] >>> --- a/lib/librte_eal/common/include/rte_dev.h >>> +++ b/lib/librte_eal/common/include/rte_dev.h >>> @@ -47,6 +47,7 @@ extern "C" { >>> #endif >>> >>> #include <sys/queue.h> >>> +#include <rte_pci.h> >>> >>> /** Double linked list of device drivers. */ >>> TAILQ_HEAD(rte_driver_list, rte_driver); >>> @@ -57,6 +58,11 @@ TAILQ_HEAD(rte_driver_list, rte_driver); >>> typedef int (rte_dev_init_t)(const char *name, const char *args); >>> >>> /** >>> + * Uninitilization function called for each device driver once. >>> + */ >>> +typedef int (rte_dev_uninit_t)(const char *name); >> Why using name as parameter and not port_id? > This function pointer will be implemented in PMDs. > For example, in pcap PMD, rte_pmd_pcap_devuninit() is the function. > > static struct rte_driver pmd_pcap_drv = { > .name = "eth_pcap", > .type = PMD_VDEV, > .init = rte_pmd_pcap_devinit, > .uninit = rte_pmd_pcap_devuninit, > }; > > "port_id" isn't needed in PMD. > >> [...] >>> --- a/lib/librte_eal/linuxapp/eal/Makefile >>> +++ b/lib/librte_eal/linuxapp/eal/Makefile >>> @@ -45,6 +45,7 @@ CFLAGS += -I$(RTE_SDK)/lib/librte_eal/common/include >>> CFLAGS += -I$(RTE_SDK)/lib/librte_ring >>> CFLAGS += -I$(RTE_SDK)/lib/librte_mempool >>> CFLAGS += -I$(RTE_SDK)/lib/librte_malloc >>> +CFLAGS += -I$(RTE_SDK)/lib/librte_mbuf >> Why do you need mbuf? > To include rte_ethdev.h in this code, rte_mbuf.h is also needed. > >> [...] >>> --- a/lib/librte_eal/linuxapp/eal/rte_eal_version.map >>> +++ b/lib/librte_eal/linuxapp/eal/rte_eal_version.map >>> @@ -21,6 +21,8 @@ DPDK_2.0 { >>> rte_eal_alarm_cancel; >>> rte_eal_alarm_set; >>> rte_eal_dev_init; >>> + rte_eal_dev_attach; >>> + rte_eal_dev_detach; >> Please keep alphabetical order. >> > Sure, I will. > > Thanks, > Tetsuya >
> -----Original Message----- > From: Tetsuya Mukawa [mailto:mukawa@igel.co.jp] > Sent: Thursday, February 19, 2015 2:50 AM > To: dev@dpdk.org > Cc: Qiu, Michael; Iremonger, Bernard; thomas.monjalon@6wind.com; Tetsuya Mukawa > Subject: [PATCH v9 07/14] eal,ethdev: Add a function and function pointers to close ether device > > The patch adds function pointer to rte_pci_driver and eth_driver structure. These function pointers > are used when ports are detached. > Also, the patch adds rte_eth_dev_uninit(). So far, it's not called by anywhere, but it will be called > when port hotplug function is implemented. > > v9: > - Change parameter of pci_devuninit_t and rte_eth_dev_uninit. > - Remove code that initiaize callback of ethdev from > rte_eth_dev_uninit(). > - Add a function to create a unique device name. > (Thanks to Thomas Monjalon) > v6: > - Fix rte_eth_dev_uninit() to handle a return value of uninit > function of PMD. > v4: > - Add parameter checking. > - Change function names. > > Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp> > --- > lib/librte_eal/common/include/rte_pci.h | 6 ++++ > lib/librte_ether/rte_ethdev.c | 62 +++++++++++++++++++++++++++++++-- > lib/librte_ether/rte_ethdev.h | 24 +++++++++++++ > 3 files changed, 90 insertions(+), 2 deletions(-) > > diff --git a/lib/librte_eal/common/include/rte_pci.h b/lib/librte_eal/common/include/rte_pci.h > index c609ef3..376f66a 100644 > --- a/lib/librte_eal/common/include/rte_pci.h > +++ b/lib/librte_eal/common/include/rte_pci.h > @@ -189,12 +189,18 @@ struct rte_pci_driver; typedef int (pci_devinit_t)(struct rte_pci_driver *, > struct rte_pci_device *); > > /** > + * Uninitialisation function for the driver called during hotplugging. > + */ > +typedef int (pci_devuninit_t)(struct rte_pci_device *); > + > +/** > * A structure describing a PCI driver. > */ > struct rte_pci_driver { > TAILQ_ENTRY(rte_pci_driver) next; /**< Next in list. */ > const char *name; /**< Driver name. */ > pci_devinit_t *devinit; /**< Device init. function. */ > + pci_devuninit_t *devuninit; /**< Device uninit function. */ > struct rte_pci_id *id_table; /**< ID table, NULL terminated. */ > uint32_t drv_flags; /**< Flags contolling handling of device. */ > }; > diff --git a/lib/librte_ether/rte_ethdev.c b/lib/librte_ether/rte_ethdev.c index be5aa18..ef5d226 > 100644 > --- a/lib/librte_ether/rte_ethdev.c > +++ b/lib/librte_ether/rte_ethdev.c > @@ -266,6 +266,24 @@ rte_eth_dev_release_port(struct rte_eth_dev *eth_dev) > return 0; > } > > +static inline int > +rte_eth_dev_create_unique_device_name(char *name, > + struct rte_pci_device *pci_dev) > +{ > + int ret; > + > + if ((name == NULL) || (pci_dev == NULL)) > + return -EINVAL; > + Hi Tetsuya, It would be safer to pass in the size of the name buffer and use it in the snprintf() call . Regards, Bernard. > + ret = snprintf(name, RTE_ETH_NAME_MAX_LEN, "%d:%d.%d", > + pci_dev->addr.bus, pci_dev->addr.devid, > + pci_dev->addr.function); > + if (ret < 0) > + return ret; > + > + return 0; > +} > + > static int > rte_eth_dev_init(struct rte_pci_driver *pci_drv, > struct rte_pci_device *pci_dev) > @@ -279,8 +297,7 @@ rte_eth_dev_init(struct rte_pci_driver *pci_drv, > eth_drv = (struct eth_driver *)pci_drv; > > /* Create unique Ethernet device name using PCI address */ > - snprintf(ethdev_name, RTE_ETH_NAME_MAX_LEN, "%d:%d.%d", > - pci_dev->addr.bus, pci_dev->addr.devid, pci_dev->addr.function); > + rte_eth_dev_create_unique_device_name(ethdev_name, pci_dev); > > eth_dev = rte_eth_dev_allocate(ethdev_name); > if (eth_dev == NULL) > @@ -321,6 +338,46 @@ rte_eth_dev_init(struct rte_pci_driver *pci_drv, > return diag; > } > > +static int > +rte_eth_dev_uninit(struct rte_pci_device *pci_dev) { > + const struct eth_driver *eth_drv; > + struct rte_eth_dev *eth_dev; > + char ethdev_name[RTE_ETH_NAME_MAX_LEN]; > + int ret; > + > + if (pci_dev == NULL) > + return -EINVAL; > + > + /* Create unique Ethernet device name using PCI address */ > + rte_eth_dev_create_unique_device_name(ethdev_name, pci_dev); > + > + eth_dev = rte_eth_dev_allocated(ethdev_name); > + if (eth_dev == NULL) > + return -ENODEV; > + > + eth_drv = (const struct eth_driver *)pci_dev->driver; > + > + /* Invoke PMD device uninit function */ > + if (*eth_drv->eth_dev_uninit) { > + ret = (*eth_drv->eth_dev_uninit)(eth_drv, eth_dev); > + if (ret) > + return ret; > + } > + > + /* free ether device */ > + rte_eth_dev_release_port(eth_dev); > + > + if (rte_eal_process_type() == RTE_PROC_PRIMARY) > + rte_free(eth_dev->data->dev_private); > + > + eth_dev->pci_dev = NULL; > + eth_dev->driver = NULL; > + eth_dev->data = NULL; > + > + return 0; > +} > + > /** > * Register an Ethernet [Poll Mode] driver. > * > @@ -339,6 +396,7 @@ void > rte_eth_driver_register(struct eth_driver *eth_drv) { > eth_drv->pci_drv.devinit = rte_eth_dev_init; > + eth_drv->pci_drv.devuninit = rte_eth_dev_uninit; > rte_eal_pci_register(ð_drv->pci_drv); > } > > diff --git a/lib/librte_ether/rte_ethdev.h b/lib/librte_ether/rte_ethdev.h index 28ecafd..f403780 > 100644 > --- a/lib/librte_ether/rte_ethdev.h > +++ b/lib/librte_ether/rte_ethdev.h > @@ -1677,6 +1677,27 @@ typedef int (*eth_dev_init_t)(struct eth_driver *eth_drv, > > /** > * @internal > + * Finalization function of an Ethernet driver invoked for each > +matching > + * Ethernet PCI device detected during the PCI closing phase. > + * > + * @param eth_drv > + * The pointer to the [matching] Ethernet driver structure supplied by > + * the PMD when it registered itself. > + * @param eth_dev > + * The *eth_dev* pointer is the address of the *rte_eth_dev* structure > + * associated with the matching device and which have been [automatically] > + * allocated in the *rte_eth_devices* array. > + * @return > + * - 0: Success, the device is properly finalized by the driver. > + * In particular, the driver MUST free the *dev_ops* pointer > + * of the *eth_dev* structure. > + * - <0: Error code of the device initialization failure. > + */ > +typedef int (*eth_dev_uninit_t)(const struct eth_driver *eth_drv, > + struct rte_eth_dev *eth_dev); > + > +/** > + * @internal > * The structure associated with a PMD Ethernet driver. > * > * Each Ethernet driver acts as a PCI driver and is represented by a generic @@ -1686,11 +1707,14 @@ > typedef int (*eth_dev_init_t)(struct eth_driver *eth_drv, > * > * - The *eth_dev_init* function invoked for each matching PCI device. > * > + * - The *eth_dev_uninit* function invoked for each matching PCI device. > + * > * - The size of the private data to allocate for each matching device. > */ > struct eth_driver { > struct rte_pci_driver pci_drv; /**< The PMD is also a PCI driver. */ > eth_dev_init_t eth_dev_init; /**< Device init function. */ > + eth_dev_uninit_t eth_dev_uninit; /**< Device uninit function. */ > unsigned int dev_private_size; /**< Size of device private data. */ > }; > > -- > 1.9.1
> -----Original Message----- > From: Tetsuya Mukawa [mailto:mukawa@igel.co.jp] > Sent: Thursday, February 19, 2015 2:50 AM > To: dev@dpdk.org > Cc: Qiu, Michael; Iremonger, Bernard; thomas.monjalon@6wind.com; Tetsuya Mukawa > Subject: [PATCH v9 12/14] ethdev: Add one dev_type parameter to rte_eth_dev_allocate > > This new parameter is needed to keep device type like PCI or virtual. > Port detaching processes are different between PCI device and virtual device. > RTE_ETH_DEV_PCI indicates device type is PCI. RTE_ETH_DEV_VIRTUAL indicates device is virtual. > > v9: > - Fix commit log. > - RTE_ETH_DEV_PHYSICAL is replaced by RTE_ETH_DEV_PCI. > (Thanks to Thomas Monjalon) > v8: > - NONE_TRACE is replaced by NO_TRACE. > - Add missing symbol in version map. > (Thanks to Qiu, Michael and Iremonger, Bernard) > v4: > - Fix comments of rte_eth_dev_type. > > Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp> > --- > app/test/virtual_pmd.c | 2 +- > lib/librte_ether/rte_ethdev.c | 25 +++++++++++++++++++++++-- > lib/librte_ether/rte_ethdev.h | 25 ++++++++++++++++++++++++- > lib/librte_ether/rte_ether_version.map | 1 + > lib/librte_pmd_af_packet/rte_eth_af_packet.c | 2 +- > lib/librte_pmd_bond/rte_eth_bond_api.c | 2 +- > lib/librte_pmd_pcap/rte_eth_pcap.c | 2 +- > lib/librte_pmd_ring/rte_eth_ring.c | 2 +- > lib/librte_pmd_xenvirt/rte_eth_xenvirt.c | 2 +- > 9 files changed, 54 insertions(+), 9 deletions(-) > > diff --git a/app/test/virtual_pmd.c b/app/test/virtual_pmd.c index 9fac95d..c02644a 100644 > --- a/app/test/virtual_pmd.c > +++ b/app/test/virtual_pmd.c > @@ -556,7 +556,7 @@ virtual_ethdev_create(const char *name, struct ether_addr *mac_addr, > goto err; > > /* reserve an ethdev entry */ > - eth_dev = rte_eth_dev_allocate(name); > + eth_dev = rte_eth_dev_allocate(name, RTE_ETH_DEV_PCI); > if (eth_dev == NULL) > goto err; > > diff --git a/lib/librte_ether/rte_ethdev.c b/lib/librte_ether/rte_ethdev.c index 3b64f3a..201c04a > 100644 > --- a/lib/librte_ether/rte_ethdev.c > +++ b/lib/librte_ether/rte_ethdev.c > @@ -227,7 +227,7 @@ rte_eth_dev_find_free_port(void) } > > struct rte_eth_dev * > -rte_eth_dev_allocate(const char *name) > +rte_eth_dev_allocate(const char *name, enum rte_eth_dev_type type) > { > uint8_t port_id; > struct rte_eth_dev *eth_dev; > @@ -251,6 +251,7 @@ rte_eth_dev_allocate(const char *name) > snprintf(eth_dev->data->name, sizeof(eth_dev->data->name), "%s", name); > eth_dev->data->port_id = port_id; > eth_dev->attached = DEV_ATTACHED; > + eth_dev->dev_type = type; > nb_ports++; > return eth_dev; > } > @@ -262,6 +263,7 @@ rte_eth_dev_release_port(struct rte_eth_dev *eth_dev) > return -EINVAL; > > eth_dev->attached = 0; > + eth_dev->dev_type = RTE_ETH_DEV_UNKNOWN; > nb_ports--; > return 0; > } > @@ -299,7 +301,7 @@ rte_eth_dev_init(struct rte_pci_driver *pci_drv, > /* Create unique Ethernet device name using PCI address */ > rte_eth_dev_create_unique_device_name(ethdev_name, pci_dev); > > - eth_dev = rte_eth_dev_allocate(ethdev_name); > + eth_dev = rte_eth_dev_allocate(ethdev_name, RTE_ETH_DEV_PCI); > if (eth_dev == NULL) > return -ENOMEM; > > @@ -424,6 +426,14 @@ rte_eth_dev_count(void) > return (nb_ports); > } > > +enum rte_eth_dev_type > +rte_eth_dev_get_device_type(uint8_t port_id) { > + if (!rte_eth_dev_is_valid_port(port_id)) > + return -1; > + return rte_eth_devices[port_id].dev_type; > +} > + > int > rte_eth_dev_save(struct rte_eth_dev *devs, size_t size) { @@ -521,6 +531,17 @@ > rte_eth_dev_is_detachable(uint8_t port_id) > return -EINVAL; > } > > + if (rte_eth_devices[port_id].dev_type == RTE_ETH_DEV_PCI) { > + switch (rte_eth_devices[port_id].pci_dev->pt_driver) { > + case RTE_PT_IGB_UIO: > + case RTE_PT_UIO_GENERIC: > + break; > + case RTE_PT_VFIO: > + default: > + return -ENOTSUP; > + } > + } > + > drv_flags = rte_eth_devices[port_id].driver->pci_drv.drv_flags; > return !(drv_flags & RTE_PCI_DRV_DETACHABLE); } diff --git a/lib/librte_ether/rte_ethdev.h > b/lib/librte_ether/rte_ethdev.h index 90b7f25..4f66bb6 100644 > --- a/lib/librte_ether/rte_ethdev.h > +++ b/lib/librte_ether/rte_ethdev.h > @@ -1523,6 +1523,17 @@ struct eth_dev_ops { }; > > /** > + * The eth device type > + */ > +enum rte_eth_dev_type { > + RTE_ETH_DEV_UNKNOWN, /**< unknown device type */ > + RTE_ETH_DEV_PCI, > + /**< Physical function and Virtual function of PCI devices */ > + RTE_ETH_DEV_VIRTUAL, /**< non hardware device */ > + RTE_ETH_DEV_MAX /**< max value of this enum */ > +}; > + > +/** > * @internal > * The generic data structure associated with each ethernet device. > * > @@ -1541,6 +1552,7 @@ struct rte_eth_dev { > struct rte_pci_device *pci_dev; /**< PCI info. supplied by probing */ > struct rte_eth_dev_cb_list callbacks; /**< User application callbacks */ > uint8_t attached; /**< Flag indicating the port is attached */ > + enum rte_eth_dev_type dev_type; /**< Flag indicating the device type > +*/ > }; > > struct rte_eth_dev_sriov { > @@ -1618,6 +1630,15 @@ extern uint8_t rte_eth_dev_count(void); > > /** > * Function for internal use by port hotplug functions. > + * Get the device type to know whether the device is physical or virtual. > + * @param port_id The pointer to the port id Hi Tetsuya, " The pointer to the port id" should be removed. Regards, Bernard. > + * @return > + * - Device type. > + */ > +extern enum rte_eth_dev_type rte_eth_dev_get_device_type(uint8_t > +port_id); > + > +/** > + * Function for internal use by port hotplug functions. > * Copies current ethdev structures to the specified pointer. > * > * @param devs The pointer to the ethdev structures > @@ -1706,10 +1727,12 @@ extern struct rte_eth_dev *rte_eth_dev_allocated(const char *name); > * to that slot for the driver to use. > * > * @param name Unique identifier name for each Ethernet device > + * @param type Device type of this Ethernet device > * @return > * - Slot in the rte_dev_devices array for a new device; > */ > -struct rte_eth_dev *rte_eth_dev_allocate(const char *name); > +struct rte_eth_dev *rte_eth_dev_allocate(const char *name, > + enum rte_eth_dev_type type); > > /** > * Function for internal use by dummy drivers primarily, e.g. ring-based diff --git > a/lib/librte_ether/rte_ether_version.map b/lib/librte_ether/rte_ether_version.map > index 1d039ed..b78f5e3 100644 > --- a/lib/librte_ether/rte_ether_version.map > +++ b/lib/librte_ether/rte_ether_version.map > @@ -116,6 +116,7 @@ DPDK_2.0 { > rte_eth_dev_get_port_by_addr; > rte_eth_dev_get_changed_port; > rte_eth_dev_save; > + rte_eth_dev_get_device_type; > > local: *; > }; > diff --git a/lib/librte_pmd_af_packet/rte_eth_af_packet.c > b/lib/librte_pmd_af_packet/rte_eth_af_packet.c > index 1ffe1cd..80e9bdf 100644 > --- a/lib/librte_pmd_af_packet/rte_eth_af_packet.c > +++ b/lib/librte_pmd_af_packet/rte_eth_af_packet.c > @@ -649,7 +649,7 @@ rte_pmd_init_internals(const char *name, > } > > /* reserve an ethdev entry */ > - *eth_dev = rte_eth_dev_allocate(name); > + *eth_dev = rte_eth_dev_allocate(name, RTE_ETH_DEV_VIRTUAL); > if (*eth_dev == NULL) > goto error; > > diff --git a/lib/librte_pmd_bond/rte_eth_bond_api.c b/lib/librte_pmd_bond/rte_eth_bond_api.c > index 4ab3267..7a6a5f7 100644 > --- a/lib/librte_pmd_bond/rte_eth_bond_api.c > +++ b/lib/librte_pmd_bond/rte_eth_bond_api.c > @@ -235,7 +235,7 @@ rte_eth_bond_create(const char *name, uint8_t mode, uint8_t socket_id) > } > > /* reserve an ethdev entry */ > - eth_dev = rte_eth_dev_allocate(name); > + eth_dev = rte_eth_dev_allocate(name, RTE_ETH_DEV_VIRTUAL); > if (eth_dev == NULL) { > RTE_BOND_LOG(ERR, "Unable to allocate rte_eth_dev"); > goto err; > diff --git a/lib/librte_pmd_pcap/rte_eth_pcap.c b/lib/librte_pmd_pcap/rte_eth_pcap.c > index d299288..af7fae8 100644 > --- a/lib/librte_pmd_pcap/rte_eth_pcap.c > +++ b/lib/librte_pmd_pcap/rte_eth_pcap.c > @@ -709,7 +709,7 @@ rte_pmd_init_internals(const char *name, const unsigned nb_rx_queues, > goto error; > > /* reserve an ethdev entry */ > - *eth_dev = rte_eth_dev_allocate(name); > + *eth_dev = rte_eth_dev_allocate(name, RTE_ETH_DEV_VIRTUAL); > if (*eth_dev == NULL) > goto error; > > diff --git a/lib/librte_pmd_ring/rte_eth_ring.c b/lib/librte_pmd_ring/rte_eth_ring.c > index d5b1686..c837fcd 100644 > --- a/lib/librte_pmd_ring/rte_eth_ring.c > +++ b/lib/librte_pmd_ring/rte_eth_ring.c > @@ -255,7 +255,7 @@ rte_eth_from_rings(const char *name, struct rte_ring *const rx_queues[], > goto error; > > /* reserve an ethdev entry */ > - eth_dev = rte_eth_dev_allocate(name); > + eth_dev = rte_eth_dev_allocate(name, RTE_ETH_DEV_VIRTUAL); > if (eth_dev == NULL) > goto error; > > diff --git a/lib/librte_pmd_xenvirt/rte_eth_xenvirt.c b/lib/librte_pmd_xenvirt/rte_eth_xenvirt.c > index 04e30c9..bc403d6 100644 > --- a/lib/librte_pmd_xenvirt/rte_eth_xenvirt.c > +++ b/lib/librte_pmd_xenvirt/rte_eth_xenvirt.c > @@ -648,7 +648,7 @@ eth_dev_xenvirt_create(const char *name, const char *params, > goto err; > > /* reserve an ethdev entry */ > - eth_dev = rte_eth_dev_allocate(name); > + eth_dev = rte_eth_dev_allocate(name, RTE_ETH_DEV_VIRTUAL); > if (eth_dev == NULL) > goto err; > > -- > 1.9.1
On 2015/02/19 23:31, Iremonger, Bernard wrote: > >> -----Original Message----- >> From: Tetsuya Mukawa [mailto:mukawa@igel.co.jp] >> Sent: Thursday, February 19, 2015 2:50 AM >> To: dev@dpdk.org >> Cc: Qiu, Michael; Iremonger, Bernard; thomas.monjalon@6wind.com; Tetsuya Mukawa >> Subject: [PATCH v9 07/14] eal,ethdev: Add a function and function pointers to close ether device >> >> The patch adds function pointer to rte_pci_driver and eth_driver structure. These function pointers >> are used when ports are detached. >> Also, the patch adds rte_eth_dev_uninit(). So far, it's not called by anywhere, but it will be called >> when port hotplug function is implemented. >> >> v9: >> - Change parameter of pci_devuninit_t and rte_eth_dev_uninit. >> - Remove code that initiaize callback of ethdev from >> rte_eth_dev_uninit(). >> - Add a function to create a unique device name. >> (Thanks to Thomas Monjalon) >> v6: >> - Fix rte_eth_dev_uninit() to handle a return value of uninit >> function of PMD. >> v4: >> - Add parameter checking. >> - Change function names. >> >> Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp> >> --- >> lib/librte_eal/common/include/rte_pci.h | 6 ++++ >> lib/librte_ether/rte_ethdev.c | 62 +++++++++++++++++++++++++++++++-- >> lib/librte_ether/rte_ethdev.h | 24 +++++++++++++ >> 3 files changed, 90 insertions(+), 2 deletions(-) >> >> diff --git a/lib/librte_eal/common/include/rte_pci.h b/lib/librte_eal/common/include/rte_pci.h >> index c609ef3..376f66a 100644 >> --- a/lib/librte_eal/common/include/rte_pci.h >> +++ b/lib/librte_eal/common/include/rte_pci.h >> @@ -189,12 +189,18 @@ struct rte_pci_driver; typedef int (pci_devinit_t)(struct rte_pci_driver *, >> struct rte_pci_device *); >> >> /** >> + * Uninitialisation function for the driver called during hotplugging. >> + */ >> +typedef int (pci_devuninit_t)(struct rte_pci_device *); >> + >> +/** >> * A structure describing a PCI driver. >> */ >> struct rte_pci_driver { >> TAILQ_ENTRY(rte_pci_driver) next; /**< Next in list. */ >> const char *name; /**< Driver name. */ >> pci_devinit_t *devinit; /**< Device init. function. */ >> + pci_devuninit_t *devuninit; /**< Device uninit function. */ >> struct rte_pci_id *id_table; /**< ID table, NULL terminated. */ >> uint32_t drv_flags; /**< Flags contolling handling of device. */ >> }; >> diff --git a/lib/librte_ether/rte_ethdev.c b/lib/librte_ether/rte_ethdev.c index be5aa18..ef5d226 >> 100644 >> --- a/lib/librte_ether/rte_ethdev.c >> +++ b/lib/librte_ether/rte_ethdev.c >> @@ -266,6 +266,24 @@ rte_eth_dev_release_port(struct rte_eth_dev *eth_dev) >> return 0; >> } >> >> +static inline int >> +rte_eth_dev_create_unique_device_name(char *name, >> + struct rte_pci_device *pci_dev) >> +{ >> + int ret; >> + >> + if ((name == NULL) || (pci_dev == NULL)) >> + return -EINVAL; >> + > Hi Tetsuya, > > It would be safer to pass in the size of the name buffer and use it in the snprintf() call . Hi Bernard, Thanks, I will do it. Tetsuya. > > Regards, > > Bernard. > > >> + ret = snprintf(name, RTE_ETH_NAME_MAX_LEN, "%d:%d.%d", >> + pci_dev->addr.bus, pci_dev->addr.devid, >> + pci_dev->addr.function); >> + if (ret < 0) >> + return ret; >> + >> + return 0; >> +} >> + >> static int >> rte_eth_dev_init(struct rte_pci_driver *pci_drv, >> struct rte_pci_device *pci_dev) >> @@ -279,8 +297,7 @@ rte_eth_dev_init(struct rte_pci_driver *pci_drv, >> eth_drv = (struct eth_driver *)pci_drv; >> >> /* Create unique Ethernet device name using PCI address */ >> - snprintf(ethdev_name, RTE_ETH_NAME_MAX_LEN, "%d:%d.%d", >> - pci_dev->addr.bus, pci_dev->addr.devid, pci_dev->addr.function); >> + rte_eth_dev_create_unique_device_name(ethdev_name, pci_dev); >> >> eth_dev = rte_eth_dev_allocate(ethdev_name); >> if (eth_dev == NULL) >> @@ -321,6 +338,46 @@ rte_eth_dev_init(struct rte_pci_driver *pci_drv, >> return diag; >> } >> >> +static int >> +rte_eth_dev_uninit(struct rte_pci_device *pci_dev) { >> + const struct eth_driver *eth_drv; >> + struct rte_eth_dev *eth_dev; >> + char ethdev_name[RTE_ETH_NAME_MAX_LEN]; >> + int ret; >> + >> + if (pci_dev == NULL) >> + return -EINVAL; >> + >> + /* Create unique Ethernet device name using PCI address */ >> + rte_eth_dev_create_unique_device_name(ethdev_name, pci_dev); >> + >> + eth_dev = rte_eth_dev_allocated(ethdev_name); >> + if (eth_dev == NULL) >> + return -ENODEV; >> + >> + eth_drv = (const struct eth_driver *)pci_dev->driver; >> + >> + /* Invoke PMD device uninit function */ >> + if (*eth_drv->eth_dev_uninit) { >> + ret = (*eth_drv->eth_dev_uninit)(eth_drv, eth_dev); >> + if (ret) >> + return ret; >> + } >> + >> + /* free ether device */ >> + rte_eth_dev_release_port(eth_dev); >> + >> + if (rte_eal_process_type() == RTE_PROC_PRIMARY) >> + rte_free(eth_dev->data->dev_private); >> + >> + eth_dev->pci_dev = NULL; >> + eth_dev->driver = NULL; >> + eth_dev->data = NULL; >> + >> + return 0; >> +} >> + >> /** >> * Register an Ethernet [Poll Mode] driver. >> * >> @@ -339,6 +396,7 @@ void >> rte_eth_driver_register(struct eth_driver *eth_drv) { >> eth_drv->pci_drv.devinit = rte_eth_dev_init; >> + eth_drv->pci_drv.devuninit = rte_eth_dev_uninit; >> rte_eal_pci_register(ð_drv->pci_drv); >> } >> >> diff --git a/lib/librte_ether/rte_ethdev.h b/lib/librte_ether/rte_ethdev.h index 28ecafd..f403780 >> 100644 >> --- a/lib/librte_ether/rte_ethdev.h >> +++ b/lib/librte_ether/rte_ethdev.h >> @@ -1677,6 +1677,27 @@ typedef int (*eth_dev_init_t)(struct eth_driver *eth_drv, >> >> /** >> * @internal >> + * Finalization function of an Ethernet driver invoked for each >> +matching >> + * Ethernet PCI device detected during the PCI closing phase. >> + * >> + * @param eth_drv >> + * The pointer to the [matching] Ethernet driver structure supplied by >> + * the PMD when it registered itself. >> + * @param eth_dev >> + * The *eth_dev* pointer is the address of the *rte_eth_dev* structure >> + * associated with the matching device and which have been [automatically] >> + * allocated in the *rte_eth_devices* array. >> + * @return >> + * - 0: Success, the device is properly finalized by the driver. >> + * In particular, the driver MUST free the *dev_ops* pointer >> + * of the *eth_dev* structure. >> + * - <0: Error code of the device initialization failure. >> + */ >> +typedef int (*eth_dev_uninit_t)(const struct eth_driver *eth_drv, >> + struct rte_eth_dev *eth_dev); >> + >> +/** >> + * @internal >> * The structure associated with a PMD Ethernet driver. >> * >> * Each Ethernet driver acts as a PCI driver and is represented by a generic @@ -1686,11 +1707,14 @@ >> typedef int (*eth_dev_init_t)(struct eth_driver *eth_drv, >> * >> * - The *eth_dev_init* function invoked for each matching PCI device. >> * >> + * - The *eth_dev_uninit* function invoked for each matching PCI device. >> + * >> * - The size of the private data to allocate for each matching device. >> */ >> struct eth_driver { >> struct rte_pci_driver pci_drv; /**< The PMD is also a PCI driver. */ >> eth_dev_init_t eth_dev_init; /**< Device init function. */ >> + eth_dev_uninit_t eth_dev_uninit; /**< Device uninit function. */ >> unsigned int dev_private_size; /**< Size of device private data. */ >> }; >> >> -- >> 1.9.1
On 2015/02/19 23:51, Iremonger, Bernard wrote: > >> -----Original Message----- >> From: Tetsuya Mukawa [mailto:mukawa@igel.co.jp] >> Sent: Thursday, February 19, 2015 2:50 AM >> To: dev@dpdk.org >> Cc: Qiu, Michael; Iremonger, Bernard; thomas.monjalon@6wind.com; Tetsuya Mukawa >> Subject: [PATCH v9 12/14] ethdev: Add one dev_type parameter to rte_eth_dev_allocate >> >> This new parameter is needed to keep device type like PCI or virtual. >> Port detaching processes are different between PCI device and virtual device. >> RTE_ETH_DEV_PCI indicates device type is PCI. RTE_ETH_DEV_VIRTUAL indicates device is virtual. >> >> v9: >> - Fix commit log. >> - RTE_ETH_DEV_PHYSICAL is replaced by RTE_ETH_DEV_PCI. >> (Thanks to Thomas Monjalon) >> v8: >> - NONE_TRACE is replaced by NO_TRACE. >> - Add missing symbol in version map. >> (Thanks to Qiu, Michael and Iremonger, Bernard) >> v4: >> - Fix comments of rte_eth_dev_type. >> >> Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp> >> --- >> app/test/virtual_pmd.c | 2 +- >> lib/librte_ether/rte_ethdev.c | 25 +++++++++++++++++++++++-- >> lib/librte_ether/rte_ethdev.h | 25 ++++++++++++++++++++++++- >> lib/librte_ether/rte_ether_version.map | 1 + >> lib/librte_pmd_af_packet/rte_eth_af_packet.c | 2 +- >> lib/librte_pmd_bond/rte_eth_bond_api.c | 2 +- >> lib/librte_pmd_pcap/rte_eth_pcap.c | 2 +- >> lib/librte_pmd_ring/rte_eth_ring.c | 2 +- >> lib/librte_pmd_xenvirt/rte_eth_xenvirt.c | 2 +- >> 9 files changed, 54 insertions(+), 9 deletions(-) >> >> diff --git a/app/test/virtual_pmd.c b/app/test/virtual_pmd.c index 9fac95d..c02644a 100644 >> --- a/app/test/virtual_pmd.c >> +++ b/app/test/virtual_pmd.c >> @@ -556,7 +556,7 @@ virtual_ethdev_create(const char *name, struct ether_addr *mac_addr, >> goto err; >> >> /* reserve an ethdev entry */ >> - eth_dev = rte_eth_dev_allocate(name); >> + eth_dev = rte_eth_dev_allocate(name, RTE_ETH_DEV_PCI); >> if (eth_dev == NULL) >> goto err; >> >> diff --git a/lib/librte_ether/rte_ethdev.c b/lib/librte_ether/rte_ethdev.c index 3b64f3a..201c04a >> 100644 >> --- a/lib/librte_ether/rte_ethdev.c >> +++ b/lib/librte_ether/rte_ethdev.c >> @@ -227,7 +227,7 @@ rte_eth_dev_find_free_port(void) } >> >> struct rte_eth_dev * >> -rte_eth_dev_allocate(const char *name) >> +rte_eth_dev_allocate(const char *name, enum rte_eth_dev_type type) >> { >> uint8_t port_id; >> struct rte_eth_dev *eth_dev; >> @@ -251,6 +251,7 @@ rte_eth_dev_allocate(const char *name) >> snprintf(eth_dev->data->name, sizeof(eth_dev->data->name), "%s", name); >> eth_dev->data->port_id = port_id; >> eth_dev->attached = DEV_ATTACHED; >> + eth_dev->dev_type = type; >> nb_ports++; >> return eth_dev; >> } >> @@ -262,6 +263,7 @@ rte_eth_dev_release_port(struct rte_eth_dev *eth_dev) >> return -EINVAL; >> >> eth_dev->attached = 0; >> + eth_dev->dev_type = RTE_ETH_DEV_UNKNOWN; >> nb_ports--; >> return 0; >> } >> @@ -299,7 +301,7 @@ rte_eth_dev_init(struct rte_pci_driver *pci_drv, >> /* Create unique Ethernet device name using PCI address */ >> rte_eth_dev_create_unique_device_name(ethdev_name, pci_dev); >> >> - eth_dev = rte_eth_dev_allocate(ethdev_name); >> + eth_dev = rte_eth_dev_allocate(ethdev_name, RTE_ETH_DEV_PCI); >> if (eth_dev == NULL) >> return -ENOMEM; >> >> @@ -424,6 +426,14 @@ rte_eth_dev_count(void) >> return (nb_ports); >> } >> >> +enum rte_eth_dev_type >> +rte_eth_dev_get_device_type(uint8_t port_id) { >> + if (!rte_eth_dev_is_valid_port(port_id)) >> + return -1; >> + return rte_eth_devices[port_id].dev_type; >> +} >> + >> int >> rte_eth_dev_save(struct rte_eth_dev *devs, size_t size) { @@ -521,6 +531,17 @@ >> rte_eth_dev_is_detachable(uint8_t port_id) >> return -EINVAL; >> } >> >> + if (rte_eth_devices[port_id].dev_type == RTE_ETH_DEV_PCI) { >> + switch (rte_eth_devices[port_id].pci_dev->pt_driver) { >> + case RTE_PT_IGB_UIO: >> + case RTE_PT_UIO_GENERIC: >> + break; >> + case RTE_PT_VFIO: >> + default: >> + return -ENOTSUP; >> + } >> + } >> + >> drv_flags = rte_eth_devices[port_id].driver->pci_drv.drv_flags; >> return !(drv_flags & RTE_PCI_DRV_DETACHABLE); } diff --git a/lib/librte_ether/rte_ethdev.h >> b/lib/librte_ether/rte_ethdev.h index 90b7f25..4f66bb6 100644 >> --- a/lib/librte_ether/rte_ethdev.h >> +++ b/lib/librte_ether/rte_ethdev.h >> @@ -1523,6 +1523,17 @@ struct eth_dev_ops { }; >> >> /** >> + * The eth device type >> + */ >> +enum rte_eth_dev_type { >> + RTE_ETH_DEV_UNKNOWN, /**< unknown device type */ >> + RTE_ETH_DEV_PCI, >> + /**< Physical function and Virtual function of PCI devices */ >> + RTE_ETH_DEV_VIRTUAL, /**< non hardware device */ >> + RTE_ETH_DEV_MAX /**< max value of this enum */ >> +}; >> + >> +/** >> * @internal >> * The generic data structure associated with each ethernet device. >> * >> @@ -1541,6 +1552,7 @@ struct rte_eth_dev { >> struct rte_pci_device *pci_dev; /**< PCI info. supplied by probing */ >> struct rte_eth_dev_cb_list callbacks; /**< User application callbacks */ >> uint8_t attached; /**< Flag indicating the port is attached */ >> + enum rte_eth_dev_type dev_type; /**< Flag indicating the device type >> +*/ >> }; >> >> struct rte_eth_dev_sriov { >> @@ -1618,6 +1630,15 @@ extern uint8_t rte_eth_dev_count(void); >> >> /** >> * Function for internal use by port hotplug functions. >> + * Get the device type to know whether the device is physical or virtual. >> + * @param port_id The pointer to the port id > Hi Tetsuya, > > " The pointer to the port id" should be removed. I will fix it in next patch. Tetsuya > Regards, > > Bernard. > > >> + * @return >> + * - Device type. >> + */ >> +extern enum rte_eth_dev_type rte_eth_dev_get_device_type(uint8_t >> +port_id); >> + >> +/** >> + * Function for internal use by port hotplug functions. >> * Copies current ethdev structures to the specified pointer. >> * >> * @param devs The pointer to the ethdev structures >> @@ -1706,10 +1727,12 @@ extern struct rte_eth_dev *rte_eth_dev_allocated(const char *name); >> * to that slot for the driver to use. >> * >> * @param name Unique identifier name for each Ethernet device >> + * @param type Device type of this Ethernet device >> * @return >> * - Slot in the rte_dev_devices array for a new device; >> */ >> -struct rte_eth_dev *rte_eth_dev_allocate(const char *name); >> +struct rte_eth_dev *rte_eth_dev_allocate(const char *name, >> + enum rte_eth_dev_type type); >> >> /** >> * Function for internal use by dummy drivers primarily, e.g. ring-based diff --git >> a/lib/librte_ether/rte_ether_version.map b/lib/librte_ether/rte_ether_version.map >> index 1d039ed..b78f5e3 100644 >> --- a/lib/librte_ether/rte_ether_version.map >> +++ b/lib/librte_ether/rte_ether_version.map >> @@ -116,6 +116,7 @@ DPDK_2.0 { >> rte_eth_dev_get_port_by_addr; >> rte_eth_dev_get_changed_port; >> rte_eth_dev_save; >> + rte_eth_dev_get_device_type; >> >> local: *; >> }; >> diff --git a/lib/librte_pmd_af_packet/rte_eth_af_packet.c >> b/lib/librte_pmd_af_packet/rte_eth_af_packet.c >> index 1ffe1cd..80e9bdf 100644 >> --- a/lib/librte_pmd_af_packet/rte_eth_af_packet.c >> +++ b/lib/librte_pmd_af_packet/rte_eth_af_packet.c >> @@ -649,7 +649,7 @@ rte_pmd_init_internals(const char *name, >> } >> >> /* reserve an ethdev entry */ >> - *eth_dev = rte_eth_dev_allocate(name); >> + *eth_dev = rte_eth_dev_allocate(name, RTE_ETH_DEV_VIRTUAL); >> if (*eth_dev == NULL) >> goto error; >> >> diff --git a/lib/librte_pmd_bond/rte_eth_bond_api.c b/lib/librte_pmd_bond/rte_eth_bond_api.c >> index 4ab3267..7a6a5f7 100644 >> --- a/lib/librte_pmd_bond/rte_eth_bond_api.c >> +++ b/lib/librte_pmd_bond/rte_eth_bond_api.c >> @@ -235,7 +235,7 @@ rte_eth_bond_create(const char *name, uint8_t mode, uint8_t socket_id) >> } >> >> /* reserve an ethdev entry */ >> - eth_dev = rte_eth_dev_allocate(name); >> + eth_dev = rte_eth_dev_allocate(name, RTE_ETH_DEV_VIRTUAL); >> if (eth_dev == NULL) { >> RTE_BOND_LOG(ERR, "Unable to allocate rte_eth_dev"); >> goto err; >> diff --git a/lib/librte_pmd_pcap/rte_eth_pcap.c b/lib/librte_pmd_pcap/rte_eth_pcap.c >> index d299288..af7fae8 100644 >> --- a/lib/librte_pmd_pcap/rte_eth_pcap.c >> +++ b/lib/librte_pmd_pcap/rte_eth_pcap.c >> @@ -709,7 +709,7 @@ rte_pmd_init_internals(const char *name, const unsigned nb_rx_queues, >> goto error; >> >> /* reserve an ethdev entry */ >> - *eth_dev = rte_eth_dev_allocate(name); >> + *eth_dev = rte_eth_dev_allocate(name, RTE_ETH_DEV_VIRTUAL); >> if (*eth_dev == NULL) >> goto error; >> >> diff --git a/lib/librte_pmd_ring/rte_eth_ring.c b/lib/librte_pmd_ring/rte_eth_ring.c >> index d5b1686..c837fcd 100644 >> --- a/lib/librte_pmd_ring/rte_eth_ring.c >> +++ b/lib/librte_pmd_ring/rte_eth_ring.c >> @@ -255,7 +255,7 @@ rte_eth_from_rings(const char *name, struct rte_ring *const rx_queues[], >> goto error; >> >> /* reserve an ethdev entry */ >> - eth_dev = rte_eth_dev_allocate(name); >> + eth_dev = rte_eth_dev_allocate(name, RTE_ETH_DEV_VIRTUAL); >> if (eth_dev == NULL) >> goto error; >> >> diff --git a/lib/librte_pmd_xenvirt/rte_eth_xenvirt.c b/lib/librte_pmd_xenvirt/rte_eth_xenvirt.c >> index 04e30c9..bc403d6 100644 >> --- a/lib/librte_pmd_xenvirt/rte_eth_xenvirt.c >> +++ b/lib/librte_pmd_xenvirt/rte_eth_xenvirt.c >> @@ -648,7 +648,7 @@ eth_dev_xenvirt_create(const char *name, const char *params, >> goto err; >> >> /* reserve an ethdev entry */ >> - eth_dev = rte_eth_dev_allocate(name); >> + eth_dev = rte_eth_dev_allocate(name, RTE_ETH_DEV_VIRTUAL); >> if (eth_dev == NULL) >> goto err; >> >> -- >> 1.9.1
On 2015/02/19 22:29, Tetsuya Mukawa wrote: > On 2015/02/19 20:17, Thomas Monjalon wrote: >>> @@ -152,6 +159,7 @@ struct rte_pci_device { >>> uint16_t max_vfs; /**< sriov enable if not zero */ >>> int numa_node; /**< NUMA node connection */ >>> struct rte_devargs *devargs; /**< Device user arguments */ >>> + enum rte_pt_driver pt_driver; /**< Driver of passthrough */ >> [...] >>> +static int >>> +pci_get_kernel_driver_by_path(const char *filename, char *dri_name) >> I think "kernel driver" is a good name. Why not using this name in the >> pci_device struct to be more consistent? > Hi Michael, > > Could you please let me know what do you think about it? Hi Thomas, Could you please check below reply from Michael? He has already replied it. http://dpdk.org/dev/patchwork/patch/3363/ According to Tim's email, he might be out of office until middle of next week. I cannot rewrite his patch without his agreement. So I will submit patches without this. If we decide to change it when he comes back, I will send v11 patches. Thanks, Tetsuya > Thanks > Tetsuya > >> Thanks
This patch series adds a dynamic port hotplug framework to DPDK. With the patches, DPDK apps can attach or detach ports at runtime. The basic concept of the port hotplug is like followings. - DPDK apps must have responsibility to manage ports. DPDK apps only know which ports are attached or detached at the moment. The port hotplug framework is implemented to allow DPDK apps to manage ports. For example, when DPDK apps call port attach function, attached port number will be returned. Also, DPDK apps can detach port by port number. - Kernel support is needed for attaching or detaching physical device ports. To attach a new physical device port, the device will be recognized by userspace directly I/O framework in kernel at first. Then DPDK apps can call the port hotplug functions to attach ports. For detaching, steps are vice versa. - Before detach ports, ports must be stopped and closed. DPDK application must call rte_eth_dev_stop() and rte_eth_dev_close() before detaching ports. These function will call finalization codes of PMDs. But so far, no PMD frees all resources allocated by initialization. It means PMDs are needed to be fixed to support the port hotplug. 'RTE_PCI_DRV_DETACHABLE' is a new flag indicating a PMD supports detaching. Without this flag, detaching will be failed. - Mustn't affect legacy DPDK apps. No DPDK EAL behavior is changed, if the port hotplug functions are't called. So all legacy DPDK apps can still work without modifications. And a few limitations. - The port hotplug functions are not thread safe. DPDK apps should handle it. - Only support Linux and igb_uio so far. BSD and VFIO is not supported. I will send VFIO patches at least, but I don't have a plan to submit BSD patch so far. Here is port hotplug APIs. ------------------------------------------------------------------------------- /** * Attach a new device. * * @param devargs * A pointer to a strings array describing the new device * to be attached. The strings should be a pci address like * '0000:01:00.0' or virtual device name like 'eth_pcap0'. * @param port_id * A pointer to a port identifier actually attached. * @return * 0 on success and port_id is filled, negative on error */ int rte_eal_dev_attach(const char *devargs, uint8_t *port_id); /** * Detach a device. * * @param port_id * The port identifier of the device to detach. * @param addr * A pointer to a device name actually detached. * @return * 0 on success and devname is filled, negative on error */ int rte_eal_dev_detach(uint8_t port_id, char *devname); ------------------------------------------------------------------------------- This patch series are for DPDK EAL. To use port hotplug function by DPDK apps, each PMD should be fixed to support 'RTE_PCI_DRV_DETACHABLE' flag. Please check a patch for pcap PMD. Also, please check testpmd patch. It will show you how to fix your legacy applications to support port hotplug feature. PATCH v10 changes - Add comments. - Chagne order of version.map. - Fix comment of "rte_ethdev.h". (Thanks to Thomas Monjalon) - Add size parameter to rte_eth_dev_create_unique_device_name(). (Thanks to Iremonger, Bernard) PATCH v9 changes - Fix commit title. - Fix commit log. - Fix comments. - Define CONFIG_RTE_LIBRTE_EAL_HOTPLUG at the top of this patch series. - DEV_INVALID/VALID are removed. - DEV_DISCONNECTED is replaced by DEV_DETACHED. - DEV_CONNECTED is replaced by DEV_ATTACHED. - rte_eth_dev_allocate_new_port() is renamed to rte_eth_dev_find_free_port(). - rte_eth_dev_validate_port() is renamed to rte_eth_dev_is_valid_port(). - rte_eth_dev_is_valid_port() is changed not to handle log toggle. - eal_compare_pci_addr() is replaced by rte_eal_compare_pci_addr(). - rte_eth_dev_free() is replaced by rte_eth_dev_release_port(). - Add a function to create a unique device name. - Change parameter of pci_devuninit_t and rte_eth_dev_uninit. - Remove code that initiaize callback of ethdev from rte_eth_dev_uninit(). - Remove pci_unmap_device(). It will be implemented in later patch. - rte_eth_dev_check_detachable() is replaced by rte_eth_dev_is_detachable(). - strncpy() is replaced by strcpy(). - Implement pci_unmap_device() in this patch. - Remove "rte_dev_hotplug.h". - Remove needless "#ifdef". - Remove RTE_EAL_INVOKE_TYPE_PROBE/CLOSE. - RTE_ETH_DEV_PHYSICAL is replaced by RTE_ETH_DEV_PCI. - Use strcmp() instead of strncmp(). - Remove RTE_EAL_INVOKE_TYPE_PROBE/CLOSE. (Thanks to Thomas Monjalon) - Change definition of rte_dev_uninit_t. (Thanks to Thomas Monjalon and Maxime Leroy) - Add missing symbol in version map. (Thanks to Nail Horman) PATCH v8 changes - Fix Makefile and add version map file. - Add missing symbol in version map. - Fix pci_scan_one() to update sysfs values. (Thanks to Qiu, Michael and Iremonger, Bernard) - NONE_TRACE is replaced by NO_TRACE. - Fix typo. - Add size parameter to rte_eth_dev_save(). (Thanks to Iremonger, Bernard) PATCH v7 changes - Add a new section to programmer's guide. (Thanks to Iremonger, Bernard) - Fix port checking implementation of star_port(). - Fix typo of warning messages. - Add pt_driver checking to rte_eth_dev_check_detachable(). (Thanks to Qiu, Michael) PATCH v6 changes - Fix rte_eth_dev_uninit() to handle a return value of uninit function of PMD. To do this, below changes also be applied. - Fix a parameter of rte_eth_dev_free(). - Use rte_eth_dev structure as the paramter of rte_eth_dev_free(). PATCH v5 changes - Add runtime check passthrough driver type, like vfio-pci, igb_uio and uio_pci_generic. This was done by Qiu, Michael. Thanks a lot. - Change function names like below. - rte_eal_dev_find_and_invoke() to rte_eal_vdev_find_and_invoke(). - rte_eal_dev_invoke() to rte_eal_vdev_invoke(). - Add code to handle a return value of rte_eal_devargs_remove(). - Fix pci address format in rte_eal_dev_detach(). - Remove RTE_EAL_INVOKE_TYPE_UNKNOWN, because it's unused. - Change function definition of rte_eal_devargs_remove(). - Fix pci_unmap_device() to check pt_driver. - Fix return value of below functions. - rte_eth_dev_get_changed_port(). - rte_eth_dev_get_port_by_addr(). - Change paramters of rte_eth_dev_validate_port() to cleanup code. - Fix pci_scan_one to handle pt_driver correctly. (Thanks to Qiu, Michael for above suggestions) PATCH v4 changes - Merge patches to review easier. - Fix indent of 'if' statement. - Fix calculation method of eal_compare_pci_addr(). - Fix header file declaration. - Add header file to determine if hotplug can be enabled. (Thanks to Qiu, Michael) - Use braces with 'for' loop. - Add parameter checking. - Fix sanity check code - Fix comments of rte_eth_dev_type. - Change function names. (Thanks to Iremonger, Bernard) PATCH v3 changes: - Fix enum definition used in rte_ethdev.c. (Thanks to Zhang, Helin) PATCH v2 changes: - Replace rte_eal_dev_attach_pdev(), rte_eal_dev_detach_pdev, rte_eal_dev_attach_vdev() and rte_eal_dev_detach_vdev() to rte_eal_dev_attach() and rte_eal_dev_detach(). - Add parameter values checking. - Refashion a few functions. (Thanks to Iremonger, Bernard) PATCH v1 Changes: - Fix error checking code of librte_eth APIs. - Fix issue that port from pcap PMD cannot be detached correctly. - Fix issue that testpmd could hang after forwarding, if attaching and detaching is repeatedly. - Fix if-condition of rte_eth_dev_get_port_by_addr(). (Thanks to Mark Enright) RFC PATCH v2 Changes: - remove 'rte_eth_dev_validate_port()', and cleanup codes. Michael Qiu (2): eal_pci: Add flag to hold kernel driver type eal_pci: pci memory map work with driver type Tetsuya Mukawa (12): eal: Enable port Hotplug framework in Linux eal/pci,ethdev: Remove assumption that port will not be detached eal/pci: Consolidate pci address comparison APIs ethdev: Add rte_eth_dev_release_port to release specified port eal,ethdev: Add a function and function pointers to close ether device ethdev: Add functions that will be used by port hotplug functions eal/linux/pci: Add functions for unmapping igb_uio resources eal/pci: Add a function to remove the entry of devargs list eal/pci: Add probe and close functions of pci driver ethdev: Add one dev_type parameter to rte_eth_dev_allocate eal/pci: Add rte_eal_dev_attach/detach() functions doc: Add port hotplug framework section to programmers guide app/test/virtual_pmd.c | 2 +- config/common_bsdapp | 6 + config/common_linuxapp | 5 + doc/guides/prog_guide/index.rst | 1 + doc/guides/prog_guide/port_hotplug_framework.rst | 110 ++++++ lib/librte_eal/bsdapp/eal/eal_pci.c | 29 +- lib/librte_eal/common/eal_common_dev.c | 315 +++++++++++++++ lib/librte_eal/common/eal_common_devargs.c | 61 +++ lib/librte_eal/common/eal_common_pci.c | 100 ++++- lib/librte_eal/common/eal_private.h | 26 ++ lib/librte_eal/common/include/rte_dev.h | 33 ++ lib/librte_eal/common/include/rte_devargs.h | 21 + lib/librte_eal/common/include/rte_pci.h | 82 ++++ lib/librte_eal/linuxapp/eal/Makefile | 1 + lib/librte_eal/linuxapp/eal/eal_pci.c | 230 +++++++++-- lib/librte_eal/linuxapp/eal/eal_pci_init.h | 7 + lib/librte_eal/linuxapp/eal/eal_pci_uio.c | 67 +++- lib/librte_eal/linuxapp/eal/rte_eal_version.map | 2 + lib/librte_ether/rte_ethdev.c | 476 ++++++++++++++++++----- lib/librte_ether/rte_ethdev.h | 149 ++++++- lib/librte_ether/rte_ether_version.map | 8 + lib/librte_pmd_af_packet/rte_eth_af_packet.c | 2 +- lib/librte_pmd_bond/rte_eth_bond_api.c | 2 +- lib/librte_pmd_pcap/rte_eth_pcap.c | 2 +- lib/librte_pmd_ring/rte_eth_ring.c | 2 +- lib/librte_pmd_xenvirt/rte_eth_xenvirt.c | 2 +- 26 files changed, 1573 insertions(+), 168 deletions(-) create mode 100644 doc/guides/prog_guide/port_hotplug_framework.rst -- 1.9.1
The patch adds CONFIG_RTE_LIBRTE_EAL_HOTPLUG in Linux and BSD configuration. So far, Hotplug functions only support linux. v9: - Move this patch at the top of this patch series. (Thanks to Thomas Monjalon) Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp> --- config/common_bsdapp | 6 ++++++ config/common_linuxapp | 5 +++++ 2 files changed, 11 insertions(+) diff --git a/config/common_bsdapp b/config/common_bsdapp index f11ff39..e9d07e4 100644 --- a/config/common_bsdapp +++ b/config/common_bsdapp @@ -116,6 +116,12 @@ CONFIG_RTE_LIBRTE_EAL_BSDAPP=y CONFIG_RTE_LIBRTE_EAL_LINUXAPP=n # +# Compile Environment Abstraction Layer to support hotplug +# So far, Hotplug functions only support linux +# +CONFIG_RTE_LIBRTE_EAL_HOTPLUG=n + +# # Compile Environment Abstraction Layer to support Vmware TSC map # CONFIG_RTE_LIBRTE_EAL_VMWARE_TSC_MAP_SUPPORT=y diff --git a/config/common_linuxapp b/config/common_linuxapp index f921d8c..5f7fe28 100644 --- a/config/common_linuxapp +++ b/config/common_linuxapp @@ -114,6 +114,11 @@ CONFIG_RTE_PCI_MAX_READ_REQUEST_SIZE=0 CONFIG_RTE_LIBRTE_EAL_LINUXAPP=y # +# Compile Environment Abstraction Layer to support hotplug +# +CONFIG_RTE_LIBRTE_EAL_HOTPLUG=y + +# # Compile Environment Abstraction Layer to support Vmware TSC map # CONFIG_RTE_LIBRTE_EAL_VMWARE_TSC_MAP_SUPPORT=y -- 1.9.1
From: Michael Qiu <michael.qiu@intel.com> Currently, dpdk has no ability to know which type of driver( vfio-pci/igb_uio/uio_pci_generic) the device used. It only can check whether vfio is enabled or not staticly. It really useful to have the flag, becasue different type need to handle differently in runtime. For example, pci memory map, pot hotplug, and so on. This patch add a flag field for pci device to solve above issue. Signed-off-by: Michael Qiu <michael.qiu@intel.com> Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp> --- lib/librte_eal/common/include/rte_pci.h | 8 +++++ lib/librte_eal/linuxapp/eal/eal_pci.c | 53 +++++++++++++++++++++++++++++++-- 2 files changed, 59 insertions(+), 2 deletions(-) diff --git a/lib/librte_eal/common/include/rte_pci.h b/lib/librte_eal/common/include/rte_pci.h index 66ed793..7b48b55 100644 --- a/lib/librte_eal/common/include/rte_pci.h +++ b/lib/librte_eal/common/include/rte_pci.h @@ -139,6 +139,13 @@ struct rte_pci_addr { struct rte_devargs; +enum rte_pt_driver { + RTE_PT_UNKNOWN = 0, + RTE_PT_IGB_UIO = 1, + RTE_PT_VFIO = 2, + RTE_PT_UIO_GENERIC = 3, +}; + /** * A structure describing a PCI device. */ @@ -152,6 +159,7 @@ struct rte_pci_device { uint16_t max_vfs; /**< sriov enable if not zero */ int numa_node; /**< NUMA node connection */ struct rte_devargs *devargs; /**< Device user arguments */ + enum rte_pt_driver pt_driver; /**< Driver of passthrough */ }; /** Any PCI device identifier (vendor, device, ...) */ diff --git a/lib/librte_eal/linuxapp/eal/eal_pci.c b/lib/librte_eal/linuxapp/eal/eal_pci.c index 15db9c4..e760452 100644 --- a/lib/librte_eal/linuxapp/eal/eal_pci.c +++ b/lib/librte_eal/linuxapp/eal/eal_pci.c @@ -97,6 +97,35 @@ error: return -1; } +static int +pci_get_kernel_driver_by_path(const char *filename, char *dri_name) +{ + int count; + char path[PATH_MAX]; + char *name; + + if (!filename || !dri_name) + return -1; + + count = readlink(filename, path, PATH_MAX); + if (count >= PATH_MAX) + return -1; + + /* For device does not have a driver */ + if (count < 0) + return 1; + + path[count] = '\0'; + + name = strrchr(path, '/'); + if (name) { + strncpy(dri_name, name + 1, strlen(name + 1) + 1); + return 0; + } + + return -1; +} + void * pci_find_max_end_va(void) { @@ -222,11 +251,12 @@ pci_scan_one(const char *dirname, uint16_t domain, uint8_t bus, char filename[PATH_MAX]; unsigned long tmp; struct rte_pci_device *dev; + char driver[PATH_MAX]; + int ret; dev = malloc(sizeof(*dev)); - if (dev == NULL) { + if (dev == NULL) return -1; - } memset(dev, 0, sizeof(*dev)); dev->addr.domain = domain; @@ -305,6 +335,25 @@ pci_scan_one(const char *dirname, uint16_t domain, uint8_t bus, return -1; } + /* parse driver */ + snprintf(filename, sizeof(filename), "%s/driver", dirname); + ret = pci_get_kernel_driver_by_path(filename, driver); + if (!ret) { + if (!strcmp(driver, "vfio-pci")) + dev->pt_driver = RTE_PT_VFIO; + else if (!strcmp(driver, "igb_uio")) + dev->pt_driver = RTE_PT_IGB_UIO; + else if (!strcmp(driver, "uio_pci_generic")) + dev->pt_driver = RTE_PT_UIO_GENERIC; + else + dev->pt_driver = RTE_PT_UNKNOWN; + } else if (ret < 0) { + RTE_LOG(ERR, EAL, "Fail to get kernel driver\n"); + free(dev); + return -1; + } else + dev->pt_driver = RTE_PT_UNKNOWN; + /* device is valid, add in list (sorted) */ if (TAILQ_EMPTY(&pci_device_list)) { TAILQ_INSERT_TAIL(&pci_device_list, dev, next); -- 1.9.1
From: Michael Qiu <michael.qiu@intel.com> With the driver type flag in struct rte_pci_dev, we do not need to always map uio devices with vfio related function when vfio enabled. Signed-off-by: Michael Qiu <michael.qiu@intel.com> Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp> --- lib/librte_eal/linuxapp/eal/eal_pci.c | 30 +++++++++++++++++------------- 1 file changed, 17 insertions(+), 13 deletions(-) diff --git a/lib/librte_eal/linuxapp/eal/eal_pci.c b/lib/librte_eal/linuxapp/eal/eal_pci.c index e760452..3c463b2 100644 --- a/lib/librte_eal/linuxapp/eal/eal_pci.c +++ b/lib/librte_eal/linuxapp/eal/eal_pci.c @@ -556,25 +556,29 @@ pci_config_space_set(struct rte_pci_device *dev) static int pci_map_device(struct rte_pci_device *dev) { - int ret, mapped = 0; + int ret = -1; /* try mapping the NIC resources using VFIO if it exists */ + switch (dev->pt_driver) { + case RTE_PT_VFIO: #ifdef VFIO_PRESENT - if (pci_vfio_is_enabled()) { - ret = pci_vfio_map_resource(dev); - if (ret == 0) - mapped = 1; - else if (ret < 0) - return ret; - } + if (pci_vfio_is_enabled()) + ret = pci_vfio_map_resource(dev); #endif - /* map resources for devices that use igb_uio */ - if (!mapped) { + break; + case RTE_PT_IGB_UIO: + case RTE_PT_UIO_GENERIC: + /* map resources for devices that use uio */ ret = pci_uio_map_resource(dev); - if (ret != 0) - return ret; + break; + default: + RTE_LOG(DEBUG, EAL, " Not managed by known pt driver," + " skipped\n"); + ret = 1; + break; } - return 0; + + return ret; } /* -- 1.9.1
To remove assumption, do like followings. This patch adds "RTE_PCI_DRV_DETACHABLE" to drv_flags of rte_pci_driver structure. The flags indicate the driver can detach devices at runtime. Also, remove assumption that port will not be detached. To remove the assumption. - Add 'attached' member to rte_eth_dev structure. This member is used for indicating the port is attached, or not. DEV_ATTACHED indicates a port is attached. DEV_DETACHED indicates a port is detached. - Add rte_eth_dev_allocate_new_port(). This function is used for allocating new port. v9: - DEV_INVALID/VALID are removed. - DEV_DISCONNECTED is replaced by DEV_DETACHED. - DEV_CONNECTED is replaced by DEV_ATTACHED. - rte_eth_dev_allocate_new_port() is renamed to rte_eth_dev_find_free_port(). - rte_eth_dev_validate_port() is renamed to rte_eth_dev_is_valid_port(). - rte_eth_dev_is_valid_port() is changed not to handle log toggle. - Fix commit log to describe DEV_ATACHED and DEV_DETACHED. (Thanks to Thomas Monjalon) v8: - NONE_TRACE is changed to NO_TRACE. (Thanks to Iremonger, Bernard) v5: - Change parameters of rte_eth_dev_validate_port() to cleanup code. v4: - Use braces with 'for' loop. - Fix indent of 'if' statement. Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp> --- lib/librte_eal/common/include/rte_pci.h | 2 + lib/librte_ether/rte_ethdev.c | 273 ++++++++++++++++++++------------ lib/librte_ether/rte_ethdev.h | 5 + 3 files changed, 177 insertions(+), 103 deletions(-) diff --git a/lib/librte_eal/common/include/rte_pci.h b/lib/librte_eal/common/include/rte_pci.h index 7b48b55..7f2d699 100644 --- a/lib/librte_eal/common/include/rte_pci.h +++ b/lib/librte_eal/common/include/rte_pci.h @@ -207,6 +207,8 @@ struct rte_pci_driver { #define RTE_PCI_DRV_FORCE_UNBIND 0x0004 /** Device driver supports link state interrupt */ #define RTE_PCI_DRV_INTR_LSC 0x0008 +/** Device driver supports detaching capability */ +#define RTE_PCI_DRV_DETACHABLE 0x0010 /**< Internal use only - Macro used by pci addr parsing functions **/ #define GET_PCIADDR_FIELD(in, fd, lim, dlm) \ diff --git a/lib/librte_ether/rte_ethdev.c b/lib/librte_ether/rte_ethdev.c index 17be2f3..6e22216 100644 --- a/lib/librte_ether/rte_ethdev.c +++ b/lib/librte_ether/rte_ethdev.c @@ -175,6 +175,11 @@ enum { STAT_QMAP_RX }; +enum { + DEV_DETACHED = 0, + DEV_ATTACHED +}; + static inline void rte_eth_dev_data_alloc(void) { @@ -201,19 +206,34 @@ rte_eth_dev_allocated(const char *name) { unsigned i; - for (i = 0; i < nb_ports; i++) { - if (strcmp(rte_eth_devices[i].data->name, name) == 0) + for (i = 0; i < RTE_MAX_ETHPORTS; i++) { + if ((rte_eth_devices[i].attached == DEV_ATTACHED) && + strcmp(rte_eth_devices[i].data->name, name) == 0) return &rte_eth_devices[i]; } return NULL; } +static uint8_t +rte_eth_dev_find_free_port(void) +{ + unsigned i; + + for (i = 0; i < RTE_MAX_ETHPORTS; i++) { + if (rte_eth_devices[i].attached == DEV_DETACHED) + return i; + } + return RTE_MAX_ETHPORTS; +} + struct rte_eth_dev * rte_eth_dev_allocate(const char *name) { + uint8_t port_id; struct rte_eth_dev *eth_dev; - if (nb_ports == RTE_MAX_ETHPORTS) { + port_id = rte_eth_dev_find_free_port(); + if (port_id == RTE_MAX_ETHPORTS) { PMD_DEBUG_TRACE("Reached maximum number of Ethernet ports\n"); return NULL; } @@ -226,10 +246,12 @@ rte_eth_dev_allocate(const char *name) return NULL; } - eth_dev = &rte_eth_devices[nb_ports]; - eth_dev->data = &rte_eth_dev_data[nb_ports]; + eth_dev = &rte_eth_devices[port_id]; + eth_dev->data = &rte_eth_dev_data[port_id]; snprintf(eth_dev->data->name, sizeof(eth_dev->data->name), "%s", name); - eth_dev->data->port_id = nb_ports++; + eth_dev->data->port_id = port_id; + eth_dev->attached = DEV_ATTACHED; + nb_ports++; return eth_dev; } @@ -283,6 +305,7 @@ rte_eth_dev_init(struct rte_pci_driver *pci_drv, (unsigned) pci_dev->id.device_id); if (rte_eal_process_type() == RTE_PROC_PRIMARY) rte_free(eth_dev->data->dev_private); + eth_dev->attached = DEV_DETACHED; nb_ports--; return diag; } @@ -308,10 +331,20 @@ rte_eth_driver_register(struct eth_driver *eth_drv) rte_eal_pci_register(ð_drv->pci_drv); } +static int +rte_eth_dev_is_valid_port(uint8_t port_id) +{ + if (port_id >= RTE_MAX_ETHPORTS || + rte_eth_devices[port_id].attached != DEV_ATTACHED) + return 0; + else + return 1; +} + int rte_eth_dev_socket_id(uint8_t port_id) { - if (port_id >= nb_ports) + if (!rte_eth_dev_is_valid_port(port_id)) return -1; return rte_eth_devices[port_id].pci_dev->numa_node; } @@ -369,7 +402,7 @@ rte_eth_dev_rx_queue_start(uint8_t port_id, uint16_t rx_queue_id) * in a multi-process setup*/ PROC_PRIMARY_OR_ERR_RET(-E_RTE_SECONDARY); - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return -EINVAL; } @@ -395,7 +428,7 @@ rte_eth_dev_rx_queue_stop(uint8_t port_id, uint16_t rx_queue_id) * in a multi-process setup*/ PROC_PRIMARY_OR_ERR_RET(-E_RTE_SECONDARY); - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return -EINVAL; } @@ -421,7 +454,7 @@ rte_eth_dev_tx_queue_start(uint8_t port_id, uint16_t tx_queue_id) * in a multi-process setup*/ PROC_PRIMARY_OR_ERR_RET(-E_RTE_SECONDARY); - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return -EINVAL; } @@ -447,7 +480,7 @@ rte_eth_dev_tx_queue_stop(uint8_t port_id, uint16_t tx_queue_id) * in a multi-process setup*/ PROC_PRIMARY_OR_ERR_RET(-E_RTE_SECONDARY); - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return -EINVAL; } @@ -703,10 +736,11 @@ rte_eth_dev_configure(uint8_t port_id, uint16_t nb_rx_q, uint16_t nb_tx_q, * in a multi-process setup*/ PROC_PRIMARY_OR_ERR_RET(-E_RTE_SECONDARY); - if (port_id >= nb_ports || port_id >= RTE_MAX_ETHPORTS) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-EINVAL); } + dev = &rte_eth_devices[port_id]; FUNC_PTR_OR_ERR_RET(*dev->dev_ops->dev_infos_get, -ENOTSUP); @@ -888,10 +922,11 @@ rte_eth_dev_start(uint8_t port_id) * in a multi-process setup*/ PROC_PRIMARY_OR_ERR_RET(-E_RTE_SECONDARY); - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%" PRIu8 "\n", port_id); + if (!rte_eth_dev_is_valid_port(port_id)) { + PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-EINVAL); } + dev = &rte_eth_devices[port_id]; FUNC_PTR_OR_ERR_RET(*dev->dev_ops->dev_start, -ENOTSUP); @@ -927,10 +962,11 @@ rte_eth_dev_stop(uint8_t port_id) * in a multi-process setup*/ PROC_PRIMARY_OR_RET(); - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%" PRIu8 "\n", port_id); + if (!rte_eth_dev_is_valid_port(port_id)) { + PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return; } + dev = &rte_eth_devices[port_id]; FUNC_PTR_OR_RET(*dev->dev_ops->dev_stop); @@ -955,10 +991,11 @@ rte_eth_dev_set_link_up(uint8_t port_id) * in a multi-process setup*/ PROC_PRIMARY_OR_ERR_RET(-E_RTE_SECONDARY); - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return -EINVAL; } + dev = &rte_eth_devices[port_id]; FUNC_PTR_OR_ERR_RET(*dev->dev_ops->dev_set_link_up, -ENOTSUP); @@ -974,10 +1011,11 @@ rte_eth_dev_set_link_down(uint8_t port_id) * in a multi-process setup*/ PROC_PRIMARY_OR_ERR_RET(-E_RTE_SECONDARY); - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return -EINVAL; } + dev = &rte_eth_devices[port_id]; FUNC_PTR_OR_ERR_RET(*dev->dev_ops->dev_set_link_down, -ENOTSUP); @@ -993,7 +1031,7 @@ rte_eth_dev_close(uint8_t port_id) * in a multi-process setup*/ PROC_PRIMARY_OR_RET(); - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return; } @@ -1021,10 +1059,11 @@ rte_eth_rx_queue_setup(uint8_t port_id, uint16_t rx_queue_id, * in a multi-process setup*/ PROC_PRIMARY_OR_ERR_RET(-E_RTE_SECONDARY); - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-EINVAL); } + dev = &rte_eth_devices[port_id]; if (rx_queue_id >= dev->data->nb_rx_queues) { PMD_DEBUG_TRACE("Invalid RX queue_id=%d\n", rx_queue_id); @@ -1094,10 +1133,11 @@ rte_eth_tx_queue_setup(uint8_t port_id, uint16_t tx_queue_id, * in a multi-process setup*/ PROC_PRIMARY_OR_ERR_RET(-E_RTE_SECONDARY); - if (port_id >= RTE_MAX_ETHPORTS || port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-EINVAL); } + dev = &rte_eth_devices[port_id]; if (tx_queue_id >= dev->data->nb_tx_queues) { PMD_DEBUG_TRACE("Invalid TX queue_id=%d\n", tx_queue_id); @@ -1127,10 +1167,11 @@ rte_eth_promiscuous_enable(uint8_t port_id) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return; } + dev = &rte_eth_devices[port_id]; FUNC_PTR_OR_RET(*dev->dev_ops->promiscuous_enable); @@ -1143,10 +1184,11 @@ rte_eth_promiscuous_disable(uint8_t port_id) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return; } + dev = &rte_eth_devices[port_id]; FUNC_PTR_OR_RET(*dev->dev_ops->promiscuous_disable); @@ -1159,7 +1201,7 @@ rte_eth_promiscuous_get(uint8_t port_id) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return -1; } @@ -1173,10 +1215,11 @@ rte_eth_allmulticast_enable(uint8_t port_id) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return; } + dev = &rte_eth_devices[port_id]; FUNC_PTR_OR_RET(*dev->dev_ops->allmulticast_enable); @@ -1189,10 +1232,11 @@ rte_eth_allmulticast_disable(uint8_t port_id) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return; } + dev = &rte_eth_devices[port_id]; FUNC_PTR_OR_RET(*dev->dev_ops->allmulticast_disable); @@ -1205,7 +1249,7 @@ rte_eth_allmulticast_get(uint8_t port_id) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return -1; } @@ -1233,10 +1277,11 @@ rte_eth_link_get(uint8_t port_id, struct rte_eth_link *eth_link) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return; } + dev = &rte_eth_devices[port_id]; if (dev->data->dev_conf.intr_conf.lsc != 0) @@ -1253,10 +1298,11 @@ rte_eth_link_get_nowait(uint8_t port_id, struct rte_eth_link *eth_link) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return; } + dev = &rte_eth_devices[port_id]; if (dev->data->dev_conf.intr_conf.lsc != 0) @@ -1273,10 +1319,11 @@ rte_eth_stats_get(uint8_t port_id, struct rte_eth_stats *stats) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-ENODEV); } + dev = &rte_eth_devices[port_id]; memset(stats, 0, sizeof(*stats)); @@ -1291,10 +1338,11 @@ rte_eth_stats_reset(uint8_t port_id) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return; } + dev = &rte_eth_devices[port_id]; FUNC_PTR_OR_RET(*dev->dev_ops->stats_reset); @@ -1312,10 +1360,11 @@ rte_eth_xstats_get(uint8_t port_id, struct rte_eth_xstats *xstats, uint64_t val; char *stats_ptr; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return -1; } + dev = &rte_eth_devices[port_id]; /* implemented by the driver */ @@ -1381,10 +1430,11 @@ rte_eth_xstats_reset(uint8_t port_id) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return; } + dev = &rte_eth_devices[port_id]; /* implemented by the driver */ @@ -1403,10 +1453,11 @@ set_queue_stats_mapping(uint8_t port_id, uint16_t queue_id, uint8_t stat_idx, { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return -ENODEV; } + dev = &rte_eth_devices[port_id]; FUNC_PTR_OR_ERR_RET(*dev->dev_ops->queue_stats_mapping_set, -ENOTSUP); @@ -1438,10 +1489,11 @@ rte_eth_dev_info_get(uint8_t port_id, struct rte_eth_dev_info *dev_info) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return; } + dev = &rte_eth_devices[port_id]; memset(dev_info, 0, sizeof(struct rte_eth_dev_info)); @@ -1458,10 +1510,11 @@ rte_eth_macaddr_get(uint8_t port_id, struct ether_addr *mac_addr) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return; } + dev = &rte_eth_devices[port_id]; ether_addr_copy(&dev->data->mac_addrs[0], mac_addr); } @@ -1472,7 +1525,7 @@ rte_eth_dev_get_mtu(uint8_t port_id, uint16_t *mtu) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-ENODEV); } @@ -1488,7 +1541,7 @@ rte_eth_dev_set_mtu(uint8_t port_id, uint16_t mtu) int ret; struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-ENODEV); } @@ -1508,10 +1561,11 @@ rte_eth_dev_vlan_filter(uint8_t port_id, uint16_t vlan_id, int on) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-ENODEV); } + dev = &rte_eth_devices[port_id]; if (! (dev->data->dev_conf.rxmode.hw_vlan_filter)) { PMD_DEBUG_TRACE("port %d: vlan-filtering disabled\n", port_id); @@ -1533,7 +1587,7 @@ rte_eth_dev_set_vlan_strip_on_queue(uint8_t port_id, uint16_t rx_queue_id, int o { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-ENODEV); } @@ -1555,7 +1609,7 @@ rte_eth_dev_set_vlan_ether_type(uint8_t port_id, uint16_t tpid) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-ENODEV); } @@ -1575,7 +1629,7 @@ rte_eth_dev_set_vlan_offload(uint8_t port_id, int offload_mask) int mask = 0; int cur, org = 0; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-ENODEV); } @@ -1620,7 +1674,7 @@ rte_eth_dev_get_vlan_offload(uint8_t port_id) struct rte_eth_dev *dev; int ret = 0; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-ENODEV); } @@ -1644,10 +1698,11 @@ rte_eth_dev_set_vlan_pvid(uint8_t port_id, uint16_t pvid, int on) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-ENODEV); } + dev = &rte_eth_devices[port_id]; FUNC_PTR_OR_ERR_RET(*dev->dev_ops->vlan_pvid_set, -ENOTSUP); (*dev->dev_ops->vlan_pvid_set)(dev, pvid, on); @@ -1662,7 +1717,7 @@ rte_eth_dev_fdir_add_signature_filter(uint8_t port_id, { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-ENODEV); } @@ -1696,7 +1751,7 @@ rte_eth_dev_fdir_update_signature_filter(uint8_t port_id, { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-ENODEV); } @@ -1730,7 +1785,7 @@ rte_eth_dev_fdir_remove_signature_filter(uint8_t port_id, { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-ENODEV); } @@ -1761,7 +1816,7 @@ rte_eth_dev_fdir_get_infos(uint8_t port_id, struct rte_eth_fdir *fdir) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-ENODEV); } @@ -1786,7 +1841,7 @@ rte_eth_dev_fdir_add_perfect_filter(uint8_t port_id, { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-ENODEV); } @@ -1826,7 +1881,7 @@ rte_eth_dev_fdir_update_perfect_filter(uint8_t port_id, { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-ENODEV); } @@ -1864,7 +1919,7 @@ rte_eth_dev_fdir_remove_perfect_filter(uint8_t port_id, { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-ENODEV); } @@ -1900,7 +1955,7 @@ rte_eth_dev_fdir_set_masks(uint8_t port_id, struct rte_fdir_masks *fdir_mask) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-ENODEV); } @@ -1920,7 +1975,7 @@ rte_eth_dev_flow_ctrl_get(uint8_t port_id, struct rte_eth_fc_conf *fc_conf) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-ENODEV); } @@ -1936,7 +1991,7 @@ rte_eth_dev_flow_ctrl_set(uint8_t port_id, struct rte_eth_fc_conf *fc_conf) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-ENODEV); } @@ -1956,7 +2011,7 @@ rte_eth_dev_priority_flow_ctrl_set(uint8_t port_id, struct rte_eth_pfc_conf *pfc { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-ENODEV); } @@ -2035,7 +2090,7 @@ rte_eth_dev_rss_reta_update(uint8_t port_id, struct rte_eth_dev *dev; int ret; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return -ENODEV; } @@ -2086,10 +2141,11 @@ rte_eth_dev_rss_hash_update(uint8_t port_id, struct rte_eth_rss_conf *rss_conf) struct rte_eth_dev *dev; uint16_t rss_hash_protos; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-ENODEV); } + rss_hash_protos = rss_conf->rss_hf; if ((rss_hash_protos != 0) && ((rss_hash_protos & ETH_RSS_PROTO_MASK) == 0)) { @@ -2108,10 +2164,11 @@ rte_eth_dev_rss_hash_conf_get(uint8_t port_id, { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-ENODEV); } + dev = &rte_eth_devices[port_id]; FUNC_PTR_OR_ERR_RET(*dev->dev_ops->rss_hash_conf_get, -ENOTSUP); return (*dev->dev_ops->rss_hash_conf_get)(dev, rss_conf); @@ -2123,7 +2180,7 @@ rte_eth_dev_udp_tunnel_add(uint8_t port_id, { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return -ENODEV; } @@ -2149,10 +2206,11 @@ rte_eth_dev_udp_tunnel_delete(uint8_t port_id, { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return -ENODEV; } + dev = &rte_eth_devices[port_id]; if (udp_tunnel == NULL) { @@ -2174,7 +2232,7 @@ rte_eth_led_on(uint8_t port_id) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-ENODEV); } @@ -2189,7 +2247,7 @@ rte_eth_led_off(uint8_t port_id) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-ENODEV); } @@ -2229,10 +2287,11 @@ rte_eth_dev_mac_addr_add(uint8_t port_id, struct ether_addr *addr, int index; uint64_t pool_mask; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-ENODEV); } + dev = &rte_eth_devices[port_id]; FUNC_PTR_OR_ERR_RET(*dev->dev_ops->mac_addr_add, -ENOTSUP); @@ -2280,10 +2339,11 @@ rte_eth_dev_mac_addr_remove(uint8_t port_id, struct ether_addr *addr) struct rte_eth_dev *dev; int index; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-ENODEV); } + dev = &rte_eth_devices[port_id]; FUNC_PTR_OR_ERR_RET(*dev->dev_ops->mac_addr_remove, -ENOTSUP); @@ -2314,7 +2374,7 @@ rte_eth_dev_set_vf_rxmode(uint8_t port_id, uint16_t vf, struct rte_eth_dev *dev; struct rte_eth_dev_info dev_info; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("set VF RX mode:Invalid port_id=%d\n", port_id); return (-ENODEV); @@ -2369,7 +2429,7 @@ rte_eth_dev_uc_hash_table_set(uint8_t port_id, struct ether_addr *addr, int ret; struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("unicast hash setting:Invalid port_id=%d\n", port_id); return (-ENODEV); @@ -2422,7 +2482,7 @@ rte_eth_dev_uc_all_hash_table_set(uint8_t port_id, uint8_t on) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("unicast hash setting:Invalid port_id=%d\n", port_id); return (-ENODEV); @@ -2441,7 +2501,7 @@ rte_eth_dev_set_vf_rx(uint8_t port_id,uint16_t vf, uint8_t on) struct rte_eth_dev *dev; struct rte_eth_dev_info dev_info; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-ENODEV); } @@ -2467,7 +2527,7 @@ rte_eth_dev_set_vf_tx(uint8_t port_id,uint16_t vf, uint8_t on) struct rte_eth_dev *dev; struct rte_eth_dev_info dev_info; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("set pool tx:Invalid port_id=%d\n", port_id); return (-ENODEV); } @@ -2492,7 +2552,7 @@ rte_eth_dev_set_vf_vlan_filter(uint8_t port_id, uint16_t vlan_id, { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("VF VLAN filter:invalid port id=%d\n", port_id); return (-ENODEV); @@ -2523,7 +2583,7 @@ int rte_eth_set_queue_rate_limit(uint8_t port_id, uint16_t queue_idx, struct rte_eth_dev_info dev_info; struct rte_eth_link link; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("set queue rate limit:invalid port id=%d\n", port_id); return -ENODEV; @@ -2560,7 +2620,7 @@ int rte_eth_set_vf_rate_limit(uint8_t port_id, uint16_t vf, uint16_t tx_rate, if (q_msk == 0) return 0; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("set VF rate limit:invalid port id=%d\n", port_id); return -ENODEV; @@ -2594,7 +2654,7 @@ rte_eth_mirror_rule_set(uint8_t port_id, { struct rte_eth_dev *dev = &rte_eth_devices[port_id]; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-ENODEV); } @@ -2635,7 +2695,7 @@ rte_eth_mirror_rule_reset(uint8_t port_id, uint8_t rule_id) { struct rte_eth_dev *dev = &rte_eth_devices[port_id]; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-ENODEV); } @@ -2660,10 +2720,11 @@ rte_eth_rx_burst(uint8_t port_id, uint16_t queue_id, { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return 0; } + dev = &rte_eth_devices[port_id]; FUNC_PTR_OR_ERR_RET(*dev->rx_pkt_burst, 0); if (queue_id >= dev->data->nb_rx_queues) { @@ -2680,10 +2741,11 @@ rte_eth_tx_burst(uint8_t port_id, uint16_t queue_id, { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return 0; } + dev = &rte_eth_devices[port_id]; FUNC_PTR_OR_ERR_RET(*dev->tx_pkt_burst, 0); @@ -2700,10 +2762,11 @@ rte_eth_rx_queue_count(uint8_t port_id, uint16_t queue_id) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return 0; } + dev = &rte_eth_devices[port_id]; FUNC_PTR_OR_ERR_RET(*dev->dev_ops->rx_queue_count, 0); return (*dev->dev_ops->rx_queue_count)(dev, queue_id); @@ -2714,10 +2777,11 @@ rte_eth_rx_descriptor_done(uint8_t port_id, uint16_t queue_id, uint16_t offset) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-ENODEV); } + dev = &rte_eth_devices[port_id]; FUNC_PTR_OR_ERR_RET(*dev->dev_ops->rx_descriptor_done, -ENOTSUP); return (*dev->dev_ops->rx_descriptor_done)( \ @@ -2735,7 +2799,8 @@ rte_eth_dev_callback_register(uint8_t port_id, if (!cb_fn) return (-EINVAL); - if (port_id >= nb_ports) { + + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-EINVAL); } @@ -2775,7 +2840,8 @@ rte_eth_dev_callback_unregister(uint8_t port_id, if (!cb_fn) return (-EINVAL); - if (port_id >= nb_ports) { + + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-EINVAL); } @@ -2835,7 +2901,7 @@ int rte_eth_dev_bypass_init(uint8_t port_id) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-ENODEV); } @@ -2855,7 +2921,7 @@ rte_eth_dev_bypass_state_show(uint8_t port_id, uint32_t *state) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-ENODEV); } @@ -2874,7 +2940,7 @@ rte_eth_dev_bypass_state_set(uint8_t port_id, uint32_t *new_state) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-ENODEV); } @@ -2894,7 +2960,7 @@ rte_eth_dev_bypass_event_show(uint8_t port_id, uint32_t event, uint32_t *state) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-ENODEV); } @@ -2914,7 +2980,7 @@ rte_eth_dev_bypass_event_store(uint8_t port_id, uint32_t event, uint32_t state) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-ENODEV); } @@ -2934,7 +3000,7 @@ rte_eth_dev_wd_timeout_store(uint8_t port_id, uint32_t timeout) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-ENODEV); } @@ -2954,7 +3020,7 @@ rte_eth_dev_bypass_ver_show(uint8_t port_id, uint32_t *ver) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-ENODEV); } @@ -2974,7 +3040,7 @@ rte_eth_dev_bypass_wd_timeout_show(uint8_t port_id, uint32_t *wd_timeout) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-ENODEV); } @@ -2994,7 +3060,7 @@ rte_eth_dev_bypass_wd_reset(uint8_t port_id) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-ENODEV); } @@ -3016,7 +3082,7 @@ rte_eth_dev_add_syn_filter(uint8_t port_id, { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return -ENODEV; } @@ -3031,7 +3097,7 @@ rte_eth_dev_remove_syn_filter(uint8_t port_id) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return -ENODEV; } @@ -3050,7 +3116,7 @@ rte_eth_dev_get_syn_filter(uint8_t port_id, if (filter == NULL || rx_queue == NULL) return -EINVAL; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return -ENODEV; } @@ -3066,10 +3132,11 @@ rte_eth_dev_add_2tuple_filter(uint8_t port_id, uint16_t index, { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return -ENODEV; } + if (filter->protocol != IPPROTO_TCP && filter->tcp_flags != 0){ PMD_DEBUG_TRACE("tcp flags is 0x%x, but the protocol value" @@ -3088,7 +3155,7 @@ rte_eth_dev_remove_2tuple_filter(uint8_t port_id, uint16_t index) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return -ENODEV; } @@ -3107,7 +3174,7 @@ rte_eth_dev_get_2tuple_filter(uint8_t port_id, uint16_t index, if (filter == NULL || rx_queue == NULL) return -EINVAL; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return -ENODEV; } @@ -3123,7 +3190,7 @@ rte_eth_dev_add_5tuple_filter(uint8_t port_id, uint16_t index, { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return -ENODEV; } @@ -3146,7 +3213,7 @@ rte_eth_dev_remove_5tuple_filter(uint8_t port_id, uint16_t index) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return -ENODEV; } @@ -3165,7 +3232,7 @@ rte_eth_dev_get_5tuple_filter(uint8_t port_id, uint16_t index, if (filter == NULL || rx_queue == NULL) return -EINVAL; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return -ENODEV; } @@ -3182,7 +3249,7 @@ rte_eth_dev_add_flex_filter(uint8_t port_id, uint16_t index, { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return -ENODEV; } @@ -3197,7 +3264,7 @@ rte_eth_dev_remove_flex_filter(uint8_t port_id, uint16_t index) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return -ENODEV; } @@ -3216,7 +3283,7 @@ rte_eth_dev_get_flex_filter(uint8_t port_id, uint16_t index, if (filter == NULL || rx_queue == NULL) return -EINVAL; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return -ENODEV; } @@ -3232,7 +3299,7 @@ rte_eth_dev_filter_supported(uint8_t port_id, enum rte_filter_type filter_type) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return -ENODEV; } @@ -3249,7 +3316,7 @@ rte_eth_dev_filter_ctrl(uint8_t port_id, enum rte_filter_type filter_type, { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return -ENODEV; } diff --git a/lib/librte_ether/rte_ethdev.h b/lib/librte_ether/rte_ethdev.h index 6e454e8..13bdb67 100644 --- a/lib/librte_ether/rte_ethdev.h +++ b/lib/librte_ether/rte_ethdev.h @@ -1540,6 +1540,7 @@ struct rte_eth_dev { struct eth_dev_ops *dev_ops; /**< Functions exported by PMD */ struct rte_pci_device *pci_dev; /**< PCI info. supplied by probing */ struct rte_eth_dev_cb_list callbacks; /**< User application callbacks */ + uint8_t attached; /**< Flag indicating the port is attached */ }; struct rte_eth_dev_sriov { @@ -1605,6 +1606,10 @@ extern struct rte_eth_dev rte_eth_devices[]; * initialized by the [matching] Ethernet driver during the PCI probing phase. * All devices whose port identifier is in the range * [0, rte_eth_dev_count() - 1] can be operated on by network applications. + * immediately after invoking rte_eal_init(). + * If the application unplugs a port using hotplug function, The enabled port + * numbers may be noncontiguous. In the case, the applications need to manage + * enabled port by themselves. * * @return * - The total number of usable Ethernet devices. -- 1.9.1
This patch replaces pci_addr_comparison() and memcmp() of pci addresses by rte_eal_compare_pci_addr(). To compare PCI addresses, rte_eal_compare_pci_addr() doesn't use memcmp(). This is because sizeof(struct rte_pci_addr) returns 6, but actually this structure is like below. struct rte_pci_addr { uint16_t domain; /**< Device domain */ uint8_t bus; /**< Device bus */ uint8_t devid; /**< Device ID */ uint8_t function; /**< Device function. */ }; If the structure is dynamically allocated in a function without bzero, last 1 byte may have value. As a result, memcmp may not work. To avoid such a case, rte_eal_compare_pci_addr() compare following values. dev_addr = (addr->domain << 24) | (addr->bus << 16) | (addr->devid << 8) | addr->function; v9: - eal_compare_pci_addr() is replaced by rte_eal_compare_pci_addr(). - Fix commit log. (Thanks to Thomas Monjalon) v8: - Fix pci_scan_one() to update sysfs values. (Thanks to Qiu, Michael and Iremonger, Bernard) v5: - Fix pci_scan_one to handle pt_driver correctly. v4: - Fix calculation method of eal_compare_pci_addr(). - Add parameter checking. Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp> --- lib/librte_eal/bsdapp/eal/eal_pci.c | 29 ++++++++++++-------------- lib/librte_eal/common/eal_common_pci.c | 2 +- lib/librte_eal/common/include/rte_pci.h | 34 +++++++++++++++++++++++++++++++ lib/librte_eal/linuxapp/eal/eal_pci.c | 30 +++++++++++++-------------- lib/librte_eal/linuxapp/eal/eal_pci_uio.c | 2 +- 5 files changed, 63 insertions(+), 34 deletions(-) diff --git a/lib/librte_eal/bsdapp/eal/eal_pci.c b/lib/librte_eal/bsdapp/eal/eal_pci.c index 74ecce7..9193f80 100644 --- a/lib/librte_eal/bsdapp/eal/eal_pci.c +++ b/lib/librte_eal/bsdapp/eal/eal_pci.c @@ -270,20 +270,6 @@ pci_uio_map_resource(struct rte_pci_device *dev) return (0); } -/* Compare two PCI device addresses. */ -static int -pci_addr_comparison(struct rte_pci_addr *addr, struct rte_pci_addr *addr2) -{ - uint64_t dev_addr = (addr->domain << 24) + (addr->bus << 16) + (addr->devid << 8) + addr->function; - uint64_t dev_addr2 = (addr2->domain << 24) + (addr2->bus << 16) + (addr2->devid << 8) + addr2->function; - - if (dev_addr > dev_addr2) - return 1; - else - return 0; -} - - /* Scan one pci sysfs entry, and fill the devices list from it. */ static int pci_scan_one(int dev_pci_fd, struct pci_conf *conf) @@ -356,13 +342,24 @@ pci_scan_one(int dev_pci_fd, struct pci_conf *conf) } else { struct rte_pci_device *dev2 = NULL; + int ret; TAILQ_FOREACH(dev2, &pci_device_list, next) { - if (pci_addr_comparison(&dev->addr, &dev2->addr)) + ret = rte_eal_compare_pci_addr(&dev->addr, &dev2->addr); + if (ret > 0) continue; - else { + else if (ret < 0) { TAILQ_INSERT_BEFORE(dev2, dev, next); return 0; + } else { /* already registered */ + /* update pt_driver */ + dev2->pt_driver = dev->pt_driver; + dev2->max_vfs = dev->max_vfs; + memmove(dev2->mem_resource, + dev->mem_resource, + sizeof(dev->mem_resource)); + free(dev); + return 0; } } TAILQ_INSERT_TAIL(&pci_device_list, dev, next); diff --git a/lib/librte_eal/common/eal_common_pci.c b/lib/librte_eal/common/eal_common_pci.c index f3c7f71..bf2793f 100644 --- a/lib/librte_eal/common/eal_common_pci.c +++ b/lib/librte_eal/common/eal_common_pci.c @@ -93,7 +93,7 @@ static struct rte_devargs *pci_devargs_lookup(struct rte_pci_device *dev) if (devargs->type != RTE_DEVTYPE_BLACKLISTED_PCI && devargs->type != RTE_DEVTYPE_WHITELISTED_PCI) continue; - if (!memcmp(&dev->addr, &devargs->pci.addr, sizeof(dev->addr))) + if (!rte_eal_compare_pci_addr(&dev->addr, &devargs->pci.addr)) return devargs; } return NULL; diff --git a/lib/librte_eal/common/include/rte_pci.h b/lib/librte_eal/common/include/rte_pci.h index 7f2d699..c609ef3 100644 --- a/lib/librte_eal/common/include/rte_pci.h +++ b/lib/librte_eal/common/include/rte_pci.h @@ -269,6 +269,40 @@ eal_parse_pci_DomBDF(const char *input, struct rte_pci_addr *dev_addr) } #undef GET_PCIADDR_FIELD +/* Compare two PCI device addresses. */ +/** + * Utility function to compare two PCI device addresses. + * + * @param addr + * The PCI Bus-Device-Function address to compare + * @param addr2 + * The PCI Bus-Device-Function address to compare + * @return + * 0 on equal PCI address. + * Positive on addr is greater than addr2. + * Negative on addr is less than addr2, or error. + */ +static inline int +rte_eal_compare_pci_addr(struct rte_pci_addr *addr, struct rte_pci_addr *addr2) +{ + uint64_t dev_addr, dev_addr2; + + if ((addr == NULL) || (addr2 == NULL)) + return -1; + + dev_addr = (addr->domain << 24) | (addr->bus << 16) | + (addr->devid << 8) | addr->function; + dev_addr2 = (addr2->domain << 24) | (addr2->bus << 16) | + (addr2->devid << 8) | addr2->function; + + if (dev_addr > dev_addr2) + return 1; + else if (dev_addr < dev_addr2) + return -1; + else + return 0; +} + /** * Probe the PCI bus for registered drivers. * diff --git a/lib/librte_eal/linuxapp/eal/eal_pci.c b/lib/librte_eal/linuxapp/eal/eal_pci.c index 3c463b2..f593f2c 100644 --- a/lib/librte_eal/linuxapp/eal/eal_pci.c +++ b/lib/librte_eal/linuxapp/eal/eal_pci.c @@ -41,6 +41,7 @@ #include <rte_eal_memconfig.h> #include <rte_malloc.h> #include <rte_devargs.h> +#include <rte_memcpy.h> #include "rte_pci_dev_ids.h" #include "eal_filesystem.h" @@ -229,20 +230,6 @@ error: return -1; } -/* Compare two PCI device addresses. */ -static int -pci_addr_comparison(struct rte_pci_addr *addr, struct rte_pci_addr *addr2) -{ - uint64_t dev_addr = (addr->domain << 24) + (addr->bus << 16) + (addr->devid << 8) + addr->function; - uint64_t dev_addr2 = (addr2->domain << 24) + (addr2->bus << 16) + (addr2->devid << 8) + addr2->function; - - if (dev_addr > dev_addr2) - return 1; - else - return 0; -} - - /* Scan one pci sysfs entry, and fill the devices list from it. */ static int pci_scan_one(const char *dirname, uint16_t domain, uint8_t bus, @@ -360,13 +347,24 @@ pci_scan_one(const char *dirname, uint16_t domain, uint8_t bus, } else { struct rte_pci_device *dev2 = NULL; + int ret; TAILQ_FOREACH(dev2, &pci_device_list, next) { - if (pci_addr_comparison(&dev->addr, &dev2->addr)) + ret = rte_eal_compare_pci_addr(&dev->addr, &dev2->addr); + if (ret > 0) continue; - else { + else if (ret < 0) { TAILQ_INSERT_BEFORE(dev2, dev, next); return 0; + } else { /* already registered */ + /* update pt_driver */ + dev2->pt_driver = dev->pt_driver; + dev2->max_vfs = dev->max_vfs; + memmove(dev2->mem_resource, + dev->mem_resource, + sizeof(dev->mem_resource)); + free(dev); + return 0; } } TAILQ_INSERT_TAIL(&pci_device_list, dev, next); diff --git a/lib/librte_eal/linuxapp/eal/eal_pci_uio.c b/lib/librte_eal/linuxapp/eal/eal_pci_uio.c index e53f06b..d75eb59 100644 --- a/lib/librte_eal/linuxapp/eal/eal_pci_uio.c +++ b/lib/librte_eal/linuxapp/eal/eal_pci_uio.c @@ -123,7 +123,7 @@ pci_uio_map_secondary(struct rte_pci_device *dev) TAILQ_FOREACH(uio_res, pci_res_list, next) { /* skip this element if it doesn't match our PCI address */ - if (memcmp(&uio_res->pci_addr, &dev->addr, sizeof(dev->addr))) + if (rte_eal_compare_pci_addr(&uio_res->pci_addr, &dev->addr)) continue; for (i = 0; i != uio_res->nb_maps; i++) { -- 1.9.1
This patch adds rte_eth_dev_release_port(). The function is used for changing an attached status of the device that has specified name. v9: - rte_eth_dev_free() is replaced by rte_eth_dev_release_port(). (Thanks to Thomas Monjalon) v6: - Use rte_eth_dev structure as the paramter of rte_eth_dev_free(). v4: - Add parameter checking. Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp> --- lib/librte_ether/rte_ethdev.c | 11 +++++++++++ lib/librte_ether/rte_ethdev.h | 12 ++++++++++++ 2 files changed, 23 insertions(+) diff --git a/lib/librte_ether/rte_ethdev.c b/lib/librte_ether/rte_ethdev.c index 6e22216..72eed1b 100644 --- a/lib/librte_ether/rte_ethdev.c +++ b/lib/librte_ether/rte_ethdev.c @@ -255,6 +255,17 @@ rte_eth_dev_allocate(const char *name) return eth_dev; } +int +rte_eth_dev_release_port(struct rte_eth_dev *eth_dev) +{ + if (eth_dev == NULL) + return -EINVAL; + + eth_dev->attached = 0; + nb_ports--; + return 0; +} + static int rte_eth_dev_init(struct rte_pci_driver *pci_drv, struct rte_pci_device *pci_dev) diff --git a/lib/librte_ether/rte_ethdev.h b/lib/librte_ether/rte_ethdev.h index 13bdb67..cd210eb 100644 --- a/lib/librte_ether/rte_ethdev.h +++ b/lib/librte_ether/rte_ethdev.h @@ -1628,6 +1628,18 @@ extern uint8_t rte_eth_dev_count(void); */ struct rte_eth_dev *rte_eth_dev_allocate(const char *name); +/** + * Function for internal use by dummy drivers primarily, e.g. ring-based + * driver. + * Release the specified ethdev port. + * + * @param eth_dev + * The *eth_dev* pointer is the address of the *rte_eth_dev* structure. + * @return + * - 0 on success, negative on error + */ +int rte_eth_dev_release_port(struct rte_eth_dev *eth_dev); + struct eth_driver; /** * @internal -- 1.9.1
The patch adds function pointer to rte_pci_driver and eth_driver structure. These function pointers are used when ports are detached. Also, the patch adds rte_eth_dev_uninit(). So far, it's not called by anywhere, but it will be called when port hotplug function is implemented. v10: - Add size parameter to rte_eth_dev_create_unique_device_name(). (Thanks to Iremonger, Bernard) v9: - Change parameter of pci_devuninit_t and rte_eth_dev_uninit. - Remove code that initiaize callback of ethdev from rte_eth_dev_uninit(). - Add a function to create a unique device name. (Thanks to Thomas Monjalon) v6: - Fix rte_eth_dev_uninit() to handle a return value of uninit function of PMD. v4: - Add parameter checking. - Change function names. Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp> --- lib/librte_eal/common/include/rte_pci.h | 6 ++++ lib/librte_ether/rte_ethdev.c | 64 +++++++++++++++++++++++++++++++-- lib/librte_ether/rte_ethdev.h | 24 +++++++++++++ 3 files changed, 92 insertions(+), 2 deletions(-) diff --git a/lib/librte_eal/common/include/rte_pci.h b/lib/librte_eal/common/include/rte_pci.h index c609ef3..376f66a 100644 --- a/lib/librte_eal/common/include/rte_pci.h +++ b/lib/librte_eal/common/include/rte_pci.h @@ -189,12 +189,18 @@ struct rte_pci_driver; typedef int (pci_devinit_t)(struct rte_pci_driver *, struct rte_pci_device *); /** + * Uninitialisation function for the driver called during hotplugging. + */ +typedef int (pci_devuninit_t)(struct rte_pci_device *); + +/** * A structure describing a PCI driver. */ struct rte_pci_driver { TAILQ_ENTRY(rte_pci_driver) next; /**< Next in list. */ const char *name; /**< Driver name. */ pci_devinit_t *devinit; /**< Device init. function. */ + pci_devuninit_t *devuninit; /**< Device uninit function. */ struct rte_pci_id *id_table; /**< ID table, NULL terminated. */ uint32_t drv_flags; /**< Flags contolling handling of device. */ }; diff --git a/lib/librte_ether/rte_ethdev.c b/lib/librte_ether/rte_ethdev.c index 72eed1b..bdb5a7b 100644 --- a/lib/librte_ether/rte_ethdev.c +++ b/lib/librte_ether/rte_ethdev.c @@ -266,6 +266,24 @@ rte_eth_dev_release_port(struct rte_eth_dev *eth_dev) return 0; } +static inline int +rte_eth_dev_create_unique_device_name(char *name, size_t size, + struct rte_pci_device *pci_dev) +{ + int ret; + + if ((name == NULL) || (pci_dev == NULL)) + return -EINVAL; + + ret = snprintf(name, size, "%d:%d.%d", + pci_dev->addr.bus, pci_dev->addr.devid, + pci_dev->addr.function); + if (ret < 0) + return ret; + + return 0; +} + static int rte_eth_dev_init(struct rte_pci_driver *pci_drv, struct rte_pci_device *pci_dev) @@ -279,8 +297,8 @@ rte_eth_dev_init(struct rte_pci_driver *pci_drv, eth_drv = (struct eth_driver *)pci_drv; /* Create unique Ethernet device name using PCI address */ - snprintf(ethdev_name, RTE_ETH_NAME_MAX_LEN, "%d:%d.%d", - pci_dev->addr.bus, pci_dev->addr.devid, pci_dev->addr.function); + rte_eth_dev_create_unique_device_name(ethdev_name, + sizeof(ethdev_name), pci_dev); eth_dev = rte_eth_dev_allocate(ethdev_name); if (eth_dev == NULL) @@ -321,6 +339,47 @@ rte_eth_dev_init(struct rte_pci_driver *pci_drv, return diag; } +static int +rte_eth_dev_uninit(struct rte_pci_device *pci_dev) +{ + const struct eth_driver *eth_drv; + struct rte_eth_dev *eth_dev; + char ethdev_name[RTE_ETH_NAME_MAX_LEN]; + int ret; + + if (pci_dev == NULL) + return -EINVAL; + + /* Create unique Ethernet device name using PCI address */ + rte_eth_dev_create_unique_device_name(ethdev_name, + sizeof(ethdev_name), pci_dev); + + eth_dev = rte_eth_dev_allocated(ethdev_name); + if (eth_dev == NULL) + return -ENODEV; + + eth_drv = (const struct eth_driver *)pci_dev->driver; + + /* Invoke PMD device uninit function */ + if (*eth_drv->eth_dev_uninit) { + ret = (*eth_drv->eth_dev_uninit)(eth_drv, eth_dev); + if (ret) + return ret; + } + + /* free ether device */ + rte_eth_dev_release_port(eth_dev); + + if (rte_eal_process_type() == RTE_PROC_PRIMARY) + rte_free(eth_dev->data->dev_private); + + eth_dev->pci_dev = NULL; + eth_dev->driver = NULL; + eth_dev->data = NULL; + + return 0; +} + /** * Register an Ethernet [Poll Mode] driver. * @@ -339,6 +398,7 @@ void rte_eth_driver_register(struct eth_driver *eth_drv) { eth_drv->pci_drv.devinit = rte_eth_dev_init; + eth_drv->pci_drv.devuninit = rte_eth_dev_uninit; rte_eal_pci_register(ð_drv->pci_drv); } diff --git a/lib/librte_ether/rte_ethdev.h b/lib/librte_ether/rte_ethdev.h index cd210eb..fa3ecab 100644 --- a/lib/librte_ether/rte_ethdev.h +++ b/lib/librte_ether/rte_ethdev.h @@ -1677,6 +1677,27 @@ typedef int (*eth_dev_init_t)(struct eth_driver *eth_drv, /** * @internal + * Finalization function of an Ethernet driver invoked for each matching + * Ethernet PCI device detected during the PCI closing phase. + * + * @param eth_drv + * The pointer to the [matching] Ethernet driver structure supplied by + * the PMD when it registered itself. + * @param eth_dev + * The *eth_dev* pointer is the address of the *rte_eth_dev* structure + * associated with the matching device and which have been [automatically] + * allocated in the *rte_eth_devices* array. + * @return + * - 0: Success, the device is properly finalized by the driver. + * In particular, the driver MUST free the *dev_ops* pointer + * of the *eth_dev* structure. + * - <0: Error code of the device initialization failure. + */ +typedef int (*eth_dev_uninit_t)(const struct eth_driver *eth_drv, + struct rte_eth_dev *eth_dev); + +/** + * @internal * The structure associated with a PMD Ethernet driver. * * Each Ethernet driver acts as a PCI driver and is represented by a generic @@ -1686,11 +1707,14 @@ typedef int (*eth_dev_init_t)(struct eth_driver *eth_drv, * * - The *eth_dev_init* function invoked for each matching PCI device. * + * - The *eth_dev_uninit* function invoked for each matching PCI device. + * * - The size of the private data to allocate for each matching device. */ struct eth_driver { struct rte_pci_driver pci_drv; /**< The PMD is also a PCI driver. */ eth_dev_init_t eth_dev_init; /**< Device init function. */ + eth_dev_uninit_t eth_dev_uninit; /**< Device uninit function. */ unsigned int dev_private_size; /**< Size of device private data. */ }; -- 1.9.1
The patch adds following functions. - rte_eth_dev_save() The function is used for saving current rte_eth_dev structures. - rte_eth_dev_get_changed_port() The function receives the rte_eth_dev structures, then compare these with current values to know which port is actually attached or detached. - rte_eth_dev_get_addr_by_port() The function returns a pci address of an ethdev specified by port identifier. - rte_eth_dev_get_port_by_addr() The function returns a port identifier of an ethdev specified by pci address. - rte_eth_dev_get_name_by_port() The function returns a unique identifier name of an ethdev specified by port identifier. - Add rte_eth_dev_is_detachable() The function returns whether a PMD supports detach function. Also, the patch changes scope of rte_eth_dev_allocated() to global. This function will be called by virtual PMDs to support port hotplug. So change scope of the function to global. v10: - Change order of version.map. (Thanks to Thomas Monjalon) v9: - rte_eth_dev_check_detachable() is replaced by rte_eth_dev_is_detachable(). - strncpy() is replaced by strcpy(). (Thanks to Thomas Monjalon) - Add missing symbol in version map. (Thanks to Nail Horman) v8: - Add size parameter to rte_eth_dev_save(). - Add missing symbol in version map. (Thanks to Qiu, Michael and Iremonger, Bernard) v7: - Add pt_driver checking to rte_eth_dev_check_detachable(). (Thanks to Qiu, Michael) v5: - Fix return value of below functions. rte_eth_dev_get_changed_port(). rte_eth_dev_get_port_by_addr(). v4: - Add parameter checking. v3: - Fix if-condition bug while comparing pci addresses. - Add error checking codes. Reported-by: Mark Enright <menrigh@brocade.com> Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp> --- lib/librte_ether/rte_ethdev.c | 103 ++++++++++++++++++++++++++++++++- lib/librte_ether/rte_ethdev.h | 83 ++++++++++++++++++++++++++ lib/librte_ether/rte_ether_version.map | 7 +++ 3 files changed, 192 insertions(+), 1 deletion(-) diff --git a/lib/librte_ether/rte_ethdev.c b/lib/librte_ether/rte_ethdev.c index bdb5a7b..5f166d6 100644 --- a/lib/librte_ether/rte_ethdev.c +++ b/lib/librte_ether/rte_ethdev.c @@ -201,7 +201,7 @@ rte_eth_dev_data_alloc(void) RTE_MAX_ETHPORTS * sizeof(*rte_eth_dev_data)); } -static struct rte_eth_dev * +struct rte_eth_dev * rte_eth_dev_allocated(const char *name) { unsigned i; @@ -426,6 +426,107 @@ rte_eth_dev_count(void) return (nb_ports); } +int +rte_eth_dev_save(struct rte_eth_dev *devs, size_t size) +{ + if ((devs == NULL) || + (size != sizeof(struct rte_eth_dev) * RTE_MAX_ETHPORTS)) + return -EINVAL; + + /* save current rte_eth_devices */ + memcpy(devs, rte_eth_devices, size); + return 0; +} + +int +rte_eth_dev_get_changed_port(struct rte_eth_dev *devs, uint8_t *port_id) +{ + if ((devs == NULL) || (port_id == NULL)) + return -EINVAL; + + /* check which port was attached or detached */ + for (*port_id = 0; *port_id < RTE_MAX_ETHPORTS; (*port_id)++, devs++) { + if (rte_eth_devices[*port_id].attached ^ devs->attached) + return 0; + } + return -ENODEV; +} + +int +rte_eth_dev_get_addr_by_port(uint8_t port_id, struct rte_pci_addr *addr) +{ + if (!rte_eth_dev_is_valid_port(port_id)) { + PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + return -EINVAL; + } + + if (addr == NULL) { + PMD_DEBUG_TRACE("Null pointer is specified\n"); + return -EINVAL; + } + + *addr = rte_eth_devices[port_id].pci_dev->addr; + return 0; +} + +int +rte_eth_dev_get_port_by_addr(struct rte_pci_addr *addr, uint8_t *port_id) +{ + struct rte_pci_addr *tmp; + + if ((addr == NULL) || (port_id == NULL)) { + PMD_DEBUG_TRACE("Null pointer is specified\n"); + return -EINVAL; + } + + for (*port_id = 0; *port_id < RTE_MAX_ETHPORTS; (*port_id)++) { + if (!rte_eth_devices[*port_id].attached) + continue; + if (!rte_eth_devices[*port_id].pci_dev) + continue; + tmp = &rte_eth_devices[*port_id].pci_dev->addr; + if (rte_eal_compare_pci_addr(tmp, addr) == 0) + return 0; + } + return -ENODEV; +} + +int +rte_eth_dev_get_name_by_port(uint8_t port_id, char *name) +{ + char *tmp; + + if (!rte_eth_dev_is_valid_port(port_id)) { + PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + return -EINVAL; + } + + if (name == NULL) { + PMD_DEBUG_TRACE("Null pointer is specified\n"); + return -EINVAL; + } + + /* shouldn't check 'rte_eth_devices[i].data', + * because it might be overwritten by VDEV PMD */ + tmp = rte_eth_dev_data[port_id].name; + strcpy(name, tmp); + return 0; +} + +int +rte_eth_dev_is_detachable(uint8_t port_id) +{ + uint32_t drv_flags; + + if (port_id >= RTE_MAX_ETHPORTS) { + PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + return -EINVAL; + } + + drv_flags = rte_eth_devices[port_id].driver->pci_drv.drv_flags; + return !(drv_flags & RTE_PCI_DRV_DETACHABLE); +} + static int rte_eth_dev_rx_queue_config(struct rte_eth_dev *dev, uint16_t nb_queues) { diff --git a/lib/librte_ether/rte_ethdev.h b/lib/librte_ether/rte_ethdev.h index fa3ecab..65a3484 100644 --- a/lib/librte_ether/rte_ethdev.h +++ b/lib/librte_ether/rte_ethdev.h @@ -1617,6 +1617,89 @@ extern struct rte_eth_dev rte_eth_devices[]; extern uint8_t rte_eth_dev_count(void); /** + * Function for internal use by port hotplug functions. + * Copies current ethdev structures to the specified pointer. + * + * @param devs The pointer to the ethdev structures + * @param size The size of ethdev structures + * @return + * - 0 on success, negative on error + */ +extern int rte_eth_dev_save(struct rte_eth_dev *devs, size_t size); + +/** + * Function for internal use by port hotplug functions. + * Compare the specified ethdev structures with currents. Then + * if there is a port which status is changed, fill the specified pointer + * with the port id of that port. + * @param devs The pointer to the ethdev structures + * @param port_id The pointer to the port id + * @return + * - 0 on success, negative on error + */ +extern int rte_eth_dev_get_changed_port( + struct rte_eth_dev *devs, uint8_t *port_id); + +/** + * Function for internal use by port hotplug functions. + * Returns a pci address of a ethdev specified by port identifier. + * @param port_id + * The port identifier of the Ethernet device + * @param addr + * The pointer to the pci address + * @return + * - 0 on success, negative on error + */ +extern int rte_eth_dev_get_addr_by_port( + uint8_t port_id, struct rte_pci_addr *addr); + +/** + * Function for internal use by port hotplug functions. + * Returns a port identifier of a ethdev specified by pci address. + * @param addr + * The pointer to the pci address of the Ethernet device. + * @param port_id + * The pointer to the port identifier + * @return + * - 0 on success, negative on error + */ +extern int rte_eth_dev_get_port_by_addr( + struct rte_pci_addr *addr, uint8_t *port_id); + +/** + * Function for internal use by port hotplug functions. + * Returns a unique identifier name of a ethdev specified by port identifier. + * @param port_id + * The port identifier. + * @param name + * The pointer to the Unique identifier name for each Ethernet device + * @return + * - 0 on success, negative on error + */ +extern int rte_eth_dev_get_name_by_port(uint8_t port_id, char *name); + +/** + * Function for internal use by port hotplug functions. + * Check whether or not, a PMD that is handling the ethdev specified by port + * identifier can support detach function. + * @param port_id + * The port identifier + * @return + * - 0 on supporting detach function, negative on not supporting + */ +extern int rte_eth_dev_is_detachable(uint8_t port_id); + +/** + * Function for internal use by port hotplug functions. + * Returns a ethdev slot specified by the unique identifier name. + * @param name + * The pointer to the Unique identifier name for each Ethernet device + * @return + * - The pointer to the ethdev slot, on success. NULL on error + */ +extern struct rte_eth_dev *rte_eth_dev_allocated(const char *name); + +/** * Function for internal use by dummy drivers primarily, e.g. ring-based * driver. * Allocates a new ethdev slot for an ethernet device and returns the pointer diff --git a/lib/librte_ether/rte_ether_version.map b/lib/librte_ether/rte_ether_version.map index 7316530..666c3af 100644 --- a/lib/librte_ether/rte_ether_version.map +++ b/lib/librte_ether/rte_ether_version.map @@ -11,6 +11,7 @@ DPDK_2.0 { rte_eth_dev_add_flex_filter; rte_eth_dev_add_syn_filter; rte_eth_dev_allocate; + rte_eth_dev_allocated; rte_eth_dev_bypass_event_show; rte_eth_dev_bypass_event_store; rte_eth_dev_bypass_init; @@ -39,12 +40,17 @@ DPDK_2.0 { rte_eth_dev_flow_ctrl_set; rte_eth_dev_get_2tuple_filter; rte_eth_dev_get_5tuple_filter; + rte_eth_dev_get_addr_by_port; + rte_eth_dev_get_changed_port; rte_eth_dev_get_ethertype_filter; rte_eth_dev_get_flex_filter; rte_eth_dev_get_mtu; + rte_eth_dev_get_name_by_port; + rte_eth_dev_get_port_by_addr; rte_eth_dev_get_syn_filter; rte_eth_dev_get_vlan_offload; rte_eth_dev_info_get; + rte_eth_dev_is_detachable; rte_eth_dev_mac_addr_add; rte_eth_dev_mac_addr_remove; rte_eth_dev_priority_flow_ctrl_set; @@ -59,6 +65,7 @@ DPDK_2.0 { rte_eth_dev_rss_reta_update; rte_eth_dev_rx_queue_start; rte_eth_dev_rx_queue_stop; + rte_eth_dev_save; rte_eth_dev_set_link_down; rte_eth_dev_set_link_up; rte_eth_dev_set_mtu; -- 1.9.1
The patch adds functions for unmapping igb_uio resources. The patch is only for Linux and igb_uio environment. VFIO and BSD are not supported. v9: - Remove "rte_dev_hotplug.h". - Remove needless "#ifdef". (Thanks to Thomas Monjalon and Neil Horman) - Remove pci_unmap_device(). It will be implemented in later patch. v8: - Fix typo. (Thanks to Iremonger, Bernard) v5: - Fix pci_unmap_device() to check pt_driver. v4: - Add parameter checking. - Add header file to determine if hotplug can be enabled. Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp> --- lib/librte_eal/linuxapp/eal/eal_pci.c | 17 ++++++++ lib/librte_eal/linuxapp/eal/eal_pci_init.h | 7 ++++ lib/librte_eal/linuxapp/eal/eal_pci_uio.c | 65 ++++++++++++++++++++++++++++++ 3 files changed, 89 insertions(+) diff --git a/lib/librte_eal/linuxapp/eal/eal_pci.c b/lib/librte_eal/linuxapp/eal/eal_pci.c index f593f2c..7349a60 100644 --- a/lib/librte_eal/linuxapp/eal/eal_pci.c +++ b/lib/librte_eal/linuxapp/eal/eal_pci.c @@ -167,6 +167,23 @@ pci_map_resource(void *requested_addr, int fd, off_t offset, size_t size) return mapaddr; } +/* unmap a particular resource */ +void +pci_unmap_resource(void *requested_addr, size_t size) +{ + if (requested_addr == NULL) + return; + + /* Unmap the PCI memory resource of device */ + if (munmap(requested_addr, size)) { + RTE_LOG(ERR, EAL, "%s(): cannot munmap(%p, 0x%lx): %s\n", + __func__, requested_addr, (unsigned long)size, + strerror(errno)); + } else + RTE_LOG(DEBUG, EAL, " PCI memory unmapped at %p\n", + requested_addr); +} + /* parse the "resource" sysfs file */ #define IORESOURCE_MEM 0x00000200 diff --git a/lib/librte_eal/linuxapp/eal/eal_pci_init.h b/lib/librte_eal/linuxapp/eal/eal_pci_init.h index 1070eb8..e2dd8a5 100644 --- a/lib/librte_eal/linuxapp/eal/eal_pci_init.h +++ b/lib/librte_eal/linuxapp/eal/eal_pci_init.h @@ -71,6 +71,13 @@ void *pci_map_resource(void *requested_addr, int fd, off_t offset, /* map IGB_UIO resource prototype */ int pci_uio_map_resource(struct rte_pci_device *dev); +void pci_unmap_resource(void *requested_addr, size_t size); + +#ifdef RTE_LIBRTE_EAL_HOTPLUG +/* unmap IGB_UIO resource prototype */ +void pci_uio_unmap_resource(struct rte_pci_device *dev); +#endif /* RTE_LIBRTE_EAL_HOTPLUG */ + #ifdef VFIO_PRESENT #define VFIO_MAX_GROUPS 64 diff --git a/lib/librte_eal/linuxapp/eal/eal_pci_uio.c b/lib/librte_eal/linuxapp/eal/eal_pci_uio.c index d75eb59..43f47dc 100644 --- a/lib/librte_eal/linuxapp/eal/eal_pci_uio.c +++ b/lib/librte_eal/linuxapp/eal/eal_pci_uio.c @@ -404,6 +404,71 @@ pci_uio_map_resource(struct rte_pci_device *dev) return 0; } +#ifdef RTE_LIBRTE_EAL_HOTPLUG +static void +pci_uio_unmap(struct mapped_pci_resource *uio_res) +{ + int i; + + if (uio_res == NULL) + return; + + for (i = 0; i != uio_res->nb_maps; i++) + pci_unmap_resource(uio_res->maps[i].addr, + (size_t)uio_res->maps[i].size); +} + +static struct mapped_pci_resource * +pci_uio_find_resource(struct rte_pci_device *dev) +{ + struct mapped_pci_resource *uio_res; + + if (dev == NULL) + return NULL; + + TAILQ_FOREACH(uio_res, pci_res_list, next) { + + /* skip this element if it doesn't match our PCI address */ + if (!rte_eal_compare_pci_addr(&uio_res->pci_addr, &dev->addr)) + return uio_res; + } + return NULL; +} + +/* unmap the PCI resource of a PCI device in virtual memory */ +void +pci_uio_unmap_resource(struct rte_pci_device *dev) +{ + struct mapped_pci_resource *uio_res; + + if (dev == NULL) + return; + + /* find an entry for the device */ + uio_res = pci_uio_find_resource(dev); + if (uio_res == NULL) + return; + + /* secondary processes - just free maps */ + if (rte_eal_process_type() != RTE_PROC_PRIMARY) + return pci_uio_unmap(uio_res); + + TAILQ_REMOVE(pci_res_list, uio_res, next); + + /* unmap all resources */ + pci_uio_unmap(uio_res); + + /* free uio resource */ + rte_free(uio_res); + + /* close fd if in primary process */ + close(dev->intr_handle.fd); + + dev->intr_handle.fd = -1; + dev->intr_handle.type = RTE_INTR_HANDLE_UNKNOWN; +} +#endif /* RTE_LIBRTE_EAL_HOTPLUG */ + /* * parse a sysfs file containing one integer value * different to the eal version, as it needs to work with 64-bit values -- 1.9.1
The function removes the specified devargs entry from devargs_list. Also, the patch adds sanity checking to rte_eal_devargs_add(). v5: - Change function definition of rte_eal_devargs_remove(). v4: - Fix sanity check code. Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp> --- lib/librte_eal/common/eal_common_devargs.c | 61 +++++++++++++++++++++++++++++ lib/librte_eal/common/include/rte_devargs.h | 21 ++++++++++ 2 files changed, 82 insertions(+) diff --git a/lib/librte_eal/common/eal_common_devargs.c b/lib/librte_eal/common/eal_common_devargs.c index 3aace08..06954c3 100644 --- a/lib/librte_eal/common/eal_common_devargs.c +++ b/lib/librte_eal/common/eal_common_devargs.c @@ -44,6 +44,36 @@ struct rte_devargs_list devargs_list = TAILQ_HEAD_INITIALIZER(devargs_list); + +/* find a entry specified by pci address or device name */ +static struct rte_devargs * +rte_eal_devargs_find(enum rte_devtype devtype, void *args) +{ + struct rte_devargs *devargs; + + if (args == NULL) + return NULL; + + TAILQ_FOREACH(devargs, &devargs_list, next) { + switch (devtype) { + case RTE_DEVTYPE_WHITELISTED_PCI: + case RTE_DEVTYPE_BLACKLISTED_PCI: + if (rte_eal_compare_pci_addr( + &devargs->pci.addr, args) == 0) + goto found; + break; + case RTE_DEVTYPE_VIRTUAL: + if (memcmp(&devargs->virtual.drv_name, args, + strlen((char *)args)) == 0) + goto found; + break; + } + } + return NULL; +found: + return devargs; +} + /* store a whitelist parameter for later parsing */ int rte_eal_devargs_add(enum rte_devtype devtype, const char *devargs_str) @@ -88,6 +118,12 @@ rte_eal_devargs_add(enum rte_devtype devtype, const char *devargs_str) RTE_LOG(ERR, EAL, "invalid PCI identifier <%s>\n", buf); goto fail; } + /* make sure there is no same entry */ + if (rte_eal_devargs_find(devtype, &devargs->pci.addr)) { + RTE_LOG(ERR, EAL, + "device already registered: <%s>\n", buf); + return -1; + } break; case RTE_DEVTYPE_VIRTUAL: /* save driver name */ @@ -97,6 +133,12 @@ rte_eal_devargs_add(enum rte_devtype devtype, const char *devargs_str) RTE_LOG(ERR, EAL, "driver name too large: <%s>\n", buf); goto fail; } + /* make sure there is no same entry */ + if (rte_eal_devargs_find(devtype, &devargs->virtual.drv_name)) { + RTE_LOG(ERR, EAL, + "device already registered: <%s>\n", buf); + return -1; + } break; } @@ -114,6 +156,25 @@ fail: return -1; } +/* remove it from the devargs_list */ +int +rte_eal_devargs_remove(enum rte_devtype devtype, void *args) +{ + struct rte_devargs *devargs; + + if (args == NULL) + return -EINVAL; + + devargs = rte_eal_devargs_find(devtype, args); + if (devargs == NULL) { + RTE_LOG(ERR, EAL, "device not found\n"); + return -ENODEV; + } + + TAILQ_REMOVE(&devargs_list, devargs, next); + return 0; +} + /* count the number of devices of a specified type */ unsigned int rte_eal_devargs_type_count(enum rte_devtype devtype) diff --git a/lib/librte_eal/common/include/rte_devargs.h b/lib/librte_eal/common/include/rte_devargs.h index 996e180..abfa3b1 100644 --- a/lib/librte_eal/common/include/rte_devargs.h +++ b/lib/librte_eal/common/include/rte_devargs.h @@ -123,6 +123,27 @@ extern struct rte_devargs_list devargs_list; int rte_eal_devargs_add(enum rte_devtype devtype, const char *devargs_str); /** + * Remove a device from the user device list + * + * For PCI devices, the format of arguments string is "PCI_ADDR". It shouldn't + * involve parameters for the device. Example: "08:00.1". + * + * For virtual devices, the format of arguments string is "DRIVER_NAME*". It + * shouldn't involve parameters for the device. Example: "eth_ring". The + * validity of the driver name is not checked by this function, it is done + * when closing the drivers. + * + * @param devtype + * The type of the device. + * @param name + * The name of the device. + * + * @return + * - 0 on success, negative on error + */ +int rte_eal_devargs_remove(enum rte_devtype devtype, void *args); + +/** * Count the number of user devices of a specified type * * @param devtype -- 1.9.1
- Add pci_close_all_drivers() The function tries to find a driver for the specified device, and then close the driver. - Add rte_eal_pci_probe_one() and rte_eal_pci_close_one() The functions are used for probe and close a device. First the function tries to find a device that has the specified PCI address. Then, probe or close the device. v9: - Fix commit title. - Remove RTE_EAL_INVOKE_TYPE_PROBE/CLOSE. (Thanks to Thomas Monjalon) - Implement pci_unmap_device() in this patch. v5: - Remove RTE_EAL_INVOKE_TYPE_UNKNOWN, because it's unused. v4: - Fix parameter checking. - Fix indent of 'if' statement. Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp> --- lib/librte_eal/common/eal_common_pci.c | 98 ++++++++++++++++++++++++++++++++- lib/librte_eal/common/eal_private.h | 15 +++++ lib/librte_eal/common/include/rte_pci.h | 32 +++++++++++ lib/librte_eal/linuxapp/eal/eal_pci.c | 94 +++++++++++++++++++++++++++++++ 4 files changed, 238 insertions(+), 1 deletion(-) diff --git a/lib/librte_eal/common/eal_common_pci.c b/lib/librte_eal/common/eal_common_pci.c index bf2793f..5b6b55d 100644 --- a/lib/librte_eal/common/eal_common_pci.c +++ b/lib/librte_eal/common/eal_common_pci.c @@ -108,7 +108,10 @@ static int pci_probe_all_drivers(struct rte_pci_device *dev) { struct rte_pci_driver *dr = NULL; - int rc; + int rc = 0; + + if (dev == NULL) + return -1; TAILQ_FOREACH(dr, &pci_driver_list, next) { rc = rte_eal_pci_probe_one_driver(dr, dev); @@ -123,6 +126,99 @@ pci_probe_all_drivers(struct rte_pci_device *dev) return 1; } +#ifdef RTE_LIBRTE_EAL_HOTPLUG +/* + * If vendor/device ID match, call the devuninit() function of all + * registered driver for the given device. Return -1 if initialization + * failed, return 1 if no driver is found for this device. + */ +static int +pci_close_all_drivers(struct rte_pci_device *dev) +{ + struct rte_pci_driver *dr = NULL; + int rc = 0; + + if (dev == NULL) + return -1; + + TAILQ_FOREACH(dr, &pci_driver_list, next) { + rc = rte_eal_pci_close_one_driver(dr, dev); + if (rc < 0) + /* negative value is an error */ + return -1; + if (rc > 0) + /* positive value means driver not found */ + continue; + return 0; + } + return 1; +} + +/* + * Find the pci device specified by pci address, then invoke probe function of + * the driver of the devive. + */ +int +rte_eal_pci_probe_one(struct rte_pci_addr *addr) +{ + struct rte_pci_device *dev = NULL; + int ret = 0; + + if (addr == NULL) + return -1; + + TAILQ_FOREACH(dev, &pci_device_list, next) { + if (rte_eal_compare_pci_addr(&dev->addr, addr)) + continue; + + ret = pci_probe_all_drivers(dev); + if (ret < 0) + goto err_return; + return 0; + } + return -1; + +err_return: + RTE_LOG(WARNING, EAL, "Requested device " PCI_PRI_FMT + " cannot be used\n", dev->addr.domain, dev->addr.bus, + dev->addr.devid, dev->addr.function); + return -1; +} + +/* + * Find the pci device specified by pci address, then invoke close function of + * the driver of the devive. + */ +int +rte_eal_pci_close_one(struct rte_pci_addr *addr) +{ + struct rte_pci_device *dev = NULL; + int ret = 0; + + if (addr == NULL) + return -1; + + TAILQ_FOREACH(dev, &pci_device_list, next) { + if (rte_eal_compare_pci_addr(&dev->addr, addr)) + continue; + + ret = pci_close_all_drivers(dev); + if (ret < 0) + goto err_return; + + TAILQ_REMOVE(&pci_device_list, dev, next); + return 0; + } + return -1; + +err_return: + RTE_LOG(WARNING, EAL, "Requested device " PCI_PRI_FMT + " cannot be used\n", dev->addr.domain, dev->addr.bus, + dev->addr.devid, dev->addr.function); + return -1; +} +#endif /* RTE_LIBRTE_EAL_HOTPLUG */ + /* * Scan the content of the PCI bus, and call the devinit() function for * all registered drivers that have a matching entry in its id_table diff --git a/lib/librte_eal/common/eal_private.h b/lib/librte_eal/common/eal_private.h index 159cd66..4acf5a0 100644 --- a/lib/librte_eal/common/eal_private.h +++ b/lib/librte_eal/common/eal_private.h @@ -165,6 +165,21 @@ int rte_eal_pci_probe_one_driver(struct rte_pci_driver *dr, struct rte_pci_device *dev); /** + * Munmap memory for single PCI device + * + * This function is private to EAL. + * + * @param dr + * The pointer to the pci driver structure + * @param dev + * The pointer to the pci device structure + * @return + * 0 on success, negative on error + */ +int rte_eal_pci_close_one_driver(struct rte_pci_driver *dr, + struct rte_pci_device *dev); + +/** * Init tail queues for non-EAL library structures. This is to allow * the rings, mempools, etc. lists to be shared among multiple processes * diff --git a/lib/librte_eal/common/include/rte_pci.h b/lib/librte_eal/common/include/rte_pci.h index 376f66a..4aed70a 100644 --- a/lib/librte_eal/common/include/rte_pci.h +++ b/lib/librte_eal/common/include/rte_pci.h @@ -322,6 +322,38 @@ rte_eal_compare_pci_addr(struct rte_pci_addr *addr, struct rte_pci_addr *addr2) */ int rte_eal_pci_probe(void); +#ifdef RTE_LIBRTE_EAL_HOTPLUG +/** + * Probe the single PCI device. + * + * Scan the content of the PCI bus, and find the pci device specified by pci + * address, then call the probe() function for registered driver that has a + * matching entry in its id_table for discovered device. + * + * @param addr + * The PCI Bus-Device-Function address to probe. + * @return + * - 0 on success. + * - Negative on error. + */ +int rte_eal_pci_probe_one(struct rte_pci_addr *addr); + +/** + * Close the single PCI device. + * + * Scan the content of the PCI bus, and find the pci device specified by pci + * address, then call the close() function for registered driver that has a + * matching entry in its id_table for discovered device. + * + * @param addr + * The PCI Bus-Device-Function address to close. + * @return + * - 0 on success. + * - Negative on error. + */ +int rte_eal_pci_close_one(struct rte_pci_addr *addr); +#endif /* RTE_LIBRTE_EAL_HOTPLUG */ + /** * Dump the content of the PCI bus. * diff --git a/lib/librte_eal/linuxapp/eal/eal_pci.c b/lib/librte_eal/linuxapp/eal/eal_pci.c index 7349a60..4bdf51b 100644 --- a/lib/librte_eal/linuxapp/eal/eal_pci.c +++ b/lib/librte_eal/linuxapp/eal/eal_pci.c @@ -596,6 +596,31 @@ pci_map_device(struct rte_pci_device *dev) return ret; } +#ifdef RTE_LIBRTE_EAL_HOTPLUG +static void +pci_unmap_device(struct rte_pci_device *dev) +{ + if (dev == NULL) + return; + + /* try unmapping the NIC resources using VFIO if it exists */ + switch (dev->pt_driver) { + case RTE_PT_VFIO: + RTE_LOG(ERR, EAL, "Hotplug doesn't support vfio yet\n"); + break; + case RTE_PT_IGB_UIO: + case RTE_PT_UIO_GENERIC: + /* unmap resources for devices that use uio */ + pci_uio_unmap_resource(dev); + break; + default: + RTE_LOG(DEBUG, EAL, " Not managed by known pt driver," + " skipped\n"); + break; + } +} +#endif /* RTE_LIBRTE_EAL_HOTPLUG */ + /* * If vendor/device ID match, call the devinit() function of the * driver. @@ -667,6 +692,75 @@ rte_eal_pci_probe_one_driver(struct rte_pci_driver *dr, struct rte_pci_device *d return 1; } +#ifdef RTE_LIBRTE_EAL_HOTPLUG +/* + * If vendor/device ID match, call the devuninit() function of the + * driver. + */ +int +rte_eal_pci_close_one_driver(struct rte_pci_driver *dr, + struct rte_pci_device *dev) +{ + struct rte_pci_id *id_table; + + if ((dr == NULL) || (dev == NULL)) + return -EINVAL; + + for (id_table = dr->id_table ; id_table->vendor_id != 0; id_table++) { + + /* check if device's identifiers match the driver's ones */ + if (id_table->vendor_id != dev->id.vendor_id && + id_table->vendor_id != PCI_ANY_ID) + continue; + if (id_table->device_id != dev->id.device_id && + id_table->device_id != PCI_ANY_ID) + continue; + if (id_table->subsystem_vendor_id != + dev->id.subsystem_vendor_id && + id_table->subsystem_vendor_id != PCI_ANY_ID) + continue; + if (id_table->subsystem_device_id != + dev->id.subsystem_device_id && + id_table->subsystem_device_id != PCI_ANY_ID) + continue; + + struct rte_pci_addr *loc = &dev->addr; + + RTE_LOG(DEBUG, EAL, + "PCI device "PCI_PRI_FMT" on NUMA socket %i\n", + loc->domain, loc->bus, loc->devid, + loc->function, dev->numa_node); + + RTE_LOG(DEBUG, EAL, " remove driver: %x:%x %s\n", + dev->id.vendor_id, dev->id.device_id, + dr->name); + + /* call the driver devuninit() function */ + if (dr->devuninit && (dr->devuninit(dev) < 0)) + return -1; /* negative value is an error */ + + /* clear driver structure */ + dev->driver = NULL; + + if (dr->drv_flags & RTE_PCI_DRV_NEED_MAPPING) + /* unmap resources for devices that use igb_uio */ + pci_unmap_device(dev); + + return 0; + } + /* return positive value if driver is not found */ + return 1; +} +#else /* RTE_LIBRTE_EAL_HOTPLUG */ +int +rte_eal_pci_close_one_driver(struct rte_pci_driver *dr __rte_unused, + struct rte_pci_device *dev __rte_unused) +{ + RTE_LOG(ERR, EAL, "Hotplug support isn't enabled\n"); + return -1; +} +#endif /* RTE_LIBRTE_EAL_HOTPLUG */ + /* Init the PCI EAL subsystem */ int rte_eal_pci_init(void) -- 1.9.1
This new parameter is needed to keep device type like PCI or virtual. Port detaching processes are different between PCI device and virtual device. RTE_ETH_DEV_PCI indicates device type is PCI. RTE_ETH_DEV_VIRTUAL indicates device is virtual. v10: - Change order of version.map. (Thanks to Thomas Monjalon) - Fix comment of "rte_ethdev.h". (Thanks to Thomas Monjalon) v9: - Fix commit log. - RTE_ETH_DEV_PHYSICAL is replaced by RTE_ETH_DEV_PCI. (Thanks to Thomas Monjalon) v8: - NONE_TRACE is replaced by NO_TRACE. - Add missing symbol in version map. (Thanks to Qiu, Michael and Iremonger, Bernard) v4: - Fix comments of rte_eth_dev_type. Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp> --- app/test/virtual_pmd.c | 2 +- lib/librte_ether/rte_ethdev.c | 25 +++++++++++++++++++++++-- lib/librte_ether/rte_ethdev.h | 25 ++++++++++++++++++++++++- lib/librte_ether/rte_ether_version.map | 1 + lib/librte_pmd_af_packet/rte_eth_af_packet.c | 2 +- lib/librte_pmd_bond/rte_eth_bond_api.c | 2 +- lib/librte_pmd_pcap/rte_eth_pcap.c | 2 +- lib/librte_pmd_ring/rte_eth_ring.c | 2 +- lib/librte_pmd_xenvirt/rte_eth_xenvirt.c | 2 +- 9 files changed, 54 insertions(+), 9 deletions(-) diff --git a/app/test/virtual_pmd.c b/app/test/virtual_pmd.c index 9fac95d..c02644a 100644 --- a/app/test/virtual_pmd.c +++ b/app/test/virtual_pmd.c @@ -556,7 +556,7 @@ virtual_ethdev_create(const char *name, struct ether_addr *mac_addr, goto err; /* reserve an ethdev entry */ - eth_dev = rte_eth_dev_allocate(name); + eth_dev = rte_eth_dev_allocate(name, RTE_ETH_DEV_PCI); if (eth_dev == NULL) goto err; diff --git a/lib/librte_ether/rte_ethdev.c b/lib/librte_ether/rte_ethdev.c index 5f166d6..69776b3 100644 --- a/lib/librte_ether/rte_ethdev.c +++ b/lib/librte_ether/rte_ethdev.c @@ -227,7 +227,7 @@ rte_eth_dev_find_free_port(void) } struct rte_eth_dev * -rte_eth_dev_allocate(const char *name) +rte_eth_dev_allocate(const char *name, enum rte_eth_dev_type type) { uint8_t port_id; struct rte_eth_dev *eth_dev; @@ -251,6 +251,7 @@ rte_eth_dev_allocate(const char *name) snprintf(eth_dev->data->name, sizeof(eth_dev->data->name), "%s", name); eth_dev->data->port_id = port_id; eth_dev->attached = DEV_ATTACHED; + eth_dev->dev_type = type; nb_ports++; return eth_dev; } @@ -262,6 +263,7 @@ rte_eth_dev_release_port(struct rte_eth_dev *eth_dev) return -EINVAL; eth_dev->attached = 0; + eth_dev->dev_type = RTE_ETH_DEV_UNKNOWN; nb_ports--; return 0; } @@ -300,7 +302,7 @@ rte_eth_dev_init(struct rte_pci_driver *pci_drv, rte_eth_dev_create_unique_device_name(ethdev_name, sizeof(ethdev_name), pci_dev); - eth_dev = rte_eth_dev_allocate(ethdev_name); + eth_dev = rte_eth_dev_allocate(ethdev_name, RTE_ETH_DEV_PCI); if (eth_dev == NULL) return -ENOMEM; @@ -426,6 +428,14 @@ rte_eth_dev_count(void) return (nb_ports); } +enum rte_eth_dev_type +rte_eth_dev_get_device_type(uint8_t port_id) +{ + if (!rte_eth_dev_is_valid_port(port_id)) + return -1; + return rte_eth_devices[port_id].dev_type; +} + int rte_eth_dev_save(struct rte_eth_dev *devs, size_t size) { @@ -523,6 +533,17 @@ rte_eth_dev_is_detachable(uint8_t port_id) return -EINVAL; } + if (rte_eth_devices[port_id].dev_type == RTE_ETH_DEV_PCI) { + switch (rte_eth_devices[port_id].pci_dev->pt_driver) { + case RTE_PT_IGB_UIO: + case RTE_PT_UIO_GENERIC: + break; + case RTE_PT_VFIO: + default: + return -ENOTSUP; + } + } + drv_flags = rte_eth_devices[port_id].driver->pci_drv.drv_flags; return !(drv_flags & RTE_PCI_DRV_DETACHABLE); } diff --git a/lib/librte_ether/rte_ethdev.h b/lib/librte_ether/rte_ethdev.h index 65a3484..8f31984 100644 --- a/lib/librte_ether/rte_ethdev.h +++ b/lib/librte_ether/rte_ethdev.h @@ -1523,6 +1523,17 @@ struct eth_dev_ops { }; /** + * The eth device type + */ +enum rte_eth_dev_type { + RTE_ETH_DEV_UNKNOWN, /**< unknown device type */ + RTE_ETH_DEV_PCI, + /**< Physical function and Virtual function of PCI devices */ + RTE_ETH_DEV_VIRTUAL, /**< non hardware device */ + RTE_ETH_DEV_MAX /**< max value of this enum */ +}; + +/** * @internal * The generic data structure associated with each ethernet device. * @@ -1541,6 +1552,7 @@ struct rte_eth_dev { struct rte_pci_device *pci_dev; /**< PCI info. supplied by probing */ struct rte_eth_dev_cb_list callbacks; /**< User application callbacks */ uint8_t attached; /**< Flag indicating the port is attached */ + enum rte_eth_dev_type dev_type; /**< Flag indicating the device type */ }; struct rte_eth_dev_sriov { @@ -1618,6 +1630,15 @@ extern uint8_t rte_eth_dev_count(void); /** * Function for internal use by port hotplug functions. + * Get the device type to know whether the device is physical or virtual. + * @param port_id The port identifier of the Ethernet device + * @return + * - Device type. + */ +extern enum rte_eth_dev_type rte_eth_dev_get_device_type(uint8_t port_id); + +/** + * Function for internal use by port hotplug functions. * Copies current ethdev structures to the specified pointer. * * @param devs The pointer to the ethdev structures @@ -1706,10 +1727,12 @@ extern struct rte_eth_dev *rte_eth_dev_allocated(const char *name); * to that slot for the driver to use. * * @param name Unique identifier name for each Ethernet device + * @param type Device type of this Ethernet device * @return * - Slot in the rte_dev_devices array for a new device; */ -struct rte_eth_dev *rte_eth_dev_allocate(const char *name); +struct rte_eth_dev *rte_eth_dev_allocate(const char *name, + enum rte_eth_dev_type type); /** * Function for internal use by dummy drivers primarily, e.g. ring-based diff --git a/lib/librte_ether/rte_ether_version.map b/lib/librte_ether/rte_ether_version.map index 666c3af..8fa523b 100644 --- a/lib/librte_ether/rte_ether_version.map +++ b/lib/librte_ether/rte_ether_version.map @@ -42,6 +42,7 @@ DPDK_2.0 { rte_eth_dev_get_5tuple_filter; rte_eth_dev_get_addr_by_port; rte_eth_dev_get_changed_port; + rte_eth_dev_get_device_type; rte_eth_dev_get_ethertype_filter; rte_eth_dev_get_flex_filter; rte_eth_dev_get_mtu; diff --git a/lib/librte_pmd_af_packet/rte_eth_af_packet.c b/lib/librte_pmd_af_packet/rte_eth_af_packet.c index 1ffe1cd..80e9bdf 100644 --- a/lib/librte_pmd_af_packet/rte_eth_af_packet.c +++ b/lib/librte_pmd_af_packet/rte_eth_af_packet.c @@ -649,7 +649,7 @@ rte_pmd_init_internals(const char *name, } /* reserve an ethdev entry */ - *eth_dev = rte_eth_dev_allocate(name); + *eth_dev = rte_eth_dev_allocate(name, RTE_ETH_DEV_VIRTUAL); if (*eth_dev == NULL) goto error; diff --git a/lib/librte_pmd_bond/rte_eth_bond_api.c b/lib/librte_pmd_bond/rte_eth_bond_api.c index 4ab3267..7a6a5f7 100644 --- a/lib/librte_pmd_bond/rte_eth_bond_api.c +++ b/lib/librte_pmd_bond/rte_eth_bond_api.c @@ -235,7 +235,7 @@ rte_eth_bond_create(const char *name, uint8_t mode, uint8_t socket_id) } /* reserve an ethdev entry */ - eth_dev = rte_eth_dev_allocate(name); + eth_dev = rte_eth_dev_allocate(name, RTE_ETH_DEV_VIRTUAL); if (eth_dev == NULL) { RTE_BOND_LOG(ERR, "Unable to allocate rte_eth_dev"); goto err; diff --git a/lib/librte_pmd_pcap/rte_eth_pcap.c b/lib/librte_pmd_pcap/rte_eth_pcap.c index d299288..af7fae8 100644 --- a/lib/librte_pmd_pcap/rte_eth_pcap.c +++ b/lib/librte_pmd_pcap/rte_eth_pcap.c @@ -709,7 +709,7 @@ rte_pmd_init_internals(const char *name, const unsigned nb_rx_queues, goto error; /* reserve an ethdev entry */ - *eth_dev = rte_eth_dev_allocate(name); + *eth_dev = rte_eth_dev_allocate(name, RTE_ETH_DEV_VIRTUAL); if (*eth_dev == NULL) goto error; diff --git a/lib/librte_pmd_ring/rte_eth_ring.c b/lib/librte_pmd_ring/rte_eth_ring.c index a23e933..cd0913b 100644 --- a/lib/librte_pmd_ring/rte_eth_ring.c +++ b/lib/librte_pmd_ring/rte_eth_ring.c @@ -297,7 +297,7 @@ rte_eth_from_rings(const char *name, struct rte_ring *const rx_queues[], goto error; /* reserve an ethdev entry */ - eth_dev = rte_eth_dev_allocate(name); + eth_dev = rte_eth_dev_allocate(name, RTE_ETH_DEV_VIRTUAL); if (eth_dev == NULL) goto error; diff --git a/lib/librte_pmd_xenvirt/rte_eth_xenvirt.c b/lib/librte_pmd_xenvirt/rte_eth_xenvirt.c index 04e30c9..bc403d6 100644 --- a/lib/librte_pmd_xenvirt/rte_eth_xenvirt.c +++ b/lib/librte_pmd_xenvirt/rte_eth_xenvirt.c @@ -648,7 +648,7 @@ eth_dev_xenvirt_create(const char *name, const char *params, goto err; /* reserve an ethdev entry */ - eth_dev = rte_eth_dev_allocate(name); + eth_dev = rte_eth_dev_allocate(name, RTE_ETH_DEV_VIRTUAL); if (eth_dev == NULL) goto err; -- 1.9.1
These functions are used for attaching or detaching a port. When rte_eal_dev_attach() is called, the function tries to realize the device name as pci address. If this is done successfully, rte_eal_dev_attach() will attach physical device port. If not, attaches virtual devive port. When rte_eal_dev_detach() is called, the function gets the device type of this port to know whether the port is come from physical or virtual. And then specific detaching function will be called. v10: - Add comments. - Change order of version.map. (Thanks to Thomas Monjalon) v9: - Fix comments. - Use strcmp() instead of strncmp(). - Remove RTE_EAL_INVOKE_TYPE_PROBE/CLOSE. - Change definition of rte_dev_uninit_t. (Thanks to Thomas Monjalon and Maxime Leroy) v8: - Add missing symbol in version map. (Thanks to Qiu, Michael and Iremonger, Bernard) v7: - Fix typo of warning messages. (Thanks to Qiu, Michael) v5: - Change function names like below. rte_eal_dev_find_and_invoke() to rte_eal_vdev_find_and_invoke(). rte_eal_dev_invoke() to rte_eal_vdev_invoke(). - Add code to handle a return value of rte_eal_devargs_remove(). - Fix pci address format in rte_eal_dev_detach(). v4: - Fix comment. - Add error checking. - Fix indent of 'if' statement. - Change function name. Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp> --- lib/librte_eal/common/eal_common_dev.c | 315 ++++++++++++++++++++++++ lib/librte_eal/common/eal_private.h | 11 + lib/librte_eal/common/include/rte_dev.h | 33 +++ lib/librte_eal/linuxapp/eal/Makefile | 1 + lib/librte_eal/linuxapp/eal/eal_pci.c | 6 +- lib/librte_eal/linuxapp/eal/rte_eal_version.map | 2 + 6 files changed, 365 insertions(+), 3 deletions(-) diff --git a/lib/librte_eal/common/eal_common_dev.c b/lib/librte_eal/common/eal_common_dev.c index eae5656..0a623f6 100644 --- a/lib/librte_eal/common/eal_common_dev.c +++ b/lib/librte_eal/common/eal_common_dev.c @@ -32,10 +32,13 @@ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ +#include <stdio.h> +#include <limits.h> #include <string.h> #include <inttypes.h> #include <sys/queue.h> +#include <rte_ethdev.h> #include <rte_dev.h> #include <rte_devargs.h> #include <rte_debug.h> @@ -107,3 +110,315 @@ rte_eal_dev_init(void) } return 0; } + +/* So far, DPDK hotplug function only supports linux */ +#ifdef RTE_LIBRTE_EAL_HOTPLUG +static int +rte_eal_vdev_find_and_init(const char *name) +{ + struct rte_devargs *devargs; + struct rte_driver *driver; + + if (name == NULL) + return -EINVAL; + + /* find the specified device and call init function */ + TAILQ_FOREACH(devargs, &devargs_list, next) { + + if (devargs->type != RTE_DEVTYPE_VIRTUAL) + continue; + + if (strcmp(name, devargs->virtual.drv_name)) + continue; + + TAILQ_FOREACH(driver, &dev_driver_list, next) { + if (driver->type != PMD_VDEV) + continue; + + /* + * search a driver prefix in virtual device name. + * For example, if the driver is pcap PMD, driver->name + * will be "eth_pcap", but devargs->virtual.drv_name + * will be "eth_pcapN". So use strncmp to compare. + */ + if (!strncmp(driver->name, devargs->virtual.drv_name, + strlen(driver->name))) { + driver->init(devargs->virtual.drv_name, + devargs->args); + break; + } + } + + if (driver == NULL) { + RTE_LOG(WARNING, EAL, "no driver found for %s\n", + devargs->virtual.drv_name); + } + return 0; + } + return 1; +} + +static int +rte_eal_vdev_find_and_uninit(const char *name) +{ + struct rte_devargs *devargs; + struct rte_driver *driver; + + if (name == NULL) + return -EINVAL; + + /* find the specified device and call uninit function */ + TAILQ_FOREACH(devargs, &devargs_list, next) { + + if (devargs->type != RTE_DEVTYPE_VIRTUAL) + continue; + + if (strcmp(name, devargs->virtual.drv_name)) + continue; + + TAILQ_FOREACH(driver, &dev_driver_list, next) { + if (driver->type != PMD_VDEV) + continue; + + /* + * search a driver prefix in virtual device name. + * For example, if the driver is pcap PMD, driver->name + * will be "eth_pcap", but devargs->virtual.drv_name + * will be "eth_pcapN". So use strncmp to compare. + */ + if (!strncmp(driver->name, devargs->virtual.drv_name, + strlen(driver->name))) { + driver->uninit(devargs->virtual.drv_name); + break; + } + } + + if (driver == NULL) { + RTE_LOG(WARNING, EAL, "no driver found for %s\n", + devargs->virtual.drv_name); + } + return 0; + } + return 1; +} + +/* attach the new physical device, then store port_id of the device */ +static int +rte_eal_dev_attach_pdev(struct rte_pci_addr *addr, uint8_t *port_id) +{ + uint8_t new_port_id; + struct rte_eth_dev devs[RTE_MAX_ETHPORTS]; + + if ((addr == NULL) || (port_id == NULL)) + goto err; + + /* save current port status */ + if (rte_eth_dev_save(devs, sizeof(devs))) + goto err; + /* re-construct pci_device_list */ + if (rte_eal_pci_scan()) + goto err; + /* invoke probe func of the driver can handle the new device. + * TODO: + * rte_eal_pci_probe_one() should return port_id. + * And rte_eth_dev_save() and rte_eth_dev_get_changed_port() + * should be removed. */ + if (rte_eal_pci_probe_one(addr)) + goto err; + /* get port_id enabled by above procedures */ + if (rte_eth_dev_get_changed_port(devs, &new_port_id)) + goto err; + + *port_id = new_port_id; + return 0; +err: + RTE_LOG(ERR, EAL, "Driver, cannot attach the device\n"); + return -1; +} + +/* detach the new physical device, then store pci_addr of the device */ +static int +rte_eal_dev_detach_pdev(uint8_t port_id, struct rte_pci_addr *addr) +{ + struct rte_pci_addr freed_addr; + struct rte_pci_addr vp; + + if (addr == NULL) + goto err; + + /* check whether the driver supports detach feature, or not */ + if (rte_eth_dev_is_detachable(port_id)) + goto err; + + /* get pci address by port id */ + if (rte_eth_dev_get_addr_by_port(port_id, &freed_addr)) + goto err; + + /* Zerod pci addr means the port comes from virtual device */ + vp.domain = vp.bus = vp.devid = vp.function = 0; + if (rte_eal_compare_pci_addr(&vp, &freed_addr) == 0) + goto err; + + /* invoke close func of the driver, + * also remove the device from pci_device_list */ + if (rte_eal_pci_close_one(&freed_addr)) + goto err; + + *addr = freed_addr; + return 0; +err: + RTE_LOG(ERR, EAL, "Driver, cannot detach the device\n"); + return -1; +} + +static void +get_vdev_name(char *vdevargs) +{ + char *sep; + + if (vdevargs == NULL) + return; + + /* set the first ',' to '\0' to split name and arguments */ + sep = strchr(vdevargs, ','); + if (sep != NULL) + sep[0] = '\0'; +} + +/* attach the new virtual device, then store port_id of the device */ +static int +rte_eal_dev_attach_vdev(const char *vdevargs, uint8_t *port_id) +{ + char *args; + uint8_t new_port_id; + struct rte_eth_dev devs[RTE_MAX_ETHPORTS]; + + if ((vdevargs == NULL) || (port_id == NULL)) + goto err0; + + args = strdup(vdevargs); + if (args == NULL) + goto err0; + + /* save current port status */ + if (rte_eth_dev_save(devs, sizeof(devs))) + goto err1; + /* add the vdevargs to devargs_list */ + if (rte_eal_devargs_add(RTE_DEVTYPE_VIRTUAL, args)) + goto err1; + /* parse vdevargs, then retrieve device name */ + get_vdev_name(args); + /* walk around dev_driver_list to find the driver of the device, + * then invoke probe function o the driver. + * TODO: + * rte_eal_vdev_find_and_init() should return port_id, + * And rte_eth_dev_save() and rte_eth_dev_get_changed_port() + * should be removed. */ + if (rte_eal_vdev_find_and_init(args)) + goto err2; + /* get port_id enabled by above procedures */ + if (rte_eth_dev_get_changed_port(devs, &new_port_id)) + goto err2; + + free(args); + *port_id = new_port_id; + return 0; +err2: + rte_eal_devargs_remove(RTE_DEVTYPE_VIRTUAL, args); +err1: + free(args); +err0: + RTE_LOG(ERR, EAL, "Driver, cannot attach the device\n"); + return -1; +} + +/* detach the new virtual device, then store the name of the device */ +static int +rte_eal_dev_detach_vdev(uint8_t port_id, char *vdevname) +{ + char name[RTE_ETH_NAME_MAX_LEN]; + + if (vdevname == NULL) + goto err; + + /* check whether the driver supports detach feature, or not */ + if (rte_eth_dev_is_detachable(port_id)) + goto err; + + /* get device name by port id */ + if (rte_eth_dev_get_name_by_port(port_id, name)) + goto err; + /* walk around dev_driver_list to find the driver of the device, + * then invoke close function o the driver */ + if (rte_eal_vdev_find_and_uninit(name)) + goto err; + /* remove the vdevname from devargs_list */ + if (rte_eal_devargs_remove(RTE_DEVTYPE_VIRTUAL, name)) + goto err; + + strncpy(vdevname, name, sizeof(name)); + return 0; +err: + RTE_LOG(ERR, EAL, "Driver, cannot detach the device\n"); + return -1; +} + +/* attach the new device, then store port_id of the device */ +int +rte_eal_dev_attach(const char *devargs, uint8_t *port_id) +{ + struct rte_pci_addr addr; + + if ((devargs == NULL) || (port_id == NULL)) + return -EINVAL; + + if (eal_parse_pci_DomBDF(devargs, &addr) == 0) + return rte_eal_dev_attach_pdev(&addr, port_id); + else + return rte_eal_dev_attach_vdev(devargs, port_id); +} + +/* detach the device, then store the name of the device */ +int +rte_eal_dev_detach(uint8_t port_id, char *name) +{ + struct rte_pci_addr addr; + int ret; + + if (name == NULL) + return -EINVAL; + + if (rte_eth_dev_get_device_type(port_id) == RTE_ETH_DEV_PCI) { + ret = rte_eth_dev_get_addr_by_port(port_id, &addr); + if (ret < 0) + return ret; + + ret = rte_eal_dev_detach_pdev(port_id, &addr); + if (ret == 0) + snprintf(name, RTE_ETH_NAME_MAX_LEN, + "%04x:%02x:%02x.%d", + addr.domain, addr.bus, + addr.devid, addr.function); + + return ret; + } else + return rte_eal_dev_detach_vdev(port_id, name); +} +#else /* RTE_LIBRTE_EAL_HOTPLUG */ +int +rte_eal_dev_attach(const char *devargs __rte_unused, + uint8_t *port_id __rte_unused) +{ + RTE_LOG(ERR, EAL, "Hotplug support isn't enabled\n"); + return -1; +} + +/* detach the device, then store the name of the device */ +int +rte_eal_dev_detach(uint8_t port_id __rte_unused, + char *name __rte_unused) +{ + RTE_LOG(ERR, EAL, "Hotplug support isn't enabled\n"); + return -1; +} +#endif /* RTE_LIBRTE_EAL_HOTPLUG */ diff --git a/lib/librte_eal/common/eal_private.h b/lib/librte_eal/common/eal_private.h index 4acf5a0..98b286a 100644 --- a/lib/librte_eal/common/eal_private.h +++ b/lib/librte_eal/common/eal_private.h @@ -154,6 +154,17 @@ struct rte_pci_driver; struct rte_pci_device; /** + * Scan the content of the PCI bus, and the devices in the devices + * list + * + * This function is private to EAL. + * + * @return + * 0 on success, negative on error + */ +int rte_eal_pci_scan(void); + +/** * Mmap memory for single PCI device * * This function is private to EAL. diff --git a/lib/librte_eal/common/include/rte_dev.h b/lib/librte_eal/common/include/rte_dev.h index f7e3a10..a5ac770 100644 --- a/lib/librte_eal/common/include/rte_dev.h +++ b/lib/librte_eal/common/include/rte_dev.h @@ -47,6 +47,7 @@ extern "C" { #endif #include <sys/queue.h> +#include <rte_pci.h> /** Double linked list of device drivers. */ TAILQ_HEAD(rte_driver_list, rte_driver); @@ -57,6 +58,11 @@ TAILQ_HEAD(rte_driver_list, rte_driver); typedef int (rte_dev_init_t)(const char *name, const char *args); /** + * Uninitilization function called for each device driver once. + */ +typedef int (rte_dev_uninit_t)(const char *name); + +/** * Driver type enumeration */ enum pmd_type { @@ -72,6 +78,7 @@ struct rte_driver { enum pmd_type type; /**< PMD Driver type */ const char *name; /**< Driver name. */ rte_dev_init_t *init; /**< Device init. function. */ + rte_dev_uninit_t *uninit; /**< Device uninit. function. */ }; /** @@ -93,6 +100,32 @@ void rte_eal_driver_register(struct rte_driver *driver); void rte_eal_driver_unregister(struct rte_driver *driver); /** + * Attach a new device. + * + * @param devargs + * A pointer to a strings array describing the new device + * to be attached. The strings should be a pci address like + * '0000:01:00.0' or virtual device name like 'eth_pcap0'. + * @param port_id + * A pointer to a port identifier actually attached. + * @return + * 0 on success and port_id is filled, negative on error + */ +int rte_eal_dev_attach(const char *devargs, uint8_t *port_id); + +/** + * Detach a device. + * + * @param port_id + * The port identifier of the device to detach. + * @param addr + * A pointer to a device name actually detached. + * @return + * 0 on success and devname is filled, negative on error + */ +int rte_eal_dev_detach(uint8_t port_id, char *devname); + +/** * Initalize all the registered drivers in this process */ int rte_eal_dev_init(void); diff --git a/lib/librte_eal/linuxapp/eal/Makefile b/lib/librte_eal/linuxapp/eal/Makefile index e117cec..b59b201 100644 --- a/lib/librte_eal/linuxapp/eal/Makefile +++ b/lib/librte_eal/linuxapp/eal/Makefile @@ -45,6 +45,7 @@ CFLAGS += -I$(RTE_SDK)/lib/librte_eal/common/include CFLAGS += -I$(RTE_SDK)/lib/librte_ring CFLAGS += -I$(RTE_SDK)/lib/librte_mempool CFLAGS += -I$(RTE_SDK)/lib/librte_malloc +CFLAGS += -I$(RTE_SDK)/lib/librte_mbuf CFLAGS += -I$(RTE_SDK)/lib/librte_ether CFLAGS += -I$(RTE_SDK)/lib/librte_ivshmem CFLAGS += -I$(RTE_SDK)/lib/librte_pmd_ring diff --git a/lib/librte_eal/linuxapp/eal/eal_pci.c b/lib/librte_eal/linuxapp/eal/eal_pci.c index 4bdf51b..21d3e85 100644 --- a/lib/librte_eal/linuxapp/eal/eal_pci.c +++ b/lib/librte_eal/linuxapp/eal/eal_pci.c @@ -441,8 +441,8 @@ error: * Scan the content of the PCI bus, and the devices in the devices * list */ -static int -pci_scan(void) +int +rte_eal_pci_scan(void) { struct dirent *e; DIR *dir; @@ -774,7 +774,7 @@ rte_eal_pci_init(void) if (internal_config.no_pci) return 0; - if (pci_scan() < 0) { + if (rte_eal_pci_scan() < 0) { RTE_LOG(ERR, EAL, "%s(): Cannot scan PCI bus\n", __func__); return -1; } diff --git a/lib/librte_eal/linuxapp/eal/rte_eal_version.map b/lib/librte_eal/linuxapp/eal/rte_eal_version.map index d36286e..bc6b82d 100644 --- a/lib/librte_eal/linuxapp/eal/rte_eal_version.map +++ b/lib/librte_eal/linuxapp/eal/rte_eal_version.map @@ -20,6 +20,8 @@ DPDK_2.0 { rte_dump_tailq; rte_eal_alarm_cancel; rte_eal_alarm_set; + rte_eal_dev_attach; + rte_eal_dev_detach; rte_eal_dev_init; rte_eal_devargs_add; rte_eal_devargs_dump; -- 1.9.1
This patch adds a new section for describing port hotplug framework. Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp> --- doc/guides/prog_guide/index.rst | 1 + doc/guides/prog_guide/port_hotplug_framework.rst | 110 +++++++++++++++++++++++ 2 files changed, 111 insertions(+) create mode 100644 doc/guides/prog_guide/port_hotplug_framework.rst diff --git a/doc/guides/prog_guide/index.rst b/doc/guides/prog_guide/index.rst index de69682..60a6ac5 100644 --- a/doc/guides/prog_guide/index.rst +++ b/doc/guides/prog_guide/index.rst @@ -71,6 +71,7 @@ Programmer's Guide packet_classif_access_ctrl packet_framework vhost_lib + port_hotplug_framework source_org dev_kit_build_system dev_kit_root_make_help diff --git a/doc/guides/prog_guide/port_hotplug_framework.rst b/doc/guides/prog_guide/port_hotplug_framework.rst new file mode 100644 index 0000000..355ae28 --- /dev/null +++ b/doc/guides/prog_guide/port_hotplug_framework.rst @@ -0,0 +1,110 @@ +.. BSD LICENSE + Copyright(c) 2015 IGEL Co.,Ltd. All rights reserved. + All rights reserved. + + Redistribution and use in source and binary forms, with or without + modification, are permitted provided that the following conditions + are met: + + * Redistributions of source code must retain the above copyright + notice, this list of conditions and the following disclaimer. + * Redistributions in binary form must reproduce the above copyright + notice, this list of conditions and the following disclaimer in + the documentation and/or other materials provided with the + distribution. + * Neither the name of IGEL Co.,Ltd. nor the names of its + contributors may be used to endorse or promote products derived + from this software without specific prior written permission. + + THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS + "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT + LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR + A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT + OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, + SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT + LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, + DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY + THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT + (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE + OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + +Port Hotplug Framework +====================== + +The Port Hotplug Framework provides DPDK applications with the ability to +attach and detach ports at runtime. Because the framework depends on PMD +implementation, the ports that PMDs cannot handle are out of scope of this +framework. Furthermore, after detaching a port from a DPDK application, the +framework doesn't provide a way for removing the devices from the system. +For the ports backed by a physical NIC, the kernel will need to support PCI +Hotplug feature. + +Overview +-------- + +The basic requirements of the Port Hotplug Framework are: + +* DPDK applications that use the Port Hotplug Framework must manage their + own ports. + + The Port Hotplug Framework is implemented to allow DPDK applications to + manage ports. For example, when DPDK applications call the port attach + function, the attached port number is returned. DPDK applications can + also detach the port by port number. + +* Kernel support is needed for attaching or detaching physical device + ports. + + To attach new physical device ports, the device will be recognized by + userspace driver I/O framework in kernel at first. Then DPDK + applications can call the Port Hotplug functions to attach the ports. + For detaching, steps are vice versa. + +* Before detaching, they must be stopped and closed. + + DPDK applications must call "rte_eth_dev_stop()" and + "rte_eth_dev_close()" APIs before detaching ports. These functions will + start finalization sequence of the PMDs. + +* The framework doesn't affect legacy DPDK applications behavior. + + If the Port Hotplug functions aren't called, all legacy DPDK apps can + still work without modifications. + +Port Hotplug API overview +------------------------- + +* Attaching a port + + "rte_eal_dev_attach()" API attaches a port to DPDK application, and + returns the attached port number. Before calling the API, the device + should be recognized by an userspace driver I/O framework. The API + receives a pci address like "0000:01:00.0" or a virtual device name + like "eth_pcap0,iface=eth0". In the case of virtual device name, the + format is the same as the general "--vdev" option of DPDK. + +* Detaching a port + + "rte_eal_dev_detach()" API detaches a port from DPDK application, and + returns a pci address of the detached device or a virtual device name + of the device. + +Reference +--------- + + "testpmd" supports the Port Hotplug Framework. + +Limitations +----------- + +* The Port Hotplug APIs are not thread safe. + +* The framework can only be enabled with Linux. BSD is not supported. + +* To detach a port, the port should be backed by a device that igb_uio + manages. VFIO is not supported. + +* Not all PMDs support detaching feature. + To know whether a PMD can support detaching, search for the + "RTE_PCI_DRV_DETACHABLE" flag in PMD implementation. If the flag is + defined in the PMD, detaching is supported. -- 1.9.1
This patch adds finalization code to free resources allocated by the PMD. v6: - Fix a paramter of rte_eth_dev_free(). v4: - Change function name. Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp> --- lib/librte_pmd_pcap/rte_eth_pcap.c | 40 ++++++++++++++++++++++++++++++++++++++ 1 file changed, 40 insertions(+) diff --git a/lib/librte_pmd_pcap/rte_eth_pcap.c b/lib/librte_pmd_pcap/rte_eth_pcap.c index af7fae8..5e94930 100644 --- a/lib/librte_pmd_pcap/rte_eth_pcap.c +++ b/lib/librte_pmd_pcap/rte_eth_pcap.c @@ -498,6 +498,13 @@ static struct eth_dev_ops ops = { .stats_reset = eth_stats_reset, }; +static struct eth_driver rte_pcap_pmd = { + .pci_drv = { + .name = "rte_pcap_pmd", + .drv_flags = RTE_PCI_DRV_DETACHABLE, + }, +}; + /* * Function handler that opens the pcap file for reading a stores a * reference of it for use it later on. @@ -713,6 +720,10 @@ rte_pmd_init_internals(const char *name, const unsigned nb_rx_queues, if (*eth_dev == NULL) goto error; + /* check length of device name */ + if ((strlen((*eth_dev)->data->name) + 1) > sizeof(data->name)) + goto error; + /* now put it all together * - store queue data in internals, * - store numa_node info in pci_driver @@ -739,10 +750,13 @@ rte_pmd_init_internals(const char *name, const unsigned nb_rx_queues, data->nb_tx_queues = (uint16_t)nb_tx_queues; data->dev_link = pmd_link; data->mac_addrs = ð_addr; + strncpy(data->name, + (*eth_dev)->data->name, strlen((*eth_dev)->data->name)); (*eth_dev)->data = data; (*eth_dev)->dev_ops = &ops; (*eth_dev)->pci_dev = pci_dev; + (*eth_dev)->driver = &rte_pcap_pmd; return 0; @@ -927,10 +941,36 @@ rte_pmd_pcap_devinit(const char *name, const char *params) } +static int +rte_pmd_pcap_devuninit(const char *name) +{ + struct rte_eth_dev *eth_dev = NULL; + + RTE_LOG(INFO, PMD, "Closing pcap ethdev on numa socket %u\n", + rte_socket_id()); + + if (name == NULL) + return -1; + + /* reserve an ethdev entry */ + eth_dev = rte_eth_dev_allocated(name); + if (eth_dev == NULL) + return -1; + + rte_free(eth_dev->data->dev_private); + rte_free(eth_dev->data); + rte_free(eth_dev->pci_dev); + + rte_eth_dev_release_port(eth_dev); + + return 0; +} + static struct rte_driver pmd_pcap_drv = { .name = "eth_pcap", .type = PMD_VDEV, .init = rte_pmd_pcap_devinit, + .uninit = rte_pmd_pcap_devuninit, }; PMD_REGISTER_DRIVER(pmd_pcap_drv); -- 1.9.1
The patch introduces following commands. - port attach [ident] - port detach [port_id] - attach: attaching a port - detach: detaching a port - ident: pci address of physical device. Or device name and parameters of virtual device. (ex. 0000:02:00.0, eth_pcap0,iface=eth0) - port_id: port identifier v7: - Fix doc. (Thanks to Iremonger, Bernard) - Fix port checking implementation of star_port(); (Thanks to Qiu, Michael) v5: - Add testpmd documentation. (Thanks to Iremonger, Bernard) v4: - Fix strings of command help. Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp> --- app/test-pmd/cmdline.c | 137 +++++++++++++++---- app/test-pmd/config.c | 116 +++++++++------- app/test-pmd/parameters.c | 22 ++- app/test-pmd/testpmd.c | 199 +++++++++++++++++++++------- app/test-pmd/testpmd.h | 18 ++- doc/guides/testpmd_app_ug/testpmd_funcs.rst | 57 ++++++++ 6 files changed, 416 insertions(+), 133 deletions(-) diff --git a/app/test-pmd/cmdline.c b/app/test-pmd/cmdline.c index 4753bb4..fa6e3a6 100644 --- a/app/test-pmd/cmdline.c +++ b/app/test-pmd/cmdline.c @@ -579,6 +579,12 @@ static void cmd_help_long_parsed(void *parsed_result, "port close (port_id|all)\n" " Close all ports or port_id.\n\n" + "port attach (ident)\n" + " Attach physical or virtual dev by pci address or virtual device name\n\n" + + "port detach (port_id)\n" + " Detach physical or virtual dev by port_id\n\n" + "port config (port_id|all)" " speed (10|100|1000|10000|40000|auto)" " duplex (half|full|auto)\n" @@ -870,6 +876,89 @@ cmdline_parse_inst_t cmd_operate_specific_port = { }, }; +/* *** attach a specified port *** */ +struct cmd_operate_attach_port_result { + cmdline_fixed_string_t port; + cmdline_fixed_string_t keyword; + cmdline_fixed_string_t identifier; +}; + +static void cmd_operate_attach_port_parsed(void *parsed_result, + __attribute__((unused)) struct cmdline *cl, + __attribute__((unused)) void *data) +{ + struct cmd_operate_attach_port_result *res = parsed_result; + + if (!strcmp(res->keyword, "attach")) + attach_port(res->identifier); + else + printf("Unknown parameter\n"); +} + +cmdline_parse_token_string_t cmd_operate_attach_port_port = + TOKEN_STRING_INITIALIZER(struct cmd_operate_attach_port_result, + port, "port"); +cmdline_parse_token_string_t cmd_operate_attach_port_keyword = + TOKEN_STRING_INITIALIZER(struct cmd_operate_attach_port_result, + keyword, "attach"); +cmdline_parse_token_string_t cmd_operate_attach_port_identifier = + TOKEN_STRING_INITIALIZER(struct cmd_operate_attach_port_result, + identifier, NULL); + +cmdline_parse_inst_t cmd_operate_attach_port = { + .f = cmd_operate_attach_port_parsed, + .data = NULL, + .help_str = "port attach identifier, " + "identifier: pci address or virtual dev name", + .tokens = { + (void *)&cmd_operate_attach_port_port, + (void *)&cmd_operate_attach_port_keyword, + (void *)&cmd_operate_attach_port_identifier, + NULL, + }, +}; + +/* *** detach a specified port *** */ +struct cmd_operate_detach_port_result { + cmdline_fixed_string_t port; + cmdline_fixed_string_t keyword; + uint8_t port_id; +}; + +static void cmd_operate_detach_port_parsed(void *parsed_result, + __attribute__((unused)) struct cmdline *cl, + __attribute__((unused)) void *data) +{ + struct cmd_operate_detach_port_result *res = parsed_result; + + if (!strcmp(res->keyword, "detach")) + detach_port(res->port_id); + else + printf("Unknown parameter\n"); +} + +cmdline_parse_token_string_t cmd_operate_detach_port_port = + TOKEN_STRING_INITIALIZER(struct cmd_operate_detach_port_result, + port, "port"); +cmdline_parse_token_string_t cmd_operate_detach_port_keyword = + TOKEN_STRING_INITIALIZER(struct cmd_operate_detach_port_result, + keyword, "detach"); +cmdline_parse_token_num_t cmd_operate_detach_port_port_id = + TOKEN_NUM_INITIALIZER(struct cmd_operate_detach_port_result, + port_id, UINT8); + +cmdline_parse_inst_t cmd_operate_detach_port = { + .f = cmd_operate_detach_port_parsed, + .data = NULL, + .help_str = "port detach port_id", + .tokens = { + (void *)&cmd_operate_detach_port_port, + (void *)&cmd_operate_detach_port_keyword, + (void *)&cmd_operate_detach_port_port_id, + NULL, + }, +}; + /* *** configure speed for all ports *** */ struct cmd_config_speed_all { cmdline_fixed_string_t port; @@ -924,7 +1013,7 @@ cmd_config_speed_all_parsed(void *parsed_result, return; } - for (pid = 0; pid < nb_ports; pid++) { + FOREACH_PORT(pid, ports) { ports[pid].dev_conf.link_speed = link_speed; ports[pid].dev_conf.link_duplex = link_duplex; } @@ -992,10 +1081,8 @@ cmd_config_speed_specific_parsed(void *parsed_result, return; } - if (res->id >= nb_ports) { - printf("Port id %d must be less than %d\n", res->id, nb_ports); + if (port_id_is_invalid(res->id, ENABLED_WARN)) return; - } if (!strcmp(res->value1, "10")) link_speed = ETH_LINK_SPEED_10; @@ -1555,7 +1642,7 @@ cmd_config_rxtx_queue_parsed(void *parsed_result, return; } - if (port_id_is_invalid(res->portid)) + if (port_id_is_invalid(res->portid, ENABLED_WARN)) return; if (port_is_started(res->portid) != 1) { @@ -2955,7 +3042,7 @@ cmd_csum_parsed(void *parsed_result, int hw = 0; uint16_t mask = 0; - if (port_id_is_invalid(res->port_id)) { + if (port_id_is_invalid(res->port_id, ENABLED_WARN)) { printf("invalid port %d\n", res->port_id); return; } @@ -3047,10 +3134,8 @@ cmd_csum_tunnel_parsed(void *parsed_result, { struct cmd_csum_tunnel_result *res = parsed_result; - if (port_id_is_invalid(res->port_id)) { - printf("invalid port %d\n", res->port_id); + if (port_id_is_invalid(res->port_id, ENABLED_WARN)) return; - } if (!strcmp(res->onoff, "on")) ports[res->port_id].tx_ol_flags |= @@ -3105,7 +3190,7 @@ cmd_tso_set_parsed(void *parsed_result, struct cmd_tso_set_result *res = parsed_result; struct rte_eth_dev_info dev_info; - if (port_id_is_invalid(res->port_id)) + if (port_id_is_invalid(res->port_id, ENABLED_WARN)) return; if (!strcmp(res->mode, "set")) @@ -4081,10 +4166,8 @@ static void cmd_set_bond_mac_addr_parsed(void *parsed_result, struct cmd_set_bond_mac_addr_result *res = parsed_result; int ret; - if (res->port_num >= nb_ports) { - printf("Port id %d must be less than %d\n", res->port_num, nb_ports); + if (port_id_is_invalid(res->port_num, ENABLED_WARN)) return; - } ret = rte_eth_bond_mac_address_set(res->port_num, &res->address); @@ -4321,7 +4404,7 @@ static void cmd_set_promisc_mode_parsed(void *parsed_result, /* all ports */ if (allports) { - for (i = 0; i < nb_ports; i++) { + FOREACH_PORT(i, ports) { if (enable) rte_eth_promiscuous_enable(i); else @@ -4401,7 +4484,7 @@ static void cmd_set_allmulti_mode_parsed(void *parsed_result, /* all ports */ if (allports) { - for (i = 0; i < nb_ports; i++) { + FOREACH_PORT(i, ports) { if (enable) rte_eth_allmulticast_enable(i); else @@ -5586,25 +5669,25 @@ static void cmd_showportall_parsed(void *parsed_result, struct cmd_showportall_result *res = parsed_result; if (!strcmp(res->show, "clear")) { if (!strcmp(res->what, "stats")) - for (i = 0; i < nb_ports; i++) + FOREACH_PORT(i, ports) nic_stats_clear(i); else if (!strcmp(res->what, "xstats")) - for (i = 0; i < nb_ports; i++) + FOREACH_PORT(i, ports) nic_xstats_clear(i); } else if (!strcmp(res->what, "info")) - for (i = 0; i < nb_ports; i++) + FOREACH_PORT(i, ports) port_infos_display(i); else if (!strcmp(res->what, "stats")) - for (i = 0; i < nb_ports; i++) + FOREACH_PORT(i, ports) nic_stats_display(i); else if (!strcmp(res->what, "xstats")) - for (i = 0; i < nb_ports; i++) + FOREACH_PORT(i, ports) nic_xstats_display(i); else if (!strcmp(res->what, "fdir")) - for (i = 0; i < nb_ports; i++) + FOREACH_PORT(i, ports) fdir_get_infos(i); else if (!strcmp(res->what, "stat_qmap")) - for (i = 0; i < nb_ports; i++) + FOREACH_PORT(i, ports) nic_stats_mapping_display(i); } @@ -9172,6 +9255,8 @@ cmdline_parse_ctx_t main_ctx[] = { (cmdline_parse_inst_t *)&cmd_set_qmap, (cmdline_parse_inst_t *)&cmd_operate_port, (cmdline_parse_inst_t *)&cmd_operate_specific_port, + (cmdline_parse_inst_t *)&cmd_operate_attach_port, + (cmdline_parse_inst_t *)&cmd_operate_detach_port, (cmdline_parse_inst_t *)&cmd_config_speed_all, (cmdline_parse_inst_t *)&cmd_config_speed_specific, (cmdline_parse_inst_t *)&cmd_config_rx_tx, @@ -9250,7 +9335,7 @@ prompt(void) static void cmd_reconfig_device_queue(portid_t id, uint8_t dev, uint8_t queue) { - if (id < nb_ports) { + if (!port_id_is_invalid(id, DISABLED_WARN)) { /* check if need_reconfig has been set to 1 */ if (ports[id].need_reconfig == 0) ports[id].need_reconfig = dev; @@ -9260,7 +9345,7 @@ cmd_reconfig_device_queue(portid_t id, uint8_t dev, uint8_t queue) } else { portid_t pid; - for (pid = 0; pid < nb_ports; pid++) { + FOREACH_PORT(pid, ports) { /* check if need_reconfig has been set to 1 */ if (ports[pid].need_reconfig == 0) ports[pid].need_reconfig = dev; @@ -9278,10 +9363,8 @@ bypass_is_supported(portid_t port_id) struct rte_port *port; struct rte_pci_id *pci_id; - if (port_id >= nb_ports) { - printf("\tPort id must be less than %d.\n", nb_ports); + if (port_id_is_invalid(port_id, ENABLED_WARN)) return 0; - } /* Get the device id. */ port = &ports[port_id]; diff --git a/app/test-pmd/config.c b/app/test-pmd/config.c index 6bcd23c..718167c 100644 --- a/app/test-pmd/config.c +++ b/app/test-pmd/config.c @@ -124,11 +124,15 @@ nic_stats_display(portid_t port_id) struct rte_eth_stats stats; struct rte_port *port = &ports[port_id]; uint8_t i; + portid_t pid; static const char *nic_stats_border = "########################"; - if (port_id >= nb_ports) { - printf("Invalid port, range is [0, %d]\n", nb_ports - 1); + if (port_id_is_invalid(port_id, ENABLED_WARN)) { + printf("Valid port range is [0"); + FOREACH_PORT(pid, ports) + printf(", %d", pid); + printf("]\n"); return; } rte_eth_stats_get(port_id, &stats); @@ -201,8 +205,13 @@ nic_stats_display(portid_t port_id) void nic_stats_clear(portid_t port_id) { - if (port_id >= nb_ports) { - printf("Invalid port, range is [0, %d]\n", nb_ports - 1); + portid_t pid; + + if (port_id_is_invalid(port_id, ENABLED_WARN)) { + printf("Valid port range is [0"); + FOREACH_PORT(pid, ports) + printf(", %d", pid); + printf("]\n"); return; } rte_eth_stats_reset(port_id); @@ -249,11 +258,15 @@ nic_stats_mapping_display(portid_t port_id) { struct rte_port *port = &ports[port_id]; uint16_t i; + portid_t pid; static const char *nic_stats_mapping_border = "########################"; - if (port_id >= nb_ports) { - printf("Invalid port, range is [0, %d]\n", nb_ports - 1); + if (port_id_is_invalid(port_id, ENABLED_WARN)) { + printf("Valid port range is [0"); + FOREACH_PORT(pid, ports) + printf(", %d", pid); + printf("]\n"); return; } @@ -302,9 +315,13 @@ port_infos_display(portid_t port_id) int vlan_offload; struct rte_mempool * mp; static const char *info_border = "*********************"; + portid_t pid; - if (port_id >= nb_ports) { - printf("Invalid port, range is [0, %d]\n", nb_ports - 1); + if (port_id_is_invalid(port_id, ENABLED_WARN)) { + printf("Valid port range is [0"); + FOREACH_PORT(pid, ports) + printf(", %d", pid); + printf("]\n"); return; } port = &ports[port_id]; @@ -362,11 +379,14 @@ port_infos_display(portid_t port_id) } int -port_id_is_invalid(portid_t port_id) +port_id_is_invalid(portid_t port_id, enum print_warning warning) { - if (port_id < nb_ports) + if (ports[port_id].enabled) return 0; - printf("Invalid port %d (must be < nb_ports=%d)\n", port_id, nb_ports); + + if (warning == ENABLED_WARN) + printf("Invalid port %d\n", port_id); + return 1; } @@ -425,7 +445,7 @@ port_reg_bit_display(portid_t port_id, uint32_t reg_off, uint8_t bit_x) uint32_t reg_v; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; if (port_reg_off_is_invalid(port_id, reg_off)) return; @@ -444,7 +464,7 @@ port_reg_bit_field_display(portid_t port_id, uint32_t reg_off, uint8_t l_bit; uint8_t h_bit; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; if (port_reg_off_is_invalid(port_id, reg_off)) return; @@ -471,7 +491,7 @@ port_reg_display(portid_t port_id, uint32_t reg_off) { uint32_t reg_v; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; if (port_reg_off_is_invalid(port_id, reg_off)) return; @@ -485,7 +505,7 @@ port_reg_bit_set(portid_t port_id, uint32_t reg_off, uint8_t bit_pos, { uint32_t reg_v; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; if (port_reg_off_is_invalid(port_id, reg_off)) return; @@ -513,7 +533,7 @@ port_reg_bit_field_set(portid_t port_id, uint32_t reg_off, uint8_t l_bit; uint8_t h_bit; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; if (port_reg_off_is_invalid(port_id, reg_off)) return; @@ -547,7 +567,7 @@ port_reg_bit_field_set(portid_t port_id, uint32_t reg_off, void port_reg_set(portid_t port_id, uint32_t reg_off, uint32_t reg_v) { - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; if (port_reg_off_is_invalid(port_id, reg_off)) return; @@ -560,7 +580,7 @@ port_mtu_set(portid_t port_id, uint16_t mtu) { int diag; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; diag = rte_eth_dev_set_mtu(port_id, mtu); if (diag == 0) @@ -723,7 +743,7 @@ rx_ring_desc_display(portid_t port_id, queueid_t rxq_id, uint16_t rxd_id) { const struct rte_memzone *rx_mz; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; if (rx_queue_id_is_invalid(rxq_id)) return; @@ -740,7 +760,7 @@ tx_ring_desc_display(portid_t port_id, queueid_t txq_id, uint16_t txd_id) { const struct rte_memzone *tx_mz; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; if (tx_queue_id_is_invalid(txq_id)) return; @@ -801,7 +821,7 @@ port_rss_reta_info(portid_t port_id, uint16_t i, idx, shift; int ret; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; ret = rte_eth_dev_rss_reta_query(port_id, reta_conf, nb_entries); @@ -833,7 +853,7 @@ port_rss_hash_conf_show(portid_t port_id, int show_rss_key) uint8_t i; int diag; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; /* Get RSS hash key if asked to display it */ rss_conf.rss_key = (show_rss_key) ? rss_key : NULL; @@ -1411,12 +1431,8 @@ set_fwd_ports_list(unsigned int *portlist, unsigned int nb_pt) again: for (i = 0; i < nb_pt; i++) { port_id = (portid_t) portlist[i]; - if (port_id >= nb_ports) { - printf("Invalid port id %u >= %u\n", - (unsigned int) port_id, - (unsigned int) nb_ports); + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; - } if (record_now) fwd_ports_ids[i] = port_id; } @@ -1574,7 +1590,7 @@ vlan_extend_set(portid_t port_id, int on) int diag; int vlan_offload; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; vlan_offload = rte_eth_dev_get_vlan_offload(port_id); @@ -1596,7 +1612,7 @@ rx_vlan_strip_set(portid_t port_id, int on) int diag; int vlan_offload; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; vlan_offload = rte_eth_dev_get_vlan_offload(port_id); @@ -1617,7 +1633,7 @@ rx_vlan_strip_set_on_queue(portid_t port_id, uint16_t queue_id, int on) { int diag; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; diag = rte_eth_dev_set_vlan_strip_on_queue(port_id, queue_id, on); @@ -1632,7 +1648,7 @@ rx_vlan_filter_set(portid_t port_id, int on) int diag; int vlan_offload; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; vlan_offload = rte_eth_dev_get_vlan_offload(port_id); @@ -1653,7 +1669,7 @@ rx_vft_set(portid_t port_id, uint16_t vlan_id, int on) { int diag; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; if (vlan_id_is_invalid(vlan_id)) return; @@ -1670,7 +1686,7 @@ rx_vlan_all_filter_set(portid_t port_id, int on) { uint16_t vlan_id; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; for (vlan_id = 0; vlan_id < 4096; vlan_id++) rx_vft_set(port_id, vlan_id, on); @@ -1680,7 +1696,7 @@ void vlan_tpid_set(portid_t port_id, uint16_t tp_id) { int diag; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; diag = rte_eth_dev_set_vlan_ether_type(port_id, tp_id); @@ -1695,7 +1711,7 @@ vlan_tpid_set(portid_t port_id, uint16_t tp_id) void tx_vlan_set(portid_t port_id, uint16_t vlan_id) { - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; if (vlan_id_is_invalid(vlan_id)) return; @@ -1706,7 +1722,7 @@ tx_vlan_set(portid_t port_id, uint16_t vlan_id) void tx_vlan_reset(portid_t port_id) { - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; ports[port_id].tx_ol_flags &= ~TESTPMD_TX_OFFLOAD_INSERT_VLAN; } @@ -1714,7 +1730,7 @@ tx_vlan_reset(portid_t port_id) void tx_vlan_pvid_set(portid_t port_id, uint16_t vlan_id, int on) { - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; rte_eth_dev_set_vlan_pvid(port_id, vlan_id, on); @@ -1726,7 +1742,7 @@ set_qmap(portid_t port_id, uint8_t is_rx, uint16_t queue_id, uint8_t map_value) uint16_t i; uint8_t existing_mapping_found = 0; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; if (is_rx ? (rx_queue_id_is_invalid(queue_id)) : (tx_queue_id_is_invalid(queue_id))) @@ -1778,7 +1794,7 @@ fdir_add_signature_filter(portid_t port_id, uint8_t queue_id, { int diag; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; diag = rte_eth_dev_fdir_add_signature_filter(port_id, fdir_filter, @@ -1796,7 +1812,7 @@ fdir_update_signature_filter(portid_t port_id, uint8_t queue_id, { int diag; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; diag = rte_eth_dev_fdir_update_signature_filter(port_id, fdir_filter, @@ -1814,7 +1830,7 @@ fdir_remove_signature_filter(portid_t port_id, { int diag; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; diag = rte_eth_dev_fdir_remove_signature_filter(port_id, fdir_filter); @@ -1886,7 +1902,7 @@ fdir_get_infos(portid_t port_id) static const char *fdir_stats_border = "########################"; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; ret = rte_eth_dev_filter_supported(port_id, RTE_ETH_FILTER_FDIR); if (ret < 0) { @@ -1960,7 +1976,7 @@ fdir_add_perfect_filter(portid_t port_id, uint16_t soft_id, uint8_t queue_id, { int diag; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; diag = rte_eth_dev_fdir_add_perfect_filter(port_id, fdir_filter, @@ -1978,7 +1994,7 @@ fdir_update_perfect_filter(portid_t port_id, uint16_t soft_id, uint8_t queue_id, { int diag; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; diag = rte_eth_dev_fdir_update_perfect_filter(port_id, fdir_filter, @@ -1996,7 +2012,7 @@ fdir_remove_perfect_filter(portid_t port_id, uint16_t soft_id, { int diag; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; diag = rte_eth_dev_fdir_remove_perfect_filter(port_id, fdir_filter, @@ -2013,7 +2029,7 @@ fdir_set_masks(portid_t port_id, struct rte_fdir_masks *fdir_masks) { int diag; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; diag = rte_eth_dev_fdir_set_masks(port_id, fdir_masks); @@ -2090,7 +2106,7 @@ set_vf_traffic(portid_t port_id, uint8_t is_rx, uint16_t vf, uint8_t on) { int diag; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; if (is_rx) diag = rte_eth_dev_set_vf_rx(port_id,vf,on); @@ -2112,7 +2128,7 @@ set_vf_rx_vlan(portid_t port_id, uint16_t vlan_id, uint64_t vf_mask, uint8_t on) { int diag; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; if (vlan_id_is_invalid(vlan_id)) return; @@ -2129,7 +2145,7 @@ set_queue_rate_limit(portid_t port_id, uint16_t queue_idx, uint16_t rate) int diag; struct rte_eth_link link; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return 1; rte_eth_link_get_nowait(port_id, &link); if (rate > link.link_speed) { @@ -2154,7 +2170,7 @@ set_vf_rate_limit(portid_t port_id, uint16_t vf, uint16_t rate, uint64_t q_msk) if (q_msk == 0) return 0; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return 1; rte_eth_link_get_nowait(port_id, &link); if (rate > link.link_speed) { diff --git a/app/test-pmd/parameters.c b/app/test-pmd/parameters.c index 19fbf46..07740c4 100644 --- a/app/test-pmd/parameters.c +++ b/app/test-pmd/parameters.c @@ -370,6 +370,7 @@ parse_portnuma_config(const char *q_arg) }; unsigned long int_fld[_NUM_FLD]; char *str_fld[_NUM_FLD]; + portid_t pid; /* reset from value set at definition */ while ((p = strchr(p0,'(')) != NULL) { @@ -391,8 +392,11 @@ parse_portnuma_config(const char *q_arg) return -1; } port_id = (uint8_t)int_fld[FLD_PORT]; - if (port_id >= nb_ports) { - printf("Invalid port, range is [0, %d]\n", nb_ports - 1); + if (port_id_is_invalid(port_id, ENABLED_WARN)) { + printf("Valid port range is [0"); + FOREACH_PORT(pid, ports) + printf(", %d", pid); + printf("]\n"); return -1; } socket_id = (uint8_t)int_fld[FLD_SOCKET]; @@ -423,6 +427,7 @@ parse_ringnuma_config(const char *q_arg) }; unsigned long int_fld[_NUM_FLD]; char *str_fld[_NUM_FLD]; + portid_t pid; #define RX_RING_ONLY 0x1 #define TX_RING_ONLY 0x2 #define RXTX_RING 0x3 @@ -447,8 +452,11 @@ parse_ringnuma_config(const char *q_arg) return -1; } port_id = (uint8_t)int_fld[FLD_PORT]; - if (port_id >= nb_ports) { - printf("Invalid port, range is [0, %d]\n", nb_ports - 1); + if (port_id_is_invalid(port_id, ENABLED_WARN)) { + printf("Valid port range is [0"); + FOREACH_PORT(pid, ports) + printf(", %d", pid); + printf("]\n"); return -1; } socket_id = (uint8_t)int_fld[FLD_SOCKET]; @@ -620,12 +628,12 @@ launch_args_parse(int argc, char** argv) #endif if (!strcmp(lgopts[opt_idx].name, "nb-ports")) { n = atoi(optarg); - if (n > 0 && n <= nb_ports) + if (n > 0 && + !port_id_is_invalid(n, DISABLED_WARN)) nb_fwd_ports = (uint8_t) n; else rte_exit(EXIT_FAILURE, - "nb-ports should be > 0 and <= %d\n", - nb_ports); + "Invalid port %d\n", n); } if (!strcmp(lgopts[opt_idx].name, "nb-cores")) { n = atoi(optarg); diff --git a/app/test-pmd/testpmd.c b/app/test-pmd/testpmd.c index 3aebea6..48e726e 100644 --- a/app/test-pmd/testpmd.c +++ b/app/test-pmd/testpmd.c @@ -71,6 +71,7 @@ #include <rte_pci.h> #include <rte_ether.h> #include <rte_ethdev.h> +#include <rte_dev.h> #include <rte_string_fns.h> #ifdef RTE_LIBRTE_PMD_XENVIRT #include <rte_eth_xenvirt.h> @@ -303,7 +304,7 @@ uint16_t nb_rx_queue_stats_mappings = 0; /* Forward function declarations */ static void map_port_queue_stats_mapping_registers(uint8_t pi, struct rte_port *port); -static void check_all_ports_link_status(uint8_t port_num, uint32_t port_mask); +static void check_all_ports_link_status(uint32_t port_mask); /* * Check if all the ports are started. @@ -312,6 +313,20 @@ static void check_all_ports_link_status(uint8_t port_num, uint32_t port_mask); static int all_ports_started(void); /* + * Find next enabled port + */ +portid_t +find_next_port(portid_t p, struct rte_port *ports, int size) +{ + if (ports == NULL) + rte_exit(-EINVAL, "failed to find a next port id\n"); + + while ((ports[p].enabled == 0) && (p < size)) + p++; + return p; +} + +/* * Setup default configuration. */ static void @@ -540,7 +555,8 @@ init_config(void) + RTE_TEST_TX_DESC_MAX + MAX_PKT_BURST; if (!numa_support) - nb_mbuf_per_pool = (nb_mbuf_per_pool * nb_ports); + nb_mbuf_per_pool = + (nb_mbuf_per_pool * RTE_MAX_ETHPORTS); } if (!numa_support) { @@ -553,14 +569,19 @@ init_config(void) /* Configuration of Ethernet ports. */ ports = rte_zmalloc("testpmd: ports", - sizeof(struct rte_port) * nb_ports, + sizeof(struct rte_port) * RTE_MAX_ETHPORTS, RTE_CACHE_LINE_SIZE); if (ports == NULL) { - rte_exit(EXIT_FAILURE, "rte_zmalloc(%d struct rte_port) " - "failed\n", nb_ports); + rte_exit(EXIT_FAILURE, + "rte_zmalloc(%d struct rte_port) failed\n", + RTE_MAX_ETHPORTS); } - for (pid = 0; pid < nb_ports; pid++) { + /* enabled allocated ports */ + for (pid = 0; pid < nb_ports; pid++) + ports[pid].enabled = 1; + + FOREACH_PORT(pid, ports) { port = &ports[pid]; rte_eth_dev_info_get(pid, &port->dev_info); @@ -590,8 +611,7 @@ init_config(void) nb_mbuf_per_pool = nb_mbuf_per_pool/nb_ports; for (i = 0; i < MAX_SOCKET; i++) { - nb_mbuf = (nb_mbuf_per_pool * - port_per_socket[i]); + nb_mbuf = (nb_mbuf_per_pool * RTE_MAX_ETHPORTS); if (nb_mbuf) mbuf_pool_create(mbuf_data_size, nb_mbuf,i); @@ -623,14 +643,6 @@ reconfig(portid_t new_port_id, unsigned socket_id) struct rte_port *port; /* Reconfiguration of Ethernet ports. */ - ports = rte_realloc(ports, - sizeof(struct rte_port) * nb_ports, - RTE_CACHE_LINE_SIZE); - if (ports == NULL) { - rte_exit(EXIT_FAILURE, "rte_realloc(%d struct rte_port) failed\n", - nb_ports); - } - port = &ports[new_port_id]; rte_eth_dev_info_get(new_port_id, &port->dev_info); @@ -651,7 +663,7 @@ init_fwd_streams(void) streamid_t sm_id, nb_fwd_streams_new; /* set socket id according to numa or not */ - for (pid = 0; pid < nb_ports; pid++) { + FOREACH_PORT(pid, ports) { port = &ports[pid]; if (nb_rxq > port->dev_info.max_rx_queues) { printf("Fail: nb_rxq(%d) is greater than " @@ -1252,7 +1264,7 @@ all_ports_started(void) portid_t pi; struct rte_port *port; - for (pi = 0; pi < nb_ports; pi++) { + FOREACH_PORT(pi, ports) { port = &ports[pi]; /* Check if there is a port which is not started */ if (port->port_status != RTE_PORT_STARTED) @@ -1264,6 +1276,45 @@ all_ports_started(void) } int +all_ports_stopped(void) +{ + portid_t pi; + struct rte_port *port; + + FOREACH_PORT(pi, ports) { + port = &ports[pi]; + if (port->port_status != RTE_PORT_STOPPED) + return 0; + } + + return 1; +} + +int +port_is_started(portid_t port_id) +{ + if (port_id_is_invalid(port_id, ENABLED_WARN)) + return 0; + + if (ports[port_id].port_status != RTE_PORT_STARTED) + return 0; + + return 1; +} + +static int +port_is_closed(portid_t port_id) +{ + if (port_id_is_invalid(port_id, ENABLED_WARN)) + return 0; + + if (ports[port_id].port_status != RTE_PORT_CLOSED) + return 0; + + return 1; +} + +int start_port(portid_t pid) { int diag, need_check_link_status = 0; @@ -1284,8 +1335,8 @@ start_port(portid_t pid) if(dcb_config) dcb_test = 1; - for (pi = 0; pi < nb_ports; pi++) { - if (pid < nb_ports && pid != pi) + FOREACH_PORT(pi, ports) { + if (pid != pi && pid != (portid_t)RTE_PORT_ALL) continue; port = &ports[pi]; @@ -1409,7 +1460,7 @@ start_port(portid_t pid) } if (need_check_link_status && !no_link_check) - check_all_ports_link_status(nb_ports, RTE_PORT_ALL); + check_all_ports_link_status(RTE_PORT_ALL); else printf("Please stop the ports first\n"); @@ -1434,8 +1485,8 @@ stop_port(portid_t pid) } printf("Stopping ports...\n"); - for (pi = 0; pi < nb_ports; pi++) { - if (pid < nb_ports && pid != pi) + FOREACH_PORT(pi, ports) { + if (!port_id_is_invalid(pid, DISABLED_WARN) && pid != pi) continue; port = &ports[pi]; @@ -1451,7 +1502,7 @@ stop_port(portid_t pid) need_check_link_status = 1; } if (need_check_link_status && !no_link_check) - check_all_ports_link_status(nb_ports, RTE_PORT_ALL); + check_all_ports_link_status(RTE_PORT_ALL); printf("Done\n"); } @@ -1469,8 +1520,8 @@ close_port(portid_t pid) printf("Closing ports...\n"); - for (pi = 0; pi < nb_ports; pi++) { - if (pid < nb_ports && pid != pi) + FOREACH_PORT(pi, ports) { + if (!port_id_is_invalid(pid, DISABLED_WARN) && pid != pi) continue; port = &ports[pi]; @@ -1490,31 +1541,83 @@ close_port(portid_t pid) printf("Done\n"); } -int -all_ports_stopped(void) +void +attach_port(char *identifier) { - portid_t pi; - struct rte_port *port; + portid_t i, j, pi = 0; - for (pi = 0; pi < nb_ports; pi++) { - port = &ports[pi]; - if (port->port_status != RTE_PORT_STOPPED) - return 0; + printf("Attaching a new port...\n"); + + if (identifier == NULL) { + printf("Invalid parameters are specified\n"); + return; } - return 1; + if (test_done == 0) { + printf("Please stop forwarding first\n"); + return; + } + + if (rte_eal_dev_attach(identifier, &pi)) + return; + + ports[pi].enabled = 1; + reconfig(pi, rte_eth_dev_socket_id(pi)); + rte_eth_promiscuous_enable(pi); + + nb_ports = rte_eth_dev_count(); + + /* set_default_fwd_ports_config(); */ + bzero(fwd_ports_ids, sizeof(fwd_ports_ids)); + i = 0; + FOREACH_PORT(j, ports) { + fwd_ports_ids[i] = j; + i++; + } + nb_cfg_ports = nb_ports; + nb_fwd_ports++; + + ports[pi].port_status = RTE_PORT_STOPPED; + + printf("Port %d is attached. Now total ports is %d\n", pi, nb_ports); + printf("Done\n"); } -int -port_is_started(portid_t port_id) +void +detach_port(uint8_t port_id) { - if (port_id_is_invalid(port_id)) - return -1; + portid_t i, pi = 0; + char name[RTE_ETH_NAME_MAX_LEN]; - if (ports[port_id].port_status != RTE_PORT_STARTED) - return 0; + printf("Detaching a port...\n"); - return 1; + if (!port_is_closed(port_id)) { + printf("Please close port first\n"); + return; + } + + rte_eth_promiscuous_disable(port_id); + + if (rte_eal_dev_detach(port_id, name)) + return; + + ports[port_id].enabled = 0; + nb_ports = rte_eth_dev_count(); + + /* set_default_fwd_ports_config(); */ + bzero(fwd_ports_ids, sizeof(fwd_ports_ids)); + i = 0; + FOREACH_PORT(pi, ports) { + fwd_ports_ids[i] = pi; + i++; + } + nb_cfg_ports = nb_ports; + nb_fwd_ports--; + + printf("Port '%s' is detached. Now total ports is %d\n", + name, nb_ports); + printf("Done\n"); + return; } void @@ -1522,7 +1625,7 @@ pmd_test_exit(void) { portid_t pt_id; - for (pt_id = 0; pt_id < nb_ports; pt_id++) { + FOREACH_PORT(pt_id, ports) { printf("Stopping port %d...", pt_id); fflush(stdout); rte_eth_dev_close(pt_id); @@ -1541,7 +1644,7 @@ struct pmd_test_command { /* Check the link status of all ports in up to 9s, and print them finally */ static void -check_all_ports_link_status(uint8_t port_num, uint32_t port_mask) +check_all_ports_link_status(uint32_t port_mask) { #define CHECK_INTERVAL 100 /* 100ms */ #define MAX_CHECK_TIME 90 /* 9s (90 * 100ms) in total */ @@ -1552,7 +1655,7 @@ check_all_ports_link_status(uint8_t port_num, uint32_t port_mask) fflush(stdout); for (count = 0; count <= MAX_CHECK_TIME; count++) { all_ports_up = 1; - for (portid = 0; portid < port_num; portid++) { + FOREACH_PORT(portid, ports) { if ((port_mask & (1 << portid)) == 0) continue; memset(&link, 0, sizeof(link)); @@ -1717,7 +1820,7 @@ init_port_config(void) portid_t pid; struct rte_port *port; - for (pid = 0; pid < nb_ports; pid++) { + FOREACH_PORT(pid, ports) { port = &ports[pid]; port->dev_conf.rxmode = rx_mode; port->dev_conf.fdir_conf = fdir_conf; @@ -1896,7 +1999,7 @@ main(int argc, char** argv) nb_ports = (portid_t) rte_eth_dev_count(); if (nb_ports == 0) - rte_exit(EXIT_FAILURE, "No probed ethernet device\n"); + RTE_LOG(WARNING, EAL, "No probed ethernet devices\n"); set_def_fwd_config(); if (nb_lcores == 0) @@ -1918,7 +2021,7 @@ main(int argc, char** argv) rte_exit(EXIT_FAILURE, "Start ports failed\n"); /* set all ports to promiscuous mode by default */ - for (port_id = 0; port_id < nb_ports; port_id++) + FOREACH_PORT(port_id, ports) rte_eth_promiscuous_enable(port_id); #ifdef RTE_LIBRTE_CMDLINE diff --git a/app/test-pmd/testpmd.h b/app/test-pmd/testpmd.h index 581130b..29b6755 100644 --- a/app/test-pmd/testpmd.h +++ b/app/test-pmd/testpmd.h @@ -137,6 +137,7 @@ struct fwd_stream { * The data structure associated with each port. */ struct rte_port { + uint8_t enabled; /**< Port enabled or not */ struct rte_eth_dev_info dev_info; /**< PCI info + driver name */ struct rte_eth_conf dev_conf; /**< Port configuration. */ struct ether_addr eth_addr; /**< Port ethernet address */ @@ -162,6 +163,14 @@ struct rte_port { struct rte_eth_txconf tx_conf; /**< tx configuration */ }; +extern portid_t __rte_unused +find_next_port(portid_t p, struct rte_port *ports, int size); + +#define FOREACH_PORT(p, ports) \ + for (p = find_next_port(0, ports, RTE_MAX_ETHPORTS); \ + p < RTE_MAX_ETHPORTS; \ + p = find_next_port(p + 1, ports, RTE_MAX_ETHPORTS)) + /** * The data structure associated with each forwarding logical core. * The logical cores are internally numbered by a core index from 0 to @@ -522,6 +531,8 @@ int init_port_dcb_config(portid_t pid,struct dcb_config *dcb_conf); int start_port(portid_t pid); void stop_port(portid_t pid); void close_port(portid_t pid); +void attach_port(char *identifier); +void detach_port(uint8_t port_id); int all_ports_stopped(void); int port_is_started(portid_t port_id); void pmd_test_exit(void); @@ -565,10 +576,15 @@ void get_ethertype_filter(uint8_t port_id, uint16_t index); void get_2tuple_filter(uint8_t port_id, uint16_t index); void get_5tuple_filter(uint8_t port_id, uint16_t index); void get_flex_filter(uint8_t port_id, uint16_t index); -int port_id_is_invalid(portid_t port_id); int rx_queue_id_is_invalid(queueid_t rxq_id); int tx_queue_id_is_invalid(queueid_t txq_id); +enum print_warning { + ENABLED_WARN = 0, + DISABLED_WARN +}; +int port_id_is_invalid(portid_t port_id, enum print_warning warning); + /* * Work-around of a compilation error with ICC on invocations of the * rte_be_to_cpu_16() function. diff --git a/doc/guides/testpmd_app_ug/testpmd_funcs.rst b/doc/guides/testpmd_app_ug/testpmd_funcs.rst index 218835a..936f9e0 100644 --- a/doc/guides/testpmd_app_ug/testpmd_funcs.rst +++ b/doc/guides/testpmd_app_ug/testpmd_funcs.rst @@ -808,6 +808,63 @@ The following sections show functions for configuring ports. Port configuration changes only become active when forwarding is started/restarted. +port attach +~~~~~~~~~~~ + +Attach a port specified by pci address or virtual device args. + +To attach a new pci device, the device should be recognized by kernel first. +Then it should be moved under DPDK management. +Finally the port can be attached to testpmd. +On the other hand, to attach a port created by virtual device, above steps are not needed. + +port attach (identifier) + +For example, to attach a port whose pci address is 0000:02:00.0. + +.. code-block:: console + + testpmd> port attach 0000:02:00.0 + Attaching a new port... + ... snip ... + Port 0 is attached. Now total ports is 1 + Done + +For example, to attach a port created by pcap PMD. + +.. code-block:: console + + testpmd> port attach eth_pcap0,iface=eth0 + Attaching a new port... + ... snip ... + Port 0 is attached. Now total ports is 1 + Done + +In this case, identifier is "eth_pcap0,iface=eth0". +This identifier format is the same as "--vdev" format of DPDK applications. + +port detach +~~~~~~~~~~~ + +Detach a specific port. + +Before detaching a port, the port should be closed. +Also to remove a pci device completely from the system, first detach the port from testpmd. +Then the device should be moved under kernel management. +Finally the device can be removed using kernel pci hotplug functionality. +On the other hand, to remove a port created by a virtual device, above steps are not needed. + +port detach (port_id) + +For example, to detach a port 0. + +.. code-block:: console + + testpmd> port detach 0 + Detaching a port... + ... snip ... + Done + port start ~~~~~~~~~~ -- 1.9.1
Hi Tetsuya, On Fri, Feb 20, 2015 at 7:39 AM, Tetsuya Mukawa <mukawa@igel.co.jp> wrote: > These functions are used for attaching or detaching a port. [..] > + > +static void > +get_vdev_name(char *vdevargs) > +{ > + char *sep; > + > + if (vdevargs == NULL) > + return; > + > + /* set the first ',' to '\0' to split name and arguments */ > + sep = strchr(vdevargs, ','); > + if (sep != NULL) > + sep[0] = '\0'; > +} > + > +/* attach the new virtual device, then store port_id of the device */ > +static int > +rte_eal_dev_attach_vdev(const char *vdevargs, uint8_t *port_id) > +{ > + char *args; > + uint8_t new_port_id; > + struct rte_eth_dev devs[RTE_MAX_ETHPORTS]; > + > + if ((vdevargs == NULL) || (port_id == NULL)) > + goto err0; > + > + args = strdup(vdevargs); > + if (args == NULL) > + goto err0; > + > + /* save current port status */ > + if (rte_eth_dev_save(devs, sizeof(devs))) > + goto err1; > + /* add the vdevargs to devargs_list */ > + if (rte_eal_devargs_add(RTE_DEVTYPE_VIRTUAL, args)) > + goto err1; You don't need to add devargs to the devargs_list. The devargs_list is only needed at the init to store the arguments when we parse the command line. Then, at initialization, rte_eal_dev_init creates the devices from this list . Instead of adding the devargs in the list, you could have the following code: static int rte_eal_dev_attach_vdev(const char *vdevargs, uint8_t *port_id) { ... /* save current port status */ if (rte_eth_dev_save(devs, sizeof(devs))) goto err; devargs = rte_eal_parse_devargs_str(RTE_ DEVTYPE_VIRTUAL, vdevargs); if (devargs == NULL) goto err; if (rte_eal_vdev_devinit(devargs) < 0) goto err; if (rte_eth_dev_get_changed_port(devs, &new_port_id)) goto err; ... } What do you think ? > + /* parse vdevargs, then retrieve device name */ > + get_vdev_name(args); > + /* walk around dev_driver_list to find the driver of the device, > + * then invoke probe function o the driver. > + * TODO: > + * rte_eal_vdev_find_and_init() should return port_id, > + * And rte_eth_dev_save() and rte_eth_dev_get_changed_port() > + * should be removed. */ > + if (rte_eal_vdev_find_and_init(args)) > + goto err2; > + /* get port_id enabled by above procedures */ > + if (rte_eth_dev_get_changed_port(devs, &new_port_id)) > + goto err2; > + > + free(args); > + *port_id = new_port_id; > + return 0; > +err2: > + rte_eal_devargs_remove(RTE_DEVTYPE_VIRTUAL, args); > +err1: > + free(args); > +err0: > + RTE_LOG(ERR, EAL, "Driver, cannot attach the device\n"); > + return -1; > +} > + > +/* detach the new virtual device, then store the name of the device */ > +static int > +rte_eal_dev_detach_vdev(uint8_t port_id, char *vdevname) > +{ > + char name[RTE_ETH_NAME_MAX_LEN]; > + > + if (vdevname == NULL) > + goto err; > + > + /* check whether the driver supports detach feature, or not */ > + if (rte_eth_dev_is_detachable(port_id)) > + goto err; > + > + /* get device name by port id */ > + if (rte_eth_dev_get_name_by_port(port_id, name)) > + goto err; > + /* walk around dev_driver_list to find the driver of the device, > + * then invoke close function o the driver */ > + if (rte_eal_vdev_find_and_uninit(name)) > + goto err; > + /* remove the vdevname from devargs_list */ > + if (rte_eal_devargs_remove(RTE_DEVTYPE_VIRTUAL, name)) > + goto err; The same here, you don't need to remove devargs from the devargs_list. Instead of removing the devargs in the list, you could have the following code:: static int rte_eal_dev_detach_vdev(uint8_t port_id, char *vdevname) { ... /* check whether the driver supports detach feature, or not */ if (rte_eth_dev_is_detachable(port_id)) goto err; /* get device name by port id */ if (rte_eth_dev_get_name_by_port(port_id, name)) goto err; if (rte_eal_vdev_uninit(name)) goto err; ... } What do you think ? Regards, Maxime
On 2015/02/20 19:14, Maxime Leroy wrote: > Hi Tetsuya, > > On Fri, Feb 20, 2015 at 7:39 AM, Tetsuya Mukawa <mukawa@igel.co.jp> wrote: >> These functions are used for attaching or detaching a port. > [..] >> + >> +static void >> +get_vdev_name(char *vdevargs) >> +{ >> + char *sep; >> + >> + if (vdevargs == NULL) >> + return; >> + >> + /* set the first ',' to '\0' to split name and arguments */ >> + sep = strchr(vdevargs, ','); >> + if (sep != NULL) >> + sep[0] = '\0'; >> +} >> + >> +/* attach the new virtual device, then store port_id of the device */ >> +static int >> +rte_eal_dev_attach_vdev(const char *vdevargs, uint8_t *port_id) >> +{ >> + char *args; >> + uint8_t new_port_id; >> + struct rte_eth_dev devs[RTE_MAX_ETHPORTS]; >> + >> + if ((vdevargs == NULL) || (port_id == NULL)) >> + goto err0; >> + >> + args = strdup(vdevargs); >> + if (args == NULL) >> + goto err0; >> + >> + /* save current port status */ >> + if (rte_eth_dev_save(devs, sizeof(devs))) >> + goto err1; >> + /* add the vdevargs to devargs_list */ >> + if (rte_eal_devargs_add(RTE_DEVTYPE_VIRTUAL, args)) >> + goto err1; > You don't need to add devargs to the devargs_list. > > The devargs_list is only needed at the init to store the arguments > when we parse the command line. Then, at initialization, > rte_eal_dev_init creates the devices from this list . > > Instead of adding the devargs in the list, you could have the following code: > > static int > rte_eal_dev_attach_vdev(const char *vdevargs, uint8_t *port_id) > { > ... > /* save current port status */ > if (rte_eth_dev_save(devs, sizeof(devs))) > goto err; > > devargs = rte_eal_parse_devargs_str(RTE_ > DEVTYPE_VIRTUAL, vdevargs); > if (devargs == NULL) > goto err; > > if (rte_eal_vdev_devinit(devargs) < 0) > goto err; > > if (rte_eth_dev_get_changed_port(devs, &new_port_id)) > goto err; > > ... > } > > What do you think ? Hi Maxime, I appreciate for your comment. When rte_eal_init() is called, if we have "--vdev" options, these will be stored in vdevargs as you describe above. I agree this is the current behavior of DPDK. When we call hotplug functions, I guess doing same thing will be more consistent design. For example, we can do like below. 1. $ ./testpmd --vdev 'eth_pcap' -- -i 2. testpmd>port detach Also we can do like below. 1. $ ./testpmd -- -i 2. testpmd> port attach eth_pcap 3. testpmd> port detach After doing above cases, we have no port. But in only first case, vdevargs still has a "--vdev" option value. So I guess current hotplug implementation is more consistent design. How about? Regards, Tetsuya >> + /* parse vdevargs, then retrieve device name */ >> + get_vdev_name(args); >> + /* walk around dev_driver_list to find the driver of the device, >> + * then invoke probe function o the driver. >> + * TODO: >> + * rte_eal_vdev_find_and_init() should return port_id, >> + * And rte_eth_dev_save() and rte_eth_dev_get_changed_port() >> + * should be removed. */ >> + if (rte_eal_vdev_find_and_init(args)) >> + goto err2; >> + /* get port_id enabled by above procedures */ >> + if (rte_eth_dev_get_changed_port(devs, &new_port_id)) >> + goto err2; >> + >> + free(args); >> + *port_id = new_port_id; >> + return 0; >> +err2: >> + rte_eal_devargs_remove(RTE_DEVTYPE_VIRTUAL, args); >> +err1: >> + free(args); >> +err0: >> + RTE_LOG(ERR, EAL, "Driver, cannot attach the device\n"); >> + return -1; >> +} >> + >> +/* detach the new virtual device, then store the name of the device */ >> +static int >> +rte_eal_dev_detach_vdev(uint8_t port_id, char *vdevname) >> +{ >> + char name[RTE_ETH_NAME_MAX_LEN]; >> + >> + if (vdevname == NULL) >> + goto err; >> + >> + /* check whether the driver supports detach feature, or not */ >> + if (rte_eth_dev_is_detachable(port_id)) >> + goto err; >> + >> + /* get device name by port id */ >> + if (rte_eth_dev_get_name_by_port(port_id, name)) >> + goto err; >> + /* walk around dev_driver_list to find the driver of the device, >> + * then invoke close function o the driver */ >> + if (rte_eal_vdev_find_and_uninit(name)) >> + goto err; >> + /* remove the vdevname from devargs_list */ >> + if (rte_eal_devargs_remove(RTE_DEVTYPE_VIRTUAL, name)) >> + goto err; > The same here, you don't need to remove devargs from the devargs_list. > > Instead of removing the devargs in the list, you could have the following code:: > > static int > rte_eal_dev_detach_vdev(uint8_t port_id, char *vdevname) > { > ... > /* check whether the driver supports detach feature, or not */ > if (rte_eth_dev_is_detachable(port_id)) > goto err; > > /* get device name by port id */ > if (rte_eth_dev_get_name_by_port(port_id, name)) > goto err; > > if (rte_eal_vdev_uninit(name)) > goto err; > ... > } > > What do you think ? > > > Regards, > > Maxime
Hi Tetsuya, Thanks for your comment. On Fri, Feb 20, 2015 at 11:32 AM, Tetsuya Mukawa <mukawa@igel.co.jp> wrote: > On 2015/02/20 19:14, Maxime Leroy wrote: >> Hi Tetsuya, [...] >> > > Hi Maxime, > > I appreciate for your comment. > > When rte_eal_init() is called, if we have "--vdev" options, these will > be stored in vdevargs as you describe above. > I agree this is the current behavior of DPDK. > > When we call hotplug functions, I guess doing same thing will be more > consistent design. The rte_eal_devargs_add is here to store a white list parameters for later creating the devices. It means that the devargs_list is only needed at the init to store the devargs parsed . I think we should not use the devargs_list after eal initialization. Why you want to add devargs in the devargs_list, if there are no needs to store this information ? At the end, it adds extra codes for nothing. > > For example, we can do like below. > 1. $ ./testpmd --vdev 'eth_pcap' -- -i > 2. testpmd>port detach It's exactly the same for physical device: 1. $./testpmd -w 0000:08:00:1 -- -i 2. testpmd> port detach But you don't call rte_eal_devargs_add with RTE_DEVTYPE_WHILISTED_PCI in rte_eal_dev_attach_pdev ? Thus it makes the hotplug implementation not coherent for me. What do you think ? Regards, Maxime
On 2015/02/21 0:20, Maxime Leroy wrote: > Hi Tetsuya, > > Thanks for your comment. > > On Fri, Feb 20, 2015 at 11:32 AM, Tetsuya Mukawa <mukawa@igel.co.jp> wrote: >> On 2015/02/20 19:14, Maxime Leroy wrote: >>> Hi Tetsuya, > [...] >> Hi Maxime, >> >> I appreciate for your comment. >> >> When rte_eal_init() is called, if we have "--vdev" options, these will >> be stored in vdevargs as you describe above. >> I agree this is the current behavior of DPDK. >> >> When we call hotplug functions, I guess doing same thing will be more >> consistent design. > The rte_eal_devargs_add is here to store a white list parameters for > later creating the devices. > It means that the devargs_list is only needed at the init to store the > devargs parsed . > I think we should not use the devargs_list after eal initialization. > > Why you want to add devargs in the devargs_list, if there are no needs > to store this information ? In eal initialization code, virtual device names stored in devargs are checked not to register a same device name twice. And each init function of PMD just trust a device name received by eal. So there is no code in PMD to check whether device name is unique. For example, according to your suggestion, how to prevent below case? $ ./testpmd -c f -n 1 -- -i testpmd> port attach eth_pcap0,iface=eth0 testpmd> port attach eth_pcap0,iface=eth1 Also, type below, after doing above. testpmd> port detach 0 Probably port 0 will be "eth_pcap0,iface=eth0". But uninit code of PMD only receives a device name like 'eth_pcap0'. (We have 2 'eth_pcap0' devices in PMD.) To prevent above case, probably we have 2 options at least. One is changing init code of all virtual PMDs not to register same device name. The other is to use devargs_list in EAL, and call init code of PMD with a unique device name. But first case, eal initialization code can calls init of PMD with unique name, but hotplug cannot. It's not so consistent behavior. Also, we need to have almost same code that assure unique name in each PMD. > At the end, it adds extra codes for nothing. > >> For example, we can do like below. >> 1. $ ./testpmd --vdev 'eth_pcap' -- -i >> 2. testpmd>port detach > It's exactly the same for physical device: > 1. $./testpmd -w 0000:08:00:1 -- -i > 2. testpmd> port detach > > But you don't call rte_eal_devargs_add with RTE_DEVTYPE_WHILISTED_PCI > in rte_eal_dev_attach_pdev ? Yes, I don't. Hotplug functions should not change BLACKLIST and WHITELIST. So not to touch the list is correct behavior. I guess you feel something strange about usage of devargs_list. If so, I agree with it. Probably the issue is that devargs_list is used for not only storing command parameters, but also not to register same virtual device name twice. > Thus it makes the hotplug implementation not coherent for me. > > What do you think ? Anyway, here is my guess. Current DPDK uses devargs not to register same virtual device name. If we follow this, probably using devargs is straight forward. But if we should change the design itself, could you check below. http://dpdk.org/dev/patchwork/patch/3374/ Thomas and I agreed that I will try to change design in post-rc1. In a future patch, probably it might be nice to have virtualdev_list, then we can use devargs_list only for storing command arguments. Regards, Tetsuya > Regards, > > Maxime
Hi Tetsuya, On Sat, Feb 21, 2015 at 4:49 AM, Tetsuya Mukawa <mukawa@igel.co.jp> wrote: > On 2015/02/21 0:20, Maxime Leroy wrote: [...] >> Why you want to add devargs in the devargs_list, if there are no needs >> to store this information ? > > In eal initialization code, virtual device names stored in devargs are > checked not to register a same device name twice. > And each init function of PMD just trust a device name received by eal. > So there is no code in PMD to check whether device name is unique. > I disagree with you. This check is not present in the master branch. You have added this check in your hotplug patchset, in this patch: [PATCH v10 10/14] eal/pci: Add a function to remove the entry of devargs list See: http://dpdk.org/ml/archives/dev/2015-February/013712.html Thus the problem should be already exist without your patches in the master branch. For example according to you, this testpmd command should create 2 devices with the same name: testpmd -c 0xc --vdev eth_pcap0,iface=eth0 --vdev eth_pcap0,iface=eth1 -n 2 -- -i But it's not the case: PMD: Initializing pmd_pcap for eth_pcap0 PMD: Creating pcap-backed ethdev on numa socket 0 PMD: Initializing pmd_pcap for eth_pcap0 PMD: Creating pcap-backed ethdev on numa socket 0 PMD: rte_eth_dev_allocate: Ethernet Device with name eth_pcap0 already allocated! In fact, it's not possible for any PMD_VDEV in the dpdk repo to create 2 devices with the same name. All the virtual device initialization functions use the rte_eth_dev_allocate function. This function prevents to create two ethernet devices with the same name: if (rte_eth_dev_allocated(name) != NULL) { PMD_DEBUG_TRACE("Ethernet Device with name %s already allocated!\n", name); return NULL; } > For example, according to your suggestion, how to prevent below case? > $ ./testpmd -c f -n 1 -- -i > testpmd> port attach eth_pcap0,iface=eth0 > testpmd> port attach eth_pcap0,iface=eth1 > > Also, type below, after doing above. > testpmd> port detach 0 > > Probably port 0 will be "eth_pcap0,iface=eth0". > But uninit code of PMD only receives a device name like 'eth_pcap0'. > (We have 2 'eth_pcap0' devices in PMD.) > > To prevent above case, probably we have 2 options at least. > One is changing init code of all virtual PMDs not to register same > device name. There are no need to change init code of all virtual PMDs to not register the same device name 2 times. Because it's already not possible to create 2 virtual device with the same name. (see my point above) > The other is to use devargs_list in EAL, and call init code of PMD with > a unique device name. Thus there are no needs to use the devargs_list for that. > [..] >> >> But you don't call rte_eal_devargs_add with RTE_DEVTYPE_WHILISTED_PCI >> in rte_eal_dev_attach_pdev ? > > Yes, I don't. > Hotplug functions should not change BLACKLIST and WHITELIST. > So not to touch the list is correct behavior. Yes the correct behaviour for Hotplug functions is to not use the devargs_list for physical and virtual devices ! Regards, Maxime
On 2015/02/21 21:49, Maxime Leroy wrote: > Hi Tetsuya, > > On Sat, Feb 21, 2015 at 4:49 AM, Tetsuya Mukawa <mukawa@igel.co.jp> wrote: >> On 2015/02/21 0:20, Maxime Leroy wrote: > [...] >>> Why you want to add devargs in the devargs_list, if there are no needs >>> to store this information ? >> In eal initialization code, virtual device names stored in devargs are >> checked not to register a same device name twice. >> And each init function of PMD just trust a device name received by eal. >> So there is no code in PMD to check whether device name is unique. >> > I disagree with you. This check is not present in the master branch. > > You have added this check in your hotplug patchset, in this patch: > [PATCH v10 10/14] eal/pci: Add a function to remove the entry of > devargs list > See: http://dpdk.org/ml/archives/dev/2015-February/013712.html > > Thus the problem should be already exist without your patches in the > master branch. > > For example according to you, this testpmd command should create 2 > devices with the same name: > testpmd -c 0xc --vdev eth_pcap0,iface=eth0 --vdev eth_pcap0,iface=eth1 > -n 2 -- -i > > But it's not the case: > PMD: Initializing pmd_pcap for eth_pcap0 > PMD: Creating pcap-backed ethdev on numa socket 0 > PMD: Initializing pmd_pcap for eth_pcap0 > PMD: Creating pcap-backed ethdev on numa socket 0 > PMD: rte_eth_dev_allocate: Ethernet Device with name eth_pcap0 already > allocated! > > In fact, it's not possible for any PMD_VDEV in the dpdk repo to create > 2 devices with the same name. > All the virtual device initialization functions use the > rte_eth_dev_allocate function. This function prevents to create two > ethernet devices with the same name: > > if (rte_eth_dev_allocated(name) != NULL) { > PMD_DEBUG_TRACE("Ethernet Device with name %s already > allocated!\n", name); > return NULL; > } > Ah, You are right. >> For example, according to your suggestion, how to prevent below case? >> $ ./testpmd -c f -n 1 -- -i >> testpmd> port attach eth_pcap0,iface=eth0 >> testpmd> port attach eth_pcap0,iface=eth1 >> >> Also, type below, after doing above. >> testpmd> port detach 0 >> >> Probably port 0 will be "eth_pcap0,iface=eth0". >> But uninit code of PMD only receives a device name like 'eth_pcap0'. >> (We have 2 'eth_pcap0' devices in PMD.) >> >> To prevent above case, probably we have 2 options at least. >> One is changing init code of all virtual PMDs not to register same >> device name. > There are no need to change init code of all virtual PMDs to not > register the same device name 2 times. > Because it's already not possible to create 2 virtual device with the > same name. (see my point above) > >> The other is to use devargs_list in EAL, and call init code of PMD with >> a unique device name. > Thus there are no needs to use the devargs_list for that. > > [..] >>> But you don't call rte_eal_devargs_add with RTE_DEVTYPE_WHILISTED_PCI >>> in rte_eal_dev_attach_pdev ? >> Yes, I don't. >> Hotplug functions should not change BLACKLIST and WHITELIST. >> So not to touch the list is correct behavior. > Yes the correct behaviour for Hotplug functions is to not use the > devargs_list for physical and virtual devices ! I totally agree with you now. It seems I have a missunderstanding about code not to duplicate a vdev name. Thanks for your suggestions, I will fix it in next patches. Regards, Tetsuya > > Regards, > > Maxime
This patch series adds a dynamic port hotplug framework to DPDK. With the patches, DPDK apps can attach or detach ports at runtime. The basic concept of the port hotplug is like followings. - DPDK apps must have responsibility to manage ports. DPDK apps only know which ports are attached or detached at the moment. The port hotplug framework is implemented to allow DPDK apps to manage ports. For example, when DPDK apps call port attach function, attached port number will be returned. Also, DPDK apps can detach port by port number. - Kernel support is needed for attaching or detaching physical device ports. To attach a new physical device port, the device will be recognized by userspace directly I/O framework in kernel at first. Then DPDK apps can call the port hotplug functions to attach ports. For detaching, steps are vice versa. - Before detach ports, ports must be stopped and closed. DPDK application must call rte_eth_dev_stop() and rte_eth_dev_close() before detaching ports. These function will call finalization codes of PMDs. But so far, no PMD frees all resources allocated by initialization. It means PMDs are needed to be fixed to support the port hotplug. 'RTE_PCI_DRV_DETACHABLE' is a new flag indicating a PMD supports detaching. Without this flag, detaching will be failed. - Mustn't affect legacy DPDK apps. No DPDK EAL behavior is changed, if the port hotplug functions are't called. So all legacy DPDK apps can still work without modifications. And a few limitations. - The port hotplug functions are not thread safe. DPDK apps should handle it. - Only support Linux and igb_uio so far. BSD and VFIO is not supported. I will send VFIO patches at least, but I don't have a plan to submit BSD patch so far. Here is port hotplug APIs. ------------------------------------------------------------------------------- /** * Attach a new device. * * @param devargs * A pointer to a strings array describing the new device * to be attached. The strings should be a pci address like * '0000:01:00.0' or virtual device name like 'eth_pcap0'. * @param port_id * A pointer to a port identifier actually attached. * @return * 0 on success and port_id is filled, negative on error */ int rte_eal_dev_attach(const char *devargs, uint8_t *port_id); /** * Detach a device. * * @param port_id * The port identifier of the device to detach. * @param addr * A pointer to a device name actually detached. * @return * 0 on success and devname is filled, negative on error */ int rte_eal_dev_detach(uint8_t port_id, char *devname); ------------------------------------------------------------------------------- This patch series are for DPDK EAL. To use port hotplug function by DPDK apps, each PMD should be fixed to support 'RTE_PCI_DRV_DETACHABLE' flag. Please check a patch for pcap PMD. Also, please check testpmd patch. It will show you how to fix your legacy applications to support port hotplug feature. PATCH v11 changes - Remove needless devargs handling codes. - Replace get_vdev_name() by rte_eal_parse_devargs_str(). - Replace rte_eal_vdev_find_and_init by rte_eal_vdev_init() - Replace rte_eal_vdev_find_and_uninit by rte_eal_vdev_uninit() - Fix rte_eal_dev_init() to use rte_eal_vdev_init(). - Remove needless patch. (Thanks to Maxime Leroy) PATCH v10 changes - Add comments. - Chagne order of version.map. - Fix comment of "rte_ethdev.h". (Thanks to Thomas Monjalon) - Add size parameter to rte_eth_dev_create_unique_device_name(). (Thanks to Iremonger, Bernard) PATCH v9 changes - Fix commit title. - Fix commit log. - Fix comments. - Define CONFIG_RTE_LIBRTE_EAL_HOTPLUG at the top of this patch series. - DEV_INVALID/VALID are removed. - DEV_DISCONNECTED is replaced by DEV_DETACHED. - DEV_CONNECTED is replaced by DEV_ATTACHED. - rte_eth_dev_allocate_new_port() is renamed to rte_eth_dev_find_free_port(). - rte_eth_dev_validate_port() is renamed to rte_eth_dev_is_valid_port(). - rte_eth_dev_is_valid_port() is changed not to handle log toggle. - eal_compare_pci_addr() is replaced by rte_eal_compare_pci_addr(). - rte_eth_dev_free() is replaced by rte_eth_dev_release_port(). - Add a function to create a unique device name. - Change parameter of pci_devuninit_t and rte_eth_dev_uninit. - Remove code that initiaize callback of ethdev from rte_eth_dev_uninit(). - Remove pci_unmap_device(). It will be implemented in later patch. - rte_eth_dev_check_detachable() is replaced by rte_eth_dev_is_detachable(). - strncpy() is replaced by strcpy(). - Implement pci_unmap_device() in this patch. - Remove "rte_dev_hotplug.h". - Remove needless "#ifdef". - Remove RTE_EAL_INVOKE_TYPE_PROBE/CLOSE. - RTE_ETH_DEV_PHYSICAL is replaced by RTE_ETH_DEV_PCI. - Use strcmp() instead of strncmp(). - Remove RTE_EAL_INVOKE_TYPE_PROBE/CLOSE. (Thanks to Thomas Monjalon) - Change definition of rte_dev_uninit_t. (Thanks to Thomas Monjalon and Maxime Leroy) - Add missing symbol in version map. (Thanks to Nail Horman) PATCH v8 changes - Fix Makefile and add version map file. - Add missing symbol in version map. - Fix pci_scan_one() to update sysfs values. (Thanks to Qiu, Michael and Iremonger, Bernard) - NONE_TRACE is replaced by NO_TRACE. - Fix typo. - Add size parameter to rte_eth_dev_save(). (Thanks to Iremonger, Bernard) PATCH v7 changes - Add a new section to programmer's guide. (Thanks to Iremonger, Bernard) - Fix port checking implementation of star_port(). - Fix typo of warning messages. - Add pt_driver checking to rte_eth_dev_check_detachable(). (Thanks to Qiu, Michael) PATCH v6 changes - Fix rte_eth_dev_uninit() to handle a return value of uninit function of PMD. To do this, below changes also be applied. - Fix a parameter of rte_eth_dev_free(). - Use rte_eth_dev structure as the paramter of rte_eth_dev_free(). PATCH v5 changes - Add runtime check passthrough driver type, like vfio-pci, igb_uio and uio_pci_generic. This was done by Qiu, Michael. Thanks a lot. - Change function names like below. - rte_eal_dev_find_and_invoke() to rte_eal_vdev_find_and_invoke(). - rte_eal_dev_invoke() to rte_eal_vdev_invoke(). - Add code to handle a return value of rte_eal_devargs_remove(). - Fix pci address format in rte_eal_dev_detach(). - Remove RTE_EAL_INVOKE_TYPE_UNKNOWN, because it's unused. - Change function definition of rte_eal_devargs_remove(). - Fix pci_unmap_device() to check pt_driver. - Fix return value of below functions. - rte_eth_dev_get_changed_port(). - rte_eth_dev_get_port_by_addr(). - Change paramters of rte_eth_dev_validate_port() to cleanup code. - Fix pci_scan_one to handle pt_driver correctly. (Thanks to Qiu, Michael for above suggestions) PATCH v4 changes - Merge patches to review easier. - Fix indent of 'if' statement. - Fix calculation method of eal_compare_pci_addr(). - Fix header file declaration. - Add header file to determine if hotplug can be enabled. (Thanks to Qiu, Michael) - Use braces with 'for' loop. - Add parameter checking. - Fix sanity check code - Fix comments of rte_eth_dev_type. - Change function names. (Thanks to Iremonger, Bernard) PATCH v3 changes: - Fix enum definition used in rte_ethdev.c. (Thanks to Zhang, Helin) PATCH v2 changes: - Replace rte_eal_dev_attach_pdev(), rte_eal_dev_detach_pdev, rte_eal_dev_attach_vdev() and rte_eal_dev_detach_vdev() to rte_eal_dev_attach() and rte_eal_dev_detach(). - Add parameter values checking. - Refashion a few functions. (Thanks to Iremonger, Bernard) PATCH v1 Changes: - Fix error checking code of librte_eth APIs. - Fix issue that port from pcap PMD cannot be detached correctly. - Fix issue that testpmd could hang after forwarding, if attaching and detaching is repeatedly. - Fix if-condition of rte_eth_dev_get_port_by_addr(). (Thanks to Mark Enright) RFC PATCH v2 Changes: - remove 'rte_eth_dev_validate_port()', and cleanup codes. Michael Qiu (2): eal_pci: Add flag to hold kernel driver type eal_pci: pci memory map work with driver type Tetsuya Mukawa (11): eal: Enable port Hotplug framework in Linux eal/pci,ethdev: Remove assumption that port will not be detached eal/pci: Consolidate pci address comparison APIs ethdev: Add rte_eth_dev_release_port to release specified port eal,ethdev: Add a function and function pointers to close ether device ethdev: Add functions that will be used by port hotplug functions eal/linux/pci: Add functions for unmapping igb_uio resources eal/pci: Add probe and close functions of pci driver ethdev: Add one dev_type parameter to rte_eth_dev_allocate eal/pci: Add rte_eal_dev_attach/detach() functions doc: Add port hotplug framework section to programmers guide app/test/virtual_pmd.c | 2 +- config/common_bsdapp | 6 + config/common_linuxapp | 5 + doc/guides/prog_guide/index.rst | 1 + doc/guides/prog_guide/port_hotplug_framework.rst | 110 ++++++ lib/librte_eal/bsdapp/eal/eal_pci.c | 29 +- lib/librte_eal/common/eal_common_dev.c | 285 +++++++++++++- lib/librte_eal/common/eal_common_devargs.c | 46 ++- lib/librte_eal/common/eal_common_pci.c | 100 ++++- lib/librte_eal/common/eal_private.h | 26 ++ lib/librte_eal/common/include/rte_dev.h | 33 ++ lib/librte_eal/common/include/rte_devargs.h | 28 ++ lib/librte_eal/common/include/rte_pci.h | 82 +++++ lib/librte_eal/linuxapp/eal/Makefile | 1 + lib/librte_eal/linuxapp/eal/eal_pci.c | 230 ++++++++++-- lib/librte_eal/linuxapp/eal/eal_pci_init.h | 7 + lib/librte_eal/linuxapp/eal/eal_pci_uio.c | 67 +++- lib/librte_eal/linuxapp/eal/rte_eal_version.map | 2 + lib/librte_ether/rte_ethdev.c | 451 ++++++++++++++++++----- lib/librte_ether/rte_ethdev.h | 149 +++++++- lib/librte_ether/rte_ether_version.map | 7 + lib/librte_pmd_af_packet/rte_eth_af_packet.c | 2 +- lib/librte_pmd_bond/rte_eth_bond_api.c | 2 +- lib/librte_pmd_pcap/rte_eth_pcap.c | 2 +- lib/librte_pmd_ring/rte_eth_ring.c | 2 +- lib/librte_pmd_xenvirt/rte_eth_xenvirt.c | 2 +- 26 files changed, 1490 insertions(+), 187 deletions(-) create mode 100644 doc/guides/prog_guide/port_hotplug_framework.rst -- 1.9.1
The patch adds CONFIG_RTE_LIBRTE_EAL_HOTPLUG in Linux and BSD configuration. So far, Hotplug functions only support linux. v9: - Move this patch at the top of this patch series. (Thanks to Thomas Monjalon) Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp> --- config/common_bsdapp | 6 ++++++ config/common_linuxapp | 5 +++++ 2 files changed, 11 insertions(+) diff --git a/config/common_bsdapp b/config/common_bsdapp index 4c0cfc0..c24f687 100644 --- a/config/common_bsdapp +++ b/config/common_bsdapp @@ -116,6 +116,12 @@ CONFIG_RTE_LIBRTE_EAL_BSDAPP=y CONFIG_RTE_LIBRTE_EAL_LINUXAPP=n # +# Compile Environment Abstraction Layer to support hotplug +# So far, Hotplug functions only support linux +# +CONFIG_RTE_LIBRTE_EAL_HOTPLUG=n + +# # Compile Environment Abstraction Layer to support Vmware TSC map # CONFIG_RTE_LIBRTE_EAL_VMWARE_TSC_MAP_SUPPORT=y diff --git a/config/common_linuxapp b/config/common_linuxapp index 0234236..d66b008 100644 --- a/config/common_linuxapp +++ b/config/common_linuxapp @@ -114,6 +114,11 @@ CONFIG_RTE_PCI_MAX_READ_REQUEST_SIZE=0 CONFIG_RTE_LIBRTE_EAL_LINUXAPP=y # +# Compile Environment Abstraction Layer to support hotplug +# +CONFIG_RTE_LIBRTE_EAL_HOTPLUG=y + +# # Compile Environment Abstraction Layer to support Vmware TSC map # CONFIG_RTE_LIBRTE_EAL_VMWARE_TSC_MAP_SUPPORT=y -- 1.9.1
From: Michael Qiu <michael.qiu@intel.com> Currently, dpdk has no ability to know which type of driver( vfio-pci/igb_uio/uio_pci_generic) the device used. It only can check whether vfio is enabled or not staticly. It really useful to have the flag, becasue different type need to handle differently in runtime. For example, pci memory map, pot hotplug, and so on. This patch add a flag field for pci device to solve above issue. Signed-off-by: Michael Qiu <michael.qiu@intel.com> Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp> --- lib/librte_eal/common/include/rte_pci.h | 8 +++++ lib/librte_eal/linuxapp/eal/eal_pci.c | 53 +++++++++++++++++++++++++++++++-- 2 files changed, 59 insertions(+), 2 deletions(-) diff --git a/lib/librte_eal/common/include/rte_pci.h b/lib/librte_eal/common/include/rte_pci.h index 4301c16..5e0ba00 100644 --- a/lib/librte_eal/common/include/rte_pci.h +++ b/lib/librte_eal/common/include/rte_pci.h @@ -142,6 +142,13 @@ struct rte_pci_addr { struct rte_devargs; +enum rte_pt_driver { + RTE_PT_UNKNOWN = 0, + RTE_PT_IGB_UIO = 1, + RTE_PT_VFIO = 2, + RTE_PT_UIO_GENERIC = 3, +}; + /** * A structure describing a PCI device. */ @@ -155,6 +162,7 @@ struct rte_pci_device { uint16_t max_vfs; /**< sriov enable if not zero */ int numa_node; /**< NUMA node connection */ struct rte_devargs *devargs; /**< Device user arguments */ + enum rte_pt_driver pt_driver; /**< Driver of passthrough */ }; /** Any PCI device identifier (vendor, device, ...) */ diff --git a/lib/librte_eal/linuxapp/eal/eal_pci.c b/lib/librte_eal/linuxapp/eal/eal_pci.c index 63bcbce..9fe2851 100644 --- a/lib/librte_eal/linuxapp/eal/eal_pci.c +++ b/lib/librte_eal/linuxapp/eal/eal_pci.c @@ -97,6 +97,35 @@ error: return -1; } +static int +pci_get_kernel_driver_by_path(const char *filename, char *dri_name) +{ + int count; + char path[PATH_MAX]; + char *name; + + if (!filename || !dri_name) + return -1; + + count = readlink(filename, path, PATH_MAX); + if (count >= PATH_MAX) + return -1; + + /* For device does not have a driver */ + if (count < 0) + return 1; + + path[count] = '\0'; + + name = strrchr(path, '/'); + if (name) { + strncpy(dri_name, name + 1, strlen(name + 1) + 1); + return 0; + } + + return -1; +} + void * pci_find_max_end_va(void) { @@ -220,11 +249,12 @@ pci_scan_one(const char *dirname, uint16_t domain, uint8_t bus, char filename[PATH_MAX]; unsigned long tmp; struct rte_pci_device *dev; + char driver[PATH_MAX]; + int ret; dev = malloc(sizeof(*dev)); - if (dev == NULL) { + if (dev == NULL) return -1; - } memset(dev, 0, sizeof(*dev)); dev->addr.domain = domain; @@ -303,6 +333,25 @@ pci_scan_one(const char *dirname, uint16_t domain, uint8_t bus, return -1; } + /* parse driver */ + snprintf(filename, sizeof(filename), "%s/driver", dirname); + ret = pci_get_kernel_driver_by_path(filename, driver); + if (!ret) { + if (!strcmp(driver, "vfio-pci")) + dev->pt_driver = RTE_PT_VFIO; + else if (!strcmp(driver, "igb_uio")) + dev->pt_driver = RTE_PT_IGB_UIO; + else if (!strcmp(driver, "uio_pci_generic")) + dev->pt_driver = RTE_PT_UIO_GENERIC; + else + dev->pt_driver = RTE_PT_UNKNOWN; + } else if (ret < 0) { + RTE_LOG(ERR, EAL, "Fail to get kernel driver\n"); + free(dev); + return -1; + } else + dev->pt_driver = RTE_PT_UNKNOWN; + /* device is valid, add in list (sorted) */ if (TAILQ_EMPTY(&pci_device_list)) { TAILQ_INSERT_TAIL(&pci_device_list, dev, next); -- 1.9.1
From: Michael Qiu <michael.qiu@intel.com> With the driver type flag in struct rte_pci_dev, we do not need to always map uio devices with vfio related function when vfio enabled. Signed-off-by: Michael Qiu <michael.qiu@intel.com> Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp> --- lib/librte_eal/linuxapp/eal/eal_pci.c | 30 +++++++++++++++++------------- 1 file changed, 17 insertions(+), 13 deletions(-) diff --git a/lib/librte_eal/linuxapp/eal/eal_pci.c b/lib/librte_eal/linuxapp/eal/eal_pci.c index 9fe2851..c04f897 100644 --- a/lib/librte_eal/linuxapp/eal/eal_pci.c +++ b/lib/librte_eal/linuxapp/eal/eal_pci.c @@ -554,25 +554,29 @@ pci_config_space_set(struct rte_pci_device *dev) static int pci_map_device(struct rte_pci_device *dev) { - int ret, mapped = 0; + int ret = -1; /* try mapping the NIC resources using VFIO if it exists */ + switch (dev->pt_driver) { + case RTE_PT_VFIO: #ifdef VFIO_PRESENT - if (pci_vfio_is_enabled()) { - ret = pci_vfio_map_resource(dev); - if (ret == 0) - mapped = 1; - else if (ret < 0) - return ret; - } + if (pci_vfio_is_enabled()) + ret = pci_vfio_map_resource(dev); #endif - /* map resources for devices that use uio_pci_generic or igb_uio */ - if (!mapped) { + break; + case RTE_PT_IGB_UIO: + case RTE_PT_UIO_GENERIC: + /* map resources for devices that use uio */ ret = pci_uio_map_resource(dev); - if (ret != 0) - return ret; + break; + default: + RTE_LOG(DEBUG, EAL, " Not managed by known pt driver," + " skipped\n"); + ret = 1; + break; } - return 0; + + return ret; } /* -- 1.9.1
To remove assumption, do like followings. This patch adds "RTE_PCI_DRV_DETACHABLE" to drv_flags of rte_pci_driver structure. The flags indicate the driver can detach devices at runtime. Also, remove assumption that port will not be detached. To remove the assumption. - Add 'attached' member to rte_eth_dev structure. This member is used for indicating the port is attached, or not. DEV_ATTACHED indicates a port is attached. DEV_DETACHED indicates a port is detached. - Add rte_eth_dev_allocate_new_port(). This function is used for allocating new port. v9: - DEV_INVALID/VALID are removed. - DEV_DISCONNECTED is replaced by DEV_DETACHED. - DEV_CONNECTED is replaced by DEV_ATTACHED. - rte_eth_dev_allocate_new_port() is renamed to rte_eth_dev_find_free_port(). - rte_eth_dev_validate_port() is renamed to rte_eth_dev_is_valid_port(). - rte_eth_dev_is_valid_port() is changed not to handle log toggle. - Fix commit log to describe DEV_ATACHED and DEV_DETACHED. (Thanks to Thomas Monjalon) v8: - NONE_TRACE is changed to NO_TRACE. (Thanks to Iremonger, Bernard) v5: - Change parameters of rte_eth_dev_validate_port() to cleanup code. v4: - Use braces with 'for' loop. - Fix indent of 'if' statement. Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp> --- lib/librte_eal/common/include/rte_pci.h | 2 + lib/librte_ether/rte_ethdev.c | 248 ++++++++++++++++++++------------ lib/librte_ether/rte_ethdev.h | 5 + 3 files changed, 164 insertions(+), 91 deletions(-) diff --git a/lib/librte_eal/common/include/rte_pci.h b/lib/librte_eal/common/include/rte_pci.h index 5e0ba00..ffd13d9 100644 --- a/lib/librte_eal/common/include/rte_pci.h +++ b/lib/librte_eal/common/include/rte_pci.h @@ -210,6 +210,8 @@ struct rte_pci_driver { #define RTE_PCI_DRV_FORCE_UNBIND 0x0004 /** Device driver supports link state interrupt */ #define RTE_PCI_DRV_INTR_LSC 0x0008 +/** Device driver supports detaching capability */ +#define RTE_PCI_DRV_DETACHABLE 0x0010 /**< Internal use only - Macro used by pci addr parsing functions **/ #define GET_PCIADDR_FIELD(in, fd, lim, dlm) \ diff --git a/lib/librte_ether/rte_ethdev.c b/lib/librte_ether/rte_ethdev.c index 27bbb0b..0e1e5c9 100644 --- a/lib/librte_ether/rte_ethdev.c +++ b/lib/librte_ether/rte_ethdev.c @@ -175,6 +175,11 @@ enum { STAT_QMAP_RX }; +enum { + DEV_DETACHED = 0, + DEV_ATTACHED +}; + static inline void rte_eth_dev_data_alloc(void) { @@ -201,19 +206,34 @@ rte_eth_dev_allocated(const char *name) { unsigned i; - for (i = 0; i < nb_ports; i++) { - if (strcmp(rte_eth_devices[i].data->name, name) == 0) + for (i = 0; i < RTE_MAX_ETHPORTS; i++) { + if ((rte_eth_devices[i].attached == DEV_ATTACHED) && + strcmp(rte_eth_devices[i].data->name, name) == 0) return &rte_eth_devices[i]; } return NULL; } +static uint8_t +rte_eth_dev_find_free_port(void) +{ + unsigned i; + + for (i = 0; i < RTE_MAX_ETHPORTS; i++) { + if (rte_eth_devices[i].attached == DEV_DETACHED) + return i; + } + return RTE_MAX_ETHPORTS; +} + struct rte_eth_dev * rte_eth_dev_allocate(const char *name) { + uint8_t port_id; struct rte_eth_dev *eth_dev; - if (nb_ports == RTE_MAX_ETHPORTS) { + port_id = rte_eth_dev_find_free_port(); + if (port_id == RTE_MAX_ETHPORTS) { PMD_DEBUG_TRACE("Reached maximum number of Ethernet ports\n"); return NULL; } @@ -226,10 +246,12 @@ rte_eth_dev_allocate(const char *name) return NULL; } - eth_dev = &rte_eth_devices[nb_ports]; - eth_dev->data = &rte_eth_dev_data[nb_ports]; + eth_dev = &rte_eth_devices[port_id]; + eth_dev->data = &rte_eth_dev_data[port_id]; snprintf(eth_dev->data->name, sizeof(eth_dev->data->name), "%s", name); - eth_dev->data->port_id = nb_ports++; + eth_dev->data->port_id = port_id; + eth_dev->attached = DEV_ATTACHED; + nb_ports++; return eth_dev; } @@ -283,6 +305,7 @@ rte_eth_dev_init(struct rte_pci_driver *pci_drv, (unsigned) pci_dev->id.device_id); if (rte_eal_process_type() == RTE_PROC_PRIMARY) rte_free(eth_dev->data->dev_private); + eth_dev->attached = DEV_DETACHED; nb_ports--; return diag; } @@ -308,10 +331,20 @@ rte_eth_driver_register(struct eth_driver *eth_drv) rte_eal_pci_register(ð_drv->pci_drv); } +static int +rte_eth_dev_is_valid_port(uint8_t port_id) +{ + if (port_id >= RTE_MAX_ETHPORTS || + rte_eth_devices[port_id].attached != DEV_ATTACHED) + return 0; + else + return 1; +} + int rte_eth_dev_socket_id(uint8_t port_id) { - if (port_id >= nb_ports) + if (!rte_eth_dev_is_valid_port(port_id)) return -1; return rte_eth_devices[port_id].pci_dev->numa_node; } @@ -369,7 +402,7 @@ rte_eth_dev_rx_queue_start(uint8_t port_id, uint16_t rx_queue_id) * in a multi-process setup*/ PROC_PRIMARY_OR_ERR_RET(-E_RTE_SECONDARY); - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return -EINVAL; } @@ -395,7 +428,7 @@ rte_eth_dev_rx_queue_stop(uint8_t port_id, uint16_t rx_queue_id) * in a multi-process setup*/ PROC_PRIMARY_OR_ERR_RET(-E_RTE_SECONDARY); - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return -EINVAL; } @@ -421,7 +454,7 @@ rte_eth_dev_tx_queue_start(uint8_t port_id, uint16_t tx_queue_id) * in a multi-process setup*/ PROC_PRIMARY_OR_ERR_RET(-E_RTE_SECONDARY); - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return -EINVAL; } @@ -447,7 +480,7 @@ rte_eth_dev_tx_queue_stop(uint8_t port_id, uint16_t tx_queue_id) * in a multi-process setup*/ PROC_PRIMARY_OR_ERR_RET(-E_RTE_SECONDARY); - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return -EINVAL; } @@ -703,10 +736,11 @@ rte_eth_dev_configure(uint8_t port_id, uint16_t nb_rx_q, uint16_t nb_tx_q, * in a multi-process setup*/ PROC_PRIMARY_OR_ERR_RET(-E_RTE_SECONDARY); - if (port_id >= nb_ports || port_id >= RTE_MAX_ETHPORTS) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-EINVAL); } + dev = &rte_eth_devices[port_id]; FUNC_PTR_OR_ERR_RET(*dev->dev_ops->dev_infos_get, -ENOTSUP); @@ -888,10 +922,11 @@ rte_eth_dev_start(uint8_t port_id) * in a multi-process setup*/ PROC_PRIMARY_OR_ERR_RET(-E_RTE_SECONDARY); - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%" PRIu8 "\n", port_id); + if (!rte_eth_dev_is_valid_port(port_id)) { + PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-EINVAL); } + dev = &rte_eth_devices[port_id]; FUNC_PTR_OR_ERR_RET(*dev->dev_ops->dev_start, -ENOTSUP); @@ -927,10 +962,11 @@ rte_eth_dev_stop(uint8_t port_id) * in a multi-process setup*/ PROC_PRIMARY_OR_RET(); - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%" PRIu8 "\n", port_id); + if (!rte_eth_dev_is_valid_port(port_id)) { + PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return; } + dev = &rte_eth_devices[port_id]; FUNC_PTR_OR_RET(*dev->dev_ops->dev_stop); @@ -955,10 +991,11 @@ rte_eth_dev_set_link_up(uint8_t port_id) * in a multi-process setup*/ PROC_PRIMARY_OR_ERR_RET(-E_RTE_SECONDARY); - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return -EINVAL; } + dev = &rte_eth_devices[port_id]; FUNC_PTR_OR_ERR_RET(*dev->dev_ops->dev_set_link_up, -ENOTSUP); @@ -974,10 +1011,11 @@ rte_eth_dev_set_link_down(uint8_t port_id) * in a multi-process setup*/ PROC_PRIMARY_OR_ERR_RET(-E_RTE_SECONDARY); - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return -EINVAL; } + dev = &rte_eth_devices[port_id]; FUNC_PTR_OR_ERR_RET(*dev->dev_ops->dev_set_link_down, -ENOTSUP); @@ -993,7 +1031,7 @@ rte_eth_dev_close(uint8_t port_id) * in a multi-process setup*/ PROC_PRIMARY_OR_RET(); - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return; } @@ -1021,10 +1059,11 @@ rte_eth_rx_queue_setup(uint8_t port_id, uint16_t rx_queue_id, * in a multi-process setup*/ PROC_PRIMARY_OR_ERR_RET(-E_RTE_SECONDARY); - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-EINVAL); } + dev = &rte_eth_devices[port_id]; if (rx_queue_id >= dev->data->nb_rx_queues) { PMD_DEBUG_TRACE("Invalid RX queue_id=%d\n", rx_queue_id); @@ -1094,10 +1133,11 @@ rte_eth_tx_queue_setup(uint8_t port_id, uint16_t tx_queue_id, * in a multi-process setup*/ PROC_PRIMARY_OR_ERR_RET(-E_RTE_SECONDARY); - if (port_id >= RTE_MAX_ETHPORTS || port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-EINVAL); } + dev = &rte_eth_devices[port_id]; if (tx_queue_id >= dev->data->nb_tx_queues) { PMD_DEBUG_TRACE("Invalid TX queue_id=%d\n", tx_queue_id); @@ -1127,10 +1167,11 @@ rte_eth_promiscuous_enable(uint8_t port_id) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return; } + dev = &rte_eth_devices[port_id]; FUNC_PTR_OR_RET(*dev->dev_ops->promiscuous_enable); @@ -1143,10 +1184,11 @@ rte_eth_promiscuous_disable(uint8_t port_id) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return; } + dev = &rte_eth_devices[port_id]; FUNC_PTR_OR_RET(*dev->dev_ops->promiscuous_disable); @@ -1159,7 +1201,7 @@ rte_eth_promiscuous_get(uint8_t port_id) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return -1; } @@ -1173,10 +1215,11 @@ rte_eth_allmulticast_enable(uint8_t port_id) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return; } + dev = &rte_eth_devices[port_id]; FUNC_PTR_OR_RET(*dev->dev_ops->allmulticast_enable); @@ -1189,10 +1232,11 @@ rte_eth_allmulticast_disable(uint8_t port_id) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return; } + dev = &rte_eth_devices[port_id]; FUNC_PTR_OR_RET(*dev->dev_ops->allmulticast_disable); @@ -1205,7 +1249,7 @@ rte_eth_allmulticast_get(uint8_t port_id) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return -1; } @@ -1233,10 +1277,11 @@ rte_eth_link_get(uint8_t port_id, struct rte_eth_link *eth_link) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return; } + dev = &rte_eth_devices[port_id]; if (dev->data->dev_conf.intr_conf.lsc != 0) @@ -1253,10 +1298,11 @@ rte_eth_link_get_nowait(uint8_t port_id, struct rte_eth_link *eth_link) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return; } + dev = &rte_eth_devices[port_id]; if (dev->data->dev_conf.intr_conf.lsc != 0) @@ -1273,10 +1319,11 @@ rte_eth_stats_get(uint8_t port_id, struct rte_eth_stats *stats) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-ENODEV); } + dev = &rte_eth_devices[port_id]; memset(stats, 0, sizeof(*stats)); @@ -1291,10 +1338,11 @@ rte_eth_stats_reset(uint8_t port_id) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return; } + dev = &rte_eth_devices[port_id]; FUNC_PTR_OR_RET(*dev->dev_ops->stats_reset); @@ -1312,10 +1360,11 @@ rte_eth_xstats_get(uint8_t port_id, struct rte_eth_xstats *xstats, uint64_t val; char *stats_ptr; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return -1; } + dev = &rte_eth_devices[port_id]; /* implemented by the driver */ @@ -1381,10 +1430,11 @@ rte_eth_xstats_reset(uint8_t port_id) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return; } + dev = &rte_eth_devices[port_id]; /* implemented by the driver */ @@ -1403,10 +1453,11 @@ set_queue_stats_mapping(uint8_t port_id, uint16_t queue_id, uint8_t stat_idx, { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return -ENODEV; } + dev = &rte_eth_devices[port_id]; FUNC_PTR_OR_ERR_RET(*dev->dev_ops->queue_stats_mapping_set, -ENOTSUP); @@ -1438,10 +1489,11 @@ rte_eth_dev_info_get(uint8_t port_id, struct rte_eth_dev_info *dev_info) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return; } + dev = &rte_eth_devices[port_id]; memset(dev_info, 0, sizeof(struct rte_eth_dev_info)); @@ -1458,10 +1510,11 @@ rte_eth_macaddr_get(uint8_t port_id, struct ether_addr *mac_addr) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return; } + dev = &rte_eth_devices[port_id]; ether_addr_copy(&dev->data->mac_addrs[0], mac_addr); } @@ -1472,7 +1525,7 @@ rte_eth_dev_get_mtu(uint8_t port_id, uint16_t *mtu) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-ENODEV); } @@ -1488,7 +1541,7 @@ rte_eth_dev_set_mtu(uint8_t port_id, uint16_t mtu) int ret; struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-ENODEV); } @@ -1508,10 +1561,11 @@ rte_eth_dev_vlan_filter(uint8_t port_id, uint16_t vlan_id, int on) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-ENODEV); } + dev = &rte_eth_devices[port_id]; if (! (dev->data->dev_conf.rxmode.hw_vlan_filter)) { PMD_DEBUG_TRACE("port %d: vlan-filtering disabled\n", port_id); @@ -1533,7 +1587,7 @@ rte_eth_dev_set_vlan_strip_on_queue(uint8_t port_id, uint16_t rx_queue_id, int o { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-ENODEV); } @@ -1555,7 +1609,7 @@ rte_eth_dev_set_vlan_ether_type(uint8_t port_id, uint16_t tpid) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-ENODEV); } @@ -1575,7 +1629,7 @@ rte_eth_dev_set_vlan_offload(uint8_t port_id, int offload_mask) int mask = 0; int cur, org = 0; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-ENODEV); } @@ -1620,7 +1674,7 @@ rte_eth_dev_get_vlan_offload(uint8_t port_id) struct rte_eth_dev *dev; int ret = 0; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-ENODEV); } @@ -1644,10 +1698,11 @@ rte_eth_dev_set_vlan_pvid(uint8_t port_id, uint16_t pvid, int on) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-ENODEV); } + dev = &rte_eth_devices[port_id]; FUNC_PTR_OR_ERR_RET(*dev->dev_ops->vlan_pvid_set, -ENOTSUP); (*dev->dev_ops->vlan_pvid_set)(dev, pvid, on); @@ -1662,7 +1717,7 @@ rte_eth_dev_fdir_add_signature_filter(uint8_t port_id, { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-ENODEV); } @@ -1696,7 +1751,7 @@ rte_eth_dev_fdir_update_signature_filter(uint8_t port_id, { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-ENODEV); } @@ -1730,7 +1785,7 @@ rte_eth_dev_fdir_remove_signature_filter(uint8_t port_id, { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-ENODEV); } @@ -1761,7 +1816,7 @@ rte_eth_dev_fdir_get_infos(uint8_t port_id, struct rte_eth_fdir *fdir) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-ENODEV); } @@ -1786,7 +1841,7 @@ rte_eth_dev_fdir_add_perfect_filter(uint8_t port_id, { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-ENODEV); } @@ -1826,7 +1881,7 @@ rte_eth_dev_fdir_update_perfect_filter(uint8_t port_id, { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-ENODEV); } @@ -1864,7 +1919,7 @@ rte_eth_dev_fdir_remove_perfect_filter(uint8_t port_id, { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-ENODEV); } @@ -1900,7 +1955,7 @@ rte_eth_dev_fdir_set_masks(uint8_t port_id, struct rte_fdir_masks *fdir_mask) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-ENODEV); } @@ -1920,7 +1975,7 @@ rte_eth_dev_flow_ctrl_get(uint8_t port_id, struct rte_eth_fc_conf *fc_conf) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-ENODEV); } @@ -1936,7 +1991,7 @@ rte_eth_dev_flow_ctrl_set(uint8_t port_id, struct rte_eth_fc_conf *fc_conf) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-ENODEV); } @@ -1956,7 +2011,7 @@ rte_eth_dev_priority_flow_ctrl_set(uint8_t port_id, struct rte_eth_pfc_conf *pfc { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-ENODEV); } @@ -2035,7 +2090,7 @@ rte_eth_dev_rss_reta_update(uint8_t port_id, struct rte_eth_dev *dev; int ret; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return -ENODEV; } @@ -2086,10 +2141,11 @@ rte_eth_dev_rss_hash_update(uint8_t port_id, struct rte_eth_rss_conf *rss_conf) struct rte_eth_dev *dev; uint16_t rss_hash_protos; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-ENODEV); } + rss_hash_protos = rss_conf->rss_hf; if ((rss_hash_protos != 0) && ((rss_hash_protos & ETH_RSS_PROTO_MASK) == 0)) { @@ -2108,10 +2164,11 @@ rte_eth_dev_rss_hash_conf_get(uint8_t port_id, { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-ENODEV); } + dev = &rte_eth_devices[port_id]; FUNC_PTR_OR_ERR_RET(*dev->dev_ops->rss_hash_conf_get, -ENOTSUP); return (*dev->dev_ops->rss_hash_conf_get)(dev, rss_conf); @@ -2123,7 +2180,7 @@ rte_eth_dev_udp_tunnel_add(uint8_t port_id, { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return -ENODEV; } @@ -2149,10 +2206,11 @@ rte_eth_dev_udp_tunnel_delete(uint8_t port_id, { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return -ENODEV; } + dev = &rte_eth_devices[port_id]; if (udp_tunnel == NULL) { @@ -2174,7 +2232,7 @@ rte_eth_led_on(uint8_t port_id) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-ENODEV); } @@ -2189,7 +2247,7 @@ rte_eth_led_off(uint8_t port_id) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-ENODEV); } @@ -2229,10 +2287,11 @@ rte_eth_dev_mac_addr_add(uint8_t port_id, struct ether_addr *addr, int index; uint64_t pool_mask; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-ENODEV); } + dev = &rte_eth_devices[port_id]; FUNC_PTR_OR_ERR_RET(*dev->dev_ops->mac_addr_add, -ENOTSUP); @@ -2280,10 +2339,11 @@ rte_eth_dev_mac_addr_remove(uint8_t port_id, struct ether_addr *addr) struct rte_eth_dev *dev; int index; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-ENODEV); } + dev = &rte_eth_devices[port_id]; FUNC_PTR_OR_ERR_RET(*dev->dev_ops->mac_addr_remove, -ENOTSUP); @@ -2314,7 +2374,7 @@ rte_eth_dev_set_vf_rxmode(uint8_t port_id, uint16_t vf, struct rte_eth_dev *dev; struct rte_eth_dev_info dev_info; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("set VF RX mode:Invalid port_id=%d\n", port_id); return (-ENODEV); @@ -2369,7 +2429,7 @@ rte_eth_dev_uc_hash_table_set(uint8_t port_id, struct ether_addr *addr, int ret; struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("unicast hash setting:Invalid port_id=%d\n", port_id); return (-ENODEV); @@ -2422,7 +2482,7 @@ rte_eth_dev_uc_all_hash_table_set(uint8_t port_id, uint8_t on) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("unicast hash setting:Invalid port_id=%d\n", port_id); return (-ENODEV); @@ -2441,7 +2501,7 @@ rte_eth_dev_set_vf_rx(uint8_t port_id,uint16_t vf, uint8_t on) struct rte_eth_dev *dev; struct rte_eth_dev_info dev_info; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-ENODEV); } @@ -2467,7 +2527,7 @@ rte_eth_dev_set_vf_tx(uint8_t port_id,uint16_t vf, uint8_t on) struct rte_eth_dev *dev; struct rte_eth_dev_info dev_info; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("set pool tx:Invalid port_id=%d\n", port_id); return (-ENODEV); } @@ -2492,7 +2552,7 @@ rte_eth_dev_set_vf_vlan_filter(uint8_t port_id, uint16_t vlan_id, { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("VF VLAN filter:invalid port id=%d\n", port_id); return (-ENODEV); @@ -2523,7 +2583,7 @@ int rte_eth_set_queue_rate_limit(uint8_t port_id, uint16_t queue_idx, struct rte_eth_dev_info dev_info; struct rte_eth_link link; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("set queue rate limit:invalid port id=%d\n", port_id); return -ENODEV; @@ -2560,7 +2620,7 @@ int rte_eth_set_vf_rate_limit(uint8_t port_id, uint16_t vf, uint16_t tx_rate, if (q_msk == 0) return 0; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("set VF rate limit:invalid port id=%d\n", port_id); return -ENODEV; @@ -2594,7 +2654,7 @@ rte_eth_mirror_rule_set(uint8_t port_id, { struct rte_eth_dev *dev = &rte_eth_devices[port_id]; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-ENODEV); } @@ -2635,7 +2695,7 @@ rte_eth_mirror_rule_reset(uint8_t port_id, uint8_t rule_id) { struct rte_eth_dev *dev = &rte_eth_devices[port_id]; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-ENODEV); } @@ -2660,10 +2720,11 @@ rte_eth_rx_burst(uint8_t port_id, uint16_t queue_id, { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return 0; } + dev = &rte_eth_devices[port_id]; FUNC_PTR_OR_ERR_RET(*dev->rx_pkt_burst, 0); if (queue_id >= dev->data->nb_rx_queues) { @@ -2680,10 +2741,11 @@ rte_eth_tx_burst(uint8_t port_id, uint16_t queue_id, { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return 0; } + dev = &rte_eth_devices[port_id]; FUNC_PTR_OR_ERR_RET(*dev->tx_pkt_burst, 0); @@ -2700,10 +2762,11 @@ rte_eth_rx_queue_count(uint8_t port_id, uint16_t queue_id) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return 0; } + dev = &rte_eth_devices[port_id]; FUNC_PTR_OR_ERR_RET(*dev->dev_ops->rx_queue_count, 0); return (*dev->dev_ops->rx_queue_count)(dev, queue_id); @@ -2714,10 +2777,11 @@ rte_eth_rx_descriptor_done(uint8_t port_id, uint16_t queue_id, uint16_t offset) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-ENODEV); } + dev = &rte_eth_devices[port_id]; FUNC_PTR_OR_ERR_RET(*dev->dev_ops->rx_descriptor_done, -ENOTSUP); return (*dev->dev_ops->rx_descriptor_done)( \ @@ -2735,7 +2799,8 @@ rte_eth_dev_callback_register(uint8_t port_id, if (!cb_fn) return (-EINVAL); - if (port_id >= nb_ports) { + + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-EINVAL); } @@ -2775,7 +2840,8 @@ rte_eth_dev_callback_unregister(uint8_t port_id, if (!cb_fn) return (-EINVAL); - if (port_id >= nb_ports) { + + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-EINVAL); } @@ -2835,7 +2901,7 @@ int rte_eth_dev_bypass_init(uint8_t port_id) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-ENODEV); } @@ -2855,7 +2921,7 @@ rte_eth_dev_bypass_state_show(uint8_t port_id, uint32_t *state) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-ENODEV); } @@ -2874,7 +2940,7 @@ rte_eth_dev_bypass_state_set(uint8_t port_id, uint32_t *new_state) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-ENODEV); } @@ -2894,7 +2960,7 @@ rte_eth_dev_bypass_event_show(uint8_t port_id, uint32_t event, uint32_t *state) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-ENODEV); } @@ -2914,7 +2980,7 @@ rte_eth_dev_bypass_event_store(uint8_t port_id, uint32_t event, uint32_t state) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-ENODEV); } @@ -2934,7 +3000,7 @@ rte_eth_dev_wd_timeout_store(uint8_t port_id, uint32_t timeout) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-ENODEV); } @@ -2954,7 +3020,7 @@ rte_eth_dev_bypass_ver_show(uint8_t port_id, uint32_t *ver) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-ENODEV); } @@ -2974,7 +3040,7 @@ rte_eth_dev_bypass_wd_timeout_show(uint8_t port_id, uint32_t *wd_timeout) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-ENODEV); } @@ -2994,7 +3060,7 @@ rte_eth_dev_bypass_wd_reset(uint8_t port_id) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-ENODEV); } @@ -3015,7 +3081,7 @@ rte_eth_dev_filter_supported(uint8_t port_id, enum rte_filter_type filter_type) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return -ENODEV; } @@ -3032,7 +3098,7 @@ rte_eth_dev_filter_ctrl(uint8_t port_id, enum rte_filter_type filter_type, { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return -ENODEV; } diff --git a/lib/librte_ether/rte_ethdev.h b/lib/librte_ether/rte_ethdev.h index 4acd595..ef31bda 100644 --- a/lib/librte_ether/rte_ethdev.h +++ b/lib/librte_ether/rte_ethdev.h @@ -1399,6 +1399,7 @@ struct rte_eth_dev { struct eth_dev_ops *dev_ops; /**< Functions exported by PMD */ struct rte_pci_device *pci_dev; /**< PCI info. supplied by probing */ struct rte_eth_dev_cb_list callbacks; /**< User application callbacks */ + uint8_t attached; /**< Flag indicating the port is attached */ }; struct rte_eth_dev_sriov { @@ -1464,6 +1465,10 @@ extern struct rte_eth_dev rte_eth_devices[]; * initialized by the [matching] Ethernet driver during the PCI probing phase. * All devices whose port identifier is in the range * [0, rte_eth_dev_count() - 1] can be operated on by network applications. + * immediately after invoking rte_eal_init(). + * If the application unplugs a port using hotplug function, The enabled port + * numbers may be noncontiguous. In the case, the applications need to manage + * enabled port by themselves. * * @return * - The total number of usable Ethernet devices. -- 1.9.1
This patch replaces pci_addr_comparison() and memcmp() of pci addresses by rte_eal_compare_pci_addr(). To compare PCI addresses, rte_eal_compare_pci_addr() doesn't use memcmp(). This is because sizeof(struct rte_pci_addr) returns 6, but actually this structure is like below. struct rte_pci_addr { uint16_t domain; /**< Device domain */ uint8_t bus; /**< Device bus */ uint8_t devid; /**< Device ID */ uint8_t function; /**< Device function. */ }; If the structure is dynamically allocated in a function without bzero, last 1 byte may have value. As a result, memcmp may not work. To avoid such a case, rte_eal_compare_pci_addr() compare following values. dev_addr = (addr->domain << 24) | (addr->bus << 16) | (addr->devid << 8) | addr->function; v9: - eal_compare_pci_addr() is replaced by rte_eal_compare_pci_addr(). - Fix commit log. (Thanks to Thomas Monjalon) v8: - Fix pci_scan_one() to update sysfs values. (Thanks to Qiu, Michael and Iremonger, Bernard) v5: - Fix pci_scan_one to handle pt_driver correctly. v4: - Fix calculation method of eal_compare_pci_addr(). - Add parameter checking. Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp> --- lib/librte_eal/bsdapp/eal/eal_pci.c | 29 ++++++++++++-------------- lib/librte_eal/common/eal_common_pci.c | 2 +- lib/librte_eal/common/include/rte_pci.h | 34 +++++++++++++++++++++++++++++++ lib/librte_eal/linuxapp/eal/eal_pci.c | 30 +++++++++++++-------------- lib/librte_eal/linuxapp/eal/eal_pci_uio.c | 2 +- 5 files changed, 63 insertions(+), 34 deletions(-) diff --git a/lib/librte_eal/bsdapp/eal/eal_pci.c b/lib/librte_eal/bsdapp/eal/eal_pci.c index 74ecce7..9193f80 100644 --- a/lib/librte_eal/bsdapp/eal/eal_pci.c +++ b/lib/librte_eal/bsdapp/eal/eal_pci.c @@ -270,20 +270,6 @@ pci_uio_map_resource(struct rte_pci_device *dev) return (0); } -/* Compare two PCI device addresses. */ -static int -pci_addr_comparison(struct rte_pci_addr *addr, struct rte_pci_addr *addr2) -{ - uint64_t dev_addr = (addr->domain << 24) + (addr->bus << 16) + (addr->devid << 8) + addr->function; - uint64_t dev_addr2 = (addr2->domain << 24) + (addr2->bus << 16) + (addr2->devid << 8) + addr2->function; - - if (dev_addr > dev_addr2) - return 1; - else - return 0; -} - - /* Scan one pci sysfs entry, and fill the devices list from it. */ static int pci_scan_one(int dev_pci_fd, struct pci_conf *conf) @@ -356,13 +342,24 @@ pci_scan_one(int dev_pci_fd, struct pci_conf *conf) } else { struct rte_pci_device *dev2 = NULL; + int ret; TAILQ_FOREACH(dev2, &pci_device_list, next) { - if (pci_addr_comparison(&dev->addr, &dev2->addr)) + ret = rte_eal_compare_pci_addr(&dev->addr, &dev2->addr); + if (ret > 0) continue; - else { + else if (ret < 0) { TAILQ_INSERT_BEFORE(dev2, dev, next); return 0; + } else { /* already registered */ + /* update pt_driver */ + dev2->pt_driver = dev->pt_driver; + dev2->max_vfs = dev->max_vfs; + memmove(dev2->mem_resource, + dev->mem_resource, + sizeof(dev->mem_resource)); + free(dev); + return 0; } } TAILQ_INSERT_TAIL(&pci_device_list, dev, next); diff --git a/lib/librte_eal/common/eal_common_pci.c b/lib/librte_eal/common/eal_common_pci.c index f3c7f71..bf2793f 100644 --- a/lib/librte_eal/common/eal_common_pci.c +++ b/lib/librte_eal/common/eal_common_pci.c @@ -93,7 +93,7 @@ static struct rte_devargs *pci_devargs_lookup(struct rte_pci_device *dev) if (devargs->type != RTE_DEVTYPE_BLACKLISTED_PCI && devargs->type != RTE_DEVTYPE_WHITELISTED_PCI) continue; - if (!memcmp(&dev->addr, &devargs->pci.addr, sizeof(dev->addr))) + if (!rte_eal_compare_pci_addr(&dev->addr, &devargs->pci.addr)) return devargs; } return NULL; diff --git a/lib/librte_eal/common/include/rte_pci.h b/lib/librte_eal/common/include/rte_pci.h index ffd13d9..6814e91 100644 --- a/lib/librte_eal/common/include/rte_pci.h +++ b/lib/librte_eal/common/include/rte_pci.h @@ -272,6 +272,40 @@ eal_parse_pci_DomBDF(const char *input, struct rte_pci_addr *dev_addr) } #undef GET_PCIADDR_FIELD +/* Compare two PCI device addresses. */ +/** + * Utility function to compare two PCI device addresses. + * + * @param addr + * The PCI Bus-Device-Function address to compare + * @param addr2 + * The PCI Bus-Device-Function address to compare + * @return + * 0 on equal PCI address. + * Positive on addr is greater than addr2. + * Negative on addr is less than addr2, or error. + */ +static inline int +rte_eal_compare_pci_addr(struct rte_pci_addr *addr, struct rte_pci_addr *addr2) +{ + uint64_t dev_addr, dev_addr2; + + if ((addr == NULL) || (addr2 == NULL)) + return -1; + + dev_addr = (addr->domain << 24) | (addr->bus << 16) | + (addr->devid << 8) | addr->function; + dev_addr2 = (addr2->domain << 24) | (addr2->bus << 16) | + (addr2->devid << 8) | addr2->function; + + if (dev_addr > dev_addr2) + return 1; + else if (dev_addr < dev_addr2) + return -1; + else + return 0; +} + /** * Probe the PCI bus for registered drivers. * diff --git a/lib/librte_eal/linuxapp/eal/eal_pci.c b/lib/librte_eal/linuxapp/eal/eal_pci.c index c04f897..e6cead1 100644 --- a/lib/librte_eal/linuxapp/eal/eal_pci.c +++ b/lib/librte_eal/linuxapp/eal/eal_pci.c @@ -41,6 +41,7 @@ #include <rte_eal_memconfig.h> #include <rte_malloc.h> #include <rte_devargs.h> +#include <rte_memcpy.h> #include "rte_pci_dev_ids.h" #include "eal_filesystem.h" @@ -227,20 +228,6 @@ error: return -1; } -/* Compare two PCI device addresses. */ -static int -pci_addr_comparison(struct rte_pci_addr *addr, struct rte_pci_addr *addr2) -{ - uint64_t dev_addr = (addr->domain << 24) + (addr->bus << 16) + (addr->devid << 8) + addr->function; - uint64_t dev_addr2 = (addr2->domain << 24) + (addr2->bus << 16) + (addr2->devid << 8) + addr2->function; - - if (dev_addr > dev_addr2) - return 1; - else - return 0; -} - - /* Scan one pci sysfs entry, and fill the devices list from it. */ static int pci_scan_one(const char *dirname, uint16_t domain, uint8_t bus, @@ -358,13 +345,24 @@ pci_scan_one(const char *dirname, uint16_t domain, uint8_t bus, } else { struct rte_pci_device *dev2 = NULL; + int ret; TAILQ_FOREACH(dev2, &pci_device_list, next) { - if (pci_addr_comparison(&dev->addr, &dev2->addr)) + ret = rte_eal_compare_pci_addr(&dev->addr, &dev2->addr); + if (ret > 0) continue; - else { + else if (ret < 0) { TAILQ_INSERT_BEFORE(dev2, dev, next); return 0; + } else { /* already registered */ + /* update pt_driver */ + dev2->pt_driver = dev->pt_driver; + dev2->max_vfs = dev->max_vfs; + memmove(dev2->mem_resource, + dev->mem_resource, + sizeof(dev->mem_resource)); + free(dev); + return 0; } } TAILQ_INSERT_TAIL(&pci_device_list, dev, next); diff --git a/lib/librte_eal/linuxapp/eal/eal_pci_uio.c b/lib/librte_eal/linuxapp/eal/eal_pci_uio.c index 2b16fcb..f7acc55 100644 --- a/lib/librte_eal/linuxapp/eal/eal_pci_uio.c +++ b/lib/librte_eal/linuxapp/eal/eal_pci_uio.c @@ -126,7 +126,7 @@ pci_uio_map_secondary(struct rte_pci_device *dev) TAILQ_FOREACH(uio_res, pci_res_list, next) { /* skip this element if it doesn't match our PCI address */ - if (memcmp(&uio_res->pci_addr, &dev->addr, sizeof(dev->addr))) + if (rte_eal_compare_pci_addr(&uio_res->pci_addr, &dev->addr)) continue; for (i = 0; i != uio_res->nb_maps; i++) { -- 1.9.1
This patch adds rte_eth_dev_release_port(). The function is used for changing an attached status of the device that has specified name. v9: - rte_eth_dev_free() is replaced by rte_eth_dev_release_port(). (Thanks to Thomas Monjalon) v6: - Use rte_eth_dev structure as the paramter of rte_eth_dev_free(). v4: - Add parameter checking. Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp> --- lib/librte_ether/rte_ethdev.c | 11 +++++++++++ lib/librte_ether/rte_ethdev.h | 12 ++++++++++++ 2 files changed, 23 insertions(+) diff --git a/lib/librte_ether/rte_ethdev.c b/lib/librte_ether/rte_ethdev.c index 0e1e5c9..8d271ae 100644 --- a/lib/librte_ether/rte_ethdev.c +++ b/lib/librte_ether/rte_ethdev.c @@ -255,6 +255,17 @@ rte_eth_dev_allocate(const char *name) return eth_dev; } +int +rte_eth_dev_release_port(struct rte_eth_dev *eth_dev) +{ + if (eth_dev == NULL) + return -EINVAL; + + eth_dev->attached = 0; + nb_ports--; + return 0; +} + static int rte_eth_dev_init(struct rte_pci_driver *pci_drv, struct rte_pci_device *pci_dev) diff --git a/lib/librte_ether/rte_ethdev.h b/lib/librte_ether/rte_ethdev.h index ef31bda..8016a51 100644 --- a/lib/librte_ether/rte_ethdev.h +++ b/lib/librte_ether/rte_ethdev.h @@ -1487,6 +1487,18 @@ extern uint8_t rte_eth_dev_count(void); */ struct rte_eth_dev *rte_eth_dev_allocate(const char *name); +/** + * Function for internal use by dummy drivers primarily, e.g. ring-based + * driver. + * Release the specified ethdev port. + * + * @param eth_dev + * The *eth_dev* pointer is the address of the *rte_eth_dev* structure. + * @return + * - 0 on success, negative on error + */ +int rte_eth_dev_release_port(struct rte_eth_dev *eth_dev); + struct eth_driver; /** * @internal -- 1.9.1
The patch adds function pointer to rte_pci_driver and eth_driver structure. These function pointers are used when ports are detached. Also, the patch adds rte_eth_dev_uninit(). So far, it's not called by anywhere, but it will be called when port hotplug function is implemented. v10: - Add size parameter to rte_eth_dev_create_unique_device_name(). (Thanks to Iremonger, Bernard) v9: - Change parameter of pci_devuninit_t and rte_eth_dev_uninit. - Remove code that initiaize callback of ethdev from rte_eth_dev_uninit(). - Add a function to create a unique device name. (Thanks to Thomas Monjalon) v6: - Fix rte_eth_dev_uninit() to handle a return value of uninit function of PMD. v4: - Add parameter checking. - Change function names. Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp> --- lib/librte_eal/common/include/rte_pci.h | 6 ++++ lib/librte_ether/rte_ethdev.c | 64 +++++++++++++++++++++++++++++++-- lib/librte_ether/rte_ethdev.h | 24 +++++++++++++ 3 files changed, 92 insertions(+), 2 deletions(-) diff --git a/lib/librte_eal/common/include/rte_pci.h b/lib/librte_eal/common/include/rte_pci.h index 6814e91..4ea57cb 100644 --- a/lib/librte_eal/common/include/rte_pci.h +++ b/lib/librte_eal/common/include/rte_pci.h @@ -192,12 +192,18 @@ struct rte_pci_driver; typedef int (pci_devinit_t)(struct rte_pci_driver *, struct rte_pci_device *); /** + * Uninitialisation function for the driver called during hotplugging. + */ +typedef int (pci_devuninit_t)(struct rte_pci_device *); + +/** * A structure describing a PCI driver. */ struct rte_pci_driver { TAILQ_ENTRY(rte_pci_driver) next; /**< Next in list. */ const char *name; /**< Driver name. */ pci_devinit_t *devinit; /**< Device init. function. */ + pci_devuninit_t *devuninit; /**< Device uninit function. */ struct rte_pci_id *id_table; /**< ID table, NULL terminated. */ uint32_t drv_flags; /**< Flags contolling handling of device. */ }; diff --git a/lib/librte_ether/rte_ethdev.c b/lib/librte_ether/rte_ethdev.c index 8d271ae..3d148e2 100644 --- a/lib/librte_ether/rte_ethdev.c +++ b/lib/librte_ether/rte_ethdev.c @@ -266,6 +266,24 @@ rte_eth_dev_release_port(struct rte_eth_dev *eth_dev) return 0; } +static inline int +rte_eth_dev_create_unique_device_name(char *name, size_t size, + struct rte_pci_device *pci_dev) +{ + int ret; + + if ((name == NULL) || (pci_dev == NULL)) + return -EINVAL; + + ret = snprintf(name, size, "%d:%d.%d", + pci_dev->addr.bus, pci_dev->addr.devid, + pci_dev->addr.function); + if (ret < 0) + return ret; + + return 0; +} + static int rte_eth_dev_init(struct rte_pci_driver *pci_drv, struct rte_pci_device *pci_dev) @@ -279,8 +297,8 @@ rte_eth_dev_init(struct rte_pci_driver *pci_drv, eth_drv = (struct eth_driver *)pci_drv; /* Create unique Ethernet device name using PCI address */ - snprintf(ethdev_name, RTE_ETH_NAME_MAX_LEN, "%d:%d.%d", - pci_dev->addr.bus, pci_dev->addr.devid, pci_dev->addr.function); + rte_eth_dev_create_unique_device_name(ethdev_name, + sizeof(ethdev_name), pci_dev); eth_dev = rte_eth_dev_allocate(ethdev_name); if (eth_dev == NULL) @@ -321,6 +339,47 @@ rte_eth_dev_init(struct rte_pci_driver *pci_drv, return diag; } +static int +rte_eth_dev_uninit(struct rte_pci_device *pci_dev) +{ + const struct eth_driver *eth_drv; + struct rte_eth_dev *eth_dev; + char ethdev_name[RTE_ETH_NAME_MAX_LEN]; + int ret; + + if (pci_dev == NULL) + return -EINVAL; + + /* Create unique Ethernet device name using PCI address */ + rte_eth_dev_create_unique_device_name(ethdev_name, + sizeof(ethdev_name), pci_dev); + + eth_dev = rte_eth_dev_allocated(ethdev_name); + if (eth_dev == NULL) + return -ENODEV; + + eth_drv = (const struct eth_driver *)pci_dev->driver; + + /* Invoke PMD device uninit function */ + if (*eth_drv->eth_dev_uninit) { + ret = (*eth_drv->eth_dev_uninit)(eth_drv, eth_dev); + if (ret) + return ret; + } + + /* free ether device */ + rte_eth_dev_release_port(eth_dev); + + if (rte_eal_process_type() == RTE_PROC_PRIMARY) + rte_free(eth_dev->data->dev_private); + + eth_dev->pci_dev = NULL; + eth_dev->driver = NULL; + eth_dev->data = NULL; + + return 0; +} + /** * Register an Ethernet [Poll Mode] driver. * @@ -339,6 +398,7 @@ void rte_eth_driver_register(struct eth_driver *eth_drv) { eth_drv->pci_drv.devinit = rte_eth_dev_init; + eth_drv->pci_drv.devuninit = rte_eth_dev_uninit; rte_eal_pci_register(ð_drv->pci_drv); } diff --git a/lib/librte_ether/rte_ethdev.h b/lib/librte_ether/rte_ethdev.h index 8016a51..6cba06d 100644 --- a/lib/librte_ether/rte_ethdev.h +++ b/lib/librte_ether/rte_ethdev.h @@ -1536,6 +1536,27 @@ typedef int (*eth_dev_init_t)(struct eth_driver *eth_drv, /** * @internal + * Finalization function of an Ethernet driver invoked for each matching + * Ethernet PCI device detected during the PCI closing phase. + * + * @param eth_drv + * The pointer to the [matching] Ethernet driver structure supplied by + * the PMD when it registered itself. + * @param eth_dev + * The *eth_dev* pointer is the address of the *rte_eth_dev* structure + * associated with the matching device and which have been [automatically] + * allocated in the *rte_eth_devices* array. + * @return + * - 0: Success, the device is properly finalized by the driver. + * In particular, the driver MUST free the *dev_ops* pointer + * of the *eth_dev* structure. + * - <0: Error code of the device initialization failure. + */ +typedef int (*eth_dev_uninit_t)(const struct eth_driver *eth_drv, + struct rte_eth_dev *eth_dev); + +/** + * @internal * The structure associated with a PMD Ethernet driver. * * Each Ethernet driver acts as a PCI driver and is represented by a generic @@ -1545,11 +1566,14 @@ typedef int (*eth_dev_init_t)(struct eth_driver *eth_drv, * * - The *eth_dev_init* function invoked for each matching PCI device. * + * - The *eth_dev_uninit* function invoked for each matching PCI device. + * * - The size of the private data to allocate for each matching device. */ struct eth_driver { struct rte_pci_driver pci_drv; /**< The PMD is also a PCI driver. */ eth_dev_init_t eth_dev_init; /**< Device init function. */ + eth_dev_uninit_t eth_dev_uninit; /**< Device uninit function. */ unsigned int dev_private_size; /**< Size of device private data. */ }; -- 1.9.1
The patch adds following functions. - rte_eth_dev_save() The function is used for saving current rte_eth_dev structures. - rte_eth_dev_get_changed_port() The function receives the rte_eth_dev structures, then compare these with current values to know which port is actually attached or detached. - rte_eth_dev_get_addr_by_port() The function returns a pci address of an ethdev specified by port identifier. - rte_eth_dev_get_port_by_addr() The function returns a port identifier of an ethdev specified by pci address. - rte_eth_dev_get_name_by_port() The function returns a unique identifier name of an ethdev specified by port identifier. - Add rte_eth_dev_is_detachable() The function returns whether a PMD supports detach function. Also, the patch changes scope of rte_eth_dev_allocated() to global. This function will be called by virtual PMDs to support port hotplug. So change scope of the function to global. v10: - Change order of version.map. (Thanks to Thomas Monjalon) v9: - rte_eth_dev_check_detachable() is replaced by rte_eth_dev_is_detachable(). - strncpy() is replaced by strcpy(). (Thanks to Thomas Monjalon) - Add missing symbol in version map. (Thanks to Nail Horman) v8: - Add size parameter to rte_eth_dev_save(). - Add missing symbol in version map. (Thanks to Qiu, Michael and Iremonger, Bernard) v7: - Add pt_driver checking to rte_eth_dev_check_detachable(). (Thanks to Qiu, Michael) v5: - Fix return value of below functions. rte_eth_dev_get_changed_port(). rte_eth_dev_get_port_by_addr(). v4: - Add parameter checking. v3: - Fix if-condition bug while comparing pci addresses. - Add error checking codes. Reported-by: Mark Enright <menrigh@brocade.com> Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp> --- lib/librte_ether/rte_ethdev.c | 103 ++++++++++++++++++++++++++++++++- lib/librte_ether/rte_ethdev.h | 83 ++++++++++++++++++++++++++ lib/librte_ether/rte_ether_version.map | 7 +++ 3 files changed, 192 insertions(+), 1 deletion(-) diff --git a/lib/librte_ether/rte_ethdev.c b/lib/librte_ether/rte_ethdev.c index 3d148e2..7067620 100644 --- a/lib/librte_ether/rte_ethdev.c +++ b/lib/librte_ether/rte_ethdev.c @@ -201,7 +201,7 @@ rte_eth_dev_data_alloc(void) RTE_MAX_ETHPORTS * sizeof(*rte_eth_dev_data)); } -static struct rte_eth_dev * +struct rte_eth_dev * rte_eth_dev_allocated(const char *name) { unsigned i; @@ -426,6 +426,107 @@ rte_eth_dev_count(void) return (nb_ports); } +int +rte_eth_dev_save(struct rte_eth_dev *devs, size_t size) +{ + if ((devs == NULL) || + (size != sizeof(struct rte_eth_dev) * RTE_MAX_ETHPORTS)) + return -EINVAL; + + /* save current rte_eth_devices */ + memcpy(devs, rte_eth_devices, size); + return 0; +} + +int +rte_eth_dev_get_changed_port(struct rte_eth_dev *devs, uint8_t *port_id) +{ + if ((devs == NULL) || (port_id == NULL)) + return -EINVAL; + + /* check which port was attached or detached */ + for (*port_id = 0; *port_id < RTE_MAX_ETHPORTS; (*port_id)++, devs++) { + if (rte_eth_devices[*port_id].attached ^ devs->attached) + return 0; + } + return -ENODEV; +} + +int +rte_eth_dev_get_addr_by_port(uint8_t port_id, struct rte_pci_addr *addr) +{ + if (!rte_eth_dev_is_valid_port(port_id)) { + PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + return -EINVAL; + } + + if (addr == NULL) { + PMD_DEBUG_TRACE("Null pointer is specified\n"); + return -EINVAL; + } + + *addr = rte_eth_devices[port_id].pci_dev->addr; + return 0; +} + +int +rte_eth_dev_get_port_by_addr(struct rte_pci_addr *addr, uint8_t *port_id) +{ + struct rte_pci_addr *tmp; + + if ((addr == NULL) || (port_id == NULL)) { + PMD_DEBUG_TRACE("Null pointer is specified\n"); + return -EINVAL; + } + + for (*port_id = 0; *port_id < RTE_MAX_ETHPORTS; (*port_id)++) { + if (!rte_eth_devices[*port_id].attached) + continue; + if (!rte_eth_devices[*port_id].pci_dev) + continue; + tmp = &rte_eth_devices[*port_id].pci_dev->addr; + if (rte_eal_compare_pci_addr(tmp, addr) == 0) + return 0; + } + return -ENODEV; +} + +int +rte_eth_dev_get_name_by_port(uint8_t port_id, char *name) +{ + char *tmp; + + if (!rte_eth_dev_is_valid_port(port_id)) { + PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + return -EINVAL; + } + + if (name == NULL) { + PMD_DEBUG_TRACE("Null pointer is specified\n"); + return -EINVAL; + } + + /* shouldn't check 'rte_eth_devices[i].data', + * because it might be overwritten by VDEV PMD */ + tmp = rte_eth_dev_data[port_id].name; + strcpy(name, tmp); + return 0; +} + +int +rte_eth_dev_is_detachable(uint8_t port_id) +{ + uint32_t drv_flags; + + if (port_id >= RTE_MAX_ETHPORTS) { + PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + return -EINVAL; + } + + drv_flags = rte_eth_devices[port_id].driver->pci_drv.drv_flags; + return !(drv_flags & RTE_PCI_DRV_DETACHABLE); +} + static int rte_eth_dev_rx_queue_config(struct rte_eth_dev *dev, uint16_t nb_queues) { diff --git a/lib/librte_ether/rte_ethdev.h b/lib/librte_ether/rte_ethdev.h index 6cba06d..d4cfafb 100644 --- a/lib/librte_ether/rte_ethdev.h +++ b/lib/librte_ether/rte_ethdev.h @@ -1476,6 +1476,89 @@ extern struct rte_eth_dev rte_eth_devices[]; extern uint8_t rte_eth_dev_count(void); /** + * Function for internal use by port hotplug functions. + * Copies current ethdev structures to the specified pointer. + * + * @param devs The pointer to the ethdev structures + * @param size The size of ethdev structures + * @return + * - 0 on success, negative on error + */ +extern int rte_eth_dev_save(struct rte_eth_dev *devs, size_t size); + +/** + * Function for internal use by port hotplug functions. + * Compare the specified ethdev structures with currents. Then + * if there is a port which status is changed, fill the specified pointer + * with the port id of that port. + * @param devs The pointer to the ethdev structures + * @param port_id The pointer to the port id + * @return + * - 0 on success, negative on error + */ +extern int rte_eth_dev_get_changed_port( + struct rte_eth_dev *devs, uint8_t *port_id); + +/** + * Function for internal use by port hotplug functions. + * Returns a pci address of a ethdev specified by port identifier. + * @param port_id + * The port identifier of the Ethernet device + * @param addr + * The pointer to the pci address + * @return + * - 0 on success, negative on error + */ +extern int rte_eth_dev_get_addr_by_port( + uint8_t port_id, struct rte_pci_addr *addr); + +/** + * Function for internal use by port hotplug functions. + * Returns a port identifier of a ethdev specified by pci address. + * @param addr + * The pointer to the pci address of the Ethernet device. + * @param port_id + * The pointer to the port identifier + * @return + * - 0 on success, negative on error + */ +extern int rte_eth_dev_get_port_by_addr( + struct rte_pci_addr *addr, uint8_t *port_id); + +/** + * Function for internal use by port hotplug functions. + * Returns a unique identifier name of a ethdev specified by port identifier. + * @param port_id + * The port identifier. + * @param name + * The pointer to the Unique identifier name for each Ethernet device + * @return + * - 0 on success, negative on error + */ +extern int rte_eth_dev_get_name_by_port(uint8_t port_id, char *name); + +/** + * Function for internal use by port hotplug functions. + * Check whether or not, a PMD that is handling the ethdev specified by port + * identifier can support detach function. + * @param port_id + * The port identifier + * @return + * - 0 on supporting detach function, negative on not supporting + */ +extern int rte_eth_dev_is_detachable(uint8_t port_id); + +/** + * Function for internal use by port hotplug functions. + * Returns a ethdev slot specified by the unique identifier name. + * @param name + * The pointer to the Unique identifier name for each Ethernet device + * @return + * - The pointer to the ethdev slot, on success. NULL on error + */ +extern struct rte_eth_dev *rte_eth_dev_allocated(const char *name); + +/** * Function for internal use by dummy drivers primarily, e.g. ring-based * driver. * Allocates a new ethdev slot for an ethernet device and returns the pointer diff --git a/lib/librte_ether/rte_ether_version.map b/lib/librte_ether/rte_ether_version.map index f66fd2d..099c769 100644 --- a/lib/librte_ether/rte_ether_version.map +++ b/lib/librte_ether/rte_ether_version.map @@ -6,6 +6,7 @@ DPDK_2.0 { rte_eth_allmulticast_enable; rte_eth_allmulticast_get; rte_eth_dev_allocate; + rte_eth_dev_allocated; rte_eth_dev_bypass_event_show; rte_eth_dev_bypass_event_store; rte_eth_dev_bypass_init; @@ -32,9 +33,14 @@ DPDK_2.0 { rte_eth_dev_filter_supported; rte_eth_dev_flow_ctrl_get; rte_eth_dev_flow_ctrl_set; + rte_eth_dev_get_addr_by_port; + rte_eth_dev_get_changed_port; rte_eth_dev_get_mtu; + rte_eth_dev_get_name_by_port; + rte_eth_dev_get_port_by_addr; rte_eth_dev_get_vlan_offload; rte_eth_dev_info_get; + rte_eth_dev_is_detachable; rte_eth_dev_mac_addr_add; rte_eth_dev_mac_addr_remove; rte_eth_dev_priority_flow_ctrl_set; @@ -44,6 +50,7 @@ DPDK_2.0 { rte_eth_dev_rss_reta_update; rte_eth_dev_rx_queue_start; rte_eth_dev_rx_queue_stop; + rte_eth_dev_save; rte_eth_dev_set_link_down; rte_eth_dev_set_link_up; rte_eth_dev_set_mtu; -- 1.9.1
The patch adds functions for unmapping igb_uio resources. The patch is only for Linux and igb_uio environment. VFIO and BSD are not supported. v9: - Remove "rte_dev_hotplug.h". - Remove needless "#ifdef". (Thanks to Thomas Monjalon and Neil Horman) - Remove pci_unmap_device(). It will be implemented in later patch. v8: - Fix typo. (Thanks to Iremonger, Bernard) v5: - Fix pci_unmap_device() to check pt_driver. v4: - Add parameter checking. - Add header file to determine if hotplug can be enabled. Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp> --- lib/librte_eal/linuxapp/eal/eal_pci.c | 17 ++++++++ lib/librte_eal/linuxapp/eal/eal_pci_init.h | 7 ++++ lib/librte_eal/linuxapp/eal/eal_pci_uio.c | 65 ++++++++++++++++++++++++++++++ 3 files changed, 89 insertions(+) diff --git a/lib/librte_eal/linuxapp/eal/eal_pci.c b/lib/librte_eal/linuxapp/eal/eal_pci.c index e6cead1..17f32c0 100644 --- a/lib/librte_eal/linuxapp/eal/eal_pci.c +++ b/lib/librte_eal/linuxapp/eal/eal_pci.c @@ -167,6 +167,23 @@ pci_map_resource(void *requested_addr, int fd, off_t offset, size_t size) return mapaddr; } +/* unmap a particular resource */ +void +pci_unmap_resource(void *requested_addr, size_t size) +{ + if (requested_addr == NULL) + return; + + /* Unmap the PCI memory resource of device */ + if (munmap(requested_addr, size)) { + RTE_LOG(ERR, EAL, "%s(): cannot munmap(%p, 0x%lx): %s\n", + __func__, requested_addr, (unsigned long)size, + strerror(errno)); + } else + RTE_LOG(DEBUG, EAL, " PCI memory unmapped at %p\n", + requested_addr); +} + /* parse the "resource" sysfs file */ static int pci_parse_sysfs_resource(const char *filename, struct rte_pci_device *dev) diff --git a/lib/librte_eal/linuxapp/eal/eal_pci_init.h b/lib/librte_eal/linuxapp/eal/eal_pci_init.h index 1070eb8..e2dd8a5 100644 --- a/lib/librte_eal/linuxapp/eal/eal_pci_init.h +++ b/lib/librte_eal/linuxapp/eal/eal_pci_init.h @@ -71,6 +71,13 @@ void *pci_map_resource(void *requested_addr, int fd, off_t offset, /* map IGB_UIO resource prototype */ int pci_uio_map_resource(struct rte_pci_device *dev); +void pci_unmap_resource(void *requested_addr, size_t size); + +#ifdef RTE_LIBRTE_EAL_HOTPLUG +/* unmap IGB_UIO resource prototype */ +void pci_uio_unmap_resource(struct rte_pci_device *dev); +#endif /* RTE_LIBRTE_EAL_HOTPLUG */ + #ifdef VFIO_PRESENT #define VFIO_MAX_GROUPS 64 diff --git a/lib/librte_eal/linuxapp/eal/eal_pci_uio.c b/lib/librte_eal/linuxapp/eal/eal_pci_uio.c index f7acc55..ff4d0e8 100644 --- a/lib/librte_eal/linuxapp/eal/eal_pci_uio.c +++ b/lib/librte_eal/linuxapp/eal/eal_pci_uio.c @@ -433,3 +433,68 @@ pci_uio_map_resource(struct rte_pci_device *dev) return 0; } + +#ifdef RTE_LIBRTE_EAL_HOTPLUG +static void +pci_uio_unmap(struct mapped_pci_resource *uio_res) +{ + int i; + + if (uio_res == NULL) + return; + + for (i = 0; i != uio_res->nb_maps; i++) + pci_unmap_resource(uio_res->maps[i].addr, + (size_t)uio_res->maps[i].size); +} + +static struct mapped_pci_resource * +pci_uio_find_resource(struct rte_pci_device *dev) +{ + struct mapped_pci_resource *uio_res; + + if (dev == NULL) + return NULL; + + TAILQ_FOREACH(uio_res, pci_res_list, next) { + + /* skip this element if it doesn't match our PCI address */ + if (!rte_eal_compare_pci_addr(&uio_res->pci_addr, &dev->addr)) + return uio_res; + } + return NULL; +} + +/* unmap the PCI resource of a PCI device in virtual memory */ +void +pci_uio_unmap_resource(struct rte_pci_device *dev) +{ + struct mapped_pci_resource *uio_res; + + if (dev == NULL) + return; + + /* find an entry for the device */ + uio_res = pci_uio_find_resource(dev); + if (uio_res == NULL) + return; + + /* secondary processes - just free maps */ + if (rte_eal_process_type() != RTE_PROC_PRIMARY) + return pci_uio_unmap(uio_res); + + TAILQ_REMOVE(pci_res_list, uio_res, next); + + /* unmap all resources */ + pci_uio_unmap(uio_res); + + /* free uio resource */ + rte_free(uio_res); + + /* close fd if in primary process */ + close(dev->intr_handle.fd); + + dev->intr_handle.fd = -1; + dev->intr_handle.type = RTE_INTR_HANDLE_UNKNOWN; +} +#endif /* RTE_LIBRTE_EAL_HOTPLUG */ -- 1.9.1
- Add pci_close_all_drivers() The function tries to find a driver for the specified device, and then close the driver. - Add rte_eal_pci_probe_one() and rte_eal_pci_close_one() The functions are used for probe and close a device. First the function tries to find a device that has the specified PCI address. Then, probe or close the device. v9: - Fix commit title. - Remove RTE_EAL_INVOKE_TYPE_PROBE/CLOSE. (Thanks to Thomas Monjalon) - Implement pci_unmap_device() in this patch. v5: - Remove RTE_EAL_INVOKE_TYPE_UNKNOWN, because it's unused. v4: - Fix parameter checking. - Fix indent of 'if' statement. Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp> --- lib/librte_eal/common/eal_common_pci.c | 98 ++++++++++++++++++++++++++++++++- lib/librte_eal/common/eal_private.h | 15 +++++ lib/librte_eal/common/include/rte_pci.h | 32 +++++++++++ lib/librte_eal/linuxapp/eal/eal_pci.c | 94 +++++++++++++++++++++++++++++++ 4 files changed, 238 insertions(+), 1 deletion(-) diff --git a/lib/librte_eal/common/eal_common_pci.c b/lib/librte_eal/common/eal_common_pci.c index bf2793f..5b6b55d 100644 --- a/lib/librte_eal/common/eal_common_pci.c +++ b/lib/librte_eal/common/eal_common_pci.c @@ -108,7 +108,10 @@ static int pci_probe_all_drivers(struct rte_pci_device *dev) { struct rte_pci_driver *dr = NULL; - int rc; + int rc = 0; + + if (dev == NULL) + return -1; TAILQ_FOREACH(dr, &pci_driver_list, next) { rc = rte_eal_pci_probe_one_driver(dr, dev); @@ -123,6 +126,99 @@ pci_probe_all_drivers(struct rte_pci_device *dev) return 1; } +#ifdef RTE_LIBRTE_EAL_HOTPLUG +/* + * If vendor/device ID match, call the devuninit() function of all + * registered driver for the given device. Return -1 if initialization + * failed, return 1 if no driver is found for this device. + */ +static int +pci_close_all_drivers(struct rte_pci_device *dev) +{ + struct rte_pci_driver *dr = NULL; + int rc = 0; + + if (dev == NULL) + return -1; + + TAILQ_FOREACH(dr, &pci_driver_list, next) { + rc = rte_eal_pci_close_one_driver(dr, dev); + if (rc < 0) + /* negative value is an error */ + return -1; + if (rc > 0) + /* positive value means driver not found */ + continue; + return 0; + } + return 1; +} + +/* + * Find the pci device specified by pci address, then invoke probe function of + * the driver of the devive. + */ +int +rte_eal_pci_probe_one(struct rte_pci_addr *addr) +{ + struct rte_pci_device *dev = NULL; + int ret = 0; + + if (addr == NULL) + return -1; + + TAILQ_FOREACH(dev, &pci_device_list, next) { + if (rte_eal_compare_pci_addr(&dev->addr, addr)) + continue; + + ret = pci_probe_all_drivers(dev); + if (ret < 0) + goto err_return; + return 0; + } + return -1; + +err_return: + RTE_LOG(WARNING, EAL, "Requested device " PCI_PRI_FMT + " cannot be used\n", dev->addr.domain, dev->addr.bus, + dev->addr.devid, dev->addr.function); + return -1; +} + +/* + * Find the pci device specified by pci address, then invoke close function of + * the driver of the devive. + */ +int +rte_eal_pci_close_one(struct rte_pci_addr *addr) +{ + struct rte_pci_device *dev = NULL; + int ret = 0; + + if (addr == NULL) + return -1; + + TAILQ_FOREACH(dev, &pci_device_list, next) { + if (rte_eal_compare_pci_addr(&dev->addr, addr)) + continue; + + ret = pci_close_all_drivers(dev); + if (ret < 0) + goto err_return; + + TAILQ_REMOVE(&pci_device_list, dev, next); + return 0; + } + return -1; + +err_return: + RTE_LOG(WARNING, EAL, "Requested device " PCI_PRI_FMT + " cannot be used\n", dev->addr.domain, dev->addr.bus, + dev->addr.devid, dev->addr.function); + return -1; +} +#endif /* RTE_LIBRTE_EAL_HOTPLUG */ + /* * Scan the content of the PCI bus, and call the devinit() function for * all registered drivers that have a matching entry in its id_table diff --git a/lib/librte_eal/common/eal_private.h b/lib/librte_eal/common/eal_private.h index 159cd66..4acf5a0 100644 --- a/lib/librte_eal/common/eal_private.h +++ b/lib/librte_eal/common/eal_private.h @@ -165,6 +165,21 @@ int rte_eal_pci_probe_one_driver(struct rte_pci_driver *dr, struct rte_pci_device *dev); /** + * Munmap memory for single PCI device + * + * This function is private to EAL. + * + * @param dr + * The pointer to the pci driver structure + * @param dev + * The pointer to the pci device structure + * @return + * 0 on success, negative on error + */ +int rte_eal_pci_close_one_driver(struct rte_pci_driver *dr, + struct rte_pci_device *dev); + +/** * Init tail queues for non-EAL library structures. This is to allow * the rings, mempools, etc. lists to be shared among multiple processes * diff --git a/lib/librte_eal/common/include/rte_pci.h b/lib/librte_eal/common/include/rte_pci.h index 4ea57cb..49f2e7f 100644 --- a/lib/librte_eal/common/include/rte_pci.h +++ b/lib/librte_eal/common/include/rte_pci.h @@ -325,6 +325,38 @@ rte_eal_compare_pci_addr(struct rte_pci_addr *addr, struct rte_pci_addr *addr2) */ int rte_eal_pci_probe(void); +#ifdef RTE_LIBRTE_EAL_HOTPLUG +/** + * Probe the single PCI device. + * + * Scan the content of the PCI bus, and find the pci device specified by pci + * address, then call the probe() function for registered driver that has a + * matching entry in its id_table for discovered device. + * + * @param addr + * The PCI Bus-Device-Function address to probe. + * @return + * - 0 on success. + * - Negative on error. + */ +int rte_eal_pci_probe_one(struct rte_pci_addr *addr); + +/** + * Close the single PCI device. + * + * Scan the content of the PCI bus, and find the pci device specified by pci + * address, then call the close() function for registered driver that has a + * matching entry in its id_table for discovered device. + * + * @param addr + * The PCI Bus-Device-Function address to close. + * @return + * - 0 on success. + * - Negative on error. + */ +int rte_eal_pci_close_one(struct rte_pci_addr *addr); +#endif /* RTE_LIBRTE_EAL_HOTPLUG */ + /** * Dump the content of the PCI bus. * diff --git a/lib/librte_eal/linuxapp/eal/eal_pci.c b/lib/librte_eal/linuxapp/eal/eal_pci.c index 17f32c0..85d9d09 100644 --- a/lib/librte_eal/linuxapp/eal/eal_pci.c +++ b/lib/librte_eal/linuxapp/eal/eal_pci.c @@ -594,6 +594,31 @@ pci_map_device(struct rte_pci_device *dev) return ret; } +#ifdef RTE_LIBRTE_EAL_HOTPLUG +static void +pci_unmap_device(struct rte_pci_device *dev) +{ + if (dev == NULL) + return; + + /* try unmapping the NIC resources using VFIO if it exists */ + switch (dev->pt_driver) { + case RTE_PT_VFIO: + RTE_LOG(ERR, EAL, "Hotplug doesn't support vfio yet\n"); + break; + case RTE_PT_IGB_UIO: + case RTE_PT_UIO_GENERIC: + /* unmap resources for devices that use uio */ + pci_uio_unmap_resource(dev); + break; + default: + RTE_LOG(DEBUG, EAL, " Not managed by known pt driver," + " skipped\n"); + break; + } +} +#endif /* RTE_LIBRTE_EAL_HOTPLUG */ + /* * If vendor/device ID match, call the devinit() function of the * driver. @@ -665,6 +690,75 @@ rte_eal_pci_probe_one_driver(struct rte_pci_driver *dr, struct rte_pci_device *d return 1; } +#ifdef RTE_LIBRTE_EAL_HOTPLUG +/* + * If vendor/device ID match, call the devuninit() function of the + * driver. + */ +int +rte_eal_pci_close_one_driver(struct rte_pci_driver *dr, + struct rte_pci_device *dev) +{ + struct rte_pci_id *id_table; + + if ((dr == NULL) || (dev == NULL)) + return -EINVAL; + + for (id_table = dr->id_table ; id_table->vendor_id != 0; id_table++) { + + /* check if device's identifiers match the driver's ones */ + if (id_table->vendor_id != dev->id.vendor_id && + id_table->vendor_id != PCI_ANY_ID) + continue; + if (id_table->device_id != dev->id.device_id && + id_table->device_id != PCI_ANY_ID) + continue; + if (id_table->subsystem_vendor_id != + dev->id.subsystem_vendor_id && + id_table->subsystem_vendor_id != PCI_ANY_ID) + continue; + if (id_table->subsystem_device_id != + dev->id.subsystem_device_id && + id_table->subsystem_device_id != PCI_ANY_ID) + continue; + + struct rte_pci_addr *loc = &dev->addr; + + RTE_LOG(DEBUG, EAL, + "PCI device "PCI_PRI_FMT" on NUMA socket %i\n", + loc->domain, loc->bus, loc->devid, + loc->function, dev->numa_node); + + RTE_LOG(DEBUG, EAL, " remove driver: %x:%x %s\n", + dev->id.vendor_id, dev->id.device_id, + dr->name); + + /* call the driver devuninit() function */ + if (dr->devuninit && (dr->devuninit(dev) < 0)) + return -1; /* negative value is an error */ + + /* clear driver structure */ + dev->driver = NULL; + + if (dr->drv_flags & RTE_PCI_DRV_NEED_MAPPING) + /* unmap resources for devices that use igb_uio */ + pci_unmap_device(dev); + + return 0; + } + /* return positive value if driver is not found */ + return 1; +} +#else /* RTE_LIBRTE_EAL_HOTPLUG */ +int +rte_eal_pci_close_one_driver(struct rte_pci_driver *dr __rte_unused, + struct rte_pci_device *dev __rte_unused) +{ + RTE_LOG(ERR, EAL, "Hotplug support isn't enabled\n"); + return -1; +} +#endif /* RTE_LIBRTE_EAL_HOTPLUG */ + /* Init the PCI EAL subsystem */ int rte_eal_pci_init(void) -- 1.9.1
This new parameter is needed to keep device type like PCI or virtual. Port detaching processes are different between PCI device and virtual device. RTE_ETH_DEV_PCI indicates device type is PCI. RTE_ETH_DEV_VIRTUAL indicates device is virtual. v10: - Change order of version.map. (Thanks to Thomas Monjalon) - Fix comment of "rte_ethdev.h". (Thanks to Thomas Monjalon) v9: - Fix commit log. - RTE_ETH_DEV_PHYSICAL is replaced by RTE_ETH_DEV_PCI. (Thanks to Thomas Monjalon) v8: - NONE_TRACE is replaced by NO_TRACE. - Add missing symbol in version map. (Thanks to Qiu, Michael and Iremonger, Bernard) v4: - Fix comments of rte_eth_dev_type. Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp> --- app/test/virtual_pmd.c | 2 +- lib/librte_ether/rte_ethdev.c | 25 +++++++++++++++++++++++-- lib/librte_ether/rte_ethdev.h | 25 ++++++++++++++++++++++++- lib/librte_pmd_af_packet/rte_eth_af_packet.c | 2 +- lib/librte_pmd_bond/rte_eth_bond_api.c | 2 +- lib/librte_pmd_pcap/rte_eth_pcap.c | 2 +- lib/librte_pmd_ring/rte_eth_ring.c | 2 +- lib/librte_pmd_xenvirt/rte_eth_xenvirt.c | 2 +- 8 files changed, 53 insertions(+), 9 deletions(-) diff --git a/app/test/virtual_pmd.c b/app/test/virtual_pmd.c index cd9faf3..01a3913 100644 --- a/app/test/virtual_pmd.c +++ b/app/test/virtual_pmd.c @@ -580,7 +580,7 @@ virtual_ethdev_create(const char *name, struct ether_addr *mac_addr, goto err; /* reserve an ethdev entry */ - eth_dev = rte_eth_dev_allocate(name); + eth_dev = rte_eth_dev_allocate(name, RTE_ETH_DEV_PCI); if (eth_dev == NULL) goto err; diff --git a/lib/librte_ether/rte_ethdev.c b/lib/librte_ether/rte_ethdev.c index 7067620..f176f1e 100644 --- a/lib/librte_ether/rte_ethdev.c +++ b/lib/librte_ether/rte_ethdev.c @@ -227,7 +227,7 @@ rte_eth_dev_find_free_port(void) } struct rte_eth_dev * -rte_eth_dev_allocate(const char *name) +rte_eth_dev_allocate(const char *name, enum rte_eth_dev_type type) { uint8_t port_id; struct rte_eth_dev *eth_dev; @@ -251,6 +251,7 @@ rte_eth_dev_allocate(const char *name) snprintf(eth_dev->data->name, sizeof(eth_dev->data->name), "%s", name); eth_dev->data->port_id = port_id; eth_dev->attached = DEV_ATTACHED; + eth_dev->dev_type = type; nb_ports++; return eth_dev; } @@ -262,6 +263,7 @@ rte_eth_dev_release_port(struct rte_eth_dev *eth_dev) return -EINVAL; eth_dev->attached = 0; + eth_dev->dev_type = RTE_ETH_DEV_UNKNOWN; nb_ports--; return 0; } @@ -300,7 +302,7 @@ rte_eth_dev_init(struct rte_pci_driver *pci_drv, rte_eth_dev_create_unique_device_name(ethdev_name, sizeof(ethdev_name), pci_dev); - eth_dev = rte_eth_dev_allocate(ethdev_name); + eth_dev = rte_eth_dev_allocate(ethdev_name, RTE_ETH_DEV_PCI); if (eth_dev == NULL) return -ENOMEM; @@ -426,6 +428,14 @@ rte_eth_dev_count(void) return (nb_ports); } +enum rte_eth_dev_type +rte_eth_dev_get_device_type(uint8_t port_id) +{ + if (!rte_eth_dev_is_valid_port(port_id)) + return -1; + return rte_eth_devices[port_id].dev_type; +} + int rte_eth_dev_save(struct rte_eth_dev *devs, size_t size) { @@ -523,6 +533,17 @@ rte_eth_dev_is_detachable(uint8_t port_id) return -EINVAL; } + if (rte_eth_devices[port_id].dev_type == RTE_ETH_DEV_PCI) { + switch (rte_eth_devices[port_id].pci_dev->pt_driver) { + case RTE_PT_IGB_UIO: + case RTE_PT_UIO_GENERIC: + break; + case RTE_PT_VFIO: + default: + return -ENOTSUP; + } + } + drv_flags = rte_eth_devices[port_id].driver->pci_drv.drv_flags; return !(drv_flags & RTE_PCI_DRV_DETACHABLE); } diff --git a/lib/librte_ether/rte_ethdev.h b/lib/librte_ether/rte_ethdev.h index d4cfafb..1a978ed 100644 --- a/lib/librte_ether/rte_ethdev.h +++ b/lib/librte_ether/rte_ethdev.h @@ -1382,6 +1382,17 @@ struct eth_dev_ops { }; /** + * The eth device type + */ +enum rte_eth_dev_type { + RTE_ETH_DEV_UNKNOWN, /**< unknown device type */ + RTE_ETH_DEV_PCI, + /**< Physical function and Virtual function of PCI devices */ + RTE_ETH_DEV_VIRTUAL, /**< non hardware device */ + RTE_ETH_DEV_MAX /**< max value of this enum */ +}; + +/** * @internal * The generic data structure associated with each ethernet device. * @@ -1400,6 +1411,7 @@ struct rte_eth_dev { struct rte_pci_device *pci_dev; /**< PCI info. supplied by probing */ struct rte_eth_dev_cb_list callbacks; /**< User application callbacks */ uint8_t attached; /**< Flag indicating the port is attached */ + enum rte_eth_dev_type dev_type; /**< Flag indicating the device type */ }; struct rte_eth_dev_sriov { @@ -1477,6 +1489,15 @@ extern uint8_t rte_eth_dev_count(void); /** * Function for internal use by port hotplug functions. + * Get the device type to know whether the device is physical or virtual. + * @param port_id The port identifier of the Ethernet device + * @return + * - Device type. + */ +extern enum rte_eth_dev_type rte_eth_dev_get_device_type(uint8_t port_id); + +/** + * Function for internal use by port hotplug functions. * Copies current ethdev structures to the specified pointer. * * @param devs The pointer to the ethdev structures @@ -1565,10 +1586,12 @@ extern struct rte_eth_dev *rte_eth_dev_allocated(const char *name); * to that slot for the driver to use. * * @param name Unique identifier name for each Ethernet device + * @param type Device type of this Ethernet device * @return * - Slot in the rte_dev_devices array for a new device; */ -struct rte_eth_dev *rte_eth_dev_allocate(const char *name); +struct rte_eth_dev *rte_eth_dev_allocate(const char *name, + enum rte_eth_dev_type type); /** * Function for internal use by dummy drivers primarily, e.g. ring-based diff --git a/lib/librte_pmd_af_packet/rte_eth_af_packet.c b/lib/librte_pmd_af_packet/rte_eth_af_packet.c index 1ffe1cd..80e9bdf 100644 --- a/lib/librte_pmd_af_packet/rte_eth_af_packet.c +++ b/lib/librte_pmd_af_packet/rte_eth_af_packet.c @@ -649,7 +649,7 @@ rte_pmd_init_internals(const char *name, } /* reserve an ethdev entry */ - *eth_dev = rte_eth_dev_allocate(name); + *eth_dev = rte_eth_dev_allocate(name, RTE_ETH_DEV_VIRTUAL); if (*eth_dev == NULL) goto error; diff --git a/lib/librte_pmd_bond/rte_eth_bond_api.c b/lib/librte_pmd_bond/rte_eth_bond_api.c index b1d0bd3..f026c8d 100644 --- a/lib/librte_pmd_bond/rte_eth_bond_api.c +++ b/lib/librte_pmd_bond/rte_eth_bond_api.c @@ -257,7 +257,7 @@ rte_eth_bond_create(const char *name, uint8_t mode, uint8_t socket_id) } /* reserve an ethdev entry */ - eth_dev = rte_eth_dev_allocate(name); + eth_dev = rte_eth_dev_allocate(name, RTE_ETH_DEV_VIRTUAL); if (eth_dev == NULL) { RTE_BOND_LOG(ERR, "Unable to allocate rte_eth_dev"); goto err; diff --git a/lib/librte_pmd_pcap/rte_eth_pcap.c b/lib/librte_pmd_pcap/rte_eth_pcap.c index d299288..af7fae8 100644 --- a/lib/librte_pmd_pcap/rte_eth_pcap.c +++ b/lib/librte_pmd_pcap/rte_eth_pcap.c @@ -709,7 +709,7 @@ rte_pmd_init_internals(const char *name, const unsigned nb_rx_queues, goto error; /* reserve an ethdev entry */ - *eth_dev = rte_eth_dev_allocate(name); + *eth_dev = rte_eth_dev_allocate(name, RTE_ETH_DEV_VIRTUAL); if (*eth_dev == NULL) goto error; diff --git a/lib/librte_pmd_ring/rte_eth_ring.c b/lib/librte_pmd_ring/rte_eth_ring.c index a23e933..cd0913b 100644 --- a/lib/librte_pmd_ring/rte_eth_ring.c +++ b/lib/librte_pmd_ring/rte_eth_ring.c @@ -297,7 +297,7 @@ rte_eth_from_rings(const char *name, struct rte_ring *const rx_queues[], goto error; /* reserve an ethdev entry */ - eth_dev = rte_eth_dev_allocate(name); + eth_dev = rte_eth_dev_allocate(name, RTE_ETH_DEV_VIRTUAL); if (eth_dev == NULL) goto error; diff --git a/lib/librte_pmd_xenvirt/rte_eth_xenvirt.c b/lib/librte_pmd_xenvirt/rte_eth_xenvirt.c index 04e30c9..bc403d6 100644 --- a/lib/librte_pmd_xenvirt/rte_eth_xenvirt.c +++ b/lib/librte_pmd_xenvirt/rte_eth_xenvirt.c @@ -648,7 +648,7 @@ eth_dev_xenvirt_create(const char *name, const char *params, goto err; /* reserve an ethdev entry */ - eth_dev = rte_eth_dev_allocate(name); + eth_dev = rte_eth_dev_allocate(name, RTE_ETH_DEV_VIRTUAL); if (eth_dev == NULL) goto err; -- 1.9.1
These functions are used for attaching or detaching a port. When rte_eal_dev_attach() is called, the function tries to realize the device name as pci address. If this is done successfully, rte_eal_dev_attach() will attach physical device port. If not, attaches virtual devive port. When rte_eal_dev_detach() is called, the function gets the device type of this port to know whether the port is come from physical or virtual. And then specific detaching function will be called. v11: - Remove needless devargs handling codes. - Replace get_vdev_name() by rte_eal_parse_devargs_str(). - Replace rte_eal_vdev_find_and_init by rte_eal_vdev_init() - Replace rte_eal_vdev_find_and_uninit by rte_eal_vdev_uninit() - Fix rte_eal_dev_init() to use rte_eal_vdev_init(). (Thanks to Maxime Leroy) v10: - Add comments. - Change order of version.map. (Thanks to Thomas Monjalon) v9: - Fix comments. - Use strcmp() instead of strncmp(). - Remove RTE_EAL_INVOKE_TYPE_PROBE/CLOSE. - Change definition of rte_dev_uninit_t. (Thanks to Thomas Monjalon and Maxime Leroy) v8: - Add missing symbol in version map. (Thanks to Qiu, Michael and Iremonger, Bernard) v7: - Fix typo of warning messages. (Thanks to Qiu, Michael) v5: - Change function names like below. rte_eal_dev_find_and_invoke() to rte_eal_vdev_find_and_invoke(). rte_eal_dev_invoke() to rte_eal_vdev_invoke(). - Add code to handle a return value of rte_eal_devargs_remove(). - Fix pci address format in rte_eal_dev_detach(). v4: - Fix comment. - Add error checking. - Fix indent of 'if' statement. - Change function name. Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp> --- lib/librte_eal/common/eal_common_dev.c | 285 ++++++++++++++++++++++-- lib/librte_eal/common/eal_common_devargs.c | 46 ++-- lib/librte_eal/common/eal_private.h | 11 + lib/librte_eal/common/include/rte_dev.h | 33 +++ lib/librte_eal/common/include/rte_devargs.h | 28 +++ lib/librte_eal/linuxapp/eal/Makefile | 1 + lib/librte_eal/linuxapp/eal/eal_pci.c | 6 +- lib/librte_eal/linuxapp/eal/rte_eal_version.map | 2 + 8 files changed, 378 insertions(+), 34 deletions(-) diff --git a/lib/librte_eal/common/eal_common_dev.c b/lib/librte_eal/common/eal_common_dev.c index eae5656..7d4dce6 100644 --- a/lib/librte_eal/common/eal_common_dev.c +++ b/lib/librte_eal/common/eal_common_dev.c @@ -32,10 +32,13 @@ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ +#include <stdio.h> +#include <limits.h> #include <string.h> #include <inttypes.h> #include <sys/queue.h> +#include <rte_ethdev.h> #include <rte_dev.h> #include <rte_devargs.h> #include <rte_debug.h> @@ -61,6 +64,37 @@ rte_eal_driver_unregister(struct rte_driver *driver) TAILQ_REMOVE(&dev_driver_list, driver, next); } +static int +rte_eal_vdev_init(const char *name, const char *args) +{ + struct rte_driver *driver; + + if (name == NULL) + return -EINVAL; + + TAILQ_FOREACH(driver, &dev_driver_list, next) { + if (driver->type != PMD_VDEV) + continue; + + /* + * search a driver prefix in virtual device name. + * For example, if the driver is pcap PMD, driver->name + * will be "eth_pcap", but "name" will be "eth_pcapN". + * So use strncmp to compare. + */ + if (!strncmp(driver->name, name, strlen(driver->name))) { + driver->init(name, args); + break; + } + } + + if (driver == NULL) { + RTE_LOG(WARNING, EAL, "no driver found for %s\n", name); + return -EINVAL; + } + return 0; +} + int rte_eal_dev_init(void) { @@ -79,23 +113,10 @@ rte_eal_dev_init(void) if (devargs->type != RTE_DEVTYPE_VIRTUAL) continue; - TAILQ_FOREACH(driver, &dev_driver_list, next) { - if (driver->type != PMD_VDEV) - continue; - - /* search a driver prefix in virtual device name */ - if (!strncmp(driver->name, devargs->virtual.drv_name, - strlen(driver->name))) { - driver->init(devargs->virtual.drv_name, - devargs->args); - break; - } - } - - if (driver == NULL) { + if (rte_eal_vdev_init(devargs->virtual.drv_name, + devargs->args)) rte_panic("no driver found for %s\n", devargs->virtual.drv_name); - } } /* Once the vdevs are initalized, start calling all the pdev drivers */ @@ -107,3 +128,237 @@ rte_eal_dev_init(void) } return 0; } + +/* So far, DPDK hotplug function only supports linux */ +#ifdef RTE_LIBRTE_EAL_HOTPLUG +static int +rte_eal_vdev_uninit(const char *name) +{ + struct rte_driver *driver; + + if (name == NULL) + return -EINVAL; + + TAILQ_FOREACH(driver, &dev_driver_list, next) { + if (driver->type != PMD_VDEV) + continue; + + /* + * search a driver prefix in virtual device name. + * For example, if the driver is pcap PMD, driver->name + * will be "eth_pcap", but "name" will be "eth_pcapN". + * So use strncmp to compare. + */ + if (!strncmp(driver->name, name, strlen(driver->name))) { + driver->uninit(name); + break; + } + } + + if (driver == NULL) { + RTE_LOG(WARNING, EAL, "no driver found for %s\n", name); + return 1; + } + return 0; +} + +/* attach the new physical device, then store port_id of the device */ +static int +rte_eal_dev_attach_pdev(struct rte_pci_addr *addr, uint8_t *port_id) +{ + uint8_t new_port_id; + struct rte_eth_dev devs[RTE_MAX_ETHPORTS]; + + if ((addr == NULL) || (port_id == NULL)) + goto err; + + /* save current port status */ + if (rte_eth_dev_save(devs, sizeof(devs))) + goto err; + /* re-construct pci_device_list */ + if (rte_eal_pci_scan()) + goto err; + /* invoke probe func of the driver can handle the new device. + * TODO: + * rte_eal_pci_probe_one() should return port_id. + * And rte_eth_dev_save() and rte_eth_dev_get_changed_port() + * should be removed. */ + if (rte_eal_pci_probe_one(addr)) + goto err; + /* get port_id enabled by above procedures */ + if (rte_eth_dev_get_changed_port(devs, &new_port_id)) + goto err; + + *port_id = new_port_id; + return 0; +err: + RTE_LOG(ERR, EAL, "Driver, cannot attach the device\n"); + return -1; +} + +/* detach the new physical device, then store pci_addr of the device */ +static int +rte_eal_dev_detach_pdev(uint8_t port_id, struct rte_pci_addr *addr) +{ + struct rte_pci_addr freed_addr; + struct rte_pci_addr vp; + + if (addr == NULL) + goto err; + + /* check whether the driver supports detach feature, or not */ + if (rte_eth_dev_is_detachable(port_id)) + goto err; + + /* get pci address by port id */ + if (rte_eth_dev_get_addr_by_port(port_id, &freed_addr)) + goto err; + + /* Zerod pci addr means the port comes from virtual device */ + vp.domain = vp.bus = vp.devid = vp.function = 0; + if (rte_eal_compare_pci_addr(&vp, &freed_addr) == 0) + goto err; + + /* invoke close func of the driver, + * also remove the device from pci_device_list */ + if (rte_eal_pci_close_one(&freed_addr)) + goto err; + + *addr = freed_addr; + return 0; +err: + RTE_LOG(ERR, EAL, "Driver, cannot detach the device\n"); + return -1; +} + +/* attach the new virtual device, then store port_id of the device */ +static int +rte_eal_dev_attach_vdev(const char *vdevargs, uint8_t *port_id) +{ + char *name = NULL, *args = NULL; + uint8_t new_port_id; + struct rte_eth_dev devs[RTE_MAX_ETHPORTS]; + int ret = -1; + + if ((vdevargs == NULL) || (port_id == NULL)) + goto end; + + /* parse vdevargs, then retrieve device name and args */ + if (rte_eal_parse_devargs_str(vdevargs, &name, &args)) + goto end; + + /* save current port status */ + if (rte_eth_dev_save(devs, sizeof(devs))) + goto end; + /* walk around dev_driver_list to find the driver of the device, + * then invoke probe function o the driver. + * TODO: + * rte_eal_vdev_init() should return port_id, + * And rte_eth_dev_save() and rte_eth_dev_get_changed_port() + * should be removed. */ + if (rte_eal_vdev_init(name, args)) + goto end; + /* get port_id enabled by above procedures */ + if (rte_eth_dev_get_changed_port(devs, &new_port_id)) + goto end; + ret = 0; + +end: + if (name) + free(name); + if (args) + free(args); + + *port_id = new_port_id; + if (ret < 0) + RTE_LOG(ERR, EAL, "Driver, cannot attach the device\n"); + return ret; +} + +/* detach the new virtual device, then store the name of the device */ +static int +rte_eal_dev_detach_vdev(uint8_t port_id, char *vdevname) +{ + char name[RTE_ETH_NAME_MAX_LEN]; + + if (vdevname == NULL) + goto err; + + /* check whether the driver supports detach feature, or not */ + if (rte_eth_dev_is_detachable(port_id)) + goto err; + + /* get device name by port id */ + if (rte_eth_dev_get_name_by_port(port_id, name)) + goto err; + /* walk around dev_driver_list to find the driver of the device, + * then invoke close function o the driver */ + if (rte_eal_vdev_uninit(name)) + goto err; + + strncpy(vdevname, name, sizeof(name)); + return 0; +err: + RTE_LOG(ERR, EAL, "Driver, cannot detach the device\n"); + return -1; +} + +/* attach the new device, then store port_id of the device */ +int +rte_eal_dev_attach(const char *devargs, uint8_t *port_id) +{ + struct rte_pci_addr addr; + + if ((devargs == NULL) || (port_id == NULL)) + return -EINVAL; + + if (eal_parse_pci_DomBDF(devargs, &addr) == 0) + return rte_eal_dev_attach_pdev(&addr, port_id); + else + return rte_eal_dev_attach_vdev(devargs, port_id); +} + +/* detach the device, then store the name of the device */ +int +rte_eal_dev_detach(uint8_t port_id, char *name) +{ + struct rte_pci_addr addr; + int ret; + + if (name == NULL) + return -EINVAL; + + if (rte_eth_dev_get_device_type(port_id) == RTE_ETH_DEV_PCI) { + ret = rte_eth_dev_get_addr_by_port(port_id, &addr); + if (ret < 0) + return ret; + + ret = rte_eal_dev_detach_pdev(port_id, &addr); + if (ret == 0) + snprintf(name, RTE_ETH_NAME_MAX_LEN, + "%04x:%02x:%02x.%d", + addr.domain, addr.bus, + addr.devid, addr.function); + + return ret; + } else + return rte_eal_dev_detach_vdev(port_id, name); +} +#else /* RTE_LIBRTE_EAL_HOTPLUG */ +int +rte_eal_dev_attach(const char *devargs __rte_unused, + uint8_t *port_id __rte_unused) +{ + RTE_LOG(ERR, EAL, "Hotplug support isn't enabled\n"); + return -1; +} + +/* detach the device, then store the name of the device */ +int +rte_eal_dev_detach(uint8_t port_id __rte_unused, + char *name __rte_unused) +{ + RTE_LOG(ERR, EAL, "Hotplug support isn't enabled\n"); + return -1; +} +#endif /* RTE_LIBRTE_EAL_HOTPLUG */ diff --git a/lib/librte_eal/common/eal_common_devargs.c b/lib/librte_eal/common/eal_common_devargs.c index 3aace08..2988a70 100644 --- a/lib/librte_eal/common/eal_common_devargs.c +++ b/lib/librte_eal/common/eal_common_devargs.c @@ -44,13 +44,41 @@ struct rte_devargs_list devargs_list = TAILQ_HEAD_INITIALIZER(devargs_list); +int +rte_eal_parse_devargs_str(const char *devargs_str, + char **drvname, char **drvargs) +{ + char *sep; + + if ((devargs_str == NULL) || drvname == NULL || drvargs == NULL) + return -1; + + *drvname = strdup(devargs_str); + if (drvname == NULL) { + RTE_LOG(ERR, EAL, "cannot allocate temp memory for devargs\n"); + return -1; + } + + /* set the first ',' to '\0' to split name and arguments */ + sep = strchr(*drvname, ','); + if (sep != NULL) { + sep[0] = '\0'; + *drvargs = strdup(sep + 1); + if (*drvargs == NULL) { + RTE_LOG(ERR, EAL, "cannot allocate for devargs args\n"); + free(*drvname); + return -1; + } + } + return 0; +} + /* store a whitelist parameter for later parsing */ int rte_eal_devargs_add(enum rte_devtype devtype, const char *devargs_str) { struct rte_devargs *devargs = NULL; char *buf = NULL; - char *sep; int ret; /* use malloc instead of rte_malloc as it's called early at init */ @@ -62,22 +90,8 @@ rte_eal_devargs_add(enum rte_devtype devtype, const char *devargs_str) memset(devargs, 0, sizeof(*devargs)); devargs->type = devtype; - buf = strdup(devargs_str); - if (buf == NULL) { - RTE_LOG(ERR, EAL, "cannot allocate temp memory for devargs\n"); + if (rte_eal_parse_devargs_str(devargs_str, &buf, &devargs->args)) goto fail; - } - - /* set the first ',' to '\0' to split name and arguments */ - sep = strchr(buf, ','); - if (sep != NULL) { - sep[0] = '\0'; - devargs->args = strdup(sep + 1); - if (devargs->args == NULL) { - RTE_LOG(ERR, EAL, "cannot allocate for devargs args\n"); - goto fail; - } - } switch (devargs->type) { case RTE_DEVTYPE_WHITELISTED_PCI: diff --git a/lib/librte_eal/common/eal_private.h b/lib/librte_eal/common/eal_private.h index 4acf5a0..98b286a 100644 --- a/lib/librte_eal/common/eal_private.h +++ b/lib/librte_eal/common/eal_private.h @@ -154,6 +154,17 @@ struct rte_pci_driver; struct rte_pci_device; /** + * Scan the content of the PCI bus, and the devices in the devices + * list + * + * This function is private to EAL. + * + * @return + * 0 on success, negative on error + */ +int rte_eal_pci_scan(void); + +/** * Mmap memory for single PCI device * * This function is private to EAL. diff --git a/lib/librte_eal/common/include/rte_dev.h b/lib/librte_eal/common/include/rte_dev.h index f7e3a10..a5ac770 100644 --- a/lib/librte_eal/common/include/rte_dev.h +++ b/lib/librte_eal/common/include/rte_dev.h @@ -47,6 +47,7 @@ extern "C" { #endif #include <sys/queue.h> +#include <rte_pci.h> /** Double linked list of device drivers. */ TAILQ_HEAD(rte_driver_list, rte_driver); @@ -57,6 +58,11 @@ TAILQ_HEAD(rte_driver_list, rte_driver); typedef int (rte_dev_init_t)(const char *name, const char *args); /** + * Uninitilization function called for each device driver once. + */ +typedef int (rte_dev_uninit_t)(const char *name); + +/** * Driver type enumeration */ enum pmd_type { @@ -72,6 +78,7 @@ struct rte_driver { enum pmd_type type; /**< PMD Driver type */ const char *name; /**< Driver name. */ rte_dev_init_t *init; /**< Device init. function. */ + rte_dev_uninit_t *uninit; /**< Device uninit. function. */ }; /** @@ -93,6 +100,32 @@ void rte_eal_driver_register(struct rte_driver *driver); void rte_eal_driver_unregister(struct rte_driver *driver); /** + * Attach a new device. + * + * @param devargs + * A pointer to a strings array describing the new device + * to be attached. The strings should be a pci address like + * '0000:01:00.0' or virtual device name like 'eth_pcap0'. + * @param port_id + * A pointer to a port identifier actually attached. + * @return + * 0 on success and port_id is filled, negative on error + */ +int rte_eal_dev_attach(const char *devargs, uint8_t *port_id); + +/** + * Detach a device. + * + * @param port_id + * The port identifier of the device to detach. + * @param addr + * A pointer to a device name actually detached. + * @return + * 0 on success and devname is filled, negative on error + */ +int rte_eal_dev_detach(uint8_t port_id, char *devname); + +/** * Initalize all the registered drivers in this process */ int rte_eal_dev_init(void); diff --git a/lib/librte_eal/common/include/rte_devargs.h b/lib/librte_eal/common/include/rte_devargs.h index 996e180..274a92c 100644 --- a/lib/librte_eal/common/include/rte_devargs.h +++ b/lib/librte_eal/common/include/rte_devargs.h @@ -99,6 +99,34 @@ TAILQ_HEAD(rte_devargs_list, rte_devargs); extern struct rte_devargs_list devargs_list; /** + * Parse a devargs string. + * + * For PCI devices, the format of arguments string is "PCI_ADDR" or + * "PCI_ADDR,key=val,key2=val2,...". Examples: "08:00.1", "0000:5:00.0", + * "04:00.0,arg=val". + * + * For virtual devices, the format of arguments string is "DRIVER_NAME*" + * or "DRIVER_NAME*,key=val,key2=val2,...". Examples: "eth_ring", + * "eth_ring0", "eth_pmdAnything,arg=0:arg2=1". + * + * The function parses the arguments string to get driver name and driver + * arguments. + * + * @param devargs_str + * The arguments as given by the user. + * @param drvname + * The pointer to the string to store parsed driver name. + * @param drvargs + * The pointer to the string to store parsed driver arguments. + * + * @return + * - 0 on success + * - A negative value on error + */ +int rte_eal_parse_devargs_str(const char *devargs_str, + char **drvname, char **drvargs); + +/** * Add a device to the user device list * * For PCI devices, the format of arguments string is "PCI_ADDR" or diff --git a/lib/librte_eal/linuxapp/eal/Makefile b/lib/librte_eal/linuxapp/eal/Makefile index e117cec..b59b201 100644 --- a/lib/librte_eal/linuxapp/eal/Makefile +++ b/lib/librte_eal/linuxapp/eal/Makefile @@ -45,6 +45,7 @@ CFLAGS += -I$(RTE_SDK)/lib/librte_eal/common/include CFLAGS += -I$(RTE_SDK)/lib/librte_ring CFLAGS += -I$(RTE_SDK)/lib/librte_mempool CFLAGS += -I$(RTE_SDK)/lib/librte_malloc +CFLAGS += -I$(RTE_SDK)/lib/librte_mbuf CFLAGS += -I$(RTE_SDK)/lib/librte_ether CFLAGS += -I$(RTE_SDK)/lib/librte_ivshmem CFLAGS += -I$(RTE_SDK)/lib/librte_pmd_ring diff --git a/lib/librte_eal/linuxapp/eal/eal_pci.c b/lib/librte_eal/linuxapp/eal/eal_pci.c index 85d9d09..ab77b68 100644 --- a/lib/librte_eal/linuxapp/eal/eal_pci.c +++ b/lib/librte_eal/linuxapp/eal/eal_pci.c @@ -439,8 +439,8 @@ error: * Scan the content of the PCI bus, and the devices in the devices * list */ -static int -pci_scan(void) +int +rte_eal_pci_scan(void) { struct dirent *e; DIR *dir; @@ -772,7 +772,7 @@ rte_eal_pci_init(void) if (internal_config.no_pci) return 0; - if (pci_scan() < 0) { + if (rte_eal_pci_scan() < 0) { RTE_LOG(ERR, EAL, "%s(): Cannot scan PCI bus\n", __func__); return -1; } diff --git a/lib/librte_eal/linuxapp/eal/rte_eal_version.map b/lib/librte_eal/linuxapp/eal/rte_eal_version.map index 5ed6e4d..5d40e02 100644 --- a/lib/librte_eal/linuxapp/eal/rte_eal_version.map +++ b/lib/librte_eal/linuxapp/eal/rte_eal_version.map @@ -20,6 +20,8 @@ DPDK_2.0 { rte_dump_tailq; rte_eal_alarm_cancel; rte_eal_alarm_set; + rte_eal_dev_attach; + rte_eal_dev_detach; rte_eal_dev_init; rte_eal_devargs_add; rte_eal_devargs_dump; -- 1.9.1
This patch adds a new section for describing port hotplug framework. Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp> --- doc/guides/prog_guide/index.rst | 1 + doc/guides/prog_guide/port_hotplug_framework.rst | 110 +++++++++++++++++++++++ 2 files changed, 111 insertions(+) create mode 100644 doc/guides/prog_guide/port_hotplug_framework.rst diff --git a/doc/guides/prog_guide/index.rst b/doc/guides/prog_guide/index.rst index de69682..60a6ac5 100644 --- a/doc/guides/prog_guide/index.rst +++ b/doc/guides/prog_guide/index.rst @@ -71,6 +71,7 @@ Programmer's Guide packet_classif_access_ctrl packet_framework vhost_lib + port_hotplug_framework source_org dev_kit_build_system dev_kit_root_make_help diff --git a/doc/guides/prog_guide/port_hotplug_framework.rst b/doc/guides/prog_guide/port_hotplug_framework.rst new file mode 100644 index 0000000..355ae28 --- /dev/null +++ b/doc/guides/prog_guide/port_hotplug_framework.rst @@ -0,0 +1,110 @@ +.. BSD LICENSE + Copyright(c) 2015 IGEL Co.,Ltd. All rights reserved. + All rights reserved. + + Redistribution and use in source and binary forms, with or without + modification, are permitted provided that the following conditions + are met: + + * Redistributions of source code must retain the above copyright + notice, this list of conditions and the following disclaimer. + * Redistributions in binary form must reproduce the above copyright + notice, this list of conditions and the following disclaimer in + the documentation and/or other materials provided with the + distribution. + * Neither the name of IGEL Co.,Ltd. nor the names of its + contributors may be used to endorse or promote products derived + from this software without specific prior written permission. + + THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS + "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT + LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR + A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT + OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, + SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT + LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, + DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY + THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT + (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE + OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + +Port Hotplug Framework +====================== + +The Port Hotplug Framework provides DPDK applications with the ability to +attach and detach ports at runtime. Because the framework depends on PMD +implementation, the ports that PMDs cannot handle are out of scope of this +framework. Furthermore, after detaching a port from a DPDK application, the +framework doesn't provide a way for removing the devices from the system. +For the ports backed by a physical NIC, the kernel will need to support PCI +Hotplug feature. + +Overview +-------- + +The basic requirements of the Port Hotplug Framework are: + +* DPDK applications that use the Port Hotplug Framework must manage their + own ports. + + The Port Hotplug Framework is implemented to allow DPDK applications to + manage ports. For example, when DPDK applications call the port attach + function, the attached port number is returned. DPDK applications can + also detach the port by port number. + +* Kernel support is needed for attaching or detaching physical device + ports. + + To attach new physical device ports, the device will be recognized by + userspace driver I/O framework in kernel at first. Then DPDK + applications can call the Port Hotplug functions to attach the ports. + For detaching, steps are vice versa. + +* Before detaching, they must be stopped and closed. + + DPDK applications must call "rte_eth_dev_stop()" and + "rte_eth_dev_close()" APIs before detaching ports. These functions will + start finalization sequence of the PMDs. + +* The framework doesn't affect legacy DPDK applications behavior. + + If the Port Hotplug functions aren't called, all legacy DPDK apps can + still work without modifications. + +Port Hotplug API overview +------------------------- + +* Attaching a port + + "rte_eal_dev_attach()" API attaches a port to DPDK application, and + returns the attached port number. Before calling the API, the device + should be recognized by an userspace driver I/O framework. The API + receives a pci address like "0000:01:00.0" or a virtual device name + like "eth_pcap0,iface=eth0". In the case of virtual device name, the + format is the same as the general "--vdev" option of DPDK. + +* Detaching a port + + "rte_eal_dev_detach()" API detaches a port from DPDK application, and + returns a pci address of the detached device or a virtual device name + of the device. + +Reference +--------- + + "testpmd" supports the Port Hotplug Framework. + +Limitations +----------- + +* The Port Hotplug APIs are not thread safe. + +* The framework can only be enabled with Linux. BSD is not supported. + +* To detach a port, the port should be backed by a device that igb_uio + manages. VFIO is not supported. + +* Not all PMDs support detaching feature. + To know whether a PMD can support detaching, search for the + "RTE_PCI_DRV_DETACHABLE" flag in PMD implementation. If the flag is + defined in the PMD, detaching is supported. -- 1.9.1
This patch adds finalization code to free resources allocated by the PMD. v6: - Fix a paramter of rte_eth_dev_free(). v4: - Change function name. Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp> --- lib/librte_pmd_pcap/rte_eth_pcap.c | 40 ++++++++++++++++++++++++++++++++++++++ 1 file changed, 40 insertions(+) diff --git a/lib/librte_pmd_pcap/rte_eth_pcap.c b/lib/librte_pmd_pcap/rte_eth_pcap.c index af7fae8..5e94930 100644 --- a/lib/librte_pmd_pcap/rte_eth_pcap.c +++ b/lib/librte_pmd_pcap/rte_eth_pcap.c @@ -498,6 +498,13 @@ static struct eth_dev_ops ops = { .stats_reset = eth_stats_reset, }; +static struct eth_driver rte_pcap_pmd = { + .pci_drv = { + .name = "rte_pcap_pmd", + .drv_flags = RTE_PCI_DRV_DETACHABLE, + }, +}; + /* * Function handler that opens the pcap file for reading a stores a * reference of it for use it later on. @@ -713,6 +720,10 @@ rte_pmd_init_internals(const char *name, const unsigned nb_rx_queues, if (*eth_dev == NULL) goto error; + /* check length of device name */ + if ((strlen((*eth_dev)->data->name) + 1) > sizeof(data->name)) + goto error; + /* now put it all together * - store queue data in internals, * - store numa_node info in pci_driver @@ -739,10 +750,13 @@ rte_pmd_init_internals(const char *name, const unsigned nb_rx_queues, data->nb_tx_queues = (uint16_t)nb_tx_queues; data->dev_link = pmd_link; data->mac_addrs = ð_addr; + strncpy(data->name, + (*eth_dev)->data->name, strlen((*eth_dev)->data->name)); (*eth_dev)->data = data; (*eth_dev)->dev_ops = &ops; (*eth_dev)->pci_dev = pci_dev; + (*eth_dev)->driver = &rte_pcap_pmd; return 0; @@ -927,10 +941,36 @@ rte_pmd_pcap_devinit(const char *name, const char *params) } +static int +rte_pmd_pcap_devuninit(const char *name) +{ + struct rte_eth_dev *eth_dev = NULL; + + RTE_LOG(INFO, PMD, "Closing pcap ethdev on numa socket %u\n", + rte_socket_id()); + + if (name == NULL) + return -1; + + /* reserve an ethdev entry */ + eth_dev = rte_eth_dev_allocated(name); + if (eth_dev == NULL) + return -1; + + rte_free(eth_dev->data->dev_private); + rte_free(eth_dev->data); + rte_free(eth_dev->pci_dev); + + rte_eth_dev_release_port(eth_dev); + + return 0; +} + static struct rte_driver pmd_pcap_drv = { .name = "eth_pcap", .type = PMD_VDEV, .init = rte_pmd_pcap_devinit, + .uninit = rte_pmd_pcap_devuninit, }; PMD_REGISTER_DRIVER(pmd_pcap_drv); -- 1.9.1
The patch introduces following commands. - port attach [ident] - port detach [port_id] - attach: attaching a port - detach: detaching a port - ident: pci address of physical device. Or device name and parameters of virtual device. (ex. 0000:02:00.0, eth_pcap0,iface=eth0) - port_id: port identifier v7: - Fix doc. (Thanks to Iremonger, Bernard) - Fix port checking implementation of star_port(); (Thanks to Qiu, Michael) v5: - Add testpmd documentation. (Thanks to Iremonger, Bernard) v4: - Fix strings of command help. Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp> --- app/test-pmd/cmdline.c | 137 +++++++++++++++---- app/test-pmd/config.c | 102 ++++++++------ app/test-pmd/parameters.c | 22 ++- app/test-pmd/testpmd.c | 199 +++++++++++++++++++++------- app/test-pmd/testpmd.h | 18 ++- doc/guides/testpmd_app_ug/testpmd_funcs.rst | 57 ++++++++ 6 files changed, 409 insertions(+), 126 deletions(-) diff --git a/app/test-pmd/cmdline.c b/app/test-pmd/cmdline.c index c6a1627..b78c659 100644 --- a/app/test-pmd/cmdline.c +++ b/app/test-pmd/cmdline.c @@ -513,6 +513,12 @@ static void cmd_help_long_parsed(void *parsed_result, "port close (port_id|all)\n" " Close all ports or port_id.\n\n" + "port attach (ident)\n" + " Attach physical or virtual dev by pci address or virtual device name\n\n" + + "port detach (port_id)\n" + " Detach physical or virtual dev by port_id\n\n" + "port config (port_id|all)" " speed (10|100|1000|10000|40000|auto)" " duplex (half|full|auto)\n" @@ -793,6 +799,89 @@ cmdline_parse_inst_t cmd_operate_specific_port = { }, }; +/* *** attach a specified port *** */ +struct cmd_operate_attach_port_result { + cmdline_fixed_string_t port; + cmdline_fixed_string_t keyword; + cmdline_fixed_string_t identifier; +}; + +static void cmd_operate_attach_port_parsed(void *parsed_result, + __attribute__((unused)) struct cmdline *cl, + __attribute__((unused)) void *data) +{ + struct cmd_operate_attach_port_result *res = parsed_result; + + if (!strcmp(res->keyword, "attach")) + attach_port(res->identifier); + else + printf("Unknown parameter\n"); +} + +cmdline_parse_token_string_t cmd_operate_attach_port_port = + TOKEN_STRING_INITIALIZER(struct cmd_operate_attach_port_result, + port, "port"); +cmdline_parse_token_string_t cmd_operate_attach_port_keyword = + TOKEN_STRING_INITIALIZER(struct cmd_operate_attach_port_result, + keyword, "attach"); +cmdline_parse_token_string_t cmd_operate_attach_port_identifier = + TOKEN_STRING_INITIALIZER(struct cmd_operate_attach_port_result, + identifier, NULL); + +cmdline_parse_inst_t cmd_operate_attach_port = { + .f = cmd_operate_attach_port_parsed, + .data = NULL, + .help_str = "port attach identifier, " + "identifier: pci address or virtual dev name", + .tokens = { + (void *)&cmd_operate_attach_port_port, + (void *)&cmd_operate_attach_port_keyword, + (void *)&cmd_operate_attach_port_identifier, + NULL, + }, +}; + +/* *** detach a specified port *** */ +struct cmd_operate_detach_port_result { + cmdline_fixed_string_t port; + cmdline_fixed_string_t keyword; + uint8_t port_id; +}; + +static void cmd_operate_detach_port_parsed(void *parsed_result, + __attribute__((unused)) struct cmdline *cl, + __attribute__((unused)) void *data) +{ + struct cmd_operate_detach_port_result *res = parsed_result; + + if (!strcmp(res->keyword, "detach")) + detach_port(res->port_id); + else + printf("Unknown parameter\n"); +} + +cmdline_parse_token_string_t cmd_operate_detach_port_port = + TOKEN_STRING_INITIALIZER(struct cmd_operate_detach_port_result, + port, "port"); +cmdline_parse_token_string_t cmd_operate_detach_port_keyword = + TOKEN_STRING_INITIALIZER(struct cmd_operate_detach_port_result, + keyword, "detach"); +cmdline_parse_token_num_t cmd_operate_detach_port_port_id = + TOKEN_NUM_INITIALIZER(struct cmd_operate_detach_port_result, + port_id, UINT8); + +cmdline_parse_inst_t cmd_operate_detach_port = { + .f = cmd_operate_detach_port_parsed, + .data = NULL, + .help_str = "port detach port_id", + .tokens = { + (void *)&cmd_operate_detach_port_port, + (void *)&cmd_operate_detach_port_keyword, + (void *)&cmd_operate_detach_port_port_id, + NULL, + }, +}; + /* *** configure speed for all ports *** */ struct cmd_config_speed_all { cmdline_fixed_string_t port; @@ -847,7 +936,7 @@ cmd_config_speed_all_parsed(void *parsed_result, return; } - for (pid = 0; pid < nb_ports; pid++) { + FOREACH_PORT(pid, ports) { ports[pid].dev_conf.link_speed = link_speed; ports[pid].dev_conf.link_duplex = link_duplex; } @@ -915,10 +1004,8 @@ cmd_config_speed_specific_parsed(void *parsed_result, return; } - if (res->id >= nb_ports) { - printf("Port id %d must be less than %d\n", res->id, nb_ports); + if (port_id_is_invalid(res->id, ENABLED_WARN)) return; - } if (!strcmp(res->value1, "10")) link_speed = ETH_LINK_SPEED_10; @@ -1489,7 +1576,7 @@ cmd_config_rxtx_queue_parsed(void *parsed_result, return; } - if (port_id_is_invalid(res->portid)) + if (port_id_is_invalid(res->portid, ENABLED_WARN)) return; if (port_is_started(res->portid) != 1) { @@ -2889,7 +2976,7 @@ cmd_csum_parsed(void *parsed_result, int hw = 0; uint16_t mask = 0; - if (port_id_is_invalid(res->port_id)) { + if (port_id_is_invalid(res->port_id, ENABLED_WARN)) { printf("invalid port %d\n", res->port_id); return; } @@ -2981,10 +3068,8 @@ cmd_csum_tunnel_parsed(void *parsed_result, { struct cmd_csum_tunnel_result *res = parsed_result; - if (port_id_is_invalid(res->port_id)) { - printf("invalid port %d\n", res->port_id); + if (port_id_is_invalid(res->port_id, ENABLED_WARN)) return; - } if (!strcmp(res->onoff, "on")) ports[res->port_id].tx_ol_flags |= @@ -3039,7 +3124,7 @@ cmd_tso_set_parsed(void *parsed_result, struct cmd_tso_set_result *res = parsed_result; struct rte_eth_dev_info dev_info; - if (port_id_is_invalid(res->port_id)) + if (port_id_is_invalid(res->port_id, ENABLED_WARN)) return; if (!strcmp(res->mode, "set")) @@ -4015,10 +4100,8 @@ static void cmd_set_bond_mac_addr_parsed(void *parsed_result, struct cmd_set_bond_mac_addr_result *res = parsed_result; int ret; - if (res->port_num >= nb_ports) { - printf("Port id %d must be less than %d\n", res->port_num, nb_ports); + if (port_id_is_invalid(res->port_num, ENABLED_WARN)) return; - } ret = rte_eth_bond_mac_address_set(res->port_num, &res->address); @@ -4255,7 +4338,7 @@ static void cmd_set_promisc_mode_parsed(void *parsed_result, /* all ports */ if (allports) { - for (i = 0; i < nb_ports; i++) { + FOREACH_PORT(i, ports) { if (enable) rte_eth_promiscuous_enable(i); else @@ -4335,7 +4418,7 @@ static void cmd_set_allmulti_mode_parsed(void *parsed_result, /* all ports */ if (allports) { - for (i = 0; i < nb_ports; i++) { + FOREACH_PORT(i, ports) { if (enable) rte_eth_allmulticast_enable(i); else @@ -5023,25 +5106,25 @@ static void cmd_showportall_parsed(void *parsed_result, struct cmd_showportall_result *res = parsed_result; if (!strcmp(res->show, "clear")) { if (!strcmp(res->what, "stats")) - for (i = 0; i < nb_ports; i++) + FOREACH_PORT(i, ports) nic_stats_clear(i); else if (!strcmp(res->what, "xstats")) - for (i = 0; i < nb_ports; i++) + FOREACH_PORT(i, ports) nic_xstats_clear(i); } else if (!strcmp(res->what, "info")) - for (i = 0; i < nb_ports; i++) + FOREACH_PORT(i, ports) port_infos_display(i); else if (!strcmp(res->what, "stats")) - for (i = 0; i < nb_ports; i++) + FOREACH_PORT(i, ports) nic_stats_display(i); else if (!strcmp(res->what, "xstats")) - for (i = 0; i < nb_ports; i++) + FOREACH_PORT(i, ports) nic_xstats_display(i); else if (!strcmp(res->what, "fdir")) - for (i = 0; i < nb_ports; i++) + FOREACH_PORT(i, ports) fdir_get_infos(i); else if (!strcmp(res->what, "stat_qmap")) - for (i = 0; i < nb_ports; i++) + FOREACH_PORT(i, ports) nic_stats_mapping_display(i); } @@ -8685,6 +8768,8 @@ cmdline_parse_ctx_t main_ctx[] = { (cmdline_parse_inst_t *)&cmd_set_qmap, (cmdline_parse_inst_t *)&cmd_operate_port, (cmdline_parse_inst_t *)&cmd_operate_specific_port, + (cmdline_parse_inst_t *)&cmd_operate_attach_port, + (cmdline_parse_inst_t *)&cmd_operate_detach_port, (cmdline_parse_inst_t *)&cmd_config_speed_all, (cmdline_parse_inst_t *)&cmd_config_speed_specific, (cmdline_parse_inst_t *)&cmd_config_rx_tx, @@ -8756,7 +8841,7 @@ prompt(void) static void cmd_reconfig_device_queue(portid_t id, uint8_t dev, uint8_t queue) { - if (id < nb_ports) { + if (!port_id_is_invalid(id, DISABLED_WARN)) { /* check if need_reconfig has been set to 1 */ if (ports[id].need_reconfig == 0) ports[id].need_reconfig = dev; @@ -8766,7 +8851,7 @@ cmd_reconfig_device_queue(portid_t id, uint8_t dev, uint8_t queue) } else { portid_t pid; - for (pid = 0; pid < nb_ports; pid++) { + FOREACH_PORT(pid, ports) { /* check if need_reconfig has been set to 1 */ if (ports[pid].need_reconfig == 0) ports[pid].need_reconfig = dev; @@ -8784,10 +8869,8 @@ bypass_is_supported(portid_t port_id) struct rte_port *port; struct rte_pci_id *pci_id; - if (port_id >= nb_ports) { - printf("\tPort id must be less than %d.\n", nb_ports); + if (port_id_is_invalid(port_id, ENABLED_WARN)) return 0; - } /* Get the device id. */ port = &ports[port_id]; diff --git a/app/test-pmd/config.c b/app/test-pmd/config.c index d436ce8..49be819 100644 --- a/app/test-pmd/config.c +++ b/app/test-pmd/config.c @@ -112,11 +112,15 @@ nic_stats_display(portid_t port_id) struct rte_eth_stats stats; struct rte_port *port = &ports[port_id]; uint8_t i; + portid_t pid; static const char *nic_stats_border = "########################"; - if (port_id >= nb_ports) { - printf("Invalid port, range is [0, %d]\n", nb_ports - 1); + if (port_id_is_invalid(port_id, ENABLED_WARN)) { + printf("Valid port range is [0"); + FOREACH_PORT(pid, ports) + printf(", %d", pid); + printf("]\n"); return; } rte_eth_stats_get(port_id, &stats); @@ -189,8 +193,13 @@ nic_stats_display(portid_t port_id) void nic_stats_clear(portid_t port_id) { - if (port_id >= nb_ports) { - printf("Invalid port, range is [0, %d]\n", nb_ports - 1); + portid_t pid; + + if (port_id_is_invalid(port_id, ENABLED_WARN)) { + printf("Valid port range is [0"); + FOREACH_PORT(pid, ports) + printf(", %d", pid); + printf("]\n"); return; } rte_eth_stats_reset(port_id); @@ -237,11 +246,15 @@ nic_stats_mapping_display(portid_t port_id) { struct rte_port *port = &ports[port_id]; uint16_t i; + portid_t pid; static const char *nic_stats_mapping_border = "########################"; - if (port_id >= nb_ports) { - printf("Invalid port, range is [0, %d]\n", nb_ports - 1); + if (port_id_is_invalid(port_id, ENABLED_WARN)) { + printf("Valid port range is [0"); + FOREACH_PORT(pid, ports) + printf(", %d", pid); + printf("]\n"); return; } @@ -290,9 +303,13 @@ port_infos_display(portid_t port_id) int vlan_offload; struct rte_mempool * mp; static const char *info_border = "*********************"; + portid_t pid; - if (port_id >= nb_ports) { - printf("Invalid port, range is [0, %d]\n", nb_ports - 1); + if (port_id_is_invalid(port_id, ENABLED_WARN)) { + printf("Valid port range is [0"); + FOREACH_PORT(pid, ports) + printf(", %d", pid); + printf("]\n"); return; } port = &ports[port_id]; @@ -365,11 +382,14 @@ port_infos_display(portid_t port_id) } int -port_id_is_invalid(portid_t port_id) +port_id_is_invalid(portid_t port_id, enum print_warning warning) { - if (port_id < nb_ports) + if (ports[port_id].enabled) return 0; - printf("Invalid port %d (must be < nb_ports=%d)\n", port_id, nb_ports); + + if (warning == ENABLED_WARN) + printf("Invalid port %d\n", port_id); + return 1; } @@ -428,7 +448,7 @@ port_reg_bit_display(portid_t port_id, uint32_t reg_off, uint8_t bit_x) uint32_t reg_v; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; if (port_reg_off_is_invalid(port_id, reg_off)) return; @@ -447,7 +467,7 @@ port_reg_bit_field_display(portid_t port_id, uint32_t reg_off, uint8_t l_bit; uint8_t h_bit; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; if (port_reg_off_is_invalid(port_id, reg_off)) return; @@ -474,7 +494,7 @@ port_reg_display(portid_t port_id, uint32_t reg_off) { uint32_t reg_v; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; if (port_reg_off_is_invalid(port_id, reg_off)) return; @@ -488,7 +508,7 @@ port_reg_bit_set(portid_t port_id, uint32_t reg_off, uint8_t bit_pos, { uint32_t reg_v; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; if (port_reg_off_is_invalid(port_id, reg_off)) return; @@ -516,7 +536,7 @@ port_reg_bit_field_set(portid_t port_id, uint32_t reg_off, uint8_t l_bit; uint8_t h_bit; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; if (port_reg_off_is_invalid(port_id, reg_off)) return; @@ -550,7 +570,7 @@ port_reg_bit_field_set(portid_t port_id, uint32_t reg_off, void port_reg_set(portid_t port_id, uint32_t reg_off, uint32_t reg_v) { - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; if (port_reg_off_is_invalid(port_id, reg_off)) return; @@ -563,7 +583,7 @@ port_mtu_set(portid_t port_id, uint16_t mtu) { int diag; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; diag = rte_eth_dev_set_mtu(port_id, mtu); if (diag == 0) @@ -726,7 +746,7 @@ rx_ring_desc_display(portid_t port_id, queueid_t rxq_id, uint16_t rxd_id) { const struct rte_memzone *rx_mz; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; if (rx_queue_id_is_invalid(rxq_id)) return; @@ -743,7 +763,7 @@ tx_ring_desc_display(portid_t port_id, queueid_t txq_id, uint16_t txd_id) { const struct rte_memzone *tx_mz; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; if (tx_queue_id_is_invalid(txq_id)) return; @@ -804,7 +824,7 @@ port_rss_reta_info(portid_t port_id, uint16_t i, idx, shift; int ret; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; ret = rte_eth_dev_rss_reta_query(port_id, reta_conf, nb_entries); @@ -859,7 +879,7 @@ port_rss_hash_conf_show(portid_t port_id, int show_rss_key) uint8_t i; int diag; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; /* Get RSS hash key if asked to display it */ rss_conf.rss_key = (show_rss_key) ? rss_key : NULL; @@ -1423,12 +1443,8 @@ set_fwd_ports_list(unsigned int *portlist, unsigned int nb_pt) again: for (i = 0; i < nb_pt; i++) { port_id = (portid_t) portlist[i]; - if (port_id >= nb_ports) { - printf("Invalid port id %u >= %u\n", - (unsigned int) port_id, - (unsigned int) nb_ports); + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; - } if (record_now) fwd_ports_ids[i] = port_id; } @@ -1586,7 +1602,7 @@ vlan_extend_set(portid_t port_id, int on) int diag; int vlan_offload; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; vlan_offload = rte_eth_dev_get_vlan_offload(port_id); @@ -1608,7 +1624,7 @@ rx_vlan_strip_set(portid_t port_id, int on) int diag; int vlan_offload; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; vlan_offload = rte_eth_dev_get_vlan_offload(port_id); @@ -1629,7 +1645,7 @@ rx_vlan_strip_set_on_queue(portid_t port_id, uint16_t queue_id, int on) { int diag; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; diag = rte_eth_dev_set_vlan_strip_on_queue(port_id, queue_id, on); @@ -1644,7 +1660,7 @@ rx_vlan_filter_set(portid_t port_id, int on) int diag; int vlan_offload; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; vlan_offload = rte_eth_dev_get_vlan_offload(port_id); @@ -1665,7 +1681,7 @@ rx_vft_set(portid_t port_id, uint16_t vlan_id, int on) { int diag; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; if (vlan_id_is_invalid(vlan_id)) return; @@ -1682,7 +1698,7 @@ rx_vlan_all_filter_set(portid_t port_id, int on) { uint16_t vlan_id; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; for (vlan_id = 0; vlan_id < 4096; vlan_id++) rx_vft_set(port_id, vlan_id, on); @@ -1692,7 +1708,7 @@ void vlan_tpid_set(portid_t port_id, uint16_t tp_id) { int diag; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; diag = rte_eth_dev_set_vlan_ether_type(port_id, tp_id); @@ -1707,7 +1723,7 @@ vlan_tpid_set(portid_t port_id, uint16_t tp_id) void tx_vlan_set(portid_t port_id, uint16_t vlan_id) { - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; if (vlan_id_is_invalid(vlan_id)) return; @@ -1718,7 +1734,7 @@ tx_vlan_set(portid_t port_id, uint16_t vlan_id) void tx_vlan_reset(portid_t port_id) { - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; ports[port_id].tx_ol_flags &= ~TESTPMD_TX_OFFLOAD_INSERT_VLAN; } @@ -1726,7 +1742,7 @@ tx_vlan_reset(portid_t port_id) void tx_vlan_pvid_set(portid_t port_id, uint16_t vlan_id, int on) { - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; rte_eth_dev_set_vlan_pvid(port_id, vlan_id, on); @@ -1738,7 +1754,7 @@ set_qmap(portid_t port_id, uint8_t is_rx, uint16_t queue_id, uint8_t map_value) uint16_t i; uint8_t existing_mapping_found = 0; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; if (is_rx ? (rx_queue_id_is_invalid(queue_id)) : (tx_queue_id_is_invalid(queue_id))) @@ -1904,7 +1920,7 @@ fdir_get_infos(portid_t port_id) static const char *fdir_stats_border = "########################"; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; ret = rte_eth_dev_filter_supported(port_id, RTE_ETH_FILTER_FDIR); if (ret < 0) { @@ -2030,7 +2046,7 @@ set_vf_traffic(portid_t port_id, uint8_t is_rx, uint16_t vf, uint8_t on) { int diag; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; if (is_rx) diag = rte_eth_dev_set_vf_rx(port_id,vf,on); @@ -2052,7 +2068,7 @@ set_vf_rx_vlan(portid_t port_id, uint16_t vlan_id, uint64_t vf_mask, uint8_t on) { int diag; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; if (vlan_id_is_invalid(vlan_id)) return; @@ -2069,7 +2085,7 @@ set_queue_rate_limit(portid_t port_id, uint16_t queue_idx, uint16_t rate) int diag; struct rte_eth_link link; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return 1; rte_eth_link_get_nowait(port_id, &link); if (rate > link.link_speed) { @@ -2094,7 +2110,7 @@ set_vf_rate_limit(portid_t port_id, uint16_t vf, uint16_t rate, uint64_t q_msk) if (q_msk == 0) return 0; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return 1; rte_eth_link_get_nowait(port_id, &link); if (rate > link.link_speed) { diff --git a/app/test-pmd/parameters.c b/app/test-pmd/parameters.c index 473f824..fa5f2a8 100644 --- a/app/test-pmd/parameters.c +++ b/app/test-pmd/parameters.c @@ -366,6 +366,7 @@ parse_portnuma_config(const char *q_arg) }; unsigned long int_fld[_NUM_FLD]; char *str_fld[_NUM_FLD]; + portid_t pid; /* reset from value set at definition */ while ((p = strchr(p0,'(')) != NULL) { @@ -387,8 +388,11 @@ parse_portnuma_config(const char *q_arg) return -1; } port_id = (uint8_t)int_fld[FLD_PORT]; - if (port_id >= nb_ports) { - printf("Invalid port, range is [0, %d]\n", nb_ports - 1); + if (port_id_is_invalid(port_id, ENABLED_WARN)) { + printf("Valid port range is [0"); + FOREACH_PORT(pid, ports) + printf(", %d", pid); + printf("]\n"); return -1; } socket_id = (uint8_t)int_fld[FLD_SOCKET]; @@ -419,6 +423,7 @@ parse_ringnuma_config(const char *q_arg) }; unsigned long int_fld[_NUM_FLD]; char *str_fld[_NUM_FLD]; + portid_t pid; #define RX_RING_ONLY 0x1 #define TX_RING_ONLY 0x2 #define RXTX_RING 0x3 @@ -443,8 +448,11 @@ parse_ringnuma_config(const char *q_arg) return -1; } port_id = (uint8_t)int_fld[FLD_PORT]; - if (port_id >= nb_ports) { - printf("Invalid port, range is [0, %d]\n", nb_ports - 1); + if (port_id_is_invalid(port_id, ENABLED_WARN)) { + printf("Valid port range is [0"); + FOREACH_PORT(pid, ports) + printf(", %d", pid); + printf("]\n"); return -1; } socket_id = (uint8_t)int_fld[FLD_SOCKET]; @@ -615,12 +623,12 @@ launch_args_parse(int argc, char** argv) #endif if (!strcmp(lgopts[opt_idx].name, "nb-ports")) { n = atoi(optarg); - if (n > 0 && n <= nb_ports) + if (n > 0 && + !port_id_is_invalid(n, DISABLED_WARN)) nb_fwd_ports = (uint8_t) n; else rte_exit(EXIT_FAILURE, - "nb-ports should be > 0 and <= %d\n", - nb_ports); + "Invalid port %d\n", n); } if (!strcmp(lgopts[opt_idx].name, "nb-cores")) { n = atoi(optarg); diff --git a/app/test-pmd/testpmd.c b/app/test-pmd/testpmd.c index 605163b..44bb8b3 100644 --- a/app/test-pmd/testpmd.c +++ b/app/test-pmd/testpmd.c @@ -71,6 +71,7 @@ #include <rte_pci.h> #include <rte_ether.h> #include <rte_ethdev.h> +#include <rte_dev.h> #include <rte_string_fns.h> #ifdef RTE_LIBRTE_PMD_XENVIRT #include <rte_eth_xenvirt.h> @@ -315,7 +316,7 @@ uint16_t nb_rx_queue_stats_mappings = 0; /* Forward function declarations */ static void map_port_queue_stats_mapping_registers(uint8_t pi, struct rte_port *port); -static void check_all_ports_link_status(uint8_t port_num, uint32_t port_mask); +static void check_all_ports_link_status(uint32_t port_mask); /* * Check if all the ports are started. @@ -324,6 +325,20 @@ static void check_all_ports_link_status(uint8_t port_num, uint32_t port_mask); static int all_ports_started(void); /* + * Find next enabled port + */ +portid_t +find_next_port(portid_t p, struct rte_port *ports, int size) +{ + if (ports == NULL) + rte_exit(-EINVAL, "failed to find a next port id\n"); + + while ((ports[p].enabled == 0) && (p < size)) + p++; + return p; +} + +/* * Setup default configuration. */ static void @@ -552,7 +567,8 @@ init_config(void) + RTE_TEST_TX_DESC_MAX + MAX_PKT_BURST; if (!numa_support) - nb_mbuf_per_pool = (nb_mbuf_per_pool * nb_ports); + nb_mbuf_per_pool = + (nb_mbuf_per_pool * RTE_MAX_ETHPORTS); } if (!numa_support) { @@ -565,14 +581,19 @@ init_config(void) /* Configuration of Ethernet ports. */ ports = rte_zmalloc("testpmd: ports", - sizeof(struct rte_port) * nb_ports, + sizeof(struct rte_port) * RTE_MAX_ETHPORTS, RTE_CACHE_LINE_SIZE); if (ports == NULL) { - rte_exit(EXIT_FAILURE, "rte_zmalloc(%d struct rte_port) " - "failed\n", nb_ports); + rte_exit(EXIT_FAILURE, + "rte_zmalloc(%d struct rte_port) failed\n", + RTE_MAX_ETHPORTS); } - for (pid = 0; pid < nb_ports; pid++) { + /* enabled allocated ports */ + for (pid = 0; pid < nb_ports; pid++) + ports[pid].enabled = 1; + + FOREACH_PORT(pid, ports) { port = &ports[pid]; rte_eth_dev_info_get(pid, &port->dev_info); @@ -602,8 +623,7 @@ init_config(void) nb_mbuf_per_pool = nb_mbuf_per_pool/nb_ports; for (i = 0; i < MAX_SOCKET; i++) { - nb_mbuf = (nb_mbuf_per_pool * - port_per_socket[i]); + nb_mbuf = (nb_mbuf_per_pool * RTE_MAX_ETHPORTS); if (nb_mbuf) mbuf_pool_create(mbuf_data_size, nb_mbuf,i); @@ -635,14 +655,6 @@ reconfig(portid_t new_port_id, unsigned socket_id) struct rte_port *port; /* Reconfiguration of Ethernet ports. */ - ports = rte_realloc(ports, - sizeof(struct rte_port) * nb_ports, - RTE_CACHE_LINE_SIZE); - if (ports == NULL) { - rte_exit(EXIT_FAILURE, "rte_realloc(%d struct rte_port) failed\n", - nb_ports); - } - port = &ports[new_port_id]; rte_eth_dev_info_get(new_port_id, &port->dev_info); @@ -663,7 +675,7 @@ init_fwd_streams(void) streamid_t sm_id, nb_fwd_streams_new; /* set socket id according to numa or not */ - for (pid = 0; pid < nb_ports; pid++) { + FOREACH_PORT(pid, ports) { port = &ports[pid]; if (nb_rxq > port->dev_info.max_rx_queues) { printf("Fail: nb_rxq(%d) is greater than " @@ -1264,7 +1276,7 @@ all_ports_started(void) portid_t pi; struct rte_port *port; - for (pi = 0; pi < nb_ports; pi++) { + FOREACH_PORT(pi, ports) { port = &ports[pi]; /* Check if there is a port which is not started */ if (port->port_status != RTE_PORT_STARTED) @@ -1276,6 +1288,45 @@ all_ports_started(void) } int +all_ports_stopped(void) +{ + portid_t pi; + struct rte_port *port; + + FOREACH_PORT(pi, ports) { + port = &ports[pi]; + if (port->port_status != RTE_PORT_STOPPED) + return 0; + } + + return 1; +} + +int +port_is_started(portid_t port_id) +{ + if (port_id_is_invalid(port_id, ENABLED_WARN)) + return 0; + + if (ports[port_id].port_status != RTE_PORT_STARTED) + return 0; + + return 1; +} + +static int +port_is_closed(portid_t port_id) +{ + if (port_id_is_invalid(port_id, ENABLED_WARN)) + return 0; + + if (ports[port_id].port_status != RTE_PORT_CLOSED) + return 0; + + return 1; +} + +int start_port(portid_t pid) { int diag, need_check_link_status = 0; @@ -1296,8 +1347,8 @@ start_port(portid_t pid) if(dcb_config) dcb_test = 1; - for (pi = 0; pi < nb_ports; pi++) { - if (pid < nb_ports && pid != pi) + FOREACH_PORT(pi, ports) { + if (pid != pi && pid != (portid_t)RTE_PORT_ALL) continue; port = &ports[pi]; @@ -1421,7 +1472,7 @@ start_port(portid_t pid) } if (need_check_link_status && !no_link_check) - check_all_ports_link_status(nb_ports, RTE_PORT_ALL); + check_all_ports_link_status(RTE_PORT_ALL); else printf("Please stop the ports first\n"); @@ -1446,8 +1497,8 @@ stop_port(portid_t pid) } printf("Stopping ports...\n"); - for (pi = 0; pi < nb_ports; pi++) { - if (pid < nb_ports && pid != pi) + FOREACH_PORT(pi, ports) { + if (!port_id_is_invalid(pid, DISABLED_WARN) && pid != pi) continue; port = &ports[pi]; @@ -1463,7 +1514,7 @@ stop_port(portid_t pid) need_check_link_status = 1; } if (need_check_link_status && !no_link_check) - check_all_ports_link_status(nb_ports, RTE_PORT_ALL); + check_all_ports_link_status(RTE_PORT_ALL); printf("Done\n"); } @@ -1481,8 +1532,8 @@ close_port(portid_t pid) printf("Closing ports...\n"); - for (pi = 0; pi < nb_ports; pi++) { - if (pid < nb_ports && pid != pi) + FOREACH_PORT(pi, ports) { + if (!port_id_is_invalid(pid, DISABLED_WARN) && pid != pi) continue; port = &ports[pi]; @@ -1502,31 +1553,83 @@ close_port(portid_t pid) printf("Done\n"); } -int -all_ports_stopped(void) +void +attach_port(char *identifier) { - portid_t pi; - struct rte_port *port; + portid_t i, j, pi = 0; - for (pi = 0; pi < nb_ports; pi++) { - port = &ports[pi]; - if (port->port_status != RTE_PORT_STOPPED) - return 0; + printf("Attaching a new port...\n"); + + if (identifier == NULL) { + printf("Invalid parameters are specified\n"); + return; } - return 1; + if (test_done == 0) { + printf("Please stop forwarding first\n"); + return; + } + + if (rte_eal_dev_attach(identifier, &pi)) + return; + + ports[pi].enabled = 1; + reconfig(pi, rte_eth_dev_socket_id(pi)); + rte_eth_promiscuous_enable(pi); + + nb_ports = rte_eth_dev_count(); + + /* set_default_fwd_ports_config(); */ + bzero(fwd_ports_ids, sizeof(fwd_ports_ids)); + i = 0; + FOREACH_PORT(j, ports) { + fwd_ports_ids[i] = j; + i++; + } + nb_cfg_ports = nb_ports; + nb_fwd_ports++; + + ports[pi].port_status = RTE_PORT_STOPPED; + + printf("Port %d is attached. Now total ports is %d\n", pi, nb_ports); + printf("Done\n"); } -int -port_is_started(portid_t port_id) +void +detach_port(uint8_t port_id) { - if (port_id_is_invalid(port_id)) - return -1; + portid_t i, pi = 0; + char name[RTE_ETH_NAME_MAX_LEN]; - if (ports[port_id].port_status != RTE_PORT_STARTED) - return 0; + printf("Detaching a port...\n"); - return 1; + if (!port_is_closed(port_id)) { + printf("Please close port first\n"); + return; + } + + rte_eth_promiscuous_disable(port_id); + + if (rte_eal_dev_detach(port_id, name)) + return; + + ports[port_id].enabled = 0; + nb_ports = rte_eth_dev_count(); + + /* set_default_fwd_ports_config(); */ + bzero(fwd_ports_ids, sizeof(fwd_ports_ids)); + i = 0; + FOREACH_PORT(pi, ports) { + fwd_ports_ids[i] = pi; + i++; + } + nb_cfg_ports = nb_ports; + nb_fwd_ports--; + + printf("Port '%s' is detached. Now total ports is %d\n", + name, nb_ports); + printf("Done\n"); + return; } void @@ -1534,7 +1637,7 @@ pmd_test_exit(void) { portid_t pt_id; - for (pt_id = 0; pt_id < nb_ports; pt_id++) { + FOREACH_PORT(pt_id, ports) { printf("Stopping port %d...", pt_id); fflush(stdout); rte_eth_dev_close(pt_id); @@ -1553,7 +1656,7 @@ struct pmd_test_command { /* Check the link status of all ports in up to 9s, and print them finally */ static void -check_all_ports_link_status(uint8_t port_num, uint32_t port_mask) +check_all_ports_link_status(uint32_t port_mask) { #define CHECK_INTERVAL 100 /* 100ms */ #define MAX_CHECK_TIME 90 /* 9s (90 * 100ms) in total */ @@ -1564,7 +1667,7 @@ check_all_ports_link_status(uint8_t port_num, uint32_t port_mask) fflush(stdout); for (count = 0; count <= MAX_CHECK_TIME; count++) { all_ports_up = 1; - for (portid = 0; portid < port_num; portid++) { + FOREACH_PORT(portid, ports) { if ((port_mask & (1 << portid)) == 0) continue; memset(&link, 0, sizeof(link)); @@ -1729,7 +1832,7 @@ init_port_config(void) portid_t pid; struct rte_port *port; - for (pid = 0; pid < nb_ports; pid++) { + FOREACH_PORT(pid, ports) { port = &ports[pid]; port->dev_conf.rxmode = rx_mode; port->dev_conf.fdir_conf = fdir_conf; @@ -1908,7 +2011,7 @@ main(int argc, char** argv) nb_ports = (portid_t) rte_eth_dev_count(); if (nb_ports == 0) - rte_exit(EXIT_FAILURE, "No probed ethernet device\n"); + RTE_LOG(WARNING, EAL, "No probed ethernet devices\n"); set_def_fwd_config(); if (nb_lcores == 0) @@ -1930,7 +2033,7 @@ main(int argc, char** argv) rte_exit(EXIT_FAILURE, "Start ports failed\n"); /* set all ports to promiscuous mode by default */ - for (port_id = 0; port_id < nb_ports; port_id++) + FOREACH_PORT(port_id, ports) rte_eth_promiscuous_enable(port_id); #ifdef RTE_LIBRTE_CMDLINE diff --git a/app/test-pmd/testpmd.h b/app/test-pmd/testpmd.h index 126bef7..0d5a526 100644 --- a/app/test-pmd/testpmd.h +++ b/app/test-pmd/testpmd.h @@ -137,6 +137,7 @@ struct fwd_stream { * The data structure associated with each port. */ struct rte_port { + uint8_t enabled; /**< Port enabled or not */ struct rte_eth_dev_info dev_info; /**< PCI info + driver name */ struct rte_eth_conf dev_conf; /**< Port configuration. */ struct ether_addr eth_addr; /**< Port ethernet address */ @@ -162,6 +163,14 @@ struct rte_port { struct rte_eth_txconf tx_conf; /**< tx configuration */ }; +extern portid_t __rte_unused +find_next_port(portid_t p, struct rte_port *ports, int size); + +#define FOREACH_PORT(p, ports) \ + for (p = find_next_port(0, ports, RTE_MAX_ETHPORTS); \ + p < RTE_MAX_ETHPORTS; \ + p = find_next_port(p + 1, ports, RTE_MAX_ETHPORTS)) + /** * The data structure associated with each forwarding logical core. * The logical cores are internally numbered by a core index from 0 to @@ -522,6 +531,8 @@ int init_port_dcb_config(portid_t pid,struct dcb_config *dcb_conf); int start_port(portid_t pid); void stop_port(portid_t pid); void close_port(portid_t pid); +void attach_port(char *identifier); +void detach_port(uint8_t port_id); int all_ports_stopped(void); int port_is_started(portid_t port_id); void pmd_test_exit(void); @@ -548,10 +559,15 @@ void get_syn_filter(uint8_t port_id); void get_ethertype_filter(uint8_t port_id, uint16_t index); void get_2tuple_filter(uint8_t port_id, uint16_t index); void get_5tuple_filter(uint8_t port_id, uint16_t index); -int port_id_is_invalid(portid_t port_id); int rx_queue_id_is_invalid(queueid_t rxq_id); int tx_queue_id_is_invalid(queueid_t txq_id); +enum print_warning { + ENABLED_WARN = 0, + DISABLED_WARN +}; +int port_id_is_invalid(portid_t port_id, enum print_warning warning); + /* * Work-around of a compilation error with ICC on invocations of the * rte_be_to_cpu_16() function. diff --git a/doc/guides/testpmd_app_ug/testpmd_funcs.rst b/doc/guides/testpmd_app_ug/testpmd_funcs.rst index afe1970..a99e14d 100644 --- a/doc/guides/testpmd_app_ug/testpmd_funcs.rst +++ b/doc/guides/testpmd_app_ug/testpmd_funcs.rst @@ -807,6 +807,63 @@ The following sections show functions for configuring ports. Port configuration changes only become active when forwarding is started/restarted. +port attach +~~~~~~~~~~~ + +Attach a port specified by pci address or virtual device args. + +To attach a new pci device, the device should be recognized by kernel first. +Then it should be moved under DPDK management. +Finally the port can be attached to testpmd. +On the other hand, to attach a port created by virtual device, above steps are not needed. + +port attach (identifier) + +For example, to attach a port whose pci address is 0000:02:00.0. + +.. code-block:: console + + testpmd> port attach 0000:02:00.0 + Attaching a new port... + ... snip ... + Port 0 is attached. Now total ports is 1 + Done + +For example, to attach a port created by pcap PMD. + +.. code-block:: console + + testpmd> port attach eth_pcap0,iface=eth0 + Attaching a new port... + ... snip ... + Port 0 is attached. Now total ports is 1 + Done + +In this case, identifier is "eth_pcap0,iface=eth0". +This identifier format is the same as "--vdev" format of DPDK applications. + +port detach +~~~~~~~~~~~ + +Detach a specific port. + +Before detaching a port, the port should be closed. +Also to remove a pci device completely from the system, first detach the port from testpmd. +Then the device should be moved under kernel management. +Finally the device can be removed using kernel pci hotplug functionality. +On the other hand, to remove a port created by a virtual device, above steps are not needed. + +port detach (port_id) + +For example, to detach a port 0. + +.. code-block:: console + + testpmd> port detach 0 + Detaching a port... + ... snip ... + Done + port start ~~~~~~~~~~ -- 1.9.1
> -----Original Message----- > From: Tetsuya Mukawa [mailto:mukawa@igel.co.jp] > Sent: Monday, February 23, 2015 5:09 AM > To: dev@dpdk.org > Cc: Qiu, Michael; Iremonger, Bernard; maxime.leroy@6wind.com; Tetsuya Mukawa > Subject: [PATCH v11 08/13] ethdev: Add functions that will be used by port hotplug functions > > The patch adds following functions. > > - rte_eth_dev_save() > The function is used for saving current rte_eth_dev structures. > - rte_eth_dev_get_changed_port() > The function receives the rte_eth_dev structures, then compare > these with current values to know which port is actually > attached or detached. > - rte_eth_dev_get_addr_by_port() > The function returns a pci address of an ethdev specified by port > identifier. > - rte_eth_dev_get_port_by_addr() > The function returns a port identifier of an ethdev specified by > pci address. > - rte_eth_dev_get_name_by_port() > The function returns a unique identifier name of an ethdev > specified by port identifier. > - Add rte_eth_dev_is_detachable() > The function returns whether a PMD supports detach function. > > Also, the patch changes scope of rte_eth_dev_allocated() to global. > This function will be called by virtual PMDs to support port hotplug. > So change scope of the function to global. > > v10: > - Change order of version.map. > (Thanks to Thomas Monjalon) > v9: > - rte_eth_dev_check_detachable() is replaced by > rte_eth_dev_is_detachable(). > - strncpy() is replaced by strcpy(). > (Thanks to Thomas Monjalon) > - Add missing symbol in version map. > (Thanks to Nail Horman) > v8: > - Add size parameter to rte_eth_dev_save(). > - Add missing symbol in version map. > (Thanks to Qiu, Michael and Iremonger, Bernard) > v7: > - Add pt_driver checking to rte_eth_dev_check_detachable(). > (Thanks to Qiu, Michael) > v5: > - Fix return value of below functions. > rte_eth_dev_get_changed_port(). > rte_eth_dev_get_port_by_addr(). > v4: > - Add parameter checking. > v3: > - Fix if-condition bug while comparing pci addresses. > - Add error checking codes. > Reported-by: Mark Enright <menrigh@brocade.com> > > Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp> > --- > lib/librte_ether/rte_ethdev.c | 103 ++++++++++++++++++++++++++++++++- > lib/librte_ether/rte_ethdev.h | 83 ++++++++++++++++++++++++++ > lib/librte_ether/rte_ether_version.map | 7 +++ > 3 files changed, 192 insertions(+), 1 deletion(-) > > diff --git a/lib/librte_ether/rte_ethdev.c b/lib/librte_ether/rte_ethdev.c index 3d148e2..7067620 > 100644 > --- a/lib/librte_ether/rte_ethdev.c > +++ b/lib/librte_ether/rte_ethdev.c > @@ -201,7 +201,7 @@ rte_eth_dev_data_alloc(void) > RTE_MAX_ETHPORTS * sizeof(*rte_eth_dev_data)); } > > -static struct rte_eth_dev * > +struct rte_eth_dev * > rte_eth_dev_allocated(const char *name) { > unsigned i; > @@ -426,6 +426,107 @@ rte_eth_dev_count(void) > return (nb_ports); > } > > +int > +rte_eth_dev_save(struct rte_eth_dev *devs, size_t size) { > + if ((devs == NULL) || > + (size != sizeof(struct rte_eth_dev) * RTE_MAX_ETHPORTS)) > + return -EINVAL; > + > + /* save current rte_eth_devices */ > + memcpy(devs, rte_eth_devices, size); > + return 0; > +} > + > +int > +rte_eth_dev_get_changed_port(struct rte_eth_dev *devs, uint8_t > +*port_id) { > + if ((devs == NULL) || (port_id == NULL)) > + return -EINVAL; > + > + /* check which port was attached or detached */ > + for (*port_id = 0; *port_id < RTE_MAX_ETHPORTS; (*port_id)++, devs++) { > + if (rte_eth_devices[*port_id].attached ^ devs->attached) > + return 0; > + } > + return -ENODEV; > +} > + > +int > +rte_eth_dev_get_addr_by_port(uint8_t port_id, struct rte_pci_addr > +*addr) { > + if (!rte_eth_dev_is_valid_port(port_id)) { > + PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); > + return -EINVAL; > + } > + > + if (addr == NULL) { > + PMD_DEBUG_TRACE("Null pointer is specified\n"); > + return -EINVAL; > + } > + > + *addr = rte_eth_devices[port_id].pci_dev->addr; > + return 0; > +} > + > +int > +rte_eth_dev_get_port_by_addr(struct rte_pci_addr *addr, uint8_t > +*port_id) { > + struct rte_pci_addr *tmp; > + > + if ((addr == NULL) || (port_id == NULL)) { > + PMD_DEBUG_TRACE("Null pointer is specified\n"); > + return -EINVAL; > + } > + > + for (*port_id = 0; *port_id < RTE_MAX_ETHPORTS; (*port_id)++) { > + if (!rte_eth_devices[*port_id].attached) > + continue; > + if (!rte_eth_devices[*port_id].pci_dev) > + continue; > + tmp = &rte_eth_devices[*port_id].pci_dev->addr; > + if (rte_eal_compare_pci_addr(tmp, addr) == 0) > + return 0; > + } > + return -ENODEV; > +} > + > +int > +rte_eth_dev_get_name_by_port(uint8_t port_id, char *name) { > + char *tmp; > + > + if (!rte_eth_dev_is_valid_port(port_id)) { > + PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); > + return -EINVAL; > + } > + > + if (name == NULL) { > + PMD_DEBUG_TRACE("Null pointer is specified\n"); > + return -EINVAL; > + } > + > + /* shouldn't check 'rte_eth_devices[i].data', > + * because it might be overwritten by VDEV PMD */ > + tmp = rte_eth_dev_data[port_id].name; > + strcpy(name, tmp); > + return 0; > +} > + > +int > +rte_eth_dev_is_detachable(uint8_t port_id) { > + uint32_t drv_flags; > + > + if (port_id >= RTE_MAX_ETHPORTS) { > + PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); > + return -EINVAL; > + } > + > + drv_flags = rte_eth_devices[port_id].driver->pci_drv.drv_flags; > + return !(drv_flags & RTE_PCI_DRV_DETACHABLE); } > + > static int > rte_eth_dev_rx_queue_config(struct rte_eth_dev *dev, uint16_t nb_queues) { diff --git > a/lib/librte_ether/rte_ethdev.h b/lib/librte_ether/rte_ethdev.h index 6cba06d..d4cfafb 100644 > --- a/lib/librte_ether/rte_ethdev.h > +++ b/lib/librte_ether/rte_ethdev.h > @@ -1476,6 +1476,89 @@ extern struct rte_eth_dev rte_eth_devices[]; extern uint8_t > rte_eth_dev_count(void); > > /** > + * Function for internal use by port hotplug functions. > + * Copies current ethdev structures to the specified pointer. > + * > + * @param devs The pointer to the ethdev structures > + * @param size The size of ethdev structures > + * @return > + * - 0 on success, negative on error > + */ > +extern int rte_eth_dev_save(struct rte_eth_dev *devs, size_t size); > + > +/** > + * Function for internal use by port hotplug functions. > + * Compare the specified ethdev structures with currents. Then > + * if there is a port which status is changed, fill the specified > +pointer > + * with the port id of that port. > + * @param devs The pointer to the ethdev structures > + * @param port_id The pointer to the port id > + * @return > + * - 0 on success, negative on error > + */ > +extern int rte_eth_dev_get_changed_port( > + struct rte_eth_dev *devs, uint8_t *port_id); > + > +/** > + * Function for internal use by port hotplug functions. > + * Returns a pci address of a ethdev specified by port identifier. > + * @param port_id > + * The port identifier of the Ethernet device > + * @param addr > + * The pointer to the pci address > + * @return > + * - 0 on success, negative on error > + */ > +extern int rte_eth_dev_get_addr_by_port( > + uint8_t port_id, struct rte_pci_addr *addr); > + > +/** > + * Function for internal use by port hotplug functions. > + * Returns a port identifier of a ethdev specified by pci address. > + * @param addr > + * The pointer to the pci address of the Ethernet device. > + * @param port_id > + * The pointer to the port identifier > + * @return > + * - 0 on success, negative on error > + */ > +extern int rte_eth_dev_get_port_by_addr( > + struct rte_pci_addr *addr, uint8_t *port_id); > + > +/** > + * Function for internal use by port hotplug functions. > + * Returns a unique identifier name of a ethdev specified by port identifier. > + * @param port_id > + * The port identifier. > + * @param name > + * The pointer to the Unique identifier name for each Ethernet device > + * @return > + * - 0 on success, negative on error > + */ > +extern int rte_eth_dev_get_name_by_port(uint8_t port_id, char *name); > + > +/** > + * Function for internal use by port hotplug functions. > + * Check whether or not, a PMD that is handling the ethdev specified by > +port > + * identifier can support detach function. > + * @param port_id > + * The port identifier > + * @return > + * - 0 on supporting detach function, negative on not supporting > + */ > +extern int rte_eth_dev_is_detachable(uint8_t port_id); > + > +/** > + * Function for internal use by port hotplug functions. > + * Returns a ethdev slot specified by the unique identifier name. > + * @param name > + * The pointer to the Unique identifier name for each Ethernet device > + * @return > + * - The pointer to the ethdev slot, on success. NULL on error > + */ > +extern struct rte_eth_dev *rte_eth_dev_allocated(const char *name); > + > +/** > * Function for internal use by dummy drivers primarily, e.g. ring-based > * driver. > * Allocates a new ethdev slot for an ethernet device and returns the pointer diff --git > a/lib/librte_ether/rte_ether_version.map b/lib/librte_ether/rte_ether_version.map > index f66fd2d..099c769 100644 > --- a/lib/librte_ether/rte_ether_version.map > +++ b/lib/librte_ether/rte_ether_version.map > @@ -6,6 +6,7 @@ DPDK_2.0 { > rte_eth_allmulticast_enable; > rte_eth_allmulticast_get; > rte_eth_dev_allocate; > + rte_eth_dev_allocated; > rte_eth_dev_bypass_event_show; > rte_eth_dev_bypass_event_store; > rte_eth_dev_bypass_init; > @@ -32,9 +33,14 @@ DPDK_2.0 { > rte_eth_dev_filter_supported; > rte_eth_dev_flow_ctrl_get; > rte_eth_dev_flow_ctrl_set; > + rte_eth_dev_get_addr_by_port; > + rte_eth_dev_get_changed_port; Hi Tetsuya, rte_eth_dev_get_device_type; needs to be added to rte_ether_version map to solve linking issue. Regards, Bernard. > rte_eth_dev_get_mtu; > + rte_eth_dev_get_name_by_port; > + rte_eth_dev_get_port_by_addr; > rte_eth_dev_get_vlan_offload; > rte_eth_dev_info_get; > + rte_eth_dev_is_detachable; > rte_eth_dev_mac_addr_add; > rte_eth_dev_mac_addr_remove; > rte_eth_dev_priority_flow_ctrl_set; > @@ -44,6 +50,7 @@ DPDK_2.0 { > rte_eth_dev_rss_reta_update; > rte_eth_dev_rx_queue_start; > rte_eth_dev_rx_queue_stop; > + rte_eth_dev_save; > rte_eth_dev_set_link_down; > rte_eth_dev_set_link_up; > rte_eth_dev_set_mtu; > -- > 1.9.1
On 2015/02/23 20:01, Iremonger, Bernard wrote: >> -----Original Message----- >> From: Tetsuya Mukawa [mailto:mukawa@igel.co.jp] >> Sent: Monday, February 23, 2015 5:09 AM >> To: dev@dpdk.org >> Cc: Qiu, Michael; Iremonger, Bernard; maxime.leroy@6wind.com; Tetsuya Mukawa >> Subject: [PATCH v11 08/13] ethdev: Add functions that will be used by port hotplug functions >> >> The patch adds following functions. >> >> - rte_eth_dev_save() >> The function is used for saving current rte_eth_dev structures. >> - rte_eth_dev_get_changed_port() >> The function receives the rte_eth_dev structures, then compare >> these with current values to know which port is actually >> attached or detached. >> - rte_eth_dev_get_addr_by_port() >> The function returns a pci address of an ethdev specified by port >> identifier. >> - rte_eth_dev_get_port_by_addr() >> The function returns a port identifier of an ethdev specified by >> pci address. >> - rte_eth_dev_get_name_by_port() >> The function returns a unique identifier name of an ethdev >> specified by port identifier. >> - Add rte_eth_dev_is_detachable() >> The function returns whether a PMD supports detach function. >> >> Also, the patch changes scope of rte_eth_dev_allocated() to global. >> This function will be called by virtual PMDs to support port hotplug. >> So change scope of the function to global. >> >> v10: >> - Change order of version.map. >> (Thanks to Thomas Monjalon) >> v9: >> - rte_eth_dev_check_detachable() is replaced by >> rte_eth_dev_is_detachable(). >> - strncpy() is replaced by strcpy(). >> (Thanks to Thomas Monjalon) >> - Add missing symbol in version map. >> (Thanks to Nail Horman) >> v8: >> - Add size parameter to rte_eth_dev_save(). >> - Add missing symbol in version map. >> (Thanks to Qiu, Michael and Iremonger, Bernard) >> v7: >> - Add pt_driver checking to rte_eth_dev_check_detachable(). >> (Thanks to Qiu, Michael) >> v5: >> - Fix return value of below functions. >> rte_eth_dev_get_changed_port(). >> rte_eth_dev_get_port_by_addr(). >> v4: >> - Add parameter checking. >> v3: >> - Fix if-condition bug while comparing pci addresses. >> - Add error checking codes. >> Reported-by: Mark Enright <menrigh@brocade.com> >> >> Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp> >> --- >> lib/librte_ether/rte_ethdev.c | 103 ++++++++++++++++++++++++++++++++- >> lib/librte_ether/rte_ethdev.h | 83 ++++++++++++++++++++++++++ >> lib/librte_ether/rte_ether_version.map | 7 +++ >> 3 files changed, 192 insertions(+), 1 deletion(-) >> >> diff --git a/lib/librte_ether/rte_ethdev.c b/lib/librte_ether/rte_ethdev.c index 3d148e2..7067620 >> 100644 >> --- a/lib/librte_ether/rte_ethdev.c >> +++ b/lib/librte_ether/rte_ethdev.c >> @@ -201,7 +201,7 @@ rte_eth_dev_data_alloc(void) >> RTE_MAX_ETHPORTS * sizeof(*rte_eth_dev_data)); } >> >> -static struct rte_eth_dev * >> +struct rte_eth_dev * >> rte_eth_dev_allocated(const char *name) { >> unsigned i; >> @@ -426,6 +426,107 @@ rte_eth_dev_count(void) >> return (nb_ports); >> } >> >> +int >> +rte_eth_dev_save(struct rte_eth_dev *devs, size_t size) { >> + if ((devs == NULL) || >> + (size != sizeof(struct rte_eth_dev) * RTE_MAX_ETHPORTS)) >> + return -EINVAL; >> + >> + /* save current rte_eth_devices */ >> + memcpy(devs, rte_eth_devices, size); >> + return 0; >> +} >> + >> +int >> +rte_eth_dev_get_changed_port(struct rte_eth_dev *devs, uint8_t >> +*port_id) { >> + if ((devs == NULL) || (port_id == NULL)) >> + return -EINVAL; >> + >> + /* check which port was attached or detached */ >> + for (*port_id = 0; *port_id < RTE_MAX_ETHPORTS; (*port_id)++, devs++) { >> + if (rte_eth_devices[*port_id].attached ^ devs->attached) >> + return 0; >> + } >> + return -ENODEV; >> +} >> + >> +int >> +rte_eth_dev_get_addr_by_port(uint8_t port_id, struct rte_pci_addr >> +*addr) { >> + if (!rte_eth_dev_is_valid_port(port_id)) { >> + PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); >> + return -EINVAL; >> + } >> + >> + if (addr == NULL) { >> + PMD_DEBUG_TRACE("Null pointer is specified\n"); >> + return -EINVAL; >> + } >> + >> + *addr = rte_eth_devices[port_id].pci_dev->addr; >> + return 0; >> +} >> + >> +int >> +rte_eth_dev_get_port_by_addr(struct rte_pci_addr *addr, uint8_t >> +*port_id) { >> + struct rte_pci_addr *tmp; >> + >> + if ((addr == NULL) || (port_id == NULL)) { >> + PMD_DEBUG_TRACE("Null pointer is specified\n"); >> + return -EINVAL; >> + } >> + >> + for (*port_id = 0; *port_id < RTE_MAX_ETHPORTS; (*port_id)++) { >> + if (!rte_eth_devices[*port_id].attached) >> + continue; >> + if (!rte_eth_devices[*port_id].pci_dev) >> + continue; >> + tmp = &rte_eth_devices[*port_id].pci_dev->addr; >> + if (rte_eal_compare_pci_addr(tmp, addr) == 0) >> + return 0; >> + } >> + return -ENODEV; >> +} >> + >> +int >> +rte_eth_dev_get_name_by_port(uint8_t port_id, char *name) { >> + char *tmp; >> + >> + if (!rte_eth_dev_is_valid_port(port_id)) { >> + PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); >> + return -EINVAL; >> + } >> + >> + if (name == NULL) { >> + PMD_DEBUG_TRACE("Null pointer is specified\n"); >> + return -EINVAL; >> + } >> + >> + /* shouldn't check 'rte_eth_devices[i].data', >> + * because it might be overwritten by VDEV PMD */ >> + tmp = rte_eth_dev_data[port_id].name; >> + strcpy(name, tmp); >> + return 0; >> +} >> + >> +int >> +rte_eth_dev_is_detachable(uint8_t port_id) { >> + uint32_t drv_flags; >> + >> + if (port_id >= RTE_MAX_ETHPORTS) { >> + PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); >> + return -EINVAL; >> + } >> + >> + drv_flags = rte_eth_devices[port_id].driver->pci_drv.drv_flags; >> + return !(drv_flags & RTE_PCI_DRV_DETACHABLE); } >> + >> static int >> rte_eth_dev_rx_queue_config(struct rte_eth_dev *dev, uint16_t nb_queues) { diff --git >> a/lib/librte_ether/rte_ethdev.h b/lib/librte_ether/rte_ethdev.h index 6cba06d..d4cfafb 100644 >> --- a/lib/librte_ether/rte_ethdev.h >> +++ b/lib/librte_ether/rte_ethdev.h >> @@ -1476,6 +1476,89 @@ extern struct rte_eth_dev rte_eth_devices[]; extern uint8_t >> rte_eth_dev_count(void); >> >> /** >> + * Function for internal use by port hotplug functions. >> + * Copies current ethdev structures to the specified pointer. >> + * >> + * @param devs The pointer to the ethdev structures >> + * @param size The size of ethdev structures >> + * @return >> + * - 0 on success, negative on error >> + */ >> +extern int rte_eth_dev_save(struct rte_eth_dev *devs, size_t size); >> + >> +/** >> + * Function for internal use by port hotplug functions. >> + * Compare the specified ethdev structures with currents. Then >> + * if there is a port which status is changed, fill the specified >> +pointer >> + * with the port id of that port. >> + * @param devs The pointer to the ethdev structures >> + * @param port_id The pointer to the port id >> + * @return >> + * - 0 on success, negative on error >> + */ >> +extern int rte_eth_dev_get_changed_port( >> + struct rte_eth_dev *devs, uint8_t *port_id); >> + >> +/** >> + * Function for internal use by port hotplug functions. >> + * Returns a pci address of a ethdev specified by port identifier. >> + * @param port_id >> + * The port identifier of the Ethernet device >> + * @param addr >> + * The pointer to the pci address >> + * @return >> + * - 0 on success, negative on error >> + */ >> +extern int rte_eth_dev_get_addr_by_port( >> + uint8_t port_id, struct rte_pci_addr *addr); >> + >> +/** >> + * Function for internal use by port hotplug functions. >> + * Returns a port identifier of a ethdev specified by pci address. >> + * @param addr >> + * The pointer to the pci address of the Ethernet device. >> + * @param port_id >> + * The pointer to the port identifier >> + * @return >> + * - 0 on success, negative on error >> + */ >> +extern int rte_eth_dev_get_port_by_addr( >> + struct rte_pci_addr *addr, uint8_t *port_id); >> + >> +/** >> + * Function for internal use by port hotplug functions. >> + * Returns a unique identifier name of a ethdev specified by port identifier. >> + * @param port_id >> + * The port identifier. >> + * @param name >> + * The pointer to the Unique identifier name for each Ethernet device >> + * @return >> + * - 0 on success, negative on error >> + */ >> +extern int rte_eth_dev_get_name_by_port(uint8_t port_id, char *name); >> + >> +/** >> + * Function for internal use by port hotplug functions. >> + * Check whether or not, a PMD that is handling the ethdev specified by >> +port >> + * identifier can support detach function. >> + * @param port_id >> + * The port identifier >> + * @return >> + * - 0 on supporting detach function, negative on not supporting >> + */ >> +extern int rte_eth_dev_is_detachable(uint8_t port_id); >> + >> +/** >> + * Function for internal use by port hotplug functions. >> + * Returns a ethdev slot specified by the unique identifier name. >> + * @param name >> + * The pointer to the Unique identifier name for each Ethernet device >> + * @return >> + * - The pointer to the ethdev slot, on success. NULL on error >> + */ >> +extern struct rte_eth_dev *rte_eth_dev_allocated(const char *name); >> + >> +/** >> * Function for internal use by dummy drivers primarily, e.g. ring-based >> * driver. >> * Allocates a new ethdev slot for an ethernet device and returns the pointer diff --git >> a/lib/librte_ether/rte_ether_version.map b/lib/librte_ether/rte_ether_version.map >> index f66fd2d..099c769 100644 >> --- a/lib/librte_ether/rte_ether_version.map >> +++ b/lib/librte_ether/rte_ether_version.map >> @@ -6,6 +6,7 @@ DPDK_2.0 { >> rte_eth_allmulticast_enable; >> rte_eth_allmulticast_get; >> rte_eth_dev_allocate; >> + rte_eth_dev_allocated; >> rte_eth_dev_bypass_event_show; >> rte_eth_dev_bypass_event_store; >> rte_eth_dev_bypass_init; >> @@ -32,9 +33,14 @@ DPDK_2.0 { >> rte_eth_dev_filter_supported; >> rte_eth_dev_flow_ctrl_get; >> rte_eth_dev_flow_ctrl_set; >> + rte_eth_dev_get_addr_by_port; >> + rte_eth_dev_get_changed_port; > Hi Tetsuya, > > rte_eth_dev_get_device_type; > > needs to be added to rte_ether_version map to solve linking issue. Hi Bernard, Thanks. Could you please let me know how can I check this linking issue on my environment? I will add it on my test. Thanks, Tetsuya > > Regards, > > Bernard. > > >> rte_eth_dev_get_mtu; >> + rte_eth_dev_get_name_by_port; >> + rte_eth_dev_get_port_by_addr; >> rte_eth_dev_get_vlan_offload; >> rte_eth_dev_info_get; >> + rte_eth_dev_is_detachable; >> rte_eth_dev_mac_addr_add; >> rte_eth_dev_mac_addr_remove; >> rte_eth_dev_priority_flow_ctrl_set; >> @@ -44,6 +50,7 @@ DPDK_2.0 { >> rte_eth_dev_rss_reta_update; >> rte_eth_dev_rx_queue_start; >> rte_eth_dev_rx_queue_stop; >> + rte_eth_dev_save; >> rte_eth_dev_set_link_down; >> rte_eth_dev_set_link_up; >> rte_eth_dev_set_mtu; >> -- >> 1.9.1
> >> the pointer diff --git a/lib/librte_ether/rte_ether_version.map > >> b/lib/librte_ether/rte_ether_version.map > >> index f66fd2d..099c769 100644 > >> --- a/lib/librte_ether/rte_ether_version.map > >> +++ b/lib/librte_ether/rte_ether_version.map > >> @@ -6,6 +6,7 @@ DPDK_2.0 { > >> rte_eth_allmulticast_enable; > >> rte_eth_allmulticast_get; > >> rte_eth_dev_allocate; > >> + rte_eth_dev_allocated; > >> rte_eth_dev_bypass_event_show; > >> rte_eth_dev_bypass_event_store; > >> rte_eth_dev_bypass_init; > >> @@ -32,9 +33,14 @@ DPDK_2.0 { > >> rte_eth_dev_filter_supported; > >> rte_eth_dev_flow_ctrl_get; > >> rte_eth_dev_flow_ctrl_set; > >> + rte_eth_dev_get_addr_by_port; > >> + rte_eth_dev_get_changed_port; > > Hi Tetsuya, > > > > rte_eth_dev_get_device_type; > > > > needs to be added to rte_ether_version map to solve linking issue. > > Hi Bernard, > > Thanks. > Could you please let me know how can I check this linking issue on my environment? > I will add it on my test. > > Thanks, > Tetsuya > Hi Tetsuya, In config/common_linuxapp With "CONFIG_RTE_BUILD_SHARED_LIB=y" the follow linking error is occurring LD test /root/dpdk_sforge_2/x86_64-native-linuxapp-gcc/lib/librte_eal.so: undefined reference to `rte_eth_dev_get_device_type' collect2: error: ld returned 1 exit status make[5]: *** [test] Error 1 make[4]: *** [test] Error 2 make[3]: *** [app] Error 2 make[2]: *** [all] Error 2 make[1]: *** [x86_64-native-linuxapp-gcc_install] Error 2 make: *** [install] Error 2 Regards, Bernard. > > > > Regards, > > > > Bernard. > > > > > >> rte_eth_dev_get_mtu; > >> + rte_eth_dev_get_name_by_port; > >> + rte_eth_dev_get_port_by_addr; > >> rte_eth_dev_get_vlan_offload; > >> rte_eth_dev_info_get; > >> + rte_eth_dev_is_detachable; > >> rte_eth_dev_mac_addr_add; > >> rte_eth_dev_mac_addr_remove; > >> rte_eth_dev_priority_flow_ctrl_set; > >> @@ -44,6 +50,7 @@ DPDK_2.0 { > >> rte_eth_dev_rss_reta_update; > >> rte_eth_dev_rx_queue_start; > >> rte_eth_dev_rx_queue_stop; > >> + rte_eth_dev_save; > >> rte_eth_dev_set_link_down; > >> rte_eth_dev_set_link_up; > >> rte_eth_dev_set_mtu; > >> -- > >> 1.9.1
On 2015/02/23 20:39, Iremonger, Bernard wrote: >>>> the pointer diff --git a/lib/librte_ether/rte_ether_version.map >>>> b/lib/librte_ether/rte_ether_version.map >>>> index f66fd2d..099c769 100644 >>>> --- a/lib/librte_ether/rte_ether_version.map >>>> +++ b/lib/librte_ether/rte_ether_version.map >>>> @@ -6,6 +6,7 @@ DPDK_2.0 { >>>> rte_eth_allmulticast_enable; >>>> rte_eth_allmulticast_get; >>>> rte_eth_dev_allocate; >>>> + rte_eth_dev_allocated; >>>> rte_eth_dev_bypass_event_show; >>>> rte_eth_dev_bypass_event_store; >>>> rte_eth_dev_bypass_init; >>>> @@ -32,9 +33,14 @@ DPDK_2.0 { >>>> rte_eth_dev_filter_supported; >>>> rte_eth_dev_flow_ctrl_get; >>>> rte_eth_dev_flow_ctrl_set; >>>> + rte_eth_dev_get_addr_by_port; >>>> + rte_eth_dev_get_changed_port; >>> Hi Tetsuya, >>> >>> rte_eth_dev_get_device_type; >>> >>> needs to be added to rte_ether_version map to solve linking issue. >> Hi Bernard, >> >> Thanks. >> Could you please let me know how can I check this linking issue on my environment? >> I will add it on my test. >> >> Thanks, >> Tetsuya >> > Hi Tetsuya, > > In config/common_linuxapp > With "CONFIG_RTE_BUILD_SHARED_LIB=y" the follow linking error is occurring > > LD test > /root/dpdk_sforge_2/x86_64-native-linuxapp-gcc/lib/librte_eal.so: undefined reference to `rte_eth_dev_get_device_type' > collect2: error: ld returned 1 exit status > make[5]: *** [test] Error 1 > make[4]: *** [test] Error 2 > make[3]: *** [app] Error 2 > make[2]: *** [all] Error 2 > make[1]: *** [x86_64-native-linuxapp-gcc_install] Error 2 > make: *** [install] Error 2 Okay, I could reproduce it. Thanks, Tetsuya > Regards, > > Bernard. >>> Regards, >>> >>> Bernard. >>> >>> >>>> rte_eth_dev_get_mtu; >>>> + rte_eth_dev_get_name_by_port; >>>> + rte_eth_dev_get_port_by_addr; >>>> rte_eth_dev_get_vlan_offload; >>>> rte_eth_dev_info_get; >>>> + rte_eth_dev_is_detachable; >>>> rte_eth_dev_mac_addr_add; >>>> rte_eth_dev_mac_addr_remove; >>>> rte_eth_dev_priority_flow_ctrl_set; >>>> @@ -44,6 +50,7 @@ DPDK_2.0 { >>>> rte_eth_dev_rss_reta_update; >>>> rte_eth_dev_rx_queue_start; >>>> rte_eth_dev_rx_queue_stop; >>>> + rte_eth_dev_save; >>>> rte_eth_dev_set_link_down; >>>> rte_eth_dev_set_link_up; >>>> rte_eth_dev_set_mtu; >>>> -- >>>> 1.9.1
This patch series adds a dynamic port hotplug framework to DPDK. With the patches, DPDK apps can attach or detach ports at runtime. The basic concept of the port hotplug is like followings. - DPDK apps must have responsibility to manage ports. DPDK apps only know which ports are attached or detached at the moment. The port hotplug framework is implemented to allow DPDK apps to manage ports. For example, when DPDK apps call port attach function, attached port number will be returned. Also, DPDK apps can detach port by port number. - Kernel support is needed for attaching or detaching physical device ports. To attach a new physical device port, the device will be recognized by userspace directly I/O framework in kernel at first. Then DPDK apps can call the port hotplug functions to attach ports. For detaching, steps are vice versa. - Before detach ports, ports must be stopped and closed. DPDK application must call rte_eth_dev_stop() and rte_eth_dev_close() before detaching ports. These function will call finalization codes of PMDs. But so far, no PMD frees all resources allocated by initialization. It means PMDs are needed to be fixed to support the port hotplug. 'RTE_PCI_DRV_DETACHABLE' is a new flag indicating a PMD supports detaching. Without this flag, detaching will be failed. - Mustn't affect legacy DPDK apps. No DPDK EAL behavior is changed, if the port hotplug functions are't called. So all legacy DPDK apps can still work without modifications. And a few limitations. - The port hotplug functions are not thread safe. DPDK apps should handle it. - Only support Linux and igb_uio so far. BSD and VFIO is not supported. I will send VFIO patches at least, but I don't have a plan to submit BSD patch so far. Here is port hotplug APIs. ------------------------------------------------------------------------------- /** * Attach a new device. * * @param devargs * A pointer to a strings array describing the new device * to be attached. The strings should be a pci address like * '0000:01:00.0' or virtual device name like 'eth_pcap0'. * @param port_id * A pointer to a port identifier actually attached. * @return * 0 on success and port_id is filled, negative on error */ int rte_eal_dev_attach(const char *devargs, uint8_t *port_id); /** * Detach a device. * * @param port_id * The port identifier of the device to detach. * @param addr * A pointer to a device name actually detached. * @return * 0 on success and devname is filled, negative on error */ int rte_eal_dev_detach(uint8_t port_id, char *devname); ------------------------------------------------------------------------------- This patch series are for DPDK EAL. To use port hotplug function by DPDK apps, each PMD should be fixed to support 'RTE_PCI_DRV_DETACHABLE' flag. Please check a patch for pcap PMD. Also, please check testpmd patch. It will show you how to fix your legacy applications to support port hotplug feature. PATCH v12 changes - Add missing symbol in version map. (Thanks to Iremonger, Bernard) PATCH v11 changes - Remove needless devargs handling codes. - Replace get_vdev_name() by rte_eal_parse_devargs_str(). - Replace rte_eal_vdev_find_and_init by rte_eal_vdev_init() - Replace rte_eal_vdev_find_and_uninit by rte_eal_vdev_uninit() - Fix rte_eal_dev_init() to use rte_eal_vdev_init(). - Remove needless patch. (Thanks to Maxime Leroy) PATCH v10 changes - Add comments. - Chagne order of version.map. - Fix comment of "rte_ethdev.h". (Thanks to Thomas Monjalon) - Add size parameter to rte_eth_dev_create_unique_device_name(). (Thanks to Iremonger, Bernard) PATCH v9 changes - Fix commit title. - Fix commit log. - Fix comments. - Define CONFIG_RTE_LIBRTE_EAL_HOTPLUG at the top of this patch series. - DEV_INVALID/VALID are removed. - DEV_DISCONNECTED is replaced by DEV_DETACHED. - DEV_CONNECTED is replaced by DEV_ATTACHED. - rte_eth_dev_allocate_new_port() is renamed to rte_eth_dev_find_free_port(). - rte_eth_dev_validate_port() is renamed to rte_eth_dev_is_valid_port(). - rte_eth_dev_is_valid_port() is changed not to handle log toggle. - eal_compare_pci_addr() is replaced by rte_eal_compare_pci_addr(). - rte_eth_dev_free() is replaced by rte_eth_dev_release_port(). - Add a function to create a unique device name. - Change parameter of pci_devuninit_t and rte_eth_dev_uninit. - Remove code that initiaize callback of ethdev from rte_eth_dev_uninit(). - Remove pci_unmap_device(). It will be implemented in later patch. - rte_eth_dev_check_detachable() is replaced by rte_eth_dev_is_detachable(). - strncpy() is replaced by strcpy(). - Implement pci_unmap_device() in this patch. - Remove "rte_dev_hotplug.h". - Remove needless "#ifdef". - Remove RTE_EAL_INVOKE_TYPE_PROBE/CLOSE. - RTE_ETH_DEV_PHYSICAL is replaced by RTE_ETH_DEV_PCI. - Use strcmp() instead of strncmp(). - Remove RTE_EAL_INVOKE_TYPE_PROBE/CLOSE. (Thanks to Thomas Monjalon) - Change definition of rte_dev_uninit_t. (Thanks to Thomas Monjalon and Maxime Leroy) - Add missing symbol in version map. (Thanks to Nail Horman) PATCH v8 changes - Fix Makefile and add version map file. - Add missing symbol in version map. - Fix pci_scan_one() to update sysfs values. (Thanks to Qiu, Michael and Iremonger, Bernard) - NONE_TRACE is replaced by NO_TRACE. - Fix typo. - Add size parameter to rte_eth_dev_save(). (Thanks to Iremonger, Bernard) PATCH v7 changes - Add a new section to programmer's guide. (Thanks to Iremonger, Bernard) - Fix port checking implementation of star_port(). - Fix typo of warning messages. - Add pt_driver checking to rte_eth_dev_check_detachable(). (Thanks to Qiu, Michael) PATCH v6 changes - Fix rte_eth_dev_uninit() to handle a return value of uninit function of PMD. To do this, below changes also be applied. - Fix a parameter of rte_eth_dev_free(). - Use rte_eth_dev structure as the paramter of rte_eth_dev_free(). PATCH v5 changes - Add runtime check passthrough driver type, like vfio-pci, igb_uio and uio_pci_generic. This was done by Qiu, Michael. Thanks a lot. - Change function names like below. - rte_eal_dev_find_and_invoke() to rte_eal_vdev_find_and_invoke(). - rte_eal_dev_invoke() to rte_eal_vdev_invoke(). - Add code to handle a return value of rte_eal_devargs_remove(). - Fix pci address format in rte_eal_dev_detach(). - Remove RTE_EAL_INVOKE_TYPE_UNKNOWN, because it's unused. - Change function definition of rte_eal_devargs_remove(). - Fix pci_unmap_device() to check pt_driver. - Fix return value of below functions. - rte_eth_dev_get_changed_port(). - rte_eth_dev_get_port_by_addr(). - Change paramters of rte_eth_dev_validate_port() to cleanup code. - Fix pci_scan_one to handle pt_driver correctly. (Thanks to Qiu, Michael for above suggestions) PATCH v4 changes - Merge patches to review easier. - Fix indent of 'if' statement. - Fix calculation method of eal_compare_pci_addr(). - Fix header file declaration. - Add header file to determine if hotplug can be enabled. (Thanks to Qiu, Michael) - Use braces with 'for' loop. - Add parameter checking. - Fix sanity check code - Fix comments of rte_eth_dev_type. - Change function names. (Thanks to Iremonger, Bernard) PATCH v3 changes: - Fix enum definition used in rte_ethdev.c. (Thanks to Zhang, Helin) PATCH v2 changes: - Replace rte_eal_dev_attach_pdev(), rte_eal_dev_detach_pdev, rte_eal_dev_attach_vdev() and rte_eal_dev_detach_vdev() to rte_eal_dev_attach() and rte_eal_dev_detach(). - Add parameter values checking. - Refashion a few functions. (Thanks to Iremonger, Bernard) PATCH v1 Changes: - Fix error checking code of librte_eth APIs. - Fix issue that port from pcap PMD cannot be detached correctly. - Fix issue that testpmd could hang after forwarding, if attaching and detaching is repeatedly. - Fix if-condition of rte_eth_dev_get_port_by_addr(). (Thanks to Mark Enright) RFC PATCH v2 Changes: - remove 'rte_eth_dev_validate_port()', and cleanup codes. Michael Qiu (2): eal_pci: Add flag to hold kernel driver type eal_pci: pci memory map work with driver type Tetsuya Mukawa (11): eal: Enable port Hotplug framework in Linux eal/pci,ethdev: Remove assumption that port will not be detached eal/pci: Consolidate pci address comparison APIs ethdev: Add rte_eth_dev_release_port to release specified port eal,ethdev: Add a function and function pointers to close ether device ethdev: Add functions that will be used by port hotplug functions eal/linux/pci: Add functions for unmapping igb_uio resources eal/pci: Add probe and close functions of pci driver ethdev: Add one dev_type parameter to rte_eth_dev_allocate eal/pci: Add rte_eal_dev_attach/detach() functions doc: Add port hotplug framework section to programmers guide app/test/virtual_pmd.c | 2 +- config/common_bsdapp | 6 + config/common_linuxapp | 5 + doc/guides/prog_guide/index.rst | 1 + doc/guides/prog_guide/port_hotplug_framework.rst | 110 ++++++ lib/librte_eal/bsdapp/eal/eal_pci.c | 29 +- lib/librte_eal/common/eal_common_dev.c | 285 +++++++++++++- lib/librte_eal/common/eal_common_devargs.c | 46 ++- lib/librte_eal/common/eal_common_pci.c | 100 ++++- lib/librte_eal/common/eal_private.h | 26 ++ lib/librte_eal/common/include/rte_dev.h | 33 ++ lib/librte_eal/common/include/rte_devargs.h | 28 ++ lib/librte_eal/common/include/rte_pci.h | 82 +++++ lib/librte_eal/linuxapp/eal/Makefile | 1 + lib/librte_eal/linuxapp/eal/eal_pci.c | 230 ++++++++++-- lib/librte_eal/linuxapp/eal/eal_pci_init.h | 7 + lib/librte_eal/linuxapp/eal/eal_pci_uio.c | 67 +++- lib/librte_eal/linuxapp/eal/rte_eal_version.map | 2 + lib/librte_ether/rte_ethdev.c | 451 ++++++++++++++++++----- lib/librte_ether/rte_ethdev.h | 149 +++++++- lib/librte_ether/rte_ether_version.map | 8 + lib/librte_pmd_af_packet/rte_eth_af_packet.c | 2 +- lib/librte_pmd_bond/rte_eth_bond_api.c | 2 +- lib/librte_pmd_pcap/rte_eth_pcap.c | 2 +- lib/librte_pmd_ring/rte_eth_ring.c | 2 +- lib/librte_pmd_xenvirt/rte_eth_xenvirt.c | 2 +- 26 files changed, 1491 insertions(+), 187 deletions(-) create mode 100644 doc/guides/prog_guide/port_hotplug_framework.rst -- 1.9.1
The patch adds CONFIG_RTE_LIBRTE_EAL_HOTPLUG in Linux and BSD configuration. So far, Hotplug functions only support linux. v9: - Move this patch at the top of this patch series. (Thanks to Thomas Monjalon) Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp> --- config/common_bsdapp | 6 ++++++ config/common_linuxapp | 5 +++++ 2 files changed, 11 insertions(+) diff --git a/config/common_bsdapp b/config/common_bsdapp index 4c0cfc0..c24f687 100644 --- a/config/common_bsdapp +++ b/config/common_bsdapp @@ -116,6 +116,12 @@ CONFIG_RTE_LIBRTE_EAL_BSDAPP=y CONFIG_RTE_LIBRTE_EAL_LINUXAPP=n # +# Compile Environment Abstraction Layer to support hotplug +# So far, Hotplug functions only support linux +# +CONFIG_RTE_LIBRTE_EAL_HOTPLUG=n + +# # Compile Environment Abstraction Layer to support Vmware TSC map # CONFIG_RTE_LIBRTE_EAL_VMWARE_TSC_MAP_SUPPORT=y diff --git a/config/common_linuxapp b/config/common_linuxapp index 0234236..d66b008 100644 --- a/config/common_linuxapp +++ b/config/common_linuxapp @@ -114,6 +114,11 @@ CONFIG_RTE_PCI_MAX_READ_REQUEST_SIZE=0 CONFIG_RTE_LIBRTE_EAL_LINUXAPP=y # +# Compile Environment Abstraction Layer to support hotplug +# +CONFIG_RTE_LIBRTE_EAL_HOTPLUG=y + +# # Compile Environment Abstraction Layer to support Vmware TSC map # CONFIG_RTE_LIBRTE_EAL_VMWARE_TSC_MAP_SUPPORT=y -- 1.9.1
From: Michael Qiu <michael.qiu@intel.com> Currently, dpdk has no ability to know which type of driver( vfio-pci/igb_uio/uio_pci_generic) the device used. It only can check whether vfio is enabled or not staticly. It really useful to have the flag, becasue different type need to handle differently in runtime. For example, pci memory map, pot hotplug, and so on. This patch add a flag field for pci device to solve above issue. Signed-off-by: Michael Qiu <michael.qiu@intel.com> Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp> --- lib/librte_eal/common/include/rte_pci.h | 8 +++++ lib/librte_eal/linuxapp/eal/eal_pci.c | 53 +++++++++++++++++++++++++++++++-- 2 files changed, 59 insertions(+), 2 deletions(-) diff --git a/lib/librte_eal/common/include/rte_pci.h b/lib/librte_eal/common/include/rte_pci.h index 4301c16..5e0ba00 100644 --- a/lib/librte_eal/common/include/rte_pci.h +++ b/lib/librte_eal/common/include/rte_pci.h @@ -142,6 +142,13 @@ struct rte_pci_addr { struct rte_devargs; +enum rte_pt_driver { + RTE_PT_UNKNOWN = 0, + RTE_PT_IGB_UIO = 1, + RTE_PT_VFIO = 2, + RTE_PT_UIO_GENERIC = 3, +}; + /** * A structure describing a PCI device. */ @@ -155,6 +162,7 @@ struct rte_pci_device { uint16_t max_vfs; /**< sriov enable if not zero */ int numa_node; /**< NUMA node connection */ struct rte_devargs *devargs; /**< Device user arguments */ + enum rte_pt_driver pt_driver; /**< Driver of passthrough */ }; /** Any PCI device identifier (vendor, device, ...) */ diff --git a/lib/librte_eal/linuxapp/eal/eal_pci.c b/lib/librte_eal/linuxapp/eal/eal_pci.c index 63bcbce..9fe2851 100644 --- a/lib/librte_eal/linuxapp/eal/eal_pci.c +++ b/lib/librte_eal/linuxapp/eal/eal_pci.c @@ -97,6 +97,35 @@ error: return -1; } +static int +pci_get_kernel_driver_by_path(const char *filename, char *dri_name) +{ + int count; + char path[PATH_MAX]; + char *name; + + if (!filename || !dri_name) + return -1; + + count = readlink(filename, path, PATH_MAX); + if (count >= PATH_MAX) + return -1; + + /* For device does not have a driver */ + if (count < 0) + return 1; + + path[count] = '\0'; + + name = strrchr(path, '/'); + if (name) { + strncpy(dri_name, name + 1, strlen(name + 1) + 1); + return 0; + } + + return -1; +} + void * pci_find_max_end_va(void) { @@ -220,11 +249,12 @@ pci_scan_one(const char *dirname, uint16_t domain, uint8_t bus, char filename[PATH_MAX]; unsigned long tmp; struct rte_pci_device *dev; + char driver[PATH_MAX]; + int ret; dev = malloc(sizeof(*dev)); - if (dev == NULL) { + if (dev == NULL) return -1; - } memset(dev, 0, sizeof(*dev)); dev->addr.domain = domain; @@ -303,6 +333,25 @@ pci_scan_one(const char *dirname, uint16_t domain, uint8_t bus, return -1; } + /* parse driver */ + snprintf(filename, sizeof(filename), "%s/driver", dirname); + ret = pci_get_kernel_driver_by_path(filename, driver); + if (!ret) { + if (!strcmp(driver, "vfio-pci")) + dev->pt_driver = RTE_PT_VFIO; + else if (!strcmp(driver, "igb_uio")) + dev->pt_driver = RTE_PT_IGB_UIO; + else if (!strcmp(driver, "uio_pci_generic")) + dev->pt_driver = RTE_PT_UIO_GENERIC; + else + dev->pt_driver = RTE_PT_UNKNOWN; + } else if (ret < 0) { + RTE_LOG(ERR, EAL, "Fail to get kernel driver\n"); + free(dev); + return -1; + } else + dev->pt_driver = RTE_PT_UNKNOWN; + /* device is valid, add in list (sorted) */ if (TAILQ_EMPTY(&pci_device_list)) { TAILQ_INSERT_TAIL(&pci_device_list, dev, next); -- 1.9.1
From: Michael Qiu <michael.qiu@intel.com> With the driver type flag in struct rte_pci_dev, we do not need to always map uio devices with vfio related function when vfio enabled. Signed-off-by: Michael Qiu <michael.qiu@intel.com> Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp> --- lib/librte_eal/linuxapp/eal/eal_pci.c | 30 +++++++++++++++++------------- 1 file changed, 17 insertions(+), 13 deletions(-) diff --git a/lib/librte_eal/linuxapp/eal/eal_pci.c b/lib/librte_eal/linuxapp/eal/eal_pci.c index 9fe2851..c04f897 100644 --- a/lib/librte_eal/linuxapp/eal/eal_pci.c +++ b/lib/librte_eal/linuxapp/eal/eal_pci.c @@ -554,25 +554,29 @@ pci_config_space_set(struct rte_pci_device *dev) static int pci_map_device(struct rte_pci_device *dev) { - int ret, mapped = 0; + int ret = -1; /* try mapping the NIC resources using VFIO if it exists */ + switch (dev->pt_driver) { + case RTE_PT_VFIO: #ifdef VFIO_PRESENT - if (pci_vfio_is_enabled()) { - ret = pci_vfio_map_resource(dev); - if (ret == 0) - mapped = 1; - else if (ret < 0) - return ret; - } + if (pci_vfio_is_enabled()) + ret = pci_vfio_map_resource(dev); #endif - /* map resources for devices that use uio_pci_generic or igb_uio */ - if (!mapped) { + break; + case RTE_PT_IGB_UIO: + case RTE_PT_UIO_GENERIC: + /* map resources for devices that use uio */ ret = pci_uio_map_resource(dev); - if (ret != 0) - return ret; + break; + default: + RTE_LOG(DEBUG, EAL, " Not managed by known pt driver," + " skipped\n"); + ret = 1; + break; } - return 0; + + return ret; } /* -- 1.9.1
To remove assumption, do like followings. This patch adds "RTE_PCI_DRV_DETACHABLE" to drv_flags of rte_pci_driver structure. The flags indicate the driver can detach devices at runtime. Also, remove assumption that port will not be detached. To remove the assumption. - Add 'attached' member to rte_eth_dev structure. This member is used for indicating the port is attached, or not. DEV_ATTACHED indicates a port is attached. DEV_DETACHED indicates a port is detached. - Add rte_eth_dev_allocate_new_port(). This function is used for allocating new port. v9: - DEV_INVALID/VALID are removed. - DEV_DISCONNECTED is replaced by DEV_DETACHED. - DEV_CONNECTED is replaced by DEV_ATTACHED. - rte_eth_dev_allocate_new_port() is renamed to rte_eth_dev_find_free_port(). - rte_eth_dev_validate_port() is renamed to rte_eth_dev_is_valid_port(). - rte_eth_dev_is_valid_port() is changed not to handle log toggle. - Fix commit log to describe DEV_ATACHED and DEV_DETACHED. (Thanks to Thomas Monjalon) v8: - NONE_TRACE is changed to NO_TRACE. (Thanks to Iremonger, Bernard) v5: - Change parameters of rte_eth_dev_validate_port() to cleanup code. v4: - Use braces with 'for' loop. - Fix indent of 'if' statement. Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp> --- lib/librte_eal/common/include/rte_pci.h | 2 + lib/librte_ether/rte_ethdev.c | 248 ++++++++++++++++++++------------ lib/librte_ether/rte_ethdev.h | 5 + 3 files changed, 164 insertions(+), 91 deletions(-) diff --git a/lib/librte_eal/common/include/rte_pci.h b/lib/librte_eal/common/include/rte_pci.h index 5e0ba00..ffd13d9 100644 --- a/lib/librte_eal/common/include/rte_pci.h +++ b/lib/librte_eal/common/include/rte_pci.h @@ -210,6 +210,8 @@ struct rte_pci_driver { #define RTE_PCI_DRV_FORCE_UNBIND 0x0004 /** Device driver supports link state interrupt */ #define RTE_PCI_DRV_INTR_LSC 0x0008 +/** Device driver supports detaching capability */ +#define RTE_PCI_DRV_DETACHABLE 0x0010 /**< Internal use only - Macro used by pci addr parsing functions **/ #define GET_PCIADDR_FIELD(in, fd, lim, dlm) \ diff --git a/lib/librte_ether/rte_ethdev.c b/lib/librte_ether/rte_ethdev.c index 27bbb0b..0e1e5c9 100644 --- a/lib/librte_ether/rte_ethdev.c +++ b/lib/librte_ether/rte_ethdev.c @@ -175,6 +175,11 @@ enum { STAT_QMAP_RX }; +enum { + DEV_DETACHED = 0, + DEV_ATTACHED +}; + static inline void rte_eth_dev_data_alloc(void) { @@ -201,19 +206,34 @@ rte_eth_dev_allocated(const char *name) { unsigned i; - for (i = 0; i < nb_ports; i++) { - if (strcmp(rte_eth_devices[i].data->name, name) == 0) + for (i = 0; i < RTE_MAX_ETHPORTS; i++) { + if ((rte_eth_devices[i].attached == DEV_ATTACHED) && + strcmp(rte_eth_devices[i].data->name, name) == 0) return &rte_eth_devices[i]; } return NULL; } +static uint8_t +rte_eth_dev_find_free_port(void) +{ + unsigned i; + + for (i = 0; i < RTE_MAX_ETHPORTS; i++) { + if (rte_eth_devices[i].attached == DEV_DETACHED) + return i; + } + return RTE_MAX_ETHPORTS; +} + struct rte_eth_dev * rte_eth_dev_allocate(const char *name) { + uint8_t port_id; struct rte_eth_dev *eth_dev; - if (nb_ports == RTE_MAX_ETHPORTS) { + port_id = rte_eth_dev_find_free_port(); + if (port_id == RTE_MAX_ETHPORTS) { PMD_DEBUG_TRACE("Reached maximum number of Ethernet ports\n"); return NULL; } @@ -226,10 +246,12 @@ rte_eth_dev_allocate(const char *name) return NULL; } - eth_dev = &rte_eth_devices[nb_ports]; - eth_dev->data = &rte_eth_dev_data[nb_ports]; + eth_dev = &rte_eth_devices[port_id]; + eth_dev->data = &rte_eth_dev_data[port_id]; snprintf(eth_dev->data->name, sizeof(eth_dev->data->name), "%s", name); - eth_dev->data->port_id = nb_ports++; + eth_dev->data->port_id = port_id; + eth_dev->attached = DEV_ATTACHED; + nb_ports++; return eth_dev; } @@ -283,6 +305,7 @@ rte_eth_dev_init(struct rte_pci_driver *pci_drv, (unsigned) pci_dev->id.device_id); if (rte_eal_process_type() == RTE_PROC_PRIMARY) rte_free(eth_dev->data->dev_private); + eth_dev->attached = DEV_DETACHED; nb_ports--; return diag; } @@ -308,10 +331,20 @@ rte_eth_driver_register(struct eth_driver *eth_drv) rte_eal_pci_register(ð_drv->pci_drv); } +static int +rte_eth_dev_is_valid_port(uint8_t port_id) +{ + if (port_id >= RTE_MAX_ETHPORTS || + rte_eth_devices[port_id].attached != DEV_ATTACHED) + return 0; + else + return 1; +} + int rte_eth_dev_socket_id(uint8_t port_id) { - if (port_id >= nb_ports) + if (!rte_eth_dev_is_valid_port(port_id)) return -1; return rte_eth_devices[port_id].pci_dev->numa_node; } @@ -369,7 +402,7 @@ rte_eth_dev_rx_queue_start(uint8_t port_id, uint16_t rx_queue_id) * in a multi-process setup*/ PROC_PRIMARY_OR_ERR_RET(-E_RTE_SECONDARY); - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return -EINVAL; } @@ -395,7 +428,7 @@ rte_eth_dev_rx_queue_stop(uint8_t port_id, uint16_t rx_queue_id) * in a multi-process setup*/ PROC_PRIMARY_OR_ERR_RET(-E_RTE_SECONDARY); - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return -EINVAL; } @@ -421,7 +454,7 @@ rte_eth_dev_tx_queue_start(uint8_t port_id, uint16_t tx_queue_id) * in a multi-process setup*/ PROC_PRIMARY_OR_ERR_RET(-E_RTE_SECONDARY); - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return -EINVAL; } @@ -447,7 +480,7 @@ rte_eth_dev_tx_queue_stop(uint8_t port_id, uint16_t tx_queue_id) * in a multi-process setup*/ PROC_PRIMARY_OR_ERR_RET(-E_RTE_SECONDARY); - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return -EINVAL; } @@ -703,10 +736,11 @@ rte_eth_dev_configure(uint8_t port_id, uint16_t nb_rx_q, uint16_t nb_tx_q, * in a multi-process setup*/ PROC_PRIMARY_OR_ERR_RET(-E_RTE_SECONDARY); - if (port_id >= nb_ports || port_id >= RTE_MAX_ETHPORTS) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-EINVAL); } + dev = &rte_eth_devices[port_id]; FUNC_PTR_OR_ERR_RET(*dev->dev_ops->dev_infos_get, -ENOTSUP); @@ -888,10 +922,11 @@ rte_eth_dev_start(uint8_t port_id) * in a multi-process setup*/ PROC_PRIMARY_OR_ERR_RET(-E_RTE_SECONDARY); - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%" PRIu8 "\n", port_id); + if (!rte_eth_dev_is_valid_port(port_id)) { + PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-EINVAL); } + dev = &rte_eth_devices[port_id]; FUNC_PTR_OR_ERR_RET(*dev->dev_ops->dev_start, -ENOTSUP); @@ -927,10 +962,11 @@ rte_eth_dev_stop(uint8_t port_id) * in a multi-process setup*/ PROC_PRIMARY_OR_RET(); - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%" PRIu8 "\n", port_id); + if (!rte_eth_dev_is_valid_port(port_id)) { + PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return; } + dev = &rte_eth_devices[port_id]; FUNC_PTR_OR_RET(*dev->dev_ops->dev_stop); @@ -955,10 +991,11 @@ rte_eth_dev_set_link_up(uint8_t port_id) * in a multi-process setup*/ PROC_PRIMARY_OR_ERR_RET(-E_RTE_SECONDARY); - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return -EINVAL; } + dev = &rte_eth_devices[port_id]; FUNC_PTR_OR_ERR_RET(*dev->dev_ops->dev_set_link_up, -ENOTSUP); @@ -974,10 +1011,11 @@ rte_eth_dev_set_link_down(uint8_t port_id) * in a multi-process setup*/ PROC_PRIMARY_OR_ERR_RET(-E_RTE_SECONDARY); - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return -EINVAL; } + dev = &rte_eth_devices[port_id]; FUNC_PTR_OR_ERR_RET(*dev->dev_ops->dev_set_link_down, -ENOTSUP); @@ -993,7 +1031,7 @@ rte_eth_dev_close(uint8_t port_id) * in a multi-process setup*/ PROC_PRIMARY_OR_RET(); - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return; } @@ -1021,10 +1059,11 @@ rte_eth_rx_queue_setup(uint8_t port_id, uint16_t rx_queue_id, * in a multi-process setup*/ PROC_PRIMARY_OR_ERR_RET(-E_RTE_SECONDARY); - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-EINVAL); } + dev = &rte_eth_devices[port_id]; if (rx_queue_id >= dev->data->nb_rx_queues) { PMD_DEBUG_TRACE("Invalid RX queue_id=%d\n", rx_queue_id); @@ -1094,10 +1133,11 @@ rte_eth_tx_queue_setup(uint8_t port_id, uint16_t tx_queue_id, * in a multi-process setup*/ PROC_PRIMARY_OR_ERR_RET(-E_RTE_SECONDARY); - if (port_id >= RTE_MAX_ETHPORTS || port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-EINVAL); } + dev = &rte_eth_devices[port_id]; if (tx_queue_id >= dev->data->nb_tx_queues) { PMD_DEBUG_TRACE("Invalid TX queue_id=%d\n", tx_queue_id); @@ -1127,10 +1167,11 @@ rte_eth_promiscuous_enable(uint8_t port_id) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return; } + dev = &rte_eth_devices[port_id]; FUNC_PTR_OR_RET(*dev->dev_ops->promiscuous_enable); @@ -1143,10 +1184,11 @@ rte_eth_promiscuous_disable(uint8_t port_id) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return; } + dev = &rte_eth_devices[port_id]; FUNC_PTR_OR_RET(*dev->dev_ops->promiscuous_disable); @@ -1159,7 +1201,7 @@ rte_eth_promiscuous_get(uint8_t port_id) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return -1; } @@ -1173,10 +1215,11 @@ rte_eth_allmulticast_enable(uint8_t port_id) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return; } + dev = &rte_eth_devices[port_id]; FUNC_PTR_OR_RET(*dev->dev_ops->allmulticast_enable); @@ -1189,10 +1232,11 @@ rte_eth_allmulticast_disable(uint8_t port_id) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return; } + dev = &rte_eth_devices[port_id]; FUNC_PTR_OR_RET(*dev->dev_ops->allmulticast_disable); @@ -1205,7 +1249,7 @@ rte_eth_allmulticast_get(uint8_t port_id) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return -1; } @@ -1233,10 +1277,11 @@ rte_eth_link_get(uint8_t port_id, struct rte_eth_link *eth_link) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return; } + dev = &rte_eth_devices[port_id]; if (dev->data->dev_conf.intr_conf.lsc != 0) @@ -1253,10 +1298,11 @@ rte_eth_link_get_nowait(uint8_t port_id, struct rte_eth_link *eth_link) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return; } + dev = &rte_eth_devices[port_id]; if (dev->data->dev_conf.intr_conf.lsc != 0) @@ -1273,10 +1319,11 @@ rte_eth_stats_get(uint8_t port_id, struct rte_eth_stats *stats) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-ENODEV); } + dev = &rte_eth_devices[port_id]; memset(stats, 0, sizeof(*stats)); @@ -1291,10 +1338,11 @@ rte_eth_stats_reset(uint8_t port_id) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return; } + dev = &rte_eth_devices[port_id]; FUNC_PTR_OR_RET(*dev->dev_ops->stats_reset); @@ -1312,10 +1360,11 @@ rte_eth_xstats_get(uint8_t port_id, struct rte_eth_xstats *xstats, uint64_t val; char *stats_ptr; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return -1; } + dev = &rte_eth_devices[port_id]; /* implemented by the driver */ @@ -1381,10 +1430,11 @@ rte_eth_xstats_reset(uint8_t port_id) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return; } + dev = &rte_eth_devices[port_id]; /* implemented by the driver */ @@ -1403,10 +1453,11 @@ set_queue_stats_mapping(uint8_t port_id, uint16_t queue_id, uint8_t stat_idx, { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return -ENODEV; } + dev = &rte_eth_devices[port_id]; FUNC_PTR_OR_ERR_RET(*dev->dev_ops->queue_stats_mapping_set, -ENOTSUP); @@ -1438,10 +1489,11 @@ rte_eth_dev_info_get(uint8_t port_id, struct rte_eth_dev_info *dev_info) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return; } + dev = &rte_eth_devices[port_id]; memset(dev_info, 0, sizeof(struct rte_eth_dev_info)); @@ -1458,10 +1510,11 @@ rte_eth_macaddr_get(uint8_t port_id, struct ether_addr *mac_addr) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return; } + dev = &rte_eth_devices[port_id]; ether_addr_copy(&dev->data->mac_addrs[0], mac_addr); } @@ -1472,7 +1525,7 @@ rte_eth_dev_get_mtu(uint8_t port_id, uint16_t *mtu) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-ENODEV); } @@ -1488,7 +1541,7 @@ rte_eth_dev_set_mtu(uint8_t port_id, uint16_t mtu) int ret; struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-ENODEV); } @@ -1508,10 +1561,11 @@ rte_eth_dev_vlan_filter(uint8_t port_id, uint16_t vlan_id, int on) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-ENODEV); } + dev = &rte_eth_devices[port_id]; if (! (dev->data->dev_conf.rxmode.hw_vlan_filter)) { PMD_DEBUG_TRACE("port %d: vlan-filtering disabled\n", port_id); @@ -1533,7 +1587,7 @@ rte_eth_dev_set_vlan_strip_on_queue(uint8_t port_id, uint16_t rx_queue_id, int o { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-ENODEV); } @@ -1555,7 +1609,7 @@ rte_eth_dev_set_vlan_ether_type(uint8_t port_id, uint16_t tpid) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-ENODEV); } @@ -1575,7 +1629,7 @@ rte_eth_dev_set_vlan_offload(uint8_t port_id, int offload_mask) int mask = 0; int cur, org = 0; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-ENODEV); } @@ -1620,7 +1674,7 @@ rte_eth_dev_get_vlan_offload(uint8_t port_id) struct rte_eth_dev *dev; int ret = 0; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-ENODEV); } @@ -1644,10 +1698,11 @@ rte_eth_dev_set_vlan_pvid(uint8_t port_id, uint16_t pvid, int on) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-ENODEV); } + dev = &rte_eth_devices[port_id]; FUNC_PTR_OR_ERR_RET(*dev->dev_ops->vlan_pvid_set, -ENOTSUP); (*dev->dev_ops->vlan_pvid_set)(dev, pvid, on); @@ -1662,7 +1717,7 @@ rte_eth_dev_fdir_add_signature_filter(uint8_t port_id, { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-ENODEV); } @@ -1696,7 +1751,7 @@ rte_eth_dev_fdir_update_signature_filter(uint8_t port_id, { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-ENODEV); } @@ -1730,7 +1785,7 @@ rte_eth_dev_fdir_remove_signature_filter(uint8_t port_id, { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-ENODEV); } @@ -1761,7 +1816,7 @@ rte_eth_dev_fdir_get_infos(uint8_t port_id, struct rte_eth_fdir *fdir) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-ENODEV); } @@ -1786,7 +1841,7 @@ rte_eth_dev_fdir_add_perfect_filter(uint8_t port_id, { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-ENODEV); } @@ -1826,7 +1881,7 @@ rte_eth_dev_fdir_update_perfect_filter(uint8_t port_id, { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-ENODEV); } @@ -1864,7 +1919,7 @@ rte_eth_dev_fdir_remove_perfect_filter(uint8_t port_id, { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-ENODEV); } @@ -1900,7 +1955,7 @@ rte_eth_dev_fdir_set_masks(uint8_t port_id, struct rte_fdir_masks *fdir_mask) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-ENODEV); } @@ -1920,7 +1975,7 @@ rte_eth_dev_flow_ctrl_get(uint8_t port_id, struct rte_eth_fc_conf *fc_conf) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-ENODEV); } @@ -1936,7 +1991,7 @@ rte_eth_dev_flow_ctrl_set(uint8_t port_id, struct rte_eth_fc_conf *fc_conf) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-ENODEV); } @@ -1956,7 +2011,7 @@ rte_eth_dev_priority_flow_ctrl_set(uint8_t port_id, struct rte_eth_pfc_conf *pfc { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-ENODEV); } @@ -2035,7 +2090,7 @@ rte_eth_dev_rss_reta_update(uint8_t port_id, struct rte_eth_dev *dev; int ret; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return -ENODEV; } @@ -2086,10 +2141,11 @@ rte_eth_dev_rss_hash_update(uint8_t port_id, struct rte_eth_rss_conf *rss_conf) struct rte_eth_dev *dev; uint16_t rss_hash_protos; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-ENODEV); } + rss_hash_protos = rss_conf->rss_hf; if ((rss_hash_protos != 0) && ((rss_hash_protos & ETH_RSS_PROTO_MASK) == 0)) { @@ -2108,10 +2164,11 @@ rte_eth_dev_rss_hash_conf_get(uint8_t port_id, { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-ENODEV); } + dev = &rte_eth_devices[port_id]; FUNC_PTR_OR_ERR_RET(*dev->dev_ops->rss_hash_conf_get, -ENOTSUP); return (*dev->dev_ops->rss_hash_conf_get)(dev, rss_conf); @@ -2123,7 +2180,7 @@ rte_eth_dev_udp_tunnel_add(uint8_t port_id, { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return -ENODEV; } @@ -2149,10 +2206,11 @@ rte_eth_dev_udp_tunnel_delete(uint8_t port_id, { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return -ENODEV; } + dev = &rte_eth_devices[port_id]; if (udp_tunnel == NULL) { @@ -2174,7 +2232,7 @@ rte_eth_led_on(uint8_t port_id) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-ENODEV); } @@ -2189,7 +2247,7 @@ rte_eth_led_off(uint8_t port_id) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-ENODEV); } @@ -2229,10 +2287,11 @@ rte_eth_dev_mac_addr_add(uint8_t port_id, struct ether_addr *addr, int index; uint64_t pool_mask; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-ENODEV); } + dev = &rte_eth_devices[port_id]; FUNC_PTR_OR_ERR_RET(*dev->dev_ops->mac_addr_add, -ENOTSUP); @@ -2280,10 +2339,11 @@ rte_eth_dev_mac_addr_remove(uint8_t port_id, struct ether_addr *addr) struct rte_eth_dev *dev; int index; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-ENODEV); } + dev = &rte_eth_devices[port_id]; FUNC_PTR_OR_ERR_RET(*dev->dev_ops->mac_addr_remove, -ENOTSUP); @@ -2314,7 +2374,7 @@ rte_eth_dev_set_vf_rxmode(uint8_t port_id, uint16_t vf, struct rte_eth_dev *dev; struct rte_eth_dev_info dev_info; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("set VF RX mode:Invalid port_id=%d\n", port_id); return (-ENODEV); @@ -2369,7 +2429,7 @@ rte_eth_dev_uc_hash_table_set(uint8_t port_id, struct ether_addr *addr, int ret; struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("unicast hash setting:Invalid port_id=%d\n", port_id); return (-ENODEV); @@ -2422,7 +2482,7 @@ rte_eth_dev_uc_all_hash_table_set(uint8_t port_id, uint8_t on) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("unicast hash setting:Invalid port_id=%d\n", port_id); return (-ENODEV); @@ -2441,7 +2501,7 @@ rte_eth_dev_set_vf_rx(uint8_t port_id,uint16_t vf, uint8_t on) struct rte_eth_dev *dev; struct rte_eth_dev_info dev_info; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-ENODEV); } @@ -2467,7 +2527,7 @@ rte_eth_dev_set_vf_tx(uint8_t port_id,uint16_t vf, uint8_t on) struct rte_eth_dev *dev; struct rte_eth_dev_info dev_info; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("set pool tx:Invalid port_id=%d\n", port_id); return (-ENODEV); } @@ -2492,7 +2552,7 @@ rte_eth_dev_set_vf_vlan_filter(uint8_t port_id, uint16_t vlan_id, { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("VF VLAN filter:invalid port id=%d\n", port_id); return (-ENODEV); @@ -2523,7 +2583,7 @@ int rte_eth_set_queue_rate_limit(uint8_t port_id, uint16_t queue_idx, struct rte_eth_dev_info dev_info; struct rte_eth_link link; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("set queue rate limit:invalid port id=%d\n", port_id); return -ENODEV; @@ -2560,7 +2620,7 @@ int rte_eth_set_vf_rate_limit(uint8_t port_id, uint16_t vf, uint16_t tx_rate, if (q_msk == 0) return 0; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("set VF rate limit:invalid port id=%d\n", port_id); return -ENODEV; @@ -2594,7 +2654,7 @@ rte_eth_mirror_rule_set(uint8_t port_id, { struct rte_eth_dev *dev = &rte_eth_devices[port_id]; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-ENODEV); } @@ -2635,7 +2695,7 @@ rte_eth_mirror_rule_reset(uint8_t port_id, uint8_t rule_id) { struct rte_eth_dev *dev = &rte_eth_devices[port_id]; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-ENODEV); } @@ -2660,10 +2720,11 @@ rte_eth_rx_burst(uint8_t port_id, uint16_t queue_id, { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return 0; } + dev = &rte_eth_devices[port_id]; FUNC_PTR_OR_ERR_RET(*dev->rx_pkt_burst, 0); if (queue_id >= dev->data->nb_rx_queues) { @@ -2680,10 +2741,11 @@ rte_eth_tx_burst(uint8_t port_id, uint16_t queue_id, { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return 0; } + dev = &rte_eth_devices[port_id]; FUNC_PTR_OR_ERR_RET(*dev->tx_pkt_burst, 0); @@ -2700,10 +2762,11 @@ rte_eth_rx_queue_count(uint8_t port_id, uint16_t queue_id) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return 0; } + dev = &rte_eth_devices[port_id]; FUNC_PTR_OR_ERR_RET(*dev->dev_ops->rx_queue_count, 0); return (*dev->dev_ops->rx_queue_count)(dev, queue_id); @@ -2714,10 +2777,11 @@ rte_eth_rx_descriptor_done(uint8_t port_id, uint16_t queue_id, uint16_t offset) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-ENODEV); } + dev = &rte_eth_devices[port_id]; FUNC_PTR_OR_ERR_RET(*dev->dev_ops->rx_descriptor_done, -ENOTSUP); return (*dev->dev_ops->rx_descriptor_done)( \ @@ -2735,7 +2799,8 @@ rte_eth_dev_callback_register(uint8_t port_id, if (!cb_fn) return (-EINVAL); - if (port_id >= nb_ports) { + + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-EINVAL); } @@ -2775,7 +2840,8 @@ rte_eth_dev_callback_unregister(uint8_t port_id, if (!cb_fn) return (-EINVAL); - if (port_id >= nb_ports) { + + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-EINVAL); } @@ -2835,7 +2901,7 @@ int rte_eth_dev_bypass_init(uint8_t port_id) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-ENODEV); } @@ -2855,7 +2921,7 @@ rte_eth_dev_bypass_state_show(uint8_t port_id, uint32_t *state) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-ENODEV); } @@ -2874,7 +2940,7 @@ rte_eth_dev_bypass_state_set(uint8_t port_id, uint32_t *new_state) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-ENODEV); } @@ -2894,7 +2960,7 @@ rte_eth_dev_bypass_event_show(uint8_t port_id, uint32_t event, uint32_t *state) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-ENODEV); } @@ -2914,7 +2980,7 @@ rte_eth_dev_bypass_event_store(uint8_t port_id, uint32_t event, uint32_t state) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-ENODEV); } @@ -2934,7 +3000,7 @@ rte_eth_dev_wd_timeout_store(uint8_t port_id, uint32_t timeout) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-ENODEV); } @@ -2954,7 +3020,7 @@ rte_eth_dev_bypass_ver_show(uint8_t port_id, uint32_t *ver) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-ENODEV); } @@ -2974,7 +3040,7 @@ rte_eth_dev_bypass_wd_timeout_show(uint8_t port_id, uint32_t *wd_timeout) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-ENODEV); } @@ -2994,7 +3060,7 @@ rte_eth_dev_bypass_wd_reset(uint8_t port_id) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-ENODEV); } @@ -3015,7 +3081,7 @@ rte_eth_dev_filter_supported(uint8_t port_id, enum rte_filter_type filter_type) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return -ENODEV; } @@ -3032,7 +3098,7 @@ rte_eth_dev_filter_ctrl(uint8_t port_id, enum rte_filter_type filter_type, { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return -ENODEV; } diff --git a/lib/librte_ether/rte_ethdev.h b/lib/librte_ether/rte_ethdev.h index 4acd595..ef31bda 100644 --- a/lib/librte_ether/rte_ethdev.h +++ b/lib/librte_ether/rte_ethdev.h @@ -1399,6 +1399,7 @@ struct rte_eth_dev { struct eth_dev_ops *dev_ops; /**< Functions exported by PMD */ struct rte_pci_device *pci_dev; /**< PCI info. supplied by probing */ struct rte_eth_dev_cb_list callbacks; /**< User application callbacks */ + uint8_t attached; /**< Flag indicating the port is attached */ }; struct rte_eth_dev_sriov { @@ -1464,6 +1465,10 @@ extern struct rte_eth_dev rte_eth_devices[]; * initialized by the [matching] Ethernet driver during the PCI probing phase. * All devices whose port identifier is in the range * [0, rte_eth_dev_count() - 1] can be operated on by network applications. + * immediately after invoking rte_eal_init(). + * If the application unplugs a port using hotplug function, The enabled port + * numbers may be noncontiguous. In the case, the applications need to manage + * enabled port by themselves. * * @return * - The total number of usable Ethernet devices. -- 1.9.1
This patch replaces pci_addr_comparison() and memcmp() of pci addresses by rte_eal_compare_pci_addr(). To compare PCI addresses, rte_eal_compare_pci_addr() doesn't use memcmp(). This is because sizeof(struct rte_pci_addr) returns 6, but actually this structure is like below. struct rte_pci_addr { uint16_t domain; /**< Device domain */ uint8_t bus; /**< Device bus */ uint8_t devid; /**< Device ID */ uint8_t function; /**< Device function. */ }; If the structure is dynamically allocated in a function without bzero, last 1 byte may have value. As a result, memcmp may not work. To avoid such a case, rte_eal_compare_pci_addr() compare following values. dev_addr = (addr->domain << 24) | (addr->bus << 16) | (addr->devid << 8) | addr->function; v9: - eal_compare_pci_addr() is replaced by rte_eal_compare_pci_addr(). - Fix commit log. (Thanks to Thomas Monjalon) v8: - Fix pci_scan_one() to update sysfs values. (Thanks to Qiu, Michael and Iremonger, Bernard) v5: - Fix pci_scan_one to handle pt_driver correctly. v4: - Fix calculation method of eal_compare_pci_addr(). - Add parameter checking. Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp> --- lib/librte_eal/bsdapp/eal/eal_pci.c | 29 ++++++++++++-------------- lib/librte_eal/common/eal_common_pci.c | 2 +- lib/librte_eal/common/include/rte_pci.h | 34 +++++++++++++++++++++++++++++++ lib/librte_eal/linuxapp/eal/eal_pci.c | 30 +++++++++++++-------------- lib/librte_eal/linuxapp/eal/eal_pci_uio.c | 2 +- 5 files changed, 63 insertions(+), 34 deletions(-) diff --git a/lib/librte_eal/bsdapp/eal/eal_pci.c b/lib/librte_eal/bsdapp/eal/eal_pci.c index 74ecce7..9193f80 100644 --- a/lib/librte_eal/bsdapp/eal/eal_pci.c +++ b/lib/librte_eal/bsdapp/eal/eal_pci.c @@ -270,20 +270,6 @@ pci_uio_map_resource(struct rte_pci_device *dev) return (0); } -/* Compare two PCI device addresses. */ -static int -pci_addr_comparison(struct rte_pci_addr *addr, struct rte_pci_addr *addr2) -{ - uint64_t dev_addr = (addr->domain << 24) + (addr->bus << 16) + (addr->devid << 8) + addr->function; - uint64_t dev_addr2 = (addr2->domain << 24) + (addr2->bus << 16) + (addr2->devid << 8) + addr2->function; - - if (dev_addr > dev_addr2) - return 1; - else - return 0; -} - - /* Scan one pci sysfs entry, and fill the devices list from it. */ static int pci_scan_one(int dev_pci_fd, struct pci_conf *conf) @@ -356,13 +342,24 @@ pci_scan_one(int dev_pci_fd, struct pci_conf *conf) } else { struct rte_pci_device *dev2 = NULL; + int ret; TAILQ_FOREACH(dev2, &pci_device_list, next) { - if (pci_addr_comparison(&dev->addr, &dev2->addr)) + ret = rte_eal_compare_pci_addr(&dev->addr, &dev2->addr); + if (ret > 0) continue; - else { + else if (ret < 0) { TAILQ_INSERT_BEFORE(dev2, dev, next); return 0; + } else { /* already registered */ + /* update pt_driver */ + dev2->pt_driver = dev->pt_driver; + dev2->max_vfs = dev->max_vfs; + memmove(dev2->mem_resource, + dev->mem_resource, + sizeof(dev->mem_resource)); + free(dev); + return 0; } } TAILQ_INSERT_TAIL(&pci_device_list, dev, next); diff --git a/lib/librte_eal/common/eal_common_pci.c b/lib/librte_eal/common/eal_common_pci.c index f3c7f71..bf2793f 100644 --- a/lib/librte_eal/common/eal_common_pci.c +++ b/lib/librte_eal/common/eal_common_pci.c @@ -93,7 +93,7 @@ static struct rte_devargs *pci_devargs_lookup(struct rte_pci_device *dev) if (devargs->type != RTE_DEVTYPE_BLACKLISTED_PCI && devargs->type != RTE_DEVTYPE_WHITELISTED_PCI) continue; - if (!memcmp(&dev->addr, &devargs->pci.addr, sizeof(dev->addr))) + if (!rte_eal_compare_pci_addr(&dev->addr, &devargs->pci.addr)) return devargs; } return NULL; diff --git a/lib/librte_eal/common/include/rte_pci.h b/lib/librte_eal/common/include/rte_pci.h index ffd13d9..6814e91 100644 --- a/lib/librte_eal/common/include/rte_pci.h +++ b/lib/librte_eal/common/include/rte_pci.h @@ -272,6 +272,40 @@ eal_parse_pci_DomBDF(const char *input, struct rte_pci_addr *dev_addr) } #undef GET_PCIADDR_FIELD +/* Compare two PCI device addresses. */ +/** + * Utility function to compare two PCI device addresses. + * + * @param addr + * The PCI Bus-Device-Function address to compare + * @param addr2 + * The PCI Bus-Device-Function address to compare + * @return + * 0 on equal PCI address. + * Positive on addr is greater than addr2. + * Negative on addr is less than addr2, or error. + */ +static inline int +rte_eal_compare_pci_addr(struct rte_pci_addr *addr, struct rte_pci_addr *addr2) +{ + uint64_t dev_addr, dev_addr2; + + if ((addr == NULL) || (addr2 == NULL)) + return -1; + + dev_addr = (addr->domain << 24) | (addr->bus << 16) | + (addr->devid << 8) | addr->function; + dev_addr2 = (addr2->domain << 24) | (addr2->bus << 16) | + (addr2->devid << 8) | addr2->function; + + if (dev_addr > dev_addr2) + return 1; + else if (dev_addr < dev_addr2) + return -1; + else + return 0; +} + /** * Probe the PCI bus for registered drivers. * diff --git a/lib/librte_eal/linuxapp/eal/eal_pci.c b/lib/librte_eal/linuxapp/eal/eal_pci.c index c04f897..e6cead1 100644 --- a/lib/librte_eal/linuxapp/eal/eal_pci.c +++ b/lib/librte_eal/linuxapp/eal/eal_pci.c @@ -41,6 +41,7 @@ #include <rte_eal_memconfig.h> #include <rte_malloc.h> #include <rte_devargs.h> +#include <rte_memcpy.h> #include "rte_pci_dev_ids.h" #include "eal_filesystem.h" @@ -227,20 +228,6 @@ error: return -1; } -/* Compare two PCI device addresses. */ -static int -pci_addr_comparison(struct rte_pci_addr *addr, struct rte_pci_addr *addr2) -{ - uint64_t dev_addr = (addr->domain << 24) + (addr->bus << 16) + (addr->devid << 8) + addr->function; - uint64_t dev_addr2 = (addr2->domain << 24) + (addr2->bus << 16) + (addr2->devid << 8) + addr2->function; - - if (dev_addr > dev_addr2) - return 1; - else - return 0; -} - - /* Scan one pci sysfs entry, and fill the devices list from it. */ static int pci_scan_one(const char *dirname, uint16_t domain, uint8_t bus, @@ -358,13 +345,24 @@ pci_scan_one(const char *dirname, uint16_t domain, uint8_t bus, } else { struct rte_pci_device *dev2 = NULL; + int ret; TAILQ_FOREACH(dev2, &pci_device_list, next) { - if (pci_addr_comparison(&dev->addr, &dev2->addr)) + ret = rte_eal_compare_pci_addr(&dev->addr, &dev2->addr); + if (ret > 0) continue; - else { + else if (ret < 0) { TAILQ_INSERT_BEFORE(dev2, dev, next); return 0; + } else { /* already registered */ + /* update pt_driver */ + dev2->pt_driver = dev->pt_driver; + dev2->max_vfs = dev->max_vfs; + memmove(dev2->mem_resource, + dev->mem_resource, + sizeof(dev->mem_resource)); + free(dev); + return 0; } } TAILQ_INSERT_TAIL(&pci_device_list, dev, next); diff --git a/lib/librte_eal/linuxapp/eal/eal_pci_uio.c b/lib/librte_eal/linuxapp/eal/eal_pci_uio.c index 2b16fcb..f7acc55 100644 --- a/lib/librte_eal/linuxapp/eal/eal_pci_uio.c +++ b/lib/librte_eal/linuxapp/eal/eal_pci_uio.c @@ -126,7 +126,7 @@ pci_uio_map_secondary(struct rte_pci_device *dev) TAILQ_FOREACH(uio_res, pci_res_list, next) { /* skip this element if it doesn't match our PCI address */ - if (memcmp(&uio_res->pci_addr, &dev->addr, sizeof(dev->addr))) + if (rte_eal_compare_pci_addr(&uio_res->pci_addr, &dev->addr)) continue; for (i = 0; i != uio_res->nb_maps; i++) { -- 1.9.1
This patch adds rte_eth_dev_release_port(). The function is used for changing an attached status of the device that has specified name. v9: - rte_eth_dev_free() is replaced by rte_eth_dev_release_port(). (Thanks to Thomas Monjalon) v6: - Use rte_eth_dev structure as the paramter of rte_eth_dev_free(). v4: - Add parameter checking. Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp> --- lib/librte_ether/rte_ethdev.c | 11 +++++++++++ lib/librte_ether/rte_ethdev.h | 12 ++++++++++++ 2 files changed, 23 insertions(+) diff --git a/lib/librte_ether/rte_ethdev.c b/lib/librte_ether/rte_ethdev.c index 0e1e5c9..8d271ae 100644 --- a/lib/librte_ether/rte_ethdev.c +++ b/lib/librte_ether/rte_ethdev.c @@ -255,6 +255,17 @@ rte_eth_dev_allocate(const char *name) return eth_dev; } +int +rte_eth_dev_release_port(struct rte_eth_dev *eth_dev) +{ + if (eth_dev == NULL) + return -EINVAL; + + eth_dev->attached = 0; + nb_ports--; + return 0; +} + static int rte_eth_dev_init(struct rte_pci_driver *pci_drv, struct rte_pci_device *pci_dev) diff --git a/lib/librte_ether/rte_ethdev.h b/lib/librte_ether/rte_ethdev.h index ef31bda..8016a51 100644 --- a/lib/librte_ether/rte_ethdev.h +++ b/lib/librte_ether/rte_ethdev.h @@ -1487,6 +1487,18 @@ extern uint8_t rte_eth_dev_count(void); */ struct rte_eth_dev *rte_eth_dev_allocate(const char *name); +/** + * Function for internal use by dummy drivers primarily, e.g. ring-based + * driver. + * Release the specified ethdev port. + * + * @param eth_dev + * The *eth_dev* pointer is the address of the *rte_eth_dev* structure. + * @return + * - 0 on success, negative on error + */ +int rte_eth_dev_release_port(struct rte_eth_dev *eth_dev); + struct eth_driver; /** * @internal -- 1.9.1
The patch adds function pointer to rte_pci_driver and eth_driver structure. These function pointers are used when ports are detached. Also, the patch adds rte_eth_dev_uninit(). So far, it's not called by anywhere, but it will be called when port hotplug function is implemented. v10: - Add size parameter to rte_eth_dev_create_unique_device_name(). (Thanks to Iremonger, Bernard) v9: - Change parameter of pci_devuninit_t and rte_eth_dev_uninit. - Remove code that initiaize callback of ethdev from rte_eth_dev_uninit(). - Add a function to create a unique device name. (Thanks to Thomas Monjalon) v6: - Fix rte_eth_dev_uninit() to handle a return value of uninit function of PMD. v4: - Add parameter checking. - Change function names. Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp> --- lib/librte_eal/common/include/rte_pci.h | 6 ++++ lib/librte_ether/rte_ethdev.c | 64 +++++++++++++++++++++++++++++++-- lib/librte_ether/rte_ethdev.h | 24 +++++++++++++ 3 files changed, 92 insertions(+), 2 deletions(-) diff --git a/lib/librte_eal/common/include/rte_pci.h b/lib/librte_eal/common/include/rte_pci.h index 6814e91..4ea57cb 100644 --- a/lib/librte_eal/common/include/rte_pci.h +++ b/lib/librte_eal/common/include/rte_pci.h @@ -192,12 +192,18 @@ struct rte_pci_driver; typedef int (pci_devinit_t)(struct rte_pci_driver *, struct rte_pci_device *); /** + * Uninitialisation function for the driver called during hotplugging. + */ +typedef int (pci_devuninit_t)(struct rte_pci_device *); + +/** * A structure describing a PCI driver. */ struct rte_pci_driver { TAILQ_ENTRY(rte_pci_driver) next; /**< Next in list. */ const char *name; /**< Driver name. */ pci_devinit_t *devinit; /**< Device init. function. */ + pci_devuninit_t *devuninit; /**< Device uninit function. */ struct rte_pci_id *id_table; /**< ID table, NULL terminated. */ uint32_t drv_flags; /**< Flags contolling handling of device. */ }; diff --git a/lib/librte_ether/rte_ethdev.c b/lib/librte_ether/rte_ethdev.c index 8d271ae..3d148e2 100644 --- a/lib/librte_ether/rte_ethdev.c +++ b/lib/librte_ether/rte_ethdev.c @@ -266,6 +266,24 @@ rte_eth_dev_release_port(struct rte_eth_dev *eth_dev) return 0; } +static inline int +rte_eth_dev_create_unique_device_name(char *name, size_t size, + struct rte_pci_device *pci_dev) +{ + int ret; + + if ((name == NULL) || (pci_dev == NULL)) + return -EINVAL; + + ret = snprintf(name, size, "%d:%d.%d", + pci_dev->addr.bus, pci_dev->addr.devid, + pci_dev->addr.function); + if (ret < 0) + return ret; + + return 0; +} + static int rte_eth_dev_init(struct rte_pci_driver *pci_drv, struct rte_pci_device *pci_dev) @@ -279,8 +297,8 @@ rte_eth_dev_init(struct rte_pci_driver *pci_drv, eth_drv = (struct eth_driver *)pci_drv; /* Create unique Ethernet device name using PCI address */ - snprintf(ethdev_name, RTE_ETH_NAME_MAX_LEN, "%d:%d.%d", - pci_dev->addr.bus, pci_dev->addr.devid, pci_dev->addr.function); + rte_eth_dev_create_unique_device_name(ethdev_name, + sizeof(ethdev_name), pci_dev); eth_dev = rte_eth_dev_allocate(ethdev_name); if (eth_dev == NULL) @@ -321,6 +339,47 @@ rte_eth_dev_init(struct rte_pci_driver *pci_drv, return diag; } +static int +rte_eth_dev_uninit(struct rte_pci_device *pci_dev) +{ + const struct eth_driver *eth_drv; + struct rte_eth_dev *eth_dev; + char ethdev_name[RTE_ETH_NAME_MAX_LEN]; + int ret; + + if (pci_dev == NULL) + return -EINVAL; + + /* Create unique Ethernet device name using PCI address */ + rte_eth_dev_create_unique_device_name(ethdev_name, + sizeof(ethdev_name), pci_dev); + + eth_dev = rte_eth_dev_allocated(ethdev_name); + if (eth_dev == NULL) + return -ENODEV; + + eth_drv = (const struct eth_driver *)pci_dev->driver; + + /* Invoke PMD device uninit function */ + if (*eth_drv->eth_dev_uninit) { + ret = (*eth_drv->eth_dev_uninit)(eth_drv, eth_dev); + if (ret) + return ret; + } + + /* free ether device */ + rte_eth_dev_release_port(eth_dev); + + if (rte_eal_process_type() == RTE_PROC_PRIMARY) + rte_free(eth_dev->data->dev_private); + + eth_dev->pci_dev = NULL; + eth_dev->driver = NULL; + eth_dev->data = NULL; + + return 0; +} + /** * Register an Ethernet [Poll Mode] driver. * @@ -339,6 +398,7 @@ void rte_eth_driver_register(struct eth_driver *eth_drv) { eth_drv->pci_drv.devinit = rte_eth_dev_init; + eth_drv->pci_drv.devuninit = rte_eth_dev_uninit; rte_eal_pci_register(ð_drv->pci_drv); } diff --git a/lib/librte_ether/rte_ethdev.h b/lib/librte_ether/rte_ethdev.h index 8016a51..6cba06d 100644 --- a/lib/librte_ether/rte_ethdev.h +++ b/lib/librte_ether/rte_ethdev.h @@ -1536,6 +1536,27 @@ typedef int (*eth_dev_init_t)(struct eth_driver *eth_drv, /** * @internal + * Finalization function of an Ethernet driver invoked for each matching + * Ethernet PCI device detected during the PCI closing phase. + * + * @param eth_drv + * The pointer to the [matching] Ethernet driver structure supplied by + * the PMD when it registered itself. + * @param eth_dev + * The *eth_dev* pointer is the address of the *rte_eth_dev* structure + * associated with the matching device and which have been [automatically] + * allocated in the *rte_eth_devices* array. + * @return + * - 0: Success, the device is properly finalized by the driver. + * In particular, the driver MUST free the *dev_ops* pointer + * of the *eth_dev* structure. + * - <0: Error code of the device initialization failure. + */ +typedef int (*eth_dev_uninit_t)(const struct eth_driver *eth_drv, + struct rte_eth_dev *eth_dev); + +/** + * @internal * The structure associated with a PMD Ethernet driver. * * Each Ethernet driver acts as a PCI driver and is represented by a generic @@ -1545,11 +1566,14 @@ typedef int (*eth_dev_init_t)(struct eth_driver *eth_drv, * * - The *eth_dev_init* function invoked for each matching PCI device. * + * - The *eth_dev_uninit* function invoked for each matching PCI device. + * * - The size of the private data to allocate for each matching device. */ struct eth_driver { struct rte_pci_driver pci_drv; /**< The PMD is also a PCI driver. */ eth_dev_init_t eth_dev_init; /**< Device init function. */ + eth_dev_uninit_t eth_dev_uninit; /**< Device uninit function. */ unsigned int dev_private_size; /**< Size of device private data. */ }; -- 1.9.1
The patch adds following functions. - rte_eth_dev_save() The function is used for saving current rte_eth_dev structures. - rte_eth_dev_get_changed_port() The function receives the rte_eth_dev structures, then compare these with current values to know which port is actually attached or detached. - rte_eth_dev_get_addr_by_port() The function returns a pci address of an ethdev specified by port identifier. - rte_eth_dev_get_port_by_addr() The function returns a port identifier of an ethdev specified by pci address. - rte_eth_dev_get_name_by_port() The function returns a unique identifier name of an ethdev specified by port identifier. - Add rte_eth_dev_is_detachable() The function returns whether a PMD supports detach function. Also, the patch changes scope of rte_eth_dev_allocated() to global. This function will be called by virtual PMDs to support port hotplug. So change scope of the function to global. v10: - Change order of version.map. (Thanks to Thomas Monjalon) v9: - rte_eth_dev_check_detachable() is replaced by rte_eth_dev_is_detachable(). - strncpy() is replaced by strcpy(). (Thanks to Thomas Monjalon) - Add missing symbol in version map. (Thanks to Nail Horman) v8: - Add size parameter to rte_eth_dev_save(). - Add missing symbol in version map. (Thanks to Qiu, Michael and Iremonger, Bernard) v7: - Add pt_driver checking to rte_eth_dev_check_detachable(). (Thanks to Qiu, Michael) v5: - Fix return value of below functions. rte_eth_dev_get_changed_port(). rte_eth_dev_get_port_by_addr(). v4: - Add parameter checking. v3: - Fix if-condition bug while comparing pci addresses. - Add error checking codes. Reported-by: Mark Enright <menrigh@brocade.com> Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp> --- lib/librte_ether/rte_ethdev.c | 103 ++++++++++++++++++++++++++++++++- lib/librte_ether/rte_ethdev.h | 83 ++++++++++++++++++++++++++ lib/librte_ether/rte_ether_version.map | 7 +++ 3 files changed, 192 insertions(+), 1 deletion(-) diff --git a/lib/librte_ether/rte_ethdev.c b/lib/librte_ether/rte_ethdev.c index 3d148e2..7067620 100644 --- a/lib/librte_ether/rte_ethdev.c +++ b/lib/librte_ether/rte_ethdev.c @@ -201,7 +201,7 @@ rte_eth_dev_data_alloc(void) RTE_MAX_ETHPORTS * sizeof(*rte_eth_dev_data)); } -static struct rte_eth_dev * +struct rte_eth_dev * rte_eth_dev_allocated(const char *name) { unsigned i; @@ -426,6 +426,107 @@ rte_eth_dev_count(void) return (nb_ports); } +int +rte_eth_dev_save(struct rte_eth_dev *devs, size_t size) +{ + if ((devs == NULL) || + (size != sizeof(struct rte_eth_dev) * RTE_MAX_ETHPORTS)) + return -EINVAL; + + /* save current rte_eth_devices */ + memcpy(devs, rte_eth_devices, size); + return 0; +} + +int +rte_eth_dev_get_changed_port(struct rte_eth_dev *devs, uint8_t *port_id) +{ + if ((devs == NULL) || (port_id == NULL)) + return -EINVAL; + + /* check which port was attached or detached */ + for (*port_id = 0; *port_id < RTE_MAX_ETHPORTS; (*port_id)++, devs++) { + if (rte_eth_devices[*port_id].attached ^ devs->attached) + return 0; + } + return -ENODEV; +} + +int +rte_eth_dev_get_addr_by_port(uint8_t port_id, struct rte_pci_addr *addr) +{ + if (!rte_eth_dev_is_valid_port(port_id)) { + PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + return -EINVAL; + } + + if (addr == NULL) { + PMD_DEBUG_TRACE("Null pointer is specified\n"); + return -EINVAL; + } + + *addr = rte_eth_devices[port_id].pci_dev->addr; + return 0; +} + +int +rte_eth_dev_get_port_by_addr(struct rte_pci_addr *addr, uint8_t *port_id) +{ + struct rte_pci_addr *tmp; + + if ((addr == NULL) || (port_id == NULL)) { + PMD_DEBUG_TRACE("Null pointer is specified\n"); + return -EINVAL; + } + + for (*port_id = 0; *port_id < RTE_MAX_ETHPORTS; (*port_id)++) { + if (!rte_eth_devices[*port_id].attached) + continue; + if (!rte_eth_devices[*port_id].pci_dev) + continue; + tmp = &rte_eth_devices[*port_id].pci_dev->addr; + if (rte_eal_compare_pci_addr(tmp, addr) == 0) + return 0; + } + return -ENODEV; +} + +int +rte_eth_dev_get_name_by_port(uint8_t port_id, char *name) +{ + char *tmp; + + if (!rte_eth_dev_is_valid_port(port_id)) { + PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + return -EINVAL; + } + + if (name == NULL) { + PMD_DEBUG_TRACE("Null pointer is specified\n"); + return -EINVAL; + } + + /* shouldn't check 'rte_eth_devices[i].data', + * because it might be overwritten by VDEV PMD */ + tmp = rte_eth_dev_data[port_id].name; + strcpy(name, tmp); + return 0; +} + +int +rte_eth_dev_is_detachable(uint8_t port_id) +{ + uint32_t drv_flags; + + if (port_id >= RTE_MAX_ETHPORTS) { + PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + return -EINVAL; + } + + drv_flags = rte_eth_devices[port_id].driver->pci_drv.drv_flags; + return !(drv_flags & RTE_PCI_DRV_DETACHABLE); +} + static int rte_eth_dev_rx_queue_config(struct rte_eth_dev *dev, uint16_t nb_queues) { diff --git a/lib/librte_ether/rte_ethdev.h b/lib/librte_ether/rte_ethdev.h index 6cba06d..d4cfafb 100644 --- a/lib/librte_ether/rte_ethdev.h +++ b/lib/librte_ether/rte_ethdev.h @@ -1476,6 +1476,89 @@ extern struct rte_eth_dev rte_eth_devices[]; extern uint8_t rte_eth_dev_count(void); /** + * Function for internal use by port hotplug functions. + * Copies current ethdev structures to the specified pointer. + * + * @param devs The pointer to the ethdev structures + * @param size The size of ethdev structures + * @return + * - 0 on success, negative on error + */ +extern int rte_eth_dev_save(struct rte_eth_dev *devs, size_t size); + +/** + * Function for internal use by port hotplug functions. + * Compare the specified ethdev structures with currents. Then + * if there is a port which status is changed, fill the specified pointer + * with the port id of that port. + * @param devs The pointer to the ethdev structures + * @param port_id The pointer to the port id + * @return + * - 0 on success, negative on error + */ +extern int rte_eth_dev_get_changed_port( + struct rte_eth_dev *devs, uint8_t *port_id); + +/** + * Function for internal use by port hotplug functions. + * Returns a pci address of a ethdev specified by port identifier. + * @param port_id + * The port identifier of the Ethernet device + * @param addr + * The pointer to the pci address + * @return + * - 0 on success, negative on error + */ +extern int rte_eth_dev_get_addr_by_port( + uint8_t port_id, struct rte_pci_addr *addr); + +/** + * Function for internal use by port hotplug functions. + * Returns a port identifier of a ethdev specified by pci address. + * @param addr + * The pointer to the pci address of the Ethernet device. + * @param port_id + * The pointer to the port identifier + * @return + * - 0 on success, negative on error + */ +extern int rte_eth_dev_get_port_by_addr( + struct rte_pci_addr *addr, uint8_t *port_id); + +/** + * Function for internal use by port hotplug functions. + * Returns a unique identifier name of a ethdev specified by port identifier. + * @param port_id + * The port identifier. + * @param name + * The pointer to the Unique identifier name for each Ethernet device + * @return + * - 0 on success, negative on error + */ +extern int rte_eth_dev_get_name_by_port(uint8_t port_id, char *name); + +/** + * Function for internal use by port hotplug functions. + * Check whether or not, a PMD that is handling the ethdev specified by port + * identifier can support detach function. + * @param port_id + * The port identifier + * @return + * - 0 on supporting detach function, negative on not supporting + */ +extern int rte_eth_dev_is_detachable(uint8_t port_id); + +/** + * Function for internal use by port hotplug functions. + * Returns a ethdev slot specified by the unique identifier name. + * @param name + * The pointer to the Unique identifier name for each Ethernet device + * @return + * - The pointer to the ethdev slot, on success. NULL on error + */ +extern struct rte_eth_dev *rte_eth_dev_allocated(const char *name); + +/** * Function for internal use by dummy drivers primarily, e.g. ring-based * driver. * Allocates a new ethdev slot for an ethernet device and returns the pointer diff --git a/lib/librte_ether/rte_ether_version.map b/lib/librte_ether/rte_ether_version.map index f66fd2d..099c769 100644 --- a/lib/librte_ether/rte_ether_version.map +++ b/lib/librte_ether/rte_ether_version.map @@ -6,6 +6,7 @@ DPDK_2.0 { rte_eth_allmulticast_enable; rte_eth_allmulticast_get; rte_eth_dev_allocate; + rte_eth_dev_allocated; rte_eth_dev_bypass_event_show; rte_eth_dev_bypass_event_store; rte_eth_dev_bypass_init; @@ -32,9 +33,14 @@ DPDK_2.0 { rte_eth_dev_filter_supported; rte_eth_dev_flow_ctrl_get; rte_eth_dev_flow_ctrl_set; + rte_eth_dev_get_addr_by_port; + rte_eth_dev_get_changed_port; rte_eth_dev_get_mtu; + rte_eth_dev_get_name_by_port; + rte_eth_dev_get_port_by_addr; rte_eth_dev_get_vlan_offload; rte_eth_dev_info_get; + rte_eth_dev_is_detachable; rte_eth_dev_mac_addr_add; rte_eth_dev_mac_addr_remove; rte_eth_dev_priority_flow_ctrl_set; @@ -44,6 +50,7 @@ DPDK_2.0 { rte_eth_dev_rss_reta_update; rte_eth_dev_rx_queue_start; rte_eth_dev_rx_queue_stop; + rte_eth_dev_save; rte_eth_dev_set_link_down; rte_eth_dev_set_link_up; rte_eth_dev_set_mtu; -- 1.9.1
The patch adds functions for unmapping igb_uio resources. The patch is only for Linux and igb_uio environment. VFIO and BSD are not supported. v9: - Remove "rte_dev_hotplug.h". - Remove needless "#ifdef". (Thanks to Thomas Monjalon and Neil Horman) - Remove pci_unmap_device(). It will be implemented in later patch. v8: - Fix typo. (Thanks to Iremonger, Bernard) v5: - Fix pci_unmap_device() to check pt_driver. v4: - Add parameter checking. - Add header file to determine if hotplug can be enabled. Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp> --- lib/librte_eal/linuxapp/eal/eal_pci.c | 17 ++++++++ lib/librte_eal/linuxapp/eal/eal_pci_init.h | 7 ++++ lib/librte_eal/linuxapp/eal/eal_pci_uio.c | 65 ++++++++++++++++++++++++++++++ 3 files changed, 89 insertions(+) diff --git a/lib/librte_eal/linuxapp/eal/eal_pci.c b/lib/librte_eal/linuxapp/eal/eal_pci.c index e6cead1..17f32c0 100644 --- a/lib/librte_eal/linuxapp/eal/eal_pci.c +++ b/lib/librte_eal/linuxapp/eal/eal_pci.c @@ -167,6 +167,23 @@ pci_map_resource(void *requested_addr, int fd, off_t offset, size_t size) return mapaddr; } +/* unmap a particular resource */ +void +pci_unmap_resource(void *requested_addr, size_t size) +{ + if (requested_addr == NULL) + return; + + /* Unmap the PCI memory resource of device */ + if (munmap(requested_addr, size)) { + RTE_LOG(ERR, EAL, "%s(): cannot munmap(%p, 0x%lx): %s\n", + __func__, requested_addr, (unsigned long)size, + strerror(errno)); + } else + RTE_LOG(DEBUG, EAL, " PCI memory unmapped at %p\n", + requested_addr); +} + /* parse the "resource" sysfs file */ static int pci_parse_sysfs_resource(const char *filename, struct rte_pci_device *dev) diff --git a/lib/librte_eal/linuxapp/eal/eal_pci_init.h b/lib/librte_eal/linuxapp/eal/eal_pci_init.h index 1070eb8..e2dd8a5 100644 --- a/lib/librte_eal/linuxapp/eal/eal_pci_init.h +++ b/lib/librte_eal/linuxapp/eal/eal_pci_init.h @@ -71,6 +71,13 @@ void *pci_map_resource(void *requested_addr, int fd, off_t offset, /* map IGB_UIO resource prototype */ int pci_uio_map_resource(struct rte_pci_device *dev); +void pci_unmap_resource(void *requested_addr, size_t size); + +#ifdef RTE_LIBRTE_EAL_HOTPLUG +/* unmap IGB_UIO resource prototype */ +void pci_uio_unmap_resource(struct rte_pci_device *dev); +#endif /* RTE_LIBRTE_EAL_HOTPLUG */ + #ifdef VFIO_PRESENT #define VFIO_MAX_GROUPS 64 diff --git a/lib/librte_eal/linuxapp/eal/eal_pci_uio.c b/lib/librte_eal/linuxapp/eal/eal_pci_uio.c index f7acc55..ff4d0e8 100644 --- a/lib/librte_eal/linuxapp/eal/eal_pci_uio.c +++ b/lib/librte_eal/linuxapp/eal/eal_pci_uio.c @@ -433,3 +433,68 @@ pci_uio_map_resource(struct rte_pci_device *dev) return 0; } + +#ifdef RTE_LIBRTE_EAL_HOTPLUG +static void +pci_uio_unmap(struct mapped_pci_resource *uio_res) +{ + int i; + + if (uio_res == NULL) + return; + + for (i = 0; i != uio_res->nb_maps; i++) + pci_unmap_resource(uio_res->maps[i].addr, + (size_t)uio_res->maps[i].size); +} + +static struct mapped_pci_resource * +pci_uio_find_resource(struct rte_pci_device *dev) +{ + struct mapped_pci_resource *uio_res; + + if (dev == NULL) + return NULL; + + TAILQ_FOREACH(uio_res, pci_res_list, next) { + + /* skip this element if it doesn't match our PCI address */ + if (!rte_eal_compare_pci_addr(&uio_res->pci_addr, &dev->addr)) + return uio_res; + } + return NULL; +} + +/* unmap the PCI resource of a PCI device in virtual memory */ +void +pci_uio_unmap_resource(struct rte_pci_device *dev) +{ + struct mapped_pci_resource *uio_res; + + if (dev == NULL) + return; + + /* find an entry for the device */ + uio_res = pci_uio_find_resource(dev); + if (uio_res == NULL) + return; + + /* secondary processes - just free maps */ + if (rte_eal_process_type() != RTE_PROC_PRIMARY) + return pci_uio_unmap(uio_res); + + TAILQ_REMOVE(pci_res_list, uio_res, next); + + /* unmap all resources */ + pci_uio_unmap(uio_res); + + /* free uio resource */ + rte_free(uio_res); + + /* close fd if in primary process */ + close(dev->intr_handle.fd); + + dev->intr_handle.fd = -1; + dev->intr_handle.type = RTE_INTR_HANDLE_UNKNOWN; +} +#endif /* RTE_LIBRTE_EAL_HOTPLUG */ -- 1.9.1
- Add pci_close_all_drivers() The function tries to find a driver for the specified device, and then close the driver. - Add rte_eal_pci_probe_one() and rte_eal_pci_close_one() The functions are used for probe and close a device. First the function tries to find a device that has the specified PCI address. Then, probe or close the device. v9: - Fix commit title. - Remove RTE_EAL_INVOKE_TYPE_PROBE/CLOSE. (Thanks to Thomas Monjalon) - Implement pci_unmap_device() in this patch. v5: - Remove RTE_EAL_INVOKE_TYPE_UNKNOWN, because it's unused. v4: - Fix parameter checking. - Fix indent of 'if' statement. Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp> --- lib/librte_eal/common/eal_common_pci.c | 98 ++++++++++++++++++++++++++++++++- lib/librte_eal/common/eal_private.h | 15 +++++ lib/librte_eal/common/include/rte_pci.h | 32 +++++++++++ lib/librte_eal/linuxapp/eal/eal_pci.c | 94 +++++++++++++++++++++++++++++++ 4 files changed, 238 insertions(+), 1 deletion(-) diff --git a/lib/librte_eal/common/eal_common_pci.c b/lib/librte_eal/common/eal_common_pci.c index bf2793f..5b6b55d 100644 --- a/lib/librte_eal/common/eal_common_pci.c +++ b/lib/librte_eal/common/eal_common_pci.c @@ -108,7 +108,10 @@ static int pci_probe_all_drivers(struct rte_pci_device *dev) { struct rte_pci_driver *dr = NULL; - int rc; + int rc = 0; + + if (dev == NULL) + return -1; TAILQ_FOREACH(dr, &pci_driver_list, next) { rc = rte_eal_pci_probe_one_driver(dr, dev); @@ -123,6 +126,99 @@ pci_probe_all_drivers(struct rte_pci_device *dev) return 1; } +#ifdef RTE_LIBRTE_EAL_HOTPLUG +/* + * If vendor/device ID match, call the devuninit() function of all + * registered driver for the given device. Return -1 if initialization + * failed, return 1 if no driver is found for this device. + */ +static int +pci_close_all_drivers(struct rte_pci_device *dev) +{ + struct rte_pci_driver *dr = NULL; + int rc = 0; + + if (dev == NULL) + return -1; + + TAILQ_FOREACH(dr, &pci_driver_list, next) { + rc = rte_eal_pci_close_one_driver(dr, dev); + if (rc < 0) + /* negative value is an error */ + return -1; + if (rc > 0) + /* positive value means driver not found */ + continue; + return 0; + } + return 1; +} + +/* + * Find the pci device specified by pci address, then invoke probe function of + * the driver of the devive. + */ +int +rte_eal_pci_probe_one(struct rte_pci_addr *addr) +{ + struct rte_pci_device *dev = NULL; + int ret = 0; + + if (addr == NULL) + return -1; + + TAILQ_FOREACH(dev, &pci_device_list, next) { + if (rte_eal_compare_pci_addr(&dev->addr, addr)) + continue; + + ret = pci_probe_all_drivers(dev); + if (ret < 0) + goto err_return; + return 0; + } + return -1; + +err_return: + RTE_LOG(WARNING, EAL, "Requested device " PCI_PRI_FMT + " cannot be used\n", dev->addr.domain, dev->addr.bus, + dev->addr.devid, dev->addr.function); + return -1; +} + +/* + * Find the pci device specified by pci address, then invoke close function of + * the driver of the devive. + */ +int +rte_eal_pci_close_one(struct rte_pci_addr *addr) +{ + struct rte_pci_device *dev = NULL; + int ret = 0; + + if (addr == NULL) + return -1; + + TAILQ_FOREACH(dev, &pci_device_list, next) { + if (rte_eal_compare_pci_addr(&dev->addr, addr)) + continue; + + ret = pci_close_all_drivers(dev); + if (ret < 0) + goto err_return; + + TAILQ_REMOVE(&pci_device_list, dev, next); + return 0; + } + return -1; + +err_return: + RTE_LOG(WARNING, EAL, "Requested device " PCI_PRI_FMT + " cannot be used\n", dev->addr.domain, dev->addr.bus, + dev->addr.devid, dev->addr.function); + return -1; +} +#endif /* RTE_LIBRTE_EAL_HOTPLUG */ + /* * Scan the content of the PCI bus, and call the devinit() function for * all registered drivers that have a matching entry in its id_table diff --git a/lib/librte_eal/common/eal_private.h b/lib/librte_eal/common/eal_private.h index 159cd66..4acf5a0 100644 --- a/lib/librte_eal/common/eal_private.h +++ b/lib/librte_eal/common/eal_private.h @@ -165,6 +165,21 @@ int rte_eal_pci_probe_one_driver(struct rte_pci_driver *dr, struct rte_pci_device *dev); /** + * Munmap memory for single PCI device + * + * This function is private to EAL. + * + * @param dr + * The pointer to the pci driver structure + * @param dev + * The pointer to the pci device structure + * @return + * 0 on success, negative on error + */ +int rte_eal_pci_close_one_driver(struct rte_pci_driver *dr, + struct rte_pci_device *dev); + +/** * Init tail queues for non-EAL library structures. This is to allow * the rings, mempools, etc. lists to be shared among multiple processes * diff --git a/lib/librte_eal/common/include/rte_pci.h b/lib/librte_eal/common/include/rte_pci.h index 4ea57cb..49f2e7f 100644 --- a/lib/librte_eal/common/include/rte_pci.h +++ b/lib/librte_eal/common/include/rte_pci.h @@ -325,6 +325,38 @@ rte_eal_compare_pci_addr(struct rte_pci_addr *addr, struct rte_pci_addr *addr2) */ int rte_eal_pci_probe(void); +#ifdef RTE_LIBRTE_EAL_HOTPLUG +/** + * Probe the single PCI device. + * + * Scan the content of the PCI bus, and find the pci device specified by pci + * address, then call the probe() function for registered driver that has a + * matching entry in its id_table for discovered device. + * + * @param addr + * The PCI Bus-Device-Function address to probe. + * @return + * - 0 on success. + * - Negative on error. + */ +int rte_eal_pci_probe_one(struct rte_pci_addr *addr); + +/** + * Close the single PCI device. + * + * Scan the content of the PCI bus, and find the pci device specified by pci + * address, then call the close() function for registered driver that has a + * matching entry in its id_table for discovered device. + * + * @param addr + * The PCI Bus-Device-Function address to close. + * @return + * - 0 on success. + * - Negative on error. + */ +int rte_eal_pci_close_one(struct rte_pci_addr *addr); +#endif /* RTE_LIBRTE_EAL_HOTPLUG */ + /** * Dump the content of the PCI bus. * diff --git a/lib/librte_eal/linuxapp/eal/eal_pci.c b/lib/librte_eal/linuxapp/eal/eal_pci.c index 17f32c0..85d9d09 100644 --- a/lib/librte_eal/linuxapp/eal/eal_pci.c +++ b/lib/librte_eal/linuxapp/eal/eal_pci.c @@ -594,6 +594,31 @@ pci_map_device(struct rte_pci_device *dev) return ret; } +#ifdef RTE_LIBRTE_EAL_HOTPLUG +static void +pci_unmap_device(struct rte_pci_device *dev) +{ + if (dev == NULL) + return; + + /* try unmapping the NIC resources using VFIO if it exists */ + switch (dev->pt_driver) { + case RTE_PT_VFIO: + RTE_LOG(ERR, EAL, "Hotplug doesn't support vfio yet\n"); + break; + case RTE_PT_IGB_UIO: + case RTE_PT_UIO_GENERIC: + /* unmap resources for devices that use uio */ + pci_uio_unmap_resource(dev); + break; + default: + RTE_LOG(DEBUG, EAL, " Not managed by known pt driver," + " skipped\n"); + break; + } +} +#endif /* RTE_LIBRTE_EAL_HOTPLUG */ + /* * If vendor/device ID match, call the devinit() function of the * driver. @@ -665,6 +690,75 @@ rte_eal_pci_probe_one_driver(struct rte_pci_driver *dr, struct rte_pci_device *d return 1; } +#ifdef RTE_LIBRTE_EAL_HOTPLUG +/* + * If vendor/device ID match, call the devuninit() function of the + * driver. + */ +int +rte_eal_pci_close_one_driver(struct rte_pci_driver *dr, + struct rte_pci_device *dev) +{ + struct rte_pci_id *id_table; + + if ((dr == NULL) || (dev == NULL)) + return -EINVAL; + + for (id_table = dr->id_table ; id_table->vendor_id != 0; id_table++) { + + /* check if device's identifiers match the driver's ones */ + if (id_table->vendor_id != dev->id.vendor_id && + id_table->vendor_id != PCI_ANY_ID) + continue; + if (id_table->device_id != dev->id.device_id && + id_table->device_id != PCI_ANY_ID) + continue; + if (id_table->subsystem_vendor_id != + dev->id.subsystem_vendor_id && + id_table->subsystem_vendor_id != PCI_ANY_ID) + continue; + if (id_table->subsystem_device_id != + dev->id.subsystem_device_id && + id_table->subsystem_device_id != PCI_ANY_ID) + continue; + + struct rte_pci_addr *loc = &dev->addr; + + RTE_LOG(DEBUG, EAL, + "PCI device "PCI_PRI_FMT" on NUMA socket %i\n", + loc->domain, loc->bus, loc->devid, + loc->function, dev->numa_node); + + RTE_LOG(DEBUG, EAL, " remove driver: %x:%x %s\n", + dev->id.vendor_id, dev->id.device_id, + dr->name); + + /* call the driver devuninit() function */ + if (dr->devuninit && (dr->devuninit(dev) < 0)) + return -1; /* negative value is an error */ + + /* clear driver structure */ + dev->driver = NULL; + + if (dr->drv_flags & RTE_PCI_DRV_NEED_MAPPING) + /* unmap resources for devices that use igb_uio */ + pci_unmap_device(dev); + + return 0; + } + /* return positive value if driver is not found */ + return 1; +} +#else /* RTE_LIBRTE_EAL_HOTPLUG */ +int +rte_eal_pci_close_one_driver(struct rte_pci_driver *dr __rte_unused, + struct rte_pci_device *dev __rte_unused) +{ + RTE_LOG(ERR, EAL, "Hotplug support isn't enabled\n"); + return -1; +} +#endif /* RTE_LIBRTE_EAL_HOTPLUG */ + /* Init the PCI EAL subsystem */ int rte_eal_pci_init(void) -- 1.9.1
This new parameter is needed to keep device type like PCI or virtual. Port detaching processes are different between PCI device and virtual device. RTE_ETH_DEV_PCI indicates device type is PCI. RTE_ETH_DEV_VIRTUAL indicates device is virtual. v12: - Add missing symbol in version map. (Thanks to Iremonger, Bernard) v10: - Change order of version.map. (Thanks to Thomas Monjalon) - Fix comment of "rte_ethdev.h". (Thanks to Thomas Monjalon) v9: - Fix commit log. - RTE_ETH_DEV_PHYSICAL is replaced by RTE_ETH_DEV_PCI. (Thanks to Thomas Monjalon) v8: - NONE_TRACE is replaced by NO_TRACE. - Add missing symbol in version map. (Thanks to Qiu, Michael and Iremonger, Bernard) v4: - Fix comments of rte_eth_dev_type. Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp> --- app/test/virtual_pmd.c | 2 +- lib/librte_ether/rte_ethdev.c | 25 +++++++++++++++++++++++-- lib/librte_ether/rte_ethdev.h | 25 ++++++++++++++++++++++++- lib/librte_ether/rte_ether_version.map | 1 + lib/librte_pmd_af_packet/rte_eth_af_packet.c | 2 +- lib/librte_pmd_bond/rte_eth_bond_api.c | 2 +- lib/librte_pmd_pcap/rte_eth_pcap.c | 2 +- lib/librte_pmd_ring/rte_eth_ring.c | 2 +- lib/librte_pmd_xenvirt/rte_eth_xenvirt.c | 2 +- 9 files changed, 54 insertions(+), 9 deletions(-) diff --git a/app/test/virtual_pmd.c b/app/test/virtual_pmd.c index cd9faf3..01a3913 100644 --- a/app/test/virtual_pmd.c +++ b/app/test/virtual_pmd.c @@ -580,7 +580,7 @@ virtual_ethdev_create(const char *name, struct ether_addr *mac_addr, goto err; /* reserve an ethdev entry */ - eth_dev = rte_eth_dev_allocate(name); + eth_dev = rte_eth_dev_allocate(name, RTE_ETH_DEV_PCI); if (eth_dev == NULL) goto err; diff --git a/lib/librte_ether/rte_ethdev.c b/lib/librte_ether/rte_ethdev.c index 7067620..f176f1e 100644 --- a/lib/librte_ether/rte_ethdev.c +++ b/lib/librte_ether/rte_ethdev.c @@ -227,7 +227,7 @@ rte_eth_dev_find_free_port(void) } struct rte_eth_dev * -rte_eth_dev_allocate(const char *name) +rte_eth_dev_allocate(const char *name, enum rte_eth_dev_type type) { uint8_t port_id; struct rte_eth_dev *eth_dev; @@ -251,6 +251,7 @@ rte_eth_dev_allocate(const char *name) snprintf(eth_dev->data->name, sizeof(eth_dev->data->name), "%s", name); eth_dev->data->port_id = port_id; eth_dev->attached = DEV_ATTACHED; + eth_dev->dev_type = type; nb_ports++; return eth_dev; } @@ -262,6 +263,7 @@ rte_eth_dev_release_port(struct rte_eth_dev *eth_dev) return -EINVAL; eth_dev->attached = 0; + eth_dev->dev_type = RTE_ETH_DEV_UNKNOWN; nb_ports--; return 0; } @@ -300,7 +302,7 @@ rte_eth_dev_init(struct rte_pci_driver *pci_drv, rte_eth_dev_create_unique_device_name(ethdev_name, sizeof(ethdev_name), pci_dev); - eth_dev = rte_eth_dev_allocate(ethdev_name); + eth_dev = rte_eth_dev_allocate(ethdev_name, RTE_ETH_DEV_PCI); if (eth_dev == NULL) return -ENOMEM; @@ -426,6 +428,14 @@ rte_eth_dev_count(void) return (nb_ports); } +enum rte_eth_dev_type +rte_eth_dev_get_device_type(uint8_t port_id) +{ + if (!rte_eth_dev_is_valid_port(port_id)) + return -1; + return rte_eth_devices[port_id].dev_type; +} + int rte_eth_dev_save(struct rte_eth_dev *devs, size_t size) { @@ -523,6 +533,17 @@ rte_eth_dev_is_detachable(uint8_t port_id) return -EINVAL; } + if (rte_eth_devices[port_id].dev_type == RTE_ETH_DEV_PCI) { + switch (rte_eth_devices[port_id].pci_dev->pt_driver) { + case RTE_PT_IGB_UIO: + case RTE_PT_UIO_GENERIC: + break; + case RTE_PT_VFIO: + default: + return -ENOTSUP; + } + } + drv_flags = rte_eth_devices[port_id].driver->pci_drv.drv_flags; return !(drv_flags & RTE_PCI_DRV_DETACHABLE); } diff --git a/lib/librte_ether/rte_ethdev.h b/lib/librte_ether/rte_ethdev.h index d4cfafb..1a978ed 100644 --- a/lib/librte_ether/rte_ethdev.h +++ b/lib/librte_ether/rte_ethdev.h @@ -1382,6 +1382,17 @@ struct eth_dev_ops { }; /** + * The eth device type + */ +enum rte_eth_dev_type { + RTE_ETH_DEV_UNKNOWN, /**< unknown device type */ + RTE_ETH_DEV_PCI, + /**< Physical function and Virtual function of PCI devices */ + RTE_ETH_DEV_VIRTUAL, /**< non hardware device */ + RTE_ETH_DEV_MAX /**< max value of this enum */ +}; + +/** * @internal * The generic data structure associated with each ethernet device. * @@ -1400,6 +1411,7 @@ struct rte_eth_dev { struct rte_pci_device *pci_dev; /**< PCI info. supplied by probing */ struct rte_eth_dev_cb_list callbacks; /**< User application callbacks */ uint8_t attached; /**< Flag indicating the port is attached */ + enum rte_eth_dev_type dev_type; /**< Flag indicating the device type */ }; struct rte_eth_dev_sriov { @@ -1477,6 +1489,15 @@ extern uint8_t rte_eth_dev_count(void); /** * Function for internal use by port hotplug functions. + * Get the device type to know whether the device is physical or virtual. + * @param port_id The port identifier of the Ethernet device + * @return + * - Device type. + */ +extern enum rte_eth_dev_type rte_eth_dev_get_device_type(uint8_t port_id); + +/** + * Function for internal use by port hotplug functions. * Copies current ethdev structures to the specified pointer. * * @param devs The pointer to the ethdev structures @@ -1565,10 +1586,12 @@ extern struct rte_eth_dev *rte_eth_dev_allocated(const char *name); * to that slot for the driver to use. * * @param name Unique identifier name for each Ethernet device + * @param type Device type of this Ethernet device * @return * - Slot in the rte_dev_devices array for a new device; */ -struct rte_eth_dev *rte_eth_dev_allocate(const char *name); +struct rte_eth_dev *rte_eth_dev_allocate(const char *name, + enum rte_eth_dev_type type); /** * Function for internal use by dummy drivers primarily, e.g. ring-based diff --git a/lib/librte_ether/rte_ether_version.map b/lib/librte_ether/rte_ether_version.map index 099c769..112644f 100644 --- a/lib/librte_ether/rte_ether_version.map +++ b/lib/librte_ether/rte_ether_version.map @@ -35,6 +35,7 @@ DPDK_2.0 { rte_eth_dev_flow_ctrl_set; rte_eth_dev_get_addr_by_port; rte_eth_dev_get_changed_port; + rte_eth_dev_get_device_type; rte_eth_dev_get_mtu; rte_eth_dev_get_name_by_port; rte_eth_dev_get_port_by_addr; diff --git a/lib/librte_pmd_af_packet/rte_eth_af_packet.c b/lib/librte_pmd_af_packet/rte_eth_af_packet.c index 1ffe1cd..80e9bdf 100644 --- a/lib/librte_pmd_af_packet/rte_eth_af_packet.c +++ b/lib/librte_pmd_af_packet/rte_eth_af_packet.c @@ -649,7 +649,7 @@ rte_pmd_init_internals(const char *name, } /* reserve an ethdev entry */ - *eth_dev = rte_eth_dev_allocate(name); + *eth_dev = rte_eth_dev_allocate(name, RTE_ETH_DEV_VIRTUAL); if (*eth_dev == NULL) goto error; diff --git a/lib/librte_pmd_bond/rte_eth_bond_api.c b/lib/librte_pmd_bond/rte_eth_bond_api.c index b1d0bd3..f026c8d 100644 --- a/lib/librte_pmd_bond/rte_eth_bond_api.c +++ b/lib/librte_pmd_bond/rte_eth_bond_api.c @@ -257,7 +257,7 @@ rte_eth_bond_create(const char *name, uint8_t mode, uint8_t socket_id) } /* reserve an ethdev entry */ - eth_dev = rte_eth_dev_allocate(name); + eth_dev = rte_eth_dev_allocate(name, RTE_ETH_DEV_VIRTUAL); if (eth_dev == NULL) { RTE_BOND_LOG(ERR, "Unable to allocate rte_eth_dev"); goto err; diff --git a/lib/librte_pmd_pcap/rte_eth_pcap.c b/lib/librte_pmd_pcap/rte_eth_pcap.c index d299288..af7fae8 100644 --- a/lib/librte_pmd_pcap/rte_eth_pcap.c +++ b/lib/librte_pmd_pcap/rte_eth_pcap.c @@ -709,7 +709,7 @@ rte_pmd_init_internals(const char *name, const unsigned nb_rx_queues, goto error; /* reserve an ethdev entry */ - *eth_dev = rte_eth_dev_allocate(name); + *eth_dev = rte_eth_dev_allocate(name, RTE_ETH_DEV_VIRTUAL); if (*eth_dev == NULL) goto error; diff --git a/lib/librte_pmd_ring/rte_eth_ring.c b/lib/librte_pmd_ring/rte_eth_ring.c index a23e933..cd0913b 100644 --- a/lib/librte_pmd_ring/rte_eth_ring.c +++ b/lib/librte_pmd_ring/rte_eth_ring.c @@ -297,7 +297,7 @@ rte_eth_from_rings(const char *name, struct rte_ring *const rx_queues[], goto error; /* reserve an ethdev entry */ - eth_dev = rte_eth_dev_allocate(name); + eth_dev = rte_eth_dev_allocate(name, RTE_ETH_DEV_VIRTUAL); if (eth_dev == NULL) goto error; diff --git a/lib/librte_pmd_xenvirt/rte_eth_xenvirt.c b/lib/librte_pmd_xenvirt/rte_eth_xenvirt.c index 04e30c9..bc403d6 100644 --- a/lib/librte_pmd_xenvirt/rte_eth_xenvirt.c +++ b/lib/librte_pmd_xenvirt/rte_eth_xenvirt.c @@ -648,7 +648,7 @@ eth_dev_xenvirt_create(const char *name, const char *params, goto err; /* reserve an ethdev entry */ - eth_dev = rte_eth_dev_allocate(name); + eth_dev = rte_eth_dev_allocate(name, RTE_ETH_DEV_VIRTUAL); if (eth_dev == NULL) goto err; -- 1.9.1
These functions are used for attaching or detaching a port. When rte_eal_dev_attach() is called, the function tries to realize the device name as pci address. If this is done successfully, rte_eal_dev_attach() will attach physical device port. If not, attaches virtual devive port. When rte_eal_dev_detach() is called, the function gets the device type of this port to know whether the port is come from physical or virtual. And then specific detaching function will be called. v11: - Remove needless devargs handling codes. - Replace get_vdev_name() by rte_eal_parse_devargs_str(). - Replace rte_eal_vdev_find_and_init by rte_eal_vdev_init() - Replace rte_eal_vdev_find_and_uninit by rte_eal_vdev_uninit() - Fix rte_eal_dev_init() to use rte_eal_vdev_init(). (Thanks to Maxime Leroy) v10: - Add comments. - Change order of version.map. (Thanks to Thomas Monjalon) v9: - Fix comments. - Use strcmp() instead of strncmp(). - Remove RTE_EAL_INVOKE_TYPE_PROBE/CLOSE. - Change definition of rte_dev_uninit_t. (Thanks to Thomas Monjalon and Maxime Leroy) v8: - Add missing symbol in version map. (Thanks to Qiu, Michael and Iremonger, Bernard) v7: - Fix typo of warning messages. (Thanks to Qiu, Michael) v5: - Change function names like below. rte_eal_dev_find_and_invoke() to rte_eal_vdev_find_and_invoke(). rte_eal_dev_invoke() to rte_eal_vdev_invoke(). - Add code to handle a return value of rte_eal_devargs_remove(). - Fix pci address format in rte_eal_dev_detach(). v4: - Fix comment. - Add error checking. - Fix indent of 'if' statement. - Change function name. Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp> --- lib/librte_eal/common/eal_common_dev.c | 285 ++++++++++++++++++++++-- lib/librte_eal/common/eal_common_devargs.c | 46 ++-- lib/librte_eal/common/eal_private.h | 11 + lib/librte_eal/common/include/rte_dev.h | 33 +++ lib/librte_eal/common/include/rte_devargs.h | 28 +++ lib/librte_eal/linuxapp/eal/Makefile | 1 + lib/librte_eal/linuxapp/eal/eal_pci.c | 6 +- lib/librte_eal/linuxapp/eal/rte_eal_version.map | 2 + 8 files changed, 378 insertions(+), 34 deletions(-) diff --git a/lib/librte_eal/common/eal_common_dev.c b/lib/librte_eal/common/eal_common_dev.c index eae5656..7d4dce6 100644 --- a/lib/librte_eal/common/eal_common_dev.c +++ b/lib/librte_eal/common/eal_common_dev.c @@ -32,10 +32,13 @@ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ +#include <stdio.h> +#include <limits.h> #include <string.h> #include <inttypes.h> #include <sys/queue.h> +#include <rte_ethdev.h> #include <rte_dev.h> #include <rte_devargs.h> #include <rte_debug.h> @@ -61,6 +64,37 @@ rte_eal_driver_unregister(struct rte_driver *driver) TAILQ_REMOVE(&dev_driver_list, driver, next); } +static int +rte_eal_vdev_init(const char *name, const char *args) +{ + struct rte_driver *driver; + + if (name == NULL) + return -EINVAL; + + TAILQ_FOREACH(driver, &dev_driver_list, next) { + if (driver->type != PMD_VDEV) + continue; + + /* + * search a driver prefix in virtual device name. + * For example, if the driver is pcap PMD, driver->name + * will be "eth_pcap", but "name" will be "eth_pcapN". + * So use strncmp to compare. + */ + if (!strncmp(driver->name, name, strlen(driver->name))) { + driver->init(name, args); + break; + } + } + + if (driver == NULL) { + RTE_LOG(WARNING, EAL, "no driver found for %s\n", name); + return -EINVAL; + } + return 0; +} + int rte_eal_dev_init(void) { @@ -79,23 +113,10 @@ rte_eal_dev_init(void) if (devargs->type != RTE_DEVTYPE_VIRTUAL) continue; - TAILQ_FOREACH(driver, &dev_driver_list, next) { - if (driver->type != PMD_VDEV) - continue; - - /* search a driver prefix in virtual device name */ - if (!strncmp(driver->name, devargs->virtual.drv_name, - strlen(driver->name))) { - driver->init(devargs->virtual.drv_name, - devargs->args); - break; - } - } - - if (driver == NULL) { + if (rte_eal_vdev_init(devargs->virtual.drv_name, + devargs->args)) rte_panic("no driver found for %s\n", devargs->virtual.drv_name); - } } /* Once the vdevs are initalized, start calling all the pdev drivers */ @@ -107,3 +128,237 @@ rte_eal_dev_init(void) } return 0; } + +/* So far, DPDK hotplug function only supports linux */ +#ifdef RTE_LIBRTE_EAL_HOTPLUG +static int +rte_eal_vdev_uninit(const char *name) +{ + struct rte_driver *driver; + + if (name == NULL) + return -EINVAL; + + TAILQ_FOREACH(driver, &dev_driver_list, next) { + if (driver->type != PMD_VDEV) + continue; + + /* + * search a driver prefix in virtual device name. + * For example, if the driver is pcap PMD, driver->name + * will be "eth_pcap", but "name" will be "eth_pcapN". + * So use strncmp to compare. + */ + if (!strncmp(driver->name, name, strlen(driver->name))) { + driver->uninit(name); + break; + } + } + + if (driver == NULL) { + RTE_LOG(WARNING, EAL, "no driver found for %s\n", name); + return 1; + } + return 0; +} + +/* attach the new physical device, then store port_id of the device */ +static int +rte_eal_dev_attach_pdev(struct rte_pci_addr *addr, uint8_t *port_id) +{ + uint8_t new_port_id; + struct rte_eth_dev devs[RTE_MAX_ETHPORTS]; + + if ((addr == NULL) || (port_id == NULL)) + goto err; + + /* save current port status */ + if (rte_eth_dev_save(devs, sizeof(devs))) + goto err; + /* re-construct pci_device_list */ + if (rte_eal_pci_scan()) + goto err; + /* invoke probe func of the driver can handle the new device. + * TODO: + * rte_eal_pci_probe_one() should return port_id. + * And rte_eth_dev_save() and rte_eth_dev_get_changed_port() + * should be removed. */ + if (rte_eal_pci_probe_one(addr)) + goto err; + /* get port_id enabled by above procedures */ + if (rte_eth_dev_get_changed_port(devs, &new_port_id)) + goto err; + + *port_id = new_port_id; + return 0; +err: + RTE_LOG(ERR, EAL, "Driver, cannot attach the device\n"); + return -1; +} + +/* detach the new physical device, then store pci_addr of the device */ +static int +rte_eal_dev_detach_pdev(uint8_t port_id, struct rte_pci_addr *addr) +{ + struct rte_pci_addr freed_addr; + struct rte_pci_addr vp; + + if (addr == NULL) + goto err; + + /* check whether the driver supports detach feature, or not */ + if (rte_eth_dev_is_detachable(port_id)) + goto err; + + /* get pci address by port id */ + if (rte_eth_dev_get_addr_by_port(port_id, &freed_addr)) + goto err; + + /* Zerod pci addr means the port comes from virtual device */ + vp.domain = vp.bus = vp.devid = vp.function = 0; + if (rte_eal_compare_pci_addr(&vp, &freed_addr) == 0) + goto err; + + /* invoke close func of the driver, + * also remove the device from pci_device_list */ + if (rte_eal_pci_close_one(&freed_addr)) + goto err; + + *addr = freed_addr; + return 0; +err: + RTE_LOG(ERR, EAL, "Driver, cannot detach the device\n"); + return -1; +} + +/* attach the new virtual device, then store port_id of the device */ +static int +rte_eal_dev_attach_vdev(const char *vdevargs, uint8_t *port_id) +{ + char *name = NULL, *args = NULL; + uint8_t new_port_id; + struct rte_eth_dev devs[RTE_MAX_ETHPORTS]; + int ret = -1; + + if ((vdevargs == NULL) || (port_id == NULL)) + goto end; + + /* parse vdevargs, then retrieve device name and args */ + if (rte_eal_parse_devargs_str(vdevargs, &name, &args)) + goto end; + + /* save current port status */ + if (rte_eth_dev_save(devs, sizeof(devs))) + goto end; + /* walk around dev_driver_list to find the driver of the device, + * then invoke probe function o the driver. + * TODO: + * rte_eal_vdev_init() should return port_id, + * And rte_eth_dev_save() and rte_eth_dev_get_changed_port() + * should be removed. */ + if (rte_eal_vdev_init(name, args)) + goto end; + /* get port_id enabled by above procedures */ + if (rte_eth_dev_get_changed_port(devs, &new_port_id)) + goto end; + ret = 0; + +end: + if (name) + free(name); + if (args) + free(args); + + *port_id = new_port_id; + if (ret < 0) + RTE_LOG(ERR, EAL, "Driver, cannot attach the device\n"); + return ret; +} + +/* detach the new virtual device, then store the name of the device */ +static int +rte_eal_dev_detach_vdev(uint8_t port_id, char *vdevname) +{ + char name[RTE_ETH_NAME_MAX_LEN]; + + if (vdevname == NULL) + goto err; + + /* check whether the driver supports detach feature, or not */ + if (rte_eth_dev_is_detachable(port_id)) + goto err; + + /* get device name by port id */ + if (rte_eth_dev_get_name_by_port(port_id, name)) + goto err; + /* walk around dev_driver_list to find the driver of the device, + * then invoke close function o the driver */ + if (rte_eal_vdev_uninit(name)) + goto err; + + strncpy(vdevname, name, sizeof(name)); + return 0; +err: + RTE_LOG(ERR, EAL, "Driver, cannot detach the device\n"); + return -1; +} + +/* attach the new device, then store port_id of the device */ +int +rte_eal_dev_attach(const char *devargs, uint8_t *port_id) +{ + struct rte_pci_addr addr; + + if ((devargs == NULL) || (port_id == NULL)) + return -EINVAL; + + if (eal_parse_pci_DomBDF(devargs, &addr) == 0) + return rte_eal_dev_attach_pdev(&addr, port_id); + else + return rte_eal_dev_attach_vdev(devargs, port_id); +} + +/* detach the device, then store the name of the device */ +int +rte_eal_dev_detach(uint8_t port_id, char *name) +{ + struct rte_pci_addr addr; + int ret; + + if (name == NULL) + return -EINVAL; + + if (rte_eth_dev_get_device_type(port_id) == RTE_ETH_DEV_PCI) { + ret = rte_eth_dev_get_addr_by_port(port_id, &addr); + if (ret < 0) + return ret; + + ret = rte_eal_dev_detach_pdev(port_id, &addr); + if (ret == 0) + snprintf(name, RTE_ETH_NAME_MAX_LEN, + "%04x:%02x:%02x.%d", + addr.domain, addr.bus, + addr.devid, addr.function); + + return ret; + } else + return rte_eal_dev_detach_vdev(port_id, name); +} +#else /* RTE_LIBRTE_EAL_HOTPLUG */ +int +rte_eal_dev_attach(const char *devargs __rte_unused, + uint8_t *port_id __rte_unused) +{ + RTE_LOG(ERR, EAL, "Hotplug support isn't enabled\n"); + return -1; +} + +/* detach the device, then store the name of the device */ +int +rte_eal_dev_detach(uint8_t port_id __rte_unused, + char *name __rte_unused) +{ + RTE_LOG(ERR, EAL, "Hotplug support isn't enabled\n"); + return -1; +} +#endif /* RTE_LIBRTE_EAL_HOTPLUG */ diff --git a/lib/librte_eal/common/eal_common_devargs.c b/lib/librte_eal/common/eal_common_devargs.c index 3aace08..2988a70 100644 --- a/lib/librte_eal/common/eal_common_devargs.c +++ b/lib/librte_eal/common/eal_common_devargs.c @@ -44,13 +44,41 @@ struct rte_devargs_list devargs_list = TAILQ_HEAD_INITIALIZER(devargs_list); +int +rte_eal_parse_devargs_str(const char *devargs_str, + char **drvname, char **drvargs) +{ + char *sep; + + if ((devargs_str == NULL) || drvname == NULL || drvargs == NULL) + return -1; + + *drvname = strdup(devargs_str); + if (drvname == NULL) { + RTE_LOG(ERR, EAL, "cannot allocate temp memory for devargs\n"); + return -1; + } + + /* set the first ',' to '\0' to split name and arguments */ + sep = strchr(*drvname, ','); + if (sep != NULL) { + sep[0] = '\0'; + *drvargs = strdup(sep + 1); + if (*drvargs == NULL) { + RTE_LOG(ERR, EAL, "cannot allocate for devargs args\n"); + free(*drvname); + return -1; + } + } + return 0; +} + /* store a whitelist parameter for later parsing */ int rte_eal_devargs_add(enum rte_devtype devtype, const char *devargs_str) { struct rte_devargs *devargs = NULL; char *buf = NULL; - char *sep; int ret; /* use malloc instead of rte_malloc as it's called early at init */ @@ -62,22 +90,8 @@ rte_eal_devargs_add(enum rte_devtype devtype, const char *devargs_str) memset(devargs, 0, sizeof(*devargs)); devargs->type = devtype; - buf = strdup(devargs_str); - if (buf == NULL) { - RTE_LOG(ERR, EAL, "cannot allocate temp memory for devargs\n"); + if (rte_eal_parse_devargs_str(devargs_str, &buf, &devargs->args)) goto fail; - } - - /* set the first ',' to '\0' to split name and arguments */ - sep = strchr(buf, ','); - if (sep != NULL) { - sep[0] = '\0'; - devargs->args = strdup(sep + 1); - if (devargs->args == NULL) { - RTE_LOG(ERR, EAL, "cannot allocate for devargs args\n"); - goto fail; - } - } switch (devargs->type) { case RTE_DEVTYPE_WHITELISTED_PCI: diff --git a/lib/librte_eal/common/eal_private.h b/lib/librte_eal/common/eal_private.h index 4acf5a0..98b286a 100644 --- a/lib/librte_eal/common/eal_private.h +++ b/lib/librte_eal/common/eal_private.h @@ -154,6 +154,17 @@ struct rte_pci_driver; struct rte_pci_device; /** + * Scan the content of the PCI bus, and the devices in the devices + * list + * + * This function is private to EAL. + * + * @return + * 0 on success, negative on error + */ +int rte_eal_pci_scan(void); + +/** * Mmap memory for single PCI device * * This function is private to EAL. diff --git a/lib/librte_eal/common/include/rte_dev.h b/lib/librte_eal/common/include/rte_dev.h index f7e3a10..a5ac770 100644 --- a/lib/librte_eal/common/include/rte_dev.h +++ b/lib/librte_eal/common/include/rte_dev.h @@ -47,6 +47,7 @@ extern "C" { #endif #include <sys/queue.h> +#include <rte_pci.h> /** Double linked list of device drivers. */ TAILQ_HEAD(rte_driver_list, rte_driver); @@ -57,6 +58,11 @@ TAILQ_HEAD(rte_driver_list, rte_driver); typedef int (rte_dev_init_t)(const char *name, const char *args); /** + * Uninitilization function called for each device driver once. + */ +typedef int (rte_dev_uninit_t)(const char *name); + +/** * Driver type enumeration */ enum pmd_type { @@ -72,6 +78,7 @@ struct rte_driver { enum pmd_type type; /**< PMD Driver type */ const char *name; /**< Driver name. */ rte_dev_init_t *init; /**< Device init. function. */ + rte_dev_uninit_t *uninit; /**< Device uninit. function. */ }; /** @@ -93,6 +100,32 @@ void rte_eal_driver_register(struct rte_driver *driver); void rte_eal_driver_unregister(struct rte_driver *driver); /** + * Attach a new device. + * + * @param devargs + * A pointer to a strings array describing the new device + * to be attached. The strings should be a pci address like + * '0000:01:00.0' or virtual device name like 'eth_pcap0'. + * @param port_id + * A pointer to a port identifier actually attached. + * @return + * 0 on success and port_id is filled, negative on error + */ +int rte_eal_dev_attach(const char *devargs, uint8_t *port_id); + +/** + * Detach a device. + * + * @param port_id + * The port identifier of the device to detach. + * @param addr + * A pointer to a device name actually detached. + * @return + * 0 on success and devname is filled, negative on error + */ +int rte_eal_dev_detach(uint8_t port_id, char *devname); + +/** * Initalize all the registered drivers in this process */ int rte_eal_dev_init(void); diff --git a/lib/librte_eal/common/include/rte_devargs.h b/lib/librte_eal/common/include/rte_devargs.h index 996e180..274a92c 100644 --- a/lib/librte_eal/common/include/rte_devargs.h +++ b/lib/librte_eal/common/include/rte_devargs.h @@ -99,6 +99,34 @@ TAILQ_HEAD(rte_devargs_list, rte_devargs); extern struct rte_devargs_list devargs_list; /** + * Parse a devargs string. + * + * For PCI devices, the format of arguments string is "PCI_ADDR" or + * "PCI_ADDR,key=val,key2=val2,...". Examples: "08:00.1", "0000:5:00.0", + * "04:00.0,arg=val". + * + * For virtual devices, the format of arguments string is "DRIVER_NAME*" + * or "DRIVER_NAME*,key=val,key2=val2,...". Examples: "eth_ring", + * "eth_ring0", "eth_pmdAnything,arg=0:arg2=1". + * + * The function parses the arguments string to get driver name and driver + * arguments. + * + * @param devargs_str + * The arguments as given by the user. + * @param drvname + * The pointer to the string to store parsed driver name. + * @param drvargs + * The pointer to the string to store parsed driver arguments. + * + * @return + * - 0 on success + * - A negative value on error + */ +int rte_eal_parse_devargs_str(const char *devargs_str, + char **drvname, char **drvargs); + +/** * Add a device to the user device list * * For PCI devices, the format of arguments string is "PCI_ADDR" or diff --git a/lib/librte_eal/linuxapp/eal/Makefile b/lib/librte_eal/linuxapp/eal/Makefile index e117cec..b59b201 100644 --- a/lib/librte_eal/linuxapp/eal/Makefile +++ b/lib/librte_eal/linuxapp/eal/Makefile @@ -45,6 +45,7 @@ CFLAGS += -I$(RTE_SDK)/lib/librte_eal/common/include CFLAGS += -I$(RTE_SDK)/lib/librte_ring CFLAGS += -I$(RTE_SDK)/lib/librte_mempool CFLAGS += -I$(RTE_SDK)/lib/librte_malloc +CFLAGS += -I$(RTE_SDK)/lib/librte_mbuf CFLAGS += -I$(RTE_SDK)/lib/librte_ether CFLAGS += -I$(RTE_SDK)/lib/librte_ivshmem CFLAGS += -I$(RTE_SDK)/lib/librte_pmd_ring diff --git a/lib/librte_eal/linuxapp/eal/eal_pci.c b/lib/librte_eal/linuxapp/eal/eal_pci.c index 85d9d09..ab77b68 100644 --- a/lib/librte_eal/linuxapp/eal/eal_pci.c +++ b/lib/librte_eal/linuxapp/eal/eal_pci.c @@ -439,8 +439,8 @@ error: * Scan the content of the PCI bus, and the devices in the devices * list */ -static int -pci_scan(void) +int +rte_eal_pci_scan(void) { struct dirent *e; DIR *dir; @@ -772,7 +772,7 @@ rte_eal_pci_init(void) if (internal_config.no_pci) return 0; - if (pci_scan() < 0) { + if (rte_eal_pci_scan() < 0) { RTE_LOG(ERR, EAL, "%s(): Cannot scan PCI bus\n", __func__); return -1; } diff --git a/lib/librte_eal/linuxapp/eal/rte_eal_version.map b/lib/librte_eal/linuxapp/eal/rte_eal_version.map index 5ed6e4d..5d40e02 100644 --- a/lib/librte_eal/linuxapp/eal/rte_eal_version.map +++ b/lib/librte_eal/linuxapp/eal/rte_eal_version.map @@ -20,6 +20,8 @@ DPDK_2.0 { rte_dump_tailq; rte_eal_alarm_cancel; rte_eal_alarm_set; + rte_eal_dev_attach; + rte_eal_dev_detach; rte_eal_dev_init; rte_eal_devargs_add; rte_eal_devargs_dump; -- 1.9.1
This patch adds a new section for describing port hotplug framework. Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp> --- doc/guides/prog_guide/index.rst | 1 + doc/guides/prog_guide/port_hotplug_framework.rst | 110 +++++++++++++++++++++++ 2 files changed, 111 insertions(+) create mode 100644 doc/guides/prog_guide/port_hotplug_framework.rst diff --git a/doc/guides/prog_guide/index.rst b/doc/guides/prog_guide/index.rst index de69682..60a6ac5 100644 --- a/doc/guides/prog_guide/index.rst +++ b/doc/guides/prog_guide/index.rst @@ -71,6 +71,7 @@ Programmer's Guide packet_classif_access_ctrl packet_framework vhost_lib + port_hotplug_framework source_org dev_kit_build_system dev_kit_root_make_help diff --git a/doc/guides/prog_guide/port_hotplug_framework.rst b/doc/guides/prog_guide/port_hotplug_framework.rst new file mode 100644 index 0000000..355ae28 --- /dev/null +++ b/doc/guides/prog_guide/port_hotplug_framework.rst @@ -0,0 +1,110 @@ +.. BSD LICENSE + Copyright(c) 2015 IGEL Co.,Ltd. All rights reserved. + All rights reserved. + + Redistribution and use in source and binary forms, with or without + modification, are permitted provided that the following conditions + are met: + + * Redistributions of source code must retain the above copyright + notice, this list of conditions and the following disclaimer. + * Redistributions in binary form must reproduce the above copyright + notice, this list of conditions and the following disclaimer in + the documentation and/or other materials provided with the + distribution. + * Neither the name of IGEL Co.,Ltd. nor the names of its + contributors may be used to endorse or promote products derived + from this software without specific prior written permission. + + THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS + "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT + LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR + A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT + OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, + SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT + LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, + DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY + THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT + (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE + OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + +Port Hotplug Framework +====================== + +The Port Hotplug Framework provides DPDK applications with the ability to +attach and detach ports at runtime. Because the framework depends on PMD +implementation, the ports that PMDs cannot handle are out of scope of this +framework. Furthermore, after detaching a port from a DPDK application, the +framework doesn't provide a way for removing the devices from the system. +For the ports backed by a physical NIC, the kernel will need to support PCI +Hotplug feature. + +Overview +-------- + +The basic requirements of the Port Hotplug Framework are: + +* DPDK applications that use the Port Hotplug Framework must manage their + own ports. + + The Port Hotplug Framework is implemented to allow DPDK applications to + manage ports. For example, when DPDK applications call the port attach + function, the attached port number is returned. DPDK applications can + also detach the port by port number. + +* Kernel support is needed for attaching or detaching physical device + ports. + + To attach new physical device ports, the device will be recognized by + userspace driver I/O framework in kernel at first. Then DPDK + applications can call the Port Hotplug functions to attach the ports. + For detaching, steps are vice versa. + +* Before detaching, they must be stopped and closed. + + DPDK applications must call "rte_eth_dev_stop()" and + "rte_eth_dev_close()" APIs before detaching ports. These functions will + start finalization sequence of the PMDs. + +* The framework doesn't affect legacy DPDK applications behavior. + + If the Port Hotplug functions aren't called, all legacy DPDK apps can + still work without modifications. + +Port Hotplug API overview +------------------------- + +* Attaching a port + + "rte_eal_dev_attach()" API attaches a port to DPDK application, and + returns the attached port number. Before calling the API, the device + should be recognized by an userspace driver I/O framework. The API + receives a pci address like "0000:01:00.0" or a virtual device name + like "eth_pcap0,iface=eth0". In the case of virtual device name, the + format is the same as the general "--vdev" option of DPDK. + +* Detaching a port + + "rte_eal_dev_detach()" API detaches a port from DPDK application, and + returns a pci address of the detached device or a virtual device name + of the device. + +Reference +--------- + + "testpmd" supports the Port Hotplug Framework. + +Limitations +----------- + +* The Port Hotplug APIs are not thread safe. + +* The framework can only be enabled with Linux. BSD is not supported. + +* To detach a port, the port should be backed by a device that igb_uio + manages. VFIO is not supported. + +* Not all PMDs support detaching feature. + To know whether a PMD can support detaching, search for the + "RTE_PCI_DRV_DETACHABLE" flag in PMD implementation. If the flag is + defined in the PMD, detaching is supported. -- 1.9.1
This patch adds finalization code to free resources allocated by the PMD. v6: - Fix a paramter of rte_eth_dev_free(). v4: - Change function name. Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp> --- lib/librte_pmd_pcap/rte_eth_pcap.c | 40 ++++++++++++++++++++++++++++++++++++++ 1 file changed, 40 insertions(+) diff --git a/lib/librte_pmd_pcap/rte_eth_pcap.c b/lib/librte_pmd_pcap/rte_eth_pcap.c index af7fae8..5e94930 100644 --- a/lib/librte_pmd_pcap/rte_eth_pcap.c +++ b/lib/librte_pmd_pcap/rte_eth_pcap.c @@ -498,6 +498,13 @@ static struct eth_dev_ops ops = { .stats_reset = eth_stats_reset, }; +static struct eth_driver rte_pcap_pmd = { + .pci_drv = { + .name = "rte_pcap_pmd", + .drv_flags = RTE_PCI_DRV_DETACHABLE, + }, +}; + /* * Function handler that opens the pcap file for reading a stores a * reference of it for use it later on. @@ -713,6 +720,10 @@ rte_pmd_init_internals(const char *name, const unsigned nb_rx_queues, if (*eth_dev == NULL) goto error; + /* check length of device name */ + if ((strlen((*eth_dev)->data->name) + 1) > sizeof(data->name)) + goto error; + /* now put it all together * - store queue data in internals, * - store numa_node info in pci_driver @@ -739,10 +750,13 @@ rte_pmd_init_internals(const char *name, const unsigned nb_rx_queues, data->nb_tx_queues = (uint16_t)nb_tx_queues; data->dev_link = pmd_link; data->mac_addrs = ð_addr; + strncpy(data->name, + (*eth_dev)->data->name, strlen((*eth_dev)->data->name)); (*eth_dev)->data = data; (*eth_dev)->dev_ops = &ops; (*eth_dev)->pci_dev = pci_dev; + (*eth_dev)->driver = &rte_pcap_pmd; return 0; @@ -927,10 +941,36 @@ rte_pmd_pcap_devinit(const char *name, const char *params) } +static int +rte_pmd_pcap_devuninit(const char *name) +{ + struct rte_eth_dev *eth_dev = NULL; + + RTE_LOG(INFO, PMD, "Closing pcap ethdev on numa socket %u\n", + rte_socket_id()); + + if (name == NULL) + return -1; + + /* reserve an ethdev entry */ + eth_dev = rte_eth_dev_allocated(name); + if (eth_dev == NULL) + return -1; + + rte_free(eth_dev->data->dev_private); + rte_free(eth_dev->data); + rte_free(eth_dev->pci_dev); + + rte_eth_dev_release_port(eth_dev); + + return 0; +} + static struct rte_driver pmd_pcap_drv = { .name = "eth_pcap", .type = PMD_VDEV, .init = rte_pmd_pcap_devinit, + .uninit = rte_pmd_pcap_devuninit, }; PMD_REGISTER_DRIVER(pmd_pcap_drv); -- 1.9.1
The patch introduces following commands. - port attach [ident] - port detach [port_id] - attach: attaching a port - detach: detaching a port - ident: pci address of physical device. Or device name and parameters of virtual device. (ex. 0000:02:00.0, eth_pcap0,iface=eth0) - port_id: port identifier v7: - Fix doc. (Thanks to Iremonger, Bernard) - Fix port checking implementation of star_port(); (Thanks to Qiu, Michael) v5: - Add testpmd documentation. (Thanks to Iremonger, Bernard) v4: - Fix strings of command help. Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp> --- app/test-pmd/cmdline.c | 137 +++++++++++++++---- app/test-pmd/config.c | 102 ++++++++------ app/test-pmd/parameters.c | 22 ++- app/test-pmd/testpmd.c | 199 +++++++++++++++++++++------- app/test-pmd/testpmd.h | 18 ++- doc/guides/testpmd_app_ug/testpmd_funcs.rst | 57 ++++++++ 6 files changed, 409 insertions(+), 126 deletions(-) diff --git a/app/test-pmd/cmdline.c b/app/test-pmd/cmdline.c index c6a1627..b78c659 100644 --- a/app/test-pmd/cmdline.c +++ b/app/test-pmd/cmdline.c @@ -513,6 +513,12 @@ static void cmd_help_long_parsed(void *parsed_result, "port close (port_id|all)\n" " Close all ports or port_id.\n\n" + "port attach (ident)\n" + " Attach physical or virtual dev by pci address or virtual device name\n\n" + + "port detach (port_id)\n" + " Detach physical or virtual dev by port_id\n\n" + "port config (port_id|all)" " speed (10|100|1000|10000|40000|auto)" " duplex (half|full|auto)\n" @@ -793,6 +799,89 @@ cmdline_parse_inst_t cmd_operate_specific_port = { }, }; +/* *** attach a specified port *** */ +struct cmd_operate_attach_port_result { + cmdline_fixed_string_t port; + cmdline_fixed_string_t keyword; + cmdline_fixed_string_t identifier; +}; + +static void cmd_operate_attach_port_parsed(void *parsed_result, + __attribute__((unused)) struct cmdline *cl, + __attribute__((unused)) void *data) +{ + struct cmd_operate_attach_port_result *res = parsed_result; + + if (!strcmp(res->keyword, "attach")) + attach_port(res->identifier); + else + printf("Unknown parameter\n"); +} + +cmdline_parse_token_string_t cmd_operate_attach_port_port = + TOKEN_STRING_INITIALIZER(struct cmd_operate_attach_port_result, + port, "port"); +cmdline_parse_token_string_t cmd_operate_attach_port_keyword = + TOKEN_STRING_INITIALIZER(struct cmd_operate_attach_port_result, + keyword, "attach"); +cmdline_parse_token_string_t cmd_operate_attach_port_identifier = + TOKEN_STRING_INITIALIZER(struct cmd_operate_attach_port_result, + identifier, NULL); + +cmdline_parse_inst_t cmd_operate_attach_port = { + .f = cmd_operate_attach_port_parsed, + .data = NULL, + .help_str = "port attach identifier, " + "identifier: pci address or virtual dev name", + .tokens = { + (void *)&cmd_operate_attach_port_port, + (void *)&cmd_operate_attach_port_keyword, + (void *)&cmd_operate_attach_port_identifier, + NULL, + }, +}; + +/* *** detach a specified port *** */ +struct cmd_operate_detach_port_result { + cmdline_fixed_string_t port; + cmdline_fixed_string_t keyword; + uint8_t port_id; +}; + +static void cmd_operate_detach_port_parsed(void *parsed_result, + __attribute__((unused)) struct cmdline *cl, + __attribute__((unused)) void *data) +{ + struct cmd_operate_detach_port_result *res = parsed_result; + + if (!strcmp(res->keyword, "detach")) + detach_port(res->port_id); + else + printf("Unknown parameter\n"); +} + +cmdline_parse_token_string_t cmd_operate_detach_port_port = + TOKEN_STRING_INITIALIZER(struct cmd_operate_detach_port_result, + port, "port"); +cmdline_parse_token_string_t cmd_operate_detach_port_keyword = + TOKEN_STRING_INITIALIZER(struct cmd_operate_detach_port_result, + keyword, "detach"); +cmdline_parse_token_num_t cmd_operate_detach_port_port_id = + TOKEN_NUM_INITIALIZER(struct cmd_operate_detach_port_result, + port_id, UINT8); + +cmdline_parse_inst_t cmd_operate_detach_port = { + .f = cmd_operate_detach_port_parsed, + .data = NULL, + .help_str = "port detach port_id", + .tokens = { + (void *)&cmd_operate_detach_port_port, + (void *)&cmd_operate_detach_port_keyword, + (void *)&cmd_operate_detach_port_port_id, + NULL, + }, +}; + /* *** configure speed for all ports *** */ struct cmd_config_speed_all { cmdline_fixed_string_t port; @@ -847,7 +936,7 @@ cmd_config_speed_all_parsed(void *parsed_result, return; } - for (pid = 0; pid < nb_ports; pid++) { + FOREACH_PORT(pid, ports) { ports[pid].dev_conf.link_speed = link_speed; ports[pid].dev_conf.link_duplex = link_duplex; } @@ -915,10 +1004,8 @@ cmd_config_speed_specific_parsed(void *parsed_result, return; } - if (res->id >= nb_ports) { - printf("Port id %d must be less than %d\n", res->id, nb_ports); + if (port_id_is_invalid(res->id, ENABLED_WARN)) return; - } if (!strcmp(res->value1, "10")) link_speed = ETH_LINK_SPEED_10; @@ -1489,7 +1576,7 @@ cmd_config_rxtx_queue_parsed(void *parsed_result, return; } - if (port_id_is_invalid(res->portid)) + if (port_id_is_invalid(res->portid, ENABLED_WARN)) return; if (port_is_started(res->portid) != 1) { @@ -2889,7 +2976,7 @@ cmd_csum_parsed(void *parsed_result, int hw = 0; uint16_t mask = 0; - if (port_id_is_invalid(res->port_id)) { + if (port_id_is_invalid(res->port_id, ENABLED_WARN)) { printf("invalid port %d\n", res->port_id); return; } @@ -2981,10 +3068,8 @@ cmd_csum_tunnel_parsed(void *parsed_result, { struct cmd_csum_tunnel_result *res = parsed_result; - if (port_id_is_invalid(res->port_id)) { - printf("invalid port %d\n", res->port_id); + if (port_id_is_invalid(res->port_id, ENABLED_WARN)) return; - } if (!strcmp(res->onoff, "on")) ports[res->port_id].tx_ol_flags |= @@ -3039,7 +3124,7 @@ cmd_tso_set_parsed(void *parsed_result, struct cmd_tso_set_result *res = parsed_result; struct rte_eth_dev_info dev_info; - if (port_id_is_invalid(res->port_id)) + if (port_id_is_invalid(res->port_id, ENABLED_WARN)) return; if (!strcmp(res->mode, "set")) @@ -4015,10 +4100,8 @@ static void cmd_set_bond_mac_addr_parsed(void *parsed_result, struct cmd_set_bond_mac_addr_result *res = parsed_result; int ret; - if (res->port_num >= nb_ports) { - printf("Port id %d must be less than %d\n", res->port_num, nb_ports); + if (port_id_is_invalid(res->port_num, ENABLED_WARN)) return; - } ret = rte_eth_bond_mac_address_set(res->port_num, &res->address); @@ -4255,7 +4338,7 @@ static void cmd_set_promisc_mode_parsed(void *parsed_result, /* all ports */ if (allports) { - for (i = 0; i < nb_ports; i++) { + FOREACH_PORT(i, ports) { if (enable) rte_eth_promiscuous_enable(i); else @@ -4335,7 +4418,7 @@ static void cmd_set_allmulti_mode_parsed(void *parsed_result, /* all ports */ if (allports) { - for (i = 0; i < nb_ports; i++) { + FOREACH_PORT(i, ports) { if (enable) rte_eth_allmulticast_enable(i); else @@ -5023,25 +5106,25 @@ static void cmd_showportall_parsed(void *parsed_result, struct cmd_showportall_result *res = parsed_result; if (!strcmp(res->show, "clear")) { if (!strcmp(res->what, "stats")) - for (i = 0; i < nb_ports; i++) + FOREACH_PORT(i, ports) nic_stats_clear(i); else if (!strcmp(res->what, "xstats")) - for (i = 0; i < nb_ports; i++) + FOREACH_PORT(i, ports) nic_xstats_clear(i); } else if (!strcmp(res->what, "info")) - for (i = 0; i < nb_ports; i++) + FOREACH_PORT(i, ports) port_infos_display(i); else if (!strcmp(res->what, "stats")) - for (i = 0; i < nb_ports; i++) + FOREACH_PORT(i, ports) nic_stats_display(i); else if (!strcmp(res->what, "xstats")) - for (i = 0; i < nb_ports; i++) + FOREACH_PORT(i, ports) nic_xstats_display(i); else if (!strcmp(res->what, "fdir")) - for (i = 0; i < nb_ports; i++) + FOREACH_PORT(i, ports) fdir_get_infos(i); else if (!strcmp(res->what, "stat_qmap")) - for (i = 0; i < nb_ports; i++) + FOREACH_PORT(i, ports) nic_stats_mapping_display(i); } @@ -8685,6 +8768,8 @@ cmdline_parse_ctx_t main_ctx[] = { (cmdline_parse_inst_t *)&cmd_set_qmap, (cmdline_parse_inst_t *)&cmd_operate_port, (cmdline_parse_inst_t *)&cmd_operate_specific_port, + (cmdline_parse_inst_t *)&cmd_operate_attach_port, + (cmdline_parse_inst_t *)&cmd_operate_detach_port, (cmdline_parse_inst_t *)&cmd_config_speed_all, (cmdline_parse_inst_t *)&cmd_config_speed_specific, (cmdline_parse_inst_t *)&cmd_config_rx_tx, @@ -8756,7 +8841,7 @@ prompt(void) static void cmd_reconfig_device_queue(portid_t id, uint8_t dev, uint8_t queue) { - if (id < nb_ports) { + if (!port_id_is_invalid(id, DISABLED_WARN)) { /* check if need_reconfig has been set to 1 */ if (ports[id].need_reconfig == 0) ports[id].need_reconfig = dev; @@ -8766,7 +8851,7 @@ cmd_reconfig_device_queue(portid_t id, uint8_t dev, uint8_t queue) } else { portid_t pid; - for (pid = 0; pid < nb_ports; pid++) { + FOREACH_PORT(pid, ports) { /* check if need_reconfig has been set to 1 */ if (ports[pid].need_reconfig == 0) ports[pid].need_reconfig = dev; @@ -8784,10 +8869,8 @@ bypass_is_supported(portid_t port_id) struct rte_port *port; struct rte_pci_id *pci_id; - if (port_id >= nb_ports) { - printf("\tPort id must be less than %d.\n", nb_ports); + if (port_id_is_invalid(port_id, ENABLED_WARN)) return 0; - } /* Get the device id. */ port = &ports[port_id]; diff --git a/app/test-pmd/config.c b/app/test-pmd/config.c index d436ce8..49be819 100644 --- a/app/test-pmd/config.c +++ b/app/test-pmd/config.c @@ -112,11 +112,15 @@ nic_stats_display(portid_t port_id) struct rte_eth_stats stats; struct rte_port *port = &ports[port_id]; uint8_t i; + portid_t pid; static const char *nic_stats_border = "########################"; - if (port_id >= nb_ports) { - printf("Invalid port, range is [0, %d]\n", nb_ports - 1); + if (port_id_is_invalid(port_id, ENABLED_WARN)) { + printf("Valid port range is [0"); + FOREACH_PORT(pid, ports) + printf(", %d", pid); + printf("]\n"); return; } rte_eth_stats_get(port_id, &stats); @@ -189,8 +193,13 @@ nic_stats_display(portid_t port_id) void nic_stats_clear(portid_t port_id) { - if (port_id >= nb_ports) { - printf("Invalid port, range is [0, %d]\n", nb_ports - 1); + portid_t pid; + + if (port_id_is_invalid(port_id, ENABLED_WARN)) { + printf("Valid port range is [0"); + FOREACH_PORT(pid, ports) + printf(", %d", pid); + printf("]\n"); return; } rte_eth_stats_reset(port_id); @@ -237,11 +246,15 @@ nic_stats_mapping_display(portid_t port_id) { struct rte_port *port = &ports[port_id]; uint16_t i; + portid_t pid; static const char *nic_stats_mapping_border = "########################"; - if (port_id >= nb_ports) { - printf("Invalid port, range is [0, %d]\n", nb_ports - 1); + if (port_id_is_invalid(port_id, ENABLED_WARN)) { + printf("Valid port range is [0"); + FOREACH_PORT(pid, ports) + printf(", %d", pid); + printf("]\n"); return; } @@ -290,9 +303,13 @@ port_infos_display(portid_t port_id) int vlan_offload; struct rte_mempool * mp; static const char *info_border = "*********************"; + portid_t pid; - if (port_id >= nb_ports) { - printf("Invalid port, range is [0, %d]\n", nb_ports - 1); + if (port_id_is_invalid(port_id, ENABLED_WARN)) { + printf("Valid port range is [0"); + FOREACH_PORT(pid, ports) + printf(", %d", pid); + printf("]\n"); return; } port = &ports[port_id]; @@ -365,11 +382,14 @@ port_infos_display(portid_t port_id) } int -port_id_is_invalid(portid_t port_id) +port_id_is_invalid(portid_t port_id, enum print_warning warning) { - if (port_id < nb_ports) + if (ports[port_id].enabled) return 0; - printf("Invalid port %d (must be < nb_ports=%d)\n", port_id, nb_ports); + + if (warning == ENABLED_WARN) + printf("Invalid port %d\n", port_id); + return 1; } @@ -428,7 +448,7 @@ port_reg_bit_display(portid_t port_id, uint32_t reg_off, uint8_t bit_x) uint32_t reg_v; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; if (port_reg_off_is_invalid(port_id, reg_off)) return; @@ -447,7 +467,7 @@ port_reg_bit_field_display(portid_t port_id, uint32_t reg_off, uint8_t l_bit; uint8_t h_bit; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; if (port_reg_off_is_invalid(port_id, reg_off)) return; @@ -474,7 +494,7 @@ port_reg_display(portid_t port_id, uint32_t reg_off) { uint32_t reg_v; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; if (port_reg_off_is_invalid(port_id, reg_off)) return; @@ -488,7 +508,7 @@ port_reg_bit_set(portid_t port_id, uint32_t reg_off, uint8_t bit_pos, { uint32_t reg_v; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; if (port_reg_off_is_invalid(port_id, reg_off)) return; @@ -516,7 +536,7 @@ port_reg_bit_field_set(portid_t port_id, uint32_t reg_off, uint8_t l_bit; uint8_t h_bit; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; if (port_reg_off_is_invalid(port_id, reg_off)) return; @@ -550,7 +570,7 @@ port_reg_bit_field_set(portid_t port_id, uint32_t reg_off, void port_reg_set(portid_t port_id, uint32_t reg_off, uint32_t reg_v) { - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; if (port_reg_off_is_invalid(port_id, reg_off)) return; @@ -563,7 +583,7 @@ port_mtu_set(portid_t port_id, uint16_t mtu) { int diag; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; diag = rte_eth_dev_set_mtu(port_id, mtu); if (diag == 0) @@ -726,7 +746,7 @@ rx_ring_desc_display(portid_t port_id, queueid_t rxq_id, uint16_t rxd_id) { const struct rte_memzone *rx_mz; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; if (rx_queue_id_is_invalid(rxq_id)) return; @@ -743,7 +763,7 @@ tx_ring_desc_display(portid_t port_id, queueid_t txq_id, uint16_t txd_id) { const struct rte_memzone *tx_mz; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; if (tx_queue_id_is_invalid(txq_id)) return; @@ -804,7 +824,7 @@ port_rss_reta_info(portid_t port_id, uint16_t i, idx, shift; int ret; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; ret = rte_eth_dev_rss_reta_query(port_id, reta_conf, nb_entries); @@ -859,7 +879,7 @@ port_rss_hash_conf_show(portid_t port_id, int show_rss_key) uint8_t i; int diag; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; /* Get RSS hash key if asked to display it */ rss_conf.rss_key = (show_rss_key) ? rss_key : NULL; @@ -1423,12 +1443,8 @@ set_fwd_ports_list(unsigned int *portlist, unsigned int nb_pt) again: for (i = 0; i < nb_pt; i++) { port_id = (portid_t) portlist[i]; - if (port_id >= nb_ports) { - printf("Invalid port id %u >= %u\n", - (unsigned int) port_id, - (unsigned int) nb_ports); + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; - } if (record_now) fwd_ports_ids[i] = port_id; } @@ -1586,7 +1602,7 @@ vlan_extend_set(portid_t port_id, int on) int diag; int vlan_offload; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; vlan_offload = rte_eth_dev_get_vlan_offload(port_id); @@ -1608,7 +1624,7 @@ rx_vlan_strip_set(portid_t port_id, int on) int diag; int vlan_offload; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; vlan_offload = rte_eth_dev_get_vlan_offload(port_id); @@ -1629,7 +1645,7 @@ rx_vlan_strip_set_on_queue(portid_t port_id, uint16_t queue_id, int on) { int diag; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; diag = rte_eth_dev_set_vlan_strip_on_queue(port_id, queue_id, on); @@ -1644,7 +1660,7 @@ rx_vlan_filter_set(portid_t port_id, int on) int diag; int vlan_offload; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; vlan_offload = rte_eth_dev_get_vlan_offload(port_id); @@ -1665,7 +1681,7 @@ rx_vft_set(portid_t port_id, uint16_t vlan_id, int on) { int diag; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; if (vlan_id_is_invalid(vlan_id)) return; @@ -1682,7 +1698,7 @@ rx_vlan_all_filter_set(portid_t port_id, int on) { uint16_t vlan_id; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; for (vlan_id = 0; vlan_id < 4096; vlan_id++) rx_vft_set(port_id, vlan_id, on); @@ -1692,7 +1708,7 @@ void vlan_tpid_set(portid_t port_id, uint16_t tp_id) { int diag; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; diag = rte_eth_dev_set_vlan_ether_type(port_id, tp_id); @@ -1707,7 +1723,7 @@ vlan_tpid_set(portid_t port_id, uint16_t tp_id) void tx_vlan_set(portid_t port_id, uint16_t vlan_id) { - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; if (vlan_id_is_invalid(vlan_id)) return; @@ -1718,7 +1734,7 @@ tx_vlan_set(portid_t port_id, uint16_t vlan_id) void tx_vlan_reset(portid_t port_id) { - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; ports[port_id].tx_ol_flags &= ~TESTPMD_TX_OFFLOAD_INSERT_VLAN; } @@ -1726,7 +1742,7 @@ tx_vlan_reset(portid_t port_id) void tx_vlan_pvid_set(portid_t port_id, uint16_t vlan_id, int on) { - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; rte_eth_dev_set_vlan_pvid(port_id, vlan_id, on); @@ -1738,7 +1754,7 @@ set_qmap(portid_t port_id, uint8_t is_rx, uint16_t queue_id, uint8_t map_value) uint16_t i; uint8_t existing_mapping_found = 0; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; if (is_rx ? (rx_queue_id_is_invalid(queue_id)) : (tx_queue_id_is_invalid(queue_id))) @@ -1904,7 +1920,7 @@ fdir_get_infos(portid_t port_id) static const char *fdir_stats_border = "########################"; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; ret = rte_eth_dev_filter_supported(port_id, RTE_ETH_FILTER_FDIR); if (ret < 0) { @@ -2030,7 +2046,7 @@ set_vf_traffic(portid_t port_id, uint8_t is_rx, uint16_t vf, uint8_t on) { int diag; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; if (is_rx) diag = rte_eth_dev_set_vf_rx(port_id,vf,on); @@ -2052,7 +2068,7 @@ set_vf_rx_vlan(portid_t port_id, uint16_t vlan_id, uint64_t vf_mask, uint8_t on) { int diag; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; if (vlan_id_is_invalid(vlan_id)) return; @@ -2069,7 +2085,7 @@ set_queue_rate_limit(portid_t port_id, uint16_t queue_idx, uint16_t rate) int diag; struct rte_eth_link link; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return 1; rte_eth_link_get_nowait(port_id, &link); if (rate > link.link_speed) { @@ -2094,7 +2110,7 @@ set_vf_rate_limit(portid_t port_id, uint16_t vf, uint16_t rate, uint64_t q_msk) if (q_msk == 0) return 0; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return 1; rte_eth_link_get_nowait(port_id, &link); if (rate > link.link_speed) { diff --git a/app/test-pmd/parameters.c b/app/test-pmd/parameters.c index 473f824..fa5f2a8 100644 --- a/app/test-pmd/parameters.c +++ b/app/test-pmd/parameters.c @@ -366,6 +366,7 @@ parse_portnuma_config(const char *q_arg) }; unsigned long int_fld[_NUM_FLD]; char *str_fld[_NUM_FLD]; + portid_t pid; /* reset from value set at definition */ while ((p = strchr(p0,'(')) != NULL) { @@ -387,8 +388,11 @@ parse_portnuma_config(const char *q_arg) return -1; } port_id = (uint8_t)int_fld[FLD_PORT]; - if (port_id >= nb_ports) { - printf("Invalid port, range is [0, %d]\n", nb_ports - 1); + if (port_id_is_invalid(port_id, ENABLED_WARN)) { + printf("Valid port range is [0"); + FOREACH_PORT(pid, ports) + printf(", %d", pid); + printf("]\n"); return -1; } socket_id = (uint8_t)int_fld[FLD_SOCKET]; @@ -419,6 +423,7 @@ parse_ringnuma_config(const char *q_arg) }; unsigned long int_fld[_NUM_FLD]; char *str_fld[_NUM_FLD]; + portid_t pid; #define RX_RING_ONLY 0x1 #define TX_RING_ONLY 0x2 #define RXTX_RING 0x3 @@ -443,8 +448,11 @@ parse_ringnuma_config(const char *q_arg) return -1; } port_id = (uint8_t)int_fld[FLD_PORT]; - if (port_id >= nb_ports) { - printf("Invalid port, range is [0, %d]\n", nb_ports - 1); + if (port_id_is_invalid(port_id, ENABLED_WARN)) { + printf("Valid port range is [0"); + FOREACH_PORT(pid, ports) + printf(", %d", pid); + printf("]\n"); return -1; } socket_id = (uint8_t)int_fld[FLD_SOCKET]; @@ -615,12 +623,12 @@ launch_args_parse(int argc, char** argv) #endif if (!strcmp(lgopts[opt_idx].name, "nb-ports")) { n = atoi(optarg); - if (n > 0 && n <= nb_ports) + if (n > 0 && + !port_id_is_invalid(n, DISABLED_WARN)) nb_fwd_ports = (uint8_t) n; else rte_exit(EXIT_FAILURE, - "nb-ports should be > 0 and <= %d\n", - nb_ports); + "Invalid port %d\n", n); } if (!strcmp(lgopts[opt_idx].name, "nb-cores")) { n = atoi(optarg); diff --git a/app/test-pmd/testpmd.c b/app/test-pmd/testpmd.c index 605163b..44bb8b3 100644 --- a/app/test-pmd/testpmd.c +++ b/app/test-pmd/testpmd.c @@ -71,6 +71,7 @@ #include <rte_pci.h> #include <rte_ether.h> #include <rte_ethdev.h> +#include <rte_dev.h> #include <rte_string_fns.h> #ifdef RTE_LIBRTE_PMD_XENVIRT #include <rte_eth_xenvirt.h> @@ -315,7 +316,7 @@ uint16_t nb_rx_queue_stats_mappings = 0; /* Forward function declarations */ static void map_port_queue_stats_mapping_registers(uint8_t pi, struct rte_port *port); -static void check_all_ports_link_status(uint8_t port_num, uint32_t port_mask); +static void check_all_ports_link_status(uint32_t port_mask); /* * Check if all the ports are started. @@ -324,6 +325,20 @@ static void check_all_ports_link_status(uint8_t port_num, uint32_t port_mask); static int all_ports_started(void); /* + * Find next enabled port + */ +portid_t +find_next_port(portid_t p, struct rte_port *ports, int size) +{ + if (ports == NULL) + rte_exit(-EINVAL, "failed to find a next port id\n"); + + while ((ports[p].enabled == 0) && (p < size)) + p++; + return p; +} + +/* * Setup default configuration. */ static void @@ -552,7 +567,8 @@ init_config(void) + RTE_TEST_TX_DESC_MAX + MAX_PKT_BURST; if (!numa_support) - nb_mbuf_per_pool = (nb_mbuf_per_pool * nb_ports); + nb_mbuf_per_pool = + (nb_mbuf_per_pool * RTE_MAX_ETHPORTS); } if (!numa_support) { @@ -565,14 +581,19 @@ init_config(void) /* Configuration of Ethernet ports. */ ports = rte_zmalloc("testpmd: ports", - sizeof(struct rte_port) * nb_ports, + sizeof(struct rte_port) * RTE_MAX_ETHPORTS, RTE_CACHE_LINE_SIZE); if (ports == NULL) { - rte_exit(EXIT_FAILURE, "rte_zmalloc(%d struct rte_port) " - "failed\n", nb_ports); + rte_exit(EXIT_FAILURE, + "rte_zmalloc(%d struct rte_port) failed\n", + RTE_MAX_ETHPORTS); } - for (pid = 0; pid < nb_ports; pid++) { + /* enabled allocated ports */ + for (pid = 0; pid < nb_ports; pid++) + ports[pid].enabled = 1; + + FOREACH_PORT(pid, ports) { port = &ports[pid]; rte_eth_dev_info_get(pid, &port->dev_info); @@ -602,8 +623,7 @@ init_config(void) nb_mbuf_per_pool = nb_mbuf_per_pool/nb_ports; for (i = 0; i < MAX_SOCKET; i++) { - nb_mbuf = (nb_mbuf_per_pool * - port_per_socket[i]); + nb_mbuf = (nb_mbuf_per_pool * RTE_MAX_ETHPORTS); if (nb_mbuf) mbuf_pool_create(mbuf_data_size, nb_mbuf,i); @@ -635,14 +655,6 @@ reconfig(portid_t new_port_id, unsigned socket_id) struct rte_port *port; /* Reconfiguration of Ethernet ports. */ - ports = rte_realloc(ports, - sizeof(struct rte_port) * nb_ports, - RTE_CACHE_LINE_SIZE); - if (ports == NULL) { - rte_exit(EXIT_FAILURE, "rte_realloc(%d struct rte_port) failed\n", - nb_ports); - } - port = &ports[new_port_id]; rte_eth_dev_info_get(new_port_id, &port->dev_info); @@ -663,7 +675,7 @@ init_fwd_streams(void) streamid_t sm_id, nb_fwd_streams_new; /* set socket id according to numa or not */ - for (pid = 0; pid < nb_ports; pid++) { + FOREACH_PORT(pid, ports) { port = &ports[pid]; if (nb_rxq > port->dev_info.max_rx_queues) { printf("Fail: nb_rxq(%d) is greater than " @@ -1264,7 +1276,7 @@ all_ports_started(void) portid_t pi; struct rte_port *port; - for (pi = 0; pi < nb_ports; pi++) { + FOREACH_PORT(pi, ports) { port = &ports[pi]; /* Check if there is a port which is not started */ if (port->port_status != RTE_PORT_STARTED) @@ -1276,6 +1288,45 @@ all_ports_started(void) } int +all_ports_stopped(void) +{ + portid_t pi; + struct rte_port *port; + + FOREACH_PORT(pi, ports) { + port = &ports[pi]; + if (port->port_status != RTE_PORT_STOPPED) + return 0; + } + + return 1; +} + +int +port_is_started(portid_t port_id) +{ + if (port_id_is_invalid(port_id, ENABLED_WARN)) + return 0; + + if (ports[port_id].port_status != RTE_PORT_STARTED) + return 0; + + return 1; +} + +static int +port_is_closed(portid_t port_id) +{ + if (port_id_is_invalid(port_id, ENABLED_WARN)) + return 0; + + if (ports[port_id].port_status != RTE_PORT_CLOSED) + return 0; + + return 1; +} + +int start_port(portid_t pid) { int diag, need_check_link_status = 0; @@ -1296,8 +1347,8 @@ start_port(portid_t pid) if(dcb_config) dcb_test = 1; - for (pi = 0; pi < nb_ports; pi++) { - if (pid < nb_ports && pid != pi) + FOREACH_PORT(pi, ports) { + if (pid != pi && pid != (portid_t)RTE_PORT_ALL) continue; port = &ports[pi]; @@ -1421,7 +1472,7 @@ start_port(portid_t pid) } if (need_check_link_status && !no_link_check) - check_all_ports_link_status(nb_ports, RTE_PORT_ALL); + check_all_ports_link_status(RTE_PORT_ALL); else printf("Please stop the ports first\n"); @@ -1446,8 +1497,8 @@ stop_port(portid_t pid) } printf("Stopping ports...\n"); - for (pi = 0; pi < nb_ports; pi++) { - if (pid < nb_ports && pid != pi) + FOREACH_PORT(pi, ports) { + if (!port_id_is_invalid(pid, DISABLED_WARN) && pid != pi) continue; port = &ports[pi]; @@ -1463,7 +1514,7 @@ stop_port(portid_t pid) need_check_link_status = 1; } if (need_check_link_status && !no_link_check) - check_all_ports_link_status(nb_ports, RTE_PORT_ALL); + check_all_ports_link_status(RTE_PORT_ALL); printf("Done\n"); } @@ -1481,8 +1532,8 @@ close_port(portid_t pid) printf("Closing ports...\n"); - for (pi = 0; pi < nb_ports; pi++) { - if (pid < nb_ports && pid != pi) + FOREACH_PORT(pi, ports) { + if (!port_id_is_invalid(pid, DISABLED_WARN) && pid != pi) continue; port = &ports[pi]; @@ -1502,31 +1553,83 @@ close_port(portid_t pid) printf("Done\n"); } -int -all_ports_stopped(void) +void +attach_port(char *identifier) { - portid_t pi; - struct rte_port *port; + portid_t i, j, pi = 0; - for (pi = 0; pi < nb_ports; pi++) { - port = &ports[pi]; - if (port->port_status != RTE_PORT_STOPPED) - return 0; + printf("Attaching a new port...\n"); + + if (identifier == NULL) { + printf("Invalid parameters are specified\n"); + return; } - return 1; + if (test_done == 0) { + printf("Please stop forwarding first\n"); + return; + } + + if (rte_eal_dev_attach(identifier, &pi)) + return; + + ports[pi].enabled = 1; + reconfig(pi, rte_eth_dev_socket_id(pi)); + rte_eth_promiscuous_enable(pi); + + nb_ports = rte_eth_dev_count(); + + /* set_default_fwd_ports_config(); */ + bzero(fwd_ports_ids, sizeof(fwd_ports_ids)); + i = 0; + FOREACH_PORT(j, ports) { + fwd_ports_ids[i] = j; + i++; + } + nb_cfg_ports = nb_ports; + nb_fwd_ports++; + + ports[pi].port_status = RTE_PORT_STOPPED; + + printf("Port %d is attached. Now total ports is %d\n", pi, nb_ports); + printf("Done\n"); } -int -port_is_started(portid_t port_id) +void +detach_port(uint8_t port_id) { - if (port_id_is_invalid(port_id)) - return -1; + portid_t i, pi = 0; + char name[RTE_ETH_NAME_MAX_LEN]; - if (ports[port_id].port_status != RTE_PORT_STARTED) - return 0; + printf("Detaching a port...\n"); - return 1; + if (!port_is_closed(port_id)) { + printf("Please close port first\n"); + return; + } + + rte_eth_promiscuous_disable(port_id); + + if (rte_eal_dev_detach(port_id, name)) + return; + + ports[port_id].enabled = 0; + nb_ports = rte_eth_dev_count(); + + /* set_default_fwd_ports_config(); */ + bzero(fwd_ports_ids, sizeof(fwd_ports_ids)); + i = 0; + FOREACH_PORT(pi, ports) { + fwd_ports_ids[i] = pi; + i++; + } + nb_cfg_ports = nb_ports; + nb_fwd_ports--; + + printf("Port '%s' is detached. Now total ports is %d\n", + name, nb_ports); + printf("Done\n"); + return; } void @@ -1534,7 +1637,7 @@ pmd_test_exit(void) { portid_t pt_id; - for (pt_id = 0; pt_id < nb_ports; pt_id++) { + FOREACH_PORT(pt_id, ports) { printf("Stopping port %d...", pt_id); fflush(stdout); rte_eth_dev_close(pt_id); @@ -1553,7 +1656,7 @@ struct pmd_test_command { /* Check the link status of all ports in up to 9s, and print them finally */ static void -check_all_ports_link_status(uint8_t port_num, uint32_t port_mask) +check_all_ports_link_status(uint32_t port_mask) { #define CHECK_INTERVAL 100 /* 100ms */ #define MAX_CHECK_TIME 90 /* 9s (90 * 100ms) in total */ @@ -1564,7 +1667,7 @@ check_all_ports_link_status(uint8_t port_num, uint32_t port_mask) fflush(stdout); for (count = 0; count <= MAX_CHECK_TIME; count++) { all_ports_up = 1; - for (portid = 0; portid < port_num; portid++) { + FOREACH_PORT(portid, ports) { if ((port_mask & (1 << portid)) == 0) continue; memset(&link, 0, sizeof(link)); @@ -1729,7 +1832,7 @@ init_port_config(void) portid_t pid; struct rte_port *port; - for (pid = 0; pid < nb_ports; pid++) { + FOREACH_PORT(pid, ports) { port = &ports[pid]; port->dev_conf.rxmode = rx_mode; port->dev_conf.fdir_conf = fdir_conf; @@ -1908,7 +2011,7 @@ main(int argc, char** argv) nb_ports = (portid_t) rte_eth_dev_count(); if (nb_ports == 0) - rte_exit(EXIT_FAILURE, "No probed ethernet device\n"); + RTE_LOG(WARNING, EAL, "No probed ethernet devices\n"); set_def_fwd_config(); if (nb_lcores == 0) @@ -1930,7 +2033,7 @@ main(int argc, char** argv) rte_exit(EXIT_FAILURE, "Start ports failed\n"); /* set all ports to promiscuous mode by default */ - for (port_id = 0; port_id < nb_ports; port_id++) + FOREACH_PORT(port_id, ports) rte_eth_promiscuous_enable(port_id); #ifdef RTE_LIBRTE_CMDLINE diff --git a/app/test-pmd/testpmd.h b/app/test-pmd/testpmd.h index 126bef7..0d5a526 100644 --- a/app/test-pmd/testpmd.h +++ b/app/test-pmd/testpmd.h @@ -137,6 +137,7 @@ struct fwd_stream { * The data structure associated with each port. */ struct rte_port { + uint8_t enabled; /**< Port enabled or not */ struct rte_eth_dev_info dev_info; /**< PCI info + driver name */ struct rte_eth_conf dev_conf; /**< Port configuration. */ struct ether_addr eth_addr; /**< Port ethernet address */ @@ -162,6 +163,14 @@ struct rte_port { struct rte_eth_txconf tx_conf; /**< tx configuration */ }; +extern portid_t __rte_unused +find_next_port(portid_t p, struct rte_port *ports, int size); + +#define FOREACH_PORT(p, ports) \ + for (p = find_next_port(0, ports, RTE_MAX_ETHPORTS); \ + p < RTE_MAX_ETHPORTS; \ + p = find_next_port(p + 1, ports, RTE_MAX_ETHPORTS)) + /** * The data structure associated with each forwarding logical core. * The logical cores are internally numbered by a core index from 0 to @@ -522,6 +531,8 @@ int init_port_dcb_config(portid_t pid,struct dcb_config *dcb_conf); int start_port(portid_t pid); void stop_port(portid_t pid); void close_port(portid_t pid); +void attach_port(char *identifier); +void detach_port(uint8_t port_id); int all_ports_stopped(void); int port_is_started(portid_t port_id); void pmd_test_exit(void); @@ -548,10 +559,15 @@ void get_syn_filter(uint8_t port_id); void get_ethertype_filter(uint8_t port_id, uint16_t index); void get_2tuple_filter(uint8_t port_id, uint16_t index); void get_5tuple_filter(uint8_t port_id, uint16_t index); -int port_id_is_invalid(portid_t port_id); int rx_queue_id_is_invalid(queueid_t rxq_id); int tx_queue_id_is_invalid(queueid_t txq_id); +enum print_warning { + ENABLED_WARN = 0, + DISABLED_WARN +}; +int port_id_is_invalid(portid_t port_id, enum print_warning warning); + /* * Work-around of a compilation error with ICC on invocations of the * rte_be_to_cpu_16() function. diff --git a/doc/guides/testpmd_app_ug/testpmd_funcs.rst b/doc/guides/testpmd_app_ug/testpmd_funcs.rst index afe1970..a99e14d 100644 --- a/doc/guides/testpmd_app_ug/testpmd_funcs.rst +++ b/doc/guides/testpmd_app_ug/testpmd_funcs.rst @@ -807,6 +807,63 @@ The following sections show functions for configuring ports. Port configuration changes only become active when forwarding is started/restarted. +port attach +~~~~~~~~~~~ + +Attach a port specified by pci address or virtual device args. + +To attach a new pci device, the device should be recognized by kernel first. +Then it should be moved under DPDK management. +Finally the port can be attached to testpmd. +On the other hand, to attach a port created by virtual device, above steps are not needed. + +port attach (identifier) + +For example, to attach a port whose pci address is 0000:02:00.0. + +.. code-block:: console + + testpmd> port attach 0000:02:00.0 + Attaching a new port... + ... snip ... + Port 0 is attached. Now total ports is 1 + Done + +For example, to attach a port created by pcap PMD. + +.. code-block:: console + + testpmd> port attach eth_pcap0,iface=eth0 + Attaching a new port... + ... snip ... + Port 0 is attached. Now total ports is 1 + Done + +In this case, identifier is "eth_pcap0,iface=eth0". +This identifier format is the same as "--vdev" format of DPDK applications. + +port detach +~~~~~~~~~~~ + +Detach a specific port. + +Before detaching a port, the port should be closed. +Also to remove a pci device completely from the system, first detach the port from testpmd. +Then the device should be moved under kernel management. +Finally the device can be removed using kernel pci hotplug functionality. +On the other hand, to remove a port created by a virtual device, above steps are not needed. + +port detach (port_id) + +For example, to detach a port 0. + +.. code-block:: console + + testpmd> port detach 0 + Detaching a port... + ... snip ... + Done + port start ~~~~~~~~~~ -- 1.9.1
Hi Tetsuya, On Mon, Feb 23, 2015 at 6:09 AM, Tetsuya Mukawa <mukawa@igel.co.jp> wrote: > These functions are used for attaching or detaching a port. [...] > > +static int > +rte_eal_vdev_init(const char *name, const char *args) > +{ > + struct rte_driver *driver; > + > + if (name == NULL) > + return -EINVAL; > + > + TAILQ_FOREACH(driver, &dev_driver_list, next) { > + if (driver->type != PMD_VDEV) > + continue; > + > + /* > + * search a driver prefix in virtual device name. > + * For example, if the driver is pcap PMD, driver->name > + * will be "eth_pcap", but "name" will be "eth_pcapN". > + * So use strncmp to compare. > + */ > + if (!strncmp(driver->name, name, strlen(driver->name))) { > + driver->init(name, args); > + break; Please return the value given by init: return driver->init(name, args); . > + } > + } > + > + if (driver == NULL) { > + RTE_LOG(WARNING, EAL, "no driver found for %s\n", name); --> should be : RTE_LOG(ERR . > + return -EINVAL; > + } > + return 0; > +} > + > int > rte_eal_dev_init(void) > { > @@ -79,23 +113,10 @@ rte_eal_dev_init(void) > if (devargs->type != RTE_DEVTYPE_VIRTUAL) > continue; > > - TAILQ_FOREACH(driver, &dev_driver_list, next) { > - if (driver->type != PMD_VDEV) > - continue; > - > - /* search a driver prefix in virtual device name */ > - if (!strncmp(driver->name, devargs->virtual.drv_name, > - strlen(driver->name))) { > - driver->init(devargs->virtual.drv_name, > - devargs->args); > - break; > - } > - } > - > - if (driver == NULL) { > + if (rte_eal_vdev_init(devargs->virtual.drv_name, > + devargs->args)) > rte_panic("no driver found for %s\n", > devargs->virtual.drv_name); instead of that: if (rte_eal_vdev_init(devargs->virtual.drv_name, devargs->args)) { RTE_LOG(ERR, "failed to initialize %s device\n", devargs->virtual.drv_name); return -1; } > - } > } > > /* Once the vdevs are initalized, start calling all the pdev drivers */ > @@ -107,3 +128,237 @@ rte_eal_dev_init(void) > } > return 0; > } > + > +/* So far, DPDK hotplug function only supports linux */ > +#ifdef RTE_LIBRTE_EAL_HOTPLUG > +static int > +rte_eal_vdev_uninit(const char *name) > +{ > + struct rte_driver *driver; > + > + if (name == NULL) > + return -EINVAL; > + > + TAILQ_FOREACH(driver, &dev_driver_list, next) { > + if (driver->type != PMD_VDEV) > + continue; > + > + /* > + * search a driver prefix in virtual device name. > + * For example, if the driver is pcap PMD, driver->name > + * will be "eth_pcap", but "name" will be "eth_pcapN". > + * So use strncmp to compare. > + */ > + if (!strncmp(driver->name, name, strlen(driver->name))) { > + driver->uninit(name); Please return the value given by uninit: return driver->uninit(name, args); > + break; > + } > + } > + > + if (driver == NULL) { > + RTE_LOG(WARNING, EAL, "no driver found for %s\n", name); > + return 1; As it's an error, the function should return a negative value ( i.e. -EINVAL). Please set the log level to ERR. > + } > + return 0; > +} > + [...] > +} > + > +/* attach the new virtual device, then store port_id of the device */ > +static int > +rte_eal_dev_attach_vdev(const char *vdevargs, uint8_t *port_id) > +{ > + char *name = NULL, *args = NULL; > + uint8_t new_port_id; > + struct rte_eth_dev devs[RTE_MAX_ETHPORTS]; > + int ret = -1; > + > + if ((vdevargs == NULL) || (port_id == NULL)) > + goto end; > + > + /* parse vdevargs, then retrieve device name and args */ > + if (rte_eal_parse_devargs_str(vdevargs, &name, &args)) > + goto end; > + > + /* save current port status */ > + if (rte_eth_dev_save(devs, sizeof(devs))) > + goto end; > + /* walk around dev_driver_list to find the driver of the device, > + * then invoke probe function o the driver. > + * TODO: > + * rte_eal_vdev_init() should return port_id, > + * And rte_eth_dev_save() and rte_eth_dev_get_changed_port() > + * should be removed. */ > + if (rte_eal_vdev_init(name, args)) > + goto end; > + /* get port_id enabled by above procedures */ > + if (rte_eth_dev_get_changed_port(devs, &new_port_id)) > + goto end; > + ret = 0; > + Please set port_id here, i.e. : *port_id = new_port_id; > +end: > + if (name) > + free(name); > + if (args) > + free(args); > + > + *port_id = new_port_id; and not here. > + if (ret < 0) > + RTE_LOG(ERR, EAL, "Driver, cannot attach the device\n"); > + return ret; > +} > + e\n"); > + return -1; > +} > + Regards, Maxime
On 2015/02/23 22:29, Maxime Leroy wrote: > Hi Tetsuya, > > On Mon, Feb 23, 2015 at 6:09 AM, Tetsuya Mukawa <mukawa@igel.co.jp> wrote: >> These functions are used for attaching or detaching a port. > [...] >> +static int >> +rte_eal_vdev_init(const char *name, const char *args) >> +{ >> + struct rte_driver *driver; >> + >> + if (name == NULL) >> + return -EINVAL; >> + >> + TAILQ_FOREACH(driver, &dev_driver_list, next) { >> + if (driver->type != PMD_VDEV) >> + continue; >> + >> + /* >> + * search a driver prefix in virtual device name. >> + * For example, if the driver is pcap PMD, driver->name >> + * will be "eth_pcap", but "name" will be "eth_pcapN". >> + * So use strncmp to compare. >> + */ >> + if (!strncmp(driver->name, name, strlen(driver->name))) { >> + driver->init(name, args); >> + break; > Please return the value given by init: return driver->init(name, args); . > >> + } >> + } >> + >> + if (driver == NULL) { >> + RTE_LOG(WARNING, EAL, "no driver found for %s\n", name); > --> should be : RTE_LOG(ERR . > > >> + return -EINVAL; >> + } >> + return 0; >> +} >> + >> int >> rte_eal_dev_init(void) >> { >> @@ -79,23 +113,10 @@ rte_eal_dev_init(void) >> if (devargs->type != RTE_DEVTYPE_VIRTUAL) >> continue; >> >> - TAILQ_FOREACH(driver, &dev_driver_list, next) { >> - if (driver->type != PMD_VDEV) >> - continue; >> - >> - /* search a driver prefix in virtual device name */ >> - if (!strncmp(driver->name, devargs->virtual.drv_name, >> - strlen(driver->name))) { >> - driver->init(devargs->virtual.drv_name, >> - devargs->args); >> - break; >> - } >> - } >> - >> - if (driver == NULL) { >> + if (rte_eal_vdev_init(devargs->virtual.drv_name, >> + devargs->args)) >> rte_panic("no driver found for %s\n", >> devargs->virtual.drv_name); > instead of that: > > if (rte_eal_vdev_init(devargs->virtual.drv_name, devargs->args)) { > RTE_LOG(ERR, "failed to initialize %s device\n", > devargs->virtual.drv_name); > return -1; > } > >> - } >> } >> >> /* Once the vdevs are initalized, start calling all the pdev drivers */ >> @@ -107,3 +128,237 @@ rte_eal_dev_init(void) >> } >> return 0; >> } >> + >> +/* So far, DPDK hotplug function only supports linux */ >> +#ifdef RTE_LIBRTE_EAL_HOTPLUG >> +static int >> +rte_eal_vdev_uninit(const char *name) >> +{ >> + struct rte_driver *driver; >> + >> + if (name == NULL) >> + return -EINVAL; >> + >> + TAILQ_FOREACH(driver, &dev_driver_list, next) { >> + if (driver->type != PMD_VDEV) >> + continue; >> + >> + /* >> + * search a driver prefix in virtual device name. >> + * For example, if the driver is pcap PMD, driver->name >> + * will be "eth_pcap", but "name" will be "eth_pcapN". >> + * So use strncmp to compare. >> + */ >> + if (!strncmp(driver->name, name, strlen(driver->name))) { >> + driver->uninit(name); > Please return the value given by uninit: return driver->uninit(name, args); > >> + break; >> + } >> + } >> + >> + if (driver == NULL) { >> + RTE_LOG(WARNING, EAL, "no driver found for %s\n", name); >> + return 1; > As it's an error, the function should return a negative value ( i.e. -EINVAL). > Please set the log level to ERR. > >> + } >> + return 0; >> +} >> + > [...] >> +} >> + >> +/* attach the new virtual device, then store port_id of the device */ >> +static int >> +rte_eal_dev_attach_vdev(const char *vdevargs, uint8_t *port_id) >> +{ >> + char *name = NULL, *args = NULL; >> + uint8_t new_port_id; >> + struct rte_eth_dev devs[RTE_MAX_ETHPORTS]; >> + int ret = -1; >> + >> + if ((vdevargs == NULL) || (port_id == NULL)) >> + goto end; >> + >> + /* parse vdevargs, then retrieve device name and args */ >> + if (rte_eal_parse_devargs_str(vdevargs, &name, &args)) >> + goto end; >> + >> + /* save current port status */ >> + if (rte_eth_dev_save(devs, sizeof(devs))) >> + goto end; >> + /* walk around dev_driver_list to find the driver of the device, >> + * then invoke probe function o the driver. >> + * TODO: >> + * rte_eal_vdev_init() should return port_id, >> + * And rte_eth_dev_save() and rte_eth_dev_get_changed_port() >> + * should be removed. */ >> + if (rte_eal_vdev_init(name, args)) >> + goto end; >> + /* get port_id enabled by above procedures */ >> + if (rte_eth_dev_get_changed_port(devs, &new_port_id)) >> + goto end; >> + ret = 0; >> + > Please set port_id here, i.e. : *port_id = new_port_id; > >> +end: >> + if (name) >> + free(name); >> + if (args) >> + free(args); >> + >> + *port_id = new_port_id; > and not here. > > >> + if (ret < 0) >> + RTE_LOG(ERR, EAL, "Driver, cannot attach the device\n"); >> + return ret; >> +} >> + > e\n"); >> + return -1; >> +} >> + Hi Maxime, I appreciate your comments. I've change like your comments, and send new one soon. Regards, Tetsuya > Regards, > > Maxime
This patch series adds a dynamic port hotplug framework to DPDK. With the patches, DPDK apps can attach or detach ports at runtime. The basic concept of the port hotplug is like followings. - DPDK apps must have responsibility to manage ports. DPDK apps only know which ports are attached or detached at the moment. The port hotplug framework is implemented to allow DPDK apps to manage ports. For example, when DPDK apps call port attach function, attached port number will be returned. Also, DPDK apps can detach port by port number. - Kernel support is needed for attaching or detaching physical device ports. To attach a new physical device port, the device will be recognized by userspace directly I/O framework in kernel at first. Then DPDK apps can call the port hotplug functions to attach ports. For detaching, steps are vice versa. - Before detach ports, ports must be stopped and closed. DPDK application must call rte_eth_dev_stop() and rte_eth_dev_close() before detaching ports. These function will call finalization codes of PMDs. But so far, no PMD frees all resources allocated by initialization. It means PMDs are needed to be fixed to support the port hotplug. 'RTE_PCI_DRV_DETACHABLE' is a new flag indicating a PMD supports detaching. Without this flag, detaching will be failed. - Mustn't affect legacy DPDK apps. No DPDK EAL behavior is changed, if the port hotplug functions are't called. So all legacy DPDK apps can still work without modifications. And a few limitations. - The port hotplug functions are not thread safe. DPDK apps should handle it. - Only support Linux and igb_uio so far. BSD and VFIO is not supported. I will send VFIO patches at least, but I don't have a plan to submit BSD patch so far. Here is port hotplug APIs. ------------------------------------------------------------------------------- /** * Attach a new device. * * @param devargs * A pointer to a strings array describing the new device * to be attached. The strings should be a pci address like * '0000:01:00.0' or virtual device name like 'eth_pcap0'. * @param port_id * A pointer to a port identifier actually attached. * @return * 0 on success and port_id is filled, negative on error */ int rte_eal_dev_attach(const char *devargs, uint8_t *port_id); /** * Detach a device. * * @param port_id * The port identifier of the device to detach. * @param addr * A pointer to a device name actually detached. * @return * 0 on success and devname is filled, negative on error */ int rte_eal_dev_detach(uint8_t port_id, char *devname); ------------------------------------------------------------------------------- This patch series are for DPDK EAL. To use port hotplug function by DPDK apps, each PMD should be fixed to support 'RTE_PCI_DRV_DETACHABLE' flag. Please check a patch for pcap PMD. Also, please check testpmd patch. It will show you how to fix your legacy applications to support port hotplug feature. PATCH v13 changes - Change log level when error occurs in rte_eal_vdev_init() and rte_eal_dev_init(). - Return value of driver init and uninit functions. - Replace rte_panic by RTE_LOG in rte_eal_dev_init() - Fix return value of rte_eal_vdev_uninit(). - Fix rte_eal_dev_attach_vdev to set port_id correctly. (Thanks to Maxime Leroy) PATCH v12 changes - Add missing symbol in version map. (Thanks to Iremonger, Bernard) PATCH v11 changes - Remove needless devargs handling codes. - Replace get_vdev_name() by rte_eal_parse_devargs_str(). - Replace rte_eal_vdev_find_and_init by rte_eal_vdev_init() - Replace rte_eal_vdev_find_and_uninit by rte_eal_vdev_uninit() - Fix rte_eal_dev_init() to use rte_eal_vdev_init(). - Remove needless patch. (Thanks to Maxime Leroy) PATCH v10 changes - Add comments. - Chagne order of version.map. - Fix comment of "rte_ethdev.h". (Thanks to Thomas Monjalon) - Add size parameter to rte_eth_dev_create_unique_device_name(). (Thanks to Iremonger, Bernard) PATCH v9 changes - Fix commit title. - Fix commit log. - Fix comments. - Define CONFIG_RTE_LIBRTE_EAL_HOTPLUG at the top of this patch series. - DEV_INVALID/VALID are removed. - DEV_DISCONNECTED is replaced by DEV_DETACHED. - DEV_CONNECTED is replaced by DEV_ATTACHED. - rte_eth_dev_allocate_new_port() is renamed to rte_eth_dev_find_free_port(). - rte_eth_dev_validate_port() is renamed to rte_eth_dev_is_valid_port(). - rte_eth_dev_is_valid_port() is changed not to handle log toggle. - eal_compare_pci_addr() is replaced by rte_eal_compare_pci_addr(). - rte_eth_dev_free() is replaced by rte_eth_dev_release_port(). - Add a function to create a unique device name. - Change parameter of pci_devuninit_t and rte_eth_dev_uninit. - Remove code that initiaize callback of ethdev from rte_eth_dev_uninit(). - Remove pci_unmap_device(). It will be implemented in later patch. - rte_eth_dev_check_detachable() is replaced by rte_eth_dev_is_detachable(). - strncpy() is replaced by strcpy(). - Implement pci_unmap_device() in this patch. - Remove "rte_dev_hotplug.h". - Remove needless "#ifdef". - Remove RTE_EAL_INVOKE_TYPE_PROBE/CLOSE. - RTE_ETH_DEV_PHYSICAL is replaced by RTE_ETH_DEV_PCI. - Use strcmp() instead of strncmp(). - Remove RTE_EAL_INVOKE_TYPE_PROBE/CLOSE. (Thanks to Thomas Monjalon) - Change definition of rte_dev_uninit_t. (Thanks to Thomas Monjalon and Maxime Leroy) - Add missing symbol in version map. (Thanks to Nail Horman) PATCH v8 changes - Fix Makefile and add version map file. - Add missing symbol in version map. - Fix pci_scan_one() to update sysfs values. (Thanks to Qiu, Michael and Iremonger, Bernard) - NONE_TRACE is replaced by NO_TRACE. - Fix typo. - Add size parameter to rte_eth_dev_save(). (Thanks to Iremonger, Bernard) PATCH v7 changes - Add a new section to programmer's guide. (Thanks to Iremonger, Bernard) - Fix port checking implementation of star_port(). - Fix typo of warning messages. - Add pt_driver checking to rte_eth_dev_check_detachable(). (Thanks to Qiu, Michael) PATCH v6 changes - Fix rte_eth_dev_uninit() to handle a return value of uninit function of PMD. To do this, below changes also be applied. - Fix a parameter of rte_eth_dev_free(). - Use rte_eth_dev structure as the paramter of rte_eth_dev_free(). PATCH v5 changes - Add runtime check passthrough driver type, like vfio-pci, igb_uio and uio_pci_generic. This was done by Qiu, Michael. Thanks a lot. - Change function names like below. - rte_eal_dev_find_and_invoke() to rte_eal_vdev_find_and_invoke(). - rte_eal_dev_invoke() to rte_eal_vdev_invoke(). - Add code to handle a return value of rte_eal_devargs_remove(). - Fix pci address format in rte_eal_dev_detach(). - Remove RTE_EAL_INVOKE_TYPE_UNKNOWN, because it's unused. - Change function definition of rte_eal_devargs_remove(). - Fix pci_unmap_device() to check pt_driver. - Fix return value of below functions. - rte_eth_dev_get_changed_port(). - rte_eth_dev_get_port_by_addr(). - Change paramters of rte_eth_dev_validate_port() to cleanup code. - Fix pci_scan_one to handle pt_driver correctly. (Thanks to Qiu, Michael for above suggestions) PATCH v4 changes - Merge patches to review easier. - Fix indent of 'if' statement. - Fix calculation method of eal_compare_pci_addr(). - Fix header file declaration. - Add header file to determine if hotplug can be enabled. (Thanks to Qiu, Michael) - Use braces with 'for' loop. - Add parameter checking. - Fix sanity check code - Fix comments of rte_eth_dev_type. - Change function names. (Thanks to Iremonger, Bernard) PATCH v3 changes: - Fix enum definition used in rte_ethdev.c. (Thanks to Zhang, Helin) PATCH v2 changes: - Replace rte_eal_dev_attach_pdev(), rte_eal_dev_detach_pdev, rte_eal_dev_attach_vdev() and rte_eal_dev_detach_vdev() to rte_eal_dev_attach() and rte_eal_dev_detach(). - Add parameter values checking. - Refashion a few functions. (Thanks to Iremonger, Bernard) PATCH v1 Changes: - Fix error checking code of librte_eth APIs. - Fix issue that port from pcap PMD cannot be detached correctly. - Fix issue that testpmd could hang after forwarding, if attaching and detaching is repeatedly. - Fix if-condition of rte_eth_dev_get_port_by_addr(). (Thanks to Mark Enright) RFC PATCH v2 Changes: - remove 'rte_eth_dev_validate_port()', and cleanup codes. Michael Qiu (2): eal_pci: Add flag to hold kernel driver type eal_pci: pci memory map work with driver type Tetsuya Mukawa (11): eal: Enable port Hotplug framework in Linux eal/pci,ethdev: Remove assumption that port will not be detached eal/pci: Consolidate pci address comparison APIs ethdev: Add rte_eth_dev_release_port to release specified port eal,ethdev: Add a function and function pointers to close ether device ethdev: Add functions that will be used by port hotplug functions eal/linux/pci: Add functions for unmapping igb_uio resources eal/pci: Add probe and close functions of pci driver ethdev: Add one dev_type parameter to rte_eth_dev_allocate eal/pci: Add rte_eal_dev_attach/detach() functions doc: Add port hotplug framework section to programmers guide app/test/virtual_pmd.c | 2 +- config/common_bsdapp | 6 + config/common_linuxapp | 5 + doc/guides/prog_guide/index.rst | 1 + doc/guides/prog_guide/port_hotplug_framework.rst | 110 ++++++ lib/librte_eal/bsdapp/eal/eal_pci.c | 29 +- lib/librte_eal/common/eal_common_dev.c | 284 +++++++++++++- lib/librte_eal/common/eal_common_devargs.c | 46 ++- lib/librte_eal/common/eal_common_pci.c | 100 ++++- lib/librte_eal/common/eal_private.h | 26 ++ lib/librte_eal/common/include/rte_dev.h | 33 ++ lib/librte_eal/common/include/rte_devargs.h | 28 ++ lib/librte_eal/common/include/rte_pci.h | 82 +++++ lib/librte_eal/linuxapp/eal/Makefile | 1 + lib/librte_eal/linuxapp/eal/eal_pci.c | 230 ++++++++++-- lib/librte_eal/linuxapp/eal/eal_pci_init.h | 7 + lib/librte_eal/linuxapp/eal/eal_pci_uio.c | 67 +++- lib/librte_eal/linuxapp/eal/rte_eal_version.map | 2 + lib/librte_ether/rte_ethdev.c | 451 ++++++++++++++++++----- lib/librte_ether/rte_ethdev.h | 149 +++++++- lib/librte_ether/rte_ether_version.map | 8 + lib/librte_pmd_af_packet/rte_eth_af_packet.c | 2 +- lib/librte_pmd_bond/rte_eth_bond_api.c | 2 +- lib/librte_pmd_pcap/rte_eth_pcap.c | 2 +- lib/librte_pmd_ring/rte_eth_ring.c | 2 +- lib/librte_pmd_xenvirt/rte_eth_xenvirt.c | 2 +- 26 files changed, 1489 insertions(+), 188 deletions(-) create mode 100644 doc/guides/prog_guide/port_hotplug_framework.rst -- 1.9.1
The patch adds CONFIG_RTE_LIBRTE_EAL_HOTPLUG in Linux and BSD configuration. So far, Hotplug functions only support linux. v9: - Move this patch at the top of this patch series. (Thanks to Thomas Monjalon) Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp> --- config/common_bsdapp | 6 ++++++ config/common_linuxapp | 5 +++++ 2 files changed, 11 insertions(+) diff --git a/config/common_bsdapp b/config/common_bsdapp index 56c5372..053caef 100644 --- a/config/common_bsdapp +++ b/config/common_bsdapp @@ -116,6 +116,12 @@ CONFIG_RTE_LIBRTE_EAL_BSDAPP=y CONFIG_RTE_LIBRTE_EAL_LINUXAPP=n # +# Compile Environment Abstraction Layer to support hotplug +# So far, Hotplug functions only support linux +# +CONFIG_RTE_LIBRTE_EAL_HOTPLUG=n + +# # Compile Environment Abstraction Layer to support Vmware TSC map # CONFIG_RTE_LIBRTE_EAL_VMWARE_TSC_MAP_SUPPORT=y diff --git a/config/common_linuxapp b/config/common_linuxapp index fc29425..7e84e5f 100644 --- a/config/common_linuxapp +++ b/config/common_linuxapp @@ -114,6 +114,11 @@ CONFIG_RTE_PCI_MAX_READ_REQUEST_SIZE=0 CONFIG_RTE_LIBRTE_EAL_LINUXAPP=y # +# Compile Environment Abstraction Layer to support hotplug +# +CONFIG_RTE_LIBRTE_EAL_HOTPLUG=y + +# # Compile Environment Abstraction Layer to support Vmware TSC map # CONFIG_RTE_LIBRTE_EAL_VMWARE_TSC_MAP_SUPPORT=y -- 1.9.1
From: Michael Qiu <michael.qiu@intel.com> Currently, dpdk has no ability to know which type of driver( vfio-pci/igb_uio/uio_pci_generic) the device used. It only can check whether vfio is enabled or not staticly. It really useful to have the flag, becasue different type need to handle differently in runtime. For example, pci memory map, pot hotplug, and so on. This patch add a flag field for pci device to solve above issue. Signed-off-by: Michael Qiu <michael.qiu@intel.com> Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp> --- lib/librte_eal/common/include/rte_pci.h | 8 +++++ lib/librte_eal/linuxapp/eal/eal_pci.c | 53 +++++++++++++++++++++++++++++++-- 2 files changed, 59 insertions(+), 2 deletions(-) diff --git a/lib/librte_eal/common/include/rte_pci.h b/lib/librte_eal/common/include/rte_pci.h index 4301c16..5e0ba00 100644 --- a/lib/librte_eal/common/include/rte_pci.h +++ b/lib/librte_eal/common/include/rte_pci.h @@ -142,6 +142,13 @@ struct rte_pci_addr { struct rte_devargs; +enum rte_pt_driver { + RTE_PT_UNKNOWN = 0, + RTE_PT_IGB_UIO = 1, + RTE_PT_VFIO = 2, + RTE_PT_UIO_GENERIC = 3, +}; + /** * A structure describing a PCI device. */ @@ -155,6 +162,7 @@ struct rte_pci_device { uint16_t max_vfs; /**< sriov enable if not zero */ int numa_node; /**< NUMA node connection */ struct rte_devargs *devargs; /**< Device user arguments */ + enum rte_pt_driver pt_driver; /**< Driver of passthrough */ }; /** Any PCI device identifier (vendor, device, ...) */ diff --git a/lib/librte_eal/linuxapp/eal/eal_pci.c b/lib/librte_eal/linuxapp/eal/eal_pci.c index a4fd5f5..4615756 100644 --- a/lib/librte_eal/linuxapp/eal/eal_pci.c +++ b/lib/librte_eal/linuxapp/eal/eal_pci.c @@ -97,6 +97,35 @@ error: return -1; } +static int +pci_get_kernel_driver_by_path(const char *filename, char *dri_name) +{ + int count; + char path[PATH_MAX]; + char *name; + + if (!filename || !dri_name) + return -1; + + count = readlink(filename, path, PATH_MAX); + if (count >= PATH_MAX) + return -1; + + /* For device does not have a driver */ + if (count < 0) + return 1; + + path[count] = '\0'; + + name = strrchr(path, '/'); + if (name) { + strncpy(dri_name, name + 1, strlen(name + 1) + 1); + return 0; + } + + return -1; +} + void * pci_find_max_end_va(void) { @@ -221,11 +250,12 @@ pci_scan_one(const char *dirname, uint16_t domain, uint8_t bus, char filename[PATH_MAX]; unsigned long tmp; struct rte_pci_device *dev; + char driver[PATH_MAX]; + int ret; dev = malloc(sizeof(*dev)); - if (dev == NULL) { + if (dev == NULL) return -1; - } memset(dev, 0, sizeof(*dev)); dev->addr.domain = domain; @@ -304,6 +334,25 @@ pci_scan_one(const char *dirname, uint16_t domain, uint8_t bus, return -1; } + /* parse driver */ + snprintf(filename, sizeof(filename), "%s/driver", dirname); + ret = pci_get_kernel_driver_by_path(filename, driver); + if (!ret) { + if (!strcmp(driver, "vfio-pci")) + dev->pt_driver = RTE_PT_VFIO; + else if (!strcmp(driver, "igb_uio")) + dev->pt_driver = RTE_PT_IGB_UIO; + else if (!strcmp(driver, "uio_pci_generic")) + dev->pt_driver = RTE_PT_UIO_GENERIC; + else + dev->pt_driver = RTE_PT_UNKNOWN; + } else if (ret < 0) { + RTE_LOG(ERR, EAL, "Fail to get kernel driver\n"); + free(dev); + return -1; + } else + dev->pt_driver = RTE_PT_UNKNOWN; + /* device is valid, add in list (sorted) */ if (TAILQ_EMPTY(&pci_device_list)) { TAILQ_INSERT_TAIL(&pci_device_list, dev, next); -- 1.9.1
From: Michael Qiu <michael.qiu@intel.com> With the driver type flag in struct rte_pci_dev, we do not need to always map uio devices with vfio related function when vfio enabled. Signed-off-by: Michael Qiu <michael.qiu@intel.com> Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp> --- lib/librte_eal/linuxapp/eal/eal_pci.c | 30 +++++++++++++++++------------- 1 file changed, 17 insertions(+), 13 deletions(-) diff --git a/lib/librte_eal/linuxapp/eal/eal_pci.c b/lib/librte_eal/linuxapp/eal/eal_pci.c index 4615756..3291c68 100644 --- a/lib/librte_eal/linuxapp/eal/eal_pci.c +++ b/lib/librte_eal/linuxapp/eal/eal_pci.c @@ -555,25 +555,29 @@ pci_config_space_set(struct rte_pci_device *dev) static int pci_map_device(struct rte_pci_device *dev) { - int ret, mapped = 0; + int ret = -1; /* try mapping the NIC resources using VFIO if it exists */ + switch (dev->pt_driver) { + case RTE_PT_VFIO: #ifdef VFIO_PRESENT - if (pci_vfio_is_enabled()) { - ret = pci_vfio_map_resource(dev); - if (ret == 0) - mapped = 1; - else if (ret < 0) - return ret; - } + if (pci_vfio_is_enabled()) + ret = pci_vfio_map_resource(dev); #endif - /* map resources for devices that use uio_pci_generic or igb_uio */ - if (!mapped) { + break; + case RTE_PT_IGB_UIO: + case RTE_PT_UIO_GENERIC: + /* map resources for devices that use uio */ ret = pci_uio_map_resource(dev); - if (ret != 0) - return ret; + break; + default: + RTE_LOG(DEBUG, EAL, " Not managed by known pt driver," + " skipped\n"); + ret = 1; + break; } - return 0; + + return ret; } /* -- 1.9.1
To remove assumption, do like followings. This patch adds "RTE_PCI_DRV_DETACHABLE" to drv_flags of rte_pci_driver structure. The flags indicate the driver can detach devices at runtime. Also, remove assumption that port will not be detached. To remove the assumption. - Add 'attached' member to rte_eth_dev structure. This member is used for indicating the port is attached, or not. DEV_ATTACHED indicates a port is attached. DEV_DETACHED indicates a port is detached. - Add rte_eth_dev_allocate_new_port(). This function is used for allocating new port. v9: - DEV_INVALID/VALID are removed. - DEV_DISCONNECTED is replaced by DEV_DETACHED. - DEV_CONNECTED is replaced by DEV_ATTACHED. - rte_eth_dev_allocate_new_port() is renamed to rte_eth_dev_find_free_port(). - rte_eth_dev_validate_port() is renamed to rte_eth_dev_is_valid_port(). - rte_eth_dev_is_valid_port() is changed not to handle log toggle. - Fix commit log to describe DEV_ATACHED and DEV_DETACHED. (Thanks to Thomas Monjalon) v8: - NONE_TRACE is changed to NO_TRACE. (Thanks to Iremonger, Bernard) v5: - Change parameters of rte_eth_dev_validate_port() to cleanup code. v4: - Use braces with 'for' loop. - Fix indent of 'if' statement. Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp> --- lib/librte_eal/common/include/rte_pci.h | 2 + lib/librte_ether/rte_ethdev.c | 248 ++++++++++++++++++++------------ lib/librte_ether/rte_ethdev.h | 5 + 3 files changed, 164 insertions(+), 91 deletions(-) diff --git a/lib/librte_eal/common/include/rte_pci.h b/lib/librte_eal/common/include/rte_pci.h index 5e0ba00..ffd13d9 100644 --- a/lib/librte_eal/common/include/rte_pci.h +++ b/lib/librte_eal/common/include/rte_pci.h @@ -210,6 +210,8 @@ struct rte_pci_driver { #define RTE_PCI_DRV_FORCE_UNBIND 0x0004 /** Device driver supports link state interrupt */ #define RTE_PCI_DRV_INTR_LSC 0x0008 +/** Device driver supports detaching capability */ +#define RTE_PCI_DRV_DETACHABLE 0x0010 /**< Internal use only - Macro used by pci addr parsing functions **/ #define GET_PCIADDR_FIELD(in, fd, lim, dlm) \ diff --git a/lib/librte_ether/rte_ethdev.c b/lib/librte_ether/rte_ethdev.c index ecbe93c..b702039 100644 --- a/lib/librte_ether/rte_ethdev.c +++ b/lib/librte_ether/rte_ethdev.c @@ -175,6 +175,11 @@ enum { STAT_QMAP_RX }; +enum { + DEV_DETACHED = 0, + DEV_ATTACHED +}; + static inline void rte_eth_dev_data_alloc(void) { @@ -201,19 +206,34 @@ rte_eth_dev_allocated(const char *name) { unsigned i; - for (i = 0; i < nb_ports; i++) { - if (strcmp(rte_eth_devices[i].data->name, name) == 0) + for (i = 0; i < RTE_MAX_ETHPORTS; i++) { + if ((rte_eth_devices[i].attached == DEV_ATTACHED) && + strcmp(rte_eth_devices[i].data->name, name) == 0) return &rte_eth_devices[i]; } return NULL; } +static uint8_t +rte_eth_dev_find_free_port(void) +{ + unsigned i; + + for (i = 0; i < RTE_MAX_ETHPORTS; i++) { + if (rte_eth_devices[i].attached == DEV_DETACHED) + return i; + } + return RTE_MAX_ETHPORTS; +} + struct rte_eth_dev * rte_eth_dev_allocate(const char *name) { + uint8_t port_id; struct rte_eth_dev *eth_dev; - if (nb_ports == RTE_MAX_ETHPORTS) { + port_id = rte_eth_dev_find_free_port(); + if (port_id == RTE_MAX_ETHPORTS) { PMD_DEBUG_TRACE("Reached maximum number of Ethernet ports\n"); return NULL; } @@ -226,10 +246,12 @@ rte_eth_dev_allocate(const char *name) return NULL; } - eth_dev = &rte_eth_devices[nb_ports]; - eth_dev->data = &rte_eth_dev_data[nb_ports]; + eth_dev = &rte_eth_devices[port_id]; + eth_dev->data = &rte_eth_dev_data[port_id]; snprintf(eth_dev->data->name, sizeof(eth_dev->data->name), "%s", name); - eth_dev->data->port_id = nb_ports++; + eth_dev->data->port_id = port_id; + eth_dev->attached = DEV_ATTACHED; + nb_ports++; return eth_dev; } @@ -283,6 +305,7 @@ rte_eth_dev_init(struct rte_pci_driver *pci_drv, (unsigned) pci_dev->id.device_id); if (rte_eal_process_type() == RTE_PROC_PRIMARY) rte_free(eth_dev->data->dev_private); + eth_dev->attached = DEV_DETACHED; nb_ports--; return diag; } @@ -308,10 +331,20 @@ rte_eth_driver_register(struct eth_driver *eth_drv) rte_eal_pci_register(ð_drv->pci_drv); } +static int +rte_eth_dev_is_valid_port(uint8_t port_id) +{ + if (port_id >= RTE_MAX_ETHPORTS || + rte_eth_devices[port_id].attached != DEV_ATTACHED) + return 0; + else + return 1; +} + int rte_eth_dev_socket_id(uint8_t port_id) { - if (port_id >= nb_ports) + if (!rte_eth_dev_is_valid_port(port_id)) return -1; return rte_eth_devices[port_id].pci_dev->numa_node; } @@ -395,7 +428,7 @@ rte_eth_dev_rx_queue_start(uint8_t port_id, uint16_t rx_queue_id) * in a multi-process setup*/ PROC_PRIMARY_OR_ERR_RET(-E_RTE_SECONDARY); - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return -EINVAL; } @@ -421,7 +454,7 @@ rte_eth_dev_rx_queue_stop(uint8_t port_id, uint16_t rx_queue_id) * in a multi-process setup*/ PROC_PRIMARY_OR_ERR_RET(-E_RTE_SECONDARY); - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return -EINVAL; } @@ -447,7 +480,7 @@ rte_eth_dev_tx_queue_start(uint8_t port_id, uint16_t tx_queue_id) * in a multi-process setup*/ PROC_PRIMARY_OR_ERR_RET(-E_RTE_SECONDARY); - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return -EINVAL; } @@ -473,7 +506,7 @@ rte_eth_dev_tx_queue_stop(uint8_t port_id, uint16_t tx_queue_id) * in a multi-process setup*/ PROC_PRIMARY_OR_ERR_RET(-E_RTE_SECONDARY); - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return -EINVAL; } @@ -755,10 +788,11 @@ rte_eth_dev_configure(uint8_t port_id, uint16_t nb_rx_q, uint16_t nb_tx_q, * in a multi-process setup*/ PROC_PRIMARY_OR_ERR_RET(-E_RTE_SECONDARY); - if (port_id >= nb_ports || port_id >= RTE_MAX_ETHPORTS) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-EINVAL); } + dev = &rte_eth_devices[port_id]; FUNC_PTR_OR_ERR_RET(*dev->dev_ops->dev_infos_get, -ENOTSUP); @@ -940,10 +974,11 @@ rte_eth_dev_start(uint8_t port_id) * in a multi-process setup*/ PROC_PRIMARY_OR_ERR_RET(-E_RTE_SECONDARY); - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%" PRIu8 "\n", port_id); + if (!rte_eth_dev_is_valid_port(port_id)) { + PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-EINVAL); } + dev = &rte_eth_devices[port_id]; FUNC_PTR_OR_ERR_RET(*dev->dev_ops->dev_start, -ENOTSUP); @@ -979,10 +1014,11 @@ rte_eth_dev_stop(uint8_t port_id) * in a multi-process setup*/ PROC_PRIMARY_OR_RET(); - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%" PRIu8 "\n", port_id); + if (!rte_eth_dev_is_valid_port(port_id)) { + PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return; } + dev = &rte_eth_devices[port_id]; FUNC_PTR_OR_RET(*dev->dev_ops->dev_stop); @@ -1007,10 +1043,11 @@ rte_eth_dev_set_link_up(uint8_t port_id) * in a multi-process setup*/ PROC_PRIMARY_OR_ERR_RET(-E_RTE_SECONDARY); - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return -EINVAL; } + dev = &rte_eth_devices[port_id]; FUNC_PTR_OR_ERR_RET(*dev->dev_ops->dev_set_link_up, -ENOTSUP); @@ -1026,10 +1063,11 @@ rte_eth_dev_set_link_down(uint8_t port_id) * in a multi-process setup*/ PROC_PRIMARY_OR_ERR_RET(-E_RTE_SECONDARY); - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return -EINVAL; } + dev = &rte_eth_devices[port_id]; FUNC_PTR_OR_ERR_RET(*dev->dev_ops->dev_set_link_down, -ENOTSUP); @@ -1045,7 +1083,7 @@ rte_eth_dev_close(uint8_t port_id) * in a multi-process setup*/ PROC_PRIMARY_OR_RET(); - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return; } @@ -1073,10 +1111,11 @@ rte_eth_rx_queue_setup(uint8_t port_id, uint16_t rx_queue_id, * in a multi-process setup*/ PROC_PRIMARY_OR_ERR_RET(-E_RTE_SECONDARY); - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-EINVAL); } + dev = &rte_eth_devices[port_id]; if (rx_queue_id >= dev->data->nb_rx_queues) { PMD_DEBUG_TRACE("Invalid RX queue_id=%d\n", rx_queue_id); @@ -1146,10 +1185,11 @@ rte_eth_tx_queue_setup(uint8_t port_id, uint16_t tx_queue_id, * in a multi-process setup*/ PROC_PRIMARY_OR_ERR_RET(-E_RTE_SECONDARY); - if (port_id >= RTE_MAX_ETHPORTS || port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-EINVAL); } + dev = &rte_eth_devices[port_id]; if (tx_queue_id >= dev->data->nb_tx_queues) { PMD_DEBUG_TRACE("Invalid TX queue_id=%d\n", tx_queue_id); @@ -1179,10 +1219,11 @@ rte_eth_promiscuous_enable(uint8_t port_id) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return; } + dev = &rte_eth_devices[port_id]; FUNC_PTR_OR_RET(*dev->dev_ops->promiscuous_enable); @@ -1195,10 +1236,11 @@ rte_eth_promiscuous_disable(uint8_t port_id) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return; } + dev = &rte_eth_devices[port_id]; FUNC_PTR_OR_RET(*dev->dev_ops->promiscuous_disable); @@ -1211,7 +1253,7 @@ rte_eth_promiscuous_get(uint8_t port_id) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return -1; } @@ -1225,10 +1267,11 @@ rte_eth_allmulticast_enable(uint8_t port_id) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return; } + dev = &rte_eth_devices[port_id]; FUNC_PTR_OR_RET(*dev->dev_ops->allmulticast_enable); @@ -1241,10 +1284,11 @@ rte_eth_allmulticast_disable(uint8_t port_id) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return; } + dev = &rte_eth_devices[port_id]; FUNC_PTR_OR_RET(*dev->dev_ops->allmulticast_disable); @@ -1257,7 +1301,7 @@ rte_eth_allmulticast_get(uint8_t port_id) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return -1; } @@ -1285,10 +1329,11 @@ rte_eth_link_get(uint8_t port_id, struct rte_eth_link *eth_link) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return; } + dev = &rte_eth_devices[port_id]; if (dev->data->dev_conf.intr_conf.lsc != 0) @@ -1305,10 +1350,11 @@ rte_eth_link_get_nowait(uint8_t port_id, struct rte_eth_link *eth_link) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return; } + dev = &rte_eth_devices[port_id]; if (dev->data->dev_conf.intr_conf.lsc != 0) @@ -1325,10 +1371,11 @@ rte_eth_stats_get(uint8_t port_id, struct rte_eth_stats *stats) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-ENODEV); } + dev = &rte_eth_devices[port_id]; memset(stats, 0, sizeof(*stats)); @@ -1343,10 +1390,11 @@ rte_eth_stats_reset(uint8_t port_id) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return; } + dev = &rte_eth_devices[port_id]; FUNC_PTR_OR_RET(*dev->dev_ops->stats_reset); @@ -1364,10 +1412,11 @@ rte_eth_xstats_get(uint8_t port_id, struct rte_eth_xstats *xstats, uint64_t val; char *stats_ptr; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return -1; } + dev = &rte_eth_devices[port_id]; /* implemented by the driver */ @@ -1433,10 +1482,11 @@ rte_eth_xstats_reset(uint8_t port_id) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return; } + dev = &rte_eth_devices[port_id]; /* implemented by the driver */ @@ -1455,10 +1505,11 @@ set_queue_stats_mapping(uint8_t port_id, uint16_t queue_id, uint8_t stat_idx, { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return -ENODEV; } + dev = &rte_eth_devices[port_id]; FUNC_PTR_OR_ERR_RET(*dev->dev_ops->queue_stats_mapping_set, -ENOTSUP); @@ -1490,10 +1541,11 @@ rte_eth_dev_info_get(uint8_t port_id, struct rte_eth_dev_info *dev_info) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return; } + dev = &rte_eth_devices[port_id]; memset(dev_info, 0, sizeof(struct rte_eth_dev_info)); @@ -1510,10 +1562,11 @@ rte_eth_macaddr_get(uint8_t port_id, struct ether_addr *mac_addr) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return; } + dev = &rte_eth_devices[port_id]; ether_addr_copy(&dev->data->mac_addrs[0], mac_addr); } @@ -1524,7 +1577,7 @@ rte_eth_dev_get_mtu(uint8_t port_id, uint16_t *mtu) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-ENODEV); } @@ -1540,7 +1593,7 @@ rte_eth_dev_set_mtu(uint8_t port_id, uint16_t mtu) int ret; struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-ENODEV); } @@ -1560,10 +1613,11 @@ rte_eth_dev_vlan_filter(uint8_t port_id, uint16_t vlan_id, int on) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-ENODEV); } + dev = &rte_eth_devices[port_id]; if (! (dev->data->dev_conf.rxmode.hw_vlan_filter)) { PMD_DEBUG_TRACE("port %d: vlan-filtering disabled\n", port_id); @@ -1585,7 +1639,7 @@ rte_eth_dev_set_vlan_strip_on_queue(uint8_t port_id, uint16_t rx_queue_id, int o { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-ENODEV); } @@ -1607,7 +1661,7 @@ rte_eth_dev_set_vlan_ether_type(uint8_t port_id, uint16_t tpid) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-ENODEV); } @@ -1627,7 +1681,7 @@ rte_eth_dev_set_vlan_offload(uint8_t port_id, int offload_mask) int mask = 0; int cur, org = 0; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-ENODEV); } @@ -1672,7 +1726,7 @@ rte_eth_dev_get_vlan_offload(uint8_t port_id) struct rte_eth_dev *dev; int ret = 0; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-ENODEV); } @@ -1696,10 +1750,11 @@ rte_eth_dev_set_vlan_pvid(uint8_t port_id, uint16_t pvid, int on) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-ENODEV); } + dev = &rte_eth_devices[port_id]; FUNC_PTR_OR_ERR_RET(*dev->dev_ops->vlan_pvid_set, -ENOTSUP); (*dev->dev_ops->vlan_pvid_set)(dev, pvid, on); @@ -1714,7 +1769,7 @@ rte_eth_dev_fdir_add_signature_filter(uint8_t port_id, { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-ENODEV); } @@ -1748,7 +1803,7 @@ rte_eth_dev_fdir_update_signature_filter(uint8_t port_id, { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-ENODEV); } @@ -1782,7 +1837,7 @@ rte_eth_dev_fdir_remove_signature_filter(uint8_t port_id, { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-ENODEV); } @@ -1813,7 +1868,7 @@ rte_eth_dev_fdir_get_infos(uint8_t port_id, struct rte_eth_fdir *fdir) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-ENODEV); } @@ -1838,7 +1893,7 @@ rte_eth_dev_fdir_add_perfect_filter(uint8_t port_id, { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-ENODEV); } @@ -1878,7 +1933,7 @@ rte_eth_dev_fdir_update_perfect_filter(uint8_t port_id, { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-ENODEV); } @@ -1916,7 +1971,7 @@ rte_eth_dev_fdir_remove_perfect_filter(uint8_t port_id, { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-ENODEV); } @@ -1952,7 +2007,7 @@ rte_eth_dev_fdir_set_masks(uint8_t port_id, struct rte_fdir_masks *fdir_mask) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-ENODEV); } @@ -1972,7 +2027,7 @@ rte_eth_dev_flow_ctrl_get(uint8_t port_id, struct rte_eth_fc_conf *fc_conf) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-ENODEV); } @@ -1988,7 +2043,7 @@ rte_eth_dev_flow_ctrl_set(uint8_t port_id, struct rte_eth_fc_conf *fc_conf) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-ENODEV); } @@ -2008,7 +2063,7 @@ rte_eth_dev_priority_flow_ctrl_set(uint8_t port_id, struct rte_eth_pfc_conf *pfc { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-ENODEV); } @@ -2087,7 +2142,7 @@ rte_eth_dev_rss_reta_update(uint8_t port_id, struct rte_eth_dev *dev; int ret; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return -ENODEV; } @@ -2138,10 +2193,11 @@ rte_eth_dev_rss_hash_update(uint8_t port_id, struct rte_eth_rss_conf *rss_conf) struct rte_eth_dev *dev; uint16_t rss_hash_protos; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-ENODEV); } + rss_hash_protos = rss_conf->rss_hf; if ((rss_hash_protos != 0) && ((rss_hash_protos & ETH_RSS_PROTO_MASK) == 0)) { @@ -2160,10 +2216,11 @@ rte_eth_dev_rss_hash_conf_get(uint8_t port_id, { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-ENODEV); } + dev = &rte_eth_devices[port_id]; FUNC_PTR_OR_ERR_RET(*dev->dev_ops->rss_hash_conf_get, -ENOTSUP); return (*dev->dev_ops->rss_hash_conf_get)(dev, rss_conf); @@ -2175,7 +2232,7 @@ rte_eth_dev_udp_tunnel_add(uint8_t port_id, { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return -ENODEV; } @@ -2201,10 +2258,11 @@ rte_eth_dev_udp_tunnel_delete(uint8_t port_id, { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return -ENODEV; } + dev = &rte_eth_devices[port_id]; if (udp_tunnel == NULL) { @@ -2226,7 +2284,7 @@ rte_eth_led_on(uint8_t port_id) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-ENODEV); } @@ -2241,7 +2299,7 @@ rte_eth_led_off(uint8_t port_id) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-ENODEV); } @@ -2281,10 +2339,11 @@ rte_eth_dev_mac_addr_add(uint8_t port_id, struct ether_addr *addr, int index; uint64_t pool_mask; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-ENODEV); } + dev = &rte_eth_devices[port_id]; FUNC_PTR_OR_ERR_RET(*dev->dev_ops->mac_addr_add, -ENOTSUP); @@ -2332,10 +2391,11 @@ rte_eth_dev_mac_addr_remove(uint8_t port_id, struct ether_addr *addr) struct rte_eth_dev *dev; int index; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-ENODEV); } + dev = &rte_eth_devices[port_id]; FUNC_PTR_OR_ERR_RET(*dev->dev_ops->mac_addr_remove, -ENOTSUP); @@ -2366,7 +2426,7 @@ rte_eth_dev_set_vf_rxmode(uint8_t port_id, uint16_t vf, struct rte_eth_dev *dev; struct rte_eth_dev_info dev_info; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("set VF RX mode:Invalid port_id=%d\n", port_id); return (-ENODEV); @@ -2421,7 +2481,7 @@ rte_eth_dev_uc_hash_table_set(uint8_t port_id, struct ether_addr *addr, int ret; struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("unicast hash setting:Invalid port_id=%d\n", port_id); return (-ENODEV); @@ -2474,7 +2534,7 @@ rte_eth_dev_uc_all_hash_table_set(uint8_t port_id, uint8_t on) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("unicast hash setting:Invalid port_id=%d\n", port_id); return (-ENODEV); @@ -2493,7 +2553,7 @@ rte_eth_dev_set_vf_rx(uint8_t port_id,uint16_t vf, uint8_t on) struct rte_eth_dev *dev; struct rte_eth_dev_info dev_info; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-ENODEV); } @@ -2519,7 +2579,7 @@ rte_eth_dev_set_vf_tx(uint8_t port_id,uint16_t vf, uint8_t on) struct rte_eth_dev *dev; struct rte_eth_dev_info dev_info; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("set pool tx:Invalid port_id=%d\n", port_id); return (-ENODEV); } @@ -2544,7 +2604,7 @@ rte_eth_dev_set_vf_vlan_filter(uint8_t port_id, uint16_t vlan_id, { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("VF VLAN filter:invalid port id=%d\n", port_id); return (-ENODEV); @@ -2575,7 +2635,7 @@ int rte_eth_set_queue_rate_limit(uint8_t port_id, uint16_t queue_idx, struct rte_eth_dev_info dev_info; struct rte_eth_link link; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("set queue rate limit:invalid port id=%d\n", port_id); return -ENODEV; @@ -2612,7 +2672,7 @@ int rte_eth_set_vf_rate_limit(uint8_t port_id, uint16_t vf, uint16_t tx_rate, if (q_msk == 0) return 0; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("set VF rate limit:invalid port id=%d\n", port_id); return -ENODEV; @@ -2646,7 +2706,7 @@ rte_eth_mirror_rule_set(uint8_t port_id, { struct rte_eth_dev *dev = &rte_eth_devices[port_id]; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-ENODEV); } @@ -2687,7 +2747,7 @@ rte_eth_mirror_rule_reset(uint8_t port_id, uint8_t rule_id) { struct rte_eth_dev *dev = &rte_eth_devices[port_id]; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-ENODEV); } @@ -2712,10 +2772,11 @@ rte_eth_rx_burst(uint8_t port_id, uint16_t queue_id, { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return 0; } + dev = &rte_eth_devices[port_id]; FUNC_PTR_OR_ERR_RET(*dev->rx_pkt_burst, 0); if (queue_id >= dev->data->nb_rx_queues) { @@ -2732,10 +2793,11 @@ rte_eth_tx_burst(uint8_t port_id, uint16_t queue_id, { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return 0; } + dev = &rte_eth_devices[port_id]; FUNC_PTR_OR_ERR_RET(*dev->tx_pkt_burst, 0); @@ -2752,10 +2814,11 @@ rte_eth_rx_queue_count(uint8_t port_id, uint16_t queue_id) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return 0; } + dev = &rte_eth_devices[port_id]; FUNC_PTR_OR_ERR_RET(*dev->dev_ops->rx_queue_count, 0); return (*dev->dev_ops->rx_queue_count)(dev, queue_id); @@ -2766,10 +2829,11 @@ rte_eth_rx_descriptor_done(uint8_t port_id, uint16_t queue_id, uint16_t offset) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-ENODEV); } + dev = &rte_eth_devices[port_id]; FUNC_PTR_OR_ERR_RET(*dev->dev_ops->rx_descriptor_done, -ENOTSUP); return (*dev->dev_ops->rx_descriptor_done)( \ @@ -2787,7 +2851,8 @@ rte_eth_dev_callback_register(uint8_t port_id, if (!cb_fn) return (-EINVAL); - if (port_id >= nb_ports) { + + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-EINVAL); } @@ -2827,7 +2892,8 @@ rte_eth_dev_callback_unregister(uint8_t port_id, if (!cb_fn) return (-EINVAL); - if (port_id >= nb_ports) { + + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-EINVAL); } @@ -2887,7 +2953,7 @@ int rte_eth_dev_bypass_init(uint8_t port_id) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-ENODEV); } @@ -2907,7 +2973,7 @@ rte_eth_dev_bypass_state_show(uint8_t port_id, uint32_t *state) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-ENODEV); } @@ -2926,7 +2992,7 @@ rte_eth_dev_bypass_state_set(uint8_t port_id, uint32_t *new_state) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-ENODEV); } @@ -2946,7 +3012,7 @@ rte_eth_dev_bypass_event_show(uint8_t port_id, uint32_t event, uint32_t *state) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-ENODEV); } @@ -2966,7 +3032,7 @@ rte_eth_dev_bypass_event_store(uint8_t port_id, uint32_t event, uint32_t state) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-ENODEV); } @@ -2986,7 +3052,7 @@ rte_eth_dev_wd_timeout_store(uint8_t port_id, uint32_t timeout) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-ENODEV); } @@ -3006,7 +3072,7 @@ rte_eth_dev_bypass_ver_show(uint8_t port_id, uint32_t *ver) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-ENODEV); } @@ -3026,7 +3092,7 @@ rte_eth_dev_bypass_wd_timeout_show(uint8_t port_id, uint32_t *wd_timeout) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-ENODEV); } @@ -3046,7 +3112,7 @@ rte_eth_dev_bypass_wd_reset(uint8_t port_id) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-ENODEV); } @@ -3067,7 +3133,7 @@ rte_eth_dev_filter_supported(uint8_t port_id, enum rte_filter_type filter_type) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return -ENODEV; } @@ -3084,7 +3150,7 @@ rte_eth_dev_filter_ctrl(uint8_t port_id, enum rte_filter_type filter_type, { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return -ENODEV; } diff --git a/lib/librte_ether/rte_ethdev.h b/lib/librte_ether/rte_ethdev.h index cce1a53..110ddba 100644 --- a/lib/librte_ether/rte_ethdev.h +++ b/lib/librte_ether/rte_ethdev.h @@ -1451,6 +1451,7 @@ struct rte_eth_dev { * received packets before passing them to the driver for transmission. */ struct rte_eth_rxtx_callback **pre_tx_burst_cbs; + uint8_t attached; /**< Flag indicating the port is attached */ }; struct rte_eth_dev_sriov { @@ -1516,6 +1517,10 @@ extern struct rte_eth_dev rte_eth_devices[]; * initialized by the [matching] Ethernet driver during the PCI probing phase. * All devices whose port identifier is in the range * [0, rte_eth_dev_count() - 1] can be operated on by network applications. + * immediately after invoking rte_eal_init(). + * If the application unplugs a port using hotplug function, The enabled port + * numbers may be noncontiguous. In the case, the applications need to manage + * enabled port by themselves. * * @return * - The total number of usable Ethernet devices. -- 1.9.1
This patch replaces pci_addr_comparison() and memcmp() of pci addresses by rte_eal_compare_pci_addr(). To compare PCI addresses, rte_eal_compare_pci_addr() doesn't use memcmp(). This is because sizeof(struct rte_pci_addr) returns 6, but actually this structure is like below. struct rte_pci_addr { uint16_t domain; /**< Device domain */ uint8_t bus; /**< Device bus */ uint8_t devid; /**< Device ID */ uint8_t function; /**< Device function. */ }; If the structure is dynamically allocated in a function without bzero, last 1 byte may have value. As a result, memcmp may not work. To avoid such a case, rte_eal_compare_pci_addr() compare following values. dev_addr = (addr->domain << 24) | (addr->bus << 16) | (addr->devid << 8) | addr->function; v9: - eal_compare_pci_addr() is replaced by rte_eal_compare_pci_addr(). - Fix commit log. (Thanks to Thomas Monjalon) v8: - Fix pci_scan_one() to update sysfs values. (Thanks to Qiu, Michael and Iremonger, Bernard) v5: - Fix pci_scan_one to handle pt_driver correctly. v4: - Fix calculation method of eal_compare_pci_addr(). - Add parameter checking. Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp> --- lib/librte_eal/bsdapp/eal/eal_pci.c | 29 ++++++++++++-------------- lib/librte_eal/common/eal_common_pci.c | 2 +- lib/librte_eal/common/include/rte_pci.h | 34 +++++++++++++++++++++++++++++++ lib/librte_eal/linuxapp/eal/eal_pci.c | 30 +++++++++++++-------------- lib/librte_eal/linuxapp/eal/eal_pci_uio.c | 2 +- 5 files changed, 63 insertions(+), 34 deletions(-) diff --git a/lib/librte_eal/bsdapp/eal/eal_pci.c b/lib/librte_eal/bsdapp/eal/eal_pci.c index 74ecce7..9193f80 100644 --- a/lib/librte_eal/bsdapp/eal/eal_pci.c +++ b/lib/librte_eal/bsdapp/eal/eal_pci.c @@ -270,20 +270,6 @@ pci_uio_map_resource(struct rte_pci_device *dev) return (0); } -/* Compare two PCI device addresses. */ -static int -pci_addr_comparison(struct rte_pci_addr *addr, struct rte_pci_addr *addr2) -{ - uint64_t dev_addr = (addr->domain << 24) + (addr->bus << 16) + (addr->devid << 8) + addr->function; - uint64_t dev_addr2 = (addr2->domain << 24) + (addr2->bus << 16) + (addr2->devid << 8) + addr2->function; - - if (dev_addr > dev_addr2) - return 1; - else - return 0; -} - - /* Scan one pci sysfs entry, and fill the devices list from it. */ static int pci_scan_one(int dev_pci_fd, struct pci_conf *conf) @@ -356,13 +342,24 @@ pci_scan_one(int dev_pci_fd, struct pci_conf *conf) } else { struct rte_pci_device *dev2 = NULL; + int ret; TAILQ_FOREACH(dev2, &pci_device_list, next) { - if (pci_addr_comparison(&dev->addr, &dev2->addr)) + ret = rte_eal_compare_pci_addr(&dev->addr, &dev2->addr); + if (ret > 0) continue; - else { + else if (ret < 0) { TAILQ_INSERT_BEFORE(dev2, dev, next); return 0; + } else { /* already registered */ + /* update pt_driver */ + dev2->pt_driver = dev->pt_driver; + dev2->max_vfs = dev->max_vfs; + memmove(dev2->mem_resource, + dev->mem_resource, + sizeof(dev->mem_resource)); + free(dev); + return 0; } } TAILQ_INSERT_TAIL(&pci_device_list, dev, next); diff --git a/lib/librte_eal/common/eal_common_pci.c b/lib/librte_eal/common/eal_common_pci.c index f3c7f71..bf2793f 100644 --- a/lib/librte_eal/common/eal_common_pci.c +++ b/lib/librte_eal/common/eal_common_pci.c @@ -93,7 +93,7 @@ static struct rte_devargs *pci_devargs_lookup(struct rte_pci_device *dev) if (devargs->type != RTE_DEVTYPE_BLACKLISTED_PCI && devargs->type != RTE_DEVTYPE_WHITELISTED_PCI) continue; - if (!memcmp(&dev->addr, &devargs->pci.addr, sizeof(dev->addr))) + if (!rte_eal_compare_pci_addr(&dev->addr, &devargs->pci.addr)) return devargs; } return NULL; diff --git a/lib/librte_eal/common/include/rte_pci.h b/lib/librte_eal/common/include/rte_pci.h index ffd13d9..6814e91 100644 --- a/lib/librte_eal/common/include/rte_pci.h +++ b/lib/librte_eal/common/include/rte_pci.h @@ -272,6 +272,40 @@ eal_parse_pci_DomBDF(const char *input, struct rte_pci_addr *dev_addr) } #undef GET_PCIADDR_FIELD +/* Compare two PCI device addresses. */ +/** + * Utility function to compare two PCI device addresses. + * + * @param addr + * The PCI Bus-Device-Function address to compare + * @param addr2 + * The PCI Bus-Device-Function address to compare + * @return + * 0 on equal PCI address. + * Positive on addr is greater than addr2. + * Negative on addr is less than addr2, or error. + */ +static inline int +rte_eal_compare_pci_addr(struct rte_pci_addr *addr, struct rte_pci_addr *addr2) +{ + uint64_t dev_addr, dev_addr2; + + if ((addr == NULL) || (addr2 == NULL)) + return -1; + + dev_addr = (addr->domain << 24) | (addr->bus << 16) | + (addr->devid << 8) | addr->function; + dev_addr2 = (addr2->domain << 24) | (addr2->bus << 16) | + (addr2->devid << 8) | addr2->function; + + if (dev_addr > dev_addr2) + return 1; + else if (dev_addr < dev_addr2) + return -1; + else + return 0; +} + /** * Probe the PCI bus for registered drivers. * diff --git a/lib/librte_eal/linuxapp/eal/eal_pci.c b/lib/librte_eal/linuxapp/eal/eal_pci.c index 3291c68..06bfc1a 100644 --- a/lib/librte_eal/linuxapp/eal/eal_pci.c +++ b/lib/librte_eal/linuxapp/eal/eal_pci.c @@ -41,6 +41,7 @@ #include <rte_eal_memconfig.h> #include <rte_malloc.h> #include <rte_devargs.h> +#include <rte_memcpy.h> #include "rte_pci_dev_ids.h" #include "eal_filesystem.h" @@ -228,20 +229,6 @@ error: return -1; } -/* Compare two PCI device addresses. */ -static int -pci_addr_comparison(struct rte_pci_addr *addr, struct rte_pci_addr *addr2) -{ - uint64_t dev_addr = (addr->domain << 24) + (addr->bus << 16) + (addr->devid << 8) + addr->function; - uint64_t dev_addr2 = (addr2->domain << 24) + (addr2->bus << 16) + (addr2->devid << 8) + addr2->function; - - if (dev_addr > dev_addr2) - return 1; - else - return 0; -} - - /* Scan one pci sysfs entry, and fill the devices list from it. */ static int pci_scan_one(const char *dirname, uint16_t domain, uint8_t bus, @@ -359,13 +346,24 @@ pci_scan_one(const char *dirname, uint16_t domain, uint8_t bus, } else { struct rte_pci_device *dev2 = NULL; + int ret; TAILQ_FOREACH(dev2, &pci_device_list, next) { - if (pci_addr_comparison(&dev->addr, &dev2->addr)) + ret = rte_eal_compare_pci_addr(&dev->addr, &dev2->addr); + if (ret > 0) continue; - else { + else if (ret < 0) { TAILQ_INSERT_BEFORE(dev2, dev, next); return 0; + } else { /* already registered */ + /* update pt_driver */ + dev2->pt_driver = dev->pt_driver; + dev2->max_vfs = dev->max_vfs; + memmove(dev2->mem_resource, + dev->mem_resource, + sizeof(dev->mem_resource)); + free(dev); + return 0; } } TAILQ_INSERT_TAIL(&pci_device_list, dev, next); diff --git a/lib/librte_eal/linuxapp/eal/eal_pci_uio.c b/lib/librte_eal/linuxapp/eal/eal_pci_uio.c index ff903d2..275c8cf 100644 --- a/lib/librte_eal/linuxapp/eal/eal_pci_uio.c +++ b/lib/librte_eal/linuxapp/eal/eal_pci_uio.c @@ -126,7 +126,7 @@ pci_uio_map_secondary(struct rte_pci_device *dev) TAILQ_FOREACH(uio_res, pci_res_list, next) { /* skip this element if it doesn't match our PCI address */ - if (memcmp(&uio_res->pci_addr, &dev->addr, sizeof(dev->addr))) + if (rte_eal_compare_pci_addr(&uio_res->pci_addr, &dev->addr)) continue; for (i = 0; i != uio_res->nb_maps; i++) { -- 1.9.1
This patch adds rte_eth_dev_release_port(). The function is used for changing an attached status of the device that has specified name. v9: - rte_eth_dev_free() is replaced by rte_eth_dev_release_port(). (Thanks to Thomas Monjalon) v6: - Use rte_eth_dev structure as the paramter of rte_eth_dev_free(). v4: - Add parameter checking. Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp> --- lib/librte_ether/rte_ethdev.c | 11 +++++++++++ lib/librte_ether/rte_ethdev.h | 12 ++++++++++++ 2 files changed, 23 insertions(+) diff --git a/lib/librte_ether/rte_ethdev.c b/lib/librte_ether/rte_ethdev.c index b702039..a089557 100644 --- a/lib/librte_ether/rte_ethdev.c +++ b/lib/librte_ether/rte_ethdev.c @@ -255,6 +255,17 @@ rte_eth_dev_allocate(const char *name) return eth_dev; } +int +rte_eth_dev_release_port(struct rte_eth_dev *eth_dev) +{ + if (eth_dev == NULL) + return -EINVAL; + + eth_dev->attached = 0; + nb_ports--; + return 0; +} + static int rte_eth_dev_init(struct rte_pci_driver *pci_drv, struct rte_pci_device *pci_dev) diff --git a/lib/librte_ether/rte_ethdev.h b/lib/librte_ether/rte_ethdev.h index 110ddba..7963e56 100644 --- a/lib/librte_ether/rte_ethdev.h +++ b/lib/librte_ether/rte_ethdev.h @@ -1539,6 +1539,18 @@ extern uint8_t rte_eth_dev_count(void); */ struct rte_eth_dev *rte_eth_dev_allocate(const char *name); +/** + * Function for internal use by dummy drivers primarily, e.g. ring-based + * driver. + * Release the specified ethdev port. + * + * @param eth_dev + * The *eth_dev* pointer is the address of the *rte_eth_dev* structure. + * @return + * - 0 on success, negative on error + */ +int rte_eth_dev_release_port(struct rte_eth_dev *eth_dev); + struct eth_driver; /** * @internal -- 1.9.1
The patch adds function pointer to rte_pci_driver and eth_driver structure. These function pointers are used when ports are detached. Also, the patch adds rte_eth_dev_uninit(). So far, it's not called by anywhere, but it will be called when port hotplug function is implemented. v10: - Add size parameter to rte_eth_dev_create_unique_device_name(). (Thanks to Iremonger, Bernard) v9: - Change parameter of pci_devuninit_t and rte_eth_dev_uninit. - Remove code that initiaize callback of ethdev from rte_eth_dev_uninit(). - Add a function to create a unique device name. (Thanks to Thomas Monjalon) v6: - Fix rte_eth_dev_uninit() to handle a return value of uninit function of PMD. v4: - Add parameter checking. - Change function names. Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp> --- lib/librte_eal/common/include/rte_pci.h | 6 ++++ lib/librte_ether/rte_ethdev.c | 64 +++++++++++++++++++++++++++++++-- lib/librte_ether/rte_ethdev.h | 24 +++++++++++++ 3 files changed, 92 insertions(+), 2 deletions(-) diff --git a/lib/librte_eal/common/include/rte_pci.h b/lib/librte_eal/common/include/rte_pci.h index 6814e91..4ea57cb 100644 --- a/lib/librte_eal/common/include/rte_pci.h +++ b/lib/librte_eal/common/include/rte_pci.h @@ -192,12 +192,18 @@ struct rte_pci_driver; typedef int (pci_devinit_t)(struct rte_pci_driver *, struct rte_pci_device *); /** + * Uninitialisation function for the driver called during hotplugging. + */ +typedef int (pci_devuninit_t)(struct rte_pci_device *); + +/** * A structure describing a PCI driver. */ struct rte_pci_driver { TAILQ_ENTRY(rte_pci_driver) next; /**< Next in list. */ const char *name; /**< Driver name. */ pci_devinit_t *devinit; /**< Device init. function. */ + pci_devuninit_t *devuninit; /**< Device uninit function. */ struct rte_pci_id *id_table; /**< ID table, NULL terminated. */ uint32_t drv_flags; /**< Flags contolling handling of device. */ }; diff --git a/lib/librte_ether/rte_ethdev.c b/lib/librte_ether/rte_ethdev.c index a089557..165ec74 100644 --- a/lib/librte_ether/rte_ethdev.c +++ b/lib/librte_ether/rte_ethdev.c @@ -266,6 +266,24 @@ rte_eth_dev_release_port(struct rte_eth_dev *eth_dev) return 0; } +static inline int +rte_eth_dev_create_unique_device_name(char *name, size_t size, + struct rte_pci_device *pci_dev) +{ + int ret; + + if ((name == NULL) || (pci_dev == NULL)) + return -EINVAL; + + ret = snprintf(name, size, "%d:%d.%d", + pci_dev->addr.bus, pci_dev->addr.devid, + pci_dev->addr.function); + if (ret < 0) + return ret; + + return 0; +} + static int rte_eth_dev_init(struct rte_pci_driver *pci_drv, struct rte_pci_device *pci_dev) @@ -279,8 +297,8 @@ rte_eth_dev_init(struct rte_pci_driver *pci_drv, eth_drv = (struct eth_driver *)pci_drv; /* Create unique Ethernet device name using PCI address */ - snprintf(ethdev_name, RTE_ETH_NAME_MAX_LEN, "%d:%d.%d", - pci_dev->addr.bus, pci_dev->addr.devid, pci_dev->addr.function); + rte_eth_dev_create_unique_device_name(ethdev_name, + sizeof(ethdev_name), pci_dev); eth_dev = rte_eth_dev_allocate(ethdev_name); if (eth_dev == NULL) @@ -321,6 +339,47 @@ rte_eth_dev_init(struct rte_pci_driver *pci_drv, return diag; } +static int +rte_eth_dev_uninit(struct rte_pci_device *pci_dev) +{ + const struct eth_driver *eth_drv; + struct rte_eth_dev *eth_dev; + char ethdev_name[RTE_ETH_NAME_MAX_LEN]; + int ret; + + if (pci_dev == NULL) + return -EINVAL; + + /* Create unique Ethernet device name using PCI address */ + rte_eth_dev_create_unique_device_name(ethdev_name, + sizeof(ethdev_name), pci_dev); + + eth_dev = rte_eth_dev_allocated(ethdev_name); + if (eth_dev == NULL) + return -ENODEV; + + eth_drv = (const struct eth_driver *)pci_dev->driver; + + /* Invoke PMD device uninit function */ + if (*eth_drv->eth_dev_uninit) { + ret = (*eth_drv->eth_dev_uninit)(eth_drv, eth_dev); + if (ret) + return ret; + } + + /* free ether device */ + rte_eth_dev_release_port(eth_dev); + + if (rte_eal_process_type() == RTE_PROC_PRIMARY) + rte_free(eth_dev->data->dev_private); + + eth_dev->pci_dev = NULL; + eth_dev->driver = NULL; + eth_dev->data = NULL; + + return 0; +} + /** * Register an Ethernet [Poll Mode] driver. * @@ -339,6 +398,7 @@ void rte_eth_driver_register(struct eth_driver *eth_drv) { eth_drv->pci_drv.devinit = rte_eth_dev_init; + eth_drv->pci_drv.devuninit = rte_eth_dev_uninit; rte_eal_pci_register(ð_drv->pci_drv); } diff --git a/lib/librte_ether/rte_ethdev.h b/lib/librte_ether/rte_ethdev.h index 7963e56..37c3765 100644 --- a/lib/librte_ether/rte_ethdev.h +++ b/lib/librte_ether/rte_ethdev.h @@ -1588,6 +1588,27 @@ typedef int (*eth_dev_init_t)(struct eth_driver *eth_drv, /** * @internal + * Finalization function of an Ethernet driver invoked for each matching + * Ethernet PCI device detected during the PCI closing phase. + * + * @param eth_drv + * The pointer to the [matching] Ethernet driver structure supplied by + * the PMD when it registered itself. + * @param eth_dev + * The *eth_dev* pointer is the address of the *rte_eth_dev* structure + * associated with the matching device and which have been [automatically] + * allocated in the *rte_eth_devices* array. + * @return + * - 0: Success, the device is properly finalized by the driver. + * In particular, the driver MUST free the *dev_ops* pointer + * of the *eth_dev* structure. + * - <0: Error code of the device initialization failure. + */ +typedef int (*eth_dev_uninit_t)(const struct eth_driver *eth_drv, + struct rte_eth_dev *eth_dev); + +/** + * @internal * The structure associated with a PMD Ethernet driver. * * Each Ethernet driver acts as a PCI driver and is represented by a generic @@ -1597,11 +1618,14 @@ typedef int (*eth_dev_init_t)(struct eth_driver *eth_drv, * * - The *eth_dev_init* function invoked for each matching PCI device. * + * - The *eth_dev_uninit* function invoked for each matching PCI device. + * * - The size of the private data to allocate for each matching device. */ struct eth_driver { struct rte_pci_driver pci_drv; /**< The PMD is also a PCI driver. */ eth_dev_init_t eth_dev_init; /**< Device init function. */ + eth_dev_uninit_t eth_dev_uninit; /**< Device uninit function. */ unsigned int dev_private_size; /**< Size of device private data. */ }; -- 1.9.1
The patch adds following functions. - rte_eth_dev_save() The function is used for saving current rte_eth_dev structures. - rte_eth_dev_get_changed_port() The function receives the rte_eth_dev structures, then compare these with current values to know which port is actually attached or detached. - rte_eth_dev_get_addr_by_port() The function returns a pci address of an ethdev specified by port identifier. - rte_eth_dev_get_port_by_addr() The function returns a port identifier of an ethdev specified by pci address. - rte_eth_dev_get_name_by_port() The function returns a unique identifier name of an ethdev specified by port identifier. - Add rte_eth_dev_is_detachable() The function returns whether a PMD supports detach function. Also, the patch changes scope of rte_eth_dev_allocated() to global. This function will be called by virtual PMDs to support port hotplug. So change scope of the function to global. v10: - Change order of version.map. (Thanks to Thomas Monjalon) v9: - rte_eth_dev_check_detachable() is replaced by rte_eth_dev_is_detachable(). - strncpy() is replaced by strcpy(). (Thanks to Thomas Monjalon) - Add missing symbol in version map. (Thanks to Nail Horman) v8: - Add size parameter to rte_eth_dev_save(). - Add missing symbol in version map. (Thanks to Qiu, Michael and Iremonger, Bernard) v7: - Add pt_driver checking to rte_eth_dev_check_detachable(). (Thanks to Qiu, Michael) v5: - Fix return value of below functions. rte_eth_dev_get_changed_port(). rte_eth_dev_get_port_by_addr(). v4: - Add parameter checking. v3: - Fix if-condition bug while comparing pci addresses. - Add error checking codes. Reported-by: Mark Enright <menrigh@brocade.com> Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp> --- lib/librte_ether/rte_ethdev.c | 103 ++++++++++++++++++++++++++++++++- lib/librte_ether/rte_ethdev.h | 83 ++++++++++++++++++++++++++ lib/librte_ether/rte_ether_version.map | 7 +++ 3 files changed, 192 insertions(+), 1 deletion(-) diff --git a/lib/librte_ether/rte_ethdev.c b/lib/librte_ether/rte_ethdev.c index 165ec74..1f6a066 100644 --- a/lib/librte_ether/rte_ethdev.c +++ b/lib/librte_ether/rte_ethdev.c @@ -201,7 +201,7 @@ rte_eth_dev_data_alloc(void) RTE_MAX_ETHPORTS * sizeof(*rte_eth_dev_data)); } -static struct rte_eth_dev * +struct rte_eth_dev * rte_eth_dev_allocated(const char *name) { unsigned i; @@ -426,6 +426,107 @@ rte_eth_dev_count(void) return (nb_ports); } +int +rte_eth_dev_save(struct rte_eth_dev *devs, size_t size) +{ + if ((devs == NULL) || + (size != sizeof(struct rte_eth_dev) * RTE_MAX_ETHPORTS)) + return -EINVAL; + + /* save current rte_eth_devices */ + memcpy(devs, rte_eth_devices, size); + return 0; +} + +int +rte_eth_dev_get_changed_port(struct rte_eth_dev *devs, uint8_t *port_id) +{ + if ((devs == NULL) || (port_id == NULL)) + return -EINVAL; + + /* check which port was attached or detached */ + for (*port_id = 0; *port_id < RTE_MAX_ETHPORTS; (*port_id)++, devs++) { + if (rte_eth_devices[*port_id].attached ^ devs->attached) + return 0; + } + return -ENODEV; +} + +int +rte_eth_dev_get_addr_by_port(uint8_t port_id, struct rte_pci_addr *addr) +{ + if (!rte_eth_dev_is_valid_port(port_id)) { + PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + return -EINVAL; + } + + if (addr == NULL) { + PMD_DEBUG_TRACE("Null pointer is specified\n"); + return -EINVAL; + } + + *addr = rte_eth_devices[port_id].pci_dev->addr; + return 0; +} + +int +rte_eth_dev_get_port_by_addr(struct rte_pci_addr *addr, uint8_t *port_id) +{ + struct rte_pci_addr *tmp; + + if ((addr == NULL) || (port_id == NULL)) { + PMD_DEBUG_TRACE("Null pointer is specified\n"); + return -EINVAL; + } + + for (*port_id = 0; *port_id < RTE_MAX_ETHPORTS; (*port_id)++) { + if (!rte_eth_devices[*port_id].attached) + continue; + if (!rte_eth_devices[*port_id].pci_dev) + continue; + tmp = &rte_eth_devices[*port_id].pci_dev->addr; + if (rte_eal_compare_pci_addr(tmp, addr) == 0) + return 0; + } + return -ENODEV; +} + +int +rte_eth_dev_get_name_by_port(uint8_t port_id, char *name) +{ + char *tmp; + + if (!rte_eth_dev_is_valid_port(port_id)) { + PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + return -EINVAL; + } + + if (name == NULL) { + PMD_DEBUG_TRACE("Null pointer is specified\n"); + return -EINVAL; + } + + /* shouldn't check 'rte_eth_devices[i].data', + * because it might be overwritten by VDEV PMD */ + tmp = rte_eth_dev_data[port_id].name; + strcpy(name, tmp); + return 0; +} + +int +rte_eth_dev_is_detachable(uint8_t port_id) +{ + uint32_t drv_flags; + + if (port_id >= RTE_MAX_ETHPORTS) { + PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + return -EINVAL; + } + + drv_flags = rte_eth_devices[port_id].driver->pci_drv.drv_flags; + return !(drv_flags & RTE_PCI_DRV_DETACHABLE); +} + static int rte_eth_dev_rx_queue_config(struct rte_eth_dev *dev, uint16_t nb_queues) { diff --git a/lib/librte_ether/rte_ethdev.h b/lib/librte_ether/rte_ethdev.h index 37c3765..5519ce0 100644 --- a/lib/librte_ether/rte_ethdev.h +++ b/lib/librte_ether/rte_ethdev.h @@ -1528,6 +1528,89 @@ extern struct rte_eth_dev rte_eth_devices[]; extern uint8_t rte_eth_dev_count(void); /** + * Function for internal use by port hotplug functions. + * Copies current ethdev structures to the specified pointer. + * + * @param devs The pointer to the ethdev structures + * @param size The size of ethdev structures + * @return + * - 0 on success, negative on error + */ +extern int rte_eth_dev_save(struct rte_eth_dev *devs, size_t size); + +/** + * Function for internal use by port hotplug functions. + * Compare the specified ethdev structures with currents. Then + * if there is a port which status is changed, fill the specified pointer + * with the port id of that port. + * @param devs The pointer to the ethdev structures + * @param port_id The pointer to the port id + * @return + * - 0 on success, negative on error + */ +extern int rte_eth_dev_get_changed_port( + struct rte_eth_dev *devs, uint8_t *port_id); + +/** + * Function for internal use by port hotplug functions. + * Returns a pci address of a ethdev specified by port identifier. + * @param port_id + * The port identifier of the Ethernet device + * @param addr + * The pointer to the pci address + * @return + * - 0 on success, negative on error + */ +extern int rte_eth_dev_get_addr_by_port( + uint8_t port_id, struct rte_pci_addr *addr); + +/** + * Function for internal use by port hotplug functions. + * Returns a port identifier of a ethdev specified by pci address. + * @param addr + * The pointer to the pci address of the Ethernet device. + * @param port_id + * The pointer to the port identifier + * @return + * - 0 on success, negative on error + */ +extern int rte_eth_dev_get_port_by_addr( + struct rte_pci_addr *addr, uint8_t *port_id); + +/** + * Function for internal use by port hotplug functions. + * Returns a unique identifier name of a ethdev specified by port identifier. + * @param port_id + * The port identifier. + * @param name + * The pointer to the Unique identifier name for each Ethernet device + * @return + * - 0 on success, negative on error + */ +extern int rte_eth_dev_get_name_by_port(uint8_t port_id, char *name); + +/** + * Function for internal use by port hotplug functions. + * Check whether or not, a PMD that is handling the ethdev specified by port + * identifier can support detach function. + * @param port_id + * The port identifier + * @return + * - 0 on supporting detach function, negative on not supporting + */ +extern int rte_eth_dev_is_detachable(uint8_t port_id); + +/** + * Function for internal use by port hotplug functions. + * Returns a ethdev slot specified by the unique identifier name. + * @param name + * The pointer to the Unique identifier name for each Ethernet device + * @return + * - The pointer to the ethdev slot, on success. NULL on error + */ +extern struct rte_eth_dev *rte_eth_dev_allocated(const char *name); + +/** * Function for internal use by dummy drivers primarily, e.g. ring-based * driver. * Allocates a new ethdev slot for an ethernet device and returns the pointer diff --git a/lib/librte_ether/rte_ether_version.map b/lib/librte_ether/rte_ether_version.map index 94fd685..7db7085 100644 --- a/lib/librte_ether/rte_ether_version.map +++ b/lib/librte_ether/rte_ether_version.map @@ -8,6 +8,7 @@ DPDK_2.0 { rte_eth_allmulticast_enable; rte_eth_allmulticast_get; rte_eth_dev_allocate; + rte_eth_dev_allocated; rte_eth_dev_bypass_event_show; rte_eth_dev_bypass_event_store; rte_eth_dev_bypass_init; @@ -34,9 +35,14 @@ DPDK_2.0 { rte_eth_dev_filter_supported; rte_eth_dev_flow_ctrl_get; rte_eth_dev_flow_ctrl_set; + rte_eth_dev_get_addr_by_port; + rte_eth_dev_get_changed_port; rte_eth_dev_get_mtu; + rte_eth_dev_get_name_by_port; + rte_eth_dev_get_port_by_addr; rte_eth_dev_get_vlan_offload; rte_eth_dev_info_get; + rte_eth_dev_is_detachable; rte_eth_dev_mac_addr_add; rte_eth_dev_mac_addr_remove; rte_eth_dev_priority_flow_ctrl_set; @@ -46,6 +52,7 @@ DPDK_2.0 { rte_eth_dev_rss_reta_update; rte_eth_dev_rx_queue_start; rte_eth_dev_rx_queue_stop; + rte_eth_dev_save; rte_eth_dev_set_link_down; rte_eth_dev_set_link_up; rte_eth_dev_set_mtu; -- 1.9.1
The patch adds functions for unmapping igb_uio resources. The patch is only for Linux and igb_uio environment. VFIO and BSD are not supported. v9: - Remove "rte_dev_hotplug.h". - Remove needless "#ifdef". (Thanks to Thomas Monjalon and Neil Horman) - Remove pci_unmap_device(). It will be implemented in later patch. v8: - Fix typo. (Thanks to Iremonger, Bernard) v5: - Fix pci_unmap_device() to check pt_driver. v4: - Add parameter checking. - Add header file to determine if hotplug can be enabled. Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp> --- lib/librte_eal/linuxapp/eal/eal_pci.c | 17 ++++++++ lib/librte_eal/linuxapp/eal/eal_pci_init.h | 7 ++++ lib/librte_eal/linuxapp/eal/eal_pci_uio.c | 65 ++++++++++++++++++++++++++++++ 3 files changed, 89 insertions(+) diff --git a/lib/librte_eal/linuxapp/eal/eal_pci.c b/lib/librte_eal/linuxapp/eal/eal_pci.c index 06bfc1a..d03429c 100644 --- a/lib/librte_eal/linuxapp/eal/eal_pci.c +++ b/lib/librte_eal/linuxapp/eal/eal_pci.c @@ -168,6 +168,23 @@ pci_map_resource(void *requested_addr, int fd, off_t offset, size_t size, return mapaddr; } +/* unmap a particular resource */ +void +pci_unmap_resource(void *requested_addr, size_t size) +{ + if (requested_addr == NULL) + return; + + /* Unmap the PCI memory resource of device */ + if (munmap(requested_addr, size)) { + RTE_LOG(ERR, EAL, "%s(): cannot munmap(%p, 0x%lx): %s\n", + __func__, requested_addr, (unsigned long)size, + strerror(errno)); + } else + RTE_LOG(DEBUG, EAL, " PCI memory unmapped at %p\n", + requested_addr); +} + /* parse the "resource" sysfs file */ static int pci_parse_sysfs_resource(const char *filename, struct rte_pci_device *dev) diff --git a/lib/librte_eal/linuxapp/eal/eal_pci_init.h b/lib/librte_eal/linuxapp/eal/eal_pci_init.h index 0a0853d..cc7c564 100644 --- a/lib/librte_eal/linuxapp/eal/eal_pci_init.h +++ b/lib/librte_eal/linuxapp/eal/eal_pci_init.h @@ -71,6 +71,13 @@ void *pci_map_resource(void *requested_addr, int fd, off_t offset, /* map IGB_UIO resource prototype */ int pci_uio_map_resource(struct rte_pci_device *dev); +void pci_unmap_resource(void *requested_addr, size_t size); + +#ifdef RTE_LIBRTE_EAL_HOTPLUG +/* unmap IGB_UIO resource prototype */ +void pci_uio_unmap_resource(struct rte_pci_device *dev); +#endif /* RTE_LIBRTE_EAL_HOTPLUG */ + #ifdef VFIO_PRESENT #define VFIO_MAX_GROUPS 64 diff --git a/lib/librte_eal/linuxapp/eal/eal_pci_uio.c b/lib/librte_eal/linuxapp/eal/eal_pci_uio.c index 275c8cf..5110d26 100644 --- a/lib/librte_eal/linuxapp/eal/eal_pci_uio.c +++ b/lib/librte_eal/linuxapp/eal/eal_pci_uio.c @@ -433,3 +433,68 @@ pci_uio_map_resource(struct rte_pci_device *dev) return 0; } + +#ifdef RTE_LIBRTE_EAL_HOTPLUG +static void +pci_uio_unmap(struct mapped_pci_resource *uio_res) +{ + int i; + + if (uio_res == NULL) + return; + + for (i = 0; i != uio_res->nb_maps; i++) + pci_unmap_resource(uio_res->maps[i].addr, + (size_t)uio_res->maps[i].size); +} + +static struct mapped_pci_resource * +pci_uio_find_resource(struct rte_pci_device *dev) +{ + struct mapped_pci_resource *uio_res; + + if (dev == NULL) + return NULL; + + TAILQ_FOREACH(uio_res, pci_res_list, next) { + + /* skip this element if it doesn't match our PCI address */ + if (!rte_eal_compare_pci_addr(&uio_res->pci_addr, &dev->addr)) + return uio_res; + } + return NULL; +} + +/* unmap the PCI resource of a PCI device in virtual memory */ +void +pci_uio_unmap_resource(struct rte_pci_device *dev) +{ + struct mapped_pci_resource *uio_res; + + if (dev == NULL) + return; + + /* find an entry for the device */ + uio_res = pci_uio_find_resource(dev); + if (uio_res == NULL) + return; + + /* secondary processes - just free maps */ + if (rte_eal_process_type() != RTE_PROC_PRIMARY) + return pci_uio_unmap(uio_res); + + TAILQ_REMOVE(pci_res_list, uio_res, next); + + /* unmap all resources */ + pci_uio_unmap(uio_res); + + /* free uio resource */ + rte_free(uio_res); + + /* close fd if in primary process */ + close(dev->intr_handle.fd); + + dev->intr_handle.fd = -1; + dev->intr_handle.type = RTE_INTR_HANDLE_UNKNOWN; +} +#endif /* RTE_LIBRTE_EAL_HOTPLUG */ -- 1.9.1
- Add pci_close_all_drivers() The function tries to find a driver for the specified device, and then close the driver. - Add rte_eal_pci_probe_one() and rte_eal_pci_close_one() The functions are used for probe and close a device. First the function tries to find a device that has the specified PCI address. Then, probe or close the device. v9: - Fix commit title. - Remove RTE_EAL_INVOKE_TYPE_PROBE/CLOSE. (Thanks to Thomas Monjalon) - Implement pci_unmap_device() in this patch. v5: - Remove RTE_EAL_INVOKE_TYPE_UNKNOWN, because it's unused. v4: - Fix parameter checking. - Fix indent of 'if' statement. Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp> --- lib/librte_eal/common/eal_common_pci.c | 98 ++++++++++++++++++++++++++++++++- lib/librte_eal/common/eal_private.h | 15 +++++ lib/librte_eal/common/include/rte_pci.h | 32 +++++++++++ lib/librte_eal/linuxapp/eal/eal_pci.c | 94 +++++++++++++++++++++++++++++++ 4 files changed, 238 insertions(+), 1 deletion(-) diff --git a/lib/librte_eal/common/eal_common_pci.c b/lib/librte_eal/common/eal_common_pci.c index bf2793f..5b6b55d 100644 --- a/lib/librte_eal/common/eal_common_pci.c +++ b/lib/librte_eal/common/eal_common_pci.c @@ -108,7 +108,10 @@ static int pci_probe_all_drivers(struct rte_pci_device *dev) { struct rte_pci_driver *dr = NULL; - int rc; + int rc = 0; + + if (dev == NULL) + return -1; TAILQ_FOREACH(dr, &pci_driver_list, next) { rc = rte_eal_pci_probe_one_driver(dr, dev); @@ -123,6 +126,99 @@ pci_probe_all_drivers(struct rte_pci_device *dev) return 1; } +#ifdef RTE_LIBRTE_EAL_HOTPLUG +/* + * If vendor/device ID match, call the devuninit() function of all + * registered driver for the given device. Return -1 if initialization + * failed, return 1 if no driver is found for this device. + */ +static int +pci_close_all_drivers(struct rte_pci_device *dev) +{ + struct rte_pci_driver *dr = NULL; + int rc = 0; + + if (dev == NULL) + return -1; + + TAILQ_FOREACH(dr, &pci_driver_list, next) { + rc = rte_eal_pci_close_one_driver(dr, dev); + if (rc < 0) + /* negative value is an error */ + return -1; + if (rc > 0) + /* positive value means driver not found */ + continue; + return 0; + } + return 1; +} + +/* + * Find the pci device specified by pci address, then invoke probe function of + * the driver of the devive. + */ +int +rte_eal_pci_probe_one(struct rte_pci_addr *addr) +{ + struct rte_pci_device *dev = NULL; + int ret = 0; + + if (addr == NULL) + return -1; + + TAILQ_FOREACH(dev, &pci_device_list, next) { + if (rte_eal_compare_pci_addr(&dev->addr, addr)) + continue; + + ret = pci_probe_all_drivers(dev); + if (ret < 0) + goto err_return; + return 0; + } + return -1; + +err_return: + RTE_LOG(WARNING, EAL, "Requested device " PCI_PRI_FMT + " cannot be used\n", dev->addr.domain, dev->addr.bus, + dev->addr.devid, dev->addr.function); + return -1; +} + +/* + * Find the pci device specified by pci address, then invoke close function of + * the driver of the devive. + */ +int +rte_eal_pci_close_one(struct rte_pci_addr *addr) +{ + struct rte_pci_device *dev = NULL; + int ret = 0; + + if (addr == NULL) + return -1; + + TAILQ_FOREACH(dev, &pci_device_list, next) { + if (rte_eal_compare_pci_addr(&dev->addr, addr)) + continue; + + ret = pci_close_all_drivers(dev); + if (ret < 0) + goto err_return; + + TAILQ_REMOVE(&pci_device_list, dev, next); + return 0; + } + return -1; + +err_return: + RTE_LOG(WARNING, EAL, "Requested device " PCI_PRI_FMT + " cannot be used\n", dev->addr.domain, dev->addr.bus, + dev->addr.devid, dev->addr.function); + return -1; +} +#endif /* RTE_LIBRTE_EAL_HOTPLUG */ + /* * Scan the content of the PCI bus, and call the devinit() function for * all registered drivers that have a matching entry in its id_table diff --git a/lib/librte_eal/common/eal_private.h b/lib/librte_eal/common/eal_private.h index 159cd66..4acf5a0 100644 --- a/lib/librte_eal/common/eal_private.h +++ b/lib/librte_eal/common/eal_private.h @@ -165,6 +165,21 @@ int rte_eal_pci_probe_one_driver(struct rte_pci_driver *dr, struct rte_pci_device *dev); /** + * Munmap memory for single PCI device + * + * This function is private to EAL. + * + * @param dr + * The pointer to the pci driver structure + * @param dev + * The pointer to the pci device structure + * @return + * 0 on success, negative on error + */ +int rte_eal_pci_close_one_driver(struct rte_pci_driver *dr, + struct rte_pci_device *dev); + +/** * Init tail queues for non-EAL library structures. This is to allow * the rings, mempools, etc. lists to be shared among multiple processes * diff --git a/lib/librte_eal/common/include/rte_pci.h b/lib/librte_eal/common/include/rte_pci.h index 4ea57cb..49f2e7f 100644 --- a/lib/librte_eal/common/include/rte_pci.h +++ b/lib/librte_eal/common/include/rte_pci.h @@ -325,6 +325,38 @@ rte_eal_compare_pci_addr(struct rte_pci_addr *addr, struct rte_pci_addr *addr2) */ int rte_eal_pci_probe(void); +#ifdef RTE_LIBRTE_EAL_HOTPLUG +/** + * Probe the single PCI device. + * + * Scan the content of the PCI bus, and find the pci device specified by pci + * address, then call the probe() function for registered driver that has a + * matching entry in its id_table for discovered device. + * + * @param addr + * The PCI Bus-Device-Function address to probe. + * @return + * - 0 on success. + * - Negative on error. + */ +int rte_eal_pci_probe_one(struct rte_pci_addr *addr); + +/** + * Close the single PCI device. + * + * Scan the content of the PCI bus, and find the pci device specified by pci + * address, then call the close() function for registered driver that has a + * matching entry in its id_table for discovered device. + * + * @param addr + * The PCI Bus-Device-Function address to close. + * @return + * - 0 on success. + * - Negative on error. + */ +int rte_eal_pci_close_one(struct rte_pci_addr *addr); +#endif /* RTE_LIBRTE_EAL_HOTPLUG */ + /** * Dump the content of the PCI bus. * diff --git a/lib/librte_eal/linuxapp/eal/eal_pci.c b/lib/librte_eal/linuxapp/eal/eal_pci.c index d03429c..f880f90 100644 --- a/lib/librte_eal/linuxapp/eal/eal_pci.c +++ b/lib/librte_eal/linuxapp/eal/eal_pci.c @@ -595,6 +595,31 @@ pci_map_device(struct rte_pci_device *dev) return ret; } +#ifdef RTE_LIBRTE_EAL_HOTPLUG +static void +pci_unmap_device(struct rte_pci_device *dev) +{ + if (dev == NULL) + return; + + /* try unmapping the NIC resources using VFIO if it exists */ + switch (dev->pt_driver) { + case RTE_PT_VFIO: + RTE_LOG(ERR, EAL, "Hotplug doesn't support vfio yet\n"); + break; + case RTE_PT_IGB_UIO: + case RTE_PT_UIO_GENERIC: + /* unmap resources for devices that use uio */ + pci_uio_unmap_resource(dev); + break; + default: + RTE_LOG(DEBUG, EAL, " Not managed by known pt driver," + " skipped\n"); + break; + } +} +#endif /* RTE_LIBRTE_EAL_HOTPLUG */ + /* * If vendor/device ID match, call the devinit() function of the * driver. @@ -666,6 +691,75 @@ rte_eal_pci_probe_one_driver(struct rte_pci_driver *dr, struct rte_pci_device *d return 1; } +#ifdef RTE_LIBRTE_EAL_HOTPLUG +/* + * If vendor/device ID match, call the devuninit() function of the + * driver. + */ +int +rte_eal_pci_close_one_driver(struct rte_pci_driver *dr, + struct rte_pci_device *dev) +{ + struct rte_pci_id *id_table; + + if ((dr == NULL) || (dev == NULL)) + return -EINVAL; + + for (id_table = dr->id_table ; id_table->vendor_id != 0; id_table++) { + + /* check if device's identifiers match the driver's ones */ + if (id_table->vendor_id != dev->id.vendor_id && + id_table->vendor_id != PCI_ANY_ID) + continue; + if (id_table->device_id != dev->id.device_id && + id_table->device_id != PCI_ANY_ID) + continue; + if (id_table->subsystem_vendor_id != + dev->id.subsystem_vendor_id && + id_table->subsystem_vendor_id != PCI_ANY_ID) + continue; + if (id_table->subsystem_device_id != + dev->id.subsystem_device_id && + id_table->subsystem_device_id != PCI_ANY_ID) + continue; + + struct rte_pci_addr *loc = &dev->addr; + + RTE_LOG(DEBUG, EAL, + "PCI device "PCI_PRI_FMT" on NUMA socket %i\n", + loc->domain, loc->bus, loc->devid, + loc->function, dev->numa_node); + + RTE_LOG(DEBUG, EAL, " remove driver: %x:%x %s\n", + dev->id.vendor_id, dev->id.device_id, + dr->name); + + /* call the driver devuninit() function */ + if (dr->devuninit && (dr->devuninit(dev) < 0)) + return -1; /* negative value is an error */ + + /* clear driver structure */ + dev->driver = NULL; + + if (dr->drv_flags & RTE_PCI_DRV_NEED_MAPPING) + /* unmap resources for devices that use igb_uio */ + pci_unmap_device(dev); + + return 0; + } + /* return positive value if driver is not found */ + return 1; +} +#else /* RTE_LIBRTE_EAL_HOTPLUG */ +int +rte_eal_pci_close_one_driver(struct rte_pci_driver *dr __rte_unused, + struct rte_pci_device *dev __rte_unused) +{ + RTE_LOG(ERR, EAL, "Hotplug support isn't enabled\n"); + return -1; +} +#endif /* RTE_LIBRTE_EAL_HOTPLUG */ + /* Init the PCI EAL subsystem */ int rte_eal_pci_init(void) -- 1.9.1
This new parameter is needed to keep device type like PCI or virtual. Port detaching processes are different between PCI device and virtual device. RTE_ETH_DEV_PCI indicates device type is PCI. RTE_ETH_DEV_VIRTUAL indicates device is virtual. v12: - Add missing symbol in version map. (Thanks to Iremonger, Bernard) v10: - Change order of version.map. (Thanks to Thomas Monjalon) - Fix comment of "rte_ethdev.h". (Thanks to Thomas Monjalon) v9: - Fix commit log. - RTE_ETH_DEV_PHYSICAL is replaced by RTE_ETH_DEV_PCI. (Thanks to Thomas Monjalon) v8: - NONE_TRACE is replaced by NO_TRACE. - Add missing symbol in version map. (Thanks to Qiu, Michael and Iremonger, Bernard) v4: - Fix comments of rte_eth_dev_type. Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp> --- app/test/virtual_pmd.c | 2 +- lib/librte_ether/rte_ethdev.c | 25 +++++++++++++++++++++++-- lib/librte_ether/rte_ethdev.h | 25 ++++++++++++++++++++++++- lib/librte_ether/rte_ether_version.map | 1 + lib/librte_pmd_af_packet/rte_eth_af_packet.c | 2 +- lib/librte_pmd_bond/rte_eth_bond_api.c | 2 +- lib/librte_pmd_pcap/rte_eth_pcap.c | 2 +- lib/librte_pmd_ring/rte_eth_ring.c | 2 +- lib/librte_pmd_xenvirt/rte_eth_xenvirt.c | 2 +- 9 files changed, 54 insertions(+), 9 deletions(-) diff --git a/app/test/virtual_pmd.c b/app/test/virtual_pmd.c index 785bccc..9b07ab1 100644 --- a/app/test/virtual_pmd.c +++ b/app/test/virtual_pmd.c @@ -580,7 +580,7 @@ virtual_ethdev_create(const char *name, struct ether_addr *mac_addr, goto err; /* reserve an ethdev entry */ - eth_dev = rte_eth_dev_allocate(name); + eth_dev = rte_eth_dev_allocate(name, RTE_ETH_DEV_PCI); if (eth_dev == NULL) goto err; diff --git a/lib/librte_ether/rte_ethdev.c b/lib/librte_ether/rte_ethdev.c index 1f6a066..4ebdd9f 100644 --- a/lib/librte_ether/rte_ethdev.c +++ b/lib/librte_ether/rte_ethdev.c @@ -227,7 +227,7 @@ rte_eth_dev_find_free_port(void) } struct rte_eth_dev * -rte_eth_dev_allocate(const char *name) +rte_eth_dev_allocate(const char *name, enum rte_eth_dev_type type) { uint8_t port_id; struct rte_eth_dev *eth_dev; @@ -251,6 +251,7 @@ rte_eth_dev_allocate(const char *name) snprintf(eth_dev->data->name, sizeof(eth_dev->data->name), "%s", name); eth_dev->data->port_id = port_id; eth_dev->attached = DEV_ATTACHED; + eth_dev->dev_type = type; nb_ports++; return eth_dev; } @@ -262,6 +263,7 @@ rte_eth_dev_release_port(struct rte_eth_dev *eth_dev) return -EINVAL; eth_dev->attached = 0; + eth_dev->dev_type = RTE_ETH_DEV_UNKNOWN; nb_ports--; return 0; } @@ -300,7 +302,7 @@ rte_eth_dev_init(struct rte_pci_driver *pci_drv, rte_eth_dev_create_unique_device_name(ethdev_name, sizeof(ethdev_name), pci_dev); - eth_dev = rte_eth_dev_allocate(ethdev_name); + eth_dev = rte_eth_dev_allocate(ethdev_name, RTE_ETH_DEV_PCI); if (eth_dev == NULL) return -ENOMEM; @@ -426,6 +428,14 @@ rte_eth_dev_count(void) return (nb_ports); } +enum rte_eth_dev_type +rte_eth_dev_get_device_type(uint8_t port_id) +{ + if (!rte_eth_dev_is_valid_port(port_id)) + return -1; + return rte_eth_devices[port_id].dev_type; +} + int rte_eth_dev_save(struct rte_eth_dev *devs, size_t size) { @@ -523,6 +533,17 @@ rte_eth_dev_is_detachable(uint8_t port_id) return -EINVAL; } + if (rte_eth_devices[port_id].dev_type == RTE_ETH_DEV_PCI) { + switch (rte_eth_devices[port_id].pci_dev->pt_driver) { + case RTE_PT_IGB_UIO: + case RTE_PT_UIO_GENERIC: + break; + case RTE_PT_VFIO: + default: + return -ENOTSUP; + } + } + drv_flags = rte_eth_devices[port_id].driver->pci_drv.drv_flags; return !(drv_flags & RTE_PCI_DRV_DETACHABLE); } diff --git a/lib/librte_ether/rte_ethdev.h b/lib/librte_ether/rte_ethdev.h index 5519ce0..d8e5543 100644 --- a/lib/librte_ether/rte_ethdev.h +++ b/lib/librte_ether/rte_ethdev.h @@ -1422,6 +1422,17 @@ struct rte_eth_rxtx_callback { void *param; }; +/* + * The eth device type + */ +enum rte_eth_dev_type { + RTE_ETH_DEV_UNKNOWN, /**< unknown device type */ + RTE_ETH_DEV_PCI, + /**< Physical function and Virtual function of PCI devices */ + RTE_ETH_DEV_VIRTUAL, /**< non hardware device */ + RTE_ETH_DEV_MAX /**< max value of this enum */ +}; + /** * @internal * The generic data structure associated with each ethernet device. @@ -1452,6 +1463,7 @@ struct rte_eth_dev { */ struct rte_eth_rxtx_callback **pre_tx_burst_cbs; uint8_t attached; /**< Flag indicating the port is attached */ + enum rte_eth_dev_type dev_type; /**< Flag indicating the device type */ }; struct rte_eth_dev_sriov { @@ -1529,6 +1541,15 @@ extern uint8_t rte_eth_dev_count(void); /** * Function for internal use by port hotplug functions. + * Get the device type to know whether the device is physical or virtual. + * @param port_id The port identifier of the Ethernet device + * @return + * - Device type. + */ +extern enum rte_eth_dev_type rte_eth_dev_get_device_type(uint8_t port_id); + +/** + * Function for internal use by port hotplug functions. * Copies current ethdev structures to the specified pointer. * * @param devs The pointer to the ethdev structures @@ -1617,10 +1638,12 @@ extern struct rte_eth_dev *rte_eth_dev_allocated(const char *name); * to that slot for the driver to use. * * @param name Unique identifier name for each Ethernet device + * @param type Device type of this Ethernet device * @return * - Slot in the rte_dev_devices array for a new device; */ -struct rte_eth_dev *rte_eth_dev_allocate(const char *name); +struct rte_eth_dev *rte_eth_dev_allocate(const char *name, + enum rte_eth_dev_type type); /** * Function for internal use by dummy drivers primarily, e.g. ring-based diff --git a/lib/librte_ether/rte_ether_version.map b/lib/librte_ether/rte_ether_version.map index 7db7085..c71eeec 100644 --- a/lib/librte_ether/rte_ether_version.map +++ b/lib/librte_ether/rte_ether_version.map @@ -37,6 +37,7 @@ DPDK_2.0 { rte_eth_dev_flow_ctrl_set; rte_eth_dev_get_addr_by_port; rte_eth_dev_get_changed_port; + rte_eth_dev_get_device_type; rte_eth_dev_get_mtu; rte_eth_dev_get_name_by_port; rte_eth_dev_get_port_by_addr; diff --git a/lib/librte_pmd_af_packet/rte_eth_af_packet.c b/lib/librte_pmd_af_packet/rte_eth_af_packet.c index 1ffe1cd..80e9bdf 100644 --- a/lib/librte_pmd_af_packet/rte_eth_af_packet.c +++ b/lib/librte_pmd_af_packet/rte_eth_af_packet.c @@ -649,7 +649,7 @@ rte_pmd_init_internals(const char *name, } /* reserve an ethdev entry */ - *eth_dev = rte_eth_dev_allocate(name); + *eth_dev = rte_eth_dev_allocate(name, RTE_ETH_DEV_VIRTUAL); if (*eth_dev == NULL) goto error; diff --git a/lib/librte_pmd_bond/rte_eth_bond_api.c b/lib/librte_pmd_bond/rte_eth_bond_api.c index 230d36c..903b7c3 100644 --- a/lib/librte_pmd_bond/rte_eth_bond_api.c +++ b/lib/librte_pmd_bond/rte_eth_bond_api.c @@ -257,7 +257,7 @@ rte_eth_bond_create(const char *name, uint8_t mode, uint8_t socket_id) } /* reserve an ethdev entry */ - eth_dev = rte_eth_dev_allocate(name); + eth_dev = rte_eth_dev_allocate(name, RTE_ETH_DEV_VIRTUAL); if (eth_dev == NULL) { RTE_BOND_LOG(ERR, "Unable to allocate rte_eth_dev"); goto err; diff --git a/lib/librte_pmd_pcap/rte_eth_pcap.c b/lib/librte_pmd_pcap/rte_eth_pcap.c index d299288..af7fae8 100644 --- a/lib/librte_pmd_pcap/rte_eth_pcap.c +++ b/lib/librte_pmd_pcap/rte_eth_pcap.c @@ -709,7 +709,7 @@ rte_pmd_init_internals(const char *name, const unsigned nb_rx_queues, goto error; /* reserve an ethdev entry */ - *eth_dev = rte_eth_dev_allocate(name); + *eth_dev = rte_eth_dev_allocate(name, RTE_ETH_DEV_VIRTUAL); if (*eth_dev == NULL) goto error; diff --git a/lib/librte_pmd_ring/rte_eth_ring.c b/lib/librte_pmd_ring/rte_eth_ring.c index a5dc71e..83ecfa8 100644 --- a/lib/librte_pmd_ring/rte_eth_ring.c +++ b/lib/librte_pmd_ring/rte_eth_ring.c @@ -297,7 +297,7 @@ rte_eth_from_rings(const char *name, struct rte_ring *const rx_queues[], goto error; /* reserve an ethdev entry */ - eth_dev = rte_eth_dev_allocate(name); + eth_dev = rte_eth_dev_allocate(name, RTE_ETH_DEV_VIRTUAL); if (eth_dev == NULL) goto error; diff --git a/lib/librte_pmd_xenvirt/rte_eth_xenvirt.c b/lib/librte_pmd_xenvirt/rte_eth_xenvirt.c index 04e30c9..bc403d6 100644 --- a/lib/librte_pmd_xenvirt/rte_eth_xenvirt.c +++ b/lib/librte_pmd_xenvirt/rte_eth_xenvirt.c @@ -648,7 +648,7 @@ eth_dev_xenvirt_create(const char *name, const char *params, goto err; /* reserve an ethdev entry */ - eth_dev = rte_eth_dev_allocate(name); + eth_dev = rte_eth_dev_allocate(name, RTE_ETH_DEV_VIRTUAL); if (eth_dev == NULL) goto err; -- 1.9.1
These functions are used for attaching or detaching a port. When rte_eal_dev_attach() is called, the function tries to realize the device name as pci address. If this is done successfully, rte_eal_dev_attach() will attach physical device port. If not, attaches virtual devive port. When rte_eal_dev_detach() is called, the function gets the device type of this port to know whether the port is come from physical or virtual. And then specific detaching function will be called. v13: - Change log level when error occurs in rte_eal_vdev_init() and rte_eal_dev_init(). - Return value of driver init and uninit functions. - Replace rte_panic by RTE_LOG in rte_eal_dev_init() - Fix return value of rte_eal_vdev_uninit(). - Fix rte_eal_dev_attach_vdev to set port_id correctly. (Thanks to Maxime Leroy) v11: - Remove needless devargs handling codes. - Replace get_vdev_name() by rte_eal_parse_devargs_str(). - Replace rte_eal_vdev_find_and_init by rte_eal_vdev_init() - Replace rte_eal_vdev_find_and_uninit by rte_eal_vdev_uninit() - Fix rte_eal_dev_init() to use rte_eal_vdev_init(). (Thanks to Maxime Leroy) v10: - Add comments. - Change order of version.map. (Thanks to Thomas Monjalon) v9: - Fix comments. - Use strcmp() instead of strncmp(). - Remove RTE_EAL_INVOKE_TYPE_PROBE/CLOSE. - Change definition of rte_dev_uninit_t. (Thanks to Thomas Monjalon and Maxime Leroy) v8: - Add missing symbol in version map. (Thanks to Qiu, Michael and Iremonger, Bernard) v7: - Fix typo of warning messages. (Thanks to Qiu, Michael) v5: - Change function names like below. rte_eal_dev_find_and_invoke() to rte_eal_vdev_find_and_invoke(). rte_eal_dev_invoke() to rte_eal_vdev_invoke(). - Add code to handle a return value of rte_eal_devargs_remove(). - Fix pci address format in rte_eal_dev_detach(). v4: - Fix comment. - Add error checking. - Fix indent of 'if' statement. - Change function name. Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp> --- lib/librte_eal/common/eal_common_dev.c | 284 ++++++++++++++++++++++-- lib/librte_eal/common/eal_common_devargs.c | 46 ++-- lib/librte_eal/common/eal_private.h | 11 + lib/librte_eal/common/include/rte_dev.h | 33 +++ lib/librte_eal/common/include/rte_devargs.h | 28 +++ lib/librte_eal/linuxapp/eal/Makefile | 1 + lib/librte_eal/linuxapp/eal/eal_pci.c | 6 +- lib/librte_eal/linuxapp/eal/rte_eal_version.map | 2 + 8 files changed, 376 insertions(+), 35 deletions(-) diff --git a/lib/librte_eal/common/eal_common_dev.c b/lib/librte_eal/common/eal_common_dev.c index eae5656..ca0558c 100644 --- a/lib/librte_eal/common/eal_common_dev.c +++ b/lib/librte_eal/common/eal_common_dev.c @@ -32,10 +32,13 @@ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ +#include <stdio.h> +#include <limits.h> #include <string.h> #include <inttypes.h> #include <sys/queue.h> +#include <rte_ethdev.h> #include <rte_dev.h> #include <rte_devargs.h> #include <rte_debug.h> @@ -61,6 +64,35 @@ rte_eal_driver_unregister(struct rte_driver *driver) TAILQ_REMOVE(&dev_driver_list, driver, next); } +static int +rte_eal_vdev_init(const char *name, const char *args) +{ + struct rte_driver *driver; + + if (name == NULL) + return -EINVAL; + + TAILQ_FOREACH(driver, &dev_driver_list, next) { + if (driver->type != PMD_VDEV) + continue; + + /* + * search a driver prefix in virtual device name. + * For example, if the driver is pcap PMD, driver->name + * will be "eth_pcap", but "name" will be "eth_pcapN". + * So use strncmp to compare. + */ + if (!strncmp(driver->name, name, strlen(driver->name))) + return driver->init(name, args); + } + + if (driver == NULL) { + RTE_LOG(ERR, EAL, "no driver found for %s\n", name); + return -EINVAL; + } + return 0; +} + int rte_eal_dev_init(void) { @@ -79,22 +111,11 @@ rte_eal_dev_init(void) if (devargs->type != RTE_DEVTYPE_VIRTUAL) continue; - TAILQ_FOREACH(driver, &dev_driver_list, next) { - if (driver->type != PMD_VDEV) - continue; - - /* search a driver prefix in virtual device name */ - if (!strncmp(driver->name, devargs->virtual.drv_name, - strlen(driver->name))) { - driver->init(devargs->virtual.drv_name, - devargs->args); - break; - } - } - - if (driver == NULL) { - rte_panic("no driver found for %s\n", - devargs->virtual.drv_name); + if (rte_eal_vdev_init(devargs->virtual.drv_name, + devargs->args)) { + RTE_LOG(ERR, EAL, "failed to initialize %s device\n", + devargs->virtual.drv_name); + return -1; } } @@ -107,3 +128,234 @@ rte_eal_dev_init(void) } return 0; } + +/* So far, DPDK hotplug function only supports linux */ +#ifdef RTE_LIBRTE_EAL_HOTPLUG +static int +rte_eal_vdev_uninit(const char *name) +{ + struct rte_driver *driver; + + if (name == NULL) + return -EINVAL; + + TAILQ_FOREACH(driver, &dev_driver_list, next) { + if (driver->type != PMD_VDEV) + continue; + + /* + * search a driver prefix in virtual device name. + * For example, if the driver is pcap PMD, driver->name + * will be "eth_pcap", but "name" will be "eth_pcapN". + * So use strncmp to compare. + */ + if (!strncmp(driver->name, name, strlen(driver->name))) + return driver->uninit(name); + } + + if (driver == NULL) { + RTE_LOG(ERR, EAL, "no driver found for %s\n", name); + return -EINVAL; + } + return 0; +} + +/* attach the new physical device, then store port_id of the device */ +static int +rte_eal_dev_attach_pdev(struct rte_pci_addr *addr, uint8_t *port_id) +{ + uint8_t new_port_id; + struct rte_eth_dev devs[RTE_MAX_ETHPORTS]; + + if ((addr == NULL) || (port_id == NULL)) + goto err; + + /* save current port status */ + if (rte_eth_dev_save(devs, sizeof(devs))) + goto err; + /* re-construct pci_device_list */ + if (rte_eal_pci_scan()) + goto err; + /* invoke probe func of the driver can handle the new device. + * TODO: + * rte_eal_pci_probe_one() should return port_id. + * And rte_eth_dev_save() and rte_eth_dev_get_changed_port() + * should be removed. */ + if (rte_eal_pci_probe_one(addr)) + goto err; + /* get port_id enabled by above procedures */ + if (rte_eth_dev_get_changed_port(devs, &new_port_id)) + goto err; + + *port_id = new_port_id; + return 0; +err: + RTE_LOG(ERR, EAL, "Driver, cannot attach the device\n"); + return -1; +} + +/* detach the new physical device, then store pci_addr of the device */ +static int +rte_eal_dev_detach_pdev(uint8_t port_id, struct rte_pci_addr *addr) +{ + struct rte_pci_addr freed_addr; + struct rte_pci_addr vp; + + if (addr == NULL) + goto err; + + /* check whether the driver supports detach feature, or not */ + if (rte_eth_dev_is_detachable(port_id)) + goto err; + + /* get pci address by port id */ + if (rte_eth_dev_get_addr_by_port(port_id, &freed_addr)) + goto err; + + /* Zerod pci addr means the port comes from virtual device */ + vp.domain = vp.bus = vp.devid = vp.function = 0; + if (rte_eal_compare_pci_addr(&vp, &freed_addr) == 0) + goto err; + + /* invoke close func of the driver, + * also remove the device from pci_device_list */ + if (rte_eal_pci_close_one(&freed_addr)) + goto err; + + *addr = freed_addr; + return 0; +err: + RTE_LOG(ERR, EAL, "Driver, cannot detach the device\n"); + return -1; +} + +/* attach the new virtual device, then store port_id of the device */ +static int +rte_eal_dev_attach_vdev(const char *vdevargs, uint8_t *port_id) +{ + char *name = NULL, *args = NULL; + uint8_t new_port_id; + struct rte_eth_dev devs[RTE_MAX_ETHPORTS]; + int ret = -1; + + if ((vdevargs == NULL) || (port_id == NULL)) + goto end; + + /* parse vdevargs, then retrieve device name and args */ + if (rte_eal_parse_devargs_str(vdevargs, &name, &args)) + goto end; + + /* save current port status */ + if (rte_eth_dev_save(devs, sizeof(devs))) + goto end; + /* walk around dev_driver_list to find the driver of the device, + * then invoke probe function o the driver. + * TODO: + * rte_eal_vdev_init() should return port_id, + * And rte_eth_dev_save() and rte_eth_dev_get_changed_port() + * should be removed. */ + if (rte_eal_vdev_init(name, args)) + goto end; + /* get port_id enabled by above procedures */ + if (rte_eth_dev_get_changed_port(devs, &new_port_id)) + goto end; + ret = 0; + *port_id = new_port_id; +end: + if (name) + free(name); + if (args) + free(args); + + if (ret < 0) + RTE_LOG(ERR, EAL, "Driver, cannot attach the device\n"); + return ret; +} + +/* detach the new virtual device, then store the name of the device */ +static int +rte_eal_dev_detach_vdev(uint8_t port_id, char *vdevname) +{ + char name[RTE_ETH_NAME_MAX_LEN]; + + if (vdevname == NULL) + goto err; + + /* check whether the driver supports detach feature, or not */ + if (rte_eth_dev_is_detachable(port_id)) + goto err; + + /* get device name by port id */ + if (rte_eth_dev_get_name_by_port(port_id, name)) + goto err; + /* walk around dev_driver_list to find the driver of the device, + * then invoke close function o the driver */ + if (rte_eal_vdev_uninit(name)) + goto err; + + strncpy(vdevname, name, sizeof(name)); + return 0; +err: + RTE_LOG(ERR, EAL, "Driver, cannot detach the device\n"); + return -1; +} + +/* attach the new device, then store port_id of the device */ +int +rte_eal_dev_attach(const char *devargs, uint8_t *port_id) +{ + struct rte_pci_addr addr; + + if ((devargs == NULL) || (port_id == NULL)) + return -EINVAL; + + if (eal_parse_pci_DomBDF(devargs, &addr) == 0) + return rte_eal_dev_attach_pdev(&addr, port_id); + else + return rte_eal_dev_attach_vdev(devargs, port_id); +} + +/* detach the device, then store the name of the device */ +int +rte_eal_dev_detach(uint8_t port_id, char *name) +{ + struct rte_pci_addr addr; + int ret; + + if (name == NULL) + return -EINVAL; + + if (rte_eth_dev_get_device_type(port_id) == RTE_ETH_DEV_PCI) { + ret = rte_eth_dev_get_addr_by_port(port_id, &addr); + if (ret < 0) + return ret; + + ret = rte_eal_dev_detach_pdev(port_id, &addr); + if (ret == 0) + snprintf(name, RTE_ETH_NAME_MAX_LEN, + "%04x:%02x:%02x.%d", + addr.domain, addr.bus, + addr.devid, addr.function); + + return ret; + } else + return rte_eal_dev_detach_vdev(port_id, name); +} +#else /* RTE_LIBRTE_EAL_HOTPLUG */ +int +rte_eal_dev_attach(const char *devargs __rte_unused, + uint8_t *port_id __rte_unused) +{ + RTE_LOG(ERR, EAL, "Hotplug support isn't enabled\n"); + return -1; +} + +/* detach the device, then store the name of the device */ +int +rte_eal_dev_detach(uint8_t port_id __rte_unused, + char *name __rte_unused) +{ + RTE_LOG(ERR, EAL, "Hotplug support isn't enabled\n"); + return -1; +} +#endif /* RTE_LIBRTE_EAL_HOTPLUG */ diff --git a/lib/librte_eal/common/eal_common_devargs.c b/lib/librte_eal/common/eal_common_devargs.c index 3aace08..2988a70 100644 --- a/lib/librte_eal/common/eal_common_devargs.c +++ b/lib/librte_eal/common/eal_common_devargs.c @@ -44,13 +44,41 @@ struct rte_devargs_list devargs_list = TAILQ_HEAD_INITIALIZER(devargs_list); +int +rte_eal_parse_devargs_str(const char *devargs_str, + char **drvname, char **drvargs) +{ + char *sep; + + if ((devargs_str == NULL) || drvname == NULL || drvargs == NULL) + return -1; + + *drvname = strdup(devargs_str); + if (drvname == NULL) { + RTE_LOG(ERR, EAL, "cannot allocate temp memory for devargs\n"); + return -1; + } + + /* set the first ',' to '\0' to split name and arguments */ + sep = strchr(*drvname, ','); + if (sep != NULL) { + sep[0] = '\0'; + *drvargs = strdup(sep + 1); + if (*drvargs == NULL) { + RTE_LOG(ERR, EAL, "cannot allocate for devargs args\n"); + free(*drvname); + return -1; + } + } + return 0; +} + /* store a whitelist parameter for later parsing */ int rte_eal_devargs_add(enum rte_devtype devtype, const char *devargs_str) { struct rte_devargs *devargs = NULL; char *buf = NULL; - char *sep; int ret; /* use malloc instead of rte_malloc as it's called early at init */ @@ -62,22 +90,8 @@ rte_eal_devargs_add(enum rte_devtype devtype, const char *devargs_str) memset(devargs, 0, sizeof(*devargs)); devargs->type = devtype; - buf = strdup(devargs_str); - if (buf == NULL) { - RTE_LOG(ERR, EAL, "cannot allocate temp memory for devargs\n"); + if (rte_eal_parse_devargs_str(devargs_str, &buf, &devargs->args)) goto fail; - } - - /* set the first ',' to '\0' to split name and arguments */ - sep = strchr(buf, ','); - if (sep != NULL) { - sep[0] = '\0'; - devargs->args = strdup(sep + 1); - if (devargs->args == NULL) { - RTE_LOG(ERR, EAL, "cannot allocate for devargs args\n"); - goto fail; - } - } switch (devargs->type) { case RTE_DEVTYPE_WHITELISTED_PCI: diff --git a/lib/librte_eal/common/eal_private.h b/lib/librte_eal/common/eal_private.h index 4acf5a0..98b286a 100644 --- a/lib/librte_eal/common/eal_private.h +++ b/lib/librte_eal/common/eal_private.h @@ -154,6 +154,17 @@ struct rte_pci_driver; struct rte_pci_device; /** + * Scan the content of the PCI bus, and the devices in the devices + * list + * + * This function is private to EAL. + * + * @return + * 0 on success, negative on error + */ +int rte_eal_pci_scan(void); + +/** * Mmap memory for single PCI device * * This function is private to EAL. diff --git a/lib/librte_eal/common/include/rte_dev.h b/lib/librte_eal/common/include/rte_dev.h index f7e3a10..a5ac770 100644 --- a/lib/librte_eal/common/include/rte_dev.h +++ b/lib/librte_eal/common/include/rte_dev.h @@ -47,6 +47,7 @@ extern "C" { #endif #include <sys/queue.h> +#include <rte_pci.h> /** Double linked list of device drivers. */ TAILQ_HEAD(rte_driver_list, rte_driver); @@ -57,6 +58,11 @@ TAILQ_HEAD(rte_driver_list, rte_driver); typedef int (rte_dev_init_t)(const char *name, const char *args); /** + * Uninitilization function called for each device driver once. + */ +typedef int (rte_dev_uninit_t)(const char *name); + +/** * Driver type enumeration */ enum pmd_type { @@ -72,6 +78,7 @@ struct rte_driver { enum pmd_type type; /**< PMD Driver type */ const char *name; /**< Driver name. */ rte_dev_init_t *init; /**< Device init. function. */ + rte_dev_uninit_t *uninit; /**< Device uninit. function. */ }; /** @@ -93,6 +100,32 @@ void rte_eal_driver_register(struct rte_driver *driver); void rte_eal_driver_unregister(struct rte_driver *driver); /** + * Attach a new device. + * + * @param devargs + * A pointer to a strings array describing the new device + * to be attached. The strings should be a pci address like + * '0000:01:00.0' or virtual device name like 'eth_pcap0'. + * @param port_id + * A pointer to a port identifier actually attached. + * @return + * 0 on success and port_id is filled, negative on error + */ +int rte_eal_dev_attach(const char *devargs, uint8_t *port_id); + +/** + * Detach a device. + * + * @param port_id + * The port identifier of the device to detach. + * @param addr + * A pointer to a device name actually detached. + * @return + * 0 on success and devname is filled, negative on error + */ +int rte_eal_dev_detach(uint8_t port_id, char *devname); + +/** * Initalize all the registered drivers in this process */ int rte_eal_dev_init(void); diff --git a/lib/librte_eal/common/include/rte_devargs.h b/lib/librte_eal/common/include/rte_devargs.h index 996e180..274a92c 100644 --- a/lib/librte_eal/common/include/rte_devargs.h +++ b/lib/librte_eal/common/include/rte_devargs.h @@ -99,6 +99,34 @@ TAILQ_HEAD(rte_devargs_list, rte_devargs); extern struct rte_devargs_list devargs_list; /** + * Parse a devargs string. + * + * For PCI devices, the format of arguments string is "PCI_ADDR" or + * "PCI_ADDR,key=val,key2=val2,...". Examples: "08:00.1", "0000:5:00.0", + * "04:00.0,arg=val". + * + * For virtual devices, the format of arguments string is "DRIVER_NAME*" + * or "DRIVER_NAME*,key=val,key2=val2,...". Examples: "eth_ring", + * "eth_ring0", "eth_pmdAnything,arg=0:arg2=1". + * + * The function parses the arguments string to get driver name and driver + * arguments. + * + * @param devargs_str + * The arguments as given by the user. + * @param drvname + * The pointer to the string to store parsed driver name. + * @param drvargs + * The pointer to the string to store parsed driver arguments. + * + * @return + * - 0 on success + * - A negative value on error + */ +int rte_eal_parse_devargs_str(const char *devargs_str, + char **drvname, char **drvargs); + +/** * Add a device to the user device list * * For PCI devices, the format of arguments string is "PCI_ADDR" or diff --git a/lib/librte_eal/linuxapp/eal/Makefile b/lib/librte_eal/linuxapp/eal/Makefile index e117cec..b59b201 100644 --- a/lib/librte_eal/linuxapp/eal/Makefile +++ b/lib/librte_eal/linuxapp/eal/Makefile @@ -45,6 +45,7 @@ CFLAGS += -I$(RTE_SDK)/lib/librte_eal/common/include CFLAGS += -I$(RTE_SDK)/lib/librte_ring CFLAGS += -I$(RTE_SDK)/lib/librte_mempool CFLAGS += -I$(RTE_SDK)/lib/librte_malloc +CFLAGS += -I$(RTE_SDK)/lib/librte_mbuf CFLAGS += -I$(RTE_SDK)/lib/librte_ether CFLAGS += -I$(RTE_SDK)/lib/librte_ivshmem CFLAGS += -I$(RTE_SDK)/lib/librte_pmd_ring diff --git a/lib/librte_eal/linuxapp/eal/eal_pci.c b/lib/librte_eal/linuxapp/eal/eal_pci.c index f880f90..6d4932d 100644 --- a/lib/librte_eal/linuxapp/eal/eal_pci.c +++ b/lib/librte_eal/linuxapp/eal/eal_pci.c @@ -440,8 +440,8 @@ error: * Scan the content of the PCI bus, and the devices in the devices * list */ -static int -pci_scan(void) +int +rte_eal_pci_scan(void) { struct dirent *e; DIR *dir; @@ -773,7 +773,7 @@ rte_eal_pci_init(void) if (internal_config.no_pci) return 0; - if (pci_scan() < 0) { + if (rte_eal_pci_scan() < 0) { RTE_LOG(ERR, EAL, "%s(): Cannot scan PCI bus\n", __func__); return -1; } diff --git a/lib/librte_eal/linuxapp/eal/rte_eal_version.map b/lib/librte_eal/linuxapp/eal/rte_eal_version.map index 5ed6e4d..5d40e02 100644 --- a/lib/librte_eal/linuxapp/eal/rte_eal_version.map +++ b/lib/librte_eal/linuxapp/eal/rte_eal_version.map @@ -20,6 +20,8 @@ DPDK_2.0 { rte_dump_tailq; rte_eal_alarm_cancel; rte_eal_alarm_set; + rte_eal_dev_attach; + rte_eal_dev_detach; rte_eal_dev_init; rte_eal_devargs_add; rte_eal_devargs_dump; -- 1.9.1
This patch adds a new section for describing port hotplug framework. Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp> --- doc/guides/prog_guide/index.rst | 1 + doc/guides/prog_guide/port_hotplug_framework.rst | 110 +++++++++++++++++++++++ 2 files changed, 111 insertions(+) create mode 100644 doc/guides/prog_guide/port_hotplug_framework.rst diff --git a/doc/guides/prog_guide/index.rst b/doc/guides/prog_guide/index.rst index de69682..60a6ac5 100644 --- a/doc/guides/prog_guide/index.rst +++ b/doc/guides/prog_guide/index.rst @@ -71,6 +71,7 @@ Programmer's Guide packet_classif_access_ctrl packet_framework vhost_lib + port_hotplug_framework source_org dev_kit_build_system dev_kit_root_make_help diff --git a/doc/guides/prog_guide/port_hotplug_framework.rst b/doc/guides/prog_guide/port_hotplug_framework.rst new file mode 100644 index 0000000..355ae28 --- /dev/null +++ b/doc/guides/prog_guide/port_hotplug_framework.rst @@ -0,0 +1,110 @@ +.. BSD LICENSE + Copyright(c) 2015 IGEL Co.,Ltd. All rights reserved. + All rights reserved. + + Redistribution and use in source and binary forms, with or without + modification, are permitted provided that the following conditions + are met: + + * Redistributions of source code must retain the above copyright + notice, this list of conditions and the following disclaimer. + * Redistributions in binary form must reproduce the above copyright + notice, this list of conditions and the following disclaimer in + the documentation and/or other materials provided with the + distribution. + * Neither the name of IGEL Co.,Ltd. nor the names of its + contributors may be used to endorse or promote products derived + from this software without specific prior written permission. + + THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS + "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT + LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR + A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT + OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, + SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT + LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, + DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY + THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT + (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE + OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + +Port Hotplug Framework +====================== + +The Port Hotplug Framework provides DPDK applications with the ability to +attach and detach ports at runtime. Because the framework depends on PMD +implementation, the ports that PMDs cannot handle are out of scope of this +framework. Furthermore, after detaching a port from a DPDK application, the +framework doesn't provide a way for removing the devices from the system. +For the ports backed by a physical NIC, the kernel will need to support PCI +Hotplug feature. + +Overview +-------- + +The basic requirements of the Port Hotplug Framework are: + +* DPDK applications that use the Port Hotplug Framework must manage their + own ports. + + The Port Hotplug Framework is implemented to allow DPDK applications to + manage ports. For example, when DPDK applications call the port attach + function, the attached port number is returned. DPDK applications can + also detach the port by port number. + +* Kernel support is needed for attaching or detaching physical device + ports. + + To attach new physical device ports, the device will be recognized by + userspace driver I/O framework in kernel at first. Then DPDK + applications can call the Port Hotplug functions to attach the ports. + For detaching, steps are vice versa. + +* Before detaching, they must be stopped and closed. + + DPDK applications must call "rte_eth_dev_stop()" and + "rte_eth_dev_close()" APIs before detaching ports. These functions will + start finalization sequence of the PMDs. + +* The framework doesn't affect legacy DPDK applications behavior. + + If the Port Hotplug functions aren't called, all legacy DPDK apps can + still work without modifications. + +Port Hotplug API overview +------------------------- + +* Attaching a port + + "rte_eal_dev_attach()" API attaches a port to DPDK application, and + returns the attached port number. Before calling the API, the device + should be recognized by an userspace driver I/O framework. The API + receives a pci address like "0000:01:00.0" or a virtual device name + like "eth_pcap0,iface=eth0". In the case of virtual device name, the + format is the same as the general "--vdev" option of DPDK. + +* Detaching a port + + "rte_eal_dev_detach()" API detaches a port from DPDK application, and + returns a pci address of the detached device or a virtual device name + of the device. + +Reference +--------- + + "testpmd" supports the Port Hotplug Framework. + +Limitations +----------- + +* The Port Hotplug APIs are not thread safe. + +* The framework can only be enabled with Linux. BSD is not supported. + +* To detach a port, the port should be backed by a device that igb_uio + manages. VFIO is not supported. + +* Not all PMDs support detaching feature. + To know whether a PMD can support detaching, search for the + "RTE_PCI_DRV_DETACHABLE" flag in PMD implementation. If the flag is + defined in the PMD, detaching is supported. -- 1.9.1
This patch adds finalization code to free resources allocated by the PMD. v6: - Fix a paramter of rte_eth_dev_free(). v4: - Change function name. Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp> --- lib/librte_pmd_pcap/rte_eth_pcap.c | 40 ++++++++++++++++++++++++++++++++++++++ 1 file changed, 40 insertions(+) diff --git a/lib/librte_pmd_pcap/rte_eth_pcap.c b/lib/librte_pmd_pcap/rte_eth_pcap.c index af7fae8..5e94930 100644 --- a/lib/librte_pmd_pcap/rte_eth_pcap.c +++ b/lib/librte_pmd_pcap/rte_eth_pcap.c @@ -498,6 +498,13 @@ static struct eth_dev_ops ops = { .stats_reset = eth_stats_reset, }; +static struct eth_driver rte_pcap_pmd = { + .pci_drv = { + .name = "rte_pcap_pmd", + .drv_flags = RTE_PCI_DRV_DETACHABLE, + }, +}; + /* * Function handler that opens the pcap file for reading a stores a * reference of it for use it later on. @@ -713,6 +720,10 @@ rte_pmd_init_internals(const char *name, const unsigned nb_rx_queues, if (*eth_dev == NULL) goto error; + /* check length of device name */ + if ((strlen((*eth_dev)->data->name) + 1) > sizeof(data->name)) + goto error; + /* now put it all together * - store queue data in internals, * - store numa_node info in pci_driver @@ -739,10 +750,13 @@ rte_pmd_init_internals(const char *name, const unsigned nb_rx_queues, data->nb_tx_queues = (uint16_t)nb_tx_queues; data->dev_link = pmd_link; data->mac_addrs = ð_addr; + strncpy(data->name, + (*eth_dev)->data->name, strlen((*eth_dev)->data->name)); (*eth_dev)->data = data; (*eth_dev)->dev_ops = &ops; (*eth_dev)->pci_dev = pci_dev; + (*eth_dev)->driver = &rte_pcap_pmd; return 0; @@ -927,10 +941,36 @@ rte_pmd_pcap_devinit(const char *name, const char *params) } +static int +rte_pmd_pcap_devuninit(const char *name) +{ + struct rte_eth_dev *eth_dev = NULL; + + RTE_LOG(INFO, PMD, "Closing pcap ethdev on numa socket %u\n", + rte_socket_id()); + + if (name == NULL) + return -1; + + /* reserve an ethdev entry */ + eth_dev = rte_eth_dev_allocated(name); + if (eth_dev == NULL) + return -1; + + rte_free(eth_dev->data->dev_private); + rte_free(eth_dev->data); + rte_free(eth_dev->pci_dev); + + rte_eth_dev_release_port(eth_dev); + + return 0; +} + static struct rte_driver pmd_pcap_drv = { .name = "eth_pcap", .type = PMD_VDEV, .init = rte_pmd_pcap_devinit, + .uninit = rte_pmd_pcap_devuninit, }; PMD_REGISTER_DRIVER(pmd_pcap_drv); -- 1.9.1
The patch introduces following commands. - port attach [ident] - port detach [port_id] - attach: attaching a port - detach: detaching a port - ident: pci address of physical device. Or device name and parameters of virtual device. (ex. 0000:02:00.0, eth_pcap0,iface=eth0) - port_id: port identifier v7: - Fix doc. (Thanks to Iremonger, Bernard) - Fix port checking implementation of star_port(); (Thanks to Qiu, Michael) v5: - Add testpmd documentation. (Thanks to Iremonger, Bernard) v4: - Fix strings of command help. Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp> --- app/test-pmd/cmdline.c | 137 +++++++++++++++---- app/test-pmd/config.c | 102 ++++++++------ app/test-pmd/parameters.c | 22 ++- app/test-pmd/testpmd.c | 199 +++++++++++++++++++++------- app/test-pmd/testpmd.h | 18 ++- doc/guides/testpmd_app_ug/testpmd_funcs.rst | 57 ++++++++ 6 files changed, 409 insertions(+), 126 deletions(-) diff --git a/app/test-pmd/cmdline.c b/app/test-pmd/cmdline.c index 4c9f423..c8312be 100644 --- a/app/test-pmd/cmdline.c +++ b/app/test-pmd/cmdline.c @@ -513,6 +513,12 @@ static void cmd_help_long_parsed(void *parsed_result, "port close (port_id|all)\n" " Close all ports or port_id.\n\n" + "port attach (ident)\n" + " Attach physical or virtual dev by pci address or virtual device name\n\n" + + "port detach (port_id)\n" + " Detach physical or virtual dev by port_id\n\n" + "port config (port_id|all)" " speed (10|100|1000|10000|40000|auto)" " duplex (half|full|auto)\n" @@ -793,6 +799,89 @@ cmdline_parse_inst_t cmd_operate_specific_port = { }, }; +/* *** attach a specified port *** */ +struct cmd_operate_attach_port_result { + cmdline_fixed_string_t port; + cmdline_fixed_string_t keyword; + cmdline_fixed_string_t identifier; +}; + +static void cmd_operate_attach_port_parsed(void *parsed_result, + __attribute__((unused)) struct cmdline *cl, + __attribute__((unused)) void *data) +{ + struct cmd_operate_attach_port_result *res = parsed_result; + + if (!strcmp(res->keyword, "attach")) + attach_port(res->identifier); + else + printf("Unknown parameter\n"); +} + +cmdline_parse_token_string_t cmd_operate_attach_port_port = + TOKEN_STRING_INITIALIZER(struct cmd_operate_attach_port_result, + port, "port"); +cmdline_parse_token_string_t cmd_operate_attach_port_keyword = + TOKEN_STRING_INITIALIZER(struct cmd_operate_attach_port_result, + keyword, "attach"); +cmdline_parse_token_string_t cmd_operate_attach_port_identifier = + TOKEN_STRING_INITIALIZER(struct cmd_operate_attach_port_result, + identifier, NULL); + +cmdline_parse_inst_t cmd_operate_attach_port = { + .f = cmd_operate_attach_port_parsed, + .data = NULL, + .help_str = "port attach identifier, " + "identifier: pci address or virtual dev name", + .tokens = { + (void *)&cmd_operate_attach_port_port, + (void *)&cmd_operate_attach_port_keyword, + (void *)&cmd_operate_attach_port_identifier, + NULL, + }, +}; + +/* *** detach a specified port *** */ +struct cmd_operate_detach_port_result { + cmdline_fixed_string_t port; + cmdline_fixed_string_t keyword; + uint8_t port_id; +}; + +static void cmd_operate_detach_port_parsed(void *parsed_result, + __attribute__((unused)) struct cmdline *cl, + __attribute__((unused)) void *data) +{ + struct cmd_operate_detach_port_result *res = parsed_result; + + if (!strcmp(res->keyword, "detach")) + detach_port(res->port_id); + else + printf("Unknown parameter\n"); +} + +cmdline_parse_token_string_t cmd_operate_detach_port_port = + TOKEN_STRING_INITIALIZER(struct cmd_operate_detach_port_result, + port, "port"); +cmdline_parse_token_string_t cmd_operate_detach_port_keyword = + TOKEN_STRING_INITIALIZER(struct cmd_operate_detach_port_result, + keyword, "detach"); +cmdline_parse_token_num_t cmd_operate_detach_port_port_id = + TOKEN_NUM_INITIALIZER(struct cmd_operate_detach_port_result, + port_id, UINT8); + +cmdline_parse_inst_t cmd_operate_detach_port = { + .f = cmd_operate_detach_port_parsed, + .data = NULL, + .help_str = "port detach port_id", + .tokens = { + (void *)&cmd_operate_detach_port_port, + (void *)&cmd_operate_detach_port_keyword, + (void *)&cmd_operate_detach_port_port_id, + NULL, + }, +}; + /* *** configure speed for all ports *** */ struct cmd_config_speed_all { cmdline_fixed_string_t port; @@ -847,7 +936,7 @@ cmd_config_speed_all_parsed(void *parsed_result, return; } - for (pid = 0; pid < nb_ports; pid++) { + FOREACH_PORT(pid, ports) { ports[pid].dev_conf.link_speed = link_speed; ports[pid].dev_conf.link_duplex = link_duplex; } @@ -915,10 +1004,8 @@ cmd_config_speed_specific_parsed(void *parsed_result, return; } - if (res->id >= nb_ports) { - printf("Port id %d must be less than %d\n", res->id, nb_ports); + if (port_id_is_invalid(res->id, ENABLED_WARN)) return; - } if (!strcmp(res->value1, "10")) link_speed = ETH_LINK_SPEED_10; @@ -1489,7 +1576,7 @@ cmd_config_rxtx_queue_parsed(void *parsed_result, return; } - if (port_id_is_invalid(res->portid)) + if (port_id_is_invalid(res->portid, ENABLED_WARN)) return; if (port_is_started(res->portid) != 1) { @@ -2889,7 +2976,7 @@ cmd_csum_parsed(void *parsed_result, int hw = 0; uint16_t mask = 0; - if (port_id_is_invalid(res->port_id)) { + if (port_id_is_invalid(res->port_id, ENABLED_WARN)) { printf("invalid port %d\n", res->port_id); return; } @@ -2981,10 +3068,8 @@ cmd_csum_tunnel_parsed(void *parsed_result, { struct cmd_csum_tunnel_result *res = parsed_result; - if (port_id_is_invalid(res->port_id)) { - printf("invalid port %d\n", res->port_id); + if (port_id_is_invalid(res->port_id, ENABLED_WARN)) return; - } if (!strcmp(res->onoff, "on")) ports[res->port_id].tx_ol_flags |= @@ -3039,7 +3124,7 @@ cmd_tso_set_parsed(void *parsed_result, struct cmd_tso_set_result *res = parsed_result; struct rte_eth_dev_info dev_info; - if (port_id_is_invalid(res->port_id)) + if (port_id_is_invalid(res->port_id, ENABLED_WARN)) return; if (!strcmp(res->mode, "set")) @@ -4015,10 +4100,8 @@ static void cmd_set_bond_mac_addr_parsed(void *parsed_result, struct cmd_set_bond_mac_addr_result *res = parsed_result; int ret; - if (res->port_num >= nb_ports) { - printf("Port id %d must be less than %d\n", res->port_num, nb_ports); + if (port_id_is_invalid(res->port_num, ENABLED_WARN)) return; - } ret = rte_eth_bond_mac_address_set(res->port_num, &res->address); @@ -4255,7 +4338,7 @@ static void cmd_set_promisc_mode_parsed(void *parsed_result, /* all ports */ if (allports) { - for (i = 0; i < nb_ports; i++) { + FOREACH_PORT(i, ports) { if (enable) rte_eth_promiscuous_enable(i); else @@ -4335,7 +4418,7 @@ static void cmd_set_allmulti_mode_parsed(void *parsed_result, /* all ports */ if (allports) { - for (i = 0; i < nb_ports; i++) { + FOREACH_PORT(i, ports) { if (enable) rte_eth_allmulticast_enable(i); else @@ -5023,25 +5106,25 @@ static void cmd_showportall_parsed(void *parsed_result, struct cmd_showportall_result *res = parsed_result; if (!strcmp(res->show, "clear")) { if (!strcmp(res->what, "stats")) - for (i = 0; i < nb_ports; i++) + FOREACH_PORT(i, ports) nic_stats_clear(i); else if (!strcmp(res->what, "xstats")) - for (i = 0; i < nb_ports; i++) + FOREACH_PORT(i, ports) nic_xstats_clear(i); } else if (!strcmp(res->what, "info")) - for (i = 0; i < nb_ports; i++) + FOREACH_PORT(i, ports) port_infos_display(i); else if (!strcmp(res->what, "stats")) - for (i = 0; i < nb_ports; i++) + FOREACH_PORT(i, ports) nic_stats_display(i); else if (!strcmp(res->what, "xstats")) - for (i = 0; i < nb_ports; i++) + FOREACH_PORT(i, ports) nic_xstats_display(i); else if (!strcmp(res->what, "fdir")) - for (i = 0; i < nb_ports; i++) + FOREACH_PORT(i, ports) fdir_get_infos(i); else if (!strcmp(res->what, "stat_qmap")) - for (i = 0; i < nb_ports; i++) + FOREACH_PORT(i, ports) nic_stats_mapping_display(i); } @@ -8687,6 +8770,8 @@ cmdline_parse_ctx_t main_ctx[] = { (cmdline_parse_inst_t *)&cmd_set_qmap, (cmdline_parse_inst_t *)&cmd_operate_port, (cmdline_parse_inst_t *)&cmd_operate_specific_port, + (cmdline_parse_inst_t *)&cmd_operate_attach_port, + (cmdline_parse_inst_t *)&cmd_operate_detach_port, (cmdline_parse_inst_t *)&cmd_config_speed_all, (cmdline_parse_inst_t *)&cmd_config_speed_specific, (cmdline_parse_inst_t *)&cmd_config_rx_tx, @@ -8758,7 +8843,7 @@ prompt(void) static void cmd_reconfig_device_queue(portid_t id, uint8_t dev, uint8_t queue) { - if (id < nb_ports) { + if (!port_id_is_invalid(id, DISABLED_WARN)) { /* check if need_reconfig has been set to 1 */ if (ports[id].need_reconfig == 0) ports[id].need_reconfig = dev; @@ -8768,7 +8853,7 @@ cmd_reconfig_device_queue(portid_t id, uint8_t dev, uint8_t queue) } else { portid_t pid; - for (pid = 0; pid < nb_ports; pid++) { + FOREACH_PORT(pid, ports) { /* check if need_reconfig has been set to 1 */ if (ports[pid].need_reconfig == 0) ports[pid].need_reconfig = dev; @@ -8786,10 +8871,8 @@ bypass_is_supported(portid_t port_id) struct rte_port *port; struct rte_pci_id *pci_id; - if (port_id >= nb_ports) { - printf("\tPort id must be less than %d.\n", nb_ports); + if (port_id_is_invalid(port_id, ENABLED_WARN)) return 0; - } /* Get the device id. */ port = &ports[port_id]; diff --git a/app/test-pmd/config.c b/app/test-pmd/config.c index d436ce8..49be819 100644 --- a/app/test-pmd/config.c +++ b/app/test-pmd/config.c @@ -112,11 +112,15 @@ nic_stats_display(portid_t port_id) struct rte_eth_stats stats; struct rte_port *port = &ports[port_id]; uint8_t i; + portid_t pid; static const char *nic_stats_border = "########################"; - if (port_id >= nb_ports) { - printf("Invalid port, range is [0, %d]\n", nb_ports - 1); + if (port_id_is_invalid(port_id, ENABLED_WARN)) { + printf("Valid port range is [0"); + FOREACH_PORT(pid, ports) + printf(", %d", pid); + printf("]\n"); return; } rte_eth_stats_get(port_id, &stats); @@ -189,8 +193,13 @@ nic_stats_display(portid_t port_id) void nic_stats_clear(portid_t port_id) { - if (port_id >= nb_ports) { - printf("Invalid port, range is [0, %d]\n", nb_ports - 1); + portid_t pid; + + if (port_id_is_invalid(port_id, ENABLED_WARN)) { + printf("Valid port range is [0"); + FOREACH_PORT(pid, ports) + printf(", %d", pid); + printf("]\n"); return; } rte_eth_stats_reset(port_id); @@ -237,11 +246,15 @@ nic_stats_mapping_display(portid_t port_id) { struct rte_port *port = &ports[port_id]; uint16_t i; + portid_t pid; static const char *nic_stats_mapping_border = "########################"; - if (port_id >= nb_ports) { - printf("Invalid port, range is [0, %d]\n", nb_ports - 1); + if (port_id_is_invalid(port_id, ENABLED_WARN)) { + printf("Valid port range is [0"); + FOREACH_PORT(pid, ports) + printf(", %d", pid); + printf("]\n"); return; } @@ -290,9 +303,13 @@ port_infos_display(portid_t port_id) int vlan_offload; struct rte_mempool * mp; static const char *info_border = "*********************"; + portid_t pid; - if (port_id >= nb_ports) { - printf("Invalid port, range is [0, %d]\n", nb_ports - 1); + if (port_id_is_invalid(port_id, ENABLED_WARN)) { + printf("Valid port range is [0"); + FOREACH_PORT(pid, ports) + printf(", %d", pid); + printf("]\n"); return; } port = &ports[port_id]; @@ -365,11 +382,14 @@ port_infos_display(portid_t port_id) } int -port_id_is_invalid(portid_t port_id) +port_id_is_invalid(portid_t port_id, enum print_warning warning) { - if (port_id < nb_ports) + if (ports[port_id].enabled) return 0; - printf("Invalid port %d (must be < nb_ports=%d)\n", port_id, nb_ports); + + if (warning == ENABLED_WARN) + printf("Invalid port %d\n", port_id); + return 1; } @@ -428,7 +448,7 @@ port_reg_bit_display(portid_t port_id, uint32_t reg_off, uint8_t bit_x) uint32_t reg_v; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; if (port_reg_off_is_invalid(port_id, reg_off)) return; @@ -447,7 +467,7 @@ port_reg_bit_field_display(portid_t port_id, uint32_t reg_off, uint8_t l_bit; uint8_t h_bit; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; if (port_reg_off_is_invalid(port_id, reg_off)) return; @@ -474,7 +494,7 @@ port_reg_display(portid_t port_id, uint32_t reg_off) { uint32_t reg_v; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; if (port_reg_off_is_invalid(port_id, reg_off)) return; @@ -488,7 +508,7 @@ port_reg_bit_set(portid_t port_id, uint32_t reg_off, uint8_t bit_pos, { uint32_t reg_v; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; if (port_reg_off_is_invalid(port_id, reg_off)) return; @@ -516,7 +536,7 @@ port_reg_bit_field_set(portid_t port_id, uint32_t reg_off, uint8_t l_bit; uint8_t h_bit; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; if (port_reg_off_is_invalid(port_id, reg_off)) return; @@ -550,7 +570,7 @@ port_reg_bit_field_set(portid_t port_id, uint32_t reg_off, void port_reg_set(portid_t port_id, uint32_t reg_off, uint32_t reg_v) { - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; if (port_reg_off_is_invalid(port_id, reg_off)) return; @@ -563,7 +583,7 @@ port_mtu_set(portid_t port_id, uint16_t mtu) { int diag; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; diag = rte_eth_dev_set_mtu(port_id, mtu); if (diag == 0) @@ -726,7 +746,7 @@ rx_ring_desc_display(portid_t port_id, queueid_t rxq_id, uint16_t rxd_id) { const struct rte_memzone *rx_mz; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; if (rx_queue_id_is_invalid(rxq_id)) return; @@ -743,7 +763,7 @@ tx_ring_desc_display(portid_t port_id, queueid_t txq_id, uint16_t txd_id) { const struct rte_memzone *tx_mz; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; if (tx_queue_id_is_invalid(txq_id)) return; @@ -804,7 +824,7 @@ port_rss_reta_info(portid_t port_id, uint16_t i, idx, shift; int ret; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; ret = rte_eth_dev_rss_reta_query(port_id, reta_conf, nb_entries); @@ -859,7 +879,7 @@ port_rss_hash_conf_show(portid_t port_id, int show_rss_key) uint8_t i; int diag; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; /* Get RSS hash key if asked to display it */ rss_conf.rss_key = (show_rss_key) ? rss_key : NULL; @@ -1423,12 +1443,8 @@ set_fwd_ports_list(unsigned int *portlist, unsigned int nb_pt) again: for (i = 0; i < nb_pt; i++) { port_id = (portid_t) portlist[i]; - if (port_id >= nb_ports) { - printf("Invalid port id %u >= %u\n", - (unsigned int) port_id, - (unsigned int) nb_ports); + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; - } if (record_now) fwd_ports_ids[i] = port_id; } @@ -1586,7 +1602,7 @@ vlan_extend_set(portid_t port_id, int on) int diag; int vlan_offload; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; vlan_offload = rte_eth_dev_get_vlan_offload(port_id); @@ -1608,7 +1624,7 @@ rx_vlan_strip_set(portid_t port_id, int on) int diag; int vlan_offload; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; vlan_offload = rte_eth_dev_get_vlan_offload(port_id); @@ -1629,7 +1645,7 @@ rx_vlan_strip_set_on_queue(portid_t port_id, uint16_t queue_id, int on) { int diag; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; diag = rte_eth_dev_set_vlan_strip_on_queue(port_id, queue_id, on); @@ -1644,7 +1660,7 @@ rx_vlan_filter_set(portid_t port_id, int on) int diag; int vlan_offload; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; vlan_offload = rte_eth_dev_get_vlan_offload(port_id); @@ -1665,7 +1681,7 @@ rx_vft_set(portid_t port_id, uint16_t vlan_id, int on) { int diag; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; if (vlan_id_is_invalid(vlan_id)) return; @@ -1682,7 +1698,7 @@ rx_vlan_all_filter_set(portid_t port_id, int on) { uint16_t vlan_id; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; for (vlan_id = 0; vlan_id < 4096; vlan_id++) rx_vft_set(port_id, vlan_id, on); @@ -1692,7 +1708,7 @@ void vlan_tpid_set(portid_t port_id, uint16_t tp_id) { int diag; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; diag = rte_eth_dev_set_vlan_ether_type(port_id, tp_id); @@ -1707,7 +1723,7 @@ vlan_tpid_set(portid_t port_id, uint16_t tp_id) void tx_vlan_set(portid_t port_id, uint16_t vlan_id) { - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; if (vlan_id_is_invalid(vlan_id)) return; @@ -1718,7 +1734,7 @@ tx_vlan_set(portid_t port_id, uint16_t vlan_id) void tx_vlan_reset(portid_t port_id) { - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; ports[port_id].tx_ol_flags &= ~TESTPMD_TX_OFFLOAD_INSERT_VLAN; } @@ -1726,7 +1742,7 @@ tx_vlan_reset(portid_t port_id) void tx_vlan_pvid_set(portid_t port_id, uint16_t vlan_id, int on) { - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; rte_eth_dev_set_vlan_pvid(port_id, vlan_id, on); @@ -1738,7 +1754,7 @@ set_qmap(portid_t port_id, uint8_t is_rx, uint16_t queue_id, uint8_t map_value) uint16_t i; uint8_t existing_mapping_found = 0; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; if (is_rx ? (rx_queue_id_is_invalid(queue_id)) : (tx_queue_id_is_invalid(queue_id))) @@ -1904,7 +1920,7 @@ fdir_get_infos(portid_t port_id) static const char *fdir_stats_border = "########################"; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; ret = rte_eth_dev_filter_supported(port_id, RTE_ETH_FILTER_FDIR); if (ret < 0) { @@ -2030,7 +2046,7 @@ set_vf_traffic(portid_t port_id, uint8_t is_rx, uint16_t vf, uint8_t on) { int diag; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; if (is_rx) diag = rte_eth_dev_set_vf_rx(port_id,vf,on); @@ -2052,7 +2068,7 @@ set_vf_rx_vlan(portid_t port_id, uint16_t vlan_id, uint64_t vf_mask, uint8_t on) { int diag; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; if (vlan_id_is_invalid(vlan_id)) return; @@ -2069,7 +2085,7 @@ set_queue_rate_limit(portid_t port_id, uint16_t queue_idx, uint16_t rate) int diag; struct rte_eth_link link; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return 1; rte_eth_link_get_nowait(port_id, &link); if (rate > link.link_speed) { @@ -2094,7 +2110,7 @@ set_vf_rate_limit(portid_t port_id, uint16_t vf, uint16_t rate, uint64_t q_msk) if (q_msk == 0) return 0; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return 1; rte_eth_link_get_nowait(port_id, &link); if (rate > link.link_speed) { diff --git a/app/test-pmd/parameters.c b/app/test-pmd/parameters.c index 473f824..fa5f2a8 100644 --- a/app/test-pmd/parameters.c +++ b/app/test-pmd/parameters.c @@ -366,6 +366,7 @@ parse_portnuma_config(const char *q_arg) }; unsigned long int_fld[_NUM_FLD]; char *str_fld[_NUM_FLD]; + portid_t pid; /* reset from value set at definition */ while ((p = strchr(p0,'(')) != NULL) { @@ -387,8 +388,11 @@ parse_portnuma_config(const char *q_arg) return -1; } port_id = (uint8_t)int_fld[FLD_PORT]; - if (port_id >= nb_ports) { - printf("Invalid port, range is [0, %d]\n", nb_ports - 1); + if (port_id_is_invalid(port_id, ENABLED_WARN)) { + printf("Valid port range is [0"); + FOREACH_PORT(pid, ports) + printf(", %d", pid); + printf("]\n"); return -1; } socket_id = (uint8_t)int_fld[FLD_SOCKET]; @@ -419,6 +423,7 @@ parse_ringnuma_config(const char *q_arg) }; unsigned long int_fld[_NUM_FLD]; char *str_fld[_NUM_FLD]; + portid_t pid; #define RX_RING_ONLY 0x1 #define TX_RING_ONLY 0x2 #define RXTX_RING 0x3 @@ -443,8 +448,11 @@ parse_ringnuma_config(const char *q_arg) return -1; } port_id = (uint8_t)int_fld[FLD_PORT]; - if (port_id >= nb_ports) { - printf("Invalid port, range is [0, %d]\n", nb_ports - 1); + if (port_id_is_invalid(port_id, ENABLED_WARN)) { + printf("Valid port range is [0"); + FOREACH_PORT(pid, ports) + printf(", %d", pid); + printf("]\n"); return -1; } socket_id = (uint8_t)int_fld[FLD_SOCKET]; @@ -615,12 +623,12 @@ launch_args_parse(int argc, char** argv) #endif if (!strcmp(lgopts[opt_idx].name, "nb-ports")) { n = atoi(optarg); - if (n > 0 && n <= nb_ports) + if (n > 0 && + !port_id_is_invalid(n, DISABLED_WARN)) nb_fwd_ports = (uint8_t) n; else rte_exit(EXIT_FAILURE, - "nb-ports should be > 0 and <= %d\n", - nb_ports); + "Invalid port %d\n", n); } if (!strcmp(lgopts[opt_idx].name, "nb-cores")) { n = atoi(optarg); diff --git a/app/test-pmd/testpmd.c b/app/test-pmd/testpmd.c index 605163b..44bb8b3 100644 --- a/app/test-pmd/testpmd.c +++ b/app/test-pmd/testpmd.c @@ -71,6 +71,7 @@ #include <rte_pci.h> #include <rte_ether.h> #include <rte_ethdev.h> +#include <rte_dev.h> #include <rte_string_fns.h> #ifdef RTE_LIBRTE_PMD_XENVIRT #include <rte_eth_xenvirt.h> @@ -315,7 +316,7 @@ uint16_t nb_rx_queue_stats_mappings = 0; /* Forward function declarations */ static void map_port_queue_stats_mapping_registers(uint8_t pi, struct rte_port *port); -static void check_all_ports_link_status(uint8_t port_num, uint32_t port_mask); +static void check_all_ports_link_status(uint32_t port_mask); /* * Check if all the ports are started. @@ -324,6 +325,20 @@ static void check_all_ports_link_status(uint8_t port_num, uint32_t port_mask); static int all_ports_started(void); /* + * Find next enabled port + */ +portid_t +find_next_port(portid_t p, struct rte_port *ports, int size) +{ + if (ports == NULL) + rte_exit(-EINVAL, "failed to find a next port id\n"); + + while ((ports[p].enabled == 0) && (p < size)) + p++; + return p; +} + +/* * Setup default configuration. */ static void @@ -552,7 +567,8 @@ init_config(void) + RTE_TEST_TX_DESC_MAX + MAX_PKT_BURST; if (!numa_support) - nb_mbuf_per_pool = (nb_mbuf_per_pool * nb_ports); + nb_mbuf_per_pool = + (nb_mbuf_per_pool * RTE_MAX_ETHPORTS); } if (!numa_support) { @@ -565,14 +581,19 @@ init_config(void) /* Configuration of Ethernet ports. */ ports = rte_zmalloc("testpmd: ports", - sizeof(struct rte_port) * nb_ports, + sizeof(struct rte_port) * RTE_MAX_ETHPORTS, RTE_CACHE_LINE_SIZE); if (ports == NULL) { - rte_exit(EXIT_FAILURE, "rte_zmalloc(%d struct rte_port) " - "failed\n", nb_ports); + rte_exit(EXIT_FAILURE, + "rte_zmalloc(%d struct rte_port) failed\n", + RTE_MAX_ETHPORTS); } - for (pid = 0; pid < nb_ports; pid++) { + /* enabled allocated ports */ + for (pid = 0; pid < nb_ports; pid++) + ports[pid].enabled = 1; + + FOREACH_PORT(pid, ports) { port = &ports[pid]; rte_eth_dev_info_get(pid, &port->dev_info); @@ -602,8 +623,7 @@ init_config(void) nb_mbuf_per_pool = nb_mbuf_per_pool/nb_ports; for (i = 0; i < MAX_SOCKET; i++) { - nb_mbuf = (nb_mbuf_per_pool * - port_per_socket[i]); + nb_mbuf = (nb_mbuf_per_pool * RTE_MAX_ETHPORTS); if (nb_mbuf) mbuf_pool_create(mbuf_data_size, nb_mbuf,i); @@ -635,14 +655,6 @@ reconfig(portid_t new_port_id, unsigned socket_id) struct rte_port *port; /* Reconfiguration of Ethernet ports. */ - ports = rte_realloc(ports, - sizeof(struct rte_port) * nb_ports, - RTE_CACHE_LINE_SIZE); - if (ports == NULL) { - rte_exit(EXIT_FAILURE, "rte_realloc(%d struct rte_port) failed\n", - nb_ports); - } - port = &ports[new_port_id]; rte_eth_dev_info_get(new_port_id, &port->dev_info); @@ -663,7 +675,7 @@ init_fwd_streams(void) streamid_t sm_id, nb_fwd_streams_new; /* set socket id according to numa or not */ - for (pid = 0; pid < nb_ports; pid++) { + FOREACH_PORT(pid, ports) { port = &ports[pid]; if (nb_rxq > port->dev_info.max_rx_queues) { printf("Fail: nb_rxq(%d) is greater than " @@ -1264,7 +1276,7 @@ all_ports_started(void) portid_t pi; struct rte_port *port; - for (pi = 0; pi < nb_ports; pi++) { + FOREACH_PORT(pi, ports) { port = &ports[pi]; /* Check if there is a port which is not started */ if (port->port_status != RTE_PORT_STARTED) @@ -1276,6 +1288,45 @@ all_ports_started(void) } int +all_ports_stopped(void) +{ + portid_t pi; + struct rte_port *port; + + FOREACH_PORT(pi, ports) { + port = &ports[pi]; + if (port->port_status != RTE_PORT_STOPPED) + return 0; + } + + return 1; +} + +int +port_is_started(portid_t port_id) +{ + if (port_id_is_invalid(port_id, ENABLED_WARN)) + return 0; + + if (ports[port_id].port_status != RTE_PORT_STARTED) + return 0; + + return 1; +} + +static int +port_is_closed(portid_t port_id) +{ + if (port_id_is_invalid(port_id, ENABLED_WARN)) + return 0; + + if (ports[port_id].port_status != RTE_PORT_CLOSED) + return 0; + + return 1; +} + +int start_port(portid_t pid) { int diag, need_check_link_status = 0; @@ -1296,8 +1347,8 @@ start_port(portid_t pid) if(dcb_config) dcb_test = 1; - for (pi = 0; pi < nb_ports; pi++) { - if (pid < nb_ports && pid != pi) + FOREACH_PORT(pi, ports) { + if (pid != pi && pid != (portid_t)RTE_PORT_ALL) continue; port = &ports[pi]; @@ -1421,7 +1472,7 @@ start_port(portid_t pid) } if (need_check_link_status && !no_link_check) - check_all_ports_link_status(nb_ports, RTE_PORT_ALL); + check_all_ports_link_status(RTE_PORT_ALL); else printf("Please stop the ports first\n"); @@ -1446,8 +1497,8 @@ stop_port(portid_t pid) } printf("Stopping ports...\n"); - for (pi = 0; pi < nb_ports; pi++) { - if (pid < nb_ports && pid != pi) + FOREACH_PORT(pi, ports) { + if (!port_id_is_invalid(pid, DISABLED_WARN) && pid != pi) continue; port = &ports[pi]; @@ -1463,7 +1514,7 @@ stop_port(portid_t pid) need_check_link_status = 1; } if (need_check_link_status && !no_link_check) - check_all_ports_link_status(nb_ports, RTE_PORT_ALL); + check_all_ports_link_status(RTE_PORT_ALL); printf("Done\n"); } @@ -1481,8 +1532,8 @@ close_port(portid_t pid) printf("Closing ports...\n"); - for (pi = 0; pi < nb_ports; pi++) { - if (pid < nb_ports && pid != pi) + FOREACH_PORT(pi, ports) { + if (!port_id_is_invalid(pid, DISABLED_WARN) && pid != pi) continue; port = &ports[pi]; @@ -1502,31 +1553,83 @@ close_port(portid_t pid) printf("Done\n"); } -int -all_ports_stopped(void) +void +attach_port(char *identifier) { - portid_t pi; - struct rte_port *port; + portid_t i, j, pi = 0; - for (pi = 0; pi < nb_ports; pi++) { - port = &ports[pi]; - if (port->port_status != RTE_PORT_STOPPED) - return 0; + printf("Attaching a new port...\n"); + + if (identifier == NULL) { + printf("Invalid parameters are specified\n"); + return; } - return 1; + if (test_done == 0) { + printf("Please stop forwarding first\n"); + return; + } + + if (rte_eal_dev_attach(identifier, &pi)) + return; + + ports[pi].enabled = 1; + reconfig(pi, rte_eth_dev_socket_id(pi)); + rte_eth_promiscuous_enable(pi); + + nb_ports = rte_eth_dev_count(); + + /* set_default_fwd_ports_config(); */ + bzero(fwd_ports_ids, sizeof(fwd_ports_ids)); + i = 0; + FOREACH_PORT(j, ports) { + fwd_ports_ids[i] = j; + i++; + } + nb_cfg_ports = nb_ports; + nb_fwd_ports++; + + ports[pi].port_status = RTE_PORT_STOPPED; + + printf("Port %d is attached. Now total ports is %d\n", pi, nb_ports); + printf("Done\n"); } -int -port_is_started(portid_t port_id) +void +detach_port(uint8_t port_id) { - if (port_id_is_invalid(port_id)) - return -1; + portid_t i, pi = 0; + char name[RTE_ETH_NAME_MAX_LEN]; - if (ports[port_id].port_status != RTE_PORT_STARTED) - return 0; + printf("Detaching a port...\n"); - return 1; + if (!port_is_closed(port_id)) { + printf("Please close port first\n"); + return; + } + + rte_eth_promiscuous_disable(port_id); + + if (rte_eal_dev_detach(port_id, name)) + return; + + ports[port_id].enabled = 0; + nb_ports = rte_eth_dev_count(); + + /* set_default_fwd_ports_config(); */ + bzero(fwd_ports_ids, sizeof(fwd_ports_ids)); + i = 0; + FOREACH_PORT(pi, ports) { + fwd_ports_ids[i] = pi; + i++; + } + nb_cfg_ports = nb_ports; + nb_fwd_ports--; + + printf("Port '%s' is detached. Now total ports is %d\n", + name, nb_ports); + printf("Done\n"); + return; } void @@ -1534,7 +1637,7 @@ pmd_test_exit(void) { portid_t pt_id; - for (pt_id = 0; pt_id < nb_ports; pt_id++) { + FOREACH_PORT(pt_id, ports) { printf("Stopping port %d...", pt_id); fflush(stdout); rte_eth_dev_close(pt_id); @@ -1553,7 +1656,7 @@ struct pmd_test_command { /* Check the link status of all ports in up to 9s, and print them finally */ static void -check_all_ports_link_status(uint8_t port_num, uint32_t port_mask) +check_all_ports_link_status(uint32_t port_mask) { #define CHECK_INTERVAL 100 /* 100ms */ #define MAX_CHECK_TIME 90 /* 9s (90 * 100ms) in total */ @@ -1564,7 +1667,7 @@ check_all_ports_link_status(uint8_t port_num, uint32_t port_mask) fflush(stdout); for (count = 0; count <= MAX_CHECK_TIME; count++) { all_ports_up = 1; - for (portid = 0; portid < port_num; portid++) { + FOREACH_PORT(portid, ports) { if ((port_mask & (1 << portid)) == 0) continue; memset(&link, 0, sizeof(link)); @@ -1729,7 +1832,7 @@ init_port_config(void) portid_t pid; struct rte_port *port; - for (pid = 0; pid < nb_ports; pid++) { + FOREACH_PORT(pid, ports) { port = &ports[pid]; port->dev_conf.rxmode = rx_mode; port->dev_conf.fdir_conf = fdir_conf; @@ -1908,7 +2011,7 @@ main(int argc, char** argv) nb_ports = (portid_t) rte_eth_dev_count(); if (nb_ports == 0) - rte_exit(EXIT_FAILURE, "No probed ethernet device\n"); + RTE_LOG(WARNING, EAL, "No probed ethernet devices\n"); set_def_fwd_config(); if (nb_lcores == 0) @@ -1930,7 +2033,7 @@ main(int argc, char** argv) rte_exit(EXIT_FAILURE, "Start ports failed\n"); /* set all ports to promiscuous mode by default */ - for (port_id = 0; port_id < nb_ports; port_id++) + FOREACH_PORT(port_id, ports) rte_eth_promiscuous_enable(port_id); #ifdef RTE_LIBRTE_CMDLINE diff --git a/app/test-pmd/testpmd.h b/app/test-pmd/testpmd.h index 126bef7..0d5a526 100644 --- a/app/test-pmd/testpmd.h +++ b/app/test-pmd/testpmd.h @@ -137,6 +137,7 @@ struct fwd_stream { * The data structure associated with each port. */ struct rte_port { + uint8_t enabled; /**< Port enabled or not */ struct rte_eth_dev_info dev_info; /**< PCI info + driver name */ struct rte_eth_conf dev_conf; /**< Port configuration. */ struct ether_addr eth_addr; /**< Port ethernet address */ @@ -162,6 +163,14 @@ struct rte_port { struct rte_eth_txconf tx_conf; /**< tx configuration */ }; +extern portid_t __rte_unused +find_next_port(portid_t p, struct rte_port *ports, int size); + +#define FOREACH_PORT(p, ports) \ + for (p = find_next_port(0, ports, RTE_MAX_ETHPORTS); \ + p < RTE_MAX_ETHPORTS; \ + p = find_next_port(p + 1, ports, RTE_MAX_ETHPORTS)) + /** * The data structure associated with each forwarding logical core. * The logical cores are internally numbered by a core index from 0 to @@ -522,6 +531,8 @@ int init_port_dcb_config(portid_t pid,struct dcb_config *dcb_conf); int start_port(portid_t pid); void stop_port(portid_t pid); void close_port(portid_t pid); +void attach_port(char *identifier); +void detach_port(uint8_t port_id); int all_ports_stopped(void); int port_is_started(portid_t port_id); void pmd_test_exit(void); @@ -548,10 +559,15 @@ void get_syn_filter(uint8_t port_id); void get_ethertype_filter(uint8_t port_id, uint16_t index); void get_2tuple_filter(uint8_t port_id, uint16_t index); void get_5tuple_filter(uint8_t port_id, uint16_t index); -int port_id_is_invalid(portid_t port_id); int rx_queue_id_is_invalid(queueid_t rxq_id); int tx_queue_id_is_invalid(queueid_t txq_id); +enum print_warning { + ENABLED_WARN = 0, + DISABLED_WARN +}; +int port_id_is_invalid(portid_t port_id, enum print_warning warning); + /* * Work-around of a compilation error with ICC on invocations of the * rte_be_to_cpu_16() function. diff --git a/doc/guides/testpmd_app_ug/testpmd_funcs.rst b/doc/guides/testpmd_app_ug/testpmd_funcs.rst index afe1970..a99e14d 100644 --- a/doc/guides/testpmd_app_ug/testpmd_funcs.rst +++ b/doc/guides/testpmd_app_ug/testpmd_funcs.rst @@ -807,6 +807,63 @@ The following sections show functions for configuring ports. Port configuration changes only become active when forwarding is started/restarted. +port attach +~~~~~~~~~~~ + +Attach a port specified by pci address or virtual device args. + +To attach a new pci device, the device should be recognized by kernel first. +Then it should be moved under DPDK management. +Finally the port can be attached to testpmd. +On the other hand, to attach a port created by virtual device, above steps are not needed. + +port attach (identifier) + +For example, to attach a port whose pci address is 0000:02:00.0. + +.. code-block:: console + + testpmd> port attach 0000:02:00.0 + Attaching a new port... + ... snip ... + Port 0 is attached. Now total ports is 1 + Done + +For example, to attach a port created by pcap PMD. + +.. code-block:: console + + testpmd> port attach eth_pcap0,iface=eth0 + Attaching a new port... + ... snip ... + Port 0 is attached. Now total ports is 1 + Done + +In this case, identifier is "eth_pcap0,iface=eth0". +This identifier format is the same as "--vdev" format of DPDK applications. + +port detach +~~~~~~~~~~~ + +Detach a specific port. + +Before detaching a port, the port should be closed. +Also to remove a pci device completely from the system, first detach the port from testpmd. +Then the device should be moved under kernel management. +Finally the device can be removed using kernel pci hotplug functionality. +On the other hand, to remove a port created by a virtual device, above steps are not needed. + +port detach (port_id) + +For example, to detach a port 0. + +.. code-block:: console + + testpmd> port detach 0 + Detaching a port... + ... snip ... + Done + port start ~~~~~~~~~~ -- 1.9.1
Hi Tetsuya, On Tue, Feb 24, 2015 at 5:49 AM, Tetsuya Mukawa <mukawa@igel.co.jp> wrote: > These functions are used for attaching or detaching a port. [...] > +static int > +rte_eal_vdev_init(const char *name, const char *args) > +{ > + struct rte_driver *driver; > + > + if (name == NULL) > + return -EINVAL; > + > + TAILQ_FOREACH(driver, &dev_driver_list, next) { > + if (driver->type != PMD_VDEV) > + continue; > + > + /* > + * search a driver prefix in virtual device name. > + * For example, if the driver is pcap PMD, driver->name > + * will be "eth_pcap", but "name" will be "eth_pcapN". > + * So use strncmp to compare. > + */ > + if (!strncmp(driver->name, name, strlen(driver->name))) > + return driver->init(name, args); > + } > + > + if (driver == NULL) { This test is not needed anymore. You should remove it. > + RTE_LOG(ERR, EAL, "no driver found for %s\n", name); > + return -EINVAL; > + } > + return 0; > +} [...] > } > + > +/* So far, DPDK hotplug function only supports linux */ > +#ifdef RTE_LIBRTE_EAL_HOTPLUG > +static int > +rte_eal_vdev_uninit(const char *name) > +{ > + struct rte_driver *driver; > + > + if (name == NULL) > + return -EINVAL; > + > + TAILQ_FOREACH(driver, &dev_driver_list, next) { > + if (driver->type != PMD_VDEV) > + continue; > + > + /* > + * search a driver prefix in virtual device name. > + * For example, if the driver is pcap PMD, driver->name > + * will be "eth_pcap", but "name" will be "eth_pcapN". > + * So use strncmp to compare. > + */ > + if (!strncmp(driver->name, name, strlen(driver->name))) > + return driver->uninit(name); > + } > + > + if (driver == NULL) { This test is not needed anymore . You should remove it. > + RTE_LOG(ERR, EAL, "no driver found for %s\n", name); > + return -EINVAL; > + } > + return 0; > +} > + [...] > diff --git a/lib/librte_eal/linuxapp/eal/Makefile b/lib/librte_eal/linuxapp/eal/Makefile > index e117cec..b59b201 100644 > --- a/lib/librte_eal/linuxapp/eal/Makefile > +++ b/lib/librte_eal/linuxapp/eal/Makefile > @@ -45,6 +45,7 @@ CFLAGS += -I$(RTE_SDK)/lib/librte_eal/common/include > CFLAGS += -I$(RTE_SDK)/lib/librte_ring > CFLAGS += -I$(RTE_SDK)/lib/librte_mempool > CFLAGS += -I$(RTE_SDK)/lib/librte_malloc > +CFLAGS += -I$(RTE_SDK)/lib/librte_mbuf Why do you need to add librte_mbuf into EAL Makefile ? > CFLAGS += -I$(RTE_SDK)/lib/librte_ether > CFLAGS += -I$(RTE_SDK)/lib/librte_ivshmem > CFLAGS += -I$(RTE_SDK)/lib/librte_pmd_ring > diff --git a/lib/librte_eal/linuxapp/eal/eal_pci.c b/lib/librte_eal/linuxapp/eal/eal_pci.c [...] > -- > 1.9.1 > Except these 3 points, ack for this patch. Regards, Maxime
2015-02-24 22:25 GMT+09:00 Maxime Leroy <maxime.leroy@6wind.com>: > Hi Tetsuya, > > On Tue, Feb 24, 2015 at 5:49 AM, Tetsuya Mukawa <mukawa@igel.co.jp> wrote: >> These functions are used for attaching or detaching a port. > [...] >> +static int >> +rte_eal_vdev_init(const char *name, const char *args) >> +{ >> + struct rte_driver *driver; >> + >> + if (name == NULL) >> + return -EINVAL; >> + >> + TAILQ_FOREACH(driver, &dev_driver_list, next) { >> + if (driver->type != PMD_VDEV) >> + continue; >> + >> + /* >> + * search a driver prefix in virtual device name. >> + * For example, if the driver is pcap PMD, driver->name >> + * will be "eth_pcap", but "name" will be "eth_pcapN". >> + * So use strncmp to compare. >> + */ >> + if (!strncmp(driver->name, name, strlen(driver->name))) >> + return driver->init(name, args); >> + } >> + >> + if (driver == NULL) { > > This test is not needed anymore. You should remove it. > Sure I will. >> + RTE_LOG(ERR, EAL, "no driver found for %s\n", name); >> + return -EINVAL; >> + } >> + return 0; >> +} > [...] >> } >> + >> +/* So far, DPDK hotplug function only supports linux */ >> +#ifdef RTE_LIBRTE_EAL_HOTPLUG >> +static int >> +rte_eal_vdev_uninit(const char *name) >> +{ >> + struct rte_driver *driver; >> + >> + if (name == NULL) >> + return -EINVAL; >> + >> + TAILQ_FOREACH(driver, &dev_driver_list, next) { >> + if (driver->type != PMD_VDEV) >> + continue; >> + >> + /* >> + * search a driver prefix in virtual device name. >> + * For example, if the driver is pcap PMD, driver->name >> + * will be "eth_pcap", but "name" will be "eth_pcapN". >> + * So use strncmp to compare. >> + */ >> + if (!strncmp(driver->name, name, strlen(driver->name))) >> + return driver->uninit(name); >> + } >> + >> + if (driver == NULL) { > > This test is not needed anymore . You should remove it. > I will. >> + RTE_LOG(ERR, EAL, "no driver found for %s\n", name); >> + return -EINVAL; >> + } >> + return 0; >> +} >> + > [...] >> diff --git a/lib/librte_eal/linuxapp/eal/Makefile b/lib/librte_eal/linuxapp/eal/Makefile >> index e117cec..b59b201 100644 >> --- a/lib/librte_eal/linuxapp/eal/Makefile >> +++ b/lib/librte_eal/linuxapp/eal/Makefile >> @@ -45,6 +45,7 @@ CFLAGS += -I$(RTE_SDK)/lib/librte_eal/common/include >> CFLAGS += -I$(RTE_SDK)/lib/librte_ring >> CFLAGS += -I$(RTE_SDK)/lib/librte_mempool >> CFLAGS += -I$(RTE_SDK)/lib/librte_malloc >> +CFLAGS += -I$(RTE_SDK)/lib/librte_mbuf > > Why do you need to add librte_mbuf into EAL Makefile ? > I need to call some ethdev library APIs like rte_eth_dev_is_detachable(). So rte_ethdev.h is needed to be included. Also, rte_mbuf.h is included in rte_ethdev.h. This is because, I need to refer librte_mbuf library also. I am out of office now, so I will submit new patches tomorrow. Regards, Tetsuya >> CFLAGS += -I$(RTE_SDK)/lib/librte_ether >> CFLAGS += -I$(RTE_SDK)/lib/librte_ivshmem >> CFLAGS += -I$(RTE_SDK)/lib/librte_pmd_ring >> diff --git a/lib/librte_eal/linuxapp/eal/eal_pci.c b/lib/librte_eal/linuxapp/eal/eal_pci.c > [...] >> -- >> 1.9.1 >> > > Except these 3 points, ack for this patch. > > Regards, > > Maxime
This patch series adds a dynamic port hotplug framework to DPDK. With the patches, DPDK apps can attach or detach ports at runtime. The basic concept of the port hotplug is like followings. - DPDK apps must have responsibility to manage ports. DPDK apps only know which ports are attached or detached at the moment. The port hotplug framework is implemented to allow DPDK apps to manage ports. For example, when DPDK apps call port attach function, attached port number will be returned. Also, DPDK apps can detach port by port number. - Kernel support is needed for attaching or detaching physical device ports. To attach a new physical device port, the device will be recognized by userspace directly I/O framework in kernel at first. Then DPDK apps can call the port hotplug functions to attach ports. For detaching, steps are vice versa. - Before detach ports, ports must be stopped and closed. DPDK application must call rte_eth_dev_stop() and rte_eth_dev_close() before detaching ports. These function will call finalization codes of PMDs. But so far, no PMD frees all resources allocated by initialization. It means PMDs are needed to be fixed to support the port hotplug. 'RTE_PCI_DRV_DETACHABLE' is a new flag indicating a PMD supports detaching. Without this flag, detaching will be failed. - Mustn't affect legacy DPDK apps. No DPDK EAL behavior is changed, if the port hotplug functions are't called. So all legacy DPDK apps can still work without modifications. And a few limitations. - The port hotplug functions are not thread safe. DPDK apps should handle it. - Only support Linux and igb_uio so far. BSD and VFIO is not supported. I will send VFIO patches at least, but I don't have a plan to submit BSD patch so far. Here is port hotplug APIs. ------------------------------------------------------------------------------- /** * Attach a new device. * * @param devargs * A pointer to a strings array describing the new device * to be attached. The strings should be a pci address like * '0000:01:00.0' or virtual device name like 'eth_pcap0'. * @param port_id * A pointer to a port identifier actually attached. * @return * 0 on success and port_id is filled, negative on error */ int rte_eal_dev_attach(const char *devargs, uint8_t *port_id); /** * Detach a device. * * @param port_id * The port identifier of the device to detach. * @param addr * A pointer to a device name actually detached. * @return * 0 on success and devname is filled, negative on error */ int rte_eal_dev_detach(uint8_t port_id, char *devname); ------------------------------------------------------------------------------- This patch series are for DPDK EAL. To use port hotplug function by DPDK apps, each PMD should be fixed to support 'RTE_PCI_DRV_DETACHABLE' flag. Please check a patch for pcap PMD. Also, please check testpmd patch. It will show you how to fix your legacy applications to support port hotplug feature. PATCH v14 changes - Remove needless if statement. (Thanks to Maxime Leroy) PATCH v13 changes - Change log level when error occurs in rte_eal_vdev_init() and rte_eal_dev_init(). - Return value of driver init and uninit functions. - Replace rte_panic by RTE_LOG in rte_eal_dev_init() - Fix return value of rte_eal_vdev_uninit(). - Fix rte_eal_dev_attach_vdev to set port_id correctly. (Thanks to Maxime Leroy) PATCH v12 changes - Add missing symbol in version map. (Thanks to Iremonger, Bernard) PATCH v11 changes - Remove needless devargs handling codes. - Replace get_vdev_name() by rte_eal_parse_devargs_str(). - Replace rte_eal_vdev_find_and_init by rte_eal_vdev_init() - Replace rte_eal_vdev_find_and_uninit by rte_eal_vdev_uninit() - Fix rte_eal_dev_init() to use rte_eal_vdev_init(). - Remove needless patch. (Thanks to Maxime Leroy) PATCH v10 changes - Add comments. - Chagne order of version.map. - Fix comment of "rte_ethdev.h". (Thanks to Thomas Monjalon) - Add size parameter to rte_eth_dev_create_unique_device_name(). (Thanks to Iremonger, Bernard) PATCH v9 changes - Fix commit title. - Fix commit log. - Fix comments. - Define CONFIG_RTE_LIBRTE_EAL_HOTPLUG at the top of this patch series. - DEV_INVALID/VALID are removed. - DEV_DISCONNECTED is replaced by DEV_DETACHED. - DEV_CONNECTED is replaced by DEV_ATTACHED. - rte_eth_dev_allocate_new_port() is renamed to rte_eth_dev_find_free_port(). - rte_eth_dev_validate_port() is renamed to rte_eth_dev_is_valid_port(). - rte_eth_dev_is_valid_port() is changed not to handle log toggle. - eal_compare_pci_addr() is replaced by rte_eal_compare_pci_addr(). - rte_eth_dev_free() is replaced by rte_eth_dev_release_port(). - Add a function to create a unique device name. - Change parameter of pci_devuninit_t and rte_eth_dev_uninit. - Remove code that initiaize callback of ethdev from rte_eth_dev_uninit(). - Remove pci_unmap_device(). It will be implemented in later patch. - rte_eth_dev_check_detachable() is replaced by rte_eth_dev_is_detachable(). - strncpy() is replaced by strcpy(). - Implement pci_unmap_device() in this patch. - Remove "rte_dev_hotplug.h". - Remove needless "#ifdef". - Remove RTE_EAL_INVOKE_TYPE_PROBE/CLOSE. - RTE_ETH_DEV_PHYSICAL is replaced by RTE_ETH_DEV_PCI. - Use strcmp() instead of strncmp(). - Remove RTE_EAL_INVOKE_TYPE_PROBE/CLOSE. (Thanks to Thomas Monjalon) - Change definition of rte_dev_uninit_t. (Thanks to Thomas Monjalon and Maxime Leroy) - Add missing symbol in version map. (Thanks to Nail Horman) PATCH v8 changes - Fix Makefile and add version map file. - Add missing symbol in version map. - Fix pci_scan_one() to update sysfs values. (Thanks to Qiu, Michael and Iremonger, Bernard) - NONE_TRACE is replaced by NO_TRACE. - Fix typo. - Add size parameter to rte_eth_dev_save(). (Thanks to Iremonger, Bernard) PATCH v7 changes - Add a new section to programmer's guide. (Thanks to Iremonger, Bernard) - Fix port checking implementation of star_port(). - Fix typo of warning messages. - Add pt_driver checking to rte_eth_dev_check_detachable(). (Thanks to Qiu, Michael) PATCH v6 changes - Fix rte_eth_dev_uninit() to handle a return value of uninit function of PMD. To do this, below changes also be applied. - Fix a parameter of rte_eth_dev_free(). - Use rte_eth_dev structure as the paramter of rte_eth_dev_free(). PATCH v5 changes - Add runtime check passthrough driver type, like vfio-pci, igb_uio and uio_pci_generic. This was done by Qiu, Michael. Thanks a lot. - Change function names like below. - rte_eal_dev_find_and_invoke() to rte_eal_vdev_find_and_invoke(). - rte_eal_dev_invoke() to rte_eal_vdev_invoke(). - Add code to handle a return value of rte_eal_devargs_remove(). - Fix pci address format in rte_eal_dev_detach(). - Remove RTE_EAL_INVOKE_TYPE_UNKNOWN, because it's unused. - Change function definition of rte_eal_devargs_remove(). - Fix pci_unmap_device() to check pt_driver. - Fix return value of below functions. - rte_eth_dev_get_changed_port(). - rte_eth_dev_get_port_by_addr(). - Change paramters of rte_eth_dev_validate_port() to cleanup code. - Fix pci_scan_one to handle pt_driver correctly. (Thanks to Qiu, Michael for above suggestions) PATCH v4 changes - Merge patches to review easier. - Fix indent of 'if' statement. - Fix calculation method of eal_compare_pci_addr(). - Fix header file declaration. - Add header file to determine if hotplug can be enabled. (Thanks to Qiu, Michael) - Use braces with 'for' loop. - Add parameter checking. - Fix sanity check code - Fix comments of rte_eth_dev_type. - Change function names. (Thanks to Iremonger, Bernard) PATCH v3 changes: - Fix enum definition used in rte_ethdev.c. (Thanks to Zhang, Helin) PATCH v2 changes: - Replace rte_eal_dev_attach_pdev(), rte_eal_dev_detach_pdev, rte_eal_dev_attach_vdev() and rte_eal_dev_detach_vdev() to rte_eal_dev_attach() and rte_eal_dev_detach(). - Add parameter values checking. - Refashion a few functions. (Thanks to Iremonger, Bernard) PATCH v1 Changes: - Fix error checking code of librte_eth APIs. - Fix issue that port from pcap PMD cannot be detached correctly. - Fix issue that testpmd could hang after forwarding, if attaching and detaching is repeatedly. - Fix if-condition of rte_eth_dev_get_port_by_addr(). (Thanks to Mark Enright) RFC PATCH v2 Changes: - remove 'rte_eth_dev_validate_port()', and cleanup codes. Michael Qiu (2): eal_pci: Add flag to hold kernel driver type eal_pci: pci memory map work with driver type Tetsuya Mukawa (11): eal: Enable port Hotplug framework in Linux eal/pci,ethdev: Remove assumption that port will not be detached eal/pci: Consolidate pci address comparison APIs ethdev: Add rte_eth_dev_release_port to release specified port eal,ethdev: Add a function and function pointers to close ether device ethdev: Add functions that will be used by port hotplug functions eal/linux/pci: Add functions for unmapping igb_uio resources eal/pci: Add probe and close functions of pci driver ethdev: Add one dev_type parameter to rte_eth_dev_allocate eal/pci: Add rte_eal_dev_attach/detach() functions doc: Add port hotplug framework section to programmers guide app/test/virtual_pmd.c | 2 +- config/common_bsdapp | 6 + config/common_linuxapp | 5 + doc/guides/prog_guide/index.rst | 1 + doc/guides/prog_guide/port_hotplug_framework.rst | 110 ++++++ lib/librte_eal/bsdapp/eal/eal_pci.c | 29 +- lib/librte_eal/common/eal_common_dev.c | 278 +++++++++++++- lib/librte_eal/common/eal_common_devargs.c | 54 ++- lib/librte_eal/common/eal_common_pci.c | 100 ++++- lib/librte_eal/common/eal_private.h | 26 ++ lib/librte_eal/common/include/rte_dev.h | 33 ++ lib/librte_eal/common/include/rte_devargs.h | 28 ++ lib/librte_eal/common/include/rte_pci.h | 82 +++++ lib/librte_eal/linuxapp/eal/Makefile | 1 + lib/librte_eal/linuxapp/eal/eal_pci.c | 230 ++++++++++-- lib/librte_eal/linuxapp/eal/eal_pci_init.h | 7 + lib/librte_eal/linuxapp/eal/eal_pci_uio.c | 67 +++- lib/librte_eal/linuxapp/eal/rte_eal_version.map | 2 + lib/librte_ether/rte_ethdev.c | 451 ++++++++++++++++++----- lib/librte_ether/rte_ethdev.h | 149 +++++++- lib/librte_ether/rte_ether_version.map | 8 + lib/librte_pmd_af_packet/rte_eth_af_packet.c | 2 +- lib/librte_pmd_bond/rte_eth_bond_api.c | 2 +- lib/librte_pmd_pcap/rte_eth_pcap.c | 2 +- lib/librte_pmd_ring/rte_eth_ring.c | 2 +- lib/librte_pmd_xenvirt/rte_eth_xenvirt.c | 2 +- 26 files changed, 1488 insertions(+), 191 deletions(-) create mode 100644 doc/guides/prog_guide/port_hotplug_framework.rst -- 1.9.1
The patch adds CONFIG_RTE_LIBRTE_EAL_HOTPLUG in Linux and BSD configuration. So far, Hotplug functions only support linux. v9: - Move this patch at the top of this patch series. (Thanks to Thomas Monjalon) Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp> --- config/common_bsdapp | 6 ++++++ config/common_linuxapp | 5 +++++ 2 files changed, 11 insertions(+) diff --git a/config/common_bsdapp b/config/common_bsdapp index 83a62a6..4108c01 100644 --- a/config/common_bsdapp +++ b/config/common_bsdapp @@ -116,6 +116,12 @@ CONFIG_RTE_LIBRTE_EAL_BSDAPP=y CONFIG_RTE_LIBRTE_EAL_LINUXAPP=n # +# Compile Environment Abstraction Layer to support hotplug +# So far, Hotplug functions only support linux +# +CONFIG_RTE_LIBRTE_EAL_HOTPLUG=n + +# # Compile Environment Abstraction Layer to support Vmware TSC map # CONFIG_RTE_LIBRTE_EAL_VMWARE_TSC_MAP_SUPPORT=y diff --git a/config/common_linuxapp b/config/common_linuxapp index 2716381..8ba0258 100644 --- a/config/common_linuxapp +++ b/config/common_linuxapp @@ -114,6 +114,11 @@ CONFIG_RTE_PCI_MAX_READ_REQUEST_SIZE=0 CONFIG_RTE_LIBRTE_EAL_LINUXAPP=y # +# Compile Environment Abstraction Layer to support hotplug +# +CONFIG_RTE_LIBRTE_EAL_HOTPLUG=y + +# # Compile Environment Abstraction Layer to support Vmware TSC map # CONFIG_RTE_LIBRTE_EAL_VMWARE_TSC_MAP_SUPPORT=y -- 1.9.1
From: Michael Qiu <michael.qiu@intel.com> Currently, dpdk has no ability to know which type of driver( vfio-pci/igb_uio/uio_pci_generic) the device used. It only can check whether vfio is enabled or not staticly. It really useful to have the flag, becasue different type need to handle differently in runtime. For example, pci memory map, pot hotplug, and so on. This patch add a flag field for pci device to solve above issue. Signed-off-by: Michael Qiu <michael.qiu@intel.com> Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp> --- lib/librte_eal/common/include/rte_pci.h | 8 +++++ lib/librte_eal/linuxapp/eal/eal_pci.c | 53 +++++++++++++++++++++++++++++++-- 2 files changed, 59 insertions(+), 2 deletions(-) diff --git a/lib/librte_eal/common/include/rte_pci.h b/lib/librte_eal/common/include/rte_pci.h index 3df07e8..a87b4b3 100644 --- a/lib/librte_eal/common/include/rte_pci.h +++ b/lib/librte_eal/common/include/rte_pci.h @@ -142,6 +142,13 @@ struct rte_pci_addr { struct rte_devargs; +enum rte_pt_driver { + RTE_PT_UNKNOWN = 0, + RTE_PT_IGB_UIO = 1, + RTE_PT_VFIO = 2, + RTE_PT_UIO_GENERIC = 3, +}; + /** * A structure describing a PCI device. */ @@ -155,6 +162,7 @@ struct rte_pci_device { uint16_t max_vfs; /**< sriov enable if not zero */ int numa_node; /**< NUMA node connection */ struct rte_devargs *devargs; /**< Device user arguments */ + enum rte_pt_driver pt_driver; /**< Driver of passthrough */ }; /** Any PCI device identifier (vendor, device, ...) */ diff --git a/lib/librte_eal/linuxapp/eal/eal_pci.c b/lib/librte_eal/linuxapp/eal/eal_pci.c index a4fd5f5..4615756 100644 --- a/lib/librte_eal/linuxapp/eal/eal_pci.c +++ b/lib/librte_eal/linuxapp/eal/eal_pci.c @@ -97,6 +97,35 @@ error: return -1; } +static int +pci_get_kernel_driver_by_path(const char *filename, char *dri_name) +{ + int count; + char path[PATH_MAX]; + char *name; + + if (!filename || !dri_name) + return -1; + + count = readlink(filename, path, PATH_MAX); + if (count >= PATH_MAX) + return -1; + + /* For device does not have a driver */ + if (count < 0) + return 1; + + path[count] = '\0'; + + name = strrchr(path, '/'); + if (name) { + strncpy(dri_name, name + 1, strlen(name + 1) + 1); + return 0; + } + + return -1; +} + void * pci_find_max_end_va(void) { @@ -221,11 +250,12 @@ pci_scan_one(const char *dirname, uint16_t domain, uint8_t bus, char filename[PATH_MAX]; unsigned long tmp; struct rte_pci_device *dev; + char driver[PATH_MAX]; + int ret; dev = malloc(sizeof(*dev)); - if (dev == NULL) { + if (dev == NULL) return -1; - } memset(dev, 0, sizeof(*dev)); dev->addr.domain = domain; @@ -304,6 +334,25 @@ pci_scan_one(const char *dirname, uint16_t domain, uint8_t bus, return -1; } + /* parse driver */ + snprintf(filename, sizeof(filename), "%s/driver", dirname); + ret = pci_get_kernel_driver_by_path(filename, driver); + if (!ret) { + if (!strcmp(driver, "vfio-pci")) + dev->pt_driver = RTE_PT_VFIO; + else if (!strcmp(driver, "igb_uio")) + dev->pt_driver = RTE_PT_IGB_UIO; + else if (!strcmp(driver, "uio_pci_generic")) + dev->pt_driver = RTE_PT_UIO_GENERIC; + else + dev->pt_driver = RTE_PT_UNKNOWN; + } else if (ret < 0) { + RTE_LOG(ERR, EAL, "Fail to get kernel driver\n"); + free(dev); + return -1; + } else + dev->pt_driver = RTE_PT_UNKNOWN; + /* device is valid, add in list (sorted) */ if (TAILQ_EMPTY(&pci_device_list)) { TAILQ_INSERT_TAIL(&pci_device_list, dev, next); -- 1.9.1
From: Michael Qiu <michael.qiu@intel.com> With the driver type flag in struct rte_pci_dev, we do not need to always map uio devices with vfio related function when vfio enabled. Signed-off-by: Michael Qiu <michael.qiu@intel.com> Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp> --- lib/librte_eal/linuxapp/eal/eal_pci.c | 30 +++++++++++++++++------------- 1 file changed, 17 insertions(+), 13 deletions(-) diff --git a/lib/librte_eal/linuxapp/eal/eal_pci.c b/lib/librte_eal/linuxapp/eal/eal_pci.c index 4615756..3291c68 100644 --- a/lib/librte_eal/linuxapp/eal/eal_pci.c +++ b/lib/librte_eal/linuxapp/eal/eal_pci.c @@ -555,25 +555,29 @@ pci_config_space_set(struct rte_pci_device *dev) static int pci_map_device(struct rte_pci_device *dev) { - int ret, mapped = 0; + int ret = -1; /* try mapping the NIC resources using VFIO if it exists */ + switch (dev->pt_driver) { + case RTE_PT_VFIO: #ifdef VFIO_PRESENT - if (pci_vfio_is_enabled()) { - ret = pci_vfio_map_resource(dev); - if (ret == 0) - mapped = 1; - else if (ret < 0) - return ret; - } + if (pci_vfio_is_enabled()) + ret = pci_vfio_map_resource(dev); #endif - /* map resources for devices that use uio_pci_generic or igb_uio */ - if (!mapped) { + break; + case RTE_PT_IGB_UIO: + case RTE_PT_UIO_GENERIC: + /* map resources for devices that use uio */ ret = pci_uio_map_resource(dev); - if (ret != 0) - return ret; + break; + default: + RTE_LOG(DEBUG, EAL, " Not managed by known pt driver," + " skipped\n"); + ret = 1; + break; } - return 0; + + return ret; } /* -- 1.9.1
To remove assumption, do like followings. This patch adds "RTE_PCI_DRV_DETACHABLE" to drv_flags of rte_pci_driver structure. The flags indicate the driver can detach devices at runtime. Also, remove assumption that port will not be detached. To remove the assumption. - Add 'attached' member to rte_eth_dev structure. This member is used for indicating the port is attached, or not. DEV_ATTACHED indicates a port is attached. DEV_DETACHED indicates a port is detached. - Add rte_eth_dev_allocate_new_port(). This function is used for allocating new port. v9: - DEV_INVALID/VALID are removed. - DEV_DISCONNECTED is replaced by DEV_DETACHED. - DEV_CONNECTED is replaced by DEV_ATTACHED. - rte_eth_dev_allocate_new_port() is renamed to rte_eth_dev_find_free_port(). - rte_eth_dev_validate_port() is renamed to rte_eth_dev_is_valid_port(). - rte_eth_dev_is_valid_port() is changed not to handle log toggle. - Fix commit log to describe DEV_ATACHED and DEV_DETACHED. (Thanks to Thomas Monjalon) v8: - NONE_TRACE is changed to NO_TRACE. (Thanks to Iremonger, Bernard) v5: - Change parameters of rte_eth_dev_validate_port() to cleanup code. v4: - Use braces with 'for' loop. - Fix indent of 'if' statement. Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp> --- lib/librte_eal/common/include/rte_pci.h | 2 + lib/librte_ether/rte_ethdev.c | 248 ++++++++++++++++++++------------ lib/librte_ether/rte_ethdev.h | 5 + 3 files changed, 164 insertions(+), 91 deletions(-) diff --git a/lib/librte_eal/common/include/rte_pci.h b/lib/librte_eal/common/include/rte_pci.h index a87b4b3..255a77b 100644 --- a/lib/librte_eal/common/include/rte_pci.h +++ b/lib/librte_eal/common/include/rte_pci.h @@ -210,6 +210,8 @@ struct rte_pci_driver { #define RTE_PCI_DRV_FORCE_UNBIND 0x0004 /** Device driver supports link state interrupt */ #define RTE_PCI_DRV_INTR_LSC 0x0008 +/** Device driver supports detaching capability */ +#define RTE_PCI_DRV_DETACHABLE 0x0010 /**< Internal use only - Macro used by pci addr parsing functions **/ #define GET_PCIADDR_FIELD(in, fd, lim, dlm) \ diff --git a/lib/librte_ether/rte_ethdev.c b/lib/librte_ether/rte_ethdev.c index ecbe93c..b702039 100644 --- a/lib/librte_ether/rte_ethdev.c +++ b/lib/librte_ether/rte_ethdev.c @@ -175,6 +175,11 @@ enum { STAT_QMAP_RX }; +enum { + DEV_DETACHED = 0, + DEV_ATTACHED +}; + static inline void rte_eth_dev_data_alloc(void) { @@ -201,19 +206,34 @@ rte_eth_dev_allocated(const char *name) { unsigned i; - for (i = 0; i < nb_ports; i++) { - if (strcmp(rte_eth_devices[i].data->name, name) == 0) + for (i = 0; i < RTE_MAX_ETHPORTS; i++) { + if ((rte_eth_devices[i].attached == DEV_ATTACHED) && + strcmp(rte_eth_devices[i].data->name, name) == 0) return &rte_eth_devices[i]; } return NULL; } +static uint8_t +rte_eth_dev_find_free_port(void) +{ + unsigned i; + + for (i = 0; i < RTE_MAX_ETHPORTS; i++) { + if (rte_eth_devices[i].attached == DEV_DETACHED) + return i; + } + return RTE_MAX_ETHPORTS; +} + struct rte_eth_dev * rte_eth_dev_allocate(const char *name) { + uint8_t port_id; struct rte_eth_dev *eth_dev; - if (nb_ports == RTE_MAX_ETHPORTS) { + port_id = rte_eth_dev_find_free_port(); + if (port_id == RTE_MAX_ETHPORTS) { PMD_DEBUG_TRACE("Reached maximum number of Ethernet ports\n"); return NULL; } @@ -226,10 +246,12 @@ rte_eth_dev_allocate(const char *name) return NULL; } - eth_dev = &rte_eth_devices[nb_ports]; - eth_dev->data = &rte_eth_dev_data[nb_ports]; + eth_dev = &rte_eth_devices[port_id]; + eth_dev->data = &rte_eth_dev_data[port_id]; snprintf(eth_dev->data->name, sizeof(eth_dev->data->name), "%s", name); - eth_dev->data->port_id = nb_ports++; + eth_dev->data->port_id = port_id; + eth_dev->attached = DEV_ATTACHED; + nb_ports++; return eth_dev; } @@ -283,6 +305,7 @@ rte_eth_dev_init(struct rte_pci_driver *pci_drv, (unsigned) pci_dev->id.device_id); if (rte_eal_process_type() == RTE_PROC_PRIMARY) rte_free(eth_dev->data->dev_private); + eth_dev->attached = DEV_DETACHED; nb_ports--; return diag; } @@ -308,10 +331,20 @@ rte_eth_driver_register(struct eth_driver *eth_drv) rte_eal_pci_register(ð_drv->pci_drv); } +static int +rte_eth_dev_is_valid_port(uint8_t port_id) +{ + if (port_id >= RTE_MAX_ETHPORTS || + rte_eth_devices[port_id].attached != DEV_ATTACHED) + return 0; + else + return 1; +} + int rte_eth_dev_socket_id(uint8_t port_id) { - if (port_id >= nb_ports) + if (!rte_eth_dev_is_valid_port(port_id)) return -1; return rte_eth_devices[port_id].pci_dev->numa_node; } @@ -395,7 +428,7 @@ rte_eth_dev_rx_queue_start(uint8_t port_id, uint16_t rx_queue_id) * in a multi-process setup*/ PROC_PRIMARY_OR_ERR_RET(-E_RTE_SECONDARY); - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return -EINVAL; } @@ -421,7 +454,7 @@ rte_eth_dev_rx_queue_stop(uint8_t port_id, uint16_t rx_queue_id) * in a multi-process setup*/ PROC_PRIMARY_OR_ERR_RET(-E_RTE_SECONDARY); - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return -EINVAL; } @@ -447,7 +480,7 @@ rte_eth_dev_tx_queue_start(uint8_t port_id, uint16_t tx_queue_id) * in a multi-process setup*/ PROC_PRIMARY_OR_ERR_RET(-E_RTE_SECONDARY); - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return -EINVAL; } @@ -473,7 +506,7 @@ rte_eth_dev_tx_queue_stop(uint8_t port_id, uint16_t tx_queue_id) * in a multi-process setup*/ PROC_PRIMARY_OR_ERR_RET(-E_RTE_SECONDARY); - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return -EINVAL; } @@ -755,10 +788,11 @@ rte_eth_dev_configure(uint8_t port_id, uint16_t nb_rx_q, uint16_t nb_tx_q, * in a multi-process setup*/ PROC_PRIMARY_OR_ERR_RET(-E_RTE_SECONDARY); - if (port_id >= nb_ports || port_id >= RTE_MAX_ETHPORTS) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-EINVAL); } + dev = &rte_eth_devices[port_id]; FUNC_PTR_OR_ERR_RET(*dev->dev_ops->dev_infos_get, -ENOTSUP); @@ -940,10 +974,11 @@ rte_eth_dev_start(uint8_t port_id) * in a multi-process setup*/ PROC_PRIMARY_OR_ERR_RET(-E_RTE_SECONDARY); - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%" PRIu8 "\n", port_id); + if (!rte_eth_dev_is_valid_port(port_id)) { + PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-EINVAL); } + dev = &rte_eth_devices[port_id]; FUNC_PTR_OR_ERR_RET(*dev->dev_ops->dev_start, -ENOTSUP); @@ -979,10 +1014,11 @@ rte_eth_dev_stop(uint8_t port_id) * in a multi-process setup*/ PROC_PRIMARY_OR_RET(); - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%" PRIu8 "\n", port_id); + if (!rte_eth_dev_is_valid_port(port_id)) { + PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return; } + dev = &rte_eth_devices[port_id]; FUNC_PTR_OR_RET(*dev->dev_ops->dev_stop); @@ -1007,10 +1043,11 @@ rte_eth_dev_set_link_up(uint8_t port_id) * in a multi-process setup*/ PROC_PRIMARY_OR_ERR_RET(-E_RTE_SECONDARY); - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return -EINVAL; } + dev = &rte_eth_devices[port_id]; FUNC_PTR_OR_ERR_RET(*dev->dev_ops->dev_set_link_up, -ENOTSUP); @@ -1026,10 +1063,11 @@ rte_eth_dev_set_link_down(uint8_t port_id) * in a multi-process setup*/ PROC_PRIMARY_OR_ERR_RET(-E_RTE_SECONDARY); - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return -EINVAL; } + dev = &rte_eth_devices[port_id]; FUNC_PTR_OR_ERR_RET(*dev->dev_ops->dev_set_link_down, -ENOTSUP); @@ -1045,7 +1083,7 @@ rte_eth_dev_close(uint8_t port_id) * in a multi-process setup*/ PROC_PRIMARY_OR_RET(); - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return; } @@ -1073,10 +1111,11 @@ rte_eth_rx_queue_setup(uint8_t port_id, uint16_t rx_queue_id, * in a multi-process setup*/ PROC_PRIMARY_OR_ERR_RET(-E_RTE_SECONDARY); - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-EINVAL); } + dev = &rte_eth_devices[port_id]; if (rx_queue_id >= dev->data->nb_rx_queues) { PMD_DEBUG_TRACE("Invalid RX queue_id=%d\n", rx_queue_id); @@ -1146,10 +1185,11 @@ rte_eth_tx_queue_setup(uint8_t port_id, uint16_t tx_queue_id, * in a multi-process setup*/ PROC_PRIMARY_OR_ERR_RET(-E_RTE_SECONDARY); - if (port_id >= RTE_MAX_ETHPORTS || port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-EINVAL); } + dev = &rte_eth_devices[port_id]; if (tx_queue_id >= dev->data->nb_tx_queues) { PMD_DEBUG_TRACE("Invalid TX queue_id=%d\n", tx_queue_id); @@ -1179,10 +1219,11 @@ rte_eth_promiscuous_enable(uint8_t port_id) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return; } + dev = &rte_eth_devices[port_id]; FUNC_PTR_OR_RET(*dev->dev_ops->promiscuous_enable); @@ -1195,10 +1236,11 @@ rte_eth_promiscuous_disable(uint8_t port_id) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return; } + dev = &rte_eth_devices[port_id]; FUNC_PTR_OR_RET(*dev->dev_ops->promiscuous_disable); @@ -1211,7 +1253,7 @@ rte_eth_promiscuous_get(uint8_t port_id) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return -1; } @@ -1225,10 +1267,11 @@ rte_eth_allmulticast_enable(uint8_t port_id) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return; } + dev = &rte_eth_devices[port_id]; FUNC_PTR_OR_RET(*dev->dev_ops->allmulticast_enable); @@ -1241,10 +1284,11 @@ rte_eth_allmulticast_disable(uint8_t port_id) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return; } + dev = &rte_eth_devices[port_id]; FUNC_PTR_OR_RET(*dev->dev_ops->allmulticast_disable); @@ -1257,7 +1301,7 @@ rte_eth_allmulticast_get(uint8_t port_id) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return -1; } @@ -1285,10 +1329,11 @@ rte_eth_link_get(uint8_t port_id, struct rte_eth_link *eth_link) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return; } + dev = &rte_eth_devices[port_id]; if (dev->data->dev_conf.intr_conf.lsc != 0) @@ -1305,10 +1350,11 @@ rte_eth_link_get_nowait(uint8_t port_id, struct rte_eth_link *eth_link) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return; } + dev = &rte_eth_devices[port_id]; if (dev->data->dev_conf.intr_conf.lsc != 0) @@ -1325,10 +1371,11 @@ rte_eth_stats_get(uint8_t port_id, struct rte_eth_stats *stats) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-ENODEV); } + dev = &rte_eth_devices[port_id]; memset(stats, 0, sizeof(*stats)); @@ -1343,10 +1390,11 @@ rte_eth_stats_reset(uint8_t port_id) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return; } + dev = &rte_eth_devices[port_id]; FUNC_PTR_OR_RET(*dev->dev_ops->stats_reset); @@ -1364,10 +1412,11 @@ rte_eth_xstats_get(uint8_t port_id, struct rte_eth_xstats *xstats, uint64_t val; char *stats_ptr; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return -1; } + dev = &rte_eth_devices[port_id]; /* implemented by the driver */ @@ -1433,10 +1482,11 @@ rte_eth_xstats_reset(uint8_t port_id) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return; } + dev = &rte_eth_devices[port_id]; /* implemented by the driver */ @@ -1455,10 +1505,11 @@ set_queue_stats_mapping(uint8_t port_id, uint16_t queue_id, uint8_t stat_idx, { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return -ENODEV; } + dev = &rte_eth_devices[port_id]; FUNC_PTR_OR_ERR_RET(*dev->dev_ops->queue_stats_mapping_set, -ENOTSUP); @@ -1490,10 +1541,11 @@ rte_eth_dev_info_get(uint8_t port_id, struct rte_eth_dev_info *dev_info) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return; } + dev = &rte_eth_devices[port_id]; memset(dev_info, 0, sizeof(struct rte_eth_dev_info)); @@ -1510,10 +1562,11 @@ rte_eth_macaddr_get(uint8_t port_id, struct ether_addr *mac_addr) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return; } + dev = &rte_eth_devices[port_id]; ether_addr_copy(&dev->data->mac_addrs[0], mac_addr); } @@ -1524,7 +1577,7 @@ rte_eth_dev_get_mtu(uint8_t port_id, uint16_t *mtu) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-ENODEV); } @@ -1540,7 +1593,7 @@ rte_eth_dev_set_mtu(uint8_t port_id, uint16_t mtu) int ret; struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-ENODEV); } @@ -1560,10 +1613,11 @@ rte_eth_dev_vlan_filter(uint8_t port_id, uint16_t vlan_id, int on) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-ENODEV); } + dev = &rte_eth_devices[port_id]; if (! (dev->data->dev_conf.rxmode.hw_vlan_filter)) { PMD_DEBUG_TRACE("port %d: vlan-filtering disabled\n", port_id); @@ -1585,7 +1639,7 @@ rte_eth_dev_set_vlan_strip_on_queue(uint8_t port_id, uint16_t rx_queue_id, int o { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-ENODEV); } @@ -1607,7 +1661,7 @@ rte_eth_dev_set_vlan_ether_type(uint8_t port_id, uint16_t tpid) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-ENODEV); } @@ -1627,7 +1681,7 @@ rte_eth_dev_set_vlan_offload(uint8_t port_id, int offload_mask) int mask = 0; int cur, org = 0; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-ENODEV); } @@ -1672,7 +1726,7 @@ rte_eth_dev_get_vlan_offload(uint8_t port_id) struct rte_eth_dev *dev; int ret = 0; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-ENODEV); } @@ -1696,10 +1750,11 @@ rte_eth_dev_set_vlan_pvid(uint8_t port_id, uint16_t pvid, int on) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-ENODEV); } + dev = &rte_eth_devices[port_id]; FUNC_PTR_OR_ERR_RET(*dev->dev_ops->vlan_pvid_set, -ENOTSUP); (*dev->dev_ops->vlan_pvid_set)(dev, pvid, on); @@ -1714,7 +1769,7 @@ rte_eth_dev_fdir_add_signature_filter(uint8_t port_id, { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-ENODEV); } @@ -1748,7 +1803,7 @@ rte_eth_dev_fdir_update_signature_filter(uint8_t port_id, { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-ENODEV); } @@ -1782,7 +1837,7 @@ rte_eth_dev_fdir_remove_signature_filter(uint8_t port_id, { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-ENODEV); } @@ -1813,7 +1868,7 @@ rte_eth_dev_fdir_get_infos(uint8_t port_id, struct rte_eth_fdir *fdir) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-ENODEV); } @@ -1838,7 +1893,7 @@ rte_eth_dev_fdir_add_perfect_filter(uint8_t port_id, { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-ENODEV); } @@ -1878,7 +1933,7 @@ rte_eth_dev_fdir_update_perfect_filter(uint8_t port_id, { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-ENODEV); } @@ -1916,7 +1971,7 @@ rte_eth_dev_fdir_remove_perfect_filter(uint8_t port_id, { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-ENODEV); } @@ -1952,7 +2007,7 @@ rte_eth_dev_fdir_set_masks(uint8_t port_id, struct rte_fdir_masks *fdir_mask) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-ENODEV); } @@ -1972,7 +2027,7 @@ rte_eth_dev_flow_ctrl_get(uint8_t port_id, struct rte_eth_fc_conf *fc_conf) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-ENODEV); } @@ -1988,7 +2043,7 @@ rte_eth_dev_flow_ctrl_set(uint8_t port_id, struct rte_eth_fc_conf *fc_conf) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-ENODEV); } @@ -2008,7 +2063,7 @@ rte_eth_dev_priority_flow_ctrl_set(uint8_t port_id, struct rte_eth_pfc_conf *pfc { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-ENODEV); } @@ -2087,7 +2142,7 @@ rte_eth_dev_rss_reta_update(uint8_t port_id, struct rte_eth_dev *dev; int ret; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return -ENODEV; } @@ -2138,10 +2193,11 @@ rte_eth_dev_rss_hash_update(uint8_t port_id, struct rte_eth_rss_conf *rss_conf) struct rte_eth_dev *dev; uint16_t rss_hash_protos; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-ENODEV); } + rss_hash_protos = rss_conf->rss_hf; if ((rss_hash_protos != 0) && ((rss_hash_protos & ETH_RSS_PROTO_MASK) == 0)) { @@ -2160,10 +2216,11 @@ rte_eth_dev_rss_hash_conf_get(uint8_t port_id, { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-ENODEV); } + dev = &rte_eth_devices[port_id]; FUNC_PTR_OR_ERR_RET(*dev->dev_ops->rss_hash_conf_get, -ENOTSUP); return (*dev->dev_ops->rss_hash_conf_get)(dev, rss_conf); @@ -2175,7 +2232,7 @@ rte_eth_dev_udp_tunnel_add(uint8_t port_id, { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return -ENODEV; } @@ -2201,10 +2258,11 @@ rte_eth_dev_udp_tunnel_delete(uint8_t port_id, { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return -ENODEV; } + dev = &rte_eth_devices[port_id]; if (udp_tunnel == NULL) { @@ -2226,7 +2284,7 @@ rte_eth_led_on(uint8_t port_id) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-ENODEV); } @@ -2241,7 +2299,7 @@ rte_eth_led_off(uint8_t port_id) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-ENODEV); } @@ -2281,10 +2339,11 @@ rte_eth_dev_mac_addr_add(uint8_t port_id, struct ether_addr *addr, int index; uint64_t pool_mask; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-ENODEV); } + dev = &rte_eth_devices[port_id]; FUNC_PTR_OR_ERR_RET(*dev->dev_ops->mac_addr_add, -ENOTSUP); @@ -2332,10 +2391,11 @@ rte_eth_dev_mac_addr_remove(uint8_t port_id, struct ether_addr *addr) struct rte_eth_dev *dev; int index; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-ENODEV); } + dev = &rte_eth_devices[port_id]; FUNC_PTR_OR_ERR_RET(*dev->dev_ops->mac_addr_remove, -ENOTSUP); @@ -2366,7 +2426,7 @@ rte_eth_dev_set_vf_rxmode(uint8_t port_id, uint16_t vf, struct rte_eth_dev *dev; struct rte_eth_dev_info dev_info; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("set VF RX mode:Invalid port_id=%d\n", port_id); return (-ENODEV); @@ -2421,7 +2481,7 @@ rte_eth_dev_uc_hash_table_set(uint8_t port_id, struct ether_addr *addr, int ret; struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("unicast hash setting:Invalid port_id=%d\n", port_id); return (-ENODEV); @@ -2474,7 +2534,7 @@ rte_eth_dev_uc_all_hash_table_set(uint8_t port_id, uint8_t on) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("unicast hash setting:Invalid port_id=%d\n", port_id); return (-ENODEV); @@ -2493,7 +2553,7 @@ rte_eth_dev_set_vf_rx(uint8_t port_id,uint16_t vf, uint8_t on) struct rte_eth_dev *dev; struct rte_eth_dev_info dev_info; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-ENODEV); } @@ -2519,7 +2579,7 @@ rte_eth_dev_set_vf_tx(uint8_t port_id,uint16_t vf, uint8_t on) struct rte_eth_dev *dev; struct rte_eth_dev_info dev_info; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("set pool tx:Invalid port_id=%d\n", port_id); return (-ENODEV); } @@ -2544,7 +2604,7 @@ rte_eth_dev_set_vf_vlan_filter(uint8_t port_id, uint16_t vlan_id, { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("VF VLAN filter:invalid port id=%d\n", port_id); return (-ENODEV); @@ -2575,7 +2635,7 @@ int rte_eth_set_queue_rate_limit(uint8_t port_id, uint16_t queue_idx, struct rte_eth_dev_info dev_info; struct rte_eth_link link; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("set queue rate limit:invalid port id=%d\n", port_id); return -ENODEV; @@ -2612,7 +2672,7 @@ int rte_eth_set_vf_rate_limit(uint8_t port_id, uint16_t vf, uint16_t tx_rate, if (q_msk == 0) return 0; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("set VF rate limit:invalid port id=%d\n", port_id); return -ENODEV; @@ -2646,7 +2706,7 @@ rte_eth_mirror_rule_set(uint8_t port_id, { struct rte_eth_dev *dev = &rte_eth_devices[port_id]; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-ENODEV); } @@ -2687,7 +2747,7 @@ rte_eth_mirror_rule_reset(uint8_t port_id, uint8_t rule_id) { struct rte_eth_dev *dev = &rte_eth_devices[port_id]; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-ENODEV); } @@ -2712,10 +2772,11 @@ rte_eth_rx_burst(uint8_t port_id, uint16_t queue_id, { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return 0; } + dev = &rte_eth_devices[port_id]; FUNC_PTR_OR_ERR_RET(*dev->rx_pkt_burst, 0); if (queue_id >= dev->data->nb_rx_queues) { @@ -2732,10 +2793,11 @@ rte_eth_tx_burst(uint8_t port_id, uint16_t queue_id, { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return 0; } + dev = &rte_eth_devices[port_id]; FUNC_PTR_OR_ERR_RET(*dev->tx_pkt_burst, 0); @@ -2752,10 +2814,11 @@ rte_eth_rx_queue_count(uint8_t port_id, uint16_t queue_id) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return 0; } + dev = &rte_eth_devices[port_id]; FUNC_PTR_OR_ERR_RET(*dev->dev_ops->rx_queue_count, 0); return (*dev->dev_ops->rx_queue_count)(dev, queue_id); @@ -2766,10 +2829,11 @@ rte_eth_rx_descriptor_done(uint8_t port_id, uint16_t queue_id, uint16_t offset) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-ENODEV); } + dev = &rte_eth_devices[port_id]; FUNC_PTR_OR_ERR_RET(*dev->dev_ops->rx_descriptor_done, -ENOTSUP); return (*dev->dev_ops->rx_descriptor_done)( \ @@ -2787,7 +2851,8 @@ rte_eth_dev_callback_register(uint8_t port_id, if (!cb_fn) return (-EINVAL); - if (port_id >= nb_ports) { + + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-EINVAL); } @@ -2827,7 +2892,8 @@ rte_eth_dev_callback_unregister(uint8_t port_id, if (!cb_fn) return (-EINVAL); - if (port_id >= nb_ports) { + + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-EINVAL); } @@ -2887,7 +2953,7 @@ int rte_eth_dev_bypass_init(uint8_t port_id) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-ENODEV); } @@ -2907,7 +2973,7 @@ rte_eth_dev_bypass_state_show(uint8_t port_id, uint32_t *state) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-ENODEV); } @@ -2926,7 +2992,7 @@ rte_eth_dev_bypass_state_set(uint8_t port_id, uint32_t *new_state) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-ENODEV); } @@ -2946,7 +3012,7 @@ rte_eth_dev_bypass_event_show(uint8_t port_id, uint32_t event, uint32_t *state) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-ENODEV); } @@ -2966,7 +3032,7 @@ rte_eth_dev_bypass_event_store(uint8_t port_id, uint32_t event, uint32_t state) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-ENODEV); } @@ -2986,7 +3052,7 @@ rte_eth_dev_wd_timeout_store(uint8_t port_id, uint32_t timeout) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-ENODEV); } @@ -3006,7 +3072,7 @@ rte_eth_dev_bypass_ver_show(uint8_t port_id, uint32_t *ver) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-ENODEV); } @@ -3026,7 +3092,7 @@ rte_eth_dev_bypass_wd_timeout_show(uint8_t port_id, uint32_t *wd_timeout) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-ENODEV); } @@ -3046,7 +3112,7 @@ rte_eth_dev_bypass_wd_reset(uint8_t port_id) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-ENODEV); } @@ -3067,7 +3133,7 @@ rte_eth_dev_filter_supported(uint8_t port_id, enum rte_filter_type filter_type) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return -ENODEV; } @@ -3084,7 +3150,7 @@ rte_eth_dev_filter_ctrl(uint8_t port_id, enum rte_filter_type filter_type, { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return -ENODEV; } diff --git a/lib/librte_ether/rte_ethdev.h b/lib/librte_ether/rte_ethdev.h index cce1a53..110ddba 100644 --- a/lib/librte_ether/rte_ethdev.h +++ b/lib/librte_ether/rte_ethdev.h @@ -1451,6 +1451,7 @@ struct rte_eth_dev { * received packets before passing them to the driver for transmission. */ struct rte_eth_rxtx_callback **pre_tx_burst_cbs; + uint8_t attached; /**< Flag indicating the port is attached */ }; struct rte_eth_dev_sriov { @@ -1516,6 +1517,10 @@ extern struct rte_eth_dev rte_eth_devices[]; * initialized by the [matching] Ethernet driver during the PCI probing phase. * All devices whose port identifier is in the range * [0, rte_eth_dev_count() - 1] can be operated on by network applications. + * immediately after invoking rte_eal_init(). + * If the application unplugs a port using hotplug function, The enabled port + * numbers may be noncontiguous. In the case, the applications need to manage + * enabled port by themselves. * * @return * - The total number of usable Ethernet devices. -- 1.9.1
This patch replaces pci_addr_comparison() and memcmp() of pci addresses by rte_eal_compare_pci_addr(). To compare PCI addresses, rte_eal_compare_pci_addr() doesn't use memcmp(). This is because sizeof(struct rte_pci_addr) returns 6, but actually this structure is like below. struct rte_pci_addr { uint16_t domain; /**< Device domain */ uint8_t bus; /**< Device bus */ uint8_t devid; /**< Device ID */ uint8_t function; /**< Device function. */ }; If the structure is dynamically allocated in a function without bzero, last 1 byte may have value. As a result, memcmp may not work. To avoid such a case, rte_eal_compare_pci_addr() compare following values. dev_addr = (addr->domain << 24) | (addr->bus << 16) | (addr->devid << 8) | addr->function; v9: - eal_compare_pci_addr() is replaced by rte_eal_compare_pci_addr(). - Fix commit log. (Thanks to Thomas Monjalon) v8: - Fix pci_scan_one() to update sysfs values. (Thanks to Qiu, Michael and Iremonger, Bernard) v5: - Fix pci_scan_one to handle pt_driver correctly. v4: - Fix calculation method of eal_compare_pci_addr(). - Add parameter checking. Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp> --- lib/librte_eal/bsdapp/eal/eal_pci.c | 29 ++++++++++++-------------- lib/librte_eal/common/eal_common_pci.c | 2 +- lib/librte_eal/common/include/rte_pci.h | 34 +++++++++++++++++++++++++++++++ lib/librte_eal/linuxapp/eal/eal_pci.c | 30 +++++++++++++-------------- lib/librte_eal/linuxapp/eal/eal_pci_uio.c | 2 +- 5 files changed, 63 insertions(+), 34 deletions(-) diff --git a/lib/librte_eal/bsdapp/eal/eal_pci.c b/lib/librte_eal/bsdapp/eal/eal_pci.c index 74ecce7..9193f80 100644 --- a/lib/librte_eal/bsdapp/eal/eal_pci.c +++ b/lib/librte_eal/bsdapp/eal/eal_pci.c @@ -270,20 +270,6 @@ pci_uio_map_resource(struct rte_pci_device *dev) return (0); } -/* Compare two PCI device addresses. */ -static int -pci_addr_comparison(struct rte_pci_addr *addr, struct rte_pci_addr *addr2) -{ - uint64_t dev_addr = (addr->domain << 24) + (addr->bus << 16) + (addr->devid << 8) + addr->function; - uint64_t dev_addr2 = (addr2->domain << 24) + (addr2->bus << 16) + (addr2->devid << 8) + addr2->function; - - if (dev_addr > dev_addr2) - return 1; - else - return 0; -} - - /* Scan one pci sysfs entry, and fill the devices list from it. */ static int pci_scan_one(int dev_pci_fd, struct pci_conf *conf) @@ -356,13 +342,24 @@ pci_scan_one(int dev_pci_fd, struct pci_conf *conf) } else { struct rte_pci_device *dev2 = NULL; + int ret; TAILQ_FOREACH(dev2, &pci_device_list, next) { - if (pci_addr_comparison(&dev->addr, &dev2->addr)) + ret = rte_eal_compare_pci_addr(&dev->addr, &dev2->addr); + if (ret > 0) continue; - else { + else if (ret < 0) { TAILQ_INSERT_BEFORE(dev2, dev, next); return 0; + } else { /* already registered */ + /* update pt_driver */ + dev2->pt_driver = dev->pt_driver; + dev2->max_vfs = dev->max_vfs; + memmove(dev2->mem_resource, + dev->mem_resource, + sizeof(dev->mem_resource)); + free(dev); + return 0; } } TAILQ_INSERT_TAIL(&pci_device_list, dev, next); diff --git a/lib/librte_eal/common/eal_common_pci.c b/lib/librte_eal/common/eal_common_pci.c index f3c7f71..bf2793f 100644 --- a/lib/librte_eal/common/eal_common_pci.c +++ b/lib/librte_eal/common/eal_common_pci.c @@ -93,7 +93,7 @@ static struct rte_devargs *pci_devargs_lookup(struct rte_pci_device *dev) if (devargs->type != RTE_DEVTYPE_BLACKLISTED_PCI && devargs->type != RTE_DEVTYPE_WHITELISTED_PCI) continue; - if (!memcmp(&dev->addr, &devargs->pci.addr, sizeof(dev->addr))) + if (!rte_eal_compare_pci_addr(&dev->addr, &devargs->pci.addr)) return devargs; } return NULL; diff --git a/lib/librte_eal/common/include/rte_pci.h b/lib/librte_eal/common/include/rte_pci.h index 255a77b..dcf9c81 100644 --- a/lib/librte_eal/common/include/rte_pci.h +++ b/lib/librte_eal/common/include/rte_pci.h @@ -272,6 +272,40 @@ eal_parse_pci_DomBDF(const char *input, struct rte_pci_addr *dev_addr) } #undef GET_PCIADDR_FIELD +/* Compare two PCI device addresses. */ +/** + * Utility function to compare two PCI device addresses. + * + * @param addr + * The PCI Bus-Device-Function address to compare + * @param addr2 + * The PCI Bus-Device-Function address to compare + * @return + * 0 on equal PCI address. + * Positive on addr is greater than addr2. + * Negative on addr is less than addr2, or error. + */ +static inline int +rte_eal_compare_pci_addr(struct rte_pci_addr *addr, struct rte_pci_addr *addr2) +{ + uint64_t dev_addr, dev_addr2; + + if ((addr == NULL) || (addr2 == NULL)) + return -1; + + dev_addr = (addr->domain << 24) | (addr->bus << 16) | + (addr->devid << 8) | addr->function; + dev_addr2 = (addr2->domain << 24) | (addr2->bus << 16) | + (addr2->devid << 8) | addr2->function; + + if (dev_addr > dev_addr2) + return 1; + else if (dev_addr < dev_addr2) + return -1; + else + return 0; +} + /** * Probe the PCI bus for registered drivers. * diff --git a/lib/librte_eal/linuxapp/eal/eal_pci.c b/lib/librte_eal/linuxapp/eal/eal_pci.c index 3291c68..06bfc1a 100644 --- a/lib/librte_eal/linuxapp/eal/eal_pci.c +++ b/lib/librte_eal/linuxapp/eal/eal_pci.c @@ -41,6 +41,7 @@ #include <rte_eal_memconfig.h> #include <rte_malloc.h> #include <rte_devargs.h> +#include <rte_memcpy.h> #include "rte_pci_dev_ids.h" #include "eal_filesystem.h" @@ -228,20 +229,6 @@ error: return -1; } -/* Compare two PCI device addresses. */ -static int -pci_addr_comparison(struct rte_pci_addr *addr, struct rte_pci_addr *addr2) -{ - uint64_t dev_addr = (addr->domain << 24) + (addr->bus << 16) + (addr->devid << 8) + addr->function; - uint64_t dev_addr2 = (addr2->domain << 24) + (addr2->bus << 16) + (addr2->devid << 8) + addr2->function; - - if (dev_addr > dev_addr2) - return 1; - else - return 0; -} - - /* Scan one pci sysfs entry, and fill the devices list from it. */ static int pci_scan_one(const char *dirname, uint16_t domain, uint8_t bus, @@ -359,13 +346,24 @@ pci_scan_one(const char *dirname, uint16_t domain, uint8_t bus, } else { struct rte_pci_device *dev2 = NULL; + int ret; TAILQ_FOREACH(dev2, &pci_device_list, next) { - if (pci_addr_comparison(&dev->addr, &dev2->addr)) + ret = rte_eal_compare_pci_addr(&dev->addr, &dev2->addr); + if (ret > 0) continue; - else { + else if (ret < 0) { TAILQ_INSERT_BEFORE(dev2, dev, next); return 0; + } else { /* already registered */ + /* update pt_driver */ + dev2->pt_driver = dev->pt_driver; + dev2->max_vfs = dev->max_vfs; + memmove(dev2->mem_resource, + dev->mem_resource, + sizeof(dev->mem_resource)); + free(dev); + return 0; } } TAILQ_INSERT_TAIL(&pci_device_list, dev, next); diff --git a/lib/librte_eal/linuxapp/eal/eal_pci_uio.c b/lib/librte_eal/linuxapp/eal/eal_pci_uio.c index 0a95376..c5e0cf3 100644 --- a/lib/librte_eal/linuxapp/eal/eal_pci_uio.c +++ b/lib/librte_eal/linuxapp/eal/eal_pci_uio.c @@ -91,7 +91,7 @@ pci_uio_map_secondary(struct rte_pci_device *dev) TAILQ_FOREACH(uio_res, pci_res_list, next) { /* skip this element if it doesn't match our PCI address */ - if (memcmp(&uio_res->pci_addr, &dev->addr, sizeof(dev->addr))) + if (rte_eal_compare_pci_addr(&uio_res->pci_addr, &dev->addr)) continue; for (i = 0; i != uio_res->nb_maps; i++) { -- 1.9.1
This patch adds rte_eth_dev_release_port(). The function is used for changing an attached status of the device that has specified name. v9: - rte_eth_dev_free() is replaced by rte_eth_dev_release_port(). (Thanks to Thomas Monjalon) v6: - Use rte_eth_dev structure as the paramter of rte_eth_dev_free(). v4: - Add parameter checking. Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp> --- lib/librte_ether/rte_ethdev.c | 11 +++++++++++ lib/librte_ether/rte_ethdev.h | 12 ++++++++++++ 2 files changed, 23 insertions(+) diff --git a/lib/librte_ether/rte_ethdev.c b/lib/librte_ether/rte_ethdev.c index b702039..a089557 100644 --- a/lib/librte_ether/rte_ethdev.c +++ b/lib/librte_ether/rte_ethdev.c @@ -255,6 +255,17 @@ rte_eth_dev_allocate(const char *name) return eth_dev; } +int +rte_eth_dev_release_port(struct rte_eth_dev *eth_dev) +{ + if (eth_dev == NULL) + return -EINVAL; + + eth_dev->attached = 0; + nb_ports--; + return 0; +} + static int rte_eth_dev_init(struct rte_pci_driver *pci_drv, struct rte_pci_device *pci_dev) diff --git a/lib/librte_ether/rte_ethdev.h b/lib/librte_ether/rte_ethdev.h index 110ddba..7963e56 100644 --- a/lib/librte_ether/rte_ethdev.h +++ b/lib/librte_ether/rte_ethdev.h @@ -1539,6 +1539,18 @@ extern uint8_t rte_eth_dev_count(void); */ struct rte_eth_dev *rte_eth_dev_allocate(const char *name); +/** + * Function for internal use by dummy drivers primarily, e.g. ring-based + * driver. + * Release the specified ethdev port. + * + * @param eth_dev + * The *eth_dev* pointer is the address of the *rte_eth_dev* structure. + * @return + * - 0 on success, negative on error + */ +int rte_eth_dev_release_port(struct rte_eth_dev *eth_dev); + struct eth_driver; /** * @internal -- 1.9.1
The patch adds function pointer to rte_pci_driver and eth_driver structure. These function pointers are used when ports are detached. Also, the patch adds rte_eth_dev_uninit(). So far, it's not called by anywhere, but it will be called when port hotplug function is implemented. v10: - Add size parameter to rte_eth_dev_create_unique_device_name(). (Thanks to Iremonger, Bernard) v9: - Change parameter of pci_devuninit_t and rte_eth_dev_uninit. - Remove code that initiaize callback of ethdev from rte_eth_dev_uninit(). - Add a function to create a unique device name. (Thanks to Thomas Monjalon) v6: - Fix rte_eth_dev_uninit() to handle a return value of uninit function of PMD. v4: - Add parameter checking. - Change function names. Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp> --- lib/librte_eal/common/include/rte_pci.h | 6 ++++ lib/librte_ether/rte_ethdev.c | 64 +++++++++++++++++++++++++++++++-- lib/librte_ether/rte_ethdev.h | 24 +++++++++++++ 3 files changed, 92 insertions(+), 2 deletions(-) diff --git a/lib/librte_eal/common/include/rte_pci.h b/lib/librte_eal/common/include/rte_pci.h index dcf9c81..ecde36f 100644 --- a/lib/librte_eal/common/include/rte_pci.h +++ b/lib/librte_eal/common/include/rte_pci.h @@ -192,12 +192,18 @@ struct rte_pci_driver; typedef int (pci_devinit_t)(struct rte_pci_driver *, struct rte_pci_device *); /** + * Uninitialisation function for the driver called during hotplugging. + */ +typedef int (pci_devuninit_t)(struct rte_pci_device *); + +/** * A structure describing a PCI driver. */ struct rte_pci_driver { TAILQ_ENTRY(rte_pci_driver) next; /**< Next in list. */ const char *name; /**< Driver name. */ pci_devinit_t *devinit; /**< Device init. function. */ + pci_devuninit_t *devuninit; /**< Device uninit function. */ struct rte_pci_id *id_table; /**< ID table, NULL terminated. */ uint32_t drv_flags; /**< Flags contolling handling of device. */ }; diff --git a/lib/librte_ether/rte_ethdev.c b/lib/librte_ether/rte_ethdev.c index a089557..165ec74 100644 --- a/lib/librte_ether/rte_ethdev.c +++ b/lib/librte_ether/rte_ethdev.c @@ -266,6 +266,24 @@ rte_eth_dev_release_port(struct rte_eth_dev *eth_dev) return 0; } +static inline int +rte_eth_dev_create_unique_device_name(char *name, size_t size, + struct rte_pci_device *pci_dev) +{ + int ret; + + if ((name == NULL) || (pci_dev == NULL)) + return -EINVAL; + + ret = snprintf(name, size, "%d:%d.%d", + pci_dev->addr.bus, pci_dev->addr.devid, + pci_dev->addr.function); + if (ret < 0) + return ret; + + return 0; +} + static int rte_eth_dev_init(struct rte_pci_driver *pci_drv, struct rte_pci_device *pci_dev) @@ -279,8 +297,8 @@ rte_eth_dev_init(struct rte_pci_driver *pci_drv, eth_drv = (struct eth_driver *)pci_drv; /* Create unique Ethernet device name using PCI address */ - snprintf(ethdev_name, RTE_ETH_NAME_MAX_LEN, "%d:%d.%d", - pci_dev->addr.bus, pci_dev->addr.devid, pci_dev->addr.function); + rte_eth_dev_create_unique_device_name(ethdev_name, + sizeof(ethdev_name), pci_dev); eth_dev = rte_eth_dev_allocate(ethdev_name); if (eth_dev == NULL) @@ -321,6 +339,47 @@ rte_eth_dev_init(struct rte_pci_driver *pci_drv, return diag; } +static int +rte_eth_dev_uninit(struct rte_pci_device *pci_dev) +{ + const struct eth_driver *eth_drv; + struct rte_eth_dev *eth_dev; + char ethdev_name[RTE_ETH_NAME_MAX_LEN]; + int ret; + + if (pci_dev == NULL) + return -EINVAL; + + /* Create unique Ethernet device name using PCI address */ + rte_eth_dev_create_unique_device_name(ethdev_name, + sizeof(ethdev_name), pci_dev); + + eth_dev = rte_eth_dev_allocated(ethdev_name); + if (eth_dev == NULL) + return -ENODEV; + + eth_drv = (const struct eth_driver *)pci_dev->driver; + + /* Invoke PMD device uninit function */ + if (*eth_drv->eth_dev_uninit) { + ret = (*eth_drv->eth_dev_uninit)(eth_drv, eth_dev); + if (ret) + return ret; + } + + /* free ether device */ + rte_eth_dev_release_port(eth_dev); + + if (rte_eal_process_type() == RTE_PROC_PRIMARY) + rte_free(eth_dev->data->dev_private); + + eth_dev->pci_dev = NULL; + eth_dev->driver = NULL; + eth_dev->data = NULL; + + return 0; +} + /** * Register an Ethernet [Poll Mode] driver. * @@ -339,6 +398,7 @@ void rte_eth_driver_register(struct eth_driver *eth_drv) { eth_drv->pci_drv.devinit = rte_eth_dev_init; + eth_drv->pci_drv.devuninit = rte_eth_dev_uninit; rte_eal_pci_register(ð_drv->pci_drv); } diff --git a/lib/librte_ether/rte_ethdev.h b/lib/librte_ether/rte_ethdev.h index 7963e56..37c3765 100644 --- a/lib/librte_ether/rte_ethdev.h +++ b/lib/librte_ether/rte_ethdev.h @@ -1588,6 +1588,27 @@ typedef int (*eth_dev_init_t)(struct eth_driver *eth_drv, /** * @internal + * Finalization function of an Ethernet driver invoked for each matching + * Ethernet PCI device detected during the PCI closing phase. + * + * @param eth_drv + * The pointer to the [matching] Ethernet driver structure supplied by + * the PMD when it registered itself. + * @param eth_dev + * The *eth_dev* pointer is the address of the *rte_eth_dev* structure + * associated with the matching device and which have been [automatically] + * allocated in the *rte_eth_devices* array. + * @return + * - 0: Success, the device is properly finalized by the driver. + * In particular, the driver MUST free the *dev_ops* pointer + * of the *eth_dev* structure. + * - <0: Error code of the device initialization failure. + */ +typedef int (*eth_dev_uninit_t)(const struct eth_driver *eth_drv, + struct rte_eth_dev *eth_dev); + +/** + * @internal * The structure associated with a PMD Ethernet driver. * * Each Ethernet driver acts as a PCI driver and is represented by a generic @@ -1597,11 +1618,14 @@ typedef int (*eth_dev_init_t)(struct eth_driver *eth_drv, * * - The *eth_dev_init* function invoked for each matching PCI device. * + * - The *eth_dev_uninit* function invoked for each matching PCI device. + * * - The size of the private data to allocate for each matching device. */ struct eth_driver { struct rte_pci_driver pci_drv; /**< The PMD is also a PCI driver. */ eth_dev_init_t eth_dev_init; /**< Device init function. */ + eth_dev_uninit_t eth_dev_uninit; /**< Device uninit function. */ unsigned int dev_private_size; /**< Size of device private data. */ }; -- 1.9.1
The patch adds following functions. - rte_eth_dev_save() The function is used for saving current rte_eth_dev structures. - rte_eth_dev_get_changed_port() The function receives the rte_eth_dev structures, then compare these with current values to know which port is actually attached or detached. - rte_eth_dev_get_addr_by_port() The function returns a pci address of an ethdev specified by port identifier. - rte_eth_dev_get_port_by_addr() The function returns a port identifier of an ethdev specified by pci address. - rte_eth_dev_get_name_by_port() The function returns a unique identifier name of an ethdev specified by port identifier. - Add rte_eth_dev_is_detachable() The function returns whether a PMD supports detach function. Also, the patch changes scope of rte_eth_dev_allocated() to global. This function will be called by virtual PMDs to support port hotplug. So change scope of the function to global. v10: - Change order of version.map. (Thanks to Thomas Monjalon) v9: - rte_eth_dev_check_detachable() is replaced by rte_eth_dev_is_detachable(). - strncpy() is replaced by strcpy(). (Thanks to Thomas Monjalon) - Add missing symbol in version map. (Thanks to Nail Horman) v8: - Add size parameter to rte_eth_dev_save(). - Add missing symbol in version map. (Thanks to Qiu, Michael and Iremonger, Bernard) v7: - Add pt_driver checking to rte_eth_dev_check_detachable(). (Thanks to Qiu, Michael) v5: - Fix return value of below functions. rte_eth_dev_get_changed_port(). rte_eth_dev_get_port_by_addr(). v4: - Add parameter checking. v3: - Fix if-condition bug while comparing pci addresses. - Add error checking codes. Reported-by: Mark Enright <menrigh@brocade.com> Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp> --- lib/librte_ether/rte_ethdev.c | 103 ++++++++++++++++++++++++++++++++- lib/librte_ether/rte_ethdev.h | 83 ++++++++++++++++++++++++++ lib/librte_ether/rte_ether_version.map | 7 +++ 3 files changed, 192 insertions(+), 1 deletion(-) diff --git a/lib/librte_ether/rte_ethdev.c b/lib/librte_ether/rte_ethdev.c index 165ec74..1f6a066 100644 --- a/lib/librte_ether/rte_ethdev.c +++ b/lib/librte_ether/rte_ethdev.c @@ -201,7 +201,7 @@ rte_eth_dev_data_alloc(void) RTE_MAX_ETHPORTS * sizeof(*rte_eth_dev_data)); } -static struct rte_eth_dev * +struct rte_eth_dev * rte_eth_dev_allocated(const char *name) { unsigned i; @@ -426,6 +426,107 @@ rte_eth_dev_count(void) return (nb_ports); } +int +rte_eth_dev_save(struct rte_eth_dev *devs, size_t size) +{ + if ((devs == NULL) || + (size != sizeof(struct rte_eth_dev) * RTE_MAX_ETHPORTS)) + return -EINVAL; + + /* save current rte_eth_devices */ + memcpy(devs, rte_eth_devices, size); + return 0; +} + +int +rte_eth_dev_get_changed_port(struct rte_eth_dev *devs, uint8_t *port_id) +{ + if ((devs == NULL) || (port_id == NULL)) + return -EINVAL; + + /* check which port was attached or detached */ + for (*port_id = 0; *port_id < RTE_MAX_ETHPORTS; (*port_id)++, devs++) { + if (rte_eth_devices[*port_id].attached ^ devs->attached) + return 0; + } + return -ENODEV; +} + +int +rte_eth_dev_get_addr_by_port(uint8_t port_id, struct rte_pci_addr *addr) +{ + if (!rte_eth_dev_is_valid_port(port_id)) { + PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + return -EINVAL; + } + + if (addr == NULL) { + PMD_DEBUG_TRACE("Null pointer is specified\n"); + return -EINVAL; + } + + *addr = rte_eth_devices[port_id].pci_dev->addr; + return 0; +} + +int +rte_eth_dev_get_port_by_addr(struct rte_pci_addr *addr, uint8_t *port_id) +{ + struct rte_pci_addr *tmp; + + if ((addr == NULL) || (port_id == NULL)) { + PMD_DEBUG_TRACE("Null pointer is specified\n"); + return -EINVAL; + } + + for (*port_id = 0; *port_id < RTE_MAX_ETHPORTS; (*port_id)++) { + if (!rte_eth_devices[*port_id].attached) + continue; + if (!rte_eth_devices[*port_id].pci_dev) + continue; + tmp = &rte_eth_devices[*port_id].pci_dev->addr; + if (rte_eal_compare_pci_addr(tmp, addr) == 0) + return 0; + } + return -ENODEV; +} + +int +rte_eth_dev_get_name_by_port(uint8_t port_id, char *name) +{ + char *tmp; + + if (!rte_eth_dev_is_valid_port(port_id)) { + PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + return -EINVAL; + } + + if (name == NULL) { + PMD_DEBUG_TRACE("Null pointer is specified\n"); + return -EINVAL; + } + + /* shouldn't check 'rte_eth_devices[i].data', + * because it might be overwritten by VDEV PMD */ + tmp = rte_eth_dev_data[port_id].name; + strcpy(name, tmp); + return 0; +} + +int +rte_eth_dev_is_detachable(uint8_t port_id) +{ + uint32_t drv_flags; + + if (port_id >= RTE_MAX_ETHPORTS) { + PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + return -EINVAL; + } + + drv_flags = rte_eth_devices[port_id].driver->pci_drv.drv_flags; + return !(drv_flags & RTE_PCI_DRV_DETACHABLE); +} + static int rte_eth_dev_rx_queue_config(struct rte_eth_dev *dev, uint16_t nb_queues) { diff --git a/lib/librte_ether/rte_ethdev.h b/lib/librte_ether/rte_ethdev.h index 37c3765..5519ce0 100644 --- a/lib/librte_ether/rte_ethdev.h +++ b/lib/librte_ether/rte_ethdev.h @@ -1528,6 +1528,89 @@ extern struct rte_eth_dev rte_eth_devices[]; extern uint8_t rte_eth_dev_count(void); /** + * Function for internal use by port hotplug functions. + * Copies current ethdev structures to the specified pointer. + * + * @param devs The pointer to the ethdev structures + * @param size The size of ethdev structures + * @return + * - 0 on success, negative on error + */ +extern int rte_eth_dev_save(struct rte_eth_dev *devs, size_t size); + +/** + * Function for internal use by port hotplug functions. + * Compare the specified ethdev structures with currents. Then + * if there is a port which status is changed, fill the specified pointer + * with the port id of that port. + * @param devs The pointer to the ethdev structures + * @param port_id The pointer to the port id + * @return + * - 0 on success, negative on error + */ +extern int rte_eth_dev_get_changed_port( + struct rte_eth_dev *devs, uint8_t *port_id); + +/** + * Function for internal use by port hotplug functions. + * Returns a pci address of a ethdev specified by port identifier. + * @param port_id + * The port identifier of the Ethernet device + * @param addr + * The pointer to the pci address + * @return + * - 0 on success, negative on error + */ +extern int rte_eth_dev_get_addr_by_port( + uint8_t port_id, struct rte_pci_addr *addr); + +/** + * Function for internal use by port hotplug functions. + * Returns a port identifier of a ethdev specified by pci address. + * @param addr + * The pointer to the pci address of the Ethernet device. + * @param port_id + * The pointer to the port identifier + * @return + * - 0 on success, negative on error + */ +extern int rte_eth_dev_get_port_by_addr( + struct rte_pci_addr *addr, uint8_t *port_id); + +/** + * Function for internal use by port hotplug functions. + * Returns a unique identifier name of a ethdev specified by port identifier. + * @param port_id + * The port identifier. + * @param name + * The pointer to the Unique identifier name for each Ethernet device + * @return + * - 0 on success, negative on error + */ +extern int rte_eth_dev_get_name_by_port(uint8_t port_id, char *name); + +/** + * Function for internal use by port hotplug functions. + * Check whether or not, a PMD that is handling the ethdev specified by port + * identifier can support detach function. + * @param port_id + * The port identifier + * @return + * - 0 on supporting detach function, negative on not supporting + */ +extern int rte_eth_dev_is_detachable(uint8_t port_id); + +/** + * Function for internal use by port hotplug functions. + * Returns a ethdev slot specified by the unique identifier name. + * @param name + * The pointer to the Unique identifier name for each Ethernet device + * @return + * - The pointer to the ethdev slot, on success. NULL on error + */ +extern struct rte_eth_dev *rte_eth_dev_allocated(const char *name); + +/** * Function for internal use by dummy drivers primarily, e.g. ring-based * driver. * Allocates a new ethdev slot for an ethernet device and returns the pointer diff --git a/lib/librte_ether/rte_ether_version.map b/lib/librte_ether/rte_ether_version.map index 94fd685..7db7085 100644 --- a/lib/librte_ether/rte_ether_version.map +++ b/lib/librte_ether/rte_ether_version.map @@ -8,6 +8,7 @@ DPDK_2.0 { rte_eth_allmulticast_enable; rte_eth_allmulticast_get; rte_eth_dev_allocate; + rte_eth_dev_allocated; rte_eth_dev_bypass_event_show; rte_eth_dev_bypass_event_store; rte_eth_dev_bypass_init; @@ -34,9 +35,14 @@ DPDK_2.0 { rte_eth_dev_filter_supported; rte_eth_dev_flow_ctrl_get; rte_eth_dev_flow_ctrl_set; + rte_eth_dev_get_addr_by_port; + rte_eth_dev_get_changed_port; rte_eth_dev_get_mtu; + rte_eth_dev_get_name_by_port; + rte_eth_dev_get_port_by_addr; rte_eth_dev_get_vlan_offload; rte_eth_dev_info_get; + rte_eth_dev_is_detachable; rte_eth_dev_mac_addr_add; rte_eth_dev_mac_addr_remove; rte_eth_dev_priority_flow_ctrl_set; @@ -46,6 +52,7 @@ DPDK_2.0 { rte_eth_dev_rss_reta_update; rte_eth_dev_rx_queue_start; rte_eth_dev_rx_queue_stop; + rte_eth_dev_save; rte_eth_dev_set_link_down; rte_eth_dev_set_link_up; rte_eth_dev_set_mtu; -- 1.9.1
The patch adds functions for unmapping igb_uio resources. The patch is only for Linux and igb_uio environment. VFIO and BSD are not supported. v9: - Remove "rte_dev_hotplug.h". - Remove needless "#ifdef". (Thanks to Thomas Monjalon and Neil Horman) - Remove pci_unmap_device(). It will be implemented in later patch. v8: - Fix typo. (Thanks to Iremonger, Bernard) v5: - Fix pci_unmap_device() to check pt_driver. v4: - Add parameter checking. - Add header file to determine if hotplug can be enabled. Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp> --- lib/librte_eal/linuxapp/eal/eal_pci.c | 17 ++++++++ lib/librte_eal/linuxapp/eal/eal_pci_init.h | 7 ++++ lib/librte_eal/linuxapp/eal/eal_pci_uio.c | 65 ++++++++++++++++++++++++++++++ 3 files changed, 89 insertions(+) diff --git a/lib/librte_eal/linuxapp/eal/eal_pci.c b/lib/librte_eal/linuxapp/eal/eal_pci.c index 06bfc1a..d03429c 100644 --- a/lib/librte_eal/linuxapp/eal/eal_pci.c +++ b/lib/librte_eal/linuxapp/eal/eal_pci.c @@ -168,6 +168,23 @@ pci_map_resource(void *requested_addr, int fd, off_t offset, size_t size, return mapaddr; } +/* unmap a particular resource */ +void +pci_unmap_resource(void *requested_addr, size_t size) +{ + if (requested_addr == NULL) + return; + + /* Unmap the PCI memory resource of device */ + if (munmap(requested_addr, size)) { + RTE_LOG(ERR, EAL, "%s(): cannot munmap(%p, 0x%lx): %s\n", + __func__, requested_addr, (unsigned long)size, + strerror(errno)); + } else + RTE_LOG(DEBUG, EAL, " PCI memory unmapped at %p\n", + requested_addr); +} + /* parse the "resource" sysfs file */ static int pci_parse_sysfs_resource(const char *filename, struct rte_pci_device *dev) diff --git a/lib/librte_eal/linuxapp/eal/eal_pci_init.h b/lib/librte_eal/linuxapp/eal/eal_pci_init.h index 03d2b52..6af84d1 100644 --- a/lib/librte_eal/linuxapp/eal/eal_pci_init.h +++ b/lib/librte_eal/linuxapp/eal/eal_pci_init.h @@ -72,6 +72,13 @@ void *pci_map_resource(void *requested_addr, int fd, off_t offset, /* map IGB_UIO resource prototype */ int pci_uio_map_resource(struct rte_pci_device *dev); +void pci_unmap_resource(void *requested_addr, size_t size); + +#ifdef RTE_LIBRTE_EAL_HOTPLUG +/* unmap IGB_UIO resource prototype */ +void pci_uio_unmap_resource(struct rte_pci_device *dev); +#endif /* RTE_LIBRTE_EAL_HOTPLUG */ + #ifdef VFIO_PRESENT #define VFIO_MAX_GROUPS 64 diff --git a/lib/librte_eal/linuxapp/eal/eal_pci_uio.c b/lib/librte_eal/linuxapp/eal/eal_pci_uio.c index c5e0cf3..35d31c5 100644 --- a/lib/librte_eal/linuxapp/eal/eal_pci_uio.c +++ b/lib/librte_eal/linuxapp/eal/eal_pci_uio.c @@ -386,3 +386,68 @@ pci_uio_map_resource(struct rte_pci_device *dev) return 0; } + +#ifdef RTE_LIBRTE_EAL_HOTPLUG +static void +pci_uio_unmap(struct mapped_pci_resource *uio_res) +{ + int i; + + if (uio_res == NULL) + return; + + for (i = 0; i != uio_res->nb_maps; i++) + pci_unmap_resource(uio_res->maps[i].addr, + (size_t)uio_res->maps[i].size); +} + +static struct mapped_pci_resource * +pci_uio_find_resource(struct rte_pci_device *dev) +{ + struct mapped_pci_resource *uio_res; + + if (dev == NULL) + return NULL; + + TAILQ_FOREACH(uio_res, pci_res_list, next) { + + /* skip this element if it doesn't match our PCI address */ + if (!rte_eal_compare_pci_addr(&uio_res->pci_addr, &dev->addr)) + return uio_res; + } + return NULL; +} + +/* unmap the PCI resource of a PCI device in virtual memory */ +void +pci_uio_unmap_resource(struct rte_pci_device *dev) +{ + struct mapped_pci_resource *uio_res; + + if (dev == NULL) + return; + + /* find an entry for the device */ + uio_res = pci_uio_find_resource(dev); + if (uio_res == NULL) + return; + + /* secondary processes - just free maps */ + if (rte_eal_process_type() != RTE_PROC_PRIMARY) + return pci_uio_unmap(uio_res); + + TAILQ_REMOVE(pci_res_list, uio_res, next); + + /* unmap all resources */ + pci_uio_unmap(uio_res); + + /* free uio resource */ + rte_free(uio_res); + + /* close fd if in primary process */ + close(dev->intr_handle.fd); + + dev->intr_handle.fd = -1; + dev->intr_handle.type = RTE_INTR_HANDLE_UNKNOWN; +} +#endif /* RTE_LIBRTE_EAL_HOTPLUG */ -- 1.9.1
- Add pci_close_all_drivers() The function tries to find a driver for the specified device, and then close the driver. - Add rte_eal_pci_probe_one() and rte_eal_pci_close_one() The functions are used for probe and close a device. First the function tries to find a device that has the specified PCI address. Then, probe or close the device. v9: - Fix commit title. - Remove RTE_EAL_INVOKE_TYPE_PROBE/CLOSE. (Thanks to Thomas Monjalon) - Implement pci_unmap_device() in this patch. v5: - Remove RTE_EAL_INVOKE_TYPE_UNKNOWN, because it's unused. v4: - Fix parameter checking. - Fix indent of 'if' statement. Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp> --- lib/librte_eal/common/eal_common_pci.c | 98 ++++++++++++++++++++++++++++++++- lib/librte_eal/common/eal_private.h | 15 +++++ lib/librte_eal/common/include/rte_pci.h | 32 +++++++++++ lib/librte_eal/linuxapp/eal/eal_pci.c | 94 +++++++++++++++++++++++++++++++ 4 files changed, 238 insertions(+), 1 deletion(-) diff --git a/lib/librte_eal/common/eal_common_pci.c b/lib/librte_eal/common/eal_common_pci.c index bf2793f..5b6b55d 100644 --- a/lib/librte_eal/common/eal_common_pci.c +++ b/lib/librte_eal/common/eal_common_pci.c @@ -108,7 +108,10 @@ static int pci_probe_all_drivers(struct rte_pci_device *dev) { struct rte_pci_driver *dr = NULL; - int rc; + int rc = 0; + + if (dev == NULL) + return -1; TAILQ_FOREACH(dr, &pci_driver_list, next) { rc = rte_eal_pci_probe_one_driver(dr, dev); @@ -123,6 +126,99 @@ pci_probe_all_drivers(struct rte_pci_device *dev) return 1; } +#ifdef RTE_LIBRTE_EAL_HOTPLUG +/* + * If vendor/device ID match, call the devuninit() function of all + * registered driver for the given device. Return -1 if initialization + * failed, return 1 if no driver is found for this device. + */ +static int +pci_close_all_drivers(struct rte_pci_device *dev) +{ + struct rte_pci_driver *dr = NULL; + int rc = 0; + + if (dev == NULL) + return -1; + + TAILQ_FOREACH(dr, &pci_driver_list, next) { + rc = rte_eal_pci_close_one_driver(dr, dev); + if (rc < 0) + /* negative value is an error */ + return -1; + if (rc > 0) + /* positive value means driver not found */ + continue; + return 0; + } + return 1; +} + +/* + * Find the pci device specified by pci address, then invoke probe function of + * the driver of the devive. + */ +int +rte_eal_pci_probe_one(struct rte_pci_addr *addr) +{ + struct rte_pci_device *dev = NULL; + int ret = 0; + + if (addr == NULL) + return -1; + + TAILQ_FOREACH(dev, &pci_device_list, next) { + if (rte_eal_compare_pci_addr(&dev->addr, addr)) + continue; + + ret = pci_probe_all_drivers(dev); + if (ret < 0) + goto err_return; + return 0; + } + return -1; + +err_return: + RTE_LOG(WARNING, EAL, "Requested device " PCI_PRI_FMT + " cannot be used\n", dev->addr.domain, dev->addr.bus, + dev->addr.devid, dev->addr.function); + return -1; +} + +/* + * Find the pci device specified by pci address, then invoke close function of + * the driver of the devive. + */ +int +rte_eal_pci_close_one(struct rte_pci_addr *addr) +{ + struct rte_pci_device *dev = NULL; + int ret = 0; + + if (addr == NULL) + return -1; + + TAILQ_FOREACH(dev, &pci_device_list, next) { + if (rte_eal_compare_pci_addr(&dev->addr, addr)) + continue; + + ret = pci_close_all_drivers(dev); + if (ret < 0) + goto err_return; + + TAILQ_REMOVE(&pci_device_list, dev, next); + return 0; + } + return -1; + +err_return: + RTE_LOG(WARNING, EAL, "Requested device " PCI_PRI_FMT + " cannot be used\n", dev->addr.domain, dev->addr.bus, + dev->addr.devid, dev->addr.function); + return -1; +} +#endif /* RTE_LIBRTE_EAL_HOTPLUG */ + /* * Scan the content of the PCI bus, and call the devinit() function for * all registered drivers that have a matching entry in its id_table diff --git a/lib/librte_eal/common/eal_private.h b/lib/librte_eal/common/eal_private.h index 159cd66..4acf5a0 100644 --- a/lib/librte_eal/common/eal_private.h +++ b/lib/librte_eal/common/eal_private.h @@ -165,6 +165,21 @@ int rte_eal_pci_probe_one_driver(struct rte_pci_driver *dr, struct rte_pci_device *dev); /** + * Munmap memory for single PCI device + * + * This function is private to EAL. + * + * @param dr + * The pointer to the pci driver structure + * @param dev + * The pointer to the pci device structure + * @return + * 0 on success, negative on error + */ +int rte_eal_pci_close_one_driver(struct rte_pci_driver *dr, + struct rte_pci_device *dev); + +/** * Init tail queues for non-EAL library structures. This is to allow * the rings, mempools, etc. lists to be shared among multiple processes * diff --git a/lib/librte_eal/common/include/rte_pci.h b/lib/librte_eal/common/include/rte_pci.h index ecde36f..ac30925 100644 --- a/lib/librte_eal/common/include/rte_pci.h +++ b/lib/librte_eal/common/include/rte_pci.h @@ -325,6 +325,38 @@ rte_eal_compare_pci_addr(struct rte_pci_addr *addr, struct rte_pci_addr *addr2) */ int rte_eal_pci_probe(void); +#ifdef RTE_LIBRTE_EAL_HOTPLUG +/** + * Probe the single PCI device. + * + * Scan the content of the PCI bus, and find the pci device specified by pci + * address, then call the probe() function for registered driver that has a + * matching entry in its id_table for discovered device. + * + * @param addr + * The PCI Bus-Device-Function address to probe. + * @return + * - 0 on success. + * - Negative on error. + */ +int rte_eal_pci_probe_one(struct rte_pci_addr *addr); + +/** + * Close the single PCI device. + * + * Scan the content of the PCI bus, and find the pci device specified by pci + * address, then call the close() function for registered driver that has a + * matching entry in its id_table for discovered device. + * + * @param addr + * The PCI Bus-Device-Function address to close. + * @return + * - 0 on success. + * - Negative on error. + */ +int rte_eal_pci_close_one(struct rte_pci_addr *addr); +#endif /* RTE_LIBRTE_EAL_HOTPLUG */ + /** * Dump the content of the PCI bus. * diff --git a/lib/librte_eal/linuxapp/eal/eal_pci.c b/lib/librte_eal/linuxapp/eal/eal_pci.c index d03429c..f880f90 100644 --- a/lib/librte_eal/linuxapp/eal/eal_pci.c +++ b/lib/librte_eal/linuxapp/eal/eal_pci.c @@ -595,6 +595,31 @@ pci_map_device(struct rte_pci_device *dev) return ret; } +#ifdef RTE_LIBRTE_EAL_HOTPLUG +static void +pci_unmap_device(struct rte_pci_device *dev) +{ + if (dev == NULL) + return; + + /* try unmapping the NIC resources using VFIO if it exists */ + switch (dev->pt_driver) { + case RTE_PT_VFIO: + RTE_LOG(ERR, EAL, "Hotplug doesn't support vfio yet\n"); + break; + case RTE_PT_IGB_UIO: + case RTE_PT_UIO_GENERIC: + /* unmap resources for devices that use uio */ + pci_uio_unmap_resource(dev); + break; + default: + RTE_LOG(DEBUG, EAL, " Not managed by known pt driver," + " skipped\n"); + break; + } +} +#endif /* RTE_LIBRTE_EAL_HOTPLUG */ + /* * If vendor/device ID match, call the devinit() function of the * driver. @@ -666,6 +691,75 @@ rte_eal_pci_probe_one_driver(struct rte_pci_driver *dr, struct rte_pci_device *d return 1; } +#ifdef RTE_LIBRTE_EAL_HOTPLUG +/* + * If vendor/device ID match, call the devuninit() function of the + * driver. + */ +int +rte_eal_pci_close_one_driver(struct rte_pci_driver *dr, + struct rte_pci_device *dev) +{ + struct rte_pci_id *id_table; + + if ((dr == NULL) || (dev == NULL)) + return -EINVAL; + + for (id_table = dr->id_table ; id_table->vendor_id != 0; id_table++) { + + /* check if device's identifiers match the driver's ones */ + if (id_table->vendor_id != dev->id.vendor_id && + id_table->vendor_id != PCI_ANY_ID) + continue; + if (id_table->device_id != dev->id.device_id && + id_table->device_id != PCI_ANY_ID) + continue; + if (id_table->subsystem_vendor_id != + dev->id.subsystem_vendor_id && + id_table->subsystem_vendor_id != PCI_ANY_ID) + continue; + if (id_table->subsystem_device_id != + dev->id.subsystem_device_id && + id_table->subsystem_device_id != PCI_ANY_ID) + continue; + + struct rte_pci_addr *loc = &dev->addr; + + RTE_LOG(DEBUG, EAL, + "PCI device "PCI_PRI_FMT" on NUMA socket %i\n", + loc->domain, loc->bus, loc->devid, + loc->function, dev->numa_node); + + RTE_LOG(DEBUG, EAL, " remove driver: %x:%x %s\n", + dev->id.vendor_id, dev->id.device_id, + dr->name); + + /* call the driver devuninit() function */ + if (dr->devuninit && (dr->devuninit(dev) < 0)) + return -1; /* negative value is an error */ + + /* clear driver structure */ + dev->driver = NULL; + + if (dr->drv_flags & RTE_PCI_DRV_NEED_MAPPING) + /* unmap resources for devices that use igb_uio */ + pci_unmap_device(dev); + + return 0; + } + /* return positive value if driver is not found */ + return 1; +} +#else /* RTE_LIBRTE_EAL_HOTPLUG */ +int +rte_eal_pci_close_one_driver(struct rte_pci_driver *dr __rte_unused, + struct rte_pci_device *dev __rte_unused) +{ + RTE_LOG(ERR, EAL, "Hotplug support isn't enabled\n"); + return -1; +} +#endif /* RTE_LIBRTE_EAL_HOTPLUG */ + /* Init the PCI EAL subsystem */ int rte_eal_pci_init(void) -- 1.9.1
This new parameter is needed to keep device type like PCI or virtual. Port detaching processes are different between PCI device and virtual device. RTE_ETH_DEV_PCI indicates device type is PCI. RTE_ETH_DEV_VIRTUAL indicates device is virtual. v12: - Add missing symbol in version map. (Thanks to Iremonger, Bernard) v10: - Change order of version.map. (Thanks to Thomas Monjalon) - Fix comment of "rte_ethdev.h". (Thanks to Thomas Monjalon) v9: - Fix commit log. - RTE_ETH_DEV_PHYSICAL is replaced by RTE_ETH_DEV_PCI. (Thanks to Thomas Monjalon) v8: - NONE_TRACE is replaced by NO_TRACE. - Add missing symbol in version map. (Thanks to Qiu, Michael and Iremonger, Bernard) v4: - Fix comments of rte_eth_dev_type. Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp> --- app/test/virtual_pmd.c | 2 +- lib/librte_ether/rte_ethdev.c | 25 +++++++++++++++++++++++-- lib/librte_ether/rte_ethdev.h | 25 ++++++++++++++++++++++++- lib/librte_ether/rte_ether_version.map | 1 + lib/librte_pmd_af_packet/rte_eth_af_packet.c | 2 +- lib/librte_pmd_bond/rte_eth_bond_api.c | 2 +- lib/librte_pmd_pcap/rte_eth_pcap.c | 2 +- lib/librte_pmd_ring/rte_eth_ring.c | 2 +- lib/librte_pmd_xenvirt/rte_eth_xenvirt.c | 2 +- 9 files changed, 54 insertions(+), 9 deletions(-) diff --git a/app/test/virtual_pmd.c b/app/test/virtual_pmd.c index 785bccc..9b07ab1 100644 --- a/app/test/virtual_pmd.c +++ b/app/test/virtual_pmd.c @@ -580,7 +580,7 @@ virtual_ethdev_create(const char *name, struct ether_addr *mac_addr, goto err; /* reserve an ethdev entry */ - eth_dev = rte_eth_dev_allocate(name); + eth_dev = rte_eth_dev_allocate(name, RTE_ETH_DEV_PCI); if (eth_dev == NULL) goto err; diff --git a/lib/librte_ether/rte_ethdev.c b/lib/librte_ether/rte_ethdev.c index 1f6a066..4ebdd9f 100644 --- a/lib/librte_ether/rte_ethdev.c +++ b/lib/librte_ether/rte_ethdev.c @@ -227,7 +227,7 @@ rte_eth_dev_find_free_port(void) } struct rte_eth_dev * -rte_eth_dev_allocate(const char *name) +rte_eth_dev_allocate(const char *name, enum rte_eth_dev_type type) { uint8_t port_id; struct rte_eth_dev *eth_dev; @@ -251,6 +251,7 @@ rte_eth_dev_allocate(const char *name) snprintf(eth_dev->data->name, sizeof(eth_dev->data->name), "%s", name); eth_dev->data->port_id = port_id; eth_dev->attached = DEV_ATTACHED; + eth_dev->dev_type = type; nb_ports++; return eth_dev; } @@ -262,6 +263,7 @@ rte_eth_dev_release_port(struct rte_eth_dev *eth_dev) return -EINVAL; eth_dev->attached = 0; + eth_dev->dev_type = RTE_ETH_DEV_UNKNOWN; nb_ports--; return 0; } @@ -300,7 +302,7 @@ rte_eth_dev_init(struct rte_pci_driver *pci_drv, rte_eth_dev_create_unique_device_name(ethdev_name, sizeof(ethdev_name), pci_dev); - eth_dev = rte_eth_dev_allocate(ethdev_name); + eth_dev = rte_eth_dev_allocate(ethdev_name, RTE_ETH_DEV_PCI); if (eth_dev == NULL) return -ENOMEM; @@ -426,6 +428,14 @@ rte_eth_dev_count(void) return (nb_ports); } +enum rte_eth_dev_type +rte_eth_dev_get_device_type(uint8_t port_id) +{ + if (!rte_eth_dev_is_valid_port(port_id)) + return -1; + return rte_eth_devices[port_id].dev_type; +} + int rte_eth_dev_save(struct rte_eth_dev *devs, size_t size) { @@ -523,6 +533,17 @@ rte_eth_dev_is_detachable(uint8_t port_id) return -EINVAL; } + if (rte_eth_devices[port_id].dev_type == RTE_ETH_DEV_PCI) { + switch (rte_eth_devices[port_id].pci_dev->pt_driver) { + case RTE_PT_IGB_UIO: + case RTE_PT_UIO_GENERIC: + break; + case RTE_PT_VFIO: + default: + return -ENOTSUP; + } + } + drv_flags = rte_eth_devices[port_id].driver->pci_drv.drv_flags; return !(drv_flags & RTE_PCI_DRV_DETACHABLE); } diff --git a/lib/librte_ether/rte_ethdev.h b/lib/librte_ether/rte_ethdev.h index 5519ce0..d8e5543 100644 --- a/lib/librte_ether/rte_ethdev.h +++ b/lib/librte_ether/rte_ethdev.h @@ -1422,6 +1422,17 @@ struct rte_eth_rxtx_callback { void *param; }; +/* + * The eth device type + */ +enum rte_eth_dev_type { + RTE_ETH_DEV_UNKNOWN, /**< unknown device type */ + RTE_ETH_DEV_PCI, + /**< Physical function and Virtual function of PCI devices */ + RTE_ETH_DEV_VIRTUAL, /**< non hardware device */ + RTE_ETH_DEV_MAX /**< max value of this enum */ +}; + /** * @internal * The generic data structure associated with each ethernet device. @@ -1452,6 +1463,7 @@ struct rte_eth_dev { */ struct rte_eth_rxtx_callback **pre_tx_burst_cbs; uint8_t attached; /**< Flag indicating the port is attached */ + enum rte_eth_dev_type dev_type; /**< Flag indicating the device type */ }; struct rte_eth_dev_sriov { @@ -1529,6 +1541,15 @@ extern uint8_t rte_eth_dev_count(void); /** * Function for internal use by port hotplug functions. + * Get the device type to know whether the device is physical or virtual. + * @param port_id The port identifier of the Ethernet device + * @return + * - Device type. + */ +extern enum rte_eth_dev_type rte_eth_dev_get_device_type(uint8_t port_id); + +/** + * Function for internal use by port hotplug functions. * Copies current ethdev structures to the specified pointer. * * @param devs The pointer to the ethdev structures @@ -1617,10 +1638,12 @@ extern struct rte_eth_dev *rte_eth_dev_allocated(const char *name); * to that slot for the driver to use. * * @param name Unique identifier name for each Ethernet device + * @param type Device type of this Ethernet device * @return * - Slot in the rte_dev_devices array for a new device; */ -struct rte_eth_dev *rte_eth_dev_allocate(const char *name); +struct rte_eth_dev *rte_eth_dev_allocate(const char *name, + enum rte_eth_dev_type type); /** * Function for internal use by dummy drivers primarily, e.g. ring-based diff --git a/lib/librte_ether/rte_ether_version.map b/lib/librte_ether/rte_ether_version.map index 7db7085..c71eeec 100644 --- a/lib/librte_ether/rte_ether_version.map +++ b/lib/librte_ether/rte_ether_version.map @@ -37,6 +37,7 @@ DPDK_2.0 { rte_eth_dev_flow_ctrl_set; rte_eth_dev_get_addr_by_port; rte_eth_dev_get_changed_port; + rte_eth_dev_get_device_type; rte_eth_dev_get_mtu; rte_eth_dev_get_name_by_port; rte_eth_dev_get_port_by_addr; diff --git a/lib/librte_pmd_af_packet/rte_eth_af_packet.c b/lib/librte_pmd_af_packet/rte_eth_af_packet.c index 1ffe1cd..80e9bdf 100644 --- a/lib/librte_pmd_af_packet/rte_eth_af_packet.c +++ b/lib/librte_pmd_af_packet/rte_eth_af_packet.c @@ -649,7 +649,7 @@ rte_pmd_init_internals(const char *name, } /* reserve an ethdev entry */ - *eth_dev = rte_eth_dev_allocate(name); + *eth_dev = rte_eth_dev_allocate(name, RTE_ETH_DEV_VIRTUAL); if (*eth_dev == NULL) goto error; diff --git a/lib/librte_pmd_bond/rte_eth_bond_api.c b/lib/librte_pmd_bond/rte_eth_bond_api.c index 230d36c..903b7c3 100644 --- a/lib/librte_pmd_bond/rte_eth_bond_api.c +++ b/lib/librte_pmd_bond/rte_eth_bond_api.c @@ -257,7 +257,7 @@ rte_eth_bond_create(const char *name, uint8_t mode, uint8_t socket_id) } /* reserve an ethdev entry */ - eth_dev = rte_eth_dev_allocate(name); + eth_dev = rte_eth_dev_allocate(name, RTE_ETH_DEV_VIRTUAL); if (eth_dev == NULL) { RTE_BOND_LOG(ERR, "Unable to allocate rte_eth_dev"); goto err; diff --git a/lib/librte_pmd_pcap/rte_eth_pcap.c b/lib/librte_pmd_pcap/rte_eth_pcap.c index d299288..af7fae8 100644 --- a/lib/librte_pmd_pcap/rte_eth_pcap.c +++ b/lib/librte_pmd_pcap/rte_eth_pcap.c @@ -709,7 +709,7 @@ rte_pmd_init_internals(const char *name, const unsigned nb_rx_queues, goto error; /* reserve an ethdev entry */ - *eth_dev = rte_eth_dev_allocate(name); + *eth_dev = rte_eth_dev_allocate(name, RTE_ETH_DEV_VIRTUAL); if (*eth_dev == NULL) goto error; diff --git a/lib/librte_pmd_ring/rte_eth_ring.c b/lib/librte_pmd_ring/rte_eth_ring.c index a5dc71e..83ecfa8 100644 --- a/lib/librte_pmd_ring/rte_eth_ring.c +++ b/lib/librte_pmd_ring/rte_eth_ring.c @@ -297,7 +297,7 @@ rte_eth_from_rings(const char *name, struct rte_ring *const rx_queues[], goto error; /* reserve an ethdev entry */ - eth_dev = rte_eth_dev_allocate(name); + eth_dev = rte_eth_dev_allocate(name, RTE_ETH_DEV_VIRTUAL); if (eth_dev == NULL) goto error; diff --git a/lib/librte_pmd_xenvirt/rte_eth_xenvirt.c b/lib/librte_pmd_xenvirt/rte_eth_xenvirt.c index 04e30c9..bc403d6 100644 --- a/lib/librte_pmd_xenvirt/rte_eth_xenvirt.c +++ b/lib/librte_pmd_xenvirt/rte_eth_xenvirt.c @@ -648,7 +648,7 @@ eth_dev_xenvirt_create(const char *name, const char *params, goto err; /* reserve an ethdev entry */ - eth_dev = rte_eth_dev_allocate(name); + eth_dev = rte_eth_dev_allocate(name, RTE_ETH_DEV_VIRTUAL); if (eth_dev == NULL) goto err; -- 1.9.1
These functions are used for attaching or detaching a port. When rte_eal_dev_attach() is called, the function tries to realize the device name as pci address. If this is done successfully, rte_eal_dev_attach() will attach physical device port. If not, attaches virtual devive port. When rte_eal_dev_detach() is called, the function gets the device type of this port to know whether the port is come from physical or virtual. And then specific detaching function will be called. v14: - Remove needless if statement. (Thanks to Maxime Leroy) v13: - Change log level when error occurs in rte_eal_vdev_init() and rte_eal_dev_init(). - Return value of driver init and uninit functions. - Replace rte_panic by RTE_LOG in rte_eal_dev_init() - Fix return value of rte_eal_vdev_uninit(). - Fix rte_eal_dev_attach_vdev to set port_id correctly. (Thanks to Maxime Leroy) v11: - Remove needless devargs handling codes. - Replace get_vdev_name() by rte_eal_parse_devargs_str(). - Replace rte_eal_vdev_find_and_init by rte_eal_vdev_init() - Replace rte_eal_vdev_find_and_uninit by rte_eal_vdev_uninit() - Fix rte_eal_dev_init() to use rte_eal_vdev_init(). (Thanks to Maxime Leroy) v10: - Add comments. - Change order of version.map. (Thanks to Thomas Monjalon) v9: - Fix comments. - Use strcmp() instead of strncmp(). - Remove RTE_EAL_INVOKE_TYPE_PROBE/CLOSE. - Change definition of rte_dev_uninit_t. (Thanks to Thomas Monjalon and Maxime Leroy) v8: - Add missing symbol in version map. (Thanks to Qiu, Michael and Iremonger, Bernard) v7: - Fix typo of warning messages. (Thanks to Qiu, Michael) v5: - Change function names like below. rte_eal_dev_find_and_invoke() to rte_eal_vdev_find_and_invoke(). rte_eal_dev_invoke() to rte_eal_vdev_invoke(). - Add code to handle a return value of rte_eal_devargs_remove(). - Fix pci address format in rte_eal_dev_detach(). v4: - Fix comment. - Add error checking. - Fix indent of 'if' statement. - Change function name. Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp> --- lib/librte_eal/common/eal_common_dev.c | 278 ++++++++++++++++++++++-- lib/librte_eal/common/eal_common_devargs.c | 54 +++-- lib/librte_eal/common/eal_private.h | 11 + lib/librte_eal/common/include/rte_dev.h | 33 +++ lib/librte_eal/common/include/rte_devargs.h | 28 +++ lib/librte_eal/linuxapp/eal/Makefile | 1 + lib/librte_eal/linuxapp/eal/eal_pci.c | 6 +- lib/librte_eal/linuxapp/eal/rte_eal_version.map | 2 + 8 files changed, 375 insertions(+), 38 deletions(-) diff --git a/lib/librte_eal/common/eal_common_dev.c b/lib/librte_eal/common/eal_common_dev.c index eae5656..6d805aa 100644 --- a/lib/librte_eal/common/eal_common_dev.c +++ b/lib/librte_eal/common/eal_common_dev.c @@ -32,10 +32,13 @@ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ +#include <stdio.h> +#include <limits.h> #include <string.h> #include <inttypes.h> #include <sys/queue.h> +#include <rte_ethdev.h> #include <rte_dev.h> #include <rte_devargs.h> #include <rte_debug.h> @@ -61,6 +64,32 @@ rte_eal_driver_unregister(struct rte_driver *driver) TAILQ_REMOVE(&dev_driver_list, driver, next); } +static int +rte_eal_vdev_init(const char *name, const char *args) +{ + struct rte_driver *driver; + + if (name == NULL) + return -EINVAL; + + TAILQ_FOREACH(driver, &dev_driver_list, next) { + if (driver->type != PMD_VDEV) + continue; + + /* + * search a driver prefix in virtual device name. + * For example, if the driver is pcap PMD, driver->name + * will be "eth_pcap", but "name" will be "eth_pcapN". + * So use strncmp to compare. + */ + if (!strncmp(driver->name, name, strlen(driver->name))) + return driver->init(name, args); + } + + RTE_LOG(ERR, EAL, "no driver found for %s\n", name); + return -EINVAL; +} + int rte_eal_dev_init(void) { @@ -79,22 +108,11 @@ rte_eal_dev_init(void) if (devargs->type != RTE_DEVTYPE_VIRTUAL) continue; - TAILQ_FOREACH(driver, &dev_driver_list, next) { - if (driver->type != PMD_VDEV) - continue; - - /* search a driver prefix in virtual device name */ - if (!strncmp(driver->name, devargs->virtual.drv_name, - strlen(driver->name))) { - driver->init(devargs->virtual.drv_name, - devargs->args); - break; - } - } - - if (driver == NULL) { - rte_panic("no driver found for %s\n", - devargs->virtual.drv_name); + if (rte_eal_vdev_init(devargs->virtual.drv_name, + devargs->args)) { + RTE_LOG(ERR, EAL, "failed to initialize %s device\n", + devargs->virtual.drv_name); + return -1; } } @@ -107,3 +125,231 @@ rte_eal_dev_init(void) } return 0; } + +/* So far, DPDK hotplug function only supports linux */ +#ifdef RTE_LIBRTE_EAL_HOTPLUG +static int +rte_eal_vdev_uninit(const char *name) +{ + struct rte_driver *driver; + + if (name == NULL) + return -EINVAL; + + TAILQ_FOREACH(driver, &dev_driver_list, next) { + if (driver->type != PMD_VDEV) + continue; + + /* + * search a driver prefix in virtual device name. + * For example, if the driver is pcap PMD, driver->name + * will be "eth_pcap", but "name" will be "eth_pcapN". + * So use strncmp to compare. + */ + if (!strncmp(driver->name, name, strlen(driver->name))) + return driver->uninit(name); + } + + RTE_LOG(ERR, EAL, "no driver found for %s\n", name); + return -EINVAL; +} + +/* attach the new physical device, then store port_id of the device */ +static int +rte_eal_dev_attach_pdev(struct rte_pci_addr *addr, uint8_t *port_id) +{ + uint8_t new_port_id; + struct rte_eth_dev devs[RTE_MAX_ETHPORTS]; + + if ((addr == NULL) || (port_id == NULL)) + goto err; + + /* save current port status */ + if (rte_eth_dev_save(devs, sizeof(devs))) + goto err; + /* re-construct pci_device_list */ + if (rte_eal_pci_scan()) + goto err; + /* invoke probe func of the driver can handle the new device. + * TODO: + * rte_eal_pci_probe_one() should return port_id. + * And rte_eth_dev_save() and rte_eth_dev_get_changed_port() + * should be removed. */ + if (rte_eal_pci_probe_one(addr)) + goto err; + /* get port_id enabled by above procedures */ + if (rte_eth_dev_get_changed_port(devs, &new_port_id)) + goto err; + + *port_id = new_port_id; + return 0; +err: + RTE_LOG(ERR, EAL, "Driver, cannot attach the device\n"); + return -1; +} + +/* detach the new physical device, then store pci_addr of the device */ +static int +rte_eal_dev_detach_pdev(uint8_t port_id, struct rte_pci_addr *addr) +{ + struct rte_pci_addr freed_addr; + struct rte_pci_addr vp; + + if (addr == NULL) + goto err; + + /* check whether the driver supports detach feature, or not */ + if (rte_eth_dev_is_detachable(port_id)) + goto err; + + /* get pci address by port id */ + if (rte_eth_dev_get_addr_by_port(port_id, &freed_addr)) + goto err; + + /* Zerod pci addr means the port comes from virtual device */ + vp.domain = vp.bus = vp.devid = vp.function = 0; + if (rte_eal_compare_pci_addr(&vp, &freed_addr) == 0) + goto err; + + /* invoke close func of the driver, + * also remove the device from pci_device_list */ + if (rte_eal_pci_close_one(&freed_addr)) + goto err; + + *addr = freed_addr; + return 0; +err: + RTE_LOG(ERR, EAL, "Driver, cannot detach the device\n"); + return -1; +} + +/* attach the new virtual device, then store port_id of the device */ +static int +rte_eal_dev_attach_vdev(const char *vdevargs, uint8_t *port_id) +{ + char *name = NULL, *args = NULL; + uint8_t new_port_id; + struct rte_eth_dev devs[RTE_MAX_ETHPORTS]; + int ret = -1; + + if ((vdevargs == NULL) || (port_id == NULL)) + goto end; + + /* parse vdevargs, then retrieve device name and args */ + if (rte_eal_parse_devargs_str(vdevargs, &name, &args)) + goto end; + + /* save current port status */ + if (rte_eth_dev_save(devs, sizeof(devs))) + goto end; + /* walk around dev_driver_list to find the driver of the device, + * then invoke probe function o the driver. + * TODO: + * rte_eal_vdev_init() should return port_id, + * And rte_eth_dev_save() and rte_eth_dev_get_changed_port() + * should be removed. */ + if (rte_eal_vdev_init(name, args)) + goto end; + /* get port_id enabled by above procedures */ + if (rte_eth_dev_get_changed_port(devs, &new_port_id)) + goto end; + ret = 0; + *port_id = new_port_id; +end: + if (name) + free(name); + if (args) + free(args); + + if (ret < 0) + RTE_LOG(ERR, EAL, "Driver, cannot attach the device\n"); + return ret; +} + +/* detach the new virtual device, then store the name of the device */ +static int +rte_eal_dev_detach_vdev(uint8_t port_id, char *vdevname) +{ + char name[RTE_ETH_NAME_MAX_LEN]; + + if (vdevname == NULL) + goto err; + + /* check whether the driver supports detach feature, or not */ + if (rte_eth_dev_is_detachable(port_id)) + goto err; + + /* get device name by port id */ + if (rte_eth_dev_get_name_by_port(port_id, name)) + goto err; + /* walk around dev_driver_list to find the driver of the device, + * then invoke close function o the driver */ + if (rte_eal_vdev_uninit(name)) + goto err; + + strncpy(vdevname, name, sizeof(name)); + return 0; +err: + RTE_LOG(ERR, EAL, "Driver, cannot detach the device\n"); + return -1; +} + +/* attach the new device, then store port_id of the device */ +int +rte_eal_dev_attach(const char *devargs, uint8_t *port_id) +{ + struct rte_pci_addr addr; + + if ((devargs == NULL) || (port_id == NULL)) + return -EINVAL; + + if (eal_parse_pci_DomBDF(devargs, &addr) == 0) + return rte_eal_dev_attach_pdev(&addr, port_id); + else + return rte_eal_dev_attach_vdev(devargs, port_id); +} + +/* detach the device, then store the name of the device */ +int +rte_eal_dev_detach(uint8_t port_id, char *name) +{ + struct rte_pci_addr addr; + int ret; + + if (name == NULL) + return -EINVAL; + + if (rte_eth_dev_get_device_type(port_id) == RTE_ETH_DEV_PCI) { + ret = rte_eth_dev_get_addr_by_port(port_id, &addr); + if (ret < 0) + return ret; + + ret = rte_eal_dev_detach_pdev(port_id, &addr); + if (ret == 0) + snprintf(name, RTE_ETH_NAME_MAX_LEN, + "%04x:%02x:%02x.%d", + addr.domain, addr.bus, + addr.devid, addr.function); + + return ret; + } else + return rte_eal_dev_detach_vdev(port_id, name); +} +#else /* RTE_LIBRTE_EAL_HOTPLUG */ +int +rte_eal_dev_attach(const char *devargs __rte_unused, + uint8_t *port_id __rte_unused) +{ + RTE_LOG(ERR, EAL, "Hotplug support isn't enabled\n"); + return -1; +} + +/* detach the device, then store the name of the device */ +int +rte_eal_dev_detach(uint8_t port_id __rte_unused, + char *name __rte_unused) +{ + RTE_LOG(ERR, EAL, "Hotplug support isn't enabled\n"); + return -1; +} +#endif /* RTE_LIBRTE_EAL_HOTPLUG */ diff --git a/lib/librte_eal/common/eal_common_devargs.c b/lib/librte_eal/common/eal_common_devargs.c index eadd719..9b110f7 100644 --- a/lib/librte_eal/common/eal_common_devargs.c +++ b/lib/librte_eal/common/eal_common_devargs.c @@ -44,13 +44,46 @@ struct rte_devargs_list devargs_list = TAILQ_HEAD_INITIALIZER(devargs_list); +int +rte_eal_parse_devargs_str(const char *devargs_str, + char **drvname, char **drvargs) +{ + char *sep; + + if ((devargs_str) == NULL || (drvname) == NULL || (drvargs == NULL)) + return -1; + + *drvname = strdup(devargs_str); + if (drvname == NULL) { + RTE_LOG(ERR, EAL, + "cannot allocate temp memory for driver name\n"); + return -1; + } + + /* set the first ',' to '\0' to split name and arguments */ + sep = strchr(*drvname, ','); + if (sep != NULL) { + sep[0] = '\0'; + *drvargs = strdup(sep + 1); + } else { + *drvargs = strdup(""); + } + + if (*drvargs == NULL) { + RTE_LOG(ERR, EAL, + "cannot allocate temp memory for driver arguments\n"); + free(*drvname); + return -1; + } + return 0; +} + /* store a whitelist parameter for later parsing */ int rte_eal_devargs_add(enum rte_devtype devtype, const char *devargs_str) { struct rte_devargs *devargs = NULL; char *buf = NULL; - char *sep; int ret; /* use malloc instead of rte_malloc as it's called early at init */ @@ -62,25 +95,8 @@ rte_eal_devargs_add(enum rte_devtype devtype, const char *devargs_str) memset(devargs, 0, sizeof(*devargs)); devargs->type = devtype; - buf = strdup(devargs_str); - if (buf == NULL) { - RTE_LOG(ERR, EAL, "cannot allocate temp memory for devargs\n"); - goto fail; - } - - /* set the first ',' to '\0' to split name and arguments */ - sep = strchr(buf, ','); - if (sep != NULL) { - sep[0] = '\0'; - devargs->args = strdup(sep + 1); - } else { - devargs->args = strdup(""); - } - - if (devargs->args == NULL) { - RTE_LOG(ERR, EAL, "cannot allocate for devargs args\n"); + if (rte_eal_parse_devargs_str(devargs_str, &buf, &devargs->args)) goto fail; - } switch (devargs->type) { case RTE_DEVTYPE_WHITELISTED_PCI: diff --git a/lib/librte_eal/common/eal_private.h b/lib/librte_eal/common/eal_private.h index 4acf5a0..98b286a 100644 --- a/lib/librte_eal/common/eal_private.h +++ b/lib/librte_eal/common/eal_private.h @@ -154,6 +154,17 @@ struct rte_pci_driver; struct rte_pci_device; /** + * Scan the content of the PCI bus, and the devices in the devices + * list + * + * This function is private to EAL. + * + * @return + * 0 on success, negative on error + */ +int rte_eal_pci_scan(void); + +/** * Mmap memory for single PCI device * * This function is private to EAL. diff --git a/lib/librte_eal/common/include/rte_dev.h b/lib/librte_eal/common/include/rte_dev.h index f7e3a10..a5ac770 100644 --- a/lib/librte_eal/common/include/rte_dev.h +++ b/lib/librte_eal/common/include/rte_dev.h @@ -47,6 +47,7 @@ extern "C" { #endif #include <sys/queue.h> +#include <rte_pci.h> /** Double linked list of device drivers. */ TAILQ_HEAD(rte_driver_list, rte_driver); @@ -57,6 +58,11 @@ TAILQ_HEAD(rte_driver_list, rte_driver); typedef int (rte_dev_init_t)(const char *name, const char *args); /** + * Uninitilization function called for each device driver once. + */ +typedef int (rte_dev_uninit_t)(const char *name); + +/** * Driver type enumeration */ enum pmd_type { @@ -72,6 +78,7 @@ struct rte_driver { enum pmd_type type; /**< PMD Driver type */ const char *name; /**< Driver name. */ rte_dev_init_t *init; /**< Device init. function. */ + rte_dev_uninit_t *uninit; /**< Device uninit. function. */ }; /** @@ -93,6 +100,32 @@ void rte_eal_driver_register(struct rte_driver *driver); void rte_eal_driver_unregister(struct rte_driver *driver); /** + * Attach a new device. + * + * @param devargs + * A pointer to a strings array describing the new device + * to be attached. The strings should be a pci address like + * '0000:01:00.0' or virtual device name like 'eth_pcap0'. + * @param port_id + * A pointer to a port identifier actually attached. + * @return + * 0 on success and port_id is filled, negative on error + */ +int rte_eal_dev_attach(const char *devargs, uint8_t *port_id); + +/** + * Detach a device. + * + * @param port_id + * The port identifier of the device to detach. + * @param addr + * A pointer to a device name actually detached. + * @return + * 0 on success and devname is filled, negative on error + */ +int rte_eal_dev_detach(uint8_t port_id, char *devname); + +/** * Initalize all the registered drivers in this process */ int rte_eal_dev_init(void); diff --git a/lib/librte_eal/common/include/rte_devargs.h b/lib/librte_eal/common/include/rte_devargs.h index 6834333..039f728 100644 --- a/lib/librte_eal/common/include/rte_devargs.h +++ b/lib/librte_eal/common/include/rte_devargs.h @@ -99,6 +99,34 @@ TAILQ_HEAD(rte_devargs_list, rte_devargs); extern struct rte_devargs_list devargs_list; /** + * Parse a devargs string. + * + * For PCI devices, the format of arguments string is "PCI_ADDR" or + * "PCI_ADDR,key=val,key2=val2,...". Examples: "08:00.1", "0000:5:00.0", + * "04:00.0,arg=val". + * + * For virtual devices, the format of arguments string is "DRIVER_NAME*" + * or "DRIVER_NAME*,key=val,key2=val2,...". Examples: "eth_ring", + * "eth_ring0", "eth_pmdAnything,arg=0:arg2=1". + * + * The function parses the arguments string to get driver name and driver + * arguments. + * + * @param devargs_str + * The arguments as given by the user. + * @param drvname + * The pointer to the string to store parsed driver name. + * @param drvargs + * The pointer to the string to store parsed driver arguments. + * + * @return + * - 0 on success + * - A negative value on error + */ +int rte_eal_parse_devargs_str(const char *devargs_str, + char **drvname, char **drvargs); + +/** * Add a device to the user device list * * For PCI devices, the format of arguments string is "PCI_ADDR" or diff --git a/lib/librte_eal/linuxapp/eal/Makefile b/lib/librte_eal/linuxapp/eal/Makefile index 23c2d48..a30bbf7 100644 --- a/lib/librte_eal/linuxapp/eal/Makefile +++ b/lib/librte_eal/linuxapp/eal/Makefile @@ -45,6 +45,7 @@ CFLAGS += -I$(RTE_SDK)/lib/librte_eal/common/include CFLAGS += -I$(RTE_SDK)/lib/librte_ring CFLAGS += -I$(RTE_SDK)/lib/librte_mempool CFLAGS += -I$(RTE_SDK)/lib/librte_malloc +CFLAGS += -I$(RTE_SDK)/lib/librte_mbuf CFLAGS += -I$(RTE_SDK)/lib/librte_ether CFLAGS += -I$(RTE_SDK)/lib/librte_ivshmem CFLAGS += -I$(RTE_SDK)/lib/librte_pmd_ring diff --git a/lib/librte_eal/linuxapp/eal/eal_pci.c b/lib/librte_eal/linuxapp/eal/eal_pci.c index f880f90..6d4932d 100644 --- a/lib/librte_eal/linuxapp/eal/eal_pci.c +++ b/lib/librte_eal/linuxapp/eal/eal_pci.c @@ -440,8 +440,8 @@ error: * Scan the content of the PCI bus, and the devices in the devices * list */ -static int -pci_scan(void) +int +rte_eal_pci_scan(void) { struct dirent *e; DIR *dir; @@ -773,7 +773,7 @@ rte_eal_pci_init(void) if (internal_config.no_pci) return 0; - if (pci_scan() < 0) { + if (rte_eal_pci_scan() < 0) { RTE_LOG(ERR, EAL, "%s(): Cannot scan PCI bus\n", __func__); return -1; } diff --git a/lib/librte_eal/linuxapp/eal/rte_eal_version.map b/lib/librte_eal/linuxapp/eal/rte_eal_version.map index c207cee..d2f7ded 100644 --- a/lib/librte_eal/linuxapp/eal/rte_eal_version.map +++ b/lib/librte_eal/linuxapp/eal/rte_eal_version.map @@ -20,6 +20,8 @@ DPDK_2.0 { rte_dump_tailq; rte_eal_alarm_cancel; rte_eal_alarm_set; + rte_eal_dev_attach; + rte_eal_dev_detach; rte_eal_dev_init; rte_eal_devargs_add; rte_eal_devargs_dump; -- 1.9.1
This patch adds a new section for describing port hotplug framework. Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp> --- doc/guides/prog_guide/index.rst | 1 + doc/guides/prog_guide/port_hotplug_framework.rst | 110 +++++++++++++++++++++++ 2 files changed, 111 insertions(+) create mode 100644 doc/guides/prog_guide/port_hotplug_framework.rst diff --git a/doc/guides/prog_guide/index.rst b/doc/guides/prog_guide/index.rst index de69682..60a6ac5 100644 --- a/doc/guides/prog_guide/index.rst +++ b/doc/guides/prog_guide/index.rst @@ -71,6 +71,7 @@ Programmer's Guide packet_classif_access_ctrl packet_framework vhost_lib + port_hotplug_framework source_org dev_kit_build_system dev_kit_root_make_help diff --git a/doc/guides/prog_guide/port_hotplug_framework.rst b/doc/guides/prog_guide/port_hotplug_framework.rst new file mode 100644 index 0000000..355ae28 --- /dev/null +++ b/doc/guides/prog_guide/port_hotplug_framework.rst @@ -0,0 +1,110 @@ +.. BSD LICENSE + Copyright(c) 2015 IGEL Co.,Ltd. All rights reserved. + All rights reserved. + + Redistribution and use in source and binary forms, with or without + modification, are permitted provided that the following conditions + are met: + + * Redistributions of source code must retain the above copyright + notice, this list of conditions and the following disclaimer. + * Redistributions in binary form must reproduce the above copyright + notice, this list of conditions and the following disclaimer in + the documentation and/or other materials provided with the + distribution. + * Neither the name of IGEL Co.,Ltd. nor the names of its + contributors may be used to endorse or promote products derived + from this software without specific prior written permission. + + THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS + "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT + LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR + A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT + OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, + SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT + LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, + DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY + THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT + (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE + OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + +Port Hotplug Framework +====================== + +The Port Hotplug Framework provides DPDK applications with the ability to +attach and detach ports at runtime. Because the framework depends on PMD +implementation, the ports that PMDs cannot handle are out of scope of this +framework. Furthermore, after detaching a port from a DPDK application, the +framework doesn't provide a way for removing the devices from the system. +For the ports backed by a physical NIC, the kernel will need to support PCI +Hotplug feature. + +Overview +-------- + +The basic requirements of the Port Hotplug Framework are: + +* DPDK applications that use the Port Hotplug Framework must manage their + own ports. + + The Port Hotplug Framework is implemented to allow DPDK applications to + manage ports. For example, when DPDK applications call the port attach + function, the attached port number is returned. DPDK applications can + also detach the port by port number. + +* Kernel support is needed for attaching or detaching physical device + ports. + + To attach new physical device ports, the device will be recognized by + userspace driver I/O framework in kernel at first. Then DPDK + applications can call the Port Hotplug functions to attach the ports. + For detaching, steps are vice versa. + +* Before detaching, they must be stopped and closed. + + DPDK applications must call "rte_eth_dev_stop()" and + "rte_eth_dev_close()" APIs before detaching ports. These functions will + start finalization sequence of the PMDs. + +* The framework doesn't affect legacy DPDK applications behavior. + + If the Port Hotplug functions aren't called, all legacy DPDK apps can + still work without modifications. + +Port Hotplug API overview +------------------------- + +* Attaching a port + + "rte_eal_dev_attach()" API attaches a port to DPDK application, and + returns the attached port number. Before calling the API, the device + should be recognized by an userspace driver I/O framework. The API + receives a pci address like "0000:01:00.0" or a virtual device name + like "eth_pcap0,iface=eth0". In the case of virtual device name, the + format is the same as the general "--vdev" option of DPDK. + +* Detaching a port + + "rte_eal_dev_detach()" API detaches a port from DPDK application, and + returns a pci address of the detached device or a virtual device name + of the device. + +Reference +--------- + + "testpmd" supports the Port Hotplug Framework. + +Limitations +----------- + +* The Port Hotplug APIs are not thread safe. + +* The framework can only be enabled with Linux. BSD is not supported. + +* To detach a port, the port should be backed by a device that igb_uio + manages. VFIO is not supported. + +* Not all PMDs support detaching feature. + To know whether a PMD can support detaching, search for the + "RTE_PCI_DRV_DETACHABLE" flag in PMD implementation. If the flag is + defined in the PMD, detaching is supported. -- 1.9.1
This patch adds finalization code to free resources allocated by the PMD. v6: - Fix a paramter of rte_eth_dev_free(). v4: - Change function name. Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp> --- lib/librte_pmd_pcap/rte_eth_pcap.c | 40 ++++++++++++++++++++++++++++++++++++++ 1 file changed, 40 insertions(+) diff --git a/lib/librte_pmd_pcap/rte_eth_pcap.c b/lib/librte_pmd_pcap/rte_eth_pcap.c index af7fae8..5e94930 100644 --- a/lib/librte_pmd_pcap/rte_eth_pcap.c +++ b/lib/librte_pmd_pcap/rte_eth_pcap.c @@ -498,6 +498,13 @@ static struct eth_dev_ops ops = { .stats_reset = eth_stats_reset, }; +static struct eth_driver rte_pcap_pmd = { + .pci_drv = { + .name = "rte_pcap_pmd", + .drv_flags = RTE_PCI_DRV_DETACHABLE, + }, +}; + /* * Function handler that opens the pcap file for reading a stores a * reference of it for use it later on. @@ -713,6 +720,10 @@ rte_pmd_init_internals(const char *name, const unsigned nb_rx_queues, if (*eth_dev == NULL) goto error; + /* check length of device name */ + if ((strlen((*eth_dev)->data->name) + 1) > sizeof(data->name)) + goto error; + /* now put it all together * - store queue data in internals, * - store numa_node info in pci_driver @@ -739,10 +750,13 @@ rte_pmd_init_internals(const char *name, const unsigned nb_rx_queues, data->nb_tx_queues = (uint16_t)nb_tx_queues; data->dev_link = pmd_link; data->mac_addrs = ð_addr; + strncpy(data->name, + (*eth_dev)->data->name, strlen((*eth_dev)->data->name)); (*eth_dev)->data = data; (*eth_dev)->dev_ops = &ops; (*eth_dev)->pci_dev = pci_dev; + (*eth_dev)->driver = &rte_pcap_pmd; return 0; @@ -927,10 +941,36 @@ rte_pmd_pcap_devinit(const char *name, const char *params) } +static int +rte_pmd_pcap_devuninit(const char *name) +{ + struct rte_eth_dev *eth_dev = NULL; + + RTE_LOG(INFO, PMD, "Closing pcap ethdev on numa socket %u\n", + rte_socket_id()); + + if (name == NULL) + return -1; + + /* reserve an ethdev entry */ + eth_dev = rte_eth_dev_allocated(name); + if (eth_dev == NULL) + return -1; + + rte_free(eth_dev->data->dev_private); + rte_free(eth_dev->data); + rte_free(eth_dev->pci_dev); + + rte_eth_dev_release_port(eth_dev); + + return 0; +} + static struct rte_driver pmd_pcap_drv = { .name = "eth_pcap", .type = PMD_VDEV, .init = rte_pmd_pcap_devinit, + .uninit = rte_pmd_pcap_devuninit, }; PMD_REGISTER_DRIVER(pmd_pcap_drv); -- 1.9.1
The patch introduces following commands. - port attach [ident] - port detach [port_id] - attach: attaching a port - detach: detaching a port - ident: pci address of physical device. Or device name and parameters of virtual device. (ex. 0000:02:00.0, eth_pcap0,iface=eth0) - port_id: port identifier v7: - Fix doc. (Thanks to Iremonger, Bernard) - Fix port checking implementation of star_port(); (Thanks to Qiu, Michael) v5: - Add testpmd documentation. (Thanks to Iremonger, Bernard) v4: - Fix strings of command help. Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp> --- app/test-pmd/cmdline.c | 137 +++++++++++++++---- app/test-pmd/config.c | 102 ++++++++------ app/test-pmd/parameters.c | 22 ++- app/test-pmd/testpmd.c | 199 +++++++++++++++++++++------- app/test-pmd/testpmd.h | 18 ++- doc/guides/testpmd_app_ug/testpmd_funcs.rst | 57 ++++++++ 6 files changed, 409 insertions(+), 126 deletions(-) diff --git a/app/test-pmd/cmdline.c b/app/test-pmd/cmdline.c index 4c9f423..c8312be 100644 --- a/app/test-pmd/cmdline.c +++ b/app/test-pmd/cmdline.c @@ -513,6 +513,12 @@ static void cmd_help_long_parsed(void *parsed_result, "port close (port_id|all)\n" " Close all ports or port_id.\n\n" + "port attach (ident)\n" + " Attach physical or virtual dev by pci address or virtual device name\n\n" + + "port detach (port_id)\n" + " Detach physical or virtual dev by port_id\n\n" + "port config (port_id|all)" " speed (10|100|1000|10000|40000|auto)" " duplex (half|full|auto)\n" @@ -793,6 +799,89 @@ cmdline_parse_inst_t cmd_operate_specific_port = { }, }; +/* *** attach a specified port *** */ +struct cmd_operate_attach_port_result { + cmdline_fixed_string_t port; + cmdline_fixed_string_t keyword; + cmdline_fixed_string_t identifier; +}; + +static void cmd_operate_attach_port_parsed(void *parsed_result, + __attribute__((unused)) struct cmdline *cl, + __attribute__((unused)) void *data) +{ + struct cmd_operate_attach_port_result *res = parsed_result; + + if (!strcmp(res->keyword, "attach")) + attach_port(res->identifier); + else + printf("Unknown parameter\n"); +} + +cmdline_parse_token_string_t cmd_operate_attach_port_port = + TOKEN_STRING_INITIALIZER(struct cmd_operate_attach_port_result, + port, "port"); +cmdline_parse_token_string_t cmd_operate_attach_port_keyword = + TOKEN_STRING_INITIALIZER(struct cmd_operate_attach_port_result, + keyword, "attach"); +cmdline_parse_token_string_t cmd_operate_attach_port_identifier = + TOKEN_STRING_INITIALIZER(struct cmd_operate_attach_port_result, + identifier, NULL); + +cmdline_parse_inst_t cmd_operate_attach_port = { + .f = cmd_operate_attach_port_parsed, + .data = NULL, + .help_str = "port attach identifier, " + "identifier: pci address or virtual dev name", + .tokens = { + (void *)&cmd_operate_attach_port_port, + (void *)&cmd_operate_attach_port_keyword, + (void *)&cmd_operate_attach_port_identifier, + NULL, + }, +}; + +/* *** detach a specified port *** */ +struct cmd_operate_detach_port_result { + cmdline_fixed_string_t port; + cmdline_fixed_string_t keyword; + uint8_t port_id; +}; + +static void cmd_operate_detach_port_parsed(void *parsed_result, + __attribute__((unused)) struct cmdline *cl, + __attribute__((unused)) void *data) +{ + struct cmd_operate_detach_port_result *res = parsed_result; + + if (!strcmp(res->keyword, "detach")) + detach_port(res->port_id); + else + printf("Unknown parameter\n"); +} + +cmdline_parse_token_string_t cmd_operate_detach_port_port = + TOKEN_STRING_INITIALIZER(struct cmd_operate_detach_port_result, + port, "port"); +cmdline_parse_token_string_t cmd_operate_detach_port_keyword = + TOKEN_STRING_INITIALIZER(struct cmd_operate_detach_port_result, + keyword, "detach"); +cmdline_parse_token_num_t cmd_operate_detach_port_port_id = + TOKEN_NUM_INITIALIZER(struct cmd_operate_detach_port_result, + port_id, UINT8); + +cmdline_parse_inst_t cmd_operate_detach_port = { + .f = cmd_operate_detach_port_parsed, + .data = NULL, + .help_str = "port detach port_id", + .tokens = { + (void *)&cmd_operate_detach_port_port, + (void *)&cmd_operate_detach_port_keyword, + (void *)&cmd_operate_detach_port_port_id, + NULL, + }, +}; + /* *** configure speed for all ports *** */ struct cmd_config_speed_all { cmdline_fixed_string_t port; @@ -847,7 +936,7 @@ cmd_config_speed_all_parsed(void *parsed_result, return; } - for (pid = 0; pid < nb_ports; pid++) { + FOREACH_PORT(pid, ports) { ports[pid].dev_conf.link_speed = link_speed; ports[pid].dev_conf.link_duplex = link_duplex; } @@ -915,10 +1004,8 @@ cmd_config_speed_specific_parsed(void *parsed_result, return; } - if (res->id >= nb_ports) { - printf("Port id %d must be less than %d\n", res->id, nb_ports); + if (port_id_is_invalid(res->id, ENABLED_WARN)) return; - } if (!strcmp(res->value1, "10")) link_speed = ETH_LINK_SPEED_10; @@ -1489,7 +1576,7 @@ cmd_config_rxtx_queue_parsed(void *parsed_result, return; } - if (port_id_is_invalid(res->portid)) + if (port_id_is_invalid(res->portid, ENABLED_WARN)) return; if (port_is_started(res->portid) != 1) { @@ -2889,7 +2976,7 @@ cmd_csum_parsed(void *parsed_result, int hw = 0; uint16_t mask = 0; - if (port_id_is_invalid(res->port_id)) { + if (port_id_is_invalid(res->port_id, ENABLED_WARN)) { printf("invalid port %d\n", res->port_id); return; } @@ -2981,10 +3068,8 @@ cmd_csum_tunnel_parsed(void *parsed_result, { struct cmd_csum_tunnel_result *res = parsed_result; - if (port_id_is_invalid(res->port_id)) { - printf("invalid port %d\n", res->port_id); + if (port_id_is_invalid(res->port_id, ENABLED_WARN)) return; - } if (!strcmp(res->onoff, "on")) ports[res->port_id].tx_ol_flags |= @@ -3039,7 +3124,7 @@ cmd_tso_set_parsed(void *parsed_result, struct cmd_tso_set_result *res = parsed_result; struct rte_eth_dev_info dev_info; - if (port_id_is_invalid(res->port_id)) + if (port_id_is_invalid(res->port_id, ENABLED_WARN)) return; if (!strcmp(res->mode, "set")) @@ -4015,10 +4100,8 @@ static void cmd_set_bond_mac_addr_parsed(void *parsed_result, struct cmd_set_bond_mac_addr_result *res = parsed_result; int ret; - if (res->port_num >= nb_ports) { - printf("Port id %d must be less than %d\n", res->port_num, nb_ports); + if (port_id_is_invalid(res->port_num, ENABLED_WARN)) return; - } ret = rte_eth_bond_mac_address_set(res->port_num, &res->address); @@ -4255,7 +4338,7 @@ static void cmd_set_promisc_mode_parsed(void *parsed_result, /* all ports */ if (allports) { - for (i = 0; i < nb_ports; i++) { + FOREACH_PORT(i, ports) { if (enable) rte_eth_promiscuous_enable(i); else @@ -4335,7 +4418,7 @@ static void cmd_set_allmulti_mode_parsed(void *parsed_result, /* all ports */ if (allports) { - for (i = 0; i < nb_ports; i++) { + FOREACH_PORT(i, ports) { if (enable) rte_eth_allmulticast_enable(i); else @@ -5023,25 +5106,25 @@ static void cmd_showportall_parsed(void *parsed_result, struct cmd_showportall_result *res = parsed_result; if (!strcmp(res->show, "clear")) { if (!strcmp(res->what, "stats")) - for (i = 0; i < nb_ports; i++) + FOREACH_PORT(i, ports) nic_stats_clear(i); else if (!strcmp(res->what, "xstats")) - for (i = 0; i < nb_ports; i++) + FOREACH_PORT(i, ports) nic_xstats_clear(i); } else if (!strcmp(res->what, "info")) - for (i = 0; i < nb_ports; i++) + FOREACH_PORT(i, ports) port_infos_display(i); else if (!strcmp(res->what, "stats")) - for (i = 0; i < nb_ports; i++) + FOREACH_PORT(i, ports) nic_stats_display(i); else if (!strcmp(res->what, "xstats")) - for (i = 0; i < nb_ports; i++) + FOREACH_PORT(i, ports) nic_xstats_display(i); else if (!strcmp(res->what, "fdir")) - for (i = 0; i < nb_ports; i++) + FOREACH_PORT(i, ports) fdir_get_infos(i); else if (!strcmp(res->what, "stat_qmap")) - for (i = 0; i < nb_ports; i++) + FOREACH_PORT(i, ports) nic_stats_mapping_display(i); } @@ -8687,6 +8770,8 @@ cmdline_parse_ctx_t main_ctx[] = { (cmdline_parse_inst_t *)&cmd_set_qmap, (cmdline_parse_inst_t *)&cmd_operate_port, (cmdline_parse_inst_t *)&cmd_operate_specific_port, + (cmdline_parse_inst_t *)&cmd_operate_attach_port, + (cmdline_parse_inst_t *)&cmd_operate_detach_port, (cmdline_parse_inst_t *)&cmd_config_speed_all, (cmdline_parse_inst_t *)&cmd_config_speed_specific, (cmdline_parse_inst_t *)&cmd_config_rx_tx, @@ -8758,7 +8843,7 @@ prompt(void) static void cmd_reconfig_device_queue(portid_t id, uint8_t dev, uint8_t queue) { - if (id < nb_ports) { + if (!port_id_is_invalid(id, DISABLED_WARN)) { /* check if need_reconfig has been set to 1 */ if (ports[id].need_reconfig == 0) ports[id].need_reconfig = dev; @@ -8768,7 +8853,7 @@ cmd_reconfig_device_queue(portid_t id, uint8_t dev, uint8_t queue) } else { portid_t pid; - for (pid = 0; pid < nb_ports; pid++) { + FOREACH_PORT(pid, ports) { /* check if need_reconfig has been set to 1 */ if (ports[pid].need_reconfig == 0) ports[pid].need_reconfig = dev; @@ -8786,10 +8871,8 @@ bypass_is_supported(portid_t port_id) struct rte_port *port; struct rte_pci_id *pci_id; - if (port_id >= nb_ports) { - printf("\tPort id must be less than %d.\n", nb_ports); + if (port_id_is_invalid(port_id, ENABLED_WARN)) return 0; - } /* Get the device id. */ port = &ports[port_id]; diff --git a/app/test-pmd/config.c b/app/test-pmd/config.c index d436ce8..49be819 100644 --- a/app/test-pmd/config.c +++ b/app/test-pmd/config.c @@ -112,11 +112,15 @@ nic_stats_display(portid_t port_id) struct rte_eth_stats stats; struct rte_port *port = &ports[port_id]; uint8_t i; + portid_t pid; static const char *nic_stats_border = "########################"; - if (port_id >= nb_ports) { - printf("Invalid port, range is [0, %d]\n", nb_ports - 1); + if (port_id_is_invalid(port_id, ENABLED_WARN)) { + printf("Valid port range is [0"); + FOREACH_PORT(pid, ports) + printf(", %d", pid); + printf("]\n"); return; } rte_eth_stats_get(port_id, &stats); @@ -189,8 +193,13 @@ nic_stats_display(portid_t port_id) void nic_stats_clear(portid_t port_id) { - if (port_id >= nb_ports) { - printf("Invalid port, range is [0, %d]\n", nb_ports - 1); + portid_t pid; + + if (port_id_is_invalid(port_id, ENABLED_WARN)) { + printf("Valid port range is [0"); + FOREACH_PORT(pid, ports) + printf(", %d", pid); + printf("]\n"); return; } rte_eth_stats_reset(port_id); @@ -237,11 +246,15 @@ nic_stats_mapping_display(portid_t port_id) { struct rte_port *port = &ports[port_id]; uint16_t i; + portid_t pid; static const char *nic_stats_mapping_border = "########################"; - if (port_id >= nb_ports) { - printf("Invalid port, range is [0, %d]\n", nb_ports - 1); + if (port_id_is_invalid(port_id, ENABLED_WARN)) { + printf("Valid port range is [0"); + FOREACH_PORT(pid, ports) + printf(", %d", pid); + printf("]\n"); return; } @@ -290,9 +303,13 @@ port_infos_display(portid_t port_id) int vlan_offload; struct rte_mempool * mp; static const char *info_border = "*********************"; + portid_t pid; - if (port_id >= nb_ports) { - printf("Invalid port, range is [0, %d]\n", nb_ports - 1); + if (port_id_is_invalid(port_id, ENABLED_WARN)) { + printf("Valid port range is [0"); + FOREACH_PORT(pid, ports) + printf(", %d", pid); + printf("]\n"); return; } port = &ports[port_id]; @@ -365,11 +382,14 @@ port_infos_display(portid_t port_id) } int -port_id_is_invalid(portid_t port_id) +port_id_is_invalid(portid_t port_id, enum print_warning warning) { - if (port_id < nb_ports) + if (ports[port_id].enabled) return 0; - printf("Invalid port %d (must be < nb_ports=%d)\n", port_id, nb_ports); + + if (warning == ENABLED_WARN) + printf("Invalid port %d\n", port_id); + return 1; } @@ -428,7 +448,7 @@ port_reg_bit_display(portid_t port_id, uint32_t reg_off, uint8_t bit_x) uint32_t reg_v; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; if (port_reg_off_is_invalid(port_id, reg_off)) return; @@ -447,7 +467,7 @@ port_reg_bit_field_display(portid_t port_id, uint32_t reg_off, uint8_t l_bit; uint8_t h_bit; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; if (port_reg_off_is_invalid(port_id, reg_off)) return; @@ -474,7 +494,7 @@ port_reg_display(portid_t port_id, uint32_t reg_off) { uint32_t reg_v; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; if (port_reg_off_is_invalid(port_id, reg_off)) return; @@ -488,7 +508,7 @@ port_reg_bit_set(portid_t port_id, uint32_t reg_off, uint8_t bit_pos, { uint32_t reg_v; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; if (port_reg_off_is_invalid(port_id, reg_off)) return; @@ -516,7 +536,7 @@ port_reg_bit_field_set(portid_t port_id, uint32_t reg_off, uint8_t l_bit; uint8_t h_bit; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; if (port_reg_off_is_invalid(port_id, reg_off)) return; @@ -550,7 +570,7 @@ port_reg_bit_field_set(portid_t port_id, uint32_t reg_off, void port_reg_set(portid_t port_id, uint32_t reg_off, uint32_t reg_v) { - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; if (port_reg_off_is_invalid(port_id, reg_off)) return; @@ -563,7 +583,7 @@ port_mtu_set(portid_t port_id, uint16_t mtu) { int diag; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; diag = rte_eth_dev_set_mtu(port_id, mtu); if (diag == 0) @@ -726,7 +746,7 @@ rx_ring_desc_display(portid_t port_id, queueid_t rxq_id, uint16_t rxd_id) { const struct rte_memzone *rx_mz; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; if (rx_queue_id_is_invalid(rxq_id)) return; @@ -743,7 +763,7 @@ tx_ring_desc_display(portid_t port_id, queueid_t txq_id, uint16_t txd_id) { const struct rte_memzone *tx_mz; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; if (tx_queue_id_is_invalid(txq_id)) return; @@ -804,7 +824,7 @@ port_rss_reta_info(portid_t port_id, uint16_t i, idx, shift; int ret; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; ret = rte_eth_dev_rss_reta_query(port_id, reta_conf, nb_entries); @@ -859,7 +879,7 @@ port_rss_hash_conf_show(portid_t port_id, int show_rss_key) uint8_t i; int diag; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; /* Get RSS hash key if asked to display it */ rss_conf.rss_key = (show_rss_key) ? rss_key : NULL; @@ -1423,12 +1443,8 @@ set_fwd_ports_list(unsigned int *portlist, unsigned int nb_pt) again: for (i = 0; i < nb_pt; i++) { port_id = (portid_t) portlist[i]; - if (port_id >= nb_ports) { - printf("Invalid port id %u >= %u\n", - (unsigned int) port_id, - (unsigned int) nb_ports); + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; - } if (record_now) fwd_ports_ids[i] = port_id; } @@ -1586,7 +1602,7 @@ vlan_extend_set(portid_t port_id, int on) int diag; int vlan_offload; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; vlan_offload = rte_eth_dev_get_vlan_offload(port_id); @@ -1608,7 +1624,7 @@ rx_vlan_strip_set(portid_t port_id, int on) int diag; int vlan_offload; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; vlan_offload = rte_eth_dev_get_vlan_offload(port_id); @@ -1629,7 +1645,7 @@ rx_vlan_strip_set_on_queue(portid_t port_id, uint16_t queue_id, int on) { int diag; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; diag = rte_eth_dev_set_vlan_strip_on_queue(port_id, queue_id, on); @@ -1644,7 +1660,7 @@ rx_vlan_filter_set(portid_t port_id, int on) int diag; int vlan_offload; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; vlan_offload = rte_eth_dev_get_vlan_offload(port_id); @@ -1665,7 +1681,7 @@ rx_vft_set(portid_t port_id, uint16_t vlan_id, int on) { int diag; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; if (vlan_id_is_invalid(vlan_id)) return; @@ -1682,7 +1698,7 @@ rx_vlan_all_filter_set(portid_t port_id, int on) { uint16_t vlan_id; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; for (vlan_id = 0; vlan_id < 4096; vlan_id++) rx_vft_set(port_id, vlan_id, on); @@ -1692,7 +1708,7 @@ void vlan_tpid_set(portid_t port_id, uint16_t tp_id) { int diag; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; diag = rte_eth_dev_set_vlan_ether_type(port_id, tp_id); @@ -1707,7 +1723,7 @@ vlan_tpid_set(portid_t port_id, uint16_t tp_id) void tx_vlan_set(portid_t port_id, uint16_t vlan_id) { - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; if (vlan_id_is_invalid(vlan_id)) return; @@ -1718,7 +1734,7 @@ tx_vlan_set(portid_t port_id, uint16_t vlan_id) void tx_vlan_reset(portid_t port_id) { - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; ports[port_id].tx_ol_flags &= ~TESTPMD_TX_OFFLOAD_INSERT_VLAN; } @@ -1726,7 +1742,7 @@ tx_vlan_reset(portid_t port_id) void tx_vlan_pvid_set(portid_t port_id, uint16_t vlan_id, int on) { - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; rte_eth_dev_set_vlan_pvid(port_id, vlan_id, on); @@ -1738,7 +1754,7 @@ set_qmap(portid_t port_id, uint8_t is_rx, uint16_t queue_id, uint8_t map_value) uint16_t i; uint8_t existing_mapping_found = 0; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; if (is_rx ? (rx_queue_id_is_invalid(queue_id)) : (tx_queue_id_is_invalid(queue_id))) @@ -1904,7 +1920,7 @@ fdir_get_infos(portid_t port_id) static const char *fdir_stats_border = "########################"; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; ret = rte_eth_dev_filter_supported(port_id, RTE_ETH_FILTER_FDIR); if (ret < 0) { @@ -2030,7 +2046,7 @@ set_vf_traffic(portid_t port_id, uint8_t is_rx, uint16_t vf, uint8_t on) { int diag; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; if (is_rx) diag = rte_eth_dev_set_vf_rx(port_id,vf,on); @@ -2052,7 +2068,7 @@ set_vf_rx_vlan(portid_t port_id, uint16_t vlan_id, uint64_t vf_mask, uint8_t on) { int diag; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; if (vlan_id_is_invalid(vlan_id)) return; @@ -2069,7 +2085,7 @@ set_queue_rate_limit(portid_t port_id, uint16_t queue_idx, uint16_t rate) int diag; struct rte_eth_link link; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return 1; rte_eth_link_get_nowait(port_id, &link); if (rate > link.link_speed) { @@ -2094,7 +2110,7 @@ set_vf_rate_limit(portid_t port_id, uint16_t vf, uint16_t rate, uint64_t q_msk) if (q_msk == 0) return 0; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return 1; rte_eth_link_get_nowait(port_id, &link); if (rate > link.link_speed) { diff --git a/app/test-pmd/parameters.c b/app/test-pmd/parameters.c index 473f824..fa5f2a8 100644 --- a/app/test-pmd/parameters.c +++ b/app/test-pmd/parameters.c @@ -366,6 +366,7 @@ parse_portnuma_config(const char *q_arg) }; unsigned long int_fld[_NUM_FLD]; char *str_fld[_NUM_FLD]; + portid_t pid; /* reset from value set at definition */ while ((p = strchr(p0,'(')) != NULL) { @@ -387,8 +388,11 @@ parse_portnuma_config(const char *q_arg) return -1; } port_id = (uint8_t)int_fld[FLD_PORT]; - if (port_id >= nb_ports) { - printf("Invalid port, range is [0, %d]\n", nb_ports - 1); + if (port_id_is_invalid(port_id, ENABLED_WARN)) { + printf("Valid port range is [0"); + FOREACH_PORT(pid, ports) + printf(", %d", pid); + printf("]\n"); return -1; } socket_id = (uint8_t)int_fld[FLD_SOCKET]; @@ -419,6 +423,7 @@ parse_ringnuma_config(const char *q_arg) }; unsigned long int_fld[_NUM_FLD]; char *str_fld[_NUM_FLD]; + portid_t pid; #define RX_RING_ONLY 0x1 #define TX_RING_ONLY 0x2 #define RXTX_RING 0x3 @@ -443,8 +448,11 @@ parse_ringnuma_config(const char *q_arg) return -1; } port_id = (uint8_t)int_fld[FLD_PORT]; - if (port_id >= nb_ports) { - printf("Invalid port, range is [0, %d]\n", nb_ports - 1); + if (port_id_is_invalid(port_id, ENABLED_WARN)) { + printf("Valid port range is [0"); + FOREACH_PORT(pid, ports) + printf(", %d", pid); + printf("]\n"); return -1; } socket_id = (uint8_t)int_fld[FLD_SOCKET]; @@ -615,12 +623,12 @@ launch_args_parse(int argc, char** argv) #endif if (!strcmp(lgopts[opt_idx].name, "nb-ports")) { n = atoi(optarg); - if (n > 0 && n <= nb_ports) + if (n > 0 && + !port_id_is_invalid(n, DISABLED_WARN)) nb_fwd_ports = (uint8_t) n; else rte_exit(EXIT_FAILURE, - "nb-ports should be > 0 and <= %d\n", - nb_ports); + "Invalid port %d\n", n); } if (!strcmp(lgopts[opt_idx].name, "nb-cores")) { n = atoi(optarg); diff --git a/app/test-pmd/testpmd.c b/app/test-pmd/testpmd.c index 605163b..44bb8b3 100644 --- a/app/test-pmd/testpmd.c +++ b/app/test-pmd/testpmd.c @@ -71,6 +71,7 @@ #include <rte_pci.h> #include <rte_ether.h> #include <rte_ethdev.h> +#include <rte_dev.h> #include <rte_string_fns.h> #ifdef RTE_LIBRTE_PMD_XENVIRT #include <rte_eth_xenvirt.h> @@ -315,7 +316,7 @@ uint16_t nb_rx_queue_stats_mappings = 0; /* Forward function declarations */ static void map_port_queue_stats_mapping_registers(uint8_t pi, struct rte_port *port); -static void check_all_ports_link_status(uint8_t port_num, uint32_t port_mask); +static void check_all_ports_link_status(uint32_t port_mask); /* * Check if all the ports are started. @@ -324,6 +325,20 @@ static void check_all_ports_link_status(uint8_t port_num, uint32_t port_mask); static int all_ports_started(void); /* + * Find next enabled port + */ +portid_t +find_next_port(portid_t p, struct rte_port *ports, int size) +{ + if (ports == NULL) + rte_exit(-EINVAL, "failed to find a next port id\n"); + + while ((ports[p].enabled == 0) && (p < size)) + p++; + return p; +} + +/* * Setup default configuration. */ static void @@ -552,7 +567,8 @@ init_config(void) + RTE_TEST_TX_DESC_MAX + MAX_PKT_BURST; if (!numa_support) - nb_mbuf_per_pool = (nb_mbuf_per_pool * nb_ports); + nb_mbuf_per_pool = + (nb_mbuf_per_pool * RTE_MAX_ETHPORTS); } if (!numa_support) { @@ -565,14 +581,19 @@ init_config(void) /* Configuration of Ethernet ports. */ ports = rte_zmalloc("testpmd: ports", - sizeof(struct rte_port) * nb_ports, + sizeof(struct rte_port) * RTE_MAX_ETHPORTS, RTE_CACHE_LINE_SIZE); if (ports == NULL) { - rte_exit(EXIT_FAILURE, "rte_zmalloc(%d struct rte_port) " - "failed\n", nb_ports); + rte_exit(EXIT_FAILURE, + "rte_zmalloc(%d struct rte_port) failed\n", + RTE_MAX_ETHPORTS); } - for (pid = 0; pid < nb_ports; pid++) { + /* enabled allocated ports */ + for (pid = 0; pid < nb_ports; pid++) + ports[pid].enabled = 1; + + FOREACH_PORT(pid, ports) { port = &ports[pid]; rte_eth_dev_info_get(pid, &port->dev_info); @@ -602,8 +623,7 @@ init_config(void) nb_mbuf_per_pool = nb_mbuf_per_pool/nb_ports; for (i = 0; i < MAX_SOCKET; i++) { - nb_mbuf = (nb_mbuf_per_pool * - port_per_socket[i]); + nb_mbuf = (nb_mbuf_per_pool * RTE_MAX_ETHPORTS); if (nb_mbuf) mbuf_pool_create(mbuf_data_size, nb_mbuf,i); @@ -635,14 +655,6 @@ reconfig(portid_t new_port_id, unsigned socket_id) struct rte_port *port; /* Reconfiguration of Ethernet ports. */ - ports = rte_realloc(ports, - sizeof(struct rte_port) * nb_ports, - RTE_CACHE_LINE_SIZE); - if (ports == NULL) { - rte_exit(EXIT_FAILURE, "rte_realloc(%d struct rte_port) failed\n", - nb_ports); - } - port = &ports[new_port_id]; rte_eth_dev_info_get(new_port_id, &port->dev_info); @@ -663,7 +675,7 @@ init_fwd_streams(void) streamid_t sm_id, nb_fwd_streams_new; /* set socket id according to numa or not */ - for (pid = 0; pid < nb_ports; pid++) { + FOREACH_PORT(pid, ports) { port = &ports[pid]; if (nb_rxq > port->dev_info.max_rx_queues) { printf("Fail: nb_rxq(%d) is greater than " @@ -1264,7 +1276,7 @@ all_ports_started(void) portid_t pi; struct rte_port *port; - for (pi = 0; pi < nb_ports; pi++) { + FOREACH_PORT(pi, ports) { port = &ports[pi]; /* Check if there is a port which is not started */ if (port->port_status != RTE_PORT_STARTED) @@ -1276,6 +1288,45 @@ all_ports_started(void) } int +all_ports_stopped(void) +{ + portid_t pi; + struct rte_port *port; + + FOREACH_PORT(pi, ports) { + port = &ports[pi]; + if (port->port_status != RTE_PORT_STOPPED) + return 0; + } + + return 1; +} + +int +port_is_started(portid_t port_id) +{ + if (port_id_is_invalid(port_id, ENABLED_WARN)) + return 0; + + if (ports[port_id].port_status != RTE_PORT_STARTED) + return 0; + + return 1; +} + +static int +port_is_closed(portid_t port_id) +{ + if (port_id_is_invalid(port_id, ENABLED_WARN)) + return 0; + + if (ports[port_id].port_status != RTE_PORT_CLOSED) + return 0; + + return 1; +} + +int start_port(portid_t pid) { int diag, need_check_link_status = 0; @@ -1296,8 +1347,8 @@ start_port(portid_t pid) if(dcb_config) dcb_test = 1; - for (pi = 0; pi < nb_ports; pi++) { - if (pid < nb_ports && pid != pi) + FOREACH_PORT(pi, ports) { + if (pid != pi && pid != (portid_t)RTE_PORT_ALL) continue; port = &ports[pi]; @@ -1421,7 +1472,7 @@ start_port(portid_t pid) } if (need_check_link_status && !no_link_check) - check_all_ports_link_status(nb_ports, RTE_PORT_ALL); + check_all_ports_link_status(RTE_PORT_ALL); else printf("Please stop the ports first\n"); @@ -1446,8 +1497,8 @@ stop_port(portid_t pid) } printf("Stopping ports...\n"); - for (pi = 0; pi < nb_ports; pi++) { - if (pid < nb_ports && pid != pi) + FOREACH_PORT(pi, ports) { + if (!port_id_is_invalid(pid, DISABLED_WARN) && pid != pi) continue; port = &ports[pi]; @@ -1463,7 +1514,7 @@ stop_port(portid_t pid) need_check_link_status = 1; } if (need_check_link_status && !no_link_check) - check_all_ports_link_status(nb_ports, RTE_PORT_ALL); + check_all_ports_link_status(RTE_PORT_ALL); printf("Done\n"); } @@ -1481,8 +1532,8 @@ close_port(portid_t pid) printf("Closing ports...\n"); - for (pi = 0; pi < nb_ports; pi++) { - if (pid < nb_ports && pid != pi) + FOREACH_PORT(pi, ports) { + if (!port_id_is_invalid(pid, DISABLED_WARN) && pid != pi) continue; port = &ports[pi]; @@ -1502,31 +1553,83 @@ close_port(portid_t pid) printf("Done\n"); } -int -all_ports_stopped(void) +void +attach_port(char *identifier) { - portid_t pi; - struct rte_port *port; + portid_t i, j, pi = 0; - for (pi = 0; pi < nb_ports; pi++) { - port = &ports[pi]; - if (port->port_status != RTE_PORT_STOPPED) - return 0; + printf("Attaching a new port...\n"); + + if (identifier == NULL) { + printf("Invalid parameters are specified\n"); + return; } - return 1; + if (test_done == 0) { + printf("Please stop forwarding first\n"); + return; + } + + if (rte_eal_dev_attach(identifier, &pi)) + return; + + ports[pi].enabled = 1; + reconfig(pi, rte_eth_dev_socket_id(pi)); + rte_eth_promiscuous_enable(pi); + + nb_ports = rte_eth_dev_count(); + + /* set_default_fwd_ports_config(); */ + bzero(fwd_ports_ids, sizeof(fwd_ports_ids)); + i = 0; + FOREACH_PORT(j, ports) { + fwd_ports_ids[i] = j; + i++; + } + nb_cfg_ports = nb_ports; + nb_fwd_ports++; + + ports[pi].port_status = RTE_PORT_STOPPED; + + printf("Port %d is attached. Now total ports is %d\n", pi, nb_ports); + printf("Done\n"); } -int -port_is_started(portid_t port_id) +void +detach_port(uint8_t port_id) { - if (port_id_is_invalid(port_id)) - return -1; + portid_t i, pi = 0; + char name[RTE_ETH_NAME_MAX_LEN]; - if (ports[port_id].port_status != RTE_PORT_STARTED) - return 0; + printf("Detaching a port...\n"); - return 1; + if (!port_is_closed(port_id)) { + printf("Please close port first\n"); + return; + } + + rte_eth_promiscuous_disable(port_id); + + if (rte_eal_dev_detach(port_id, name)) + return; + + ports[port_id].enabled = 0; + nb_ports = rte_eth_dev_count(); + + /* set_default_fwd_ports_config(); */ + bzero(fwd_ports_ids, sizeof(fwd_ports_ids)); + i = 0; + FOREACH_PORT(pi, ports) { + fwd_ports_ids[i] = pi; + i++; + } + nb_cfg_ports = nb_ports; + nb_fwd_ports--; + + printf("Port '%s' is detached. Now total ports is %d\n", + name, nb_ports); + printf("Done\n"); + return; } void @@ -1534,7 +1637,7 @@ pmd_test_exit(void) { portid_t pt_id; - for (pt_id = 0; pt_id < nb_ports; pt_id++) { + FOREACH_PORT(pt_id, ports) { printf("Stopping port %d...", pt_id); fflush(stdout); rte_eth_dev_close(pt_id); @@ -1553,7 +1656,7 @@ struct pmd_test_command { /* Check the link status of all ports in up to 9s, and print them finally */ static void -check_all_ports_link_status(uint8_t port_num, uint32_t port_mask) +check_all_ports_link_status(uint32_t port_mask) { #define CHECK_INTERVAL 100 /* 100ms */ #define MAX_CHECK_TIME 90 /* 9s (90 * 100ms) in total */ @@ -1564,7 +1667,7 @@ check_all_ports_link_status(uint8_t port_num, uint32_t port_mask) fflush(stdout); for (count = 0; count <= MAX_CHECK_TIME; count++) { all_ports_up = 1; - for (portid = 0; portid < port_num; portid++) { + FOREACH_PORT(portid, ports) { if ((port_mask & (1 << portid)) == 0) continue; memset(&link, 0, sizeof(link)); @@ -1729,7 +1832,7 @@ init_port_config(void) portid_t pid; struct rte_port *port; - for (pid = 0; pid < nb_ports; pid++) { + FOREACH_PORT(pid, ports) { port = &ports[pid]; port->dev_conf.rxmode = rx_mode; port->dev_conf.fdir_conf = fdir_conf; @@ -1908,7 +2011,7 @@ main(int argc, char** argv) nb_ports = (portid_t) rte_eth_dev_count(); if (nb_ports == 0) - rte_exit(EXIT_FAILURE, "No probed ethernet device\n"); + RTE_LOG(WARNING, EAL, "No probed ethernet devices\n"); set_def_fwd_config(); if (nb_lcores == 0) @@ -1930,7 +2033,7 @@ main(int argc, char** argv) rte_exit(EXIT_FAILURE, "Start ports failed\n"); /* set all ports to promiscuous mode by default */ - for (port_id = 0; port_id < nb_ports; port_id++) + FOREACH_PORT(port_id, ports) rte_eth_promiscuous_enable(port_id); #ifdef RTE_LIBRTE_CMDLINE diff --git a/app/test-pmd/testpmd.h b/app/test-pmd/testpmd.h index 126bef7..0d5a526 100644 --- a/app/test-pmd/testpmd.h +++ b/app/test-pmd/testpmd.h @@ -137,6 +137,7 @@ struct fwd_stream { * The data structure associated with each port. */ struct rte_port { + uint8_t enabled; /**< Port enabled or not */ struct rte_eth_dev_info dev_info; /**< PCI info + driver name */ struct rte_eth_conf dev_conf; /**< Port configuration. */ struct ether_addr eth_addr; /**< Port ethernet address */ @@ -162,6 +163,14 @@ struct rte_port { struct rte_eth_txconf tx_conf; /**< tx configuration */ }; +extern portid_t __rte_unused +find_next_port(portid_t p, struct rte_port *ports, int size); + +#define FOREACH_PORT(p, ports) \ + for (p = find_next_port(0, ports, RTE_MAX_ETHPORTS); \ + p < RTE_MAX_ETHPORTS; \ + p = find_next_port(p + 1, ports, RTE_MAX_ETHPORTS)) + /** * The data structure associated with each forwarding logical core. * The logical cores are internally numbered by a core index from 0 to @@ -522,6 +531,8 @@ int init_port_dcb_config(portid_t pid,struct dcb_config *dcb_conf); int start_port(portid_t pid); void stop_port(portid_t pid); void close_port(portid_t pid); +void attach_port(char *identifier); +void detach_port(uint8_t port_id); int all_ports_stopped(void); int port_is_started(portid_t port_id); void pmd_test_exit(void); @@ -548,10 +559,15 @@ void get_syn_filter(uint8_t port_id); void get_ethertype_filter(uint8_t port_id, uint16_t index); void get_2tuple_filter(uint8_t port_id, uint16_t index); void get_5tuple_filter(uint8_t port_id, uint16_t index); -int port_id_is_invalid(portid_t port_id); int rx_queue_id_is_invalid(queueid_t rxq_id); int tx_queue_id_is_invalid(queueid_t txq_id); +enum print_warning { + ENABLED_WARN = 0, + DISABLED_WARN +}; +int port_id_is_invalid(portid_t port_id, enum print_warning warning); + /* * Work-around of a compilation error with ICC on invocations of the * rte_be_to_cpu_16() function. diff --git a/doc/guides/testpmd_app_ug/testpmd_funcs.rst b/doc/guides/testpmd_app_ug/testpmd_funcs.rst index afe1970..a99e14d 100644 --- a/doc/guides/testpmd_app_ug/testpmd_funcs.rst +++ b/doc/guides/testpmd_app_ug/testpmd_funcs.rst @@ -807,6 +807,63 @@ The following sections show functions for configuring ports. Port configuration changes only become active when forwarding is started/restarted. +port attach +~~~~~~~~~~~ + +Attach a port specified by pci address or virtual device args. + +To attach a new pci device, the device should be recognized by kernel first. +Then it should be moved under DPDK management. +Finally the port can be attached to testpmd. +On the other hand, to attach a port created by virtual device, above steps are not needed. + +port attach (identifier) + +For example, to attach a port whose pci address is 0000:02:00.0. + +.. code-block:: console + + testpmd> port attach 0000:02:00.0 + Attaching a new port... + ... snip ... + Port 0 is attached. Now total ports is 1 + Done + +For example, to attach a port created by pcap PMD. + +.. code-block:: console + + testpmd> port attach eth_pcap0,iface=eth0 + Attaching a new port... + ... snip ... + Port 0 is attached. Now total ports is 1 + Done + +In this case, identifier is "eth_pcap0,iface=eth0". +This identifier format is the same as "--vdev" format of DPDK applications. + +port detach +~~~~~~~~~~~ + +Detach a specific port. + +Before detaching a port, the port should be closed. +Also to remove a pci device completely from the system, first detach the port from testpmd. +Then the device should be moved under kernel management. +Finally the device can be removed using kernel pci hotplug functionality. +On the other hand, to remove a port created by a virtual device, above steps are not needed. + +port detach (port_id) + +For example, to detach a port 0. + +.. code-block:: console + + testpmd> port detach 0 + Detaching a port... + ... snip ... + Done + port start ~~~~~~~~~~ -- 1.9.1
2015-02-25 13:04, Tetsuya Mukawa: > --- a/lib/librte_eal/common/eal_common_dev.c > +++ b/lib/librte_eal/common/eal_common_dev.c > @@ -32,10 +32,13 @@ > * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. > */ > > +#include <stdio.h> > +#include <limits.h> > #include <string.h> > #include <inttypes.h> > #include <sys/queue.h> > > +#include <rte_ethdev.h> > #include <rte_dev.h> > #include <rte_devargs.h> No, you must not include ethdev in EAL. The ethdev layer is by design on top of EAL. Maxime already asked why you did it. He was implicitly asking to remove it. You said that you are calling ethdev_is_detachable() but you should call a function eal_is_detachable() or something like that. The detachable state must be only device-related, i.e. in EAL. The ethdev API is only a wrapper (with port id) in such case. > --- a/lib/librte_eal/linuxapp/eal/Makefile > +++ b/lib/librte_eal/linuxapp/eal/Makefile > @@ -45,6 +45,7 @@ CFLAGS += -I$(RTE_SDK)/lib/librte_eal/common/include > CFLAGS += -I$(RTE_SDK)/lib/librte_ring > CFLAGS += -I$(RTE_SDK)/lib/librte_mempool > CFLAGS += -I$(RTE_SDK)/lib/librte_malloc > +CFLAGS += -I$(RTE_SDK)/lib/librte_mbuf By removing ethdev dependency, you can remove this ugly mbuf dependency. Thanks Tetsuya
2015-02-25 20:21 GMT+09:00 Thomas Monjalon <thomas.monjalon@6wind.com>: > 2015-02-25 13:04, Tetsuya Mukawa: >> --- a/lib/librte_eal/common/eal_common_dev.c >> +++ b/lib/librte_eal/common/eal_common_dev.c >> @@ -32,10 +32,13 @@ >> * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. >> */ >> >> +#include <stdio.h> >> +#include <limits.h> >> #include <string.h> >> #include <inttypes.h> >> #include <sys/queue.h> >> >> +#include <rte_ethdev.h> >> #include <rte_dev.h> >> #include <rte_devargs.h> > > No, you must not include ethdev in EAL. > The ethdev layer is by design on top of EAL. > Maxime already asked why you did it. He was implicitly asking to remove it. > You said that you are calling ethdev_is_detachable() but you should > call a function eal_is_detachable() or something like that. > The detachable state must be only device-related, i.e. in EAL. > The ethdev API is only a wrapper (with port id) in such case. > Hi Thomas, If ethdev library is on top of EAL, hotplug functions like rte_eal_dev_attach/detach should be implemented in ethdev library. Is it right? If so, I will move rte_eal_dev_attach/detach to ethdev library. And I will change names like rte_eth_dev_attach/detach. Also, I will add "rte_dev.h" and "rte_pci.h" in rte_ethdev.h, and call below EAL functions from ethdev library. - For virtual device initialization and finalization -- rte_eth_vdev_init -- rte_eth_vdev_uninit() - For physical NIC initialization and finalization -- rte_eal_pci_probe_one() -- rte_eal_pci_close_one() I guess this will fix this design violation. Is this ok? Thanks, Tetsuya >> --- a/lib/librte_eal/linuxapp/eal/Makefile >> +++ b/lib/librte_eal/linuxapp/eal/Makefile >> @@ -45,6 +45,7 @@ CFLAGS += -I$(RTE_SDK)/lib/librte_eal/common/include >> CFLAGS += -I$(RTE_SDK)/lib/librte_ring >> CFLAGS += -I$(RTE_SDK)/lib/librte_mempool >> CFLAGS += -I$(RTE_SDK)/lib/librte_malloc >> +CFLAGS += -I$(RTE_SDK)/lib/librte_mbuf > > By removing ethdev dependency, you can remove this ugly mbuf dependency. > > Thanks Tetsuya >
2015-02-25 21:32, Tetsuya Mukawa: > 2015-02-25 20:21 GMT+09:00 Thomas Monjalon <thomas.monjalon@6wind.com>: > > 2015-02-25 13:04, Tetsuya Mukawa: > >> --- a/lib/librte_eal/common/eal_common_dev.c > >> +++ b/lib/librte_eal/common/eal_common_dev.c > >> @@ -32,10 +32,13 @@ > >> * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. > >> */ > >> > >> +#include <stdio.h> > >> +#include <limits.h> > >> #include <string.h> > >> #include <inttypes.h> > >> #include <sys/queue.h> > >> > >> +#include <rte_ethdev.h> > >> #include <rte_dev.h> > >> #include <rte_devargs.h> > > > > No, you must not include ethdev in EAL. > > The ethdev layer is by design on top of EAL. > > Maxime already asked why you did it. He was implicitly asking to remove it. > > You said that you are calling ethdev_is_detachable() but you should > > call a function eal_is_detachable() or something like that. > > The detachable state must be only device-related, i.e. in EAL. > > The ethdev API is only a wrapper (with port id) in such case. > > > > Hi Thomas, > > If ethdev library is on top of EAL, hotplug functions like > rte_eal_dev_attach/detach should be implemented in ethdev library. > Is it right? Yes you're right. > If so, I will move rte_eal_dev_attach/detach to ethdev library. > And I will change names like rte_eth_dev_attach/detach. It seems to be the right thing to do. > Also, I will add "rte_dev.h" and "rte_pci.h" in rte_ethdev.h, and call > below EAL functions from ethdev library. > > - For virtual device initialization and finalization > -- rte_eth_vdev_init > -- rte_eth_vdev_uninit() > - For physical NIC initialization and finalization > -- rte_eal_pci_probe_one() > -- rte_eal_pci_close_one() > > I guess this will fix this design violation. > Is this ok? I think yes. If needed, we could do some cleanup after RC1. I'm just waiting for you fixing this, to avoid introducing a layering violation. Would you able to do it today? Thanks > >> --- a/lib/librte_eal/linuxapp/eal/Makefile > >> +++ b/lib/librte_eal/linuxapp/eal/Makefile > >> @@ -45,6 +45,7 @@ CFLAGS += -I$(RTE_SDK)/lib/librte_eal/common/include > >> CFLAGS += -I$(RTE_SDK)/lib/librte_ring > >> CFLAGS += -I$(RTE_SDK)/lib/librte_mempool > >> CFLAGS += -I$(RTE_SDK)/lib/librte_malloc > >> +CFLAGS += -I$(RTE_SDK)/lib/librte_mbuf > > > > By removing ethdev dependency, you can remove this ugly mbuf dependency. > > > > Thanks Tetsuya > >
On 2015/02/25 23:00, Thomas Monjalon wrote: > 2015-02-25 21:32, Tetsuya Mukawa: >> 2015-02-25 20:21 GMT+09:00 Thomas Monjalon <thomas.monjalon@6wind.com>: >>> 2015-02-25 13:04, Tetsuya Mukawa: >>>> --- a/lib/librte_eal/common/eal_common_dev.c >>>> +++ b/lib/librte_eal/common/eal_common_dev.c >>>> @@ -32,10 +32,13 @@ >>>> * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. >>>> */ >>>> >>>> +#include <stdio.h> >>>> +#include <limits.h> >>>> #include <string.h> >>>> #include <inttypes.h> >>>> #include <sys/queue.h> >>>> >>>> +#include <rte_ethdev.h> >>>> #include <rte_dev.h> >>>> #include <rte_devargs.h> >>> No, you must not include ethdev in EAL. >>> The ethdev layer is by design on top of EAL. >>> Maxime already asked why you did it. He was implicitly asking to remove it. >>> You said that you are calling ethdev_is_detachable() but you should >>> call a function eal_is_detachable() or something like that. >>> The detachable state must be only device-related, i.e. in EAL. >>> The ethdev API is only a wrapper (with port id) in such case. >>> >> Hi Thomas, >> >> If ethdev library is on top of EAL, hotplug functions like >> rte_eal_dev_attach/detach should be implemented in ethdev library. >> Is it right? > Yes you're right. > >> If so, I will move rte_eal_dev_attach/detach to ethdev library. >> And I will change names like rte_eth_dev_attach/detach. > It seems to be the right thing to do. > >> Also, I will add "rte_dev.h" and "rte_pci.h" in rte_ethdev.h, and call >> below EAL functions from ethdev library. >> >> - For virtual device initialization and finalization >> -- rte_eth_vdev_init >> -- rte_eth_vdev_uninit() >> - For physical NIC initialization and finalization >> -- rte_eal_pci_probe_one() >> -- rte_eal_pci_close_one() >> >> I guess this will fix this design violation. >> Is this ok? > I think yes. > If needed, we could do some cleanup after RC1. > I'm just waiting for you fixing this, to avoid introducing > a layering violation. > Would you able to do it today? Hi Thomas, I appreciate for your reply. I start trying it. Thanks, Tetsuya > Thanks > >>>> --- a/lib/librte_eal/linuxapp/eal/Makefile >>>> +++ b/lib/librte_eal/linuxapp/eal/Makefile >>>> @@ -45,6 +45,7 @@ CFLAGS += -I$(RTE_SDK)/lib/librte_eal/common/include >>>> CFLAGS += -I$(RTE_SDK)/lib/librte_ring >>>> CFLAGS += -I$(RTE_SDK)/lib/librte_mempool >>>> CFLAGS += -I$(RTE_SDK)/lib/librte_malloc >>>> +CFLAGS += -I$(RTE_SDK)/lib/librte_mbuf >>> By removing ethdev dependency, you can remove this ugly mbuf dependency. >>> >>> Thanks Tetsuya >>> >
This patch series adds a dynamic port hotplug framework to DPDK. With the patches, DPDK apps can attach or detach ports at runtime. The basic concept of the port hotplug is like followings. - DPDK apps must have responsibility to manage ports. DPDK apps only know which ports are attached or detached at the moment. The port hotplug framework is implemented to allow DPDK apps to manage ports. For example, when DPDK apps call port attach function, attached port number will be returned. Also, DPDK apps can detach port by port number. - Kernel support is needed for attaching or detaching physical device ports. To attach a new physical device port, the device will be recognized by userspace directly I/O framework in kernel at first. Then DPDK apps can call the port hotplug functions to attach ports. For detaching, steps are vice versa. - Before detach ports, ports must be stopped and closed. DPDK application must call rte_eth_dev_stop() and rte_eth_dev_close() before detaching ports. These function will call finalization codes of PMDs. But so far, no PMD frees all resources allocated by initialization. It means PMDs are needed to be fixed to support the port hotplug. 'RTE_PCI_DRV_DETACHABLE' is a new flag indicating a PMD supports detaching. Without this flag, detaching will be failed. - Mustn't affect legacy DPDK apps. No DPDK EAL behavior is changed, if the port hotplug functions are't called. So all legacy DPDK apps can still work without modifications. And a few limitations. - The port hotplug functions are not thread safe. DPDK apps should handle it. - Only support Linux and igb_uio so far. BSD and VFIO is not supported. I will send VFIO patches at least, but I don't have a plan to submit BSD patch so far. Here is port hotplug APIs. ------------------------------------------------------------------------------- /** * Attach a new device. * * @param devargs * A pointer to a strings array describing the new device * to be attached. The strings should be a pci address like * '0000:01:00.0' or virtual device name like 'eth_pcap0'. * @param port_id * A pointer to a port identifier actually attached. * @return * 0 on success and port_id is filled, negative on error */ int rte_eth_dev_attach(const char *devargs, uint8_t *port_id); /** * Detach a device. * * @param port_id * The port identifier of the device to detach. * @param addr * A pointer to a device name actually detached. * @return * 0 on success and devname is filled, negative on error */ int rte_eth_dev_detach(uint8_t port_id, char *devname); ------------------------------------------------------------------------------- This patch series are for DPDK EAL. To use port hotplug function by DPDK apps, each PMD should be fixed to support 'RTE_PCI_DRV_DETACHABLE' flag. Please check a patch for pcap PMD. Also, please check testpmd patch. It will show you how to fix your legacy applications to support port hotplug feature. PATCH v15 changes - Fix issue that eal calls ethedv library APIs. - Remove rte_eal_dev_attach(), and add rte_eth_dev_attach(). - Remove rte_eal_dev_detach(), and add rte_eth_dev_detach(). - Call rte_eal_vdev_init/uninit from ethdev library. (Thanks to Thomas Monjalon) - Fix version.map - Reorder and squash patches to compile correctly. - Remove needless symbols in version.map - Fix version.map. PATCH v14 changes - Remove needless if statement. (Thanks to Maxime Leroy) PATCH v13 changes - Change log level when error occurs in rte_eal_vdev_init() and rte_eal_dev_init(). - Return value of driver init and uninit functions. - Replace rte_panic by RTE_LOG in rte_eal_dev_init() - Fix return value of rte_eal_vdev_uninit(). - Fix rte_eal_dev_attach_vdev to set port_id correctly. (Thanks to Maxime Leroy) PATCH v12 changes - Add missing symbol in version map. (Thanks to Iremonger, Bernard) PATCH v11 changes - Remove needless devargs handling codes. - Replace get_vdev_name() by rte_eal_parse_devargs_str(). - Replace rte_eal_vdev_find_and_init by rte_eal_vdev_init() - Replace rte_eal_vdev_find_and_uninit by rte_eal_vdev_uninit() - Fix rte_eal_dev_init() to use rte_eal_vdev_init(). - Remove needless patch. (Thanks to Maxime Leroy) PATCH v10 changes - Add comments. - Chagne order of version.map. - Fix comment of "rte_ethdev.h". (Thanks to Thomas Monjalon) - Add size parameter to rte_eth_dev_create_unique_device_name(). (Thanks to Iremonger, Bernard) PATCH v9 changes - Fix commit title. - Fix commit log. - Fix comments. - Define CONFIG_RTE_LIBRTE_EAL_HOTPLUG at the top of this patch series. - DEV_INVALID/VALID are removed. - DEV_DISCONNECTED is replaced by DEV_DETACHED. - DEV_CONNECTED is replaced by DEV_ATTACHED. - rte_eth_dev_allocate_new_port() is renamed to rte_eth_dev_find_free_port(). - rte_eth_dev_validate_port() is renamed to rte_eth_dev_is_valid_port(). - rte_eth_dev_is_valid_port() is changed not to handle log toggle. - eal_compare_pci_addr() is replaced by rte_eal_compare_pci_addr(). - rte_eth_dev_free() is replaced by rte_eth_dev_release_port(). - Add a function to create a unique device name. - Change parameter of pci_devuninit_t and rte_eth_dev_uninit. - Remove code that initiaize callback of ethdev from rte_eth_dev_uninit(). - Remove pci_unmap_device(). It will be implemented in later patch. - rte_eth_dev_check_detachable() is replaced by rte_eth_dev_is_detachable(). - strncpy() is replaced by strcpy(). - Implement pci_unmap_device() in this patch. - Remove "rte_dev_hotplug.h". - Remove needless "#ifdef". - Remove RTE_EAL_INVOKE_TYPE_PROBE/CLOSE. - RTE_ETH_DEV_PHYSICAL is replaced by RTE_ETH_DEV_PCI. - Use strcmp() instead of strncmp(). - Remove RTE_EAL_INVOKE_TYPE_PROBE/CLOSE. (Thanks to Thomas Monjalon) - Change definition of rte_dev_uninit_t. (Thanks to Thomas Monjalon and Maxime Leroy) - Add missing symbol in version map. (Thanks to Nail Horman) PATCH v8 changes - Fix Makefile and add version map file. - Add missing symbol in version map. - Fix pci_scan_one() to update sysfs values. (Thanks to Qiu, Michael and Iremonger, Bernard) - NONE_TRACE is replaced by NO_TRACE. - Fix typo. - Add size parameter to rte_eth_dev_save(). (Thanks to Iremonger, Bernard) PATCH v7 changes - Add a new section to programmer's guide. (Thanks to Iremonger, Bernard) - Fix port checking implementation of star_port(). - Fix typo of warning messages. - Add pt_driver checking to rte_eth_dev_check_detachable(). (Thanks to Qiu, Michael) PATCH v6 changes - Fix rte_eth_dev_uninit() to handle a return value of uninit function of PMD. To do this, below changes also be applied. - Fix a parameter of rte_eth_dev_free(). - Use rte_eth_dev structure as the paramter of rte_eth_dev_free(). PATCH v5 changes - Add runtime check passthrough driver type, like vfio-pci, igb_uio and uio_pci_generic. This was done by Qiu, Michael. Thanks a lot. - Change function names like below. - rte_eal_dev_find_and_invoke() to rte_eal_vdev_find_and_invoke(). - rte_eal_dev_invoke() to rte_eal_vdev_invoke(). - Add code to handle a return value of rte_eal_devargs_remove(). - Fix pci address format in rte_eal_dev_detach(). - Remove RTE_EAL_INVOKE_TYPE_UNKNOWN, because it's unused. - Change function definition of rte_eal_devargs_remove(). - Fix pci_unmap_device() to check pt_driver. - Fix return value of below functions. - rte_eth_dev_get_changed_port(). - rte_eth_dev_get_port_by_addr(). - Change paramters of rte_eth_dev_validate_port() to cleanup code. - Fix pci_scan_one to handle pt_driver correctly. (Thanks to Qiu, Michael for above suggestions) PATCH v4 changes - Merge patches to review easier. - Fix indent of 'if' statement. - Fix calculation method of eal_compare_pci_addr(). - Fix header file declaration. - Add header file to determine if hotplug can be enabled. (Thanks to Qiu, Michael) - Use braces with 'for' loop. - Add parameter checking. - Fix sanity check code - Fix comments of rte_eth_dev_type. - Change function names. (Thanks to Iremonger, Bernard) PATCH v3 changes: - Fix enum definition used in rte_ethdev.c. (Thanks to Zhang, Helin) PATCH v2 changes: - Replace rte_eal_dev_attach_pdev(), rte_eal_dev_detach_pdev, rte_eal_dev_attach_vdev() and rte_eal_dev_detach_vdev() to rte_eal_dev_attach() and rte_eal_dev_detach(). - Add parameter values checking. - Refashion a few functions. (Thanks to Iremonger, Bernard) PATCH v1 Changes: - Fix error checking code of librte_eth APIs. - Fix issue that port from pcap PMD cannot be detached correctly. - Fix issue that testpmd could hang after forwarding, if attaching and detaching is repeatedly. - Fix if-condition of rte_eth_dev_get_port_by_addr(). (Thanks to Mark Enright) RFC PATCH v2 Changes: - remove 'rte_eth_dev_validate_port()', and cleanup codes. Michael Qiu (2): eal_pci: Add flag to hold kernel driver type eal_pci: pci memory map work with driver type Tetsuya Mukawa (11): eal: Enable port Hotplug framework in Linux eal/pci,ethdev: Remove assumption that port will not be detached eal/pci: Consolidate pci address comparison APIs ethdev: Add rte_eth_dev_release_port to release specified port eal,ethdev: Add a function and function pointers to close ether device eal/linux/pci: Add functions for unmapping igb_uio resources eal/pci: Add probe and close functions of pci driver ethdev: Add one dev_type parameter to rte_eth_dev_allocate eal/pci: Add vdev driver initialization and uninitialization functions ethdev: Add rte_eth_dev_attach/detach() functions doc: Add port hotplug framework section to programmers guide app/test/virtual_pmd.c | 2 +- config/common_bsdapp | 6 + config/common_linuxapp | 5 + doc/guides/prog_guide/index.rst | 1 + doc/guides/prog_guide/port_hotplug_framework.rst | 110 ++++ lib/librte_eal/bsdapp/eal/eal_pci.c | 29 +- lib/librte_eal/common/eal_common_dev.c | 77 ++- lib/librte_eal/common/eal_common_devargs.c | 54 +- lib/librte_eal/common/eal_common_pci.c | 100 +++- lib/librte_eal/common/eal_private.h | 15 + lib/librte_eal/common/include/rte_dev.h | 28 + lib/librte_eal/common/include/rte_devargs.h | 28 + lib/librte_eal/common/include/rte_pci.h | 91 ++++ lib/librte_eal/linuxapp/eal/eal_pci.c | 230 ++++++-- lib/librte_eal/linuxapp/eal/eal_pci_init.h | 7 + lib/librte_eal/linuxapp/eal/eal_pci_uio.c | 67 ++- lib/librte_eal/linuxapp/eal/rte_eal_version.map | 6 + lib/librte_ether/rte_ethdev.c | 635 +++++++++++++++++++---- lib/librte_ether/rte_ethdev.h | 95 +++- lib/librte_ether/rte_ether_version.map | 3 + lib/librte_pmd_af_packet/rte_eth_af_packet.c | 2 +- lib/librte_pmd_bond/rte_eth_bond_api.c | 2 +- lib/librte_pmd_pcap/rte_eth_pcap.c | 2 +- lib/librte_pmd_ring/rte_eth_ring.c | 2 +- lib/librte_pmd_xenvirt/rte_eth_xenvirt.c | 2 +- 25 files changed, 1405 insertions(+), 194 deletions(-) create mode 100644 doc/guides/prog_guide/port_hotplug_framework.rst -- 1.9.1
The patch adds CONFIG_RTE_LIBRTE_EAL_HOTPLUG in Linux and BSD configuration. So far, Hotplug functions only support linux. v9: - Move this patch at the top of this patch series. (Thanks to Thomas Monjalon) Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp> --- config/common_bsdapp | 6 ++++++ config/common_linuxapp | 5 +++++ 2 files changed, 11 insertions(+) diff --git a/config/common_bsdapp b/config/common_bsdapp index 83a62a6..4108c01 100644 --- a/config/common_bsdapp +++ b/config/common_bsdapp @@ -116,6 +116,12 @@ CONFIG_RTE_LIBRTE_EAL_BSDAPP=y CONFIG_RTE_LIBRTE_EAL_LINUXAPP=n # +# Compile Environment Abstraction Layer to support hotplug +# So far, Hotplug functions only support linux +# +CONFIG_RTE_LIBRTE_EAL_HOTPLUG=n + +# # Compile Environment Abstraction Layer to support Vmware TSC map # CONFIG_RTE_LIBRTE_EAL_VMWARE_TSC_MAP_SUPPORT=y diff --git a/config/common_linuxapp b/config/common_linuxapp index 2716381..8ba0258 100644 --- a/config/common_linuxapp +++ b/config/common_linuxapp @@ -114,6 +114,11 @@ CONFIG_RTE_PCI_MAX_READ_REQUEST_SIZE=0 CONFIG_RTE_LIBRTE_EAL_LINUXAPP=y # +# Compile Environment Abstraction Layer to support hotplug +# +CONFIG_RTE_LIBRTE_EAL_HOTPLUG=y + +# # Compile Environment Abstraction Layer to support Vmware TSC map # CONFIG_RTE_LIBRTE_EAL_VMWARE_TSC_MAP_SUPPORT=y -- 1.9.1
From: Michael Qiu <michael.qiu@intel.com> Currently, dpdk has no ability to know which type of driver( vfio-pci/igb_uio/uio_pci_generic) the device used. It only can check whether vfio is enabled or not staticly. It really useful to have the flag, becasue different type need to handle differently in runtime. For example, pci memory map, pot hotplug, and so on. This patch add a flag field for pci device to solve above issue. Signed-off-by: Michael Qiu <michael.qiu@intel.com> Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp> --- lib/librte_eal/common/include/rte_pci.h | 8 +++++ lib/librte_eal/linuxapp/eal/eal_pci.c | 53 +++++++++++++++++++++++++++++++-- 2 files changed, 59 insertions(+), 2 deletions(-) diff --git a/lib/librte_eal/common/include/rte_pci.h b/lib/librte_eal/common/include/rte_pci.h index 3df07e8..a87b4b3 100644 --- a/lib/librte_eal/common/include/rte_pci.h +++ b/lib/librte_eal/common/include/rte_pci.h @@ -142,6 +142,13 @@ struct rte_pci_addr { struct rte_devargs; +enum rte_pt_driver { + RTE_PT_UNKNOWN = 0, + RTE_PT_IGB_UIO = 1, + RTE_PT_VFIO = 2, + RTE_PT_UIO_GENERIC = 3, +}; + /** * A structure describing a PCI device. */ @@ -155,6 +162,7 @@ struct rte_pci_device { uint16_t max_vfs; /**< sriov enable if not zero */ int numa_node; /**< NUMA node connection */ struct rte_devargs *devargs; /**< Device user arguments */ + enum rte_pt_driver pt_driver; /**< Driver of passthrough */ }; /** Any PCI device identifier (vendor, device, ...) */ diff --git a/lib/librte_eal/linuxapp/eal/eal_pci.c b/lib/librte_eal/linuxapp/eal/eal_pci.c index a4fd5f5..4615756 100644 --- a/lib/librte_eal/linuxapp/eal/eal_pci.c +++ b/lib/librte_eal/linuxapp/eal/eal_pci.c @@ -97,6 +97,35 @@ error: return -1; } +static int +pci_get_kernel_driver_by_path(const char *filename, char *dri_name) +{ + int count; + char path[PATH_MAX]; + char *name; + + if (!filename || !dri_name) + return -1; + + count = readlink(filename, path, PATH_MAX); + if (count >= PATH_MAX) + return -1; + + /* For device does not have a driver */ + if (count < 0) + return 1; + + path[count] = '\0'; + + name = strrchr(path, '/'); + if (name) { + strncpy(dri_name, name + 1, strlen(name + 1) + 1); + return 0; + } + + return -1; +} + void * pci_find_max_end_va(void) { @@ -221,11 +250,12 @@ pci_scan_one(const char *dirname, uint16_t domain, uint8_t bus, char filename[PATH_MAX]; unsigned long tmp; struct rte_pci_device *dev; + char driver[PATH_MAX]; + int ret; dev = malloc(sizeof(*dev)); - if (dev == NULL) { + if (dev == NULL) return -1; - } memset(dev, 0, sizeof(*dev)); dev->addr.domain = domain; @@ -304,6 +334,25 @@ pci_scan_one(const char *dirname, uint16_t domain, uint8_t bus, return -1; } + /* parse driver */ + snprintf(filename, sizeof(filename), "%s/driver", dirname); + ret = pci_get_kernel_driver_by_path(filename, driver); + if (!ret) { + if (!strcmp(driver, "vfio-pci")) + dev->pt_driver = RTE_PT_VFIO; + else if (!strcmp(driver, "igb_uio")) + dev->pt_driver = RTE_PT_IGB_UIO; + else if (!strcmp(driver, "uio_pci_generic")) + dev->pt_driver = RTE_PT_UIO_GENERIC; + else + dev->pt_driver = RTE_PT_UNKNOWN; + } else if (ret < 0) { + RTE_LOG(ERR, EAL, "Fail to get kernel driver\n"); + free(dev); + return -1; + } else + dev->pt_driver = RTE_PT_UNKNOWN; + /* device is valid, add in list (sorted) */ if (TAILQ_EMPTY(&pci_device_list)) { TAILQ_INSERT_TAIL(&pci_device_list, dev, next); -- 1.9.1
From: Michael Qiu <michael.qiu@intel.com> With the driver type flag in struct rte_pci_dev, we do not need to always map uio devices with vfio related function when vfio enabled. Signed-off-by: Michael Qiu <michael.qiu@intel.com> Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp> --- lib/librte_eal/linuxapp/eal/eal_pci.c | 30 +++++++++++++++++------------- 1 file changed, 17 insertions(+), 13 deletions(-) diff --git a/lib/librte_eal/linuxapp/eal/eal_pci.c b/lib/librte_eal/linuxapp/eal/eal_pci.c index 4615756..3291c68 100644 --- a/lib/librte_eal/linuxapp/eal/eal_pci.c +++ b/lib/librte_eal/linuxapp/eal/eal_pci.c @@ -555,25 +555,29 @@ pci_config_space_set(struct rte_pci_device *dev) static int pci_map_device(struct rte_pci_device *dev) { - int ret, mapped = 0; + int ret = -1; /* try mapping the NIC resources using VFIO if it exists */ + switch (dev->pt_driver) { + case RTE_PT_VFIO: #ifdef VFIO_PRESENT - if (pci_vfio_is_enabled()) { - ret = pci_vfio_map_resource(dev); - if (ret == 0) - mapped = 1; - else if (ret < 0) - return ret; - } + if (pci_vfio_is_enabled()) + ret = pci_vfio_map_resource(dev); #endif - /* map resources for devices that use uio_pci_generic or igb_uio */ - if (!mapped) { + break; + case RTE_PT_IGB_UIO: + case RTE_PT_UIO_GENERIC: + /* map resources for devices that use uio */ ret = pci_uio_map_resource(dev); - if (ret != 0) - return ret; + break; + default: + RTE_LOG(DEBUG, EAL, " Not managed by known pt driver," + " skipped\n"); + ret = 1; + break; } - return 0; + + return ret; } /* -- 1.9.1
To remove assumption, do like followings. This patch adds "RTE_PCI_DRV_DETACHABLE" to drv_flags of rte_pci_driver structure. The flags indicate the driver can detach devices at runtime. Also, remove assumption that port will not be detached. To remove the assumption. - Add 'attached' member to rte_eth_dev structure. This member is used for indicating the port is attached, or not. DEV_ATTACHED indicates a port is attached. DEV_DETACHED indicates a port is detached. - Add rte_eth_dev_allocate_new_port(). This function is used for allocating new port. v9: - DEV_INVALID/VALID are removed. - DEV_DISCONNECTED is replaced by DEV_DETACHED. - DEV_CONNECTED is replaced by DEV_ATTACHED. - rte_eth_dev_allocate_new_port() is renamed to rte_eth_dev_find_free_port(). - rte_eth_dev_validate_port() is renamed to rte_eth_dev_is_valid_port(). - rte_eth_dev_is_valid_port() is changed not to handle log toggle. - Fix commit log to describe DEV_ATACHED and DEV_DETACHED. (Thanks to Thomas Monjalon) v8: - NONE_TRACE is changed to NO_TRACE. (Thanks to Iremonger, Bernard) v5: - Change parameters of rte_eth_dev_validate_port() to cleanup code. v4: - Use braces with 'for' loop. - Fix indent of 'if' statement. Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp> --- lib/librte_eal/common/include/rte_pci.h | 2 + lib/librte_ether/rte_ethdev.c | 248 ++++++++++++++++++++------------ lib/librte_ether/rte_ethdev.h | 5 + 3 files changed, 164 insertions(+), 91 deletions(-) diff --git a/lib/librte_eal/common/include/rte_pci.h b/lib/librte_eal/common/include/rte_pci.h index a87b4b3..255a77b 100644 --- a/lib/librte_eal/common/include/rte_pci.h +++ b/lib/librte_eal/common/include/rte_pci.h @@ -210,6 +210,8 @@ struct rte_pci_driver { #define RTE_PCI_DRV_FORCE_UNBIND 0x0004 /** Device driver supports link state interrupt */ #define RTE_PCI_DRV_INTR_LSC 0x0008 +/** Device driver supports detaching capability */ +#define RTE_PCI_DRV_DETACHABLE 0x0010 /**< Internal use only - Macro used by pci addr parsing functions **/ #define GET_PCIADDR_FIELD(in, fd, lim, dlm) \ diff --git a/lib/librte_ether/rte_ethdev.c b/lib/librte_ether/rte_ethdev.c index ecbe93c..b702039 100644 --- a/lib/librte_ether/rte_ethdev.c +++ b/lib/librte_ether/rte_ethdev.c @@ -175,6 +175,11 @@ enum { STAT_QMAP_RX }; +enum { + DEV_DETACHED = 0, + DEV_ATTACHED +}; + static inline void rte_eth_dev_data_alloc(void) { @@ -201,19 +206,34 @@ rte_eth_dev_allocated(const char *name) { unsigned i; - for (i = 0; i < nb_ports; i++) { - if (strcmp(rte_eth_devices[i].data->name, name) == 0) + for (i = 0; i < RTE_MAX_ETHPORTS; i++) { + if ((rte_eth_devices[i].attached == DEV_ATTACHED) && + strcmp(rte_eth_devices[i].data->name, name) == 0) return &rte_eth_devices[i]; } return NULL; } +static uint8_t +rte_eth_dev_find_free_port(void) +{ + unsigned i; + + for (i = 0; i < RTE_MAX_ETHPORTS; i++) { + if (rte_eth_devices[i].attached == DEV_DETACHED) + return i; + } + return RTE_MAX_ETHPORTS; +} + struct rte_eth_dev * rte_eth_dev_allocate(const char *name) { + uint8_t port_id; struct rte_eth_dev *eth_dev; - if (nb_ports == RTE_MAX_ETHPORTS) { + port_id = rte_eth_dev_find_free_port(); + if (port_id == RTE_MAX_ETHPORTS) { PMD_DEBUG_TRACE("Reached maximum number of Ethernet ports\n"); return NULL; } @@ -226,10 +246,12 @@ rte_eth_dev_allocate(const char *name) return NULL; } - eth_dev = &rte_eth_devices[nb_ports]; - eth_dev->data = &rte_eth_dev_data[nb_ports]; + eth_dev = &rte_eth_devices[port_id]; + eth_dev->data = &rte_eth_dev_data[port_id]; snprintf(eth_dev->data->name, sizeof(eth_dev->data->name), "%s", name); - eth_dev->data->port_id = nb_ports++; + eth_dev->data->port_id = port_id; + eth_dev->attached = DEV_ATTACHED; + nb_ports++; return eth_dev; } @@ -283,6 +305,7 @@ rte_eth_dev_init(struct rte_pci_driver *pci_drv, (unsigned) pci_dev->id.device_id); if (rte_eal_process_type() == RTE_PROC_PRIMARY) rte_free(eth_dev->data->dev_private); + eth_dev->attached = DEV_DETACHED; nb_ports--; return diag; } @@ -308,10 +331,20 @@ rte_eth_driver_register(struct eth_driver *eth_drv) rte_eal_pci_register(ð_drv->pci_drv); } +static int +rte_eth_dev_is_valid_port(uint8_t port_id) +{ + if (port_id >= RTE_MAX_ETHPORTS || + rte_eth_devices[port_id].attached != DEV_ATTACHED) + return 0; + else + return 1; +} + int rte_eth_dev_socket_id(uint8_t port_id) { - if (port_id >= nb_ports) + if (!rte_eth_dev_is_valid_port(port_id)) return -1; return rte_eth_devices[port_id].pci_dev->numa_node; } @@ -395,7 +428,7 @@ rte_eth_dev_rx_queue_start(uint8_t port_id, uint16_t rx_queue_id) * in a multi-process setup*/ PROC_PRIMARY_OR_ERR_RET(-E_RTE_SECONDARY); - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return -EINVAL; } @@ -421,7 +454,7 @@ rte_eth_dev_rx_queue_stop(uint8_t port_id, uint16_t rx_queue_id) * in a multi-process setup*/ PROC_PRIMARY_OR_ERR_RET(-E_RTE_SECONDARY); - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return -EINVAL; } @@ -447,7 +480,7 @@ rte_eth_dev_tx_queue_start(uint8_t port_id, uint16_t tx_queue_id) * in a multi-process setup*/ PROC_PRIMARY_OR_ERR_RET(-E_RTE_SECONDARY); - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return -EINVAL; } @@ -473,7 +506,7 @@ rte_eth_dev_tx_queue_stop(uint8_t port_id, uint16_t tx_queue_id) * in a multi-process setup*/ PROC_PRIMARY_OR_ERR_RET(-E_RTE_SECONDARY); - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return -EINVAL; } @@ -755,10 +788,11 @@ rte_eth_dev_configure(uint8_t port_id, uint16_t nb_rx_q, uint16_t nb_tx_q, * in a multi-process setup*/ PROC_PRIMARY_OR_ERR_RET(-E_RTE_SECONDARY); - if (port_id >= nb_ports || port_id >= RTE_MAX_ETHPORTS) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-EINVAL); } + dev = &rte_eth_devices[port_id]; FUNC_PTR_OR_ERR_RET(*dev->dev_ops->dev_infos_get, -ENOTSUP); @@ -940,10 +974,11 @@ rte_eth_dev_start(uint8_t port_id) * in a multi-process setup*/ PROC_PRIMARY_OR_ERR_RET(-E_RTE_SECONDARY); - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%" PRIu8 "\n", port_id); + if (!rte_eth_dev_is_valid_port(port_id)) { + PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-EINVAL); } + dev = &rte_eth_devices[port_id]; FUNC_PTR_OR_ERR_RET(*dev->dev_ops->dev_start, -ENOTSUP); @@ -979,10 +1014,11 @@ rte_eth_dev_stop(uint8_t port_id) * in a multi-process setup*/ PROC_PRIMARY_OR_RET(); - if (port_id >= nb_ports) { - PMD_DEBUG_TRACE("Invalid port_id=%" PRIu8 "\n", port_id); + if (!rte_eth_dev_is_valid_port(port_id)) { + PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return; } + dev = &rte_eth_devices[port_id]; FUNC_PTR_OR_RET(*dev->dev_ops->dev_stop); @@ -1007,10 +1043,11 @@ rte_eth_dev_set_link_up(uint8_t port_id) * in a multi-process setup*/ PROC_PRIMARY_OR_ERR_RET(-E_RTE_SECONDARY); - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return -EINVAL; } + dev = &rte_eth_devices[port_id]; FUNC_PTR_OR_ERR_RET(*dev->dev_ops->dev_set_link_up, -ENOTSUP); @@ -1026,10 +1063,11 @@ rte_eth_dev_set_link_down(uint8_t port_id) * in a multi-process setup*/ PROC_PRIMARY_OR_ERR_RET(-E_RTE_SECONDARY); - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return -EINVAL; } + dev = &rte_eth_devices[port_id]; FUNC_PTR_OR_ERR_RET(*dev->dev_ops->dev_set_link_down, -ENOTSUP); @@ -1045,7 +1083,7 @@ rte_eth_dev_close(uint8_t port_id) * in a multi-process setup*/ PROC_PRIMARY_OR_RET(); - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return; } @@ -1073,10 +1111,11 @@ rte_eth_rx_queue_setup(uint8_t port_id, uint16_t rx_queue_id, * in a multi-process setup*/ PROC_PRIMARY_OR_ERR_RET(-E_RTE_SECONDARY); - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-EINVAL); } + dev = &rte_eth_devices[port_id]; if (rx_queue_id >= dev->data->nb_rx_queues) { PMD_DEBUG_TRACE("Invalid RX queue_id=%d\n", rx_queue_id); @@ -1146,10 +1185,11 @@ rte_eth_tx_queue_setup(uint8_t port_id, uint16_t tx_queue_id, * in a multi-process setup*/ PROC_PRIMARY_OR_ERR_RET(-E_RTE_SECONDARY); - if (port_id >= RTE_MAX_ETHPORTS || port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-EINVAL); } + dev = &rte_eth_devices[port_id]; if (tx_queue_id >= dev->data->nb_tx_queues) { PMD_DEBUG_TRACE("Invalid TX queue_id=%d\n", tx_queue_id); @@ -1179,10 +1219,11 @@ rte_eth_promiscuous_enable(uint8_t port_id) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return; } + dev = &rte_eth_devices[port_id]; FUNC_PTR_OR_RET(*dev->dev_ops->promiscuous_enable); @@ -1195,10 +1236,11 @@ rte_eth_promiscuous_disable(uint8_t port_id) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return; } + dev = &rte_eth_devices[port_id]; FUNC_PTR_OR_RET(*dev->dev_ops->promiscuous_disable); @@ -1211,7 +1253,7 @@ rte_eth_promiscuous_get(uint8_t port_id) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return -1; } @@ -1225,10 +1267,11 @@ rte_eth_allmulticast_enable(uint8_t port_id) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return; } + dev = &rte_eth_devices[port_id]; FUNC_PTR_OR_RET(*dev->dev_ops->allmulticast_enable); @@ -1241,10 +1284,11 @@ rte_eth_allmulticast_disable(uint8_t port_id) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return; } + dev = &rte_eth_devices[port_id]; FUNC_PTR_OR_RET(*dev->dev_ops->allmulticast_disable); @@ -1257,7 +1301,7 @@ rte_eth_allmulticast_get(uint8_t port_id) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return -1; } @@ -1285,10 +1329,11 @@ rte_eth_link_get(uint8_t port_id, struct rte_eth_link *eth_link) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return; } + dev = &rte_eth_devices[port_id]; if (dev->data->dev_conf.intr_conf.lsc != 0) @@ -1305,10 +1350,11 @@ rte_eth_link_get_nowait(uint8_t port_id, struct rte_eth_link *eth_link) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return; } + dev = &rte_eth_devices[port_id]; if (dev->data->dev_conf.intr_conf.lsc != 0) @@ -1325,10 +1371,11 @@ rte_eth_stats_get(uint8_t port_id, struct rte_eth_stats *stats) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-ENODEV); } + dev = &rte_eth_devices[port_id]; memset(stats, 0, sizeof(*stats)); @@ -1343,10 +1390,11 @@ rte_eth_stats_reset(uint8_t port_id) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return; } + dev = &rte_eth_devices[port_id]; FUNC_PTR_OR_RET(*dev->dev_ops->stats_reset); @@ -1364,10 +1412,11 @@ rte_eth_xstats_get(uint8_t port_id, struct rte_eth_xstats *xstats, uint64_t val; char *stats_ptr; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return -1; } + dev = &rte_eth_devices[port_id]; /* implemented by the driver */ @@ -1433,10 +1482,11 @@ rte_eth_xstats_reset(uint8_t port_id) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return; } + dev = &rte_eth_devices[port_id]; /* implemented by the driver */ @@ -1455,10 +1505,11 @@ set_queue_stats_mapping(uint8_t port_id, uint16_t queue_id, uint8_t stat_idx, { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return -ENODEV; } + dev = &rte_eth_devices[port_id]; FUNC_PTR_OR_ERR_RET(*dev->dev_ops->queue_stats_mapping_set, -ENOTSUP); @@ -1490,10 +1541,11 @@ rte_eth_dev_info_get(uint8_t port_id, struct rte_eth_dev_info *dev_info) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return; } + dev = &rte_eth_devices[port_id]; memset(dev_info, 0, sizeof(struct rte_eth_dev_info)); @@ -1510,10 +1562,11 @@ rte_eth_macaddr_get(uint8_t port_id, struct ether_addr *mac_addr) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return; } + dev = &rte_eth_devices[port_id]; ether_addr_copy(&dev->data->mac_addrs[0], mac_addr); } @@ -1524,7 +1577,7 @@ rte_eth_dev_get_mtu(uint8_t port_id, uint16_t *mtu) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-ENODEV); } @@ -1540,7 +1593,7 @@ rte_eth_dev_set_mtu(uint8_t port_id, uint16_t mtu) int ret; struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-ENODEV); } @@ -1560,10 +1613,11 @@ rte_eth_dev_vlan_filter(uint8_t port_id, uint16_t vlan_id, int on) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-ENODEV); } + dev = &rte_eth_devices[port_id]; if (! (dev->data->dev_conf.rxmode.hw_vlan_filter)) { PMD_DEBUG_TRACE("port %d: vlan-filtering disabled\n", port_id); @@ -1585,7 +1639,7 @@ rte_eth_dev_set_vlan_strip_on_queue(uint8_t port_id, uint16_t rx_queue_id, int o { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-ENODEV); } @@ -1607,7 +1661,7 @@ rte_eth_dev_set_vlan_ether_type(uint8_t port_id, uint16_t tpid) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-ENODEV); } @@ -1627,7 +1681,7 @@ rte_eth_dev_set_vlan_offload(uint8_t port_id, int offload_mask) int mask = 0; int cur, org = 0; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-ENODEV); } @@ -1672,7 +1726,7 @@ rte_eth_dev_get_vlan_offload(uint8_t port_id) struct rte_eth_dev *dev; int ret = 0; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-ENODEV); } @@ -1696,10 +1750,11 @@ rte_eth_dev_set_vlan_pvid(uint8_t port_id, uint16_t pvid, int on) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-ENODEV); } + dev = &rte_eth_devices[port_id]; FUNC_PTR_OR_ERR_RET(*dev->dev_ops->vlan_pvid_set, -ENOTSUP); (*dev->dev_ops->vlan_pvid_set)(dev, pvid, on); @@ -1714,7 +1769,7 @@ rte_eth_dev_fdir_add_signature_filter(uint8_t port_id, { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-ENODEV); } @@ -1748,7 +1803,7 @@ rte_eth_dev_fdir_update_signature_filter(uint8_t port_id, { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-ENODEV); } @@ -1782,7 +1837,7 @@ rte_eth_dev_fdir_remove_signature_filter(uint8_t port_id, { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-ENODEV); } @@ -1813,7 +1868,7 @@ rte_eth_dev_fdir_get_infos(uint8_t port_id, struct rte_eth_fdir *fdir) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-ENODEV); } @@ -1838,7 +1893,7 @@ rte_eth_dev_fdir_add_perfect_filter(uint8_t port_id, { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-ENODEV); } @@ -1878,7 +1933,7 @@ rte_eth_dev_fdir_update_perfect_filter(uint8_t port_id, { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-ENODEV); } @@ -1916,7 +1971,7 @@ rte_eth_dev_fdir_remove_perfect_filter(uint8_t port_id, { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-ENODEV); } @@ -1952,7 +2007,7 @@ rte_eth_dev_fdir_set_masks(uint8_t port_id, struct rte_fdir_masks *fdir_mask) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-ENODEV); } @@ -1972,7 +2027,7 @@ rte_eth_dev_flow_ctrl_get(uint8_t port_id, struct rte_eth_fc_conf *fc_conf) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-ENODEV); } @@ -1988,7 +2043,7 @@ rte_eth_dev_flow_ctrl_set(uint8_t port_id, struct rte_eth_fc_conf *fc_conf) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-ENODEV); } @@ -2008,7 +2063,7 @@ rte_eth_dev_priority_flow_ctrl_set(uint8_t port_id, struct rte_eth_pfc_conf *pfc { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-ENODEV); } @@ -2087,7 +2142,7 @@ rte_eth_dev_rss_reta_update(uint8_t port_id, struct rte_eth_dev *dev; int ret; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return -ENODEV; } @@ -2138,10 +2193,11 @@ rte_eth_dev_rss_hash_update(uint8_t port_id, struct rte_eth_rss_conf *rss_conf) struct rte_eth_dev *dev; uint16_t rss_hash_protos; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-ENODEV); } + rss_hash_protos = rss_conf->rss_hf; if ((rss_hash_protos != 0) && ((rss_hash_protos & ETH_RSS_PROTO_MASK) == 0)) { @@ -2160,10 +2216,11 @@ rte_eth_dev_rss_hash_conf_get(uint8_t port_id, { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-ENODEV); } + dev = &rte_eth_devices[port_id]; FUNC_PTR_OR_ERR_RET(*dev->dev_ops->rss_hash_conf_get, -ENOTSUP); return (*dev->dev_ops->rss_hash_conf_get)(dev, rss_conf); @@ -2175,7 +2232,7 @@ rte_eth_dev_udp_tunnel_add(uint8_t port_id, { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return -ENODEV; } @@ -2201,10 +2258,11 @@ rte_eth_dev_udp_tunnel_delete(uint8_t port_id, { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return -ENODEV; } + dev = &rte_eth_devices[port_id]; if (udp_tunnel == NULL) { @@ -2226,7 +2284,7 @@ rte_eth_led_on(uint8_t port_id) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-ENODEV); } @@ -2241,7 +2299,7 @@ rte_eth_led_off(uint8_t port_id) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-ENODEV); } @@ -2281,10 +2339,11 @@ rte_eth_dev_mac_addr_add(uint8_t port_id, struct ether_addr *addr, int index; uint64_t pool_mask; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-ENODEV); } + dev = &rte_eth_devices[port_id]; FUNC_PTR_OR_ERR_RET(*dev->dev_ops->mac_addr_add, -ENOTSUP); @@ -2332,10 +2391,11 @@ rte_eth_dev_mac_addr_remove(uint8_t port_id, struct ether_addr *addr) struct rte_eth_dev *dev; int index; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-ENODEV); } + dev = &rte_eth_devices[port_id]; FUNC_PTR_OR_ERR_RET(*dev->dev_ops->mac_addr_remove, -ENOTSUP); @@ -2366,7 +2426,7 @@ rte_eth_dev_set_vf_rxmode(uint8_t port_id, uint16_t vf, struct rte_eth_dev *dev; struct rte_eth_dev_info dev_info; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("set VF RX mode:Invalid port_id=%d\n", port_id); return (-ENODEV); @@ -2421,7 +2481,7 @@ rte_eth_dev_uc_hash_table_set(uint8_t port_id, struct ether_addr *addr, int ret; struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("unicast hash setting:Invalid port_id=%d\n", port_id); return (-ENODEV); @@ -2474,7 +2534,7 @@ rte_eth_dev_uc_all_hash_table_set(uint8_t port_id, uint8_t on) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("unicast hash setting:Invalid port_id=%d\n", port_id); return (-ENODEV); @@ -2493,7 +2553,7 @@ rte_eth_dev_set_vf_rx(uint8_t port_id,uint16_t vf, uint8_t on) struct rte_eth_dev *dev; struct rte_eth_dev_info dev_info; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-ENODEV); } @@ -2519,7 +2579,7 @@ rte_eth_dev_set_vf_tx(uint8_t port_id,uint16_t vf, uint8_t on) struct rte_eth_dev *dev; struct rte_eth_dev_info dev_info; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("set pool tx:Invalid port_id=%d\n", port_id); return (-ENODEV); } @@ -2544,7 +2604,7 @@ rte_eth_dev_set_vf_vlan_filter(uint8_t port_id, uint16_t vlan_id, { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("VF VLAN filter:invalid port id=%d\n", port_id); return (-ENODEV); @@ -2575,7 +2635,7 @@ int rte_eth_set_queue_rate_limit(uint8_t port_id, uint16_t queue_idx, struct rte_eth_dev_info dev_info; struct rte_eth_link link; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("set queue rate limit:invalid port id=%d\n", port_id); return -ENODEV; @@ -2612,7 +2672,7 @@ int rte_eth_set_vf_rate_limit(uint8_t port_id, uint16_t vf, uint16_t tx_rate, if (q_msk == 0) return 0; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("set VF rate limit:invalid port id=%d\n", port_id); return -ENODEV; @@ -2646,7 +2706,7 @@ rte_eth_mirror_rule_set(uint8_t port_id, { struct rte_eth_dev *dev = &rte_eth_devices[port_id]; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-ENODEV); } @@ -2687,7 +2747,7 @@ rte_eth_mirror_rule_reset(uint8_t port_id, uint8_t rule_id) { struct rte_eth_dev *dev = &rte_eth_devices[port_id]; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-ENODEV); } @@ -2712,10 +2772,11 @@ rte_eth_rx_burst(uint8_t port_id, uint16_t queue_id, { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return 0; } + dev = &rte_eth_devices[port_id]; FUNC_PTR_OR_ERR_RET(*dev->rx_pkt_burst, 0); if (queue_id >= dev->data->nb_rx_queues) { @@ -2732,10 +2793,11 @@ rte_eth_tx_burst(uint8_t port_id, uint16_t queue_id, { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return 0; } + dev = &rte_eth_devices[port_id]; FUNC_PTR_OR_ERR_RET(*dev->tx_pkt_burst, 0); @@ -2752,10 +2814,11 @@ rte_eth_rx_queue_count(uint8_t port_id, uint16_t queue_id) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return 0; } + dev = &rte_eth_devices[port_id]; FUNC_PTR_OR_ERR_RET(*dev->dev_ops->rx_queue_count, 0); return (*dev->dev_ops->rx_queue_count)(dev, queue_id); @@ -2766,10 +2829,11 @@ rte_eth_rx_descriptor_done(uint8_t port_id, uint16_t queue_id, uint16_t offset) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-ENODEV); } + dev = &rte_eth_devices[port_id]; FUNC_PTR_OR_ERR_RET(*dev->dev_ops->rx_descriptor_done, -ENOTSUP); return (*dev->dev_ops->rx_descriptor_done)( \ @@ -2787,7 +2851,8 @@ rte_eth_dev_callback_register(uint8_t port_id, if (!cb_fn) return (-EINVAL); - if (port_id >= nb_ports) { + + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-EINVAL); } @@ -2827,7 +2892,8 @@ rte_eth_dev_callback_unregister(uint8_t port_id, if (!cb_fn) return (-EINVAL); - if (port_id >= nb_ports) { + + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-EINVAL); } @@ -2887,7 +2953,7 @@ int rte_eth_dev_bypass_init(uint8_t port_id) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-ENODEV); } @@ -2907,7 +2973,7 @@ rte_eth_dev_bypass_state_show(uint8_t port_id, uint32_t *state) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-ENODEV); } @@ -2926,7 +2992,7 @@ rte_eth_dev_bypass_state_set(uint8_t port_id, uint32_t *new_state) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-ENODEV); } @@ -2946,7 +3012,7 @@ rte_eth_dev_bypass_event_show(uint8_t port_id, uint32_t event, uint32_t *state) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-ENODEV); } @@ -2966,7 +3032,7 @@ rte_eth_dev_bypass_event_store(uint8_t port_id, uint32_t event, uint32_t state) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-ENODEV); } @@ -2986,7 +3052,7 @@ rte_eth_dev_wd_timeout_store(uint8_t port_id, uint32_t timeout) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-ENODEV); } @@ -3006,7 +3072,7 @@ rte_eth_dev_bypass_ver_show(uint8_t port_id, uint32_t *ver) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-ENODEV); } @@ -3026,7 +3092,7 @@ rte_eth_dev_bypass_wd_timeout_show(uint8_t port_id, uint32_t *wd_timeout) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-ENODEV); } @@ -3046,7 +3112,7 @@ rte_eth_dev_bypass_wd_reset(uint8_t port_id) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return (-ENODEV); } @@ -3067,7 +3133,7 @@ rte_eth_dev_filter_supported(uint8_t port_id, enum rte_filter_type filter_type) { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return -ENODEV; } @@ -3084,7 +3150,7 @@ rte_eth_dev_filter_ctrl(uint8_t port_id, enum rte_filter_type filter_type, { struct rte_eth_dev *dev; - if (port_id >= nb_ports) { + if (!rte_eth_dev_is_valid_port(port_id)) { PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); return -ENODEV; } diff --git a/lib/librte_ether/rte_ethdev.h b/lib/librte_ether/rte_ethdev.h index cce1a53..110ddba 100644 --- a/lib/librte_ether/rte_ethdev.h +++ b/lib/librte_ether/rte_ethdev.h @@ -1451,6 +1451,7 @@ struct rte_eth_dev { * received packets before passing them to the driver for transmission. */ struct rte_eth_rxtx_callback **pre_tx_burst_cbs; + uint8_t attached; /**< Flag indicating the port is attached */ }; struct rte_eth_dev_sriov { @@ -1516,6 +1517,10 @@ extern struct rte_eth_dev rte_eth_devices[]; * initialized by the [matching] Ethernet driver during the PCI probing phase. * All devices whose port identifier is in the range * [0, rte_eth_dev_count() - 1] can be operated on by network applications. + * immediately after invoking rte_eal_init(). + * If the application unplugs a port using hotplug function, The enabled port + * numbers may be noncontiguous. In the case, the applications need to manage + * enabled port by themselves. * * @return * - The total number of usable Ethernet devices. -- 1.9.1
This patch replaces pci_addr_comparison() and memcmp() of pci addresses by rte_eal_compare_pci_addr(). To compare PCI addresses, rte_eal_compare_pci_addr() doesn't use memcmp(). This is because sizeof(struct rte_pci_addr) returns 6, but actually this structure is like below. struct rte_pci_addr { uint16_t domain; /**< Device domain */ uint8_t bus; /**< Device bus */ uint8_t devid; /**< Device ID */ uint8_t function; /**< Device function. */ }; If the structure is dynamically allocated in a function without bzero, last 1 byte may have value. As a result, memcmp may not work. To avoid such a case, rte_eal_compare_pci_addr() compare following values. dev_addr = (addr->domain << 24) | (addr->bus << 16) | (addr->devid << 8) | addr->function; v9: - eal_compare_pci_addr() is replaced by rte_eal_compare_pci_addr(). - Fix commit log. (Thanks to Thomas Monjalon) v8: - Fix pci_scan_one() to update sysfs values. (Thanks to Qiu, Michael and Iremonger, Bernard) v5: - Fix pci_scan_one to handle pt_driver correctly. v4: - Fix calculation method of eal_compare_pci_addr(). - Add parameter checking. Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp> --- lib/librte_eal/bsdapp/eal/eal_pci.c | 29 ++++++++++++-------------- lib/librte_eal/common/eal_common_pci.c | 2 +- lib/librte_eal/common/include/rte_pci.h | 34 +++++++++++++++++++++++++++++++ lib/librte_eal/linuxapp/eal/eal_pci.c | 30 +++++++++++++-------------- lib/librte_eal/linuxapp/eal/eal_pci_uio.c | 2 +- 5 files changed, 63 insertions(+), 34 deletions(-) diff --git a/lib/librte_eal/bsdapp/eal/eal_pci.c b/lib/librte_eal/bsdapp/eal/eal_pci.c index 74ecce7..9193f80 100644 --- a/lib/librte_eal/bsdapp/eal/eal_pci.c +++ b/lib/librte_eal/bsdapp/eal/eal_pci.c @@ -270,20 +270,6 @@ pci_uio_map_resource(struct rte_pci_device *dev) return (0); } -/* Compare two PCI device addresses. */ -static int -pci_addr_comparison(struct rte_pci_addr *addr, struct rte_pci_addr *addr2) -{ - uint64_t dev_addr = (addr->domain << 24) + (addr->bus << 16) + (addr->devid << 8) + addr->function; - uint64_t dev_addr2 = (addr2->domain << 24) + (addr2->bus << 16) + (addr2->devid << 8) + addr2->function; - - if (dev_addr > dev_addr2) - return 1; - else - return 0; -} - - /* Scan one pci sysfs entry, and fill the devices list from it. */ static int pci_scan_one(int dev_pci_fd, struct pci_conf *conf) @@ -356,13 +342,24 @@ pci_scan_one(int dev_pci_fd, struct pci_conf *conf) } else { struct rte_pci_device *dev2 = NULL; + int ret; TAILQ_FOREACH(dev2, &pci_device_list, next) { - if (pci_addr_comparison(&dev->addr, &dev2->addr)) + ret = rte_eal_compare_pci_addr(&dev->addr, &dev2->addr); + if (ret > 0) continue; - else { + else if (ret < 0) { TAILQ_INSERT_BEFORE(dev2, dev, next); return 0; + } else { /* already registered */ + /* update pt_driver */ + dev2->pt_driver = dev->pt_driver; + dev2->max_vfs = dev->max_vfs; + memmove(dev2->mem_resource, + dev->mem_resource, + sizeof(dev->mem_resource)); + free(dev); + return 0; } } TAILQ_INSERT_TAIL(&pci_device_list, dev, next); diff --git a/lib/librte_eal/common/eal_common_pci.c b/lib/librte_eal/common/eal_common_pci.c index f3c7f71..bf2793f 100644 --- a/lib/librte_eal/common/eal_common_pci.c +++ b/lib/librte_eal/common/eal_common_pci.c @@ -93,7 +93,7 @@ static struct rte_devargs *pci_devargs_lookup(struct rte_pci_device *dev) if (devargs->type != RTE_DEVTYPE_BLACKLISTED_PCI && devargs->type != RTE_DEVTYPE_WHITELISTED_PCI) continue; - if (!memcmp(&dev->addr, &devargs->pci.addr, sizeof(dev->addr))) + if (!rte_eal_compare_pci_addr(&dev->addr, &devargs->pci.addr)) return devargs; } return NULL; diff --git a/lib/librte_eal/common/include/rte_pci.h b/lib/librte_eal/common/include/rte_pci.h index 255a77b..dcf9c81 100644 --- a/lib/librte_eal/common/include/rte_pci.h +++ b/lib/librte_eal/common/include/rte_pci.h @@ -272,6 +272,40 @@ eal_parse_pci_DomBDF(const char *input, struct rte_pci_addr *dev_addr) } #undef GET_PCIADDR_FIELD +/* Compare two PCI device addresses. */ +/** + * Utility function to compare two PCI device addresses. + * + * @param addr + * The PCI Bus-Device-Function address to compare + * @param addr2 + * The PCI Bus-Device-Function address to compare + * @return + * 0 on equal PCI address. + * Positive on addr is greater than addr2. + * Negative on addr is less than addr2, or error. + */ +static inline int +rte_eal_compare_pci_addr(struct rte_pci_addr *addr, struct rte_pci_addr *addr2) +{ + uint64_t dev_addr, dev_addr2; + + if ((addr == NULL) || (addr2 == NULL)) + return -1; + + dev_addr = (addr->domain << 24) | (addr->bus << 16) | + (addr->devid << 8) | addr->function; + dev_addr2 = (addr2->domain << 24) | (addr2->bus << 16) | + (addr2->devid << 8) | addr2->function; + + if (dev_addr > dev_addr2) + return 1; + else if (dev_addr < dev_addr2) + return -1; + else + return 0; +} + /** * Probe the PCI bus for registered drivers. * diff --git a/lib/librte_eal/linuxapp/eal/eal_pci.c b/lib/librte_eal/linuxapp/eal/eal_pci.c index 3291c68..06bfc1a 100644 --- a/lib/librte_eal/linuxapp/eal/eal_pci.c +++ b/lib/librte_eal/linuxapp/eal/eal_pci.c @@ -41,6 +41,7 @@ #include <rte_eal_memconfig.h> #include <rte_malloc.h> #include <rte_devargs.h> +#include <rte_memcpy.h> #include "rte_pci_dev_ids.h" #include "eal_filesystem.h" @@ -228,20 +229,6 @@ error: return -1; } -/* Compare two PCI device addresses. */ -static int -pci_addr_comparison(struct rte_pci_addr *addr, struct rte_pci_addr *addr2) -{ - uint64_t dev_addr = (addr->domain << 24) + (addr->bus << 16) + (addr->devid << 8) + addr->function; - uint64_t dev_addr2 = (addr2->domain << 24) + (addr2->bus << 16) + (addr2->devid << 8) + addr2->function; - - if (dev_addr > dev_addr2) - return 1; - else - return 0; -} - - /* Scan one pci sysfs entry, and fill the devices list from it. */ static int pci_scan_one(const char *dirname, uint16_t domain, uint8_t bus, @@ -359,13 +346,24 @@ pci_scan_one(const char *dirname, uint16_t domain, uint8_t bus, } else { struct rte_pci_device *dev2 = NULL; + int ret; TAILQ_FOREACH(dev2, &pci_device_list, next) { - if (pci_addr_comparison(&dev->addr, &dev2->addr)) + ret = rte_eal_compare_pci_addr(&dev->addr, &dev2->addr); + if (ret > 0) continue; - else { + else if (ret < 0) { TAILQ_INSERT_BEFORE(dev2, dev, next); return 0; + } else { /* already registered */ + /* update pt_driver */ + dev2->pt_driver = dev->pt_driver; + dev2->max_vfs = dev->max_vfs; + memmove(dev2->mem_resource, + dev->mem_resource, + sizeof(dev->mem_resource)); + free(dev); + return 0; } } TAILQ_INSERT_TAIL(&pci_device_list, dev, next); diff --git a/lib/librte_eal/linuxapp/eal/eal_pci_uio.c b/lib/librte_eal/linuxapp/eal/eal_pci_uio.c index 0a95376..c5e0cf3 100644 --- a/lib/librte_eal/linuxapp/eal/eal_pci_uio.c +++ b/lib/librte_eal/linuxapp/eal/eal_pci_uio.c @@ -91,7 +91,7 @@ pci_uio_map_secondary(struct rte_pci_device *dev) TAILQ_FOREACH(uio_res, pci_res_list, next) { /* skip this element if it doesn't match our PCI address */ - if (memcmp(&uio_res->pci_addr, &dev->addr, sizeof(dev->addr))) + if (rte_eal_compare_pci_addr(&uio_res->pci_addr, &dev->addr)) continue; for (i = 0; i != uio_res->nb_maps; i++) { -- 1.9.1
This patch adds rte_eth_dev_release_port(). The function is used for changing an attached status of the device that has specified name. v9: - rte_eth_dev_free() is replaced by rte_eth_dev_release_port(). (Thanks to Thomas Monjalon) v6: - Use rte_eth_dev structure as the paramter of rte_eth_dev_free(). v4: - Add parameter checking. Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp> --- lib/librte_ether/rte_ethdev.c | 11 +++++++++++ lib/librte_ether/rte_ethdev.h | 12 ++++++++++++ 2 files changed, 23 insertions(+) diff --git a/lib/librte_ether/rte_ethdev.c b/lib/librte_ether/rte_ethdev.c index b702039..a089557 100644 --- a/lib/librte_ether/rte_ethdev.c +++ b/lib/librte_ether/rte_ethdev.c @@ -255,6 +255,17 @@ rte_eth_dev_allocate(const char *name) return eth_dev; } +int +rte_eth_dev_release_port(struct rte_eth_dev *eth_dev) +{ + if (eth_dev == NULL) + return -EINVAL; + + eth_dev->attached = 0; + nb_ports--; + return 0; +} + static int rte_eth_dev_init(struct rte_pci_driver *pci_drv, struct rte_pci_device *pci_dev) diff --git a/lib/librte_ether/rte_ethdev.h b/lib/librte_ether/rte_ethdev.h index 110ddba..7963e56 100644 --- a/lib/librte_ether/rte_ethdev.h +++ b/lib/librte_ether/rte_ethdev.h @@ -1539,6 +1539,18 @@ extern uint8_t rte_eth_dev_count(void); */ struct rte_eth_dev *rte_eth_dev_allocate(const char *name); +/** + * Function for internal use by dummy drivers primarily, e.g. ring-based + * driver. + * Release the specified ethdev port. + * + * @param eth_dev + * The *eth_dev* pointer is the address of the *rte_eth_dev* structure. + * @return + * - 0 on success, negative on error + */ +int rte_eth_dev_release_port(struct rte_eth_dev *eth_dev); + struct eth_driver; /** * @internal -- 1.9.1
The patch adds function pointer to rte_pci_driver and eth_driver structure. These function pointers are used when ports are detached. Also, the patch adds rte_eth_dev_uninit(). So far, it's not called by anywhere, but it will be called when port hotplug function is implemented. v10: - Add size parameter to rte_eth_dev_create_unique_device_name(). (Thanks to Iremonger, Bernard) v9: - Change parameter of pci_devuninit_t and rte_eth_dev_uninit. - Remove code that initiaize callback of ethdev from rte_eth_dev_uninit(). - Add a function to create a unique device name. (Thanks to Thomas Monjalon) v6: - Fix rte_eth_dev_uninit() to handle a return value of uninit function of PMD. v4: - Add parameter checking. - Change function names. Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp> --- lib/librte_eal/common/include/rte_pci.h | 6 ++++ lib/librte_ether/rte_ethdev.c | 64 +++++++++++++++++++++++++++++++-- lib/librte_ether/rte_ethdev.h | 24 +++++++++++++ 3 files changed, 92 insertions(+), 2 deletions(-) diff --git a/lib/librte_eal/common/include/rte_pci.h b/lib/librte_eal/common/include/rte_pci.h index dcf9c81..ecde36f 100644 --- a/lib/librte_eal/common/include/rte_pci.h +++ b/lib/librte_eal/common/include/rte_pci.h @@ -192,12 +192,18 @@ struct rte_pci_driver; typedef int (pci_devinit_t)(struct rte_pci_driver *, struct rte_pci_device *); /** + * Uninitialisation function for the driver called during hotplugging. + */ +typedef int (pci_devuninit_t)(struct rte_pci_device *); + +/** * A structure describing a PCI driver. */ struct rte_pci_driver { TAILQ_ENTRY(rte_pci_driver) next; /**< Next in list. */ const char *name; /**< Driver name. */ pci_devinit_t *devinit; /**< Device init. function. */ + pci_devuninit_t *devuninit; /**< Device uninit function. */ struct rte_pci_id *id_table; /**< ID table, NULL terminated. */ uint32_t drv_flags; /**< Flags contolling handling of device. */ }; diff --git a/lib/librte_ether/rte_ethdev.c b/lib/librte_ether/rte_ethdev.c index a089557..a26af75 100644 --- a/lib/librte_ether/rte_ethdev.c +++ b/lib/librte_ether/rte_ethdev.c @@ -255,6 +255,24 @@ rte_eth_dev_allocate(const char *name) return eth_dev; } +static inline int +rte_eth_dev_create_unique_device_name(char *name, size_t size, + struct rte_pci_device *pci_dev) +{ + int ret; + + if ((name == NULL) || (pci_dev == NULL)) + return -EINVAL; + + ret = snprintf(name, size, "%d:%d.%d", + pci_dev->addr.bus, pci_dev->addr.devid, + pci_dev->addr.function); + if (ret < 0) + return ret; + + return 0; +} + int rte_eth_dev_release_port(struct rte_eth_dev *eth_dev) { @@ -279,8 +297,8 @@ rte_eth_dev_init(struct rte_pci_driver *pci_drv, eth_drv = (struct eth_driver *)pci_drv; /* Create unique Ethernet device name using PCI address */ - snprintf(ethdev_name, RTE_ETH_NAME_MAX_LEN, "%d:%d.%d", - pci_dev->addr.bus, pci_dev->addr.devid, pci_dev->addr.function); + rte_eth_dev_create_unique_device_name(ethdev_name, + sizeof(ethdev_name), pci_dev); eth_dev = rte_eth_dev_allocate(ethdev_name); if (eth_dev == NULL) @@ -321,6 +339,47 @@ rte_eth_dev_init(struct rte_pci_driver *pci_drv, return diag; } +static int +rte_eth_dev_uninit(struct rte_pci_device *pci_dev) +{ + const struct eth_driver *eth_drv; + struct rte_eth_dev *eth_dev; + char ethdev_name[RTE_ETH_NAME_MAX_LEN]; + int ret; + + if (pci_dev == NULL) + return -EINVAL; + + /* Create unique Ethernet device name using PCI address */ + rte_eth_dev_create_unique_device_name(ethdev_name, + sizeof(ethdev_name), pci_dev); + + eth_dev = rte_eth_dev_allocated(ethdev_name); + if (eth_dev == NULL) + return -ENODEV; + + eth_drv = (const struct eth_driver *)pci_dev->driver; + + /* Invoke PMD device uninit function */ + if (*eth_drv->eth_dev_uninit) { + ret = (*eth_drv->eth_dev_uninit)(eth_drv, eth_dev); + if (ret) + return ret; + } + + /* free ether device */ + rte_eth_dev_release_port(eth_dev); + + if (rte_eal_process_type() == RTE_PROC_PRIMARY) + rte_free(eth_dev->data->dev_private); + + eth_dev->pci_dev = NULL; + eth_dev->driver = NULL; + eth_dev->data = NULL; + + return 0; +} + /** * Register an Ethernet [Poll Mode] driver. * @@ -339,6 +398,7 @@ void rte_eth_driver_register(struct eth_driver *eth_drv) { eth_drv->pci_drv.devinit = rte_eth_dev_init; + eth_drv->pci_drv.devuninit = rte_eth_dev_uninit; rte_eal_pci_register(ð_drv->pci_drv); } diff --git a/lib/librte_ether/rte_ethdev.h b/lib/librte_ether/rte_ethdev.h index 7963e56..37c3765 100644 --- a/lib/librte_ether/rte_ethdev.h +++ b/lib/librte_ether/rte_ethdev.h @@ -1588,6 +1588,27 @@ typedef int (*eth_dev_init_t)(struct eth_driver *eth_drv, /** * @internal + * Finalization function of an Ethernet driver invoked for each matching + * Ethernet PCI device detected during the PCI closing phase. + * + * @param eth_drv + * The pointer to the [matching] Ethernet driver structure supplied by + * the PMD when it registered itself. + * @param eth_dev + * The *eth_dev* pointer is the address of the *rte_eth_dev* structure + * associated with the matching device and which have been [automatically] + * allocated in the *rte_eth_devices* array. + * @return + * - 0: Success, the device is properly finalized by the driver. + * In particular, the driver MUST free the *dev_ops* pointer + * of the *eth_dev* structure. + * - <0: Error code of the device initialization failure. + */ +typedef int (*eth_dev_uninit_t)(const struct eth_driver *eth_drv, + struct rte_eth_dev *eth_dev); + +/** + * @internal * The structure associated with a PMD Ethernet driver. * * Each Ethernet driver acts as a PCI driver and is represented by a generic @@ -1597,11 +1618,14 @@ typedef int (*eth_dev_init_t)(struct eth_driver *eth_drv, * * - The *eth_dev_init* function invoked for each matching PCI device. * + * - The *eth_dev_uninit* function invoked for each matching PCI device. + * * - The size of the private data to allocate for each matching device. */ struct eth_driver { struct rte_pci_driver pci_drv; /**< The PMD is also a PCI driver. */ eth_dev_init_t eth_dev_init; /**< Device init function. */ + eth_dev_uninit_t eth_dev_uninit; /**< Device uninit function. */ unsigned int dev_private_size; /**< Size of device private data. */ }; -- 1.9.1
The patch adds functions for unmapping igb_uio resources. The patch is only for Linux and igb_uio environment. VFIO and BSD are not supported. v9: - Remove "rte_dev_hotplug.h". - Remove needless "#ifdef". (Thanks to Thomas Monjalon and Neil Horman) - Remove pci_unmap_device(). It will be implemented in later patch. v8: - Fix typo. (Thanks to Iremonger, Bernard) v5: - Fix pci_unmap_device() to check pt_driver. v4: - Add parameter checking. - Add header file to determine if hotplug can be enabled. Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp> --- lib/librte_eal/linuxapp/eal/eal_pci.c | 17 ++++++++ lib/librte_eal/linuxapp/eal/eal_pci_init.h | 7 ++++ lib/librte_eal/linuxapp/eal/eal_pci_uio.c | 65 ++++++++++++++++++++++++++++++ 3 files changed, 89 insertions(+) diff --git a/lib/librte_eal/linuxapp/eal/eal_pci.c b/lib/librte_eal/linuxapp/eal/eal_pci.c index 06bfc1a..d03429c 100644 --- a/lib/librte_eal/linuxapp/eal/eal_pci.c +++ b/lib/librte_eal/linuxapp/eal/eal_pci.c @@ -168,6 +168,23 @@ pci_map_resource(void *requested_addr, int fd, off_t offset, size_t size, return mapaddr; } +/* unmap a particular resource */ +void +pci_unmap_resource(void *requested_addr, size_t size) +{ + if (requested_addr == NULL) + return; + + /* Unmap the PCI memory resource of device */ + if (munmap(requested_addr, size)) { + RTE_LOG(ERR, EAL, "%s(): cannot munmap(%p, 0x%lx): %s\n", + __func__, requested_addr, (unsigned long)size, + strerror(errno)); + } else + RTE_LOG(DEBUG, EAL, " PCI memory unmapped at %p\n", + requested_addr); +} + /* parse the "resource" sysfs file */ static int pci_parse_sysfs_resource(const char *filename, struct rte_pci_device *dev) diff --git a/lib/librte_eal/linuxapp/eal/eal_pci_init.h b/lib/librte_eal/linuxapp/eal/eal_pci_init.h index 03d2b52..6af84d1 100644 --- a/lib/librte_eal/linuxapp/eal/eal_pci_init.h +++ b/lib/librte_eal/linuxapp/eal/eal_pci_init.h @@ -72,6 +72,13 @@ void *pci_map_resource(void *requested_addr, int fd, off_t offset, /* map IGB_UIO resource prototype */ int pci_uio_map_resource(struct rte_pci_device *dev); +void pci_unmap_resource(void *requested_addr, size_t size); + +#ifdef RTE_LIBRTE_EAL_HOTPLUG +/* unmap IGB_UIO resource prototype */ +void pci_uio_unmap_resource(struct rte_pci_device *dev); +#endif /* RTE_LIBRTE_EAL_HOTPLUG */ + #ifdef VFIO_PRESENT #define VFIO_MAX_GROUPS 64 diff --git a/lib/librte_eal/linuxapp/eal/eal_pci_uio.c b/lib/librte_eal/linuxapp/eal/eal_pci_uio.c index c5e0cf3..35d31c5 100644 --- a/lib/librte_eal/linuxapp/eal/eal_pci_uio.c +++ b/lib/librte_eal/linuxapp/eal/eal_pci_uio.c @@ -386,3 +386,68 @@ pci_uio_map_resource(struct rte_pci_device *dev) return 0; } + +#ifdef RTE_LIBRTE_EAL_HOTPLUG +static void +pci_uio_unmap(struct mapped_pci_resource *uio_res) +{ + int i; + + if (uio_res == NULL) + return; + + for (i = 0; i != uio_res->nb_maps; i++) + pci_unmap_resource(uio_res->maps[i].addr, + (size_t)uio_res->maps[i].size); +} + +static struct mapped_pci_resource * +pci_uio_find_resource(struct rte_pci_device *dev) +{ + struct mapped_pci_resource *uio_res; + + if (dev == NULL) + return NULL; + + TAILQ_FOREACH(uio_res, pci_res_list, next) { + + /* skip this element if it doesn't match our PCI address */ + if (!rte_eal_compare_pci_addr(&uio_res->pci_addr, &dev->addr)) + return uio_res; + } + return NULL; +} + +/* unmap the PCI resource of a PCI device in virtual memory */ +void +pci_uio_unmap_resource(struct rte_pci_device *dev) +{ + struct mapped_pci_resource *uio_res; + + if (dev == NULL) + return; + + /* find an entry for the device */ + uio_res = pci_uio_find_resource(dev); + if (uio_res == NULL) + return; + + /* secondary processes - just free maps */ + if (rte_eal_process_type() != RTE_PROC_PRIMARY) + return pci_uio_unmap(uio_res); + + TAILQ_REMOVE(pci_res_list, uio_res, next); + + /* unmap all resources */ + pci_uio_unmap(uio_res); + + /* free uio resource */ + rte_free(uio_res); + + /* close fd if in primary process */ + close(dev->intr_handle.fd); + + dev->intr_handle.fd = -1; + dev->intr_handle.type = RTE_INTR_HANDLE_UNKNOWN; +} +#endif /* RTE_LIBRTE_EAL_HOTPLUG */ -- 1.9.1
- Add pci_close_all_drivers() The function tries to find a driver for the specified device, and then close the driver. - Add rte_eal_pci_probe_one() and rte_eal_pci_close_one() The functions are used for probe and close a device. First the function tries to find a device that has the specified PCI address. Then, probe or close the device. v15: - Fix version.map. v9: - Fix commit title. - Remove RTE_EAL_INVOKE_TYPE_PROBE/CLOSE. (Thanks to Thomas Monjalon) - Implement pci_unmap_device() in this patch. v5: - Remove RTE_EAL_INVOKE_TYPE_UNKNOWN, because it's unused. v4: - Fix parameter checking. - Fix indent of 'if' statement. Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp> --- lib/librte_eal/common/eal_common_pci.c | 98 ++++++++++++++++++++++++- lib/librte_eal/common/eal_private.h | 15 ++++ lib/librte_eal/common/include/rte_pci.h | 32 ++++++++ lib/librte_eal/linuxapp/eal/eal_pci.c | 94 ++++++++++++++++++++++++ lib/librte_eal/linuxapp/eal/rte_eal_version.map | 2 + lib/librte_ether/rte_ethdev.c | 6 +- 6 files changed, 243 insertions(+), 4 deletions(-) diff --git a/lib/librte_eal/common/eal_common_pci.c b/lib/librte_eal/common/eal_common_pci.c index bf2793f..5b6b55d 100644 --- a/lib/librte_eal/common/eal_common_pci.c +++ b/lib/librte_eal/common/eal_common_pci.c @@ -108,7 +108,10 @@ static int pci_probe_all_drivers(struct rte_pci_device *dev) { struct rte_pci_driver *dr = NULL; - int rc; + int rc = 0; + + if (dev == NULL) + return -1; TAILQ_FOREACH(dr, &pci_driver_list, next) { rc = rte_eal_pci_probe_one_driver(dr, dev); @@ -123,6 +126,99 @@ pci_probe_all_drivers(struct rte_pci_device *dev) return 1; } +#ifdef RTE_LIBRTE_EAL_HOTPLUG +/* + * If vendor/device ID match, call the devuninit() function of all + * registered driver for the given device. Return -1 if initialization + * failed, return 1 if no driver is found for this device. + */ +static int +pci_close_all_drivers(struct rte_pci_device *dev) +{ + struct rte_pci_driver *dr = NULL; + int rc = 0; + + if (dev == NULL) + return -1; + + TAILQ_FOREACH(dr, &pci_driver_list, next) { + rc = rte_eal_pci_close_one_driver(dr, dev); + if (rc < 0) + /* negative value is an error */ + return -1; + if (rc > 0) + /* positive value means driver not found */ + continue; + return 0; + } + return 1; +} + +/* + * Find the pci device specified by pci address, then invoke probe function of + * the driver of the devive. + */ +int +rte_eal_pci_probe_one(struct rte_pci_addr *addr) +{ + struct rte_pci_device *dev = NULL; + int ret = 0; + + if (addr == NULL) + return -1; + + TAILQ_FOREACH(dev, &pci_device_list, next) { + if (rte_eal_compare_pci_addr(&dev->addr, addr)) + continue; + + ret = pci_probe_all_drivers(dev); + if (ret < 0) + goto err_return; + return 0; + } + return -1; + +err_return: + RTE_LOG(WARNING, EAL, "Requested device " PCI_PRI_FMT + " cannot be used\n", dev->addr.domain, dev->addr.bus, + dev->addr.devid, dev->addr.function); + return -1; +} + +/* + * Find the pci device specified by pci address, then invoke close function of + * the driver of the devive. + */ +int +rte_eal_pci_close_one(struct rte_pci_addr *addr) +{ + struct rte_pci_device *dev = NULL; + int ret = 0; + + if (addr == NULL) + return -1; + + TAILQ_FOREACH(dev, &pci_device_list, next) { + if (rte_eal_compare_pci_addr(&dev->addr, addr)) + continue; + + ret = pci_close_all_drivers(dev); + if (ret < 0) + goto err_return; + + TAILQ_REMOVE(&pci_device_list, dev, next); + return 0; + } + return -1; + +err_return: + RTE_LOG(WARNING, EAL, "Requested device " PCI_PRI_FMT + " cannot be used\n", dev->addr.domain, dev->addr.bus, + dev->addr.devid, dev->addr.function); + return -1; +} +#endif /* RTE_LIBRTE_EAL_HOTPLUG */ + /* * Scan the content of the PCI bus, and call the devinit() function for * all registered drivers that have a matching entry in its id_table diff --git a/lib/librte_eal/common/eal_private.h b/lib/librte_eal/common/eal_private.h index 159cd66..4acf5a0 100644 --- a/lib/librte_eal/common/eal_private.h +++ b/lib/librte_eal/common/eal_private.h @@ -165,6 +165,21 @@ int rte_eal_pci_probe_one_driver(struct rte_pci_driver *dr, struct rte_pci_device *dev); /** + * Munmap memory for single PCI device + * + * This function is private to EAL. + * + * @param dr + * The pointer to the pci driver structure + * @param dev + * The pointer to the pci device structure + * @return + * 0 on success, negative on error + */ +int rte_eal_pci_close_one_driver(struct rte_pci_driver *dr, + struct rte_pci_device *dev); + +/** * Init tail queues for non-EAL library structures. This is to allow * the rings, mempools, etc. lists to be shared among multiple processes * diff --git a/lib/librte_eal/common/include/rte_pci.h b/lib/librte_eal/common/include/rte_pci.h index ecde36f..ac30925 100644 --- a/lib/librte_eal/common/include/rte_pci.h +++ b/lib/librte_eal/common/include/rte_pci.h @@ -325,6 +325,38 @@ rte_eal_compare_pci_addr(struct rte_pci_addr *addr, struct rte_pci_addr *addr2) */ int rte_eal_pci_probe(void); +#ifdef RTE_LIBRTE_EAL_HOTPLUG +/** + * Probe the single PCI device. + * + * Scan the content of the PCI bus, and find the pci device specified by pci + * address, then call the probe() function for registered driver that has a + * matching entry in its id_table for discovered device. + * + * @param addr + * The PCI Bus-Device-Function address to probe. + * @return + * - 0 on success. + * - Negative on error. + */ +int rte_eal_pci_probe_one(struct rte_pci_addr *addr); + +/** + * Close the single PCI device. + * + * Scan the content of the PCI bus, and find the pci device specified by pci + * address, then call the close() function for registered driver that has a + * matching entry in its id_table for discovered device. + * + * @param addr + * The PCI Bus-Device-Function address to close. + * @return + * - 0 on success. + * - Negative on error. + */ +int rte_eal_pci_close_one(struct rte_pci_addr *addr); +#endif /* RTE_LIBRTE_EAL_HOTPLUG */ + /** * Dump the content of the PCI bus. * diff --git a/lib/librte_eal/linuxapp/eal/eal_pci.c b/lib/librte_eal/linuxapp/eal/eal_pci.c index d03429c..f880f90 100644 --- a/lib/librte_eal/linuxapp/eal/eal_pci.c +++ b/lib/librte_eal/linuxapp/eal/eal_pci.c @@ -595,6 +595,31 @@ pci_map_device(struct rte_pci_device *dev) return ret; } +#ifdef RTE_LIBRTE_EAL_HOTPLUG +static void +pci_unmap_device(struct rte_pci_device *dev) +{ + if (dev == NULL) + return; + + /* try unmapping the NIC resources using VFIO if it exists */ + switch (dev->pt_driver) { + case RTE_PT_VFIO: + RTE_LOG(ERR, EAL, "Hotplug doesn't support vfio yet\n"); + break; + case RTE_PT_IGB_UIO: + case RTE_PT_UIO_GENERIC: + /* unmap resources for devices that use uio */ + pci_uio_unmap_resource(dev); + break; + default: + RTE_LOG(DEBUG, EAL, " Not managed by known pt driver," + " skipped\n"); + break; + } +} +#endif /* RTE_LIBRTE_EAL_HOTPLUG */ + /* * If vendor/device ID match, call the devinit() function of the * driver. @@ -666,6 +691,75 @@ rte_eal_pci_probe_one_driver(struct rte_pci_driver *dr, struct rte_pci_device *d return 1; } +#ifdef RTE_LIBRTE_EAL_HOTPLUG +/* + * If vendor/device ID match, call the devuninit() function of the + * driver. + */ +int +rte_eal_pci_close_one_driver(struct rte_pci_driver *dr, + struct rte_pci_device *dev) +{ + struct rte_pci_id *id_table; + + if ((dr == NULL) || (dev == NULL)) + return -EINVAL; + + for (id_table = dr->id_table ; id_table->vendor_id != 0; id_table++) { + + /* check if device's identifiers match the driver's ones */ + if (id_table->vendor_id != dev->id.vendor_id && + id_table->vendor_id != PCI_ANY_ID) + continue; + if (id_table->device_id != dev->id.device_id && + id_table->device_id != PCI_ANY_ID) + continue; + if (id_table->subsystem_vendor_id != + dev->id.subsystem_vendor_id && + id_table->subsystem_vendor_id != PCI_ANY_ID) + continue; + if (id_table->subsystem_device_id != + dev->id.subsystem_device_id && + id_table->subsystem_device_id != PCI_ANY_ID) + continue; + + struct rte_pci_addr *loc = &dev->addr; + + RTE_LOG(DEBUG, EAL, + "PCI device "PCI_PRI_FMT" on NUMA socket %i\n", + loc->domain, loc->bus, loc->devid, + loc->function, dev->numa_node); + + RTE_LOG(DEBUG, EAL, " remove driver: %x:%x %s\n", + dev->id.vendor_id, dev->id.device_id, + dr->name); + + /* call the driver devuninit() function */ + if (dr->devuninit && (dr->devuninit(dev) < 0)) + return -1; /* negative value is an error */ + + /* clear driver structure */ + dev->driver = NULL; + + if (dr->drv_flags & RTE_PCI_DRV_NEED_MAPPING) + /* unmap resources for devices that use igb_uio */ + pci_unmap_device(dev); + + return 0; + } + /* return positive value if driver is not found */ + return 1; +} +#else /* RTE_LIBRTE_EAL_HOTPLUG */ +int +rte_eal_pci_close_one_driver(struct rte_pci_driver *dr __rte_unused, + struct rte_pci_device *dev __rte_unused) +{ + RTE_LOG(ERR, EAL, "Hotplug support isn't enabled\n"); + return -1; +} +#endif /* RTE_LIBRTE_EAL_HOTPLUG */ + /* Init the PCI EAL subsystem */ int rte_eal_pci_init(void) diff --git a/lib/librte_eal/linuxapp/eal/rte_eal_version.map b/lib/librte_eal/linuxapp/eal/rte_eal_version.map index 17515a9..5478492 100644 --- a/lib/librte_eal/linuxapp/eal/rte_eal_version.map +++ b/lib/librte_eal/linuxapp/eal/rte_eal_version.map @@ -38,8 +38,10 @@ DPDK_2.0 { rte_eal_lcore_role; rte_eal_mp_remote_launch; rte_eal_mp_wait_lcore; + rte_eal_pci_close_one; rte_eal_pci_dump; rte_eal_pci_probe; + rte_eal_pci_probe_one; rte_eal_pci_register; rte_eal_pci_unregister; rte_eal_process_type; diff --git a/lib/librte_ether/rte_ethdev.c b/lib/librte_ether/rte_ethdev.c index a26af75..0ca7147 100644 --- a/lib/librte_ether/rte_ethdev.c +++ b/lib/librte_ether/rte_ethdev.c @@ -286,7 +286,7 @@ rte_eth_dev_release_port(struct rte_eth_dev *eth_dev) static int rte_eth_dev_init(struct rte_pci_driver *pci_drv, - struct rte_pci_device *pci_dev) + struct rte_pci_device *pci_dev) { struct eth_driver *eth_drv; struct rte_eth_dev *eth_dev; @@ -306,8 +306,8 @@ rte_eth_dev_init(struct rte_pci_driver *pci_drv, if (rte_eal_process_type() == RTE_PROC_PRIMARY){ eth_dev->data->dev_private = rte_zmalloc("ethdev private structure", - eth_drv->dev_private_size, - RTE_CACHE_LINE_SIZE); + eth_drv->dev_private_size, + RTE_CACHE_LINE_SIZE); if (eth_dev->data->dev_private == NULL) rte_panic("Cannot allocate memzone for private port data\n"); } -- 1.9.1
This new parameter is needed to keep device type like PCI or virtual. Port detaching processes are different between PCI device and virtual device. RTE_ETH_DEV_PCI indicates device type is PCI. RTE_ETH_DEV_VIRTUAL indicates device is virtual. v15: - Remove needless symbols in version.map - Move rte_eth_dev_get_device_type() to later patch to compile correctly. v12: - Add missing symbol in version map. (Thanks to Iremonger, Bernard) v10: - Change order of version.map. (Thanks to Thomas Monjalon) - Fix comment of "rte_ethdev.h". (Thanks to Thomas Monjalon) v9: - Fix commit log. - RTE_ETH_DEV_PHYSICAL is replaced by RTE_ETH_DEV_PCI. (Thanks to Thomas Monjalon) v8: - NONE_TRACE is replaced by NO_TRACE. - Add missing symbol in version map. (Thanks to Qiu, Michael and Iremonger, Bernard) v4: - Fix comments of rte_eth_dev_type. Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp> --- app/test/virtual_pmd.c | 2 +- lib/librte_ether/rte_ethdev.c | 5 +++-- lib/librte_ether/rte_ethdev.h | 16 +++++++++++++++- lib/librte_pmd_af_packet/rte_eth_af_packet.c | 2 +- lib/librte_pmd_bond/rte_eth_bond_api.c | 2 +- lib/librte_pmd_pcap/rte_eth_pcap.c | 2 +- lib/librte_pmd_ring/rte_eth_ring.c | 2 +- lib/librte_pmd_xenvirt/rte_eth_xenvirt.c | 2 +- 8 files changed, 24 insertions(+), 9 deletions(-) diff --git a/app/test/virtual_pmd.c b/app/test/virtual_pmd.c index 785bccc..9b07ab1 100644 --- a/app/test/virtual_pmd.c +++ b/app/test/virtual_pmd.c @@ -580,7 +580,7 @@ virtual_ethdev_create(const char *name, struct ether_addr *mac_addr, goto err; /* reserve an ethdev entry */ - eth_dev = rte_eth_dev_allocate(name); + eth_dev = rte_eth_dev_allocate(name, RTE_ETH_DEV_PCI); if (eth_dev == NULL) goto err; diff --git a/lib/librte_ether/rte_ethdev.c b/lib/librte_ether/rte_ethdev.c index 0ca7147..b07f2df 100644 --- a/lib/librte_ether/rte_ethdev.c +++ b/lib/librte_ether/rte_ethdev.c @@ -227,7 +227,7 @@ rte_eth_dev_find_free_port(void) } struct rte_eth_dev * -rte_eth_dev_allocate(const char *name) +rte_eth_dev_allocate(const char *name, enum rte_eth_dev_type type) { uint8_t port_id; struct rte_eth_dev *eth_dev; @@ -251,6 +251,7 @@ rte_eth_dev_allocate(const char *name) snprintf(eth_dev->data->name, sizeof(eth_dev->data->name), "%s", name); eth_dev->data->port_id = port_id; eth_dev->attached = DEV_ATTACHED; + eth_dev->dev_type = type; nb_ports++; return eth_dev; } @@ -300,7 +301,7 @@ rte_eth_dev_init(struct rte_pci_driver *pci_drv, rte_eth_dev_create_unique_device_name(ethdev_name, sizeof(ethdev_name), pci_dev); - eth_dev = rte_eth_dev_allocate(ethdev_name); + eth_dev = rte_eth_dev_allocate(ethdev_name, RTE_ETH_DEV_PCI); if (eth_dev == NULL) return -ENOMEM; diff --git a/lib/librte_ether/rte_ethdev.h b/lib/librte_ether/rte_ethdev.h index 37c3765..50c2dce 100644 --- a/lib/librte_ether/rte_ethdev.h +++ b/lib/librte_ether/rte_ethdev.h @@ -1422,6 +1422,17 @@ struct rte_eth_rxtx_callback { void *param; }; +/* + * The eth device type + */ +enum rte_eth_dev_type { + RTE_ETH_DEV_UNKNOWN, /**< unknown device type */ + RTE_ETH_DEV_PCI, + /**< Physical function and Virtual function of PCI devices */ + RTE_ETH_DEV_VIRTUAL, /**< non hardware device */ + RTE_ETH_DEV_MAX /**< max value of this enum */ +}; + /** * @internal * The generic data structure associated with each ethernet device. @@ -1452,6 +1463,7 @@ struct rte_eth_dev { */ struct rte_eth_rxtx_callback **pre_tx_burst_cbs; uint8_t attached; /**< Flag indicating the port is attached */ + enum rte_eth_dev_type dev_type; /**< Flag indicating the device type */ }; struct rte_eth_dev_sriov { @@ -1534,10 +1546,12 @@ extern uint8_t rte_eth_dev_count(void); * to that slot for the driver to use. * * @param name Unique identifier name for each Ethernet device + * @param type Device type of this Ethernet device * @return * - Slot in the rte_dev_devices array for a new device; */ -struct rte_eth_dev *rte_eth_dev_allocate(const char *name); +struct rte_eth_dev *rte_eth_dev_allocate(const char *name, + enum rte_eth_dev_type type); /** * Function for internal use by dummy drivers primarily, e.g. ring-based diff --git a/lib/librte_pmd_af_packet/rte_eth_af_packet.c b/lib/librte_pmd_af_packet/rte_eth_af_packet.c index 1ffe1cd..80e9bdf 100644 --- a/lib/librte_pmd_af_packet/rte_eth_af_packet.c +++ b/lib/librte_pmd_af_packet/rte_eth_af_packet.c @@ -649,7 +649,7 @@ rte_pmd_init_internals(const char *name, } /* reserve an ethdev entry */ - *eth_dev = rte_eth_dev_allocate(name); + *eth_dev = rte_eth_dev_allocate(name, RTE_ETH_DEV_VIRTUAL); if (*eth_dev == NULL) goto error; diff --git a/lib/librte_pmd_bond/rte_eth_bond_api.c b/lib/librte_pmd_bond/rte_eth_bond_api.c index 230d36c..903b7c3 100644 --- a/lib/librte_pmd_bond/rte_eth_bond_api.c +++ b/lib/librte_pmd_bond/rte_eth_bond_api.c @@ -257,7 +257,7 @@ rte_eth_bond_create(const char *name, uint8_t mode, uint8_t socket_id) } /* reserve an ethdev entry */ - eth_dev = rte_eth_dev_allocate(name); + eth_dev = rte_eth_dev_allocate(name, RTE_ETH_DEV_VIRTUAL); if (eth_dev == NULL) { RTE_BOND_LOG(ERR, "Unable to allocate rte_eth_dev"); goto err; diff --git a/lib/librte_pmd_pcap/rte_eth_pcap.c b/lib/librte_pmd_pcap/rte_eth_pcap.c index d299288..af7fae8 100644 --- a/lib/librte_pmd_pcap/rte_eth_pcap.c +++ b/lib/librte_pmd_pcap/rte_eth_pcap.c @@ -709,7 +709,7 @@ rte_pmd_init_internals(const char *name, const unsigned nb_rx_queues, goto error; /* reserve an ethdev entry */ - *eth_dev = rte_eth_dev_allocate(name); + *eth_dev = rte_eth_dev_allocate(name, RTE_ETH_DEV_VIRTUAL); if (*eth_dev == NULL) goto error; diff --git a/lib/librte_pmd_ring/rte_eth_ring.c b/lib/librte_pmd_ring/rte_eth_ring.c index a5dc71e..83ecfa8 100644 --- a/lib/librte_pmd_ring/rte_eth_ring.c +++ b/lib/librte_pmd_ring/rte_eth_ring.c @@ -297,7 +297,7 @@ rte_eth_from_rings(const char *name, struct rte_ring *const rx_queues[], goto error; /* reserve an ethdev entry */ - eth_dev = rte_eth_dev_allocate(name); + eth_dev = rte_eth_dev_allocate(name, RTE_ETH_DEV_VIRTUAL); if (eth_dev == NULL) goto error; diff --git a/lib/librte_pmd_xenvirt/rte_eth_xenvirt.c b/lib/librte_pmd_xenvirt/rte_eth_xenvirt.c index 04e30c9..bc403d6 100644 --- a/lib/librte_pmd_xenvirt/rte_eth_xenvirt.c +++ b/lib/librte_pmd_xenvirt/rte_eth_xenvirt.c @@ -648,7 +648,7 @@ eth_dev_xenvirt_create(const char *name, const char *params, goto err; /* reserve an ethdev entry */ - eth_dev = rte_eth_dev_allocate(name); + eth_dev = rte_eth_dev_allocate(name, RTE_ETH_DEV_VIRTUAL); if (eth_dev == NULL) goto err; -- 1.9.1
The patch adds following functions. - rte_eal_vdev_init(); - rte_eal_vdev_uninit(); - rte_eal_parse_devargs_str(). These functions are used for driver initialization and finalization. v15: - Add this patch to hotplug series. Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp> --- lib/librte_eal/common/eal_common_dev.c | 77 ++++++++++++++++++++----- lib/librte_eal/common/eal_common_devargs.c | 54 +++++++++++------ lib/librte_eal/common/include/rte_dev.h | 28 +++++++++ lib/librte_eal/common/include/rte_devargs.h | 28 +++++++++ lib/librte_eal/linuxapp/eal/rte_eal_version.map | 3 + 5 files changed, 155 insertions(+), 35 deletions(-) diff --git a/lib/librte_eal/common/eal_common_dev.c b/lib/librte_eal/common/eal_common_dev.c index eae5656..92a5a94 100644 --- a/lib/librte_eal/common/eal_common_dev.c +++ b/lib/librte_eal/common/eal_common_dev.c @@ -32,6 +32,7 @@ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ +#include <stdio.h> #include <string.h> #include <inttypes.h> #include <sys/queue.h> @@ -40,6 +41,7 @@ #include <rte_devargs.h> #include <rte_debug.h> #include <rte_devargs.h> +#include <rte_log.h> #include "eal_private.h" @@ -62,6 +64,32 @@ rte_eal_driver_unregister(struct rte_driver *driver) } int +rte_eal_vdev_init(const char *name, const char *args) +{ + struct rte_driver *driver; + + if (name == NULL) + return -EINVAL; + + TAILQ_FOREACH(driver, &dev_driver_list, next) { + if (driver->type != PMD_VDEV) + continue; + + /* + * search a driver prefix in virtual device name. + * For example, if the driver is pcap PMD, driver->name + * will be "eth_pcap", but "name" will be "eth_pcapN". + * So use strncmp to compare. + */ + if (!strncmp(driver->name, name, strlen(driver->name))) + return driver->init(name, args); + } + + RTE_LOG(ERR, EAL, "no driver found for %s\n", name); + return -EINVAL; +} + +int rte_eal_dev_init(void) { struct rte_devargs *devargs; @@ -79,22 +107,11 @@ rte_eal_dev_init(void) if (devargs->type != RTE_DEVTYPE_VIRTUAL) continue; - TAILQ_FOREACH(driver, &dev_driver_list, next) { - if (driver->type != PMD_VDEV) - continue; - - /* search a driver prefix in virtual device name */ - if (!strncmp(driver->name, devargs->virtual.drv_name, - strlen(driver->name))) { - driver->init(devargs->virtual.drv_name, - devargs->args); - break; - } - } - - if (driver == NULL) { - rte_panic("no driver found for %s\n", - devargs->virtual.drv_name); + if (rte_eal_vdev_init(devargs->virtual.drv_name, + devargs->args)) { + RTE_LOG(ERR, EAL, "failed to initialize %s device\n", + devargs->virtual.drv_name); + return -1; } } @@ -107,3 +124,31 @@ rte_eal_dev_init(void) } return 0; } + +#ifdef RTE_LIBRTE_EAL_HOTPLUG +int +rte_eal_vdev_uninit(const char *name) +{ + struct rte_driver *driver; + + if (name == NULL) + return -EINVAL; + + TAILQ_FOREACH(driver, &dev_driver_list, next) { + if (driver->type != PMD_VDEV) + continue; + + /* + * search a driver prefix in virtual device name. + * For example, if the driver is pcap PMD, driver->name + * will be "eth_pcap", but "name" will be "eth_pcapN". + * So use strncmp to compare. + */ + if (!strncmp(driver->name, name, strlen(driver->name))) + return driver->uninit(name); + } + + RTE_LOG(ERR, EAL, "no driver found for %s\n", name); + return -EINVAL; +} +#endif /* RTE_LIBRTE_EAL_HOTPLUG */ diff --git a/lib/librte_eal/common/eal_common_devargs.c b/lib/librte_eal/common/eal_common_devargs.c index eadd719..9b110f7 100644 --- a/lib/librte_eal/common/eal_common_devargs.c +++ b/lib/librte_eal/common/eal_common_devargs.c @@ -44,13 +44,46 @@ struct rte_devargs_list devargs_list = TAILQ_HEAD_INITIALIZER(devargs_list); +int +rte_eal_parse_devargs_str(const char *devargs_str, + char **drvname, char **drvargs) +{ + char *sep; + + if ((devargs_str) == NULL || (drvname) == NULL || (drvargs == NULL)) + return -1; + + *drvname = strdup(devargs_str); + if (drvname == NULL) { + RTE_LOG(ERR, EAL, + "cannot allocate temp memory for driver name\n"); + return -1; + } + + /* set the first ',' to '\0' to split name and arguments */ + sep = strchr(*drvname, ','); + if (sep != NULL) { + sep[0] = '\0'; + *drvargs = strdup(sep + 1); + } else { + *drvargs = strdup(""); + } + + if (*drvargs == NULL) { + RTE_LOG(ERR, EAL, + "cannot allocate temp memory for driver arguments\n"); + free(*drvname); + return -1; + } + return 0; +} + /* store a whitelist parameter for later parsing */ int rte_eal_devargs_add(enum rte_devtype devtype, const char *devargs_str) { struct rte_devargs *devargs = NULL; char *buf = NULL; - char *sep; int ret; /* use malloc instead of rte_malloc as it's called early at init */ @@ -62,25 +95,8 @@ rte_eal_devargs_add(enum rte_devtype devtype, const char *devargs_str) memset(devargs, 0, sizeof(*devargs)); devargs->type = devtype; - buf = strdup(devargs_str); - if (buf == NULL) { - RTE_LOG(ERR, EAL, "cannot allocate temp memory for devargs\n"); - goto fail; - } - - /* set the first ',' to '\0' to split name and arguments */ - sep = strchr(buf, ','); - if (sep != NULL) { - sep[0] = '\0'; - devargs->args = strdup(sep + 1); - } else { - devargs->args = strdup(""); - } - - if (devargs->args == NULL) { - RTE_LOG(ERR, EAL, "cannot allocate for devargs args\n"); + if (rte_eal_parse_devargs_str(devargs_str, &buf, &devargs->args)) goto fail; - } switch (devargs->type) { case RTE_DEVTYPE_WHITELISTED_PCI: diff --git a/lib/librte_eal/common/include/rte_dev.h b/lib/librte_eal/common/include/rte_dev.h index f7e3a10..f601d21 100644 --- a/lib/librte_eal/common/include/rte_dev.h +++ b/lib/librte_eal/common/include/rte_dev.h @@ -57,6 +57,11 @@ TAILQ_HEAD(rte_driver_list, rte_driver); typedef int (rte_dev_init_t)(const char *name, const char *args); /** + * Uninitilization function called for each device driver once. + */ +typedef int (rte_dev_uninit_t)(const char *name); + +/** * Driver type enumeration */ enum pmd_type { @@ -72,6 +77,7 @@ struct rte_driver { enum pmd_type type; /**< PMD Driver type */ const char *name; /**< Driver name. */ rte_dev_init_t *init; /**< Device init. function. */ + rte_dev_uninit_t *uninit; /**< Device uninit. function. */ }; /** @@ -97,6 +103,28 @@ void rte_eal_driver_unregister(struct rte_driver *driver); */ int rte_eal_dev_init(void); +/** + * Initialize a driver specified by name. + * + * @param name + * The pointer to a driver name to be initialized. + * @param args + * The pointer to arguments used by driver initialization. + * @return + * 0 on success, negative on error + */ +int rte_eal_vdev_init(const char *name, const char *args); + +/** + * Uninitalize a driver specified by name. + * + * @param name + * The pointer to a driver name to be initialized. + * @return + * 0 on success, negative on error + */ +int rte_eal_vdev_uninit(const char *name); + #define PMD_REGISTER_DRIVER(d)\ void devinitfn_ ##d(void);\ void __attribute__((constructor, used)) devinitfn_ ##d(void)\ diff --git a/lib/librte_eal/common/include/rte_devargs.h b/lib/librte_eal/common/include/rte_devargs.h index 6834333..039f728 100644 --- a/lib/librte_eal/common/include/rte_devargs.h +++ b/lib/librte_eal/common/include/rte_devargs.h @@ -99,6 +99,34 @@ TAILQ_HEAD(rte_devargs_list, rte_devargs); extern struct rte_devargs_list devargs_list; /** + * Parse a devargs string. + * + * For PCI devices, the format of arguments string is "PCI_ADDR" or + * "PCI_ADDR,key=val,key2=val2,...". Examples: "08:00.1", "0000:5:00.0", + * "04:00.0,arg=val". + * + * For virtual devices, the format of arguments string is "DRIVER_NAME*" + * or "DRIVER_NAME*,key=val,key2=val2,...". Examples: "eth_ring", + * "eth_ring0", "eth_pmdAnything,arg=0:arg2=1". + * + * The function parses the arguments string to get driver name and driver + * arguments. + * + * @param devargs_str + * The arguments as given by the user. + * @param drvname + * The pointer to the string to store parsed driver name. + * @param drvargs + * The pointer to the string to store parsed driver arguments. + * + * @return + * - 0 on success + * - A negative value on error + */ +int rte_eal_parse_devargs_str(const char *devargs_str, + char **drvname, char **drvargs); + +/** * Add a device to the user device list * * For PCI devices, the format of arguments string is "PCI_ADDR" or diff --git a/lib/librte_eal/linuxapp/eal/rte_eal_version.map b/lib/librte_eal/linuxapp/eal/rte_eal_version.map index 5478492..214643d 100644 --- a/lib/librte_eal/linuxapp/eal/rte_eal_version.map +++ b/lib/librte_eal/linuxapp/eal/rte_eal_version.map @@ -38,6 +38,7 @@ DPDK_2.0 { rte_eal_lcore_role; rte_eal_mp_remote_launch; rte_eal_mp_wait_lcore; + rte_eal_parse_devargs_str; rte_eal_pci_close_one; rte_eal_pci_dump; rte_eal_pci_probe; @@ -50,6 +51,8 @@ DPDK_2.0 { rte_eal_tailq_lookup_by_idx; rte_eal_tailq_reserve; rte_eal_tailq_reserve_by_idx; + rte_eal_vdev_init; + rte_eal_vdev_uninit; rte_eal_wait_lcore; rte_exit; rte_get_hpet_cycles; -- 1.9.1
These functions are used for attaching or detaching a port. When rte_eth_dev_attach() is called, the function tries to realize the device name as pci address. If this is done successfully, rte_eth_dev_attach() will attach physical device port. If not, attaches virtual devive port. When rte_eth_dev_detach() is called, the function gets the device type of this port to know whether the port is come from physical or virtual. And then specific detaching function will be called. v15: - Fix issue that eal calls ethedv library APIs. - Remove rte_eal_dev_attach(), and add rte_eth_dev_attach(). - Remove rte_eal_dev_detach(), and add rte_eth_dev_detach(). - Call rte_eal_vdev_init/uninit from ethdev library. (Thanks to Thomas Monjalon) - Fix version.map - Squash below patch to compile. - ethdev: Add functions that will be used by port hotplug functions v14: - Remove needless if statement. (Thanks to Maxime Leroy) v13: - Change log level when error occurs in rte_eal_vdev_init() and rte_eal_dev_init(). - Return value of driver init and uninit functions. - Replace rte_panic by RTE_LOG in rte_eal_dev_init() - Fix return value of rte_eal_vdev_uninit(). - Fix rte_eal_dev_attach_vdev to set port_id correctly. (Thanks to Maxime Leroy) v11: - Remove needless devargs handling codes. - Replace get_vdev_name() by rte_eal_parse_devargs_str(). - Replace rte_eal_vdev_find_and_init by rte_eal_vdev_init() - Replace rte_eal_vdev_find_and_uninit by rte_eal_vdev_uninit() - Fix rte_eal_dev_init() to use rte_eal_vdev_init(). (Thanks to Maxime Leroy) v10: - Add comments. - Change order of version.map. (Thanks to Thomas Monjalon) v9: - Fix comments. - Use strcmp() instead of strncmp(). - Remove RTE_EAL_INVOKE_TYPE_PROBE/CLOSE. - Change definition of rte_dev_uninit_t. (Thanks to Thomas Monjalon and Maxime Leroy) v8: - Add missing symbol in version map. (Thanks to Qiu, Michael and Iremonger, Bernard) v7: - Fix typo of warning messages. (Thanks to Qiu, Michael) v5: - Change function names like below. rte_eal_dev_find_and_invoke() to rte_eal_vdev_find_and_invoke(). rte_eal_dev_invoke() to rte_eal_vdev_invoke(). - Add code to handle a return value of rte_eal_devargs_remove(). - Fix pci address format in rte_eal_dev_detach(). v4: - Fix comment. - Add error checking. - Fix indent of 'if' statement. - Change function name. Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp> --- lib/librte_eal/common/include/rte_pci.h | 9 + lib/librte_eal/linuxapp/eal/eal_pci.c | 6 +- lib/librte_eal/linuxapp/eal/rte_eal_version.map | 1 + lib/librte_ether/rte_ethdev.c | 303 +++++++++++++++++++++++- lib/librte_ether/rte_ethdev.h | 38 +++ lib/librte_ether/rte_ether_version.map | 3 + 6 files changed, 355 insertions(+), 5 deletions(-) diff --git a/lib/librte_eal/common/include/rte_pci.h b/lib/librte_eal/common/include/rte_pci.h index ac30925..b9cdf8b 100644 --- a/lib/librte_eal/common/include/rte_pci.h +++ b/lib/librte_eal/common/include/rte_pci.h @@ -313,6 +313,15 @@ rte_eal_compare_pci_addr(struct rte_pci_addr *addr, struct rte_pci_addr *addr2) } /** + * Scan the content of the PCI bus, and the devices in the devices + * list + * + * @return + * 0 on success, negative on error + */ +int rte_eal_pci_scan(void); + +/** * Probe the PCI bus for registered drivers. * * Scan the content of the PCI bus, and call the probe() function for diff --git a/lib/librte_eal/linuxapp/eal/eal_pci.c b/lib/librte_eal/linuxapp/eal/eal_pci.c index f880f90..6d4932d 100644 --- a/lib/librte_eal/linuxapp/eal/eal_pci.c +++ b/lib/librte_eal/linuxapp/eal/eal_pci.c @@ -440,8 +440,8 @@ error: * Scan the content of the PCI bus, and the devices in the devices * list */ -static int -pci_scan(void) +int +rte_eal_pci_scan(void) { struct dirent *e; DIR *dir; @@ -773,7 +773,7 @@ rte_eal_pci_init(void) if (internal_config.no_pci) return 0; - if (pci_scan() < 0) { + if (rte_eal_pci_scan() < 0) { RTE_LOG(ERR, EAL, "%s(): Cannot scan PCI bus\n", __func__); return -1; } diff --git a/lib/librte_eal/linuxapp/eal/rte_eal_version.map b/lib/librte_eal/linuxapp/eal/rte_eal_version.map index 214643d..7c2aac3 100644 --- a/lib/librte_eal/linuxapp/eal/rte_eal_version.map +++ b/lib/librte_eal/linuxapp/eal/rte_eal_version.map @@ -45,6 +45,7 @@ DPDK_2.0 { rte_eal_pci_probe_one; rte_eal_pci_register; rte_eal_pci_unregister; + rte_eal_pci_scan; rte_eal_process_type; rte_eal_remote_launch; rte_eal_tailq_lookup; diff --git a/lib/librte_ether/rte_ethdev.c b/lib/librte_ether/rte_ethdev.c index b07f2df..868289c 100644 --- a/lib/librte_ether/rte_ethdev.c +++ b/lib/librte_ether/rte_ethdev.c @@ -201,7 +201,7 @@ rte_eth_dev_data_alloc(void) RTE_MAX_ETHPORTS * sizeof(*rte_eth_dev_data)); } -static struct rte_eth_dev * +struct rte_eth_dev * rte_eth_dev_allocated(const char *name) { unsigned i; @@ -270,7 +270,6 @@ rte_eth_dev_create_unique_device_name(char *name, size_t size, pci_dev->addr.function); if (ret < 0) return ret; - return 0; } @@ -427,6 +426,306 @@ rte_eth_dev_count(void) return (nb_ports); } +static enum rte_eth_dev_type +rte_eth_dev_get_device_type(uint8_t port_id) +{ + if (!rte_eth_dev_is_valid_port(port_id)) + return -1; + return rte_eth_devices[port_id].dev_type; +} + +static int +rte_eth_dev_save(struct rte_eth_dev *devs, size_t size) +{ + if ((devs == NULL) || + (size != sizeof(struct rte_eth_dev) * RTE_MAX_ETHPORTS)) + return -EINVAL; + + /* save current rte_eth_devices */ + memcpy(devs, rte_eth_devices, size); + return 0; +} + +static int +rte_eth_dev_get_changed_port(struct rte_eth_dev *devs, uint8_t *port_id) +{ + if ((devs == NULL) || (port_id == NULL)) + return -EINVAL; + + /* check which port was attached or detached */ + for (*port_id = 0; *port_id < RTE_MAX_ETHPORTS; (*port_id)++, devs++) { + if (rte_eth_devices[*port_id].attached ^ devs->attached) + return 0; + } + return -ENODEV; +} + +static int +rte_eth_dev_get_addr_by_port(uint8_t port_id, struct rte_pci_addr *addr) +{ + if (!rte_eth_dev_is_valid_port(port_id)) { + PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + return -EINVAL; + } + + if (addr == NULL) { + PMD_DEBUG_TRACE("Null pointer is specified\n"); + return -EINVAL; + } + + *addr = rte_eth_devices[port_id].pci_dev->addr; + return 0; +} + +static int +rte_eth_dev_get_name_by_port(uint8_t port_id, char *name) +{ + char *tmp; + + if (!rte_eth_dev_is_valid_port(port_id)) { + PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + return -EINVAL; + } + + if (name == NULL) { + PMD_DEBUG_TRACE("Null pointer is specified\n"); + return -EINVAL; + } + + /* shouldn't check 'rte_eth_devices[i].data', + * because it might be overwritten by VDEV PMD */ + tmp = rte_eth_dev_data[port_id].name; + strcpy(name, tmp); + return 0; +} + +static int +rte_eth_dev_is_detachable(uint8_t port_id) +{ + uint32_t drv_flags; + + if (port_id >= RTE_MAX_ETHPORTS) { + PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id); + return -EINVAL; + } + + if (rte_eth_devices[port_id].dev_type == RTE_ETH_DEV_PCI) { + switch (rte_eth_devices[port_id].pci_dev->pt_driver) { + case RTE_PT_IGB_UIO: + case RTE_PT_UIO_GENERIC: + break; + case RTE_PT_VFIO: + default: + return -ENOTSUP; + } + } + + drv_flags = rte_eth_devices[port_id].driver->pci_drv.drv_flags; + return !(drv_flags & RTE_PCI_DRV_DETACHABLE); +} + +/* So far, DPDK hotplug function only supports linux */ +#ifdef RTE_LIBRTE_EAL_HOTPLUG +/* attach the new physical device, then store port_id of the device */ +static int +rte_eth_dev_attach_pdev(struct rte_pci_addr *addr, uint8_t *port_id) +{ + uint8_t new_port_id; + struct rte_eth_dev devs[RTE_MAX_ETHPORTS]; + + if ((addr == NULL) || (port_id == NULL)) + goto err; + + /* save current port status */ + if (rte_eth_dev_save(devs, sizeof(devs))) + goto err; + /* re-construct pci_device_list */ + if (rte_eal_pci_scan()) + goto err; + /* invoke probe func of the driver can handle the new device. + * TODO: + * rte_eal_pci_probe_one() should return port_id. + * And rte_eth_dev_save() and rte_eth_dev_get_changed_port() + * should be removed. */ + if (rte_eal_pci_probe_one(addr)) + goto err; + /* get port_id enabled by above procedures */ + if (rte_eth_dev_get_changed_port(devs, &new_port_id)) + goto err; + + *port_id = new_port_id; + return 0; +err: + RTE_LOG(ERR, EAL, "Driver, cannot attach the device\n"); + return -1; +} + +/* detach the new physical device, then store pci_addr of the device */ +static int +rte_eth_dev_detach_pdev(uint8_t port_id, struct rte_pci_addr *addr) +{ + struct rte_pci_addr freed_addr; + struct rte_pci_addr vp; + + if (addr == NULL) + goto err; + + /* check whether the driver supports detach feature, or not */ + if (rte_eth_dev_is_detachable(port_id)) + goto err; + + /* get pci address by port id */ + if (rte_eth_dev_get_addr_by_port(port_id, &freed_addr)) + goto err; + + /* Zerod pci addr means the port comes from virtual device */ + vp.domain = vp.bus = vp.devid = vp.function = 0; + if (rte_eal_compare_pci_addr(&vp, &freed_addr) == 0) + goto err; + + /* invoke close func of the driver, + * also remove the device from pci_device_list */ + if (rte_eal_pci_close_one(&freed_addr)) + goto err; + + *addr = freed_addr; + return 0; +err: + RTE_LOG(ERR, EAL, "Driver, cannot detach the device\n"); + return -1; +} + +/* attach the new virtual device, then store port_id of the device */ +static int +rte_eth_dev_attach_vdev(const char *vdevargs, uint8_t *port_id) +{ + char *name = NULL, *args = NULL; + uint8_t new_port_id; + struct rte_eth_dev devs[RTE_MAX_ETHPORTS]; + int ret = -1; + + if ((vdevargs == NULL) || (port_id == NULL)) + goto end; + + /* parse vdevargs, then retrieve device name and args */ + if (rte_eal_parse_devargs_str(vdevargs, &name, &args)) + goto end; + + /* save current port status */ + if (rte_eth_dev_save(devs, sizeof(devs))) + goto end; + /* walk around dev_driver_list to find the driver of the device, + * then invoke probe function o the driver. + * TODO: + * rte_eal_vdev_init() should return port_id, + * And rte_eth_dev_save() and rte_eth_dev_get_changed_port() + * should be removed. */ + if (rte_eal_vdev_init(name, args)) + goto end; + /* get port_id enabled by above procedures */ + if (rte_eth_dev_get_changed_port(devs, &new_port_id)) + goto end; + ret = 0; + *port_id = new_port_id; +end: + if (name) + free(name); + if (args) + free(args); + + if (ret < 0) + RTE_LOG(ERR, EAL, "Driver, cannot attach the device\n"); + return ret; +} + +/* detach the new virtual device, then store the name of the device */ +static int +rte_eth_dev_detach_vdev(uint8_t port_id, char *vdevname) +{ + char name[RTE_ETH_NAME_MAX_LEN]; + + if (vdevname == NULL) + goto err; + + /* check whether the driver supports detach feature, or not */ + if (rte_eth_dev_is_detachable(port_id)) + goto err; + + /* get device name by port id */ + if (rte_eth_dev_get_name_by_port(port_id, name)) + goto err; + /* walk around dev_driver_list to find the driver of the device, + * then invoke close function o the driver */ + if (rte_eal_vdev_uninit(name)) + goto err; + + strncpy(vdevname, name, sizeof(name)); + return 0; +err: + RTE_LOG(ERR, EAL, "Driver, cannot detach the device\n"); + return -1; +} + +/* attach the new device, then store port_id of the device */ +int +rte_eth_dev_attach(const char *devargs, uint8_t *port_id) +{ + struct rte_pci_addr addr; + + if ((devargs == NULL) || (port_id == NULL)) + return -EINVAL; + + if (eal_parse_pci_DomBDF(devargs, &addr) == 0) + return rte_eth_dev_attach_pdev(&addr, port_id); + else + return rte_eth_dev_attach_vdev(devargs, port_id); +} + +/* detach the device, then store the name of the device */ +int +rte_eth_dev_detach(uint8_t port_id, char *name) +{ + struct rte_pci_addr addr; + int ret; + + if (name == NULL) + return -EINVAL; + + if (rte_eth_dev_get_device_type(port_id) == RTE_ETH_DEV_PCI) { + ret = rte_eth_dev_get_addr_by_port(port_id, &addr); + if (ret < 0) + return ret; + + ret = rte_eth_dev_detach_pdev(port_id, &addr); + if (ret == 0) + snprintf(name, RTE_ETH_NAME_MAX_LEN, + "%04x:%02x:%02x.%d", + addr.domain, addr.bus, + addr.devid, addr.function); + + return ret; + } else + return rte_eth_dev_detach_vdev(port_id, name); +} +#else /* RTE_LIBRTE_EAL_HOTPLUG */ +int +rte_eth_dev_attach(const char *devargs __rte_unused, + uint8_t *port_id __rte_unused) +{ + RTE_LOG(ERR, EAL, "Hotplug support isn't enabled\n"); + return -1; +} + +/* detach the device, then store the name of the device */ +int +rte_eth_dev_detach(uint8_t port_id __rte_unused, + char *name __rte_unused) +{ + RTE_LOG(ERR, EAL, "Hotplug support isn't enabled\n"); + return -1; +} +#endif /* RTE_LIBRTE_EAL_HOTPLUG */ + static int rte_eth_dev_rx_queue_config(struct rte_eth_dev *dev, uint16_t nb_queues) { diff --git a/lib/librte_ether/rte_ethdev.h b/lib/librte_ether/rte_ethdev.h index 50c2dce..53d146c 100644 --- a/lib/librte_ether/rte_ethdev.h +++ b/lib/librte_ether/rte_ethdev.h @@ -175,6 +175,8 @@ extern "C" { #include <rte_log.h> #include <rte_interrupts.h> #include <rte_pci.h> +#include <rte_dev.h> +#include <rte_devargs.h> #include <rte_mbuf.h> #include "rte_ether.h" #include "rte_eth_ctrl.h" @@ -1540,6 +1542,16 @@ extern struct rte_eth_dev rte_eth_devices[]; extern uint8_t rte_eth_dev_count(void); /** + * Function for internal use by port hotplug functions. + * Returns a ethdev slot specified by the unique identifier name. + * @param name + * The pointer to the Unique identifier name for each Ethernet device + * @return + * - The pointer to the ethdev slot, on success. NULL on error + */ +extern struct rte_eth_dev *rte_eth_dev_allocated(const char *name); + +/** * Function for internal use by dummy drivers primarily, e.g. ring-based * driver. * Allocates a new ethdev slot for an ethernet device and returns the pointer @@ -1565,6 +1577,32 @@ struct rte_eth_dev *rte_eth_dev_allocate(const char *name, */ int rte_eth_dev_release_port(struct rte_eth_dev *eth_dev); +/** + * Attach a new Ethernet device specified by aruguments. + * + * @param devargs + * A pointer to a strings array describing the new device + * to be attached. The strings should be a pci address like + * '0000:01:00.0' or virtual device name like 'eth_pcap0'. + * @param port_id + * A pointer to a port identifier actually attached. + * @return + * 0 on success and port_id is filled, negative on error + */ +int rte_eth_dev_attach(const char *devargs, uint8_t *port_id); + +/** + * Detach a Ethernet device specified by port identifier. + * + * @param port_id + * The port identifier of the device to detach. + * @param addr + * A pointer to a device name actually detached. + * @return + * 0 on success and devname is filled, negative on error + */ +int rte_eth_dev_detach(uint8_t port_id, char *devname); + struct eth_driver; /** * @internal diff --git a/lib/librte_ether/rte_ether_version.map b/lib/librte_ether/rte_ether_version.map index 94fd685..0d46578 100644 --- a/lib/librte_ether/rte_ether_version.map +++ b/lib/librte_ether/rte_ether_version.map @@ -8,6 +8,8 @@ DPDK_2.0 { rte_eth_allmulticast_enable; rte_eth_allmulticast_get; rte_eth_dev_allocate; + rte_eth_dev_allocated; + rte_eth_dev_attach; rte_eth_dev_bypass_event_show; rte_eth_dev_bypass_event_store; rte_eth_dev_bypass_init; @@ -22,6 +24,7 @@ DPDK_2.0 { rte_eth_dev_close; rte_eth_dev_configure; rte_eth_dev_count; + rte_eth_dev_detach; rte_eth_dev_fdir_add_perfect_filter; rte_eth_dev_fdir_add_signature_filter; rte_eth_dev_fdir_get_infos; -- 1.9.1
This patch adds a new section for describing port hotplug framework. v15: - Fix function names like below. - rte_eal_dev_attach() to rte_eth_dev_attach(). - rte_eal_dev_detach() to rte_eth_dev_detach(). Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp> --- doc/guides/prog_guide/index.rst | 1 + doc/guides/prog_guide/port_hotplug_framework.rst | 110 +++++++++++++++++++++++ 2 files changed, 111 insertions(+) create mode 100644 doc/guides/prog_guide/port_hotplug_framework.rst diff --git a/doc/guides/prog_guide/index.rst b/doc/guides/prog_guide/index.rst index de69682..60a6ac5 100644 --- a/doc/guides/prog_guide/index.rst +++ b/doc/guides/prog_guide/index.rst @@ -71,6 +71,7 @@ Programmer's Guide packet_classif_access_ctrl packet_framework vhost_lib + port_hotplug_framework source_org dev_kit_build_system dev_kit_root_make_help diff --git a/doc/guides/prog_guide/port_hotplug_framework.rst b/doc/guides/prog_guide/port_hotplug_framework.rst new file mode 100644 index 0000000..fe6d72a --- /dev/null +++ b/doc/guides/prog_guide/port_hotplug_framework.rst @@ -0,0 +1,110 @@ +.. BSD LICENSE + Copyright(c) 2015 IGEL Co.,Ltd. All rights reserved. + All rights reserved. + + Redistribution and use in source and binary forms, with or without + modification, are permitted provided that the following conditions + are met: + + * Redistributions of source code must retain the above copyright + notice, this list of conditions and the following disclaimer. + * Redistributions in binary form must reproduce the above copyright + notice, this list of conditions and the following disclaimer in + the documentation and/or other materials provided with the + distribution. + * Neither the name of IGEL Co.,Ltd. nor the names of its + contributors may be used to endorse or promote products derived + from this software without specific prior written permission. + + THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS + "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT + LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR + A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT + OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, + SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT + LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, + DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY + THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT + (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE + OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + +Port Hotplug Framework +====================== + +The Port Hotplug Framework provides DPDK applications with the ability to +attach and detach ports at runtime. Because the framework depends on PMD +implementation, the ports that PMDs cannot handle are out of scope of this +framework. Furthermore, after detaching a port from a DPDK application, the +framework doesn't provide a way for removing the devices from the system. +For the ports backed by a physical NIC, the kernel will need to support PCI +Hotplug feature. + +Overview +-------- + +The basic requirements of the Port Hotplug Framework are: + +* DPDK applications that use the Port Hotplug Framework must manage their + own ports. + + The Port Hotplug Framework is implemented to allow DPDK applications to + manage ports. For example, when DPDK applications call the port attach + function, the attached port number is returned. DPDK applications can + also detach the port by port number. + +* Kernel support is needed for attaching or detaching physical device + ports. + + To attach new physical device ports, the device will be recognized by + userspace driver I/O framework in kernel at first. Then DPDK + applications can call the Port Hotplug functions to attach the ports. + For detaching, steps are vice versa. + +* Before detaching, they must be stopped and closed. + + DPDK applications must call "rte_eth_dev_stop()" and + "rte_eth_dev_close()" APIs before detaching ports. These functions will + start finalization sequence of the PMDs. + +* The framework doesn't affect legacy DPDK applications behavior. + + If the Port Hotplug functions aren't called, all legacy DPDK apps can + still work without modifications. + +Port Hotplug API overview +------------------------- + +* Attaching a port + + "rte_eth_dev_attach()" API attaches a port to DPDK application, and + returns the attached port number. Before calling the API, the device + should be recognized by an userspace driver I/O framework. The API + receives a pci address like "0000:01:00.0" or a virtual device name + like "eth_pcap0,iface=eth0". In the case of virtual device name, the + format is the same as the general "--vdev" option of DPDK. + +* Detaching a port + + "rte_eth_dev_detach()" API detaches a port from DPDK application, and + returns a pci address of the detached device or a virtual device name + of the device. + +Reference +--------- + + "testpmd" supports the Port Hotplug Framework. + +Limitations +----------- + +* The Port Hotplug APIs are not thread safe. + +* The framework can only be enabled with Linux. BSD is not supported. + +* To detach a port, the port should be backed by a device that igb_uio + manages. VFIO is not supported. + +* Not all PMDs support detaching feature. + To know whether a PMD can support detaching, search for the + "RTE_PCI_DRV_DETACHABLE" flag in PMD implementation. If the flag is + defined in the PMD, detaching is supported. -- 1.9.1
This patch adds finalization code to free resources allocated by the PMD. v6: - Fix a paramter of rte_eth_dev_free(). v4: - Change function name. Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp> --- lib/librte_pmd_pcap/rte_eth_pcap.c | 40 ++++++++++++++++++++++++++++++++++++++ 1 file changed, 40 insertions(+) diff --git a/lib/librte_pmd_pcap/rte_eth_pcap.c b/lib/librte_pmd_pcap/rte_eth_pcap.c index af7fae8..5e94930 100644 --- a/lib/librte_pmd_pcap/rte_eth_pcap.c +++ b/lib/librte_pmd_pcap/rte_eth_pcap.c @@ -498,6 +498,13 @@ static struct eth_dev_ops ops = { .stats_reset = eth_stats_reset, }; +static struct eth_driver rte_pcap_pmd = { + .pci_drv = { + .name = "rte_pcap_pmd", + .drv_flags = RTE_PCI_DRV_DETACHABLE, + }, +}; + /* * Function handler that opens the pcap file for reading a stores a * reference of it for use it later on. @@ -713,6 +720,10 @@ rte_pmd_init_internals(const char *name, const unsigned nb_rx_queues, if (*eth_dev == NULL) goto error; + /* check length of device name */ + if ((strlen((*eth_dev)->data->name) + 1) > sizeof(data->name)) + goto error; + /* now put it all together * - store queue data in internals, * - store numa_node info in pci_driver @@ -739,10 +750,13 @@ rte_pmd_init_internals(const char *name, const unsigned nb_rx_queues, data->nb_tx_queues = (uint16_t)nb_tx_queues; data->dev_link = pmd_link; data->mac_addrs = ð_addr; + strncpy(data->name, + (*eth_dev)->data->name, strlen((*eth_dev)->data->name)); (*eth_dev)->data = data; (*eth_dev)->dev_ops = &ops; (*eth_dev)->pci_dev = pci_dev; + (*eth_dev)->driver = &rte_pcap_pmd; return 0; @@ -927,10 +941,36 @@ rte_pmd_pcap_devinit(const char *name, const char *params) } +static int +rte_pmd_pcap_devuninit(const char *name) +{ + struct rte_eth_dev *eth_dev = NULL; + + RTE_LOG(INFO, PMD, "Closing pcap ethdev on numa socket %u\n", + rte_socket_id()); + + if (name == NULL) + return -1; + + /* reserve an ethdev entry */ + eth_dev = rte_eth_dev_allocated(name); + if (eth_dev == NULL) + return -1; + + rte_free(eth_dev->data->dev_private); + rte_free(eth_dev->data); + rte_free(eth_dev->pci_dev); + + rte_eth_dev_release_port(eth_dev); + + return 0; +} + static struct rte_driver pmd_pcap_drv = { .name = "eth_pcap", .type = PMD_VDEV, .init = rte_pmd_pcap_devinit, + .uninit = rte_pmd_pcap_devuninit, }; PMD_REGISTER_DRIVER(pmd_pcap_drv); -- 1.9.1
The patch introduces following commands. - port attach [ident] - port detach [port_id] - attach: attaching a port - detach: detaching a port - ident: pci address of physical device. Or device name and parameters of virtual device. (ex. 0000:02:00.0, eth_pcap0,iface=eth0) - port_id: port identifier v15: - Replace rte_eal_dev_attach() by rte_eth_dev_attach() - Replace rte_eal_dev_detach() by rte_eth_dev_detach() v7: - Fix doc. (Thanks to Iremonger, Bernard) - Fix port checking implementation of star_port(); (Thanks to Qiu, Michael) v5: - Add testpmd documentation. (Thanks to Iremonger, Bernard) v4: - Fix strings of command help. Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp> --- app/test-pmd/cmdline.c | 137 +++++++++++++++---- app/test-pmd/config.c | 102 ++++++++------ app/test-pmd/parameters.c | 22 ++- app/test-pmd/testpmd.c | 199 +++++++++++++++++++++------- app/test-pmd/testpmd.h | 18 ++- doc/guides/testpmd_app_ug/testpmd_funcs.rst | 57 ++++++++ 6 files changed, 409 insertions(+), 126 deletions(-) diff --git a/app/test-pmd/cmdline.c b/app/test-pmd/cmdline.c index 4c9f423..c8312be 100644 --- a/app/test-pmd/cmdline.c +++ b/app/test-pmd/cmdline.c @@ -513,6 +513,12 @@ static void cmd_help_long_parsed(void *parsed_result, "port close (port_id|all)\n" " Close all ports or port_id.\n\n" + "port attach (ident)\n" + " Attach physical or virtual dev by pci address or virtual device name\n\n" + + "port detach (port_id)\n" + " Detach physical or virtual dev by port_id\n\n" + "port config (port_id|all)" " speed (10|100|1000|10000|40000|auto)" " duplex (half|full|auto)\n" @@ -793,6 +799,89 @@ cmdline_parse_inst_t cmd_operate_specific_port = { }, }; +/* *** attach a specified port *** */ +struct cmd_operate_attach_port_result { + cmdline_fixed_string_t port; + cmdline_fixed_string_t keyword; + cmdline_fixed_string_t identifier; +}; + +static void cmd_operate_attach_port_parsed(void *parsed_result, + __attribute__((unused)) struct cmdline *cl, + __attribute__((unused)) void *data) +{ + struct cmd_operate_attach_port_result *res = parsed_result; + + if (!strcmp(res->keyword, "attach")) + attach_port(res->identifier); + else + printf("Unknown parameter\n"); +} + +cmdline_parse_token_string_t cmd_operate_attach_port_port = + TOKEN_STRING_INITIALIZER(struct cmd_operate_attach_port_result, + port, "port"); +cmdline_parse_token_string_t cmd_operate_attach_port_keyword = + TOKEN_STRING_INITIALIZER(struct cmd_operate_attach_port_result, + keyword, "attach"); +cmdline_parse_token_string_t cmd_operate_attach_port_identifier = + TOKEN_STRING_INITIALIZER(struct cmd_operate_attach_port_result, + identifier, NULL); + +cmdline_parse_inst_t cmd_operate_attach_port = { + .f = cmd_operate_attach_port_parsed, + .data = NULL, + .help_str = "port attach identifier, " + "identifier: pci address or virtual dev name", + .tokens = { + (void *)&cmd_operate_attach_port_port, + (void *)&cmd_operate_attach_port_keyword, + (void *)&cmd_operate_attach_port_identifier, + NULL, + }, +}; + +/* *** detach a specified port *** */ +struct cmd_operate_detach_port_result { + cmdline_fixed_string_t port; + cmdline_fixed_string_t keyword; + uint8_t port_id; +}; + +static void cmd_operate_detach_port_parsed(void *parsed_result, + __attribute__((unused)) struct cmdline *cl, + __attribute__((unused)) void *data) +{ + struct cmd_operate_detach_port_result *res = parsed_result; + + if (!strcmp(res->keyword, "detach")) + detach_port(res->port_id); + else + printf("Unknown parameter\n"); +} + +cmdline_parse_token_string_t cmd_operate_detach_port_port = + TOKEN_STRING_INITIALIZER(struct cmd_operate_detach_port_result, + port, "port"); +cmdline_parse_token_string_t cmd_operate_detach_port_keyword = + TOKEN_STRING_INITIALIZER(struct cmd_operate_detach_port_result, + keyword, "detach"); +cmdline_parse_token_num_t cmd_operate_detach_port_port_id = + TOKEN_NUM_INITIALIZER(struct cmd_operate_detach_port_result, + port_id, UINT8); + +cmdline_parse_inst_t cmd_operate_detach_port = { + .f = cmd_operate_detach_port_parsed, + .data = NULL, + .help_str = "port detach port_id", + .tokens = { + (void *)&cmd_operate_detach_port_port, + (void *)&cmd_operate_detach_port_keyword, + (void *)&cmd_operate_detach_port_port_id, + NULL, + }, +}; + /* *** configure speed for all ports *** */ struct cmd_config_speed_all { cmdline_fixed_string_t port; @@ -847,7 +936,7 @@ cmd_config_speed_all_parsed(void *parsed_result, return; } - for (pid = 0; pid < nb_ports; pid++) { + FOREACH_PORT(pid, ports) { ports[pid].dev_conf.link_speed = link_speed; ports[pid].dev_conf.link_duplex = link_duplex; } @@ -915,10 +1004,8 @@ cmd_config_speed_specific_parsed(void *parsed_result, return; } - if (res->id >= nb_ports) { - printf("Port id %d must be less than %d\n", res->id, nb_ports); + if (port_id_is_invalid(res->id, ENABLED_WARN)) return; - } if (!strcmp(res->value1, "10")) link_speed = ETH_LINK_SPEED_10; @@ -1489,7 +1576,7 @@ cmd_config_rxtx_queue_parsed(void *parsed_result, return; } - if (port_id_is_invalid(res->portid)) + if (port_id_is_invalid(res->portid, ENABLED_WARN)) return; if (port_is_started(res->portid) != 1) { @@ -2889,7 +2976,7 @@ cmd_csum_parsed(void *parsed_result, int hw = 0; uint16_t mask = 0; - if (port_id_is_invalid(res->port_id)) { + if (port_id_is_invalid(res->port_id, ENABLED_WARN)) { printf("invalid port %d\n", res->port_id); return; } @@ -2981,10 +3068,8 @@ cmd_csum_tunnel_parsed(void *parsed_result, { struct cmd_csum_tunnel_result *res = parsed_result; - if (port_id_is_invalid(res->port_id)) { - printf("invalid port %d\n", res->port_id); + if (port_id_is_invalid(res->port_id, ENABLED_WARN)) return; - } if (!strcmp(res->onoff, "on")) ports[res->port_id].tx_ol_flags |= @@ -3039,7 +3124,7 @@ cmd_tso_set_parsed(void *parsed_result, struct cmd_tso_set_result *res = parsed_result; struct rte_eth_dev_info dev_info; - if (port_id_is_invalid(res->port_id)) + if (port_id_is_invalid(res->port_id, ENABLED_WARN)) return; if (!strcmp(res->mode, "set")) @@ -4015,10 +4100,8 @@ static void cmd_set_bond_mac_addr_parsed(void *parsed_result, struct cmd_set_bond_mac_addr_result *res = parsed_result; int ret; - if (res->port_num >= nb_ports) { - printf("Port id %d must be less than %d\n", res->port_num, nb_ports); + if (port_id_is_invalid(res->port_num, ENABLED_WARN)) return; - } ret = rte_eth_bond_mac_address_set(res->port_num, &res->address); @@ -4255,7 +4338,7 @@ static void cmd_set_promisc_mode_parsed(void *parsed_result, /* all ports */ if (allports) { - for (i = 0; i < nb_ports; i++) { + FOREACH_PORT(i, ports) { if (enable) rte_eth_promiscuous_enable(i); else @@ -4335,7 +4418,7 @@ static void cmd_set_allmulti_mode_parsed(void *parsed_result, /* all ports */ if (allports) { - for (i = 0; i < nb_ports; i++) { + FOREACH_PORT(i, ports) { if (enable) rte_eth_allmulticast_enable(i); else @@ -5023,25 +5106,25 @@ static void cmd_showportall_parsed(void *parsed_result, struct cmd_showportall_result *res = parsed_result; if (!strcmp(res->show, "clear")) { if (!strcmp(res->what, "stats")) - for (i = 0; i < nb_ports; i++) + FOREACH_PORT(i, ports) nic_stats_clear(i); else if (!strcmp(res->what, "xstats")) - for (i = 0; i < nb_ports; i++) + FOREACH_PORT(i, ports) nic_xstats_clear(i); } else if (!strcmp(res->what, "info")) - for (i = 0; i < nb_ports; i++) + FOREACH_PORT(i, ports) port_infos_display(i); else if (!strcmp(res->what, "stats")) - for (i = 0; i < nb_ports; i++) + FOREACH_PORT(i, ports) nic_stats_display(i); else if (!strcmp(res->what, "xstats")) - for (i = 0; i < nb_ports; i++) + FOREACH_PORT(i, ports) nic_xstats_display(i); else if (!strcmp(res->what, "fdir")) - for (i = 0; i < nb_ports; i++) + FOREACH_PORT(i, ports) fdir_get_infos(i); else if (!strcmp(res->what, "stat_qmap")) - for (i = 0; i < nb_ports; i++) + FOREACH_PORT(i, ports) nic_stats_mapping_display(i); } @@ -8687,6 +8770,8 @@ cmdline_parse_ctx_t main_ctx[] = { (cmdline_parse_inst_t *)&cmd_set_qmap, (cmdline_parse_inst_t *)&cmd_operate_port, (cmdline_parse_inst_t *)&cmd_operate_specific_port, + (cmdline_parse_inst_t *)&cmd_operate_attach_port, + (cmdline_parse_inst_t *)&cmd_operate_detach_port, (cmdline_parse_inst_t *)&cmd_config_speed_all, (cmdline_parse_inst_t *)&cmd_config_speed_specific, (cmdline_parse_inst_t *)&cmd_config_rx_tx, @@ -8758,7 +8843,7 @@ prompt(void) static void cmd_reconfig_device_queue(portid_t id, uint8_t dev, uint8_t queue) { - if (id < nb_ports) { + if (!port_id_is_invalid(id, DISABLED_WARN)) { /* check if need_reconfig has been set to 1 */ if (ports[id].need_reconfig == 0) ports[id].need_reconfig = dev; @@ -8768,7 +8853,7 @@ cmd_reconfig_device_queue(portid_t id, uint8_t dev, uint8_t queue) } else { portid_t pid; - for (pid = 0; pid < nb_ports; pid++) { + FOREACH_PORT(pid, ports) { /* check if need_reconfig has been set to 1 */ if (ports[pid].need_reconfig == 0) ports[pid].need_reconfig = dev; @@ -8786,10 +8871,8 @@ bypass_is_supported(portid_t port_id) struct rte_port *port; struct rte_pci_id *pci_id; - if (port_id >= nb_ports) { - printf("\tPort id must be less than %d.\n", nb_ports); + if (port_id_is_invalid(port_id, ENABLED_WARN)) return 0; - } /* Get the device id. */ port = &ports[port_id]; diff --git a/app/test-pmd/config.c b/app/test-pmd/config.c index d436ce8..49be819 100644 --- a/app/test-pmd/config.c +++ b/app/test-pmd/config.c @@ -112,11 +112,15 @@ nic_stats_display(portid_t port_id) struct rte_eth_stats stats; struct rte_port *port = &ports[port_id]; uint8_t i; + portid_t pid; static const char *nic_stats_border = "########################"; - if (port_id >= nb_ports) { - printf("Invalid port, range is [0, %d]\n", nb_ports - 1); + if (port_id_is_invalid(port_id, ENABLED_WARN)) { + printf("Valid port range is [0"); + FOREACH_PORT(pid, ports) + printf(", %d", pid); + printf("]\n"); return; } rte_eth_stats_get(port_id, &stats); @@ -189,8 +193,13 @@ nic_stats_display(portid_t port_id) void nic_stats_clear(portid_t port_id) { - if (port_id >= nb_ports) { - printf("Invalid port, range is [0, %d]\n", nb_ports - 1); + portid_t pid; + + if (port_id_is_invalid(port_id, ENABLED_WARN)) { + printf("Valid port range is [0"); + FOREACH_PORT(pid, ports) + printf(", %d", pid); + printf("]\n"); return; } rte_eth_stats_reset(port_id); @@ -237,11 +246,15 @@ nic_stats_mapping_display(portid_t port_id) { struct rte_port *port = &ports[port_id]; uint16_t i; + portid_t pid; static const char *nic_stats_mapping_border = "########################"; - if (port_id >= nb_ports) { - printf("Invalid port, range is [0, %d]\n", nb_ports - 1); + if (port_id_is_invalid(port_id, ENABLED_WARN)) { + printf("Valid port range is [0"); + FOREACH_PORT(pid, ports) + printf(", %d", pid); + printf("]\n"); return; } @@ -290,9 +303,13 @@ port_infos_display(portid_t port_id) int vlan_offload; struct rte_mempool * mp; static const char *info_border = "*********************"; + portid_t pid; - if (port_id >= nb_ports) { - printf("Invalid port, range is [0, %d]\n", nb_ports - 1); + if (port_id_is_invalid(port_id, ENABLED_WARN)) { + printf("Valid port range is [0"); + FOREACH_PORT(pid, ports) + printf(", %d", pid); + printf("]\n"); return; } port = &ports[port_id]; @@ -365,11 +382,14 @@ port_infos_display(portid_t port_id) } int -port_id_is_invalid(portid_t port_id) +port_id_is_invalid(portid_t port_id, enum print_warning warning) { - if (port_id < nb_ports) + if (ports[port_id].enabled) return 0; - printf("Invalid port %d (must be < nb_ports=%d)\n", port_id, nb_ports); + + if (warning == ENABLED_WARN) + printf("Invalid port %d\n", port_id); + return 1; } @@ -428,7 +448,7 @@ port_reg_bit_display(portid_t port_id, uint32_t reg_off, uint8_t bit_x) uint32_t reg_v; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; if (port_reg_off_is_invalid(port_id, reg_off)) return; @@ -447,7 +467,7 @@ port_reg_bit_field_display(portid_t port_id, uint32_t reg_off, uint8_t l_bit; uint8_t h_bit; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; if (port_reg_off_is_invalid(port_id, reg_off)) return; @@ -474,7 +494,7 @@ port_reg_display(portid_t port_id, uint32_t reg_off) { uint32_t reg_v; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; if (port_reg_off_is_invalid(port_id, reg_off)) return; @@ -488,7 +508,7 @@ port_reg_bit_set(portid_t port_id, uint32_t reg_off, uint8_t bit_pos, { uint32_t reg_v; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; if (port_reg_off_is_invalid(port_id, reg_off)) return; @@ -516,7 +536,7 @@ port_reg_bit_field_set(portid_t port_id, uint32_t reg_off, uint8_t l_bit; uint8_t h_bit; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; if (port_reg_off_is_invalid(port_id, reg_off)) return; @@ -550,7 +570,7 @@ port_reg_bit_field_set(portid_t port_id, uint32_t reg_off, void port_reg_set(portid_t port_id, uint32_t reg_off, uint32_t reg_v) { - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; if (port_reg_off_is_invalid(port_id, reg_off)) return; @@ -563,7 +583,7 @@ port_mtu_set(portid_t port_id, uint16_t mtu) { int diag; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; diag = rte_eth_dev_set_mtu(port_id, mtu); if (diag == 0) @@ -726,7 +746,7 @@ rx_ring_desc_display(portid_t port_id, queueid_t rxq_id, uint16_t rxd_id) { const struct rte_memzone *rx_mz; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; if (rx_queue_id_is_invalid(rxq_id)) return; @@ -743,7 +763,7 @@ tx_ring_desc_display(portid_t port_id, queueid_t txq_id, uint16_t txd_id) { const struct rte_memzone *tx_mz; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; if (tx_queue_id_is_invalid(txq_id)) return; @@ -804,7 +824,7 @@ port_rss_reta_info(portid_t port_id, uint16_t i, idx, shift; int ret; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; ret = rte_eth_dev_rss_reta_query(port_id, reta_conf, nb_entries); @@ -859,7 +879,7 @@ port_rss_hash_conf_show(portid_t port_id, int show_rss_key) uint8_t i; int diag; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; /* Get RSS hash key if asked to display it */ rss_conf.rss_key = (show_rss_key) ? rss_key : NULL; @@ -1423,12 +1443,8 @@ set_fwd_ports_list(unsigned int *portlist, unsigned int nb_pt) again: for (i = 0; i < nb_pt; i++) { port_id = (portid_t) portlist[i]; - if (port_id >= nb_ports) { - printf("Invalid port id %u >= %u\n", - (unsigned int) port_id, - (unsigned int) nb_ports); + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; - } if (record_now) fwd_ports_ids[i] = port_id; } @@ -1586,7 +1602,7 @@ vlan_extend_set(portid_t port_id, int on) int diag; int vlan_offload; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; vlan_offload = rte_eth_dev_get_vlan_offload(port_id); @@ -1608,7 +1624,7 @@ rx_vlan_strip_set(portid_t port_id, int on) int diag; int vlan_offload; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; vlan_offload = rte_eth_dev_get_vlan_offload(port_id); @@ -1629,7 +1645,7 @@ rx_vlan_strip_set_on_queue(portid_t port_id, uint16_t queue_id, int on) { int diag; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; diag = rte_eth_dev_set_vlan_strip_on_queue(port_id, queue_id, on); @@ -1644,7 +1660,7 @@ rx_vlan_filter_set(portid_t port_id, int on) int diag; int vlan_offload; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; vlan_offload = rte_eth_dev_get_vlan_offload(port_id); @@ -1665,7 +1681,7 @@ rx_vft_set(portid_t port_id, uint16_t vlan_id, int on) { int diag; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; if (vlan_id_is_invalid(vlan_id)) return; @@ -1682,7 +1698,7 @@ rx_vlan_all_filter_set(portid_t port_id, int on) { uint16_t vlan_id; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; for (vlan_id = 0; vlan_id < 4096; vlan_id++) rx_vft_set(port_id, vlan_id, on); @@ -1692,7 +1708,7 @@ void vlan_tpid_set(portid_t port_id, uint16_t tp_id) { int diag; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; diag = rte_eth_dev_set_vlan_ether_type(port_id, tp_id); @@ -1707,7 +1723,7 @@ vlan_tpid_set(portid_t port_id, uint16_t tp_id) void tx_vlan_set(portid_t port_id, uint16_t vlan_id) { - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; if (vlan_id_is_invalid(vlan_id)) return; @@ -1718,7 +1734,7 @@ tx_vlan_set(portid_t port_id, uint16_t vlan_id) void tx_vlan_reset(portid_t port_id) { - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; ports[port_id].tx_ol_flags &= ~TESTPMD_TX_OFFLOAD_INSERT_VLAN; } @@ -1726,7 +1742,7 @@ tx_vlan_reset(portid_t port_id) void tx_vlan_pvid_set(portid_t port_id, uint16_t vlan_id, int on) { - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; rte_eth_dev_set_vlan_pvid(port_id, vlan_id, on); @@ -1738,7 +1754,7 @@ set_qmap(portid_t port_id, uint8_t is_rx, uint16_t queue_id, uint8_t map_value) uint16_t i; uint8_t existing_mapping_found = 0; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; if (is_rx ? (rx_queue_id_is_invalid(queue_id)) : (tx_queue_id_is_invalid(queue_id))) @@ -1904,7 +1920,7 @@ fdir_get_infos(portid_t port_id) static const char *fdir_stats_border = "########################"; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; ret = rte_eth_dev_filter_supported(port_id, RTE_ETH_FILTER_FDIR); if (ret < 0) { @@ -2030,7 +2046,7 @@ set_vf_traffic(portid_t port_id, uint8_t is_rx, uint16_t vf, uint8_t on) { int diag; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; if (is_rx) diag = rte_eth_dev_set_vf_rx(port_id,vf,on); @@ -2052,7 +2068,7 @@ set_vf_rx_vlan(portid_t port_id, uint16_t vlan_id, uint64_t vf_mask, uint8_t on) { int diag; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return; if (vlan_id_is_invalid(vlan_id)) return; @@ -2069,7 +2085,7 @@ set_queue_rate_limit(portid_t port_id, uint16_t queue_idx, uint16_t rate) int diag; struct rte_eth_link link; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return 1; rte_eth_link_get_nowait(port_id, &link); if (rate > link.link_speed) { @@ -2094,7 +2110,7 @@ set_vf_rate_limit(portid_t port_id, uint16_t vf, uint16_t rate, uint64_t q_msk) if (q_msk == 0) return 0; - if (port_id_is_invalid(port_id)) + if (port_id_is_invalid(port_id, ENABLED_WARN)) return 1; rte_eth_link_get_nowait(port_id, &link); if (rate > link.link_speed) { diff --git a/app/test-pmd/parameters.c b/app/test-pmd/parameters.c index 473f824..fa5f2a8 100644 --- a/app/test-pmd/parameters.c +++ b/app/test-pmd/parameters.c @@ -366,6 +366,7 @@ parse_portnuma_config(const char *q_arg) }; unsigned long int_fld[_NUM_FLD]; char *str_fld[_NUM_FLD]; + portid_t pid; /* reset from value set at definition */ while ((p = strchr(p0,'(')) != NULL) { @@ -387,8 +388,11 @@ parse_portnuma_config(const char *q_arg) return -1; } port_id = (uint8_t)int_fld[FLD_PORT]; - if (port_id >= nb_ports) { - printf("Invalid port, range is [0, %d]\n", nb_ports - 1); + if (port_id_is_invalid(port_id, ENABLED_WARN)) { + printf("Valid port range is [0"); + FOREACH_PORT(pid, ports) + printf(", %d", pid); + printf("]\n"); return -1; } socket_id = (uint8_t)int_fld[FLD_SOCKET]; @@ -419,6 +423,7 @@ parse_ringnuma_config(const char *q_arg) }; unsigned long int_fld[_NUM_FLD]; char *str_fld[_NUM_FLD]; + portid_t pid; #define RX_RING_ONLY 0x1 #define TX_RING_ONLY 0x2 #define RXTX_RING 0x3 @@ -443,8 +448,11 @@ parse_ringnuma_config(const char *q_arg) return -1; } port_id = (uint8_t)int_fld[FLD_PORT]; - if (port_id >= nb_ports) { - printf("Invalid port, range is [0, %d]\n", nb_ports - 1); + if (port_id_is_invalid(port_id, ENABLED_WARN)) { + printf("Valid port range is [0"); + FOREACH_PORT(pid, ports) + printf(", %d", pid); + printf("]\n"); return -1; } socket_id = (uint8_t)int_fld[FLD_SOCKET]; @@ -615,12 +623,12 @@ launch_args_parse(int argc, char** argv) #endif if (!strcmp(lgopts[opt_idx].name, "nb-ports")) { n = atoi(optarg); - if (n > 0 && n <= nb_ports) + if (n > 0 && + !port_id_is_invalid(n, DISABLED_WARN)) nb_fwd_ports = (uint8_t) n; else rte_exit(EXIT_FAILURE, - "nb-ports should be > 0 and <= %d\n", - nb_ports); + "Invalid port %d\n", n); } if (!strcmp(lgopts[opt_idx].name, "nb-cores")) { n = atoi(optarg); diff --git a/app/test-pmd/testpmd.c b/app/test-pmd/testpmd.c index 605163b..43329ed 100644 --- a/app/test-pmd/testpmd.c +++ b/app/test-pmd/testpmd.c @@ -71,6 +71,7 @@ #include <rte_pci.h> #include <rte_ether.h> #include <rte_ethdev.h> +#include <rte_dev.h> #include <rte_string_fns.h> #ifdef RTE_LIBRTE_PMD_XENVIRT #include <rte_eth_xenvirt.h> @@ -315,7 +316,7 @@ uint16_t nb_rx_queue_stats_mappings = 0; /* Forward function declarations */ static void map_port_queue_stats_mapping_registers(uint8_t pi, struct rte_port *port); -static void check_all_ports_link_status(uint8_t port_num, uint32_t port_mask); +static void check_all_ports_link_status(uint32_t port_mask); /* * Check if all the ports are started. @@ -324,6 +325,20 @@ static void check_all_ports_link_status(uint8_t port_num, uint32_t port_mask); static int all_ports_started(void); /* + * Find next enabled port + */ +portid_t +find_next_port(portid_t p, struct rte_port *ports, int size) +{ + if (ports == NULL) + rte_exit(-EINVAL, "failed to find a next port id\n"); + + while ((ports[p].enabled == 0) && (p < size)) + p++; + return p; +} + +/* * Setup default configuration. */ static void @@ -552,7 +567,8 @@ init_config(void) + RTE_TEST_TX_DESC_MAX + MAX_PKT_BURST; if (!numa_support) - nb_mbuf_per_pool = (nb_mbuf_per_pool * nb_ports); + nb_mbuf_per_pool = + (nb_mbuf_per_pool * RTE_MAX_ETHPORTS); } if (!numa_support) { @@ -565,14 +581,19 @@ init_config(void) /* Configuration of Ethernet ports. */ ports = rte_zmalloc("testpmd: ports", - sizeof(struct rte_port) * nb_ports, + sizeof(struct rte_port) * RTE_MAX_ETHPORTS, RTE_CACHE_LINE_SIZE); if (ports == NULL) { - rte_exit(EXIT_FAILURE, "rte_zmalloc(%d struct rte_port) " - "failed\n", nb_ports); + rte_exit(EXIT_FAILURE, + "rte_zmalloc(%d struct rte_port) failed\n", + RTE_MAX_ETHPORTS); } - for (pid = 0; pid < nb_ports; pid++) { + /* enabled allocated ports */ + for (pid = 0; pid < nb_ports; pid++) + ports[pid].enabled = 1; + + FOREACH_PORT(pid, ports) { port = &ports[pid]; rte_eth_dev_info_get(pid, &port->dev_info); @@ -602,8 +623,7 @@ init_config(void) nb_mbuf_per_pool = nb_mbuf_per_pool/nb_ports; for (i = 0; i < MAX_SOCKET; i++) { - nb_mbuf = (nb_mbuf_per_pool * - port_per_socket[i]); + nb_mbuf = (nb_mbuf_per_pool * RTE_MAX_ETHPORTS); if (nb_mbuf) mbuf_pool_create(mbuf_data_size, nb_mbuf,i); @@ -635,14 +655,6 @@ reconfig(portid_t new_port_id, unsigned socket_id) struct rte_port *port; /* Reconfiguration of Ethernet ports. */ - ports = rte_realloc(ports, - sizeof(struct rte_port) * nb_ports, - RTE_CACHE_LINE_SIZE); - if (ports == NULL) { - rte_exit(EXIT_FAILURE, "rte_realloc(%d struct rte_port) failed\n", - nb_ports); - } - port = &ports[new_port_id]; rte_eth_dev_info_get(new_port_id, &port->dev_info); @@ -663,7 +675,7 @@ init_fwd_streams(void) streamid_t sm_id, nb_fwd_streams_new; /* set socket id according to numa or not */ - for (pid = 0; pid < nb_ports; pid++) { + FOREACH_PORT(pid, ports) { port = &ports[pid]; if (nb_rxq > port->dev_info.max_rx_queues) { printf("Fail: nb_rxq(%d) is greater than " @@ -1264,7 +1276,7 @@ all_ports_started(void) portid_t pi; struct rte_port *port; - for (pi = 0; pi < nb_ports; pi++) { + FOREACH_PORT(pi, ports) { port = &ports[pi]; /* Check if there is a port which is not started */ if (port->port_status != RTE_PORT_STARTED) @@ -1276,6 +1288,45 @@ all_ports_started(void) } int +all_ports_stopped(void) +{ + portid_t pi; + struct rte_port *port; + + FOREACH_PORT(pi, ports) { + port = &ports[pi]; + if (port->port_status != RTE_PORT_STOPPED) + return 0; + } + + return 1; +} + +int +port_is_started(portid_t port_id) +{ + if (port_id_is_invalid(port_id, ENABLED_WARN)) + return 0; + + if (ports[port_id].port_status != RTE_PORT_STARTED) + return 0; + + return 1; +} + +static int +port_is_closed(portid_t port_id) +{ + if (port_id_is_invalid(port_id, ENABLED_WARN)) + return 0; + + if (ports[port_id].port_status != RTE_PORT_CLOSED) + return 0; + + return 1; +} + +int start_port(portid_t pid) { int diag, need_check_link_status = 0; @@ -1296,8 +1347,8 @@ start_port(portid_t pid) if(dcb_config) dcb_test = 1; - for (pi = 0; pi < nb_ports; pi++) { - if (pid < nb_ports && pid != pi) + FOREACH_PORT(pi, ports) { + if (pid != pi && pid != (portid_t)RTE_PORT_ALL) continue; port = &ports[pi]; @@ -1421,7 +1472,7 @@ start_port(portid_t pid) } if (need_check_link_status && !no_link_check) - check_all_ports_link_status(nb_ports, RTE_PORT_ALL); + check_all_ports_link_status(RTE_PORT_ALL); else printf("Please stop the ports first\n"); @@ -1446,8 +1497,8 @@ stop_port(portid_t pid) } printf("Stopping ports...\n"); - for (pi = 0; pi < nb_ports; pi++) { - if (pid < nb_ports && pid != pi) + FOREACH_PORT(pi, ports) { + if (!port_id_is_invalid(pid, DISABLED_WARN) && pid != pi) continue; port = &ports[pi]; @@ -1463,7 +1514,7 @@ stop_port(portid_t pid) need_check_link_status = 1; } if (need_check_link_status && !no_link_check) - check_all_ports_link_status(nb_ports, RTE_PORT_ALL); + check_all_ports_link_status(RTE_PORT_ALL); printf("Done\n"); } @@ -1481,8 +1532,8 @@ close_port(portid_t pid) printf("Closing ports...\n"); - for (pi = 0; pi < nb_ports; pi++) { - if (pid < nb_ports && pid != pi) + FOREACH_PORT(pi, ports) { + if (!port_id_is_invalid(pid, DISABLED_WARN) && pid != pi) continue; port = &ports[pi]; @@ -1502,31 +1553,83 @@ close_port(portid_t pid) printf("Done\n"); } -int -all_ports_stopped(void) +void +attach_port(char *identifier) { - portid_t pi; - struct rte_port *port; + portid_t i, j, pi = 0; - for (pi = 0; pi < nb_ports; pi++) { - port = &ports[pi]; - if (port->port_status != RTE_PORT_STOPPED) - return 0; + printf("Attaching a new port...\n"); + + if (identifier == NULL) { + printf("Invalid parameters are specified\n"); + return; } - return 1; + if (test_done == 0) { + printf("Please stop forwarding first\n"); + return; + } + + if (rte_eth_dev_attach(identifier, &pi)) + return; + + ports[pi].enabled = 1; + reconfig(pi, rte_eth_dev_socket_id(pi)); + rte_eth_promiscuous_enable(pi); + + nb_ports = rte_eth_dev_count(); + + /* set_default_fwd_ports_config(); */ + bzero(fwd_ports_ids, sizeof(fwd_ports_ids)); + i = 0; + FOREACH_PORT(j, ports) { + fwd_ports_ids[i] = j; + i++; + } + nb_cfg_ports = nb_ports; + nb_fwd_ports++; + + ports[pi].port_status = RTE_PORT_STOPPED; + + printf("Port %d is attached. Now total ports is %d\n", pi, nb_ports); + printf("Done\n"); } -int -port_is_started(portid_t port_id) +void +detach_port(uint8_t port_id) { - if (port_id_is_invalid(port_id)) - return -1; + portid_t i, pi = 0; + char name[RTE_ETH_NAME_MAX_LEN]; - if (ports[port_id].port_status != RTE_PORT_STARTED) - return 0; + printf("Detaching a port...\n"); - return 1; + if (!port_is_closed(port_id)) { + printf("Please close port first\n"); + return; + } + + rte_eth_promiscuous_disable(port_id); + + if (rte_eth_dev_detach(port_id, name)) + return; + + ports[port_id].enabled = 0; + nb_ports = rte_eth_dev_count(); + + /* set_default_fwd_ports_config(); */ + bzero(fwd_ports_ids, sizeof(fwd_ports_ids)); + i = 0; + FOREACH_PORT(pi, ports) { + fwd_ports_ids[i] = pi; + i++; + } + nb_cfg_ports = nb_ports; + nb_fwd_ports--; + + printf("Port '%s' is detached. Now total ports is %d\n", + name, nb_ports); + printf("Done\n"); + return; } void @@ -1534,7 +1637,7 @@ pmd_test_exit(void) { portid_t pt_id; - for (pt_id = 0; pt_id < nb_ports; pt_id++) { + FOREACH_PORT(pt_id, ports) { printf("Stopping port %d...", pt_id); fflush(stdout); rte_eth_dev_close(pt_id); @@ -1553,7 +1656,7 @@ struct pmd_test_command { /* Check the link status of all ports in up to 9s, and print them finally */ static void -check_all_ports_link_status(uint8_t port_num, uint32_t port_mask) +check_all_ports_link_status(uint32_t port_mask) { #define CHECK_INTERVAL 100 /* 100ms */ #define MAX_CHECK_TIME 90 /* 9s (90 * 100ms) in total */ @@ -1564,7 +1667,7 @@ check_all_ports_link_status(uint8_t port_num, uint32_t port_mask) fflush(stdout); for (count = 0; count <= MAX_CHECK_TIME; count++) { all_ports_up = 1; - for (portid = 0; portid < port_num; portid++) { + FOREACH_PORT(portid, ports) { if ((port_mask & (1 << portid)) == 0) continue; memset(&link, 0, sizeof(link)); @@ -1729,7 +1832,7 @@ init_port_config(void) portid_t pid; struct rte_port *port; - for (pid = 0; pid < nb_ports; pid++) { + FOREACH_PORT(pid, ports) { port = &ports[pid]; port->dev_conf.rxmode = rx_mode; port->dev_conf.fdir_conf = fdir_conf; @@ -1908,7 +2011,7 @@ main(int argc, char** argv) nb_ports = (portid_t) rte_eth_dev_count(); if (nb_ports == 0) - rte_exit(EXIT_FAILURE, "No probed ethernet device\n"); + RTE_LOG(WARNING, EAL, "No probed ethernet devices\n"); set_def_fwd_config(); if (nb_lcores == 0) @@ -1930,7 +2033,7 @@ main(int argc, char** argv) rte_exit(EXIT_FAILURE, "Start ports failed\n"); /* set all ports to promiscuous mode by default */ - for (port_id = 0; port_id < nb_ports; port_id++) + FOREACH_PORT(port_id, ports) rte_eth_promiscuous_enable(port_id); #ifdef RTE_LIBRTE_CMDLINE diff --git a/app/test-pmd/testpmd.h b/app/test-pmd/testpmd.h index 126bef7..0d5a526 100644 --- a/app/test-pmd/testpmd.h +++ b/app/test-pmd/testpmd.h @@ -137,6 +137,7 @@ struct fwd_stream { * The data structure associated with each port. */ struct rte_port { + uint8_t enabled; /**< Port enabled or not */ struct rte_eth_dev_info dev_info; /**< PCI info + driver name */ struct rte_eth_conf dev_conf; /**< Port configuration. */ struct ether_addr eth_addr; /**< Port ethernet address */ @@ -162,6 +163,14 @@ struct rte_port { struct rte_eth_txconf tx_conf; /**< tx configuration */ }; +extern portid_t __rte_unused +find_next_port(portid_t p, struct rte_port *ports, int size); + +#define FOREACH_PORT(p, ports) \ + for (p = find_next_port(0, ports, RTE_MAX_ETHPORTS); \ + p < RTE_MAX_ETHPORTS; \ + p = find_next_port(p + 1, ports, RTE_MAX_ETHPORTS)) + /** * The data structure associated with each forwarding logical core. * The logical cores are internally numbered by a core index from 0 to @@ -522,6 +531,8 @@ int init_port_dcb_config(portid_t pid,struct dcb_config *dcb_conf); int start_port(portid_t pid); void stop_port(portid_t pid); void close_port(portid_t pid); +void attach_port(char *identifier); +void detach_port(uint8_t port_id); int all_ports_stopped(void); int port_is_started(portid_t port_id); void pmd_test_exit(void); @@ -548,10 +559,15 @@ void get_syn_filter(uint8_t port_id); void get_ethertype_filter(uint8_t port_id, uint16_t index); void get_2tuple_filter(uint8_t port_id, uint16_t index); void get_5tuple_filter(uint8_t port_id, uint16_t index); -int port_id_is_invalid(portid_t port_id); int rx_queue_id_is_invalid(queueid_t rxq_id); int tx_queue_id_is_invalid(queueid_t txq_id); +enum print_warning { + ENABLED_WARN = 0, + DISABLED_WARN +}; +int port_id_is_invalid(portid_t port_id, enum print_warning warning); + /* * Work-around of a compilation error with ICC on invocations of the * rte_be_to_cpu_16() function. diff --git a/doc/guides/testpmd_app_ug/testpmd_funcs.rst b/doc/guides/testpmd_app_ug/testpmd_funcs.rst index afe1970..a99e14d 100644 --- a/doc/guides/testpmd_app_ug/testpmd_funcs.rst +++ b/doc/guides/testpmd_app_ug/testpmd_funcs.rst @@ -807,6 +807,63 @@ The following sections show functions for configuring ports. Port configuration changes only become active when forwarding is started/restarted. +port attach +~~~~~~~~~~~ + +Attach a port specified by pci address or virtual device args. + +To attach a new pci device, the device should be recognized by kernel first. +Then it should be moved under DPDK management. +Finally the port can be attached to testpmd. +On the other hand, to attach a port created by virtual device, above steps are not needed. + +port attach (identifier) + +For example, to attach a port whose pci address is 0000:02:00.0. + +.. code-block:: console + + testpmd> port attach 0000:02:00.0 + Attaching a new port... + ... snip ... + Port 0 is attached. Now total ports is 1 + Done + +For example, to attach a port created by pcap PMD. + +.. code-block:: console + + testpmd> port attach eth_pcap0,iface=eth0 + Attaching a new port... + ... snip ... + Port 0 is attached. Now total ports is 1 + Done + +In this case, identifier is "eth_pcap0,iface=eth0". +This identifier format is the same as "--vdev" format of DPDK applications. + +port detach +~~~~~~~~~~~ + +Detach a specific port. + +Before detaching a port, the port should be closed. +Also to remove a pci device completely from the system, first detach the port from testpmd. +Then the device should be moved under kernel management. +Finally the device can be removed using kernel pci hotplug functionality. +On the other hand, to remove a port created by a virtual device, above steps are not needed. + +port detach (port_id) + +For example, to detach a port 0. + +.. code-block:: console + + testpmd> port detach 0 + Detaching a port... + ... snip ... + Done + port start ~~~~~~~~~~ -- 1.9.1
2015-02-26 04:32, Tetsuya Mukawa:
> This patch series adds a dynamic port hotplug framework to DPDK.
> With the patches, DPDK apps can attach or detach ports at runtime.
>
> The basic concept of the port hotplug is like followings.
> - DPDK apps must have responsibility to manage ports.
> DPDK apps only know which ports are attached or detached at the moment.
> The port hotplug framework is implemented to allow DPDK apps to manage ports.
> For example, when DPDK apps call port attach function, attached port number
> will be returned. Also, DPDK apps can detach port by port number.
> - Kernel support is needed for attaching or detaching physical device ports.
> To attach a new physical device port, the device will be recognized by
> userspace directly I/O framework in kernel at first. Then DPDK apps can
> call the port hotplug functions to attach ports.
> For detaching, steps are vice versa.
> - Before detach ports, ports must be stopped and closed.
> DPDK application must call rte_eth_dev_stop() and rte_eth_dev_close() before
> detaching ports. These function will call finalization codes of PMDs.
> But so far, no PMD frees all resources allocated by initialization.
> It means PMDs are needed to be fixed to support the port hotplug.
> 'RTE_PCI_DRV_DETACHABLE' is a new flag indicating a PMD supports detaching.
> Without this flag, detaching will be failed.
> - Mustn't affect legacy DPDK apps.
> No DPDK EAL behavior is changed, if the port hotplug functions are't called.
> So all legacy DPDK apps can still work without modifications.
>
> And a few limitations.
> - The port hotplug functions are not thread safe.
> DPDK apps should handle it.
> - Only support Linux and igb_uio so far.
> BSD and VFIO is not supported. I will send VFIO patches at least, but I don't
> have a plan to submit BSD patch so far.
>
>
> Here is port hotplug APIs.
> -------------------------------------------------------------------------------
> /**
> * Attach a new device.
> *
> * @param devargs
> * A pointer to a strings array describing the new device
> * to be attached. The strings should be a pci address like
> * '0000:01:00.0' or virtual device name like 'eth_pcap0'.
> * @param port_id
> * A pointer to a port identifier actually attached.
> * @return
> * 0 on success and port_id is filled, negative on error
> */
> int rte_eth_dev_attach(const char *devargs, uint8_t *port_id);
>
> /**
> * Detach a device.
> *
> * @param port_id
> * The port identifier of the device to detach.
> * @param addr
> * A pointer to a device name actually detached.
> * @return
> * 0 on success and devname is filled, negative on error
> */
> int rte_eth_dev_detach(uint8_t port_id, char *devname);
> -------------------------------------------------------------------------------
>
> This patch series are for DPDK EAL. To use port hotplug function by DPDK apps,
> each PMD should be fixed to support 'RTE_PCI_DRV_DETACHABLE' flag. Please check
> a patch for pcap PMD.
>
> Also, please check testpmd patch. It will show you how to fix your legacy
> applications to support port hotplug feature.
>
> PATCH v15 changes
> - Fix issue that eal calls ethedv library APIs.
> - Remove rte_eal_dev_attach(), and add rte_eth_dev_attach().
> - Remove rte_eal_dev_detach(), and add rte_eth_dev_detach().
> - Call rte_eal_vdev_init/uninit from ethdev library.
> (Thanks to Thomas Monjalon)
> - Fix version.map
> - Reorder and squash patches to compile correctly.
> - Remove needless symbols in version.map
> - Fix version.map.
>
> PATCH v14 changes
> - Remove needless if statement.
> (Thanks to Maxime Leroy)
>
> PATCH v13 changes
> - Change log level when error occurs in rte_eal_vdev_init() and
> rte_eal_dev_init().
> - Return value of driver init and uninit functions.
> - Replace rte_panic by RTE_LOG in rte_eal_dev_init()
> - Fix return value of rte_eal_vdev_uninit().
> - Fix rte_eal_dev_attach_vdev to set port_id correctly.
> (Thanks to Maxime Leroy)
>
> PATCH v12 changes
> - Add missing symbol in version map.
> (Thanks to Iremonger, Bernard)
>
> PATCH v11 changes
> - Remove needless devargs handling codes.
> - Replace get_vdev_name() by rte_eal_parse_devargs_str().
> - Replace rte_eal_vdev_find_and_init by rte_eal_vdev_init()
> - Replace rte_eal_vdev_find_and_uninit by rte_eal_vdev_uninit()
> - Fix rte_eal_dev_init() to use rte_eal_vdev_init().
> - Remove needless patch.
> (Thanks to Maxime Leroy)
>
> PATCH v10 changes
> - Add comments.
> - Chagne order of version.map.
> - Fix comment of "rte_ethdev.h".
> (Thanks to Thomas Monjalon)
> - Add size parameter to rte_eth_dev_create_unique_device_name().
> (Thanks to Iremonger, Bernard)
>
> PATCH v9 changes
> - Fix commit title.
> - Fix commit log.
> - Fix comments.
> - Define CONFIG_RTE_LIBRTE_EAL_HOTPLUG at the top of this patch series.
> - DEV_INVALID/VALID are removed.
> - DEV_DISCONNECTED is replaced by DEV_DETACHED.
> - DEV_CONNECTED is replaced by DEV_ATTACHED.
> - rte_eth_dev_allocate_new_port() is renamed to
> rte_eth_dev_find_free_port().
> - rte_eth_dev_validate_port() is renamed to rte_eth_dev_is_valid_port().
> - rte_eth_dev_is_valid_port() is changed not to handle log toggle.
> - eal_compare_pci_addr() is replaced by rte_eal_compare_pci_addr().
> - rte_eth_dev_free() is replaced by rte_eth_dev_release_port().
> - Add a function to create a unique device name.
> - Change parameter of pci_devuninit_t and rte_eth_dev_uninit.
> - Remove code that initiaize callback of ethdev from
> rte_eth_dev_uninit().
> - Remove pci_unmap_device(). It will be implemented in later patch.
> - rte_eth_dev_check_detachable() is replaced by
> rte_eth_dev_is_detachable().
> - strncpy() is replaced by strcpy().
> - Implement pci_unmap_device() in this patch.
> - Remove "rte_dev_hotplug.h".
> - Remove needless "#ifdef".
> - Remove RTE_EAL_INVOKE_TYPE_PROBE/CLOSE.
> - RTE_ETH_DEV_PHYSICAL is replaced by RTE_ETH_DEV_PCI.
> - Use strcmp() instead of strncmp().
> - Remove RTE_EAL_INVOKE_TYPE_PROBE/CLOSE.
> (Thanks to Thomas Monjalon)
> - Change definition of rte_dev_uninit_t.
> (Thanks to Thomas Monjalon and Maxime Leroy)
> - Add missing symbol in version map.
> (Thanks to Nail Horman)
>
> PATCH v8 changes
> - Fix Makefile and add version map file.
> - Add missing symbol in version map.
> - Fix pci_scan_one() to update sysfs values.
> (Thanks to Qiu, Michael and Iremonger, Bernard)
> - NONE_TRACE is replaced by NO_TRACE.
> - Fix typo.
> - Add size parameter to rte_eth_dev_save().
> (Thanks to Iremonger, Bernard)
>
> PATCH v7 changes
> - Add a new section to programmer's guide.
> (Thanks to Iremonger, Bernard)
> - Fix port checking implementation of star_port().
> - Fix typo of warning messages.
> - Add pt_driver checking to rte_eth_dev_check_detachable().
> (Thanks to Qiu, Michael)
>
> PATCH v6 changes
> - Fix rte_eth_dev_uninit() to handle a return value of uninit
> function of PMD. To do this, below changes also be applied.
> - Fix a parameter of rte_eth_dev_free().
> - Use rte_eth_dev structure as the paramter of rte_eth_dev_free().
>
> PATCH v5 changes
> - Add runtime check passthrough driver type, like vfio-pci, igb_uio
> and uio_pci_generic.
> This was done by Qiu, Michael. Thanks a lot.
> - Change function names like below.
> - rte_eal_dev_find_and_invoke() to rte_eal_vdev_find_and_invoke().
> - rte_eal_dev_invoke() to rte_eal_vdev_invoke().
> - Add code to handle a return value of rte_eal_devargs_remove().
> - Fix pci address format in rte_eal_dev_detach().
> - Remove RTE_EAL_INVOKE_TYPE_UNKNOWN, because it's unused.
> - Change function definition of rte_eal_devargs_remove().
> - Fix pci_unmap_device() to check pt_driver.
> - Fix return value of below functions.
> - rte_eth_dev_get_changed_port().
> - rte_eth_dev_get_port_by_addr().
> - Change paramters of rte_eth_dev_validate_port() to cleanup code.
> - Fix pci_scan_one to handle pt_driver correctly.
> (Thanks to Qiu, Michael for above suggestions)
>
> PATCH v4 changes
> - Merge patches to review easier.
> - Fix indent of 'if' statement.
> - Fix calculation method of eal_compare_pci_addr().
> - Fix header file declaration.
> - Add header file to determine if hotplug can be enabled.
> (Thanks to Qiu, Michael)
> - Use braces with 'for' loop.
> - Add parameter checking.
> - Fix sanity check code
> - Fix comments of rte_eth_dev_type.
> - Change function names.
> (Thanks to Iremonger, Bernard)
>
> PATCH v3 changes:
> - Fix enum definition used in rte_ethdev.c.
> (Thanks to Zhang, Helin)
>
> PATCH v2 changes:
> - Replace rte_eal_dev_attach_pdev(), rte_eal_dev_detach_pdev,
> rte_eal_dev_attach_vdev() and rte_eal_dev_detach_vdev() to
> rte_eal_dev_attach() and rte_eal_dev_detach().
> - Add parameter values checking.
> - Refashion a few functions.
> (Thanks to Iremonger, Bernard)
>
> PATCH v1 Changes:
> - Fix error checking code of librte_eth APIs.
> - Fix issue that port from pcap PMD cannot be detached correctly.
> - Fix issue that testpmd could hang after forwarding, if attaching and detaching
> is repeatedly.
> - Fix if-condition of rte_eth_dev_get_port_by_addr().
> (Thanks to Mark Enright)
>
> RFC PATCH v2 Changes:
> - remove 'rte_eth_dev_validate_port()', and cleanup codes.
>
>
> Michael Qiu (2):
> eal_pci: Add flag to hold kernel driver type
> eal_pci: pci memory map work with driver type
>
> Tetsuya Mukawa (11):
> eal: Enable port Hotplug framework in Linux
> eal/pci,ethdev: Remove assumption that port will not be detached
> eal/pci: Consolidate pci address comparison APIs
> ethdev: Add rte_eth_dev_release_port to release specified port
> eal,ethdev: Add a function and function pointers to close ether device
> eal/linux/pci: Add functions for unmapping igb_uio resources
> eal/pci: Add probe and close functions of pci driver
> ethdev: Add one dev_type parameter to rte_eth_dev_allocate
> eal/pci: Add vdev driver initialization and uninitialization functions
> ethdev: Add rte_eth_dev_attach/detach() functions
> doc: Add port hotplug framework section to programmers guide
Applied, thanks Tetsuya for this long work
> This patch adds finalization code to free resources allocated by the
> PMD.
>
> v6:
> - Fix a paramter of rte_eth_dev_free().
> v4:
> - Change function name.
>
> Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp>
Applied, thanks
> The patch introduces following commands.
> - port attach [ident]
> - port detach [port_id]
> - attach: attaching a port
> - detach: detaching a port
> - ident: pci address of physical device.
> Or device name and parameters of virtual device.
> (ex. 0000:02:00.0, eth_pcap0,iface=eth0)
> - port_id: port identifier
>
> v15:
> - Replace rte_eal_dev_attach() by rte_eth_dev_attach()
> - Replace rte_eal_dev_detach() by rte_eth_dev_detach()
>
> v7:
> - Fix doc.
> (Thanks to Iremonger, Bernard)
> - Fix port checking implementation of star_port();
> (Thanks to Qiu, Michael)
> v5:
> - Add testpmd documentation.
> (Thanks to Iremonger, Bernard)
> v4:
> - Fix strings of command help.
>
> Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp>
Applied, thanks
> -----Original Message----- > From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Tetsuya Mukawa > Sent: Wednesday, February 25, 2015 7:32 PM > To: dev@dpdk.org > Subject: [dpdk-dev] [PATCH v15] testpmd: Add port hotplug support > > The patch introduces following commands. > - port attach [ident] > - port detach [port_id] > - attach: attaching a port > - detach: detaching a port > - ident: pci address of physical device. > Or device name and parameters of virtual device. > (ex. 0000:02:00.0, eth_pcap0,iface=eth0) > - port_id: port identifier > > v15: > - Replace rte_eal_dev_attach() by rte_eth_dev_attach() > - Replace rte_eal_dev_detach() by rte_eth_dev_detach() > > v7: > - Fix doc. > (Thanks to Iremonger, Bernard) > - Fix port checking implementation of star_port(); > (Thanks to Qiu, Michael) > v5: > - Add testpmd documentation. > (Thanks to Iremonger, Bernard) > v4: > - Fix strings of command help. > > Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp> > --- > app/test-pmd/cmdline.c | 137 +++++++++++++++---- > app/test-pmd/config.c | 102 ++++++++------ > app/test-pmd/parameters.c | 22 ++- > app/test-pmd/testpmd.c | 199 +++++++++++++++++++++------- > app/test-pmd/testpmd.h | 18 ++- > doc/guides/testpmd_app_ug/testpmd_funcs.rst | 57 ++++++++ > 6 files changed, 409 insertions(+), 126 deletions(-) > [...] > @@ -1423,12 +1443,8 @@ set_fwd_ports_list(unsigned int *portlist, > unsigned int nb_pt) > again: > for (i = 0; i < nb_pt; i++) { > port_id = (portid_t) portlist[i]; > - if (port_id >= nb_ports) { > - printf("Invalid port id %u >= %u\n", > - (unsigned int) port_id, > - (unsigned int) nb_ports); > + if (port_id_is_invalid(port_id, ENABLED_WARN)) Sorry for catching this late, but there is a segmentation fault when this function gets called when parsing "portmask" at testpmd initialization, since port_id_is_invalid function needs to have array "ports" initialized. I would revert the change, but not sure if this is need for other reasons. Thanks, Pablo > return; > - } > if (record_now) > fwd_ports_ids[i] = port_id; > }
On 2015/02/27 3:49, De Lara Guarch, Pablo wrote: > >> -----Original Message----- >> From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Tetsuya Mukawa >> Sent: Wednesday, February 25, 2015 7:32 PM >> To: dev@dpdk.org >> Subject: [dpdk-dev] [PATCH v15] testpmd: Add port hotplug support >> >> The patch introduces following commands. >> - port attach [ident] >> - port detach [port_id] >> - attach: attaching a port >> - detach: detaching a port >> - ident: pci address of physical device. >> Or device name and parameters of virtual device. >> (ex. 0000:02:00.0, eth_pcap0,iface=eth0) >> - port_id: port identifier >> >> v15: >> - Replace rte_eal_dev_attach() by rte_eth_dev_attach() >> - Replace rte_eal_dev_detach() by rte_eth_dev_detach() >> >> v7: >> - Fix doc. >> (Thanks to Iremonger, Bernard) >> - Fix port checking implementation of star_port(); >> (Thanks to Qiu, Michael) >> v5: >> - Add testpmd documentation. >> (Thanks to Iremonger, Bernard) >> v4: >> - Fix strings of command help. >> >> Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp> >> --- >> app/test-pmd/cmdline.c | 137 +++++++++++++++---- >> app/test-pmd/config.c | 102 ++++++++------ >> app/test-pmd/parameters.c | 22 ++- >> app/test-pmd/testpmd.c | 199 +++++++++++++++++++++------- >> app/test-pmd/testpmd.h | 18 ++- >> doc/guides/testpmd_app_ug/testpmd_funcs.rst | 57 ++++++++ >> 6 files changed, 409 insertions(+), 126 deletions(-) >> > [...] > >> @@ -1423,12 +1443,8 @@ set_fwd_ports_list(unsigned int *portlist, >> unsigned int nb_pt) >> again: >> for (i = 0; i < nb_pt; i++) { >> port_id = (portid_t) portlist[i]; >> - if (port_id >= nb_ports) { >> - printf("Invalid port id %u >= %u\n", >> - (unsigned int) port_id, >> - (unsigned int) nb_ports); >> + if (port_id_is_invalid(port_id, ENABLED_WARN)) > Sorry for catching this late, but there is a segmentation fault when this function gets called > when parsing "portmask" at testpmd initialization, since port_id_is_invalid function needs to have array "ports" initialized. > I would revert the change, but not sure if this is need for other reasons. Hi Pablo, I appreciate for your reporting. I could reproduce the issue with portmask option like below $ sudo ./testpmd -c f -n 1 -- --portmask 0x1 -i As a result of investigation, when 'launch_args_parse()' is called, port structure hasn't been allocated yet. As you said, this causes the issue. I guess we can have 2 options to fix the issue. Option1: Fix 'set_fwd_ports_list()' to work even when port structure hasn't been initialized yet. I guess this implementation is much complex for readers of test-pmd code. Option2: Move initialization code of ports like below. int main () { init_port(); /* only initialize port structure here*/ launch_args_parse(); init_config(); /* So far, port initialization code is implemented in init_config() */ } I guess 2nd option may be better. How do you think? I will send a patch to implement 2nd option. if you are ok to fix like above, could you please check and acked it? Thanks, Tetsuya > Thanks, > Pablo >> return; >> - } >> if (record_now) >> fwd_ports_ids[i] = port_id; >> }
2015-02-27 15:14, Tetsuya Mukawa: > On 2015/02/27 3:49, De Lara Guarch, Pablo wrote: > > > >> -----Original Message----- > >> From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Tetsuya Mukawa > >> Sent: Wednesday, February 25, 2015 7:32 PM > >> To: dev@dpdk.org > >> Subject: [dpdk-dev] [PATCH v15] testpmd: Add port hotplug support > >> > >> The patch introduces following commands. > >> - port attach [ident] > >> - port detach [port_id] > >> - attach: attaching a port > >> - detach: detaching a port > >> - ident: pci address of physical device. > >> Or device name and parameters of virtual device. > >> (ex. 0000:02:00.0, eth_pcap0,iface=eth0) > >> - port_id: port identifier > >> > >> v15: > >> - Replace rte_eal_dev_attach() by rte_eth_dev_attach() > >> - Replace rte_eal_dev_detach() by rte_eth_dev_detach() > >> > >> v7: > >> - Fix doc. > >> (Thanks to Iremonger, Bernard) > >> - Fix port checking implementation of star_port(); > >> (Thanks to Qiu, Michael) > >> v5: > >> - Add testpmd documentation. > >> (Thanks to Iremonger, Bernard) > >> v4: > >> - Fix strings of command help. > >> > >> Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp> > >> --- > >> app/test-pmd/cmdline.c | 137 +++++++++++++++---- > >> app/test-pmd/config.c | 102 ++++++++------ > >> app/test-pmd/parameters.c | 22 ++- > >> app/test-pmd/testpmd.c | 199 +++++++++++++++++++++------- > >> app/test-pmd/testpmd.h | 18 ++- > >> doc/guides/testpmd_app_ug/testpmd_funcs.rst | 57 ++++++++ > >> 6 files changed, 409 insertions(+), 126 deletions(-) > >> > > [...] > > > >> @@ -1423,12 +1443,8 @@ set_fwd_ports_list(unsigned int *portlist, > >> unsigned int nb_pt) > >> again: > >> for (i = 0; i < nb_pt; i++) { > >> port_id = (portid_t) portlist[i]; > >> - if (port_id >= nb_ports) { > >> - printf("Invalid port id %u >= %u\n", > >> - (unsigned int) port_id, > >> - (unsigned int) nb_ports); > >> + if (port_id_is_invalid(port_id, ENABLED_WARN)) > > Sorry for catching this late, but there is a segmentation fault when this function gets called > > when parsing "portmask" at testpmd initialization, since port_id_is_invalid function needs to have array "ports" initialized. > > I would revert the change, but not sure if this is need for other reasons. > > Hi Pablo, > > I appreciate for your reporting. > I could reproduce the issue with portmask option like below > $ sudo ./testpmd -c f -n 1 -- --portmask 0x1 -i > > As a result of investigation, when 'launch_args_parse()' is called, port > structure hasn't been allocated yet. > As you said, this causes the issue. > > I guess we can have 2 options to fix the issue. > > Option1: > Fix 'set_fwd_ports_list()' to work even when port structure hasn't been > initialized yet. > I guess this implementation is much complex for readers of test-pmd code. > > Option2: > Move initialization code of ports like below. > > int main () { > > init_port(); /* only initialize port structure here*/ > launch_args_parse(); > init_config(); /* So far, port initialization code is > implemented in init_config() */ > > } > > I guess 2nd option may be better. > How do you think? > I will send a patch to implement 2nd option. > if you are ok to fix like above, could you please check and acked it? For the record, Tetsuya chose the second option: http://dpdk.org/browse/dpdk/commit/?id=ffc468ff3cfe768
> -----Original Message-----
> From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Tetsuya Mukawa
> Sent: Wednesday, February 25, 2015 7:32 PM
> To: dev@dpdk.org
> Subject: [dpdk-dev] [PATCH v15 13/13] doc: Add port hotplug framework
> section to programmers guide
>
> This patch adds a new section for describing port hotplug framework.
>
> v15:
> - Fix function names like below.
> - rte_eal_dev_attach() to rte_eth_dev_attach().
> - rte_eal_dev_detach() to rte_eth_dev_detach().
>
> Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp>
> ---
> doc/guides/prog_guide/index.rst | 1 +
> doc/guides/prog_guide/port_hotplug_framework.rst | 110
> +++++++++++++++++++++++
> 2 files changed, 111 insertions(+)
> create mode 100644 doc/guides/prog_guide/port_hotplug_framework.rst
>
> diff --git a/doc/guides/prog_guide/index.rst
> b/doc/guides/prog_guide/index.rst index de69682..60a6ac5 100644
> --- a/doc/guides/prog_guide/index.rst
> +++ b/doc/guides/prog_guide/index.rst
> @@ -71,6 +71,7 @@ Programmer's Guide
> packet_classif_access_ctrl
> packet_framework
> vhost_lib
> + port_hotplug_framework
> source_org
> dev_kit_build_system
> dev_kit_root_make_help
> diff --git a/doc/guides/prog_guide/port_hotplug_framework.rst
> b/doc/guides/prog_guide/port_hotplug_framework.rst
> new file mode 100644
> index 0000000..fe6d72a
> --- /dev/null
> +++ b/doc/guides/prog_guide/port_hotplug_framework.rst
> @@ -0,0 +1,110 @@
> +.. BSD LICENSE
> + Copyright(c) 2015 IGEL Co.,Ltd. All rights reserved.
> + All rights reserved.
> +
> + Redistribution and use in source and binary forms, with or without
> + modification, are permitted provided that the following conditions
> + are met:
> +
> + * Redistributions of source code must retain the above copyright
> + notice, this list of conditions and the following disclaimer.
> + * Redistributions in binary form must reproduce the above copyright
> + notice, this list of conditions and the following disclaimer in
> + the documentation and/or other materials provided with the
> + distribution.
> + * Neither the name of IGEL Co.,Ltd. nor the names of its
> + contributors may be used to endorse or promote products derived
> + from this software without specific prior written permission.
> +
> + THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND
> CONTRIBUTORS
> + "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT
> NOT
> + LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND
> FITNESS FOR
> + A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE
> COPYRIGHT
> + OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT,
> INCIDENTAL,
> + SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
> NOT
> + LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS
> OF USE,
> + DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED
> AND ON ANY
> + THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
> + (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF
> THE USE
> + OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH
> DAMAGE.
> +
> +Port Hotplug Framework
> +======================
> +
> +The Port Hotplug Framework provides DPDK applications with the ability
> +to attach and detach ports at runtime. Because the framework depends on
> +PMD implementation, the ports that PMDs cannot handle are out of scope
> +of this framework. Furthermore, after detaching a port from a DPDK
> +application, the framework doesn't provide a way for removing the devices
> from the system.
> +For the ports backed by a physical NIC, the kernel will need to support
> +PCI Hotplug feature.
> +
> +Overview
> +--------
> +
> +The basic requirements of the Port Hotplug Framework are:
> +
> +* DPDK applications that use the Port Hotplug Framework must manage
> their
> + own ports.
> +
> + The Port Hotplug Framework is implemented to allow DPDK applications
> to
> + manage ports. For example, when DPDK applications call the port attach
> + function, the attached port number is returned. DPDK applications can
> + also detach the port by port number.
> +
> +* Kernel support is needed for attaching or detaching physical device
> + ports.
> +
> + To attach new physical device ports, the device will be recognized by
> + userspace driver I/O framework in kernel at first. Then DPDK
> + applications can call the Port Hotplug functions to attach the ports.
> + For detaching, steps are vice versa.
> +
> +* Before detaching, they must be stopped and closed.
> +
> + DPDK applications must call "rte_eth_dev_stop()" and
> + "rte_eth_dev_close()" APIs before detaching ports. These functions
> will
> + start finalization sequence of the PMDs.
> +
> +* The framework doesn't affect legacy DPDK applications behavior.
> +
> + If the Port Hotplug functions aren't called, all legacy DPDK apps can
> + still work without modifications.
> +
> +Port Hotplug API overview
> +-------------------------
> +
> +* Attaching a port
> +
> + "rte_eth_dev_attach()" API attaches a port to DPDK application, and
> + returns the attached port number. Before calling the API, the device
> + should be recognized by an userspace driver I/O framework. The API
> + receives a pci address like "0000:01:00.0" or a virtual device name
> + like "eth_pcap0,iface=eth0". In the case of virtual device name, the
> + format is the same as the general "--vdev" option of DPDK.
> +
> +* Detaching a port
> +
> + "rte_eth_dev_detach()" API detaches a port from DPDK application,
> and
> + returns a pci address of the detached device or a virtual device name
> + of the device.
> +
> +Reference
> +---------
> +
> + "testpmd" supports the Port Hotplug Framework.
> +
> +Limitations
> +-----------
> +
> +* The Port Hotplug APIs are not thread safe.
> +
> +* The framework can only be enabled with Linux. BSD is not supported.
> +
> +* To detach a port, the port should be backed by a device that igb_uio
> + manages. VFIO is not supported.
> +
> +* Not all PMDs support detaching feature.
> + To know whether a PMD can support detaching, search for the
> + "RTE_PCI_DRV_DETACHABLE" flag in PMD implementation. If the flag is
> + defined in the PMD, detaching is supported.
> --
> 1.9.1
Acked-by: Siobhan Butler <siobhan.a.butler@intel.com>
Hi Tetsuya, > -----Original Message----- > From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Tetsuya Mukawa > Sent: Wednesday, February 25, 2015 7:32 PM > To: dev@dpdk.org > Subject: [dpdk-dev] [PATCH v15] testpmd: Add port hotplug support > > The patch introduces following commands. > - port attach [ident] > - port detach [port_id] > - attach: attaching a port > - detach: detaching a port > - ident: pci address of physical device. > Or device name and parameters of virtual device. > (ex. 0000:02:00.0, eth_pcap0,iface=eth0) > - port_id: port identifier > > v15: > - Replace rte_eal_dev_attach() by rte_eth_dev_attach() > - Replace rte_eal_dev_detach() by rte_eth_dev_detach() > > v7: > - Fix doc. > (Thanks to Iremonger, Bernard) > - Fix port checking implementation of star_port(); > (Thanks to Qiu, Michael) > v5: > - Add testpmd documentation. > (Thanks to Iremonger, Bernard) > v4: > - Fix strings of command help. > > Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp> > --- > app/test-pmd/cmdline.c | 137 +++++++++++++++---- > app/test-pmd/config.c | 102 ++++++++------ > app/test-pmd/parameters.c | 22 ++- > app/test-pmd/testpmd.c | 199 +++++++++++++++++++++------- > app/test-pmd/testpmd.h | 18 ++- > doc/guides/testpmd_app_ug/testpmd_funcs.rst | 57 ++++++++ [...] > @@ -1446,8 +1497,8 @@ stop_port(portid_t pid) > } > printf("Stopping ports...\n"); > > - for (pi = 0; pi < nb_ports; pi++) { > - if (pid < nb_ports && pid != pi) > + FOREACH_PORT(pi, ports) { > + if (!port_id_is_invalid(pid, DISABLED_WARN) && pid != pi) If using "port stop all", this function does not work as it should. Problem is that pid = RTE_PORT_ALL = 255, and then ports[255].enabled is undefined, as ports array is allocated only for RTE_MAX_ETHPORTS (32 by default). So, the solution could be either increasing the ports array to 256 items, or check if we are passing RTE_PORT_ALL in pid. What do you think? Thanks, Pablo > continue; > > port = &ports[pi]; > @@ -1463,7 +1514,7 @@ stop_port(portid_t pid) > need_check_link_status = 1; > } > if (need_check_link_status && !no_link_check) > - check_all_ports_link_status(nb_ports, RTE_PORT_ALL); > + check_all_ports_link_status(RTE_PORT_ALL); > > printf("Done\n"); > }
On 2015/03/04 0:54, De Lara Guarch, Pablo wrote: > Hi Tetsuya, > >> -----Original Message----- >> From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Tetsuya Mukawa >> Sent: Wednesday, February 25, 2015 7:32 PM >> To: dev@dpdk.org >> Subject: [dpdk-dev] [PATCH v15] testpmd: Add port hotplug support >> >> The patch introduces following commands. >> - port attach [ident] >> - port detach [port_id] >> - attach: attaching a port >> - detach: detaching a port >> - ident: pci address of physical device. >> Or device name and parameters of virtual device. >> (ex. 0000:02:00.0, eth_pcap0,iface=eth0) >> - port_id: port identifier >> >> v15: >> - Replace rte_eal_dev_attach() by rte_eth_dev_attach() >> - Replace rte_eal_dev_detach() by rte_eth_dev_detach() >> >> v7: >> - Fix doc. >> (Thanks to Iremonger, Bernard) >> - Fix port checking implementation of star_port(); >> (Thanks to Qiu, Michael) >> v5: >> - Add testpmd documentation. >> (Thanks to Iremonger, Bernard) >> v4: >> - Fix strings of command help. >> >> Signed-off-by: Tetsuya Mukawa <mukawa@igel.co.jp> >> --- >> app/test-pmd/cmdline.c | 137 +++++++++++++++---- >> app/test-pmd/config.c | 102 ++++++++------ >> app/test-pmd/parameters.c | 22 ++- >> app/test-pmd/testpmd.c | 199 +++++++++++++++++++++------- >> app/test-pmd/testpmd.h | 18 ++- >> doc/guides/testpmd_app_ug/testpmd_funcs.rst | 57 ++++++++ > [...] > >> @@ -1446,8 +1497,8 @@ stop_port(portid_t pid) >> } >> printf("Stopping ports...\n"); >> >> - for (pi = 0; pi < nb_ports; pi++) { >> - if (pid < nb_ports && pid != pi) >> + FOREACH_PORT(pi, ports) { >> + if (!port_id_is_invalid(pid, DISABLED_WARN) && pid != pi) > If using "port stop all", this function does not work as it should. > Problem is that pid = RTE_PORT_ALL = 255, and then ports[255].enabled is undefined, > as ports array is allocated only for RTE_MAX_ETHPORTS (32 by default). > > So, the solution could be either increasing the ports array to 256 items, > or check if we are passing RTE_PORT_ALL in pid. > > What do you think? Hi Pablo, Thanks for reporting. Yes, it should be fixed. Could you please check a patch I will send later? I'll fix it same as start_port(). Thanks, Tetsuya > > Thanks, > Pablo > >> continue; >> >> port = &ports[pi]; >> @@ -1463,7 +1514,7 @@ stop_port(portid_t pid) >> need_check_link_status = 1; >> } >> if (need_check_link_status && !no_link_check) >> - check_all_ports_link_status(nb_ports, RTE_PORT_ALL); >> + check_all_ports_link_status(RTE_PORT_ALL); >> >> printf("Done\n"); >> }