* [dpdk-dev] [PATCH 0/8] Dynamic RSS Configuration for Bonding
@ 2015-06-03 10:58 Tomasz Kulasek
2015-06-03 10:59 ` [dpdk-dev] [PATCH 1/8] bond: dynamic rss configuration Tomasz Kulasek
` (9 more replies)
0 siblings, 10 replies; 126+ messages in thread
From: Tomasz Kulasek @ 2015-06-03 10:58 UTC (permalink / raw)
To: dev
OVERVIEW
--------
1) Setting .rxmode.mq_mode for bonding device to ETH_MQ_RX_RSS makes bonding
device fully RSS-capable, so all slaves are synchronized with its configuration.
This mode is intended to provide RSS configuration as known from "dynamic RSS
configuration for one port" and made slaves transparent for client application
implementation.
2) If .rxmode.mq_mode for bonding device isn't ETH_MQ_RX_RSS, slaves are not
synchronized. That provides an ability to configure them manually. This mode may
be useful when application wants to manage RSS in an unusual way and the
consistency of RSS configuration for slaves isn't required.
Turning on/off RSS mode for slaves when bonding is started is not possible.
Other RSS configuration is propagated over slaves, when bonding device API is
used to do it.
Tomasz Kulasek (8):
bond: dynamic rss configuration
ring: dynamic rss configuration
test: dynamic rss configuration
bond: queue stats mapping
ring: queue stats mapping set dummy implementation
examples: dynamic rss configuration for bonding
doc: fixed spellings and typos
doc: dynamic rss configuration for bonding
app/test/Makefile | 1 +
app/test/test_link_bonding_rssconf.c | 674 ++++++++++++++
.../prog_guide/link_bonding_poll_mode_drv_lib.rst | 40 +-
drivers/net/bonding/rte_eth_bond_api.c | 22 +
drivers/net/bonding/rte_eth_bond_pmd.c | 222 ++++-
drivers/net/bonding/rte_eth_bond_private.h | 11 +
drivers/net/ring/rte_eth_ring.c | 133 ++-
examples/bond_rss/Makefile | 59 ++
examples/bond_rss/bondrss.c | 293 +++++++
examples/bond_rss/bondrss.h | 163 ++++
examples/bond_rss/config.c | 251 ++++++
examples/bond_rss/ui.c | 915 ++++++++++++++++++++
12 files changed, 2759 insertions(+), 25 deletions(-)
create mode 100644 app/test/test_link_bonding_rssconf.c
create mode 100644 examples/bond_rss/Makefile
create mode 100644 examples/bond_rss/bondrss.c
create mode 100644 examples/bond_rss/bondrss.h
create mode 100644 examples/bond_rss/config.c
create mode 100644 examples/bond_rss/ui.c
--
1.7.9.5
^ permalink raw reply [flat|nested] 126+ messages in thread* [dpdk-dev] [PATCH 1/8] bond: dynamic rss configuration 2015-06-03 10:58 [dpdk-dev] [PATCH 0/8] Dynamic RSS Configuration for Bonding Tomasz Kulasek @ 2015-06-03 10:59 ` Tomasz Kulasek 2015-06-03 10:59 ` [dpdk-dev] [PATCH 2/8] ring: " Tomasz Kulasek ` (8 subsequent siblings) 9 siblings, 0 replies; 126+ messages in thread From: Tomasz Kulasek @ 2015-06-03 10:59 UTC (permalink / raw) To: dev Bonding device implements independent management of RSS settings. It stores its own copies of settings i.e. RETA, RSS hash function and RSS key. It’s required to ensure consistency. 1) RSS hash function set for bonding device is maximal set of RSS hash functions supported by all bonded devices. That mean, to have RSS support for bonding, all slaves should be RSS-capable. 2) RSS key is propagated over the slaves "as is", and is always 40 bytes long. 3) RETA for bonding is an internal table managed by bonding API, and is used as a pattern to set up slaves. Its size is GCD of all RETA sizes, so it can be easily used as a pattern providing expected behavior, even if slaves RETA sizes are different. Signed-off-by: Tomasz Kulasek <tomaszx.kulasek@intel.com> --- drivers/net/bonding/rte_eth_bond_api.c | 22 ++++ drivers/net/bonding/rte_eth_bond_pmd.c | 192 ++++++++++++++++++++++++++-- drivers/net/bonding/rte_eth_bond_private.h | 11 ++ 3 files changed, 211 insertions(+), 14 deletions(-) diff --git a/drivers/net/bonding/rte_eth_bond_api.c b/drivers/net/bonding/rte_eth_bond_api.c index e91a623..a31b098 100644 --- a/drivers/net/bonding/rte_eth_bond_api.c +++ b/drivers/net/bonding/rte_eth_bond_api.c @@ -301,6 +301,7 @@ rte_eth_bond_create(const char *name, uint8_t mode, uint8_t socket_id) internals->active_slave_count = 0; internals->rx_offload_capa = 0; internals->tx_offload_capa = 0; + internals->flow_type_rss_offloads = 0; memset(internals->active_slaves, 0, sizeof(internals->active_slaves)); memset(internals->slaves, 0, sizeof(internals->slaves)); @@ -366,6 +367,11 @@ __eth_bond_slave_add_lock_free(uint8_t bonded_port_id, uint8_t slave_port_id) memset(&dev_info, 0, sizeof(dev_info)); rte_eth_dev_info_get(slave_port_id, &dev_info); + /* We need to store slaves reta_size to be able to synchronize RETA for all + * slave devices even if its sizes are different. + */ + internals->slaves[internals->slave_count].reta_size = dev_info.reta_size; + if (internals->slave_count < 1) { /* if MAC is not user defined then use MAC of first slave add to * bonded device */ @@ -379,9 +385,16 @@ __eth_bond_slave_add_lock_free(uint8_t bonded_port_id, uint8_t slave_port_id) /* Make primary slave */ internals->primary_port = slave_port_id; + /* Inherit queues settings from first slave */ + internals->nb_rx_queues = slave_eth_dev->data->nb_rx_queues; + internals->nb_tx_queues = slave_eth_dev->data->nb_tx_queues; + + internals->reta_size = dev_info.reta_size; + /* Take the first dev's offload capabilities */ internals->rx_offload_capa = dev_info.rx_offload_capa; internals->tx_offload_capa = dev_info.tx_offload_capa; + internals->flow_type_rss_offloads = dev_info.flow_type_rss_offloads; } else { /* Check slave link properties are supported if props are set, @@ -400,6 +413,13 @@ __eth_bond_slave_add_lock_free(uint8_t bonded_port_id, uint8_t slave_port_id) } internals->rx_offload_capa &= dev_info.rx_offload_capa; internals->tx_offload_capa &= dev_info.tx_offload_capa; + internals->flow_type_rss_offloads &= dev_info.flow_type_rss_offloads; + + /* RETA size is GCD of all slaves RETA sizes, + * so, if all sizes will be the power of 2, the lower one is GCD */ + if (internals->reta_size > dev_info.reta_size) + internals->reta_size = dev_info.reta_size; + } internals->slave_count++; @@ -528,6 +548,8 @@ __eth_bond_slave_remove_lock_free(uint8_t bonded_port_id, uint8_t slave_port_id) if (internals->slave_count == 0) { internals->rx_offload_capa = 0; internals->tx_offload_capa = 0; + internals->flow_type_rss_offloads = 0; + internals->reta_size = 0; } return 0; } diff --git a/drivers/net/bonding/rte_eth_bond_pmd.c b/drivers/net/bonding/rte_eth_bond_pmd.c index c937e6b..4f6fc68 100644 --- a/drivers/net/bonding/rte_eth_bond_pmd.c +++ b/drivers/net/bonding/rte_eth_bond_pmd.c @@ -1306,6 +1306,15 @@ slave_configure(struct rte_eth_dev *bonded_eth_dev, if (slave_eth_dev->driver->pci_drv.drv_flags & RTE_PCI_DRV_INTR_LSC) slave_eth_dev->data->dev_conf.intr_conf.lsc = 1; + /* If RSS is enabled for bonding, try to enable it for slaves */ + if (bonded_eth_dev->data->dev_conf.rxmode.mq_mode & ETH_MQ_RX_RSS) { + memcpy( + &(slave_eth_dev->data->dev_conf.rx_adv_conf.rss_conf), + &(bonded_eth_dev->data->dev_conf.rx_adv_conf.rss_conf), + sizeof(bonded_eth_dev->data->dev_conf.rx_adv_conf.rss_conf)); + slave_eth_dev->data->dev_conf.rxmode.mq_mode |= ETH_MQ_RX_RSS; + } + /* Configure device */ errval = rte_eth_dev_configure(slave_eth_dev->data->port_id, bonded_eth_dev->data->nb_rx_queues, @@ -1349,6 +1358,29 @@ slave_configure(struct rte_eth_dev *bonded_eth_dev, } } + /* If RSS is enabled for bonding, synchronize RETA */ + if (bonded_eth_dev->data->dev_conf.rxmode.mq_mode & ETH_MQ_RX_RSS) { + int i; + struct bond_dev_private *internals; + internals = bonded_eth_dev->data->dev_private; + + for (i = 0; i < internals->slave_count; i++) { + if (internals->slaves[i].port_id == slave_eth_dev->data->port_id) { + errval = rte_eth_dev_rss_reta_update( + slave_eth_dev->data->port_id, + &(internals->reta_conf[0]), + internals->slaves[i].reta_size); + if (errval != 0) { + RTE_BOND_LOG(ERR, + "rte_eth_dev_rss_reta_update: port=%d, err (%d)", + slave_eth_dev->data->port_id, errval); + return -1; + } + break; + } + } + } + /* Start device */ errval = rte_eth_dev_start(slave_eth_dev->data->port_id); if (errval != 0) { @@ -1569,6 +1601,9 @@ bond_ethdev_info(struct rte_eth_dev *dev, struct rte_eth_dev_info *dev_info) dev_info->rx_offload_capa = internals->rx_offload_capa; dev_info->tx_offload_capa = internals->tx_offload_capa; + dev_info->flow_type_rss_offloads = internals->flow_type_rss_offloads; + + dev_info->reta_size = internals->reta_size; } static int @@ -1954,21 +1989,126 @@ bond_ethdev_lsc_event_callback(uint8_t port_id, enum rte_eth_event_type type, } } +static int +bond_ethdev_rss_reta_update(struct rte_eth_dev *dev, + struct rte_eth_rss_reta_entry64 *reta_conf, + uint16_t reta_size) +{ + int i, j, result = 0; + int slave_reta_size; + int reta_count; + struct bond_dev_private *internals = dev->data->dev_private; + + if (reta_size != internals->reta_size) + return -EINVAL; + + /* Copy RETA table */ + reta_count = reta_size / RTE_RETA_GROUP_SIZE; + + for (i = 0; i < reta_count; i++) { + internals->reta_conf[i].mask = reta_conf[i].mask; + for (j = 0; j < RTE_RETA_GROUP_SIZE; j++) + if ((reta_conf[i].mask >> j) & 0x01) + internals->reta_conf[i].reta[j] = reta_conf[i].reta[j]; + } + + /* Fill rest of array (512 - max slaves RETA size) */ + for (; i < 512 / RTE_RETA_GROUP_SIZE; i += reta_count) + memcpy(&(internals->reta_conf[i]), &(internals->reta_conf[0]), + sizeof(internals->reta_conf[0]) * reta_count); + + /* Propagate RETA over slaves */ + for (i = 0; i < internals->slave_count; i++) { + slave_reta_size = internals->slaves[i].reta_size; + result = rte_eth_dev_rss_reta_update(internals->slaves[i].port_id, + &(internals->reta_conf[0]), slave_reta_size); + if (result < 0) + return result; + } + + return 0; +} + +static int +bond_ethdev_rss_reta_query(struct rte_eth_dev *dev, + struct rte_eth_rss_reta_entry64 *reta_conf, + uint16_t reta_size) +{ + int i, j; + struct bond_dev_private *internals = dev->data->dev_private; + + if (reta_size != internals->reta_size) + return -EINVAL; + + /* Copy RETA table */ + for (i = 0; i < reta_size / RTE_RETA_GROUP_SIZE; i++) + for (j = 0; j < RTE_RETA_GROUP_SIZE; j++) + if ((reta_conf[i].mask >> j) & 0x01) + reta_conf[i].reta[j] = internals->reta_conf[i].reta[j]; + + return 0; +} + +static int +bond_ethdev_rss_hash_update(struct rte_eth_dev *dev, + struct rte_eth_rss_conf *rss_conf) +{ + int i, result = 0; + struct bond_dev_private *internals = dev->data->dev_private; + struct rte_eth_rss_conf bond_rss_conf; + + memcpy(&bond_rss_conf, rss_conf, sizeof(struct rte_eth_rss_conf)); + + bond_rss_conf.rss_hf &= internals->flow_type_rss_offloads; + + if (bond_rss_conf.rss_hf != 0) + dev->data->dev_conf.rx_adv_conf.rss_conf.rss_hf = bond_rss_conf.rss_hf; + + if (bond_rss_conf.rss_key != NULL) + memcpy(internals->rss_key, bond_rss_conf.rss_key, 40); + + for (i = 0; i < internals->slave_count; i++) { + result = rte_eth_dev_rss_hash_update(internals->slaves[i].port_id, + &bond_rss_conf); + if (result < 0) + return result; + } + + return 0; +} + +static int +bond_ethdev_rss_hash_conf_get(struct rte_eth_dev *dev, + struct rte_eth_rss_conf *rss_conf) +{ + struct bond_dev_private *internals = dev->data->dev_private; + + rss_conf->rss_hf = dev->data->dev_conf.rx_adv_conf.rss_conf.rss_hf; + if (rss_conf->rss_key != NULL) + memcpy(rss_conf->rss_key, internals->rss_key, 40); + + return 0; +} + struct eth_dev_ops default_dev_ops = { - .dev_start = bond_ethdev_start, - .dev_stop = bond_ethdev_stop, - .dev_close = bond_ethdev_close, - .dev_configure = bond_ethdev_configure, - .dev_infos_get = bond_ethdev_info, - .rx_queue_setup = bond_ethdev_rx_queue_setup, - .tx_queue_setup = bond_ethdev_tx_queue_setup, - .rx_queue_release = bond_ethdev_rx_queue_release, - .tx_queue_release = bond_ethdev_tx_queue_release, - .link_update = bond_ethdev_link_update, - .stats_get = bond_ethdev_stats_get, - .stats_reset = bond_ethdev_stats_reset, - .promiscuous_enable = bond_ethdev_promiscuous_enable, - .promiscuous_disable = bond_ethdev_promiscuous_disable + .dev_start = bond_ethdev_start, + .dev_stop = bond_ethdev_stop, + .dev_close = bond_ethdev_close, + .dev_configure = bond_ethdev_configure, + .dev_infos_get = bond_ethdev_info, + .rx_queue_setup = bond_ethdev_rx_queue_setup, + .tx_queue_setup = bond_ethdev_tx_queue_setup, + .rx_queue_release = bond_ethdev_rx_queue_release, + .tx_queue_release = bond_ethdev_tx_queue_release, + .link_update = bond_ethdev_link_update, + .stats_get = bond_ethdev_stats_get, + .stats_reset = bond_ethdev_stats_reset, + .promiscuous_enable = bond_ethdev_promiscuous_enable, + .promiscuous_disable = bond_ethdev_promiscuous_disable, + .reta_update = bond_ethdev_rss_reta_update, + .reta_query = bond_ethdev_rss_reta_query, + .rss_hash_update = bond_ethdev_rss_hash_update, + .rss_hash_conf_get = bond_ethdev_rss_hash_conf_get }; static int @@ -2049,6 +2189,30 @@ bond_ethdev_configure(struct rte_eth_dev *dev) int arg_count; uint8_t port_id = dev - rte_eth_devices; + static const uint8_t default_rss_key[40] = { + 0x6D, 0x5A, 0x56, 0xDA, 0x25, 0x5B, 0x0E, 0xC2, 0x41, 0x67, 0x25, 0x3D, + 0x43, 0xA3, 0x8F, 0xB0, 0xD0, 0xCA, 0x2B, 0xCB, 0xAE, 0x7B, 0x30, 0xB4, + 0x77, 0xCB, 0x2D, 0xA3, 0x80, 0x30, 0xF2, 0x0C, 0x6A, 0x42, 0xB7, 0x3B, + 0xBE, 0xAC, 0x01, 0xFA + }; + + int i, j; + + /* If RSS is enabled, fill table and key with default values */ + if (dev->data->dev_conf.rxmode.mq_mode & ETH_MQ_RX_RSS) { + + dev->data->dev_conf.rx_adv_conf.rss_conf.rss_key = internals->rss_key; + dev->data->dev_conf.rx_adv_conf.rss_conf.rss_key_len = 40; + memcpy(internals->rss_key, default_rss_key, 40); + + for (i = 0; i < 512 / RTE_RETA_GROUP_SIZE; i++) { + internals->reta_conf[i].mask = ~0LL; + for (j = 0; j < RTE_RETA_GROUP_SIZE; j++) + internals->reta_conf[i].reta[j] = j % dev->data->nb_rx_queues; + } + + } + /* * if no kvlist, it means that this bonded device has been created * through the bonding api. diff --git a/drivers/net/bonding/rte_eth_bond_private.h b/drivers/net/bonding/rte_eth_bond_private.h index 45e5c65..26f4b28 100644 --- a/drivers/net/bonding/rte_eth_bond_private.h +++ b/drivers/net/bonding/rte_eth_bond_private.h @@ -103,6 +103,8 @@ struct bond_slave_details { uint8_t last_link_status; /**< Port Id of slave eth_dev */ struct ether_addr persisted_mac_addr; + + uint16_t reta_size; }; @@ -155,6 +157,15 @@ struct bond_dev_private { uint32_t rx_offload_capa; /** Rx offload capability */ uint32_t tx_offload_capa; /** Tx offload capability */ + /** Bit mask of RSS offloads, the bit offset also means flow type */ + uint64_t flow_type_rss_offloads; + + uint16_t reta_size; + struct rte_eth_rss_reta_entry64 reta_conf[512 / RTE_RETA_GROUP_SIZE]; + + uint8_t rss_key[40]; /**< 40-byte hash key. */ + uint8_t rss_key_len; /**< hash key length in bytes. */ + struct rte_kvargs *kvlist; uint8_t slave_update_idx; }; -- 1.7.9.5 ^ permalink raw reply [flat|nested] 126+ messages in thread
* [dpdk-dev] [PATCH 2/8] ring: dynamic rss configuration 2015-06-03 10:58 [dpdk-dev] [PATCH 0/8] Dynamic RSS Configuration for Bonding Tomasz Kulasek 2015-06-03 10:59 ` [dpdk-dev] [PATCH 1/8] bond: dynamic rss configuration Tomasz Kulasek @ 2015-06-03 10:59 ` Tomasz Kulasek 2015-06-03 10:59 ` [dpdk-dev] [PATCH 3/8] test: " Tomasz Kulasek ` (7 subsequent siblings) 9 siblings, 0 replies; 126+ messages in thread From: Tomasz Kulasek @ 2015-06-03 10:59 UTC (permalink / raw) To: dev This implementation allows to set and read RSS configuration for ring device, and is used to validate right values propagation over the slaves, in test units for dynamic RSS configuration for bonding. It have no impact on packet processing by ring device. Signed-off-by: Tomasz Kulasek <tomaszx.kulasek@intel.com> --- drivers/net/ring/rte_eth_ring.c | 122 +++++++++++++++++++++++++++++++++++++-- 1 file changed, 118 insertions(+), 4 deletions(-) diff --git a/drivers/net/ring/rte_eth_ring.c b/drivers/net/ring/rte_eth_ring.c index 6832f01..6d2182f 100644 --- a/drivers/net/ring/rte_eth_ring.c +++ b/drivers/net/ring/rte_eth_ring.c @@ -39,6 +39,7 @@ #include <rte_string_fns.h> #include <rte_dev.h> #include <rte_kvargs.h> +#include <rte_spinlock.h> #define ETH_RING_NUMA_NODE_ACTION_ARG "nodeaction" #define ETH_RING_ACTION_CREATE "CREATE" @@ -66,6 +67,17 @@ struct pmd_internals { struct ring_queue tx_ring_queues[RTE_PMD_RING_MAX_TX_RINGS]; struct ether_addr address; + + /** Bit mask of RSS offloads, the bit offset also means flow type */ + uint64_t flow_type_rss_offloads; + + rte_spinlock_t rss_lock; + + uint16_t reta_size; + struct rte_eth_rss_reta_entry64 reta_conf[ETH_RSS_RETA_SIZE_128 / + RTE_RETA_GROUP_SIZE]; + + uint8_t rss_key[40]; /**< 40-byte hash key. */ }; @@ -173,6 +185,8 @@ eth_dev_info(struct rte_eth_dev *dev, dev_info->max_tx_queues = (uint16_t)internals->nb_tx_queues; dev_info->min_rx_bufsize = 0; dev_info->pci_dev = NULL; + dev_info->flow_type_rss_offloads = internals->flow_type_rss_offloads; + dev_info->reta_size = ETH_RSS_RETA_SIZE_128; } static void @@ -234,6 +248,93 @@ static int eth_link_update(struct rte_eth_dev *dev __rte_unused, int wait_to_complete __rte_unused) { return 0; } +static int +eth_rss_reta_update(struct rte_eth_dev *dev, + struct rte_eth_rss_reta_entry64 *reta_conf, + uint16_t reta_size) +{ + int i, j; + struct pmd_internals *internal = dev->data->dev_private; + + if (reta_size != internal->reta_size) + return -EINVAL; + + rte_spinlock_lock(&internal->rss_lock); + + /* Copy RETA table */ + for (i = 0; i < (internal->reta_size / RTE_RETA_GROUP_SIZE); i++) { + internal->reta_conf[i].mask = reta_conf[i].mask; + for (j = 0; j < RTE_RETA_GROUP_SIZE; j++) + if ((reta_conf[i].mask >> j) & 0x01) + internal->reta_conf[i].reta[j] = reta_conf[i].reta[j]; + } + + rte_spinlock_unlock(&internal->rss_lock); + + return 0; +} + +static int +eth_rss_reta_query(struct rte_eth_dev *dev, + struct rte_eth_rss_reta_entry64 *reta_conf, + uint16_t reta_size) +{ + int i, j; + struct pmd_internals *internal = dev->data->dev_private; + + if (reta_size != internal->reta_size) return -EINVAL; + + rte_spinlock_lock(&internal->rss_lock); + + /* Copy RETA table */ + for (i = 0; i < (internal->reta_size / RTE_RETA_GROUP_SIZE); i++) { + for (j = 0; j < RTE_RETA_GROUP_SIZE; j++) + if ((reta_conf[i].mask >> j) & 0x01) + reta_conf[i].reta[j] = internal->reta_conf[i].reta[j]; + } + + rte_spinlock_unlock(&internal->rss_lock); + + return 0; +} + +static int +eth_rss_hash_update(struct rte_eth_dev *dev, + struct rte_eth_rss_conf *rss_conf) +{ + struct pmd_internals *internal = dev->data->dev_private; + + rte_spinlock_lock(&internal->rss_lock); + + if ((rss_conf->rss_hf & internal->flow_type_rss_offloads) != 0) + dev->data->dev_conf.rx_adv_conf.rss_conf.rss_hf = + rss_conf->rss_hf & internal->flow_type_rss_offloads; + + if (rss_conf->rss_key != NULL) + memcpy(internal->rss_key, rss_conf->rss_key, 40); + + rte_spinlock_unlock(&internal->rss_lock); + + return 0; +} + +static int +eth_rss_hash_conf_get(struct rte_eth_dev *dev, + struct rte_eth_rss_conf *rss_conf) +{ + struct pmd_internals *internal = dev->data->dev_private; + + rte_spinlock_lock(&internal->rss_lock); + + rss_conf->rss_hf = dev->data->dev_conf.rx_adv_conf.rss_conf.rss_hf; + if (rss_conf->rss_key != NULL) + memcpy(rss_conf->rss_key, internal->rss_key, 40); + + rte_spinlock_unlock(&internal->rss_lock); + + return 0; +} + static const struct eth_dev_ops ops = { .dev_start = eth_dev_start, .dev_stop = eth_dev_stop, @@ -250,6 +351,10 @@ static const struct eth_dev_ops ops = { .stats_reset = eth_stats_reset, .mac_addr_remove = eth_mac_addr_remove, .mac_addr_add = eth_mac_addr_add, + .reta_update = eth_rss_reta_update, + .reta_query = eth_rss_reta_query, + .rss_hash_update = eth_rss_hash_update, + .rss_hash_conf_get = eth_rss_hash_conf_get }; int @@ -268,6 +373,13 @@ rte_eth_from_rings(const char *name, struct rte_ring *const rx_queues[], unsigned i; + static const uint8_t default_rss_key[40] = { + 0x6D, 0x5A, 0x56, 0xDA, 0x25, 0x5B, 0x0E, 0xC2, 0x41, 0x67, 0x25, 0x3D, + 0x43, 0xA3, 0x8F, 0xB0, 0xD0, 0xCA, 0x2B, 0xCB, 0xAE, 0x7B, 0x30, 0xB4, + 0x77, 0xCB, 0x2D, 0xA3, 0x80, 0x30, 0xF2, 0x0C, 0x6A, 0x42, 0xB7, 0x3B, + 0xBE, 0xAC, 0x01, 0xFA + }; + /* do some parameter checking */ if (rx_queues == NULL && nb_rx_queues > 0) goto error; @@ -316,12 +428,14 @@ rte_eth_from_rings(const char *name, struct rte_ring *const rx_queues[], internals->nb_rx_queues = nb_rx_queues; internals->nb_tx_queues = nb_tx_queues; - for (i = 0; i < nb_rx_queues; i++) { + for (i = 0; i < nb_rx_queues; i++) internals->rx_ring_queues[i].rng = rx_queues[i]; - } - for (i = 0; i < nb_tx_queues; i++) { + for (i = 0; i < nb_tx_queues; i++) internals->tx_ring_queues[i].rng = tx_queues[i]; - } + internals->flow_type_rss_offloads = ETH_RSS_PROTO_MASK; + internals->reta_size = ETH_RSS_RETA_SIZE_128; + + memcpy(internals->rss_key, default_rss_key, 40); eth_drv->pci_drv.name = ring_ethdev_driver_name; eth_drv->pci_drv.id_table = id_table; -- 1.7.9.5 ^ permalink raw reply [flat|nested] 126+ messages in thread
* [dpdk-dev] [PATCH 3/8] test: dynamic rss configuration 2015-06-03 10:58 [dpdk-dev] [PATCH 0/8] Dynamic RSS Configuration for Bonding Tomasz Kulasek 2015-06-03 10:59 ` [dpdk-dev] [PATCH 1/8] bond: dynamic rss configuration Tomasz Kulasek 2015-06-03 10:59 ` [dpdk-dev] [PATCH 2/8] ring: " Tomasz Kulasek @ 2015-06-03 10:59 ` Tomasz Kulasek 2015-06-03 10:59 ` [dpdk-dev] [PATCH 4/8] bond: queue stats mapping Tomasz Kulasek ` (6 subsequent siblings) 9 siblings, 0 replies; 126+ messages in thread From: Tomasz Kulasek @ 2015-06-03 10:59 UTC (permalink / raw) To: dev This test module uses ring device to check right RSS configuration propagation. 1) Propagation test a) Set RSS hash function for bonding, fetch RSS hash function from bonded slave, check if same. Do it for all slaves. b) Repeat above for RSS key and RETA. 2)Dynamic adding slave to the bonding a) Remove slave from bonding. b) Change its configuration to other than bonding. c) Add slave to the started bonding device. d) Check if slaves configuration changed. 3) Repeat point 1) and 2) with RSS multi-queue mode enabled/disabled for bonding port. Signed-off-by: Tomasz Kulasek <tomaszx.kulasek@intel.com> --- app/test/Makefile | 1 + app/test/test_link_bonding_rssconf.c | 674 ++++++++++++++++++++++++++++++++++ 2 files changed, 675 insertions(+) create mode 100644 app/test/test_link_bonding_rssconf.c diff --git a/app/test/Makefile b/app/test/Makefile index 3c777bf..aef9ae7 100644 --- a/app/test/Makefile +++ b/app/test/Makefile @@ -134,6 +134,7 @@ SRCS-$(CONFIG_RTE_LIBRTE_ACL) += test_acl.c ifeq ($(CONFIG_RTE_LIBRTE_PMD_RING),y) SRCS-$(CONFIG_RTE_LIBRTE_PMD_BOND) += test_link_bonding.c SRCS-$(CONFIG_RTE_LIBRTE_PMD_BOND) += test_link_bonding_mode4.c +SRCS-$(CONFIG_RTE_LIBRTE_PMD_BOND) += test_link_bonding_rssconf.c endif SRCS-$(CONFIG_RTE_LIBRTE_PMD_RING) += test_pmd_ring.c diff --git a/app/test/test_link_bonding_rssconf.c b/app/test/test_link_bonding_rssconf.c new file mode 100644 index 0000000..c0be755 --- /dev/null +++ b/app/test/test_link_bonding_rssconf.c @@ -0,0 +1,674 @@ +/*- + * BSD LICENSE + * + * Copyright(c) 2015 Intel Corporation. All rights reserved. + * All rights reserved. + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * + * * Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * * Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in + * the documentation and/or other materials provided with the + * distribution. + * * Neither the name of Intel Corporation nor the names of its + * contributors may be used to endorse or promote products derived + * from this software without specific prior written permission. + * + * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS + * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT + * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR + * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT + * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, + * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT + * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, + * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY + * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT + * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE + * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + */ + +#include <string.h> +#include <stdarg.h> +#include <stdio.h> +#include <stdlib.h> +#include <stdint.h> +#include <inttypes.h> +#include <errno.h> +#include <rte_cycles.h> +#include <sys/queue.h> + +#include <rte_byteorder.h> +#include <rte_common.h> +#include <rte_debug.h> +#include <rte_ethdev.h> +#include <rte_log.h> +#include <rte_lcore.h> +#include <rte_memory.h> + +#include <rte_string_fns.h> + +#include <rte_eth_ring.h> +#include <rte_errno.h> +#include <rte_eth_bond.h> + +#include "test.h" + +#define SLAVE_COUNT (4) + +#define RXTX_RING_SIZE 1024 +#define RXTX_QUEUE_COUNT 4 + +#define BONDED_DEV_NAME ("rssconf_bond_dev") + +#define SLAVE_DEV_NAME_FMT ("rssconf_slave%d") +#define SLAVE_RXTX_QUEUE_FMT ("rssconf_slave%d_q%d") + +#define NUM_MBUFS 8191 +#define MBUF_SIZE (1600 + sizeof(struct rte_mbuf) + RTE_PKTMBUF_HEADROOM) +#define MBUF_CACHE_SIZE 250 +#define BURST_SIZE 32 + +#define INVALID_SOCKET_ID (-1) +#define INVALID_PORT_ID (0xFF) +#define INVALID_BONDING_MODE (-1) + +struct slave_conf { + uint8_t port_id; + struct rte_eth_dev_info dev_info; + + struct rte_eth_rss_conf rss_conf; + uint8_t rss_key[40]; + struct rte_eth_rss_reta_entry64 reta_conf[512 / RTE_RETA_GROUP_SIZE]; + + uint8_t is_slave; + struct rte_ring *rxtx_queue[RXTX_QUEUE_COUNT]; +}; + +struct link_bonding_rssconf_unittest_params { + uint8_t bond_port_id; + struct rte_eth_dev_info bond_dev_info; + struct rte_eth_rss_reta_entry64 bond_reta_conf[512 / RTE_RETA_GROUP_SIZE]; + struct slave_conf slave_ports[SLAVE_COUNT]; + + struct rte_mempool *mbuf_pool; +}; + +static struct link_bonding_rssconf_unittest_params test_params = { + .bond_port_id = INVALID_PORT_ID, + .slave_ports = { + [0 ... SLAVE_COUNT - 1] = { .port_id = INVALID_PORT_ID, .is_slave = 0} + }, + .mbuf_pool = NULL, +}; + +/** + * Default port configuration with RSS turned off + */ +static struct rte_eth_conf default_pmd_conf = { + .rxmode = { + .mq_mode = ETH_MQ_RX_NONE, + .max_rx_pkt_len = ETHER_MAX_LEN, + .split_hdr_size = 0, + .header_split = 0, /**< Header Split disabled */ + .hw_ip_checksum = 0, /**< IP checksum offload enabled */ + .hw_vlan_filter = 0, /**< VLAN filtering disabled */ + .jumbo_frame = 0, /**< Jumbo Frame Support disabled */ + .hw_strip_crc = 0, /**< CRC stripped by hardware */ + }, + .txmode = { + .mq_mode = ETH_MQ_TX_NONE, + }, + .lpbk_mode = 0, +}; + +static struct rte_eth_conf rss_pmd_conf = { + .rxmode = { + .mq_mode = ETH_MQ_RX_RSS, + .max_rx_pkt_len = ETHER_MAX_LEN, + .split_hdr_size = 0, + .header_split = 0, /**< Header Split disabled */ + .hw_ip_checksum = 0, /**< IP checksum offload enabled */ + .hw_vlan_filter = 0, /**< VLAN filtering disabled */ + .jumbo_frame = 0, /**< Jumbo Frame Support disabled */ + .hw_strip_crc = 0, /**< CRC stripped by hardware */ + }, + .txmode = { + .mq_mode = ETH_MQ_TX_NONE, + }, + .rx_adv_conf = { + .rss_conf = { + .rss_key = NULL, + .rss_hf = ETH_RSS_IPV6, + }, + }, + .lpbk_mode = 0, +}; + +#define FOR_EACH(_i, _item, _array, _size) \ + for (_i = 0, _item = &_array[0]; _i < _size && (_item = &_array[_i]); _i++) + +/* Macro for iterating over every port that can be used as a slave + * in this test. + * _i variable used as an index in test_params->slave_ports + * _slave pointer to &test_params->slave_ports[_idx] + */ +#define FOR_EACH_PORT(_i, _port) \ + FOR_EACH(_i, _port, test_params.slave_ports, \ + RTE_DIM(test_params.slave_ports)) + +static int +configure_ethdev(uint8_t port_id, struct rte_eth_conf *eth_conf, uint8_t start) +{ + int rxq, txq; + + TEST_ASSERT(rte_eth_dev_configure(port_id, RXTX_QUEUE_COUNT, + RXTX_QUEUE_COUNT, eth_conf) == 0, "Failed to configure device %u", + port_id); + + for (rxq = 0; rxq < RXTX_QUEUE_COUNT; rxq++) { + TEST_ASSERT(rte_eth_rx_queue_setup(port_id, rxq, RXTX_RING_SIZE, + rte_eth_dev_socket_id(port_id), NULL, + test_params.mbuf_pool) == 0, "Failed to setup rx queue."); + } + + for (txq = 0; txq < RXTX_QUEUE_COUNT; txq++) { + TEST_ASSERT(rte_eth_tx_queue_setup(port_id, txq, RXTX_RING_SIZE, + rte_eth_dev_socket_id(port_id), NULL) == 0, + "Failed to setup tx queue."); + } + + if (start) { + TEST_ASSERT(rte_eth_dev_start(port_id) == 0, + "Failed to start device (%d).", port_id); + } + + return 0; +} + +/** + * Remove all slaves from bonding + */ +static int +remove_slaves(void) +{ + unsigned n; + struct slave_conf *port; + + FOR_EACH_PORT(n, port) { + port = &test_params.slave_ports[n]; + if (port->is_slave) { + TEST_ASSERT_SUCCESS(rte_eth_bond_slave_remove( + test_params.bond_port_id, port->port_id), + "Cannot remove slave %d from bonding", port->port_id); + port->is_slave = 0; + } + } + + return 0; +} + +static int +remove_slaves_and_stop_bonded_device(void) +{ + TEST_ASSERT_SUCCESS(remove_slaves(), "Removing slaves"); + rte_eth_dev_stop(test_params.bond_port_id); + return TEST_SUCCESS; +} + +/** + * Add all slaves to bonding + */ +static int +bond_slaves(void) +{ + unsigned n; + struct slave_conf *port; + + FOR_EACH_PORT(n, port) { + port = &test_params.slave_ports[n]; + if (!port->is_slave) { + TEST_ASSERT_SUCCESS(rte_eth_bond_slave_add(test_params.bond_port_id, + port->port_id), "Cannot attach slave %d to the bonding", + port->port_id); + port->is_slave = 1; + } + } + + return 0; +} + +/** + * Set all RETA values in port_id to value + */ +static int +reta_set(uint8_t port_id, uint8_t value, int reta_size) +{ + struct rte_eth_rss_reta_entry64 reta_conf[512/RTE_RETA_GROUP_SIZE]; + int i, j; + + for (i = 0; i < reta_size / RTE_RETA_GROUP_SIZE; i++) { + /* select all fields to set */ + reta_conf[i].mask = ~0LL; + for (j = 0; j < RTE_RETA_GROUP_SIZE; j++) + reta_conf[i].reta[j] = value; + } + + return rte_eth_dev_rss_reta_update(port_id, reta_conf, reta_size); +} + +/** + * Check if slaves RETA is synchronized with bonding port. Returns 1 if slave + * port is synced with bonding port. + */ +static int +reta_check_synced(struct slave_conf *port) +{ + unsigned i; + + for (i = 0; i < test_params.bond_dev_info.reta_size; + i++) { + + int index = i / RTE_RETA_GROUP_SIZE; + int shift = i % RTE_RETA_GROUP_SIZE; + + if (port->reta_conf[index].reta[shift] != + test_params.bond_reta_conf[index].reta[shift]) + return 0; + + } + + return 1; +} + +/** + * Fetch bonding ports RETA + */ +static int +bond_reta_fetch(void) { + unsigned j; + + for (j = 0; j < test_params.bond_dev_info.reta_size / RTE_RETA_GROUP_SIZE; + j++) + test_params.bond_reta_conf[j].mask = ~0LL; + + TEST_ASSERT_SUCCESS(rte_eth_dev_rss_reta_query(test_params.bond_port_id, + test_params.bond_reta_conf, test_params.bond_dev_info.reta_size), + "Cannot take bonding ports RSS configuration"); + return 0; +} + +/** + * Fetch slaves RETA + */ +static int +slave_reta_fetch(struct slave_conf *port) { + unsigned j; + + for (j = 0; j < port->dev_info.reta_size / RTE_RETA_GROUP_SIZE; j++) + port->reta_conf[j].mask = ~0LL; + + TEST_ASSERT_SUCCESS(rte_eth_dev_rss_reta_query(port->port_id, + port->reta_conf, port->dev_info.reta_size), + "Cannot take bonding ports RSS configuration"); + return 0; +} + +/** + * Remove and add slave to check if slaves configuration is synced with + * the bonding ports values after adding new slave. + */ +static int +slave_remove_and_add(void) +{ + struct slave_conf *port = &(test_params.slave_ports[0]); + + /* 1. Remove first slave from bonding */ + TEST_ASSERT_SUCCESS(rte_eth_bond_slave_remove(test_params.bond_port_id, + port->port_id), "Cannot remove slave #d from bonding"); + + /* 2. Change removed (ex-)slave and bonding configuration to different + * values + */ + reta_set(test_params.bond_port_id, 1, test_params.bond_dev_info.reta_size); + bond_reta_fetch(); + + reta_set(port->port_id, 2, port->dev_info.reta_size); + slave_reta_fetch(port); + + TEST_ASSERT(reta_check_synced(port) == 0, + "Removed slave didn't should be synchronized with bonding port"); + + /* 3. Add (ex-)slave and check if configuration changed*/ + TEST_ASSERT_SUCCESS(rte_eth_bond_slave_add(test_params.bond_port_id, + port->port_id), "Cannot add slave"); + + bond_reta_fetch(); + slave_reta_fetch(port); + + return reta_check_synced(port); +} + +/** + * Test configuration propagation over slaves. + */ +static int +test_propagate(void) +{ + unsigned i; + uint8_t n; + struct slave_conf *port; + uint8_t bond_rss_key[40]; + struct rte_eth_rss_conf bond_rss_conf; + + int retval = 0; + uint64_t rss_hf = 0; + uint64_t default_rss_hf = 0; + + rte_eth_dev_info_get(test_params.bond_port_id, &test_params.bond_dev_info); + + /* + * Test hash function propagation + */ + for (i = 0; i < sizeof(test_params.bond_dev_info.flow_type_rss_offloads)*8; + i++) { + + rss_hf = test_params.bond_dev_info.flow_type_rss_offloads & (1<<i); + if (rss_hf) { + bond_rss_conf.rss_key = NULL; + bond_rss_conf.rss_hf = rss_hf; + + retval = rte_eth_dev_rss_hash_update(test_params.bond_port_id, + &bond_rss_conf); + TEST_ASSERT_SUCCESS(retval, "Cannot set slaves hash function"); + + FOR_EACH_PORT(n, port) { + port = &test_params.slave_ports[n]; + + retval = rte_eth_dev_rss_hash_conf_get(port->port_id, + &port->rss_conf); + TEST_ASSERT_SUCCESS(retval, + "Cannot take slaves RSS configuration"); + + TEST_ASSERT(port->rss_conf.rss_hf == rss_hf, + "Hash function not propagated for slave %d", + port->port_id); + } + + default_rss_hf = rss_hf; + } + + } + + /* + * Test key propagation + */ + for (i = 1; i < 10; i++) { + + /* Set all keys to zero */ + FOR_EACH_PORT(n, port) { + port = &test_params.slave_ports[n]; + memset(port->rss_conf.rss_key, 0, 40); + retval = rte_eth_dev_rss_hash_update(port->port_id, + &port->rss_conf); + TEST_ASSERT_SUCCESS(retval, "Cannot set slaves RSS keys"); + } + + memset(bond_rss_key, i, sizeof(bond_rss_key)); + bond_rss_conf.rss_hf = default_rss_hf, + bond_rss_conf.rss_key = bond_rss_key; + bond_rss_conf.rss_key_len = 40; + + retval = rte_eth_dev_rss_hash_update(test_params.bond_port_id, + &bond_rss_conf); + TEST_ASSERT_SUCCESS(retval, "Cannot set bonded port RSS keys"); + + FOR_EACH_PORT(n, port) { + port = &test_params.slave_ports[n]; + + retval = rte_eth_dev_rss_hash_conf_get(port->port_id, + &(port->rss_conf)); + + TEST_ASSERT_SUCCESS(retval, + "Cannot take slaves RSS configuration"); + + /* compare keys */ + retval = memcmp(port->rss_conf.rss_key, bond_rss_key, + sizeof(bond_rss_key)); + TEST_ASSERT(retval == 0, "Key value not propagated for slave %d", + port->port_id); + } + } + + /* + * Test RETA propagation + */ + for (i = 0; i < RXTX_QUEUE_COUNT; i++) { + + /* Set all keys to zero */ + FOR_EACH_PORT(n, port) { + port = &test_params.slave_ports[n]; + retval = reta_set(port->port_id, (i + 1) % RXTX_QUEUE_COUNT, + port->dev_info.reta_size); + TEST_ASSERT_SUCCESS(retval, "Cannot set slaves RETA"); + } + + TEST_ASSERT_SUCCESS(reta_set(test_params.bond_port_id, + i % RXTX_QUEUE_COUNT, test_params.bond_dev_info.reta_size), + "Cannot set bonded port RETA"); + + bond_reta_fetch(); + + FOR_EACH_PORT(n, port) { + port = &test_params.slave_ports[n]; + + slave_reta_fetch(port); + TEST_ASSERT(reta_check_synced(port) == 1, "RETAs inconsistent"); + } + } + + return TEST_SUCCESS; +} + +/** + * Test propagation logic, when RX_RSS mq_mode is turned on for bonding port + */ +static int +test_rss(void) +{ + /** + * Configure bonding port in RSS mq mode + */ + TEST_ASSERT_SUCCESS(configure_ethdev(test_params.bond_port_id, + &rss_pmd_conf, 0), "Failed to configure bonding device\n"); + + rte_eth_dev_info_get(test_params.bond_port_id, &test_params.bond_dev_info); + + TEST_ASSERT_SUCCESS(bond_slaves(), "Bonding slaves failed"); + + TEST_ASSERT_SUCCESS(rte_eth_dev_start(test_params.bond_port_id), + "Failed to start bonding port (%d).", test_params.bond_port_id); + + TEST_ASSERT_SUCCESS(test_propagate(), "Propagation test failed"); + + TEST_ASSERT(slave_remove_and_add() == 1, "New slave should be synced"); + + remove_slaves_and_stop_bonded_device(); + + return TEST_SUCCESS; +} + +/** + * Test propagation logic, when RX_RSS mq_mode is turned off for bonding port + */ +static int +test_rss_lazy(void) +{ + TEST_ASSERT_SUCCESS(configure_ethdev(test_params.bond_port_id, + &default_pmd_conf, 0), "Failed to configure bonding device\n"); + + rte_eth_dev_info_get(test_params.bond_port_id, &test_params.bond_dev_info); + + TEST_ASSERT_SUCCESS(bond_slaves(), "Bonding slaves failed"); + + TEST_ASSERT_SUCCESS(rte_eth_dev_start(test_params.bond_port_id), + "Failed to start bonding port (%d).", test_params.bond_port_id); + + TEST_ASSERT_SUCCESS(test_propagate(), "Propagation test failed"); + + TEST_ASSERT(slave_remove_and_add() == 0, "New slave shouldn't be synced"); + + remove_slaves_and_stop_bonded_device(); + + return TEST_SUCCESS; +} + +static int +test_setup(void) +{ + unsigned i, n; + int retval; + char name[256]; + char rng_name[RTE_RING_NAMESIZE]; + struct slave_conf *port; + + if (test_params.mbuf_pool == NULL) { + + test_params.mbuf_pool = rte_mempool_create("MBUF_POOL", NUM_MBUFS * + SLAVE_COUNT, MBUF_SIZE, MBUF_CACHE_SIZE, + sizeof(struct rte_pktmbuf_pool_private), rte_pktmbuf_pool_init, + NULL, rte_pktmbuf_init, NULL, rte_socket_id(), 0); + + TEST_ASSERT(test_params.mbuf_pool != NULL, + "rte_mempool_create failed\n"); + } + + /* Create / initialize ring eth devs. */ + FOR_EACH_PORT(n, port) { + port = &test_params.slave_ports[n]; + + snprintf(name, sizeof(name), SLAVE_DEV_NAME_FMT, n); + + for (i = 0; i < RXTX_QUEUE_COUNT; i++) { + snprintf(rng_name, sizeof(rng_name), SLAVE_RXTX_QUEUE_FMT, n, i); + + if (port->rxtx_queue[i] == NULL) { + port->rxtx_queue[i] = rte_ring_create(rng_name, RXTX_RING_SIZE, + 0, RING_F_SP_ENQ|RING_F_SC_DEQ); + TEST_ASSERT(port->rxtx_queue[i] != NULL, + "Failed to allocate rx ring '%s': %s", name, + rte_strerror(rte_errno)); + } + } + + if (rte_eth_from_rings(name, port->rxtx_queue, RXTX_QUEUE_COUNT, + port->rxtx_queue, RXTX_QUEUE_COUNT, 0)) { + TEST_ASSERT(retval >= 0, + "Failed to create ring ethdev '%s'\n", name); + } + + port->port_id = rte_eth_dev_count() - 1; + port->rss_conf.rss_key = port->rss_key; + port->rss_conf.rss_key_len = 40; + + retval = configure_ethdev(port->port_id, &default_pmd_conf, 1); + TEST_ASSERT_SUCCESS(retval, "Failed to configure virtual ethdev %s\n", + name); + + rte_eth_dev_info_get(port->port_id, &port->dev_info); + } + + if (test_params.bond_port_id == INVALID_PORT_ID) { + retval = rte_eth_bond_create(BONDED_DEV_NAME, 0, rte_socket_id()); + + TEST_ASSERT(retval >= 0, "Failed to create bonded ethdev %s", + BONDED_DEV_NAME); + + test_params.bond_port_id = retval; + + TEST_ASSERT_SUCCESS(configure_ethdev(test_params.bond_port_id, + &default_pmd_conf, 0), "Failed to configure bonding device\n"); + + rte_eth_dev_info_get(test_params.bond_port_id, + &test_params.bond_dev_info); + } + + return TEST_SUCCESS; +} + +static int +check_environment(void) +{ + return TEST_SUCCESS; +} + +static int +test_rssconf_executor(int (*test_func)(void)) +{ + int test_result; + + /* Check if environment is clean. Fail to launch a test if there was + * a critical error before that prevented to reset environment. */ + TEST_ASSERT_SUCCESS(check_environment(), + "Refusing to launch test in dirty environment."); + + RTE_VERIFY(test_func != NULL); + test_result = (*test_func)(); + + /* If test succeed check if environment wast left in good condition. */ + if (test_result == TEST_SUCCESS) + test_result = check_environment(); + + /* Reset environment in case test failed to do that. */ + if (test_result != TEST_SUCCESS) { + TEST_ASSERT_SUCCESS(remove_slaves_and_stop_bonded_device(), + "Failed to stop bonded device"); + } + + return test_result; +} + +static int +test_setup_wrapper(void) +{ + return test_rssconf_executor(&test_setup); +} + +static int +test_rss_wrapper(void) +{ + return test_rssconf_executor(&test_rss); +} + +static int +test_rss_lazy_wrapper(void) +{ + return test_rssconf_executor(&test_rss_lazy); +} + +static struct unit_test_suite link_bonding_rssconf_test_suite = { + .suite_name = "RSS Dynamic Configuration for Bonding Unit Test Suite", + .setup = test_setup, + .unit_test_cases = { + TEST_CASE_NAMED("test_setup", test_setup_wrapper), + TEST_CASE_NAMED("test_rss", test_rss_wrapper), + TEST_CASE_NAMED("test_rss_lazy", test_rss_lazy_wrapper), + { NULL, NULL, NULL, NULL, NULL } /**< NULL terminate unit test array */ + } +}; + +static int +test_link_bonding_rssconf(void) +{ + return unit_test_suite_runner(&link_bonding_rssconf_test_suite); +} + +static struct test_command test_link_bonding_rssconf_cmd = { + .command = "link_bonding_rssconf_autotest", + .callback = test_link_bonding_rssconf, +}; + +REGISTER_TEST_COMMAND(test_link_bonding_rssconf_cmd); -- 1.7.9.5 ^ permalink raw reply [flat|nested] 126+ messages in thread
* [dpdk-dev] [PATCH 4/8] bond: queue stats mapping 2015-06-03 10:58 [dpdk-dev] [PATCH 0/8] Dynamic RSS Configuration for Bonding Tomasz Kulasek ` (2 preceding siblings ...) 2015-06-03 10:59 ` [dpdk-dev] [PATCH 3/8] test: " Tomasz Kulasek @ 2015-06-03 10:59 ` Tomasz Kulasek 2015-06-03 10:59 ` [dpdk-dev] [PATCH 5/8] ring: queue stats mapping set dummy implementation Tomasz Kulasek ` (5 subsequent siblings) 9 siblings, 0 replies; 126+ messages in thread From: Tomasz Kulasek @ 2015-06-03 10:59 UTC (permalink / raw) To: dev Queue stats mapping is used in example application. This patch adds propagation of mapping over the slaves, and fills bonding port's stats with a sum of corresponding values taken from bonded slaves, when stats are requested for bonding port. Signed-off-by: Tomasz Kulasek <tomaszx.kulasek@intel.com> --- drivers/net/bonding/rte_eth_bond_pmd.c | 30 +++++++++++++++++++++++++++++- 1 file changed, 29 insertions(+), 1 deletion(-) diff --git a/drivers/net/bonding/rte_eth_bond_pmd.c b/drivers/net/bonding/rte_eth_bond_pmd.c index 4f6fc68..7a4ecb8 100644 --- a/drivers/net/bonding/rte_eth_bond_pmd.c +++ b/drivers/net/bonding/rte_eth_bond_pmd.c @@ -1766,7 +1766,7 @@ bond_ethdev_stats_get(struct rte_eth_dev *dev, struct rte_eth_stats *stats) struct bond_dev_private *internals = dev->data->dev_private; struct rte_eth_stats slave_stats; - int i; + int i, j; /* clear bonded stats before populating from slaves */ memset(stats, 0, sizeof(*stats)); @@ -1788,6 +1788,15 @@ bond_ethdev_stats_get(struct rte_eth_dev *dev, struct rte_eth_stats *stats) stats->rx_pause_xon += slave_stats.rx_pause_xon; stats->tx_pause_xoff += slave_stats.tx_pause_xoff; stats->rx_pause_xoff += slave_stats.rx_pause_xoff; + + for (j = 0; j < RTE_ETHDEV_QUEUE_STAT_CNTRS; j++) { + stats->q_ipackets[j] += slave_stats.q_ipackets[j]; + stats->q_opackets[j] += slave_stats.q_ipackets[j]; + stats->q_ibytes[j] += slave_stats.q_ipackets[j]; + stats->q_obytes[j] += slave_stats.q_ipackets[j]; + stats->q_errors[j] += slave_stats.q_ipackets[j]; + } + } } @@ -2090,6 +2099,24 @@ bond_ethdev_rss_hash_conf_get(struct rte_eth_dev *dev, return 0; } +static int +bond_ethdev_queue_stats_mapping_set(struct rte_eth_dev *dev, + uint16_t queue_id, uint8_t stat_idx, uint8_t is_rx) +{ + int i; + struct bond_dev_private *internals = dev->data->dev_private; + + for (i = 0; i < internals->slave_count; i++) + if (is_rx) + rte_eth_dev_set_rx_queue_stats_mapping ( + internals->slaves[i].port_id, queue_id, stat_idx); + else + rte_eth_dev_set_tx_queue_stats_mapping ( + internals->slaves[i].port_id, queue_id, stat_idx); + + return 0; +} + struct eth_dev_ops default_dev_ops = { .dev_start = bond_ethdev_start, .dev_stop = bond_ethdev_stop, @@ -2103,6 +2130,7 @@ struct eth_dev_ops default_dev_ops = { .link_update = bond_ethdev_link_update, .stats_get = bond_ethdev_stats_get, .stats_reset = bond_ethdev_stats_reset, + .queue_stats_mapping_set = bond_ethdev_queue_stats_mapping_set, .promiscuous_enable = bond_ethdev_promiscuous_enable, .promiscuous_disable = bond_ethdev_promiscuous_disable, .reta_update = bond_ethdev_rss_reta_update, -- 1.7.9.5 ^ permalink raw reply [flat|nested] 126+ messages in thread
* [dpdk-dev] [PATCH 5/8] ring: queue stats mapping set dummy implementation 2015-06-03 10:58 [dpdk-dev] [PATCH 0/8] Dynamic RSS Configuration for Bonding Tomasz Kulasek ` (3 preceding siblings ...) 2015-06-03 10:59 ` [dpdk-dev] [PATCH 4/8] bond: queue stats mapping Tomasz Kulasek @ 2015-06-03 10:59 ` Tomasz Kulasek 2015-06-03 10:59 ` [dpdk-dev] [PATCH 6/8] examples: dynamic rss configuration for bonding Tomasz Kulasek ` (4 subsequent siblings) 9 siblings, 0 replies; 126+ messages in thread From: Tomasz Kulasek @ 2015-06-03 10:59 UTC (permalink / raw) To: dev Per queue statistics are already implemented for ring device, but with static mapping (stat_idx == queue_id). This fix is required, if you want to use ring device in test application and is used only to point that per queue statistics are provided for this device. Signed-off-by: Tomasz Kulasek <tomaszx.kulasek@intel.com> --- drivers/net/ring/rte_eth_ring.c | 11 +++++++++++ 1 file changed, 11 insertions(+) diff --git a/drivers/net/ring/rte_eth_ring.c b/drivers/net/ring/rte_eth_ring.c index 6d2182f..07695c3 100644 --- a/drivers/net/ring/rte_eth_ring.c +++ b/drivers/net/ring/rte_eth_ring.c @@ -335,6 +335,16 @@ eth_rss_hash_conf_get(struct rte_eth_dev *dev, return 0; } +static int +eth_queue_stats_mapping_set(__rte_unused struct rte_eth_dev *dev, + __rte_unused uint16_t queue_id, + __rte_unused uint8_t stat_idx, + __rte_unused uint8_t is_rx) +{ + /* Do nothing, just return ok */ + return 0; +} + static const struct eth_dev_ops ops = { .dev_start = eth_dev_start, .dev_stop = eth_dev_stop, @@ -347,6 +357,7 @@ static const struct eth_dev_ops ops = { .rx_queue_release = eth_queue_release, .tx_queue_release = eth_queue_release, .link_update = eth_link_update, + .queue_stats_mapping_set = eth_queue_stats_mapping_set, .stats_get = eth_stats_get, .stats_reset = eth_stats_reset, .mac_addr_remove = eth_mac_addr_remove, -- 1.7.9.5 ^ permalink raw reply [flat|nested] 126+ messages in thread
* [dpdk-dev] [PATCH 6/8] examples: dynamic rss configuration for bonding 2015-06-03 10:58 [dpdk-dev] [PATCH 0/8] Dynamic RSS Configuration for Bonding Tomasz Kulasek ` (4 preceding siblings ...) 2015-06-03 10:59 ` [dpdk-dev] [PATCH 5/8] ring: queue stats mapping set dummy implementation Tomasz Kulasek @ 2015-06-03 10:59 ` Tomasz Kulasek 2015-06-03 10:59 ` [dpdk-dev] [PATCH 7/8] doc: fixed spellings and typos Tomasz Kulasek ` (3 subsequent siblings) 9 siblings, 0 replies; 126+ messages in thread From: Tomasz Kulasek @ 2015-06-03 10:59 UTC (permalink / raw) To: dev This application allows you to test RSS configuration for bonded devices changing configuration dynamically, during receiving packets on slaves and observe the changes in its distribution over queues. After initialization process, all accessible ports are attached to one bonding as slaves. Monitor screen is divided into five main parts: - Port selection (on the very top) - RSS Configuration for selected port including hash function, key and RETA - Incoming packets statistics - Incoming packets list for selected port - Status bar with contextual information about selected part Signed-off-by: Tomasz Kulasek <tomaszx.kulasek@intel.com> --- examples/bond_rss/Makefile | 59 +++ examples/bond_rss/bondrss.c | 293 ++++++++++++++ examples/bond_rss/bondrss.h | 163 ++++++++ examples/bond_rss/config.c | 251 ++++++++++++ examples/bond_rss/ui.c | 915 +++++++++++++++++++++++++++++++++++++++++++ 5 files changed, 1681 insertions(+) create mode 100644 examples/bond_rss/Makefile create mode 100644 examples/bond_rss/bondrss.c create mode 100644 examples/bond_rss/bondrss.h create mode 100644 examples/bond_rss/config.c create mode 100644 examples/bond_rss/ui.c diff --git a/examples/bond_rss/Makefile b/examples/bond_rss/Makefile new file mode 100644 index 0000000..db457a3 --- /dev/null +++ b/examples/bond_rss/Makefile @@ -0,0 +1,59 @@ +# BSD LICENSE +# +# Copyright(c) 2010-2015 Intel Corporation. All rights reserved. +# All rights reserved. +# +# Redistribution and use in source and binary forms, with or without +# modification, are permitted provided that the following conditions +# are met: +# +# * Redistributions of source code must retain the above copyright +# notice, this list of conditions and the following disclaimer. +# * Redistributions in binary form must reproduce the above copyright +# notice, this list of conditions and the following disclaimer in +# the documentation and/or other materials provided with the +# distribution. +# * Neither the name of Intel Corporation nor the names of its +# contributors may be used to endorse or promote products derived +# from this software without specific prior written permission. +# +# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS +# "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT +# LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR +# A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT +# OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, +# SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT +# LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, +# DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY +# THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT +# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE +# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + +ifeq ($(RTE_SDK),) +$(error "Please define RTE_SDK environment variable") +endif + +# Default target, can be overridden by command line or environment +RTE_TARGET ?= x86_64-native-linuxapp-gcc + +include $(RTE_SDK)/mk/rte.vars.mk + +# binary name +APP = bondrss + +# all source are stored in SRCS-y +SRCS-y := bondrss.c +SRCS-y += ui.c +SRCS-y += config.c + +CFLAGS += $(WERROR_FLAGS) + +# workaround for a gcc bug with noreturn attribute +# http://gcc.gnu.org/bugzilla/show_bug.cgi?id=12603 +ifeq ($(CONFIG_RTE_TOOLCHAIN_GCC),y) +CFLAGS_main.o += -Wno-return-type +endif + +LDLIBS += -lncurses + +include $(RTE_SDK)/mk/rte.extapp.mk diff --git a/examples/bond_rss/bondrss.c b/examples/bond_rss/bondrss.c new file mode 100644 index 0000000..2dc9979 --- /dev/null +++ b/examples/bond_rss/bondrss.c @@ -0,0 +1,293 @@ +/*- + * BSD LICENSE + * + * Copyright(c) 2010-2015 Intel Corporation. All rights reserved. + * All rights reserved. + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * + * * Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * * Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in + * the documentation and/or other materials provided with the + * distribution. + * * Neither the name of Intel Corporation nor the names of its + * contributors may be used to endorse or promote products derived + * from this software without specific prior written permission. + * + * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS + * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT + * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR + * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT + * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, + * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT + * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, + * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY + * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT + * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE + * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + */ + +#include "bondrss.h" + +#define RSS_HASH_KEY_LENGTH 40 + +static const struct rte_eth_conf port_conf_default = { + .rxmode = { + .mq_mode = ETH_MQ_RX_RSS, + .max_rx_pkt_len = ETHER_MAX_LEN, + .split_hdr_size = 0, + .header_split = 0, /**< Header Split disabled. */ + .hw_ip_checksum = 0, /**< IP checksum offload disabled. */ + .hw_vlan_filter = 1, /**< VLAN filtering enabled. */ + .hw_vlan_strip = 1, /**< VLAN strip enabled. */ + .hw_vlan_extend = 0, /**< Extended VLAN disabled. */ + .jumbo_frame = 0, /**< Jumbo Frame Support disabled. */ + .hw_strip_crc = 0, /**< CRC stripping by hardware disabled. */ + .enable_scatter = 0, /**< scatter rx disabled */ + }, + .rx_adv_conf = { + .rss_conf = { + .rss_key = NULL, + .rss_hf = ETH_RSS_IPV4, + }, + }, +}; + +static const struct rte_eth_conf port_conf_bonding = { + .rxmode = { + .mq_mode = ETH_MQ_RX_RSS, + .max_rx_pkt_len = ETHER_MAX_LEN, + .split_hdr_size = 0, + .header_split = 0, /**< Header Split disabled. */ + .hw_ip_checksum = 0, /**< IP checksum offload disabled. */ + .hw_vlan_filter = 1, /**< VLAN filtering enabled. */ + .hw_vlan_strip = 1, /**< VLAN strip enabled. */ + .hw_vlan_extend = 0, /**< Extended VLAN disabled. */ + .jumbo_frame = 0, /**< Jumbo Frame Support disabled. */ + .hw_strip_crc = 0, /**< CRC stripping by hardware disabled. */ + .enable_scatter = 0, /**< scatter rx disabled */ + }, + .rx_adv_conf = { + .rss_conf = { + .rss_key = NULL, + .rss_hf = ETH_RSS_IPV6, + }, + }, +}; + +/* + * Configurable number of RX/TX queues. + */ +int nb_rxq = 4; /**< Number of RX queues per port. */ +int nb_txq = 1; /**< Number of TX queues per port. */ + +int bond_port_id = -1; /**< Bonded device port id */ + +int nb_ports; +struct rte_port *ports; +rte_atomic64_t counter; /**< Received packet's counter */ + +/* + * Initializes a given port using global settings and with the rx buffers + * coming from the mbuf_pool passed as parameter + */ +static inline +int init_port(uint8_t port_id, struct rte_mempool *mbuf_pool, + struct rte_eth_conf port_conf) +{ + + int retval; + uint16_t q; + struct rte_eth_rxconf rxconf; + + struct rte_port *port; + + if (port_id >= rte_eth_dev_count()) + return -1; + + port = &ports[port_id]; + port->nb_rx_queues = nb_rxq; + port->nb_tx_queues = nb_txq; + + memcpy(&(port->dev_conf), &port_conf, sizeof(port_conf)); + retval = rte_eth_dev_configure(port_id, port->nb_rx_queues, + port->nb_tx_queues, &port_conf); + + if (retval != 0) + return retval; + + rte_eth_dev_info_get(port_id, &(port->dev_info)); + rxconf = (port->dev_info).default_rxconf; + rxconf.rx_free_thresh = 32; + + for (q = 0; q < port->nb_rx_queues; q++) { + retval = rte_eth_rx_queue_setup(port_id, q, RX_RING_SIZE, + rte_eth_dev_socket_id(port_id), &rxconf, + mbuf_pool); + if (retval < 0) + return retval; + } + + for (q = 0; q < port->nb_tx_queues; q++) { + retval = rte_eth_tx_queue_setup(port_id, q, TX_RING_SIZE, + rte_eth_dev_socket_id(port_id), + &((port->dev_info).default_txconf)); + if (retval < 0) + return retval; + } + + port->bond_mode = -1; + port->promiscuous = 0; + port->enabled = 1; + + return 0; +} + +static int +rx_loop(__attribute__((unused)) void *dummy) +{ + uint8_t port_id; + int nq; + int n; + + for (;;) + for (port_id = 0; port_id < nb_ports; port_id++) { + /* Pool only bonding ports */ + if (ports[port_id].bond_mode >= 0) + for (nq = 0; nq < ports[port_id].nb_rx_queues; nq++) { + struct rte_mbuf *bufs[BURST_SIZE]; + + const uint16_t nb_rx = rte_eth_rx_burst(port_id, nq, bufs, + BURST_SIZE); + if (unlikely(nb_rx == 0)) + continue; + + ports[port_id].rxq_ipackets[nq] += nb_rx; + for (n = 0; n < nb_rx; n++) { + record_pkt(port_id, nq, bufs[n]); + rte_pktmbuf_free(bufs[n]); + } + } + } + + return 0; +} + +static uint16_t +rx_callback(uint8_t port_id, uint16_t qidx, struct rte_mbuf **pkts, + uint16_t nb_pkts, uint16_t max_pkts __rte_unused, void *_ __rte_unused) +{ + int n; + + ports[port_id].rxq_ipackets[qidx] += nb_pkts; + for (n = 0; n < nb_pkts; n++) { + rte_atomic64_inc(&counter); + pkts[n]->udata64 = counter.cnt; + record_pkt(port_id, qidx, pkts[n]); + } + + return nb_pkts; +} + +int +main(int argc, char *argv[]) +{ + struct rte_mempool *mbuf_pool; + unsigned lcore_id = 0; + uint8_t port_id, queue_id; + + /* init EAL */ + int ret = rte_eal_init(argc, argv); + if (ret < 0) + rte_exit(EXIT_FAILURE, "Error with EAL initialization\n"); + argc -= ret; + argv += ret; + + nb_ports = rte_eth_dev_count(); + if (nb_ports < 1) + rte_exit(EXIT_FAILURE, + "At least one port must be available to create a bonding\n"); + + mbuf_pool = rte_mempool_create("MBUF_POOL", NUM_MBUFS * nb_ports, MBUF_SIZE, + MBUF_CACHE_SIZE, + sizeof(struct rte_pktmbuf_pool_private), + rte_pktmbuf_pool_init, NULL, + rte_pktmbuf_init, NULL, rte_socket_id(), 0); + if (mbuf_pool == NULL) + rte_exit(EXIT_FAILURE, "Cannot create mbuf pool\n"); + + /* Configuration of Ethernet ports. */ + ports = rte_zmalloc("bondrss: ports", + sizeof(struct rte_port) * RTE_MAX_ETHPORTS, + RTE_CACHE_LINE_SIZE); + if (ports == NULL) + rte_exit(EXIT_FAILURE, "rte_zmalloc(%d struct rte_port) failed\n", + RTE_MAX_ETHPORTS); + + /* enabled allocated ports */ + for (port_id = 0; port_id < RTE_MAX_ETHPORTS; port_id++) + ports[port_id].enabled = 0; + + /* initialize all ports */ + for (port_id = 0; port_id < nb_ports; port_id++) { + if (init_port(port_id, mbuf_pool, port_conf_default) != 0) + rte_exit(EXIT_FAILURE, "Cannot init port %"PRIu8"\n", port_id); + + /* Add rx callbacks to the slave's port */ + for (queue_id = 0; queue_id < ports[port_id].nb_rx_queues; queue_id++) + rte_eth_add_rx_callback(port_id, queue_id, rx_callback, NULL); + + } + + /* create bonding port */ + bond_port_id = rte_eth_bond_create("eth_bond", 0, 0); + if (bond_port_id < 0) + rte_exit(EXIT_FAILURE, "Cannot create bonding port\n"); + + for (port_id = 0; port_id < nb_ports; port_id++) + rte_eth_bond_slave_add(bond_port_id, port_id); + + /* count again */ + nb_ports = rte_eth_dev_count(); + + init_port(bond_port_id, mbuf_pool, port_conf_bonding); + + /* start bonding port*/ + ret = rte_eth_dev_start(bond_port_id); + + /* enable promiscuous by default */ + rte_eth_promiscuous_enable(bond_port_id); + + for (port_id = 0; port_id < nb_ports; port_id++) { + ports[port_id].bond_mode = rte_eth_bond_mode_get(port_id); + ports[port_id].promiscuous = rte_eth_promiscuous_get(port_id); + + /* map queues to stats */ + if (ports[port_id].bond_mode >= 0) { + for (queue_id = 0; queue_id < ports[port_id].nb_rx_queues; + queue_id++) { + rte_eth_dev_set_rx_queue_stats_mapping(port_id, queue_id, + queue_id); + } + } + + } + + /* Initialize received packet's counter */ + rte_atomic64_init(&counter); + + if (rte_lcore_count() <= 1) + rte_exit(EXIT_FAILURE, "More than one free lcore is needed\n"); + + lcore_id = rte_get_next_lcore(rte_lcore_id(), 1, 0); + rte_eal_remote_launch(rx_loop, NULL, lcore_id); + + /* call lcore_main on master core only */ + ui_loop(); + + return 0; +} diff --git a/examples/bond_rss/bondrss.h b/examples/bond_rss/bondrss.h new file mode 100644 index 0000000..3e2ef73 --- /dev/null +++ b/examples/bond_rss/bondrss.h @@ -0,0 +1,163 @@ +/*- + * BSD LICENSE + * + * Copyright(c) 2015 Intel Corporation. All rights reserved. + * All rights reserved. + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * + * * Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * * Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in + * the documentation and/or other materials provided with the + * distribution. + * * Neither the name of Intel Corporation nor the names of its + * contributors may be used to endorse or promote products derived + * from this software without specific prior written permission. + * + * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS + * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT + * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR + * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT + * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, + * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT + * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, + * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY + * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT + * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE + * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + */ + +#ifndef BONDRSS_H_ +#define BONDRSS_H_ + +#include <stdint.h> +#include <inttypes.h> +#include <rte_eal.h> +#include <rte_ethdev.h> +#include <rte_cycles.h> +#include <rte_lcore.h> +#include <rte_mbuf.h> +#include <rte_malloc.h> + +#include <rte_eth_bond.h> + +#define RX_RING_SIZE 128 +#define TX_RING_SIZE 512 + +#define NUM_MBUFS 8191 +#define MBUF_SIZE (1600 + sizeof(struct rte_mbuf) + RTE_PKTMBUF_HEADROOM) +#define MBUF_CACHE_SIZE 250 +#define BURST_SIZE 32 + +/* + * Configurable number of RX/TX queues. + */ +extern int nb_rxq; /**< Number of RX queues per port. */ +extern int nb_txq; /**< Number of TX queues per port. */ + +extern int bond_port_id; /**< Bonded device port id */ +extern int bond_mode; + +struct rss_type_info { + char str[32]; + uint64_t rss_type; +}; + +static struct rss_type_info rss_type_table[] = { + {"ipv4", ETH_RSS_IPV4}, + {"ipv4-frag", ETH_RSS_FRAG_IPV4}, + {"ipv4-tcp", ETH_RSS_NONFRAG_IPV4_TCP}, + {"ipv4-udp", ETH_RSS_NONFRAG_IPV4_UDP}, + {"ipv4-sctp", ETH_RSS_NONFRAG_IPV4_SCTP}, + {"ipv4-other", ETH_RSS_NONFRAG_IPV4_OTHER}, + {"ipv6", ETH_RSS_IPV6}, + {"ipv6-frag", ETH_RSS_FRAG_IPV6}, + {"ipv6-tcp", ETH_RSS_NONFRAG_IPV6_TCP}, + {"ipv6-udp", ETH_RSS_NONFRAG_IPV6_UDP}, + {"ipv6-sctp", ETH_RSS_NONFRAG_IPV6_SCTP}, + {"ipv6-other", ETH_RSS_NONFRAG_IPV6_OTHER}, + {"l2-payload", ETH_RSS_L2_PAYLOAD}, + {"ipv6-ex", ETH_RSS_IPV6_EX}, + {"ipv6-tcp-ex", ETH_RSS_IPV6_TCP_EX}, + {"ipv6-udp-ex", ETH_RSS_IPV6_UDP_EX}, +}; + +struct rss_type_fn { + uint8_t enabled; + struct rss_type_info *info; +}; + +/** + * Buffer for last received packets + */ +#define PKT_RECORD_SIZE 256 + +struct pkt_info { + uint64_t n; /**< Packet number */ + uint8_t port_id; + uint8_t queue_id; + uint64_t ol_flags; /**< Offload features. */ + uint32_t rss; /**< RSS hash result if RSS enabled */ +}; + +struct pkt_record { + uint64_t count; /**< Total number of received packets */ + int top; + struct pkt_info pkt_info[PKT_RECORD_SIZE]; +}; + +/** + * The data structure associated with each port. + */ +struct rte_port { + struct rte_eth_dev_info dev_info; /**< PCI info + driver name */ + struct rte_eth_conf dev_conf; /**< Port configuration. */ + + uint16_t nb_rx_queues; /**< Total number of rx queues */ + uint16_t nb_tx_queues; /**< Total number of tx queues*/ + + int bond_mode; + struct ether_addr addr; + + int rss_type_count; + struct rss_type_fn rss_type_fn[RTE_DIM(rss_type_table)]; + struct rte_eth_rss_reta_entry64 reta_conf[512 / RTE_RETA_GROUP_SIZE]; + uint8_t rss_key[40]; + + + struct rte_eth_stats stats; + uint64_t rxq_ipackets[128]; + + struct pkt_record record; + + uint8_t rss; /**< RSS enabled for port */ + uint8_t promiscuous; /**< promiscuous mode enabled for port */ + uint8_t enabled; /**< port is enabled */ +}; + +extern int nb_ports; +extern struct rte_port *ports; +extern rte_atomic64_t counter; /**< Received packet's counter */ + +void update_port_info(uint8_t port_id); + +int rss_enable(uint8_t port_id, int enabled); + +int key_set(uint8_t port_id, uint8_t *key, uint8_t len); +int key_set_random(uint8_t port_id, uint8_t len); + +int reta_set_default(uint8_t port_id); +int reta_set_random(uint8_t port_id); +int reta_set_all(uint8_t port_id, uint8_t value); +int reta_set_weights(uint8_t port_id, uint64_t *weights); + +struct pkt_info *record_pkt(uint8_t port_id, int queue_id, struct rte_mbuf *mb); + +void ui_loop(void); + + +#endif /* BONDRSS_H_ */ diff --git a/examples/bond_rss/config.c b/examples/bond_rss/config.c new file mode 100644 index 0000000..39c00e5 --- /dev/null +++ b/examples/bond_rss/config.c @@ -0,0 +1,251 @@ +/*- + * BSD LICENSE + * + * Copyright(c) 2015 Intel Corporation. All rights reserved. + * All rights reserved. + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * + * * Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * * Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in + * the documentation and/or other materials provided with the + * distribution. + * * Neither the name of Intel Corporation nor the names of its + * contributors may be used to endorse or promote products derived + * from this software without specific prior written permission. + * + * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS + * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT + * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR + * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT + * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, + * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT + * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, + * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY + * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT + * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE + * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + */ + +#include <time.h> +#include <stdlib.h> +#include "bondrss.h" + +static const struct rte_eth_conf port_conf_default = { + .rxmode = { + .max_rx_pkt_len = ETHER_MAX_LEN, + .split_hdr_size = 0, + .header_split = 0, /**< Header Split disabled. */ + .hw_ip_checksum = 0, /**< IP checksum offload disabled. */ + .hw_vlan_filter = 1, /**< VLAN filtering enabled. */ + .hw_vlan_strip = 1, /**< VLAN strip enabled. */ + .hw_vlan_extend = 0, /**< Extended VLAN disabled. */ + .jumbo_frame = 0, /**< Jumbo Frame Support disabled. */ + .hw_strip_crc = 0, /**< CRC stripping by hardware disabled. */ + .enable_scatter = 0, /**< scatter rx disabled */ + }, + .rx_adv_conf = { + .rss_conf = { + .rss_key = NULL, + .rss_hf = ETH_RSS_IPV4, + }, + }, +}; + + +void +update_port_info(uint8_t port_id) +{ + int i; + struct rte_port *port; + + port = &ports[port_id]; + + rte_eth_dev_info_get(port_id, &(port->dev_info)); + + port->promiscuous = rte_eth_promiscuous_get(port_id); + + /* Update device information */ + rte_eth_dev_info_get(port_id, &port->dev_info); + rte_eth_macaddr_get(port_id, &port->addr); + + port->dev_conf.rx_adv_conf.rss_conf.rss_key = port->rss_key; + port->dev_conf.rx_adv_conf.rss_conf.rss_key_len = 40; + rte_eth_dev_rss_hash_conf_get(port_id, + &(port->dev_conf.rx_adv_conf.rss_conf)); + + /* select all fields to be fetched */ + for (i = 0; i < port->dev_info.reta_size / RTE_RETA_GROUP_SIZE; i++) + port->reta_conf[i].mask = ~0LL; + + rte_eth_dev_rss_reta_query(port_id, port->reta_conf, + port->dev_info.reta_size); + +} + +/** + * Try to enable RSS for port_id. + */ +int +rss_enable(uint8_t port_id, int enabled) +{ + struct rte_port *port; + int retval; + + port = &ports[port_id]; + + rte_eth_dev_stop(port_id); + + if (enabled) { + port->dev_conf.rxmode.mq_mode = ETH_MQ_RX_RSS; + port->dev_conf.rx_adv_conf.rss_conf.rss_hf = ETH_RSS_IPV4; + } else { + port->dev_conf.rxmode.mq_mode = 0; + port->dev_conf.rx_adv_conf.rss_conf.rss_hf = 0; + } + + retval = rte_eth_dev_configure(port_id, port->nb_rx_queues, + port->nb_tx_queues, &port->dev_conf); + if (retval != 0) + return retval; + + retval = rte_eth_dev_start(port_id); + + return 0; +} + +int +key_set(uint8_t port_id, uint8_t *key, uint8_t len) +{ + struct rte_eth_rss_conf rss_conf; + + int i; + + rss_conf.rss_key = NULL; + rss_conf.rss_key_len = 0; + i = rte_eth_dev_rss_hash_conf_get(port_id, &rss_conf); + + rss_conf.rss_key = key; + rss_conf.rss_key_len = len; + + i = rte_eth_dev_rss_hash_update(port_id, + &rss_conf); + + return i; +} + +/** + * Set random RSS key value + */ +int +key_set_random(uint8_t port_id, uint8_t len) +{ + int i; + uint8_t key[40]; + + for (i = 0; i < len; i++) + key[i] = rand() % 256; + + return key_set(port_id, key, len); +} + +/** + * Set random RETA values + */ +int +reta_set_random(uint8_t port_id) +{ + struct rte_eth_rss_reta_entry64 reta_conf[512/RTE_RETA_GROUP_SIZE]; + int i, j; + + int reta_size = ports[port_id].dev_info.reta_size; + + for (i = 0; i < reta_size / RTE_RETA_GROUP_SIZE; i++) { + /* select all fields to set */ + reta_conf[i].mask = ~0LL; + for (j = 0; j < RTE_RETA_GROUP_SIZE; j++) + reta_conf[i].reta[j] = rand() % ports[port_id].nb_rx_queues; + } + + return rte_eth_dev_rss_reta_update(port_id, reta_conf, reta_size); +} + +/** + * Set default RETA values + */ +int +reta_set_default(uint8_t port_id) +{ + struct rte_eth_rss_reta_entry64 reta_conf[512/RTE_RETA_GROUP_SIZE]; + int i, j; + + int reta_size = ports[port_id].dev_info.reta_size; + + for (i = 0; i < reta_size / RTE_RETA_GROUP_SIZE; i++) { + /* select all fields to set */ + reta_conf[i].mask = ~0LL; + for (j = 0; j < RTE_RETA_GROUP_SIZE; j++) + reta_conf[i].reta[j] = j % nb_rxq; + } + + return rte_eth_dev_rss_reta_update(port_id, reta_conf, reta_size); +} + +/** + * Fill the RETA with values + */ +int +reta_set_all(uint8_t port_id, uint8_t value) +{ + struct rte_eth_rss_reta_entry64 reta_conf[512/RTE_RETA_GROUP_SIZE]; + int i, j; + + int reta_size = ports[port_id].dev_info.reta_size; + + for (i = 0; i < reta_size / RTE_RETA_GROUP_SIZE; i++) { + /* select all fields to set */ + reta_conf[i].mask = ~0LL; + for (j = 0; j < RTE_RETA_GROUP_SIZE; j++) + reta_conf[i].reta[j] = value; + } + + return rte_eth_dev_rss_reta_update(port_id, reta_conf, reta_size); +} + +/** + * Set the RETA basing on weights table + */ +int +reta_set_weights(uint8_t port_id, uint64_t *weights) +{ + struct rte_eth_rss_reta_entry64 reta_conf[512/RTE_RETA_GROUP_SIZE]; + unsigned i, j, k; + + unsigned reta_size = ports[port_id].dev_info.reta_size; + + uint64_t sum = 0, sum2; + + for (i = 0; i < ports[port_id].nb_rx_queues; i++) + sum += weights[i]; + sum2 = sum * (ports[port_id].nb_rx_queues-1); + + if (sum2 == 0) + return 0; + + for (i = 0; i < reta_size / RTE_RETA_GROUP_SIZE; i++) + reta_conf[i].mask = ~0LL; + k = 0; + for (i = 0; i < ports[port_id].nb_rx_queues; i++) { + for (j = 0; k < reta_size && + j < ((sum - weights[i]) * reta_size + 1) / sum2; j++) { + reta_conf[k/RTE_RETA_GROUP_SIZE].reta[k%RTE_RETA_GROUP_SIZE] = i; + k++; + } + } + + return rte_eth_dev_rss_reta_update(port_id, reta_conf, reta_size); +} diff --git a/examples/bond_rss/ui.c b/examples/bond_rss/ui.c new file mode 100644 index 0000000..be3767c --- /dev/null +++ b/examples/bond_rss/ui.c @@ -0,0 +1,915 @@ +/*- + * BSD LICENSE + * + * Copyright(c) 2015 Intel Corporation. All rights reserved. + * All rights reserved. + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * + * * Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * * Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in + * the documentation and/or other materials provided with the + * distribution. + * * Neither the name of Intel Corporation nor the names of its + * contributors may be used to endorse or promote products derived + * from this software without specific prior written permission. + * + * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS + * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT + * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR + * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT + * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, + * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT + * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, + * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY + * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT + * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE + * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + */ + +#include <rte_atomic.h> + +#include <ncurses.h> +#include <string.h> +#include <signal.h> +#include <sys/ioctl.h> +#include <menu.h> +#include "bondrss.h" + +WINDOW *win_footer = NULL; +WINDOW *win_header = NULL; +WINDOW *win_list = NULL; +WINDOW *win_stats = NULL; +WINDOW *win_reta = NULL; +WINDOW *win_err = NULL; + +int scr_lines = 0; +int scr_cols = 0; + +unsigned win_header_lines; +unsigned win_list_lines; +unsigned win_stats_lines; +unsigned win_reta_lines; +unsigned win_err_lines = 0; + +const char *ui_status_text; +static char *ui_err_text[5]; + +struct rte_port *selected_port; +static uint8_t selected_port_id; + +struct rte_port *focused_port; +static uint8_t focused_port_id; + +#define UI_MODE_PORT 0 +#define UI_MODE_RSS_FN 1 +#define UI_MODE_RSS_KEY 2 +#define UI_MODE_RETA 3 +#define UI_MODE_HELP 4 + +int ui_mode = UI_MODE_PORT; + +static uint16_t _is_lock; +static uint16_t _is_ready; + +static uint16_t _do_configure; +static uint16_t _do_repaint; +static uint16_t _do_repaint_stats; +static uint16_t _do_repaint_list; +static uint16_t _do_repaint_info; +static uint16_t _do_repaint_err; + + +struct help_page { + const char *title; + const char *body; +}; + +static struct help_page help_pages[3] = { + { + .title = "About", + + .body = + "\n\n" + " ______ ______ ______ _____ ______ ______ ______\n" + "| | | \\ / | | \\ | | \\ \\ | | \\ \\ | | | \\ / | / |\n" + "| |--| < | | | | | | | | | | | | | |__| | '------. '------.\n" + "|_|__|_/ \\_|__|_/ |_| |_| |_|_/_/ |_| \\_\\ ____|_/ ____|_/\n" + "\n\n" + "This application allows you to test RSS configuration for " + "bonded devices\nchanging configuration dynamically, during " + "receiving packets on slaves and\nobserve the changes in its " + "distribution over queues.\n" + "\n" + + }, + { + .title = "Introduction", + .body = "\n\n" + "After initialization process, all accessible ports are " + "attached to one\nbonding as slaves.\n" + "\n" + + "Monitor screen is divided into five main parts:\n" + " - Port selection (on the very top)\n" + " - RSS Configuration for selected port including hash " + "function, key and\n RETA\n" + " - Incoming packets statistics\n" + " - Incoming packets list for selected port\n" + " - Status bar with contextual information about selected " + "part\n" + "\n\n" + + "[up], [down] arrows key lets you highlight a part " + "and parameter to change.\n\n" + }, + { + .title = "Key bindings", + .body = + "_Port_selection_\n\n" + " [left], [right] - select port,\n" + " [space] - force turning on/off RSS mode,\n" + " [p] - turn on/off promiscuous mode,\n" + " [c] - clear statistics and incoming packet list,\n" + "\n" + "_RSS_hash_function_\n\n" + " [1]-[9] - toggle selected hash function,\n" + " [a] - select all,\n" + "\n" + "_RSS_key_\n\n" + " [1]-[4] - set one of predefined key value,\n" + " [r] - randomize value,\n" + "\n" + "_RSS_RETA_\n\n" + " [number] - fill RETA with number,\n" + " [r] - randomize value,\n" + " [d] - set default value," + + } +}; + +static int +ui_lock(int wait) +{ + int result; + + while (!(result = rte_atomic16_cmpset(&_is_lock, 0, 1)) && wait) + rte_pause(); + + return result; +} + +static int +ui_unlock(void) +{ + return rte_atomic16_cmpset(&_is_lock, 1, 0); +} + +/** + * Predefined key values + */ +static uint8_t RSS_KEY[4][40] = { + { 0x6D, 0x5A, 0x56, 0xDA, 0x25, 0x5B, 0x0E, 0xC2, 0x41, 0x67, 0x25, 0x3D, + 0x43, 0xA3, 0x8F, 0xB0, 0xD0, 0xCA, 0x2B, 0xCB, 0xAE, 0x7B, 0x30, 0xB4, + 0x77, 0xCB, 0x2D, 0xA3, 0x80, 0x30, 0xF2, 0x0C, 0x6A, 0x42, 0xB7, 0x3B, + 0xBE, 0xAC, 0x01, 0xFA + }, + { 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, + }, + { 0x6D, 0x5A, 0x6D, 0x5A, 0x6D, 0x5A, 0x6D, 0x5A, 0x6D, 0x5A, 0x6D, 0x5A, + 0x6D, 0x5A, 0x6D, 0x5A, 0x6D, 0x5A, 0x6D, 0x5A, 0x6D, 0x5A, 0x6D, 0x5A, + 0x6D, 0x5A, 0x6D, 0x5A, 0x6D, 0x5A, 0x6D, 0x5A, 0x6D, 0x5A, 0x6D, 0x5A, + 0x6D, 0x5A, 0x6D, 0x5A, + }, + { 0x01, 0x02, 0x03, 0x04, 0x05, 0x06, 0x07, 0x08, 0x09, 0x0A, 0x0B, 0x0C, + 0x0D, 0x0E, 0x0F, 0x10, 0x11, 0x12, 0x13, 0x14, 0x15, 0x16, 0x17, 0x18, + 0x19, 0x1A, 0x1B, 0x1C, 0x1D, 0x1E, 0x1F, 0x20, 0x21, 0x22, 0x23, 0x24, + 0x25, 0x26, 0x27, 0x28, + } +}; + +static void +win_err_addline(const char *text) +{ + int i; + + if (win_err_lines < 5) { + win_err_lines++; + _do_configure = 1; + } else { + free(ui_err_text[0]); + for (i = 1; i < 5; i++) + ui_err_text[i - 1] = ui_err_text[i]; + } + + ui_err_text[win_err_lines - 1] = strdup(text); + _do_repaint_err = 1; +} + +static void +win_err_fini(void) +{ + int i; + for (i = 0; i < (int)win_err_lines; i++) + free(ui_err_text[i]); +} + +/** + * Draw full information for port + */ +static void +win_header_draw(uint8_t port_id) +{ + int i, j; + attr_t a; + + update_port_info(selected_port_id); + + /* Draw title bar */ + if (ui_mode == UI_MODE_PORT) + wattron(win_header, COLOR_PAIR(2)); + else + wattron(win_header, COLOR_PAIR(4)); + + mvwprintw(win_header, 0, 0, "::"); + + for (i = 0; i < nb_ports; i++) { + if (ports[i].enabled) { + const char *name = (ports[i].bond_mode >= 0 ? "BOND" : "SLAVE"); + + if (i == port_id) { + wattron(win_header, A_REVERSE); + wprintw(win_header, " %s-%d ", name, (unsigned) i); + wattroff(win_header, A_REVERSE); + } else + wprintw(win_header, " %s-%d ", name, (unsigned) i); + } + } + + for (i = scr_cols - getcurx(win_header) - 3; i > 0; i--) + waddch(win_header, ':'); + + waddch(win_header, + (ports[port_id].rss ? 'R' : '-')); + waddch(win_header, ports[port_id].bond_mode >= 0 ? + ports[port_id].bond_mode + '0' : '-'); + waddch(win_header, ports[port_id].promiscuous == 0 ? '-' : 'P'); + + if (ui_mode == UI_MODE_PORT) + wattroff(win_header, COLOR_PAIR(2)); + else + wattroff(win_header, COLOR_PAIR(4)); + + /* Redraw RSS-Fn */ + selected_port->rss_type_count = 0; + for (i = 0; i < (int) RTE_DIM(rss_type_table); i++) { + if (selected_port->dev_info.flow_type_rss_offloads + & rss_type_table[i].rss_type) { + selected_port->rss_type_fn[selected_port->rss_type_count].info = + &rss_type_table[i]; + selected_port->rss_type_fn[selected_port->rss_type_count].enabled = + ((selected_port->dev_conf.rx_adv_conf.rss_conf.rss_hf + & rss_type_table[i].rss_type) ? 1 : 0); + selected_port->rss_type_count++; + } + } + + a = (ui_mode == UI_MODE_RSS_FN ? COLOR_PAIR(2) : COLOR_PAIR(1)); + + wattron(win_header, a); + mvwprintw(win_header, 1, 0, "FN: "); + for (i = 0; i < selected_port->rss_type_count; i++) { + if (selected_port->rss_type_fn[i].enabled) + wattron(win_header, COLOR_PAIR(3)); + waddstr(win_header, selected_port->rss_type_fn[i].info->str); + if (selected_port->rss_type_fn[i].enabled) + wattron(win_header, a); + waddch(win_header, ' '); + } + wattroff(win_header, a); + + /* Redraw RSS-Key */ + if (ui_mode == UI_MODE_RSS_KEY) + wattron(win_header, COLOR_PAIR(2)); + mvwprintw(win_header, 2, 0, "KEY: "); + for (i = 0; i < 40; i++) + wprintw(win_header, "%02X", + selected_port->dev_conf.rx_adv_conf.rss_conf.rss_key[i]); + if (ui_mode == UI_MODE_RSS_KEY) + wattroff(win_header, COLOR_PAIR(2)); + + + /* Redraw RETA window */ + int idx, shift; + + if (ui_mode == UI_MODE_RETA) + wattron(win_reta, COLOR_PAIR(2)); + + for (j = 0; j < selected_port->dev_info.reta_size / 16; j++) { + wmove(win_reta, j, 0); + for (i = 0; i < 16; i++) { + idx = (j*16 + i) / RTE_RETA_GROUP_SIZE; + shift = (j*16 + i) % RTE_RETA_GROUP_SIZE; + waddch(win_reta, ACS_VLINE); + wprintw(win_reta, "%d", selected_port->reta_conf[idx].reta[shift]); + } + waddch(win_reta, ACS_VLINE); + } + + if (ui_mode == UI_MODE_RETA) + wattroff(win_reta, COLOR_PAIR(2)); + wnoutrefresh(win_reta); + + + /* Stats repaint */ + if (_do_repaint_stats) { + uint64_t total; + + rte_eth_stats_get(selected_port_id, &(selected_port->stats)); + + wmove(win_stats, 0, 0); + total = 0; + for (i = 0; i < selected_port->nb_rx_queues; i++) { + wprintw(win_stats, "Queue %d: %10"PRIu64 + " (%10" PRIu64 ")\n", i, selected_port->rxq_ipackets[i], + selected_port->stats.q_ipackets[i]); + total += selected_port->rxq_ipackets[i]; + } + + wprintw(win_stats, " Total: %10"PRIu64 + " (%10" PRIu64 ")\n", total, selected_port->stats.ipackets); + + _do_repaint_stats = 0; + wnoutrefresh(win_stats); + } + + if (_do_repaint_err && win_err_lines > 0) { + for (i = 0; i < (int)win_err_lines; i++) { + mvwprintw(win_err, i, 0, ui_err_text[i]); + wclrtoeol(win_err); + } + _do_repaint_err = 0; + wnoutrefresh(win_err); + } + + mvwhline(win_header, win_header_lines - 1, 0, 0, scr_cols); +} + +static void +win_footer_draw(const char *text) +{ + if (text != NULL) + ui_status_text = text; + + wclear(win_footer); + wmove(win_footer, 0, 0); + wprintw(win_footer, ui_status_text); + + wprintw(win_footer, "; [arrows]navigate; [q]quit"); + wnoutrefresh(win_footer); +} + +static void +win_list_draw(void) +{ + unsigned n, i; + + struct pkt_info *pkt_info; + + struct rte_port *port = &ports[focused_port_id]; + struct pkt_record *record = &(port->record); + + n = (win_list_lines > record->count) ? record->count : win_list_lines; + + wmove(win_list, 0, 0); + + for (i = 0; i < n; i++) { + + pkt_info = &record->pkt_info[(record->top - n + i + PKT_RECORD_SIZE) + % PKT_RECORD_SIZE]; + + wmove(win_list, i, 0); + + wprintw(win_list, "%5"PRIu64, (unsigned) pkt_info->n); + waddch(win_list, ACS_VLINE); + + wprintw(win_list, "%2d: ", (unsigned) pkt_info->queue_id); + wprintw(win_list, "%08x ", (unsigned) pkt_info->rss); + + if (pkt_info->ol_flags != 0) { + unsigned rxf; + const char *name; + + for (rxf = 0; rxf < sizeof(pkt_info->ol_flags) * 8; rxf++) { + if ((pkt_info->ol_flags & (1ULL << rxf)) == 0) + continue; + name = rte_get_rx_ol_flag_name(1ULL << rxf); + if (name == NULL) + continue; + wprintw(win_list, "%s ", name); + } + } + wclrtoeol(win_list); + } + + wclrtobot(win_list); + wnoutrefresh(win_list); +} + +static void +ui_repaint(void) +{ + switch (ui_mode) { + default: + win_header_draw(selected_port_id); + wnoutrefresh(win_header); + + if (_do_repaint_list) { + win_list_draw(); + _do_repaint_list = 0; + } + + break; + } + + win_footer_draw(NULL); + _do_repaint = 0; +} + +static void +ui_mode_set(int mode) +{ + ui_mode = (mode + 4) % 4; + switch (ui_mode) { + case UI_MODE_PORT: + ui_status_text = "Port: [p]promiscuous; [c]clear stats"; + break; + case UI_MODE_RSS_FN: + ui_status_text = "RSS Fn: [1-9]toggle fn"; + break; + case UI_MODE_RETA: + ui_status_text = + "RETA: [d]set default; [r]randomize; [0-9]fill table with value"; + break; + case UI_MODE_RSS_KEY: + ui_status_text = "RSS Key: [r]randomize; [1-4]select predefined key"; + break; + } + _do_repaint = 1; +} + +static void +ui_configure(void) +{ + int win_reta_cols = (16 * 2 + 1); + win_reta_lines = selected_port->dev_info.reta_size / 16; + win_stats_lines = selected_port->nb_rx_queues + 1; + win_header_lines = win_reta_lines; + if (win_header_lines < win_stats_lines) + win_header_lines = win_stats_lines; + win_header_lines += 5; + win_list_lines = scr_lines - win_header_lines - win_err_lines - 1; + + if (win_footer == NULL) { + win_footer = newwin(1, scr_cols, scr_lines - 1, 0); + ui_status_text = ""; + wbkgdset(win_footer, COLOR_PAIR(1)); + } else { + wresize(win_footer, 1, scr_cols); + mvwin(win_footer, scr_lines - 1, 0); + } + + if (win_err == NULL) { + win_err = newwin(1, scr_cols, scr_lines - 1 - win_err_lines, 0); + ui_status_text = ""; + wbkgdset(win_err, COLOR_PAIR(5)); + } else { + wresize(win_err, win_err_lines, scr_cols); + mvwin(win_err, scr_lines - win_err_lines - 1, 0); + } + + + if (win_header == NULL) { + win_header = newwin(win_header_lines, scr_cols, 0, 0); + wbkgdset(win_header, COLOR_PAIR(1)); + } else + wresize(win_header, win_header_lines, scr_cols); + + if (win_stats == NULL) + win_stats = subwin(win_header, win_stats_lines, + scr_cols - win_reta_cols - 7, 4, win_reta_cols + 7); + else + wresize(win_stats, win_stats_lines, scr_cols - win_reta_cols - 7); + + if (win_reta == NULL) + win_reta = subwin(win_header, win_reta_lines, win_reta_cols, + 4, 5); + else + wresize(win_reta, win_reta_lines, win_reta_cols); + + if (win_list == NULL) + win_list = newwin(win_list_lines, scr_cols, win_header_lines, 0); + else { + wresize(win_list, win_list_lines, scr_cols); + mvwin(win_list, win_header_lines, 0); + } + + wclear(win_header); + wclear(win_reta); + + _do_configure = 0; + _do_repaint = 1; + _do_repaint_list = 1; + _do_repaint_stats = 1; + _do_repaint_info = 1; + _do_repaint_err = 1; + +} + +static void +ui_resize(void) +{ + if (COLS != scr_cols || LINES != scr_lines) { + scr_cols = COLS; + scr_lines = LINES; + _do_configure = 1; + } +} + +static void +ui_update(void) +{ + if (ui_lock(1)) { + if (_is_ready) { + if (_do_configure) + ui_configure(); + if (_do_repaint) { + ui_repaint(); + doupdate(); + } + } + ui_unlock(); + } +} + +static void +ui_fini(void) +{ + win_err_fini(); + endwin(); +} + +static void +ui_init(void) +{ + initscr(); + start_color(); + init_pair(1, COLOR_YELLOW, COLOR_BLUE); + init_pair(2, COLOR_BLACK, COLOR_CYAN); + init_pair(3, COLOR_BLACK, COLOR_YELLOW); + init_pair(4, COLOR_BLACK, COLOR_WHITE); + init_pair(5, COLOR_YELLOW, COLOR_RED); + + cbreak(); + noecho(); + curs_set(FALSE); + keypad(stdscr, TRUE); + timeout(100); + + scr_cols = COLS; + scr_lines = LINES; + _do_configure = 1; + _is_ready = 1; +} + +static int +port_select(uint8_t port_id) +{ + if (ports[port_id].enabled == 1) { + focused_port_id = selected_port_id = port_id; + + selected_port = &ports[selected_port_id]; + focused_port = &ports[focused_port_id]; + + update_port_info(selected_port_id); + + _do_configure = 1; + _do_repaint = 1; + _do_repaint_info = 1; + _do_repaint_list = 1; + _do_repaint_stats = 1; + return selected_port_id; + } + return -1; +} + +/** + * Record packet from mbuf + */ +struct pkt_info * +record_pkt(uint8_t port_id, int queue_id, struct rte_mbuf *mb) +{ + struct rte_port *port = &ports[port_id]; + struct pkt_record *record = &(port->record); + struct pkt_info *pkt_info = &(record->pkt_info[record->top]); + + ui_lock(1); + + record->count++; + pkt_info->n = mb->udata64; + pkt_info->ol_flags = mb->ol_flags; + pkt_info->queue_id = queue_id; + pkt_info->rss = mb->hash.rss; + + record->top = (record->top + 1) % PKT_RECORD_SIZE; + + _do_repaint = 1; + _do_repaint_list = 1; + _do_repaint_stats = 1; + ui_unlock(); + + return pkt_info; +} + +static void +ui_help(void) { + timeout(-1); + + WINDOW *win_help_border = NULL; + WINDOW *win_help = NULL; + char title[256]; + int page = 0; + int page_prev; + int page_next; + int c; + int cols; + int lines; + int top; + int left; + + endwin(); + initscr(); + ui_resize(); + ui_update(); + + cols = scr_cols > 80 ? 80 : scr_cols; + lines = scr_lines > 25 ? 25 : scr_lines; + top = (scr_lines - lines) / 2; + left = (scr_cols - cols) / 2; + + win_help_border = newwin(lines, cols, top, left); + win_help = derwin(win_help_border, lines - 2, cols - 4, 1, 2); + wbkgdset(win_help_border, COLOR_PAIR(2)); + wbkgdset(win_help, COLOR_PAIR(2)); + + do { + page_prev = (page + RTE_DIM(help_pages) - 1) % RTE_DIM(help_pages); + page_next = (page + 1) % RTE_DIM(help_pages); + + wclear(win_help_border); + box(win_help_border, 0 , 0); + + sprintf(title, "[ Help: %s ]", help_pages[page].title); + mvwprintw(win_help_border, 0, (cols - strlen(title)) / 2 - 2, "%s", + title); + + sprintf(title, "< %s ]", help_pages[page_prev].title); + mvwprintw(win_help_border, lines - 1, 1, "%s", title); + + sprintf(title, "[ Page %d of %d ]", page + 1, (int)RTE_DIM(help_pages)); + mvwprintw(win_help_border, lines - 1, (cols - strlen(title)) / 2 - 2, + "%s", title); + + sprintf(title, "[ %s >", help_pages[page_next].title); + mvwprintw(win_help_border, lines - 1, cols - strlen(title) - 1, "%s", + title); + + wrefresh(win_help_border); + + wclear(win_help); + + wmove(win_help, 0, 0); + + wprintw(win_help, + help_pages[page].body + ); + + wrefresh(win_help); + + switch (c = getch()) { + + case KEY_RESIZE: + endwin(); + initscr(); + ui_resize(); + ui_update(); + + cols = scr_cols > 80 ? 80 : scr_cols; + lines = scr_lines > 25 ? 25 : scr_cols; + top = (scr_lines - lines) / 2; + left = (scr_cols - cols) / 2; + + wresize(win_help_border, lines, cols); + wresize(win_help, lines - 2, cols - 4); + break; + + case KEY_LEFT: + page = page_prev; + break; + + case KEY_RIGHT: + page = page_next; + break; + + default: + c = -1; /* Exit */ + break; + } + + + } while (c != -1); + + timeout(100); + delwin(win_help); + delwin(win_help_border); + _do_configure = 1; + _do_repaint = 1; +} + +/* main processing loop */ +void +ui_loop(void) +{ + int c; + int i; + int port_id; + + ui_init(); + port_select(bond_port_id); + + ui_mode_set(UI_MODE_PORT); + + while ((c = getch()) != 'q') { + switch (c) { + case -1: + ui_update(); + break; + + case 'c': + /* clear stats */ + ui_lock(1); + + rte_atomic64_clear(&counter); + for (port_id = 0; port_id < RTE_MAX_ETHPORTS; port_id++) { + if (ports[port_id].enabled) { + ports[port_id].record.count = 0; + ports[port_id].record.top = 0; + for (i = 0; i < ports[port_id].nb_rx_queues; i++) + ports[port_id].rxq_ipackets[i] = 0; + rte_eth_stats_reset(port_id); + } + } + + _do_repaint = 1; + ui_unlock(); + break; + + case ' ': + focused_port->rss ^= 1; + rss_enable(focused_port_id, focused_port->rss); + _do_configure = 1; + _do_repaint = 1; + break; + + case 'p': + if (focused_port->promiscuous) + rte_eth_promiscuous_disable(focused_port_id); + else + rte_eth_promiscuous_enable(focused_port_id); + focused_port->promiscuous ^= 1; + _do_configure = 1; + _do_repaint = 1; + break; + + case KEY_RESIZE: + ui_resize(); + break; + + case KEY_UP: + ui_mode_set(ui_mode - 1); + break; + case KEY_DOWN: + ui_mode_set(ui_mode + 1); + break; + case 'h': + case '?': + ui_help(); + break; + + default: + switch (ui_mode) { + case UI_MODE_PORT: + switch (c) { + case KEY_LEFT: + for (i = 1; i < nb_ports; i++) + if (port_select( + (selected_port_id - i + nb_ports) % nb_ports) + != -1) + break; + break; + + case KEY_RIGHT: + for (i = 1; i < nb_ports; i++) + if (port_select( + (selected_port_id + i + nb_ports) % nb_ports) + != -1) + break; + break; + + default: + i = (c - '1'); + port_select(i); + break; + } + break; + + case UI_MODE_RSS_KEY: + switch (c) { + case 'r': + i = key_set_random(focused_port_id, 40); + if (i < 0) + win_err_addline("Cannot update RSS key"); + break; + + default: + c = c - '1'; + if (c >= 0 && c < (int) RTE_DIM(RSS_KEY)) { + i = key_set(focused_port_id, RSS_KEY[c], 40); + if (i < 0) + win_err_addline("Cannot update RSS key"); + } + break; + } + update_port_info(focused_port_id); + break; + + case UI_MODE_RETA: + switch (c) { + case 'r': + reta_set_random(focused_port_id); + break; + case 'd': + reta_set_default(focused_port_id); + break; + default: + c = c - '0'; + reta_set_all(focused_port_id, c); + break; + } + break; + + case UI_MODE_RSS_FN: + switch (c) { + case 'a': + /* Set all */ + focused_port->dev_conf.rx_adv_conf.rss_conf.rss_hf = + ports[focused_port_id].dev_info.flow_type_rss_offloads; + rte_eth_dev_rss_hash_update(focused_port_id, + &focused_port->dev_conf.rx_adv_conf.rss_conf); + break; + + default: + c -= '1'; + if (c >= 0 && c < focused_port->rss_type_count) { + uint64_t rss_type = 0; + + focused_port->rss_type_fn[c].enabled ^= 1; /* toggle */ + + for (i = 0; i < focused_port->rss_type_count; i++) + if (focused_port->rss_type_fn[i].enabled) + rss_type |= + focused_port->rss_type_fn[i].info->rss_type; + focused_port->dev_conf.rx_adv_conf.rss_conf.rss_hf = + rss_type; + + rte_eth_dev_rss_hash_update(focused_port_id, + &focused_port->dev_conf.rx_adv_conf.rss_conf); + + } + } + break; + + } + _do_repaint = 1; + break; + } + } + ui_fini(); +} -- 1.7.9.5 ^ permalink raw reply [flat|nested] 126+ messages in thread
* [dpdk-dev] [PATCH 7/8] doc: fixed spellings and typos 2015-06-03 10:58 [dpdk-dev] [PATCH 0/8] Dynamic RSS Configuration for Bonding Tomasz Kulasek ` (5 preceding siblings ...) 2015-06-03 10:59 ` [dpdk-dev] [PATCH 6/8] examples: dynamic rss configuration for bonding Tomasz Kulasek @ 2015-06-03 10:59 ` Tomasz Kulasek 2015-06-03 10:59 ` [dpdk-dev] [PATCH 8/8] doc: dynamic rss configuration for bonding Tomasz Kulasek ` (2 subsequent siblings) 9 siblings, 0 replies; 126+ messages in thread From: Tomasz Kulasek @ 2015-06-03 10:59 UTC (permalink / raw) To: dev Signed-off-by: Tomasz Kulasek <tomaszx.kulasek@intel.com> --- .../prog_guide/link_bonding_poll_mode_drv_lib.rst | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/doc/guides/prog_guide/link_bonding_poll_mode_drv_lib.rst b/doc/guides/prog_guide/link_bonding_poll_mode_drv_lib.rst index 96e554f..03baf90 100644 --- a/doc/guides/prog_guide/link_bonding_poll_mode_drv_lib.rst +++ b/doc/guides/prog_guide/link_bonding_poll_mode_drv_lib.rst @@ -188,7 +188,7 @@ conditions are not met. If a user wishes to monitor individual slaves then they must register callbacks with that slave directly. The link bonding library also supports devices which do not implement link -status change interrupts, this is achieve by polling the devices link status at +status change interrupts, this is achieved by polling the devices link status at a defined period which is set using the ``rte_eth_bond_link_monitoring_set`` API, the default polling interval is 10ms. When a device is added as a slave to a bonding device it is determined using the ``RTE_PCI_DRV_INTR_LSC`` flag @@ -286,7 +286,7 @@ and UDP protocols for load balancing. Using Link Bonding Devices -------------------------- -The librte_pmd_bond library support two modes of device creation, the libraries +The librte_pmd_bond library supports two modes of device creation, the libraries export full C API or using the EAL command line to statically configure link bonding devices at application startup. Using the EAL option it is possible to use link bonding functionality transparently without specific knowledge of the @@ -299,7 +299,7 @@ Using the Poll Mode Driver from an Application Using the librte_pmd_bond libraries API it is possible to dynamically create and manage link bonding device from within any application. Link bonding -device are created using the ``rte_eth_bond_create`` API which requires a +devices are created using the ``rte_eth_bond_create`` API which requires a unique device name, the link bonding mode to initial the device in and finally the socket Id which to allocate the devices resources onto. After successful creation of a bonding device it must be configured using the generic Ethernet @@ -362,7 +362,7 @@ The different options are: mode=2 * slave: Defines the PMD device which will be added as slave to the bonded - device. This option can be selected multiple time, for each device to be + device. This option can be selected multiple times, for each device to be added as a slave. Physical devices should be specified using their PCI address, in the format domain:bus:devid.function -- 1.7.9.5 ^ permalink raw reply [flat|nested] 126+ messages in thread
* [dpdk-dev] [PATCH 8/8] doc: dynamic rss configuration for bonding 2015-06-03 10:58 [dpdk-dev] [PATCH 0/8] Dynamic RSS Configuration for Bonding Tomasz Kulasek ` (6 preceding siblings ...) 2015-06-03 10:59 ` [dpdk-dev] [PATCH 7/8] doc: fixed spellings and typos Tomasz Kulasek @ 2015-06-03 10:59 ` Tomasz Kulasek 2015-06-12 5:36 ` [dpdk-dev] [PATCH 0/8] Dynamic RSS Configuration for Bonding Xu, HuilongX 2015-06-19 14:13 ` [dpdk-dev] [PATCH v2 " Tomasz Kulasek 9 siblings, 0 replies; 126+ messages in thread From: Tomasz Kulasek @ 2015-06-03 10:59 UTC (permalink / raw) To: dev Documentation update about implementation details and requirements for Dynamic RSS Configuration for Bonding. Signed-off-by: Tomasz Kulasek <tomaszx.kulasek@intel.com> --- .../prog_guide/link_bonding_poll_mode_drv_lib.rst | 32 ++++++++++++++++++-- 1 file changed, 30 insertions(+), 2 deletions(-) diff --git a/doc/guides/prog_guide/link_bonding_poll_mode_drv_lib.rst b/doc/guides/prog_guide/link_bonding_poll_mode_drv_lib.rst index 03baf90..7f06cbe 100644 --- a/doc/guides/prog_guide/link_bonding_poll_mode_drv_lib.rst +++ b/doc/guides/prog_guide/link_bonding_poll_mode_drv_lib.rst @@ -1,5 +1,5 @@ .. BSD LICENSE - Copyright(c) 2010-2014 Intel Corporation. All rights reserved. + Copyright(c) 2010-2015 Intel Corporation. All rights reserved. All rights reserved. Redistribution and use in source and binary forms, with or without @@ -173,7 +173,27 @@ After a slave device is added to a bonded device slave is stopped using ``rte_eth_dev_stop`` and then reconfigured using ``rte_eth_dev_configure`` the RX and TX queues are also reconfigured using ``rte_eth_tx_queue_setup`` / ``rte_eth_rx_queue_setup`` with the parameters use to configure the bonding -device. +device. If RSS is enabled for bonding device, this mode is also enabled on new +slave and configured as well. + +Setting up multi-queue mode for bonding device to RSS, makes it fully +RSS-capable, so all slaves are synchronized with its configuration. This mode is +intended to provide RSS configuration on slaves transparent for client +application implementation. + +Bonding device stores its own version of RSS settings i.e. RETA, RSS hash +function and RSS key, used to set up its slaves. That let to define the meaning +of RSS configuration of bonding device as desired configuration of whole bonding +(as one unit), without pointing any of slave inside. It is required to ensure +consistency and made it more errorproof. + +RSS hash function set for bonding device, is a maximal set of RSS hash functions +supported by all bonded slaves. RETA size is a GCD of all its RETA's sizes, so +it can be easily used as a pattern providing expected behavior, even if slave +RETAs' sizes are different. RSS key is always 40 bytes long. + +All settings are managed through the bonding port API and always are propagated +in one direction (from bonding to slaves). Link Status Change Interrupts / Polling ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ @@ -207,6 +227,14 @@ these parameters. A bonding device must have a minimum of one slave before the bonding device itself can be started. +To use a bonding device dynamic RSS configuration feature effectively, it is +also required, that all slaves should be RSS-capable and support, at least one +common hash function available for each of them. + +To prevent inconsistency on how slaves process packets, once a device is added +to a bonding device, RSS configuration should be managed through the bonding +device API, and not directly on the slave. + Like all other PMD, all functions exported by a PMD are lock-free functions that are assumed not to be invoked in parallel on different logical cores to work on the same target object. -- 1.7.9.5 ^ permalink raw reply [flat|nested] 126+ messages in thread
* Re: [dpdk-dev] [PATCH 0/8] Dynamic RSS Configuration for Bonding 2015-06-03 10:58 [dpdk-dev] [PATCH 0/8] Dynamic RSS Configuration for Bonding Tomasz Kulasek ` (7 preceding siblings ...) 2015-06-03 10:59 ` [dpdk-dev] [PATCH 8/8] doc: dynamic rss configuration for bonding Tomasz Kulasek @ 2015-06-12 5:36 ` Xu, HuilongX 2015-06-25 9:20 ` Kulasek, TomaszX 2015-06-19 14:13 ` [dpdk-dev] [PATCH v2 " Tomasz Kulasek 9 siblings, 1 reply; 126+ messages in thread From: Xu, HuilongX @ 2015-06-12 5:36 UTC (permalink / raw) To: Kulasek, TomaszX, dev Tested-by: huilong xu <huilongx.xu@intel.com> - Tested Commit: 1a1109404e702d3ad1ccc1033df55c59bec1f89a + PATCH - OS: Linux dpdk-fedora20 3.11.10-301.fc20.x86_64 - GCC: gcc version 4.8.3 20140624 (Red Hat 4.8.3-1) (GCC) - CPU: Intel(R) Xeon(R) CPU E5-2680 v2 @ 2.80GHz - NIC: Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection [8086:10fb] - NIC: Intel Corporation Ethernet Controller XL710 for 40GbE QSFP+ [8086:1583] - Default x86_64-native-linuxapp-gcc configuration - Total 4 cases, 2 passed, 2 failed. Niantic NIC case all passed, but Fortville NIC case all failed. First case: This is a unit test case, not need NIC 1. build dpdk driver and insmod driver 2. set 1024*2M hugepage 3. compile test app in app/test 4. run test ./test -c f -n 4 5. exec dynamic rss confif unit test link_bonding_rssconf_autotest 6. print "test ok" 7. this case passed Second case: This is a function test case, used Fortville NIC(8086:1583) 1. build dpdk driver and insmod driver 2. bind dpdk driver to Fortville nic 3. set 1024*2M hugepage 4. run test pmd ./x86_64-native-linuxapp-gcc/app/testpmd -c 0xff -n 4 -- -i --txqflags=0x2 --mbcache=128 --burst=32 --txfreet=32 --rxfreet=64 --rxq=4 --txq=4 5. exec testpmd cmd a) create bonded device 0 0 b) add bonding slave 0 3 c) add bonding slave 1 3 d) port start 3 port can start, and link stats is down, so this case failed. Thirdly case: This is a function test case, used Fortville NIC(8086:1583) 1. build dpdk driver and insmod driver 2. bind dpdk driver to Fortville nic 3. set 1024*2M hugepage 4. run test pmd ./x86_64-native-linuxapp-gcc/app/testpmd -c 0xff -n 4 -- -i --txqflags=0x2 --mbcache=128 --burst=32 --txfreet=32 --rxfreet=64 --rxq=4 --txq=4 5. exec testpmd cmd a) create bonded device 0 0 b) add bonding slave 0 3 c) add bonding slave 1 3 d) port config all rss ip e) show port 3 rss-hash printf: RSS functions: ipv4-frag ipv4-other ipv6-frag ipv6-other f) show port 0 rss-hash printf: RSS disabled Slave rss not enable, so this case failed Fourthly case: This is a function test case, used Niantic NIC(8086:10fb) 1. build dpdk driver and insmod driver 2. bind dpdk driver to Fortville nic 3. set 1024*2M hugepage 4. run test pmd ./x86_64-native-linuxapp-gcc/app/testpmd -c 0xff -n 4 -- -i --txqflags=0x2 --mbcache=128 --burst=32 --txfreet=32 --rxfreet=64 --rxq=4 --txq=4 5. exec testpmd cmd a) create bonded device 0 0 b) add bonding slave 0 3 c) add bonding slave 1 3 d) port start 3 e) port stop all f) set verbose 8 g) set fwd rxonly h) set stat_qmap 3 0 0 i) set stat_qmap 3 1 1 j) set stat_qmap 3 1 1 k) set stat_qmap 3 1 1 l) port config all rss ip m) port start all n)start 6. send 50 ip packages to salve 0 by ixia, the package config as below a) dst mac: bond device (port 3) mac address. b) src mac: 00:00:00:12:34:56 c) package type:0800 e) dst ip: 192.168.1.1 f) src ip: from 192.168.1.2 to 192.168.1.51, one package, this ip add 1 7. stop Port 3 queue 0 received 9 packages Port 3 queue 1 received 9 packages Port 3 queue 2 received 16 packages Port 3 queue 3 received 16 packages 8. send 50 ip packages to slave 1 by ixia, the package config as same 9. stop and check port 3 received packages again Form slave 0 and slave 1 the rss are some, the test passed > -----Original Message----- > From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Tomasz Kulasek > Sent: Wednesday, June 03, 2015 6:59 PM > To: dev@dpdk.org > Subject: [dpdk-dev] [PATCH 0/8] Dynamic RSS Configuration for Bonding > > OVERVIEW > -------- > 1) Setting .rxmode.mq_mode for bonding device to ETH_MQ_RX_RSS makes > bonding > device fully RSS-capable, so all slaves are synchronized with its > configuration. > This mode is intended to provide RSS configuration as known from "dynamic > RSS > configuration for one port" and made slaves transparent for client > application > implementation. > > 2) If .rxmode.mq_mode for bonding device isn't ETH_MQ_RX_RSS, slaves are > not > synchronized. That provides an ability to configure them manually. This > mode may > be useful when application wants to manage RSS in an unusual way and the > consistency of RSS configuration for slaves isn't required. > > Turning on/off RSS mode for slaves when bonding is started is not possible. > Other RSS configuration is propagated over slaves, when bonding device API > is > used to do it. > > Tomasz Kulasek (8): > bond: dynamic rss configuration > ring: dynamic rss configuration > test: dynamic rss configuration > bond: queue stats mapping > ring: queue stats mapping set dummy implementation > examples: dynamic rss configuration for bonding > doc: fixed spellings and typos > doc: dynamic rss configuration for bonding > > app/test/Makefile | 1 + > app/test/test_link_bonding_rssconf.c | 674 ++++++++++++++ > .../prog_guide/link_bonding_poll_mode_drv_lib.rst | 40 +- > drivers/net/bonding/rte_eth_bond_api.c | 22 + > drivers/net/bonding/rte_eth_bond_pmd.c | 222 ++++- > drivers/net/bonding/rte_eth_bond_private.h | 11 + > drivers/net/ring/rte_eth_ring.c | 133 ++- > examples/bond_rss/Makefile | 59 ++ > examples/bond_rss/bondrss.c | 293 +++++++ > examples/bond_rss/bondrss.h | 163 ++++ > examples/bond_rss/config.c | 251 ++++++ > examples/bond_rss/ui.c | 915 > ++++++++++++++++++++ > 12 files changed, 2759 insertions(+), 25 deletions(-) > create mode 100644 app/test/test_link_bonding_rssconf.c > create mode 100644 examples/bond_rss/Makefile > create mode 100644 examples/bond_rss/bondrss.c > create mode 100644 examples/bond_rss/bondrss.h > create mode 100644 examples/bond_rss/config.c > create mode 100644 examples/bond_rss/ui.c > > -- > 1.7.9.5 ^ permalink raw reply [flat|nested] 126+ messages in thread
* Re: [dpdk-dev] [PATCH 0/8] Dynamic RSS Configuration for Bonding 2015-06-12 5:36 ` [dpdk-dev] [PATCH 0/8] Dynamic RSS Configuration for Bonding Xu, HuilongX @ 2015-06-25 9:20 ` Kulasek, TomaszX 0 siblings, 0 replies; 126+ messages in thread From: Kulasek, TomaszX @ 2015-06-25 9:20 UTC (permalink / raw) To: Xu, HuilongX, dev There's a bug in bonding itself, which prevents the bonding, made of Fortville NICs, start and is not related to Dynamic RSS Configuration. This problem solves separate patch "bond: fix check initial link status of slave". -----Original Message----- From: Xu, HuilongX Sent: Friday, June 12, 2015 07:36 To: Kulasek, TomaszX; dev@dpdk.org Subject: RE: [dpdk-dev] [PATCH 0/8] Dynamic RSS Configuration for Bonding Tested-by: huilong xu <huilongx.xu@intel.com> - Tested Commit: 1a1109404e702d3ad1ccc1033df55c59bec1f89a + PATCH - OS: Linux dpdk-fedora20 3.11.10-301.fc20.x86_64 - GCC: gcc version 4.8.3 20140624 (Red Hat 4.8.3-1) (GCC) - CPU: Intel(R) Xeon(R) CPU E5-2680 v2 @ 2.80GHz - NIC: Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection [8086:10fb] - NIC: Intel Corporation Ethernet Controller XL710 for 40GbE QSFP+ [8086:1583] - Default x86_64-native-linuxapp-gcc configuration - Total 4 cases, 2 passed, 2 failed. Niantic NIC case all passed, but Fortville NIC case all failed. First case: This is a unit test case, not need NIC 1. build dpdk driver and insmod driver 2. set 1024*2M hugepage 3. compile test app in app/test 4. run test ./test -c f -n 4 5. exec dynamic rss confif unit test link_bonding_rssconf_autotest 6. print "test ok" 7. this case passed Second case: This is a function test case, used Fortville NIC(8086:1583) 1. build dpdk driver and insmod driver 2. bind dpdk driver to Fortville nic 3. set 1024*2M hugepage 4. run test pmd ./x86_64-native-linuxapp-gcc/app/testpmd -c 0xff -n 4 -- -i --txqflags=0x2 --mbcache=128 --burst=32 --txfreet=32 --rxfreet=64 --rxq=4 --txq=4 5. exec testpmd cmd a) create bonded device 0 0 b) add bonding slave 0 3 c) add bonding slave 1 3 d) port start 3 port can start, and link stats is down, so this case failed. Thirdly case: This is a function test case, used Fortville NIC(8086:1583) 1. build dpdk driver and insmod driver 2. bind dpdk driver to Fortville nic 3. set 1024*2M hugepage 4. run test pmd ./x86_64-native-linuxapp-gcc/app/testpmd -c 0xff -n 4 -- -i --txqflags=0x2 --mbcache=128 --burst=32 --txfreet=32 --rxfreet=64 --rxq=4 --txq=4 5. exec testpmd cmd a) create bonded device 0 0 b) add bonding slave 0 3 c) add bonding slave 1 3 d) port config all rss ip e) show port 3 rss-hash printf: RSS functions: ipv4-frag ipv4-other ipv6-frag ipv6-other f) show port 0 rss-hash printf: RSS disabled Slave rss not enable, so this case failed Fourthly case: This is a function test case, used Niantic NIC(8086:10fb) 1. build dpdk driver and insmod driver 2. bind dpdk driver to Fortville nic 3. set 1024*2M hugepage 4. run test pmd ./x86_64-native-linuxapp-gcc/app/testpmd -c 0xff -n 4 -- -i --txqflags=0x2 --mbcache=128 --burst=32 --txfreet=32 --rxfreet=64 --rxq=4 --txq=4 5. exec testpmd cmd a) create bonded device 0 0 b) add bonding slave 0 3 c) add bonding slave 1 3 d) port start 3 e) port stop all f) set verbose 8 g) set fwd rxonly h) set stat_qmap 3 0 0 i) set stat_qmap 3 1 1 j) set stat_qmap 3 1 1 k) set stat_qmap 3 1 1 l) port config all rss ip m) port start all n)start 6. send 50 ip packages to salve 0 by ixia, the package config as below a) dst mac: bond device (port 3) mac address. b) src mac: 00:00:00:12:34:56 c) package type:0800 e) dst ip: 192.168.1.1 f) src ip: from 192.168.1.2 to 192.168.1.51, one package, this ip add 1 7. stop Port 3 queue 0 received 9 packages Port 3 queue 1 received 9 packages Port 3 queue 2 received 16 packages Port 3 queue 3 received 16 packages 8. send 50 ip packages to slave 1 by ixia, the package config as same 9. stop and check port 3 received packages again Form slave 0 and slave 1 the rss are some, the test passed > -----Original Message----- > From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Tomasz Kulasek > Sent: Wednesday, June 03, 2015 6:59 PM > To: dev@dpdk.org > Subject: [dpdk-dev] [PATCH 0/8] Dynamic RSS Configuration for Bonding > > OVERVIEW > -------- > 1) Setting .rxmode.mq_mode for bonding device to ETH_MQ_RX_RSS makes > bonding device fully RSS-capable, so all slaves are synchronized with > its configuration. > This mode is intended to provide RSS configuration as known from > "dynamic RSS configuration for one port" and made slaves transparent > for client application implementation. > > 2) If .rxmode.mq_mode for bonding device isn't ETH_MQ_RX_RSS, slaves > are not synchronized. That provides an ability to configure them > manually. This mode may be useful when application wants to manage RSS > in an unusual way and the consistency of RSS configuration for slaves > isn't required. > > Turning on/off RSS mode for slaves when bonding is started is not possible. > Other RSS configuration is propagated over slaves, when bonding device > API is used to do it. > > Tomasz Kulasek (8): > bond: dynamic rss configuration > ring: dynamic rss configuration > test: dynamic rss configuration > bond: queue stats mapping > ring: queue stats mapping set dummy implementation > examples: dynamic rss configuration for bonding > doc: fixed spellings and typos > doc: dynamic rss configuration for bonding > > app/test/Makefile | 1 + > app/test/test_link_bonding_rssconf.c | 674 ++++++++++++++ > .../prog_guide/link_bonding_poll_mode_drv_lib.rst | 40 +- > drivers/net/bonding/rte_eth_bond_api.c | 22 + > drivers/net/bonding/rte_eth_bond_pmd.c | 222 ++++- > drivers/net/bonding/rte_eth_bond_private.h | 11 + > drivers/net/ring/rte_eth_ring.c | 133 ++- > examples/bond_rss/Makefile | 59 ++ > examples/bond_rss/bondrss.c | 293 +++++++ > examples/bond_rss/bondrss.h | 163 ++++ > examples/bond_rss/config.c | 251 ++++++ > examples/bond_rss/ui.c | 915 > ++++++++++++++++++++ > 12 files changed, 2759 insertions(+), 25 deletions(-) create mode > 100644 app/test/test_link_bonding_rssconf.c > create mode 100644 examples/bond_rss/Makefile create mode 100644 > examples/bond_rss/bondrss.c create mode 100644 > examples/bond_rss/bondrss.h create mode 100644 > examples/bond_rss/config.c create mode 100644 examples/bond_rss/ui.c > > -- > 1.7.9.5 ^ permalink raw reply [flat|nested] 126+ messages in thread
* [dpdk-dev] [PATCH v2 0/8] Dynamic RSS Configuration for Bonding 2015-06-03 10:58 [dpdk-dev] [PATCH 0/8] Dynamic RSS Configuration for Bonding Tomasz Kulasek ` (8 preceding siblings ...) 2015-06-12 5:36 ` [dpdk-dev] [PATCH 0/8] Dynamic RSS Configuration for Bonding Xu, HuilongX @ 2015-06-19 14:13 ` Tomasz Kulasek 2015-06-19 14:13 ` [dpdk-dev] [PATCH v2 1/8] bond: rss dynamic configuration Tomasz Kulasek ` (9 more replies) 9 siblings, 10 replies; 126+ messages in thread From: Tomasz Kulasek @ 2015-06-19 14:13 UTC (permalink / raw) To: dev OVERVIEW -------- 1) Setting .rxmode.mq_mode for bonding device to ETH_MQ_RX_RSS makes bonding device fully RSS-capable, so all slaves are synchronized with its configuration. This mode is intended to provide RSS configuration as known from "dynamic RSS configuration for one port" and made slaves transparent for client application implementation. 2) If .rxmode.mq_mode for bonding device isn't ETH_MQ_RX_RSS, slaves are not synchronized. That provides an ability to configure them manually. This mode may be useful when application wants to manage RSS in an unusual way and the consistency of RSS configuration for slaves isn't required. Turning on/off RSS mode for slaves when bonding is started is not possible. Other RSS configuration is propagated over slaves, when bonding device API is used to do it. v2 changes: - added support for keys other than 40 bytes long, - now, if RSS key is not set for bonding, it is not set also for slaves, - fix - full initial RSS configuration before any slave is added was not possible due to the initially zeroed flow_type_rss_offloads for bonding, - fix - changed error to warning when slave is synchronizing due to the bonding's initial configuration (to allow use slaves' drivers not supporting dynamic RSS configuration in bonding), - some code cleanups, - updated documentation, Tomasz Kulasek (8): bond: rss dynamic configuration ring: dynamic rss configuration test: dynamic rss configuration bond: queue stats mapping ring: queue stats mapping set dummy implementation examples: dynamic rss configuration for bonding doc: fixed spellings and typos doc: dynamic rss configuration for bonding app/test/Makefile | 1 + app/test/test_link_bonding_rssconf.c | 674 ++++++++++++++ .../prog_guide/link_bonding_poll_mode_drv_lib.rst | 42 +- drivers/net/bonding/rte_eth_bond_api.c | 27 + drivers/net/bonding/rte_eth_bond_pmd.c | 239 ++++- drivers/net/bonding/rte_eth_bond_private.h | 12 + drivers/net/ring/rte_eth_ring.c | 133 ++- examples/bond_rss/Makefile | 59 ++ examples/bond_rss/bondrss.c | 293 +++++++ examples/bond_rss/bondrss.h | 163 ++++ examples/bond_rss/config.c | 251 ++++++ examples/bond_rss/ui.c | 920 ++++++++++++++++++++ 12 files changed, 2789 insertions(+), 25 deletions(-) create mode 100644 app/test/test_link_bonding_rssconf.c create mode 100644 examples/bond_rss/Makefile create mode 100644 examples/bond_rss/bondrss.c create mode 100644 examples/bond_rss/bondrss.h create mode 100644 examples/bond_rss/config.c create mode 100644 examples/bond_rss/ui.c -- 1.7.9.5 ^ permalink raw reply [flat|nested] 126+ messages in thread
* [dpdk-dev] [PATCH v2 1/8] bond: rss dynamic configuration 2015-06-19 14:13 ` [dpdk-dev] [PATCH v2 " Tomasz Kulasek @ 2015-06-19 14:13 ` Tomasz Kulasek 2015-06-19 14:13 ` [dpdk-dev] [PATCH v2 2/8] ring: dynamic rss configuration Tomasz Kulasek ` (8 subsequent siblings) 9 siblings, 0 replies; 126+ messages in thread From: Tomasz Kulasek @ 2015-06-19 14:13 UTC (permalink / raw) To: dev Bonding device implements independent management of RSS settings. It stores its own copies of settings i.e. RETA, RSS hash function and RSS key. It’s required to ensure consistency. 1) RSS hash function set for bonding device is maximal set of RSS hash functions supported by all bonded devices. That mean, to have RSS support for bonding, all slaves should be RSS-capable. 2) RSS key is propagated over the slaves "as is". 3) RETA for bonding is an internal table managed by bonding API, and is used as a pattern to set up slaves. Its size is GCD of all RETA sizes, so it can be easily used as a pattern providing expected behavior, even if slaves RETA sizes are different. Signed-off-by: Tomasz Kulasek <tomaszx.kulasek@intel.com> --- drivers/net/bonding/rte_eth_bond_api.c | 27 ++++ drivers/net/bonding/rte_eth_bond_pmd.c | 209 ++++++++++++++++++++++++++-- drivers/net/bonding/rte_eth_bond_private.h | 12 ++ 3 files changed, 234 insertions(+), 14 deletions(-) diff --git a/drivers/net/bonding/rte_eth_bond_api.c b/drivers/net/bonding/rte_eth_bond_api.c index d4caa83..1d8083d 100644 --- a/drivers/net/bonding/rte_eth_bond_api.c +++ b/drivers/net/bonding/rte_eth_bond_api.c @@ -302,6 +302,9 @@ rte_eth_bond_create(const char *name, uint8_t mode, uint8_t socket_id) internals->rx_offload_capa = 0; internals->tx_offload_capa = 0; + /* Initially allow to choose any offload type */ + internals->flow_type_rss_offloads = ETH_RSS_PROTO_MASK; + memset(internals->active_slaves, 0, sizeof(internals->active_slaves)); memset(internals->slaves, 0, sizeof(internals->slaves)); @@ -365,6 +368,11 @@ __eth_bond_slave_add_lock_free(uint8_t bonded_port_id, uint8_t slave_port_id) rte_eth_dev_info_get(slave_port_id, &dev_info); + /* We need to store slaves reta_size to be able to synchronize RETA for all + * slave devices even if its sizes are different. + */ + internals->slaves[internals->slave_count].reta_size = dev_info.reta_size; + if (internals->slave_count < 1) { /* if MAC is not user defined then use MAC of first slave add to * bonded device */ @@ -378,9 +386,16 @@ __eth_bond_slave_add_lock_free(uint8_t bonded_port_id, uint8_t slave_port_id) /* Make primary slave */ internals->primary_port = slave_port_id; + /* Inherit queues settings from first slave */ + internals->nb_rx_queues = slave_eth_dev->data->nb_rx_queues; + internals->nb_tx_queues = slave_eth_dev->data->nb_tx_queues; + + internals->reta_size = dev_info.reta_size; + /* Take the first dev's offload capabilities */ internals->rx_offload_capa = dev_info.rx_offload_capa; internals->tx_offload_capa = dev_info.tx_offload_capa; + internals->flow_type_rss_offloads = dev_info.flow_type_rss_offloads; } else { /* Check slave link properties are supported if props are set, @@ -399,8 +414,18 @@ __eth_bond_slave_add_lock_free(uint8_t bonded_port_id, uint8_t slave_port_id) } internals->rx_offload_capa &= dev_info.rx_offload_capa; internals->tx_offload_capa &= dev_info.tx_offload_capa; + internals->flow_type_rss_offloads &= dev_info.flow_type_rss_offloads; + + /* RETA size is GCD of all slaves RETA sizes, + * so, if all sizes will be the power of 2, the lower one is GCD */ + if (internals->reta_size > dev_info.reta_size) + internals->reta_size = dev_info.reta_size; + } + bonded_eth_dev->data->dev_conf.rx_adv_conf.rss_conf.rss_hf &= + internals->flow_type_rss_offloads; + internals->slave_count++; /* Update all slave devices MACs*/ @@ -527,6 +552,8 @@ __eth_bond_slave_remove_lock_free(uint8_t bonded_port_id, uint8_t slave_port_id) if (internals->slave_count == 0) { internals->rx_offload_capa = 0; internals->tx_offload_capa = 0; + internals->flow_type_rss_offloads = ETH_RSS_PROTO_MASK; + internals->reta_size = 0; } return 0; } diff --git a/drivers/net/bonding/rte_eth_bond_pmd.c b/drivers/net/bonding/rte_eth_bond_pmd.c index 8bad2e1..5de2f30 100644 --- a/drivers/net/bonding/rte_eth_bond_pmd.c +++ b/drivers/net/bonding/rte_eth_bond_pmd.c @@ -1306,6 +1306,22 @@ slave_configure(struct rte_eth_dev *bonded_eth_dev, if (slave_eth_dev->driver->pci_drv.drv_flags & RTE_PCI_DRV_INTR_LSC) slave_eth_dev->data->dev_conf.intr_conf.lsc = 1; + /* If RSS is enabled for bonding, try to enable it for slaves */ + if (bonded_eth_dev->data->dev_conf.rxmode.mq_mode & ETH_MQ_RX_RSS) { + if (bonded_eth_dev->data->dev_conf.rx_adv_conf.rss_conf.rss_key_len != 0) { + slave_eth_dev->data->dev_conf.rx_adv_conf.rss_conf.rss_key_len = + bonded_eth_dev->data->dev_conf.rx_adv_conf.rss_conf.rss_key_len; + slave_eth_dev->data->dev_conf.rx_adv_conf.rss_conf.rss_key = + bonded_eth_dev->data->dev_conf.rx_adv_conf.rss_conf.rss_key; + } + else + slave_eth_dev->data->dev_conf.rx_adv_conf.rss_conf.rss_key = NULL; + + slave_eth_dev->data->dev_conf.rx_adv_conf.rss_conf.rss_hf = + bonded_eth_dev->data->dev_conf.rx_adv_conf.rss_conf.rss_hf; + slave_eth_dev->data->dev_conf.rxmode.mq_mode |= ETH_MQ_RX_RSS; + } + /* Configure device */ errval = rte_eth_dev_configure(slave_eth_dev->data->port_id, bonded_eth_dev->data->nb_rx_queues, @@ -1357,6 +1373,31 @@ slave_configure(struct rte_eth_dev *bonded_eth_dev, return -1; } + /* If RSS is enabled for bonding, synchronize RETA */ + if (bonded_eth_dev->data->dev_conf.rxmode.mq_mode & ETH_MQ_RX_RSS) { + int i; + struct bond_dev_private *internals; + internals = bonded_eth_dev->data->dev_private; + + for (i = 0; i < internals->slave_count; i++) { + if (internals->slaves[i].port_id == slave_eth_dev->data->port_id) { + errval = rte_eth_dev_rss_reta_update( + slave_eth_dev->data->port_id, + &(internals->reta_conf[0]), + internals->slaves[i].reta_size); + if (errval != 0) { + RTE_LOG(WARNING, PMD, + "Can't update slave's RSS configuration " + "(rte_eth_dev_rss_reta_update on slave port fails: " + "port=%d, err %d). RSS Configuration for bonding " + "may be inconsistent.\n", + slave_eth_dev->data->port_id, errval); + } + break; + } + } + } + return 0; } @@ -1569,6 +1610,9 @@ bond_ethdev_info(struct rte_eth_dev *dev, struct rte_eth_dev_info *dev_info) dev_info->rx_offload_capa = internals->rx_offload_capa; dev_info->tx_offload_capa = internals->tx_offload_capa; + dev_info->flow_type_rss_offloads = internals->flow_type_rss_offloads; + + dev_info->reta_size = internals->reta_size; } static int @@ -1950,21 +1994,134 @@ bond_ethdev_lsc_event_callback(uint8_t port_id, enum rte_eth_event_type type, } } +static int +bond_ethdev_rss_reta_update(struct rte_eth_dev *dev, + struct rte_eth_rss_reta_entry64 *reta_conf, + uint16_t reta_size) +{ + unsigned i, j; + int result = 0; + int slave_reta_size; + unsigned reta_count; + struct bond_dev_private *internals = dev->data->dev_private; + + if (reta_size != internals->reta_size) + return -EINVAL; + + /* Copy RETA table */ + reta_count = reta_size / RTE_RETA_GROUP_SIZE; + + for (i = 0; i < reta_count; i++) { + internals->reta_conf[i].mask = reta_conf[i].mask; + for (j = 0; j < RTE_RETA_GROUP_SIZE; j++) + if ((reta_conf[i].mask >> j) & 0x01) + internals->reta_conf[i].reta[j] = reta_conf[i].reta[j]; + } + + /* Fill rest of array */ + for (; i < RTE_DIM(internals->reta_conf); i += reta_count) + memcpy(&(internals->reta_conf[i]), &(internals->reta_conf[0]), + sizeof(internals->reta_conf[0]) * reta_count); + + /* Propagate RETA over slaves */ + for (i = 0; i < internals->slave_count; i++) { + slave_reta_size = internals->slaves[i].reta_size; + result = rte_eth_dev_rss_reta_update(internals->slaves[i].port_id, + &(internals->reta_conf[0]), slave_reta_size); + if (result < 0) + return result; + } + + return 0; +} + +static int +bond_ethdev_rss_reta_query(struct rte_eth_dev *dev, + struct rte_eth_rss_reta_entry64 *reta_conf, + uint16_t reta_size) +{ + int i, j; + struct bond_dev_private *internals = dev->data->dev_private; + + if (reta_size != internals->reta_size) + return -EINVAL; + + /* Copy RETA table */ + for (i = 0; i < reta_size / RTE_RETA_GROUP_SIZE; i++) + for (j = 0; j < RTE_RETA_GROUP_SIZE; j++) + if ((reta_conf[i].mask >> j) & 0x01) + reta_conf[i].reta[j] = internals->reta_conf[i].reta[j]; + + return 0; +} + +static int +bond_ethdev_rss_hash_update(struct rte_eth_dev *dev, + struct rte_eth_rss_conf *rss_conf) +{ + int i, result = 0; + struct bond_dev_private *internals = dev->data->dev_private; + struct rte_eth_rss_conf bond_rss_conf; + + memcpy(&bond_rss_conf, rss_conf, sizeof(struct rte_eth_rss_conf)); + + bond_rss_conf.rss_hf &= internals->flow_type_rss_offloads; + + if (bond_rss_conf.rss_hf != 0) + dev->data->dev_conf.rx_adv_conf.rss_conf.rss_hf = bond_rss_conf.rss_hf; + + if (bond_rss_conf.rss_key != NULL && bond_rss_conf.rss_key_len < + sizeof(internals->rss_key)) { + if (bond_rss_conf.rss_key_len == 0) + bond_rss_conf.rss_key_len = 40; + internals->rss_key_len = bond_rss_conf.rss_key_len; + memcpy(internals->rss_key, bond_rss_conf.rss_key, + internals->rss_key_len); + } + + for (i = 0; i < internals->slave_count; i++) { + result = rte_eth_dev_rss_hash_update(internals->slaves[i].port_id, + &bond_rss_conf); + if (result < 0) + return result; + } + + return 0; +} + +static int +bond_ethdev_rss_hash_conf_get(struct rte_eth_dev *dev, + struct rte_eth_rss_conf *rss_conf) +{ + struct bond_dev_private *internals = dev->data->dev_private; + + rss_conf->rss_hf = dev->data->dev_conf.rx_adv_conf.rss_conf.rss_hf; + rss_conf->rss_key_len = internals->rss_key_len; + if (rss_conf->rss_key != NULL) + memcpy(rss_conf->rss_key, internals->rss_key, internals->rss_key_len); + + return 0; +} + struct eth_dev_ops default_dev_ops = { - .dev_start = bond_ethdev_start, - .dev_stop = bond_ethdev_stop, - .dev_close = bond_ethdev_close, - .dev_configure = bond_ethdev_configure, - .dev_infos_get = bond_ethdev_info, - .rx_queue_setup = bond_ethdev_rx_queue_setup, - .tx_queue_setup = bond_ethdev_tx_queue_setup, - .rx_queue_release = bond_ethdev_rx_queue_release, - .tx_queue_release = bond_ethdev_tx_queue_release, - .link_update = bond_ethdev_link_update, - .stats_get = bond_ethdev_stats_get, - .stats_reset = bond_ethdev_stats_reset, - .promiscuous_enable = bond_ethdev_promiscuous_enable, - .promiscuous_disable = bond_ethdev_promiscuous_disable + .dev_start = bond_ethdev_start, + .dev_stop = bond_ethdev_stop, + .dev_close = bond_ethdev_close, + .dev_configure = bond_ethdev_configure, + .dev_infos_get = bond_ethdev_info, + .rx_queue_setup = bond_ethdev_rx_queue_setup, + .tx_queue_setup = bond_ethdev_tx_queue_setup, + .rx_queue_release = bond_ethdev_rx_queue_release, + .tx_queue_release = bond_ethdev_tx_queue_release, + .link_update = bond_ethdev_link_update, + .stats_get = bond_ethdev_stats_get, + .stats_reset = bond_ethdev_stats_reset, + .promiscuous_enable = bond_ethdev_promiscuous_enable, + .promiscuous_disable = bond_ethdev_promiscuous_disable, + .reta_update = bond_ethdev_rss_reta_update, + .reta_query = bond_ethdev_rss_reta_query, + .rss_hash_update = bond_ethdev_rss_hash_update, + .rss_hash_conf_get = bond_ethdev_rss_hash_conf_get }; static int @@ -2045,6 +2202,30 @@ bond_ethdev_configure(struct rte_eth_dev *dev) int arg_count; uint8_t port_id = dev - rte_eth_devices; + static const uint8_t default_rss_key[40] = { + 0x6D, 0x5A, 0x56, 0xDA, 0x25, 0x5B, 0x0E, 0xC2, 0x41, 0x67, 0x25, 0x3D, + 0x43, 0xA3, 0x8F, 0xB0, 0xD0, 0xCA, 0x2B, 0xCB, 0xAE, 0x7B, 0x30, 0xB4, + 0x77, 0xCB, 0x2D, 0xA3, 0x80, 0x30, 0xF2, 0x0C, 0x6A, 0x42, 0xB7, 0x3B, + 0xBE, 0xAC, 0x01, 0xFA + }; + + unsigned i, j; + + /* If RSS is enabled, fill table and key with default values */ + if (dev->data->dev_conf.rxmode.mq_mode & ETH_MQ_RX_RSS) { + + dev->data->dev_conf.rx_adv_conf.rss_conf.rss_key = internals->rss_key; + dev->data->dev_conf.rx_adv_conf.rss_conf.rss_key_len = 0; + memcpy(internals->rss_key, default_rss_key, 40); + + for (i = 0; i < RTE_DIM(internals->reta_conf); i++) { + internals->reta_conf[i].mask = ~0LL; + for (j = 0; j < RTE_RETA_GROUP_SIZE; j++) + internals->reta_conf[i].reta[j] = j % dev->data->nb_rx_queues; + } + + } + /* * if no kvlist, it means that this bonded device has been created * through the bonding api. diff --git a/drivers/net/bonding/rte_eth_bond_private.h b/drivers/net/bonding/rte_eth_bond_private.h index 45e5c65..f2ce94f 100644 --- a/drivers/net/bonding/rte_eth_bond_private.h +++ b/drivers/net/bonding/rte_eth_bond_private.h @@ -103,6 +103,8 @@ struct bond_slave_details { uint8_t last_link_status; /**< Port Id of slave eth_dev */ struct ether_addr persisted_mac_addr; + + uint16_t reta_size; }; @@ -155,6 +157,16 @@ struct bond_dev_private { uint32_t rx_offload_capa; /** Rx offload capability */ uint32_t tx_offload_capa; /** Tx offload capability */ + /** Bit mask of RSS offloads, the bit offset also means flow type */ + uint64_t flow_type_rss_offloads; + + uint16_t reta_size; + struct rte_eth_rss_reta_entry64 reta_conf[ETH_RSS_RETA_SIZE_512 / + RTE_RETA_GROUP_SIZE]; + + uint8_t rss_key[52]; /**< 52-byte hash key buffer. */ + uint8_t rss_key_len; /**< hash key length in bytes. */ + struct rte_kvargs *kvlist; uint8_t slave_update_idx; }; -- 1.7.9.5 ^ permalink raw reply [flat|nested] 126+ messages in thread
* [dpdk-dev] [PATCH v2 2/8] ring: dynamic rss configuration 2015-06-19 14:13 ` [dpdk-dev] [PATCH v2 " Tomasz Kulasek 2015-06-19 14:13 ` [dpdk-dev] [PATCH v2 1/8] bond: rss dynamic configuration Tomasz Kulasek @ 2015-06-19 14:13 ` Tomasz Kulasek 2015-06-19 14:13 ` [dpdk-dev] [PATCH v2 3/8] test: " Tomasz Kulasek ` (7 subsequent siblings) 9 siblings, 0 replies; 126+ messages in thread From: Tomasz Kulasek @ 2015-06-19 14:13 UTC (permalink / raw) To: dev This implementation allows to set and read RSS configuration for ring device, and is used to validate right values propagation over the slaves, in test units for dynamic RSS configuration for bonding. It have no impact on packet processing by ring device. Signed-off-by: Tomasz Kulasek <tomaszx.kulasek@intel.com> --- drivers/net/ring/rte_eth_ring.c | 122 +++++++++++++++++++++++++++++++++++++-- 1 file changed, 118 insertions(+), 4 deletions(-) diff --git a/drivers/net/ring/rte_eth_ring.c b/drivers/net/ring/rte_eth_ring.c index 6832f01..d7e7d9c 100644 --- a/drivers/net/ring/rte_eth_ring.c +++ b/drivers/net/ring/rte_eth_ring.c @@ -39,6 +39,7 @@ #include <rte_string_fns.h> #include <rte_dev.h> #include <rte_kvargs.h> +#include <rte_spinlock.h> #define ETH_RING_NUMA_NODE_ACTION_ARG "nodeaction" #define ETH_RING_ACTION_CREATE "CREATE" @@ -66,6 +67,17 @@ struct pmd_internals { struct ring_queue tx_ring_queues[RTE_PMD_RING_MAX_TX_RINGS]; struct ether_addr address; + + /** Bit mask of RSS offloads, the bit offset also means flow type */ + uint64_t flow_type_rss_offloads; + + rte_spinlock_t rss_lock; + + uint16_t reta_size; + struct rte_eth_rss_reta_entry64 reta_conf[ETH_RSS_RETA_SIZE_128 / + RTE_RETA_GROUP_SIZE]; + + uint8_t rss_key[40]; /**< 40-byte hash key. */ }; @@ -173,6 +185,8 @@ eth_dev_info(struct rte_eth_dev *dev, dev_info->max_tx_queues = (uint16_t)internals->nb_tx_queues; dev_info->min_rx_bufsize = 0; dev_info->pci_dev = NULL; + dev_info->flow_type_rss_offloads = internals->flow_type_rss_offloads; + dev_info->reta_size = internals->reta_size; } static void @@ -234,6 +248,93 @@ static int eth_link_update(struct rte_eth_dev *dev __rte_unused, int wait_to_complete __rte_unused) { return 0; } +static int +eth_rss_reta_update(struct rte_eth_dev *dev, + struct rte_eth_rss_reta_entry64 *reta_conf, + uint16_t reta_size) +{ + int i, j; + struct pmd_internals *internal = dev->data->dev_private; + + if (reta_size != internal->reta_size) + return -EINVAL; + + rte_spinlock_lock(&internal->rss_lock); + + /* Copy RETA table */ + for (i = 0; i < (internal->reta_size / RTE_RETA_GROUP_SIZE); i++) { + internal->reta_conf[i].mask = reta_conf[i].mask; + for (j = 0; j < RTE_RETA_GROUP_SIZE; j++) + if ((reta_conf[i].mask >> j) & 0x01) + internal->reta_conf[i].reta[j] = reta_conf[i].reta[j]; + } + + rte_spinlock_unlock(&internal->rss_lock); + + return 0; +} + +static int +eth_rss_reta_query(struct rte_eth_dev *dev, + struct rte_eth_rss_reta_entry64 *reta_conf, + uint16_t reta_size) +{ + int i, j; + struct pmd_internals *internal = dev->data->dev_private; + + if (reta_size != internal->reta_size) return -EINVAL; + + rte_spinlock_lock(&internal->rss_lock); + + /* Copy RETA table */ + for (i = 0; i < (internal->reta_size / RTE_RETA_GROUP_SIZE); i++) { + for (j = 0; j < RTE_RETA_GROUP_SIZE; j++) + if ((reta_conf[i].mask >> j) & 0x01) + reta_conf[i].reta[j] = internal->reta_conf[i].reta[j]; + } + + rte_spinlock_unlock(&internal->rss_lock); + + return 0; +} + +static int +eth_rss_hash_update(struct rte_eth_dev *dev, + struct rte_eth_rss_conf *rss_conf) +{ + struct pmd_internals *internal = dev->data->dev_private; + + rte_spinlock_lock(&internal->rss_lock); + + if ((rss_conf->rss_hf & internal->flow_type_rss_offloads) != 0) + dev->data->dev_conf.rx_adv_conf.rss_conf.rss_hf = + rss_conf->rss_hf & internal->flow_type_rss_offloads; + + if (rss_conf->rss_key != NULL) + memcpy(internal->rss_key, rss_conf->rss_key, 40); + + rte_spinlock_unlock(&internal->rss_lock); + + return 0; +} + +static int +eth_rss_hash_conf_get(struct rte_eth_dev *dev, + struct rte_eth_rss_conf *rss_conf) +{ + struct pmd_internals *internal = dev->data->dev_private; + + rte_spinlock_lock(&internal->rss_lock); + + rss_conf->rss_hf = dev->data->dev_conf.rx_adv_conf.rss_conf.rss_hf; + if (rss_conf->rss_key != NULL) + memcpy(rss_conf->rss_key, internal->rss_key, 40); + + rte_spinlock_unlock(&internal->rss_lock); + + return 0; +} + static const struct eth_dev_ops ops = { .dev_start = eth_dev_start, .dev_stop = eth_dev_stop, @@ -250,6 +351,10 @@ static const struct eth_dev_ops ops = { .stats_reset = eth_stats_reset, .mac_addr_remove = eth_mac_addr_remove, .mac_addr_add = eth_mac_addr_add, + .reta_update = eth_rss_reta_update, + .reta_query = eth_rss_reta_query, + .rss_hash_update = eth_rss_hash_update, + .rss_hash_conf_get = eth_rss_hash_conf_get }; int @@ -268,6 +373,13 @@ rte_eth_from_rings(const char *name, struct rte_ring *const rx_queues[], unsigned i; + static const uint8_t default_rss_key[40] = { + 0x6D, 0x5A, 0x56, 0xDA, 0x25, 0x5B, 0x0E, 0xC2, 0x41, 0x67, 0x25, 0x3D, + 0x43, 0xA3, 0x8F, 0xB0, 0xD0, 0xCA, 0x2B, 0xCB, 0xAE, 0x7B, 0x30, 0xB4, + 0x77, 0xCB, 0x2D, 0xA3, 0x80, 0x30, 0xF2, 0x0C, 0x6A, 0x42, 0xB7, 0x3B, + 0xBE, 0xAC, 0x01, 0xFA + }; + /* do some parameter checking */ if (rx_queues == NULL && nb_rx_queues > 0) goto error; @@ -316,12 +428,14 @@ rte_eth_from_rings(const char *name, struct rte_ring *const rx_queues[], internals->nb_rx_queues = nb_rx_queues; internals->nb_tx_queues = nb_tx_queues; - for (i = 0; i < nb_rx_queues; i++) { + for (i = 0; i < nb_rx_queues; i++) internals->rx_ring_queues[i].rng = rx_queues[i]; - } - for (i = 0; i < nb_tx_queues; i++) { + for (i = 0; i < nb_tx_queues; i++) internals->tx_ring_queues[i].rng = tx_queues[i]; - } + internals->flow_type_rss_offloads = ETH_RSS_PROTO_MASK; + internals->reta_size = RTE_DIM(internals->reta_conf) * RTE_RETA_GROUP_SIZE; + + memcpy(internals->rss_key, default_rss_key, 40); eth_drv->pci_drv.name = ring_ethdev_driver_name; eth_drv->pci_drv.id_table = id_table; -- 1.7.9.5 ^ permalink raw reply [flat|nested] 126+ messages in thread
* [dpdk-dev] [PATCH v2 3/8] test: dynamic rss configuration 2015-06-19 14:13 ` [dpdk-dev] [PATCH v2 " Tomasz Kulasek 2015-06-19 14:13 ` [dpdk-dev] [PATCH v2 1/8] bond: rss dynamic configuration Tomasz Kulasek 2015-06-19 14:13 ` [dpdk-dev] [PATCH v2 2/8] ring: dynamic rss configuration Tomasz Kulasek @ 2015-06-19 14:13 ` Tomasz Kulasek 2015-06-19 14:13 ` [dpdk-dev] [PATCH v2 4/8] bond: queue stats mapping Tomasz Kulasek ` (6 subsequent siblings) 9 siblings, 0 replies; 126+ messages in thread From: Tomasz Kulasek @ 2015-06-19 14:13 UTC (permalink / raw) To: dev This test module uses ring device to check right RSS configuration propagation. 1) Propagation test a) Set RSS hash function for bonding, fetch RSS hash function from bonded slave, check if same. Do it for all slaves. b) Repeat above for RSS key and RETA. 2)Dynamic adding slave to the bonding a) Remove slave from bonding. b) Change its configuration to other than bonding. c) Add slave to the started bonding device. d) Check if slaves configuration changed. 3) Repeat point 1) and 2) with RSS multi-queue mode enabled/disabled for bonding port. Signed-off-by: Tomasz Kulasek <tomaszx.kulasek@intel.com> --- app/test/Makefile | 1 + app/test/test_link_bonding_rssconf.c | 674 ++++++++++++++++++++++++++++++++++ 2 files changed, 675 insertions(+) create mode 100644 app/test/test_link_bonding_rssconf.c diff --git a/app/test/Makefile b/app/test/Makefile index 5cf8296..64e5fa7 100644 --- a/app/test/Makefile +++ b/app/test/Makefile @@ -135,6 +135,7 @@ SRCS-$(CONFIG_RTE_LIBRTE_ACL) += test_acl.c ifeq ($(CONFIG_RTE_LIBRTE_PMD_RING),y) SRCS-$(CONFIG_RTE_LIBRTE_PMD_BOND) += test_link_bonding.c SRCS-$(CONFIG_RTE_LIBRTE_PMD_BOND) += test_link_bonding_mode4.c +SRCS-$(CONFIG_RTE_LIBRTE_PMD_BOND) += test_link_bonding_rssconf.c endif SRCS-$(CONFIG_RTE_LIBRTE_PMD_RING) += test_pmd_ring.c diff --git a/app/test/test_link_bonding_rssconf.c b/app/test/test_link_bonding_rssconf.c new file mode 100644 index 0000000..c0be755 --- /dev/null +++ b/app/test/test_link_bonding_rssconf.c @@ -0,0 +1,674 @@ +/*- + * BSD LICENSE + * + * Copyright(c) 2015 Intel Corporation. All rights reserved. + * All rights reserved. + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * + * * Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * * Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in + * the documentation and/or other materials provided with the + * distribution. + * * Neither the name of Intel Corporation nor the names of its + * contributors may be used to endorse or promote products derived + * from this software without specific prior written permission. + * + * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS + * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT + * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR + * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT + * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, + * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT + * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, + * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY + * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT + * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE + * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + */ + +#include <string.h> +#include <stdarg.h> +#include <stdio.h> +#include <stdlib.h> +#include <stdint.h> +#include <inttypes.h> +#include <errno.h> +#include <rte_cycles.h> +#include <sys/queue.h> + +#include <rte_byteorder.h> +#include <rte_common.h> +#include <rte_debug.h> +#include <rte_ethdev.h> +#include <rte_log.h> +#include <rte_lcore.h> +#include <rte_memory.h> + +#include <rte_string_fns.h> + +#include <rte_eth_ring.h> +#include <rte_errno.h> +#include <rte_eth_bond.h> + +#include "test.h" + +#define SLAVE_COUNT (4) + +#define RXTX_RING_SIZE 1024 +#define RXTX_QUEUE_COUNT 4 + +#define BONDED_DEV_NAME ("rssconf_bond_dev") + +#define SLAVE_DEV_NAME_FMT ("rssconf_slave%d") +#define SLAVE_RXTX_QUEUE_FMT ("rssconf_slave%d_q%d") + +#define NUM_MBUFS 8191 +#define MBUF_SIZE (1600 + sizeof(struct rte_mbuf) + RTE_PKTMBUF_HEADROOM) +#define MBUF_CACHE_SIZE 250 +#define BURST_SIZE 32 + +#define INVALID_SOCKET_ID (-1) +#define INVALID_PORT_ID (0xFF) +#define INVALID_BONDING_MODE (-1) + +struct slave_conf { + uint8_t port_id; + struct rte_eth_dev_info dev_info; + + struct rte_eth_rss_conf rss_conf; + uint8_t rss_key[40]; + struct rte_eth_rss_reta_entry64 reta_conf[512 / RTE_RETA_GROUP_SIZE]; + + uint8_t is_slave; + struct rte_ring *rxtx_queue[RXTX_QUEUE_COUNT]; +}; + +struct link_bonding_rssconf_unittest_params { + uint8_t bond_port_id; + struct rte_eth_dev_info bond_dev_info; + struct rte_eth_rss_reta_entry64 bond_reta_conf[512 / RTE_RETA_GROUP_SIZE]; + struct slave_conf slave_ports[SLAVE_COUNT]; + + struct rte_mempool *mbuf_pool; +}; + +static struct link_bonding_rssconf_unittest_params test_params = { + .bond_port_id = INVALID_PORT_ID, + .slave_ports = { + [0 ... SLAVE_COUNT - 1] = { .port_id = INVALID_PORT_ID, .is_slave = 0} + }, + .mbuf_pool = NULL, +}; + +/** + * Default port configuration with RSS turned off + */ +static struct rte_eth_conf default_pmd_conf = { + .rxmode = { + .mq_mode = ETH_MQ_RX_NONE, + .max_rx_pkt_len = ETHER_MAX_LEN, + .split_hdr_size = 0, + .header_split = 0, /**< Header Split disabled */ + .hw_ip_checksum = 0, /**< IP checksum offload enabled */ + .hw_vlan_filter = 0, /**< VLAN filtering disabled */ + .jumbo_frame = 0, /**< Jumbo Frame Support disabled */ + .hw_strip_crc = 0, /**< CRC stripped by hardware */ + }, + .txmode = { + .mq_mode = ETH_MQ_TX_NONE, + }, + .lpbk_mode = 0, +}; + +static struct rte_eth_conf rss_pmd_conf = { + .rxmode = { + .mq_mode = ETH_MQ_RX_RSS, + .max_rx_pkt_len = ETHER_MAX_LEN, + .split_hdr_size = 0, + .header_split = 0, /**< Header Split disabled */ + .hw_ip_checksum = 0, /**< IP checksum offload enabled */ + .hw_vlan_filter = 0, /**< VLAN filtering disabled */ + .jumbo_frame = 0, /**< Jumbo Frame Support disabled */ + .hw_strip_crc = 0, /**< CRC stripped by hardware */ + }, + .txmode = { + .mq_mode = ETH_MQ_TX_NONE, + }, + .rx_adv_conf = { + .rss_conf = { + .rss_key = NULL, + .rss_hf = ETH_RSS_IPV6, + }, + }, + .lpbk_mode = 0, +}; + +#define FOR_EACH(_i, _item, _array, _size) \ + for (_i = 0, _item = &_array[0]; _i < _size && (_item = &_array[_i]); _i++) + +/* Macro for iterating over every port that can be used as a slave + * in this test. + * _i variable used as an index in test_params->slave_ports + * _slave pointer to &test_params->slave_ports[_idx] + */ +#define FOR_EACH_PORT(_i, _port) \ + FOR_EACH(_i, _port, test_params.slave_ports, \ + RTE_DIM(test_params.slave_ports)) + +static int +configure_ethdev(uint8_t port_id, struct rte_eth_conf *eth_conf, uint8_t start) +{ + int rxq, txq; + + TEST_ASSERT(rte_eth_dev_configure(port_id, RXTX_QUEUE_COUNT, + RXTX_QUEUE_COUNT, eth_conf) == 0, "Failed to configure device %u", + port_id); + + for (rxq = 0; rxq < RXTX_QUEUE_COUNT; rxq++) { + TEST_ASSERT(rte_eth_rx_queue_setup(port_id, rxq, RXTX_RING_SIZE, + rte_eth_dev_socket_id(port_id), NULL, + test_params.mbuf_pool) == 0, "Failed to setup rx queue."); + } + + for (txq = 0; txq < RXTX_QUEUE_COUNT; txq++) { + TEST_ASSERT(rte_eth_tx_queue_setup(port_id, txq, RXTX_RING_SIZE, + rte_eth_dev_socket_id(port_id), NULL) == 0, + "Failed to setup tx queue."); + } + + if (start) { + TEST_ASSERT(rte_eth_dev_start(port_id) == 0, + "Failed to start device (%d).", port_id); + } + + return 0; +} + +/** + * Remove all slaves from bonding + */ +static int +remove_slaves(void) +{ + unsigned n; + struct slave_conf *port; + + FOR_EACH_PORT(n, port) { + port = &test_params.slave_ports[n]; + if (port->is_slave) { + TEST_ASSERT_SUCCESS(rte_eth_bond_slave_remove( + test_params.bond_port_id, port->port_id), + "Cannot remove slave %d from bonding", port->port_id); + port->is_slave = 0; + } + } + + return 0; +} + +static int +remove_slaves_and_stop_bonded_device(void) +{ + TEST_ASSERT_SUCCESS(remove_slaves(), "Removing slaves"); + rte_eth_dev_stop(test_params.bond_port_id); + return TEST_SUCCESS; +} + +/** + * Add all slaves to bonding + */ +static int +bond_slaves(void) +{ + unsigned n; + struct slave_conf *port; + + FOR_EACH_PORT(n, port) { + port = &test_params.slave_ports[n]; + if (!port->is_slave) { + TEST_ASSERT_SUCCESS(rte_eth_bond_slave_add(test_params.bond_port_id, + port->port_id), "Cannot attach slave %d to the bonding", + port->port_id); + port->is_slave = 1; + } + } + + return 0; +} + +/** + * Set all RETA values in port_id to value + */ +static int +reta_set(uint8_t port_id, uint8_t value, int reta_size) +{ + struct rte_eth_rss_reta_entry64 reta_conf[512/RTE_RETA_GROUP_SIZE]; + int i, j; + + for (i = 0; i < reta_size / RTE_RETA_GROUP_SIZE; i++) { + /* select all fields to set */ + reta_conf[i].mask = ~0LL; + for (j = 0; j < RTE_RETA_GROUP_SIZE; j++) + reta_conf[i].reta[j] = value; + } + + return rte_eth_dev_rss_reta_update(port_id, reta_conf, reta_size); +} + +/** + * Check if slaves RETA is synchronized with bonding port. Returns 1 if slave + * port is synced with bonding port. + */ +static int +reta_check_synced(struct slave_conf *port) +{ + unsigned i; + + for (i = 0; i < test_params.bond_dev_info.reta_size; + i++) { + + int index = i / RTE_RETA_GROUP_SIZE; + int shift = i % RTE_RETA_GROUP_SIZE; + + if (port->reta_conf[index].reta[shift] != + test_params.bond_reta_conf[index].reta[shift]) + return 0; + + } + + return 1; +} + +/** + * Fetch bonding ports RETA + */ +static int +bond_reta_fetch(void) { + unsigned j; + + for (j = 0; j < test_params.bond_dev_info.reta_size / RTE_RETA_GROUP_SIZE; + j++) + test_params.bond_reta_conf[j].mask = ~0LL; + + TEST_ASSERT_SUCCESS(rte_eth_dev_rss_reta_query(test_params.bond_port_id, + test_params.bond_reta_conf, test_params.bond_dev_info.reta_size), + "Cannot take bonding ports RSS configuration"); + return 0; +} + +/** + * Fetch slaves RETA + */ +static int +slave_reta_fetch(struct slave_conf *port) { + unsigned j; + + for (j = 0; j < port->dev_info.reta_size / RTE_RETA_GROUP_SIZE; j++) + port->reta_conf[j].mask = ~0LL; + + TEST_ASSERT_SUCCESS(rte_eth_dev_rss_reta_query(port->port_id, + port->reta_conf, port->dev_info.reta_size), + "Cannot take bonding ports RSS configuration"); + return 0; +} + +/** + * Remove and add slave to check if slaves configuration is synced with + * the bonding ports values after adding new slave. + */ +static int +slave_remove_and_add(void) +{ + struct slave_conf *port = &(test_params.slave_ports[0]); + + /* 1. Remove first slave from bonding */ + TEST_ASSERT_SUCCESS(rte_eth_bond_slave_remove(test_params.bond_port_id, + port->port_id), "Cannot remove slave #d from bonding"); + + /* 2. Change removed (ex-)slave and bonding configuration to different + * values + */ + reta_set(test_params.bond_port_id, 1, test_params.bond_dev_info.reta_size); + bond_reta_fetch(); + + reta_set(port->port_id, 2, port->dev_info.reta_size); + slave_reta_fetch(port); + + TEST_ASSERT(reta_check_synced(port) == 0, + "Removed slave didn't should be synchronized with bonding port"); + + /* 3. Add (ex-)slave and check if configuration changed*/ + TEST_ASSERT_SUCCESS(rte_eth_bond_slave_add(test_params.bond_port_id, + port->port_id), "Cannot add slave"); + + bond_reta_fetch(); + slave_reta_fetch(port); + + return reta_check_synced(port); +} + +/** + * Test configuration propagation over slaves. + */ +static int +test_propagate(void) +{ + unsigned i; + uint8_t n; + struct slave_conf *port; + uint8_t bond_rss_key[40]; + struct rte_eth_rss_conf bond_rss_conf; + + int retval = 0; + uint64_t rss_hf = 0; + uint64_t default_rss_hf = 0; + + rte_eth_dev_info_get(test_params.bond_port_id, &test_params.bond_dev_info); + + /* + * Test hash function propagation + */ + for (i = 0; i < sizeof(test_params.bond_dev_info.flow_type_rss_offloads)*8; + i++) { + + rss_hf = test_params.bond_dev_info.flow_type_rss_offloads & (1<<i); + if (rss_hf) { + bond_rss_conf.rss_key = NULL; + bond_rss_conf.rss_hf = rss_hf; + + retval = rte_eth_dev_rss_hash_update(test_params.bond_port_id, + &bond_rss_conf); + TEST_ASSERT_SUCCESS(retval, "Cannot set slaves hash function"); + + FOR_EACH_PORT(n, port) { + port = &test_params.slave_ports[n]; + + retval = rte_eth_dev_rss_hash_conf_get(port->port_id, + &port->rss_conf); + TEST_ASSERT_SUCCESS(retval, + "Cannot take slaves RSS configuration"); + + TEST_ASSERT(port->rss_conf.rss_hf == rss_hf, + "Hash function not propagated for slave %d", + port->port_id); + } + + default_rss_hf = rss_hf; + } + + } + + /* + * Test key propagation + */ + for (i = 1; i < 10; i++) { + + /* Set all keys to zero */ + FOR_EACH_PORT(n, port) { + port = &test_params.slave_ports[n]; + memset(port->rss_conf.rss_key, 0, 40); + retval = rte_eth_dev_rss_hash_update(port->port_id, + &port->rss_conf); + TEST_ASSERT_SUCCESS(retval, "Cannot set slaves RSS keys"); + } + + memset(bond_rss_key, i, sizeof(bond_rss_key)); + bond_rss_conf.rss_hf = default_rss_hf, + bond_rss_conf.rss_key = bond_rss_key; + bond_rss_conf.rss_key_len = 40; + + retval = rte_eth_dev_rss_hash_update(test_params.bond_port_id, + &bond_rss_conf); + TEST_ASSERT_SUCCESS(retval, "Cannot set bonded port RSS keys"); + + FOR_EACH_PORT(n, port) { + port = &test_params.slave_ports[n]; + + retval = rte_eth_dev_rss_hash_conf_get(port->port_id, + &(port->rss_conf)); + + TEST_ASSERT_SUCCESS(retval, + "Cannot take slaves RSS configuration"); + + /* compare keys */ + retval = memcmp(port->rss_conf.rss_key, bond_rss_key, + sizeof(bond_rss_key)); + TEST_ASSERT(retval == 0, "Key value not propagated for slave %d", + port->port_id); + } + } + + /* + * Test RETA propagation + */ + for (i = 0; i < RXTX_QUEUE_COUNT; i++) { + + /* Set all keys to zero */ + FOR_EACH_PORT(n, port) { + port = &test_params.slave_ports[n]; + retval = reta_set(port->port_id, (i + 1) % RXTX_QUEUE_COUNT, + port->dev_info.reta_size); + TEST_ASSERT_SUCCESS(retval, "Cannot set slaves RETA"); + } + + TEST_ASSERT_SUCCESS(reta_set(test_params.bond_port_id, + i % RXTX_QUEUE_COUNT, test_params.bond_dev_info.reta_size), + "Cannot set bonded port RETA"); + + bond_reta_fetch(); + + FOR_EACH_PORT(n, port) { + port = &test_params.slave_ports[n]; + + slave_reta_fetch(port); + TEST_ASSERT(reta_check_synced(port) == 1, "RETAs inconsistent"); + } + } + + return TEST_SUCCESS; +} + +/** + * Test propagation logic, when RX_RSS mq_mode is turned on for bonding port + */ +static int +test_rss(void) +{ + /** + * Configure bonding port in RSS mq mode + */ + TEST_ASSERT_SUCCESS(configure_ethdev(test_params.bond_port_id, + &rss_pmd_conf, 0), "Failed to configure bonding device\n"); + + rte_eth_dev_info_get(test_params.bond_port_id, &test_params.bond_dev_info); + + TEST_ASSERT_SUCCESS(bond_slaves(), "Bonding slaves failed"); + + TEST_ASSERT_SUCCESS(rte_eth_dev_start(test_params.bond_port_id), + "Failed to start bonding port (%d).", test_params.bond_port_id); + + TEST_ASSERT_SUCCESS(test_propagate(), "Propagation test failed"); + + TEST_ASSERT(slave_remove_and_add() == 1, "New slave should be synced"); + + remove_slaves_and_stop_bonded_device(); + + return TEST_SUCCESS; +} + +/** + * Test propagation logic, when RX_RSS mq_mode is turned off for bonding port + */ +static int +test_rss_lazy(void) +{ + TEST_ASSERT_SUCCESS(configure_ethdev(test_params.bond_port_id, + &default_pmd_conf, 0), "Failed to configure bonding device\n"); + + rte_eth_dev_info_get(test_params.bond_port_id, &test_params.bond_dev_info); + + TEST_ASSERT_SUCCESS(bond_slaves(), "Bonding slaves failed"); + + TEST_ASSERT_SUCCESS(rte_eth_dev_start(test_params.bond_port_id), + "Failed to start bonding port (%d).", test_params.bond_port_id); + + TEST_ASSERT_SUCCESS(test_propagate(), "Propagation test failed"); + + TEST_ASSERT(slave_remove_and_add() == 0, "New slave shouldn't be synced"); + + remove_slaves_and_stop_bonded_device(); + + return TEST_SUCCESS; +} + +static int +test_setup(void) +{ + unsigned i, n; + int retval; + char name[256]; + char rng_name[RTE_RING_NAMESIZE]; + struct slave_conf *port; + + if (test_params.mbuf_pool == NULL) { + + test_params.mbuf_pool = rte_mempool_create("MBUF_POOL", NUM_MBUFS * + SLAVE_COUNT, MBUF_SIZE, MBUF_CACHE_SIZE, + sizeof(struct rte_pktmbuf_pool_private), rte_pktmbuf_pool_init, + NULL, rte_pktmbuf_init, NULL, rte_socket_id(), 0); + + TEST_ASSERT(test_params.mbuf_pool != NULL, + "rte_mempool_create failed\n"); + } + + /* Create / initialize ring eth devs. */ + FOR_EACH_PORT(n, port) { + port = &test_params.slave_ports[n]; + + snprintf(name, sizeof(name), SLAVE_DEV_NAME_FMT, n); + + for (i = 0; i < RXTX_QUEUE_COUNT; i++) { + snprintf(rng_name, sizeof(rng_name), SLAVE_RXTX_QUEUE_FMT, n, i); + + if (port->rxtx_queue[i] == NULL) { + port->rxtx_queue[i] = rte_ring_create(rng_name, RXTX_RING_SIZE, + 0, RING_F_SP_ENQ|RING_F_SC_DEQ); + TEST_ASSERT(port->rxtx_queue[i] != NULL, + "Failed to allocate rx ring '%s': %s", name, + rte_strerror(rte_errno)); + } + } + + if (rte_eth_from_rings(name, port->rxtx_queue, RXTX_QUEUE_COUNT, + port->rxtx_queue, RXTX_QUEUE_COUNT, 0)) { + TEST_ASSERT(retval >= 0, + "Failed to create ring ethdev '%s'\n", name); + } + + port->port_id = rte_eth_dev_count() - 1; + port->rss_conf.rss_key = port->rss_key; + port->rss_conf.rss_key_len = 40; + + retval = configure_ethdev(port->port_id, &default_pmd_conf, 1); + TEST_ASSERT_SUCCESS(retval, "Failed to configure virtual ethdev %s\n", + name); + + rte_eth_dev_info_get(port->port_id, &port->dev_info); + } + + if (test_params.bond_port_id == INVALID_PORT_ID) { + retval = rte_eth_bond_create(BONDED_DEV_NAME, 0, rte_socket_id()); + + TEST_ASSERT(retval >= 0, "Failed to create bonded ethdev %s", + BONDED_DEV_NAME); + + test_params.bond_port_id = retval; + + TEST_ASSERT_SUCCESS(configure_ethdev(test_params.bond_port_id, + &default_pmd_conf, 0), "Failed to configure bonding device\n"); + + rte_eth_dev_info_get(test_params.bond_port_id, + &test_params.bond_dev_info); + } + + return TEST_SUCCESS; +} + +static int +check_environment(void) +{ + return TEST_SUCCESS; +} + +static int +test_rssconf_executor(int (*test_func)(void)) +{ + int test_result; + + /* Check if environment is clean. Fail to launch a test if there was + * a critical error before that prevented to reset environment. */ + TEST_ASSERT_SUCCESS(check_environment(), + "Refusing to launch test in dirty environment."); + + RTE_VERIFY(test_func != NULL); + test_result = (*test_func)(); + + /* If test succeed check if environment wast left in good condition. */ + if (test_result == TEST_SUCCESS) + test_result = check_environment(); + + /* Reset environment in case test failed to do that. */ + if (test_result != TEST_SUCCESS) { + TEST_ASSERT_SUCCESS(remove_slaves_and_stop_bonded_device(), + "Failed to stop bonded device"); + } + + return test_result; +} + +static int +test_setup_wrapper(void) +{ + return test_rssconf_executor(&test_setup); +} + +static int +test_rss_wrapper(void) +{ + return test_rssconf_executor(&test_rss); +} + +static int +test_rss_lazy_wrapper(void) +{ + return test_rssconf_executor(&test_rss_lazy); +} + +static struct unit_test_suite link_bonding_rssconf_test_suite = { + .suite_name = "RSS Dynamic Configuration for Bonding Unit Test Suite", + .setup = test_setup, + .unit_test_cases = { + TEST_CASE_NAMED("test_setup", test_setup_wrapper), + TEST_CASE_NAMED("test_rss", test_rss_wrapper), + TEST_CASE_NAMED("test_rss_lazy", test_rss_lazy_wrapper), + { NULL, NULL, NULL, NULL, NULL } /**< NULL terminate unit test array */ + } +}; + +static int +test_link_bonding_rssconf(void) +{ + return unit_test_suite_runner(&link_bonding_rssconf_test_suite); +} + +static struct test_command test_link_bonding_rssconf_cmd = { + .command = "link_bonding_rssconf_autotest", + .callback = test_link_bonding_rssconf, +}; + +REGISTER_TEST_COMMAND(test_link_bonding_rssconf_cmd); -- 1.7.9.5 ^ permalink raw reply [flat|nested] 126+ messages in thread
* [dpdk-dev] [PATCH v2 4/8] bond: queue stats mapping 2015-06-19 14:13 ` [dpdk-dev] [PATCH v2 " Tomasz Kulasek ` (2 preceding siblings ...) 2015-06-19 14:13 ` [dpdk-dev] [PATCH v2 3/8] test: " Tomasz Kulasek @ 2015-06-19 14:13 ` Tomasz Kulasek 2015-06-19 14:13 ` [dpdk-dev] [PATCH v2 5/8] ring: queue stats mapping set dummy implementation Tomasz Kulasek ` (5 subsequent siblings) 9 siblings, 0 replies; 126+ messages in thread From: Tomasz Kulasek @ 2015-06-19 14:13 UTC (permalink / raw) To: dev Queue stats mapping is used in example application. This patch adds propagation of mapping over the slaves, and fills bonding port's stats with a sum of corresponding values taken from bonded slaves, when stats are requested for bonding port. Signed-off-by: Tomasz Kulasek <tomaszx.kulasek@intel.com> --- drivers/net/bonding/rte_eth_bond_pmd.c | 30 +++++++++++++++++++++++++++++- 1 file changed, 29 insertions(+), 1 deletion(-) diff --git a/drivers/net/bonding/rte_eth_bond_pmd.c b/drivers/net/bonding/rte_eth_bond_pmd.c index 5de2f30..90a2ac7 100644 --- a/drivers/net/bonding/rte_eth_bond_pmd.c +++ b/drivers/net/bonding/rte_eth_bond_pmd.c @@ -1774,7 +1774,7 @@ bond_ethdev_stats_get(struct rte_eth_dev *dev, struct rte_eth_stats *stats) { struct bond_dev_private *internals = dev->data->dev_private; struct rte_eth_stats slave_stats; - int i; + int i, j; for (i = 0; i < internals->slave_count; i++) { rte_eth_stats_get(internals->slaves[i].port_id, &slave_stats); @@ -1793,6 +1793,15 @@ bond_ethdev_stats_get(struct rte_eth_dev *dev, struct rte_eth_stats *stats) stats->rx_pause_xon += slave_stats.rx_pause_xon; stats->tx_pause_xoff += slave_stats.tx_pause_xoff; stats->rx_pause_xoff += slave_stats.rx_pause_xoff; + + for (j = 0; j < RTE_ETHDEV_QUEUE_STAT_CNTRS; j++) { + stats->q_ipackets[j] += slave_stats.q_ipackets[j]; + stats->q_opackets[j] += slave_stats.q_ipackets[j]; + stats->q_ibytes[j] += slave_stats.q_ipackets[j]; + stats->q_obytes[j] += slave_stats.q_ipackets[j]; + stats->q_errors[j] += slave_stats.q_ipackets[j]; + } + } } @@ -2103,6 +2112,24 @@ bond_ethdev_rss_hash_conf_get(struct rte_eth_dev *dev, return 0; } +static int +bond_ethdev_queue_stats_mapping_set(struct rte_eth_dev *dev, + uint16_t queue_id, uint8_t stat_idx, uint8_t is_rx) +{ + int i; + struct bond_dev_private *internals = dev->data->dev_private; + + for (i = 0; i < internals->slave_count; i++) + if (is_rx) + rte_eth_dev_set_rx_queue_stats_mapping ( + internals->slaves[i].port_id, queue_id, stat_idx); + else + rte_eth_dev_set_tx_queue_stats_mapping ( + internals->slaves[i].port_id, queue_id, stat_idx); + + return 0; +} + struct eth_dev_ops default_dev_ops = { .dev_start = bond_ethdev_start, .dev_stop = bond_ethdev_stop, @@ -2116,6 +2143,7 @@ struct eth_dev_ops default_dev_ops = { .link_update = bond_ethdev_link_update, .stats_get = bond_ethdev_stats_get, .stats_reset = bond_ethdev_stats_reset, + .queue_stats_mapping_set = bond_ethdev_queue_stats_mapping_set, .promiscuous_enable = bond_ethdev_promiscuous_enable, .promiscuous_disable = bond_ethdev_promiscuous_disable, .reta_update = bond_ethdev_rss_reta_update, -- 1.7.9.5 ^ permalink raw reply [flat|nested] 126+ messages in thread
* [dpdk-dev] [PATCH v2 5/8] ring: queue stats mapping set dummy implementation 2015-06-19 14:13 ` [dpdk-dev] [PATCH v2 " Tomasz Kulasek ` (3 preceding siblings ...) 2015-06-19 14:13 ` [dpdk-dev] [PATCH v2 4/8] bond: queue stats mapping Tomasz Kulasek @ 2015-06-19 14:13 ` Tomasz Kulasek 2015-06-19 14:13 ` [dpdk-dev] [PATCH v2 6/8] examples: dynamic rss configuration for bonding Tomasz Kulasek ` (4 subsequent siblings) 9 siblings, 0 replies; 126+ messages in thread From: Tomasz Kulasek @ 2015-06-19 14:13 UTC (permalink / raw) To: dev Per queue statistics are already implemented for ring device, but with static mapping (stat_idx == queue_id). This fix is required, if you want to use ring device in test application and is used only to point that per queue statistics are provided for this device. Signed-off-by: Tomasz Kulasek <tomaszx.kulasek@intel.com> --- drivers/net/ring/rte_eth_ring.c | 11 +++++++++++ 1 file changed, 11 insertions(+) diff --git a/drivers/net/ring/rte_eth_ring.c b/drivers/net/ring/rte_eth_ring.c index d7e7d9c..975bba7 100644 --- a/drivers/net/ring/rte_eth_ring.c +++ b/drivers/net/ring/rte_eth_ring.c @@ -335,6 +335,16 @@ eth_rss_hash_conf_get(struct rte_eth_dev *dev, return 0; } +static int +eth_queue_stats_mapping_set(__rte_unused struct rte_eth_dev *dev, + __rte_unused uint16_t queue_id, + __rte_unused uint8_t stat_idx, + __rte_unused uint8_t is_rx) +{ + /* Do nothing, just return ok */ + return 0; +} + static const struct eth_dev_ops ops = { .dev_start = eth_dev_start, .dev_stop = eth_dev_stop, @@ -347,6 +357,7 @@ static const struct eth_dev_ops ops = { .rx_queue_release = eth_queue_release, .tx_queue_release = eth_queue_release, .link_update = eth_link_update, + .queue_stats_mapping_set = eth_queue_stats_mapping_set, .stats_get = eth_stats_get, .stats_reset = eth_stats_reset, .mac_addr_remove = eth_mac_addr_remove, -- 1.7.9.5 ^ permalink raw reply [flat|nested] 126+ messages in thread
* [dpdk-dev] [PATCH v2 6/8] examples: dynamic rss configuration for bonding 2015-06-19 14:13 ` [dpdk-dev] [PATCH v2 " Tomasz Kulasek ` (4 preceding siblings ...) 2015-06-19 14:13 ` [dpdk-dev] [PATCH v2 5/8] ring: queue stats mapping set dummy implementation Tomasz Kulasek @ 2015-06-19 14:13 ` Tomasz Kulasek 2015-06-19 14:13 ` [dpdk-dev] [PATCH v2 7/8] doc: fixed spellings and typos Tomasz Kulasek ` (3 subsequent siblings) 9 siblings, 0 replies; 126+ messages in thread From: Tomasz Kulasek @ 2015-06-19 14:13 UTC (permalink / raw) To: dev This application allows you to test RSS configuration for bonded devices changing configuration dynamically, during receiving packets on slaves and observe the changes in its distribution over queues. After initialization process, all accessible ports are attached to one bonding as slaves. Monitor screen is divided into five main parts: - Port selection (on the very top) - RSS Configuration for selected port including hash function, key and RETA - Incoming packets statistics - Incoming packets list for selected port - Status bar with contextual information about selected part Signed-off-by: Tomasz Kulasek <tomaszx.kulasek@intel.com> --- examples/bond_rss/Makefile | 59 +++ examples/bond_rss/bondrss.c | 293 ++++++++++++++ examples/bond_rss/bondrss.h | 163 ++++++++ examples/bond_rss/config.c | 251 ++++++++++++ examples/bond_rss/ui.c | 920 +++++++++++++++++++++++++++++++++++++++++++ 5 files changed, 1686 insertions(+) create mode 100644 examples/bond_rss/Makefile create mode 100644 examples/bond_rss/bondrss.c create mode 100644 examples/bond_rss/bondrss.h create mode 100644 examples/bond_rss/config.c create mode 100644 examples/bond_rss/ui.c diff --git a/examples/bond_rss/Makefile b/examples/bond_rss/Makefile new file mode 100644 index 0000000..db457a3 --- /dev/null +++ b/examples/bond_rss/Makefile @@ -0,0 +1,59 @@ +# BSD LICENSE +# +# Copyright(c) 2010-2015 Intel Corporation. All rights reserved. +# All rights reserved. +# +# Redistribution and use in source and binary forms, with or without +# modification, are permitted provided that the following conditions +# are met: +# +# * Redistributions of source code must retain the above copyright +# notice, this list of conditions and the following disclaimer. +# * Redistributions in binary form must reproduce the above copyright +# notice, this list of conditions and the following disclaimer in +# the documentation and/or other materials provided with the +# distribution. +# * Neither the name of Intel Corporation nor the names of its +# contributors may be used to endorse or promote products derived +# from this software without specific prior written permission. +# +# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS +# "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT +# LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR +# A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT +# OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, +# SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT +# LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, +# DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY +# THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT +# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE +# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + +ifeq ($(RTE_SDK),) +$(error "Please define RTE_SDK environment variable") +endif + +# Default target, can be overridden by command line or environment +RTE_TARGET ?= x86_64-native-linuxapp-gcc + +include $(RTE_SDK)/mk/rte.vars.mk + +# binary name +APP = bondrss + +# all source are stored in SRCS-y +SRCS-y := bondrss.c +SRCS-y += ui.c +SRCS-y += config.c + +CFLAGS += $(WERROR_FLAGS) + +# workaround for a gcc bug with noreturn attribute +# http://gcc.gnu.org/bugzilla/show_bug.cgi?id=12603 +ifeq ($(CONFIG_RTE_TOOLCHAIN_GCC),y) +CFLAGS_main.o += -Wno-return-type +endif + +LDLIBS += -lncurses + +include $(RTE_SDK)/mk/rte.extapp.mk diff --git a/examples/bond_rss/bondrss.c b/examples/bond_rss/bondrss.c new file mode 100644 index 0000000..2dc9979 --- /dev/null +++ b/examples/bond_rss/bondrss.c @@ -0,0 +1,293 @@ +/*- + * BSD LICENSE + * + * Copyright(c) 2010-2015 Intel Corporation. All rights reserved. + * All rights reserved. + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * + * * Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * * Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in + * the documentation and/or other materials provided with the + * distribution. + * * Neither the name of Intel Corporation nor the names of its + * contributors may be used to endorse or promote products derived + * from this software without specific prior written permission. + * + * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS + * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT + * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR + * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT + * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, + * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT + * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, + * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY + * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT + * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE + * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + */ + +#include "bondrss.h" + +#define RSS_HASH_KEY_LENGTH 40 + +static const struct rte_eth_conf port_conf_default = { + .rxmode = { + .mq_mode = ETH_MQ_RX_RSS, + .max_rx_pkt_len = ETHER_MAX_LEN, + .split_hdr_size = 0, + .header_split = 0, /**< Header Split disabled. */ + .hw_ip_checksum = 0, /**< IP checksum offload disabled. */ + .hw_vlan_filter = 1, /**< VLAN filtering enabled. */ + .hw_vlan_strip = 1, /**< VLAN strip enabled. */ + .hw_vlan_extend = 0, /**< Extended VLAN disabled. */ + .jumbo_frame = 0, /**< Jumbo Frame Support disabled. */ + .hw_strip_crc = 0, /**< CRC stripping by hardware disabled. */ + .enable_scatter = 0, /**< scatter rx disabled */ + }, + .rx_adv_conf = { + .rss_conf = { + .rss_key = NULL, + .rss_hf = ETH_RSS_IPV4, + }, + }, +}; + +static const struct rte_eth_conf port_conf_bonding = { + .rxmode = { + .mq_mode = ETH_MQ_RX_RSS, + .max_rx_pkt_len = ETHER_MAX_LEN, + .split_hdr_size = 0, + .header_split = 0, /**< Header Split disabled. */ + .hw_ip_checksum = 0, /**< IP checksum offload disabled. */ + .hw_vlan_filter = 1, /**< VLAN filtering enabled. */ + .hw_vlan_strip = 1, /**< VLAN strip enabled. */ + .hw_vlan_extend = 0, /**< Extended VLAN disabled. */ + .jumbo_frame = 0, /**< Jumbo Frame Support disabled. */ + .hw_strip_crc = 0, /**< CRC stripping by hardware disabled. */ + .enable_scatter = 0, /**< scatter rx disabled */ + }, + .rx_adv_conf = { + .rss_conf = { + .rss_key = NULL, + .rss_hf = ETH_RSS_IPV6, + }, + }, +}; + +/* + * Configurable number of RX/TX queues. + */ +int nb_rxq = 4; /**< Number of RX queues per port. */ +int nb_txq = 1; /**< Number of TX queues per port. */ + +int bond_port_id = -1; /**< Bonded device port id */ + +int nb_ports; +struct rte_port *ports; +rte_atomic64_t counter; /**< Received packet's counter */ + +/* + * Initializes a given port using global settings and with the rx buffers + * coming from the mbuf_pool passed as parameter + */ +static inline +int init_port(uint8_t port_id, struct rte_mempool *mbuf_pool, + struct rte_eth_conf port_conf) +{ + + int retval; + uint16_t q; + struct rte_eth_rxconf rxconf; + + struct rte_port *port; + + if (port_id >= rte_eth_dev_count()) + return -1; + + port = &ports[port_id]; + port->nb_rx_queues = nb_rxq; + port->nb_tx_queues = nb_txq; + + memcpy(&(port->dev_conf), &port_conf, sizeof(port_conf)); + retval = rte_eth_dev_configure(port_id, port->nb_rx_queues, + port->nb_tx_queues, &port_conf); + + if (retval != 0) + return retval; + + rte_eth_dev_info_get(port_id, &(port->dev_info)); + rxconf = (port->dev_info).default_rxconf; + rxconf.rx_free_thresh = 32; + + for (q = 0; q < port->nb_rx_queues; q++) { + retval = rte_eth_rx_queue_setup(port_id, q, RX_RING_SIZE, + rte_eth_dev_socket_id(port_id), &rxconf, + mbuf_pool); + if (retval < 0) + return retval; + } + + for (q = 0; q < port->nb_tx_queues; q++) { + retval = rte_eth_tx_queue_setup(port_id, q, TX_RING_SIZE, + rte_eth_dev_socket_id(port_id), + &((port->dev_info).default_txconf)); + if (retval < 0) + return retval; + } + + port->bond_mode = -1; + port->promiscuous = 0; + port->enabled = 1; + + return 0; +} + +static int +rx_loop(__attribute__((unused)) void *dummy) +{ + uint8_t port_id; + int nq; + int n; + + for (;;) + for (port_id = 0; port_id < nb_ports; port_id++) { + /* Pool only bonding ports */ + if (ports[port_id].bond_mode >= 0) + for (nq = 0; nq < ports[port_id].nb_rx_queues; nq++) { + struct rte_mbuf *bufs[BURST_SIZE]; + + const uint16_t nb_rx = rte_eth_rx_burst(port_id, nq, bufs, + BURST_SIZE); + if (unlikely(nb_rx == 0)) + continue; + + ports[port_id].rxq_ipackets[nq] += nb_rx; + for (n = 0; n < nb_rx; n++) { + record_pkt(port_id, nq, bufs[n]); + rte_pktmbuf_free(bufs[n]); + } + } + } + + return 0; +} + +static uint16_t +rx_callback(uint8_t port_id, uint16_t qidx, struct rte_mbuf **pkts, + uint16_t nb_pkts, uint16_t max_pkts __rte_unused, void *_ __rte_unused) +{ + int n; + + ports[port_id].rxq_ipackets[qidx] += nb_pkts; + for (n = 0; n < nb_pkts; n++) { + rte_atomic64_inc(&counter); + pkts[n]->udata64 = counter.cnt; + record_pkt(port_id, qidx, pkts[n]); + } + + return nb_pkts; +} + +int +main(int argc, char *argv[]) +{ + struct rte_mempool *mbuf_pool; + unsigned lcore_id = 0; + uint8_t port_id, queue_id; + + /* init EAL */ + int ret = rte_eal_init(argc, argv); + if (ret < 0) + rte_exit(EXIT_FAILURE, "Error with EAL initialization\n"); + argc -= ret; + argv += ret; + + nb_ports = rte_eth_dev_count(); + if (nb_ports < 1) + rte_exit(EXIT_FAILURE, + "At least one port must be available to create a bonding\n"); + + mbuf_pool = rte_mempool_create("MBUF_POOL", NUM_MBUFS * nb_ports, MBUF_SIZE, + MBUF_CACHE_SIZE, + sizeof(struct rte_pktmbuf_pool_private), + rte_pktmbuf_pool_init, NULL, + rte_pktmbuf_init, NULL, rte_socket_id(), 0); + if (mbuf_pool == NULL) + rte_exit(EXIT_FAILURE, "Cannot create mbuf pool\n"); + + /* Configuration of Ethernet ports. */ + ports = rte_zmalloc("bondrss: ports", + sizeof(struct rte_port) * RTE_MAX_ETHPORTS, + RTE_CACHE_LINE_SIZE); + if (ports == NULL) + rte_exit(EXIT_FAILURE, "rte_zmalloc(%d struct rte_port) failed\n", + RTE_MAX_ETHPORTS); + + /* enabled allocated ports */ + for (port_id = 0; port_id < RTE_MAX_ETHPORTS; port_id++) + ports[port_id].enabled = 0; + + /* initialize all ports */ + for (port_id = 0; port_id < nb_ports; port_id++) { + if (init_port(port_id, mbuf_pool, port_conf_default) != 0) + rte_exit(EXIT_FAILURE, "Cannot init port %"PRIu8"\n", port_id); + + /* Add rx callbacks to the slave's port */ + for (queue_id = 0; queue_id < ports[port_id].nb_rx_queues; queue_id++) + rte_eth_add_rx_callback(port_id, queue_id, rx_callback, NULL); + + } + + /* create bonding port */ + bond_port_id = rte_eth_bond_create("eth_bond", 0, 0); + if (bond_port_id < 0) + rte_exit(EXIT_FAILURE, "Cannot create bonding port\n"); + + for (port_id = 0; port_id < nb_ports; port_id++) + rte_eth_bond_slave_add(bond_port_id, port_id); + + /* count again */ + nb_ports = rte_eth_dev_count(); + + init_port(bond_port_id, mbuf_pool, port_conf_bonding); + + /* start bonding port*/ + ret = rte_eth_dev_start(bond_port_id); + + /* enable promiscuous by default */ + rte_eth_promiscuous_enable(bond_port_id); + + for (port_id = 0; port_id < nb_ports; port_id++) { + ports[port_id].bond_mode = rte_eth_bond_mode_get(port_id); + ports[port_id].promiscuous = rte_eth_promiscuous_get(port_id); + + /* map queues to stats */ + if (ports[port_id].bond_mode >= 0) { + for (queue_id = 0; queue_id < ports[port_id].nb_rx_queues; + queue_id++) { + rte_eth_dev_set_rx_queue_stats_mapping(port_id, queue_id, + queue_id); + } + } + + } + + /* Initialize received packet's counter */ + rte_atomic64_init(&counter); + + if (rte_lcore_count() <= 1) + rte_exit(EXIT_FAILURE, "More than one free lcore is needed\n"); + + lcore_id = rte_get_next_lcore(rte_lcore_id(), 1, 0); + rte_eal_remote_launch(rx_loop, NULL, lcore_id); + + /* call lcore_main on master core only */ + ui_loop(); + + return 0; +} diff --git a/examples/bond_rss/bondrss.h b/examples/bond_rss/bondrss.h new file mode 100644 index 0000000..305a690 --- /dev/null +++ b/examples/bond_rss/bondrss.h @@ -0,0 +1,163 @@ +/*- + * BSD LICENSE + * + * Copyright(c) 2015 Intel Corporation. All rights reserved. + * All rights reserved. + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * + * * Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * * Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in + * the documentation and/or other materials provided with the + * distribution. + * * Neither the name of Intel Corporation nor the names of its + * contributors may be used to endorse or promote products derived + * from this software without specific prior written permission. + * + * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS + * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT + * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR + * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT + * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, + * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT + * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, + * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY + * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT + * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE + * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + */ + +#ifndef BONDRSS_H_ +#define BONDRSS_H_ + +#include <stdint.h> +#include <inttypes.h> +#include <rte_eal.h> +#include <rte_ethdev.h> +#include <rte_cycles.h> +#include <rte_lcore.h> +#include <rte_mbuf.h> +#include <rte_malloc.h> + +#include <rte_eth_bond.h> + +#define RX_RING_SIZE 128 +#define TX_RING_SIZE 512 + +#define NUM_MBUFS 8191 +#define MBUF_SIZE (1600 + sizeof(struct rte_mbuf) + RTE_PKTMBUF_HEADROOM) +#define MBUF_CACHE_SIZE 250 +#define BURST_SIZE 32 + +/* + * Configurable number of RX/TX queues. + */ +extern int nb_rxq; /**< Number of RX queues per port. */ +extern int nb_txq; /**< Number of TX queues per port. */ + +extern int bond_port_id; /**< Bonded device port id */ +extern int bond_mode; + +struct rss_type_info { + char str[32]; + uint64_t rss_type; +}; + +static struct rss_type_info rss_type_table[] = { + {"ipv4", ETH_RSS_IPV4}, + {"ipv4-frag", ETH_RSS_FRAG_IPV4}, + {"ipv4-tcp", ETH_RSS_NONFRAG_IPV4_TCP}, + {"ipv4-udp", ETH_RSS_NONFRAG_IPV4_UDP}, + {"ipv4-sctp", ETH_RSS_NONFRAG_IPV4_SCTP}, + {"ipv4-other", ETH_RSS_NONFRAG_IPV4_OTHER}, + {"ipv6", ETH_RSS_IPV6}, + {"ipv6-frag", ETH_RSS_FRAG_IPV6}, + {"ipv6-tcp", ETH_RSS_NONFRAG_IPV6_TCP}, + {"ipv6-udp", ETH_RSS_NONFRAG_IPV6_UDP}, + {"ipv6-sctp", ETH_RSS_NONFRAG_IPV6_SCTP}, + {"ipv6-other", ETH_RSS_NONFRAG_IPV6_OTHER}, + {"l2-payload", ETH_RSS_L2_PAYLOAD}, + {"ipv6-ex", ETH_RSS_IPV6_EX}, + {"ipv6-tcp-ex", ETH_RSS_IPV6_TCP_EX}, + {"ipv6-udp-ex", ETH_RSS_IPV6_UDP_EX}, +}; + +struct rss_type_fn { + uint8_t enabled; + struct rss_type_info *info; +}; + +/** + * Buffer for last received packets + */ +#define PKT_RECORD_SIZE 256 + +struct pkt_info { + uint64_t n; /**< Packet number */ + uint8_t port_id; + uint8_t queue_id; + uint64_t ol_flags; /**< Offload features. */ + uint32_t rss; /**< RSS hash result if RSS enabled */ +}; + +struct pkt_record { + uint64_t count; /**< Total number of received packets */ + int top; + struct pkt_info pkt_info[PKT_RECORD_SIZE]; +}; + +/** + * The data structure associated with each port. + */ +struct rte_port { + struct rte_eth_dev_info dev_info; /**< PCI info + driver name */ + struct rte_eth_conf dev_conf; /**< Port configuration. */ + + uint16_t nb_rx_queues; /**< Total number of rx queues */ + uint16_t nb_tx_queues; /**< Total number of tx queues*/ + + int bond_mode; + struct ether_addr addr; + + int rss_type_count; + struct rss_type_fn rss_type_fn[RTE_DIM(rss_type_table)]; + struct rte_eth_rss_reta_entry64 reta_conf[512 / RTE_RETA_GROUP_SIZE]; + uint8_t rss_key[52]; + + + struct rte_eth_stats stats; + uint64_t rxq_ipackets[128]; + + struct pkt_record record; + + uint8_t rss; /**< RSS enabled for port */ + uint8_t promiscuous; /**< promiscuous mode enabled for port */ + uint8_t enabled; /**< port is enabled */ +}; + +extern int nb_ports; +extern struct rte_port *ports; +extern rte_atomic64_t counter; /**< Received packet's counter */ + +void update_port_info(uint8_t port_id); + +int rss_enable(uint8_t port_id, int enabled); + +int key_set(uint8_t port_id, uint8_t *key, uint8_t len); +int key_set_random(uint8_t port_id, uint8_t len); + +int reta_set_default(uint8_t port_id); +int reta_set_random(uint8_t port_id); +int reta_set_all(uint8_t port_id, uint8_t value); +int reta_set_weights(uint8_t port_id, uint64_t *weights); + +struct pkt_info *record_pkt(uint8_t port_id, int queue_id, struct rte_mbuf *mb); + +void ui_loop(void); + + +#endif /* BONDRSS_H_ */ diff --git a/examples/bond_rss/config.c b/examples/bond_rss/config.c new file mode 100644 index 0000000..39c00e5 --- /dev/null +++ b/examples/bond_rss/config.c @@ -0,0 +1,251 @@ +/*- + * BSD LICENSE + * + * Copyright(c) 2015 Intel Corporation. All rights reserved. + * All rights reserved. + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * + * * Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * * Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in + * the documentation and/or other materials provided with the + * distribution. + * * Neither the name of Intel Corporation nor the names of its + * contributors may be used to endorse or promote products derived + * from this software without specific prior written permission. + * + * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS + * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT + * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR + * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT + * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, + * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT + * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, + * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY + * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT + * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE + * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + */ + +#include <time.h> +#include <stdlib.h> +#include "bondrss.h" + +static const struct rte_eth_conf port_conf_default = { + .rxmode = { + .max_rx_pkt_len = ETHER_MAX_LEN, + .split_hdr_size = 0, + .header_split = 0, /**< Header Split disabled. */ + .hw_ip_checksum = 0, /**< IP checksum offload disabled. */ + .hw_vlan_filter = 1, /**< VLAN filtering enabled. */ + .hw_vlan_strip = 1, /**< VLAN strip enabled. */ + .hw_vlan_extend = 0, /**< Extended VLAN disabled. */ + .jumbo_frame = 0, /**< Jumbo Frame Support disabled. */ + .hw_strip_crc = 0, /**< CRC stripping by hardware disabled. */ + .enable_scatter = 0, /**< scatter rx disabled */ + }, + .rx_adv_conf = { + .rss_conf = { + .rss_key = NULL, + .rss_hf = ETH_RSS_IPV4, + }, + }, +}; + + +void +update_port_info(uint8_t port_id) +{ + int i; + struct rte_port *port; + + port = &ports[port_id]; + + rte_eth_dev_info_get(port_id, &(port->dev_info)); + + port->promiscuous = rte_eth_promiscuous_get(port_id); + + /* Update device information */ + rte_eth_dev_info_get(port_id, &port->dev_info); + rte_eth_macaddr_get(port_id, &port->addr); + + port->dev_conf.rx_adv_conf.rss_conf.rss_key = port->rss_key; + port->dev_conf.rx_adv_conf.rss_conf.rss_key_len = 40; + rte_eth_dev_rss_hash_conf_get(port_id, + &(port->dev_conf.rx_adv_conf.rss_conf)); + + /* select all fields to be fetched */ + for (i = 0; i < port->dev_info.reta_size / RTE_RETA_GROUP_SIZE; i++) + port->reta_conf[i].mask = ~0LL; + + rte_eth_dev_rss_reta_query(port_id, port->reta_conf, + port->dev_info.reta_size); + +} + +/** + * Try to enable RSS for port_id. + */ +int +rss_enable(uint8_t port_id, int enabled) +{ + struct rte_port *port; + int retval; + + port = &ports[port_id]; + + rte_eth_dev_stop(port_id); + + if (enabled) { + port->dev_conf.rxmode.mq_mode = ETH_MQ_RX_RSS; + port->dev_conf.rx_adv_conf.rss_conf.rss_hf = ETH_RSS_IPV4; + } else { + port->dev_conf.rxmode.mq_mode = 0; + port->dev_conf.rx_adv_conf.rss_conf.rss_hf = 0; + } + + retval = rte_eth_dev_configure(port_id, port->nb_rx_queues, + port->nb_tx_queues, &port->dev_conf); + if (retval != 0) + return retval; + + retval = rte_eth_dev_start(port_id); + + return 0; +} + +int +key_set(uint8_t port_id, uint8_t *key, uint8_t len) +{ + struct rte_eth_rss_conf rss_conf; + + int i; + + rss_conf.rss_key = NULL; + rss_conf.rss_key_len = 0; + i = rte_eth_dev_rss_hash_conf_get(port_id, &rss_conf); + + rss_conf.rss_key = key; + rss_conf.rss_key_len = len; + + i = rte_eth_dev_rss_hash_update(port_id, + &rss_conf); + + return i; +} + +/** + * Set random RSS key value + */ +int +key_set_random(uint8_t port_id, uint8_t len) +{ + int i; + uint8_t key[40]; + + for (i = 0; i < len; i++) + key[i] = rand() % 256; + + return key_set(port_id, key, len); +} + +/** + * Set random RETA values + */ +int +reta_set_random(uint8_t port_id) +{ + struct rte_eth_rss_reta_entry64 reta_conf[512/RTE_RETA_GROUP_SIZE]; + int i, j; + + int reta_size = ports[port_id].dev_info.reta_size; + + for (i = 0; i < reta_size / RTE_RETA_GROUP_SIZE; i++) { + /* select all fields to set */ + reta_conf[i].mask = ~0LL; + for (j = 0; j < RTE_RETA_GROUP_SIZE; j++) + reta_conf[i].reta[j] = rand() % ports[port_id].nb_rx_queues; + } + + return rte_eth_dev_rss_reta_update(port_id, reta_conf, reta_size); +} + +/** + * Set default RETA values + */ +int +reta_set_default(uint8_t port_id) +{ + struct rte_eth_rss_reta_entry64 reta_conf[512/RTE_RETA_GROUP_SIZE]; + int i, j; + + int reta_size = ports[port_id].dev_info.reta_size; + + for (i = 0; i < reta_size / RTE_RETA_GROUP_SIZE; i++) { + /* select all fields to set */ + reta_conf[i].mask = ~0LL; + for (j = 0; j < RTE_RETA_GROUP_SIZE; j++) + reta_conf[i].reta[j] = j % nb_rxq; + } + + return rte_eth_dev_rss_reta_update(port_id, reta_conf, reta_size); +} + +/** + * Fill the RETA with values + */ +int +reta_set_all(uint8_t port_id, uint8_t value) +{ + struct rte_eth_rss_reta_entry64 reta_conf[512/RTE_RETA_GROUP_SIZE]; + int i, j; + + int reta_size = ports[port_id].dev_info.reta_size; + + for (i = 0; i < reta_size / RTE_RETA_GROUP_SIZE; i++) { + /* select all fields to set */ + reta_conf[i].mask = ~0LL; + for (j = 0; j < RTE_RETA_GROUP_SIZE; j++) + reta_conf[i].reta[j] = value; + } + + return rte_eth_dev_rss_reta_update(port_id, reta_conf, reta_size); +} + +/** + * Set the RETA basing on weights table + */ +int +reta_set_weights(uint8_t port_id, uint64_t *weights) +{ + struct rte_eth_rss_reta_entry64 reta_conf[512/RTE_RETA_GROUP_SIZE]; + unsigned i, j, k; + + unsigned reta_size = ports[port_id].dev_info.reta_size; + + uint64_t sum = 0, sum2; + + for (i = 0; i < ports[port_id].nb_rx_queues; i++) + sum += weights[i]; + sum2 = sum * (ports[port_id].nb_rx_queues-1); + + if (sum2 == 0) + return 0; + + for (i = 0; i < reta_size / RTE_RETA_GROUP_SIZE; i++) + reta_conf[i].mask = ~0LL; + k = 0; + for (i = 0; i < ports[port_id].nb_rx_queues; i++) { + for (j = 0; k < reta_size && + j < ((sum - weights[i]) * reta_size + 1) / sum2; j++) { + reta_conf[k/RTE_RETA_GROUP_SIZE].reta[k%RTE_RETA_GROUP_SIZE] = i; + k++; + } + } + + return rte_eth_dev_rss_reta_update(port_id, reta_conf, reta_size); +} diff --git a/examples/bond_rss/ui.c b/examples/bond_rss/ui.c new file mode 100644 index 0000000..90cae09 --- /dev/null +++ b/examples/bond_rss/ui.c @@ -0,0 +1,920 @@ +/*- + * BSD LICENSE + * + * Copyright(c) 2015 Intel Corporation. All rights reserved. + * All rights reserved. + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * + * * Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * * Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in + * the documentation and/or other materials provided with the + * distribution. + * * Neither the name of Intel Corporation nor the names of its + * contributors may be used to endorse or promote products derived + * from this software without specific prior written permission. + * + * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS + * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT + * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR + * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT + * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, + * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT + * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, + * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY + * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT + * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE + * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + */ + +#include <rte_atomic.h> + +#include <ncurses.h> +#include <string.h> +#include <signal.h> +#include <sys/ioctl.h> +#include <menu.h> +#include "bondrss.h" + +WINDOW *win_footer = NULL; +WINDOW *win_header = NULL; +WINDOW *win_list = NULL; +WINDOW *win_stats = NULL; +WINDOW *win_reta = NULL; +WINDOW *win_err = NULL; + +int scr_lines = 0; +int scr_cols = 0; + +unsigned win_header_lines; +unsigned win_list_lines; +unsigned win_stats_lines; +unsigned win_reta_lines; +unsigned win_err_lines = 0; + +const char *ui_status_text; +static char *ui_err_text[5]; + +struct rte_port *selected_port; +static uint8_t selected_port_id; + +struct rte_port *focused_port; +static uint8_t focused_port_id; + +#define UI_MODE_PORT 0 +#define UI_MODE_RSS_FN 1 +#define UI_MODE_RSS_KEY 2 +#define UI_MODE_RETA 3 +#define UI_MODE_HELP 4 + +int ui_mode = UI_MODE_PORT; + +static uint16_t _is_lock; +static uint16_t _is_ready; + +static uint16_t _do_configure; +static uint16_t _do_repaint; +static uint16_t _do_repaint_stats; +static uint16_t _do_repaint_list; +static uint16_t _do_repaint_info; +static uint16_t _do_repaint_err; + + +struct help_page { + const char *title; + const char *body; +}; + +static struct help_page help_pages[3] = { + { + .title = "About", + + .body = + "\n\n" + " ______ ______ ______ _____ ______ ______ ______\n" + "| | | \\ / | | \\ | | \\ \\ | | \\ \\ | | | \\ / | / |\n" + "| |--| < | | | | | | | | | | | | | |__| | '------. '------.\n" + "|_|__|_/ \\_|__|_/ |_| |_| |_|_/_/ |_| \\_\\ ____|_/ ____|_/\n" + "\n\n" + "This application allows you to test RSS configuration for " + "bonded devices\nchanging configuration dynamically, during " + "receiving packets on slaves and\nobserve the changes in its " + "distribution over queues.\n" + "\n" + + }, + { + .title = "Introduction", + .body = "\n\n" + "After initialization process, all accessible ports are " + "attached to one\nbonding as slaves.\n" + "\n" + + "Monitor screen is divided into five main parts:\n" + " - Port selection (on the very top)\n" + " - RSS Configuration for selected port including hash " + "function, key and\n RETA\n" + " - Incoming packets statistics\n" + " - Incoming packets list for selected port\n" + " - Status bar with contextual information about selected " + "part\n" + "\n\n" + + "[up], [down] arrows key lets you highlight a part " + "and parameter to change.\n\n" + }, + { + .title = "Key bindings", + .body = + "_Port_selection_\n\n" + " [left], [right] - select port,\n" + " [space] - force turning on/off RSS mode,\n" + " [p] - turn on/off promiscuous mode,\n" + " [c] - clear statistics and incoming packet list,\n" + "\n" + "_RSS_hash_function_\n\n" + " [1]-[9] - toggle selected hash function,\n" + " [a] - select all,\n" + "\n" + "_RSS_key_\n\n" + " [1]-[4] - set one of predefined key value,\n" + " [r] - randomize value,\n" + "\n" + "_RSS_RETA_\n\n" + " [number] - fill RETA with number,\n" + " [r] - randomize value,\n" + " [d] - set default value," + + } +}; + +static int +ui_lock(int wait) +{ + int result; + + while (!(result = rte_atomic16_cmpset(&_is_lock, 0, 1)) && wait) + rte_pause(); + + return result; +} + +static int +ui_unlock(void) +{ + return rte_atomic16_cmpset(&_is_lock, 1, 0); +} + +/** + * Predefined key values + */ +static uint8_t RSS_KEY[4][40] = { + { 0x6D, 0x5A, 0x56, 0xDA, 0x25, 0x5B, 0x0E, 0xC2, 0x41, 0x67, 0x25, 0x3D, + 0x43, 0xA3, 0x8F, 0xB0, 0xD0, 0xCA, 0x2B, 0xCB, 0xAE, 0x7B, 0x30, 0xB4, + 0x77, 0xCB, 0x2D, 0xA3, 0x80, 0x30, 0xF2, 0x0C, 0x6A, 0x42, 0xB7, 0x3B, + 0xBE, 0xAC, 0x01, 0xFA + }, + { 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, + }, + { 0x6D, 0x5A, 0x6D, 0x5A, 0x6D, 0x5A, 0x6D, 0x5A, 0x6D, 0x5A, 0x6D, 0x5A, + 0x6D, 0x5A, 0x6D, 0x5A, 0x6D, 0x5A, 0x6D, 0x5A, 0x6D, 0x5A, 0x6D, 0x5A, + 0x6D, 0x5A, 0x6D, 0x5A, 0x6D, 0x5A, 0x6D, 0x5A, 0x6D, 0x5A, 0x6D, 0x5A, + 0x6D, 0x5A, 0x6D, 0x5A, + }, + { 0x01, 0x02, 0x03, 0x04, 0x05, 0x06, 0x07, 0x08, 0x09, 0x0A, 0x0B, 0x0C, + 0x0D, 0x0E, 0x0F, 0x10, 0x11, 0x12, 0x13, 0x14, 0x15, 0x16, 0x17, 0x18, + 0x19, 0x1A, 0x1B, 0x1C, 0x1D, 0x1E, 0x1F, 0x20, 0x21, 0x22, 0x23, 0x24, + 0x25, 0x26, 0x27, 0x28, + } +}; + +static void +win_err_addline(const char *text) +{ + int i; + + if (win_err_lines < 5) { + win_err_lines++; + _do_configure = 1; + } else { + free(ui_err_text[0]); + for (i = 1; i < 5; i++) + ui_err_text[i - 1] = ui_err_text[i]; + } + + ui_err_text[win_err_lines - 1] = strdup(text); + _do_repaint_err = 1; +} + +static void +win_err_fini(void) +{ + int i; + for (i = 0; i < (int)win_err_lines; i++) + free(ui_err_text[i]); +} + +/** + * Draw full information for port + */ +static void +win_header_draw(uint8_t port_id) +{ + int i, j; + attr_t a; + + update_port_info(selected_port_id); + + /* Draw title bar */ + if (ui_mode == UI_MODE_PORT) + wattron(win_header, COLOR_PAIR(2)); + else + wattron(win_header, COLOR_PAIR(4)); + + mvwprintw(win_header, 0, 0, "::"); + + for (i = 0; i < nb_ports; i++) { + if (ports[i].enabled) { + const char *name = (ports[i].bond_mode >= 0 ? "BOND" : "SLAVE"); + + if (i == port_id) { + wattron(win_header, A_REVERSE); + wprintw(win_header, " %s-%d ", name, (unsigned) i); + wattroff(win_header, A_REVERSE); + } else + wprintw(win_header, " %s-%d ", name, (unsigned) i); + } + } + + for (i = scr_cols - getcurx(win_header) - 3; i > 0; i--) + waddch(win_header, ':'); + + waddch(win_header, + (ports[port_id].rss ? 'R' : '-')); + waddch(win_header, ports[port_id].bond_mode >= 0 ? + ports[port_id].bond_mode + '0' : '-'); + waddch(win_header, ports[port_id].promiscuous == 0 ? '-' : 'P'); + + if (ui_mode == UI_MODE_PORT) + wattroff(win_header, COLOR_PAIR(2)); + else + wattroff(win_header, COLOR_PAIR(4)); + + /* Redraw RSS-Fn */ + selected_port->rss_type_count = 0; + for (i = 0; i < (int) RTE_DIM(rss_type_table); i++) { + if (selected_port->dev_info.flow_type_rss_offloads + & rss_type_table[i].rss_type) { + selected_port->rss_type_fn[selected_port->rss_type_count].info = + &rss_type_table[i]; + selected_port->rss_type_fn[selected_port->rss_type_count].enabled = + ((selected_port->dev_conf.rx_adv_conf.rss_conf.rss_hf + & rss_type_table[i].rss_type) ? 1 : 0); + selected_port->rss_type_count++; + } + } + + a = (ui_mode == UI_MODE_RSS_FN ? COLOR_PAIR(2) : COLOR_PAIR(1)); + + wattron(win_header, a); + mvwprintw(win_header, 1, 0, "FN: "); + for (i = 0; i < selected_port->rss_type_count; i++) { + if (selected_port->rss_type_fn[i].enabled) + wattron(win_header, COLOR_PAIR(3)); + waddstr(win_header, selected_port->rss_type_fn[i].info->str); + if (selected_port->rss_type_fn[i].enabled) + wattron(win_header, a); + waddch(win_header, ' '); + } + wattroff(win_header, a); + + /* Redraw RSS-Key */ + if (ui_mode == UI_MODE_RSS_KEY) + wattron(win_header, COLOR_PAIR(2)); + mvwprintw(win_header, 2, 0, "KEY: "); + for (i = 0; i < 40; i++) + wprintw(win_header, "%02X", + selected_port->dev_conf.rx_adv_conf.rss_conf.rss_key[i]); + if (ui_mode == UI_MODE_RSS_KEY) + wattroff(win_header, COLOR_PAIR(2)); + + + /* Redraw RETA window */ + int idx, shift; + + if (ui_mode == UI_MODE_RETA) + wattron(win_reta, COLOR_PAIR(2)); + + for (j = 0; j < selected_port->dev_info.reta_size / 16; j++) { + wmove(win_reta, j, 0); + for (i = 0; i < 16; i++) { + idx = (j*16 + i) / RTE_RETA_GROUP_SIZE; + shift = (j*16 + i) % RTE_RETA_GROUP_SIZE; + waddch(win_reta, ACS_VLINE); + wprintw(win_reta, "%d", selected_port->reta_conf[idx].reta[shift]); + } + waddch(win_reta, ACS_VLINE); + } + + if (ui_mode == UI_MODE_RETA) + wattroff(win_reta, COLOR_PAIR(2)); + wnoutrefresh(win_reta); + + + /* Stats repaint */ + if (_do_repaint_stats) { + uint64_t total; + + rte_eth_stats_get(selected_port_id, &(selected_port->stats)); + + wmove(win_stats, 0, 0); + total = 0; + for (i = 0; i < selected_port->nb_rx_queues; i++) { + wprintw(win_stats, "Queue %d: %10"PRIu64 + " (%10" PRIu64 ")\n", i, selected_port->rxq_ipackets[i], + selected_port->stats.q_ipackets[i]); + total += selected_port->rxq_ipackets[i]; + } + + wprintw(win_stats, " Total: %10"PRIu64 + " (%10" PRIu64 ")\n", total, selected_port->stats.ipackets); + + _do_repaint_stats = 0; + wnoutrefresh(win_stats); + } + + if (_do_repaint_err && win_err_lines > 0) { + for (i = 0; i < (int)win_err_lines; i++) { + mvwprintw(win_err, i, 0, ui_err_text[i]); + wclrtoeol(win_err); + } + _do_repaint_err = 0; + wnoutrefresh(win_err); + } + + mvwhline(win_header, win_header_lines - 1, 0, 0, scr_cols); +} + +static void +win_footer_draw(const char *text) +{ + if (text != NULL) + ui_status_text = text; + + wclear(win_footer); + wmove(win_footer, 0, 0); + wprintw(win_footer, ui_status_text); + + wprintw(win_footer, "; [arrows]navigate; [q]quit"); + wnoutrefresh(win_footer); +} + +static void +win_list_draw(void) +{ + unsigned n, i; + + struct pkt_info *pkt_info; + + struct rte_port *port = &ports[focused_port_id]; + struct pkt_record *record = &(port->record); + + n = (win_list_lines > record->count) ? record->count : win_list_lines; + + wmove(win_list, 0, 0); + + for (i = 0; i < n; i++) { + + pkt_info = &record->pkt_info[(record->top - n + i + PKT_RECORD_SIZE) + % PKT_RECORD_SIZE]; + + wmove(win_list, i, 0); + + wprintw(win_list, "%5"PRIu64, (unsigned) pkt_info->n); + waddch(win_list, ACS_VLINE); + + wprintw(win_list, "%2d: ", (unsigned) pkt_info->queue_id); + wprintw(win_list, "%08x ", (unsigned) pkt_info->rss); + + if (pkt_info->ol_flags != 0) { + unsigned rxf; + const char *name; + + for (rxf = 0; rxf < sizeof(pkt_info->ol_flags) * 8; rxf++) { + if ((pkt_info->ol_flags & (1ULL << rxf)) == 0) + continue; + name = rte_get_rx_ol_flag_name(1ULL << rxf); + if (name == NULL) + continue; + wprintw(win_list, "%s ", name); + } + } + wclrtoeol(win_list); + } + + wclrtobot(win_list); + wnoutrefresh(win_list); +} + +static void +ui_repaint(void) +{ + switch (ui_mode) { + default: + win_header_draw(selected_port_id); + wnoutrefresh(win_header); + + if (_do_repaint_list) { + win_list_draw(); + _do_repaint_list = 0; + } + + break; + } + + win_footer_draw(NULL); + _do_repaint = 0; +} + +static void +ui_mode_set(int mode) +{ + ui_mode = (mode + 4) % 4; + switch (ui_mode) { + case UI_MODE_PORT: + ui_status_text = "Port: [p]promiscuous; [c]clear stats"; + break; + case UI_MODE_RSS_FN: + ui_status_text = "RSS Fn: [1-9]toggle fn"; + break; + case UI_MODE_RETA: + ui_status_text = + "RETA: [d]set default; [r]randomize; [0-9]fill table with value"; + break; + case UI_MODE_RSS_KEY: + ui_status_text = "RSS Key: [r]randomize; [1-4]select predefined key"; + break; + } + _do_repaint = 1; +} + +static void +ui_configure(void) +{ + int win_reta_cols = (16 * 2 + 1); + win_reta_lines = selected_port->dev_info.reta_size / 16; + win_stats_lines = selected_port->nb_rx_queues + 1; + win_header_lines = win_reta_lines; + if (win_header_lines < win_stats_lines) + win_header_lines = win_stats_lines; + win_header_lines += 5; + win_list_lines = scr_lines - win_header_lines - win_err_lines - 1; + + if (win_footer == NULL) { + win_footer = newwin(1, scr_cols, scr_lines - 1, 0); + ui_status_text = ""; + wbkgdset(win_footer, COLOR_PAIR(1)); + } else { + wresize(win_footer, 1, scr_cols); + mvwin(win_footer, scr_lines - 1, 0); + } + + if (win_err == NULL) { + win_err = newwin(1, scr_cols, scr_lines - 1 - win_err_lines, 0); + ui_status_text = ""; + wbkgdset(win_err, COLOR_PAIR(5)); + } else { + wresize(win_err, win_err_lines, scr_cols); + mvwin(win_err, scr_lines - win_err_lines - 1, 0); + } + + + if (win_header == NULL) { + win_header = newwin(win_header_lines, scr_cols, 0, 0); + wbkgdset(win_header, COLOR_PAIR(1)); + } else + wresize(win_header, win_header_lines, scr_cols); + + if (win_stats == NULL) + win_stats = subwin(win_header, win_stats_lines, + scr_cols - win_reta_cols - 7, 4, win_reta_cols + 7); + else + wresize(win_stats, win_stats_lines, scr_cols - win_reta_cols - 7); + + if (win_reta == NULL) + win_reta = subwin(win_header, win_reta_lines, win_reta_cols, + 4, 5); + else + wresize(win_reta, win_reta_lines, win_reta_cols); + + if (win_list == NULL) + win_list = newwin(win_list_lines, scr_cols, win_header_lines, 0); + else { + wresize(win_list, win_list_lines, scr_cols); + mvwin(win_list, win_header_lines, 0); + } + + wclear(win_header); + wclear(win_reta); + + _do_configure = 0; + _do_repaint = 1; + _do_repaint_list = 1; + _do_repaint_stats = 1; + _do_repaint_info = 1; + _do_repaint_err = 1; + +} + +static void +ui_resize(void) +{ + if (COLS != scr_cols || LINES != scr_lines) { + scr_cols = COLS; + scr_lines = LINES; + _do_configure = 1; + } +} + +static void +ui_update(void) +{ + if (ui_lock(1)) { + if (_is_ready) { + if (_do_configure) + ui_configure(); + if (_do_repaint) { + ui_repaint(); + doupdate(); + } + } + ui_unlock(); + } +} + +static void +ui_fini(void) +{ + win_err_fini(); + endwin(); +} + +static void +ui_init(void) +{ + initscr(); + start_color(); + init_pair(1, COLOR_YELLOW, COLOR_BLUE); + init_pair(2, COLOR_BLACK, COLOR_CYAN); + init_pair(3, COLOR_BLACK, COLOR_YELLOW); + init_pair(4, COLOR_BLACK, COLOR_WHITE); + init_pair(5, COLOR_YELLOW, COLOR_RED); + + cbreak(); + noecho(); + curs_set(FALSE); + keypad(stdscr, TRUE); + timeout(100); + + scr_cols = COLS; + scr_lines = LINES; + _do_configure = 1; + _is_ready = 1; +} + +static int +port_select(uint8_t port_id) +{ + if (ports[port_id].enabled == 1) { + focused_port_id = selected_port_id = port_id; + + selected_port = &ports[selected_port_id]; + focused_port = &ports[focused_port_id]; + + update_port_info(selected_port_id); + + _do_configure = 1; + _do_repaint = 1; + _do_repaint_info = 1; + _do_repaint_list = 1; + _do_repaint_stats = 1; + return selected_port_id; + } + return -1; +} + +/** + * Record packet from mbuf + */ +struct pkt_info * +record_pkt(uint8_t port_id, int queue_id, struct rte_mbuf *mb) +{ + struct rte_port *port = &ports[port_id]; + struct pkt_record *record = &(port->record); + struct pkt_info *pkt_info = &(record->pkt_info[record->top]); + + ui_lock(1); + + record->count++; + pkt_info->n = mb->udata64; + pkt_info->ol_flags = mb->ol_flags; + pkt_info->queue_id = queue_id; + pkt_info->rss = mb->hash.rss; + + record->top = (record->top + 1) % PKT_RECORD_SIZE; + + _do_repaint = 1; + _do_repaint_list = 1; + _do_repaint_stats = 1; + ui_unlock(); + + return pkt_info; +} + +static void +ui_help(void) { + timeout(-1); + + WINDOW *win_help_border = NULL; + WINDOW *win_help = NULL; + char title[256]; + int page = 0; + int page_prev; + int page_next; + int c; + int cols; + int lines; + int top; + int left; + + endwin(); + initscr(); + ui_resize(); + ui_update(); + + cols = scr_cols > 80 ? 80 : scr_cols; + lines = scr_lines > 25 ? 25 : scr_lines; + top = (scr_lines - lines) / 2; + left = (scr_cols - cols) / 2; + + win_help_border = newwin(lines, cols, top, left); + win_help = derwin(win_help_border, lines - 2, cols - 4, 1, 2); + wbkgdset(win_help_border, COLOR_PAIR(2)); + wbkgdset(win_help, COLOR_PAIR(2)); + + do { + page_prev = (page + RTE_DIM(help_pages) - 1) % RTE_DIM(help_pages); + page_next = (page + 1) % RTE_DIM(help_pages); + + wclear(win_help_border); + box(win_help_border, 0 , 0); + + sprintf(title, "[ Help: %s ]", help_pages[page].title); + mvwprintw(win_help_border, 0, (cols - strlen(title)) / 2 - 2, "%s", + title); + + sprintf(title, "< %s ]", help_pages[page_prev].title); + mvwprintw(win_help_border, lines - 1, 1, "%s", title); + + sprintf(title, "[ Page %d of %d ]", page + 1, (int)RTE_DIM(help_pages)); + mvwprintw(win_help_border, lines - 1, (cols - strlen(title)) / 2 - 2, + "%s", title); + + sprintf(title, "[ %s >", help_pages[page_next].title); + mvwprintw(win_help_border, lines - 1, cols - strlen(title) - 1, "%s", + title); + + wrefresh(win_help_border); + + wclear(win_help); + + wmove(win_help, 0, 0); + + wprintw(win_help, + help_pages[page].body + ); + + wrefresh(win_help); + + switch (c = getch()) { + + case KEY_RESIZE: + endwin(); + initscr(); + ui_resize(); + ui_update(); + + cols = scr_cols > 80 ? 80 : scr_cols; + lines = scr_lines > 25 ? 25 : scr_cols; + top = (scr_lines - lines) / 2; + left = (scr_cols - cols) / 2; + + wresize(win_help_border, lines, cols); + wresize(win_help, lines - 2, cols - 4); + break; + + case KEY_LEFT: + page = page_prev; + break; + + case KEY_RIGHT: + page = page_next; + break; + + default: + c = -1; /* Exit */ + break; + } + + + } while (c != -1); + + timeout(100); + delwin(win_help); + delwin(win_help_border); + _do_configure = 1; + _do_repaint = 1; +} + +/* main processing loop */ +void +ui_loop(void) +{ + int c; + int i; + int port_id; + int refresh = 0; + + ui_init(); + port_select(bond_port_id); + + ui_mode_set(UI_MODE_PORT); + + while ((c = getch()) != 'q') { + switch (c) { + case -1: + refresh++; + if (refresh % 10) { + _do_configure = 1; + } + ui_update(); + break; + + case 'c': + /* clear stats */ + ui_lock(1); + + rte_atomic64_clear(&counter); + for (port_id = 0; port_id < RTE_MAX_ETHPORTS; port_id++) { + if (ports[port_id].enabled) { + ports[port_id].record.count = 0; + ports[port_id].record.top = 0; + for (i = 0; i < ports[port_id].nb_rx_queues; i++) + ports[port_id].rxq_ipackets[i] = 0; + rte_eth_stats_reset(port_id); + } + } + + _do_repaint = 1; + ui_unlock(); + break; + + case ' ': + focused_port->rss ^= 1; + rss_enable(focused_port_id, focused_port->rss); + _do_configure = 1; + _do_repaint = 1; + break; + + case 'p': + if (focused_port->promiscuous) + rte_eth_promiscuous_disable(focused_port_id); + else + rte_eth_promiscuous_enable(focused_port_id); + focused_port->promiscuous ^= 1; + _do_configure = 1; + _do_repaint = 1; + break; + + case KEY_RESIZE: + ui_resize(); + break; + + case KEY_UP: + ui_mode_set(ui_mode - 1); + break; + case KEY_DOWN: + ui_mode_set(ui_mode + 1); + break; + case 'h': + case '?': + ui_help(); + break; + + default: + switch (ui_mode) { + case UI_MODE_PORT: + switch (c) { + case KEY_LEFT: + for (i = 1; i < nb_ports; i++) + if (port_select( + (selected_port_id - i + nb_ports) % nb_ports) + != -1) + break; + break; + + case KEY_RIGHT: + for (i = 1; i < nb_ports; i++) + if (port_select( + (selected_port_id + i + nb_ports) % nb_ports) + != -1) + break; + break; + + default: + i = (c - '1'); + port_select(i); + break; + } + break; + + case UI_MODE_RSS_KEY: + switch (c) { + case 'r': + i = key_set_random(focused_port_id, 40); + if (i < 0) + win_err_addline("Cannot update RSS key"); + break; + + default: + c = c - '1'; + if (c >= 0 && c < (int) RTE_DIM(RSS_KEY)) { + i = key_set(focused_port_id, RSS_KEY[c], 40); + if (i < 0) + win_err_addline("Cannot update RSS key"); + } + break; + } + update_port_info(focused_port_id); + break; + + case UI_MODE_RETA: + switch (c) { + case 'r': + reta_set_random(focused_port_id); + break; + case 'd': + reta_set_default(focused_port_id); + break; + default: + c = c - '0'; + reta_set_all(focused_port_id, c); + break; + } + break; + + case UI_MODE_RSS_FN: + switch (c) { + case 'a': + /* Set all */ + focused_port->dev_conf.rx_adv_conf.rss_conf.rss_hf = + ports[focused_port_id].dev_info.flow_type_rss_offloads; + rte_eth_dev_rss_hash_update(focused_port_id, + &focused_port->dev_conf.rx_adv_conf.rss_conf); + break; + + default: + c -= '1'; + if (c >= 0 && c < focused_port->rss_type_count) { + uint64_t rss_type = 0; + + focused_port->rss_type_fn[c].enabled ^= 1; /* toggle */ + + for (i = 0; i < focused_port->rss_type_count; i++) + if (focused_port->rss_type_fn[i].enabled) + rss_type |= + focused_port->rss_type_fn[i].info->rss_type; + focused_port->dev_conf.rx_adv_conf.rss_conf.rss_hf = + rss_type; + + rte_eth_dev_rss_hash_update(focused_port_id, + &focused_port->dev_conf.rx_adv_conf.rss_conf); + + } + } + break; + + } + _do_repaint = 1; + break; + } + } + ui_fini(); +} -- 1.7.9.5 ^ permalink raw reply [flat|nested] 126+ messages in thread
* [dpdk-dev] [PATCH v2 7/8] doc: fixed spellings and typos 2015-06-19 14:13 ` [dpdk-dev] [PATCH v2 " Tomasz Kulasek ` (5 preceding siblings ...) 2015-06-19 14:13 ` [dpdk-dev] [PATCH v2 6/8] examples: dynamic rss configuration for bonding Tomasz Kulasek @ 2015-06-19 14:13 ` Tomasz Kulasek 2015-06-19 14:13 ` [dpdk-dev] [PATCH v2 8/8] doc: dynamic rss configuration for bonding Tomasz Kulasek ` (2 subsequent siblings) 9 siblings, 0 replies; 126+ messages in thread From: Tomasz Kulasek @ 2015-06-19 14:13 UTC (permalink / raw) To: dev Signed-off-by: Tomasz Kulasek <tomaszx.kulasek@intel.com> --- .../prog_guide/link_bonding_poll_mode_drv_lib.rst | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/doc/guides/prog_guide/link_bonding_poll_mode_drv_lib.rst b/doc/guides/prog_guide/link_bonding_poll_mode_drv_lib.rst index 96e554f..03baf90 100644 --- a/doc/guides/prog_guide/link_bonding_poll_mode_drv_lib.rst +++ b/doc/guides/prog_guide/link_bonding_poll_mode_drv_lib.rst @@ -188,7 +188,7 @@ conditions are not met. If a user wishes to monitor individual slaves then they must register callbacks with that slave directly. The link bonding library also supports devices which do not implement link -status change interrupts, this is achieve by polling the devices link status at +status change interrupts, this is achieved by polling the devices link status at a defined period which is set using the ``rte_eth_bond_link_monitoring_set`` API, the default polling interval is 10ms. When a device is added as a slave to a bonding device it is determined using the ``RTE_PCI_DRV_INTR_LSC`` flag @@ -286,7 +286,7 @@ and UDP protocols for load balancing. Using Link Bonding Devices -------------------------- -The librte_pmd_bond library support two modes of device creation, the libraries +The librte_pmd_bond library supports two modes of device creation, the libraries export full C API or using the EAL command line to statically configure link bonding devices at application startup. Using the EAL option it is possible to use link bonding functionality transparently without specific knowledge of the @@ -299,7 +299,7 @@ Using the Poll Mode Driver from an Application Using the librte_pmd_bond libraries API it is possible to dynamically create and manage link bonding device from within any application. Link bonding -device are created using the ``rte_eth_bond_create`` API which requires a +devices are created using the ``rte_eth_bond_create`` API which requires a unique device name, the link bonding mode to initial the device in and finally the socket Id which to allocate the devices resources onto. After successful creation of a bonding device it must be configured using the generic Ethernet @@ -362,7 +362,7 @@ The different options are: mode=2 * slave: Defines the PMD device which will be added as slave to the bonded - device. This option can be selected multiple time, for each device to be + device. This option can be selected multiple times, for each device to be added as a slave. Physical devices should be specified using their PCI address, in the format domain:bus:devid.function -- 1.7.9.5 ^ permalink raw reply [flat|nested] 126+ messages in thread
* [dpdk-dev] [PATCH v2 8/8] doc: dynamic rss configuration for bonding 2015-06-19 14:13 ` [dpdk-dev] [PATCH v2 " Tomasz Kulasek ` (6 preceding siblings ...) 2015-06-19 14:13 ` [dpdk-dev] [PATCH v2 7/8] doc: fixed spellings and typos Tomasz Kulasek @ 2015-06-19 14:13 ` Tomasz Kulasek 2015-06-26 7:33 ` [dpdk-dev] [PATCH v2 0/8] Dynamic RSS Configuration for Bonding Doherty, Declan 2015-06-29 14:50 ` [dpdk-dev] [PATCH v3 " Tomasz Kulasek 9 siblings, 0 replies; 126+ messages in thread From: Tomasz Kulasek @ 2015-06-19 14:13 UTC (permalink / raw) To: dev Documentation update about implementation details and requirements for Dynamic RSS Configuration for Bonding. Signed-off-by: Tomasz Kulasek <tomaszx.kulasek@intel.com> --- .../prog_guide/link_bonding_poll_mode_drv_lib.rst | 34 ++++++++++++++++++-- 1 file changed, 32 insertions(+), 2 deletions(-) diff --git a/doc/guides/prog_guide/link_bonding_poll_mode_drv_lib.rst b/doc/guides/prog_guide/link_bonding_poll_mode_drv_lib.rst index 03baf90..46f0296 100644 --- a/doc/guides/prog_guide/link_bonding_poll_mode_drv_lib.rst +++ b/doc/guides/prog_guide/link_bonding_poll_mode_drv_lib.rst @@ -1,5 +1,5 @@ .. BSD LICENSE - Copyright(c) 2010-2014 Intel Corporation. All rights reserved. + Copyright(c) 2010-2015 Intel Corporation. All rights reserved. All rights reserved. Redistribution and use in source and binary forms, with or without @@ -173,7 +173,28 @@ After a slave device is added to a bonded device slave is stopped using ``rte_eth_dev_stop`` and then reconfigured using ``rte_eth_dev_configure`` the RX and TX queues are also reconfigured using ``rte_eth_tx_queue_setup`` / ``rte_eth_rx_queue_setup`` with the parameters use to configure the bonding -device. +device. If RSS is enabled for bonding device, this mode is also enabled on new +slave and configured as well. + +Setting up multi-queue mode for bonding device to RSS, makes it fully +RSS-capable, so all slaves are synchronized with its configuration. This mode is +intended to provide RSS configuration on slaves transparent for client +application implementation. + +Bonding device stores its own version of RSS settings i.e. RETA, RSS hash +function and RSS key, used to set up its slaves. That let to define the meaning +of RSS configuration of bonding device as desired configuration of whole bonding +(as one unit), without pointing any of slave inside. It is required to ensure +consistency and made it more errorproof. + +RSS hash function set for bonding device, is a maximal set of RSS hash functions +supported by all bonded slaves. RETA size is a GCD of all its RETA's sizes, so +it can be easily used as a pattern providing expected behavior, even if slave +RETAs' sizes are different. If RSS Key is not set for bonded device, it's not +changed on the slaves and default key for device is used. + +All settings are managed through the bonding port API and always are propagated +in one direction (from bonding to slaves). Link Status Change Interrupts / Polling ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ @@ -207,6 +228,15 @@ these parameters. A bonding device must have a minimum of one slave before the bonding device itself can be started. +To use a bonding device dynamic RSS configuration feature effectively, it is +also required, that all slaves should be RSS-capable and support, at least one +common hash function available for each of them. Changing RSS key is only +possible, when all slave devices support the same key size. + +To prevent inconsistency on how slaves process packets, once a device is added +to a bonding device, RSS configuration should be managed through the bonding +device API, and not directly on the slave. + Like all other PMD, all functions exported by a PMD are lock-free functions that are assumed not to be invoked in parallel on different logical cores to work on the same target object. -- 1.7.9.5 ^ permalink raw reply [flat|nested] 126+ messages in thread
* Re: [dpdk-dev] [PATCH v2 0/8] Dynamic RSS Configuration for Bonding 2015-06-19 14:13 ` [dpdk-dev] [PATCH v2 " Tomasz Kulasek ` (7 preceding siblings ...) 2015-06-19 14:13 ` [dpdk-dev] [PATCH v2 8/8] doc: dynamic rss configuration for bonding Tomasz Kulasek @ 2015-06-26 7:33 ` Doherty, Declan 2015-06-28 21:54 ` Thomas Monjalon 2015-06-29 14:50 ` [dpdk-dev] [PATCH v3 " Tomasz Kulasek 9 siblings, 1 reply; 126+ messages in thread From: Doherty, Declan @ 2015-06-26 7:33 UTC (permalink / raw) To: Kulasek, TomaszX, dev > -----Original Message----- > From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Tomasz Kulasek > Sent: Friday, June 19, 2015 7:13 AM > To: dev@dpdk.org > Subject: [dpdk-dev] [PATCH v2 0/8] Dynamic RSS Configuration for Bonding > > OVERVIEW > -------- > 1) Setting .rxmode.mq_mode for bonding device to ETH_MQ_RX_RSS makes > bonding > device fully RSS-capable, so all slaves are synchronized with its configuration. > This mode is intended to provide RSS configuration as known from "dynamic RSS > configuration for one port" and made slaves transparent for client application > implementation. > > 2) If .rxmode.mq_mode for bonding device isn't ETH_MQ_RX_RSS, slaves are not > synchronized. That provides an ability to configure them manually. This mode may > be useful when application wants to manage RSS in an unusual way and the > consistency of RSS configuration for slaves isn't required. > > Turning on/off RSS mode for slaves when bonding is started is not possible. > Other RSS configuration is propagated over slaves, when bonding device API is > used to do it. > > v2 changes: > - added support for keys other than 40 bytes long, > - now, if RSS key is not set for bonding, it is not set also for slaves, > - fix - full initial RSS configuration before any slave is added was not > possible due to the initially zeroed flow_type_rss_offloads for bonding, > - fix - changed error to warning when slave is synchronizing due to the > bonding's initial configuration (to allow use slaves' drivers not supporting > dynamic RSS configuration in bonding), > - some code cleanups, > - updated documentation, > > Tomasz Kulasek (8): > bond: rss dynamic configuration > ring: dynamic rss configuration > test: dynamic rss configuration > bond: queue stats mapping > ring: queue stats mapping set dummy implementation > examples: dynamic rss configuration for bonding > doc: fixed spellings and typos > doc: dynamic rss configuration for bonding > > app/test/Makefile | 1 + > app/test/test_link_bonding_rssconf.c | 674 ++++++++++++++ > .../prog_guide/link_bonding_poll_mode_drv_lib.rst | 42 +- > drivers/net/bonding/rte_eth_bond_api.c | 27 + > drivers/net/bonding/rte_eth_bond_pmd.c | 239 ++++- > drivers/net/bonding/rte_eth_bond_private.h | 12 + > drivers/net/ring/rte_eth_ring.c | 133 ++- > examples/bond_rss/Makefile | 59 ++ > examples/bond_rss/bondrss.c | 293 +++++++ > examples/bond_rss/bondrss.h | 163 ++++ > examples/bond_rss/config.c | 251 ++++++ > examples/bond_rss/ui.c | 920 ++++++++++++++++++++ > 12 files changed, 2789 insertions(+), 25 deletions(-) > create mode 100644 app/test/test_link_bonding_rssconf.c > create mode 100644 examples/bond_rss/Makefile > create mode 100644 examples/bond_rss/bondrss.c > create mode 100644 examples/bond_rss/bondrss.h > create mode 100644 examples/bond_rss/config.c > create mode 100644 examples/bond_rss/ui.c > > -- > 1.7.9.5 Acked-by : Declan Doherty <declan.doherty@intel.com> ^ permalink raw reply [flat|nested] 126+ messages in thread
* Re: [dpdk-dev] [PATCH v2 0/8] Dynamic RSS Configuration for Bonding 2015-06-26 7:33 ` [dpdk-dev] [PATCH v2 0/8] Dynamic RSS Configuration for Bonding Doherty, Declan @ 2015-06-28 21:54 ` Thomas Monjalon 0 siblings, 0 replies; 126+ messages in thread From: Thomas Monjalon @ 2015-06-28 21:54 UTC (permalink / raw) To: Kulasek, TomaszX; +Cc: dev 2015-06-26 07:33, Doherty, Declan: > From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Tomasz Kulasek > > Acked-by : Declan Doherty <declan.doherty@intel.com> Please use checkpatch to fix formatting. ^ permalink raw reply [flat|nested] 126+ messages in thread
* [dpdk-dev] [PATCH v3 0/8] Dynamic RSS Configuration for Bonding 2015-06-19 14:13 ` [dpdk-dev] [PATCH v2 " Tomasz Kulasek ` (8 preceding siblings ...) 2015-06-26 7:33 ` [dpdk-dev] [PATCH v2 0/8] Dynamic RSS Configuration for Bonding Doherty, Declan @ 2015-06-29 14:50 ` Tomasz Kulasek 2015-06-29 14:50 ` [dpdk-dev] [PATCH v3 1/8] bonding: rss dynamic configuration Tomasz Kulasek ` (9 more replies) 9 siblings, 10 replies; 126+ messages in thread From: Tomasz Kulasek @ 2015-06-29 14:50 UTC (permalink / raw) To: dev OVERVIEW -------- 1) Setting .rxmode.mq_mode for bonding device to ETH_MQ_RX_RSS makes bonding device fully RSS-capable, so all slaves are synchronized with its configuration. This mode is intended to provide RSS configuration as known from "dynamic RSS configuration for one port" and made slaves transparent for client application implementation. 2) If .rxmode.mq_mode for bonding device isn't ETH_MQ_RX_RSS, slaves are not synchronized. That provides an ability to configure them manually. This mode may be useful when application wants to manage RSS in an unusual way and the consistency of RSS configuration for slaves isn't required. Turning on/off RSS mode for slaves when bonding is started is not possible. Other RSS configuration is propagated over slaves, when bonding device API is used to do it. v3 changes: - checkpatch cleanups v2 changes: - added support for keys other than 40 bytes long, - now, if RSS key is not set for bonding, it is not set also for slaves, - fix - full initial RSS configuration before any slave is added was not possible due to the initially zeroed flow_type_rss_offloads for bonding, - fix - changed error to warning when slave is synchronizing due to the bonding's initial configuration (to allow use slaves' drivers not supporting dynamic RSS configuration in bonding), - some code cleanups, - updated documentation, Tomasz Kulasek (8): bonding: rss dynamic configuration ring: dynamic rss configuration test: dynamic rss configuration bonding: queue stats mapping ring: queue stats mapping set dummy implementation examples: dynamic rss configuration for bonding doc: fixed spellings and typos doc: dynamic rss configuration for bonding app/test/Makefile | 1 + app/test/test_link_bonding_rssconf.c | 674 ++++++++++++++ .../prog_guide/link_bonding_poll_mode_drv_lib.rst | 42 +- drivers/net/bonding/rte_eth_bond_api.c | 28 + drivers/net/bonding/rte_eth_bond_pmd.c | 235 ++++- drivers/net/bonding/rte_eth_bond_private.h | 12 + drivers/net/ring/rte_eth_ring.c | 130 ++- examples/bond_rss/Makefile | 59 ++ examples/bond_rss/bondrss.c | 293 ++++++ examples/bond_rss/bondrss.h | 163 ++++ examples/bond_rss/config.c | 251 ++++++ examples/bond_rss/ui.c | 945 ++++++++++++++++++++ 12 files changed, 2808 insertions(+), 25 deletions(-) create mode 100644 app/test/test_link_bonding_rssconf.c create mode 100644 examples/bond_rss/Makefile create mode 100644 examples/bond_rss/bondrss.c create mode 100644 examples/bond_rss/bondrss.h create mode 100644 examples/bond_rss/config.c create mode 100644 examples/bond_rss/ui.c -- 1.7.9.5 ^ permalink raw reply [flat|nested] 126+ messages in thread
* [dpdk-dev] [PATCH v3 1/8] bonding: rss dynamic configuration 2015-06-29 14:50 ` [dpdk-dev] [PATCH v3 " Tomasz Kulasek @ 2015-06-29 14:50 ` Tomasz Kulasek 2015-06-29 14:50 ` [dpdk-dev] [PATCH v3 2/8] ring: dynamic rss configuration Tomasz Kulasek ` (8 subsequent siblings) 9 siblings, 0 replies; 126+ messages in thread From: Tomasz Kulasek @ 2015-06-29 14:50 UTC (permalink / raw) To: dev Bonding device implements independent management of RSS settings. It stores its own copies of settings i.e. RETA, RSS hash function and RSS key. It’s required to ensure consistency. 1) RSS hash function set for bonding device is maximal set of RSS hash functions supported by all bonded devices. That mean, to have RSS support for bonding, all slaves should be RSS-capable. 2) RSS key is propagated over the slaves "as is". 3) RETA for bonding is an internal table managed by bonding API, and is used as a pattern to set up slaves. Its size is GCD of all RETA sizes, so it can be easily used as a pattern providing expected behavior, even if slaves RETA sizes are different. Signed-off-by: Tomasz Kulasek <tomaszx.kulasek@intel.com> --- drivers/net/bonding/rte_eth_bond_api.c | 28 ++++ drivers/net/bonding/rte_eth_bond_pmd.c | 205 ++++++++++++++++++++++++++-- drivers/net/bonding/rte_eth_bond_private.h | 12 ++ 3 files changed, 231 insertions(+), 14 deletions(-) diff --git a/drivers/net/bonding/rte_eth_bond_api.c b/drivers/net/bonding/rte_eth_bond_api.c index d810ec4..22eb575 100644 --- a/drivers/net/bonding/rte_eth_bond_api.c +++ b/drivers/net/bonding/rte_eth_bond_api.c @@ -303,6 +303,9 @@ rte_eth_bond_create(const char *name, uint8_t mode, uint8_t socket_id) internals->rx_offload_capa = 0; internals->tx_offload_capa = 0; + /* Initially allow to choose any offload type */ + internals->flow_type_rss_offloads = ETH_RSS_PROTO_MASK; + memset(internals->active_slaves, 0, sizeof(internals->active_slaves)); memset(internals->slaves, 0, sizeof(internals->slaves)); @@ -366,6 +369,11 @@ __eth_bond_slave_add_lock_free(uint8_t bonded_port_id, uint8_t slave_port_id) rte_eth_dev_info_get(slave_port_id, &dev_info); + /* We need to store slaves reta_size to be able to synchronize RETA for all + * slave devices even if its sizes are different. + */ + internals->slaves[internals->slave_count].reta_size = dev_info.reta_size; + if (internals->slave_count < 1) { /* if MAC is not user defined then use MAC of first slave add to * bonded device */ @@ -379,9 +387,16 @@ __eth_bond_slave_add_lock_free(uint8_t bonded_port_id, uint8_t slave_port_id) /* Make primary slave */ internals->primary_port = slave_port_id; + /* Inherit queues settings from first slave */ + internals->nb_rx_queues = slave_eth_dev->data->nb_rx_queues; + internals->nb_tx_queues = slave_eth_dev->data->nb_tx_queues; + + internals->reta_size = dev_info.reta_size; + /* Take the first dev's offload capabilities */ internals->rx_offload_capa = dev_info.rx_offload_capa; internals->tx_offload_capa = dev_info.tx_offload_capa; + internals->flow_type_rss_offloads = dev_info.flow_type_rss_offloads; } else { /* Check slave link properties are supported if props are set, @@ -400,8 +415,19 @@ __eth_bond_slave_add_lock_free(uint8_t bonded_port_id, uint8_t slave_port_id) } internals->rx_offload_capa &= dev_info.rx_offload_capa; internals->tx_offload_capa &= dev_info.tx_offload_capa; + internals->flow_type_rss_offloads &= dev_info.flow_type_rss_offloads; + + /* RETA size is GCD of all slaves RETA sizes, so, if all sizes will be + * the power of 2, the lower one is GCD + */ + if (internals->reta_size > dev_info.reta_size) + internals->reta_size = dev_info.reta_size; + } + bonded_eth_dev->data->dev_conf.rx_adv_conf.rss_conf.rss_hf &= + internals->flow_type_rss_offloads; + internals->slave_count++; /* Update all slave devices MACs*/ @@ -528,6 +554,8 @@ __eth_bond_slave_remove_lock_free(uint8_t bonded_port_id, uint8_t slave_port_id) if (internals->slave_count == 0) { internals->rx_offload_capa = 0; internals->tx_offload_capa = 0; + internals->flow_type_rss_offloads = ETH_RSS_PROTO_MASK; + internals->reta_size = 0; } return 0; } diff --git a/drivers/net/bonding/rte_eth_bond_pmd.c b/drivers/net/bonding/rte_eth_bond_pmd.c index 989e878..cd23f42 100644 --- a/drivers/net/bonding/rte_eth_bond_pmd.c +++ b/drivers/net/bonding/rte_eth_bond_pmd.c @@ -1310,6 +1310,23 @@ slave_configure(struct rte_eth_dev *bonded_eth_dev, if (slave_eth_dev->driver->pci_drv.drv_flags & RTE_PCI_DRV_INTR_LSC) slave_eth_dev->data->dev_conf.intr_conf.lsc = 1; + /* If RSS is enabled for bonding, try to enable it for slaves */ + if (bonded_eth_dev->data->dev_conf.rxmode.mq_mode & ETH_MQ_RX_RSS) { + if (bonded_eth_dev->data->dev_conf.rx_adv_conf.rss_conf.rss_key_len + != 0) { + slave_eth_dev->data->dev_conf.rx_adv_conf.rss_conf.rss_key_len = + bonded_eth_dev->data->dev_conf.rx_adv_conf.rss_conf.rss_key_len; + slave_eth_dev->data->dev_conf.rx_adv_conf.rss_conf.rss_key = + bonded_eth_dev->data->dev_conf.rx_adv_conf.rss_conf.rss_key; + } else { + slave_eth_dev->data->dev_conf.rx_adv_conf.rss_conf.rss_key = NULL; + } + + slave_eth_dev->data->dev_conf.rx_adv_conf.rss_conf.rss_hf = + bonded_eth_dev->data->dev_conf.rx_adv_conf.rss_conf.rss_hf; + slave_eth_dev->data->dev_conf.rxmode.mq_mode |= ETH_MQ_RX_RSS; + } + /* Configure device */ errval = rte_eth_dev_configure(slave_eth_dev->data->port_id, bonded_eth_dev->data->nb_rx_queues, @@ -1361,6 +1378,30 @@ slave_configure(struct rte_eth_dev *bonded_eth_dev, return -1; } + /* If RSS is enabled for bonding, synchronize RETA */ + if (bonded_eth_dev->data->dev_conf.rxmode.mq_mode & ETH_MQ_RX_RSS) { + int i; + struct bond_dev_private *internals; + + internals = bonded_eth_dev->data->dev_private; + + for (i = 0; i < internals->slave_count; i++) { + if (internals->slaves[i].port_id == slave_eth_dev->data->port_id) { + errval = rte_eth_dev_rss_reta_update( + slave_eth_dev->data->port_id, + &internals->reta_conf[0], + internals->slaves[i].reta_size); + if (errval != 0) { + RTE_LOG(WARNING, PMD, + "rte_eth_dev_rss_reta_update on slave port %d fails (err %d)." + " RSS Configuration for bonding may be inconsistent.\n", + slave_eth_dev->data->port_id, errval); + } + break; + } + } + } + /* If lsc interrupt is set, check initial slave's link status */ if (slave_eth_dev->driver->pci_drv.drv_flags & RTE_PCI_DRV_INTR_LSC) bond_ethdev_lsc_event_callback(slave_eth_dev->data->port_id, @@ -1578,6 +1619,9 @@ bond_ethdev_info(struct rte_eth_dev *dev, struct rte_eth_dev_info *dev_info) dev_info->rx_offload_capa = internals->rx_offload_capa; dev_info->tx_offload_capa = internals->tx_offload_capa; + dev_info->flow_type_rss_offloads = internals->flow_type_rss_offloads; + + dev_info->reta_size = internals->reta_size; } static int @@ -1959,21 +2003,132 @@ bond_ethdev_lsc_event_callback(uint8_t port_id, enum rte_eth_event_type type, } } +static int +bond_ethdev_rss_reta_update(struct rte_eth_dev *dev, + struct rte_eth_rss_reta_entry64 *reta_conf, uint16_t reta_size) +{ + unsigned i, j; + int result = 0; + int slave_reta_size; + unsigned reta_count; + struct bond_dev_private *internals = dev->data->dev_private; + + if (reta_size != internals->reta_size) + return -EINVAL; + + /* Copy RETA table */ + reta_count = reta_size / RTE_RETA_GROUP_SIZE; + + for (i = 0; i < reta_count; i++) { + internals->reta_conf[i].mask = reta_conf[i].mask; + for (j = 0; j < RTE_RETA_GROUP_SIZE; j++) + if ((reta_conf[i].mask >> j) & 0x01) + internals->reta_conf[i].reta[j] = reta_conf[i].reta[j]; + } + + /* Fill rest of array */ + for (; i < RTE_DIM(internals->reta_conf); i += reta_count) + memcpy(&internals->reta_conf[i], &internals->reta_conf[0], + sizeof(internals->reta_conf[0]) * reta_count); + + /* Propagate RETA over slaves */ + for (i = 0; i < internals->slave_count; i++) { + slave_reta_size = internals->slaves[i].reta_size; + result = rte_eth_dev_rss_reta_update(internals->slaves[i].port_id, + &internals->reta_conf[0], slave_reta_size); + if (result < 0) + return result; + } + + return 0; +} + +static int +bond_ethdev_rss_reta_query(struct rte_eth_dev *dev, + struct rte_eth_rss_reta_entry64 *reta_conf, uint16_t reta_size) +{ + int i, j; + struct bond_dev_private *internals = dev->data->dev_private; + + if (reta_size != internals->reta_size) + return -EINVAL; + + /* Copy RETA table */ + for (i = 0; i < reta_size / RTE_RETA_GROUP_SIZE; i++) + for (j = 0; j < RTE_RETA_GROUP_SIZE; j++) + if ((reta_conf[i].mask >> j) & 0x01) + reta_conf[i].reta[j] = internals->reta_conf[i].reta[j]; + + return 0; +} + +static int +bond_ethdev_rss_hash_update(struct rte_eth_dev *dev, + struct rte_eth_rss_conf *rss_conf) +{ + int i, result = 0; + struct bond_dev_private *internals = dev->data->dev_private; + struct rte_eth_rss_conf bond_rss_conf; + + memcpy(&bond_rss_conf, rss_conf, sizeof(struct rte_eth_rss_conf)); + + bond_rss_conf.rss_hf &= internals->flow_type_rss_offloads; + + if (bond_rss_conf.rss_hf != 0) + dev->data->dev_conf.rx_adv_conf.rss_conf.rss_hf = bond_rss_conf.rss_hf; + + if (bond_rss_conf.rss_key && bond_rss_conf.rss_key_len < + sizeof(internals->rss_key)) { + if (bond_rss_conf.rss_key_len == 0) + bond_rss_conf.rss_key_len = 40; + internals->rss_key_len = bond_rss_conf.rss_key_len; + memcpy(internals->rss_key, bond_rss_conf.rss_key, + internals->rss_key_len); + } + + for (i = 0; i < internals->slave_count; i++) { + result = rte_eth_dev_rss_hash_update(internals->slaves[i].port_id, + &bond_rss_conf); + if (result < 0) + return result; + } + + return 0; +} + +static int +bond_ethdev_rss_hash_conf_get(struct rte_eth_dev *dev, + struct rte_eth_rss_conf *rss_conf) +{ + struct bond_dev_private *internals = dev->data->dev_private; + + rss_conf->rss_hf = dev->data->dev_conf.rx_adv_conf.rss_conf.rss_hf; + rss_conf->rss_key_len = internals->rss_key_len; + if (rss_conf->rss_key) + memcpy(rss_conf->rss_key, internals->rss_key, internals->rss_key_len); + + return 0; +} + struct eth_dev_ops default_dev_ops = { - .dev_start = bond_ethdev_start, - .dev_stop = bond_ethdev_stop, - .dev_close = bond_ethdev_close, - .dev_configure = bond_ethdev_configure, - .dev_infos_get = bond_ethdev_info, - .rx_queue_setup = bond_ethdev_rx_queue_setup, - .tx_queue_setup = bond_ethdev_tx_queue_setup, - .rx_queue_release = bond_ethdev_rx_queue_release, - .tx_queue_release = bond_ethdev_tx_queue_release, - .link_update = bond_ethdev_link_update, - .stats_get = bond_ethdev_stats_get, - .stats_reset = bond_ethdev_stats_reset, - .promiscuous_enable = bond_ethdev_promiscuous_enable, - .promiscuous_disable = bond_ethdev_promiscuous_disable + .dev_start = bond_ethdev_start, + .dev_stop = bond_ethdev_stop, + .dev_close = bond_ethdev_close, + .dev_configure = bond_ethdev_configure, + .dev_infos_get = bond_ethdev_info, + .rx_queue_setup = bond_ethdev_rx_queue_setup, + .tx_queue_setup = bond_ethdev_tx_queue_setup, + .rx_queue_release = bond_ethdev_rx_queue_release, + .tx_queue_release = bond_ethdev_tx_queue_release, + .link_update = bond_ethdev_link_update, + .stats_get = bond_ethdev_stats_get, + .stats_reset = bond_ethdev_stats_reset, + .promiscuous_enable = bond_ethdev_promiscuous_enable, + .promiscuous_disable = bond_ethdev_promiscuous_disable, + .reta_update = bond_ethdev_rss_reta_update, + .reta_query = bond_ethdev_rss_reta_query, + .rss_hash_update = bond_ethdev_rss_hash_update, + .rss_hash_conf_get = bond_ethdev_rss_hash_conf_get }; static int @@ -2054,6 +2209,28 @@ bond_ethdev_configure(struct rte_eth_dev *dev) int arg_count; uint8_t port_id = dev - rte_eth_devices; + static const uint8_t default_rss_key[40] = { + 0x6D, 0x5A, 0x56, 0xDA, 0x25, 0x5B, 0x0E, 0xC2, 0x41, 0x67, 0x25, 0x3D, + 0x43, 0xA3, 0x8F, 0xB0, 0xD0, 0xCA, 0x2B, 0xCB, 0xAE, 0x7B, 0x30, 0xB4, + 0x77, 0xCB, 0x2D, 0xA3, 0x80, 0x30, 0xF2, 0x0C, 0x6A, 0x42, 0xB7, 0x3B, + 0xBE, 0xAC, 0x01, 0xFA + }; + + unsigned i, j; + + /* If RSS is enabled, fill table and key with default values */ + if (dev->data->dev_conf.rxmode.mq_mode & ETH_MQ_RX_RSS) { + dev->data->dev_conf.rx_adv_conf.rss_conf.rss_key = internals->rss_key; + dev->data->dev_conf.rx_adv_conf.rss_conf.rss_key_len = 0; + memcpy(internals->rss_key, default_rss_key, 40); + + for (i = 0; i < RTE_DIM(internals->reta_conf); i++) { + internals->reta_conf[i].mask = ~0LL; + for (j = 0; j < RTE_RETA_GROUP_SIZE; j++) + internals->reta_conf[i].reta[j] = j % dev->data->nb_rx_queues; + } + } + /* * if no kvlist, it means that this bonded device has been created * through the bonding api. diff --git a/drivers/net/bonding/rte_eth_bond_private.h b/drivers/net/bonding/rte_eth_bond_private.h index 45e5c65..98bb64d 100644 --- a/drivers/net/bonding/rte_eth_bond_private.h +++ b/drivers/net/bonding/rte_eth_bond_private.h @@ -103,6 +103,8 @@ struct bond_slave_details { uint8_t last_link_status; /**< Port Id of slave eth_dev */ struct ether_addr persisted_mac_addr; + + uint16_t reta_size; }; @@ -155,6 +157,16 @@ struct bond_dev_private { uint32_t rx_offload_capa; /** Rx offload capability */ uint32_t tx_offload_capa; /** Tx offload capability */ + /** Bit mask of RSS offloads, the bit offset also means flow type */ + uint64_t flow_type_rss_offloads; + + uint16_t reta_size; + struct rte_eth_rss_reta_entry64 reta_conf[ETH_RSS_RETA_SIZE_512 / + RTE_RETA_GROUP_SIZE]; + + uint8_t rss_key[52]; /**< 52-byte hash key buffer. */ + uint8_t rss_key_len; /**< hash key length in bytes. */ + struct rte_kvargs *kvlist; uint8_t slave_update_idx; }; -- 1.7.9.5 ^ permalink raw reply [flat|nested] 126+ messages in thread
* [dpdk-dev] [PATCH v3 2/8] ring: dynamic rss configuration 2015-06-29 14:50 ` [dpdk-dev] [PATCH v3 " Tomasz Kulasek 2015-06-29 14:50 ` [dpdk-dev] [PATCH v3 1/8] bonding: rss dynamic configuration Tomasz Kulasek @ 2015-06-29 14:50 ` Tomasz Kulasek 2015-07-13 12:36 ` Thomas Monjalon 2015-06-29 14:50 ` [dpdk-dev] [PATCH v3 3/8] test: " Tomasz Kulasek ` (7 subsequent siblings) 9 siblings, 1 reply; 126+ messages in thread From: Tomasz Kulasek @ 2015-06-29 14:50 UTC (permalink / raw) To: dev This implementation allows to set and read RSS configuration for ring device, and is used to validate right values propagation over the slaves, in test units for dynamic RSS configuration for bonding. It have no impact on packet processing by ring device. Signed-off-by: Tomasz Kulasek <tomaszx.kulasek@intel.com> --- drivers/net/ring/rte_eth_ring.c | 120 +++++++++++++++++++++++++++++++++++++-- 1 file changed, 116 insertions(+), 4 deletions(-) diff --git a/drivers/net/ring/rte_eth_ring.c b/drivers/net/ring/rte_eth_ring.c index ada025f..29d7bd8 100644 --- a/drivers/net/ring/rte_eth_ring.c +++ b/drivers/net/ring/rte_eth_ring.c @@ -39,6 +39,7 @@ #include <rte_string_fns.h> #include <rte_dev.h> #include <rte_kvargs.h> +#include <rte_spinlock.h> #define ETH_RING_NUMA_NODE_ACTION_ARG "nodeaction" #define ETH_RING_ACTION_CREATE "CREATE" @@ -66,6 +67,17 @@ struct pmd_internals { struct ring_queue tx_ring_queues[RTE_PMD_RING_MAX_TX_RINGS]; struct ether_addr address; + + /** Bit mask of RSS offloads, the bit offset also means flow type */ + uint64_t flow_type_rss_offloads; + + rte_spinlock_t rss_lock; + + uint16_t reta_size; + struct rte_eth_rss_reta_entry64 reta_conf[ETH_RSS_RETA_SIZE_128 / + RTE_RETA_GROUP_SIZE]; + + uint8_t rss_key[40]; /**< 40-byte hash key. */ }; @@ -173,6 +185,8 @@ eth_dev_info(struct rte_eth_dev *dev, dev_info->max_tx_queues = (uint16_t)internals->nb_tx_queues; dev_info->min_rx_bufsize = 0; dev_info->pci_dev = NULL; + dev_info->flow_type_rss_offloads = internals->flow_type_rss_offloads; + dev_info->reta_size = internals->reta_size; } static void @@ -234,6 +248,91 @@ static int eth_link_update(struct rte_eth_dev *dev __rte_unused, int wait_to_complete __rte_unused) { return 0; } +static int +eth_rss_reta_update(struct rte_eth_dev *dev, + struct rte_eth_rss_reta_entry64 *reta_conf, uint16_t reta_size) +{ + int i, j; + struct pmd_internals *internal = dev->data->dev_private; + + if (reta_size != internal->reta_size) + return -EINVAL; + + rte_spinlock_lock(&internal->rss_lock); + + /* Copy RETA table */ + for (i = 0; i < (internal->reta_size / RTE_RETA_GROUP_SIZE); i++) { + internal->reta_conf[i].mask = reta_conf[i].mask; + for (j = 0; j < RTE_RETA_GROUP_SIZE; j++) + if ((reta_conf[i].mask >> j) & 0x01) + internal->reta_conf[i].reta[j] = reta_conf[i].reta[j]; + } + + rte_spinlock_unlock(&internal->rss_lock); + + return 0; +} + +static int +eth_rss_reta_query(struct rte_eth_dev *dev, + struct rte_eth_rss_reta_entry64 *reta_conf, uint16_t reta_size) +{ + int i, j; + struct pmd_internals *internal = dev->data->dev_private; + + if (reta_size != internal->reta_size) + return -EINVAL; + + rte_spinlock_lock(&internal->rss_lock); + + /* Copy RETA table */ + for (i = 0; i < (internal->reta_size / RTE_RETA_GROUP_SIZE); i++) { + for (j = 0; j < RTE_RETA_GROUP_SIZE; j++) + if ((reta_conf[i].mask >> j) & 0x01) + reta_conf[i].reta[j] = internal->reta_conf[i].reta[j]; + } + + rte_spinlock_unlock(&internal->rss_lock); + + return 0; +} + +static int +eth_rss_hash_update(struct rte_eth_dev *dev, struct rte_eth_rss_conf *rss_conf) +{ + struct pmd_internals *internal = dev->data->dev_private; + + rte_spinlock_lock(&internal->rss_lock); + + if ((rss_conf->rss_hf & internal->flow_type_rss_offloads) != 0) + dev->data->dev_conf.rx_adv_conf.rss_conf.rss_hf = + rss_conf->rss_hf & internal->flow_type_rss_offloads; + + if (rss_conf->rss_key) + memcpy(internal->rss_key, rss_conf->rss_key, 40); + + rte_spinlock_unlock(&internal->rss_lock); + + return 0; +} + +static int +eth_rss_hash_conf_get(struct rte_eth_dev *dev, + struct rte_eth_rss_conf *rss_conf) +{ + struct pmd_internals *internal = dev->data->dev_private; + + rte_spinlock_lock(&internal->rss_lock); + + rss_conf->rss_hf = dev->data->dev_conf.rx_adv_conf.rss_conf.rss_hf; + if (rss_conf->rss_key) + memcpy(rss_conf->rss_key, internal->rss_key, 40); + + rte_spinlock_unlock(&internal->rss_lock); + + return 0; +} + static const struct eth_dev_ops ops = { .dev_start = eth_dev_start, .dev_stop = eth_dev_stop, @@ -250,6 +349,10 @@ static const struct eth_dev_ops ops = { .stats_reset = eth_stats_reset, .mac_addr_remove = eth_mac_addr_remove, .mac_addr_add = eth_mac_addr_add, + .reta_update = eth_rss_reta_update, + .reta_query = eth_rss_reta_query, + .rss_hash_update = eth_rss_hash_update, + .rss_hash_conf_get = eth_rss_hash_conf_get }; int @@ -268,6 +371,13 @@ rte_eth_from_rings(const char *name, struct rte_ring *const rx_queues[], unsigned i; + static const uint8_t default_rss_key[40] = { + 0x6D, 0x5A, 0x56, 0xDA, 0x25, 0x5B, 0x0E, 0xC2, 0x41, 0x67, 0x25, 0x3D, + 0x43, 0xA3, 0x8F, 0xB0, 0xD0, 0xCA, 0x2B, 0xCB, 0xAE, 0x7B, 0x30, 0xB4, + 0x77, 0xCB, 0x2D, 0xA3, 0x80, 0x30, 0xF2, 0x0C, 0x6A, 0x42, 0xB7, 0x3B, + 0xBE, 0xAC, 0x01, 0xFA + }; + /* do some parameter checking */ if (rx_queues == NULL && nb_rx_queues > 0) goto error; @@ -316,12 +426,14 @@ rte_eth_from_rings(const char *name, struct rte_ring *const rx_queues[], internals->nb_rx_queues = nb_rx_queues; internals->nb_tx_queues = nb_tx_queues; - for (i = 0; i < nb_rx_queues; i++) { + for (i = 0; i < nb_rx_queues; i++) internals->rx_ring_queues[i].rng = rx_queues[i]; - } - for (i = 0; i < nb_tx_queues; i++) { + for (i = 0; i < nb_tx_queues; i++) internals->tx_ring_queues[i].rng = tx_queues[i]; - } + internals->flow_type_rss_offloads = ETH_RSS_PROTO_MASK; + internals->reta_size = RTE_DIM(internals->reta_conf) * RTE_RETA_GROUP_SIZE; + + memcpy(internals->rss_key, default_rss_key, 40); eth_drv->pci_drv.name = ring_ethdev_driver_name; eth_drv->pci_drv.id_table = id_table; -- 1.7.9.5 ^ permalink raw reply [flat|nested] 126+ messages in thread
* Re: [dpdk-dev] [PATCH v3 2/8] ring: dynamic rss configuration 2015-06-29 14:50 ` [dpdk-dev] [PATCH v3 2/8] ring: dynamic rss configuration Tomasz Kulasek @ 2015-07-13 12:36 ` Thomas Monjalon 2015-07-13 14:43 ` Kulasek, TomaszX 0 siblings, 1 reply; 126+ messages in thread From: Thomas Monjalon @ 2015-07-13 12:36 UTC (permalink / raw) To: Tomasz Kulasek; +Cc: dev 2015-06-29 16:50, Tomasz Kulasek: > This implementation allows to set and read RSS configuration for ring > device, and is used to validate right values propagation over the slaves, > in test units for dynamic RSS configuration for bonding. > > It have no impact on packet processing by ring device. Adding some fake RSS to the ring PMD (in order to test bonding) is weird. The ring PMD is not a driver for testing. Maybe that the null PMD would be more appropriate. By the way the current RSS implementation is really bound to Intel devices. Before applying it to more drivers, we have to make sure it is generic enough. Maybe the RETA needs more abstraction. ^ permalink raw reply [flat|nested] 126+ messages in thread
* Re: [dpdk-dev] [PATCH v3 2/8] ring: dynamic rss configuration 2015-07-13 12:36 ` Thomas Monjalon @ 2015-07-13 14:43 ` Kulasek, TomaszX 2015-07-13 15:11 ` Thomas Monjalon 0 siblings, 1 reply; 126+ messages in thread From: Kulasek, TomaszX @ 2015-07-13 14:43 UTC (permalink / raw) To: Thomas Monjalon; +Cc: dev > -----Original Message----- > From: Thomas Monjalon [mailto:thomas.monjalon@6wind.com] > Sent: Monday, July 13, 2015 14:36 > To: Kulasek, TomaszX > Cc: dev@dpdk.org > Subject: Re: [dpdk-dev] [PATCH v3 2/8] ring: dynamic rss configuration > > 2015-06-29 16:50, Tomasz Kulasek: > > This implementation allows to set and read RSS configuration for ring > > device, and is used to validate right values propagation over the > > slaves, in test units for dynamic RSS configuration for bonding. > > > > It have no impact on packet processing by ring device. > > Adding some fake RSS to the ring PMD (in order to test bonding) is weird. > The ring PMD is not a driver for testing. Maybe that the null PMD would be > more appropriate. > By the way the current RSS implementation is really bound to Intel > devices. > Before applying it to more drivers, we have to make sure it is generic > enough. Maybe the RETA needs more abstraction. This is not RSS implementation, but implementation of Dynamic RSS Configuration, already existing, and well defined in official documentation for single port. I don't know where you see bounding to Intel device. It's an implementation of official API. Anyway I will appreciate for any comment and opinion on that and clarification what means more RETA abstraction in context of official API. I will check the possibility of moving it to the null pmd driver and will prepare new version for easier reviewing. Meantime I'm waiting for more opinions. ^ permalink raw reply [flat|nested] 126+ messages in thread
* Re: [dpdk-dev] [PATCH v3 2/8] ring: dynamic rss configuration 2015-07-13 14:43 ` Kulasek, TomaszX @ 2015-07-13 15:11 ` Thomas Monjalon 2015-07-16 13:02 ` Kulasek, TomaszX 0 siblings, 1 reply; 126+ messages in thread From: Thomas Monjalon @ 2015-07-13 15:11 UTC (permalink / raw) To: Kulasek, TomaszX; +Cc: dev 2015-07-13 14:43, Kulasek, TomaszX: > From: Thomas Monjalon [mailto:thomas.monjalon@6wind.com] > > 2015-06-29 16:50, Tomasz Kulasek: > > > This implementation allows to set and read RSS configuration for ring > > > device, and is used to validate right values propagation over the > > > slaves, in test units for dynamic RSS configuration for bonding. > > > > > > It have no impact on packet processing by ring device. > > > > Adding some fake RSS to the ring PMD (in order to test bonding) is weird. > > The ring PMD is not a driver for testing. Maybe that the null PMD would be > > more appropriate. > > By the way the current RSS implementation is really bound to Intel > > devices. > > Before applying it to more drivers, we have to make sure it is generic > > enough. Maybe the RETA needs more abstraction. > > This is not RSS implementation, but implementation of Dynamic RSS > Configuration, already existing, and well defined in official documentation > for single port. > I don't know where you see bounding to Intel device. > It's an implementation of official API. What do you mean by "official"? My concern is that the RSS API in ethdev comes from a time where DPDK was Intel DPDK. I may be wrong but I think that other devices could need something more generic and better defined. > Anyway I will appreciate for any comment and opinion on that and > clarification what means more RETA abstraction in context of official API. > > I will check the possibility of moving it to the null pmd driver and will > prepare new version for easier reviewing. Meantime I'm waiting for more > opinions. Exact, we need more opinions on this topic. ^ permalink raw reply [flat|nested] 126+ messages in thread
* Re: [dpdk-dev] [PATCH v3 2/8] ring: dynamic rss configuration 2015-07-13 15:11 ` Thomas Monjalon @ 2015-07-16 13:02 ` Kulasek, TomaszX 0 siblings, 0 replies; 126+ messages in thread From: Kulasek, TomaszX @ 2015-07-16 13:02 UTC (permalink / raw) To: Thomas Monjalon; +Cc: dev > -----Original Message----- > From: Thomas Monjalon [mailto:thomas.monjalon@6wind.com] > Sent: Monday, July 13, 2015 17:11 > To: Kulasek, TomaszX > Cc: dev@dpdk.org > Subject: Re: [dpdk-dev] [PATCH v3 2/8] ring: dynamic rss configuration > > 2015-07-13 14:43, Kulasek, TomaszX: > > From: Thomas Monjalon [mailto:thomas.monjalon@6wind.com] > > > 2015-06-29 16:50, Tomasz Kulasek: > > > > This implementation allows to set and read RSS configuration for > > > > ring device, and is used to validate right values propagation over > > > > the slaves, in test units for dynamic RSS configuration for bonding. > > > > > > > > It have no impact on packet processing by ring device. > > > > > > Adding some fake RSS to the ring PMD (in order to test bonding) is > weird. > > > The ring PMD is not a driver for testing. Maybe that the null PMD > > > would be more appropriate. > > > By the way the current RSS implementation is really bound to Intel > > > devices. > > > Before applying it to more drivers, we have to make sure it is > > > generic enough. Maybe the RETA needs more abstraction. > > > > This is not RSS implementation, but implementation of Dynamic RSS > > Configuration, already existing, and well defined in official > > documentation for single port. > > I don't know where you see bounding to Intel device. > > It's an implementation of official API. > > What do you mean by "official"? > My concern is that the RSS API in ethdev comes from a time where DPDK was > Intel DPDK. I may be wrong but I think that other devices could need > something more generic and better defined. > > > Anyway I will appreciate for any comment and opinion on that and > > clarification what means more RETA abstraction in context of official > API. > > > > I will check the possibility of moving it to the null pmd driver and > > will prepare new version for easier reviewing. Meantime I'm waiting > > for more opinions. > > Exact, we need more opinions on this topic. Hi Thomas, I have fixed the error with wrong copy/paste. Also I changed usage of ring pmd with null pmd like you suggested. You are right, it is better place. Thanks. I sent a v4 patch-set. This implementation is using an existing ethdev API for RSS and introduces generic enhancements for configuration of bonding slaves. If other devices need a more generic ethdev API then this should be treated separately and they can submit separate patches for this in a future release. ^ permalink raw reply [flat|nested] 126+ messages in thread
* [dpdk-dev] [PATCH v3 3/8] test: dynamic rss configuration 2015-06-29 14:50 ` [dpdk-dev] [PATCH v3 " Tomasz Kulasek 2015-06-29 14:50 ` [dpdk-dev] [PATCH v3 1/8] bonding: rss dynamic configuration Tomasz Kulasek 2015-06-29 14:50 ` [dpdk-dev] [PATCH v3 2/8] ring: dynamic rss configuration Tomasz Kulasek @ 2015-06-29 14:50 ` Tomasz Kulasek 2015-06-29 14:50 ` [dpdk-dev] [PATCH v3 4/8] bonding: queue stats mapping Tomasz Kulasek ` (6 subsequent siblings) 9 siblings, 0 replies; 126+ messages in thread From: Tomasz Kulasek @ 2015-06-29 14:50 UTC (permalink / raw) To: dev This test module uses ring device to check right RSS configuration propagation. 1) Propagation test a) Set RSS hash function for bonding, fetch RSS hash function from bonded slave, check if same. Do it for all slaves. b) Repeat above for RSS key and RETA. 2)Dynamic adding slave to the bonding a) Remove slave from bonding. b) Change its configuration to other than bonding. c) Add slave to the started bonding device. d) Check if slaves configuration changed. 3) Repeat point 1) and 2) with RSS multi-queue mode enabled/disabled for bonding port. Signed-off-by: Tomasz Kulasek <tomaszx.kulasek@intel.com> --- app/test/Makefile | 1 + app/test/test_link_bonding_rssconf.c | 674 ++++++++++++++++++++++++++++++++++ 2 files changed, 675 insertions(+) create mode 100644 app/test/test_link_bonding_rssconf.c diff --git a/app/test/Makefile b/app/test/Makefile index 2e2758c..15926fe 100644 --- a/app/test/Makefile +++ b/app/test/Makefile @@ -136,6 +136,7 @@ SRCS-$(CONFIG_RTE_LIBRTE_ACL) += test_acl.c ifeq ($(CONFIG_RTE_LIBRTE_PMD_RING),y) SRCS-$(CONFIG_RTE_LIBRTE_PMD_BOND) += test_link_bonding.c SRCS-$(CONFIG_RTE_LIBRTE_PMD_BOND) += test_link_bonding_mode4.c +SRCS-$(CONFIG_RTE_LIBRTE_PMD_BOND) += test_link_bonding_rssconf.c endif SRCS-$(CONFIG_RTE_LIBRTE_PMD_RING) += test_pmd_ring.c diff --git a/app/test/test_link_bonding_rssconf.c b/app/test/test_link_bonding_rssconf.c new file mode 100644 index 0000000..96bc5c6 --- /dev/null +++ b/app/test/test_link_bonding_rssconf.c @@ -0,0 +1,674 @@ +/*- + * BSD LICENSE + * + * Copyright(c) 2015 Intel Corporation. All rights reserved. + * All rights reserved. + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * + * * Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * * Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in + * the documentation and/or other materials provided with the + * distribution. + * * Neither the name of Intel Corporation nor the names of its + * contributors may be used to endorse or promote products derived + * from this software without specific prior written permission. + * + * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS + * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT + * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR + * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT + * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, + * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT + * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, + * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY + * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT + * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE + * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + */ + +#include <string.h> +#include <stdarg.h> +#include <stdio.h> +#include <stdlib.h> +#include <stdint.h> +#include <inttypes.h> +#include <errno.h> +#include <rte_cycles.h> +#include <sys/queue.h> + +#include <rte_byteorder.h> +#include <rte_common.h> +#include <rte_debug.h> +#include <rte_ethdev.h> +#include <rte_log.h> +#include <rte_lcore.h> +#include <rte_memory.h> + +#include <rte_string_fns.h> + +#include <rte_eth_ring.h> +#include <rte_errno.h> +#include <rte_eth_bond.h> + +#include "test.h" + +#define SLAVE_COUNT (4) + +#define RXTX_RING_SIZE 1024 +#define RXTX_QUEUE_COUNT 4 + +#define BONDED_DEV_NAME ("rssconf_bond_dev") + +#define SLAVE_DEV_NAME_FMT ("rssconf_slave%d") +#define SLAVE_RXTX_QUEUE_FMT ("rssconf_slave%d_q%d") + +#define NUM_MBUFS 8191 +#define MBUF_SIZE (1600 + sizeof(struct rte_mbuf) + RTE_PKTMBUF_HEADROOM) +#define MBUF_CACHE_SIZE 250 +#define BURST_SIZE 32 + +#define INVALID_SOCKET_ID (-1) +#define INVALID_PORT_ID (0xFF) +#define INVALID_BONDING_MODE (-1) + +struct slave_conf { + uint8_t port_id; + struct rte_eth_dev_info dev_info; + + struct rte_eth_rss_conf rss_conf; + uint8_t rss_key[40]; + struct rte_eth_rss_reta_entry64 reta_conf[512 / RTE_RETA_GROUP_SIZE]; + + uint8_t is_slave; + struct rte_ring *rxtx_queue[RXTX_QUEUE_COUNT]; +}; + +struct link_bonding_rssconf_unittest_params { + uint8_t bond_port_id; + struct rte_eth_dev_info bond_dev_info; + struct rte_eth_rss_reta_entry64 bond_reta_conf[512 / RTE_RETA_GROUP_SIZE]; + struct slave_conf slave_ports[SLAVE_COUNT]; + + struct rte_mempool *mbuf_pool; +}; + +static struct link_bonding_rssconf_unittest_params test_params = { + .bond_port_id = INVALID_PORT_ID, + .slave_ports = { + [0 ... SLAVE_COUNT - 1] = { .port_id = INVALID_PORT_ID, .is_slave = 0} + }, + .mbuf_pool = NULL, +}; + +/** + * Default port configuration with RSS turned off + */ +static struct rte_eth_conf default_pmd_conf = { + .rxmode = { + .mq_mode = ETH_MQ_RX_NONE, + .max_rx_pkt_len = ETHER_MAX_LEN, + .split_hdr_size = 0, + .header_split = 0, /**< Header Split disabled */ + .hw_ip_checksum = 0, /**< IP checksum offload enabled */ + .hw_vlan_filter = 0, /**< VLAN filtering disabled */ + .jumbo_frame = 0, /**< Jumbo Frame Support disabled */ + .hw_strip_crc = 0, /**< CRC stripped by hardware */ + }, + .txmode = { + .mq_mode = ETH_MQ_TX_NONE, + }, + .lpbk_mode = 0, +}; + +static struct rte_eth_conf rss_pmd_conf = { + .rxmode = { + .mq_mode = ETH_MQ_RX_RSS, + .max_rx_pkt_len = ETHER_MAX_LEN, + .split_hdr_size = 0, + .header_split = 0, /**< Header Split disabled */ + .hw_ip_checksum = 0, /**< IP checksum offload enabled */ + .hw_vlan_filter = 0, /**< VLAN filtering disabled */ + .jumbo_frame = 0, /**< Jumbo Frame Support disabled */ + .hw_strip_crc = 0, /**< CRC stripped by hardware */ + }, + .txmode = { + .mq_mode = ETH_MQ_TX_NONE, + }, + .rx_adv_conf = { + .rss_conf = { + .rss_key = NULL, + .rss_hf = ETH_RSS_IPV6, + }, + }, + .lpbk_mode = 0, +}; + +#define FOR_EACH(_i, _item, _array, _size) \ + for (_i = 0, _item = &_array[0]; _i < _size && (_item = &_array[_i]); _i++) + +/* Macro for iterating over every port that can be used as a slave + * in this test. + * _i variable used as an index in test_params->slave_ports + * _slave pointer to &test_params->slave_ports[_idx] + */ +#define FOR_EACH_PORT(_i, _port) \ + FOR_EACH(_i, _port, test_params.slave_ports, \ + RTE_DIM(test_params.slave_ports)) + +static int +configure_ethdev(uint8_t port_id, struct rte_eth_conf *eth_conf, uint8_t start) +{ + int rxq, txq; + + TEST_ASSERT(rte_eth_dev_configure(port_id, RXTX_QUEUE_COUNT, + RXTX_QUEUE_COUNT, eth_conf) == 0, "Failed to configure device %u", + port_id); + + for (rxq = 0; rxq < RXTX_QUEUE_COUNT; rxq++) { + TEST_ASSERT(rte_eth_rx_queue_setup(port_id, rxq, RXTX_RING_SIZE, + rte_eth_dev_socket_id(port_id), NULL, + test_params.mbuf_pool) == 0, "Failed to setup rx queue."); + } + + for (txq = 0; txq < RXTX_QUEUE_COUNT; txq++) { + TEST_ASSERT(rte_eth_tx_queue_setup(port_id, txq, RXTX_RING_SIZE, + rte_eth_dev_socket_id(port_id), NULL) == 0, + "Failed to setup tx queue."); + } + + if (start) { + TEST_ASSERT(rte_eth_dev_start(port_id) == 0, + "Failed to start device (%d).", port_id); + } + + return 0; +} + +/** + * Remove all slaves from bonding + */ +static int +remove_slaves(void) +{ + unsigned n; + struct slave_conf *port; + + FOR_EACH_PORT(n, port) { + port = &test_params.slave_ports[n]; + if (port->is_slave) { + TEST_ASSERT_SUCCESS(rte_eth_bond_slave_remove( + test_params.bond_port_id, port->port_id), + "Cannot remove slave %d from bonding", port->port_id); + port->is_slave = 0; + } + } + + return 0; +} + +static int +remove_slaves_and_stop_bonded_device(void) +{ + TEST_ASSERT_SUCCESS(remove_slaves(), "Removing slaves"); + rte_eth_dev_stop(test_params.bond_port_id); + return TEST_SUCCESS; +} + +/** + * Add all slaves to bonding + */ +static int +bond_slaves(void) +{ + unsigned n; + struct slave_conf *port; + + FOR_EACH_PORT(n, port) { + port = &test_params.slave_ports[n]; + if (!port->is_slave) { + TEST_ASSERT_SUCCESS(rte_eth_bond_slave_add(test_params.bond_port_id, + port->port_id), "Cannot attach slave %d to the bonding", + port->port_id); + port->is_slave = 1; + } + } + + return 0; +} + +/** + * Set all RETA values in port_id to value + */ +static int +reta_set(uint8_t port_id, uint8_t value, int reta_size) +{ + struct rte_eth_rss_reta_entry64 reta_conf[512/RTE_RETA_GROUP_SIZE]; + int i, j; + + for (i = 0; i < reta_size / RTE_RETA_GROUP_SIZE; i++) { + /* select all fields to set */ + reta_conf[i].mask = ~0LL; + for (j = 0; j < RTE_RETA_GROUP_SIZE; j++) + reta_conf[i].reta[j] = value; + } + + return rte_eth_dev_rss_reta_update(port_id, reta_conf, reta_size); +} + +/** + * Check if slaves RETA is synchronized with bonding port. Returns 1 if slave + * port is synced with bonding port. + */ +static int +reta_check_synced(struct slave_conf *port) +{ + unsigned i; + + for (i = 0; i < test_params.bond_dev_info.reta_size; + i++) { + + int index = i / RTE_RETA_GROUP_SIZE; + int shift = i % RTE_RETA_GROUP_SIZE; + + if (port->reta_conf[index].reta[shift] != + test_params.bond_reta_conf[index].reta[shift]) + return 0; + + } + + return 1; +} + +/** + * Fetch bonding ports RETA + */ +static int +bond_reta_fetch(void) { + unsigned j; + + for (j = 0; j < test_params.bond_dev_info.reta_size / RTE_RETA_GROUP_SIZE; + j++) + test_params.bond_reta_conf[j].mask = ~0LL; + + TEST_ASSERT_SUCCESS(rte_eth_dev_rss_reta_query(test_params.bond_port_id, + test_params.bond_reta_conf, test_params.bond_dev_info.reta_size), + "Cannot take bonding ports RSS configuration"); + return 0; +} + +/** + * Fetch slaves RETA + */ +static int +slave_reta_fetch(struct slave_conf *port) { + unsigned j; + + for (j = 0; j < port->dev_info.reta_size / RTE_RETA_GROUP_SIZE; j++) + port->reta_conf[j].mask = ~0LL; + + TEST_ASSERT_SUCCESS(rte_eth_dev_rss_reta_query(port->port_id, + port->reta_conf, port->dev_info.reta_size), + "Cannot take bonding ports RSS configuration"); + return 0; +} + +/** + * Remove and add slave to check if slaves configuration is synced with + * the bonding ports values after adding new slave. + */ +static int +slave_remove_and_add(void) +{ + struct slave_conf *port = &(test_params.slave_ports[0]); + + /* 1. Remove first slave from bonding */ + TEST_ASSERT_SUCCESS(rte_eth_bond_slave_remove(test_params.bond_port_id, + port->port_id), "Cannot remove slave #d from bonding"); + + /* 2. Change removed (ex-)slave and bonding configuration to different + * values + */ + reta_set(test_params.bond_port_id, 1, test_params.bond_dev_info.reta_size); + bond_reta_fetch(); + + reta_set(port->port_id, 2, port->dev_info.reta_size); + slave_reta_fetch(port); + + TEST_ASSERT(reta_check_synced(port) == 0, + "Removed slave didn't should be synchronized with bonding port"); + + /* 3. Add (ex-)slave and check if configuration changed*/ + TEST_ASSERT_SUCCESS(rte_eth_bond_slave_add(test_params.bond_port_id, + port->port_id), "Cannot add slave"); + + bond_reta_fetch(); + slave_reta_fetch(port); + + return reta_check_synced(port); +} + +/** + * Test configuration propagation over slaves. + */ +static int +test_propagate(void) +{ + unsigned i; + uint8_t n; + struct slave_conf *port; + uint8_t bond_rss_key[40]; + struct rte_eth_rss_conf bond_rss_conf; + + int retval = 0; + uint64_t rss_hf = 0; + uint64_t default_rss_hf = 0; + + rte_eth_dev_info_get(test_params.bond_port_id, &test_params.bond_dev_info); + + /* + * Test hash function propagation + */ + for (i = 0; i < sizeof(test_params.bond_dev_info.flow_type_rss_offloads)*8; + i++) { + + rss_hf = test_params.bond_dev_info.flow_type_rss_offloads & (1<<i); + if (rss_hf) { + bond_rss_conf.rss_key = NULL; + bond_rss_conf.rss_hf = rss_hf; + + retval = rte_eth_dev_rss_hash_update(test_params.bond_port_id, + &bond_rss_conf); + TEST_ASSERT_SUCCESS(retval, "Cannot set slaves hash function"); + + FOR_EACH_PORT(n, port) { + port = &test_params.slave_ports[n]; + + retval = rte_eth_dev_rss_hash_conf_get(port->port_id, + &port->rss_conf); + TEST_ASSERT_SUCCESS(retval, + "Cannot take slaves RSS configuration"); + + TEST_ASSERT(port->rss_conf.rss_hf == rss_hf, + "Hash function not propagated for slave %d", + port->port_id); + } + + default_rss_hf = rss_hf; + } + + } + + /* + * Test key propagation + */ + for (i = 1; i < 10; i++) { + + /* Set all keys to zero */ + FOR_EACH_PORT(n, port) { + port = &test_params.slave_ports[n]; + memset(port->rss_conf.rss_key, 0, 40); + retval = rte_eth_dev_rss_hash_update(port->port_id, + &port->rss_conf); + TEST_ASSERT_SUCCESS(retval, "Cannot set slaves RSS keys"); + } + + memset(bond_rss_key, i, sizeof(bond_rss_key)); + bond_rss_conf.rss_hf = default_rss_hf, + bond_rss_conf.rss_key = bond_rss_key; + bond_rss_conf.rss_key_len = 40; + + retval = rte_eth_dev_rss_hash_update(test_params.bond_port_id, + &bond_rss_conf); + TEST_ASSERT_SUCCESS(retval, "Cannot set bonded port RSS keys"); + + FOR_EACH_PORT(n, port) { + port = &test_params.slave_ports[n]; + + retval = rte_eth_dev_rss_hash_conf_get(port->port_id, + &(port->rss_conf)); + + TEST_ASSERT_SUCCESS(retval, + "Cannot take slaves RSS configuration"); + + /* compare keys */ + retval = memcmp(port->rss_conf.rss_key, bond_rss_key, + sizeof(bond_rss_key)); + TEST_ASSERT(retval == 0, "Key value not propagated for slave %d", + port->port_id); + } + } + + /* + * Test RETA propagation + */ + for (i = 0; i < RXTX_QUEUE_COUNT; i++) { + + /* Set all keys to zero */ + FOR_EACH_PORT(n, port) { + port = &test_params.slave_ports[n]; + retval = reta_set(port->port_id, (i + 1) % RXTX_QUEUE_COUNT, + port->dev_info.reta_size); + TEST_ASSERT_SUCCESS(retval, "Cannot set slaves RETA"); + } + + TEST_ASSERT_SUCCESS(reta_set(test_params.bond_port_id, + i % RXTX_QUEUE_COUNT, test_params.bond_dev_info.reta_size), + "Cannot set bonded port RETA"); + + bond_reta_fetch(); + + FOR_EACH_PORT(n, port) { + port = &test_params.slave_ports[n]; + + slave_reta_fetch(port); + TEST_ASSERT(reta_check_synced(port) == 1, "RETAs inconsistent"); + } + } + + return TEST_SUCCESS; +} + +/** + * Test propagation logic, when RX_RSS mq_mode is turned on for bonding port + */ +static int +test_rss(void) +{ + /** + * Configure bonding port in RSS mq mode + */ + TEST_ASSERT_SUCCESS(configure_ethdev(test_params.bond_port_id, + &rss_pmd_conf, 0), "Failed to configure bonding device\n"); + + rte_eth_dev_info_get(test_params.bond_port_id, &test_params.bond_dev_info); + + TEST_ASSERT_SUCCESS(bond_slaves(), "Bonding slaves failed"); + + TEST_ASSERT_SUCCESS(rte_eth_dev_start(test_params.bond_port_id), + "Failed to start bonding port (%d).", test_params.bond_port_id); + + TEST_ASSERT_SUCCESS(test_propagate(), "Propagation test failed"); + + TEST_ASSERT(slave_remove_and_add() == 1, "New slave should be synced"); + + remove_slaves_and_stop_bonded_device(); + + return TEST_SUCCESS; +} + +/** + * Test propagation logic, when RX_RSS mq_mode is turned off for bonding port + */ +static int +test_rss_lazy(void) +{ + TEST_ASSERT_SUCCESS(configure_ethdev(test_params.bond_port_id, + &default_pmd_conf, 0), "Failed to configure bonding device\n"); + + rte_eth_dev_info_get(test_params.bond_port_id, &test_params.bond_dev_info); + + TEST_ASSERT_SUCCESS(bond_slaves(), "Bonding slaves failed"); + + TEST_ASSERT_SUCCESS(rte_eth_dev_start(test_params.bond_port_id), + "Failed to start bonding port (%d).", test_params.bond_port_id); + + TEST_ASSERT_SUCCESS(test_propagate(), "Propagation test failed"); + + TEST_ASSERT(slave_remove_and_add() == 0, "New slave shouldn't be synced"); + + remove_slaves_and_stop_bonded_device(); + + return TEST_SUCCESS; +} + +static int +test_setup(void) +{ + unsigned i, n; + int retval; + char name[256]; + char rng_name[RTE_RING_NAMESIZE]; + struct slave_conf *port; + + if (test_params.mbuf_pool == NULL) { + + test_params.mbuf_pool = rte_mempool_create("MBUF_POOL", NUM_MBUFS * + SLAVE_COUNT, MBUF_SIZE, MBUF_CACHE_SIZE, + sizeof(struct rte_pktmbuf_pool_private), rte_pktmbuf_pool_init, + NULL, rte_pktmbuf_init, NULL, rte_socket_id(), 0); + + TEST_ASSERT(test_params.mbuf_pool != NULL, + "rte_mempool_create failed\n"); + } + + /* Create / initialize ring eth devs. */ + FOR_EACH_PORT(n, port) { + port = &test_params.slave_ports[n]; + + snprintf(name, sizeof(name), SLAVE_DEV_NAME_FMT, n); + + for (i = 0; i < RXTX_QUEUE_COUNT; i++) { + snprintf(rng_name, sizeof(rng_name), SLAVE_RXTX_QUEUE_FMT, n, i); + + if (port->rxtx_queue[i] == NULL) { + port->rxtx_queue[i] = rte_ring_create(rng_name, RXTX_RING_SIZE, + 0, RING_F_SP_ENQ|RING_F_SC_DEQ); + TEST_ASSERT(port->rxtx_queue[i] != NULL, + "Failed to allocate rx ring '%s': %s", name, + rte_strerror(rte_errno)); + } + } + + if (rte_eth_from_rings(name, port->rxtx_queue, RXTX_QUEUE_COUNT, + port->rxtx_queue, RXTX_QUEUE_COUNT, 0)) { + TEST_ASSERT(retval >= 0, + "Failed to create ring ethdev '%s'\n", name); + } + + port->port_id = rte_eth_dev_count() - 1; + port->rss_conf.rss_key = port->rss_key; + port->rss_conf.rss_key_len = 40; + + retval = configure_ethdev(port->port_id, &default_pmd_conf, 1); + TEST_ASSERT_SUCCESS(retval, "Failed to configure virtual ethdev %s\n", + name); + + rte_eth_dev_info_get(port->port_id, &port->dev_info); + } + + if (test_params.bond_port_id == INVALID_PORT_ID) { + retval = rte_eth_bond_create(BONDED_DEV_NAME, 0, rte_socket_id()); + + TEST_ASSERT(retval >= 0, "Failed to create bonded ethdev %s", + BONDED_DEV_NAME); + + test_params.bond_port_id = retval; + + TEST_ASSERT_SUCCESS(configure_ethdev(test_params.bond_port_id, + &default_pmd_conf, 0), "Failed to configure bonding device\n"); + + rte_eth_dev_info_get(test_params.bond_port_id, + &test_params.bond_dev_info); + } + + return TEST_SUCCESS; +} + +static int +check_environment(void) +{ + return TEST_SUCCESS; +} + +static int +test_rssconf_executor(int (*test_func)(void)) +{ + int test_result; + + /* Check if environment is clean. Fail to launch a test if there was + * a critical error before that prevented to reset environment. */ + TEST_ASSERT_SUCCESS(check_environment(), + "Refusing to launch test in dirty environment."); + + RTE_VERIFY(test_func != NULL); + test_result = (*test_func)(); + + /* If test succeed check if environment wast left in good condition. */ + if (test_result == TEST_SUCCESS) + test_result = check_environment(); + + /* Reset environment in case test failed to do that. */ + if (test_result != TEST_SUCCESS) { + TEST_ASSERT_SUCCESS(remove_slaves_and_stop_bonded_device(), + "Failed to stop bonded device"); + } + + return test_result; +} + +static int +test_setup_wrapper(void) +{ + return test_rssconf_executor(&test_setup); +} + +static int +test_rss_wrapper(void) +{ + return test_rssconf_executor(&test_rss); +} + +static int +test_rss_lazy_wrapper(void) +{ + return test_rssconf_executor(&test_rss_lazy); +} + +static struct unit_test_suite link_bonding_rssconf_test_suite = { + .suite_name = "RSS Dynamic Configuration for Bonding Unit Test Suite", + .setup = test_setup, + .unit_test_cases = { + TEST_CASE_NAMED("test_setup", test_setup_wrapper), + TEST_CASE_NAMED("test_rss", test_rss_wrapper), + TEST_CASE_NAMED("test_rss_lazy", test_rss_lazy_wrapper), + { NULL, NULL, NULL, NULL, NULL } /**< NULL terminate unit test array */ + } +}; + +static int +test_link_bonding_rssconf(void) +{ + return unit_test_suite_runner(&link_bonding_rssconf_test_suite); +} + +static struct test_command test_link_bonding_rssconf_cmd = { + .command = "link_bonding_rssconf_autotest", + .callback = test_link_bonding_rssconf, +}; + +REGISTER_TEST_COMMAND(test_link_bonding_rssconf_cmd); -- 1.7.9.5 ^ permalink raw reply [flat|nested] 126+ messages in thread
* [dpdk-dev] [PATCH v3 4/8] bonding: queue stats mapping 2015-06-29 14:50 ` [dpdk-dev] [PATCH v3 " Tomasz Kulasek ` (2 preceding siblings ...) 2015-06-29 14:50 ` [dpdk-dev] [PATCH v3 3/8] test: " Tomasz Kulasek @ 2015-06-29 14:50 ` Tomasz Kulasek 2015-07-13 11:18 ` Thomas Monjalon 2015-06-29 14:50 ` [dpdk-dev] [PATCH v3 5/8] ring: queue stats mapping set dummy implementation Tomasz Kulasek ` (5 subsequent siblings) 9 siblings, 1 reply; 126+ messages in thread From: Tomasz Kulasek @ 2015-06-29 14:50 UTC (permalink / raw) To: dev Queue stats mapping is used in example application. This patch adds propagation of mapping over the slaves, and fills bonding port's stats with a sum of corresponding values taken from bonded slaves, when stats are requested for bonding port. Signed-off-by: Tomasz Kulasek <tomaszx.kulasek@intel.com> --- drivers/net/bonding/rte_eth_bond_pmd.c | 30 +++++++++++++++++++++++++++++- 1 file changed, 29 insertions(+), 1 deletion(-) diff --git a/drivers/net/bonding/rte_eth_bond_pmd.c b/drivers/net/bonding/rte_eth_bond_pmd.c index cd23f42..ade84c4 100644 --- a/drivers/net/bonding/rte_eth_bond_pmd.c +++ b/drivers/net/bonding/rte_eth_bond_pmd.c @@ -1783,7 +1783,7 @@ bond_ethdev_stats_get(struct rte_eth_dev *dev, struct rte_eth_stats *stats) { struct bond_dev_private *internals = dev->data->dev_private; struct rte_eth_stats slave_stats; - int i; + int i, j; for (i = 0; i < internals->slave_count; i++) { rte_eth_stats_get(internals->slaves[i].port_id, &slave_stats); @@ -1802,6 +1802,15 @@ bond_ethdev_stats_get(struct rte_eth_dev *dev, struct rte_eth_stats *stats) stats->rx_pause_xon += slave_stats.rx_pause_xon; stats->tx_pause_xoff += slave_stats.tx_pause_xoff; stats->rx_pause_xoff += slave_stats.rx_pause_xoff; + + for (j = 0; j < RTE_ETHDEV_QUEUE_STAT_CNTRS; j++) { + stats->q_ipackets[j] += slave_stats.q_ipackets[j]; + stats->q_opackets[j] += slave_stats.q_ipackets[j]; + stats->q_ibytes[j] += slave_stats.q_ipackets[j]; + stats->q_obytes[j] += slave_stats.q_ipackets[j]; + stats->q_errors[j] += slave_stats.q_ipackets[j]; + } + } } @@ -2110,6 +2119,24 @@ bond_ethdev_rss_hash_conf_get(struct rte_eth_dev *dev, return 0; } +static int +bond_ethdev_queue_stats_mapping_set(struct rte_eth_dev *dev, uint16_t queue_id, + uint8_t stat_idx, uint8_t is_rx) +{ + int i; + struct bond_dev_private *internals = dev->data->dev_private; + + for (i = 0; i < internals->slave_count; i++) + if (is_rx) + rte_eth_dev_set_rx_queue_stats_mapping( + internals->slaves[i].port_id, queue_id, stat_idx); + else + rte_eth_dev_set_tx_queue_stats_mapping( + internals->slaves[i].port_id, queue_id, stat_idx); + + return 0; +} + struct eth_dev_ops default_dev_ops = { .dev_start = bond_ethdev_start, .dev_stop = bond_ethdev_stop, @@ -2123,6 +2150,7 @@ struct eth_dev_ops default_dev_ops = { .link_update = bond_ethdev_link_update, .stats_get = bond_ethdev_stats_get, .stats_reset = bond_ethdev_stats_reset, + .queue_stats_mapping_set = bond_ethdev_queue_stats_mapping_set, .promiscuous_enable = bond_ethdev_promiscuous_enable, .promiscuous_disable = bond_ethdev_promiscuous_disable, .reta_update = bond_ethdev_rss_reta_update, -- 1.7.9.5 ^ permalink raw reply [flat|nested] 126+ messages in thread
* Re: [dpdk-dev] [PATCH v3 4/8] bonding: queue stats mapping 2015-06-29 14:50 ` [dpdk-dev] [PATCH v3 4/8] bonding: queue stats mapping Tomasz Kulasek @ 2015-07-13 11:18 ` Thomas Monjalon 2015-07-13 12:00 ` Kulasek, TomaszX 0 siblings, 1 reply; 126+ messages in thread From: Thomas Monjalon @ 2015-07-13 11:18 UTC (permalink / raw) To: Tomasz Kulasek; +Cc: dev 2015-06-29 16:50, Tomasz Kulasek: > + for (j = 0; j < RTE_ETHDEV_QUEUE_STAT_CNTRS; j++) { > + stats->q_ipackets[j] += slave_stats.q_ipackets[j]; > + stats->q_opackets[j] += slave_stats.q_ipackets[j]; > + stats->q_ibytes[j] += slave_stats.q_ipackets[j]; > + stats->q_obytes[j] += slave_stats.q_ipackets[j]; > + stats->q_errors[j] += slave_stats.q_ipackets[j]; Wrong copy/paste. ^ permalink raw reply [flat|nested] 126+ messages in thread
* Re: [dpdk-dev] [PATCH v3 4/8] bonding: queue stats mapping 2015-07-13 11:18 ` Thomas Monjalon @ 2015-07-13 12:00 ` Kulasek, TomaszX 0 siblings, 0 replies; 126+ messages in thread From: Kulasek, TomaszX @ 2015-07-13 12:00 UTC (permalink / raw) To: Thomas Monjalon; +Cc: dev > -----Original Message----- > From: Thomas Monjalon [mailto:thomas.monjalon@6wind.com] > Sent: Monday, July 13, 2015 13:18 > To: Kulasek, TomaszX > Cc: dev@dpdk.org > Subject: Re: [dpdk-dev] [PATCH v3 4/8] bonding: queue stats mapping > > 2015-06-29 16:50, Tomasz Kulasek: > > + for (j = 0; j < RTE_ETHDEV_QUEUE_STAT_CNTRS; j++) { > > + stats->q_ipackets[j] += slave_stats.q_ipackets[j]; > > + stats->q_opackets[j] += slave_stats.q_ipackets[j]; > > + stats->q_ibytes[j] += slave_stats.q_ipackets[j]; > > + stats->q_obytes[j] += slave_stats.q_ipackets[j]; > > + stats->q_errors[j] += slave_stats.q_ipackets[j]; > > Wrong copy/paste. Ok, my fault, sorry. I'll fix it. ^ permalink raw reply [flat|nested] 126+ messages in thread
* [dpdk-dev] [PATCH v3 5/8] ring: queue stats mapping set dummy implementation 2015-06-29 14:50 ` [dpdk-dev] [PATCH v3 " Tomasz Kulasek ` (3 preceding siblings ...) 2015-06-29 14:50 ` [dpdk-dev] [PATCH v3 4/8] bonding: queue stats mapping Tomasz Kulasek @ 2015-06-29 14:50 ` Tomasz Kulasek 2015-07-09 1:58 ` Thomas Monjalon 2015-06-29 14:50 ` [dpdk-dev] [PATCH v3 6/8] examples: dynamic rss configuration for bonding Tomasz Kulasek ` (4 subsequent siblings) 9 siblings, 1 reply; 126+ messages in thread From: Tomasz Kulasek @ 2015-06-29 14:50 UTC (permalink / raw) To: dev Per queue statistics are already implemented for ring device, but with static mapping (stat_idx == queue_id). This fix is required, if you want to use ring device in test application and is used only to point that per queue statistics are provided for this device. Signed-off-by: Tomasz Kulasek <tomaszx.kulasek@intel.com> --- drivers/net/ring/rte_eth_ring.c | 10 ++++++++++ 1 file changed, 10 insertions(+) diff --git a/drivers/net/ring/rte_eth_ring.c b/drivers/net/ring/rte_eth_ring.c index 29d7bd8..7e6bf14 100644 --- a/drivers/net/ring/rte_eth_ring.c +++ b/drivers/net/ring/rte_eth_ring.c @@ -333,6 +333,15 @@ eth_rss_hash_conf_get(struct rte_eth_dev *dev, return 0; } +static int +eth_queue_stats_mapping_set(__rte_unused struct rte_eth_dev *dev, + __rte_unused uint16_t queue_id, __rte_unused uint8_t stat_idx, + __rte_unused uint8_t is_rx) +{ + /* Do nothing, just return ok */ + return 0; +} + static const struct eth_dev_ops ops = { .dev_start = eth_dev_start, .dev_stop = eth_dev_stop, @@ -345,6 +354,7 @@ static const struct eth_dev_ops ops = { .rx_queue_release = eth_queue_release, .tx_queue_release = eth_queue_release, .link_update = eth_link_update, + .queue_stats_mapping_set = eth_queue_stats_mapping_set, .stats_get = eth_stats_get, .stats_reset = eth_stats_reset, .mac_addr_remove = eth_mac_addr_remove, -- 1.7.9.5 ^ permalink raw reply [flat|nested] 126+ messages in thread
* Re: [dpdk-dev] [PATCH v3 5/8] ring: queue stats mapping set dummy implementation 2015-06-29 14:50 ` [dpdk-dev] [PATCH v3 5/8] ring: queue stats mapping set dummy implementation Tomasz Kulasek @ 2015-07-09 1:58 ` Thomas Monjalon 2015-07-09 9:55 ` Kulasek, TomaszX 0 siblings, 1 reply; 126+ messages in thread From: Thomas Monjalon @ 2015-07-09 1:58 UTC (permalink / raw) To: Tomasz Kulasek; +Cc: dev 2015-06-29 16:50, Tomasz Kulasek: > Per queue statistics are already implemented for ring device, but with > static mapping (stat_idx == queue_id). > > This fix is required, if you want to use ring device in test application > and is used only to point that per queue statistics are provided for this > device. > > Signed-off-by: Tomasz Kulasek <tomaszx.kulasek@intel.com> [...] > +static int > +eth_queue_stats_mapping_set(__rte_unused struct rte_eth_dev *dev, > + __rte_unused uint16_t queue_id, __rte_unused uint8_t stat_idx, > + __rte_unused uint8_t is_rx) > +{ > + /* Do nothing, just return ok */ > + return 0; > +} I've just realized how this is broken. Some Intel devices use a mapping to select hardware queues which will have some stats. But we may have stats per queues without requiring such mapping. I may miss something but I suggest these 3 actions: - remove this patch - replace checks on stats_mapping by an ethdev flag - remove device-specific stats_mapping from ethdev ^ permalink raw reply [flat|nested] 126+ messages in thread
* Re: [dpdk-dev] [PATCH v3 5/8] ring: queue stats mapping set dummy implementation 2015-07-09 1:58 ` Thomas Monjalon @ 2015-07-09 9:55 ` Kulasek, TomaszX 2015-07-09 10:13 ` Thomas Monjalon 0 siblings, 1 reply; 126+ messages in thread From: Kulasek, TomaszX @ 2015-07-09 9:55 UTC (permalink / raw) To: Thomas Monjalon; +Cc: dev > From: Thomas Monjalon [mailto:thomas.monjalon@6wind.com] > Sent: Thursday, July 9, 2015 03:58 > To: Kulasek, TomaszX > Cc: dev@dpdk.org > Subject: Re: [dpdk-dev] [PATCH v3 5/8] ring: queue stats mapping set dummy > implementation > > 2015-06-29 16:50, Tomasz Kulasek: > > Per queue statistics are already implemented for ring device, but with > > static mapping (stat_idx == queue_id). > > > > This fix is required, if you want to use ring device in test > > application and is used only to point that per queue statistics are > > provided for this device. > > > > Signed-off-by: Tomasz Kulasek <tomaszx.kulasek@intel.com> > [...] > > +static int > > +eth_queue_stats_mapping_set(__rte_unused struct rte_eth_dev *dev, > > + __rte_unused uint16_t queue_id, __rte_unused uint8_t stat_idx, > > + __rte_unused uint8_t is_rx) > > +{ > > + /* Do nothing, just return ok */ > > + return 0; > > +} > > I've just realized how this is broken. > Some Intel devices use a mapping to select hardware queues which will have > some stats. But we may have stats per queues without requiring such > mapping. > > I may miss something but I suggest these 3 actions: > - remove this patch > - replace checks on stats_mapping by an ethdev flag > - remove device-specific stats_mapping from ethdev For Niantic NICs all queues stats for port are mapped to stats_idx=0 by default, and stats mapping is required to have per-queue statistics (even in testpmd). Anyway, this patch, for ring pmd, was intended more as a cleanup than feature and was inspired by implementation in virtio driver: virtio_ethdev.c: /* * It enables testpmd to collect per queue stats. */ static int virtio_dev_queue_stats_mapping_set(__rte_unused struct rte_eth_dev *eth_dev, __rte_unused uint16_t queue_id, __rte_unused uint8_t stat_idx, __rte_unused uint8_t is_rx) { return 0; } This patch can be safely removed, if you think such a cleanup is not required, or lack of this implementation should be common behavior for this case. That will cause only few more warning messages, if you want to use ring pmd as a slave in example application. Do you need v4? ^ permalink raw reply [flat|nested] 126+ messages in thread
* Re: [dpdk-dev] [PATCH v3 5/8] ring: queue stats mapping set dummy implementation 2015-07-09 9:55 ` Kulasek, TomaszX @ 2015-07-09 10:13 ` Thomas Monjalon 0 siblings, 0 replies; 126+ messages in thread From: Thomas Monjalon @ 2015-07-09 10:13 UTC (permalink / raw) To: Kulasek, TomaszX; +Cc: dev 2015-07-09 09:55, Kulasek, TomaszX: > > From: Thomas Monjalon [mailto:thomas.monjalon@6wind.com] > > Sent: Thursday, July 9, 2015 03:58 > > To: Kulasek, TomaszX > > Cc: dev@dpdk.org > > Subject: Re: [dpdk-dev] [PATCH v3 5/8] ring: queue stats mapping set dummy > > implementation > > > > 2015-06-29 16:50, Tomasz Kulasek: > > > Per queue statistics are already implemented for ring device, but with > > > static mapping (stat_idx == queue_id). > > > > > > This fix is required, if you want to use ring device in test > > > application and is used only to point that per queue statistics are > > > provided for this device. > > > > > > Signed-off-by: Tomasz Kulasek <tomaszx.kulasek@intel.com> > > [...] > > > +static int > > > +eth_queue_stats_mapping_set(__rte_unused struct rte_eth_dev *dev, > > > + __rte_unused uint16_t queue_id, __rte_unused uint8_t stat_idx, > > > + __rte_unused uint8_t is_rx) > > > +{ > > > + /* Do nothing, just return ok */ > > > + return 0; > > > +} > > > > I've just realized how this is broken. > > Some Intel devices use a mapping to select hardware queues which will have > > some stats. But we may have stats per queues without requiring such > > mapping. > > > > I may miss something but I suggest these 3 actions: > > - remove this patch > > - replace checks on stats_mapping by an ethdev flag > > - remove device-specific stats_mapping from ethdev > > > For Niantic NICs all queues stats for port are mapped to stats_idx=0 by default, and stats mapping is required to have per-queue statistics (even in testpmd). > > Anyway, this patch, for ring pmd, was intended more as a cleanup than feature and was inspired by implementation in virtio driver: > > virtio_ethdev.c: > > /* > * It enables testpmd to collect per queue stats. > */ It should be removed in virtio. > static int > virtio_dev_queue_stats_mapping_set(__rte_unused struct rte_eth_dev *eth_dev, > __rte_unused uint16_t queue_id, __rte_unused uint8_t stat_idx, > __rte_unused uint8_t is_rx) > { > return 0; > } > > > This patch can be safely removed, if you think such a cleanup is not required, OK > or lack of this implementation should be common behavior for this case. > That will cause only few more warning messages, if you want to use ring pmd > as a slave in example application. > > Do you need v4? No, thank you ^ permalink raw reply [flat|nested] 126+ messages in thread
* [dpdk-dev] [PATCH v3 6/8] examples: dynamic rss configuration for bonding 2015-06-29 14:50 ` [dpdk-dev] [PATCH v3 " Tomasz Kulasek ` (4 preceding siblings ...) 2015-06-29 14:50 ` [dpdk-dev] [PATCH v3 5/8] ring: queue stats mapping set dummy implementation Tomasz Kulasek @ 2015-06-29 14:50 ` Tomasz Kulasek 2015-07-13 11:20 ` Thomas Monjalon 2015-06-29 14:50 ` [dpdk-dev] [PATCH v3 7/8] doc: fixed spellings and typos Tomasz Kulasek ` (3 subsequent siblings) 9 siblings, 1 reply; 126+ messages in thread From: Tomasz Kulasek @ 2015-06-29 14:50 UTC (permalink / raw) To: dev This application allows you to test RSS configuration for bonded devices changing configuration dynamically, during receiving packets on slaves and observe the changes in its distribution over queues. After initialization process, all accessible ports are attached to one bonding as slaves. Monitor screen is divided into five main parts: - Port selection (on the very top) - RSS Configuration for selected port including hash function, key and RETA - Incoming packets statistics - Incoming packets list for selected port - Status bar with contextual information about selected part Signed-off-by: Tomasz Kulasek <tomaszx.kulasek@intel.com> --- examples/bond_rss/Makefile | 59 +++ examples/bond_rss/bondrss.c | 293 ++++++++++++++ examples/bond_rss/bondrss.h | 163 ++++++++ examples/bond_rss/config.c | 251 ++++++++++++ examples/bond_rss/ui.c | 945 +++++++++++++++++++++++++++++++++++++++++++ 5 files changed, 1711 insertions(+) create mode 100644 examples/bond_rss/Makefile create mode 100644 examples/bond_rss/bondrss.c create mode 100644 examples/bond_rss/bondrss.h create mode 100644 examples/bond_rss/config.c create mode 100644 examples/bond_rss/ui.c diff --git a/examples/bond_rss/Makefile b/examples/bond_rss/Makefile new file mode 100644 index 0000000..db457a3 --- /dev/null +++ b/examples/bond_rss/Makefile @@ -0,0 +1,59 @@ +# BSD LICENSE +# +# Copyright(c) 2010-2015 Intel Corporation. All rights reserved. +# All rights reserved. +# +# Redistribution and use in source and binary forms, with or without +# modification, are permitted provided that the following conditions +# are met: +# +# * Redistributions of source code must retain the above copyright +# notice, this list of conditions and the following disclaimer. +# * Redistributions in binary form must reproduce the above copyright +# notice, this list of conditions and the following disclaimer in +# the documentation and/or other materials provided with the +# distribution. +# * Neither the name of Intel Corporation nor the names of its +# contributors may be used to endorse or promote products derived +# from this software without specific prior written permission. +# +# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS +# "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT +# LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR +# A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT +# OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, +# SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT +# LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, +# DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY +# THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT +# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE +# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + +ifeq ($(RTE_SDK),) +$(error "Please define RTE_SDK environment variable") +endif + +# Default target, can be overridden by command line or environment +RTE_TARGET ?= x86_64-native-linuxapp-gcc + +include $(RTE_SDK)/mk/rte.vars.mk + +# binary name +APP = bondrss + +# all source are stored in SRCS-y +SRCS-y := bondrss.c +SRCS-y += ui.c +SRCS-y += config.c + +CFLAGS += $(WERROR_FLAGS) + +# workaround for a gcc bug with noreturn attribute +# http://gcc.gnu.org/bugzilla/show_bug.cgi?id=12603 +ifeq ($(CONFIG_RTE_TOOLCHAIN_GCC),y) +CFLAGS_main.o += -Wno-return-type +endif + +LDLIBS += -lncurses + +include $(RTE_SDK)/mk/rte.extapp.mk diff --git a/examples/bond_rss/bondrss.c b/examples/bond_rss/bondrss.c new file mode 100644 index 0000000..225bf24 --- /dev/null +++ b/examples/bond_rss/bondrss.c @@ -0,0 +1,293 @@ +/*- + * BSD LICENSE + * + * Copyright(c) 2010-2015 Intel Corporation. All rights reserved. + * All rights reserved. + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * + * * Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * * Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in + * the documentation and/or other materials provided with the + * distribution. + * * Neither the name of Intel Corporation nor the names of its + * contributors may be used to endorse or promote products derived + * from this software without specific prior written permission. + * + * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS + * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT + * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR + * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT + * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, + * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT + * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, + * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY + * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT + * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE + * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + */ + +#include "bondrss.h" + +#define RSS_HASH_KEY_LENGTH 40 + +static const struct rte_eth_conf port_conf_default = { + .rxmode = { + .mq_mode = ETH_MQ_RX_RSS, + .max_rx_pkt_len = ETHER_MAX_LEN, + .split_hdr_size = 0, + .header_split = 0, /**< Header Split disabled. */ + .hw_ip_checksum = 0, /**< IP checksum offload disabled. */ + .hw_vlan_filter = 1, /**< VLAN filtering enabled. */ + .hw_vlan_strip = 1, /**< VLAN strip enabled. */ + .hw_vlan_extend = 0, /**< Extended VLAN disabled. */ + .jumbo_frame = 0, /**< Jumbo Frame Support disabled. */ + .hw_strip_crc = 0, /**< CRC stripping by hardware disabled. */ + .enable_scatter = 0, /**< scatter rx disabled */ + }, + .rx_adv_conf = { + .rss_conf = { + .rss_key = NULL, + .rss_hf = ETH_RSS_IPV4, + }, + }, +}; + +static const struct rte_eth_conf port_conf_bonding = { + .rxmode = { + .mq_mode = ETH_MQ_RX_RSS, + .max_rx_pkt_len = ETHER_MAX_LEN, + .split_hdr_size = 0, + .header_split = 0, /**< Header Split disabled. */ + .hw_ip_checksum = 0, /**< IP checksum offload disabled. */ + .hw_vlan_filter = 1, /**< VLAN filtering enabled. */ + .hw_vlan_strip = 1, /**< VLAN strip enabled. */ + .hw_vlan_extend = 0, /**< Extended VLAN disabled. */ + .jumbo_frame = 0, /**< Jumbo Frame Support disabled. */ + .hw_strip_crc = 0, /**< CRC stripping by hardware disabled. */ + .enable_scatter = 0, /**< scatter rx disabled */ + }, + .rx_adv_conf = { + .rss_conf = { + .rss_key = NULL, + .rss_hf = ETH_RSS_IPV6, + }, + }, +}; + +/* + * Configurable number of RX/TX queues. + */ +int nb_rxq = 4; /**< Number of RX queues per port. */ +int nb_txq = 1; /**< Number of TX queues per port. */ + +int bond_port_id = -1; /**< Bonded device port id */ + +int nb_ports; +struct rte_port *ports; +rte_atomic64_t counter; /**< Received packet's counter */ + +/* + * Initializes a given port using global settings and with the rx buffers + * coming from the mbuf_pool passed as parameter + */ +static inline +int init_port(uint8_t port_id, struct rte_mempool *mbuf_pool, + struct rte_eth_conf port_conf) +{ + + int retval; + uint16_t q; + struct rte_eth_rxconf rxconf; + + struct rte_port *port; + + if (port_id >= rte_eth_dev_count()) + return -1; + + port = &ports[port_id]; + port->nb_rx_queues = nb_rxq; + port->nb_tx_queues = nb_txq; + + memcpy(&(port->dev_conf), &port_conf, sizeof(port_conf)); + retval = rte_eth_dev_configure(port_id, port->nb_rx_queues, + port->nb_tx_queues, &port_conf); + + if (retval != 0) + return retval; + + rte_eth_dev_info_get(port_id, &(port->dev_info)); + rxconf = (port->dev_info).default_rxconf; + rxconf.rx_free_thresh = 32; + + for (q = 0; q < port->nb_rx_queues; q++) { + retval = rte_eth_rx_queue_setup(port_id, q, RX_RING_SIZE, + rte_eth_dev_socket_id(port_id), &rxconf, mbuf_pool); + if (retval < 0) + return retval; + } + + for (q = 0; q < port->nb_tx_queues; q++) { + retval = rte_eth_tx_queue_setup(port_id, q, TX_RING_SIZE, + rte_eth_dev_socket_id(port_id), + &((port->dev_info).default_txconf)); + if (retval < 0) + return retval; + } + + port->bond_mode = -1; + port->promiscuous = 0; + port->enabled = 1; + + return 0; +} + +static int +rx_loop(__attribute__((unused)) void *dummy) +{ + uint8_t port_id; + int nq; + int n; + + for (;;) + for (port_id = 0; port_id < nb_ports; port_id++) { + /* Pool only bonding ports */ + if (ports[port_id].bond_mode >= 0) + for (nq = 0; nq < ports[port_id].nb_rx_queues; nq++) { + struct rte_mbuf *bufs[BURST_SIZE]; + + const uint16_t nb_rx = rte_eth_rx_burst(port_id, nq, bufs, + BURST_SIZE); + if (unlikely(nb_rx == 0)) + continue; + + ports[port_id].rxq_ipackets[nq] += nb_rx; + for (n = 0; n < nb_rx; n++) { + record_pkt(port_id, nq, bufs[n]); + rte_pktmbuf_free(bufs[n]); + } + } + } + + return 0; +} + +static uint16_t +rx_callback(uint8_t port_id, uint16_t qidx, struct rte_mbuf **pkts, + uint16_t nb_pkts, uint16_t max_pkts __rte_unused, void *_ __rte_unused) +{ + int n; + + ports[port_id].rxq_ipackets[qidx] += nb_pkts; + for (n = 0; n < nb_pkts; n++) { + rte_atomic64_inc(&counter); + pkts[n]->udata64 = counter.cnt; + record_pkt(port_id, qidx, pkts[n]); + } + + return nb_pkts; +} + +int +main(int argc, char *argv[]) +{ + struct rte_mempool *mbuf_pool; + unsigned lcore_id = 0; + uint8_t port_id, queue_id; + + /* init EAL */ + int ret = rte_eal_init(argc, argv); + + if (ret < 0) + rte_exit(EXIT_FAILURE, "Error with EAL initialization\n"); + argc -= ret; + argv += ret; + + nb_ports = rte_eth_dev_count(); + if (nb_ports < 1) + rte_exit(EXIT_FAILURE, + "At least one port must be available to create a bonding\n"); + + mbuf_pool = rte_mempool_create("MBUF_POOL", NUM_MBUFS * nb_ports, MBUF_SIZE, + MBUF_CACHE_SIZE, sizeof(struct rte_pktmbuf_pool_private), + rte_pktmbuf_pool_init, NULL, rte_pktmbuf_init, NULL, + rte_socket_id(), 0); + + if (mbuf_pool == NULL) + rte_exit(EXIT_FAILURE, "Cannot create mbuf pool\n"); + + /* Configuration of Ethernet ports. */ + ports = rte_zmalloc("bondrss: ports", + sizeof(struct rte_port) * RTE_MAX_ETHPORTS, RTE_CACHE_LINE_SIZE); + + if (ports == NULL) + rte_exit(EXIT_FAILURE, "rte_zmalloc(%d struct rte_port) failed\n", + RTE_MAX_ETHPORTS); + + /* enabled allocated ports */ + for (port_id = 0; port_id < RTE_MAX_ETHPORTS; port_id++) + ports[port_id].enabled = 0; + + /* initialize all ports */ + for (port_id = 0; port_id < nb_ports; port_id++) { + if (init_port(port_id, mbuf_pool, port_conf_default) != 0) + rte_exit(EXIT_FAILURE, "Cannot init port %"PRIu8"\n", port_id); + + /* Add rx callbacks to the slave's port */ + for (queue_id = 0; queue_id < ports[port_id].nb_rx_queues; queue_id++) + rte_eth_add_rx_callback(port_id, queue_id, rx_callback, NULL); + + } + + /* create bonding port */ + bond_port_id = rte_eth_bond_create("eth_bond", 0, 0); + if (bond_port_id < 0) + rte_exit(EXIT_FAILURE, "Cannot create bonding port\n"); + + for (port_id = 0; port_id < nb_ports; port_id++) + rte_eth_bond_slave_add(bond_port_id, port_id); + + /* count again */ + nb_ports = rte_eth_dev_count(); + + init_port(bond_port_id, mbuf_pool, port_conf_bonding); + + /* start bonding port*/ + ret = rte_eth_dev_start(bond_port_id); + + /* enable promiscuous by default */ + rte_eth_promiscuous_enable(bond_port_id); + + for (port_id = 0; port_id < nb_ports; port_id++) { + ports[port_id].bond_mode = rte_eth_bond_mode_get(port_id); + ports[port_id].promiscuous = rte_eth_promiscuous_get(port_id); + + /* map queues to stats */ + if (ports[port_id].bond_mode >= 0) { + for (queue_id = 0; queue_id < ports[port_id].nb_rx_queues; + queue_id++) { + rte_eth_dev_set_rx_queue_stats_mapping(port_id, queue_id, + queue_id); + } + } + + } + + /* Initialize received packet's counter */ + rte_atomic64_init(&counter); + + if (rte_lcore_count() <= 1) + rte_exit(EXIT_FAILURE, "More than one free lcore is needed\n"); + + lcore_id = rte_get_next_lcore(rte_lcore_id(), 1, 0); + rte_eal_remote_launch(rx_loop, NULL, lcore_id); + + /* call lcore_main on master core only */ + ui_loop(); + + return 0; +} diff --git a/examples/bond_rss/bondrss.h b/examples/bond_rss/bondrss.h new file mode 100644 index 0000000..305a690 --- /dev/null +++ b/examples/bond_rss/bondrss.h @@ -0,0 +1,163 @@ +/*- + * BSD LICENSE + * + * Copyright(c) 2015 Intel Corporation. All rights reserved. + * All rights reserved. + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * + * * Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * * Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in + * the documentation and/or other materials provided with the + * distribution. + * * Neither the name of Intel Corporation nor the names of its + * contributors may be used to endorse or promote products derived + * from this software without specific prior written permission. + * + * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS + * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT + * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR + * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT + * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, + * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT + * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, + * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY + * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT + * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE + * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + */ + +#ifndef BONDRSS_H_ +#define BONDRSS_H_ + +#include <stdint.h> +#include <inttypes.h> +#include <rte_eal.h> +#include <rte_ethdev.h> +#include <rte_cycles.h> +#include <rte_lcore.h> +#include <rte_mbuf.h> +#include <rte_malloc.h> + +#include <rte_eth_bond.h> + +#define RX_RING_SIZE 128 +#define TX_RING_SIZE 512 + +#define NUM_MBUFS 8191 +#define MBUF_SIZE (1600 + sizeof(struct rte_mbuf) + RTE_PKTMBUF_HEADROOM) +#define MBUF_CACHE_SIZE 250 +#define BURST_SIZE 32 + +/* + * Configurable number of RX/TX queues. + */ +extern int nb_rxq; /**< Number of RX queues per port. */ +extern int nb_txq; /**< Number of TX queues per port. */ + +extern int bond_port_id; /**< Bonded device port id */ +extern int bond_mode; + +struct rss_type_info { + char str[32]; + uint64_t rss_type; +}; + +static struct rss_type_info rss_type_table[] = { + {"ipv4", ETH_RSS_IPV4}, + {"ipv4-frag", ETH_RSS_FRAG_IPV4}, + {"ipv4-tcp", ETH_RSS_NONFRAG_IPV4_TCP}, + {"ipv4-udp", ETH_RSS_NONFRAG_IPV4_UDP}, + {"ipv4-sctp", ETH_RSS_NONFRAG_IPV4_SCTP}, + {"ipv4-other", ETH_RSS_NONFRAG_IPV4_OTHER}, + {"ipv6", ETH_RSS_IPV6}, + {"ipv6-frag", ETH_RSS_FRAG_IPV6}, + {"ipv6-tcp", ETH_RSS_NONFRAG_IPV6_TCP}, + {"ipv6-udp", ETH_RSS_NONFRAG_IPV6_UDP}, + {"ipv6-sctp", ETH_RSS_NONFRAG_IPV6_SCTP}, + {"ipv6-other", ETH_RSS_NONFRAG_IPV6_OTHER}, + {"l2-payload", ETH_RSS_L2_PAYLOAD}, + {"ipv6-ex", ETH_RSS_IPV6_EX}, + {"ipv6-tcp-ex", ETH_RSS_IPV6_TCP_EX}, + {"ipv6-udp-ex", ETH_RSS_IPV6_UDP_EX}, +}; + +struct rss_type_fn { + uint8_t enabled; + struct rss_type_info *info; +}; + +/** + * Buffer for last received packets + */ +#define PKT_RECORD_SIZE 256 + +struct pkt_info { + uint64_t n; /**< Packet number */ + uint8_t port_id; + uint8_t queue_id; + uint64_t ol_flags; /**< Offload features. */ + uint32_t rss; /**< RSS hash result if RSS enabled */ +}; + +struct pkt_record { + uint64_t count; /**< Total number of received packets */ + int top; + struct pkt_info pkt_info[PKT_RECORD_SIZE]; +}; + +/** + * The data structure associated with each port. + */ +struct rte_port { + struct rte_eth_dev_info dev_info; /**< PCI info + driver name */ + struct rte_eth_conf dev_conf; /**< Port configuration. */ + + uint16_t nb_rx_queues; /**< Total number of rx queues */ + uint16_t nb_tx_queues; /**< Total number of tx queues*/ + + int bond_mode; + struct ether_addr addr; + + int rss_type_count; + struct rss_type_fn rss_type_fn[RTE_DIM(rss_type_table)]; + struct rte_eth_rss_reta_entry64 reta_conf[512 / RTE_RETA_GROUP_SIZE]; + uint8_t rss_key[52]; + + + struct rte_eth_stats stats; + uint64_t rxq_ipackets[128]; + + struct pkt_record record; + + uint8_t rss; /**< RSS enabled for port */ + uint8_t promiscuous; /**< promiscuous mode enabled for port */ + uint8_t enabled; /**< port is enabled */ +}; + +extern int nb_ports; +extern struct rte_port *ports; +extern rte_atomic64_t counter; /**< Received packet's counter */ + +void update_port_info(uint8_t port_id); + +int rss_enable(uint8_t port_id, int enabled); + +int key_set(uint8_t port_id, uint8_t *key, uint8_t len); +int key_set_random(uint8_t port_id, uint8_t len); + +int reta_set_default(uint8_t port_id); +int reta_set_random(uint8_t port_id); +int reta_set_all(uint8_t port_id, uint8_t value); +int reta_set_weights(uint8_t port_id, uint64_t *weights); + +struct pkt_info *record_pkt(uint8_t port_id, int queue_id, struct rte_mbuf *mb); + +void ui_loop(void); + + +#endif /* BONDRSS_H_ */ diff --git a/examples/bond_rss/config.c b/examples/bond_rss/config.c new file mode 100644 index 0000000..fc10d2b --- /dev/null +++ b/examples/bond_rss/config.c @@ -0,0 +1,251 @@ +/*- + * BSD LICENSE + * + * Copyright(c) 2015 Intel Corporation. All rights reserved. + * All rights reserved. + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * + * * Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * * Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in + * the documentation and/or other materials provided with the + * distribution. + * * Neither the name of Intel Corporation nor the names of its + * contributors may be used to endorse or promote products derived + * from this software without specific prior written permission. + * + * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS + * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT + * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR + * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT + * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, + * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT + * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, + * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY + * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT + * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE + * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + */ + +#include <time.h> +#include <stdlib.h> +#include "bondrss.h" + +static const struct rte_eth_conf port_conf_default = { + .rxmode = { + .max_rx_pkt_len = ETHER_MAX_LEN, + .split_hdr_size = 0, + .header_split = 0, /**< Header Split disabled. */ + .hw_ip_checksum = 0, /**< IP checksum offload disabled. */ + .hw_vlan_filter = 1, /**< VLAN filtering enabled. */ + .hw_vlan_strip = 1, /**< VLAN strip enabled. */ + .hw_vlan_extend = 0, /**< Extended VLAN disabled. */ + .jumbo_frame = 0, /**< Jumbo Frame Support disabled. */ + .hw_strip_crc = 0, /**< CRC stripping by hardware disabled. */ + .enable_scatter = 0, /**< scatter rx disabled */ + }, + .rx_adv_conf = { + .rss_conf = { + .rss_key = NULL, + .rss_hf = ETH_RSS_IPV4, + }, + }, +}; + + +void +update_port_info(uint8_t port_id) +{ + int i; + struct rte_port *port; + + port = &ports[port_id]; + + rte_eth_dev_info_get(port_id, &(port->dev_info)); + + port->promiscuous = rte_eth_promiscuous_get(port_id); + + /* Update device information */ + rte_eth_dev_info_get(port_id, &port->dev_info); + rte_eth_macaddr_get(port_id, &port->addr); + + port->dev_conf.rx_adv_conf.rss_conf.rss_key = port->rss_key; + port->dev_conf.rx_adv_conf.rss_conf.rss_key_len = 40; + rte_eth_dev_rss_hash_conf_get(port_id, + &(port->dev_conf.rx_adv_conf.rss_conf)); + + /* select all fields to be fetched */ + for (i = 0; i < port->dev_info.reta_size / RTE_RETA_GROUP_SIZE; i++) + port->reta_conf[i].mask = ~0LL; + + rte_eth_dev_rss_reta_query(port_id, port->reta_conf, + port->dev_info.reta_size); + +} + +/** + * Try to enable RSS for port_id. + */ +int +rss_enable(uint8_t port_id, int enabled) +{ + struct rte_port *port; + int retval; + + port = &ports[port_id]; + + rte_eth_dev_stop(port_id); + + if (enabled) { + port->dev_conf.rxmode.mq_mode = ETH_MQ_RX_RSS; + port->dev_conf.rx_adv_conf.rss_conf.rss_hf = ETH_RSS_IPV4; + } else { + port->dev_conf.rxmode.mq_mode = 0; + port->dev_conf.rx_adv_conf.rss_conf.rss_hf = 0; + } + + retval = rte_eth_dev_configure(port_id, port->nb_rx_queues, + port->nb_tx_queues, &port->dev_conf); + if (retval != 0) + return retval; + + retval = rte_eth_dev_start(port_id); + + return 0; +} + +int +key_set(uint8_t port_id, uint8_t *key, uint8_t len) +{ + struct rte_eth_rss_conf rss_conf; + + int i; + + rss_conf.rss_key = NULL; + rss_conf.rss_key_len = 0; + i = rte_eth_dev_rss_hash_conf_get(port_id, &rss_conf); + + rss_conf.rss_key = key; + rss_conf.rss_key_len = len; + + i = rte_eth_dev_rss_hash_update(port_id, + &rss_conf); + + return i; +} + +/** + * Set random RSS key value + */ +int +key_set_random(uint8_t port_id, uint8_t len) +{ + int i; + uint8_t key[40]; + + for (i = 0; i < len; i++) + key[i] = rand() % 256; + + return key_set(port_id, key, len); +} + +/** + * Set random RETA values + */ +int +reta_set_random(uint8_t port_id) +{ + struct rte_eth_rss_reta_entry64 reta_conf[512/RTE_RETA_GROUP_SIZE]; + int i, j; + + int reta_size = ports[port_id].dev_info.reta_size; + + for (i = 0; i < reta_size / RTE_RETA_GROUP_SIZE; i++) { + /* select all fields to set */ + reta_conf[i].mask = ~0LL; + for (j = 0; j < RTE_RETA_GROUP_SIZE; j++) + reta_conf[i].reta[j] = rand() % ports[port_id].nb_rx_queues; + } + + return rte_eth_dev_rss_reta_update(port_id, reta_conf, reta_size); +} + +/** + * Set default RETA values + */ +int +reta_set_default(uint8_t port_id) +{ + struct rte_eth_rss_reta_entry64 reta_conf[512/RTE_RETA_GROUP_SIZE]; + int i, j; + + int reta_size = ports[port_id].dev_info.reta_size; + + for (i = 0; i < reta_size / RTE_RETA_GROUP_SIZE; i++) { + /* select all fields to set */ + reta_conf[i].mask = ~0LL; + for (j = 0; j < RTE_RETA_GROUP_SIZE; j++) + reta_conf[i].reta[j] = j % nb_rxq; + } + + return rte_eth_dev_rss_reta_update(port_id, reta_conf, reta_size); +} + +/** + * Fill the RETA with values + */ +int +reta_set_all(uint8_t port_id, uint8_t value) +{ + struct rte_eth_rss_reta_entry64 reta_conf[512/RTE_RETA_GROUP_SIZE]; + int i, j; + + int reta_size = ports[port_id].dev_info.reta_size; + + for (i = 0; i < reta_size / RTE_RETA_GROUP_SIZE; i++) { + /* select all fields to set */ + reta_conf[i].mask = ~0LL; + for (j = 0; j < RTE_RETA_GROUP_SIZE; j++) + reta_conf[i].reta[j] = value; + } + + return rte_eth_dev_rss_reta_update(port_id, reta_conf, reta_size); +} + +/** + * Set the RETA basing on weights table + */ +int +reta_set_weights(uint8_t port_id, uint64_t *weights) +{ + struct rte_eth_rss_reta_entry64 reta_conf[512/RTE_RETA_GROUP_SIZE]; + unsigned i, j, k; + + unsigned reta_size = ports[port_id].dev_info.reta_size; + + uint64_t sum = 0, sum2; + + for (i = 0; i < ports[port_id].nb_rx_queues; i++) + sum += weights[i]; + sum2 = sum * (ports[port_id].nb_rx_queues-1); + + if (sum2 == 0) + return 0; + + for (i = 0; i < reta_size / RTE_RETA_GROUP_SIZE; i++) + reta_conf[i].mask = ~0LL; + k = 0; + for (i = 0; i < ports[port_id].nb_rx_queues; i++) { + for (j = 0; k < reta_size && + j < ((sum - weights[i]) * reta_size + 1) / sum2; j++) { + reta_conf[k/RTE_RETA_GROUP_SIZE].reta[k%RTE_RETA_GROUP_SIZE] = i; + k++; + } + } + + return rte_eth_dev_rss_reta_update(port_id, reta_conf, reta_size); +} diff --git a/examples/bond_rss/ui.c b/examples/bond_rss/ui.c new file mode 100644 index 0000000..d2babe5 --- /dev/null +++ b/examples/bond_rss/ui.c @@ -0,0 +1,945 @@ +/*- + * BSD LICENSE + * + * Copyright(c) 2015 Intel Corporation. All rights reserved. + * All rights reserved. + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * + * * Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * * Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in + * the documentation and/or other materials provided with the + * distribution. + * * Neither the name of Intel Corporation nor the names of its + * contributors may be used to endorse or promote products derived + * from this software without specific prior written permission. + * + * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS + * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT + * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR + * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT + * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, + * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT + * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, + * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY + * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT + * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE + * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + */ + +#include <rte_atomic.h> + +#include <ncurses.h> +#include <string.h> +#include <signal.h> +#include <sys/ioctl.h> +#include <menu.h> +#include "bondrss.h" + +WINDOW *win_footer; +WINDOW *win_header; +WINDOW *win_list; +WINDOW *win_stats; +WINDOW *win_reta; +WINDOW *win_err; + +int scr_lines; +int scr_cols; + +unsigned win_header_lines; +unsigned win_list_lines; +unsigned win_stats_lines; +unsigned win_reta_lines; +unsigned win_err_lines; + +const char *ui_status_text; +static char *ui_err_text[5]; + +struct rte_port *selected_port; +static uint8_t selected_port_id; + +struct rte_port *focused_port; +static uint8_t focused_port_id; + +#define UI_MODE_PORT 0 +#define UI_MODE_RSS_FN 1 +#define UI_MODE_RSS_KEY 2 +#define UI_MODE_RETA 3 +#define UI_MODE_HELP 4 + +int ui_mode = UI_MODE_PORT; + +static uint16_t _is_lock; +static uint16_t _is_ready; + +static uint16_t _do_configure; +static uint16_t _do_repaint; +static uint16_t _do_repaint_stats; +static uint16_t _do_repaint_list; +static uint16_t _do_repaint_info; +static uint16_t _do_repaint_err; + + +struct help_page { + const char *title; + const char *body; +}; + +static struct help_page help_pages[3] = { + { + .title = "About", + + .body = + "\n\n" + " ______ ______ ______ _____ ______ ______ ______\n" + "| | | \\ / | | \\ | | \\ \\ | | \\ \\ | | | \\ / | / |\n" + "| |--| < | | | | | | | | | | | | | |__| | '------. '------.\n" + "|_|__|_/ \\_|__|_/ |_| |_| |_|_/_/ |_| \\_\\ ____|_/ ____|_/\n" + "\n\n" + "This application allows you to test RSS configuration for bonded devices\n" + "changing configuration dynamically, during receiving packets on slaves and\n" + "observe the changes in its distribution over queues.\n" + "\n" + + }, + { + .title = "Introduction", + .body = "\n\n" + "After initialization process, all accessible ports are attached to one\n" + "bonding as slaves.\n" + "\n" + + "Monitor screen is divided into five main parts:\n" + " - Port selection (on the very top)\n" + " - RSS Configuration for selected port including hash function, key and\n" + " RETA\n" + " - Incoming packets statistics\n" + " - Incoming packets list for selected port\n" + " - Status bar with contextual information about selected part\n" + "\n\n" + + "[up], [down] arrows key lets you highlight a part and parameter to change." + }, + { + .title = "Key bindings", + .body = + "_Port_selection_\n\n" + " [left], [right] - select port,\n" + " [space] - force turning on/off RSS mode,\n" + " [p] - turn on/off promiscuous mode,\n" + " [c] - clear statistics and incoming packet list,\n" + "\n" + "_RSS_hash_function_\n\n" + " [1]-[9] - toggle selected hash function,\n" + " [a] - select all,\n" + "\n" + "_RSS_key_\n\n" + " [1]-[4] - set one of predefined key value,\n" + " [r] - randomize value,\n" + "\n" + "_RSS_RETA_\n\n" + " [number] - fill RETA with number,\n" + " [r] - randomize value,\n" + " [d] - set default value," + + } +}; + +static int +ui_lock(int wait) +{ + int result; + + while (!(result = rte_atomic16_cmpset(&_is_lock, 0, 1)) && wait) + rte_pause(); + + return result; +} + +static int +ui_unlock(void) +{ + return rte_atomic16_cmpset(&_is_lock, 1, 0); +} + +/** + * Predefined key values + */ +static uint8_t RSS_KEY[4][40] = { + { 0x6D, 0x5A, 0x56, 0xDA, 0x25, 0x5B, 0x0E, 0xC2, 0x41, 0x67, 0x25, 0x3D, + 0x43, 0xA3, 0x8F, 0xB0, 0xD0, 0xCA, 0x2B, 0xCB, 0xAE, 0x7B, 0x30, 0xB4, + 0x77, 0xCB, 0x2D, 0xA3, 0x80, 0x30, 0xF2, 0x0C, 0x6A, 0x42, 0xB7, 0x3B, + 0xBE, 0xAC, 0x01, 0xFA + }, + { 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, + }, + { 0x6D, 0x5A, 0x6D, 0x5A, 0x6D, 0x5A, 0x6D, 0x5A, 0x6D, 0x5A, 0x6D, 0x5A, + 0x6D, 0x5A, 0x6D, 0x5A, 0x6D, 0x5A, 0x6D, 0x5A, 0x6D, 0x5A, 0x6D, 0x5A, + 0x6D, 0x5A, 0x6D, 0x5A, 0x6D, 0x5A, 0x6D, 0x5A, 0x6D, 0x5A, 0x6D, 0x5A, + 0x6D, 0x5A, 0x6D, 0x5A, + }, + { 0x01, 0x02, 0x03, 0x04, 0x05, 0x06, 0x07, 0x08, 0x09, 0x0A, 0x0B, 0x0C, + 0x0D, 0x0E, 0x0F, 0x10, 0x11, 0x12, 0x13, 0x14, 0x15, 0x16, 0x17, 0x18, + 0x19, 0x1A, 0x1B, 0x1C, 0x1D, 0x1E, 0x1F, 0x20, 0x21, 0x22, 0x23, 0x24, + 0x25, 0x26, 0x27, 0x28, + } +}; + +static void +win_err_addline(const char *text) +{ + int i; + + if (win_err_lines < 5) { + win_err_lines++; + _do_configure = 1; + } else { + free(ui_err_text[0]); + for (i = 1; i < 5; i++) + ui_err_text[i - 1] = ui_err_text[i]; + } + + ui_err_text[win_err_lines - 1] = strdup(text); + _do_repaint_err = 1; +} + +static void +win_err_fini(void) +{ + int i; + + for (i = 0; i < (int)win_err_lines; i++) + free(ui_err_text[i]); +} + +/** + * Draw full information for port + */ +static void +win_header_draw(uint8_t port_id) +{ + int i, j; + attr_t a; + + update_port_info(selected_port_id); + + /* Draw title bar */ + if (ui_mode == UI_MODE_PORT) + wattron(win_header, COLOR_PAIR(2)); + else + wattron(win_header, COLOR_PAIR(4)); + + mvwprintw(win_header, 0, 0, "::"); + + for (i = 0; i < nb_ports; i++) { + if (ports[i].enabled) { + const char *name = (ports[i].bond_mode >= 0 ? "BOND" : "SLAVE"); + + if (i == port_id) { + wattron(win_header, A_REVERSE); + wprintw(win_header, " %s-%d ", name, (unsigned) i); + wattroff(win_header, A_REVERSE); + } else + wprintw(win_header, " %s-%d ", name, (unsigned) i); + } + } + + for (i = scr_cols - getcurx(win_header) - 3; i > 0; i--) + waddch(win_header, ':'); + + waddch(win_header, + (ports[port_id].rss ? 'R' : '-')); + waddch(win_header, ports[port_id].bond_mode >= 0 ? + ports[port_id].bond_mode + '0' : '-'); + waddch(win_header, ports[port_id].promiscuous == 0 ? '-' : 'P'); + + if (ui_mode == UI_MODE_PORT) + wattroff(win_header, COLOR_PAIR(2)); + else + wattroff(win_header, COLOR_PAIR(4)); + + /* Redraw RSS-Fn */ + selected_port->rss_type_count = 0; + for (i = 0; i < (int) RTE_DIM(rss_type_table); i++) { + if (selected_port->dev_info.flow_type_rss_offloads + & rss_type_table[i].rss_type) { + selected_port->rss_type_fn[selected_port->rss_type_count].info = + &rss_type_table[i]; + selected_port->rss_type_fn[selected_port->rss_type_count].enabled = + ((selected_port->dev_conf.rx_adv_conf.rss_conf.rss_hf + & rss_type_table[i].rss_type) ? 1 : 0); + selected_port->rss_type_count++; + } + } + + a = (ui_mode == UI_MODE_RSS_FN ? COLOR_PAIR(2) : COLOR_PAIR(1)); + + wattron(win_header, a); + mvwprintw(win_header, 1, 0, "FN: "); + for (i = 0; i < selected_port->rss_type_count; i++) { + if (selected_port->rss_type_fn[i].enabled) + wattron(win_header, COLOR_PAIR(3)); + waddstr(win_header, selected_port->rss_type_fn[i].info->str); + if (selected_port->rss_type_fn[i].enabled) + wattron(win_header, a); + waddch(win_header, ' '); + } + wattroff(win_header, a); + + /* Redraw RSS-Key */ + if (ui_mode == UI_MODE_RSS_KEY) + wattron(win_header, COLOR_PAIR(2)); + mvwprintw(win_header, 2, 0, "KEY: "); + for (i = 0; i < 40; i++) + wprintw(win_header, "%02X", + selected_port->dev_conf.rx_adv_conf.rss_conf.rss_key[i]); + if (ui_mode == UI_MODE_RSS_KEY) + wattroff(win_header, COLOR_PAIR(2)); + + + /* Redraw RETA window */ + int idx, shift; + + if (ui_mode == UI_MODE_RETA) + wattron(win_reta, COLOR_PAIR(2)); + + for (j = 0; j < selected_port->dev_info.reta_size / 16; j++) { + wmove(win_reta, j, 0); + for (i = 0; i < 16; i++) { + idx = (j*16 + i) / RTE_RETA_GROUP_SIZE; + shift = (j*16 + i) % RTE_RETA_GROUP_SIZE; + waddch(win_reta, ACS_VLINE); + wprintw(win_reta, "%d", selected_port->reta_conf[idx].reta[shift]); + } + waddch(win_reta, ACS_VLINE); + } + + if (ui_mode == UI_MODE_RETA) + wattroff(win_reta, COLOR_PAIR(2)); + wnoutrefresh(win_reta); + + + /* Stats repaint */ + if (_do_repaint_stats) { + uint64_t total; + + rte_eth_stats_get(selected_port_id, &(selected_port->stats)); + + wmove(win_stats, 0, 0); + total = 0; + for (i = 0; i < selected_port->nb_rx_queues; i++) { + wprintw(win_stats, "Queue %d: %10"PRIu64 + " (%10" PRIu64 ")\n", i, selected_port->rxq_ipackets[i], + selected_port->stats.q_ipackets[i]); + total += selected_port->rxq_ipackets[i]; + } + + wprintw(win_stats, " Total: %10"PRIu64 + " (%10" PRIu64 ")\n", total, selected_port->stats.ipackets); + + _do_repaint_stats = 0; + wnoutrefresh(win_stats); + } + + if (_do_repaint_err && win_err_lines > 0) { + for (i = 0; i < (int)win_err_lines; i++) { + mvwprintw(win_err, i, 0, ui_err_text[i]); + wclrtoeol(win_err); + } + _do_repaint_err = 0; + wnoutrefresh(win_err); + } + + mvwhline(win_header, win_header_lines - 1, 0, 0, scr_cols); +} + +static void +win_footer_draw(const char *text) +{ + if (text != NULL) + ui_status_text = text; + + wclear(win_footer); + wmove(win_footer, 0, 0); + wprintw(win_footer, ui_status_text); + + wprintw(win_footer, "; [arrows]navigate; [q]quit"); + wnoutrefresh(win_footer); +} + +static void +win_list_draw(void) +{ + unsigned n, i; + + struct pkt_info *pkt_info; + + struct rte_port *port = &ports[focused_port_id]; + struct pkt_record *record = &(port->record); + + n = (win_list_lines > record->count) ? record->count : win_list_lines; + + wmove(win_list, 0, 0); + + for (i = 0; i < n; i++) { + + pkt_info = &record->pkt_info[(record->top - n + i + PKT_RECORD_SIZE) + % PKT_RECORD_SIZE]; + + wmove(win_list, i, 0); + + wprintw(win_list, "%5"PRIu64, (unsigned) pkt_info->n); + waddch(win_list, ACS_VLINE); + + wprintw(win_list, "%2d: ", (unsigned) pkt_info->queue_id); + wprintw(win_list, "%08x ", (unsigned) pkt_info->rss); + + if (pkt_info->ol_flags != 0) { + unsigned rxf; + const char *name; + + for (rxf = 0; rxf < sizeof(pkt_info->ol_flags) * 8; rxf++) { + if ((pkt_info->ol_flags & (1ULL << rxf)) == 0) + continue; + name = rte_get_rx_ol_flag_name(1ULL << rxf); + if (name == NULL) + continue; + wprintw(win_list, "%s ", name); + } + } + wclrtoeol(win_list); + } + + wclrtobot(win_list); + wnoutrefresh(win_list); +} + +static void +ui_repaint(void) +{ + switch (ui_mode) { + default: + win_header_draw(selected_port_id); + wnoutrefresh(win_header); + + if (_do_repaint_list) { + win_list_draw(); + _do_repaint_list = 0; + } + + break; + } + + win_footer_draw(NULL); + _do_repaint = 0; +} + +static void +ui_mode_set(int mode) +{ + ui_mode = (mode + 4) % 4; + switch (ui_mode) { + case UI_MODE_PORT: + ui_status_text = "Port: [p]promiscuous; [c]clear stats"; + break; + case UI_MODE_RSS_FN: + ui_status_text = "RSS Fn: [1-9]toggle fn"; + break; + case UI_MODE_RETA: + ui_status_text = + "RETA: [d]set default; [r]randomize; [0-9]fill table with value"; + break; + case UI_MODE_RSS_KEY: + ui_status_text = "RSS Key: [r]randomize; [1-4]select predefined key"; + break; + } + _do_repaint = 1; +} + +static void +ui_configure(void) +{ + int win_reta_cols = (16 * 2 + 1); + + win_reta_lines = selected_port->dev_info.reta_size / 16; + win_stats_lines = selected_port->nb_rx_queues + 1; + win_header_lines = win_reta_lines; + if (win_header_lines < win_stats_lines) + win_header_lines = win_stats_lines; + win_header_lines += 5; + win_list_lines = scr_lines - win_header_lines - win_err_lines - 1; + + if (win_footer == NULL) { + win_footer = newwin(1, scr_cols, scr_lines - 1, 0); + ui_status_text = ""; + wbkgdset(win_footer, COLOR_PAIR(1)); + } else { + wresize(win_footer, 1, scr_cols); + mvwin(win_footer, scr_lines - 1, 0); + } + + if (win_err == NULL) { + win_err = newwin(1, scr_cols, scr_lines - 1 - win_err_lines, 0); + ui_status_text = ""; + wbkgdset(win_err, COLOR_PAIR(5)); + } else { + wresize(win_err, win_err_lines, scr_cols); + mvwin(win_err, scr_lines - win_err_lines - 1, 0); + } + + + if (win_header == NULL) { + win_header = newwin(win_header_lines, scr_cols, 0, 0); + wbkgdset(win_header, COLOR_PAIR(1)); + } else + wresize(win_header, win_header_lines, scr_cols); + + if (win_stats == NULL) + win_stats = subwin(win_header, win_stats_lines, + scr_cols - win_reta_cols - 7, 4, win_reta_cols + 7); + else + wresize(win_stats, win_stats_lines, scr_cols - win_reta_cols - 7); + + if (win_reta == NULL) + win_reta = subwin(win_header, win_reta_lines, win_reta_cols, + 4, 5); + else + wresize(win_reta, win_reta_lines, win_reta_cols); + + if (win_list == NULL) + win_list = newwin(win_list_lines, scr_cols, win_header_lines, 0); + else { + wresize(win_list, win_list_lines, scr_cols); + mvwin(win_list, win_header_lines, 0); + } + + wclear(win_header); + wclear(win_reta); + + _do_configure = 0; + _do_repaint = 1; + _do_repaint_list = 1; + _do_repaint_stats = 1; + _do_repaint_info = 1; + _do_repaint_err = 1; + +} + +static void +ui_resize(void) +{ + if (COLS != scr_cols || LINES != scr_lines) { + scr_cols = COLS; + scr_lines = LINES; + _do_configure = 1; + } +} + +static void +ui_update(void) +{ + if (ui_lock(1)) { + if (_is_ready) { + if (_do_configure) + ui_configure(); + if (_do_repaint) { + ui_repaint(); + doupdate(); + } + } + ui_unlock(); + } +} + +static void +ui_fini(void) +{ + win_err_fini(); + endwin(); +} + +static void +ui_init(void) +{ + initscr(); + start_color(); + init_pair(1, COLOR_YELLOW, COLOR_BLUE); + init_pair(2, COLOR_BLACK, COLOR_CYAN); + init_pair(3, COLOR_BLACK, COLOR_YELLOW); + init_pair(4, COLOR_BLACK, COLOR_WHITE); + init_pair(5, COLOR_YELLOW, COLOR_RED); + + cbreak(); + noecho(); + curs_set(FALSE); + keypad(stdscr, TRUE); + timeout(100); + + scr_cols = COLS; + scr_lines = LINES; + _do_configure = 1; + _is_ready = 1; +} + +static int +port_select(uint8_t port_id) +{ + if (ports[port_id].enabled == 1) { + focused_port_id = selected_port_id = port_id; + + selected_port = &ports[selected_port_id]; + focused_port = &ports[focused_port_id]; + + update_port_info(selected_port_id); + + _do_configure = 1; + _do_repaint = 1; + _do_repaint_info = 1; + _do_repaint_list = 1; + _do_repaint_stats = 1; + return selected_port_id; + } + return -1; +} + +static void +port_select_prev(void) +{ + int i; + + for (i = 1; i < nb_ports; i++) + if (port_select((selected_port_id - i + nb_ports) % nb_ports) != -1) + break; +} + +static void +port_select_next(void) +{ + int i; + + for (i = 1; i < nb_ports; i++) + if (port_select((selected_port_id + i) % nb_ports) != -1) + break; +} + +/** + * Toggle on/off RSS fn for focused port + */ +static void +rss_fn_toggle(int index) +{ + int i; + uint64_t rss_type = 0; + + if (index < 0 || index >= focused_port->rss_type_count) + return; + + focused_port->rss_type_fn[index].enabled ^= 1; /* toggle */ + + for (i = 0; i < focused_port->rss_type_count; i++) + if (focused_port->rss_type_fn[i].enabled) + rss_type |= focused_port->rss_type_fn[i].info->rss_type; + + focused_port->dev_conf.rx_adv_conf.rss_conf.rss_hf = rss_type; + + rte_eth_dev_rss_hash_update(focused_port_id, + &focused_port->dev_conf.rx_adv_conf.rss_conf); +} + +/** + * Set RSS key of focused port from predefined table of keys + */ +static int +key_set_predefined(int index) +{ + if (index < 0 || index >= (int) RTE_DIM(RSS_KEY)) + return -1; + + return key_set(focused_port_id, RSS_KEY[index], 40); +} + +/** + * Record packet from mbuf + */ +struct pkt_info * +record_pkt(uint8_t port_id, int queue_id, struct rte_mbuf *mb) +{ + struct rte_port *port = &ports[port_id]; + struct pkt_record *record = &(port->record); + struct pkt_info *pkt_info = &(record->pkt_info[record->top]); + + ui_lock(1); + + record->count++; + pkt_info->n = mb->udata64; + pkt_info->ol_flags = mb->ol_flags; + pkt_info->queue_id = queue_id; + pkt_info->rss = mb->hash.rss; + + record->top = (record->top + 1) % PKT_RECORD_SIZE; + + _do_repaint = 1; + _do_repaint_list = 1; + _do_repaint_stats = 1; + ui_unlock(); + + return pkt_info; +} + +static void +ui_help(void) { + timeout(-1); + + WINDOW *win_help_border = NULL; + WINDOW *win_help = NULL; + char title[256]; + int page = 0; + int page_prev; + int page_next; + int c; + int cols; + int lines; + int top; + int left; + + endwin(); + initscr(); + ui_resize(); + ui_update(); + + cols = scr_cols > 80 ? 80 : scr_cols; + lines = scr_lines > 25 ? 25 : scr_lines; + top = (scr_lines - lines) / 2; + left = (scr_cols - cols) / 2; + + win_help_border = newwin(lines, cols, top, left); + win_help = derwin(win_help_border, lines - 2, cols - 4, 1, 2); + wbkgdset(win_help_border, COLOR_PAIR(2)); + wbkgdset(win_help, COLOR_PAIR(2)); + + do { + page_prev = (page + RTE_DIM(help_pages) - 1) % RTE_DIM(help_pages); + page_next = (page + 1) % RTE_DIM(help_pages); + + wclear(win_help_border); + box(win_help_border, 0, 0); + + sprintf(title, "[ Help: %s ]", help_pages[page].title); + mvwprintw(win_help_border, 0, (cols - strlen(title)) / 2 - 2, "%s", + title); + + sprintf(title, "< %s ]", help_pages[page_prev].title); + mvwprintw(win_help_border, lines - 1, 1, "%s", title); + + sprintf(title, "[ Page %d of %d ]", page + 1, (int)RTE_DIM(help_pages)); + mvwprintw(win_help_border, lines - 1, (cols - strlen(title)) / 2 - 2, + "%s", title); + + sprintf(title, "[ %s >", help_pages[page_next].title); + mvwprintw(win_help_border, lines - 1, cols - strlen(title) - 1, "%s", + title); + + wrefresh(win_help_border); + + wclear(win_help); + + wmove(win_help, 0, 0); + + wprintw(win_help, help_pages[page].body); + + wrefresh(win_help); + + switch (c = getch()) { + + case KEY_RESIZE: + endwin(); + initscr(); + ui_resize(); + ui_update(); + + cols = scr_cols > 80 ? 80 : scr_cols; + lines = scr_lines > 25 ? 25 : scr_cols; + top = (scr_lines - lines) / 2; + left = (scr_cols - cols) / 2; + + wresize(win_help_border, lines, cols); + wresize(win_help, lines - 2, cols - 4); + break; + + case KEY_LEFT: + page = page_prev; + break; + + case KEY_RIGHT: + page = page_next; + break; + + default: + c = -1; /* Exit */ + break; + } + + + } while (c != -1); + + timeout(100); + delwin(win_help); + delwin(win_help_border); + _do_configure = 1; + _do_repaint = 1; +} + +/* main processing loop */ +void +ui_loop(void) +{ + int c; + int i; + int port_id; + int refresh = 0; + + ui_init(); + port_select(bond_port_id); + + ui_mode_set(UI_MODE_PORT); + + while ((c = getch()) != 'q') { + switch (c) { + case -1: + refresh++; + if (refresh % 10) + _do_configure = 1; + ui_update(); + break; + + case 'c': + /* clear stats */ + ui_lock(1); + + rte_atomic64_clear(&counter); + for (port_id = 0; port_id < RTE_MAX_ETHPORTS; port_id++) { + if (ports[port_id].enabled) { + ports[port_id].record.count = 0; + ports[port_id].record.top = 0; + for (i = 0; i < ports[port_id].nb_rx_queues; i++) + ports[port_id].rxq_ipackets[i] = 0; + rte_eth_stats_reset(port_id); + } + } + + _do_repaint = 1; + ui_unlock(); + break; + + case ' ': + focused_port->rss ^= 1; + rss_enable(focused_port_id, focused_port->rss); + _do_configure = 1; + _do_repaint = 1; + break; + + case 'p': + if (focused_port->promiscuous) + rte_eth_promiscuous_disable(focused_port_id); + else + rte_eth_promiscuous_enable(focused_port_id); + focused_port->promiscuous ^= 1; + _do_configure = 1; + _do_repaint = 1; + break; + + case KEY_RESIZE: + ui_resize(); + break; + + case KEY_UP: + ui_mode_set(ui_mode - 1); + break; + case KEY_DOWN: + ui_mode_set(ui_mode + 1); + break; + case 'h': + case '?': + ui_help(); + break; + + default: + switch (ui_mode) { + case UI_MODE_PORT: + switch (c) { + case KEY_LEFT: + port_select_prev(); + break; + + case KEY_RIGHT: + port_select_next(); + break; + + default: + i = (c - '1'); + port_select(i); + break; + } + break; + + case UI_MODE_RSS_KEY: + switch (c) { + case 'r': + if (key_set_random(focused_port_id, 40) < 0) + win_err_addline("Cannot update RSS key"); + break; + + default: + c = c - '1'; + if (key_set_predefined(c) < 0) + win_err_addline("Cannot update RSS key"); + break; + } + update_port_info(focused_port_id); + break; + + case UI_MODE_RETA: + switch (c) { + case 'r': + reta_set_random(focused_port_id); + break; + case 'd': + reta_set_default(focused_port_id); + break; + default: + c = c - '0'; + reta_set_all(focused_port_id, c); + break; + } + break; + + case UI_MODE_RSS_FN: + switch (c) { + case 'a': + /* Set all */ + focused_port->dev_conf.rx_adv_conf.rss_conf.rss_hf = + ports[focused_port_id].dev_info.flow_type_rss_offloads; + rte_eth_dev_rss_hash_update(focused_port_id, + &focused_port->dev_conf.rx_adv_conf.rss_conf); + break; + + default: + c -= '1'; + rss_fn_toggle(c); + } + break; + + } + _do_repaint = 1; + break; + } + } + ui_fini(); +} -- 1.7.9.5 ^ permalink raw reply [flat|nested] 126+ messages in thread
* Re: [dpdk-dev] [PATCH v3 6/8] examples: dynamic rss configuration for bonding 2015-06-29 14:50 ` [dpdk-dev] [PATCH v3 6/8] examples: dynamic rss configuration for bonding Tomasz Kulasek @ 2015-07-13 11:20 ` Thomas Monjalon 0 siblings, 0 replies; 126+ messages in thread From: Thomas Monjalon @ 2015-07-13 11:20 UTC (permalink / raw) To: Tomasz Kulasek; +Cc: dev 2015-06-29 16:50, Tomasz Kulasek: > This application allows you to test RSS configuration for bonded devices > changing configuration dynamically, during receiving packets on slaves and > observe the changes in its distribution over queues. > > After initialization process, all accessible ports are attached to one > bonding as slaves. > > Monitor screen is divided into five main parts: > - Port selection (on the very top) > - RSS Configuration for selected port including hash function, key and > RETA > - Incoming packets statistics > - Incoming packets list for selected port > - Status bar with contextual information about selected part > > Signed-off-by: Tomasz Kulasek <tomaszx.kulasek@intel.com> > --- > examples/bond_rss/Makefile | 59 +++ > examples/bond_rss/bondrss.c | 293 ++++++++++++++ > examples/bond_rss/bondrss.h | 163 ++++++++ > examples/bond_rss/config.c | 251 ++++++++++++ > examples/bond_rss/ui.c | 945 +++++++++++++++++++++++++++++++++++++++++++ > 5 files changed, 1711 insertions(+) This new example is not added in examples/Makefile. It introduces a new dependency (ncurses). Examples are useful to show how to use some features. Maybe you can show bonding RSS in the existing bonding example without adding bells and whistles. The example directory must be kept reasonnably maintainable. REJECTED ^ permalink raw reply [flat|nested] 126+ messages in thread
* [dpdk-dev] [PATCH v3 7/8] doc: fixed spellings and typos 2015-06-29 14:50 ` [dpdk-dev] [PATCH v3 " Tomasz Kulasek ` (5 preceding siblings ...) 2015-06-29 14:50 ` [dpdk-dev] [PATCH v3 6/8] examples: dynamic rss configuration for bonding Tomasz Kulasek @ 2015-06-29 14:50 ` Tomasz Kulasek 2015-06-29 14:50 ` [dpdk-dev] [PATCH v3 8/8] doc: dynamic rss configuration for bonding Tomasz Kulasek ` (2 subsequent siblings) 9 siblings, 0 replies; 126+ messages in thread From: Tomasz Kulasek @ 2015-06-29 14:50 UTC (permalink / raw) To: dev Signed-off-by: Tomasz Kulasek <tomaszx.kulasek@intel.com> --- .../prog_guide/link_bonding_poll_mode_drv_lib.rst | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/doc/guides/prog_guide/link_bonding_poll_mode_drv_lib.rst b/doc/guides/prog_guide/link_bonding_poll_mode_drv_lib.rst index 96e554f..03baf90 100644 --- a/doc/guides/prog_guide/link_bonding_poll_mode_drv_lib.rst +++ b/doc/guides/prog_guide/link_bonding_poll_mode_drv_lib.rst @@ -188,7 +188,7 @@ conditions are not met. If a user wishes to monitor individual slaves then they must register callbacks with that slave directly. The link bonding library also supports devices which do not implement link -status change interrupts, this is achieve by polling the devices link status at +status change interrupts, this is achieved by polling the devices link status at a defined period which is set using the ``rte_eth_bond_link_monitoring_set`` API, the default polling interval is 10ms. When a device is added as a slave to a bonding device it is determined using the ``RTE_PCI_DRV_INTR_LSC`` flag @@ -286,7 +286,7 @@ and UDP protocols for load balancing. Using Link Bonding Devices -------------------------- -The librte_pmd_bond library support two modes of device creation, the libraries +The librte_pmd_bond library supports two modes of device creation, the libraries export full C API or using the EAL command line to statically configure link bonding devices at application startup. Using the EAL option it is possible to use link bonding functionality transparently without specific knowledge of the @@ -299,7 +299,7 @@ Using the Poll Mode Driver from an Application Using the librte_pmd_bond libraries API it is possible to dynamically create and manage link bonding device from within any application. Link bonding -device are created using the ``rte_eth_bond_create`` API which requires a +devices are created using the ``rte_eth_bond_create`` API which requires a unique device name, the link bonding mode to initial the device in and finally the socket Id which to allocate the devices resources onto. After successful creation of a bonding device it must be configured using the generic Ethernet @@ -362,7 +362,7 @@ The different options are: mode=2 * slave: Defines the PMD device which will be added as slave to the bonded - device. This option can be selected multiple time, for each device to be + device. This option can be selected multiple times, for each device to be added as a slave. Physical devices should be specified using their PCI address, in the format domain:bus:devid.function -- 1.7.9.5 ^ permalink raw reply [flat|nested] 126+ messages in thread
* [dpdk-dev] [PATCH v3 8/8] doc: dynamic rss configuration for bonding 2015-06-29 14:50 ` [dpdk-dev] [PATCH v3 " Tomasz Kulasek ` (6 preceding siblings ...) 2015-06-29 14:50 ` [dpdk-dev] [PATCH v3 7/8] doc: fixed spellings and typos Tomasz Kulasek @ 2015-06-29 14:50 ` Tomasz Kulasek 2015-07-01 10:05 ` [dpdk-dev] [PATCH v3 0/8] Dynamic RSS Configuration for Bonding Declan Doherty 2015-07-15 17:26 ` [dpdk-dev] [PATCHv4 0/9] " Tomasz Kulasek 9 siblings, 0 replies; 126+ messages in thread From: Tomasz Kulasek @ 2015-06-29 14:50 UTC (permalink / raw) To: dev Documentation update about implementation details and requirements for Dynamic RSS Configuration for Bonding. Signed-off-by: Tomasz Kulasek <tomaszx.kulasek@intel.com> --- .../prog_guide/link_bonding_poll_mode_drv_lib.rst | 34 ++++++++++++++++++-- 1 file changed, 32 insertions(+), 2 deletions(-) diff --git a/doc/guides/prog_guide/link_bonding_poll_mode_drv_lib.rst b/doc/guides/prog_guide/link_bonding_poll_mode_drv_lib.rst index 03baf90..46f0296 100644 --- a/doc/guides/prog_guide/link_bonding_poll_mode_drv_lib.rst +++ b/doc/guides/prog_guide/link_bonding_poll_mode_drv_lib.rst @@ -1,5 +1,5 @@ .. BSD LICENSE - Copyright(c) 2010-2014 Intel Corporation. All rights reserved. + Copyright(c) 2010-2015 Intel Corporation. All rights reserved. All rights reserved. Redistribution and use in source and binary forms, with or without @@ -173,7 +173,28 @@ After a slave device is added to a bonded device slave is stopped using ``rte_eth_dev_stop`` and then reconfigured using ``rte_eth_dev_configure`` the RX and TX queues are also reconfigured using ``rte_eth_tx_queue_setup`` / ``rte_eth_rx_queue_setup`` with the parameters use to configure the bonding -device. +device. If RSS is enabled for bonding device, this mode is also enabled on new +slave and configured as well. + +Setting up multi-queue mode for bonding device to RSS, makes it fully +RSS-capable, so all slaves are synchronized with its configuration. This mode is +intended to provide RSS configuration on slaves transparent for client +application implementation. + +Bonding device stores its own version of RSS settings i.e. RETA, RSS hash +function and RSS key, used to set up its slaves. That let to define the meaning +of RSS configuration of bonding device as desired configuration of whole bonding +(as one unit), without pointing any of slave inside. It is required to ensure +consistency and made it more errorproof. + +RSS hash function set for bonding device, is a maximal set of RSS hash functions +supported by all bonded slaves. RETA size is a GCD of all its RETA's sizes, so +it can be easily used as a pattern providing expected behavior, even if slave +RETAs' sizes are different. If RSS Key is not set for bonded device, it's not +changed on the slaves and default key for device is used. + +All settings are managed through the bonding port API and always are propagated +in one direction (from bonding to slaves). Link Status Change Interrupts / Polling ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ @@ -207,6 +228,15 @@ these parameters. A bonding device must have a minimum of one slave before the bonding device itself can be started. +To use a bonding device dynamic RSS configuration feature effectively, it is +also required, that all slaves should be RSS-capable and support, at least one +common hash function available for each of them. Changing RSS key is only +possible, when all slave devices support the same key size. + +To prevent inconsistency on how slaves process packets, once a device is added +to a bonding device, RSS configuration should be managed through the bonding +device API, and not directly on the slave. + Like all other PMD, all functions exported by a PMD are lock-free functions that are assumed not to be invoked in parallel on different logical cores to work on the same target object. -- 1.7.9.5 ^ permalink raw reply [flat|nested] 126+ messages in thread
* Re: [dpdk-dev] [PATCH v3 0/8] Dynamic RSS Configuration for Bonding 2015-06-29 14:50 ` [dpdk-dev] [PATCH v3 " Tomasz Kulasek ` (7 preceding siblings ...) 2015-06-29 14:50 ` [dpdk-dev] [PATCH v3 8/8] doc: dynamic rss configuration for bonding Tomasz Kulasek @ 2015-07-01 10:05 ` Declan Doherty 2015-07-13 11:03 ` Thomas Monjalon 2015-07-15 17:26 ` [dpdk-dev] [PATCHv4 0/9] " Tomasz Kulasek 9 siblings, 1 reply; 126+ messages in thread From: Declan Doherty @ 2015-07-01 10:05 UTC (permalink / raw) To: Tomasz Kulasek, dev On 29/06/15 15:50, Tomasz Kulasek wrote: > OVERVIEW > -------- > 1) Setting .rxmode.mq_mode for bonding device to ETH_MQ_RX_RSS makes bonding > device fully RSS-capable, so all slaves are synchronized with its configuration. > This mode is intended to provide RSS configuration as known from "dynamic RSS > configuration for one port" and made slaves transparent for client application > implementation. > > 2) If .rxmode.mq_mode for bonding device isn't ETH_MQ_RX_RSS, slaves are not > synchronized. That provides an ability to configure them manually. This mode may > be useful when application wants to manage RSS in an unusual way and the > consistency of RSS configuration for slaves isn't required. > > Turning on/off RSS mode for slaves when bonding is started is not possible. > Other RSS configuration is propagated over slaves, when bonding device API is > used to do it. > > v3 changes: > - checkpatch cleanups ... > Acked-by : Declan Doherty <declan.doherty@intel.com> ^ permalink raw reply [flat|nested] 126+ messages in thread
* Re: [dpdk-dev] [PATCH v3 0/8] Dynamic RSS Configuration for Bonding 2015-07-01 10:05 ` [dpdk-dev] [PATCH v3 0/8] Dynamic RSS Configuration for Bonding Declan Doherty @ 2015-07-13 11:03 ` Thomas Monjalon 2015-07-13 11:18 ` Thomas Monjalon 0 siblings, 1 reply; 126+ messages in thread From: Thomas Monjalon @ 2015-07-13 11:03 UTC (permalink / raw) To: Tomasz Kulasek; +Cc: dev 2015-07-01 11:05, Declan Doherty: > On 29/06/15 15:50, Tomasz Kulasek wrote: > > OVERVIEW > > -------- > > 1) Setting .rxmode.mq_mode for bonding device to ETH_MQ_RX_RSS makes bonding > > device fully RSS-capable, so all slaves are synchronized with its configuration. > > This mode is intended to provide RSS configuration as known from "dynamic RSS > > configuration for one port" and made slaves transparent for client application > > implementation. > > > > 2) If .rxmode.mq_mode for bonding device isn't ETH_MQ_RX_RSS, slaves are not > > synchronized. That provides an ability to configure them manually. This mode may > > be useful when application wants to manage RSS in an unusual way and the > > consistency of RSS configuration for slaves isn't required. > > > > Turning on/off RSS mode for slaves when bonding is started is not possible. > > Other RSS configuration is propagated over slaves, when bonding device API is > > used to do it. > > > > v3 changes: > > - checkpatch cleanups > ... > > Acked-by : Declan Doherty <declan.doherty@intel.com> Applied without patches 5 and 6: - As discussed earlier, patch 5 workaround a missing flag to announce stats per queue availability. - Patch 6 introduce a new dependency (ncurses) to build a new bond_rss example. Examples are useful to show how to use some features. Maybe you can show bonding RSS in the existing bonding example without adding bells and whistles. The example directory must be kept reasonnably maintainable. ^ permalink raw reply [flat|nested] 126+ messages in thread
* Re: [dpdk-dev] [PATCH v3 0/8] Dynamic RSS Configuration for Bonding 2015-07-13 11:03 ` Thomas Monjalon @ 2015-07-13 11:18 ` Thomas Monjalon 2015-07-13 12:14 ` Kulasek, TomaszX 0 siblings, 1 reply; 126+ messages in thread From: Thomas Monjalon @ 2015-07-13 11:18 UTC (permalink / raw) To: Tomasz Kulasek; +Cc: dev 2015-07-13 13:03, Thomas Monjalon: > 2015-07-01 11:05, Declan Doherty: > > On 29/06/15 15:50, Tomasz Kulasek wrote: > > > OVERVIEW > > > -------- > > > 1) Setting .rxmode.mq_mode for bonding device to ETH_MQ_RX_RSS makes bonding > > > device fully RSS-capable, so all slaves are synchronized with its configuration. > > > This mode is intended to provide RSS configuration as known from "dynamic RSS > > > configuration for one port" and made slaves transparent for client application > > > implementation. > > > > > > 2) If .rxmode.mq_mode for bonding device isn't ETH_MQ_RX_RSS, slaves are not > > > synchronized. That provides an ability to configure them manually. This mode may > > > be useful when application wants to manage RSS in an unusual way and the > > > consistency of RSS configuration for slaves isn't required. > > > > > > Turning on/off RSS mode for slaves when bonding is started is not possible. > > > Other RSS configuration is propagated over slaves, when bonding device API is > > > used to do it. > > > > > > v3 changes: > > > - checkpatch cleanups > > ... > > > > Acked-by : Declan Doherty <declan.doherty@intel.com> > > Applied without patches 5 and 6: Sorry, after more review, this series won't be pushed at all. There is an obvious error in patch 4, and patch 2/3 must be discussed. > - As discussed earlier, patch 5 workaround a missing flag to announce stats > per queue availability. > > - Patch 6 introduce a new dependency (ncurses) to build a new bond_rss example. > Examples are useful to show how to use some features. Maybe you can show > bonding RSS in the existing bonding example without adding bells and whistles. > The example directory must be kept reasonnably maintainable. ^ permalink raw reply [flat|nested] 126+ messages in thread
* Re: [dpdk-dev] [PATCH v3 0/8] Dynamic RSS Configuration for Bonding 2015-07-13 11:18 ` Thomas Monjalon @ 2015-07-13 12:14 ` Kulasek, TomaszX 0 siblings, 0 replies; 126+ messages in thread From: Kulasek, TomaszX @ 2015-07-13 12:14 UTC (permalink / raw) To: Thomas Monjalon; +Cc: dev > -----Original Message----- > From: Thomas Monjalon [mailto:thomas.monjalon@6wind.com] > Sent: Monday, July 13, 2015 13:18 > To: Kulasek, TomaszX > Cc: dev@dpdk.org; Doherty, Declan > Subject: Re: [dpdk-dev] [PATCH v3 0/8] Dynamic RSS Configuration for > Bonding > > 2015-07-13 13:03, Thomas Monjalon: > > 2015-07-01 11:05, Declan Doherty: > > > On 29/06/15 15:50, Tomasz Kulasek wrote: > > > > OVERVIEW > > > > -------- > > > > 1) Setting .rxmode.mq_mode for bonding device to ETH_MQ_RX_RSS > > > > makes bonding device fully RSS-capable, so all slaves are > synchronized with its configuration. > > > > This mode is intended to provide RSS configuration as known from > > > > "dynamic RSS configuration for one port" and made slaves > > > > transparent for client application implementation. > > > > > > > > 2) If .rxmode.mq_mode for bonding device isn't ETH_MQ_RX_RSS, > > > > slaves are not synchronized. That provides an ability to configure > > > > them manually. This mode may be useful when application wants to > > > > manage RSS in an unusual way and the consistency of RSS > configuration for slaves isn't required. > > > > > > > > Turning on/off RSS mode for slaves when bonding is started is not > possible. > > > > Other RSS configuration is propagated over slaves, when bonding > > > > device API is used to do it. > > > > > > > > v3 changes: > > > > - checkpatch cleanups > > > ... > > > > > > Acked-by : Declan Doherty <declan.doherty@intel.com> > > > > Applied without patches 5 and 6: > > Sorry, after more review, this series won't be pushed at all. > There is an obvious error in patch 4, and patch 2/3 must be discussed. What's wrong with patches 2/3? Some details? > > > - As discussed earlier, patch 5 workaround a missing flag to announce > > stats per queue availability. > > > > - Patch 6 introduce a new dependency (ncurses) to build a new bond_rss > example. > > Examples are useful to show how to use some features. Maybe you can > > show bonding RSS in the existing bonding example without adding bells > and whistles. > > The example directory must be kept reasonnably maintainable. > Ok. I understand that. This application was created to help me in development process and using ncurses made it a lot easier. The intention wasn't "adding bells and whistles" but made it usable in dynamic environment. ^ permalink raw reply [flat|nested] 126+ messages in thread
* [dpdk-dev] [PATCHv4 0/9] Dynamic RSS Configuration for Bonding 2015-06-29 14:50 ` [dpdk-dev] [PATCH v3 " Tomasz Kulasek ` (8 preceding siblings ...) 2015-07-01 10:05 ` [dpdk-dev] [PATCH v3 0/8] Dynamic RSS Configuration for Bonding Declan Doherty @ 2015-07-15 17:26 ` Tomasz Kulasek 2015-07-15 17:26 ` [dpdk-dev] [PATCHv4 1/9] bonding: rss dynamic configuration Tomasz Kulasek ` (10 more replies) 9 siblings, 11 replies; 126+ messages in thread From: Tomasz Kulasek @ 2015-07-15 17:26 UTC (permalink / raw) To: dev OVERVIEW -------- 1) Setting .rxmode.mq_mode for bonding device to ETH_MQ_RX_RSS makes bonding device fully RSS-capable, so all slaves are synchronized with its configuration. This mode is intended to provide RSS configuration as known from "dynamic RSS configuration for one port" and made slaves transparent for client application implementation. 2) If .rxmode.mq_mode for bonding device isn't ETH_MQ_RX_RSS, slaves are not synchronized. That provides an ability to configure them manually. This mode may be useful when application wants to manage RSS in an unusual way and the consistency of RSS configuration for slaves isn't required. Turning on/off RSS mode for slaves when bonding is started is not possible. Other RSS configuration is propagated over slaves, when bonding device API is used to do it. v4 changes: - fixed copy-paste error, - removed example application as too complex and introducing a new dependency, - addapted null pmd to be used as testing device for dynamic RSS configuration, - addapted test units to use null pmd instead of ring pmd, - ring pmd is not used and changed in this patchset, Tomasz Kulasek (9): bonding: rss dynamic configuration null: fix segfault when null_pmd added to bonding null: extend number of virtual queues null: virtual dynamic rss configuration null: export eth_dev_null_create test: dynamic rss configuration bonding: queue stats mapping doc: fixed spellings and typos doc: dynamic rss configuration for bonding app/test/Makefile | 8 + app/test/test_link_bonding_rssconf.c | 679 ++++++++++++++++++++ .../prog_guide/link_bonding_poll_mode_drv_lib.rst | 42 +- drivers/net/bonding/rte_eth_bond_api.c | 28 + drivers/net/bonding/rte_eth_bond_pmd.c | 239 ++++++- drivers/net/bonding/rte_eth_bond_private.h | 12 + drivers/net/null/Makefile | 2 +- drivers/net/null/rte_eth_null.c | 142 +++- drivers/net/null/rte_eth_null.h | 40 ++ drivers/net/null/rte_pmd_null_version.map | 7 + 10 files changed, 1170 insertions(+), 29 deletions(-) create mode 100644 app/test/test_link_bonding_rssconf.c create mode 100644 drivers/net/null/rte_eth_null.h -- 1.7.9.5 ^ permalink raw reply [flat|nested] 126+ messages in thread
* [dpdk-dev] [PATCHv4 1/9] bonding: rss dynamic configuration 2015-07-15 17:26 ` [dpdk-dev] [PATCHv4 0/9] " Tomasz Kulasek @ 2015-07-15 17:26 ` Tomasz Kulasek 2015-07-15 17:26 ` [dpdk-dev] [PATCHv4 2/9] null: fix segfault when null_pmd added to bonding Tomasz Kulasek ` (9 subsequent siblings) 10 siblings, 0 replies; 126+ messages in thread From: Tomasz Kulasek @ 2015-07-15 17:26 UTC (permalink / raw) To: dev Bonding device implements independent management of RSS settings. It stores its own copies of settings i.e. RETA, RSS hash function and RSS key. It’s required to ensure consistency. 1) RSS hash function set for bonding device is maximal set of RSS hash functions supported by all bonded devices. That mean, to have RSS support for bonding, all slaves should be RSS-capable. 2) RSS key is propagated over the slaves "as is". 3) RETA for bonding is an internal table managed by bonding API, and is used as a pattern to set up slaves. Its size is GCD of all RETA sizes, so it can be easily used as a pattern providing expected behavior, even if slaves RETA sizes are different. Signed-off-by: Tomasz Kulasek <tomaszx.kulasek@intel.com> --- drivers/net/bonding/rte_eth_bond_api.c | 28 ++++ drivers/net/bonding/rte_eth_bond_pmd.c | 205 ++++++++++++++++++++++++++-- drivers/net/bonding/rte_eth_bond_private.h | 12 ++ 3 files changed, 231 insertions(+), 14 deletions(-) diff --git a/drivers/net/bonding/rte_eth_bond_api.c b/drivers/net/bonding/rte_eth_bond_api.c index d810ec4..22eb575 100644 --- a/drivers/net/bonding/rte_eth_bond_api.c +++ b/drivers/net/bonding/rte_eth_bond_api.c @@ -303,6 +303,9 @@ rte_eth_bond_create(const char *name, uint8_t mode, uint8_t socket_id) internals->rx_offload_capa = 0; internals->tx_offload_capa = 0; + /* Initially allow to choose any offload type */ + internals->flow_type_rss_offloads = ETH_RSS_PROTO_MASK; + memset(internals->active_slaves, 0, sizeof(internals->active_slaves)); memset(internals->slaves, 0, sizeof(internals->slaves)); @@ -366,6 +369,11 @@ __eth_bond_slave_add_lock_free(uint8_t bonded_port_id, uint8_t slave_port_id) rte_eth_dev_info_get(slave_port_id, &dev_info); + /* We need to store slaves reta_size to be able to synchronize RETA for all + * slave devices even if its sizes are different. + */ + internals->slaves[internals->slave_count].reta_size = dev_info.reta_size; + if (internals->slave_count < 1) { /* if MAC is not user defined then use MAC of first slave add to * bonded device */ @@ -379,9 +387,16 @@ __eth_bond_slave_add_lock_free(uint8_t bonded_port_id, uint8_t slave_port_id) /* Make primary slave */ internals->primary_port = slave_port_id; + /* Inherit queues settings from first slave */ + internals->nb_rx_queues = slave_eth_dev->data->nb_rx_queues; + internals->nb_tx_queues = slave_eth_dev->data->nb_tx_queues; + + internals->reta_size = dev_info.reta_size; + /* Take the first dev's offload capabilities */ internals->rx_offload_capa = dev_info.rx_offload_capa; internals->tx_offload_capa = dev_info.tx_offload_capa; + internals->flow_type_rss_offloads = dev_info.flow_type_rss_offloads; } else { /* Check slave link properties are supported if props are set, @@ -400,8 +415,19 @@ __eth_bond_slave_add_lock_free(uint8_t bonded_port_id, uint8_t slave_port_id) } internals->rx_offload_capa &= dev_info.rx_offload_capa; internals->tx_offload_capa &= dev_info.tx_offload_capa; + internals->flow_type_rss_offloads &= dev_info.flow_type_rss_offloads; + + /* RETA size is GCD of all slaves RETA sizes, so, if all sizes will be + * the power of 2, the lower one is GCD + */ + if (internals->reta_size > dev_info.reta_size) + internals->reta_size = dev_info.reta_size; + } + bonded_eth_dev->data->dev_conf.rx_adv_conf.rss_conf.rss_hf &= + internals->flow_type_rss_offloads; + internals->slave_count++; /* Update all slave devices MACs*/ @@ -528,6 +554,8 @@ __eth_bond_slave_remove_lock_free(uint8_t bonded_port_id, uint8_t slave_port_id) if (internals->slave_count == 0) { internals->rx_offload_capa = 0; internals->tx_offload_capa = 0; + internals->flow_type_rss_offloads = ETH_RSS_PROTO_MASK; + internals->reta_size = 0; } return 0; } diff --git a/drivers/net/bonding/rte_eth_bond_pmd.c b/drivers/net/bonding/rte_eth_bond_pmd.c index 989e878..cd23f42 100644 --- a/drivers/net/bonding/rte_eth_bond_pmd.c +++ b/drivers/net/bonding/rte_eth_bond_pmd.c @@ -1310,6 +1310,23 @@ slave_configure(struct rte_eth_dev *bonded_eth_dev, if (slave_eth_dev->driver->pci_drv.drv_flags & RTE_PCI_DRV_INTR_LSC) slave_eth_dev->data->dev_conf.intr_conf.lsc = 1; + /* If RSS is enabled for bonding, try to enable it for slaves */ + if (bonded_eth_dev->data->dev_conf.rxmode.mq_mode & ETH_MQ_RX_RSS) { + if (bonded_eth_dev->data->dev_conf.rx_adv_conf.rss_conf.rss_key_len + != 0) { + slave_eth_dev->data->dev_conf.rx_adv_conf.rss_conf.rss_key_len = + bonded_eth_dev->data->dev_conf.rx_adv_conf.rss_conf.rss_key_len; + slave_eth_dev->data->dev_conf.rx_adv_conf.rss_conf.rss_key = + bonded_eth_dev->data->dev_conf.rx_adv_conf.rss_conf.rss_key; + } else { + slave_eth_dev->data->dev_conf.rx_adv_conf.rss_conf.rss_key = NULL; + } + + slave_eth_dev->data->dev_conf.rx_adv_conf.rss_conf.rss_hf = + bonded_eth_dev->data->dev_conf.rx_adv_conf.rss_conf.rss_hf; + slave_eth_dev->data->dev_conf.rxmode.mq_mode |= ETH_MQ_RX_RSS; + } + /* Configure device */ errval = rte_eth_dev_configure(slave_eth_dev->data->port_id, bonded_eth_dev->data->nb_rx_queues, @@ -1361,6 +1378,30 @@ slave_configure(struct rte_eth_dev *bonded_eth_dev, return -1; } + /* If RSS is enabled for bonding, synchronize RETA */ + if (bonded_eth_dev->data->dev_conf.rxmode.mq_mode & ETH_MQ_RX_RSS) { + int i; + struct bond_dev_private *internals; + + internals = bonded_eth_dev->data->dev_private; + + for (i = 0; i < internals->slave_count; i++) { + if (internals->slaves[i].port_id == slave_eth_dev->data->port_id) { + errval = rte_eth_dev_rss_reta_update( + slave_eth_dev->data->port_id, + &internals->reta_conf[0], + internals->slaves[i].reta_size); + if (errval != 0) { + RTE_LOG(WARNING, PMD, + "rte_eth_dev_rss_reta_update on slave port %d fails (err %d)." + " RSS Configuration for bonding may be inconsistent.\n", + slave_eth_dev->data->port_id, errval); + } + break; + } + } + } + /* If lsc interrupt is set, check initial slave's link status */ if (slave_eth_dev->driver->pci_drv.drv_flags & RTE_PCI_DRV_INTR_LSC) bond_ethdev_lsc_event_callback(slave_eth_dev->data->port_id, @@ -1578,6 +1619,9 @@ bond_ethdev_info(struct rte_eth_dev *dev, struct rte_eth_dev_info *dev_info) dev_info->rx_offload_capa = internals->rx_offload_capa; dev_info->tx_offload_capa = internals->tx_offload_capa; + dev_info->flow_type_rss_offloads = internals->flow_type_rss_offloads; + + dev_info->reta_size = internals->reta_size; } static int @@ -1959,21 +2003,132 @@ bond_ethdev_lsc_event_callback(uint8_t port_id, enum rte_eth_event_type type, } } +static int +bond_ethdev_rss_reta_update(struct rte_eth_dev *dev, + struct rte_eth_rss_reta_entry64 *reta_conf, uint16_t reta_size) +{ + unsigned i, j; + int result = 0; + int slave_reta_size; + unsigned reta_count; + struct bond_dev_private *internals = dev->data->dev_private; + + if (reta_size != internals->reta_size) + return -EINVAL; + + /* Copy RETA table */ + reta_count = reta_size / RTE_RETA_GROUP_SIZE; + + for (i = 0; i < reta_count; i++) { + internals->reta_conf[i].mask = reta_conf[i].mask; + for (j = 0; j < RTE_RETA_GROUP_SIZE; j++) + if ((reta_conf[i].mask >> j) & 0x01) + internals->reta_conf[i].reta[j] = reta_conf[i].reta[j]; + } + + /* Fill rest of array */ + for (; i < RTE_DIM(internals->reta_conf); i += reta_count) + memcpy(&internals->reta_conf[i], &internals->reta_conf[0], + sizeof(internals->reta_conf[0]) * reta_count); + + /* Propagate RETA over slaves */ + for (i = 0; i < internals->slave_count; i++) { + slave_reta_size = internals->slaves[i].reta_size; + result = rte_eth_dev_rss_reta_update(internals->slaves[i].port_id, + &internals->reta_conf[0], slave_reta_size); + if (result < 0) + return result; + } + + return 0; +} + +static int +bond_ethdev_rss_reta_query(struct rte_eth_dev *dev, + struct rte_eth_rss_reta_entry64 *reta_conf, uint16_t reta_size) +{ + int i, j; + struct bond_dev_private *internals = dev->data->dev_private; + + if (reta_size != internals->reta_size) + return -EINVAL; + + /* Copy RETA table */ + for (i = 0; i < reta_size / RTE_RETA_GROUP_SIZE; i++) + for (j = 0; j < RTE_RETA_GROUP_SIZE; j++) + if ((reta_conf[i].mask >> j) & 0x01) + reta_conf[i].reta[j] = internals->reta_conf[i].reta[j]; + + return 0; +} + +static int +bond_ethdev_rss_hash_update(struct rte_eth_dev *dev, + struct rte_eth_rss_conf *rss_conf) +{ + int i, result = 0; + struct bond_dev_private *internals = dev->data->dev_private; + struct rte_eth_rss_conf bond_rss_conf; + + memcpy(&bond_rss_conf, rss_conf, sizeof(struct rte_eth_rss_conf)); + + bond_rss_conf.rss_hf &= internals->flow_type_rss_offloads; + + if (bond_rss_conf.rss_hf != 0) + dev->data->dev_conf.rx_adv_conf.rss_conf.rss_hf = bond_rss_conf.rss_hf; + + if (bond_rss_conf.rss_key && bond_rss_conf.rss_key_len < + sizeof(internals->rss_key)) { + if (bond_rss_conf.rss_key_len == 0) + bond_rss_conf.rss_key_len = 40; + internals->rss_key_len = bond_rss_conf.rss_key_len; + memcpy(internals->rss_key, bond_rss_conf.rss_key, + internals->rss_key_len); + } + + for (i = 0; i < internals->slave_count; i++) { + result = rte_eth_dev_rss_hash_update(internals->slaves[i].port_id, + &bond_rss_conf); + if (result < 0) + return result; + } + + return 0; +} + +static int +bond_ethdev_rss_hash_conf_get(struct rte_eth_dev *dev, + struct rte_eth_rss_conf *rss_conf) +{ + struct bond_dev_private *internals = dev->data->dev_private; + + rss_conf->rss_hf = dev->data->dev_conf.rx_adv_conf.rss_conf.rss_hf; + rss_conf->rss_key_len = internals->rss_key_len; + if (rss_conf->rss_key) + memcpy(rss_conf->rss_key, internals->rss_key, internals->rss_key_len); + + return 0; +} + struct eth_dev_ops default_dev_ops = { - .dev_start = bond_ethdev_start, - .dev_stop = bond_ethdev_stop, - .dev_close = bond_ethdev_close, - .dev_configure = bond_ethdev_configure, - .dev_infos_get = bond_ethdev_info, - .rx_queue_setup = bond_ethdev_rx_queue_setup, - .tx_queue_setup = bond_ethdev_tx_queue_setup, - .rx_queue_release = bond_ethdev_rx_queue_release, - .tx_queue_release = bond_ethdev_tx_queue_release, - .link_update = bond_ethdev_link_update, - .stats_get = bond_ethdev_stats_get, - .stats_reset = bond_ethdev_stats_reset, - .promiscuous_enable = bond_ethdev_promiscuous_enable, - .promiscuous_disable = bond_ethdev_promiscuous_disable + .dev_start = bond_ethdev_start, + .dev_stop = bond_ethdev_stop, + .dev_close = bond_ethdev_close, + .dev_configure = bond_ethdev_configure, + .dev_infos_get = bond_ethdev_info, + .rx_queue_setup = bond_ethdev_rx_queue_setup, + .tx_queue_setup = bond_ethdev_tx_queue_setup, + .rx_queue_release = bond_ethdev_rx_queue_release, + .tx_queue_release = bond_ethdev_tx_queue_release, + .link_update = bond_ethdev_link_update, + .stats_get = bond_ethdev_stats_get, + .stats_reset = bond_ethdev_stats_reset, + .promiscuous_enable = bond_ethdev_promiscuous_enable, + .promiscuous_disable = bond_ethdev_promiscuous_disable, + .reta_update = bond_ethdev_rss_reta_update, + .reta_query = bond_ethdev_rss_reta_query, + .rss_hash_update = bond_ethdev_rss_hash_update, + .rss_hash_conf_get = bond_ethdev_rss_hash_conf_get }; static int @@ -2054,6 +2209,28 @@ bond_ethdev_configure(struct rte_eth_dev *dev) int arg_count; uint8_t port_id = dev - rte_eth_devices; + static const uint8_t default_rss_key[40] = { + 0x6D, 0x5A, 0x56, 0xDA, 0x25, 0x5B, 0x0E, 0xC2, 0x41, 0x67, 0x25, 0x3D, + 0x43, 0xA3, 0x8F, 0xB0, 0xD0, 0xCA, 0x2B, 0xCB, 0xAE, 0x7B, 0x30, 0xB4, + 0x77, 0xCB, 0x2D, 0xA3, 0x80, 0x30, 0xF2, 0x0C, 0x6A, 0x42, 0xB7, 0x3B, + 0xBE, 0xAC, 0x01, 0xFA + }; + + unsigned i, j; + + /* If RSS is enabled, fill table and key with default values */ + if (dev->data->dev_conf.rxmode.mq_mode & ETH_MQ_RX_RSS) { + dev->data->dev_conf.rx_adv_conf.rss_conf.rss_key = internals->rss_key; + dev->data->dev_conf.rx_adv_conf.rss_conf.rss_key_len = 0; + memcpy(internals->rss_key, default_rss_key, 40); + + for (i = 0; i < RTE_DIM(internals->reta_conf); i++) { + internals->reta_conf[i].mask = ~0LL; + for (j = 0; j < RTE_RETA_GROUP_SIZE; j++) + internals->reta_conf[i].reta[j] = j % dev->data->nb_rx_queues; + } + } + /* * if no kvlist, it means that this bonded device has been created * through the bonding api. diff --git a/drivers/net/bonding/rte_eth_bond_private.h b/drivers/net/bonding/rte_eth_bond_private.h index 45e5c65..98bb64d 100644 --- a/drivers/net/bonding/rte_eth_bond_private.h +++ b/drivers/net/bonding/rte_eth_bond_private.h @@ -103,6 +103,8 @@ struct bond_slave_details { uint8_t last_link_status; /**< Port Id of slave eth_dev */ struct ether_addr persisted_mac_addr; + + uint16_t reta_size; }; @@ -155,6 +157,16 @@ struct bond_dev_private { uint32_t rx_offload_capa; /** Rx offload capability */ uint32_t tx_offload_capa; /** Tx offload capability */ + /** Bit mask of RSS offloads, the bit offset also means flow type */ + uint64_t flow_type_rss_offloads; + + uint16_t reta_size; + struct rte_eth_rss_reta_entry64 reta_conf[ETH_RSS_RETA_SIZE_512 / + RTE_RETA_GROUP_SIZE]; + + uint8_t rss_key[52]; /**< 52-byte hash key buffer. */ + uint8_t rss_key_len; /**< hash key length in bytes. */ + struct rte_kvargs *kvlist; uint8_t slave_update_idx; }; -- 1.7.9.5 ^ permalink raw reply [flat|nested] 126+ messages in thread
* [dpdk-dev] [PATCHv4 2/9] null: fix segfault when null_pmd added to bonding 2015-07-15 17:26 ` [dpdk-dev] [PATCHv4 0/9] " Tomasz Kulasek 2015-07-15 17:26 ` [dpdk-dev] [PATCHv4 1/9] bonding: rss dynamic configuration Tomasz Kulasek @ 2015-07-15 17:26 ` Tomasz Kulasek 2015-09-29 2:24 ` Tetsuya Mukawa 2015-07-15 17:26 ` [dpdk-dev] [PATCHv4 3/9] null: extend number of virtual queues Tomasz Kulasek ` (8 subsequent siblings) 10 siblings, 1 reply; 126+ messages in thread From: Tomasz Kulasek @ 2015-07-15 17:26 UTC (permalink / raw) To: dev When device is added to the bonding, the link status callback is added to the slave's eth_dev->link_intr_cbs list. This list is not initialized for null pmd and adding it to the bonding segfaults application. This patch allocates and sets up required structures. Signed-off-by: Tomasz Kulasek <tomaszx.kulasek@intel.com> --- drivers/net/null/rte_eth_null.c | 11 +++++++++++ 1 file changed, 11 insertions(+) diff --git a/drivers/net/null/rte_eth_null.c b/drivers/net/null/rte_eth_null.c index e244595..a8b3191 100644 --- a/drivers/net/null/rte_eth_null.c +++ b/drivers/net/null/rte_eth_null.c @@ -386,6 +386,7 @@ eth_dev_null_create(const char *name, const unsigned nb_rx_queues = 1; const unsigned nb_tx_queues = 1; struct rte_eth_dev_data *data = NULL; + struct eth_driver *eth_drv = NULL; struct rte_pci_device *pci_dev = NULL; struct pmd_internals *internals = NULL; struct rte_eth_dev *eth_dev = NULL; @@ -416,6 +417,10 @@ eth_dev_null_create(const char *name, if (eth_dev == NULL) goto error; + eth_drv = rte_zmalloc_socket(name, sizeof(*eth_drv), 0, numa_node); + if (!eth_drv) + goto error; + /* now put it all together * - store queue data in internals, * - store numa_node info in pci_driver @@ -431,7 +436,10 @@ eth_dev_null_create(const char *name, internals->packet_copy = packet_copy; internals->numa_node = numa_node; + eth_drv->pci_drv.name = drivername; + pci_dev->numa_node = numa_node; + pci_dev->driver = ð_drv->pci_drv; data->dev_private = internals; data->port_id = eth_dev->data->port_id; @@ -445,6 +453,7 @@ eth_dev_null_create(const char *name, eth_dev->dev_ops = &ops; eth_dev->pci_dev = pci_dev; eth_dev->driver = &rte_null_pmd; + TAILQ_INIT(ð_dev->link_intr_cbs); /* finally assign rx and tx ops */ if (packet_copy) { @@ -461,6 +470,8 @@ error: rte_free(data); rte_free(pci_dev); rte_free(internals); + rte_free(eth_dev); + rte_free(eth_drv); return -1; } -- 1.7.9.5 ^ permalink raw reply [flat|nested] 126+ messages in thread
* Re: [dpdk-dev] [PATCHv4 2/9] null: fix segfault when null_pmd added to bonding 2015-07-15 17:26 ` [dpdk-dev] [PATCHv4 2/9] null: fix segfault when null_pmd added to bonding Tomasz Kulasek @ 2015-09-29 2:24 ` Tetsuya Mukawa 2015-09-29 9:39 ` Kulasek, TomaszX 0 siblings, 1 reply; 126+ messages in thread From: Tetsuya Mukawa @ 2015-09-29 2:24 UTC (permalink / raw) To: Tomasz Kulasek; +Cc: dev On 2015/07/16 2:26, Tomasz Kulasek wrote: > When device is added to the bonding, the link status callback is added to > the slave's eth_dev->link_intr_cbs list. This list is not initialized for > null pmd and adding it to the bonding segfaults application. > > This patch allocates and sets up required structures. > > Signed-off-by: Tomasz Kulasek <tomaszx.kulasek@intel.com> > --- > drivers/net/null/rte_eth_null.c | 11 +++++++++++ > 1 file changed, 11 insertions(+) > > diff --git a/drivers/net/null/rte_eth_null.c b/drivers/net/null/rte_eth_null.c > index e244595..a8b3191 100644 > --- a/drivers/net/null/rte_eth_null.c > +++ b/drivers/net/null/rte_eth_null.c > @@ -386,6 +386,7 @@ eth_dev_null_create(const char *name, > const unsigned nb_rx_queues = 1; > const unsigned nb_tx_queues = 1; > struct rte_eth_dev_data *data = NULL; > + struct eth_driver *eth_drv = NULL; Hi Tomasz, Thanks for extending null pmd features. Is it possible to use rte_null_pmd here? Could you please check ring pmd? It may also uses rte_ring_pmd for link status callback. Tetsuya ^ permalink raw reply [flat|nested] 126+ messages in thread
* Re: [dpdk-dev] [PATCHv4 2/9] null: fix segfault when null_pmd added to bonding 2015-09-29 2:24 ` Tetsuya Mukawa @ 2015-09-29 9:39 ` Kulasek, TomaszX 2015-09-29 10:32 ` Tetsuya Mukawa 0 siblings, 1 reply; 126+ messages in thread From: Kulasek, TomaszX @ 2015-09-29 9:39 UTC (permalink / raw) To: Tetsuya Mukawa; +Cc: dev Hi Tetsuya, > Thanks for extending null pmd features. > Is it possible to use rte_null_pmd here? > Could you please check ring pmd? It may also uses rte_ring_pmd for link > status callback. > > Tetsuya My first attempt was to use ring pmd, and there's no such an issue with it. It works pretty well in bonding. Tomasz. ^ permalink raw reply [flat|nested] 126+ messages in thread
* Re: [dpdk-dev] [PATCHv4 2/9] null: fix segfault when null_pmd added to bonding 2015-09-29 9:39 ` Kulasek, TomaszX @ 2015-09-29 10:32 ` Tetsuya Mukawa 2015-09-29 11:29 ` Kulasek, TomaszX 2015-09-29 15:10 ` Kulasek, TomaszX 0 siblings, 2 replies; 126+ messages in thread From: Tetsuya Mukawa @ 2015-09-29 10:32 UTC (permalink / raw) To: Kulasek, TomaszX; +Cc: dev On 2015/09/29 18:39, Kulasek, TomaszX wrote: > Hi Tetsuya, > >> Thanks for extending null pmd features. >> Is it possible to use rte_null_pmd here? >> Could you please check ring pmd? It may also uses rte_ring_pmd for link >> status callback. >> >> Tetsuya > My first attempt was to use ring pmd, and there's no such an issue with it. It works pretty well in bonding. > > Tomasz. HI Tomasz, Sorry, my English is wrong. 'rte_null_pmd' is defined like below. static struct eth_driver rte_null_pmd = { .pci_drv = { .name = "rte_null_pmd", .drv_flags = RTE_PCI_DRV_DETACHABLE, }, }; I guess you may be able to use 'rte_null_pmd' instead of allocating one more eth_driver structure like below. struct eth_driver *eth_drv = NULL; eth_drv = rte_zmalloc_socket(name, sizeof(*eth_drv), 0, numa_node); Is it possible to use 'rte_null_pmd'? Tetsuya ^ permalink raw reply [flat|nested] 126+ messages in thread
* Re: [dpdk-dev] [PATCHv4 2/9] null: fix segfault when null_pmd added to bonding 2015-09-29 10:32 ` Tetsuya Mukawa @ 2015-09-29 11:29 ` Kulasek, TomaszX 2015-09-29 15:10 ` Kulasek, TomaszX 1 sibling, 0 replies; 126+ messages in thread From: Kulasek, TomaszX @ 2015-09-29 11:29 UTC (permalink / raw) To: Tetsuya Mukawa; +Cc: dev > -----Original Message----- > From: Tetsuya Mukawa [mailto:mukawa@igel.co.jp] > Sent: Tuesday, September 29, 2015 12:33 > To: Kulasek, TomaszX > Cc: dev@dpdk.org > Subject: Re: [dpdk-dev] [PATCHv4 2/9] null: fix segfault when null_pmd > added to bonding > > On 2015/09/29 18:39, Kulasek, TomaszX wrote: > > Hi Tetsuya, > > > >> Thanks for extending null pmd features. > >> Is it possible to use rte_null_pmd here? > >> Could you please check ring pmd? It may also uses rte_ring_pmd for > >> link status callback. > >> > >> Tetsuya > > My first attempt was to use ring pmd, and there's no such an issue > with it. It works pretty well in bonding. > > > > Tomasz. > > HI Tomasz, > > > Sorry, my English is wrong. > 'rte_null_pmd' is defined like below. > > static struct eth_driver rte_null_pmd = { > .pci_drv = { > .name = "rte_null_pmd", > .drv_flags = RTE_PCI_DRV_DETACHABLE, > }, > }; > > I guess you may be able to use 'rte_null_pmd' instead of allocating one > more eth_driver structure like below. > > struct eth_driver *eth_drv = NULL; > eth_drv = rte_zmalloc_socket(name, sizeof(*eth_drv), 0, numa_node); > > Is it possible to use 'rte_null_pmd'? > > Tetsuya Yes, you're right. This malloc can be removed. Tomasz ^ permalink raw reply [flat|nested] 126+ messages in thread
* Re: [dpdk-dev] [PATCHv4 2/9] null: fix segfault when null_pmd added to bonding 2015-09-29 10:32 ` Tetsuya Mukawa 2015-09-29 11:29 ` Kulasek, TomaszX @ 2015-09-29 15:10 ` Kulasek, TomaszX 2015-09-30 1:24 ` Tetsuya Mukawa 1 sibling, 1 reply; 126+ messages in thread From: Kulasek, TomaszX @ 2015-09-29 15:10 UTC (permalink / raw) To: Tetsuya Mukawa; +Cc: dev Hi Tetsuya, Changes from patch below, solves the problem. I've removed malloc. Tomasz. --- >From e03e77a7dc0e47ac9d750545a834624f88f61966 Mon Sep 17 00:00:00 2001 From: Tomasz Kulasek <tomaszx.kulasek@intel.com> Date: Tue, 29 Sep 2015 16:48:31 +0200 Subject: [PATCH 1/4] null: fix segfault when null_pmd added to bonding --- drivers/net/null/rte_eth_null.c | 3 +++ 1 file changed, 3 insertions(+) diff --git a/drivers/net/null/rte_eth_null.c b/drivers/net/null/rte_eth_null.c index e244595..b498ef1 100644 --- a/drivers/net/null/rte_eth_null.c +++ b/drivers/net/null/rte_eth_null.c @@ -432,6 +432,7 @@ eth_dev_null_create(const char *name, internals->numa_node = numa_node; pci_dev->numa_node = numa_node; + pci_dev->driver = &rte_null_pmd.pci_drv; data->dev_private = internals; data->port_id = eth_dev->data->port_id; @@ -445,6 +446,7 @@ eth_dev_null_create(const char *name, eth_dev->dev_ops = &ops; eth_dev->pci_dev = pci_dev; eth_dev->driver = &rte_null_pmd; + TAILQ_INIT(ð_dev->link_intr_cbs); /* finally assign rx and tx ops */ if (packet_copy) { @@ -461,6 +463,7 @@ error: rte_free(data); rte_free(pci_dev); rte_free(internals); + rte_free(eth_dev); return -1; } -- ^ permalink raw reply [flat|nested] 126+ messages in thread
* Re: [dpdk-dev] [PATCHv4 2/9] null: fix segfault when null_pmd added to bonding 2015-09-29 15:10 ` Kulasek, TomaszX @ 2015-09-30 1:24 ` Tetsuya Mukawa 0 siblings, 0 replies; 126+ messages in thread From: Tetsuya Mukawa @ 2015-09-30 1:24 UTC (permalink / raw) To: Kulasek, TomaszX; +Cc: dev On 2015/09/30 0:10, Kulasek, TomaszX wrote: > Hi Tetsuya, > > Changes from patch below, solves the problem. I've removed malloc. > > Tomasz. > > --- > From e03e77a7dc0e47ac9d750545a834624f88f61966 Mon Sep 17 00:00:00 2001 > From: Tomasz Kulasek <tomaszx.kulasek@intel.com> > Date: Tue, 29 Sep 2015 16:48:31 +0200 > Subject: [PATCH 1/4] null: fix segfault when null_pmd added to bonding > > --- > drivers/net/null/rte_eth_null.c | 3 +++ > 1 file changed, 3 insertions(+) > > diff --git a/drivers/net/null/rte_eth_null.c b/drivers/net/null/rte_eth_null.c > index e244595..b498ef1 100644 > --- a/drivers/net/null/rte_eth_null.c > +++ b/drivers/net/null/rte_eth_null.c > @@ -432,6 +432,7 @@ eth_dev_null_create(const char *name, > internals->numa_node = numa_node; > > pci_dev->numa_node = numa_node; > + pci_dev->driver = &rte_null_pmd.pci_drv; > > data->dev_private = internals; > data->port_id = eth_dev->data->port_id; > @@ -445,6 +446,7 @@ eth_dev_null_create(const char *name, > eth_dev->dev_ops = &ops; > eth_dev->pci_dev = pci_dev; > eth_dev->driver = &rte_null_pmd; > + TAILQ_INIT(ð_dev->link_intr_cbs); > > /* finally assign rx and tx ops */ > if (packet_copy) { > @@ -461,6 +463,7 @@ error: > rte_free(data); > rte_free(pci_dev); > rte_free(internals); > + rte_free(eth_dev); Hi Tomasz, We can remove rte_free(eth_dev), because if eth_dev is allocated correctly, we cannot reach here. I don't see any issues except for it. Thanks, Tetsuya ^ permalink raw reply [flat|nested] 126+ messages in thread
* [dpdk-dev] [PATCHv4 3/9] null: extend number of virtual queues 2015-07-15 17:26 ` [dpdk-dev] [PATCHv4 0/9] " Tomasz Kulasek 2015-07-15 17:26 ` [dpdk-dev] [PATCHv4 1/9] bonding: rss dynamic configuration Tomasz Kulasek 2015-07-15 17:26 ` [dpdk-dev] [PATCHv4 2/9] null: fix segfault when null_pmd added to bonding Tomasz Kulasek @ 2015-07-15 17:26 ` Tomasz Kulasek 2015-09-29 2:24 ` Tetsuya Mukawa 2015-07-15 17:26 ` [dpdk-dev] [PATCHv4 4/9] null: virtual dynamic rss configuration Tomasz Kulasek ` (7 subsequent siblings) 10 siblings, 1 reply; 126+ messages in thread From: Tomasz Kulasek @ 2015-07-15 17:26 UTC (permalink / raw) To: dev This patch adds a possibility to configure more than one queue on null device. Signed-off-by: Tomasz Kulasek <tomaszx.kulasek@intel.com> --- drivers/net/null/rte_eth_null.c | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/drivers/net/null/rte_eth_null.c b/drivers/net/null/rte_eth_null.c index a8b3191..39ffcde 100644 --- a/drivers/net/null/rte_eth_null.c +++ b/drivers/net/null/rte_eth_null.c @@ -71,8 +71,8 @@ struct pmd_internals { unsigned nb_rx_queues; unsigned nb_tx_queues; - struct null_queue rx_null_queues[1]; - struct null_queue tx_null_queues[1]; + struct null_queue rx_null_queues[RTE_MAX_QUEUES_PER_PORT]; + struct null_queue tx_null_queues[RTE_MAX_QUEUES_PER_PORT]; }; @@ -213,7 +213,7 @@ eth_rx_queue_setup(struct rte_eth_dev *dev, uint16_t rx_queue_id, if ((dev == NULL) || (mb_pool == NULL)) return -EINVAL; - if (rx_queue_id != 0) + if (rx_queue_id >= dev->data->nb_rx_queues) return -ENODEV; internals = dev->data->dev_private; @@ -246,7 +246,7 @@ eth_tx_queue_setup(struct rte_eth_dev *dev, uint16_t tx_queue_id, if (dev == NULL) return -EINVAL; - if (tx_queue_id != 0) + if (tx_queue_id >= dev->data->nb_tx_queues) return -ENODEV; internals = dev->data->dev_private; @@ -279,8 +279,8 @@ eth_dev_info(struct rte_eth_dev *dev, dev_info->driver_name = drivername; dev_info->max_mac_addrs = 1; dev_info->max_rx_pktlen = (uint32_t)-1; - dev_info->max_rx_queues = (uint16_t)internals->nb_rx_queues; - dev_info->max_tx_queues = (uint16_t)internals->nb_tx_queues; + dev_info->max_rx_queues = RTE_DIM(internals->rx_null_queues); + dev_info->max_tx_queues = RTE_DIM(internals->tx_null_queues); dev_info->min_rx_bufsize = 0; dev_info->pci_dev = NULL; } -- 1.7.9.5 ^ permalink raw reply [flat|nested] 126+ messages in thread
* Re: [dpdk-dev] [PATCHv4 3/9] null: extend number of virtual queues 2015-07-15 17:26 ` [dpdk-dev] [PATCHv4 3/9] null: extend number of virtual queues Tomasz Kulasek @ 2015-09-29 2:24 ` Tetsuya Mukawa 2015-09-29 9:46 ` Kulasek, TomaszX 0 siblings, 1 reply; 126+ messages in thread From: Tetsuya Mukawa @ 2015-09-29 2:24 UTC (permalink / raw) To: Tomasz Kulasek, dev On 2015/07/16 2:26, Tomasz Kulasek wrote: > This patch adds a possibility to configure more than one queue on null > device. > > Signed-off-by: Tomasz Kulasek <tomaszx.kulasek@intel.com> > --- > drivers/net/null/rte_eth_null.c | 12 ++++++------ > 1 file changed, 6 insertions(+), 6 deletions(-) > > diff --git a/drivers/net/null/rte_eth_null.c b/drivers/net/null/rte_eth_null.c > index a8b3191..39ffcde 100644 > --- a/drivers/net/null/rte_eth_null.c > +++ b/drivers/net/null/rte_eth_null.c > @@ -71,8 +71,8 @@ struct pmd_internals { > unsigned nb_rx_queues; > unsigned nb_tx_queues; > > - struct null_queue rx_null_queues[1]; > - struct null_queue tx_null_queues[1]; > + struct null_queue rx_null_queues[RTE_MAX_QUEUES_PER_PORT]; > + struct null_queue tx_null_queues[RTE_MAX_QUEUES_PER_PORT]; > }; > > > @@ -213,7 +213,7 @@ eth_rx_queue_setup(struct rte_eth_dev *dev, uint16_t rx_queue_id, > if ((dev == NULL) || (mb_pool == NULL)) > return -EINVAL; > > - if (rx_queue_id != 0) > + if (rx_queue_id >= dev->data->nb_rx_queues) > return -ENODEV; > > internals = dev->data->dev_private; > @@ -246,7 +246,7 @@ eth_tx_queue_setup(struct rte_eth_dev *dev, uint16_t tx_queue_id, > if (dev == NULL) > return -EINVAL; > > - if (tx_queue_id != 0) > + if (tx_queue_id >= dev->data->nb_tx_queues) > return -ENODEV; > > internals = dev->data->dev_private; > @@ -279,8 +279,8 @@ eth_dev_info(struct rte_eth_dev *dev, > dev_info->driver_name = drivername; > dev_info->max_mac_addrs = 1; > dev_info->max_rx_pktlen = (uint32_t)-1; > - dev_info->max_rx_queues = (uint16_t)internals->nb_rx_queues; > - dev_info->max_tx_queues = (uint16_t)internals->nb_tx_queues; > + dev_info->max_rx_queues = RTE_DIM(internals->rx_null_queues); > + dev_info->max_tx_queues = RTE_DIM(internals->tx_null_queues); > dev_info->min_rx_bufsize = 0; > dev_info->pci_dev = NULL; > } Hi Thomasz, To do like above, should we change below variables also? static int eth_dev_null_create(const char *name, const unsigned numa_node, unsigned packet_size, unsigned packet_copy) { const unsigned nb_rx_queues = 1; const unsigned nb_tx_queues = 1; (snip) } Tetsuya ^ permalink raw reply [flat|nested] 126+ messages in thread
* Re: [dpdk-dev] [PATCHv4 3/9] null: extend number of virtual queues 2015-09-29 2:24 ` Tetsuya Mukawa @ 2015-09-29 9:46 ` Kulasek, TomaszX 2015-09-29 10:34 ` Tetsuya Mukawa 0 siblings, 1 reply; 126+ messages in thread From: Kulasek, TomaszX @ 2015-09-29 9:46 UTC (permalink / raw) To: Tetsuya Mukawa, dev > -----Original Message----- > From: Tetsuya Mukawa [mailto:mukawa@igel.co.jp] > Sent: Tuesday, September 29, 2015 04:25 > To: Kulasek, TomaszX; dev@dpdk.org > Subject: Re: [dpdk-dev] [PATCHv4 3/9] null: extend number of virtual > queues > > On 2015/07/16 2:26, Tomasz Kulasek wrote: > > This patch adds a possibility to configure more than one queue on null > > device. > > > > Signed-off-by: Tomasz Kulasek <tomaszx.kulasek@intel.com> > > --- > > drivers/net/null/rte_eth_null.c | 12 ++++++------ > > 1 file changed, 6 insertions(+), 6 deletions(-) > > > > diff --git a/drivers/net/null/rte_eth_null.c > > b/drivers/net/null/rte_eth_null.c index a8b3191..39ffcde 100644 > > --- a/drivers/net/null/rte_eth_null.c > > +++ b/drivers/net/null/rte_eth_null.c > > @@ -71,8 +71,8 @@ struct pmd_internals { > > unsigned nb_rx_queues; > > unsigned nb_tx_queues; > > > > - struct null_queue rx_null_queues[1]; > > - struct null_queue tx_null_queues[1]; > > + struct null_queue rx_null_queues[RTE_MAX_QUEUES_PER_PORT]; > > + struct null_queue tx_null_queues[RTE_MAX_QUEUES_PER_PORT]; > > }; > > > > > > @@ -213,7 +213,7 @@ eth_rx_queue_setup(struct rte_eth_dev *dev, > uint16_t rx_queue_id, > > if ((dev == NULL) || (mb_pool == NULL)) > > return -EINVAL; > > > > - if (rx_queue_id != 0) > > + if (rx_queue_id >= dev->data->nb_rx_queues) > > return -ENODEV; > > > > internals = dev->data->dev_private; > > @@ -246,7 +246,7 @@ eth_tx_queue_setup(struct rte_eth_dev *dev, > uint16_t tx_queue_id, > > if (dev == NULL) > > return -EINVAL; > > > > - if (tx_queue_id != 0) > > + if (tx_queue_id >= dev->data->nb_tx_queues) > > return -ENODEV; > > > > internals = dev->data->dev_private; > > @@ -279,8 +279,8 @@ eth_dev_info(struct rte_eth_dev *dev, > > dev_info->driver_name = drivername; > > dev_info->max_mac_addrs = 1; > > dev_info->max_rx_pktlen = (uint32_t)-1; > > - dev_info->max_rx_queues = (uint16_t)internals->nb_rx_queues; > > - dev_info->max_tx_queues = (uint16_t)internals->nb_tx_queues; > > + dev_info->max_rx_queues = RTE_DIM(internals->rx_null_queues); > > + dev_info->max_tx_queues = RTE_DIM(internals->tx_null_queues); > > dev_info->min_rx_bufsize = 0; > > dev_info->pci_dev = NULL; > > } > > Hi Thomasz, > > To do like above, should we change below variables also? > > static int > eth_dev_null_create(const char *name, > const unsigned numa_node, > unsigned packet_size, > unsigned packet_copy) > { > const unsigned nb_rx_queues = 1; > const unsigned nb_tx_queues = 1; > > (snip) > > } > > Tetsuya Hi Tetsuya, The number of requested queues are configured later, through the rte eth_dev_configure API. Here these values mean the default number of queues after null pmd creation. Tomasz. ^ permalink raw reply [flat|nested] 126+ messages in thread
* Re: [dpdk-dev] [PATCHv4 3/9] null: extend number of virtual queues 2015-09-29 9:46 ` Kulasek, TomaszX @ 2015-09-29 10:34 ` Tetsuya Mukawa 2015-09-29 11:56 ` Kulasek, TomaszX 0 siblings, 1 reply; 126+ messages in thread From: Tetsuya Mukawa @ 2015-09-29 10:34 UTC (permalink / raw) To: Kulasek, TomaszX, dev On 2015/09/29 18:46, Kulasek, TomaszX wrote: >> -----Original Message----- >> From: Tetsuya Mukawa [mailto:mukawa@igel.co.jp] >> Sent: Tuesday, September 29, 2015 04:25 >> To: Kulasek, TomaszX; dev@dpdk.org >> Subject: Re: [dpdk-dev] [PATCHv4 3/9] null: extend number of virtual >> queues >> >> On 2015/07/16 2:26, Tomasz Kulasek wrote: >>> This patch adds a possibility to configure more than one queue on null >>> device. >>> >>> Signed-off-by: Tomasz Kulasek <tomaszx.kulasek@intel.com> >>> --- >>> drivers/net/null/rte_eth_null.c | 12 ++++++------ >>> 1 file changed, 6 insertions(+), 6 deletions(-) >>> >>> diff --git a/drivers/net/null/rte_eth_null.c >>> b/drivers/net/null/rte_eth_null.c index a8b3191..39ffcde 100644 >>> --- a/drivers/net/null/rte_eth_null.c >>> +++ b/drivers/net/null/rte_eth_null.c >>> @@ -71,8 +71,8 @@ struct pmd_internals { >>> unsigned nb_rx_queues; >>> unsigned nb_tx_queues; >>> >>> - struct null_queue rx_null_queues[1]; >>> - struct null_queue tx_null_queues[1]; >>> + struct null_queue rx_null_queues[RTE_MAX_QUEUES_PER_PORT]; >>> + struct null_queue tx_null_queues[RTE_MAX_QUEUES_PER_PORT]; >>> }; >>> >>> >>> @@ -213,7 +213,7 @@ eth_rx_queue_setup(struct rte_eth_dev *dev, >> uint16_t rx_queue_id, >>> if ((dev == NULL) || (mb_pool == NULL)) >>> return -EINVAL; >>> >>> - if (rx_queue_id != 0) >>> + if (rx_queue_id >= dev->data->nb_rx_queues) >>> return -ENODEV; >>> >>> internals = dev->data->dev_private; >>> @@ -246,7 +246,7 @@ eth_tx_queue_setup(struct rte_eth_dev *dev, >> uint16_t tx_queue_id, >>> if (dev == NULL) >>> return -EINVAL; >>> >>> - if (tx_queue_id != 0) >>> + if (tx_queue_id >= dev->data->nb_tx_queues) >>> return -ENODEV; >>> >>> internals = dev->data->dev_private; >>> @@ -279,8 +279,8 @@ eth_dev_info(struct rte_eth_dev *dev, >>> dev_info->driver_name = drivername; >>> dev_info->max_mac_addrs = 1; >>> dev_info->max_rx_pktlen = (uint32_t)-1; >>> - dev_info->max_rx_queues = (uint16_t)internals->nb_rx_queues; >>> - dev_info->max_tx_queues = (uint16_t)internals->nb_tx_queues; >>> + dev_info->max_rx_queues = RTE_DIM(internals->rx_null_queues); >>> + dev_info->max_tx_queues = RTE_DIM(internals->tx_null_queues); >>> dev_info->min_rx_bufsize = 0; >>> dev_info->pci_dev = NULL; >>> } >> Hi Thomasz, >> >> To do like above, should we change below variables also? >> >> static int >> eth_dev_null_create(const char *name, >> const unsigned numa_node, >> unsigned packet_size, >> unsigned packet_copy) >> { >> const unsigned nb_rx_queues = 1; >> const unsigned nb_tx_queues = 1; >> >> (snip) >> >> } >> >> Tetsuya > Hi Tetsuya, > > The number of requested queues are configured later, through the rte eth_dev_configure API. Here these values mean the default number of queues after null pmd creation. > > Tomasz. > Hi Tomasz, I guess we may need to change below variables when a queue is configured. - internals->nb_rx_queues; - internals->nb_tx_queues; Without changing, 'num_stats' in eth_stats_get() might be wrong value. Tetsuya ^ permalink raw reply [flat|nested] 126+ messages in thread
* Re: [dpdk-dev] [PATCHv4 3/9] null: extend number of virtual queues 2015-09-29 10:34 ` Tetsuya Mukawa @ 2015-09-29 11:56 ` Kulasek, TomaszX 2015-09-29 15:06 ` Kulasek, TomaszX 0 siblings, 1 reply; 126+ messages in thread From: Kulasek, TomaszX @ 2015-09-29 11:56 UTC (permalink / raw) To: Tetsuya Mukawa, dev > > Hi Tomasz, > > I guess we may need to change below variables when a queue is > configured. > - internals->nb_rx_queues; > - internals->nb_tx_queues; > Without changing, 'num_stats' in eth_stats_get() might be wrong value. > > Tetsuya Hi Tetsuya, I see it now. I was too focused on one case and I missed it. I'll modify it. Thanks for your help, by the way. Tomasz ^ permalink raw reply [flat|nested] 126+ messages in thread
* Re: [dpdk-dev] [PATCHv4 3/9] null: extend number of virtual queues 2015-09-29 11:56 ` Kulasek, TomaszX @ 2015-09-29 15:06 ` Kulasek, TomaszX 2015-09-30 1:24 ` Tetsuya Mukawa 0 siblings, 1 reply; 126+ messages in thread From: Kulasek, TomaszX @ 2015-09-29 15:06 UTC (permalink / raw) To: Kulasek, TomaszX, Tetsuya Mukawa, dev Hi Tetsuya, Can you check patch below? Thanks, Tomasz. --- From d50b0594fa34a576602c457b51cac80e7d462eed Mon Sep 17 00:00:00 2001 From: Tomasz Kulasek <tomaszx.kulasek@intel.com> Date: Tue, 29 Sep 2015 16:52:12 +0200 Subject: [PATCH 2/4] null: extend number of virtual queues --- drivers/net/null/rte_eth_null.c | 28 +++++++++++++++++++--------- 1 file changed, 19 insertions(+), 9 deletions(-) diff --git a/drivers/net/null/rte_eth_null.c b/drivers/net/null/rte_eth_null.c index b498ef1..68cb723 100644 --- a/drivers/net/null/rte_eth_null.c +++ b/drivers/net/null/rte_eth_null.c @@ -71,8 +71,8 @@ struct pmd_internals { unsigned nb_rx_queues; unsigned nb_tx_queues; - struct null_queue rx_null_queues[1]; - struct null_queue tx_null_queues[1]; + struct null_queue rx_null_queues[RTE_MAX_QUEUES_PER_PORT]; + struct null_queue tx_null_queues[RTE_MAX_QUEUES_PER_PORT]; }; @@ -178,7 +178,15 @@ eth_null_copy_tx(void *q, struct rte_mbuf **bufs, uint16_t nb_bufs) } static int -eth_dev_configure(struct rte_eth_dev *dev __rte_unused) { return 0; } +eth_dev_configure(struct rte_eth_dev *dev) { + struct pmd_internals *internals; + + internals = dev->data->dev_private; + internals->nb_rx_queues = dev->data->nb_rx_queues; + internals->nb_tx_queues = dev->data->nb_tx_queues; + + return 0; +} static int eth_dev_start(struct rte_eth_dev *dev) @@ -213,10 +221,11 @@ eth_rx_queue_setup(struct rte_eth_dev *dev, uint16_t rx_queue_id, if ((dev == NULL) || (mb_pool == NULL)) return -EINVAL; - if (rx_queue_id != 0) + internals = dev->data->dev_private; + + if (rx_queue_id >= internals->nb_rx_queues) return -ENODEV; - internals = dev->data->dev_private; packet_size = internals->packet_size; internals->rx_null_queues[rx_queue_id].mb_pool = mb_pool; @@ -246,10 +255,11 @@ eth_tx_queue_setup(struct rte_eth_dev *dev, uint16_t tx_queue_id, if (dev == NULL) return -EINVAL; - if (tx_queue_id != 0) + internals = dev->data->dev_private; + + if (tx_queue_id >= internals->nb_tx_queues) return -ENODEV; - internals = dev->data->dev_private; packet_size = internals->packet_size; dev->data->tx_queues[tx_queue_id] = @@ -279,8 +289,8 @@ eth_dev_info(struct rte_eth_dev *dev, dev_info->driver_name = drivername; dev_info->max_mac_addrs = 1; dev_info->max_rx_pktlen = (uint32_t)-1; - dev_info->max_rx_queues = (uint16_t)internals->nb_rx_queues; - dev_info->max_tx_queues = (uint16_t)internals->nb_tx_queues; + dev_info->max_rx_queues = RTE_DIM(internals->rx_null_queues); + dev_info->max_tx_queues = RTE_DIM(internals->tx_null_queues); dev_info->min_rx_bufsize = 0; dev_info->pci_dev = NULL; } -- ^ permalink raw reply [flat|nested] 126+ messages in thread
* Re: [dpdk-dev] [PATCHv4 3/9] null: extend number of virtual queues 2015-09-29 15:06 ` Kulasek, TomaszX @ 2015-09-30 1:24 ` Tetsuya Mukawa 0 siblings, 0 replies; 126+ messages in thread From: Tetsuya Mukawa @ 2015-09-30 1:24 UTC (permalink / raw) To: Kulasek, TomaszX, dev On 2015/09/30 0:06, Kulasek, TomaszX wrote: > Hi Tetsuya, > > Can you check patch below? > > Thanks, > Tomasz. > --- > > From d50b0594fa34a576602c457b51cac80e7d462eed Mon Sep 17 00:00:00 2001 > From: Tomasz Kulasek <tomaszx.kulasek@intel.com> > Date: Tue, 29 Sep 2015 16:52:12 +0200 > Subject: [PATCH 2/4] null: extend number of virtual queues > > --- > drivers/net/null/rte_eth_null.c | 28 +++++++++++++++++++--------- > 1 file changed, 19 insertions(+), 9 deletions(-) > > diff --git a/drivers/net/null/rte_eth_null.c b/drivers/net/null/rte_eth_null.c > index b498ef1..68cb723 100644 > --- a/drivers/net/null/rte_eth_null.c > +++ b/drivers/net/null/rte_eth_null.c > @@ -71,8 +71,8 @@ struct pmd_internals { > unsigned nb_rx_queues; > unsigned nb_tx_queues; > > - struct null_queue rx_null_queues[1]; > - struct null_queue tx_null_queues[1]; > + struct null_queue rx_null_queues[RTE_MAX_QUEUES_PER_PORT]; > + struct null_queue tx_null_queues[RTE_MAX_QUEUES_PER_PORT]; > }; > > > @@ -178,7 +178,15 @@ eth_null_copy_tx(void *q, struct rte_mbuf **bufs, uint16_t nb_bufs) > } > > static int > -eth_dev_configure(struct rte_eth_dev *dev __rte_unused) { return 0; } > +eth_dev_configure(struct rte_eth_dev *dev) { > + struct pmd_internals *internals; > + > + internals = dev->data->dev_private; > + internals->nb_rx_queues = dev->data->nb_rx_queues; > + internals->nb_tx_queues = dev->data->nb_tx_queues; > + > + return 0; > +} > > static int > eth_dev_start(struct rte_eth_dev *dev) > @@ -213,10 +221,11 @@ eth_rx_queue_setup(struct rte_eth_dev *dev, uint16_t rx_queue_id, > if ((dev == NULL) || (mb_pool == NULL)) > return -EINVAL; > > - if (rx_queue_id != 0) > + internals = dev->data->dev_private; > + > + if (rx_queue_id >= internals->nb_rx_queues) > return -ENODEV; > > - internals = dev->data->dev_private; > packet_size = internals->packet_size; > > internals->rx_null_queues[rx_queue_id].mb_pool = mb_pool; > @@ -246,10 +255,11 @@ eth_tx_queue_setup(struct rte_eth_dev *dev, uint16_t tx_queue_id, > if (dev == NULL) > return -EINVAL; > > - if (tx_queue_id != 0) > + internals = dev->data->dev_private; > + > + if (tx_queue_id >= internals->nb_tx_queues) > return -ENODEV; > > - internals = dev->data->dev_private; > packet_size = internals->packet_size; > > dev->data->tx_queues[tx_queue_id] = > @@ -279,8 +289,8 @@ eth_dev_info(struct rte_eth_dev *dev, > dev_info->driver_name = drivername; > dev_info->max_mac_addrs = 1; > dev_info->max_rx_pktlen = (uint32_t)-1; > - dev_info->max_rx_queues = (uint16_t)internals->nb_rx_queues; > - dev_info->max_tx_queues = (uint16_t)internals->nb_tx_queues; > + dev_info->max_rx_queues = RTE_DIM(internals->rx_null_queues); > + dev_info->max_tx_queues = RTE_DIM(internals->tx_null_queues); > dev_info->min_rx_bufsize = 0; > dev_info->pci_dev = NULL; > } > -- > Acked-by: Tetsuya Mukawa <mukawa@igel.co.jp> ^ permalink raw reply [flat|nested] 126+ messages in thread
* [dpdk-dev] [PATCHv4 4/9] null: virtual dynamic rss configuration 2015-07-15 17:26 ` [dpdk-dev] [PATCHv4 0/9] " Tomasz Kulasek ` (2 preceding siblings ...) 2015-07-15 17:26 ` [dpdk-dev] [PATCHv4 3/9] null: extend number of virtual queues Tomasz Kulasek @ 2015-07-15 17:26 ` Tomasz Kulasek 2015-09-29 2:24 ` Tetsuya Mukawa 2015-07-15 17:26 ` [dpdk-dev] [PATCHv4 5/9] null: export eth_dev_null_create Tomasz Kulasek ` (6 subsequent siblings) 10 siblings, 1 reply; 126+ messages in thread From: Tomasz Kulasek @ 2015-07-15 17:26 UTC (permalink / raw) To: dev This implementation allows to set and read RSS configuration for null device, and is used to validate right values propagation over the slaves, in test units for dynamic RSS configuration for bonding. Signed-off-by: Tomasz Kulasek <tomaszx.kulasek@intel.com> --- drivers/net/null/rte_eth_null.c | 116 +++++++++++++++++++++++++++++++++++++++ 1 file changed, 116 insertions(+) diff --git a/drivers/net/null/rte_eth_null.c b/drivers/net/null/rte_eth_null.c index 39ffcde..f393422 100644 --- a/drivers/net/null/rte_eth_null.c +++ b/drivers/net/null/rte_eth_null.c @@ -37,6 +37,8 @@ #include <rte_memcpy.h> #include <rte_dev.h> #include <rte_kvargs.h> +#include <rte_eth_null.h> +#include <rte_spinlock.h> #define ETH_NULL_PACKET_SIZE_ARG "size" #define ETH_NULL_PACKET_COPY_ARG "copy" @@ -73,6 +75,17 @@ struct pmd_internals { struct null_queue rx_null_queues[RTE_MAX_QUEUES_PER_PORT]; struct null_queue tx_null_queues[RTE_MAX_QUEUES_PER_PORT]; + + /** Bit mask of RSS offloads, the bit offset also means flow type */ + uint64_t flow_type_rss_offloads; + + rte_spinlock_t rss_lock; + + uint16_t reta_size; + struct rte_eth_rss_reta_entry64 reta_conf[ETH_RSS_RETA_SIZE_128 / + RTE_RETA_GROUP_SIZE]; + + uint8_t rss_key[40]; /**< 40-byte hash key. */ }; @@ -283,6 +296,8 @@ eth_dev_info(struct rte_eth_dev *dev, dev_info->max_tx_queues = RTE_DIM(internals->tx_null_queues); dev_info->min_rx_bufsize = 0; dev_info->pci_dev = NULL; + dev_info->reta_size = internals->reta_size; + dev_info->flow_type_rss_offloads = internals->flow_type_rss_offloads; } static void @@ -363,6 +378,91 @@ static int eth_link_update(struct rte_eth_dev *dev __rte_unused, int wait_to_complete __rte_unused) { return 0; } +static int +eth_rss_reta_update(struct rte_eth_dev *dev, + struct rte_eth_rss_reta_entry64 *reta_conf, uint16_t reta_size) +{ + int i, j; + struct pmd_internals *internal = dev->data->dev_private; + + if (reta_size != internal->reta_size) + return -EINVAL; + + rte_spinlock_lock(&internal->rss_lock); + + /* Copy RETA table */ + for (i = 0; i < (internal->reta_size / RTE_RETA_GROUP_SIZE); i++) { + internal->reta_conf[i].mask = reta_conf[i].mask; + for (j = 0; j < RTE_RETA_GROUP_SIZE; j++) + if ((reta_conf[i].mask >> j) & 0x01) + internal->reta_conf[i].reta[j] = reta_conf[i].reta[j]; + } + + rte_spinlock_unlock(&internal->rss_lock); + + return 0; +} + +static int +eth_rss_reta_query(struct rte_eth_dev *dev, + struct rte_eth_rss_reta_entry64 *reta_conf, uint16_t reta_size) +{ + int i, j; + struct pmd_internals *internal = dev->data->dev_private; + + if (reta_size != internal->reta_size) + return -EINVAL; + + rte_spinlock_lock(&internal->rss_lock); + + /* Copy RETA table */ + for (i = 0; i < (internal->reta_size / RTE_RETA_GROUP_SIZE); i++) { + for (j = 0; j < RTE_RETA_GROUP_SIZE; j++) + if ((reta_conf[i].mask >> j) & 0x01) + reta_conf[i].reta[j] = internal->reta_conf[i].reta[j]; + } + + rte_spinlock_unlock(&internal->rss_lock); + + return 0; +} + +static int +eth_rss_hash_update(struct rte_eth_dev *dev, struct rte_eth_rss_conf *rss_conf) +{ + struct pmd_internals *internal = dev->data->dev_private; + + rte_spinlock_lock(&internal->rss_lock); + + if ((rss_conf->rss_hf & internal->flow_type_rss_offloads) != 0) + dev->data->dev_conf.rx_adv_conf.rss_conf.rss_hf = + rss_conf->rss_hf & internal->flow_type_rss_offloads; + + if (rss_conf->rss_key) + memcpy(internal->rss_key, rss_conf->rss_key, 40); + + rte_spinlock_unlock(&internal->rss_lock); + + return 0; +} + +static int +eth_rss_hash_conf_get(struct rte_eth_dev *dev, + struct rte_eth_rss_conf *rss_conf) +{ + struct pmd_internals *internal = dev->data->dev_private; + + rte_spinlock_lock(&internal->rss_lock); + + rss_conf->rss_hf = dev->data->dev_conf.rx_adv_conf.rss_conf.rss_hf; + if (rss_conf->rss_key) + memcpy(rss_conf->rss_key, internal->rss_key, 40); + + rte_spinlock_unlock(&internal->rss_lock); + + return 0; +} + static const struct eth_dev_ops ops = { .dev_start = eth_dev_start, .dev_stop = eth_dev_stop, @@ -375,6 +475,10 @@ static const struct eth_dev_ops ops = { .link_update = eth_link_update, .stats_get = eth_stats_get, .stats_reset = eth_stats_reset, + .reta_update = eth_rss_reta_update, + .reta_query = eth_rss_reta_query, + .rss_hash_update = eth_rss_hash_update, + .rss_hash_conf_get = eth_rss_hash_conf_get }; static int @@ -391,6 +495,13 @@ eth_dev_null_create(const char *name, struct pmd_internals *internals = NULL; struct rte_eth_dev *eth_dev = NULL; + static const uint8_t default_rss_key[40] = { + 0x6D, 0x5A, 0x56, 0xDA, 0x25, 0x5B, 0x0E, 0xC2, 0x41, 0x67, 0x25, 0x3D, + 0x43, 0xA3, 0x8F, 0xB0, 0xD0, 0xCA, 0x2B, 0xCB, 0xAE, 0x7B, 0x30, 0xB4, + 0x77, 0xCB, 0x2D, 0xA3, 0x80, 0x30, 0xF2, 0x0C, 0x6A, 0x42, 0xB7, 0x3B, + 0xBE, 0xAC, 0x01, 0xFA + }; + if (name == NULL) return -EINVAL; @@ -436,6 +547,11 @@ eth_dev_null_create(const char *name, internals->packet_copy = packet_copy; internals->numa_node = numa_node; + internals->flow_type_rss_offloads = ETH_RSS_PROTO_MASK; + internals->reta_size = RTE_DIM(internals->reta_conf) * RTE_RETA_GROUP_SIZE; + + memcpy(internals->rss_key, default_rss_key, 40); + eth_drv->pci_drv.name = drivername; pci_dev->numa_node = numa_node; -- 1.7.9.5 ^ permalink raw reply [flat|nested] 126+ messages in thread
* Re: [dpdk-dev] [PATCHv4 4/9] null: virtual dynamic rss configuration 2015-07-15 17:26 ` [dpdk-dev] [PATCHv4 4/9] null: virtual dynamic rss configuration Tomasz Kulasek @ 2015-09-29 2:24 ` Tetsuya Mukawa 2015-09-29 10:04 ` Kulasek, TomaszX 2015-10-12 9:05 ` Jastrzebski, MichalX K 0 siblings, 2 replies; 126+ messages in thread From: Tetsuya Mukawa @ 2015-09-29 2:24 UTC (permalink / raw) To: Tomasz Kulasek, dev On 2015/07/16 2:26, Tomasz Kulasek wrote: > This implementation allows to set and read RSS configuration for null > device, and is used to validate right values propagation over the slaves, > in test units for dynamic RSS configuration for bonding. > > Signed-off-by: Tomasz Kulasek <tomaszx.kulasek@intel.com> > --- > drivers/net/null/rte_eth_null.c | 116 +++++++++++++++++++++++++++++++++++++++ > 1 file changed, 116 insertions(+) > > diff --git a/drivers/net/null/rte_eth_null.c b/drivers/net/null/rte_eth_null.c > index 39ffcde..f393422 100644 > --- a/drivers/net/null/rte_eth_null.c > +++ b/drivers/net/null/rte_eth_null.c > +static int > +eth_rss_hash_update(struct rte_eth_dev *dev, struct rte_eth_rss_conf *rss_conf) > +{ > + struct pmd_internals *internal = dev->data->dev_private; > + > + rte_spinlock_lock(&internal->rss_lock); > + > + if ((rss_conf->rss_hf & internal->flow_type_rss_offloads) != 0) > + dev->data->dev_conf.rx_adv_conf.rss_conf.rss_hf = > + rss_conf->rss_hf & internal->flow_type_rss_offloads; > + > + if (rss_conf->rss_key) > + memcpy(internal->rss_key, rss_conf->rss_key, 40); > + > + rte_spinlock_unlock(&internal->rss_lock); > + > + return 0; > +} > + > +static int > +eth_rss_hash_conf_get(struct rte_eth_dev *dev, > + struct rte_eth_rss_conf *rss_conf) > +{ > + struct pmd_internals *internal = dev->data->dev_private; > + > + rte_spinlock_lock(&internal->rss_lock); > + > + rss_conf->rss_hf = dev->data->dev_conf.rx_adv_conf.rss_conf.rss_hf; > + if (rss_conf->rss_key) > + memcpy(rss_conf->rss_key, internal->rss_key, 40); > + > + rte_spinlock_unlock(&internal->rss_lock); > + > + return 0; > +} > + > static const struct eth_dev_ops ops = { > .dev_start = eth_dev_start, > .dev_stop = eth_dev_stop, > @@ -436,6 +547,11 @@ eth_dev_null_create(const char *name, > internals->packet_copy = packet_copy; > internals->numa_node = numa_node; > > + internals->flow_type_rss_offloads = ETH_RSS_PROTO_MASK; > + internals->reta_size = RTE_DIM(internals->reta_conf) * RTE_RETA_GROUP_SIZE; > + > + memcpy(internals->rss_key, default_rss_key, 40); > + > eth_drv->pci_drv.name = drivername; > > pci_dev->numa_node = numa_node; Hi Thomasz, I am just curious. Is it possible to use rte_memcpy instead of memcpy? if we can, rte_memcpy may be faster. Tetsuya ^ permalink raw reply [flat|nested] 126+ messages in thread
* Re: [dpdk-dev] [PATCHv4 4/9] null: virtual dynamic rss configuration 2015-09-29 2:24 ` Tetsuya Mukawa @ 2015-09-29 10:04 ` Kulasek, TomaszX 2015-10-12 9:05 ` Jastrzebski, MichalX K 1 sibling, 0 replies; 126+ messages in thread From: Kulasek, TomaszX @ 2015-09-29 10:04 UTC (permalink / raw) To: Tetsuya Mukawa, dev > Hi Thomasz, > > I am just curious. Is it possible to use rte_memcpy instead of memcpy? > if we can, rte_memcpy may be faster. > > Tetsuya Hi Tetsuya, I've just tested it and seems to work, so there's no objections to use rte_memcpy. Tomasz. ^ permalink raw reply [flat|nested] 126+ messages in thread
* Re: [dpdk-dev] [PATCHv4 4/9] null: virtual dynamic rss configuration 2015-09-29 2:24 ` Tetsuya Mukawa 2015-09-29 10:04 ` Kulasek, TomaszX @ 2015-10-12 9:05 ` Jastrzebski, MichalX K 1 sibling, 0 replies; 126+ messages in thread From: Jastrzebski, MichalX K @ 2015-10-12 9:05 UTC (permalink / raw) To: Tetsuya Mukawa, Kulasek, TomaszX, dev > -----Original Message----- > From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Tetsuya Mukawa > Sent: Tuesday, September 29, 2015 4:25 AM > To: Kulasek, TomaszX; dev@dpdk.org > Subject: Re: [dpdk-dev] [PATCHv4 4/9] null: virtual dynamic rss configuration > > On 2015/07/16 2:26, Tomasz Kulasek wrote: > > This implementation allows to set and read RSS configuration for null > > device, and is used to validate right values propagation over the slaves, > > in test units for dynamic RSS configuration for bonding. > > > > Signed-off-by: Tomasz Kulasek <tomaszx.kulasek@intel.com> > > --- > > drivers/net/null/rte_eth_null.c | 116 > +++++++++++++++++++++++++++++++++++++++ > > 1 file changed, 116 insertions(+) > > > > diff --git a/drivers/net/null/rte_eth_null.c b/drivers/net/null/rte_eth_null.c > > index 39ffcde..f393422 100644 > > --- a/drivers/net/null/rte_eth_null.c > > +++ b/drivers/net/null/rte_eth_null.c > > +static int > > +eth_rss_hash_update(struct rte_eth_dev *dev, struct rte_eth_rss_conf > *rss_conf) > > +{ > > + struct pmd_internals *internal = dev->data->dev_private; > > + > > + rte_spinlock_lock(&internal->rss_lock); > > + > > + if ((rss_conf->rss_hf & internal->flow_type_rss_offloads) != 0) > > + dev->data->dev_conf.rx_adv_conf.rss_conf.rss_hf = > > + rss_conf->rss_hf & internal- > >flow_type_rss_offloads; > > + > > + if (rss_conf->rss_key) > > + memcpy(internal->rss_key, rss_conf->rss_key, 40); > > + > > + rte_spinlock_unlock(&internal->rss_lock); > > + > > + return 0; > > +} > > + > > +static int > > +eth_rss_hash_conf_get(struct rte_eth_dev *dev, > > + struct rte_eth_rss_conf *rss_conf) > > +{ > > + struct pmd_internals *internal = dev->data->dev_private; > > + > > + rte_spinlock_lock(&internal->rss_lock); > > + > > + rss_conf->rss_hf = dev->data- > >dev_conf.rx_adv_conf.rss_conf.rss_hf; > > + if (rss_conf->rss_key) > > + memcpy(rss_conf->rss_key, internal->rss_key, 40); > > + > > + rte_spinlock_unlock(&internal->rss_lock); > > + > > + return 0; > > +} > > + > > static const struct eth_dev_ops ops = { > > .dev_start = eth_dev_start, > > .dev_stop = eth_dev_stop, > > @@ -436,6 +547,11 @@ eth_dev_null_create(const char *name, > > internals->packet_copy = packet_copy; > > internals->numa_node = numa_node; > > > > + internals->flow_type_rss_offloads = ETH_RSS_PROTO_MASK; > > + internals->reta_size = RTE_DIM(internals->reta_conf) * > RTE_RETA_GROUP_SIZE; > > + > > + memcpy(internals->rss_key, default_rss_key, 40); > > + > > eth_drv->pci_drv.name = drivername; > > > > pci_dev->numa_node = numa_node; > > Hi Thomasz, > > I am just curious. Is it possible to use rte_memcpy instead of memcpy? > if we can, rte_memcpy may be faster. > > Tetsuya Hi Tetsuya, Could You please review v5 that Tomasz sent and if You agree with this implementation could You also ACK this patch-set? ^ permalink raw reply [flat|nested] 126+ messages in thread
* [dpdk-dev] [PATCHv4 5/9] null: export eth_dev_null_create 2015-07-15 17:26 ` [dpdk-dev] [PATCHv4 0/9] " Tomasz Kulasek ` (3 preceding siblings ...) 2015-07-15 17:26 ` [dpdk-dev] [PATCHv4 4/9] null: virtual dynamic rss configuration Tomasz Kulasek @ 2015-07-15 17:26 ` Tomasz Kulasek 2015-09-25 10:11 ` Thomas Monjalon 2015-09-29 2:28 ` Tetsuya Mukawa 2015-07-15 17:26 ` [dpdk-dev] [PATCHv4 6/9] test: dynamic rss configuration Tomasz Kulasek ` (5 subsequent siblings) 10 siblings, 2 replies; 126+ messages in thread From: Tomasz Kulasek @ 2015-07-15 17:26 UTC (permalink / raw) To: dev To use eth_dev_null_create in application this method needs to be exported. Signed-off-by: Tomasz Kulasek <tomaszx.kulasek@intel.com> --- drivers/net/null/Makefile | 2 +- drivers/net/null/rte_eth_null.c | 3 ++- drivers/net/null/rte_eth_null.h | 40 +++++++++++++++++++++++++++++ drivers/net/null/rte_pmd_null_version.map | 7 +++++ 4 files changed, 50 insertions(+), 2 deletions(-) create mode 100644 drivers/net/null/rte_eth_null.h diff --git a/drivers/net/null/Makefile b/drivers/net/null/Makefile index 6472015..b33f9fd 100644 --- a/drivers/net/null/Makefile +++ b/drivers/net/null/Makefile @@ -51,7 +51,7 @@ SRCS-$(CONFIG_RTE_LIBRTE_PMD_NULL) += rte_eth_null.c # # Export include files # -SYMLINK-y-include += +SYMLINK-y-include += rte_eth_null.h # this lib depends upon: DEPDIRS-$(CONFIG_RTE_LIBRTE_PMD_NULL) += lib/librte_mbuf diff --git a/drivers/net/null/rte_eth_null.c b/drivers/net/null/rte_eth_null.c index f393422..8ae6ebf 100644 --- a/drivers/net/null/rte_eth_null.c +++ b/drivers/net/null/rte_eth_null.c @@ -2,6 +2,7 @@ * BSD LICENSE * * Copyright (C) IGEL Co.,Ltd. + * Copyright (c) 2015 Intel Corporation. * All rights reserved. * * Redistribution and use in source and binary forms, with or without @@ -481,7 +482,7 @@ static const struct eth_dev_ops ops = { .rss_hash_conf_get = eth_rss_hash_conf_get }; -static int +int eth_dev_null_create(const char *name, const unsigned numa_node, unsigned packet_size, diff --git a/drivers/net/null/rte_eth_null.h b/drivers/net/null/rte_eth_null.h new file mode 100644 index 0000000..abada8c --- /dev/null +++ b/drivers/net/null/rte_eth_null.h @@ -0,0 +1,40 @@ +/*- + * BSD LICENSE + * + * Copyright(c) 2015 Intel Corporation. All rights reserved. + * All rights reserved. + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * + * * Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * * Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in + * the documentation and/or other materials provided with the + * distribution. + * * Neither the name of Intel Corporation nor the names of its + * contributors may be used to endorse or promote products derived + * from this software without specific prior written permission. + * + * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS + * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT + * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR + * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT + * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, + * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT + * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, + * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY + * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT + * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE + * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + */ + +#ifndef RTE_ETH_NULL_H_ +#define RTE_ETH_NULL_H_ + +int eth_dev_null_create(const char *name, const unsigned numa_node, + unsigned packet_size, unsigned packet_copy); + +#endif /* RTE_ETH_NULL_H_ */ diff --git a/drivers/net/null/rte_pmd_null_version.map b/drivers/net/null/rte_pmd_null_version.map index ef35398..2b2a743 100644 --- a/drivers/net/null/rte_pmd_null_version.map +++ b/drivers/net/null/rte_pmd_null_version.map @@ -2,3 +2,10 @@ DPDK_2.0 { local: *; }; + +DPDK_2.1 { + global: + + eth_dev_null_create; + +} DPDK_2.0; -- 1.7.9.5 ^ permalink raw reply [flat|nested] 126+ messages in thread
* Re: [dpdk-dev] [PATCHv4 5/9] null: export eth_dev_null_create 2015-07-15 17:26 ` [dpdk-dev] [PATCHv4 5/9] null: export eth_dev_null_create Tomasz Kulasek @ 2015-09-25 10:11 ` Thomas Monjalon 2015-09-25 13:24 ` Kulasek, TomaszX 2015-09-29 2:28 ` Tetsuya Mukawa 1 sibling, 1 reply; 126+ messages in thread From: Thomas Monjalon @ 2015-09-25 10:11 UTC (permalink / raw) To: Tomasz Kulasek; +Cc: dev Hi Tomasz, 2015-07-15 19:26, Tomasz Kulasek: > --- a/drivers/net/null/rte_eth_null.c > +++ b/drivers/net/null/rte_eth_null.c > @@ -2,6 +2,7 @@ > * BSD LICENSE > * > * Copyright (C) IGEL Co.,Ltd. > + * Copyright (c) 2015 Intel Corporation. > * All rights reserved. > * > * Redistribution and use in source and binary forms, with or without > @@ -481,7 +482,7 @@ static const struct eth_dev_ops ops = { > .rss_hash_conf_get = eth_rss_hash_conf_get > }; > > -static int > +int > eth_dev_null_create(const char *name, > const unsigned numa_node, > unsigned packet_size, There is no rule to update a copyright but I think it is not justified here. > --- a/drivers/net/null/rte_pmd_null_version.map > +++ b/drivers/net/null/rte_pmd_null_version.map > @@ -2,3 +2,10 @@ DPDK_2.0 { > > local: *; > }; > + > +DPDK_2.1 { > + global: > + > + eth_dev_null_create; > + > +} DPDK_2.0; Please, could you update it to DPDK_2.2? Thanks ^ permalink raw reply [flat|nested] 126+ messages in thread
* Re: [dpdk-dev] [PATCHv4 5/9] null: export eth_dev_null_create 2015-09-25 10:11 ` Thomas Monjalon @ 2015-09-25 13:24 ` Kulasek, TomaszX 2015-09-25 13:43 ` Thomas Monjalon 0 siblings, 1 reply; 126+ messages in thread From: Kulasek, TomaszX @ 2015-09-25 13:24 UTC (permalink / raw) To: Thomas Monjalon; +Cc: dev Hi Thomas, > -----Original Message----- > From: Thomas Monjalon [mailto:thomas.monjalon@6wind.com] > Sent: Friday, September 25, 2015 12:11 > To: Kulasek, TomaszX > Cc: dev@dpdk.org > Subject: Re: [dpdk-dev] [PATCHv4 5/9] null: export eth_dev_null_create > > > > --- a/drivers/net/null/rte_pmd_null_version.map > > +++ b/drivers/net/null/rte_pmd_null_version.map > > @@ -2,3 +2,10 @@ DPDK_2.0 { > > > > local: *; > > }; > > + > > +DPDK_2.1 { > > + global: > > + > > + eth_dev_null_create; > > + > > +} DPDK_2.0; > > Please, could you update it to DPDK_2.2? > > Thanks > --- a/drivers/net/null/rte_pmd_null_version.map +++ b/drivers/net/null/rte_pmd_null_version.map @@ -2,3 +2,10 @@ DPDK_2.0 { local: *; }; + +DPDK_2.2 { + global: + + eth_dev_null_create; + +} DPDK_2.0; This change is fine? Tomasz. ^ permalink raw reply [flat|nested] 126+ messages in thread
* Re: [dpdk-dev] [PATCHv4 5/9] null: export eth_dev_null_create 2015-09-25 13:24 ` Kulasek, TomaszX @ 2015-09-25 13:43 ` Thomas Monjalon 0 siblings, 0 replies; 126+ messages in thread From: Thomas Monjalon @ 2015-09-25 13:43 UTC (permalink / raw) To: Kulasek, TomaszX; +Cc: dev > > Please, could you update it to DPDK_2.2? > +DPDK_2.2 { > + global: > + > + eth_dev_null_create; > + > +} DPDK_2.0; > > This change is fine? Yes, only one letter to change :) ^ permalink raw reply [flat|nested] 126+ messages in thread
* Re: [dpdk-dev] [PATCHv4 5/9] null: export eth_dev_null_create 2015-07-15 17:26 ` [dpdk-dev] [PATCHv4 5/9] null: export eth_dev_null_create Tomasz Kulasek 2015-09-25 10:11 ` Thomas Monjalon @ 2015-09-29 2:28 ` Tetsuya Mukawa 1 sibling, 0 replies; 126+ messages in thread From: Tetsuya Mukawa @ 2015-09-29 2:28 UTC (permalink / raw) To: Tomasz Kulasek, dev On 2015/07/16 2:26, Tomasz Kulasek wrote: > To use eth_dev_null_create in application this method needs to be exported. > > Signed-off-by: Tomasz Kulasek <tomaszx.kulasek@intel.com> > --- > drivers/net/null/Makefile | 2 +- > drivers/net/null/rte_eth_null.c | 3 ++- > drivers/net/null/rte_eth_null.h | 40 +++++++++++++++++++++++++++++ > drivers/net/null/rte_pmd_null_version.map | 7 +++++ > 4 files changed, 50 insertions(+), 2 deletions(-) > create mode 100644 drivers/net/null/rte_eth_null.h > > diff --git a/drivers/net/null/Makefile b/drivers/net/null/Makefile > index 6472015..b33f9fd 100644 > --- a/drivers/net/null/Makefile > +++ b/drivers/net/null/Makefile > @@ -51,7 +51,7 @@ SRCS-$(CONFIG_RTE_LIBRTE_PMD_NULL) += rte_eth_null.c > # > # Export include files > # > -SYMLINK-y-include += > +SYMLINK-y-include += rte_eth_null.h > > # this lib depends upon: > DEPDIRS-$(CONFIG_RTE_LIBRTE_PMD_NULL) += lib/librte_mbuf > diff --git a/drivers/net/null/rte_eth_null.c b/drivers/net/null/rte_eth_null.c > index f393422..8ae6ebf 100644 > --- a/drivers/net/null/rte_eth_null.c > +++ b/drivers/net/null/rte_eth_null.c > @@ -2,6 +2,7 @@ > * BSD LICENSE > * > * Copyright (C) IGEL Co.,Ltd. > + * Copyright (c) 2015 Intel Corporation. > * All rights reserved. > * > * Redistribution and use in source and binary forms, with or without > @@ -481,7 +482,7 @@ static const struct eth_dev_ops ops = { > .rss_hash_conf_get = eth_rss_hash_conf_get > }; > > -static int > +int > eth_dev_null_create(const char *name, > const unsigned numa_node, > unsigned packet_size, > diff --git a/drivers/net/null/rte_eth_null.h b/drivers/net/null/rte_eth_null.h > new file mode 100644 > index 0000000..abada8c > --- /dev/null > +++ b/drivers/net/null/rte_eth_null.h > @@ -0,0 +1,40 @@ > +/*- > + * BSD LICENSE > + * > + * Copyright(c) 2015 Intel Corporation. All rights reserved. > + * All rights reserved. > + * > + * Redistribution and use in source and binary forms, with or without > + * modification, are permitted provided that the following conditions > + * are met: > + * > + * * Redistributions of source code must retain the above copyright > + * notice, this list of conditions and the following disclaimer. > + * * Redistributions in binary form must reproduce the above copyright > + * notice, this list of conditions and the following disclaimer in > + * the documentation and/or other materials provided with the > + * distribution. > + * * Neither the name of Intel Corporation nor the names of its > + * contributors may be used to endorse or promote products derived > + * from this software without specific prior written permission. > + * > + * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS > + * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT > + * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR > + * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT > + * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, > + * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT > + * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, > + * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY > + * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT > + * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE > + * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. > + */ > + > +#ifndef RTE_ETH_NULL_H_ > +#define RTE_ETH_NULL_H_ > + > +int eth_dev_null_create(const char *name, const unsigned numa_node, > + unsigned packet_size, unsigned packet_copy); > + > +#endif /* RTE_ETH_NULL_H_ */ > diff --git a/drivers/net/null/rte_pmd_null_version.map b/drivers/net/null/rte_pmd_null_version.map > index ef35398..2b2a743 100644 > --- a/drivers/net/null/rte_pmd_null_version.map > +++ b/drivers/net/null/rte_pmd_null_version.map > @@ -2,3 +2,10 @@ DPDK_2.0 { > > local: *; > }; > + > +DPDK_2.1 { > + global: > + > + eth_dev_null_create; > + > +} DPDK_2.0; Hi Thomasz, I don't have any comments without the version miss Thomas has already commented. Tetsuya ^ permalink raw reply [flat|nested] 126+ messages in thread
* [dpdk-dev] [PATCHv4 6/9] test: dynamic rss configuration 2015-07-15 17:26 ` [dpdk-dev] [PATCHv4 0/9] " Tomasz Kulasek ` (4 preceding siblings ...) 2015-07-15 17:26 ` [dpdk-dev] [PATCHv4 5/9] null: export eth_dev_null_create Tomasz Kulasek @ 2015-07-15 17:26 ` Tomasz Kulasek 2015-07-15 17:26 ` [dpdk-dev] [PATCHv4 7/9] bonding: queue stats mapping Tomasz Kulasek ` (4 subsequent siblings) 10 siblings, 0 replies; 126+ messages in thread From: Tomasz Kulasek @ 2015-07-15 17:26 UTC (permalink / raw) To: dev This test module uses modified null device to check right RSS configuration propagation. 1) Propagation test a) Set RSS hash function for bonding, fetch RSS hash function from bonded slave, check if same. Do it for all slaves. b) Repeat above for RSS key and RETA. 2)Dynamic adding slave to the bonding a) Remove slave from bonding. b) Change its configuration to other than bonding. c) Add slave to the started bonding device. d) Check if slaves configuration changed. 3) Repeat point 1) and 2) with RSS multi-queue mode enabled/disabled for bonding port. v4 changes: - adapted to use null pmd driver in place of ring pmd Signed-off-by: Tomasz Kulasek <tomaszx.kulasek@intel.com> --- app/test/Makefile | 8 + app/test/test_link_bonding_rssconf.c | 679 ++++++++++++++++++++++++++++++++++ 2 files changed, 687 insertions(+) create mode 100644 app/test/test_link_bonding_rssconf.c diff --git a/app/test/Makefile b/app/test/Makefile index caa359c..4d87d85 100644 --- a/app/test/Makefile +++ b/app/test/Makefile @@ -137,6 +137,7 @@ SRCS-$(CONFIG_RTE_LIBRTE_ACL) += test_acl.c ifeq ($(CONFIG_RTE_LIBRTE_PMD_RING),y) SRCS-$(CONFIG_RTE_LIBRTE_PMD_BOND) += test_link_bonding.c SRCS-$(CONFIG_RTE_LIBRTE_PMD_BOND) += test_link_bonding_mode4.c +SRCS-$(CONFIG_RTE_LIBRTE_PMD_BOND) += test_link_bonding_rssconf.c endif SRCS-$(CONFIG_RTE_LIBRTE_PMD_RING) += test_pmd_ring.c @@ -173,6 +174,13 @@ ifeq ($(CONFIG_RTE_BUILD_SHARED_LIB),y) LDLIBS += -lrte_pmd_ring endif endif +ifneq ($(CONFIG_RTE_LIBRTE_PMD_NULL),y) +$(error Link bonding rssconf tests require CONFIG_RTE_LIBRTE_PMD_NULL=y) +else +ifeq ($(CONFIG_RTE_BUILD_SHARED_LIB),y) +LDLIBS += -lrte_pmd_null +endif +endif endif include $(RTE_SDK)/mk/rte.app.mk diff --git a/app/test/test_link_bonding_rssconf.c b/app/test/test_link_bonding_rssconf.c new file mode 100644 index 0000000..e6714b4 --- /dev/null +++ b/app/test/test_link_bonding_rssconf.c @@ -0,0 +1,679 @@ +/*- + * BSD LICENSE + * + * Copyright(c) 2015 Intel Corporation. All rights reserved. + * All rights reserved. + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * + * * Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * * Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in + * the documentation and/or other materials provided with the + * distribution. + * * Neither the name of Intel Corporation nor the names of its + * contributors may be used to endorse or promote products derived + * from this software without specific prior written permission. + * + * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS + * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT + * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR + * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT + * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, + * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT + * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, + * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY + * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT + * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE + * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + */ + +#include <string.h> +#include <stdarg.h> +#include <stdio.h> +#include <stdlib.h> +#include <stdint.h> +#include <inttypes.h> +#include <errno.h> +#include <rte_cycles.h> +#include <sys/queue.h> + +#include <rte_byteorder.h> +#include <rte_common.h> +#include <rte_debug.h> +#include <rte_ethdev.h> +#include <rte_log.h> +#include <rte_lcore.h> +#include <rte_memory.h> + +#include <rte_string_fns.h> +#include <rte_errno.h> +#include <rte_eth_bond.h> +#include <rte_eth_null.h> + +#include "test.h" + +#define SLAVE_COUNT (4) + +#define RXTX_RING_SIZE 1024 +#define RXTX_QUEUE_COUNT 4 + +#define BONDED_DEV_NAME ("rssconf_bond_dev") + +#define SLAVE_DEV_NAME_FMT ("rssconf_slave%d") +#define SLAVE_RXTX_QUEUE_FMT ("rssconf_slave%d_q%d") + +#define NUM_MBUFS 8191 +#define MBUF_SIZE (1600 + sizeof(struct rte_mbuf) + RTE_PKTMBUF_HEADROOM) +#define MBUF_CACHE_SIZE 250 +#define BURST_SIZE 32 + +#define INVALID_SOCKET_ID (-1) +#define INVALID_PORT_ID (0xFF) +#define INVALID_BONDING_MODE (-1) + +struct slave_conf { + uint8_t port_id; + struct rte_eth_dev_info dev_info; + + struct rte_eth_rss_conf rss_conf; + uint8_t rss_key[40]; + struct rte_eth_rss_reta_entry64 reta_conf[512 / RTE_RETA_GROUP_SIZE]; + + uint8_t is_slave; + struct rte_ring *rxtx_queue[RXTX_QUEUE_COUNT]; +}; + +struct link_bonding_rssconf_unittest_params { + uint8_t bond_port_id; + struct rte_eth_dev_info bond_dev_info; + struct rte_eth_rss_reta_entry64 bond_reta_conf[512 / RTE_RETA_GROUP_SIZE]; + struct slave_conf slave_ports[SLAVE_COUNT]; + + struct rte_mempool *mbuf_pool; +}; + +static struct link_bonding_rssconf_unittest_params test_params = { + .bond_port_id = INVALID_PORT_ID, + .slave_ports = { + [0 ... SLAVE_COUNT - 1] = { .port_id = INVALID_PORT_ID, .is_slave = 0} + }, + .mbuf_pool = NULL, +}; + +/** + * Default port configuration with RSS turned off + */ +static struct rte_eth_conf default_pmd_conf = { + .rxmode = { + .mq_mode = ETH_MQ_RX_NONE, + .max_rx_pkt_len = ETHER_MAX_LEN, + .split_hdr_size = 0, + .header_split = 0, /**< Header Split disabled */ + .hw_ip_checksum = 0, /**< IP checksum offload enabled */ + .hw_vlan_filter = 0, /**< VLAN filtering disabled */ + .jumbo_frame = 0, /**< Jumbo Frame Support disabled */ + .hw_strip_crc = 0, /**< CRC stripped by hardware */ + }, + .txmode = { + .mq_mode = ETH_MQ_TX_NONE, + }, + .lpbk_mode = 0, +}; + +static struct rte_eth_conf rss_pmd_conf = { + .rxmode = { + .mq_mode = ETH_MQ_RX_RSS, + .max_rx_pkt_len = ETHER_MAX_LEN, + .split_hdr_size = 0, + .header_split = 0, /**< Header Split disabled */ + .hw_ip_checksum = 0, /**< IP checksum offload enabled */ + .hw_vlan_filter = 0, /**< VLAN filtering disabled */ + .jumbo_frame = 0, /**< Jumbo Frame Support disabled */ + .hw_strip_crc = 0, /**< CRC stripped by hardware */ + }, + .txmode = { + .mq_mode = ETH_MQ_TX_NONE, + }, + .rx_adv_conf = { + .rss_conf = { + .rss_key = NULL, + .rss_hf = ETH_RSS_IPV6, + }, + }, + .lpbk_mode = 0, +}; + +#define FOR_EACH(_i, _item, _array, _size) \ + for (_i = 0, _item = &_array[0]; _i < _size && (_item = &_array[_i]); _i++) + +/* Macro for iterating over every port that can be used as a slave + * in this test. + * _i variable used as an index in test_params->slave_ports + * _slave pointer to &test_params->slave_ports[_idx] + */ +#define FOR_EACH_PORT(_i, _port) \ + FOR_EACH(_i, _port, test_params.slave_ports, \ + RTE_DIM(test_params.slave_ports)) + +static int +configure_ethdev(uint8_t port_id, struct rte_eth_conf *eth_conf, uint8_t start) +{ + int rxq, txq; + + TEST_ASSERT(rte_eth_dev_configure(port_id, RXTX_QUEUE_COUNT, + RXTX_QUEUE_COUNT, eth_conf) == 0, "Failed to configure device %u", + port_id); + + for (rxq = 0; rxq < RXTX_QUEUE_COUNT; rxq++) { + TEST_ASSERT(rte_eth_rx_queue_setup(port_id, rxq, RXTX_RING_SIZE, + rte_eth_dev_socket_id(port_id), NULL, + test_params.mbuf_pool) == 0, "Failed to setup rx queue."); + } + + for (txq = 0; txq < RXTX_QUEUE_COUNT; txq++) { + TEST_ASSERT(rte_eth_tx_queue_setup(port_id, txq, RXTX_RING_SIZE, + rte_eth_dev_socket_id(port_id), NULL) == 0, + "Failed to setup tx queue."); + } + + if (start) { + TEST_ASSERT(rte_eth_dev_start(port_id) == 0, + "Failed to start device (%d).", port_id); + } + + return 0; +} + +/** + * Remove all slaves from bonding + */ +static int +remove_slaves(void) +{ + unsigned n; + struct slave_conf *port; + + FOR_EACH_PORT(n, port) { + port = &test_params.slave_ports[n]; + if (port->is_slave) { + TEST_ASSERT_SUCCESS(rte_eth_bond_slave_remove( + test_params.bond_port_id, port->port_id), + "Cannot remove slave %d from bonding", port->port_id); + port->is_slave = 0; + } + } + + return 0; +} + +static int +remove_slaves_and_stop_bonded_device(void) +{ + TEST_ASSERT_SUCCESS(remove_slaves(), "Removing slaves"); + rte_eth_dev_stop(test_params.bond_port_id); + return TEST_SUCCESS; +} + +/** + * Add all slaves to bonding + */ +static int +bond_slaves(void) +{ + unsigned n; + struct slave_conf *port; + + FOR_EACH_PORT(n, port) { + port = &test_params.slave_ports[n]; + if (!port->is_slave) { + TEST_ASSERT_SUCCESS(rte_eth_bond_slave_add(test_params.bond_port_id, + port->port_id), "Cannot attach slave %d to the bonding", + port->port_id); + port->is_slave = 1; + } + } + + return 0; +} + +/** + * Set all RETA values in port_id to value + */ +static int +reta_set(uint8_t port_id, uint8_t value, int reta_size) +{ + struct rte_eth_rss_reta_entry64 reta_conf[512/RTE_RETA_GROUP_SIZE]; + int i, j; + + for (i = 0; i < reta_size / RTE_RETA_GROUP_SIZE; i++) { + /* select all fields to set */ + reta_conf[i].mask = ~0LL; + for (j = 0; j < RTE_RETA_GROUP_SIZE; j++) + reta_conf[i].reta[j] = value; + } + + return rte_eth_dev_rss_reta_update(port_id, reta_conf, reta_size); +} + +/** + * Check if slaves RETA is synchronized with bonding port. Returns 1 if slave + * port is synced with bonding port. + */ +static int +reta_check_synced(struct slave_conf *port) +{ + unsigned i; + + for (i = 0; i < test_params.bond_dev_info.reta_size; + i++) { + + int index = i / RTE_RETA_GROUP_SIZE; + int shift = i % RTE_RETA_GROUP_SIZE; + + if (port->reta_conf[index].reta[shift] != + test_params.bond_reta_conf[index].reta[shift]) + return 0; + + } + + return 1; +} + +/** + * Fetch bonding ports RETA + */ +static int +bond_reta_fetch(void) { + unsigned j; + + for (j = 0; j < test_params.bond_dev_info.reta_size / RTE_RETA_GROUP_SIZE; + j++) + test_params.bond_reta_conf[j].mask = ~0LL; + + TEST_ASSERT_SUCCESS(rte_eth_dev_rss_reta_query(test_params.bond_port_id, + test_params.bond_reta_conf, test_params.bond_dev_info.reta_size), + "Cannot take bonding ports RSS configuration"); + return 0; +} + +/** + * Fetch slaves RETA + */ +static int +slave_reta_fetch(struct slave_conf *port) { + unsigned j; + + for (j = 0; j < port->dev_info.reta_size / RTE_RETA_GROUP_SIZE; j++) + port->reta_conf[j].mask = ~0LL; + + TEST_ASSERT_SUCCESS(rte_eth_dev_rss_reta_query(port->port_id, + port->reta_conf, port->dev_info.reta_size), + "Cannot take bonding ports RSS configuration"); + return 0; +} + +/** + * Remove and add slave to check if slaves configuration is synced with + * the bonding ports values after adding new slave. + */ +static int +slave_remove_and_add(void) +{ + struct slave_conf *port = &(test_params.slave_ports[0]); + + /* 1. Remove first slave from bonding */ + TEST_ASSERT_SUCCESS(rte_eth_bond_slave_remove(test_params.bond_port_id, + port->port_id), "Cannot remove slave #d from bonding"); + + /* 2. Change removed (ex-)slave and bonding configuration to different + * values + */ + reta_set(test_params.bond_port_id, 1, test_params.bond_dev_info.reta_size); + bond_reta_fetch(); + + reta_set(port->port_id, 2, port->dev_info.reta_size); + slave_reta_fetch(port); + + TEST_ASSERT(reta_check_synced(port) == 0, + "Removed slave didn't should be synchronized with bonding port"); + + /* 3. Add (ex-)slave and check if configuration changed*/ + TEST_ASSERT_SUCCESS(rte_eth_bond_slave_add(test_params.bond_port_id, + port->port_id), "Cannot add slave"); + + bond_reta_fetch(); + slave_reta_fetch(port); + + return reta_check_synced(port); +} + +/** + * Test configuration propagation over slaves. + */ +static int +test_propagate(void) +{ + unsigned i; + uint8_t n; + struct slave_conf *port; + uint8_t bond_rss_key[40]; + struct rte_eth_rss_conf bond_rss_conf; + + int retval = 0; + uint64_t rss_hf = 0; + uint64_t default_rss_hf = 0; + + rte_eth_dev_info_get(test_params.bond_port_id, &test_params.bond_dev_info); + + /* + * Test hash function propagation + */ + for (i = 0; i < sizeof(test_params.bond_dev_info.flow_type_rss_offloads)*8; + i++) { + + rss_hf = test_params.bond_dev_info.flow_type_rss_offloads & (1<<i); + if (rss_hf) { + bond_rss_conf.rss_key = NULL; + bond_rss_conf.rss_hf = rss_hf; + + retval = rte_eth_dev_rss_hash_update(test_params.bond_port_id, + &bond_rss_conf); + TEST_ASSERT_SUCCESS(retval, "Cannot set slaves hash function"); + + FOR_EACH_PORT(n, port) { + port = &test_params.slave_ports[n]; + + retval = rte_eth_dev_rss_hash_conf_get(port->port_id, + &port->rss_conf); + TEST_ASSERT_SUCCESS(retval, + "Cannot take slaves RSS configuration"); + + TEST_ASSERT(port->rss_conf.rss_hf == rss_hf, + "Hash function not propagated for slave %d", + port->port_id); + } + + default_rss_hf = rss_hf; + } + + } + + /* + * Test key propagation + */ + for (i = 1; i < 10; i++) { + + /* Set all keys to zero */ + FOR_EACH_PORT(n, port) { + port = &test_params.slave_ports[n]; + memset(port->rss_conf.rss_key, 0, 40); + retval = rte_eth_dev_rss_hash_update(port->port_id, + &port->rss_conf); + TEST_ASSERT_SUCCESS(retval, "Cannot set slaves RSS keys"); + } + + memset(bond_rss_key, i, sizeof(bond_rss_key)); + bond_rss_conf.rss_hf = default_rss_hf, + bond_rss_conf.rss_key = bond_rss_key; + bond_rss_conf.rss_key_len = 40; + + retval = rte_eth_dev_rss_hash_update(test_params.bond_port_id, + &bond_rss_conf); + TEST_ASSERT_SUCCESS(retval, "Cannot set bonded port RSS keys"); + + FOR_EACH_PORT(n, port) { + port = &test_params.slave_ports[n]; + + retval = rte_eth_dev_rss_hash_conf_get(port->port_id, + &(port->rss_conf)); + + TEST_ASSERT_SUCCESS(retval, + "Cannot take slaves RSS configuration"); + + /* compare keys */ + retval = memcmp(port->rss_conf.rss_key, bond_rss_key, + sizeof(bond_rss_key)); + TEST_ASSERT(retval == 0, "Key value not propagated for slave %d", + port->port_id); + } + } + + /* + * Test RETA propagation + */ + for (i = 0; i < RXTX_QUEUE_COUNT; i++) { + + /* Set all keys to zero */ + FOR_EACH_PORT(n, port) { + port = &test_params.slave_ports[n]; + retval = reta_set(port->port_id, (i + 1) % RXTX_QUEUE_COUNT, + port->dev_info.reta_size); + TEST_ASSERT_SUCCESS(retval, "Cannot set slaves RETA"); + } + + TEST_ASSERT_SUCCESS(reta_set(test_params.bond_port_id, + i % RXTX_QUEUE_COUNT, test_params.bond_dev_info.reta_size), + "Cannot set bonded port RETA"); + + bond_reta_fetch(); + + FOR_EACH_PORT(n, port) { + port = &test_params.slave_ports[n]; + + slave_reta_fetch(port); + TEST_ASSERT(reta_check_synced(port) == 1, "RETAs inconsistent"); + } + } + + return TEST_SUCCESS; +} + +/** + * Test propagation logic, when RX_RSS mq_mode is turned on for bonding port + */ +static int +test_rss(void) +{ + /** + * Configure bonding port in RSS mq mode + */ + TEST_ASSERT_SUCCESS(configure_ethdev(test_params.bond_port_id, + &rss_pmd_conf, 0), "Failed to configure bonding device\n"); + + rte_eth_dev_info_get(test_params.bond_port_id, &test_params.bond_dev_info); + + TEST_ASSERT_SUCCESS(bond_slaves(), "Bonding slaves failed"); + + TEST_ASSERT_SUCCESS(rte_eth_dev_start(test_params.bond_port_id), + "Failed to start bonding port (%d).", test_params.bond_port_id); + + TEST_ASSERT_SUCCESS(test_propagate(), "Propagation test failed"); + + TEST_ASSERT(slave_remove_and_add() == 1, "New slave should be synced"); + + remove_slaves_and_stop_bonded_device(); + + return TEST_SUCCESS; +} + +/** + * Test propagation logic, when RX_RSS mq_mode is turned off for bonding port + */ +static int +test_rss_lazy(void) +{ + TEST_ASSERT_SUCCESS(configure_ethdev(test_params.bond_port_id, + &default_pmd_conf, 0), "Failed to configure bonding device\n"); + + rte_eth_dev_info_get(test_params.bond_port_id, &test_params.bond_dev_info); + + TEST_ASSERT_SUCCESS(bond_slaves(), "Bonding slaves failed"); + + TEST_ASSERT_SUCCESS(rte_eth_dev_start(test_params.bond_port_id), + "Failed to start bonding port (%d).", test_params.bond_port_id); + + TEST_ASSERT_SUCCESS(test_propagate(), "Propagation test failed"); + + TEST_ASSERT(slave_remove_and_add() == 0, "New slave shouldn't be synced"); + + remove_slaves_and_stop_bonded_device(); + + return TEST_SUCCESS; +} + +static int +test_setup(void) +{ + unsigned n; + int retval; + int port_id; + char name[256]; + struct slave_conf *port; + + if (test_params.mbuf_pool == NULL) { + + test_params.mbuf_pool = rte_mempool_create("RSS_MBUF_POOL", NUM_MBUFS * + SLAVE_COUNT, MBUF_SIZE, MBUF_CACHE_SIZE, + sizeof(struct rte_pktmbuf_pool_private), rte_pktmbuf_pool_init, + NULL, rte_pktmbuf_init, NULL, rte_socket_id(), 0); + + TEST_ASSERT(test_params.mbuf_pool != NULL, + "rte_mempool_create failed\n"); + } + + /* Create / initialize ring eth devs. */ + FOR_EACH_PORT(n, port) { + port = &test_params.slave_ports[n]; + + port_id = rte_eth_dev_count(); + snprintf(name, sizeof(name), SLAVE_DEV_NAME_FMT, port_id); + + retval = eth_dev_null_create(name, 0, 64, 0); + TEST_ASSERT_SUCCESS(retval, "Failed to create null device '%s'\n", + name); + + port->port_id = port_id; + + port->rss_conf.rss_key = port->rss_key; + port->rss_conf.rss_key_len = 40; + + retval = configure_ethdev(port->port_id, &default_pmd_conf, 0); + TEST_ASSERT_SUCCESS(retval, "Failed to configure virtual ethdev %s\n", + name); + + rte_eth_dev_info_get(port->port_id, &port->dev_info); + } + + if (test_params.bond_port_id == INVALID_PORT_ID) { + retval = rte_eth_bond_create(BONDED_DEV_NAME, 0, rte_socket_id()); + + TEST_ASSERT(retval >= 0, "Failed to create bonded ethdev %s", + BONDED_DEV_NAME); + + test_params.bond_port_id = retval; + + TEST_ASSERT_SUCCESS(configure_ethdev(test_params.bond_port_id, + &default_pmd_conf, 0), "Failed to configure bonding device\n"); + + rte_eth_dev_info_get(test_params.bond_port_id, + &test_params.bond_dev_info); + } + + return TEST_SUCCESS; +} + +static int +testsuite_teardown(void) +{ + struct slave_conf *port; + uint8_t i; + + /* Only stop ports. + * Any cleanup/reset state is done when particular test is + * started. */ + + rte_eth_dev_stop(test_params.bond_port_id); + + FOR_EACH_PORT(i, port) + rte_eth_dev_stop(port->port_id); + + return 0; +} + +static int +check_environment(void) +{ + return TEST_SUCCESS; +} + +static int +test_rssconf_executor(int (*test_func)(void)) +{ + int test_result; + + /* Check if environment is clean. Fail to launch a test if there was + * a critical error before that prevented to reset environment. */ + TEST_ASSERT_SUCCESS(check_environment(), + "Refusing to launch test in dirty environment."); + + RTE_VERIFY(test_func != NULL); + test_result = (*test_func)(); + + /* If test succeed check if environment wast left in good condition. */ + if (test_result == TEST_SUCCESS) + test_result = check_environment(); + + /* Reset environment in case test failed to do that. */ + if (test_result != TEST_SUCCESS) { + TEST_ASSERT_SUCCESS(remove_slaves_and_stop_bonded_device(), + "Failed to stop bonded device"); + } + + return test_result; +} + +static int +test_setup_wrapper(void) +{ + return test_rssconf_executor(&test_setup); +} + +static int +test_rss_wrapper(void) +{ + return test_rssconf_executor(&test_rss); +} + +static int +test_rss_lazy_wrapper(void) +{ + return test_rssconf_executor(&test_rss_lazy); +} + +static struct unit_test_suite link_bonding_rssconf_test_suite = { + .suite_name = "RSS Dynamic Configuration for Bonding Unit Test Suite", + .teardown = testsuite_teardown, + .unit_test_cases = { + TEST_CASE_NAMED("test_setup", test_setup_wrapper), + TEST_CASE_NAMED("test_rss", test_rss_wrapper), + TEST_CASE_NAMED("test_rss_lazy", test_rss_lazy_wrapper), + { NULL, NULL, NULL, NULL, NULL } /**< NULL terminate unit test array */ + } +}; + +static int +test_link_bonding_rssconf(void) +{ + return unit_test_suite_runner(&link_bonding_rssconf_test_suite); +} + +static struct test_command test_link_bonding_rssconf_cmd = { + .command = "link_bonding_rssconf_autotest", + .callback = test_link_bonding_rssconf, +}; + +REGISTER_TEST_COMMAND(test_link_bonding_rssconf_cmd); -- 1.7.9.5 ^ permalink raw reply [flat|nested] 126+ messages in thread
* [dpdk-dev] [PATCHv4 7/9] bonding: queue stats mapping 2015-07-15 17:26 ` [dpdk-dev] [PATCHv4 0/9] " Tomasz Kulasek ` (5 preceding siblings ...) 2015-07-15 17:26 ` [dpdk-dev] [PATCHv4 6/9] test: dynamic rss configuration Tomasz Kulasek @ 2015-07-15 17:26 ` Tomasz Kulasek 2015-09-25 10:14 ` Thomas Monjalon 2015-07-15 17:26 ` [dpdk-dev] [PATCHv4 8/9] doc: fixed spellings and typos Tomasz Kulasek ` (3 subsequent siblings) 10 siblings, 1 reply; 126+ messages in thread From: Tomasz Kulasek @ 2015-07-15 17:26 UTC (permalink / raw) To: dev This patch adds propagation of mapping over the slaves, and fills bonding port's stats with a sum of corresponding values taken from bonded slaves, when stats are requested for bonding port. Signed-off-by: Tomasz Kulasek <tomaszx.kulasek@intel.com> --- drivers/net/bonding/rte_eth_bond_pmd.c | 34 +++++++++++++++++++++++++++++++- 1 file changed, 33 insertions(+), 1 deletion(-) diff --git a/drivers/net/bonding/rte_eth_bond_pmd.c b/drivers/net/bonding/rte_eth_bond_pmd.c index cd23f42..ff92011 100644 --- a/drivers/net/bonding/rte_eth_bond_pmd.c +++ b/drivers/net/bonding/rte_eth_bond_pmd.c @@ -1783,7 +1783,7 @@ bond_ethdev_stats_get(struct rte_eth_dev *dev, struct rte_eth_stats *stats) { struct bond_dev_private *internals = dev->data->dev_private; struct rte_eth_stats slave_stats; - int i; + int i, j; for (i = 0; i < internals->slave_count; i++) { rte_eth_stats_get(internals->slaves[i].port_id, &slave_stats); @@ -1802,6 +1802,15 @@ bond_ethdev_stats_get(struct rte_eth_dev *dev, struct rte_eth_stats *stats) stats->rx_pause_xon += slave_stats.rx_pause_xon; stats->tx_pause_xoff += slave_stats.tx_pause_xoff; stats->rx_pause_xoff += slave_stats.rx_pause_xoff; + + for (j = 0; j < RTE_ETHDEV_QUEUE_STAT_CNTRS; j++) { + stats->q_ipackets[j] += slave_stats.q_ipackets[j]; + stats->q_opackets[j] += slave_stats.q_opackets[j]; + stats->q_ibytes[j] += slave_stats.q_ibytes[j]; + stats->q_obytes[j] += slave_stats.q_obytes[j]; + stats->q_errors[j] += slave_stats.q_errors[j]; + } + } } @@ -2110,6 +2119,28 @@ bond_ethdev_rss_hash_conf_get(struct rte_eth_dev *dev, return 0; } +static int +bond_ethdev_queue_stats_mapping_set(struct rte_eth_dev *dev, uint16_t queue_id, + uint8_t stat_idx, uint8_t is_rx) +{ + int i; + int retval; + struct bond_dev_private *internals = dev->data->dev_private; + + for (i = 0; i < internals->slave_count; i++) { + if (is_rx) + retval = rte_eth_dev_set_rx_queue_stats_mapping( + internals->slaves[i].port_id, queue_id, stat_idx); + else + retval = rte_eth_dev_set_tx_queue_stats_mapping( + internals->slaves[i].port_id, queue_id, stat_idx); + if (retval != 0) + return retval; + } + + return 0; +} + struct eth_dev_ops default_dev_ops = { .dev_start = bond_ethdev_start, .dev_stop = bond_ethdev_stop, @@ -2123,6 +2154,7 @@ struct eth_dev_ops default_dev_ops = { .link_update = bond_ethdev_link_update, .stats_get = bond_ethdev_stats_get, .stats_reset = bond_ethdev_stats_reset, + .queue_stats_mapping_set = bond_ethdev_queue_stats_mapping_set, .promiscuous_enable = bond_ethdev_promiscuous_enable, .promiscuous_disable = bond_ethdev_promiscuous_disable, .reta_update = bond_ethdev_rss_reta_update, -- 1.7.9.5 ^ permalink raw reply [flat|nested] 126+ messages in thread
* Re: [dpdk-dev] [PATCHv4 7/9] bonding: queue stats mapping 2015-07-15 17:26 ` [dpdk-dev] [PATCHv4 7/9] bonding: queue stats mapping Tomasz Kulasek @ 2015-09-25 10:14 ` Thomas Monjalon 2015-09-25 13:26 ` Kulasek, TomaszX 0 siblings, 1 reply; 126+ messages in thread From: Thomas Monjalon @ 2015-09-25 10:14 UTC (permalink / raw) To: Tomasz Kulasek; +Cc: dev 2015-07-15 19:26, Tomasz Kulasek: > + .queue_stats_mapping_set = bond_ethdev_queue_stats_mapping_set, As explained with previous version of this patchset, this API should be removed. It is specific to some Intel devices. Please do not use it in bonding. Then we could discuss in another thread how to remove it from ethdev. ^ permalink raw reply [flat|nested] 126+ messages in thread
* Re: [dpdk-dev] [PATCHv4 7/9] bonding: queue stats mapping 2015-09-25 10:14 ` Thomas Monjalon @ 2015-09-25 13:26 ` Kulasek, TomaszX 0 siblings, 0 replies; 126+ messages in thread From: Kulasek, TomaszX @ 2015-09-25 13:26 UTC (permalink / raw) To: Thomas Monjalon; +Cc: dev > -----Original Message----- > From: Thomas Monjalon [mailto:thomas.monjalon@6wind.com] > Sent: Friday, September 25, 2015 12:15 > To: Kulasek, TomaszX > Cc: dev@dpdk.org > Subject: Re: [dpdk-dev] [PATCHv4 7/9] bonding: queue stats mapping > > 2015-07-15 19:26, Tomasz Kulasek: > > + .queue_stats_mapping_set = > bond_ethdev_queue_stats_mapping_set, > > As explained with previous version of this patchset, this API should be > removed. It is specific to some Intel devices. > Please do not use it in bonding. > Then we could discuss in another thread how to remove it from ethdev. Ok, I remove it in v5. ^ permalink raw reply [flat|nested] 126+ messages in thread
* [dpdk-dev] [PATCHv4 8/9] doc: fixed spellings and typos 2015-07-15 17:26 ` [dpdk-dev] [PATCHv4 0/9] " Tomasz Kulasek ` (6 preceding siblings ...) 2015-07-15 17:26 ` [dpdk-dev] [PATCHv4 7/9] bonding: queue stats mapping Tomasz Kulasek @ 2015-07-15 17:26 ` Tomasz Kulasek 2015-07-15 17:26 ` [dpdk-dev] [PATCHv4 9/9] doc: dynamic rss configuration for bonding Tomasz Kulasek ` (2 subsequent siblings) 10 siblings, 0 replies; 126+ messages in thread From: Tomasz Kulasek @ 2015-07-15 17:26 UTC (permalink / raw) To: dev Signed-off-by: Tomasz Kulasek <tomaszx.kulasek@intel.com> --- .../prog_guide/link_bonding_poll_mode_drv_lib.rst | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/doc/guides/prog_guide/link_bonding_poll_mode_drv_lib.rst b/doc/guides/prog_guide/link_bonding_poll_mode_drv_lib.rst index 96e554f..03baf90 100644 --- a/doc/guides/prog_guide/link_bonding_poll_mode_drv_lib.rst +++ b/doc/guides/prog_guide/link_bonding_poll_mode_drv_lib.rst @@ -188,7 +188,7 @@ conditions are not met. If a user wishes to monitor individual slaves then they must register callbacks with that slave directly. The link bonding library also supports devices which do not implement link -status change interrupts, this is achieve by polling the devices link status at +status change interrupts, this is achieved by polling the devices link status at a defined period which is set using the ``rte_eth_bond_link_monitoring_set`` API, the default polling interval is 10ms. When a device is added as a slave to a bonding device it is determined using the ``RTE_PCI_DRV_INTR_LSC`` flag @@ -286,7 +286,7 @@ and UDP protocols for load balancing. Using Link Bonding Devices -------------------------- -The librte_pmd_bond library support two modes of device creation, the libraries +The librte_pmd_bond library supports two modes of device creation, the libraries export full C API or using the EAL command line to statically configure link bonding devices at application startup. Using the EAL option it is possible to use link bonding functionality transparently without specific knowledge of the @@ -299,7 +299,7 @@ Using the Poll Mode Driver from an Application Using the librte_pmd_bond libraries API it is possible to dynamically create and manage link bonding device from within any application. Link bonding -device are created using the ``rte_eth_bond_create`` API which requires a +devices are created using the ``rte_eth_bond_create`` API which requires a unique device name, the link bonding mode to initial the device in and finally the socket Id which to allocate the devices resources onto. After successful creation of a bonding device it must be configured using the generic Ethernet @@ -362,7 +362,7 @@ The different options are: mode=2 * slave: Defines the PMD device which will be added as slave to the bonded - device. This option can be selected multiple time, for each device to be + device. This option can be selected multiple times, for each device to be added as a slave. Physical devices should be specified using their PCI address, in the format domain:bus:devid.function -- 1.7.9.5 ^ permalink raw reply [flat|nested] 126+ messages in thread
* [dpdk-dev] [PATCHv4 9/9] doc: dynamic rss configuration for bonding 2015-07-15 17:26 ` [dpdk-dev] [PATCHv4 0/9] " Tomasz Kulasek ` (7 preceding siblings ...) 2015-07-15 17:26 ` [dpdk-dev] [PATCHv4 8/9] doc: fixed spellings and typos Tomasz Kulasek @ 2015-07-15 17:26 ` Tomasz Kulasek 2015-09-25 10:16 ` [dpdk-dev] [PATCHv4 0/9] Dynamic RSS Configuration for Bonding Thomas Monjalon 2015-09-30 14:04 ` [dpdk-dev] [PATCH v5 " Tomasz Kulasek 10 siblings, 0 replies; 126+ messages in thread From: Tomasz Kulasek @ 2015-07-15 17:26 UTC (permalink / raw) To: dev Documentation update about implementation details and requirements for Dynamic RSS Configuration for Bonding. Signed-off-by: Tomasz Kulasek <tomaszx.kulasek@intel.com> --- .../prog_guide/link_bonding_poll_mode_drv_lib.rst | 34 ++++++++++++++++++-- 1 file changed, 32 insertions(+), 2 deletions(-) diff --git a/doc/guides/prog_guide/link_bonding_poll_mode_drv_lib.rst b/doc/guides/prog_guide/link_bonding_poll_mode_drv_lib.rst index 03baf90..46f0296 100644 --- a/doc/guides/prog_guide/link_bonding_poll_mode_drv_lib.rst +++ b/doc/guides/prog_guide/link_bonding_poll_mode_drv_lib.rst @@ -1,5 +1,5 @@ .. BSD LICENSE - Copyright(c) 2010-2014 Intel Corporation. All rights reserved. + Copyright(c) 2010-2015 Intel Corporation. All rights reserved. All rights reserved. Redistribution and use in source and binary forms, with or without @@ -173,7 +173,28 @@ After a slave device is added to a bonded device slave is stopped using ``rte_eth_dev_stop`` and then reconfigured using ``rte_eth_dev_configure`` the RX and TX queues are also reconfigured using ``rte_eth_tx_queue_setup`` / ``rte_eth_rx_queue_setup`` with the parameters use to configure the bonding -device. +device. If RSS is enabled for bonding device, this mode is also enabled on new +slave and configured as well. + +Setting up multi-queue mode for bonding device to RSS, makes it fully +RSS-capable, so all slaves are synchronized with its configuration. This mode is +intended to provide RSS configuration on slaves transparent for client +application implementation. + +Bonding device stores its own version of RSS settings i.e. RETA, RSS hash +function and RSS key, used to set up its slaves. That let to define the meaning +of RSS configuration of bonding device as desired configuration of whole bonding +(as one unit), without pointing any of slave inside. It is required to ensure +consistency and made it more errorproof. + +RSS hash function set for bonding device, is a maximal set of RSS hash functions +supported by all bonded slaves. RETA size is a GCD of all its RETA's sizes, so +it can be easily used as a pattern providing expected behavior, even if slave +RETAs' sizes are different. If RSS Key is not set for bonded device, it's not +changed on the slaves and default key for device is used. + +All settings are managed through the bonding port API and always are propagated +in one direction (from bonding to slaves). Link Status Change Interrupts / Polling ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ @@ -207,6 +228,15 @@ these parameters. A bonding device must have a minimum of one slave before the bonding device itself can be started. +To use a bonding device dynamic RSS configuration feature effectively, it is +also required, that all slaves should be RSS-capable and support, at least one +common hash function available for each of them. Changing RSS key is only +possible, when all slave devices support the same key size. + +To prevent inconsistency on how slaves process packets, once a device is added +to a bonding device, RSS configuration should be managed through the bonding +device API, and not directly on the slave. + Like all other PMD, all functions exported by a PMD are lock-free functions that are assumed not to be invoked in parallel on different logical cores to work on the same target object. -- 1.7.9.5 ^ permalink raw reply [flat|nested] 126+ messages in thread
* Re: [dpdk-dev] [PATCHv4 0/9] Dynamic RSS Configuration for Bonding 2015-07-15 17:26 ` [dpdk-dev] [PATCHv4 0/9] " Tomasz Kulasek ` (8 preceding siblings ...) 2015-07-15 17:26 ` [dpdk-dev] [PATCHv4 9/9] doc: dynamic rss configuration for bonding Tomasz Kulasek @ 2015-09-25 10:16 ` Thomas Monjalon 2015-09-25 13:28 ` Kulasek, TomaszX 2015-09-30 14:04 ` [dpdk-dev] [PATCH v5 " Tomasz Kulasek 10 siblings, 1 reply; 126+ messages in thread From: Thomas Monjalon @ 2015-09-25 10:16 UTC (permalink / raw) To: Tetsuya Mukawa; +Cc: dev 2015-07-15 19:26, Tomasz Kulasek: > Tomasz Kulasek (9): > bonding: rss dynamic configuration > null: fix segfault when null_pmd added to bonding > null: extend number of virtual queues > null: virtual dynamic rss configuration > null: export eth_dev_null_create Please Tetsuya, could you review the patches on null PMD? Thanks ^ permalink raw reply [flat|nested] 126+ messages in thread
* Re: [dpdk-dev] [PATCHv4 0/9] Dynamic RSS Configuration for Bonding 2015-09-25 10:16 ` [dpdk-dev] [PATCHv4 0/9] Dynamic RSS Configuration for Bonding Thomas Monjalon @ 2015-09-25 13:28 ` Kulasek, TomaszX 2015-09-25 13:44 ` Thomas Monjalon 0 siblings, 1 reply; 126+ messages in thread From: Kulasek, TomaszX @ 2015-09-25 13:28 UTC (permalink / raw) To: Thomas Monjalon, Tetsuya Mukawa; +Cc: dev > -----Original Message----- > From: Thomas Monjalon [mailto:thomas.monjalon@6wind.com] > Sent: Friday, September 25, 2015 12:17 > To: Tetsuya Mukawa > Cc: dev@dpdk.org; Kulasek, TomaszX > Subject: Re: [dpdk-dev] [PATCHv4 0/9] Dynamic RSS Configuration for > Bonding > > 2015-07-15 19:26, Tomasz Kulasek: > > Tomasz Kulasek (9): > > bonding: rss dynamic configuration > > null: fix segfault when null_pmd added to bonding > > null: extend number of virtual queues > > null: virtual dynamic rss configuration > > null: export eth_dev_null_create > > Please Tetsuya, could you review the patches on null PMD? > Thanks Do you need patch-set v5 or should I wait? Tomasz ^ permalink raw reply [flat|nested] 126+ messages in thread
* Re: [dpdk-dev] [PATCHv4 0/9] Dynamic RSS Configuration for Bonding 2015-09-25 13:28 ` Kulasek, TomaszX @ 2015-09-25 13:44 ` Thomas Monjalon 0 siblings, 0 replies; 126+ messages in thread From: Thomas Monjalon @ 2015-09-25 13:44 UTC (permalink / raw) To: Kulasek, TomaszX; +Cc: dev 2015-09-25 13:28, Kulasek, TomaszX: > From: Thomas Monjalon [mailto:thomas.monjalon@6wind.com] > > Please Tetsuya, could you review the patches on null PMD? > > Thanks > > Do you need patch-set v5 or should I wait? I think you should wait for Tetsuya feedback. ^ permalink raw reply [flat|nested] 126+ messages in thread
* [dpdk-dev] [PATCH v5 0/9] Dynamic RSS Configuration for Bonding 2015-07-15 17:26 ` [dpdk-dev] [PATCHv4 0/9] " Tomasz Kulasek ` (9 preceding siblings ...) 2015-09-25 10:16 ` [dpdk-dev] [PATCHv4 0/9] Dynamic RSS Configuration for Bonding Thomas Monjalon @ 2015-09-30 14:04 ` Tomasz Kulasek 2015-09-30 14:04 ` [dpdk-dev] [PATCH v5 1/9] bonding: rss dynamic configuration Tomasz Kulasek ` (9 more replies) 10 siblings, 10 replies; 126+ messages in thread From: Tomasz Kulasek @ 2015-09-30 14:04 UTC (permalink / raw) To: dev OVERVIEW -------- 1) Setting .rxmode.mq_mode for bonding device to ETH_MQ_RX_RSS makes bonding device fully RSS-capable, so all slaves are synchronized with its configuration. This mode is intended to provide RSS configuration as known from "dynamic RSS configuration for one port" and made slaves transparent for client application implementation. 2) If .rxmode.mq_mode for bonding device isn't ETH_MQ_RX_RSS, slaves are not synchronized. That provides an ability to configure them manually. This mode may be useful when application wants to manage RSS in an unusual way and the consistency of RSS configuration for slaves isn't required. Turning on/off RSS mode for slaves when bonding is started is not possible. Other RSS configuration is propagated over slaves, when bonding device API is used to do it. v5 changes: - updated to DPDK 2.2 - removed copyright change from null device source - removed queue_stats_mapping_set from eth_dev_ops of bonding device - null pmd cleanups (removed unnecessary malloc, replaced memcpy with rte_memcpy) - fixed queues number configuration in null pmd v4 changes: - fixed copy-paste error, - removed example application as too complex and introducing a new dependency, - addapted null pmd to be used as testing device for dynamic RSS configuration, - addapted test units to use null pmd instead of ring pmd, - ring pmd is not used and changed in this patchset v3 changes: - checkpatch cleanups v2 changes: - added support for keys other than 40 bytes long, - now, if RSS key is not set for bonding, it is not set also for slaves, - fix - full initial RSS configuration before any slave is added was not possible due to the initially zeroed flow_type_rss_offloads for bonding, - fix - changed error to warning when slave is synchronizing due to the bonding's initial configuration (to allow use slaves' drivers not supporting dynamic RSS configuration in bonding), - some code cleanups, - updated documentation, Tomasz Kulasek (9): bonding: rss dynamic configuration null: fix segfault when null_pmd added to bonding null: extend number of virtual queues null: virtual dynamic rss configuration null: export eth_dev_null_create test: dynamic rss configuration bonding: per queue stats doc: fixed spellings and typos doc: dynamic rss configuration for bonding app/test/Makefile | 8 + app/test/test_link_bonding_rssconf.c | 679 ++++++++++++++++++++ .../prog_guide/link_bonding_poll_mode_drv_lib.rst | 42 +- drivers/net/bonding/rte_eth_bond_api.c | 28 + drivers/net/bonding/rte_eth_bond_pmd.c | 216 ++++++- drivers/net/bonding/rte_eth_bond_private.h | 12 + drivers/net/null/Makefile | 2 +- drivers/net/null/rte_eth_null.c | 148 ++++- drivers/net/null/rte_eth_null.h | 40 ++ drivers/net/null/rte_pmd_null_version.map | 7 + 10 files changed, 1150 insertions(+), 32 deletions(-) create mode 100644 app/test/test_link_bonding_rssconf.c create mode 100644 drivers/net/null/rte_eth_null.h -- 1.7.9.5 ^ permalink raw reply [flat|nested] 126+ messages in thread
* [dpdk-dev] [PATCH v5 1/9] bonding: rss dynamic configuration 2015-09-30 14:04 ` [dpdk-dev] [PATCH v5 " Tomasz Kulasek @ 2015-09-30 14:04 ` Tomasz Kulasek 2015-09-30 14:04 ` [dpdk-dev] [PATCH v5 2/9] null: fix segfault when null_pmd added to bonding Tomasz Kulasek ` (8 subsequent siblings) 9 siblings, 0 replies; 126+ messages in thread From: Tomasz Kulasek @ 2015-09-30 14:04 UTC (permalink / raw) To: dev Bonding device implements independent management of RSS settings. It stores its own copies of settings i.e. RETA, RSS hash function and RSS key. It’s required to ensure consistency. 1) RSS hash function set for bonding device is maximal set of RSS hash functions supported by all bonded devices. That mean, to have RSS support for bonding, all slaves should be RSS-capable. 2) RSS key is propagated over the slaves "as is". 3) RETA for bonding is an internal table managed by bonding API, and is used as a pattern to set up slaves. Its size is GCD of all RETA sizes, so it can be easily used as a pattern providing expected behavior, even if slaves RETA sizes are different. Signed-off-by: Tomasz Kulasek <tomaszx.kulasek@intel.com> --- drivers/net/bonding/rte_eth_bond_api.c | 28 ++++ drivers/net/bonding/rte_eth_bond_pmd.c | 205 ++++++++++++++++++++++++++-- drivers/net/bonding/rte_eth_bond_private.h | 12 ++ 3 files changed, 231 insertions(+), 14 deletions(-) diff --git a/drivers/net/bonding/rte_eth_bond_api.c b/drivers/net/bonding/rte_eth_bond_api.c index 0681d1a..92073df 100644 --- a/drivers/net/bonding/rte_eth_bond_api.c +++ b/drivers/net/bonding/rte_eth_bond_api.c @@ -273,6 +273,9 @@ rte_eth_bond_create(const char *name, uint8_t mode, uint8_t socket_id) internals->rx_offload_capa = 0; internals->tx_offload_capa = 0; + /* Initially allow to choose any offload type */ + internals->flow_type_rss_offloads = ETH_RSS_PROTO_MASK; + memset(internals->active_slaves, 0, sizeof(internals->active_slaves)); memset(internals->slaves, 0, sizeof(internals->slaves)); @@ -369,6 +372,11 @@ __eth_bond_slave_add_lock_free(uint8_t bonded_port_id, uint8_t slave_port_id) rte_eth_dev_info_get(slave_port_id, &dev_info); + /* We need to store slaves reta_size to be able to synchronize RETA for all + * slave devices even if its sizes are different. + */ + internals->slaves[internals->slave_count].reta_size = dev_info.reta_size; + if (internals->slave_count < 1) { /* if MAC is not user defined then use MAC of first slave add to * bonded device */ @@ -382,9 +390,16 @@ __eth_bond_slave_add_lock_free(uint8_t bonded_port_id, uint8_t slave_port_id) /* Make primary slave */ internals->primary_port = slave_port_id; + /* Inherit queues settings from first slave */ + internals->nb_rx_queues = slave_eth_dev->data->nb_rx_queues; + internals->nb_tx_queues = slave_eth_dev->data->nb_tx_queues; + + internals->reta_size = dev_info.reta_size; + /* Take the first dev's offload capabilities */ internals->rx_offload_capa = dev_info.rx_offload_capa; internals->tx_offload_capa = dev_info.tx_offload_capa; + internals->flow_type_rss_offloads = dev_info.flow_type_rss_offloads; } else { /* Check slave link properties are supported if props are set, @@ -403,8 +418,19 @@ __eth_bond_slave_add_lock_free(uint8_t bonded_port_id, uint8_t slave_port_id) } internals->rx_offload_capa &= dev_info.rx_offload_capa; internals->tx_offload_capa &= dev_info.tx_offload_capa; + internals->flow_type_rss_offloads &= dev_info.flow_type_rss_offloads; + + /* RETA size is GCD of all slaves RETA sizes, so, if all sizes will be + * the power of 2, the lower one is GCD + */ + if (internals->reta_size > dev_info.reta_size) + internals->reta_size = dev_info.reta_size; + } + bonded_eth_dev->data->dev_conf.rx_adv_conf.rss_conf.rss_hf &= + internals->flow_type_rss_offloads; + internals->slave_count++; /* Update all slave devices MACs*/ @@ -531,6 +557,8 @@ __eth_bond_slave_remove_lock_free(uint8_t bonded_port_id, uint8_t slave_port_id) if (internals->slave_count == 0) { internals->rx_offload_capa = 0; internals->tx_offload_capa = 0; + internals->flow_type_rss_offloads = ETH_RSS_PROTO_MASK; + internals->reta_size = 0; } return 0; } diff --git a/drivers/net/bonding/rte_eth_bond_pmd.c b/drivers/net/bonding/rte_eth_bond_pmd.c index 5cc6372..2880f5c 100644 --- a/drivers/net/bonding/rte_eth_bond_pmd.c +++ b/drivers/net/bonding/rte_eth_bond_pmd.c @@ -1310,6 +1310,23 @@ slave_configure(struct rte_eth_dev *bonded_eth_dev, if (slave_eth_dev->driver->pci_drv.drv_flags & RTE_PCI_DRV_INTR_LSC) slave_eth_dev->data->dev_conf.intr_conf.lsc = 1; + /* If RSS is enabled for bonding, try to enable it for slaves */ + if (bonded_eth_dev->data->dev_conf.rxmode.mq_mode & ETH_MQ_RX_RSS) { + if (bonded_eth_dev->data->dev_conf.rx_adv_conf.rss_conf.rss_key_len + != 0) { + slave_eth_dev->data->dev_conf.rx_adv_conf.rss_conf.rss_key_len = + bonded_eth_dev->data->dev_conf.rx_adv_conf.rss_conf.rss_key_len; + slave_eth_dev->data->dev_conf.rx_adv_conf.rss_conf.rss_key = + bonded_eth_dev->data->dev_conf.rx_adv_conf.rss_conf.rss_key; + } else { + slave_eth_dev->data->dev_conf.rx_adv_conf.rss_conf.rss_key = NULL; + } + + slave_eth_dev->data->dev_conf.rx_adv_conf.rss_conf.rss_hf = + bonded_eth_dev->data->dev_conf.rx_adv_conf.rss_conf.rss_hf; + slave_eth_dev->data->dev_conf.rxmode.mq_mode |= ETH_MQ_RX_RSS; + } + /* Configure device */ errval = rte_eth_dev_configure(slave_eth_dev->data->port_id, bonded_eth_dev->data->nb_rx_queues, @@ -1361,6 +1378,30 @@ slave_configure(struct rte_eth_dev *bonded_eth_dev, return -1; } + /* If RSS is enabled for bonding, synchronize RETA */ + if (bonded_eth_dev->data->dev_conf.rxmode.mq_mode & ETH_MQ_RX_RSS) { + int i; + struct bond_dev_private *internals; + + internals = bonded_eth_dev->data->dev_private; + + for (i = 0; i < internals->slave_count; i++) { + if (internals->slaves[i].port_id == slave_eth_dev->data->port_id) { + errval = rte_eth_dev_rss_reta_update( + slave_eth_dev->data->port_id, + &internals->reta_conf[0], + internals->slaves[i].reta_size); + if (errval != 0) { + RTE_LOG(WARNING, PMD, + "rte_eth_dev_rss_reta_update on slave port %d fails (err %d)." + " RSS Configuration for bonding may be inconsistent.\n", + slave_eth_dev->data->port_id, errval); + } + break; + } + } + } + /* If lsc interrupt is set, check initial slave's link status */ if (slave_eth_dev->driver->pci_drv.drv_flags & RTE_PCI_DRV_INTR_LSC) bond_ethdev_lsc_event_callback(slave_eth_dev->data->port_id, @@ -1596,6 +1637,9 @@ bond_ethdev_info(struct rte_eth_dev *dev, struct rte_eth_dev_info *dev_info) dev_info->rx_offload_capa = internals->rx_offload_capa; dev_info->tx_offload_capa = internals->tx_offload_capa; + dev_info->flow_type_rss_offloads = internals->flow_type_rss_offloads; + + dev_info->reta_size = internals->reta_size; } static int @@ -1977,21 +2021,132 @@ bond_ethdev_lsc_event_callback(uint8_t port_id, enum rte_eth_event_type type, } } +static int +bond_ethdev_rss_reta_update(struct rte_eth_dev *dev, + struct rte_eth_rss_reta_entry64 *reta_conf, uint16_t reta_size) +{ + unsigned i, j; + int result = 0; + int slave_reta_size; + unsigned reta_count; + struct bond_dev_private *internals = dev->data->dev_private; + + if (reta_size != internals->reta_size) + return -EINVAL; + + /* Copy RETA table */ + reta_count = reta_size / RTE_RETA_GROUP_SIZE; + + for (i = 0; i < reta_count; i++) { + internals->reta_conf[i].mask = reta_conf[i].mask; + for (j = 0; j < RTE_RETA_GROUP_SIZE; j++) + if ((reta_conf[i].mask >> j) & 0x01) + internals->reta_conf[i].reta[j] = reta_conf[i].reta[j]; + } + + /* Fill rest of array */ + for (; i < RTE_DIM(internals->reta_conf); i += reta_count) + memcpy(&internals->reta_conf[i], &internals->reta_conf[0], + sizeof(internals->reta_conf[0]) * reta_count); + + /* Propagate RETA over slaves */ + for (i = 0; i < internals->slave_count; i++) { + slave_reta_size = internals->slaves[i].reta_size; + result = rte_eth_dev_rss_reta_update(internals->slaves[i].port_id, + &internals->reta_conf[0], slave_reta_size); + if (result < 0) + return result; + } + + return 0; +} + +static int +bond_ethdev_rss_reta_query(struct rte_eth_dev *dev, + struct rte_eth_rss_reta_entry64 *reta_conf, uint16_t reta_size) +{ + int i, j; + struct bond_dev_private *internals = dev->data->dev_private; + + if (reta_size != internals->reta_size) + return -EINVAL; + + /* Copy RETA table */ + for (i = 0; i < reta_size / RTE_RETA_GROUP_SIZE; i++) + for (j = 0; j < RTE_RETA_GROUP_SIZE; j++) + if ((reta_conf[i].mask >> j) & 0x01) + reta_conf[i].reta[j] = internals->reta_conf[i].reta[j]; + + return 0; +} + +static int +bond_ethdev_rss_hash_update(struct rte_eth_dev *dev, + struct rte_eth_rss_conf *rss_conf) +{ + int i, result = 0; + struct bond_dev_private *internals = dev->data->dev_private; + struct rte_eth_rss_conf bond_rss_conf; + + memcpy(&bond_rss_conf, rss_conf, sizeof(struct rte_eth_rss_conf)); + + bond_rss_conf.rss_hf &= internals->flow_type_rss_offloads; + + if (bond_rss_conf.rss_hf != 0) + dev->data->dev_conf.rx_adv_conf.rss_conf.rss_hf = bond_rss_conf.rss_hf; + + if (bond_rss_conf.rss_key && bond_rss_conf.rss_key_len < + sizeof(internals->rss_key)) { + if (bond_rss_conf.rss_key_len == 0) + bond_rss_conf.rss_key_len = 40; + internals->rss_key_len = bond_rss_conf.rss_key_len; + memcpy(internals->rss_key, bond_rss_conf.rss_key, + internals->rss_key_len); + } + + for (i = 0; i < internals->slave_count; i++) { + result = rte_eth_dev_rss_hash_update(internals->slaves[i].port_id, + &bond_rss_conf); + if (result < 0) + return result; + } + + return 0; +} + +static int +bond_ethdev_rss_hash_conf_get(struct rte_eth_dev *dev, + struct rte_eth_rss_conf *rss_conf) +{ + struct bond_dev_private *internals = dev->data->dev_private; + + rss_conf->rss_hf = dev->data->dev_conf.rx_adv_conf.rss_conf.rss_hf; + rss_conf->rss_key_len = internals->rss_key_len; + if (rss_conf->rss_key) + memcpy(rss_conf->rss_key, internals->rss_key, internals->rss_key_len); + + return 0; +} + struct eth_dev_ops default_dev_ops = { - .dev_start = bond_ethdev_start, - .dev_stop = bond_ethdev_stop, - .dev_close = bond_ethdev_close, - .dev_configure = bond_ethdev_configure, - .dev_infos_get = bond_ethdev_info, - .rx_queue_setup = bond_ethdev_rx_queue_setup, - .tx_queue_setup = bond_ethdev_tx_queue_setup, - .rx_queue_release = bond_ethdev_rx_queue_release, - .tx_queue_release = bond_ethdev_tx_queue_release, - .link_update = bond_ethdev_link_update, - .stats_get = bond_ethdev_stats_get, - .stats_reset = bond_ethdev_stats_reset, - .promiscuous_enable = bond_ethdev_promiscuous_enable, - .promiscuous_disable = bond_ethdev_promiscuous_disable + .dev_start = bond_ethdev_start, + .dev_stop = bond_ethdev_stop, + .dev_close = bond_ethdev_close, + .dev_configure = bond_ethdev_configure, + .dev_infos_get = bond_ethdev_info, + .rx_queue_setup = bond_ethdev_rx_queue_setup, + .tx_queue_setup = bond_ethdev_tx_queue_setup, + .rx_queue_release = bond_ethdev_rx_queue_release, + .tx_queue_release = bond_ethdev_tx_queue_release, + .link_update = bond_ethdev_link_update, + .stats_get = bond_ethdev_stats_get, + .stats_reset = bond_ethdev_stats_reset, + .promiscuous_enable = bond_ethdev_promiscuous_enable, + .promiscuous_disable = bond_ethdev_promiscuous_disable, + .reta_update = bond_ethdev_rss_reta_update, + .reta_query = bond_ethdev_rss_reta_query, + .rss_hash_update = bond_ethdev_rss_hash_update, + .rss_hash_conf_get = bond_ethdev_rss_hash_conf_get }; static int @@ -2090,6 +2245,28 @@ bond_ethdev_configure(struct rte_eth_dev *dev) int arg_count; uint8_t port_id = dev - rte_eth_devices; + static const uint8_t default_rss_key[40] = { + 0x6D, 0x5A, 0x56, 0xDA, 0x25, 0x5B, 0x0E, 0xC2, 0x41, 0x67, 0x25, 0x3D, + 0x43, 0xA3, 0x8F, 0xB0, 0xD0, 0xCA, 0x2B, 0xCB, 0xAE, 0x7B, 0x30, 0xB4, + 0x77, 0xCB, 0x2D, 0xA3, 0x80, 0x30, 0xF2, 0x0C, 0x6A, 0x42, 0xB7, 0x3B, + 0xBE, 0xAC, 0x01, 0xFA + }; + + unsigned i, j; + + /* If RSS is enabled, fill table and key with default values */ + if (dev->data->dev_conf.rxmode.mq_mode & ETH_MQ_RX_RSS) { + dev->data->dev_conf.rx_adv_conf.rss_conf.rss_key = internals->rss_key; + dev->data->dev_conf.rx_adv_conf.rss_conf.rss_key_len = 0; + memcpy(internals->rss_key, default_rss_key, 40); + + for (i = 0; i < RTE_DIM(internals->reta_conf); i++) { + internals->reta_conf[i].mask = ~0LL; + for (j = 0; j < RTE_RETA_GROUP_SIZE; j++) + internals->reta_conf[i].reta[j] = j % dev->data->nb_rx_queues; + } + } + /* * if no kvlist, it means that this bonded device has been created * through the bonding api. diff --git a/drivers/net/bonding/rte_eth_bond_private.h b/drivers/net/bonding/rte_eth_bond_private.h index 038bca6..e7af809 100644 --- a/drivers/net/bonding/rte_eth_bond_private.h +++ b/drivers/net/bonding/rte_eth_bond_private.h @@ -103,6 +103,8 @@ struct bond_slave_details { uint8_t last_link_status; /**< Port Id of slave eth_dev */ struct ether_addr persisted_mac_addr; + + uint16_t reta_size; }; @@ -155,6 +157,16 @@ struct bond_dev_private { uint32_t rx_offload_capa; /** Rx offload capability */ uint32_t tx_offload_capa; /** Tx offload capability */ + /** Bit mask of RSS offloads, the bit offset also means flow type */ + uint64_t flow_type_rss_offloads; + + uint16_t reta_size; + struct rte_eth_rss_reta_entry64 reta_conf[ETH_RSS_RETA_SIZE_512 / + RTE_RETA_GROUP_SIZE]; + + uint8_t rss_key[52]; /**< 52-byte hash key buffer. */ + uint8_t rss_key_len; /**< hash key length in bytes. */ + struct rte_kvargs *kvlist; uint8_t slave_update_idx; }; -- 1.7.9.5 ^ permalink raw reply [flat|nested] 126+ messages in thread
* [dpdk-dev] [PATCH v5 2/9] null: fix segfault when null_pmd added to bonding 2015-09-30 14:04 ` [dpdk-dev] [PATCH v5 " Tomasz Kulasek 2015-09-30 14:04 ` [dpdk-dev] [PATCH v5 1/9] bonding: rss dynamic configuration Tomasz Kulasek @ 2015-09-30 14:04 ` Tomasz Kulasek 2015-09-30 14:04 ` [dpdk-dev] [PATCH v5 3/9] null: extend number of virtual queues Tomasz Kulasek ` (7 subsequent siblings) 9 siblings, 0 replies; 126+ messages in thread From: Tomasz Kulasek @ 2015-09-30 14:04 UTC (permalink / raw) To: dev This patch initializes eth_dev->link_intr_cbs queue used when null pmd is added to the bonding. v5 changes: - removed unnecessary malloc for eth_driver (rte_null_pmd) Signed-off-by: Tomasz Kulasek <tomaszx.kulasek@intel.com> --- drivers/net/null/rte_eth_null.c | 2 ++ 1 file changed, 2 insertions(+) diff --git a/drivers/net/null/rte_eth_null.c b/drivers/net/null/rte_eth_null.c index e244595..c748101 100644 --- a/drivers/net/null/rte_eth_null.c +++ b/drivers/net/null/rte_eth_null.c @@ -432,6 +432,7 @@ eth_dev_null_create(const char *name, internals->numa_node = numa_node; pci_dev->numa_node = numa_node; + pci_dev->driver = &rte_null_pmd.pci_drv; data->dev_private = internals; data->port_id = eth_dev->data->port_id; @@ -445,6 +446,7 @@ eth_dev_null_create(const char *name, eth_dev->dev_ops = &ops; eth_dev->pci_dev = pci_dev; eth_dev->driver = &rte_null_pmd; + TAILQ_INIT(ð_dev->link_intr_cbs); /* finally assign rx and tx ops */ if (packet_copy) { -- 1.7.9.5 ^ permalink raw reply [flat|nested] 126+ messages in thread
* [dpdk-dev] [PATCH v5 3/9] null: extend number of virtual queues 2015-09-30 14:04 ` [dpdk-dev] [PATCH v5 " Tomasz Kulasek 2015-09-30 14:04 ` [dpdk-dev] [PATCH v5 1/9] bonding: rss dynamic configuration Tomasz Kulasek 2015-09-30 14:04 ` [dpdk-dev] [PATCH v5 2/9] null: fix segfault when null_pmd added to bonding Tomasz Kulasek @ 2015-09-30 14:04 ` Tomasz Kulasek 2015-10-14 1:34 ` Tetsuya Mukawa 2015-09-30 14:05 ` [dpdk-dev] [PATCH v5 4/9] null: virtual dynamic rss configuration Tomasz Kulasek ` (6 subsequent siblings) 9 siblings, 1 reply; 126+ messages in thread From: Tomasz Kulasek @ 2015-09-30 14:04 UTC (permalink / raw) To: dev This patch adds a possibility to configure more than one queue on null device. v5 changes: - fixed queues number configuration (using internals->nb_*_queues instead of dev->data->nb_*_queues) Signed-off-by: Tomasz Kulasek <tomaszx.kulasek@intel.com> --- drivers/net/null/rte_eth_null.c | 28 +++++++++++++++++++--------- 1 file changed, 19 insertions(+), 9 deletions(-) diff --git a/drivers/net/null/rte_eth_null.c b/drivers/net/null/rte_eth_null.c index c748101..bf81b1b 100644 --- a/drivers/net/null/rte_eth_null.c +++ b/drivers/net/null/rte_eth_null.c @@ -71,8 +71,8 @@ struct pmd_internals { unsigned nb_rx_queues; unsigned nb_tx_queues; - struct null_queue rx_null_queues[1]; - struct null_queue tx_null_queues[1]; + struct null_queue rx_null_queues[RTE_MAX_QUEUES_PER_PORT]; + struct null_queue tx_null_queues[RTE_MAX_QUEUES_PER_PORT]; }; @@ -178,7 +178,15 @@ eth_null_copy_tx(void *q, struct rte_mbuf **bufs, uint16_t nb_bufs) } static int -eth_dev_configure(struct rte_eth_dev *dev __rte_unused) { return 0; } +eth_dev_configure(struct rte_eth_dev *dev) { + struct pmd_internals *internals; + + internals = dev->data->dev_private; + internals->nb_rx_queues = dev->data->nb_rx_queues; + internals->nb_tx_queues = dev->data->nb_tx_queues; + + return 0; +} static int eth_dev_start(struct rte_eth_dev *dev) @@ -213,10 +221,11 @@ eth_rx_queue_setup(struct rte_eth_dev *dev, uint16_t rx_queue_id, if ((dev == NULL) || (mb_pool == NULL)) return -EINVAL; - if (rx_queue_id != 0) + internals = dev->data->dev_private; + + if (rx_queue_id >= internals->nb_rx_queues) return -ENODEV; - internals = dev->data->dev_private; packet_size = internals->packet_size; internals->rx_null_queues[rx_queue_id].mb_pool = mb_pool; @@ -246,10 +255,11 @@ eth_tx_queue_setup(struct rte_eth_dev *dev, uint16_t tx_queue_id, if (dev == NULL) return -EINVAL; - if (tx_queue_id != 0) + internals = dev->data->dev_private; + + if (tx_queue_id >= internals->nb_tx_queues) return -ENODEV; - internals = dev->data->dev_private; packet_size = internals->packet_size; dev->data->tx_queues[tx_queue_id] = @@ -279,8 +289,8 @@ eth_dev_info(struct rte_eth_dev *dev, dev_info->driver_name = drivername; dev_info->max_mac_addrs = 1; dev_info->max_rx_pktlen = (uint32_t)-1; - dev_info->max_rx_queues = (uint16_t)internals->nb_rx_queues; - dev_info->max_tx_queues = (uint16_t)internals->nb_tx_queues; + dev_info->max_rx_queues = RTE_DIM(internals->rx_null_queues); + dev_info->max_tx_queues = RTE_DIM(internals->tx_null_queues); dev_info->min_rx_bufsize = 0; dev_info->pci_dev = NULL; } -- 1.7.9.5 ^ permalink raw reply [flat|nested] 126+ messages in thread
* Re: [dpdk-dev] [PATCH v5 3/9] null: extend number of virtual queues 2015-09-30 14:04 ` [dpdk-dev] [PATCH v5 3/9] null: extend number of virtual queues Tomasz Kulasek @ 2015-10-14 1:34 ` Tetsuya Mukawa 0 siblings, 0 replies; 126+ messages in thread From: Tetsuya Mukawa @ 2015-10-14 1:34 UTC (permalink / raw) To: Tomasz Kulasek, dev On 2015/09/30 23:04, Tomasz Kulasek wrote: > This patch adds a possibility to configure more than one queue on null > device. > > v5 changes: > - fixed queues number configuration (using internals->nb_*_queues instead > of dev->data->nb_*_queues) > > Signed-off-by: Tomasz Kulasek <tomaszx.kulasek@intel.com> > --- > drivers/net/null/rte_eth_null.c | 28 +++++++++++++++++++--------- > 1 file changed, 19 insertions(+), 9 deletions(-) Acked-by: Tetsuya Mukawa <mukawa@igel.co.jp> ^ permalink raw reply [flat|nested] 126+ messages in thread
* [dpdk-dev] [PATCH v5 4/9] null: virtual dynamic rss configuration 2015-09-30 14:04 ` [dpdk-dev] [PATCH v5 " Tomasz Kulasek ` (2 preceding siblings ...) 2015-09-30 14:04 ` [dpdk-dev] [PATCH v5 3/9] null: extend number of virtual queues Tomasz Kulasek @ 2015-09-30 14:05 ` Tomasz Kulasek 2015-10-14 1:34 ` Tetsuya Mukawa 2015-10-15 7:46 ` Tetsuya Mukawa 2015-09-30 14:05 ` [dpdk-dev] [PATCH v5 5/9] null: export eth_dev_null_create Tomasz Kulasek ` (5 subsequent siblings) 9 siblings, 2 replies; 126+ messages in thread From: Tomasz Kulasek @ 2015-09-30 14:05 UTC (permalink / raw) To: dev This implementation allows to set and read RSS configuration for null device, and is used to validate right values propagation over the slaves, in test units for dynamic RSS configuration for bonding. v5 changes: - replaced memcpy with rte_memcpy Signed-off-by: Tomasz Kulasek <tomaszx.kulasek@intel.com> --- drivers/net/null/rte_eth_null.c | 116 +++++++++++++++++++++++++++++++++++++++ 1 file changed, 116 insertions(+) diff --git a/drivers/net/null/rte_eth_null.c b/drivers/net/null/rte_eth_null.c index bf81b1b..b01f647 100644 --- a/drivers/net/null/rte_eth_null.c +++ b/drivers/net/null/rte_eth_null.c @@ -37,6 +37,8 @@ #include <rte_memcpy.h> #include <rte_dev.h> #include <rte_kvargs.h> +#include <rte_eth_null.h> +#include <rte_spinlock.h> #define ETH_NULL_PACKET_SIZE_ARG "size" #define ETH_NULL_PACKET_COPY_ARG "copy" @@ -73,6 +75,17 @@ struct pmd_internals { struct null_queue rx_null_queues[RTE_MAX_QUEUES_PER_PORT]; struct null_queue tx_null_queues[RTE_MAX_QUEUES_PER_PORT]; + + /** Bit mask of RSS offloads, the bit offset also means flow type */ + uint64_t flow_type_rss_offloads; + + rte_spinlock_t rss_lock; + + uint16_t reta_size; + struct rte_eth_rss_reta_entry64 reta_conf[ETH_RSS_RETA_SIZE_128 / + RTE_RETA_GROUP_SIZE]; + + uint8_t rss_key[40]; /**< 40-byte hash key. */ }; @@ -293,6 +306,8 @@ eth_dev_info(struct rte_eth_dev *dev, dev_info->max_tx_queues = RTE_DIM(internals->tx_null_queues); dev_info->min_rx_bufsize = 0; dev_info->pci_dev = NULL; + dev_info->reta_size = internals->reta_size; + dev_info->flow_type_rss_offloads = internals->flow_type_rss_offloads; } static void @@ -373,6 +388,91 @@ static int eth_link_update(struct rte_eth_dev *dev __rte_unused, int wait_to_complete __rte_unused) { return 0; } +static int +eth_rss_reta_update(struct rte_eth_dev *dev, + struct rte_eth_rss_reta_entry64 *reta_conf, uint16_t reta_size) +{ + int i, j; + struct pmd_internals *internal = dev->data->dev_private; + + if (reta_size != internal->reta_size) + return -EINVAL; + + rte_spinlock_lock(&internal->rss_lock); + + /* Copy RETA table */ + for (i = 0; i < (internal->reta_size / RTE_RETA_GROUP_SIZE); i++) { + internal->reta_conf[i].mask = reta_conf[i].mask; + for (j = 0; j < RTE_RETA_GROUP_SIZE; j++) + if ((reta_conf[i].mask >> j) & 0x01) + internal->reta_conf[i].reta[j] = reta_conf[i].reta[j]; + } + + rte_spinlock_unlock(&internal->rss_lock); + + return 0; +} + +static int +eth_rss_reta_query(struct rte_eth_dev *dev, + struct rte_eth_rss_reta_entry64 *reta_conf, uint16_t reta_size) +{ + int i, j; + struct pmd_internals *internal = dev->data->dev_private; + + if (reta_size != internal->reta_size) + return -EINVAL; + + rte_spinlock_lock(&internal->rss_lock); + + /* Copy RETA table */ + for (i = 0; i < (internal->reta_size / RTE_RETA_GROUP_SIZE); i++) { + for (j = 0; j < RTE_RETA_GROUP_SIZE; j++) + if ((reta_conf[i].mask >> j) & 0x01) + reta_conf[i].reta[j] = internal->reta_conf[i].reta[j]; + } + + rte_spinlock_unlock(&internal->rss_lock); + + return 0; +} + +static int +eth_rss_hash_update(struct rte_eth_dev *dev, struct rte_eth_rss_conf *rss_conf) +{ + struct pmd_internals *internal = dev->data->dev_private; + + rte_spinlock_lock(&internal->rss_lock); + + if ((rss_conf->rss_hf & internal->flow_type_rss_offloads) != 0) + dev->data->dev_conf.rx_adv_conf.rss_conf.rss_hf = + rss_conf->rss_hf & internal->flow_type_rss_offloads; + + if (rss_conf->rss_key) + rte_memcpy(internal->rss_key, rss_conf->rss_key, 40); + + rte_spinlock_unlock(&internal->rss_lock); + + return 0; +} + +static int +eth_rss_hash_conf_get(struct rte_eth_dev *dev, + struct rte_eth_rss_conf *rss_conf) +{ + struct pmd_internals *internal = dev->data->dev_private; + + rte_spinlock_lock(&internal->rss_lock); + + rss_conf->rss_hf = dev->data->dev_conf.rx_adv_conf.rss_conf.rss_hf; + if (rss_conf->rss_key) + rte_memcpy(rss_conf->rss_key, internal->rss_key, 40); + + rte_spinlock_unlock(&internal->rss_lock); + + return 0; +} + static const struct eth_dev_ops ops = { .dev_start = eth_dev_start, .dev_stop = eth_dev_stop, @@ -385,6 +485,10 @@ static const struct eth_dev_ops ops = { .link_update = eth_link_update, .stats_get = eth_stats_get, .stats_reset = eth_stats_reset, + .reta_update = eth_rss_reta_update, + .reta_query = eth_rss_reta_query, + .rss_hash_update = eth_rss_hash_update, + .rss_hash_conf_get = eth_rss_hash_conf_get }; static int @@ -400,6 +504,13 @@ eth_dev_null_create(const char *name, struct pmd_internals *internals = NULL; struct rte_eth_dev *eth_dev = NULL; + static const uint8_t default_rss_key[40] = { + 0x6D, 0x5A, 0x56, 0xDA, 0x25, 0x5B, 0x0E, 0xC2, 0x41, 0x67, 0x25, 0x3D, + 0x43, 0xA3, 0x8F, 0xB0, 0xD0, 0xCA, 0x2B, 0xCB, 0xAE, 0x7B, 0x30, 0xB4, + 0x77, 0xCB, 0x2D, 0xA3, 0x80, 0x30, 0xF2, 0x0C, 0x6A, 0x42, 0xB7, 0x3B, + 0xBE, 0xAC, 0x01, 0xFA + }; + if (name == NULL) return -EINVAL; @@ -441,6 +552,11 @@ eth_dev_null_create(const char *name, internals->packet_copy = packet_copy; internals->numa_node = numa_node; + internals->flow_type_rss_offloads = ETH_RSS_PROTO_MASK; + internals->reta_size = RTE_DIM(internals->reta_conf) * RTE_RETA_GROUP_SIZE; + + rte_memcpy(internals->rss_key, default_rss_key, 40); + pci_dev->numa_node = numa_node; pci_dev->driver = &rte_null_pmd.pci_drv; -- 1.7.9.5 ^ permalink raw reply [flat|nested] 126+ messages in thread
* Re: [dpdk-dev] [PATCH v5 4/9] null: virtual dynamic rss configuration 2015-09-30 14:05 ` [dpdk-dev] [PATCH v5 4/9] null: virtual dynamic rss configuration Tomasz Kulasek @ 2015-10-14 1:34 ` Tetsuya Mukawa 2015-10-15 7:46 ` Tetsuya Mukawa 1 sibling, 0 replies; 126+ messages in thread From: Tetsuya Mukawa @ 2015-10-14 1:34 UTC (permalink / raw) To: Tomasz Kulasek, dev On 2015/09/30 23:05, Tomasz Kulasek wrote: > This implementation allows to set and read RSS configuration for null > device, and is used to validate right values propagation over the slaves, > in test units for dynamic RSS configuration for bonding. > > v5 changes: > - replaced memcpy with rte_memcpy > > Signed-off-by: Tomasz Kulasek <tomaszx.kulasek@intel.com> > --- > drivers/net/null/rte_eth_null.c | 116 +++++++++++++++++++++++++++++++++++++++ > 1 file changed, 116 insertions(+) Acked-by: Tetsuya Mukawa <mukawa@igel.co.jp> ^ permalink raw reply [flat|nested] 126+ messages in thread
* Re: [dpdk-dev] [PATCH v5 4/9] null: virtual dynamic rss configuration 2015-09-30 14:05 ` [dpdk-dev] [PATCH v5 4/9] null: virtual dynamic rss configuration Tomasz Kulasek 2015-10-14 1:34 ` Tetsuya Mukawa @ 2015-10-15 7:46 ` Tetsuya Mukawa 2015-10-15 8:42 ` Kulasek, TomaszX 1 sibling, 1 reply; 126+ messages in thread From: Tetsuya Mukawa @ 2015-10-15 7:46 UTC (permalink / raw) To: Tomasz Kulasek, dev On 2015/09/30 23:05, Tomasz Kulasek wrote: > This implementation allows to set and read RSS configuration for null > device, and is used to validate right values propagation over the slaves, > in test units for dynamic RSS configuration for bonding. > > v5 changes: > - replaced memcpy with rte_memcpy > > Signed-off-by: Tomasz Kulasek <tomaszx.kulasek@intel.com> > --- > drivers/net/null/rte_eth_null.c | 116 +++++++++++++++++++++++++++++++++++++++ > 1 file changed, 116 insertions(+) > > diff --git a/drivers/net/null/rte_eth_null.c b/drivers/net/null/rte_eth_null.c > index bf81b1b..b01f647 100644 > --- a/drivers/net/null/rte_eth_null.c > +++ b/drivers/net/null/rte_eth_null.c > @@ -37,6 +37,8 @@ > #include <rte_memcpy.h> > #include <rte_dev.h> > #include <rte_kvargs.h> > +#include <rte_eth_null.h> > +#include <rte_spinlock.h> > > Hi Tomasz, We don't have "rte_eth_null.h" at this point. (The header file will be added next patch) Probably, we also need "rte_pmd_null_version.map" to compile correctly. (To make sure, please compile DPDK with "CONFIG_RTE_BUILD_SHARED_LIB=y" option.) Also, it seems 'rte_eth_null.h' should be included like below. #include "rte_eth_null.h" Without it, we cannot compile. Thanks, Tetsuya ^ permalink raw reply [flat|nested] 126+ messages in thread
* Re: [dpdk-dev] [PATCH v5 4/9] null: virtual dynamic rss configuration 2015-10-15 7:46 ` Tetsuya Mukawa @ 2015-10-15 8:42 ` Kulasek, TomaszX 2015-10-15 9:21 ` Tetsuya Mukawa 0 siblings, 1 reply; 126+ messages in thread From: Kulasek, TomaszX @ 2015-10-15 8:42 UTC (permalink / raw) To: Tetsuya Mukawa, dev > -----Original Message----- > From: Tetsuya Mukawa [mailto:mukawa@igel.co.jp] > Sent: Thursday, October 15, 2015 09:46 > To: Kulasek, TomaszX; dev@dpdk.org > Subject: Re: [dpdk-dev] [PATCH v5 4/9] null: virtual dynamic rss > configuration > > On 2015/09/30 23:05, Tomasz Kulasek wrote: > > This implementation allows to set and read RSS configuration for null > > device, and is used to validate right values propagation over the > > slaves, in test units for dynamic RSS configuration for bonding. > > > > v5 changes: > > - replaced memcpy with rte_memcpy > > > > Signed-off-by: Tomasz Kulasek <tomaszx.kulasek@intel.com> > > --- > > drivers/net/null/rte_eth_null.c | 116 > > +++++++++++++++++++++++++++++++++++++++ > > 1 file changed, 116 insertions(+) > > > > diff --git a/drivers/net/null/rte_eth_null.c > > b/drivers/net/null/rte_eth_null.c index bf81b1b..b01f647 100644 > > --- a/drivers/net/null/rte_eth_null.c > > +++ b/drivers/net/null/rte_eth_null.c > > @@ -37,6 +37,8 @@ > > #include <rte_memcpy.h> > > #include <rte_dev.h> > > #include <rte_kvargs.h> > > +#include <rte_eth_null.h> > > +#include <rte_spinlock.h> > > > > > > > Hi Tomasz, > > We don't have "rte_eth_null.h" at this point. > (The header file will be added next patch) Probably, we also need > "rte_pmd_null_version.map" to compile correctly. > (To make sure, please compile DPDK with "CONFIG_RTE_BUILD_SHARED_LIB=y" > option.) > > Also, it seems 'rte_eth_null.h' should be included like below. > #include "rte_eth_null.h" > Without it, we cannot compile. > > Thanks, > Tetsuya Hi Tetsuya, This file and modifications are already included in "[dpdk-dev,v5,5/9] null: export eth_dev_null_create" in the patch set, also the required symlink is created like below: --- diff --git a/drivers/net/null/Makefile b/drivers/net/null/Makefile index 96ba01c..2202389 100644 --- a/drivers/net/null/Makefile +++ b/drivers/net/null/Makefile @@ -51,7 +51,7 @@ SRCS-$(CONFIG_RTE_LIBRTE_PMD_NULL) += rte_eth_null.c # # Export include files # -SYMLINK-y-include += +SYMLINK-y-include += rte_eth_null.h # this lib depends upon: DEPDIRS-$(CONFIG_RTE_LIBRTE_PMD_NULL) += lib/librte_mbuf --- Tomasz. ^ permalink raw reply [flat|nested] 126+ messages in thread
* Re: [dpdk-dev] [PATCH v5 4/9] null: virtual dynamic rss configuration 2015-10-15 8:42 ` Kulasek, TomaszX @ 2015-10-15 9:21 ` Tetsuya Mukawa 0 siblings, 0 replies; 126+ messages in thread From: Tetsuya Mukawa @ 2015-10-15 9:21 UTC (permalink / raw) To: Kulasek, TomaszX, dev On 2015/10/15 17:42, Kulasek, TomaszX wrote: >> -----Original Message----- >> From: Tetsuya Mukawa [mailto:mukawa@igel.co.jp] >> Sent: Thursday, October 15, 2015 09:46 >> To: Kulasek, TomaszX; dev@dpdk.org >> Subject: Re: [dpdk-dev] [PATCH v5 4/9] null: virtual dynamic rss >> configuration >> >> On 2015/09/30 23:05, Tomasz Kulasek wrote: >>> This implementation allows to set and read RSS configuration for null >>> device, and is used to validate right values propagation over the >>> slaves, in test units for dynamic RSS configuration for bonding. >>> >>> v5 changes: >>> - replaced memcpy with rte_memcpy >>> >>> Signed-off-by: Tomasz Kulasek <tomaszx.kulasek@intel.com> >>> --- >>> drivers/net/null/rte_eth_null.c | 116 >>> +++++++++++++++++++++++++++++++++++++++ >>> 1 file changed, 116 insertions(+) >>> >>> diff --git a/drivers/net/null/rte_eth_null.c >>> b/drivers/net/null/rte_eth_null.c index bf81b1b..b01f647 100644 >>> --- a/drivers/net/null/rte_eth_null.c >>> +++ b/drivers/net/null/rte_eth_null.c >>> @@ -37,6 +37,8 @@ >>> #include <rte_memcpy.h> >>> #include <rte_dev.h> >>> #include <rte_kvargs.h> >>> +#include <rte_eth_null.h> >>> +#include <rte_spinlock.h> >>> >>> >> >> Hi Tomasz, >> >> We don't have "rte_eth_null.h" at this point. >> (The header file will be added next patch) Probably, we also need >> "rte_pmd_null_version.map" to compile correctly. >> (To make sure, please compile DPDK with "CONFIG_RTE_BUILD_SHARED_LIB=y" >> option.) >> >> Also, it seems 'rte_eth_null.h' should be included like below. >> #include "rte_eth_null.h" >> Without it, we cannot compile. >> >> Thanks, >> Tetsuya > Hi Tetsuya, > > This file and modifications are already included in "[dpdk-dev,v5,5/9] null: export eth_dev_null_create" in the patch set, also the required symlink is created like below: Hi Tomasz, But "rte_eth_null.h" is used in " [PATCH v5 4/9] null: virtual dynamic rss configuration". It means we cannot compile DPDK before applying "[dpdk-dev,v5,5/9]". When we need to use git-bisect, this will be problem. So could you please keep DPDK being able to compile at any point? Thanks, Tetsuya, ^ permalink raw reply [flat|nested] 126+ messages in thread
* [dpdk-dev] [PATCH v5 5/9] null: export eth_dev_null_create 2015-09-30 14:04 ` [dpdk-dev] [PATCH v5 " Tomasz Kulasek ` (3 preceding siblings ...) 2015-09-30 14:05 ` [dpdk-dev] [PATCH v5 4/9] null: virtual dynamic rss configuration Tomasz Kulasek @ 2015-09-30 14:05 ` Tomasz Kulasek 2015-10-14 1:35 ` Tetsuya Mukawa 2015-09-30 14:05 ` [dpdk-dev] [PATCH v5 6/9] test: dynamic rss configuration Tomasz Kulasek ` (4 subsequent siblings) 9 siblings, 1 reply; 126+ messages in thread From: Tomasz Kulasek @ 2015-09-30 14:05 UTC (permalink / raw) To: dev Signed-off-by: Tomasz Kulasek <tomaszx.kulasek@intel.com> --- drivers/net/null/Makefile | 2 +- drivers/net/null/rte_eth_null.c | 2 +- drivers/net/null/rte_eth_null.h | 40 +++++++++++++++++++++++++++++ drivers/net/null/rte_pmd_null_version.map | 7 +++++ 4 files changed, 49 insertions(+), 2 deletions(-) create mode 100644 drivers/net/null/rte_eth_null.h diff --git a/drivers/net/null/Makefile b/drivers/net/null/Makefile index 96ba01c..2202389 100644 --- a/drivers/net/null/Makefile +++ b/drivers/net/null/Makefile @@ -51,7 +51,7 @@ SRCS-$(CONFIG_RTE_LIBRTE_PMD_NULL) += rte_eth_null.c # # Export include files # -SYMLINK-y-include += +SYMLINK-y-include += rte_eth_null.h # this lib depends upon: DEPDIRS-$(CONFIG_RTE_LIBRTE_PMD_NULL) += lib/librte_mbuf diff --git a/drivers/net/null/rte_eth_null.c b/drivers/net/null/rte_eth_null.c index b01f647..56e4278 100644 --- a/drivers/net/null/rte_eth_null.c +++ b/drivers/net/null/rte_eth_null.c @@ -491,7 +491,7 @@ static const struct eth_dev_ops ops = { .rss_hash_conf_get = eth_rss_hash_conf_get }; -static int +int eth_dev_null_create(const char *name, const unsigned numa_node, unsigned packet_size, diff --git a/drivers/net/null/rte_eth_null.h b/drivers/net/null/rte_eth_null.h new file mode 100644 index 0000000..abada8c --- /dev/null +++ b/drivers/net/null/rte_eth_null.h @@ -0,0 +1,40 @@ +/*- + * BSD LICENSE + * + * Copyright(c) 2015 Intel Corporation. All rights reserved. + * All rights reserved. + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * + * * Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * * Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in + * the documentation and/or other materials provided with the + * distribution. + * * Neither the name of Intel Corporation nor the names of its + * contributors may be used to endorse or promote products derived + * from this software without specific prior written permission. + * + * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS + * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT + * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR + * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT + * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, + * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT + * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, + * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY + * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT + * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE + * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + */ + +#ifndef RTE_ETH_NULL_H_ +#define RTE_ETH_NULL_H_ + +int eth_dev_null_create(const char *name, const unsigned numa_node, + unsigned packet_size, unsigned packet_copy); + +#endif /* RTE_ETH_NULL_H_ */ diff --git a/drivers/net/null/rte_pmd_null_version.map b/drivers/net/null/rte_pmd_null_version.map index ef35398..84b1d0f 100644 --- a/drivers/net/null/rte_pmd_null_version.map +++ b/drivers/net/null/rte_pmd_null_version.map @@ -2,3 +2,10 @@ DPDK_2.0 { local: *; }; + +DPDK_2.2 { + global: + + eth_dev_null_create; + +} DPDK_2.0; -- 1.7.9.5 ^ permalink raw reply [flat|nested] 126+ messages in thread
* Re: [dpdk-dev] [PATCH v5 5/9] null: export eth_dev_null_create 2015-09-30 14:05 ` [dpdk-dev] [PATCH v5 5/9] null: export eth_dev_null_create Tomasz Kulasek @ 2015-10-14 1:35 ` Tetsuya Mukawa 2015-10-14 12:42 ` Jastrzebski, MichalX K 0 siblings, 1 reply; 126+ messages in thread From: Tetsuya Mukawa @ 2015-10-14 1:35 UTC (permalink / raw) To: Tomasz Kulasek, dev On 2015/09/30 23:05, Tomasz Kulasek wrote: > Signed-off-by: Tomasz Kulasek <tomaszx.kulasek@intel.com> > --- > drivers/net/null/Makefile | 2 +- > drivers/net/null/rte_eth_null.c | 2 +- > drivers/net/null/rte_eth_null.h | 40 +++++++++++++++++++++++++++++ > drivers/net/null/rte_pmd_null_version.map | 7 +++++ > 4 files changed, 49 insertions(+), 2 deletions(-) > create mode 100644 drivers/net/null/rte_eth_null.h Acked-by: Tetsuya Mukawa <mukawa@igel.co.jp> ^ permalink raw reply [flat|nested] 126+ messages in thread
* Re: [dpdk-dev] [PATCH v5 5/9] null: export eth_dev_null_create 2015-10-14 1:35 ` Tetsuya Mukawa @ 2015-10-14 12:42 ` Jastrzebski, MichalX K 2015-10-15 8:16 ` Tetsuya Mukawa 0 siblings, 1 reply; 126+ messages in thread From: Jastrzebski, MichalX K @ 2015-10-14 12:42 UTC (permalink / raw) To: Tetsuya Mukawa; +Cc: dev > -----Original Message----- > From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Tetsuya Mukawa > Sent: Wednesday, October 14, 2015 3:35 AM > To: Kulasek, TomaszX; dev@dpdk.org > Subject: Re: [dpdk-dev] [PATCH v5 5/9] null: export eth_dev_null_create > > On 2015/09/30 23:05, Tomasz Kulasek wrote: > > Signed-off-by: Tomasz Kulasek <tomaszx.kulasek@intel.com> > > --- > > drivers/net/null/Makefile | 2 +- > > drivers/net/null/rte_eth_null.c | 2 +- > > drivers/net/null/rte_eth_null.h | 40 > +++++++++++++++++++++++++++++ > > drivers/net/null/rte_pmd_null_version.map | 7 +++++ > > 4 files changed, 49 insertions(+), 2 deletions(-) > > create mode 100644 drivers/net/null/rte_eth_null.h > Acked-by: Tetsuya Mukawa <mukawa@igel.co.jp> Hi Tetsuya, Thank You for a review and acking patches. There are 5 remaining patches in a patch-set: [dpdk-dev] [PATCH v5 1/9] bonding: rss dynamic configuration [dpdk-dev] [PATCH v5 6/9] test: dynamic rss configuration [dpdk-dev] [PATCH v5 7/9] bonding: per queue stats [dpdk-dev] [PATCH v5 8/9] doc: fixed spellings and typos [dpdk-dev] [PATCH v5 9/9] doc: dynamic rss configuration for bonding Could You also review them and ack if You agree with that implementation, please? ^ permalink raw reply [flat|nested] 126+ messages in thread
* Re: [dpdk-dev] [PATCH v5 5/9] null: export eth_dev_null_create 2015-10-14 12:42 ` Jastrzebski, MichalX K @ 2015-10-15 8:16 ` Tetsuya Mukawa 2015-10-15 10:35 ` Jastrzebski, MichalX K 0 siblings, 1 reply; 126+ messages in thread From: Tetsuya Mukawa @ 2015-10-15 8:16 UTC (permalink / raw) To: Jastrzebski, MichalX K; +Cc: dev On 2015/10/14 21:42, Jastrzebski, MichalX K wrote: >> -----Original Message----- >> From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Tetsuya Mukawa >> Sent: Wednesday, October 14, 2015 3:35 AM >> To: Kulasek, TomaszX; dev@dpdk.org >> Subject: Re: [dpdk-dev] [PATCH v5 5/9] null: export eth_dev_null_create >> >> On 2015/09/30 23:05, Tomasz Kulasek wrote: >>> Signed-off-by: Tomasz Kulasek <tomaszx.kulasek@intel.com> >>> --- >>> drivers/net/null/Makefile | 2 +- >>> drivers/net/null/rte_eth_null.c | 2 +- >>> drivers/net/null/rte_eth_null.h | 40 >> +++++++++++++++++++++++++++++ >>> drivers/net/null/rte_pmd_null_version.map | 7 +++++ >>> 4 files changed, 49 insertions(+), 2 deletions(-) >>> create mode 100644 drivers/net/null/rte_eth_null.h >> Acked-by: Tetsuya Mukawa <mukawa@igel.co.jp> > Hi Tetsuya, > Thank You for a review and acking patches. > There are 5 remaining patches in a patch-set: > > [dpdk-dev] [PATCH v5 1/9] bonding: rss dynamic configuration > [dpdk-dev] [PATCH v5 6/9] test: dynamic rss configuration > [dpdk-dev] [PATCH v5 7/9] bonding: per queue stats > [dpdk-dev] [PATCH v5 8/9] doc: fixed spellings and typos > [dpdk-dev] [PATCH v5 9/9] doc: dynamic rss configuration for bonding > > Could You also review them and ack if You agree with that implementation, please? Hi MichalX, I've checked above patches. But I am sorry that I am not a correct person to ack your patches. To review it correctly, at least knowledge about RSS and bonding PMD are needed, but honestly I am not familiar with RSS. Is it possible to find a correct person who can ack it? BTW, I've checked the patch from the following point of view. It may help other reviewers. (I've checked after fixing null PMD issues I've found today, please check an email I've sent to Tomasz) - No fatal errors are reported by checkpatch.pl. - Some style warnings are reported, but I guess we can ignore it. - No compile error with gcc-4.6/4.7/4.8/4.9/5, clang and icc. - No missing symbol. - No compile error for docs. - No compile error for examples. Thanks, Tetsuya ^ permalink raw reply [flat|nested] 126+ messages in thread
* Re: [dpdk-dev] [PATCH v5 5/9] null: export eth_dev_null_create 2015-10-15 8:16 ` Tetsuya Mukawa @ 2015-10-15 10:35 ` Jastrzebski, MichalX K 0 siblings, 0 replies; 126+ messages in thread From: Jastrzebski, MichalX K @ 2015-10-15 10:35 UTC (permalink / raw) To: Tetsuya Mukawa; +Cc: dev > -----Original Message----- > From: Tetsuya Mukawa [mailto:mukawa@igel.co.jp] > Sent: Thursday, October 15, 2015 10:17 AM > To: Jastrzebski, MichalX K > Cc: Glynn, Michael J; Kulasek, TomaszX; dev@dpdk.org > Subject: Re: [dpdk-dev] [PATCH v5 5/9] null: export eth_dev_null_create > > On 2015/10/14 21:42, Jastrzebski, MichalX K wrote: > >> -----Original Message----- > >> From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Tetsuya Mukawa > >> Sent: Wednesday, October 14, 2015 3:35 AM > >> To: Kulasek, TomaszX; dev@dpdk.org > >> Subject: Re: [dpdk-dev] [PATCH v5 5/9] null: export eth_dev_null_create > >> > >> On 2015/09/30 23:05, Tomasz Kulasek wrote: > >>> Signed-off-by: Tomasz Kulasek <tomaszx.kulasek@intel.com> > >>> --- > >>> drivers/net/null/Makefile | 2 +- > >>> drivers/net/null/rte_eth_null.c | 2 +- > >>> drivers/net/null/rte_eth_null.h | 40 > >> +++++++++++++++++++++++++++++ > >>> drivers/net/null/rte_pmd_null_version.map | 7 +++++ > >>> 4 files changed, 49 insertions(+), 2 deletions(-) > >>> create mode 100644 drivers/net/null/rte_eth_null.h > >> Acked-by: Tetsuya Mukawa <mukawa@igel.co.jp> > > Hi Tetsuya, > > Thank You for a review and acking patches. > > There are 5 remaining patches in a patch-set: > > > > [dpdk-dev] [PATCH v5 1/9] bonding: rss dynamic configuration > > [dpdk-dev] [PATCH v5 6/9] test: dynamic rss configuration > > [dpdk-dev] [PATCH v5 7/9] bonding: per queue stats > > [dpdk-dev] [PATCH v5 8/9] doc: fixed spellings and typos > > [dpdk-dev] [PATCH v5 9/9] doc: dynamic rss configuration for bonding > > > > Could You also review them and ack if You agree with that implementation, > please? > > Hi MichalX, > > I've checked above patches. But I am sorry that I am not a correct > person to ack your patches. > To review it correctly, at least knowledge about RSS and bonding PMD are > needed, but honestly I am not familiar with RSS. > Is it possible to find a correct person who can ack it? > > BTW, I've checked the patch from the following point of view. It may > help other reviewers. > (I've checked after fixing null PMD issues I've found today, please > check an email I've sent to Tomasz) > - No fatal errors are reported by checkpatch.pl. > - Some style warnings are reported, but I guess we can ignore it. > - No compile error with gcc-4.6/4.7/4.8/4.9/5, clang and icc. > - No missing symbol. > - No compile error for docs. > - No compile error for examples. > > Thanks, > Tetsuya Hi Tetsuya, Thank You very much for verifications and Your comments. I will ask maintainers in term of reviewing remaining patches. Regarding an issue with rte_eth_null.h I think that reordering of patches 4 and 5 may be sufficient. Best regards Michal ^ permalink raw reply [flat|nested] 126+ messages in thread
* [dpdk-dev] [PATCH v5 6/9] test: dynamic rss configuration 2015-09-30 14:04 ` [dpdk-dev] [PATCH v5 " Tomasz Kulasek ` (4 preceding siblings ...) 2015-09-30 14:05 ` [dpdk-dev] [PATCH v5 5/9] null: export eth_dev_null_create Tomasz Kulasek @ 2015-09-30 14:05 ` Tomasz Kulasek 2015-09-30 14:05 ` [dpdk-dev] [PATCH v5 7/9] bonding: per queue stats Tomasz Kulasek ` (3 subsequent siblings) 9 siblings, 0 replies; 126+ messages in thread From: Tomasz Kulasek @ 2015-09-30 14:05 UTC (permalink / raw) To: dev Signed-off-by: Tomasz Kulasek <tomaszx.kulasek@intel.com> --- app/test/Makefile | 8 + app/test/test_link_bonding_rssconf.c | 679 ++++++++++++++++++++++++++++++++++ 2 files changed, 687 insertions(+) create mode 100644 app/test/test_link_bonding_rssconf.c diff --git a/app/test/Makefile b/app/test/Makefile index 294618f..c122f28 100644 --- a/app/test/Makefile +++ b/app/test/Makefile @@ -138,6 +138,7 @@ SRCS-$(CONFIG_RTE_LIBRTE_ACL) += test_acl.c ifeq ($(CONFIG_RTE_LIBRTE_PMD_RING),y) SRCS-$(CONFIG_RTE_LIBRTE_PMD_BOND) += test_link_bonding.c SRCS-$(CONFIG_RTE_LIBRTE_PMD_BOND) += test_link_bonding_mode4.c +SRCS-$(CONFIG_RTE_LIBRTE_PMD_BOND) += test_link_bonding_rssconf.c endif SRCS-$(CONFIG_RTE_LIBRTE_PMD_RING) += test_pmd_ring.c @@ -168,6 +169,13 @@ ifeq ($(CONFIG_RTE_BUILD_SHARED_LIB),y) LDLIBS += -lrte_pmd_ring endif endif +ifneq ($(CONFIG_RTE_LIBRTE_PMD_NULL),y) +$(error Link bonding rssconf tests require CONFIG_RTE_LIBRTE_PMD_NULL=y) +else +ifeq ($(CONFIG_RTE_BUILD_SHARED_LIB),y) +LDLIBS += -lrte_pmd_null +endif +endif endif include $(RTE_SDK)/mk/rte.app.mk diff --git a/app/test/test_link_bonding_rssconf.c b/app/test/test_link_bonding_rssconf.c new file mode 100644 index 0000000..e6714b4 --- /dev/null +++ b/app/test/test_link_bonding_rssconf.c @@ -0,0 +1,679 @@ +/*- + * BSD LICENSE + * + * Copyright(c) 2015 Intel Corporation. All rights reserved. + * All rights reserved. + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * + * * Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * * Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in + * the documentation and/or other materials provided with the + * distribution. + * * Neither the name of Intel Corporation nor the names of its + * contributors may be used to endorse or promote products derived + * from this software without specific prior written permission. + * + * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS + * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT + * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR + * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT + * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, + * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT + * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, + * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY + * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT + * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE + * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + */ + +#include <string.h> +#include <stdarg.h> +#include <stdio.h> +#include <stdlib.h> +#include <stdint.h> +#include <inttypes.h> +#include <errno.h> +#include <rte_cycles.h> +#include <sys/queue.h> + +#include <rte_byteorder.h> +#include <rte_common.h> +#include <rte_debug.h> +#include <rte_ethdev.h> +#include <rte_log.h> +#include <rte_lcore.h> +#include <rte_memory.h> + +#include <rte_string_fns.h> +#include <rte_errno.h> +#include <rte_eth_bond.h> +#include <rte_eth_null.h> + +#include "test.h" + +#define SLAVE_COUNT (4) + +#define RXTX_RING_SIZE 1024 +#define RXTX_QUEUE_COUNT 4 + +#define BONDED_DEV_NAME ("rssconf_bond_dev") + +#define SLAVE_DEV_NAME_FMT ("rssconf_slave%d") +#define SLAVE_RXTX_QUEUE_FMT ("rssconf_slave%d_q%d") + +#define NUM_MBUFS 8191 +#define MBUF_SIZE (1600 + sizeof(struct rte_mbuf) + RTE_PKTMBUF_HEADROOM) +#define MBUF_CACHE_SIZE 250 +#define BURST_SIZE 32 + +#define INVALID_SOCKET_ID (-1) +#define INVALID_PORT_ID (0xFF) +#define INVALID_BONDING_MODE (-1) + +struct slave_conf { + uint8_t port_id; + struct rte_eth_dev_info dev_info; + + struct rte_eth_rss_conf rss_conf; + uint8_t rss_key[40]; + struct rte_eth_rss_reta_entry64 reta_conf[512 / RTE_RETA_GROUP_SIZE]; + + uint8_t is_slave; + struct rte_ring *rxtx_queue[RXTX_QUEUE_COUNT]; +}; + +struct link_bonding_rssconf_unittest_params { + uint8_t bond_port_id; + struct rte_eth_dev_info bond_dev_info; + struct rte_eth_rss_reta_entry64 bond_reta_conf[512 / RTE_RETA_GROUP_SIZE]; + struct slave_conf slave_ports[SLAVE_COUNT]; + + struct rte_mempool *mbuf_pool; +}; + +static struct link_bonding_rssconf_unittest_params test_params = { + .bond_port_id = INVALID_PORT_ID, + .slave_ports = { + [0 ... SLAVE_COUNT - 1] = { .port_id = INVALID_PORT_ID, .is_slave = 0} + }, + .mbuf_pool = NULL, +}; + +/** + * Default port configuration with RSS turned off + */ +static struct rte_eth_conf default_pmd_conf = { + .rxmode = { + .mq_mode = ETH_MQ_RX_NONE, + .max_rx_pkt_len = ETHER_MAX_LEN, + .split_hdr_size = 0, + .header_split = 0, /**< Header Split disabled */ + .hw_ip_checksum = 0, /**< IP checksum offload enabled */ + .hw_vlan_filter = 0, /**< VLAN filtering disabled */ + .jumbo_frame = 0, /**< Jumbo Frame Support disabled */ + .hw_strip_crc = 0, /**< CRC stripped by hardware */ + }, + .txmode = { + .mq_mode = ETH_MQ_TX_NONE, + }, + .lpbk_mode = 0, +}; + +static struct rte_eth_conf rss_pmd_conf = { + .rxmode = { + .mq_mode = ETH_MQ_RX_RSS, + .max_rx_pkt_len = ETHER_MAX_LEN, + .split_hdr_size = 0, + .header_split = 0, /**< Header Split disabled */ + .hw_ip_checksum = 0, /**< IP checksum offload enabled */ + .hw_vlan_filter = 0, /**< VLAN filtering disabled */ + .jumbo_frame = 0, /**< Jumbo Frame Support disabled */ + .hw_strip_crc = 0, /**< CRC stripped by hardware */ + }, + .txmode = { + .mq_mode = ETH_MQ_TX_NONE, + }, + .rx_adv_conf = { + .rss_conf = { + .rss_key = NULL, + .rss_hf = ETH_RSS_IPV6, + }, + }, + .lpbk_mode = 0, +}; + +#define FOR_EACH(_i, _item, _array, _size) \ + for (_i = 0, _item = &_array[0]; _i < _size && (_item = &_array[_i]); _i++) + +/* Macro for iterating over every port that can be used as a slave + * in this test. + * _i variable used as an index in test_params->slave_ports + * _slave pointer to &test_params->slave_ports[_idx] + */ +#define FOR_EACH_PORT(_i, _port) \ + FOR_EACH(_i, _port, test_params.slave_ports, \ + RTE_DIM(test_params.slave_ports)) + +static int +configure_ethdev(uint8_t port_id, struct rte_eth_conf *eth_conf, uint8_t start) +{ + int rxq, txq; + + TEST_ASSERT(rte_eth_dev_configure(port_id, RXTX_QUEUE_COUNT, + RXTX_QUEUE_COUNT, eth_conf) == 0, "Failed to configure device %u", + port_id); + + for (rxq = 0; rxq < RXTX_QUEUE_COUNT; rxq++) { + TEST_ASSERT(rte_eth_rx_queue_setup(port_id, rxq, RXTX_RING_SIZE, + rte_eth_dev_socket_id(port_id), NULL, + test_params.mbuf_pool) == 0, "Failed to setup rx queue."); + } + + for (txq = 0; txq < RXTX_QUEUE_COUNT; txq++) { + TEST_ASSERT(rte_eth_tx_queue_setup(port_id, txq, RXTX_RING_SIZE, + rte_eth_dev_socket_id(port_id), NULL) == 0, + "Failed to setup tx queue."); + } + + if (start) { + TEST_ASSERT(rte_eth_dev_start(port_id) == 0, + "Failed to start device (%d).", port_id); + } + + return 0; +} + +/** + * Remove all slaves from bonding + */ +static int +remove_slaves(void) +{ + unsigned n; + struct slave_conf *port; + + FOR_EACH_PORT(n, port) { + port = &test_params.slave_ports[n]; + if (port->is_slave) { + TEST_ASSERT_SUCCESS(rte_eth_bond_slave_remove( + test_params.bond_port_id, port->port_id), + "Cannot remove slave %d from bonding", port->port_id); + port->is_slave = 0; + } + } + + return 0; +} + +static int +remove_slaves_and_stop_bonded_device(void) +{ + TEST_ASSERT_SUCCESS(remove_slaves(), "Removing slaves"); + rte_eth_dev_stop(test_params.bond_port_id); + return TEST_SUCCESS; +} + +/** + * Add all slaves to bonding + */ +static int +bond_slaves(void) +{ + unsigned n; + struct slave_conf *port; + + FOR_EACH_PORT(n, port) { + port = &test_params.slave_ports[n]; + if (!port->is_slave) { + TEST_ASSERT_SUCCESS(rte_eth_bond_slave_add(test_params.bond_port_id, + port->port_id), "Cannot attach slave %d to the bonding", + port->port_id); + port->is_slave = 1; + } + } + + return 0; +} + +/** + * Set all RETA values in port_id to value + */ +static int +reta_set(uint8_t port_id, uint8_t value, int reta_size) +{ + struct rte_eth_rss_reta_entry64 reta_conf[512/RTE_RETA_GROUP_SIZE]; + int i, j; + + for (i = 0; i < reta_size / RTE_RETA_GROUP_SIZE; i++) { + /* select all fields to set */ + reta_conf[i].mask = ~0LL; + for (j = 0; j < RTE_RETA_GROUP_SIZE; j++) + reta_conf[i].reta[j] = value; + } + + return rte_eth_dev_rss_reta_update(port_id, reta_conf, reta_size); +} + +/** + * Check if slaves RETA is synchronized with bonding port. Returns 1 if slave + * port is synced with bonding port. + */ +static int +reta_check_synced(struct slave_conf *port) +{ + unsigned i; + + for (i = 0; i < test_params.bond_dev_info.reta_size; + i++) { + + int index = i / RTE_RETA_GROUP_SIZE; + int shift = i % RTE_RETA_GROUP_SIZE; + + if (port->reta_conf[index].reta[shift] != + test_params.bond_reta_conf[index].reta[shift]) + return 0; + + } + + return 1; +} + +/** + * Fetch bonding ports RETA + */ +static int +bond_reta_fetch(void) { + unsigned j; + + for (j = 0; j < test_params.bond_dev_info.reta_size / RTE_RETA_GROUP_SIZE; + j++) + test_params.bond_reta_conf[j].mask = ~0LL; + + TEST_ASSERT_SUCCESS(rte_eth_dev_rss_reta_query(test_params.bond_port_id, + test_params.bond_reta_conf, test_params.bond_dev_info.reta_size), + "Cannot take bonding ports RSS configuration"); + return 0; +} + +/** + * Fetch slaves RETA + */ +static int +slave_reta_fetch(struct slave_conf *port) { + unsigned j; + + for (j = 0; j < port->dev_info.reta_size / RTE_RETA_GROUP_SIZE; j++) + port->reta_conf[j].mask = ~0LL; + + TEST_ASSERT_SUCCESS(rte_eth_dev_rss_reta_query(port->port_id, + port->reta_conf, port->dev_info.reta_size), + "Cannot take bonding ports RSS configuration"); + return 0; +} + +/** + * Remove and add slave to check if slaves configuration is synced with + * the bonding ports values after adding new slave. + */ +static int +slave_remove_and_add(void) +{ + struct slave_conf *port = &(test_params.slave_ports[0]); + + /* 1. Remove first slave from bonding */ + TEST_ASSERT_SUCCESS(rte_eth_bond_slave_remove(test_params.bond_port_id, + port->port_id), "Cannot remove slave #d from bonding"); + + /* 2. Change removed (ex-)slave and bonding configuration to different + * values + */ + reta_set(test_params.bond_port_id, 1, test_params.bond_dev_info.reta_size); + bond_reta_fetch(); + + reta_set(port->port_id, 2, port->dev_info.reta_size); + slave_reta_fetch(port); + + TEST_ASSERT(reta_check_synced(port) == 0, + "Removed slave didn't should be synchronized with bonding port"); + + /* 3. Add (ex-)slave and check if configuration changed*/ + TEST_ASSERT_SUCCESS(rte_eth_bond_slave_add(test_params.bond_port_id, + port->port_id), "Cannot add slave"); + + bond_reta_fetch(); + slave_reta_fetch(port); + + return reta_check_synced(port); +} + +/** + * Test configuration propagation over slaves. + */ +static int +test_propagate(void) +{ + unsigned i; + uint8_t n; + struct slave_conf *port; + uint8_t bond_rss_key[40]; + struct rte_eth_rss_conf bond_rss_conf; + + int retval = 0; + uint64_t rss_hf = 0; + uint64_t default_rss_hf = 0; + + rte_eth_dev_info_get(test_params.bond_port_id, &test_params.bond_dev_info); + + /* + * Test hash function propagation + */ + for (i = 0; i < sizeof(test_params.bond_dev_info.flow_type_rss_offloads)*8; + i++) { + + rss_hf = test_params.bond_dev_info.flow_type_rss_offloads & (1<<i); + if (rss_hf) { + bond_rss_conf.rss_key = NULL; + bond_rss_conf.rss_hf = rss_hf; + + retval = rte_eth_dev_rss_hash_update(test_params.bond_port_id, + &bond_rss_conf); + TEST_ASSERT_SUCCESS(retval, "Cannot set slaves hash function"); + + FOR_EACH_PORT(n, port) { + port = &test_params.slave_ports[n]; + + retval = rte_eth_dev_rss_hash_conf_get(port->port_id, + &port->rss_conf); + TEST_ASSERT_SUCCESS(retval, + "Cannot take slaves RSS configuration"); + + TEST_ASSERT(port->rss_conf.rss_hf == rss_hf, + "Hash function not propagated for slave %d", + port->port_id); + } + + default_rss_hf = rss_hf; + } + + } + + /* + * Test key propagation + */ + for (i = 1; i < 10; i++) { + + /* Set all keys to zero */ + FOR_EACH_PORT(n, port) { + port = &test_params.slave_ports[n]; + memset(port->rss_conf.rss_key, 0, 40); + retval = rte_eth_dev_rss_hash_update(port->port_id, + &port->rss_conf); + TEST_ASSERT_SUCCESS(retval, "Cannot set slaves RSS keys"); + } + + memset(bond_rss_key, i, sizeof(bond_rss_key)); + bond_rss_conf.rss_hf = default_rss_hf, + bond_rss_conf.rss_key = bond_rss_key; + bond_rss_conf.rss_key_len = 40; + + retval = rte_eth_dev_rss_hash_update(test_params.bond_port_id, + &bond_rss_conf); + TEST_ASSERT_SUCCESS(retval, "Cannot set bonded port RSS keys"); + + FOR_EACH_PORT(n, port) { + port = &test_params.slave_ports[n]; + + retval = rte_eth_dev_rss_hash_conf_get(port->port_id, + &(port->rss_conf)); + + TEST_ASSERT_SUCCESS(retval, + "Cannot take slaves RSS configuration"); + + /* compare keys */ + retval = memcmp(port->rss_conf.rss_key, bond_rss_key, + sizeof(bond_rss_key)); + TEST_ASSERT(retval == 0, "Key value not propagated for slave %d", + port->port_id); + } + } + + /* + * Test RETA propagation + */ + for (i = 0; i < RXTX_QUEUE_COUNT; i++) { + + /* Set all keys to zero */ + FOR_EACH_PORT(n, port) { + port = &test_params.slave_ports[n]; + retval = reta_set(port->port_id, (i + 1) % RXTX_QUEUE_COUNT, + port->dev_info.reta_size); + TEST_ASSERT_SUCCESS(retval, "Cannot set slaves RETA"); + } + + TEST_ASSERT_SUCCESS(reta_set(test_params.bond_port_id, + i % RXTX_QUEUE_COUNT, test_params.bond_dev_info.reta_size), + "Cannot set bonded port RETA"); + + bond_reta_fetch(); + + FOR_EACH_PORT(n, port) { + port = &test_params.slave_ports[n]; + + slave_reta_fetch(port); + TEST_ASSERT(reta_check_synced(port) == 1, "RETAs inconsistent"); + } + } + + return TEST_SUCCESS; +} + +/** + * Test propagation logic, when RX_RSS mq_mode is turned on for bonding port + */ +static int +test_rss(void) +{ + /** + * Configure bonding port in RSS mq mode + */ + TEST_ASSERT_SUCCESS(configure_ethdev(test_params.bond_port_id, + &rss_pmd_conf, 0), "Failed to configure bonding device\n"); + + rte_eth_dev_info_get(test_params.bond_port_id, &test_params.bond_dev_info); + + TEST_ASSERT_SUCCESS(bond_slaves(), "Bonding slaves failed"); + + TEST_ASSERT_SUCCESS(rte_eth_dev_start(test_params.bond_port_id), + "Failed to start bonding port (%d).", test_params.bond_port_id); + + TEST_ASSERT_SUCCESS(test_propagate(), "Propagation test failed"); + + TEST_ASSERT(slave_remove_and_add() == 1, "New slave should be synced"); + + remove_slaves_and_stop_bonded_device(); + + return TEST_SUCCESS; +} + +/** + * Test propagation logic, when RX_RSS mq_mode is turned off for bonding port + */ +static int +test_rss_lazy(void) +{ + TEST_ASSERT_SUCCESS(configure_ethdev(test_params.bond_port_id, + &default_pmd_conf, 0), "Failed to configure bonding device\n"); + + rte_eth_dev_info_get(test_params.bond_port_id, &test_params.bond_dev_info); + + TEST_ASSERT_SUCCESS(bond_slaves(), "Bonding slaves failed"); + + TEST_ASSERT_SUCCESS(rte_eth_dev_start(test_params.bond_port_id), + "Failed to start bonding port (%d).", test_params.bond_port_id); + + TEST_ASSERT_SUCCESS(test_propagate(), "Propagation test failed"); + + TEST_ASSERT(slave_remove_and_add() == 0, "New slave shouldn't be synced"); + + remove_slaves_and_stop_bonded_device(); + + return TEST_SUCCESS; +} + +static int +test_setup(void) +{ + unsigned n; + int retval; + int port_id; + char name[256]; + struct slave_conf *port; + + if (test_params.mbuf_pool == NULL) { + + test_params.mbuf_pool = rte_mempool_create("RSS_MBUF_POOL", NUM_MBUFS * + SLAVE_COUNT, MBUF_SIZE, MBUF_CACHE_SIZE, + sizeof(struct rte_pktmbuf_pool_private), rte_pktmbuf_pool_init, + NULL, rte_pktmbuf_init, NULL, rte_socket_id(), 0); + + TEST_ASSERT(test_params.mbuf_pool != NULL, + "rte_mempool_create failed\n"); + } + + /* Create / initialize ring eth devs. */ + FOR_EACH_PORT(n, port) { + port = &test_params.slave_ports[n]; + + port_id = rte_eth_dev_count(); + snprintf(name, sizeof(name), SLAVE_DEV_NAME_FMT, port_id); + + retval = eth_dev_null_create(name, 0, 64, 0); + TEST_ASSERT_SUCCESS(retval, "Failed to create null device '%s'\n", + name); + + port->port_id = port_id; + + port->rss_conf.rss_key = port->rss_key; + port->rss_conf.rss_key_len = 40; + + retval = configure_ethdev(port->port_id, &default_pmd_conf, 0); + TEST_ASSERT_SUCCESS(retval, "Failed to configure virtual ethdev %s\n", + name); + + rte_eth_dev_info_get(port->port_id, &port->dev_info); + } + + if (test_params.bond_port_id == INVALID_PORT_ID) { + retval = rte_eth_bond_create(BONDED_DEV_NAME, 0, rte_socket_id()); + + TEST_ASSERT(retval >= 0, "Failed to create bonded ethdev %s", + BONDED_DEV_NAME); + + test_params.bond_port_id = retval; + + TEST_ASSERT_SUCCESS(configure_ethdev(test_params.bond_port_id, + &default_pmd_conf, 0), "Failed to configure bonding device\n"); + + rte_eth_dev_info_get(test_params.bond_port_id, + &test_params.bond_dev_info); + } + + return TEST_SUCCESS; +} + +static int +testsuite_teardown(void) +{ + struct slave_conf *port; + uint8_t i; + + /* Only stop ports. + * Any cleanup/reset state is done when particular test is + * started. */ + + rte_eth_dev_stop(test_params.bond_port_id); + + FOR_EACH_PORT(i, port) + rte_eth_dev_stop(port->port_id); + + return 0; +} + +static int +check_environment(void) +{ + return TEST_SUCCESS; +} + +static int +test_rssconf_executor(int (*test_func)(void)) +{ + int test_result; + + /* Check if environment is clean. Fail to launch a test if there was + * a critical error before that prevented to reset environment. */ + TEST_ASSERT_SUCCESS(check_environment(), + "Refusing to launch test in dirty environment."); + + RTE_VERIFY(test_func != NULL); + test_result = (*test_func)(); + + /* If test succeed check if environment wast left in good condition. */ + if (test_result == TEST_SUCCESS) + test_result = check_environment(); + + /* Reset environment in case test failed to do that. */ + if (test_result != TEST_SUCCESS) { + TEST_ASSERT_SUCCESS(remove_slaves_and_stop_bonded_device(), + "Failed to stop bonded device"); + } + + return test_result; +} + +static int +test_setup_wrapper(void) +{ + return test_rssconf_executor(&test_setup); +} + +static int +test_rss_wrapper(void) +{ + return test_rssconf_executor(&test_rss); +} + +static int +test_rss_lazy_wrapper(void) +{ + return test_rssconf_executor(&test_rss_lazy); +} + +static struct unit_test_suite link_bonding_rssconf_test_suite = { + .suite_name = "RSS Dynamic Configuration for Bonding Unit Test Suite", + .teardown = testsuite_teardown, + .unit_test_cases = { + TEST_CASE_NAMED("test_setup", test_setup_wrapper), + TEST_CASE_NAMED("test_rss", test_rss_wrapper), + TEST_CASE_NAMED("test_rss_lazy", test_rss_lazy_wrapper), + { NULL, NULL, NULL, NULL, NULL } /**< NULL terminate unit test array */ + } +}; + +static int +test_link_bonding_rssconf(void) +{ + return unit_test_suite_runner(&link_bonding_rssconf_test_suite); +} + +static struct test_command test_link_bonding_rssconf_cmd = { + .command = "link_bonding_rssconf_autotest", + .callback = test_link_bonding_rssconf, +}; + +REGISTER_TEST_COMMAND(test_link_bonding_rssconf_cmd); -- 1.7.9.5 ^ permalink raw reply [flat|nested] 126+ messages in thread
* [dpdk-dev] [PATCH v5 7/9] bonding: per queue stats 2015-09-30 14:04 ` [dpdk-dev] [PATCH v5 " Tomasz Kulasek ` (5 preceding siblings ...) 2015-09-30 14:05 ` [dpdk-dev] [PATCH v5 6/9] test: dynamic rss configuration Tomasz Kulasek @ 2015-09-30 14:05 ` Tomasz Kulasek 2015-09-30 14:05 ` [dpdk-dev] [PATCH v5 8/9] doc: fixed spellings and typos Tomasz Kulasek ` (2 subsequent siblings) 9 siblings, 0 replies; 126+ messages in thread From: Tomasz Kulasek @ 2015-09-30 14:05 UTC (permalink / raw) To: dev This patch fills bonding port's stats with a sum of corresponding values taken from bonded slaves, when stats are requested for bonding port. v5 changes: - removed queue_stats_mapping_set from eth_dev_ops of bonding device Signed-off-by: Tomasz Kulasek <tomaszx.kulasek@intel.com> --- drivers/net/bonding/rte_eth_bond_pmd.c | 11 ++++++++++- 1 file changed, 10 insertions(+), 1 deletion(-) diff --git a/drivers/net/bonding/rte_eth_bond_pmd.c b/drivers/net/bonding/rte_eth_bond_pmd.c index 2880f5c..eecb381 100644 --- a/drivers/net/bonding/rte_eth_bond_pmd.c +++ b/drivers/net/bonding/rte_eth_bond_pmd.c @@ -1801,7 +1801,7 @@ bond_ethdev_stats_get(struct rte_eth_dev *dev, struct rte_eth_stats *stats) { struct bond_dev_private *internals = dev->data->dev_private; struct rte_eth_stats slave_stats; - int i; + int i, j; for (i = 0; i < internals->slave_count; i++) { rte_eth_stats_get(internals->slaves[i].port_id, &slave_stats); @@ -1820,6 +1820,15 @@ bond_ethdev_stats_get(struct rte_eth_dev *dev, struct rte_eth_stats *stats) stats->rx_pause_xon += slave_stats.rx_pause_xon; stats->tx_pause_xoff += slave_stats.tx_pause_xoff; stats->rx_pause_xoff += slave_stats.rx_pause_xoff; + + for (j = 0; j < RTE_ETHDEV_QUEUE_STAT_CNTRS; j++) { + stats->q_ipackets[j] += slave_stats.q_ipackets[j]; + stats->q_opackets[j] += slave_stats.q_opackets[j]; + stats->q_ibytes[j] += slave_stats.q_ibytes[j]; + stats->q_obytes[j] += slave_stats.q_obytes[j]; + stats->q_errors[j] += slave_stats.q_errors[j]; + } + } } -- 1.7.9.5 ^ permalink raw reply [flat|nested] 126+ messages in thread
* [dpdk-dev] [PATCH v5 8/9] doc: fixed spellings and typos 2015-09-30 14:04 ` [dpdk-dev] [PATCH v5 " Tomasz Kulasek ` (6 preceding siblings ...) 2015-09-30 14:05 ` [dpdk-dev] [PATCH v5 7/9] bonding: per queue stats Tomasz Kulasek @ 2015-09-30 14:05 ` Tomasz Kulasek 2015-09-30 14:05 ` [dpdk-dev] [PATCH v5 9/9] doc: dynamic rss configuration for bonding Tomasz Kulasek 2015-10-16 10:00 ` [dpdk-dev] [PATCH v6 0/9] Dynamic RSS Configuration for Bonding Tomasz Kulasek 9 siblings, 0 replies; 126+ messages in thread From: Tomasz Kulasek @ 2015-09-30 14:05 UTC (permalink / raw) To: dev Signed-off-by: Tomasz Kulasek <tomaszx.kulasek@intel.com> --- .../prog_guide/link_bonding_poll_mode_drv_lib.rst | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/doc/guides/prog_guide/link_bonding_poll_mode_drv_lib.rst b/doc/guides/prog_guide/link_bonding_poll_mode_drv_lib.rst index 96e554f..03baf90 100644 --- a/doc/guides/prog_guide/link_bonding_poll_mode_drv_lib.rst +++ b/doc/guides/prog_guide/link_bonding_poll_mode_drv_lib.rst @@ -188,7 +188,7 @@ conditions are not met. If a user wishes to monitor individual slaves then they must register callbacks with that slave directly. The link bonding library also supports devices which do not implement link -status change interrupts, this is achieve by polling the devices link status at +status change interrupts, this is achieved by polling the devices link status at a defined period which is set using the ``rte_eth_bond_link_monitoring_set`` API, the default polling interval is 10ms. When a device is added as a slave to a bonding device it is determined using the ``RTE_PCI_DRV_INTR_LSC`` flag @@ -286,7 +286,7 @@ and UDP protocols for load balancing. Using Link Bonding Devices -------------------------- -The librte_pmd_bond library support two modes of device creation, the libraries +The librte_pmd_bond library supports two modes of device creation, the libraries export full C API or using the EAL command line to statically configure link bonding devices at application startup. Using the EAL option it is possible to use link bonding functionality transparently without specific knowledge of the @@ -299,7 +299,7 @@ Using the Poll Mode Driver from an Application Using the librte_pmd_bond libraries API it is possible to dynamically create and manage link bonding device from within any application. Link bonding -device are created using the ``rte_eth_bond_create`` API which requires a +devices are created using the ``rte_eth_bond_create`` API which requires a unique device name, the link bonding mode to initial the device in and finally the socket Id which to allocate the devices resources onto. After successful creation of a bonding device it must be configured using the generic Ethernet @@ -362,7 +362,7 @@ The different options are: mode=2 * slave: Defines the PMD device which will be added as slave to the bonded - device. This option can be selected multiple time, for each device to be + device. This option can be selected multiple times, for each device to be added as a slave. Physical devices should be specified using their PCI address, in the format domain:bus:devid.function -- 1.7.9.5 ^ permalink raw reply [flat|nested] 126+ messages in thread
* [dpdk-dev] [PATCH v5 9/9] doc: dynamic rss configuration for bonding 2015-09-30 14:04 ` [dpdk-dev] [PATCH v5 " Tomasz Kulasek ` (7 preceding siblings ...) 2015-09-30 14:05 ` [dpdk-dev] [PATCH v5 8/9] doc: fixed spellings and typos Tomasz Kulasek @ 2015-09-30 14:05 ` Tomasz Kulasek 2015-10-16 10:00 ` [dpdk-dev] [PATCH v6 0/9] Dynamic RSS Configuration for Bonding Tomasz Kulasek 9 siblings, 0 replies; 126+ messages in thread From: Tomasz Kulasek @ 2015-09-30 14:05 UTC (permalink / raw) To: dev Documentation update about implementation details and requirements for Dynamic RSS Configuration for Bonding. Signed-off-by: Tomasz Kulasek <tomaszx.kulasek@intel.com> --- .../prog_guide/link_bonding_poll_mode_drv_lib.rst | 34 ++++++++++++++++++-- 1 file changed, 32 insertions(+), 2 deletions(-) diff --git a/doc/guides/prog_guide/link_bonding_poll_mode_drv_lib.rst b/doc/guides/prog_guide/link_bonding_poll_mode_drv_lib.rst index 03baf90..46f0296 100644 --- a/doc/guides/prog_guide/link_bonding_poll_mode_drv_lib.rst +++ b/doc/guides/prog_guide/link_bonding_poll_mode_drv_lib.rst @@ -1,5 +1,5 @@ .. BSD LICENSE - Copyright(c) 2010-2014 Intel Corporation. All rights reserved. + Copyright(c) 2010-2015 Intel Corporation. All rights reserved. All rights reserved. Redistribution and use in source and binary forms, with or without @@ -173,7 +173,28 @@ After a slave device is added to a bonded device slave is stopped using ``rte_eth_dev_stop`` and then reconfigured using ``rte_eth_dev_configure`` the RX and TX queues are also reconfigured using ``rte_eth_tx_queue_setup`` / ``rte_eth_rx_queue_setup`` with the parameters use to configure the bonding -device. +device. If RSS is enabled for bonding device, this mode is also enabled on new +slave and configured as well. + +Setting up multi-queue mode for bonding device to RSS, makes it fully +RSS-capable, so all slaves are synchronized with its configuration. This mode is +intended to provide RSS configuration on slaves transparent for client +application implementation. + +Bonding device stores its own version of RSS settings i.e. RETA, RSS hash +function and RSS key, used to set up its slaves. That let to define the meaning +of RSS configuration of bonding device as desired configuration of whole bonding +(as one unit), without pointing any of slave inside. It is required to ensure +consistency and made it more errorproof. + +RSS hash function set for bonding device, is a maximal set of RSS hash functions +supported by all bonded slaves. RETA size is a GCD of all its RETA's sizes, so +it can be easily used as a pattern providing expected behavior, even if slave +RETAs' sizes are different. If RSS Key is not set for bonded device, it's not +changed on the slaves and default key for device is used. + +All settings are managed through the bonding port API and always are propagated +in one direction (from bonding to slaves). Link Status Change Interrupts / Polling ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ @@ -207,6 +228,15 @@ these parameters. A bonding device must have a minimum of one slave before the bonding device itself can be started. +To use a bonding device dynamic RSS configuration feature effectively, it is +also required, that all slaves should be RSS-capable and support, at least one +common hash function available for each of them. Changing RSS key is only +possible, when all slave devices support the same key size. + +To prevent inconsistency on how slaves process packets, once a device is added +to a bonding device, RSS configuration should be managed through the bonding +device API, and not directly on the slave. + Like all other PMD, all functions exported by a PMD are lock-free functions that are assumed not to be invoked in parallel on different logical cores to work on the same target object. -- 1.7.9.5 ^ permalink raw reply [flat|nested] 126+ messages in thread
* [dpdk-dev] [PATCH v6 0/9] Dynamic RSS Configuration for Bonding 2015-09-30 14:04 ` [dpdk-dev] [PATCH v5 " Tomasz Kulasek ` (8 preceding siblings ...) 2015-09-30 14:05 ` [dpdk-dev] [PATCH v5 9/9] doc: dynamic rss configuration for bonding Tomasz Kulasek @ 2015-10-16 10:00 ` Tomasz Kulasek 2015-10-16 10:00 ` [dpdk-dev] [PATCH v6 1/9] bonding: rss dynamic configuration Tomasz Kulasek ` (10 more replies) 9 siblings, 11 replies; 126+ messages in thread From: Tomasz Kulasek @ 2015-10-16 10:00 UTC (permalink / raw) To: dev OVERVIEW -------- 1) Setting .rxmode.mq_mode for bonding device to ETH_MQ_RX_RSS makes bonding device fully RSS-capable, so all slaves are synchronized with its configuration. This mode is intended to provide RSS configuration as known from "dynamic RSS configuration for one port" and made slaves transparent for client application implementation. 2) If .rxmode.mq_mode for bonding device isn't ETH_MQ_RX_RSS, slaves are not synchronized. That provides an ability to configure them manually. This mode may be useful when application wants to manage RSS in an unusual way and the consistency of RSS configuration for slaves isn't required. Turning on/off RSS mode for slaves when bonding is started is not possible. Other RSS configuration is propagated over slaves, when bonding device API is used to do it. v6 changes: - patchset reordered - fixed forward dependency between patch 4/9 and 5/9 v5 changes: - updated to DPDK 2.2 - removed copyright change from null device source - removed queue_stats_mapping_set from eth_dev_ops of bonding device - null pmd cleanups (removed unnecessary malloc, replaced memcpy with rte_memcpy) - fixed queues number configuration in null pmd v4 changes: - fixed copy-paste error, - removed example application as too complex and introducing a new dependency, - addapted null pmd to be used as testing device for dynamic RSS configuration, - addapted test units to use null pmd instead of ring pmd, - ring pmd is not used and changed in this patchset v3 changes: - checkpatch cleanups v2 changes: - added support for keys other than 40 bytes long, - now, if RSS key is not set for bonding, it is not set also for slaves, - fix - full initial RSS configuration before any slave is added was not possible due to the initially zeroed flow_type_rss_offloads for bonding, - fix - changed error to warning when slave is synchronizing due to the bonding's initial configuration (to allow use slaves' drivers not supporting dynamic RSS configuration in bonding), - some code cleanups, - updated documentation, Tomasz Kulasek (9): bonding: rss dynamic configuration null: fix segfault when null_pmd added to bonding null: extend number of virtual queues null: export eth_dev_null_create null: virtual dynamic rss configuration test: dynamic rss configuration bonding: per queue stats doc: fixed spellings and typos doc: dynamic rss configuration for bonding app/test/Makefile | 8 + app/test/test_link_bonding_rssconf.c | 679 ++++++++++++++++++++ .../prog_guide/link_bonding_poll_mode_drv_lib.rst | 42 +- drivers/net/bonding/rte_eth_bond_api.c | 28 + drivers/net/bonding/rte_eth_bond_pmd.c | 216 ++++++- drivers/net/bonding/rte_eth_bond_private.h | 12 + drivers/net/null/Makefile | 2 +- drivers/net/null/rte_eth_null.c | 149 ++++- drivers/net/null/rte_eth_null.h | 40 ++ drivers/net/null/rte_pmd_null_version.map | 7 + 10 files changed, 1151 insertions(+), 32 deletions(-) create mode 100644 app/test/test_link_bonding_rssconf.c create mode 100644 drivers/net/null/rte_eth_null.h -- 1.7.9.5 ^ permalink raw reply [flat|nested] 126+ messages in thread
* [dpdk-dev] [PATCH v6 1/9] bonding: rss dynamic configuration 2015-10-16 10:00 ` [dpdk-dev] [PATCH v6 0/9] Dynamic RSS Configuration for Bonding Tomasz Kulasek @ 2015-10-16 10:00 ` Tomasz Kulasek 2015-10-16 10:00 ` [dpdk-dev] [PATCH v6 2/9] null: fix segfault when null_pmd added to bonding Tomasz Kulasek ` (9 subsequent siblings) 10 siblings, 0 replies; 126+ messages in thread From: Tomasz Kulasek @ 2015-10-16 10:00 UTC (permalink / raw) To: dev Bonding device implements independent management of RSS settings. It stores its own copies of settings i.e. RETA, RSS hash function and RSS key. It’s required to ensure consistency. 1) RSS hash function set for bonding device is maximal set of RSS hash functions supported by all bonded devices. That mean, to have RSS support for bonding, all slaves should be RSS-capable. 2) RSS key is propagated over the slaves "as is". 3) RETA for bonding is an internal table managed by bonding API, and is used as a pattern to set up slaves. Its size is GCD of all RETA sizes, so it can be easily used as a pattern providing expected behavior, even if slaves RETA sizes are different. Signed-off-by: Tomasz Kulasek <tomaszx.kulasek@intel.com> --- drivers/net/bonding/rte_eth_bond_api.c | 28 ++++ drivers/net/bonding/rte_eth_bond_pmd.c | 205 ++++++++++++++++++++++++++-- drivers/net/bonding/rte_eth_bond_private.h | 12 ++ 3 files changed, 231 insertions(+), 14 deletions(-) diff --git a/drivers/net/bonding/rte_eth_bond_api.c b/drivers/net/bonding/rte_eth_bond_api.c index 0681d1a..92073df 100644 --- a/drivers/net/bonding/rte_eth_bond_api.c +++ b/drivers/net/bonding/rte_eth_bond_api.c @@ -273,6 +273,9 @@ rte_eth_bond_create(const char *name, uint8_t mode, uint8_t socket_id) internals->rx_offload_capa = 0; internals->tx_offload_capa = 0; + /* Initially allow to choose any offload type */ + internals->flow_type_rss_offloads = ETH_RSS_PROTO_MASK; + memset(internals->active_slaves, 0, sizeof(internals->active_slaves)); memset(internals->slaves, 0, sizeof(internals->slaves)); @@ -369,6 +372,11 @@ __eth_bond_slave_add_lock_free(uint8_t bonded_port_id, uint8_t slave_port_id) rte_eth_dev_info_get(slave_port_id, &dev_info); + /* We need to store slaves reta_size to be able to synchronize RETA for all + * slave devices even if its sizes are different. + */ + internals->slaves[internals->slave_count].reta_size = dev_info.reta_size; + if (internals->slave_count < 1) { /* if MAC is not user defined then use MAC of first slave add to * bonded device */ @@ -382,9 +390,16 @@ __eth_bond_slave_add_lock_free(uint8_t bonded_port_id, uint8_t slave_port_id) /* Make primary slave */ internals->primary_port = slave_port_id; + /* Inherit queues settings from first slave */ + internals->nb_rx_queues = slave_eth_dev->data->nb_rx_queues; + internals->nb_tx_queues = slave_eth_dev->data->nb_tx_queues; + + internals->reta_size = dev_info.reta_size; + /* Take the first dev's offload capabilities */ internals->rx_offload_capa = dev_info.rx_offload_capa; internals->tx_offload_capa = dev_info.tx_offload_capa; + internals->flow_type_rss_offloads = dev_info.flow_type_rss_offloads; } else { /* Check slave link properties are supported if props are set, @@ -403,8 +418,19 @@ __eth_bond_slave_add_lock_free(uint8_t bonded_port_id, uint8_t slave_port_id) } internals->rx_offload_capa &= dev_info.rx_offload_capa; internals->tx_offload_capa &= dev_info.tx_offload_capa; + internals->flow_type_rss_offloads &= dev_info.flow_type_rss_offloads; + + /* RETA size is GCD of all slaves RETA sizes, so, if all sizes will be + * the power of 2, the lower one is GCD + */ + if (internals->reta_size > dev_info.reta_size) + internals->reta_size = dev_info.reta_size; + } + bonded_eth_dev->data->dev_conf.rx_adv_conf.rss_conf.rss_hf &= + internals->flow_type_rss_offloads; + internals->slave_count++; /* Update all slave devices MACs*/ @@ -531,6 +557,8 @@ __eth_bond_slave_remove_lock_free(uint8_t bonded_port_id, uint8_t slave_port_id) if (internals->slave_count == 0) { internals->rx_offload_capa = 0; internals->tx_offload_capa = 0; + internals->flow_type_rss_offloads = ETH_RSS_PROTO_MASK; + internals->reta_size = 0; } return 0; } diff --git a/drivers/net/bonding/rte_eth_bond_pmd.c b/drivers/net/bonding/rte_eth_bond_pmd.c index 5cc6372..2880f5c 100644 --- a/drivers/net/bonding/rte_eth_bond_pmd.c +++ b/drivers/net/bonding/rte_eth_bond_pmd.c @@ -1310,6 +1310,23 @@ slave_configure(struct rte_eth_dev *bonded_eth_dev, if (slave_eth_dev->driver->pci_drv.drv_flags & RTE_PCI_DRV_INTR_LSC) slave_eth_dev->data->dev_conf.intr_conf.lsc = 1; + /* If RSS is enabled for bonding, try to enable it for slaves */ + if (bonded_eth_dev->data->dev_conf.rxmode.mq_mode & ETH_MQ_RX_RSS) { + if (bonded_eth_dev->data->dev_conf.rx_adv_conf.rss_conf.rss_key_len + != 0) { + slave_eth_dev->data->dev_conf.rx_adv_conf.rss_conf.rss_key_len = + bonded_eth_dev->data->dev_conf.rx_adv_conf.rss_conf.rss_key_len; + slave_eth_dev->data->dev_conf.rx_adv_conf.rss_conf.rss_key = + bonded_eth_dev->data->dev_conf.rx_adv_conf.rss_conf.rss_key; + } else { + slave_eth_dev->data->dev_conf.rx_adv_conf.rss_conf.rss_key = NULL; + } + + slave_eth_dev->data->dev_conf.rx_adv_conf.rss_conf.rss_hf = + bonded_eth_dev->data->dev_conf.rx_adv_conf.rss_conf.rss_hf; + slave_eth_dev->data->dev_conf.rxmode.mq_mode |= ETH_MQ_RX_RSS; + } + /* Configure device */ errval = rte_eth_dev_configure(slave_eth_dev->data->port_id, bonded_eth_dev->data->nb_rx_queues, @@ -1361,6 +1378,30 @@ slave_configure(struct rte_eth_dev *bonded_eth_dev, return -1; } + /* If RSS is enabled for bonding, synchronize RETA */ + if (bonded_eth_dev->data->dev_conf.rxmode.mq_mode & ETH_MQ_RX_RSS) { + int i; + struct bond_dev_private *internals; + + internals = bonded_eth_dev->data->dev_private; + + for (i = 0; i < internals->slave_count; i++) { + if (internals->slaves[i].port_id == slave_eth_dev->data->port_id) { + errval = rte_eth_dev_rss_reta_update( + slave_eth_dev->data->port_id, + &internals->reta_conf[0], + internals->slaves[i].reta_size); + if (errval != 0) { + RTE_LOG(WARNING, PMD, + "rte_eth_dev_rss_reta_update on slave port %d fails (err %d)." + " RSS Configuration for bonding may be inconsistent.\n", + slave_eth_dev->data->port_id, errval); + } + break; + } + } + } + /* If lsc interrupt is set, check initial slave's link status */ if (slave_eth_dev->driver->pci_drv.drv_flags & RTE_PCI_DRV_INTR_LSC) bond_ethdev_lsc_event_callback(slave_eth_dev->data->port_id, @@ -1596,6 +1637,9 @@ bond_ethdev_info(struct rte_eth_dev *dev, struct rte_eth_dev_info *dev_info) dev_info->rx_offload_capa = internals->rx_offload_capa; dev_info->tx_offload_capa = internals->tx_offload_capa; + dev_info->flow_type_rss_offloads = internals->flow_type_rss_offloads; + + dev_info->reta_size = internals->reta_size; } static int @@ -1977,21 +2021,132 @@ bond_ethdev_lsc_event_callback(uint8_t port_id, enum rte_eth_event_type type, } } +static int +bond_ethdev_rss_reta_update(struct rte_eth_dev *dev, + struct rte_eth_rss_reta_entry64 *reta_conf, uint16_t reta_size) +{ + unsigned i, j; + int result = 0; + int slave_reta_size; + unsigned reta_count; + struct bond_dev_private *internals = dev->data->dev_private; + + if (reta_size != internals->reta_size) + return -EINVAL; + + /* Copy RETA table */ + reta_count = reta_size / RTE_RETA_GROUP_SIZE; + + for (i = 0; i < reta_count; i++) { + internals->reta_conf[i].mask = reta_conf[i].mask; + for (j = 0; j < RTE_RETA_GROUP_SIZE; j++) + if ((reta_conf[i].mask >> j) & 0x01) + internals->reta_conf[i].reta[j] = reta_conf[i].reta[j]; + } + + /* Fill rest of array */ + for (; i < RTE_DIM(internals->reta_conf); i += reta_count) + memcpy(&internals->reta_conf[i], &internals->reta_conf[0], + sizeof(internals->reta_conf[0]) * reta_count); + + /* Propagate RETA over slaves */ + for (i = 0; i < internals->slave_count; i++) { + slave_reta_size = internals->slaves[i].reta_size; + result = rte_eth_dev_rss_reta_update(internals->slaves[i].port_id, + &internals->reta_conf[0], slave_reta_size); + if (result < 0) + return result; + } + + return 0; +} + +static int +bond_ethdev_rss_reta_query(struct rte_eth_dev *dev, + struct rte_eth_rss_reta_entry64 *reta_conf, uint16_t reta_size) +{ + int i, j; + struct bond_dev_private *internals = dev->data->dev_private; + + if (reta_size != internals->reta_size) + return -EINVAL; + + /* Copy RETA table */ + for (i = 0; i < reta_size / RTE_RETA_GROUP_SIZE; i++) + for (j = 0; j < RTE_RETA_GROUP_SIZE; j++) + if ((reta_conf[i].mask >> j) & 0x01) + reta_conf[i].reta[j] = internals->reta_conf[i].reta[j]; + + return 0; +} + +static int +bond_ethdev_rss_hash_update(struct rte_eth_dev *dev, + struct rte_eth_rss_conf *rss_conf) +{ + int i, result = 0; + struct bond_dev_private *internals = dev->data->dev_private; + struct rte_eth_rss_conf bond_rss_conf; + + memcpy(&bond_rss_conf, rss_conf, sizeof(struct rte_eth_rss_conf)); + + bond_rss_conf.rss_hf &= internals->flow_type_rss_offloads; + + if (bond_rss_conf.rss_hf != 0) + dev->data->dev_conf.rx_adv_conf.rss_conf.rss_hf = bond_rss_conf.rss_hf; + + if (bond_rss_conf.rss_key && bond_rss_conf.rss_key_len < + sizeof(internals->rss_key)) { + if (bond_rss_conf.rss_key_len == 0) + bond_rss_conf.rss_key_len = 40; + internals->rss_key_len = bond_rss_conf.rss_key_len; + memcpy(internals->rss_key, bond_rss_conf.rss_key, + internals->rss_key_len); + } + + for (i = 0; i < internals->slave_count; i++) { + result = rte_eth_dev_rss_hash_update(internals->slaves[i].port_id, + &bond_rss_conf); + if (result < 0) + return result; + } + + return 0; +} + +static int +bond_ethdev_rss_hash_conf_get(struct rte_eth_dev *dev, + struct rte_eth_rss_conf *rss_conf) +{ + struct bond_dev_private *internals = dev->data->dev_private; + + rss_conf->rss_hf = dev->data->dev_conf.rx_adv_conf.rss_conf.rss_hf; + rss_conf->rss_key_len = internals->rss_key_len; + if (rss_conf->rss_key) + memcpy(rss_conf->rss_key, internals->rss_key, internals->rss_key_len); + + return 0; +} + struct eth_dev_ops default_dev_ops = { - .dev_start = bond_ethdev_start, - .dev_stop = bond_ethdev_stop, - .dev_close = bond_ethdev_close, - .dev_configure = bond_ethdev_configure, - .dev_infos_get = bond_ethdev_info, - .rx_queue_setup = bond_ethdev_rx_queue_setup, - .tx_queue_setup = bond_ethdev_tx_queue_setup, - .rx_queue_release = bond_ethdev_rx_queue_release, - .tx_queue_release = bond_ethdev_tx_queue_release, - .link_update = bond_ethdev_link_update, - .stats_get = bond_ethdev_stats_get, - .stats_reset = bond_ethdev_stats_reset, - .promiscuous_enable = bond_ethdev_promiscuous_enable, - .promiscuous_disable = bond_ethdev_promiscuous_disable + .dev_start = bond_ethdev_start, + .dev_stop = bond_ethdev_stop, + .dev_close = bond_ethdev_close, + .dev_configure = bond_ethdev_configure, + .dev_infos_get = bond_ethdev_info, + .rx_queue_setup = bond_ethdev_rx_queue_setup, + .tx_queue_setup = bond_ethdev_tx_queue_setup, + .rx_queue_release = bond_ethdev_rx_queue_release, + .tx_queue_release = bond_ethdev_tx_queue_release, + .link_update = bond_ethdev_link_update, + .stats_get = bond_ethdev_stats_get, + .stats_reset = bond_ethdev_stats_reset, + .promiscuous_enable = bond_ethdev_promiscuous_enable, + .promiscuous_disable = bond_ethdev_promiscuous_disable, + .reta_update = bond_ethdev_rss_reta_update, + .reta_query = bond_ethdev_rss_reta_query, + .rss_hash_update = bond_ethdev_rss_hash_update, + .rss_hash_conf_get = bond_ethdev_rss_hash_conf_get }; static int @@ -2090,6 +2245,28 @@ bond_ethdev_configure(struct rte_eth_dev *dev) int arg_count; uint8_t port_id = dev - rte_eth_devices; + static const uint8_t default_rss_key[40] = { + 0x6D, 0x5A, 0x56, 0xDA, 0x25, 0x5B, 0x0E, 0xC2, 0x41, 0x67, 0x25, 0x3D, + 0x43, 0xA3, 0x8F, 0xB0, 0xD0, 0xCA, 0x2B, 0xCB, 0xAE, 0x7B, 0x30, 0xB4, + 0x77, 0xCB, 0x2D, 0xA3, 0x80, 0x30, 0xF2, 0x0C, 0x6A, 0x42, 0xB7, 0x3B, + 0xBE, 0xAC, 0x01, 0xFA + }; + + unsigned i, j; + + /* If RSS is enabled, fill table and key with default values */ + if (dev->data->dev_conf.rxmode.mq_mode & ETH_MQ_RX_RSS) { + dev->data->dev_conf.rx_adv_conf.rss_conf.rss_key = internals->rss_key; + dev->data->dev_conf.rx_adv_conf.rss_conf.rss_key_len = 0; + memcpy(internals->rss_key, default_rss_key, 40); + + for (i = 0; i < RTE_DIM(internals->reta_conf); i++) { + internals->reta_conf[i].mask = ~0LL; + for (j = 0; j < RTE_RETA_GROUP_SIZE; j++) + internals->reta_conf[i].reta[j] = j % dev->data->nb_rx_queues; + } + } + /* * if no kvlist, it means that this bonded device has been created * through the bonding api. diff --git a/drivers/net/bonding/rte_eth_bond_private.h b/drivers/net/bonding/rte_eth_bond_private.h index 038bca6..e7af809 100644 --- a/drivers/net/bonding/rte_eth_bond_private.h +++ b/drivers/net/bonding/rte_eth_bond_private.h @@ -103,6 +103,8 @@ struct bond_slave_details { uint8_t last_link_status; /**< Port Id of slave eth_dev */ struct ether_addr persisted_mac_addr; + + uint16_t reta_size; }; @@ -155,6 +157,16 @@ struct bond_dev_private { uint32_t rx_offload_capa; /** Rx offload capability */ uint32_t tx_offload_capa; /** Tx offload capability */ + /** Bit mask of RSS offloads, the bit offset also means flow type */ + uint64_t flow_type_rss_offloads; + + uint16_t reta_size; + struct rte_eth_rss_reta_entry64 reta_conf[ETH_RSS_RETA_SIZE_512 / + RTE_RETA_GROUP_SIZE]; + + uint8_t rss_key[52]; /**< 52-byte hash key buffer. */ + uint8_t rss_key_len; /**< hash key length in bytes. */ + struct rte_kvargs *kvlist; uint8_t slave_update_idx; }; -- 1.7.9.5 ^ permalink raw reply [flat|nested] 126+ messages in thread
* [dpdk-dev] [PATCH v6 2/9] null: fix segfault when null_pmd added to bonding 2015-10-16 10:00 ` [dpdk-dev] [PATCH v6 0/9] Dynamic RSS Configuration for Bonding Tomasz Kulasek 2015-10-16 10:00 ` [dpdk-dev] [PATCH v6 1/9] bonding: rss dynamic configuration Tomasz Kulasek @ 2015-10-16 10:00 ` Tomasz Kulasek 2015-10-27 16:58 ` Thomas Monjalon 2015-10-16 10:00 ` [dpdk-dev] [PATCH v6 3/9] null: extend number of virtual queues Tomasz Kulasek ` (8 subsequent siblings) 10 siblings, 1 reply; 126+ messages in thread From: Tomasz Kulasek @ 2015-10-16 10:00 UTC (permalink / raw) To: dev This patch initializes eth_dev->link_intr_cbs queue used when null pmd is added to the bonding. v5 changes: - removed unnecessary malloc for eth_driver (rte_null_pmd) Signed-off-by: Tomasz Kulasek <tomaszx.kulasek@intel.com> --- drivers/net/null/rte_eth_null.c | 2 ++ 1 file changed, 2 insertions(+) diff --git a/drivers/net/null/rte_eth_null.c b/drivers/net/null/rte_eth_null.c index e244595..c748101 100644 --- a/drivers/net/null/rte_eth_null.c +++ b/drivers/net/null/rte_eth_null.c @@ -432,6 +432,7 @@ eth_dev_null_create(const char *name, internals->numa_node = numa_node; pci_dev->numa_node = numa_node; + pci_dev->driver = &rte_null_pmd.pci_drv; data->dev_private = internals; data->port_id = eth_dev->data->port_id; @@ -445,6 +446,7 @@ eth_dev_null_create(const char *name, eth_dev->dev_ops = &ops; eth_dev->pci_dev = pci_dev; eth_dev->driver = &rte_null_pmd; + TAILQ_INIT(ð_dev->link_intr_cbs); /* finally assign rx and tx ops */ if (packet_copy) { -- 1.7.9.5 ^ permalink raw reply [flat|nested] 126+ messages in thread
* Re: [dpdk-dev] [PATCH v6 2/9] null: fix segfault when null_pmd added to bonding 2015-10-16 10:00 ` [dpdk-dev] [PATCH v6 2/9] null: fix segfault when null_pmd added to bonding Tomasz Kulasek @ 2015-10-27 16:58 ` Thomas Monjalon 2015-10-28 13:43 ` Kulasek, TomaszX 0 siblings, 1 reply; 126+ messages in thread From: Thomas Monjalon @ 2015-10-27 16:58 UTC (permalink / raw) To: Tomasz Kulasek; +Cc: dev Hi, There is no change in v6 for this patch which was acked by Tetsuya. So why not keep the Acked-by below your Signed-off-by? It seems patches 2, 3, 4 and 5 were Acked by Tetsuya. Other acks I'm missing? 2015-10-16 12:00, Tomasz Kulasek: > This patch initializes eth_dev->link_intr_cbs queue used when null pmd is > added to the bonding. > > v5 changes: > - removed unnecessary malloc for eth_driver (rte_null_pmd) > > Signed-off-by: Tomasz Kulasek <tomaszx.kulasek@intel.com> ^ permalink raw reply [flat|nested] 126+ messages in thread
* Re: [dpdk-dev] [PATCH v6 2/9] null: fix segfault when null_pmd added to bonding 2015-10-27 16:58 ` Thomas Monjalon @ 2015-10-28 13:43 ` Kulasek, TomaszX 2015-10-29 7:19 ` Tetsuya Mukawa 0 siblings, 1 reply; 126+ messages in thread From: Kulasek, TomaszX @ 2015-10-28 13:43 UTC (permalink / raw) To: Thomas Monjalon; +Cc: dev > -----Original Message----- > From: Thomas Monjalon [mailto:thomas.monjalon@6wind.com] > Sent: Tuesday, October 27, 2015 17:59 > To: Kulasek, TomaszX > Cc: dev@dpdk.org; Tetsuya Mukawa > Subject: Re: [dpdk-dev] [PATCH v6 2/9] null: fix segfault when null_pmd > added to bonding > > Hi, > There is no change in v6 for this patch which was acked by Tetsuya. > So why not keep the Acked-by below your Signed-off-by? > > It seems patches 2, 3, 4 and 5 were Acked by Tetsuya. > Other acks I'm missing? > Hi, Patches 4 and 5 were changed due to the Tetsuya's suggestions and already reviewed. There are not big changes, but I'm not sure if it should be reacked by Tetsuya, or I can just copy ack? Tomasz ^ permalink raw reply [flat|nested] 126+ messages in thread
* Re: [dpdk-dev] [PATCH v6 2/9] null: fix segfault when null_pmd added to bonding 2015-10-28 13:43 ` Kulasek, TomaszX @ 2015-10-29 7:19 ` Tetsuya Mukawa 2015-10-30 13:50 ` Kulasek, TomaszX 0 siblings, 1 reply; 126+ messages in thread From: Tetsuya Mukawa @ 2015-10-29 7:19 UTC (permalink / raw) To: Kulasek, TomaszX; +Cc: dev On 2015/10/28 22:43, Kulasek, TomaszX wrote: >> -----Original Message----- >> From: Thomas Monjalon [mailto:thomas.monjalon@6wind.com] >> Sent: Tuesday, October 27, 2015 17:59 >> To: Kulasek, TomaszX >> Cc: dev@dpdk.org; Tetsuya Mukawa >> Subject: Re: [dpdk-dev] [PATCH v6 2/9] null: fix segfault when null_pmd >> added to bonding >> >> Hi, >> There is no change in v6 for this patch which was acked by Tetsuya. >> So why not keep the Acked-by below your Signed-off-by? >> >> It seems patches 2, 3, 4 and 5 were Acked by Tetsuya. >> Other acks I'm missing? >> > Hi, > > Patches 4 and 5 were changed due to the Tetsuya's suggestions and already reviewed. There are not big changes, but I'm not sure if it should be reacked by Tetsuya, or I can just copy ack? > > Tomasz Hi Tomasz, Could you please check Bernard's patch series? - [PATCH v5 00/28] remove pci driver from vdevs He tries to remove pci_drv from PMD. It seems your below patch will be affected by his patch. - null: fix segfault when null_pmd added to bonding I just wonder your bonding RSS patch works without below fixing. drivers/net/null/rte_eth_null.c : + pci_dev->driver = &rte_null_pmd.pci_drv; Anyway, Could you please check him patches? Thanks, Tetsuya ^ permalink raw reply [flat|nested] 126+ messages in thread
* Re: [dpdk-dev] [PATCH v6 2/9] null: fix segfault when null_pmd added to bonding 2015-10-29 7:19 ` Tetsuya Mukawa @ 2015-10-30 13:50 ` Kulasek, TomaszX 0 siblings, 0 replies; 126+ messages in thread From: Kulasek, TomaszX @ 2015-10-30 13:50 UTC (permalink / raw) To: Tetsuya Mukawa; +Cc: dev > -----Original Message----- > From: Tetsuya Mukawa [mailto:mukawa@igel.co.jp] > Sent: Thursday, October 29, 2015 08:19 > To: Kulasek, TomaszX > Cc: Thomas Monjalon; dev@dpdk.org > Subject: Re: [dpdk-dev] [PATCH v6 2/9] null: fix segfault when null_pmd > added to bonding > > On 2015/10/28 22:43, Kulasek, TomaszX wrote: > >> -----Original Message----- > >> From: Thomas Monjalon [mailto:thomas.monjalon@6wind.com] > >> Sent: Tuesday, October 27, 2015 17:59 > >> To: Kulasek, TomaszX > >> Cc: dev@dpdk.org; Tetsuya Mukawa > >> Subject: Re: [dpdk-dev] [PATCH v6 2/9] null: fix segfault when > >> null_pmd added to bonding > >> > >> Hi, > >> There is no change in v6 for this patch which was acked by Tetsuya. > >> So why not keep the Acked-by below your Signed-off-by? > >> > >> It seems patches 2, 3, 4 and 5 were Acked by Tetsuya. > >> Other acks I'm missing? > >> > > Hi, > > > > Patches 4 and 5 were changed due to the Tetsuya's suggestions and > already reviewed. There are not big changes, but I'm not sure if it > should be reacked by Tetsuya, or I can just copy ack? > > > > Tomasz > > Hi Tomasz, > > Could you please check Bernard's patch series? > - [PATCH v5 00/28] remove pci driver from vdevs He tries to remove > pci_drv from PMD. > > It seems your below patch will be affected by his patch. > - null: fix segfault when null_pmd added to bonding > > I just wonder your bonding RSS patch works without below fixing. > drivers/net/null/rte_eth_null.c : > + pci_dev->driver = &rte_null_pmd.pci_drv; > > Anyway, Could you please check him patches? > > Thanks, > Tetsuya Hi Tetsuya, I've just checked Bernard's patches and it seems that it can be safely removed from here. It's not used in RSS implementation, so have no impact on it. I'm sending patchset v7 with copied your acks, as Thomas suggested, if you have no other objections. Tomasz ^ permalink raw reply [flat|nested] 126+ messages in thread
* [dpdk-dev] [PATCH v6 3/9] null: extend number of virtual queues 2015-10-16 10:00 ` [dpdk-dev] [PATCH v6 0/9] Dynamic RSS Configuration for Bonding Tomasz Kulasek 2015-10-16 10:00 ` [dpdk-dev] [PATCH v6 1/9] bonding: rss dynamic configuration Tomasz Kulasek 2015-10-16 10:00 ` [dpdk-dev] [PATCH v6 2/9] null: fix segfault when null_pmd added to bonding Tomasz Kulasek @ 2015-10-16 10:00 ` Tomasz Kulasek 2015-10-16 10:00 ` [dpdk-dev] [PATCH v6 4/9] null: export eth_dev_null_create Tomasz Kulasek ` (7 subsequent siblings) 10 siblings, 0 replies; 126+ messages in thread From: Tomasz Kulasek @ 2015-10-16 10:00 UTC (permalink / raw) To: dev This patch adds a possibility to configure more than one queue on null device. v5 changes: - fixed queues number configuration (using internals->nb_*_queues instead of dev->data->nb_*_queues) Signed-off-by: Tomasz Kulasek <tomaszx.kulasek@intel.com> --- drivers/net/null/rte_eth_null.c | 28 +++++++++++++++++++--------- 1 file changed, 19 insertions(+), 9 deletions(-) diff --git a/drivers/net/null/rte_eth_null.c b/drivers/net/null/rte_eth_null.c index c748101..bf81b1b 100644 --- a/drivers/net/null/rte_eth_null.c +++ b/drivers/net/null/rte_eth_null.c @@ -71,8 +71,8 @@ struct pmd_internals { unsigned nb_rx_queues; unsigned nb_tx_queues; - struct null_queue rx_null_queues[1]; - struct null_queue tx_null_queues[1]; + struct null_queue rx_null_queues[RTE_MAX_QUEUES_PER_PORT]; + struct null_queue tx_null_queues[RTE_MAX_QUEUES_PER_PORT]; }; @@ -178,7 +178,15 @@ eth_null_copy_tx(void *q, struct rte_mbuf **bufs, uint16_t nb_bufs) } static int -eth_dev_configure(struct rte_eth_dev *dev __rte_unused) { return 0; } +eth_dev_configure(struct rte_eth_dev *dev) { + struct pmd_internals *internals; + + internals = dev->data->dev_private; + internals->nb_rx_queues = dev->data->nb_rx_queues; + internals->nb_tx_queues = dev->data->nb_tx_queues; + + return 0; +} static int eth_dev_start(struct rte_eth_dev *dev) @@ -213,10 +221,11 @@ eth_rx_queue_setup(struct rte_eth_dev *dev, uint16_t rx_queue_id, if ((dev == NULL) || (mb_pool == NULL)) return -EINVAL; - if (rx_queue_id != 0) + internals = dev->data->dev_private; + + if (rx_queue_id >= internals->nb_rx_queues) return -ENODEV; - internals = dev->data->dev_private; packet_size = internals->packet_size; internals->rx_null_queues[rx_queue_id].mb_pool = mb_pool; @@ -246,10 +255,11 @@ eth_tx_queue_setup(struct rte_eth_dev *dev, uint16_t tx_queue_id, if (dev == NULL) return -EINVAL; - if (tx_queue_id != 0) + internals = dev->data->dev_private; + + if (tx_queue_id >= internals->nb_tx_queues) return -ENODEV; - internals = dev->data->dev_private; packet_size = internals->packet_size; dev->data->tx_queues[tx_queue_id] = @@ -279,8 +289,8 @@ eth_dev_info(struct rte_eth_dev *dev, dev_info->driver_name = drivername; dev_info->max_mac_addrs = 1; dev_info->max_rx_pktlen = (uint32_t)-1; - dev_info->max_rx_queues = (uint16_t)internals->nb_rx_queues; - dev_info->max_tx_queues = (uint16_t)internals->nb_tx_queues; + dev_info->max_rx_queues = RTE_DIM(internals->rx_null_queues); + dev_info->max_tx_queues = RTE_DIM(internals->tx_null_queues); dev_info->min_rx_bufsize = 0; dev_info->pci_dev = NULL; } -- 1.7.9.5 ^ permalink raw reply [flat|nested] 126+ messages in thread
* [dpdk-dev] [PATCH v6 4/9] null: export eth_dev_null_create 2015-10-16 10:00 ` [dpdk-dev] [PATCH v6 0/9] Dynamic RSS Configuration for Bonding Tomasz Kulasek ` (2 preceding siblings ...) 2015-10-16 10:00 ` [dpdk-dev] [PATCH v6 3/9] null: extend number of virtual queues Tomasz Kulasek @ 2015-10-16 10:00 ` Tomasz Kulasek 2015-10-16 10:00 ` [dpdk-dev] [PATCH v6 5/9] null: virtual dynamic rss configuration Tomasz Kulasek ` (6 subsequent siblings) 10 siblings, 0 replies; 126+ messages in thread From: Tomasz Kulasek @ 2015-10-16 10:00 UTC (permalink / raw) To: dev v6 changes: - reordered with patch 5/9 - fixed forward dependency to the patch 5/9 Signed-off-by: Tomasz Kulasek <tomaszx.kulasek@intel.com> --- drivers/net/null/Makefile | 2 +- drivers/net/null/rte_eth_null.c | 4 ++- drivers/net/null/rte_eth_null.h | 40 +++++++++++++++++++++++++++++ drivers/net/null/rte_pmd_null_version.map | 7 +++++ 4 files changed, 51 insertions(+), 2 deletions(-) create mode 100644 drivers/net/null/rte_eth_null.h diff --git a/drivers/net/null/Makefile b/drivers/net/null/Makefile index 96ba01c..2202389 100644 --- a/drivers/net/null/Makefile +++ b/drivers/net/null/Makefile @@ -51,7 +51,7 @@ SRCS-$(CONFIG_RTE_LIBRTE_PMD_NULL) += rte_eth_null.c # # Export include files # -SYMLINK-y-include += +SYMLINK-y-include += rte_eth_null.h # this lib depends upon: DEPDIRS-$(CONFIG_RTE_LIBRTE_PMD_NULL) += lib/librte_mbuf diff --git a/drivers/net/null/rte_eth_null.c b/drivers/net/null/rte_eth_null.c index bf81b1b..236d998 100644 --- a/drivers/net/null/rte_eth_null.c +++ b/drivers/net/null/rte_eth_null.c @@ -38,6 +38,8 @@ #include <rte_dev.h> #include <rte_kvargs.h> +#include "rte_eth_null.h" + #define ETH_NULL_PACKET_SIZE_ARG "size" #define ETH_NULL_PACKET_COPY_ARG "copy" @@ -387,7 +389,7 @@ static const struct eth_dev_ops ops = { .stats_reset = eth_stats_reset, }; -static int +int eth_dev_null_create(const char *name, const unsigned numa_node, unsigned packet_size, diff --git a/drivers/net/null/rte_eth_null.h b/drivers/net/null/rte_eth_null.h new file mode 100644 index 0000000..abada8c --- /dev/null +++ b/drivers/net/null/rte_eth_null.h @@ -0,0 +1,40 @@ +/*- + * BSD LICENSE + * + * Copyright(c) 2015 Intel Corporation. All rights reserved. + * All rights reserved. + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * + * * Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * * Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in + * the documentation and/or other materials provided with the + * distribution. + * * Neither the name of Intel Corporation nor the names of its + * contributors may be used to endorse or promote products derived + * from this software without specific prior written permission. + * + * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS + * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT + * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR + * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT + * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, + * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT + * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, + * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY + * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT + * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE + * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + */ + +#ifndef RTE_ETH_NULL_H_ +#define RTE_ETH_NULL_H_ + +int eth_dev_null_create(const char *name, const unsigned numa_node, + unsigned packet_size, unsigned packet_copy); + +#endif /* RTE_ETH_NULL_H_ */ diff --git a/drivers/net/null/rte_pmd_null_version.map b/drivers/net/null/rte_pmd_null_version.map index ef35398..84b1d0f 100644 --- a/drivers/net/null/rte_pmd_null_version.map +++ b/drivers/net/null/rte_pmd_null_version.map @@ -2,3 +2,10 @@ DPDK_2.0 { local: *; }; + +DPDK_2.2 { + global: + + eth_dev_null_create; + +} DPDK_2.0; -- 1.7.9.5 ^ permalink raw reply [flat|nested] 126+ messages in thread
* [dpdk-dev] [PATCH v6 5/9] null: virtual dynamic rss configuration 2015-10-16 10:00 ` [dpdk-dev] [PATCH v6 0/9] Dynamic RSS Configuration for Bonding Tomasz Kulasek ` (3 preceding siblings ...) 2015-10-16 10:00 ` [dpdk-dev] [PATCH v6 4/9] null: export eth_dev_null_create Tomasz Kulasek @ 2015-10-16 10:00 ` Tomasz Kulasek 2015-10-16 10:00 ` [dpdk-dev] [PATCH v6 6/9] test: " Tomasz Kulasek ` (5 subsequent siblings) 10 siblings, 0 replies; 126+ messages in thread From: Tomasz Kulasek @ 2015-10-16 10:00 UTC (permalink / raw) To: dev This implementation allows to set and read RSS configuration for null device, and is used to validate right values propagation over the slaves, in test units for dynamic RSS configuration for bonding. v6 changes: - reordered with patch 4/9 - recreated due to the changes in patch 4/9 v5 changes: - replaced memcpy with rte_memcpy Signed-off-by: Tomasz Kulasek <tomaszx.kulasek@intel.com> --- drivers/net/null/rte_eth_null.c | 115 +++++++++++++++++++++++++++++++++++++++ 1 file changed, 115 insertions(+) diff --git a/drivers/net/null/rte_eth_null.c b/drivers/net/null/rte_eth_null.c index 236d998..64e9000 100644 --- a/drivers/net/null/rte_eth_null.c +++ b/drivers/net/null/rte_eth_null.c @@ -37,6 +37,7 @@ #include <rte_memcpy.h> #include <rte_dev.h> #include <rte_kvargs.h> +#include <rte_spinlock.h> #include "rte_eth_null.h" @@ -75,6 +76,17 @@ struct pmd_internals { struct null_queue rx_null_queues[RTE_MAX_QUEUES_PER_PORT]; struct null_queue tx_null_queues[RTE_MAX_QUEUES_PER_PORT]; + + /** Bit mask of RSS offloads, the bit offset also means flow type */ + uint64_t flow_type_rss_offloads; + + rte_spinlock_t rss_lock; + + uint16_t reta_size; + struct rte_eth_rss_reta_entry64 reta_conf[ETH_RSS_RETA_SIZE_128 / + RTE_RETA_GROUP_SIZE]; + + uint8_t rss_key[40]; /**< 40-byte hash key. */ }; @@ -295,6 +307,8 @@ eth_dev_info(struct rte_eth_dev *dev, dev_info->max_tx_queues = RTE_DIM(internals->tx_null_queues); dev_info->min_rx_bufsize = 0; dev_info->pci_dev = NULL; + dev_info->reta_size = internals->reta_size; + dev_info->flow_type_rss_offloads = internals->flow_type_rss_offloads; } static void @@ -375,6 +389,91 @@ static int eth_link_update(struct rte_eth_dev *dev __rte_unused, int wait_to_complete __rte_unused) { return 0; } +static int +eth_rss_reta_update(struct rte_eth_dev *dev, + struct rte_eth_rss_reta_entry64 *reta_conf, uint16_t reta_size) +{ + int i, j; + struct pmd_internals *internal = dev->data->dev_private; + + if (reta_size != internal->reta_size) + return -EINVAL; + + rte_spinlock_lock(&internal->rss_lock); + + /* Copy RETA table */ + for (i = 0; i < (internal->reta_size / RTE_RETA_GROUP_SIZE); i++) { + internal->reta_conf[i].mask = reta_conf[i].mask; + for (j = 0; j < RTE_RETA_GROUP_SIZE; j++) + if ((reta_conf[i].mask >> j) & 0x01) + internal->reta_conf[i].reta[j] = reta_conf[i].reta[j]; + } + + rte_spinlock_unlock(&internal->rss_lock); + + return 0; +} + +static int +eth_rss_reta_query(struct rte_eth_dev *dev, + struct rte_eth_rss_reta_entry64 *reta_conf, uint16_t reta_size) +{ + int i, j; + struct pmd_internals *internal = dev->data->dev_private; + + if (reta_size != internal->reta_size) + return -EINVAL; + + rte_spinlock_lock(&internal->rss_lock); + + /* Copy RETA table */ + for (i = 0; i < (internal->reta_size / RTE_RETA_GROUP_SIZE); i++) { + for (j = 0; j < RTE_RETA_GROUP_SIZE; j++) + if ((reta_conf[i].mask >> j) & 0x01) + reta_conf[i].reta[j] = internal->reta_conf[i].reta[j]; + } + + rte_spinlock_unlock(&internal->rss_lock); + + return 0; +} + +static int +eth_rss_hash_update(struct rte_eth_dev *dev, struct rte_eth_rss_conf *rss_conf) +{ + struct pmd_internals *internal = dev->data->dev_private; + + rte_spinlock_lock(&internal->rss_lock); + + if ((rss_conf->rss_hf & internal->flow_type_rss_offloads) != 0) + dev->data->dev_conf.rx_adv_conf.rss_conf.rss_hf = + rss_conf->rss_hf & internal->flow_type_rss_offloads; + + if (rss_conf->rss_key) + rte_memcpy(internal->rss_key, rss_conf->rss_key, 40); + + rte_spinlock_unlock(&internal->rss_lock); + + return 0; +} + +static int +eth_rss_hash_conf_get(struct rte_eth_dev *dev, + struct rte_eth_rss_conf *rss_conf) +{ + struct pmd_internals *internal = dev->data->dev_private; + + rte_spinlock_lock(&internal->rss_lock); + + rss_conf->rss_hf = dev->data->dev_conf.rx_adv_conf.rss_conf.rss_hf; + if (rss_conf->rss_key) + rte_memcpy(rss_conf->rss_key, internal->rss_key, 40); + + rte_spinlock_unlock(&internal->rss_lock); + + return 0; +} + static const struct eth_dev_ops ops = { .dev_start = eth_dev_start, .dev_stop = eth_dev_stop, @@ -387,6 +486,10 @@ static const struct eth_dev_ops ops = { .link_update = eth_link_update, .stats_get = eth_stats_get, .stats_reset = eth_stats_reset, + .reta_update = eth_rss_reta_update, + .reta_query = eth_rss_reta_query, + .rss_hash_update = eth_rss_hash_update, + .rss_hash_conf_get = eth_rss_hash_conf_get }; int @@ -402,6 +505,13 @@ eth_dev_null_create(const char *name, struct pmd_internals *internals = NULL; struct rte_eth_dev *eth_dev = NULL; + static const uint8_t default_rss_key[40] = { + 0x6D, 0x5A, 0x56, 0xDA, 0x25, 0x5B, 0x0E, 0xC2, 0x41, 0x67, 0x25, 0x3D, + 0x43, 0xA3, 0x8F, 0xB0, 0xD0, 0xCA, 0x2B, 0xCB, 0xAE, 0x7B, 0x30, 0xB4, + 0x77, 0xCB, 0x2D, 0xA3, 0x80, 0x30, 0xF2, 0x0C, 0x6A, 0x42, 0xB7, 0x3B, + 0xBE, 0xAC, 0x01, 0xFA + }; + if (name == NULL) return -EINVAL; @@ -443,6 +553,11 @@ eth_dev_null_create(const char *name, internals->packet_copy = packet_copy; internals->numa_node = numa_node; + internals->flow_type_rss_offloads = ETH_RSS_PROTO_MASK; + internals->reta_size = RTE_DIM(internals->reta_conf) * RTE_RETA_GROUP_SIZE; + + rte_memcpy(internals->rss_key, default_rss_key, 40); + pci_dev->numa_node = numa_node; pci_dev->driver = &rte_null_pmd.pci_drv; -- 1.7.9.5 ^ permalink raw reply [flat|nested] 126+ messages in thread
* [dpdk-dev] [PATCH v6 6/9] test: dynamic rss configuration 2015-10-16 10:00 ` [dpdk-dev] [PATCH v6 0/9] Dynamic RSS Configuration for Bonding Tomasz Kulasek ` (4 preceding siblings ...) 2015-10-16 10:00 ` [dpdk-dev] [PATCH v6 5/9] null: virtual dynamic rss configuration Tomasz Kulasek @ 2015-10-16 10:00 ` Tomasz Kulasek 2015-10-27 17:34 ` Thomas Monjalon 2015-10-16 10:00 ` [dpdk-dev] [PATCH v6 7/9] bonding: per queue stats Tomasz Kulasek ` (4 subsequent siblings) 10 siblings, 1 reply; 126+ messages in thread From: Tomasz Kulasek @ 2015-10-16 10:00 UTC (permalink / raw) To: dev Signed-off-by: Tomasz Kulasek <tomaszx.kulasek@intel.com> --- app/test/Makefile | 8 + app/test/test_link_bonding_rssconf.c | 679 ++++++++++++++++++++++++++++++++++ 2 files changed, 687 insertions(+) create mode 100644 app/test/test_link_bonding_rssconf.c diff --git a/app/test/Makefile b/app/test/Makefile index 294618f..c122f28 100644 --- a/app/test/Makefile +++ b/app/test/Makefile @@ -138,6 +138,7 @@ SRCS-$(CONFIG_RTE_LIBRTE_ACL) += test_acl.c ifeq ($(CONFIG_RTE_LIBRTE_PMD_RING),y) SRCS-$(CONFIG_RTE_LIBRTE_PMD_BOND) += test_link_bonding.c SRCS-$(CONFIG_RTE_LIBRTE_PMD_BOND) += test_link_bonding_mode4.c +SRCS-$(CONFIG_RTE_LIBRTE_PMD_BOND) += test_link_bonding_rssconf.c endif SRCS-$(CONFIG_RTE_LIBRTE_PMD_RING) += test_pmd_ring.c @@ -168,6 +169,13 @@ ifeq ($(CONFIG_RTE_BUILD_SHARED_LIB),y) LDLIBS += -lrte_pmd_ring endif endif +ifneq ($(CONFIG_RTE_LIBRTE_PMD_NULL),y) +$(error Link bonding rssconf tests require CONFIG_RTE_LIBRTE_PMD_NULL=y) +else +ifeq ($(CONFIG_RTE_BUILD_SHARED_LIB),y) +LDLIBS += -lrte_pmd_null +endif +endif endif include $(RTE_SDK)/mk/rte.app.mk diff --git a/app/test/test_link_bonding_rssconf.c b/app/test/test_link_bonding_rssconf.c new file mode 100644 index 0000000..e6714b4 --- /dev/null +++ b/app/test/test_link_bonding_rssconf.c @@ -0,0 +1,679 @@ +/*- + * BSD LICENSE + * + * Copyright(c) 2015 Intel Corporation. All rights reserved. + * All rights reserved. + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * + * * Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * * Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in + * the documentation and/or other materials provided with the + * distribution. + * * Neither the name of Intel Corporation nor the names of its + * contributors may be used to endorse or promote products derived + * from this software without specific prior written permission. + * + * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS + * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT + * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR + * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT + * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, + * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT + * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, + * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY + * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT + * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE + * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + */ + +#include <string.h> +#include <stdarg.h> +#include <stdio.h> +#include <stdlib.h> +#include <stdint.h> +#include <inttypes.h> +#include <errno.h> +#include <rte_cycles.h> +#include <sys/queue.h> + +#include <rte_byteorder.h> +#include <rte_common.h> +#include <rte_debug.h> +#include <rte_ethdev.h> +#include <rte_log.h> +#include <rte_lcore.h> +#include <rte_memory.h> + +#include <rte_string_fns.h> +#include <rte_errno.h> +#include <rte_eth_bond.h> +#include <rte_eth_null.h> + +#include "test.h" + +#define SLAVE_COUNT (4) + +#define RXTX_RING_SIZE 1024 +#define RXTX_QUEUE_COUNT 4 + +#define BONDED_DEV_NAME ("rssconf_bond_dev") + +#define SLAVE_DEV_NAME_FMT ("rssconf_slave%d") +#define SLAVE_RXTX_QUEUE_FMT ("rssconf_slave%d_q%d") + +#define NUM_MBUFS 8191 +#define MBUF_SIZE (1600 + sizeof(struct rte_mbuf) + RTE_PKTMBUF_HEADROOM) +#define MBUF_CACHE_SIZE 250 +#define BURST_SIZE 32 + +#define INVALID_SOCKET_ID (-1) +#define INVALID_PORT_ID (0xFF) +#define INVALID_BONDING_MODE (-1) + +struct slave_conf { + uint8_t port_id; + struct rte_eth_dev_info dev_info; + + struct rte_eth_rss_conf rss_conf; + uint8_t rss_key[40]; + struct rte_eth_rss_reta_entry64 reta_conf[512 / RTE_RETA_GROUP_SIZE]; + + uint8_t is_slave; + struct rte_ring *rxtx_queue[RXTX_QUEUE_COUNT]; +}; + +struct link_bonding_rssconf_unittest_params { + uint8_t bond_port_id; + struct rte_eth_dev_info bond_dev_info; + struct rte_eth_rss_reta_entry64 bond_reta_conf[512 / RTE_RETA_GROUP_SIZE]; + struct slave_conf slave_ports[SLAVE_COUNT]; + + struct rte_mempool *mbuf_pool; +}; + +static struct link_bonding_rssconf_unittest_params test_params = { + .bond_port_id = INVALID_PORT_ID, + .slave_ports = { + [0 ... SLAVE_COUNT - 1] = { .port_id = INVALID_PORT_ID, .is_slave = 0} + }, + .mbuf_pool = NULL, +}; + +/** + * Default port configuration with RSS turned off + */ +static struct rte_eth_conf default_pmd_conf = { + .rxmode = { + .mq_mode = ETH_MQ_RX_NONE, + .max_rx_pkt_len = ETHER_MAX_LEN, + .split_hdr_size = 0, + .header_split = 0, /**< Header Split disabled */ + .hw_ip_checksum = 0, /**< IP checksum offload enabled */ + .hw_vlan_filter = 0, /**< VLAN filtering disabled */ + .jumbo_frame = 0, /**< Jumbo Frame Support disabled */ + .hw_strip_crc = 0, /**< CRC stripped by hardware */ + }, + .txmode = { + .mq_mode = ETH_MQ_TX_NONE, + }, + .lpbk_mode = 0, +}; + +static struct rte_eth_conf rss_pmd_conf = { + .rxmode = { + .mq_mode = ETH_MQ_RX_RSS, + .max_rx_pkt_len = ETHER_MAX_LEN, + .split_hdr_size = 0, + .header_split = 0, /**< Header Split disabled */ + .hw_ip_checksum = 0, /**< IP checksum offload enabled */ + .hw_vlan_filter = 0, /**< VLAN filtering disabled */ + .jumbo_frame = 0, /**< Jumbo Frame Support disabled */ + .hw_strip_crc = 0, /**< CRC stripped by hardware */ + }, + .txmode = { + .mq_mode = ETH_MQ_TX_NONE, + }, + .rx_adv_conf = { + .rss_conf = { + .rss_key = NULL, + .rss_hf = ETH_RSS_IPV6, + }, + }, + .lpbk_mode = 0, +}; + +#define FOR_EACH(_i, _item, _array, _size) \ + for (_i = 0, _item = &_array[0]; _i < _size && (_item = &_array[_i]); _i++) + +/* Macro for iterating over every port that can be used as a slave + * in this test. + * _i variable used as an index in test_params->slave_ports + * _slave pointer to &test_params->slave_ports[_idx] + */ +#define FOR_EACH_PORT(_i, _port) \ + FOR_EACH(_i, _port, test_params.slave_ports, \ + RTE_DIM(test_params.slave_ports)) + +static int +configure_ethdev(uint8_t port_id, struct rte_eth_conf *eth_conf, uint8_t start) +{ + int rxq, txq; + + TEST_ASSERT(rte_eth_dev_configure(port_id, RXTX_QUEUE_COUNT, + RXTX_QUEUE_COUNT, eth_conf) == 0, "Failed to configure device %u", + port_id); + + for (rxq = 0; rxq < RXTX_QUEUE_COUNT; rxq++) { + TEST_ASSERT(rte_eth_rx_queue_setup(port_id, rxq, RXTX_RING_SIZE, + rte_eth_dev_socket_id(port_id), NULL, + test_params.mbuf_pool) == 0, "Failed to setup rx queue."); + } + + for (txq = 0; txq < RXTX_QUEUE_COUNT; txq++) { + TEST_ASSERT(rte_eth_tx_queue_setup(port_id, txq, RXTX_RING_SIZE, + rte_eth_dev_socket_id(port_id), NULL) == 0, + "Failed to setup tx queue."); + } + + if (start) { + TEST_ASSERT(rte_eth_dev_start(port_id) == 0, + "Failed to start device (%d).", port_id); + } + + return 0; +} + +/** + * Remove all slaves from bonding + */ +static int +remove_slaves(void) +{ + unsigned n; + struct slave_conf *port; + + FOR_EACH_PORT(n, port) { + port = &test_params.slave_ports[n]; + if (port->is_slave) { + TEST_ASSERT_SUCCESS(rte_eth_bond_slave_remove( + test_params.bond_port_id, port->port_id), + "Cannot remove slave %d from bonding", port->port_id); + port->is_slave = 0; + } + } + + return 0; +} + +static int +remove_slaves_and_stop_bonded_device(void) +{ + TEST_ASSERT_SUCCESS(remove_slaves(), "Removing slaves"); + rte_eth_dev_stop(test_params.bond_port_id); + return TEST_SUCCESS; +} + +/** + * Add all slaves to bonding + */ +static int +bond_slaves(void) +{ + unsigned n; + struct slave_conf *port; + + FOR_EACH_PORT(n, port) { + port = &test_params.slave_ports[n]; + if (!port->is_slave) { + TEST_ASSERT_SUCCESS(rte_eth_bond_slave_add(test_params.bond_port_id, + port->port_id), "Cannot attach slave %d to the bonding", + port->port_id); + port->is_slave = 1; + } + } + + return 0; +} + +/** + * Set all RETA values in port_id to value + */ +static int +reta_set(uint8_t port_id, uint8_t value, int reta_size) +{ + struct rte_eth_rss_reta_entry64 reta_conf[512/RTE_RETA_GROUP_SIZE]; + int i, j; + + for (i = 0; i < reta_size / RTE_RETA_GROUP_SIZE; i++) { + /* select all fields to set */ + reta_conf[i].mask = ~0LL; + for (j = 0; j < RTE_RETA_GROUP_SIZE; j++) + reta_conf[i].reta[j] = value; + } + + return rte_eth_dev_rss_reta_update(port_id, reta_conf, reta_size); +} + +/** + * Check if slaves RETA is synchronized with bonding port. Returns 1 if slave + * port is synced with bonding port. + */ +static int +reta_check_synced(struct slave_conf *port) +{ + unsigned i; + + for (i = 0; i < test_params.bond_dev_info.reta_size; + i++) { + + int index = i / RTE_RETA_GROUP_SIZE; + int shift = i % RTE_RETA_GROUP_SIZE; + + if (port->reta_conf[index].reta[shift] != + test_params.bond_reta_conf[index].reta[shift]) + return 0; + + } + + return 1; +} + +/** + * Fetch bonding ports RETA + */ +static int +bond_reta_fetch(void) { + unsigned j; + + for (j = 0; j < test_params.bond_dev_info.reta_size / RTE_RETA_GROUP_SIZE; + j++) + test_params.bond_reta_conf[j].mask = ~0LL; + + TEST_ASSERT_SUCCESS(rte_eth_dev_rss_reta_query(test_params.bond_port_id, + test_params.bond_reta_conf, test_params.bond_dev_info.reta_size), + "Cannot take bonding ports RSS configuration"); + return 0; +} + +/** + * Fetch slaves RETA + */ +static int +slave_reta_fetch(struct slave_conf *port) { + unsigned j; + + for (j = 0; j < port->dev_info.reta_size / RTE_RETA_GROUP_SIZE; j++) + port->reta_conf[j].mask = ~0LL; + + TEST_ASSERT_SUCCESS(rte_eth_dev_rss_reta_query(port->port_id, + port->reta_conf, port->dev_info.reta_size), + "Cannot take bonding ports RSS configuration"); + return 0; +} + +/** + * Remove and add slave to check if slaves configuration is synced with + * the bonding ports values after adding new slave. + */ +static int +slave_remove_and_add(void) +{ + struct slave_conf *port = &(test_params.slave_ports[0]); + + /* 1. Remove first slave from bonding */ + TEST_ASSERT_SUCCESS(rte_eth_bond_slave_remove(test_params.bond_port_id, + port->port_id), "Cannot remove slave #d from bonding"); + + /* 2. Change removed (ex-)slave and bonding configuration to different + * values + */ + reta_set(test_params.bond_port_id, 1, test_params.bond_dev_info.reta_size); + bond_reta_fetch(); + + reta_set(port->port_id, 2, port->dev_info.reta_size); + slave_reta_fetch(port); + + TEST_ASSERT(reta_check_synced(port) == 0, + "Removed slave didn't should be synchronized with bonding port"); + + /* 3. Add (ex-)slave and check if configuration changed*/ + TEST_ASSERT_SUCCESS(rte_eth_bond_slave_add(test_params.bond_port_id, + port->port_id), "Cannot add slave"); + + bond_reta_fetch(); + slave_reta_fetch(port); + + return reta_check_synced(port); +} + +/** + * Test configuration propagation over slaves. + */ +static int +test_propagate(void) +{ + unsigned i; + uint8_t n; + struct slave_conf *port; + uint8_t bond_rss_key[40]; + struct rte_eth_rss_conf bond_rss_conf; + + int retval = 0; + uint64_t rss_hf = 0; + uint64_t default_rss_hf = 0; + + rte_eth_dev_info_get(test_params.bond_port_id, &test_params.bond_dev_info); + + /* + * Test hash function propagation + */ + for (i = 0; i < sizeof(test_params.bond_dev_info.flow_type_rss_offloads)*8; + i++) { + + rss_hf = test_params.bond_dev_info.flow_type_rss_offloads & (1<<i); + if (rss_hf) { + bond_rss_conf.rss_key = NULL; + bond_rss_conf.rss_hf = rss_hf; + + retval = rte_eth_dev_rss_hash_update(test_params.bond_port_id, + &bond_rss_conf); + TEST_ASSERT_SUCCESS(retval, "Cannot set slaves hash function"); + + FOR_EACH_PORT(n, port) { + port = &test_params.slave_ports[n]; + + retval = rte_eth_dev_rss_hash_conf_get(port->port_id, + &port->rss_conf); + TEST_ASSERT_SUCCESS(retval, + "Cannot take slaves RSS configuration"); + + TEST_ASSERT(port->rss_conf.rss_hf == rss_hf, + "Hash function not propagated for slave %d", + port->port_id); + } + + default_rss_hf = rss_hf; + } + + } + + /* + * Test key propagation + */ + for (i = 1; i < 10; i++) { + + /* Set all keys to zero */ + FOR_EACH_PORT(n, port) { + port = &test_params.slave_ports[n]; + memset(port->rss_conf.rss_key, 0, 40); + retval = rte_eth_dev_rss_hash_update(port->port_id, + &port->rss_conf); + TEST_ASSERT_SUCCESS(retval, "Cannot set slaves RSS keys"); + } + + memset(bond_rss_key, i, sizeof(bond_rss_key)); + bond_rss_conf.rss_hf = default_rss_hf, + bond_rss_conf.rss_key = bond_rss_key; + bond_rss_conf.rss_key_len = 40; + + retval = rte_eth_dev_rss_hash_update(test_params.bond_port_id, + &bond_rss_conf); + TEST_ASSERT_SUCCESS(retval, "Cannot set bonded port RSS keys"); + + FOR_EACH_PORT(n, port) { + port = &test_params.slave_ports[n]; + + retval = rte_eth_dev_rss_hash_conf_get(port->port_id, + &(port->rss_conf)); + + TEST_ASSERT_SUCCESS(retval, + "Cannot take slaves RSS configuration"); + + /* compare keys */ + retval = memcmp(port->rss_conf.rss_key, bond_rss_key, + sizeof(bond_rss_key)); + TEST_ASSERT(retval == 0, "Key value not propagated for slave %d", + port->port_id); + } + } + + /* + * Test RETA propagation + */ + for (i = 0; i < RXTX_QUEUE_COUNT; i++) { + + /* Set all keys to zero */ + FOR_EACH_PORT(n, port) { + port = &test_params.slave_ports[n]; + retval = reta_set(port->port_id, (i + 1) % RXTX_QUEUE_COUNT, + port->dev_info.reta_size); + TEST_ASSERT_SUCCESS(retval, "Cannot set slaves RETA"); + } + + TEST_ASSERT_SUCCESS(reta_set(test_params.bond_port_id, + i % RXTX_QUEUE_COUNT, test_params.bond_dev_info.reta_size), + "Cannot set bonded port RETA"); + + bond_reta_fetch(); + + FOR_EACH_PORT(n, port) { + port = &test_params.slave_ports[n]; + + slave_reta_fetch(port); + TEST_ASSERT(reta_check_synced(port) == 1, "RETAs inconsistent"); + } + } + + return TEST_SUCCESS; +} + +/** + * Test propagation logic, when RX_RSS mq_mode is turned on for bonding port + */ +static int +test_rss(void) +{ + /** + * Configure bonding port in RSS mq mode + */ + TEST_ASSERT_SUCCESS(configure_ethdev(test_params.bond_port_id, + &rss_pmd_conf, 0), "Failed to configure bonding device\n"); + + rte_eth_dev_info_get(test_params.bond_port_id, &test_params.bond_dev_info); + + TEST_ASSERT_SUCCESS(bond_slaves(), "Bonding slaves failed"); + + TEST_ASSERT_SUCCESS(rte_eth_dev_start(test_params.bond_port_id), + "Failed to start bonding port (%d).", test_params.bond_port_id); + + TEST_ASSERT_SUCCESS(test_propagate(), "Propagation test failed"); + + TEST_ASSERT(slave_remove_and_add() == 1, "New slave should be synced"); + + remove_slaves_and_stop_bonded_device(); + + return TEST_SUCCESS; +} + +/** + * Test propagation logic, when RX_RSS mq_mode is turned off for bonding port + */ +static int +test_rss_lazy(void) +{ + TEST_ASSERT_SUCCESS(configure_ethdev(test_params.bond_port_id, + &default_pmd_conf, 0), "Failed to configure bonding device\n"); + + rte_eth_dev_info_get(test_params.bond_port_id, &test_params.bond_dev_info); + + TEST_ASSERT_SUCCESS(bond_slaves(), "Bonding slaves failed"); + + TEST_ASSERT_SUCCESS(rte_eth_dev_start(test_params.bond_port_id), + "Failed to start bonding port (%d).", test_params.bond_port_id); + + TEST_ASSERT_SUCCESS(test_propagate(), "Propagation test failed"); + + TEST_ASSERT(slave_remove_and_add() == 0, "New slave shouldn't be synced"); + + remove_slaves_and_stop_bonded_device(); + + return TEST_SUCCESS; +} + +static int +test_setup(void) +{ + unsigned n; + int retval; + int port_id; + char name[256]; + struct slave_conf *port; + + if (test_params.mbuf_pool == NULL) { + + test_params.mbuf_pool = rte_mempool_create("RSS_MBUF_POOL", NUM_MBUFS * + SLAVE_COUNT, MBUF_SIZE, MBUF_CACHE_SIZE, + sizeof(struct rte_pktmbuf_pool_private), rte_pktmbuf_pool_init, + NULL, rte_pktmbuf_init, NULL, rte_socket_id(), 0); + + TEST_ASSERT(test_params.mbuf_pool != NULL, + "rte_mempool_create failed\n"); + } + + /* Create / initialize ring eth devs. */ + FOR_EACH_PORT(n, port) { + port = &test_params.slave_ports[n]; + + port_id = rte_eth_dev_count(); + snprintf(name, sizeof(name), SLAVE_DEV_NAME_FMT, port_id); + + retval = eth_dev_null_create(name, 0, 64, 0); + TEST_ASSERT_SUCCESS(retval, "Failed to create null device '%s'\n", + name); + + port->port_id = port_id; + + port->rss_conf.rss_key = port->rss_key; + port->rss_conf.rss_key_len = 40; + + retval = configure_ethdev(port->port_id, &default_pmd_conf, 0); + TEST_ASSERT_SUCCESS(retval, "Failed to configure virtual ethdev %s\n", + name); + + rte_eth_dev_info_get(port->port_id, &port->dev_info); + } + + if (test_params.bond_port_id == INVALID_PORT_ID) { + retval = rte_eth_bond_create(BONDED_DEV_NAME, 0, rte_socket_id()); + + TEST_ASSERT(retval >= 0, "Failed to create bonded ethdev %s", + BONDED_DEV_NAME); + + test_params.bond_port_id = retval; + + TEST_ASSERT_SUCCESS(configure_ethdev(test_params.bond_port_id, + &default_pmd_conf, 0), "Failed to configure bonding device\n"); + + rte_eth_dev_info_get(test_params.bond_port_id, + &test_params.bond_dev_info); + } + + return TEST_SUCCESS; +} + +static int +testsuite_teardown(void) +{ + struct slave_conf *port; + uint8_t i; + + /* Only stop ports. + * Any cleanup/reset state is done when particular test is + * started. */ + + rte_eth_dev_stop(test_params.bond_port_id); + + FOR_EACH_PORT(i, port) + rte_eth_dev_stop(port->port_id); + + return 0; +} + +static int +check_environment(void) +{ + return TEST_SUCCESS; +} + +static int +test_rssconf_executor(int (*test_func)(void)) +{ + int test_result; + + /* Check if environment is clean. Fail to launch a test if there was + * a critical error before that prevented to reset environment. */ + TEST_ASSERT_SUCCESS(check_environment(), + "Refusing to launch test in dirty environment."); + + RTE_VERIFY(test_func != NULL); + test_result = (*test_func)(); + + /* If test succeed check if environment wast left in good condition. */ + if (test_result == TEST_SUCCESS) + test_result = check_environment(); + + /* Reset environment in case test failed to do that. */ + if (test_result != TEST_SUCCESS) { + TEST_ASSERT_SUCCESS(remove_slaves_and_stop_bonded_device(), + "Failed to stop bonded device"); + } + + return test_result; +} + +static int +test_setup_wrapper(void) +{ + return test_rssconf_executor(&test_setup); +} + +static int +test_rss_wrapper(void) +{ + return test_rssconf_executor(&test_rss); +} + +static int +test_rss_lazy_wrapper(void) +{ + return test_rssconf_executor(&test_rss_lazy); +} + +static struct unit_test_suite link_bonding_rssconf_test_suite = { + .suite_name = "RSS Dynamic Configuration for Bonding Unit Test Suite", + .teardown = testsuite_teardown, + .unit_test_cases = { + TEST_CASE_NAMED("test_setup", test_setup_wrapper), + TEST_CASE_NAMED("test_rss", test_rss_wrapper), + TEST_CASE_NAMED("test_rss_lazy", test_rss_lazy_wrapper), + { NULL, NULL, NULL, NULL, NULL } /**< NULL terminate unit test array */ + } +}; + +static int +test_link_bonding_rssconf(void) +{ + return unit_test_suite_runner(&link_bonding_rssconf_test_suite); +} + +static struct test_command test_link_bonding_rssconf_cmd = { + .command = "link_bonding_rssconf_autotest", + .callback = test_link_bonding_rssconf, +}; + +REGISTER_TEST_COMMAND(test_link_bonding_rssconf_cmd); -- 1.7.9.5 ^ permalink raw reply [flat|nested] 126+ messages in thread
* Re: [dpdk-dev] [PATCH v6 6/9] test: dynamic rss configuration 2015-10-16 10:00 ` [dpdk-dev] [PATCH v6 6/9] test: " Tomasz Kulasek @ 2015-10-27 17:34 ` Thomas Monjalon 0 siblings, 0 replies; 126+ messages in thread From: Thomas Monjalon @ 2015-10-27 17:34 UTC (permalink / raw) To: Tomasz Kulasek; +Cc: dev This new test depends on null PMD. The dependency should be checked gracefully, see below. 2015-10-16 12:00, Tomasz Kulasek: > --- a/app/test/Makefile > +++ b/app/test/Makefile > @@ -138,6 +138,7 @@ SRCS-$(CONFIG_RTE_LIBRTE_ACL) += test_acl.c > ifeq ($(CONFIG_RTE_LIBRTE_PMD_RING),y) > SRCS-$(CONFIG_RTE_LIBRTE_PMD_BOND) += test_link_bonding.c > SRCS-$(CONFIG_RTE_LIBRTE_PMD_BOND) += test_link_bonding_mode4.c Why not enclose in ifeq ($(CONFIG_RTE_LIBRTE_PMD_NULL),y)? > +SRCS-$(CONFIG_RTE_LIBRTE_PMD_BOND) += test_link_bonding_rssconf.c > endif > > SRCS-$(CONFIG_RTE_LIBRTE_PMD_RING) += test_pmd_ring.c > @@ -168,6 +169,13 @@ ifeq ($(CONFIG_RTE_BUILD_SHARED_LIB),y) > LDLIBS += -lrte_pmd_ring > endif > endif > +ifneq ($(CONFIG_RTE_LIBRTE_PMD_NULL),y) > +$(error Link bonding rssconf tests require CONFIG_RTE_LIBRTE_PMD_NULL=y) Not needed if handled as suggested above. The build should not fail because a module is disabled. > +else > +ifeq ($(CONFIG_RTE_BUILD_SHARED_LIB),y) > +LDLIBS += -lrte_pmd_null > +endif > +endif ^ permalink raw reply [flat|nested] 126+ messages in thread
* [dpdk-dev] [PATCH v6 7/9] bonding: per queue stats 2015-10-16 10:00 ` [dpdk-dev] [PATCH v6 0/9] Dynamic RSS Configuration for Bonding Tomasz Kulasek ` (5 preceding siblings ...) 2015-10-16 10:00 ` [dpdk-dev] [PATCH v6 6/9] test: " Tomasz Kulasek @ 2015-10-16 10:00 ` Tomasz Kulasek 2015-10-16 10:00 ` [dpdk-dev] [PATCH v6 8/9] doc: fixed spellings and typos Tomasz Kulasek ` (3 subsequent siblings) 10 siblings, 0 replies; 126+ messages in thread From: Tomasz Kulasek @ 2015-10-16 10:00 UTC (permalink / raw) To: dev This patch adds fills bonding port's stats with a sum of corresponding values taken from bonded slaves, when stats are requested for bonding port. v5 changes: - removed queue_stats_mapping_set from eth_dev_ops of bonding device Signed-off-by: Tomasz Kulasek <tomaszx.kulasek@intel.com> --- drivers/net/bonding/rte_eth_bond_pmd.c | 11 ++++++++++- 1 file changed, 10 insertions(+), 1 deletion(-) diff --git a/drivers/net/bonding/rte_eth_bond_pmd.c b/drivers/net/bonding/rte_eth_bond_pmd.c index 2880f5c..eecb381 100644 --- a/drivers/net/bonding/rte_eth_bond_pmd.c +++ b/drivers/net/bonding/rte_eth_bond_pmd.c @@ -1801,7 +1801,7 @@ bond_ethdev_stats_get(struct rte_eth_dev *dev, struct rte_eth_stats *stats) { struct bond_dev_private *internals = dev->data->dev_private; struct rte_eth_stats slave_stats; - int i; + int i, j; for (i = 0; i < internals->slave_count; i++) { rte_eth_stats_get(internals->slaves[i].port_id, &slave_stats); @@ -1820,6 +1820,15 @@ bond_ethdev_stats_get(struct rte_eth_dev *dev, struct rte_eth_stats *stats) stats->rx_pause_xon += slave_stats.rx_pause_xon; stats->tx_pause_xoff += slave_stats.tx_pause_xoff; stats->rx_pause_xoff += slave_stats.rx_pause_xoff; + + for (j = 0; j < RTE_ETHDEV_QUEUE_STAT_CNTRS; j++) { + stats->q_ipackets[j] += slave_stats.q_ipackets[j]; + stats->q_opackets[j] += slave_stats.q_opackets[j]; + stats->q_ibytes[j] += slave_stats.q_ibytes[j]; + stats->q_obytes[j] += slave_stats.q_obytes[j]; + stats->q_errors[j] += slave_stats.q_errors[j]; + } + } } -- 1.7.9.5 ^ permalink raw reply [flat|nested] 126+ messages in thread
* [dpdk-dev] [PATCH v6 8/9] doc: fixed spellings and typos 2015-10-16 10:00 ` [dpdk-dev] [PATCH v6 0/9] Dynamic RSS Configuration for Bonding Tomasz Kulasek ` (6 preceding siblings ...) 2015-10-16 10:00 ` [dpdk-dev] [PATCH v6 7/9] bonding: per queue stats Tomasz Kulasek @ 2015-10-16 10:00 ` Tomasz Kulasek 2015-10-16 10:00 ` [dpdk-dev] [PATCH v6 9/9] doc: dynamic rss configuration for bonding Tomasz Kulasek ` (2 subsequent siblings) 10 siblings, 0 replies; 126+ messages in thread From: Tomasz Kulasek @ 2015-10-16 10:00 UTC (permalink / raw) To: dev Signed-off-by: Tomasz Kulasek <tomaszx.kulasek@intel.com> --- .../prog_guide/link_bonding_poll_mode_drv_lib.rst | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/doc/guides/prog_guide/link_bonding_poll_mode_drv_lib.rst b/doc/guides/prog_guide/link_bonding_poll_mode_drv_lib.rst index 96e554f..03baf90 100644 --- a/doc/guides/prog_guide/link_bonding_poll_mode_drv_lib.rst +++ b/doc/guides/prog_guide/link_bonding_poll_mode_drv_lib.rst @@ -188,7 +188,7 @@ conditions are not met. If a user wishes to monitor individual slaves then they must register callbacks with that slave directly. The link bonding library also supports devices which do not implement link -status change interrupts, this is achieve by polling the devices link status at +status change interrupts, this is achieved by polling the devices link status at a defined period which is set using the ``rte_eth_bond_link_monitoring_set`` API, the default polling interval is 10ms. When a device is added as a slave to a bonding device it is determined using the ``RTE_PCI_DRV_INTR_LSC`` flag @@ -286,7 +286,7 @@ and UDP protocols for load balancing. Using Link Bonding Devices -------------------------- -The librte_pmd_bond library support two modes of device creation, the libraries +The librte_pmd_bond library supports two modes of device creation, the libraries export full C API or using the EAL command line to statically configure link bonding devices at application startup. Using the EAL option it is possible to use link bonding functionality transparently without specific knowledge of the @@ -299,7 +299,7 @@ Using the Poll Mode Driver from an Application Using the librte_pmd_bond libraries API it is possible to dynamically create and manage link bonding device from within any application. Link bonding -device are created using the ``rte_eth_bond_create`` API which requires a +devices are created using the ``rte_eth_bond_create`` API which requires a unique device name, the link bonding mode to initial the device in and finally the socket Id which to allocate the devices resources onto. After successful creation of a bonding device it must be configured using the generic Ethernet @@ -362,7 +362,7 @@ The different options are: mode=2 * slave: Defines the PMD device which will be added as slave to the bonded - device. This option can be selected multiple time, for each device to be + device. This option can be selected multiple times, for each device to be added as a slave. Physical devices should be specified using their PCI address, in the format domain:bus:devid.function -- 1.7.9.5 ^ permalink raw reply [flat|nested] 126+ messages in thread
* [dpdk-dev] [PATCH v6 9/9] doc: dynamic rss configuration for bonding 2015-10-16 10:00 ` [dpdk-dev] [PATCH v6 0/9] Dynamic RSS Configuration for Bonding Tomasz Kulasek ` (7 preceding siblings ...) 2015-10-16 10:00 ` [dpdk-dev] [PATCH v6 8/9] doc: fixed spellings and typos Tomasz Kulasek @ 2015-10-16 10:00 ` Tomasz Kulasek 2015-10-19 13:22 ` [dpdk-dev] [PATCH v6 0/9] Dynamic RSS Configuration for Bonding Declan Doherty 2015-10-30 14:25 ` [dpdk-dev] [PATCH v7 " Tomasz Kulasek 10 siblings, 0 replies; 126+ messages in thread From: Tomasz Kulasek @ 2015-10-16 10:00 UTC (permalink / raw) To: dev Documentation update about implementation details and requirements for Dynamic RSS Configuration for Bonding. Signed-off-by: Tomasz Kulasek <tomaszx.kulasek@intel.com> --- .../prog_guide/link_bonding_poll_mode_drv_lib.rst | 34 ++++++++++++++++++-- 1 file changed, 32 insertions(+), 2 deletions(-) diff --git a/doc/guides/prog_guide/link_bonding_poll_mode_drv_lib.rst b/doc/guides/prog_guide/link_bonding_poll_mode_drv_lib.rst index 03baf90..46f0296 100644 --- a/doc/guides/prog_guide/link_bonding_poll_mode_drv_lib.rst +++ b/doc/guides/prog_guide/link_bonding_poll_mode_drv_lib.rst @@ -1,5 +1,5 @@ .. BSD LICENSE - Copyright(c) 2010-2014 Intel Corporation. All rights reserved. + Copyright(c) 2010-2015 Intel Corporation. All rights reserved. All rights reserved. Redistribution and use in source and binary forms, with or without @@ -173,7 +173,28 @@ After a slave device is added to a bonded device slave is stopped using ``rte_eth_dev_stop`` and then reconfigured using ``rte_eth_dev_configure`` the RX and TX queues are also reconfigured using ``rte_eth_tx_queue_setup`` / ``rte_eth_rx_queue_setup`` with the parameters use to configure the bonding -device. +device. If RSS is enabled for bonding device, this mode is also enabled on new +slave and configured as well. + +Setting up multi-queue mode for bonding device to RSS, makes it fully +RSS-capable, so all slaves are synchronized with its configuration. This mode is +intended to provide RSS configuration on slaves transparent for client +application implementation. + +Bonding device stores its own version of RSS settings i.e. RETA, RSS hash +function and RSS key, used to set up its slaves. That let to define the meaning +of RSS configuration of bonding device as desired configuration of whole bonding +(as one unit), without pointing any of slave inside. It is required to ensure +consistency and made it more errorproof. + +RSS hash function set for bonding device, is a maximal set of RSS hash functions +supported by all bonded slaves. RETA size is a GCD of all its RETA's sizes, so +it can be easily used as a pattern providing expected behavior, even if slave +RETAs' sizes are different. If RSS Key is not set for bonded device, it's not +changed on the slaves and default key for device is used. + +All settings are managed through the bonding port API and always are propagated +in one direction (from bonding to slaves). Link Status Change Interrupts / Polling ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ @@ -207,6 +228,15 @@ these parameters. A bonding device must have a minimum of one slave before the bonding device itself can be started. +To use a bonding device dynamic RSS configuration feature effectively, it is +also required, that all slaves should be RSS-capable and support, at least one +common hash function available for each of them. Changing RSS key is only +possible, when all slave devices support the same key size. + +To prevent inconsistency on how slaves process packets, once a device is added +to a bonding device, RSS configuration should be managed through the bonding +device API, and not directly on the slave. + Like all other PMD, all functions exported by a PMD are lock-free functions that are assumed not to be invoked in parallel on different logical cores to work on the same target object. -- 1.7.9.5 ^ permalink raw reply [flat|nested] 126+ messages in thread
* Re: [dpdk-dev] [PATCH v6 0/9] Dynamic RSS Configuration for Bonding 2015-10-16 10:00 ` [dpdk-dev] [PATCH v6 0/9] Dynamic RSS Configuration for Bonding Tomasz Kulasek ` (8 preceding siblings ...) 2015-10-16 10:00 ` [dpdk-dev] [PATCH v6 9/9] doc: dynamic rss configuration for bonding Tomasz Kulasek @ 2015-10-19 13:22 ` Declan Doherty 2015-10-30 14:25 ` [dpdk-dev] [PATCH v7 " Tomasz Kulasek 10 siblings, 0 replies; 126+ messages in thread From: Declan Doherty @ 2015-10-19 13:22 UTC (permalink / raw) To: Tomasz Kulasek, dev On 16/10/15 11:00, Tomasz Kulasek wrote: > OVERVIEW > -------- > 1) Setting .rxmode.mq_mode for bonding device to ETH_MQ_RX_RSS makes bonding > device fully RSS-capable, so all slaves are synchronized with its configuration. > This mode is intended to provide RSS configuration as known from "dynamic RSS > configuration for one port" and made slaves transparent for client application > implementation. > > 2) If .rxmode.mq_mode for bonding device isn't ETH_MQ_RX_RSS, slaves are not > synchronized. That provides an ability to configure them manually. This mode may > be useful when application wants to manage RSS in an unusual way and the > consistency of RSS configuration for slaves isn't required. > > Turning on/off RSS mode for slaves when bonding is started is not possible. > Other RSS configuration is propagated over slaves, when bonding device API is > used to do it. > > v6 changes: > - patchset reordered > - fixed forward dependency between patch 4/9 and 5/9 > > v5 changes: > - updated to DPDK 2.2 > - removed copyright change from null device source > - removed queue_stats_mapping_set from eth_dev_ops of bonding device > - null pmd cleanups (removed unnecessary malloc, replaced memcpy with > rte_memcpy) > - fixed queues number configuration in null pmd > > v4 changes: > - fixed copy-paste error, > - removed example application as too complex and introducing a new > dependency, > - addapted null pmd to be used as testing device for dynamic RSS configuration, > - addapted test units to use null pmd instead of ring pmd, > - ring pmd is not used and changed in this patchset > > v3 changes: > - checkpatch cleanups > > v2 changes: > - added support for keys other than 40 bytes long, > - now, if RSS key is not set for bonding, it is not set also for slaves, > - fix - full initial RSS configuration before any slave is added was not > possible due to the initially zeroed flow_type_rss_offloads for bonding, > - fix - changed error to warning when slave is synchronizing due to the > bonding's initial configuration (to allow use slaves' drivers not supporting > dynamic RSS configuration in bonding), > - some code cleanups, > - updated documentation, > > Tomasz Kulasek (9): > bonding: rss dynamic configuration > null: fix segfault when null_pmd added to bonding > null: extend number of virtual queues > null: export eth_dev_null_create > null: virtual dynamic rss configuration > test: dynamic rss configuration > bonding: per queue stats > doc: fixed spellings and typos > doc: dynamic rss configuration for bonding > > app/test/Makefile | 8 + > app/test/test_link_bonding_rssconf.c | 679 ++++++++++++++++++++ > .../prog_guide/link_bonding_poll_mode_drv_lib.rst | 42 +- > drivers/net/bonding/rte_eth_bond_api.c | 28 + > drivers/net/bonding/rte_eth_bond_pmd.c | 216 ++++++- > drivers/net/bonding/rte_eth_bond_private.h | 12 + > drivers/net/null/Makefile | 2 +- > drivers/net/null/rte_eth_null.c | 149 ++++- > drivers/net/null/rte_eth_null.h | 40 ++ > drivers/net/null/rte_pmd_null_version.map | 7 + > 10 files changed, 1151 insertions(+), 32 deletions(-) > create mode 100644 app/test/test_link_bonding_rssconf.c > create mode 100644 drivers/net/null/rte_eth_null.h > Series Acked-by: Declan Doherty <declan.doherty@intel.com> ^ permalink raw reply [flat|nested] 126+ messages in thread
* [dpdk-dev] [PATCH v7 0/9] Dynamic RSS Configuration for Bonding 2015-10-16 10:00 ` [dpdk-dev] [PATCH v6 0/9] Dynamic RSS Configuration for Bonding Tomasz Kulasek ` (9 preceding siblings ...) 2015-10-19 13:22 ` [dpdk-dev] [PATCH v6 0/9] Dynamic RSS Configuration for Bonding Declan Doherty @ 2015-10-30 14:25 ` Tomasz Kulasek 2015-10-30 14:25 ` [dpdk-dev] [PATCH v7 1/9] bonding: rss dynamic configuration Tomasz Kulasek ` (9 more replies) 10 siblings, 10 replies; 126+ messages in thread From: Tomasz Kulasek @ 2015-10-30 14:25 UTC (permalink / raw) To: dev OVERVIEW -------- 1) Setting .rxmode.mq_mode for bonding device to ETH_MQ_RX_RSS makes bonding device fully RSS-capable, so all slaves are synchronized with its configuration. This mode is intended to provide RSS configuration as known from "dynamic RSS configuration for one port" and made slaves transparent for client application implementation. 2) If .rxmode.mq_mode for bonding device isn't ETH_MQ_RX_RSS, slaves are not synchronized. That provides an ability to configure them manually. This mode may be useful when application wants to manage RSS in an unusual way and the consistency of RSS configuration for slaves isn't required. Turning on/off RSS mode for slaves when bonding is started is not possible. Other RSS configuration is propagated over slaves, when bonding device API is used to do it. v7 changes: - Makefile for test application changed to not fail when requirements for test units not met v6 changes: - patchset reordered - fixed forward dependency between patch 4/9 and 5/9 v5 changes: - updated to DPDK 2.2 - removed copyright change from null device source - removed queue_stats_mapping_set from eth_dev_ops of bonding device - null pmd cleanups (removed unnecessary malloc, replaced memcpy with rte_memcpy) - fixed queues number configuration in null pmd v4 changes: - fixed copy-paste error, - removed example application as too complex and introducing a new dependency, - addapted null pmd to be used as testing device for dynamic RSS configuration, - addapted test units to use null pmd instead of ring pmd, - ring pmd is not used and changed in this patchset v3 changes: - checkpatch cleanups v2 changes: - added support for keys other than 40 bytes long, - now, if RSS key is not set for bonding, it is not set also for slaves, - fix - full initial RSS configuration before any slave is added was not possible due to the initially zeroed flow_type_rss_offloads for bonding, - fix - changed error to warning when slave is synchronizing due to the bonding's initial configuration (to allow use slaves' drivers not supporting dynamic RSS configuration in bonding), - some code cleanups, - updated documentation, Tomasz Kulasek (9): bonding: rss dynamic configuration null: fix segfault when null_pmd added to bonding null: extend number of virtual queues null: export eth_dev_null_create null: virtual dynamic rss configuration test: dynamic rss configuration bonding: per queue stats doc: fixed spellings and typos doc: dynamic rss configuration for bonding app/test/Makefile | 7 + app/test/test_link_bonding_rssconf.c | 679 ++++++++++++++++++++ .../prog_guide/link_bonding_poll_mode_drv_lib.rst | 42 +- drivers/net/bonding/rte_eth_bond_api.c | 28 + drivers/net/bonding/rte_eth_bond_pmd.c | 216 ++++++- drivers/net/bonding/rte_eth_bond_private.h | 12 + drivers/net/null/Makefile | 2 +- drivers/net/null/rte_eth_null.c | 149 ++++- drivers/net/null/rte_eth_null.h | 40 ++ drivers/net/null/rte_pmd_null_version.map | 7 + 10 files changed, 1150 insertions(+), 32 deletions(-) create mode 100644 app/test/test_link_bonding_rssconf.c create mode 100644 drivers/net/null/rte_eth_null.h -- 1.7.9.5 ^ permalink raw reply [flat|nested] 126+ messages in thread
* [dpdk-dev] [PATCH v7 1/9] bonding: rss dynamic configuration 2015-10-30 14:25 ` [dpdk-dev] [PATCH v7 " Tomasz Kulasek @ 2015-10-30 14:25 ` Tomasz Kulasek 2015-10-30 14:25 ` [dpdk-dev] [PATCH v7 2/9] null: fix segfault when null_pmd added to bonding Tomasz Kulasek ` (8 subsequent siblings) 9 siblings, 0 replies; 126+ messages in thread From: Tomasz Kulasek @ 2015-10-30 14:25 UTC (permalink / raw) To: dev Bonding device implements independent management of RSS settings. It stores its own copies of settings i.e. RETA, RSS hash function and RSS key. It’s required to ensure consistency. 1) RSS hash function set for bonding device is maximal set of RSS hash functions supported by all bonded devices. That mean, to have RSS support for bonding, all slaves should be RSS-capable. 2) RSS key is propagated over the slaves "as is". 3) RETA for bonding is an internal table managed by bonding API, and is used as a pattern to set up slaves. Its size is GCD of all RETA sizes, so it can be easily used as a pattern providing expected behavior, even if slaves RETA sizes are different. Signed-off-by: Tomasz Kulasek <tomaszx.kulasek@intel.com> Acked-by: Declan Doherty <declan.doherty@intel.com> --- drivers/net/bonding/rte_eth_bond_api.c | 28 ++++ drivers/net/bonding/rte_eth_bond_pmd.c | 205 ++++++++++++++++++++++++++-- drivers/net/bonding/rte_eth_bond_private.h | 12 ++ 3 files changed, 231 insertions(+), 14 deletions(-) diff --git a/drivers/net/bonding/rte_eth_bond_api.c b/drivers/net/bonding/rte_eth_bond_api.c index 0681d1a..92073df 100644 --- a/drivers/net/bonding/rte_eth_bond_api.c +++ b/drivers/net/bonding/rte_eth_bond_api.c @@ -273,6 +273,9 @@ rte_eth_bond_create(const char *name, uint8_t mode, uint8_t socket_id) internals->rx_offload_capa = 0; internals->tx_offload_capa = 0; + /* Initially allow to choose any offload type */ + internals->flow_type_rss_offloads = ETH_RSS_PROTO_MASK; + memset(internals->active_slaves, 0, sizeof(internals->active_slaves)); memset(internals->slaves, 0, sizeof(internals->slaves)); @@ -369,6 +372,11 @@ __eth_bond_slave_add_lock_free(uint8_t bonded_port_id, uint8_t slave_port_id) rte_eth_dev_info_get(slave_port_id, &dev_info); + /* We need to store slaves reta_size to be able to synchronize RETA for all + * slave devices even if its sizes are different. + */ + internals->slaves[internals->slave_count].reta_size = dev_info.reta_size; + if (internals->slave_count < 1) { /* if MAC is not user defined then use MAC of first slave add to * bonded device */ @@ -382,9 +390,16 @@ __eth_bond_slave_add_lock_free(uint8_t bonded_port_id, uint8_t slave_port_id) /* Make primary slave */ internals->primary_port = slave_port_id; + /* Inherit queues settings from first slave */ + internals->nb_rx_queues = slave_eth_dev->data->nb_rx_queues; + internals->nb_tx_queues = slave_eth_dev->data->nb_tx_queues; + + internals->reta_size = dev_info.reta_size; + /* Take the first dev's offload capabilities */ internals->rx_offload_capa = dev_info.rx_offload_capa; internals->tx_offload_capa = dev_info.tx_offload_capa; + internals->flow_type_rss_offloads = dev_info.flow_type_rss_offloads; } else { /* Check slave link properties are supported if props are set, @@ -403,8 +418,19 @@ __eth_bond_slave_add_lock_free(uint8_t bonded_port_id, uint8_t slave_port_id) } internals->rx_offload_capa &= dev_info.rx_offload_capa; internals->tx_offload_capa &= dev_info.tx_offload_capa; + internals->flow_type_rss_offloads &= dev_info.flow_type_rss_offloads; + + /* RETA size is GCD of all slaves RETA sizes, so, if all sizes will be + * the power of 2, the lower one is GCD + */ + if (internals->reta_size > dev_info.reta_size) + internals->reta_size = dev_info.reta_size; + } + bonded_eth_dev->data->dev_conf.rx_adv_conf.rss_conf.rss_hf &= + internals->flow_type_rss_offloads; + internals->slave_count++; /* Update all slave devices MACs*/ @@ -531,6 +557,8 @@ __eth_bond_slave_remove_lock_free(uint8_t bonded_port_id, uint8_t slave_port_id) if (internals->slave_count == 0) { internals->rx_offload_capa = 0; internals->tx_offload_capa = 0; + internals->flow_type_rss_offloads = ETH_RSS_PROTO_MASK; + internals->reta_size = 0; } return 0; } diff --git a/drivers/net/bonding/rte_eth_bond_pmd.c b/drivers/net/bonding/rte_eth_bond_pmd.c index 5cc6372..2880f5c 100644 --- a/drivers/net/bonding/rte_eth_bond_pmd.c +++ b/drivers/net/bonding/rte_eth_bond_pmd.c @@ -1310,6 +1310,23 @@ slave_configure(struct rte_eth_dev *bonded_eth_dev, if (slave_eth_dev->driver->pci_drv.drv_flags & RTE_PCI_DRV_INTR_LSC) slave_eth_dev->data->dev_conf.intr_conf.lsc = 1; + /* If RSS is enabled for bonding, try to enable it for slaves */ + if (bonded_eth_dev->data->dev_conf.rxmode.mq_mode & ETH_MQ_RX_RSS) { + if (bonded_eth_dev->data->dev_conf.rx_adv_conf.rss_conf.rss_key_len + != 0) { + slave_eth_dev->data->dev_conf.rx_adv_conf.rss_conf.rss_key_len = + bonded_eth_dev->data->dev_conf.rx_adv_conf.rss_conf.rss_key_len; + slave_eth_dev->data->dev_conf.rx_adv_conf.rss_conf.rss_key = + bonded_eth_dev->data->dev_conf.rx_adv_conf.rss_conf.rss_key; + } else { + slave_eth_dev->data->dev_conf.rx_adv_conf.rss_conf.rss_key = NULL; + } + + slave_eth_dev->data->dev_conf.rx_adv_conf.rss_conf.rss_hf = + bonded_eth_dev->data->dev_conf.rx_adv_conf.rss_conf.rss_hf; + slave_eth_dev->data->dev_conf.rxmode.mq_mode |= ETH_MQ_RX_RSS; + } + /* Configure device */ errval = rte_eth_dev_configure(slave_eth_dev->data->port_id, bonded_eth_dev->data->nb_rx_queues, @@ -1361,6 +1378,30 @@ slave_configure(struct rte_eth_dev *bonded_eth_dev, return -1; } + /* If RSS is enabled for bonding, synchronize RETA */ + if (bonded_eth_dev->data->dev_conf.rxmode.mq_mode & ETH_MQ_RX_RSS) { + int i; + struct bond_dev_private *internals; + + internals = bonded_eth_dev->data->dev_private; + + for (i = 0; i < internals->slave_count; i++) { + if (internals->slaves[i].port_id == slave_eth_dev->data->port_id) { + errval = rte_eth_dev_rss_reta_update( + slave_eth_dev->data->port_id, + &internals->reta_conf[0], + internals->slaves[i].reta_size); + if (errval != 0) { + RTE_LOG(WARNING, PMD, + "rte_eth_dev_rss_reta_update on slave port %d fails (err %d)." + " RSS Configuration for bonding may be inconsistent.\n", + slave_eth_dev->data->port_id, errval); + } + break; + } + } + } + /* If lsc interrupt is set, check initial slave's link status */ if (slave_eth_dev->driver->pci_drv.drv_flags & RTE_PCI_DRV_INTR_LSC) bond_ethdev_lsc_event_callback(slave_eth_dev->data->port_id, @@ -1596,6 +1637,9 @@ bond_ethdev_info(struct rte_eth_dev *dev, struct rte_eth_dev_info *dev_info) dev_info->rx_offload_capa = internals->rx_offload_capa; dev_info->tx_offload_capa = internals->tx_offload_capa; + dev_info->flow_type_rss_offloads = internals->flow_type_rss_offloads; + + dev_info->reta_size = internals->reta_size; } static int @@ -1977,21 +2021,132 @@ bond_ethdev_lsc_event_callback(uint8_t port_id, enum rte_eth_event_type type, } } +static int +bond_ethdev_rss_reta_update(struct rte_eth_dev *dev, + struct rte_eth_rss_reta_entry64 *reta_conf, uint16_t reta_size) +{ + unsigned i, j; + int result = 0; + int slave_reta_size; + unsigned reta_count; + struct bond_dev_private *internals = dev->data->dev_private; + + if (reta_size != internals->reta_size) + return -EINVAL; + + /* Copy RETA table */ + reta_count = reta_size / RTE_RETA_GROUP_SIZE; + + for (i = 0; i < reta_count; i++) { + internals->reta_conf[i].mask = reta_conf[i].mask; + for (j = 0; j < RTE_RETA_GROUP_SIZE; j++) + if ((reta_conf[i].mask >> j) & 0x01) + internals->reta_conf[i].reta[j] = reta_conf[i].reta[j]; + } + + /* Fill rest of array */ + for (; i < RTE_DIM(internals->reta_conf); i += reta_count) + memcpy(&internals->reta_conf[i], &internals->reta_conf[0], + sizeof(internals->reta_conf[0]) * reta_count); + + /* Propagate RETA over slaves */ + for (i = 0; i < internals->slave_count; i++) { + slave_reta_size = internals->slaves[i].reta_size; + result = rte_eth_dev_rss_reta_update(internals->slaves[i].port_id, + &internals->reta_conf[0], slave_reta_size); + if (result < 0) + return result; + } + + return 0; +} + +static int +bond_ethdev_rss_reta_query(struct rte_eth_dev *dev, + struct rte_eth_rss_reta_entry64 *reta_conf, uint16_t reta_size) +{ + int i, j; + struct bond_dev_private *internals = dev->data->dev_private; + + if (reta_size != internals->reta_size) + return -EINVAL; + + /* Copy RETA table */ + for (i = 0; i < reta_size / RTE_RETA_GROUP_SIZE; i++) + for (j = 0; j < RTE_RETA_GROUP_SIZE; j++) + if ((reta_conf[i].mask >> j) & 0x01) + reta_conf[i].reta[j] = internals->reta_conf[i].reta[j]; + + return 0; +} + +static int +bond_ethdev_rss_hash_update(struct rte_eth_dev *dev, + struct rte_eth_rss_conf *rss_conf) +{ + int i, result = 0; + struct bond_dev_private *internals = dev->data->dev_private; + struct rte_eth_rss_conf bond_rss_conf; + + memcpy(&bond_rss_conf, rss_conf, sizeof(struct rte_eth_rss_conf)); + + bond_rss_conf.rss_hf &= internals->flow_type_rss_offloads; + + if (bond_rss_conf.rss_hf != 0) + dev->data->dev_conf.rx_adv_conf.rss_conf.rss_hf = bond_rss_conf.rss_hf; + + if (bond_rss_conf.rss_key && bond_rss_conf.rss_key_len < + sizeof(internals->rss_key)) { + if (bond_rss_conf.rss_key_len == 0) + bond_rss_conf.rss_key_len = 40; + internals->rss_key_len = bond_rss_conf.rss_key_len; + memcpy(internals->rss_key, bond_rss_conf.rss_key, + internals->rss_key_len); + } + + for (i = 0; i < internals->slave_count; i++) { + result = rte_eth_dev_rss_hash_update(internals->slaves[i].port_id, + &bond_rss_conf); + if (result < 0) + return result; + } + + return 0; +} + +static int +bond_ethdev_rss_hash_conf_get(struct rte_eth_dev *dev, + struct rte_eth_rss_conf *rss_conf) +{ + struct bond_dev_private *internals = dev->data->dev_private; + + rss_conf->rss_hf = dev->data->dev_conf.rx_adv_conf.rss_conf.rss_hf; + rss_conf->rss_key_len = internals->rss_key_len; + if (rss_conf->rss_key) + memcpy(rss_conf->rss_key, internals->rss_key, internals->rss_key_len); + + return 0; +} + struct eth_dev_ops default_dev_ops = { - .dev_start = bond_ethdev_start, - .dev_stop = bond_ethdev_stop, - .dev_close = bond_ethdev_close, - .dev_configure = bond_ethdev_configure, - .dev_infos_get = bond_ethdev_info, - .rx_queue_setup = bond_ethdev_rx_queue_setup, - .tx_queue_setup = bond_ethdev_tx_queue_setup, - .rx_queue_release = bond_ethdev_rx_queue_release, - .tx_queue_release = bond_ethdev_tx_queue_release, - .link_update = bond_ethdev_link_update, - .stats_get = bond_ethdev_stats_get, - .stats_reset = bond_ethdev_stats_reset, - .promiscuous_enable = bond_ethdev_promiscuous_enable, - .promiscuous_disable = bond_ethdev_promiscuous_disable + .dev_start = bond_ethdev_start, + .dev_stop = bond_ethdev_stop, + .dev_close = bond_ethdev_close, + .dev_configure = bond_ethdev_configure, + .dev_infos_get = bond_ethdev_info, + .rx_queue_setup = bond_ethdev_rx_queue_setup, + .tx_queue_setup = bond_ethdev_tx_queue_setup, + .rx_queue_release = bond_ethdev_rx_queue_release, + .tx_queue_release = bond_ethdev_tx_queue_release, + .link_update = bond_ethdev_link_update, + .stats_get = bond_ethdev_stats_get, + .stats_reset = bond_ethdev_stats_reset, + .promiscuous_enable = bond_ethdev_promiscuous_enable, + .promiscuous_disable = bond_ethdev_promiscuous_disable, + .reta_update = bond_ethdev_rss_reta_update, + .reta_query = bond_ethdev_rss_reta_query, + .rss_hash_update = bond_ethdev_rss_hash_update, + .rss_hash_conf_get = bond_ethdev_rss_hash_conf_get }; static int @@ -2090,6 +2245,28 @@ bond_ethdev_configure(struct rte_eth_dev *dev) int arg_count; uint8_t port_id = dev - rte_eth_devices; + static const uint8_t default_rss_key[40] = { + 0x6D, 0x5A, 0x56, 0xDA, 0x25, 0x5B, 0x0E, 0xC2, 0x41, 0x67, 0x25, 0x3D, + 0x43, 0xA3, 0x8F, 0xB0, 0xD0, 0xCA, 0x2B, 0xCB, 0xAE, 0x7B, 0x30, 0xB4, + 0x77, 0xCB, 0x2D, 0xA3, 0x80, 0x30, 0xF2, 0x0C, 0x6A, 0x42, 0xB7, 0x3B, + 0xBE, 0xAC, 0x01, 0xFA + }; + + unsigned i, j; + + /* If RSS is enabled, fill table and key with default values */ + if (dev->data->dev_conf.rxmode.mq_mode & ETH_MQ_RX_RSS) { + dev->data->dev_conf.rx_adv_conf.rss_conf.rss_key = internals->rss_key; + dev->data->dev_conf.rx_adv_conf.rss_conf.rss_key_len = 0; + memcpy(internals->rss_key, default_rss_key, 40); + + for (i = 0; i < RTE_DIM(internals->reta_conf); i++) { + internals->reta_conf[i].mask = ~0LL; + for (j = 0; j < RTE_RETA_GROUP_SIZE; j++) + internals->reta_conf[i].reta[j] = j % dev->data->nb_rx_queues; + } + } + /* * if no kvlist, it means that this bonded device has been created * through the bonding api. diff --git a/drivers/net/bonding/rte_eth_bond_private.h b/drivers/net/bonding/rte_eth_bond_private.h index 038bca6..e7af809 100644 --- a/drivers/net/bonding/rte_eth_bond_private.h +++ b/drivers/net/bonding/rte_eth_bond_private.h @@ -103,6 +103,8 @@ struct bond_slave_details { uint8_t last_link_status; /**< Port Id of slave eth_dev */ struct ether_addr persisted_mac_addr; + + uint16_t reta_size; }; @@ -155,6 +157,16 @@ struct bond_dev_private { uint32_t rx_offload_capa; /** Rx offload capability */ uint32_t tx_offload_capa; /** Tx offload capability */ + /** Bit mask of RSS offloads, the bit offset also means flow type */ + uint64_t flow_type_rss_offloads; + + uint16_t reta_size; + struct rte_eth_rss_reta_entry64 reta_conf[ETH_RSS_RETA_SIZE_512 / + RTE_RETA_GROUP_SIZE]; + + uint8_t rss_key[52]; /**< 52-byte hash key buffer. */ + uint8_t rss_key_len; /**< hash key length in bytes. */ + struct rte_kvargs *kvlist; uint8_t slave_update_idx; }; -- 1.7.9.5 ^ permalink raw reply [flat|nested] 126+ messages in thread
* [dpdk-dev] [PATCH v7 2/9] null: fix segfault when null_pmd added to bonding 2015-10-30 14:25 ` [dpdk-dev] [PATCH v7 " Tomasz Kulasek 2015-10-30 14:25 ` [dpdk-dev] [PATCH v7 1/9] bonding: rss dynamic configuration Tomasz Kulasek @ 2015-10-30 14:25 ` Tomasz Kulasek 2015-10-30 14:25 ` [dpdk-dev] [PATCH v7 3/9] null: extend number of virtual queues Tomasz Kulasek ` (7 subsequent siblings) 9 siblings, 0 replies; 126+ messages in thread From: Tomasz Kulasek @ 2015-10-30 14:25 UTC (permalink / raw) To: dev This patch initializes eth_dev->link_intr_cbs queue used when null pmd is added to the bonding. v5 changes: - removed unnecessary malloc for eth_driver (rte_null_pmd) Signed-off-by: Tomasz Kulasek <tomaszx.kulasek@intel.com> Acked-by: Tetsuya Mukawa <mukawa@igel.co.jp> --- drivers/net/null/rte_eth_null.c | 2 ++ 1 file changed, 2 insertions(+) diff --git a/drivers/net/null/rte_eth_null.c b/drivers/net/null/rte_eth_null.c index e244595..c748101 100644 --- a/drivers/net/null/rte_eth_null.c +++ b/drivers/net/null/rte_eth_null.c @@ -432,6 +432,7 @@ eth_dev_null_create(const char *name, internals->numa_node = numa_node; pci_dev->numa_node = numa_node; + pci_dev->driver = &rte_null_pmd.pci_drv; data->dev_private = internals; data->port_id = eth_dev->data->port_id; @@ -445,6 +446,7 @@ eth_dev_null_create(const char *name, eth_dev->dev_ops = &ops; eth_dev->pci_dev = pci_dev; eth_dev->driver = &rte_null_pmd; + TAILQ_INIT(ð_dev->link_intr_cbs); /* finally assign rx and tx ops */ if (packet_copy) { -- 1.7.9.5 ^ permalink raw reply [flat|nested] 126+ messages in thread
* [dpdk-dev] [PATCH v7 3/9] null: extend number of virtual queues 2015-10-30 14:25 ` [dpdk-dev] [PATCH v7 " Tomasz Kulasek 2015-10-30 14:25 ` [dpdk-dev] [PATCH v7 1/9] bonding: rss dynamic configuration Tomasz Kulasek 2015-10-30 14:25 ` [dpdk-dev] [PATCH v7 2/9] null: fix segfault when null_pmd added to bonding Tomasz Kulasek @ 2015-10-30 14:25 ` Tomasz Kulasek 2015-10-30 14:25 ` [dpdk-dev] [PATCH v7 4/9] null: export eth_dev_null_create Tomasz Kulasek ` (6 subsequent siblings) 9 siblings, 0 replies; 126+ messages in thread From: Tomasz Kulasek @ 2015-10-30 14:25 UTC (permalink / raw) To: dev This patch adds a possibility to configure more than one queue on null device. v5 changes: - fixed queues number configuration (using internals->nb_*_queues instead of dev->data->nb_*_queues) Signed-off-by: Tomasz Kulasek <tomaszx.kulasek@intel.com> Acked-by: Tetsuya Mukawa <mukawa@igel.co.jp> --- drivers/net/null/rte_eth_null.c | 28 +++++++++++++++++++--------- 1 file changed, 19 insertions(+), 9 deletions(-) diff --git a/drivers/net/null/rte_eth_null.c b/drivers/net/null/rte_eth_null.c index c748101..bf81b1b 100644 --- a/drivers/net/null/rte_eth_null.c +++ b/drivers/net/null/rte_eth_null.c @@ -71,8 +71,8 @@ struct pmd_internals { unsigned nb_rx_queues; unsigned nb_tx_queues; - struct null_queue rx_null_queues[1]; - struct null_queue tx_null_queues[1]; + struct null_queue rx_null_queues[RTE_MAX_QUEUES_PER_PORT]; + struct null_queue tx_null_queues[RTE_MAX_QUEUES_PER_PORT]; }; @@ -178,7 +178,15 @@ eth_null_copy_tx(void *q, struct rte_mbuf **bufs, uint16_t nb_bufs) } static int -eth_dev_configure(struct rte_eth_dev *dev __rte_unused) { return 0; } +eth_dev_configure(struct rte_eth_dev *dev) { + struct pmd_internals *internals; + + internals = dev->data->dev_private; + internals->nb_rx_queues = dev->data->nb_rx_queues; + internals->nb_tx_queues = dev->data->nb_tx_queues; + + return 0; +} static int eth_dev_start(struct rte_eth_dev *dev) @@ -213,10 +221,11 @@ eth_rx_queue_setup(struct rte_eth_dev *dev, uint16_t rx_queue_id, if ((dev == NULL) || (mb_pool == NULL)) return -EINVAL; - if (rx_queue_id != 0) + internals = dev->data->dev_private; + + if (rx_queue_id >= internals->nb_rx_queues) return -ENODEV; - internals = dev->data->dev_private; packet_size = internals->packet_size; internals->rx_null_queues[rx_queue_id].mb_pool = mb_pool; @@ -246,10 +255,11 @@ eth_tx_queue_setup(struct rte_eth_dev *dev, uint16_t tx_queue_id, if (dev == NULL) return -EINVAL; - if (tx_queue_id != 0) + internals = dev->data->dev_private; + + if (tx_queue_id >= internals->nb_tx_queues) return -ENODEV; - internals = dev->data->dev_private; packet_size = internals->packet_size; dev->data->tx_queues[tx_queue_id] = @@ -279,8 +289,8 @@ eth_dev_info(struct rte_eth_dev *dev, dev_info->driver_name = drivername; dev_info->max_mac_addrs = 1; dev_info->max_rx_pktlen = (uint32_t)-1; - dev_info->max_rx_queues = (uint16_t)internals->nb_rx_queues; - dev_info->max_tx_queues = (uint16_t)internals->nb_tx_queues; + dev_info->max_rx_queues = RTE_DIM(internals->rx_null_queues); + dev_info->max_tx_queues = RTE_DIM(internals->tx_null_queues); dev_info->min_rx_bufsize = 0; dev_info->pci_dev = NULL; } -- 1.7.9.5 ^ permalink raw reply [flat|nested] 126+ messages in thread
* [dpdk-dev] [PATCH v7 4/9] null: export eth_dev_null_create 2015-10-30 14:25 ` [dpdk-dev] [PATCH v7 " Tomasz Kulasek ` (2 preceding siblings ...) 2015-10-30 14:25 ` [dpdk-dev] [PATCH v7 3/9] null: extend number of virtual queues Tomasz Kulasek @ 2015-10-30 14:25 ` Tomasz Kulasek 2015-10-30 14:25 ` [dpdk-dev] [PATCH v7 5/9] null: virtual dynamic rss configuration Tomasz Kulasek ` (5 subsequent siblings) 9 siblings, 0 replies; 126+ messages in thread From: Tomasz Kulasek @ 2015-10-30 14:25 UTC (permalink / raw) To: dev v6 changes: - reordered with patch 5/9 - fixed forward dependency to patch patch 5/9 Signed-off-by: Tomasz Kulasek <tomaszx.kulasek@intel.com> Acked-by: Tetsuya Mukawa <mukawa@igel.co.jp> --- drivers/net/null/Makefile | 2 +- drivers/net/null/rte_eth_null.c | 4 ++- drivers/net/null/rte_eth_null.h | 40 +++++++++++++++++++++++++++++ drivers/net/null/rte_pmd_null_version.map | 7 +++++ 4 files changed, 51 insertions(+), 2 deletions(-) create mode 100644 drivers/net/null/rte_eth_null.h diff --git a/drivers/net/null/Makefile b/drivers/net/null/Makefile index 96ba01c..2202389 100644 --- a/drivers/net/null/Makefile +++ b/drivers/net/null/Makefile @@ -51,7 +51,7 @@ SRCS-$(CONFIG_RTE_LIBRTE_PMD_NULL) += rte_eth_null.c # # Export include files # -SYMLINK-y-include += +SYMLINK-y-include += rte_eth_null.h # this lib depends upon: DEPDIRS-$(CONFIG_RTE_LIBRTE_PMD_NULL) += lib/librte_mbuf diff --git a/drivers/net/null/rte_eth_null.c b/drivers/net/null/rte_eth_null.c index bf81b1b..236d998 100644 --- a/drivers/net/null/rte_eth_null.c +++ b/drivers/net/null/rte_eth_null.c @@ -38,6 +38,8 @@ #include <rte_dev.h> #include <rte_kvargs.h> +#include "rte_eth_null.h" + #define ETH_NULL_PACKET_SIZE_ARG "size" #define ETH_NULL_PACKET_COPY_ARG "copy" @@ -387,7 +389,7 @@ static const struct eth_dev_ops ops = { .stats_reset = eth_stats_reset, }; -static int +int eth_dev_null_create(const char *name, const unsigned numa_node, unsigned packet_size, diff --git a/drivers/net/null/rte_eth_null.h b/drivers/net/null/rte_eth_null.h new file mode 100644 index 0000000..abada8c --- /dev/null +++ b/drivers/net/null/rte_eth_null.h @@ -0,0 +1,40 @@ +/*- + * BSD LICENSE + * + * Copyright(c) 2015 Intel Corporation. All rights reserved. + * All rights reserved. + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * + * * Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * * Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in + * the documentation and/or other materials provided with the + * distribution. + * * Neither the name of Intel Corporation nor the names of its + * contributors may be used to endorse or promote products derived + * from this software without specific prior written permission. + * + * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS + * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT + * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR + * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT + * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, + * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT + * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, + * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY + * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT + * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE + * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + */ + +#ifndef RTE_ETH_NULL_H_ +#define RTE_ETH_NULL_H_ + +int eth_dev_null_create(const char *name, const unsigned numa_node, + unsigned packet_size, unsigned packet_copy); + +#endif /* RTE_ETH_NULL_H_ */ diff --git a/drivers/net/null/rte_pmd_null_version.map b/drivers/net/null/rte_pmd_null_version.map index ef35398..84b1d0f 100644 --- a/drivers/net/null/rte_pmd_null_version.map +++ b/drivers/net/null/rte_pmd_null_version.map @@ -2,3 +2,10 @@ DPDK_2.0 { local: *; }; + +DPDK_2.2 { + global: + + eth_dev_null_create; + +} DPDK_2.0; -- 1.7.9.5 ^ permalink raw reply [flat|nested] 126+ messages in thread
* [dpdk-dev] [PATCH v7 5/9] null: virtual dynamic rss configuration 2015-10-30 14:25 ` [dpdk-dev] [PATCH v7 " Tomasz Kulasek ` (3 preceding siblings ...) 2015-10-30 14:25 ` [dpdk-dev] [PATCH v7 4/9] null: export eth_dev_null_create Tomasz Kulasek @ 2015-10-30 14:25 ` Tomasz Kulasek 2015-10-30 14:25 ` [dpdk-dev] [PATCH v7 6/9] test: " Tomasz Kulasek ` (4 subsequent siblings) 9 siblings, 0 replies; 126+ messages in thread From: Tomasz Kulasek @ 2015-10-30 14:25 UTC (permalink / raw) To: dev This implementation allows to set and read RSS configuration for null device, and is used to validate right values propagation over the slaves, in test units for dynamic RSS configuration for bonding. v6 changes: - reordered with patch 4/9 - recreated due to the changes in patch 4/9 v5 changes: - replaced memcpy with rte_memcpy Signed-off-by: Tomasz Kulasek <tomaszx.kulasek@intel.com> Acked-by: Tetsuya Mukawa <mukawa@igel.co.jp> --- drivers/net/null/rte_eth_null.c | 115 +++++++++++++++++++++++++++++++++++++++ 1 file changed, 115 insertions(+) diff --git a/drivers/net/null/rte_eth_null.c b/drivers/net/null/rte_eth_null.c index 236d998..64e9000 100644 --- a/drivers/net/null/rte_eth_null.c +++ b/drivers/net/null/rte_eth_null.c @@ -37,6 +37,7 @@ #include <rte_memcpy.h> #include <rte_dev.h> #include <rte_kvargs.h> +#include <rte_spinlock.h> #include "rte_eth_null.h" @@ -75,6 +76,17 @@ struct pmd_internals { struct null_queue rx_null_queues[RTE_MAX_QUEUES_PER_PORT]; struct null_queue tx_null_queues[RTE_MAX_QUEUES_PER_PORT]; + + /** Bit mask of RSS offloads, the bit offset also means flow type */ + uint64_t flow_type_rss_offloads; + + rte_spinlock_t rss_lock; + + uint16_t reta_size; + struct rte_eth_rss_reta_entry64 reta_conf[ETH_RSS_RETA_SIZE_128 / + RTE_RETA_GROUP_SIZE]; + + uint8_t rss_key[40]; /**< 40-byte hash key. */ }; @@ -295,6 +307,8 @@ eth_dev_info(struct rte_eth_dev *dev, dev_info->max_tx_queues = RTE_DIM(internals->tx_null_queues); dev_info->min_rx_bufsize = 0; dev_info->pci_dev = NULL; + dev_info->reta_size = internals->reta_size; + dev_info->flow_type_rss_offloads = internals->flow_type_rss_offloads; } static void @@ -375,6 +389,91 @@ static int eth_link_update(struct rte_eth_dev *dev __rte_unused, int wait_to_complete __rte_unused) { return 0; } +static int +eth_rss_reta_update(struct rte_eth_dev *dev, + struct rte_eth_rss_reta_entry64 *reta_conf, uint16_t reta_size) +{ + int i, j; + struct pmd_internals *internal = dev->data->dev_private; + + if (reta_size != internal->reta_size) + return -EINVAL; + + rte_spinlock_lock(&internal->rss_lock); + + /* Copy RETA table */ + for (i = 0; i < (internal->reta_size / RTE_RETA_GROUP_SIZE); i++) { + internal->reta_conf[i].mask = reta_conf[i].mask; + for (j = 0; j < RTE_RETA_GROUP_SIZE; j++) + if ((reta_conf[i].mask >> j) & 0x01) + internal->reta_conf[i].reta[j] = reta_conf[i].reta[j]; + } + + rte_spinlock_unlock(&internal->rss_lock); + + return 0; +} + +static int +eth_rss_reta_query(struct rte_eth_dev *dev, + struct rte_eth_rss_reta_entry64 *reta_conf, uint16_t reta_size) +{ + int i, j; + struct pmd_internals *internal = dev->data->dev_private; + + if (reta_size != internal->reta_size) + return -EINVAL; + + rte_spinlock_lock(&internal->rss_lock); + + /* Copy RETA table */ + for (i = 0; i < (internal->reta_size / RTE_RETA_GROUP_SIZE); i++) { + for (j = 0; j < RTE_RETA_GROUP_SIZE; j++) + if ((reta_conf[i].mask >> j) & 0x01) + reta_conf[i].reta[j] = internal->reta_conf[i].reta[j]; + } + + rte_spinlock_unlock(&internal->rss_lock); + + return 0; +} + +static int +eth_rss_hash_update(struct rte_eth_dev *dev, struct rte_eth_rss_conf *rss_conf) +{ + struct pmd_internals *internal = dev->data->dev_private; + + rte_spinlock_lock(&internal->rss_lock); + + if ((rss_conf->rss_hf & internal->flow_type_rss_offloads) != 0) + dev->data->dev_conf.rx_adv_conf.rss_conf.rss_hf = + rss_conf->rss_hf & internal->flow_type_rss_offloads; + + if (rss_conf->rss_key) + rte_memcpy(internal->rss_key, rss_conf->rss_key, 40); + + rte_spinlock_unlock(&internal->rss_lock); + + return 0; +} + +static int +eth_rss_hash_conf_get(struct rte_eth_dev *dev, + struct rte_eth_rss_conf *rss_conf) +{ + struct pmd_internals *internal = dev->data->dev_private; + + rte_spinlock_lock(&internal->rss_lock); + + rss_conf->rss_hf = dev->data->dev_conf.rx_adv_conf.rss_conf.rss_hf; + if (rss_conf->rss_key) + rte_memcpy(rss_conf->rss_key, internal->rss_key, 40); + + rte_spinlock_unlock(&internal->rss_lock); + + return 0; +} + static const struct eth_dev_ops ops = { .dev_start = eth_dev_start, .dev_stop = eth_dev_stop, @@ -387,6 +486,10 @@ static const struct eth_dev_ops ops = { .link_update = eth_link_update, .stats_get = eth_stats_get, .stats_reset = eth_stats_reset, + .reta_update = eth_rss_reta_update, + .reta_query = eth_rss_reta_query, + .rss_hash_update = eth_rss_hash_update, + .rss_hash_conf_get = eth_rss_hash_conf_get }; int @@ -402,6 +505,13 @@ eth_dev_null_create(const char *name, struct pmd_internals *internals = NULL; struct rte_eth_dev *eth_dev = NULL; + static const uint8_t default_rss_key[40] = { + 0x6D, 0x5A, 0x56, 0xDA, 0x25, 0x5B, 0x0E, 0xC2, 0x41, 0x67, 0x25, 0x3D, + 0x43, 0xA3, 0x8F, 0xB0, 0xD0, 0xCA, 0x2B, 0xCB, 0xAE, 0x7B, 0x30, 0xB4, + 0x77, 0xCB, 0x2D, 0xA3, 0x80, 0x30, 0xF2, 0x0C, 0x6A, 0x42, 0xB7, 0x3B, + 0xBE, 0xAC, 0x01, 0xFA + }; + if (name == NULL) return -EINVAL; @@ -443,6 +553,11 @@ eth_dev_null_create(const char *name, internals->packet_copy = packet_copy; internals->numa_node = numa_node; + internals->flow_type_rss_offloads = ETH_RSS_PROTO_MASK; + internals->reta_size = RTE_DIM(internals->reta_conf) * RTE_RETA_GROUP_SIZE; + + rte_memcpy(internals->rss_key, default_rss_key, 40); + pci_dev->numa_node = numa_node; pci_dev->driver = &rte_null_pmd.pci_drv; -- 1.7.9.5 ^ permalink raw reply [flat|nested] 126+ messages in thread
* [dpdk-dev] [PATCH v7 6/9] test: dynamic rss configuration 2015-10-30 14:25 ` [dpdk-dev] [PATCH v7 " Tomasz Kulasek ` (4 preceding siblings ...) 2015-10-30 14:25 ` [dpdk-dev] [PATCH v7 5/9] null: virtual dynamic rss configuration Tomasz Kulasek @ 2015-10-30 14:25 ` Tomasz Kulasek 2015-10-30 14:25 ` [dpdk-dev] [PATCH v7 7/9] bonding: per queue stats Tomasz Kulasek ` (3 subsequent siblings) 9 siblings, 0 replies; 126+ messages in thread From: Tomasz Kulasek @ 2015-10-30 14:25 UTC (permalink / raw) To: dev v7 changes: - Makefile for test app changed to not break compilation when CONFIG_RTE_LIBRTE_PMD_NULL=n (now module is silently omitted when requirements not met) Signed-off-by: Tomasz Kulasek <tomaszx.kulasek@intel.com> Acked-by: Declan Doherty <declan.doherty@intel.com> --- app/test/Makefile | 7 + app/test/test_link_bonding_rssconf.c | 679 ++++++++++++++++++++++++++++++++++ 2 files changed, 686 insertions(+) create mode 100644 app/test/test_link_bonding_rssconf.c diff --git a/app/test/Makefile b/app/test/Makefile index 294618f..ef9125d 100644 --- a/app/test/Makefile +++ b/app/test/Makefile @@ -140,6 +140,13 @@ SRCS-$(CONFIG_RTE_LIBRTE_PMD_BOND) += test_link_bonding.c SRCS-$(CONFIG_RTE_LIBRTE_PMD_BOND) += test_link_bonding_mode4.c endif +ifeq ($(CONFIG_RTE_LIBRTE_PMD_NULL),y) +SRCS-$(CONFIG_RTE_LIBRTE_PMD_BOND) += test_link_bonding_rssconf.c +ifeq ($(CONFIG_RTE_BUILD_SHARED_LIB),y) +LDLIBS += -lrte_pmd_null +endif +endif + SRCS-$(CONFIG_RTE_LIBRTE_PMD_RING) += test_pmd_ring.c SRCS-$(CONFIG_RTE_LIBRTE_KVARGS) += test_kvargs.c diff --git a/app/test/test_link_bonding_rssconf.c b/app/test/test_link_bonding_rssconf.c new file mode 100644 index 0000000..e6714b4 --- /dev/null +++ b/app/test/test_link_bonding_rssconf.c @@ -0,0 +1,679 @@ +/*- + * BSD LICENSE + * + * Copyright(c) 2015 Intel Corporation. All rights reserved. + * All rights reserved. + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * + * * Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * * Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in + * the documentation and/or other materials provided with the + * distribution. + * * Neither the name of Intel Corporation nor the names of its + * contributors may be used to endorse or promote products derived + * from this software without specific prior written permission. + * + * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS + * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT + * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR + * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT + * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, + * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT + * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, + * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY + * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT + * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE + * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + */ + +#include <string.h> +#include <stdarg.h> +#include <stdio.h> +#include <stdlib.h> +#include <stdint.h> +#include <inttypes.h> +#include <errno.h> +#include <rte_cycles.h> +#include <sys/queue.h> + +#include <rte_byteorder.h> +#include <rte_common.h> +#include <rte_debug.h> +#include <rte_ethdev.h> +#include <rte_log.h> +#include <rte_lcore.h> +#include <rte_memory.h> + +#include <rte_string_fns.h> +#include <rte_errno.h> +#include <rte_eth_bond.h> +#include <rte_eth_null.h> + +#include "test.h" + +#define SLAVE_COUNT (4) + +#define RXTX_RING_SIZE 1024 +#define RXTX_QUEUE_COUNT 4 + +#define BONDED_DEV_NAME ("rssconf_bond_dev") + +#define SLAVE_DEV_NAME_FMT ("rssconf_slave%d") +#define SLAVE_RXTX_QUEUE_FMT ("rssconf_slave%d_q%d") + +#define NUM_MBUFS 8191 +#define MBUF_SIZE (1600 + sizeof(struct rte_mbuf) + RTE_PKTMBUF_HEADROOM) +#define MBUF_CACHE_SIZE 250 +#define BURST_SIZE 32 + +#define INVALID_SOCKET_ID (-1) +#define INVALID_PORT_ID (0xFF) +#define INVALID_BONDING_MODE (-1) + +struct slave_conf { + uint8_t port_id; + struct rte_eth_dev_info dev_info; + + struct rte_eth_rss_conf rss_conf; + uint8_t rss_key[40]; + struct rte_eth_rss_reta_entry64 reta_conf[512 / RTE_RETA_GROUP_SIZE]; + + uint8_t is_slave; + struct rte_ring *rxtx_queue[RXTX_QUEUE_COUNT]; +}; + +struct link_bonding_rssconf_unittest_params { + uint8_t bond_port_id; + struct rte_eth_dev_info bond_dev_info; + struct rte_eth_rss_reta_entry64 bond_reta_conf[512 / RTE_RETA_GROUP_SIZE]; + struct slave_conf slave_ports[SLAVE_COUNT]; + + struct rte_mempool *mbuf_pool; +}; + +static struct link_bonding_rssconf_unittest_params test_params = { + .bond_port_id = INVALID_PORT_ID, + .slave_ports = { + [0 ... SLAVE_COUNT - 1] = { .port_id = INVALID_PORT_ID, .is_slave = 0} + }, + .mbuf_pool = NULL, +}; + +/** + * Default port configuration with RSS turned off + */ +static struct rte_eth_conf default_pmd_conf = { + .rxmode = { + .mq_mode = ETH_MQ_RX_NONE, + .max_rx_pkt_len = ETHER_MAX_LEN, + .split_hdr_size = 0, + .header_split = 0, /**< Header Split disabled */ + .hw_ip_checksum = 0, /**< IP checksum offload enabled */ + .hw_vlan_filter = 0, /**< VLAN filtering disabled */ + .jumbo_frame = 0, /**< Jumbo Frame Support disabled */ + .hw_strip_crc = 0, /**< CRC stripped by hardware */ + }, + .txmode = { + .mq_mode = ETH_MQ_TX_NONE, + }, + .lpbk_mode = 0, +}; + +static struct rte_eth_conf rss_pmd_conf = { + .rxmode = { + .mq_mode = ETH_MQ_RX_RSS, + .max_rx_pkt_len = ETHER_MAX_LEN, + .split_hdr_size = 0, + .header_split = 0, /**< Header Split disabled */ + .hw_ip_checksum = 0, /**< IP checksum offload enabled */ + .hw_vlan_filter = 0, /**< VLAN filtering disabled */ + .jumbo_frame = 0, /**< Jumbo Frame Support disabled */ + .hw_strip_crc = 0, /**< CRC stripped by hardware */ + }, + .txmode = { + .mq_mode = ETH_MQ_TX_NONE, + }, + .rx_adv_conf = { + .rss_conf = { + .rss_key = NULL, + .rss_hf = ETH_RSS_IPV6, + }, + }, + .lpbk_mode = 0, +}; + +#define FOR_EACH(_i, _item, _array, _size) \ + for (_i = 0, _item = &_array[0]; _i < _size && (_item = &_array[_i]); _i++) + +/* Macro for iterating over every port that can be used as a slave + * in this test. + * _i variable used as an index in test_params->slave_ports + * _slave pointer to &test_params->slave_ports[_idx] + */ +#define FOR_EACH_PORT(_i, _port) \ + FOR_EACH(_i, _port, test_params.slave_ports, \ + RTE_DIM(test_params.slave_ports)) + +static int +configure_ethdev(uint8_t port_id, struct rte_eth_conf *eth_conf, uint8_t start) +{ + int rxq, txq; + + TEST_ASSERT(rte_eth_dev_configure(port_id, RXTX_QUEUE_COUNT, + RXTX_QUEUE_COUNT, eth_conf) == 0, "Failed to configure device %u", + port_id); + + for (rxq = 0; rxq < RXTX_QUEUE_COUNT; rxq++) { + TEST_ASSERT(rte_eth_rx_queue_setup(port_id, rxq, RXTX_RING_SIZE, + rte_eth_dev_socket_id(port_id), NULL, + test_params.mbuf_pool) == 0, "Failed to setup rx queue."); + } + + for (txq = 0; txq < RXTX_QUEUE_COUNT; txq++) { + TEST_ASSERT(rte_eth_tx_queue_setup(port_id, txq, RXTX_RING_SIZE, + rte_eth_dev_socket_id(port_id), NULL) == 0, + "Failed to setup tx queue."); + } + + if (start) { + TEST_ASSERT(rte_eth_dev_start(port_id) == 0, + "Failed to start device (%d).", port_id); + } + + return 0; +} + +/** + * Remove all slaves from bonding + */ +static int +remove_slaves(void) +{ + unsigned n; + struct slave_conf *port; + + FOR_EACH_PORT(n, port) { + port = &test_params.slave_ports[n]; + if (port->is_slave) { + TEST_ASSERT_SUCCESS(rte_eth_bond_slave_remove( + test_params.bond_port_id, port->port_id), + "Cannot remove slave %d from bonding", port->port_id); + port->is_slave = 0; + } + } + + return 0; +} + +static int +remove_slaves_and_stop_bonded_device(void) +{ + TEST_ASSERT_SUCCESS(remove_slaves(), "Removing slaves"); + rte_eth_dev_stop(test_params.bond_port_id); + return TEST_SUCCESS; +} + +/** + * Add all slaves to bonding + */ +static int +bond_slaves(void) +{ + unsigned n; + struct slave_conf *port; + + FOR_EACH_PORT(n, port) { + port = &test_params.slave_ports[n]; + if (!port->is_slave) { + TEST_ASSERT_SUCCESS(rte_eth_bond_slave_add(test_params.bond_port_id, + port->port_id), "Cannot attach slave %d to the bonding", + port->port_id); + port->is_slave = 1; + } + } + + return 0; +} + +/** + * Set all RETA values in port_id to value + */ +static int +reta_set(uint8_t port_id, uint8_t value, int reta_size) +{ + struct rte_eth_rss_reta_entry64 reta_conf[512/RTE_RETA_GROUP_SIZE]; + int i, j; + + for (i = 0; i < reta_size / RTE_RETA_GROUP_SIZE; i++) { + /* select all fields to set */ + reta_conf[i].mask = ~0LL; + for (j = 0; j < RTE_RETA_GROUP_SIZE; j++) + reta_conf[i].reta[j] = value; + } + + return rte_eth_dev_rss_reta_update(port_id, reta_conf, reta_size); +} + +/** + * Check if slaves RETA is synchronized with bonding port. Returns 1 if slave + * port is synced with bonding port. + */ +static int +reta_check_synced(struct slave_conf *port) +{ + unsigned i; + + for (i = 0; i < test_params.bond_dev_info.reta_size; + i++) { + + int index = i / RTE_RETA_GROUP_SIZE; + int shift = i % RTE_RETA_GROUP_SIZE; + + if (port->reta_conf[index].reta[shift] != + test_params.bond_reta_conf[index].reta[shift]) + return 0; + + } + + return 1; +} + +/** + * Fetch bonding ports RETA + */ +static int +bond_reta_fetch(void) { + unsigned j; + + for (j = 0; j < test_params.bond_dev_info.reta_size / RTE_RETA_GROUP_SIZE; + j++) + test_params.bond_reta_conf[j].mask = ~0LL; + + TEST_ASSERT_SUCCESS(rte_eth_dev_rss_reta_query(test_params.bond_port_id, + test_params.bond_reta_conf, test_params.bond_dev_info.reta_size), + "Cannot take bonding ports RSS configuration"); + return 0; +} + +/** + * Fetch slaves RETA + */ +static int +slave_reta_fetch(struct slave_conf *port) { + unsigned j; + + for (j = 0; j < port->dev_info.reta_size / RTE_RETA_GROUP_SIZE; j++) + port->reta_conf[j].mask = ~0LL; + + TEST_ASSERT_SUCCESS(rte_eth_dev_rss_reta_query(port->port_id, + port->reta_conf, port->dev_info.reta_size), + "Cannot take bonding ports RSS configuration"); + return 0; +} + +/** + * Remove and add slave to check if slaves configuration is synced with + * the bonding ports values after adding new slave. + */ +static int +slave_remove_and_add(void) +{ + struct slave_conf *port = &(test_params.slave_ports[0]); + + /* 1. Remove first slave from bonding */ + TEST_ASSERT_SUCCESS(rte_eth_bond_slave_remove(test_params.bond_port_id, + port->port_id), "Cannot remove slave #d from bonding"); + + /* 2. Change removed (ex-)slave and bonding configuration to different + * values + */ + reta_set(test_params.bond_port_id, 1, test_params.bond_dev_info.reta_size); + bond_reta_fetch(); + + reta_set(port->port_id, 2, port->dev_info.reta_size); + slave_reta_fetch(port); + + TEST_ASSERT(reta_check_synced(port) == 0, + "Removed slave didn't should be synchronized with bonding port"); + + /* 3. Add (ex-)slave and check if configuration changed*/ + TEST_ASSERT_SUCCESS(rte_eth_bond_slave_add(test_params.bond_port_id, + port->port_id), "Cannot add slave"); + + bond_reta_fetch(); + slave_reta_fetch(port); + + return reta_check_synced(port); +} + +/** + * Test configuration propagation over slaves. + */ +static int +test_propagate(void) +{ + unsigned i; + uint8_t n; + struct slave_conf *port; + uint8_t bond_rss_key[40]; + struct rte_eth_rss_conf bond_rss_conf; + + int retval = 0; + uint64_t rss_hf = 0; + uint64_t default_rss_hf = 0; + + rte_eth_dev_info_get(test_params.bond_port_id, &test_params.bond_dev_info); + + /* + * Test hash function propagation + */ + for (i = 0; i < sizeof(test_params.bond_dev_info.flow_type_rss_offloads)*8; + i++) { + + rss_hf = test_params.bond_dev_info.flow_type_rss_offloads & (1<<i); + if (rss_hf) { + bond_rss_conf.rss_key = NULL; + bond_rss_conf.rss_hf = rss_hf; + + retval = rte_eth_dev_rss_hash_update(test_params.bond_port_id, + &bond_rss_conf); + TEST_ASSERT_SUCCESS(retval, "Cannot set slaves hash function"); + + FOR_EACH_PORT(n, port) { + port = &test_params.slave_ports[n]; + + retval = rte_eth_dev_rss_hash_conf_get(port->port_id, + &port->rss_conf); + TEST_ASSERT_SUCCESS(retval, + "Cannot take slaves RSS configuration"); + + TEST_ASSERT(port->rss_conf.rss_hf == rss_hf, + "Hash function not propagated for slave %d", + port->port_id); + } + + default_rss_hf = rss_hf; + } + + } + + /* + * Test key propagation + */ + for (i = 1; i < 10; i++) { + + /* Set all keys to zero */ + FOR_EACH_PORT(n, port) { + port = &test_params.slave_ports[n]; + memset(port->rss_conf.rss_key, 0, 40); + retval = rte_eth_dev_rss_hash_update(port->port_id, + &port->rss_conf); + TEST_ASSERT_SUCCESS(retval, "Cannot set slaves RSS keys"); + } + + memset(bond_rss_key, i, sizeof(bond_rss_key)); + bond_rss_conf.rss_hf = default_rss_hf, + bond_rss_conf.rss_key = bond_rss_key; + bond_rss_conf.rss_key_len = 40; + + retval = rte_eth_dev_rss_hash_update(test_params.bond_port_id, + &bond_rss_conf); + TEST_ASSERT_SUCCESS(retval, "Cannot set bonded port RSS keys"); + + FOR_EACH_PORT(n, port) { + port = &test_params.slave_ports[n]; + + retval = rte_eth_dev_rss_hash_conf_get(port->port_id, + &(port->rss_conf)); + + TEST_ASSERT_SUCCESS(retval, + "Cannot take slaves RSS configuration"); + + /* compare keys */ + retval = memcmp(port->rss_conf.rss_key, bond_rss_key, + sizeof(bond_rss_key)); + TEST_ASSERT(retval == 0, "Key value not propagated for slave %d", + port->port_id); + } + } + + /* + * Test RETA propagation + */ + for (i = 0; i < RXTX_QUEUE_COUNT; i++) { + + /* Set all keys to zero */ + FOR_EACH_PORT(n, port) { + port = &test_params.slave_ports[n]; + retval = reta_set(port->port_id, (i + 1) % RXTX_QUEUE_COUNT, + port->dev_info.reta_size); + TEST_ASSERT_SUCCESS(retval, "Cannot set slaves RETA"); + } + + TEST_ASSERT_SUCCESS(reta_set(test_params.bond_port_id, + i % RXTX_QUEUE_COUNT, test_params.bond_dev_info.reta_size), + "Cannot set bonded port RETA"); + + bond_reta_fetch(); + + FOR_EACH_PORT(n, port) { + port = &test_params.slave_ports[n]; + + slave_reta_fetch(port); + TEST_ASSERT(reta_check_synced(port) == 1, "RETAs inconsistent"); + } + } + + return TEST_SUCCESS; +} + +/** + * Test propagation logic, when RX_RSS mq_mode is turned on for bonding port + */ +static int +test_rss(void) +{ + /** + * Configure bonding port in RSS mq mode + */ + TEST_ASSERT_SUCCESS(configure_ethdev(test_params.bond_port_id, + &rss_pmd_conf, 0), "Failed to configure bonding device\n"); + + rte_eth_dev_info_get(test_params.bond_port_id, &test_params.bond_dev_info); + + TEST_ASSERT_SUCCESS(bond_slaves(), "Bonding slaves failed"); + + TEST_ASSERT_SUCCESS(rte_eth_dev_start(test_params.bond_port_id), + "Failed to start bonding port (%d).", test_params.bond_port_id); + + TEST_ASSERT_SUCCESS(test_propagate(), "Propagation test failed"); + + TEST_ASSERT(slave_remove_and_add() == 1, "New slave should be synced"); + + remove_slaves_and_stop_bonded_device(); + + return TEST_SUCCESS; +} + +/** + * Test propagation logic, when RX_RSS mq_mode is turned off for bonding port + */ +static int +test_rss_lazy(void) +{ + TEST_ASSERT_SUCCESS(configure_ethdev(test_params.bond_port_id, + &default_pmd_conf, 0), "Failed to configure bonding device\n"); + + rte_eth_dev_info_get(test_params.bond_port_id, &test_params.bond_dev_info); + + TEST_ASSERT_SUCCESS(bond_slaves(), "Bonding slaves failed"); + + TEST_ASSERT_SUCCESS(rte_eth_dev_start(test_params.bond_port_id), + "Failed to start bonding port (%d).", test_params.bond_port_id); + + TEST_ASSERT_SUCCESS(test_propagate(), "Propagation test failed"); + + TEST_ASSERT(slave_remove_and_add() == 0, "New slave shouldn't be synced"); + + remove_slaves_and_stop_bonded_device(); + + return TEST_SUCCESS; +} + +static int +test_setup(void) +{ + unsigned n; + int retval; + int port_id; + char name[256]; + struct slave_conf *port; + + if (test_params.mbuf_pool == NULL) { + + test_params.mbuf_pool = rte_mempool_create("RSS_MBUF_POOL", NUM_MBUFS * + SLAVE_COUNT, MBUF_SIZE, MBUF_CACHE_SIZE, + sizeof(struct rte_pktmbuf_pool_private), rte_pktmbuf_pool_init, + NULL, rte_pktmbuf_init, NULL, rte_socket_id(), 0); + + TEST_ASSERT(test_params.mbuf_pool != NULL, + "rte_mempool_create failed\n"); + } + + /* Create / initialize ring eth devs. */ + FOR_EACH_PORT(n, port) { + port = &test_params.slave_ports[n]; + + port_id = rte_eth_dev_count(); + snprintf(name, sizeof(name), SLAVE_DEV_NAME_FMT, port_id); + + retval = eth_dev_null_create(name, 0, 64, 0); + TEST_ASSERT_SUCCESS(retval, "Failed to create null device '%s'\n", + name); + + port->port_id = port_id; + + port->rss_conf.rss_key = port->rss_key; + port->rss_conf.rss_key_len = 40; + + retval = configure_ethdev(port->port_id, &default_pmd_conf, 0); + TEST_ASSERT_SUCCESS(retval, "Failed to configure virtual ethdev %s\n", + name); + + rte_eth_dev_info_get(port->port_id, &port->dev_info); + } + + if (test_params.bond_port_id == INVALID_PORT_ID) { + retval = rte_eth_bond_create(BONDED_DEV_NAME, 0, rte_socket_id()); + + TEST_ASSERT(retval >= 0, "Failed to create bonded ethdev %s", + BONDED_DEV_NAME); + + test_params.bond_port_id = retval; + + TEST_ASSERT_SUCCESS(configure_ethdev(test_params.bond_port_id, + &default_pmd_conf, 0), "Failed to configure bonding device\n"); + + rte_eth_dev_info_get(test_params.bond_port_id, + &test_params.bond_dev_info); + } + + return TEST_SUCCESS; +} + +static int +testsuite_teardown(void) +{ + struct slave_conf *port; + uint8_t i; + + /* Only stop ports. + * Any cleanup/reset state is done when particular test is + * started. */ + + rte_eth_dev_stop(test_params.bond_port_id); + + FOR_EACH_PORT(i, port) + rte_eth_dev_stop(port->port_id); + + return 0; +} + +static int +check_environment(void) +{ + return TEST_SUCCESS; +} + +static int +test_rssconf_executor(int (*test_func)(void)) +{ + int test_result; + + /* Check if environment is clean. Fail to launch a test if there was + * a critical error before that prevented to reset environment. */ + TEST_ASSERT_SUCCESS(check_environment(), + "Refusing to launch test in dirty environment."); + + RTE_VERIFY(test_func != NULL); + test_result = (*test_func)(); + + /* If test succeed check if environment wast left in good condition. */ + if (test_result == TEST_SUCCESS) + test_result = check_environment(); + + /* Reset environment in case test failed to do that. */ + if (test_result != TEST_SUCCESS) { + TEST_ASSERT_SUCCESS(remove_slaves_and_stop_bonded_device(), + "Failed to stop bonded device"); + } + + return test_result; +} + +static int +test_setup_wrapper(void) +{ + return test_rssconf_executor(&test_setup); +} + +static int +test_rss_wrapper(void) +{ + return test_rssconf_executor(&test_rss); +} + +static int +test_rss_lazy_wrapper(void) +{ + return test_rssconf_executor(&test_rss_lazy); +} + +static struct unit_test_suite link_bonding_rssconf_test_suite = { + .suite_name = "RSS Dynamic Configuration for Bonding Unit Test Suite", + .teardown = testsuite_teardown, + .unit_test_cases = { + TEST_CASE_NAMED("test_setup", test_setup_wrapper), + TEST_CASE_NAMED("test_rss", test_rss_wrapper), + TEST_CASE_NAMED("test_rss_lazy", test_rss_lazy_wrapper), + { NULL, NULL, NULL, NULL, NULL } /**< NULL terminate unit test array */ + } +}; + +static int +test_link_bonding_rssconf(void) +{ + return unit_test_suite_runner(&link_bonding_rssconf_test_suite); +} + +static struct test_command test_link_bonding_rssconf_cmd = { + .command = "link_bonding_rssconf_autotest", + .callback = test_link_bonding_rssconf, +}; + +REGISTER_TEST_COMMAND(test_link_bonding_rssconf_cmd); -- 1.7.9.5 ^ permalink raw reply [flat|nested] 126+ messages in thread
* [dpdk-dev] [PATCH v7 7/9] bonding: per queue stats 2015-10-30 14:25 ` [dpdk-dev] [PATCH v7 " Tomasz Kulasek ` (5 preceding siblings ...) 2015-10-30 14:25 ` [dpdk-dev] [PATCH v7 6/9] test: " Tomasz Kulasek @ 2015-10-30 14:25 ` Tomasz Kulasek 2015-10-30 14:25 ` [dpdk-dev] [PATCH v7 8/9] doc: fixed spellings and typos Tomasz Kulasek ` (2 subsequent siblings) 9 siblings, 0 replies; 126+ messages in thread From: Tomasz Kulasek @ 2015-10-30 14:25 UTC (permalink / raw) To: dev This patch adds fills bonding port's stats with a sum of corresponding values taken from bonded slaves, when stats are requested for bonding port. v5 changes: - removed queue_stats_mapping_set from eth_dev_ops of bonding device Signed-off-by: Tomasz Kulasek <tomaszx.kulasek@intel.com> Acked-by: Declan Doherty <declan.doherty@intel.com> --- drivers/net/bonding/rte_eth_bond_pmd.c | 11 ++++++++++- 1 file changed, 10 insertions(+), 1 deletion(-) diff --git a/drivers/net/bonding/rte_eth_bond_pmd.c b/drivers/net/bonding/rte_eth_bond_pmd.c index 2880f5c..eecb381 100644 --- a/drivers/net/bonding/rte_eth_bond_pmd.c +++ b/drivers/net/bonding/rte_eth_bond_pmd.c @@ -1801,7 +1801,7 @@ bond_ethdev_stats_get(struct rte_eth_dev *dev, struct rte_eth_stats *stats) { struct bond_dev_private *internals = dev->data->dev_private; struct rte_eth_stats slave_stats; - int i; + int i, j; for (i = 0; i < internals->slave_count; i++) { rte_eth_stats_get(internals->slaves[i].port_id, &slave_stats); @@ -1820,6 +1820,15 @@ bond_ethdev_stats_get(struct rte_eth_dev *dev, struct rte_eth_stats *stats) stats->rx_pause_xon += slave_stats.rx_pause_xon; stats->tx_pause_xoff += slave_stats.tx_pause_xoff; stats->rx_pause_xoff += slave_stats.rx_pause_xoff; + + for (j = 0; j < RTE_ETHDEV_QUEUE_STAT_CNTRS; j++) { + stats->q_ipackets[j] += slave_stats.q_ipackets[j]; + stats->q_opackets[j] += slave_stats.q_opackets[j]; + stats->q_ibytes[j] += slave_stats.q_ibytes[j]; + stats->q_obytes[j] += slave_stats.q_obytes[j]; + stats->q_errors[j] += slave_stats.q_errors[j]; + } + } } -- 1.7.9.5 ^ permalink raw reply [flat|nested] 126+ messages in thread
* [dpdk-dev] [PATCH v7 8/9] doc: fixed spellings and typos 2015-10-30 14:25 ` [dpdk-dev] [PATCH v7 " Tomasz Kulasek ` (6 preceding siblings ...) 2015-10-30 14:25 ` [dpdk-dev] [PATCH v7 7/9] bonding: per queue stats Tomasz Kulasek @ 2015-10-30 14:25 ` Tomasz Kulasek 2015-10-30 14:25 ` [dpdk-dev] [PATCH v7 9/9] doc: dynamic rss configuration for bonding Tomasz Kulasek 2015-11-01 17:29 ` [dpdk-dev] [PATCH v7 0/9] Dynamic RSS Configuration for Bonding Thomas Monjalon 9 siblings, 0 replies; 126+ messages in thread From: Tomasz Kulasek @ 2015-10-30 14:25 UTC (permalink / raw) To: dev Signed-off-by: Tomasz Kulasek <tomaszx.kulasek@intel.com> Acked-by: Declan Doherty <declan.doherty@intel.com> --- .../prog_guide/link_bonding_poll_mode_drv_lib.rst | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/doc/guides/prog_guide/link_bonding_poll_mode_drv_lib.rst b/doc/guides/prog_guide/link_bonding_poll_mode_drv_lib.rst index 96e554f..03baf90 100644 --- a/doc/guides/prog_guide/link_bonding_poll_mode_drv_lib.rst +++ b/doc/guides/prog_guide/link_bonding_poll_mode_drv_lib.rst @@ -188,7 +188,7 @@ conditions are not met. If a user wishes to monitor individual slaves then they must register callbacks with that slave directly. The link bonding library also supports devices which do not implement link -status change interrupts, this is achieve by polling the devices link status at +status change interrupts, this is achieved by polling the devices link status at a defined period which is set using the ``rte_eth_bond_link_monitoring_set`` API, the default polling interval is 10ms. When a device is added as a slave to a bonding device it is determined using the ``RTE_PCI_DRV_INTR_LSC`` flag @@ -286,7 +286,7 @@ and UDP protocols for load balancing. Using Link Bonding Devices -------------------------- -The librte_pmd_bond library support two modes of device creation, the libraries +The librte_pmd_bond library supports two modes of device creation, the libraries export full C API or using the EAL command line to statically configure link bonding devices at application startup. Using the EAL option it is possible to use link bonding functionality transparently without specific knowledge of the @@ -299,7 +299,7 @@ Using the Poll Mode Driver from an Application Using the librte_pmd_bond libraries API it is possible to dynamically create and manage link bonding device from within any application. Link bonding -device are created using the ``rte_eth_bond_create`` API which requires a +devices are created using the ``rte_eth_bond_create`` API which requires a unique device name, the link bonding mode to initial the device in and finally the socket Id which to allocate the devices resources onto. After successful creation of a bonding device it must be configured using the generic Ethernet @@ -362,7 +362,7 @@ The different options are: mode=2 * slave: Defines the PMD device which will be added as slave to the bonded - device. This option can be selected multiple time, for each device to be + device. This option can be selected multiple times, for each device to be added as a slave. Physical devices should be specified using their PCI address, in the format domain:bus:devid.function -- 1.7.9.5 ^ permalink raw reply [flat|nested] 126+ messages in thread
* [dpdk-dev] [PATCH v7 9/9] doc: dynamic rss configuration for bonding 2015-10-30 14:25 ` [dpdk-dev] [PATCH v7 " Tomasz Kulasek ` (7 preceding siblings ...) 2015-10-30 14:25 ` [dpdk-dev] [PATCH v7 8/9] doc: fixed spellings and typos Tomasz Kulasek @ 2015-10-30 14:25 ` Tomasz Kulasek 2015-11-01 17:29 ` [dpdk-dev] [PATCH v7 0/9] Dynamic RSS Configuration for Bonding Thomas Monjalon 9 siblings, 0 replies; 126+ messages in thread From: Tomasz Kulasek @ 2015-10-30 14:25 UTC (permalink / raw) To: dev Documentation update about implementation details and requirements for Dynamic RSS Configuration for Bonding. Signed-off-by: Tomasz Kulasek <tomaszx.kulasek@intel.com> Acked-by: Declan Doherty <declan.doherty@intel.com> --- .../prog_guide/link_bonding_poll_mode_drv_lib.rst | 34 ++++++++++++++++++-- 1 file changed, 32 insertions(+), 2 deletions(-) diff --git a/doc/guides/prog_guide/link_bonding_poll_mode_drv_lib.rst b/doc/guides/prog_guide/link_bonding_poll_mode_drv_lib.rst index 03baf90..46f0296 100644 --- a/doc/guides/prog_guide/link_bonding_poll_mode_drv_lib.rst +++ b/doc/guides/prog_guide/link_bonding_poll_mode_drv_lib.rst @@ -1,5 +1,5 @@ .. BSD LICENSE - Copyright(c) 2010-2014 Intel Corporation. All rights reserved. + Copyright(c) 2010-2015 Intel Corporation. All rights reserved. All rights reserved. Redistribution and use in source and binary forms, with or without @@ -173,7 +173,28 @@ After a slave device is added to a bonded device slave is stopped using ``rte_eth_dev_stop`` and then reconfigured using ``rte_eth_dev_configure`` the RX and TX queues are also reconfigured using ``rte_eth_tx_queue_setup`` / ``rte_eth_rx_queue_setup`` with the parameters use to configure the bonding -device. +device. If RSS is enabled for bonding device, this mode is also enabled on new +slave and configured as well. + +Setting up multi-queue mode for bonding device to RSS, makes it fully +RSS-capable, so all slaves are synchronized with its configuration. This mode is +intended to provide RSS configuration on slaves transparent for client +application implementation. + +Bonding device stores its own version of RSS settings i.e. RETA, RSS hash +function and RSS key, used to set up its slaves. That let to define the meaning +of RSS configuration of bonding device as desired configuration of whole bonding +(as one unit), without pointing any of slave inside. It is required to ensure +consistency and made it more errorproof. + +RSS hash function set for bonding device, is a maximal set of RSS hash functions +supported by all bonded slaves. RETA size is a GCD of all its RETA's sizes, so +it can be easily used as a pattern providing expected behavior, even if slave +RETAs' sizes are different. If RSS Key is not set for bonded device, it's not +changed on the slaves and default key for device is used. + +All settings are managed through the bonding port API and always are propagated +in one direction (from bonding to slaves). Link Status Change Interrupts / Polling ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ @@ -207,6 +228,15 @@ these parameters. A bonding device must have a minimum of one slave before the bonding device itself can be started. +To use a bonding device dynamic RSS configuration feature effectively, it is +also required, that all slaves should be RSS-capable and support, at least one +common hash function available for each of them. Changing RSS key is only +possible, when all slave devices support the same key size. + +To prevent inconsistency on how slaves process packets, once a device is added +to a bonding device, RSS configuration should be managed through the bonding +device API, and not directly on the slave. + Like all other PMD, all functions exported by a PMD are lock-free functions that are assumed not to be invoked in parallel on different logical cores to work on the same target object. -- 1.7.9.5 ^ permalink raw reply [flat|nested] 126+ messages in thread
* Re: [dpdk-dev] [PATCH v7 0/9] Dynamic RSS Configuration for Bonding 2015-10-30 14:25 ` [dpdk-dev] [PATCH v7 " Tomasz Kulasek ` (8 preceding siblings ...) 2015-10-30 14:25 ` [dpdk-dev] [PATCH v7 9/9] doc: dynamic rss configuration for bonding Tomasz Kulasek @ 2015-11-01 17:29 ` Thomas Monjalon 9 siblings, 0 replies; 126+ messages in thread From: Thomas Monjalon @ 2015-11-01 17:29 UTC (permalink / raw) To: Tomasz Kulasek; +Cc: dev > Tomasz Kulasek (9): > bonding: rss dynamic configuration > null: fix segfault when null_pmd added to bonding > null: extend number of virtual queues > null: export eth_dev_null_create > null: virtual dynamic rss configuration > test: dynamic rss configuration > bonding: per queue stats > doc: fixed spellings and typos > doc: dynamic rss configuration for bonding Applied, thanks PS: I've added an entry in the release notes ^ permalink raw reply [flat|nested] 126+ messages in thread
* [dpdk-dev] [PATCH v7 9/9] doc: dynamic rss configuration for bonding
@ 2015-10-30 13:56 Tomasz Kulasek
0 siblings, 0 replies; 126+ messages in thread
From: Tomasz Kulasek @ 2015-10-30 13:56 UTC (permalink / raw)
To: dev
Date: Fri, 30 Oct 2015 14:56:04 +0100
Message-Id: <1446213364-11856-10-git-send-email-tomaszx.kulasek@intel.com>
X-Mailer: git-send-email 2.1.4
In-Reply-To: <1446213364-11856-1-git-send-email-tomaszx.kulasek@intel.com>
References: <1444989651-6236-1-git-send-email-tomaszx.kulasek@intel.com>
<1446213364-11856-1-git-send-email-tomaszx.kulasek@intel.com>
Documentation update about implementation details and requirements for
Dynamic RSS Configuration for Bonding.
Signed-off-by: Tomasz Kulasek <tomaszx.kulasek@intel.com>
Acked-by: Declan Doherty <declan.doherty@intel.com>
---
.../prog_guide/link_bonding_poll_mode_drv_lib.rst | 34 ++++++++++++++++++--
1 file changed, 32 insertions(+), 2 deletions(-)
diff --git a/doc/guides/prog_guide/link_bonding_poll_mode_drv_lib.rst b/doc/guides/prog_guide/link_bonding_poll_mode_drv_lib.rst
index 03baf90..46f0296 100644
--- a/doc/guides/prog_guide/link_bonding_poll_mode_drv_lib.rst
+++ b/doc/guides/prog_guide/link_bonding_poll_mode_drv_lib.rst
@@ -1,5 +1,5 @@
.. BSD LICENSE
- Copyright(c) 2010-2014 Intel Corporation. All rights reserved.
+ Copyright(c) 2010-2015 Intel Corporation. All rights reserved.
All rights reserved.
Redistribution and use in source and binary forms, with or without
@@ -173,7 +173,28 @@ After a slave device is added to a bonded device slave is stopped using
``rte_eth_dev_stop`` and then reconfigured using ``rte_eth_dev_configure``
the RX and TX queues are also reconfigured using ``rte_eth_tx_queue_setup`` /
``rte_eth_rx_queue_setup`` with the parameters use to configure the bonding
-device.
+device. If RSS is enabled for bonding device, this mode is also enabled on new
+slave and configured as well.
+
+Setting up multi-queue mode for bonding device to RSS, makes it fully
+RSS-capable, so all slaves are synchronized with its configuration. This mode is
+intended to provide RSS configuration on slaves transparent for client
+application implementation.
+
+Bonding device stores its own version of RSS settings i.e. RETA, RSS hash
+function and RSS key, used to set up its slaves. That let to define the meaning
+of RSS configuration of bonding device as desired configuration of whole bonding
+(as one unit), without pointing any of slave inside. It is required to ensure
+consistency and made it more errorproof.
+
+RSS hash function set for bonding device, is a maximal set of RSS hash functions
+supported by all bonded slaves. RETA size is a GCD of all its RETA's sizes, so
+it can be easily used as a pattern providing expected behavior, even if slave
+RETAs' sizes are different. If RSS Key is not set for bonded device, it's not
+changed on the slaves and default key for device is used.
+
+All settings are managed through the bonding port API and always are propagated
+in one direction (from bonding to slaves).
Link Status Change Interrupts / Polling
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
@@ -207,6 +228,15 @@ these parameters.
A bonding device must have a minimum of one slave before the bonding device
itself can be started.
+To use a bonding device dynamic RSS configuration feature effectively, it is
+also required, that all slaves should be RSS-capable and support, at least one
+common hash function available for each of them. Changing RSS key is only
+possible, when all slave devices support the same key size.
+
+To prevent inconsistency on how slaves process packets, once a device is added
+to a bonding device, RSS configuration should be managed through the bonding
+device API, and not directly on the slave.
+
Like all other PMD, all functions exported by a PMD are lock-free functions
that are assumed not to be invoked in parallel on different logical cores to
work on the same target object.
--
1.7.9.5
^ permalink raw reply [flat|nested] 126+ messages in threadend of thread, other threads:[~2015-11-01 17:30 UTC | newest] Thread overview: 126+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2015-06-03 10:58 [dpdk-dev] [PATCH 0/8] Dynamic RSS Configuration for Bonding Tomasz Kulasek 2015-06-03 10:59 ` [dpdk-dev] [PATCH 1/8] bond: dynamic rss configuration Tomasz Kulasek 2015-06-03 10:59 ` [dpdk-dev] [PATCH 2/8] ring: " Tomasz Kulasek 2015-06-03 10:59 ` [dpdk-dev] [PATCH 3/8] test: " Tomasz Kulasek 2015-06-03 10:59 ` [dpdk-dev] [PATCH 4/8] bond: queue stats mapping Tomasz Kulasek 2015-06-03 10:59 ` [dpdk-dev] [PATCH 5/8] ring: queue stats mapping set dummy implementation Tomasz Kulasek 2015-06-03 10:59 ` [dpdk-dev] [PATCH 6/8] examples: dynamic rss configuration for bonding Tomasz Kulasek 2015-06-03 10:59 ` [dpdk-dev] [PATCH 7/8] doc: fixed spellings and typos Tomasz Kulasek 2015-06-03 10:59 ` [dpdk-dev] [PATCH 8/8] doc: dynamic rss configuration for bonding Tomasz Kulasek 2015-06-12 5:36 ` [dpdk-dev] [PATCH 0/8] Dynamic RSS Configuration for Bonding Xu, HuilongX 2015-06-25 9:20 ` Kulasek, TomaszX 2015-06-19 14:13 ` [dpdk-dev] [PATCH v2 " Tomasz Kulasek 2015-06-19 14:13 ` [dpdk-dev] [PATCH v2 1/8] bond: rss dynamic configuration Tomasz Kulasek 2015-06-19 14:13 ` [dpdk-dev] [PATCH v2 2/8] ring: dynamic rss configuration Tomasz Kulasek 2015-06-19 14:13 ` [dpdk-dev] [PATCH v2 3/8] test: " Tomasz Kulasek 2015-06-19 14:13 ` [dpdk-dev] [PATCH v2 4/8] bond: queue stats mapping Tomasz Kulasek 2015-06-19 14:13 ` [dpdk-dev] [PATCH v2 5/8] ring: queue stats mapping set dummy implementation Tomasz Kulasek 2015-06-19 14:13 ` [dpdk-dev] [PATCH v2 6/8] examples: dynamic rss configuration for bonding Tomasz Kulasek 2015-06-19 14:13 ` [dpdk-dev] [PATCH v2 7/8] doc: fixed spellings and typos Tomasz Kulasek 2015-06-19 14:13 ` [dpdk-dev] [PATCH v2 8/8] doc: dynamic rss configuration for bonding Tomasz Kulasek 2015-06-26 7:33 ` [dpdk-dev] [PATCH v2 0/8] Dynamic RSS Configuration for Bonding Doherty, Declan 2015-06-28 21:54 ` Thomas Monjalon 2015-06-29 14:50 ` [dpdk-dev] [PATCH v3 " Tomasz Kulasek 2015-06-29 14:50 ` [dpdk-dev] [PATCH v3 1/8] bonding: rss dynamic configuration Tomasz Kulasek 2015-06-29 14:50 ` [dpdk-dev] [PATCH v3 2/8] ring: dynamic rss configuration Tomasz Kulasek 2015-07-13 12:36 ` Thomas Monjalon 2015-07-13 14:43 ` Kulasek, TomaszX 2015-07-13 15:11 ` Thomas Monjalon 2015-07-16 13:02 ` Kulasek, TomaszX 2015-06-29 14:50 ` [dpdk-dev] [PATCH v3 3/8] test: " Tomasz Kulasek 2015-06-29 14:50 ` [dpdk-dev] [PATCH v3 4/8] bonding: queue stats mapping Tomasz Kulasek 2015-07-13 11:18 ` Thomas Monjalon 2015-07-13 12:00 ` Kulasek, TomaszX 2015-06-29 14:50 ` [dpdk-dev] [PATCH v3 5/8] ring: queue stats mapping set dummy implementation Tomasz Kulasek 2015-07-09 1:58 ` Thomas Monjalon 2015-07-09 9:55 ` Kulasek, TomaszX 2015-07-09 10:13 ` Thomas Monjalon 2015-06-29 14:50 ` [dpdk-dev] [PATCH v3 6/8] examples: dynamic rss configuration for bonding Tomasz Kulasek 2015-07-13 11:20 ` Thomas Monjalon 2015-06-29 14:50 ` [dpdk-dev] [PATCH v3 7/8] doc: fixed spellings and typos Tomasz Kulasek 2015-06-29 14:50 ` [dpdk-dev] [PATCH v3 8/8] doc: dynamic rss configuration for bonding Tomasz Kulasek 2015-07-01 10:05 ` [dpdk-dev] [PATCH v3 0/8] Dynamic RSS Configuration for Bonding Declan Doherty 2015-07-13 11:03 ` Thomas Monjalon 2015-07-13 11:18 ` Thomas Monjalon 2015-07-13 12:14 ` Kulasek, TomaszX 2015-07-15 17:26 ` [dpdk-dev] [PATCHv4 0/9] " Tomasz Kulasek 2015-07-15 17:26 ` [dpdk-dev] [PATCHv4 1/9] bonding: rss dynamic configuration Tomasz Kulasek 2015-07-15 17:26 ` [dpdk-dev] [PATCHv4 2/9] null: fix segfault when null_pmd added to bonding Tomasz Kulasek 2015-09-29 2:24 ` Tetsuya Mukawa 2015-09-29 9:39 ` Kulasek, TomaszX 2015-09-29 10:32 ` Tetsuya Mukawa 2015-09-29 11:29 ` Kulasek, TomaszX 2015-09-29 15:10 ` Kulasek, TomaszX 2015-09-30 1:24 ` Tetsuya Mukawa 2015-07-15 17:26 ` [dpdk-dev] [PATCHv4 3/9] null: extend number of virtual queues Tomasz Kulasek 2015-09-29 2:24 ` Tetsuya Mukawa 2015-09-29 9:46 ` Kulasek, TomaszX 2015-09-29 10:34 ` Tetsuya Mukawa 2015-09-29 11:56 ` Kulasek, TomaszX 2015-09-29 15:06 ` Kulasek, TomaszX 2015-09-30 1:24 ` Tetsuya Mukawa 2015-07-15 17:26 ` [dpdk-dev] [PATCHv4 4/9] null: virtual dynamic rss configuration Tomasz Kulasek 2015-09-29 2:24 ` Tetsuya Mukawa 2015-09-29 10:04 ` Kulasek, TomaszX 2015-10-12 9:05 ` Jastrzebski, MichalX K 2015-07-15 17:26 ` [dpdk-dev] [PATCHv4 5/9] null: export eth_dev_null_create Tomasz Kulasek 2015-09-25 10:11 ` Thomas Monjalon 2015-09-25 13:24 ` Kulasek, TomaszX 2015-09-25 13:43 ` Thomas Monjalon 2015-09-29 2:28 ` Tetsuya Mukawa 2015-07-15 17:26 ` [dpdk-dev] [PATCHv4 6/9] test: dynamic rss configuration Tomasz Kulasek 2015-07-15 17:26 ` [dpdk-dev] [PATCHv4 7/9] bonding: queue stats mapping Tomasz Kulasek 2015-09-25 10:14 ` Thomas Monjalon 2015-09-25 13:26 ` Kulasek, TomaszX 2015-07-15 17:26 ` [dpdk-dev] [PATCHv4 8/9] doc: fixed spellings and typos Tomasz Kulasek 2015-07-15 17:26 ` [dpdk-dev] [PATCHv4 9/9] doc: dynamic rss configuration for bonding Tomasz Kulasek 2015-09-25 10:16 ` [dpdk-dev] [PATCHv4 0/9] Dynamic RSS Configuration for Bonding Thomas Monjalon 2015-09-25 13:28 ` Kulasek, TomaszX 2015-09-25 13:44 ` Thomas Monjalon 2015-09-30 14:04 ` [dpdk-dev] [PATCH v5 " Tomasz Kulasek 2015-09-30 14:04 ` [dpdk-dev] [PATCH v5 1/9] bonding: rss dynamic configuration Tomasz Kulasek 2015-09-30 14:04 ` [dpdk-dev] [PATCH v5 2/9] null: fix segfault when null_pmd added to bonding Tomasz Kulasek 2015-09-30 14:04 ` [dpdk-dev] [PATCH v5 3/9] null: extend number of virtual queues Tomasz Kulasek 2015-10-14 1:34 ` Tetsuya Mukawa 2015-09-30 14:05 ` [dpdk-dev] [PATCH v5 4/9] null: virtual dynamic rss configuration Tomasz Kulasek 2015-10-14 1:34 ` Tetsuya Mukawa 2015-10-15 7:46 ` Tetsuya Mukawa 2015-10-15 8:42 ` Kulasek, TomaszX 2015-10-15 9:21 ` Tetsuya Mukawa 2015-09-30 14:05 ` [dpdk-dev] [PATCH v5 5/9] null: export eth_dev_null_create Tomasz Kulasek 2015-10-14 1:35 ` Tetsuya Mukawa 2015-10-14 12:42 ` Jastrzebski, MichalX K 2015-10-15 8:16 ` Tetsuya Mukawa 2015-10-15 10:35 ` Jastrzebski, MichalX K 2015-09-30 14:05 ` [dpdk-dev] [PATCH v5 6/9] test: dynamic rss configuration Tomasz Kulasek 2015-09-30 14:05 ` [dpdk-dev] [PATCH v5 7/9] bonding: per queue stats Tomasz Kulasek 2015-09-30 14:05 ` [dpdk-dev] [PATCH v5 8/9] doc: fixed spellings and typos Tomasz Kulasek 2015-09-30 14:05 ` [dpdk-dev] [PATCH v5 9/9] doc: dynamic rss configuration for bonding Tomasz Kulasek 2015-10-16 10:00 ` [dpdk-dev] [PATCH v6 0/9] Dynamic RSS Configuration for Bonding Tomasz Kulasek 2015-10-16 10:00 ` [dpdk-dev] [PATCH v6 1/9] bonding: rss dynamic configuration Tomasz Kulasek 2015-10-16 10:00 ` [dpdk-dev] [PATCH v6 2/9] null: fix segfault when null_pmd added to bonding Tomasz Kulasek 2015-10-27 16:58 ` Thomas Monjalon 2015-10-28 13:43 ` Kulasek, TomaszX 2015-10-29 7:19 ` Tetsuya Mukawa 2015-10-30 13:50 ` Kulasek, TomaszX 2015-10-16 10:00 ` [dpdk-dev] [PATCH v6 3/9] null: extend number of virtual queues Tomasz Kulasek 2015-10-16 10:00 ` [dpdk-dev] [PATCH v6 4/9] null: export eth_dev_null_create Tomasz Kulasek 2015-10-16 10:00 ` [dpdk-dev] [PATCH v6 5/9] null: virtual dynamic rss configuration Tomasz Kulasek 2015-10-16 10:00 ` [dpdk-dev] [PATCH v6 6/9] test: " Tomasz Kulasek 2015-10-27 17:34 ` Thomas Monjalon 2015-10-16 10:00 ` [dpdk-dev] [PATCH v6 7/9] bonding: per queue stats Tomasz Kulasek 2015-10-16 10:00 ` [dpdk-dev] [PATCH v6 8/9] doc: fixed spellings and typos Tomasz Kulasek 2015-10-16 10:00 ` [dpdk-dev] [PATCH v6 9/9] doc: dynamic rss configuration for bonding Tomasz Kulasek 2015-10-19 13:22 ` [dpdk-dev] [PATCH v6 0/9] Dynamic RSS Configuration for Bonding Declan Doherty 2015-10-30 14:25 ` [dpdk-dev] [PATCH v7 " Tomasz Kulasek 2015-10-30 14:25 ` [dpdk-dev] [PATCH v7 1/9] bonding: rss dynamic configuration Tomasz Kulasek 2015-10-30 14:25 ` [dpdk-dev] [PATCH v7 2/9] null: fix segfault when null_pmd added to bonding Tomasz Kulasek 2015-10-30 14:25 ` [dpdk-dev] [PATCH v7 3/9] null: extend number of virtual queues Tomasz Kulasek 2015-10-30 14:25 ` [dpdk-dev] [PATCH v7 4/9] null: export eth_dev_null_create Tomasz Kulasek 2015-10-30 14:25 ` [dpdk-dev] [PATCH v7 5/9] null: virtual dynamic rss configuration Tomasz Kulasek 2015-10-30 14:25 ` [dpdk-dev] [PATCH v7 6/9] test: " Tomasz Kulasek 2015-10-30 14:25 ` [dpdk-dev] [PATCH v7 7/9] bonding: per queue stats Tomasz Kulasek 2015-10-30 14:25 ` [dpdk-dev] [PATCH v7 8/9] doc: fixed spellings and typos Tomasz Kulasek 2015-10-30 14:25 ` [dpdk-dev] [PATCH v7 9/9] doc: dynamic rss configuration for bonding Tomasz Kulasek 2015-11-01 17:29 ` [dpdk-dev] [PATCH v7 0/9] Dynamic RSS Configuration for Bonding Thomas Monjalon 2015-10-30 13:56 [dpdk-dev] [PATCH v7 9/9] doc: dynamic rss configuration for bonding Tomasz Kulasek
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).