From: Adrien Mazarguil <adrien.mazarguil@6wind.com>
To: dev@dpdk.org
Subject: [dpdk-dev] [PATCH 06/17] mlx5: adapt indirection table size depending on RX queues number
Date: Mon, 5 Oct 2015 19:54:41 +0200 [thread overview]
Message-ID: <1444067692-29645-7-git-send-email-adrien.mazarguil@6wind.com> (raw)
In-Reply-To: <1444067692-29645-1-git-send-email-adrien.mazarguil@6wind.com>
From: Nelio Laranjeiro <nelio.laranjeiro@6wind.com>
Use the maximum size of the indirection table when the number of requested
RX queues is not a power of two, this help to improve RSS balancing.
A message informs users that balancing is not optimal in such cases.
Signed-off-by: Nelio Laranjeiro <nelio.laranjeiro@6wind.com>
Signed-off-by: Adrien Mazarguil <adrien.mazarguil@6wind.com>
---
drivers/net/mlx5/mlx5.c | 10 +++++++++-
drivers/net/mlx5/mlx5.h | 1 +
drivers/net/mlx5/mlx5_defs.h | 3 +++
drivers/net/mlx5/mlx5_rxq.c | 21 ++++++++++++++-------
4 files changed, 27 insertions(+), 8 deletions(-)
diff --git a/drivers/net/mlx5/mlx5.c b/drivers/net/mlx5/mlx5.c
index a316989..167e14b 100644
--- a/drivers/net/mlx5/mlx5.c
+++ b/drivers/net/mlx5/mlx5.c
@@ -301,7 +301,9 @@ mlx5_pci_devinit(struct rte_pci_driver *pci_drv, struct rte_pci_device *pci_dev)
struct ether_addr mac;
#ifdef HAVE_EXP_QUERY_DEVICE
- exp_device_attr.comp_mask = IBV_EXP_DEVICE_ATTR_EXP_CAP_FLAGS;
+ exp_device_attr.comp_mask =
+ IBV_EXP_DEVICE_ATTR_EXP_CAP_FLAGS |
+ IBV_EXP_DEVICE_ATTR_RX_HASH;
#endif /* HAVE_EXP_QUERY_DEVICE */
DEBUG("using port %u (%08" PRIx32 ")", port, test);
@@ -365,6 +367,12 @@ mlx5_pci_devinit(struct rte_pci_driver *pci_drv, struct rte_pci_device *pci_dev)
DEBUG("L2 tunnel checksum offloads are %ssupported",
(priv->hw_csum_l2tun ? "" : "not "));
+ priv->ind_table_max_size = exp_device_attr.rx_hash_caps.max_rwq_indirection_table_size;
+ DEBUG("maximum RX indirection table size is %u",
+ priv->ind_table_max_size);
+
+#else /* HAVE_EXP_QUERY_DEVICE */
+ priv->ind_table_max_size = RSS_INDIRECTION_TABLE_SIZE;
#endif /* HAVE_EXP_QUERY_DEVICE */
priv->vf = vf;
diff --git a/drivers/net/mlx5/mlx5.h b/drivers/net/mlx5/mlx5.h
index 08900f5..b099dac 100644
--- a/drivers/net/mlx5/mlx5.h
+++ b/drivers/net/mlx5/mlx5.h
@@ -114,6 +114,7 @@ struct priv {
/* Indirection tables referencing all RX WQs. */
struct ibv_exp_rwq_ind_table *(*ind_tables)[];
unsigned int ind_tables_n; /* Number of indirection tables. */
+ unsigned int ind_table_max_size; /* Maximum indirection table size. */
/* Hash RX QPs feeding the indirection table. */
struct hash_rxq (*hash_rxqs)[];
unsigned int hash_rxqs_n; /* Hash RX QPs array size. */
diff --git a/drivers/net/mlx5/mlx5_defs.h b/drivers/net/mlx5/mlx5_defs.h
index 79de609..e697764 100644
--- a/drivers/net/mlx5/mlx5_defs.h
+++ b/drivers/net/mlx5/mlx5_defs.h
@@ -53,6 +53,9 @@
/* Request send completion once in every 64 sends, might be less. */
#define MLX5_PMD_TX_PER_COMP_REQ 64
+/* RSS Indirection table size. */
+#define RSS_INDIRECTION_TABLE_SIZE 128
+
/* Maximum number of Scatter/Gather Elements per Work Request. */
#ifndef MLX5_PMD_SGE_WR_N
#define MLX5_PMD_SGE_WR_N 4
diff --git a/drivers/net/mlx5/mlx5_rxq.c b/drivers/net/mlx5/mlx5_rxq.c
index b5084f8..606367c 100644
--- a/drivers/net/mlx5/mlx5_rxq.c
+++ b/drivers/net/mlx5/mlx5_rxq.c
@@ -224,7 +224,13 @@ priv_make_ind_table_init(struct priv *priv,
int
priv_create_hash_rxqs(struct priv *priv)
{
- unsigned int wqs_n = (1 << log2above(priv->rxqs_n));
+ /* If the requested number of WQs is not a power of two, use the
+ * maximum indirection table size for better balancing.
+ * The result is always rounded to the next power of two. */
+ unsigned int wqs_n =
+ (1 << log2above((priv->rxqs_n & (priv->rxqs_n - 1)) ?
+ priv->ind_table_max_size :
+ priv->rxqs_n));
struct ibv_exp_wq *wqs[wqs_n];
struct ind_table_init ind_table_init[IND_TABLE_INIT_N];
unsigned int ind_tables_n =
@@ -251,16 +257,17 @@ priv_create_hash_rxqs(struct priv *priv)
" indirection table cannot be created");
return EINVAL;
}
- if (wqs_n < priv->rxqs_n) {
+ if ((wqs_n < priv->rxqs_n) || (wqs_n > priv->ind_table_max_size)) {
ERROR("cannot handle this many RX queues (%u)", priv->rxqs_n);
err = ERANGE;
goto error;
}
- if (wqs_n != priv->rxqs_n)
- WARN("%u RX queues are configured, consider rounding this"
- " number to the next power of two (%u) for optimal"
- " performance",
- priv->rxqs_n, wqs_n);
+ if (wqs_n != priv->rxqs_n) {
+ INFO("%u RX queues are configured, consider rounding this"
+ " number to the next power of two for better balancing",
+ priv->rxqs_n);
+ DEBUG("indirection table extended to assume %u WQs", wqs_n);
+ }
/* When the number of RX queues is not a power of two, the remaining
* table entries are padded with reused WQs and hashes are not spread
* uniformly. */
--
2.1.0
next prev parent reply other threads:[~2015-10-05 17:55 UTC|newest]
Thread overview: 39+ messages / expand[flat|nested] mbox.gz Atom feed top
2015-10-05 17:54 [dpdk-dev] [PATCH 00/17] Enhance mlx5 with Mellanox OFED 3.1 Adrien Mazarguil
2015-10-05 17:54 ` [dpdk-dev] [PATCH 01/17] mlx5: use fast Verbs interface for scattered RX operation Adrien Mazarguil
2015-10-05 17:54 ` [dpdk-dev] [PATCH 02/17] mlx5: get rid of the WR structure in RX queue elements Adrien Mazarguil
2015-10-05 17:54 ` [dpdk-dev] [PATCH 03/17] mlx5: refactor RX code for the new Verbs RSS API Adrien Mazarguil
2015-10-05 17:54 ` [dpdk-dev] [PATCH 04/17] mlx5: restore allmulti and promisc modes after device restart Adrien Mazarguil
2015-10-05 17:54 ` [dpdk-dev] [PATCH 05/17] mlx5: use separate indirection table for default hash RX queue Adrien Mazarguil
2015-10-05 17:54 ` Adrien Mazarguil [this message]
2015-10-05 17:54 ` [dpdk-dev] [PATCH 07/17] mlx5: define specific flow steering rules for each hash RX QP Adrien Mazarguil
2015-10-05 17:54 ` [dpdk-dev] [PATCH 08/17] mlx5: use alternate method to configure promiscuous mode Adrien Mazarguil
2015-10-05 17:54 ` [dpdk-dev] [PATCH 09/17] mlx5: add RSS hash update/get Adrien Mazarguil
2015-10-05 17:54 ` [dpdk-dev] [PATCH 10/17] mlx5: use one RSS hash key per flow type Adrien Mazarguil
2015-10-05 17:54 ` [dpdk-dev] [PATCH 11/17] app/testpmd: add missing type to RSS hash commands Adrien Mazarguil
2015-10-05 17:54 ` [dpdk-dev] [PATCH 12/17] app/testpmd: fix missing initialization in the RSS hash show command Adrien Mazarguil
2015-10-05 17:54 ` [dpdk-dev] [PATCH 13/17] mlx5: remove normal MAC flows when enabling promiscuous mode Adrien Mazarguil
2015-10-05 17:54 ` [dpdk-dev] [PATCH 14/17] mlx5: use experimental flows in hash RX queues Adrien Mazarguil
2015-10-05 17:54 ` [dpdk-dev] [PATCH 15/17] mlx5: enable multi packet send WR in TX CQ Adrien Mazarguil
2015-10-05 17:54 ` [dpdk-dev] [PATCH 16/17] mlx5: fix compilation error with GCC < 4.6 Adrien Mazarguil
2015-10-05 17:54 ` [dpdk-dev] [PATCH 17/17] doc: update mlx5 documentation Adrien Mazarguil
2015-10-06 8:54 ` [dpdk-dev] [PATCH 00/17] Enhance mlx5 with Mellanox OFED 3.1 Stephen Hemminger
2015-10-06 9:58 ` Vincent JARDIN
2015-10-07 13:30 ` Joongi Kim
2015-10-30 18:55 ` [dpdk-dev] [PATCH v2 00/16] " Adrien Mazarguil
2015-10-30 18:55 ` [dpdk-dev] [PATCH v2 01/16] mlx5: use fast Verbs interface for scattered RX operation Adrien Mazarguil
2015-10-30 18:55 ` [dpdk-dev] [PATCH v2 02/16] mlx5: get rid of the WR structure in RX queue elements Adrien Mazarguil
2015-10-30 18:55 ` [dpdk-dev] [PATCH v2 03/16] mlx5: refactor RX code for the new Verbs RSS API Adrien Mazarguil
2015-10-30 18:55 ` [dpdk-dev] [PATCH v2 04/16] mlx5: use separate indirection table for default hash RX queue Adrien Mazarguil
2015-10-30 18:55 ` [dpdk-dev] [PATCH v2 05/16] mlx5: adapt indirection table size depending on RX queues number Adrien Mazarguil
2015-10-30 18:55 ` [dpdk-dev] [PATCH v2 06/16] mlx5: define specific flow steering rules for each hash RX QP Adrien Mazarguil
2015-10-30 18:55 ` [dpdk-dev] [PATCH v2 07/16] mlx5: use alternate method to configure promisc and allmulti modes Adrien Mazarguil
2015-10-30 18:55 ` [dpdk-dev] [PATCH v2 08/16] mlx5: add RSS hash update/get Adrien Mazarguil
2015-10-30 18:55 ` [dpdk-dev] [PATCH v2 09/16] mlx5: use one RSS hash key per flow type Adrien Mazarguil
2015-10-30 18:55 ` [dpdk-dev] [PATCH v2 10/16] app/testpmd: add missing type to RSS hash commands Adrien Mazarguil
2015-10-30 18:55 ` [dpdk-dev] [PATCH v2 11/16] app/testpmd: fix missing initialization in the RSS hash show command Adrien Mazarguil
2015-10-30 18:55 ` [dpdk-dev] [PATCH v2 12/16] mlx5: disable useless flows in promiscuous mode Adrien Mazarguil
2015-10-30 18:55 ` [dpdk-dev] [PATCH v2 13/16] mlx5: add IPv6 RSS support using experimental flows Adrien Mazarguil
2015-10-30 18:55 ` [dpdk-dev] [PATCH v2 14/16] mlx5: enable multi packet send WR in TX CQ Adrien Mazarguil
2015-10-30 18:55 ` [dpdk-dev] [PATCH v2 15/16] mlx5: fix compilation error with GCC < 4.6 Adrien Mazarguil
2015-10-30 18:55 ` [dpdk-dev] [PATCH v2 16/16] doc: update mlx5 documentation Adrien Mazarguil
2015-11-01 10:26 ` [dpdk-dev] [PATCH v2 00/16] Enhance mlx5 with Mellanox OFED 3.1 Thomas Monjalon
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=1444067692-29645-7-git-send-email-adrien.mazarguil@6wind.com \
--to=adrien.mazarguil@6wind.com \
--cc=dev@dpdk.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).