From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-wi0-f170.google.com (mail-wi0-f170.google.com [209.85.212.170]) by dpdk.org (Postfix) with ESMTP id 8E8D293BA for ; Mon, 5 Oct 2015 19:55:26 +0200 (CEST) Received: by wicfx3 with SMTP id fx3so131837544wic.1 for ; Mon, 05 Oct 2015 10:55:26 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=WD8Qos+YtuevYZvyVxTE3792NDgE1GVc63O0ipibivk=; b=DoWqIr9fvzN9wg2OU02wEgrlHLKjDrFXOMCLZx4f83bTKn+lH+rG0Ri0HKkF5ycUl+ LWepSAGZpaCPgJ6rm30zRsEUt7T8w2GWdxXeIr2ZJ1oEPGO10CBTqOSVzg/XPnsMSVB8 Fbbzq7BhfWLdBtqpW7ML2sWyc8uas7LGziXYYYe0CMB2utPfaScMTDDKvfs12MrKp1AG ICdjFlmNseX9LadTWzFZhoslepwNtmSZofF6EgWOJT5dd7kppSiHbsDdfYx/uCF6oxCC 37ykKLrThWUVY21cPPC4o6onUqyC1UboaL/dTQVn/H3M7PBvGLDoYueR7VFVEUg9C5zH tJBQ== X-Gm-Message-State: ALoCoQm6coi37121lunc7skOmVDeyDXBQO8wpyR8OVz7pPaad0qdfAWhs3Lw+MAXyD2hjAqnKuVD X-Received: by 10.180.39.193 with SMTP id r1mr12191549wik.57.1444067726471; Mon, 05 Oct 2015 10:55:26 -0700 (PDT) Received: from 6wind.com (guy78-3-82-239-227-177.fbx.proxad.net. [82.239.227.177]) by smtp.gmail.com with ESMTPSA id pk7sm28129867wjb.2.2015.10.05.10.55.25 (version=TLSv1.2 cipher=RC4-SHA bits=128/128); Mon, 05 Oct 2015 10:55:26 -0700 (PDT) From: Adrien Mazarguil To: dev@dpdk.org Date: Mon, 5 Oct 2015 19:54:41 +0200 Message-Id: <1444067692-29645-7-git-send-email-adrien.mazarguil@6wind.com> X-Mailer: git-send-email 2.1.0 In-Reply-To: <1444067692-29645-1-git-send-email-adrien.mazarguil@6wind.com> References: <1444067692-29645-1-git-send-email-adrien.mazarguil@6wind.com> Subject: [dpdk-dev] [PATCH 06/17] mlx5: adapt indirection table size depending on RX queues number X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: patches and discussions about DPDK List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 05 Oct 2015 17:55:26 -0000 From: Nelio Laranjeiro Use the maximum size of the indirection table when the number of requested RX queues is not a power of two, this help to improve RSS balancing. A message informs users that balancing is not optimal in such cases. Signed-off-by: Nelio Laranjeiro Signed-off-by: Adrien Mazarguil --- drivers/net/mlx5/mlx5.c | 10 +++++++++- drivers/net/mlx5/mlx5.h | 1 + drivers/net/mlx5/mlx5_defs.h | 3 +++ drivers/net/mlx5/mlx5_rxq.c | 21 ++++++++++++++------- 4 files changed, 27 insertions(+), 8 deletions(-) diff --git a/drivers/net/mlx5/mlx5.c b/drivers/net/mlx5/mlx5.c index a316989..167e14b 100644 --- a/drivers/net/mlx5/mlx5.c +++ b/drivers/net/mlx5/mlx5.c @@ -301,7 +301,9 @@ mlx5_pci_devinit(struct rte_pci_driver *pci_drv, struct rte_pci_device *pci_dev) struct ether_addr mac; #ifdef HAVE_EXP_QUERY_DEVICE - exp_device_attr.comp_mask = IBV_EXP_DEVICE_ATTR_EXP_CAP_FLAGS; + exp_device_attr.comp_mask = + IBV_EXP_DEVICE_ATTR_EXP_CAP_FLAGS | + IBV_EXP_DEVICE_ATTR_RX_HASH; #endif /* HAVE_EXP_QUERY_DEVICE */ DEBUG("using port %u (%08" PRIx32 ")", port, test); @@ -365,6 +367,12 @@ mlx5_pci_devinit(struct rte_pci_driver *pci_drv, struct rte_pci_device *pci_dev) DEBUG("L2 tunnel checksum offloads are %ssupported", (priv->hw_csum_l2tun ? "" : "not ")); + priv->ind_table_max_size = exp_device_attr.rx_hash_caps.max_rwq_indirection_table_size; + DEBUG("maximum RX indirection table size is %u", + priv->ind_table_max_size); + +#else /* HAVE_EXP_QUERY_DEVICE */ + priv->ind_table_max_size = RSS_INDIRECTION_TABLE_SIZE; #endif /* HAVE_EXP_QUERY_DEVICE */ priv->vf = vf; diff --git a/drivers/net/mlx5/mlx5.h b/drivers/net/mlx5/mlx5.h index 08900f5..b099dac 100644 --- a/drivers/net/mlx5/mlx5.h +++ b/drivers/net/mlx5/mlx5.h @@ -114,6 +114,7 @@ struct priv { /* Indirection tables referencing all RX WQs. */ struct ibv_exp_rwq_ind_table *(*ind_tables)[]; unsigned int ind_tables_n; /* Number of indirection tables. */ + unsigned int ind_table_max_size; /* Maximum indirection table size. */ /* Hash RX QPs feeding the indirection table. */ struct hash_rxq (*hash_rxqs)[]; unsigned int hash_rxqs_n; /* Hash RX QPs array size. */ diff --git a/drivers/net/mlx5/mlx5_defs.h b/drivers/net/mlx5/mlx5_defs.h index 79de609..e697764 100644 --- a/drivers/net/mlx5/mlx5_defs.h +++ b/drivers/net/mlx5/mlx5_defs.h @@ -53,6 +53,9 @@ /* Request send completion once in every 64 sends, might be less. */ #define MLX5_PMD_TX_PER_COMP_REQ 64 +/* RSS Indirection table size. */ +#define RSS_INDIRECTION_TABLE_SIZE 128 + /* Maximum number of Scatter/Gather Elements per Work Request. */ #ifndef MLX5_PMD_SGE_WR_N #define MLX5_PMD_SGE_WR_N 4 diff --git a/drivers/net/mlx5/mlx5_rxq.c b/drivers/net/mlx5/mlx5_rxq.c index b5084f8..606367c 100644 --- a/drivers/net/mlx5/mlx5_rxq.c +++ b/drivers/net/mlx5/mlx5_rxq.c @@ -224,7 +224,13 @@ priv_make_ind_table_init(struct priv *priv, int priv_create_hash_rxqs(struct priv *priv) { - unsigned int wqs_n = (1 << log2above(priv->rxqs_n)); + /* If the requested number of WQs is not a power of two, use the + * maximum indirection table size for better balancing. + * The result is always rounded to the next power of two. */ + unsigned int wqs_n = + (1 << log2above((priv->rxqs_n & (priv->rxqs_n - 1)) ? + priv->ind_table_max_size : + priv->rxqs_n)); struct ibv_exp_wq *wqs[wqs_n]; struct ind_table_init ind_table_init[IND_TABLE_INIT_N]; unsigned int ind_tables_n = @@ -251,16 +257,17 @@ priv_create_hash_rxqs(struct priv *priv) " indirection table cannot be created"); return EINVAL; } - if (wqs_n < priv->rxqs_n) { + if ((wqs_n < priv->rxqs_n) || (wqs_n > priv->ind_table_max_size)) { ERROR("cannot handle this many RX queues (%u)", priv->rxqs_n); err = ERANGE; goto error; } - if (wqs_n != priv->rxqs_n) - WARN("%u RX queues are configured, consider rounding this" - " number to the next power of two (%u) for optimal" - " performance", - priv->rxqs_n, wqs_n); + if (wqs_n != priv->rxqs_n) { + INFO("%u RX queues are configured, consider rounding this" + " number to the next power of two for better balancing", + priv->rxqs_n); + DEBUG("indirection table extended to assume %u WQs", wqs_n); + } /* When the number of RX queues is not a power of two, the remaining * table entries are padded with reused WQs and hashes are not spread * uniformly. */ -- 2.1.0