From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 1413CA055A for ; Thu, 27 Feb 2020 10:35:03 +0100 (CET) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 085E81C01; Thu, 27 Feb 2020 10:35:03 +0100 (CET) Received: from mail-wr1-f67.google.com (mail-wr1-f67.google.com [209.85.221.67]) by dpdk.org (Postfix) with ESMTP id DD9B61BFE5 for ; Thu, 27 Feb 2020 10:35:01 +0100 (CET) Received: by mail-wr1-f67.google.com with SMTP id u6so2453242wrt.0 for ; Thu, 27 Feb 2020 01:35:01 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=ohmtxvEQgMemP3h+5sjRFCdhCY/4XBQbPj1v+pqGilU=; b=Tff4Zlnwmg9diFzrZoNVgM4f90yWl801pxtp7WfzWvWjwGXjBWDOgmzYjbQpmOexbq lDjnC+Dg5rjG8OiAXJMX7xLaTYJ3Tdae0bg5Sgczu9CwbS1FDtx3474KRkA828AYXnD7 bfl/0fmIj6VgDmr5OJT4Oz/w7HLBE4xMr15LVz7MuTWjEksDb8lHPHseeIxvPyje3W/a vYc9SHitZQS7opbm14SP4vhS8uZ5SWqKzCj53zNam/opX+q7sMtDCPC0R5uAksjajgC+ G5JVXwDTg9LpPLG5iqRn7fObXiuq7kwKZdhoZsRt1bBEe6wkS/Q2o9OtNpA/IWkIuwlJ kETQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=ohmtxvEQgMemP3h+5sjRFCdhCY/4XBQbPj1v+pqGilU=; b=TDhObfy+iwrEgdUsE/sM/y1RqzfFTdi8ozm+f/jic4L2FAmaJS7p/l718lvriEJ9Lu mD/F/xIqCvYuBhtAT1EtG3xiqFPicFwMONzMCjqSZNRp7JElN/BNNRXbLKlEEvx5K/q8 LgXq6CJgEIz0PdFSBc/1u96hA1xo4814CR2YogtMg7yjUimYaVSvXScUmxs3T+Hb3S4K RqrpuBxT0KibwZu28tT08f08JWpJZWk8tbxxdDpU9v6oSPvhpBeT4ljNy6889Z2Nw+d6 PQHsQQGnMcG0trjHdglmELyu9aLGLr0NdixDqv+LS/oNRjKBANj+Xx5/nL6viZ1mK29G Y/CQ== X-Gm-Message-State: APjAAAUQAVV72al+0ZjvW27TU2BYIvrxauG/yshcbHzSliOeIL0zy3vP 8VGp39CyvH9E9V/MQb0V4rXv60n7mAs= X-Google-Smtp-Source: APXvYqwnyWex8izuIvuDKlZbcuxrChlDdef5cfhFjWtYroCU1r5fnwPhCsZaa3h3Jg9tOsIHJXh6ng== X-Received: by 2002:adf:ed84:: with SMTP id c4mr3773961wro.24.1582796101598; Thu, 27 Feb 2020 01:35:01 -0800 (PST) Received: from localhost ([88.98.246.218]) by smtp.gmail.com with ESMTPSA id q6sm7267000wrf.67.2020.02.27.01.35.00 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 27 Feb 2020 01:35:00 -0800 (PST) From: luca.boccassi@gmail.com To: Bing Zhao Cc: Ori Kam , dpdk stable Date: Thu, 27 Feb 2020 09:33:38 +0000 Message-Id: <20200227093402.17690-18-luca.boccassi@gmail.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20200227093402.17690-1-luca.boccassi@gmail.com> References: <20200217174546.25334-54-luca.boccassi@gmail.com> <20200227093402.17690-1-luca.boccassi@gmail.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Subject: [dpdk-stable] patch 'net/mlx5: fix hairpin queue capacity' has been queued to stable release 19.11.1 X-BeenThere: stable@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: patches for DPDK stable branches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: stable-bounces@dpdk.org Sender: "stable" Hi, FYI, your patch has been queued to stable release 19.11.1 Note it hasn't been pushed to http://dpdk.org/browse/dpdk-stable yet. It will be pushed if I get no objections before 02/29/20. So please shout if anyone has objections. Also note that after the patch there's a diff of the upstream commit vs the patch applied to the branch. This will indicate if there was any rebasing needed to apply to the stable branch. If there were code changes for rebasing (ie: not only metadata diffs), please double check that the rebase was correctly done. Thanks. Luca Boccassi --- >From 6455e561a6becf768147f4749074d3b90ad6e39c Mon Sep 17 00:00:00 2001 From: Bing Zhao Date: Wed, 19 Feb 2020 16:28:39 +0800 Subject: [PATCH] net/mlx5: fix hairpin queue capacity [ upstream commit c288d7b5ebc645e45c0477e6f0263020c033abc2 ] The hairpin TX/RX queue depth and packet size is fixed in the past. When the firmware has some fix or improvement, the PMD will not make full use of it. And also, 32 packets for a single queue will not guarantee a good performance for hairpin flows. It will make the stride size larger and for small packets, it is a waste of memory. The recommended stride size is 64B now. The parameter of hairpin queue setup needs to be adjusted. 1. A proper buffer size should support the standard jumbo frame with 9KB, and also more than 1 jumbo frame packet for performance. 2. Number of packets of a single queue should be the maximum supported value (total buffer size / stride size). There is no need to support the max capacity of total buffer size because the memory consumption should also be taken into consideration. Fixes: e79c9be91515 ("net/mlx5: support Rx hairpin queues") Signed-off-by: Bing Zhao Acked-by: Ori Kam --- drivers/net/mlx5/mlx5_defs.h | 4 ++++ drivers/net/mlx5/mlx5_rxq.c | 13 +++++++++---- drivers/net/mlx5/mlx5_txq.c | 13 +++++++++---- 3 files changed, 22 insertions(+), 8 deletions(-) diff --git a/drivers/net/mlx5/mlx5_defs.h b/drivers/net/mlx5/mlx5_defs.h index 042e1f31ee..330666cd60 100644 --- a/drivers/net/mlx5/mlx5_defs.h +++ b/drivers/net/mlx5/mlx5_defs.h @@ -176,6 +176,10 @@ #define MLX5_FLOW_MREG_HNAME "MARK_COPY_TABLE" #define MLX5_DEFAULT_COPY_ID UINT32_MAX +/* Hairpin TX/RX queue configuration parameters. */ +#define MLX5_HAIRPIN_QUEUE_STRIDE 6 +#define MLX5_HAIRPIN_JUMBO_LOG_SIZE (15 + 2) + /* Definition of static_assert found in /usr/include/assert.h */ #ifndef HAVE_STATIC_ASSERT #define static_assert _Static_assert diff --git a/drivers/net/mlx5/mlx5_rxq.c b/drivers/net/mlx5/mlx5_rxq.c index da020a0aa0..5f3740b812 100644 --- a/drivers/net/mlx5/mlx5_rxq.c +++ b/drivers/net/mlx5/mlx5_rxq.c @@ -1261,6 +1261,7 @@ mlx5_rxq_obj_hairpin_new(struct rte_eth_dev *dev, uint16_t idx) struct mlx5_devx_create_rq_attr attr = { 0 }; struct mlx5_rxq_obj *tmpl = NULL; int ret = 0; + uint32_t max_wq_data; assert(rxq_data); assert(!rxq_ctrl->obj); @@ -1276,11 +1277,15 @@ mlx5_rxq_obj_hairpin_new(struct rte_eth_dev *dev, uint16_t idx) tmpl->type = MLX5_RXQ_OBJ_TYPE_DEVX_HAIRPIN; tmpl->rxq_ctrl = rxq_ctrl; attr.hairpin = 1; - /* Workaround for hairpin startup */ - attr.wq_attr.log_hairpin_num_packets = log2above(32); - /* Workaround for packets larger than 1KB */ + max_wq_data = priv->config.hca_attr.log_max_hairpin_wq_data_sz; + /* Jumbo frames > 9KB should be supported, and more packets. */ attr.wq_attr.log_hairpin_data_sz = - priv->config.hca_attr.log_max_hairpin_wq_data_sz; + (max_wq_data < MLX5_HAIRPIN_JUMBO_LOG_SIZE) ? + max_wq_data : MLX5_HAIRPIN_JUMBO_LOG_SIZE; + /* Set the packets number to the maximum value for performance. */ + attr.wq_attr.log_hairpin_num_packets = + attr.wq_attr.log_hairpin_data_sz - + MLX5_HAIRPIN_QUEUE_STRIDE; tmpl->rq = mlx5_devx_cmd_create_rq(priv->sh->ctx, &attr, rxq_ctrl->socket); if (!tmpl->rq) { diff --git a/drivers/net/mlx5/mlx5_txq.c b/drivers/net/mlx5/mlx5_txq.c index 1a76f6ea83..50d2464fa3 100644 --- a/drivers/net/mlx5/mlx5_txq.c +++ b/drivers/net/mlx5/mlx5_txq.c @@ -491,6 +491,7 @@ mlx5_txq_obj_hairpin_new(struct rte_eth_dev *dev, uint16_t idx) struct mlx5_devx_create_sq_attr attr = { 0 }; struct mlx5_txq_obj *tmpl = NULL; int ret = 0; + uint32_t max_wq_data; assert(txq_data); assert(!txq_ctrl->obj); @@ -507,11 +508,15 @@ mlx5_txq_obj_hairpin_new(struct rte_eth_dev *dev, uint16_t idx) tmpl->txq_ctrl = txq_ctrl; attr.hairpin = 1; attr.tis_lst_sz = 1; - /* Workaround for hairpin startup */ - attr.wq_attr.log_hairpin_num_packets = log2above(32); - /* Workaround for packets larger than 1KB */ + max_wq_data = priv->config.hca_attr.log_max_hairpin_wq_data_sz; + /* Jumbo frames > 9KB should be supported, and more packets. */ attr.wq_attr.log_hairpin_data_sz = - priv->config.hca_attr.log_max_hairpin_wq_data_sz; + (max_wq_data < MLX5_HAIRPIN_JUMBO_LOG_SIZE) ? + max_wq_data : MLX5_HAIRPIN_JUMBO_LOG_SIZE; + /* Set the packets number to the maximum value for performance. */ + attr.wq_attr.log_hairpin_num_packets = + attr.wq_attr.log_hairpin_data_sz - + MLX5_HAIRPIN_QUEUE_STRIDE; attr.tis_num = priv->sh->tis->id; tmpl->sq = mlx5_devx_cmd_create_sq(priv->sh->ctx, &attr); if (!tmpl->sq) { -- 2.20.1 --- Diff of the applied patch vs upstream commit (please double-check if non-empty: --- --- - 2020-02-27 09:31:56.426157115 +0000 +++ 0018-net-mlx5-fix-hairpin-queue-capacity.patch 2020-02-27 09:31:55.731945844 +0000 @@ -1,8 +1,10 @@ -From c288d7b5ebc645e45c0477e6f0263020c033abc2 Mon Sep 17 00:00:00 2001 +From 6455e561a6becf768147f4749074d3b90ad6e39c Mon Sep 17 00:00:00 2001 From: Bing Zhao Date: Wed, 19 Feb 2020 16:28:39 +0800 Subject: [PATCH] net/mlx5: fix hairpin queue capacity +[ upstream commit c288d7b5ebc645e45c0477e6f0263020c033abc2 ] + The hairpin TX/RX queue depth and packet size is fixed in the past. When the firmware has some fix or improvement, the PMD will not make full use of it. And also, 32 packets for a single queue will not @@ -21,7 +23,6 @@ consideration. Fixes: e79c9be91515 ("net/mlx5: support Rx hairpin queues") -Cc: stable@dpdk.org Signed-off-by: Bing Zhao Acked-by: Ori Kam @@ -32,10 +33,10 @@ 3 files changed, 22 insertions(+), 8 deletions(-) diff --git a/drivers/net/mlx5/mlx5_defs.h b/drivers/net/mlx5/mlx5_defs.h -index 9b392ed036..83ca367dba 100644 +index 042e1f31ee..330666cd60 100644 --- a/drivers/net/mlx5/mlx5_defs.h +++ b/drivers/net/mlx5/mlx5_defs.h -@@ -173,6 +173,10 @@ +@@ -176,6 +176,10 @@ #define MLX5_FLOW_MREG_HNAME "MARK_COPY_TABLE" #define MLX5_DEFAULT_COPY_ID UINT32_MAX @@ -47,18 +48,18 @@ #ifndef HAVE_STATIC_ASSERT #define static_assert _Static_assert diff --git a/drivers/net/mlx5/mlx5_rxq.c b/drivers/net/mlx5/mlx5_rxq.c -index dc0fd82113..8a6b410eff 100644 +index da020a0aa0..5f3740b812 100644 --- a/drivers/net/mlx5/mlx5_rxq.c +++ b/drivers/net/mlx5/mlx5_rxq.c -@@ -1268,6 +1268,7 @@ mlx5_rxq_obj_hairpin_new(struct rte_eth_dev *dev, uint16_t idx) +@@ -1261,6 +1261,7 @@ mlx5_rxq_obj_hairpin_new(struct rte_eth_dev *dev, uint16_t idx) struct mlx5_devx_create_rq_attr attr = { 0 }; struct mlx5_rxq_obj *tmpl = NULL; int ret = 0; + uint32_t max_wq_data; - MLX5_ASSERT(rxq_data); - MLX5_ASSERT(!rxq_ctrl->obj); -@@ -1283,11 +1284,15 @@ mlx5_rxq_obj_hairpin_new(struct rte_eth_dev *dev, uint16_t idx) + assert(rxq_data); + assert(!rxq_ctrl->obj); +@@ -1276,11 +1277,15 @@ mlx5_rxq_obj_hairpin_new(struct rte_eth_dev *dev, uint16_t idx) tmpl->type = MLX5_RXQ_OBJ_TYPE_DEVX_HAIRPIN; tmpl->rxq_ctrl = rxq_ctrl; attr.hairpin = 1; @@ -79,18 +80,18 @@ rxq_ctrl->socket); if (!tmpl->rq) { diff --git a/drivers/net/mlx5/mlx5_txq.c b/drivers/net/mlx5/mlx5_txq.c -index bc13abfe63..2ad849a229 100644 +index 1a76f6ea83..50d2464fa3 100644 --- a/drivers/net/mlx5/mlx5_txq.c +++ b/drivers/net/mlx5/mlx5_txq.c -@@ -493,6 +493,7 @@ mlx5_txq_obj_hairpin_new(struct rte_eth_dev *dev, uint16_t idx) +@@ -491,6 +491,7 @@ mlx5_txq_obj_hairpin_new(struct rte_eth_dev *dev, uint16_t idx) struct mlx5_devx_create_sq_attr attr = { 0 }; struct mlx5_txq_obj *tmpl = NULL; int ret = 0; + uint32_t max_wq_data; - MLX5_ASSERT(txq_data); - MLX5_ASSERT(!txq_ctrl->obj); -@@ -509,11 +510,15 @@ mlx5_txq_obj_hairpin_new(struct rte_eth_dev *dev, uint16_t idx) + assert(txq_data); + assert(!txq_ctrl->obj); +@@ -507,11 +508,15 @@ mlx5_txq_obj_hairpin_new(struct rte_eth_dev *dev, uint16_t idx) tmpl->txq_ctrl = txq_ctrl; attr.hairpin = 1; attr.tis_lst_sz = 1;