From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 29F72468E4 for ; Thu, 12 Jun 2025 23:08:55 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 2535042E0C; Thu, 12 Jun 2025 23:08:55 +0200 (CEST) Received: from mail-wm1-f42.google.com (mail-wm1-f42.google.com [209.85.128.42]) by mails.dpdk.org (Postfix) with ESMTP id A7FD142DD9 for ; Thu, 12 Jun 2025 23:08:53 +0200 (CEST) Received: by mail-wm1-f42.google.com with SMTP id 5b1f17b1804b1-451d41e1ad1so12279455e9.1 for ; Thu, 12 Jun 2025 14:08:53 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1749762533; x=1750367333; darn=dpdk.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=deKpedU07SVLMtTy4mlm8niCA5H8EoQXJVoUOIgvKUA=; b=IMJOBYfkbxd7szT5wcJb1YB7O3ALAnnyug4JmdufgETKJwP6hSkvll4EGCfQk7qnRp 5MqXsWkdbB6XQ1ZcLhsUKEinA6VQ8ESO7OJrwm9ZMyCgKJAHFW0DrMC+5PjvpGzU9OjL v36tWdhHqwsa0n//aMG0LahQEO+obF1+Jq+x+o/buhUFuZ1WcvoXoxpLtM5uy7EaMEe5 jv1RnyzGxgdSQ03WdkePB86vCgfyjWG1k3B7RnHI8DVKE5b9+5dsNOTz0QFBHtYAR3KJ 3p/jii2VLcpZuzy+yRw7dS5UhCjKcHUg+v+ScaSR6T5v4WQtd6wfmHrAmqEgpECNCzj5 RArg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1749762533; x=1750367333; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=deKpedU07SVLMtTy4mlm8niCA5H8EoQXJVoUOIgvKUA=; b=jcvzQ3ovZlmIAvrTPKc+oDcqwZofZwUSArHG8uXULo4H45DL6BFvDCqg6CgWdZnPOV GhvYMbDw1LnN3zxwuus2mbDBDyZWJVFoyQIc3ItIlAAaMl/3KcqDo+zebOz4cVmrwBpS Tbcf0wmnsnggOxCmwWR1ybfrJCjjhURNw4fg5ilXdg9HqIPAFKZJjh2bysL4LxX+JMQ3 1PwsxZaOFKhYr+XooT1IMg9PQSkuqsynwAi+Vt9MvKDtVMA6510PovEvrg/SC5kFMvXZ 57L+Otg4YDq4Gdrxia00PEOdPPCx0Cc6OxZ/cw/xIAuWylWJzgquym+JNOT0F9eoxbBt q2Ww== X-Forwarded-Encrypted: i=1; AJvYcCXMFaHWIvEtvOw8ujXL8xikwz9jztDnXVTAJeYwlYsvUcyzZK/hRYBCihaDMbmpq9MoRPOhujg=@dpdk.org X-Gm-Message-State: AOJu0YxiP+dl+ADAfYlUh+4d97rufCcfbj1NA1MyOLuuX/4EMfq3Nfek ZFzl0mlXf0WSM93KFJg9Tq2lvZy2aJGmTZHI402HXiQpOz40LCr9KH8Z X-Gm-Gg: ASbGncuqpfXhBB3HQqZPY0SBup55tqxfgBGQROlzJ0lB1isMatVsuQd9zaWbnUrod3c 3YWfMTeLyUSeCKvqTohIJxpUjlh6MDCH8C+gC7vO/1TDI5G5gdHWN0kr9BD/WgKTeBmFVxZHbd0 aw5E4EitFbn57sldYnplktsTwapFHsPl0bFuJNq/s3xifvi9f8sJ4rxrulstJijKVVZJW5v9CTv HZciIh3WZBH3cdvsKmj0XGRB5QL58G3Yrzrb2NZ0CWgebVA607xyeQ60zOkH1Zvbp0KspuMZJ+2 1TgRfyu6lkw1BLdlluQLX43j7a1vxHLZSTesYuHQ6RH2UuqtAs6L8p2D4t7iEo3vYKYH X-Google-Smtp-Source: AGHT+IHwOxcuAN3feHQuG0JfhLOkiuIXcazxgbpnNU/Goo77E+Vz2viOxXPXC976LEHVbnKNaOnJ+g== X-Received: by 2002:a05:600c:c8d:b0:43c:f1b8:16ad with SMTP id 5b1f17b1804b1-45334af6e5emr4575005e9.30.1749762533039; Thu, 12 Jun 2025 14:08:53 -0700 (PDT) Received: from localhost ([2a01:4b00:d036:ae00:f2df:571a:ae4c:bef2]) by smtp.gmail.com with UTF8SMTPSA id ffacd0b85a97d-3a568a72cd3sm401137f8f.32.2025.06.12.14.08.52 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 12 Jun 2025 14:08:52 -0700 (PDT) From: luca.boccassi@gmail.com To: Viacheslav Ovsiienko Cc: Edwin Brossette , Dariusz Sosnowski , dpdk stable Subject: patch 'net/mlx5: fix maximal queue size query' has been queued to stable release 22.11.9 Date: Thu, 12 Jun 2025 22:06:40 +0100 Message-ID: <20250612210733.2506558-23-luca.boccassi@gmail.com> X-Mailer: git-send-email 2.47.2 In-Reply-To: <20250612210733.2506558-1-luca.boccassi@gmail.com> References: <20250612210733.2506558-1-luca.boccassi@gmail.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-BeenThere: stable@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: patches for DPDK stable branches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: stable-bounces@dpdk.org Hi, FYI, your patch has been queued to stable release 22.11.9 Note it hasn't been pushed to http://dpdk.org/browse/dpdk-stable yet. It will be pushed if I get no objections before 06/14/25. So please shout if anyone has objections. Also note that after the patch there's a diff of the upstream commit vs the patch applied to the branch. This will indicate if there was any rebasing needed to apply to the stable branch. If there were code changes for rebasing (ie: not only metadata diffs), please double check that the rebase was correctly done. Queued patches are on a temporary branch at: https://github.com/bluca/dpdk-stable This queued commit can be viewed at: https://github.com/bluca/dpdk-stable/commit/e9c54705c3230bcc5eb5585347faec0ed7778c9e Thanks. Luca Boccassi --- >From e9c54705c3230bcc5eb5585347faec0ed7778c9e Mon Sep 17 00:00:00 2001 From: Viacheslav Ovsiienko Date: Wed, 14 May 2025 10:55:30 +0300 Subject: [PATCH] net/mlx5: fix maximal queue size query [ upstream commit 9de8acd30d5adfc5b9703d15a3e1babc7d4ddacc ] The mlx5 PMD manages the device using two modes: the Verbs API and the DevX API. Each API offers its own method for querying the maximum work queue size (in descriptors). The corrected patch enhanced the rte_eth_dev_info_get() API support in mlx5 PMD to return the true maximum number of descriptors. It also implemented a limit check during queue creation, but this was applied only to "DevX mode." Consequently, the "Verbs mode" was overlooked, leading to malfunction on legacy NICs that do not support DevX. This patch adds support for Verbs mode, and all limit checks are updated accordingly. Fixes: 4c3d7961d900 ("net/mlx5: fix reported Rx/Tx descriptor limits") Reported-by: Edwin Brossette Signed-off-by: Viacheslav Ovsiienko Acked-by: Dariusz Sosnowski --- drivers/common/mlx5/mlx5_prm.h | 1 + drivers/net/mlx5/mlx5.h | 1 + drivers/net/mlx5/mlx5_devx.c | 2 +- drivers/net/mlx5/mlx5_ethdev.c | 39 +++++++++++++++++++++++++++++---- drivers/net/mlx5/mlx5_rxq.c | 2 +- drivers/net/mlx5/mlx5_trigger.c | 4 ++-- drivers/net/mlx5/mlx5_txq.c | 12 +++++----- 7 files changed, 47 insertions(+), 14 deletions(-) diff --git a/drivers/common/mlx5/mlx5_prm.h b/drivers/common/mlx5/mlx5_prm.h index cf525c14df..a57fb235ec 100644 --- a/drivers/common/mlx5/mlx5_prm.h +++ b/drivers/common/mlx5/mlx5_prm.h @@ -35,6 +35,7 @@ /* Hardware index widths. */ #define MLX5_CQ_INDEX_WIDTH 24 #define MLX5_WQ_INDEX_WIDTH 16 +#define MLX5_WQ_INDEX_MAX (1u << (MLX5_WQ_INDEX_WIDTH - 1)) /* WQE Segment sizes in bytes. */ #define MLX5_WSEG_SIZE 16u diff --git a/drivers/net/mlx5/mlx5.h b/drivers/net/mlx5/mlx5.h index 7053db5fa7..8052d8c426 100644 --- a/drivers/net/mlx5/mlx5.h +++ b/drivers/net/mlx5/mlx5.h @@ -1941,6 +1941,7 @@ int mlx5_representor_info_get(struct rte_eth_dev *dev, (((repr_id) >> 12) & 3) uint16_t mlx5_representor_id_encode(const struct mlx5_switch_info *info, enum rte_eth_representor_type hpf_type); +uint16_t mlx5_dev_get_max_wq_size(struct mlx5_dev_ctx_shared *sh); int mlx5_dev_infos_get(struct rte_eth_dev *dev, struct rte_eth_dev_info *info); int mlx5_fw_version_get(struct rte_eth_dev *dev, char *fw_ver, size_t fw_size); const uint32_t *mlx5_dev_supported_ptypes_get(struct rte_eth_dev *dev); diff --git a/drivers/net/mlx5/mlx5_devx.c b/drivers/net/mlx5/mlx5_devx.c index 7e0ec91328..cca7b86649 100644 --- a/drivers/net/mlx5/mlx5_devx.c +++ b/drivers/net/mlx5/mlx5_devx.c @@ -1513,7 +1513,7 @@ mlx5_txq_devx_obj_new(struct rte_eth_dev *dev, uint16_t idx) wqe_size = RTE_ALIGN(wqe_size, MLX5_WQE_SIZE) / MLX5_WQE_SIZE; /* Create Send Queue object with DevX. */ wqe_n = RTE_MIN((1UL << txq_data->elts_n) * wqe_size, - (uint32_t)priv->sh->dev_cap.max_qp_wr); + (uint32_t)mlx5_dev_get_max_wq_size(priv->sh)); log_desc_n = log2above(wqe_n); ret = mlx5_txq_create_devx_sq_resources(dev, idx, log_desc_n); if (ret) { diff --git a/drivers/net/mlx5/mlx5_ethdev.c b/drivers/net/mlx5/mlx5_ethdev.c index 08c6b18975..169612ca44 100644 --- a/drivers/net/mlx5/mlx5_ethdev.c +++ b/drivers/net/mlx5/mlx5_ethdev.c @@ -306,6 +306,37 @@ mlx5_set_txlimit_params(struct rte_eth_dev *dev, struct rte_eth_dev_info *info) info->tx_desc_lim.nb_mtu_seg_max = nb_max; } +/** + * Get maximal work queue size in WQEs + * + * @param sh + * Pointer to the device shared context. + * @return + * Maximal number of WQEs in queue + */ +uint16_t +mlx5_dev_get_max_wq_size(struct mlx5_dev_ctx_shared *sh) +{ + uint16_t max_wqe = MLX5_WQ_INDEX_MAX; + + if (sh->cdev->config.devx) { + /* use HCA properties for DevX config */ + MLX5_ASSERT(sh->cdev->config.hca_attr.log_max_wq_sz != 0); + MLX5_ASSERT(sh->cdev->config.hca_attr.log_max_wq_sz < MLX5_WQ_INDEX_WIDTH); + if (sh->cdev->config.hca_attr.log_max_wq_sz != 0 && + sh->cdev->config.hca_attr.log_max_wq_sz < MLX5_WQ_INDEX_WIDTH) + max_wqe = 1u << sh->cdev->config.hca_attr.log_max_wq_sz; + } else { + /* use IB device capabilities */ + MLX5_ASSERT(sh->dev_cap.max_qp_wr > 0); + MLX5_ASSERT((unsigned int)sh->dev_cap.max_qp_wr <= MLX5_WQ_INDEX_MAX); + if (sh->dev_cap.max_qp_wr > 0 && + (uint32_t)sh->dev_cap.max_qp_wr <= MLX5_WQ_INDEX_MAX) + max_wqe = (uint16_t)sh->dev_cap.max_qp_wr; + } + return max_wqe; +} + /** * DPDK callback to get information about the device. * @@ -319,6 +350,7 @@ mlx5_dev_infos_get(struct rte_eth_dev *dev, struct rte_eth_dev_info *info) { struct mlx5_priv *priv = dev->data->dev_private; unsigned int max; + uint16_t max_wqe; /* FIXME: we should ask the device for these values. */ info->min_rx_bufsize = 32; @@ -351,10 +383,9 @@ mlx5_dev_infos_get(struct rte_eth_dev *dev, struct rte_eth_dev_info *info) info->flow_type_rss_offloads = ~MLX5_RSS_HF_MASK; mlx5_set_default_params(dev, info); mlx5_set_txlimit_params(dev, info); - info->rx_desc_lim.nb_max = - 1 << priv->sh->cdev->config.hca_attr.log_max_wq_sz; - info->tx_desc_lim.nb_max = - 1 << priv->sh->cdev->config.hca_attr.log_max_wq_sz; + max_wqe = mlx5_dev_get_max_wq_size(priv->sh); + info->rx_desc_lim.nb_max = max_wqe; + info->tx_desc_lim.nb_max = max_wqe; if (priv->sh->cdev->config.hca_attr.mem_rq_rmp && priv->obj_ops.rxq_obj_new == devx_obj_ops.rxq_obj_new) info->dev_capa |= RTE_ETH_DEV_CAPA_RXQ_SHARE; diff --git a/drivers/net/mlx5/mlx5_rxq.c b/drivers/net/mlx5/mlx5_rxq.c index fcf6ab54b6..dfbb3b9a16 100644 --- a/drivers/net/mlx5/mlx5_rxq.c +++ b/drivers/net/mlx5/mlx5_rxq.c @@ -652,7 +652,7 @@ mlx5_rx_queue_pre_setup(struct rte_eth_dev *dev, uint16_t idx, uint16_t *desc, struct mlx5_rxq_priv *rxq; bool empty; - if (*desc > 1 << priv->sh->cdev->config.hca_attr.log_max_wq_sz) { + if (*desc > mlx5_dev_get_max_wq_size(priv->sh)) { DRV_LOG(ERR, "port %u number of descriptors requested for Rx queue" " %u is more than supported", diff --git a/drivers/net/mlx5/mlx5_trigger.c b/drivers/net/mlx5/mlx5_trigger.c index 786e638f95..b2e860d149 100644 --- a/drivers/net/mlx5/mlx5_trigger.c +++ b/drivers/net/mlx5/mlx5_trigger.c @@ -215,8 +215,8 @@ mlx5_rxq_start(struct rte_eth_dev *dev) /* Should not release Rx queues but return immediately. */ return -rte_errno; } - DRV_LOG(DEBUG, "Port %u dev_cap.max_qp_wr is %d.", - dev->data->port_id, priv->sh->dev_cap.max_qp_wr); + DRV_LOG(DEBUG, "Port %u max work queue size is %d.", + dev->data->port_id, mlx5_dev_get_max_wq_size(priv->sh)); DRV_LOG(DEBUG, "Port %u dev_cap.max_sge is %d.", dev->data->port_id, priv->sh->dev_cap.max_sge); for (i = 0; i != priv->rxqs_n; ++i) { diff --git a/drivers/net/mlx5/mlx5_txq.c b/drivers/net/mlx5/mlx5_txq.c index cdc9755fe0..9869cd4577 100644 --- a/drivers/net/mlx5/mlx5_txq.c +++ b/drivers/net/mlx5/mlx5_txq.c @@ -332,7 +332,7 @@ mlx5_tx_queue_pre_setup(struct rte_eth_dev *dev, uint16_t idx, uint16_t *desc) { struct mlx5_priv *priv = dev->data->dev_private; - if (*desc > 1 << priv->sh->cdev->config.hca_attr.log_max_wq_sz) { + if (*desc > mlx5_dev_get_max_wq_size(priv->sh)) { DRV_LOG(ERR, "port %u number of descriptors requested for Tx queue" " %u is more than supported", @@ -726,7 +726,7 @@ txq_calc_inline_max(struct mlx5_txq_ctrl *txq_ctrl) struct mlx5_priv *priv = txq_ctrl->priv; unsigned int wqe_size; - wqe_size = priv->sh->dev_cap.max_qp_wr / desc; + wqe_size = mlx5_dev_get_max_wq_size(priv->sh) / desc; if (!wqe_size) return 0; /* @@ -1081,6 +1081,7 @@ mlx5_txq_new(struct rte_eth_dev *dev, uint16_t idx, uint16_t desc, { struct mlx5_priv *priv = dev->data->dev_private; struct mlx5_txq_ctrl *tmpl; + uint16_t max_wqe; tmpl = mlx5_malloc(MLX5_MEM_RTE | MLX5_MEM_ZERO, sizeof(*tmpl) + desc * sizeof(struct rte_mbuf *), 0, socket); @@ -1106,13 +1107,12 @@ mlx5_txq_new(struct rte_eth_dev *dev, uint16_t idx, uint16_t desc, txq_set_params(tmpl); if (txq_adjust_params(tmpl)) goto error; - if (txq_calc_wqebb_cnt(tmpl) > - priv->sh->dev_cap.max_qp_wr) { + max_wqe = mlx5_dev_get_max_wq_size(priv->sh); + if (txq_calc_wqebb_cnt(tmpl) > max_wqe) { DRV_LOG(ERR, "port %u Tx WQEBB count (%d) exceeds the limit (%d)," " try smaller queue size", - dev->data->port_id, txq_calc_wqebb_cnt(tmpl), - priv->sh->dev_cap.max_qp_wr); + dev->data->port_id, txq_calc_wqebb_cnt(tmpl), max_wqe); rte_errno = ENOMEM; goto error; } -- 2.47.2 --- Diff of the applied patch vs upstream commit (please double-check if non-empty: --- --- - 2025-06-12 22:06:24.754365555 +0100 +++ 0023-net-mlx5-fix-maximal-queue-size-query.patch 2025-06-12 22:06:23.838043962 +0100 @@ -1 +1 @@ -From 9de8acd30d5adfc5b9703d15a3e1babc7d4ddacc Mon Sep 17 00:00:00 2001 +From e9c54705c3230bcc5eb5585347faec0ed7778c9e Mon Sep 17 00:00:00 2001 @@ -5,0 +6,2 @@ +[ upstream commit 9de8acd30d5adfc5b9703d15a3e1babc7d4ddacc ] + @@ -21 +22,0 @@ -Cc: stable@dpdk.org @@ -37 +38 @@ -index 742c274a85..7accdeab87 100644 +index cf525c14df..a57fb235ec 100644 @@ -40 +41 @@ -@@ -41,6 +41,7 @@ +@@ -35,6 +35,7 @@ @@ -49 +50 @@ -index 36f11b9c51..5695d0f54a 100644 +index 7053db5fa7..8052d8c426 100644 @@ -52 +53 @@ -@@ -2303,6 +2303,7 @@ int mlx5_representor_info_get(struct rte_eth_dev *dev, +@@ -1941,6 +1941,7 @@ int mlx5_representor_info_get(struct rte_eth_dev *dev, @@ -59 +60 @@ - const uint32_t *mlx5_dev_supported_ptypes_get(struct rte_eth_dev *dev, + const uint32_t *mlx5_dev_supported_ptypes_get(struct rte_eth_dev *dev); @@ -61 +62 @@ -index a12891a983..9711746edb 100644 +index 7e0ec91328..cca7b86649 100644 @@ -64 +65 @@ -@@ -1593,7 +1593,7 @@ mlx5_txq_devx_obj_new(struct rte_eth_dev *dev, uint16_t idx) +@@ -1513,7 +1513,7 @@ mlx5_txq_devx_obj_new(struct rte_eth_dev *dev, uint16_t idx) @@ -74 +75 @@ -index 7708a0b808..a50320075c 100644 +index 08c6b18975..169612ca44 100644 @@ -77 +78 @@ -@@ -314,6 +314,37 @@ mlx5_set_txlimit_params(struct rte_eth_dev *dev, struct rte_eth_dev_info *info) +@@ -306,6 +306,37 @@ mlx5_set_txlimit_params(struct rte_eth_dev *dev, struct rte_eth_dev_info *info) @@ -115 +116 @@ -@@ -327,6 +358,7 @@ mlx5_dev_infos_get(struct rte_eth_dev *dev, struct rte_eth_dev_info *info) +@@ -319,6 +350,7 @@ mlx5_dev_infos_get(struct rte_eth_dev *dev, struct rte_eth_dev_info *info) @@ -123 +124 @@ -@@ -359,10 +391,9 @@ mlx5_dev_infos_get(struct rte_eth_dev *dev, struct rte_eth_dev_info *info) +@@ -351,10 +383,9 @@ mlx5_dev_infos_get(struct rte_eth_dev *dev, struct rte_eth_dev_info *info) @@ -138 +139 @@ -index ab29b43875..b676e5394b 100644 +index fcf6ab54b6..dfbb3b9a16 100644 @@ -141 +142 @@ -@@ -656,7 +656,7 @@ mlx5_rx_queue_pre_setup(struct rte_eth_dev *dev, uint16_t idx, uint16_t *desc, +@@ -652,7 +652,7 @@ mlx5_rx_queue_pre_setup(struct rte_eth_dev *dev, uint16_t idx, uint16_t *desc, @@ -151 +152 @@ -index 4ee44e9165..8145ad4233 100644 +index 786e638f95..b2e860d149 100644 @@ -154 +155 @@ -@@ -217,8 +217,8 @@ mlx5_rxq_start(struct rte_eth_dev *dev) +@@ -215,8 +215,8 @@ mlx5_rxq_start(struct rte_eth_dev *dev) @@ -166 +167 @@ -index ddd3a66282..5fee5bc4e8 100644 +index cdc9755fe0..9869cd4577 100644 @@ -169 +170 @@ -@@ -334,7 +334,7 @@ mlx5_tx_queue_pre_setup(struct rte_eth_dev *dev, uint16_t idx, uint16_t *desc) +@@ -332,7 +332,7 @@ mlx5_tx_queue_pre_setup(struct rte_eth_dev *dev, uint16_t idx, uint16_t *desc) @@ -178 +179 @@ -@@ -728,7 +728,7 @@ txq_calc_inline_max(struct mlx5_txq_ctrl *txq_ctrl) +@@ -726,7 +726,7 @@ txq_calc_inline_max(struct mlx5_txq_ctrl *txq_ctrl) @@ -187 +188 @@ -@@ -1054,6 +1054,7 @@ mlx5_txq_new(struct rte_eth_dev *dev, uint16_t idx, uint16_t desc, +@@ -1081,6 +1081,7 @@ mlx5_txq_new(struct rte_eth_dev *dev, uint16_t idx, uint16_t desc, @@ -195,2 +196 @@ -@@ -1078,13 +1079,12 @@ mlx5_txq_new(struct rte_eth_dev *dev, uint16_t idx, uint16_t desc, - tmpl->txq.idx = idx; +@@ -1106,13 +1107,12 @@ mlx5_txq_new(struct rte_eth_dev *dev, uint16_t idx, uint16_t desc, @@ -198 +198,2 @@ - txq_adjust_params(tmpl); + if (txq_adjust_params(tmpl)) + goto error;