From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 76E88A0553 for ; Mon, 17 Feb 2020 18:46:30 +0100 (CET) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 70C561DA12; Mon, 17 Feb 2020 18:46:30 +0100 (CET) Received: from mail-wr1-f52.google.com (mail-wr1-f52.google.com [209.85.221.52]) by dpdk.org (Postfix) with ESMTP id 433E91DA12 for ; Mon, 17 Feb 2020 18:46:29 +0100 (CET) Received: by mail-wr1-f52.google.com with SMTP id z7so20757685wrl.13 for ; Mon, 17 Feb 2020 09:46:29 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=nkNt9wZe1HYF4Mamvnn4lZazB24MntkiYMXMMryLGxc=; b=X97BafOsB4//WaO1tpD7vCbEPefBMoBIhKNVTsupKnIt4DanFs6+i54hd+h4bqZ/rK AnWMglS5yTyLyCNkHZTNXJoIcz+x+VJ41OAHRHtMDodBDlUk/zXDzDoLr5ztu98KtZwK XAFeMQjtxD83JD9FzpCBpIdwZ1wLjwsEO3FJCxAFOj3LnOqEZciLwVHLieogN4UzJZN/ +qHiySpylJV+zWMpd02IgoyfDCUtk/xBKJtDkiAmBOpV3T6u5TsmpqmCmFI0KIgN3La9 ZOQMxft2VrEbPiuHMYICNyC6yYg7wMmbFr4/cCJmc+FLAOGlh22KKrk42p//qlrucCQB SO8A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=nkNt9wZe1HYF4Mamvnn4lZazB24MntkiYMXMMryLGxc=; b=QoV1VL6B00ln/t6CsdAw6uTNfLe5b+AK+Zr+heqFU7e5LcSnQ8nB59CRsoJtsKYWdz eChNWMtn5i7ngAAERoxxfpbceWEyjjhqylIgnYCByj1/NUtaRRF1EkmIVHNf++WPC7MN Gvjdx9okq02tZdpr6+SZcsiJd2bxa5fHdtn/siT6vpKqZeh2DKM93BWVaA+4iLmHkSai n6dK0R4mF+kfofeRESmSuze2tSoA8gV7jigbeigOoISorx3ytaioBgYSmEijy7emnmz9 LB5W/qYgFn+4zE/7ceBRLRApiWsT92mrqlahwdlFvRN8iKjVXAv7e4mfnkuRrb1IS0XU fwfA== X-Gm-Message-State: APjAAAXWnSnCIANed/MsLWlfMH1W2XwwfrM/3pkrukoKxDGs8GzLtLtl qQQyMfUFfphwBuhheOb360IULISFT+mKKg== X-Google-Smtp-Source: APXvYqw2jRHx0Di5yrdN1stlTG8K3L30G5w4L/cspBe9lomQ771JXwfasAq6LVVtNH7yNMuuxDpBMg== X-Received: by 2002:a5d:6789:: with SMTP id v9mr24576637wru.55.1581961588881; Mon, 17 Feb 2020 09:46:28 -0800 (PST) Received: from localhost (userh634.uk.uudial.com. [194.69.103.7]) by smtp.gmail.com with ESMTPSA id w1sm2015419wro.72.2020.02.17.09.46.28 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 17 Feb 2020 09:46:28 -0800 (PST) From: luca.boccassi@gmail.com To: Viacheslav Ovsiienko Cc: Matan Azrad , dpdk stable Date: Mon, 17 Feb 2020 17:45:16 +0000 Message-Id: <20200217174546.25334-24-luca.boccassi@gmail.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20200217174546.25334-1-luca.boccassi@gmail.com> References: <20200211112216.3929-190-luca.boccassi@gmail.com> <20200217174546.25334-1-luca.boccassi@gmail.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Subject: [dpdk-stable] patch 'net/mlx5: fix legacy multi-packet write session' has been queued to stable release 19.11.1 X-BeenThere: stable@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: patches for DPDK stable branches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: stable-bounces@dpdk.org Sender: "stable" Hi, FYI, your patch has been queued to stable release 19.11.1 Note it hasn't been pushed to http://dpdk.org/browse/dpdk-stable yet. It will be pushed if I get no objections before 02/19/20. So please shout if anyone has objections. Also note that after the patch there's a diff of the upstream commit vs the patch applied to the branch. This will indicate if there was any rebasing needed to apply to the stable branch. If there were code changes for rebasing (ie: not only metadata diffs), please double check that the rebase was correctly done. Thanks. Luca Boccassi --- >From 2fa4a87b409237f05dfb9586c03e28dcf47f93d3 Mon Sep 17 00:00:00 2001 From: Viacheslav Ovsiienko Date: Sun, 9 Feb 2020 22:54:53 +0000 Subject: [PATCH] net/mlx5: fix legacy multi-packet write session [ upstream commit 7593cf1d3500dfce7c9faacfc01f5797556b6a80 ] To provide the better PCIe bandwidth utilization the ConnectX-4LX NIC supports the multi-packet write (MPW) sessions allowing to pack multiple packets into one descriptor (WQE). This is legacy feature and it has some limitations on the packets and data description segments. To provide the best performance all inline packets must be put into shared data segment and the total length of MPW session must be limited. The limit is controlled with txq_inline_mpw devarg. Fixes: 82e75f8323bf ("net/mlx5: fix legacy multi-packet Tx descriptors") Signed-off-by: Viacheslav Ovsiienko Acked-by: Matan Azrad --- drivers/net/mlx5/mlx5_prm.h | 2 +- drivers/net/mlx5/mlx5_rxtx.c | 91 +++++++++++++++++++++++++++++++----- 2 files changed, 81 insertions(+), 12 deletions(-) diff --git a/drivers/net/mlx5/mlx5_prm.h b/drivers/net/mlx5/mlx5_prm.h index a805363757..9f1d122df0 100644 --- a/drivers/net/mlx5/mlx5_prm.h +++ b/drivers/net/mlx5/mlx5_prm.h @@ -100,7 +100,7 @@ */ #define MLX5_EMPW_MAX_PACKETS MLX5_TX_COMP_THRESH #define MLX5_MPW_MAX_PACKETS 6 -#define MLX5_MPW_INLINE_MAX_PACKETS 2 +#define MLX5_MPW_INLINE_MAX_PACKETS 6 /* * Default packet length threshold to be inlined with diff --git a/drivers/net/mlx5/mlx5_rxtx.c b/drivers/net/mlx5/mlx5_rxtx.c index 67cafd1e2c..a69729f010 100644 --- a/drivers/net/mlx5/mlx5_rxtx.c +++ b/drivers/net/mlx5/mlx5_rxtx.c @@ -2821,8 +2821,14 @@ mlx5_tx_dseg_empw(struct mlx5_txq_data *restrict txq, unsigned int part; uint8_t *pdst; - dseg->bcount = rte_cpu_to_be_32(len | MLX5_ETH_WQE_DATA_INLINE); - pdst = &dseg->inline_data[0]; + if (!MLX5_TXOFF_CONFIG(MPW)) { + /* Store the descriptor byte counter for eMPW sessions. */ + dseg->bcount = rte_cpu_to_be_32(len | MLX5_ETH_WQE_DATA_INLINE); + pdst = &dseg->inline_data[0]; + } else { + /* The entire legacy MPW session counter is stored on close. */ + pdst = (uint8_t *)dseg; + } /* * The WQEBB space availability is checked by caller. * Here we should be aware of WQE ring buffer wraparound only. @@ -2834,7 +2840,8 @@ mlx5_tx_dseg_empw(struct mlx5_txq_data *restrict txq, len -= part; if (likely(!len)) { pdst += part; - pdst = RTE_PTR_ALIGN(pdst, MLX5_WSEG_SIZE); + if (!MLX5_TXOFF_CONFIG(MPW)) + pdst = RTE_PTR_ALIGN(pdst, MLX5_WSEG_SIZE); /* Note: no final wraparound check here. */ return (struct mlx5_wqe_dseg *)pdst; } @@ -2882,9 +2889,16 @@ mlx5_tx_dseg_vlan(struct mlx5_txq_data *restrict txq, static_assert(MLX5_DSEG_MIN_INLINE_SIZE == (2 * RTE_ETHER_ADDR_LEN), "invalid Data Segment data size"); - dseg->bcount = rte_cpu_to_be_32((len + sizeof(struct rte_vlan_hdr)) | - MLX5_ETH_WQE_DATA_INLINE); - pdst = &dseg->inline_data[0]; + if (!MLX5_TXOFF_CONFIG(MPW)) { + /* Store the descriptor byte counter for eMPW sessions. */ + dseg->bcount = rte_cpu_to_be_32 + ((len + sizeof(struct rte_vlan_hdr)) | + MLX5_ETH_WQE_DATA_INLINE); + pdst = &dseg->inline_data[0]; + } else { + /* The entire legacy MPW session counter is stored on close. */ + pdst = (uint8_t *)dseg; + } memcpy(pdst, buf, MLX5_DSEG_MIN_INLINE_SIZE); buf += MLX5_DSEG_MIN_INLINE_SIZE; pdst += MLX5_DSEG_MIN_INLINE_SIZE; @@ -2907,7 +2921,8 @@ mlx5_tx_dseg_vlan(struct mlx5_txq_data *restrict txq, len -= part; if (likely(!len)) { pdst += part; - pdst = RTE_PTR_ALIGN(pdst, MLX5_WSEG_SIZE); + if (!MLX5_TXOFF_CONFIG(MPW)) + pdst = RTE_PTR_ALIGN(pdst, MLX5_WSEG_SIZE); /* Note: no final wraparound check here. */ return (struct mlx5_wqe_dseg *)pdst; } @@ -3790,15 +3805,33 @@ mlx5_tx_idone_empw(struct mlx5_txq_data *restrict txq, unsigned int slen, unsigned int olx __rte_unused) { + struct mlx5_wqe_dseg *dseg = &loc->wqe_last->dseg[0]; + assert(MLX5_TXOFF_CONFIG(INLINE)); - assert((len % MLX5_WSEG_SIZE) == 0); #ifdef MLX5_PMD_SOFT_COUNTERS /* Update sent data bytes counter. */ txq->stats.obytes += slen; #else (void)slen; #endif - len = len / MLX5_WSEG_SIZE + 2; + if (MLX5_TXOFF_CONFIG(MPW) && dseg->bcount == RTE_BE32(0)) { + /* + * If the legacy MPW session contains the inline packets + * we should set the only inline data segment length + * and align the total length to the segment size. + */ + assert(len > sizeof(dseg->bcount)); + dseg->bcount = rte_cpu_to_be_32((len - sizeof(dseg->bcount)) | + MLX5_ETH_WQE_DATA_INLINE); + len = (len + MLX5_WSEG_SIZE - 1) / MLX5_WSEG_SIZE + 2; + } else { + /* + * The session is not legacy MPW or contains the + * data buffer pointer segments. + */ + assert((len % MLX5_WSEG_SIZE) == 0); + len = len / MLX5_WSEG_SIZE + 2; + } loc->wqe_last->cseg.sq_ds = rte_cpu_to_be_32(txq->qp_num_8s | len); txq->wqe_ci += (len + 3) / 4; loc->wqe_free -= (len + 3) / 4; @@ -4077,6 +4110,15 @@ mlx5_tx_burst_empw_inline(struct mlx5_txq_data *restrict txq, loc->wqe_free) * MLX5_WQE_SIZE - MLX5_WQE_CSEG_SIZE - MLX5_WQE_ESEG_SIZE; + /* Limit the room for legacy MPW sessions for performance. */ + if (MLX5_TXOFF_CONFIG(MPW)) + room = RTE_MIN(room, + RTE_MAX(txq->inlen_empw + + sizeof(dseg->bcount) + + (MLX5_TXOFF_CONFIG(VLAN) ? + sizeof(struct rte_vlan_hdr) : 0), + MLX5_MPW_INLINE_MAX_PACKETS * + MLX5_WQE_DSEG_SIZE)); /* Build WQE till we have space, packets and resources. */ part = room; for (;;) { @@ -4106,8 +4148,26 @@ mlx5_tx_burst_empw_inline(struct mlx5_txq_data *restrict txq, /* Inline or not inline - that's the Question. */ if (dlen > txq->inlen_empw) goto pointer_empw; + if (MLX5_TXOFF_CONFIG(MPW)) { + tlen = dlen; + if (part == room) { + /* Open new inline MPW session. */ + tlen += sizeof(dseg->bcount); + dseg->bcount = RTE_BE32(0); + dseg = RTE_PTR_ADD + (dseg, sizeof(dseg->bcount)); + } else { + /* + * No pointer and inline descriptor + * intermix for legacy MPW sessions. + */ + if (loc->wqe_last->dseg[0].bcount) + break; + } + } else { + tlen = sizeof(dseg->bcount) + dlen; + } /* Inline entire packet, optional VLAN insertion. */ - tlen = sizeof(dseg->bcount) + dlen; if (MLX5_TXOFF_CONFIG(VLAN) && loc->mbuf->ol_flags & PKT_TX_VLAN_PKT) { /* @@ -4132,7 +4192,8 @@ mlx5_tx_burst_empw_inline(struct mlx5_txq_data *restrict txq, dseg = mlx5_tx_dseg_empw(txq, loc, dseg, dptr, dlen, olx); } - tlen = RTE_ALIGN(tlen, MLX5_WSEG_SIZE); + if (!MLX5_TXOFF_CONFIG(MPW)) + tlen = RTE_ALIGN(tlen, MLX5_WSEG_SIZE); assert(room >= tlen); room -= tlen; /* @@ -4142,6 +4203,14 @@ mlx5_tx_burst_empw_inline(struct mlx5_txq_data *restrict txq, rte_pktmbuf_free_seg(loc->mbuf); goto next_mbuf; pointer_empw: + /* + * No pointer and inline descriptor + * intermix for legacy MPW sessions. + */ + if (MLX5_TXOFF_CONFIG(MPW) && + part != room && + loc->wqe_last->dseg[0].bcount == RTE_BE32(0)) + break; /* * Not inlinable VLAN packets are * proceeded outside of this routine. -- 2.20.1 --- Diff of the applied patch vs upstream commit (please double-check if non-empty: --- --- - 2020-02-17 17:00:15.988041003 +0000 +++ 0024-net-mlx5-fix-legacy-multi-packet-write-session.patch 2020-02-17 17:00:15.315950569 +0000 @@ -1,8 +1,10 @@ -From 7593cf1d3500dfce7c9faacfc01f5797556b6a80 Mon Sep 17 00:00:00 2001 +From 2fa4a87b409237f05dfb9586c03e28dcf47f93d3 Mon Sep 17 00:00:00 2001 From: Viacheslav Ovsiienko Date: Sun, 9 Feb 2020 22:54:53 +0000 Subject: [PATCH] net/mlx5: fix legacy multi-packet write session +[ upstream commit 7593cf1d3500dfce7c9faacfc01f5797556b6a80 ] + To provide the better PCIe bandwidth utilization the ConnectX-4LX NIC supports the multi-packet write (MPW) sessions allowing to pack multiple packets into one descriptor (WQE). This is legacy @@ -13,19 +15,18 @@ txq_inline_mpw devarg. Fixes: 82e75f8323bf ("net/mlx5: fix legacy multi-packet Tx descriptors") -Cc: stable@dpdk.org Signed-off-by: Viacheslav Ovsiienko Acked-by: Matan Azrad --- - drivers/common/mlx5/mlx5_prm.h | 2 +- - drivers/net/mlx5/mlx5_rxtx.c | 91 ++++++++++++++++++++++++++++++---- + drivers/net/mlx5/mlx5_prm.h | 2 +- + drivers/net/mlx5/mlx5_rxtx.c | 91 +++++++++++++++++++++++++++++++----- 2 files changed, 81 insertions(+), 12 deletions(-) -diff --git a/drivers/common/mlx5/mlx5_prm.h b/drivers/common/mlx5/mlx5_prm.h -index 855b37aff1..4ac3d4bd50 100644 ---- a/drivers/common/mlx5/mlx5_prm.h -+++ b/drivers/common/mlx5/mlx5_prm.h +diff --git a/drivers/net/mlx5/mlx5_prm.h b/drivers/net/mlx5/mlx5_prm.h +index a805363757..9f1d122df0 100644 +--- a/drivers/net/mlx5/mlx5_prm.h ++++ b/drivers/net/mlx5/mlx5_prm.h @@ -100,7 +100,7 @@ */ #define MLX5_EMPW_MAX_PACKETS MLX5_TX_COMP_THRESH @@ -36,10 +37,10 @@ /* * Default packet length threshold to be inlined with diff --git a/drivers/net/mlx5/mlx5_rxtx.c b/drivers/net/mlx5/mlx5_rxtx.c -index 5eea932d44..b55db4fa74 100644 +index 67cafd1e2c..a69729f010 100644 --- a/drivers/net/mlx5/mlx5_rxtx.c +++ b/drivers/net/mlx5/mlx5_rxtx.c -@@ -2949,8 +2949,14 @@ mlx5_tx_dseg_empw(struct mlx5_txq_data *restrict txq, +@@ -2821,8 +2821,14 @@ mlx5_tx_dseg_empw(struct mlx5_txq_data *restrict txq, unsigned int part; uint8_t *pdst; @@ -56,7 +57,7 @@ /* * The WQEBB space availability is checked by caller. * Here we should be aware of WQE ring buffer wraparound only. -@@ -2962,7 +2968,8 @@ mlx5_tx_dseg_empw(struct mlx5_txq_data *restrict txq, +@@ -2834,7 +2840,8 @@ mlx5_tx_dseg_empw(struct mlx5_txq_data *restrict txq, len -= part; if (likely(!len)) { pdst += part; @@ -66,7 +67,7 @@ /* Note: no final wraparound check here. */ return (struct mlx5_wqe_dseg *)pdst; } -@@ -3010,9 +3017,16 @@ mlx5_tx_dseg_vlan(struct mlx5_txq_data *restrict txq, +@@ -2882,9 +2889,16 @@ mlx5_tx_dseg_vlan(struct mlx5_txq_data *restrict txq, static_assert(MLX5_DSEG_MIN_INLINE_SIZE == (2 * RTE_ETHER_ADDR_LEN), "invalid Data Segment data size"); @@ -86,7 +87,7 @@ memcpy(pdst, buf, MLX5_DSEG_MIN_INLINE_SIZE); buf += MLX5_DSEG_MIN_INLINE_SIZE; pdst += MLX5_DSEG_MIN_INLINE_SIZE; -@@ -3035,7 +3049,8 @@ mlx5_tx_dseg_vlan(struct mlx5_txq_data *restrict txq, +@@ -2907,7 +2921,8 @@ mlx5_tx_dseg_vlan(struct mlx5_txq_data *restrict txq, len -= part; if (likely(!len)) { pdst += part; @@ -96,14 +97,14 @@ /* Note: no final wraparound check here. */ return (struct mlx5_wqe_dseg *)pdst; } -@@ -3921,15 +3936,33 @@ mlx5_tx_idone_empw(struct mlx5_txq_data *restrict txq, +@@ -3790,15 +3805,33 @@ mlx5_tx_idone_empw(struct mlx5_txq_data *restrict txq, unsigned int slen, unsigned int olx __rte_unused) { + struct mlx5_wqe_dseg *dseg = &loc->wqe_last->dseg[0]; + - MLX5_ASSERT(MLX5_TXOFF_CONFIG(INLINE)); -- MLX5_ASSERT((len % MLX5_WSEG_SIZE) == 0); + assert(MLX5_TXOFF_CONFIG(INLINE)); +- assert((len % MLX5_WSEG_SIZE) == 0); #ifdef MLX5_PMD_SOFT_COUNTERS /* Update sent data bytes counter. */ txq->stats.obytes += slen; @@ -117,7 +118,7 @@ + * we should set the only inline data segment length + * and align the total length to the segment size. + */ -+ MLX5_ASSERT(len > sizeof(dseg->bcount)); ++ assert(len > sizeof(dseg->bcount)); + dseg->bcount = rte_cpu_to_be_32((len - sizeof(dseg->bcount)) | + MLX5_ETH_WQE_DATA_INLINE); + len = (len + MLX5_WSEG_SIZE - 1) / MLX5_WSEG_SIZE + 2; @@ -126,13 +127,13 @@ + * The session is not legacy MPW or contains the + * data buffer pointer segments. + */ -+ MLX5_ASSERT((len % MLX5_WSEG_SIZE) == 0); ++ assert((len % MLX5_WSEG_SIZE) == 0); + len = len / MLX5_WSEG_SIZE + 2; + } loc->wqe_last->cseg.sq_ds = rte_cpu_to_be_32(txq->qp_num_8s | len); txq->wqe_ci += (len + 3) / 4; loc->wqe_free -= (len + 3) / 4; -@@ -4208,6 +4241,15 @@ mlx5_tx_burst_empw_inline(struct mlx5_txq_data *restrict txq, +@@ -4077,6 +4110,15 @@ mlx5_tx_burst_empw_inline(struct mlx5_txq_data *restrict txq, loc->wqe_free) * MLX5_WQE_SIZE - MLX5_WQE_CSEG_SIZE - MLX5_WQE_ESEG_SIZE; @@ -148,9 +149,9 @@ /* Build WQE till we have space, packets and resources. */ part = room; for (;;) { -@@ -4238,8 +4280,26 @@ mlx5_tx_burst_empw_inline(struct mlx5_txq_data *restrict txq, - if (dlen > txq->inlen_empw || - loc->mbuf->ol_flags & PKT_TX_DYNF_NOINLINE) +@@ -4106,8 +4148,26 @@ mlx5_tx_burst_empw_inline(struct mlx5_txq_data *restrict txq, + /* Inline or not inline - that's the Question. */ + if (dlen > txq->inlen_empw) goto pointer_empw; + if (MLX5_TXOFF_CONFIG(MPW)) { + tlen = dlen; @@ -176,17 +177,17 @@ if (MLX5_TXOFF_CONFIG(VLAN) && loc->mbuf->ol_flags & PKT_TX_VLAN_PKT) { /* -@@ -4265,7 +4325,8 @@ mlx5_tx_burst_empw_inline(struct mlx5_txq_data *restrict txq, +@@ -4132,7 +4192,8 @@ mlx5_tx_burst_empw_inline(struct mlx5_txq_data *restrict txq, dseg = mlx5_tx_dseg_empw(txq, loc, dseg, dptr, dlen, olx); } - tlen = RTE_ALIGN(tlen, MLX5_WSEG_SIZE); + if (!MLX5_TXOFF_CONFIG(MPW)) + tlen = RTE_ALIGN(tlen, MLX5_WSEG_SIZE); - MLX5_ASSERT(room >= tlen); + assert(room >= tlen); room -= tlen; /* -@@ -4275,6 +4336,14 @@ mlx5_tx_burst_empw_inline(struct mlx5_txq_data *restrict txq, +@@ -4142,6 +4203,14 @@ mlx5_tx_burst_empw_inline(struct mlx5_txq_data *restrict txq, rte_pktmbuf_free_seg(loc->mbuf); goto next_mbuf; pointer_empw: