[dpdk-stable] [PATCH 18.11] net/mlx5: fix instruction hotspot on replenishing Rx buffer

[dpdk-stable] [PATCH 18.11] net/mlx5: fix instruction hotspot on replenishing Rx buffer

[Yongseok Koh]

[ backported from upstream commit 12d468a62bc19ca08ee9964dcb923f67f87fba7d ]

On replenishing Rx buffers for vectorized Rx, mbuf->buf_addr isn't needed
to be accessed as it is static and easily calculated from the mbuf address.
Accessing the mbuf content causes unnecessary load stall and it is worsened
on ARM.

Fixes: 545b884b1da3 ("net/mlx5: fix buffer address posting in SSE Rx")

Signed-off-by: Yongseok Koh <yskoh@mellanox.com>
Acked-by: Shahaf Shuler <shahafs@mellanox.com>
---
 drivers/net/mlx5/mlx5_rxtx_vec.h | 8 ++++++--
 1 file changed, 6 insertions(+), 2 deletions(-)

diff --git a/drivers/net/mlx5/mlx5_rxtx_vec.h b/drivers/net/mlx5/mlx5_rxtx_vec.h
index fda7004e2d..da447a63e2 100644
--- a/drivers/net/mlx5/mlx5_rxtx_vec.h
+++ b/drivers/net/mlx5/mlx5_rxtx_vec.h
@@ -102,8 +102,12 @@ mlx5_rx_replenish_bulk_mbuf(struct mlx5_rxq_data *rxq, uint16_t n)
 		return;
 	}
 	for (i = 0; i < n; ++i) {
-		wq[i].addr = rte_cpu_to_be_64((uintptr_t)elts[i]->buf_addr +
-					      RTE_PKTMBUF_HEADROOM);
+		uintptr_t buf_addr =
+			(uintptr_t)elts[i] + sizeof(struct rte_mbuf) +
+			rte_pktmbuf_priv_size(rxq->mp);
+
+		assert(buf_addr == (uintptr_t)elts[i]->buf_addr);
+		wq[i].addr = rte_cpu_to_be_64(buf_addr + RTE_PKTMBUF_HEADROOM);
 		/* If there's only one MR, no need to replace LKey in WQE. */
 		if (unlikely(mlx5_mr_btree_len(&rxq->mr_ctrl.cache_bh) > 1))
 			wq[i].lkey = mlx5_rx_mb2mr(rxq, elts[i]);
-- 
2.11.0

Re: [dpdk-stable] [PATCH 18.11] net/mlx5: fix instruction hotspot on replenishing Rx buffer

[Kevin Traynor]

On 30/04/2019 21:22, Yongseok Koh wrote:
> On replenishing Rx buffers for vectorized Rx, mbuf->buf_addr isn't needed
> to be accessed as it is static and easily calculated from the mbuf address.
> Accessing the mbuf content causes unnecessary load stall. non-x86
> processors (mostly RISC such as ARM and Power) are more vulnerable to load
> stall. For x86, reducing the number of instructions seems to matter most.
> 
> Fixes: 545b884b1da3 ("net/mlx5: fix buffer address posting in SSE Rx")
> 
> Signed-off-by: Yongseok Koh <yskoh@mellanox.com>
> Acked-by: Shahaf Shuler <shahafs@mellanox.com>

Thanks. Added a reference to the dpdk master commits in the commit msg
and queued for 18.11 stable branch. Will push to dpdk-stable next week
as part of next batch.

[dpdk-stable] [PATCH 18.11] net/mlx5: fix instruction hotspot on replenishing Rx buffer

[Yongseok Koh]

On replenishing Rx buffers for vectorized Rx, mbuf->buf_addr isn't needed
to be accessed as it is static and easily calculated from the mbuf address.
Accessing the mbuf content causes unnecessary load stall. non-x86
processors (mostly RISC such as ARM and Power) are more vulnerable to load
stall. For x86, reducing the number of instructions seems to matter most.

Fixes: 545b884b1da3 ("net/mlx5: fix buffer address posting in SSE Rx")

Signed-off-by: Yongseok Koh <yskoh@mellanox.com>
Acked-by: Shahaf Shuler <shahafs@mellanox.com>
---
 drivers/net/mlx5/mlx5_rxtx_vec.h | 17 ++++++++++++++++-
 1 file changed, 16 insertions(+), 1 deletion(-)

diff --git a/drivers/net/mlx5/mlx5_rxtx_vec.h b/drivers/net/mlx5/mlx5_rxtx_vec.h
index fda7004e2..86735044d 100644
--- a/drivers/net/mlx5/mlx5_rxtx_vec.h
+++ b/drivers/net/mlx5/mlx5_rxtx_vec.h
@@ -102,7 +102,22 @@ mlx5_rx_replenish_bulk_mbuf(struct mlx5_rxq_data *rxq, uint16_t n)
 		return;
 	}
 	for (i = 0; i < n; ++i) {
-		wq[i].addr = rte_cpu_to_be_64((uintptr_t)elts[i]->buf_addr +
+		void *buf_addr;
+
+		/*
+		 * Load the virtual address for Rx WQE. non-x86 processors
+		 * (mostly RISC such as ARM and Power) are more vulnerable to
+		 * load stall. For x86, reducing the number of instructions
+		 * seems to matter most.
+		 */
+#ifdef RTE_ARCH_X86_64
+		buf_addr = elts[i]->buf_addr;
+#else
+		buf_addr = (char *)elts[i] + sizeof(struct rte_mbuf) +
+			   rte_pktmbuf_priv_size(rxq->mp);
+		assert(buf_addr == elts[i]->buf_addr);
+#endif
+		wq[i].addr = rte_cpu_to_be_64((uintptr_t)buf_addr +
 					      RTE_PKTMBUF_HEADROOM);
 		/* If there's only one MR, no need to replace LKey in WQE. */
 		if (unlikely(mlx5_mr_btree_len(&rxq->mr_ctrl.cache_bh) > 1))
-- 
2.11.0