[dpdk-dev] [PATCH] net/mlx5: fix wrong completion buffer size

[dpdk-dev] [PATCH] net/mlx5: fix wrong completion buffer size

[Shahaf Shuler]

Completion buffer size was computed wrongly, causing
completion polling to wraparound too early and miss entries.

Fixing it by using Direct Verbs to query the CQ info.

Fixes: 6218063b39a6 ("net/mlx5: refactor Rx data path")
Fixes: 1d88ba171942 ("net/mlx5: refactor Tx data path")
Cc: stable@dpdk.org

Signed-off-by: Shahaf Shuler <shahafs@mellanox.com>
Acked-by: Yongseok Koh <yskoh@mellanox.com>
---
 drivers/net/mlx5/mlx5_rxq.c | 14 +++++++++-----
 drivers/net/mlx5/mlx5_txq.c | 14 +++++++++-----
 2 files changed, 18 insertions(+), 10 deletions(-)

diff --git a/drivers/net/mlx5/mlx5_rxq.c b/drivers/net/mlx5/mlx5_rxq.c
index 8b7823360..2a2683985 100644
--- a/drivers/net/mlx5/mlx5_rxq.c
+++ b/drivers/net/mlx5/mlx5_rxq.c
@@ -838,12 +838,16 @@ static inline int
 rxq_setup(struct rxq_ctrl *tmpl)
 {
 	struct ibv_cq *ibcq = tmpl->cq;
-	struct mlx5_cq *cq = to_mxxx(cq, cq);
+	struct ibv_mlx5_cq_info cq_info;
 	struct mlx5_rwq *rwq = container_of(tmpl->wq, struct mlx5_rwq, wq);
 	struct rte_mbuf *(*elts)[1 << tmpl->rxq.elts_n] =
 		rte_calloc_socket("RXQ", 1, sizeof(*elts), 0, tmpl->socket);
 
-	if (cq->cqe_sz != RTE_CACHE_LINE_SIZE) {
+	if (ibv_mlx5_exp_get_cq_info(ibcq, &cq_info)) {
+		ERROR("Unable to query CQ info. check your OFED.");
+		return ENOTSUP;
+	}
+	if (cq_info.cqe_size != RTE_CACHE_LINE_SIZE) {
 		ERROR("Wrong MLX5_CQE_SIZE environment variable value: "
 		      "it should be set to %u", RTE_CACHE_LINE_SIZE);
 		return EINVAL;
@@ -851,16 +855,16 @@ rxq_setup(struct rxq_ctrl *tmpl)
 	if (elts == NULL)
 		return ENOMEM;
 	tmpl->rxq.rq_db = rwq->rq.db;
-	tmpl->rxq.cqe_n = log2above(ibcq->cqe);
+	tmpl->rxq.cqe_n = log2above(cq_info.cqe_cnt);
 	tmpl->rxq.cq_ci = 0;
 	tmpl->rxq.rq_ci = 0;
-	tmpl->rxq.cq_db = cq->dbrec;
+	tmpl->rxq.cq_db = cq_info.dbrec;
 	tmpl->rxq.wqes =
 		(volatile struct mlx5_wqe_data_seg (*)[])
 		(uintptr_t)rwq->rq.buff;
 	tmpl->rxq.cqes =
 		(volatile struct mlx5_cqe (*)[])
-		(uintptr_t)cq->active_buf->buf;
+		(uintptr_t)cq_info.buf;
 	tmpl->rxq.elts = elts;
 	return 0;
 }
diff --git a/drivers/net/mlx5/mlx5_txq.c b/drivers/net/mlx5/mlx5_txq.c
index de7e28be6..6c1387e56 100644
--- a/drivers/net/mlx5/mlx5_txq.c
+++ b/drivers/net/mlx5/mlx5_txq.c
@@ -173,23 +173,27 @@ txq_setup(struct txq_ctrl *tmpl, struct txq_ctrl *txq_ctrl)
 {
 	struct mlx5_qp *qp = to_mqp(tmpl->qp);
 	struct ibv_cq *ibcq = tmpl->cq;
-	struct mlx5_cq *cq = to_mxxx(cq, cq);
+	struct ibv_mlx5_cq_info cq_info;
 
-	if (cq->cqe_sz != RTE_CACHE_LINE_SIZE) {
+	if (ibv_mlx5_exp_get_cq_info(ibcq, &cq_info)) {
+		ERROR("Unable to query CQ info. check your OFED.");
+		return ENOTSUP;
+	}
+	if (cq_info.cqe_size != RTE_CACHE_LINE_SIZE) {
 		ERROR("Wrong MLX5_CQE_SIZE environment variable value: "
 		      "it should be set to %u", RTE_CACHE_LINE_SIZE);
 		return EINVAL;
 	}
-	tmpl->txq.cqe_n = log2above(ibcq->cqe);
+	tmpl->txq.cqe_n = log2above(cq_info.cqe_cnt);
 	tmpl->txq.qp_num_8s = qp->ctrl_seg.qp_num << 8;
 	tmpl->txq.wqes = qp->gen_data.sqstart;
 	tmpl->txq.wqe_n = log2above(qp->sq.wqe_cnt);
 	tmpl->txq.qp_db = &qp->gen_data.db[MLX5_SND_DBR];
 	tmpl->txq.bf_reg = qp->gen_data.bf->reg;
-	tmpl->txq.cq_db = cq->dbrec;
+	tmpl->txq.cq_db = cq_info.dbrec;
 	tmpl->txq.cqes =
 		(volatile struct mlx5_cqe (*)[])
-		(uintptr_t)cq->active_buf->buf;
+		(uintptr_t)cq_info.buf;
 	tmpl->txq.elts =
 		(struct rte_mbuf *(*)[1 << tmpl->txq.elts_n])
 		((uintptr_t)txq_ctrl + sizeof(*txq_ctrl));
-- 
2.12.0

Re: [dpdk-dev] [PATCH] net/mlx5: fix wrong completion buffer size

[Nélio Laranjeiro]

On Thu, May 18, 2017 at 01:44:45PM +0300, Shahaf Shuler wrote:
> Completion buffer size was computed wrongly, causing
> completion polling to wraparound too early and miss entries.
> 
> Fixing it by using Direct Verbs to query the CQ info.
> 
> Fixes: 6218063b39a6 ("net/mlx5: refactor Rx data path")
> Fixes: 1d88ba171942 ("net/mlx5: refactor Tx data path")
> Cc: stable@dpdk.org
> 
> Signed-off-by: Shahaf Shuler <shahafs@mellanox.com>
> Acked-by: Yongseok Koh <yskoh@mellanox.com>

Acked-by: Nelio Laranjeiro <nelio.laranjeiro@6wind.com>

-- 
Nélio Laranjeiro
6WIND

Re: [dpdk-dev] [dpdk-stable] [PATCH] net/mlx5: fix wrong completion buffer size

[Ferruh Yigit]

On 5/23/2017 1:09 PM, Nélio Laranjeiro wrote:
> On Thu, May 18, 2017 at 01:44:45PM +0300, Shahaf Shuler wrote:
>> Completion buffer size was computed wrongly, causing
>> completion polling to wraparound too early and miss entries.
>>
>> Fixing it by using Direct Verbs to query the CQ info.
>>
>> Fixes: 6218063b39a6 ("net/mlx5: refactor Rx data path")
>> Fixes: 1d88ba171942 ("net/mlx5: refactor Tx data path")
>> Cc: stable@dpdk.org
>>
>> Signed-off-by: Shahaf Shuler <shahafs@mellanox.com>
>> Acked-by: Yongseok Koh <yskoh@mellanox.com>
> 
> Acked-by: Nelio Laranjeiro <nelio.laranjeiro@6wind.com>

Series applied to dpdk-next-net/master, thanks.