[PATCH] common/mlx5: fix shared MR ranges allocation

[PATCH] common/mlx5: fix shared MR ranges allocation

[Bing Zhao]

Memory regions (MRs) were allocated in one chunk of memory with a
mempool registration object. However, MRs can be reused among
different mempool registrations.

When the registration that allocated the MRs originally was
destroyed, the dangling pointers to the MRs could be left in other
registrations sharing these MRs.

Splitting the memory allocation of registration structure and MRs in
this commit solves this pointer reference issue.

Fixes: 690b2a88c2f7 ("common/mlx5: add mempool registration facilities")

Signed-off-by: Bing Zhao <bingz@nvidia.com>
Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
Reviewed-by: Dmitry Kozlyuk <dkozlyuk@nvidia.com>
---
 drivers/common/mlx5/mlx5_common_mr.c | 13 +++++++++++--
 1 file changed, 11 insertions(+), 2 deletions(-)

diff --git a/drivers/common/mlx5/mlx5_common_mr.c b/drivers/common/mlx5/mlx5_common_mr.c
index 47121d8ff7..01f35ebcdf 100644
--- a/drivers/common/mlx5/mlx5_common_mr.c
+++ b/drivers/common/mlx5/mlx5_common_mr.c
@@ -1515,15 +1515,23 @@ mlx5_mempool_reg_create(struct rte_mempool *mp, unsigned int mrs_n,
 	struct mlx5_mempool_reg *mpr = NULL;
 
 	mpr = mlx5_malloc(MLX5_MEM_RTE | MLX5_MEM_ZERO,
-			  sizeof(*mpr) + mrs_n * sizeof(mpr->mrs[0]),
+			  sizeof(struct mlx5_mempool_reg),
 			  RTE_CACHE_LINE_SIZE, SOCKET_ID_ANY);
 	if (mpr == NULL) {
 		DRV_LOG(ERR, "Cannot allocate mempool %s registration object",
 			mp->name);
 		return NULL;
 	}
+	mpr->mrs = mlx5_malloc(MLX5_MEM_RTE | MLX5_MEM_ZERO,
+			       mrs_n * sizeof(struct mlx5_mempool_mr),
+			       RTE_CACHE_LINE_SIZE, SOCKET_ID_ANY);
+	if (!mpr->mrs) {
+		DRV_LOG(ERR, "Cannot allocate mempool %s registration MRs",
+			mp->name);
+		mlx5_free(mpr);
+		return NULL;
+	}
 	mpr->mp = mp;
-	mpr->mrs = (struct mlx5_mempool_mr *)(mpr + 1);
 	mpr->mrs_n = mrs_n;
 	mpr->is_extmem = is_extmem;
 	return mpr;
@@ -1544,6 +1552,7 @@ mlx5_mempool_reg_destroy(struct mlx5_mr_share_cache *share_cache,
 
 		for (i = 0; i < mpr->mrs_n; i++)
 			share_cache->dereg_mr_cb(&mpr->mrs[i].pmd_mr);
+		mlx5_free(mpr->mrs);
 	}
 	mlx5_free(mpr);
 }
-- 
2.27.0

RE: [PATCH] common/mlx5: fix shared MR ranges allocation

[Raslan Darawsheh]

Hi,

> -----Original Message-----
> From: Bing Zhao <bingz@nvidia.com>
> Sent: Tuesday, November 23, 2021 2:49 PM
> To: Slava Ovsiienko <viacheslavo@nvidia.com>; Matan Azrad
> <matan@nvidia.com>
> Cc: dev@dpdk.org; Raslan Darawsheh <rasland@nvidia.com>; Dmitry Kozlyuk
> <dkozlyuk@nvidia.com>
> Subject: [PATCH] common/mlx5: fix shared MR ranges allocation
> 
> Memory regions (MRs) were allocated in one chunk of memory with a mempool
> registration object. However, MRs can be reused among different mempool
> registrations.
> 
> When the registration that allocated the MRs originally was destroyed, the
> dangling pointers to the MRs could be left in other registrations sharing these
> MRs.
> 
> Splitting the memory allocation of registration structure and MRs in this commit
> solves this pointer reference issue.
> 
> Fixes: 690b2a88c2f7 ("common/mlx5: add mempool registration facilities")
> 
> Signed-off-by: Bing Zhao <bingz@nvidia.com>
> Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
> Reviewed-by: Dmitry Kozlyuk <dkozlyuk@nvidia.com>

Patch applied to next-net-mlx,

Kindest regards,
Raslan Darawsheh