patches for DPDK stable branches
 help / color / mirror / Atom feed
* [dpdk-stable] [PATCH] common/mlx5: free MR resource while device DMA unmap
@ 2020-10-28 13:14 Jiawei Wang
  2020-10-28 13:16 ` Slava Ovsiienko
  2020-11-02  2:22 ` [dpdk-stable] [PATCH v2] " Jiawei Wang
  0 siblings, 2 replies; 4+ messages in thread
From: Jiawei Wang @ 2020-10-28 13:14 UTC (permalink / raw)
  To: viacheslavo, matan, shahafs, orika; +Cc: dev, rasland, stable

mlx5 PMD created the MR (Memory Region) resource on the
mlx5_dma_map call to make the memory available for DMA
operations. On the mlx5_dma_unmap call the MR resource
was not freed but inserted to MR Free list for further
garbage collection.
Actual MR resource destroying happened on device stop
call. That caused the runtime out of memory in case of
application performed multiple DMA map/unmap calls.

The fix immediately frees the MR resource on mlx5_dma_unmap
call not engaging the list. The export for mlx5_mr_free
function from common PMD part is added as well.

Redmine: 2330226
Fixes: 989e999d9305 ("net/mlx5: support PCI device DMA map and unmap")
Cc: stable@dpdk.org

Signed-off-by: Jiawei Wang <jiaweiw@nvidia.com>
---
 drivers/common/mlx5/mlx5_common_mr.c | 12 ++++++------
 drivers/common/mlx5/mlx5_common_mr.h |  4 ++++
 drivers/common/mlx5/version.map      |  1 +
 drivers/net/mlx5/mlx5_mr.c           |  2 +-
 4 files changed, 12 insertions(+), 7 deletions(-)

diff --git a/drivers/common/mlx5/mlx5_common_mr.c b/drivers/common/mlx5/mlx5_common_mr.c
index 23324c0..7c25541 100644
--- a/drivers/common/mlx5/mlx5_common_mr.c
+++ b/drivers/common/mlx5/mlx5_common_mr.c
@@ -436,8 +436,8 @@ struct mlx5_mr *
  * @param mr
  *   Pointer to MR to free.
  */
-static void
-mr_free(struct mlx5_mr *mr, mlx5_dereg_mr_t dereg_mr_cb)
+void
+mlx5_mr_free(struct mlx5_mr *mr, mlx5_dereg_mr_t dereg_mr_cb)
 {
 	if (mr == NULL)
 		return;
@@ -492,7 +492,7 @@ struct mlx5_mr *
 		struct mlx5_mr *mr = mr_next;
 
 		mr_next = LIST_NEXT(mr, mr);
-		mr_free(mr, share_cache->dereg_mr_cb);
+		mlx5_mr_free(mr, share_cache->dereg_mr_cb);
 	}
 }
 
@@ -702,7 +702,7 @@ struct mlx5_mr *
 		data.start = RTE_ALIGN_FLOOR(addr, msl->page_sz);
 		data.end = data.start + msl->page_sz;
 		rte_mcfg_mem_read_unlock();
-		mr_free(mr, share_cache->dereg_mr_cb);
+		mlx5_mr_free(mr, share_cache->dereg_mr_cb);
 		goto alloc_resources;
 	}
 	MLX5_ASSERT(data.msl == data_re.msl);
@@ -725,7 +725,7 @@ struct mlx5_mr *
 		 * Must be unlocked before calling rte_free() because
 		 * mlx5_mr_mem_event_free_cb() can be called inside.
 		 */
-		mr_free(mr, share_cache->dereg_mr_cb);
+		mlx5_mr_free(mr, share_cache->dereg_mr_cb);
 		return entry->lkey;
 	}
 	/*
@@ -801,7 +801,7 @@ struct mlx5_mr *
 	 * calling rte_free() because mlx5_mr_mem_event_free_cb() can be called
 	 * inside.
 	 */
-	mr_free(mr, share_cache->dereg_mr_cb);
+	mlx5_mr_free(mr, share_cache->dereg_mr_cb);
 	return UINT32_MAX;
 }
 
diff --git a/drivers/common/mlx5/mlx5_common_mr.h b/drivers/common/mlx5/mlx5_common_mr.h
index a2c426d..da0a0f0 100644
--- a/drivers/common/mlx5/mlx5_common_mr.h
+++ b/drivers/common/mlx5/mlx5_common_mr.h
@@ -171,4 +171,8 @@ struct mlx5_mr *
 __rte_internal
 void
 mlx5_common_verbs_dereg_mr(struct mlx5_pmd_mr *pmd_mr);
+
+__rte_internal
+void
+mlx5_mr_free(struct mlx5_mr *mr, mlx5_dereg_mr_t dereg_mr_cb);
 #endif /* RTE_PMD_MLX5_COMMON_MR_H_ */
diff --git a/drivers/common/mlx5/version.map b/drivers/common/mlx5/version.map
index 884001c..f0f3ca9 100644
--- a/drivers/common/mlx5/version.map
+++ b/drivers/common/mlx5/version.map
@@ -63,6 +63,7 @@ INTERNAL {
 	mlx5_mr_lookup_list;
 	mlx5_mr_create_primary;
 	mlx5_mr_flush_local_cache;
+	mlx5_mr_free;
 
 	mlx5_nl_allmulti;
 	mlx5_nl_devlink_family_id_get;
diff --git a/drivers/net/mlx5/mlx5_mr.c b/drivers/net/mlx5/mlx5_mr.c
index c308ecc..8b20ee3 100644
--- a/drivers/net/mlx5/mlx5_mr.c
+++ b/drivers/net/mlx5/mlx5_mr.c
@@ -404,7 +404,7 @@ struct mr_update_mp_data {
 		return -1;
 	}
 	LIST_REMOVE(mr, mr);
-	LIST_INSERT_HEAD(&sh->share_cache.mr_free_list, mr, mr);
+	mlx5_mr_free(mr, sh->share_cache.dereg_mr_cb);
 	DEBUG("port %u remove MR(%p) from list", dev->data->port_id,
 	      (void *)mr);
 	mlx5_mr_rebuild_cache(&sh->share_cache);
-- 
1.8.3.1


^ permalink raw reply	[flat|nested] 4+ messages in thread

* Re: [dpdk-stable] [PATCH] common/mlx5: free MR resource while device DMA unmap
  2020-10-28 13:14 [dpdk-stable] [PATCH] common/mlx5: free MR resource while device DMA unmap Jiawei Wang
@ 2020-10-28 13:16 ` Slava Ovsiienko
  2020-11-02  2:22 ` [dpdk-stable] [PATCH v2] " Jiawei Wang
  1 sibling, 0 replies; 4+ messages in thread
From: Slava Ovsiienko @ 2020-10-28 13:16 UTC (permalink / raw)
  To: Jiawei(Jonny) Wang, Matan Azrad, Shahaf Shuler, Ori Kam
  Cc: dev, Raslan Darawsheh, stable

> -----Original Message-----
> From: Jiawei Wang <jiaweiw@nvidia.com>
> Sent: Wednesday, October 28, 2020 15:14
> To: Slava Ovsiienko <viacheslavo@nvidia.com>; Matan Azrad
> <matan@nvidia.com>; Shahaf Shuler <shahafs@nvidia.com>; Ori Kam
> <orika@nvidia.com>
> Cc: dev@dpdk.org; Raslan Darawsheh <rasland@nvidia.com>; stable@dpdk.org
> Subject: [PATCH] common/mlx5: free MR resource while device DMA unmap
> 
> mlx5 PMD created the MR (Memory Region) resource on the mlx5_dma_map
> call to make the memory available for DMA operations. On the
> mlx5_dma_unmap call the MR resource was not freed but inserted to MR Free
> list for further garbage collection.
> Actual MR resource destroying happened on device stop call. That caused the
> runtime out of memory in case of application performed multiple DMA
> map/unmap calls.
> 
> The fix immediately frees the MR resource on mlx5_dma_unmap call not
> engaging the list. The export for mlx5_mr_free function from common PMD
> part is added as well.
> 
> Redmine: 2330226
> Fixes: 989e999d9305 ("net/mlx5: support PCI device DMA map and unmap")
> Cc: stable@dpdk.org
> 
> Signed-off-by: Jiawei Wang <jiaweiw@nvidia.com>
Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>

^ permalink raw reply	[flat|nested] 4+ messages in thread

* [dpdk-stable] [PATCH v2] common/mlx5: free MR resource while device DMA unmap
  2020-10-28 13:14 [dpdk-stable] [PATCH] common/mlx5: free MR resource while device DMA unmap Jiawei Wang
  2020-10-28 13:16 ` Slava Ovsiienko
@ 2020-11-02  2:22 ` Jiawei Wang
  2020-11-05 13:25   ` Raslan Darawsheh
  1 sibling, 1 reply; 4+ messages in thread
From: Jiawei Wang @ 2020-11-02  2:22 UTC (permalink / raw)
  To: viacheslavo, matan, shahafs, orika; +Cc: dev, rasland, stable

mlx5 PMD created the MR (Memory Region) resource on the
mlx5_dma_map call to make the memory available for DMA
operations. On the mlx5_dma_unmap call the MR resource
was not freed but inserted to MR Free list for further
garbage collection.
Actual MR resource destroying happened on device stop
call. That caused the runtime out of memory in case of
application performed multiple DMA map/unmap calls.

The fix immediately frees the MR resource on mlx5_dma_unmap
call not engaging the list. The export for mlx5_mr_free
function from common PMD part is added as well.

Fixes: 989e999d9305 ("net/mlx5: support PCI device DMA map and unmap")
Cc: stable@dpdk.org

Signed-off-by: Jiawei Wang <jiaweiw@nvidia.com>
Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
---
 drivers/common/mlx5/mlx5_common_mr.c | 12 ++++++------
 drivers/common/mlx5/mlx5_common_mr.h |  4 ++++
 drivers/common/mlx5/version.map      |  1 +
 drivers/net/mlx5/mlx5_mr.c           |  2 +-
 4 files changed, 12 insertions(+), 7 deletions(-)

diff --git a/drivers/common/mlx5/mlx5_common_mr.c b/drivers/common/mlx5/mlx5_common_mr.c
index 23324c0..7c25541 100644
--- a/drivers/common/mlx5/mlx5_common_mr.c
+++ b/drivers/common/mlx5/mlx5_common_mr.c
@@ -436,8 +436,8 @@ struct mlx5_mr *
  * @param mr
  *   Pointer to MR to free.
  */
-static void
-mr_free(struct mlx5_mr *mr, mlx5_dereg_mr_t dereg_mr_cb)
+void
+mlx5_mr_free(struct mlx5_mr *mr, mlx5_dereg_mr_t dereg_mr_cb)
 {
 	if (mr == NULL)
 		return;
@@ -492,7 +492,7 @@ struct mlx5_mr *
 		struct mlx5_mr *mr = mr_next;
 
 		mr_next = LIST_NEXT(mr, mr);
-		mr_free(mr, share_cache->dereg_mr_cb);
+		mlx5_mr_free(mr, share_cache->dereg_mr_cb);
 	}
 }
 
@@ -702,7 +702,7 @@ struct mlx5_mr *
 		data.start = RTE_ALIGN_FLOOR(addr, msl->page_sz);
 		data.end = data.start + msl->page_sz;
 		rte_mcfg_mem_read_unlock();
-		mr_free(mr, share_cache->dereg_mr_cb);
+		mlx5_mr_free(mr, share_cache->dereg_mr_cb);
 		goto alloc_resources;
 	}
 	MLX5_ASSERT(data.msl == data_re.msl);
@@ -725,7 +725,7 @@ struct mlx5_mr *
 		 * Must be unlocked before calling rte_free() because
 		 * mlx5_mr_mem_event_free_cb() can be called inside.
 		 */
-		mr_free(mr, share_cache->dereg_mr_cb);
+		mlx5_mr_free(mr, share_cache->dereg_mr_cb);
 		return entry->lkey;
 	}
 	/*
@@ -801,7 +801,7 @@ struct mlx5_mr *
 	 * calling rte_free() because mlx5_mr_mem_event_free_cb() can be called
 	 * inside.
 	 */
-	mr_free(mr, share_cache->dereg_mr_cb);
+	mlx5_mr_free(mr, share_cache->dereg_mr_cb);
 	return UINT32_MAX;
 }
 
diff --git a/drivers/common/mlx5/mlx5_common_mr.h b/drivers/common/mlx5/mlx5_common_mr.h
index a2c426d..da0a0f0 100644
--- a/drivers/common/mlx5/mlx5_common_mr.h
+++ b/drivers/common/mlx5/mlx5_common_mr.h
@@ -171,4 +171,8 @@ struct mlx5_mr *
 __rte_internal
 void
 mlx5_common_verbs_dereg_mr(struct mlx5_pmd_mr *pmd_mr);
+
+__rte_internal
+void
+mlx5_mr_free(struct mlx5_mr *mr, mlx5_dereg_mr_t dereg_mr_cb);
 #endif /* RTE_PMD_MLX5_COMMON_MR_H_ */
diff --git a/drivers/common/mlx5/version.map b/drivers/common/mlx5/version.map
index 884001c..f0f3ca9 100644
--- a/drivers/common/mlx5/version.map
+++ b/drivers/common/mlx5/version.map
@@ -63,6 +63,7 @@ INTERNAL {
 	mlx5_mr_lookup_list;
 	mlx5_mr_create_primary;
 	mlx5_mr_flush_local_cache;
+	mlx5_mr_free;
 
 	mlx5_nl_allmulti;
 	mlx5_nl_devlink_family_id_get;
diff --git a/drivers/net/mlx5/mlx5_mr.c b/drivers/net/mlx5/mlx5_mr.c
index c308ecc..8b20ee3 100644
--- a/drivers/net/mlx5/mlx5_mr.c
+++ b/drivers/net/mlx5/mlx5_mr.c
@@ -404,7 +404,7 @@ struct mr_update_mp_data {
 		return -1;
 	}
 	LIST_REMOVE(mr, mr);
-	LIST_INSERT_HEAD(&sh->share_cache.mr_free_list, mr, mr);
+	mlx5_mr_free(mr, sh->share_cache.dereg_mr_cb);
 	DEBUG("port %u remove MR(%p) from list", dev->data->port_id,
 	      (void *)mr);
 	mlx5_mr_rebuild_cache(&sh->share_cache);
-- 
1.8.3.1


^ permalink raw reply	[flat|nested] 4+ messages in thread

* Re: [dpdk-stable] [PATCH v2] common/mlx5: free MR resource while device DMA unmap
  2020-11-02  2:22 ` [dpdk-stable] [PATCH v2] " Jiawei Wang
@ 2020-11-05 13:25   ` Raslan Darawsheh
  0 siblings, 0 replies; 4+ messages in thread
From: Raslan Darawsheh @ 2020-11-05 13:25 UTC (permalink / raw)
  To: Jiawei(Jonny) Wang, Slava Ovsiienko, Matan Azrad, Shahaf Shuler, Ori Kam
  Cc: dev, stable

Hi,

> -----Original Message-----
> From: Jiawei Wang <jiaweiw@nvidia.com>
> Sent: Monday, November 2, 2020 4:22 AM
> To: Slava Ovsiienko <viacheslavo@nvidia.com>; Matan Azrad
> <matan@nvidia.com>; Shahaf Shuler <shahafs@nvidia.com>; Ori Kam
> <orika@nvidia.com>
> Cc: dev@dpdk.org; Raslan Darawsheh <rasland@nvidia.com>;
> stable@dpdk.org
> Subject: [PATCH v2] common/mlx5: free MR resource while device DMA
> unmap
> 
> mlx5 PMD created the MR (Memory Region) resource on the
> mlx5_dma_map call to make the memory available for DMA
> operations. On the mlx5_dma_unmap call the MR resource
> was not freed but inserted to MR Free list for further
> garbage collection.
> Actual MR resource destroying happened on device stop
> call. That caused the runtime out of memory in case of
> application performed multiple DMA map/unmap calls.
> 
> The fix immediately frees the MR resource on mlx5_dma_unmap
> call not engaging the list. The export for mlx5_mr_free
> function from common PMD part is added as well.
> 
> Fixes: 989e999d9305 ("net/mlx5: support PCI device DMA map and unmap")
> Cc: stable@dpdk.org
> 
> Signed-off-by: Jiawei Wang <jiaweiw@nvidia.com>
> Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
> ---
>  drivers/common/mlx5/mlx5_common_mr.c | 12 ++++++------
>  drivers/common/mlx5/mlx5_common_mr.h |  4 ++++
>  drivers/common/mlx5/version.map      |  1 +
>  drivers/net/mlx5/mlx5_mr.c           |  2 +-
>  4 files changed, 12 insertions(+), 7 deletions(-)
> 

Patch applied to next-net-mlx,

Kindest regards,
Raslan Darawsheh

^ permalink raw reply	[flat|nested] 4+ messages in thread

end of thread, other threads:[~2020-11-05 13:25 UTC | newest]

Thread overview: 4+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2020-10-28 13:14 [dpdk-stable] [PATCH] common/mlx5: free MR resource while device DMA unmap Jiawei Wang
2020-10-28 13:16 ` Slava Ovsiienko
2020-11-02  2:22 ` [dpdk-stable] [PATCH v2] " Jiawei Wang
2020-11-05 13:25   ` Raslan Darawsheh

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).