[dpdk-dev] [PATCH 1/2] dmadev: hide devices array

[dpdk-dev] [PATCH 1/2] dmadev: hide devices array

[David Marchand]

No need to expose rte_dma_devices out of the dmadev library.
Existing helpers should be enough, and inlines make use of
rte_dma_fp_objs.

Signed-off-by: David Marchand <david.marchand@redhat.com>
---
 app/test/test_dmadev.c      | 5 +++--
 lib/dmadev/rte_dmadev.c     | 2 +-
 lib/dmadev/rte_dmadev_pmd.h | 2 --
 lib/dmadev/version.map      | 1 -
 4 files changed, 4 insertions(+), 6 deletions(-)

diff --git a/app/test/test_dmadev.c b/app/test/test_dmadev.c
index 1e327bd45f..8b58256afc 100644
--- a/app/test/test_dmadev.c
+++ b/app/test/test_dmadev.c
@@ -747,10 +747,11 @@ test_dmadev_instance(int16_t dev_id)
 	};
 	const int vchan = 0;
 
+	rte_dma_info_get(dev_id, &info);
+
 	printf("\n### Test dmadev instance %u [%s]\n",
-			dev_id, rte_dma_devices[dev_id].data->dev_name);
+			dev_id, info.dev_name);
 
-	rte_dma_info_get(dev_id, &info);
 	if (info.max_vchans < 1)
 		ERR_RETURN("Error, no channels available on device id %u\n", dev_id);
 
diff --git a/lib/dmadev/rte_dmadev.c b/lib/dmadev/rte_dmadev.c
index 182d32aedb..d4b32b2971 100644
--- a/lib/dmadev/rte_dmadev.c
+++ b/lib/dmadev/rte_dmadev.c
@@ -18,7 +18,7 @@
 static int16_t dma_devices_max;
 
 struct rte_dma_fp_object *rte_dma_fp_objs;
-struct rte_dma_dev *rte_dma_devices;
+static struct rte_dma_dev *rte_dma_devices;
 static struct {
 	/* Hold the dev_max information of the primary process. This field is
 	 * set by the primary process and is read by the secondary process.
diff --git a/lib/dmadev/rte_dmadev_pmd.h b/lib/dmadev/rte_dmadev_pmd.h
index b97b5bf10b..5316ad5b5f 100644
--- a/lib/dmadev/rte_dmadev_pmd.h
+++ b/lib/dmadev/rte_dmadev_pmd.h
@@ -131,8 +131,6 @@ struct rte_dma_dev {
 	uint64_t reserved[2]; /**< Reserved for future fields. */
 } __rte_cache_aligned;
 
-extern struct rte_dma_dev *rte_dma_devices;
-
 /**
  * @internal
  * Allocate a new dmadev slot for an DMA device and return the pointer to that
diff --git a/lib/dmadev/version.map b/lib/dmadev/version.map
index ef561acd46..89f7a5b1d3 100644
--- a/lib/dmadev/version.map
+++ b/lib/dmadev/version.map
@@ -30,7 +30,6 @@ EXPERIMENTAL {
 INTERNAL {
 	global:
 
-	rte_dma_devices;
 	rte_dma_fp_objs;
 	rte_dma_pmd_allocate;
 	rte_dma_pmd_release;
-- 
2.23.0

[dpdk-dev] [PATCH 2/2] dmadev: remove symbol versioning for inline helpers

[David Marchand]

Inline helpers have no global symbols in shared libraries.
There is no reason to ask for versioning (plus this library would not
build on Windows).

Fixes: 91e581e5c924 ("dmadev: add data plane API")
Fixes: ea8cf0f8536d ("dmadev: add burst capacity API")

Signed-off-by: David Marchand <david.marchand@redhat.com>
---
 lib/dmadev/version.map | 7 -------
 1 file changed, 7 deletions(-)

diff --git a/lib/dmadev/version.map b/lib/dmadev/version.map
index 89f7a5b1d3..7031d6b335 100644
--- a/lib/dmadev/version.map
+++ b/lib/dmadev/version.map
@@ -1,17 +1,11 @@
 EXPERIMENTAL {
 	global:
 
-	rte_dma_burst_capacity;
 	rte_dma_close;
-	rte_dma_completed;
-	rte_dma_completed_status;
 	rte_dma_configure;
-	rte_dma_copy;
-	rte_dma_copy_sg;
 	rte_dma_count_avail;
 	rte_dma_dev_max;
 	rte_dma_dump;
-	rte_dma_fill;
 	rte_dma_get_dev_id_by_name;
 	rte_dma_info_get;
 	rte_dma_is_valid;
@@ -20,7 +14,6 @@ EXPERIMENTAL {
 	rte_dma_stats_get;
 	rte_dma_stats_reset;
 	rte_dma_stop;
-	rte_dma_submit;
 	rte_dma_vchan_setup;
 	rte_dma_vchan_status;
 
-- 
2.23.0

Re: [dpdk-dev] [PATCH 2/2] dmadev: remove symbol versioning for inline helpers

[Bruce Richardson]

On Wed, Oct 20, 2021 at 08:59:44AM +0200, David Marchand wrote:
> Inline helpers have no global symbols in shared libraries.
> There is no reason to ask for versioning (plus this library would not
> build on Windows).
> 
> Fixes: 91e581e5c924 ("dmadev: add data plane API")
> Fixes: ea8cf0f8536d ("dmadev: add burst capacity API")
> 
> Signed-off-by: David Marchand <david.marchand@redhat.com>

Acked-by: Bruce Richardson <bruce.richardson@intel.com>

Re: [dpdk-dev] [PATCH 1/2] dmadev: hide devices array

[fengchengwen]

On 2021/10/20 14:59, David Marchand wrote:
> No need to expose rte_dma_devices out of the dmadev library.
> Existing helpers should be enough, and inlines make use of
> rte_dma_fp_objs.
> 
> Signed-off-by: David Marchand <david.marchand@redhat.com>
> ---
>  app/test/test_dmadev.c      | 5 +++--
>  lib/dmadev/rte_dmadev.c     | 2 +-
>  lib/dmadev/rte_dmadev_pmd.h | 2 --
>  lib/dmadev/version.map      | 1 -
>  4 files changed, 4 insertions(+), 6 deletions(-)
> 
> diff --git a/app/test/test_dmadev.c b/app/test/test_dmadev.c
> index 1e327bd45f..8b58256afc 100644
> --- a/app/test/test_dmadev.c
> +++ b/app/test/test_dmadev.c
> @@ -747,10 +747,11 @@ test_dmadev_instance(int16_t dev_id)
>  	};
>  	const int vchan = 0;
>  
> +	rte_dma_info_get(dev_id, &info);

suggest declare info as: struct rte_dma_stats info = { 0 };
so that the following %s will display NULL if rte_dma_info_get call fail.

anyway,
Reviewed-by: Chengwen Feng <fengchengwen@huawei.com>

> +
>  	printf("\n### Test dmadev instance %u [%s]\n",
> -			dev_id, rte_dma_devices[dev_id].data->dev_name);
> +			dev_id, info.dev_name);
>  
> -	rte_dma_info_get(dev_id, &info);
>  	if (info.max_vchans < 1)
>  		ERR_RETURN("Error, no channels available on device id %u\n", dev_id);
>  
> diff --git a/lib/dmadev/rte_dmadev.c b/lib/dmadev/rte_dmadev.c
> index 182d32aedb..d4b32b2971 100644

[snip]

Re: [dpdk-dev] [PATCH 1/2] dmadev: hide devices array

[David Marchand]

On Wed, Oct 20, 2021 at 11:47 AM fengchengwen <fengchengwen@huawei.com> wrote:
>
> On 2021/10/20 14:59, David Marchand wrote:
> > No need to expose rte_dma_devices out of the dmadev library.
> > Existing helpers should be enough, and inlines make use of
> > rte_dma_fp_objs.
> >
> > Signed-off-by: David Marchand <david.marchand@redhat.com>
> > ---
> >  app/test/test_dmadev.c      | 5 +++--
> >  lib/dmadev/rte_dmadev.c     | 2 +-
> >  lib/dmadev/rte_dmadev_pmd.h | 2 --
> >  lib/dmadev/version.map      | 1 -
> >  4 files changed, 4 insertions(+), 6 deletions(-)
> >
> > diff --git a/app/test/test_dmadev.c b/app/test/test_dmadev.c
> > index 1e327bd45f..8b58256afc 100644
> > --- a/app/test/test_dmadev.c
> > +++ b/app/test/test_dmadev.c
> > @@ -747,10 +747,11 @@ test_dmadev_instance(int16_t dev_id)
> >       };
> >       const int vchan = 0;
> >
> > +     rte_dma_info_get(dev_id, &info);
>
> suggest declare info as: struct rte_dma_stats info = { 0 };
> so that the following %s will display NULL if rte_dma_info_get call fail.

The problem is more generic.
Other info fields are used by the test.
If rte_dma_info_get can fail, its return code must be checked.

Worth a followup patch, can you send it?

Thanks.


-- 
David Marchand

Re: [dpdk-dev] [PATCH 1/2] dmadev: hide devices array

[Walsh, Conor]

> Subject: [dpdk-dev] [PATCH 1/2] dmadev: hide devices array
> 
> No need to expose rte_dma_devices out of the dmadev library.
> Existing helpers should be enough, and inlines make use of
> rte_dma_fp_objs.
> 
> Signed-off-by: David Marchand <david.marchand@redhat.com>
> ---

The devices array is not needed by the drivers.

From a drivers point of view:
Tested-by: Conor Walsh <conor.walsh@intel.com>

[dpdk-dev] [PATCH v2 1/3] dmadev: hide devices array

[Chengwen Feng]

From: David Marchand <david.marchand@redhat.com>

No need to expose rte_dma_devices out of the dmadev library.
Existing helpers should be enough, and inlines make use of
rte_dma_fp_objs.

Signed-off-by: David Marchand <david.marchand@redhat.com>
Reviewed-by: Chengwen Feng <fengchengwen@huawei.com>
Tested-by: Conor Walsh <conor.walsh@intel.com>
---
 app/test/test_dmadev.c      | 5 +++--
 lib/dmadev/rte_dmadev.c     | 2 +-
 lib/dmadev/rte_dmadev_pmd.h | 2 --
 lib/dmadev/version.map      | 1 -
 4 files changed, 4 insertions(+), 6 deletions(-)

diff --git a/app/test/test_dmadev.c b/app/test/test_dmadev.c
index 1e327bd45f..8b58256afc 100644
--- a/app/test/test_dmadev.c
+++ b/app/test/test_dmadev.c
@@ -747,10 +747,11 @@ test_dmadev_instance(int16_t dev_id)
 	};
 	const int vchan = 0;
 
+	rte_dma_info_get(dev_id, &info);
+
 	printf("\n### Test dmadev instance %u [%s]\n",
-			dev_id, rte_dma_devices[dev_id].data->dev_name);
+			dev_id, info.dev_name);
 
-	rte_dma_info_get(dev_id, &info);
 	if (info.max_vchans < 1)
 		ERR_RETURN("Error, no channels available on device id %u\n", dev_id);
 
diff --git a/lib/dmadev/rte_dmadev.c b/lib/dmadev/rte_dmadev.c
index 182d32aedb..d4b32b2971 100644
--- a/lib/dmadev/rte_dmadev.c
+++ b/lib/dmadev/rte_dmadev.c
@@ -18,7 +18,7 @@
 static int16_t dma_devices_max;
 
 struct rte_dma_fp_object *rte_dma_fp_objs;
-struct rte_dma_dev *rte_dma_devices;
+static struct rte_dma_dev *rte_dma_devices;
 static struct {
 	/* Hold the dev_max information of the primary process. This field is
 	 * set by the primary process and is read by the secondary process.
diff --git a/lib/dmadev/rte_dmadev_pmd.h b/lib/dmadev/rte_dmadev_pmd.h
index b97b5bf10b..5316ad5b5f 100644
--- a/lib/dmadev/rte_dmadev_pmd.h
+++ b/lib/dmadev/rte_dmadev_pmd.h
@@ -131,8 +131,6 @@ struct rte_dma_dev {
 	uint64_t reserved[2]; /**< Reserved for future fields. */
 } __rte_cache_aligned;
 
-extern struct rte_dma_dev *rte_dma_devices;
-
 /**
  * @internal
  * Allocate a new dmadev slot for an DMA device and return the pointer to that
diff --git a/lib/dmadev/version.map b/lib/dmadev/version.map
index ef561acd46..89f7a5b1d3 100644
--- a/lib/dmadev/version.map
+++ b/lib/dmadev/version.map
@@ -30,7 +30,6 @@ EXPERIMENTAL {
 INTERNAL {
 	global:
 
-	rte_dma_devices;
 	rte_dma_fp_objs;
 	rte_dma_pmd_allocate;
 	rte_dma_pmd_release;
-- 
2.33.0

[dpdk-dev] [PATCH v2 2/3] app/test: add check for DMA info get API

[Chengwen Feng]

This patch add check for rte_dma_info_get() API.

Signed-off-by: Chengwen Feng <fengchengwen@huawei.com>
---
 app/test/test_dmadev.c | 9 ++++++---
 1 file changed, 6 insertions(+), 3 deletions(-)

diff --git a/app/test/test_dmadev.c b/app/test/test_dmadev.c
index 8b58256afc..d6d5dfc803 100644
--- a/app/test/test_dmadev.c
+++ b/app/test/test_dmadev.c
@@ -746,8 +746,11 @@ test_dmadev_instance(int16_t dev_id)
 			.nb_desc = TEST_RINGSIZE,
 	};
 	const int vchan = 0;
+	int ret;
 
-	rte_dma_info_get(dev_id, &info);
+	ret = rte_dma_info_get(dev_id, &info);
+	if (ret)
+		ERR_RETURN("Error with rte_dma_info_get()\n");
 
 	printf("\n### Test dmadev instance %u [%s]\n",
 			dev_id, info.dev_name);
@@ -761,8 +764,8 @@ test_dmadev_instance(int16_t dev_id)
 	if (rte_dma_vchan_setup(dev_id, vchan, &qconf) < 0)
 		ERR_RETURN("Error with queue configuration\n");
 
-	rte_dma_info_get(dev_id, &info);
-	if (info.nb_vchans != 1)
+	ret = rte_dma_info_get(dev_id, &info);
+	if (ret || info.nb_vchans != 1)
 		ERR_RETURN("Error, no configured queues reported on device id %u\n", dev_id);
 
 	if (rte_dma_start(dev_id) != 0)
-- 
2.33.0

Re: [dpdk-dev] [PATCH v2 2/3] app/test: add check for DMA info get API

[David Marchand]

On Thu, Oct 21, 2021 at 3:04 PM Chengwen Feng <fengchengwen@huawei.com> wrote:
>
> This patch add check for rte_dma_info_get() API.

Fixes: 718f7804841f ("test/dma: add basic dmadev instance tests")

>
> Signed-off-by: Chengwen Feng <fengchengwen@huawei.com>

Thanks for following up.

> ---
>  app/test/test_dmadev.c | 9 ++++++---
>  1 file changed, 6 insertions(+), 3 deletions(-)
>
> diff --git a/app/test/test_dmadev.c b/app/test/test_dmadev.c
> index 8b58256afc..d6d5dfc803 100644
> --- a/app/test/test_dmadev.c
> +++ b/app/test/test_dmadev.c
> @@ -746,8 +746,11 @@ test_dmadev_instance(int16_t dev_id)
>                         .nb_desc = TEST_RINGSIZE,
>         };
>         const int vchan = 0;
> +       int ret;
>
> -       rte_dma_info_get(dev_id, &info);
> +       ret = rte_dma_info_get(dev_id, &info);
> +       if (ret)

Nit: ret != 0

> +               ERR_RETURN("Error with rte_dma_info_get()\n");
>
>         printf("\n### Test dmadev instance %u [%s]\n",
>                         dev_id, info.dev_name);
> @@ -761,8 +764,8 @@ test_dmadev_instance(int16_t dev_id)
>         if (rte_dma_vchan_setup(dev_id, vchan, &qconf) < 0)
>                 ERR_RETURN("Error with queue configuration\n");
>
> -       rte_dma_info_get(dev_id, &info);
> -       if (info.nb_vchans != 1)
> +       ret = rte_dma_info_get(dev_id, &info);
> +       if (ret || info.nb_vchans != 1)

Nit: ret != 0

>                 ERR_RETURN("Error, no configured queues reported on device id %u\n", dev_id);
>
>         if (rte_dma_start(dev_id) != 0)
> --
> 2.33.0
>

Reviewed-by: David Marchand <david.marchand@redhat.com>


-- 
David Marchand

Re: [dpdk-dev] [PATCH v2 2/3] app/test: add check for DMA info get API

[Kevin Laatz]

On 21/10/2021 13:59, Chengwen Feng wrote:
> This patch add check for rte_dma_info_get() API.
>
> Signed-off-by: Chengwen Feng <fengchengwen@huawei.com>
> ---
>   app/test/test_dmadev.c | 9 ++++++---
>   1 file changed, 6 insertions(+), 3 deletions(-)
>

Acked-by: Kevin Laatz <kevin.laatz@intel.com>

Re: [dpdk-dev] [PATCH v2 2/3] app/test: add check for DMA info get API

[Walsh, Conor]

> From: Laatz, Kevin <kevin.laatz@intel.com>
> Sent: Thursday 21 October 2021 15:04
> To: Chengwen Feng <fengchengwen@huawei.com>; dev@dpdk.org
> Cc: david.marchand@redhat.com; mdr@ashroe.eu; Walsh, Conor
> <conor.walsh@intel.com>; Richardson, Bruce <bruce.richardson@intel.com>
> Subject: Re: [PATCH v2 2/3] app/test: add check for DMA info get API
> 
> On 21/10/2021 13:59, Chengwen Feng wrote:
> > This patch add check for rte_dma_info_get() API.
> >
> > Signed-off-by: Chengwen Feng <fengchengwen@huawei.com>
> > ---
> >   app/test/test_dmadev.c | 9 ++++++---
> >   1 file changed, 6 insertions(+), 3 deletions(-)

Acked-by: Conor Walsh <conor.walsh@intel.com>

[dpdk-dev] [PATCH v2 3/3] dmadev: fix compile error in debug mode

[Chengwen Feng]

This patch fix compile error when enable RTE_DMADEV_DEBUG.

Fixes: ea8cf0f8536d ("dmadev: add burst capacity API")

Signed-off-by: Chengwen Feng <fengchengwen@huawei.com>
---
 lib/dmadev/rte_dmadev.h | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/lib/dmadev/rte_dmadev.h b/lib/dmadev/rte_dmadev.h
index f5d23017b1..9942c6ec21 100644
--- a/lib/dmadev/rte_dmadev.h
+++ b/lib/dmadev/rte_dmadev.h
@@ -1126,7 +1126,7 @@ rte_dma_burst_capacity(int16_t dev_id, uint16_t vchan)
 #ifdef RTE_DMADEV_DEBUG
 	if (!rte_dma_is_valid(dev_id))
 		return 0;
-	RTE_FUNC_PTR_OR_ERR_RET(*obbj->burst_capacity, 0);
+	RTE_FUNC_PTR_OR_ERR_RET(*obj->burst_capacity, 0);
 #endif
 	return (*obj->burst_capacity)(obj->dev_private, vchan);
 }
-- 
2.33.0

Re: [dpdk-dev] [PATCH v2 3/3] dmadev: fix compile error in debug mode

[David Marchand]

On Thu, Oct 21, 2021 at 3:04 PM Chengwen Feng <fengchengwen@huawei.com> wrote:
>
> This patch fix compile error when enable RTE_DMADEV_DEBUG.
>
> Fixes: ea8cf0f8536d ("dmadev: add burst capacity API")
>
> Signed-off-by: Chengwen Feng <fengchengwen@huawei.com>

Reviewed-by: David Marchand <david.marchand@redhat.com>


-- 
David Marchand

Re: [dpdk-dev] [PATCH v2 3/3] dmadev: fix compile error in debug mode

[Kevin Laatz]

On 21/10/2021 13:59, Chengwen Feng wrote:
> This patch fix compile error when enable RTE_DMADEV_DEBUG.
>
> Fixes: ea8cf0f8536d ("dmadev: add burst capacity API")
>
> Signed-off-by: Chengwen Feng <fengchengwen@huawei.com>
> ---
>   lib/dmadev/rte_dmadev.h | 2 +-
>   1 file changed, 1 insertion(+), 1 deletion(-)
>

Acked-by: Kevin Laatz <kevin.laatz@intel.com>

Re: [dpdk-dev] [PATCH v2 3/3] dmadev: fix compile error in debug mode

[Walsh, Conor]

> From: Laatz, Kevin <kevin.laatz@intel.com>
> Sent: Thursday 21 October 2021 15:04
> To: Chengwen Feng <fengchengwen@huawei.com>; dev@dpdk.org
> Cc: david.marchand@redhat.com; mdr@ashroe.eu; Walsh, Conor
> <conor.walsh@intel.com>; Richardson, Bruce <bruce.richardson@intel.com>
> Subject: Re: [PATCH v2 3/3] dmadev: fix compile error in debug mode
> 
> On 21/10/2021 13:59, Chengwen Feng wrote:
> > This patch fix compile error when enable RTE_DMADEV_DEBUG.
> >
> > Fixes: ea8cf0f8536d ("dmadev: add burst capacity API")
> >
> > Signed-off-by: Chengwen Feng <fengchengwen@huawei.com>
> > ---

Acked-by: Conor Walsh <conor.walsh@intel.com>

Re: [dpdk-dev] [PATCH v2 3/3] dmadev: fix compile error in debug mode

[David Marchand]

On Thu, Oct 21, 2021 at 3:04 PM Chengwen Feng <fengchengwen@huawei.com> wrote:
>
> This patch fix compile error when enable RTE_DMADEV_DEBUG.
>
> Fixes: ea8cf0f8536d ("dmadev: add burst capacity API")
>
> Signed-off-by: Chengwen Feng <fengchengwen@huawei.com>

Series applied, thanks.


-- 
David Marchand

Re: [dpdk-dev] [PATCH v2 1/3] dmadev: hide devices array

[Kevin Laatz]

On 21/10/2021 13:59, Chengwen Feng wrote:
> From: David Marchand <david.marchand@redhat.com>
>
> No need to expose rte_dma_devices out of the dmadev library.
> Existing helpers should be enough, and inlines make use of
> rte_dma_fp_objs.
>
> Signed-off-by: David Marchand <david.marchand@redhat.com>
> Reviewed-by: Chengwen Feng <fengchengwen@huawei.com>
> Tested-by: Conor Walsh <conor.walsh@intel.com>
> ---
>   app/test/test_dmadev.c      | 5 +++--
>   lib/dmadev/rte_dmadev.c     | 2 +-
>   lib/dmadev/rte_dmadev_pmd.h | 2 --
>   lib/dmadev/version.map      | 1 -
>   4 files changed, 4 insertions(+), 6 deletions(-)
>

Acked-by: Kevin Laatz <kevin.laatz@intel.com>

Re: [dpdk-dev] [PATCH v2 1/3] dmadev: hide devices array

[Radha Mohan]

On Thu, Oct 21, 2021 at 7:04 AM Kevin Laatz <kevin.laatz@intel.com> wrote:
>
> On 21/10/2021 13:59, Chengwen Feng wrote:
> > From: David Marchand <david.marchand@redhat.com>
> >
> > No need to expose rte_dma_devices out of the dmadev library.
> > Existing helpers should be enough, and inlines make use of
> > rte_dma_fp_objs.

I asked this earlier and was asked to use rte_dma_devices[] array by Kevin.
Now that it has been made static is there a different way to get the
fp_obj->private_data from a PMD .remove function ?
I need the private data for DMA queue state cleanup.

> >
> > Signed-off-by: David Marchand <david.marchand@redhat.com>
> > Reviewed-by: Chengwen Feng <fengchengwen@huawei.com>
> > Tested-by: Conor Walsh <conor.walsh@intel.com>
> > ---
> >   app/test/test_dmadev.c      | 5 +++--
> >   lib/dmadev/rte_dmadev.c     | 2 +-
> >   lib/dmadev/rte_dmadev_pmd.h | 2 --
> >   lib/dmadev/version.map      | 1 -
> >   4 files changed, 4 insertions(+), 6 deletions(-)
> >
>
> Acked-by: Kevin Laatz <kevin.laatz@intel.com>

Re: [dpdk-dev] [PATCH v2 1/3] dmadev: hide devices array

[fengchengwen]

On 2021/10/26 14:04, Radha Mohan wrote:
> On Thu, Oct 21, 2021 at 7:04 AM Kevin Laatz <kevin.laatz@intel.com> wrote:
>>
>> On 21/10/2021 13:59, Chengwen Feng wrote:
>>> From: David Marchand <david.marchand@redhat.com>
>>>
>>> No need to expose rte_dma_devices out of the dmadev library.
>>> Existing helpers should be enough, and inlines make use of
>>> rte_dma_fp_objs.
> 
> I asked this earlier and was asked to use rte_dma_devices[] array by Kevin.
> Now that it has been made static is there a different way to get the
> fp_obj->private_data from a PMD .remove function ?
> I need the private data for DMA queue state cleanup.

PMD .remove function just call rte_dma_pmd_release() and it's will
invoke PMD .dev_close ops:
  rte_dma_pmd_release()
    --> rte_dma_close()
      --> PMD .dev_close ops

PMD could implemente cleanup in dev_close ops.

If the PMD want support multi-process, just make sure only the primary
could do really cleanup in the dev_close ops.

> 
>>>
>>> Signed-off-by: David Marchand <david.marchand@redhat.com>
>>> Reviewed-by: Chengwen Feng <fengchengwen@huawei.com>
>>> Tested-by: Conor Walsh <conor.walsh@intel.com>
>>> ---
>>>   app/test/test_dmadev.c      | 5 +++--
>>>   lib/dmadev/rte_dmadev.c     | 2 +-
>>>   lib/dmadev/rte_dmadev_pmd.h | 2 --
>>>   lib/dmadev/version.map      | 1 -
>>>   4 files changed, 4 insertions(+), 6 deletions(-)
>>>
>>
>> Acked-by: Kevin Laatz <kevin.laatz@intel.com>
> 
> .
>