[PATCH 0/3] idxd driver update fixes

[PATCH 0/3] idxd driver update fixes

[Bruce Richardson]

This set contains a number of small fixes and updates for the idxd dma
driver, mainly in the area of properly handling cases where the default
max batch size is configured to a lower than expected value.

Bruce Richardson (3):
  app/test: fix missing checks for DMA device capacity
  dma/idxd: configure max batch size to high value
  doc/dmadev/idxd: improve configuration examples

 app/test/test_dmadev.c            |  8 +++++++-
 doc/guides/dmadevs/idxd.rst       | 29 +++++++++++++++++++++++++++--
 drivers/dma/idxd/dpdk_idxd_cfg.py |  1 +
 3 files changed, 35 insertions(+), 3 deletions(-)

--
2.32.0

[PATCH 1/3] app/test: fix missing checks for DMA device capacity

[Bruce Richardson]

For some DMA HW devices, e.g. those using the idxd driver, the maximum
burst size is configurable, which can lead to test failures if the value
is set too small. Add explicit check for this to give reasonable error
messages for devices which need their config adjusted.

Fixes: 1b86a66a30c2 ("test/dma: add more comprehensive copy tests")
Fixes: 8fa5d2683940 ("test/dma: add burst capacity test")
Cc: kevin.laatz@intel.com
Cc: stable@dpdk.org

Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
---
 app/test/test_dmadev.c | 8 +++++++-
 1 file changed, 7 insertions(+), 1 deletion(-)

diff --git a/app/test/test_dmadev.c b/app/test/test_dmadev.c
index db5aff701c..2b097e0f47 100644
--- a/app/test/test_dmadev.c
+++ b/app/test/test_dmadev.c
@@ -775,6 +775,9 @@ test_dmadev_instance(int16_t dev_id)
 	if (rte_dma_stats_get(dev_id, vchan, &stats) != 0)
 		ERR_RETURN("Error with rte_dma_stats_get()\n");
 
+	if (rte_dma_burst_capacity(dev_id, vchan) < 32)
+		ERR_RETURN("Error: Device does not have sufficient burst capacity to run tests");
+
 	if (stats.completed != 0 || stats.submitted != 0 || stats.errors != 0)
 		ERR_RETURN("Error device stats are not all zero: completed = %"PRIu64", "
 				"submitted = %"PRIu64", errors = %"PRIu64"\n",
@@ -796,7 +799,10 @@ test_dmadev_instance(int16_t dev_id)
 		goto err;
 
 	/* run some burst capacity tests */
-	if (runtest("burst capacity", test_burst_capacity, 1, dev_id, vchan, CHECK_ERRS) < 0)
+	if (rte_dma_burst_capacity(dev_id, vchan) < 64)
+		printf("DMA Dev %u: insufficient burst capacity (64 required), skipping tests\n",
+				dev_id);
+	else if (runtest("burst capacity", test_burst_capacity, 1, dev_id, vchan, CHECK_ERRS) < 0)
 		goto err;
 
 	/* to test error handling we can provide null pointers for source or dest in copies. This
-- 
2.32.0

RE: [PATCH 1/3] app/test: fix missing checks for DMA device capacity

[Walsh, Conor]

Reviewed-by: Conor Walsh <conor.walsh@intel.com>

Re: [PATCH 1/3] app/test: fix missing checks for DMA device capacity

[Kevin Laatz]

On 16/02/2022 16:06, Bruce Richardson wrote:
> For some DMA HW devices, e.g. those using the idxd driver, the maximum
> burst size is configurable, which can lead to test failures if the value
> is set too small. Add explicit check for this to give reasonable error
> messages for devices which need their config adjusted.
>
> Fixes: 1b86a66a30c2 ("test/dma: add more comprehensive copy tests")
> Fixes: 8fa5d2683940 ("test/dma: add burst capacity test")
> Cc: kevin.laatz@intel.com
> Cc: stable@dpdk.org
>
> Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
> ---
>   app/test/test_dmadev.c | 8 +++++++-
>   1 file changed, 7 insertions(+), 1 deletion(-)
>
Reviewed-by: Kevin Laatz <kevin.laatz@intel.com>

[PATCH 2/3] dma/idxd: configure max batch size to high value

[Bruce Richardson]

When configuring an Intel DSA instance using the utility script
dpdk_idxd_cfg.py, explicitly set the max supported batch size value to a
high value, to ensure large bursts are supported if so desired. The
default in the linux kernel is now just 32 [1], which may not be
sufficient for all DPDK apps.

[1] https://lore.kernel.org/r/163528473483.3926048.7950067926287180976.stgit@djiang5-desk3.ch.intel.com

Cc: stable@dpdk.org

Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
---
Note: although not fixing a bug in DPDK itself, due to kernel changes
this patch should be considered for backport as a fix, so cc'ing stable.
---
 drivers/dma/idxd/dpdk_idxd_cfg.py | 1 +
 1 file changed, 1 insertion(+)

diff --git a/drivers/dma/idxd/dpdk_idxd_cfg.py b/drivers/dma/idxd/dpdk_idxd_cfg.py
index 34537cb980..3f5d5ee752 100755
--- a/drivers/dma/idxd/dpdk_idxd_cfg.py
+++ b/drivers/dma/idxd/dpdk_idxd_cfg.py
@@ -89,6 +89,7 @@ def configure_dsa(dsa_id, queues, prefix):
                              "mode": "dedicated",
                              "name": f"{prefix}_wq{dsa_id}.{q}",
                              "priority": 1,
+                             "max_batch_size": 1024,
                              "size": int(max_work_queues_size / nb_queues)})
 
     # enable device and then queues
-- 
2.32.0

Re: [PATCH 2/3] dma/idxd: configure max batch size to high value

[Kevin Laatz]

On 16/02/2022 16:06, Bruce Richardson wrote:
> When configuring an Intel DSA instance using the utility script
> dpdk_idxd_cfg.py, explicitly set the max supported batch size value to a
> high value, to ensure large bursts are supported if so desired. The
> default in the linux kernel is now just 32 [1], which may not be
> sufficient for all DPDK apps.
>
> [1] https://lore.kernel.org/r/163528473483.3926048.7950067926287180976.stgit@djiang5-desk3.ch.intel.com
>
> Cc: stable@dpdk.org
>
> Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
> ---
> Note: although not fixing a bug in DPDK itself, due to kernel changes
> this patch should be considered for backport as a fix, so cc'ing stable.
> ---
>   drivers/dma/idxd/dpdk_idxd_cfg.py | 1 +
>   1 file changed, 1 insertion(+)
>
Reviewed-by: Kevin Laatz <kevin.laatz@intel.com>

[PATCH 3/3] doc/dmadev/idxd: improve configuration examples

[Bruce Richardson]

The documentation on how to configure device instances using
accel-config can be improved by a number of changes:

* For initial example, when only configuring one queue, omit
  configuration of a second engine, which is unused later.
* Add the "max-batch-size" setting to the options being configured for
  each queue
* Add a final, more complete example, showing configuration of multiple
  queues on a device.

Cc: stable@dpdk.org

Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
---
Note: although not strictly a bug fix, this doc update is a good
candidate for backport to 21.11, so cc'ing stable
---
 doc/guides/dmadevs/idxd.rst | 29 +++++++++++++++++++++++++++--
 1 file changed, 27 insertions(+), 2 deletions(-)

diff --git a/doc/guides/dmadevs/idxd.rst b/doc/guides/dmadevs/idxd.rst
index 8d7cea6583..bdfd3e78ad 100644
--- a/doc/guides/dmadevs/idxd.rst
+++ b/doc/guides/dmadevs/idxd.rst
@@ -55,7 +55,6 @@ such as priority or queue depth, need to be set for each queue.
 To assign an engine to a group::
 
         $ accel-config config-engine dsa0/engine0.0 --group-id=0
-        $ accel-config config-engine dsa0/engine0.1 --group-id=1
 
 To assign work queues to groups for passing descriptors to the engines a similar accel-config command can be used.
 However, the work queues also need to be configured depending on the use case.
@@ -71,7 +70,7 @@ Example configuration for a work queue::
 
         $ accel-config config-wq dsa0/wq0.0 --group-id=0 \
            --mode=dedicated --priority=10 --wq-size=8 \
-           --type=user --name=dpdk_app1
+           --max-batch-size=512 --type=user --name=dpdk_app1
 
 Once the devices have been configured, they need to be enabled::
 
@@ -82,6 +81,32 @@ Check the device configuration::
 
         $ accel-config list
 
+Every Intel\ |reg| DSA instance supports multiple queues and each should be similarly configured.
+As a further example, the following set of commands will configure and enable 4 queues on instance 0,
+giving each an equal share of resources::
+
+        # configure 4 groups, each with one engine
+        accel-config config-engine dsa0/engine0.0 --group-id=0
+        accel-config config-engine dsa0/engine0.1 --group-id=1
+        accel-config config-engine dsa0/engine0.2 --group-id=2
+        accel-config config-engine dsa0/engine0.3 --group-id=3
+
+        # configure 4 queues, putting each in a different group, so each
+        # is backed by a single engine
+        accel-config config-wq dsa0/wq0.0 --group-id=0 --type=user --wq-size=32 \
+            --priority=10 --max-batch-size=1024 --mode=dedicated --name=dpdk_app1
+        accel-config config-wq dsa0/wq0.1 --group-id=1 --type=user --wq-size=32 \
+            --priority=10 --max-batch-size=1024 --mode=dedicated --name=dpdk_app1
+        accel-config config-wq dsa0/wq0.2 --group-id=2 --type=user --wq-size=32 \
+            --priority=10 --max-batch-size=1024 --mode=dedicated --name=dpdk_app1
+        accel-config config-wq dsa0/wq0.3 --group-id=3 --type=user --wq-size=32 \
+            --priority=10 --max-batch-size=1024 --mode=dedicated --name=dpdk_app1
+
+        # enable device and queues
+        accel-config enable-device dsa0
+        accel-config enable-wq dsa0/wq0.0 dsa0/wq0.1 dsa0/wq0.2 dsa0/wq0.3
+
+
 Devices using VFIO/UIO drivers
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 
-- 
2.32.0

Re: [PATCH 3/3] doc/dmadev/idxd: improve configuration examples

[Kevin Laatz]

On 16/02/2022 16:06, Bruce Richardson wrote:
> The documentation on how to configure device instances using
> accel-config can be improved by a number of changes:
>
> * For initial example, when only configuring one queue, omit
>    configuration of a second engine, which is unused later.
> * Add the "max-batch-size" setting to the options being configured for
>    each queue
> * Add a final, more complete example, showing configuration of multiple
>    queues on a device.
>
> Cc: stable@dpdk.org
>
> Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
> ---
> Note: although not strictly a bug fix, this doc update is a good
> candidate for backport to 21.11, so cc'ing stable
> ---
>   doc/guides/dmadevs/idxd.rst | 29 +++++++++++++++++++++++++++--
>   1 file changed, 27 insertions(+), 2 deletions(-)
>
Reviewed-by: Kevin Laatz <kevin.laatz@intel.com>

Re: [PATCH 0/3] idxd driver update fixes

[Thomas Monjalon]

16/02/2022 17:06, Bruce Richardson:
> This set contains a number of small fixes and updates for the idxd dma
> driver, mainly in the area of properly handling cases where the default
> max batch size is configured to a lower than expected value.
> 
> Bruce Richardson (3):
>   app/test: fix missing checks for DMA device capacity
>   dma/idxd: configure max batch size to high value
>   doc/dmadev/idxd: improve configuration examples

Applied, thanks.