* [PATCH v1 1/7] eventdev: introduce DMA event adapter library
@ 2023-09-19 13:42 Amit Prakash Shukla
2023-09-19 13:42 ` [PATCH v1 2/7] eventdev: api to get DMA capabilities Amit Prakash Shukla
` (7 more replies)
0 siblings, 8 replies; 110+ messages in thread
From: Amit Prakash Shukla @ 2023-09-19 13:42 UTC (permalink / raw)
To: Jerin Jacob
Cc: dev, fengchengwen, kevin.laatz, bruce.richardson, conor.walsh,
vattunuru, g.singh, sachin.saxena, hemant.agrawal, cheng1.jiang,
ndabilpuram, anoobj, mb, Amit Prakash Shukla
Introduce event DMA adapter APIs. The change provides information
on adapter modes and usage. Application can use this event adapter
interface to transfer packets between DMA device and event device.
Signed-off-by: Amit Prakash Shukla <amitprakashs@marvell.com>
---
doc/api/doxy-api-index.md | 1 +
doc/guides/eventdevs/features/default.ini | 8 +
doc/guides/prog_guide/event_dma_adapter.rst | 268 ++++
doc/guides/prog_guide/eventdev.rst | 8 +-
.../img/event_dma_adapter_op_forward.svg | 1086 +++++++++++++++++
.../img/event_dma_adapter_op_new.svg | 1079 ++++++++++++++++
doc/guides/prog_guide/index.rst | 1 +
lib/eventdev/eventdev_pmd.h | 171 ++-
lib/eventdev/eventdev_private.c | 10 +
lib/eventdev/rte_event_dma_adapter.h | 641 ++++++++++
lib/eventdev/rte_eventdev.h | 45 +
lib/eventdev/rte_eventdev_core.h | 8 +-
lib/eventdev/version.map | 15 +
13 files changed, 3336 insertions(+), 5 deletions(-)
create mode 100644 doc/guides/prog_guide/event_dma_adapter.rst
create mode 100644 doc/guides/prog_guide/img/event_dma_adapter_op_forward.svg
create mode 100644 doc/guides/prog_guide/img/event_dma_adapter_op_new.svg
create mode 100644 lib/eventdev/rte_event_dma_adapter.h
diff --git a/doc/api/doxy-api-index.md b/doc/api/doxy-api-index.md
index fdeda13932..b7df7be4d9 100644
--- a/doc/api/doxy-api-index.md
+++ b/doc/api/doxy-api-index.md
@@ -29,6 +29,7 @@ The public API headers are grouped by topics:
[event_eth_tx_adapter](@ref rte_event_eth_tx_adapter.h),
[event_timer_adapter](@ref rte_event_timer_adapter.h),
[event_crypto_adapter](@ref rte_event_crypto_adapter.h),
+ [event_dma_adapter](@ref rte_event_dma_adapter.h),
[rawdev](@ref rte_rawdev.h),
[metrics](@ref rte_metrics.h),
[bitrate](@ref rte_bitrate.h),
diff --git a/doc/guides/eventdevs/features/default.ini b/doc/guides/eventdevs/features/default.ini
index 00360f60c6..fda8baf487 100644
--- a/doc/guides/eventdevs/features/default.ini
+++ b/doc/guides/eventdevs/features/default.ini
@@ -44,6 +44,14 @@ internal_port_op_fwd =
internal_port_qp_ev_bind =
session_private_data =
+;
+; Features of a default DMA adapter.
+;
+[DMA adapter Features]
+internal_port_op_new =
+internal_port_op_fwd =
+internal_port_qp_ev_bind =
+
;
; Features of a default Timer adapter.
;
diff --git a/doc/guides/prog_guide/event_dma_adapter.rst b/doc/guides/prog_guide/event_dma_adapter.rst
new file mode 100644
index 0000000000..7ac3ce744b
--- /dev/null
+++ b/doc/guides/prog_guide/event_dma_adapter.rst
@@ -0,0 +1,268 @@
+.. SPDX-License-Identifier: BSD-3-Clause
+ Copyright (c) 2023 Marvell.
+
+Event DMA Adapter Library
+=========================
+
+DPDK :doc:`Eventdev library <eventdev>` provides event driven programming model with features
+to schedule events. :doc:`DMA Device library <dmadev>` provides an interface to DMA poll mode
+drivers that support DMA operations. Event DMA Adapter is intended to bridge between the event
+device and the DMA device.
+
+Packet flow from DMA device to the event device can be accomplished using software and hardware
+based transfer mechanisms. The adapter queries an eventdev PMD to determine which mechanism to
+be used. The adapter uses an EAL service core function for software based packet transfer and
+uses the eventdev PMD functions to configure hardware based packet transfer between DMA device
+and the event device. DMA adapter uses a new event type called ``RTE_EVENT_TYPE_DMADEV`` to
+indicate the source of event.
+
+Application can choose to submit an DMA operation directly to an DMA device or send it to an DMA
+adapter via eventdev based on RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP_FWD capability. The
+first mode is known as the event new (RTE_EVENT_DMA_ADAPTER_OP_NEW) mode and the second as the
+event forward (RTE_EVENT_DMA_ADAPTER_OP_FORWARD) mode. Choice of mode can be specified while
+creating the adapter. In the former mode, it is the application's responsibility to enable
+ingress packet ordering. In the latter mode, it is the adapter's responsibility to enable
+ingress packet ordering.
+
+
+Adapter Modes
+-------------
+
+RTE_EVENT_DMA_ADAPTER_OP_NEW mode
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+In the RTE_EVENT_DMA_ADAPTER_OP_NEW mode, application submits DMA operations directly to an DMA
+device. The adapter then dequeues DMA completions from the DMA device and enqueues them as events
+to the event device. This mode does not ensure ingress ordering as the application directly
+enqueues to the dmadev without going through DMA/atomic stage. In this mode, events dequeued
+from the adapter are treated as new events. The application has to specify event information
+(response information) which is needed to enqueue an event after the DMA operation is completed.
+
+.. _figure_event_dma_adapter_op_new:
+
+.. figure:: img/event_dma_adapter_op_new.*
+
+ Working model of ``RTE_EVENT_DMA_ADAPTER_OP_NEW`` mode
+
+
+RTE_EVENT_DMA_ADAPTER_OP_FORWARD mode
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+In the ``RTE_EVENT_DMA_ADAPTER_OP_FORWARD`` mode, if the event PMD and DMA PMD supports internal
+event port (``RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP_FWD``), the application should use
+``rte_event_dma_adapter_enqueue()`` API to enqueue DMA operations as events to DMA adapter. If
+not, application retrieves DMA adapter's event port using ``rte_event_dma_adapter_event_port_get()``
+API, links its event queue to this port and starts enqueuing DMA operations as events to eventdev
+using ``rte_event_enqueue_burst()``. The adapter then dequeues the events and submits the DMA
+operations to the dmadev. After the DMA operation is complete, the adapter enqueues events to the
+event device.
+
+Applications can use this mode when ingress packet ordering is needed. In this mode, events
+dequeued from the adapter will be treated as forwarded events. Application has to specify
+the dmadev ID and vchan ID (request information) needed to enqueue a DMA operation in
+addition to the event information (response information) needed to enqueue the event after
+the DMA operation has completed.
+
+.. _figure_event_dma_adapter_op_forward:
+
+.. figure:: img/event_dma_adapter_op_forward.*
+
+ Working model of ``RTE_EVENT_DMA_ADAPTER_OP_FORWARD`` mode
+
+
+API Overview
+------------
+
+This section has a brief introduction to the event DMA adapter APIs. The application is expected
+to create an adapter which is associated with a single eventdev, then add dmadev and vchan to the
+adapter instance.
+
+
+Create an adapter instance
+~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+An adapter instance is created using ``rte_event_dma_adapter_create()``. This function is called
+with event device to be associated with the adapter and port configuration for the adapter to
+setup an event port (if the adapter needs to use a service function).
+
+Adapter can be started in ``RTE_EVENT_DMA_ADAPTER_OP_NEW`` or ``RTE_EVENT_DMA_ADAPTER_OP_FORWARD``
+mode.
+
+.. code-block:: c
+
+ enum rte_event_dma_adapter_mode mode;
+ struct rte_event_dev_info dev_info;
+ struct rte_event_port_conf conf;
+ uint8_t evdev_id;
+ uint8_t dma_id;
+ int ret;
+
+ ret = rte_event_dev_info_get(dma_id, &dev_info);
+
+ conf.new_event_threshold = dev_info.max_num_events;
+ conf.dequeue_depth = dev_info.max_event_port_dequeue_depth;
+ conf.enqueue_depth = dev_info.max_event_port_enqueue_depth;
+ mode = RTE_EVENT_DMA_ADAPTER_OP_FORWARD;
+ ret = rte_event_dma_adapter_create(dma_id, evdev_id, &conf, mode);
+
+
+``rte_event_dma_adapter_create_ext()`` function can be used by the application to have a finer
+control on eventdev port allocation and setup. The ``rte_event_dma_adapter_create_ext()``
+function is passed a callback function. The callback function is invoked if the adapter creates
+a service function and uses an event port for it. The callback is expected to fill the
+``struct rte_event_dma_adapter_conf`` structure passed to it.
+
+In the ``RTE_EVENT_DMA_ADAPTER_OP_FORWARD`` mode, if the event PMD and DMA PMD supports internal
+event port (``RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP_FWD``), events with DMA operations should
+be enqueued to the DMA adapter using ``rte_event_dma_adapter_enqueue()`` API. If not, the event port
+created by the adapter can be retrieved using ``rte_event_dma_adapter_event_port_get()`` API. An
+application can use this event port to link with an event queue, on which it enqueues events
+towards the DMA adapter using ``rte_event_enqueue_burst()``.
+
+.. code-block:: c
+
+ uint8_t dma_id, evdev_id, cdev_id, dma_ev_port_id, app_qid;
+ struct rte_event ev;
+ uint32_t cap;
+ int ret;
+
+ // Fill in event info and update event_ptr with rte_dma_op
+ memset(&ev, 0, sizeof(ev));
+ .
+ .
+ ev.event_ptr = op;
+
+ ret = rte_event_dma_adapter_caps_get(evdev_id, cdev_id, &cap);
+ if (cap & RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP_FWD) {
+ ret = rte_event_dma_adapter_enqueue(evdev_id, app_ev_port_id, ev, nb_events);
+ } else {
+ ret = rte_event_dma_adapter_event_port_get(dma_id, &dma_ev_port_id);
+ ret = rte_event_queue_setup(evdev_id, app_qid, NULL);
+ ret = rte_event_port_link(evdev_id, dma_ev_port_id, &app_qid, NULL, 1);
+ ev.queue_id = app_qid;
+ ret = rte_event_enqueue_burst(evdev_id, app_ev_port_id, ev, nb_events);
+ }
+
+
+Event device configuration for service based adapter
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+When ``rte_event_dma_adapter_create()`` is used for creating adapter instance,
+``rte_event_dev_config::nb_event_ports`` is automatically incremented, and event device is
+reconfigured with additional event port during service initialization. This event device
+reconfigure logic also increments the ``rte_event_dev_config::nb_single_link_event_port_queues``
+parameter if the adapter event port config is of type ``RTE_EVENT_PORT_CFG_SINGLE_LINK``.
+
+Applications using this mode of adapter creation need not configure the event device with
+``rte_event_dev_config::nb_event_ports`` and
+``rte_event_dev_config::nb_single_link_event_port_queues`` parameters required for DMA adapter when
+the adapter is created using the above-mentioned API.
+
+
+Querying adapter capabilities
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+The ``rte_event_dma_adapter_caps_get()`` function allows the application to query the adapter
+capabilities for an eventdev and dmadev combination. This API provides whether dmadev and eventdev
+are connected using internal HW port or not.
+
+.. code-block:: c
+
+ rte_event_dma_adapter_caps_get(dev_id, cdev_id, &cap);
+
+
+Adding vchan queue to the adapter instance
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+dmadev device id and vchan queue are configured using dmadev APIs. For more information
+see :doc:`here <dmadev>`.
+
+.. code-block:: c
+
+ struct rte_dma_vchan_conf vchan_conf;
+ struct rte_dma_conf dev_conf;
+ uint8_t dev_id = 0;
+ uint16_t vchan = 0;
+
+ rte_dma_configure(dev_id, &dev_conf);
+ rte_dma_vchan_setup(dev_id, vhcan, &vchan_conf);
+
+These dmadev id and vchan are added to the instance using the
+``rte_event_dma_adapter_vchan_queue_add()`` API. The same is removed using
+``rte_event_dma_adapter_vchan_queue_del()`` API. If hardware supports
+``RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_QP_EV_BIND`` capability, event information must be passed to
+the add API.
+
+.. code-block:: c
+
+ uint32_t cap;
+ int ret;
+
+ ret = rte_event_dma_adapter_caps_get(evdev_id, dma_dev_id, &cap);
+ if (cap & RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_QP_EV_BIND) {
+ struct rte_event event;
+
+ rte_event_dma_adapter_vchan_queue_add(id, dma_dev_id, vchan, &conf);
+ } else
+ rte_event_dma_adapter_vchan_queue_add(id, dma_dev_id, vchan, NULL);
+
+
+Configuring service function
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+If the adapter uses a service function, the application is required to assign a service core to
+the service function as show below.
+
+.. code-block:: c
+
+ uint32_t service_id;
+
+ if (rte_event_dma_adapter_service_id_get(dma_id, &service_id) == 0)
+ rte_service_map_lcore_set(service_id, CORE_ID);
+
+
+Set event request / response information
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+In the RTE_EVENT_DMA_ADAPTER_OP_FORWARD mode, the application specifies the dmadev ID and
+vchan ID (request information) in addition to the event information (response information)
+needed to enqueue an event after the DMA operation has completed. The request and response
+information are specified in the ``struct rte_event_dma_metadata``.
+
+In the RTE_EVENT_DMA_ADAPTER_OP_NEW mode, the application is required to provide only the response
+information.
+
+
+Start the adapter instance
+~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+The application calls ``rte_event_dma_adapter_start()`` to start the adapter. This function calls
+the start callbacks of the eventdev PMDs for hardware based eventdev-dmadev connections and
+``rte_service_run_state_set()`` to enable the service function if one exists.
+
+.. code-block:: c
+
+ rte_event_dma_adapter_start(id);
+
+.. Note::
+
+ The eventdev to which the event_dma_adapter is connected should be started before calling
+ rte_event_dma_adapter_start().
+
+
+Get adapter statistics
+~~~~~~~~~~~~~~~~~~~~~~
+
+The ``rte_event_dma_adapter_stats_get()`` function reports counters defined in struct
+``rte_event_dma_adapter_stats``. The received packet and enqueued event counts are a sum of the
+counts from the eventdev PMD callbacks if the callback is supported, and the counts maintained by
+the service function, if one exists.
+
+Set/Get adapter runtime configuration parameters
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+The runtime configuration parameters of adapter can be set/get using
+``rte_event_dma_adapter_runtime_params_set()`` and
+``rte_event_dma_adapter_runtime_params_get()`` respectively.
+The parameters that can be set/get are defined in
+``struct rte_event_dma_adapter_runtime_params``.
diff --git a/doc/guides/prog_guide/eventdev.rst b/doc/guides/prog_guide/eventdev.rst
index 2c83176846..ff55115d0d 100644
--- a/doc/guides/prog_guide/eventdev.rst
+++ b/doc/guides/prog_guide/eventdev.rst
@@ -333,7 +333,8 @@ eventdev.
.. Note::
EventDev needs to be started before starting the event producers such
- as event_eth_rx_adapter, event_timer_adapter and event_crypto_adapter.
+ as event_eth_rx_adapter, event_timer_adapter, event_crypto_adapter and
+ event_dma_adapter.
Ingress of New Events
~~~~~~~~~~~~~~~~~~~~~
@@ -445,8 +446,9 @@ using ``rte_event_dev_stop_flush_callback_register()`` function.
.. Note::
The event producers such as ``event_eth_rx_adapter``,
- ``event_timer_adapter`` and ``event_crypto_adapter``
- need to be stopped before stopping the event device.
+ ``event_timer_adapter``, ``event_crypto_adapter`` and
+ ``event_dma_adapter`` need to be stopped before stopping
+ the event device.
Summary
-------
diff --git a/doc/guides/prog_guide/img/event_dma_adapter_op_forward.svg b/doc/guides/prog_guide/img/event_dma_adapter_op_forward.svg
new file mode 100644
index 0000000000..b7fe1fecf2
--- /dev/null
+++ b/doc/guides/prog_guide/img/event_dma_adapter_op_forward.svg
@@ -0,0 +1,1086 @@
+<?xml version="1.0" encoding="UTF-8" standalone="no"?>
+<!-- SPDX-License-Identifier: BSD-3-Clause -->
+<!-- Copyright (c) 2023 Marvell. -->
+<!-- Created with Inkscape (http://www.inkscape.org/) -->
+
+<svg
+ width="720px"
+ height="486px"
+ id="svg13237"
+ version="1.1"
+ inkscape:version="1.2.1 (9c6d41e410, 2022-07-14)"
+ sodipodi:docname="event_dma_adapter_op_forward.svg"
+ xmlns:inkscape="http://www.inkscape.org/namespaces/inkscape"
+ xmlns:sodipodi="http://sodipodi.sourceforge.net/DTD/sodipodi-0.dtd"
+ xmlns="http://www.w3.org/2000/svg"
+ xmlns:svg="http://www.w3.org/2000/svg"
+ xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#"
+ xmlns:cc="http://creativecommons.org/ns#"
+ xmlns:dc="http://purl.org/dc/elements/1.1/"
+ xmlns:v="http://schemas.microsoft.com/visio/2003/SVGExtensions/">
+ <defs
+ id="defs13239">
+ <marker
+ inkscape:stockid="Arrow1Sstart"
+ orient="auto"
+ refY="0.0"
+ refX="0.0"
+ id="Arrow1Sstart"
+ style="overflow:visible">
+ <path
+ id="path8416"
+ d="M 0.0,0.0 L 5.0,-5.0 L -12.5,0.0 L 5.0,5.0 L 0.0,0.0 z "
+ style="fill-rule:evenodd;stroke:#000000;stroke-width:1.0pt"
+ transform="scale(0.2) translate(6,0)" />
+ </marker>
+ <marker
+ inkscape:stockid="Arrow1Send"
+ orient="auto"
+ refY="0.0"
+ refX="0.0"
+ id="Arrow1Send"
+ style="overflow:visible;">
+ <path
+ id="path8419"
+ d="M 0.0,0.0 L 5.0,-5.0 L -12.5,0.0 L 5.0,5.0 L 0.0,0.0 z "
+ style="fill-rule:evenodd;stroke:#000000;stroke-width:1.0pt;"
+ transform="scale(0.2) rotate(180) translate(6,0)" />
+ </marker>
+ <marker
+ inkscape:stockid="DiamondL"
+ orient="auto"
+ refY="0.0"
+ refX="0.0"
+ id="DiamondL"
+ style="overflow:visible">
+ <path
+ id="path8483"
+ d="M 0,-7.0710768 L -7.0710894,0 L 0,7.0710589 L 7.0710462,0 L 0,-7.0710768 z "
+ style="fill-rule:evenodd;stroke:#000000;stroke-width:1.0pt"
+ transform="scale(0.8)" />
+ </marker>
+ <marker
+ inkscape:stockid="DotL"
+ orient="auto"
+ refY="0.0"
+ refX="0.0"
+ id="DotL"
+ style="overflow:visible">
+ <path
+ id="path8465"
+ d="M -2.5,-1.0 C -2.5,1.7600000 -4.7400000,4.0 -7.5,4.0 C -10.260000,4.0 -12.5,1.7600000 -12.5,-1.0 C -12.5,-3.7600000 -10.260000,-6.0 -7.5,-6.0 C -4.7400000,-6.0 -2.5,-3.7600000 -2.5,-1.0 z "
+ style="fill-rule:evenodd;stroke:#000000;stroke-width:1.0pt"
+ transform="scale(0.8) translate(7.4, 1)" />
+ </marker>
+ <marker
+ inkscape:stockid="SquareL"
+ orient="auto"
+ refY="0.0"
+ refX="0.0"
+ id="SquareL"
+ style="overflow:visible">
+ <path
+ id="path8474"
+ d="M -5.0,-5.0 L -5.0,5.0 L 5.0,5.0 L 5.0,-5.0 L -5.0,-5.0 z "
+ style="fill-rule:evenodd;stroke:#000000;stroke-width:1.0pt"
+ transform="scale(0.8)" />
+ </marker>
+ <marker
+ inkscape:stockid="TriangleOutL"
+ orient="auto"
+ refY="0.0"
+ refX="0.0"
+ id="TriangleOutL"
+ style="overflow:visible">
+ <path
+ id="path8546"
+ d="M 5.77,0.0 L -2.88,5.0 L -2.88,-5.0 L 5.77,0.0 z "
+ style="fill-rule:evenodd;stroke:#000000;stroke-width:1.0pt"
+ transform="scale(0.8)" />
+ </marker>
+ <marker
+ inkscape:stockid="Arrow1Lstart"
+ orient="auto"
+ refY="0.0"
+ refX="0.0"
+ id="Arrow1Lstart"
+ style="overflow:visible">
+ <path
+ id="path8404"
+ d="M 0.0,0.0 L 5.0,-5.0 L -12.5,0.0 L 5.0,5.0 L 0.0,0.0 z "
+ style="fill-rule:evenodd;stroke:#000000;stroke-width:1.0pt"
+ transform="scale(0.8) translate(12.5,0)" />
+ </marker>
+ <marker
+ inkscape:stockid="Arrow1Mend"
+ orient="auto"
+ refY="0.0"
+ refX="0.0"
+ id="Arrow1Mend"
+ style="overflow:visible;">
+ <path
+ id="path8413"
+ d="M 0.0,0.0 L 5.0,-5.0 L -12.5,0.0 L 5.0,5.0 L 0.0,0.0 z "
+ style="fill-rule:evenodd;stroke:#000000;stroke-width:1.0pt;"
+ transform="scale(0.4) rotate(180) translate(10,0)" />
+ </marker>
+ <marker
+ inkscape:stockid="Arrow2Lend"
+ orient="auto"
+ refY="0.0"
+ refX="0.0"
+ id="Arrow2Lend"
+ style="overflow:visible;">
+ <path
+ id="path8425"
+ style="fill-rule:evenodd;stroke-width:0.62500000;stroke-linejoin:round;"
+ d="M 8.7185878,4.0337352 L -2.2072895,0.016013256 L 8.7185884,-4.0017078 C 6.9730900,-1.6296469 6.9831476,1.6157441 8.7185878,4.0337352 z "
+ transform="scale(1.1) rotate(180) translate(1,0)" />
+ </marker>
+ <marker
+ inkscape:stockid="Arrow1Lend"
+ orient="auto"
+ refY="0.0"
+ refX="0.0"
+ id="Arrow1Lend"
+ style="overflow:visible;">
+ <path
+ id="path8407"
+ d="M 0.0,0.0 L 5.0,-5.0 L -12.5,0.0 L 5.0,5.0 L 0.0,0.0 z "
+ style="fill-rule:evenodd;stroke:#000000;stroke-width:1.0pt;"
+ transform="scale(0.8) rotate(180) translate(12.5,0)" />
+ </marker>
+ <filter
+ id="filter_2"
+ color-interpolation-filters="sRGB">
+ <feGaussianBlur
+ stdDeviation="2"
+ id="feGaussianBlur15" />
+ </filter>
+ <filter
+ id="filter_2-3"
+ color-interpolation-filters="sRGB"
+ x="-0.086178862"
+ y="-0.086178862"
+ width="1.1723577"
+ height="1.1723577">
+ <feGaussianBlur
+ stdDeviation="2"
+ id="feGaussianBlur15-1" />
+ </filter>
+ <filter
+ id="filter_2-0"
+ color-interpolation-filters="sRGB">
+ <feGaussianBlur
+ stdDeviation="2"
+ id="feGaussianBlur15-7" />
+ </filter>
+ <filter
+ id="filter_2-0-8"
+ color-interpolation-filters="sRGB">
+ <feGaussianBlur
+ stdDeviation="2"
+ id="feGaussianBlur15-7-7" />
+ </filter>
+ <filter
+ id="filter_2-3-9"
+ color-interpolation-filters="sRGB">
+ <feGaussianBlur
+ stdDeviation="2"
+ id="feGaussianBlur15-1-6" />
+ </filter>
+ <filter
+ id="filter_2-3-6"
+ color-interpolation-filters="sRGB">
+ <feGaussianBlur
+ stdDeviation="2"
+ id="feGaussianBlur15-1-63" />
+ </filter>
+ <filter
+ id="filter_2-3-91"
+ color-interpolation-filters="sRGB"
+ x="-0.086178862"
+ y="-0.086178862"
+ width="1.1723577"
+ height="1.1723577">
+ <feGaussianBlur
+ stdDeviation="2"
+ id="feGaussianBlur15-1-3" />
+ </filter>
+ <marker
+ inkscape:stockid="Arrow1Lend"
+ orient="auto"
+ refY="0"
+ refX="0"
+ id="Arrow1Lend-5"
+ style="overflow:visible">
+ <path
+ inkscape:connector-curvature="0"
+ id="path8407-3"
+ d="M 0,0 5,-5 -12.5,0 5,5 0,0 z"
+ style="fill-rule:evenodd;stroke:#000000;stroke-width:1pt"
+ transform="matrix(-0.8,0,0,-0.8,-10,0)" />
+ </marker>
+ <marker
+ inkscape:stockid="Arrow1Lend"
+ orient="auto"
+ refY="0"
+ refX="0"
+ id="Arrow1Lend-6"
+ style="overflow:visible">
+ <path
+ inkscape:connector-curvature="0"
+ id="path8407-0"
+ d="M 0,0 5,-5 -12.5,0 5,5 0,0 z"
+ style="fill-rule:evenodd;stroke:#000000;stroke-width:1pt"
+ transform="matrix(-0.8,0,0,-0.8,-10,0)" />
+ </marker>
+ <marker
+ inkscape:stockid="Arrow1Lstart"
+ orient="auto"
+ refY="0"
+ refX="0"
+ id="Arrow1Lstart-7"
+ style="overflow:visible">
+ <path
+ inkscape:connector-curvature="0"
+ id="path8404-0"
+ d="M 0,0 5,-5 -12.5,0 5,5 0,0 z"
+ style="fill-rule:evenodd;stroke:#000000;stroke-width:1pt"
+ transform="matrix(0.8,0,0,0.8,10,0)" />
+ </marker>
+ <marker
+ inkscape:stockid="Arrow1Lend"
+ orient="auto"
+ refY="0"
+ refX="0"
+ id="Arrow1Lend-51"
+ style="overflow:visible">
+ <path
+ inkscape:connector-curvature="0"
+ id="path8407-1"
+ d="M 0,0 5,-5 -12.5,0 5,5 0,0 z"
+ style="fill-rule:evenodd;stroke:#000000;stroke-width:1pt"
+ transform="matrix(-0.8,0,0,-0.8,-10,0)" />
+ </marker>
+ <marker
+ inkscape:stockid="Arrow1Lend"
+ orient="auto"
+ refY="0"
+ refX="0"
+ id="Arrow1Lend-3"
+ style="overflow:visible">
+ <path
+ inkscape:connector-curvature="0"
+ id="path8407-6"
+ d="M 0,0 5,-5 -12.5,0 5,5 0,0 z"
+ style="fill-rule:evenodd;stroke:#000000;stroke-width:1pt"
+ transform="matrix(-0.8,0,0,-0.8,-10,0)" />
+ </marker>
+ <marker
+ inkscape:stockid="Arrow1Lend"
+ orient="auto"
+ refY="0"
+ refX="0"
+ id="Arrow1Lend-62"
+ style="overflow:visible">
+ <path
+ inkscape:connector-curvature="0"
+ id="path8407-9"
+ d="M 0,0 5,-5 -12.5,0 5,5 0,0 z"
+ style="fill-rule:evenodd;stroke:#000000;stroke-width:1pt"
+ transform="matrix(-0.8,0,0,-0.8,-10,0)" />
+ </marker>
+ <marker
+ inkscape:stockid="Arrow1Lend"
+ orient="auto"
+ refY="0"
+ refX="0"
+ id="Arrow1Lend-2"
+ style="overflow:visible">
+ <path
+ inkscape:connector-curvature="0"
+ id="path8407-7"
+ d="M 0,0 5,-5 -12.5,0 5,5 0,0 z"
+ style="fill-rule:evenodd;stroke:#000000;stroke-width:1pt"
+ transform="matrix(-0.8,0,0,-0.8,-10,0)" />
+ </marker>
+ <marker
+ inkscape:stockid="Arrow1Lstart"
+ orient="auto"
+ refY="0"
+ refX="0"
+ id="Arrow1Lstart-7-9"
+ style="overflow:visible">
+ <path
+ inkscape:connector-curvature="0"
+ id="path8404-0-3"
+ d="M 0,0 5,-5 -12.5,0 5,5 0,0 z"
+ style="fill-rule:evenodd;stroke:#000000;stroke-width:1pt"
+ transform="matrix(0.8,0,0,0.8,10,0)" />
+ </marker>
+ <filter
+ id="filter_2-3-6-1"
+ color-interpolation-filters="sRGB">
+ <feGaussianBlur
+ stdDeviation="2"
+ id="feGaussianBlur15-1-63-8" />
+ </filter>
+ <filter
+ id="filter_2-3-92"
+ color-interpolation-filters="sRGB"
+ x="-0.086178862"
+ y="-0.086178862"
+ width="1.1723577"
+ height="1.1723577">
+ <feGaussianBlur
+ stdDeviation="2"
+ id="feGaussianBlur15-1-2" />
+ </filter>
+ <filter
+ id="filter_2-3-94"
+ color-interpolation-filters="sRGB"
+ x="-0.086178862"
+ y="-0.086178862"
+ width="1.1723577"
+ height="1.1723577">
+ <feGaussianBlur
+ stdDeviation="2"
+ id="feGaussianBlur15-1-7" />
+ </filter>
+ <marker
+ inkscape:stockid="Arrow1Lstart"
+ orient="auto"
+ refY="0"
+ refX="0"
+ id="Arrow1Lstart-7-6"
+ style="overflow:visible">
+ <path
+ inkscape:connector-curvature="0"
+ id="path8404-0-1"
+ d="M 0,0 5,-5 -12.5,0 5,5 0,0 z"
+ style="fill-rule:evenodd;stroke:#000000;stroke-width:1pt"
+ transform="matrix(0.8,0,0,0.8,10,0)" />
+ </marker>
+ <marker
+ inkscape:stockid="Arrow1Lend"
+ orient="auto"
+ refY="0"
+ refX="0"
+ id="Arrow1Lend-55"
+ style="overflow:visible">
+ <path
+ inkscape:connector-curvature="0"
+ id="path8407-4"
+ d="M 0,0 5,-5 -12.5,0 5,5 0,0 z"
+ style="fill-rule:evenodd;stroke:#000000;stroke-width:1pt"
+ transform="matrix(-0.8,0,0,-0.8,-10,0)" />
+ </marker>
+ </defs>
+ <sodipodi:namedview
+ id="base"
+ pagecolor="#ffffff"
+ bordercolor="#666666"
+ borderopacity="1.0"
+ inkscape:pageopacity="0.0"
+ inkscape:pageshadow="2"
+ inkscape:zoom="1"
+ inkscape:cx="233.5"
+ inkscape:cy="288"
+ inkscape:document-units="px"
+ inkscape:current-layer="layer1"
+ showgrid="false"
+ inkscape:window-width="1920"
+ inkscape:window-height="1017"
+ inkscape:window-x="-8"
+ inkscape:window-y="-8"
+ inkscape:window-maximized="1"
+ inkscape:snap-nodes="false"
+ inkscape:showpageshadow="2"
+ inkscape:pagecheckerboard="0"
+ inkscape:deskcolor="#d1d1d1">
+ <inkscape:grid
+ type="xygrid"
+ id="grid13454" />
+ </sodipodi:namedview>
+ <metadata
+ id="metadata13242">
+ <rdf:RDF>
+ <cc:Work
+ rdf:about="">
+ <dc:format>image/svg+xml</dc:format>
+ <dc:type
+ rdf:resource="http://purl.org/dc/dcmitype/StillImage" />
+ <dc:title />
+ </cc:Work>
+ </rdf:RDF>
+ </metadata>
+ <g
+ id="layer1"
+ inkscape:label="Layer 1"
+ inkscape:groupmode="layer">
+ <g
+ style="font-size:12px;fill:none;stroke-linecap:square;stroke-miterlimit:3;overflow:visible"
+ id="shape1-1-2-4"
+ v:mID="1"
+ v:groupContext="shape"
+ transform="matrix(2.1604167,0,0,1.5671361,88.874699,-812.39909)">
+ <title
+ id="title22-7-5">Square</title>
+ <desc
+ id="desc24-7-8">Atomic Queue #1</desc>
+ <v:userDefs>
+ <v:ud
+ v:nameU="visVersion"
+ v:val="VT0(15):26" />
+ </v:userDefs>
+ <v:textBlock
+ v:margins="rect(4,4,4,4)" />
+ <v:textRect
+ cx="30.75"
+ cy="581.25"
+ width="61.5"
+ height="61.5" />
+ <g
+ id="shadow1-2-9-5"
+ v:groupContext="shadow"
+ v:shadowOffsetX="0.345598"
+ v:shadowOffsetY="-1.97279"
+ v:shadowType="1"
+ transform="translate(0.345598,1.97279)"
+ class="st1"
+ style="visibility:visible">
+ <rect
+ x="0"
+ y="550.5"
+ width="61.5"
+ height="61.5"
+ class="st2"
+ id="rect27-8-7"
+ style="fill:#5b9bd5;fill-opacity:0.22000002;stroke:#5b9bd5;stroke-opacity:0.22000002;filter:url(#filter_2-3-91)" />
+ </g>
+ <g
+ id="g13515-33">
+ <g
+ id="g13534-8">
+ <rect
+ x="0"
+ y="550.5"
+ width="61.5"
+ height="61.5"
+ class="st3"
+ id="rect29-1-95"
+ style="fill:#5b9bd5;stroke:#c7c8c8;stroke-width:0.25" />
+ </g>
+ </g>
+ </g>
+ <path
+ style="fill:none;stroke:#000000;stroke-width:0.712265;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:4;stroke-dasharray:none;stroke-opacity:1;marker-start:url(#Arrow1Lstart-7);marker-end:none"
+ d="M 312.28671,240.74335 H 227.99897"
+ id="path17209"
+ inkscape:connector-type="orthogonal"
+ inkscape:connector-curvature="0" />
+ <path
+ style="fill:none;stroke:#000000;stroke-width:0.718986px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1;marker-mid:none;marker-end:url(#Arrow1Lend)"
+ d="m 221.6484,77.57125 h 94.28101"
+ id="path17209-8"
+ inkscape:connector-type="orthogonal"
+ inkscape:connector-curvature="0" />
+ <g
+ style="font-size:12px;fill:none;stroke-linecap:square;stroke-miterlimit:3;overflow:visible"
+ id="shape1-1-2"
+ v:mID="1"
+ v:groupContext="shape"
+ transform="matrix(2.1604167,0,0,1.5671361,314.24227,-811.89589)">
+ <title
+ id="title22-7">Square</title>
+ <desc
+ id="desc24-7">Atomic Queue #1</desc>
+ <v:userDefs>
+ <v:ud
+ v:nameU="visVersion"
+ v:val="VT0(15):26" />
+ </v:userDefs>
+ <v:textBlock
+ v:margins="rect(4,4,4,4)" />
+ <v:textRect
+ cx="30.75"
+ cy="581.25"
+ width="61.5"
+ height="61.5" />
+ <g
+ id="shadow1-2-9"
+ v:groupContext="shadow"
+ v:shadowOffsetX="0.345598"
+ v:shadowOffsetY="-1.97279"
+ v:shadowType="1"
+ transform="translate(0.345598,1.97279)"
+ class="st1"
+ style="visibility:visible">
+ <rect
+ x="0"
+ y="550.5"
+ width="61.5"
+ height="61.5"
+ class="st2"
+ id="rect27-8"
+ style="fill:#5b9bd5;fill-opacity:0.22000002;stroke:#5b9bd5;stroke-opacity:0.22000002;filter:url(#filter_2-3)" />
+ </g>
+ <g
+ id="g13515">
+ <g
+ id="g13534">
+ <rect
+ x="0"
+ y="550.5"
+ width="61.5"
+ height="61.5"
+ class="st3"
+ id="rect29-1"
+ style="fill:#5b9bd5;stroke:#c7c8c8;stroke-width:0.25" />
+ </g>
+ </g>
+ </g>
+ <path
+ style="fill:none;stroke:#000000;stroke-width:0.724714;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:4;stroke-dasharray:none;stroke-opacity:1;marker-start:url(#Arrow1Lstart);marker-end:none"
+ d="M 89.025329,74.39932 H 24.750043"
+ id="path17209-3"
+ inkscape:connector-type="orthogonal"
+ inkscape:connector-curvature="0" />
+ <path
+ transform="matrix(0.73232502,0,0,0.75477602,-4.325033,28.642983)"
+ sodipodi:type="arc"
+ style="fill:#539de6;fill-opacity:1;stroke:#0000ea;stroke-width:1;stroke-linecap:square;stroke-miterlimit:4;stroke-opacity:1;stroke-dasharray:none"
+ id="path29161-3"
+ sodipodi:cx="371"
+ sodipodi:cy="64.5"
+ sodipodi:rx="17"
+ sodipodi:ry="15.5"
+ d="m 388,64.5 a 17,15.5 0 1 1 -34,0 17,15.5 0 1 1 34,0 z" />
+ <path
+ transform="matrix(0.73232502,0,0,0.75477602,-1.93108,192.80833)"
+ sodipodi:type="arc"
+ style="fill:#539de6;fill-opacity:1;stroke:#0000ea;stroke-width:1;stroke-linecap:square;stroke-miterlimit:4;stroke-opacity:1;stroke-dasharray:none"
+ id="path29161-1"
+ sodipodi:cx="371"
+ sodipodi:cy="64.5"
+ sodipodi:rx="17"
+ sodipodi:ry="15.5"
+ d="m 388,64.5 a 17,15.5 0 1 1 -34,0 17,15.5 0 1 1 34,0 z" />
+ <path
+ style="fill:none;stroke:#000000;stroke-width:0.751412;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:4;stroke-dasharray:none;stroke-opacity:1;marker-start:url(#Arrow1Lstart-7);marker-end:none"
+ d="M 18.763392,120.7432 H 87.758545"
+ id="path17209-3-0"
+ inkscape:connector-type="orthogonal"
+ inkscape:connector-curvature="0" />
+ <path
+ sodipodi:type="arc"
+ style="fill:#539de6;fill-opacity:1;stroke:#0000ea;stroke-width:1;stroke-linecap:square;stroke-miterlimit:4;stroke-opacity:1;stroke-dasharray:none"
+ id="path29161"
+ sodipodi:cx="371"
+ sodipodi:cy="64.5"
+ sodipodi:rx="17"
+ sodipodi:ry="15.5"
+ d="m 388,64.5 a 17,15.5 0 1 1 -34,0 17,15.5 0 1 1 34,0 z"
+ transform="matrix(0.73232502,0,0,0.75477602,-218.16394,72.68276)" />
+ <path
+ sodipodi:type="arc"
+ style="fill:#539de6;fill-opacity:1;stroke:#0000ea;stroke-width:1;stroke-linecap:square;stroke-miterlimit:4;stroke-opacity:1;stroke-dasharray:none"
+ id="path29161-2"
+ sodipodi:cx="371"
+ sodipodi:cy="64.5"
+ sodipodi:rx="17"
+ sodipodi:ry="15.5"
+ d="m 388,64.5 a 17,15.5 0 1 1 -34,0 17,15.5 0 1 1 34,0 z"
+ transform="matrix(0.73232502,0,0,0.75477602,-217.40136,26.716271)" />
+ <g
+ id="g29167-4"
+ transform="matrix(0.73232502,0,0,0.75477602,-217.31662,28.007562)">
+ <text
+ id="text29163-9"
+ y="70"
+ x="365"
+ style="font-style:normal;font-weight:normal;line-height:0%;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none"
+ xml:space="preserve"><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="70"
+ x="365"
+ id="tspan29165-9"
+ sodipodi:role="line">1</tspan></text>
+ </g>
+ <g
+ id="g29167-9"
+ transform="matrix(0.73232502,0,0,0.75477602,-4.9726112,28.689051)">
+ <text
+ id="text29163-3"
+ y="70"
+ x="365"
+ style="font-style:normal;font-weight:normal;line-height:0%;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none"
+ xml:space="preserve"><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="70"
+ x="365"
+ id="tspan29165-3"
+ sodipodi:role="line">2</tspan></text>
+ </g>
+ <path
+ style="fill:none;stroke:#000000;stroke-width:0.678033px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1;marker-start:url(#Arrow1Lstart-7);marker-end:none"
+ d="M 181,214.66098 V 145.33902"
+ id="path17211-7-1-6"
+ inkscape:connector-type="orthogonal"
+ inkscape:connector-curvature="0" />
+ <g
+ id="g29167"
+ transform="matrix(0.73232502,0,0,0.75477602,-218.07919,73.10621)">
+ <text
+ id="text29163"
+ y="70"
+ x="365"
+ style="font-style:normal;font-weight:normal;line-height:0%;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none"
+ xml:space="preserve"><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="70"
+ x="365"
+ id="tspan29165"
+ sodipodi:role="line">8</tspan></text>
+ </g>
+ <path
+ style="fill:none;stroke:#000000;stroke-width:0.678033px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1;marker-start:url(#Arrow1Lstart-7);marker-end:none"
+ d="m 131,145.8531 v 69.32197"
+ id="path17211-7-1"
+ inkscape:connector-type="orthogonal"
+ inkscape:connector-curvature="0" />
+ <path
+ transform="matrix(0.73232502,0,0,0.75477602,-140.37076,129.97088)"
+ sodipodi:type="arc"
+ style="fill:#539de6;fill-opacity:1;stroke:#0000ea;stroke-width:1;stroke-linecap:square;stroke-miterlimit:4;stroke-opacity:1;stroke-dasharray:none"
+ id="path29161-8"
+ sodipodi:cx="371"
+ sodipodi:cy="64.5"
+ sodipodi:rx="17"
+ sodipodi:ry="15.5"
+ d="m 388,64.5 a 17,15.5 0 1 1 -34,0 17,15.5 0 1 1 34,0 z" />
+ <g
+ id="g29167-2"
+ transform="matrix(0.73232502,0,0,0.75477602,-140.28602,131.01695)">
+ <text
+ id="text29163-92"
+ y="70"
+ x="365"
+ style="font-style:normal;font-weight:normal;line-height:0%;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none"
+ xml:space="preserve"><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="70"
+ x="365"
+ id="tspan29165-8"
+ sodipodi:role="line">7</tspan></text>
+ </g>
+ <path
+ style="fill:none;stroke:#000000;stroke-width:0.718986px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1;marker-mid:none;marker-end:url(#Arrow1Lend)"
+ d="m 317.1405,116 h -94.281"
+ id="path17209-8-0"
+ inkscape:connector-type="orthogonal"
+ inkscape:connector-curvature="0" />
+ <path
+ transform="matrix(0.73232502,0,0,0.75477602,-3.4914,66.68745)"
+ sodipodi:type="arc"
+ style="fill:#539de6;fill-opacity:1;stroke:#0000ea;stroke-width:1;stroke-linecap:square;stroke-miterlimit:4;stroke-opacity:1;stroke-dasharray:none"
+ id="path29161-6"
+ sodipodi:cx="371"
+ sodipodi:cy="64.5"
+ sodipodi:rx="17"
+ sodipodi:ry="15.5"
+ d="m 388,64.5 a 17,15.5 0 1 1 -34,0 17,15.5 0 1 1 34,0 z" />
+ <g
+ id="g29167-46"
+ transform="matrix(0.73232502,0,0,0.75477602,-4.40666,67.48829)">
+ <text
+ id="text29163-1"
+ y="70"
+ x="365"
+ style="font-style:normal;font-weight:normal;line-height:0%;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none"
+ xml:space="preserve"><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="70"
+ x="365"
+ id="tspan29165-5"
+ sodipodi:role="line">3</tspan></text>
+ </g>
+ <path
+ transform="matrix(0.73232502,0,0,0.75477602,-90.692582,130.31695)"
+ sodipodi:type="arc"
+ style="fill:#539de6;fill-opacity:1;stroke:#0000ea;stroke-width:1;stroke-linecap:square;stroke-miterlimit:4;stroke-opacity:1;stroke-dasharray:none"
+ id="path29161-8-6"
+ sodipodi:cx="371"
+ sodipodi:cy="64.5"
+ sodipodi:rx="17"
+ sodipodi:ry="15.5"
+ d="m 388,64.5 a 17,15.5 0 1 1 -34,0 17,15.5 0 1 1 34,0 z" />
+ <g
+ id="g29167-6"
+ transform="matrix(0.73232502,0,0,0.75477602,-90.84634,131.60918)">
+ <text
+ id="text29163-17"
+ y="70"
+ x="365"
+ style="font-style:normal;font-weight:normal;line-height:0%;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none"
+ xml:space="preserve"><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="70"
+ x="365"
+ id="tspan29165-2"
+ sodipodi:role="line">4</tspan></text>
+ </g>
+ <g
+ id="g29167-2-0"
+ transform="matrix(0.73232502,0,0,0.75477602,-2.424397,194.0216)">
+ <text
+ id="text29163-92-6"
+ y="70"
+ x="365"
+ style="font-style:normal;font-weight:normal;line-height:0%;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none"
+ xml:space="preserve"><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="70"
+ x="365"
+ id="tspan29165-8-2"
+ sodipodi:role="line">5</tspan></text>
+ </g>
+ <g
+ style="font-size:12px;fill:none;stroke-linecap:square;stroke-miterlimit:3;overflow:visible"
+ id="shape1-1-2-8"
+ v:mID="1"
+ v:groupContext="shape"
+ transform="matrix(2.1604167,0,0,1.5671361,93.82055,-648.98949)">
+ <title
+ id="title22-7-97">Square</title>
+ <desc
+ id="desc24-7-3">Atomic Queue #1</desc>
+ <v:userDefs>
+ <v:ud
+ v:nameU="visVersion"
+ v:val="VT0(15):26" />
+ </v:userDefs>
+ <v:textBlock
+ v:margins="rect(4,4,4,4)" />
+ <v:textRect
+ cx="30.75"
+ cy="581.25"
+ width="61.5"
+ height="61.5" />
+ <g
+ id="shadow1-2-9-6"
+ v:groupContext="shadow"
+ v:shadowOffsetX="0.345598"
+ v:shadowOffsetY="-1.97279"
+ v:shadowType="1"
+ transform="translate(0.345598,1.97279)"
+ class="st1"
+ style="visibility:visible">
+ <rect
+ x="0"
+ y="550.5"
+ width="61.5"
+ height="61.5"
+ class="st2"
+ id="rect27-8-12"
+ style="fill:#5b9bd5;fill-opacity:0.22000002;stroke:#5b9bd5;stroke-opacity:0.22000002;filter:url(#filter_2-3-92)" />
+ </g>
+ <g
+ id="g13515-9">
+ <g
+ id="g13534-3">
+ <rect
+ x="0"
+ y="550.5"
+ width="61.5"
+ height="61.5"
+ class="st3"
+ id="rect29-1-1"
+ style="fill:#5b9bd5;stroke:#c7c8c8;stroke-width:0.25" />
+ </g>
+ </g>
+ </g>
+ <g
+ style="font-size:12px;fill:none;stroke-linecap:square;stroke-miterlimit:3;overflow:visible"
+ id="shape1-1-2-84"
+ v:mID="1"
+ v:groupContext="shape"
+ transform="matrix(2.1604167,0,0,1.5671361,314.82055,-648.98949)">
+ <title
+ id="title22-7-50">Square</title>
+ <desc
+ id="desc24-7-36">Atomic Queue #1</desc>
+ <v:userDefs>
+ <v:ud
+ v:nameU="visVersion"
+ v:val="VT0(15):26" />
+ </v:userDefs>
+ <v:textBlock
+ v:margins="rect(4,4,4,4)" />
+ <v:textRect
+ cx="30.75"
+ cy="581.25"
+ width="61.5"
+ height="61.5" />
+ <g
+ id="shadow1-2-9-1"
+ v:groupContext="shadow"
+ v:shadowOffsetX="0.345598"
+ v:shadowOffsetY="-1.97279"
+ v:shadowType="1"
+ transform="translate(0.345598,1.97279)"
+ class="st1"
+ style="visibility:visible">
+ <rect
+ x="0"
+ y="550.5"
+ width="61.5"
+ height="61.5"
+ class="st2"
+ id="rect27-8-0"
+ style="fill:#5b9bd5;fill-opacity:0.22000002;stroke:#5b9bd5;stroke-opacity:0.22000002;filter:url(#filter_2-3-94)" />
+ </g>
+ <g
+ id="g13515-6">
+ <g
+ id="g13534-32">
+ <rect
+ x="0"
+ y="550.5"
+ width="61.5"
+ height="61.5"
+ class="st3"
+ id="rect29-1-0"
+ style="fill:#5b9bd5;stroke:#c7c8c8;stroke-width:0.25" />
+ </g>
+ </g>
+ </g>
+ <path
+ style="fill:none;stroke:#000000;stroke-width:0.712265;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:4;stroke-dasharray:none;stroke-opacity:1;marker-start:none;marker-end:url(#Arrow1Lend)"
+ d="M 313.14387,285 H 228.85613"
+ id="path17209-7"
+ inkscape:connector-type="orthogonal"
+ inkscape:connector-curvature="0" />
+ <path
+ transform="matrix(0.73232502,0,0,0.75477602,-2.692582,236.31695)"
+ sodipodi:type="arc"
+ style="fill:#539de6;fill-opacity:1;stroke:#0000ea;stroke-width:1;stroke-linecap:square;stroke-miterlimit:4;stroke-opacity:1;stroke-dasharray:none"
+ id="path29161-1-6"
+ sodipodi:cx="371"
+ sodipodi:cy="64.5"
+ sodipodi:rx="17"
+ sodipodi:ry="15.5"
+ d="m 388,64.5 a 17,15.5 0 1 1 -34,0 17,15.5 0 1 1 34,0 z" />
+ <g
+ id="g29167-2-0-5"
+ transform="matrix(0.73232502,0,0,0.75477602,-2.424397,237.0216)">
+ <text
+ id="text29163-92-6-6"
+ y="70"
+ x="365"
+ style="font-style:normal;font-weight:normal;line-height:0%;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none"
+ xml:space="preserve"><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="70"
+ x="365"
+ id="tspan29165-8-2-9"
+ sodipodi:role="line">6</tspan></text>
+ </g>
+ <g
+ id="g29167-4-3"
+ transform="matrix(0.73232502,0,0,0.75477602,-154.60784,51.117791)">
+ <text
+ id="text29163-9-6"
+ y="70"
+ x="365"
+ style="font-style:normal;font-weight:normal;line-height:0%;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none"
+ xml:space="preserve"><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:24.2109px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="70"
+ x="365"
+ id="tspan29165-9-7"
+ sodipodi:role="line">Eventdev</tspan></text>
+ </g>
+ <g
+ id="g29167-4-3-5"
+ transform="matrix(0.73232502,0,0,0.75477602,-144.65044,201.97821)">
+ <text
+ id="text29163-9-6-3"
+ y="70"
+ x="412.93716"
+ style="font-style:normal;font-weight:normal;line-height:0%;font-family:sans-serif;text-align:center;letter-spacing:0px;word-spacing:0px;text-anchor:middle;fill:#000000;fill-opacity:1;stroke:none"
+ xml:space="preserve"><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:24.2109px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:center;writing-mode:lr-tb;text-anchor:middle"
+ y="70"
+ x="412.93716"
+ id="tspan29165-9-7-5"
+ sodipodi:role="line">DMA</tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:24.2109px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:center;writing-mode:lr-tb;text-anchor:middle"
+ y="100.26363"
+ x="412.93716"
+ sodipodi:role="line"
+ id="tspan3201">Adapter</tspan></text>
+ </g>
+ <g
+ id="g29167-4-3-5-6"
+ transform="matrix(0.73232502,0,0,0.75477602,79.53518,46.62529)">
+ <text
+ id="text29163-9-6-3-2"
+ y="48.801659"
+ x="412.93716"
+ style="font-style:normal;font-weight:normal;line-height:0%;font-family:sans-serif;text-align:center;letter-spacing:0px;word-spacing:0px;text-anchor:middle;fill:#000000;fill-opacity:1;stroke:none"
+ xml:space="preserve"><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:24.2109px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:center;writing-mode:lr-tb;text-anchor:middle"
+ y="48.801659"
+ x="412.93716"
+ sodipodi:role="line"
+ id="tspan3155">Application</tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:24.2109px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:center;writing-mode:lr-tb;text-anchor:middle"
+ y="98.801659"
+ x="412.93716"
+ sodipodi:role="line"
+ id="tspan3201-1">in ordered</tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:24.2109px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:center;writing-mode:lr-tb;text-anchor:middle"
+ y="148.80167"
+ x="412.93716"
+ sodipodi:role="line"
+ id="tspan3161">stage</tspan></text>
+ </g>
+ <g
+ id="g29167-4-3-5-2"
+ transform="matrix(0.73232502,0,0,0.75477602,77.535182,213.62529)">
+ <text
+ id="text29163-9-6-3-7"
+ y="70"
+ x="412.93716"
+ style="font-style:normal;font-weight:normal;line-height:0%;font-family:sans-serif;text-align:center;letter-spacing:0px;word-spacing:0px;text-anchor:middle;fill:#000000;fill-opacity:1;stroke:none"
+ xml:space="preserve"><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:24.2109px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:center;writing-mode:lr-tb;text-anchor:middle"
+ y="70"
+ x="412.93716"
+ sodipodi:role="line"
+ id="tspan3201-9">DMA Device</tspan></text>
+ </g>
+ <g
+ id="g29167-4-3-5-3"
+ transform="matrix(0.73232502,0,0,0.75477602,188.53518,-3.37471)">
+ <text
+ id="text29163-9-6-3-6"
+ y="70"
+ x="375.65271"
+ style="font-style:normal;font-weight:normal;line-height:0%;font-family:sans-serif;text-align:center;letter-spacing:0px;word-spacing:0px;text-anchor:middle;fill:#000000;fill-opacity:1;stroke:none"
+ xml:space="preserve"><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18.8307px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="70"
+ x="375.65271"
+ sodipodi:role="line"
+ id="tspan3201-6">1. Events from the previous stage.</tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18.8307px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="93.538376"
+ x="375.65271"
+ sodipodi:role="line"
+ id="tspan3260"> </tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18.8307px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="117.07675"
+ x="375.65271"
+ sodipodi:role="line"
+ id="tspan3262">2. Application in ordered stage</tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18.8307px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="140.61513"
+ x="375.65271"
+ sodipodi:role="line"
+ id="tspan3288"> dequeues events from eventdev.</tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18.8307px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="164.1535"
+ x="375.65271"
+ sodipodi:role="line"
+ id="tspan3264"> </tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18.8307px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="187.69188"
+ x="375.65271"
+ sodipodi:role="line"
+ id="tspan3266">3. Application enqueues DMA</tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18.8307px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="211.23026"
+ x="375.65271"
+ sodipodi:role="line"
+ id="tspan3290"> operations as events to eventdev.</tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18.8307px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="234.76863"
+ x="375.65271"
+ sodipodi:role="line"
+ id="tspan3268"> </tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18.8307px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="258.30701"
+ x="375.65271"
+ sodipodi:role="line"
+ id="tspan3270">4. DMA adapter dequeues event</tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18.8307px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="281.84537"
+ x="375.65271"
+ sodipodi:role="line"
+ id="tspan3292"> from eventdev.</tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18.8307px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="305.38376"
+ x="375.65271"
+ sodipodi:role="line"
+ id="tspan3272"> </tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18.8307px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="328.92212"
+ x="375.65271"
+ sodipodi:role="line"
+ id="tspan3274">5. DMA adapter submits DMA</tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18.8307px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="352.46051"
+ x="375.65271"
+ sodipodi:role="line"
+ id="tspan3294"> operations to DMA Device (Atomic</tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18.8307px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="375.99887"
+ x="375.65271"
+ sodipodi:role="line"
+ id="tspan3296"> stage)</tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18.8307px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="399.53726"
+ x="375.65271"
+ sodipodi:role="line"
+ id="tspan3276"> </tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18.8307px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="423.07562"
+ x="375.65271"
+ sodipodi:role="line"
+ id="tspan3278">6. DMA adapter dequeues DMA</tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18.8307px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="446.61401"
+ x="375.65271"
+ sodipodi:role="line"
+ id="tspan3298"> completions from DMA Device</tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18.8307px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="470.15237"
+ x="375.65271"
+ sodipodi:role="line"
+ id="tspan3280"> </tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18.8307px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="493.69073"
+ x="375.65271"
+ sodipodi:role="line"
+ id="tspan3282">7. DMA adapter enqueues events</tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18.8307px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="517.22913"
+ x="375.65271"
+ sodipodi:role="line"
+ id="tspan3300"> to the eventdev</tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18.8307px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="540.76752"
+ x="375.65271"
+ sodipodi:role="line"
+ id="tspan3284"> </tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18.8307px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="564.30585"
+ x="375.65271"
+ sodipodi:role="line"
+ id="tspan3286">8. Events to the next stage</tspan></text>
+ </g>
+ </g>
+</svg>
diff --git a/doc/guides/prog_guide/img/event_dma_adapter_op_new.svg b/doc/guides/prog_guide/img/event_dma_adapter_op_new.svg
new file mode 100644
index 0000000000..e9e8bb2b98
--- /dev/null
+++ b/doc/guides/prog_guide/img/event_dma_adapter_op_new.svg
@@ -0,0 +1,1079 @@
+<?xml version="1.0" encoding="UTF-8" standalone="no"?>
+<!-- SPDX-License-Identifier: BSD-3-Clause -->
+<!-- Copyright (c) 2023 Marvell. -->
+<!-- Created with Inkscape (http://www.inkscape.org/) -->
+
+<svg
+ width="720px"
+ height="486px"
+ id="svg13237"
+ version="1.1"
+ inkscape:version="1.2.1 (9c6d41e410, 2022-07-14)"
+ sodipodi:docname="event_dma_adapter_op_new.svg"
+ xmlns:inkscape="http://www.inkscape.org/namespaces/inkscape"
+ xmlns:sodipodi="http://sodipodi.sourceforge.net/DTD/sodipodi-0.dtd"
+ xmlns="http://www.w3.org/2000/svg"
+ xmlns:svg="http://www.w3.org/2000/svg"
+ xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#"
+ xmlns:cc="http://creativecommons.org/ns#"
+ xmlns:dc="http://purl.org/dc/elements/1.1/"
+ xmlns:v="http://schemas.microsoft.com/visio/2003/SVGExtensions/">
+ <defs
+ id="defs13239">
+ <marker
+ inkscape:stockid="Arrow1Sstart"
+ orient="auto"
+ refY="0.0"
+ refX="0.0"
+ id="Arrow1Sstart"
+ style="overflow:visible">
+ <path
+ id="path8416"
+ d="M 0.0,0.0 L 5.0,-5.0 L -12.5,0.0 L 5.0,5.0 L 0.0,0.0 z "
+ style="fill-rule:evenodd;stroke:#000000;stroke-width:1.0pt"
+ transform="scale(0.2) translate(6,0)" />
+ </marker>
+ <marker
+ inkscape:stockid="Arrow1Send"
+ orient="auto"
+ refY="0.0"
+ refX="0.0"
+ id="Arrow1Send"
+ style="overflow:visible;">
+ <path
+ id="path8419"
+ d="M 0.0,0.0 L 5.0,-5.0 L -12.5,0.0 L 5.0,5.0 L 0.0,0.0 z "
+ style="fill-rule:evenodd;stroke:#000000;stroke-width:1.0pt;"
+ transform="scale(0.2) rotate(180) translate(6,0)" />
+ </marker>
+ <marker
+ inkscape:stockid="DiamondL"
+ orient="auto"
+ refY="0.0"
+ refX="0.0"
+ id="DiamondL"
+ style="overflow:visible">
+ <path
+ id="path8483"
+ d="M 0,-7.0710768 L -7.0710894,0 L 0,7.0710589 L 7.0710462,0 L 0,-7.0710768 z "
+ style="fill-rule:evenodd;stroke:#000000;stroke-width:1.0pt"
+ transform="scale(0.8)" />
+ </marker>
+ <marker
+ inkscape:stockid="DotL"
+ orient="auto"
+ refY="0.0"
+ refX="0.0"
+ id="DotL"
+ style="overflow:visible">
+ <path
+ id="path8465"
+ d="M -2.5,-1.0 C -2.5,1.7600000 -4.7400000,4.0 -7.5,4.0 C -10.260000,4.0 -12.5,1.7600000 -12.5,-1.0 C -12.5,-3.7600000 -10.260000,-6.0 -7.5,-6.0 C -4.7400000,-6.0 -2.5,-3.7600000 -2.5,-1.0 z "
+ style="fill-rule:evenodd;stroke:#000000;stroke-width:1.0pt"
+ transform="scale(0.8) translate(7.4, 1)" />
+ </marker>
+ <marker
+ inkscape:stockid="SquareL"
+ orient="auto"
+ refY="0.0"
+ refX="0.0"
+ id="SquareL"
+ style="overflow:visible">
+ <path
+ id="path8474"
+ d="M -5.0,-5.0 L -5.0,5.0 L 5.0,5.0 L 5.0,-5.0 L -5.0,-5.0 z "
+ style="fill-rule:evenodd;stroke:#000000;stroke-width:1.0pt"
+ transform="scale(0.8)" />
+ </marker>
+ <marker
+ inkscape:stockid="TriangleOutL"
+ orient="auto"
+ refY="0.0"
+ refX="0.0"
+ id="TriangleOutL"
+ style="overflow:visible">
+ <path
+ id="path8546"
+ d="M 5.77,0.0 L -2.88,5.0 L -2.88,-5.0 L 5.77,0.0 z "
+ style="fill-rule:evenodd;stroke:#000000;stroke-width:1.0pt"
+ transform="scale(0.8)" />
+ </marker>
+ <marker
+ inkscape:stockid="Arrow1Lstart"
+ orient="auto"
+ refY="0.0"
+ refX="0.0"
+ id="Arrow1Lstart"
+ style="overflow:visible">
+ <path
+ id="path8404"
+ d="M 0.0,0.0 L 5.0,-5.0 L -12.5,0.0 L 5.0,5.0 L 0.0,0.0 z "
+ style="fill-rule:evenodd;stroke:#000000;stroke-width:1.0pt"
+ transform="scale(0.8) translate(12.5,0)" />
+ </marker>
+ <marker
+ inkscape:stockid="Arrow1Mend"
+ orient="auto"
+ refY="0.0"
+ refX="0.0"
+ id="Arrow1Mend"
+ style="overflow:visible;">
+ <path
+ id="path8413"
+ d="M 0.0,0.0 L 5.0,-5.0 L -12.5,0.0 L 5.0,5.0 L 0.0,0.0 z "
+ style="fill-rule:evenodd;stroke:#000000;stroke-width:1.0pt;"
+ transform="scale(0.4) rotate(180) translate(10,0)" />
+ </marker>
+ <marker
+ inkscape:stockid="Arrow2Lend"
+ orient="auto"
+ refY="0.0"
+ refX="0.0"
+ id="Arrow2Lend"
+ style="overflow:visible;">
+ <path
+ id="path8425"
+ style="fill-rule:evenodd;stroke-width:0.62500000;stroke-linejoin:round;"
+ d="M 8.7185878,4.0337352 L -2.2072895,0.016013256 L 8.7185884,-4.0017078 C 6.9730900,-1.6296469 6.9831476,1.6157441 8.7185878,4.0337352 z "
+ transform="scale(1.1) rotate(180) translate(1,0)" />
+ </marker>
+ <marker
+ inkscape:stockid="Arrow1Lend"
+ orient="auto"
+ refY="0.0"
+ refX="0.0"
+ id="Arrow1Lend"
+ style="overflow:visible;">
+ <path
+ id="path8407"
+ d="M 0.0,0.0 L 5.0,-5.0 L -12.5,0.0 L 5.0,5.0 L 0.0,0.0 z "
+ style="fill-rule:evenodd;stroke:#000000;stroke-width:1.0pt;"
+ transform="scale(0.8) rotate(180) translate(12.5,0)" />
+ </marker>
+ <filter
+ id="filter_2"
+ color-interpolation-filters="sRGB">
+ <feGaussianBlur
+ stdDeviation="2"
+ id="feGaussianBlur15" />
+ </filter>
+ <filter
+ id="filter_2-3"
+ color-interpolation-filters="sRGB"
+ x="-0.086178862"
+ y="-0.086178862"
+ width="1.1723577"
+ height="1.1723577">
+ <feGaussianBlur
+ stdDeviation="2"
+ id="feGaussianBlur15-1" />
+ </filter>
+ <filter
+ id="filter_2-0"
+ color-interpolation-filters="sRGB">
+ <feGaussianBlur
+ stdDeviation="2"
+ id="feGaussianBlur15-7" />
+ </filter>
+ <filter
+ id="filter_2-0-8"
+ color-interpolation-filters="sRGB">
+ <feGaussianBlur
+ stdDeviation="2"
+ id="feGaussianBlur15-7-7" />
+ </filter>
+ <filter
+ id="filter_2-3-9"
+ color-interpolation-filters="sRGB"
+ x="-0.086178862"
+ y="-0.086178862"
+ width="1.1723577"
+ height="1.1723577">
+ <feGaussianBlur
+ stdDeviation="2"
+ id="feGaussianBlur15-1-6" />
+ </filter>
+ <filter
+ id="filter_2-3-6"
+ color-interpolation-filters="sRGB"
+ x="-0.086178862"
+ y="-0.086178862"
+ width="1.1723577"
+ height="1.1723577">
+ <feGaussianBlur
+ stdDeviation="2"
+ id="feGaussianBlur15-1-63" />
+ </filter>
+ <filter
+ id="filter_2-3-91"
+ color-interpolation-filters="sRGB"
+ x="-0.086178862"
+ y="-0.086178862"
+ width="1.1723577"
+ height="1.1723577">
+ <feGaussianBlur
+ stdDeviation="2"
+ id="feGaussianBlur15-1-3" />
+ </filter>
+ <marker
+ inkscape:stockid="Arrow1Lend"
+ orient="auto"
+ refY="0"
+ refX="0"
+ id="Arrow1Lend-5"
+ style="overflow:visible">
+ <path
+ inkscape:connector-curvature="0"
+ id="path8407-3"
+ d="M 0,0 5,-5 -12.5,0 5,5 0,0 z"
+ style="fill-rule:evenodd;stroke:#000000;stroke-width:1pt"
+ transform="matrix(-0.8,0,0,-0.8,-10,0)" />
+ </marker>
+ <marker
+ inkscape:stockid="Arrow1Lend"
+ orient="auto"
+ refY="0"
+ refX="0"
+ id="Arrow1Lend-6"
+ style="overflow:visible">
+ <path
+ inkscape:connector-curvature="0"
+ id="path8407-0"
+ d="M 0,0 5,-5 -12.5,0 5,5 0,0 z"
+ style="fill-rule:evenodd;stroke:#000000;stroke-width:1pt"
+ transform="matrix(-0.8,0,0,-0.8,-10,0)" />
+ </marker>
+ <marker
+ inkscape:stockid="Arrow1Lstart"
+ orient="auto"
+ refY="0"
+ refX="0"
+ id="Arrow1Lstart-7"
+ style="overflow:visible">
+ <path
+ inkscape:connector-curvature="0"
+ id="path8404-0"
+ d="M 0,0 5,-5 -12.5,0 5,5 0,0 z"
+ style="fill-rule:evenodd;stroke:#000000;stroke-width:1pt"
+ transform="matrix(0.8,0,0,0.8,10,0)" />
+ </marker>
+ <marker
+ inkscape:stockid="Arrow1Lend"
+ orient="auto"
+ refY="0"
+ refX="0"
+ id="Arrow1Lend-51"
+ style="overflow:visible">
+ <path
+ inkscape:connector-curvature="0"
+ id="path8407-1"
+ d="M 0,0 5,-5 -12.5,0 5,5 0,0 z"
+ style="fill-rule:evenodd;stroke:#000000;stroke-width:1pt"
+ transform="matrix(-0.8,0,0,-0.8,-10,0)" />
+ </marker>
+ <marker
+ inkscape:stockid="Arrow1Lend"
+ orient="auto"
+ refY="0"
+ refX="0"
+ id="Arrow1Lend-3"
+ style="overflow:visible">
+ <path
+ inkscape:connector-curvature="0"
+ id="path8407-6"
+ d="M 0,0 5,-5 -12.5,0 5,5 0,0 z"
+ style="fill-rule:evenodd;stroke:#000000;stroke-width:1pt"
+ transform="matrix(-0.8,0,0,-0.8,-10,0)" />
+ </marker>
+ <marker
+ inkscape:stockid="Arrow1Lend"
+ orient="auto"
+ refY="0"
+ refX="0"
+ id="Arrow1Lend-62"
+ style="overflow:visible">
+ <path
+ inkscape:connector-curvature="0"
+ id="path8407-9"
+ d="M 0,0 5,-5 -12.5,0 5,5 0,0 z"
+ style="fill-rule:evenodd;stroke:#000000;stroke-width:1pt"
+ transform="matrix(-0.8,0,0,-0.8,-10,0)" />
+ </marker>
+ <marker
+ inkscape:stockid="Arrow1Lend"
+ orient="auto"
+ refY="0"
+ refX="0"
+ id="Arrow1Lend-2"
+ style="overflow:visible">
+ <path
+ inkscape:connector-curvature="0"
+ id="path8407-7"
+ d="M 0,0 5,-5 -12.5,0 5,5 0,0 z"
+ style="fill-rule:evenodd;stroke:#000000;stroke-width:1pt"
+ transform="matrix(-0.8,0,0,-0.8,-10,0)" />
+ </marker>
+ <filter
+ id="filter_2-3-91-3"
+ color-interpolation-filters="sRGB"
+ x="-0.086178862"
+ y="-0.086178862"
+ width="1.1723577"
+ height="1.1723577">
+ <feGaussianBlur
+ stdDeviation="2"
+ id="feGaussianBlur15-1-3-6" />
+ </filter>
+ </defs>
+ <sodipodi:namedview
+ id="base"
+ pagecolor="#ffffff"
+ bordercolor="#666666"
+ borderopacity="1.0"
+ inkscape:pageopacity="0.0"
+ inkscape:pageshadow="2"
+ inkscape:zoom="1"
+ inkscape:cx="385.5"
+ inkscape:cy="234"
+ inkscape:document-units="px"
+ inkscape:current-layer="layer1"
+ showgrid="false"
+ inkscape:window-width="1920"
+ inkscape:window-height="1017"
+ inkscape:window-x="-8"
+ inkscape:window-y="-8"
+ inkscape:window-maximized="1"
+ inkscape:snap-nodes="false"
+ inkscape:showpageshadow="2"
+ inkscape:pagecheckerboard="0"
+ inkscape:deskcolor="#d1d1d1">
+ <inkscape:grid
+ type="xygrid"
+ id="grid13454" />
+ </sodipodi:namedview>
+ <metadata
+ id="metadata13242">
+ <rdf:RDF>
+ <cc:Work
+ rdf:about="">
+ <dc:format>image/svg+xml</dc:format>
+ <dc:type
+ rdf:resource="http://purl.org/dc/dcmitype/StillImage" />
+ <dc:title />
+ </cc:Work>
+ </rdf:RDF>
+ </metadata>
+ <g
+ id="layer1"
+ inkscape:label="Layer 1"
+ inkscape:groupmode="layer">
+ <g
+ style="font-size:12px;fill:none;stroke-linecap:square;stroke-miterlimit:3;overflow:visible"
+ id="shape1-1-2-0"
+ v:mID="1"
+ v:groupContext="shape"
+ transform="matrix(2.1604167,0,0,1.5671361,323.2187,-540.25927)">
+ <title
+ id="title22-7-6">Square</title>
+ <desc
+ id="desc24-7-4">Atomic Queue #1</desc>
+ <v:userDefs>
+ <v:ud
+ v:nameU="visVersion"
+ v:val="VT0(15):26" />
+ </v:userDefs>
+ <v:textBlock
+ v:margins="rect(4,4,4,4)" />
+ <v:textRect
+ cx="30.75"
+ cy="581.25"
+ width="61.5"
+ height="61.5" />
+ <g
+ id="shadow1-2-9-0"
+ v:groupContext="shadow"
+ v:shadowOffsetX="0.345598"
+ v:shadowOffsetY="-1.97279"
+ v:shadowType="1"
+ transform="translate(0.345598,1.97279)"
+ class="st1"
+ style="visibility:visible">
+ <rect
+ x="0"
+ y="550.5"
+ width="61.5"
+ height="61.5"
+ class="st2"
+ id="rect27-8-9"
+ style="fill:#5b9bd5;fill-opacity:0.22000002;stroke:#5b9bd5;stroke-opacity:0.22000002;filter:url(#filter_2-3-9)" />
+ </g>
+ <g
+ id="g13515-4">
+ <g
+ id="g13534-5">
+ <rect
+ x="0"
+ y="550.5"
+ width="61.5"
+ height="61.5"
+ class="st3"
+ id="rect29-1-4"
+ style="fill:#5b9bd5;stroke:#c7c8c8;stroke-width:0.25" />
+ </g>
+ </g>
+ </g>
+ <g
+ style="font-size:12px;fill:none;stroke-linecap:square;stroke-miterlimit:3;overflow:visible"
+ id="shape1-1-2-4"
+ v:mID="1"
+ v:groupContext="shape"
+ transform="matrix(2.1604167,0,0,1.165886,88.874699,-447.8809)">
+ <title
+ id="title22-7-5">Square</title>
+ <desc
+ id="desc24-7-8">Atomic Queue #1</desc>
+ <v:userDefs>
+ <v:ud
+ v:nameU="visVersion"
+ v:val="VT0(15):26" />
+ </v:userDefs>
+ <v:textBlock
+ v:margins="rect(4,4,4,4)" />
+ <v:textRect
+ cx="30.75"
+ cy="581.25"
+ width="61.5"
+ height="61.5" />
+ <g
+ id="shadow1-2-9-5"
+ v:groupContext="shadow"
+ v:shadowOffsetX="0.345598"
+ v:shadowOffsetY="-1.97279"
+ v:shadowType="1"
+ transform="translate(0.345598,1.97279)"
+ class="st1"
+ style="visibility:visible">
+ <rect
+ x="0"
+ y="550.5"
+ width="61.5"
+ height="61.5"
+ class="st2"
+ id="rect27-8-7"
+ style="fill:#5b9bd5;fill-opacity:0.22000002;stroke:#5b9bd5;stroke-opacity:0.22000002;filter:url(#filter_2-3-91)" />
+ </g>
+ <g
+ id="g13515-33">
+ <g
+ id="g13534-8">
+ <rect
+ x="0"
+ y="550.5"
+ width="61.5"
+ height="61.5"
+ class="st3"
+ id="rect29-1-95"
+ style="fill:#5b9bd5;stroke:#c7c8c8;stroke-width:0.25" />
+ </g>
+ </g>
+ </g>
+ <g
+ style="font-size:12px;fill:none;stroke-linecap:square;stroke-miterlimit:3;overflow:visible"
+ id="shape1-1-2-9"
+ v:mID="1"
+ v:groupContext="shape"
+ transform="matrix(2.1604167,0,0,1.5671361,88.874699,-538.24651)">
+ <title
+ id="title22-7-9">Square</title>
+ <desc
+ id="desc24-7-5">Atomic Queue #1</desc>
+ <v:userDefs>
+ <v:ud
+ v:nameU="visVersion"
+ v:val="VT0(15):26" />
+ </v:userDefs>
+ <v:textBlock
+ v:margins="rect(4,4,4,4)" />
+ <v:textRect
+ cx="30.75"
+ cy="581.25"
+ width="61.5"
+ height="61.5" />
+ <g
+ id="shadow1-2-9-2"
+ v:groupContext="shadow"
+ v:shadowOffsetX="0.345598"
+ v:shadowOffsetY="-1.97279"
+ v:shadowType="1"
+ transform="translate(0.345598,1.97279)"
+ class="st1"
+ style="visibility:visible">
+ <rect
+ x="0"
+ y="550.5"
+ width="61.5"
+ height="61.5"
+ class="st2"
+ id="rect27-8-1"
+ style="fill:#5b9bd5;fill-opacity:0.22000002;stroke:#5b9bd5;stroke-opacity:0.22000002;filter:url(#filter_2-3-6)" />
+ </g>
+ <g
+ id="g13515-3">
+ <g
+ id="g13534-0">
+ <rect
+ x="0"
+ y="550.5"
+ width="61.5"
+ height="61.5"
+ class="st3"
+ id="rect29-1-9"
+ style="fill:#5b9bd5;stroke:#c7c8c8;stroke-width:0.25" />
+ </g>
+ </g>
+ </g>
+ <path
+ style="fill:none;stroke:#000000;stroke-width:0.743466px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1;marker-mid:none;marker-end:url(#Arrow1Lend)"
+ d="M 220.66064,98.57125 H 321.88592"
+ id="path17209-8"
+ inkscape:connector-type="orthogonal"
+ inkscape:connector-curvature="0" />
+ <g
+ style="font-size:12px;fill:none;stroke-linecap:square;stroke-miterlimit:3;overflow:visible"
+ id="shape1-1-2"
+ v:mID="1"
+ v:groupContext="shape"
+ transform="matrix(2.1604167,0,0,1.5671361,322.24227,-811.89589)">
+ <title
+ id="title22-7">Square</title>
+ <desc
+ id="desc24-7">Atomic Queue #1</desc>
+ <v:userDefs>
+ <v:ud
+ v:nameU="visVersion"
+ v:val="VT0(15):26" />
+ </v:userDefs>
+ <v:textBlock
+ v:margins="rect(4,4,4,4)" />
+ <v:textRect
+ cx="30.75"
+ cy="581.25"
+ width="61.5"
+ height="61.5" />
+ <g
+ id="shadow1-2-9"
+ v:groupContext="shadow"
+ v:shadowOffsetX="0.345598"
+ v:shadowOffsetY="-1.97279"
+ v:shadowType="1"
+ transform="translate(0.345598,1.97279)"
+ class="st1"
+ style="visibility:visible">
+ <rect
+ x="0"
+ y="550.5"
+ width="61.5"
+ height="61.5"
+ class="st2"
+ id="rect27-8"
+ style="fill:#5b9bd5;fill-opacity:0.22000002;stroke:#5b9bd5;stroke-opacity:0.22000002;filter:url(#filter_2-3)" />
+ </g>
+ <g
+ id="g13515">
+ <g
+ id="g13534">
+ <rect
+ x="0"
+ y="550.5"
+ width="61.5"
+ height="61.5"
+ class="st3"
+ id="rect29-1"
+ style="fill:#5b9bd5;stroke:#c7c8c8;stroke-width:0.25" />
+ </g>
+ </g>
+ </g>
+ <g
+ id="g13518"
+ transform="matrix(0.73232502,0,0,0.75477602,25.29268,348.89752)">
+ <g
+ id="g13526">
+ <flowRoot
+ xml:space="preserve"
+ id="flowRoot13464-9"
+ style="font-style:normal;font-weight:normal;line-height:0.01%;font-family:sans-serif;text-align:center;letter-spacing:0px;word-spacing:0px;text-anchor:middle;fill:#000000;fill-opacity:1;stroke:none"
+ transform="translate(-12.00521,-129.65179)"><flowRegion
+ id="flowRegion13466-1"
+ style="font-family:sans-serif"><rect
+ id="rect13468-2"
+ width="195.99997"
+ height="112.00001"
+ x="273.33334"
+ y="175.33333"
+ style="text-align:center;text-anchor:middle" /></flowRegion><flowPara
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:20px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:center;writing-mode:lr-tb;text-anchor:middle"
+ id="flowPara13511"> </flowPara></flowRoot>
+ </g>
+ </g>
+ <path
+ style="fill:none;stroke:#000000;stroke-width:0.751455;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:4;stroke-dasharray:none;stroke-opacity:1;marker-start:url(#Arrow1Lstart);marker-end:none"
+ d="m 176.26096,124.64833 v 69.24854"
+ id="path17209-3"
+ inkscape:connector-type="orthogonal"
+ inkscape:connector-curvature="0" />
+ <path
+ transform="matrix(0.73232502,0,0,0.75477602,-4.325033,50.642983)"
+ sodipodi:type="arc"
+ style="fill:#539de6;fill-opacity:1;stroke:#0000ea;stroke-width:1;stroke-linecap:square;stroke-miterlimit:4;stroke-opacity:1;stroke-dasharray:none"
+ id="path29161-3"
+ sodipodi:cx="371"
+ sodipodi:cy="64.5"
+ sodipodi:rx="17"
+ sodipodi:ry="15.5"
+ d="m 388,64.5 a 17,15.5 0 1 1 -34,0 17,15.5 0 1 1 34,0 z" />
+ <path
+ style="fill:none;stroke:#000000;stroke-width:0.743466px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1;marker-mid:none;marker-end:url(#Arrow1Lend)"
+ d="M 322.61264,375 H 221.38736"
+ id="path17209-8-0"
+ inkscape:connector-type="orthogonal"
+ inkscape:connector-curvature="0" />
+ <path
+ transform="matrix(0.73232502,0,0,0.75477602,0.0689171,324.80833)"
+ sodipodi:type="arc"
+ style="fill:#539de6;fill-opacity:1;stroke:#0000ea;stroke-width:1;stroke-linecap:square;stroke-miterlimit:4;stroke-opacity:1;stroke-dasharray:none"
+ id="path29161-1"
+ sodipodi:cx="371"
+ sodipodi:cy="64.5"
+ sodipodi:rx="17"
+ sodipodi:ry="15.5"
+ d="m 388,64.5 a 17,15.5 0 1 1 -34,0 17,15.5 0 1 1 34,0 z" />
+ <path
+ style="fill:none;stroke:#000000;stroke-width:0.629082px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1;marker-end:url(#Arrow1Lend)"
+ d="M 155,324.19955 V 264.82863"
+ id="path17211-7-1"
+ inkscape:connector-type="orthogonal"
+ inkscape:connector-curvature="0" />
+ <path
+ transform="matrix(0.73232502,0,0,0.75477602,-116.37076,245.97088)"
+ sodipodi:type="arc"
+ style="fill:#539de6;fill-opacity:1;stroke:#0000ea;stroke-width:1;stroke-linecap:square;stroke-miterlimit:4;stroke-opacity:1;stroke-dasharray:none"
+ id="path29161-8"
+ sodipodi:cx="371"
+ sodipodi:cy="64.5"
+ sodipodi:rx="17"
+ sodipodi:ry="15.5"
+ d="m 388,64.5 a 17,15.5 0 1 1 -34,0 17,15.5 0 1 1 34,0 z" />
+ <path
+ style="fill:none;stroke:#000000;stroke-width:0.75059;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:4;stroke-dasharray:none;stroke-opacity:1;marker-start:url(#Arrow1Lstart-7);marker-end:none"
+ d="m 126.26097,124.99178 v 69.24941"
+ id="path17209-3-0"
+ inkscape:connector-type="orthogonal"
+ inkscape:connector-curvature="0" />
+ <path
+ sodipodi:type="arc"
+ style="fill:#539de6;fill-opacity:1;stroke:#0000ea;stroke-width:1;stroke-linecap:square;stroke-miterlimit:4;stroke-opacity:1;stroke-dasharray:none"
+ id="path29161"
+ sodipodi:cx="371"
+ sodipodi:cy="64.5"
+ sodipodi:rx="17"
+ sodipodi:ry="15.5"
+ d="m 388,64.5 a 17,15.5 0 1 1 -34,0 17,15.5 0 1 1 34,0 z"
+ transform="matrix(0.73232502,0,0,0.75477602,-146.16394,110.68276)" />
+ <path
+ sodipodi:type="arc"
+ style="fill:#539de6;fill-opacity:1;stroke:#0000ea;stroke-width:1;stroke-linecap:square;stroke-miterlimit:4;stroke-opacity:1;stroke-dasharray:none"
+ id="path29161-2"
+ sodipodi:cx="371"
+ sodipodi:cy="64.5"
+ sodipodi:rx="17"
+ sodipodi:ry="15.5"
+ d="m 388,64.5 a 17,15.5 0 1 1 -34,0 17,15.5 0 1 1 34,0 z"
+ transform="matrix(0.73232502,0,0,0.75477602,-95.40136,110.71627)" />
+ <g
+ id="g29167-4"
+ transform="matrix(0.73232502,0,0,0.75477602,-95.31662,112.00756)">
+ <text
+ id="text29163-9"
+ y="70"
+ x="365"
+ style="font-style:normal;font-weight:normal;line-height:0%;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none"
+ xml:space="preserve"><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="70"
+ x="365"
+ id="tspan29165-9"
+ sodipodi:role="line">1</tspan></text>
+ </g>
+ <g
+ id="g29167-9"
+ transform="matrix(0.73232502,0,0,0.75477602,-4.9726112,50.689051)">
+ <text
+ id="text29163-3"
+ y="70"
+ x="365"
+ style="font-style:normal;font-weight:normal;line-height:0%;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none"
+ xml:space="preserve"><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="70"
+ x="365"
+ id="tspan29165-3"
+ sodipodi:role="line">2</tspan></text>
+ </g>
+ <path
+ style="fill:none;stroke:#000000;stroke-width:1.04033px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1;marker-end:url(#Arrow1Lend)"
+ d="M 388.20118,147.93341 V 321.89308"
+ id="path17211-7"
+ inkscape:connector-type="orthogonal"
+ inkscape:connector-curvature="0" />
+ <path
+ transform="matrix(0.73232502,0,0,0.75477602,116.5086,136.68745)"
+ sodipodi:type="arc"
+ style="fill:#539de6;fill-opacity:1;stroke:#0000ea;stroke-width:1;stroke-linecap:square;stroke-miterlimit:4;stroke-opacity:1;stroke-dasharray:none"
+ id="path29161-6"
+ sodipodi:cx="371"
+ sodipodi:cy="64.5"
+ sodipodi:rx="17"
+ sodipodi:ry="15.5"
+ d="m 388,64.5 a 17,15.5 0 1 1 -34,0 17,15.5 0 1 1 34,0 z" />
+ <g
+ id="g29167-46"
+ transform="matrix(0.73232502,0,0,0.75477602,116.59334,137.48829)">
+ <text
+ id="text29163-1"
+ y="70"
+ x="365"
+ style="font-style:normal;font-weight:normal;line-height:0%;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none"
+ xml:space="preserve"><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="70"
+ x="365"
+ id="tspan29165-5"
+ sodipodi:role="line">3</tspan></text>
+ </g>
+ <g
+ id="g29167-6"
+ transform="matrix(0.73232502,0,0,0.75477602,0.1536639,325.60918)">
+ <text
+ id="text29163-17"
+ y="70"
+ x="365"
+ style="font-style:normal;font-weight:normal;line-height:0%;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none"
+ xml:space="preserve"><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="70"
+ x="365"
+ id="tspan29165-2"
+ sodipodi:role="line">4</tspan></text>
+ </g>
+ <g
+ id="g29167"
+ transform="matrix(0.73232502,0,0,0.75477602,-146.07919,111.10621)">
+ <text
+ id="text29163"
+ y="70"
+ x="365"
+ style="font-style:normal;font-weight:normal;line-height:0%;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none"
+ xml:space="preserve"><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="70"
+ x="365"
+ id="tspan29165"
+ sodipodi:role="line">6</tspan></text>
+ </g>
+ <g
+ id="g29167-4-3"
+ transform="matrix(0.73232502,0,0,0.75477602,-117.60784,180.11779)">
+ <text
+ id="text29163-9-6"
+ y="70"
+ x="321.30356"
+ style="font-style:normal;font-weight:normal;line-height:0%;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none"
+ xml:space="preserve"><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:24.2109px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="70"
+ x="321.30356"
+ id="tspan29165-9-7"
+ sodipodi:role="line">Eventdev</tspan></text>
+ </g>
+ <g
+ id="g29167-4-3-5"
+ transform="matrix(0.73232502,0,0,0.75477602,55.34956,26.97821)">
+ <text
+ id="text29163-9-6-3"
+ y="70"
+ x="454.74152"
+ style="font-style:normal;font-weight:normal;line-height:0%;font-family:sans-serif;text-align:center;letter-spacing:0px;word-spacing:0px;text-anchor:middle;fill:#000000;fill-opacity:1;stroke:none"
+ xml:space="preserve"><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:21.5208px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:center;writing-mode:lr-tb;text-anchor:middle"
+ y="70"
+ x="454.74152"
+ id="tspan29165-9-7-5"
+ sodipodi:role="line">A<tspan
+ style="line-height:100%;font-family:sans-serif"
+ id="tspan3374">tomic Stage</tspan></tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:21.5208px;line-height:100%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:center;writing-mode:lr-tb;text-anchor:middle"
+ y="94.210899"
+ x="454.74152"
+ sodipodi:role="line"
+ id="tspan3320">+</tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:21.5208px;line-height:100%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:center;writing-mode:lr-tb;text-anchor:middle"
+ y="115.7317"
+ x="454.74152"
+ sodipodi:role="line"
+ id="tspan3322">Enqueue to</tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:21.5208px;line-height:100%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:center;writing-mode:lr-tb;text-anchor:middle"
+ y="137.2525"
+ x="454.74152"
+ sodipodi:role="line"
+ id="tspan3324">DMA Device</tspan></text>
+ </g>
+ <g
+ id="g29167-2"
+ transform="matrix(0.73232502,0,0,0.75477602,-116.28602,248.01695)">
+ <text
+ id="text29163-92"
+ y="70"
+ x="365"
+ style="font-style:normal;font-weight:normal;line-height:0%;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none"
+ xml:space="preserve"><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="70"
+ x="365"
+ id="tspan29165-8"
+ sodipodi:role="line">5</tspan></text>
+ </g>
+ <flowRoot
+ xml:space="preserve"
+ id="flowRoot3376"
+ style="font-style:normal;font-weight:normal;line-height:0.01%;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none;stroke-width:1px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1"><flowRegion
+ id="flowRegion3378"
+ style="font-family:sans-serif"><rect
+ id="rect3380"
+ width="100"
+ height="37"
+ x="109"
+ y="259" /></flowRegion><flowPara
+ id="flowPara3382"
+ style="font-size:18px;line-height:1.25;font-family:sans-serif"> </flowPara></flowRoot>
+ <g
+ id="g29167-4-3-1"
+ transform="matrix(0.73232502,0,0,0.75477602,109.34956,323.97821)">
+ <text
+ id="text29163-9-6-8"
+ y="70"
+ x="321.30356"
+ style="font-style:normal;font-weight:normal;line-height:0%;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none"
+ xml:space="preserve"><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:24.2109px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="70"
+ x="321.30356"
+ id="tspan29165-9-7-7"
+ sodipodi:role="line">DMA Device</tspan></text>
+ </g>
+ <g
+ id="g29167-4-3-1-9"
+ transform="matrix(0.73232502,0,0,0.75477602,-114.48565,314.20704)">
+ <text
+ id="text29163-9-6-8-2"
+ y="70"
+ x="368.01718"
+ style="font-style:normal;font-weight:normal;line-height:0%;font-family:sans-serif;text-align:center;letter-spacing:0px;word-spacing:0px;text-anchor:middle;fill:#000000;fill-opacity:1;stroke:none"
+ xml:space="preserve"><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:24.2109px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:center;writing-mode:lr-tb;text-anchor:middle"
+ y="70"
+ x="368.01718"
+ id="tspan29165-9-7-7-0"
+ sodipodi:role="line">DMA</tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:24.2109px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:center;writing-mode:lr-tb;text-anchor:middle"
+ y="100.26363"
+ x="368.01718"
+ sodipodi:role="line"
+ id="tspan3488">Adapter</tspan></text>
+ </g>
+ <g
+ id="g29167-4-3-1-9-2"
+ transform="matrix(0.73232502,0,0,0.75477602,250.96804,192.62529)">
+ <text
+ id="text29163-9-6-8-2-3"
+ y="-188.35481"
+ x="318.61978"
+ style="font-style:normal;font-weight:normal;line-height:0%;font-family:sans-serif;text-align:center;letter-spacing:0px;word-spacing:0px;text-anchor:middle;fill:#000000;fill-opacity:1;stroke:none"
+ xml:space="preserve"><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:21.5208px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="-188.35481"
+ x="318.61978"
+ sodipodi:role="line"
+ id="tspan3543">1. Application dequeues</tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:21.5208px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="-161.45381"
+ x="318.61978"
+ sodipodi:role="line"
+ id="tspan3196"> events from the previous</tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:21.5208px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="-134.55281"
+ x="318.61978"
+ sodipodi:role="line"
+ id="tspan3232"> stage</tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:21.5208px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="-107.65182"
+ x="318.61978"
+ sodipodi:role="line"
+ id="tspan3519"> </tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:21.5208px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="-80.750816"
+ x="318.61978"
+ sodipodi:role="line"
+ id="tspan3551">2. Application prepares the</tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:21.5208px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="-53.849815"
+ x="318.61978"
+ sodipodi:role="line"
+ id="tspan3203"> DMA operations.</tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:21.5208px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="-26.948814"
+ x="318.61978"
+ sodipodi:role="line"
+ id="tspan3523"> </tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:21.5208px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="-0.0478158"
+ x="318.61978"
+ sodipodi:role="line"
+ id="tspan3541">3. DMA operations are</tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:21.5208px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="26.853184"
+ x="318.61978"
+ sodipodi:role="line"
+ id="tspan3207"> submitted to dmadev</tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:21.5208px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="53.754185"
+ x="318.61978"
+ sodipodi:role="line"
+ id="tspan3209"> by application.</tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:21.5208px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="80.655182"
+ x="318.61978"
+ sodipodi:role="line"
+ id="tspan3527"> </tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:21.5208px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="107.55618"
+ x="318.61978"
+ sodipodi:role="line"
+ id="tspan3547">4. DMA adapter dequeues</tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:21.5208px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="134.45718"
+ x="318.61978"
+ sodipodi:role="line"
+ id="tspan3216"> DMA completions from</tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:21.5208px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="161.35818"
+ x="318.61978"
+ sodipodi:role="line"
+ id="tspan3218"> DMA device.</tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:21.5208px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="188.25919"
+ x="318.61978"
+ sodipodi:role="line"
+ id="tspan3531"> </tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:21.5208px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="215.16019"
+ x="318.61978"
+ sodipodi:role="line"
+ id="tspan3549">5. DMA adapter enqueues</tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:21.5208px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="242.06119"
+ x="318.61978"
+ sodipodi:role="line"
+ id="tspan3222"> events to the eventdev.</tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:21.5208px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="268.96219"
+ x="318.61978"
+ sodipodi:role="line"
+ id="tspan3535"> </tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:21.5208px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="295.86319"
+ x="318.61978"
+ sodipodi:role="line"
+ id="tspan3537">6. Application dequeues from</tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:21.5208px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="322.76419"
+ x="318.61978"
+ sodipodi:role="line"
+ id="tspan3224"> eventdev and prepare for</tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:21.5208px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="349.66519"
+ x="318.61978"
+ sodipodi:role="line"
+ id="tspan3226"> further processing</tspan></text>
+ </g>
+ <g
+ style="font-size:12px;fill:none;stroke-linecap:square;stroke-miterlimit:3;overflow:visible"
+ id="shape1-1-2-4-7"
+ v:mID="1"
+ v:groupContext="shape"
+ transform="matrix(2.1604167,0,0,1.165886,90.820551,-587.97129)">
+ <title
+ id="title22-7-5-5">Square</title>
+ <desc
+ id="desc24-7-8-3">Atomic Queue #1</desc>
+ <v:userDefs>
+ <v:ud
+ v:nameU="visVersion"
+ v:val="VT0(15):26" />
+ </v:userDefs>
+ <v:textBlock
+ v:margins="rect(4,4,4,4)" />
+ <v:textRect
+ cx="30.75"
+ cy="581.25"
+ width="61.5"
+ height="61.5" />
+ <g
+ id="shadow1-2-9-5-5"
+ v:groupContext="shadow"
+ v:shadowOffsetX="0.345598"
+ v:shadowOffsetY="-1.97279"
+ v:shadowType="1"
+ transform="translate(0.345598,1.97279)"
+ class="st1"
+ style="visibility:visible">
+ <rect
+ x="0"
+ y="550.5"
+ width="61.5"
+ height="61.5"
+ class="st2"
+ id="rect27-8-7-6"
+ style="fill:#5b9bd5;fill-opacity:0.22000002;stroke:#5b9bd5;stroke-opacity:0.22000002;filter:url(#filter_2-3-91-3)" />
+ </g>
+ <g
+ id="g13515-33-2">
+ <g
+ id="g13534-8-9">
+ <rect
+ x="0"
+ y="550.5"
+ width="61.5"
+ height="61.5"
+ class="st3"
+ id="rect29-1-95-1"
+ style="fill:#5b9bd5;stroke:#c7c8c8;stroke-width:0.25" />
+ </g>
+ </g>
+ </g>
+ <g
+ id="g29167-4-3-2"
+ transform="matrix(0.73232502,0,0,0.75477602,-125.66199,44.027402)">
+ <text
+ id="text29163-9-6-7"
+ y="70"
+ x="321.30356"
+ style="font-style:normal;font-weight:normal;line-height:0%;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none"
+ xml:space="preserve"><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:24.2109px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="70"
+ x="321.30356"
+ id="tspan29165-9-7-0"
+ sodipodi:role="line">Application</tspan></text>
+ </g>
+ </g>
+</svg>
diff --git a/doc/guides/prog_guide/index.rst b/doc/guides/prog_guide/index.rst
index 52a6d9e7aa..beaa4b8869 100644
--- a/doc/guides/prog_guide/index.rst
+++ b/doc/guides/prog_guide/index.rst
@@ -60,6 +60,7 @@ Programmer's Guide
event_ethernet_tx_adapter
event_timer_adapter
event_crypto_adapter
+ event_dma_adapter
qos_framework
power_man
packet_classif_access_ctrl
diff --git a/lib/eventdev/eventdev_pmd.h b/lib/eventdev/eventdev_pmd.h
index f62f42e140..6c77c128ac 100644
--- a/lib/eventdev/eventdev_pmd.h
+++ b/lib/eventdev/eventdev_pmd.h
@@ -178,8 +178,12 @@ struct rte_eventdev {
event_tx_adapter_enqueue_t txa_enqueue;
/**< Pointer to PMD eth Tx adapter enqueue function. */
event_crypto_adapter_enqueue_t ca_enqueue;
+ /**< Pointer to PMD crypto adapter enqueue function. */
- uint64_t reserved_64s[4]; /**< Reserved for future fields */
+ event_dma_adapter_enqueue_t dma_enqueue;
+ /**< Pointer to PMD DMA adapter enqueue function. */
+
+ uint64_t reserved_64s[3]; /**< Reserved for future fields */
void *reserved_ptrs[3]; /**< Reserved for future fields */
} __rte_cache_aligned;
@@ -1320,6 +1324,156 @@ typedef int (*eventdev_eth_tx_adapter_queue_stop)
#define eventdev_stop_flush_t rte_eventdev_stop_flush_t
+struct rte_dma_dev;
+
+/**
+ * Retrieve the event device's DMA adapter capabilities for the
+ * specified DMA device
+ *
+ * @param dev
+ * Event device pointer
+ *
+ * @param dmadev
+ * DMA device pointer
+ *
+ * @param[out] caps
+ * A pointer to memory filled with event adapter capabilities.
+ * It is expected to be pre-allocated & initialized by caller.
+ *
+ * @return
+ * - 0: Success, driver provides event adapter capabilities for the
+ * DMADEV.
+ * - <0: Error code returned by the driver function.
+ *
+ */
+typedef int (*eventdev_dma_adapter_caps_get_t)(const struct rte_eventdev *dev,
+ const struct rte_dma_dev *dmadev, uint32_t *caps);
+
+/**
+ * This API may change without prior notice
+ *
+ * Add DMA queue pair to event device. This callback is invoked if
+ * the caps returned from rte_event_dma_adapter_caps_get(, dmadev_id)
+ * has RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_* set.
+ *
+ * @param dev
+ * Event device pointer
+ *
+ * @param dmadev
+ * DMADEV pointer
+ *
+ * @param queue_pair_id
+ * DMADEV queue pair identifier.
+ *
+ * @param event
+ * Event information required for binding dmadev queue pair to event queue.
+ * This structure will have a valid value for only those HW PMDs supporting
+ * @see RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_QP_EV_BIND capability.
+ *
+ * @return
+ * - 0: Success, dmadev queue pair added successfully.
+ * - <0: Error code returned by the driver function.
+ *
+ */
+typedef int (*eventdev_dma_adapter_queue_pair_add_t)(const struct rte_eventdev *dev,
+ const struct rte_dma_dev *dmadev,
+ int32_t queue_pair_id,
+ const struct rte_event *event);
+
+/**
+ * This API may change without prior notice
+ *
+ * Delete DMA queue pair to event device. This callback is invoked if
+ * the caps returned from rte_event_dma_adapter_caps_get(, dmadev_id)
+ * has RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_* set.
+ *
+ * @param queue_pair_id
+ * dmadev queue pair identifier.
+ *
+ * @return
+ * - 0: Success, dmadev queue pair deleted successfully.
+ * - <0: Error code returned by the driver function.
+ *
+ */
+typedef int (*eventdev_dma_adapter_queue_pair_del_t)(const struct rte_eventdev *dev,
+ const struct rte_dma_dev *cdev,
+ int32_t queue_pair_id);
+
+/**
+ * Start DMA adapter. This callback is invoked if
+ * the caps returned from rte_event_dma_adapter_caps_get(.., dmadev_id)
+ * has RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_* set and queue pairs
+ * from dmadev_id have been added to the event device.
+ *
+ * @param dev
+ * Event device pointer
+ *
+ * @param dmadev
+ * DMA device pointer
+ *
+ * @return
+ * - 0: Success, DMA adapter started successfully.
+ * - <0: Error code returned by the driver function.
+ */
+typedef int (*eventdev_dma_adapter_start_t)(const struct rte_eventdev *dev,
+ const struct rte_dma_dev *dmadev);
+
+/**
+ * Stop DMA adapter. This callback is invoked if
+ * the caps returned from rte_event_dma_adapter_caps_get(.., dmadev_id)
+ * has RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_* set and queue pairs
+ * from dmadev_id have been added to the event device.
+ *
+ * @param dev
+ * Event device pointer
+ *
+ * @param dmadev
+ * DMA device pointer
+ *
+ * @return
+ * - 0: Success, DMA adapter stopped successfully.
+ * - <0: Error code returned by the driver function.
+ */
+typedef int (*eventdev_dma_adapter_stop_t)(const struct rte_eventdev *dev,
+ const struct rte_dma_dev *dmadev);
+
+struct rte_event_dma_adapter_stats;
+
+/**
+ * Retrieve DMA adapter statistics.
+ *
+ * @param dev
+ * Event device pointer
+ *
+ * @param dmadev
+ * DMA device pointer
+ *
+ * @param[out] stats
+ * Pointer to stats structure
+ *
+ * @return
+ * Return 0 on success.
+ */
+typedef int (*eventdev_dma_adapter_stats_get)(const struct rte_eventdev *dev,
+ const struct rte_dma_dev *dmadev,
+ struct rte_event_dma_adapter_stats *stats);
+
+/**
+ * Reset DMA adapter statistics.
+ *
+ * @param dev
+ * Event device pointer
+ *
+ * @param dmadev
+ * DMA device pointer
+ *
+ * @return
+ * Return 0 on success.
+ */
+typedef int (*eventdev_dma_adapter_stats_reset)(const struct rte_eventdev *dev,
+ const struct rte_dma_dev *dmadev);
+
+
/** Event device operations function pointer table */
struct eventdev_ops {
eventdev_info_get_t dev_infos_get; /**< Get device info. */
@@ -1440,6 +1594,21 @@ struct eventdev_ops {
eventdev_eth_tx_adapter_queue_stop eth_tx_adapter_queue_stop;
/**< Stop Tx queue assigned to Tx adapter instance */
+ eventdev_dma_adapter_caps_get_t dma_adapter_caps_get;
+ /**< Get DMA adapter capabilities */
+ eventdev_dma_adapter_queue_pair_add_t dma_adapter_queue_pair_add;
+ /**< Add queue pair to DMA adapter */
+ eventdev_dma_adapter_queue_pair_del_t dma_adapter_queue_pair_del;
+ /**< Delete queue pair from DMA adapter */
+ eventdev_dma_adapter_start_t dma_adapter_start;
+ /**< Start DMA adapter */
+ eventdev_dma_adapter_stop_t dma_adapter_stop;
+ /**< Stop DMA adapter */
+ eventdev_dma_adapter_stats_get dma_adapter_stats_get;
+ /**< Get DMA stats */
+ eventdev_dma_adapter_stats_reset dma_adapter_stats_reset;
+ /**< Reset DMA stats */
+
eventdev_selftest dev_selftest;
/**< Start eventdev Selftest */
diff --git a/lib/eventdev/eventdev_private.c b/lib/eventdev/eventdev_private.c
index 1d3d9d357e..18ed8bf3c8 100644
--- a/lib/eventdev/eventdev_private.c
+++ b/lib/eventdev/eventdev_private.c
@@ -81,6 +81,14 @@ dummy_event_crypto_adapter_enqueue(__rte_unused void *port,
return 0;
}
+static uint16_t
+dummy_event_dma_adapter_enqueue(__rte_unused void *port, __rte_unused struct rte_event ev[],
+ __rte_unused uint16_t nb_events)
+{
+ RTE_EDEV_LOG_ERR("event DMA adapter enqueue requested for unconfigured event device");
+ return 0;
+}
+
void
event_dev_fp_ops_reset(struct rte_event_fp_ops *fp_op)
{
@@ -97,6 +105,7 @@ event_dev_fp_ops_reset(struct rte_event_fp_ops *fp_op)
.txa_enqueue_same_dest =
dummy_event_tx_adapter_enqueue_same_dest,
.ca_enqueue = dummy_event_crypto_adapter_enqueue,
+ .dma_enqueue = dummy_event_dma_adapter_enqueue,
.data = dummy_data,
};
@@ -117,5 +126,6 @@ event_dev_fp_ops_set(struct rte_event_fp_ops *fp_op,
fp_op->txa_enqueue = dev->txa_enqueue;
fp_op->txa_enqueue_same_dest = dev->txa_enqueue_same_dest;
fp_op->ca_enqueue = dev->ca_enqueue;
+ fp_op->dma_enqueue = dev->dma_enqueue;
fp_op->data = dev->data->ports;
}
diff --git a/lib/eventdev/rte_event_dma_adapter.h b/lib/eventdev/rte_event_dma_adapter.h
new file mode 100644
index 0000000000..c667398d08
--- /dev/null
+++ b/lib/eventdev/rte_event_dma_adapter.h
@@ -0,0 +1,641 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright (c) 2023 Marvell.
+ */
+
+#ifndef RTE_EVENT_DMA_ADAPTER
+#define RTE_EVENT_DMA_ADAPTER
+
+/**
+ * @file rte_event_dma_adapter.h
+ *
+ * @warning
+ * @b EXPERIMENTAL:
+ * All functions in this file may be changed or removed without prior notice.
+ *
+ * DMA Event Adapter API.
+ *
+ * Eventdev library provides adapters to bridge between various components for providing new
+ * event source. The event DMA adapter is one of those adapters which is intended to bridge
+ * between event devices and DMA devices.
+ *
+ * The DMA adapter adds support to enqueue / dequeue DMA operations to / from event device. The
+ * packet flow between DMA device and the event device can be accomplished using both SW and HW
+ * based transfer mechanisms. The adapter uses an EAL service core function for SW based packet
+ * transfer and uses the eventdev PMD functions to configure HW based packet transfer between the
+ * DMA device and the event device.
+ *
+ * The application can choose to submit a DMA operation directly to an DMA device or send it to the
+ * DMA adapter via eventdev based on RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP_FWD capability. The
+ * first mode is known as the event new (RTE_EVENT_DMA_ADAPTER_OP_NEW) mode and the second as the
+ * event forward (RTE_EVENT_DMA_ADAPTER_OP_FORWARD) mode. The choice of mode can be specified while
+ * creating the adapter. In the former mode, it is an application responsibility to enable ingress
+ * packet ordering. In the latter mode, it is the adapter responsibility to enable the ingress
+ * packet ordering.
+ *
+ *
+ * Working model of RTE_EVENT_DMA_ADAPTER_OP_NEW mode:
+ *
+ * +--------------+ +--------------+
+ * | | | DMA stage |
+ * | Application |---[2]-->| + enqueue to |
+ * | | | dmadev |
+ * +--------------+ +--------------+
+ * ^ ^ |
+ * | | [3]
+ * [6] [1] |
+ * | | |
+ * +--------------+ |
+ * | | |
+ * | Event device | |
+ * | | |
+ * +--------------+ |
+ * ^ |
+ * | |
+ * [5] |
+ * | v
+ * +--------------+ +--------------+
+ * | | | |
+ * | DMA adapter |<--[4]---| dmadev |
+ * | | | |
+ * +--------------+ +--------------+
+ *
+ *
+ * [1] Application dequeues events from the previous stage.
+ * [2] Application prepares the DMA operations.
+ * [3] DMA operations are submitted to dmadev by application.
+ * [4] DMA adapter dequeues DMA completions from dmadev.
+ * [5] DMA adapter enqueues events to the eventdev.
+ * [6] Application dequeues from eventdev for further processing.
+ *
+ * In the RTE_EVENT_DMA_ADAPTER_OP_NEW mode, application submits DMA operations directly to DMA
+ * device. The DMA adapter then dequeues DMA completions from DMA device and enqueue events to the
+ * event device. This mode does not ensure ingress ordering, if the application directly enqueues
+ * to dmadev without going through DMA / atomic stage i.e. removing item [1] and [2].
+ *
+ * Events dequeued from the adapter will be treated as new events. In this mode, application needs
+ * to specify event information (response information) which is needed to enqueue an event after the
+ * DMA operation is completed.
+ *
+ *
+ * Working model of RTE_EVENT_DMA_ADAPTER_OP_FORWARD mode:
+ *
+ * +--------------+ +--------------+
+ * --[1]-->| |---[2]-->| Application |
+ * | Event device | | in |
+ * <--[8]--| |<--[3]---| Ordered stage|
+ * +--------------+ +--------------+
+ * ^ |
+ * | [4]
+ * [7] |
+ * | v
+ * +----------------+ +--------------+
+ * | |--[5]->| |
+ * | DMA adapter | | dmadev |
+ * | |<-[6]--| |
+ * +----------------+ +--------------+
+ *
+ *
+ * [1] Events from the previous stage.
+ * [2] Application in ordered stage dequeues events from eventdev.
+ * [3] Application enqueues DMA operations as events to eventdev.
+ * [4] DMA adapter dequeues event from eventdev.
+ * [5] DMA adapter submits DMA operations to dmadev (Atomic stage).
+ * [6] DMA adapter dequeues DMA completions from dmadev
+ * [7] DMA adapter enqueues events to the eventdev
+ * [8] Events to the next stage
+ *
+ * In the event forward (RTE_EVENT_DMA_ADAPTER_OP_FORWARD) mode, if the HW supports the capability
+ * RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP_FWD, application can directly submit the DMA
+ * operations to the dmadev. If not, application retrieves the event port of the DMA adapter
+ * through the API, rte_event_DMA_adapter_event_port_get(). Then, links its event queue to this
+ * port and starts enqueuing DMA operations as events to the eventdev. The adapter then dequeues
+ * the events and submits the DMA operations to the dmadev. After the DMA completions, the adapter
+ * enqueues events to the event device.
+ *
+ * Application can use this mode, when ingress packet ordering is needed. Events dequeued from the
+ * adapter will be treated as forwarded events. In this mode, the application needs to specify the
+ * dmadev ID and queue pair ID (request information) needed to enqueue an DMA operation in addition
+ * to the event information (response information) needed to enqueue an event after the DMA
+ * operation has completed.
+ *
+ * The event DMA adapter provides common APIs to configure the packet flow from the DMA device to
+ * event devices for both SW and HW based transfers. The DMA event adapter's functions are:
+ *
+ * - rte_event_dma_adapter_create_ext()
+ * - rte_event_dma_adapter_create()
+ * - rte_event_dma_adapter_free()
+ * - rte_event_dma_adapter_vchan_queue_add()
+ * - rte_event_dma_adapter_vchan_queue_del()
+ * - rte_event_dma_adapter_start()
+ * - rte_event_dma_adapter_stop()
+ * - rte_event_dma_adapter_stats_get()
+ * - rte_event_dma_adapter_stats_reset()
+ *
+ * The application creates an instance using rte_event_dma_adapter_create() or
+ * rte_event_dma_adapter_create_ext().
+ *
+ * dmadev queue pair addition / deletion is done using the rte_event_dma_adapter_vchan_queue_add() /
+ * rte_event_dma_adapter_vchan_queue_del() APIs. If HW supports the capability
+ * RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_QP_EV_BIND, event information must be passed to the add
+ * API.
+ *
+ */
+
+#include <stdint.h>
+
+#include "rte_eventdev.h"
+#include <rte_common.h>
+#include <rte_dmadev_pmd.h>
+
+#ifdef __cplusplus
+extern "C" {
+#endif
+
+/**
+ * A structure used to hold event based DMA operation entry.
+ */
+struct rte_event_dma_adapter_op {
+ struct rte_dma_sge *src_seg;
+ /**< Source segments. */
+ struct rte_dma_sge *dst_seg;
+ /**< Destination segments. */
+ uint16_t nb_src;
+ /**< Number of source segments. */
+ uint16_t nb_dst;
+ /**< Number of destination segments. */
+ uint64_t flags;
+ /**< Flags related to the operation.
+ * @see RTE_DMA_OP_FLAG_*
+ */
+ struct rte_mempool *op_mp;
+ /**< Mempool from which op is allocated. */
+};
+
+/**
+ * DMA event adapter mode
+ */
+enum rte_event_dma_adapter_mode {
+ RTE_EVENT_DMA_ADAPTER_OP_NEW,
+ /**< Start the DMA adapter in event new mode.
+ * @see RTE_EVENT_OP_NEW.
+ *
+ * Application submits DMA operations to the dmadev. Adapter only dequeues the DMA
+ * completions from dmadev and enqueue events to the eventdev.
+ */
+
+ RTE_EVENT_DMA_ADAPTER_OP_FORWARD,
+ /**< Start the DMA adapter in event forward mode.
+ * @see RTE_EVENT_OP_FORWARD.
+ *
+ * Application submits DMA requests as events to the DMA adapter or DMA device based on
+ * RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP_FWD capability. DMA completions are enqueued
+ * back to the eventdev by DMA adapter.
+ */
+};
+
+/**
+ * DMA event request structure will be filled by application to provide event request information to
+ * the adapter.
+ */
+struct rte_event_dma_request {
+ uint8_t resv[8];
+ /**< Overlaps with first 8 bytes of struct rte_event that encode the response event
+ * information. Application is expected to fill in struct rte_event response_info.
+ */
+
+ int16_t dmadev_id;
+ /**< DMA device ID to be used */
+
+ uint16_t queue_pair_id;
+ /**< DMA queue pair ID to be used */
+
+ uint32_t rsvd;
+ /**< Reserved bits */
+};
+
+/**
+ * Adapter configuration structure that the adapter configuration callback function is expected to
+ * fill out.
+ *
+ * @see rte_event_dma_adapter_conf_cb
+ */
+struct rte_event_dma_adapter_conf {
+ uint8_t event_port_id;
+ /** < Event port identifier, the adapter enqueues events to this port and dequeues DMA
+ * request events in RTE_EVENT_DMA_ADAPTER_OP_FORWARD mode.
+ */
+
+ uint32_t max_nb;
+ /**< The adapter can return early if it has processed at least max_nb DMA ops. This isn't
+ * treated as a requirement; batching may cause the adapter to process more than max_nb DMA
+ * ops.
+ */
+};
+
+/**
+ * Adapter runtime configuration parameters
+ */
+struct rte_event_dma_adapter_runtime_params {
+ uint32_t max_nb;
+ /**< The adapter can return early if it has processed at least max_nb DMA ops. This isn't
+ * treated as a requirement; batching may cause the adapter to process more than max_nb DMA
+ * ops.
+ *
+ * Callback function passed to rte_event_dma_adapter_create_ext() configures the adapter
+ * with default value of max_nb.
+ * rte_event_dma_adapter_runtime_params_set() allows to re-configure max_nb during runtime
+ * (after adding at least one queue pair)
+ *
+ * This is valid for the devices without RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP_FWD or
+ * RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP_NEW capability.
+ */
+
+ uint32_t rsvd[15];
+ /**< Reserved fields for future expansion */
+};
+
+/**
+ * Function type used for adapter configuration callback. The callback is used to fill in members of
+ * the struct rte_event_dma_adapter_conf, this callback is invoked when creating a SW service for
+ * packet transfer from dmadev vchan queue to the event device. The SW service is created within the
+ * function, rte_event_dma_adapter_vchan_queue_add(), if SW based packet
+ * transfers from dmadev vchan queue to the event device are required.
+ *
+ * @param id
+ * Adapter identifier.
+ * @param evdev_id
+ * Event device identifier.
+ * @param conf
+ * Structure that needs to be populated by this callback.
+ * @param arg
+ * Argument to the callback. This is the same as the conf_arg passed to the
+ * rte_event_dma_adapter_create_ext().
+ */
+typedef int (*rte_event_dma_adapter_conf_cb)(uint8_t id, uint8_t evdev_id,
+ struct rte_event_dma_adapter_conf *conf, void *arg);
+
+/**
+ * A structure used to retrieve statistics for an event DMA adapter instance.
+ */
+struct rte_event_dma_adapter_stats {
+ uint64_t event_poll_count;
+ /**< Event port poll count */
+
+ uint64_t event_deq_count;
+ /**< Event dequeue count */
+
+ uint64_t dma_enq_count;
+ /**< dmadev enqueue count */
+
+ uint64_t dma_enq_fail_count;
+ /**< dmadev enqueue failed count */
+
+ uint64_t dma_deq_count;
+ /**< dmadev dequeue count */
+
+ uint64_t event_enq_count;
+ /**< Event enqueue count */
+
+ uint64_t event_enq_retry_count;
+ /**< Event enqueue retry count */
+
+ uint64_t event_enq_fail_count;
+ /**< Event enqueue fail count */
+};
+
+/**
+ * @warning
+ * @b EXPERIMENTAL: this API may change without prior notice.
+ *
+ * Create a new event DMA adapter with the specified identifier.
+ *
+ * @param id
+ * Adapter identifier.
+ * @param evdev_id
+ * Event device identifier.
+ * @param conf_cb
+ * Callback function that fills in members of a struct rte_event_dma_adapter_conf struct passed
+ * into it.
+ * @param mode
+ * Flag to indicate the mode of the adapter.
+ * @see rte_event_dma_adapter_mode
+ * @param conf_arg
+ * Argument that is passed to the conf_cb function.
+ *
+ * @return
+ * - 0: Success
+ * - <0: Error code on failure
+ */
+__rte_experimental
+int rte_event_dma_adapter_create_ext(uint8_t id, uint8_t evdev_id,
+ rte_event_dma_adapter_conf_cb conf_cb,
+ enum rte_event_dma_adapter_mode mode, void *conf_arg);
+
+/**
+ * @warning
+ * @b EXPERIMENTAL: this API may change without prior notice.
+ *
+ * Create a new event DMA adapter with the specified identifier. This function uses an internal
+ * configuration function that creates an event port. This default function reconfigures the event
+ * device with an additional event port and set up the event port using the port_config parameter
+ * passed into this function. In case the application needs more control in configuration of the
+ * service, it should use the rte_event_dma_adapter_create_ext() version.
+ *
+ * @param id
+ * Adapter identifier.
+ * @param evdev_id
+ * Event device identifier.
+ * @param port_config
+ * Argument of type *rte_event_port_conf* that is passed to the conf_cb function.
+ * @param mode
+ * Flag to indicate the mode of the adapter.
+ * @see rte_event_dma_adapter_mode
+ *
+ * @return
+ * - 0: Success
+ * - <0: Error code on failure
+ */
+__rte_experimental
+int rte_event_dma_adapter_create(uint8_t id, uint8_t evdev_id,
+ struct rte_event_port_conf *port_config,
+ enum rte_event_dma_adapter_mode mode);
+
+/**
+ * @warning
+ * @b EXPERIMENTAL: this API may change without prior notice.
+ *
+ * Free an event DMA adapter
+ *
+ * @param id
+ * Adapter identifier.
+ * @return
+ * - 0: Success
+ * - <0: Error code on failure, If the adapter still has queue pairs added to it, the function
+ * returns -EBUSY.
+ */
+__rte_experimental
+int rte_event_dma_adapter_free(uint8_t id);
+
+/**
+ * @warning
+ * @b EXPERIMENTAL: this API may change without prior notice.
+ *
+ * Retrieve the event port of an adapter.
+ *
+ * @param id
+ * Adapter identifier.
+ *
+ * @param [out] event_port_id
+ * Application links its event queue to this adapter port which is used in
+ * RTE_EVENT_DMA_ADAPTER_OP_FORWARD mode.
+ *
+ * @return
+ * - 0: Success
+ * - <0: Error code on failure.
+ */
+__rte_experimental
+int rte_event_dma_adapter_event_port_get(uint8_t id, uint8_t *event_port_id);
+
+/**
+ * @warning
+ * @b EXPERIMENTAL: this API may change without prior notice.
+ *
+ * Add a vchan queue to an event DMA adapter.
+ *
+ * @param id
+ * Adapter identifier.
+ * @param dmadev_id
+ * dmadev identifier.
+ * @param queue_pair_id
+ * DMA device vchan queue identifier. If queue_pair_id is set -1, adapter adds all the
+ * preconfigured queue pairs to the instance.
+ * @param event
+ * If HW supports dmadev queue pair to event queue binding, application is expected to fill in
+ * event information, else it will be NULL.
+ * @see RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_QP_EV_BIND
+ *
+ * @return
+ * - 0: Success, vchan queue added correctly.
+ * - <0: Error code on failure.
+ */
+__rte_experimental
+int rte_event_dma_adapter_vchan_queue_add(uint8_t id, int16_t dmadev_id, int32_t queue_pair_id,
+ const struct rte_event *event);
+
+/**
+ * @warning
+ * @b EXPERIMENTAL: this API may change without prior notice.
+ *
+ * Delete a vchan queue from an event DMA adapter.
+ *
+ * @param id
+ * Adapter identifier.
+ * @param dmadev_id
+ * DMA device identifier.
+ * @param queue_pair_id
+ * DMA device vchan queue identifier.
+ *
+ * @return
+ * - 0: Success, vchan queue deleted successfully.
+ * - <0: Error code on failure.
+ */
+__rte_experimental
+int rte_event_dma_adapter_vchan_queue_del(uint8_t id, int16_t dmadev_id, int32_t queue_pair_id);
+
+/**
+ * @warning
+ * @b EXPERIMENTAL: this API may change without prior notice.
+ *
+ * Retrieve the service ID of an adapter. If the adapter doesn't use a rte_service function, this
+ * function returns -ESRCH.
+ *
+ * @param id
+ * Adapter identifier.
+ * @param [out] service_id
+ * A pointer to a uint32_t, to be filled in with the service id.
+ *
+ * @return
+ * - 0: Success
+ * - <0: Error code on failure, if the adapter doesn't use a rte_service function, this function
+ * returns -ESRCH.
+ */
+__rte_experimental
+int rte_event_dma_adapter_service_id_get(uint8_t id, uint32_t *service_id);
+
+/**
+ * @warning
+ * @b EXPERIMENTAL: this API may change without prior notice.
+ *
+ * Start event DMA adapter
+ *
+ * @param id
+ * Adapter identifier.
+ *
+ * @return
+ * - 0: Success, adapter started successfully.
+ * - <0: Error code on failure.
+ *
+ * @note The eventdev and dmadev to which the event_dma_adapter is connected should be started
+ * before calling rte_event_dma_adapter_start().
+ */
+__rte_experimental
+int rte_event_dma_adapter_start(uint8_t id);
+
+/**
+ * @warning
+ * @b EXPERIMENTAL: this API may change without prior notice.
+ *
+ * Stop event DMA adapter
+ *
+ * @param id
+ * Adapter identifier.
+ *
+ * @return
+ * - 0: Success, adapter stopped successfully.
+ * - <0: Error code on failure.
+ */
+__rte_experimental
+int rte_event_dma_adapter_stop(uint8_t id);
+
+/**
+ * @warning
+ * @b EXPERIMENTAL: this API may change without prior notice.
+ *
+ * Initialize the adapter runtime configuration parameters
+ *
+ * @param params
+ * A pointer to structure of type struct rte_event_dma_adapter_runtime_params
+ *
+ * @return
+ * - 0: Success
+ * - <0: Error code on failure
+ */
+__rte_experimental
+int rte_event_dma_adapter_runtime_params_init(struct rte_event_dma_adapter_runtime_params *params);
+
+/**
+ * @warning
+ * @b EXPERIMENTAL: this API may change without prior notice.
+ *
+ * Set the adapter runtime configuration parameters
+ *
+ * @param id
+ * Adapter identifier
+ *
+ * @param params
+ * A pointer to structure of type struct rte_event_dma_adapter_runtime_params with configuration
+ * parameter values. The reserved fields of this structure must be initialized to zero and the valid
+ * fields need to be set appropriately. This struct can be initialized using
+ * rte_event_dma_adapter_runtime_params_init() API to default values or application may reset this
+ * struct and update required fields.
+ *
+ * @return
+ * - 0: Success
+ * - <0: Error code on failure
+ */
+__rte_experimental
+int rte_event_dma_adapter_runtime_params_set(uint8_t id,
+ struct rte_event_dma_adapter_runtime_params *params);
+
+/**
+ * @warning
+ * @b EXPERIMENTAL: this API may change without prior notice.
+ *
+ * Get the adapter runtime configuration parameters
+ *
+ * @param id
+ * Adapter identifier
+ *
+ * @param[out] params
+ * A pointer to structure of type struct rte_event_dma_adapter_runtime_params containing valid
+ * adapter parameters when return value is 0.
+ *
+ * @return
+ * - 0: Success
+ * - <0: Error code on failure
+ */
+__rte_experimental
+int rte_event_dma_adapter_runtime_params_get(uint8_t id,
+ struct rte_event_dma_adapter_runtime_params *params);
+
+/**
+ * @warning
+ * @b EXPERIMENTAL: this API may change without prior notice.
+ *
+ * Retrieve statistics for an adapter
+ *
+ * @param id
+ * Adapter identifier.
+ * @param [out] stats
+ * A pointer to structure used to retrieve statistics for an adapter.
+ *
+ * @return
+ * - 0: Success, retrieved successfully.
+ * - <0: Error code on failure.
+ */
+__rte_experimental
+int rte_event_dma_adapter_stats_get(uint8_t id, struct rte_event_dma_adapter_stats *stats);
+
+/**
+ * @warning
+ * @b EXPERIMENTAL: this API may change without prior notice.
+ *
+ * Reset statistics for an adapter.
+ *
+ * @param id
+ * Adapter identifier.
+ *
+ * @return
+ * - 0: Success, statistics reset successfully.
+ * - <0: Error code on failure.
+ */
+__rte_experimental
+int rte_event_dma_adapter_stats_reset(uint8_t id);
+
+/**
+ * @warning
+ * @b EXPERIMENTAL: this API may change without prior notice.
+ *
+ * Enqueue a burst of DMA operations as event objects supplied in *rte_event* structure on an event
+ * DMA adapter designated by its event *evdev_id* through the event port specified by *port_id*.
+ * This function is supported if the eventdev PMD has the
+ * #RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP_FWD capability flag set.
+ *
+ * The *nb_events* parameter is the number of event objects to enqueue that are supplied in the
+ * *ev* array of *rte_event* structure.
+ *
+ * The rte_event_dma_adapter_enqueue() function returns the number of event objects it actually
+ * enqueued. A return value equal to *nb_events* means that all event objects have been enqueued.
+ *
+ * @param evdev_id
+ * The identifier of the device.
+ * @param port_id
+ * The identifier of the event port.
+ * @param ev
+ * Points to an array of *nb_events* objects of type *rte_event* structure which contain the
+ * event object enqueue operations to be processed.
+ * @param nb_events
+ * The number of event objects to enqueue, typically number of
+ * rte_event_port_attr_get(...RTE_EVENT_PORT_ATTR_ENQ_DEPTH...) available for this port.
+ *
+ * @return
+ * The number of event objects actually enqueued on the event device. The return value can be
+ * less than the value of the *nb_events* parameter when the event devices queue is full or if
+ * invalid parameters are specified in a *rte_event*. If the return value is less than *nb_events*,
+ * the remaining events at the end of ev[] are not consumed and the caller has to take care of them,
+ * and rte_errno is set accordingly. Possible errno values include:
+ *
+ * - EINVAL: The port ID is invalid, device ID is invalid, an event's queue ID is invalid, or an
+ * event's sched type doesn't match the capabilities of the destination queue.
+ * - ENOSPC: The event port was backpressured and unable to enqueue one or more events. This
+ * error code is only applicable to closed systems.
+ */
+__rte_experimental
+uint16_t rte_event_dma_adapter_enqueue(uint8_t evdev_id, uint8_t port_id, struct rte_event ev[],
+ uint16_t nb_events);
+
+#ifdef __cplusplus
+}
+#endif
+
+#endif /* RTE_EVENT_DMA_ADAPTER */
diff --git a/lib/eventdev/rte_eventdev.h b/lib/eventdev/rte_eventdev.h
index 2ba8a7b090..d231f527ae 100644
--- a/lib/eventdev/rte_eventdev.h
+++ b/lib/eventdev/rte_eventdev.h
@@ -1197,6 +1197,8 @@ struct rte_event_vector {
*/
#define RTE_EVENT_TYPE_ETH_RX_ADAPTER 0x4
/**< The event generated from event eth Rx adapter */
+#define RTE_EVENT_TYPE_DMADEV 0x5
+/**< The event generated from dma subsystem */
#define RTE_EVENT_TYPE_VECTOR 0x8
/**< Indicates that event is a vector.
* All vector event types should be a logical OR of EVENT_TYPE_VECTOR.
@@ -1462,6 +1464,49 @@ int
rte_event_crypto_adapter_caps_get(uint8_t dev_id, uint8_t cdev_id,
uint32_t *caps);
+/* DMA adapter capability bitmap flag */
+#define RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP_NEW 0x1
+/**< Flag indicates HW is capable of generating events in
+ * RTE_EVENT_OP_NEW enqueue operation. DMADEV will send
+ * packets to the event device as new events using an
+ * internal event port.
+ */
+
+#define RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP_FWD 0x2
+/**< Flag indicates HW is capable of generating events in
+ * RTE_EVENT_OP_FORWARD enqueue operation. DMADEV will send
+ * packets to the event device as forwarded event using an
+ * internal event port.
+ */
+
+#define RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_QP_EV_BIND 0x4
+/**< Flag indicates HW is capable of mapping DMA queue pair to
+ * event queue.
+ */
+
+/**
+ * Retrieve the event device's DMA adapter capabilities for the
+ * specified dmadev device
+ *
+ * @param dev_id
+ * The identifier of the device.
+ *
+ * @param dmadev_id
+ * The identifier of the dmadev device.
+ *
+ * @param[out] caps
+ * A pointer to memory filled with event adapter capabilities.
+ * It is expected to be pre-allocated & initialized by caller.
+ *
+ * @return
+ * - 0: Success, driver provides event adapter capabilities for the
+ * dmadev device.
+ * - <0: Error code returned by the driver function.
+ *
+ */
+int
+rte_event_dma_adapter_caps_get(uint8_t dev_id, int16_t dmadev_id, uint32_t *caps);
+
/* Ethdev Tx adapter capability bitmap flags */
#define RTE_EVENT_ETH_TX_ADAPTER_CAP_INTERNAL_PORT 0x1
/**< This flag is sent when the PMD supports a packet transmit callback
diff --git a/lib/eventdev/rte_eventdev_core.h b/lib/eventdev/rte_eventdev_core.h
index c27a52ccc0..83e8736c71 100644
--- a/lib/eventdev/rte_eventdev_core.h
+++ b/lib/eventdev/rte_eventdev_core.h
@@ -42,6 +42,10 @@ typedef uint16_t (*event_crypto_adapter_enqueue_t)(void *port,
uint16_t nb_events);
/**< @internal Enqueue burst of events on crypto adapter */
+typedef uint16_t (*event_dma_adapter_enqueue_t)(void *port, struct rte_event ev[],
+ uint16_t nb_events);
+/**< @internal Enqueue burst of events on DMA adapter */
+
struct rte_event_fp_ops {
void **data;
/**< points to array of internal port data pointers */
@@ -65,7 +69,9 @@ struct rte_event_fp_ops {
/**< PMD Tx adapter enqueue same destination function. */
event_crypto_adapter_enqueue_t ca_enqueue;
/**< PMD Crypto adapter enqueue function. */
- uintptr_t reserved[5];
+ event_dma_adapter_enqueue_t dma_enqueue;
+ /**< PMD DMA adapter enqueue function. */
+ uintptr_t reserved[4];
} __rte_cache_aligned;
extern struct rte_event_fp_ops rte_event_fp_ops[RTE_EVENT_MAX_DEVS];
diff --git a/lib/eventdev/version.map b/lib/eventdev/version.map
index 7ce09a87bb..597a5c9cda 100644
--- a/lib/eventdev/version.map
+++ b/lib/eventdev/version.map
@@ -134,6 +134,21 @@ EXPERIMENTAL {
# added in 23.11
rte_event_eth_rx_adapter_create_ext_with_params;
+ rte_event_dma_adapter_create_ext;
+ rte_event_dma_adapter_create;
+ rte_event_dma_adapter_free;
+ rte_event_dma_adapter_event_port_get;
+ rte_event_dma_adapter_vchan_queue_add;
+ rte_event_dma_adapter_vchan_queue_del;
+ rte_event_dma_adapter_service_id_get;
+ rte_event_dma_adapter_start;
+ rte_event_dma_adapter_stop;
+ rte_event_dma_adapter_runtime_params_init;
+ rte_event_dma_adapter_runtime_params_set;
+ rte_event_dma_adapter_runtime_params_get;
+ rte_event_dma_adapter_stats_get;
+ rte_event_dma_adapter_stats_reset;
+ rte_event_dma_adapter_enqueue;
};
INTERNAL {
--
2.25.1
^ permalink raw reply [flat|nested] 110+ messages in thread
* [PATCH v1 2/7] eventdev: api to get DMA capabilities
2023-09-19 13:42 [PATCH v1 1/7] eventdev: introduce DMA event adapter library Amit Prakash Shukla
@ 2023-09-19 13:42 ` Amit Prakash Shukla
2023-09-19 13:42 ` [PATCH v1 3/7] eventdev: add DMA adapter implementation Amit Prakash Shukla
` (6 subsequent siblings)
7 siblings, 0 replies; 110+ messages in thread
From: Amit Prakash Shukla @ 2023-09-19 13:42 UTC (permalink / raw)
To: Jerin Jacob
Cc: dev, fengchengwen, kevin.laatz, bruce.richardson, conor.walsh,
vattunuru, g.singh, sachin.saxena, hemant.agrawal, cheng1.jiang,
ndabilpuram, anoobj, mb, Amit Prakash Shukla
Added a new eventdev API rte_event_dma_adapter_caps_get(), to get
DMA adapter capabilities supported by the driver.
Signed-off-by: Amit Prakash Shukla <amitprakashs@marvell.com>
---
lib/eventdev/meson.build | 2 +-
lib/eventdev/rte_eventdev.c | 25 +++++++++++++++++++++++++
lib/eventdev/rte_eventdev.h | 2 +-
lib/meson.build | 2 +-
4 files changed, 28 insertions(+), 3 deletions(-)
diff --git a/lib/eventdev/meson.build b/lib/eventdev/meson.build
index 6edf98dfa5..fbab3a42ad 100644
--- a/lib/eventdev/meson.build
+++ b/lib/eventdev/meson.build
@@ -42,5 +42,5 @@ driver_sdk_headers += files(
'event_timer_adapter_pmd.h',
)
-deps += ['ring', 'ethdev', 'hash', 'mempool', 'mbuf', 'timer', 'cryptodev']
+deps += ['ring', 'ethdev', 'hash', 'mempool', 'mbuf', 'timer', 'cryptodev', 'dmadev']
deps += ['telemetry']
diff --git a/lib/eventdev/rte_eventdev.c b/lib/eventdev/rte_eventdev.c
index 6ab4524332..9415788b6a 100644
--- a/lib/eventdev/rte_eventdev.c
+++ b/lib/eventdev/rte_eventdev.c
@@ -21,6 +21,7 @@
#include <ethdev_driver.h>
#include <rte_cryptodev.h>
#include <cryptodev_pmd.h>
+#include <rte_dmadev_pmd.h>
#include <rte_telemetry.h>
#include "rte_eventdev.h"
@@ -224,6 +225,30 @@ rte_event_eth_tx_adapter_caps_get(uint8_t dev_id, uint16_t eth_port_id,
: 0;
}
+int
+rte_event_dma_adapter_caps_get(uint8_t dev_id, uint8_t dma_dev_id, uint32_t *caps)
+{
+ struct rte_eventdev *dev;
+ struct rte_dma_dev *dma_dev;
+
+ RTE_EVENTDEV_VALID_DEVID_OR_ERR_RET(dev_id, -EINVAL);
+ if (!rte_dma_is_valid(dma_dev_id))
+ return -EINVAL;
+
+ dev = &rte_eventdevs[dev_id];
+ dma_dev = rte_dma_pmd_dev_get(dma_dev_id);
+
+ if (caps == NULL || dma_dev == NULL)
+ return -EINVAL;
+
+ *caps = 0;
+
+ if (dev->dev_ops->dma_adapter_caps_get)
+ return (*dev->dev_ops->dma_adapter_caps_get)(dev, dma_dev, caps);
+
+ return 0;
+}
+
static inline int
event_dev_queue_config(struct rte_eventdev *dev, uint8_t nb_queues)
{
diff --git a/lib/eventdev/rte_eventdev.h b/lib/eventdev/rte_eventdev.h
index d231f527ae..5611880872 100644
--- a/lib/eventdev/rte_eventdev.h
+++ b/lib/eventdev/rte_eventdev.h
@@ -1505,7 +1505,7 @@ rte_event_crypto_adapter_caps_get(uint8_t dev_id, uint8_t cdev_id,
*
*/
int
-rte_event_dma_adapter_caps_get(uint8_t dev_id, int16_t dmadev_id, uint32_t *caps);
+rte_event_dma_adapter_caps_get(uint8_t dev_id, uint8_t dmadev_id, uint32_t *caps);
/* Ethdev Tx adapter capability bitmap flags */
#define RTE_EVENT_ETH_TX_ADAPTER_CAP_INTERNAL_PORT 0x1
diff --git a/lib/meson.build b/lib/meson.build
index 53155be8e9..f3191f10b6 100644
--- a/lib/meson.build
+++ b/lib/meson.build
@@ -33,6 +33,7 @@ libraries = [
'compressdev',
'cryptodev',
'distributor',
+ 'dmadev',
'efd',
'eventdev',
'gpudev',
@@ -48,7 +49,6 @@ libraries = [
'rawdev',
'regexdev',
'mldev',
- 'dmadev',
'rib',
'reorder',
'sched',
--
2.25.1
^ permalink raw reply [flat|nested] 110+ messages in thread
* [PATCH v1 3/7] eventdev: add DMA adapter implementation
2023-09-19 13:42 [PATCH v1 1/7] eventdev: introduce DMA event adapter library Amit Prakash Shukla
2023-09-19 13:42 ` [PATCH v1 2/7] eventdev: api to get DMA capabilities Amit Prakash Shukla
@ 2023-09-19 13:42 ` Amit Prakash Shukla
2023-09-19 13:42 ` [PATCH v1 4/7] app/test: add event DMA adapter auto-test Amit Prakash Shukla
` (5 subsequent siblings)
7 siblings, 0 replies; 110+ messages in thread
From: Amit Prakash Shukla @ 2023-09-19 13:42 UTC (permalink / raw)
To: Bruce Richardson, Jerin Jacob
Cc: dev, fengchengwen, kevin.laatz, conor.walsh, vattunuru, g.singh,
sachin.saxena, hemant.agrawal, cheng1.jiang, ndabilpuram, anoobj,
mb, Amit Prakash Shukla
This patch adds common code for the dma adapter to support
SW and HW based transfer mechanisms. The adapter uses an EAL
service core function for SW based packet transfer and uses
the eventdev PMD functions to configure HW based packet
transfer between the dma device and the event device.
Signed-off-by: Amit Prakash Shukla <amitprakashs@marvell.com>
---
config/rte_config.h | 1 +
lib/eventdev/meson.build | 2 +
lib/eventdev/rte_event_dma_adapter.c | 1423 ++++++++++++++++++++++++++
lib/eventdev/rte_event_dma_adapter.h | 41 +-
4 files changed, 1458 insertions(+), 9 deletions(-)
create mode 100644 lib/eventdev/rte_event_dma_adapter.c
diff --git a/config/rte_config.h b/config/rte_config.h
index 400e44e3cf..401727703f 100644
--- a/config/rte_config.h
+++ b/config/rte_config.h
@@ -77,6 +77,7 @@
#define RTE_EVENT_ETH_INTR_RING_SIZE 1024
#define RTE_EVENT_CRYPTO_ADAPTER_MAX_INSTANCE 32
#define RTE_EVENT_ETH_TX_ADAPTER_MAX_INSTANCE 32
+#define RTE_EVENT_DMA_ADAPTER_MAX_INSTANCE 32
/* rawdev defines */
#define RTE_RAWDEV_MAX_DEVS 64
diff --git a/lib/eventdev/meson.build b/lib/eventdev/meson.build
index fbab3a42ad..822cd83857 100644
--- a/lib/eventdev/meson.build
+++ b/lib/eventdev/meson.build
@@ -19,6 +19,7 @@ sources = files(
'rte_event_crypto_adapter.c',
'rte_event_eth_rx_adapter.c',
'rte_event_eth_tx_adapter.c',
+ 'rte_event_dma_adapter.c',
'rte_event_ring.c',
'rte_event_timer_adapter.c',
'rte_eventdev.c',
@@ -27,6 +28,7 @@ headers = files(
'rte_event_crypto_adapter.h',
'rte_event_eth_rx_adapter.h',
'rte_event_eth_tx_adapter.h',
+ 'rte_event_dma_adapter.h',
'rte_event_ring.h',
'rte_event_timer_adapter.h',
'rte_eventdev.h',
diff --git a/lib/eventdev/rte_event_dma_adapter.c b/lib/eventdev/rte_event_dma_adapter.c
new file mode 100644
index 0000000000..e13283726f
--- /dev/null
+++ b/lib/eventdev/rte_event_dma_adapter.c
@@ -0,0 +1,1423 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright (c) 2023 Marvell.
+ */
+
+#include <string.h>
+#include <stdbool.h>
+#include <rte_common.h>
+#include <dev_driver.h>
+#include <rte_errno.h>
+#include <rte_dmadev_pmd.h>
+#include <rte_log.h>
+#include <rte_malloc.h>
+#include <rte_service_component.h>
+
+#include "rte_eventdev.h"
+#include "eventdev_pmd.h"
+#include "rte_event_dma_adapter.h"
+
+#define DMA_BATCH_SIZE 32
+#define DMA_DEFAULT_MAX_NB 128
+#define DMA_ADAPTER_NAME_LEN 32
+#define DMA_ADAPTER_BUFFER_SIZE 1024
+
+#define DMA_ADAPTER_OPS_BUFFER_SIZE (DMA_BATCH_SIZE + DMA_BATCH_SIZE)
+
+#define DMA_ADAPTER_ARRAY "event_dma_adapter_array"
+
+/* Macros to check for valid adapter */
+#define EVENT_DMA_ADAPTER_ID_VALID_OR_ERR_RET(id, retval) \
+ do { \
+ if (!edma_adapter_valid_id(id)) { \
+ RTE_EDEV_LOG_ERR("Invalid DMA adapter id = %d\n", id); \
+ return retval; \
+ } \
+ } while (0)
+
+/* DMA ops circular buffer */
+struct dma_ops_circular_buffer {
+ /* Index of head element */
+ uint16_t head;
+
+ /* Index of tail element */
+ uint16_t tail;
+
+ /* Number of elements in buffer */
+ uint16_t count;
+
+ /* Size of circular buffer */
+ uint16_t size;
+
+ /* Pointer to hold rte_event_dma_adapter_op for processing */
+ struct rte_event_dma_adapter_op **op_buffer;
+} __rte_cache_aligned;
+
+/* Queue pair information */
+struct dma_vchan_queue_info {
+ /* Set to indicate vchan queue is enabled */
+ bool vq_enabled;
+
+ /* Circular buffer for batching DMA ops to dma_dev */
+ struct dma_ops_circular_buffer dma_buf;
+} __rte_cache_aligned;
+
+/* DMA device information */
+struct dma_device_info {
+ /* Pointer to dma_dev */
+ struct rte_dma_dev *dev;
+
+ /* Pointer to vchan queue info */
+ struct dma_vchan_queue_info *vchanq;
+
+ /* Pointer to vchan queue info.
+ * This holds ops passed by application.
+ */
+ struct dma_vchan_queue_info *tqmap;
+
+ /* Next queue pair to be processed */
+ uint16_t next_queue_pair_id;
+
+ /* If num_vchanq > 0, the start callback will
+ * be invoked if not already invoked
+ */
+ uint16_t num_vchanq;
+
+ /* Set to indicate processing has been started */
+ uint8_t dev_started;
+
+ /* Set to indicate dmadev->eventdev packet
+ * transfer uses a hardware mechanism
+ */
+ uint8_t internal_event_port;
+} __rte_cache_aligned;
+
+struct event_dma_adapter {
+ /* Event device identifier */
+ uint8_t eventdev_id;
+
+ /* Event port identifier */
+ uint8_t event_port_id;
+
+ /* Adapter mode */
+ enum rte_event_dma_adapter_mode mode;
+
+ /* Memory allocation name */
+ char mem_name[DMA_ADAPTER_NAME_LEN];
+
+ /* Socket identifier cached from eventdev */
+ int socket_id;
+
+ /* Lock to serialize config updates with service function */
+ rte_spinlock_t lock;
+
+ /* Next dma device to be processed */
+ uint16_t next_dmadev_id;
+
+ /* DMA device structure array */
+ struct dma_device_info *dma_devs;
+
+ /* Circular buffer for processing DMA ops to eventdev */
+ struct dma_ops_circular_buffer ebuf;
+
+ /* Configuration callback for rte_service configuration */
+ rte_event_dma_adapter_conf_cb conf_cb;
+
+ /* Configuration callback argument */
+ void *conf_arg;
+
+ /* Set if default_cb is being used */
+ int default_cb_arg;
+
+ /* No. of vchan queue configured */
+ uint16_t nb_vchanq;
+
+ /* Per adapter EAL service ID */
+ uint32_t service_id;
+
+ /* Service initialization state */
+ uint8_t service_initialized;
+
+ /* Max DMA ops processed in any service function invocation */
+ uint32_t max_nb;
+
+ /* Store event port's implicit release capability */
+ uint8_t implicit_release_disabled;
+
+ /* Flag to indicate backpressure at dma_dev
+ * Stop further dequeuing events from eventdev
+ */
+ bool stop_enq_to_dma_dev;
+
+ /* Loop counter to flush dma ops */
+ uint16_t transmit_loop_count;
+
+ /* Per instance stats structure */
+ struct rte_event_dma_adapter_stats dma_stats;
+} __rte_cache_aligned;
+
+static struct event_dma_adapter **event_dma_adapter;
+
+static inline int
+edma_adapter_valid_id(uint8_t id)
+{
+ return id < RTE_EVENT_DMA_ADAPTER_MAX_INSTANCE;
+}
+
+static inline struct event_dma_adapter *
+edma_id_to_adapter(uint8_t id)
+{
+ return event_dma_adapter ? event_dma_adapter[id] : NULL;
+}
+
+static int
+edma_array_init(void)
+{
+ const struct rte_memzone *mz;
+ uint32_t sz;
+
+ mz = rte_memzone_lookup(DMA_ADAPTER_ARRAY);
+ if (mz == NULL) {
+ sz = sizeof(struct event_dma_adapter *) * RTE_EVENT_DMA_ADAPTER_MAX_INSTANCE;
+ sz = RTE_ALIGN(sz, RTE_CACHE_LINE_SIZE);
+
+ mz = rte_memzone_reserve_aligned(DMA_ADAPTER_ARRAY, sz, rte_socket_id(), 0,
+ RTE_CACHE_LINE_SIZE);
+ if (mz == NULL) {
+ RTE_EDEV_LOG_ERR("Failed to reserve memzone : %s, err = %d",
+ DMA_ADAPTER_ARRAY, rte_errno);
+ return -rte_errno;
+ }
+ }
+
+ event_dma_adapter = mz->addr;
+
+ return 0;
+}
+
+static inline bool
+edma_circular_buffer_batch_ready(struct dma_ops_circular_buffer *bufp)
+{
+ return bufp->count >= DMA_BATCH_SIZE;
+}
+
+static inline bool
+edma_circular_buffer_space_for_batch(struct dma_ops_circular_buffer *bufp)
+{
+ return (bufp->size - bufp->count) >= DMA_BATCH_SIZE;
+}
+
+static inline int
+edma_circular_buffer_init(const char *name, struct dma_ops_circular_buffer *buf, uint16_t sz)
+{
+ buf->op_buffer = rte_zmalloc(name, sizeof(struct rte_event_dma_adapter_op *) * sz, 0);
+ if (buf->op_buffer == NULL)
+ return -ENOMEM;
+
+ buf->size = sz;
+
+ return 0;
+}
+
+static inline void
+edma_circular_buffer_free(struct dma_ops_circular_buffer *buf)
+{
+ rte_free(buf->op_buffer);
+}
+
+static inline int
+edma_circular_buffer_add(struct dma_ops_circular_buffer *bufp, struct rte_event_dma_adapter_op *op)
+{
+ uint16_t *tail = &bufp->tail;
+
+ bufp->op_buffer[*tail] = op;
+
+ /* circular buffer, go round */
+ *tail = (*tail + 1) % bufp->size;
+ bufp->count++;
+
+ return 0;
+}
+
+static inline int
+edma_circular_buffer_flush_to_dma_dev(struct event_dma_adapter *adapter,
+ struct dma_ops_circular_buffer *bufp, uint8_t dma_dev_id,
+ uint16_t vchan, uint16_t *nb_ops_flushed)
+{
+ struct rte_event_dma_adapter_op *op;
+ struct dma_vchan_queue_info *tq;
+ uint16_t *head = &bufp->head;
+ uint16_t *tail = &bufp->tail;
+ uint16_t n;
+ uint16_t i;
+ int ret;
+
+ if (*tail > *head)
+ n = *tail - *head;
+ else if (*tail < *head)
+ n = bufp->size - *head;
+ else {
+ *nb_ops_flushed = 0;
+ return 0; /* buffer empty */
+ }
+
+ tq = &adapter->dma_devs[dma_dev_id].tqmap[vchan];
+
+ for (i = 0; i < n; i++) {
+ op = bufp->op_buffer[*head];
+ ret = rte_dma_copy_sg(dma_dev_id, vchan, op->src_seg, op->dst_seg,
+ op->nb_src, op->nb_dst, op->flags);
+ if (ret < 0)
+ break;
+
+ /* Enqueue in transaction queue. */
+ edma_circular_buffer_add(&tq->dma_buf, op);
+
+ *head = (*head + 1) % bufp->size;
+ }
+
+ *nb_ops_flushed = i;
+ bufp->count -= *nb_ops_flushed;
+ if (!bufp->count) {
+ *head = 0;
+ *tail = 0;
+ }
+
+ return *nb_ops_flushed == n ? 0 : -1;
+}
+
+static int
+edma_default_config_cb(uint8_t id, uint8_t evdev_id, struct rte_event_dma_adapter_conf *conf,
+ void *arg)
+{
+ struct rte_event_port_conf *port_conf;
+ struct rte_event_dev_config dev_conf;
+ struct event_dma_adapter *adapter;
+ struct rte_eventdev *dev;
+ uint8_t port_id;
+ int started;
+ int ret;
+
+ adapter = edma_id_to_adapter(id);
+ if (adapter == NULL)
+ return -EINVAL;
+
+ dev = &rte_eventdevs[adapter->eventdev_id];
+ dev_conf = dev->data->dev_conf;
+
+ started = dev->data->dev_started;
+ if (started)
+ rte_event_dev_stop(evdev_id);
+
+ port_id = dev_conf.nb_event_ports;
+ dev_conf.nb_event_ports += 1;
+
+ port_conf = arg;
+ if (port_conf->event_port_cfg & RTE_EVENT_PORT_CFG_SINGLE_LINK)
+ dev_conf.nb_single_link_event_port_queues += 1;
+
+ ret = rte_event_dev_configure(evdev_id, &dev_conf);
+ if (ret) {
+ RTE_EDEV_LOG_ERR("Failed to configure event dev %u\n", evdev_id);
+ if (started) {
+ if (rte_event_dev_start(evdev_id))
+ return -EIO;
+ }
+ return ret;
+ }
+
+ ret = rte_event_port_setup(evdev_id, port_id, port_conf);
+ if (ret) {
+ RTE_EDEV_LOG_ERR("Failed to setup event port %u\n", port_id);
+ return ret;
+ }
+
+ conf->event_port_id = port_id;
+ conf->max_nb = DMA_DEFAULT_MAX_NB;
+ if (started)
+ ret = rte_event_dev_start(evdev_id);
+
+ adapter->default_cb_arg = 1;
+ adapter->event_port_id = conf->event_port_id;
+
+ return ret;
+}
+
+int
+rte_event_dma_adapter_create_ext(uint8_t id, uint8_t evdev_id,
+ rte_event_dma_adapter_conf_cb conf_cb,
+ enum rte_event_dma_adapter_mode mode, void *conf_arg)
+{
+ struct rte_event_dev_info dev_info;
+ struct event_dma_adapter *adapter;
+ char name[DMA_ADAPTER_NAME_LEN];
+ uint16_t num_dma_dev;
+ int socket_id;
+ uint8_t i;
+ int ret;
+
+ EVENT_DMA_ADAPTER_ID_VALID_OR_ERR_RET(id, -EINVAL);
+ RTE_EVENTDEV_VALID_DEVID_OR_ERR_RET(evdev_id, -EINVAL);
+
+ if (conf_cb == NULL)
+ return -EINVAL;
+
+ if (event_dma_adapter == NULL) {
+ ret = edma_array_init();
+ if (ret)
+ return ret;
+ }
+
+ adapter = edma_id_to_adapter(id);
+ if (adapter != NULL) {
+ RTE_EDEV_LOG_ERR("ML adapter ID %d already exists!", id);
+ return -EEXIST;
+ }
+
+ socket_id = rte_event_dev_socket_id(evdev_id);
+ snprintf(name, DMA_ADAPTER_NAME_LEN, "rte_event_dma_adapter_%d", id);
+ adapter = rte_zmalloc_socket(name, sizeof(struct event_dma_adapter), RTE_CACHE_LINE_SIZE,
+ socket_id);
+ if (adapter == NULL) {
+ RTE_EDEV_LOG_ERR("Failed to get mem for event ML adapter!");
+ return -ENOMEM;
+ }
+
+ if (edma_circular_buffer_init("edma_circular_buffer", &adapter->ebuf,
+ DMA_ADAPTER_BUFFER_SIZE)) {
+ RTE_EDEV_LOG_ERR("Failed to get memory for event adapter circular buffer");
+ rte_free(adapter);
+ return -ENOMEM;
+ }
+
+ ret = rte_event_dev_info_get(evdev_id, &dev_info);
+ if (ret < 0) {
+ RTE_EDEV_LOG_ERR("Failed to get info for eventdev %d: %s", evdev_id,
+ dev_info.driver_name);
+ edma_circular_buffer_free(&adapter->ebuf);
+ rte_free(adapter);
+ return ret;
+ }
+
+ num_dma_dev = rte_dma_count_avail();
+
+ adapter->eventdev_id = evdev_id;
+ adapter->mode = mode;
+ strcpy(adapter->mem_name, name);
+ adapter->socket_id = socket_id;
+ adapter->conf_cb = conf_cb;
+ adapter->conf_arg = conf_arg;
+ adapter->dma_devs = rte_zmalloc_socket(adapter->mem_name,
+ num_dma_dev * sizeof(struct dma_device_info), 0,
+ socket_id);
+ if (adapter->dma_devs == NULL) {
+ RTE_EDEV_LOG_ERR("Failed to get memory for DMA devices\n");
+ edma_circular_buffer_free(&adapter->ebuf);
+ rte_free(adapter);
+ return -ENOMEM;
+ }
+
+ rte_spinlock_init(&adapter->lock);
+ for (i = 0; i < num_dma_dev; i++)
+ adapter->dma_devs[i].dev = rte_dma_pmd_dev_get(i);
+
+ event_dma_adapter[id] = adapter;
+
+ return 0;
+}
+
+int
+rte_event_dma_adapter_create(uint8_t id, uint8_t evdev_id, struct rte_event_port_conf *port_config,
+ enum rte_event_dma_adapter_mode mode)
+{
+ struct rte_event_port_conf *pc;
+ int ret;
+
+ if (port_config == NULL)
+ return -EINVAL;
+
+ EVENT_DMA_ADAPTER_ID_VALID_OR_ERR_RET(id, -EINVAL);
+
+ pc = rte_malloc(NULL, sizeof(struct rte_event_port_conf), 0);
+ if (pc == NULL)
+ return -ENOMEM;
+
+ rte_memcpy(pc, port_config, sizeof(struct rte_event_port_conf));
+ ret = rte_event_dma_adapter_create_ext(id, evdev_id, edma_default_config_cb, mode, pc);
+ if (ret != 0)
+ rte_free(pc);
+
+ return ret;
+}
+
+int
+rte_event_dma_adapter_free(uint8_t id)
+{
+ struct event_dma_adapter *adapter;
+
+ EVENT_DMA_ADAPTER_ID_VALID_OR_ERR_RET(id, -EINVAL);
+
+ adapter = edma_id_to_adapter(id);
+ if (adapter == NULL)
+ return -EINVAL;
+
+ rte_free(adapter->conf_arg);
+ rte_free(adapter->dma_devs);
+ edma_circular_buffer_free(&adapter->ebuf);
+ rte_free(adapter);
+ event_dma_adapter[id] = NULL;
+
+ return 0;
+}
+
+int
+rte_event_dma_adapter_event_port_get(uint8_t id, uint8_t *event_port_id)
+{
+ struct event_dma_adapter *adapter;
+
+ EVENT_DMA_ADAPTER_ID_VALID_OR_ERR_RET(id, -EINVAL);
+
+ adapter = edma_id_to_adapter(id);
+ if (adapter == NULL || event_port_id == NULL)
+ return -EINVAL;
+
+ *event_port_id = adapter->event_port_id;
+
+ return 0;
+}
+
+static inline unsigned int
+edma_enq_to_dma_dev(struct event_dma_adapter *adapter, struct rte_event *ev, unsigned int cnt)
+{
+ struct rte_event_dma_adapter_stats *stats = &adapter->dma_stats;
+ union rte_event_dma_metadata *m_data = NULL;
+ struct dma_vchan_queue_info *vchan_qinfo = NULL;
+ struct rte_event_dma_adapter_op *dma_op;
+ uint16_t vchan, nb_enqueued = 0;
+ int16_t dma_dev_id;
+ unsigned int i, n;
+ int ret;
+
+ ret = 0;
+ n = 0;
+ stats->event_deq_count += cnt;
+
+ for (i = 0; i < cnt; i++) {
+ dma_op = ev[i].event_ptr;
+ if (dma_op == NULL)
+ continue;
+
+ /* Expected to have metadata appended to dma_op. */
+ m_data = (union rte_event_dma_metadata *)((uint8_t *)dma_op +
+ sizeof(struct rte_event_dma_adapter_op));
+ if (m_data == NULL) {
+ if (dma_op != NULL && dma_op->op_mp != NULL)
+ rte_mempool_put(dma_op->op_mp, dma_op);
+ continue;
+ }
+
+ dma_dev_id = m_data->request_info.dma_dev_id;
+ vchan = m_data->request_info.vchan;
+ vchan_qinfo = &adapter->dma_devs[dma_dev_id].vchanq[vchan];
+ if (!vchan_qinfo->vq_enabled) {
+ if (dma_op != NULL && dma_op->op_mp != NULL)
+ rte_mempool_put(dma_op->op_mp, dma_op);
+ continue;
+ }
+ edma_circular_buffer_add(&vchan_qinfo->dma_buf, dma_op);
+
+ if (edma_circular_buffer_batch_ready(&vchan_qinfo->dma_buf)) {
+ ret = edma_circular_buffer_flush_to_dma_dev(adapter, &vchan_qinfo->dma_buf,
+ dma_dev_id, vchan,
+ &nb_enqueued);
+ stats->dma_enq_count += nb_enqueued;
+ n += nb_enqueued;
+
+ /**
+ * If some dma ops failed to flush to dma_dev and
+ * space for another batch is not available, stop
+ * dequeue from eventdev momentarily
+ */
+ if (unlikely(ret < 0 &&
+ !edma_circular_buffer_space_for_batch(&vchan_qinfo->dma_buf)))
+ adapter->stop_enq_to_dma_dev = true;
+ }
+ }
+
+ return n;
+}
+
+static unsigned int
+edma_adapter_dev_flush(struct event_dma_adapter *adapter, int16_t dma_dev_id,
+ uint16_t *nb_ops_flushed)
+{
+ struct dma_vchan_queue_info *vchan_queue;
+ struct dma_device_info *dev_info;
+ uint16_t nb = 0, nb_enqueued = 0;
+ uint16_t vchan, nb_vchans;
+ struct rte_dma_dev *dev;
+
+ dev_info = &adapter->dma_devs[dma_dev_id];
+ dev = rte_dma_pmd_dev_get(dma_dev_id);
+ nb_vchans = dev->data->dev_conf.nb_vchans;
+
+ for (vchan = 0; vchan < nb_vchans; vchan++) {
+
+ vchan_queue = &dev_info->vchanq[vchan];
+ if (unlikely(vchan_queue == NULL || !vchan_queue->vq_enabled))
+ continue;
+
+ edma_circular_buffer_flush_to_dma_dev(adapter, &vchan_queue->dma_buf, dma_dev_id,
+ vchan, &nb_enqueued);
+ *nb_ops_flushed += vchan_queue->dma_buf.count;
+ nb += nb_enqueued;
+ }
+
+ return nb;
+}
+
+static unsigned int
+edma_adapter_enq_flush(struct event_dma_adapter *adapter)
+{
+ struct rte_event_dma_adapter_stats *stats = &adapter->dma_stats;
+ int16_t dma_dev_id;
+ uint16_t nb_enqueued = 0;
+ uint16_t nb_ops_flushed = 0;
+ uint16_t num_dma_dev = rte_dma_count_avail();
+
+ for (dma_dev_id = 0; dma_dev_id < num_dma_dev; dma_dev_id++)
+ nb_enqueued += edma_adapter_dev_flush(adapter, dma_dev_id, &nb_ops_flushed);
+ /**
+ * Enable dequeue from eventdev if all ops from circular
+ * buffer flushed to dma_dev
+ */
+ if (!nb_ops_flushed)
+ adapter->stop_enq_to_dma_dev = false;
+
+ stats->dma_enq_count += nb_enqueued;
+
+ return nb_enqueued;
+}
+
+/* Flush an instance's enqueue buffers every DMA_ENQ_FLUSH_THRESHOLD
+ * iterations of edma_adapter_enq_run()
+ */
+#define DMA_ENQ_FLUSH_THRESHOLD 1024
+
+static int
+edma_adapter_enq_run(struct event_dma_adapter *adapter, unsigned int max_enq)
+{
+ struct rte_event_dma_adapter_stats *stats = &adapter->dma_stats;
+ uint8_t event_port_id = adapter->event_port_id;
+ uint8_t event_dev_id = adapter->eventdev_id;
+ struct rte_event ev[DMA_BATCH_SIZE];
+ unsigned int nb_enq, nb_enqueued;
+ uint16_t n;
+
+ if (adapter->mode == RTE_EVENT_DMA_ADAPTER_OP_NEW)
+ return 0;
+
+ nb_enqueued = 0;
+ for (nb_enq = 0; nb_enq < max_enq; nb_enq += n) {
+
+ if (unlikely(adapter->stop_enq_to_dma_dev)) {
+ nb_enqueued += edma_adapter_enq_flush(adapter);
+
+ if (unlikely(adapter->stop_enq_to_dma_dev))
+ break;
+ }
+
+ stats->event_poll_count++;
+ n = rte_event_dequeue_burst(event_dev_id, event_port_id, ev, DMA_BATCH_SIZE, 0);
+
+ if (!n)
+ break;
+
+ nb_enqueued += edma_enq_to_dma_dev(adapter, ev, n);
+ }
+
+ if ((++adapter->transmit_loop_count & (DMA_ENQ_FLUSH_THRESHOLD - 1)) == 0)
+ nb_enqueued += edma_adapter_enq_flush(adapter);
+
+ return nb_enqueued;
+}
+
+#define DMA_ADAPTER_MAX_EV_ENQ_RETRIES 100
+
+static inline uint16_t
+edma_ops_enqueue_burst(struct event_dma_adapter *adapter, struct rte_event_dma_adapter_op **ops,
+ uint16_t num)
+{
+ struct rte_event_dma_adapter_stats *stats = &adapter->dma_stats;
+ uint8_t event_port_id = adapter->event_port_id;
+ union rte_event_dma_metadata *m_data = NULL;
+ uint8_t event_dev_id = adapter->eventdev_id;
+ struct rte_event events[DMA_BATCH_SIZE];
+ uint16_t nb_enqueued, nb_ev;
+ uint8_t retry;
+ uint8_t i;
+
+ nb_ev = 0;
+ retry = 0;
+ nb_enqueued = 0;
+ num = RTE_MIN(num, DMA_BATCH_SIZE);
+ for (i = 0; i < num; i++) {
+ struct rte_event *ev = &events[nb_ev++];
+
+ /* Expected to have metadata appended to dma_op. */
+ m_data = (union rte_event_dma_metadata *)((uint8_t *)ops[i] +
+ sizeof(struct rte_event_dma_adapter_op));
+ if (unlikely(m_data == NULL)) {
+ if (ops[i] != NULL && ops[i]->op_mp != NULL)
+ rte_mempool_put(ops[i]->op_mp, ops[i]);
+ continue;
+ }
+
+ rte_memcpy(ev, &m_data->response_info, sizeof(*ev));
+ ev->event_ptr = ops[i];
+ ev->event_type = RTE_EVENT_TYPE_DMADEV;
+ if (adapter->implicit_release_disabled)
+ ev->op = RTE_EVENT_OP_FORWARD;
+ else
+ ev->op = RTE_EVENT_OP_NEW;
+ }
+
+ do {
+ nb_enqueued += rte_event_enqueue_burst(event_dev_id, event_port_id,
+ &events[nb_enqueued], nb_ev - nb_enqueued);
+
+ } while (retry++ < DMA_ADAPTER_MAX_EV_ENQ_RETRIES && nb_enqueued < nb_ev);
+
+ stats->event_enq_fail_count += nb_ev - nb_enqueued;
+ stats->event_enq_count += nb_enqueued;
+ stats->event_enq_retry_count += retry - 1;
+
+ return nb_enqueued;
+}
+
+static int
+edma_circular_buffer_flush_to_evdev(struct event_dma_adapter *adapter,
+ struct dma_ops_circular_buffer *bufp,
+ uint16_t *enqueue_count)
+{
+ struct rte_event_dma_adapter_op **ops = bufp->op_buffer;
+ uint16_t n = 0, nb_ops_flushed;
+ uint16_t *head = &bufp->head;
+ uint16_t *tail = &bufp->tail;
+
+ if (*tail > *head)
+ n = *tail - *head;
+ else if (*tail < *head)
+ n = bufp->size - *head;
+ else {
+ if (enqueue_count)
+ *enqueue_count = 0;
+ return 0; /* buffer empty */
+ }
+
+ if (enqueue_count && n > *enqueue_count)
+ n = *enqueue_count;
+
+ nb_ops_flushed = edma_ops_enqueue_burst(adapter, &ops[*head], n);
+ if (enqueue_count)
+ *enqueue_count = nb_ops_flushed;
+
+ bufp->count -= nb_ops_flushed;
+ if (!bufp->count) {
+ *head = 0;
+ *tail = 0;
+ return 0; /* buffer empty */
+ }
+
+ *head = (*head + nb_ops_flushed) % bufp->size;
+ return 1;
+}
+
+static void
+edma_ops_buffer_flush(struct event_dma_adapter *adapter)
+{
+ if (likely(adapter->ebuf.count == 0))
+ return;
+
+ while (edma_circular_buffer_flush_to_evdev(adapter, &adapter->ebuf, NULL))
+ ;
+}
+
+static inline unsigned int
+edma_adapter_deq_run(struct event_dma_adapter *adapter, unsigned int max_deq)
+{
+ struct rte_event_dma_adapter_stats *stats = &adapter->dma_stats;
+ struct dma_vchan_queue_info *vchan_queue;
+ struct dma_ops_circular_buffer *tq_buf;
+ struct rte_event_dma_adapter_op *ops;
+ uint16_t n, nb_deq, nb_enqueued, i;
+ struct dma_device_info *dev_info;
+ uint16_t vchan, num_vchan;
+ struct rte_dma_dev *dev;
+ uint16_t num_dma_dev;
+ int16_t dma_dev_id;
+ uint16_t index;
+ bool done;
+ bool err;
+
+ nb_deq = 0;
+ edma_ops_buffer_flush(adapter);
+
+ num_dma_dev = rte_dma_count_avail();
+ do {
+ done = true;
+
+ for (dma_dev_id = adapter->next_dmadev_id; dma_dev_id < num_dma_dev; dma_dev_id++) {
+ uint16_t queues = 0;
+ dev_info = &adapter->dma_devs[dma_dev_id];
+ dev = dev_info->dev;
+ if (unlikely(dev == NULL))
+ continue;
+
+ num_vchan = dev->data->dev_conf.nb_vchans;
+ for (vchan = dev_info->next_queue_pair_id; queues < num_vchan;
+ vchan = (vchan + 1) % num_vchan, queues++) {
+
+ vchan_queue = &dev_info->vchanq[vchan];
+ if (unlikely(vchan_queue == NULL || !vchan_queue->vq_enabled))
+ continue;
+
+ n = rte_dma_completed(dma_dev_id, vchan, DMA_BATCH_SIZE,
+ &index, &err);
+ if (!n)
+ continue;
+
+ done = false;
+ stats->dma_deq_count += n;
+
+ tq_buf = &dev_info->tqmap[vchan].dma_buf;
+
+ nb_enqueued = n;
+ if (unlikely(!adapter->ebuf.count))
+ edma_circular_buffer_flush_to_evdev(adapter, tq_buf,
+ &nb_enqueued);
+
+ if (likely(nb_enqueued == n))
+ goto check;
+
+ /* Failed to enqueue events case */
+ for (i = nb_enqueued; i < n; i++) {
+ ops = tq_buf->op_buffer[tq_buf->head];
+ edma_circular_buffer_add(&adapter->ebuf, ops);
+ tq_buf->head = (tq_buf->head + 1) % tq_buf->size;
+ }
+
+check:
+ nb_deq += n;
+ if (nb_deq >= max_deq) {
+ if ((vchan + 1) == num_vchan)
+ adapter->next_dmadev_id =
+ (dma_dev_id + 1) % num_dma_dev;
+
+ dev_info->next_queue_pair_id = (vchan + 1) % num_vchan;
+
+ return nb_deq;
+ }
+ }
+ }
+ adapter->next_dmadev_id = 0;
+
+ } while (done == false);
+
+ return nb_deq;
+}
+
+static int
+edma_adapter_run(struct event_dma_adapter *adapter, unsigned int max_ops)
+{
+ unsigned int ops_left = max_ops;
+
+ while (ops_left > 0) {
+ unsigned int e_cnt, d_cnt;
+
+ e_cnt = edma_adapter_deq_run(adapter, ops_left);
+ ops_left -= RTE_MIN(ops_left, e_cnt);
+
+ d_cnt = edma_adapter_enq_run(adapter, ops_left);
+ ops_left -= RTE_MIN(ops_left, d_cnt);
+
+ if (e_cnt == 0 && d_cnt == 0)
+ break;
+ }
+
+ if (ops_left == max_ops) {
+ rte_event_maintain(adapter->eventdev_id, adapter->event_port_id, 0);
+ return -EAGAIN;
+ } else
+ return 0;
+}
+
+static int
+edma_service_func(void *args)
+{
+ struct event_dma_adapter *adapter = args;
+ int ret;
+
+ if (rte_spinlock_trylock(&adapter->lock) == 0)
+ return 0;
+ ret = edma_adapter_run(adapter, adapter->max_nb);
+ rte_spinlock_unlock(&adapter->lock);
+
+ return ret;
+}
+
+static int
+edma_init_service(struct event_dma_adapter *adapter, uint8_t id)
+{
+ struct rte_event_dma_adapter_conf adapter_conf;
+ struct rte_service_spec service;
+ uint32_t impl_rel;
+ int ret;
+
+ if (adapter->service_initialized)
+ return 0;
+
+ memset(&service, 0, sizeof(service));
+ snprintf(service.name, DMA_ADAPTER_NAME_LEN, "rte_event_dma_adapter_%d", id);
+ service.socket_id = adapter->socket_id;
+ service.callback = edma_service_func;
+ service.callback_userdata = adapter;
+
+ /* Service function handles locking for queue add/del updates */
+ service.capabilities = RTE_SERVICE_CAP_MT_SAFE;
+ ret = rte_service_component_register(&service, &adapter->service_id);
+ if (ret) {
+ RTE_EDEV_LOG_ERR("failed to register service %s err = %" PRId32, service.name, ret);
+ return ret;
+ }
+
+ ret = adapter->conf_cb(id, adapter->eventdev_id, &adapter_conf, adapter->conf_arg);
+ if (ret) {
+ RTE_EDEV_LOG_ERR("configuration callback failed err = %" PRId32, ret);
+ return ret;
+ }
+
+ adapter->max_nb = adapter_conf.max_nb;
+ adapter->event_port_id = adapter_conf.event_port_id;
+
+ if (rte_event_port_attr_get(adapter->eventdev_id, adapter->event_port_id,
+ RTE_EVENT_PORT_ATTR_IMPLICIT_RELEASE_DISABLE, &impl_rel)) {
+ RTE_EDEV_LOG_ERR("Failed to get port info for eventdev %" PRId32,
+ adapter->eventdev_id);
+ edma_circular_buffer_free(&adapter->ebuf);
+ rte_free(adapter);
+ return -EINVAL;
+ }
+
+ adapter->implicit_release_disabled = (uint8_t)impl_rel;
+ adapter->service_initialized = 1;
+
+ return ret;
+}
+
+static void
+edma_update_vchanq_info(struct event_dma_adapter *adapter, struct dma_device_info *dev_info,
+ uint16_t vchan, uint8_t add)
+{
+ struct dma_vchan_queue_info *vchan_info;
+ struct dma_vchan_queue_info *tqmap_info;
+ int enabled;
+ uint16_t i;
+
+ if (dev_info->vchanq == NULL)
+ return;
+
+ if (vchan == RTE_DMA_ALL_VCHAN) {
+ for (i = 0; i < dev_info->dev->data->dev_conf.nb_vchans; i++)
+ edma_update_vchanq_info(adapter, dev_info, i, add);
+ } else {
+ tqmap_info = &dev_info->tqmap[vchan];
+ vchan_info = &dev_info->vchanq[vchan];
+ enabled = vchan_info->vq_enabled;
+ if (add) {
+ adapter->nb_vchanq += !enabled;
+ dev_info->num_vchanq += !enabled;
+ } else {
+ adapter->nb_vchanq -= enabled;
+ dev_info->num_vchanq -= enabled;
+ }
+ vchan_info->vq_enabled = !!add;
+ tqmap_info->vq_enabled = !!add;
+ }
+}
+
+static int
+edma_add_queue_pair(struct event_dma_adapter *adapter, int16_t dma_dev_id, uint16_t vchan)
+{
+ struct dma_device_info *dev_info = &adapter->dma_devs[dma_dev_id];
+ struct dma_vchan_queue_info *vchanq;
+ struct dma_vchan_queue_info *tqmap;
+ uint16_t nb_vchans;
+ uint32_t i;
+
+ if (dev_info->vchanq == NULL) {
+ nb_vchans = dev_info->dev->data->dev_conf.nb_vchans;
+
+ dev_info->vchanq = rte_zmalloc_socket(adapter->mem_name,
+ nb_vchans * sizeof(struct dma_vchan_queue_info),
+ 0, adapter->socket_id);
+ if (dev_info->vchanq == NULL)
+ return -ENOMEM;
+
+ dev_info->tqmap = rte_zmalloc_socket(adapter->mem_name,
+ nb_vchans * sizeof(struct dma_vchan_queue_info),
+ 0, adapter->socket_id);
+ if (dev_info->tqmap == NULL)
+ return -ENOMEM;
+
+ for (i = 0; i < nb_vchans; i++) {
+ vchanq = &dev_info->vchanq[i];
+
+ if (edma_circular_buffer_init("dma_dev_circular_buffer", &vchanq->dma_buf,
+ DMA_ADAPTER_OPS_BUFFER_SIZE)) {
+ RTE_EDEV_LOG_ERR("Failed to get memory for dma_dev buffer");
+ rte_free(vchanq);
+ return -ENOMEM;
+ }
+
+ tqmap = &dev_info->tqmap[i];
+ if (edma_circular_buffer_init("dma_dev_circular_trans_buf", &tqmap->dma_buf,
+ DMA_ADAPTER_OPS_BUFFER_SIZE)) {
+ RTE_EDEV_LOG_ERR(
+ "Failed to get memory for dma_dev transction buffer");
+ rte_free(tqmap);
+ return -ENOMEM;
+ }
+ }
+ }
+
+ if (vchan == RTE_DMA_ALL_VCHAN) {
+ for (i = 0; i < dev_info->dev->data->dev_conf.nb_vchans; i++)
+ edma_update_vchanq_info(adapter, dev_info, i, 1);
+ } else
+ edma_update_vchanq_info(adapter, dev_info, vchan, 1);
+
+ return 0;
+}
+
+int
+rte_event_dma_adapter_vchan_queue_add(uint8_t id, int16_t dma_dev_id, uint16_t vchan,
+ const struct rte_event *event)
+{
+ struct event_dma_adapter *adapter;
+ struct dma_device_info *dev_info;
+ uint32_t cap;
+ int ret;
+
+ EVENT_DMA_ADAPTER_ID_VALID_OR_ERR_RET(id, -EINVAL);
+
+ if (!rte_dma_is_valid(dma_dev_id)) {
+ RTE_EDEV_LOG_ERR("Invalid dma_dev_id = %" PRIu8, dma_dev_id);
+ return -EINVAL;
+ }
+
+ adapter = edma_id_to_adapter(id);
+ if (adapter == NULL)
+ return -EINVAL;
+
+ ret = rte_event_dma_adapter_caps_get(adapter->eventdev_id, dma_dev_id, &cap);
+ if (ret) {
+ RTE_EDEV_LOG_ERR("Failed to get adapter caps dev %u dma_dev %u", id, dma_dev_id);
+ return ret;
+ }
+
+ if ((cap & RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_QP_EV_BIND) && (event == NULL)) {
+ RTE_EDEV_LOG_ERR("Event can not be NULL for dma_dev_id = %u", dma_dev_id);
+ return -EINVAL;
+ }
+
+ dev_info = &adapter->dma_devs[dma_dev_id];
+ if (vchan != RTE_DMA_ALL_VCHAN && vchan >= dev_info->dev->data->dev_conf.nb_vchans) {
+ RTE_EDEV_LOG_ERR("Invalid vhcan %u", vchan);
+ return -EINVAL;
+ }
+
+ /* In case HW cap is RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP_FWD, no
+ * need of service core as HW supports event forward capability.
+ */
+ if ((cap & RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP_FWD) ||
+ (cap & RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_QP_EV_BIND &&
+ adapter->mode == RTE_EVENT_DMA_ADAPTER_OP_NEW) ||
+ (cap & RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP_NEW &&
+ adapter->mode == RTE_EVENT_DMA_ADAPTER_OP_NEW)) {
+ if (dev_info->vchanq == NULL) {
+ dev_info->vchanq = rte_zmalloc_socket(adapter->mem_name,
+ dev_info->dev->data->dev_conf.nb_vchans *
+ sizeof(struct dma_vchan_queue_info),
+ 0, adapter->socket_id);
+ if (dev_info->vchanq == NULL) {
+ printf("Queue pair add not supported\n");
+ return -ENOMEM;
+ }
+ }
+
+ if (dev_info->tqmap == NULL) {
+ dev_info->tqmap = rte_zmalloc_socket(adapter->mem_name,
+ dev_info->dev->data->dev_conf.nb_vchans *
+ sizeof(struct dma_vchan_queue_info),
+ 0, adapter->socket_id);
+ if (dev_info->tqmap == NULL) {
+ printf("tq pair add not supported\n");
+ return -ENOMEM;
+ }
+ }
+
+ edma_update_vchanq_info(adapter, &adapter->dma_devs[dma_dev_id], vchan, 1);
+ }
+
+ /* In case HW cap is RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP_NEW, or SW adapter, initiate
+ * services so the application can choose which ever way it wants to use the adapter.
+ *
+ * Case 1: RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP_NEW. Application may wants to use one
+ * of below two modes
+ *
+ * a. OP_FORWARD mode -> HW Dequeue + SW enqueue
+ * b. OP_NEW mode -> HW Dequeue
+ *
+ * Case 2: No HW caps, use SW adapter
+ *
+ * a. OP_FORWARD mode -> SW enqueue & dequeue
+ * b. OP_NEW mode -> SW Dequeue
+ */
+ if ((cap & RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP_NEW &&
+ !(cap & RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP_FWD) &&
+ adapter->mode == RTE_EVENT_DMA_ADAPTER_OP_FORWARD) ||
+ (!(cap & RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP_NEW) &&
+ !(cap & RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP_FWD) &&
+ !(cap & RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_QP_EV_BIND))) {
+ rte_spinlock_lock(&adapter->lock);
+ ret = edma_init_service(adapter, id);
+ if (ret == 0)
+ ret = edma_add_queue_pair(adapter, dma_dev_id, vchan);
+ rte_spinlock_unlock(&adapter->lock);
+
+ if (ret)
+ return ret;
+
+ rte_service_component_runstate_set(adapter->service_id, 1);
+ }
+
+ return 0;
+}
+
+int
+rte_event_dma_adapter_vchan_queue_del(uint8_t id, int16_t dma_dev_id, uint16_t vchan)
+{
+ struct event_dma_adapter *adapter;
+ struct dma_device_info *dev_info;
+ uint32_t cap;
+ int ret;
+
+ EVENT_DMA_ADAPTER_ID_VALID_OR_ERR_RET(id, -EINVAL);
+
+ if (!rte_dma_is_valid(dma_dev_id)) {
+ RTE_EDEV_LOG_ERR("Invalid dma_dev_id = %" PRIu8, dma_dev_id);
+ return -EINVAL;
+ }
+
+ adapter = edma_id_to_adapter(id);
+ if (adapter == NULL)
+ return -EINVAL;
+
+ ret = rte_event_dma_adapter_caps_get(adapter->eventdev_id, dma_dev_id, &cap);
+ if (ret)
+ return ret;
+
+ dev_info = &adapter->dma_devs[dma_dev_id];
+
+ if (vchan != RTE_DMA_ALL_VCHAN && vchan >= dev_info->dev->data->dev_conf.nb_vchans) {
+ RTE_EDEV_LOG_ERR("Invalid vhcan %" PRIu16, vchan);
+ return -EINVAL;
+ }
+
+ if ((cap & RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP_FWD) ||
+ (cap & RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP_NEW &&
+ adapter->mode == RTE_EVENT_DMA_ADAPTER_OP_NEW)) {
+ edma_update_vchanq_info(adapter, dev_info, vchan, 0);
+ if (dev_info->num_vchanq == 0) {
+ rte_free(dev_info->vchanq);
+ dev_info->vchanq = NULL;
+ }
+ } else {
+ if (adapter->nb_vchanq == 0)
+ return 0;
+
+ rte_spinlock_lock(&adapter->lock);
+ edma_update_vchanq_info(adapter, dev_info, vchan, 0);
+
+ if (dev_info->num_vchanq == 0) {
+ rte_free(dev_info->vchanq);
+ rte_free(dev_info->tqmap);
+ dev_info->vchanq = NULL;
+ dev_info->tqmap = NULL;
+ }
+
+ rte_spinlock_unlock(&adapter->lock);
+ rte_service_component_runstate_set(adapter->service_id, adapter->nb_vchanq);
+ }
+
+ return ret;
+}
+
+int
+rte_event_dma_adapter_service_id_get(uint8_t id, uint32_t *service_id)
+{
+ struct event_dma_adapter *adapter;
+
+ EVENT_DMA_ADAPTER_ID_VALID_OR_ERR_RET(id, -EINVAL);
+
+ adapter = edma_id_to_adapter(id);
+ if (adapter == NULL || service_id == NULL)
+ return -EINVAL;
+
+ if (adapter->service_initialized)
+ *service_id = adapter->service_id;
+
+ return adapter->service_initialized ? 0 : -ESRCH;
+}
+
+static int
+edma_adapter_ctrl(uint8_t id, int start)
+{
+ struct event_dma_adapter *adapter;
+ struct dma_device_info *dev_info;
+ struct rte_eventdev *dev;
+ uint16_t num_dma_dev;
+ int stop = !start;
+ int use_service;
+ uint32_t i;
+
+ use_service = 0;
+ EVENT_DMA_ADAPTER_ID_VALID_OR_ERR_RET(id, -EINVAL);
+ adapter = edma_id_to_adapter(id);
+ if (adapter == NULL)
+ return -EINVAL;
+
+ num_dma_dev = rte_dma_count_avail();
+ dev = &rte_eventdevs[adapter->eventdev_id];
+
+ for (i = 0; i < num_dma_dev; i++) {
+ dev_info = &adapter->dma_devs[i];
+ /* start check for num queue pairs */
+ if (start && !dev_info->num_vchanq)
+ continue;
+ /* stop check if dev has been started */
+ if (stop && !dev_info->dev_started)
+ continue;
+ use_service |= !dev_info->internal_event_port;
+ dev_info->dev_started = start;
+ if (dev_info->internal_event_port == 0)
+ continue;
+ start ? (*dev->dev_ops->dma_adapter_start)(dev, &dev_info->dev[i]) :
+ (*dev->dev_ops->dma_adapter_stop)(dev, &dev_info->dev[i]);
+ }
+
+ if (use_service)
+ rte_service_runstate_set(adapter->service_id, start);
+
+ return 0;
+}
+
+int
+rte_event_dma_adapter_start(uint8_t id)
+{
+ struct event_dma_adapter *adapter;
+
+ EVENT_DMA_ADAPTER_ID_VALID_OR_ERR_RET(id, -EINVAL);
+
+ adapter = edma_id_to_adapter(id);
+ if (adapter == NULL)
+ return -EINVAL;
+
+ return edma_adapter_ctrl(id, 1);
+}
+
+int
+rte_event_dma_adapter_stop(uint8_t id)
+{
+ return edma_adapter_ctrl(id, 0);
+}
+
+#define DEFAULT_MAX_NB 128
+
+int
+rte_event_dma_adapter_runtime_params_init(struct rte_event_dma_adapter_runtime_params *params)
+{
+ if (params == NULL)
+ return -EINVAL;
+
+ memset(params, 0, sizeof(*params));
+ params->max_nb = DEFAULT_MAX_NB;
+
+ return 0;
+}
+
+static int
+dma_adapter_cap_check(struct event_dma_adapter *adapter)
+{
+ uint32_t caps;
+ int ret;
+
+ if (!adapter->nb_vchanq)
+ return -EINVAL;
+
+ ret = rte_event_dma_adapter_caps_get(adapter->eventdev_id, adapter->next_dmadev_id, &caps);
+ if (ret) {
+ RTE_EDEV_LOG_ERR("Failed to get adapter caps dev %" PRIu8 " cdev %" PRIu8,
+ adapter->eventdev_id, adapter->next_dmadev_id);
+ return ret;
+ }
+
+ if ((caps & RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP_FWD) ||
+ (caps & RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP_NEW))
+ return -ENOTSUP;
+
+ return 0;
+}
+
+int
+rte_event_dma_adapter_runtime_params_set(uint8_t id,
+ struct rte_event_dma_adapter_runtime_params *params)
+{
+ struct event_dma_adapter *adapter;
+ int ret;
+
+ EVENT_DMA_ADAPTER_ID_VALID_OR_ERR_RET(id, -EINVAL);
+
+ if (params == NULL) {
+ RTE_EDEV_LOG_ERR("params pointer is NULL\n");
+ return -EINVAL;
+ }
+
+ adapter = edma_id_to_adapter(id);
+ if (adapter == NULL)
+ return -EINVAL;
+
+ ret = dma_adapter_cap_check(adapter);
+ if (ret)
+ return ret;
+
+ rte_spinlock_lock(&adapter->lock);
+ adapter->max_nb = params->max_nb;
+ rte_spinlock_unlock(&adapter->lock);
+
+ return 0;
+}
+
+int
+rte_event_dma_adapter_runtime_params_get(uint8_t id,
+ struct rte_event_dma_adapter_runtime_params *params)
+{
+ struct event_dma_adapter *adapter;
+ int ret;
+
+ EVENT_DMA_ADAPTER_ID_VALID_OR_ERR_RET(id, -EINVAL);
+
+ if (params == NULL) {
+ RTE_EDEV_LOG_ERR("params pointer is NULL\n");
+ return -EINVAL;
+ }
+
+ adapter = edma_id_to_adapter(id);
+ if (adapter == NULL)
+ return -EINVAL;
+
+ ret = dma_adapter_cap_check(adapter);
+ if (ret)
+ return ret;
+
+ params->max_nb = adapter->max_nb;
+
+ return 0;
+}
+
+int
+rte_event_dma_adapter_stats_get(uint8_t id, struct rte_event_dma_adapter_stats *stats)
+{
+ struct rte_event_dma_adapter_stats dev_stats_sum = {0};
+ struct rte_event_dma_adapter_stats dev_stats;
+ struct event_dma_adapter *adapter;
+ struct dma_device_info *dev_info;
+ struct rte_eventdev *dev;
+ uint16_t num_dma_dev;
+ uint32_t i;
+ int ret;
+
+ EVENT_DMA_ADAPTER_ID_VALID_OR_ERR_RET(id, -EINVAL);
+
+ adapter = edma_id_to_adapter(id);
+ if (adapter == NULL || stats == NULL)
+ return -EINVAL;
+
+ num_dma_dev = rte_dma_count_avail();
+ dev = &rte_eventdevs[adapter->eventdev_id];
+ memset(stats, 0, sizeof(*stats));
+ for (i = 0; i < num_dma_dev; i++) {
+ dev_info = &adapter->dma_devs[i];
+
+ if (dev_info->internal_event_port == 0 ||
+ dev->dev_ops->dma_adapter_stats_get == NULL)
+ continue;
+
+ ret = (*dev->dev_ops->dma_adapter_stats_get)(dev, dev_info->dev, &dev_stats);
+ if (ret)
+ continue;
+
+ dev_stats_sum.dma_deq_count += dev_stats.dma_deq_count;
+ dev_stats_sum.event_enq_count += dev_stats.event_enq_count;
+ }
+
+ stats->dma_deq_count += dev_stats_sum.dma_deq_count;
+ stats->event_enq_count += dev_stats_sum.event_enq_count;
+
+ return 0;
+}
+
+int
+rte_event_dma_adapter_stats_reset(uint8_t id)
+{
+ struct event_dma_adapter *adapter;
+ struct dma_device_info *dev_info;
+ struct rte_eventdev *dev;
+ uint16_t num_dma_dev;
+ uint32_t i;
+
+ EVENT_DMA_ADAPTER_ID_VALID_OR_ERR_RET(id, -EINVAL);
+
+ adapter = edma_id_to_adapter(id);
+ if (adapter == NULL)
+ return -EINVAL;
+
+ num_dma_dev = rte_dma_count_avail();
+ dev = &rte_eventdevs[adapter->eventdev_id];
+ for (i = 0; i < num_dma_dev; i++) {
+ dev_info = &adapter->dma_devs[i];
+
+ if (dev_info->internal_event_port == 0 ||
+ dev->dev_ops->dma_adapter_stats_reset == NULL)
+ continue;
+
+ (*dev->dev_ops->dma_adapter_stats_reset)(dev, dev_info->dev);
+ }
+
+ memset(&adapter->dma_stats, 0, sizeof(adapter->dma_stats));
+
+ return 0;
+}
+
+uint16_t
+rte_event_dma_adapter_enqueue(uint8_t dev_id, uint8_t port_id, struct rte_event ev[],
+ uint16_t nb_events)
+{
+ const struct rte_event_fp_ops *fp_ops;
+ void *port;
+
+ fp_ops = &rte_event_fp_ops[dev_id];
+ port = fp_ops->data[port_id];
+
+ return fp_ops->dma_enqueue(port, ev, nb_events);
+}
diff --git a/lib/eventdev/rte_event_dma_adapter.h b/lib/eventdev/rte_event_dma_adapter.h
index c667398d08..d3d8b2dbbd 100644
--- a/lib/eventdev/rte_event_dma_adapter.h
+++ b/lib/eventdev/rte_event_dma_adapter.h
@@ -203,16 +203,39 @@ struct rte_event_dma_request {
* information. Application is expected to fill in struct rte_event response_info.
*/
- int16_t dmadev_id;
+ int16_t dma_dev_id;
/**< DMA device ID to be used */
- uint16_t queue_pair_id;
- /**< DMA queue pair ID to be used */
+ uint16_t vchan;
+ /**< DMA vchan ID to be used */
uint32_t rsvd;
/**< Reserved bits */
};
+/**
+ * DMA event metadata structure will be filled by application
+ * to provide dma request and event response information.
+ *
+ * If dma events are enqueued using a HW mechanism, the dmadev
+ * PMD will use the event response information to set up the event
+ * that is enqueued back to eventdev after completion of the dma
+ * operation. If the transfer is done by SW, event response information
+ * will be used by the adapter.
+ */
+union rte_event_dma_metadata {
+ struct rte_event_dma_request request_info;
+ /**< Request information to be filled in by application
+ * for RTE_EVENT_DMA_ADAPTER_OP_FORWARD mode.
+ * First 8 bytes of request_info is reserved for response_info.
+ */
+ struct rte_event response_info;
+ /**< Response information to be filled in by application
+ * for RTE_EVENT_DMA_ADAPTER_OP_NEW and
+ * RTE_EVENT_DMA_ADAPTER_OP_FORWARD mode.
+ */
+};
+
/**
* Adapter configuration structure that the adapter configuration callback function is expected to
* fill out.
@@ -406,9 +429,9 @@ int rte_event_dma_adapter_event_port_get(uint8_t id, uint8_t *event_port_id);
* Adapter identifier.
* @param dmadev_id
* dmadev identifier.
- * @param queue_pair_id
- * DMA device vchan queue identifier. If queue_pair_id is set -1, adapter adds all the
- * preconfigured queue pairs to the instance.
+ * @param vchan
+ * DMA device vchan queue identifier. If vchan is set -1, adapter adds all the
+ * preconfigured vchan queue to the instance.
* @param event
* If HW supports dmadev queue pair to event queue binding, application is expected to fill in
* event information, else it will be NULL.
@@ -419,7 +442,7 @@ int rte_event_dma_adapter_event_port_get(uint8_t id, uint8_t *event_port_id);
* - <0: Error code on failure.
*/
__rte_experimental
-int rte_event_dma_adapter_vchan_queue_add(uint8_t id, int16_t dmadev_id, int32_t queue_pair_id,
+int rte_event_dma_adapter_vchan_queue_add(uint8_t id, int16_t dmadev_id, uint16_t vchan,
const struct rte_event *event);
/**
@@ -432,7 +455,7 @@ int rte_event_dma_adapter_vchan_queue_add(uint8_t id, int16_t dmadev_id, int32_t
* Adapter identifier.
* @param dmadev_id
* DMA device identifier.
- * @param queue_pair_id
+ * @param vchan
* DMA device vchan queue identifier.
*
* @return
@@ -440,7 +463,7 @@ int rte_event_dma_adapter_vchan_queue_add(uint8_t id, int16_t dmadev_id, int32_t
* - <0: Error code on failure.
*/
__rte_experimental
-int rte_event_dma_adapter_vchan_queue_del(uint8_t id, int16_t dmadev_id, int32_t queue_pair_id);
+int rte_event_dma_adapter_vchan_queue_del(uint8_t id, int16_t dmadev_id, uint16_t vchan);
/**
* @warning
--
2.25.1
^ permalink raw reply [flat|nested] 110+ messages in thread
* [PATCH v1 4/7] app/test: add event DMA adapter auto-test
2023-09-19 13:42 [PATCH v1 1/7] eventdev: introduce DMA event adapter library Amit Prakash Shukla
2023-09-19 13:42 ` [PATCH v1 2/7] eventdev: api to get DMA capabilities Amit Prakash Shukla
2023-09-19 13:42 ` [PATCH v1 3/7] eventdev: add DMA adapter implementation Amit Prakash Shukla
@ 2023-09-19 13:42 ` Amit Prakash Shukla
2023-09-19 13:42 ` [PATCH v1 5/7] common/cnxk: dma result to an offset of the event Amit Prakash Shukla
` (4 subsequent siblings)
7 siblings, 0 replies; 110+ messages in thread
From: Amit Prakash Shukla @ 2023-09-19 13:42 UTC (permalink / raw)
Cc: dev, jerinj, fengchengwen, kevin.laatz, bruce.richardson,
conor.walsh, vattunuru, g.singh, sachin.saxena, hemant.agrawal,
cheng1.jiang, ndabilpuram, anoobj, mb, Amit Prakash Shukla
Added testsuite to test the dma adapter functionality.
The testsuite detects event and DMA device capability
and accordingly dma adapter is configured and modes are
tested.
Signed-off-by: Amit Prakash Shukla <amitprakashs@marvell.com>
---
app/test/meson.build | 1 +
app/test/test_event_dma_adapter.c | 814 ++++++++++++++++++++++++++++++
2 files changed, 815 insertions(+)
create mode 100644 app/test/test_event_dma_adapter.c
diff --git a/app/test/meson.build b/app/test/meson.build
index 05bae9216d..eccd3b72d8 100644
--- a/app/test/meson.build
+++ b/app/test/meson.build
@@ -66,6 +66,7 @@ source_file_deps = {
'test_errno.c': [],
'test_ethdev_link.c': ['ethdev'],
'test_event_crypto_adapter.c': ['cryptodev', 'eventdev', 'bus_vdev'],
+ 'test_event_dma_adapter.c': ['dmadev', 'eventdev'],
'test_event_eth_rx_adapter.c': ['ethdev', 'eventdev', 'bus_vdev'],
'test_event_eth_tx_adapter.c': ['bus_vdev', 'ethdev', 'net_ring', 'eventdev'],
'test_event_ring.c': ['eventdev'],
diff --git a/app/test/test_event_dma_adapter.c b/app/test/test_event_dma_adapter.c
new file mode 100644
index 0000000000..5b8649fe14
--- /dev/null
+++ b/app/test/test_event_dma_adapter.c
@@ -0,0 +1,814 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright (c) 2023 Marvell.
+ */
+
+#include "test.h"
+#include <string.h>
+#include <rte_common.h>
+#include <rte_malloc.h>
+#include <rte_mempool.h>
+#include <rte_mbuf.h>
+#include <rte_random.h>
+
+#ifdef RTE_EXEC_ENV_WINDOWS
+static int
+test_event_dma_adapter(void)
+{
+ printf("event_dma_adapter not supported on Windows, skipping test\n");
+ return TEST_SKIPPED;
+}
+
+#else
+
+#include <rte_bus_vdev.h>
+#include <rte_dmadev.h>
+#include <rte_eventdev.h>
+#include <rte_event_dma_adapter.h>
+#include <rte_service.h>
+
+#define NUM_MBUFS (8191)
+#define MBUF_CACHE_SIZE (256)
+#define TEST_APP_PORT_ID 0
+#define TEST_APP_EV_QUEUE_ID 0
+#define TEST_APP_EV_PRIORITY 0
+#define TEST_APP_EV_FLOWID 0xAABB
+#define TEST_DMA_EV_QUEUE_ID 1
+#define TEST_ADAPTER_ID 0
+#define TEST_DMA_DEV_ID 0
+#define TEST_DMA_VCHAN_ID 0
+#define PACKET_LENGTH 64
+#define NB_TEST_PORTS 1
+#define NB_TEST_QUEUES 2
+#define NUM_CORES 1
+#define DMA_OP_POOL_SIZE 64
+#define TEST_MAX_OP 64
+#define TEST_RINGSIZE 512
+
+#define MBUF_SIZE (RTE_PKTMBUF_HEADROOM + PACKET_LENGTH)
+
+/* Handle log statements in same manner as test macros */
+#define LOG_DBG(...) RTE_LOG(DEBUG, EAL, __VA_ARGS__)
+
+struct event_dma_adapter_test_params {
+ struct rte_mempool *src_mbuf_pool;
+ struct rte_mempool *dst_mbuf_pool;
+ struct rte_mempool *op_mpool;
+ uint8_t dma_event_port_id;
+ uint8_t internal_port_op_fwd;
+};
+
+struct rte_event dma_response_info = {
+ .queue_id = TEST_APP_EV_QUEUE_ID,
+ .sched_type = RTE_SCHED_TYPE_ATOMIC,
+ .flow_id = TEST_APP_EV_FLOWID,
+ .priority = TEST_APP_EV_PRIORITY
+};
+
+struct rte_event_dma_request dma_request_info = {
+ .dma_dev_id = TEST_DMA_DEV_ID,
+ .vchan = TEST_DMA_VCHAN_ID
+};
+
+static struct event_dma_adapter_test_params params;
+static uint8_t dma_adapter_setup_done;
+static uint32_t slcore_id;
+static int evdev;
+
+static int
+send_recv_ev(struct rte_event *ev)
+{
+ struct rte_event recv_ev[TEST_MAX_OP] = {0};
+ struct rte_event_dma_adapter_op *op;
+ uint16_t nb_enqueued = 0;
+ int ret, i = 0;
+
+ if (params.internal_port_op_fwd) {
+ nb_enqueued = rte_event_dma_adapter_enqueue(evdev, TEST_APP_PORT_ID, ev,
+ TEST_MAX_OP);
+ } else {
+ while (nb_enqueued < TEST_MAX_OP) {
+ nb_enqueued += rte_event_enqueue_burst(evdev, TEST_APP_PORT_ID,
+ &ev[nb_enqueued], TEST_MAX_OP -
+ nb_enqueued);
+ }
+ }
+
+ TEST_ASSERT_EQUAL(nb_enqueued, TEST_MAX_OP, "Failed to send event to dma adapter\n");
+
+ while (i < TEST_MAX_OP) {
+ if (rte_event_dequeue_burst(evdev, TEST_APP_PORT_ID, &recv_ev[i], 1, 0) != 1)
+ continue;
+
+ op = recv_ev[i].event_ptr;
+ ret = memcmp((uint8_t *)op->src_seg->addr, (uint8_t *)op->dst_seg->addr,
+ op->src_seg->length);
+ TEST_ASSERT_EQUAL(ret, 0, "Data mismatch for dma adapter\n");
+ i++;
+ }
+
+ return TEST_SUCCESS;
+}
+
+static int
+test_dma_adapter_stats(void)
+{
+ struct rte_event_dma_adapter_stats stats;
+
+ rte_event_dma_adapter_stats_get(TEST_ADAPTER_ID, &stats);
+ printf(" +------------------------------------------------------+\n");
+ printf(" + DMA adapter stats for instance %u:\n", TEST_ADAPTER_ID);
+ printf(" + Event port poll count %" PRIx64 "\n",
+ stats.event_poll_count);
+ printf(" + Event dequeue count %" PRIx64 "\n",
+ stats.event_deq_count);
+ printf(" + DMA dev enqueue count %" PRIx64 "\n",
+ stats.dma_enq_count);
+ printf(" + DMA dev enqueue failed count %" PRIx64 "\n",
+ stats.dma_enq_fail_count);
+ printf(" + DMA dev dequeue count %" PRIx64 "\n",
+ stats.dma_deq_count);
+ printf(" + Event enqueue count %" PRIx64 "\n",
+ stats.event_enq_count);
+ printf(" + Event enqueue retry count %" PRIx64 "\n",
+ stats.event_enq_retry_count);
+ printf(" + Event enqueue fail count %" PRIx64 "\n",
+ stats.event_enq_fail_count);
+ printf(" +------------------------------------------------------+\n");
+
+ rte_event_dma_adapter_stats_reset(TEST_ADAPTER_ID);
+ return TEST_SUCCESS;
+}
+
+static int
+test_dma_adapter_params(void)
+{
+ struct rte_event_dma_adapter_runtime_params in_params;
+ struct rte_event_dma_adapter_runtime_params out_params;
+ struct rte_event event;
+ uint32_t cap;
+ int err, rc;
+
+ err = rte_event_dma_adapter_caps_get(evdev, TEST_DMA_DEV_ID, &cap);
+ TEST_ASSERT_SUCCESS(err, "Failed to get adapter capabilities\n");
+
+ if (cap & RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_QP_EV_BIND) {
+ err = rte_event_dma_adapter_vchan_queue_add(TEST_ADAPTER_ID, TEST_DMA_DEV_ID,
+ TEST_DMA_VCHAN_ID, &event);
+ } else
+ err = rte_event_dma_adapter_vchan_queue_add(TEST_ADAPTER_ID, TEST_DMA_DEV_ID,
+ TEST_DMA_VCHAN_ID, NULL);
+
+ TEST_ASSERT_SUCCESS(err, "Failed to add queue pair\n");
+
+ err = rte_event_dma_adapter_runtime_params_init(&in_params);
+ TEST_ASSERT(err == 0, "Expected 0 got %d", err);
+ err = rte_event_dma_adapter_runtime_params_init(&out_params);
+ TEST_ASSERT(err == 0, "Expected 0 got %d", err);
+
+ /* Case 1: Get the default value of mbufs processed by adapter */
+ err = rte_event_dma_adapter_runtime_params_get(TEST_ADAPTER_ID, &out_params);
+ if (err == -ENOTSUP) {
+ rc = TEST_SKIPPED;
+ goto queue_pair_del;
+ }
+ TEST_ASSERT(err == 0, "Expected 0 got %d", err);
+
+ /* Case 2: Set max_nb = 32 (=BATCH_SEIZE) */
+ in_params.max_nb = 32;
+
+ err = rte_event_dma_adapter_runtime_params_set(TEST_ADAPTER_ID, &in_params);
+ TEST_ASSERT(err == 0, "Expected 0 got %d", err);
+
+ err = rte_event_dma_adapter_runtime_params_get(TEST_ADAPTER_ID, &out_params);
+ TEST_ASSERT(err == 0, "Expected 0 got %d", err);
+ TEST_ASSERT(in_params.max_nb == out_params.max_nb, "Expected %u got %u",
+ in_params.max_nb, out_params.max_nb);
+
+ /* Case 3: Set max_nb = 192 */
+ in_params.max_nb = 192;
+
+ err = rte_event_dma_adapter_runtime_params_set(TEST_ADAPTER_ID, &in_params);
+ TEST_ASSERT(err == 0, "Expected 0 got %d", err);
+
+ err = rte_event_dma_adapter_runtime_params_get(TEST_ADAPTER_ID, &out_params);
+ TEST_ASSERT(err == 0, "Expected 0 got %d", err);
+ TEST_ASSERT(in_params.max_nb == out_params.max_nb, "Expected %u got %u",
+ in_params.max_nb, out_params.max_nb);
+
+ /* Case 4: Set max_nb = 256 */
+ in_params.max_nb = 256;
+
+ err = rte_event_dma_adapter_runtime_params_set(TEST_ADAPTER_ID, &in_params);
+ TEST_ASSERT(err == 0, "Expected 0 got %d", err);
+
+ err = rte_event_dma_adapter_runtime_params_get(TEST_ADAPTER_ID, &out_params);
+ TEST_ASSERT(err == 0, "Expected 0 got %d", err);
+ TEST_ASSERT(in_params.max_nb == out_params.max_nb, "Expected %u got %u",
+ in_params.max_nb, out_params.max_nb);
+
+ /* Case 5: Set max_nb = 30(<BATCH_SIZE) */
+ in_params.max_nb = 30;
+
+ err = rte_event_dma_adapter_runtime_params_set(TEST_ADAPTER_ID, &in_params);
+ TEST_ASSERT(err == 0, "Expected 0 got %d", err);
+
+ err = rte_event_dma_adapter_runtime_params_get(TEST_ADAPTER_ID, &out_params);
+ TEST_ASSERT(err == 0, "Expected 0 got %d", err);
+ TEST_ASSERT(in_params.max_nb == out_params.max_nb, "Expected %u got %u",
+ in_params.max_nb, out_params.max_nb);
+
+ /* Case 6: Set max_nb = 512 */
+ in_params.max_nb = 512;
+
+ err = rte_event_dma_adapter_runtime_params_set(TEST_ADAPTER_ID, &in_params);
+ TEST_ASSERT(err == 0, "Expected 0 got %d", err);
+
+ err = rte_event_dma_adapter_runtime_params_get(TEST_ADAPTER_ID, &out_params);
+ TEST_ASSERT(err == 0, "Expected 0 got %d", err);
+ TEST_ASSERT(in_params.max_nb == out_params.max_nb, "Expected %u got %u",
+ in_params.max_nb, out_params.max_nb);
+
+ rc = TEST_SUCCESS;
+queue_pair_del:
+ err = rte_event_dma_adapter_vchan_queue_del(TEST_ADAPTER_ID, TEST_DMA_DEV_ID,
+ TEST_DMA_VCHAN_ID);
+ TEST_ASSERT_SUCCESS(err, "Failed to delete add queue pair\n");
+
+ return rc;
+}
+
+static int
+test_op_forward_mode(void)
+{
+ struct rte_mbuf *src_mbuf[TEST_MAX_OP];
+ struct rte_mbuf *dst_mbuf[TEST_MAX_OP];
+ struct rte_event ev[TEST_MAX_OP] = {0};
+ union rte_event_dma_metadata m_data;
+ struct rte_event_dma_adapter_op *op;
+ int ret, i;
+
+ ret = rte_pktmbuf_alloc_bulk(params.src_mbuf_pool, src_mbuf, TEST_MAX_OP);
+ TEST_ASSERT_SUCCESS(ret, "alloc src mbufs failed.\n");
+
+ ret = rte_pktmbuf_alloc_bulk(params.dst_mbuf_pool, dst_mbuf, TEST_MAX_OP);
+ TEST_ASSERT_SUCCESS(ret, "alloc dst mbufs failed.\n");
+
+ for (i = 0; i < TEST_MAX_OP; i++) {
+ memset(rte_pktmbuf_mtod(src_mbuf[i], void *), rte_rand(), PACKET_LENGTH);
+ memset(rte_pktmbuf_mtod(dst_mbuf[i], void *), 0, PACKET_LENGTH);
+ }
+
+ for (i = 0; i < TEST_MAX_OP; i++) {
+ rte_mempool_get(params.op_mpool, (void **)&op);
+ TEST_ASSERT_NOT_NULL(op, "Failed to allocate dma operation struct\n");
+
+ op->src_seg = rte_malloc(NULL, sizeof(struct rte_dma_sge), 0);
+ op->dst_seg = rte_malloc(NULL, sizeof(struct rte_dma_sge), 0);
+
+ /* Update Op */
+ op->src_seg->addr = rte_pktmbuf_iova(src_mbuf[i]);
+ op->dst_seg->addr = rte_pktmbuf_iova(dst_mbuf[i]);
+ op->src_seg->length = PACKET_LENGTH;
+ op->dst_seg->length = PACKET_LENGTH;
+ op->nb_src = 1;
+ op->nb_dst = 1;
+ op->flags = RTE_DMA_OP_FLAG_SUBMIT;
+ op->op_mp = params.op_mpool;
+
+ memset(&m_data, 0, sizeof(m_data));
+ m_data.request_info.dma_dev_id = dma_request_info.dma_dev_id;
+ m_data.request_info.vchan = dma_request_info.vchan;
+ m_data.response_info.event = dma_response_info.event;
+ rte_memcpy((uint8_t *)op + sizeof(struct rte_event_dma_adapter_op), &m_data,
+ sizeof(union rte_event_dma_metadata));
+
+ /* Fill in event info and update event_ptr with rte_event_dma_adapter_op */
+ memset(&ev[i], 0, sizeof(struct rte_event));
+ ev[i].event = 0;
+ ev[i].event_type = RTE_EVENT_TYPE_DMADEV;
+ ev[i].queue_id = TEST_DMA_EV_QUEUE_ID;
+ ev[i].sched_type = RTE_SCHED_TYPE_ATOMIC;
+ ev[i].flow_id = 0xAABB;
+ ev[i].event_ptr = op;
+ }
+
+ ret = send_recv_ev(ev);
+ TEST_ASSERT_SUCCESS(ret, "Failed to send/receive event to dma adapter\n");
+
+ test_dma_adapter_stats();
+
+ for (i = 0; i < TEST_MAX_OP; i++) {
+ op = ev[i].event_ptr;
+ ret = memcmp((uint8_t *)op->src_seg->addr, (uint8_t *)op->dst_seg->addr,
+ op->src_seg->length);
+ TEST_ASSERT_EQUAL(ret, 0, "Data mismatch for dma adapter\n");
+
+ rte_free(op->src_seg);
+ rte_free(op->dst_seg);
+ rte_mempool_put(op->op_mp, op);
+ }
+
+ rte_pktmbuf_free_bulk(src_mbuf, TEST_MAX_OP);
+ rte_pktmbuf_free_bulk(dst_mbuf, TEST_MAX_OP);
+
+ return TEST_SUCCESS;
+}
+
+static int
+map_adapter_service_core(void)
+{
+ uint32_t adapter_service_id;
+ int ret;
+
+ if (rte_event_dma_adapter_service_id_get(TEST_ADAPTER_ID, &adapter_service_id) == 0) {
+ uint32_t core_list[NUM_CORES];
+
+ ret = rte_service_lcore_list(core_list, NUM_CORES);
+ TEST_ASSERT(ret >= 0, "Failed to get service core list!");
+
+ if (core_list[0] != slcore_id) {
+ TEST_ASSERT_SUCCESS(rte_service_lcore_add(slcore_id),
+ "Failed to add service core");
+ TEST_ASSERT_SUCCESS(rte_service_lcore_start(slcore_id),
+ "Failed to start service core");
+ }
+
+ TEST_ASSERT_SUCCESS(rte_service_map_lcore_set(
+ adapter_service_id, slcore_id, 1),
+ "Failed to map adapter service");
+ }
+
+ return TEST_SUCCESS;
+}
+
+static int
+test_with_op_forward_mode(void)
+{
+ uint32_t cap;
+ int ret;
+
+ ret = rte_event_dma_adapter_caps_get(evdev, TEST_DMA_DEV_ID, &cap);
+ TEST_ASSERT_SUCCESS(ret, "Failed to get adapter capabilities\n");
+
+ if (!(cap & RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP_FWD) &&
+ !(cap & RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP_NEW))
+ map_adapter_service_core();
+ else {
+ if (!(cap & RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP_FWD))
+ return TEST_SKIPPED;
+ }
+
+ TEST_ASSERT_SUCCESS(rte_event_dma_adapter_start(TEST_ADAPTER_ID),
+ "Failed to start event dma adapter");
+
+ ret = test_op_forward_mode();
+ TEST_ASSERT_SUCCESS(ret, "DMA - FORWARD mode test failed\n");
+ return TEST_SUCCESS;
+}
+
+static int
+configure_dmadev(void)
+{
+ const struct rte_dma_conf conf = { .nb_vchans = 1};
+ const struct rte_dma_vchan_conf qconf = {
+ .direction = RTE_DMA_DIR_MEM_TO_MEM,
+ .nb_desc = TEST_RINGSIZE,
+ };
+ struct rte_dma_info info;
+ unsigned int elt_size;
+ int ret;
+
+ ret = rte_dma_count_avail();
+ RTE_TEST_ASSERT_FAIL(ret, "No dma devices found!\n");
+
+ ret = rte_dma_info_get(TEST_DMA_DEV_ID, &info);
+ TEST_ASSERT_SUCCESS(ret, "Error with rte_dma_info_get()\n");
+
+ if (info.max_vchans < 1)
+ RTE_LOG(ERR, USER1, "Error, no channels available on device id %u\n",
+ TEST_DMA_DEV_ID);
+
+ if (rte_dma_configure(TEST_DMA_DEV_ID, &conf) != 0)
+ RTE_LOG(ERR, USER1, "Error with rte_dma_configure()\n");
+
+ if (rte_dma_vchan_setup(TEST_DMA_DEV_ID, TEST_DMA_VCHAN_ID, &qconf) < 0)
+ RTE_LOG(ERR, USER1, "Error with queue configuration\n");
+
+ ret = rte_dma_info_get(TEST_DMA_DEV_ID, &info);
+ if (ret != 0 || info.nb_vchans != 1)
+ RTE_LOG(ERR, USER1, "Error, no configured queues reported on device id %u\n",
+ TEST_DMA_DEV_ID);
+
+ params.src_mbuf_pool = rte_pktmbuf_pool_create("DMA_ADAPTER_SRC_MBUFPOOL", NUM_MBUFS,
+ MBUF_CACHE_SIZE, 0, MBUF_SIZE,
+ rte_socket_id());
+ RTE_TEST_ASSERT_NOT_NULL(params.src_mbuf_pool, "Can't create DMA_SRC_MBUFPOOL\n");
+
+ params.dst_mbuf_pool = rte_pktmbuf_pool_create("DMA_ADAPTER_DST_MBUFPOOL", NUM_MBUFS,
+ MBUF_CACHE_SIZE, 0, MBUF_SIZE,
+ rte_socket_id());
+ RTE_TEST_ASSERT_NOT_NULL(params.dst_mbuf_pool, "Can't create DMA_DST_MBUFPOOL\n");
+
+ elt_size = sizeof(struct rte_event_dma_adapter_op) + sizeof(union rte_event_dma_metadata);
+ params.op_mpool = rte_mempool_create("EVENT_DMA_OP_POOL", DMA_OP_POOL_SIZE, elt_size, 0,
+ 0, NULL, NULL, NULL, NULL, rte_socket_id(), 0);
+ RTE_TEST_ASSERT_NOT_NULL(params.op_mpool, "Can't create DMA_OP_POOL\n");
+
+ return TEST_SUCCESS;
+}
+
+static inline void
+evdev_set_conf_values(struct rte_event_dev_config *dev_conf, struct rte_event_dev_info *info)
+{
+ memset(dev_conf, 0, sizeof(struct rte_event_dev_config));
+ dev_conf->dequeue_timeout_ns = info->min_dequeue_timeout_ns;
+ dev_conf->nb_event_ports = NB_TEST_PORTS;
+ dev_conf->nb_event_queues = NB_TEST_QUEUES;
+ dev_conf->nb_event_queue_flows = info->max_event_queue_flows;
+ dev_conf->nb_event_port_dequeue_depth =
+ info->max_event_port_dequeue_depth;
+ dev_conf->nb_event_port_enqueue_depth =
+ info->max_event_port_enqueue_depth;
+ dev_conf->nb_event_port_enqueue_depth =
+ info->max_event_port_enqueue_depth;
+ dev_conf->nb_events_limit =
+ info->max_num_events;
+}
+
+static int
+configure_eventdev(void)
+{
+ struct rte_event_queue_conf queue_conf;
+ struct rte_event_dev_config devconf;
+ struct rte_event_dev_info info;
+ uint32_t queue_count;
+ uint32_t port_count;
+ uint8_t qid;
+ int ret;
+
+ if (!rte_event_dev_count()) {
+ /* If there is no hardware eventdev, or no software vdev was
+ * specified on the command line, create an instance of
+ * event_sw.
+ */
+ LOG_DBG("Failed to find a valid event device... "
+ "testing with event_sw device\n");
+ TEST_ASSERT_SUCCESS(rte_vdev_init("event_sw0", NULL),
+ "Error creating eventdev");
+ evdev = rte_event_dev_get_dev_id("event_sw0");
+ }
+
+ ret = rte_event_dev_info_get(evdev, &info);
+ TEST_ASSERT_SUCCESS(ret, "Failed to get event dev info\n");
+
+ evdev_set_conf_values(&devconf, &info);
+
+ ret = rte_event_dev_configure(evdev, &devconf);
+ TEST_ASSERT_SUCCESS(ret, "Failed to configure eventdev\n");
+
+ /* Set up event queue */
+ ret = rte_event_dev_attr_get(evdev, RTE_EVENT_DEV_ATTR_QUEUE_COUNT, &queue_count);
+ TEST_ASSERT_SUCCESS(ret, "Queue count get failed\n");
+ TEST_ASSERT_EQUAL(queue_count, 2, "Unexpected queue count\n");
+
+ qid = TEST_APP_EV_QUEUE_ID;
+ ret = rte_event_queue_setup(evdev, qid, NULL);
+ TEST_ASSERT_SUCCESS(ret, "Failed to setup queue=%d\n", qid);
+
+ queue_conf.nb_atomic_flows = info.max_event_queue_flows;
+ queue_conf.nb_atomic_order_sequences = 32;
+ queue_conf.schedule_type = RTE_SCHED_TYPE_ATOMIC;
+ queue_conf.priority = RTE_EVENT_DEV_PRIORITY_HIGHEST;
+ queue_conf.event_queue_cfg = RTE_EVENT_QUEUE_CFG_SINGLE_LINK;
+
+ qid = TEST_DMA_EV_QUEUE_ID;
+ ret = rte_event_queue_setup(evdev, qid, &queue_conf);
+ TEST_ASSERT_SUCCESS(ret, "Failed to setup queue=%u\n", qid);
+
+ /* Set up event port */
+ ret = rte_event_dev_attr_get(evdev, RTE_EVENT_DEV_ATTR_PORT_COUNT,
+ &port_count);
+ TEST_ASSERT_SUCCESS(ret, "Port count get failed\n");
+ TEST_ASSERT_EQUAL(port_count, 1, "Unexpected port count\n");
+
+ ret = rte_event_port_setup(evdev, TEST_APP_PORT_ID, NULL);
+ TEST_ASSERT_SUCCESS(ret, "Failed to setup port=%d\n",
+ TEST_APP_PORT_ID);
+
+ qid = TEST_APP_EV_QUEUE_ID;
+ ret = rte_event_port_link(evdev, TEST_APP_PORT_ID, &qid, NULL, 1);
+ TEST_ASSERT(ret >= 0, "Failed to link queue port=%d\n",
+ TEST_APP_PORT_ID);
+
+ return TEST_SUCCESS;
+}
+
+static void
+test_dma_adapter_free(void)
+{
+ rte_event_dma_adapter_free(TEST_ADAPTER_ID);
+}
+
+static int
+test_dma_adapter_create(void)
+{
+ struct rte_event_dev_info evdev_info = {0};
+ struct rte_event_port_conf conf = {0};
+ int ret;
+
+ ret = rte_event_dev_info_get(evdev, &evdev_info);
+ TEST_ASSERT_SUCCESS(ret, "Failed to create event dma adapter\n");
+
+ conf.new_event_threshold = evdev_info.max_num_events;
+ conf.dequeue_depth = evdev_info.max_event_port_dequeue_depth;
+ conf.enqueue_depth = evdev_info.max_event_port_enqueue_depth;
+
+ /* Create adapter with default port creation callback */
+ ret = rte_event_dma_adapter_create(TEST_ADAPTER_ID, evdev, &conf, 0);
+ TEST_ASSERT_SUCCESS(ret, "Failed to create event dma adapter\n");
+
+ return TEST_SUCCESS;
+}
+
+static int
+test_dma_adapter_qp_add_del(void)
+{
+ struct rte_event event;
+ uint32_t cap;
+ int ret;
+
+ ret = rte_event_dma_adapter_caps_get(evdev, TEST_DMA_DEV_ID, &cap);
+ TEST_ASSERT_SUCCESS(ret, "Failed to get adapter capabilities\n");
+
+ if (cap & RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_QP_EV_BIND) {
+ ret = rte_event_dma_adapter_vchan_queue_add(TEST_ADAPTER_ID, TEST_DMA_DEV_ID,
+ TEST_DMA_VCHAN_ID, &event);
+ } else
+ ret = rte_event_dma_adapter_vchan_queue_add(TEST_ADAPTER_ID, TEST_DMA_DEV_ID,
+ TEST_DMA_VCHAN_ID, NULL);
+
+ TEST_ASSERT_SUCCESS(ret, "Failed to create add queue pair\n");
+
+ ret = rte_event_dma_adapter_vchan_queue_del(TEST_ADAPTER_ID, TEST_DMA_DEV_ID,
+ TEST_DMA_VCHAN_ID);
+ TEST_ASSERT_SUCCESS(ret, "Failed to delete add queue pair\n");
+
+ return TEST_SUCCESS;
+}
+
+static int
+configure_event_dma_adapter(enum rte_event_dma_adapter_mode mode)
+{
+ struct rte_event_dev_info evdev_info = {0};
+ struct rte_event_port_conf conf = {0};
+ struct rte_event event;
+ uint32_t cap;
+ int ret;
+
+ ret = rte_event_dma_adapter_caps_get(evdev, TEST_DMA_DEV_ID, &cap);
+ TEST_ASSERT_SUCCESS(ret, "Failed to get adapter capabilities\n");
+
+ /* Skip mode and capability mismatch check for SW eventdev */
+ if (!(cap & RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP_NEW) &&
+ !(cap & RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP_FWD) &&
+ !(cap & RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_QP_EV_BIND))
+ goto adapter_create;
+
+ if (mode == RTE_EVENT_DMA_ADAPTER_OP_FORWARD) {
+ if (cap & RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP_FWD)
+ params.internal_port_op_fwd = 1;
+ else
+ return -ENOTSUP;
+ }
+
+adapter_create:
+ ret = rte_event_dev_info_get(evdev, &evdev_info);
+ TEST_ASSERT_SUCCESS(ret, "Failed to create event dma adapter\n");
+
+ conf.new_event_threshold = evdev_info.max_num_events;
+ conf.dequeue_depth = evdev_info.max_event_port_dequeue_depth;
+ conf.enqueue_depth = evdev_info.max_event_port_enqueue_depth;
+
+ /* Create adapter with default port creation callback */
+ ret = rte_event_dma_adapter_create(TEST_ADAPTER_ID, evdev, &conf, mode);
+ TEST_ASSERT_SUCCESS(ret, "Failed to create event dma adapter\n");
+
+ if (cap & RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_QP_EV_BIND) {
+ ret = rte_event_dma_adapter_vchan_queue_add(TEST_ADAPTER_ID, TEST_DMA_DEV_ID,
+ TEST_DMA_VCHAN_ID, &event);
+ } else
+ ret = rte_event_dma_adapter_vchan_queue_add(TEST_ADAPTER_ID, TEST_DMA_DEV_ID,
+ TEST_DMA_VCHAN_ID, NULL);
+
+ TEST_ASSERT_SUCCESS(ret, "Failed to add queue pair\n");
+
+ if (!params.internal_port_op_fwd) {
+ ret = rte_event_dma_adapter_event_port_get(TEST_ADAPTER_ID,
+ ¶ms.dma_event_port_id);
+ TEST_ASSERT_SUCCESS(ret, "Failed to get event port\n");
+ }
+
+ return TEST_SUCCESS;
+}
+
+static void
+test_dma_adapter_stop(void)
+{
+ uint32_t evdev_service_id, adapter_service_id;
+
+ /* retrieve service ids & stop services */
+ if (rte_event_dma_adapter_service_id_get(TEST_ADAPTER_ID,
+ &adapter_service_id) == 0) {
+ rte_service_runstate_set(adapter_service_id, 0);
+ rte_service_lcore_stop(slcore_id);
+ rte_service_lcore_del(slcore_id);
+ rte_event_dma_adapter_stop(TEST_ADAPTER_ID);
+ }
+
+ if (rte_event_dev_service_id_get(evdev, &evdev_service_id) == 0) {
+ rte_service_runstate_set(evdev_service_id, 0);
+ rte_service_lcore_stop(slcore_id);
+ rte_service_lcore_del(slcore_id);
+ rte_dma_stop(TEST_DMA_DEV_ID);
+ rte_event_dev_stop(evdev);
+ } else {
+ rte_dma_stop(TEST_DMA_DEV_ID);
+ rte_event_dev_stop(evdev);
+ }
+}
+
+static int
+test_dma_adapter_conf(enum rte_event_dma_adapter_mode mode)
+{
+ uint32_t evdev_service_id;
+ uint8_t qid;
+ int ret;
+
+ if (!dma_adapter_setup_done) {
+ ret = configure_event_dma_adapter(mode);
+ if (ret)
+ return ret;
+ if (!params.internal_port_op_fwd) {
+ qid = TEST_DMA_EV_QUEUE_ID;
+ ret = rte_event_port_link(evdev,
+ params.dma_event_port_id, &qid, NULL, 1);
+ TEST_ASSERT(ret >= 0, "Failed to link queue %d "
+ "port=%u\n", qid,
+ params.dma_event_port_id);
+ }
+ dma_adapter_setup_done = 1;
+ }
+
+ /* retrieve service ids */
+ if (rte_event_dev_service_id_get(evdev, &evdev_service_id) == 0) {
+ /* add a service core and start it */
+ TEST_ASSERT_SUCCESS(rte_service_lcore_add(slcore_id),
+ "Failed to add service core");
+ TEST_ASSERT_SUCCESS(rte_service_lcore_start(slcore_id),
+ "Failed to start service core");
+
+ /* map services to it */
+ TEST_ASSERT_SUCCESS(rte_service_map_lcore_set(evdev_service_id,
+ slcore_id, 1), "Failed to map evdev service");
+
+ /* set services to running */
+ TEST_ASSERT_SUCCESS(rte_service_runstate_set(evdev_service_id,
+ 1), "Failed to start evdev service");
+ }
+
+ /* start the eventdev */
+ TEST_ASSERT_SUCCESS(rte_event_dev_start(evdev),
+ "Failed to start event device");
+
+ /* start the dma dev */
+ TEST_ASSERT_SUCCESS(rte_dma_start(TEST_DMA_DEV_ID),
+ "Failed to start dma device");
+
+ return TEST_SUCCESS;
+}
+
+static int
+test_dma_adapter_conf_op_forward_mode(void)
+{
+ enum rte_event_dma_adapter_mode mode;
+
+ mode = RTE_EVENT_DMA_ADAPTER_OP_FORWARD;
+
+ return test_dma_adapter_conf(mode);
+}
+
+static int
+testsuite_setup(void)
+{
+ int ret;
+
+ slcore_id = rte_get_next_lcore(-1, 1, 0);
+ TEST_ASSERT_NOT_EQUAL(slcore_id, RTE_MAX_LCORE, "At least 2 lcores "
+ "are required to run this autotest\n");
+
+ /* Setup and start event device. */
+ ret = configure_eventdev();
+ TEST_ASSERT_SUCCESS(ret, "Failed to setup eventdev\n");
+
+ /* Setup and start dma device. */
+ ret = configure_dmadev();
+ TEST_ASSERT_SUCCESS(ret, "dmadev initialization failed\n");
+
+ return TEST_SUCCESS;
+}
+
+static void
+dma_adapter_teardown(void)
+{
+ int ret;
+
+ ret = rte_event_dma_adapter_stop(TEST_ADAPTER_ID);
+ if (ret < 0)
+ RTE_LOG(ERR, USER1, "Failed to stop adapter!");
+
+ ret = rte_event_dma_adapter_vchan_queue_del(TEST_ADAPTER_ID, TEST_DMA_DEV_ID,
+ TEST_DMA_VCHAN_ID);
+ if (ret < 0)
+ RTE_LOG(ERR, USER1, "Failed to delete queue pair!");
+
+ ret = rte_event_dma_adapter_free(TEST_ADAPTER_ID);
+ if (ret < 0)
+ RTE_LOG(ERR, USER1, "Failed to free adapter!");
+
+ dma_adapter_setup_done = 0;
+}
+
+static void
+dma_teardown(void)
+{
+ /* Free mbuf mempool */
+ if (params.src_mbuf_pool != NULL) {
+ RTE_LOG(DEBUG, USER1, "DMA_ADAPTER_SRC_MBUFPOOL count %u\n",
+ rte_mempool_avail_count(params.src_mbuf_pool));
+ rte_mempool_free(params.src_mbuf_pool);
+ params.src_mbuf_pool = NULL;
+ }
+
+ if (params.dst_mbuf_pool != NULL) {
+ RTE_LOG(DEBUG, USER1, "DMA_ADAPTER_DST_MBUFPOOL count %u\n",
+ rte_mempool_avail_count(params.dst_mbuf_pool));
+ rte_mempool_free(params.dst_mbuf_pool);
+ params.dst_mbuf_pool = NULL;
+ }
+
+ /* Free ops mempool */
+ if (params.op_mpool != NULL) {
+ RTE_LOG(DEBUG, USER1, "EVENT_DMA_OP_POOL count %u\n",
+ rte_mempool_avail_count(params.op_mpool));
+ rte_mempool_free(params.op_mpool);
+ params.op_mpool = NULL;
+ }
+}
+
+static void
+eventdev_teardown(void)
+{
+ rte_event_dev_stop(evdev);
+}
+
+static void
+testsuite_teardown(void)
+{
+ dma_adapter_teardown();
+ dma_teardown();
+ eventdev_teardown();
+}
+
+static struct unit_test_suite functional_testsuite = {
+ .suite_name = "Event dma adapter test suite",
+ .setup = testsuite_setup,
+ .teardown = testsuite_teardown,
+ .unit_test_cases = {
+
+ TEST_CASE_ST(NULL, test_dma_adapter_free, test_dma_adapter_create),
+
+ TEST_CASE_ST(test_dma_adapter_create, test_dma_adapter_free,
+ test_dma_adapter_qp_add_del),
+
+ TEST_CASE_ST(test_dma_adapter_create, test_dma_adapter_free,
+ test_dma_adapter_stats),
+
+ TEST_CASE_ST(test_dma_adapter_create, test_dma_adapter_free,
+ test_dma_adapter_params),
+
+ TEST_CASE_ST(test_dma_adapter_conf_op_forward_mode, test_dma_adapter_stop,
+ test_with_op_forward_mode),
+
+ TEST_CASES_END() /**< NULL terminate unit test array */
+ }
+};
+
+static int
+test_event_dma_adapter(void)
+{
+ return unit_test_suite_runner(&functional_testsuite);
+}
+
+#endif /* !RTE_EXEC_ENV_WINDOWS */
+
+REGISTER_TEST_COMMAND(event_dma_adapter_autotest, test_event_dma_adapter);
--
2.25.1
^ permalink raw reply [flat|nested] 110+ messages in thread
* [PATCH v1 5/7] common/cnxk: dma result to an offset of the event
2023-09-19 13:42 [PATCH v1 1/7] eventdev: introduce DMA event adapter library Amit Prakash Shukla
` (2 preceding siblings ...)
2023-09-19 13:42 ` [PATCH v1 4/7] app/test: add event DMA adapter auto-test Amit Prakash Shukla
@ 2023-09-19 13:42 ` Amit Prakash Shukla
2023-09-19 13:42 ` [PATCH v1 6/7] dma/cnxk: support for DMA event enqueue dequeue Amit Prakash Shukla
` (3 subsequent siblings)
7 siblings, 0 replies; 110+ messages in thread
From: Amit Prakash Shukla @ 2023-09-19 13:42 UTC (permalink / raw)
To: Nithin Dabilpuram, Kiran Kumar K, Sunil Kumar Kori, Satha Rao,
Vamsi Attunuru
Cc: dev, jerinj, fengchengwen, kevin.laatz, bruce.richardson,
conor.walsh, g.singh, sachin.saxena, hemant.agrawal,
cheng1.jiang, anoobj, mb, Amit Prakash Shukla
Adds support to configure writing result to offset of the DMA
response event.
Signed-off-by: Amit Prakash Shukla <amitprakashs@marvell.com>
---
drivers/common/cnxk/roc_dpi.c | 5 ++++-
drivers/common/cnxk/roc_dpi.h | 2 +-
drivers/common/cnxk/roc_dpi_priv.h | 4 ++++
drivers/common/cnxk/roc_idev.c | 20 ++++++++++++++++++++
drivers/common/cnxk/roc_idev_priv.h | 3 +++
drivers/dma/cnxk/cnxk_dmadev.c | 3 ++-
drivers/dma/cnxk/cnxk_dmadev.h | 1 +
7 files changed, 35 insertions(+), 3 deletions(-)
diff --git a/drivers/common/cnxk/roc_dpi.c b/drivers/common/cnxk/roc_dpi.c
index 2e086b3698..7bf6ac2aaf 100644
--- a/drivers/common/cnxk/roc_dpi.c
+++ b/drivers/common/cnxk/roc_dpi.c
@@ -84,6 +84,8 @@ roc_dpi_configure(struct roc_dpi *roc_dpi, uint32_t chunk_sz, uint64_t aura, uin
mbox_msg.s.aura = aura;
mbox_msg.s.sso_pf_func = idev_sso_pffunc_get();
mbox_msg.s.npa_pf_func = idev_npa_pffunc_get();
+ mbox_msg.s.wqecs = 1;
+ mbox_msg.s.wqecsoff = idev_dma_cs_offset_get();
rc = send_msg_to_pf(&pci_dev->addr, (const char *)&mbox_msg,
sizeof(dpi_mbox_msg_t));
@@ -95,7 +97,7 @@ roc_dpi_configure(struct roc_dpi *roc_dpi, uint32_t chunk_sz, uint64_t aura, uin
}
int
-roc_dpi_dev_init(struct roc_dpi *roc_dpi)
+roc_dpi_dev_init(struct roc_dpi *roc_dpi, uint8_t offset)
{
struct plt_pci_device *pci_dev = roc_dpi->pci_dev;
uint16_t vfid;
@@ -104,6 +106,7 @@ roc_dpi_dev_init(struct roc_dpi *roc_dpi)
vfid = ((pci_dev->addr.devid & 0x1F) << 3) | (pci_dev->addr.function & 0x7);
vfid -= 1;
roc_dpi->vfid = vfid;
+ idev_dma_cs_offset_set(offset);
return 0;
}
diff --git a/drivers/common/cnxk/roc_dpi.h b/drivers/common/cnxk/roc_dpi.h
index 4ebde5b8a6..978e2badb2 100644
--- a/drivers/common/cnxk/roc_dpi.h
+++ b/drivers/common/cnxk/roc_dpi.h
@@ -11,7 +11,7 @@ struct roc_dpi {
uint16_t vfid;
} __plt_cache_aligned;
-int __roc_api roc_dpi_dev_init(struct roc_dpi *roc_dpi);
+int __roc_api roc_dpi_dev_init(struct roc_dpi *roc_dpi, uint8_t offset);
int __roc_api roc_dpi_dev_fini(struct roc_dpi *roc_dpi);
int __roc_api roc_dpi_configure(struct roc_dpi *dpi, uint32_t chunk_sz, uint64_t aura,
diff --git a/drivers/common/cnxk/roc_dpi_priv.h b/drivers/common/cnxk/roc_dpi_priv.h
index 518a3e7351..52962c8bc0 100644
--- a/drivers/common/cnxk/roc_dpi_priv.h
+++ b/drivers/common/cnxk/roc_dpi_priv.h
@@ -31,6 +31,10 @@ typedef union dpi_mbox_msg_t {
uint64_t sso_pf_func : 16;
/* NPA PF function */
uint64_t npa_pf_func : 16;
+ /* WQE queue DMA completion status enable */
+ uint64_t wqecs : 1;
+ /* WQE queue DMA completion status offset */
+ uint64_t wqecsoff : 8;
} s;
} dpi_mbox_msg_t;
diff --git a/drivers/common/cnxk/roc_idev.c b/drivers/common/cnxk/roc_idev.c
index e6c6b34d78..7b922c8bae 100644
--- a/drivers/common/cnxk/roc_idev.c
+++ b/drivers/common/cnxk/roc_idev.c
@@ -301,6 +301,26 @@ idev_sso_set(struct roc_sso *sso)
__atomic_store_n(&idev->sso, sso, __ATOMIC_RELEASE);
}
+void
+idev_dma_cs_offset_set(uint8_t offset)
+{
+ struct idev_cfg *idev = idev_get_cfg();
+
+ if (idev != NULL)
+ idev->dma_cs_offset = offset;
+}
+
+uint8_t
+idev_dma_cs_offset_get(void)
+{
+ struct idev_cfg *idev = idev_get_cfg();
+
+ if (idev != NULL)
+ return idev->dma_cs_offset;
+
+ return 0;
+}
+
uint64_t
roc_idev_nix_inl_meta_aura_get(void)
{
diff --git a/drivers/common/cnxk/roc_idev_priv.h b/drivers/common/cnxk/roc_idev_priv.h
index 80f8465e1c..cf63c58d92 100644
--- a/drivers/common/cnxk/roc_idev_priv.h
+++ b/drivers/common/cnxk/roc_idev_priv.h
@@ -37,6 +37,7 @@ struct idev_cfg {
struct roc_nix_list roc_nix_list;
plt_spinlock_t nix_inl_dev_lock;
plt_spinlock_t npa_dev_lock;
+ uint8_t dma_cs_offset;
};
/* Generic */
@@ -55,6 +56,8 @@ void idev_sso_pffunc_set(uint16_t sso_pf_func);
uint16_t idev_sso_pffunc_get(void);
struct roc_sso *idev_sso_get(void);
void idev_sso_set(struct roc_sso *sso);
+void idev_dma_cs_offset_set(uint8_t offset);
+uint8_t idev_dma_cs_offset_get(void);
/* idev lmt */
uint16_t idev_lmt_pffunc_get(void);
diff --git a/drivers/dma/cnxk/cnxk_dmadev.c b/drivers/dma/cnxk/cnxk_dmadev.c
index 26680edfde..db127e056f 100644
--- a/drivers/dma/cnxk/cnxk_dmadev.c
+++ b/drivers/dma/cnxk/cnxk_dmadev.c
@@ -511,6 +511,7 @@ static const struct rte_dma_dev_ops cnxk_dmadev_ops = {
static int
cnxk_dmadev_probe(struct rte_pci_driver *pci_drv __rte_unused, struct rte_pci_device *pci_dev)
{
+ struct cnxk_dpi_compl_s *compl = NULL;
struct cnxk_dpi_vf_s *dpivf = NULL;
char name[RTE_DEV_NAME_MAX_LEN];
struct rte_dma_dev *dmadev;
@@ -556,7 +557,7 @@ cnxk_dmadev_probe(struct rte_pci_driver *pci_drv __rte_unused, struct rte_pci_de
rdpi = &dpivf->rdpi;
rdpi->pci_dev = pci_dev;
- rc = roc_dpi_dev_init(rdpi);
+ rc = roc_dpi_dev_init(rdpi, (uint64_t)&compl->wqecs);
if (rc < 0)
goto err_out_free;
diff --git a/drivers/dma/cnxk/cnxk_dmadev.h b/drivers/dma/cnxk/cnxk_dmadev.h
index 350ae73b5c..75059b8843 100644
--- a/drivers/dma/cnxk/cnxk_dmadev.h
+++ b/drivers/dma/cnxk/cnxk_dmadev.h
@@ -86,6 +86,7 @@ union cnxk_dpi_instr_cmd {
struct cnxk_dpi_compl_s {
uint64_t cdata;
void *cb_data;
+ uint32_t wqecs;
};
struct cnxk_dpi_cdesc_data_s {
--
2.25.1
^ permalink raw reply [flat|nested] 110+ messages in thread
* [PATCH v1 6/7] dma/cnxk: support for DMA event enqueue dequeue
2023-09-19 13:42 [PATCH v1 1/7] eventdev: introduce DMA event adapter library Amit Prakash Shukla
` (3 preceding siblings ...)
2023-09-19 13:42 ` [PATCH v1 5/7] common/cnxk: dma result to an offset of the event Amit Prakash Shukla
@ 2023-09-19 13:42 ` Amit Prakash Shukla
2023-09-19 13:42 ` [PATCH v1 7/7] event/cnxk: support DMA event functions Amit Prakash Shukla
` (2 subsequent siblings)
7 siblings, 0 replies; 110+ messages in thread
From: Amit Prakash Shukla @ 2023-09-19 13:42 UTC (permalink / raw)
To: Vamsi Attunuru
Cc: dev, jerinj, fengchengwen, kevin.laatz, bruce.richardson,
conor.walsh, g.singh, sachin.saxena, hemant.agrawal,
cheng1.jiang, ndabilpuram, anoobj, mb, Amit Prakash Shukla
Added cnxk driver support for dma event enqueue and dequeue.
Signed-off-by: Amit Prakash Shukla <amitprakashs@marvell.com>
---
drivers/dma/cnxk/cnxk_dma_event_dp.h | 22 +++
drivers/dma/cnxk/cnxk_dmadev.h | 9 +-
drivers/dma/cnxk/cnxk_dmadev_fp.c | 209 +++++++++++++++++++++++++++
drivers/dma/cnxk/meson.build | 6 +-
drivers/dma/cnxk/version.map | 9 ++
5 files changed, 253 insertions(+), 2 deletions(-)
create mode 100644 drivers/dma/cnxk/cnxk_dma_event_dp.h
create mode 100644 drivers/dma/cnxk/version.map
diff --git a/drivers/dma/cnxk/cnxk_dma_event_dp.h b/drivers/dma/cnxk/cnxk_dma_event_dp.h
new file mode 100644
index 0000000000..bf9b01f8f1
--- /dev/null
+++ b/drivers/dma/cnxk/cnxk_dma_event_dp.h
@@ -0,0 +1,22 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(C) 2023 Marvell.
+ */
+
+#ifndef _CNXK_DMA_EVENT_DP_H_
+#define _CNXK_DMA_EVENT_DP_H_
+
+#include <stdint.h>
+
+#include <rte_common.h>
+#include <rte_eventdev.h>
+
+__rte_internal
+uint16_t cn10k_dma_adapter_enqueue(void *ws, struct rte_event ev[], uint16_t nb_events);
+
+__rte_internal
+uint16_t cn9k_dma_adapter_enqueue(void *ws, struct rte_event ev[], uint16_t nb_events);
+
+__rte_internal
+uintptr_t cnxk_dma_adapter_dequeue(uintptr_t get_work1);
+
+#endif /* _CNXK_DMA_EVENT_DP_H_ */
diff --git a/drivers/dma/cnxk/cnxk_dmadev.h b/drivers/dma/cnxk/cnxk_dmadev.h
index 75059b8843..9cba388d02 100644
--- a/drivers/dma/cnxk/cnxk_dmadev.h
+++ b/drivers/dma/cnxk/cnxk_dmadev.h
@@ -40,6 +40,11 @@
*/
#define CNXK_DPI_REQ_CDATA 0xFF
+/* Set Completion data to 0xDEADBEEF when request submitted for SSO.
+ * This helps differentiate if the dequeue is called after cnxk enueue.
+ */
+#define CNXK_DPI_REQ_SSO_CDATA 0xDEADBEEF
+
union cnxk_dpi_instr_cmd {
uint64_t u;
struct cn9k_dpi_instr_cmd {
@@ -85,7 +90,9 @@ union cnxk_dpi_instr_cmd {
struct cnxk_dpi_compl_s {
uint64_t cdata;
- void *cb_data;
+ void *op;
+ uint16_t dev_id;
+ uint16_t vchan;
uint32_t wqecs;
};
diff --git a/drivers/dma/cnxk/cnxk_dmadev_fp.c b/drivers/dma/cnxk/cnxk_dmadev_fp.c
index 16d7b5426b..c7cd036a5b 100644
--- a/drivers/dma/cnxk/cnxk_dmadev_fp.c
+++ b/drivers/dma/cnxk/cnxk_dmadev_fp.c
@@ -5,6 +5,8 @@
#include <rte_vect.h>
#include "cnxk_dmadev.h"
+#include "cnxk_dma_event_dp.h"
+#include <rte_event_dma_adapter.h>
static __plt_always_inline void
__dpi_cpy_scalar(uint64_t *src, uint64_t *dst, uint8_t n)
@@ -434,3 +436,210 @@ cn10k_dmadev_copy_sg(void *dev_private, uint16_t vchan, const struct rte_dma_sge
return dpi_conf->desc_idx++;
}
+
+uint16_t
+cn10k_dma_adapter_enqueue(void *ws, struct rte_event ev[], uint16_t nb_events)
+{
+ union rte_event_dma_metadata *dma_mdata;
+ struct rte_event_dma_request *req_info;
+ const struct rte_dma_sge *src, *dst;
+ struct rte_event_dma_adapter_op *op;
+ struct cnxk_dpi_compl_s *comp_ptr;
+ struct cnxk_dpi_conf *dpi_conf;
+ struct cnxk_dpi_vf_s *dpivf;
+ struct rte_event *rsp_info;
+ uint16_t nb_src, nb_dst;
+ struct rte_dma_dev *dev;
+ uint64_t hdr[4];
+ uint16_t count;
+ int rc;
+
+ PLT_SET_USED(ws);
+
+ for (count = 0; count < nb_events; count++) {
+ op = ev[count].event_ptr;
+ dma_mdata = (union rte_event_dma_metadata *)((uint8_t *)op +
+ sizeof(struct rte_event_dma_adapter_op));
+ rsp_info = &dma_mdata->response_info;
+ req_info = &dma_mdata->request_info;
+ dev = rte_dma_pmd_dev_get(req_info->dma_dev_id);
+ dpivf = dev->data->dev_private;
+ dpi_conf = &dpivf->conf[req_info->vchan];
+
+ if (unlikely(((dpi_conf->c_desc.tail + 1) & dpi_conf->c_desc.max_cnt) ==
+ dpi_conf->c_desc.head))
+ return count;
+
+ comp_ptr = dpi_conf->c_desc.compl_ptr[dpi_conf->c_desc.tail];
+ CNXK_DPI_STRM_INC(dpi_conf->c_desc, tail);
+ comp_ptr->op = op;
+ comp_ptr->dev_id = req_info->dma_dev_id;
+ comp_ptr->vchan = req_info->vchan;
+ comp_ptr->cdata = CNXK_DPI_REQ_SSO_CDATA;
+
+ nb_src = op->nb_src & CNXK_DPI_MAX_POINTER;
+ nb_dst = op->nb_dst & CNXK_DPI_MAX_POINTER;
+
+ hdr[0] = dpi_conf->cmd.u | ((uint64_t)DPI_HDR_PT_WQP << 54);
+ hdr[0] |= (nb_dst << 6) | nb_src;
+ hdr[1] = ((uint64_t)comp_ptr);
+ hdr[2] = (RTE_EVENT_TYPE_DMADEV << 28 | (rsp_info->sub_event_type << 20) |
+ rsp_info->flow_id);
+ hdr[2] |= ((uint64_t)(rsp_info->sched_type & DPI_HDR_TT_MASK)) << 32;
+ hdr[2] |= ((uint64_t)(rsp_info->queue_id & DPI_HDR_GRP_MASK)) << 34;
+
+ src = &op->src_seg[0];
+ dst = &op->dst_seg[0];
+
+ rc = __dpi_queue_write_sg(dpivf, hdr, src, dst, nb_src, nb_dst);
+ if (unlikely(rc)) {
+ CNXK_DPI_STRM_DEC(dpi_conf->c_desc, tail);
+ return rc;
+ }
+
+ if (op->flags & RTE_DMA_OP_FLAG_SUBMIT) {
+ rte_wmb();
+ plt_write64(dpi_conf->pnum_words + CNXK_DPI_CMD_LEN(nb_src, nb_dst),
+ dpivf->rdpi.rbase + DPI_VDMA_DBELL);
+ dpi_conf->stats.submitted += dpi_conf->pending + 1;
+ dpi_conf->pnum_words = 0;
+ dpi_conf->pending = 0;
+ } else {
+ dpi_conf->pnum_words += CNXK_DPI_CMD_LEN(nb_src, nb_dst);
+ dpi_conf->pending++;
+ }
+ }
+
+ return count;
+}
+
+uint16_t
+cn9k_dma_adapter_enqueue(void *ws, struct rte_event ev[], uint16_t nb_events)
+{
+ union rte_event_dma_metadata *dma_mdata;
+ struct rte_event_dma_request *req_info;
+ const struct rte_dma_sge *fptr, *lptr;
+ struct rte_event_dma_adapter_op *op;
+ struct cnxk_dpi_compl_s *comp_ptr;
+ struct cnxk_dpi_conf *dpi_conf;
+ struct cnxk_dpi_vf_s *dpivf;
+ struct rte_event *rsp_info;
+ uint16_t nb_src, nb_dst;
+ struct rte_dma_dev *dev;
+ uint64_t hdr[4];
+ uint16_t count;
+ int rc;
+
+ PLT_SET_USED(ws);
+
+ for (count = 0; count < nb_events; count++) {
+ op = ev[count].event_ptr;
+ dma_mdata = (union rte_event_dma_metadata *)((uint8_t *)op +
+ sizeof(struct rte_event_dma_adapter_op));
+ rsp_info = &dma_mdata->response_info;
+ req_info = &dma_mdata->request_info;
+ dev = rte_dma_pmd_dev_get(req_info->dma_dev_id);
+ dpivf = dev->data->dev_private;
+ dpi_conf = &dpivf->conf[req_info->vchan];
+
+ if (unlikely(((dpi_conf->c_desc.tail + 1) & dpi_conf->c_desc.max_cnt) ==
+ dpi_conf->c_desc.head))
+ return count;
+
+ comp_ptr = dpi_conf->c_desc.compl_ptr[dpi_conf->c_desc.tail];
+ CNXK_DPI_STRM_INC(dpi_conf->c_desc, tail);
+ comp_ptr->op = op;
+ comp_ptr->dev_id = req_info->dma_dev_id;
+ comp_ptr->vchan = req_info->vchan;
+ comp_ptr->cdata = CNXK_DPI_REQ_SSO_CDATA;
+
+ hdr[1] = dpi_conf->cmd.u | ((uint64_t)DPI_HDR_PT_WQP << 36);
+ hdr[2] = (uint64_t)comp_ptr;
+
+ nb_src = op->nb_src & CNXK_DPI_MAX_POINTER;
+ nb_dst = op->nb_dst & CNXK_DPI_MAX_POINTER;
+ /*
+ * For inbound case, src pointers are last pointers.
+ * For all other cases, src pointers are first pointers.
+ */
+ if (((dpi_conf->cmd.u >> 48) & DPI_HDR_XTYPE_MASK) == DPI_XTYPE_INBOUND) {
+ fptr = &op->dst_seg[0];
+ lptr = &op->src_seg[0];
+ RTE_SWAP(nb_src, nb_dst);
+ } else {
+ fptr = &op->src_seg[0];
+ lptr = &op->dst_seg[0];
+ }
+
+ hdr[0] = ((uint64_t)nb_dst << 54) | (uint64_t)nb_src << 48;
+ hdr[0] |= (RTE_EVENT_TYPE_DMADEV << 28 | (rsp_info->sub_event_type << 20) |
+ rsp_info->flow_id);
+ hdr[0] |= ((uint64_t)(rsp_info->sched_type & DPI_HDR_TT_MASK)) << 32;
+ hdr[0] |= ((uint64_t)(rsp_info->queue_id & DPI_HDR_GRP_MASK)) << 34;
+
+ rc = __dpi_queue_write_sg(dpivf, hdr, fptr, lptr, nb_src, nb_dst);
+ if (unlikely(rc)) {
+ CNXK_DPI_STRM_DEC(dpi_conf->c_desc, tail);
+ return rc;
+ }
+
+ if (op->flags & RTE_DMA_OP_FLAG_SUBMIT) {
+ rte_wmb();
+ plt_write64(dpi_conf->pnum_words + CNXK_DPI_CMD_LEN(nb_src, nb_dst),
+ dpivf->rdpi.rbase + DPI_VDMA_DBELL);
+ dpi_conf->stats.submitted += dpi_conf->pending + 1;
+ dpi_conf->pnum_words = 0;
+ dpi_conf->pending = 0;
+ } else {
+ dpi_conf->pnum_words += CNXK_DPI_CMD_LEN(nb_src, nb_dst);
+ dpi_conf->pending++;
+ }
+ }
+
+ return count;
+}
+
+uintptr_t
+cnxk_dma_adapter_dequeue(uintptr_t get_work1)
+{
+ struct rte_event_dma_adapter_op *op;
+ struct cnxk_dpi_compl_s *comp_ptr;
+ struct cnxk_dpi_conf *dpi_conf;
+ struct cnxk_dpi_vf_s *dpivf;
+ struct rte_dma_dev *dev;
+ uint8_t *wqecs;
+
+ comp_ptr = (struct cnxk_dpi_compl_s *)get_work1;
+
+ /* Dequeue can be called without calling cnx_enqueue in case of
+ * dma_adapter. When its called from adapter, dma op will not be
+ * embedded in completion pointer. In those cases return op.
+ */
+ if (comp_ptr->cdata != CNXK_DPI_REQ_SSO_CDATA)
+ return (uintptr_t)comp_ptr;
+
+ dev = rte_dma_pmd_dev_get(comp_ptr->dev_id);
+ dpivf = dev->data->dev_private;
+ dpi_conf = &dpivf->conf[comp_ptr->vchan];
+
+ wqecs = (uint8_t *)&comp_ptr->wqecs;
+ if (__atomic_load_n(wqecs, __ATOMIC_RELAXED) != 0)
+ dpi_conf->stats.errors++;
+
+ op = (struct rte_event_dma_adapter_op *)comp_ptr->op;
+
+ /* We are done here. Reset completion buffer.*/
+ comp_ptr->wqecs = ~0;
+ comp_ptr->op = NULL;
+ comp_ptr->dev_id = ~0;
+ comp_ptr->vchan = ~0;
+ comp_ptr->cdata = CNXK_DPI_REQ_CDATA;
+
+ CNXK_DPI_STRM_INC(dpi_conf->c_desc, head);
+ /* Take into account errors also. This is similar to
+ * cnxk_dmadev_completed_status().
+ */
+ dpi_conf->stats.completed++;
+
+ return (uintptr_t)op;
+}
diff --git a/drivers/dma/cnxk/meson.build b/drivers/dma/cnxk/meson.build
index e557349368..9cf5453b0b 100644
--- a/drivers/dma/cnxk/meson.build
+++ b/drivers/dma/cnxk/meson.build
@@ -8,6 +8,10 @@ foreach flag: error_cflags
endif
endforeach
-deps += ['bus_pci', 'common_cnxk', 'dmadev']
+driver_sdk_headers = files(
+ 'cnxk_dma_event_dp.h',
+)
+
+deps += ['bus_pci', 'common_cnxk', 'dmadev', 'eventdev']
sources = files('cnxk_dmadev.c', 'cnxk_dmadev_fp.c')
require_iova_in_mbuf = false
diff --git a/drivers/dma/cnxk/version.map b/drivers/dma/cnxk/version.map
new file mode 100644
index 0000000000..6cc1c6aaa5
--- /dev/null
+++ b/drivers/dma/cnxk/version.map
@@ -0,0 +1,9 @@
+INTERNAL {
+ global:
+
+ cn10k_dma_adapter_enqueue;
+ cn9k_dma_adapter_enqueue;
+ cnxk_dma_adapter_dequeue;
+
+ local: *;
+};
--
2.25.1
^ permalink raw reply [flat|nested] 110+ messages in thread
* [PATCH v1 7/7] event/cnxk: support DMA event functions
2023-09-19 13:42 [PATCH v1 1/7] eventdev: introduce DMA event adapter library Amit Prakash Shukla
` (4 preceding siblings ...)
2023-09-19 13:42 ` [PATCH v1 6/7] dma/cnxk: support for DMA event enqueue dequeue Amit Prakash Shukla
@ 2023-09-19 13:42 ` Amit Prakash Shukla
2023-09-21 2:41 ` [PATCH v1 1/7] eventdev: introduce DMA event adapter library Jerin Jacob
2023-09-22 20:13 ` [PATCH v2 00/12] event DMA adapter library support Amit Prakash Shukla
7 siblings, 0 replies; 110+ messages in thread
From: Amit Prakash Shukla @ 2023-09-19 13:42 UTC (permalink / raw)
To: Pavan Nikhilesh, Shijith Thotton
Cc: dev, jerinj, fengchengwen, kevin.laatz, bruce.richardson,
conor.walsh, vattunuru, g.singh, sachin.saxena, hemant.agrawal,
cheng1.jiang, ndabilpuram, anoobj, mb, Amit Prakash Shukla
Added support of dma driver callback assignment to eventdev
enqueue and dequeue. The change also defines dma adapter
capabilities function.
Signed-off-by: Amit Prakash Shukla <amitprakashs@marvell.com>
---
drivers/event/cnxk/cn10k_eventdev.c | 20 ++++++++++++++++++++
drivers/event/cnxk/cn10k_worker.h | 3 +++
drivers/event/cnxk/cn9k_eventdev.c | 17 +++++++++++++++++
drivers/event/cnxk/cn9k_worker.h | 3 +++
drivers/event/cnxk/meson.build | 3 +--
5 files changed, 44 insertions(+), 2 deletions(-)
diff --git a/drivers/event/cnxk/cn10k_eventdev.c b/drivers/event/cnxk/cn10k_eventdev.c
index c5d4be0474..9bb8b8ff01 100644
--- a/drivers/event/cnxk/cn10k_eventdev.c
+++ b/drivers/event/cnxk/cn10k_eventdev.c
@@ -8,6 +8,9 @@
#include "cn10k_cryptodev_ops.h"
#include "cnxk_eventdev.h"
#include "cnxk_worker.h"
+#include "cnxk_dma_event_dp.h"
+
+#include <rte_dmadev_pmd.h>
#define CN10K_SET_EVDEV_DEQ_OP(dev, deq_op, deq_ops) \
deq_op = deq_ops[dev->rx_offloads & (NIX_RX_OFFLOAD_MAX - 1)]
@@ -469,6 +472,8 @@ cn10k_sso_fp_fns_set(struct rte_eventdev *event_dev)
else
event_dev->ca_enqueue = cn10k_cpt_sg_ver1_crypto_adapter_enqueue;
+ event_dev->dma_enqueue = cn10k_dma_adapter_enqueue;
+
if (dev->tx_offloads & NIX_TX_MULTI_SEG_F)
CN10K_SET_EVDEV_ENQ_OP(dev, event_dev->txa_enqueue, sso_hws_tx_adptr_enq_seg);
else
@@ -978,6 +983,19 @@ cn10k_crypto_adapter_vec_limits(const struct rte_eventdev *event_dev,
return 0;
}
+static int
+cn10k_dma_adapter_caps_get(const struct rte_eventdev *event_dev,
+ const struct rte_dma_dev *dma_dev, uint32_t *caps)
+{
+ RTE_SET_USED(dma_dev);
+
+ CNXK_VALID_DEV_OR_ERR_RET(event_dev->dev, "event_cn10k", EINVAL);
+
+ *caps = RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP_FWD;
+
+ return 0;
+}
+
static struct eventdev_ops cn10k_sso_dev_ops = {
.dev_infos_get = cn10k_sso_info_get,
.dev_configure = cn10k_sso_dev_configure,
@@ -1017,6 +1035,8 @@ static struct eventdev_ops cn10k_sso_dev_ops = {
.crypto_adapter_queue_pair_del = cn10k_crypto_adapter_qp_del,
.crypto_adapter_vector_limits_get = cn10k_crypto_adapter_vec_limits,
+ .dma_adapter_caps_get = cn10k_dma_adapter_caps_get,
+
.xstats_get = cnxk_sso_xstats_get,
.xstats_reset = cnxk_sso_xstats_reset,
.xstats_get_names = cnxk_sso_xstats_get_names,
diff --git a/drivers/event/cnxk/cn10k_worker.h b/drivers/event/cnxk/cn10k_worker.h
index e71ab3c523..3d35fcb657 100644
--- a/drivers/event/cnxk/cn10k_worker.h
+++ b/drivers/event/cnxk/cn10k_worker.h
@@ -7,6 +7,7 @@
#include <rte_eventdev.h>
#include "cn10k_cryptodev_event_dp.h"
+#include "cnxk_dma_event_dp.h"
#include "cn10k_rx.h"
#include "cnxk_worker.h"
#include "cn10k_eventdev.h"
@@ -226,6 +227,8 @@ cn10k_sso_hws_post_process(struct cn10k_sso_hws *ws, uint64_t *u64,
/* Mark vector mempool object as get */
RTE_MEMPOOL_CHECK_COOKIES(rte_mempool_from_obj((void *)u64[1]),
(void **)&u64[1], 1, 1);
+ } else if (CNXK_EVENT_TYPE_FROM_TAG(u64[0]) == RTE_EVENT_TYPE_DMADEV) {
+ u64[1] = cnxk_dma_adapter_dequeue(u64[1]);
}
}
diff --git a/drivers/event/cnxk/cn9k_eventdev.c b/drivers/event/cnxk/cn9k_eventdev.c
index f77a9d7085..980932bd12 100644
--- a/drivers/event/cnxk/cn9k_eventdev.c
+++ b/drivers/event/cnxk/cn9k_eventdev.c
@@ -510,6 +510,8 @@ cn9k_sso_fp_fns_set(struct rte_eventdev *event_dev)
sso_hws_dual_tx_adptr_enq);
}
+ event_dev->dma_enqueue = cn9k_dma_adapter_enqueue;
+
event_dev->txa_enqueue_same_dest = event_dev->txa_enqueue;
rte_mb();
#else
@@ -991,6 +993,19 @@ cn9k_tim_caps_get(const struct rte_eventdev *evdev, uint64_t flags,
cn9k_sso_set_priv_mem);
}
+static int
+cn9k_dma_adapter_caps_get(const struct rte_eventdev *event_dev,
+ const struct rte_dma_dev *dma_dev, uint32_t *caps)
+{
+ RTE_SET_USED(dma_dev);
+
+ CNXK_VALID_DEV_OR_ERR_RET(event_dev->dev, "event_cn9k");
+
+ *caps = RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP_FWD;
+
+ return 0;
+}
+
static struct eventdev_ops cn9k_sso_dev_ops = {
.dev_infos_get = cn9k_sso_info_get,
.dev_configure = cn9k_sso_dev_configure,
@@ -1027,6 +1042,8 @@ static struct eventdev_ops cn9k_sso_dev_ops = {
.crypto_adapter_queue_pair_add = cn9k_crypto_adapter_qp_add,
.crypto_adapter_queue_pair_del = cn9k_crypto_adapter_qp_del,
+ .dma_adapter_caps_get = cn9k_dma_adapter_caps_get,
+
.xstats_get = cnxk_sso_xstats_get,
.xstats_reset = cnxk_sso_xstats_reset,
.xstats_get_names = cnxk_sso_xstats_get_names,
diff --git a/drivers/event/cnxk/cn9k_worker.h b/drivers/event/cnxk/cn9k_worker.h
index 9ddab095ac..6ac6fffc86 100644
--- a/drivers/event/cnxk/cn9k_worker.h
+++ b/drivers/event/cnxk/cn9k_worker.h
@@ -11,6 +11,7 @@
#include "cnxk_ethdev.h"
#include "cnxk_eventdev.h"
#include "cnxk_worker.h"
+#include "cnxk_dma_event_dp.h"
#include "cn9k_cryptodev_ops.h"
#include "cn9k_ethdev.h"
@@ -214,6 +215,8 @@ cn9k_sso_hws_post_process(uint64_t *u64, uint64_t mbuf, const uint32_t flags,
if (flags & NIX_RX_OFFLOAD_TSTAMP_F)
cn9k_sso_process_tstamp(u64[1], mbuf, tstamp[port]);
u64[1] = mbuf;
+ } else if (CNXK_EVENT_TYPE_FROM_TAG(u64[0]) == RTE_EVENT_TYPE_DMADEV) {
+ u64[1] = cnxk_dma_adapter_dequeue(u64[1]);
}
}
diff --git a/drivers/event/cnxk/meson.build b/drivers/event/cnxk/meson.build
index 51f1be8848..649419d5d3 100644
--- a/drivers/event/cnxk/meson.build
+++ b/drivers/event/cnxk/meson.build
@@ -314,8 +314,7 @@ foreach flag: extra_flags
endif
endforeach
-deps += ['bus_pci', 'common_cnxk', 'net_cnxk', 'crypto_cnxk']
-
+deps += ['bus_pci', 'common_cnxk', 'net_cnxk', 'crypto_cnxk', 'dma_cnxk']
require_iova_in_mbuf = false
annotate_locks = false
--
2.25.1
^ permalink raw reply [flat|nested] 110+ messages in thread
* Re: [PATCH v1 1/7] eventdev: introduce DMA event adapter library
2023-09-19 13:42 [PATCH v1 1/7] eventdev: introduce DMA event adapter library Amit Prakash Shukla
` (5 preceding siblings ...)
2023-09-19 13:42 ` [PATCH v1 7/7] event/cnxk: support DMA event functions Amit Prakash Shukla
@ 2023-09-21 2:41 ` Jerin Jacob
2023-09-21 6:42 ` [EXT] " Amit Prakash Shukla
2023-09-22 20:13 ` [PATCH v2 00/12] event DMA adapter library support Amit Prakash Shukla
7 siblings, 1 reply; 110+ messages in thread
From: Jerin Jacob @ 2023-09-21 2:41 UTC (permalink / raw)
To: Amit Prakash Shukla
Cc: Jerin Jacob, dev, fengchengwen, kevin.laatz, bruce.richardson,
conor.walsh, vattunuru, g.singh, sachin.saxena, hemant.agrawal,
cheng1.jiang, ndabilpuram, anoobj, mb
On Tue, Sep 19, 2023 at 7:12 PM Amit Prakash Shukla
<amitprakashs@marvell.com> wrote:
>
> Introduce event DMA adapter APIs. The change provides information
> on adapter modes and usage. Application can use this event adapter
> interface to transfer packets between DMA device and event device.
>
> Signed-off-by: Amit Prakash Shukla <amitprakashs@marvell.com>
Please have a cover letter and document changes from RFC for easy review.
The driver changes can be accepted after rc1. So please have two
series better review
First series - Targeting to merge for rc1
-API Header file with programing guide
-SW implementation
-Test cases
Second series: - Targeting to merge for rc2.
-cnxk driver changes
> ---
> 13 files changed, 3336 insertions(+), 5 deletions(-)
> create mode 100644 doc/guides/prog_guide/event_dma_adapter.rst
> create mode 100644 doc/guides/prog_guide/img/event_dma_adapter_op_forward.svg
> create mode 100644 doc/guides/prog_guide/img/event_dma_adapter_op_new.svg
> create mode 100644 lib/eventdev/rte_event_dma_adapter.h
Missing updates to MAINTAINERS and doc/guides/rel_notes/release_23_11.rst
> diff --git a/doc/guides/eventdevs/features/default.ini b/doc/guides/eventdevs/features/default.ini
> index 00360f60c6..fda8baf487 100644
> --- a/doc/guides/eventdevs/features/default.ini
> +++ b/doc/guides/eventdevs/features/default.ini
> @@ -44,6 +44,14 @@ internal_port_op_fwd =
> internal_port_qp_ev_bind =
> session_private_data =
>
> +;
> +; Features of a default DMA adapter.
> +;
> +[DMA adapter Features]
> +internal_port_op_new =
> +internal_port_op_fwd =
> +internal_port_qp_ev_bind =
For dmadev there is qp (queue pair) change to vchan.
> diff --git a/doc/guides/prog_guide/event_dma_adapter.rst b/doc/guides/prog_guide/event_dma_adapter.rst
Snip
> +Adding vchan queue to the adapter instance
Keep only vchan (remove queue) to align with dmadev.
> +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> +
> +dmadev device id and vchan queue are configured using dmadev APIs. For more information
> +see :doc:`here <dmadev>`.
> +
> +.. code-block:: c
> +
> + struct rte_dma_vchan_conf vchan_conf;
> + struct rte_dma_conf dev_conf;
> + uint8_t dev_id = 0;
> + uint16_t vchan = 0;
> +
> + rte_dma_configure(dev_id, &dev_conf);
> + rte_dma_vchan_setup(dev_id, vhcan, &vchan_conf);
> +
> +These dmadev id and vchan are added to the instance using the
> +``rte_event_dma_adapter_vchan_queue_add()`` API. The same is removed using
Keep only vchan (remove queue) to align with dmadev.
> +``rte_event_dma_adapter_vchan_queue_del()`` API. If hardware supports
Keep only vchan (remove queue) to align with dmadev.
> +``RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_QP_EV_BIND`` capability, event information must be passed to
QP->VCHAN
> +the add API.
> +
> +.. code-block:: c
> +
> + uint32_t cap;
> + int ret;
> +
> + ret = rte_event_dma_adapter_caps_get(evdev_id, dma_dev_id, &cap);
> + if (cap & RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_QP_EV_BIND) {
> + struct rte_event event;
> +
> + rte_event_dma_adapter_vchan_queue_add(id, dma_dev_id, vchan, &conf);
Keep only vchan (remove queue) to align with dmadev.
> + } else
> + rte_event_dma_adapter_vchan_queue_add(id, dma_dev_id, vchan, NULL);
Keep only vchan (remove queue) to align with dmadev.
> +
> +
> +
> +Set event request / response information
> +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> +
> +In the RTE_EVENT_DMA_ADAPTER_OP_FORWARD mode, the application specifies the dmadev ID and
> +vchan ID (request information) in addition to the event information (response information)
> +needed to enqueue an event after the DMA operation has completed. The request and response
> +information are specified in the ``struct rte_event_dma_metadata``.
Symbol rte_event_dma_metadata not found in header file. Remove this
section if it is not used.
> +/**
> + * This API may change without prior notice
> + *
> + * Add DMA queue pair to event device. This callback is invoked if
> + * the caps returned from rte_event_dma_adapter_caps_get(, dmadev_id)
> + * has RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_* set.
> + *
> + * @param dev
> + * Event device pointer
> + *
> + * @param dmadev
> + * DMADEV pointer
> + *
> + * @param queue_pair_id
> + * DMADEV queue pair identifier.
> + *
> + * @param event
> + * Event information required for binding dmadev queue pair to event queue.
> + * This structure will have a valid value for only those HW PMDs supporting
> + * @see RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_QP_EV_BIND capability.
QP->VCHAN
> + *
> + * @return
> + * - 0: Success, dmadev queue pair added successfully.
> + * - <0: Error code returned by the driver function.
> + *
> + */
> +typedef int (*eventdev_dma_adapter_queue_pair_add_t)(const struct rte_eventdev *dev,
queue_pair->vchan
> + const struct rte_dma_dev *dmadev,
> + int32_t queue_pair_id,
queue_pair->vchan
> +++ b/lib/eventdev/rte_event_dma_adapter.h
> @@ -0,0 +1,641 @@
> +/* SPDX-License-Identifier: BSD-3-Clause
> + * Copyright (c) 2023 Marvell.
> + */
> +
> +#ifndef RTE_EVENT_DMA_ADAPTER
> +#define RTE_EVENT_DMA_ADAPTER
> +
> +/**
> + * @file rte_event_dma_adapter.h
> + *
> + * @warning
> + * @b EXPERIMENTAL:
> + * All functions in this file may be changed or removed without prior notice.
> + *
> + * DMA Event Adapter API.
> + *
> + * Eventdev library provides adapters to bridge between various components for providing new
> + * event source. The event DMA adapter is one of those adapters which is intended to bridge
> + * between event devices and DMA devices.
> + *
> + * The DMA adapter adds support to enqueue / dequeue DMA operations to / from event device. The
> + * packet flow between DMA device and the event device can be accomplished using both SW and HW
> + * based transfer mechanisms. The adapter uses an EAL service core function for SW based packet
> + * transfer and uses the eventdev PMD functions to configure HW based packet transfer between the
> + * DMA device and the event device.
> + *
> + * The application can choose to submit a DMA operation directly to an DMA device or send it to the
> + * DMA adapter via eventdev based on RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP_FWD capability. The
> + * first mode is known as the event new (RTE_EVENT_DMA_ADAPTER_OP_NEW) mode and the second as the
> + * event forward (RTE_EVENT_DMA_ADAPTER_OP_FORWARD) mode. The choice of mode can be specified while
> + * creating the adapter. In the former mode, it is an application responsibility to enable ingress
> + * packet ordering. In the latter mode, it is the adapter responsibility to enable the ingress
> + * packet ordering.
> + *
> + *
> + * Working model of RTE_EVENT_DMA_ADAPTER_OP_NEW mode:
> + *
> + * +--------------+ +--------------+
> + * | | | DMA stage |
> + * | Application |---[2]-->| + enqueue to |
> + * | | | dmadev |
> + * +--------------+ +--------------+
> + * ^ ^ |
> + * | | [3]
> + * [6] [1] |
> + * | | |
> + * +--------------+ |
> + * | | |
> + * | Event device | |
> + * | | |
> + * +--------------+ |
> + * ^ |
> + * | |
> + * [5] |
> + * | v
> + * +--------------+ +--------------+
> + * | | | |
> + * | DMA adapter |<--[4]---| dmadev |
> + * | | | |
> + * +--------------+ +--------------+
> + *
> + *
> + * [1] Application dequeues events from the previous stage.
> + * [2] Application prepares the DMA operations.
> + * [3] DMA operations are submitted to dmadev by application.
> + * [4] DMA adapter dequeues DMA completions from dmadev.
> + * [5] DMA adapter enqueues events to the eventdev.
> + * [6] Application dequeues from eventdev for further processing.
> + *
> + * In the RTE_EVENT_DMA_ADAPTER_OP_NEW mode, application submits DMA operations directly to DMA
> + * device. The DMA adapter then dequeues DMA completions from DMA device and enqueue events to the
> + * event device. This mode does not ensure ingress ordering, if the application directly enqueues
> + * to dmadev without going through DMA / atomic stage i.e. removing item [1] and [2].
> + *
> + * Events dequeued from the adapter will be treated as new events. In this mode, application needs
> + * to specify event information (response information) which is needed to enqueue an event after the
> + * DMA operation is completed.
> + *
> + *
> + * Working model of RTE_EVENT_DMA_ADAPTER_OP_FORWARD mode:
> + *
> + * +--------------+ +--------------+
> + * --[1]-->| |---[2]-->| Application |
> + * | Event device | | in |
> + * <--[8]--| |<--[3]---| Ordered stage|
> + * +--------------+ +--------------+
> + * ^ |
> + * | [4]
> + * [7] |
> + * | v
> + * +----------------+ +--------------+
> + * | |--[5]->| |
> + * | DMA adapter | | dmadev |
> + * | |<-[6]--| |
> + * +----------------+ +--------------+
> + *
> + *
> + * [1] Events from the previous stage.
> + * [2] Application in ordered stage dequeues events from eventdev.
> + * [3] Application enqueues DMA operations as events to eventdev.
> + * [4] DMA adapter dequeues event from eventdev.
> + * [5] DMA adapter submits DMA operations to dmadev (Atomic stage).
> + * [6] DMA adapter dequeues DMA completions from dmadev
> + * [7] DMA adapter enqueues events to the eventdev
> + * [8] Events to the next stage
> + *
> + * In the event forward (RTE_EVENT_DMA_ADAPTER_OP_FORWARD) mode, if the HW supports the capability
> + * RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP_FWD, application can directly submit the DMA
> + * operations to the dmadev. If not, application retrieves the event port of the DMA adapter
> + * through the API, rte_event_DMA_adapter_event_port_get(). Then, links its event queue to this
> + * port and starts enqueuing DMA operations as events to the eventdev. The adapter then dequeues
> + * the events and submits the DMA operations to the dmadev. After the DMA completions, the adapter
> + * enqueues events to the event device.
> + *
> + * Application can use this mode, when ingress packet ordering is needed. Events dequeued from the
> + * adapter will be treated as forwarded events. In this mode, the application needs to specify the
> + * dmadev ID and queue pair ID (request information) needed to enqueue an DMA operation in addition
Grep queue pair and change to vchan everywhere.
> + * to the event information (response information) needed to enqueue an event after the DMA
> + * operation has completed.
> + *
> + * The event DMA adapter provides common APIs to configure the packet flow from the DMA device to
> + * event devices for both SW and HW based transfers. The DMA event adapter's functions are:
> + *
> + * - rte_event_dma_adapter_create_ext()
> + * - rte_event_dma_adapter_create()
> + * - rte_event_dma_adapter_free()
> + * - rte_event_dma_adapter_vchan_queue_add()
> + * - rte_event_dma_adapter_vchan_queue_del()
Remove queue
> + * - rte_event_dma_adapter_start()
> + * - rte_event_dma_adapter_stop()
> + * - rte_event_dma_adapter_stats_get()
> + * - rte_event_dma_adapter_stats_reset()
> + *
> + * The application creates an instance using rte_event_dma_adapter_create() or
> + * rte_event_dma_adapter_create_ext().
> + *
> + * dmadev queue pair addition / deletion is done using the rte_event_dma_adapter_vchan_queue_add() /
> + * rte_event_dma_adapter_vchan_queue_del() APIs. If HW supports the capability
> + * RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_QP_EV_BIND, event information must be passed to the add
> + * API.
> + *
> + */
> +
> +#include <stdint.h>
> +
> +#include "rte_eventdev.h"
use <> and sort in alphanetical orer.
> +#include <rte_common.h>
> +#include <rte_dmadev_pmd.h>
> +
> +#ifdef __cplusplus
> +extern "C" {
> +#endif
> +
> +/**
> + * A structure used to hold event based DMA operation entry.
> + */
> +struct rte_event_dma_adapter_op {
> + struct rte_dma_sge *src_seg;
> + /**< Source segments. */
> + struct rte_dma_sge *dst_seg;
> + /**< Destination segments. */
> + uint16_t nb_src;
> + /**< Number of source segments. */
> + uint16_t nb_dst;
> + /**< Number of destination segments. */
> + uint64_t flags;
> + /**< Flags related to the operation.
> + * @see RTE_DMA_OP_FLAG_*
> + */
> + struct rte_mempool *op_mp;
> + /**< Mempool from which op is allocated. */
> +};
> +
> +/**
> + * DMA event adapter mode
> + */
> +enum rte_event_dma_adapter_mode {
> + RTE_EVENT_DMA_ADAPTER_OP_NEW,
> + /**< Start the DMA adapter in event new mode.
> + * @see RTE_EVENT_OP_NEW.
> + *
> + * Application submits DMA operations to the dmadev. Adapter only dequeues the DMA
> + * completions from dmadev and enqueue events to the eventdev.
> + */
> +
> + RTE_EVENT_DMA_ADAPTER_OP_FORWARD,
> + /**< Start the DMA adapter in event forward mode.
> + * @see RTE_EVENT_OP_FORWARD.
> + *
> + * Application submits DMA requests as events to the DMA adapter or DMA device based on
> + * RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP_FWD capability. DMA completions are enqueued
> + * back to the eventdev by DMA adapter.
> + */
> +};
> +
> +/**
> + * DMA event request structure will be filled by application to provide event request information to
> + * the adapter.
> + */
> +struct rte_event_dma_request {
It is got duplicated by rte_event_dma_adapter_op. Right? if so,
please remove it.
> + uint8_t resv[8];
> + /**< Overlaps with first 8 bytes of struct rte_event that encode the response event
> + * information. Application is expected to fill in struct rte_event response_info.
> + */
> +
> + int16_t dmadev_id;
> + /**< DMA device ID to be used */
> +
> + uint16_t queue_pair_id;
> + /**< DMA queue pair ID to be used */
> +
> + uint32_t rsvd;
> + /**< Reserved bits */
> +};
> +/**
> + * @warning
> + * @b EXPERIMENTAL: this API may change without prior notice.
Header file has this banner. No need to duplcate for EVERY functiion.
> +
> +/**
> + * @warning
> + * @b EXPERIMENTAL: this API may change without prior notice.
> + *
> + * Add a vchan queue to an event DMA adapter.
Remove queue
> + *
> + * @param id
> + * Adapter identifier.
> + * @param dmadev_id
> + * dmadev identifier.
> + * @param queue_pair_id
vchan_id
> + * DMA device vchan queue identifier. If queue_pair_id is set -1, adapter adds all the
> + * preconfigured queue pairs to the instance.
> + * @param event
> + * If HW supports dmadev queue pair to event queue binding, application is expected to fill in
> + * event information, else it will be NULL.
> + * @see RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_QP_EV_BIND
> + *
> + * @return
> + * - 0: Success, vchan queue added correctly.
> + * - <0: Error code on failure.
> + */
> +__rte_experimental
> +int rte_event_dma_adapter_vchan_queue_add(uint8_t id, int16_t dmadev_id, int32_t queue_pair_id,
Remove queue
> + const struct rte_event *event);
> +
> +/**
> + * @warning
> + * @b EXPERIMENTAL: this API may change without prior notice.
> + *
> + * Delete a vchan queue from an event DMA adapter.
> + *
> + * @param id
> + * Adapter identifier.
> + * @param dmadev_id
> + * DMA device identifier.
> + * @param queue_pair_id
vchan_id
> + * DMA device vchan queue identifier.
> + *
> + * @return
> + * - 0: Success, vchan queue deleted successfully.
> + * - <0: Error code on failure.
> + */
> +__rte_experimental
> +int rte_event_dma_adapter_vchan_queue_del(uint8_t id, int16_t dmadev_id, int32_t queue_pair_id);
Remove queue
> +
> +/**
> + * @warning
> + * @b EXPERIMENTAL: this API may change without prior notice.
> + *
> + * Enqueue a burst of DMA operations as event objects supplied in *rte_event* structure on an event
> + * DMA adapter designated by its event *evdev_id* through the event port specified by *port_id*.
> + * This function is supported if the eventdev PMD has the
> + * #RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP_FWD capability flag set.
> + *
> + * The *nb_events* parameter is the number of event objects to enqueue that are supplied in the
> + * *ev* array of *rte_event* structure.
> + *
> + * The rte_event_dma_adapter_enqueue() function returns the number of event objects it actually
> + * enqueued. A return value equal to *nb_events* means that all event objects have been enqueued.
> + *
> + * @param evdev_id
> + * The identifier of the device.
> + * @param port_id
> + * The identifier of the event port.
> + * @param ev
> + * Points to an array of *nb_events* objects of type *rte_event* structure which contain the
> + * event object enqueue operations to be processed.
Documenent the use of rte_event_dma_adapter_ops
> + * @param nb_events
> + * The number of event objects to enqueue, typically number of
> + * rte_event_port_attr_get(...RTE_EVENT_PORT_ATTR_ENQ_DEPTH...) available for this port.
> + *
> + * @return
> + * The number of event objects actually enqueued on the event device. The return value can be
> + * less than the value of the *nb_events* parameter when the event devices queue is full or if
> + * invalid parameters are specified in a *rte_event*. If the return value is less than *nb_events*,
> + * the remaining events at the end of ev[] are not consumed and the caller has to take care of them,
> + * and rte_errno is set accordingly. Possible errno values include:
> + *
> + * - EINVAL: The port ID is invalid, device ID is invalid, an event's queue ID is invalid, or an
> + * event's sched type doesn't match the capabilities of the destination queue.
> + * - ENOSPC: The event port was backpressured and unable to enqueue one or more events. This
> + * error code is only applicable to closed systems.
> + */
> +__rte_experimental
> +uint16_t rte_event_dma_adapter_enqueue(uint8_t evdev_id, uint8_t port_id, struct rte_event ev[],
> + uint16_t nb_events);
> +
> +#ifdef __cplusplus
> +}
> +#endif
> +
> +
> +#define RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_QP_EV_BIND 0x4
QP->VCHAN
> +/**< Flag indicates HW is capable of mapping DMA queue pair to
queue pair to vhcan
> + * event queue.
> + */
> +
> +/**
> + * Retrieve the event device's DMA adapter capabilities for the
> + * specified dmadev device
> + *
> + * @param dev_id
> + * The identifier of the device.
> + *
> + * @param dmadev_id
> + * The identifier of the dmadev device.
> + *
> + * @param[out] caps
> + * A pointer to memory filled with event adapter capabilities.
> + * It is expected to be pre-allocated & initialized by caller.
> + *
> + * @return
> + * - 0: Success, driver provides event adapter capabilities for the
> + * dmadev device.
> + * - <0: Error code returned by the driver function.
> + *
> + */
> +int
> +rte_event_dma_adapter_caps_get(uint8_t dev_id, int16_t dmadev_id, uint32_t *caps);
> +
> extern struct rte_event_fp_ops rte_event_fp_ops[RTE_EVENT_MAX_DEVS];
> diff --git a/lib/eventdev/version.map b/lib/eventdev/version.map
> index 7ce09a87bb..597a5c9cda 100644
> --- a/lib/eventdev/version.map
> +++ b/lib/eventdev/version.map
> @@ -134,6 +134,21 @@ EXPERIMENTAL {
>
> # added in 23.11
> rte_event_eth_rx_adapter_create_ext_with_params;
> + rte_event_dma_adapter_create_ext;
> + rte_event_dma_adapter_create;
> + rte_event_dma_adapter_free;
> + rte_event_dma_adapter_event_port_get;
> + rte_event_dma_adapter_vchan_queue_add;
> + rte_event_dma_adapter_vchan_queue_del;
> + rte_event_dma_adapter_service_id_get;
> + rte_event_dma_adapter_start;
> + rte_event_dma_adapter_stop;
> + rte_event_dma_adapter_runtime_params_init;
> + rte_event_dma_adapter_runtime_params_set;
> + rte_event_dma_adapter_runtime_params_get;
> + rte_event_dma_adapter_stats_get;
> + rte_event_dma_adapter_stats_reset;
> + rte_event_dma_adapter_enqueue;
sort in alphabetical order.
Phew :-)
^ permalink raw reply [flat|nested] 110+ messages in thread
* RE: [EXT] Re: [PATCH v1 1/7] eventdev: introduce DMA event adapter library
2023-09-21 2:41 ` [PATCH v1 1/7] eventdev: introduce DMA event adapter library Jerin Jacob
@ 2023-09-21 6:42 ` Amit Prakash Shukla
0 siblings, 0 replies; 110+ messages in thread
From: Amit Prakash Shukla @ 2023-09-21 6:42 UTC (permalink / raw)
To: Jerin Jacob
Cc: Jerin Jacob Kollanukkaran, dev, fengchengwen, kevin.laatz,
bruce.richardson, conor.walsh, Vamsi Krishna Attunuru, g.singh,
sachin.saxena, hemant.agrawal, cheng1.jiang,
Nithin Kumar Dabilpuram, Anoob Joseph, mb
Hi Jerin,
Thanks for the review and feedback. I will make all suggested changes in next version of patch.
Thanks,
Amit Shukla
> -----Original Message-----
> From: Jerin Jacob <jerinjacobk@gmail.com>
> Sent: Thursday, September 21, 2023 8:11 AM
> To: Amit Prakash Shukla <amitprakashs@marvell.com>
> Cc: Jerin Jacob Kollanukkaran <jerinj@marvell.com>; dev@dpdk.org;
> fengchengwen@huawei.com; kevin.laatz@intel.com;
> bruce.richardson@intel.com; conor.walsh@intel.com; Vamsi Krishna
> Attunuru <vattunuru@marvell.com>; g.singh@nxp.com;
> sachin.saxena@oss.nxp.com; hemant.agrawal@nxp.com;
> cheng1.jiang@intel.com; Nithin Kumar Dabilpuram
> <ndabilpuram@marvell.com>; Anoob Joseph <anoobj@marvell.com>;
> mb@smartsharesystems.com
> Subject: [EXT] Re: [PATCH v1 1/7] eventdev: introduce DMA event adapter
> library
>
> External Email
>
> ----------------------------------------------------------------------
> On Tue, Sep 19, 2023 at 7:12 PM Amit Prakash Shukla
> <amitprakashs@marvell.com> wrote:
> >
> > Introduce event DMA adapter APIs. The change provides information on
> > adapter modes and usage. Application can use this event adapter
> > interface to transfer packets between DMA device and event device.
> >
> > Signed-off-by: Amit Prakash Shukla <amitprakashs@marvell.com>
<snip>
^ permalink raw reply [flat|nested] 110+ messages in thread
* [PATCH v2 00/12] event DMA adapter library support
2023-09-19 13:42 [PATCH v1 1/7] eventdev: introduce DMA event adapter library Amit Prakash Shukla
` (6 preceding siblings ...)
2023-09-21 2:41 ` [PATCH v1 1/7] eventdev: introduce DMA event adapter library Jerin Jacob
@ 2023-09-22 20:13 ` Amit Prakash Shukla
2023-09-22 20:13 ` [PATCH v2 01/12] eventdev: introduce event DMA adapter library Amit Prakash Shukla
` (13 more replies)
7 siblings, 14 replies; 110+ messages in thread
From: Amit Prakash Shukla @ 2023-09-22 20:13 UTC (permalink / raw)
Cc: dev, jerinj, fengchengwen, kevin.laatz, bruce.richardson,
conor.walsh, vattunuru, g.singh, sachin.saxena, hemant.agrawal,
cheng1.jiang, ndabilpuram, anoobj, mb, Amit Prakash Shukla
This series adds support for event DMA adapter library. API's defined
as part of this library can be used by the application for DMA transfer
of data using event based mechanism.
v2:
- Resolved review comments.
- Patch split into multiple patches.
Amit Prakash Shukla (12):
eventdev: introduce event DMA adapter library
eventdev: api to get DMA adapter capabilities
eventdev: add DMA adapter API to create and free
eventdev: api support for vchan add and delete
eventdev: add support for service function
eventdev: api support for DMA adapter start stop
eventdev: api support to get DMA adapter service ID
eventdev: add DMA adapter support for runtime params
eventdev: add support for DMA adapter stats
eventdev: add support for DMA adapter enqueue
eventdev: add DMA adapter port get
app/test: add event DMA adapter auto-test
MAINTAINERS | 5 +
app/test/meson.build | 1 +
app/test/test_event_dma_adapter.c | 808 ++++++++++
config/rte_config.h | 1 +
doc/api/doxy-api-index.md | 1 +
doc/guides/eventdevs/features/default.ini | 8 +
doc/guides/prog_guide/event_dma_adapter.rst | 264 +++
doc/guides/prog_guide/eventdev.rst | 8 +-
.../img/event_dma_adapter_op_forward.svg | 1086 +++++++++++++
.../img/event_dma_adapter_op_new.svg | 1079 +++++++++++++
doc/guides/prog_guide/index.rst | 1 +
doc/guides/rel_notes/release_23_11.rst | 4 +-
lib/eventdev/eventdev_pmd.h | 175 +-
lib/eventdev/eventdev_private.c | 10 +
lib/eventdev/meson.build | 4 +-
lib/eventdev/rte_event_dma_adapter.c | 1431 +++++++++++++++++
lib/eventdev/rte_event_dma_adapter.h | 582 +++++++
lib/eventdev/rte_eventdev.c | 23 +
lib/eventdev/rte_eventdev.h | 43 +
lib/eventdev/rte_eventdev_core.h | 8 +-
lib/eventdev/version.map | 15 +
lib/meson.build | 2 +-
22 files changed, 5551 insertions(+), 8 deletions(-)
create mode 100644 app/test/test_event_dma_adapter.c
create mode 100644 doc/guides/prog_guide/event_dma_adapter.rst
create mode 100644 doc/guides/prog_guide/img/event_dma_adapter_op_forward.svg
create mode 100644 doc/guides/prog_guide/img/event_dma_adapter_op_new.svg
create mode 100644 lib/eventdev/rte_event_dma_adapter.c
create mode 100644 lib/eventdev/rte_event_dma_adapter.h
--
2.25.1
^ permalink raw reply [flat|nested] 110+ messages in thread
* [PATCH v2 01/12] eventdev: introduce event DMA adapter library
2023-09-22 20:13 ` [PATCH v2 00/12] event DMA adapter library support Amit Prakash Shukla
@ 2023-09-22 20:13 ` Amit Prakash Shukla
2023-09-22 20:13 ` [PATCH v2 02/12] eventdev: api to get DMA adapter capabilities Amit Prakash Shukla
` (12 subsequent siblings)
13 siblings, 0 replies; 110+ messages in thread
From: Amit Prakash Shukla @ 2023-09-22 20:13 UTC (permalink / raw)
To: Thomas Monjalon, Amit Prakash Shukla, Jerin Jacob
Cc: dev, fengchengwen, kevin.laatz, bruce.richardson, conor.walsh,
vattunuru, g.singh, sachin.saxena, hemant.agrawal, cheng1.jiang,
ndabilpuram, anoobj, mb
Introduce event DMA adapter APIs. The change provides information
on adapter modes and usage. Application can use this event adapter
interface to transfer packets between DMA device and event device.
Signed-off-by: Amit Prakash Shukla <amitprakashs@marvell.com>
---
MAINTAINERS | 5 +
doc/api/doxy-api-index.md | 1 +
doc/guides/eventdevs/features/default.ini | 8 +
doc/guides/prog_guide/event_dma_adapter.rst | 264 ++++
doc/guides/prog_guide/eventdev.rst | 8 +-
.../img/event_dma_adapter_op_forward.svg | 1086 +++++++++++++++++
.../img/event_dma_adapter_op_new.svg | 1079 ++++++++++++++++
doc/guides/prog_guide/index.rst | 1 +
doc/guides/rel_notes/release_23_11.rst | 4 +-
lib/eventdev/eventdev_pmd.h | 175 ++-
lib/eventdev/eventdev_private.c | 10 +
lib/eventdev/meson.build | 1 +
lib/eventdev/rte_event_dma_adapter.h | 582 +++++++++
lib/eventdev/rte_eventdev.h | 43 +
lib/eventdev/rte_eventdev_core.h | 8 +-
lib/eventdev/version.map | 15 +
16 files changed, 3284 insertions(+), 6 deletions(-)
create mode 100644 doc/guides/prog_guide/event_dma_adapter.rst
create mode 100644 doc/guides/prog_guide/img/event_dma_adapter_op_forward.svg
create mode 100644 doc/guides/prog_guide/img/event_dma_adapter_op_new.svg
create mode 100644 lib/eventdev/rte_event_dma_adapter.h
diff --git a/MAINTAINERS b/MAINTAINERS
index a926155f26..a6b8fc88d0 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -548,6 +548,11 @@ F: drivers/raw/skeleton/
F: app/test/test_rawdev.c
F: doc/guides/prog_guide/rawdev.rst
+Eventdev DMA Adapter API
+M: Amit Prakash Shukla <amitprakashs@marvell.com>
+T: git://dpdk.org/next/dpdk-next-eventdev
+F: lib/eventdev/*dma_adapter*
+F: doc/guides/prog_guide/event_dma_adapter.rst
Memory Pool Drivers
-------------------
diff --git a/doc/api/doxy-api-index.md b/doc/api/doxy-api-index.md
index fdeda13932..b7df7be4d9 100644
--- a/doc/api/doxy-api-index.md
+++ b/doc/api/doxy-api-index.md
@@ -29,6 +29,7 @@ The public API headers are grouped by topics:
[event_eth_tx_adapter](@ref rte_event_eth_tx_adapter.h),
[event_timer_adapter](@ref rte_event_timer_adapter.h),
[event_crypto_adapter](@ref rte_event_crypto_adapter.h),
+ [event_dma_adapter](@ref rte_event_dma_adapter.h),
[rawdev](@ref rte_rawdev.h),
[metrics](@ref rte_metrics.h),
[bitrate](@ref rte_bitrate.h),
diff --git a/doc/guides/eventdevs/features/default.ini b/doc/guides/eventdevs/features/default.ini
index 00360f60c6..73a52d915b 100644
--- a/doc/guides/eventdevs/features/default.ini
+++ b/doc/guides/eventdevs/features/default.ini
@@ -44,6 +44,14 @@ internal_port_op_fwd =
internal_port_qp_ev_bind =
session_private_data =
+;
+; Features of a default DMA adapter.
+;
+[DMA adapter Features]
+internal_port_op_new =
+internal_port_op_fwd =
+internal_port_vchan_ev_bind =
+
;
; Features of a default Timer adapter.
;
diff --git a/doc/guides/prog_guide/event_dma_adapter.rst b/doc/guides/prog_guide/event_dma_adapter.rst
new file mode 100644
index 0000000000..eeb9ce6dfd
--- /dev/null
+++ b/doc/guides/prog_guide/event_dma_adapter.rst
@@ -0,0 +1,264 @@
+.. SPDX-License-Identifier: BSD-3-Clause
+ Copyright (c) 2023 Marvell.
+
+Event DMA Adapter Library
+=========================
+
+DPDK :doc:`Eventdev library <eventdev>` provides event driven programming model with features
+to schedule events. :doc:`DMA Device library <dmadev>` provides an interface to DMA poll mode
+drivers that support DMA operations. Event DMA Adapter is intended to bridge between the event
+device and the DMA device.
+
+Packet flow from DMA device to the event device can be accomplished using software and hardware
+based transfer mechanisms. The adapter queries an eventdev PMD to determine which mechanism to
+be used. The adapter uses an EAL service core function for software based packet transfer and
+uses the eventdev PMD functions to configure hardware based packet transfer between DMA device
+and the event device. DMA adapter uses a new event type called ``RTE_EVENT_TYPE_DMADEV`` to
+indicate the source of event.
+
+Application can choose to submit an DMA operation directly to an DMA device or send it to an DMA
+adapter via eventdev based on RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP_FWD capability. The
+first mode is known as the event new (RTE_EVENT_DMA_ADAPTER_OP_NEW) mode and the second as the
+event forward (RTE_EVENT_DMA_ADAPTER_OP_FORWARD) mode. Choice of mode can be specified while
+creating the adapter. In the former mode, it is the application's responsibility to enable
+ingress packet ordering. In the latter mode, it is the adapter's responsibility to enable
+ingress packet ordering.
+
+
+Adapter Modes
+-------------
+
+RTE_EVENT_DMA_ADAPTER_OP_NEW mode
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+In the RTE_EVENT_DMA_ADAPTER_OP_NEW mode, application submits DMA operations directly to an DMA
+device. The adapter then dequeues DMA completions from the DMA device and enqueues them as events
+to the event device. This mode does not ensure ingress ordering as the application directly
+enqueues to the dmadev without going through DMA/atomic stage. In this mode, events dequeued
+from the adapter are treated as new events. The application has to specify event information
+(response information) which is needed to enqueue an event after the DMA operation is completed.
+
+.. _figure_event_dma_adapter_op_new:
+
+.. figure:: img/event_dma_adapter_op_new.*
+
+ Working model of ``RTE_EVENT_DMA_ADAPTER_OP_NEW`` mode
+
+
+RTE_EVENT_DMA_ADAPTER_OP_FORWARD mode
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+In the ``RTE_EVENT_DMA_ADAPTER_OP_FORWARD`` mode, if the event PMD and DMA PMD supports internal
+event port (``RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP_FWD``), the application should use
+``rte_event_dma_adapter_enqueue()`` API to enqueue DMA operations as events to DMA adapter. If
+not, application retrieves DMA adapter's event port using ``rte_event_dma_adapter_event_port_get()``
+API, links its event queue to this port and starts enqueuing DMA operations as events to eventdev
+using ``rte_event_enqueue_burst()``. The adapter then dequeues the events and submits the DMA
+operations to the dmadev. After the DMA operation is complete, the adapter enqueues events to the
+event device.
+
+Applications can use this mode when ingress packet ordering is needed. In this mode, events
+dequeued from the adapter will be treated as forwarded events. Application has to specify event
+information (response information) needed to enqueue the event after the DMA operation has
+completed.
+
+.. _figure_event_dma_adapter_op_forward:
+
+.. figure:: img/event_dma_adapter_op_forward.*
+
+ Working model of ``RTE_EVENT_DMA_ADAPTER_OP_FORWARD`` mode
+
+
+API Overview
+------------
+
+This section has a brief introduction to the event DMA adapter APIs. The application is expected
+to create an adapter which is associated with a single eventdev, then add dmadev and vchan to the
+adapter instance.
+
+
+Create an adapter instance
+~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+An adapter instance is created using ``rte_event_dma_adapter_create()``. This function is called
+with event device to be associated with the adapter and port configuration for the adapter to
+setup an event port (if the adapter needs to use a service function).
+
+Adapter can be started in ``RTE_EVENT_DMA_ADAPTER_OP_NEW`` or ``RTE_EVENT_DMA_ADAPTER_OP_FORWARD``
+mode.
+
+.. code-block:: c
+
+ enum rte_event_dma_adapter_mode mode;
+ struct rte_event_dev_info dev_info;
+ struct rte_event_port_conf conf;
+ uint8_t evdev_id;
+ uint8_t dma_id;
+ int ret;
+
+ ret = rte_event_dev_info_get(dma_id, &dev_info);
+
+ conf.new_event_threshold = dev_info.max_num_events;
+ conf.dequeue_depth = dev_info.max_event_port_dequeue_depth;
+ conf.enqueue_depth = dev_info.max_event_port_enqueue_depth;
+ mode = RTE_EVENT_DMA_ADAPTER_OP_FORWARD;
+ ret = rte_event_dma_adapter_create(dma_id, evdev_id, &conf, mode);
+
+
+``rte_event_dma_adapter_create_ext()`` function can be used by the application to have a finer
+control on eventdev port allocation and setup. The ``rte_event_dma_adapter_create_ext()``
+function is passed a callback function. The callback function is invoked if the adapter creates
+a service function and uses an event port for it. The callback is expected to fill the
+``struct rte_event_dma_adapter_conf`` structure passed to it.
+
+In the ``RTE_EVENT_DMA_ADAPTER_OP_FORWARD`` mode, if the event PMD and DMA PMD supports internal
+event port (``RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP_FWD``), events with DMA operations should
+be enqueued to the DMA adapter using ``rte_event_dma_adapter_enqueue()`` API. If not, the event port
+created by the adapter can be retrieved using ``rte_event_dma_adapter_event_port_get()`` API. An
+application can use this event port to link with an event queue, on which it enqueues events
+towards the DMA adapter using ``rte_event_enqueue_burst()``.
+
+.. code-block:: c
+
+ uint8_t dma_adpt_id, evdev_id, dma_dev_id, dma_ev_port_id, app_qid;
+ struct rte_event ev;
+ uint32_t cap;
+ int ret;
+
+ // Fill in event info and update event_ptr with rte_dma_op
+ memset(&ev, 0, sizeof(ev));
+ .
+ .
+ ev.event_ptr = op;
+
+ ret = rte_event_dma_adapter_caps_get(evdev_id, dma_dev_id, &cap);
+ if (cap & RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP_FWD) {
+ ret = rte_event_dma_adapter_enqueue(evdev_id, app_ev_port_id, ev, nb_events);
+ } else {
+ ret = rte_event_dma_adapter_event_port_get(dma_adpt_id, &dma_ev_port_id);
+ ret = rte_event_queue_setup(evdev_id, app_qid, NULL);
+ ret = rte_event_port_link(evdev_id, dma_ev_port_id, &app_qid, NULL, 1);
+ ev.queue_id = app_qid;
+ ret = rte_event_enqueue_burst(evdev_id, app_ev_port_id, ev, nb_events);
+ }
+
+
+Event device configuration for service based adapter
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+When ``rte_event_dma_adapter_create()`` is used for creating adapter instance,
+``rte_event_dev_config::nb_event_ports`` is automatically incremented, and event device is
+reconfigured with additional event port during service initialization. This event device
+reconfigure logic also increments the ``rte_event_dev_config::nb_single_link_event_port_queues``
+parameter if the adapter event port config is of type ``RTE_EVENT_PORT_CFG_SINGLE_LINK``.
+
+Applications using this mode of adapter creation need not configure the event device with
+``rte_event_dev_config::nb_event_ports`` and
+``rte_event_dev_config::nb_single_link_event_port_queues`` parameters required for DMA adapter when
+the adapter is created using the above-mentioned API.
+
+
+Querying adapter capabilities
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+The ``rte_event_dma_adapter_caps_get()`` function allows the application to query the adapter
+capabilities for an eventdev and dmadev combination. This API provides whether dmadev and eventdev
+are connected using internal HW port or not.
+
+.. code-block:: c
+
+ rte_event_dma_adapter_caps_get(dev_id, dma_dev_id, &cap);
+
+
+Adding vchan to the adapter instance
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+dmadev device id and vchan are configured using dmadev APIs. For more information
+see :doc:`here <dmadev>`.
+
+.. code-block:: c
+
+ struct rte_dma_vchan_conf vchan_conf;
+ struct rte_dma_conf dev_conf;
+ uint8_t dev_id = 0;
+ uint16_t vchan = 0;
+
+ rte_dma_configure(dev_id, &dev_conf);
+ rte_dma_vchan_setup(dev_id, vhcan, &vchan_conf);
+
+These dmadev id and vchan are added to the instance using the
+``rte_event_dma_adapter_vchan_add()`` API. The same is removed using
+``rte_event_dma_adapter_vchan_del()`` API. If hardware supports
+``RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_VCHAN_EV_BIND`` capability, event information must be passed to the add API.
+
+.. code-block:: c
+
+ uint32_t cap;
+ int ret;
+
+ ret = rte_event_dma_adapter_caps_get(evdev_id, dma_dev_id, &cap);
+ if (cap & RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_VCHAN_EV_BIND) {
+ struct rte_event event;
+
+ rte_event_dma_adapter_vchan_add(id, dma_dev_id, vchan, &conf);
+ } else
+ rte_event_dma_adapter_vchan_add(id, dma_dev_id, vchan, NULL);
+
+
+Configuring service function
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+If the adapter uses a service function, the application is required to assign a service core to
+the service function as show below.
+
+.. code-block:: c
+
+ uint32_t service_id;
+
+ if (rte_event_dma_adapter_service_id_get(dma_id, &service_id) == 0)
+ rte_service_map_lcore_set(service_id, CORE_ID);
+
+
+Set event response information
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+In the RTE_EVENT_DMA_ADAPTER_OP_FORWARD / RTE_EVENT_DMA_ADAPTER_OP_NEW mode, the application
+specifies the dmadev ID and vchan ID in ``struct rte_event_dma_adapter_op`` and the event
+information (response information) needed to enqueue an event after the DMA operation has
+completed. The response information is specified in ``struct rte_event`` and appended to the
+``struct rte_event_dma_adapter_op``.
+
+
+Start the adapter instance
+~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+The application calls ``rte_event_dma_adapter_start()`` to start the adapter. This function calls
+the start callbacks of the eventdev PMDs for hardware based eventdev-dmadev connections and
+``rte_service_run_state_set()`` to enable the service function if one exists.
+
+.. code-block:: c
+
+ rte_event_dma_adapter_start(id);
+
+.. Note::
+
+ The eventdev to which the event_dma_adapter is connected should be started before calling
+ rte_event_dma_adapter_start().
+
+
+Get adapter statistics
+~~~~~~~~~~~~~~~~~~~~~~
+
+The ``rte_event_dma_adapter_stats_get()`` function reports counters defined in struct
+``rte_event_dma_adapter_stats``. The received packet and enqueued event counts are a sum of the
+counts from the eventdev PMD callbacks if the callback is supported, and the counts maintained by
+the service function, if one exists.
+
+Set/Get adapter runtime configuration parameters
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+The runtime configuration parameters of adapter can be set/get using
+``rte_event_dma_adapter_runtime_params_set()`` and
+``rte_event_dma_adapter_runtime_params_get()`` respectively.
+The parameters that can be set/get are defined in
+``struct rte_event_dma_adapter_runtime_params``.
diff --git a/doc/guides/prog_guide/eventdev.rst b/doc/guides/prog_guide/eventdev.rst
index 2c83176846..ff55115d0d 100644
--- a/doc/guides/prog_guide/eventdev.rst
+++ b/doc/guides/prog_guide/eventdev.rst
@@ -333,7 +333,8 @@ eventdev.
.. Note::
EventDev needs to be started before starting the event producers such
- as event_eth_rx_adapter, event_timer_adapter and event_crypto_adapter.
+ as event_eth_rx_adapter, event_timer_adapter, event_crypto_adapter and
+ event_dma_adapter.
Ingress of New Events
~~~~~~~~~~~~~~~~~~~~~
@@ -445,8 +446,9 @@ using ``rte_event_dev_stop_flush_callback_register()`` function.
.. Note::
The event producers such as ``event_eth_rx_adapter``,
- ``event_timer_adapter`` and ``event_crypto_adapter``
- need to be stopped before stopping the event device.
+ ``event_timer_adapter``, ``event_crypto_adapter`` and
+ ``event_dma_adapter`` need to be stopped before stopping
+ the event device.
Summary
-------
diff --git a/doc/guides/prog_guide/img/event_dma_adapter_op_forward.svg b/doc/guides/prog_guide/img/event_dma_adapter_op_forward.svg
new file mode 100644
index 0000000000..b7fe1fecf2
--- /dev/null
+++ b/doc/guides/prog_guide/img/event_dma_adapter_op_forward.svg
@@ -0,0 +1,1086 @@
+<?xml version="1.0" encoding="UTF-8" standalone="no"?>
+<!-- SPDX-License-Identifier: BSD-3-Clause -->
+<!-- Copyright (c) 2023 Marvell. -->
+<!-- Created with Inkscape (http://www.inkscape.org/) -->
+
+<svg
+ width="720px"
+ height="486px"
+ id="svg13237"
+ version="1.1"
+ inkscape:version="1.2.1 (9c6d41e410, 2022-07-14)"
+ sodipodi:docname="event_dma_adapter_op_forward.svg"
+ xmlns:inkscape="http://www.inkscape.org/namespaces/inkscape"
+ xmlns:sodipodi="http://sodipodi.sourceforge.net/DTD/sodipodi-0.dtd"
+ xmlns="http://www.w3.org/2000/svg"
+ xmlns:svg="http://www.w3.org/2000/svg"
+ xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#"
+ xmlns:cc="http://creativecommons.org/ns#"
+ xmlns:dc="http://purl.org/dc/elements/1.1/"
+ xmlns:v="http://schemas.microsoft.com/visio/2003/SVGExtensions/">
+ <defs
+ id="defs13239">
+ <marker
+ inkscape:stockid="Arrow1Sstart"
+ orient="auto"
+ refY="0.0"
+ refX="0.0"
+ id="Arrow1Sstart"
+ style="overflow:visible">
+ <path
+ id="path8416"
+ d="M 0.0,0.0 L 5.0,-5.0 L -12.5,0.0 L 5.0,5.0 L 0.0,0.0 z "
+ style="fill-rule:evenodd;stroke:#000000;stroke-width:1.0pt"
+ transform="scale(0.2) translate(6,0)" />
+ </marker>
+ <marker
+ inkscape:stockid="Arrow1Send"
+ orient="auto"
+ refY="0.0"
+ refX="0.0"
+ id="Arrow1Send"
+ style="overflow:visible;">
+ <path
+ id="path8419"
+ d="M 0.0,0.0 L 5.0,-5.0 L -12.5,0.0 L 5.0,5.0 L 0.0,0.0 z "
+ style="fill-rule:evenodd;stroke:#000000;stroke-width:1.0pt;"
+ transform="scale(0.2) rotate(180) translate(6,0)" />
+ </marker>
+ <marker
+ inkscape:stockid="DiamondL"
+ orient="auto"
+ refY="0.0"
+ refX="0.0"
+ id="DiamondL"
+ style="overflow:visible">
+ <path
+ id="path8483"
+ d="M 0,-7.0710768 L -7.0710894,0 L 0,7.0710589 L 7.0710462,0 L 0,-7.0710768 z "
+ style="fill-rule:evenodd;stroke:#000000;stroke-width:1.0pt"
+ transform="scale(0.8)" />
+ </marker>
+ <marker
+ inkscape:stockid="DotL"
+ orient="auto"
+ refY="0.0"
+ refX="0.0"
+ id="DotL"
+ style="overflow:visible">
+ <path
+ id="path8465"
+ d="M -2.5,-1.0 C -2.5,1.7600000 -4.7400000,4.0 -7.5,4.0 C -10.260000,4.0 -12.5,1.7600000 -12.5,-1.0 C -12.5,-3.7600000 -10.260000,-6.0 -7.5,-6.0 C -4.7400000,-6.0 -2.5,-3.7600000 -2.5,-1.0 z "
+ style="fill-rule:evenodd;stroke:#000000;stroke-width:1.0pt"
+ transform="scale(0.8) translate(7.4, 1)" />
+ </marker>
+ <marker
+ inkscape:stockid="SquareL"
+ orient="auto"
+ refY="0.0"
+ refX="0.0"
+ id="SquareL"
+ style="overflow:visible">
+ <path
+ id="path8474"
+ d="M -5.0,-5.0 L -5.0,5.0 L 5.0,5.0 L 5.0,-5.0 L -5.0,-5.0 z "
+ style="fill-rule:evenodd;stroke:#000000;stroke-width:1.0pt"
+ transform="scale(0.8)" />
+ </marker>
+ <marker
+ inkscape:stockid="TriangleOutL"
+ orient="auto"
+ refY="0.0"
+ refX="0.0"
+ id="TriangleOutL"
+ style="overflow:visible">
+ <path
+ id="path8546"
+ d="M 5.77,0.0 L -2.88,5.0 L -2.88,-5.0 L 5.77,0.0 z "
+ style="fill-rule:evenodd;stroke:#000000;stroke-width:1.0pt"
+ transform="scale(0.8)" />
+ </marker>
+ <marker
+ inkscape:stockid="Arrow1Lstart"
+ orient="auto"
+ refY="0.0"
+ refX="0.0"
+ id="Arrow1Lstart"
+ style="overflow:visible">
+ <path
+ id="path8404"
+ d="M 0.0,0.0 L 5.0,-5.0 L -12.5,0.0 L 5.0,5.0 L 0.0,0.0 z "
+ style="fill-rule:evenodd;stroke:#000000;stroke-width:1.0pt"
+ transform="scale(0.8) translate(12.5,0)" />
+ </marker>
+ <marker
+ inkscape:stockid="Arrow1Mend"
+ orient="auto"
+ refY="0.0"
+ refX="0.0"
+ id="Arrow1Mend"
+ style="overflow:visible;">
+ <path
+ id="path8413"
+ d="M 0.0,0.0 L 5.0,-5.0 L -12.5,0.0 L 5.0,5.0 L 0.0,0.0 z "
+ style="fill-rule:evenodd;stroke:#000000;stroke-width:1.0pt;"
+ transform="scale(0.4) rotate(180) translate(10,0)" />
+ </marker>
+ <marker
+ inkscape:stockid="Arrow2Lend"
+ orient="auto"
+ refY="0.0"
+ refX="0.0"
+ id="Arrow2Lend"
+ style="overflow:visible;">
+ <path
+ id="path8425"
+ style="fill-rule:evenodd;stroke-width:0.62500000;stroke-linejoin:round;"
+ d="M 8.7185878,4.0337352 L -2.2072895,0.016013256 L 8.7185884,-4.0017078 C 6.9730900,-1.6296469 6.9831476,1.6157441 8.7185878,4.0337352 z "
+ transform="scale(1.1) rotate(180) translate(1,0)" />
+ </marker>
+ <marker
+ inkscape:stockid="Arrow1Lend"
+ orient="auto"
+ refY="0.0"
+ refX="0.0"
+ id="Arrow1Lend"
+ style="overflow:visible;">
+ <path
+ id="path8407"
+ d="M 0.0,0.0 L 5.0,-5.0 L -12.5,0.0 L 5.0,5.0 L 0.0,0.0 z "
+ style="fill-rule:evenodd;stroke:#000000;stroke-width:1.0pt;"
+ transform="scale(0.8) rotate(180) translate(12.5,0)" />
+ </marker>
+ <filter
+ id="filter_2"
+ color-interpolation-filters="sRGB">
+ <feGaussianBlur
+ stdDeviation="2"
+ id="feGaussianBlur15" />
+ </filter>
+ <filter
+ id="filter_2-3"
+ color-interpolation-filters="sRGB"
+ x="-0.086178862"
+ y="-0.086178862"
+ width="1.1723577"
+ height="1.1723577">
+ <feGaussianBlur
+ stdDeviation="2"
+ id="feGaussianBlur15-1" />
+ </filter>
+ <filter
+ id="filter_2-0"
+ color-interpolation-filters="sRGB">
+ <feGaussianBlur
+ stdDeviation="2"
+ id="feGaussianBlur15-7" />
+ </filter>
+ <filter
+ id="filter_2-0-8"
+ color-interpolation-filters="sRGB">
+ <feGaussianBlur
+ stdDeviation="2"
+ id="feGaussianBlur15-7-7" />
+ </filter>
+ <filter
+ id="filter_2-3-9"
+ color-interpolation-filters="sRGB">
+ <feGaussianBlur
+ stdDeviation="2"
+ id="feGaussianBlur15-1-6" />
+ </filter>
+ <filter
+ id="filter_2-3-6"
+ color-interpolation-filters="sRGB">
+ <feGaussianBlur
+ stdDeviation="2"
+ id="feGaussianBlur15-1-63" />
+ </filter>
+ <filter
+ id="filter_2-3-91"
+ color-interpolation-filters="sRGB"
+ x="-0.086178862"
+ y="-0.086178862"
+ width="1.1723577"
+ height="1.1723577">
+ <feGaussianBlur
+ stdDeviation="2"
+ id="feGaussianBlur15-1-3" />
+ </filter>
+ <marker
+ inkscape:stockid="Arrow1Lend"
+ orient="auto"
+ refY="0"
+ refX="0"
+ id="Arrow1Lend-5"
+ style="overflow:visible">
+ <path
+ inkscape:connector-curvature="0"
+ id="path8407-3"
+ d="M 0,0 5,-5 -12.5,0 5,5 0,0 z"
+ style="fill-rule:evenodd;stroke:#000000;stroke-width:1pt"
+ transform="matrix(-0.8,0,0,-0.8,-10,0)" />
+ </marker>
+ <marker
+ inkscape:stockid="Arrow1Lend"
+ orient="auto"
+ refY="0"
+ refX="0"
+ id="Arrow1Lend-6"
+ style="overflow:visible">
+ <path
+ inkscape:connector-curvature="0"
+ id="path8407-0"
+ d="M 0,0 5,-5 -12.5,0 5,5 0,0 z"
+ style="fill-rule:evenodd;stroke:#000000;stroke-width:1pt"
+ transform="matrix(-0.8,0,0,-0.8,-10,0)" />
+ </marker>
+ <marker
+ inkscape:stockid="Arrow1Lstart"
+ orient="auto"
+ refY="0"
+ refX="0"
+ id="Arrow1Lstart-7"
+ style="overflow:visible">
+ <path
+ inkscape:connector-curvature="0"
+ id="path8404-0"
+ d="M 0,0 5,-5 -12.5,0 5,5 0,0 z"
+ style="fill-rule:evenodd;stroke:#000000;stroke-width:1pt"
+ transform="matrix(0.8,0,0,0.8,10,0)" />
+ </marker>
+ <marker
+ inkscape:stockid="Arrow1Lend"
+ orient="auto"
+ refY="0"
+ refX="0"
+ id="Arrow1Lend-51"
+ style="overflow:visible">
+ <path
+ inkscape:connector-curvature="0"
+ id="path8407-1"
+ d="M 0,0 5,-5 -12.5,0 5,5 0,0 z"
+ style="fill-rule:evenodd;stroke:#000000;stroke-width:1pt"
+ transform="matrix(-0.8,0,0,-0.8,-10,0)" />
+ </marker>
+ <marker
+ inkscape:stockid="Arrow1Lend"
+ orient="auto"
+ refY="0"
+ refX="0"
+ id="Arrow1Lend-3"
+ style="overflow:visible">
+ <path
+ inkscape:connector-curvature="0"
+ id="path8407-6"
+ d="M 0,0 5,-5 -12.5,0 5,5 0,0 z"
+ style="fill-rule:evenodd;stroke:#000000;stroke-width:1pt"
+ transform="matrix(-0.8,0,0,-0.8,-10,0)" />
+ </marker>
+ <marker
+ inkscape:stockid="Arrow1Lend"
+ orient="auto"
+ refY="0"
+ refX="0"
+ id="Arrow1Lend-62"
+ style="overflow:visible">
+ <path
+ inkscape:connector-curvature="0"
+ id="path8407-9"
+ d="M 0,0 5,-5 -12.5,0 5,5 0,0 z"
+ style="fill-rule:evenodd;stroke:#000000;stroke-width:1pt"
+ transform="matrix(-0.8,0,0,-0.8,-10,0)" />
+ </marker>
+ <marker
+ inkscape:stockid="Arrow1Lend"
+ orient="auto"
+ refY="0"
+ refX="0"
+ id="Arrow1Lend-2"
+ style="overflow:visible">
+ <path
+ inkscape:connector-curvature="0"
+ id="path8407-7"
+ d="M 0,0 5,-5 -12.5,0 5,5 0,0 z"
+ style="fill-rule:evenodd;stroke:#000000;stroke-width:1pt"
+ transform="matrix(-0.8,0,0,-0.8,-10,0)" />
+ </marker>
+ <marker
+ inkscape:stockid="Arrow1Lstart"
+ orient="auto"
+ refY="0"
+ refX="0"
+ id="Arrow1Lstart-7-9"
+ style="overflow:visible">
+ <path
+ inkscape:connector-curvature="0"
+ id="path8404-0-3"
+ d="M 0,0 5,-5 -12.5,0 5,5 0,0 z"
+ style="fill-rule:evenodd;stroke:#000000;stroke-width:1pt"
+ transform="matrix(0.8,0,0,0.8,10,0)" />
+ </marker>
+ <filter
+ id="filter_2-3-6-1"
+ color-interpolation-filters="sRGB">
+ <feGaussianBlur
+ stdDeviation="2"
+ id="feGaussianBlur15-1-63-8" />
+ </filter>
+ <filter
+ id="filter_2-3-92"
+ color-interpolation-filters="sRGB"
+ x="-0.086178862"
+ y="-0.086178862"
+ width="1.1723577"
+ height="1.1723577">
+ <feGaussianBlur
+ stdDeviation="2"
+ id="feGaussianBlur15-1-2" />
+ </filter>
+ <filter
+ id="filter_2-3-94"
+ color-interpolation-filters="sRGB"
+ x="-0.086178862"
+ y="-0.086178862"
+ width="1.1723577"
+ height="1.1723577">
+ <feGaussianBlur
+ stdDeviation="2"
+ id="feGaussianBlur15-1-7" />
+ </filter>
+ <marker
+ inkscape:stockid="Arrow1Lstart"
+ orient="auto"
+ refY="0"
+ refX="0"
+ id="Arrow1Lstart-7-6"
+ style="overflow:visible">
+ <path
+ inkscape:connector-curvature="0"
+ id="path8404-0-1"
+ d="M 0,0 5,-5 -12.5,0 5,5 0,0 z"
+ style="fill-rule:evenodd;stroke:#000000;stroke-width:1pt"
+ transform="matrix(0.8,0,0,0.8,10,0)" />
+ </marker>
+ <marker
+ inkscape:stockid="Arrow1Lend"
+ orient="auto"
+ refY="0"
+ refX="0"
+ id="Arrow1Lend-55"
+ style="overflow:visible">
+ <path
+ inkscape:connector-curvature="0"
+ id="path8407-4"
+ d="M 0,0 5,-5 -12.5,0 5,5 0,0 z"
+ style="fill-rule:evenodd;stroke:#000000;stroke-width:1pt"
+ transform="matrix(-0.8,0,0,-0.8,-10,0)" />
+ </marker>
+ </defs>
+ <sodipodi:namedview
+ id="base"
+ pagecolor="#ffffff"
+ bordercolor="#666666"
+ borderopacity="1.0"
+ inkscape:pageopacity="0.0"
+ inkscape:pageshadow="2"
+ inkscape:zoom="1"
+ inkscape:cx="233.5"
+ inkscape:cy="288"
+ inkscape:document-units="px"
+ inkscape:current-layer="layer1"
+ showgrid="false"
+ inkscape:window-width="1920"
+ inkscape:window-height="1017"
+ inkscape:window-x="-8"
+ inkscape:window-y="-8"
+ inkscape:window-maximized="1"
+ inkscape:snap-nodes="false"
+ inkscape:showpageshadow="2"
+ inkscape:pagecheckerboard="0"
+ inkscape:deskcolor="#d1d1d1">
+ <inkscape:grid
+ type="xygrid"
+ id="grid13454" />
+ </sodipodi:namedview>
+ <metadata
+ id="metadata13242">
+ <rdf:RDF>
+ <cc:Work
+ rdf:about="">
+ <dc:format>image/svg+xml</dc:format>
+ <dc:type
+ rdf:resource="http://purl.org/dc/dcmitype/StillImage" />
+ <dc:title />
+ </cc:Work>
+ </rdf:RDF>
+ </metadata>
+ <g
+ id="layer1"
+ inkscape:label="Layer 1"
+ inkscape:groupmode="layer">
+ <g
+ style="font-size:12px;fill:none;stroke-linecap:square;stroke-miterlimit:3;overflow:visible"
+ id="shape1-1-2-4"
+ v:mID="1"
+ v:groupContext="shape"
+ transform="matrix(2.1604167,0,0,1.5671361,88.874699,-812.39909)">
+ <title
+ id="title22-7-5">Square</title>
+ <desc
+ id="desc24-7-8">Atomic Queue #1</desc>
+ <v:userDefs>
+ <v:ud
+ v:nameU="visVersion"
+ v:val="VT0(15):26" />
+ </v:userDefs>
+ <v:textBlock
+ v:margins="rect(4,4,4,4)" />
+ <v:textRect
+ cx="30.75"
+ cy="581.25"
+ width="61.5"
+ height="61.5" />
+ <g
+ id="shadow1-2-9-5"
+ v:groupContext="shadow"
+ v:shadowOffsetX="0.345598"
+ v:shadowOffsetY="-1.97279"
+ v:shadowType="1"
+ transform="translate(0.345598,1.97279)"
+ class="st1"
+ style="visibility:visible">
+ <rect
+ x="0"
+ y="550.5"
+ width="61.5"
+ height="61.5"
+ class="st2"
+ id="rect27-8-7"
+ style="fill:#5b9bd5;fill-opacity:0.22000002;stroke:#5b9bd5;stroke-opacity:0.22000002;filter:url(#filter_2-3-91)" />
+ </g>
+ <g
+ id="g13515-33">
+ <g
+ id="g13534-8">
+ <rect
+ x="0"
+ y="550.5"
+ width="61.5"
+ height="61.5"
+ class="st3"
+ id="rect29-1-95"
+ style="fill:#5b9bd5;stroke:#c7c8c8;stroke-width:0.25" />
+ </g>
+ </g>
+ </g>
+ <path
+ style="fill:none;stroke:#000000;stroke-width:0.712265;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:4;stroke-dasharray:none;stroke-opacity:1;marker-start:url(#Arrow1Lstart-7);marker-end:none"
+ d="M 312.28671,240.74335 H 227.99897"
+ id="path17209"
+ inkscape:connector-type="orthogonal"
+ inkscape:connector-curvature="0" />
+ <path
+ style="fill:none;stroke:#000000;stroke-width:0.718986px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1;marker-mid:none;marker-end:url(#Arrow1Lend)"
+ d="m 221.6484,77.57125 h 94.28101"
+ id="path17209-8"
+ inkscape:connector-type="orthogonal"
+ inkscape:connector-curvature="0" />
+ <g
+ style="font-size:12px;fill:none;stroke-linecap:square;stroke-miterlimit:3;overflow:visible"
+ id="shape1-1-2"
+ v:mID="1"
+ v:groupContext="shape"
+ transform="matrix(2.1604167,0,0,1.5671361,314.24227,-811.89589)">
+ <title
+ id="title22-7">Square</title>
+ <desc
+ id="desc24-7">Atomic Queue #1</desc>
+ <v:userDefs>
+ <v:ud
+ v:nameU="visVersion"
+ v:val="VT0(15):26" />
+ </v:userDefs>
+ <v:textBlock
+ v:margins="rect(4,4,4,4)" />
+ <v:textRect
+ cx="30.75"
+ cy="581.25"
+ width="61.5"
+ height="61.5" />
+ <g
+ id="shadow1-2-9"
+ v:groupContext="shadow"
+ v:shadowOffsetX="0.345598"
+ v:shadowOffsetY="-1.97279"
+ v:shadowType="1"
+ transform="translate(0.345598,1.97279)"
+ class="st1"
+ style="visibility:visible">
+ <rect
+ x="0"
+ y="550.5"
+ width="61.5"
+ height="61.5"
+ class="st2"
+ id="rect27-8"
+ style="fill:#5b9bd5;fill-opacity:0.22000002;stroke:#5b9bd5;stroke-opacity:0.22000002;filter:url(#filter_2-3)" />
+ </g>
+ <g
+ id="g13515">
+ <g
+ id="g13534">
+ <rect
+ x="0"
+ y="550.5"
+ width="61.5"
+ height="61.5"
+ class="st3"
+ id="rect29-1"
+ style="fill:#5b9bd5;stroke:#c7c8c8;stroke-width:0.25" />
+ </g>
+ </g>
+ </g>
+ <path
+ style="fill:none;stroke:#000000;stroke-width:0.724714;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:4;stroke-dasharray:none;stroke-opacity:1;marker-start:url(#Arrow1Lstart);marker-end:none"
+ d="M 89.025329,74.39932 H 24.750043"
+ id="path17209-3"
+ inkscape:connector-type="orthogonal"
+ inkscape:connector-curvature="0" />
+ <path
+ transform="matrix(0.73232502,0,0,0.75477602,-4.325033,28.642983)"
+ sodipodi:type="arc"
+ style="fill:#539de6;fill-opacity:1;stroke:#0000ea;stroke-width:1;stroke-linecap:square;stroke-miterlimit:4;stroke-opacity:1;stroke-dasharray:none"
+ id="path29161-3"
+ sodipodi:cx="371"
+ sodipodi:cy="64.5"
+ sodipodi:rx="17"
+ sodipodi:ry="15.5"
+ d="m 388,64.5 a 17,15.5 0 1 1 -34,0 17,15.5 0 1 1 34,0 z" />
+ <path
+ transform="matrix(0.73232502,0,0,0.75477602,-1.93108,192.80833)"
+ sodipodi:type="arc"
+ style="fill:#539de6;fill-opacity:1;stroke:#0000ea;stroke-width:1;stroke-linecap:square;stroke-miterlimit:4;stroke-opacity:1;stroke-dasharray:none"
+ id="path29161-1"
+ sodipodi:cx="371"
+ sodipodi:cy="64.5"
+ sodipodi:rx="17"
+ sodipodi:ry="15.5"
+ d="m 388,64.5 a 17,15.5 0 1 1 -34,0 17,15.5 0 1 1 34,0 z" />
+ <path
+ style="fill:none;stroke:#000000;stroke-width:0.751412;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:4;stroke-dasharray:none;stroke-opacity:1;marker-start:url(#Arrow1Lstart-7);marker-end:none"
+ d="M 18.763392,120.7432 H 87.758545"
+ id="path17209-3-0"
+ inkscape:connector-type="orthogonal"
+ inkscape:connector-curvature="0" />
+ <path
+ sodipodi:type="arc"
+ style="fill:#539de6;fill-opacity:1;stroke:#0000ea;stroke-width:1;stroke-linecap:square;stroke-miterlimit:4;stroke-opacity:1;stroke-dasharray:none"
+ id="path29161"
+ sodipodi:cx="371"
+ sodipodi:cy="64.5"
+ sodipodi:rx="17"
+ sodipodi:ry="15.5"
+ d="m 388,64.5 a 17,15.5 0 1 1 -34,0 17,15.5 0 1 1 34,0 z"
+ transform="matrix(0.73232502,0,0,0.75477602,-218.16394,72.68276)" />
+ <path
+ sodipodi:type="arc"
+ style="fill:#539de6;fill-opacity:1;stroke:#0000ea;stroke-width:1;stroke-linecap:square;stroke-miterlimit:4;stroke-opacity:1;stroke-dasharray:none"
+ id="path29161-2"
+ sodipodi:cx="371"
+ sodipodi:cy="64.5"
+ sodipodi:rx="17"
+ sodipodi:ry="15.5"
+ d="m 388,64.5 a 17,15.5 0 1 1 -34,0 17,15.5 0 1 1 34,0 z"
+ transform="matrix(0.73232502,0,0,0.75477602,-217.40136,26.716271)" />
+ <g
+ id="g29167-4"
+ transform="matrix(0.73232502,0,0,0.75477602,-217.31662,28.007562)">
+ <text
+ id="text29163-9"
+ y="70"
+ x="365"
+ style="font-style:normal;font-weight:normal;line-height:0%;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none"
+ xml:space="preserve"><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="70"
+ x="365"
+ id="tspan29165-9"
+ sodipodi:role="line">1</tspan></text>
+ </g>
+ <g
+ id="g29167-9"
+ transform="matrix(0.73232502,0,0,0.75477602,-4.9726112,28.689051)">
+ <text
+ id="text29163-3"
+ y="70"
+ x="365"
+ style="font-style:normal;font-weight:normal;line-height:0%;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none"
+ xml:space="preserve"><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="70"
+ x="365"
+ id="tspan29165-3"
+ sodipodi:role="line">2</tspan></text>
+ </g>
+ <path
+ style="fill:none;stroke:#000000;stroke-width:0.678033px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1;marker-start:url(#Arrow1Lstart-7);marker-end:none"
+ d="M 181,214.66098 V 145.33902"
+ id="path17211-7-1-6"
+ inkscape:connector-type="orthogonal"
+ inkscape:connector-curvature="0" />
+ <g
+ id="g29167"
+ transform="matrix(0.73232502,0,0,0.75477602,-218.07919,73.10621)">
+ <text
+ id="text29163"
+ y="70"
+ x="365"
+ style="font-style:normal;font-weight:normal;line-height:0%;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none"
+ xml:space="preserve"><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="70"
+ x="365"
+ id="tspan29165"
+ sodipodi:role="line">8</tspan></text>
+ </g>
+ <path
+ style="fill:none;stroke:#000000;stroke-width:0.678033px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1;marker-start:url(#Arrow1Lstart-7);marker-end:none"
+ d="m 131,145.8531 v 69.32197"
+ id="path17211-7-1"
+ inkscape:connector-type="orthogonal"
+ inkscape:connector-curvature="0" />
+ <path
+ transform="matrix(0.73232502,0,0,0.75477602,-140.37076,129.97088)"
+ sodipodi:type="arc"
+ style="fill:#539de6;fill-opacity:1;stroke:#0000ea;stroke-width:1;stroke-linecap:square;stroke-miterlimit:4;stroke-opacity:1;stroke-dasharray:none"
+ id="path29161-8"
+ sodipodi:cx="371"
+ sodipodi:cy="64.5"
+ sodipodi:rx="17"
+ sodipodi:ry="15.5"
+ d="m 388,64.5 a 17,15.5 0 1 1 -34,0 17,15.5 0 1 1 34,0 z" />
+ <g
+ id="g29167-2"
+ transform="matrix(0.73232502,0,0,0.75477602,-140.28602,131.01695)">
+ <text
+ id="text29163-92"
+ y="70"
+ x="365"
+ style="font-style:normal;font-weight:normal;line-height:0%;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none"
+ xml:space="preserve"><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="70"
+ x="365"
+ id="tspan29165-8"
+ sodipodi:role="line">7</tspan></text>
+ </g>
+ <path
+ style="fill:none;stroke:#000000;stroke-width:0.718986px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1;marker-mid:none;marker-end:url(#Arrow1Lend)"
+ d="m 317.1405,116 h -94.281"
+ id="path17209-8-0"
+ inkscape:connector-type="orthogonal"
+ inkscape:connector-curvature="0" />
+ <path
+ transform="matrix(0.73232502,0,0,0.75477602,-3.4914,66.68745)"
+ sodipodi:type="arc"
+ style="fill:#539de6;fill-opacity:1;stroke:#0000ea;stroke-width:1;stroke-linecap:square;stroke-miterlimit:4;stroke-opacity:1;stroke-dasharray:none"
+ id="path29161-6"
+ sodipodi:cx="371"
+ sodipodi:cy="64.5"
+ sodipodi:rx="17"
+ sodipodi:ry="15.5"
+ d="m 388,64.5 a 17,15.5 0 1 1 -34,0 17,15.5 0 1 1 34,0 z" />
+ <g
+ id="g29167-46"
+ transform="matrix(0.73232502,0,0,0.75477602,-4.40666,67.48829)">
+ <text
+ id="text29163-1"
+ y="70"
+ x="365"
+ style="font-style:normal;font-weight:normal;line-height:0%;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none"
+ xml:space="preserve"><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="70"
+ x="365"
+ id="tspan29165-5"
+ sodipodi:role="line">3</tspan></text>
+ </g>
+ <path
+ transform="matrix(0.73232502,0,0,0.75477602,-90.692582,130.31695)"
+ sodipodi:type="arc"
+ style="fill:#539de6;fill-opacity:1;stroke:#0000ea;stroke-width:1;stroke-linecap:square;stroke-miterlimit:4;stroke-opacity:1;stroke-dasharray:none"
+ id="path29161-8-6"
+ sodipodi:cx="371"
+ sodipodi:cy="64.5"
+ sodipodi:rx="17"
+ sodipodi:ry="15.5"
+ d="m 388,64.5 a 17,15.5 0 1 1 -34,0 17,15.5 0 1 1 34,0 z" />
+ <g
+ id="g29167-6"
+ transform="matrix(0.73232502,0,0,0.75477602,-90.84634,131.60918)">
+ <text
+ id="text29163-17"
+ y="70"
+ x="365"
+ style="font-style:normal;font-weight:normal;line-height:0%;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none"
+ xml:space="preserve"><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="70"
+ x="365"
+ id="tspan29165-2"
+ sodipodi:role="line">4</tspan></text>
+ </g>
+ <g
+ id="g29167-2-0"
+ transform="matrix(0.73232502,0,0,0.75477602,-2.424397,194.0216)">
+ <text
+ id="text29163-92-6"
+ y="70"
+ x="365"
+ style="font-style:normal;font-weight:normal;line-height:0%;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none"
+ xml:space="preserve"><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="70"
+ x="365"
+ id="tspan29165-8-2"
+ sodipodi:role="line">5</tspan></text>
+ </g>
+ <g
+ style="font-size:12px;fill:none;stroke-linecap:square;stroke-miterlimit:3;overflow:visible"
+ id="shape1-1-2-8"
+ v:mID="1"
+ v:groupContext="shape"
+ transform="matrix(2.1604167,0,0,1.5671361,93.82055,-648.98949)">
+ <title
+ id="title22-7-97">Square</title>
+ <desc
+ id="desc24-7-3">Atomic Queue #1</desc>
+ <v:userDefs>
+ <v:ud
+ v:nameU="visVersion"
+ v:val="VT0(15):26" />
+ </v:userDefs>
+ <v:textBlock
+ v:margins="rect(4,4,4,4)" />
+ <v:textRect
+ cx="30.75"
+ cy="581.25"
+ width="61.5"
+ height="61.5" />
+ <g
+ id="shadow1-2-9-6"
+ v:groupContext="shadow"
+ v:shadowOffsetX="0.345598"
+ v:shadowOffsetY="-1.97279"
+ v:shadowType="1"
+ transform="translate(0.345598,1.97279)"
+ class="st1"
+ style="visibility:visible">
+ <rect
+ x="0"
+ y="550.5"
+ width="61.5"
+ height="61.5"
+ class="st2"
+ id="rect27-8-12"
+ style="fill:#5b9bd5;fill-opacity:0.22000002;stroke:#5b9bd5;stroke-opacity:0.22000002;filter:url(#filter_2-3-92)" />
+ </g>
+ <g
+ id="g13515-9">
+ <g
+ id="g13534-3">
+ <rect
+ x="0"
+ y="550.5"
+ width="61.5"
+ height="61.5"
+ class="st3"
+ id="rect29-1-1"
+ style="fill:#5b9bd5;stroke:#c7c8c8;stroke-width:0.25" />
+ </g>
+ </g>
+ </g>
+ <g
+ style="font-size:12px;fill:none;stroke-linecap:square;stroke-miterlimit:3;overflow:visible"
+ id="shape1-1-2-84"
+ v:mID="1"
+ v:groupContext="shape"
+ transform="matrix(2.1604167,0,0,1.5671361,314.82055,-648.98949)">
+ <title
+ id="title22-7-50">Square</title>
+ <desc
+ id="desc24-7-36">Atomic Queue #1</desc>
+ <v:userDefs>
+ <v:ud
+ v:nameU="visVersion"
+ v:val="VT0(15):26" />
+ </v:userDefs>
+ <v:textBlock
+ v:margins="rect(4,4,4,4)" />
+ <v:textRect
+ cx="30.75"
+ cy="581.25"
+ width="61.5"
+ height="61.5" />
+ <g
+ id="shadow1-2-9-1"
+ v:groupContext="shadow"
+ v:shadowOffsetX="0.345598"
+ v:shadowOffsetY="-1.97279"
+ v:shadowType="1"
+ transform="translate(0.345598,1.97279)"
+ class="st1"
+ style="visibility:visible">
+ <rect
+ x="0"
+ y="550.5"
+ width="61.5"
+ height="61.5"
+ class="st2"
+ id="rect27-8-0"
+ style="fill:#5b9bd5;fill-opacity:0.22000002;stroke:#5b9bd5;stroke-opacity:0.22000002;filter:url(#filter_2-3-94)" />
+ </g>
+ <g
+ id="g13515-6">
+ <g
+ id="g13534-32">
+ <rect
+ x="0"
+ y="550.5"
+ width="61.5"
+ height="61.5"
+ class="st3"
+ id="rect29-1-0"
+ style="fill:#5b9bd5;stroke:#c7c8c8;stroke-width:0.25" />
+ </g>
+ </g>
+ </g>
+ <path
+ style="fill:none;stroke:#000000;stroke-width:0.712265;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:4;stroke-dasharray:none;stroke-opacity:1;marker-start:none;marker-end:url(#Arrow1Lend)"
+ d="M 313.14387,285 H 228.85613"
+ id="path17209-7"
+ inkscape:connector-type="orthogonal"
+ inkscape:connector-curvature="0" />
+ <path
+ transform="matrix(0.73232502,0,0,0.75477602,-2.692582,236.31695)"
+ sodipodi:type="arc"
+ style="fill:#539de6;fill-opacity:1;stroke:#0000ea;stroke-width:1;stroke-linecap:square;stroke-miterlimit:4;stroke-opacity:1;stroke-dasharray:none"
+ id="path29161-1-6"
+ sodipodi:cx="371"
+ sodipodi:cy="64.5"
+ sodipodi:rx="17"
+ sodipodi:ry="15.5"
+ d="m 388,64.5 a 17,15.5 0 1 1 -34,0 17,15.5 0 1 1 34,0 z" />
+ <g
+ id="g29167-2-0-5"
+ transform="matrix(0.73232502,0,0,0.75477602,-2.424397,237.0216)">
+ <text
+ id="text29163-92-6-6"
+ y="70"
+ x="365"
+ style="font-style:normal;font-weight:normal;line-height:0%;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none"
+ xml:space="preserve"><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="70"
+ x="365"
+ id="tspan29165-8-2-9"
+ sodipodi:role="line">6</tspan></text>
+ </g>
+ <g
+ id="g29167-4-3"
+ transform="matrix(0.73232502,0,0,0.75477602,-154.60784,51.117791)">
+ <text
+ id="text29163-9-6"
+ y="70"
+ x="365"
+ style="font-style:normal;font-weight:normal;line-height:0%;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none"
+ xml:space="preserve"><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:24.2109px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="70"
+ x="365"
+ id="tspan29165-9-7"
+ sodipodi:role="line">Eventdev</tspan></text>
+ </g>
+ <g
+ id="g29167-4-3-5"
+ transform="matrix(0.73232502,0,0,0.75477602,-144.65044,201.97821)">
+ <text
+ id="text29163-9-6-3"
+ y="70"
+ x="412.93716"
+ style="font-style:normal;font-weight:normal;line-height:0%;font-family:sans-serif;text-align:center;letter-spacing:0px;word-spacing:0px;text-anchor:middle;fill:#000000;fill-opacity:1;stroke:none"
+ xml:space="preserve"><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:24.2109px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:center;writing-mode:lr-tb;text-anchor:middle"
+ y="70"
+ x="412.93716"
+ id="tspan29165-9-7-5"
+ sodipodi:role="line">DMA</tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:24.2109px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:center;writing-mode:lr-tb;text-anchor:middle"
+ y="100.26363"
+ x="412.93716"
+ sodipodi:role="line"
+ id="tspan3201">Adapter</tspan></text>
+ </g>
+ <g
+ id="g29167-4-3-5-6"
+ transform="matrix(0.73232502,0,0,0.75477602,79.53518,46.62529)">
+ <text
+ id="text29163-9-6-3-2"
+ y="48.801659"
+ x="412.93716"
+ style="font-style:normal;font-weight:normal;line-height:0%;font-family:sans-serif;text-align:center;letter-spacing:0px;word-spacing:0px;text-anchor:middle;fill:#000000;fill-opacity:1;stroke:none"
+ xml:space="preserve"><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:24.2109px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:center;writing-mode:lr-tb;text-anchor:middle"
+ y="48.801659"
+ x="412.93716"
+ sodipodi:role="line"
+ id="tspan3155">Application</tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:24.2109px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:center;writing-mode:lr-tb;text-anchor:middle"
+ y="98.801659"
+ x="412.93716"
+ sodipodi:role="line"
+ id="tspan3201-1">in ordered</tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:24.2109px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:center;writing-mode:lr-tb;text-anchor:middle"
+ y="148.80167"
+ x="412.93716"
+ sodipodi:role="line"
+ id="tspan3161">stage</tspan></text>
+ </g>
+ <g
+ id="g29167-4-3-5-2"
+ transform="matrix(0.73232502,0,0,0.75477602,77.535182,213.62529)">
+ <text
+ id="text29163-9-6-3-7"
+ y="70"
+ x="412.93716"
+ style="font-style:normal;font-weight:normal;line-height:0%;font-family:sans-serif;text-align:center;letter-spacing:0px;word-spacing:0px;text-anchor:middle;fill:#000000;fill-opacity:1;stroke:none"
+ xml:space="preserve"><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:24.2109px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:center;writing-mode:lr-tb;text-anchor:middle"
+ y="70"
+ x="412.93716"
+ sodipodi:role="line"
+ id="tspan3201-9">DMA Device</tspan></text>
+ </g>
+ <g
+ id="g29167-4-3-5-3"
+ transform="matrix(0.73232502,0,0,0.75477602,188.53518,-3.37471)">
+ <text
+ id="text29163-9-6-3-6"
+ y="70"
+ x="375.65271"
+ style="font-style:normal;font-weight:normal;line-height:0%;font-family:sans-serif;text-align:center;letter-spacing:0px;word-spacing:0px;text-anchor:middle;fill:#000000;fill-opacity:1;stroke:none"
+ xml:space="preserve"><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18.8307px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="70"
+ x="375.65271"
+ sodipodi:role="line"
+ id="tspan3201-6">1. Events from the previous stage.</tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18.8307px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="93.538376"
+ x="375.65271"
+ sodipodi:role="line"
+ id="tspan3260"> </tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18.8307px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="117.07675"
+ x="375.65271"
+ sodipodi:role="line"
+ id="tspan3262">2. Application in ordered stage</tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18.8307px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="140.61513"
+ x="375.65271"
+ sodipodi:role="line"
+ id="tspan3288"> dequeues events from eventdev.</tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18.8307px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="164.1535"
+ x="375.65271"
+ sodipodi:role="line"
+ id="tspan3264"> </tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18.8307px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="187.69188"
+ x="375.65271"
+ sodipodi:role="line"
+ id="tspan3266">3. Application enqueues DMA</tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18.8307px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="211.23026"
+ x="375.65271"
+ sodipodi:role="line"
+ id="tspan3290"> operations as events to eventdev.</tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18.8307px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="234.76863"
+ x="375.65271"
+ sodipodi:role="line"
+ id="tspan3268"> </tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18.8307px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="258.30701"
+ x="375.65271"
+ sodipodi:role="line"
+ id="tspan3270">4. DMA adapter dequeues event</tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18.8307px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="281.84537"
+ x="375.65271"
+ sodipodi:role="line"
+ id="tspan3292"> from eventdev.</tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18.8307px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="305.38376"
+ x="375.65271"
+ sodipodi:role="line"
+ id="tspan3272"> </tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18.8307px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="328.92212"
+ x="375.65271"
+ sodipodi:role="line"
+ id="tspan3274">5. DMA adapter submits DMA</tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18.8307px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="352.46051"
+ x="375.65271"
+ sodipodi:role="line"
+ id="tspan3294"> operations to DMA Device (Atomic</tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18.8307px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="375.99887"
+ x="375.65271"
+ sodipodi:role="line"
+ id="tspan3296"> stage)</tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18.8307px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="399.53726"
+ x="375.65271"
+ sodipodi:role="line"
+ id="tspan3276"> </tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18.8307px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="423.07562"
+ x="375.65271"
+ sodipodi:role="line"
+ id="tspan3278">6. DMA adapter dequeues DMA</tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18.8307px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="446.61401"
+ x="375.65271"
+ sodipodi:role="line"
+ id="tspan3298"> completions from DMA Device</tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18.8307px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="470.15237"
+ x="375.65271"
+ sodipodi:role="line"
+ id="tspan3280"> </tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18.8307px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="493.69073"
+ x="375.65271"
+ sodipodi:role="line"
+ id="tspan3282">7. DMA adapter enqueues events</tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18.8307px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="517.22913"
+ x="375.65271"
+ sodipodi:role="line"
+ id="tspan3300"> to the eventdev</tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18.8307px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="540.76752"
+ x="375.65271"
+ sodipodi:role="line"
+ id="tspan3284"> </tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18.8307px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="564.30585"
+ x="375.65271"
+ sodipodi:role="line"
+ id="tspan3286">8. Events to the next stage</tspan></text>
+ </g>
+ </g>
+</svg>
diff --git a/doc/guides/prog_guide/img/event_dma_adapter_op_new.svg b/doc/guides/prog_guide/img/event_dma_adapter_op_new.svg
new file mode 100644
index 0000000000..e9e8bb2b98
--- /dev/null
+++ b/doc/guides/prog_guide/img/event_dma_adapter_op_new.svg
@@ -0,0 +1,1079 @@
+<?xml version="1.0" encoding="UTF-8" standalone="no"?>
+<!-- SPDX-License-Identifier: BSD-3-Clause -->
+<!-- Copyright (c) 2023 Marvell. -->
+<!-- Created with Inkscape (http://www.inkscape.org/) -->
+
+<svg
+ width="720px"
+ height="486px"
+ id="svg13237"
+ version="1.1"
+ inkscape:version="1.2.1 (9c6d41e410, 2022-07-14)"
+ sodipodi:docname="event_dma_adapter_op_new.svg"
+ xmlns:inkscape="http://www.inkscape.org/namespaces/inkscape"
+ xmlns:sodipodi="http://sodipodi.sourceforge.net/DTD/sodipodi-0.dtd"
+ xmlns="http://www.w3.org/2000/svg"
+ xmlns:svg="http://www.w3.org/2000/svg"
+ xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#"
+ xmlns:cc="http://creativecommons.org/ns#"
+ xmlns:dc="http://purl.org/dc/elements/1.1/"
+ xmlns:v="http://schemas.microsoft.com/visio/2003/SVGExtensions/">
+ <defs
+ id="defs13239">
+ <marker
+ inkscape:stockid="Arrow1Sstart"
+ orient="auto"
+ refY="0.0"
+ refX="0.0"
+ id="Arrow1Sstart"
+ style="overflow:visible">
+ <path
+ id="path8416"
+ d="M 0.0,0.0 L 5.0,-5.0 L -12.5,0.0 L 5.0,5.0 L 0.0,0.0 z "
+ style="fill-rule:evenodd;stroke:#000000;stroke-width:1.0pt"
+ transform="scale(0.2) translate(6,0)" />
+ </marker>
+ <marker
+ inkscape:stockid="Arrow1Send"
+ orient="auto"
+ refY="0.0"
+ refX="0.0"
+ id="Arrow1Send"
+ style="overflow:visible;">
+ <path
+ id="path8419"
+ d="M 0.0,0.0 L 5.0,-5.0 L -12.5,0.0 L 5.0,5.0 L 0.0,0.0 z "
+ style="fill-rule:evenodd;stroke:#000000;stroke-width:1.0pt;"
+ transform="scale(0.2) rotate(180) translate(6,0)" />
+ </marker>
+ <marker
+ inkscape:stockid="DiamondL"
+ orient="auto"
+ refY="0.0"
+ refX="0.0"
+ id="DiamondL"
+ style="overflow:visible">
+ <path
+ id="path8483"
+ d="M 0,-7.0710768 L -7.0710894,0 L 0,7.0710589 L 7.0710462,0 L 0,-7.0710768 z "
+ style="fill-rule:evenodd;stroke:#000000;stroke-width:1.0pt"
+ transform="scale(0.8)" />
+ </marker>
+ <marker
+ inkscape:stockid="DotL"
+ orient="auto"
+ refY="0.0"
+ refX="0.0"
+ id="DotL"
+ style="overflow:visible">
+ <path
+ id="path8465"
+ d="M -2.5,-1.0 C -2.5,1.7600000 -4.7400000,4.0 -7.5,4.0 C -10.260000,4.0 -12.5,1.7600000 -12.5,-1.0 C -12.5,-3.7600000 -10.260000,-6.0 -7.5,-6.0 C -4.7400000,-6.0 -2.5,-3.7600000 -2.5,-1.0 z "
+ style="fill-rule:evenodd;stroke:#000000;stroke-width:1.0pt"
+ transform="scale(0.8) translate(7.4, 1)" />
+ </marker>
+ <marker
+ inkscape:stockid="SquareL"
+ orient="auto"
+ refY="0.0"
+ refX="0.0"
+ id="SquareL"
+ style="overflow:visible">
+ <path
+ id="path8474"
+ d="M -5.0,-5.0 L -5.0,5.0 L 5.0,5.0 L 5.0,-5.0 L -5.0,-5.0 z "
+ style="fill-rule:evenodd;stroke:#000000;stroke-width:1.0pt"
+ transform="scale(0.8)" />
+ </marker>
+ <marker
+ inkscape:stockid="TriangleOutL"
+ orient="auto"
+ refY="0.0"
+ refX="0.0"
+ id="TriangleOutL"
+ style="overflow:visible">
+ <path
+ id="path8546"
+ d="M 5.77,0.0 L -2.88,5.0 L -2.88,-5.0 L 5.77,0.0 z "
+ style="fill-rule:evenodd;stroke:#000000;stroke-width:1.0pt"
+ transform="scale(0.8)" />
+ </marker>
+ <marker
+ inkscape:stockid="Arrow1Lstart"
+ orient="auto"
+ refY="0.0"
+ refX="0.0"
+ id="Arrow1Lstart"
+ style="overflow:visible">
+ <path
+ id="path8404"
+ d="M 0.0,0.0 L 5.0,-5.0 L -12.5,0.0 L 5.0,5.0 L 0.0,0.0 z "
+ style="fill-rule:evenodd;stroke:#000000;stroke-width:1.0pt"
+ transform="scale(0.8) translate(12.5,0)" />
+ </marker>
+ <marker
+ inkscape:stockid="Arrow1Mend"
+ orient="auto"
+ refY="0.0"
+ refX="0.0"
+ id="Arrow1Mend"
+ style="overflow:visible;">
+ <path
+ id="path8413"
+ d="M 0.0,0.0 L 5.0,-5.0 L -12.5,0.0 L 5.0,5.0 L 0.0,0.0 z "
+ style="fill-rule:evenodd;stroke:#000000;stroke-width:1.0pt;"
+ transform="scale(0.4) rotate(180) translate(10,0)" />
+ </marker>
+ <marker
+ inkscape:stockid="Arrow2Lend"
+ orient="auto"
+ refY="0.0"
+ refX="0.0"
+ id="Arrow2Lend"
+ style="overflow:visible;">
+ <path
+ id="path8425"
+ style="fill-rule:evenodd;stroke-width:0.62500000;stroke-linejoin:round;"
+ d="M 8.7185878,4.0337352 L -2.2072895,0.016013256 L 8.7185884,-4.0017078 C 6.9730900,-1.6296469 6.9831476,1.6157441 8.7185878,4.0337352 z "
+ transform="scale(1.1) rotate(180) translate(1,0)" />
+ </marker>
+ <marker
+ inkscape:stockid="Arrow1Lend"
+ orient="auto"
+ refY="0.0"
+ refX="0.0"
+ id="Arrow1Lend"
+ style="overflow:visible;">
+ <path
+ id="path8407"
+ d="M 0.0,0.0 L 5.0,-5.0 L -12.5,0.0 L 5.0,5.0 L 0.0,0.0 z "
+ style="fill-rule:evenodd;stroke:#000000;stroke-width:1.0pt;"
+ transform="scale(0.8) rotate(180) translate(12.5,0)" />
+ </marker>
+ <filter
+ id="filter_2"
+ color-interpolation-filters="sRGB">
+ <feGaussianBlur
+ stdDeviation="2"
+ id="feGaussianBlur15" />
+ </filter>
+ <filter
+ id="filter_2-3"
+ color-interpolation-filters="sRGB"
+ x="-0.086178862"
+ y="-0.086178862"
+ width="1.1723577"
+ height="1.1723577">
+ <feGaussianBlur
+ stdDeviation="2"
+ id="feGaussianBlur15-1" />
+ </filter>
+ <filter
+ id="filter_2-0"
+ color-interpolation-filters="sRGB">
+ <feGaussianBlur
+ stdDeviation="2"
+ id="feGaussianBlur15-7" />
+ </filter>
+ <filter
+ id="filter_2-0-8"
+ color-interpolation-filters="sRGB">
+ <feGaussianBlur
+ stdDeviation="2"
+ id="feGaussianBlur15-7-7" />
+ </filter>
+ <filter
+ id="filter_2-3-9"
+ color-interpolation-filters="sRGB"
+ x="-0.086178862"
+ y="-0.086178862"
+ width="1.1723577"
+ height="1.1723577">
+ <feGaussianBlur
+ stdDeviation="2"
+ id="feGaussianBlur15-1-6" />
+ </filter>
+ <filter
+ id="filter_2-3-6"
+ color-interpolation-filters="sRGB"
+ x="-0.086178862"
+ y="-0.086178862"
+ width="1.1723577"
+ height="1.1723577">
+ <feGaussianBlur
+ stdDeviation="2"
+ id="feGaussianBlur15-1-63" />
+ </filter>
+ <filter
+ id="filter_2-3-91"
+ color-interpolation-filters="sRGB"
+ x="-0.086178862"
+ y="-0.086178862"
+ width="1.1723577"
+ height="1.1723577">
+ <feGaussianBlur
+ stdDeviation="2"
+ id="feGaussianBlur15-1-3" />
+ </filter>
+ <marker
+ inkscape:stockid="Arrow1Lend"
+ orient="auto"
+ refY="0"
+ refX="0"
+ id="Arrow1Lend-5"
+ style="overflow:visible">
+ <path
+ inkscape:connector-curvature="0"
+ id="path8407-3"
+ d="M 0,0 5,-5 -12.5,0 5,5 0,0 z"
+ style="fill-rule:evenodd;stroke:#000000;stroke-width:1pt"
+ transform="matrix(-0.8,0,0,-0.8,-10,0)" />
+ </marker>
+ <marker
+ inkscape:stockid="Arrow1Lend"
+ orient="auto"
+ refY="0"
+ refX="0"
+ id="Arrow1Lend-6"
+ style="overflow:visible">
+ <path
+ inkscape:connector-curvature="0"
+ id="path8407-0"
+ d="M 0,0 5,-5 -12.5,0 5,5 0,0 z"
+ style="fill-rule:evenodd;stroke:#000000;stroke-width:1pt"
+ transform="matrix(-0.8,0,0,-0.8,-10,0)" />
+ </marker>
+ <marker
+ inkscape:stockid="Arrow1Lstart"
+ orient="auto"
+ refY="0"
+ refX="0"
+ id="Arrow1Lstart-7"
+ style="overflow:visible">
+ <path
+ inkscape:connector-curvature="0"
+ id="path8404-0"
+ d="M 0,0 5,-5 -12.5,0 5,5 0,0 z"
+ style="fill-rule:evenodd;stroke:#000000;stroke-width:1pt"
+ transform="matrix(0.8,0,0,0.8,10,0)" />
+ </marker>
+ <marker
+ inkscape:stockid="Arrow1Lend"
+ orient="auto"
+ refY="0"
+ refX="0"
+ id="Arrow1Lend-51"
+ style="overflow:visible">
+ <path
+ inkscape:connector-curvature="0"
+ id="path8407-1"
+ d="M 0,0 5,-5 -12.5,0 5,5 0,0 z"
+ style="fill-rule:evenodd;stroke:#000000;stroke-width:1pt"
+ transform="matrix(-0.8,0,0,-0.8,-10,0)" />
+ </marker>
+ <marker
+ inkscape:stockid="Arrow1Lend"
+ orient="auto"
+ refY="0"
+ refX="0"
+ id="Arrow1Lend-3"
+ style="overflow:visible">
+ <path
+ inkscape:connector-curvature="0"
+ id="path8407-6"
+ d="M 0,0 5,-5 -12.5,0 5,5 0,0 z"
+ style="fill-rule:evenodd;stroke:#000000;stroke-width:1pt"
+ transform="matrix(-0.8,0,0,-0.8,-10,0)" />
+ </marker>
+ <marker
+ inkscape:stockid="Arrow1Lend"
+ orient="auto"
+ refY="0"
+ refX="0"
+ id="Arrow1Lend-62"
+ style="overflow:visible">
+ <path
+ inkscape:connector-curvature="0"
+ id="path8407-9"
+ d="M 0,0 5,-5 -12.5,0 5,5 0,0 z"
+ style="fill-rule:evenodd;stroke:#000000;stroke-width:1pt"
+ transform="matrix(-0.8,0,0,-0.8,-10,0)" />
+ </marker>
+ <marker
+ inkscape:stockid="Arrow1Lend"
+ orient="auto"
+ refY="0"
+ refX="0"
+ id="Arrow1Lend-2"
+ style="overflow:visible">
+ <path
+ inkscape:connector-curvature="0"
+ id="path8407-7"
+ d="M 0,0 5,-5 -12.5,0 5,5 0,0 z"
+ style="fill-rule:evenodd;stroke:#000000;stroke-width:1pt"
+ transform="matrix(-0.8,0,0,-0.8,-10,0)" />
+ </marker>
+ <filter
+ id="filter_2-3-91-3"
+ color-interpolation-filters="sRGB"
+ x="-0.086178862"
+ y="-0.086178862"
+ width="1.1723577"
+ height="1.1723577">
+ <feGaussianBlur
+ stdDeviation="2"
+ id="feGaussianBlur15-1-3-6" />
+ </filter>
+ </defs>
+ <sodipodi:namedview
+ id="base"
+ pagecolor="#ffffff"
+ bordercolor="#666666"
+ borderopacity="1.0"
+ inkscape:pageopacity="0.0"
+ inkscape:pageshadow="2"
+ inkscape:zoom="1"
+ inkscape:cx="385.5"
+ inkscape:cy="234"
+ inkscape:document-units="px"
+ inkscape:current-layer="layer1"
+ showgrid="false"
+ inkscape:window-width="1920"
+ inkscape:window-height="1017"
+ inkscape:window-x="-8"
+ inkscape:window-y="-8"
+ inkscape:window-maximized="1"
+ inkscape:snap-nodes="false"
+ inkscape:showpageshadow="2"
+ inkscape:pagecheckerboard="0"
+ inkscape:deskcolor="#d1d1d1">
+ <inkscape:grid
+ type="xygrid"
+ id="grid13454" />
+ </sodipodi:namedview>
+ <metadata
+ id="metadata13242">
+ <rdf:RDF>
+ <cc:Work
+ rdf:about="">
+ <dc:format>image/svg+xml</dc:format>
+ <dc:type
+ rdf:resource="http://purl.org/dc/dcmitype/StillImage" />
+ <dc:title />
+ </cc:Work>
+ </rdf:RDF>
+ </metadata>
+ <g
+ id="layer1"
+ inkscape:label="Layer 1"
+ inkscape:groupmode="layer">
+ <g
+ style="font-size:12px;fill:none;stroke-linecap:square;stroke-miterlimit:3;overflow:visible"
+ id="shape1-1-2-0"
+ v:mID="1"
+ v:groupContext="shape"
+ transform="matrix(2.1604167,0,0,1.5671361,323.2187,-540.25927)">
+ <title
+ id="title22-7-6">Square</title>
+ <desc
+ id="desc24-7-4">Atomic Queue #1</desc>
+ <v:userDefs>
+ <v:ud
+ v:nameU="visVersion"
+ v:val="VT0(15):26" />
+ </v:userDefs>
+ <v:textBlock
+ v:margins="rect(4,4,4,4)" />
+ <v:textRect
+ cx="30.75"
+ cy="581.25"
+ width="61.5"
+ height="61.5" />
+ <g
+ id="shadow1-2-9-0"
+ v:groupContext="shadow"
+ v:shadowOffsetX="0.345598"
+ v:shadowOffsetY="-1.97279"
+ v:shadowType="1"
+ transform="translate(0.345598,1.97279)"
+ class="st1"
+ style="visibility:visible">
+ <rect
+ x="0"
+ y="550.5"
+ width="61.5"
+ height="61.5"
+ class="st2"
+ id="rect27-8-9"
+ style="fill:#5b9bd5;fill-opacity:0.22000002;stroke:#5b9bd5;stroke-opacity:0.22000002;filter:url(#filter_2-3-9)" />
+ </g>
+ <g
+ id="g13515-4">
+ <g
+ id="g13534-5">
+ <rect
+ x="0"
+ y="550.5"
+ width="61.5"
+ height="61.5"
+ class="st3"
+ id="rect29-1-4"
+ style="fill:#5b9bd5;stroke:#c7c8c8;stroke-width:0.25" />
+ </g>
+ </g>
+ </g>
+ <g
+ style="font-size:12px;fill:none;stroke-linecap:square;stroke-miterlimit:3;overflow:visible"
+ id="shape1-1-2-4"
+ v:mID="1"
+ v:groupContext="shape"
+ transform="matrix(2.1604167,0,0,1.165886,88.874699,-447.8809)">
+ <title
+ id="title22-7-5">Square</title>
+ <desc
+ id="desc24-7-8">Atomic Queue #1</desc>
+ <v:userDefs>
+ <v:ud
+ v:nameU="visVersion"
+ v:val="VT0(15):26" />
+ </v:userDefs>
+ <v:textBlock
+ v:margins="rect(4,4,4,4)" />
+ <v:textRect
+ cx="30.75"
+ cy="581.25"
+ width="61.5"
+ height="61.5" />
+ <g
+ id="shadow1-2-9-5"
+ v:groupContext="shadow"
+ v:shadowOffsetX="0.345598"
+ v:shadowOffsetY="-1.97279"
+ v:shadowType="1"
+ transform="translate(0.345598,1.97279)"
+ class="st1"
+ style="visibility:visible">
+ <rect
+ x="0"
+ y="550.5"
+ width="61.5"
+ height="61.5"
+ class="st2"
+ id="rect27-8-7"
+ style="fill:#5b9bd5;fill-opacity:0.22000002;stroke:#5b9bd5;stroke-opacity:0.22000002;filter:url(#filter_2-3-91)" />
+ </g>
+ <g
+ id="g13515-33">
+ <g
+ id="g13534-8">
+ <rect
+ x="0"
+ y="550.5"
+ width="61.5"
+ height="61.5"
+ class="st3"
+ id="rect29-1-95"
+ style="fill:#5b9bd5;stroke:#c7c8c8;stroke-width:0.25" />
+ </g>
+ </g>
+ </g>
+ <g
+ style="font-size:12px;fill:none;stroke-linecap:square;stroke-miterlimit:3;overflow:visible"
+ id="shape1-1-2-9"
+ v:mID="1"
+ v:groupContext="shape"
+ transform="matrix(2.1604167,0,0,1.5671361,88.874699,-538.24651)">
+ <title
+ id="title22-7-9">Square</title>
+ <desc
+ id="desc24-7-5">Atomic Queue #1</desc>
+ <v:userDefs>
+ <v:ud
+ v:nameU="visVersion"
+ v:val="VT0(15):26" />
+ </v:userDefs>
+ <v:textBlock
+ v:margins="rect(4,4,4,4)" />
+ <v:textRect
+ cx="30.75"
+ cy="581.25"
+ width="61.5"
+ height="61.5" />
+ <g
+ id="shadow1-2-9-2"
+ v:groupContext="shadow"
+ v:shadowOffsetX="0.345598"
+ v:shadowOffsetY="-1.97279"
+ v:shadowType="1"
+ transform="translate(0.345598,1.97279)"
+ class="st1"
+ style="visibility:visible">
+ <rect
+ x="0"
+ y="550.5"
+ width="61.5"
+ height="61.5"
+ class="st2"
+ id="rect27-8-1"
+ style="fill:#5b9bd5;fill-opacity:0.22000002;stroke:#5b9bd5;stroke-opacity:0.22000002;filter:url(#filter_2-3-6)" />
+ </g>
+ <g
+ id="g13515-3">
+ <g
+ id="g13534-0">
+ <rect
+ x="0"
+ y="550.5"
+ width="61.5"
+ height="61.5"
+ class="st3"
+ id="rect29-1-9"
+ style="fill:#5b9bd5;stroke:#c7c8c8;stroke-width:0.25" />
+ </g>
+ </g>
+ </g>
+ <path
+ style="fill:none;stroke:#000000;stroke-width:0.743466px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1;marker-mid:none;marker-end:url(#Arrow1Lend)"
+ d="M 220.66064,98.57125 H 321.88592"
+ id="path17209-8"
+ inkscape:connector-type="orthogonal"
+ inkscape:connector-curvature="0" />
+ <g
+ style="font-size:12px;fill:none;stroke-linecap:square;stroke-miterlimit:3;overflow:visible"
+ id="shape1-1-2"
+ v:mID="1"
+ v:groupContext="shape"
+ transform="matrix(2.1604167,0,0,1.5671361,322.24227,-811.89589)">
+ <title
+ id="title22-7">Square</title>
+ <desc
+ id="desc24-7">Atomic Queue #1</desc>
+ <v:userDefs>
+ <v:ud
+ v:nameU="visVersion"
+ v:val="VT0(15):26" />
+ </v:userDefs>
+ <v:textBlock
+ v:margins="rect(4,4,4,4)" />
+ <v:textRect
+ cx="30.75"
+ cy="581.25"
+ width="61.5"
+ height="61.5" />
+ <g
+ id="shadow1-2-9"
+ v:groupContext="shadow"
+ v:shadowOffsetX="0.345598"
+ v:shadowOffsetY="-1.97279"
+ v:shadowType="1"
+ transform="translate(0.345598,1.97279)"
+ class="st1"
+ style="visibility:visible">
+ <rect
+ x="0"
+ y="550.5"
+ width="61.5"
+ height="61.5"
+ class="st2"
+ id="rect27-8"
+ style="fill:#5b9bd5;fill-opacity:0.22000002;stroke:#5b9bd5;stroke-opacity:0.22000002;filter:url(#filter_2-3)" />
+ </g>
+ <g
+ id="g13515">
+ <g
+ id="g13534">
+ <rect
+ x="0"
+ y="550.5"
+ width="61.5"
+ height="61.5"
+ class="st3"
+ id="rect29-1"
+ style="fill:#5b9bd5;stroke:#c7c8c8;stroke-width:0.25" />
+ </g>
+ </g>
+ </g>
+ <g
+ id="g13518"
+ transform="matrix(0.73232502,0,0,0.75477602,25.29268,348.89752)">
+ <g
+ id="g13526">
+ <flowRoot
+ xml:space="preserve"
+ id="flowRoot13464-9"
+ style="font-style:normal;font-weight:normal;line-height:0.01%;font-family:sans-serif;text-align:center;letter-spacing:0px;word-spacing:0px;text-anchor:middle;fill:#000000;fill-opacity:1;stroke:none"
+ transform="translate(-12.00521,-129.65179)"><flowRegion
+ id="flowRegion13466-1"
+ style="font-family:sans-serif"><rect
+ id="rect13468-2"
+ width="195.99997"
+ height="112.00001"
+ x="273.33334"
+ y="175.33333"
+ style="text-align:center;text-anchor:middle" /></flowRegion><flowPara
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:20px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:center;writing-mode:lr-tb;text-anchor:middle"
+ id="flowPara13511"> </flowPara></flowRoot>
+ </g>
+ </g>
+ <path
+ style="fill:none;stroke:#000000;stroke-width:0.751455;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:4;stroke-dasharray:none;stroke-opacity:1;marker-start:url(#Arrow1Lstart);marker-end:none"
+ d="m 176.26096,124.64833 v 69.24854"
+ id="path17209-3"
+ inkscape:connector-type="orthogonal"
+ inkscape:connector-curvature="0" />
+ <path
+ transform="matrix(0.73232502,0,0,0.75477602,-4.325033,50.642983)"
+ sodipodi:type="arc"
+ style="fill:#539de6;fill-opacity:1;stroke:#0000ea;stroke-width:1;stroke-linecap:square;stroke-miterlimit:4;stroke-opacity:1;stroke-dasharray:none"
+ id="path29161-3"
+ sodipodi:cx="371"
+ sodipodi:cy="64.5"
+ sodipodi:rx="17"
+ sodipodi:ry="15.5"
+ d="m 388,64.5 a 17,15.5 0 1 1 -34,0 17,15.5 0 1 1 34,0 z" />
+ <path
+ style="fill:none;stroke:#000000;stroke-width:0.743466px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1;marker-mid:none;marker-end:url(#Arrow1Lend)"
+ d="M 322.61264,375 H 221.38736"
+ id="path17209-8-0"
+ inkscape:connector-type="orthogonal"
+ inkscape:connector-curvature="0" />
+ <path
+ transform="matrix(0.73232502,0,0,0.75477602,0.0689171,324.80833)"
+ sodipodi:type="arc"
+ style="fill:#539de6;fill-opacity:1;stroke:#0000ea;stroke-width:1;stroke-linecap:square;stroke-miterlimit:4;stroke-opacity:1;stroke-dasharray:none"
+ id="path29161-1"
+ sodipodi:cx="371"
+ sodipodi:cy="64.5"
+ sodipodi:rx="17"
+ sodipodi:ry="15.5"
+ d="m 388,64.5 a 17,15.5 0 1 1 -34,0 17,15.5 0 1 1 34,0 z" />
+ <path
+ style="fill:none;stroke:#000000;stroke-width:0.629082px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1;marker-end:url(#Arrow1Lend)"
+ d="M 155,324.19955 V 264.82863"
+ id="path17211-7-1"
+ inkscape:connector-type="orthogonal"
+ inkscape:connector-curvature="0" />
+ <path
+ transform="matrix(0.73232502,0,0,0.75477602,-116.37076,245.97088)"
+ sodipodi:type="arc"
+ style="fill:#539de6;fill-opacity:1;stroke:#0000ea;stroke-width:1;stroke-linecap:square;stroke-miterlimit:4;stroke-opacity:1;stroke-dasharray:none"
+ id="path29161-8"
+ sodipodi:cx="371"
+ sodipodi:cy="64.5"
+ sodipodi:rx="17"
+ sodipodi:ry="15.5"
+ d="m 388,64.5 a 17,15.5 0 1 1 -34,0 17,15.5 0 1 1 34,0 z" />
+ <path
+ style="fill:none;stroke:#000000;stroke-width:0.75059;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:4;stroke-dasharray:none;stroke-opacity:1;marker-start:url(#Arrow1Lstart-7);marker-end:none"
+ d="m 126.26097,124.99178 v 69.24941"
+ id="path17209-3-0"
+ inkscape:connector-type="orthogonal"
+ inkscape:connector-curvature="0" />
+ <path
+ sodipodi:type="arc"
+ style="fill:#539de6;fill-opacity:1;stroke:#0000ea;stroke-width:1;stroke-linecap:square;stroke-miterlimit:4;stroke-opacity:1;stroke-dasharray:none"
+ id="path29161"
+ sodipodi:cx="371"
+ sodipodi:cy="64.5"
+ sodipodi:rx="17"
+ sodipodi:ry="15.5"
+ d="m 388,64.5 a 17,15.5 0 1 1 -34,0 17,15.5 0 1 1 34,0 z"
+ transform="matrix(0.73232502,0,0,0.75477602,-146.16394,110.68276)" />
+ <path
+ sodipodi:type="arc"
+ style="fill:#539de6;fill-opacity:1;stroke:#0000ea;stroke-width:1;stroke-linecap:square;stroke-miterlimit:4;stroke-opacity:1;stroke-dasharray:none"
+ id="path29161-2"
+ sodipodi:cx="371"
+ sodipodi:cy="64.5"
+ sodipodi:rx="17"
+ sodipodi:ry="15.5"
+ d="m 388,64.5 a 17,15.5 0 1 1 -34,0 17,15.5 0 1 1 34,0 z"
+ transform="matrix(0.73232502,0,0,0.75477602,-95.40136,110.71627)" />
+ <g
+ id="g29167-4"
+ transform="matrix(0.73232502,0,0,0.75477602,-95.31662,112.00756)">
+ <text
+ id="text29163-9"
+ y="70"
+ x="365"
+ style="font-style:normal;font-weight:normal;line-height:0%;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none"
+ xml:space="preserve"><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="70"
+ x="365"
+ id="tspan29165-9"
+ sodipodi:role="line">1</tspan></text>
+ </g>
+ <g
+ id="g29167-9"
+ transform="matrix(0.73232502,0,0,0.75477602,-4.9726112,50.689051)">
+ <text
+ id="text29163-3"
+ y="70"
+ x="365"
+ style="font-style:normal;font-weight:normal;line-height:0%;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none"
+ xml:space="preserve"><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="70"
+ x="365"
+ id="tspan29165-3"
+ sodipodi:role="line">2</tspan></text>
+ </g>
+ <path
+ style="fill:none;stroke:#000000;stroke-width:1.04033px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1;marker-end:url(#Arrow1Lend)"
+ d="M 388.20118,147.93341 V 321.89308"
+ id="path17211-7"
+ inkscape:connector-type="orthogonal"
+ inkscape:connector-curvature="0" />
+ <path
+ transform="matrix(0.73232502,0,0,0.75477602,116.5086,136.68745)"
+ sodipodi:type="arc"
+ style="fill:#539de6;fill-opacity:1;stroke:#0000ea;stroke-width:1;stroke-linecap:square;stroke-miterlimit:4;stroke-opacity:1;stroke-dasharray:none"
+ id="path29161-6"
+ sodipodi:cx="371"
+ sodipodi:cy="64.5"
+ sodipodi:rx="17"
+ sodipodi:ry="15.5"
+ d="m 388,64.5 a 17,15.5 0 1 1 -34,0 17,15.5 0 1 1 34,0 z" />
+ <g
+ id="g29167-46"
+ transform="matrix(0.73232502,0,0,0.75477602,116.59334,137.48829)">
+ <text
+ id="text29163-1"
+ y="70"
+ x="365"
+ style="font-style:normal;font-weight:normal;line-height:0%;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none"
+ xml:space="preserve"><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="70"
+ x="365"
+ id="tspan29165-5"
+ sodipodi:role="line">3</tspan></text>
+ </g>
+ <g
+ id="g29167-6"
+ transform="matrix(0.73232502,0,0,0.75477602,0.1536639,325.60918)">
+ <text
+ id="text29163-17"
+ y="70"
+ x="365"
+ style="font-style:normal;font-weight:normal;line-height:0%;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none"
+ xml:space="preserve"><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="70"
+ x="365"
+ id="tspan29165-2"
+ sodipodi:role="line">4</tspan></text>
+ </g>
+ <g
+ id="g29167"
+ transform="matrix(0.73232502,0,0,0.75477602,-146.07919,111.10621)">
+ <text
+ id="text29163"
+ y="70"
+ x="365"
+ style="font-style:normal;font-weight:normal;line-height:0%;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none"
+ xml:space="preserve"><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="70"
+ x="365"
+ id="tspan29165"
+ sodipodi:role="line">6</tspan></text>
+ </g>
+ <g
+ id="g29167-4-3"
+ transform="matrix(0.73232502,0,0,0.75477602,-117.60784,180.11779)">
+ <text
+ id="text29163-9-6"
+ y="70"
+ x="321.30356"
+ style="font-style:normal;font-weight:normal;line-height:0%;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none"
+ xml:space="preserve"><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:24.2109px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="70"
+ x="321.30356"
+ id="tspan29165-9-7"
+ sodipodi:role="line">Eventdev</tspan></text>
+ </g>
+ <g
+ id="g29167-4-3-5"
+ transform="matrix(0.73232502,0,0,0.75477602,55.34956,26.97821)">
+ <text
+ id="text29163-9-6-3"
+ y="70"
+ x="454.74152"
+ style="font-style:normal;font-weight:normal;line-height:0%;font-family:sans-serif;text-align:center;letter-spacing:0px;word-spacing:0px;text-anchor:middle;fill:#000000;fill-opacity:1;stroke:none"
+ xml:space="preserve"><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:21.5208px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:center;writing-mode:lr-tb;text-anchor:middle"
+ y="70"
+ x="454.74152"
+ id="tspan29165-9-7-5"
+ sodipodi:role="line">A<tspan
+ style="line-height:100%;font-family:sans-serif"
+ id="tspan3374">tomic Stage</tspan></tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:21.5208px;line-height:100%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:center;writing-mode:lr-tb;text-anchor:middle"
+ y="94.210899"
+ x="454.74152"
+ sodipodi:role="line"
+ id="tspan3320">+</tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:21.5208px;line-height:100%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:center;writing-mode:lr-tb;text-anchor:middle"
+ y="115.7317"
+ x="454.74152"
+ sodipodi:role="line"
+ id="tspan3322">Enqueue to</tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:21.5208px;line-height:100%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:center;writing-mode:lr-tb;text-anchor:middle"
+ y="137.2525"
+ x="454.74152"
+ sodipodi:role="line"
+ id="tspan3324">DMA Device</tspan></text>
+ </g>
+ <g
+ id="g29167-2"
+ transform="matrix(0.73232502,0,0,0.75477602,-116.28602,248.01695)">
+ <text
+ id="text29163-92"
+ y="70"
+ x="365"
+ style="font-style:normal;font-weight:normal;line-height:0%;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none"
+ xml:space="preserve"><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="70"
+ x="365"
+ id="tspan29165-8"
+ sodipodi:role="line">5</tspan></text>
+ </g>
+ <flowRoot
+ xml:space="preserve"
+ id="flowRoot3376"
+ style="font-style:normal;font-weight:normal;line-height:0.01%;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none;stroke-width:1px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1"><flowRegion
+ id="flowRegion3378"
+ style="font-family:sans-serif"><rect
+ id="rect3380"
+ width="100"
+ height="37"
+ x="109"
+ y="259" /></flowRegion><flowPara
+ id="flowPara3382"
+ style="font-size:18px;line-height:1.25;font-family:sans-serif"> </flowPara></flowRoot>
+ <g
+ id="g29167-4-3-1"
+ transform="matrix(0.73232502,0,0,0.75477602,109.34956,323.97821)">
+ <text
+ id="text29163-9-6-8"
+ y="70"
+ x="321.30356"
+ style="font-style:normal;font-weight:normal;line-height:0%;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none"
+ xml:space="preserve"><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:24.2109px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="70"
+ x="321.30356"
+ id="tspan29165-9-7-7"
+ sodipodi:role="line">DMA Device</tspan></text>
+ </g>
+ <g
+ id="g29167-4-3-1-9"
+ transform="matrix(0.73232502,0,0,0.75477602,-114.48565,314.20704)">
+ <text
+ id="text29163-9-6-8-2"
+ y="70"
+ x="368.01718"
+ style="font-style:normal;font-weight:normal;line-height:0%;font-family:sans-serif;text-align:center;letter-spacing:0px;word-spacing:0px;text-anchor:middle;fill:#000000;fill-opacity:1;stroke:none"
+ xml:space="preserve"><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:24.2109px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:center;writing-mode:lr-tb;text-anchor:middle"
+ y="70"
+ x="368.01718"
+ id="tspan29165-9-7-7-0"
+ sodipodi:role="line">DMA</tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:24.2109px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:center;writing-mode:lr-tb;text-anchor:middle"
+ y="100.26363"
+ x="368.01718"
+ sodipodi:role="line"
+ id="tspan3488">Adapter</tspan></text>
+ </g>
+ <g
+ id="g29167-4-3-1-9-2"
+ transform="matrix(0.73232502,0,0,0.75477602,250.96804,192.62529)">
+ <text
+ id="text29163-9-6-8-2-3"
+ y="-188.35481"
+ x="318.61978"
+ style="font-style:normal;font-weight:normal;line-height:0%;font-family:sans-serif;text-align:center;letter-spacing:0px;word-spacing:0px;text-anchor:middle;fill:#000000;fill-opacity:1;stroke:none"
+ xml:space="preserve"><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:21.5208px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="-188.35481"
+ x="318.61978"
+ sodipodi:role="line"
+ id="tspan3543">1. Application dequeues</tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:21.5208px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="-161.45381"
+ x="318.61978"
+ sodipodi:role="line"
+ id="tspan3196"> events from the previous</tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:21.5208px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="-134.55281"
+ x="318.61978"
+ sodipodi:role="line"
+ id="tspan3232"> stage</tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:21.5208px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="-107.65182"
+ x="318.61978"
+ sodipodi:role="line"
+ id="tspan3519"> </tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:21.5208px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="-80.750816"
+ x="318.61978"
+ sodipodi:role="line"
+ id="tspan3551">2. Application prepares the</tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:21.5208px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="-53.849815"
+ x="318.61978"
+ sodipodi:role="line"
+ id="tspan3203"> DMA operations.</tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:21.5208px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="-26.948814"
+ x="318.61978"
+ sodipodi:role="line"
+ id="tspan3523"> </tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:21.5208px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="-0.0478158"
+ x="318.61978"
+ sodipodi:role="line"
+ id="tspan3541">3. DMA operations are</tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:21.5208px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="26.853184"
+ x="318.61978"
+ sodipodi:role="line"
+ id="tspan3207"> submitted to dmadev</tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:21.5208px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="53.754185"
+ x="318.61978"
+ sodipodi:role="line"
+ id="tspan3209"> by application.</tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:21.5208px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="80.655182"
+ x="318.61978"
+ sodipodi:role="line"
+ id="tspan3527"> </tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:21.5208px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="107.55618"
+ x="318.61978"
+ sodipodi:role="line"
+ id="tspan3547">4. DMA adapter dequeues</tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:21.5208px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="134.45718"
+ x="318.61978"
+ sodipodi:role="line"
+ id="tspan3216"> DMA completions from</tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:21.5208px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="161.35818"
+ x="318.61978"
+ sodipodi:role="line"
+ id="tspan3218"> DMA device.</tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:21.5208px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="188.25919"
+ x="318.61978"
+ sodipodi:role="line"
+ id="tspan3531"> </tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:21.5208px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="215.16019"
+ x="318.61978"
+ sodipodi:role="line"
+ id="tspan3549">5. DMA adapter enqueues</tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:21.5208px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="242.06119"
+ x="318.61978"
+ sodipodi:role="line"
+ id="tspan3222"> events to the eventdev.</tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:21.5208px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="268.96219"
+ x="318.61978"
+ sodipodi:role="line"
+ id="tspan3535"> </tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:21.5208px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="295.86319"
+ x="318.61978"
+ sodipodi:role="line"
+ id="tspan3537">6. Application dequeues from</tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:21.5208px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="322.76419"
+ x="318.61978"
+ sodipodi:role="line"
+ id="tspan3224"> eventdev and prepare for</tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:21.5208px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="349.66519"
+ x="318.61978"
+ sodipodi:role="line"
+ id="tspan3226"> further processing</tspan></text>
+ </g>
+ <g
+ style="font-size:12px;fill:none;stroke-linecap:square;stroke-miterlimit:3;overflow:visible"
+ id="shape1-1-2-4-7"
+ v:mID="1"
+ v:groupContext="shape"
+ transform="matrix(2.1604167,0,0,1.165886,90.820551,-587.97129)">
+ <title
+ id="title22-7-5-5">Square</title>
+ <desc
+ id="desc24-7-8-3">Atomic Queue #1</desc>
+ <v:userDefs>
+ <v:ud
+ v:nameU="visVersion"
+ v:val="VT0(15):26" />
+ </v:userDefs>
+ <v:textBlock
+ v:margins="rect(4,4,4,4)" />
+ <v:textRect
+ cx="30.75"
+ cy="581.25"
+ width="61.5"
+ height="61.5" />
+ <g
+ id="shadow1-2-9-5-5"
+ v:groupContext="shadow"
+ v:shadowOffsetX="0.345598"
+ v:shadowOffsetY="-1.97279"
+ v:shadowType="1"
+ transform="translate(0.345598,1.97279)"
+ class="st1"
+ style="visibility:visible">
+ <rect
+ x="0"
+ y="550.5"
+ width="61.5"
+ height="61.5"
+ class="st2"
+ id="rect27-8-7-6"
+ style="fill:#5b9bd5;fill-opacity:0.22000002;stroke:#5b9bd5;stroke-opacity:0.22000002;filter:url(#filter_2-3-91-3)" />
+ </g>
+ <g
+ id="g13515-33-2">
+ <g
+ id="g13534-8-9">
+ <rect
+ x="0"
+ y="550.5"
+ width="61.5"
+ height="61.5"
+ class="st3"
+ id="rect29-1-95-1"
+ style="fill:#5b9bd5;stroke:#c7c8c8;stroke-width:0.25" />
+ </g>
+ </g>
+ </g>
+ <g
+ id="g29167-4-3-2"
+ transform="matrix(0.73232502,0,0,0.75477602,-125.66199,44.027402)">
+ <text
+ id="text29163-9-6-7"
+ y="70"
+ x="321.30356"
+ style="font-style:normal;font-weight:normal;line-height:0%;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none"
+ xml:space="preserve"><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:24.2109px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="70"
+ x="321.30356"
+ id="tspan29165-9-7-0"
+ sodipodi:role="line">Application</tspan></text>
+ </g>
+ </g>
+</svg>
diff --git a/doc/guides/prog_guide/index.rst b/doc/guides/prog_guide/index.rst
index 52a6d9e7aa..beaa4b8869 100644
--- a/doc/guides/prog_guide/index.rst
+++ b/doc/guides/prog_guide/index.rst
@@ -60,6 +60,7 @@ Programmer's Guide
event_ethernet_tx_adapter
event_timer_adapter
event_crypto_adapter
+ event_dma_adapter
qos_framework
power_man
packet_classif_access_ctrl
diff --git a/doc/guides/rel_notes/release_23_11.rst b/doc/guides/rel_notes/release_23_11.rst
index b34ddc0860..70f2e7e03c 100644
--- a/doc/guides/rel_notes/release_23_11.rst
+++ b/doc/guides/rel_notes/release_23_11.rst
@@ -44,7 +44,9 @@ New Features
.. This section should contain new features added in this release.
Sample format:
- * **Add a title in the past tense with a full stop.**
+ * **Added eventdev DMA adapter library.**
+
+ Added API to DMA transfer data using event mechanism.
Add a short 1-2 sentence description in the past tense.
The description should be enough to allow someone scanning
diff --git a/lib/eventdev/eventdev_pmd.h b/lib/eventdev/eventdev_pmd.h
index f62f42e140..8e5fd420f6 100644
--- a/lib/eventdev/eventdev_pmd.h
+++ b/lib/eventdev/eventdev_pmd.h
@@ -178,8 +178,12 @@ struct rte_eventdev {
event_tx_adapter_enqueue_t txa_enqueue;
/**< Pointer to PMD eth Tx adapter enqueue function. */
event_crypto_adapter_enqueue_t ca_enqueue;
+ /**< Pointer to PMD crypto adapter enqueue function. */
- uint64_t reserved_64s[4]; /**< Reserved for future fields */
+ event_dma_adapter_enqueue_t dma_enqueue;
+ /**< Pointer to PMD DMA adapter enqueue function. */
+
+ uint64_t reserved_64s[3]; /**< Reserved for future fields */
void *reserved_ptrs[3]; /**< Reserved for future fields */
} __rte_cache_aligned;
@@ -1320,6 +1324,160 @@ typedef int (*eventdev_eth_tx_adapter_queue_stop)
#define eventdev_stop_flush_t rte_eventdev_stop_flush_t
+/**
+ * Retrieve the event device's DMA adapter capabilities for the
+ * specified DMA device
+ *
+ * @param dev
+ * Event device pointer
+ *
+ * @param dma_dev_id
+ * DMA device identification
+ *
+ * @param[out] caps
+ * A pointer to memory filled with event adapter capabilities.
+ * It is expected to be pre-allocated & initialized by caller.
+ *
+ * @return
+ * - 0: Success, driver provides event adapter capabilities for the
+ * DMADEV.
+ * - <0: Error code returned by the driver function.
+ *
+ */
+typedef int (*eventdev_dma_adapter_caps_get_t)(const struct rte_eventdev *dev,
+ const int16_t dma_dev_id, uint32_t *caps);
+
+/**
+ * This API may change without prior notice
+ *
+ * Add DMA vchan queue to event device. This callback is invoked if
+ * the caps returned from rte_event_dma_adapter_caps_get(, dmadev_id)
+ * has RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_* set.
+ *
+ * @param dev
+ * Event device pointer
+ *
+ * @param dma_dev_id
+ * DMADEV identification
+ *
+ * @param vchan_id
+ * DMADEV vchan queue identifier.
+ *
+ * @param event
+ * Event information required for binding dmadev queue pair to event queue.
+ * This structure will have a valid value for only those HW PMDs supporting
+ * @see RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_VCHAN_EV_BIND capability.
+ *
+ * @return
+ * - 0: Success, dmadev vchan added successfully.
+ * - <0: Error code returned by the driver function.
+ *
+ */
+typedef int (*eventdev_dma_adapter_vchan_add_t)(const struct rte_eventdev *dev,
+ const int16_t dma_dev_id,
+ uint16_t vchan_id,
+ const struct rte_event *event);
+
+/**
+ * This API may change without prior notice
+ *
+ * Delete DMA vhcan to event device. This callback is invoked if
+ * the caps returned from rte_event_dma_adapter_caps_get(, dmadev_id)
+ * has RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_* set.
+ *
+ * @param dev
+ * Event device pointer
+ *
+ * @param dma_dev_id
+ * DMADEV identification
+ *
+ * @param vchan_id
+ * dmadev vchan identifier.
+ *
+ * @return
+ * - 0: Success, dmadev vchan deleted successfully.
+ * - <0: Error code returned by the driver function.
+ *
+ */
+typedef int (*eventdev_dma_adapter_vchan_del_t)(const struct rte_eventdev *dev,
+ const int16_t dma_dev_id,
+ uint16_t vchan_id);
+
+/**
+ * Start DMA adapter. This callback is invoked if
+ * the caps returned from rte_event_dma_adapter_caps_get(.., dmadev_id)
+ * has RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_* set and vchan for dmadev_id
+ * have been added to the event device.
+ *
+ * @param dev
+ * Event device pointer
+ *
+ * @param dma_dev_id
+ * DMA device identification
+ *
+ * @return
+ * - 0: Success, DMA adapter started successfully.
+ * - <0: Error code returned by the driver function.
+ */
+typedef int (*eventdev_dma_adapter_start_t)(const struct rte_eventdev *dev,
+ const int16_t dma_dev_id);
+
+/**
+ * Stop DMA adapter. This callback is invoked if
+ * the caps returned from rte_event_dma_adapter_caps_get(.., dmadev_id)
+ * has RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_* set and vchan for dmadev_id
+ * have been added to the event device.
+ *
+ * @param dev
+ * Event device pointer
+ *
+ * @param dma_dev_id
+ * DMA device identification
+ *
+ * @return
+ * - 0: Success, DMA adapter stopped successfully.
+ * - <0: Error code returned by the driver function.
+ */
+typedef int (*eventdev_dma_adapter_stop_t)(const struct rte_eventdev *dev,
+ const int16_t dma_dev_id);
+
+struct rte_event_dma_adapter_stats;
+
+/**
+ * Retrieve DMA adapter statistics.
+ *
+ * @param dev
+ * Event device pointer
+ *
+ * @param dma_dev_id
+ * DMA device identification
+ *
+ * @param[out] stats
+ * Pointer to stats structure
+ *
+ * @return
+ * Return 0 on success.
+ */
+typedef int (*eventdev_dma_adapter_stats_get)(const struct rte_eventdev *dev,
+ const int16_t dma_dev_id,
+ struct rte_event_dma_adapter_stats *stats);
+
+/**
+ * Reset DMA adapter statistics.
+ *
+ * @param dev
+ * Event device pointer
+ *
+ * @param dma_dev_id
+ * DMA device identification
+ *
+ * @return
+ * Return 0 on success.
+ */
+typedef int (*eventdev_dma_adapter_stats_reset)(const struct rte_eventdev *dev,
+ const int16_t dma_dev_id);
+
+
/** Event device operations function pointer table */
struct eventdev_ops {
eventdev_info_get_t dev_infos_get; /**< Get device info. */
@@ -1440,6 +1598,21 @@ struct eventdev_ops {
eventdev_eth_tx_adapter_queue_stop eth_tx_adapter_queue_stop;
/**< Stop Tx queue assigned to Tx adapter instance */
+ eventdev_dma_adapter_caps_get_t dma_adapter_caps_get;
+ /**< Get DMA adapter capabilities */
+ eventdev_dma_adapter_vchan_add_t dma_adapter_vchan_add;
+ /**< Add vchan queue to DMA adapter */
+ eventdev_dma_adapter_vchan_del_t dma_adapter_vchan_del;
+ /**< Delete vchan queue from DMA adapter */
+ eventdev_dma_adapter_start_t dma_adapter_start;
+ /**< Start DMA adapter */
+ eventdev_dma_adapter_stop_t dma_adapter_stop;
+ /**< Stop DMA adapter */
+ eventdev_dma_adapter_stats_get dma_adapter_stats_get;
+ /**< Get DMA stats */
+ eventdev_dma_adapter_stats_reset dma_adapter_stats_reset;
+ /**< Reset DMA stats */
+
eventdev_selftest dev_selftest;
/**< Start eventdev Selftest */
diff --git a/lib/eventdev/eventdev_private.c b/lib/eventdev/eventdev_private.c
index 1d3d9d357e..18ed8bf3c8 100644
--- a/lib/eventdev/eventdev_private.c
+++ b/lib/eventdev/eventdev_private.c
@@ -81,6 +81,14 @@ dummy_event_crypto_adapter_enqueue(__rte_unused void *port,
return 0;
}
+static uint16_t
+dummy_event_dma_adapter_enqueue(__rte_unused void *port, __rte_unused struct rte_event ev[],
+ __rte_unused uint16_t nb_events)
+{
+ RTE_EDEV_LOG_ERR("event DMA adapter enqueue requested for unconfigured event device");
+ return 0;
+}
+
void
event_dev_fp_ops_reset(struct rte_event_fp_ops *fp_op)
{
@@ -97,6 +105,7 @@ event_dev_fp_ops_reset(struct rte_event_fp_ops *fp_op)
.txa_enqueue_same_dest =
dummy_event_tx_adapter_enqueue_same_dest,
.ca_enqueue = dummy_event_crypto_adapter_enqueue,
+ .dma_enqueue = dummy_event_dma_adapter_enqueue,
.data = dummy_data,
};
@@ -117,5 +126,6 @@ event_dev_fp_ops_set(struct rte_event_fp_ops *fp_op,
fp_op->txa_enqueue = dev->txa_enqueue;
fp_op->txa_enqueue_same_dest = dev->txa_enqueue_same_dest;
fp_op->ca_enqueue = dev->ca_enqueue;
+ fp_op->dma_enqueue = dev->dma_enqueue;
fp_op->data = dev->data->ports;
}
diff --git a/lib/eventdev/meson.build b/lib/eventdev/meson.build
index 6edf98dfa5..21347f7c4c 100644
--- a/lib/eventdev/meson.build
+++ b/lib/eventdev/meson.build
@@ -25,6 +25,7 @@ sources = files(
)
headers = files(
'rte_event_crypto_adapter.h',
+ 'rte_event_dma_adapter.h',
'rte_event_eth_rx_adapter.h',
'rte_event_eth_tx_adapter.h',
'rte_event_ring.h',
diff --git a/lib/eventdev/rte_event_dma_adapter.h b/lib/eventdev/rte_event_dma_adapter.h
new file mode 100644
index 0000000000..731439a4fe
--- /dev/null
+++ b/lib/eventdev/rte_event_dma_adapter.h
@@ -0,0 +1,582 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright (c) 2023 Marvell.
+ */
+
+#ifndef RTE_EVENT_DMA_ADAPTER
+#define RTE_EVENT_DMA_ADAPTER
+
+/**
+ * @file rte_event_dma_adapter.h
+ *
+ * @warning
+ * @b EXPERIMENTAL:
+ * All functions in this file may be changed or removed without prior notice.
+ *
+ * DMA Event Adapter API.
+ *
+ * Eventdev library provides adapters to bridge between various components for providing new
+ * event source. The event DMA adapter is one of those adapters which is intended to bridge
+ * between event devices and DMA devices.
+ *
+ * The DMA adapter adds support to enqueue / dequeue DMA operations to / from event device. The
+ * packet flow between DMA device and the event device can be accomplished using both SW and HW
+ * based transfer mechanisms. The adapter uses an EAL service core function for SW based packet
+ * transfer and uses the eventdev PMD functions to configure HW based packet transfer between the
+ * DMA device and the event device.
+ *
+ * The application can choose to submit a DMA operation directly to an DMA device or send it to the
+ * DMA adapter via eventdev based on RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP_FWD capability. The
+ * first mode is known as the event new (RTE_EVENT_DMA_ADAPTER_OP_NEW) mode and the second as the
+ * event forward (RTE_EVENT_DMA_ADAPTER_OP_FORWARD) mode. The choice of mode can be specified while
+ * creating the adapter. In the former mode, it is an application responsibility to enable ingress
+ * packet ordering. In the latter mode, it is the adapter responsibility to enable the ingress
+ * packet ordering.
+ *
+ *
+ * Working model of RTE_EVENT_DMA_ADAPTER_OP_NEW mode:
+ *
+ * +--------------+ +--------------+
+ * | | | DMA stage |
+ * | Application |---[2]-->| + enqueue to |
+ * | | | dmadev |
+ * +--------------+ +--------------+
+ * ^ ^ |
+ * | | [3]
+ * [6] [1] |
+ * | | |
+ * +--------------+ |
+ * | | |
+ * | Event device | |
+ * | | |
+ * +--------------+ |
+ * ^ |
+ * | |
+ * [5] |
+ * | v
+ * +--------------+ +--------------+
+ * | | | |
+ * | DMA adapter |<--[4]---| dmadev |
+ * | | | |
+ * +--------------+ +--------------+
+ *
+ *
+ * [1] Application dequeues events from the previous stage.
+ * [2] Application prepares the DMA operations.
+ * [3] DMA operations are submitted to dmadev by application.
+ * [4] DMA adapter dequeues DMA completions from dmadev.
+ * [5] DMA adapter enqueues events to the eventdev.
+ * [6] Application dequeues from eventdev for further processing.
+ *
+ * In the RTE_EVENT_DMA_ADAPTER_OP_NEW mode, application submits DMA operations directly to DMA
+ * device. The DMA adapter then dequeues DMA completions from DMA device and enqueue events to the
+ * event device. This mode does not ensure ingress ordering, if the application directly enqueues
+ * to dmadev without going through DMA / atomic stage i.e. removing item [1] and [2].
+ *
+ * Events dequeued from the adapter will be treated as new events. In this mode, application needs
+ * to specify event information (response information) which is needed to enqueue an event after the
+ * DMA operation is completed.
+ *
+ *
+ * Working model of RTE_EVENT_DMA_ADAPTER_OP_FORWARD mode:
+ *
+ * +--------------+ +--------------+
+ * --[1]-->| |---[2]-->| Application |
+ * | Event device | | in |
+ * <--[8]--| |<--[3]---| Ordered stage|
+ * +--------------+ +--------------+
+ * ^ |
+ * | [4]
+ * [7] |
+ * | v
+ * +----------------+ +--------------+
+ * | |--[5]->| |
+ * | DMA adapter | | dmadev |
+ * | |<-[6]--| |
+ * +----------------+ +--------------+
+ *
+ *
+ * [1] Events from the previous stage.
+ * [2] Application in ordered stage dequeues events from eventdev.
+ * [3] Application enqueues DMA operations as events to eventdev.
+ * [4] DMA adapter dequeues event from eventdev.
+ * [5] DMA adapter submits DMA operations to dmadev (Atomic stage).
+ * [6] DMA adapter dequeues DMA completions from dmadev
+ * [7] DMA adapter enqueues events to the eventdev
+ * [8] Events to the next stage
+ *
+ * In the event forward (RTE_EVENT_DMA_ADAPTER_OP_FORWARD) mode, if the HW supports the capability
+ * RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP_FWD, application can directly submit the DMA
+ * operations to the dmadev. If not, application retrieves the event port of the DMA adapter
+ * through the API, rte_event_DMA_adapter_event_port_get(). Then, links its event queue to this
+ * port and starts enqueuing DMA operations as events to the eventdev. The adapter then dequeues
+ * the events and submits the DMA operations to the dmadev. After the DMA completions, the adapter
+ * enqueues events to the event device.
+ *
+ * Application can use this mode, when ingress packet ordering is needed. Events dequeued from the
+ * adapter will be treated as forwarded events. In this mode, the application needs to specify the
+ * dmadev ID and queue pair ID (request information) needed to enqueue an DMA operation in addition
+ * to the event information (response information) needed to enqueue an event after the DMA
+ * operation has completed.
+ *
+ * The event DMA adapter provides common APIs to configure the packet flow from the DMA device to
+ * event devices for both SW and HW based transfers. The DMA event adapter's functions are:
+ *
+ * - rte_event_dma_adapter_create_ext()
+ * - rte_event_dma_adapter_create()
+ * - rte_event_dma_adapter_free()
+ * - rte_event_dma_adapter_vchan_add()
+ * - rte_event_dma_adapter_vchan_del()
+ * - rte_event_dma_adapter_start()
+ * - rte_event_dma_adapter_stop()
+ * - rte_event_dma_adapter_stats_get()
+ * - rte_event_dma_adapter_stats_reset()
+ *
+ * The application creates an instance using rte_event_dma_adapter_create() or
+ * rte_event_dma_adapter_create_ext().
+ *
+ * dmadev queue pair addition / deletion is done using the rte_event_dma_adapter_vchan_add() /
+ * rte_event_dma_adapter_vchan_del() APIs. If HW supports the capability
+ * RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_VCHAN_EV_BIND, event information must be passed to the add
+ * API.
+ *
+ */
+
+#include <stdint.h>
+
+#include <rte_eventdev.h>
+#include <rte_common.h>
+#include <rte_dmadev_pmd.h>
+
+#ifdef __cplusplus
+extern "C" {
+#endif
+
+/**
+ * A structure used to hold event based DMA operation entry. All the information
+ * required for a DMA transfer shall be be populated in "struct rte_event_dma_adapter_op"
+ * instance.
+ */
+struct rte_event_dma_adapter_op {
+ struct rte_dma_sge *src_seg;
+ /**< Source segments. */
+ struct rte_dma_sge *dst_seg;
+ /**< Destination segments. */
+ uint16_t nb_src;
+ /**< Number of source segments. */
+ uint16_t nb_dst;
+ /**< Number of destination segments. */
+ uint64_t flags;
+ /**< Flags related to the operation.
+ * @see RTE_DMA_OP_FLAG_*
+ */
+ int16_t dma_dev_id;
+ /**< DMA device ID to be used */
+ uint16_t vchan;
+ /**< DMA vchan ID to be used */
+ struct rte_mempool *op_mp;
+ /**< Mempool from which op is allocated. */
+};
+
+/**
+ * DMA event adapter mode
+ */
+enum rte_event_dma_adapter_mode {
+ RTE_EVENT_DMA_ADAPTER_OP_NEW,
+ /**< Start the DMA adapter in event new mode.
+ * @see RTE_EVENT_OP_NEW.
+ *
+ * Application submits DMA operations to the dmadev. Adapter only dequeues the DMA
+ * completions from dmadev and enqueue events to the eventdev.
+ */
+
+ RTE_EVENT_DMA_ADAPTER_OP_FORWARD,
+ /**< Start the DMA adapter in event forward mode.
+ * @see RTE_EVENT_OP_FORWARD.
+ *
+ * Application submits DMA requests as events to the DMA adapter or DMA device based on
+ * RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP_FWD capability. DMA completions are enqueued
+ * back to the eventdev by DMA adapter.
+ */
+};
+
+/**
+ * Adapter configuration structure that the adapter configuration callback function is expected to
+ * fill out.
+ *
+ * @see rte_event_dma_adapter_conf_cb
+ */
+struct rte_event_dma_adapter_conf {
+ uint8_t event_port_id;
+ /** < Event port identifier, the adapter enqueues events to this port and dequeues DMA
+ * request events in RTE_EVENT_DMA_ADAPTER_OP_FORWARD mode.
+ */
+
+ uint32_t max_nb;
+ /**< The adapter can return early if it has processed at least max_nb DMA ops. This isn't
+ * treated as a requirement; batching may cause the adapter to process more than max_nb DMA
+ * ops.
+ */
+};
+
+/**
+ * Adapter runtime configuration parameters
+ */
+struct rte_event_dma_adapter_runtime_params {
+ uint32_t max_nb;
+ /**< The adapter can return early if it has processed at least max_nb DMA ops. This isn't
+ * treated as a requirement; batching may cause the adapter to process more than max_nb DMA
+ * ops.
+ *
+ * Callback function passed to rte_event_dma_adapter_create_ext() configures the adapter
+ * with default value of max_nb.
+ * rte_event_dma_adapter_runtime_params_set() allows to re-configure max_nb during runtime
+ * (after adding at least one queue pair)
+ *
+ * This is valid for the devices without RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP_FWD or
+ * RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP_NEW capability.
+ */
+
+ uint32_t rsvd[15];
+ /**< Reserved fields for future expansion */
+};
+
+/**
+ * Function type used for adapter configuration callback. The callback is used to fill in members of
+ * the struct rte_event_dma_adapter_conf, this callback is invoked when creating a SW service for
+ * packet transfer from dmadev vchan to the event device. The SW service is created within the
+ * function, rte_event_dma_adapter_vchan_add(), if SW based packet transfers from dmadev vchan
+ * to the event device are required.
+ *
+ * @param id
+ * Adapter identifier.
+ * @param evdev_id
+ * Event device identifier.
+ * @param conf
+ * Structure that needs to be populated by this callback.
+ * @param arg
+ * Argument to the callback. This is the same as the conf_arg passed to the
+ * rte_event_dma_adapter_create_ext().
+ */
+typedef int (*rte_event_dma_adapter_conf_cb)(uint8_t id, uint8_t evdev_id,
+ struct rte_event_dma_adapter_conf *conf, void *arg);
+
+/**
+ * A structure used to retrieve statistics for an event DMA adapter instance.
+ */
+struct rte_event_dma_adapter_stats {
+ uint64_t event_poll_count;
+ /**< Event port poll count */
+
+ uint64_t event_deq_count;
+ /**< Event dequeue count */
+
+ uint64_t dma_enq_count;
+ /**< dmadev enqueue count */
+
+ uint64_t dma_enq_fail_count;
+ /**< dmadev enqueue failed count */
+
+ uint64_t dma_deq_count;
+ /**< dmadev dequeue count */
+
+ uint64_t event_enq_count;
+ /**< Event enqueue count */
+
+ uint64_t event_enq_retry_count;
+ /**< Event enqueue retry count */
+
+ uint64_t event_enq_fail_count;
+ /**< Event enqueue fail count */
+};
+
+/**
+ * Create a new event DMA adapter with the specified identifier.
+ *
+ * @param id
+ * Adapter identifier.
+ * @param evdev_id
+ * Event device identifier.
+ * @param conf_cb
+ * Callback function that fills in members of a struct rte_event_dma_adapter_conf struct passed
+ * into it.
+ * @param mode
+ * Flag to indicate the mode of the adapter.
+ * @see rte_event_dma_adapter_mode
+ * @param conf_arg
+ * Argument that is passed to the conf_cb function.
+ *
+ * @return
+ * - 0: Success
+ * - <0: Error code on failure
+ */
+__rte_experimental
+int rte_event_dma_adapter_create_ext(uint8_t id, uint8_t evdev_id,
+ rte_event_dma_adapter_conf_cb conf_cb,
+ enum rte_event_dma_adapter_mode mode, void *conf_arg);
+
+/**
+ * Create a new event DMA adapter with the specified identifier. This function uses an internal
+ * configuration function that creates an event port. This default function reconfigures the event
+ * device with an additional event port and set up the event port using the port_config parameter
+ * passed into this function. In case the application needs more control in configuration of the
+ * service, it should use the rte_event_dma_adapter_create_ext() version.
+ *
+ * @param id
+ * Adapter identifier.
+ * @param evdev_id
+ * Event device identifier.
+ * @param port_config
+ * Argument of type *rte_event_port_conf* that is passed to the conf_cb function.
+ * @param mode
+ * Flag to indicate the mode of the adapter.
+ * @see rte_event_dma_adapter_mode
+ *
+ * @return
+ * - 0: Success
+ * - <0: Error code on failure
+ */
+__rte_experimental
+int rte_event_dma_adapter_create(uint8_t id, uint8_t evdev_id,
+ struct rte_event_port_conf *port_config,
+ enum rte_event_dma_adapter_mode mode);
+
+/**
+ * Free an event DMA adapter
+ *
+ * @param id
+ * Adapter identifier.
+ * @return
+ * - 0: Success
+ * - <0: Error code on failure, If the adapter still has queue pairs added to it, the function
+ * returns -EBUSY.
+ */
+__rte_experimental
+int rte_event_dma_adapter_free(uint8_t id);
+
+/**
+ * Retrieve the event port of an adapter.
+ *
+ * @param id
+ * Adapter identifier.
+ *
+ * @param [out] event_port_id
+ * Application links its event queue to this adapter port which is used in
+ * RTE_EVENT_DMA_ADAPTER_OP_FORWARD mode.
+ *
+ * @return
+ * - 0: Success
+ * - <0: Error code on failure.
+ */
+__rte_experimental
+int rte_event_dma_adapter_event_port_get(uint8_t id, uint8_t *event_port_id);
+
+/**
+ * Add a vchan to an event DMA adapter.
+ *
+ * @param id
+ * Adapter identifier.
+ * @param dmadev_id
+ * dmadev identifier.
+ * @param vchan
+ * DMA device vchan identifier. If vchan is set -1, adapter adds all the
+ * preconfigured vchan to the instance.
+ * @param event
+ * If HW supports dmadev vchan to event queue binding, application is expected to fill in
+ * event information, else it will be NULL.
+ * @see RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_VCHAN_EV_BIND
+ *
+ * @return
+ * - 0: Success, vchan added correctly.
+ * - <0: Error code on failure.
+ */
+__rte_experimental
+int rte_event_dma_adapter_vchan_add(uint8_t id, int16_t dmadev_id, uint16_t vchan,
+ const struct rte_event *event);
+
+/**
+ * Delete a vchan from an event DMA adapter.
+ *
+ * @param id
+ * Adapter identifier.
+ * @param dmadev_id
+ * DMA device identifier.
+ * @param vchan
+ * DMA device vchan identifier.
+ *
+ * @return
+ * - 0: Success, vchan deleted successfully.
+ * - <0: Error code on failure.
+ */
+__rte_experimental
+int rte_event_dma_adapter_vchan_del(uint8_t id, int16_t dmadev_id, uint16_t vchan);
+
+/**
+ * Retrieve the service ID of an adapter. If the adapter doesn't use a rte_service function, this
+ * function returns -ESRCH.
+ *
+ * @param id
+ * Adapter identifier.
+ * @param [out] service_id
+ * A pointer to a uint32_t, to be filled in with the service id.
+ *
+ * @return
+ * - 0: Success
+ * - <0: Error code on failure, if the adapter doesn't use a rte_service function, this function
+ * returns -ESRCH.
+ */
+__rte_experimental
+int rte_event_dma_adapter_service_id_get(uint8_t id, uint32_t *service_id);
+
+/**
+ * Start event DMA adapter
+ *
+ * @param id
+ * Adapter identifier.
+ *
+ * @return
+ * - 0: Success, adapter started successfully.
+ * - <0: Error code on failure.
+ *
+ * @note The eventdev and dmadev to which the event_dma_adapter is connected should be started
+ * before calling rte_event_dma_adapter_start().
+ */
+__rte_experimental
+int rte_event_dma_adapter_start(uint8_t id);
+
+/**
+ * Stop event DMA adapter
+ *
+ * @param id
+ * Adapter identifier.
+ *
+ * @return
+ * - 0: Success, adapter stopped successfully.
+ * - <0: Error code on failure.
+ */
+__rte_experimental
+int rte_event_dma_adapter_stop(uint8_t id);
+
+/**
+ * Initialize the adapter runtime configuration parameters
+ *
+ * @param params
+ * A pointer to structure of type struct rte_event_dma_adapter_runtime_params
+ *
+ * @return
+ * - 0: Success
+ * - <0: Error code on failure
+ */
+__rte_experimental
+int rte_event_dma_adapter_runtime_params_init(struct rte_event_dma_adapter_runtime_params *params);
+
+/**
+ * Set the adapter runtime configuration parameters
+ *
+ * @param id
+ * Adapter identifier
+ *
+ * @param params
+ * A pointer to structure of type struct rte_event_dma_adapter_runtime_params with configuration
+ * parameter values. The reserved fields of this structure must be initialized to zero and the valid
+ * fields need to be set appropriately. This struct can be initialized using
+ * rte_event_dma_adapter_runtime_params_init() API to default values or application may reset this
+ * struct and update required fields.
+ *
+ * @return
+ * - 0: Success
+ * - <0: Error code on failure
+ */
+__rte_experimental
+int rte_event_dma_adapter_runtime_params_set(uint8_t id,
+ struct rte_event_dma_adapter_runtime_params *params);
+
+/**
+ * Get the adapter runtime configuration parameters
+ *
+ * @param id
+ * Adapter identifier
+ *
+ * @param[out] params
+ * A pointer to structure of type struct rte_event_dma_adapter_runtime_params containing valid
+ * adapter parameters when return value is 0.
+ *
+ * @return
+ * - 0: Success
+ * - <0: Error code on failure
+ */
+__rte_experimental
+int rte_event_dma_adapter_runtime_params_get(uint8_t id,
+ struct rte_event_dma_adapter_runtime_params *params);
+
+/**
+ * Retrieve statistics for an adapter
+ *
+ * @param id
+ * Adapter identifier.
+ * @param [out] stats
+ * A pointer to structure used to retrieve statistics for an adapter.
+ *
+ * @return
+ * - 0: Success, retrieved successfully.
+ * - <0: Error code on failure.
+ */
+__rte_experimental
+int rte_event_dma_adapter_stats_get(uint8_t id, struct rte_event_dma_adapter_stats *stats);
+
+/**
+ * Reset statistics for an adapter.
+ *
+ * @param id
+ * Adapter identifier.
+ *
+ * @return
+ * - 0: Success, statistics reset successfully.
+ * - <0: Error code on failure.
+ */
+__rte_experimental
+int rte_event_dma_adapter_stats_reset(uint8_t id);
+
+/**
+ * Enqueue a burst of DMA operations as event objects supplied in *rte_event* structure on an event
+ * DMA adapter designated by its event *evdev_id* through the event port specified by *port_id*.
+ * This function is supported if the eventdev PMD has the
+ * #RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP_FWD capability flag set.
+ *
+ * The *nb_events* parameter is the number of event objects to enqueue that are supplied in the
+ * *ev* array of *rte_event* structure.
+ *
+ * The rte_event_dma_adapter_enqueue() function returns the number of event objects it actually
+ * enqueued. A return value equal to *nb_events* means that all event objects have been enqueued.
+ *
+ * @param evdev_id
+ * The identifier of the device.
+ * @param port_id
+ * The identifier of the event port.
+ * @param ev
+ * Points to an array of *nb_events* objects of type *rte_event* structure which contain the
+ * event object enqueue operations to be processed.
+ * @param nb_events
+ * The number of event objects to enqueue, typically number of
+ * rte_event_port_attr_get(...RTE_EVENT_PORT_ATTR_ENQ_DEPTH...) available for this port.
+ *
+ * @return
+ * The number of event objects actually enqueued on the event device. The return value can be
+ * less than the value of the *nb_events* parameter when the event devices queue is full or if
+ * invalid parameters are specified in a *rte_event*. If the return value is less than *nb_events*,
+ * the remaining events at the end of ev[] are not consumed and the caller has to take care of them,
+ * and rte_errno is set accordingly. Possible errno values include:
+ *
+ * - EINVAL: The port ID is invalid, device ID is invalid, an event's queue ID is invalid, or an
+ * event's sched type doesn't match the capabilities of the destination queue.
+ * - ENOSPC: The event port was backpressured and unable to enqueue one or more events. This
+ * error code is only applicable to closed systems.
+ */
+__rte_experimental
+uint16_t rte_event_dma_adapter_enqueue(uint8_t evdev_id, uint8_t port_id, struct rte_event ev[],
+ uint16_t nb_events);
+
+#ifdef __cplusplus
+}
+#endif
+
+#endif /* RTE_EVENT_DMA_ADAPTER */
diff --git a/lib/eventdev/rte_eventdev.h b/lib/eventdev/rte_eventdev.h
index 2ba8a7b090..026f7c367d 100644
--- a/lib/eventdev/rte_eventdev.h
+++ b/lib/eventdev/rte_eventdev.h
@@ -1197,6 +1197,8 @@ struct rte_event_vector {
*/
#define RTE_EVENT_TYPE_ETH_RX_ADAPTER 0x4
/**< The event generated from event eth Rx adapter */
+#define RTE_EVENT_TYPE_DMADEV 0x5
+/**< The event generated from dma subsystem */
#define RTE_EVENT_TYPE_VECTOR 0x8
/**< Indicates that event is a vector.
* All vector event types should be a logical OR of EVENT_TYPE_VECTOR.
@@ -1462,6 +1464,47 @@ int
rte_event_crypto_adapter_caps_get(uint8_t dev_id, uint8_t cdev_id,
uint32_t *caps);
+/* DMA adapter capability bitmap flag */
+#define RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP_NEW 0x1
+/**< Flag indicates HW is capable of generating events in
+ * RTE_EVENT_OP_NEW enqueue operation. DMADEV will send
+ * packets to the event device as new events using an
+ * internal event port.
+ */
+
+#define RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP_FWD 0x2
+/**< Flag indicates HW is capable of generating events in
+ * RTE_EVENT_OP_FORWARD enqueue operation. DMADEV will send
+ * packets to the event device as forwarded event using an
+ * internal event port.
+ */
+
+#define RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_VCHAN_EV_BIND 0x4
+/**< Flag indicates HW is capable of mapping DMA vchan to event queue. */
+
+/**
+ * Retrieve the event device's DMA adapter capabilities for the
+ * specified dmadev device
+ *
+ * @param dev_id
+ * The identifier of the device.
+ *
+ * @param dmadev_id
+ * The identifier of the dmadev device.
+ *
+ * @param[out] caps
+ * A pointer to memory filled with event adapter capabilities.
+ * It is expected to be pre-allocated & initialized by caller.
+ *
+ * @return
+ * - 0: Success, driver provides event adapter capabilities for the
+ * dmadev device.
+ * - <0: Error code returned by the driver function.
+ *
+ */
+int
+rte_event_dma_adapter_caps_get(uint8_t dev_id, int16_t dmadev_id, uint32_t *caps);
+
/* Ethdev Tx adapter capability bitmap flags */
#define RTE_EVENT_ETH_TX_ADAPTER_CAP_INTERNAL_PORT 0x1
/**< This flag is sent when the PMD supports a packet transmit callback
diff --git a/lib/eventdev/rte_eventdev_core.h b/lib/eventdev/rte_eventdev_core.h
index c27a52ccc0..83e8736c71 100644
--- a/lib/eventdev/rte_eventdev_core.h
+++ b/lib/eventdev/rte_eventdev_core.h
@@ -42,6 +42,10 @@ typedef uint16_t (*event_crypto_adapter_enqueue_t)(void *port,
uint16_t nb_events);
/**< @internal Enqueue burst of events on crypto adapter */
+typedef uint16_t (*event_dma_adapter_enqueue_t)(void *port, struct rte_event ev[],
+ uint16_t nb_events);
+/**< @internal Enqueue burst of events on DMA adapter */
+
struct rte_event_fp_ops {
void **data;
/**< points to array of internal port data pointers */
@@ -65,7 +69,9 @@ struct rte_event_fp_ops {
/**< PMD Tx adapter enqueue same destination function. */
event_crypto_adapter_enqueue_t ca_enqueue;
/**< PMD Crypto adapter enqueue function. */
- uintptr_t reserved[5];
+ event_dma_adapter_enqueue_t dma_enqueue;
+ /**< PMD DMA adapter enqueue function. */
+ uintptr_t reserved[4];
} __rte_cache_aligned;
extern struct rte_event_fp_ops rte_event_fp_ops[RTE_EVENT_MAX_DEVS];
diff --git a/lib/eventdev/version.map b/lib/eventdev/version.map
index 7ce09a87bb..dc1dcc8274 100644
--- a/lib/eventdev/version.map
+++ b/lib/eventdev/version.map
@@ -133,6 +133,21 @@ EXPERIMENTAL {
rte_event_timer_remaining_ticks_get;
# added in 23.11
+ rte_event_dma_adapter_create_ext;
+ rte_event_dma_adapter_create;
+ rte_event_dma_adapter_event_port_get;
+ rte_event_dma_adapter_enqueue;
+ rte_event_dma_adapter_free;
+ rte_event_dma_adapter_runtime_params_init;
+ rte_event_dma_adapter_runtime_params_set;
+ rte_event_dma_adapter_runtime_params_get;
+ rte_event_dma_adapter_service_id_get;
+ rte_event_dma_adapter_start;
+ rte_event_dma_adapter_stop;
+ rte_event_dma_adapter_stats_get;
+ rte_event_dma_adapter_stats_reset;
+ rte_event_dma_adapter_vchan_add;
+ rte_event_dma_adapter_vchan_del;
rte_event_eth_rx_adapter_create_ext_with_params;
};
--
2.25.1
^ permalink raw reply [flat|nested] 110+ messages in thread
* [PATCH v2 02/12] eventdev: api to get DMA adapter capabilities
2023-09-22 20:13 ` [PATCH v2 00/12] event DMA adapter library support Amit Prakash Shukla
2023-09-22 20:13 ` [PATCH v2 01/12] eventdev: introduce event DMA adapter library Amit Prakash Shukla
@ 2023-09-22 20:13 ` Amit Prakash Shukla
2023-09-22 20:13 ` [PATCH v2 03/12] eventdev: add DMA adapter API to create and free Amit Prakash Shukla
` (11 subsequent siblings)
13 siblings, 0 replies; 110+ messages in thread
From: Amit Prakash Shukla @ 2023-09-22 20:13 UTC (permalink / raw)
To: Jerin Jacob
Cc: dev, fengchengwen, kevin.laatz, bruce.richardson, conor.walsh,
vattunuru, g.singh, sachin.saxena, hemant.agrawal, cheng1.jiang,
ndabilpuram, anoobj, mb, Amit Prakash Shukla
Added a new eventdev API rte_event_dma_adapter_caps_get(), to get
DMA adapter capabilities supported by the driver.
Signed-off-by: Amit Prakash Shukla <amitprakashs@marvell.com>
---
lib/eventdev/meson.build | 2 +-
lib/eventdev/rte_eventdev.c | 23 +++++++++++++++++++++++
lib/eventdev/rte_eventdev.h | 2 +-
lib/meson.build | 2 +-
4 files changed, 26 insertions(+), 3 deletions(-)
diff --git a/lib/eventdev/meson.build b/lib/eventdev/meson.build
index 21347f7c4c..b46bbbc9aa 100644
--- a/lib/eventdev/meson.build
+++ b/lib/eventdev/meson.build
@@ -43,5 +43,5 @@ driver_sdk_headers += files(
'event_timer_adapter_pmd.h',
)
-deps += ['ring', 'ethdev', 'hash', 'mempool', 'mbuf', 'timer', 'cryptodev']
+deps += ['ring', 'ethdev', 'hash', 'mempool', 'mbuf', 'timer', 'cryptodev', 'dmadev']
deps += ['telemetry']
diff --git a/lib/eventdev/rte_eventdev.c b/lib/eventdev/rte_eventdev.c
index 6ab4524332..60509c6efb 100644
--- a/lib/eventdev/rte_eventdev.c
+++ b/lib/eventdev/rte_eventdev.c
@@ -20,6 +20,7 @@
#include <rte_errno.h>
#include <ethdev_driver.h>
#include <rte_cryptodev.h>
+#include <rte_dmadev.h>
#include <cryptodev_pmd.h>
#include <rte_telemetry.h>
@@ -224,6 +225,28 @@ rte_event_eth_tx_adapter_caps_get(uint8_t dev_id, uint16_t eth_port_id,
: 0;
}
+int
+rte_event_dma_adapter_caps_get(uint8_t dev_id, uint8_t dma_dev_id, uint32_t *caps)
+{
+ struct rte_eventdev *dev;
+
+ RTE_EVENTDEV_VALID_DEVID_OR_ERR_RET(dev_id, -EINVAL);
+ if (!rte_dma_is_valid(dma_dev_id))
+ return -EINVAL;
+
+ dev = &rte_eventdevs[dev_id];
+
+ if (caps == NULL)
+ return -EINVAL;
+
+ *caps = 0;
+
+ if (dev->dev_ops->dma_adapter_caps_get)
+ return (*dev->dev_ops->dma_adapter_caps_get)(dev, dma_dev_id, caps);
+
+ return 0;
+}
+
static inline int
event_dev_queue_config(struct rte_eventdev *dev, uint8_t nb_queues)
{
diff --git a/lib/eventdev/rte_eventdev.h b/lib/eventdev/rte_eventdev.h
index 026f7c367d..cef90aebf6 100644
--- a/lib/eventdev/rte_eventdev.h
+++ b/lib/eventdev/rte_eventdev.h
@@ -1503,7 +1503,7 @@ rte_event_crypto_adapter_caps_get(uint8_t dev_id, uint8_t cdev_id,
*
*/
int
-rte_event_dma_adapter_caps_get(uint8_t dev_id, int16_t dmadev_id, uint32_t *caps);
+rte_event_dma_adapter_caps_get(uint8_t dev_id, uint8_t dmadev_id, uint32_t *caps);
/* Ethdev Tx adapter capability bitmap flags */
#define RTE_EVENT_ETH_TX_ADAPTER_CAP_INTERNAL_PORT 0x1
diff --git a/lib/meson.build b/lib/meson.build
index 53155be8e9..f3191f10b6 100644
--- a/lib/meson.build
+++ b/lib/meson.build
@@ -33,6 +33,7 @@ libraries = [
'compressdev',
'cryptodev',
'distributor',
+ 'dmadev',
'efd',
'eventdev',
'gpudev',
@@ -48,7 +49,6 @@ libraries = [
'rawdev',
'regexdev',
'mldev',
- 'dmadev',
'rib',
'reorder',
'sched',
--
2.25.1
^ permalink raw reply [flat|nested] 110+ messages in thread
* [PATCH v2 03/12] eventdev: add DMA adapter API to create and free
2023-09-22 20:13 ` [PATCH v2 00/12] event DMA adapter library support Amit Prakash Shukla
2023-09-22 20:13 ` [PATCH v2 01/12] eventdev: introduce event DMA adapter library Amit Prakash Shukla
2023-09-22 20:13 ` [PATCH v2 02/12] eventdev: api to get DMA adapter capabilities Amit Prakash Shukla
@ 2023-09-22 20:13 ` Amit Prakash Shukla
2023-09-22 20:13 ` [PATCH v2 04/12] eventdev: api support for vchan add and delete Amit Prakash Shukla
` (10 subsequent siblings)
13 siblings, 0 replies; 110+ messages in thread
From: Amit Prakash Shukla @ 2023-09-22 20:13 UTC (permalink / raw)
To: Bruce Richardson, Jerin Jacob, Amit Prakash Shukla
Cc: dev, fengchengwen, kevin.laatz, conor.walsh, vattunuru, g.singh,
sachin.saxena, hemant.agrawal, cheng1.jiang, ndabilpuram, anoobj,
mb
This patch adds API support to create and free DMA adapter.
Signed-off-by: Amit Prakash Shukla <amitprakashs@marvell.com>
---
config/rte_config.h | 1 +
lib/eventdev/meson.build | 1 +
lib/eventdev/rte_event_dma_adapter.c | 335 +++++++++++++++++++++++++++
3 files changed, 337 insertions(+)
create mode 100644 lib/eventdev/rte_event_dma_adapter.c
diff --git a/config/rte_config.h b/config/rte_config.h
index 400e44e3cf..401727703f 100644
--- a/config/rte_config.h
+++ b/config/rte_config.h
@@ -77,6 +77,7 @@
#define RTE_EVENT_ETH_INTR_RING_SIZE 1024
#define RTE_EVENT_CRYPTO_ADAPTER_MAX_INSTANCE 32
#define RTE_EVENT_ETH_TX_ADAPTER_MAX_INSTANCE 32
+#define RTE_EVENT_DMA_ADAPTER_MAX_INSTANCE 32
/* rawdev defines */
#define RTE_RAWDEV_MAX_DEVS 64
diff --git a/lib/eventdev/meson.build b/lib/eventdev/meson.build
index b46bbbc9aa..250abcb154 100644
--- a/lib/eventdev/meson.build
+++ b/lib/eventdev/meson.build
@@ -17,6 +17,7 @@ sources = files(
'eventdev_private.c',
'eventdev_trace_points.c',
'rte_event_crypto_adapter.c',
+ 'rte_event_dma_adapter.c',
'rte_event_eth_rx_adapter.c',
'rte_event_eth_tx_adapter.c',
'rte_event_ring.c',
diff --git a/lib/eventdev/rte_event_dma_adapter.c b/lib/eventdev/rte_event_dma_adapter.c
new file mode 100644
index 0000000000..c7ffba1b47
--- /dev/null
+++ b/lib/eventdev/rte_event_dma_adapter.c
@@ -0,0 +1,335 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright (c) 2023 Marvell.
+ */
+
+#include "rte_eventdev.h"
+#include "eventdev_pmd.h"
+#include "rte_event_dma_adapter.h"
+
+#define DMA_BATCH_SIZE 32
+#define DMA_DEFAULT_MAX_NB 128
+#define DMA_ADAPTER_NAME_LEN 32
+#define DMA_ADAPTER_BUFFER_SIZE 1024
+
+#define DMA_ADAPTER_OPS_BUFFER_SIZE (DMA_BATCH_SIZE + DMA_BATCH_SIZE)
+
+#define DMA_ADAPTER_ARRAY "event_dma_adapter_array"
+
+/* Macros to check for valid adapter */
+#define EVENT_DMA_ADAPTER_ID_VALID_OR_ERR_RET(id, retval) \
+ do { \
+ if (!edma_adapter_valid_id(id)) { \
+ RTE_EDEV_LOG_ERR("Invalid DMA adapter id = %d\n", id); \
+ return retval; \
+ } \
+ } while (0)
+
+/* DMA ops circular buffer */
+struct dma_ops_circular_buffer {
+ /* Index of head element */
+ uint16_t head;
+
+ /* Index of tail element */
+ uint16_t tail;
+
+ /* Number of elements in buffer */
+ uint16_t count;
+
+ /* Size of circular buffer */
+ uint16_t size;
+
+ /* Pointer to hold rte_event_dma_adapter_op for processing */
+ struct rte_event_dma_adapter_op **op_buffer;
+} __rte_cache_aligned;
+
+/* DMA device information */
+struct dma_device_info {
+ /* Number of vchans configured for a DMA device. */
+ uint16_t num_dma_dev_vchan;
+} __rte_cache_aligned;
+
+struct event_dma_adapter {
+ /* Event device identifier */
+ uint8_t eventdev_id;
+
+ /* Event port identifier */
+ uint8_t event_port_id;
+
+ /* Adapter mode */
+ enum rte_event_dma_adapter_mode mode;
+
+ /* Memory allocation name */
+ char mem_name[DMA_ADAPTER_NAME_LEN];
+
+ /* Socket identifier cached from eventdev */
+ int socket_id;
+
+ /* Lock to serialize config updates with service function */
+ rte_spinlock_t lock;
+
+ /* DMA device structure array */
+ struct dma_device_info *dma_devs;
+
+ /* Circular buffer for processing DMA ops to eventdev */
+ struct dma_ops_circular_buffer ebuf;
+
+ /* Configuration callback for rte_service configuration */
+ rte_event_dma_adapter_conf_cb conf_cb;
+
+ /* Configuration callback argument */
+ void *conf_arg;
+
+ /* Set if default_cb is being used */
+ int default_cb_arg;
+} __rte_cache_aligned;
+
+static struct event_dma_adapter **event_dma_adapter;
+
+static inline int
+edma_adapter_valid_id(uint8_t id)
+{
+ return id < RTE_EVENT_DMA_ADAPTER_MAX_INSTANCE;
+}
+
+static inline struct event_dma_adapter *
+edma_id_to_adapter(uint8_t id)
+{
+ return event_dma_adapter ? event_dma_adapter[id] : NULL;
+}
+
+static int
+edma_array_init(void)
+{
+ const struct rte_memzone *mz;
+ uint32_t sz;
+
+ mz = rte_memzone_lookup(DMA_ADAPTER_ARRAY);
+ if (mz == NULL) {
+ sz = sizeof(struct event_dma_adapter *) * RTE_EVENT_DMA_ADAPTER_MAX_INSTANCE;
+ sz = RTE_ALIGN(sz, RTE_CACHE_LINE_SIZE);
+
+ mz = rte_memzone_reserve_aligned(DMA_ADAPTER_ARRAY, sz, rte_socket_id(), 0,
+ RTE_CACHE_LINE_SIZE);
+ if (mz == NULL) {
+ RTE_EDEV_LOG_ERR("Failed to reserve memzone : %s, err = %d",
+ DMA_ADAPTER_ARRAY, rte_errno);
+ return -rte_errno;
+ }
+ }
+
+ event_dma_adapter = mz->addr;
+
+ return 0;
+}
+
+static inline int
+edma_circular_buffer_init(const char *name, struct dma_ops_circular_buffer *buf, uint16_t sz)
+{
+ buf->op_buffer = rte_zmalloc(name, sizeof(struct rte_event_dma_adapter_op *) * sz, 0);
+ if (buf->op_buffer == NULL)
+ return -ENOMEM;
+
+ buf->size = sz;
+
+ return 0;
+}
+
+static inline void
+edma_circular_buffer_free(struct dma_ops_circular_buffer *buf)
+{
+ rte_free(buf->op_buffer);
+}
+
+static int
+edma_default_config_cb(uint8_t id, uint8_t evdev_id, struct rte_event_dma_adapter_conf *conf,
+ void *arg)
+{
+ struct rte_event_port_conf *port_conf;
+ struct rte_event_dev_config dev_conf;
+ struct event_dma_adapter *adapter;
+ struct rte_eventdev *dev;
+ uint8_t port_id;
+ int started;
+ int ret;
+
+ adapter = edma_id_to_adapter(id);
+ if (adapter == NULL)
+ return -EINVAL;
+
+ dev = &rte_eventdevs[adapter->eventdev_id];
+ dev_conf = dev->data->dev_conf;
+
+ started = dev->data->dev_started;
+ if (started)
+ rte_event_dev_stop(evdev_id);
+
+ port_id = dev_conf.nb_event_ports;
+ dev_conf.nb_event_ports += 1;
+
+ port_conf = arg;
+ if (port_conf->event_port_cfg & RTE_EVENT_PORT_CFG_SINGLE_LINK)
+ dev_conf.nb_single_link_event_port_queues += 1;
+
+ ret = rte_event_dev_configure(evdev_id, &dev_conf);
+ if (ret) {
+ RTE_EDEV_LOG_ERR("Failed to configure event dev %u\n", evdev_id);
+ if (started) {
+ if (rte_event_dev_start(evdev_id))
+ return -EIO;
+ }
+ return ret;
+ }
+
+ ret = rte_event_port_setup(evdev_id, port_id, port_conf);
+ if (ret) {
+ RTE_EDEV_LOG_ERR("Failed to setup event port %u\n", port_id);
+ return ret;
+ }
+
+ conf->event_port_id = port_id;
+ conf->max_nb = DMA_DEFAULT_MAX_NB;
+ if (started)
+ ret = rte_event_dev_start(evdev_id);
+
+ adapter->default_cb_arg = 1;
+ adapter->event_port_id = conf->event_port_id;
+
+ return ret;
+}
+
+int
+rte_event_dma_adapter_create_ext(uint8_t id, uint8_t evdev_id,
+ rte_event_dma_adapter_conf_cb conf_cb,
+ enum rte_event_dma_adapter_mode mode, void *conf_arg)
+{
+ struct rte_event_dev_info dev_info;
+ struct event_dma_adapter *adapter;
+ char name[DMA_ADAPTER_NAME_LEN];
+ struct rte_dma_info info;
+ uint16_t num_dma_dev;
+ int socket_id;
+ uint8_t i;
+ int ret;
+
+ EVENT_DMA_ADAPTER_ID_VALID_OR_ERR_RET(id, -EINVAL);
+ RTE_EVENTDEV_VALID_DEVID_OR_ERR_RET(evdev_id, -EINVAL);
+
+ if (conf_cb == NULL)
+ return -EINVAL;
+
+ if (event_dma_adapter == NULL) {
+ ret = edma_array_init();
+ if (ret)
+ return ret;
+ }
+
+ adapter = edma_id_to_adapter(id);
+ if (adapter != NULL) {
+ RTE_EDEV_LOG_ERR("ML adapter ID %d already exists!", id);
+ return -EEXIST;
+ }
+
+ socket_id = rte_event_dev_socket_id(evdev_id);
+ snprintf(name, DMA_ADAPTER_NAME_LEN, "rte_event_dma_adapter_%d", id);
+ adapter = rte_zmalloc_socket(name, sizeof(struct event_dma_adapter), RTE_CACHE_LINE_SIZE,
+ socket_id);
+ if (adapter == NULL) {
+ RTE_EDEV_LOG_ERR("Failed to get mem for event ML adapter!");
+ return -ENOMEM;
+ }
+
+ if (edma_circular_buffer_init("edma_circular_buffer", &adapter->ebuf,
+ DMA_ADAPTER_BUFFER_SIZE)) {
+ RTE_EDEV_LOG_ERR("Failed to get memory for event adapter circular buffer");
+ rte_free(adapter);
+ return -ENOMEM;
+ }
+
+ ret = rte_event_dev_info_get(evdev_id, &dev_info);
+ if (ret < 0) {
+ RTE_EDEV_LOG_ERR("Failed to get info for eventdev %d: %s", evdev_id,
+ dev_info.driver_name);
+ edma_circular_buffer_free(&adapter->ebuf);
+ rte_free(adapter);
+ return ret;
+ }
+
+ num_dma_dev = rte_dma_count_avail();
+
+ adapter->eventdev_id = evdev_id;
+ adapter->mode = mode;
+ strcpy(adapter->mem_name, name);
+ adapter->socket_id = socket_id;
+ adapter->conf_cb = conf_cb;
+ adapter->conf_arg = conf_arg;
+ adapter->dma_devs = rte_zmalloc_socket(adapter->mem_name,
+ num_dma_dev * sizeof(struct dma_device_info), 0,
+ socket_id);
+ if (adapter->dma_devs == NULL) {
+ RTE_EDEV_LOG_ERR("Failed to get memory for DMA devices\n");
+ edma_circular_buffer_free(&adapter->ebuf);
+ rte_free(adapter);
+ return -ENOMEM;
+ }
+
+ rte_spinlock_init(&adapter->lock);
+ for (i = 0; i < num_dma_dev; i++) {
+ ret = rte_dma_info_get(i, &info);
+ if (ret) {
+ RTE_EDEV_LOG_ERR("Failed to get dma device info\n");
+ edma_circular_buffer_free(&adapter->ebuf);
+ rte_free(adapter);
+ return ret;
+ }
+
+ adapter->dma_devs[i].num_dma_dev_vchan = info.nb_vchans;
+ }
+
+ event_dma_adapter[id] = adapter;
+
+ return 0;
+}
+
+int
+rte_event_dma_adapter_create(uint8_t id, uint8_t evdev_id, struct rte_event_port_conf *port_config,
+ enum rte_event_dma_adapter_mode mode)
+{
+ struct rte_event_port_conf *pc;
+ int ret;
+
+ if (port_config == NULL)
+ return -EINVAL;
+
+ EVENT_DMA_ADAPTER_ID_VALID_OR_ERR_RET(id, -EINVAL);
+
+ pc = rte_malloc(NULL, sizeof(struct rte_event_port_conf), 0);
+ if (pc == NULL)
+ return -ENOMEM;
+
+ rte_memcpy(pc, port_config, sizeof(struct rte_event_port_conf));
+ ret = rte_event_dma_adapter_create_ext(id, evdev_id, edma_default_config_cb, mode, pc);
+ if (ret != 0)
+ rte_free(pc);
+
+ return ret;
+}
+
+int
+rte_event_dma_adapter_free(uint8_t id)
+{
+ struct event_dma_adapter *adapter;
+
+ EVENT_DMA_ADAPTER_ID_VALID_OR_ERR_RET(id, -EINVAL);
+
+ adapter = edma_id_to_adapter(id);
+ if (adapter == NULL)
+ return -EINVAL;
+
+ rte_free(adapter->conf_arg);
+ rte_free(adapter->dma_devs);
+ edma_circular_buffer_free(&adapter->ebuf);
+ rte_free(adapter);
+ event_dma_adapter[id] = NULL;
+
+ return 0;
+}
--
2.25.1
^ permalink raw reply [flat|nested] 110+ messages in thread
* [PATCH v2 04/12] eventdev: api support for vchan add and delete
2023-09-22 20:13 ` [PATCH v2 00/12] event DMA adapter library support Amit Prakash Shukla
` (2 preceding siblings ...)
2023-09-22 20:13 ` [PATCH v2 03/12] eventdev: add DMA adapter API to create and free Amit Prakash Shukla
@ 2023-09-22 20:13 ` Amit Prakash Shukla
2023-09-22 20:13 ` [PATCH v2 05/12] eventdev: add support for service function Amit Prakash Shukla
` (9 subsequent siblings)
13 siblings, 0 replies; 110+ messages in thread
From: Amit Prakash Shukla @ 2023-09-22 20:13 UTC (permalink / raw)
To: Amit Prakash Shukla, Jerin Jacob
Cc: dev, fengchengwen, kevin.laatz, bruce.richardson, conor.walsh,
vattunuru, g.singh, sachin.saxena, hemant.agrawal, cheng1.jiang,
ndabilpuram, anoobj, mb
Added API support to add and delete vchan's for a DMA device.
Signed-off-by: Amit Prakash Shukla <amitprakashs@marvell.com>
---
lib/eventdev/rte_event_dma_adapter.c | 204 +++++++++++++++++++++++++++
1 file changed, 204 insertions(+)
diff --git a/lib/eventdev/rte_event_dma_adapter.c b/lib/eventdev/rte_event_dma_adapter.c
index c7ffba1b47..dd58188bf3 100644
--- a/lib/eventdev/rte_event_dma_adapter.c
+++ b/lib/eventdev/rte_event_dma_adapter.c
@@ -42,8 +42,31 @@ struct dma_ops_circular_buffer {
struct rte_event_dma_adapter_op **op_buffer;
} __rte_cache_aligned;
+/* Vchan information */
+struct dma_vchan_info {
+ /* Set to indicate vchan queue is enabled */
+ bool vq_enabled;
+
+ /* Circular buffer for batching DMA ops to dma_dev */
+ struct dma_ops_circular_buffer dma_buf;
+} __rte_cache_aligned;
+
/* DMA device information */
struct dma_device_info {
+ /* Pointer to vchan queue info */
+ struct dma_vchan_info *vchanq;
+
+ /* Pointer to vchan queue info.
+ * This holds ops passed by application till the
+ * dma completion is done.
+ */
+ struct dma_vchan_info *tqmap;
+
+ /* If num_vchanq > 0, the start callback will
+ * be invoked if not already invoked
+ */
+ uint16_t num_vchanq;
+
/* Number of vchans configured for a DMA device. */
uint16_t num_dma_dev_vchan;
} __rte_cache_aligned;
@@ -81,6 +104,9 @@ struct event_dma_adapter {
/* Set if default_cb is being used */
int default_cb_arg;
+
+ /* No. of vchan queue configured */
+ uint16_t nb_vchanq;
} __rte_cache_aligned;
static struct event_dma_adapter **event_dma_adapter;
@@ -333,3 +359,181 @@ rte_event_dma_adapter_free(uint8_t id)
return 0;
}
+
+static void
+edma_update_vchanq_info(struct event_dma_adapter *adapter, struct dma_device_info *dev_info,
+ uint16_t vchan, uint8_t add)
+{
+ struct dma_vchan_info *vchan_info;
+ struct dma_vchan_info *tqmap_info;
+ int enabled;
+ uint16_t i;
+
+ if (dev_info->vchanq == NULL)
+ return;
+
+ if (vchan == RTE_DMA_ALL_VCHAN) {
+ for (i = 0; i < dev_info->num_dma_dev_vchan; i++)
+ edma_update_vchanq_info(adapter, dev_info, i, add);
+ } else {
+ tqmap_info = &dev_info->tqmap[vchan];
+ vchan_info = &dev_info->vchanq[vchan];
+ enabled = vchan_info->vq_enabled;
+ if (add) {
+ adapter->nb_vchanq += !enabled;
+ dev_info->num_vchanq += !enabled;
+ } else {
+ adapter->nb_vchanq -= enabled;
+ dev_info->num_vchanq -= enabled;
+ }
+ vchan_info->vq_enabled = !!add;
+ tqmap_info->vq_enabled = !!add;
+ }
+}
+
+int
+rte_event_dma_adapter_vchan_add(uint8_t id, int16_t dma_dev_id, uint16_t vchan,
+ const struct rte_event *event)
+{
+ struct event_dma_adapter *adapter;
+ struct dma_device_info *dev_info;
+ struct rte_eventdev *dev;
+ uint32_t cap;
+ int ret;
+
+ EVENT_DMA_ADAPTER_ID_VALID_OR_ERR_RET(id, -EINVAL);
+
+ if (!rte_dma_is_valid(dma_dev_id)) {
+ RTE_EDEV_LOG_ERR("Invalid dma_dev_id = %" PRIu8, dma_dev_id);
+ return -EINVAL;
+ }
+
+ adapter = edma_id_to_adapter(id);
+ if (adapter == NULL)
+ return -EINVAL;
+
+ dev = &rte_eventdevs[adapter->eventdev_id];
+ ret = rte_event_dma_adapter_caps_get(adapter->eventdev_id, dma_dev_id, &cap);
+ if (ret) {
+ RTE_EDEV_LOG_ERR("Failed to get adapter caps dev %u dma_dev %u", id, dma_dev_id);
+ return ret;
+ }
+
+ if ((cap & RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_VCHAN_EV_BIND) && (event == NULL)) {
+ RTE_EDEV_LOG_ERR("Event can not be NULL for dma_dev_id = %u", dma_dev_id);
+ return -EINVAL;
+ }
+
+ dev_info = &adapter->dma_devs[dma_dev_id];
+ if (vchan != RTE_DMA_ALL_VCHAN && vchan >= dev_info->num_dma_dev_vchan) {
+ RTE_EDEV_LOG_ERR("Invalid vhcan %u", vchan);
+ return -EINVAL;
+ }
+
+ /* In case HW cap is RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP_FWD, no
+ * need of service core as HW supports event forward capability.
+ */
+ if ((cap & RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP_FWD) ||
+ (cap & RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_VCHAN_EV_BIND &&
+ adapter->mode == RTE_EVENT_DMA_ADAPTER_OP_NEW) ||
+ (cap & RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP_NEW &&
+ adapter->mode == RTE_EVENT_DMA_ADAPTER_OP_NEW)) {
+ if (*dev->dev_ops->dma_adapter_vchan_add == NULL)
+ return -ENOTSUP;
+ if (dev_info->vchanq == NULL) {
+ dev_info->vchanq = rte_zmalloc_socket(adapter->mem_name,
+ dev_info->num_dma_dev_vchan *
+ sizeof(struct dma_vchan_info),
+ 0, adapter->socket_id);
+ if (dev_info->vchanq == NULL) {
+ printf("Queue pair add not supported\n");
+ return -ENOMEM;
+ }
+ }
+
+ if (dev_info->tqmap == NULL) {
+ dev_info->tqmap = rte_zmalloc_socket(adapter->mem_name,
+ dev_info->num_dma_dev_vchan *
+ sizeof(struct dma_vchan_info),
+ 0, adapter->socket_id);
+ if (dev_info->tqmap == NULL) {
+ printf("tq pair add not supported\n");
+ return -ENOMEM;
+ }
+ }
+
+ ret = (*dev->dev_ops->dma_adapter_vchan_add)(dev, dma_dev_id, vchan, event);
+ if (ret)
+ return ret;
+
+ else
+ edma_update_vchanq_info(adapter, &adapter->dma_devs[dma_dev_id], vchan, 1);
+ }
+
+ return 0;
+}
+
+int
+rte_event_dma_adapter_vchan_del(uint8_t id, int16_t dma_dev_id, uint16_t vchan)
+{
+ struct event_dma_adapter *adapter;
+ struct dma_device_info *dev_info;
+ struct rte_eventdev *dev;
+ uint32_t cap;
+ int ret;
+
+ EVENT_DMA_ADAPTER_ID_VALID_OR_ERR_RET(id, -EINVAL);
+
+ if (!rte_dma_is_valid(dma_dev_id)) {
+ RTE_EDEV_LOG_ERR("Invalid dma_dev_id = %" PRIu8, dma_dev_id);
+ return -EINVAL;
+ }
+
+ adapter = edma_id_to_adapter(id);
+ if (adapter == NULL)
+ return -EINVAL;
+
+ dev = &rte_eventdevs[adapter->eventdev_id];
+ ret = rte_event_dma_adapter_caps_get(adapter->eventdev_id, dma_dev_id, &cap);
+ if (ret)
+ return ret;
+
+ dev_info = &adapter->dma_devs[dma_dev_id];
+
+ if (vchan != RTE_DMA_ALL_VCHAN && vchan >= dev_info->num_dma_dev_vchan) {
+ RTE_EDEV_LOG_ERR("Invalid vhcan %" PRIu16, vchan);
+ return -EINVAL;
+ }
+
+ if ((cap & RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP_FWD) ||
+ (cap & RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP_NEW &&
+ adapter->mode == RTE_EVENT_DMA_ADAPTER_OP_NEW)) {
+ if (*dev->dev_ops->dma_adapter_vchan_del == NULL)
+ return -ENOTSUP;
+ ret = (*dev->dev_ops->dma_adapter_vchan_del)(dev, dma_dev_id, vchan);
+ if (ret == 0) {
+ edma_update_vchanq_info(adapter, dev_info, vchan, 0);
+ if (dev_info->num_vchanq == 0) {
+ rte_free(dev_info->vchanq);
+ dev_info->vchanq = NULL;
+ }
+ }
+ } else {
+ if (adapter->nb_vchanq == 0)
+ return 0;
+
+ rte_spinlock_lock(&adapter->lock);
+ edma_update_vchanq_info(adapter, dev_info, vchan, 0);
+
+ if (dev_info->num_vchanq == 0) {
+ rte_free(dev_info->vchanq);
+ rte_free(dev_info->tqmap);
+ dev_info->vchanq = NULL;
+ dev_info->tqmap = NULL;
+ }
+
+ rte_spinlock_unlock(&adapter->lock);
+ }
+
+ return ret;
+}
--
2.25.1
^ permalink raw reply [flat|nested] 110+ messages in thread
* [PATCH v2 05/12] eventdev: add support for service function
2023-09-22 20:13 ` [PATCH v2 00/12] event DMA adapter library support Amit Prakash Shukla
` (3 preceding siblings ...)
2023-09-22 20:13 ` [PATCH v2 04/12] eventdev: api support for vchan add and delete Amit Prakash Shukla
@ 2023-09-22 20:13 ` Amit Prakash Shukla
2023-09-22 20:13 ` [PATCH v2 06/12] eventdev: api support for DMA adapter start stop Amit Prakash Shukla
` (8 subsequent siblings)
13 siblings, 0 replies; 110+ messages in thread
From: Amit Prakash Shukla @ 2023-09-22 20:13 UTC (permalink / raw)
To: Amit Prakash Shukla, Jerin Jacob
Cc: dev, fengchengwen, kevin.laatz, bruce.richardson, conor.walsh,
vattunuru, g.singh, sachin.saxena, hemant.agrawal, cheng1.jiang,
ndabilpuram, anoobj, mb
Added support for DMA adapter service function for event devices.
Signed-off-by: Amit Prakash Shukla <amitprakashs@marvell.com>
---
lib/eventdev/rte_event_dma_adapter.c | 589 +++++++++++++++++++++++++++
1 file changed, 589 insertions(+)
diff --git a/lib/eventdev/rte_event_dma_adapter.c b/lib/eventdev/rte_event_dma_adapter.c
index dd58188bf3..a9b9452aa0 100644
--- a/lib/eventdev/rte_event_dma_adapter.c
+++ b/lib/eventdev/rte_event_dma_adapter.c
@@ -2,6 +2,8 @@
* Copyright (c) 2023 Marvell.
*/
+#include <rte_service_component.h>
+
#include "rte_eventdev.h"
#include "eventdev_pmd.h"
#include "rte_event_dma_adapter.h"
@@ -69,6 +71,10 @@ struct dma_device_info {
/* Number of vchans configured for a DMA device. */
uint16_t num_dma_dev_vchan;
+
+ /* Next queue pair to be processed */
+ uint16_t next_vchan_id;
+
} __rte_cache_aligned;
struct event_dma_adapter {
@@ -90,6 +96,9 @@ struct event_dma_adapter {
/* Lock to serialize config updates with service function */
rte_spinlock_t lock;
+ /* Next dma device to be processed */
+ uint16_t next_dmadev_id;
+
/* DMA device structure array */
struct dma_device_info *dma_devs;
@@ -107,6 +116,26 @@ struct event_dma_adapter {
/* No. of vchan queue configured */
uint16_t nb_vchanq;
+
+ /* Per adapter EAL service ID */
+ uint32_t service_id;
+
+ /* Service initialization state */
+ uint8_t service_initialized;
+
+ /* Max DMA ops processed in any service function invocation */
+ uint32_t max_nb;
+
+ /* Store event port's implicit release capability */
+ uint8_t implicit_release_disabled;
+
+ /* Flag to indicate backpressure at dma_dev
+ * Stop further dequeuing events from eventdev
+ */
+ bool stop_enq_to_dma_dev;
+
+ /* Loop counter to flush dma ops */
+ uint16_t transmit_loop_count;
} __rte_cache_aligned;
static struct event_dma_adapter **event_dma_adapter;
@@ -148,6 +177,18 @@ edma_array_init(void)
return 0;
}
+static inline bool
+edma_circular_buffer_batch_ready(struct dma_ops_circular_buffer *bufp)
+{
+ return bufp->count >= DMA_BATCH_SIZE;
+}
+
+static inline bool
+edma_circular_buffer_space_for_batch(struct dma_ops_circular_buffer *bufp)
+{
+ return (bufp->size - bufp->count) >= DMA_BATCH_SIZE;
+}
+
static inline int
edma_circular_buffer_init(const char *name, struct dma_ops_circular_buffer *buf, uint16_t sz)
{
@@ -166,6 +207,67 @@ edma_circular_buffer_free(struct dma_ops_circular_buffer *buf)
rte_free(buf->op_buffer);
}
+static inline int
+edma_circular_buffer_add(struct dma_ops_circular_buffer *bufp, struct rte_event_dma_adapter_op *op)
+{
+ uint16_t *tail = &bufp->tail;
+
+ bufp->op_buffer[*tail] = op;
+
+ /* circular buffer, go round */
+ *tail = (*tail + 1) % bufp->size;
+ bufp->count++;
+
+ return 0;
+}
+
+static inline int
+edma_circular_buffer_flush_to_dma_dev(struct event_dma_adapter *adapter,
+ struct dma_ops_circular_buffer *bufp, uint8_t dma_dev_id,
+ uint16_t vchan, uint16_t *nb_ops_flushed)
+{
+ struct rte_event_dma_adapter_op *op;
+ struct dma_vchan_info *tq;
+ uint16_t *head = &bufp->head;
+ uint16_t *tail = &bufp->tail;
+ uint16_t n;
+ uint16_t i;
+ int ret;
+
+ if (*tail > *head)
+ n = *tail - *head;
+ else if (*tail < *head)
+ n = bufp->size - *head;
+ else {
+ *nb_ops_flushed = 0;
+ return 0; /* buffer empty */
+ }
+
+ tq = &adapter->dma_devs[dma_dev_id].tqmap[vchan];
+
+ for (i = 0; i < n; i++) {
+ op = bufp->op_buffer[*head];
+ ret = rte_dma_copy_sg(dma_dev_id, vchan, op->src_seg, op->dst_seg,
+ op->nb_src, op->nb_dst, op->flags);
+ if (ret < 0)
+ break;
+
+ /* Enqueue in transaction queue. */
+ edma_circular_buffer_add(&tq->dma_buf, op);
+
+ *head = (*head + 1) % bufp->size;
+ }
+
+ *nb_ops_flushed = i;
+ bufp->count -= *nb_ops_flushed;
+ if (!bufp->count) {
+ *head = 0;
+ *tail = 0;
+ }
+
+ return *nb_ops_flushed == n ? 0 : -1;
+}
+
static int
edma_default_config_cb(uint8_t id, uint8_t evdev_id, struct rte_event_dma_adapter_conf *conf,
void *arg)
@@ -360,6 +462,406 @@ rte_event_dma_adapter_free(uint8_t id)
return 0;
}
+static inline unsigned int
+edma_enq_to_dma_dev(struct event_dma_adapter *adapter, struct rte_event *ev, unsigned int cnt)
+{
+ struct dma_vchan_info *vchan_qinfo = NULL;
+ struct rte_event_dma_adapter_op *dma_op;
+ uint16_t vchan, nb_enqueued = 0;
+ int16_t dma_dev_id;
+ unsigned int i, n;
+ int ret;
+
+ ret = 0;
+ n = 0;
+
+ for (i = 0; i < cnt; i++) {
+ dma_op = ev[i].event_ptr;
+ if (dma_op == NULL)
+ continue;
+
+ /* Expected to have reponse info appended to dma_op. */
+
+ dma_dev_id = dma_op->dma_dev_id;
+ vchan = dma_op->vchan;
+ vchan_qinfo = &adapter->dma_devs[dma_dev_id].vchanq[vchan];
+ if (!vchan_qinfo->vq_enabled) {
+ if (dma_op != NULL && dma_op->op_mp != NULL)
+ rte_mempool_put(dma_op->op_mp, dma_op);
+ continue;
+ }
+ edma_circular_buffer_add(&vchan_qinfo->dma_buf, dma_op);
+
+ if (edma_circular_buffer_batch_ready(&vchan_qinfo->dma_buf)) {
+ ret = edma_circular_buffer_flush_to_dma_dev(adapter, &vchan_qinfo->dma_buf,
+ dma_dev_id, vchan,
+ &nb_enqueued);
+ n += nb_enqueued;
+
+ /**
+ * If some dma ops failed to flush to dma_dev and
+ * space for another batch is not available, stop
+ * dequeue from eventdev momentarily
+ */
+ if (unlikely(ret < 0 &&
+ !edma_circular_buffer_space_for_batch(&vchan_qinfo->dma_buf)))
+ adapter->stop_enq_to_dma_dev = true;
+ }
+ }
+
+ return n;
+}
+
+static unsigned int
+edma_adapter_dev_flush(struct event_dma_adapter *adapter, int16_t dma_dev_id,
+ uint16_t *nb_ops_flushed)
+{
+ struct dma_vchan_info *vchan_info;
+ struct dma_device_info *dev_info;
+ uint16_t nb = 0, nb_enqueued = 0;
+ uint16_t vchan, nb_vchans;
+
+ dev_info = &adapter->dma_devs[dma_dev_id];
+ nb_vchans = dev_info->num_vchanq;
+
+ for (vchan = 0; vchan < nb_vchans; vchan++) {
+
+ vchan_info = &dev_info->vchanq[vchan];
+ if (unlikely(vchan_info == NULL || !vchan_info->vq_enabled))
+ continue;
+
+ edma_circular_buffer_flush_to_dma_dev(adapter, &vchan_info->dma_buf, dma_dev_id,
+ vchan, &nb_enqueued);
+ *nb_ops_flushed += vchan_info->dma_buf.count;
+ nb += nb_enqueued;
+ }
+
+ return nb;
+}
+
+static unsigned int
+edma_adapter_enq_flush(struct event_dma_adapter *adapter)
+{
+ int16_t dma_dev_id;
+ uint16_t nb_enqueued = 0;
+ uint16_t nb_ops_flushed = 0;
+ uint16_t num_dma_dev = rte_dma_count_avail();
+
+ for (dma_dev_id = 0; dma_dev_id < num_dma_dev; dma_dev_id++)
+ nb_enqueued += edma_adapter_dev_flush(adapter, dma_dev_id, &nb_ops_flushed);
+ /**
+ * Enable dequeue from eventdev if all ops from circular
+ * buffer flushed to dma_dev
+ */
+ if (!nb_ops_flushed)
+ adapter->stop_enq_to_dma_dev = false;
+
+ return nb_enqueued;
+}
+
+/* Flush an instance's enqueue buffers every DMA_ENQ_FLUSH_THRESHOLD
+ * iterations of edma_adapter_enq_run()
+ */
+#define DMA_ENQ_FLUSH_THRESHOLD 1024
+
+static int
+edma_adapter_enq_run(struct event_dma_adapter *adapter, unsigned int max_enq)
+{
+ uint8_t event_port_id = adapter->event_port_id;
+ uint8_t event_dev_id = adapter->eventdev_id;
+ struct rte_event ev[DMA_BATCH_SIZE];
+ unsigned int nb_enq, nb_enqueued;
+ uint16_t n;
+
+ if (adapter->mode == RTE_EVENT_DMA_ADAPTER_OP_NEW)
+ return 0;
+
+ nb_enqueued = 0;
+ for (nb_enq = 0; nb_enq < max_enq; nb_enq += n) {
+
+ if (unlikely(adapter->stop_enq_to_dma_dev)) {
+ nb_enqueued += edma_adapter_enq_flush(adapter);
+
+ if (unlikely(adapter->stop_enq_to_dma_dev))
+ break;
+ }
+
+ n = rte_event_dequeue_burst(event_dev_id, event_port_id, ev, DMA_BATCH_SIZE, 0);
+
+ if (!n)
+ break;
+
+ nb_enqueued += edma_enq_to_dma_dev(adapter, ev, n);
+ }
+
+ if ((++adapter->transmit_loop_count & (DMA_ENQ_FLUSH_THRESHOLD - 1)) == 0)
+ nb_enqueued += edma_adapter_enq_flush(adapter);
+
+ return nb_enqueued;
+}
+
+#define DMA_ADAPTER_MAX_EV_ENQ_RETRIES 100
+
+static inline uint16_t
+edma_ops_enqueue_burst(struct event_dma_adapter *adapter, struct rte_event_dma_adapter_op **ops,
+ uint16_t num)
+{
+ uint8_t event_port_id = adapter->event_port_id;
+ uint8_t event_dev_id = adapter->eventdev_id;
+ struct rte_event events[DMA_BATCH_SIZE];
+ struct rte_event *response_info;
+ uint16_t nb_enqueued, nb_ev;
+ uint8_t retry;
+ uint8_t i;
+
+ nb_ev = 0;
+ retry = 0;
+ nb_enqueued = 0;
+ num = RTE_MIN(num, DMA_BATCH_SIZE);
+ for (i = 0; i < num; i++) {
+ struct rte_event *ev = &events[nb_ev++];
+
+ /* Expected to have response info appended to dma_op. */
+ response_info = (struct rte_event *)((uint8_t *)ops[i] +
+ sizeof(struct rte_event_dma_adapter_op));
+ if (unlikely(response_info == NULL)) {
+ if (ops[i] != NULL && ops[i]->op_mp != NULL)
+ rte_mempool_put(ops[i]->op_mp, ops[i]);
+ continue;
+ }
+
+ rte_memcpy(ev, response_info, sizeof(struct rte_event));
+ ev->event_ptr = ops[i];
+ ev->event_type = RTE_EVENT_TYPE_DMADEV;
+ if (adapter->implicit_release_disabled)
+ ev->op = RTE_EVENT_OP_FORWARD;
+ else
+ ev->op = RTE_EVENT_OP_NEW;
+ }
+
+ do {
+ nb_enqueued += rte_event_enqueue_burst(event_dev_id, event_port_id,
+ &events[nb_enqueued], nb_ev - nb_enqueued);
+
+ } while (retry++ < DMA_ADAPTER_MAX_EV_ENQ_RETRIES && nb_enqueued < nb_ev);
+
+ return nb_enqueued;
+}
+
+static int
+edma_circular_buffer_flush_to_evdev(struct event_dma_adapter *adapter,
+ struct dma_ops_circular_buffer *bufp,
+ uint16_t *enqueue_count)
+{
+ struct rte_event_dma_adapter_op **ops = bufp->op_buffer;
+ uint16_t n = 0, nb_ops_flushed;
+ uint16_t *head = &bufp->head;
+ uint16_t *tail = &bufp->tail;
+
+ if (*tail > *head)
+ n = *tail - *head;
+ else if (*tail < *head)
+ n = bufp->size - *head;
+ else {
+ if (enqueue_count)
+ *enqueue_count = 0;
+ return 0; /* buffer empty */
+ }
+
+ if (enqueue_count && n > *enqueue_count)
+ n = *enqueue_count;
+
+ nb_ops_flushed = edma_ops_enqueue_burst(adapter, &ops[*head], n);
+ if (enqueue_count)
+ *enqueue_count = nb_ops_flushed;
+
+ bufp->count -= nb_ops_flushed;
+ if (!bufp->count) {
+ *head = 0;
+ *tail = 0;
+ return 0; /* buffer empty */
+ }
+
+ *head = (*head + nb_ops_flushed) % bufp->size;
+ return 1;
+}
+
+static void
+edma_ops_buffer_flush(struct event_dma_adapter *adapter)
+{
+ if (likely(adapter->ebuf.count == 0))
+ return;
+
+ while (edma_circular_buffer_flush_to_evdev(adapter, &adapter->ebuf, NULL))
+ ;
+}
+
+static inline unsigned int
+edma_adapter_deq_run(struct event_dma_adapter *adapter, unsigned int max_deq)
+{
+ struct dma_vchan_info *vchan_info;
+ struct dma_ops_circular_buffer *tq_buf;
+ struct rte_event_dma_adapter_op *ops;
+ uint16_t n, nb_deq, nb_enqueued, i;
+ struct dma_device_info *dev_info;
+ uint16_t vchan, num_vchan;
+ uint16_t num_dma_dev;
+ int16_t dma_dev_id;
+ uint16_t index;
+ bool done;
+ bool err;
+
+ nb_deq = 0;
+ edma_ops_buffer_flush(adapter);
+
+ num_dma_dev = rte_dma_count_avail();
+ do {
+ done = true;
+
+ for (dma_dev_id = adapter->next_dmadev_id; dma_dev_id < num_dma_dev; dma_dev_id++) {
+ uint16_t queues = 0;
+ dev_info = &adapter->dma_devs[dma_dev_id];
+ num_vchan = dev_info->num_vchanq;
+
+ for (vchan = dev_info->next_vchan_id; queues < num_vchan;
+ vchan = (vchan + 1) % num_vchan, queues++) {
+
+ vchan_info = &dev_info->vchanq[vchan];
+ if (unlikely(vchan_info == NULL || !vchan_info->vq_enabled))
+ continue;
+
+ n = rte_dma_completed(dma_dev_id, vchan, DMA_BATCH_SIZE,
+ &index, &err);
+ if (!n)
+ continue;
+
+ done = false;
+
+ tq_buf = &dev_info->tqmap[vchan].dma_buf;
+
+ nb_enqueued = n;
+ if (unlikely(!adapter->ebuf.count))
+ edma_circular_buffer_flush_to_evdev(adapter, tq_buf,
+ &nb_enqueued);
+
+ if (likely(nb_enqueued == n))
+ goto check;
+
+ /* Failed to enqueue events case */
+ for (i = nb_enqueued; i < n; i++) {
+ ops = tq_buf->op_buffer[tq_buf->head];
+ edma_circular_buffer_add(&adapter->ebuf, ops);
+ tq_buf->head = (tq_buf->head + 1) % tq_buf->size;
+ }
+
+check:
+ nb_deq += n;
+ if (nb_deq >= max_deq) {
+ if ((vchan + 1) == num_vchan)
+ adapter->next_dmadev_id =
+ (dma_dev_id + 1) % num_dma_dev;
+
+ dev_info->next_vchan_id = (vchan + 1) % num_vchan;
+
+ return nb_deq;
+ }
+ }
+ }
+ adapter->next_dmadev_id = 0;
+
+ } while (done == false);
+
+ return nb_deq;
+}
+
+static int
+edma_adapter_run(struct event_dma_adapter *adapter, unsigned int max_ops)
+{
+ unsigned int ops_left = max_ops;
+
+ while (ops_left > 0) {
+ unsigned int e_cnt, d_cnt;
+
+ e_cnt = edma_adapter_deq_run(adapter, ops_left);
+ ops_left -= RTE_MIN(ops_left, e_cnt);
+
+ d_cnt = edma_adapter_enq_run(adapter, ops_left);
+ ops_left -= RTE_MIN(ops_left, d_cnt);
+
+ if (e_cnt == 0 && d_cnt == 0)
+ break;
+ }
+
+ if (ops_left == max_ops) {
+ rte_event_maintain(adapter->eventdev_id, adapter->event_port_id, 0);
+ return -EAGAIN;
+ } else
+ return 0;
+}
+
+static int
+edma_service_func(void *args)
+{
+ struct event_dma_adapter *adapter = args;
+ int ret;
+
+ if (rte_spinlock_trylock(&adapter->lock) == 0)
+ return 0;
+ ret = edma_adapter_run(adapter, adapter->max_nb);
+ rte_spinlock_unlock(&adapter->lock);
+
+ return ret;
+}
+
+static int
+edma_init_service(struct event_dma_adapter *adapter, uint8_t id)
+{
+ struct rte_event_dma_adapter_conf adapter_conf;
+ struct rte_service_spec service;
+ uint32_t impl_rel;
+ int ret;
+
+ if (adapter->service_initialized)
+ return 0;
+
+ memset(&service, 0, sizeof(service));
+ snprintf(service.name, DMA_ADAPTER_NAME_LEN, "rte_event_dma_adapter_%d", id);
+ service.socket_id = adapter->socket_id;
+ service.callback = edma_service_func;
+ service.callback_userdata = adapter;
+
+ /* Service function handles locking for queue add/del updates */
+ service.capabilities = RTE_SERVICE_CAP_MT_SAFE;
+ ret = rte_service_component_register(&service, &adapter->service_id);
+ if (ret) {
+ RTE_EDEV_LOG_ERR("failed to register service %s err = %" PRId32, service.name, ret);
+ return ret;
+ }
+
+ ret = adapter->conf_cb(id, adapter->eventdev_id, &adapter_conf, adapter->conf_arg);
+ if (ret) {
+ RTE_EDEV_LOG_ERR("configuration callback failed err = %" PRId32, ret);
+ return ret;
+ }
+
+ adapter->max_nb = adapter_conf.max_nb;
+ adapter->event_port_id = adapter_conf.event_port_id;
+
+ if (rte_event_port_attr_get(adapter->eventdev_id, adapter->event_port_id,
+ RTE_EVENT_PORT_ATTR_IMPLICIT_RELEASE_DISABLE, &impl_rel)) {
+ RTE_EDEV_LOG_ERR("Failed to get port info for eventdev %" PRId32,
+ adapter->eventdev_id);
+ edma_circular_buffer_free(&adapter->ebuf);
+ rte_free(adapter);
+ return -EINVAL;
+ }
+
+ adapter->implicit_release_disabled = (uint8_t)impl_rel;
+ adapter->service_initialized = 1;
+
+ return ret;
+}
+
static void
edma_update_vchanq_info(struct event_dma_adapter *adapter, struct dma_device_info *dev_info,
uint16_t vchan, uint8_t add)
@@ -391,6 +893,60 @@ edma_update_vchanq_info(struct event_dma_adapter *adapter, struct dma_device_inf
}
}
+static int
+edma_add_vchan(struct event_dma_adapter *adapter, int16_t dma_dev_id, uint16_t vchan)
+{
+ struct dma_device_info *dev_info = &adapter->dma_devs[dma_dev_id];
+ struct dma_vchan_info *vchanq;
+ struct dma_vchan_info *tqmap;
+ uint16_t nb_vchans;
+ uint32_t i;
+
+ if (dev_info->vchanq == NULL) {
+ nb_vchans = dev_info->num_dma_dev_vchan;
+
+ dev_info->vchanq = rte_zmalloc_socket(adapter->mem_name,
+ nb_vchans * sizeof(struct dma_vchan_info),
+ 0, adapter->socket_id);
+ if (dev_info->vchanq == NULL)
+ return -ENOMEM;
+
+ dev_info->tqmap = rte_zmalloc_socket(adapter->mem_name,
+ nb_vchans * sizeof(struct dma_vchan_info),
+ 0, adapter->socket_id);
+ if (dev_info->tqmap == NULL)
+ return -ENOMEM;
+
+ for (i = 0; i < nb_vchans; i++) {
+ vchanq = &dev_info->vchanq[i];
+
+ if (edma_circular_buffer_init("dma_dev_circular_buffer", &vchanq->dma_buf,
+ DMA_ADAPTER_OPS_BUFFER_SIZE)) {
+ RTE_EDEV_LOG_ERR("Failed to get memory for dma_dev buffer");
+ rte_free(vchanq);
+ return -ENOMEM;
+ }
+
+ tqmap = &dev_info->tqmap[i];
+ if (edma_circular_buffer_init("dma_dev_circular_trans_buf", &tqmap->dma_buf,
+ DMA_ADAPTER_OPS_BUFFER_SIZE)) {
+ RTE_EDEV_LOG_ERR(
+ "Failed to get memory for dma_dev transction buffer");
+ rte_free(tqmap);
+ return -ENOMEM;
+ }
+ }
+ }
+
+ if (vchan == RTE_DMA_ALL_VCHAN) {
+ for (i = 0; i < dev_info->num_dma_dev_vchan; i++)
+ edma_update_vchanq_info(adapter, dev_info, i, 1);
+ } else
+ edma_update_vchanq_info(adapter, dev_info, vchan, 1);
+
+ return 0;
+}
+
int
rte_event_dma_adapter_vchan_add(uint8_t id, int16_t dma_dev_id, uint16_t vchan,
const struct rte_event *event)
@@ -470,6 +1026,38 @@ rte_event_dma_adapter_vchan_add(uint8_t id, int16_t dma_dev_id, uint16_t vchan,
edma_update_vchanq_info(adapter, &adapter->dma_devs[dma_dev_id], vchan, 1);
}
+ /* In case HW cap is RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP_NEW, or SW adapter, initiate
+ * services so the application can choose which ever way it wants to use the adapter.
+ *
+ * Case 1: RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP_NEW. Application may wants to use one
+ * of below two modes
+ *
+ * a. OP_FORWARD mode -> HW Dequeue + SW enqueue
+ * b. OP_NEW mode -> HW Dequeue
+ *
+ * Case 2: No HW caps, use SW adapter
+ *
+ * a. OP_FORWARD mode -> SW enqueue & dequeue
+ * b. OP_NEW mode -> SW Dequeue
+ */
+ if ((cap & RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP_NEW &&
+ !(cap & RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP_FWD) &&
+ adapter->mode == RTE_EVENT_DMA_ADAPTER_OP_FORWARD) ||
+ (!(cap & RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP_NEW) &&
+ !(cap & RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP_FWD) &&
+ !(cap & RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_VCHAN_EV_BIND))) {
+ rte_spinlock_lock(&adapter->lock);
+ ret = edma_init_service(adapter, id);
+ if (ret == 0)
+ ret = edma_add_vchan(adapter, dma_dev_id, vchan);
+ rte_spinlock_unlock(&adapter->lock);
+
+ if (ret)
+ return ret;
+
+ rte_service_component_runstate_set(adapter->service_id, 1);
+ }
+
return 0;
}
@@ -533,6 +1121,7 @@ rte_event_dma_adapter_vchan_del(uint8_t id, int16_t dma_dev_id, uint16_t vchan)
}
rte_spinlock_unlock(&adapter->lock);
+ rte_service_component_runstate_set(adapter->service_id, adapter->nb_vchanq);
}
return ret;
--
2.25.1
^ permalink raw reply [flat|nested] 110+ messages in thread
* [PATCH v2 06/12] eventdev: api support for DMA adapter start stop
2023-09-22 20:13 ` [PATCH v2 00/12] event DMA adapter library support Amit Prakash Shukla
` (4 preceding siblings ...)
2023-09-22 20:13 ` [PATCH v2 05/12] eventdev: add support for service function Amit Prakash Shukla
@ 2023-09-22 20:13 ` Amit Prakash Shukla
2023-09-22 20:13 ` [PATCH v2 07/12] eventdev: api support to get DMA adapter service ID Amit Prakash Shukla
` (7 subsequent siblings)
13 siblings, 0 replies; 110+ messages in thread
From: Amit Prakash Shukla @ 2023-09-22 20:13 UTC (permalink / raw)
To: Amit Prakash Shukla, Jerin Jacob
Cc: dev, fengchengwen, kevin.laatz, bruce.richardson, conor.walsh,
vattunuru, g.singh, sachin.saxena, hemant.agrawal, cheng1.jiang,
ndabilpuram, anoobj, mb
Added API support to start and stop DMA adapter.
Signed-off-by: Amit Prakash Shukla <amitprakashs@marvell.com>
---
lib/eventdev/rte_event_dma_adapter.c | 69 ++++++++++++++++++++++++++++
1 file changed, 69 insertions(+)
diff --git a/lib/eventdev/rte_event_dma_adapter.c b/lib/eventdev/rte_event_dma_adapter.c
index a9b9452aa0..2f574cb7ea 100644
--- a/lib/eventdev/rte_event_dma_adapter.c
+++ b/lib/eventdev/rte_event_dma_adapter.c
@@ -75,6 +75,13 @@ struct dma_device_info {
/* Next queue pair to be processed */
uint16_t next_vchan_id;
+ /* Set to indicate processing has been started */
+ uint8_t dev_started;
+
+ /* Set to indicate dmadev->eventdev packet
+ * transfer uses a hardware mechanism
+ */
+ uint8_t internal_event_port;
} __rte_cache_aligned;
struct event_dma_adapter {
@@ -1126,3 +1133,65 @@ rte_event_dma_adapter_vchan_del(uint8_t id, int16_t dma_dev_id, uint16_t vchan)
return ret;
}
+
+static int
+edma_adapter_ctrl(uint8_t id, int start)
+{
+ struct event_dma_adapter *adapter;
+ struct dma_device_info *dev_info;
+ struct rte_eventdev *dev;
+ uint16_t num_dma_dev;
+ int stop = !start;
+ int use_service;
+ uint32_t i;
+
+ use_service = 0;
+ EVENT_DMA_ADAPTER_ID_VALID_OR_ERR_RET(id, -EINVAL);
+ adapter = edma_id_to_adapter(id);
+ if (adapter == NULL)
+ return -EINVAL;
+
+ num_dma_dev = rte_dma_count_avail();
+ dev = &rte_eventdevs[adapter->eventdev_id];
+
+ for (i = 0; i < num_dma_dev; i++) {
+ dev_info = &adapter->dma_devs[i];
+ /* start check for num queue pairs */
+ if (start && !dev_info->num_vchanq)
+ continue;
+ /* stop check if dev has been started */
+ if (stop && !dev_info->dev_started)
+ continue;
+ use_service |= !dev_info->internal_event_port;
+ dev_info->dev_started = start;
+ if (dev_info->internal_event_port == 0)
+ continue;
+ start ? (*dev->dev_ops->dma_adapter_start)(dev, i) :
+ (*dev->dev_ops->dma_adapter_stop)(dev, i);
+ }
+
+ if (use_service)
+ rte_service_runstate_set(adapter->service_id, start);
+
+ return 0;
+}
+
+int
+rte_event_dma_adapter_start(uint8_t id)
+{
+ struct event_dma_adapter *adapter;
+
+ EVENT_DMA_ADAPTER_ID_VALID_OR_ERR_RET(id, -EINVAL);
+
+ adapter = edma_id_to_adapter(id);
+ if (adapter == NULL)
+ return -EINVAL;
+
+ return edma_adapter_ctrl(id, 1);
+}
+
+int
+rte_event_dma_adapter_stop(uint8_t id)
+{
+ return edma_adapter_ctrl(id, 0);
+}
--
2.25.1
^ permalink raw reply [flat|nested] 110+ messages in thread
* [PATCH v2 07/12] eventdev: api support to get DMA adapter service ID
2023-09-22 20:13 ` [PATCH v2 00/12] event DMA adapter library support Amit Prakash Shukla
` (5 preceding siblings ...)
2023-09-22 20:13 ` [PATCH v2 06/12] eventdev: api support for DMA adapter start stop Amit Prakash Shukla
@ 2023-09-22 20:13 ` Amit Prakash Shukla
2023-09-22 20:13 ` [PATCH v2 08/12] eventdev: add DMA adapter support for runtime params Amit Prakash Shukla
` (6 subsequent siblings)
13 siblings, 0 replies; 110+ messages in thread
From: Amit Prakash Shukla @ 2023-09-22 20:13 UTC (permalink / raw)
To: Amit Prakash Shukla, Jerin Jacob
Cc: dev, fengchengwen, kevin.laatz, bruce.richardson, conor.walsh,
vattunuru, g.singh, sachin.saxena, hemant.agrawal, cheng1.jiang,
ndabilpuram, anoobj, mb
Added API support to get DMA adapter service ID.
Signed-off-by: Amit Prakash Shukla <amitprakashs@marvell.com>
---
lib/eventdev/rte_event_dma_adapter.c | 17 +++++++++++++++++
1 file changed, 17 insertions(+)
diff --git a/lib/eventdev/rte_event_dma_adapter.c b/lib/eventdev/rte_event_dma_adapter.c
index 2f574cb7ea..e66e0307b9 100644
--- a/lib/eventdev/rte_event_dma_adapter.c
+++ b/lib/eventdev/rte_event_dma_adapter.c
@@ -1134,6 +1134,23 @@ rte_event_dma_adapter_vchan_del(uint8_t id, int16_t dma_dev_id, uint16_t vchan)
return ret;
}
+int
+rte_event_dma_adapter_service_id_get(uint8_t id, uint32_t *service_id)
+{
+ struct event_dma_adapter *adapter;
+
+ EVENT_DMA_ADAPTER_ID_VALID_OR_ERR_RET(id, -EINVAL);
+
+ adapter = edma_id_to_adapter(id);
+ if (adapter == NULL || service_id == NULL)
+ return -EINVAL;
+
+ if (adapter->service_initialized)
+ *service_id = adapter->service_id;
+
+ return adapter->service_initialized ? 0 : -ESRCH;
+}
+
static int
edma_adapter_ctrl(uint8_t id, int start)
{
--
2.25.1
^ permalink raw reply [flat|nested] 110+ messages in thread
* [PATCH v2 08/12] eventdev: add DMA adapter support for runtime params
2023-09-22 20:13 ` [PATCH v2 00/12] event DMA adapter library support Amit Prakash Shukla
` (6 preceding siblings ...)
2023-09-22 20:13 ` [PATCH v2 07/12] eventdev: api support to get DMA adapter service ID Amit Prakash Shukla
@ 2023-09-22 20:13 ` Amit Prakash Shukla
2023-09-22 20:13 ` [PATCH v2 09/12] eventdev: add support for DMA adapter stats Amit Prakash Shukla
` (5 subsequent siblings)
13 siblings, 0 replies; 110+ messages in thread
From: Amit Prakash Shukla @ 2023-09-22 20:13 UTC (permalink / raw)
To: Amit Prakash Shukla, Jerin Jacob
Cc: dev, fengchengwen, kevin.laatz, bruce.richardson, conor.walsh,
vattunuru, g.singh, sachin.saxena, hemant.agrawal, cheng1.jiang,
ndabilpuram, anoobj, mb
Added support to set and get runtime params for DMA adapter.
Signed-off-by: Amit Prakash Shukla <amitprakashs@marvell.com>
---
lib/eventdev/rte_event_dma_adapter.c | 93 ++++++++++++++++++++++++++++
1 file changed, 93 insertions(+)
diff --git a/lib/eventdev/rte_event_dma_adapter.c b/lib/eventdev/rte_event_dma_adapter.c
index e66e0307b9..20817768fc 100644
--- a/lib/eventdev/rte_event_dma_adapter.c
+++ b/lib/eventdev/rte_event_dma_adapter.c
@@ -1212,3 +1212,96 @@ rte_event_dma_adapter_stop(uint8_t id)
{
return edma_adapter_ctrl(id, 0);
}
+
+#define DEFAULT_MAX_NB 128
+
+int
+rte_event_dma_adapter_runtime_params_init(struct rte_event_dma_adapter_runtime_params *params)
+{
+ if (params == NULL)
+ return -EINVAL;
+
+ memset(params, 0, sizeof(*params));
+ params->max_nb = DEFAULT_MAX_NB;
+
+ return 0;
+}
+
+static int
+dma_adapter_cap_check(struct event_dma_adapter *adapter)
+{
+ uint32_t caps;
+ int ret;
+
+ if (!adapter->nb_vchanq)
+ return -EINVAL;
+
+ ret = rte_event_dma_adapter_caps_get(adapter->eventdev_id, adapter->next_dmadev_id, &caps);
+ if (ret) {
+ RTE_EDEV_LOG_ERR("Failed to get adapter caps dev %" PRIu8 " cdev %" PRIu8,
+ adapter->eventdev_id, adapter->next_dmadev_id);
+ return ret;
+ }
+
+ if ((caps & RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP_FWD) ||
+ (caps & RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP_NEW))
+ return -ENOTSUP;
+
+ return 0;
+}
+
+int
+rte_event_dma_adapter_runtime_params_set(uint8_t id,
+ struct rte_event_dma_adapter_runtime_params *params)
+{
+ struct event_dma_adapter *adapter;
+ int ret;
+
+ EVENT_DMA_ADAPTER_ID_VALID_OR_ERR_RET(id, -EINVAL);
+
+ if (params == NULL) {
+ RTE_EDEV_LOG_ERR("params pointer is NULL\n");
+ return -EINVAL;
+ }
+
+ adapter = edma_id_to_adapter(id);
+ if (adapter == NULL)
+ return -EINVAL;
+
+ ret = dma_adapter_cap_check(adapter);
+ if (ret)
+ return ret;
+
+ rte_spinlock_lock(&adapter->lock);
+ adapter->max_nb = params->max_nb;
+ rte_spinlock_unlock(&adapter->lock);
+
+ return 0;
+}
+
+int
+rte_event_dma_adapter_runtime_params_get(uint8_t id,
+ struct rte_event_dma_adapter_runtime_params *params)
+{
+ struct event_dma_adapter *adapter;
+ int ret;
+
+ EVENT_DMA_ADAPTER_ID_VALID_OR_ERR_RET(id, -EINVAL);
+
+ if (params == NULL) {
+ RTE_EDEV_LOG_ERR("params pointer is NULL\n");
+ return -EINVAL;
+ }
+
+ adapter = edma_id_to_adapter(id);
+ if (adapter == NULL)
+ return -EINVAL;
+
+ ret = dma_adapter_cap_check(adapter);
+ if (ret)
+ return ret;
+
+ params->max_nb = adapter->max_nb;
+
+ return 0;
+}
--
2.25.1
^ permalink raw reply [flat|nested] 110+ messages in thread
* [PATCH v2 09/12] eventdev: add support for DMA adapter stats
2023-09-22 20:13 ` [PATCH v2 00/12] event DMA adapter library support Amit Prakash Shukla
` (7 preceding siblings ...)
2023-09-22 20:13 ` [PATCH v2 08/12] eventdev: add DMA adapter support for runtime params Amit Prakash Shukla
@ 2023-09-22 20:13 ` Amit Prakash Shukla
2023-09-22 20:13 ` [PATCH v2 10/12] eventdev: add support for DMA adapter enqueue Amit Prakash Shukla
` (4 subsequent siblings)
13 siblings, 0 replies; 110+ messages in thread
From: Amit Prakash Shukla @ 2023-09-22 20:13 UTC (permalink / raw)
To: Amit Prakash Shukla, Jerin Jacob
Cc: dev, fengchengwen, kevin.laatz, bruce.richardson, conor.walsh,
vattunuru, g.singh, sachin.saxena, hemant.agrawal, cheng1.jiang,
ndabilpuram, anoobj, mb
Added DMA adapter stats support for get and reset functions.
Signed-off-by: Amit Prakash Shukla <amitprakashs@marvell.com>
---
lib/eventdev/rte_event_dma_adapter.c | 95 ++++++++++++++++++++++++++++
1 file changed, 95 insertions(+)
diff --git a/lib/eventdev/rte_event_dma_adapter.c b/lib/eventdev/rte_event_dma_adapter.c
index 20817768fc..06a09371d2 100644
--- a/lib/eventdev/rte_event_dma_adapter.c
+++ b/lib/eventdev/rte_event_dma_adapter.c
@@ -143,6 +143,9 @@ struct event_dma_adapter {
/* Loop counter to flush dma ops */
uint16_t transmit_loop_count;
+
+ /* Per instance stats structure */
+ struct rte_event_dma_adapter_stats dma_stats;
} __rte_cache_aligned;
static struct event_dma_adapter **event_dma_adapter;
@@ -472,6 +475,7 @@ rte_event_dma_adapter_free(uint8_t id)
static inline unsigned int
edma_enq_to_dma_dev(struct event_dma_adapter *adapter, struct rte_event *ev, unsigned int cnt)
{
+ struct rte_event_dma_adapter_stats *stats = &adapter->dma_stats;
struct dma_vchan_info *vchan_qinfo = NULL;
struct rte_event_dma_adapter_op *dma_op;
uint16_t vchan, nb_enqueued = 0;
@@ -481,6 +485,7 @@ edma_enq_to_dma_dev(struct event_dma_adapter *adapter, struct rte_event *ev, uns
ret = 0;
n = 0;
+ stats->event_deq_count += cnt;
for (i = 0; i < cnt; i++) {
dma_op = ev[i].event_ptr;
@@ -503,6 +508,7 @@ edma_enq_to_dma_dev(struct event_dma_adapter *adapter, struct rte_event *ev, uns
ret = edma_circular_buffer_flush_to_dma_dev(adapter, &vchan_qinfo->dma_buf,
dma_dev_id, vchan,
&nb_enqueued);
+ stats->dma_enq_count += nb_enqueued;
n += nb_enqueued;
/**
@@ -549,6 +555,7 @@ edma_adapter_dev_flush(struct event_dma_adapter *adapter, int16_t dma_dev_id,
static unsigned int
edma_adapter_enq_flush(struct event_dma_adapter *adapter)
{
+ struct rte_event_dma_adapter_stats *stats = &adapter->dma_stats;
int16_t dma_dev_id;
uint16_t nb_enqueued = 0;
uint16_t nb_ops_flushed = 0;
@@ -563,6 +570,8 @@ edma_adapter_enq_flush(struct event_dma_adapter *adapter)
if (!nb_ops_flushed)
adapter->stop_enq_to_dma_dev = false;
+ stats->dma_enq_count += nb_enqueued;
+
return nb_enqueued;
}
@@ -574,6 +583,7 @@ edma_adapter_enq_flush(struct event_dma_adapter *adapter)
static int
edma_adapter_enq_run(struct event_dma_adapter *adapter, unsigned int max_enq)
{
+ struct rte_event_dma_adapter_stats *stats = &adapter->dma_stats;
uint8_t event_port_id = adapter->event_port_id;
uint8_t event_dev_id = adapter->eventdev_id;
struct rte_event ev[DMA_BATCH_SIZE];
@@ -593,6 +603,7 @@ edma_adapter_enq_run(struct event_dma_adapter *adapter, unsigned int max_enq)
break;
}
+ stats->event_poll_count++;
n = rte_event_dequeue_burst(event_dev_id, event_port_id, ev, DMA_BATCH_SIZE, 0);
if (!n)
@@ -613,6 +624,7 @@ static inline uint16_t
edma_ops_enqueue_burst(struct event_dma_adapter *adapter, struct rte_event_dma_adapter_op **ops,
uint16_t num)
{
+ struct rte_event_dma_adapter_stats *stats = &adapter->dma_stats;
uint8_t event_port_id = adapter->event_port_id;
uint8_t event_dev_id = adapter->eventdev_id;
struct rte_event events[DMA_BATCH_SIZE];
@@ -652,6 +664,10 @@ edma_ops_enqueue_burst(struct event_dma_adapter *adapter, struct rte_event_dma_a
} while (retry++ < DMA_ADAPTER_MAX_EV_ENQ_RETRIES && nb_enqueued < nb_ev);
+ stats->event_enq_fail_count += nb_ev - nb_enqueued;
+ stats->event_enq_count += nb_enqueued;
+ stats->event_enq_retry_count += retry - 1;
+
return nb_enqueued;
}
@@ -706,6 +722,7 @@ edma_ops_buffer_flush(struct event_dma_adapter *adapter)
static inline unsigned int
edma_adapter_deq_run(struct event_dma_adapter *adapter, unsigned int max_deq)
{
+ struct rte_event_dma_adapter_stats *stats = &adapter->dma_stats;
struct dma_vchan_info *vchan_info;
struct dma_ops_circular_buffer *tq_buf;
struct rte_event_dma_adapter_op *ops;
@@ -743,6 +760,7 @@ edma_adapter_deq_run(struct event_dma_adapter *adapter, unsigned int max_deq)
continue;
done = false;
+ stats->dma_deq_count += n;
tq_buf = &dev_info->tqmap[vchan].dma_buf;
@@ -1305,3 +1323,80 @@ rte_event_dma_adapter_runtime_params_get(uint8_t id,
return 0;
}
+
+int
+rte_event_dma_adapter_stats_get(uint8_t id, struct rte_event_dma_adapter_stats *stats)
+{
+ struct rte_event_dma_adapter_stats dev_stats_sum = {0};
+ struct rte_event_dma_adapter_stats dev_stats;
+ struct event_dma_adapter *adapter;
+ struct dma_device_info *dev_info;
+ struct rte_eventdev *dev;
+ uint16_t num_dma_dev;
+ uint32_t i;
+ int ret;
+
+ EVENT_DMA_ADAPTER_ID_VALID_OR_ERR_RET(id, -EINVAL);
+
+ adapter = edma_id_to_adapter(id);
+ if (adapter == NULL || stats == NULL)
+ return -EINVAL;
+
+ num_dma_dev = rte_dma_count_avail();
+ dev = &rte_eventdevs[adapter->eventdev_id];
+ memset(stats, 0, sizeof(*stats));
+ for (i = 0; i < num_dma_dev; i++) {
+ dev_info = &adapter->dma_devs[i];
+
+ if (dev_info->internal_event_port == 0 ||
+ dev->dev_ops->dma_adapter_stats_get == NULL)
+ continue;
+
+ ret = (*dev->dev_ops->dma_adapter_stats_get)(dev, i, &dev_stats);
+ if (ret)
+ continue;
+
+ dev_stats_sum.dma_deq_count += dev_stats.dma_deq_count;
+ dev_stats_sum.event_enq_count += dev_stats.event_enq_count;
+ }
+
+ if (adapter->service_initialized)
+ *stats = adapter->dma_stats;
+
+ stats->dma_deq_count += dev_stats_sum.dma_deq_count;
+ stats->event_enq_count += dev_stats_sum.event_enq_count;
+
+ return 0;
+}
+
+int
+rte_event_dma_adapter_stats_reset(uint8_t id)
+{
+ struct event_dma_adapter *adapter;
+ struct dma_device_info *dev_info;
+ struct rte_eventdev *dev;
+ uint16_t num_dma_dev;
+ uint32_t i;
+
+ EVENT_DMA_ADAPTER_ID_VALID_OR_ERR_RET(id, -EINVAL);
+
+ adapter = edma_id_to_adapter(id);
+ if (adapter == NULL)
+ return -EINVAL;
+
+ num_dma_dev = rte_dma_count_avail();
+ dev = &rte_eventdevs[adapter->eventdev_id];
+ for (i = 0; i < num_dma_dev; i++) {
+ dev_info = &adapter->dma_devs[i];
+
+ if (dev_info->internal_event_port == 0 ||
+ dev->dev_ops->dma_adapter_stats_reset == NULL)
+ continue;
+
+ (*dev->dev_ops->dma_adapter_stats_reset)(dev, i);
+ }
+
+ memset(&adapter->dma_stats, 0, sizeof(adapter->dma_stats));
+
+ return 0;
+}
--
2.25.1
^ permalink raw reply [flat|nested] 110+ messages in thread
* [PATCH v2 10/12] eventdev: add support for DMA adapter enqueue
2023-09-22 20:13 ` [PATCH v2 00/12] event DMA adapter library support Amit Prakash Shukla
` (8 preceding siblings ...)
2023-09-22 20:13 ` [PATCH v2 09/12] eventdev: add support for DMA adapter stats Amit Prakash Shukla
@ 2023-09-22 20:13 ` Amit Prakash Shukla
2023-09-22 20:13 ` [PATCH v2 11/12] eventdev: add DMA adapter port get Amit Prakash Shukla
` (3 subsequent siblings)
13 siblings, 0 replies; 110+ messages in thread
From: Amit Prakash Shukla @ 2023-09-22 20:13 UTC (permalink / raw)
To: Amit Prakash Shukla, Jerin Jacob
Cc: dev, fengchengwen, kevin.laatz, bruce.richardson, conor.walsh,
vattunuru, g.singh, sachin.saxena, hemant.agrawal, cheng1.jiang,
ndabilpuram, anoobj, mb
Added support for DMA adapter enqueue.
Signed-off-by: Amit Prakash Shukla <amitprakashs@marvell.com>
---
lib/eventdev/rte_event_dma_adapter.c | 13 +++++++++++++
1 file changed, 13 insertions(+)
diff --git a/lib/eventdev/rte_event_dma_adapter.c b/lib/eventdev/rte_event_dma_adapter.c
index 06a09371d2..b55b40cd46 100644
--- a/lib/eventdev/rte_event_dma_adapter.c
+++ b/lib/eventdev/rte_event_dma_adapter.c
@@ -1400,3 +1400,16 @@ rte_event_dma_adapter_stats_reset(uint8_t id)
return 0;
}
+
+uint16_t
+rte_event_dma_adapter_enqueue(uint8_t dev_id, uint8_t port_id, struct rte_event ev[],
+ uint16_t nb_events)
+{
+ const struct rte_event_fp_ops *fp_ops;
+ void *port;
+
+ fp_ops = &rte_event_fp_ops[dev_id];
+ port = fp_ops->data[port_id];
+
+ return fp_ops->dma_enqueue(port, ev, nb_events);
+}
--
2.25.1
^ permalink raw reply [flat|nested] 110+ messages in thread
* [PATCH v2 11/12] eventdev: add DMA adapter port get
2023-09-22 20:13 ` [PATCH v2 00/12] event DMA adapter library support Amit Prakash Shukla
` (9 preceding siblings ...)
2023-09-22 20:13 ` [PATCH v2 10/12] eventdev: add support for DMA adapter enqueue Amit Prakash Shukla
@ 2023-09-22 20:13 ` Amit Prakash Shukla
2023-09-22 20:13 ` [PATCH v2 12/12] app/test: add event DMA adapter auto-test Amit Prakash Shukla
` (2 subsequent siblings)
13 siblings, 0 replies; 110+ messages in thread
From: Amit Prakash Shukla @ 2023-09-22 20:13 UTC (permalink / raw)
To: Amit Prakash Shukla, Jerin Jacob
Cc: dev, fengchengwen, kevin.laatz, bruce.richardson, conor.walsh,
vattunuru, g.singh, sachin.saxena, hemant.agrawal, cheng1.jiang,
ndabilpuram, anoobj, mb
Added support for DMA adapter port get.
Signed-off-by: Amit Prakash Shukla <amitprakashs@marvell.com>
---
lib/eventdev/rte_event_dma_adapter.c | 16 ++++++++++++++++
1 file changed, 16 insertions(+)
diff --git a/lib/eventdev/rte_event_dma_adapter.c b/lib/eventdev/rte_event_dma_adapter.c
index b55b40cd46..5f52977543 100644
--- a/lib/eventdev/rte_event_dma_adapter.c
+++ b/lib/eventdev/rte_event_dma_adapter.c
@@ -472,6 +472,22 @@ rte_event_dma_adapter_free(uint8_t id)
return 0;
}
+int
+rte_event_dma_adapter_event_port_get(uint8_t id, uint8_t *event_port_id)
+{
+ struct event_dma_adapter *adapter;
+
+ EVENT_DMA_ADAPTER_ID_VALID_OR_ERR_RET(id, -EINVAL);
+
+ adapter = edma_id_to_adapter(id);
+ if (adapter == NULL || event_port_id == NULL)
+ return -EINVAL;
+
+ *event_port_id = adapter->event_port_id;
+
+ return 0;
+}
+
static inline unsigned int
edma_enq_to_dma_dev(struct event_dma_adapter *adapter, struct rte_event *ev, unsigned int cnt)
{
--
2.25.1
^ permalink raw reply [flat|nested] 110+ messages in thread
* [PATCH v2 12/12] app/test: add event DMA adapter auto-test
2023-09-22 20:13 ` [PATCH v2 00/12] event DMA adapter library support Amit Prakash Shukla
` (10 preceding siblings ...)
2023-09-22 20:13 ` [PATCH v2 11/12] eventdev: add DMA adapter port get Amit Prakash Shukla
@ 2023-09-22 20:13 ` Amit Prakash Shukla
2023-09-23 13:34 ` [PATCH v3 00/12] event DMA adapter library support Amit Prakash Shukla
2023-09-26 5:06 ` [PATCH v2 " Jerin Jacob
13 siblings, 0 replies; 110+ messages in thread
From: Amit Prakash Shukla @ 2023-09-22 20:13 UTC (permalink / raw)
Cc: dev, jerinj, fengchengwen, kevin.laatz, bruce.richardson,
conor.walsh, vattunuru, g.singh, sachin.saxena, hemant.agrawal,
cheng1.jiang, ndabilpuram, anoobj, mb, Amit Prakash Shukla
Added testsuite to test the dma adapter functionality.
The testsuite detects event and DMA device capability
and accordingly dma adapter is configured and modes are
tested.
Signed-off-by: Amit Prakash Shukla <amitprakashs@marvell.com>
---
app/test/meson.build | 1 +
app/test/test_event_dma_adapter.c | 808 ++++++++++++++++++++++++++++++
2 files changed, 809 insertions(+)
create mode 100644 app/test/test_event_dma_adapter.c
diff --git a/app/test/meson.build b/app/test/meson.build
index 05bae9216d..eccd3b72d8 100644
--- a/app/test/meson.build
+++ b/app/test/meson.build
@@ -66,6 +66,7 @@ source_file_deps = {
'test_errno.c': [],
'test_ethdev_link.c': ['ethdev'],
'test_event_crypto_adapter.c': ['cryptodev', 'eventdev', 'bus_vdev'],
+ 'test_event_dma_adapter.c': ['dmadev', 'eventdev'],
'test_event_eth_rx_adapter.c': ['ethdev', 'eventdev', 'bus_vdev'],
'test_event_eth_tx_adapter.c': ['bus_vdev', 'ethdev', 'net_ring', 'eventdev'],
'test_event_ring.c': ['eventdev'],
diff --git a/app/test/test_event_dma_adapter.c b/app/test/test_event_dma_adapter.c
new file mode 100644
index 0000000000..801294c118
--- /dev/null
+++ b/app/test/test_event_dma_adapter.c
@@ -0,0 +1,808 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright (c) 2023 Marvell.
+ */
+
+#include "test.h"
+#include <string.h>
+#include <rte_common.h>
+#include <rte_malloc.h>
+#include <rte_mempool.h>
+#include <rte_mbuf.h>
+#include <rte_random.h>
+
+#ifdef RTE_EXEC_ENV_WINDOWS
+static int
+test_event_dma_adapter(void)
+{
+ printf("event_dma_adapter not supported on Windows, skipping test\n");
+ return TEST_SKIPPED;
+}
+
+#else
+
+#include <rte_bus_vdev.h>
+#include <rte_dmadev.h>
+#include <rte_eventdev.h>
+#include <rte_event_dma_adapter.h>
+#include <rte_service.h>
+
+#define NUM_MBUFS (8191)
+#define MBUF_CACHE_SIZE (256)
+#define TEST_APP_PORT_ID 0
+#define TEST_APP_EV_QUEUE_ID 0
+#define TEST_APP_EV_PRIORITY 0
+#define TEST_APP_EV_FLOWID 0xAABB
+#define TEST_DMA_EV_QUEUE_ID 1
+#define TEST_ADAPTER_ID 0
+#define TEST_DMA_DEV_ID 0
+#define TEST_DMA_VCHAN_ID 0
+#define PACKET_LENGTH 1024
+#define NB_TEST_PORTS 1
+#define NB_TEST_QUEUES 2
+#define NUM_CORES 2
+#define DMA_OP_POOL_SIZE 128
+#define TEST_MAX_OP 128
+#define TEST_RINGSIZE 512
+
+#define MBUF_SIZE (RTE_PKTMBUF_HEADROOM + PACKET_LENGTH)
+
+/* Handle log statements in same manner as test macros */
+#define LOG_DBG(...) RTE_LOG(DEBUG, EAL, __VA_ARGS__)
+
+struct event_dma_adapter_test_params {
+ struct rte_mempool *src_mbuf_pool;
+ struct rte_mempool *dst_mbuf_pool;
+ struct rte_mempool *op_mpool;
+ uint8_t dma_event_port_id;
+ uint8_t internal_port_op_fwd;
+};
+
+struct rte_event dma_response_info = {
+ .queue_id = TEST_APP_EV_QUEUE_ID,
+ .sched_type = RTE_SCHED_TYPE_ATOMIC,
+ .flow_id = TEST_APP_EV_FLOWID,
+ .priority = TEST_APP_EV_PRIORITY
+};
+
+static struct event_dma_adapter_test_params params;
+static uint8_t dma_adapter_setup_done;
+static uint32_t slcore_id;
+static int evdev;
+
+static int
+send_recv_ev(struct rte_event *ev)
+{
+ struct rte_event recv_ev[TEST_MAX_OP] = {0};
+ struct rte_event_dma_adapter_op *op;
+ uint16_t nb_enqueued = 0;
+ int ret, i = 0;
+
+ if (params.internal_port_op_fwd) {
+ nb_enqueued = rte_event_dma_adapter_enqueue(evdev, TEST_APP_PORT_ID, ev,
+ TEST_MAX_OP);
+ } else {
+ while (nb_enqueued < TEST_MAX_OP) {
+ nb_enqueued += rte_event_enqueue_burst(evdev, TEST_APP_PORT_ID,
+ &ev[nb_enqueued], TEST_MAX_OP -
+ nb_enqueued);
+ }
+ }
+
+ TEST_ASSERT_EQUAL(nb_enqueued, TEST_MAX_OP, "Failed to send event to dma adapter\n");
+
+ while (i < TEST_MAX_OP) {
+ if (rte_event_dequeue_burst(evdev, TEST_APP_PORT_ID, &recv_ev[i], 1, 0) != 1)
+ continue;
+
+ op = recv_ev[i].event_ptr;
+ ret = memcmp((uint8_t *)op->src_seg->addr, (uint8_t *)op->dst_seg->addr,
+ op->src_seg->length);
+ TEST_ASSERT_EQUAL(ret, 0, "Data mismatch for dma adapter\n");
+ i++;
+ }
+
+ return TEST_SUCCESS;
+}
+
+static int
+test_dma_adapter_stats(void)
+{
+ struct rte_event_dma_adapter_stats stats;
+
+ rte_event_dma_adapter_stats_get(TEST_ADAPTER_ID, &stats);
+ printf(" +------------------------------------------------------+\n");
+ printf(" + DMA adapter stats for instance %u:\n", TEST_ADAPTER_ID);
+ printf(" + Event port poll count 0x%" PRIx64 "\n",
+ stats.event_poll_count);
+ printf(" + Event dequeue count 0x%" PRIx64 "\n",
+ stats.event_deq_count);
+ printf(" + DMA dev enqueue count 0x%" PRIx64 "\n",
+ stats.dma_enq_count);
+ printf(" + DMA dev enqueue failed count 0x%" PRIx64 "\n",
+ stats.dma_enq_fail_count);
+ printf(" + DMA dev dequeue count 0x%" PRIx64 "\n",
+ stats.dma_deq_count);
+ printf(" + Event enqueue count 0x%" PRIx64 "\n",
+ stats.event_enq_count);
+ printf(" + Event enqueue retry count 0x%" PRIx64 "\n",
+ stats.event_enq_retry_count);
+ printf(" + Event enqueue fail count 0x%" PRIx64 "\n",
+ stats.event_enq_fail_count);
+ printf(" +------------------------------------------------------+\n");
+
+ rte_event_dma_adapter_stats_reset(TEST_ADAPTER_ID);
+ return TEST_SUCCESS;
+}
+
+static int
+test_dma_adapter_params(void)
+{
+ struct rte_event_dma_adapter_runtime_params in_params;
+ struct rte_event_dma_adapter_runtime_params out_params;
+ struct rte_event event;
+ uint32_t cap;
+ int err, rc;
+
+ err = rte_event_dma_adapter_caps_get(evdev, TEST_DMA_DEV_ID, &cap);
+ TEST_ASSERT_SUCCESS(err, "Failed to get adapter capabilities\n");
+
+ if (cap & RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_VCHAN_EV_BIND) {
+ err = rte_event_dma_adapter_vchan_add(TEST_ADAPTER_ID, TEST_DMA_DEV_ID,
+ TEST_DMA_VCHAN_ID, &event);
+ } else
+ err = rte_event_dma_adapter_vchan_add(TEST_ADAPTER_ID, TEST_DMA_DEV_ID,
+ TEST_DMA_VCHAN_ID, NULL);
+
+ TEST_ASSERT_SUCCESS(err, "Failed to add vchan\n");
+
+ err = rte_event_dma_adapter_runtime_params_init(&in_params);
+ TEST_ASSERT(err == 0, "Expected 0 got %d", err);
+ err = rte_event_dma_adapter_runtime_params_init(&out_params);
+ TEST_ASSERT(err == 0, "Expected 0 got %d", err);
+
+ /* Case 1: Get the default value of mbufs processed by adapter */
+ err = rte_event_dma_adapter_runtime_params_get(TEST_ADAPTER_ID, &out_params);
+ if (err == -ENOTSUP) {
+ rc = TEST_SKIPPED;
+ goto vchan_del;
+ }
+ TEST_ASSERT(err == 0, "Expected 0 got %d", err);
+
+ /* Case 2: Set max_nb = 32 (=BATCH_SEIZE) */
+ in_params.max_nb = 32;
+
+ err = rte_event_dma_adapter_runtime_params_set(TEST_ADAPTER_ID, &in_params);
+ TEST_ASSERT(err == 0, "Expected 0 got %d", err);
+
+ err = rte_event_dma_adapter_runtime_params_get(TEST_ADAPTER_ID, &out_params);
+ TEST_ASSERT(err == 0, "Expected 0 got %d", err);
+ TEST_ASSERT(in_params.max_nb == out_params.max_nb, "Expected %u got %u",
+ in_params.max_nb, out_params.max_nb);
+
+ /* Case 3: Set max_nb = 192 */
+ in_params.max_nb = 192;
+
+ err = rte_event_dma_adapter_runtime_params_set(TEST_ADAPTER_ID, &in_params);
+ TEST_ASSERT(err == 0, "Expected 0 got %d", err);
+
+ err = rte_event_dma_adapter_runtime_params_get(TEST_ADAPTER_ID, &out_params);
+ TEST_ASSERT(err == 0, "Expected 0 got %d", err);
+ TEST_ASSERT(in_params.max_nb == out_params.max_nb, "Expected %u got %u",
+ in_params.max_nb, out_params.max_nb);
+
+ /* Case 4: Set max_nb = 256 */
+ in_params.max_nb = 256;
+
+ err = rte_event_dma_adapter_runtime_params_set(TEST_ADAPTER_ID, &in_params);
+ TEST_ASSERT(err == 0, "Expected 0 got %d", err);
+
+ err = rte_event_dma_adapter_runtime_params_get(TEST_ADAPTER_ID, &out_params);
+ TEST_ASSERT(err == 0, "Expected 0 got %d", err);
+ TEST_ASSERT(in_params.max_nb == out_params.max_nb, "Expected %u got %u",
+ in_params.max_nb, out_params.max_nb);
+
+ /* Case 5: Set max_nb = 30(<BATCH_SIZE) */
+ in_params.max_nb = 30;
+
+ err = rte_event_dma_adapter_runtime_params_set(TEST_ADAPTER_ID, &in_params);
+ TEST_ASSERT(err == 0, "Expected 0 got %d", err);
+
+ err = rte_event_dma_adapter_runtime_params_get(TEST_ADAPTER_ID, &out_params);
+ TEST_ASSERT(err == 0, "Expected 0 got %d", err);
+ TEST_ASSERT(in_params.max_nb == out_params.max_nb, "Expected %u got %u",
+ in_params.max_nb, out_params.max_nb);
+
+ /* Case 6: Set max_nb = 512 */
+ in_params.max_nb = 512;
+
+ err = rte_event_dma_adapter_runtime_params_set(TEST_ADAPTER_ID, &in_params);
+ TEST_ASSERT(err == 0, "Expected 0 got %d", err);
+
+ err = rte_event_dma_adapter_runtime_params_get(TEST_ADAPTER_ID, &out_params);
+ TEST_ASSERT(err == 0, "Expected 0 got %d", err);
+ TEST_ASSERT(in_params.max_nb == out_params.max_nb, "Expected %u got %u",
+ in_params.max_nb, out_params.max_nb);
+
+ rc = TEST_SUCCESS;
+vchan_del:
+ err = rte_event_dma_adapter_vchan_del(TEST_ADAPTER_ID, TEST_DMA_DEV_ID,
+ TEST_DMA_VCHAN_ID);
+ TEST_ASSERT_SUCCESS(err, "Failed to delete vchan\n");
+
+ return rc;
+}
+
+static int
+test_op_forward_mode(void)
+{
+ struct rte_mbuf *src_mbuf[TEST_MAX_OP];
+ struct rte_mbuf *dst_mbuf[TEST_MAX_OP];
+ struct rte_event ev[TEST_MAX_OP] = {0};
+ struct rte_event response_info;
+ struct rte_event_dma_adapter_op *op;
+ int ret, i;
+
+ ret = rte_pktmbuf_alloc_bulk(params.src_mbuf_pool, src_mbuf, TEST_MAX_OP);
+ TEST_ASSERT_SUCCESS(ret, "alloc src mbufs failed.\n");
+
+ ret = rte_pktmbuf_alloc_bulk(params.dst_mbuf_pool, dst_mbuf, TEST_MAX_OP);
+ TEST_ASSERT_SUCCESS(ret, "alloc dst mbufs failed.\n");
+
+ for (i = 0; i < TEST_MAX_OP; i++) {
+ memset(rte_pktmbuf_mtod(src_mbuf[i], void *), rte_rand(), PACKET_LENGTH);
+ memset(rte_pktmbuf_mtod(dst_mbuf[i], void *), 0, PACKET_LENGTH);
+ }
+
+ for (i = 0; i < TEST_MAX_OP; i++) {
+ rte_mempool_get(params.op_mpool, (void **)&op);
+ TEST_ASSERT_NOT_NULL(op, "Failed to allocate dma operation struct\n");
+
+ op->src_seg = rte_malloc(NULL, sizeof(struct rte_dma_sge), 0);
+ op->dst_seg = rte_malloc(NULL, sizeof(struct rte_dma_sge), 0);
+
+ /* Update Op */
+ op->src_seg->addr = rte_pktmbuf_iova(src_mbuf[i]);
+ op->dst_seg->addr = rte_pktmbuf_iova(dst_mbuf[i]);
+ op->src_seg->length = PACKET_LENGTH;
+ op->dst_seg->length = PACKET_LENGTH;
+ op->nb_src = 1;
+ op->nb_dst = 1;
+ op->flags = RTE_DMA_OP_FLAG_SUBMIT;
+ op->op_mp = params.op_mpool;
+ op->dma_dev_id = TEST_DMA_DEV_ID;
+ op->vchan = TEST_DMA_VCHAN_ID;
+
+ response_info.event = dma_response_info.event;
+ rte_memcpy((uint8_t *)op + sizeof(struct rte_event_dma_adapter_op), &response_info,
+ sizeof(struct rte_event));
+
+ /* Fill in event info and update event_ptr with rte_event_dma_adapter_op */
+ memset(&ev[i], 0, sizeof(struct rte_event));
+ ev[i].event = 0;
+ ev[i].event_type = RTE_EVENT_TYPE_DMADEV;
+ ev[i].queue_id = TEST_DMA_EV_QUEUE_ID;
+ ev[i].sched_type = RTE_SCHED_TYPE_ATOMIC;
+ ev[i].flow_id = 0xAABB;
+ ev[i].event_ptr = op;
+ }
+
+ ret = send_recv_ev(ev);
+ TEST_ASSERT_SUCCESS(ret, "Failed to send/receive event to dma adapter\n");
+
+ test_dma_adapter_stats();
+
+ for (i = 0; i < TEST_MAX_OP; i++) {
+ op = ev[i].event_ptr;
+ ret = memcmp((uint8_t *)op->src_seg->addr, (uint8_t *)op->dst_seg->addr,
+ op->src_seg->length);
+ TEST_ASSERT_EQUAL(ret, 0, "Data mismatch for dma adapter\n");
+
+ rte_free(op->src_seg);
+ rte_free(op->dst_seg);
+ rte_mempool_put(op->op_mp, op);
+ }
+
+ rte_pktmbuf_free_bulk(src_mbuf, TEST_MAX_OP);
+ rte_pktmbuf_free_bulk(dst_mbuf, TEST_MAX_OP);
+
+ return TEST_SUCCESS;
+}
+
+static int
+map_adapter_service_core(void)
+{
+ uint32_t adapter_service_id;
+ int ret;
+
+ if (rte_event_dma_adapter_service_id_get(TEST_ADAPTER_ID, &adapter_service_id) == 0) {
+ uint32_t core_list[NUM_CORES];
+
+ ret = rte_service_lcore_list(core_list, NUM_CORES);
+ TEST_ASSERT(ret >= 0, "Failed to get service core list!");
+
+ if (core_list[0] != slcore_id) {
+ TEST_ASSERT_SUCCESS(rte_service_lcore_add(slcore_id),
+ "Failed to add service core");
+ TEST_ASSERT_SUCCESS(rte_service_lcore_start(slcore_id),
+ "Failed to start service core");
+ }
+
+ TEST_ASSERT_SUCCESS(rte_service_map_lcore_set(
+ adapter_service_id, slcore_id, 1),
+ "Failed to map adapter service");
+ }
+
+ return TEST_SUCCESS;
+}
+
+static int
+test_with_op_forward_mode(void)
+{
+ uint32_t cap;
+ int ret;
+
+ ret = rte_event_dma_adapter_caps_get(evdev, TEST_DMA_DEV_ID, &cap);
+ TEST_ASSERT_SUCCESS(ret, "Failed to get adapter capabilities\n");
+
+ if (!(cap & RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP_FWD) &&
+ !(cap & RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP_NEW))
+ map_adapter_service_core();
+ else {
+ if (!(cap & RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP_FWD))
+ return TEST_SKIPPED;
+ }
+
+ TEST_ASSERT_SUCCESS(rte_event_dma_adapter_start(TEST_ADAPTER_ID),
+ "Failed to start event dma adapter");
+
+ ret = test_op_forward_mode();
+ TEST_ASSERT_SUCCESS(ret, "DMA - FORWARD mode test failed\n");
+ return TEST_SUCCESS;
+}
+
+static int
+configure_dmadev(void)
+{
+ const struct rte_dma_conf conf = { .nb_vchans = 1};
+ const struct rte_dma_vchan_conf qconf = {
+ .direction = RTE_DMA_DIR_MEM_TO_MEM,
+ .nb_desc = TEST_RINGSIZE,
+ };
+ struct rte_dma_info info;
+ unsigned int elt_size;
+ int ret;
+
+ ret = rte_dma_count_avail();
+ RTE_TEST_ASSERT_FAIL(ret, "No dma devices found!\n");
+
+ ret = rte_dma_info_get(TEST_DMA_DEV_ID, &info);
+ TEST_ASSERT_SUCCESS(ret, "Error with rte_dma_info_get()\n");
+
+ if (info.max_vchans < 1)
+ RTE_LOG(ERR, USER1, "Error, no channels available on device id %u\n",
+ TEST_DMA_DEV_ID);
+
+ if (rte_dma_configure(TEST_DMA_DEV_ID, &conf) != 0)
+ RTE_LOG(ERR, USER1, "Error with rte_dma_configure()\n");
+
+ if (rte_dma_vchan_setup(TEST_DMA_DEV_ID, TEST_DMA_VCHAN_ID, &qconf) < 0)
+ RTE_LOG(ERR, USER1, "Error with vchan configuration\n");
+
+ ret = rte_dma_info_get(TEST_DMA_DEV_ID, &info);
+ if (ret != 0 || info.nb_vchans != 1)
+ RTE_LOG(ERR, USER1, "Error, no configured vhcan reported on device id %u\n",
+ TEST_DMA_DEV_ID);
+
+ params.src_mbuf_pool = rte_pktmbuf_pool_create("DMA_ADAPTER_SRC_MBUFPOOL", NUM_MBUFS,
+ MBUF_CACHE_SIZE, 0, MBUF_SIZE,
+ rte_socket_id());
+ RTE_TEST_ASSERT_NOT_NULL(params.src_mbuf_pool, "Can't create DMA_SRC_MBUFPOOL\n");
+
+ params.dst_mbuf_pool = rte_pktmbuf_pool_create("DMA_ADAPTER_DST_MBUFPOOL", NUM_MBUFS,
+ MBUF_CACHE_SIZE, 0, MBUF_SIZE,
+ rte_socket_id());
+ RTE_TEST_ASSERT_NOT_NULL(params.dst_mbuf_pool, "Can't create DMA_DST_MBUFPOOL\n");
+
+ elt_size = sizeof(struct rte_event_dma_adapter_op) + sizeof(struct rte_event);
+ params.op_mpool = rte_mempool_create("EVENT_DMA_OP_POOL", DMA_OP_POOL_SIZE, elt_size, 0,
+ 0, NULL, NULL, NULL, NULL, rte_socket_id(), 0);
+ RTE_TEST_ASSERT_NOT_NULL(params.op_mpool, "Can't create DMA_OP_POOL\n");
+
+ return TEST_SUCCESS;
+}
+
+static inline void
+evdev_set_conf_values(struct rte_event_dev_config *dev_conf, struct rte_event_dev_info *info)
+{
+ memset(dev_conf, 0, sizeof(struct rte_event_dev_config));
+ dev_conf->dequeue_timeout_ns = info->min_dequeue_timeout_ns;
+ dev_conf->nb_event_ports = NB_TEST_PORTS;
+ dev_conf->nb_event_queues = NB_TEST_QUEUES;
+ dev_conf->nb_event_queue_flows = info->max_event_queue_flows;
+ dev_conf->nb_event_port_dequeue_depth =
+ info->max_event_port_dequeue_depth;
+ dev_conf->nb_event_port_enqueue_depth =
+ info->max_event_port_enqueue_depth;
+ dev_conf->nb_event_port_enqueue_depth =
+ info->max_event_port_enqueue_depth;
+ dev_conf->nb_events_limit =
+ info->max_num_events;
+}
+
+static int
+configure_eventdev(void)
+{
+ struct rte_event_queue_conf queue_conf;
+ struct rte_event_dev_config devconf;
+ struct rte_event_dev_info info;
+ uint32_t queue_count;
+ uint32_t port_count;
+ uint8_t qid;
+ int ret;
+
+ if (!rte_event_dev_count()) {
+ /* If there is no hardware eventdev, or no software vdev was
+ * specified on the command line, create an instance of
+ * event_sw.
+ */
+ LOG_DBG("Failed to find a valid event device... "
+ "testing with event_sw device\n");
+ TEST_ASSERT_SUCCESS(rte_vdev_init("event_sw0", NULL),
+ "Error creating eventdev");
+ evdev = rte_event_dev_get_dev_id("event_sw0");
+ }
+
+ ret = rte_event_dev_info_get(evdev, &info);
+ TEST_ASSERT_SUCCESS(ret, "Failed to get event dev info\n");
+
+ evdev_set_conf_values(&devconf, &info);
+
+ ret = rte_event_dev_configure(evdev, &devconf);
+ TEST_ASSERT_SUCCESS(ret, "Failed to configure eventdev\n");
+
+ /* Set up event queue */
+ ret = rte_event_dev_attr_get(evdev, RTE_EVENT_DEV_ATTR_QUEUE_COUNT, &queue_count);
+ TEST_ASSERT_SUCCESS(ret, "Queue count get failed\n");
+ TEST_ASSERT_EQUAL(queue_count, 2, "Unexpected queue count\n");
+
+ qid = TEST_APP_EV_QUEUE_ID;
+ ret = rte_event_queue_setup(evdev, qid, NULL);
+ TEST_ASSERT_SUCCESS(ret, "Failed to setup queue=%d\n", qid);
+
+ queue_conf.nb_atomic_flows = info.max_event_queue_flows;
+ queue_conf.nb_atomic_order_sequences = 32;
+ queue_conf.schedule_type = RTE_SCHED_TYPE_ATOMIC;
+ queue_conf.priority = RTE_EVENT_DEV_PRIORITY_HIGHEST;
+ queue_conf.event_queue_cfg = RTE_EVENT_QUEUE_CFG_SINGLE_LINK;
+
+ qid = TEST_DMA_EV_QUEUE_ID;
+ ret = rte_event_queue_setup(evdev, qid, &queue_conf);
+ TEST_ASSERT_SUCCESS(ret, "Failed to setup queue=%u\n", qid);
+
+ /* Set up event port */
+ ret = rte_event_dev_attr_get(evdev, RTE_EVENT_DEV_ATTR_PORT_COUNT,
+ &port_count);
+ TEST_ASSERT_SUCCESS(ret, "Port count get failed\n");
+ TEST_ASSERT_EQUAL(port_count, 1, "Unexpected port count\n");
+
+ ret = rte_event_port_setup(evdev, TEST_APP_PORT_ID, NULL);
+ TEST_ASSERT_SUCCESS(ret, "Failed to setup port=%d\n",
+ TEST_APP_PORT_ID);
+
+ qid = TEST_APP_EV_QUEUE_ID;
+ ret = rte_event_port_link(evdev, TEST_APP_PORT_ID, &qid, NULL, 1);
+ TEST_ASSERT(ret >= 0, "Failed to link queue port=%d\n",
+ TEST_APP_PORT_ID);
+
+ return TEST_SUCCESS;
+}
+
+static void
+test_dma_adapter_free(void)
+{
+ rte_event_dma_adapter_free(TEST_ADAPTER_ID);
+}
+
+static int
+test_dma_adapter_create(void)
+{
+ struct rte_event_dev_info evdev_info = {0};
+ struct rte_event_port_conf conf = {0};
+ int ret;
+
+ ret = rte_event_dev_info_get(evdev, &evdev_info);
+ TEST_ASSERT_SUCCESS(ret, "Failed to create event dma adapter\n");
+
+ conf.new_event_threshold = evdev_info.max_num_events;
+ conf.dequeue_depth = evdev_info.max_event_port_dequeue_depth;
+ conf.enqueue_depth = evdev_info.max_event_port_enqueue_depth;
+
+ /* Create adapter with default port creation callback */
+ ret = rte_event_dma_adapter_create(TEST_ADAPTER_ID, evdev, &conf, 0);
+ TEST_ASSERT_SUCCESS(ret, "Failed to create event dma adapter\n");
+
+ return TEST_SUCCESS;
+}
+
+static int
+test_dma_adapter_qp_add_del(void)
+{
+ struct rte_event event;
+ uint32_t cap;
+ int ret;
+
+ ret = rte_event_dma_adapter_caps_get(evdev, TEST_DMA_DEV_ID, &cap);
+ TEST_ASSERT_SUCCESS(ret, "Failed to get adapter capabilities\n");
+
+ if (cap & RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_VCHAN_EV_BIND) {
+ ret = rte_event_dma_adapter_vchan_add(TEST_ADAPTER_ID, TEST_DMA_DEV_ID,
+ TEST_DMA_VCHAN_ID, &event);
+ } else
+ ret = rte_event_dma_adapter_vchan_add(TEST_ADAPTER_ID, TEST_DMA_DEV_ID,
+ TEST_DMA_VCHAN_ID, NULL);
+
+ TEST_ASSERT_SUCCESS(ret, "Failed to create add vchan\n");
+
+ ret = rte_event_dma_adapter_vchan_del(TEST_ADAPTER_ID, TEST_DMA_DEV_ID,
+ TEST_DMA_VCHAN_ID);
+ TEST_ASSERT_SUCCESS(ret, "Failed to delete vchan\n");
+
+ return TEST_SUCCESS;
+}
+
+static int
+configure_event_dma_adapter(enum rte_event_dma_adapter_mode mode)
+{
+ struct rte_event_dev_info evdev_info = {0};
+ struct rte_event_port_conf conf = {0};
+ struct rte_event event;
+ uint32_t cap;
+ int ret;
+
+ ret = rte_event_dma_adapter_caps_get(evdev, TEST_DMA_DEV_ID, &cap);
+ TEST_ASSERT_SUCCESS(ret, "Failed to get adapter capabilities\n");
+
+ /* Skip mode and capability mismatch check for SW eventdev */
+ if (!(cap & RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP_NEW) &&
+ !(cap & RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP_FWD) &&
+ !(cap & RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_VCHAN_EV_BIND))
+ goto adapter_create;
+
+ if (mode == RTE_EVENT_DMA_ADAPTER_OP_FORWARD) {
+ if (cap & RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP_FWD)
+ params.internal_port_op_fwd = 1;
+ else
+ return -ENOTSUP;
+ }
+
+adapter_create:
+ ret = rte_event_dev_info_get(evdev, &evdev_info);
+ TEST_ASSERT_SUCCESS(ret, "Failed to create event dma adapter\n");
+
+ conf.new_event_threshold = evdev_info.max_num_events;
+ conf.dequeue_depth = evdev_info.max_event_port_dequeue_depth;
+ conf.enqueue_depth = evdev_info.max_event_port_enqueue_depth;
+
+ /* Create adapter with default port creation callback */
+ ret = rte_event_dma_adapter_create(TEST_ADAPTER_ID, evdev, &conf, mode);
+ TEST_ASSERT_SUCCESS(ret, "Failed to create event dma adapter\n");
+
+ if (cap & RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_VCHAN_EV_BIND) {
+ ret = rte_event_dma_adapter_vchan_add(TEST_ADAPTER_ID, TEST_DMA_DEV_ID,
+ TEST_DMA_VCHAN_ID, &event);
+ } else
+ ret = rte_event_dma_adapter_vchan_add(TEST_ADAPTER_ID, TEST_DMA_DEV_ID,
+ TEST_DMA_VCHAN_ID, NULL);
+
+ TEST_ASSERT_SUCCESS(ret, "Failed to add vchan\n");
+
+ if (!params.internal_port_op_fwd) {
+ ret = rte_event_dma_adapter_event_port_get(TEST_ADAPTER_ID,
+ ¶ms.dma_event_port_id);
+ TEST_ASSERT_SUCCESS(ret, "Failed to get event port\n");
+ }
+
+ return TEST_SUCCESS;
+}
+
+static void
+test_dma_adapter_stop(void)
+{
+ uint32_t evdev_service_id, adapter_service_id;
+
+ /* retrieve service ids & stop services */
+ if (rte_event_dma_adapter_service_id_get(TEST_ADAPTER_ID,
+ &adapter_service_id) == 0) {
+ rte_service_runstate_set(adapter_service_id, 0);
+ rte_service_lcore_stop(slcore_id);
+ rte_service_lcore_del(slcore_id);
+ rte_event_dma_adapter_stop(TEST_ADAPTER_ID);
+ }
+
+ if (rte_event_dev_service_id_get(evdev, &evdev_service_id) == 0) {
+ rte_service_runstate_set(evdev_service_id, 0);
+ rte_service_lcore_stop(slcore_id);
+ rte_service_lcore_del(slcore_id);
+ rte_dma_stop(TEST_DMA_DEV_ID);
+ rte_event_dev_stop(evdev);
+ } else {
+ rte_dma_stop(TEST_DMA_DEV_ID);
+ rte_event_dev_stop(evdev);
+ }
+}
+
+static int
+test_dma_adapter_conf(enum rte_event_dma_adapter_mode mode)
+{
+ uint32_t evdev_service_id;
+ uint8_t qid;
+ int ret;
+
+ if (!dma_adapter_setup_done) {
+ ret = configure_event_dma_adapter(mode);
+ if (ret)
+ return ret;
+ if (!params.internal_port_op_fwd) {
+ qid = TEST_DMA_EV_QUEUE_ID;
+ ret = rte_event_port_link(evdev,
+ params.dma_event_port_id, &qid, NULL, 1);
+ TEST_ASSERT(ret >= 0, "Failed to link queue %d "
+ "port=%u\n", qid,
+ params.dma_event_port_id);
+ }
+ dma_adapter_setup_done = 1;
+ }
+
+ /* retrieve service ids */
+ if (rte_event_dev_service_id_get(evdev, &evdev_service_id) == 0) {
+ /* add a service core and start it */
+ TEST_ASSERT_SUCCESS(rte_service_lcore_add(slcore_id),
+ "Failed to add service core");
+ TEST_ASSERT_SUCCESS(rte_service_lcore_start(slcore_id),
+ "Failed to start service core");
+
+ /* map services to it */
+ TEST_ASSERT_SUCCESS(rte_service_map_lcore_set(evdev_service_id,
+ slcore_id, 1), "Failed to map evdev service");
+
+ /* set services to running */
+ TEST_ASSERT_SUCCESS(rte_service_runstate_set(evdev_service_id,
+ 1), "Failed to start evdev service");
+ }
+
+ /* start the eventdev */
+ TEST_ASSERT_SUCCESS(rte_event_dev_start(evdev),
+ "Failed to start event device");
+
+ /* start the dma dev */
+ TEST_ASSERT_SUCCESS(rte_dma_start(TEST_DMA_DEV_ID),
+ "Failed to start dma device");
+
+ return TEST_SUCCESS;
+}
+
+static int
+test_dma_adapter_conf_op_forward_mode(void)
+{
+ enum rte_event_dma_adapter_mode mode;
+
+ mode = RTE_EVENT_DMA_ADAPTER_OP_FORWARD;
+
+ return test_dma_adapter_conf(mode);
+}
+
+static int
+testsuite_setup(void)
+{
+ int ret;
+
+ slcore_id = rte_get_next_lcore(-1, 1, 0);
+ TEST_ASSERT_NOT_EQUAL(slcore_id, RTE_MAX_LCORE, "At least 2 lcores "
+ "are required to run this autotest\n");
+
+ /* Setup and start event device. */
+ ret = configure_eventdev();
+ TEST_ASSERT_SUCCESS(ret, "Failed to setup eventdev\n");
+
+ /* Setup and start dma device. */
+ ret = configure_dmadev();
+ TEST_ASSERT_SUCCESS(ret, "dmadev initialization failed\n");
+
+ return TEST_SUCCESS;
+}
+
+static void
+dma_adapter_teardown(void)
+{
+ int ret;
+
+ ret = rte_event_dma_adapter_stop(TEST_ADAPTER_ID);
+ if (ret < 0)
+ RTE_LOG(ERR, USER1, "Failed to stop adapter!");
+
+ ret = rte_event_dma_adapter_vchan_del(TEST_ADAPTER_ID, TEST_DMA_DEV_ID,
+ TEST_DMA_VCHAN_ID);
+ if (ret < 0)
+ RTE_LOG(ERR, USER1, "Failed to delete vchan!");
+
+ ret = rte_event_dma_adapter_free(TEST_ADAPTER_ID);
+ if (ret < 0)
+ RTE_LOG(ERR, USER1, "Failed to free adapter!");
+
+ dma_adapter_setup_done = 0;
+}
+
+static void
+dma_teardown(void)
+{
+ /* Free mbuf mempool */
+ if (params.src_mbuf_pool != NULL) {
+ RTE_LOG(DEBUG, USER1, "DMA_ADAPTER_SRC_MBUFPOOL count %u\n",
+ rte_mempool_avail_count(params.src_mbuf_pool));
+ rte_mempool_free(params.src_mbuf_pool);
+ params.src_mbuf_pool = NULL;
+ }
+
+ if (params.dst_mbuf_pool != NULL) {
+ RTE_LOG(DEBUG, USER1, "DMA_ADAPTER_DST_MBUFPOOL count %u\n",
+ rte_mempool_avail_count(params.dst_mbuf_pool));
+ rte_mempool_free(params.dst_mbuf_pool);
+ params.dst_mbuf_pool = NULL;
+ }
+
+ /* Free ops mempool */
+ if (params.op_mpool != NULL) {
+ RTE_LOG(DEBUG, USER1, "EVENT_DMA_OP_POOL count %u\n",
+ rte_mempool_avail_count(params.op_mpool));
+ rte_mempool_free(params.op_mpool);
+ params.op_mpool = NULL;
+ }
+}
+
+static void
+eventdev_teardown(void)
+{
+ rte_event_dev_stop(evdev);
+}
+
+static void
+testsuite_teardown(void)
+{
+ dma_adapter_teardown();
+ dma_teardown();
+ eventdev_teardown();
+}
+
+static struct unit_test_suite functional_testsuite = {
+ .suite_name = "Event dma adapter test suite",
+ .setup = testsuite_setup,
+ .teardown = testsuite_teardown,
+ .unit_test_cases = {
+
+ TEST_CASE_ST(NULL, test_dma_adapter_free, test_dma_adapter_create),
+
+ TEST_CASE_ST(test_dma_adapter_create, test_dma_adapter_free,
+ test_dma_adapter_qp_add_del),
+
+ TEST_CASE_ST(test_dma_adapter_create, test_dma_adapter_free,
+ test_dma_adapter_stats),
+
+ TEST_CASE_ST(test_dma_adapter_create, test_dma_adapter_free,
+ test_dma_adapter_params),
+
+ TEST_CASE_ST(test_dma_adapter_conf_op_forward_mode, test_dma_adapter_stop,
+ test_with_op_forward_mode),
+
+ TEST_CASES_END() /**< NULL terminate unit test array */
+ }
+};
+
+static int
+test_event_dma_adapter(void)
+{
+ return unit_test_suite_runner(&functional_testsuite);
+}
+
+#endif /* !RTE_EXEC_ENV_WINDOWS */
+
+REGISTER_TEST_COMMAND(event_dma_adapter_autotest, test_event_dma_adapter);
--
2.25.1
^ permalink raw reply [flat|nested] 110+ messages in thread
* [PATCH v3 00/12] event DMA adapter library support
2023-09-22 20:13 ` [PATCH v2 00/12] event DMA adapter library support Amit Prakash Shukla
` (11 preceding siblings ...)
2023-09-22 20:13 ` [PATCH v2 12/12] app/test: add event DMA adapter auto-test Amit Prakash Shukla
@ 2023-09-23 13:34 ` Amit Prakash Shukla
2023-09-23 13:34 ` [PATCH v3 01/12] eventdev: introduce event DMA adapter library Amit Prakash Shukla
` (12 more replies)
2023-09-26 5:06 ` [PATCH v2 " Jerin Jacob
13 siblings, 13 replies; 110+ messages in thread
From: Amit Prakash Shukla @ 2023-09-23 13:34 UTC (permalink / raw)
Cc: dev, jerinj, fengchengwen, kevin.laatz, bruce.richardson,
conor.walsh, vattunuru, g.singh, sachin.saxena, hemant.agrawal,
cheng1.jiang, ndabilpuram, anoobj, mb, Amit Prakash Shukla
This series adds support for event DMA adapter library. API's defined
as part of this library can be used by the application for DMA transfer
of data using event based mechanism.
v3:
- Resolved checkpatch warnings.
- Fixed compilation error on intel.
- Updated git commit message.
v2:
- Resolved review comments.
- Patch split into multiple patches.
Amit Prakash Shukla (12):
eventdev: introduce event DMA adapter library
eventdev: api to get DMA adapter capabilities
eventdev: create and free API for DMA adapter
eventdev: add API support for vchan add and delete
eventdev: add support for DMA adapter service function
eventdev: add support for DMA adapter start and stop
eventdev: add support for DMA adapter service ID get
eventdev: add DMA adapter support for runtime params
eventdev: add support for DMA adapter stats
eventdev: add support for DMA adapter enqueue
eventdev: add DMA adapter event port get
app/test: add event DMA adapter auto-test
MAINTAINERS | 5 +
app/test/meson.build | 1 +
app/test/test_event_dma_adapter.c | 805 ++++++++++
config/rte_config.h | 1 +
doc/api/doxy-api-index.md | 1 +
doc/guides/eventdevs/features/default.ini | 8 +
doc/guides/prog_guide/event_dma_adapter.rst | 264 +++
doc/guides/prog_guide/eventdev.rst | 8 +-
.../img/event_dma_adapter_op_forward.svg | 1086 +++++++++++++
.../img/event_dma_adapter_op_new.svg | 1079 +++++++++++++
doc/guides/prog_guide/index.rst | 1 +
doc/guides/rel_notes/release_23_11.rst | 4 +-
lib/eventdev/eventdev_pmd.h | 175 +-
lib/eventdev/eventdev_private.c | 10 +
lib/eventdev/meson.build | 4 +-
lib/eventdev/rte_event_dma_adapter.c | 1431 +++++++++++++++++
lib/eventdev/rte_event_dma_adapter.h | 582 +++++++
lib/eventdev/rte_eventdev.c | 23 +
lib/eventdev/rte_eventdev.h | 43 +
lib/eventdev/rte_eventdev_core.h | 8 +-
lib/eventdev/version.map | 15 +
lib/meson.build | 2 +-
22 files changed, 5548 insertions(+), 8 deletions(-)
create mode 100644 app/test/test_event_dma_adapter.c
create mode 100644 doc/guides/prog_guide/event_dma_adapter.rst
create mode 100644 doc/guides/prog_guide/img/event_dma_adapter_op_forward.svg
create mode 100644 doc/guides/prog_guide/img/event_dma_adapter_op_new.svg
create mode 100644 lib/eventdev/rte_event_dma_adapter.c
create mode 100644 lib/eventdev/rte_event_dma_adapter.h
--
2.25.1
^ permalink raw reply [flat|nested] 110+ messages in thread
* [PATCH v3 01/12] eventdev: introduce event DMA adapter library
2023-09-23 13:34 ` [PATCH v3 00/12] event DMA adapter library support Amit Prakash Shukla
@ 2023-09-23 13:34 ` Amit Prakash Shukla
2023-09-23 13:34 ` [PATCH v3 02/12] eventdev: api to get DMA adapter capabilities Amit Prakash Shukla
` (11 subsequent siblings)
12 siblings, 0 replies; 110+ messages in thread
From: Amit Prakash Shukla @ 2023-09-23 13:34 UTC (permalink / raw)
To: Thomas Monjalon, Amit Prakash Shukla, Jerin Jacob
Cc: dev, fengchengwen, kevin.laatz, bruce.richardson, conor.walsh,
vattunuru, g.singh, sachin.saxena, hemant.agrawal, cheng1.jiang,
ndabilpuram, anoobj, mb
Introduce event DMA adapter APIs. The change provides information
on adapter modes and usage. Application can use this event adapter
interface to transfer packets between DMA device and event device.
Signed-off-by: Amit Prakash Shukla <amitprakashs@marvell.com>
---
MAINTAINERS | 5 +
doc/api/doxy-api-index.md | 1 +
doc/guides/eventdevs/features/default.ini | 8 +
doc/guides/prog_guide/event_dma_adapter.rst | 264 ++++
doc/guides/prog_guide/eventdev.rst | 8 +-
.../img/event_dma_adapter_op_forward.svg | 1086 +++++++++++++++++
.../img/event_dma_adapter_op_new.svg | 1079 ++++++++++++++++
doc/guides/prog_guide/index.rst | 1 +
doc/guides/rel_notes/release_23_11.rst | 4 +-
lib/eventdev/eventdev_pmd.h | 175 ++-
lib/eventdev/eventdev_private.c | 10 +
lib/eventdev/meson.build | 1 +
lib/eventdev/rte_event_dma_adapter.h | 582 +++++++++
lib/eventdev/rte_eventdev.h | 43 +
lib/eventdev/rte_eventdev_core.h | 8 +-
lib/eventdev/version.map | 15 +
16 files changed, 3284 insertions(+), 6 deletions(-)
create mode 100644 doc/guides/prog_guide/event_dma_adapter.rst
create mode 100644 doc/guides/prog_guide/img/event_dma_adapter_op_forward.svg
create mode 100644 doc/guides/prog_guide/img/event_dma_adapter_op_new.svg
create mode 100644 lib/eventdev/rte_event_dma_adapter.h
diff --git a/MAINTAINERS b/MAINTAINERS
index a926155f26..a6b8fc88d0 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -548,6 +548,11 @@ F: drivers/raw/skeleton/
F: app/test/test_rawdev.c
F: doc/guides/prog_guide/rawdev.rst
+Eventdev DMA Adapter API
+M: Amit Prakash Shukla <amitprakashs@marvell.com>
+T: git://dpdk.org/next/dpdk-next-eventdev
+F: lib/eventdev/*dma_adapter*
+F: doc/guides/prog_guide/event_dma_adapter.rst
Memory Pool Drivers
-------------------
diff --git a/doc/api/doxy-api-index.md b/doc/api/doxy-api-index.md
index fdeda13932..b7df7be4d9 100644
--- a/doc/api/doxy-api-index.md
+++ b/doc/api/doxy-api-index.md
@@ -29,6 +29,7 @@ The public API headers are grouped by topics:
[event_eth_tx_adapter](@ref rte_event_eth_tx_adapter.h),
[event_timer_adapter](@ref rte_event_timer_adapter.h),
[event_crypto_adapter](@ref rte_event_crypto_adapter.h),
+ [event_dma_adapter](@ref rte_event_dma_adapter.h),
[rawdev](@ref rte_rawdev.h),
[metrics](@ref rte_metrics.h),
[bitrate](@ref rte_bitrate.h),
diff --git a/doc/guides/eventdevs/features/default.ini b/doc/guides/eventdevs/features/default.ini
index 00360f60c6..73a52d915b 100644
--- a/doc/guides/eventdevs/features/default.ini
+++ b/doc/guides/eventdevs/features/default.ini
@@ -44,6 +44,14 @@ internal_port_op_fwd =
internal_port_qp_ev_bind =
session_private_data =
+;
+; Features of a default DMA adapter.
+;
+[DMA adapter Features]
+internal_port_op_new =
+internal_port_op_fwd =
+internal_port_vchan_ev_bind =
+
;
; Features of a default Timer adapter.
;
diff --git a/doc/guides/prog_guide/event_dma_adapter.rst b/doc/guides/prog_guide/event_dma_adapter.rst
new file mode 100644
index 0000000000..eeb9ce6dfd
--- /dev/null
+++ b/doc/guides/prog_guide/event_dma_adapter.rst
@@ -0,0 +1,264 @@
+.. SPDX-License-Identifier: BSD-3-Clause
+ Copyright (c) 2023 Marvell.
+
+Event DMA Adapter Library
+=========================
+
+DPDK :doc:`Eventdev library <eventdev>` provides event driven programming model with features
+to schedule events. :doc:`DMA Device library <dmadev>` provides an interface to DMA poll mode
+drivers that support DMA operations. Event DMA Adapter is intended to bridge between the event
+device and the DMA device.
+
+Packet flow from DMA device to the event device can be accomplished using software and hardware
+based transfer mechanisms. The adapter queries an eventdev PMD to determine which mechanism to
+be used. The adapter uses an EAL service core function for software based packet transfer and
+uses the eventdev PMD functions to configure hardware based packet transfer between DMA device
+and the event device. DMA adapter uses a new event type called ``RTE_EVENT_TYPE_DMADEV`` to
+indicate the source of event.
+
+Application can choose to submit an DMA operation directly to an DMA device or send it to an DMA
+adapter via eventdev based on RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP_FWD capability. The
+first mode is known as the event new (RTE_EVENT_DMA_ADAPTER_OP_NEW) mode and the second as the
+event forward (RTE_EVENT_DMA_ADAPTER_OP_FORWARD) mode. Choice of mode can be specified while
+creating the adapter. In the former mode, it is the application's responsibility to enable
+ingress packet ordering. In the latter mode, it is the adapter's responsibility to enable
+ingress packet ordering.
+
+
+Adapter Modes
+-------------
+
+RTE_EVENT_DMA_ADAPTER_OP_NEW mode
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+In the RTE_EVENT_DMA_ADAPTER_OP_NEW mode, application submits DMA operations directly to an DMA
+device. The adapter then dequeues DMA completions from the DMA device and enqueues them as events
+to the event device. This mode does not ensure ingress ordering as the application directly
+enqueues to the dmadev without going through DMA/atomic stage. In this mode, events dequeued
+from the adapter are treated as new events. The application has to specify event information
+(response information) which is needed to enqueue an event after the DMA operation is completed.
+
+.. _figure_event_dma_adapter_op_new:
+
+.. figure:: img/event_dma_adapter_op_new.*
+
+ Working model of ``RTE_EVENT_DMA_ADAPTER_OP_NEW`` mode
+
+
+RTE_EVENT_DMA_ADAPTER_OP_FORWARD mode
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+In the ``RTE_EVENT_DMA_ADAPTER_OP_FORWARD`` mode, if the event PMD and DMA PMD supports internal
+event port (``RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP_FWD``), the application should use
+``rte_event_dma_adapter_enqueue()`` API to enqueue DMA operations as events to DMA adapter. If
+not, application retrieves DMA adapter's event port using ``rte_event_dma_adapter_event_port_get()``
+API, links its event queue to this port and starts enqueuing DMA operations as events to eventdev
+using ``rte_event_enqueue_burst()``. The adapter then dequeues the events and submits the DMA
+operations to the dmadev. After the DMA operation is complete, the adapter enqueues events to the
+event device.
+
+Applications can use this mode when ingress packet ordering is needed. In this mode, events
+dequeued from the adapter will be treated as forwarded events. Application has to specify event
+information (response information) needed to enqueue the event after the DMA operation has
+completed.
+
+.. _figure_event_dma_adapter_op_forward:
+
+.. figure:: img/event_dma_adapter_op_forward.*
+
+ Working model of ``RTE_EVENT_DMA_ADAPTER_OP_FORWARD`` mode
+
+
+API Overview
+------------
+
+This section has a brief introduction to the event DMA adapter APIs. The application is expected
+to create an adapter which is associated with a single eventdev, then add dmadev and vchan to the
+adapter instance.
+
+
+Create an adapter instance
+~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+An adapter instance is created using ``rte_event_dma_adapter_create()``. This function is called
+with event device to be associated with the adapter and port configuration for the adapter to
+setup an event port (if the adapter needs to use a service function).
+
+Adapter can be started in ``RTE_EVENT_DMA_ADAPTER_OP_NEW`` or ``RTE_EVENT_DMA_ADAPTER_OP_FORWARD``
+mode.
+
+.. code-block:: c
+
+ enum rte_event_dma_adapter_mode mode;
+ struct rte_event_dev_info dev_info;
+ struct rte_event_port_conf conf;
+ uint8_t evdev_id;
+ uint8_t dma_id;
+ int ret;
+
+ ret = rte_event_dev_info_get(dma_id, &dev_info);
+
+ conf.new_event_threshold = dev_info.max_num_events;
+ conf.dequeue_depth = dev_info.max_event_port_dequeue_depth;
+ conf.enqueue_depth = dev_info.max_event_port_enqueue_depth;
+ mode = RTE_EVENT_DMA_ADAPTER_OP_FORWARD;
+ ret = rte_event_dma_adapter_create(dma_id, evdev_id, &conf, mode);
+
+
+``rte_event_dma_adapter_create_ext()`` function can be used by the application to have a finer
+control on eventdev port allocation and setup. The ``rte_event_dma_adapter_create_ext()``
+function is passed a callback function. The callback function is invoked if the adapter creates
+a service function and uses an event port for it. The callback is expected to fill the
+``struct rte_event_dma_adapter_conf`` structure passed to it.
+
+In the ``RTE_EVENT_DMA_ADAPTER_OP_FORWARD`` mode, if the event PMD and DMA PMD supports internal
+event port (``RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP_FWD``), events with DMA operations should
+be enqueued to the DMA adapter using ``rte_event_dma_adapter_enqueue()`` API. If not, the event port
+created by the adapter can be retrieved using ``rte_event_dma_adapter_event_port_get()`` API. An
+application can use this event port to link with an event queue, on which it enqueues events
+towards the DMA adapter using ``rte_event_enqueue_burst()``.
+
+.. code-block:: c
+
+ uint8_t dma_adpt_id, evdev_id, dma_dev_id, dma_ev_port_id, app_qid;
+ struct rte_event ev;
+ uint32_t cap;
+ int ret;
+
+ // Fill in event info and update event_ptr with rte_dma_op
+ memset(&ev, 0, sizeof(ev));
+ .
+ .
+ ev.event_ptr = op;
+
+ ret = rte_event_dma_adapter_caps_get(evdev_id, dma_dev_id, &cap);
+ if (cap & RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP_FWD) {
+ ret = rte_event_dma_adapter_enqueue(evdev_id, app_ev_port_id, ev, nb_events);
+ } else {
+ ret = rte_event_dma_adapter_event_port_get(dma_adpt_id, &dma_ev_port_id);
+ ret = rte_event_queue_setup(evdev_id, app_qid, NULL);
+ ret = rte_event_port_link(evdev_id, dma_ev_port_id, &app_qid, NULL, 1);
+ ev.queue_id = app_qid;
+ ret = rte_event_enqueue_burst(evdev_id, app_ev_port_id, ev, nb_events);
+ }
+
+
+Event device configuration for service based adapter
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+When ``rte_event_dma_adapter_create()`` is used for creating adapter instance,
+``rte_event_dev_config::nb_event_ports`` is automatically incremented, and event device is
+reconfigured with additional event port during service initialization. This event device
+reconfigure logic also increments the ``rte_event_dev_config::nb_single_link_event_port_queues``
+parameter if the adapter event port config is of type ``RTE_EVENT_PORT_CFG_SINGLE_LINK``.
+
+Applications using this mode of adapter creation need not configure the event device with
+``rte_event_dev_config::nb_event_ports`` and
+``rte_event_dev_config::nb_single_link_event_port_queues`` parameters required for DMA adapter when
+the adapter is created using the above-mentioned API.
+
+
+Querying adapter capabilities
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+The ``rte_event_dma_adapter_caps_get()`` function allows the application to query the adapter
+capabilities for an eventdev and dmadev combination. This API provides whether dmadev and eventdev
+are connected using internal HW port or not.
+
+.. code-block:: c
+
+ rte_event_dma_adapter_caps_get(dev_id, dma_dev_id, &cap);
+
+
+Adding vchan to the adapter instance
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+dmadev device id and vchan are configured using dmadev APIs. For more information
+see :doc:`here <dmadev>`.
+
+.. code-block:: c
+
+ struct rte_dma_vchan_conf vchan_conf;
+ struct rte_dma_conf dev_conf;
+ uint8_t dev_id = 0;
+ uint16_t vchan = 0;
+
+ rte_dma_configure(dev_id, &dev_conf);
+ rte_dma_vchan_setup(dev_id, vhcan, &vchan_conf);
+
+These dmadev id and vchan are added to the instance using the
+``rte_event_dma_adapter_vchan_add()`` API. The same is removed using
+``rte_event_dma_adapter_vchan_del()`` API. If hardware supports
+``RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_VCHAN_EV_BIND`` capability, event information must be passed to the add API.
+
+.. code-block:: c
+
+ uint32_t cap;
+ int ret;
+
+ ret = rte_event_dma_adapter_caps_get(evdev_id, dma_dev_id, &cap);
+ if (cap & RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_VCHAN_EV_BIND) {
+ struct rte_event event;
+
+ rte_event_dma_adapter_vchan_add(id, dma_dev_id, vchan, &conf);
+ } else
+ rte_event_dma_adapter_vchan_add(id, dma_dev_id, vchan, NULL);
+
+
+Configuring service function
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+If the adapter uses a service function, the application is required to assign a service core to
+the service function as show below.
+
+.. code-block:: c
+
+ uint32_t service_id;
+
+ if (rte_event_dma_adapter_service_id_get(dma_id, &service_id) == 0)
+ rte_service_map_lcore_set(service_id, CORE_ID);
+
+
+Set event response information
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+In the RTE_EVENT_DMA_ADAPTER_OP_FORWARD / RTE_EVENT_DMA_ADAPTER_OP_NEW mode, the application
+specifies the dmadev ID and vchan ID in ``struct rte_event_dma_adapter_op`` and the event
+information (response information) needed to enqueue an event after the DMA operation has
+completed. The response information is specified in ``struct rte_event`` and appended to the
+``struct rte_event_dma_adapter_op``.
+
+
+Start the adapter instance
+~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+The application calls ``rte_event_dma_adapter_start()`` to start the adapter. This function calls
+the start callbacks of the eventdev PMDs for hardware based eventdev-dmadev connections and
+``rte_service_run_state_set()`` to enable the service function if one exists.
+
+.. code-block:: c
+
+ rte_event_dma_adapter_start(id);
+
+.. Note::
+
+ The eventdev to which the event_dma_adapter is connected should be started before calling
+ rte_event_dma_adapter_start().
+
+
+Get adapter statistics
+~~~~~~~~~~~~~~~~~~~~~~
+
+The ``rte_event_dma_adapter_stats_get()`` function reports counters defined in struct
+``rte_event_dma_adapter_stats``. The received packet and enqueued event counts are a sum of the
+counts from the eventdev PMD callbacks if the callback is supported, and the counts maintained by
+the service function, if one exists.
+
+Set/Get adapter runtime configuration parameters
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+The runtime configuration parameters of adapter can be set/get using
+``rte_event_dma_adapter_runtime_params_set()`` and
+``rte_event_dma_adapter_runtime_params_get()`` respectively.
+The parameters that can be set/get are defined in
+``struct rte_event_dma_adapter_runtime_params``.
diff --git a/doc/guides/prog_guide/eventdev.rst b/doc/guides/prog_guide/eventdev.rst
index 2c83176846..ff55115d0d 100644
--- a/doc/guides/prog_guide/eventdev.rst
+++ b/doc/guides/prog_guide/eventdev.rst
@@ -333,7 +333,8 @@ eventdev.
.. Note::
EventDev needs to be started before starting the event producers such
- as event_eth_rx_adapter, event_timer_adapter and event_crypto_adapter.
+ as event_eth_rx_adapter, event_timer_adapter, event_crypto_adapter and
+ event_dma_adapter.
Ingress of New Events
~~~~~~~~~~~~~~~~~~~~~
@@ -445,8 +446,9 @@ using ``rte_event_dev_stop_flush_callback_register()`` function.
.. Note::
The event producers such as ``event_eth_rx_adapter``,
- ``event_timer_adapter`` and ``event_crypto_adapter``
- need to be stopped before stopping the event device.
+ ``event_timer_adapter``, ``event_crypto_adapter`` and
+ ``event_dma_adapter`` need to be stopped before stopping
+ the event device.
Summary
-------
diff --git a/doc/guides/prog_guide/img/event_dma_adapter_op_forward.svg b/doc/guides/prog_guide/img/event_dma_adapter_op_forward.svg
new file mode 100644
index 0000000000..b7fe1fecf2
--- /dev/null
+++ b/doc/guides/prog_guide/img/event_dma_adapter_op_forward.svg
@@ -0,0 +1,1086 @@
+<?xml version="1.0" encoding="UTF-8" standalone="no"?>
+<!-- SPDX-License-Identifier: BSD-3-Clause -->
+<!-- Copyright (c) 2023 Marvell. -->
+<!-- Created with Inkscape (http://www.inkscape.org/) -->
+
+<svg
+ width="720px"
+ height="486px"
+ id="svg13237"
+ version="1.1"
+ inkscape:version="1.2.1 (9c6d41e410, 2022-07-14)"
+ sodipodi:docname="event_dma_adapter_op_forward.svg"
+ xmlns:inkscape="http://www.inkscape.org/namespaces/inkscape"
+ xmlns:sodipodi="http://sodipodi.sourceforge.net/DTD/sodipodi-0.dtd"
+ xmlns="http://www.w3.org/2000/svg"
+ xmlns:svg="http://www.w3.org/2000/svg"
+ xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#"
+ xmlns:cc="http://creativecommons.org/ns#"
+ xmlns:dc="http://purl.org/dc/elements/1.1/"
+ xmlns:v="http://schemas.microsoft.com/visio/2003/SVGExtensions/">
+ <defs
+ id="defs13239">
+ <marker
+ inkscape:stockid="Arrow1Sstart"
+ orient="auto"
+ refY="0.0"
+ refX="0.0"
+ id="Arrow1Sstart"
+ style="overflow:visible">
+ <path
+ id="path8416"
+ d="M 0.0,0.0 L 5.0,-5.0 L -12.5,0.0 L 5.0,5.0 L 0.0,0.0 z "
+ style="fill-rule:evenodd;stroke:#000000;stroke-width:1.0pt"
+ transform="scale(0.2) translate(6,0)" />
+ </marker>
+ <marker
+ inkscape:stockid="Arrow1Send"
+ orient="auto"
+ refY="0.0"
+ refX="0.0"
+ id="Arrow1Send"
+ style="overflow:visible;">
+ <path
+ id="path8419"
+ d="M 0.0,0.0 L 5.0,-5.0 L -12.5,0.0 L 5.0,5.0 L 0.0,0.0 z "
+ style="fill-rule:evenodd;stroke:#000000;stroke-width:1.0pt;"
+ transform="scale(0.2) rotate(180) translate(6,0)" />
+ </marker>
+ <marker
+ inkscape:stockid="DiamondL"
+ orient="auto"
+ refY="0.0"
+ refX="0.0"
+ id="DiamondL"
+ style="overflow:visible">
+ <path
+ id="path8483"
+ d="M 0,-7.0710768 L -7.0710894,0 L 0,7.0710589 L 7.0710462,0 L 0,-7.0710768 z "
+ style="fill-rule:evenodd;stroke:#000000;stroke-width:1.0pt"
+ transform="scale(0.8)" />
+ </marker>
+ <marker
+ inkscape:stockid="DotL"
+ orient="auto"
+ refY="0.0"
+ refX="0.0"
+ id="DotL"
+ style="overflow:visible">
+ <path
+ id="path8465"
+ d="M -2.5,-1.0 C -2.5,1.7600000 -4.7400000,4.0 -7.5,4.0 C -10.260000,4.0 -12.5,1.7600000 -12.5,-1.0 C -12.5,-3.7600000 -10.260000,-6.0 -7.5,-6.0 C -4.7400000,-6.0 -2.5,-3.7600000 -2.5,-1.0 z "
+ style="fill-rule:evenodd;stroke:#000000;stroke-width:1.0pt"
+ transform="scale(0.8) translate(7.4, 1)" />
+ </marker>
+ <marker
+ inkscape:stockid="SquareL"
+ orient="auto"
+ refY="0.0"
+ refX="0.0"
+ id="SquareL"
+ style="overflow:visible">
+ <path
+ id="path8474"
+ d="M -5.0,-5.0 L -5.0,5.0 L 5.0,5.0 L 5.0,-5.0 L -5.0,-5.0 z "
+ style="fill-rule:evenodd;stroke:#000000;stroke-width:1.0pt"
+ transform="scale(0.8)" />
+ </marker>
+ <marker
+ inkscape:stockid="TriangleOutL"
+ orient="auto"
+ refY="0.0"
+ refX="0.0"
+ id="TriangleOutL"
+ style="overflow:visible">
+ <path
+ id="path8546"
+ d="M 5.77,0.0 L -2.88,5.0 L -2.88,-5.0 L 5.77,0.0 z "
+ style="fill-rule:evenodd;stroke:#000000;stroke-width:1.0pt"
+ transform="scale(0.8)" />
+ </marker>
+ <marker
+ inkscape:stockid="Arrow1Lstart"
+ orient="auto"
+ refY="0.0"
+ refX="0.0"
+ id="Arrow1Lstart"
+ style="overflow:visible">
+ <path
+ id="path8404"
+ d="M 0.0,0.0 L 5.0,-5.0 L -12.5,0.0 L 5.0,5.0 L 0.0,0.0 z "
+ style="fill-rule:evenodd;stroke:#000000;stroke-width:1.0pt"
+ transform="scale(0.8) translate(12.5,0)" />
+ </marker>
+ <marker
+ inkscape:stockid="Arrow1Mend"
+ orient="auto"
+ refY="0.0"
+ refX="0.0"
+ id="Arrow1Mend"
+ style="overflow:visible;">
+ <path
+ id="path8413"
+ d="M 0.0,0.0 L 5.0,-5.0 L -12.5,0.0 L 5.0,5.0 L 0.0,0.0 z "
+ style="fill-rule:evenodd;stroke:#000000;stroke-width:1.0pt;"
+ transform="scale(0.4) rotate(180) translate(10,0)" />
+ </marker>
+ <marker
+ inkscape:stockid="Arrow2Lend"
+ orient="auto"
+ refY="0.0"
+ refX="0.0"
+ id="Arrow2Lend"
+ style="overflow:visible;">
+ <path
+ id="path8425"
+ style="fill-rule:evenodd;stroke-width:0.62500000;stroke-linejoin:round;"
+ d="M 8.7185878,4.0337352 L -2.2072895,0.016013256 L 8.7185884,-4.0017078 C 6.9730900,-1.6296469 6.9831476,1.6157441 8.7185878,4.0337352 z "
+ transform="scale(1.1) rotate(180) translate(1,0)" />
+ </marker>
+ <marker
+ inkscape:stockid="Arrow1Lend"
+ orient="auto"
+ refY="0.0"
+ refX="0.0"
+ id="Arrow1Lend"
+ style="overflow:visible;">
+ <path
+ id="path8407"
+ d="M 0.0,0.0 L 5.0,-5.0 L -12.5,0.0 L 5.0,5.0 L 0.0,0.0 z "
+ style="fill-rule:evenodd;stroke:#000000;stroke-width:1.0pt;"
+ transform="scale(0.8) rotate(180) translate(12.5,0)" />
+ </marker>
+ <filter
+ id="filter_2"
+ color-interpolation-filters="sRGB">
+ <feGaussianBlur
+ stdDeviation="2"
+ id="feGaussianBlur15" />
+ </filter>
+ <filter
+ id="filter_2-3"
+ color-interpolation-filters="sRGB"
+ x="-0.086178862"
+ y="-0.086178862"
+ width="1.1723577"
+ height="1.1723577">
+ <feGaussianBlur
+ stdDeviation="2"
+ id="feGaussianBlur15-1" />
+ </filter>
+ <filter
+ id="filter_2-0"
+ color-interpolation-filters="sRGB">
+ <feGaussianBlur
+ stdDeviation="2"
+ id="feGaussianBlur15-7" />
+ </filter>
+ <filter
+ id="filter_2-0-8"
+ color-interpolation-filters="sRGB">
+ <feGaussianBlur
+ stdDeviation="2"
+ id="feGaussianBlur15-7-7" />
+ </filter>
+ <filter
+ id="filter_2-3-9"
+ color-interpolation-filters="sRGB">
+ <feGaussianBlur
+ stdDeviation="2"
+ id="feGaussianBlur15-1-6" />
+ </filter>
+ <filter
+ id="filter_2-3-6"
+ color-interpolation-filters="sRGB">
+ <feGaussianBlur
+ stdDeviation="2"
+ id="feGaussianBlur15-1-63" />
+ </filter>
+ <filter
+ id="filter_2-3-91"
+ color-interpolation-filters="sRGB"
+ x="-0.086178862"
+ y="-0.086178862"
+ width="1.1723577"
+ height="1.1723577">
+ <feGaussianBlur
+ stdDeviation="2"
+ id="feGaussianBlur15-1-3" />
+ </filter>
+ <marker
+ inkscape:stockid="Arrow1Lend"
+ orient="auto"
+ refY="0"
+ refX="0"
+ id="Arrow1Lend-5"
+ style="overflow:visible">
+ <path
+ inkscape:connector-curvature="0"
+ id="path8407-3"
+ d="M 0,0 5,-5 -12.5,0 5,5 0,0 z"
+ style="fill-rule:evenodd;stroke:#000000;stroke-width:1pt"
+ transform="matrix(-0.8,0,0,-0.8,-10,0)" />
+ </marker>
+ <marker
+ inkscape:stockid="Arrow1Lend"
+ orient="auto"
+ refY="0"
+ refX="0"
+ id="Arrow1Lend-6"
+ style="overflow:visible">
+ <path
+ inkscape:connector-curvature="0"
+ id="path8407-0"
+ d="M 0,0 5,-5 -12.5,0 5,5 0,0 z"
+ style="fill-rule:evenodd;stroke:#000000;stroke-width:1pt"
+ transform="matrix(-0.8,0,0,-0.8,-10,0)" />
+ </marker>
+ <marker
+ inkscape:stockid="Arrow1Lstart"
+ orient="auto"
+ refY="0"
+ refX="0"
+ id="Arrow1Lstart-7"
+ style="overflow:visible">
+ <path
+ inkscape:connector-curvature="0"
+ id="path8404-0"
+ d="M 0,0 5,-5 -12.5,0 5,5 0,0 z"
+ style="fill-rule:evenodd;stroke:#000000;stroke-width:1pt"
+ transform="matrix(0.8,0,0,0.8,10,0)" />
+ </marker>
+ <marker
+ inkscape:stockid="Arrow1Lend"
+ orient="auto"
+ refY="0"
+ refX="0"
+ id="Arrow1Lend-51"
+ style="overflow:visible">
+ <path
+ inkscape:connector-curvature="0"
+ id="path8407-1"
+ d="M 0,0 5,-5 -12.5,0 5,5 0,0 z"
+ style="fill-rule:evenodd;stroke:#000000;stroke-width:1pt"
+ transform="matrix(-0.8,0,0,-0.8,-10,0)" />
+ </marker>
+ <marker
+ inkscape:stockid="Arrow1Lend"
+ orient="auto"
+ refY="0"
+ refX="0"
+ id="Arrow1Lend-3"
+ style="overflow:visible">
+ <path
+ inkscape:connector-curvature="0"
+ id="path8407-6"
+ d="M 0,0 5,-5 -12.5,0 5,5 0,0 z"
+ style="fill-rule:evenodd;stroke:#000000;stroke-width:1pt"
+ transform="matrix(-0.8,0,0,-0.8,-10,0)" />
+ </marker>
+ <marker
+ inkscape:stockid="Arrow1Lend"
+ orient="auto"
+ refY="0"
+ refX="0"
+ id="Arrow1Lend-62"
+ style="overflow:visible">
+ <path
+ inkscape:connector-curvature="0"
+ id="path8407-9"
+ d="M 0,0 5,-5 -12.5,0 5,5 0,0 z"
+ style="fill-rule:evenodd;stroke:#000000;stroke-width:1pt"
+ transform="matrix(-0.8,0,0,-0.8,-10,0)" />
+ </marker>
+ <marker
+ inkscape:stockid="Arrow1Lend"
+ orient="auto"
+ refY="0"
+ refX="0"
+ id="Arrow1Lend-2"
+ style="overflow:visible">
+ <path
+ inkscape:connector-curvature="0"
+ id="path8407-7"
+ d="M 0,0 5,-5 -12.5,0 5,5 0,0 z"
+ style="fill-rule:evenodd;stroke:#000000;stroke-width:1pt"
+ transform="matrix(-0.8,0,0,-0.8,-10,0)" />
+ </marker>
+ <marker
+ inkscape:stockid="Arrow1Lstart"
+ orient="auto"
+ refY="0"
+ refX="0"
+ id="Arrow1Lstart-7-9"
+ style="overflow:visible">
+ <path
+ inkscape:connector-curvature="0"
+ id="path8404-0-3"
+ d="M 0,0 5,-5 -12.5,0 5,5 0,0 z"
+ style="fill-rule:evenodd;stroke:#000000;stroke-width:1pt"
+ transform="matrix(0.8,0,0,0.8,10,0)" />
+ </marker>
+ <filter
+ id="filter_2-3-6-1"
+ color-interpolation-filters="sRGB">
+ <feGaussianBlur
+ stdDeviation="2"
+ id="feGaussianBlur15-1-63-8" />
+ </filter>
+ <filter
+ id="filter_2-3-92"
+ color-interpolation-filters="sRGB"
+ x="-0.086178862"
+ y="-0.086178862"
+ width="1.1723577"
+ height="1.1723577">
+ <feGaussianBlur
+ stdDeviation="2"
+ id="feGaussianBlur15-1-2" />
+ </filter>
+ <filter
+ id="filter_2-3-94"
+ color-interpolation-filters="sRGB"
+ x="-0.086178862"
+ y="-0.086178862"
+ width="1.1723577"
+ height="1.1723577">
+ <feGaussianBlur
+ stdDeviation="2"
+ id="feGaussianBlur15-1-7" />
+ </filter>
+ <marker
+ inkscape:stockid="Arrow1Lstart"
+ orient="auto"
+ refY="0"
+ refX="0"
+ id="Arrow1Lstart-7-6"
+ style="overflow:visible">
+ <path
+ inkscape:connector-curvature="0"
+ id="path8404-0-1"
+ d="M 0,0 5,-5 -12.5,0 5,5 0,0 z"
+ style="fill-rule:evenodd;stroke:#000000;stroke-width:1pt"
+ transform="matrix(0.8,0,0,0.8,10,0)" />
+ </marker>
+ <marker
+ inkscape:stockid="Arrow1Lend"
+ orient="auto"
+ refY="0"
+ refX="0"
+ id="Arrow1Lend-55"
+ style="overflow:visible">
+ <path
+ inkscape:connector-curvature="0"
+ id="path8407-4"
+ d="M 0,0 5,-5 -12.5,0 5,5 0,0 z"
+ style="fill-rule:evenodd;stroke:#000000;stroke-width:1pt"
+ transform="matrix(-0.8,0,0,-0.8,-10,0)" />
+ </marker>
+ </defs>
+ <sodipodi:namedview
+ id="base"
+ pagecolor="#ffffff"
+ bordercolor="#666666"
+ borderopacity="1.0"
+ inkscape:pageopacity="0.0"
+ inkscape:pageshadow="2"
+ inkscape:zoom="1"
+ inkscape:cx="233.5"
+ inkscape:cy="288"
+ inkscape:document-units="px"
+ inkscape:current-layer="layer1"
+ showgrid="false"
+ inkscape:window-width="1920"
+ inkscape:window-height="1017"
+ inkscape:window-x="-8"
+ inkscape:window-y="-8"
+ inkscape:window-maximized="1"
+ inkscape:snap-nodes="false"
+ inkscape:showpageshadow="2"
+ inkscape:pagecheckerboard="0"
+ inkscape:deskcolor="#d1d1d1">
+ <inkscape:grid
+ type="xygrid"
+ id="grid13454" />
+ </sodipodi:namedview>
+ <metadata
+ id="metadata13242">
+ <rdf:RDF>
+ <cc:Work
+ rdf:about="">
+ <dc:format>image/svg+xml</dc:format>
+ <dc:type
+ rdf:resource="http://purl.org/dc/dcmitype/StillImage" />
+ <dc:title />
+ </cc:Work>
+ </rdf:RDF>
+ </metadata>
+ <g
+ id="layer1"
+ inkscape:label="Layer 1"
+ inkscape:groupmode="layer">
+ <g
+ style="font-size:12px;fill:none;stroke-linecap:square;stroke-miterlimit:3;overflow:visible"
+ id="shape1-1-2-4"
+ v:mID="1"
+ v:groupContext="shape"
+ transform="matrix(2.1604167,0,0,1.5671361,88.874699,-812.39909)">
+ <title
+ id="title22-7-5">Square</title>
+ <desc
+ id="desc24-7-8">Atomic Queue #1</desc>
+ <v:userDefs>
+ <v:ud
+ v:nameU="visVersion"
+ v:val="VT0(15):26" />
+ </v:userDefs>
+ <v:textBlock
+ v:margins="rect(4,4,4,4)" />
+ <v:textRect
+ cx="30.75"
+ cy="581.25"
+ width="61.5"
+ height="61.5" />
+ <g
+ id="shadow1-2-9-5"
+ v:groupContext="shadow"
+ v:shadowOffsetX="0.345598"
+ v:shadowOffsetY="-1.97279"
+ v:shadowType="1"
+ transform="translate(0.345598,1.97279)"
+ class="st1"
+ style="visibility:visible">
+ <rect
+ x="0"
+ y="550.5"
+ width="61.5"
+ height="61.5"
+ class="st2"
+ id="rect27-8-7"
+ style="fill:#5b9bd5;fill-opacity:0.22000002;stroke:#5b9bd5;stroke-opacity:0.22000002;filter:url(#filter_2-3-91)" />
+ </g>
+ <g
+ id="g13515-33">
+ <g
+ id="g13534-8">
+ <rect
+ x="0"
+ y="550.5"
+ width="61.5"
+ height="61.5"
+ class="st3"
+ id="rect29-1-95"
+ style="fill:#5b9bd5;stroke:#c7c8c8;stroke-width:0.25" />
+ </g>
+ </g>
+ </g>
+ <path
+ style="fill:none;stroke:#000000;stroke-width:0.712265;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:4;stroke-dasharray:none;stroke-opacity:1;marker-start:url(#Arrow1Lstart-7);marker-end:none"
+ d="M 312.28671,240.74335 H 227.99897"
+ id="path17209"
+ inkscape:connector-type="orthogonal"
+ inkscape:connector-curvature="0" />
+ <path
+ style="fill:none;stroke:#000000;stroke-width:0.718986px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1;marker-mid:none;marker-end:url(#Arrow1Lend)"
+ d="m 221.6484,77.57125 h 94.28101"
+ id="path17209-8"
+ inkscape:connector-type="orthogonal"
+ inkscape:connector-curvature="0" />
+ <g
+ style="font-size:12px;fill:none;stroke-linecap:square;stroke-miterlimit:3;overflow:visible"
+ id="shape1-1-2"
+ v:mID="1"
+ v:groupContext="shape"
+ transform="matrix(2.1604167,0,0,1.5671361,314.24227,-811.89589)">
+ <title
+ id="title22-7">Square</title>
+ <desc
+ id="desc24-7">Atomic Queue #1</desc>
+ <v:userDefs>
+ <v:ud
+ v:nameU="visVersion"
+ v:val="VT0(15):26" />
+ </v:userDefs>
+ <v:textBlock
+ v:margins="rect(4,4,4,4)" />
+ <v:textRect
+ cx="30.75"
+ cy="581.25"
+ width="61.5"
+ height="61.5" />
+ <g
+ id="shadow1-2-9"
+ v:groupContext="shadow"
+ v:shadowOffsetX="0.345598"
+ v:shadowOffsetY="-1.97279"
+ v:shadowType="1"
+ transform="translate(0.345598,1.97279)"
+ class="st1"
+ style="visibility:visible">
+ <rect
+ x="0"
+ y="550.5"
+ width="61.5"
+ height="61.5"
+ class="st2"
+ id="rect27-8"
+ style="fill:#5b9bd5;fill-opacity:0.22000002;stroke:#5b9bd5;stroke-opacity:0.22000002;filter:url(#filter_2-3)" />
+ </g>
+ <g
+ id="g13515">
+ <g
+ id="g13534">
+ <rect
+ x="0"
+ y="550.5"
+ width="61.5"
+ height="61.5"
+ class="st3"
+ id="rect29-1"
+ style="fill:#5b9bd5;stroke:#c7c8c8;stroke-width:0.25" />
+ </g>
+ </g>
+ </g>
+ <path
+ style="fill:none;stroke:#000000;stroke-width:0.724714;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:4;stroke-dasharray:none;stroke-opacity:1;marker-start:url(#Arrow1Lstart);marker-end:none"
+ d="M 89.025329,74.39932 H 24.750043"
+ id="path17209-3"
+ inkscape:connector-type="orthogonal"
+ inkscape:connector-curvature="0" />
+ <path
+ transform="matrix(0.73232502,0,0,0.75477602,-4.325033,28.642983)"
+ sodipodi:type="arc"
+ style="fill:#539de6;fill-opacity:1;stroke:#0000ea;stroke-width:1;stroke-linecap:square;stroke-miterlimit:4;stroke-opacity:1;stroke-dasharray:none"
+ id="path29161-3"
+ sodipodi:cx="371"
+ sodipodi:cy="64.5"
+ sodipodi:rx="17"
+ sodipodi:ry="15.5"
+ d="m 388,64.5 a 17,15.5 0 1 1 -34,0 17,15.5 0 1 1 34,0 z" />
+ <path
+ transform="matrix(0.73232502,0,0,0.75477602,-1.93108,192.80833)"
+ sodipodi:type="arc"
+ style="fill:#539de6;fill-opacity:1;stroke:#0000ea;stroke-width:1;stroke-linecap:square;stroke-miterlimit:4;stroke-opacity:1;stroke-dasharray:none"
+ id="path29161-1"
+ sodipodi:cx="371"
+ sodipodi:cy="64.5"
+ sodipodi:rx="17"
+ sodipodi:ry="15.5"
+ d="m 388,64.5 a 17,15.5 0 1 1 -34,0 17,15.5 0 1 1 34,0 z" />
+ <path
+ style="fill:none;stroke:#000000;stroke-width:0.751412;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:4;stroke-dasharray:none;stroke-opacity:1;marker-start:url(#Arrow1Lstart-7);marker-end:none"
+ d="M 18.763392,120.7432 H 87.758545"
+ id="path17209-3-0"
+ inkscape:connector-type="orthogonal"
+ inkscape:connector-curvature="0" />
+ <path
+ sodipodi:type="arc"
+ style="fill:#539de6;fill-opacity:1;stroke:#0000ea;stroke-width:1;stroke-linecap:square;stroke-miterlimit:4;stroke-opacity:1;stroke-dasharray:none"
+ id="path29161"
+ sodipodi:cx="371"
+ sodipodi:cy="64.5"
+ sodipodi:rx="17"
+ sodipodi:ry="15.5"
+ d="m 388,64.5 a 17,15.5 0 1 1 -34,0 17,15.5 0 1 1 34,0 z"
+ transform="matrix(0.73232502,0,0,0.75477602,-218.16394,72.68276)" />
+ <path
+ sodipodi:type="arc"
+ style="fill:#539de6;fill-opacity:1;stroke:#0000ea;stroke-width:1;stroke-linecap:square;stroke-miterlimit:4;stroke-opacity:1;stroke-dasharray:none"
+ id="path29161-2"
+ sodipodi:cx="371"
+ sodipodi:cy="64.5"
+ sodipodi:rx="17"
+ sodipodi:ry="15.5"
+ d="m 388,64.5 a 17,15.5 0 1 1 -34,0 17,15.5 0 1 1 34,0 z"
+ transform="matrix(0.73232502,0,0,0.75477602,-217.40136,26.716271)" />
+ <g
+ id="g29167-4"
+ transform="matrix(0.73232502,0,0,0.75477602,-217.31662,28.007562)">
+ <text
+ id="text29163-9"
+ y="70"
+ x="365"
+ style="font-style:normal;font-weight:normal;line-height:0%;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none"
+ xml:space="preserve"><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="70"
+ x="365"
+ id="tspan29165-9"
+ sodipodi:role="line">1</tspan></text>
+ </g>
+ <g
+ id="g29167-9"
+ transform="matrix(0.73232502,0,0,0.75477602,-4.9726112,28.689051)">
+ <text
+ id="text29163-3"
+ y="70"
+ x="365"
+ style="font-style:normal;font-weight:normal;line-height:0%;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none"
+ xml:space="preserve"><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="70"
+ x="365"
+ id="tspan29165-3"
+ sodipodi:role="line">2</tspan></text>
+ </g>
+ <path
+ style="fill:none;stroke:#000000;stroke-width:0.678033px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1;marker-start:url(#Arrow1Lstart-7);marker-end:none"
+ d="M 181,214.66098 V 145.33902"
+ id="path17211-7-1-6"
+ inkscape:connector-type="orthogonal"
+ inkscape:connector-curvature="0" />
+ <g
+ id="g29167"
+ transform="matrix(0.73232502,0,0,0.75477602,-218.07919,73.10621)">
+ <text
+ id="text29163"
+ y="70"
+ x="365"
+ style="font-style:normal;font-weight:normal;line-height:0%;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none"
+ xml:space="preserve"><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="70"
+ x="365"
+ id="tspan29165"
+ sodipodi:role="line">8</tspan></text>
+ </g>
+ <path
+ style="fill:none;stroke:#000000;stroke-width:0.678033px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1;marker-start:url(#Arrow1Lstart-7);marker-end:none"
+ d="m 131,145.8531 v 69.32197"
+ id="path17211-7-1"
+ inkscape:connector-type="orthogonal"
+ inkscape:connector-curvature="0" />
+ <path
+ transform="matrix(0.73232502,0,0,0.75477602,-140.37076,129.97088)"
+ sodipodi:type="arc"
+ style="fill:#539de6;fill-opacity:1;stroke:#0000ea;stroke-width:1;stroke-linecap:square;stroke-miterlimit:4;stroke-opacity:1;stroke-dasharray:none"
+ id="path29161-8"
+ sodipodi:cx="371"
+ sodipodi:cy="64.5"
+ sodipodi:rx="17"
+ sodipodi:ry="15.5"
+ d="m 388,64.5 a 17,15.5 0 1 1 -34,0 17,15.5 0 1 1 34,0 z" />
+ <g
+ id="g29167-2"
+ transform="matrix(0.73232502,0,0,0.75477602,-140.28602,131.01695)">
+ <text
+ id="text29163-92"
+ y="70"
+ x="365"
+ style="font-style:normal;font-weight:normal;line-height:0%;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none"
+ xml:space="preserve"><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="70"
+ x="365"
+ id="tspan29165-8"
+ sodipodi:role="line">7</tspan></text>
+ </g>
+ <path
+ style="fill:none;stroke:#000000;stroke-width:0.718986px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1;marker-mid:none;marker-end:url(#Arrow1Lend)"
+ d="m 317.1405,116 h -94.281"
+ id="path17209-8-0"
+ inkscape:connector-type="orthogonal"
+ inkscape:connector-curvature="0" />
+ <path
+ transform="matrix(0.73232502,0,0,0.75477602,-3.4914,66.68745)"
+ sodipodi:type="arc"
+ style="fill:#539de6;fill-opacity:1;stroke:#0000ea;stroke-width:1;stroke-linecap:square;stroke-miterlimit:4;stroke-opacity:1;stroke-dasharray:none"
+ id="path29161-6"
+ sodipodi:cx="371"
+ sodipodi:cy="64.5"
+ sodipodi:rx="17"
+ sodipodi:ry="15.5"
+ d="m 388,64.5 a 17,15.5 0 1 1 -34,0 17,15.5 0 1 1 34,0 z" />
+ <g
+ id="g29167-46"
+ transform="matrix(0.73232502,0,0,0.75477602,-4.40666,67.48829)">
+ <text
+ id="text29163-1"
+ y="70"
+ x="365"
+ style="font-style:normal;font-weight:normal;line-height:0%;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none"
+ xml:space="preserve"><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="70"
+ x="365"
+ id="tspan29165-5"
+ sodipodi:role="line">3</tspan></text>
+ </g>
+ <path
+ transform="matrix(0.73232502,0,0,0.75477602,-90.692582,130.31695)"
+ sodipodi:type="arc"
+ style="fill:#539de6;fill-opacity:1;stroke:#0000ea;stroke-width:1;stroke-linecap:square;stroke-miterlimit:4;stroke-opacity:1;stroke-dasharray:none"
+ id="path29161-8-6"
+ sodipodi:cx="371"
+ sodipodi:cy="64.5"
+ sodipodi:rx="17"
+ sodipodi:ry="15.5"
+ d="m 388,64.5 a 17,15.5 0 1 1 -34,0 17,15.5 0 1 1 34,0 z" />
+ <g
+ id="g29167-6"
+ transform="matrix(0.73232502,0,0,0.75477602,-90.84634,131.60918)">
+ <text
+ id="text29163-17"
+ y="70"
+ x="365"
+ style="font-style:normal;font-weight:normal;line-height:0%;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none"
+ xml:space="preserve"><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="70"
+ x="365"
+ id="tspan29165-2"
+ sodipodi:role="line">4</tspan></text>
+ </g>
+ <g
+ id="g29167-2-0"
+ transform="matrix(0.73232502,0,0,0.75477602,-2.424397,194.0216)">
+ <text
+ id="text29163-92-6"
+ y="70"
+ x="365"
+ style="font-style:normal;font-weight:normal;line-height:0%;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none"
+ xml:space="preserve"><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="70"
+ x="365"
+ id="tspan29165-8-2"
+ sodipodi:role="line">5</tspan></text>
+ </g>
+ <g
+ style="font-size:12px;fill:none;stroke-linecap:square;stroke-miterlimit:3;overflow:visible"
+ id="shape1-1-2-8"
+ v:mID="1"
+ v:groupContext="shape"
+ transform="matrix(2.1604167,0,0,1.5671361,93.82055,-648.98949)">
+ <title
+ id="title22-7-97">Square</title>
+ <desc
+ id="desc24-7-3">Atomic Queue #1</desc>
+ <v:userDefs>
+ <v:ud
+ v:nameU="visVersion"
+ v:val="VT0(15):26" />
+ </v:userDefs>
+ <v:textBlock
+ v:margins="rect(4,4,4,4)" />
+ <v:textRect
+ cx="30.75"
+ cy="581.25"
+ width="61.5"
+ height="61.5" />
+ <g
+ id="shadow1-2-9-6"
+ v:groupContext="shadow"
+ v:shadowOffsetX="0.345598"
+ v:shadowOffsetY="-1.97279"
+ v:shadowType="1"
+ transform="translate(0.345598,1.97279)"
+ class="st1"
+ style="visibility:visible">
+ <rect
+ x="0"
+ y="550.5"
+ width="61.5"
+ height="61.5"
+ class="st2"
+ id="rect27-8-12"
+ style="fill:#5b9bd5;fill-opacity:0.22000002;stroke:#5b9bd5;stroke-opacity:0.22000002;filter:url(#filter_2-3-92)" />
+ </g>
+ <g
+ id="g13515-9">
+ <g
+ id="g13534-3">
+ <rect
+ x="0"
+ y="550.5"
+ width="61.5"
+ height="61.5"
+ class="st3"
+ id="rect29-1-1"
+ style="fill:#5b9bd5;stroke:#c7c8c8;stroke-width:0.25" />
+ </g>
+ </g>
+ </g>
+ <g
+ style="font-size:12px;fill:none;stroke-linecap:square;stroke-miterlimit:3;overflow:visible"
+ id="shape1-1-2-84"
+ v:mID="1"
+ v:groupContext="shape"
+ transform="matrix(2.1604167,0,0,1.5671361,314.82055,-648.98949)">
+ <title
+ id="title22-7-50">Square</title>
+ <desc
+ id="desc24-7-36">Atomic Queue #1</desc>
+ <v:userDefs>
+ <v:ud
+ v:nameU="visVersion"
+ v:val="VT0(15):26" />
+ </v:userDefs>
+ <v:textBlock
+ v:margins="rect(4,4,4,4)" />
+ <v:textRect
+ cx="30.75"
+ cy="581.25"
+ width="61.5"
+ height="61.5" />
+ <g
+ id="shadow1-2-9-1"
+ v:groupContext="shadow"
+ v:shadowOffsetX="0.345598"
+ v:shadowOffsetY="-1.97279"
+ v:shadowType="1"
+ transform="translate(0.345598,1.97279)"
+ class="st1"
+ style="visibility:visible">
+ <rect
+ x="0"
+ y="550.5"
+ width="61.5"
+ height="61.5"
+ class="st2"
+ id="rect27-8-0"
+ style="fill:#5b9bd5;fill-opacity:0.22000002;stroke:#5b9bd5;stroke-opacity:0.22000002;filter:url(#filter_2-3-94)" />
+ </g>
+ <g
+ id="g13515-6">
+ <g
+ id="g13534-32">
+ <rect
+ x="0"
+ y="550.5"
+ width="61.5"
+ height="61.5"
+ class="st3"
+ id="rect29-1-0"
+ style="fill:#5b9bd5;stroke:#c7c8c8;stroke-width:0.25" />
+ </g>
+ </g>
+ </g>
+ <path
+ style="fill:none;stroke:#000000;stroke-width:0.712265;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:4;stroke-dasharray:none;stroke-opacity:1;marker-start:none;marker-end:url(#Arrow1Lend)"
+ d="M 313.14387,285 H 228.85613"
+ id="path17209-7"
+ inkscape:connector-type="orthogonal"
+ inkscape:connector-curvature="0" />
+ <path
+ transform="matrix(0.73232502,0,0,0.75477602,-2.692582,236.31695)"
+ sodipodi:type="arc"
+ style="fill:#539de6;fill-opacity:1;stroke:#0000ea;stroke-width:1;stroke-linecap:square;stroke-miterlimit:4;stroke-opacity:1;stroke-dasharray:none"
+ id="path29161-1-6"
+ sodipodi:cx="371"
+ sodipodi:cy="64.5"
+ sodipodi:rx="17"
+ sodipodi:ry="15.5"
+ d="m 388,64.5 a 17,15.5 0 1 1 -34,0 17,15.5 0 1 1 34,0 z" />
+ <g
+ id="g29167-2-0-5"
+ transform="matrix(0.73232502,0,0,0.75477602,-2.424397,237.0216)">
+ <text
+ id="text29163-92-6-6"
+ y="70"
+ x="365"
+ style="font-style:normal;font-weight:normal;line-height:0%;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none"
+ xml:space="preserve"><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="70"
+ x="365"
+ id="tspan29165-8-2-9"
+ sodipodi:role="line">6</tspan></text>
+ </g>
+ <g
+ id="g29167-4-3"
+ transform="matrix(0.73232502,0,0,0.75477602,-154.60784,51.117791)">
+ <text
+ id="text29163-9-6"
+ y="70"
+ x="365"
+ style="font-style:normal;font-weight:normal;line-height:0%;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none"
+ xml:space="preserve"><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:24.2109px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="70"
+ x="365"
+ id="tspan29165-9-7"
+ sodipodi:role="line">Eventdev</tspan></text>
+ </g>
+ <g
+ id="g29167-4-3-5"
+ transform="matrix(0.73232502,0,0,0.75477602,-144.65044,201.97821)">
+ <text
+ id="text29163-9-6-3"
+ y="70"
+ x="412.93716"
+ style="font-style:normal;font-weight:normal;line-height:0%;font-family:sans-serif;text-align:center;letter-spacing:0px;word-spacing:0px;text-anchor:middle;fill:#000000;fill-opacity:1;stroke:none"
+ xml:space="preserve"><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:24.2109px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:center;writing-mode:lr-tb;text-anchor:middle"
+ y="70"
+ x="412.93716"
+ id="tspan29165-9-7-5"
+ sodipodi:role="line">DMA</tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:24.2109px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:center;writing-mode:lr-tb;text-anchor:middle"
+ y="100.26363"
+ x="412.93716"
+ sodipodi:role="line"
+ id="tspan3201">Adapter</tspan></text>
+ </g>
+ <g
+ id="g29167-4-3-5-6"
+ transform="matrix(0.73232502,0,0,0.75477602,79.53518,46.62529)">
+ <text
+ id="text29163-9-6-3-2"
+ y="48.801659"
+ x="412.93716"
+ style="font-style:normal;font-weight:normal;line-height:0%;font-family:sans-serif;text-align:center;letter-spacing:0px;word-spacing:0px;text-anchor:middle;fill:#000000;fill-opacity:1;stroke:none"
+ xml:space="preserve"><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:24.2109px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:center;writing-mode:lr-tb;text-anchor:middle"
+ y="48.801659"
+ x="412.93716"
+ sodipodi:role="line"
+ id="tspan3155">Application</tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:24.2109px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:center;writing-mode:lr-tb;text-anchor:middle"
+ y="98.801659"
+ x="412.93716"
+ sodipodi:role="line"
+ id="tspan3201-1">in ordered</tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:24.2109px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:center;writing-mode:lr-tb;text-anchor:middle"
+ y="148.80167"
+ x="412.93716"
+ sodipodi:role="line"
+ id="tspan3161">stage</tspan></text>
+ </g>
+ <g
+ id="g29167-4-3-5-2"
+ transform="matrix(0.73232502,0,0,0.75477602,77.535182,213.62529)">
+ <text
+ id="text29163-9-6-3-7"
+ y="70"
+ x="412.93716"
+ style="font-style:normal;font-weight:normal;line-height:0%;font-family:sans-serif;text-align:center;letter-spacing:0px;word-spacing:0px;text-anchor:middle;fill:#000000;fill-opacity:1;stroke:none"
+ xml:space="preserve"><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:24.2109px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:center;writing-mode:lr-tb;text-anchor:middle"
+ y="70"
+ x="412.93716"
+ sodipodi:role="line"
+ id="tspan3201-9">DMA Device</tspan></text>
+ </g>
+ <g
+ id="g29167-4-3-5-3"
+ transform="matrix(0.73232502,0,0,0.75477602,188.53518,-3.37471)">
+ <text
+ id="text29163-9-6-3-6"
+ y="70"
+ x="375.65271"
+ style="font-style:normal;font-weight:normal;line-height:0%;font-family:sans-serif;text-align:center;letter-spacing:0px;word-spacing:0px;text-anchor:middle;fill:#000000;fill-opacity:1;stroke:none"
+ xml:space="preserve"><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18.8307px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="70"
+ x="375.65271"
+ sodipodi:role="line"
+ id="tspan3201-6">1. Events from the previous stage.</tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18.8307px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="93.538376"
+ x="375.65271"
+ sodipodi:role="line"
+ id="tspan3260"> </tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18.8307px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="117.07675"
+ x="375.65271"
+ sodipodi:role="line"
+ id="tspan3262">2. Application in ordered stage</tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18.8307px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="140.61513"
+ x="375.65271"
+ sodipodi:role="line"
+ id="tspan3288"> dequeues events from eventdev.</tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18.8307px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="164.1535"
+ x="375.65271"
+ sodipodi:role="line"
+ id="tspan3264"> </tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18.8307px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="187.69188"
+ x="375.65271"
+ sodipodi:role="line"
+ id="tspan3266">3. Application enqueues DMA</tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18.8307px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="211.23026"
+ x="375.65271"
+ sodipodi:role="line"
+ id="tspan3290"> operations as events to eventdev.</tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18.8307px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="234.76863"
+ x="375.65271"
+ sodipodi:role="line"
+ id="tspan3268"> </tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18.8307px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="258.30701"
+ x="375.65271"
+ sodipodi:role="line"
+ id="tspan3270">4. DMA adapter dequeues event</tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18.8307px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="281.84537"
+ x="375.65271"
+ sodipodi:role="line"
+ id="tspan3292"> from eventdev.</tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18.8307px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="305.38376"
+ x="375.65271"
+ sodipodi:role="line"
+ id="tspan3272"> </tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18.8307px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="328.92212"
+ x="375.65271"
+ sodipodi:role="line"
+ id="tspan3274">5. DMA adapter submits DMA</tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18.8307px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="352.46051"
+ x="375.65271"
+ sodipodi:role="line"
+ id="tspan3294"> operations to DMA Device (Atomic</tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18.8307px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="375.99887"
+ x="375.65271"
+ sodipodi:role="line"
+ id="tspan3296"> stage)</tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18.8307px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="399.53726"
+ x="375.65271"
+ sodipodi:role="line"
+ id="tspan3276"> </tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18.8307px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="423.07562"
+ x="375.65271"
+ sodipodi:role="line"
+ id="tspan3278">6. DMA adapter dequeues DMA</tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18.8307px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="446.61401"
+ x="375.65271"
+ sodipodi:role="line"
+ id="tspan3298"> completions from DMA Device</tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18.8307px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="470.15237"
+ x="375.65271"
+ sodipodi:role="line"
+ id="tspan3280"> </tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18.8307px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="493.69073"
+ x="375.65271"
+ sodipodi:role="line"
+ id="tspan3282">7. DMA adapter enqueues events</tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18.8307px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="517.22913"
+ x="375.65271"
+ sodipodi:role="line"
+ id="tspan3300"> to the eventdev</tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18.8307px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="540.76752"
+ x="375.65271"
+ sodipodi:role="line"
+ id="tspan3284"> </tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18.8307px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="564.30585"
+ x="375.65271"
+ sodipodi:role="line"
+ id="tspan3286">8. Events to the next stage</tspan></text>
+ </g>
+ </g>
+</svg>
diff --git a/doc/guides/prog_guide/img/event_dma_adapter_op_new.svg b/doc/guides/prog_guide/img/event_dma_adapter_op_new.svg
new file mode 100644
index 0000000000..e9e8bb2b98
--- /dev/null
+++ b/doc/guides/prog_guide/img/event_dma_adapter_op_new.svg
@@ -0,0 +1,1079 @@
+<?xml version="1.0" encoding="UTF-8" standalone="no"?>
+<!-- SPDX-License-Identifier: BSD-3-Clause -->
+<!-- Copyright (c) 2023 Marvell. -->
+<!-- Created with Inkscape (http://www.inkscape.org/) -->
+
+<svg
+ width="720px"
+ height="486px"
+ id="svg13237"
+ version="1.1"
+ inkscape:version="1.2.1 (9c6d41e410, 2022-07-14)"
+ sodipodi:docname="event_dma_adapter_op_new.svg"
+ xmlns:inkscape="http://www.inkscape.org/namespaces/inkscape"
+ xmlns:sodipodi="http://sodipodi.sourceforge.net/DTD/sodipodi-0.dtd"
+ xmlns="http://www.w3.org/2000/svg"
+ xmlns:svg="http://www.w3.org/2000/svg"
+ xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#"
+ xmlns:cc="http://creativecommons.org/ns#"
+ xmlns:dc="http://purl.org/dc/elements/1.1/"
+ xmlns:v="http://schemas.microsoft.com/visio/2003/SVGExtensions/">
+ <defs
+ id="defs13239">
+ <marker
+ inkscape:stockid="Arrow1Sstart"
+ orient="auto"
+ refY="0.0"
+ refX="0.0"
+ id="Arrow1Sstart"
+ style="overflow:visible">
+ <path
+ id="path8416"
+ d="M 0.0,0.0 L 5.0,-5.0 L -12.5,0.0 L 5.0,5.0 L 0.0,0.0 z "
+ style="fill-rule:evenodd;stroke:#000000;stroke-width:1.0pt"
+ transform="scale(0.2) translate(6,0)" />
+ </marker>
+ <marker
+ inkscape:stockid="Arrow1Send"
+ orient="auto"
+ refY="0.0"
+ refX="0.0"
+ id="Arrow1Send"
+ style="overflow:visible;">
+ <path
+ id="path8419"
+ d="M 0.0,0.0 L 5.0,-5.0 L -12.5,0.0 L 5.0,5.0 L 0.0,0.0 z "
+ style="fill-rule:evenodd;stroke:#000000;stroke-width:1.0pt;"
+ transform="scale(0.2) rotate(180) translate(6,0)" />
+ </marker>
+ <marker
+ inkscape:stockid="DiamondL"
+ orient="auto"
+ refY="0.0"
+ refX="0.0"
+ id="DiamondL"
+ style="overflow:visible">
+ <path
+ id="path8483"
+ d="M 0,-7.0710768 L -7.0710894,0 L 0,7.0710589 L 7.0710462,0 L 0,-7.0710768 z "
+ style="fill-rule:evenodd;stroke:#000000;stroke-width:1.0pt"
+ transform="scale(0.8)" />
+ </marker>
+ <marker
+ inkscape:stockid="DotL"
+ orient="auto"
+ refY="0.0"
+ refX="0.0"
+ id="DotL"
+ style="overflow:visible">
+ <path
+ id="path8465"
+ d="M -2.5,-1.0 C -2.5,1.7600000 -4.7400000,4.0 -7.5,4.0 C -10.260000,4.0 -12.5,1.7600000 -12.5,-1.0 C -12.5,-3.7600000 -10.260000,-6.0 -7.5,-6.0 C -4.7400000,-6.0 -2.5,-3.7600000 -2.5,-1.0 z "
+ style="fill-rule:evenodd;stroke:#000000;stroke-width:1.0pt"
+ transform="scale(0.8) translate(7.4, 1)" />
+ </marker>
+ <marker
+ inkscape:stockid="SquareL"
+ orient="auto"
+ refY="0.0"
+ refX="0.0"
+ id="SquareL"
+ style="overflow:visible">
+ <path
+ id="path8474"
+ d="M -5.0,-5.0 L -5.0,5.0 L 5.0,5.0 L 5.0,-5.0 L -5.0,-5.0 z "
+ style="fill-rule:evenodd;stroke:#000000;stroke-width:1.0pt"
+ transform="scale(0.8)" />
+ </marker>
+ <marker
+ inkscape:stockid="TriangleOutL"
+ orient="auto"
+ refY="0.0"
+ refX="0.0"
+ id="TriangleOutL"
+ style="overflow:visible">
+ <path
+ id="path8546"
+ d="M 5.77,0.0 L -2.88,5.0 L -2.88,-5.0 L 5.77,0.0 z "
+ style="fill-rule:evenodd;stroke:#000000;stroke-width:1.0pt"
+ transform="scale(0.8)" />
+ </marker>
+ <marker
+ inkscape:stockid="Arrow1Lstart"
+ orient="auto"
+ refY="0.0"
+ refX="0.0"
+ id="Arrow1Lstart"
+ style="overflow:visible">
+ <path
+ id="path8404"
+ d="M 0.0,0.0 L 5.0,-5.0 L -12.5,0.0 L 5.0,5.0 L 0.0,0.0 z "
+ style="fill-rule:evenodd;stroke:#000000;stroke-width:1.0pt"
+ transform="scale(0.8) translate(12.5,0)" />
+ </marker>
+ <marker
+ inkscape:stockid="Arrow1Mend"
+ orient="auto"
+ refY="0.0"
+ refX="0.0"
+ id="Arrow1Mend"
+ style="overflow:visible;">
+ <path
+ id="path8413"
+ d="M 0.0,0.0 L 5.0,-5.0 L -12.5,0.0 L 5.0,5.0 L 0.0,0.0 z "
+ style="fill-rule:evenodd;stroke:#000000;stroke-width:1.0pt;"
+ transform="scale(0.4) rotate(180) translate(10,0)" />
+ </marker>
+ <marker
+ inkscape:stockid="Arrow2Lend"
+ orient="auto"
+ refY="0.0"
+ refX="0.0"
+ id="Arrow2Lend"
+ style="overflow:visible;">
+ <path
+ id="path8425"
+ style="fill-rule:evenodd;stroke-width:0.62500000;stroke-linejoin:round;"
+ d="M 8.7185878,4.0337352 L -2.2072895,0.016013256 L 8.7185884,-4.0017078 C 6.9730900,-1.6296469 6.9831476,1.6157441 8.7185878,4.0337352 z "
+ transform="scale(1.1) rotate(180) translate(1,0)" />
+ </marker>
+ <marker
+ inkscape:stockid="Arrow1Lend"
+ orient="auto"
+ refY="0.0"
+ refX="0.0"
+ id="Arrow1Lend"
+ style="overflow:visible;">
+ <path
+ id="path8407"
+ d="M 0.0,0.0 L 5.0,-5.0 L -12.5,0.0 L 5.0,5.0 L 0.0,0.0 z "
+ style="fill-rule:evenodd;stroke:#000000;stroke-width:1.0pt;"
+ transform="scale(0.8) rotate(180) translate(12.5,0)" />
+ </marker>
+ <filter
+ id="filter_2"
+ color-interpolation-filters="sRGB">
+ <feGaussianBlur
+ stdDeviation="2"
+ id="feGaussianBlur15" />
+ </filter>
+ <filter
+ id="filter_2-3"
+ color-interpolation-filters="sRGB"
+ x="-0.086178862"
+ y="-0.086178862"
+ width="1.1723577"
+ height="1.1723577">
+ <feGaussianBlur
+ stdDeviation="2"
+ id="feGaussianBlur15-1" />
+ </filter>
+ <filter
+ id="filter_2-0"
+ color-interpolation-filters="sRGB">
+ <feGaussianBlur
+ stdDeviation="2"
+ id="feGaussianBlur15-7" />
+ </filter>
+ <filter
+ id="filter_2-0-8"
+ color-interpolation-filters="sRGB">
+ <feGaussianBlur
+ stdDeviation="2"
+ id="feGaussianBlur15-7-7" />
+ </filter>
+ <filter
+ id="filter_2-3-9"
+ color-interpolation-filters="sRGB"
+ x="-0.086178862"
+ y="-0.086178862"
+ width="1.1723577"
+ height="1.1723577">
+ <feGaussianBlur
+ stdDeviation="2"
+ id="feGaussianBlur15-1-6" />
+ </filter>
+ <filter
+ id="filter_2-3-6"
+ color-interpolation-filters="sRGB"
+ x="-0.086178862"
+ y="-0.086178862"
+ width="1.1723577"
+ height="1.1723577">
+ <feGaussianBlur
+ stdDeviation="2"
+ id="feGaussianBlur15-1-63" />
+ </filter>
+ <filter
+ id="filter_2-3-91"
+ color-interpolation-filters="sRGB"
+ x="-0.086178862"
+ y="-0.086178862"
+ width="1.1723577"
+ height="1.1723577">
+ <feGaussianBlur
+ stdDeviation="2"
+ id="feGaussianBlur15-1-3" />
+ </filter>
+ <marker
+ inkscape:stockid="Arrow1Lend"
+ orient="auto"
+ refY="0"
+ refX="0"
+ id="Arrow1Lend-5"
+ style="overflow:visible">
+ <path
+ inkscape:connector-curvature="0"
+ id="path8407-3"
+ d="M 0,0 5,-5 -12.5,0 5,5 0,0 z"
+ style="fill-rule:evenodd;stroke:#000000;stroke-width:1pt"
+ transform="matrix(-0.8,0,0,-0.8,-10,0)" />
+ </marker>
+ <marker
+ inkscape:stockid="Arrow1Lend"
+ orient="auto"
+ refY="0"
+ refX="0"
+ id="Arrow1Lend-6"
+ style="overflow:visible">
+ <path
+ inkscape:connector-curvature="0"
+ id="path8407-0"
+ d="M 0,0 5,-5 -12.5,0 5,5 0,0 z"
+ style="fill-rule:evenodd;stroke:#000000;stroke-width:1pt"
+ transform="matrix(-0.8,0,0,-0.8,-10,0)" />
+ </marker>
+ <marker
+ inkscape:stockid="Arrow1Lstart"
+ orient="auto"
+ refY="0"
+ refX="0"
+ id="Arrow1Lstart-7"
+ style="overflow:visible">
+ <path
+ inkscape:connector-curvature="0"
+ id="path8404-0"
+ d="M 0,0 5,-5 -12.5,0 5,5 0,0 z"
+ style="fill-rule:evenodd;stroke:#000000;stroke-width:1pt"
+ transform="matrix(0.8,0,0,0.8,10,0)" />
+ </marker>
+ <marker
+ inkscape:stockid="Arrow1Lend"
+ orient="auto"
+ refY="0"
+ refX="0"
+ id="Arrow1Lend-51"
+ style="overflow:visible">
+ <path
+ inkscape:connector-curvature="0"
+ id="path8407-1"
+ d="M 0,0 5,-5 -12.5,0 5,5 0,0 z"
+ style="fill-rule:evenodd;stroke:#000000;stroke-width:1pt"
+ transform="matrix(-0.8,0,0,-0.8,-10,0)" />
+ </marker>
+ <marker
+ inkscape:stockid="Arrow1Lend"
+ orient="auto"
+ refY="0"
+ refX="0"
+ id="Arrow1Lend-3"
+ style="overflow:visible">
+ <path
+ inkscape:connector-curvature="0"
+ id="path8407-6"
+ d="M 0,0 5,-5 -12.5,0 5,5 0,0 z"
+ style="fill-rule:evenodd;stroke:#000000;stroke-width:1pt"
+ transform="matrix(-0.8,0,0,-0.8,-10,0)" />
+ </marker>
+ <marker
+ inkscape:stockid="Arrow1Lend"
+ orient="auto"
+ refY="0"
+ refX="0"
+ id="Arrow1Lend-62"
+ style="overflow:visible">
+ <path
+ inkscape:connector-curvature="0"
+ id="path8407-9"
+ d="M 0,0 5,-5 -12.5,0 5,5 0,0 z"
+ style="fill-rule:evenodd;stroke:#000000;stroke-width:1pt"
+ transform="matrix(-0.8,0,0,-0.8,-10,0)" />
+ </marker>
+ <marker
+ inkscape:stockid="Arrow1Lend"
+ orient="auto"
+ refY="0"
+ refX="0"
+ id="Arrow1Lend-2"
+ style="overflow:visible">
+ <path
+ inkscape:connector-curvature="0"
+ id="path8407-7"
+ d="M 0,0 5,-5 -12.5,0 5,5 0,0 z"
+ style="fill-rule:evenodd;stroke:#000000;stroke-width:1pt"
+ transform="matrix(-0.8,0,0,-0.8,-10,0)" />
+ </marker>
+ <filter
+ id="filter_2-3-91-3"
+ color-interpolation-filters="sRGB"
+ x="-0.086178862"
+ y="-0.086178862"
+ width="1.1723577"
+ height="1.1723577">
+ <feGaussianBlur
+ stdDeviation="2"
+ id="feGaussianBlur15-1-3-6" />
+ </filter>
+ </defs>
+ <sodipodi:namedview
+ id="base"
+ pagecolor="#ffffff"
+ bordercolor="#666666"
+ borderopacity="1.0"
+ inkscape:pageopacity="0.0"
+ inkscape:pageshadow="2"
+ inkscape:zoom="1"
+ inkscape:cx="385.5"
+ inkscape:cy="234"
+ inkscape:document-units="px"
+ inkscape:current-layer="layer1"
+ showgrid="false"
+ inkscape:window-width="1920"
+ inkscape:window-height="1017"
+ inkscape:window-x="-8"
+ inkscape:window-y="-8"
+ inkscape:window-maximized="1"
+ inkscape:snap-nodes="false"
+ inkscape:showpageshadow="2"
+ inkscape:pagecheckerboard="0"
+ inkscape:deskcolor="#d1d1d1">
+ <inkscape:grid
+ type="xygrid"
+ id="grid13454" />
+ </sodipodi:namedview>
+ <metadata
+ id="metadata13242">
+ <rdf:RDF>
+ <cc:Work
+ rdf:about="">
+ <dc:format>image/svg+xml</dc:format>
+ <dc:type
+ rdf:resource="http://purl.org/dc/dcmitype/StillImage" />
+ <dc:title />
+ </cc:Work>
+ </rdf:RDF>
+ </metadata>
+ <g
+ id="layer1"
+ inkscape:label="Layer 1"
+ inkscape:groupmode="layer">
+ <g
+ style="font-size:12px;fill:none;stroke-linecap:square;stroke-miterlimit:3;overflow:visible"
+ id="shape1-1-2-0"
+ v:mID="1"
+ v:groupContext="shape"
+ transform="matrix(2.1604167,0,0,1.5671361,323.2187,-540.25927)">
+ <title
+ id="title22-7-6">Square</title>
+ <desc
+ id="desc24-7-4">Atomic Queue #1</desc>
+ <v:userDefs>
+ <v:ud
+ v:nameU="visVersion"
+ v:val="VT0(15):26" />
+ </v:userDefs>
+ <v:textBlock
+ v:margins="rect(4,4,4,4)" />
+ <v:textRect
+ cx="30.75"
+ cy="581.25"
+ width="61.5"
+ height="61.5" />
+ <g
+ id="shadow1-2-9-0"
+ v:groupContext="shadow"
+ v:shadowOffsetX="0.345598"
+ v:shadowOffsetY="-1.97279"
+ v:shadowType="1"
+ transform="translate(0.345598,1.97279)"
+ class="st1"
+ style="visibility:visible">
+ <rect
+ x="0"
+ y="550.5"
+ width="61.5"
+ height="61.5"
+ class="st2"
+ id="rect27-8-9"
+ style="fill:#5b9bd5;fill-opacity:0.22000002;stroke:#5b9bd5;stroke-opacity:0.22000002;filter:url(#filter_2-3-9)" />
+ </g>
+ <g
+ id="g13515-4">
+ <g
+ id="g13534-5">
+ <rect
+ x="0"
+ y="550.5"
+ width="61.5"
+ height="61.5"
+ class="st3"
+ id="rect29-1-4"
+ style="fill:#5b9bd5;stroke:#c7c8c8;stroke-width:0.25" />
+ </g>
+ </g>
+ </g>
+ <g
+ style="font-size:12px;fill:none;stroke-linecap:square;stroke-miterlimit:3;overflow:visible"
+ id="shape1-1-2-4"
+ v:mID="1"
+ v:groupContext="shape"
+ transform="matrix(2.1604167,0,0,1.165886,88.874699,-447.8809)">
+ <title
+ id="title22-7-5">Square</title>
+ <desc
+ id="desc24-7-8">Atomic Queue #1</desc>
+ <v:userDefs>
+ <v:ud
+ v:nameU="visVersion"
+ v:val="VT0(15):26" />
+ </v:userDefs>
+ <v:textBlock
+ v:margins="rect(4,4,4,4)" />
+ <v:textRect
+ cx="30.75"
+ cy="581.25"
+ width="61.5"
+ height="61.5" />
+ <g
+ id="shadow1-2-9-5"
+ v:groupContext="shadow"
+ v:shadowOffsetX="0.345598"
+ v:shadowOffsetY="-1.97279"
+ v:shadowType="1"
+ transform="translate(0.345598,1.97279)"
+ class="st1"
+ style="visibility:visible">
+ <rect
+ x="0"
+ y="550.5"
+ width="61.5"
+ height="61.5"
+ class="st2"
+ id="rect27-8-7"
+ style="fill:#5b9bd5;fill-opacity:0.22000002;stroke:#5b9bd5;stroke-opacity:0.22000002;filter:url(#filter_2-3-91)" />
+ </g>
+ <g
+ id="g13515-33">
+ <g
+ id="g13534-8">
+ <rect
+ x="0"
+ y="550.5"
+ width="61.5"
+ height="61.5"
+ class="st3"
+ id="rect29-1-95"
+ style="fill:#5b9bd5;stroke:#c7c8c8;stroke-width:0.25" />
+ </g>
+ </g>
+ </g>
+ <g
+ style="font-size:12px;fill:none;stroke-linecap:square;stroke-miterlimit:3;overflow:visible"
+ id="shape1-1-2-9"
+ v:mID="1"
+ v:groupContext="shape"
+ transform="matrix(2.1604167,0,0,1.5671361,88.874699,-538.24651)">
+ <title
+ id="title22-7-9">Square</title>
+ <desc
+ id="desc24-7-5">Atomic Queue #1</desc>
+ <v:userDefs>
+ <v:ud
+ v:nameU="visVersion"
+ v:val="VT0(15):26" />
+ </v:userDefs>
+ <v:textBlock
+ v:margins="rect(4,4,4,4)" />
+ <v:textRect
+ cx="30.75"
+ cy="581.25"
+ width="61.5"
+ height="61.5" />
+ <g
+ id="shadow1-2-9-2"
+ v:groupContext="shadow"
+ v:shadowOffsetX="0.345598"
+ v:shadowOffsetY="-1.97279"
+ v:shadowType="1"
+ transform="translate(0.345598,1.97279)"
+ class="st1"
+ style="visibility:visible">
+ <rect
+ x="0"
+ y="550.5"
+ width="61.5"
+ height="61.5"
+ class="st2"
+ id="rect27-8-1"
+ style="fill:#5b9bd5;fill-opacity:0.22000002;stroke:#5b9bd5;stroke-opacity:0.22000002;filter:url(#filter_2-3-6)" />
+ </g>
+ <g
+ id="g13515-3">
+ <g
+ id="g13534-0">
+ <rect
+ x="0"
+ y="550.5"
+ width="61.5"
+ height="61.5"
+ class="st3"
+ id="rect29-1-9"
+ style="fill:#5b9bd5;stroke:#c7c8c8;stroke-width:0.25" />
+ </g>
+ </g>
+ </g>
+ <path
+ style="fill:none;stroke:#000000;stroke-width:0.743466px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1;marker-mid:none;marker-end:url(#Arrow1Lend)"
+ d="M 220.66064,98.57125 H 321.88592"
+ id="path17209-8"
+ inkscape:connector-type="orthogonal"
+ inkscape:connector-curvature="0" />
+ <g
+ style="font-size:12px;fill:none;stroke-linecap:square;stroke-miterlimit:3;overflow:visible"
+ id="shape1-1-2"
+ v:mID="1"
+ v:groupContext="shape"
+ transform="matrix(2.1604167,0,0,1.5671361,322.24227,-811.89589)">
+ <title
+ id="title22-7">Square</title>
+ <desc
+ id="desc24-7">Atomic Queue #1</desc>
+ <v:userDefs>
+ <v:ud
+ v:nameU="visVersion"
+ v:val="VT0(15):26" />
+ </v:userDefs>
+ <v:textBlock
+ v:margins="rect(4,4,4,4)" />
+ <v:textRect
+ cx="30.75"
+ cy="581.25"
+ width="61.5"
+ height="61.5" />
+ <g
+ id="shadow1-2-9"
+ v:groupContext="shadow"
+ v:shadowOffsetX="0.345598"
+ v:shadowOffsetY="-1.97279"
+ v:shadowType="1"
+ transform="translate(0.345598,1.97279)"
+ class="st1"
+ style="visibility:visible">
+ <rect
+ x="0"
+ y="550.5"
+ width="61.5"
+ height="61.5"
+ class="st2"
+ id="rect27-8"
+ style="fill:#5b9bd5;fill-opacity:0.22000002;stroke:#5b9bd5;stroke-opacity:0.22000002;filter:url(#filter_2-3)" />
+ </g>
+ <g
+ id="g13515">
+ <g
+ id="g13534">
+ <rect
+ x="0"
+ y="550.5"
+ width="61.5"
+ height="61.5"
+ class="st3"
+ id="rect29-1"
+ style="fill:#5b9bd5;stroke:#c7c8c8;stroke-width:0.25" />
+ </g>
+ </g>
+ </g>
+ <g
+ id="g13518"
+ transform="matrix(0.73232502,0,0,0.75477602,25.29268,348.89752)">
+ <g
+ id="g13526">
+ <flowRoot
+ xml:space="preserve"
+ id="flowRoot13464-9"
+ style="font-style:normal;font-weight:normal;line-height:0.01%;font-family:sans-serif;text-align:center;letter-spacing:0px;word-spacing:0px;text-anchor:middle;fill:#000000;fill-opacity:1;stroke:none"
+ transform="translate(-12.00521,-129.65179)"><flowRegion
+ id="flowRegion13466-1"
+ style="font-family:sans-serif"><rect
+ id="rect13468-2"
+ width="195.99997"
+ height="112.00001"
+ x="273.33334"
+ y="175.33333"
+ style="text-align:center;text-anchor:middle" /></flowRegion><flowPara
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:20px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:center;writing-mode:lr-tb;text-anchor:middle"
+ id="flowPara13511"> </flowPara></flowRoot>
+ </g>
+ </g>
+ <path
+ style="fill:none;stroke:#000000;stroke-width:0.751455;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:4;stroke-dasharray:none;stroke-opacity:1;marker-start:url(#Arrow1Lstart);marker-end:none"
+ d="m 176.26096,124.64833 v 69.24854"
+ id="path17209-3"
+ inkscape:connector-type="orthogonal"
+ inkscape:connector-curvature="0" />
+ <path
+ transform="matrix(0.73232502,0,0,0.75477602,-4.325033,50.642983)"
+ sodipodi:type="arc"
+ style="fill:#539de6;fill-opacity:1;stroke:#0000ea;stroke-width:1;stroke-linecap:square;stroke-miterlimit:4;stroke-opacity:1;stroke-dasharray:none"
+ id="path29161-3"
+ sodipodi:cx="371"
+ sodipodi:cy="64.5"
+ sodipodi:rx="17"
+ sodipodi:ry="15.5"
+ d="m 388,64.5 a 17,15.5 0 1 1 -34,0 17,15.5 0 1 1 34,0 z" />
+ <path
+ style="fill:none;stroke:#000000;stroke-width:0.743466px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1;marker-mid:none;marker-end:url(#Arrow1Lend)"
+ d="M 322.61264,375 H 221.38736"
+ id="path17209-8-0"
+ inkscape:connector-type="orthogonal"
+ inkscape:connector-curvature="0" />
+ <path
+ transform="matrix(0.73232502,0,0,0.75477602,0.0689171,324.80833)"
+ sodipodi:type="arc"
+ style="fill:#539de6;fill-opacity:1;stroke:#0000ea;stroke-width:1;stroke-linecap:square;stroke-miterlimit:4;stroke-opacity:1;stroke-dasharray:none"
+ id="path29161-1"
+ sodipodi:cx="371"
+ sodipodi:cy="64.5"
+ sodipodi:rx="17"
+ sodipodi:ry="15.5"
+ d="m 388,64.5 a 17,15.5 0 1 1 -34,0 17,15.5 0 1 1 34,0 z" />
+ <path
+ style="fill:none;stroke:#000000;stroke-width:0.629082px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1;marker-end:url(#Arrow1Lend)"
+ d="M 155,324.19955 V 264.82863"
+ id="path17211-7-1"
+ inkscape:connector-type="orthogonal"
+ inkscape:connector-curvature="0" />
+ <path
+ transform="matrix(0.73232502,0,0,0.75477602,-116.37076,245.97088)"
+ sodipodi:type="arc"
+ style="fill:#539de6;fill-opacity:1;stroke:#0000ea;stroke-width:1;stroke-linecap:square;stroke-miterlimit:4;stroke-opacity:1;stroke-dasharray:none"
+ id="path29161-8"
+ sodipodi:cx="371"
+ sodipodi:cy="64.5"
+ sodipodi:rx="17"
+ sodipodi:ry="15.5"
+ d="m 388,64.5 a 17,15.5 0 1 1 -34,0 17,15.5 0 1 1 34,0 z" />
+ <path
+ style="fill:none;stroke:#000000;stroke-width:0.75059;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:4;stroke-dasharray:none;stroke-opacity:1;marker-start:url(#Arrow1Lstart-7);marker-end:none"
+ d="m 126.26097,124.99178 v 69.24941"
+ id="path17209-3-0"
+ inkscape:connector-type="orthogonal"
+ inkscape:connector-curvature="0" />
+ <path
+ sodipodi:type="arc"
+ style="fill:#539de6;fill-opacity:1;stroke:#0000ea;stroke-width:1;stroke-linecap:square;stroke-miterlimit:4;stroke-opacity:1;stroke-dasharray:none"
+ id="path29161"
+ sodipodi:cx="371"
+ sodipodi:cy="64.5"
+ sodipodi:rx="17"
+ sodipodi:ry="15.5"
+ d="m 388,64.5 a 17,15.5 0 1 1 -34,0 17,15.5 0 1 1 34,0 z"
+ transform="matrix(0.73232502,0,0,0.75477602,-146.16394,110.68276)" />
+ <path
+ sodipodi:type="arc"
+ style="fill:#539de6;fill-opacity:1;stroke:#0000ea;stroke-width:1;stroke-linecap:square;stroke-miterlimit:4;stroke-opacity:1;stroke-dasharray:none"
+ id="path29161-2"
+ sodipodi:cx="371"
+ sodipodi:cy="64.5"
+ sodipodi:rx="17"
+ sodipodi:ry="15.5"
+ d="m 388,64.5 a 17,15.5 0 1 1 -34,0 17,15.5 0 1 1 34,0 z"
+ transform="matrix(0.73232502,0,0,0.75477602,-95.40136,110.71627)" />
+ <g
+ id="g29167-4"
+ transform="matrix(0.73232502,0,0,0.75477602,-95.31662,112.00756)">
+ <text
+ id="text29163-9"
+ y="70"
+ x="365"
+ style="font-style:normal;font-weight:normal;line-height:0%;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none"
+ xml:space="preserve"><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="70"
+ x="365"
+ id="tspan29165-9"
+ sodipodi:role="line">1</tspan></text>
+ </g>
+ <g
+ id="g29167-9"
+ transform="matrix(0.73232502,0,0,0.75477602,-4.9726112,50.689051)">
+ <text
+ id="text29163-3"
+ y="70"
+ x="365"
+ style="font-style:normal;font-weight:normal;line-height:0%;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none"
+ xml:space="preserve"><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="70"
+ x="365"
+ id="tspan29165-3"
+ sodipodi:role="line">2</tspan></text>
+ </g>
+ <path
+ style="fill:none;stroke:#000000;stroke-width:1.04033px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1;marker-end:url(#Arrow1Lend)"
+ d="M 388.20118,147.93341 V 321.89308"
+ id="path17211-7"
+ inkscape:connector-type="orthogonal"
+ inkscape:connector-curvature="0" />
+ <path
+ transform="matrix(0.73232502,0,0,0.75477602,116.5086,136.68745)"
+ sodipodi:type="arc"
+ style="fill:#539de6;fill-opacity:1;stroke:#0000ea;stroke-width:1;stroke-linecap:square;stroke-miterlimit:4;stroke-opacity:1;stroke-dasharray:none"
+ id="path29161-6"
+ sodipodi:cx="371"
+ sodipodi:cy="64.5"
+ sodipodi:rx="17"
+ sodipodi:ry="15.5"
+ d="m 388,64.5 a 17,15.5 0 1 1 -34,0 17,15.5 0 1 1 34,0 z" />
+ <g
+ id="g29167-46"
+ transform="matrix(0.73232502,0,0,0.75477602,116.59334,137.48829)">
+ <text
+ id="text29163-1"
+ y="70"
+ x="365"
+ style="font-style:normal;font-weight:normal;line-height:0%;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none"
+ xml:space="preserve"><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="70"
+ x="365"
+ id="tspan29165-5"
+ sodipodi:role="line">3</tspan></text>
+ </g>
+ <g
+ id="g29167-6"
+ transform="matrix(0.73232502,0,0,0.75477602,0.1536639,325.60918)">
+ <text
+ id="text29163-17"
+ y="70"
+ x="365"
+ style="font-style:normal;font-weight:normal;line-height:0%;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none"
+ xml:space="preserve"><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="70"
+ x="365"
+ id="tspan29165-2"
+ sodipodi:role="line">4</tspan></text>
+ </g>
+ <g
+ id="g29167"
+ transform="matrix(0.73232502,0,0,0.75477602,-146.07919,111.10621)">
+ <text
+ id="text29163"
+ y="70"
+ x="365"
+ style="font-style:normal;font-weight:normal;line-height:0%;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none"
+ xml:space="preserve"><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="70"
+ x="365"
+ id="tspan29165"
+ sodipodi:role="line">6</tspan></text>
+ </g>
+ <g
+ id="g29167-4-3"
+ transform="matrix(0.73232502,0,0,0.75477602,-117.60784,180.11779)">
+ <text
+ id="text29163-9-6"
+ y="70"
+ x="321.30356"
+ style="font-style:normal;font-weight:normal;line-height:0%;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none"
+ xml:space="preserve"><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:24.2109px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="70"
+ x="321.30356"
+ id="tspan29165-9-7"
+ sodipodi:role="line">Eventdev</tspan></text>
+ </g>
+ <g
+ id="g29167-4-3-5"
+ transform="matrix(0.73232502,0,0,0.75477602,55.34956,26.97821)">
+ <text
+ id="text29163-9-6-3"
+ y="70"
+ x="454.74152"
+ style="font-style:normal;font-weight:normal;line-height:0%;font-family:sans-serif;text-align:center;letter-spacing:0px;word-spacing:0px;text-anchor:middle;fill:#000000;fill-opacity:1;stroke:none"
+ xml:space="preserve"><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:21.5208px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:center;writing-mode:lr-tb;text-anchor:middle"
+ y="70"
+ x="454.74152"
+ id="tspan29165-9-7-5"
+ sodipodi:role="line">A<tspan
+ style="line-height:100%;font-family:sans-serif"
+ id="tspan3374">tomic Stage</tspan></tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:21.5208px;line-height:100%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:center;writing-mode:lr-tb;text-anchor:middle"
+ y="94.210899"
+ x="454.74152"
+ sodipodi:role="line"
+ id="tspan3320">+</tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:21.5208px;line-height:100%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:center;writing-mode:lr-tb;text-anchor:middle"
+ y="115.7317"
+ x="454.74152"
+ sodipodi:role="line"
+ id="tspan3322">Enqueue to</tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:21.5208px;line-height:100%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:center;writing-mode:lr-tb;text-anchor:middle"
+ y="137.2525"
+ x="454.74152"
+ sodipodi:role="line"
+ id="tspan3324">DMA Device</tspan></text>
+ </g>
+ <g
+ id="g29167-2"
+ transform="matrix(0.73232502,0,0,0.75477602,-116.28602,248.01695)">
+ <text
+ id="text29163-92"
+ y="70"
+ x="365"
+ style="font-style:normal;font-weight:normal;line-height:0%;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none"
+ xml:space="preserve"><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="70"
+ x="365"
+ id="tspan29165-8"
+ sodipodi:role="line">5</tspan></text>
+ </g>
+ <flowRoot
+ xml:space="preserve"
+ id="flowRoot3376"
+ style="font-style:normal;font-weight:normal;line-height:0.01%;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none;stroke-width:1px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1"><flowRegion
+ id="flowRegion3378"
+ style="font-family:sans-serif"><rect
+ id="rect3380"
+ width="100"
+ height="37"
+ x="109"
+ y="259" /></flowRegion><flowPara
+ id="flowPara3382"
+ style="font-size:18px;line-height:1.25;font-family:sans-serif"> </flowPara></flowRoot>
+ <g
+ id="g29167-4-3-1"
+ transform="matrix(0.73232502,0,0,0.75477602,109.34956,323.97821)">
+ <text
+ id="text29163-9-6-8"
+ y="70"
+ x="321.30356"
+ style="font-style:normal;font-weight:normal;line-height:0%;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none"
+ xml:space="preserve"><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:24.2109px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="70"
+ x="321.30356"
+ id="tspan29165-9-7-7"
+ sodipodi:role="line">DMA Device</tspan></text>
+ </g>
+ <g
+ id="g29167-4-3-1-9"
+ transform="matrix(0.73232502,0,0,0.75477602,-114.48565,314.20704)">
+ <text
+ id="text29163-9-6-8-2"
+ y="70"
+ x="368.01718"
+ style="font-style:normal;font-weight:normal;line-height:0%;font-family:sans-serif;text-align:center;letter-spacing:0px;word-spacing:0px;text-anchor:middle;fill:#000000;fill-opacity:1;stroke:none"
+ xml:space="preserve"><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:24.2109px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:center;writing-mode:lr-tb;text-anchor:middle"
+ y="70"
+ x="368.01718"
+ id="tspan29165-9-7-7-0"
+ sodipodi:role="line">DMA</tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:24.2109px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:center;writing-mode:lr-tb;text-anchor:middle"
+ y="100.26363"
+ x="368.01718"
+ sodipodi:role="line"
+ id="tspan3488">Adapter</tspan></text>
+ </g>
+ <g
+ id="g29167-4-3-1-9-2"
+ transform="matrix(0.73232502,0,0,0.75477602,250.96804,192.62529)">
+ <text
+ id="text29163-9-6-8-2-3"
+ y="-188.35481"
+ x="318.61978"
+ style="font-style:normal;font-weight:normal;line-height:0%;font-family:sans-serif;text-align:center;letter-spacing:0px;word-spacing:0px;text-anchor:middle;fill:#000000;fill-opacity:1;stroke:none"
+ xml:space="preserve"><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:21.5208px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="-188.35481"
+ x="318.61978"
+ sodipodi:role="line"
+ id="tspan3543">1. Application dequeues</tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:21.5208px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="-161.45381"
+ x="318.61978"
+ sodipodi:role="line"
+ id="tspan3196"> events from the previous</tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:21.5208px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="-134.55281"
+ x="318.61978"
+ sodipodi:role="line"
+ id="tspan3232"> stage</tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:21.5208px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="-107.65182"
+ x="318.61978"
+ sodipodi:role="line"
+ id="tspan3519"> </tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:21.5208px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="-80.750816"
+ x="318.61978"
+ sodipodi:role="line"
+ id="tspan3551">2. Application prepares the</tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:21.5208px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="-53.849815"
+ x="318.61978"
+ sodipodi:role="line"
+ id="tspan3203"> DMA operations.</tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:21.5208px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="-26.948814"
+ x="318.61978"
+ sodipodi:role="line"
+ id="tspan3523"> </tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:21.5208px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="-0.0478158"
+ x="318.61978"
+ sodipodi:role="line"
+ id="tspan3541">3. DMA operations are</tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:21.5208px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="26.853184"
+ x="318.61978"
+ sodipodi:role="line"
+ id="tspan3207"> submitted to dmadev</tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:21.5208px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="53.754185"
+ x="318.61978"
+ sodipodi:role="line"
+ id="tspan3209"> by application.</tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:21.5208px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="80.655182"
+ x="318.61978"
+ sodipodi:role="line"
+ id="tspan3527"> </tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:21.5208px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="107.55618"
+ x="318.61978"
+ sodipodi:role="line"
+ id="tspan3547">4. DMA adapter dequeues</tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:21.5208px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="134.45718"
+ x="318.61978"
+ sodipodi:role="line"
+ id="tspan3216"> DMA completions from</tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:21.5208px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="161.35818"
+ x="318.61978"
+ sodipodi:role="line"
+ id="tspan3218"> DMA device.</tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:21.5208px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="188.25919"
+ x="318.61978"
+ sodipodi:role="line"
+ id="tspan3531"> </tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:21.5208px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="215.16019"
+ x="318.61978"
+ sodipodi:role="line"
+ id="tspan3549">5. DMA adapter enqueues</tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:21.5208px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="242.06119"
+ x="318.61978"
+ sodipodi:role="line"
+ id="tspan3222"> events to the eventdev.</tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:21.5208px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="268.96219"
+ x="318.61978"
+ sodipodi:role="line"
+ id="tspan3535"> </tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:21.5208px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="295.86319"
+ x="318.61978"
+ sodipodi:role="line"
+ id="tspan3537">6. Application dequeues from</tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:21.5208px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="322.76419"
+ x="318.61978"
+ sodipodi:role="line"
+ id="tspan3224"> eventdev and prepare for</tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:21.5208px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="349.66519"
+ x="318.61978"
+ sodipodi:role="line"
+ id="tspan3226"> further processing</tspan></text>
+ </g>
+ <g
+ style="font-size:12px;fill:none;stroke-linecap:square;stroke-miterlimit:3;overflow:visible"
+ id="shape1-1-2-4-7"
+ v:mID="1"
+ v:groupContext="shape"
+ transform="matrix(2.1604167,0,0,1.165886,90.820551,-587.97129)">
+ <title
+ id="title22-7-5-5">Square</title>
+ <desc
+ id="desc24-7-8-3">Atomic Queue #1</desc>
+ <v:userDefs>
+ <v:ud
+ v:nameU="visVersion"
+ v:val="VT0(15):26" />
+ </v:userDefs>
+ <v:textBlock
+ v:margins="rect(4,4,4,4)" />
+ <v:textRect
+ cx="30.75"
+ cy="581.25"
+ width="61.5"
+ height="61.5" />
+ <g
+ id="shadow1-2-9-5-5"
+ v:groupContext="shadow"
+ v:shadowOffsetX="0.345598"
+ v:shadowOffsetY="-1.97279"
+ v:shadowType="1"
+ transform="translate(0.345598,1.97279)"
+ class="st1"
+ style="visibility:visible">
+ <rect
+ x="0"
+ y="550.5"
+ width="61.5"
+ height="61.5"
+ class="st2"
+ id="rect27-8-7-6"
+ style="fill:#5b9bd5;fill-opacity:0.22000002;stroke:#5b9bd5;stroke-opacity:0.22000002;filter:url(#filter_2-3-91-3)" />
+ </g>
+ <g
+ id="g13515-33-2">
+ <g
+ id="g13534-8-9">
+ <rect
+ x="0"
+ y="550.5"
+ width="61.5"
+ height="61.5"
+ class="st3"
+ id="rect29-1-95-1"
+ style="fill:#5b9bd5;stroke:#c7c8c8;stroke-width:0.25" />
+ </g>
+ </g>
+ </g>
+ <g
+ id="g29167-4-3-2"
+ transform="matrix(0.73232502,0,0,0.75477602,-125.66199,44.027402)">
+ <text
+ id="text29163-9-6-7"
+ y="70"
+ x="321.30356"
+ style="font-style:normal;font-weight:normal;line-height:0%;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none"
+ xml:space="preserve"><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:24.2109px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="70"
+ x="321.30356"
+ id="tspan29165-9-7-0"
+ sodipodi:role="line">Application</tspan></text>
+ </g>
+ </g>
+</svg>
diff --git a/doc/guides/prog_guide/index.rst b/doc/guides/prog_guide/index.rst
index 52a6d9e7aa..beaa4b8869 100644
--- a/doc/guides/prog_guide/index.rst
+++ b/doc/guides/prog_guide/index.rst
@@ -60,6 +60,7 @@ Programmer's Guide
event_ethernet_tx_adapter
event_timer_adapter
event_crypto_adapter
+ event_dma_adapter
qos_framework
power_man
packet_classif_access_ctrl
diff --git a/doc/guides/rel_notes/release_23_11.rst b/doc/guides/rel_notes/release_23_11.rst
index b34ddc0860..70f2e7e03c 100644
--- a/doc/guides/rel_notes/release_23_11.rst
+++ b/doc/guides/rel_notes/release_23_11.rst
@@ -44,7 +44,9 @@ New Features
.. This section should contain new features added in this release.
Sample format:
- * **Add a title in the past tense with a full stop.**
+ * **Added eventdev DMA adapter library.**
+
+ Added API to DMA transfer data using event mechanism.
Add a short 1-2 sentence description in the past tense.
The description should be enough to allow someone scanning
diff --git a/lib/eventdev/eventdev_pmd.h b/lib/eventdev/eventdev_pmd.h
index f62f42e140..8e5fd420f6 100644
--- a/lib/eventdev/eventdev_pmd.h
+++ b/lib/eventdev/eventdev_pmd.h
@@ -178,8 +178,12 @@ struct rte_eventdev {
event_tx_adapter_enqueue_t txa_enqueue;
/**< Pointer to PMD eth Tx adapter enqueue function. */
event_crypto_adapter_enqueue_t ca_enqueue;
+ /**< Pointer to PMD crypto adapter enqueue function. */
- uint64_t reserved_64s[4]; /**< Reserved for future fields */
+ event_dma_adapter_enqueue_t dma_enqueue;
+ /**< Pointer to PMD DMA adapter enqueue function. */
+
+ uint64_t reserved_64s[3]; /**< Reserved for future fields */
void *reserved_ptrs[3]; /**< Reserved for future fields */
} __rte_cache_aligned;
@@ -1320,6 +1324,160 @@ typedef int (*eventdev_eth_tx_adapter_queue_stop)
#define eventdev_stop_flush_t rte_eventdev_stop_flush_t
+/**
+ * Retrieve the event device's DMA adapter capabilities for the
+ * specified DMA device
+ *
+ * @param dev
+ * Event device pointer
+ *
+ * @param dma_dev_id
+ * DMA device identification
+ *
+ * @param[out] caps
+ * A pointer to memory filled with event adapter capabilities.
+ * It is expected to be pre-allocated & initialized by caller.
+ *
+ * @return
+ * - 0: Success, driver provides event adapter capabilities for the
+ * DMADEV.
+ * - <0: Error code returned by the driver function.
+ *
+ */
+typedef int (*eventdev_dma_adapter_caps_get_t)(const struct rte_eventdev *dev,
+ const int16_t dma_dev_id, uint32_t *caps);
+
+/**
+ * This API may change without prior notice
+ *
+ * Add DMA vchan queue to event device. This callback is invoked if
+ * the caps returned from rte_event_dma_adapter_caps_get(, dmadev_id)
+ * has RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_* set.
+ *
+ * @param dev
+ * Event device pointer
+ *
+ * @param dma_dev_id
+ * DMADEV identification
+ *
+ * @param vchan_id
+ * DMADEV vchan queue identifier.
+ *
+ * @param event
+ * Event information required for binding dmadev queue pair to event queue.
+ * This structure will have a valid value for only those HW PMDs supporting
+ * @see RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_VCHAN_EV_BIND capability.
+ *
+ * @return
+ * - 0: Success, dmadev vchan added successfully.
+ * - <0: Error code returned by the driver function.
+ *
+ */
+typedef int (*eventdev_dma_adapter_vchan_add_t)(const struct rte_eventdev *dev,
+ const int16_t dma_dev_id,
+ uint16_t vchan_id,
+ const struct rte_event *event);
+
+/**
+ * This API may change without prior notice
+ *
+ * Delete DMA vhcan to event device. This callback is invoked if
+ * the caps returned from rte_event_dma_adapter_caps_get(, dmadev_id)
+ * has RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_* set.
+ *
+ * @param dev
+ * Event device pointer
+ *
+ * @param dma_dev_id
+ * DMADEV identification
+ *
+ * @param vchan_id
+ * dmadev vchan identifier.
+ *
+ * @return
+ * - 0: Success, dmadev vchan deleted successfully.
+ * - <0: Error code returned by the driver function.
+ *
+ */
+typedef int (*eventdev_dma_adapter_vchan_del_t)(const struct rte_eventdev *dev,
+ const int16_t dma_dev_id,
+ uint16_t vchan_id);
+
+/**
+ * Start DMA adapter. This callback is invoked if
+ * the caps returned from rte_event_dma_adapter_caps_get(.., dmadev_id)
+ * has RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_* set and vchan for dmadev_id
+ * have been added to the event device.
+ *
+ * @param dev
+ * Event device pointer
+ *
+ * @param dma_dev_id
+ * DMA device identification
+ *
+ * @return
+ * - 0: Success, DMA adapter started successfully.
+ * - <0: Error code returned by the driver function.
+ */
+typedef int (*eventdev_dma_adapter_start_t)(const struct rte_eventdev *dev,
+ const int16_t dma_dev_id);
+
+/**
+ * Stop DMA adapter. This callback is invoked if
+ * the caps returned from rte_event_dma_adapter_caps_get(.., dmadev_id)
+ * has RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_* set and vchan for dmadev_id
+ * have been added to the event device.
+ *
+ * @param dev
+ * Event device pointer
+ *
+ * @param dma_dev_id
+ * DMA device identification
+ *
+ * @return
+ * - 0: Success, DMA adapter stopped successfully.
+ * - <0: Error code returned by the driver function.
+ */
+typedef int (*eventdev_dma_adapter_stop_t)(const struct rte_eventdev *dev,
+ const int16_t dma_dev_id);
+
+struct rte_event_dma_adapter_stats;
+
+/**
+ * Retrieve DMA adapter statistics.
+ *
+ * @param dev
+ * Event device pointer
+ *
+ * @param dma_dev_id
+ * DMA device identification
+ *
+ * @param[out] stats
+ * Pointer to stats structure
+ *
+ * @return
+ * Return 0 on success.
+ */
+typedef int (*eventdev_dma_adapter_stats_get)(const struct rte_eventdev *dev,
+ const int16_t dma_dev_id,
+ struct rte_event_dma_adapter_stats *stats);
+
+/**
+ * Reset DMA adapter statistics.
+ *
+ * @param dev
+ * Event device pointer
+ *
+ * @param dma_dev_id
+ * DMA device identification
+ *
+ * @return
+ * Return 0 on success.
+ */
+typedef int (*eventdev_dma_adapter_stats_reset)(const struct rte_eventdev *dev,
+ const int16_t dma_dev_id);
+
+
/** Event device operations function pointer table */
struct eventdev_ops {
eventdev_info_get_t dev_infos_get; /**< Get device info. */
@@ -1440,6 +1598,21 @@ struct eventdev_ops {
eventdev_eth_tx_adapter_queue_stop eth_tx_adapter_queue_stop;
/**< Stop Tx queue assigned to Tx adapter instance */
+ eventdev_dma_adapter_caps_get_t dma_adapter_caps_get;
+ /**< Get DMA adapter capabilities */
+ eventdev_dma_adapter_vchan_add_t dma_adapter_vchan_add;
+ /**< Add vchan queue to DMA adapter */
+ eventdev_dma_adapter_vchan_del_t dma_adapter_vchan_del;
+ /**< Delete vchan queue from DMA adapter */
+ eventdev_dma_adapter_start_t dma_adapter_start;
+ /**< Start DMA adapter */
+ eventdev_dma_adapter_stop_t dma_adapter_stop;
+ /**< Stop DMA adapter */
+ eventdev_dma_adapter_stats_get dma_adapter_stats_get;
+ /**< Get DMA stats */
+ eventdev_dma_adapter_stats_reset dma_adapter_stats_reset;
+ /**< Reset DMA stats */
+
eventdev_selftest dev_selftest;
/**< Start eventdev Selftest */
diff --git a/lib/eventdev/eventdev_private.c b/lib/eventdev/eventdev_private.c
index 1d3d9d357e..18ed8bf3c8 100644
--- a/lib/eventdev/eventdev_private.c
+++ b/lib/eventdev/eventdev_private.c
@@ -81,6 +81,14 @@ dummy_event_crypto_adapter_enqueue(__rte_unused void *port,
return 0;
}
+static uint16_t
+dummy_event_dma_adapter_enqueue(__rte_unused void *port, __rte_unused struct rte_event ev[],
+ __rte_unused uint16_t nb_events)
+{
+ RTE_EDEV_LOG_ERR("event DMA adapter enqueue requested for unconfigured event device");
+ return 0;
+}
+
void
event_dev_fp_ops_reset(struct rte_event_fp_ops *fp_op)
{
@@ -97,6 +105,7 @@ event_dev_fp_ops_reset(struct rte_event_fp_ops *fp_op)
.txa_enqueue_same_dest =
dummy_event_tx_adapter_enqueue_same_dest,
.ca_enqueue = dummy_event_crypto_adapter_enqueue,
+ .dma_enqueue = dummy_event_dma_adapter_enqueue,
.data = dummy_data,
};
@@ -117,5 +126,6 @@ event_dev_fp_ops_set(struct rte_event_fp_ops *fp_op,
fp_op->txa_enqueue = dev->txa_enqueue;
fp_op->txa_enqueue_same_dest = dev->txa_enqueue_same_dest;
fp_op->ca_enqueue = dev->ca_enqueue;
+ fp_op->dma_enqueue = dev->dma_enqueue;
fp_op->data = dev->data->ports;
}
diff --git a/lib/eventdev/meson.build b/lib/eventdev/meson.build
index 6edf98dfa5..21347f7c4c 100644
--- a/lib/eventdev/meson.build
+++ b/lib/eventdev/meson.build
@@ -25,6 +25,7 @@ sources = files(
)
headers = files(
'rte_event_crypto_adapter.h',
+ 'rte_event_dma_adapter.h',
'rte_event_eth_rx_adapter.h',
'rte_event_eth_tx_adapter.h',
'rte_event_ring.h',
diff --git a/lib/eventdev/rte_event_dma_adapter.h b/lib/eventdev/rte_event_dma_adapter.h
new file mode 100644
index 0000000000..d413d31a61
--- /dev/null
+++ b/lib/eventdev/rte_event_dma_adapter.h
@@ -0,0 +1,582 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright (c) 2023 Marvell.
+ */
+
+#ifndef RTE_EVENT_DMA_ADAPTER
+#define RTE_EVENT_DMA_ADAPTER
+
+/**
+ * @file rte_event_dma_adapter.h
+ *
+ * @warning
+ * @b EXPERIMENTAL:
+ * All functions in this file may be changed or removed without prior notice.
+ *
+ * DMA Event Adapter API.
+ *
+ * Eventdev library provides adapters to bridge between various components for providing new
+ * event source. The event DMA adapter is one of those adapters which is intended to bridge
+ * between event devices and DMA devices.
+ *
+ * The DMA adapter adds support to enqueue / dequeue DMA operations to / from event device. The
+ * packet flow between DMA device and the event device can be accomplished using both SW and HW
+ * based transfer mechanisms. The adapter uses an EAL service core function for SW based packet
+ * transfer and uses the eventdev PMD functions to configure HW based packet transfer between the
+ * DMA device and the event device.
+ *
+ * The application can choose to submit a DMA operation directly to an DMA device or send it to the
+ * DMA adapter via eventdev based on RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP_FWD capability. The
+ * first mode is known as the event new (RTE_EVENT_DMA_ADAPTER_OP_NEW) mode and the second as the
+ * event forward (RTE_EVENT_DMA_ADAPTER_OP_FORWARD) mode. The choice of mode can be specified while
+ * creating the adapter. In the former mode, it is an application responsibility to enable ingress
+ * packet ordering. In the latter mode, it is the adapter responsibility to enable the ingress
+ * packet ordering.
+ *
+ *
+ * Working model of RTE_EVENT_DMA_ADAPTER_OP_NEW mode:
+ *
+ * +--------------+ +--------------+
+ * | | | DMA stage |
+ * | Application |---[2]-->| + enqueue to |
+ * | | | dmadev |
+ * +--------------+ +--------------+
+ * ^ ^ |
+ * | | [3]
+ * [6] [1] |
+ * | | |
+ * +--------------+ |
+ * | | |
+ * | Event device | |
+ * | | |
+ * +--------------+ |
+ * ^ |
+ * | |
+ * [5] |
+ * | v
+ * +--------------+ +--------------+
+ * | | | |
+ * | DMA adapter |<--[4]---| dmadev |
+ * | | | |
+ * +--------------+ +--------------+
+ *
+ *
+ * [1] Application dequeues events from the previous stage.
+ * [2] Application prepares the DMA operations.
+ * [3] DMA operations are submitted to dmadev by application.
+ * [4] DMA adapter dequeues DMA completions from dmadev.
+ * [5] DMA adapter enqueues events to the eventdev.
+ * [6] Application dequeues from eventdev for further processing.
+ *
+ * In the RTE_EVENT_DMA_ADAPTER_OP_NEW mode, application submits DMA operations directly to DMA
+ * device. The DMA adapter then dequeues DMA completions from DMA device and enqueue events to the
+ * event device. This mode does not ensure ingress ordering, if the application directly enqueues
+ * to dmadev without going through DMA / atomic stage i.e. removing item [1] and [2].
+ *
+ * Events dequeued from the adapter will be treated as new events. In this mode, application needs
+ * to specify event information (response information) which is needed to enqueue an event after the
+ * DMA operation is completed.
+ *
+ *
+ * Working model of RTE_EVENT_DMA_ADAPTER_OP_FORWARD mode:
+ *
+ * +--------------+ +--------------+
+ * --[1]-->| |---[2]-->| Application |
+ * | Event device | | in |
+ * <--[8]--| |<--[3]---| Ordered stage|
+ * +--------------+ +--------------+
+ * ^ |
+ * | [4]
+ * [7] |
+ * | v
+ * +----------------+ +--------------+
+ * | |--[5]->| |
+ * | DMA adapter | | dmadev |
+ * | |<-[6]--| |
+ * +----------------+ +--------------+
+ *
+ *
+ * [1] Events from the previous stage.
+ * [2] Application in ordered stage dequeues events from eventdev.
+ * [3] Application enqueues DMA operations as events to eventdev.
+ * [4] DMA adapter dequeues event from eventdev.
+ * [5] DMA adapter submits DMA operations to dmadev (Atomic stage).
+ * [6] DMA adapter dequeues DMA completions from dmadev
+ * [7] DMA adapter enqueues events to the eventdev
+ * [8] Events to the next stage
+ *
+ * In the event forward (RTE_EVENT_DMA_ADAPTER_OP_FORWARD) mode, if the HW supports the capability
+ * RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP_FWD, application can directly submit the DMA
+ * operations to the dmadev. If not, application retrieves the event port of the DMA adapter
+ * through the API, rte_event_DMA_adapter_event_port_get(). Then, links its event queue to this
+ * port and starts enqueuing DMA operations as events to the eventdev. The adapter then dequeues
+ * the events and submits the DMA operations to the dmadev. After the DMA completions, the adapter
+ * enqueues events to the event device.
+ *
+ * Application can use this mode, when ingress packet ordering is needed. Events dequeued from the
+ * adapter will be treated as forwarded events. In this mode, the application needs to specify the
+ * dmadev ID and queue pair ID (request information) needed to enqueue an DMA operation in addition
+ * to the event information (response information) needed to enqueue an event after the DMA
+ * operation has completed.
+ *
+ * The event DMA adapter provides common APIs to configure the packet flow from the DMA device to
+ * event devices for both SW and HW based transfers. The DMA event adapter's functions are:
+ *
+ * - rte_event_dma_adapter_create_ext()
+ * - rte_event_dma_adapter_create()
+ * - rte_event_dma_adapter_free()
+ * - rte_event_dma_adapter_vchan_add()
+ * - rte_event_dma_adapter_vchan_del()
+ * - rte_event_dma_adapter_start()
+ * - rte_event_dma_adapter_stop()
+ * - rte_event_dma_adapter_stats_get()
+ * - rte_event_dma_adapter_stats_reset()
+ *
+ * The application creates an instance using rte_event_dma_adapter_create() or
+ * rte_event_dma_adapter_create_ext().
+ *
+ * dmadev queue pair addition / deletion is done using the rte_event_dma_adapter_vchan_add() /
+ * rte_event_dma_adapter_vchan_del() APIs. If HW supports the capability
+ * RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_VCHAN_EV_BIND, event information must be passed to the
+ * add API.
+ *
+ */
+
+#include <stdint.h>
+
+#include <rte_eventdev.h>
+#include <rte_common.h>
+#include <rte_dmadev_pmd.h>
+
+#ifdef __cplusplus
+extern "C" {
+#endif
+
+/**
+ * A structure used to hold event based DMA operation entry. All the information
+ * required for a DMA transfer shall be populated in "struct rte_event_dma_adapter_op"
+ * instance.
+ */
+struct rte_event_dma_adapter_op {
+ struct rte_dma_sge *src_seg;
+ /**< Source segments. */
+ struct rte_dma_sge *dst_seg;
+ /**< Destination segments. */
+ uint16_t nb_src;
+ /**< Number of source segments. */
+ uint16_t nb_dst;
+ /**< Number of destination segments. */
+ uint64_t flags;
+ /**< Flags related to the operation.
+ * @see RTE_DMA_OP_FLAG_*
+ */
+ int16_t dma_dev_id;
+ /**< DMA device ID to be used */
+ uint16_t vchan;
+ /**< DMA vchan ID to be used */
+ struct rte_mempool *op_mp;
+ /**< Mempool from which op is allocated. */
+};
+
+/**
+ * DMA event adapter mode
+ */
+enum rte_event_dma_adapter_mode {
+ RTE_EVENT_DMA_ADAPTER_OP_NEW,
+ /**< Start the DMA adapter in event new mode.
+ * @see RTE_EVENT_OP_NEW.
+ *
+ * Application submits DMA operations to the dmadev. Adapter only dequeues the DMA
+ * completions from dmadev and enqueue events to the eventdev.
+ */
+
+ RTE_EVENT_DMA_ADAPTER_OP_FORWARD,
+ /**< Start the DMA adapter in event forward mode.
+ * @see RTE_EVENT_OP_FORWARD.
+ *
+ * Application submits DMA requests as events to the DMA adapter or DMA device based on
+ * RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP_FWD capability. DMA completions are enqueued
+ * back to the eventdev by DMA adapter.
+ */
+};
+
+/**
+ * Adapter configuration structure that the adapter configuration callback function is expected to
+ * fill out.
+ *
+ * @see rte_event_dma_adapter_conf_cb
+ */
+struct rte_event_dma_adapter_conf {
+ uint8_t event_port_id;
+ /** < Event port identifier, the adapter enqueues events to this port and dequeues DMA
+ * request events in RTE_EVENT_DMA_ADAPTER_OP_FORWARD mode.
+ */
+
+ uint32_t max_nb;
+ /**< The adapter can return early if it has processed at least max_nb DMA ops. This isn't
+ * treated as a requirement; batching may cause the adapter to process more than max_nb DMA
+ * ops.
+ */
+};
+
+/**
+ * Adapter runtime configuration parameters
+ */
+struct rte_event_dma_adapter_runtime_params {
+ uint32_t max_nb;
+ /**< The adapter can return early if it has processed at least max_nb DMA ops. This isn't
+ * treated as a requirement; batching may cause the adapter to process more than max_nb DMA
+ * ops.
+ *
+ * Callback function passed to rte_event_dma_adapter_create_ext() configures the adapter
+ * with default value of max_nb.
+ * rte_event_dma_adapter_runtime_params_set() allows to re-configure max_nb during runtime
+ * (after adding at least one queue pair)
+ *
+ * This is valid for the devices without RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP_FWD or
+ * RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP_NEW capability.
+ */
+
+ uint32_t rsvd[15];
+ /**< Reserved fields for future expansion */
+};
+
+/**
+ * Function type used for adapter configuration callback. The callback is used to fill in members of
+ * the struct rte_event_dma_adapter_conf, this callback is invoked when creating a SW service for
+ * packet transfer from dmadev vchan to the event device. The SW service is created within the
+ * function, rte_event_dma_adapter_vchan_add(), if SW based packet transfers from dmadev vchan
+ * to the event device are required.
+ *
+ * @param id
+ * Adapter identifier.
+ * @param evdev_id
+ * Event device identifier.
+ * @param conf
+ * Structure that needs to be populated by this callback.
+ * @param arg
+ * Argument to the callback. This is the same as the conf_arg passed to the
+ * rte_event_dma_adapter_create_ext().
+ */
+typedef int (*rte_event_dma_adapter_conf_cb)(uint8_t id, uint8_t evdev_id,
+ struct rte_event_dma_adapter_conf *conf, void *arg);
+
+/**
+ * A structure used to retrieve statistics for an event DMA adapter instance.
+ */
+struct rte_event_dma_adapter_stats {
+ uint64_t event_poll_count;
+ /**< Event port poll count */
+
+ uint64_t event_deq_count;
+ /**< Event dequeue count */
+
+ uint64_t dma_enq_count;
+ /**< dmadev enqueue count */
+
+ uint64_t dma_enq_fail_count;
+ /**< dmadev enqueue failed count */
+
+ uint64_t dma_deq_count;
+ /**< dmadev dequeue count */
+
+ uint64_t event_enq_count;
+ /**< Event enqueue count */
+
+ uint64_t event_enq_retry_count;
+ /**< Event enqueue retry count */
+
+ uint64_t event_enq_fail_count;
+ /**< Event enqueue fail count */
+};
+
+/**
+ * Create a new event DMA adapter with the specified identifier.
+ *
+ * @param id
+ * Adapter identifier.
+ * @param evdev_id
+ * Event device identifier.
+ * @param conf_cb
+ * Callback function that fills in members of a struct rte_event_dma_adapter_conf struct passed
+ * into it.
+ * @param mode
+ * Flag to indicate the mode of the adapter.
+ * @see rte_event_dma_adapter_mode
+ * @param conf_arg
+ * Argument that is passed to the conf_cb function.
+ *
+ * @return
+ * - 0: Success
+ * - <0: Error code on failure
+ */
+__rte_experimental
+int rte_event_dma_adapter_create_ext(uint8_t id, uint8_t evdev_id,
+ rte_event_dma_adapter_conf_cb conf_cb,
+ enum rte_event_dma_adapter_mode mode, void *conf_arg);
+
+/**
+ * Create a new event DMA adapter with the specified identifier. This function uses an internal
+ * configuration function that creates an event port. This default function reconfigures the event
+ * device with an additional event port and set up the event port using the port_config parameter
+ * passed into this function. In case the application needs more control in configuration of the
+ * service, it should use the rte_event_dma_adapter_create_ext() version.
+ *
+ * @param id
+ * Adapter identifier.
+ * @param evdev_id
+ * Event device identifier.
+ * @param port_config
+ * Argument of type *rte_event_port_conf* that is passed to the conf_cb function.
+ * @param mode
+ * Flag to indicate the mode of the adapter.
+ * @see rte_event_dma_adapter_mode
+ *
+ * @return
+ * - 0: Success
+ * - <0: Error code on failure
+ */
+__rte_experimental
+int rte_event_dma_adapter_create(uint8_t id, uint8_t evdev_id,
+ struct rte_event_port_conf *port_config,
+ enum rte_event_dma_adapter_mode mode);
+
+/**
+ * Free an event DMA adapter
+ *
+ * @param id
+ * Adapter identifier.
+ * @return
+ * - 0: Success
+ * - <0: Error code on failure, If the adapter still has queue pairs added to it, the function
+ * returns -EBUSY.
+ */
+__rte_experimental
+int rte_event_dma_adapter_free(uint8_t id);
+
+/**
+ * Retrieve the event port of an adapter.
+ *
+ * @param id
+ * Adapter identifier.
+ *
+ * @param [out] event_port_id
+ * Application links its event queue to this adapter port which is used in
+ * RTE_EVENT_DMA_ADAPTER_OP_FORWARD mode.
+ *
+ * @return
+ * - 0: Success
+ * - <0: Error code on failure.
+ */
+__rte_experimental
+int rte_event_dma_adapter_event_port_get(uint8_t id, uint8_t *event_port_id);
+
+/**
+ * Add a vchan to an event DMA adapter.
+ *
+ * @param id
+ * Adapter identifier.
+ * @param dmadev_id
+ * dmadev identifier.
+ * @param vchan
+ * DMA device vchan identifier. If vchan is set -1, adapter adds all the
+ * preconfigured vchan to the instance.
+ * @param event
+ * If HW supports dmadev vchan to event queue binding, application is expected to fill in
+ * event information, else it will be NULL.
+ * @see RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_VCHAN_EV_BIND
+ *
+ * @return
+ * - 0: Success, vchan added correctly.
+ * - <0: Error code on failure.
+ */
+__rte_experimental
+int rte_event_dma_adapter_vchan_add(uint8_t id, int16_t dmadev_id, uint16_t vchan,
+ const struct rte_event *event);
+
+/**
+ * Delete a vchan from an event DMA adapter.
+ *
+ * @param id
+ * Adapter identifier.
+ * @param dmadev_id
+ * DMA device identifier.
+ * @param vchan
+ * DMA device vchan identifier.
+ *
+ * @return
+ * - 0: Success, vchan deleted successfully.
+ * - <0: Error code on failure.
+ */
+__rte_experimental
+int rte_event_dma_adapter_vchan_del(uint8_t id, int16_t dmadev_id, uint16_t vchan);
+
+/**
+ * Retrieve the service ID of an adapter. If the adapter doesn't use a rte_service function, this
+ * function returns -ESRCH.
+ *
+ * @param id
+ * Adapter identifier.
+ * @param [out] service_id
+ * A pointer to a uint32_t, to be filled in with the service id.
+ *
+ * @return
+ * - 0: Success
+ * - <0: Error code on failure, if the adapter doesn't use a rte_service function, this function
+ * returns -ESRCH.
+ */
+__rte_experimental
+int rte_event_dma_adapter_service_id_get(uint8_t id, uint32_t *service_id);
+
+/**
+ * Start event DMA adapter
+ *
+ * @param id
+ * Adapter identifier.
+ *
+ * @return
+ * - 0: Success, adapter started successfully.
+ * - <0: Error code on failure.
+ *
+ * @note The eventdev and dmadev to which the event_dma_adapter is connected should be started
+ * before calling rte_event_dma_adapter_start().
+ */
+__rte_experimental
+int rte_event_dma_adapter_start(uint8_t id);
+
+/**
+ * Stop event DMA adapter
+ *
+ * @param id
+ * Adapter identifier.
+ *
+ * @return
+ * - 0: Success, adapter stopped successfully.
+ * - <0: Error code on failure.
+ */
+__rte_experimental
+int rte_event_dma_adapter_stop(uint8_t id);
+
+/**
+ * Initialize the adapter runtime configuration parameters
+ *
+ * @param params
+ * A pointer to structure of type struct rte_event_dma_adapter_runtime_params
+ *
+ * @return
+ * - 0: Success
+ * - <0: Error code on failure
+ */
+__rte_experimental
+int rte_event_dma_adapter_runtime_params_init(struct rte_event_dma_adapter_runtime_params *params);
+
+/**
+ * Set the adapter runtime configuration parameters
+ *
+ * @param id
+ * Adapter identifier
+ *
+ * @param params
+ * A pointer to structure of type struct rte_event_dma_adapter_runtime_params with configuration
+ * parameter values. The reserved fields of this structure must be initialized to zero and the valid
+ * fields need to be set appropriately. This struct can be initialized using
+ * rte_event_dma_adapter_runtime_params_init() API to default values or application may reset this
+ * struct and update required fields.
+ *
+ * @return
+ * - 0: Success
+ * - <0: Error code on failure
+ */
+__rte_experimental
+int rte_event_dma_adapter_runtime_params_set(uint8_t id,
+ struct rte_event_dma_adapter_runtime_params *params);
+
+/**
+ * Get the adapter runtime configuration parameters
+ *
+ * @param id
+ * Adapter identifier
+ *
+ * @param[out] params
+ * A pointer to structure of type struct rte_event_dma_adapter_runtime_params containing valid
+ * adapter parameters when return value is 0.
+ *
+ * @return
+ * - 0: Success
+ * - <0: Error code on failure
+ */
+__rte_experimental
+int rte_event_dma_adapter_runtime_params_get(uint8_t id,
+ struct rte_event_dma_adapter_runtime_params *params);
+
+/**
+ * Retrieve statistics for an adapter
+ *
+ * @param id
+ * Adapter identifier.
+ * @param [out] stats
+ * A pointer to structure used to retrieve statistics for an adapter.
+ *
+ * @return
+ * - 0: Success, retrieved successfully.
+ * - <0: Error code on failure.
+ */
+__rte_experimental
+int rte_event_dma_adapter_stats_get(uint8_t id, struct rte_event_dma_adapter_stats *stats);
+
+/**
+ * Reset statistics for an adapter.
+ *
+ * @param id
+ * Adapter identifier.
+ *
+ * @return
+ * - 0: Success, statistics reset successfully.
+ * - <0: Error code on failure.
+ */
+__rte_experimental
+int rte_event_dma_adapter_stats_reset(uint8_t id);
+
+/**
+ * Enqueue a burst of DMA operations as event objects supplied in *rte_event* structure on an event
+ * DMA adapter designated by its event *evdev_id* through the event port specified by *port_id*.
+ * This function is supported if the eventdev PMD has the
+ * #RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP_FWD capability flag set.
+ *
+ * The *nb_events* parameter is the number of event objects to enqueue that are supplied in the
+ * *ev* array of *rte_event* structure.
+ *
+ * The rte_event_dma_adapter_enqueue() function returns the number of event objects it actually
+ * enqueued. A return value equal to *nb_events* means that all event objects have been enqueued.
+ *
+ * @param evdev_id
+ * The identifier of the device.
+ * @param port_id
+ * The identifier of the event port.
+ * @param ev
+ * Points to an array of *nb_events* objects of type *rte_event* structure which contain the
+ * event object enqueue operations to be processed.
+ * @param nb_events
+ * The number of event objects to enqueue, typically number of
+ * rte_event_port_attr_get(...RTE_EVENT_PORT_ATTR_ENQ_DEPTH...) available for this port.
+ *
+ * @return
+ * The number of event objects actually enqueued on the event device. The return value can be
+ * less than the value of the *nb_events* parameter when the event devices queue is full or if
+ * invalid parameters are specified in a *rte_event*. If the return value is less than *nb_events*,
+ * the remaining events at the end of ev[] are not consumed and the caller has to take care of them,
+ * and rte_errno is set accordingly. Possible errno values include:
+ *
+ * - EINVAL: The port ID is invalid, device ID is invalid, an event's queue ID is invalid, or an
+ * event's sched type doesn't match the capabilities of the destination queue.
+ * - ENOSPC: The event port was backpressured and unable to enqueue one or more events. This
+ * error code is only applicable to closed systems.
+ */
+__rte_experimental
+uint16_t rte_event_dma_adapter_enqueue(uint8_t evdev_id, uint8_t port_id, struct rte_event ev[],
+ uint16_t nb_events);
+
+#ifdef __cplusplus
+}
+#endif
+
+#endif /* RTE_EVENT_DMA_ADAPTER */
diff --git a/lib/eventdev/rte_eventdev.h b/lib/eventdev/rte_eventdev.h
index 2ba8a7b090..026f7c367d 100644
--- a/lib/eventdev/rte_eventdev.h
+++ b/lib/eventdev/rte_eventdev.h
@@ -1197,6 +1197,8 @@ struct rte_event_vector {
*/
#define RTE_EVENT_TYPE_ETH_RX_ADAPTER 0x4
/**< The event generated from event eth Rx adapter */
+#define RTE_EVENT_TYPE_DMADEV 0x5
+/**< The event generated from dma subsystem */
#define RTE_EVENT_TYPE_VECTOR 0x8
/**< Indicates that event is a vector.
* All vector event types should be a logical OR of EVENT_TYPE_VECTOR.
@@ -1462,6 +1464,47 @@ int
rte_event_crypto_adapter_caps_get(uint8_t dev_id, uint8_t cdev_id,
uint32_t *caps);
+/* DMA adapter capability bitmap flag */
+#define RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP_NEW 0x1
+/**< Flag indicates HW is capable of generating events in
+ * RTE_EVENT_OP_NEW enqueue operation. DMADEV will send
+ * packets to the event device as new events using an
+ * internal event port.
+ */
+
+#define RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP_FWD 0x2
+/**< Flag indicates HW is capable of generating events in
+ * RTE_EVENT_OP_FORWARD enqueue operation. DMADEV will send
+ * packets to the event device as forwarded event using an
+ * internal event port.
+ */
+
+#define RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_VCHAN_EV_BIND 0x4
+/**< Flag indicates HW is capable of mapping DMA vchan to event queue. */
+
+/**
+ * Retrieve the event device's DMA adapter capabilities for the
+ * specified dmadev device
+ *
+ * @param dev_id
+ * The identifier of the device.
+ *
+ * @param dmadev_id
+ * The identifier of the dmadev device.
+ *
+ * @param[out] caps
+ * A pointer to memory filled with event adapter capabilities.
+ * It is expected to be pre-allocated & initialized by caller.
+ *
+ * @return
+ * - 0: Success, driver provides event adapter capabilities for the
+ * dmadev device.
+ * - <0: Error code returned by the driver function.
+ *
+ */
+int
+rte_event_dma_adapter_caps_get(uint8_t dev_id, int16_t dmadev_id, uint32_t *caps);
+
/* Ethdev Tx adapter capability bitmap flags */
#define RTE_EVENT_ETH_TX_ADAPTER_CAP_INTERNAL_PORT 0x1
/**< This flag is sent when the PMD supports a packet transmit callback
diff --git a/lib/eventdev/rte_eventdev_core.h b/lib/eventdev/rte_eventdev_core.h
index c27a52ccc0..83e8736c71 100644
--- a/lib/eventdev/rte_eventdev_core.h
+++ b/lib/eventdev/rte_eventdev_core.h
@@ -42,6 +42,10 @@ typedef uint16_t (*event_crypto_adapter_enqueue_t)(void *port,
uint16_t nb_events);
/**< @internal Enqueue burst of events on crypto adapter */
+typedef uint16_t (*event_dma_adapter_enqueue_t)(void *port, struct rte_event ev[],
+ uint16_t nb_events);
+/**< @internal Enqueue burst of events on DMA adapter */
+
struct rte_event_fp_ops {
void **data;
/**< points to array of internal port data pointers */
@@ -65,7 +69,9 @@ struct rte_event_fp_ops {
/**< PMD Tx adapter enqueue same destination function. */
event_crypto_adapter_enqueue_t ca_enqueue;
/**< PMD Crypto adapter enqueue function. */
- uintptr_t reserved[5];
+ event_dma_adapter_enqueue_t dma_enqueue;
+ /**< PMD DMA adapter enqueue function. */
+ uintptr_t reserved[4];
} __rte_cache_aligned;
extern struct rte_event_fp_ops rte_event_fp_ops[RTE_EVENT_MAX_DEVS];
diff --git a/lib/eventdev/version.map b/lib/eventdev/version.map
index 7ce09a87bb..dc1dcc8274 100644
--- a/lib/eventdev/version.map
+++ b/lib/eventdev/version.map
@@ -133,6 +133,21 @@ EXPERIMENTAL {
rte_event_timer_remaining_ticks_get;
# added in 23.11
+ rte_event_dma_adapter_create_ext;
+ rte_event_dma_adapter_create;
+ rte_event_dma_adapter_event_port_get;
+ rte_event_dma_adapter_enqueue;
+ rte_event_dma_adapter_free;
+ rte_event_dma_adapter_runtime_params_init;
+ rte_event_dma_adapter_runtime_params_set;
+ rte_event_dma_adapter_runtime_params_get;
+ rte_event_dma_adapter_service_id_get;
+ rte_event_dma_adapter_start;
+ rte_event_dma_adapter_stop;
+ rte_event_dma_adapter_stats_get;
+ rte_event_dma_adapter_stats_reset;
+ rte_event_dma_adapter_vchan_add;
+ rte_event_dma_adapter_vchan_del;
rte_event_eth_rx_adapter_create_ext_with_params;
};
--
2.25.1
^ permalink raw reply [flat|nested] 110+ messages in thread
* [PATCH v3 02/12] eventdev: api to get DMA adapter capabilities
2023-09-23 13:34 ` [PATCH v3 00/12] event DMA adapter library support Amit Prakash Shukla
2023-09-23 13:34 ` [PATCH v3 01/12] eventdev: introduce event DMA adapter library Amit Prakash Shukla
@ 2023-09-23 13:34 ` Amit Prakash Shukla
2023-09-23 13:34 ` [PATCH v3 03/12] eventdev: create and free API for DMA adapter Amit Prakash Shukla
` (10 subsequent siblings)
12 siblings, 0 replies; 110+ messages in thread
From: Amit Prakash Shukla @ 2023-09-23 13:34 UTC (permalink / raw)
To: Jerin Jacob
Cc: dev, fengchengwen, kevin.laatz, bruce.richardson, conor.walsh,
vattunuru, g.singh, sachin.saxena, hemant.agrawal, cheng1.jiang,
ndabilpuram, anoobj, mb, Amit Prakash Shukla
Added a new eventdev API rte_event_dma_adapter_caps_get(), to get
DMA adapter capabilities supported by the driver.
Signed-off-by: Amit Prakash Shukla <amitprakashs@marvell.com>
---
lib/eventdev/meson.build | 2 +-
lib/eventdev/rte_eventdev.c | 23 +++++++++++++++++++++++
lib/eventdev/rte_eventdev.h | 2 +-
lib/meson.build | 2 +-
4 files changed, 26 insertions(+), 3 deletions(-)
diff --git a/lib/eventdev/meson.build b/lib/eventdev/meson.build
index 21347f7c4c..b46bbbc9aa 100644
--- a/lib/eventdev/meson.build
+++ b/lib/eventdev/meson.build
@@ -43,5 +43,5 @@ driver_sdk_headers += files(
'event_timer_adapter_pmd.h',
)
-deps += ['ring', 'ethdev', 'hash', 'mempool', 'mbuf', 'timer', 'cryptodev']
+deps += ['ring', 'ethdev', 'hash', 'mempool', 'mbuf', 'timer', 'cryptodev', 'dmadev']
deps += ['telemetry']
diff --git a/lib/eventdev/rte_eventdev.c b/lib/eventdev/rte_eventdev.c
index 6ab4524332..60509c6efb 100644
--- a/lib/eventdev/rte_eventdev.c
+++ b/lib/eventdev/rte_eventdev.c
@@ -20,6 +20,7 @@
#include <rte_errno.h>
#include <ethdev_driver.h>
#include <rte_cryptodev.h>
+#include <rte_dmadev.h>
#include <cryptodev_pmd.h>
#include <rte_telemetry.h>
@@ -224,6 +225,28 @@ rte_event_eth_tx_adapter_caps_get(uint8_t dev_id, uint16_t eth_port_id,
: 0;
}
+int
+rte_event_dma_adapter_caps_get(uint8_t dev_id, uint8_t dma_dev_id, uint32_t *caps)
+{
+ struct rte_eventdev *dev;
+
+ RTE_EVENTDEV_VALID_DEVID_OR_ERR_RET(dev_id, -EINVAL);
+ if (!rte_dma_is_valid(dma_dev_id))
+ return -EINVAL;
+
+ dev = &rte_eventdevs[dev_id];
+
+ if (caps == NULL)
+ return -EINVAL;
+
+ *caps = 0;
+
+ if (dev->dev_ops->dma_adapter_caps_get)
+ return (*dev->dev_ops->dma_adapter_caps_get)(dev, dma_dev_id, caps);
+
+ return 0;
+}
+
static inline int
event_dev_queue_config(struct rte_eventdev *dev, uint8_t nb_queues)
{
diff --git a/lib/eventdev/rte_eventdev.h b/lib/eventdev/rte_eventdev.h
index 026f7c367d..cef90aebf6 100644
--- a/lib/eventdev/rte_eventdev.h
+++ b/lib/eventdev/rte_eventdev.h
@@ -1503,7 +1503,7 @@ rte_event_crypto_adapter_caps_get(uint8_t dev_id, uint8_t cdev_id,
*
*/
int
-rte_event_dma_adapter_caps_get(uint8_t dev_id, int16_t dmadev_id, uint32_t *caps);
+rte_event_dma_adapter_caps_get(uint8_t dev_id, uint8_t dmadev_id, uint32_t *caps);
/* Ethdev Tx adapter capability bitmap flags */
#define RTE_EVENT_ETH_TX_ADAPTER_CAP_INTERNAL_PORT 0x1
diff --git a/lib/meson.build b/lib/meson.build
index 53155be8e9..f3191f10b6 100644
--- a/lib/meson.build
+++ b/lib/meson.build
@@ -33,6 +33,7 @@ libraries = [
'compressdev',
'cryptodev',
'distributor',
+ 'dmadev',
'efd',
'eventdev',
'gpudev',
@@ -48,7 +49,6 @@ libraries = [
'rawdev',
'regexdev',
'mldev',
- 'dmadev',
'rib',
'reorder',
'sched',
--
2.25.1
^ permalink raw reply [flat|nested] 110+ messages in thread
* [PATCH v3 03/12] eventdev: create and free API for DMA adapter
2023-09-23 13:34 ` [PATCH v3 00/12] event DMA adapter library support Amit Prakash Shukla
2023-09-23 13:34 ` [PATCH v3 01/12] eventdev: introduce event DMA adapter library Amit Prakash Shukla
2023-09-23 13:34 ` [PATCH v3 02/12] eventdev: api to get DMA adapter capabilities Amit Prakash Shukla
@ 2023-09-23 13:34 ` Amit Prakash Shukla
2023-09-23 13:34 ` [PATCH v3 04/12] eventdev: add API support for vchan add and delete Amit Prakash Shukla
` (9 subsequent siblings)
12 siblings, 0 replies; 110+ messages in thread
From: Amit Prakash Shukla @ 2023-09-23 13:34 UTC (permalink / raw)
To: Bruce Richardson, Jerin Jacob, Amit Prakash Shukla
Cc: dev, fengchengwen, kevin.laatz, conor.walsh, vattunuru, g.singh,
sachin.saxena, hemant.agrawal, cheng1.jiang, ndabilpuram, anoobj,
mb
Added API support to create and free DMA adapter. Create function shall be
called with event device to be associated with the adapter and port
configuration to setup an event port.
Signed-off-by: Amit Prakash Shukla <amitprakashs@marvell.com>
---
config/rte_config.h | 1 +
lib/eventdev/meson.build | 1 +
lib/eventdev/rte_event_dma_adapter.c | 335 +++++++++++++++++++++++++++
3 files changed, 337 insertions(+)
create mode 100644 lib/eventdev/rte_event_dma_adapter.c
diff --git a/config/rte_config.h b/config/rte_config.h
index 400e44e3cf..401727703f 100644
--- a/config/rte_config.h
+++ b/config/rte_config.h
@@ -77,6 +77,7 @@
#define RTE_EVENT_ETH_INTR_RING_SIZE 1024
#define RTE_EVENT_CRYPTO_ADAPTER_MAX_INSTANCE 32
#define RTE_EVENT_ETH_TX_ADAPTER_MAX_INSTANCE 32
+#define RTE_EVENT_DMA_ADAPTER_MAX_INSTANCE 32
/* rawdev defines */
#define RTE_RAWDEV_MAX_DEVS 64
diff --git a/lib/eventdev/meson.build b/lib/eventdev/meson.build
index b46bbbc9aa..250abcb154 100644
--- a/lib/eventdev/meson.build
+++ b/lib/eventdev/meson.build
@@ -17,6 +17,7 @@ sources = files(
'eventdev_private.c',
'eventdev_trace_points.c',
'rte_event_crypto_adapter.c',
+ 'rte_event_dma_adapter.c',
'rte_event_eth_rx_adapter.c',
'rte_event_eth_tx_adapter.c',
'rte_event_ring.c',
diff --git a/lib/eventdev/rte_event_dma_adapter.c b/lib/eventdev/rte_event_dma_adapter.c
new file mode 100644
index 0000000000..c7ffba1b47
--- /dev/null
+++ b/lib/eventdev/rte_event_dma_adapter.c
@@ -0,0 +1,335 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright (c) 2023 Marvell.
+ */
+
+#include "rte_eventdev.h"
+#include "eventdev_pmd.h"
+#include "rte_event_dma_adapter.h"
+
+#define DMA_BATCH_SIZE 32
+#define DMA_DEFAULT_MAX_NB 128
+#define DMA_ADAPTER_NAME_LEN 32
+#define DMA_ADAPTER_BUFFER_SIZE 1024
+
+#define DMA_ADAPTER_OPS_BUFFER_SIZE (DMA_BATCH_SIZE + DMA_BATCH_SIZE)
+
+#define DMA_ADAPTER_ARRAY "event_dma_adapter_array"
+
+/* Macros to check for valid adapter */
+#define EVENT_DMA_ADAPTER_ID_VALID_OR_ERR_RET(id, retval) \
+ do { \
+ if (!edma_adapter_valid_id(id)) { \
+ RTE_EDEV_LOG_ERR("Invalid DMA adapter id = %d\n", id); \
+ return retval; \
+ } \
+ } while (0)
+
+/* DMA ops circular buffer */
+struct dma_ops_circular_buffer {
+ /* Index of head element */
+ uint16_t head;
+
+ /* Index of tail element */
+ uint16_t tail;
+
+ /* Number of elements in buffer */
+ uint16_t count;
+
+ /* Size of circular buffer */
+ uint16_t size;
+
+ /* Pointer to hold rte_event_dma_adapter_op for processing */
+ struct rte_event_dma_adapter_op **op_buffer;
+} __rte_cache_aligned;
+
+/* DMA device information */
+struct dma_device_info {
+ /* Number of vchans configured for a DMA device. */
+ uint16_t num_dma_dev_vchan;
+} __rte_cache_aligned;
+
+struct event_dma_adapter {
+ /* Event device identifier */
+ uint8_t eventdev_id;
+
+ /* Event port identifier */
+ uint8_t event_port_id;
+
+ /* Adapter mode */
+ enum rte_event_dma_adapter_mode mode;
+
+ /* Memory allocation name */
+ char mem_name[DMA_ADAPTER_NAME_LEN];
+
+ /* Socket identifier cached from eventdev */
+ int socket_id;
+
+ /* Lock to serialize config updates with service function */
+ rte_spinlock_t lock;
+
+ /* DMA device structure array */
+ struct dma_device_info *dma_devs;
+
+ /* Circular buffer for processing DMA ops to eventdev */
+ struct dma_ops_circular_buffer ebuf;
+
+ /* Configuration callback for rte_service configuration */
+ rte_event_dma_adapter_conf_cb conf_cb;
+
+ /* Configuration callback argument */
+ void *conf_arg;
+
+ /* Set if default_cb is being used */
+ int default_cb_arg;
+} __rte_cache_aligned;
+
+static struct event_dma_adapter **event_dma_adapter;
+
+static inline int
+edma_adapter_valid_id(uint8_t id)
+{
+ return id < RTE_EVENT_DMA_ADAPTER_MAX_INSTANCE;
+}
+
+static inline struct event_dma_adapter *
+edma_id_to_adapter(uint8_t id)
+{
+ return event_dma_adapter ? event_dma_adapter[id] : NULL;
+}
+
+static int
+edma_array_init(void)
+{
+ const struct rte_memzone *mz;
+ uint32_t sz;
+
+ mz = rte_memzone_lookup(DMA_ADAPTER_ARRAY);
+ if (mz == NULL) {
+ sz = sizeof(struct event_dma_adapter *) * RTE_EVENT_DMA_ADAPTER_MAX_INSTANCE;
+ sz = RTE_ALIGN(sz, RTE_CACHE_LINE_SIZE);
+
+ mz = rte_memzone_reserve_aligned(DMA_ADAPTER_ARRAY, sz, rte_socket_id(), 0,
+ RTE_CACHE_LINE_SIZE);
+ if (mz == NULL) {
+ RTE_EDEV_LOG_ERR("Failed to reserve memzone : %s, err = %d",
+ DMA_ADAPTER_ARRAY, rte_errno);
+ return -rte_errno;
+ }
+ }
+
+ event_dma_adapter = mz->addr;
+
+ return 0;
+}
+
+static inline int
+edma_circular_buffer_init(const char *name, struct dma_ops_circular_buffer *buf, uint16_t sz)
+{
+ buf->op_buffer = rte_zmalloc(name, sizeof(struct rte_event_dma_adapter_op *) * sz, 0);
+ if (buf->op_buffer == NULL)
+ return -ENOMEM;
+
+ buf->size = sz;
+
+ return 0;
+}
+
+static inline void
+edma_circular_buffer_free(struct dma_ops_circular_buffer *buf)
+{
+ rte_free(buf->op_buffer);
+}
+
+static int
+edma_default_config_cb(uint8_t id, uint8_t evdev_id, struct rte_event_dma_adapter_conf *conf,
+ void *arg)
+{
+ struct rte_event_port_conf *port_conf;
+ struct rte_event_dev_config dev_conf;
+ struct event_dma_adapter *adapter;
+ struct rte_eventdev *dev;
+ uint8_t port_id;
+ int started;
+ int ret;
+
+ adapter = edma_id_to_adapter(id);
+ if (adapter == NULL)
+ return -EINVAL;
+
+ dev = &rte_eventdevs[adapter->eventdev_id];
+ dev_conf = dev->data->dev_conf;
+
+ started = dev->data->dev_started;
+ if (started)
+ rte_event_dev_stop(evdev_id);
+
+ port_id = dev_conf.nb_event_ports;
+ dev_conf.nb_event_ports += 1;
+
+ port_conf = arg;
+ if (port_conf->event_port_cfg & RTE_EVENT_PORT_CFG_SINGLE_LINK)
+ dev_conf.nb_single_link_event_port_queues += 1;
+
+ ret = rte_event_dev_configure(evdev_id, &dev_conf);
+ if (ret) {
+ RTE_EDEV_LOG_ERR("Failed to configure event dev %u\n", evdev_id);
+ if (started) {
+ if (rte_event_dev_start(evdev_id))
+ return -EIO;
+ }
+ return ret;
+ }
+
+ ret = rte_event_port_setup(evdev_id, port_id, port_conf);
+ if (ret) {
+ RTE_EDEV_LOG_ERR("Failed to setup event port %u\n", port_id);
+ return ret;
+ }
+
+ conf->event_port_id = port_id;
+ conf->max_nb = DMA_DEFAULT_MAX_NB;
+ if (started)
+ ret = rte_event_dev_start(evdev_id);
+
+ adapter->default_cb_arg = 1;
+ adapter->event_port_id = conf->event_port_id;
+
+ return ret;
+}
+
+int
+rte_event_dma_adapter_create_ext(uint8_t id, uint8_t evdev_id,
+ rte_event_dma_adapter_conf_cb conf_cb,
+ enum rte_event_dma_adapter_mode mode, void *conf_arg)
+{
+ struct rte_event_dev_info dev_info;
+ struct event_dma_adapter *adapter;
+ char name[DMA_ADAPTER_NAME_LEN];
+ struct rte_dma_info info;
+ uint16_t num_dma_dev;
+ int socket_id;
+ uint8_t i;
+ int ret;
+
+ EVENT_DMA_ADAPTER_ID_VALID_OR_ERR_RET(id, -EINVAL);
+ RTE_EVENTDEV_VALID_DEVID_OR_ERR_RET(evdev_id, -EINVAL);
+
+ if (conf_cb == NULL)
+ return -EINVAL;
+
+ if (event_dma_adapter == NULL) {
+ ret = edma_array_init();
+ if (ret)
+ return ret;
+ }
+
+ adapter = edma_id_to_adapter(id);
+ if (adapter != NULL) {
+ RTE_EDEV_LOG_ERR("ML adapter ID %d already exists!", id);
+ return -EEXIST;
+ }
+
+ socket_id = rte_event_dev_socket_id(evdev_id);
+ snprintf(name, DMA_ADAPTER_NAME_LEN, "rte_event_dma_adapter_%d", id);
+ adapter = rte_zmalloc_socket(name, sizeof(struct event_dma_adapter), RTE_CACHE_LINE_SIZE,
+ socket_id);
+ if (adapter == NULL) {
+ RTE_EDEV_LOG_ERR("Failed to get mem for event ML adapter!");
+ return -ENOMEM;
+ }
+
+ if (edma_circular_buffer_init("edma_circular_buffer", &adapter->ebuf,
+ DMA_ADAPTER_BUFFER_SIZE)) {
+ RTE_EDEV_LOG_ERR("Failed to get memory for event adapter circular buffer");
+ rte_free(adapter);
+ return -ENOMEM;
+ }
+
+ ret = rte_event_dev_info_get(evdev_id, &dev_info);
+ if (ret < 0) {
+ RTE_EDEV_LOG_ERR("Failed to get info for eventdev %d: %s", evdev_id,
+ dev_info.driver_name);
+ edma_circular_buffer_free(&adapter->ebuf);
+ rte_free(adapter);
+ return ret;
+ }
+
+ num_dma_dev = rte_dma_count_avail();
+
+ adapter->eventdev_id = evdev_id;
+ adapter->mode = mode;
+ strcpy(adapter->mem_name, name);
+ adapter->socket_id = socket_id;
+ adapter->conf_cb = conf_cb;
+ adapter->conf_arg = conf_arg;
+ adapter->dma_devs = rte_zmalloc_socket(adapter->mem_name,
+ num_dma_dev * sizeof(struct dma_device_info), 0,
+ socket_id);
+ if (adapter->dma_devs == NULL) {
+ RTE_EDEV_LOG_ERR("Failed to get memory for DMA devices\n");
+ edma_circular_buffer_free(&adapter->ebuf);
+ rte_free(adapter);
+ return -ENOMEM;
+ }
+
+ rte_spinlock_init(&adapter->lock);
+ for (i = 0; i < num_dma_dev; i++) {
+ ret = rte_dma_info_get(i, &info);
+ if (ret) {
+ RTE_EDEV_LOG_ERR("Failed to get dma device info\n");
+ edma_circular_buffer_free(&adapter->ebuf);
+ rte_free(adapter);
+ return ret;
+ }
+
+ adapter->dma_devs[i].num_dma_dev_vchan = info.nb_vchans;
+ }
+
+ event_dma_adapter[id] = adapter;
+
+ return 0;
+}
+
+int
+rte_event_dma_adapter_create(uint8_t id, uint8_t evdev_id, struct rte_event_port_conf *port_config,
+ enum rte_event_dma_adapter_mode mode)
+{
+ struct rte_event_port_conf *pc;
+ int ret;
+
+ if (port_config == NULL)
+ return -EINVAL;
+
+ EVENT_DMA_ADAPTER_ID_VALID_OR_ERR_RET(id, -EINVAL);
+
+ pc = rte_malloc(NULL, sizeof(struct rte_event_port_conf), 0);
+ if (pc == NULL)
+ return -ENOMEM;
+
+ rte_memcpy(pc, port_config, sizeof(struct rte_event_port_conf));
+ ret = rte_event_dma_adapter_create_ext(id, evdev_id, edma_default_config_cb, mode, pc);
+ if (ret != 0)
+ rte_free(pc);
+
+ return ret;
+}
+
+int
+rte_event_dma_adapter_free(uint8_t id)
+{
+ struct event_dma_adapter *adapter;
+
+ EVENT_DMA_ADAPTER_ID_VALID_OR_ERR_RET(id, -EINVAL);
+
+ adapter = edma_id_to_adapter(id);
+ if (adapter == NULL)
+ return -EINVAL;
+
+ rte_free(adapter->conf_arg);
+ rte_free(adapter->dma_devs);
+ edma_circular_buffer_free(&adapter->ebuf);
+ rte_free(adapter);
+ event_dma_adapter[id] = NULL;
+
+ return 0;
+}
--
2.25.1
^ permalink raw reply [flat|nested] 110+ messages in thread
* [PATCH v3 04/12] eventdev: add API support for vchan add and delete
2023-09-23 13:34 ` [PATCH v3 00/12] event DMA adapter library support Amit Prakash Shukla
` (2 preceding siblings ...)
2023-09-23 13:34 ` [PATCH v3 03/12] eventdev: create and free API for DMA adapter Amit Prakash Shukla
@ 2023-09-23 13:34 ` Amit Prakash Shukla
2023-09-23 13:34 ` [PATCH v3 05/12] eventdev: add support for DMA adapter service function Amit Prakash Shukla
` (8 subsequent siblings)
12 siblings, 0 replies; 110+ messages in thread
From: Amit Prakash Shukla @ 2023-09-23 13:34 UTC (permalink / raw)
To: Amit Prakash Shukla, Jerin Jacob
Cc: dev, fengchengwen, kevin.laatz, bruce.richardson, conor.walsh,
vattunuru, g.singh, sachin.saxena, hemant.agrawal, cheng1.jiang,
ndabilpuram, anoobj, mb
Added API support to add and delete vchan's from the DMA adapter. DMA devid
and vchan are added to the addapter instance by calling
rte_event_dma_adapter_vchan_add and
deleted using rte_event_dma_adapter_vchan_del.
Signed-off-by: Amit Prakash Shukla <amitprakashs@marvell.com>
---
lib/eventdev/rte_event_dma_adapter.c | 204 +++++++++++++++++++++++++++
1 file changed, 204 insertions(+)
diff --git a/lib/eventdev/rte_event_dma_adapter.c b/lib/eventdev/rte_event_dma_adapter.c
index c7ffba1b47..dd58188bf3 100644
--- a/lib/eventdev/rte_event_dma_adapter.c
+++ b/lib/eventdev/rte_event_dma_adapter.c
@@ -42,8 +42,31 @@ struct dma_ops_circular_buffer {
struct rte_event_dma_adapter_op **op_buffer;
} __rte_cache_aligned;
+/* Vchan information */
+struct dma_vchan_info {
+ /* Set to indicate vchan queue is enabled */
+ bool vq_enabled;
+
+ /* Circular buffer for batching DMA ops to dma_dev */
+ struct dma_ops_circular_buffer dma_buf;
+} __rte_cache_aligned;
+
/* DMA device information */
struct dma_device_info {
+ /* Pointer to vchan queue info */
+ struct dma_vchan_info *vchanq;
+
+ /* Pointer to vchan queue info.
+ * This holds ops passed by application till the
+ * dma completion is done.
+ */
+ struct dma_vchan_info *tqmap;
+
+ /* If num_vchanq > 0, the start callback will
+ * be invoked if not already invoked
+ */
+ uint16_t num_vchanq;
+
/* Number of vchans configured for a DMA device. */
uint16_t num_dma_dev_vchan;
} __rte_cache_aligned;
@@ -81,6 +104,9 @@ struct event_dma_adapter {
/* Set if default_cb is being used */
int default_cb_arg;
+
+ /* No. of vchan queue configured */
+ uint16_t nb_vchanq;
} __rte_cache_aligned;
static struct event_dma_adapter **event_dma_adapter;
@@ -333,3 +359,181 @@ rte_event_dma_adapter_free(uint8_t id)
return 0;
}
+
+static void
+edma_update_vchanq_info(struct event_dma_adapter *adapter, struct dma_device_info *dev_info,
+ uint16_t vchan, uint8_t add)
+{
+ struct dma_vchan_info *vchan_info;
+ struct dma_vchan_info *tqmap_info;
+ int enabled;
+ uint16_t i;
+
+ if (dev_info->vchanq == NULL)
+ return;
+
+ if (vchan == RTE_DMA_ALL_VCHAN) {
+ for (i = 0; i < dev_info->num_dma_dev_vchan; i++)
+ edma_update_vchanq_info(adapter, dev_info, i, add);
+ } else {
+ tqmap_info = &dev_info->tqmap[vchan];
+ vchan_info = &dev_info->vchanq[vchan];
+ enabled = vchan_info->vq_enabled;
+ if (add) {
+ adapter->nb_vchanq += !enabled;
+ dev_info->num_vchanq += !enabled;
+ } else {
+ adapter->nb_vchanq -= enabled;
+ dev_info->num_vchanq -= enabled;
+ }
+ vchan_info->vq_enabled = !!add;
+ tqmap_info->vq_enabled = !!add;
+ }
+}
+
+int
+rte_event_dma_adapter_vchan_add(uint8_t id, int16_t dma_dev_id, uint16_t vchan,
+ const struct rte_event *event)
+{
+ struct event_dma_adapter *adapter;
+ struct dma_device_info *dev_info;
+ struct rte_eventdev *dev;
+ uint32_t cap;
+ int ret;
+
+ EVENT_DMA_ADAPTER_ID_VALID_OR_ERR_RET(id, -EINVAL);
+
+ if (!rte_dma_is_valid(dma_dev_id)) {
+ RTE_EDEV_LOG_ERR("Invalid dma_dev_id = %" PRIu8, dma_dev_id);
+ return -EINVAL;
+ }
+
+ adapter = edma_id_to_adapter(id);
+ if (adapter == NULL)
+ return -EINVAL;
+
+ dev = &rte_eventdevs[adapter->eventdev_id];
+ ret = rte_event_dma_adapter_caps_get(adapter->eventdev_id, dma_dev_id, &cap);
+ if (ret) {
+ RTE_EDEV_LOG_ERR("Failed to get adapter caps dev %u dma_dev %u", id, dma_dev_id);
+ return ret;
+ }
+
+ if ((cap & RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_VCHAN_EV_BIND) && (event == NULL)) {
+ RTE_EDEV_LOG_ERR("Event can not be NULL for dma_dev_id = %u", dma_dev_id);
+ return -EINVAL;
+ }
+
+ dev_info = &adapter->dma_devs[dma_dev_id];
+ if (vchan != RTE_DMA_ALL_VCHAN && vchan >= dev_info->num_dma_dev_vchan) {
+ RTE_EDEV_LOG_ERR("Invalid vhcan %u", vchan);
+ return -EINVAL;
+ }
+
+ /* In case HW cap is RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP_FWD, no
+ * need of service core as HW supports event forward capability.
+ */
+ if ((cap & RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP_FWD) ||
+ (cap & RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_VCHAN_EV_BIND &&
+ adapter->mode == RTE_EVENT_DMA_ADAPTER_OP_NEW) ||
+ (cap & RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP_NEW &&
+ adapter->mode == RTE_EVENT_DMA_ADAPTER_OP_NEW)) {
+ if (*dev->dev_ops->dma_adapter_vchan_add == NULL)
+ return -ENOTSUP;
+ if (dev_info->vchanq == NULL) {
+ dev_info->vchanq = rte_zmalloc_socket(adapter->mem_name,
+ dev_info->num_dma_dev_vchan *
+ sizeof(struct dma_vchan_info),
+ 0, adapter->socket_id);
+ if (dev_info->vchanq == NULL) {
+ printf("Queue pair add not supported\n");
+ return -ENOMEM;
+ }
+ }
+
+ if (dev_info->tqmap == NULL) {
+ dev_info->tqmap = rte_zmalloc_socket(adapter->mem_name,
+ dev_info->num_dma_dev_vchan *
+ sizeof(struct dma_vchan_info),
+ 0, adapter->socket_id);
+ if (dev_info->tqmap == NULL) {
+ printf("tq pair add not supported\n");
+ return -ENOMEM;
+ }
+ }
+
+ ret = (*dev->dev_ops->dma_adapter_vchan_add)(dev, dma_dev_id, vchan, event);
+ if (ret)
+ return ret;
+
+ else
+ edma_update_vchanq_info(adapter, &adapter->dma_devs[dma_dev_id], vchan, 1);
+ }
+
+ return 0;
+}
+
+int
+rte_event_dma_adapter_vchan_del(uint8_t id, int16_t dma_dev_id, uint16_t vchan)
+{
+ struct event_dma_adapter *adapter;
+ struct dma_device_info *dev_info;
+ struct rte_eventdev *dev;
+ uint32_t cap;
+ int ret;
+
+ EVENT_DMA_ADAPTER_ID_VALID_OR_ERR_RET(id, -EINVAL);
+
+ if (!rte_dma_is_valid(dma_dev_id)) {
+ RTE_EDEV_LOG_ERR("Invalid dma_dev_id = %" PRIu8, dma_dev_id);
+ return -EINVAL;
+ }
+
+ adapter = edma_id_to_adapter(id);
+ if (adapter == NULL)
+ return -EINVAL;
+
+ dev = &rte_eventdevs[adapter->eventdev_id];
+ ret = rte_event_dma_adapter_caps_get(adapter->eventdev_id, dma_dev_id, &cap);
+ if (ret)
+ return ret;
+
+ dev_info = &adapter->dma_devs[dma_dev_id];
+
+ if (vchan != RTE_DMA_ALL_VCHAN && vchan >= dev_info->num_dma_dev_vchan) {
+ RTE_EDEV_LOG_ERR("Invalid vhcan %" PRIu16, vchan);
+ return -EINVAL;
+ }
+
+ if ((cap & RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP_FWD) ||
+ (cap & RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP_NEW &&
+ adapter->mode == RTE_EVENT_DMA_ADAPTER_OP_NEW)) {
+ if (*dev->dev_ops->dma_adapter_vchan_del == NULL)
+ return -ENOTSUP;
+ ret = (*dev->dev_ops->dma_adapter_vchan_del)(dev, dma_dev_id, vchan);
+ if (ret == 0) {
+ edma_update_vchanq_info(adapter, dev_info, vchan, 0);
+ if (dev_info->num_vchanq == 0) {
+ rte_free(dev_info->vchanq);
+ dev_info->vchanq = NULL;
+ }
+ }
+ } else {
+ if (adapter->nb_vchanq == 0)
+ return 0;
+
+ rte_spinlock_lock(&adapter->lock);
+ edma_update_vchanq_info(adapter, dev_info, vchan, 0);
+
+ if (dev_info->num_vchanq == 0) {
+ rte_free(dev_info->vchanq);
+ rte_free(dev_info->tqmap);
+ dev_info->vchanq = NULL;
+ dev_info->tqmap = NULL;
+ }
+
+ rte_spinlock_unlock(&adapter->lock);
+ }
+
+ return ret;
+}
--
2.25.1
^ permalink raw reply [flat|nested] 110+ messages in thread
* [PATCH v3 05/12] eventdev: add support for DMA adapter service function
2023-09-23 13:34 ` [PATCH v3 00/12] event DMA adapter library support Amit Prakash Shukla
` (3 preceding siblings ...)
2023-09-23 13:34 ` [PATCH v3 04/12] eventdev: add API support for vchan add and delete Amit Prakash Shukla
@ 2023-09-23 13:34 ` Amit Prakash Shukla
2023-09-23 13:34 ` [PATCH v3 06/12] eventdev: add support for DMA adapter start and stop Amit Prakash Shukla
` (7 subsequent siblings)
12 siblings, 0 replies; 110+ messages in thread
From: Amit Prakash Shukla @ 2023-09-23 13:34 UTC (permalink / raw)
To: Amit Prakash Shukla, Jerin Jacob
Cc: dev, fengchengwen, kevin.laatz, bruce.richardson, conor.walsh,
vattunuru, g.singh, sachin.saxena, hemant.agrawal, cheng1.jiang,
ndabilpuram, anoobj, mb
Added support for DMA adapter service function for event devices.
Enqueue and dequeue of event from eventdev and DMA device are done
based on the adapter mode and the supported HW capabilities.
Signed-off-by: Amit Prakash Shukla <amitprakashs@marvell.com>
---
lib/eventdev/rte_event_dma_adapter.c | 589 +++++++++++++++++++++++++++
1 file changed, 589 insertions(+)
diff --git a/lib/eventdev/rte_event_dma_adapter.c b/lib/eventdev/rte_event_dma_adapter.c
index dd58188bf3..8349b95796 100644
--- a/lib/eventdev/rte_event_dma_adapter.c
+++ b/lib/eventdev/rte_event_dma_adapter.c
@@ -2,6 +2,8 @@
* Copyright (c) 2023 Marvell.
*/
+#include <rte_service_component.h>
+
#include "rte_eventdev.h"
#include "eventdev_pmd.h"
#include "rte_event_dma_adapter.h"
@@ -69,6 +71,10 @@ struct dma_device_info {
/* Number of vchans configured for a DMA device. */
uint16_t num_dma_dev_vchan;
+
+ /* Next queue pair to be processed */
+ uint16_t next_vchan_id;
+
} __rte_cache_aligned;
struct event_dma_adapter {
@@ -90,6 +96,9 @@ struct event_dma_adapter {
/* Lock to serialize config updates with service function */
rte_spinlock_t lock;
+ /* Next dma device to be processed */
+ uint16_t next_dmadev_id;
+
/* DMA device structure array */
struct dma_device_info *dma_devs;
@@ -107,6 +116,26 @@ struct event_dma_adapter {
/* No. of vchan queue configured */
uint16_t nb_vchanq;
+
+ /* Per adapter EAL service ID */
+ uint32_t service_id;
+
+ /* Service initialization state */
+ uint8_t service_initialized;
+
+ /* Max DMA ops processed in any service function invocation */
+ uint32_t max_nb;
+
+ /* Store event port's implicit release capability */
+ uint8_t implicit_release_disabled;
+
+ /* Flag to indicate backpressure at dma_dev
+ * Stop further dequeuing events from eventdev
+ */
+ bool stop_enq_to_dma_dev;
+
+ /* Loop counter to flush dma ops */
+ uint16_t transmit_loop_count;
} __rte_cache_aligned;
static struct event_dma_adapter **event_dma_adapter;
@@ -148,6 +177,18 @@ edma_array_init(void)
return 0;
}
+static inline bool
+edma_circular_buffer_batch_ready(struct dma_ops_circular_buffer *bufp)
+{
+ return bufp->count >= DMA_BATCH_SIZE;
+}
+
+static inline bool
+edma_circular_buffer_space_for_batch(struct dma_ops_circular_buffer *bufp)
+{
+ return (bufp->size - bufp->count) >= DMA_BATCH_SIZE;
+}
+
static inline int
edma_circular_buffer_init(const char *name, struct dma_ops_circular_buffer *buf, uint16_t sz)
{
@@ -166,6 +207,67 @@ edma_circular_buffer_free(struct dma_ops_circular_buffer *buf)
rte_free(buf->op_buffer);
}
+static inline int
+edma_circular_buffer_add(struct dma_ops_circular_buffer *bufp, struct rte_event_dma_adapter_op *op)
+{
+ uint16_t *tail = &bufp->tail;
+
+ bufp->op_buffer[*tail] = op;
+
+ /* circular buffer, go round */
+ *tail = (*tail + 1) % bufp->size;
+ bufp->count++;
+
+ return 0;
+}
+
+static inline int
+edma_circular_buffer_flush_to_dma_dev(struct event_dma_adapter *adapter,
+ struct dma_ops_circular_buffer *bufp, uint8_t dma_dev_id,
+ uint16_t vchan, uint16_t *nb_ops_flushed)
+{
+ struct rte_event_dma_adapter_op *op;
+ struct dma_vchan_info *tq;
+ uint16_t *head = &bufp->head;
+ uint16_t *tail = &bufp->tail;
+ uint16_t n;
+ uint16_t i;
+ int ret;
+
+ if (*tail > *head)
+ n = *tail - *head;
+ else if (*tail < *head)
+ n = bufp->size - *head;
+ else {
+ *nb_ops_flushed = 0;
+ return 0; /* buffer empty */
+ }
+
+ tq = &adapter->dma_devs[dma_dev_id].tqmap[vchan];
+
+ for (i = 0; i < n; i++) {
+ op = bufp->op_buffer[*head];
+ ret = rte_dma_copy_sg(dma_dev_id, vchan, op->src_seg, op->dst_seg,
+ op->nb_src, op->nb_dst, op->flags);
+ if (ret < 0)
+ break;
+
+ /* Enqueue in transaction queue. */
+ edma_circular_buffer_add(&tq->dma_buf, op);
+
+ *head = (*head + 1) % bufp->size;
+ }
+
+ *nb_ops_flushed = i;
+ bufp->count -= *nb_ops_flushed;
+ if (!bufp->count) {
+ *head = 0;
+ *tail = 0;
+ }
+
+ return *nb_ops_flushed == n ? 0 : -1;
+}
+
static int
edma_default_config_cb(uint8_t id, uint8_t evdev_id, struct rte_event_dma_adapter_conf *conf,
void *arg)
@@ -360,6 +462,406 @@ rte_event_dma_adapter_free(uint8_t id)
return 0;
}
+static inline unsigned int
+edma_enq_to_dma_dev(struct event_dma_adapter *adapter, struct rte_event *ev, unsigned int cnt)
+{
+ struct dma_vchan_info *vchan_qinfo = NULL;
+ struct rte_event_dma_adapter_op *dma_op;
+ uint16_t vchan, nb_enqueued = 0;
+ int16_t dma_dev_id;
+ unsigned int i, n;
+ int ret;
+
+ ret = 0;
+ n = 0;
+
+ for (i = 0; i < cnt; i++) {
+ dma_op = ev[i].event_ptr;
+ if (dma_op == NULL)
+ continue;
+
+ /* Expected to have response info appended to dma_op. */
+
+ dma_dev_id = dma_op->dma_dev_id;
+ vchan = dma_op->vchan;
+ vchan_qinfo = &adapter->dma_devs[dma_dev_id].vchanq[vchan];
+ if (!vchan_qinfo->vq_enabled) {
+ if (dma_op != NULL && dma_op->op_mp != NULL)
+ rte_mempool_put(dma_op->op_mp, dma_op);
+ continue;
+ }
+ edma_circular_buffer_add(&vchan_qinfo->dma_buf, dma_op);
+
+ if (edma_circular_buffer_batch_ready(&vchan_qinfo->dma_buf)) {
+ ret = edma_circular_buffer_flush_to_dma_dev(adapter, &vchan_qinfo->dma_buf,
+ dma_dev_id, vchan,
+ &nb_enqueued);
+ n += nb_enqueued;
+
+ /**
+ * If some dma ops failed to flush to dma_dev and
+ * space for another batch is not available, stop
+ * dequeue from eventdev momentarily
+ */
+ if (unlikely(ret < 0 &&
+ !edma_circular_buffer_space_for_batch(&vchan_qinfo->dma_buf)))
+ adapter->stop_enq_to_dma_dev = true;
+ }
+ }
+
+ return n;
+}
+
+static unsigned int
+edma_adapter_dev_flush(struct event_dma_adapter *adapter, int16_t dma_dev_id,
+ uint16_t *nb_ops_flushed)
+{
+ struct dma_vchan_info *vchan_info;
+ struct dma_device_info *dev_info;
+ uint16_t nb = 0, nb_enqueued = 0;
+ uint16_t vchan, nb_vchans;
+
+ dev_info = &adapter->dma_devs[dma_dev_id];
+ nb_vchans = dev_info->num_vchanq;
+
+ for (vchan = 0; vchan < nb_vchans; vchan++) {
+
+ vchan_info = &dev_info->vchanq[vchan];
+ if (unlikely(vchan_info == NULL || !vchan_info->vq_enabled))
+ continue;
+
+ edma_circular_buffer_flush_to_dma_dev(adapter, &vchan_info->dma_buf, dma_dev_id,
+ vchan, &nb_enqueued);
+ *nb_ops_flushed += vchan_info->dma_buf.count;
+ nb += nb_enqueued;
+ }
+
+ return nb;
+}
+
+static unsigned int
+edma_adapter_enq_flush(struct event_dma_adapter *adapter)
+{
+ int16_t dma_dev_id;
+ uint16_t nb_enqueued = 0;
+ uint16_t nb_ops_flushed = 0;
+ uint16_t num_dma_dev = rte_dma_count_avail();
+
+ for (dma_dev_id = 0; dma_dev_id < num_dma_dev; dma_dev_id++)
+ nb_enqueued += edma_adapter_dev_flush(adapter, dma_dev_id, &nb_ops_flushed);
+ /**
+ * Enable dequeue from eventdev if all ops from circular
+ * buffer flushed to dma_dev
+ */
+ if (!nb_ops_flushed)
+ adapter->stop_enq_to_dma_dev = false;
+
+ return nb_enqueued;
+}
+
+/* Flush an instance's enqueue buffers every DMA_ENQ_FLUSH_THRESHOLD
+ * iterations of edma_adapter_enq_run()
+ */
+#define DMA_ENQ_FLUSH_THRESHOLD 1024
+
+static int
+edma_adapter_enq_run(struct event_dma_adapter *adapter, unsigned int max_enq)
+{
+ uint8_t event_port_id = adapter->event_port_id;
+ uint8_t event_dev_id = adapter->eventdev_id;
+ struct rte_event ev[DMA_BATCH_SIZE];
+ unsigned int nb_enq, nb_enqueued;
+ uint16_t n;
+
+ if (adapter->mode == RTE_EVENT_DMA_ADAPTER_OP_NEW)
+ return 0;
+
+ nb_enqueued = 0;
+ for (nb_enq = 0; nb_enq < max_enq; nb_enq += n) {
+
+ if (unlikely(adapter->stop_enq_to_dma_dev)) {
+ nb_enqueued += edma_adapter_enq_flush(adapter);
+
+ if (unlikely(adapter->stop_enq_to_dma_dev))
+ break;
+ }
+
+ n = rte_event_dequeue_burst(event_dev_id, event_port_id, ev, DMA_BATCH_SIZE, 0);
+
+ if (!n)
+ break;
+
+ nb_enqueued += edma_enq_to_dma_dev(adapter, ev, n);
+ }
+
+ if ((++adapter->transmit_loop_count & (DMA_ENQ_FLUSH_THRESHOLD - 1)) == 0)
+ nb_enqueued += edma_adapter_enq_flush(adapter);
+
+ return nb_enqueued;
+}
+
+#define DMA_ADAPTER_MAX_EV_ENQ_RETRIES 100
+
+static inline uint16_t
+edma_ops_enqueue_burst(struct event_dma_adapter *adapter, struct rte_event_dma_adapter_op **ops,
+ uint16_t num)
+{
+ uint8_t event_port_id = adapter->event_port_id;
+ uint8_t event_dev_id = adapter->eventdev_id;
+ struct rte_event events[DMA_BATCH_SIZE];
+ struct rte_event *response_info;
+ uint16_t nb_enqueued, nb_ev;
+ uint8_t retry;
+ uint8_t i;
+
+ nb_ev = 0;
+ retry = 0;
+ nb_enqueued = 0;
+ num = RTE_MIN(num, DMA_BATCH_SIZE);
+ for (i = 0; i < num; i++) {
+ struct rte_event *ev = &events[nb_ev++];
+
+ /* Expected to have response info appended to dma_op. */
+ response_info = (struct rte_event *)((uint8_t *)ops[i] +
+ sizeof(struct rte_event_dma_adapter_op));
+ if (unlikely(response_info == NULL)) {
+ if (ops[i] != NULL && ops[i]->op_mp != NULL)
+ rte_mempool_put(ops[i]->op_mp, ops[i]);
+ continue;
+ }
+
+ rte_memcpy(ev, response_info, sizeof(struct rte_event));
+ ev->event_ptr = ops[i];
+ ev->event_type = RTE_EVENT_TYPE_DMADEV;
+ if (adapter->implicit_release_disabled)
+ ev->op = RTE_EVENT_OP_FORWARD;
+ else
+ ev->op = RTE_EVENT_OP_NEW;
+ }
+
+ do {
+ nb_enqueued += rte_event_enqueue_burst(event_dev_id, event_port_id,
+ &events[nb_enqueued], nb_ev - nb_enqueued);
+
+ } while (retry++ < DMA_ADAPTER_MAX_EV_ENQ_RETRIES && nb_enqueued < nb_ev);
+
+ return nb_enqueued;
+}
+
+static int
+edma_circular_buffer_flush_to_evdev(struct event_dma_adapter *adapter,
+ struct dma_ops_circular_buffer *bufp,
+ uint16_t *enqueue_count)
+{
+ struct rte_event_dma_adapter_op **ops = bufp->op_buffer;
+ uint16_t n = 0, nb_ops_flushed;
+ uint16_t *head = &bufp->head;
+ uint16_t *tail = &bufp->tail;
+
+ if (*tail > *head)
+ n = *tail - *head;
+ else if (*tail < *head)
+ n = bufp->size - *head;
+ else {
+ if (enqueue_count)
+ *enqueue_count = 0;
+ return 0; /* buffer empty */
+ }
+
+ if (enqueue_count && n > *enqueue_count)
+ n = *enqueue_count;
+
+ nb_ops_flushed = edma_ops_enqueue_burst(adapter, &ops[*head], n);
+ if (enqueue_count)
+ *enqueue_count = nb_ops_flushed;
+
+ bufp->count -= nb_ops_flushed;
+ if (!bufp->count) {
+ *head = 0;
+ *tail = 0;
+ return 0; /* buffer empty */
+ }
+
+ *head = (*head + nb_ops_flushed) % bufp->size;
+ return 1;
+}
+
+static void
+edma_ops_buffer_flush(struct event_dma_adapter *adapter)
+{
+ if (likely(adapter->ebuf.count == 0))
+ return;
+
+ while (edma_circular_buffer_flush_to_evdev(adapter, &adapter->ebuf, NULL))
+ ;
+}
+
+static inline unsigned int
+edma_adapter_deq_run(struct event_dma_adapter *adapter, unsigned int max_deq)
+{
+ struct dma_vchan_info *vchan_info;
+ struct dma_ops_circular_buffer *tq_buf;
+ struct rte_event_dma_adapter_op *ops;
+ uint16_t n, nb_deq, nb_enqueued, i;
+ struct dma_device_info *dev_info;
+ uint16_t vchan, num_vchan;
+ uint16_t num_dma_dev;
+ int16_t dma_dev_id;
+ uint16_t index;
+ bool done;
+ bool err;
+
+ nb_deq = 0;
+ edma_ops_buffer_flush(adapter);
+
+ num_dma_dev = rte_dma_count_avail();
+ do {
+ done = true;
+
+ for (dma_dev_id = adapter->next_dmadev_id; dma_dev_id < num_dma_dev; dma_dev_id++) {
+ uint16_t queues = 0;
+ dev_info = &adapter->dma_devs[dma_dev_id];
+ num_vchan = dev_info->num_vchanq;
+
+ for (vchan = dev_info->next_vchan_id; queues < num_vchan;
+ vchan = (vchan + 1) % num_vchan, queues++) {
+
+ vchan_info = &dev_info->vchanq[vchan];
+ if (unlikely(vchan_info == NULL || !vchan_info->vq_enabled))
+ continue;
+
+ n = rte_dma_completed(dma_dev_id, vchan, DMA_BATCH_SIZE,
+ &index, &err);
+ if (!n)
+ continue;
+
+ done = false;
+
+ tq_buf = &dev_info->tqmap[vchan].dma_buf;
+
+ nb_enqueued = n;
+ if (unlikely(!adapter->ebuf.count))
+ edma_circular_buffer_flush_to_evdev(adapter, tq_buf,
+ &nb_enqueued);
+
+ if (likely(nb_enqueued == n))
+ goto check;
+
+ /* Failed to enqueue events case */
+ for (i = nb_enqueued; i < n; i++) {
+ ops = tq_buf->op_buffer[tq_buf->head];
+ edma_circular_buffer_add(&adapter->ebuf, ops);
+ tq_buf->head = (tq_buf->head + 1) % tq_buf->size;
+ }
+
+check:
+ nb_deq += n;
+ if (nb_deq >= max_deq) {
+ if ((vchan + 1) == num_vchan)
+ adapter->next_dmadev_id =
+ (dma_dev_id + 1) % num_dma_dev;
+
+ dev_info->next_vchan_id = (vchan + 1) % num_vchan;
+
+ return nb_deq;
+ }
+ }
+ }
+ adapter->next_dmadev_id = 0;
+
+ } while (done == false);
+
+ return nb_deq;
+}
+
+static int
+edma_adapter_run(struct event_dma_adapter *adapter, unsigned int max_ops)
+{
+ unsigned int ops_left = max_ops;
+
+ while (ops_left > 0) {
+ unsigned int e_cnt, d_cnt;
+
+ e_cnt = edma_adapter_deq_run(adapter, ops_left);
+ ops_left -= RTE_MIN(ops_left, e_cnt);
+
+ d_cnt = edma_adapter_enq_run(adapter, ops_left);
+ ops_left -= RTE_MIN(ops_left, d_cnt);
+
+ if (e_cnt == 0 && d_cnt == 0)
+ break;
+ }
+
+ if (ops_left == max_ops) {
+ rte_event_maintain(adapter->eventdev_id, adapter->event_port_id, 0);
+ return -EAGAIN;
+ } else
+ return 0;
+}
+
+static int
+edma_service_func(void *args)
+{
+ struct event_dma_adapter *adapter = args;
+ int ret;
+
+ if (rte_spinlock_trylock(&adapter->lock) == 0)
+ return 0;
+ ret = edma_adapter_run(adapter, adapter->max_nb);
+ rte_spinlock_unlock(&adapter->lock);
+
+ return ret;
+}
+
+static int
+edma_init_service(struct event_dma_adapter *adapter, uint8_t id)
+{
+ struct rte_event_dma_adapter_conf adapter_conf;
+ struct rte_service_spec service;
+ uint32_t impl_rel;
+ int ret;
+
+ if (adapter->service_initialized)
+ return 0;
+
+ memset(&service, 0, sizeof(service));
+ snprintf(service.name, DMA_ADAPTER_NAME_LEN, "rte_event_dma_adapter_%d", id);
+ service.socket_id = adapter->socket_id;
+ service.callback = edma_service_func;
+ service.callback_userdata = adapter;
+
+ /* Service function handles locking for queue add/del updates */
+ service.capabilities = RTE_SERVICE_CAP_MT_SAFE;
+ ret = rte_service_component_register(&service, &adapter->service_id);
+ if (ret) {
+ RTE_EDEV_LOG_ERR("failed to register service %s err = %" PRId32, service.name, ret);
+ return ret;
+ }
+
+ ret = adapter->conf_cb(id, adapter->eventdev_id, &adapter_conf, adapter->conf_arg);
+ if (ret) {
+ RTE_EDEV_LOG_ERR("configuration callback failed err = %" PRId32, ret);
+ return ret;
+ }
+
+ adapter->max_nb = adapter_conf.max_nb;
+ adapter->event_port_id = adapter_conf.event_port_id;
+
+ if (rte_event_port_attr_get(adapter->eventdev_id, adapter->event_port_id,
+ RTE_EVENT_PORT_ATTR_IMPLICIT_RELEASE_DISABLE, &impl_rel)) {
+ RTE_EDEV_LOG_ERR("Failed to get port info for eventdev %" PRId32,
+ adapter->eventdev_id);
+ edma_circular_buffer_free(&adapter->ebuf);
+ rte_free(adapter);
+ return -EINVAL;
+ }
+
+ adapter->implicit_release_disabled = (uint8_t)impl_rel;
+ adapter->service_initialized = 1;
+
+ return ret;
+}
+
static void
edma_update_vchanq_info(struct event_dma_adapter *adapter, struct dma_device_info *dev_info,
uint16_t vchan, uint8_t add)
@@ -391,6 +893,60 @@ edma_update_vchanq_info(struct event_dma_adapter *adapter, struct dma_device_inf
}
}
+static int
+edma_add_vchan(struct event_dma_adapter *adapter, int16_t dma_dev_id, uint16_t vchan)
+{
+ struct dma_device_info *dev_info = &adapter->dma_devs[dma_dev_id];
+ struct dma_vchan_info *vchanq;
+ struct dma_vchan_info *tqmap;
+ uint16_t nb_vchans;
+ uint32_t i;
+
+ if (dev_info->vchanq == NULL) {
+ nb_vchans = dev_info->num_dma_dev_vchan;
+
+ dev_info->vchanq = rte_zmalloc_socket(adapter->mem_name,
+ nb_vchans * sizeof(struct dma_vchan_info),
+ 0, adapter->socket_id);
+ if (dev_info->vchanq == NULL)
+ return -ENOMEM;
+
+ dev_info->tqmap = rte_zmalloc_socket(adapter->mem_name,
+ nb_vchans * sizeof(struct dma_vchan_info),
+ 0, adapter->socket_id);
+ if (dev_info->tqmap == NULL)
+ return -ENOMEM;
+
+ for (i = 0; i < nb_vchans; i++) {
+ vchanq = &dev_info->vchanq[i];
+
+ if (edma_circular_buffer_init("dma_dev_circular_buffer", &vchanq->dma_buf,
+ DMA_ADAPTER_OPS_BUFFER_SIZE)) {
+ RTE_EDEV_LOG_ERR("Failed to get memory for dma_dev buffer");
+ rte_free(vchanq);
+ return -ENOMEM;
+ }
+
+ tqmap = &dev_info->tqmap[i];
+ if (edma_circular_buffer_init("dma_dev_circular_trans_buf", &tqmap->dma_buf,
+ DMA_ADAPTER_OPS_BUFFER_SIZE)) {
+ RTE_EDEV_LOG_ERR(
+ "Failed to get memory for dma_dev transaction buffer");
+ rte_free(tqmap);
+ return -ENOMEM;
+ }
+ }
+ }
+
+ if (vchan == RTE_DMA_ALL_VCHAN) {
+ for (i = 0; i < dev_info->num_dma_dev_vchan; i++)
+ edma_update_vchanq_info(adapter, dev_info, i, 1);
+ } else
+ edma_update_vchanq_info(adapter, dev_info, vchan, 1);
+
+ return 0;
+}
+
int
rte_event_dma_adapter_vchan_add(uint8_t id, int16_t dma_dev_id, uint16_t vchan,
const struct rte_event *event)
@@ -470,6 +1026,38 @@ rte_event_dma_adapter_vchan_add(uint8_t id, int16_t dma_dev_id, uint16_t vchan,
edma_update_vchanq_info(adapter, &adapter->dma_devs[dma_dev_id], vchan, 1);
}
+ /* In case HW cap is RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP_NEW, or SW adapter, initiate
+ * services so the application can choose which ever way it wants to use the adapter.
+ *
+ * Case 1: RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP_NEW. Application may wants to use one
+ * of below two modes
+ *
+ * a. OP_FORWARD mode -> HW Dequeue + SW enqueue
+ * b. OP_NEW mode -> HW Dequeue
+ *
+ * Case 2: No HW caps, use SW adapter
+ *
+ * a. OP_FORWARD mode -> SW enqueue & dequeue
+ * b. OP_NEW mode -> SW Dequeue
+ */
+ if ((cap & RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP_NEW &&
+ !(cap & RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP_FWD) &&
+ adapter->mode == RTE_EVENT_DMA_ADAPTER_OP_FORWARD) ||
+ (!(cap & RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP_NEW) &&
+ !(cap & RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP_FWD) &&
+ !(cap & RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_VCHAN_EV_BIND))) {
+ rte_spinlock_lock(&adapter->lock);
+ ret = edma_init_service(adapter, id);
+ if (ret == 0)
+ ret = edma_add_vchan(adapter, dma_dev_id, vchan);
+ rte_spinlock_unlock(&adapter->lock);
+
+ if (ret)
+ return ret;
+
+ rte_service_component_runstate_set(adapter->service_id, 1);
+ }
+
return 0;
}
@@ -533,6 +1121,7 @@ rte_event_dma_adapter_vchan_del(uint8_t id, int16_t dma_dev_id, uint16_t vchan)
}
rte_spinlock_unlock(&adapter->lock);
+ rte_service_component_runstate_set(adapter->service_id, adapter->nb_vchanq);
}
return ret;
--
2.25.1
^ permalink raw reply [flat|nested] 110+ messages in thread
* [PATCH v3 06/12] eventdev: add support for DMA adapter start and stop
2023-09-23 13:34 ` [PATCH v3 00/12] event DMA adapter library support Amit Prakash Shukla
` (4 preceding siblings ...)
2023-09-23 13:34 ` [PATCH v3 05/12] eventdev: add support for DMA adapter service function Amit Prakash Shukla
@ 2023-09-23 13:34 ` Amit Prakash Shukla
2023-09-23 13:34 ` [PATCH v3 07/12] eventdev: add support for DMA adapter service ID get Amit Prakash Shukla
` (6 subsequent siblings)
12 siblings, 0 replies; 110+ messages in thread
From: Amit Prakash Shukla @ 2023-09-23 13:34 UTC (permalink / raw)
To: Amit Prakash Shukla, Jerin Jacob
Cc: dev, fengchengwen, kevin.laatz, bruce.richardson, conor.walsh,
vattunuru, g.singh, sachin.saxena, hemant.agrawal, cheng1.jiang,
ndabilpuram, anoobj, mb
Added API support to start and stop DMA adapter.
Signed-off-by: Amit Prakash Shukla <amitprakashs@marvell.com>
---
lib/eventdev/rte_event_dma_adapter.c | 69 ++++++++++++++++++++++++++++
1 file changed, 69 insertions(+)
diff --git a/lib/eventdev/rte_event_dma_adapter.c b/lib/eventdev/rte_event_dma_adapter.c
index 8349b95796..e5c2be3a2f 100644
--- a/lib/eventdev/rte_event_dma_adapter.c
+++ b/lib/eventdev/rte_event_dma_adapter.c
@@ -75,6 +75,13 @@ struct dma_device_info {
/* Next queue pair to be processed */
uint16_t next_vchan_id;
+ /* Set to indicate processing has been started */
+ uint8_t dev_started;
+
+ /* Set to indicate dmadev->eventdev packet
+ * transfer uses a hardware mechanism
+ */
+ uint8_t internal_event_port;
} __rte_cache_aligned;
struct event_dma_adapter {
@@ -1126,3 +1133,65 @@ rte_event_dma_adapter_vchan_del(uint8_t id, int16_t dma_dev_id, uint16_t vchan)
return ret;
}
+
+static int
+edma_adapter_ctrl(uint8_t id, int start)
+{
+ struct event_dma_adapter *adapter;
+ struct dma_device_info *dev_info;
+ struct rte_eventdev *dev;
+ uint16_t num_dma_dev;
+ int stop = !start;
+ int use_service;
+ uint32_t i;
+
+ use_service = 0;
+ EVENT_DMA_ADAPTER_ID_VALID_OR_ERR_RET(id, -EINVAL);
+ adapter = edma_id_to_adapter(id);
+ if (adapter == NULL)
+ return -EINVAL;
+
+ num_dma_dev = rte_dma_count_avail();
+ dev = &rte_eventdevs[adapter->eventdev_id];
+
+ for (i = 0; i < num_dma_dev; i++) {
+ dev_info = &adapter->dma_devs[i];
+ /* start check for num queue pairs */
+ if (start && !dev_info->num_vchanq)
+ continue;
+ /* stop check if dev has been started */
+ if (stop && !dev_info->dev_started)
+ continue;
+ use_service |= !dev_info->internal_event_port;
+ dev_info->dev_started = start;
+ if (dev_info->internal_event_port == 0)
+ continue;
+ start ? (*dev->dev_ops->dma_adapter_start)(dev, i) :
+ (*dev->dev_ops->dma_adapter_stop)(dev, i);
+ }
+
+ if (use_service)
+ rte_service_runstate_set(adapter->service_id, start);
+
+ return 0;
+}
+
+int
+rte_event_dma_adapter_start(uint8_t id)
+{
+ struct event_dma_adapter *adapter;
+
+ EVENT_DMA_ADAPTER_ID_VALID_OR_ERR_RET(id, -EINVAL);
+
+ adapter = edma_id_to_adapter(id);
+ if (adapter == NULL)
+ return -EINVAL;
+
+ return edma_adapter_ctrl(id, 1);
+}
+
+int
+rte_event_dma_adapter_stop(uint8_t id)
+{
+ return edma_adapter_ctrl(id, 0);
+}
--
2.25.1
^ permalink raw reply [flat|nested] 110+ messages in thread
* [PATCH v3 07/12] eventdev: add support for DMA adapter service ID get
2023-09-23 13:34 ` [PATCH v3 00/12] event DMA adapter library support Amit Prakash Shukla
` (5 preceding siblings ...)
2023-09-23 13:34 ` [PATCH v3 06/12] eventdev: add support for DMA adapter start and stop Amit Prakash Shukla
@ 2023-09-23 13:34 ` Amit Prakash Shukla
2023-09-23 13:34 ` [PATCH v3 08/12] eventdev: add DMA adapter support for runtime params Amit Prakash Shukla
` (5 subsequent siblings)
12 siblings, 0 replies; 110+ messages in thread
From: Amit Prakash Shukla @ 2023-09-23 13:34 UTC (permalink / raw)
To: Amit Prakash Shukla, Jerin Jacob
Cc: dev, fengchengwen, kevin.laatz, bruce.richardson, conor.walsh,
vattunuru, g.singh, sachin.saxena, hemant.agrawal, cheng1.jiang,
ndabilpuram, anoobj, mb
Added API support to get DMA adapter service ID. Service id
returned in the variable by the API call shall be used by application
to map a service core.
Signed-off-by: Amit Prakash Shukla <amitprakashs@marvell.com>
---
lib/eventdev/rte_event_dma_adapter.c | 17 +++++++++++++++++
1 file changed, 17 insertions(+)
diff --git a/lib/eventdev/rte_event_dma_adapter.c b/lib/eventdev/rte_event_dma_adapter.c
index e5c2be3a2f..9e7ed622f2 100644
--- a/lib/eventdev/rte_event_dma_adapter.c
+++ b/lib/eventdev/rte_event_dma_adapter.c
@@ -1134,6 +1134,23 @@ rte_event_dma_adapter_vchan_del(uint8_t id, int16_t dma_dev_id, uint16_t vchan)
return ret;
}
+int
+rte_event_dma_adapter_service_id_get(uint8_t id, uint32_t *service_id)
+{
+ struct event_dma_adapter *adapter;
+
+ EVENT_DMA_ADAPTER_ID_VALID_OR_ERR_RET(id, -EINVAL);
+
+ adapter = edma_id_to_adapter(id);
+ if (adapter == NULL || service_id == NULL)
+ return -EINVAL;
+
+ if (adapter->service_initialized)
+ *service_id = adapter->service_id;
+
+ return adapter->service_initialized ? 0 : -ESRCH;
+}
+
static int
edma_adapter_ctrl(uint8_t id, int start)
{
--
2.25.1
^ permalink raw reply [flat|nested] 110+ messages in thread
* [PATCH v3 08/12] eventdev: add DMA adapter support for runtime params
2023-09-23 13:34 ` [PATCH v3 00/12] event DMA adapter library support Amit Prakash Shukla
` (6 preceding siblings ...)
2023-09-23 13:34 ` [PATCH v3 07/12] eventdev: add support for DMA adapter service ID get Amit Prakash Shukla
@ 2023-09-23 13:34 ` Amit Prakash Shukla
2023-09-23 13:34 ` [PATCH v3 09/12] eventdev: add support for DMA adapter stats Amit Prakash Shukla
` (4 subsequent siblings)
12 siblings, 0 replies; 110+ messages in thread
From: Amit Prakash Shukla @ 2023-09-23 13:34 UTC (permalink / raw)
To: Amit Prakash Shukla, Jerin Jacob
Cc: dev, fengchengwen, kevin.laatz, bruce.richardson, conor.walsh,
vattunuru, g.singh, sachin.saxena, hemant.agrawal, cheng1.jiang,
ndabilpuram, anoobj, mb
Added support to set and get runtime params for DMA adapter. The
parameters that can be set/get are defined in
struct rte_event_dma_adapter_runtime_params.
Signed-off-by: Amit Prakash Shukla <amitprakashs@marvell.com>
---
lib/eventdev/rte_event_dma_adapter.c | 93 ++++++++++++++++++++++++++++
1 file changed, 93 insertions(+)
diff --git a/lib/eventdev/rte_event_dma_adapter.c b/lib/eventdev/rte_event_dma_adapter.c
index 9e7ed622f2..cf53e7f429 100644
--- a/lib/eventdev/rte_event_dma_adapter.c
+++ b/lib/eventdev/rte_event_dma_adapter.c
@@ -1212,3 +1212,96 @@ rte_event_dma_adapter_stop(uint8_t id)
{
return edma_adapter_ctrl(id, 0);
}
+
+#define DEFAULT_MAX_NB 128
+
+int
+rte_event_dma_adapter_runtime_params_init(struct rte_event_dma_adapter_runtime_params *params)
+{
+ if (params == NULL)
+ return -EINVAL;
+
+ memset(params, 0, sizeof(*params));
+ params->max_nb = DEFAULT_MAX_NB;
+
+ return 0;
+}
+
+static int
+dma_adapter_cap_check(struct event_dma_adapter *adapter)
+{
+ uint32_t caps;
+ int ret;
+
+ if (!adapter->nb_vchanq)
+ return -EINVAL;
+
+ ret = rte_event_dma_adapter_caps_get(adapter->eventdev_id, adapter->next_dmadev_id, &caps);
+ if (ret) {
+ RTE_EDEV_LOG_ERR("Failed to get adapter caps dev %" PRIu8 " cdev %" PRIu8,
+ adapter->eventdev_id, adapter->next_dmadev_id);
+ return ret;
+ }
+
+ if ((caps & RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP_FWD) ||
+ (caps & RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP_NEW))
+ return -ENOTSUP;
+
+ return 0;
+}
+
+int
+rte_event_dma_adapter_runtime_params_set(uint8_t id,
+ struct rte_event_dma_adapter_runtime_params *params)
+{
+ struct event_dma_adapter *adapter;
+ int ret;
+
+ EVENT_DMA_ADAPTER_ID_VALID_OR_ERR_RET(id, -EINVAL);
+
+ if (params == NULL) {
+ RTE_EDEV_LOG_ERR("params pointer is NULL\n");
+ return -EINVAL;
+ }
+
+ adapter = edma_id_to_adapter(id);
+ if (adapter == NULL)
+ return -EINVAL;
+
+ ret = dma_adapter_cap_check(adapter);
+ if (ret)
+ return ret;
+
+ rte_spinlock_lock(&adapter->lock);
+ adapter->max_nb = params->max_nb;
+ rte_spinlock_unlock(&adapter->lock);
+
+ return 0;
+}
+
+int
+rte_event_dma_adapter_runtime_params_get(uint8_t id,
+ struct rte_event_dma_adapter_runtime_params *params)
+{
+ struct event_dma_adapter *adapter;
+ int ret;
+
+ EVENT_DMA_ADAPTER_ID_VALID_OR_ERR_RET(id, -EINVAL);
+
+ if (params == NULL) {
+ RTE_EDEV_LOG_ERR("params pointer is NULL\n");
+ return -EINVAL;
+ }
+
+ adapter = edma_id_to_adapter(id);
+ if (adapter == NULL)
+ return -EINVAL;
+
+ ret = dma_adapter_cap_check(adapter);
+ if (ret)
+ return ret;
+
+ params->max_nb = adapter->max_nb;
+
+ return 0;
+}
--
2.25.1
^ permalink raw reply [flat|nested] 110+ messages in thread
* [PATCH v3 09/12] eventdev: add support for DMA adapter stats
2023-09-23 13:34 ` [PATCH v3 00/12] event DMA adapter library support Amit Prakash Shukla
` (7 preceding siblings ...)
2023-09-23 13:34 ` [PATCH v3 08/12] eventdev: add DMA adapter support for runtime params Amit Prakash Shukla
@ 2023-09-23 13:34 ` Amit Prakash Shukla
2023-09-23 13:34 ` [PATCH v3 10/12] eventdev: add support for DMA adapter enqueue Amit Prakash Shukla
` (3 subsequent siblings)
12 siblings, 0 replies; 110+ messages in thread
From: Amit Prakash Shukla @ 2023-09-23 13:34 UTC (permalink / raw)
To: Amit Prakash Shukla, Jerin Jacob
Cc: dev, fengchengwen, kevin.laatz, bruce.richardson, conor.walsh,
vattunuru, g.singh, sachin.saxena, hemant.agrawal, cheng1.jiang,
ndabilpuram, anoobj, mb
Added DMA adapter stats API support to get and reset stats. DMA
SW adapter stats and eventdev driver supported stats for enqueue
and dequeue are reported by get API.
Signed-off-by: Amit Prakash Shukla <amitprakashs@marvell.com>
---
lib/eventdev/rte_event_dma_adapter.c | 95 ++++++++++++++++++++++++++++
1 file changed, 95 insertions(+)
diff --git a/lib/eventdev/rte_event_dma_adapter.c b/lib/eventdev/rte_event_dma_adapter.c
index cf53e7f429..35acec6caf 100644
--- a/lib/eventdev/rte_event_dma_adapter.c
+++ b/lib/eventdev/rte_event_dma_adapter.c
@@ -143,6 +143,9 @@ struct event_dma_adapter {
/* Loop counter to flush dma ops */
uint16_t transmit_loop_count;
+
+ /* Per instance stats structure */
+ struct rte_event_dma_adapter_stats dma_stats;
} __rte_cache_aligned;
static struct event_dma_adapter **event_dma_adapter;
@@ -472,6 +475,7 @@ rte_event_dma_adapter_free(uint8_t id)
static inline unsigned int
edma_enq_to_dma_dev(struct event_dma_adapter *adapter, struct rte_event *ev, unsigned int cnt)
{
+ struct rte_event_dma_adapter_stats *stats = &adapter->dma_stats;
struct dma_vchan_info *vchan_qinfo = NULL;
struct rte_event_dma_adapter_op *dma_op;
uint16_t vchan, nb_enqueued = 0;
@@ -481,6 +485,7 @@ edma_enq_to_dma_dev(struct event_dma_adapter *adapter, struct rte_event *ev, uns
ret = 0;
n = 0;
+ stats->event_deq_count += cnt;
for (i = 0; i < cnt; i++) {
dma_op = ev[i].event_ptr;
@@ -503,6 +508,7 @@ edma_enq_to_dma_dev(struct event_dma_adapter *adapter, struct rte_event *ev, uns
ret = edma_circular_buffer_flush_to_dma_dev(adapter, &vchan_qinfo->dma_buf,
dma_dev_id, vchan,
&nb_enqueued);
+ stats->dma_enq_count += nb_enqueued;
n += nb_enqueued;
/**
@@ -549,6 +555,7 @@ edma_adapter_dev_flush(struct event_dma_adapter *adapter, int16_t dma_dev_id,
static unsigned int
edma_adapter_enq_flush(struct event_dma_adapter *adapter)
{
+ struct rte_event_dma_adapter_stats *stats = &adapter->dma_stats;
int16_t dma_dev_id;
uint16_t nb_enqueued = 0;
uint16_t nb_ops_flushed = 0;
@@ -563,6 +570,8 @@ edma_adapter_enq_flush(struct event_dma_adapter *adapter)
if (!nb_ops_flushed)
adapter->stop_enq_to_dma_dev = false;
+ stats->dma_enq_count += nb_enqueued;
+
return nb_enqueued;
}
@@ -574,6 +583,7 @@ edma_adapter_enq_flush(struct event_dma_adapter *adapter)
static int
edma_adapter_enq_run(struct event_dma_adapter *adapter, unsigned int max_enq)
{
+ struct rte_event_dma_adapter_stats *stats = &adapter->dma_stats;
uint8_t event_port_id = adapter->event_port_id;
uint8_t event_dev_id = adapter->eventdev_id;
struct rte_event ev[DMA_BATCH_SIZE];
@@ -593,6 +603,7 @@ edma_adapter_enq_run(struct event_dma_adapter *adapter, unsigned int max_enq)
break;
}
+ stats->event_poll_count++;
n = rte_event_dequeue_burst(event_dev_id, event_port_id, ev, DMA_BATCH_SIZE, 0);
if (!n)
@@ -613,6 +624,7 @@ static inline uint16_t
edma_ops_enqueue_burst(struct event_dma_adapter *adapter, struct rte_event_dma_adapter_op **ops,
uint16_t num)
{
+ struct rte_event_dma_adapter_stats *stats = &adapter->dma_stats;
uint8_t event_port_id = adapter->event_port_id;
uint8_t event_dev_id = adapter->eventdev_id;
struct rte_event events[DMA_BATCH_SIZE];
@@ -652,6 +664,10 @@ edma_ops_enqueue_burst(struct event_dma_adapter *adapter, struct rte_event_dma_a
} while (retry++ < DMA_ADAPTER_MAX_EV_ENQ_RETRIES && nb_enqueued < nb_ev);
+ stats->event_enq_fail_count += nb_ev - nb_enqueued;
+ stats->event_enq_count += nb_enqueued;
+ stats->event_enq_retry_count += retry - 1;
+
return nb_enqueued;
}
@@ -706,6 +722,7 @@ edma_ops_buffer_flush(struct event_dma_adapter *adapter)
static inline unsigned int
edma_adapter_deq_run(struct event_dma_adapter *adapter, unsigned int max_deq)
{
+ struct rte_event_dma_adapter_stats *stats = &adapter->dma_stats;
struct dma_vchan_info *vchan_info;
struct dma_ops_circular_buffer *tq_buf;
struct rte_event_dma_adapter_op *ops;
@@ -743,6 +760,7 @@ edma_adapter_deq_run(struct event_dma_adapter *adapter, unsigned int max_deq)
continue;
done = false;
+ stats->dma_deq_count += n;
tq_buf = &dev_info->tqmap[vchan].dma_buf;
@@ -1305,3 +1323,80 @@ rte_event_dma_adapter_runtime_params_get(uint8_t id,
return 0;
}
+
+int
+rte_event_dma_adapter_stats_get(uint8_t id, struct rte_event_dma_adapter_stats *stats)
+{
+ struct rte_event_dma_adapter_stats dev_stats_sum = {0};
+ struct rte_event_dma_adapter_stats dev_stats;
+ struct event_dma_adapter *adapter;
+ struct dma_device_info *dev_info;
+ struct rte_eventdev *dev;
+ uint16_t num_dma_dev;
+ uint32_t i;
+ int ret;
+
+ EVENT_DMA_ADAPTER_ID_VALID_OR_ERR_RET(id, -EINVAL);
+
+ adapter = edma_id_to_adapter(id);
+ if (adapter == NULL || stats == NULL)
+ return -EINVAL;
+
+ num_dma_dev = rte_dma_count_avail();
+ dev = &rte_eventdevs[adapter->eventdev_id];
+ memset(stats, 0, sizeof(*stats));
+ for (i = 0; i < num_dma_dev; i++) {
+ dev_info = &adapter->dma_devs[i];
+
+ if (dev_info->internal_event_port == 0 ||
+ dev->dev_ops->dma_adapter_stats_get == NULL)
+ continue;
+
+ ret = (*dev->dev_ops->dma_adapter_stats_get)(dev, i, &dev_stats);
+ if (ret)
+ continue;
+
+ dev_stats_sum.dma_deq_count += dev_stats.dma_deq_count;
+ dev_stats_sum.event_enq_count += dev_stats.event_enq_count;
+ }
+
+ if (adapter->service_initialized)
+ *stats = adapter->dma_stats;
+
+ stats->dma_deq_count += dev_stats_sum.dma_deq_count;
+ stats->event_enq_count += dev_stats_sum.event_enq_count;
+
+ return 0;
+}
+
+int
+rte_event_dma_adapter_stats_reset(uint8_t id)
+{
+ struct event_dma_adapter *adapter;
+ struct dma_device_info *dev_info;
+ struct rte_eventdev *dev;
+ uint16_t num_dma_dev;
+ uint32_t i;
+
+ EVENT_DMA_ADAPTER_ID_VALID_OR_ERR_RET(id, -EINVAL);
+
+ adapter = edma_id_to_adapter(id);
+ if (adapter == NULL)
+ return -EINVAL;
+
+ num_dma_dev = rte_dma_count_avail();
+ dev = &rte_eventdevs[adapter->eventdev_id];
+ for (i = 0; i < num_dma_dev; i++) {
+ dev_info = &adapter->dma_devs[i];
+
+ if (dev_info->internal_event_port == 0 ||
+ dev->dev_ops->dma_adapter_stats_reset == NULL)
+ continue;
+
+ (*dev->dev_ops->dma_adapter_stats_reset)(dev, i);
+ }
+
+ memset(&adapter->dma_stats, 0, sizeof(adapter->dma_stats));
+
+ return 0;
+}
--
2.25.1
^ permalink raw reply [flat|nested] 110+ messages in thread
* [PATCH v3 10/12] eventdev: add support for DMA adapter enqueue
2023-09-23 13:34 ` [PATCH v3 00/12] event DMA adapter library support Amit Prakash Shukla
` (8 preceding siblings ...)
2023-09-23 13:34 ` [PATCH v3 09/12] eventdev: add support for DMA adapter stats Amit Prakash Shukla
@ 2023-09-23 13:34 ` Amit Prakash Shukla
2023-09-23 13:34 ` [PATCH v3 11/12] eventdev: add DMA adapter event port get Amit Prakash Shukla
` (2 subsequent siblings)
12 siblings, 0 replies; 110+ messages in thread
From: Amit Prakash Shukla @ 2023-09-23 13:34 UTC (permalink / raw)
To: Amit Prakash Shukla, Jerin Jacob
Cc: dev, fengchengwen, kevin.laatz, bruce.richardson, conor.walsh,
vattunuru, g.singh, sachin.saxena, hemant.agrawal, cheng1.jiang,
ndabilpuram, anoobj, mb
Added API support to enqueue a DMA operation to the DMA driver.
Signed-off-by: Amit Prakash Shukla <amitprakashs@marvell.com>
---
lib/eventdev/rte_event_dma_adapter.c | 13 +++++++++++++
1 file changed, 13 insertions(+)
diff --git a/lib/eventdev/rte_event_dma_adapter.c b/lib/eventdev/rte_event_dma_adapter.c
index 35acec6caf..9770b0b8df 100644
--- a/lib/eventdev/rte_event_dma_adapter.c
+++ b/lib/eventdev/rte_event_dma_adapter.c
@@ -1400,3 +1400,16 @@ rte_event_dma_adapter_stats_reset(uint8_t id)
return 0;
}
+
+uint16_t
+rte_event_dma_adapter_enqueue(uint8_t dev_id, uint8_t port_id, struct rte_event ev[],
+ uint16_t nb_events)
+{
+ const struct rte_event_fp_ops *fp_ops;
+ void *port;
+
+ fp_ops = &rte_event_fp_ops[dev_id];
+ port = fp_ops->data[port_id];
+
+ return fp_ops->dma_enqueue(port, ev, nb_events);
+}
--
2.25.1
^ permalink raw reply [flat|nested] 110+ messages in thread
* [PATCH v3 11/12] eventdev: add DMA adapter event port get
2023-09-23 13:34 ` [PATCH v3 00/12] event DMA adapter library support Amit Prakash Shukla
` (9 preceding siblings ...)
2023-09-23 13:34 ` [PATCH v3 10/12] eventdev: add support for DMA adapter enqueue Amit Prakash Shukla
@ 2023-09-23 13:34 ` Amit Prakash Shukla
2023-09-23 13:34 ` [PATCH v3 12/12] app/test: add event DMA adapter auto-test Amit Prakash Shukla
2023-09-26 10:32 ` [PATCH v4 00/12] event DMA adapter library support Amit Prakash Shukla
12 siblings, 0 replies; 110+ messages in thread
From: Amit Prakash Shukla @ 2023-09-23 13:34 UTC (permalink / raw)
To: Amit Prakash Shukla, Jerin Jacob
Cc: dev, fengchengwen, kevin.laatz, bruce.richardson, conor.walsh,
vattunuru, g.singh, sachin.saxena, hemant.agrawal, cheng1.jiang,
ndabilpuram, anoobj, mb
Added support for DMA adapter event port get.
Signed-off-by: Amit Prakash Shukla <amitprakashs@marvell.com>
---
lib/eventdev/rte_event_dma_adapter.c | 16 ++++++++++++++++
1 file changed, 16 insertions(+)
diff --git a/lib/eventdev/rte_event_dma_adapter.c b/lib/eventdev/rte_event_dma_adapter.c
index 9770b0b8df..2b0919d705 100644
--- a/lib/eventdev/rte_event_dma_adapter.c
+++ b/lib/eventdev/rte_event_dma_adapter.c
@@ -472,6 +472,22 @@ rte_event_dma_adapter_free(uint8_t id)
return 0;
}
+int
+rte_event_dma_adapter_event_port_get(uint8_t id, uint8_t *event_port_id)
+{
+ struct event_dma_adapter *adapter;
+
+ EVENT_DMA_ADAPTER_ID_VALID_OR_ERR_RET(id, -EINVAL);
+
+ adapter = edma_id_to_adapter(id);
+ if (adapter == NULL || event_port_id == NULL)
+ return -EINVAL;
+
+ *event_port_id = adapter->event_port_id;
+
+ return 0;
+}
+
static inline unsigned int
edma_enq_to_dma_dev(struct event_dma_adapter *adapter, struct rte_event *ev, unsigned int cnt)
{
--
2.25.1
^ permalink raw reply [flat|nested] 110+ messages in thread
* [PATCH v3 12/12] app/test: add event DMA adapter auto-test
2023-09-23 13:34 ` [PATCH v3 00/12] event DMA adapter library support Amit Prakash Shukla
` (10 preceding siblings ...)
2023-09-23 13:34 ` [PATCH v3 11/12] eventdev: add DMA adapter event port get Amit Prakash Shukla
@ 2023-09-23 13:34 ` Amit Prakash Shukla
2023-09-26 10:32 ` [PATCH v4 00/12] event DMA adapter library support Amit Prakash Shukla
12 siblings, 0 replies; 110+ messages in thread
From: Amit Prakash Shukla @ 2023-09-23 13:34 UTC (permalink / raw)
Cc: dev, jerinj, fengchengwen, kevin.laatz, bruce.richardson,
conor.walsh, vattunuru, g.singh, sachin.saxena, hemant.agrawal,
cheng1.jiang, ndabilpuram, anoobj, mb, Amit Prakash Shukla
Added testsuite to test the dma adapter functionality.
The testsuite detects event and DMA device capability
and accordingly dma adapter is configured and modes are
tested.
Signed-off-by: Amit Prakash Shukla <amitprakashs@marvell.com>
---
app/test/meson.build | 1 +
app/test/test_event_dma_adapter.c | 805 ++++++++++++++++++++++++++++++
2 files changed, 806 insertions(+)
create mode 100644 app/test/test_event_dma_adapter.c
diff --git a/app/test/meson.build b/app/test/meson.build
index 05bae9216d..7caf5ae5fc 100644
--- a/app/test/meson.build
+++ b/app/test/meson.build
@@ -66,6 +66,7 @@ source_file_deps = {
'test_errno.c': [],
'test_ethdev_link.c': ['ethdev'],
'test_event_crypto_adapter.c': ['cryptodev', 'eventdev', 'bus_vdev'],
+ 'test_event_dma_adapter.c': ['dmadev', 'eventdev', 'bus_vdev'],
'test_event_eth_rx_adapter.c': ['ethdev', 'eventdev', 'bus_vdev'],
'test_event_eth_tx_adapter.c': ['bus_vdev', 'ethdev', 'net_ring', 'eventdev'],
'test_event_ring.c': ['eventdev'],
diff --git a/app/test/test_event_dma_adapter.c b/app/test/test_event_dma_adapter.c
new file mode 100644
index 0000000000..fe1311e7f8
--- /dev/null
+++ b/app/test/test_event_dma_adapter.c
@@ -0,0 +1,805 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright (c) 2023 Marvell.
+ */
+
+#include "test.h"
+#include <string.h>
+#include <rte_common.h>
+#include <rte_malloc.h>
+#include <rte_mempool.h>
+#include <rte_mbuf.h>
+#include <rte_random.h>
+
+#ifdef RTE_EXEC_ENV_WINDOWS
+static int
+test_event_dma_adapter(void)
+{
+ printf("event_dma_adapter not supported on Windows, skipping test\n");
+ return TEST_SKIPPED;
+}
+
+#else
+
+#include <rte_bus_vdev.h>
+#include <rte_dmadev.h>
+#include <rte_eventdev.h>
+#include <rte_event_dma_adapter.h>
+#include <rte_service.h>
+
+#define NUM_MBUFS (8191)
+#define MBUF_CACHE_SIZE (256)
+#define TEST_APP_PORT_ID 0
+#define TEST_APP_EV_QUEUE_ID 0
+#define TEST_APP_EV_PRIORITY 0
+#define TEST_APP_EV_FLOWID 0xAABB
+#define TEST_DMA_EV_QUEUE_ID 1
+#define TEST_ADAPTER_ID 0
+#define TEST_DMA_DEV_ID 0
+#define TEST_DMA_VCHAN_ID 0
+#define PACKET_LENGTH 1024
+#define NB_TEST_PORTS 1
+#define NB_TEST_QUEUES 2
+#define NUM_CORES 2
+#define DMA_OP_POOL_SIZE 128
+#define TEST_MAX_OP 128
+#define TEST_RINGSIZE 512
+
+#define MBUF_SIZE (RTE_PKTMBUF_HEADROOM + PACKET_LENGTH)
+
+/* Handle log statements in same manner as test macros */
+#define LOG_DBG(...) RTE_LOG(DEBUG, EAL, __VA_ARGS__)
+
+struct event_dma_adapter_test_params {
+ struct rte_mempool *src_mbuf_pool;
+ struct rte_mempool *dst_mbuf_pool;
+ struct rte_mempool *op_mpool;
+ uint8_t dma_event_port_id;
+ uint8_t internal_port_op_fwd;
+};
+
+struct rte_event dma_response_info = {
+ .queue_id = TEST_APP_EV_QUEUE_ID,
+ .sched_type = RTE_SCHED_TYPE_ATOMIC,
+ .flow_id = TEST_APP_EV_FLOWID,
+ .priority = TEST_APP_EV_PRIORITY
+};
+
+static struct event_dma_adapter_test_params params;
+static uint8_t dma_adapter_setup_done;
+static uint32_t slcore_id;
+static int evdev;
+
+static int
+send_recv_ev(struct rte_event *ev)
+{
+ struct rte_event recv_ev[TEST_MAX_OP];
+ uint16_t nb_enqueued = 0;
+ int i = 0;
+
+ if (params.internal_port_op_fwd) {
+ nb_enqueued = rte_event_dma_adapter_enqueue(evdev, TEST_APP_PORT_ID, ev,
+ TEST_MAX_OP);
+ } else {
+ while (nb_enqueued < TEST_MAX_OP) {
+ nb_enqueued += rte_event_enqueue_burst(evdev, TEST_APP_PORT_ID,
+ &ev[nb_enqueued], TEST_MAX_OP -
+ nb_enqueued);
+ }
+ }
+
+ TEST_ASSERT_EQUAL(nb_enqueued, TEST_MAX_OP, "Failed to send event to dma adapter\n");
+
+ while (i < TEST_MAX_OP) {
+ if (rte_event_dequeue_burst(evdev, TEST_APP_PORT_ID, &recv_ev[i], 1, 0) != 1)
+ continue;
+ i++;
+ }
+
+ TEST_ASSERT_EQUAL(i, TEST_MAX_OP, "Test failed. Failed to dequeue events.\n");
+
+ return TEST_SUCCESS;
+}
+
+static int
+test_dma_adapter_stats(void)
+{
+ struct rte_event_dma_adapter_stats stats;
+
+ rte_event_dma_adapter_stats_get(TEST_ADAPTER_ID, &stats);
+ printf(" +------------------------------------------------------+\n");
+ printf(" + DMA adapter stats for instance %u:\n", TEST_ADAPTER_ID);
+ printf(" + Event port poll count 0x%" PRIx64 "\n",
+ stats.event_poll_count);
+ printf(" + Event dequeue count 0x%" PRIx64 "\n",
+ stats.event_deq_count);
+ printf(" + DMA dev enqueue count 0x%" PRIx64 "\n",
+ stats.dma_enq_count);
+ printf(" + DMA dev enqueue failed count 0x%" PRIx64 "\n",
+ stats.dma_enq_fail_count);
+ printf(" + DMA dev dequeue count 0x%" PRIx64 "\n",
+ stats.dma_deq_count);
+ printf(" + Event enqueue count 0x%" PRIx64 "\n",
+ stats.event_enq_count);
+ printf(" + Event enqueue retry count 0x%" PRIx64 "\n",
+ stats.event_enq_retry_count);
+ printf(" + Event enqueue fail count 0x%" PRIx64 "\n",
+ stats.event_enq_fail_count);
+ printf(" +------------------------------------------------------+\n");
+
+ rte_event_dma_adapter_stats_reset(TEST_ADAPTER_ID);
+ return TEST_SUCCESS;
+}
+
+static int
+test_dma_adapter_params(void)
+{
+ struct rte_event_dma_adapter_runtime_params out_params;
+ struct rte_event_dma_adapter_runtime_params in_params;
+ struct rte_event event;
+ uint32_t cap;
+ int err, rc;
+
+ err = rte_event_dma_adapter_caps_get(evdev, TEST_DMA_DEV_ID, &cap);
+ TEST_ASSERT_SUCCESS(err, "Failed to get adapter capabilities\n");
+
+ if (cap & RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_VCHAN_EV_BIND) {
+ err = rte_event_dma_adapter_vchan_add(TEST_ADAPTER_ID, TEST_DMA_DEV_ID,
+ TEST_DMA_VCHAN_ID, &event);
+ } else
+ err = rte_event_dma_adapter_vchan_add(TEST_ADAPTER_ID, TEST_DMA_DEV_ID,
+ TEST_DMA_VCHAN_ID, NULL);
+
+ TEST_ASSERT_SUCCESS(err, "Failed to add vchan\n");
+
+ err = rte_event_dma_adapter_runtime_params_init(&in_params);
+ TEST_ASSERT(err == 0, "Expected 0 got %d", err);
+ err = rte_event_dma_adapter_runtime_params_init(&out_params);
+ TEST_ASSERT(err == 0, "Expected 0 got %d", err);
+
+ /* Case 1: Get the default value of mbufs processed by adapter */
+ err = rte_event_dma_adapter_runtime_params_get(TEST_ADAPTER_ID, &out_params);
+ if (err == -ENOTSUP) {
+ rc = TEST_SKIPPED;
+ goto vchan_del;
+ }
+ TEST_ASSERT(err == 0, "Expected 0 got %d", err);
+
+ /* Case 2: Set max_nb = 32 (=BATCH_SEIZE) */
+ in_params.max_nb = 32;
+
+ err = rte_event_dma_adapter_runtime_params_set(TEST_ADAPTER_ID, &in_params);
+ TEST_ASSERT(err == 0, "Expected 0 got %d", err);
+
+ err = rte_event_dma_adapter_runtime_params_get(TEST_ADAPTER_ID, &out_params);
+ TEST_ASSERT(err == 0, "Expected 0 got %d", err);
+ TEST_ASSERT(in_params.max_nb == out_params.max_nb, "Expected %u got %u",
+ in_params.max_nb, out_params.max_nb);
+
+ /* Case 3: Set max_nb = 192 */
+ in_params.max_nb = 192;
+
+ err = rte_event_dma_adapter_runtime_params_set(TEST_ADAPTER_ID, &in_params);
+ TEST_ASSERT(err == 0, "Expected 0 got %d", err);
+
+ err = rte_event_dma_adapter_runtime_params_get(TEST_ADAPTER_ID, &out_params);
+ TEST_ASSERT(err == 0, "Expected 0 got %d", err);
+ TEST_ASSERT(in_params.max_nb == out_params.max_nb, "Expected %u got %u",
+ in_params.max_nb, out_params.max_nb);
+
+ /* Case 4: Set max_nb = 256 */
+ in_params.max_nb = 256;
+
+ err = rte_event_dma_adapter_runtime_params_set(TEST_ADAPTER_ID, &in_params);
+ TEST_ASSERT(err == 0, "Expected 0 got %d", err);
+
+ err = rte_event_dma_adapter_runtime_params_get(TEST_ADAPTER_ID, &out_params);
+ TEST_ASSERT(err == 0, "Expected 0 got %d", err);
+ TEST_ASSERT(in_params.max_nb == out_params.max_nb, "Expected %u got %u",
+ in_params.max_nb, out_params.max_nb);
+
+ /* Case 5: Set max_nb = 30(<BATCH_SIZE) */
+ in_params.max_nb = 30;
+
+ err = rte_event_dma_adapter_runtime_params_set(TEST_ADAPTER_ID, &in_params);
+ TEST_ASSERT(err == 0, "Expected 0 got %d", err);
+
+ err = rte_event_dma_adapter_runtime_params_get(TEST_ADAPTER_ID, &out_params);
+ TEST_ASSERT(err == 0, "Expected 0 got %d", err);
+ TEST_ASSERT(in_params.max_nb == out_params.max_nb, "Expected %u got %u",
+ in_params.max_nb, out_params.max_nb);
+
+ /* Case 6: Set max_nb = 512 */
+ in_params.max_nb = 512;
+
+ err = rte_event_dma_adapter_runtime_params_set(TEST_ADAPTER_ID, &in_params);
+ TEST_ASSERT(err == 0, "Expected 0 got %d", err);
+
+ err = rte_event_dma_adapter_runtime_params_get(TEST_ADAPTER_ID, &out_params);
+ TEST_ASSERT(err == 0, "Expected 0 got %d", err);
+ TEST_ASSERT(in_params.max_nb == out_params.max_nb, "Expected %u got %u",
+ in_params.max_nb, out_params.max_nb);
+
+ rc = TEST_SUCCESS;
+vchan_del:
+ err = rte_event_dma_adapter_vchan_del(TEST_ADAPTER_ID, TEST_DMA_DEV_ID,
+ TEST_DMA_VCHAN_ID);
+ TEST_ASSERT_SUCCESS(err, "Failed to delete vchan\n");
+
+ return rc;
+}
+
+static int
+test_op_forward_mode(void)
+{
+ struct rte_mbuf *src_mbuf[TEST_MAX_OP];
+ struct rte_mbuf *dst_mbuf[TEST_MAX_OP];
+ struct rte_event_dma_adapter_op *op;
+ struct rte_event ev[TEST_MAX_OP];
+ struct rte_event response_info;
+ int ret, i;
+
+ ret = rte_pktmbuf_alloc_bulk(params.src_mbuf_pool, src_mbuf, TEST_MAX_OP);
+ TEST_ASSERT_SUCCESS(ret, "alloc src mbufs failed.\n");
+
+ ret = rte_pktmbuf_alloc_bulk(params.dst_mbuf_pool, dst_mbuf, TEST_MAX_OP);
+ TEST_ASSERT_SUCCESS(ret, "alloc dst mbufs failed.\n");
+
+ for (i = 0; i < TEST_MAX_OP; i++) {
+ memset(rte_pktmbuf_mtod(src_mbuf[i], void *), rte_rand(), PACKET_LENGTH);
+ memset(rte_pktmbuf_mtod(dst_mbuf[i], void *), 0, PACKET_LENGTH);
+ }
+
+ for (i = 0; i < TEST_MAX_OP; i++) {
+ rte_mempool_get(params.op_mpool, (void **)&op);
+ TEST_ASSERT_NOT_NULL(op, "Failed to allocate dma operation struct\n");
+
+ op->src_seg = rte_malloc(NULL, sizeof(struct rte_dma_sge), 0);
+ op->dst_seg = rte_malloc(NULL, sizeof(struct rte_dma_sge), 0);
+
+ /* Update Op */
+ op->src_seg->addr = rte_pktmbuf_iova(src_mbuf[i]);
+ op->dst_seg->addr = rte_pktmbuf_iova(dst_mbuf[i]);
+ op->src_seg->length = PACKET_LENGTH;
+ op->dst_seg->length = PACKET_LENGTH;
+ op->nb_src = 1;
+ op->nb_dst = 1;
+ op->flags = RTE_DMA_OP_FLAG_SUBMIT;
+ op->op_mp = params.op_mpool;
+ op->dma_dev_id = TEST_DMA_DEV_ID;
+ op->vchan = TEST_DMA_VCHAN_ID;
+
+ response_info.event = dma_response_info.event;
+ rte_memcpy((uint8_t *)op + sizeof(struct rte_event_dma_adapter_op), &response_info,
+ sizeof(struct rte_event));
+
+ /* Fill in event info and update event_ptr with rte_event_dma_adapter_op */
+ memset(&ev[i], 0, sizeof(struct rte_event));
+ ev[i].event = 0;
+ ev[i].event_type = RTE_EVENT_TYPE_DMADEV;
+ ev[i].queue_id = TEST_DMA_EV_QUEUE_ID;
+ ev[i].sched_type = RTE_SCHED_TYPE_ATOMIC;
+ ev[i].flow_id = 0xAABB;
+ ev[i].event_ptr = op;
+ }
+
+ ret = send_recv_ev(ev);
+ TEST_ASSERT_SUCCESS(ret, "Failed to send/receive event to dma adapter\n");
+
+ test_dma_adapter_stats();
+
+ for (i = 0; i < TEST_MAX_OP; i++) {
+ op = ev[i].event_ptr;
+ ret = memcmp(rte_pktmbuf_mtod(src_mbuf[i], void *),
+ rte_pktmbuf_mtod(dst_mbuf[i], void *), PACKET_LENGTH);
+
+ TEST_ASSERT_EQUAL(ret, 0, "Data mismatch for dma adapter\n");
+
+ rte_free(op->src_seg);
+ rte_free(op->dst_seg);
+ rte_mempool_put(op->op_mp, op);
+ }
+
+ rte_pktmbuf_free_bulk(src_mbuf, TEST_MAX_OP);
+ rte_pktmbuf_free_bulk(dst_mbuf, TEST_MAX_OP);
+
+ return TEST_SUCCESS;
+}
+
+static int
+map_adapter_service_core(void)
+{
+ uint32_t adapter_service_id;
+ int ret;
+
+ if (rte_event_dma_adapter_service_id_get(TEST_ADAPTER_ID, &adapter_service_id) == 0) {
+ uint32_t core_list[NUM_CORES];
+
+ ret = rte_service_lcore_list(core_list, NUM_CORES);
+ TEST_ASSERT(ret >= 0, "Failed to get service core list!");
+
+ if (core_list[0] != slcore_id) {
+ TEST_ASSERT_SUCCESS(rte_service_lcore_add(slcore_id),
+ "Failed to add service core");
+ TEST_ASSERT_SUCCESS(rte_service_lcore_start(slcore_id),
+ "Failed to start service core");
+ }
+
+ TEST_ASSERT_SUCCESS(rte_service_map_lcore_set(
+ adapter_service_id, slcore_id, 1),
+ "Failed to map adapter service");
+ }
+
+ return TEST_SUCCESS;
+}
+
+static int
+test_with_op_forward_mode(void)
+{
+ uint32_t cap;
+ int ret;
+
+ ret = rte_event_dma_adapter_caps_get(evdev, TEST_DMA_DEV_ID, &cap);
+ TEST_ASSERT_SUCCESS(ret, "Failed to get adapter capabilities\n");
+
+ if (!(cap & RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP_FWD) &&
+ !(cap & RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP_NEW))
+ map_adapter_service_core();
+ else {
+ if (!(cap & RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP_FWD))
+ return TEST_SKIPPED;
+ }
+
+ TEST_ASSERT_SUCCESS(rte_event_dma_adapter_start(TEST_ADAPTER_ID),
+ "Failed to start event dma adapter");
+
+ ret = test_op_forward_mode();
+ TEST_ASSERT_SUCCESS(ret, "DMA - FORWARD mode test failed\n");
+ return TEST_SUCCESS;
+}
+
+static int
+configure_dmadev(void)
+{
+ const struct rte_dma_conf conf = { .nb_vchans = 1};
+ const struct rte_dma_vchan_conf qconf = {
+ .direction = RTE_DMA_DIR_MEM_TO_MEM,
+ .nb_desc = TEST_RINGSIZE,
+ };
+ struct rte_dma_info info;
+ unsigned int elt_size;
+ int ret;
+
+ ret = rte_dma_count_avail();
+ RTE_TEST_ASSERT_FAIL(ret, "No dma devices found!\n");
+
+ ret = rte_dma_info_get(TEST_DMA_DEV_ID, &info);
+ TEST_ASSERT_SUCCESS(ret, "Error with rte_dma_info_get()\n");
+
+ if (info.max_vchans < 1)
+ RTE_LOG(ERR, USER1, "Error, no channels available on device id %u\n",
+ TEST_DMA_DEV_ID);
+
+ if (rte_dma_configure(TEST_DMA_DEV_ID, &conf) != 0)
+ RTE_LOG(ERR, USER1, "Error with rte_dma_configure()\n");
+
+ if (rte_dma_vchan_setup(TEST_DMA_DEV_ID, TEST_DMA_VCHAN_ID, &qconf) < 0)
+ RTE_LOG(ERR, USER1, "Error with vchan configuration\n");
+
+ ret = rte_dma_info_get(TEST_DMA_DEV_ID, &info);
+ if (ret != 0 || info.nb_vchans != 1)
+ RTE_LOG(ERR, USER1, "Error, no configured vhcan reported on device id %u\n",
+ TEST_DMA_DEV_ID);
+
+ params.src_mbuf_pool = rte_pktmbuf_pool_create("DMA_ADAPTER_SRC_MBUFPOOL", NUM_MBUFS,
+ MBUF_CACHE_SIZE, 0, MBUF_SIZE,
+ rte_socket_id());
+ RTE_TEST_ASSERT_NOT_NULL(params.src_mbuf_pool, "Can't create DMA_SRC_MBUFPOOL\n");
+
+ params.dst_mbuf_pool = rte_pktmbuf_pool_create("DMA_ADAPTER_DST_MBUFPOOL", NUM_MBUFS,
+ MBUF_CACHE_SIZE, 0, MBUF_SIZE,
+ rte_socket_id());
+ RTE_TEST_ASSERT_NOT_NULL(params.dst_mbuf_pool, "Can't create DMA_DST_MBUFPOOL\n");
+
+ elt_size = sizeof(struct rte_event_dma_adapter_op) + sizeof(struct rte_event);
+ params.op_mpool = rte_mempool_create("EVENT_DMA_OP_POOL", DMA_OP_POOL_SIZE, elt_size, 0,
+ 0, NULL, NULL, NULL, NULL, rte_socket_id(), 0);
+ RTE_TEST_ASSERT_NOT_NULL(params.op_mpool, "Can't create DMA_OP_POOL\n");
+
+ return TEST_SUCCESS;
+}
+
+static inline void
+evdev_set_conf_values(struct rte_event_dev_config *dev_conf, struct rte_event_dev_info *info)
+{
+ memset(dev_conf, 0, sizeof(struct rte_event_dev_config));
+ dev_conf->dequeue_timeout_ns = info->min_dequeue_timeout_ns;
+ dev_conf->nb_event_ports = NB_TEST_PORTS;
+ dev_conf->nb_event_queues = NB_TEST_QUEUES;
+ dev_conf->nb_event_queue_flows = info->max_event_queue_flows;
+ dev_conf->nb_event_port_dequeue_depth =
+ info->max_event_port_dequeue_depth;
+ dev_conf->nb_event_port_enqueue_depth =
+ info->max_event_port_enqueue_depth;
+ dev_conf->nb_event_port_enqueue_depth =
+ info->max_event_port_enqueue_depth;
+ dev_conf->nb_events_limit =
+ info->max_num_events;
+}
+
+static int
+configure_eventdev(void)
+{
+ struct rte_event_queue_conf queue_conf;
+ struct rte_event_dev_config devconf;
+ struct rte_event_dev_info info;
+ uint32_t queue_count;
+ uint32_t port_count;
+ uint8_t qid;
+ int ret;
+
+ if (!rte_event_dev_count()) {
+ /* If there is no hardware eventdev, or no software vdev was
+ * specified on the command line, create an instance of
+ * event_sw.
+ */
+ LOG_DBG("Failed to find a valid event device... "
+ "testing with event_sw device\n");
+ TEST_ASSERT_SUCCESS(rte_vdev_init("event_sw0", NULL),
+ "Error creating eventdev");
+ evdev = rte_event_dev_get_dev_id("event_sw0");
+ }
+
+ ret = rte_event_dev_info_get(evdev, &info);
+ TEST_ASSERT_SUCCESS(ret, "Failed to get event dev info\n");
+
+ evdev_set_conf_values(&devconf, &info);
+
+ ret = rte_event_dev_configure(evdev, &devconf);
+ TEST_ASSERT_SUCCESS(ret, "Failed to configure eventdev\n");
+
+ /* Set up event queue */
+ ret = rte_event_dev_attr_get(evdev, RTE_EVENT_DEV_ATTR_QUEUE_COUNT, &queue_count);
+ TEST_ASSERT_SUCCESS(ret, "Queue count get failed\n");
+ TEST_ASSERT_EQUAL(queue_count, 2, "Unexpected queue count\n");
+
+ qid = TEST_APP_EV_QUEUE_ID;
+ ret = rte_event_queue_setup(evdev, qid, NULL);
+ TEST_ASSERT_SUCCESS(ret, "Failed to setup queue=%d\n", qid);
+
+ queue_conf.nb_atomic_flows = info.max_event_queue_flows;
+ queue_conf.nb_atomic_order_sequences = 32;
+ queue_conf.schedule_type = RTE_SCHED_TYPE_ATOMIC;
+ queue_conf.priority = RTE_EVENT_DEV_PRIORITY_HIGHEST;
+ queue_conf.event_queue_cfg = RTE_EVENT_QUEUE_CFG_SINGLE_LINK;
+
+ qid = TEST_DMA_EV_QUEUE_ID;
+ ret = rte_event_queue_setup(evdev, qid, &queue_conf);
+ TEST_ASSERT_SUCCESS(ret, "Failed to setup queue=%u\n", qid);
+
+ /* Set up event port */
+ ret = rte_event_dev_attr_get(evdev, RTE_EVENT_DEV_ATTR_PORT_COUNT,
+ &port_count);
+ TEST_ASSERT_SUCCESS(ret, "Port count get failed\n");
+ TEST_ASSERT_EQUAL(port_count, 1, "Unexpected port count\n");
+
+ ret = rte_event_port_setup(evdev, TEST_APP_PORT_ID, NULL);
+ TEST_ASSERT_SUCCESS(ret, "Failed to setup port=%d\n",
+ TEST_APP_PORT_ID);
+
+ qid = TEST_APP_EV_QUEUE_ID;
+ ret = rte_event_port_link(evdev, TEST_APP_PORT_ID, &qid, NULL, 1);
+ TEST_ASSERT(ret >= 0, "Failed to link queue port=%d\n",
+ TEST_APP_PORT_ID);
+
+ return TEST_SUCCESS;
+}
+
+static void
+test_dma_adapter_free(void)
+{
+ rte_event_dma_adapter_free(TEST_ADAPTER_ID);
+}
+
+static int
+test_dma_adapter_create(void)
+{
+ struct rte_event_dev_info evdev_info = {0};
+ struct rte_event_port_conf conf = {0};
+ int ret;
+
+ ret = rte_event_dev_info_get(evdev, &evdev_info);
+ TEST_ASSERT_SUCCESS(ret, "Failed to create event dma adapter\n");
+
+ conf.new_event_threshold = evdev_info.max_num_events;
+ conf.dequeue_depth = evdev_info.max_event_port_dequeue_depth;
+ conf.enqueue_depth = evdev_info.max_event_port_enqueue_depth;
+
+ /* Create adapter with default port creation callback */
+ ret = rte_event_dma_adapter_create(TEST_ADAPTER_ID, evdev, &conf, 0);
+ TEST_ASSERT_SUCCESS(ret, "Failed to create event dma adapter\n");
+
+ return TEST_SUCCESS;
+}
+
+static int
+test_dma_adapter_vchan_add_del(void)
+{
+ struct rte_event event;
+ uint32_t cap;
+ int ret;
+
+ ret = rte_event_dma_adapter_caps_get(evdev, TEST_DMA_DEV_ID, &cap);
+ TEST_ASSERT_SUCCESS(ret, "Failed to get adapter capabilities\n");
+
+ if (cap & RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_VCHAN_EV_BIND) {
+ ret = rte_event_dma_adapter_vchan_add(TEST_ADAPTER_ID, TEST_DMA_DEV_ID,
+ TEST_DMA_VCHAN_ID, &event);
+ } else
+ ret = rte_event_dma_adapter_vchan_add(TEST_ADAPTER_ID, TEST_DMA_DEV_ID,
+ TEST_DMA_VCHAN_ID, NULL);
+
+ TEST_ASSERT_SUCCESS(ret, "Failed to create add vchan\n");
+
+ ret = rte_event_dma_adapter_vchan_del(TEST_ADAPTER_ID, TEST_DMA_DEV_ID,
+ TEST_DMA_VCHAN_ID);
+ TEST_ASSERT_SUCCESS(ret, "Failed to delete vchan\n");
+
+ return TEST_SUCCESS;
+}
+
+static int
+configure_event_dma_adapter(enum rte_event_dma_adapter_mode mode)
+{
+ struct rte_event_dev_info evdev_info = {0};
+ struct rte_event_port_conf conf = {0};
+ struct rte_event event;
+ uint32_t cap;
+ int ret;
+
+ ret = rte_event_dma_adapter_caps_get(evdev, TEST_DMA_DEV_ID, &cap);
+ TEST_ASSERT_SUCCESS(ret, "Failed to get adapter capabilities\n");
+
+ /* Skip mode and capability mismatch check for SW eventdev */
+ if (!(cap & RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP_NEW) &&
+ !(cap & RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP_FWD) &&
+ !(cap & RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_VCHAN_EV_BIND))
+ goto adapter_create;
+
+ if (mode == RTE_EVENT_DMA_ADAPTER_OP_FORWARD) {
+ if (cap & RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP_FWD)
+ params.internal_port_op_fwd = 1;
+ else
+ return -ENOTSUP;
+ }
+
+adapter_create:
+ ret = rte_event_dev_info_get(evdev, &evdev_info);
+ TEST_ASSERT_SUCCESS(ret, "Failed to create event dma adapter\n");
+
+ conf.new_event_threshold = evdev_info.max_num_events;
+ conf.dequeue_depth = evdev_info.max_event_port_dequeue_depth;
+ conf.enqueue_depth = evdev_info.max_event_port_enqueue_depth;
+
+ /* Create adapter with default port creation callback */
+ ret = rte_event_dma_adapter_create(TEST_ADAPTER_ID, evdev, &conf, mode);
+ TEST_ASSERT_SUCCESS(ret, "Failed to create event dma adapter\n");
+
+ if (cap & RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_VCHAN_EV_BIND) {
+ ret = rte_event_dma_adapter_vchan_add(TEST_ADAPTER_ID, TEST_DMA_DEV_ID,
+ TEST_DMA_VCHAN_ID, &event);
+ } else
+ ret = rte_event_dma_adapter_vchan_add(TEST_ADAPTER_ID, TEST_DMA_DEV_ID,
+ TEST_DMA_VCHAN_ID, NULL);
+
+ TEST_ASSERT_SUCCESS(ret, "Failed to add vchan\n");
+
+ if (!params.internal_port_op_fwd) {
+ ret = rte_event_dma_adapter_event_port_get(TEST_ADAPTER_ID,
+ ¶ms.dma_event_port_id);
+ TEST_ASSERT_SUCCESS(ret, "Failed to get event port\n");
+ }
+
+ return TEST_SUCCESS;
+}
+
+static void
+test_dma_adapter_stop(void)
+{
+ uint32_t evdev_service_id, adapter_service_id;
+
+ /* retrieve service ids & stop services */
+ if (rte_event_dma_adapter_service_id_get(TEST_ADAPTER_ID,
+ &adapter_service_id) == 0) {
+ rte_service_runstate_set(adapter_service_id, 0);
+ rte_service_lcore_stop(slcore_id);
+ rte_service_lcore_del(slcore_id);
+ rte_event_dma_adapter_stop(TEST_ADAPTER_ID);
+ }
+
+ if (rte_event_dev_service_id_get(evdev, &evdev_service_id) == 0) {
+ rte_service_runstate_set(evdev_service_id, 0);
+ rte_service_lcore_stop(slcore_id);
+ rte_service_lcore_del(slcore_id);
+ rte_dma_stop(TEST_DMA_DEV_ID);
+ rte_event_dev_stop(evdev);
+ } else {
+ rte_dma_stop(TEST_DMA_DEV_ID);
+ rte_event_dev_stop(evdev);
+ }
+}
+
+static int
+test_dma_adapter_conf(enum rte_event_dma_adapter_mode mode)
+{
+ uint32_t evdev_service_id;
+ uint8_t qid;
+ int ret;
+
+ if (!dma_adapter_setup_done) {
+ ret = configure_event_dma_adapter(mode);
+ if (ret)
+ return ret;
+ if (!params.internal_port_op_fwd) {
+ qid = TEST_DMA_EV_QUEUE_ID;
+ ret = rte_event_port_link(evdev,
+ params.dma_event_port_id, &qid, NULL, 1);
+ TEST_ASSERT(ret >= 0, "Failed to link queue %d "
+ "port=%u\n", qid,
+ params.dma_event_port_id);
+ }
+ dma_adapter_setup_done = 1;
+ }
+
+ /* retrieve service ids */
+ if (rte_event_dev_service_id_get(evdev, &evdev_service_id) == 0) {
+ /* add a service core and start it */
+ TEST_ASSERT_SUCCESS(rte_service_lcore_add(slcore_id),
+ "Failed to add service core");
+ TEST_ASSERT_SUCCESS(rte_service_lcore_start(slcore_id),
+ "Failed to start service core");
+
+ /* map services to it */
+ TEST_ASSERT_SUCCESS(rte_service_map_lcore_set(evdev_service_id,
+ slcore_id, 1), "Failed to map evdev service");
+
+ /* set services to running */
+ TEST_ASSERT_SUCCESS(rte_service_runstate_set(evdev_service_id,
+ 1), "Failed to start evdev service");
+ }
+
+ /* start the eventdev */
+ TEST_ASSERT_SUCCESS(rte_event_dev_start(evdev),
+ "Failed to start event device");
+
+ /* start the dma dev */
+ TEST_ASSERT_SUCCESS(rte_dma_start(TEST_DMA_DEV_ID),
+ "Failed to start dma device");
+
+ return TEST_SUCCESS;
+}
+
+static int
+test_dma_adapter_conf_op_forward_mode(void)
+{
+ enum rte_event_dma_adapter_mode mode;
+
+ mode = RTE_EVENT_DMA_ADAPTER_OP_FORWARD;
+
+ return test_dma_adapter_conf(mode);
+}
+
+static int
+testsuite_setup(void)
+{
+ int ret;
+
+ slcore_id = rte_get_next_lcore(-1, 1, 0);
+ TEST_ASSERT_NOT_EQUAL(slcore_id, RTE_MAX_LCORE, "At least 2 lcores "
+ "are required to run this autotest\n");
+
+ /* Setup and start event device. */
+ ret = configure_eventdev();
+ TEST_ASSERT_SUCCESS(ret, "Failed to setup eventdev\n");
+
+ /* Setup and start dma device. */
+ ret = configure_dmadev();
+ TEST_ASSERT_SUCCESS(ret, "dmadev initialization failed\n");
+
+ return TEST_SUCCESS;
+}
+
+static void
+dma_adapter_teardown(void)
+{
+ int ret;
+
+ ret = rte_event_dma_adapter_stop(TEST_ADAPTER_ID);
+ if (ret < 0)
+ RTE_LOG(ERR, USER1, "Failed to stop adapter!");
+
+ ret = rte_event_dma_adapter_vchan_del(TEST_ADAPTER_ID, TEST_DMA_DEV_ID,
+ TEST_DMA_VCHAN_ID);
+ if (ret < 0)
+ RTE_LOG(ERR, USER1, "Failed to delete vchan!");
+
+ ret = rte_event_dma_adapter_free(TEST_ADAPTER_ID);
+ if (ret < 0)
+ RTE_LOG(ERR, USER1, "Failed to free adapter!");
+
+ dma_adapter_setup_done = 0;
+}
+
+static void
+dma_teardown(void)
+{
+ /* Free mbuf mempool */
+ if (params.src_mbuf_pool != NULL) {
+ RTE_LOG(DEBUG, USER1, "DMA_ADAPTER_SRC_MBUFPOOL count %u\n",
+ rte_mempool_avail_count(params.src_mbuf_pool));
+ rte_mempool_free(params.src_mbuf_pool);
+ params.src_mbuf_pool = NULL;
+ }
+
+ if (params.dst_mbuf_pool != NULL) {
+ RTE_LOG(DEBUG, USER1, "DMA_ADAPTER_DST_MBUFPOOL count %u\n",
+ rte_mempool_avail_count(params.dst_mbuf_pool));
+ rte_mempool_free(params.dst_mbuf_pool);
+ params.dst_mbuf_pool = NULL;
+ }
+
+ /* Free ops mempool */
+ if (params.op_mpool != NULL) {
+ RTE_LOG(DEBUG, USER1, "EVENT_DMA_OP_POOL count %u\n",
+ rte_mempool_avail_count(params.op_mpool));
+ rte_mempool_free(params.op_mpool);
+ params.op_mpool = NULL;
+ }
+}
+
+static void
+eventdev_teardown(void)
+{
+ rte_event_dev_stop(evdev);
+}
+
+static void
+testsuite_teardown(void)
+{
+ dma_adapter_teardown();
+ dma_teardown();
+ eventdev_teardown();
+}
+
+static struct unit_test_suite functional_testsuite = {
+ .suite_name = "Event dma adapter test suite",
+ .setup = testsuite_setup,
+ .teardown = testsuite_teardown,
+ .unit_test_cases = {
+
+ TEST_CASE_ST(NULL, test_dma_adapter_free, test_dma_adapter_create),
+
+ TEST_CASE_ST(test_dma_adapter_create, test_dma_adapter_free,
+ test_dma_adapter_vchan_add_del),
+
+ TEST_CASE_ST(test_dma_adapter_create, test_dma_adapter_free,
+ test_dma_adapter_stats),
+
+ TEST_CASE_ST(test_dma_adapter_create, test_dma_adapter_free,
+ test_dma_adapter_params),
+
+ TEST_CASE_ST(test_dma_adapter_conf_op_forward_mode, test_dma_adapter_stop,
+ test_with_op_forward_mode),
+
+ TEST_CASES_END() /**< NULL terminate unit test array */
+ }
+};
+
+static int
+test_event_dma_adapter(void)
+{
+ return unit_test_suite_runner(&functional_testsuite);
+}
+
+#endif /* !RTE_EXEC_ENV_WINDOWS */
+
+REGISTER_TEST_COMMAND(event_dma_adapter_autotest, test_event_dma_adapter);
--
2.25.1
^ permalink raw reply [flat|nested] 110+ messages in thread
* Re: [PATCH v2 00/12] event DMA adapter library support
2023-09-22 20:13 ` [PATCH v2 00/12] event DMA adapter library support Amit Prakash Shukla
` (12 preceding siblings ...)
2023-09-23 13:34 ` [PATCH v3 00/12] event DMA adapter library support Amit Prakash Shukla
@ 2023-09-26 5:06 ` Jerin Jacob
13 siblings, 0 replies; 110+ messages in thread
From: Jerin Jacob @ 2023-09-26 5:06 UTC (permalink / raw)
To: Amit Prakash Shukla
Cc: dev, jerinj, fengchengwen, kevin.laatz, bruce.richardson,
conor.walsh, vattunuru, g.singh, sachin.saxena, hemant.agrawal,
cheng1.jiang, ndabilpuram, anoobj, mb
On Sat, Sep 23, 2023 at 1:43 AM Amit Prakash Shukla
<amitprakashs@marvell.com> wrote:
>
> This series adds support for event DMA adapter library. API's defined
> as part of this library can be used by the application for DMA transfer
> of data using event based mechanism.
>
> v2:
> - Resolved review comments.
> - Patch split into multiple patches.
>
Fix build issue report at
http://mails.dpdk.org/archives/test-report/2023-September/465071.html
^ permalink raw reply [flat|nested] 110+ messages in thread
* [PATCH v4 00/12] event DMA adapter library support
2023-09-23 13:34 ` [PATCH v3 00/12] event DMA adapter library support Amit Prakash Shukla
` (11 preceding siblings ...)
2023-09-23 13:34 ` [PATCH v3 12/12] app/test: add event DMA adapter auto-test Amit Prakash Shukla
@ 2023-09-26 10:32 ` Amit Prakash Shukla
2023-09-26 10:32 ` [PATCH v4 01/12] eventdev: introduce event DMA adapter library Amit Prakash Shukla
` (12 more replies)
12 siblings, 13 replies; 110+ messages in thread
From: Amit Prakash Shukla @ 2023-09-26 10:32 UTC (permalink / raw)
Cc: dev, jerinj, fengchengwen, kevin.laatz, bruce.richardson,
conor.walsh, vattunuru, g.singh, sachin.saxena, hemant.agrawal,
cheng1.jiang, ndabilpuram, anoobj, mb, Amit Prakash Shukla
This series adds support for event DMA adapter library. API's defined
as part of this library can be used by the application for DMA transfer
of data using event based mechanism.
v4:
- Fixed compilation error.
v3:
- Resolved checkpatch warnings.
- Fixed compilation error on intel.
- Updated git commit message.
v2:
- Resolved review comments.
- Patch split into multiple patches.
Amit Prakash Shukla (12):
eventdev: introduce event DMA adapter library
eventdev: api to get DMA adapter capabilities
eventdev: create and free API for DMA adapter
eventdev: add API support for vchan add and delete
eventdev: add support for DMA adapter service function
eventdev: add support for DMA adapter start and stop
eventdev: add support for DMA adapter service ID get
eventdev: add DMA adapter support for runtime params
eventdev: add support for DMA adapter stats
eventdev: add support for DMA adapter enqueue
eventdev: add DMA adapter event port get
app/test: add event DMA adapter auto-test
MAINTAINERS | 5 +
app/test/meson.build | 1 +
app/test/test_event_dma_adapter.c | 805 ++++++++++
config/rte_config.h | 1 +
doc/api/doxy-api-index.md | 1 +
doc/guides/eventdevs/features/default.ini | 8 +
doc/guides/prog_guide/event_dma_adapter.rst | 264 +++
doc/guides/prog_guide/eventdev.rst | 8 +-
.../img/event_dma_adapter_op_forward.svg | 1086 +++++++++++++
.../img/event_dma_adapter_op_new.svg | 1079 +++++++++++++
doc/guides/prog_guide/index.rst | 1 +
doc/guides/rel_notes/release_23_11.rst | 4 +-
lib/eventdev/eventdev_pmd.h | 175 +-
lib/eventdev/eventdev_private.c | 10 +
lib/eventdev/meson.build | 4 +-
lib/eventdev/rte_event_dma_adapter.c | 1431 +++++++++++++++++
lib/eventdev/rte_event_dma_adapter.h | 582 +++++++
lib/eventdev/rte_eventdev.c | 23 +
lib/eventdev/rte_eventdev.h | 44 +
lib/eventdev/rte_eventdev_core.h | 8 +-
lib/eventdev/version.map | 16 +
lib/meson.build | 2 +-
22 files changed, 5550 insertions(+), 8 deletions(-)
create mode 100644 app/test/test_event_dma_adapter.c
create mode 100644 doc/guides/prog_guide/event_dma_adapter.rst
create mode 100644 doc/guides/prog_guide/img/event_dma_adapter_op_forward.svg
create mode 100644 doc/guides/prog_guide/img/event_dma_adapter_op_new.svg
create mode 100644 lib/eventdev/rte_event_dma_adapter.c
create mode 100644 lib/eventdev/rte_event_dma_adapter.h
--
2.25.1
^ permalink raw reply [flat|nested] 110+ messages in thread
* [PATCH v4 01/12] eventdev: introduce event DMA adapter library
2023-09-26 10:32 ` [PATCH v4 00/12] event DMA adapter library support Amit Prakash Shukla
@ 2023-09-26 10:32 ` Amit Prakash Shukla
2023-09-27 18:12 ` Jerin Jacob
2023-09-26 10:32 ` [PATCH v4 02/12] eventdev: api to get DMA adapter capabilities Amit Prakash Shukla
` (11 subsequent siblings)
12 siblings, 1 reply; 110+ messages in thread
From: Amit Prakash Shukla @ 2023-09-26 10:32 UTC (permalink / raw)
To: Thomas Monjalon, Amit Prakash Shukla, Jerin Jacob
Cc: dev, fengchengwen, kevin.laatz, bruce.richardson, conor.walsh,
vattunuru, g.singh, sachin.saxena, hemant.agrawal, cheng1.jiang,
ndabilpuram, anoobj, mb
Introduce event DMA adapter APIs. The change provides information
on adapter modes and usage. Application can use this event adapter
interface to transfer packets between DMA device and event device.
Signed-off-by: Amit Prakash Shukla <amitprakashs@marvell.com>
---
MAINTAINERS | 5 +
doc/api/doxy-api-index.md | 1 +
doc/guides/eventdevs/features/default.ini | 8 +
doc/guides/prog_guide/event_dma_adapter.rst | 264 ++++
doc/guides/prog_guide/eventdev.rst | 8 +-
.../img/event_dma_adapter_op_forward.svg | 1086 +++++++++++++++++
.../img/event_dma_adapter_op_new.svg | 1079 ++++++++++++++++
doc/guides/prog_guide/index.rst | 1 +
doc/guides/rel_notes/release_23_11.rst | 4 +-
lib/eventdev/eventdev_pmd.h | 175 ++-
lib/eventdev/eventdev_private.c | 10 +
lib/eventdev/meson.build | 1 +
lib/eventdev/rte_event_dma_adapter.h | 582 +++++++++
lib/eventdev/rte_eventdev.h | 44 +
lib/eventdev/rte_eventdev_core.h | 8 +-
lib/eventdev/version.map | 16 +
16 files changed, 3286 insertions(+), 6 deletions(-)
create mode 100644 doc/guides/prog_guide/event_dma_adapter.rst
create mode 100644 doc/guides/prog_guide/img/event_dma_adapter_op_forward.svg
create mode 100644 doc/guides/prog_guide/img/event_dma_adapter_op_new.svg
create mode 100644 lib/eventdev/rte_event_dma_adapter.h
diff --git a/MAINTAINERS b/MAINTAINERS
index a926155f26..a6b8fc88d0 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -548,6 +548,11 @@ F: drivers/raw/skeleton/
F: app/test/test_rawdev.c
F: doc/guides/prog_guide/rawdev.rst
+Eventdev DMA Adapter API
+M: Amit Prakash Shukla <amitprakashs@marvell.com>
+T: git://dpdk.org/next/dpdk-next-eventdev
+F: lib/eventdev/*dma_adapter*
+F: doc/guides/prog_guide/event_dma_adapter.rst
Memory Pool Drivers
-------------------
diff --git a/doc/api/doxy-api-index.md b/doc/api/doxy-api-index.md
index fdeda13932..b7df7be4d9 100644
--- a/doc/api/doxy-api-index.md
+++ b/doc/api/doxy-api-index.md
@@ -29,6 +29,7 @@ The public API headers are grouped by topics:
[event_eth_tx_adapter](@ref rte_event_eth_tx_adapter.h),
[event_timer_adapter](@ref rte_event_timer_adapter.h),
[event_crypto_adapter](@ref rte_event_crypto_adapter.h),
+ [event_dma_adapter](@ref rte_event_dma_adapter.h),
[rawdev](@ref rte_rawdev.h),
[metrics](@ref rte_metrics.h),
[bitrate](@ref rte_bitrate.h),
diff --git a/doc/guides/eventdevs/features/default.ini b/doc/guides/eventdevs/features/default.ini
index 00360f60c6..73a52d915b 100644
--- a/doc/guides/eventdevs/features/default.ini
+++ b/doc/guides/eventdevs/features/default.ini
@@ -44,6 +44,14 @@ internal_port_op_fwd =
internal_port_qp_ev_bind =
session_private_data =
+;
+; Features of a default DMA adapter.
+;
+[DMA adapter Features]
+internal_port_op_new =
+internal_port_op_fwd =
+internal_port_vchan_ev_bind =
+
;
; Features of a default Timer adapter.
;
diff --git a/doc/guides/prog_guide/event_dma_adapter.rst b/doc/guides/prog_guide/event_dma_adapter.rst
new file mode 100644
index 0000000000..eeb9ce6dfd
--- /dev/null
+++ b/doc/guides/prog_guide/event_dma_adapter.rst
@@ -0,0 +1,264 @@
+.. SPDX-License-Identifier: BSD-3-Clause
+ Copyright (c) 2023 Marvell.
+
+Event DMA Adapter Library
+=========================
+
+DPDK :doc:`Eventdev library <eventdev>` provides event driven programming model with features
+to schedule events. :doc:`DMA Device library <dmadev>` provides an interface to DMA poll mode
+drivers that support DMA operations. Event DMA Adapter is intended to bridge between the event
+device and the DMA device.
+
+Packet flow from DMA device to the event device can be accomplished using software and hardware
+based transfer mechanisms. The adapter queries an eventdev PMD to determine which mechanism to
+be used. The adapter uses an EAL service core function for software based packet transfer and
+uses the eventdev PMD functions to configure hardware based packet transfer between DMA device
+and the event device. DMA adapter uses a new event type called ``RTE_EVENT_TYPE_DMADEV`` to
+indicate the source of event.
+
+Application can choose to submit an DMA operation directly to an DMA device or send it to an DMA
+adapter via eventdev based on RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP_FWD capability. The
+first mode is known as the event new (RTE_EVENT_DMA_ADAPTER_OP_NEW) mode and the second as the
+event forward (RTE_EVENT_DMA_ADAPTER_OP_FORWARD) mode. Choice of mode can be specified while
+creating the adapter. In the former mode, it is the application's responsibility to enable
+ingress packet ordering. In the latter mode, it is the adapter's responsibility to enable
+ingress packet ordering.
+
+
+Adapter Modes
+-------------
+
+RTE_EVENT_DMA_ADAPTER_OP_NEW mode
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+In the RTE_EVENT_DMA_ADAPTER_OP_NEW mode, application submits DMA operations directly to an DMA
+device. The adapter then dequeues DMA completions from the DMA device and enqueues them as events
+to the event device. This mode does not ensure ingress ordering as the application directly
+enqueues to the dmadev without going through DMA/atomic stage. In this mode, events dequeued
+from the adapter are treated as new events. The application has to specify event information
+(response information) which is needed to enqueue an event after the DMA operation is completed.
+
+.. _figure_event_dma_adapter_op_new:
+
+.. figure:: img/event_dma_adapter_op_new.*
+
+ Working model of ``RTE_EVENT_DMA_ADAPTER_OP_NEW`` mode
+
+
+RTE_EVENT_DMA_ADAPTER_OP_FORWARD mode
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+In the ``RTE_EVENT_DMA_ADAPTER_OP_FORWARD`` mode, if the event PMD and DMA PMD supports internal
+event port (``RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP_FWD``), the application should use
+``rte_event_dma_adapter_enqueue()`` API to enqueue DMA operations as events to DMA adapter. If
+not, application retrieves DMA adapter's event port using ``rte_event_dma_adapter_event_port_get()``
+API, links its event queue to this port and starts enqueuing DMA operations as events to eventdev
+using ``rte_event_enqueue_burst()``. The adapter then dequeues the events and submits the DMA
+operations to the dmadev. After the DMA operation is complete, the adapter enqueues events to the
+event device.
+
+Applications can use this mode when ingress packet ordering is needed. In this mode, events
+dequeued from the adapter will be treated as forwarded events. Application has to specify event
+information (response information) needed to enqueue the event after the DMA operation has
+completed.
+
+.. _figure_event_dma_adapter_op_forward:
+
+.. figure:: img/event_dma_adapter_op_forward.*
+
+ Working model of ``RTE_EVENT_DMA_ADAPTER_OP_FORWARD`` mode
+
+
+API Overview
+------------
+
+This section has a brief introduction to the event DMA adapter APIs. The application is expected
+to create an adapter which is associated with a single eventdev, then add dmadev and vchan to the
+adapter instance.
+
+
+Create an adapter instance
+~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+An adapter instance is created using ``rte_event_dma_adapter_create()``. This function is called
+with event device to be associated with the adapter and port configuration for the adapter to
+setup an event port (if the adapter needs to use a service function).
+
+Adapter can be started in ``RTE_EVENT_DMA_ADAPTER_OP_NEW`` or ``RTE_EVENT_DMA_ADAPTER_OP_FORWARD``
+mode.
+
+.. code-block:: c
+
+ enum rte_event_dma_adapter_mode mode;
+ struct rte_event_dev_info dev_info;
+ struct rte_event_port_conf conf;
+ uint8_t evdev_id;
+ uint8_t dma_id;
+ int ret;
+
+ ret = rte_event_dev_info_get(dma_id, &dev_info);
+
+ conf.new_event_threshold = dev_info.max_num_events;
+ conf.dequeue_depth = dev_info.max_event_port_dequeue_depth;
+ conf.enqueue_depth = dev_info.max_event_port_enqueue_depth;
+ mode = RTE_EVENT_DMA_ADAPTER_OP_FORWARD;
+ ret = rte_event_dma_adapter_create(dma_id, evdev_id, &conf, mode);
+
+
+``rte_event_dma_adapter_create_ext()`` function can be used by the application to have a finer
+control on eventdev port allocation and setup. The ``rte_event_dma_adapter_create_ext()``
+function is passed a callback function. The callback function is invoked if the adapter creates
+a service function and uses an event port for it. The callback is expected to fill the
+``struct rte_event_dma_adapter_conf`` structure passed to it.
+
+In the ``RTE_EVENT_DMA_ADAPTER_OP_FORWARD`` mode, if the event PMD and DMA PMD supports internal
+event port (``RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP_FWD``), events with DMA operations should
+be enqueued to the DMA adapter using ``rte_event_dma_adapter_enqueue()`` API. If not, the event port
+created by the adapter can be retrieved using ``rte_event_dma_adapter_event_port_get()`` API. An
+application can use this event port to link with an event queue, on which it enqueues events
+towards the DMA adapter using ``rte_event_enqueue_burst()``.
+
+.. code-block:: c
+
+ uint8_t dma_adpt_id, evdev_id, dma_dev_id, dma_ev_port_id, app_qid;
+ struct rte_event ev;
+ uint32_t cap;
+ int ret;
+
+ // Fill in event info and update event_ptr with rte_dma_op
+ memset(&ev, 0, sizeof(ev));
+ .
+ .
+ ev.event_ptr = op;
+
+ ret = rte_event_dma_adapter_caps_get(evdev_id, dma_dev_id, &cap);
+ if (cap & RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP_FWD) {
+ ret = rte_event_dma_adapter_enqueue(evdev_id, app_ev_port_id, ev, nb_events);
+ } else {
+ ret = rte_event_dma_adapter_event_port_get(dma_adpt_id, &dma_ev_port_id);
+ ret = rte_event_queue_setup(evdev_id, app_qid, NULL);
+ ret = rte_event_port_link(evdev_id, dma_ev_port_id, &app_qid, NULL, 1);
+ ev.queue_id = app_qid;
+ ret = rte_event_enqueue_burst(evdev_id, app_ev_port_id, ev, nb_events);
+ }
+
+
+Event device configuration for service based adapter
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+When ``rte_event_dma_adapter_create()`` is used for creating adapter instance,
+``rte_event_dev_config::nb_event_ports`` is automatically incremented, and event device is
+reconfigured with additional event port during service initialization. This event device
+reconfigure logic also increments the ``rte_event_dev_config::nb_single_link_event_port_queues``
+parameter if the adapter event port config is of type ``RTE_EVENT_PORT_CFG_SINGLE_LINK``.
+
+Applications using this mode of adapter creation need not configure the event device with
+``rte_event_dev_config::nb_event_ports`` and
+``rte_event_dev_config::nb_single_link_event_port_queues`` parameters required for DMA adapter when
+the adapter is created using the above-mentioned API.
+
+
+Querying adapter capabilities
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+The ``rte_event_dma_adapter_caps_get()`` function allows the application to query the adapter
+capabilities for an eventdev and dmadev combination. This API provides whether dmadev and eventdev
+are connected using internal HW port or not.
+
+.. code-block:: c
+
+ rte_event_dma_adapter_caps_get(dev_id, dma_dev_id, &cap);
+
+
+Adding vchan to the adapter instance
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+dmadev device id and vchan are configured using dmadev APIs. For more information
+see :doc:`here <dmadev>`.
+
+.. code-block:: c
+
+ struct rte_dma_vchan_conf vchan_conf;
+ struct rte_dma_conf dev_conf;
+ uint8_t dev_id = 0;
+ uint16_t vchan = 0;
+
+ rte_dma_configure(dev_id, &dev_conf);
+ rte_dma_vchan_setup(dev_id, vhcan, &vchan_conf);
+
+These dmadev id and vchan are added to the instance using the
+``rte_event_dma_adapter_vchan_add()`` API. The same is removed using
+``rte_event_dma_adapter_vchan_del()`` API. If hardware supports
+``RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_VCHAN_EV_BIND`` capability, event information must be passed to the add API.
+
+.. code-block:: c
+
+ uint32_t cap;
+ int ret;
+
+ ret = rte_event_dma_adapter_caps_get(evdev_id, dma_dev_id, &cap);
+ if (cap & RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_VCHAN_EV_BIND) {
+ struct rte_event event;
+
+ rte_event_dma_adapter_vchan_add(id, dma_dev_id, vchan, &conf);
+ } else
+ rte_event_dma_adapter_vchan_add(id, dma_dev_id, vchan, NULL);
+
+
+Configuring service function
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+If the adapter uses a service function, the application is required to assign a service core to
+the service function as show below.
+
+.. code-block:: c
+
+ uint32_t service_id;
+
+ if (rte_event_dma_adapter_service_id_get(dma_id, &service_id) == 0)
+ rte_service_map_lcore_set(service_id, CORE_ID);
+
+
+Set event response information
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+In the RTE_EVENT_DMA_ADAPTER_OP_FORWARD / RTE_EVENT_DMA_ADAPTER_OP_NEW mode, the application
+specifies the dmadev ID and vchan ID in ``struct rte_event_dma_adapter_op`` and the event
+information (response information) needed to enqueue an event after the DMA operation has
+completed. The response information is specified in ``struct rte_event`` and appended to the
+``struct rte_event_dma_adapter_op``.
+
+
+Start the adapter instance
+~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+The application calls ``rte_event_dma_adapter_start()`` to start the adapter. This function calls
+the start callbacks of the eventdev PMDs for hardware based eventdev-dmadev connections and
+``rte_service_run_state_set()`` to enable the service function if one exists.
+
+.. code-block:: c
+
+ rte_event_dma_adapter_start(id);
+
+.. Note::
+
+ The eventdev to which the event_dma_adapter is connected should be started before calling
+ rte_event_dma_adapter_start().
+
+
+Get adapter statistics
+~~~~~~~~~~~~~~~~~~~~~~
+
+The ``rte_event_dma_adapter_stats_get()`` function reports counters defined in struct
+``rte_event_dma_adapter_stats``. The received packet and enqueued event counts are a sum of the
+counts from the eventdev PMD callbacks if the callback is supported, and the counts maintained by
+the service function, if one exists.
+
+Set/Get adapter runtime configuration parameters
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+The runtime configuration parameters of adapter can be set/get using
+``rte_event_dma_adapter_runtime_params_set()`` and
+``rte_event_dma_adapter_runtime_params_get()`` respectively.
+The parameters that can be set/get are defined in
+``struct rte_event_dma_adapter_runtime_params``.
diff --git a/doc/guides/prog_guide/eventdev.rst b/doc/guides/prog_guide/eventdev.rst
index 2c83176846..ff55115d0d 100644
--- a/doc/guides/prog_guide/eventdev.rst
+++ b/doc/guides/prog_guide/eventdev.rst
@@ -333,7 +333,8 @@ eventdev.
.. Note::
EventDev needs to be started before starting the event producers such
- as event_eth_rx_adapter, event_timer_adapter and event_crypto_adapter.
+ as event_eth_rx_adapter, event_timer_adapter, event_crypto_adapter and
+ event_dma_adapter.
Ingress of New Events
~~~~~~~~~~~~~~~~~~~~~
@@ -445,8 +446,9 @@ using ``rte_event_dev_stop_flush_callback_register()`` function.
.. Note::
The event producers such as ``event_eth_rx_adapter``,
- ``event_timer_adapter`` and ``event_crypto_adapter``
- need to be stopped before stopping the event device.
+ ``event_timer_adapter``, ``event_crypto_adapter`` and
+ ``event_dma_adapter`` need to be stopped before stopping
+ the event device.
Summary
-------
diff --git a/doc/guides/prog_guide/img/event_dma_adapter_op_forward.svg b/doc/guides/prog_guide/img/event_dma_adapter_op_forward.svg
new file mode 100644
index 0000000000..b7fe1fecf2
--- /dev/null
+++ b/doc/guides/prog_guide/img/event_dma_adapter_op_forward.svg
@@ -0,0 +1,1086 @@
+<?xml version="1.0" encoding="UTF-8" standalone="no"?>
+<!-- SPDX-License-Identifier: BSD-3-Clause -->
+<!-- Copyright (c) 2023 Marvell. -->
+<!-- Created with Inkscape (http://www.inkscape.org/) -->
+
+<svg
+ width="720px"
+ height="486px"
+ id="svg13237"
+ version="1.1"
+ inkscape:version="1.2.1 (9c6d41e410, 2022-07-14)"
+ sodipodi:docname="event_dma_adapter_op_forward.svg"
+ xmlns:inkscape="http://www.inkscape.org/namespaces/inkscape"
+ xmlns:sodipodi="http://sodipodi.sourceforge.net/DTD/sodipodi-0.dtd"
+ xmlns="http://www.w3.org/2000/svg"
+ xmlns:svg="http://www.w3.org/2000/svg"
+ xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#"
+ xmlns:cc="http://creativecommons.org/ns#"
+ xmlns:dc="http://purl.org/dc/elements/1.1/"
+ xmlns:v="http://schemas.microsoft.com/visio/2003/SVGExtensions/">
+ <defs
+ id="defs13239">
+ <marker
+ inkscape:stockid="Arrow1Sstart"
+ orient="auto"
+ refY="0.0"
+ refX="0.0"
+ id="Arrow1Sstart"
+ style="overflow:visible">
+ <path
+ id="path8416"
+ d="M 0.0,0.0 L 5.0,-5.0 L -12.5,0.0 L 5.0,5.0 L 0.0,0.0 z "
+ style="fill-rule:evenodd;stroke:#000000;stroke-width:1.0pt"
+ transform="scale(0.2) translate(6,0)" />
+ </marker>
+ <marker
+ inkscape:stockid="Arrow1Send"
+ orient="auto"
+ refY="0.0"
+ refX="0.0"
+ id="Arrow1Send"
+ style="overflow:visible;">
+ <path
+ id="path8419"
+ d="M 0.0,0.0 L 5.0,-5.0 L -12.5,0.0 L 5.0,5.0 L 0.0,0.0 z "
+ style="fill-rule:evenodd;stroke:#000000;stroke-width:1.0pt;"
+ transform="scale(0.2) rotate(180) translate(6,0)" />
+ </marker>
+ <marker
+ inkscape:stockid="DiamondL"
+ orient="auto"
+ refY="0.0"
+ refX="0.0"
+ id="DiamondL"
+ style="overflow:visible">
+ <path
+ id="path8483"
+ d="M 0,-7.0710768 L -7.0710894,0 L 0,7.0710589 L 7.0710462,0 L 0,-7.0710768 z "
+ style="fill-rule:evenodd;stroke:#000000;stroke-width:1.0pt"
+ transform="scale(0.8)" />
+ </marker>
+ <marker
+ inkscape:stockid="DotL"
+ orient="auto"
+ refY="0.0"
+ refX="0.0"
+ id="DotL"
+ style="overflow:visible">
+ <path
+ id="path8465"
+ d="M -2.5,-1.0 C -2.5,1.7600000 -4.7400000,4.0 -7.5,4.0 C -10.260000,4.0 -12.5,1.7600000 -12.5,-1.0 C -12.5,-3.7600000 -10.260000,-6.0 -7.5,-6.0 C -4.7400000,-6.0 -2.5,-3.7600000 -2.5,-1.0 z "
+ style="fill-rule:evenodd;stroke:#000000;stroke-width:1.0pt"
+ transform="scale(0.8) translate(7.4, 1)" />
+ </marker>
+ <marker
+ inkscape:stockid="SquareL"
+ orient="auto"
+ refY="0.0"
+ refX="0.0"
+ id="SquareL"
+ style="overflow:visible">
+ <path
+ id="path8474"
+ d="M -5.0,-5.0 L -5.0,5.0 L 5.0,5.0 L 5.0,-5.0 L -5.0,-5.0 z "
+ style="fill-rule:evenodd;stroke:#000000;stroke-width:1.0pt"
+ transform="scale(0.8)" />
+ </marker>
+ <marker
+ inkscape:stockid="TriangleOutL"
+ orient="auto"
+ refY="0.0"
+ refX="0.0"
+ id="TriangleOutL"
+ style="overflow:visible">
+ <path
+ id="path8546"
+ d="M 5.77,0.0 L -2.88,5.0 L -2.88,-5.0 L 5.77,0.0 z "
+ style="fill-rule:evenodd;stroke:#000000;stroke-width:1.0pt"
+ transform="scale(0.8)" />
+ </marker>
+ <marker
+ inkscape:stockid="Arrow1Lstart"
+ orient="auto"
+ refY="0.0"
+ refX="0.0"
+ id="Arrow1Lstart"
+ style="overflow:visible">
+ <path
+ id="path8404"
+ d="M 0.0,0.0 L 5.0,-5.0 L -12.5,0.0 L 5.0,5.0 L 0.0,0.0 z "
+ style="fill-rule:evenodd;stroke:#000000;stroke-width:1.0pt"
+ transform="scale(0.8) translate(12.5,0)" />
+ </marker>
+ <marker
+ inkscape:stockid="Arrow1Mend"
+ orient="auto"
+ refY="0.0"
+ refX="0.0"
+ id="Arrow1Mend"
+ style="overflow:visible;">
+ <path
+ id="path8413"
+ d="M 0.0,0.0 L 5.0,-5.0 L -12.5,0.0 L 5.0,5.0 L 0.0,0.0 z "
+ style="fill-rule:evenodd;stroke:#000000;stroke-width:1.0pt;"
+ transform="scale(0.4) rotate(180) translate(10,0)" />
+ </marker>
+ <marker
+ inkscape:stockid="Arrow2Lend"
+ orient="auto"
+ refY="0.0"
+ refX="0.0"
+ id="Arrow2Lend"
+ style="overflow:visible;">
+ <path
+ id="path8425"
+ style="fill-rule:evenodd;stroke-width:0.62500000;stroke-linejoin:round;"
+ d="M 8.7185878,4.0337352 L -2.2072895,0.016013256 L 8.7185884,-4.0017078 C 6.9730900,-1.6296469 6.9831476,1.6157441 8.7185878,4.0337352 z "
+ transform="scale(1.1) rotate(180) translate(1,0)" />
+ </marker>
+ <marker
+ inkscape:stockid="Arrow1Lend"
+ orient="auto"
+ refY="0.0"
+ refX="0.0"
+ id="Arrow1Lend"
+ style="overflow:visible;">
+ <path
+ id="path8407"
+ d="M 0.0,0.0 L 5.0,-5.0 L -12.5,0.0 L 5.0,5.0 L 0.0,0.0 z "
+ style="fill-rule:evenodd;stroke:#000000;stroke-width:1.0pt;"
+ transform="scale(0.8) rotate(180) translate(12.5,0)" />
+ </marker>
+ <filter
+ id="filter_2"
+ color-interpolation-filters="sRGB">
+ <feGaussianBlur
+ stdDeviation="2"
+ id="feGaussianBlur15" />
+ </filter>
+ <filter
+ id="filter_2-3"
+ color-interpolation-filters="sRGB"
+ x="-0.086178862"
+ y="-0.086178862"
+ width="1.1723577"
+ height="1.1723577">
+ <feGaussianBlur
+ stdDeviation="2"
+ id="feGaussianBlur15-1" />
+ </filter>
+ <filter
+ id="filter_2-0"
+ color-interpolation-filters="sRGB">
+ <feGaussianBlur
+ stdDeviation="2"
+ id="feGaussianBlur15-7" />
+ </filter>
+ <filter
+ id="filter_2-0-8"
+ color-interpolation-filters="sRGB">
+ <feGaussianBlur
+ stdDeviation="2"
+ id="feGaussianBlur15-7-7" />
+ </filter>
+ <filter
+ id="filter_2-3-9"
+ color-interpolation-filters="sRGB">
+ <feGaussianBlur
+ stdDeviation="2"
+ id="feGaussianBlur15-1-6" />
+ </filter>
+ <filter
+ id="filter_2-3-6"
+ color-interpolation-filters="sRGB">
+ <feGaussianBlur
+ stdDeviation="2"
+ id="feGaussianBlur15-1-63" />
+ </filter>
+ <filter
+ id="filter_2-3-91"
+ color-interpolation-filters="sRGB"
+ x="-0.086178862"
+ y="-0.086178862"
+ width="1.1723577"
+ height="1.1723577">
+ <feGaussianBlur
+ stdDeviation="2"
+ id="feGaussianBlur15-1-3" />
+ </filter>
+ <marker
+ inkscape:stockid="Arrow1Lend"
+ orient="auto"
+ refY="0"
+ refX="0"
+ id="Arrow1Lend-5"
+ style="overflow:visible">
+ <path
+ inkscape:connector-curvature="0"
+ id="path8407-3"
+ d="M 0,0 5,-5 -12.5,0 5,5 0,0 z"
+ style="fill-rule:evenodd;stroke:#000000;stroke-width:1pt"
+ transform="matrix(-0.8,0,0,-0.8,-10,0)" />
+ </marker>
+ <marker
+ inkscape:stockid="Arrow1Lend"
+ orient="auto"
+ refY="0"
+ refX="0"
+ id="Arrow1Lend-6"
+ style="overflow:visible">
+ <path
+ inkscape:connector-curvature="0"
+ id="path8407-0"
+ d="M 0,0 5,-5 -12.5,0 5,5 0,0 z"
+ style="fill-rule:evenodd;stroke:#000000;stroke-width:1pt"
+ transform="matrix(-0.8,0,0,-0.8,-10,0)" />
+ </marker>
+ <marker
+ inkscape:stockid="Arrow1Lstart"
+ orient="auto"
+ refY="0"
+ refX="0"
+ id="Arrow1Lstart-7"
+ style="overflow:visible">
+ <path
+ inkscape:connector-curvature="0"
+ id="path8404-0"
+ d="M 0,0 5,-5 -12.5,0 5,5 0,0 z"
+ style="fill-rule:evenodd;stroke:#000000;stroke-width:1pt"
+ transform="matrix(0.8,0,0,0.8,10,0)" />
+ </marker>
+ <marker
+ inkscape:stockid="Arrow1Lend"
+ orient="auto"
+ refY="0"
+ refX="0"
+ id="Arrow1Lend-51"
+ style="overflow:visible">
+ <path
+ inkscape:connector-curvature="0"
+ id="path8407-1"
+ d="M 0,0 5,-5 -12.5,0 5,5 0,0 z"
+ style="fill-rule:evenodd;stroke:#000000;stroke-width:1pt"
+ transform="matrix(-0.8,0,0,-0.8,-10,0)" />
+ </marker>
+ <marker
+ inkscape:stockid="Arrow1Lend"
+ orient="auto"
+ refY="0"
+ refX="0"
+ id="Arrow1Lend-3"
+ style="overflow:visible">
+ <path
+ inkscape:connector-curvature="0"
+ id="path8407-6"
+ d="M 0,0 5,-5 -12.5,0 5,5 0,0 z"
+ style="fill-rule:evenodd;stroke:#000000;stroke-width:1pt"
+ transform="matrix(-0.8,0,0,-0.8,-10,0)" />
+ </marker>
+ <marker
+ inkscape:stockid="Arrow1Lend"
+ orient="auto"
+ refY="0"
+ refX="0"
+ id="Arrow1Lend-62"
+ style="overflow:visible">
+ <path
+ inkscape:connector-curvature="0"
+ id="path8407-9"
+ d="M 0,0 5,-5 -12.5,0 5,5 0,0 z"
+ style="fill-rule:evenodd;stroke:#000000;stroke-width:1pt"
+ transform="matrix(-0.8,0,0,-0.8,-10,0)" />
+ </marker>
+ <marker
+ inkscape:stockid="Arrow1Lend"
+ orient="auto"
+ refY="0"
+ refX="0"
+ id="Arrow1Lend-2"
+ style="overflow:visible">
+ <path
+ inkscape:connector-curvature="0"
+ id="path8407-7"
+ d="M 0,0 5,-5 -12.5,0 5,5 0,0 z"
+ style="fill-rule:evenodd;stroke:#000000;stroke-width:1pt"
+ transform="matrix(-0.8,0,0,-0.8,-10,0)" />
+ </marker>
+ <marker
+ inkscape:stockid="Arrow1Lstart"
+ orient="auto"
+ refY="0"
+ refX="0"
+ id="Arrow1Lstart-7-9"
+ style="overflow:visible">
+ <path
+ inkscape:connector-curvature="0"
+ id="path8404-0-3"
+ d="M 0,0 5,-5 -12.5,0 5,5 0,0 z"
+ style="fill-rule:evenodd;stroke:#000000;stroke-width:1pt"
+ transform="matrix(0.8,0,0,0.8,10,0)" />
+ </marker>
+ <filter
+ id="filter_2-3-6-1"
+ color-interpolation-filters="sRGB">
+ <feGaussianBlur
+ stdDeviation="2"
+ id="feGaussianBlur15-1-63-8" />
+ </filter>
+ <filter
+ id="filter_2-3-92"
+ color-interpolation-filters="sRGB"
+ x="-0.086178862"
+ y="-0.086178862"
+ width="1.1723577"
+ height="1.1723577">
+ <feGaussianBlur
+ stdDeviation="2"
+ id="feGaussianBlur15-1-2" />
+ </filter>
+ <filter
+ id="filter_2-3-94"
+ color-interpolation-filters="sRGB"
+ x="-0.086178862"
+ y="-0.086178862"
+ width="1.1723577"
+ height="1.1723577">
+ <feGaussianBlur
+ stdDeviation="2"
+ id="feGaussianBlur15-1-7" />
+ </filter>
+ <marker
+ inkscape:stockid="Arrow1Lstart"
+ orient="auto"
+ refY="0"
+ refX="0"
+ id="Arrow1Lstart-7-6"
+ style="overflow:visible">
+ <path
+ inkscape:connector-curvature="0"
+ id="path8404-0-1"
+ d="M 0,0 5,-5 -12.5,0 5,5 0,0 z"
+ style="fill-rule:evenodd;stroke:#000000;stroke-width:1pt"
+ transform="matrix(0.8,0,0,0.8,10,0)" />
+ </marker>
+ <marker
+ inkscape:stockid="Arrow1Lend"
+ orient="auto"
+ refY="0"
+ refX="0"
+ id="Arrow1Lend-55"
+ style="overflow:visible">
+ <path
+ inkscape:connector-curvature="0"
+ id="path8407-4"
+ d="M 0,0 5,-5 -12.5,0 5,5 0,0 z"
+ style="fill-rule:evenodd;stroke:#000000;stroke-width:1pt"
+ transform="matrix(-0.8,0,0,-0.8,-10,0)" />
+ </marker>
+ </defs>
+ <sodipodi:namedview
+ id="base"
+ pagecolor="#ffffff"
+ bordercolor="#666666"
+ borderopacity="1.0"
+ inkscape:pageopacity="0.0"
+ inkscape:pageshadow="2"
+ inkscape:zoom="1"
+ inkscape:cx="233.5"
+ inkscape:cy="288"
+ inkscape:document-units="px"
+ inkscape:current-layer="layer1"
+ showgrid="false"
+ inkscape:window-width="1920"
+ inkscape:window-height="1017"
+ inkscape:window-x="-8"
+ inkscape:window-y="-8"
+ inkscape:window-maximized="1"
+ inkscape:snap-nodes="false"
+ inkscape:showpageshadow="2"
+ inkscape:pagecheckerboard="0"
+ inkscape:deskcolor="#d1d1d1">
+ <inkscape:grid
+ type="xygrid"
+ id="grid13454" />
+ </sodipodi:namedview>
+ <metadata
+ id="metadata13242">
+ <rdf:RDF>
+ <cc:Work
+ rdf:about="">
+ <dc:format>image/svg+xml</dc:format>
+ <dc:type
+ rdf:resource="http://purl.org/dc/dcmitype/StillImage" />
+ <dc:title />
+ </cc:Work>
+ </rdf:RDF>
+ </metadata>
+ <g
+ id="layer1"
+ inkscape:label="Layer 1"
+ inkscape:groupmode="layer">
+ <g
+ style="font-size:12px;fill:none;stroke-linecap:square;stroke-miterlimit:3;overflow:visible"
+ id="shape1-1-2-4"
+ v:mID="1"
+ v:groupContext="shape"
+ transform="matrix(2.1604167,0,0,1.5671361,88.874699,-812.39909)">
+ <title
+ id="title22-7-5">Square</title>
+ <desc
+ id="desc24-7-8">Atomic Queue #1</desc>
+ <v:userDefs>
+ <v:ud
+ v:nameU="visVersion"
+ v:val="VT0(15):26" />
+ </v:userDefs>
+ <v:textBlock
+ v:margins="rect(4,4,4,4)" />
+ <v:textRect
+ cx="30.75"
+ cy="581.25"
+ width="61.5"
+ height="61.5" />
+ <g
+ id="shadow1-2-9-5"
+ v:groupContext="shadow"
+ v:shadowOffsetX="0.345598"
+ v:shadowOffsetY="-1.97279"
+ v:shadowType="1"
+ transform="translate(0.345598,1.97279)"
+ class="st1"
+ style="visibility:visible">
+ <rect
+ x="0"
+ y="550.5"
+ width="61.5"
+ height="61.5"
+ class="st2"
+ id="rect27-8-7"
+ style="fill:#5b9bd5;fill-opacity:0.22000002;stroke:#5b9bd5;stroke-opacity:0.22000002;filter:url(#filter_2-3-91)" />
+ </g>
+ <g
+ id="g13515-33">
+ <g
+ id="g13534-8">
+ <rect
+ x="0"
+ y="550.5"
+ width="61.5"
+ height="61.5"
+ class="st3"
+ id="rect29-1-95"
+ style="fill:#5b9bd5;stroke:#c7c8c8;stroke-width:0.25" />
+ </g>
+ </g>
+ </g>
+ <path
+ style="fill:none;stroke:#000000;stroke-width:0.712265;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:4;stroke-dasharray:none;stroke-opacity:1;marker-start:url(#Arrow1Lstart-7);marker-end:none"
+ d="M 312.28671,240.74335 H 227.99897"
+ id="path17209"
+ inkscape:connector-type="orthogonal"
+ inkscape:connector-curvature="0" />
+ <path
+ style="fill:none;stroke:#000000;stroke-width:0.718986px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1;marker-mid:none;marker-end:url(#Arrow1Lend)"
+ d="m 221.6484,77.57125 h 94.28101"
+ id="path17209-8"
+ inkscape:connector-type="orthogonal"
+ inkscape:connector-curvature="0" />
+ <g
+ style="font-size:12px;fill:none;stroke-linecap:square;stroke-miterlimit:3;overflow:visible"
+ id="shape1-1-2"
+ v:mID="1"
+ v:groupContext="shape"
+ transform="matrix(2.1604167,0,0,1.5671361,314.24227,-811.89589)">
+ <title
+ id="title22-7">Square</title>
+ <desc
+ id="desc24-7">Atomic Queue #1</desc>
+ <v:userDefs>
+ <v:ud
+ v:nameU="visVersion"
+ v:val="VT0(15):26" />
+ </v:userDefs>
+ <v:textBlock
+ v:margins="rect(4,4,4,4)" />
+ <v:textRect
+ cx="30.75"
+ cy="581.25"
+ width="61.5"
+ height="61.5" />
+ <g
+ id="shadow1-2-9"
+ v:groupContext="shadow"
+ v:shadowOffsetX="0.345598"
+ v:shadowOffsetY="-1.97279"
+ v:shadowType="1"
+ transform="translate(0.345598,1.97279)"
+ class="st1"
+ style="visibility:visible">
+ <rect
+ x="0"
+ y="550.5"
+ width="61.5"
+ height="61.5"
+ class="st2"
+ id="rect27-8"
+ style="fill:#5b9bd5;fill-opacity:0.22000002;stroke:#5b9bd5;stroke-opacity:0.22000002;filter:url(#filter_2-3)" />
+ </g>
+ <g
+ id="g13515">
+ <g
+ id="g13534">
+ <rect
+ x="0"
+ y="550.5"
+ width="61.5"
+ height="61.5"
+ class="st3"
+ id="rect29-1"
+ style="fill:#5b9bd5;stroke:#c7c8c8;stroke-width:0.25" />
+ </g>
+ </g>
+ </g>
+ <path
+ style="fill:none;stroke:#000000;stroke-width:0.724714;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:4;stroke-dasharray:none;stroke-opacity:1;marker-start:url(#Arrow1Lstart);marker-end:none"
+ d="M 89.025329,74.39932 H 24.750043"
+ id="path17209-3"
+ inkscape:connector-type="orthogonal"
+ inkscape:connector-curvature="0" />
+ <path
+ transform="matrix(0.73232502,0,0,0.75477602,-4.325033,28.642983)"
+ sodipodi:type="arc"
+ style="fill:#539de6;fill-opacity:1;stroke:#0000ea;stroke-width:1;stroke-linecap:square;stroke-miterlimit:4;stroke-opacity:1;stroke-dasharray:none"
+ id="path29161-3"
+ sodipodi:cx="371"
+ sodipodi:cy="64.5"
+ sodipodi:rx="17"
+ sodipodi:ry="15.5"
+ d="m 388,64.5 a 17,15.5 0 1 1 -34,0 17,15.5 0 1 1 34,0 z" />
+ <path
+ transform="matrix(0.73232502,0,0,0.75477602,-1.93108,192.80833)"
+ sodipodi:type="arc"
+ style="fill:#539de6;fill-opacity:1;stroke:#0000ea;stroke-width:1;stroke-linecap:square;stroke-miterlimit:4;stroke-opacity:1;stroke-dasharray:none"
+ id="path29161-1"
+ sodipodi:cx="371"
+ sodipodi:cy="64.5"
+ sodipodi:rx="17"
+ sodipodi:ry="15.5"
+ d="m 388,64.5 a 17,15.5 0 1 1 -34,0 17,15.5 0 1 1 34,0 z" />
+ <path
+ style="fill:none;stroke:#000000;stroke-width:0.751412;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:4;stroke-dasharray:none;stroke-opacity:1;marker-start:url(#Arrow1Lstart-7);marker-end:none"
+ d="M 18.763392,120.7432 H 87.758545"
+ id="path17209-3-0"
+ inkscape:connector-type="orthogonal"
+ inkscape:connector-curvature="0" />
+ <path
+ sodipodi:type="arc"
+ style="fill:#539de6;fill-opacity:1;stroke:#0000ea;stroke-width:1;stroke-linecap:square;stroke-miterlimit:4;stroke-opacity:1;stroke-dasharray:none"
+ id="path29161"
+ sodipodi:cx="371"
+ sodipodi:cy="64.5"
+ sodipodi:rx="17"
+ sodipodi:ry="15.5"
+ d="m 388,64.5 a 17,15.5 0 1 1 -34,0 17,15.5 0 1 1 34,0 z"
+ transform="matrix(0.73232502,0,0,0.75477602,-218.16394,72.68276)" />
+ <path
+ sodipodi:type="arc"
+ style="fill:#539de6;fill-opacity:1;stroke:#0000ea;stroke-width:1;stroke-linecap:square;stroke-miterlimit:4;stroke-opacity:1;stroke-dasharray:none"
+ id="path29161-2"
+ sodipodi:cx="371"
+ sodipodi:cy="64.5"
+ sodipodi:rx="17"
+ sodipodi:ry="15.5"
+ d="m 388,64.5 a 17,15.5 0 1 1 -34,0 17,15.5 0 1 1 34,0 z"
+ transform="matrix(0.73232502,0,0,0.75477602,-217.40136,26.716271)" />
+ <g
+ id="g29167-4"
+ transform="matrix(0.73232502,0,0,0.75477602,-217.31662,28.007562)">
+ <text
+ id="text29163-9"
+ y="70"
+ x="365"
+ style="font-style:normal;font-weight:normal;line-height:0%;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none"
+ xml:space="preserve"><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="70"
+ x="365"
+ id="tspan29165-9"
+ sodipodi:role="line">1</tspan></text>
+ </g>
+ <g
+ id="g29167-9"
+ transform="matrix(0.73232502,0,0,0.75477602,-4.9726112,28.689051)">
+ <text
+ id="text29163-3"
+ y="70"
+ x="365"
+ style="font-style:normal;font-weight:normal;line-height:0%;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none"
+ xml:space="preserve"><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="70"
+ x="365"
+ id="tspan29165-3"
+ sodipodi:role="line">2</tspan></text>
+ </g>
+ <path
+ style="fill:none;stroke:#000000;stroke-width:0.678033px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1;marker-start:url(#Arrow1Lstart-7);marker-end:none"
+ d="M 181,214.66098 V 145.33902"
+ id="path17211-7-1-6"
+ inkscape:connector-type="orthogonal"
+ inkscape:connector-curvature="0" />
+ <g
+ id="g29167"
+ transform="matrix(0.73232502,0,0,0.75477602,-218.07919,73.10621)">
+ <text
+ id="text29163"
+ y="70"
+ x="365"
+ style="font-style:normal;font-weight:normal;line-height:0%;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none"
+ xml:space="preserve"><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="70"
+ x="365"
+ id="tspan29165"
+ sodipodi:role="line">8</tspan></text>
+ </g>
+ <path
+ style="fill:none;stroke:#000000;stroke-width:0.678033px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1;marker-start:url(#Arrow1Lstart-7);marker-end:none"
+ d="m 131,145.8531 v 69.32197"
+ id="path17211-7-1"
+ inkscape:connector-type="orthogonal"
+ inkscape:connector-curvature="0" />
+ <path
+ transform="matrix(0.73232502,0,0,0.75477602,-140.37076,129.97088)"
+ sodipodi:type="arc"
+ style="fill:#539de6;fill-opacity:1;stroke:#0000ea;stroke-width:1;stroke-linecap:square;stroke-miterlimit:4;stroke-opacity:1;stroke-dasharray:none"
+ id="path29161-8"
+ sodipodi:cx="371"
+ sodipodi:cy="64.5"
+ sodipodi:rx="17"
+ sodipodi:ry="15.5"
+ d="m 388,64.5 a 17,15.5 0 1 1 -34,0 17,15.5 0 1 1 34,0 z" />
+ <g
+ id="g29167-2"
+ transform="matrix(0.73232502,0,0,0.75477602,-140.28602,131.01695)">
+ <text
+ id="text29163-92"
+ y="70"
+ x="365"
+ style="font-style:normal;font-weight:normal;line-height:0%;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none"
+ xml:space="preserve"><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="70"
+ x="365"
+ id="tspan29165-8"
+ sodipodi:role="line">7</tspan></text>
+ </g>
+ <path
+ style="fill:none;stroke:#000000;stroke-width:0.718986px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1;marker-mid:none;marker-end:url(#Arrow1Lend)"
+ d="m 317.1405,116 h -94.281"
+ id="path17209-8-0"
+ inkscape:connector-type="orthogonal"
+ inkscape:connector-curvature="0" />
+ <path
+ transform="matrix(0.73232502,0,0,0.75477602,-3.4914,66.68745)"
+ sodipodi:type="arc"
+ style="fill:#539de6;fill-opacity:1;stroke:#0000ea;stroke-width:1;stroke-linecap:square;stroke-miterlimit:4;stroke-opacity:1;stroke-dasharray:none"
+ id="path29161-6"
+ sodipodi:cx="371"
+ sodipodi:cy="64.5"
+ sodipodi:rx="17"
+ sodipodi:ry="15.5"
+ d="m 388,64.5 a 17,15.5 0 1 1 -34,0 17,15.5 0 1 1 34,0 z" />
+ <g
+ id="g29167-46"
+ transform="matrix(0.73232502,0,0,0.75477602,-4.40666,67.48829)">
+ <text
+ id="text29163-1"
+ y="70"
+ x="365"
+ style="font-style:normal;font-weight:normal;line-height:0%;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none"
+ xml:space="preserve"><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="70"
+ x="365"
+ id="tspan29165-5"
+ sodipodi:role="line">3</tspan></text>
+ </g>
+ <path
+ transform="matrix(0.73232502,0,0,0.75477602,-90.692582,130.31695)"
+ sodipodi:type="arc"
+ style="fill:#539de6;fill-opacity:1;stroke:#0000ea;stroke-width:1;stroke-linecap:square;stroke-miterlimit:4;stroke-opacity:1;stroke-dasharray:none"
+ id="path29161-8-6"
+ sodipodi:cx="371"
+ sodipodi:cy="64.5"
+ sodipodi:rx="17"
+ sodipodi:ry="15.5"
+ d="m 388,64.5 a 17,15.5 0 1 1 -34,0 17,15.5 0 1 1 34,0 z" />
+ <g
+ id="g29167-6"
+ transform="matrix(0.73232502,0,0,0.75477602,-90.84634,131.60918)">
+ <text
+ id="text29163-17"
+ y="70"
+ x="365"
+ style="font-style:normal;font-weight:normal;line-height:0%;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none"
+ xml:space="preserve"><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="70"
+ x="365"
+ id="tspan29165-2"
+ sodipodi:role="line">4</tspan></text>
+ </g>
+ <g
+ id="g29167-2-0"
+ transform="matrix(0.73232502,0,0,0.75477602,-2.424397,194.0216)">
+ <text
+ id="text29163-92-6"
+ y="70"
+ x="365"
+ style="font-style:normal;font-weight:normal;line-height:0%;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none"
+ xml:space="preserve"><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="70"
+ x="365"
+ id="tspan29165-8-2"
+ sodipodi:role="line">5</tspan></text>
+ </g>
+ <g
+ style="font-size:12px;fill:none;stroke-linecap:square;stroke-miterlimit:3;overflow:visible"
+ id="shape1-1-2-8"
+ v:mID="1"
+ v:groupContext="shape"
+ transform="matrix(2.1604167,0,0,1.5671361,93.82055,-648.98949)">
+ <title
+ id="title22-7-97">Square</title>
+ <desc
+ id="desc24-7-3">Atomic Queue #1</desc>
+ <v:userDefs>
+ <v:ud
+ v:nameU="visVersion"
+ v:val="VT0(15):26" />
+ </v:userDefs>
+ <v:textBlock
+ v:margins="rect(4,4,4,4)" />
+ <v:textRect
+ cx="30.75"
+ cy="581.25"
+ width="61.5"
+ height="61.5" />
+ <g
+ id="shadow1-2-9-6"
+ v:groupContext="shadow"
+ v:shadowOffsetX="0.345598"
+ v:shadowOffsetY="-1.97279"
+ v:shadowType="1"
+ transform="translate(0.345598,1.97279)"
+ class="st1"
+ style="visibility:visible">
+ <rect
+ x="0"
+ y="550.5"
+ width="61.5"
+ height="61.5"
+ class="st2"
+ id="rect27-8-12"
+ style="fill:#5b9bd5;fill-opacity:0.22000002;stroke:#5b9bd5;stroke-opacity:0.22000002;filter:url(#filter_2-3-92)" />
+ </g>
+ <g
+ id="g13515-9">
+ <g
+ id="g13534-3">
+ <rect
+ x="0"
+ y="550.5"
+ width="61.5"
+ height="61.5"
+ class="st3"
+ id="rect29-1-1"
+ style="fill:#5b9bd5;stroke:#c7c8c8;stroke-width:0.25" />
+ </g>
+ </g>
+ </g>
+ <g
+ style="font-size:12px;fill:none;stroke-linecap:square;stroke-miterlimit:3;overflow:visible"
+ id="shape1-1-2-84"
+ v:mID="1"
+ v:groupContext="shape"
+ transform="matrix(2.1604167,0,0,1.5671361,314.82055,-648.98949)">
+ <title
+ id="title22-7-50">Square</title>
+ <desc
+ id="desc24-7-36">Atomic Queue #1</desc>
+ <v:userDefs>
+ <v:ud
+ v:nameU="visVersion"
+ v:val="VT0(15):26" />
+ </v:userDefs>
+ <v:textBlock
+ v:margins="rect(4,4,4,4)" />
+ <v:textRect
+ cx="30.75"
+ cy="581.25"
+ width="61.5"
+ height="61.5" />
+ <g
+ id="shadow1-2-9-1"
+ v:groupContext="shadow"
+ v:shadowOffsetX="0.345598"
+ v:shadowOffsetY="-1.97279"
+ v:shadowType="1"
+ transform="translate(0.345598,1.97279)"
+ class="st1"
+ style="visibility:visible">
+ <rect
+ x="0"
+ y="550.5"
+ width="61.5"
+ height="61.5"
+ class="st2"
+ id="rect27-8-0"
+ style="fill:#5b9bd5;fill-opacity:0.22000002;stroke:#5b9bd5;stroke-opacity:0.22000002;filter:url(#filter_2-3-94)" />
+ </g>
+ <g
+ id="g13515-6">
+ <g
+ id="g13534-32">
+ <rect
+ x="0"
+ y="550.5"
+ width="61.5"
+ height="61.5"
+ class="st3"
+ id="rect29-1-0"
+ style="fill:#5b9bd5;stroke:#c7c8c8;stroke-width:0.25" />
+ </g>
+ </g>
+ </g>
+ <path
+ style="fill:none;stroke:#000000;stroke-width:0.712265;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:4;stroke-dasharray:none;stroke-opacity:1;marker-start:none;marker-end:url(#Arrow1Lend)"
+ d="M 313.14387,285 H 228.85613"
+ id="path17209-7"
+ inkscape:connector-type="orthogonal"
+ inkscape:connector-curvature="0" />
+ <path
+ transform="matrix(0.73232502,0,0,0.75477602,-2.692582,236.31695)"
+ sodipodi:type="arc"
+ style="fill:#539de6;fill-opacity:1;stroke:#0000ea;stroke-width:1;stroke-linecap:square;stroke-miterlimit:4;stroke-opacity:1;stroke-dasharray:none"
+ id="path29161-1-6"
+ sodipodi:cx="371"
+ sodipodi:cy="64.5"
+ sodipodi:rx="17"
+ sodipodi:ry="15.5"
+ d="m 388,64.5 a 17,15.5 0 1 1 -34,0 17,15.5 0 1 1 34,0 z" />
+ <g
+ id="g29167-2-0-5"
+ transform="matrix(0.73232502,0,0,0.75477602,-2.424397,237.0216)">
+ <text
+ id="text29163-92-6-6"
+ y="70"
+ x="365"
+ style="font-style:normal;font-weight:normal;line-height:0%;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none"
+ xml:space="preserve"><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="70"
+ x="365"
+ id="tspan29165-8-2-9"
+ sodipodi:role="line">6</tspan></text>
+ </g>
+ <g
+ id="g29167-4-3"
+ transform="matrix(0.73232502,0,0,0.75477602,-154.60784,51.117791)">
+ <text
+ id="text29163-9-6"
+ y="70"
+ x="365"
+ style="font-style:normal;font-weight:normal;line-height:0%;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none"
+ xml:space="preserve"><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:24.2109px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="70"
+ x="365"
+ id="tspan29165-9-7"
+ sodipodi:role="line">Eventdev</tspan></text>
+ </g>
+ <g
+ id="g29167-4-3-5"
+ transform="matrix(0.73232502,0,0,0.75477602,-144.65044,201.97821)">
+ <text
+ id="text29163-9-6-3"
+ y="70"
+ x="412.93716"
+ style="font-style:normal;font-weight:normal;line-height:0%;font-family:sans-serif;text-align:center;letter-spacing:0px;word-spacing:0px;text-anchor:middle;fill:#000000;fill-opacity:1;stroke:none"
+ xml:space="preserve"><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:24.2109px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:center;writing-mode:lr-tb;text-anchor:middle"
+ y="70"
+ x="412.93716"
+ id="tspan29165-9-7-5"
+ sodipodi:role="line">DMA</tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:24.2109px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:center;writing-mode:lr-tb;text-anchor:middle"
+ y="100.26363"
+ x="412.93716"
+ sodipodi:role="line"
+ id="tspan3201">Adapter</tspan></text>
+ </g>
+ <g
+ id="g29167-4-3-5-6"
+ transform="matrix(0.73232502,0,0,0.75477602,79.53518,46.62529)">
+ <text
+ id="text29163-9-6-3-2"
+ y="48.801659"
+ x="412.93716"
+ style="font-style:normal;font-weight:normal;line-height:0%;font-family:sans-serif;text-align:center;letter-spacing:0px;word-spacing:0px;text-anchor:middle;fill:#000000;fill-opacity:1;stroke:none"
+ xml:space="preserve"><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:24.2109px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:center;writing-mode:lr-tb;text-anchor:middle"
+ y="48.801659"
+ x="412.93716"
+ sodipodi:role="line"
+ id="tspan3155">Application</tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:24.2109px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:center;writing-mode:lr-tb;text-anchor:middle"
+ y="98.801659"
+ x="412.93716"
+ sodipodi:role="line"
+ id="tspan3201-1">in ordered</tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:24.2109px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:center;writing-mode:lr-tb;text-anchor:middle"
+ y="148.80167"
+ x="412.93716"
+ sodipodi:role="line"
+ id="tspan3161">stage</tspan></text>
+ </g>
+ <g
+ id="g29167-4-3-5-2"
+ transform="matrix(0.73232502,0,0,0.75477602,77.535182,213.62529)">
+ <text
+ id="text29163-9-6-3-7"
+ y="70"
+ x="412.93716"
+ style="font-style:normal;font-weight:normal;line-height:0%;font-family:sans-serif;text-align:center;letter-spacing:0px;word-spacing:0px;text-anchor:middle;fill:#000000;fill-opacity:1;stroke:none"
+ xml:space="preserve"><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:24.2109px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:center;writing-mode:lr-tb;text-anchor:middle"
+ y="70"
+ x="412.93716"
+ sodipodi:role="line"
+ id="tspan3201-9">DMA Device</tspan></text>
+ </g>
+ <g
+ id="g29167-4-3-5-3"
+ transform="matrix(0.73232502,0,0,0.75477602,188.53518,-3.37471)">
+ <text
+ id="text29163-9-6-3-6"
+ y="70"
+ x="375.65271"
+ style="font-style:normal;font-weight:normal;line-height:0%;font-family:sans-serif;text-align:center;letter-spacing:0px;word-spacing:0px;text-anchor:middle;fill:#000000;fill-opacity:1;stroke:none"
+ xml:space="preserve"><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18.8307px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="70"
+ x="375.65271"
+ sodipodi:role="line"
+ id="tspan3201-6">1. Events from the previous stage.</tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18.8307px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="93.538376"
+ x="375.65271"
+ sodipodi:role="line"
+ id="tspan3260"> </tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18.8307px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="117.07675"
+ x="375.65271"
+ sodipodi:role="line"
+ id="tspan3262">2. Application in ordered stage</tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18.8307px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="140.61513"
+ x="375.65271"
+ sodipodi:role="line"
+ id="tspan3288"> dequeues events from eventdev.</tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18.8307px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="164.1535"
+ x="375.65271"
+ sodipodi:role="line"
+ id="tspan3264"> </tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18.8307px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="187.69188"
+ x="375.65271"
+ sodipodi:role="line"
+ id="tspan3266">3. Application enqueues DMA</tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18.8307px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="211.23026"
+ x="375.65271"
+ sodipodi:role="line"
+ id="tspan3290"> operations as events to eventdev.</tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18.8307px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="234.76863"
+ x="375.65271"
+ sodipodi:role="line"
+ id="tspan3268"> </tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18.8307px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="258.30701"
+ x="375.65271"
+ sodipodi:role="line"
+ id="tspan3270">4. DMA adapter dequeues event</tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18.8307px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="281.84537"
+ x="375.65271"
+ sodipodi:role="line"
+ id="tspan3292"> from eventdev.</tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18.8307px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="305.38376"
+ x="375.65271"
+ sodipodi:role="line"
+ id="tspan3272"> </tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18.8307px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="328.92212"
+ x="375.65271"
+ sodipodi:role="line"
+ id="tspan3274">5. DMA adapter submits DMA</tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18.8307px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="352.46051"
+ x="375.65271"
+ sodipodi:role="line"
+ id="tspan3294"> operations to DMA Device (Atomic</tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18.8307px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="375.99887"
+ x="375.65271"
+ sodipodi:role="line"
+ id="tspan3296"> stage)</tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18.8307px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="399.53726"
+ x="375.65271"
+ sodipodi:role="line"
+ id="tspan3276"> </tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18.8307px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="423.07562"
+ x="375.65271"
+ sodipodi:role="line"
+ id="tspan3278">6. DMA adapter dequeues DMA</tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18.8307px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="446.61401"
+ x="375.65271"
+ sodipodi:role="line"
+ id="tspan3298"> completions from DMA Device</tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18.8307px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="470.15237"
+ x="375.65271"
+ sodipodi:role="line"
+ id="tspan3280"> </tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18.8307px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="493.69073"
+ x="375.65271"
+ sodipodi:role="line"
+ id="tspan3282">7. DMA adapter enqueues events</tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18.8307px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="517.22913"
+ x="375.65271"
+ sodipodi:role="line"
+ id="tspan3300"> to the eventdev</tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18.8307px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="540.76752"
+ x="375.65271"
+ sodipodi:role="line"
+ id="tspan3284"> </tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18.8307px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="564.30585"
+ x="375.65271"
+ sodipodi:role="line"
+ id="tspan3286">8. Events to the next stage</tspan></text>
+ </g>
+ </g>
+</svg>
diff --git a/doc/guides/prog_guide/img/event_dma_adapter_op_new.svg b/doc/guides/prog_guide/img/event_dma_adapter_op_new.svg
new file mode 100644
index 0000000000..e9e8bb2b98
--- /dev/null
+++ b/doc/guides/prog_guide/img/event_dma_adapter_op_new.svg
@@ -0,0 +1,1079 @@
+<?xml version="1.0" encoding="UTF-8" standalone="no"?>
+<!-- SPDX-License-Identifier: BSD-3-Clause -->
+<!-- Copyright (c) 2023 Marvell. -->
+<!-- Created with Inkscape (http://www.inkscape.org/) -->
+
+<svg
+ width="720px"
+ height="486px"
+ id="svg13237"
+ version="1.1"
+ inkscape:version="1.2.1 (9c6d41e410, 2022-07-14)"
+ sodipodi:docname="event_dma_adapter_op_new.svg"
+ xmlns:inkscape="http://www.inkscape.org/namespaces/inkscape"
+ xmlns:sodipodi="http://sodipodi.sourceforge.net/DTD/sodipodi-0.dtd"
+ xmlns="http://www.w3.org/2000/svg"
+ xmlns:svg="http://www.w3.org/2000/svg"
+ xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#"
+ xmlns:cc="http://creativecommons.org/ns#"
+ xmlns:dc="http://purl.org/dc/elements/1.1/"
+ xmlns:v="http://schemas.microsoft.com/visio/2003/SVGExtensions/">
+ <defs
+ id="defs13239">
+ <marker
+ inkscape:stockid="Arrow1Sstart"
+ orient="auto"
+ refY="0.0"
+ refX="0.0"
+ id="Arrow1Sstart"
+ style="overflow:visible">
+ <path
+ id="path8416"
+ d="M 0.0,0.0 L 5.0,-5.0 L -12.5,0.0 L 5.0,5.0 L 0.0,0.0 z "
+ style="fill-rule:evenodd;stroke:#000000;stroke-width:1.0pt"
+ transform="scale(0.2) translate(6,0)" />
+ </marker>
+ <marker
+ inkscape:stockid="Arrow1Send"
+ orient="auto"
+ refY="0.0"
+ refX="0.0"
+ id="Arrow1Send"
+ style="overflow:visible;">
+ <path
+ id="path8419"
+ d="M 0.0,0.0 L 5.0,-5.0 L -12.5,0.0 L 5.0,5.0 L 0.0,0.0 z "
+ style="fill-rule:evenodd;stroke:#000000;stroke-width:1.0pt;"
+ transform="scale(0.2) rotate(180) translate(6,0)" />
+ </marker>
+ <marker
+ inkscape:stockid="DiamondL"
+ orient="auto"
+ refY="0.0"
+ refX="0.0"
+ id="DiamondL"
+ style="overflow:visible">
+ <path
+ id="path8483"
+ d="M 0,-7.0710768 L -7.0710894,0 L 0,7.0710589 L 7.0710462,0 L 0,-7.0710768 z "
+ style="fill-rule:evenodd;stroke:#000000;stroke-width:1.0pt"
+ transform="scale(0.8)" />
+ </marker>
+ <marker
+ inkscape:stockid="DotL"
+ orient="auto"
+ refY="0.0"
+ refX="0.0"
+ id="DotL"
+ style="overflow:visible">
+ <path
+ id="path8465"
+ d="M -2.5,-1.0 C -2.5,1.7600000 -4.7400000,4.0 -7.5,4.0 C -10.260000,4.0 -12.5,1.7600000 -12.5,-1.0 C -12.5,-3.7600000 -10.260000,-6.0 -7.5,-6.0 C -4.7400000,-6.0 -2.5,-3.7600000 -2.5,-1.0 z "
+ style="fill-rule:evenodd;stroke:#000000;stroke-width:1.0pt"
+ transform="scale(0.8) translate(7.4, 1)" />
+ </marker>
+ <marker
+ inkscape:stockid="SquareL"
+ orient="auto"
+ refY="0.0"
+ refX="0.0"
+ id="SquareL"
+ style="overflow:visible">
+ <path
+ id="path8474"
+ d="M -5.0,-5.0 L -5.0,5.0 L 5.0,5.0 L 5.0,-5.0 L -5.0,-5.0 z "
+ style="fill-rule:evenodd;stroke:#000000;stroke-width:1.0pt"
+ transform="scale(0.8)" />
+ </marker>
+ <marker
+ inkscape:stockid="TriangleOutL"
+ orient="auto"
+ refY="0.0"
+ refX="0.0"
+ id="TriangleOutL"
+ style="overflow:visible">
+ <path
+ id="path8546"
+ d="M 5.77,0.0 L -2.88,5.0 L -2.88,-5.0 L 5.77,0.0 z "
+ style="fill-rule:evenodd;stroke:#000000;stroke-width:1.0pt"
+ transform="scale(0.8)" />
+ </marker>
+ <marker
+ inkscape:stockid="Arrow1Lstart"
+ orient="auto"
+ refY="0.0"
+ refX="0.0"
+ id="Arrow1Lstart"
+ style="overflow:visible">
+ <path
+ id="path8404"
+ d="M 0.0,0.0 L 5.0,-5.0 L -12.5,0.0 L 5.0,5.0 L 0.0,0.0 z "
+ style="fill-rule:evenodd;stroke:#000000;stroke-width:1.0pt"
+ transform="scale(0.8) translate(12.5,0)" />
+ </marker>
+ <marker
+ inkscape:stockid="Arrow1Mend"
+ orient="auto"
+ refY="0.0"
+ refX="0.0"
+ id="Arrow1Mend"
+ style="overflow:visible;">
+ <path
+ id="path8413"
+ d="M 0.0,0.0 L 5.0,-5.0 L -12.5,0.0 L 5.0,5.0 L 0.0,0.0 z "
+ style="fill-rule:evenodd;stroke:#000000;stroke-width:1.0pt;"
+ transform="scale(0.4) rotate(180) translate(10,0)" />
+ </marker>
+ <marker
+ inkscape:stockid="Arrow2Lend"
+ orient="auto"
+ refY="0.0"
+ refX="0.0"
+ id="Arrow2Lend"
+ style="overflow:visible;">
+ <path
+ id="path8425"
+ style="fill-rule:evenodd;stroke-width:0.62500000;stroke-linejoin:round;"
+ d="M 8.7185878,4.0337352 L -2.2072895,0.016013256 L 8.7185884,-4.0017078 C 6.9730900,-1.6296469 6.9831476,1.6157441 8.7185878,4.0337352 z "
+ transform="scale(1.1) rotate(180) translate(1,0)" />
+ </marker>
+ <marker
+ inkscape:stockid="Arrow1Lend"
+ orient="auto"
+ refY="0.0"
+ refX="0.0"
+ id="Arrow1Lend"
+ style="overflow:visible;">
+ <path
+ id="path8407"
+ d="M 0.0,0.0 L 5.0,-5.0 L -12.5,0.0 L 5.0,5.0 L 0.0,0.0 z "
+ style="fill-rule:evenodd;stroke:#000000;stroke-width:1.0pt;"
+ transform="scale(0.8) rotate(180) translate(12.5,0)" />
+ </marker>
+ <filter
+ id="filter_2"
+ color-interpolation-filters="sRGB">
+ <feGaussianBlur
+ stdDeviation="2"
+ id="feGaussianBlur15" />
+ </filter>
+ <filter
+ id="filter_2-3"
+ color-interpolation-filters="sRGB"
+ x="-0.086178862"
+ y="-0.086178862"
+ width="1.1723577"
+ height="1.1723577">
+ <feGaussianBlur
+ stdDeviation="2"
+ id="feGaussianBlur15-1" />
+ </filter>
+ <filter
+ id="filter_2-0"
+ color-interpolation-filters="sRGB">
+ <feGaussianBlur
+ stdDeviation="2"
+ id="feGaussianBlur15-7" />
+ </filter>
+ <filter
+ id="filter_2-0-8"
+ color-interpolation-filters="sRGB">
+ <feGaussianBlur
+ stdDeviation="2"
+ id="feGaussianBlur15-7-7" />
+ </filter>
+ <filter
+ id="filter_2-3-9"
+ color-interpolation-filters="sRGB"
+ x="-0.086178862"
+ y="-0.086178862"
+ width="1.1723577"
+ height="1.1723577">
+ <feGaussianBlur
+ stdDeviation="2"
+ id="feGaussianBlur15-1-6" />
+ </filter>
+ <filter
+ id="filter_2-3-6"
+ color-interpolation-filters="sRGB"
+ x="-0.086178862"
+ y="-0.086178862"
+ width="1.1723577"
+ height="1.1723577">
+ <feGaussianBlur
+ stdDeviation="2"
+ id="feGaussianBlur15-1-63" />
+ </filter>
+ <filter
+ id="filter_2-3-91"
+ color-interpolation-filters="sRGB"
+ x="-0.086178862"
+ y="-0.086178862"
+ width="1.1723577"
+ height="1.1723577">
+ <feGaussianBlur
+ stdDeviation="2"
+ id="feGaussianBlur15-1-3" />
+ </filter>
+ <marker
+ inkscape:stockid="Arrow1Lend"
+ orient="auto"
+ refY="0"
+ refX="0"
+ id="Arrow1Lend-5"
+ style="overflow:visible">
+ <path
+ inkscape:connector-curvature="0"
+ id="path8407-3"
+ d="M 0,0 5,-5 -12.5,0 5,5 0,0 z"
+ style="fill-rule:evenodd;stroke:#000000;stroke-width:1pt"
+ transform="matrix(-0.8,0,0,-0.8,-10,0)" />
+ </marker>
+ <marker
+ inkscape:stockid="Arrow1Lend"
+ orient="auto"
+ refY="0"
+ refX="0"
+ id="Arrow1Lend-6"
+ style="overflow:visible">
+ <path
+ inkscape:connector-curvature="0"
+ id="path8407-0"
+ d="M 0,0 5,-5 -12.5,0 5,5 0,0 z"
+ style="fill-rule:evenodd;stroke:#000000;stroke-width:1pt"
+ transform="matrix(-0.8,0,0,-0.8,-10,0)" />
+ </marker>
+ <marker
+ inkscape:stockid="Arrow1Lstart"
+ orient="auto"
+ refY="0"
+ refX="0"
+ id="Arrow1Lstart-7"
+ style="overflow:visible">
+ <path
+ inkscape:connector-curvature="0"
+ id="path8404-0"
+ d="M 0,0 5,-5 -12.5,0 5,5 0,0 z"
+ style="fill-rule:evenodd;stroke:#000000;stroke-width:1pt"
+ transform="matrix(0.8,0,0,0.8,10,0)" />
+ </marker>
+ <marker
+ inkscape:stockid="Arrow1Lend"
+ orient="auto"
+ refY="0"
+ refX="0"
+ id="Arrow1Lend-51"
+ style="overflow:visible">
+ <path
+ inkscape:connector-curvature="0"
+ id="path8407-1"
+ d="M 0,0 5,-5 -12.5,0 5,5 0,0 z"
+ style="fill-rule:evenodd;stroke:#000000;stroke-width:1pt"
+ transform="matrix(-0.8,0,0,-0.8,-10,0)" />
+ </marker>
+ <marker
+ inkscape:stockid="Arrow1Lend"
+ orient="auto"
+ refY="0"
+ refX="0"
+ id="Arrow1Lend-3"
+ style="overflow:visible">
+ <path
+ inkscape:connector-curvature="0"
+ id="path8407-6"
+ d="M 0,0 5,-5 -12.5,0 5,5 0,0 z"
+ style="fill-rule:evenodd;stroke:#000000;stroke-width:1pt"
+ transform="matrix(-0.8,0,0,-0.8,-10,0)" />
+ </marker>
+ <marker
+ inkscape:stockid="Arrow1Lend"
+ orient="auto"
+ refY="0"
+ refX="0"
+ id="Arrow1Lend-62"
+ style="overflow:visible">
+ <path
+ inkscape:connector-curvature="0"
+ id="path8407-9"
+ d="M 0,0 5,-5 -12.5,0 5,5 0,0 z"
+ style="fill-rule:evenodd;stroke:#000000;stroke-width:1pt"
+ transform="matrix(-0.8,0,0,-0.8,-10,0)" />
+ </marker>
+ <marker
+ inkscape:stockid="Arrow1Lend"
+ orient="auto"
+ refY="0"
+ refX="0"
+ id="Arrow1Lend-2"
+ style="overflow:visible">
+ <path
+ inkscape:connector-curvature="0"
+ id="path8407-7"
+ d="M 0,0 5,-5 -12.5,0 5,5 0,0 z"
+ style="fill-rule:evenodd;stroke:#000000;stroke-width:1pt"
+ transform="matrix(-0.8,0,0,-0.8,-10,0)" />
+ </marker>
+ <filter
+ id="filter_2-3-91-3"
+ color-interpolation-filters="sRGB"
+ x="-0.086178862"
+ y="-0.086178862"
+ width="1.1723577"
+ height="1.1723577">
+ <feGaussianBlur
+ stdDeviation="2"
+ id="feGaussianBlur15-1-3-6" />
+ </filter>
+ </defs>
+ <sodipodi:namedview
+ id="base"
+ pagecolor="#ffffff"
+ bordercolor="#666666"
+ borderopacity="1.0"
+ inkscape:pageopacity="0.0"
+ inkscape:pageshadow="2"
+ inkscape:zoom="1"
+ inkscape:cx="385.5"
+ inkscape:cy="234"
+ inkscape:document-units="px"
+ inkscape:current-layer="layer1"
+ showgrid="false"
+ inkscape:window-width="1920"
+ inkscape:window-height="1017"
+ inkscape:window-x="-8"
+ inkscape:window-y="-8"
+ inkscape:window-maximized="1"
+ inkscape:snap-nodes="false"
+ inkscape:showpageshadow="2"
+ inkscape:pagecheckerboard="0"
+ inkscape:deskcolor="#d1d1d1">
+ <inkscape:grid
+ type="xygrid"
+ id="grid13454" />
+ </sodipodi:namedview>
+ <metadata
+ id="metadata13242">
+ <rdf:RDF>
+ <cc:Work
+ rdf:about="">
+ <dc:format>image/svg+xml</dc:format>
+ <dc:type
+ rdf:resource="http://purl.org/dc/dcmitype/StillImage" />
+ <dc:title />
+ </cc:Work>
+ </rdf:RDF>
+ </metadata>
+ <g
+ id="layer1"
+ inkscape:label="Layer 1"
+ inkscape:groupmode="layer">
+ <g
+ style="font-size:12px;fill:none;stroke-linecap:square;stroke-miterlimit:3;overflow:visible"
+ id="shape1-1-2-0"
+ v:mID="1"
+ v:groupContext="shape"
+ transform="matrix(2.1604167,0,0,1.5671361,323.2187,-540.25927)">
+ <title
+ id="title22-7-6">Square</title>
+ <desc
+ id="desc24-7-4">Atomic Queue #1</desc>
+ <v:userDefs>
+ <v:ud
+ v:nameU="visVersion"
+ v:val="VT0(15):26" />
+ </v:userDefs>
+ <v:textBlock
+ v:margins="rect(4,4,4,4)" />
+ <v:textRect
+ cx="30.75"
+ cy="581.25"
+ width="61.5"
+ height="61.5" />
+ <g
+ id="shadow1-2-9-0"
+ v:groupContext="shadow"
+ v:shadowOffsetX="0.345598"
+ v:shadowOffsetY="-1.97279"
+ v:shadowType="1"
+ transform="translate(0.345598,1.97279)"
+ class="st1"
+ style="visibility:visible">
+ <rect
+ x="0"
+ y="550.5"
+ width="61.5"
+ height="61.5"
+ class="st2"
+ id="rect27-8-9"
+ style="fill:#5b9bd5;fill-opacity:0.22000002;stroke:#5b9bd5;stroke-opacity:0.22000002;filter:url(#filter_2-3-9)" />
+ </g>
+ <g
+ id="g13515-4">
+ <g
+ id="g13534-5">
+ <rect
+ x="0"
+ y="550.5"
+ width="61.5"
+ height="61.5"
+ class="st3"
+ id="rect29-1-4"
+ style="fill:#5b9bd5;stroke:#c7c8c8;stroke-width:0.25" />
+ </g>
+ </g>
+ </g>
+ <g
+ style="font-size:12px;fill:none;stroke-linecap:square;stroke-miterlimit:3;overflow:visible"
+ id="shape1-1-2-4"
+ v:mID="1"
+ v:groupContext="shape"
+ transform="matrix(2.1604167,0,0,1.165886,88.874699,-447.8809)">
+ <title
+ id="title22-7-5">Square</title>
+ <desc
+ id="desc24-7-8">Atomic Queue #1</desc>
+ <v:userDefs>
+ <v:ud
+ v:nameU="visVersion"
+ v:val="VT0(15):26" />
+ </v:userDefs>
+ <v:textBlock
+ v:margins="rect(4,4,4,4)" />
+ <v:textRect
+ cx="30.75"
+ cy="581.25"
+ width="61.5"
+ height="61.5" />
+ <g
+ id="shadow1-2-9-5"
+ v:groupContext="shadow"
+ v:shadowOffsetX="0.345598"
+ v:shadowOffsetY="-1.97279"
+ v:shadowType="1"
+ transform="translate(0.345598,1.97279)"
+ class="st1"
+ style="visibility:visible">
+ <rect
+ x="0"
+ y="550.5"
+ width="61.5"
+ height="61.5"
+ class="st2"
+ id="rect27-8-7"
+ style="fill:#5b9bd5;fill-opacity:0.22000002;stroke:#5b9bd5;stroke-opacity:0.22000002;filter:url(#filter_2-3-91)" />
+ </g>
+ <g
+ id="g13515-33">
+ <g
+ id="g13534-8">
+ <rect
+ x="0"
+ y="550.5"
+ width="61.5"
+ height="61.5"
+ class="st3"
+ id="rect29-1-95"
+ style="fill:#5b9bd5;stroke:#c7c8c8;stroke-width:0.25" />
+ </g>
+ </g>
+ </g>
+ <g
+ style="font-size:12px;fill:none;stroke-linecap:square;stroke-miterlimit:3;overflow:visible"
+ id="shape1-1-2-9"
+ v:mID="1"
+ v:groupContext="shape"
+ transform="matrix(2.1604167,0,0,1.5671361,88.874699,-538.24651)">
+ <title
+ id="title22-7-9">Square</title>
+ <desc
+ id="desc24-7-5">Atomic Queue #1</desc>
+ <v:userDefs>
+ <v:ud
+ v:nameU="visVersion"
+ v:val="VT0(15):26" />
+ </v:userDefs>
+ <v:textBlock
+ v:margins="rect(4,4,4,4)" />
+ <v:textRect
+ cx="30.75"
+ cy="581.25"
+ width="61.5"
+ height="61.5" />
+ <g
+ id="shadow1-2-9-2"
+ v:groupContext="shadow"
+ v:shadowOffsetX="0.345598"
+ v:shadowOffsetY="-1.97279"
+ v:shadowType="1"
+ transform="translate(0.345598,1.97279)"
+ class="st1"
+ style="visibility:visible">
+ <rect
+ x="0"
+ y="550.5"
+ width="61.5"
+ height="61.5"
+ class="st2"
+ id="rect27-8-1"
+ style="fill:#5b9bd5;fill-opacity:0.22000002;stroke:#5b9bd5;stroke-opacity:0.22000002;filter:url(#filter_2-3-6)" />
+ </g>
+ <g
+ id="g13515-3">
+ <g
+ id="g13534-0">
+ <rect
+ x="0"
+ y="550.5"
+ width="61.5"
+ height="61.5"
+ class="st3"
+ id="rect29-1-9"
+ style="fill:#5b9bd5;stroke:#c7c8c8;stroke-width:0.25" />
+ </g>
+ </g>
+ </g>
+ <path
+ style="fill:none;stroke:#000000;stroke-width:0.743466px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1;marker-mid:none;marker-end:url(#Arrow1Lend)"
+ d="M 220.66064,98.57125 H 321.88592"
+ id="path17209-8"
+ inkscape:connector-type="orthogonal"
+ inkscape:connector-curvature="0" />
+ <g
+ style="font-size:12px;fill:none;stroke-linecap:square;stroke-miterlimit:3;overflow:visible"
+ id="shape1-1-2"
+ v:mID="1"
+ v:groupContext="shape"
+ transform="matrix(2.1604167,0,0,1.5671361,322.24227,-811.89589)">
+ <title
+ id="title22-7">Square</title>
+ <desc
+ id="desc24-7">Atomic Queue #1</desc>
+ <v:userDefs>
+ <v:ud
+ v:nameU="visVersion"
+ v:val="VT0(15):26" />
+ </v:userDefs>
+ <v:textBlock
+ v:margins="rect(4,4,4,4)" />
+ <v:textRect
+ cx="30.75"
+ cy="581.25"
+ width="61.5"
+ height="61.5" />
+ <g
+ id="shadow1-2-9"
+ v:groupContext="shadow"
+ v:shadowOffsetX="0.345598"
+ v:shadowOffsetY="-1.97279"
+ v:shadowType="1"
+ transform="translate(0.345598,1.97279)"
+ class="st1"
+ style="visibility:visible">
+ <rect
+ x="0"
+ y="550.5"
+ width="61.5"
+ height="61.5"
+ class="st2"
+ id="rect27-8"
+ style="fill:#5b9bd5;fill-opacity:0.22000002;stroke:#5b9bd5;stroke-opacity:0.22000002;filter:url(#filter_2-3)" />
+ </g>
+ <g
+ id="g13515">
+ <g
+ id="g13534">
+ <rect
+ x="0"
+ y="550.5"
+ width="61.5"
+ height="61.5"
+ class="st3"
+ id="rect29-1"
+ style="fill:#5b9bd5;stroke:#c7c8c8;stroke-width:0.25" />
+ </g>
+ </g>
+ </g>
+ <g
+ id="g13518"
+ transform="matrix(0.73232502,0,0,0.75477602,25.29268,348.89752)">
+ <g
+ id="g13526">
+ <flowRoot
+ xml:space="preserve"
+ id="flowRoot13464-9"
+ style="font-style:normal;font-weight:normal;line-height:0.01%;font-family:sans-serif;text-align:center;letter-spacing:0px;word-spacing:0px;text-anchor:middle;fill:#000000;fill-opacity:1;stroke:none"
+ transform="translate(-12.00521,-129.65179)"><flowRegion
+ id="flowRegion13466-1"
+ style="font-family:sans-serif"><rect
+ id="rect13468-2"
+ width="195.99997"
+ height="112.00001"
+ x="273.33334"
+ y="175.33333"
+ style="text-align:center;text-anchor:middle" /></flowRegion><flowPara
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:20px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:center;writing-mode:lr-tb;text-anchor:middle"
+ id="flowPara13511"> </flowPara></flowRoot>
+ </g>
+ </g>
+ <path
+ style="fill:none;stroke:#000000;stroke-width:0.751455;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:4;stroke-dasharray:none;stroke-opacity:1;marker-start:url(#Arrow1Lstart);marker-end:none"
+ d="m 176.26096,124.64833 v 69.24854"
+ id="path17209-3"
+ inkscape:connector-type="orthogonal"
+ inkscape:connector-curvature="0" />
+ <path
+ transform="matrix(0.73232502,0,0,0.75477602,-4.325033,50.642983)"
+ sodipodi:type="arc"
+ style="fill:#539de6;fill-opacity:1;stroke:#0000ea;stroke-width:1;stroke-linecap:square;stroke-miterlimit:4;stroke-opacity:1;stroke-dasharray:none"
+ id="path29161-3"
+ sodipodi:cx="371"
+ sodipodi:cy="64.5"
+ sodipodi:rx="17"
+ sodipodi:ry="15.5"
+ d="m 388,64.5 a 17,15.5 0 1 1 -34,0 17,15.5 0 1 1 34,0 z" />
+ <path
+ style="fill:none;stroke:#000000;stroke-width:0.743466px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1;marker-mid:none;marker-end:url(#Arrow1Lend)"
+ d="M 322.61264,375 H 221.38736"
+ id="path17209-8-0"
+ inkscape:connector-type="orthogonal"
+ inkscape:connector-curvature="0" />
+ <path
+ transform="matrix(0.73232502,0,0,0.75477602,0.0689171,324.80833)"
+ sodipodi:type="arc"
+ style="fill:#539de6;fill-opacity:1;stroke:#0000ea;stroke-width:1;stroke-linecap:square;stroke-miterlimit:4;stroke-opacity:1;stroke-dasharray:none"
+ id="path29161-1"
+ sodipodi:cx="371"
+ sodipodi:cy="64.5"
+ sodipodi:rx="17"
+ sodipodi:ry="15.5"
+ d="m 388,64.5 a 17,15.5 0 1 1 -34,0 17,15.5 0 1 1 34,0 z" />
+ <path
+ style="fill:none;stroke:#000000;stroke-width:0.629082px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1;marker-end:url(#Arrow1Lend)"
+ d="M 155,324.19955 V 264.82863"
+ id="path17211-7-1"
+ inkscape:connector-type="orthogonal"
+ inkscape:connector-curvature="0" />
+ <path
+ transform="matrix(0.73232502,0,0,0.75477602,-116.37076,245.97088)"
+ sodipodi:type="arc"
+ style="fill:#539de6;fill-opacity:1;stroke:#0000ea;stroke-width:1;stroke-linecap:square;stroke-miterlimit:4;stroke-opacity:1;stroke-dasharray:none"
+ id="path29161-8"
+ sodipodi:cx="371"
+ sodipodi:cy="64.5"
+ sodipodi:rx="17"
+ sodipodi:ry="15.5"
+ d="m 388,64.5 a 17,15.5 0 1 1 -34,0 17,15.5 0 1 1 34,0 z" />
+ <path
+ style="fill:none;stroke:#000000;stroke-width:0.75059;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:4;stroke-dasharray:none;stroke-opacity:1;marker-start:url(#Arrow1Lstart-7);marker-end:none"
+ d="m 126.26097,124.99178 v 69.24941"
+ id="path17209-3-0"
+ inkscape:connector-type="orthogonal"
+ inkscape:connector-curvature="0" />
+ <path
+ sodipodi:type="arc"
+ style="fill:#539de6;fill-opacity:1;stroke:#0000ea;stroke-width:1;stroke-linecap:square;stroke-miterlimit:4;stroke-opacity:1;stroke-dasharray:none"
+ id="path29161"
+ sodipodi:cx="371"
+ sodipodi:cy="64.5"
+ sodipodi:rx="17"
+ sodipodi:ry="15.5"
+ d="m 388,64.5 a 17,15.5 0 1 1 -34,0 17,15.5 0 1 1 34,0 z"
+ transform="matrix(0.73232502,0,0,0.75477602,-146.16394,110.68276)" />
+ <path
+ sodipodi:type="arc"
+ style="fill:#539de6;fill-opacity:1;stroke:#0000ea;stroke-width:1;stroke-linecap:square;stroke-miterlimit:4;stroke-opacity:1;stroke-dasharray:none"
+ id="path29161-2"
+ sodipodi:cx="371"
+ sodipodi:cy="64.5"
+ sodipodi:rx="17"
+ sodipodi:ry="15.5"
+ d="m 388,64.5 a 17,15.5 0 1 1 -34,0 17,15.5 0 1 1 34,0 z"
+ transform="matrix(0.73232502,0,0,0.75477602,-95.40136,110.71627)" />
+ <g
+ id="g29167-4"
+ transform="matrix(0.73232502,0,0,0.75477602,-95.31662,112.00756)">
+ <text
+ id="text29163-9"
+ y="70"
+ x="365"
+ style="font-style:normal;font-weight:normal;line-height:0%;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none"
+ xml:space="preserve"><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="70"
+ x="365"
+ id="tspan29165-9"
+ sodipodi:role="line">1</tspan></text>
+ </g>
+ <g
+ id="g29167-9"
+ transform="matrix(0.73232502,0,0,0.75477602,-4.9726112,50.689051)">
+ <text
+ id="text29163-3"
+ y="70"
+ x="365"
+ style="font-style:normal;font-weight:normal;line-height:0%;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none"
+ xml:space="preserve"><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="70"
+ x="365"
+ id="tspan29165-3"
+ sodipodi:role="line">2</tspan></text>
+ </g>
+ <path
+ style="fill:none;stroke:#000000;stroke-width:1.04033px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1;marker-end:url(#Arrow1Lend)"
+ d="M 388.20118,147.93341 V 321.89308"
+ id="path17211-7"
+ inkscape:connector-type="orthogonal"
+ inkscape:connector-curvature="0" />
+ <path
+ transform="matrix(0.73232502,0,0,0.75477602,116.5086,136.68745)"
+ sodipodi:type="arc"
+ style="fill:#539de6;fill-opacity:1;stroke:#0000ea;stroke-width:1;stroke-linecap:square;stroke-miterlimit:4;stroke-opacity:1;stroke-dasharray:none"
+ id="path29161-6"
+ sodipodi:cx="371"
+ sodipodi:cy="64.5"
+ sodipodi:rx="17"
+ sodipodi:ry="15.5"
+ d="m 388,64.5 a 17,15.5 0 1 1 -34,0 17,15.5 0 1 1 34,0 z" />
+ <g
+ id="g29167-46"
+ transform="matrix(0.73232502,0,0,0.75477602,116.59334,137.48829)">
+ <text
+ id="text29163-1"
+ y="70"
+ x="365"
+ style="font-style:normal;font-weight:normal;line-height:0%;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none"
+ xml:space="preserve"><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="70"
+ x="365"
+ id="tspan29165-5"
+ sodipodi:role="line">3</tspan></text>
+ </g>
+ <g
+ id="g29167-6"
+ transform="matrix(0.73232502,0,0,0.75477602,0.1536639,325.60918)">
+ <text
+ id="text29163-17"
+ y="70"
+ x="365"
+ style="font-style:normal;font-weight:normal;line-height:0%;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none"
+ xml:space="preserve"><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="70"
+ x="365"
+ id="tspan29165-2"
+ sodipodi:role="line">4</tspan></text>
+ </g>
+ <g
+ id="g29167"
+ transform="matrix(0.73232502,0,0,0.75477602,-146.07919,111.10621)">
+ <text
+ id="text29163"
+ y="70"
+ x="365"
+ style="font-style:normal;font-weight:normal;line-height:0%;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none"
+ xml:space="preserve"><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="70"
+ x="365"
+ id="tspan29165"
+ sodipodi:role="line">6</tspan></text>
+ </g>
+ <g
+ id="g29167-4-3"
+ transform="matrix(0.73232502,0,0,0.75477602,-117.60784,180.11779)">
+ <text
+ id="text29163-9-6"
+ y="70"
+ x="321.30356"
+ style="font-style:normal;font-weight:normal;line-height:0%;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none"
+ xml:space="preserve"><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:24.2109px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="70"
+ x="321.30356"
+ id="tspan29165-9-7"
+ sodipodi:role="line">Eventdev</tspan></text>
+ </g>
+ <g
+ id="g29167-4-3-5"
+ transform="matrix(0.73232502,0,0,0.75477602,55.34956,26.97821)">
+ <text
+ id="text29163-9-6-3"
+ y="70"
+ x="454.74152"
+ style="font-style:normal;font-weight:normal;line-height:0%;font-family:sans-serif;text-align:center;letter-spacing:0px;word-spacing:0px;text-anchor:middle;fill:#000000;fill-opacity:1;stroke:none"
+ xml:space="preserve"><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:21.5208px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:center;writing-mode:lr-tb;text-anchor:middle"
+ y="70"
+ x="454.74152"
+ id="tspan29165-9-7-5"
+ sodipodi:role="line">A<tspan
+ style="line-height:100%;font-family:sans-serif"
+ id="tspan3374">tomic Stage</tspan></tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:21.5208px;line-height:100%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:center;writing-mode:lr-tb;text-anchor:middle"
+ y="94.210899"
+ x="454.74152"
+ sodipodi:role="line"
+ id="tspan3320">+</tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:21.5208px;line-height:100%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:center;writing-mode:lr-tb;text-anchor:middle"
+ y="115.7317"
+ x="454.74152"
+ sodipodi:role="line"
+ id="tspan3322">Enqueue to</tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:21.5208px;line-height:100%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:center;writing-mode:lr-tb;text-anchor:middle"
+ y="137.2525"
+ x="454.74152"
+ sodipodi:role="line"
+ id="tspan3324">DMA Device</tspan></text>
+ </g>
+ <g
+ id="g29167-2"
+ transform="matrix(0.73232502,0,0,0.75477602,-116.28602,248.01695)">
+ <text
+ id="text29163-92"
+ y="70"
+ x="365"
+ style="font-style:normal;font-weight:normal;line-height:0%;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none"
+ xml:space="preserve"><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="70"
+ x="365"
+ id="tspan29165-8"
+ sodipodi:role="line">5</tspan></text>
+ </g>
+ <flowRoot
+ xml:space="preserve"
+ id="flowRoot3376"
+ style="font-style:normal;font-weight:normal;line-height:0.01%;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none;stroke-width:1px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1"><flowRegion
+ id="flowRegion3378"
+ style="font-family:sans-serif"><rect
+ id="rect3380"
+ width="100"
+ height="37"
+ x="109"
+ y="259" /></flowRegion><flowPara
+ id="flowPara3382"
+ style="font-size:18px;line-height:1.25;font-family:sans-serif"> </flowPara></flowRoot>
+ <g
+ id="g29167-4-3-1"
+ transform="matrix(0.73232502,0,0,0.75477602,109.34956,323.97821)">
+ <text
+ id="text29163-9-6-8"
+ y="70"
+ x="321.30356"
+ style="font-style:normal;font-weight:normal;line-height:0%;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none"
+ xml:space="preserve"><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:24.2109px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="70"
+ x="321.30356"
+ id="tspan29165-9-7-7"
+ sodipodi:role="line">DMA Device</tspan></text>
+ </g>
+ <g
+ id="g29167-4-3-1-9"
+ transform="matrix(0.73232502,0,0,0.75477602,-114.48565,314.20704)">
+ <text
+ id="text29163-9-6-8-2"
+ y="70"
+ x="368.01718"
+ style="font-style:normal;font-weight:normal;line-height:0%;font-family:sans-serif;text-align:center;letter-spacing:0px;word-spacing:0px;text-anchor:middle;fill:#000000;fill-opacity:1;stroke:none"
+ xml:space="preserve"><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:24.2109px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:center;writing-mode:lr-tb;text-anchor:middle"
+ y="70"
+ x="368.01718"
+ id="tspan29165-9-7-7-0"
+ sodipodi:role="line">DMA</tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:24.2109px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:center;writing-mode:lr-tb;text-anchor:middle"
+ y="100.26363"
+ x="368.01718"
+ sodipodi:role="line"
+ id="tspan3488">Adapter</tspan></text>
+ </g>
+ <g
+ id="g29167-4-3-1-9-2"
+ transform="matrix(0.73232502,0,0,0.75477602,250.96804,192.62529)">
+ <text
+ id="text29163-9-6-8-2-3"
+ y="-188.35481"
+ x="318.61978"
+ style="font-style:normal;font-weight:normal;line-height:0%;font-family:sans-serif;text-align:center;letter-spacing:0px;word-spacing:0px;text-anchor:middle;fill:#000000;fill-opacity:1;stroke:none"
+ xml:space="preserve"><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:21.5208px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="-188.35481"
+ x="318.61978"
+ sodipodi:role="line"
+ id="tspan3543">1. Application dequeues</tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:21.5208px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="-161.45381"
+ x="318.61978"
+ sodipodi:role="line"
+ id="tspan3196"> events from the previous</tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:21.5208px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="-134.55281"
+ x="318.61978"
+ sodipodi:role="line"
+ id="tspan3232"> stage</tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:21.5208px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="-107.65182"
+ x="318.61978"
+ sodipodi:role="line"
+ id="tspan3519"> </tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:21.5208px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="-80.750816"
+ x="318.61978"
+ sodipodi:role="line"
+ id="tspan3551">2. Application prepares the</tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:21.5208px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="-53.849815"
+ x="318.61978"
+ sodipodi:role="line"
+ id="tspan3203"> DMA operations.</tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:21.5208px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="-26.948814"
+ x="318.61978"
+ sodipodi:role="line"
+ id="tspan3523"> </tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:21.5208px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="-0.0478158"
+ x="318.61978"
+ sodipodi:role="line"
+ id="tspan3541">3. DMA operations are</tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:21.5208px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="26.853184"
+ x="318.61978"
+ sodipodi:role="line"
+ id="tspan3207"> submitted to dmadev</tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:21.5208px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="53.754185"
+ x="318.61978"
+ sodipodi:role="line"
+ id="tspan3209"> by application.</tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:21.5208px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="80.655182"
+ x="318.61978"
+ sodipodi:role="line"
+ id="tspan3527"> </tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:21.5208px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="107.55618"
+ x="318.61978"
+ sodipodi:role="line"
+ id="tspan3547">4. DMA adapter dequeues</tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:21.5208px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="134.45718"
+ x="318.61978"
+ sodipodi:role="line"
+ id="tspan3216"> DMA completions from</tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:21.5208px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="161.35818"
+ x="318.61978"
+ sodipodi:role="line"
+ id="tspan3218"> DMA device.</tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:21.5208px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="188.25919"
+ x="318.61978"
+ sodipodi:role="line"
+ id="tspan3531"> </tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:21.5208px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="215.16019"
+ x="318.61978"
+ sodipodi:role="line"
+ id="tspan3549">5. DMA adapter enqueues</tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:21.5208px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="242.06119"
+ x="318.61978"
+ sodipodi:role="line"
+ id="tspan3222"> events to the eventdev.</tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:21.5208px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="268.96219"
+ x="318.61978"
+ sodipodi:role="line"
+ id="tspan3535"> </tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:21.5208px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="295.86319"
+ x="318.61978"
+ sodipodi:role="line"
+ id="tspan3537">6. Application dequeues from</tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:21.5208px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="322.76419"
+ x="318.61978"
+ sodipodi:role="line"
+ id="tspan3224"> eventdev and prepare for</tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:21.5208px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="349.66519"
+ x="318.61978"
+ sodipodi:role="line"
+ id="tspan3226"> further processing</tspan></text>
+ </g>
+ <g
+ style="font-size:12px;fill:none;stroke-linecap:square;stroke-miterlimit:3;overflow:visible"
+ id="shape1-1-2-4-7"
+ v:mID="1"
+ v:groupContext="shape"
+ transform="matrix(2.1604167,0,0,1.165886,90.820551,-587.97129)">
+ <title
+ id="title22-7-5-5">Square</title>
+ <desc
+ id="desc24-7-8-3">Atomic Queue #1</desc>
+ <v:userDefs>
+ <v:ud
+ v:nameU="visVersion"
+ v:val="VT0(15):26" />
+ </v:userDefs>
+ <v:textBlock
+ v:margins="rect(4,4,4,4)" />
+ <v:textRect
+ cx="30.75"
+ cy="581.25"
+ width="61.5"
+ height="61.5" />
+ <g
+ id="shadow1-2-9-5-5"
+ v:groupContext="shadow"
+ v:shadowOffsetX="0.345598"
+ v:shadowOffsetY="-1.97279"
+ v:shadowType="1"
+ transform="translate(0.345598,1.97279)"
+ class="st1"
+ style="visibility:visible">
+ <rect
+ x="0"
+ y="550.5"
+ width="61.5"
+ height="61.5"
+ class="st2"
+ id="rect27-8-7-6"
+ style="fill:#5b9bd5;fill-opacity:0.22000002;stroke:#5b9bd5;stroke-opacity:0.22000002;filter:url(#filter_2-3-91-3)" />
+ </g>
+ <g
+ id="g13515-33-2">
+ <g
+ id="g13534-8-9">
+ <rect
+ x="0"
+ y="550.5"
+ width="61.5"
+ height="61.5"
+ class="st3"
+ id="rect29-1-95-1"
+ style="fill:#5b9bd5;stroke:#c7c8c8;stroke-width:0.25" />
+ </g>
+ </g>
+ </g>
+ <g
+ id="g29167-4-3-2"
+ transform="matrix(0.73232502,0,0,0.75477602,-125.66199,44.027402)">
+ <text
+ id="text29163-9-6-7"
+ y="70"
+ x="321.30356"
+ style="font-style:normal;font-weight:normal;line-height:0%;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none"
+ xml:space="preserve"><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:24.2109px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="70"
+ x="321.30356"
+ id="tspan29165-9-7-0"
+ sodipodi:role="line">Application</tspan></text>
+ </g>
+ </g>
+</svg>
diff --git a/doc/guides/prog_guide/index.rst b/doc/guides/prog_guide/index.rst
index 52a6d9e7aa..beaa4b8869 100644
--- a/doc/guides/prog_guide/index.rst
+++ b/doc/guides/prog_guide/index.rst
@@ -60,6 +60,7 @@ Programmer's Guide
event_ethernet_tx_adapter
event_timer_adapter
event_crypto_adapter
+ event_dma_adapter
qos_framework
power_man
packet_classif_access_ctrl
diff --git a/doc/guides/rel_notes/release_23_11.rst b/doc/guides/rel_notes/release_23_11.rst
index b34ddc0860..70f2e7e03c 100644
--- a/doc/guides/rel_notes/release_23_11.rst
+++ b/doc/guides/rel_notes/release_23_11.rst
@@ -44,7 +44,9 @@ New Features
.. This section should contain new features added in this release.
Sample format:
- * **Add a title in the past tense with a full stop.**
+ * **Added eventdev DMA adapter library.**
+
+ Added API to DMA transfer data using event mechanism.
Add a short 1-2 sentence description in the past tense.
The description should be enough to allow someone scanning
diff --git a/lib/eventdev/eventdev_pmd.h b/lib/eventdev/eventdev_pmd.h
index f62f42e140..8e5fd420f6 100644
--- a/lib/eventdev/eventdev_pmd.h
+++ b/lib/eventdev/eventdev_pmd.h
@@ -178,8 +178,12 @@ struct rte_eventdev {
event_tx_adapter_enqueue_t txa_enqueue;
/**< Pointer to PMD eth Tx adapter enqueue function. */
event_crypto_adapter_enqueue_t ca_enqueue;
+ /**< Pointer to PMD crypto adapter enqueue function. */
- uint64_t reserved_64s[4]; /**< Reserved for future fields */
+ event_dma_adapter_enqueue_t dma_enqueue;
+ /**< Pointer to PMD DMA adapter enqueue function. */
+
+ uint64_t reserved_64s[3]; /**< Reserved for future fields */
void *reserved_ptrs[3]; /**< Reserved for future fields */
} __rte_cache_aligned;
@@ -1320,6 +1324,160 @@ typedef int (*eventdev_eth_tx_adapter_queue_stop)
#define eventdev_stop_flush_t rte_eventdev_stop_flush_t
+/**
+ * Retrieve the event device's DMA adapter capabilities for the
+ * specified DMA device
+ *
+ * @param dev
+ * Event device pointer
+ *
+ * @param dma_dev_id
+ * DMA device identification
+ *
+ * @param[out] caps
+ * A pointer to memory filled with event adapter capabilities.
+ * It is expected to be pre-allocated & initialized by caller.
+ *
+ * @return
+ * - 0: Success, driver provides event adapter capabilities for the
+ * DMADEV.
+ * - <0: Error code returned by the driver function.
+ *
+ */
+typedef int (*eventdev_dma_adapter_caps_get_t)(const struct rte_eventdev *dev,
+ const int16_t dma_dev_id, uint32_t *caps);
+
+/**
+ * This API may change without prior notice
+ *
+ * Add DMA vchan queue to event device. This callback is invoked if
+ * the caps returned from rte_event_dma_adapter_caps_get(, dmadev_id)
+ * has RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_* set.
+ *
+ * @param dev
+ * Event device pointer
+ *
+ * @param dma_dev_id
+ * DMADEV identification
+ *
+ * @param vchan_id
+ * DMADEV vchan queue identifier.
+ *
+ * @param event
+ * Event information required for binding dmadev queue pair to event queue.
+ * This structure will have a valid value for only those HW PMDs supporting
+ * @see RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_VCHAN_EV_BIND capability.
+ *
+ * @return
+ * - 0: Success, dmadev vchan added successfully.
+ * - <0: Error code returned by the driver function.
+ *
+ */
+typedef int (*eventdev_dma_adapter_vchan_add_t)(const struct rte_eventdev *dev,
+ const int16_t dma_dev_id,
+ uint16_t vchan_id,
+ const struct rte_event *event);
+
+/**
+ * This API may change without prior notice
+ *
+ * Delete DMA vhcan to event device. This callback is invoked if
+ * the caps returned from rte_event_dma_adapter_caps_get(, dmadev_id)
+ * has RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_* set.
+ *
+ * @param dev
+ * Event device pointer
+ *
+ * @param dma_dev_id
+ * DMADEV identification
+ *
+ * @param vchan_id
+ * dmadev vchan identifier.
+ *
+ * @return
+ * - 0: Success, dmadev vchan deleted successfully.
+ * - <0: Error code returned by the driver function.
+ *
+ */
+typedef int (*eventdev_dma_adapter_vchan_del_t)(const struct rte_eventdev *dev,
+ const int16_t dma_dev_id,
+ uint16_t vchan_id);
+
+/**
+ * Start DMA adapter. This callback is invoked if
+ * the caps returned from rte_event_dma_adapter_caps_get(.., dmadev_id)
+ * has RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_* set and vchan for dmadev_id
+ * have been added to the event device.
+ *
+ * @param dev
+ * Event device pointer
+ *
+ * @param dma_dev_id
+ * DMA device identification
+ *
+ * @return
+ * - 0: Success, DMA adapter started successfully.
+ * - <0: Error code returned by the driver function.
+ */
+typedef int (*eventdev_dma_adapter_start_t)(const struct rte_eventdev *dev,
+ const int16_t dma_dev_id);
+
+/**
+ * Stop DMA adapter. This callback is invoked if
+ * the caps returned from rte_event_dma_adapter_caps_get(.., dmadev_id)
+ * has RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_* set and vchan for dmadev_id
+ * have been added to the event device.
+ *
+ * @param dev
+ * Event device pointer
+ *
+ * @param dma_dev_id
+ * DMA device identification
+ *
+ * @return
+ * - 0: Success, DMA adapter stopped successfully.
+ * - <0: Error code returned by the driver function.
+ */
+typedef int (*eventdev_dma_adapter_stop_t)(const struct rte_eventdev *dev,
+ const int16_t dma_dev_id);
+
+struct rte_event_dma_adapter_stats;
+
+/**
+ * Retrieve DMA adapter statistics.
+ *
+ * @param dev
+ * Event device pointer
+ *
+ * @param dma_dev_id
+ * DMA device identification
+ *
+ * @param[out] stats
+ * Pointer to stats structure
+ *
+ * @return
+ * Return 0 on success.
+ */
+typedef int (*eventdev_dma_adapter_stats_get)(const struct rte_eventdev *dev,
+ const int16_t dma_dev_id,
+ struct rte_event_dma_adapter_stats *stats);
+
+/**
+ * Reset DMA adapter statistics.
+ *
+ * @param dev
+ * Event device pointer
+ *
+ * @param dma_dev_id
+ * DMA device identification
+ *
+ * @return
+ * Return 0 on success.
+ */
+typedef int (*eventdev_dma_adapter_stats_reset)(const struct rte_eventdev *dev,
+ const int16_t dma_dev_id);
+
+
/** Event device operations function pointer table */
struct eventdev_ops {
eventdev_info_get_t dev_infos_get; /**< Get device info. */
@@ -1440,6 +1598,21 @@ struct eventdev_ops {
eventdev_eth_tx_adapter_queue_stop eth_tx_adapter_queue_stop;
/**< Stop Tx queue assigned to Tx adapter instance */
+ eventdev_dma_adapter_caps_get_t dma_adapter_caps_get;
+ /**< Get DMA adapter capabilities */
+ eventdev_dma_adapter_vchan_add_t dma_adapter_vchan_add;
+ /**< Add vchan queue to DMA adapter */
+ eventdev_dma_adapter_vchan_del_t dma_adapter_vchan_del;
+ /**< Delete vchan queue from DMA adapter */
+ eventdev_dma_adapter_start_t dma_adapter_start;
+ /**< Start DMA adapter */
+ eventdev_dma_adapter_stop_t dma_adapter_stop;
+ /**< Stop DMA adapter */
+ eventdev_dma_adapter_stats_get dma_adapter_stats_get;
+ /**< Get DMA stats */
+ eventdev_dma_adapter_stats_reset dma_adapter_stats_reset;
+ /**< Reset DMA stats */
+
eventdev_selftest dev_selftest;
/**< Start eventdev Selftest */
diff --git a/lib/eventdev/eventdev_private.c b/lib/eventdev/eventdev_private.c
index 1d3d9d357e..18ed8bf3c8 100644
--- a/lib/eventdev/eventdev_private.c
+++ b/lib/eventdev/eventdev_private.c
@@ -81,6 +81,14 @@ dummy_event_crypto_adapter_enqueue(__rte_unused void *port,
return 0;
}
+static uint16_t
+dummy_event_dma_adapter_enqueue(__rte_unused void *port, __rte_unused struct rte_event ev[],
+ __rte_unused uint16_t nb_events)
+{
+ RTE_EDEV_LOG_ERR("event DMA adapter enqueue requested for unconfigured event device");
+ return 0;
+}
+
void
event_dev_fp_ops_reset(struct rte_event_fp_ops *fp_op)
{
@@ -97,6 +105,7 @@ event_dev_fp_ops_reset(struct rte_event_fp_ops *fp_op)
.txa_enqueue_same_dest =
dummy_event_tx_adapter_enqueue_same_dest,
.ca_enqueue = dummy_event_crypto_adapter_enqueue,
+ .dma_enqueue = dummy_event_dma_adapter_enqueue,
.data = dummy_data,
};
@@ -117,5 +126,6 @@ event_dev_fp_ops_set(struct rte_event_fp_ops *fp_op,
fp_op->txa_enqueue = dev->txa_enqueue;
fp_op->txa_enqueue_same_dest = dev->txa_enqueue_same_dest;
fp_op->ca_enqueue = dev->ca_enqueue;
+ fp_op->dma_enqueue = dev->dma_enqueue;
fp_op->data = dev->data->ports;
}
diff --git a/lib/eventdev/meson.build b/lib/eventdev/meson.build
index 6edf98dfa5..21347f7c4c 100644
--- a/lib/eventdev/meson.build
+++ b/lib/eventdev/meson.build
@@ -25,6 +25,7 @@ sources = files(
)
headers = files(
'rte_event_crypto_adapter.h',
+ 'rte_event_dma_adapter.h',
'rte_event_eth_rx_adapter.h',
'rte_event_eth_tx_adapter.h',
'rte_event_ring.h',
diff --git a/lib/eventdev/rte_event_dma_adapter.h b/lib/eventdev/rte_event_dma_adapter.h
new file mode 100644
index 0000000000..d413d31a61
--- /dev/null
+++ b/lib/eventdev/rte_event_dma_adapter.h
@@ -0,0 +1,582 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright (c) 2023 Marvell.
+ */
+
+#ifndef RTE_EVENT_DMA_ADAPTER
+#define RTE_EVENT_DMA_ADAPTER
+
+/**
+ * @file rte_event_dma_adapter.h
+ *
+ * @warning
+ * @b EXPERIMENTAL:
+ * All functions in this file may be changed or removed without prior notice.
+ *
+ * DMA Event Adapter API.
+ *
+ * Eventdev library provides adapters to bridge between various components for providing new
+ * event source. The event DMA adapter is one of those adapters which is intended to bridge
+ * between event devices and DMA devices.
+ *
+ * The DMA adapter adds support to enqueue / dequeue DMA operations to / from event device. The
+ * packet flow between DMA device and the event device can be accomplished using both SW and HW
+ * based transfer mechanisms. The adapter uses an EAL service core function for SW based packet
+ * transfer and uses the eventdev PMD functions to configure HW based packet transfer between the
+ * DMA device and the event device.
+ *
+ * The application can choose to submit a DMA operation directly to an DMA device or send it to the
+ * DMA adapter via eventdev based on RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP_FWD capability. The
+ * first mode is known as the event new (RTE_EVENT_DMA_ADAPTER_OP_NEW) mode and the second as the
+ * event forward (RTE_EVENT_DMA_ADAPTER_OP_FORWARD) mode. The choice of mode can be specified while
+ * creating the adapter. In the former mode, it is an application responsibility to enable ingress
+ * packet ordering. In the latter mode, it is the adapter responsibility to enable the ingress
+ * packet ordering.
+ *
+ *
+ * Working model of RTE_EVENT_DMA_ADAPTER_OP_NEW mode:
+ *
+ * +--------------+ +--------------+
+ * | | | DMA stage |
+ * | Application |---[2]-->| + enqueue to |
+ * | | | dmadev |
+ * +--------------+ +--------------+
+ * ^ ^ |
+ * | | [3]
+ * [6] [1] |
+ * | | |
+ * +--------------+ |
+ * | | |
+ * | Event device | |
+ * | | |
+ * +--------------+ |
+ * ^ |
+ * | |
+ * [5] |
+ * | v
+ * +--------------+ +--------------+
+ * | | | |
+ * | DMA adapter |<--[4]---| dmadev |
+ * | | | |
+ * +--------------+ +--------------+
+ *
+ *
+ * [1] Application dequeues events from the previous stage.
+ * [2] Application prepares the DMA operations.
+ * [3] DMA operations are submitted to dmadev by application.
+ * [4] DMA adapter dequeues DMA completions from dmadev.
+ * [5] DMA adapter enqueues events to the eventdev.
+ * [6] Application dequeues from eventdev for further processing.
+ *
+ * In the RTE_EVENT_DMA_ADAPTER_OP_NEW mode, application submits DMA operations directly to DMA
+ * device. The DMA adapter then dequeues DMA completions from DMA device and enqueue events to the
+ * event device. This mode does not ensure ingress ordering, if the application directly enqueues
+ * to dmadev without going through DMA / atomic stage i.e. removing item [1] and [2].
+ *
+ * Events dequeued from the adapter will be treated as new events. In this mode, application needs
+ * to specify event information (response information) which is needed to enqueue an event after the
+ * DMA operation is completed.
+ *
+ *
+ * Working model of RTE_EVENT_DMA_ADAPTER_OP_FORWARD mode:
+ *
+ * +--------------+ +--------------+
+ * --[1]-->| |---[2]-->| Application |
+ * | Event device | | in |
+ * <--[8]--| |<--[3]---| Ordered stage|
+ * +--------------+ +--------------+
+ * ^ |
+ * | [4]
+ * [7] |
+ * | v
+ * +----------------+ +--------------+
+ * | |--[5]->| |
+ * | DMA adapter | | dmadev |
+ * | |<-[6]--| |
+ * +----------------+ +--------------+
+ *
+ *
+ * [1] Events from the previous stage.
+ * [2] Application in ordered stage dequeues events from eventdev.
+ * [3] Application enqueues DMA operations as events to eventdev.
+ * [4] DMA adapter dequeues event from eventdev.
+ * [5] DMA adapter submits DMA operations to dmadev (Atomic stage).
+ * [6] DMA adapter dequeues DMA completions from dmadev
+ * [7] DMA adapter enqueues events to the eventdev
+ * [8] Events to the next stage
+ *
+ * In the event forward (RTE_EVENT_DMA_ADAPTER_OP_FORWARD) mode, if the HW supports the capability
+ * RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP_FWD, application can directly submit the DMA
+ * operations to the dmadev. If not, application retrieves the event port of the DMA adapter
+ * through the API, rte_event_DMA_adapter_event_port_get(). Then, links its event queue to this
+ * port and starts enqueuing DMA operations as events to the eventdev. The adapter then dequeues
+ * the events and submits the DMA operations to the dmadev. After the DMA completions, the adapter
+ * enqueues events to the event device.
+ *
+ * Application can use this mode, when ingress packet ordering is needed. Events dequeued from the
+ * adapter will be treated as forwarded events. In this mode, the application needs to specify the
+ * dmadev ID and queue pair ID (request information) needed to enqueue an DMA operation in addition
+ * to the event information (response information) needed to enqueue an event after the DMA
+ * operation has completed.
+ *
+ * The event DMA adapter provides common APIs to configure the packet flow from the DMA device to
+ * event devices for both SW and HW based transfers. The DMA event adapter's functions are:
+ *
+ * - rte_event_dma_adapter_create_ext()
+ * - rte_event_dma_adapter_create()
+ * - rte_event_dma_adapter_free()
+ * - rte_event_dma_adapter_vchan_add()
+ * - rte_event_dma_adapter_vchan_del()
+ * - rte_event_dma_adapter_start()
+ * - rte_event_dma_adapter_stop()
+ * - rte_event_dma_adapter_stats_get()
+ * - rte_event_dma_adapter_stats_reset()
+ *
+ * The application creates an instance using rte_event_dma_adapter_create() or
+ * rte_event_dma_adapter_create_ext().
+ *
+ * dmadev queue pair addition / deletion is done using the rte_event_dma_adapter_vchan_add() /
+ * rte_event_dma_adapter_vchan_del() APIs. If HW supports the capability
+ * RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_VCHAN_EV_BIND, event information must be passed to the
+ * add API.
+ *
+ */
+
+#include <stdint.h>
+
+#include <rte_eventdev.h>
+#include <rte_common.h>
+#include <rte_dmadev_pmd.h>
+
+#ifdef __cplusplus
+extern "C" {
+#endif
+
+/**
+ * A structure used to hold event based DMA operation entry. All the information
+ * required for a DMA transfer shall be populated in "struct rte_event_dma_adapter_op"
+ * instance.
+ */
+struct rte_event_dma_adapter_op {
+ struct rte_dma_sge *src_seg;
+ /**< Source segments. */
+ struct rte_dma_sge *dst_seg;
+ /**< Destination segments. */
+ uint16_t nb_src;
+ /**< Number of source segments. */
+ uint16_t nb_dst;
+ /**< Number of destination segments. */
+ uint64_t flags;
+ /**< Flags related to the operation.
+ * @see RTE_DMA_OP_FLAG_*
+ */
+ int16_t dma_dev_id;
+ /**< DMA device ID to be used */
+ uint16_t vchan;
+ /**< DMA vchan ID to be used */
+ struct rte_mempool *op_mp;
+ /**< Mempool from which op is allocated. */
+};
+
+/**
+ * DMA event adapter mode
+ */
+enum rte_event_dma_adapter_mode {
+ RTE_EVENT_DMA_ADAPTER_OP_NEW,
+ /**< Start the DMA adapter in event new mode.
+ * @see RTE_EVENT_OP_NEW.
+ *
+ * Application submits DMA operations to the dmadev. Adapter only dequeues the DMA
+ * completions from dmadev and enqueue events to the eventdev.
+ */
+
+ RTE_EVENT_DMA_ADAPTER_OP_FORWARD,
+ /**< Start the DMA adapter in event forward mode.
+ * @see RTE_EVENT_OP_FORWARD.
+ *
+ * Application submits DMA requests as events to the DMA adapter or DMA device based on
+ * RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP_FWD capability. DMA completions are enqueued
+ * back to the eventdev by DMA adapter.
+ */
+};
+
+/**
+ * Adapter configuration structure that the adapter configuration callback function is expected to
+ * fill out.
+ *
+ * @see rte_event_dma_adapter_conf_cb
+ */
+struct rte_event_dma_adapter_conf {
+ uint8_t event_port_id;
+ /** < Event port identifier, the adapter enqueues events to this port and dequeues DMA
+ * request events in RTE_EVENT_DMA_ADAPTER_OP_FORWARD mode.
+ */
+
+ uint32_t max_nb;
+ /**< The adapter can return early if it has processed at least max_nb DMA ops. This isn't
+ * treated as a requirement; batching may cause the adapter to process more than max_nb DMA
+ * ops.
+ */
+};
+
+/**
+ * Adapter runtime configuration parameters
+ */
+struct rte_event_dma_adapter_runtime_params {
+ uint32_t max_nb;
+ /**< The adapter can return early if it has processed at least max_nb DMA ops. This isn't
+ * treated as a requirement; batching may cause the adapter to process more than max_nb DMA
+ * ops.
+ *
+ * Callback function passed to rte_event_dma_adapter_create_ext() configures the adapter
+ * with default value of max_nb.
+ * rte_event_dma_adapter_runtime_params_set() allows to re-configure max_nb during runtime
+ * (after adding at least one queue pair)
+ *
+ * This is valid for the devices without RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP_FWD or
+ * RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP_NEW capability.
+ */
+
+ uint32_t rsvd[15];
+ /**< Reserved fields for future expansion */
+};
+
+/**
+ * Function type used for adapter configuration callback. The callback is used to fill in members of
+ * the struct rte_event_dma_adapter_conf, this callback is invoked when creating a SW service for
+ * packet transfer from dmadev vchan to the event device. The SW service is created within the
+ * function, rte_event_dma_adapter_vchan_add(), if SW based packet transfers from dmadev vchan
+ * to the event device are required.
+ *
+ * @param id
+ * Adapter identifier.
+ * @param evdev_id
+ * Event device identifier.
+ * @param conf
+ * Structure that needs to be populated by this callback.
+ * @param arg
+ * Argument to the callback. This is the same as the conf_arg passed to the
+ * rte_event_dma_adapter_create_ext().
+ */
+typedef int (*rte_event_dma_adapter_conf_cb)(uint8_t id, uint8_t evdev_id,
+ struct rte_event_dma_adapter_conf *conf, void *arg);
+
+/**
+ * A structure used to retrieve statistics for an event DMA adapter instance.
+ */
+struct rte_event_dma_adapter_stats {
+ uint64_t event_poll_count;
+ /**< Event port poll count */
+
+ uint64_t event_deq_count;
+ /**< Event dequeue count */
+
+ uint64_t dma_enq_count;
+ /**< dmadev enqueue count */
+
+ uint64_t dma_enq_fail_count;
+ /**< dmadev enqueue failed count */
+
+ uint64_t dma_deq_count;
+ /**< dmadev dequeue count */
+
+ uint64_t event_enq_count;
+ /**< Event enqueue count */
+
+ uint64_t event_enq_retry_count;
+ /**< Event enqueue retry count */
+
+ uint64_t event_enq_fail_count;
+ /**< Event enqueue fail count */
+};
+
+/**
+ * Create a new event DMA adapter with the specified identifier.
+ *
+ * @param id
+ * Adapter identifier.
+ * @param evdev_id
+ * Event device identifier.
+ * @param conf_cb
+ * Callback function that fills in members of a struct rte_event_dma_adapter_conf struct passed
+ * into it.
+ * @param mode
+ * Flag to indicate the mode of the adapter.
+ * @see rte_event_dma_adapter_mode
+ * @param conf_arg
+ * Argument that is passed to the conf_cb function.
+ *
+ * @return
+ * - 0: Success
+ * - <0: Error code on failure
+ */
+__rte_experimental
+int rte_event_dma_adapter_create_ext(uint8_t id, uint8_t evdev_id,
+ rte_event_dma_adapter_conf_cb conf_cb,
+ enum rte_event_dma_adapter_mode mode, void *conf_arg);
+
+/**
+ * Create a new event DMA adapter with the specified identifier. This function uses an internal
+ * configuration function that creates an event port. This default function reconfigures the event
+ * device with an additional event port and set up the event port using the port_config parameter
+ * passed into this function. In case the application needs more control in configuration of the
+ * service, it should use the rte_event_dma_adapter_create_ext() version.
+ *
+ * @param id
+ * Adapter identifier.
+ * @param evdev_id
+ * Event device identifier.
+ * @param port_config
+ * Argument of type *rte_event_port_conf* that is passed to the conf_cb function.
+ * @param mode
+ * Flag to indicate the mode of the adapter.
+ * @see rte_event_dma_adapter_mode
+ *
+ * @return
+ * - 0: Success
+ * - <0: Error code on failure
+ */
+__rte_experimental
+int rte_event_dma_adapter_create(uint8_t id, uint8_t evdev_id,
+ struct rte_event_port_conf *port_config,
+ enum rte_event_dma_adapter_mode mode);
+
+/**
+ * Free an event DMA adapter
+ *
+ * @param id
+ * Adapter identifier.
+ * @return
+ * - 0: Success
+ * - <0: Error code on failure, If the adapter still has queue pairs added to it, the function
+ * returns -EBUSY.
+ */
+__rte_experimental
+int rte_event_dma_adapter_free(uint8_t id);
+
+/**
+ * Retrieve the event port of an adapter.
+ *
+ * @param id
+ * Adapter identifier.
+ *
+ * @param [out] event_port_id
+ * Application links its event queue to this adapter port which is used in
+ * RTE_EVENT_DMA_ADAPTER_OP_FORWARD mode.
+ *
+ * @return
+ * - 0: Success
+ * - <0: Error code on failure.
+ */
+__rte_experimental
+int rte_event_dma_adapter_event_port_get(uint8_t id, uint8_t *event_port_id);
+
+/**
+ * Add a vchan to an event DMA adapter.
+ *
+ * @param id
+ * Adapter identifier.
+ * @param dmadev_id
+ * dmadev identifier.
+ * @param vchan
+ * DMA device vchan identifier. If vchan is set -1, adapter adds all the
+ * preconfigured vchan to the instance.
+ * @param event
+ * If HW supports dmadev vchan to event queue binding, application is expected to fill in
+ * event information, else it will be NULL.
+ * @see RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_VCHAN_EV_BIND
+ *
+ * @return
+ * - 0: Success, vchan added correctly.
+ * - <0: Error code on failure.
+ */
+__rte_experimental
+int rte_event_dma_adapter_vchan_add(uint8_t id, int16_t dmadev_id, uint16_t vchan,
+ const struct rte_event *event);
+
+/**
+ * Delete a vchan from an event DMA adapter.
+ *
+ * @param id
+ * Adapter identifier.
+ * @param dmadev_id
+ * DMA device identifier.
+ * @param vchan
+ * DMA device vchan identifier.
+ *
+ * @return
+ * - 0: Success, vchan deleted successfully.
+ * - <0: Error code on failure.
+ */
+__rte_experimental
+int rte_event_dma_adapter_vchan_del(uint8_t id, int16_t dmadev_id, uint16_t vchan);
+
+/**
+ * Retrieve the service ID of an adapter. If the adapter doesn't use a rte_service function, this
+ * function returns -ESRCH.
+ *
+ * @param id
+ * Adapter identifier.
+ * @param [out] service_id
+ * A pointer to a uint32_t, to be filled in with the service id.
+ *
+ * @return
+ * - 0: Success
+ * - <0: Error code on failure, if the adapter doesn't use a rte_service function, this function
+ * returns -ESRCH.
+ */
+__rte_experimental
+int rte_event_dma_adapter_service_id_get(uint8_t id, uint32_t *service_id);
+
+/**
+ * Start event DMA adapter
+ *
+ * @param id
+ * Adapter identifier.
+ *
+ * @return
+ * - 0: Success, adapter started successfully.
+ * - <0: Error code on failure.
+ *
+ * @note The eventdev and dmadev to which the event_dma_adapter is connected should be started
+ * before calling rte_event_dma_adapter_start().
+ */
+__rte_experimental
+int rte_event_dma_adapter_start(uint8_t id);
+
+/**
+ * Stop event DMA adapter
+ *
+ * @param id
+ * Adapter identifier.
+ *
+ * @return
+ * - 0: Success, adapter stopped successfully.
+ * - <0: Error code on failure.
+ */
+__rte_experimental
+int rte_event_dma_adapter_stop(uint8_t id);
+
+/**
+ * Initialize the adapter runtime configuration parameters
+ *
+ * @param params
+ * A pointer to structure of type struct rte_event_dma_adapter_runtime_params
+ *
+ * @return
+ * - 0: Success
+ * - <0: Error code on failure
+ */
+__rte_experimental
+int rte_event_dma_adapter_runtime_params_init(struct rte_event_dma_adapter_runtime_params *params);
+
+/**
+ * Set the adapter runtime configuration parameters
+ *
+ * @param id
+ * Adapter identifier
+ *
+ * @param params
+ * A pointer to structure of type struct rte_event_dma_adapter_runtime_params with configuration
+ * parameter values. The reserved fields of this structure must be initialized to zero and the valid
+ * fields need to be set appropriately. This struct can be initialized using
+ * rte_event_dma_adapter_runtime_params_init() API to default values or application may reset this
+ * struct and update required fields.
+ *
+ * @return
+ * - 0: Success
+ * - <0: Error code on failure
+ */
+__rte_experimental
+int rte_event_dma_adapter_runtime_params_set(uint8_t id,
+ struct rte_event_dma_adapter_runtime_params *params);
+
+/**
+ * Get the adapter runtime configuration parameters
+ *
+ * @param id
+ * Adapter identifier
+ *
+ * @param[out] params
+ * A pointer to structure of type struct rte_event_dma_adapter_runtime_params containing valid
+ * adapter parameters when return value is 0.
+ *
+ * @return
+ * - 0: Success
+ * - <0: Error code on failure
+ */
+__rte_experimental
+int rte_event_dma_adapter_runtime_params_get(uint8_t id,
+ struct rte_event_dma_adapter_runtime_params *params);
+
+/**
+ * Retrieve statistics for an adapter
+ *
+ * @param id
+ * Adapter identifier.
+ * @param [out] stats
+ * A pointer to structure used to retrieve statistics for an adapter.
+ *
+ * @return
+ * - 0: Success, retrieved successfully.
+ * - <0: Error code on failure.
+ */
+__rte_experimental
+int rte_event_dma_adapter_stats_get(uint8_t id, struct rte_event_dma_adapter_stats *stats);
+
+/**
+ * Reset statistics for an adapter.
+ *
+ * @param id
+ * Adapter identifier.
+ *
+ * @return
+ * - 0: Success, statistics reset successfully.
+ * - <0: Error code on failure.
+ */
+__rte_experimental
+int rte_event_dma_adapter_stats_reset(uint8_t id);
+
+/**
+ * Enqueue a burst of DMA operations as event objects supplied in *rte_event* structure on an event
+ * DMA adapter designated by its event *evdev_id* through the event port specified by *port_id*.
+ * This function is supported if the eventdev PMD has the
+ * #RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP_FWD capability flag set.
+ *
+ * The *nb_events* parameter is the number of event objects to enqueue that are supplied in the
+ * *ev* array of *rte_event* structure.
+ *
+ * The rte_event_dma_adapter_enqueue() function returns the number of event objects it actually
+ * enqueued. A return value equal to *nb_events* means that all event objects have been enqueued.
+ *
+ * @param evdev_id
+ * The identifier of the device.
+ * @param port_id
+ * The identifier of the event port.
+ * @param ev
+ * Points to an array of *nb_events* objects of type *rte_event* structure which contain the
+ * event object enqueue operations to be processed.
+ * @param nb_events
+ * The number of event objects to enqueue, typically number of
+ * rte_event_port_attr_get(...RTE_EVENT_PORT_ATTR_ENQ_DEPTH...) available for this port.
+ *
+ * @return
+ * The number of event objects actually enqueued on the event device. The return value can be
+ * less than the value of the *nb_events* parameter when the event devices queue is full or if
+ * invalid parameters are specified in a *rte_event*. If the return value is less than *nb_events*,
+ * the remaining events at the end of ev[] are not consumed and the caller has to take care of them,
+ * and rte_errno is set accordingly. Possible errno values include:
+ *
+ * - EINVAL: The port ID is invalid, device ID is invalid, an event's queue ID is invalid, or an
+ * event's sched type doesn't match the capabilities of the destination queue.
+ * - ENOSPC: The event port was backpressured and unable to enqueue one or more events. This
+ * error code is only applicable to closed systems.
+ */
+__rte_experimental
+uint16_t rte_event_dma_adapter_enqueue(uint8_t evdev_id, uint8_t port_id, struct rte_event ev[],
+ uint16_t nb_events);
+
+#ifdef __cplusplus
+}
+#endif
+
+#endif /* RTE_EVENT_DMA_ADAPTER */
diff --git a/lib/eventdev/rte_eventdev.h b/lib/eventdev/rte_eventdev.h
index 2ba8a7b090..2c6ecc7f8f 100644
--- a/lib/eventdev/rte_eventdev.h
+++ b/lib/eventdev/rte_eventdev.h
@@ -1197,6 +1197,8 @@ struct rte_event_vector {
*/
#define RTE_EVENT_TYPE_ETH_RX_ADAPTER 0x4
/**< The event generated from event eth Rx adapter */
+#define RTE_EVENT_TYPE_DMADEV 0x5
+/**< The event generated from dma subsystem */
#define RTE_EVENT_TYPE_VECTOR 0x8
/**< Indicates that event is a vector.
* All vector event types should be a logical OR of EVENT_TYPE_VECTOR.
@@ -1462,6 +1464,48 @@ int
rte_event_crypto_adapter_caps_get(uint8_t dev_id, uint8_t cdev_id,
uint32_t *caps);
+/* DMA adapter capability bitmap flag */
+#define RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP_NEW 0x1
+/**< Flag indicates HW is capable of generating events in
+ * RTE_EVENT_OP_NEW enqueue operation. DMADEV will send
+ * packets to the event device as new events using an
+ * internal event port.
+ */
+
+#define RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP_FWD 0x2
+/**< Flag indicates HW is capable of generating events in
+ * RTE_EVENT_OP_FORWARD enqueue operation. DMADEV will send
+ * packets to the event device as forwarded event using an
+ * internal event port.
+ */
+
+#define RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_VCHAN_EV_BIND 0x4
+/**< Flag indicates HW is capable of mapping DMA vchan to event queue. */
+
+/**
+ * Retrieve the event device's DMA adapter capabilities for the
+ * specified dmadev device
+ *
+ * @param dev_id
+ * The identifier of the device.
+ *
+ * @param dmadev_id
+ * The identifier of the dmadev device.
+ *
+ * @param[out] caps
+ * A pointer to memory filled with event adapter capabilities.
+ * It is expected to be pre-allocated & initialized by caller.
+ *
+ * @return
+ * - 0: Success, driver provides event adapter capabilities for the
+ * dmadev device.
+ * - <0: Error code returned by the driver function.
+ *
+ */
+__rte_experimental
+int
+rte_event_dma_adapter_caps_get(uint8_t dev_id, int16_t dmadev_id, uint32_t *caps);
+
/* Ethdev Tx adapter capability bitmap flags */
#define RTE_EVENT_ETH_TX_ADAPTER_CAP_INTERNAL_PORT 0x1
/**< This flag is sent when the PMD supports a packet transmit callback
diff --git a/lib/eventdev/rte_eventdev_core.h b/lib/eventdev/rte_eventdev_core.h
index c27a52ccc0..83e8736c71 100644
--- a/lib/eventdev/rte_eventdev_core.h
+++ b/lib/eventdev/rte_eventdev_core.h
@@ -42,6 +42,10 @@ typedef uint16_t (*event_crypto_adapter_enqueue_t)(void *port,
uint16_t nb_events);
/**< @internal Enqueue burst of events on crypto adapter */
+typedef uint16_t (*event_dma_adapter_enqueue_t)(void *port, struct rte_event ev[],
+ uint16_t nb_events);
+/**< @internal Enqueue burst of events on DMA adapter */
+
struct rte_event_fp_ops {
void **data;
/**< points to array of internal port data pointers */
@@ -65,7 +69,9 @@ struct rte_event_fp_ops {
/**< PMD Tx adapter enqueue same destination function. */
event_crypto_adapter_enqueue_t ca_enqueue;
/**< PMD Crypto adapter enqueue function. */
- uintptr_t reserved[5];
+ event_dma_adapter_enqueue_t dma_enqueue;
+ /**< PMD DMA adapter enqueue function. */
+ uintptr_t reserved[4];
} __rte_cache_aligned;
extern struct rte_event_fp_ops rte_event_fp_ops[RTE_EVENT_MAX_DEVS];
diff --git a/lib/eventdev/version.map b/lib/eventdev/version.map
index 7ce09a87bb..5d644b1759 100644
--- a/lib/eventdev/version.map
+++ b/lib/eventdev/version.map
@@ -133,6 +133,22 @@ EXPERIMENTAL {
rte_event_timer_remaining_ticks_get;
# added in 23.11
+ rte_event_dma_adapter_caps_get;
+ rte_event_dma_adapter_create_ext;
+ rte_event_dma_adapter_create;
+ rte_event_dma_adapter_event_port_get;
+ rte_event_dma_adapter_enqueue;
+ rte_event_dma_adapter_free;
+ rte_event_dma_adapter_runtime_params_init;
+ rte_event_dma_adapter_runtime_params_set;
+ rte_event_dma_adapter_runtime_params_get;
+ rte_event_dma_adapter_service_id_get;
+ rte_event_dma_adapter_start;
+ rte_event_dma_adapter_stop;
+ rte_event_dma_adapter_stats_get;
+ rte_event_dma_adapter_stats_reset;
+ rte_event_dma_adapter_vchan_add;
+ rte_event_dma_adapter_vchan_del;
rte_event_eth_rx_adapter_create_ext_with_params;
};
--
2.25.1
^ permalink raw reply [flat|nested] 110+ messages in thread
* [PATCH v4 02/12] eventdev: api to get DMA adapter capabilities
2023-09-26 10:32 ` [PATCH v4 00/12] event DMA adapter library support Amit Prakash Shukla
2023-09-26 10:32 ` [PATCH v4 01/12] eventdev: introduce event DMA adapter library Amit Prakash Shukla
@ 2023-09-26 10:32 ` Amit Prakash Shukla
2023-09-27 18:20 ` Jerin Jacob
2023-09-26 10:32 ` [PATCH v4 03/12] eventdev: create and free API for DMA adapter Amit Prakash Shukla
` (10 subsequent siblings)
12 siblings, 1 reply; 110+ messages in thread
From: Amit Prakash Shukla @ 2023-09-26 10:32 UTC (permalink / raw)
To: Jerin Jacob
Cc: dev, fengchengwen, kevin.laatz, bruce.richardson, conor.walsh,
vattunuru, g.singh, sachin.saxena, hemant.agrawal, cheng1.jiang,
ndabilpuram, anoobj, mb, Amit Prakash Shukla
Added a new eventdev API rte_event_dma_adapter_caps_get(), to get
DMA adapter capabilities supported by the driver.
Signed-off-by: Amit Prakash Shukla <amitprakashs@marvell.com>
---
lib/eventdev/meson.build | 2 +-
lib/eventdev/rte_eventdev.c | 23 +++++++++++++++++++++++
lib/eventdev/rte_eventdev.h | 2 +-
lib/meson.build | 2 +-
4 files changed, 26 insertions(+), 3 deletions(-)
diff --git a/lib/eventdev/meson.build b/lib/eventdev/meson.build
index 21347f7c4c..b46bbbc9aa 100644
--- a/lib/eventdev/meson.build
+++ b/lib/eventdev/meson.build
@@ -43,5 +43,5 @@ driver_sdk_headers += files(
'event_timer_adapter_pmd.h',
)
-deps += ['ring', 'ethdev', 'hash', 'mempool', 'mbuf', 'timer', 'cryptodev']
+deps += ['ring', 'ethdev', 'hash', 'mempool', 'mbuf', 'timer', 'cryptodev', 'dmadev']
deps += ['telemetry']
diff --git a/lib/eventdev/rte_eventdev.c b/lib/eventdev/rte_eventdev.c
index 6ab4524332..60509c6efb 100644
--- a/lib/eventdev/rte_eventdev.c
+++ b/lib/eventdev/rte_eventdev.c
@@ -20,6 +20,7 @@
#include <rte_errno.h>
#include <ethdev_driver.h>
#include <rte_cryptodev.h>
+#include <rte_dmadev.h>
#include <cryptodev_pmd.h>
#include <rte_telemetry.h>
@@ -224,6 +225,28 @@ rte_event_eth_tx_adapter_caps_get(uint8_t dev_id, uint16_t eth_port_id,
: 0;
}
+int
+rte_event_dma_adapter_caps_get(uint8_t dev_id, uint8_t dma_dev_id, uint32_t *caps)
+{
+ struct rte_eventdev *dev;
+
+ RTE_EVENTDEV_VALID_DEVID_OR_ERR_RET(dev_id, -EINVAL);
+ if (!rte_dma_is_valid(dma_dev_id))
+ return -EINVAL;
+
+ dev = &rte_eventdevs[dev_id];
+
+ if (caps == NULL)
+ return -EINVAL;
+
+ *caps = 0;
+
+ if (dev->dev_ops->dma_adapter_caps_get)
+ return (*dev->dev_ops->dma_adapter_caps_get)(dev, dma_dev_id, caps);
+
+ return 0;
+}
+
static inline int
event_dev_queue_config(struct rte_eventdev *dev, uint8_t nb_queues)
{
diff --git a/lib/eventdev/rte_eventdev.h b/lib/eventdev/rte_eventdev.h
index 2c6ecc7f8f..41743f91b1 100644
--- a/lib/eventdev/rte_eventdev.h
+++ b/lib/eventdev/rte_eventdev.h
@@ -1504,7 +1504,7 @@ rte_event_crypto_adapter_caps_get(uint8_t dev_id, uint8_t cdev_id,
*/
__rte_experimental
int
-rte_event_dma_adapter_caps_get(uint8_t dev_id, int16_t dmadev_id, uint32_t *caps);
+rte_event_dma_adapter_caps_get(uint8_t dev_id, uint8_t dmadev_id, uint32_t *caps);
/* Ethdev Tx adapter capability bitmap flags */
#define RTE_EVENT_ETH_TX_ADAPTER_CAP_INTERNAL_PORT 0x1
diff --git a/lib/meson.build b/lib/meson.build
index 53155be8e9..f3191f10b6 100644
--- a/lib/meson.build
+++ b/lib/meson.build
@@ -33,6 +33,7 @@ libraries = [
'compressdev',
'cryptodev',
'distributor',
+ 'dmadev',
'efd',
'eventdev',
'gpudev',
@@ -48,7 +49,6 @@ libraries = [
'rawdev',
'regexdev',
'mldev',
- 'dmadev',
'rib',
'reorder',
'sched',
--
2.25.1
^ permalink raw reply [flat|nested] 110+ messages in thread
* [PATCH v4 03/12] eventdev: create and free API for DMA adapter
2023-09-26 10:32 ` [PATCH v4 00/12] event DMA adapter library support Amit Prakash Shukla
2023-09-26 10:32 ` [PATCH v4 01/12] eventdev: introduce event DMA adapter library Amit Prakash Shukla
2023-09-26 10:32 ` [PATCH v4 02/12] eventdev: api to get DMA adapter capabilities Amit Prakash Shukla
@ 2023-09-26 10:32 ` Amit Prakash Shukla
2023-09-27 18:23 ` Jerin Jacob
2023-09-26 10:32 ` [PATCH v4 04/12] eventdev: add API support for vchan add and delete Amit Prakash Shukla
` (9 subsequent siblings)
12 siblings, 1 reply; 110+ messages in thread
From: Amit Prakash Shukla @ 2023-09-26 10:32 UTC (permalink / raw)
To: Bruce Richardson, Jerin Jacob, Amit Prakash Shukla
Cc: dev, fengchengwen, kevin.laatz, conor.walsh, vattunuru, g.singh,
sachin.saxena, hemant.agrawal, cheng1.jiang, ndabilpuram, anoobj,
mb
Added API support to create and free DMA adapter. Create function shall be
called with event device to be associated with the adapter and port
configuration to setup an event port.
Signed-off-by: Amit Prakash Shukla <amitprakashs@marvell.com>
---
config/rte_config.h | 1 +
lib/eventdev/meson.build | 1 +
lib/eventdev/rte_event_dma_adapter.c | 335 +++++++++++++++++++++++++++
3 files changed, 337 insertions(+)
create mode 100644 lib/eventdev/rte_event_dma_adapter.c
diff --git a/config/rte_config.h b/config/rte_config.h
index 400e44e3cf..401727703f 100644
--- a/config/rte_config.h
+++ b/config/rte_config.h
@@ -77,6 +77,7 @@
#define RTE_EVENT_ETH_INTR_RING_SIZE 1024
#define RTE_EVENT_CRYPTO_ADAPTER_MAX_INSTANCE 32
#define RTE_EVENT_ETH_TX_ADAPTER_MAX_INSTANCE 32
+#define RTE_EVENT_DMA_ADAPTER_MAX_INSTANCE 32
/* rawdev defines */
#define RTE_RAWDEV_MAX_DEVS 64
diff --git a/lib/eventdev/meson.build b/lib/eventdev/meson.build
index b46bbbc9aa..250abcb154 100644
--- a/lib/eventdev/meson.build
+++ b/lib/eventdev/meson.build
@@ -17,6 +17,7 @@ sources = files(
'eventdev_private.c',
'eventdev_trace_points.c',
'rte_event_crypto_adapter.c',
+ 'rte_event_dma_adapter.c',
'rte_event_eth_rx_adapter.c',
'rte_event_eth_tx_adapter.c',
'rte_event_ring.c',
diff --git a/lib/eventdev/rte_event_dma_adapter.c b/lib/eventdev/rte_event_dma_adapter.c
new file mode 100644
index 0000000000..c7ffba1b47
--- /dev/null
+++ b/lib/eventdev/rte_event_dma_adapter.c
@@ -0,0 +1,335 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright (c) 2023 Marvell.
+ */
+
+#include "rte_eventdev.h"
+#include "eventdev_pmd.h"
+#include "rte_event_dma_adapter.h"
+
+#define DMA_BATCH_SIZE 32
+#define DMA_DEFAULT_MAX_NB 128
+#define DMA_ADAPTER_NAME_LEN 32
+#define DMA_ADAPTER_BUFFER_SIZE 1024
+
+#define DMA_ADAPTER_OPS_BUFFER_SIZE (DMA_BATCH_SIZE + DMA_BATCH_SIZE)
+
+#define DMA_ADAPTER_ARRAY "event_dma_adapter_array"
+
+/* Macros to check for valid adapter */
+#define EVENT_DMA_ADAPTER_ID_VALID_OR_ERR_RET(id, retval) \
+ do { \
+ if (!edma_adapter_valid_id(id)) { \
+ RTE_EDEV_LOG_ERR("Invalid DMA adapter id = %d\n", id); \
+ return retval; \
+ } \
+ } while (0)
+
+/* DMA ops circular buffer */
+struct dma_ops_circular_buffer {
+ /* Index of head element */
+ uint16_t head;
+
+ /* Index of tail element */
+ uint16_t tail;
+
+ /* Number of elements in buffer */
+ uint16_t count;
+
+ /* Size of circular buffer */
+ uint16_t size;
+
+ /* Pointer to hold rte_event_dma_adapter_op for processing */
+ struct rte_event_dma_adapter_op **op_buffer;
+} __rte_cache_aligned;
+
+/* DMA device information */
+struct dma_device_info {
+ /* Number of vchans configured for a DMA device. */
+ uint16_t num_dma_dev_vchan;
+} __rte_cache_aligned;
+
+struct event_dma_adapter {
+ /* Event device identifier */
+ uint8_t eventdev_id;
+
+ /* Event port identifier */
+ uint8_t event_port_id;
+
+ /* Adapter mode */
+ enum rte_event_dma_adapter_mode mode;
+
+ /* Memory allocation name */
+ char mem_name[DMA_ADAPTER_NAME_LEN];
+
+ /* Socket identifier cached from eventdev */
+ int socket_id;
+
+ /* Lock to serialize config updates with service function */
+ rte_spinlock_t lock;
+
+ /* DMA device structure array */
+ struct dma_device_info *dma_devs;
+
+ /* Circular buffer for processing DMA ops to eventdev */
+ struct dma_ops_circular_buffer ebuf;
+
+ /* Configuration callback for rte_service configuration */
+ rte_event_dma_adapter_conf_cb conf_cb;
+
+ /* Configuration callback argument */
+ void *conf_arg;
+
+ /* Set if default_cb is being used */
+ int default_cb_arg;
+} __rte_cache_aligned;
+
+static struct event_dma_adapter **event_dma_adapter;
+
+static inline int
+edma_adapter_valid_id(uint8_t id)
+{
+ return id < RTE_EVENT_DMA_ADAPTER_MAX_INSTANCE;
+}
+
+static inline struct event_dma_adapter *
+edma_id_to_adapter(uint8_t id)
+{
+ return event_dma_adapter ? event_dma_adapter[id] : NULL;
+}
+
+static int
+edma_array_init(void)
+{
+ const struct rte_memzone *mz;
+ uint32_t sz;
+
+ mz = rte_memzone_lookup(DMA_ADAPTER_ARRAY);
+ if (mz == NULL) {
+ sz = sizeof(struct event_dma_adapter *) * RTE_EVENT_DMA_ADAPTER_MAX_INSTANCE;
+ sz = RTE_ALIGN(sz, RTE_CACHE_LINE_SIZE);
+
+ mz = rte_memzone_reserve_aligned(DMA_ADAPTER_ARRAY, sz, rte_socket_id(), 0,
+ RTE_CACHE_LINE_SIZE);
+ if (mz == NULL) {
+ RTE_EDEV_LOG_ERR("Failed to reserve memzone : %s, err = %d",
+ DMA_ADAPTER_ARRAY, rte_errno);
+ return -rte_errno;
+ }
+ }
+
+ event_dma_adapter = mz->addr;
+
+ return 0;
+}
+
+static inline int
+edma_circular_buffer_init(const char *name, struct dma_ops_circular_buffer *buf, uint16_t sz)
+{
+ buf->op_buffer = rte_zmalloc(name, sizeof(struct rte_event_dma_adapter_op *) * sz, 0);
+ if (buf->op_buffer == NULL)
+ return -ENOMEM;
+
+ buf->size = sz;
+
+ return 0;
+}
+
+static inline void
+edma_circular_buffer_free(struct dma_ops_circular_buffer *buf)
+{
+ rte_free(buf->op_buffer);
+}
+
+static int
+edma_default_config_cb(uint8_t id, uint8_t evdev_id, struct rte_event_dma_adapter_conf *conf,
+ void *arg)
+{
+ struct rte_event_port_conf *port_conf;
+ struct rte_event_dev_config dev_conf;
+ struct event_dma_adapter *adapter;
+ struct rte_eventdev *dev;
+ uint8_t port_id;
+ int started;
+ int ret;
+
+ adapter = edma_id_to_adapter(id);
+ if (adapter == NULL)
+ return -EINVAL;
+
+ dev = &rte_eventdevs[adapter->eventdev_id];
+ dev_conf = dev->data->dev_conf;
+
+ started = dev->data->dev_started;
+ if (started)
+ rte_event_dev_stop(evdev_id);
+
+ port_id = dev_conf.nb_event_ports;
+ dev_conf.nb_event_ports += 1;
+
+ port_conf = arg;
+ if (port_conf->event_port_cfg & RTE_EVENT_PORT_CFG_SINGLE_LINK)
+ dev_conf.nb_single_link_event_port_queues += 1;
+
+ ret = rte_event_dev_configure(evdev_id, &dev_conf);
+ if (ret) {
+ RTE_EDEV_LOG_ERR("Failed to configure event dev %u\n", evdev_id);
+ if (started) {
+ if (rte_event_dev_start(evdev_id))
+ return -EIO;
+ }
+ return ret;
+ }
+
+ ret = rte_event_port_setup(evdev_id, port_id, port_conf);
+ if (ret) {
+ RTE_EDEV_LOG_ERR("Failed to setup event port %u\n", port_id);
+ return ret;
+ }
+
+ conf->event_port_id = port_id;
+ conf->max_nb = DMA_DEFAULT_MAX_NB;
+ if (started)
+ ret = rte_event_dev_start(evdev_id);
+
+ adapter->default_cb_arg = 1;
+ adapter->event_port_id = conf->event_port_id;
+
+ return ret;
+}
+
+int
+rte_event_dma_adapter_create_ext(uint8_t id, uint8_t evdev_id,
+ rte_event_dma_adapter_conf_cb conf_cb,
+ enum rte_event_dma_adapter_mode mode, void *conf_arg)
+{
+ struct rte_event_dev_info dev_info;
+ struct event_dma_adapter *adapter;
+ char name[DMA_ADAPTER_NAME_LEN];
+ struct rte_dma_info info;
+ uint16_t num_dma_dev;
+ int socket_id;
+ uint8_t i;
+ int ret;
+
+ EVENT_DMA_ADAPTER_ID_VALID_OR_ERR_RET(id, -EINVAL);
+ RTE_EVENTDEV_VALID_DEVID_OR_ERR_RET(evdev_id, -EINVAL);
+
+ if (conf_cb == NULL)
+ return -EINVAL;
+
+ if (event_dma_adapter == NULL) {
+ ret = edma_array_init();
+ if (ret)
+ return ret;
+ }
+
+ adapter = edma_id_to_adapter(id);
+ if (adapter != NULL) {
+ RTE_EDEV_LOG_ERR("ML adapter ID %d already exists!", id);
+ return -EEXIST;
+ }
+
+ socket_id = rte_event_dev_socket_id(evdev_id);
+ snprintf(name, DMA_ADAPTER_NAME_LEN, "rte_event_dma_adapter_%d", id);
+ adapter = rte_zmalloc_socket(name, sizeof(struct event_dma_adapter), RTE_CACHE_LINE_SIZE,
+ socket_id);
+ if (adapter == NULL) {
+ RTE_EDEV_LOG_ERR("Failed to get mem for event ML adapter!");
+ return -ENOMEM;
+ }
+
+ if (edma_circular_buffer_init("edma_circular_buffer", &adapter->ebuf,
+ DMA_ADAPTER_BUFFER_SIZE)) {
+ RTE_EDEV_LOG_ERR("Failed to get memory for event adapter circular buffer");
+ rte_free(adapter);
+ return -ENOMEM;
+ }
+
+ ret = rte_event_dev_info_get(evdev_id, &dev_info);
+ if (ret < 0) {
+ RTE_EDEV_LOG_ERR("Failed to get info for eventdev %d: %s", evdev_id,
+ dev_info.driver_name);
+ edma_circular_buffer_free(&adapter->ebuf);
+ rte_free(adapter);
+ return ret;
+ }
+
+ num_dma_dev = rte_dma_count_avail();
+
+ adapter->eventdev_id = evdev_id;
+ adapter->mode = mode;
+ strcpy(adapter->mem_name, name);
+ adapter->socket_id = socket_id;
+ adapter->conf_cb = conf_cb;
+ adapter->conf_arg = conf_arg;
+ adapter->dma_devs = rte_zmalloc_socket(adapter->mem_name,
+ num_dma_dev * sizeof(struct dma_device_info), 0,
+ socket_id);
+ if (adapter->dma_devs == NULL) {
+ RTE_EDEV_LOG_ERR("Failed to get memory for DMA devices\n");
+ edma_circular_buffer_free(&adapter->ebuf);
+ rte_free(adapter);
+ return -ENOMEM;
+ }
+
+ rte_spinlock_init(&adapter->lock);
+ for (i = 0; i < num_dma_dev; i++) {
+ ret = rte_dma_info_get(i, &info);
+ if (ret) {
+ RTE_EDEV_LOG_ERR("Failed to get dma device info\n");
+ edma_circular_buffer_free(&adapter->ebuf);
+ rte_free(adapter);
+ return ret;
+ }
+
+ adapter->dma_devs[i].num_dma_dev_vchan = info.nb_vchans;
+ }
+
+ event_dma_adapter[id] = adapter;
+
+ return 0;
+}
+
+int
+rte_event_dma_adapter_create(uint8_t id, uint8_t evdev_id, struct rte_event_port_conf *port_config,
+ enum rte_event_dma_adapter_mode mode)
+{
+ struct rte_event_port_conf *pc;
+ int ret;
+
+ if (port_config == NULL)
+ return -EINVAL;
+
+ EVENT_DMA_ADAPTER_ID_VALID_OR_ERR_RET(id, -EINVAL);
+
+ pc = rte_malloc(NULL, sizeof(struct rte_event_port_conf), 0);
+ if (pc == NULL)
+ return -ENOMEM;
+
+ rte_memcpy(pc, port_config, sizeof(struct rte_event_port_conf));
+ ret = rte_event_dma_adapter_create_ext(id, evdev_id, edma_default_config_cb, mode, pc);
+ if (ret != 0)
+ rte_free(pc);
+
+ return ret;
+}
+
+int
+rte_event_dma_adapter_free(uint8_t id)
+{
+ struct event_dma_adapter *adapter;
+
+ EVENT_DMA_ADAPTER_ID_VALID_OR_ERR_RET(id, -EINVAL);
+
+ adapter = edma_id_to_adapter(id);
+ if (adapter == NULL)
+ return -EINVAL;
+
+ rte_free(adapter->conf_arg);
+ rte_free(adapter->dma_devs);
+ edma_circular_buffer_free(&adapter->ebuf);
+ rte_free(adapter);
+ event_dma_adapter[id] = NULL;
+
+ return 0;
+}
--
2.25.1
^ permalink raw reply [flat|nested] 110+ messages in thread
* [PATCH v4 04/12] eventdev: add API support for vchan add and delete
2023-09-26 10:32 ` [PATCH v4 00/12] event DMA adapter library support Amit Prakash Shukla
` (2 preceding siblings ...)
2023-09-26 10:32 ` [PATCH v4 03/12] eventdev: create and free API for DMA adapter Amit Prakash Shukla
@ 2023-09-26 10:32 ` Amit Prakash Shukla
2023-09-26 10:32 ` [PATCH v4 05/12] eventdev: add support for DMA adapter service function Amit Prakash Shukla
` (8 subsequent siblings)
12 siblings, 0 replies; 110+ messages in thread
From: Amit Prakash Shukla @ 2023-09-26 10:32 UTC (permalink / raw)
To: Amit Prakash Shukla, Jerin Jacob
Cc: dev, fengchengwen, kevin.laatz, bruce.richardson, conor.walsh,
vattunuru, g.singh, sachin.saxena, hemant.agrawal, cheng1.jiang,
ndabilpuram, anoobj, mb
Added API support to add and delete vchan's from the DMA adapter. DMA devid
and vchan are added to the addapter instance by calling
rte_event_dma_adapter_vchan_add and
deleted using rte_event_dma_adapter_vchan_del.
Signed-off-by: Amit Prakash Shukla <amitprakashs@marvell.com>
---
lib/eventdev/rte_event_dma_adapter.c | 204 +++++++++++++++++++++++++++
1 file changed, 204 insertions(+)
diff --git a/lib/eventdev/rte_event_dma_adapter.c b/lib/eventdev/rte_event_dma_adapter.c
index c7ffba1b47..dd58188bf3 100644
--- a/lib/eventdev/rte_event_dma_adapter.c
+++ b/lib/eventdev/rte_event_dma_adapter.c
@@ -42,8 +42,31 @@ struct dma_ops_circular_buffer {
struct rte_event_dma_adapter_op **op_buffer;
} __rte_cache_aligned;
+/* Vchan information */
+struct dma_vchan_info {
+ /* Set to indicate vchan queue is enabled */
+ bool vq_enabled;
+
+ /* Circular buffer for batching DMA ops to dma_dev */
+ struct dma_ops_circular_buffer dma_buf;
+} __rte_cache_aligned;
+
/* DMA device information */
struct dma_device_info {
+ /* Pointer to vchan queue info */
+ struct dma_vchan_info *vchanq;
+
+ /* Pointer to vchan queue info.
+ * This holds ops passed by application till the
+ * dma completion is done.
+ */
+ struct dma_vchan_info *tqmap;
+
+ /* If num_vchanq > 0, the start callback will
+ * be invoked if not already invoked
+ */
+ uint16_t num_vchanq;
+
/* Number of vchans configured for a DMA device. */
uint16_t num_dma_dev_vchan;
} __rte_cache_aligned;
@@ -81,6 +104,9 @@ struct event_dma_adapter {
/* Set if default_cb is being used */
int default_cb_arg;
+
+ /* No. of vchan queue configured */
+ uint16_t nb_vchanq;
} __rte_cache_aligned;
static struct event_dma_adapter **event_dma_adapter;
@@ -333,3 +359,181 @@ rte_event_dma_adapter_free(uint8_t id)
return 0;
}
+
+static void
+edma_update_vchanq_info(struct event_dma_adapter *adapter, struct dma_device_info *dev_info,
+ uint16_t vchan, uint8_t add)
+{
+ struct dma_vchan_info *vchan_info;
+ struct dma_vchan_info *tqmap_info;
+ int enabled;
+ uint16_t i;
+
+ if (dev_info->vchanq == NULL)
+ return;
+
+ if (vchan == RTE_DMA_ALL_VCHAN) {
+ for (i = 0; i < dev_info->num_dma_dev_vchan; i++)
+ edma_update_vchanq_info(adapter, dev_info, i, add);
+ } else {
+ tqmap_info = &dev_info->tqmap[vchan];
+ vchan_info = &dev_info->vchanq[vchan];
+ enabled = vchan_info->vq_enabled;
+ if (add) {
+ adapter->nb_vchanq += !enabled;
+ dev_info->num_vchanq += !enabled;
+ } else {
+ adapter->nb_vchanq -= enabled;
+ dev_info->num_vchanq -= enabled;
+ }
+ vchan_info->vq_enabled = !!add;
+ tqmap_info->vq_enabled = !!add;
+ }
+}
+
+int
+rte_event_dma_adapter_vchan_add(uint8_t id, int16_t dma_dev_id, uint16_t vchan,
+ const struct rte_event *event)
+{
+ struct event_dma_adapter *adapter;
+ struct dma_device_info *dev_info;
+ struct rte_eventdev *dev;
+ uint32_t cap;
+ int ret;
+
+ EVENT_DMA_ADAPTER_ID_VALID_OR_ERR_RET(id, -EINVAL);
+
+ if (!rte_dma_is_valid(dma_dev_id)) {
+ RTE_EDEV_LOG_ERR("Invalid dma_dev_id = %" PRIu8, dma_dev_id);
+ return -EINVAL;
+ }
+
+ adapter = edma_id_to_adapter(id);
+ if (adapter == NULL)
+ return -EINVAL;
+
+ dev = &rte_eventdevs[adapter->eventdev_id];
+ ret = rte_event_dma_adapter_caps_get(adapter->eventdev_id, dma_dev_id, &cap);
+ if (ret) {
+ RTE_EDEV_LOG_ERR("Failed to get adapter caps dev %u dma_dev %u", id, dma_dev_id);
+ return ret;
+ }
+
+ if ((cap & RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_VCHAN_EV_BIND) && (event == NULL)) {
+ RTE_EDEV_LOG_ERR("Event can not be NULL for dma_dev_id = %u", dma_dev_id);
+ return -EINVAL;
+ }
+
+ dev_info = &adapter->dma_devs[dma_dev_id];
+ if (vchan != RTE_DMA_ALL_VCHAN && vchan >= dev_info->num_dma_dev_vchan) {
+ RTE_EDEV_LOG_ERR("Invalid vhcan %u", vchan);
+ return -EINVAL;
+ }
+
+ /* In case HW cap is RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP_FWD, no
+ * need of service core as HW supports event forward capability.
+ */
+ if ((cap & RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP_FWD) ||
+ (cap & RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_VCHAN_EV_BIND &&
+ adapter->mode == RTE_EVENT_DMA_ADAPTER_OP_NEW) ||
+ (cap & RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP_NEW &&
+ adapter->mode == RTE_EVENT_DMA_ADAPTER_OP_NEW)) {
+ if (*dev->dev_ops->dma_adapter_vchan_add == NULL)
+ return -ENOTSUP;
+ if (dev_info->vchanq == NULL) {
+ dev_info->vchanq = rte_zmalloc_socket(adapter->mem_name,
+ dev_info->num_dma_dev_vchan *
+ sizeof(struct dma_vchan_info),
+ 0, adapter->socket_id);
+ if (dev_info->vchanq == NULL) {
+ printf("Queue pair add not supported\n");
+ return -ENOMEM;
+ }
+ }
+
+ if (dev_info->tqmap == NULL) {
+ dev_info->tqmap = rte_zmalloc_socket(adapter->mem_name,
+ dev_info->num_dma_dev_vchan *
+ sizeof(struct dma_vchan_info),
+ 0, adapter->socket_id);
+ if (dev_info->tqmap == NULL) {
+ printf("tq pair add not supported\n");
+ return -ENOMEM;
+ }
+ }
+
+ ret = (*dev->dev_ops->dma_adapter_vchan_add)(dev, dma_dev_id, vchan, event);
+ if (ret)
+ return ret;
+
+ else
+ edma_update_vchanq_info(adapter, &adapter->dma_devs[dma_dev_id], vchan, 1);
+ }
+
+ return 0;
+}
+
+int
+rte_event_dma_adapter_vchan_del(uint8_t id, int16_t dma_dev_id, uint16_t vchan)
+{
+ struct event_dma_adapter *adapter;
+ struct dma_device_info *dev_info;
+ struct rte_eventdev *dev;
+ uint32_t cap;
+ int ret;
+
+ EVENT_DMA_ADAPTER_ID_VALID_OR_ERR_RET(id, -EINVAL);
+
+ if (!rte_dma_is_valid(dma_dev_id)) {
+ RTE_EDEV_LOG_ERR("Invalid dma_dev_id = %" PRIu8, dma_dev_id);
+ return -EINVAL;
+ }
+
+ adapter = edma_id_to_adapter(id);
+ if (adapter == NULL)
+ return -EINVAL;
+
+ dev = &rte_eventdevs[adapter->eventdev_id];
+ ret = rte_event_dma_adapter_caps_get(adapter->eventdev_id, dma_dev_id, &cap);
+ if (ret)
+ return ret;
+
+ dev_info = &adapter->dma_devs[dma_dev_id];
+
+ if (vchan != RTE_DMA_ALL_VCHAN && vchan >= dev_info->num_dma_dev_vchan) {
+ RTE_EDEV_LOG_ERR("Invalid vhcan %" PRIu16, vchan);
+ return -EINVAL;
+ }
+
+ if ((cap & RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP_FWD) ||
+ (cap & RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP_NEW &&
+ adapter->mode == RTE_EVENT_DMA_ADAPTER_OP_NEW)) {
+ if (*dev->dev_ops->dma_adapter_vchan_del == NULL)
+ return -ENOTSUP;
+ ret = (*dev->dev_ops->dma_adapter_vchan_del)(dev, dma_dev_id, vchan);
+ if (ret == 0) {
+ edma_update_vchanq_info(adapter, dev_info, vchan, 0);
+ if (dev_info->num_vchanq == 0) {
+ rte_free(dev_info->vchanq);
+ dev_info->vchanq = NULL;
+ }
+ }
+ } else {
+ if (adapter->nb_vchanq == 0)
+ return 0;
+
+ rte_spinlock_lock(&adapter->lock);
+ edma_update_vchanq_info(adapter, dev_info, vchan, 0);
+
+ if (dev_info->num_vchanq == 0) {
+ rte_free(dev_info->vchanq);
+ rte_free(dev_info->tqmap);
+ dev_info->vchanq = NULL;
+ dev_info->tqmap = NULL;
+ }
+
+ rte_spinlock_unlock(&adapter->lock);
+ }
+
+ return ret;
+}
--
2.25.1
^ permalink raw reply [flat|nested] 110+ messages in thread
* [PATCH v4 05/12] eventdev: add support for DMA adapter service function
2023-09-26 10:32 ` [PATCH v4 00/12] event DMA adapter library support Amit Prakash Shukla
` (3 preceding siblings ...)
2023-09-26 10:32 ` [PATCH v4 04/12] eventdev: add API support for vchan add and delete Amit Prakash Shukla
@ 2023-09-26 10:32 ` Amit Prakash Shukla
2023-09-26 10:32 ` [PATCH v4 06/12] eventdev: add support for DMA adapter start and stop Amit Prakash Shukla
` (7 subsequent siblings)
12 siblings, 0 replies; 110+ messages in thread
From: Amit Prakash Shukla @ 2023-09-26 10:32 UTC (permalink / raw)
To: Amit Prakash Shukla, Jerin Jacob
Cc: dev, fengchengwen, kevin.laatz, bruce.richardson, conor.walsh,
vattunuru, g.singh, sachin.saxena, hemant.agrawal, cheng1.jiang,
ndabilpuram, anoobj, mb
Added support for DMA adapter service function for event devices.
Enqueue and dequeue of event from eventdev and DMA device are done
based on the adapter mode and the supported HW capabilities.
Signed-off-by: Amit Prakash Shukla <amitprakashs@marvell.com>
---
lib/eventdev/rte_event_dma_adapter.c | 589 +++++++++++++++++++++++++++
1 file changed, 589 insertions(+)
diff --git a/lib/eventdev/rte_event_dma_adapter.c b/lib/eventdev/rte_event_dma_adapter.c
index dd58188bf3..8349b95796 100644
--- a/lib/eventdev/rte_event_dma_adapter.c
+++ b/lib/eventdev/rte_event_dma_adapter.c
@@ -2,6 +2,8 @@
* Copyright (c) 2023 Marvell.
*/
+#include <rte_service_component.h>
+
#include "rte_eventdev.h"
#include "eventdev_pmd.h"
#include "rte_event_dma_adapter.h"
@@ -69,6 +71,10 @@ struct dma_device_info {
/* Number of vchans configured for a DMA device. */
uint16_t num_dma_dev_vchan;
+
+ /* Next queue pair to be processed */
+ uint16_t next_vchan_id;
+
} __rte_cache_aligned;
struct event_dma_adapter {
@@ -90,6 +96,9 @@ struct event_dma_adapter {
/* Lock to serialize config updates with service function */
rte_spinlock_t lock;
+ /* Next dma device to be processed */
+ uint16_t next_dmadev_id;
+
/* DMA device structure array */
struct dma_device_info *dma_devs;
@@ -107,6 +116,26 @@ struct event_dma_adapter {
/* No. of vchan queue configured */
uint16_t nb_vchanq;
+
+ /* Per adapter EAL service ID */
+ uint32_t service_id;
+
+ /* Service initialization state */
+ uint8_t service_initialized;
+
+ /* Max DMA ops processed in any service function invocation */
+ uint32_t max_nb;
+
+ /* Store event port's implicit release capability */
+ uint8_t implicit_release_disabled;
+
+ /* Flag to indicate backpressure at dma_dev
+ * Stop further dequeuing events from eventdev
+ */
+ bool stop_enq_to_dma_dev;
+
+ /* Loop counter to flush dma ops */
+ uint16_t transmit_loop_count;
} __rte_cache_aligned;
static struct event_dma_adapter **event_dma_adapter;
@@ -148,6 +177,18 @@ edma_array_init(void)
return 0;
}
+static inline bool
+edma_circular_buffer_batch_ready(struct dma_ops_circular_buffer *bufp)
+{
+ return bufp->count >= DMA_BATCH_SIZE;
+}
+
+static inline bool
+edma_circular_buffer_space_for_batch(struct dma_ops_circular_buffer *bufp)
+{
+ return (bufp->size - bufp->count) >= DMA_BATCH_SIZE;
+}
+
static inline int
edma_circular_buffer_init(const char *name, struct dma_ops_circular_buffer *buf, uint16_t sz)
{
@@ -166,6 +207,67 @@ edma_circular_buffer_free(struct dma_ops_circular_buffer *buf)
rte_free(buf->op_buffer);
}
+static inline int
+edma_circular_buffer_add(struct dma_ops_circular_buffer *bufp, struct rte_event_dma_adapter_op *op)
+{
+ uint16_t *tail = &bufp->tail;
+
+ bufp->op_buffer[*tail] = op;
+
+ /* circular buffer, go round */
+ *tail = (*tail + 1) % bufp->size;
+ bufp->count++;
+
+ return 0;
+}
+
+static inline int
+edma_circular_buffer_flush_to_dma_dev(struct event_dma_adapter *adapter,
+ struct dma_ops_circular_buffer *bufp, uint8_t dma_dev_id,
+ uint16_t vchan, uint16_t *nb_ops_flushed)
+{
+ struct rte_event_dma_adapter_op *op;
+ struct dma_vchan_info *tq;
+ uint16_t *head = &bufp->head;
+ uint16_t *tail = &bufp->tail;
+ uint16_t n;
+ uint16_t i;
+ int ret;
+
+ if (*tail > *head)
+ n = *tail - *head;
+ else if (*tail < *head)
+ n = bufp->size - *head;
+ else {
+ *nb_ops_flushed = 0;
+ return 0; /* buffer empty */
+ }
+
+ tq = &adapter->dma_devs[dma_dev_id].tqmap[vchan];
+
+ for (i = 0; i < n; i++) {
+ op = bufp->op_buffer[*head];
+ ret = rte_dma_copy_sg(dma_dev_id, vchan, op->src_seg, op->dst_seg,
+ op->nb_src, op->nb_dst, op->flags);
+ if (ret < 0)
+ break;
+
+ /* Enqueue in transaction queue. */
+ edma_circular_buffer_add(&tq->dma_buf, op);
+
+ *head = (*head + 1) % bufp->size;
+ }
+
+ *nb_ops_flushed = i;
+ bufp->count -= *nb_ops_flushed;
+ if (!bufp->count) {
+ *head = 0;
+ *tail = 0;
+ }
+
+ return *nb_ops_flushed == n ? 0 : -1;
+}
+
static int
edma_default_config_cb(uint8_t id, uint8_t evdev_id, struct rte_event_dma_adapter_conf *conf,
void *arg)
@@ -360,6 +462,406 @@ rte_event_dma_adapter_free(uint8_t id)
return 0;
}
+static inline unsigned int
+edma_enq_to_dma_dev(struct event_dma_adapter *adapter, struct rte_event *ev, unsigned int cnt)
+{
+ struct dma_vchan_info *vchan_qinfo = NULL;
+ struct rte_event_dma_adapter_op *dma_op;
+ uint16_t vchan, nb_enqueued = 0;
+ int16_t dma_dev_id;
+ unsigned int i, n;
+ int ret;
+
+ ret = 0;
+ n = 0;
+
+ for (i = 0; i < cnt; i++) {
+ dma_op = ev[i].event_ptr;
+ if (dma_op == NULL)
+ continue;
+
+ /* Expected to have response info appended to dma_op. */
+
+ dma_dev_id = dma_op->dma_dev_id;
+ vchan = dma_op->vchan;
+ vchan_qinfo = &adapter->dma_devs[dma_dev_id].vchanq[vchan];
+ if (!vchan_qinfo->vq_enabled) {
+ if (dma_op != NULL && dma_op->op_mp != NULL)
+ rte_mempool_put(dma_op->op_mp, dma_op);
+ continue;
+ }
+ edma_circular_buffer_add(&vchan_qinfo->dma_buf, dma_op);
+
+ if (edma_circular_buffer_batch_ready(&vchan_qinfo->dma_buf)) {
+ ret = edma_circular_buffer_flush_to_dma_dev(adapter, &vchan_qinfo->dma_buf,
+ dma_dev_id, vchan,
+ &nb_enqueued);
+ n += nb_enqueued;
+
+ /**
+ * If some dma ops failed to flush to dma_dev and
+ * space for another batch is not available, stop
+ * dequeue from eventdev momentarily
+ */
+ if (unlikely(ret < 0 &&
+ !edma_circular_buffer_space_for_batch(&vchan_qinfo->dma_buf)))
+ adapter->stop_enq_to_dma_dev = true;
+ }
+ }
+
+ return n;
+}
+
+static unsigned int
+edma_adapter_dev_flush(struct event_dma_adapter *adapter, int16_t dma_dev_id,
+ uint16_t *nb_ops_flushed)
+{
+ struct dma_vchan_info *vchan_info;
+ struct dma_device_info *dev_info;
+ uint16_t nb = 0, nb_enqueued = 0;
+ uint16_t vchan, nb_vchans;
+
+ dev_info = &adapter->dma_devs[dma_dev_id];
+ nb_vchans = dev_info->num_vchanq;
+
+ for (vchan = 0; vchan < nb_vchans; vchan++) {
+
+ vchan_info = &dev_info->vchanq[vchan];
+ if (unlikely(vchan_info == NULL || !vchan_info->vq_enabled))
+ continue;
+
+ edma_circular_buffer_flush_to_dma_dev(adapter, &vchan_info->dma_buf, dma_dev_id,
+ vchan, &nb_enqueued);
+ *nb_ops_flushed += vchan_info->dma_buf.count;
+ nb += nb_enqueued;
+ }
+
+ return nb;
+}
+
+static unsigned int
+edma_adapter_enq_flush(struct event_dma_adapter *adapter)
+{
+ int16_t dma_dev_id;
+ uint16_t nb_enqueued = 0;
+ uint16_t nb_ops_flushed = 0;
+ uint16_t num_dma_dev = rte_dma_count_avail();
+
+ for (dma_dev_id = 0; dma_dev_id < num_dma_dev; dma_dev_id++)
+ nb_enqueued += edma_adapter_dev_flush(adapter, dma_dev_id, &nb_ops_flushed);
+ /**
+ * Enable dequeue from eventdev if all ops from circular
+ * buffer flushed to dma_dev
+ */
+ if (!nb_ops_flushed)
+ adapter->stop_enq_to_dma_dev = false;
+
+ return nb_enqueued;
+}
+
+/* Flush an instance's enqueue buffers every DMA_ENQ_FLUSH_THRESHOLD
+ * iterations of edma_adapter_enq_run()
+ */
+#define DMA_ENQ_FLUSH_THRESHOLD 1024
+
+static int
+edma_adapter_enq_run(struct event_dma_adapter *adapter, unsigned int max_enq)
+{
+ uint8_t event_port_id = adapter->event_port_id;
+ uint8_t event_dev_id = adapter->eventdev_id;
+ struct rte_event ev[DMA_BATCH_SIZE];
+ unsigned int nb_enq, nb_enqueued;
+ uint16_t n;
+
+ if (adapter->mode == RTE_EVENT_DMA_ADAPTER_OP_NEW)
+ return 0;
+
+ nb_enqueued = 0;
+ for (nb_enq = 0; nb_enq < max_enq; nb_enq += n) {
+
+ if (unlikely(adapter->stop_enq_to_dma_dev)) {
+ nb_enqueued += edma_adapter_enq_flush(adapter);
+
+ if (unlikely(adapter->stop_enq_to_dma_dev))
+ break;
+ }
+
+ n = rte_event_dequeue_burst(event_dev_id, event_port_id, ev, DMA_BATCH_SIZE, 0);
+
+ if (!n)
+ break;
+
+ nb_enqueued += edma_enq_to_dma_dev(adapter, ev, n);
+ }
+
+ if ((++adapter->transmit_loop_count & (DMA_ENQ_FLUSH_THRESHOLD - 1)) == 0)
+ nb_enqueued += edma_adapter_enq_flush(adapter);
+
+ return nb_enqueued;
+}
+
+#define DMA_ADAPTER_MAX_EV_ENQ_RETRIES 100
+
+static inline uint16_t
+edma_ops_enqueue_burst(struct event_dma_adapter *adapter, struct rte_event_dma_adapter_op **ops,
+ uint16_t num)
+{
+ uint8_t event_port_id = adapter->event_port_id;
+ uint8_t event_dev_id = adapter->eventdev_id;
+ struct rte_event events[DMA_BATCH_SIZE];
+ struct rte_event *response_info;
+ uint16_t nb_enqueued, nb_ev;
+ uint8_t retry;
+ uint8_t i;
+
+ nb_ev = 0;
+ retry = 0;
+ nb_enqueued = 0;
+ num = RTE_MIN(num, DMA_BATCH_SIZE);
+ for (i = 0; i < num; i++) {
+ struct rte_event *ev = &events[nb_ev++];
+
+ /* Expected to have response info appended to dma_op. */
+ response_info = (struct rte_event *)((uint8_t *)ops[i] +
+ sizeof(struct rte_event_dma_adapter_op));
+ if (unlikely(response_info == NULL)) {
+ if (ops[i] != NULL && ops[i]->op_mp != NULL)
+ rte_mempool_put(ops[i]->op_mp, ops[i]);
+ continue;
+ }
+
+ rte_memcpy(ev, response_info, sizeof(struct rte_event));
+ ev->event_ptr = ops[i];
+ ev->event_type = RTE_EVENT_TYPE_DMADEV;
+ if (adapter->implicit_release_disabled)
+ ev->op = RTE_EVENT_OP_FORWARD;
+ else
+ ev->op = RTE_EVENT_OP_NEW;
+ }
+
+ do {
+ nb_enqueued += rte_event_enqueue_burst(event_dev_id, event_port_id,
+ &events[nb_enqueued], nb_ev - nb_enqueued);
+
+ } while (retry++ < DMA_ADAPTER_MAX_EV_ENQ_RETRIES && nb_enqueued < nb_ev);
+
+ return nb_enqueued;
+}
+
+static int
+edma_circular_buffer_flush_to_evdev(struct event_dma_adapter *adapter,
+ struct dma_ops_circular_buffer *bufp,
+ uint16_t *enqueue_count)
+{
+ struct rte_event_dma_adapter_op **ops = bufp->op_buffer;
+ uint16_t n = 0, nb_ops_flushed;
+ uint16_t *head = &bufp->head;
+ uint16_t *tail = &bufp->tail;
+
+ if (*tail > *head)
+ n = *tail - *head;
+ else if (*tail < *head)
+ n = bufp->size - *head;
+ else {
+ if (enqueue_count)
+ *enqueue_count = 0;
+ return 0; /* buffer empty */
+ }
+
+ if (enqueue_count && n > *enqueue_count)
+ n = *enqueue_count;
+
+ nb_ops_flushed = edma_ops_enqueue_burst(adapter, &ops[*head], n);
+ if (enqueue_count)
+ *enqueue_count = nb_ops_flushed;
+
+ bufp->count -= nb_ops_flushed;
+ if (!bufp->count) {
+ *head = 0;
+ *tail = 0;
+ return 0; /* buffer empty */
+ }
+
+ *head = (*head + nb_ops_flushed) % bufp->size;
+ return 1;
+}
+
+static void
+edma_ops_buffer_flush(struct event_dma_adapter *adapter)
+{
+ if (likely(adapter->ebuf.count == 0))
+ return;
+
+ while (edma_circular_buffer_flush_to_evdev(adapter, &adapter->ebuf, NULL))
+ ;
+}
+
+static inline unsigned int
+edma_adapter_deq_run(struct event_dma_adapter *adapter, unsigned int max_deq)
+{
+ struct dma_vchan_info *vchan_info;
+ struct dma_ops_circular_buffer *tq_buf;
+ struct rte_event_dma_adapter_op *ops;
+ uint16_t n, nb_deq, nb_enqueued, i;
+ struct dma_device_info *dev_info;
+ uint16_t vchan, num_vchan;
+ uint16_t num_dma_dev;
+ int16_t dma_dev_id;
+ uint16_t index;
+ bool done;
+ bool err;
+
+ nb_deq = 0;
+ edma_ops_buffer_flush(adapter);
+
+ num_dma_dev = rte_dma_count_avail();
+ do {
+ done = true;
+
+ for (dma_dev_id = adapter->next_dmadev_id; dma_dev_id < num_dma_dev; dma_dev_id++) {
+ uint16_t queues = 0;
+ dev_info = &adapter->dma_devs[dma_dev_id];
+ num_vchan = dev_info->num_vchanq;
+
+ for (vchan = dev_info->next_vchan_id; queues < num_vchan;
+ vchan = (vchan + 1) % num_vchan, queues++) {
+
+ vchan_info = &dev_info->vchanq[vchan];
+ if (unlikely(vchan_info == NULL || !vchan_info->vq_enabled))
+ continue;
+
+ n = rte_dma_completed(dma_dev_id, vchan, DMA_BATCH_SIZE,
+ &index, &err);
+ if (!n)
+ continue;
+
+ done = false;
+
+ tq_buf = &dev_info->tqmap[vchan].dma_buf;
+
+ nb_enqueued = n;
+ if (unlikely(!adapter->ebuf.count))
+ edma_circular_buffer_flush_to_evdev(adapter, tq_buf,
+ &nb_enqueued);
+
+ if (likely(nb_enqueued == n))
+ goto check;
+
+ /* Failed to enqueue events case */
+ for (i = nb_enqueued; i < n; i++) {
+ ops = tq_buf->op_buffer[tq_buf->head];
+ edma_circular_buffer_add(&adapter->ebuf, ops);
+ tq_buf->head = (tq_buf->head + 1) % tq_buf->size;
+ }
+
+check:
+ nb_deq += n;
+ if (nb_deq >= max_deq) {
+ if ((vchan + 1) == num_vchan)
+ adapter->next_dmadev_id =
+ (dma_dev_id + 1) % num_dma_dev;
+
+ dev_info->next_vchan_id = (vchan + 1) % num_vchan;
+
+ return nb_deq;
+ }
+ }
+ }
+ adapter->next_dmadev_id = 0;
+
+ } while (done == false);
+
+ return nb_deq;
+}
+
+static int
+edma_adapter_run(struct event_dma_adapter *adapter, unsigned int max_ops)
+{
+ unsigned int ops_left = max_ops;
+
+ while (ops_left > 0) {
+ unsigned int e_cnt, d_cnt;
+
+ e_cnt = edma_adapter_deq_run(adapter, ops_left);
+ ops_left -= RTE_MIN(ops_left, e_cnt);
+
+ d_cnt = edma_adapter_enq_run(adapter, ops_left);
+ ops_left -= RTE_MIN(ops_left, d_cnt);
+
+ if (e_cnt == 0 && d_cnt == 0)
+ break;
+ }
+
+ if (ops_left == max_ops) {
+ rte_event_maintain(adapter->eventdev_id, adapter->event_port_id, 0);
+ return -EAGAIN;
+ } else
+ return 0;
+}
+
+static int
+edma_service_func(void *args)
+{
+ struct event_dma_adapter *adapter = args;
+ int ret;
+
+ if (rte_spinlock_trylock(&adapter->lock) == 0)
+ return 0;
+ ret = edma_adapter_run(adapter, adapter->max_nb);
+ rte_spinlock_unlock(&adapter->lock);
+
+ return ret;
+}
+
+static int
+edma_init_service(struct event_dma_adapter *adapter, uint8_t id)
+{
+ struct rte_event_dma_adapter_conf adapter_conf;
+ struct rte_service_spec service;
+ uint32_t impl_rel;
+ int ret;
+
+ if (adapter->service_initialized)
+ return 0;
+
+ memset(&service, 0, sizeof(service));
+ snprintf(service.name, DMA_ADAPTER_NAME_LEN, "rte_event_dma_adapter_%d", id);
+ service.socket_id = adapter->socket_id;
+ service.callback = edma_service_func;
+ service.callback_userdata = adapter;
+
+ /* Service function handles locking for queue add/del updates */
+ service.capabilities = RTE_SERVICE_CAP_MT_SAFE;
+ ret = rte_service_component_register(&service, &adapter->service_id);
+ if (ret) {
+ RTE_EDEV_LOG_ERR("failed to register service %s err = %" PRId32, service.name, ret);
+ return ret;
+ }
+
+ ret = adapter->conf_cb(id, adapter->eventdev_id, &adapter_conf, adapter->conf_arg);
+ if (ret) {
+ RTE_EDEV_LOG_ERR("configuration callback failed err = %" PRId32, ret);
+ return ret;
+ }
+
+ adapter->max_nb = adapter_conf.max_nb;
+ adapter->event_port_id = adapter_conf.event_port_id;
+
+ if (rte_event_port_attr_get(adapter->eventdev_id, adapter->event_port_id,
+ RTE_EVENT_PORT_ATTR_IMPLICIT_RELEASE_DISABLE, &impl_rel)) {
+ RTE_EDEV_LOG_ERR("Failed to get port info for eventdev %" PRId32,
+ adapter->eventdev_id);
+ edma_circular_buffer_free(&adapter->ebuf);
+ rte_free(adapter);
+ return -EINVAL;
+ }
+
+ adapter->implicit_release_disabled = (uint8_t)impl_rel;
+ adapter->service_initialized = 1;
+
+ return ret;
+}
+
static void
edma_update_vchanq_info(struct event_dma_adapter *adapter, struct dma_device_info *dev_info,
uint16_t vchan, uint8_t add)
@@ -391,6 +893,60 @@ edma_update_vchanq_info(struct event_dma_adapter *adapter, struct dma_device_inf
}
}
+static int
+edma_add_vchan(struct event_dma_adapter *adapter, int16_t dma_dev_id, uint16_t vchan)
+{
+ struct dma_device_info *dev_info = &adapter->dma_devs[dma_dev_id];
+ struct dma_vchan_info *vchanq;
+ struct dma_vchan_info *tqmap;
+ uint16_t nb_vchans;
+ uint32_t i;
+
+ if (dev_info->vchanq == NULL) {
+ nb_vchans = dev_info->num_dma_dev_vchan;
+
+ dev_info->vchanq = rte_zmalloc_socket(adapter->mem_name,
+ nb_vchans * sizeof(struct dma_vchan_info),
+ 0, adapter->socket_id);
+ if (dev_info->vchanq == NULL)
+ return -ENOMEM;
+
+ dev_info->tqmap = rte_zmalloc_socket(adapter->mem_name,
+ nb_vchans * sizeof(struct dma_vchan_info),
+ 0, adapter->socket_id);
+ if (dev_info->tqmap == NULL)
+ return -ENOMEM;
+
+ for (i = 0; i < nb_vchans; i++) {
+ vchanq = &dev_info->vchanq[i];
+
+ if (edma_circular_buffer_init("dma_dev_circular_buffer", &vchanq->dma_buf,
+ DMA_ADAPTER_OPS_BUFFER_SIZE)) {
+ RTE_EDEV_LOG_ERR("Failed to get memory for dma_dev buffer");
+ rte_free(vchanq);
+ return -ENOMEM;
+ }
+
+ tqmap = &dev_info->tqmap[i];
+ if (edma_circular_buffer_init("dma_dev_circular_trans_buf", &tqmap->dma_buf,
+ DMA_ADAPTER_OPS_BUFFER_SIZE)) {
+ RTE_EDEV_LOG_ERR(
+ "Failed to get memory for dma_dev transaction buffer");
+ rte_free(tqmap);
+ return -ENOMEM;
+ }
+ }
+ }
+
+ if (vchan == RTE_DMA_ALL_VCHAN) {
+ for (i = 0; i < dev_info->num_dma_dev_vchan; i++)
+ edma_update_vchanq_info(adapter, dev_info, i, 1);
+ } else
+ edma_update_vchanq_info(adapter, dev_info, vchan, 1);
+
+ return 0;
+}
+
int
rte_event_dma_adapter_vchan_add(uint8_t id, int16_t dma_dev_id, uint16_t vchan,
const struct rte_event *event)
@@ -470,6 +1026,38 @@ rte_event_dma_adapter_vchan_add(uint8_t id, int16_t dma_dev_id, uint16_t vchan,
edma_update_vchanq_info(adapter, &adapter->dma_devs[dma_dev_id], vchan, 1);
}
+ /* In case HW cap is RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP_NEW, or SW adapter, initiate
+ * services so the application can choose which ever way it wants to use the adapter.
+ *
+ * Case 1: RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP_NEW. Application may wants to use one
+ * of below two modes
+ *
+ * a. OP_FORWARD mode -> HW Dequeue + SW enqueue
+ * b. OP_NEW mode -> HW Dequeue
+ *
+ * Case 2: No HW caps, use SW adapter
+ *
+ * a. OP_FORWARD mode -> SW enqueue & dequeue
+ * b. OP_NEW mode -> SW Dequeue
+ */
+ if ((cap & RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP_NEW &&
+ !(cap & RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP_FWD) &&
+ adapter->mode == RTE_EVENT_DMA_ADAPTER_OP_FORWARD) ||
+ (!(cap & RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP_NEW) &&
+ !(cap & RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP_FWD) &&
+ !(cap & RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_VCHAN_EV_BIND))) {
+ rte_spinlock_lock(&adapter->lock);
+ ret = edma_init_service(adapter, id);
+ if (ret == 0)
+ ret = edma_add_vchan(adapter, dma_dev_id, vchan);
+ rte_spinlock_unlock(&adapter->lock);
+
+ if (ret)
+ return ret;
+
+ rte_service_component_runstate_set(adapter->service_id, 1);
+ }
+
return 0;
}
@@ -533,6 +1121,7 @@ rte_event_dma_adapter_vchan_del(uint8_t id, int16_t dma_dev_id, uint16_t vchan)
}
rte_spinlock_unlock(&adapter->lock);
+ rte_service_component_runstate_set(adapter->service_id, adapter->nb_vchanq);
}
return ret;
--
2.25.1
^ permalink raw reply [flat|nested] 110+ messages in thread
* [PATCH v4 06/12] eventdev: add support for DMA adapter start and stop
2023-09-26 10:32 ` [PATCH v4 00/12] event DMA adapter library support Amit Prakash Shukla
` (4 preceding siblings ...)
2023-09-26 10:32 ` [PATCH v4 05/12] eventdev: add support for DMA adapter service function Amit Prakash Shukla
@ 2023-09-26 10:32 ` Amit Prakash Shukla
2023-09-26 10:32 ` [PATCH v4 07/12] eventdev: add support for DMA adapter service ID get Amit Prakash Shukla
` (6 subsequent siblings)
12 siblings, 0 replies; 110+ messages in thread
From: Amit Prakash Shukla @ 2023-09-26 10:32 UTC (permalink / raw)
To: Amit Prakash Shukla, Jerin Jacob
Cc: dev, fengchengwen, kevin.laatz, bruce.richardson, conor.walsh,
vattunuru, g.singh, sachin.saxena, hemant.agrawal, cheng1.jiang,
ndabilpuram, anoobj, mb
Added API support to start and stop DMA adapter.
Signed-off-by: Amit Prakash Shukla <amitprakashs@marvell.com>
---
lib/eventdev/rte_event_dma_adapter.c | 69 ++++++++++++++++++++++++++++
1 file changed, 69 insertions(+)
diff --git a/lib/eventdev/rte_event_dma_adapter.c b/lib/eventdev/rte_event_dma_adapter.c
index 8349b95796..e5c2be3a2f 100644
--- a/lib/eventdev/rte_event_dma_adapter.c
+++ b/lib/eventdev/rte_event_dma_adapter.c
@@ -75,6 +75,13 @@ struct dma_device_info {
/* Next queue pair to be processed */
uint16_t next_vchan_id;
+ /* Set to indicate processing has been started */
+ uint8_t dev_started;
+
+ /* Set to indicate dmadev->eventdev packet
+ * transfer uses a hardware mechanism
+ */
+ uint8_t internal_event_port;
} __rte_cache_aligned;
struct event_dma_adapter {
@@ -1126,3 +1133,65 @@ rte_event_dma_adapter_vchan_del(uint8_t id, int16_t dma_dev_id, uint16_t vchan)
return ret;
}
+
+static int
+edma_adapter_ctrl(uint8_t id, int start)
+{
+ struct event_dma_adapter *adapter;
+ struct dma_device_info *dev_info;
+ struct rte_eventdev *dev;
+ uint16_t num_dma_dev;
+ int stop = !start;
+ int use_service;
+ uint32_t i;
+
+ use_service = 0;
+ EVENT_DMA_ADAPTER_ID_VALID_OR_ERR_RET(id, -EINVAL);
+ adapter = edma_id_to_adapter(id);
+ if (adapter == NULL)
+ return -EINVAL;
+
+ num_dma_dev = rte_dma_count_avail();
+ dev = &rte_eventdevs[adapter->eventdev_id];
+
+ for (i = 0; i < num_dma_dev; i++) {
+ dev_info = &adapter->dma_devs[i];
+ /* start check for num queue pairs */
+ if (start && !dev_info->num_vchanq)
+ continue;
+ /* stop check if dev has been started */
+ if (stop && !dev_info->dev_started)
+ continue;
+ use_service |= !dev_info->internal_event_port;
+ dev_info->dev_started = start;
+ if (dev_info->internal_event_port == 0)
+ continue;
+ start ? (*dev->dev_ops->dma_adapter_start)(dev, i) :
+ (*dev->dev_ops->dma_adapter_stop)(dev, i);
+ }
+
+ if (use_service)
+ rte_service_runstate_set(adapter->service_id, start);
+
+ return 0;
+}
+
+int
+rte_event_dma_adapter_start(uint8_t id)
+{
+ struct event_dma_adapter *adapter;
+
+ EVENT_DMA_ADAPTER_ID_VALID_OR_ERR_RET(id, -EINVAL);
+
+ adapter = edma_id_to_adapter(id);
+ if (adapter == NULL)
+ return -EINVAL;
+
+ return edma_adapter_ctrl(id, 1);
+}
+
+int
+rte_event_dma_adapter_stop(uint8_t id)
+{
+ return edma_adapter_ctrl(id, 0);
+}
--
2.25.1
^ permalink raw reply [flat|nested] 110+ messages in thread
* [PATCH v4 07/12] eventdev: add support for DMA adapter service ID get
2023-09-26 10:32 ` [PATCH v4 00/12] event DMA adapter library support Amit Prakash Shukla
` (5 preceding siblings ...)
2023-09-26 10:32 ` [PATCH v4 06/12] eventdev: add support for DMA adapter start and stop Amit Prakash Shukla
@ 2023-09-26 10:32 ` Amit Prakash Shukla
2023-09-26 10:32 ` [PATCH v4 08/12] eventdev: add DMA adapter support for runtime params Amit Prakash Shukla
` (5 subsequent siblings)
12 siblings, 0 replies; 110+ messages in thread
From: Amit Prakash Shukla @ 2023-09-26 10:32 UTC (permalink / raw)
To: Amit Prakash Shukla, Jerin Jacob
Cc: dev, fengchengwen, kevin.laatz, bruce.richardson, conor.walsh,
vattunuru, g.singh, sachin.saxena, hemant.agrawal, cheng1.jiang,
ndabilpuram, anoobj, mb
Added API support to get DMA adapter service ID. Service id
returned in the variable by the API call shall be used by application
to map a service core.
Signed-off-by: Amit Prakash Shukla <amitprakashs@marvell.com>
---
lib/eventdev/rte_event_dma_adapter.c | 17 +++++++++++++++++
1 file changed, 17 insertions(+)
diff --git a/lib/eventdev/rte_event_dma_adapter.c b/lib/eventdev/rte_event_dma_adapter.c
index e5c2be3a2f..9e7ed622f2 100644
--- a/lib/eventdev/rte_event_dma_adapter.c
+++ b/lib/eventdev/rte_event_dma_adapter.c
@@ -1134,6 +1134,23 @@ rte_event_dma_adapter_vchan_del(uint8_t id, int16_t dma_dev_id, uint16_t vchan)
return ret;
}
+int
+rte_event_dma_adapter_service_id_get(uint8_t id, uint32_t *service_id)
+{
+ struct event_dma_adapter *adapter;
+
+ EVENT_DMA_ADAPTER_ID_VALID_OR_ERR_RET(id, -EINVAL);
+
+ adapter = edma_id_to_adapter(id);
+ if (adapter == NULL || service_id == NULL)
+ return -EINVAL;
+
+ if (adapter->service_initialized)
+ *service_id = adapter->service_id;
+
+ return adapter->service_initialized ? 0 : -ESRCH;
+}
+
static int
edma_adapter_ctrl(uint8_t id, int start)
{
--
2.25.1
^ permalink raw reply [flat|nested] 110+ messages in thread
* [PATCH v4 08/12] eventdev: add DMA adapter support for runtime params
2023-09-26 10:32 ` [PATCH v4 00/12] event DMA adapter library support Amit Prakash Shukla
` (6 preceding siblings ...)
2023-09-26 10:32 ` [PATCH v4 07/12] eventdev: add support for DMA adapter service ID get Amit Prakash Shukla
@ 2023-09-26 10:32 ` Amit Prakash Shukla
2023-09-26 10:32 ` [PATCH v4 09/12] eventdev: add support for DMA adapter stats Amit Prakash Shukla
` (4 subsequent siblings)
12 siblings, 0 replies; 110+ messages in thread
From: Amit Prakash Shukla @ 2023-09-26 10:32 UTC (permalink / raw)
To: Amit Prakash Shukla, Jerin Jacob
Cc: dev, fengchengwen, kevin.laatz, bruce.richardson, conor.walsh,
vattunuru, g.singh, sachin.saxena, hemant.agrawal, cheng1.jiang,
ndabilpuram, anoobj, mb
Added support to set and get runtime params for DMA adapter. The
parameters that can be set/get are defined in
struct rte_event_dma_adapter_runtime_params.
Signed-off-by: Amit Prakash Shukla <amitprakashs@marvell.com>
---
lib/eventdev/rte_event_dma_adapter.c | 93 ++++++++++++++++++++++++++++
1 file changed, 93 insertions(+)
diff --git a/lib/eventdev/rte_event_dma_adapter.c b/lib/eventdev/rte_event_dma_adapter.c
index 9e7ed622f2..cf53e7f429 100644
--- a/lib/eventdev/rte_event_dma_adapter.c
+++ b/lib/eventdev/rte_event_dma_adapter.c
@@ -1212,3 +1212,96 @@ rte_event_dma_adapter_stop(uint8_t id)
{
return edma_adapter_ctrl(id, 0);
}
+
+#define DEFAULT_MAX_NB 128
+
+int
+rte_event_dma_adapter_runtime_params_init(struct rte_event_dma_adapter_runtime_params *params)
+{
+ if (params == NULL)
+ return -EINVAL;
+
+ memset(params, 0, sizeof(*params));
+ params->max_nb = DEFAULT_MAX_NB;
+
+ return 0;
+}
+
+static int
+dma_adapter_cap_check(struct event_dma_adapter *adapter)
+{
+ uint32_t caps;
+ int ret;
+
+ if (!adapter->nb_vchanq)
+ return -EINVAL;
+
+ ret = rte_event_dma_adapter_caps_get(adapter->eventdev_id, adapter->next_dmadev_id, &caps);
+ if (ret) {
+ RTE_EDEV_LOG_ERR("Failed to get adapter caps dev %" PRIu8 " cdev %" PRIu8,
+ adapter->eventdev_id, adapter->next_dmadev_id);
+ return ret;
+ }
+
+ if ((caps & RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP_FWD) ||
+ (caps & RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP_NEW))
+ return -ENOTSUP;
+
+ return 0;
+}
+
+int
+rte_event_dma_adapter_runtime_params_set(uint8_t id,
+ struct rte_event_dma_adapter_runtime_params *params)
+{
+ struct event_dma_adapter *adapter;
+ int ret;
+
+ EVENT_DMA_ADAPTER_ID_VALID_OR_ERR_RET(id, -EINVAL);
+
+ if (params == NULL) {
+ RTE_EDEV_LOG_ERR("params pointer is NULL\n");
+ return -EINVAL;
+ }
+
+ adapter = edma_id_to_adapter(id);
+ if (adapter == NULL)
+ return -EINVAL;
+
+ ret = dma_adapter_cap_check(adapter);
+ if (ret)
+ return ret;
+
+ rte_spinlock_lock(&adapter->lock);
+ adapter->max_nb = params->max_nb;
+ rte_spinlock_unlock(&adapter->lock);
+
+ return 0;
+}
+
+int
+rte_event_dma_adapter_runtime_params_get(uint8_t id,
+ struct rte_event_dma_adapter_runtime_params *params)
+{
+ struct event_dma_adapter *adapter;
+ int ret;
+
+ EVENT_DMA_ADAPTER_ID_VALID_OR_ERR_RET(id, -EINVAL);
+
+ if (params == NULL) {
+ RTE_EDEV_LOG_ERR("params pointer is NULL\n");
+ return -EINVAL;
+ }
+
+ adapter = edma_id_to_adapter(id);
+ if (adapter == NULL)
+ return -EINVAL;
+
+ ret = dma_adapter_cap_check(adapter);
+ if (ret)
+ return ret;
+
+ params->max_nb = adapter->max_nb;
+
+ return 0;
+}
--
2.25.1
^ permalink raw reply [flat|nested] 110+ messages in thread
* [PATCH v4 09/12] eventdev: add support for DMA adapter stats
2023-09-26 10:32 ` [PATCH v4 00/12] event DMA adapter library support Amit Prakash Shukla
` (7 preceding siblings ...)
2023-09-26 10:32 ` [PATCH v4 08/12] eventdev: add DMA adapter support for runtime params Amit Prakash Shukla
@ 2023-09-26 10:32 ` Amit Prakash Shukla
2023-09-26 10:32 ` [PATCH v4 10/12] eventdev: add support for DMA adapter enqueue Amit Prakash Shukla
` (3 subsequent siblings)
12 siblings, 0 replies; 110+ messages in thread
From: Amit Prakash Shukla @ 2023-09-26 10:32 UTC (permalink / raw)
To: Amit Prakash Shukla, Jerin Jacob
Cc: dev, fengchengwen, kevin.laatz, bruce.richardson, conor.walsh,
vattunuru, g.singh, sachin.saxena, hemant.agrawal, cheng1.jiang,
ndabilpuram, anoobj, mb
Added DMA adapter stats API support to get and reset stats. DMA
SW adapter stats and eventdev driver supported stats for enqueue
and dequeue are reported by get API.
Signed-off-by: Amit Prakash Shukla <amitprakashs@marvell.com>
---
lib/eventdev/rte_event_dma_adapter.c | 95 ++++++++++++++++++++++++++++
1 file changed, 95 insertions(+)
diff --git a/lib/eventdev/rte_event_dma_adapter.c b/lib/eventdev/rte_event_dma_adapter.c
index cf53e7f429..35acec6caf 100644
--- a/lib/eventdev/rte_event_dma_adapter.c
+++ b/lib/eventdev/rte_event_dma_adapter.c
@@ -143,6 +143,9 @@ struct event_dma_adapter {
/* Loop counter to flush dma ops */
uint16_t transmit_loop_count;
+
+ /* Per instance stats structure */
+ struct rte_event_dma_adapter_stats dma_stats;
} __rte_cache_aligned;
static struct event_dma_adapter **event_dma_adapter;
@@ -472,6 +475,7 @@ rte_event_dma_adapter_free(uint8_t id)
static inline unsigned int
edma_enq_to_dma_dev(struct event_dma_adapter *adapter, struct rte_event *ev, unsigned int cnt)
{
+ struct rte_event_dma_adapter_stats *stats = &adapter->dma_stats;
struct dma_vchan_info *vchan_qinfo = NULL;
struct rte_event_dma_adapter_op *dma_op;
uint16_t vchan, nb_enqueued = 0;
@@ -481,6 +485,7 @@ edma_enq_to_dma_dev(struct event_dma_adapter *adapter, struct rte_event *ev, uns
ret = 0;
n = 0;
+ stats->event_deq_count += cnt;
for (i = 0; i < cnt; i++) {
dma_op = ev[i].event_ptr;
@@ -503,6 +508,7 @@ edma_enq_to_dma_dev(struct event_dma_adapter *adapter, struct rte_event *ev, uns
ret = edma_circular_buffer_flush_to_dma_dev(adapter, &vchan_qinfo->dma_buf,
dma_dev_id, vchan,
&nb_enqueued);
+ stats->dma_enq_count += nb_enqueued;
n += nb_enqueued;
/**
@@ -549,6 +555,7 @@ edma_adapter_dev_flush(struct event_dma_adapter *adapter, int16_t dma_dev_id,
static unsigned int
edma_adapter_enq_flush(struct event_dma_adapter *adapter)
{
+ struct rte_event_dma_adapter_stats *stats = &adapter->dma_stats;
int16_t dma_dev_id;
uint16_t nb_enqueued = 0;
uint16_t nb_ops_flushed = 0;
@@ -563,6 +570,8 @@ edma_adapter_enq_flush(struct event_dma_adapter *adapter)
if (!nb_ops_flushed)
adapter->stop_enq_to_dma_dev = false;
+ stats->dma_enq_count += nb_enqueued;
+
return nb_enqueued;
}
@@ -574,6 +583,7 @@ edma_adapter_enq_flush(struct event_dma_adapter *adapter)
static int
edma_adapter_enq_run(struct event_dma_adapter *adapter, unsigned int max_enq)
{
+ struct rte_event_dma_adapter_stats *stats = &adapter->dma_stats;
uint8_t event_port_id = adapter->event_port_id;
uint8_t event_dev_id = adapter->eventdev_id;
struct rte_event ev[DMA_BATCH_SIZE];
@@ -593,6 +603,7 @@ edma_adapter_enq_run(struct event_dma_adapter *adapter, unsigned int max_enq)
break;
}
+ stats->event_poll_count++;
n = rte_event_dequeue_burst(event_dev_id, event_port_id, ev, DMA_BATCH_SIZE, 0);
if (!n)
@@ -613,6 +624,7 @@ static inline uint16_t
edma_ops_enqueue_burst(struct event_dma_adapter *adapter, struct rte_event_dma_adapter_op **ops,
uint16_t num)
{
+ struct rte_event_dma_adapter_stats *stats = &adapter->dma_stats;
uint8_t event_port_id = adapter->event_port_id;
uint8_t event_dev_id = adapter->eventdev_id;
struct rte_event events[DMA_BATCH_SIZE];
@@ -652,6 +664,10 @@ edma_ops_enqueue_burst(struct event_dma_adapter *adapter, struct rte_event_dma_a
} while (retry++ < DMA_ADAPTER_MAX_EV_ENQ_RETRIES && nb_enqueued < nb_ev);
+ stats->event_enq_fail_count += nb_ev - nb_enqueued;
+ stats->event_enq_count += nb_enqueued;
+ stats->event_enq_retry_count += retry - 1;
+
return nb_enqueued;
}
@@ -706,6 +722,7 @@ edma_ops_buffer_flush(struct event_dma_adapter *adapter)
static inline unsigned int
edma_adapter_deq_run(struct event_dma_adapter *adapter, unsigned int max_deq)
{
+ struct rte_event_dma_adapter_stats *stats = &adapter->dma_stats;
struct dma_vchan_info *vchan_info;
struct dma_ops_circular_buffer *tq_buf;
struct rte_event_dma_adapter_op *ops;
@@ -743,6 +760,7 @@ edma_adapter_deq_run(struct event_dma_adapter *adapter, unsigned int max_deq)
continue;
done = false;
+ stats->dma_deq_count += n;
tq_buf = &dev_info->tqmap[vchan].dma_buf;
@@ -1305,3 +1323,80 @@ rte_event_dma_adapter_runtime_params_get(uint8_t id,
return 0;
}
+
+int
+rte_event_dma_adapter_stats_get(uint8_t id, struct rte_event_dma_adapter_stats *stats)
+{
+ struct rte_event_dma_adapter_stats dev_stats_sum = {0};
+ struct rte_event_dma_adapter_stats dev_stats;
+ struct event_dma_adapter *adapter;
+ struct dma_device_info *dev_info;
+ struct rte_eventdev *dev;
+ uint16_t num_dma_dev;
+ uint32_t i;
+ int ret;
+
+ EVENT_DMA_ADAPTER_ID_VALID_OR_ERR_RET(id, -EINVAL);
+
+ adapter = edma_id_to_adapter(id);
+ if (adapter == NULL || stats == NULL)
+ return -EINVAL;
+
+ num_dma_dev = rte_dma_count_avail();
+ dev = &rte_eventdevs[adapter->eventdev_id];
+ memset(stats, 0, sizeof(*stats));
+ for (i = 0; i < num_dma_dev; i++) {
+ dev_info = &adapter->dma_devs[i];
+
+ if (dev_info->internal_event_port == 0 ||
+ dev->dev_ops->dma_adapter_stats_get == NULL)
+ continue;
+
+ ret = (*dev->dev_ops->dma_adapter_stats_get)(dev, i, &dev_stats);
+ if (ret)
+ continue;
+
+ dev_stats_sum.dma_deq_count += dev_stats.dma_deq_count;
+ dev_stats_sum.event_enq_count += dev_stats.event_enq_count;
+ }
+
+ if (adapter->service_initialized)
+ *stats = adapter->dma_stats;
+
+ stats->dma_deq_count += dev_stats_sum.dma_deq_count;
+ stats->event_enq_count += dev_stats_sum.event_enq_count;
+
+ return 0;
+}
+
+int
+rte_event_dma_adapter_stats_reset(uint8_t id)
+{
+ struct event_dma_adapter *adapter;
+ struct dma_device_info *dev_info;
+ struct rte_eventdev *dev;
+ uint16_t num_dma_dev;
+ uint32_t i;
+
+ EVENT_DMA_ADAPTER_ID_VALID_OR_ERR_RET(id, -EINVAL);
+
+ adapter = edma_id_to_adapter(id);
+ if (adapter == NULL)
+ return -EINVAL;
+
+ num_dma_dev = rte_dma_count_avail();
+ dev = &rte_eventdevs[adapter->eventdev_id];
+ for (i = 0; i < num_dma_dev; i++) {
+ dev_info = &adapter->dma_devs[i];
+
+ if (dev_info->internal_event_port == 0 ||
+ dev->dev_ops->dma_adapter_stats_reset == NULL)
+ continue;
+
+ (*dev->dev_ops->dma_adapter_stats_reset)(dev, i);
+ }
+
+ memset(&adapter->dma_stats, 0, sizeof(adapter->dma_stats));
+
+ return 0;
+}
--
2.25.1
^ permalink raw reply [flat|nested] 110+ messages in thread
* [PATCH v4 10/12] eventdev: add support for DMA adapter enqueue
2023-09-26 10:32 ` [PATCH v4 00/12] event DMA adapter library support Amit Prakash Shukla
` (8 preceding siblings ...)
2023-09-26 10:32 ` [PATCH v4 09/12] eventdev: add support for DMA adapter stats Amit Prakash Shukla
@ 2023-09-26 10:32 ` Amit Prakash Shukla
2023-09-26 10:32 ` [PATCH v4 11/12] eventdev: add DMA adapter event port get Amit Prakash Shukla
` (2 subsequent siblings)
12 siblings, 0 replies; 110+ messages in thread
From: Amit Prakash Shukla @ 2023-09-26 10:32 UTC (permalink / raw)
To: Amit Prakash Shukla, Jerin Jacob
Cc: dev, fengchengwen, kevin.laatz, bruce.richardson, conor.walsh,
vattunuru, g.singh, sachin.saxena, hemant.agrawal, cheng1.jiang,
ndabilpuram, anoobj, mb
Added API support to enqueue a DMA operation to the DMA driver.
Signed-off-by: Amit Prakash Shukla <amitprakashs@marvell.com>
---
lib/eventdev/rte_event_dma_adapter.c | 13 +++++++++++++
1 file changed, 13 insertions(+)
diff --git a/lib/eventdev/rte_event_dma_adapter.c b/lib/eventdev/rte_event_dma_adapter.c
index 35acec6caf..9770b0b8df 100644
--- a/lib/eventdev/rte_event_dma_adapter.c
+++ b/lib/eventdev/rte_event_dma_adapter.c
@@ -1400,3 +1400,16 @@ rte_event_dma_adapter_stats_reset(uint8_t id)
return 0;
}
+
+uint16_t
+rte_event_dma_adapter_enqueue(uint8_t dev_id, uint8_t port_id, struct rte_event ev[],
+ uint16_t nb_events)
+{
+ const struct rte_event_fp_ops *fp_ops;
+ void *port;
+
+ fp_ops = &rte_event_fp_ops[dev_id];
+ port = fp_ops->data[port_id];
+
+ return fp_ops->dma_enqueue(port, ev, nb_events);
+}
--
2.25.1
^ permalink raw reply [flat|nested] 110+ messages in thread
* [PATCH v4 11/12] eventdev: add DMA adapter event port get
2023-09-26 10:32 ` [PATCH v4 00/12] event DMA adapter library support Amit Prakash Shukla
` (9 preceding siblings ...)
2023-09-26 10:32 ` [PATCH v4 10/12] eventdev: add support for DMA adapter enqueue Amit Prakash Shukla
@ 2023-09-26 10:32 ` Amit Prakash Shukla
2023-09-26 10:32 ` [PATCH v4 12/12] app/test: add event DMA adapter auto-test Amit Prakash Shukla
2023-09-28 10:36 ` [PATCH v5 00/12] event DMA adapter library support Amit Prakash Shukla
12 siblings, 0 replies; 110+ messages in thread
From: Amit Prakash Shukla @ 2023-09-26 10:32 UTC (permalink / raw)
To: Amit Prakash Shukla, Jerin Jacob
Cc: dev, fengchengwen, kevin.laatz, bruce.richardson, conor.walsh,
vattunuru, g.singh, sachin.saxena, hemant.agrawal, cheng1.jiang,
ndabilpuram, anoobj, mb
Added support for DMA adapter event port get.
Signed-off-by: Amit Prakash Shukla <amitprakashs@marvell.com>
---
lib/eventdev/rte_event_dma_adapter.c | 16 ++++++++++++++++
1 file changed, 16 insertions(+)
diff --git a/lib/eventdev/rte_event_dma_adapter.c b/lib/eventdev/rte_event_dma_adapter.c
index 9770b0b8df..2b0919d705 100644
--- a/lib/eventdev/rte_event_dma_adapter.c
+++ b/lib/eventdev/rte_event_dma_adapter.c
@@ -472,6 +472,22 @@ rte_event_dma_adapter_free(uint8_t id)
return 0;
}
+int
+rte_event_dma_adapter_event_port_get(uint8_t id, uint8_t *event_port_id)
+{
+ struct event_dma_adapter *adapter;
+
+ EVENT_DMA_ADAPTER_ID_VALID_OR_ERR_RET(id, -EINVAL);
+
+ adapter = edma_id_to_adapter(id);
+ if (adapter == NULL || event_port_id == NULL)
+ return -EINVAL;
+
+ *event_port_id = adapter->event_port_id;
+
+ return 0;
+}
+
static inline unsigned int
edma_enq_to_dma_dev(struct event_dma_adapter *adapter, struct rte_event *ev, unsigned int cnt)
{
--
2.25.1
^ permalink raw reply [flat|nested] 110+ messages in thread
* [PATCH v4 12/12] app/test: add event DMA adapter auto-test
2023-09-26 10:32 ` [PATCH v4 00/12] event DMA adapter library support Amit Prakash Shukla
` (10 preceding siblings ...)
2023-09-26 10:32 ` [PATCH v4 11/12] eventdev: add DMA adapter event port get Amit Prakash Shukla
@ 2023-09-26 10:32 ` Amit Prakash Shukla
2023-09-27 18:27 ` Jerin Jacob
2023-09-28 10:36 ` [PATCH v5 00/12] event DMA adapter library support Amit Prakash Shukla
12 siblings, 1 reply; 110+ messages in thread
From: Amit Prakash Shukla @ 2023-09-26 10:32 UTC (permalink / raw)
Cc: dev, jerinj, fengchengwen, kevin.laatz, bruce.richardson,
conor.walsh, vattunuru, g.singh, sachin.saxena, hemant.agrawal,
cheng1.jiang, ndabilpuram, anoobj, mb, Amit Prakash Shukla
Added testsuite to test the dma adapter functionality.
The testsuite detects event and DMA device capability
and accordingly dma adapter is configured and modes are
tested.
Signed-off-by: Amit Prakash Shukla <amitprakashs@marvell.com>
---
app/test/meson.build | 1 +
app/test/test_event_dma_adapter.c | 805 ++++++++++++++++++++++++++++++
2 files changed, 806 insertions(+)
create mode 100644 app/test/test_event_dma_adapter.c
diff --git a/app/test/meson.build b/app/test/meson.build
index 05bae9216d..7caf5ae5fc 100644
--- a/app/test/meson.build
+++ b/app/test/meson.build
@@ -66,6 +66,7 @@ source_file_deps = {
'test_errno.c': [],
'test_ethdev_link.c': ['ethdev'],
'test_event_crypto_adapter.c': ['cryptodev', 'eventdev', 'bus_vdev'],
+ 'test_event_dma_adapter.c': ['dmadev', 'eventdev', 'bus_vdev'],
'test_event_eth_rx_adapter.c': ['ethdev', 'eventdev', 'bus_vdev'],
'test_event_eth_tx_adapter.c': ['bus_vdev', 'ethdev', 'net_ring', 'eventdev'],
'test_event_ring.c': ['eventdev'],
diff --git a/app/test/test_event_dma_adapter.c b/app/test/test_event_dma_adapter.c
new file mode 100644
index 0000000000..fe1311e7f8
--- /dev/null
+++ b/app/test/test_event_dma_adapter.c
@@ -0,0 +1,805 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright (c) 2023 Marvell.
+ */
+
+#include "test.h"
+#include <string.h>
+#include <rte_common.h>
+#include <rte_malloc.h>
+#include <rte_mempool.h>
+#include <rte_mbuf.h>
+#include <rte_random.h>
+
+#ifdef RTE_EXEC_ENV_WINDOWS
+static int
+test_event_dma_adapter(void)
+{
+ printf("event_dma_adapter not supported on Windows, skipping test\n");
+ return TEST_SKIPPED;
+}
+
+#else
+
+#include <rte_bus_vdev.h>
+#include <rte_dmadev.h>
+#include <rte_eventdev.h>
+#include <rte_event_dma_adapter.h>
+#include <rte_service.h>
+
+#define NUM_MBUFS (8191)
+#define MBUF_CACHE_SIZE (256)
+#define TEST_APP_PORT_ID 0
+#define TEST_APP_EV_QUEUE_ID 0
+#define TEST_APP_EV_PRIORITY 0
+#define TEST_APP_EV_FLOWID 0xAABB
+#define TEST_DMA_EV_QUEUE_ID 1
+#define TEST_ADAPTER_ID 0
+#define TEST_DMA_DEV_ID 0
+#define TEST_DMA_VCHAN_ID 0
+#define PACKET_LENGTH 1024
+#define NB_TEST_PORTS 1
+#define NB_TEST_QUEUES 2
+#define NUM_CORES 2
+#define DMA_OP_POOL_SIZE 128
+#define TEST_MAX_OP 128
+#define TEST_RINGSIZE 512
+
+#define MBUF_SIZE (RTE_PKTMBUF_HEADROOM + PACKET_LENGTH)
+
+/* Handle log statements in same manner as test macros */
+#define LOG_DBG(...) RTE_LOG(DEBUG, EAL, __VA_ARGS__)
+
+struct event_dma_adapter_test_params {
+ struct rte_mempool *src_mbuf_pool;
+ struct rte_mempool *dst_mbuf_pool;
+ struct rte_mempool *op_mpool;
+ uint8_t dma_event_port_id;
+ uint8_t internal_port_op_fwd;
+};
+
+struct rte_event dma_response_info = {
+ .queue_id = TEST_APP_EV_QUEUE_ID,
+ .sched_type = RTE_SCHED_TYPE_ATOMIC,
+ .flow_id = TEST_APP_EV_FLOWID,
+ .priority = TEST_APP_EV_PRIORITY
+};
+
+static struct event_dma_adapter_test_params params;
+static uint8_t dma_adapter_setup_done;
+static uint32_t slcore_id;
+static int evdev;
+
+static int
+send_recv_ev(struct rte_event *ev)
+{
+ struct rte_event recv_ev[TEST_MAX_OP];
+ uint16_t nb_enqueued = 0;
+ int i = 0;
+
+ if (params.internal_port_op_fwd) {
+ nb_enqueued = rte_event_dma_adapter_enqueue(evdev, TEST_APP_PORT_ID, ev,
+ TEST_MAX_OP);
+ } else {
+ while (nb_enqueued < TEST_MAX_OP) {
+ nb_enqueued += rte_event_enqueue_burst(evdev, TEST_APP_PORT_ID,
+ &ev[nb_enqueued], TEST_MAX_OP -
+ nb_enqueued);
+ }
+ }
+
+ TEST_ASSERT_EQUAL(nb_enqueued, TEST_MAX_OP, "Failed to send event to dma adapter\n");
+
+ while (i < TEST_MAX_OP) {
+ if (rte_event_dequeue_burst(evdev, TEST_APP_PORT_ID, &recv_ev[i], 1, 0) != 1)
+ continue;
+ i++;
+ }
+
+ TEST_ASSERT_EQUAL(i, TEST_MAX_OP, "Test failed. Failed to dequeue events.\n");
+
+ return TEST_SUCCESS;
+}
+
+static int
+test_dma_adapter_stats(void)
+{
+ struct rte_event_dma_adapter_stats stats;
+
+ rte_event_dma_adapter_stats_get(TEST_ADAPTER_ID, &stats);
+ printf(" +------------------------------------------------------+\n");
+ printf(" + DMA adapter stats for instance %u:\n", TEST_ADAPTER_ID);
+ printf(" + Event port poll count 0x%" PRIx64 "\n",
+ stats.event_poll_count);
+ printf(" + Event dequeue count 0x%" PRIx64 "\n",
+ stats.event_deq_count);
+ printf(" + DMA dev enqueue count 0x%" PRIx64 "\n",
+ stats.dma_enq_count);
+ printf(" + DMA dev enqueue failed count 0x%" PRIx64 "\n",
+ stats.dma_enq_fail_count);
+ printf(" + DMA dev dequeue count 0x%" PRIx64 "\n",
+ stats.dma_deq_count);
+ printf(" + Event enqueue count 0x%" PRIx64 "\n",
+ stats.event_enq_count);
+ printf(" + Event enqueue retry count 0x%" PRIx64 "\n",
+ stats.event_enq_retry_count);
+ printf(" + Event enqueue fail count 0x%" PRIx64 "\n",
+ stats.event_enq_fail_count);
+ printf(" +------------------------------------------------------+\n");
+
+ rte_event_dma_adapter_stats_reset(TEST_ADAPTER_ID);
+ return TEST_SUCCESS;
+}
+
+static int
+test_dma_adapter_params(void)
+{
+ struct rte_event_dma_adapter_runtime_params out_params;
+ struct rte_event_dma_adapter_runtime_params in_params;
+ struct rte_event event;
+ uint32_t cap;
+ int err, rc;
+
+ err = rte_event_dma_adapter_caps_get(evdev, TEST_DMA_DEV_ID, &cap);
+ TEST_ASSERT_SUCCESS(err, "Failed to get adapter capabilities\n");
+
+ if (cap & RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_VCHAN_EV_BIND) {
+ err = rte_event_dma_adapter_vchan_add(TEST_ADAPTER_ID, TEST_DMA_DEV_ID,
+ TEST_DMA_VCHAN_ID, &event);
+ } else
+ err = rte_event_dma_adapter_vchan_add(TEST_ADAPTER_ID, TEST_DMA_DEV_ID,
+ TEST_DMA_VCHAN_ID, NULL);
+
+ TEST_ASSERT_SUCCESS(err, "Failed to add vchan\n");
+
+ err = rte_event_dma_adapter_runtime_params_init(&in_params);
+ TEST_ASSERT(err == 0, "Expected 0 got %d", err);
+ err = rte_event_dma_adapter_runtime_params_init(&out_params);
+ TEST_ASSERT(err == 0, "Expected 0 got %d", err);
+
+ /* Case 1: Get the default value of mbufs processed by adapter */
+ err = rte_event_dma_adapter_runtime_params_get(TEST_ADAPTER_ID, &out_params);
+ if (err == -ENOTSUP) {
+ rc = TEST_SKIPPED;
+ goto vchan_del;
+ }
+ TEST_ASSERT(err == 0, "Expected 0 got %d", err);
+
+ /* Case 2: Set max_nb = 32 (=BATCH_SEIZE) */
+ in_params.max_nb = 32;
+
+ err = rte_event_dma_adapter_runtime_params_set(TEST_ADAPTER_ID, &in_params);
+ TEST_ASSERT(err == 0, "Expected 0 got %d", err);
+
+ err = rte_event_dma_adapter_runtime_params_get(TEST_ADAPTER_ID, &out_params);
+ TEST_ASSERT(err == 0, "Expected 0 got %d", err);
+ TEST_ASSERT(in_params.max_nb == out_params.max_nb, "Expected %u got %u",
+ in_params.max_nb, out_params.max_nb);
+
+ /* Case 3: Set max_nb = 192 */
+ in_params.max_nb = 192;
+
+ err = rte_event_dma_adapter_runtime_params_set(TEST_ADAPTER_ID, &in_params);
+ TEST_ASSERT(err == 0, "Expected 0 got %d", err);
+
+ err = rte_event_dma_adapter_runtime_params_get(TEST_ADAPTER_ID, &out_params);
+ TEST_ASSERT(err == 0, "Expected 0 got %d", err);
+ TEST_ASSERT(in_params.max_nb == out_params.max_nb, "Expected %u got %u",
+ in_params.max_nb, out_params.max_nb);
+
+ /* Case 4: Set max_nb = 256 */
+ in_params.max_nb = 256;
+
+ err = rte_event_dma_adapter_runtime_params_set(TEST_ADAPTER_ID, &in_params);
+ TEST_ASSERT(err == 0, "Expected 0 got %d", err);
+
+ err = rte_event_dma_adapter_runtime_params_get(TEST_ADAPTER_ID, &out_params);
+ TEST_ASSERT(err == 0, "Expected 0 got %d", err);
+ TEST_ASSERT(in_params.max_nb == out_params.max_nb, "Expected %u got %u",
+ in_params.max_nb, out_params.max_nb);
+
+ /* Case 5: Set max_nb = 30(<BATCH_SIZE) */
+ in_params.max_nb = 30;
+
+ err = rte_event_dma_adapter_runtime_params_set(TEST_ADAPTER_ID, &in_params);
+ TEST_ASSERT(err == 0, "Expected 0 got %d", err);
+
+ err = rte_event_dma_adapter_runtime_params_get(TEST_ADAPTER_ID, &out_params);
+ TEST_ASSERT(err == 0, "Expected 0 got %d", err);
+ TEST_ASSERT(in_params.max_nb == out_params.max_nb, "Expected %u got %u",
+ in_params.max_nb, out_params.max_nb);
+
+ /* Case 6: Set max_nb = 512 */
+ in_params.max_nb = 512;
+
+ err = rte_event_dma_adapter_runtime_params_set(TEST_ADAPTER_ID, &in_params);
+ TEST_ASSERT(err == 0, "Expected 0 got %d", err);
+
+ err = rte_event_dma_adapter_runtime_params_get(TEST_ADAPTER_ID, &out_params);
+ TEST_ASSERT(err == 0, "Expected 0 got %d", err);
+ TEST_ASSERT(in_params.max_nb == out_params.max_nb, "Expected %u got %u",
+ in_params.max_nb, out_params.max_nb);
+
+ rc = TEST_SUCCESS;
+vchan_del:
+ err = rte_event_dma_adapter_vchan_del(TEST_ADAPTER_ID, TEST_DMA_DEV_ID,
+ TEST_DMA_VCHAN_ID);
+ TEST_ASSERT_SUCCESS(err, "Failed to delete vchan\n");
+
+ return rc;
+}
+
+static int
+test_op_forward_mode(void)
+{
+ struct rte_mbuf *src_mbuf[TEST_MAX_OP];
+ struct rte_mbuf *dst_mbuf[TEST_MAX_OP];
+ struct rte_event_dma_adapter_op *op;
+ struct rte_event ev[TEST_MAX_OP];
+ struct rte_event response_info;
+ int ret, i;
+
+ ret = rte_pktmbuf_alloc_bulk(params.src_mbuf_pool, src_mbuf, TEST_MAX_OP);
+ TEST_ASSERT_SUCCESS(ret, "alloc src mbufs failed.\n");
+
+ ret = rte_pktmbuf_alloc_bulk(params.dst_mbuf_pool, dst_mbuf, TEST_MAX_OP);
+ TEST_ASSERT_SUCCESS(ret, "alloc dst mbufs failed.\n");
+
+ for (i = 0; i < TEST_MAX_OP; i++) {
+ memset(rte_pktmbuf_mtod(src_mbuf[i], void *), rte_rand(), PACKET_LENGTH);
+ memset(rte_pktmbuf_mtod(dst_mbuf[i], void *), 0, PACKET_LENGTH);
+ }
+
+ for (i = 0; i < TEST_MAX_OP; i++) {
+ rte_mempool_get(params.op_mpool, (void **)&op);
+ TEST_ASSERT_NOT_NULL(op, "Failed to allocate dma operation struct\n");
+
+ op->src_seg = rte_malloc(NULL, sizeof(struct rte_dma_sge), 0);
+ op->dst_seg = rte_malloc(NULL, sizeof(struct rte_dma_sge), 0);
+
+ /* Update Op */
+ op->src_seg->addr = rte_pktmbuf_iova(src_mbuf[i]);
+ op->dst_seg->addr = rte_pktmbuf_iova(dst_mbuf[i]);
+ op->src_seg->length = PACKET_LENGTH;
+ op->dst_seg->length = PACKET_LENGTH;
+ op->nb_src = 1;
+ op->nb_dst = 1;
+ op->flags = RTE_DMA_OP_FLAG_SUBMIT;
+ op->op_mp = params.op_mpool;
+ op->dma_dev_id = TEST_DMA_DEV_ID;
+ op->vchan = TEST_DMA_VCHAN_ID;
+
+ response_info.event = dma_response_info.event;
+ rte_memcpy((uint8_t *)op + sizeof(struct rte_event_dma_adapter_op), &response_info,
+ sizeof(struct rte_event));
+
+ /* Fill in event info and update event_ptr with rte_event_dma_adapter_op */
+ memset(&ev[i], 0, sizeof(struct rte_event));
+ ev[i].event = 0;
+ ev[i].event_type = RTE_EVENT_TYPE_DMADEV;
+ ev[i].queue_id = TEST_DMA_EV_QUEUE_ID;
+ ev[i].sched_type = RTE_SCHED_TYPE_ATOMIC;
+ ev[i].flow_id = 0xAABB;
+ ev[i].event_ptr = op;
+ }
+
+ ret = send_recv_ev(ev);
+ TEST_ASSERT_SUCCESS(ret, "Failed to send/receive event to dma adapter\n");
+
+ test_dma_adapter_stats();
+
+ for (i = 0; i < TEST_MAX_OP; i++) {
+ op = ev[i].event_ptr;
+ ret = memcmp(rte_pktmbuf_mtod(src_mbuf[i], void *),
+ rte_pktmbuf_mtod(dst_mbuf[i], void *), PACKET_LENGTH);
+
+ TEST_ASSERT_EQUAL(ret, 0, "Data mismatch for dma adapter\n");
+
+ rte_free(op->src_seg);
+ rte_free(op->dst_seg);
+ rte_mempool_put(op->op_mp, op);
+ }
+
+ rte_pktmbuf_free_bulk(src_mbuf, TEST_MAX_OP);
+ rte_pktmbuf_free_bulk(dst_mbuf, TEST_MAX_OP);
+
+ return TEST_SUCCESS;
+}
+
+static int
+map_adapter_service_core(void)
+{
+ uint32_t adapter_service_id;
+ int ret;
+
+ if (rte_event_dma_adapter_service_id_get(TEST_ADAPTER_ID, &adapter_service_id) == 0) {
+ uint32_t core_list[NUM_CORES];
+
+ ret = rte_service_lcore_list(core_list, NUM_CORES);
+ TEST_ASSERT(ret >= 0, "Failed to get service core list!");
+
+ if (core_list[0] != slcore_id) {
+ TEST_ASSERT_SUCCESS(rte_service_lcore_add(slcore_id),
+ "Failed to add service core");
+ TEST_ASSERT_SUCCESS(rte_service_lcore_start(slcore_id),
+ "Failed to start service core");
+ }
+
+ TEST_ASSERT_SUCCESS(rte_service_map_lcore_set(
+ adapter_service_id, slcore_id, 1),
+ "Failed to map adapter service");
+ }
+
+ return TEST_SUCCESS;
+}
+
+static int
+test_with_op_forward_mode(void)
+{
+ uint32_t cap;
+ int ret;
+
+ ret = rte_event_dma_adapter_caps_get(evdev, TEST_DMA_DEV_ID, &cap);
+ TEST_ASSERT_SUCCESS(ret, "Failed to get adapter capabilities\n");
+
+ if (!(cap & RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP_FWD) &&
+ !(cap & RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP_NEW))
+ map_adapter_service_core();
+ else {
+ if (!(cap & RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP_FWD))
+ return TEST_SKIPPED;
+ }
+
+ TEST_ASSERT_SUCCESS(rte_event_dma_adapter_start(TEST_ADAPTER_ID),
+ "Failed to start event dma adapter");
+
+ ret = test_op_forward_mode();
+ TEST_ASSERT_SUCCESS(ret, "DMA - FORWARD mode test failed\n");
+ return TEST_SUCCESS;
+}
+
+static int
+configure_dmadev(void)
+{
+ const struct rte_dma_conf conf = { .nb_vchans = 1};
+ const struct rte_dma_vchan_conf qconf = {
+ .direction = RTE_DMA_DIR_MEM_TO_MEM,
+ .nb_desc = TEST_RINGSIZE,
+ };
+ struct rte_dma_info info;
+ unsigned int elt_size;
+ int ret;
+
+ ret = rte_dma_count_avail();
+ RTE_TEST_ASSERT_FAIL(ret, "No dma devices found!\n");
+
+ ret = rte_dma_info_get(TEST_DMA_DEV_ID, &info);
+ TEST_ASSERT_SUCCESS(ret, "Error with rte_dma_info_get()\n");
+
+ if (info.max_vchans < 1)
+ RTE_LOG(ERR, USER1, "Error, no channels available on device id %u\n",
+ TEST_DMA_DEV_ID);
+
+ if (rte_dma_configure(TEST_DMA_DEV_ID, &conf) != 0)
+ RTE_LOG(ERR, USER1, "Error with rte_dma_configure()\n");
+
+ if (rte_dma_vchan_setup(TEST_DMA_DEV_ID, TEST_DMA_VCHAN_ID, &qconf) < 0)
+ RTE_LOG(ERR, USER1, "Error with vchan configuration\n");
+
+ ret = rte_dma_info_get(TEST_DMA_DEV_ID, &info);
+ if (ret != 0 || info.nb_vchans != 1)
+ RTE_LOG(ERR, USER1, "Error, no configured vhcan reported on device id %u\n",
+ TEST_DMA_DEV_ID);
+
+ params.src_mbuf_pool = rte_pktmbuf_pool_create("DMA_ADAPTER_SRC_MBUFPOOL", NUM_MBUFS,
+ MBUF_CACHE_SIZE, 0, MBUF_SIZE,
+ rte_socket_id());
+ RTE_TEST_ASSERT_NOT_NULL(params.src_mbuf_pool, "Can't create DMA_SRC_MBUFPOOL\n");
+
+ params.dst_mbuf_pool = rte_pktmbuf_pool_create("DMA_ADAPTER_DST_MBUFPOOL", NUM_MBUFS,
+ MBUF_CACHE_SIZE, 0, MBUF_SIZE,
+ rte_socket_id());
+ RTE_TEST_ASSERT_NOT_NULL(params.dst_mbuf_pool, "Can't create DMA_DST_MBUFPOOL\n");
+
+ elt_size = sizeof(struct rte_event_dma_adapter_op) + sizeof(struct rte_event);
+ params.op_mpool = rte_mempool_create("EVENT_DMA_OP_POOL", DMA_OP_POOL_SIZE, elt_size, 0,
+ 0, NULL, NULL, NULL, NULL, rte_socket_id(), 0);
+ RTE_TEST_ASSERT_NOT_NULL(params.op_mpool, "Can't create DMA_OP_POOL\n");
+
+ return TEST_SUCCESS;
+}
+
+static inline void
+evdev_set_conf_values(struct rte_event_dev_config *dev_conf, struct rte_event_dev_info *info)
+{
+ memset(dev_conf, 0, sizeof(struct rte_event_dev_config));
+ dev_conf->dequeue_timeout_ns = info->min_dequeue_timeout_ns;
+ dev_conf->nb_event_ports = NB_TEST_PORTS;
+ dev_conf->nb_event_queues = NB_TEST_QUEUES;
+ dev_conf->nb_event_queue_flows = info->max_event_queue_flows;
+ dev_conf->nb_event_port_dequeue_depth =
+ info->max_event_port_dequeue_depth;
+ dev_conf->nb_event_port_enqueue_depth =
+ info->max_event_port_enqueue_depth;
+ dev_conf->nb_event_port_enqueue_depth =
+ info->max_event_port_enqueue_depth;
+ dev_conf->nb_events_limit =
+ info->max_num_events;
+}
+
+static int
+configure_eventdev(void)
+{
+ struct rte_event_queue_conf queue_conf;
+ struct rte_event_dev_config devconf;
+ struct rte_event_dev_info info;
+ uint32_t queue_count;
+ uint32_t port_count;
+ uint8_t qid;
+ int ret;
+
+ if (!rte_event_dev_count()) {
+ /* If there is no hardware eventdev, or no software vdev was
+ * specified on the command line, create an instance of
+ * event_sw.
+ */
+ LOG_DBG("Failed to find a valid event device... "
+ "testing with event_sw device\n");
+ TEST_ASSERT_SUCCESS(rte_vdev_init("event_sw0", NULL),
+ "Error creating eventdev");
+ evdev = rte_event_dev_get_dev_id("event_sw0");
+ }
+
+ ret = rte_event_dev_info_get(evdev, &info);
+ TEST_ASSERT_SUCCESS(ret, "Failed to get event dev info\n");
+
+ evdev_set_conf_values(&devconf, &info);
+
+ ret = rte_event_dev_configure(evdev, &devconf);
+ TEST_ASSERT_SUCCESS(ret, "Failed to configure eventdev\n");
+
+ /* Set up event queue */
+ ret = rte_event_dev_attr_get(evdev, RTE_EVENT_DEV_ATTR_QUEUE_COUNT, &queue_count);
+ TEST_ASSERT_SUCCESS(ret, "Queue count get failed\n");
+ TEST_ASSERT_EQUAL(queue_count, 2, "Unexpected queue count\n");
+
+ qid = TEST_APP_EV_QUEUE_ID;
+ ret = rte_event_queue_setup(evdev, qid, NULL);
+ TEST_ASSERT_SUCCESS(ret, "Failed to setup queue=%d\n", qid);
+
+ queue_conf.nb_atomic_flows = info.max_event_queue_flows;
+ queue_conf.nb_atomic_order_sequences = 32;
+ queue_conf.schedule_type = RTE_SCHED_TYPE_ATOMIC;
+ queue_conf.priority = RTE_EVENT_DEV_PRIORITY_HIGHEST;
+ queue_conf.event_queue_cfg = RTE_EVENT_QUEUE_CFG_SINGLE_LINK;
+
+ qid = TEST_DMA_EV_QUEUE_ID;
+ ret = rte_event_queue_setup(evdev, qid, &queue_conf);
+ TEST_ASSERT_SUCCESS(ret, "Failed to setup queue=%u\n", qid);
+
+ /* Set up event port */
+ ret = rte_event_dev_attr_get(evdev, RTE_EVENT_DEV_ATTR_PORT_COUNT,
+ &port_count);
+ TEST_ASSERT_SUCCESS(ret, "Port count get failed\n");
+ TEST_ASSERT_EQUAL(port_count, 1, "Unexpected port count\n");
+
+ ret = rte_event_port_setup(evdev, TEST_APP_PORT_ID, NULL);
+ TEST_ASSERT_SUCCESS(ret, "Failed to setup port=%d\n",
+ TEST_APP_PORT_ID);
+
+ qid = TEST_APP_EV_QUEUE_ID;
+ ret = rte_event_port_link(evdev, TEST_APP_PORT_ID, &qid, NULL, 1);
+ TEST_ASSERT(ret >= 0, "Failed to link queue port=%d\n",
+ TEST_APP_PORT_ID);
+
+ return TEST_SUCCESS;
+}
+
+static void
+test_dma_adapter_free(void)
+{
+ rte_event_dma_adapter_free(TEST_ADAPTER_ID);
+}
+
+static int
+test_dma_adapter_create(void)
+{
+ struct rte_event_dev_info evdev_info = {0};
+ struct rte_event_port_conf conf = {0};
+ int ret;
+
+ ret = rte_event_dev_info_get(evdev, &evdev_info);
+ TEST_ASSERT_SUCCESS(ret, "Failed to create event dma adapter\n");
+
+ conf.new_event_threshold = evdev_info.max_num_events;
+ conf.dequeue_depth = evdev_info.max_event_port_dequeue_depth;
+ conf.enqueue_depth = evdev_info.max_event_port_enqueue_depth;
+
+ /* Create adapter with default port creation callback */
+ ret = rte_event_dma_adapter_create(TEST_ADAPTER_ID, evdev, &conf, 0);
+ TEST_ASSERT_SUCCESS(ret, "Failed to create event dma adapter\n");
+
+ return TEST_SUCCESS;
+}
+
+static int
+test_dma_adapter_vchan_add_del(void)
+{
+ struct rte_event event;
+ uint32_t cap;
+ int ret;
+
+ ret = rte_event_dma_adapter_caps_get(evdev, TEST_DMA_DEV_ID, &cap);
+ TEST_ASSERT_SUCCESS(ret, "Failed to get adapter capabilities\n");
+
+ if (cap & RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_VCHAN_EV_BIND) {
+ ret = rte_event_dma_adapter_vchan_add(TEST_ADAPTER_ID, TEST_DMA_DEV_ID,
+ TEST_DMA_VCHAN_ID, &event);
+ } else
+ ret = rte_event_dma_adapter_vchan_add(TEST_ADAPTER_ID, TEST_DMA_DEV_ID,
+ TEST_DMA_VCHAN_ID, NULL);
+
+ TEST_ASSERT_SUCCESS(ret, "Failed to create add vchan\n");
+
+ ret = rte_event_dma_adapter_vchan_del(TEST_ADAPTER_ID, TEST_DMA_DEV_ID,
+ TEST_DMA_VCHAN_ID);
+ TEST_ASSERT_SUCCESS(ret, "Failed to delete vchan\n");
+
+ return TEST_SUCCESS;
+}
+
+static int
+configure_event_dma_adapter(enum rte_event_dma_adapter_mode mode)
+{
+ struct rte_event_dev_info evdev_info = {0};
+ struct rte_event_port_conf conf = {0};
+ struct rte_event event;
+ uint32_t cap;
+ int ret;
+
+ ret = rte_event_dma_adapter_caps_get(evdev, TEST_DMA_DEV_ID, &cap);
+ TEST_ASSERT_SUCCESS(ret, "Failed to get adapter capabilities\n");
+
+ /* Skip mode and capability mismatch check for SW eventdev */
+ if (!(cap & RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP_NEW) &&
+ !(cap & RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP_FWD) &&
+ !(cap & RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_VCHAN_EV_BIND))
+ goto adapter_create;
+
+ if (mode == RTE_EVENT_DMA_ADAPTER_OP_FORWARD) {
+ if (cap & RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP_FWD)
+ params.internal_port_op_fwd = 1;
+ else
+ return -ENOTSUP;
+ }
+
+adapter_create:
+ ret = rte_event_dev_info_get(evdev, &evdev_info);
+ TEST_ASSERT_SUCCESS(ret, "Failed to create event dma adapter\n");
+
+ conf.new_event_threshold = evdev_info.max_num_events;
+ conf.dequeue_depth = evdev_info.max_event_port_dequeue_depth;
+ conf.enqueue_depth = evdev_info.max_event_port_enqueue_depth;
+
+ /* Create adapter with default port creation callback */
+ ret = rte_event_dma_adapter_create(TEST_ADAPTER_ID, evdev, &conf, mode);
+ TEST_ASSERT_SUCCESS(ret, "Failed to create event dma adapter\n");
+
+ if (cap & RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_VCHAN_EV_BIND) {
+ ret = rte_event_dma_adapter_vchan_add(TEST_ADAPTER_ID, TEST_DMA_DEV_ID,
+ TEST_DMA_VCHAN_ID, &event);
+ } else
+ ret = rte_event_dma_adapter_vchan_add(TEST_ADAPTER_ID, TEST_DMA_DEV_ID,
+ TEST_DMA_VCHAN_ID, NULL);
+
+ TEST_ASSERT_SUCCESS(ret, "Failed to add vchan\n");
+
+ if (!params.internal_port_op_fwd) {
+ ret = rte_event_dma_adapter_event_port_get(TEST_ADAPTER_ID,
+ ¶ms.dma_event_port_id);
+ TEST_ASSERT_SUCCESS(ret, "Failed to get event port\n");
+ }
+
+ return TEST_SUCCESS;
+}
+
+static void
+test_dma_adapter_stop(void)
+{
+ uint32_t evdev_service_id, adapter_service_id;
+
+ /* retrieve service ids & stop services */
+ if (rte_event_dma_adapter_service_id_get(TEST_ADAPTER_ID,
+ &adapter_service_id) == 0) {
+ rte_service_runstate_set(adapter_service_id, 0);
+ rte_service_lcore_stop(slcore_id);
+ rte_service_lcore_del(slcore_id);
+ rte_event_dma_adapter_stop(TEST_ADAPTER_ID);
+ }
+
+ if (rte_event_dev_service_id_get(evdev, &evdev_service_id) == 0) {
+ rte_service_runstate_set(evdev_service_id, 0);
+ rte_service_lcore_stop(slcore_id);
+ rte_service_lcore_del(slcore_id);
+ rte_dma_stop(TEST_DMA_DEV_ID);
+ rte_event_dev_stop(evdev);
+ } else {
+ rte_dma_stop(TEST_DMA_DEV_ID);
+ rte_event_dev_stop(evdev);
+ }
+}
+
+static int
+test_dma_adapter_conf(enum rte_event_dma_adapter_mode mode)
+{
+ uint32_t evdev_service_id;
+ uint8_t qid;
+ int ret;
+
+ if (!dma_adapter_setup_done) {
+ ret = configure_event_dma_adapter(mode);
+ if (ret)
+ return ret;
+ if (!params.internal_port_op_fwd) {
+ qid = TEST_DMA_EV_QUEUE_ID;
+ ret = rte_event_port_link(evdev,
+ params.dma_event_port_id, &qid, NULL, 1);
+ TEST_ASSERT(ret >= 0, "Failed to link queue %d "
+ "port=%u\n", qid,
+ params.dma_event_port_id);
+ }
+ dma_adapter_setup_done = 1;
+ }
+
+ /* retrieve service ids */
+ if (rte_event_dev_service_id_get(evdev, &evdev_service_id) == 0) {
+ /* add a service core and start it */
+ TEST_ASSERT_SUCCESS(rte_service_lcore_add(slcore_id),
+ "Failed to add service core");
+ TEST_ASSERT_SUCCESS(rte_service_lcore_start(slcore_id),
+ "Failed to start service core");
+
+ /* map services to it */
+ TEST_ASSERT_SUCCESS(rte_service_map_lcore_set(evdev_service_id,
+ slcore_id, 1), "Failed to map evdev service");
+
+ /* set services to running */
+ TEST_ASSERT_SUCCESS(rte_service_runstate_set(evdev_service_id,
+ 1), "Failed to start evdev service");
+ }
+
+ /* start the eventdev */
+ TEST_ASSERT_SUCCESS(rte_event_dev_start(evdev),
+ "Failed to start event device");
+
+ /* start the dma dev */
+ TEST_ASSERT_SUCCESS(rte_dma_start(TEST_DMA_DEV_ID),
+ "Failed to start dma device");
+
+ return TEST_SUCCESS;
+}
+
+static int
+test_dma_adapter_conf_op_forward_mode(void)
+{
+ enum rte_event_dma_adapter_mode mode;
+
+ mode = RTE_EVENT_DMA_ADAPTER_OP_FORWARD;
+
+ return test_dma_adapter_conf(mode);
+}
+
+static int
+testsuite_setup(void)
+{
+ int ret;
+
+ slcore_id = rte_get_next_lcore(-1, 1, 0);
+ TEST_ASSERT_NOT_EQUAL(slcore_id, RTE_MAX_LCORE, "At least 2 lcores "
+ "are required to run this autotest\n");
+
+ /* Setup and start event device. */
+ ret = configure_eventdev();
+ TEST_ASSERT_SUCCESS(ret, "Failed to setup eventdev\n");
+
+ /* Setup and start dma device. */
+ ret = configure_dmadev();
+ TEST_ASSERT_SUCCESS(ret, "dmadev initialization failed\n");
+
+ return TEST_SUCCESS;
+}
+
+static void
+dma_adapter_teardown(void)
+{
+ int ret;
+
+ ret = rte_event_dma_adapter_stop(TEST_ADAPTER_ID);
+ if (ret < 0)
+ RTE_LOG(ERR, USER1, "Failed to stop adapter!");
+
+ ret = rte_event_dma_adapter_vchan_del(TEST_ADAPTER_ID, TEST_DMA_DEV_ID,
+ TEST_DMA_VCHAN_ID);
+ if (ret < 0)
+ RTE_LOG(ERR, USER1, "Failed to delete vchan!");
+
+ ret = rte_event_dma_adapter_free(TEST_ADAPTER_ID);
+ if (ret < 0)
+ RTE_LOG(ERR, USER1, "Failed to free adapter!");
+
+ dma_adapter_setup_done = 0;
+}
+
+static void
+dma_teardown(void)
+{
+ /* Free mbuf mempool */
+ if (params.src_mbuf_pool != NULL) {
+ RTE_LOG(DEBUG, USER1, "DMA_ADAPTER_SRC_MBUFPOOL count %u\n",
+ rte_mempool_avail_count(params.src_mbuf_pool));
+ rte_mempool_free(params.src_mbuf_pool);
+ params.src_mbuf_pool = NULL;
+ }
+
+ if (params.dst_mbuf_pool != NULL) {
+ RTE_LOG(DEBUG, USER1, "DMA_ADAPTER_DST_MBUFPOOL count %u\n",
+ rte_mempool_avail_count(params.dst_mbuf_pool));
+ rte_mempool_free(params.dst_mbuf_pool);
+ params.dst_mbuf_pool = NULL;
+ }
+
+ /* Free ops mempool */
+ if (params.op_mpool != NULL) {
+ RTE_LOG(DEBUG, USER1, "EVENT_DMA_OP_POOL count %u\n",
+ rte_mempool_avail_count(params.op_mpool));
+ rte_mempool_free(params.op_mpool);
+ params.op_mpool = NULL;
+ }
+}
+
+static void
+eventdev_teardown(void)
+{
+ rte_event_dev_stop(evdev);
+}
+
+static void
+testsuite_teardown(void)
+{
+ dma_adapter_teardown();
+ dma_teardown();
+ eventdev_teardown();
+}
+
+static struct unit_test_suite functional_testsuite = {
+ .suite_name = "Event dma adapter test suite",
+ .setup = testsuite_setup,
+ .teardown = testsuite_teardown,
+ .unit_test_cases = {
+
+ TEST_CASE_ST(NULL, test_dma_adapter_free, test_dma_adapter_create),
+
+ TEST_CASE_ST(test_dma_adapter_create, test_dma_adapter_free,
+ test_dma_adapter_vchan_add_del),
+
+ TEST_CASE_ST(test_dma_adapter_create, test_dma_adapter_free,
+ test_dma_adapter_stats),
+
+ TEST_CASE_ST(test_dma_adapter_create, test_dma_adapter_free,
+ test_dma_adapter_params),
+
+ TEST_CASE_ST(test_dma_adapter_conf_op_forward_mode, test_dma_adapter_stop,
+ test_with_op_forward_mode),
+
+ TEST_CASES_END() /**< NULL terminate unit test array */
+ }
+};
+
+static int
+test_event_dma_adapter(void)
+{
+ return unit_test_suite_runner(&functional_testsuite);
+}
+
+#endif /* !RTE_EXEC_ENV_WINDOWS */
+
+REGISTER_TEST_COMMAND(event_dma_adapter_autotest, test_event_dma_adapter);
--
2.25.1
^ permalink raw reply [flat|nested] 110+ messages in thread
* Re: [PATCH v4 01/12] eventdev: introduce event DMA adapter library
2023-09-26 10:32 ` [PATCH v4 01/12] eventdev: introduce event DMA adapter library Amit Prakash Shukla
@ 2023-09-27 18:12 ` Jerin Jacob
2023-09-27 20:45 ` Thomas Monjalon
0 siblings, 1 reply; 110+ messages in thread
From: Jerin Jacob @ 2023-09-27 18:12 UTC (permalink / raw)
To: Amit Prakash Shukla
Cc: Thomas Monjalon, Jerin Jacob, dev, fengchengwen, kevin.laatz,
bruce.richardson, conor.walsh, vattunuru, g.singh, sachin.saxena,
hemant.agrawal, cheng1.jiang, ndabilpuram, anoobj, mb
On Tue, Sep 26, 2023 at 4:03 PM Amit Prakash Shukla
<amitprakashs@marvell.com> wrote:
>
> Introduce event DMA adapter APIs. The change provides information
> on adapter modes and usage. Application can use this event adapter
> interface to transfer packets between DMA device and event device.
With below changes,
Acked-by: Jerin Jacob <jerinj@marvell.com>
Use ethdev/dma for all patches like other adapters.
Rewrite suggestion:
eventdev/dma: introduce event dma adapter
Introduce event dma adapter interface to transfer packets between
dma device and event device.
>
> Signed-off-by: Amit Prakash Shukla <amitprakashs@marvell.com>
> ---
>
> diff --git a/MAINTAINERS b/MAINTAINERS
> index a926155f26..a6b8fc88d0 100644
> --- a/MAINTAINERS
> +++ b/MAINTAINERS
> @@ -548,6 +548,11 @@ F: drivers/raw/skeleton/
> F: app/test/test_rawdev.c
> F: doc/guides/prog_guide/rawdev.rst
>
> +Eventdev DMA Adapter API
> +M: Amit Prakash Shukla <amitprakashs@marvell.com>
> +T: git://dpdk.org/next/dpdk-next-eventdev
> +F: lib/eventdev/*dma_adapter*
> +F: doc/guides/prog_guide/event_dma_adapter.rst
>
Move after "Eventdev Crypto Adapter API" to group all adapter frameworks.
> diff --git a/doc/guides/prog_guide/event_dma_adapter.rst b/doc/guides/prog_guide/event_dma_adapter.rst
> new file mode 100644
> index 0000000000..eeb9ce6dfd
> --- /dev/null
> +++ b/doc/guides/prog_guide/event_dma_adapter.rst
> @@ -0,0 +1,264 @@
> +.. SPDX-License-Identifier: BSD-3-Clause
> + Copyright (c) 2023 Marvell.
> +
> +Event DMA Adapter Library
> +=========================
> +
> +DPDK :doc:`Eventdev library <eventdev>` provides event driven programming model with features
> +to schedule events. :doc:`DMA Device library <dmadev>` provides an interface to DMA poll mode
> +drivers that support DMA operations. Event DMA Adapter is intended to bridge between the event
> +device and the DMA device.
> +
> +Packet flow from DMA device to the event device can be accomplished using software and hardware
> +based transfer mechanisms. The adapter queries an eventdev PMD to determine which mechanism to
> +be used. The adapter uses an EAL service core function for software based packet transfer and
> +uses the eventdev PMD functions to configure hardware based packet transfer between DMA device
> +and the event device. DMA adapter uses a new event type called ``RTE_EVENT_TYPE_DMADEV`` to
> +indicate the source of event.
> +
> +Application can choose to submit an DMA operation directly to an DMA device or send it to an DMA
> +adapter via eventdev based on RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP_FWD capability. The
Change as ``RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP_FWD``
> +first mode is known as the event new (RTE_EVENT_DMA_ADAPTER_OP_NEW) mode and the second as the
Change as ``RTE_EVENT_DMA_ADAPTER_OP_NEW``
> +event forward (RTE_EVENT_DMA_ADAPTER_OP_FORWARD) mode. Choice of mode can be specified while
Change as ``RTE_EVENT_DMA_ADAPTER_OP_FORWARD``
> +creating the adapter. In the former mode, it is the application's responsibility to enable
> +ingress packet ordering. In the latter mode, it is the adapter's responsibility to enable
> +ingress packet ordering.
> +
> +
> +Adapter Modes
> +-------------
> +
> +RTE_EVENT_DMA_ADAPTER_OP_NEW mode
> +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> +
> +In the RTE_EVENT_DMA_ADAPTER_OP_NEW mode, application submits DMA operations directly to an DMA
Change as ``RTE_EVENT_DMA_ADAPTER_OP_NEW``
> +device. The adapter then dequeues DMA completions from the DMA device and enqueues them as events
> +to the event device. This mode does not ensure ingress ordering as the application directly
> +enqueues to the dmadev without going through DMA/atomic stage. In this mode, events dequeued
> +from the adapter are treated as new events. The application has to specify event information
> +(response information) which is needed to enqueue an event after the DMA operation is completed.
> +
> +.. _figure_event_dma_adapter_op_new:
> +
> +.. figure:: img/event_dma_adapter_op_new.*
> +
> + Working model of ``RTE_EVENT_DMA_ADAPTER_OP_NEW`` mode
> +
> +
> +RTE_EVENT_DMA_ADAPTER_OP_FORWARD mode
> +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> +
> +In the ``RTE_EVENT_DMA_ADAPTER_OP_FORWARD`` mode, if the event PMD and DMA PMD supports internal
> +event port (``RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP_FWD``), the application should use
> +``rte_event_dma_adapter_enqueue()`` API to enqueue DMA operations as events to DMA adapter. If
> +not, application retrieves DMA adapter's event port using ``rte_event_dma_adapter_event_port_get()``
> +API, links its event queue to this port and starts enqueuing DMA operations as events to eventdev
> +using ``rte_event_enqueue_burst()``. The adapter then dequeues the events and submits the DMA
> +operations to the dmadev. After the DMA operation is complete, the adapter enqueues events to the
> +event device.
> +
> +Applications can use this mode when ingress packet ordering is needed. In this mode, events
> +dequeued from the adapter will be treated as forwarded events. Application has to specify event
> +information (response information) needed to enqueue the event after the DMA operation has
> +completed.
> +
> +.. _figure_event_dma_adapter_op_forward:
> +
> +.. figure:: img/event_dma_adapter_op_forward.*
> +
> + Working model of ``RTE_EVENT_DMA_ADAPTER_OP_FORWARD`` mode
> +
> +
> +API Overview
> +------------
> +
> +This section has a brief introduction to the event DMA adapter APIs. The application is expected
> +to create an adapter which is associated with a single eventdev, then add dmadev and vchan to the
> +adapter instance.
> +
> +
> +Create an adapter instance
> +~~~~~~~~~~~~~~~~~~~~~~~~~~
> +
> +An adapter instance is created using ``rte_event_dma_adapter_create()``. This function is called
> +with event device to be associated with the adapter and port configuration for the adapter to
> +setup an event port (if the adapter needs to use a service function).
> +
> +Adapter can be started in ``RTE_EVENT_DMA_ADAPTER_OP_NEW`` or ``RTE_EVENT_DMA_ADAPTER_OP_FORWARD``
> +mode.
> +
> +.. code-block:: c
> +
> + enum rte_event_dma_adapter_mode mode;
> + struct rte_event_dev_info dev_info;
> + struct rte_event_port_conf conf;
> + uint8_t evdev_id;
> + uint8_t dma_id;
> + int ret;
> +
> + ret = rte_event_dev_info_get(dma_id, &dev_info);
> +
> + conf.new_event_threshold = dev_info.max_num_events;
> + conf.dequeue_depth = dev_info.max_event_port_dequeue_depth;
> + conf.enqueue_depth = dev_info.max_event_port_enqueue_depth;
> + mode = RTE_EVENT_DMA_ADAPTER_OP_FORWARD;
> + ret = rte_event_dma_adapter_create(dma_id, evdev_id, &conf, mode);
> +
> +
> +``rte_event_dma_adapter_create_ext()`` function can be used by the application to have a finer
> +control on eventdev port allocation and setup. The ``rte_event_dma_adapter_create_ext()``
> +function is passed a callback function. The callback function is invoked if the adapter creates
> +a service function and uses an event port for it. The callback is expected to fill the
> +``struct rte_event_dma_adapter_conf`` structure passed to it.
> +
> +In the ``RTE_EVENT_DMA_ADAPTER_OP_FORWARD`` mode, if the event PMD and DMA PMD supports internal
> +event port (``RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP_FWD``), events with DMA operations should
> +be enqueued to the DMA adapter using ``rte_event_dma_adapter_enqueue()`` API. If not, the event port
> +created by the adapter can be retrieved using ``rte_event_dma_adapter_event_port_get()`` API. An
> +application can use this event port to link with an event queue, on which it enqueues events
> +towards the DMA adapter using ``rte_event_enqueue_burst()``.
> +
> +.. code-block:: c
> +
> + uint8_t dma_adpt_id, evdev_id, dma_dev_id, dma_ev_port_id, app_qid;
> + struct rte_event ev;
> + uint32_t cap;
> + int ret;
> +
> + // Fill in event info and update event_ptr with rte_dma_op
> + memset(&ev, 0, sizeof(ev));
> + .
> + .
> + ev.event_ptr = op;
> +
> + ret = rte_event_dma_adapter_caps_get(evdev_id, dma_dev_id, &cap);
> + if (cap & RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP_FWD) {
> + ret = rte_event_dma_adapter_enqueue(evdev_id, app_ev_port_id, ev, nb_events);
> + } else {
> + ret = rte_event_dma_adapter_event_port_get(dma_adpt_id, &dma_ev_port_id);
> + ret = rte_event_queue_setup(evdev_id, app_qid, NULL);
> + ret = rte_event_port_link(evdev_id, dma_ev_port_id, &app_qid, NULL, 1);
> + ev.queue_id = app_qid;
> + ret = rte_event_enqueue_burst(evdev_id, app_ev_port_id, ev, nb_events);
> + }
> +
> +
> +Event device configuration for service based adapter
> +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> +
> +When ``rte_event_dma_adapter_create()`` is used for creating adapter instance,
> +``rte_event_dev_config::nb_event_ports`` is automatically incremented, and event device is
> +reconfigured with additional event port during service initialization. This event device
> +reconfigure logic also increments the ``rte_event_dev_config::nb_single_link_event_port_queues``
> +parameter if the adapter event port config is of type ``RTE_EVENT_PORT_CFG_SINGLE_LINK``.
> +
> +Applications using this mode of adapter creation need not configure the event device with
> +``rte_event_dev_config::nb_event_ports`` and
> +``rte_event_dev_config::nb_single_link_event_port_queues`` parameters required for DMA adapter when
> +the adapter is created using the above-mentioned API.
> +
> +
> +Querying adapter capabilities
> +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> +
> +The ``rte_event_dma_adapter_caps_get()`` function allows the application to query the adapter
> +capabilities for an eventdev and dmadev combination. This API provides whether dmadev and eventdev
> +are connected using internal HW port or not.
> +
> +.. code-block:: c
> +
> + rte_event_dma_adapter_caps_get(dev_id, dma_dev_id, &cap);
> +
> +
> +Adding vchan to the adapter instance
> +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> +
> +dmadev device id and vchan are configured using dmadev APIs. For more information
> +see :doc:`here <dmadev>`.
> +
> +.. code-block:: c
> +
> + struct rte_dma_vchan_conf vchan_conf;
> + struct rte_dma_conf dev_conf;
> + uint8_t dev_id = 0;
> + uint16_t vchan = 0;
> +
> + rte_dma_configure(dev_id, &dev_conf);
> + rte_dma_vchan_setup(dev_id, vhcan, &vchan_conf);
vhcan -> vchan
> +
> +These dmadev id and vchan are added to the instance using the
> +``rte_event_dma_adapter_vchan_add()`` API. The same is removed using
> +``rte_event_dma_adapter_vchan_del()`` API. If hardware supports
> +``RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_VCHAN_EV_BIND`` capability, event information must be passed to the add API.
> +
> +.. code-block:: c
> +
> + uint32_t cap;
> + int ret;
> +
> + ret = rte_event_dma_adapter_caps_get(evdev_id, dma_dev_id, &cap);
> + if (cap & RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_VCHAN_EV_BIND) {
> + struct rte_event event;
> +
> + rte_event_dma_adapter_vchan_add(id, dma_dev_id, vchan, &conf);
> + } else
> + rte_event_dma_adapter_vchan_add(id, dma_dev_id, vchan, NULL);
> +
> +
> +Configuring service function
> +~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> +
> +If the adapter uses a service function, the application is required to assign a service core to
> +the service function as show below.
> +
> +.. code-block:: c
> +
> + uint32_t service_id;
> +
> + if (rte_event_dma_adapter_service_id_get(dma_id, &service_id) == 0)
> + rte_service_map_lcore_set(service_id, CORE_ID);
> +
> +
> +Set event response information
> +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> +
> +In the RTE_EVENT_DMA_ADAPTER_OP_FORWARD / RTE_EVENT_DMA_ADAPTER_OP_NEW mode, the application
> +specifies the dmadev ID and vchan ID in ``struct rte_event_dma_adapter_op`` and the event
> +information (response information) needed to enqueue an event after the DMA operation has
> +completed. The response information is specified in ``struct rte_event`` and appended to the
> +``struct rte_event_dma_adapter_op``.
> +
> +
> +Start the adapter instance
> +~~~~~~~~~~~~~~~~~~~~~~~~~~
> +
> +The application calls ``rte_event_dma_adapter_start()`` to start the adapter. This function calls
> +the start callbacks of the eventdev PMDs for hardware based eventdev-dmadev connections and
> +``rte_service_run_state_set()`` to enable the service function if one exists.
> +
> +.. code-block:: c
> +
> + rte_event_dma_adapter_start(id);
> +
> +.. Note::
> +
> + The eventdev to which the event_dma_adapter is connected should be started before calling
> + rte_event_dma_adapter_start().
> +
> +
> +Get adapter statistics
> +~~~~~~~~~~~~~~~~~~~~~~
> +
> +The ``rte_event_dma_adapter_stats_get()`` function reports counters defined in struct
> +``rte_event_dma_adapter_stats``. The received packet and enqueued event counts are a sum of the
> +counts from the eventdev PMD callbacks if the callback is supported, and the counts maintained by
> +the service function, if one exists.
> +
> +Set/Get adapter runtime configuration parameters
> +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> +
> +The runtime configuration parameters of adapter can be set/get using
> +``rte_event_dma_adapter_runtime_params_set()`` and
> +``rte_event_dma_adapter_runtime_params_get()`` respectively.
> +The parameters that can be set/get are defined in
> +``struct rte_event_dma_adapter_runtime_params``.
> diff --git a/doc/guides/prog_guide/eventdev.rst b/doc/guides/prog_guide/eventdev.rst
> index 2c83176846..ff55115d0d 100644
> --- a/doc/guides/prog_guide/eventdev.rst
> +++ b/doc/guides/prog_guide/eventdev.rst
> @@ -333,7 +333,8 @@ eventdev.
> .. Note::
>
> EventDev needs to be started before starting the event producers such
> - as event_eth_rx_adapter, event_timer_adapter and event_crypto_adapter.
> + as event_eth_rx_adapter, event_timer_adapter, event_crypto_adapter and
> + event_dma_adapter.
>
> Ingress of New Events
> ~~~~~~~~~~~~~~~~~~~~~
> @@ -445,8 +446,9 @@ using ``rte_event_dev_stop_flush_callback_register()`` function.
> .. Note::
>
> The event producers such as ``event_eth_rx_adapter``,
> - ``event_timer_adapter`` and ``event_crypto_adapter``
> - need to be stopped before stopping the event device.
> + ``event_timer_adapter``, ``event_crypto_adapter`` and
> + ``event_dma_adapter`` need to be stopped before stopping
> + the event device.
>
> diff --git a/doc/guides/prog_guide/index.rst b/doc/guides/prog_guide/index.rst
> index 52a6d9e7aa..beaa4b8869 100644
> --- a/doc/guides/prog_guide/index.rst
> +++ b/doc/guides/prog_guide/index.rst
> @@ -60,6 +60,7 @@ Programmer's Guide
> event_ethernet_tx_adapter
> event_timer_adapter
> event_crypto_adapter
> + event_dma_adapter
> qos_framework
> power_man
> packet_classif_access_ctrl
> diff --git a/doc/guides/rel_notes/release_23_11.rst b/doc/guides/rel_notes/release_23_11.rst
> index b34ddc0860..70f2e7e03c 100644
> --- a/doc/guides/rel_notes/release_23_11.rst
> +++ b/doc/guides/rel_notes/release_23_11.rst
> @@ -44,7 +44,9 @@ New Features
> .. This section should contain new features added in this release.
> Sample format:
>
> - * **Add a title in the past tense with a full stop.**
Dont delete this. See other commits which update the release notes
> + * **Added eventdev DMA adapter library.**
> +
> + Added API to DMA transfer data using event mechanism.
Pull the text which is add for Crypto adapter when it was addded.
>
> Add a short 1-2 sentence description in the past tense.
> The description should be enough to allow someone scanning
> diff --git a/lib/eventdev/eventdev_pmd.h b/lib/eventdev/eventdev_pmd.h
> index f62f42e140..8e5fd420f6 100644
> --- a/lib/eventdev/eventdev_pmd.h
> +++ b/lib/eventdev/eventdev_pmd.h
> @@ -178,8 +178,12 @@ struct rte_eventdev {
> event_tx_adapter_enqueue_t txa_enqueue;
> /**< Pointer to PMD eth Tx adapter enqueue function. */
> event_crypto_adapter_enqueue_t ca_enqueue;
> + /**< Pointer to PMD crypto adapter enqueue function. */
>
> - uint64_t reserved_64s[4]; /**< Reserved for future fields */
> + event_dma_adapter_enqueue_t dma_enqueue;
> + /**< Pointer to PMD DMA adapter enqueue function. */
> +
> + uint64_t reserved_64s[3]; /**< Reserved for future fields */
> void *reserved_ptrs[3]; /**< Reserved for future fields */
> } __rte_cache_aligned;
>
> @@ -1320,6 +1324,160 @@ typedef int (*eventdev_eth_tx_adapter_queue_stop)
>
> #define eventdev_stop_flush_t rte_eventdev_stop_flush_t
>
> +/**
> + * Retrieve the event device's DMA adapter capabilities for the
> + * specified DMA device
> + *
> + * @param dev
> + * Event device pointer
> + *
> + * @param dma_dev_id
> + * DMA device identification
> + *
> + * @param[out] caps
> + * A pointer to memory filled with event adapter capabilities.
> + * It is expected to be pre-allocated & initialized by caller.
> + *
> + * @return
> + * - 0: Success, driver provides event adapter capabilities for the
> + * DMADEV.
> + * - <0: Error code returned by the driver function.
> + *
> + */
> +typedef int (*eventdev_dma_adapter_caps_get_t)(const struct rte_eventdev *dev,
> + const int16_t dma_dev_id, uint32_t *caps);
> +
> +/**
> + * This API may change without prior notice
Remove this.
> + *
> + * Add DMA vchan queue to event device. This callback is invoked if
> + * the caps returned from rte_event_dma_adapter_caps_get(, dmadev_id)
> + * has RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_* set.
> + *
> + * @param dev
> + * Event device pointer
> + *
> + * @param dma_dev_id
> + * DMADEV identification
> + *
> + * @param vchan_id
> + * DMADEV vchan queue identifier.
> + *
> + * @param event
> + * Event information required for binding dmadev queue pair to event queue.
> + * This structure will have a valid value for only those HW PMDs supporting
> + * @see RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_VCHAN_EV_BIND capability.
> + *
> + * @return
> + * - 0: Success, dmadev vchan added successfully.
> + * - <0: Error code returned by the driver function.
> + *
> + */
> +typedef int (*eventdev_dma_adapter_vchan_add_t)(const struct rte_eventdev *dev,
> + const int16_t dma_dev_id,
> + uint16_t vchan_id,
> + const struct rte_event *event);
> +
> +/**
> + * This API may change without prior notice
Remove this.
> + *
> + * Delete DMA vhcan to event device. This callback is invoked if
> + * the caps returned from rte_event_dma_adapter_caps_get(, dmadev_id)
> + * has RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_* set.
> + *
> + * @param dev
> + * Event device pointer
> + *
> + * @param dma_dev_id
> + * DMADEV identification
identification -> identifier.
Change DMADEV as dmadev across this patch for documentation.
> + *
> + * @param vchan_id
> + * dmadev vchan identifier.
> + *
> + * @return
> + * - 0: Success, dmadev vchan deleted successfully.
> + * - <0: Error code returned by the driver function.
> + *
> + */
> +typedef int (*eventdev_dma_adapter_vchan_del_t)(const struct rte_eventdev *dev,
> + const int16_t dma_dev_id,
> + uint16_t vchan_id);
> +
> +/**
> + * Start DMA adapter. This callback is invoked if
> + * the caps returned from rte_event_dma_adapter_caps_get(.., dmadev_id)
> + * has RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_* set and vchan for dmadev_id
> + * have been added to the event device.
> + *
> + * @param dev
> + * Event device pointer
> + *
> + * @param dma_dev_id
> + * DMA device identification
> + *
> + * @return
> + * - 0: Success, DMA adapter started successfully.
> + * - <0: Error code returned by the driver function.
> + */
> +typedef int (*eventdev_dma_adapter_start_t)(const struct rte_eventdev *dev,
> + const int16_t dma_dev_id);
> +
> +/**
> + * Stop DMA adapter. This callback is invoked if
> + * the caps returned from rte_event_dma_adapter_caps_get(.., dmadev_id)
> + * has RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_* set and vchan for dmadev_id
> + * have been added to the event device.
> + *
> + * @param dev
> + * Event device pointer
> + *
> + * @param dma_dev_id
> + * DMA device identification
> + *
> + * @return
> + * - 0: Success, DMA adapter stopped successfully.
> + * - <0: Error code returned by the driver function.
> + */
> +typedef int (*eventdev_dma_adapter_stop_t)(const struct rte_eventdev *dev,
> + const int16_t dma_dev_id);
> +
> +struct rte_event_dma_adapter_stats;
> +
> +/**
> + * Retrieve DMA adapter statistics.
> + *
> + * @param dev
> + * Event device pointer
> + *
> + * @param dma_dev_id
> + * DMA device identification
> + *
> + * @param[out] stats
> + * Pointer to stats structure
> + *
> + * @return
> + * Return 0 on success.
> + */
> +typedef int (*eventdev_dma_adapter_stats_get)(const struct rte_eventdev *dev,
> + const int16_t dma_dev_id,
> + struct rte_event_dma_adapter_stats *stats);
> +
> +/**
> + * Reset DMA adapter statistics.
> + *
> + * @param dev
> + * Event device pointer
> + *
> + * @param dma_dev_id
> + * DMA device identification
> + *
> + * @return
> + * Return 0 on success.
> + */
> +typedef int (*eventdev_dma_adapter_stats_reset)(const struct rte_eventdev *dev,
> + const int16_t dma_dev_id);
> +
> +
> /** Event device operations function pointer table */
> struct eventdev_ops {
> eventdev_info_get_t dev_infos_get; /**< Get device info. */
> @@ -1440,6 +1598,21 @@ struct eventdev_ops {
> eventdev_eth_tx_adapter_queue_stop eth_tx_adapter_queue_stop;
> /**< Stop Tx queue assigned to Tx adapter instance */
>
> + eventdev_dma_adapter_caps_get_t dma_adapter_caps_get;
> + /**< Get DMA adapter capabilities */
> + eventdev_dma_adapter_vchan_add_t dma_adapter_vchan_add;
> + /**< Add vchan queue to DMA adapter */
> + eventdev_dma_adapter_vchan_del_t dma_adapter_vchan_del;
> + /**< Delete vchan queue from DMA adapter */
> + eventdev_dma_adapter_start_t dma_adapter_start;
> + /**< Start DMA adapter */
> + eventdev_dma_adapter_stop_t dma_adapter_stop;
> + /**< Stop DMA adapter */
> + eventdev_dma_adapter_stats_get dma_adapter_stats_get;
> + /**< Get DMA stats */
> + eventdev_dma_adapter_stats_reset dma_adapter_stats_reset;
> + /**< Reset DMA stats */
> +
> eventdev_selftest dev_selftest;
> /**< Start eventdev Selftest */
>
> diff --git a/lib/eventdev/eventdev_private.c b/lib/eventdev/eventdev_private.c
> index 1d3d9d357e..18ed8bf3c8 100644
> --- a/lib/eventdev/eventdev_private.c
> +++ b/lib/eventdev/eventdev_private.c
> @@ -81,6 +81,14 @@ dummy_event_crypto_adapter_enqueue(__rte_unused void *port,
> return 0;
> }
>
> +static uint16_t
> +dummy_event_dma_adapter_enqueue(__rte_unused void *port, __rte_unused struct rte_event ev[],
> + __rte_unused uint16_t nb_events)
> +{
> + RTE_EDEV_LOG_ERR("event DMA adapter enqueue requested for unconfigured event device");
> + return 0;
> +}
> +
> void
> event_dev_fp_ops_reset(struct rte_event_fp_ops *fp_op)
> {
> @@ -97,6 +105,7 @@ event_dev_fp_ops_reset(struct rte_event_fp_ops *fp_op)
> .txa_enqueue_same_dest =
> dummy_event_tx_adapter_enqueue_same_dest,
> .ca_enqueue = dummy_event_crypto_adapter_enqueue,
> + .dma_enqueue = dummy_event_dma_adapter_enqueue,
> .data = dummy_data,
> };
>
> @@ -117,5 +126,6 @@ event_dev_fp_ops_set(struct rte_event_fp_ops *fp_op,
> fp_op->txa_enqueue = dev->txa_enqueue;
> fp_op->txa_enqueue_same_dest = dev->txa_enqueue_same_dest;
> fp_op->ca_enqueue = dev->ca_enqueue;
> + fp_op->dma_enqueue = dev->dma_enqueue;
> fp_op->data = dev->data->ports;
> }
> diff --git a/lib/eventdev/meson.build b/lib/eventdev/meson.build
> index 6edf98dfa5..21347f7c4c 100644
> --- a/lib/eventdev/meson.build
> +++ b/lib/eventdev/meson.build
> @@ -25,6 +25,7 @@ sources = files(
> )
> headers = files(
> 'rte_event_crypto_adapter.h',
> + 'rte_event_dma_adapter.h',
> 'rte_event_eth_rx_adapter.h',
> 'rte_event_eth_tx_adapter.h',
> 'rte_event_ring.h',
> diff --git a/lib/eventdev/rte_event_dma_adapter.h b/lib/eventdev/rte_event_dma_adapter.h
> new file mode 100644
> index 0000000000..d413d31a61
> --- /dev/null
> +++ b/lib/eventdev/rte_event_dma_adapter.h
> @@ -0,0 +1,582 @@
> +/* SPDX-License-Identifier: BSD-3-Clause
> + * Copyright (c) 2023 Marvell.
> + */
> +
> +#ifndef RTE_EVENT_DMA_ADAPTER
> +#define RTE_EVENT_DMA_ADAPTER
> +
> +/**
> + * @file rte_event_dma_adapter.h
> + *
> + * @warning
> + * @b EXPERIMENTAL:
> + * All functions in this file may be changed or removed without prior notice.
> + *
> + * DMA Event Adapter API.
> + *
> + * Eventdev library provides adapters to bridge between various components for providing new
> + * event source. The event DMA adapter is one of those adapters which is intended to bridge
> + * between event devices and DMA devices.
> + *
> + * The DMA adapter adds support to enqueue / dequeue DMA operations to / from event device. The
> + * packet flow between DMA device and the event device can be accomplished using both SW and HW
> + * based transfer mechanisms. The adapter uses an EAL service core function for SW based packet
> + * transfer and uses the eventdev PMD functions to configure HW based packet transfer between the
> + * DMA device and the event device.
> + *
> + * The application can choose to submit a DMA operation directly to an DMA device or send it to the
> + * DMA adapter via eventdev based on RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP_FWD capability. The
> + * first mode is known as the event new (RTE_EVENT_DMA_ADAPTER_OP_NEW) mode and the second as the
> + * event forward (RTE_EVENT_DMA_ADAPTER_OP_FORWARD) mode. The choice of mode can be specified while
> + * creating the adapter. In the former mode, it is an application responsibility to enable ingress
> + * packet ordering. In the latter mode, it is the adapter responsibility to enable the ingress
> + * packet ordering.
> + *
> + *
> + * Working model of RTE_EVENT_DMA_ADAPTER_OP_NEW mode:
> + *
> + * +--------------+ +--------------+
> + * | | | DMA stage |
> + * | Application |---[2]-->| + enqueue to |
> + * | | | dmadev |
> + * +--------------+ +--------------+
> + * ^ ^ |
> + * | | [3]
> + * [6] [1] |
> + * | | |
> + * +--------------+ |
> + * | | |
> + * | Event device | |
> + * | | |
> + * +--------------+ |
> + * ^ |
> + * | |
> + * [5] |
> + * | v
> + * +--------------+ +--------------+
> + * | | | |
> + * | DMA adapter |<--[4]---| dmadev |
> + * | | | |
> + * +--------------+ +--------------+
> + *
> + *
> + * [1] Application dequeues events from the previous stage.
> + * [2] Application prepares the DMA operations.
> + * [3] DMA operations are submitted to dmadev by application.
> + * [4] DMA adapter dequeues DMA completions from dmadev.
> + * [5] DMA adapter enqueues events to the eventdev.
> + * [6] Application dequeues from eventdev for further processing.
> + *
> + * In the RTE_EVENT_DMA_ADAPTER_OP_NEW mode, application submits DMA operations directly to DMA
> + * device. The DMA adapter then dequeues DMA completions from DMA device and enqueue events to the
> + * event device. This mode does not ensure ingress ordering, if the application directly enqueues
> + * to dmadev without going through DMA / atomic stage i.e. removing item [1] and [2].
> + *
> + * Events dequeued from the adapter will be treated as new events. In this mode, application needs
> + * to specify event information (response information) which is needed to enqueue an event after the
> + * DMA operation is completed.
> + *
> + *
> + * Working model of RTE_EVENT_DMA_ADAPTER_OP_FORWARD mode:
> + *
> + * +--------------+ +--------------+
> + * --[1]-->| |---[2]-->| Application |
> + * | Event device | | in |
> + * <--[8]--| |<--[3]---| Ordered stage|
> + * +--------------+ +--------------+
> + * ^ |
> + * | [4]
> + * [7] |
> + * | v
> + * +----------------+ +--------------+
> + * | |--[5]->| |
> + * | DMA adapter | | dmadev |
> + * | |<-[6]--| |
> + * +----------------+ +--------------+
> + *
> + *
> + * [1] Events from the previous stage.
> + * [2] Application in ordered stage dequeues events from eventdev.
> + * [3] Application enqueues DMA operations as events to eventdev.
> + * [4] DMA adapter dequeues event from eventdev.
> + * [5] DMA adapter submits DMA operations to dmadev (Atomic stage).
> + * [6] DMA adapter dequeues DMA completions from dmadev
> + * [7] DMA adapter enqueues events to the eventdev
> + * [8] Events to the next stage
> + *
> + * In the event forward (RTE_EVENT_DMA_ADAPTER_OP_FORWARD) mode, if the HW supports the capability
> + * RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP_FWD, application can directly submit the DMA
> + * operations to the dmadev. If not, application retrieves the event port of the DMA adapter
> + * through the API, rte_event_DMA_adapter_event_port_get(). Then, links its event queue to this
> + * port and starts enqueuing DMA operations as events to the eventdev. The adapter then dequeues
> + * the events and submits the DMA operations to the dmadev. After the DMA completions, the adapter
> + * enqueues events to the event device.
> + *
> + * Application can use this mode, when ingress packet ordering is needed. Events dequeued from the
> + * adapter will be treated as forwarded events. In this mode, the application needs to specify the
> + * dmadev ID and queue pair ID (request information) needed to enqueue an DMA operation in addition
> + * to the event information (response information) needed to enqueue an event after the DMA
> + * operation has completed.
> + *
> + * The event DMA adapter provides common APIs to configure the packet flow from the DMA device to
> + * event devices for both SW and HW based transfers. The DMA event adapter's functions are:
> + *
> + * - rte_event_dma_adapter_create_ext()
> + * - rte_event_dma_adapter_create()
> + * - rte_event_dma_adapter_free()
> + * - rte_event_dma_adapter_vchan_add()
> + * - rte_event_dma_adapter_vchan_del()
> + * - rte_event_dma_adapter_start()
> + * - rte_event_dma_adapter_stop()
> + * - rte_event_dma_adapter_stats_get()
> + * - rte_event_dma_adapter_stats_reset()
> + *
> + * The application creates an instance using rte_event_dma_adapter_create() or
> + * rte_event_dma_adapter_create_ext().
> + *
> + * dmadev queue pair addition / deletion is done using the rte_event_dma_adapter_vchan_add() /
> + * rte_event_dma_adapter_vchan_del() APIs. If HW supports the capability
> + * RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_VCHAN_EV_BIND, event information must be passed to the
> + * add API.
> + *
> + */
> +
> +#include <stdint.h>
> +
> +#include <rte_eventdev.h>
> +#include <rte_common.h>
> +#include <rte_dmadev_pmd.h>
Alphabetical order.
> +
> +#ifdef __cplusplus
> +extern "C" {
> +#endif
> +
> +
> +/**
> + * Enqueue a burst of DMA operations as event objects supplied in *rte_event* structure on an event
> + * DMA adapter designated by its event *evdev_id* through the event port specified by *port_id*.
> + * This function is supported if the eventdev PMD has the
> + * #RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP_FWD capability flag set.
> + *
> + * The *nb_events* parameter is the number of event objects to enqueue that are supplied in the
> + * *ev* array of *rte_event* structure.
> + *
> + * The rte_event_dma_adapter_enqueue() function returns the number of event objects it actually
> + * enqueued. A return value equal to *nb_events* means that all event objects have been enqueued.
> + *
> + * @param evdev_id
> + * The identifier of the device.
> + * @param port_id
> + * The identifier of the event port.
> + * @param ev
> + * Points to an array of *nb_events* objects of type *rte_event* structure which contain the
> + * event object enqueue operations to be processed.
> + * @param nb_events
> + * The number of event objects to enqueue, typically number of
> + * rte_event_port_attr_get(...RTE_EVENT_PORT_ATTR_ENQ_DEPTH...) available for this port.
> + *
> + * @return
> + * The number of event objects actually enqueued on the event device. The return value can be
> + * less than the value of the *nb_events* parameter when the event devices queue is full or if
> + * invalid parameters are specified in a *rte_event*. If the return value is less than *nb_events*,
> + * the remaining events at the end of ev[] are not consumed and the caller has to take care of them,
> + * and rte_errno is set accordingly. Possible errno values include:
> + *
Due to this newline, generated doxygen coming bad. Please check
generated doxygen html page
in browser.
> + * - EINVAL: The port ID is invalid, device ID is invalid, an event's queue ID is invalid, or an
> + * event's sched type doesn't match the capabilities of the destination queue.
> + * - ENOSPC: The event port was backpressured and unable to enqueue one or more events. This
> + * error code is only applicable to closed systems.
> + */
> +__rte_experimental
> +uint16_t rte_event_dma_adapter_enqueue(uint8_t evdev_id, uint8_t port_id, struct rte_event ev[],
> + uint16_t nb_events);
> +
> +#ifdef __cplusplus
>
>
> extern struct rte_event_fp_ops rte_event_fp_ops[RTE_EVENT_MAX_DEVS];
> diff --git a/lib/eventdev/version.map b/lib/eventdev/version.map
> index 7ce09a87bb..5d644b1759 100644
> --- a/lib/eventdev/version.map
> +++ b/lib/eventdev/version.map
> @@ -133,6 +133,22 @@ EXPERIMENTAL {
> rte_event_timer_remaining_ticks_get;
>
> # added in 23.11
> + rte_event_dma_adapter_caps_get;
> + rte_event_dma_adapter_create_ext;
> + rte_event_dma_adapter_create;
> + rte_event_dma_adapter_event_port_get;
> + rte_event_dma_adapter_enqueue;
> + rte_event_dma_adapter_free;
> + rte_event_dma_adapter_runtime_params_init;
> + rte_event_dma_adapter_runtime_params_set;
> + rte_event_dma_adapter_runtime_params_get;
Sort this list in alphabetical order.
> + rte_event_dma_adapter_service_id_get;
> + rte_event_dma_adapter_start;
> + rte_event_dma_adapter_stop;
> + rte_event_dma_adapter_stats_get;
> + rte_event_dma_adapter_stats_reset;
> + rte_event_dma_adapter_vchan_add;
> + rte_event_dma_adapter_vchan_del;
> rte_event_eth_rx_adapter_create_ext_with_params;
> };
>
> --
> 2.25.1
>
^ permalink raw reply [flat|nested] 110+ messages in thread
* Re: [PATCH v4 02/12] eventdev: api to get DMA adapter capabilities
2023-09-26 10:32 ` [PATCH v4 02/12] eventdev: api to get DMA adapter capabilities Amit Prakash Shukla
@ 2023-09-27 18:20 ` Jerin Jacob
0 siblings, 0 replies; 110+ messages in thread
From: Jerin Jacob @ 2023-09-27 18:20 UTC (permalink / raw)
To: Amit Prakash Shukla
Cc: Jerin Jacob, dev, fengchengwen, kevin.laatz, bruce.richardson,
conor.walsh, vattunuru, g.singh, sachin.saxena, hemant.agrawal,
cheng1.jiang, ndabilpuram, anoobj, mb
On Tue, Sep 26, 2023 at 4:03 PM Amit Prakash Shukla
<amitprakashs@marvell.com> wrote:
>
> Added a new eventdev API rte_event_dma_adapter_caps_get(), to get
> DMA adapter capabilities supported by the driver.
Patches from 2: Keep following heading template
eventdev/dma: support ...
example:
eventdev/dma: support adapter capabilities get
> Signed-off-by: Amit Prakash Shukla <amitprakashs@marvell.com>
> ---
> lib/eventdev/meson.build | 2 +-
> lib/eventdev/rte_eventdev.c | 23 +++++++++++++++++++++++
> lib/eventdev/rte_eventdev.h | 2 +-
> lib/meson.build | 2 +-
> 4 files changed, 26 insertions(+), 3 deletions(-)
>
> diff --git a/lib/eventdev/meson.build b/lib/eventdev/meson.build
> index 21347f7c4c..b46bbbc9aa 100644
> --- a/lib/eventdev/meson.build
> +++ b/lib/eventdev/meson.build
> @@ -43,5 +43,5 @@ driver_sdk_headers += files(
> 'event_timer_adapter_pmd.h',
> )
>
> -deps += ['ring', 'ethdev', 'hash', 'mempool', 'mbuf', 'timer', 'cryptodev']
> +deps += ['ring', 'ethdev', 'hash', 'mempool', 'mbuf', 'timer', 'cryptodev', 'dmadev']
> deps += ['telemetry']
> diff --git a/lib/eventdev/rte_eventdev.c b/lib/eventdev/rte_eventdev.c
> index 6ab4524332..60509c6efb 100644
> --- a/lib/eventdev/rte_eventdev.c
> +++ b/lib/eventdev/rte_eventdev.c
> @@ -20,6 +20,7 @@
> #include <rte_errno.h>
> #include <ethdev_driver.h>
> #include <rte_cryptodev.h>
> +#include <rte_dmadev.h>
> #include <cryptodev_pmd.h>
> #include <rte_telemetry.h>
>
> @@ -224,6 +225,28 @@ rte_event_eth_tx_adapter_caps_get(uint8_t dev_id, uint16_t eth_port_id,
> : 0;
> }
>
> +int
> +rte_event_dma_adapter_caps_get(uint8_t dev_id, uint8_t dma_dev_id, uint32_t *caps)
> +{
> + struct rte_eventdev *dev;
> +
> + RTE_EVENTDEV_VALID_DEVID_OR_ERR_RET(dev_id, -EINVAL);
> + if (!rte_dma_is_valid(dma_dev_id))
> + return -EINVAL;
> +
> + dev = &rte_eventdevs[dev_id];
> +
> + if (caps == NULL)
> + return -EINVAL;
> +
> + *caps = 0;
> +
> + if (dev->dev_ops->dma_adapter_caps_get)
> + return (*dev->dev_ops->dma_adapter_caps_get)(dev, dma_dev_id, caps);
> +
> + return 0;
> +}
> +
> static inline int
> event_dev_queue_config(struct rte_eventdev *dev, uint8_t nb_queues)
> {
> diff --git a/lib/eventdev/rte_eventdev.h b/lib/eventdev/rte_eventdev.h
> index 2c6ecc7f8f..41743f91b1 100644
> --- a/lib/eventdev/rte_eventdev.h
> +++ b/lib/eventdev/rte_eventdev.h
> @@ -1504,7 +1504,7 @@ rte_event_crypto_adapter_caps_get(uint8_t dev_id, uint8_t cdev_id,
> */
> __rte_experimental
> int
> -rte_event_dma_adapter_caps_get(uint8_t dev_id, int16_t dmadev_id, uint32_t *caps);
> +rte_event_dma_adapter_caps_get(uint8_t dev_id, uint8_t dmadev_id, uint32_t *caps);
Squash this to first patch.
^ permalink raw reply [flat|nested] 110+ messages in thread
* Re: [PATCH v4 03/12] eventdev: create and free API for DMA adapter
2023-09-26 10:32 ` [PATCH v4 03/12] eventdev: create and free API for DMA adapter Amit Prakash Shukla
@ 2023-09-27 18:23 ` Jerin Jacob
0 siblings, 0 replies; 110+ messages in thread
From: Jerin Jacob @ 2023-09-27 18:23 UTC (permalink / raw)
To: Amit Prakash Shukla
Cc: Bruce Richardson, Jerin Jacob, dev, fengchengwen, kevin.laatz,
conor.walsh, vattunuru, g.singh, sachin.saxena, hemant.agrawal,
cheng1.jiang, ndabilpuram, anoobj, mb
On Tue, Sep 26, 2023 at 6:31 PM Amit Prakash Shukla
<amitprakashs@marvell.com> wrote:
>
> Added API support to create and free DMA adapter. Create function shall be
> called with event device to be associated with the adapter and port
> configuration to setup an event port.
>
> Signed-off-by: Amit Prakash Shukla <amitprakashs@marvell.com>
> ---
> config/rte_config.h | 1 +
> lib/eventdev/meson.build | 1 +
> lib/eventdev/rte_event_dma_adapter.c | 335 +++++++++++++++++++++++++++
> 3 files changed, 337 insertions(+)
> create mode 100644 lib/eventdev/rte_event_dma_adapter.c
>
> diff --git a/config/rte_config.h b/config/rte_config.h
> index 400e44e3cf..401727703f 100644
> --- a/config/rte_config.h
> +++ b/config/rte_config.h
> @@ -77,6 +77,7 @@
> #define RTE_EVENT_ETH_INTR_RING_SIZE 1024
> #define RTE_EVENT_CRYPTO_ADAPTER_MAX_INSTANCE 32
> #define RTE_EVENT_ETH_TX_ADAPTER_MAX_INSTANCE 32
> +#define RTE_EVENT_DMA_ADAPTER_MAX_INSTANCE 32
>
> /* rawdev defines */
> #define RTE_RAWDEV_MAX_DEVS 64
> diff --git a/lib/eventdev/meson.build b/lib/eventdev/meson.build
> index b46bbbc9aa..250abcb154 100644
> --- a/lib/eventdev/meson.build
> +++ b/lib/eventdev/meson.build
> @@ -17,6 +17,7 @@ sources = files(
> 'eventdev_private.c',
> 'eventdev_trace_points.c',
> 'rte_event_crypto_adapter.c',
> + 'rte_event_dma_adapter.c',
> 'rte_event_eth_rx_adapter.c',
> 'rte_event_eth_tx_adapter.c',
> 'rte_event_ring.c',
> diff --git a/lib/eventdev/rte_event_dma_adapter.c b/lib/eventdev/rte_event_dma_adapter.c
> new file mode 100644
> index 0000000000..c7ffba1b47
> --- /dev/null
> +++ b/lib/eventdev/rte_event_dma_adapter.c
> @@ -0,0 +1,335 @@
> +/* SPDX-License-Identifier: BSD-3-Clause
> + * Copyright (c) 2023 Marvell.
> + */
> +
> +#include "rte_eventdev.h"
> +#include "eventdev_pmd.h"
Move this to another block.
> +#include "rte_event_dma_adapter.h"
> +
^ permalink raw reply [flat|nested] 110+ messages in thread
* Re: [PATCH v4 12/12] app/test: add event DMA adapter auto-test
2023-09-26 10:32 ` [PATCH v4 12/12] app/test: add event DMA adapter auto-test Amit Prakash Shukla
@ 2023-09-27 18:27 ` Jerin Jacob
0 siblings, 0 replies; 110+ messages in thread
From: Jerin Jacob @ 2023-09-27 18:27 UTC (permalink / raw)
To: Amit Prakash Shukla
Cc: dev, jerinj, fengchengwen, kevin.laatz, bruce.richardson,
conor.walsh, vattunuru, g.singh, sachin.saxena, hemant.agrawal,
cheng1.jiang, ndabilpuram, anoobj, mb
On Tue, Sep 26, 2023 at 4:11 PM Amit Prakash Shukla
<amitprakashs@marvell.com> wrote:
>
> Added testsuite to test the dma adapter functionality.
> The testsuite detects event and DMA device capability
> and accordingly dma adapter is configured and modes are
> tested.
Please add example command to test this with SW driver in git commit.
>
> Signed-off-by: Amit Prakash Shukla <amitprakashs@marvell.com>
> ---
> app/test/meson.build | 1 +
> app/test/test_event_dma_adapter.c | 805 ++++++++++++++++++++++++++++++
Missing updates to MAINTAINERS file for this new file.
^ permalink raw reply [flat|nested] 110+ messages in thread
* Re: [PATCH v4 01/12] eventdev: introduce event DMA adapter library
2023-09-27 18:12 ` Jerin Jacob
@ 2023-09-27 20:45 ` Thomas Monjalon
2023-09-28 4:04 ` Jerin Jacob
0 siblings, 1 reply; 110+ messages in thread
From: Thomas Monjalon @ 2023-09-27 20:45 UTC (permalink / raw)
To: Amit Prakash Shukla, Jerin Jacob
Cc: Jerin Jacob, dev, fengchengwen, kevin.laatz, bruce.richardson,
conor.walsh, vattunuru, g.singh, sachin.saxena, hemant.agrawal,
cheng1.jiang, ndabilpuram, anoobj, mb
27/09/2023 20:12, Jerin Jacob:
> Use ethdev/dma for all patches like other adapters.
>
> Rewrite suggestion:
>
> eventdev/dma: introduce event dma adapter
While changing the title, please keep DMA uppercased after the colon.
Also, do we need "event" as it is in eventdev?
eventdev/dma: introduce DMA adapter
^ permalink raw reply [flat|nested] 110+ messages in thread
* Re: [PATCH v4 01/12] eventdev: introduce event DMA adapter library
2023-09-27 20:45 ` Thomas Monjalon
@ 2023-09-28 4:04 ` Jerin Jacob
0 siblings, 0 replies; 110+ messages in thread
From: Jerin Jacob @ 2023-09-28 4:04 UTC (permalink / raw)
To: Thomas Monjalon
Cc: Amit Prakash Shukla, Jerin Jacob, dev, fengchengwen, kevin.laatz,
bruce.richardson, conor.walsh, vattunuru, g.singh, sachin.saxena,
hemant.agrawal, cheng1.jiang, ndabilpuram, anoobj, mb
On Thu, Sep 28, 2023 at 2:15 AM Thomas Monjalon <thomas@monjalon.net> wrote:
>
> 27/09/2023 20:12, Jerin Jacob:
> > Use ethdev/dma for all patches like other adapters.
> >
> > Rewrite suggestion:
> >
> > eventdev/dma: introduce event dma adapter
>
> While changing the title, please keep DMA uppercased after the colon.
> Also, do we need "event" as it is in eventdev?
>
> eventdev/dma: introduce DMA adapter
That works..
>
>
>
^ permalink raw reply [flat|nested] 110+ messages in thread
* [PATCH v5 00/12] event DMA adapter library support
2023-09-26 10:32 ` [PATCH v4 00/12] event DMA adapter library support Amit Prakash Shukla
` (11 preceding siblings ...)
2023-09-26 10:32 ` [PATCH v4 12/12] app/test: add event DMA adapter auto-test Amit Prakash Shukla
@ 2023-09-28 10:36 ` Amit Prakash Shukla
2023-09-28 10:36 ` [PATCH v5 01/12] eventdev/dma: introduce DMA adapter Amit Prakash Shukla
` (12 more replies)
12 siblings, 13 replies; 110+ messages in thread
From: Amit Prakash Shukla @ 2023-09-28 10:36 UTC (permalink / raw)
Cc: dev, jerinj, fengchengwen, kevin.laatz, bruce.richardson,
conor.walsh, vattunuru, g.singh, sachin.saxena, hemant.agrawal,
cheng1.jiang, ndabilpuram, anoobj, mb, Amit Prakash Shukla
This series adds support for event DMA adapter library. API's defined
as part of this library can be used by the application for DMA transfer
of data using event based mechanism.
v5:
- Resolved review comments.
v4:
- Fixed compilation error.
v3:
- Resolved checkpatch warnings.
- Fixed compilation error on intel.
- Updated git commit message.
v2:
- Resolved review comments.
- Patch split into multiple patches.
Amit Prakash Shukla (12):
eventdev/dma: introduce DMA adapter
eventdev/dma: support adapter capabilities get
eventdev/dma: support adapter create and free
eventdev/dma: support for vchan add and delete
eventdev/dma: support for adapter service function
eventdev/dma: support for adapter start and stop
eventdev/dma: support for adapter service ID get
eventdev/dma: support adapter runtime params
eventdev/dma: support for adapter stats
eventdev/dma: support for adapter enqueue
eventdev/dma: support for adapter event port get
app/test: add event DMA adapter auto-test
MAINTAINERS | 8 +-
app/test/meson.build | 1 +
app/test/test_event_dma_adapter.c | 805 ++++++++++
config/rte_config.h | 1 +
doc/api/doxy-api-index.md | 1 +
doc/guides/eventdevs/features/default.ini | 8 +
doc/guides/prog_guide/event_dma_adapter.rst | 264 +++
doc/guides/prog_guide/eventdev.rst | 8 +-
.../img/event_dma_adapter_op_forward.svg | 1086 +++++++++++++
.../img/event_dma_adapter_op_new.svg | 1079 +++++++++++++
doc/guides/prog_guide/index.rst | 1 +
doc/guides/rel_notes/release_23_11.rst | 5 +
lib/eventdev/eventdev_pmd.h | 171 +-
lib/eventdev/eventdev_private.c | 10 +
lib/eventdev/meson.build | 4 +-
lib/eventdev/rte_event_dma_adapter.c | 1430 +++++++++++++++++
lib/eventdev/rte_event_dma_adapter.h | 581 +++++++
lib/eventdev/rte_eventdev.c | 23 +
lib/eventdev/rte_eventdev.h | 44 +
lib/eventdev/rte_eventdev_core.h | 8 +-
lib/eventdev/version.map | 16 +
lib/meson.build | 2 +-
22 files changed, 5548 insertions(+), 8 deletions(-)
create mode 100644 app/test/test_event_dma_adapter.c
create mode 100644 doc/guides/prog_guide/event_dma_adapter.rst
create mode 100644 doc/guides/prog_guide/img/event_dma_adapter_op_forward.svg
create mode 100644 doc/guides/prog_guide/img/event_dma_adapter_op_new.svg
create mode 100644 lib/eventdev/rte_event_dma_adapter.c
create mode 100644 lib/eventdev/rte_event_dma_adapter.h
--
2.25.1
^ permalink raw reply [flat|nested] 110+ messages in thread
* [PATCH v5 01/12] eventdev/dma: introduce DMA adapter
2023-09-28 10:36 ` [PATCH v5 00/12] event DMA adapter library support Amit Prakash Shukla
@ 2023-09-28 10:36 ` Amit Prakash Shukla
2023-09-28 15:33 ` Jerin Jacob
2023-09-28 10:36 ` [PATCH v5 02/12] eventdev/dma: support adapter capabilities get Amit Prakash Shukla
` (11 subsequent siblings)
12 siblings, 1 reply; 110+ messages in thread
From: Amit Prakash Shukla @ 2023-09-28 10:36 UTC (permalink / raw)
To: Thomas Monjalon, Amit Prakash Shukla, Jerin Jacob
Cc: dev, fengchengwen, kevin.laatz, bruce.richardson, conor.walsh,
vattunuru, g.singh, sachin.saxena, hemant.agrawal, cheng1.jiang,
ndabilpuram, anoobj, mb
Introduce event dma adapter interface to transfer packets between
dma device and event device.
Signed-off-by: Amit Prakash Shukla <amitprakashs@marvell.com>
Acked-by: Jerin Jacob <jerinj@marvell.com>
---
MAINTAINERS | 8 +-
doc/api/doxy-api-index.md | 1 +
doc/guides/eventdevs/features/default.ini | 8 +
doc/guides/prog_guide/event_dma_adapter.rst | 264 ++++
doc/guides/prog_guide/eventdev.rst | 8 +-
.../img/event_dma_adapter_op_forward.svg | 1086 +++++++++++++++++
.../img/event_dma_adapter_op_new.svg | 1079 ++++++++++++++++
doc/guides/prog_guide/index.rst | 1 +
doc/guides/rel_notes/release_23_11.rst | 5 +
lib/eventdev/eventdev_pmd.h | 171 ++-
lib/eventdev/eventdev_private.c | 10 +
lib/eventdev/meson.build | 1 +
lib/eventdev/rte_event_dma_adapter.h | 581 +++++++++
lib/eventdev/rte_eventdev.h | 44 +
lib/eventdev/rte_eventdev_core.h | 8 +-
lib/eventdev/version.map | 16 +
lib/meson.build | 2 +-
17 files changed, 3286 insertions(+), 7 deletions(-)
create mode 100644 doc/guides/prog_guide/event_dma_adapter.rst
create mode 100644 doc/guides/prog_guide/img/event_dma_adapter_op_forward.svg
create mode 100644 doc/guides/prog_guide/img/event_dma_adapter_op_new.svg
create mode 100644 lib/eventdev/rte_event_dma_adapter.h
diff --git a/MAINTAINERS b/MAINTAINERS
index a926155f26..e731a75d39 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -540,6 +540,13 @@ F: lib/eventdev/*crypto_adapter*
F: app/test/test_event_crypto_adapter.c
F: doc/guides/prog_guide/event_crypto_adapter.rst
+Eventdev DMA Adapter API
+M: Amit Prakash Shukla <amitprakashs@marvell.com>
+T: git://dpdk.org/next/dpdk-next-eventdev
+F: lib/eventdev/*dma_adapter*
+F: app/test/test_event_dma_adapter.c
+F: doc/guides/prog_guide/event_dma_adapter.rst
+
Raw device API
M: Sachin Saxena <sachin.saxena@oss.nxp.com>
M: Hemant Agrawal <hemant.agrawal@nxp.com>
@@ -548,7 +555,6 @@ F: drivers/raw/skeleton/
F: app/test/test_rawdev.c
F: doc/guides/prog_guide/rawdev.rst
-
Memory Pool Drivers
-------------------
diff --git a/doc/api/doxy-api-index.md b/doc/api/doxy-api-index.md
index fdeda13932..b7df7be4d9 100644
--- a/doc/api/doxy-api-index.md
+++ b/doc/api/doxy-api-index.md
@@ -29,6 +29,7 @@ The public API headers are grouped by topics:
[event_eth_tx_adapter](@ref rte_event_eth_tx_adapter.h),
[event_timer_adapter](@ref rte_event_timer_adapter.h),
[event_crypto_adapter](@ref rte_event_crypto_adapter.h),
+ [event_dma_adapter](@ref rte_event_dma_adapter.h),
[rawdev](@ref rte_rawdev.h),
[metrics](@ref rte_metrics.h),
[bitrate](@ref rte_bitrate.h),
diff --git a/doc/guides/eventdevs/features/default.ini b/doc/guides/eventdevs/features/default.ini
index 00360f60c6..73a52d915b 100644
--- a/doc/guides/eventdevs/features/default.ini
+++ b/doc/guides/eventdevs/features/default.ini
@@ -44,6 +44,14 @@ internal_port_op_fwd =
internal_port_qp_ev_bind =
session_private_data =
+;
+; Features of a default DMA adapter.
+;
+[DMA adapter Features]
+internal_port_op_new =
+internal_port_op_fwd =
+internal_port_vchan_ev_bind =
+
;
; Features of a default Timer adapter.
;
diff --git a/doc/guides/prog_guide/event_dma_adapter.rst b/doc/guides/prog_guide/event_dma_adapter.rst
new file mode 100644
index 0000000000..701e50d042
--- /dev/null
+++ b/doc/guides/prog_guide/event_dma_adapter.rst
@@ -0,0 +1,264 @@
+.. SPDX-License-Identifier: BSD-3-Clause
+ Copyright (c) 2023 Marvell.
+
+Event DMA Adapter Library
+=========================
+
+DPDK :doc:`Eventdev library <eventdev>` provides event driven programming model with features
+to schedule events. :doc:`DMA Device library <dmadev>` provides an interface to DMA poll mode
+drivers that support DMA operations. Event DMA Adapter is intended to bridge between the event
+device and the DMA device.
+
+Packet flow from DMA device to the event device can be accomplished using software and hardware
+based transfer mechanisms. The adapter queries an eventdev PMD to determine which mechanism to
+be used. The adapter uses an EAL service core function for software based packet transfer and
+uses the eventdev PMD functions to configure hardware based packet transfer between DMA device
+and the event device. DMA adapter uses a new event type called ``RTE_EVENT_TYPE_DMADEV`` to
+indicate the source of event.
+
+Application can choose to submit an DMA operation directly to an DMA device or send it to an DMA
+adapter via eventdev based on ``RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP_FWD`` capability. The
+first mode is known as the event new (``RTE_EVENT_DMA_ADAPTER_OP_NEW``) mode and the second as the
+event forward (``RTE_EVENT_DMA_ADAPTER_OP_FORWARD``) mode. Choice of mode can be specified while
+creating the adapter. In the former mode, it is the application's responsibility to enable
+ingress packet ordering. In the latter mode, it is the adapter's responsibility to enable
+ingress packet ordering.
+
+
+Adapter Modes
+-------------
+
+RTE_EVENT_DMA_ADAPTER_OP_NEW mode
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+In the ``RTE_EVENT_DMA_ADAPTER_OP_NEW`` mode, application submits DMA operations directly to an DMA
+device. The adapter then dequeues DMA completions from the DMA device and enqueues them as events
+to the event device. This mode does not ensure ingress ordering as the application directly
+enqueues to the dmadev without going through DMA/atomic stage. In this mode, events dequeued
+from the adapter are treated as new events. The application has to specify event information
+(response information) which is needed to enqueue an event after the DMA operation is completed.
+
+.. _figure_event_dma_adapter_op_new:
+
+.. figure:: img/event_dma_adapter_op_new.*
+
+ Working model of ``RTE_EVENT_DMA_ADAPTER_OP_NEW`` mode
+
+
+RTE_EVENT_DMA_ADAPTER_OP_FORWARD mode
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+In the ``RTE_EVENT_DMA_ADAPTER_OP_FORWARD`` mode, if the event PMD and DMA PMD supports internal
+event port (``RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP_FWD``), the application should use
+``rte_event_dma_adapter_enqueue()`` API to enqueue DMA operations as events to DMA adapter. If
+not, application retrieves DMA adapter's event port using ``rte_event_dma_adapter_event_port_get()``
+API, links its event queue to this port and starts enqueuing DMA operations as events to eventdev
+using ``rte_event_enqueue_burst()``. The adapter then dequeues the events and submits the DMA
+operations to the dmadev. After the DMA operation is complete, the adapter enqueues events to the
+event device.
+
+Applications can use this mode when ingress packet ordering is needed. In this mode, events
+dequeued from the adapter will be treated as forwarded events. Application has to specify event
+information (response information) needed to enqueue the event after the DMA operation has
+completed.
+
+.. _figure_event_dma_adapter_op_forward:
+
+.. figure:: img/event_dma_adapter_op_forward.*
+
+ Working model of ``RTE_EVENT_DMA_ADAPTER_OP_FORWARD`` mode
+
+
+API Overview
+------------
+
+This section has a brief introduction to the event DMA adapter APIs. The application is expected
+to create an adapter which is associated with a single eventdev, then add dmadev and vchan to the
+adapter instance.
+
+
+Create an adapter instance
+~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+An adapter instance is created using ``rte_event_dma_adapter_create()``. This function is called
+with event device to be associated with the adapter and port configuration for the adapter to
+setup an event port (if the adapter needs to use a service function).
+
+Adapter can be started in ``RTE_EVENT_DMA_ADAPTER_OP_NEW`` or ``RTE_EVENT_DMA_ADAPTER_OP_FORWARD``
+mode.
+
+.. code-block:: c
+
+ enum rte_event_dma_adapter_mode mode;
+ struct rte_event_dev_info dev_info;
+ struct rte_event_port_conf conf;
+ uint8_t evdev_id;
+ uint8_t dma_id;
+ int ret;
+
+ ret = rte_event_dev_info_get(dma_id, &dev_info);
+
+ conf.new_event_threshold = dev_info.max_num_events;
+ conf.dequeue_depth = dev_info.max_event_port_dequeue_depth;
+ conf.enqueue_depth = dev_info.max_event_port_enqueue_depth;
+ mode = RTE_EVENT_DMA_ADAPTER_OP_FORWARD;
+ ret = rte_event_dma_adapter_create(dma_id, evdev_id, &conf, mode);
+
+
+``rte_event_dma_adapter_create_ext()`` function can be used by the application to have a finer
+control on eventdev port allocation and setup. The ``rte_event_dma_adapter_create_ext()``
+function is passed a callback function. The callback function is invoked if the adapter creates
+a service function and uses an event port for it. The callback is expected to fill the
+``struct rte_event_dma_adapter_conf`` structure passed to it.
+
+In the ``RTE_EVENT_DMA_ADAPTER_OP_FORWARD`` mode, if the event PMD and DMA PMD supports internal
+event port (``RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP_FWD``), events with DMA operations should
+be enqueued to the DMA adapter using ``rte_event_dma_adapter_enqueue()`` API. If not, the event port
+created by the adapter can be retrieved using ``rte_event_dma_adapter_event_port_get()`` API. An
+application can use this event port to link with an event queue, on which it enqueues events
+towards the DMA adapter using ``rte_event_enqueue_burst()``.
+
+.. code-block:: c
+
+ uint8_t dma_adpt_id, evdev_id, dma_dev_id, dma_ev_port_id, app_qid;
+ struct rte_event ev;
+ uint32_t cap;
+ int ret;
+
+ // Fill in event info and update event_ptr with rte_dma_op
+ memset(&ev, 0, sizeof(ev));
+ .
+ .
+ ev.event_ptr = op;
+
+ ret = rte_event_dma_adapter_caps_get(evdev_id, dma_dev_id, &cap);
+ if (cap & RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP_FWD) {
+ ret = rte_event_dma_adapter_enqueue(evdev_id, app_ev_port_id, ev, nb_events);
+ } else {
+ ret = rte_event_dma_adapter_event_port_get(dma_adpt_id, &dma_ev_port_id);
+ ret = rte_event_queue_setup(evdev_id, app_qid, NULL);
+ ret = rte_event_port_link(evdev_id, dma_ev_port_id, &app_qid, NULL, 1);
+ ev.queue_id = app_qid;
+ ret = rte_event_enqueue_burst(evdev_id, app_ev_port_id, ev, nb_events);
+ }
+
+
+Event device configuration for service based adapter
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+When ``rte_event_dma_adapter_create()`` is used for creating adapter instance,
+``rte_event_dev_config::nb_event_ports`` is automatically incremented, and event device is
+reconfigured with additional event port during service initialization. This event device
+reconfigure logic also increments the ``rte_event_dev_config::nb_single_link_event_port_queues``
+parameter if the adapter event port config is of type ``RTE_EVENT_PORT_CFG_SINGLE_LINK``.
+
+Applications using this mode of adapter creation need not configure the event device with
+``rte_event_dev_config::nb_event_ports`` and
+``rte_event_dev_config::nb_single_link_event_port_queues`` parameters required for DMA adapter when
+the adapter is created using the above-mentioned API.
+
+
+Querying adapter capabilities
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+The ``rte_event_dma_adapter_caps_get()`` function allows the application to query the adapter
+capabilities for an eventdev and dmadev combination. This API provides whether dmadev and eventdev
+are connected using internal HW port or not.
+
+.. code-block:: c
+
+ rte_event_dma_adapter_caps_get(dev_id, dma_dev_id, &cap);
+
+
+Adding vchan to the adapter instance
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+dmadev device id and vchan are configured using dmadev APIs. For more information
+see :doc:`here <dmadev>`.
+
+.. code-block:: c
+
+ struct rte_dma_vchan_conf vchan_conf;
+ struct rte_dma_conf dev_conf;
+ uint8_t dev_id = 0;
+ uint16_t vchan = 0;
+
+ rte_dma_configure(dev_id, &dev_conf);
+ rte_dma_vchan_setup(dev_id, vchan, &vchan_conf);
+
+These dmadev id and vchan are added to the instance using the
+``rte_event_dma_adapter_vchan_add()`` API. The same is removed using
+``rte_event_dma_adapter_vchan_del()`` API. If hardware supports
+``RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_VCHAN_EV_BIND`` capability, event information must be passed to the add API.
+
+.. code-block:: c
+
+ uint32_t cap;
+ int ret;
+
+ ret = rte_event_dma_adapter_caps_get(evdev_id, dma_dev_id, &cap);
+ if (cap & RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_VCHAN_EV_BIND) {
+ struct rte_event event;
+
+ rte_event_dma_adapter_vchan_add(id, dma_dev_id, vchan, &conf);
+ } else
+ rte_event_dma_adapter_vchan_add(id, dma_dev_id, vchan, NULL);
+
+
+Configuring service function
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+If the adapter uses a service function, the application is required to assign a service core to
+the service function as show below.
+
+.. code-block:: c
+
+ uint32_t service_id;
+
+ if (rte_event_dma_adapter_service_id_get(dma_id, &service_id) == 0)
+ rte_service_map_lcore_set(service_id, CORE_ID);
+
+
+Set event response information
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+In the RTE_EVENT_DMA_ADAPTER_OP_FORWARD / RTE_EVENT_DMA_ADAPTER_OP_NEW mode, the application
+specifies the dmadev ID and vchan ID in ``struct rte_event_dma_adapter_op`` and the event
+information (response information) needed to enqueue an event after the DMA operation has
+completed. The response information is specified in ``struct rte_event`` and appended to the
+``struct rte_event_dma_adapter_op``.
+
+
+Start the adapter instance
+~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+The application calls ``rte_event_dma_adapter_start()`` to start the adapter. This function calls
+the start callbacks of the eventdev PMDs for hardware based eventdev-dmadev connections and
+``rte_service_run_state_set()`` to enable the service function if one exists.
+
+.. code-block:: c
+
+ rte_event_dma_adapter_start(id);
+
+.. Note::
+
+ The eventdev to which the event_dma_adapter is connected should be started before calling
+ rte_event_dma_adapter_start().
+
+
+Get adapter statistics
+~~~~~~~~~~~~~~~~~~~~~~
+
+The ``rte_event_dma_adapter_stats_get()`` function reports counters defined in struct
+``rte_event_dma_adapter_stats``. The received packet and enqueued event counts are a sum of the
+counts from the eventdev PMD callbacks if the callback is supported, and the counts maintained by
+the service function, if one exists.
+
+Set/Get adapter runtime configuration parameters
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+The runtime configuration parameters of adapter can be set/get using
+``rte_event_dma_adapter_runtime_params_set()`` and
+``rte_event_dma_adapter_runtime_params_get()`` respectively.
+The parameters that can be set/get are defined in
+``struct rte_event_dma_adapter_runtime_params``.
diff --git a/doc/guides/prog_guide/eventdev.rst b/doc/guides/prog_guide/eventdev.rst
index 2c83176846..ff55115d0d 100644
--- a/doc/guides/prog_guide/eventdev.rst
+++ b/doc/guides/prog_guide/eventdev.rst
@@ -333,7 +333,8 @@ eventdev.
.. Note::
EventDev needs to be started before starting the event producers such
- as event_eth_rx_adapter, event_timer_adapter and event_crypto_adapter.
+ as event_eth_rx_adapter, event_timer_adapter, event_crypto_adapter and
+ event_dma_adapter.
Ingress of New Events
~~~~~~~~~~~~~~~~~~~~~
@@ -445,8 +446,9 @@ using ``rte_event_dev_stop_flush_callback_register()`` function.
.. Note::
The event producers such as ``event_eth_rx_adapter``,
- ``event_timer_adapter`` and ``event_crypto_adapter``
- need to be stopped before stopping the event device.
+ ``event_timer_adapter``, ``event_crypto_adapter`` and
+ ``event_dma_adapter`` need to be stopped before stopping
+ the event device.
Summary
-------
diff --git a/doc/guides/prog_guide/img/event_dma_adapter_op_forward.svg b/doc/guides/prog_guide/img/event_dma_adapter_op_forward.svg
new file mode 100644
index 0000000000..b7fe1fecf2
--- /dev/null
+++ b/doc/guides/prog_guide/img/event_dma_adapter_op_forward.svg
@@ -0,0 +1,1086 @@
+<?xml version="1.0" encoding="UTF-8" standalone="no"?>
+<!-- SPDX-License-Identifier: BSD-3-Clause -->
+<!-- Copyright (c) 2023 Marvell. -->
+<!-- Created with Inkscape (http://www.inkscape.org/) -->
+
+<svg
+ width="720px"
+ height="486px"
+ id="svg13237"
+ version="1.1"
+ inkscape:version="1.2.1 (9c6d41e410, 2022-07-14)"
+ sodipodi:docname="event_dma_adapter_op_forward.svg"
+ xmlns:inkscape="http://www.inkscape.org/namespaces/inkscape"
+ xmlns:sodipodi="http://sodipodi.sourceforge.net/DTD/sodipodi-0.dtd"
+ xmlns="http://www.w3.org/2000/svg"
+ xmlns:svg="http://www.w3.org/2000/svg"
+ xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#"
+ xmlns:cc="http://creativecommons.org/ns#"
+ xmlns:dc="http://purl.org/dc/elements/1.1/"
+ xmlns:v="http://schemas.microsoft.com/visio/2003/SVGExtensions/">
+ <defs
+ id="defs13239">
+ <marker
+ inkscape:stockid="Arrow1Sstart"
+ orient="auto"
+ refY="0.0"
+ refX="0.0"
+ id="Arrow1Sstart"
+ style="overflow:visible">
+ <path
+ id="path8416"
+ d="M 0.0,0.0 L 5.0,-5.0 L -12.5,0.0 L 5.0,5.0 L 0.0,0.0 z "
+ style="fill-rule:evenodd;stroke:#000000;stroke-width:1.0pt"
+ transform="scale(0.2) translate(6,0)" />
+ </marker>
+ <marker
+ inkscape:stockid="Arrow1Send"
+ orient="auto"
+ refY="0.0"
+ refX="0.0"
+ id="Arrow1Send"
+ style="overflow:visible;">
+ <path
+ id="path8419"
+ d="M 0.0,0.0 L 5.0,-5.0 L -12.5,0.0 L 5.0,5.0 L 0.0,0.0 z "
+ style="fill-rule:evenodd;stroke:#000000;stroke-width:1.0pt;"
+ transform="scale(0.2) rotate(180) translate(6,0)" />
+ </marker>
+ <marker
+ inkscape:stockid="DiamondL"
+ orient="auto"
+ refY="0.0"
+ refX="0.0"
+ id="DiamondL"
+ style="overflow:visible">
+ <path
+ id="path8483"
+ d="M 0,-7.0710768 L -7.0710894,0 L 0,7.0710589 L 7.0710462,0 L 0,-7.0710768 z "
+ style="fill-rule:evenodd;stroke:#000000;stroke-width:1.0pt"
+ transform="scale(0.8)" />
+ </marker>
+ <marker
+ inkscape:stockid="DotL"
+ orient="auto"
+ refY="0.0"
+ refX="0.0"
+ id="DotL"
+ style="overflow:visible">
+ <path
+ id="path8465"
+ d="M -2.5,-1.0 C -2.5,1.7600000 -4.7400000,4.0 -7.5,4.0 C -10.260000,4.0 -12.5,1.7600000 -12.5,-1.0 C -12.5,-3.7600000 -10.260000,-6.0 -7.5,-6.0 C -4.7400000,-6.0 -2.5,-3.7600000 -2.5,-1.0 z "
+ style="fill-rule:evenodd;stroke:#000000;stroke-width:1.0pt"
+ transform="scale(0.8) translate(7.4, 1)" />
+ </marker>
+ <marker
+ inkscape:stockid="SquareL"
+ orient="auto"
+ refY="0.0"
+ refX="0.0"
+ id="SquareL"
+ style="overflow:visible">
+ <path
+ id="path8474"
+ d="M -5.0,-5.0 L -5.0,5.0 L 5.0,5.0 L 5.0,-5.0 L -5.0,-5.0 z "
+ style="fill-rule:evenodd;stroke:#000000;stroke-width:1.0pt"
+ transform="scale(0.8)" />
+ </marker>
+ <marker
+ inkscape:stockid="TriangleOutL"
+ orient="auto"
+ refY="0.0"
+ refX="0.0"
+ id="TriangleOutL"
+ style="overflow:visible">
+ <path
+ id="path8546"
+ d="M 5.77,0.0 L -2.88,5.0 L -2.88,-5.0 L 5.77,0.0 z "
+ style="fill-rule:evenodd;stroke:#000000;stroke-width:1.0pt"
+ transform="scale(0.8)" />
+ </marker>
+ <marker
+ inkscape:stockid="Arrow1Lstart"
+ orient="auto"
+ refY="0.0"
+ refX="0.0"
+ id="Arrow1Lstart"
+ style="overflow:visible">
+ <path
+ id="path8404"
+ d="M 0.0,0.0 L 5.0,-5.0 L -12.5,0.0 L 5.0,5.0 L 0.0,0.0 z "
+ style="fill-rule:evenodd;stroke:#000000;stroke-width:1.0pt"
+ transform="scale(0.8) translate(12.5,0)" />
+ </marker>
+ <marker
+ inkscape:stockid="Arrow1Mend"
+ orient="auto"
+ refY="0.0"
+ refX="0.0"
+ id="Arrow1Mend"
+ style="overflow:visible;">
+ <path
+ id="path8413"
+ d="M 0.0,0.0 L 5.0,-5.0 L -12.5,0.0 L 5.0,5.0 L 0.0,0.0 z "
+ style="fill-rule:evenodd;stroke:#000000;stroke-width:1.0pt;"
+ transform="scale(0.4) rotate(180) translate(10,0)" />
+ </marker>
+ <marker
+ inkscape:stockid="Arrow2Lend"
+ orient="auto"
+ refY="0.0"
+ refX="0.0"
+ id="Arrow2Lend"
+ style="overflow:visible;">
+ <path
+ id="path8425"
+ style="fill-rule:evenodd;stroke-width:0.62500000;stroke-linejoin:round;"
+ d="M 8.7185878,4.0337352 L -2.2072895,0.016013256 L 8.7185884,-4.0017078 C 6.9730900,-1.6296469 6.9831476,1.6157441 8.7185878,4.0337352 z "
+ transform="scale(1.1) rotate(180) translate(1,0)" />
+ </marker>
+ <marker
+ inkscape:stockid="Arrow1Lend"
+ orient="auto"
+ refY="0.0"
+ refX="0.0"
+ id="Arrow1Lend"
+ style="overflow:visible;">
+ <path
+ id="path8407"
+ d="M 0.0,0.0 L 5.0,-5.0 L -12.5,0.0 L 5.0,5.0 L 0.0,0.0 z "
+ style="fill-rule:evenodd;stroke:#000000;stroke-width:1.0pt;"
+ transform="scale(0.8) rotate(180) translate(12.5,0)" />
+ </marker>
+ <filter
+ id="filter_2"
+ color-interpolation-filters="sRGB">
+ <feGaussianBlur
+ stdDeviation="2"
+ id="feGaussianBlur15" />
+ </filter>
+ <filter
+ id="filter_2-3"
+ color-interpolation-filters="sRGB"
+ x="-0.086178862"
+ y="-0.086178862"
+ width="1.1723577"
+ height="1.1723577">
+ <feGaussianBlur
+ stdDeviation="2"
+ id="feGaussianBlur15-1" />
+ </filter>
+ <filter
+ id="filter_2-0"
+ color-interpolation-filters="sRGB">
+ <feGaussianBlur
+ stdDeviation="2"
+ id="feGaussianBlur15-7" />
+ </filter>
+ <filter
+ id="filter_2-0-8"
+ color-interpolation-filters="sRGB">
+ <feGaussianBlur
+ stdDeviation="2"
+ id="feGaussianBlur15-7-7" />
+ </filter>
+ <filter
+ id="filter_2-3-9"
+ color-interpolation-filters="sRGB">
+ <feGaussianBlur
+ stdDeviation="2"
+ id="feGaussianBlur15-1-6" />
+ </filter>
+ <filter
+ id="filter_2-3-6"
+ color-interpolation-filters="sRGB">
+ <feGaussianBlur
+ stdDeviation="2"
+ id="feGaussianBlur15-1-63" />
+ </filter>
+ <filter
+ id="filter_2-3-91"
+ color-interpolation-filters="sRGB"
+ x="-0.086178862"
+ y="-0.086178862"
+ width="1.1723577"
+ height="1.1723577">
+ <feGaussianBlur
+ stdDeviation="2"
+ id="feGaussianBlur15-1-3" />
+ </filter>
+ <marker
+ inkscape:stockid="Arrow1Lend"
+ orient="auto"
+ refY="0"
+ refX="0"
+ id="Arrow1Lend-5"
+ style="overflow:visible">
+ <path
+ inkscape:connector-curvature="0"
+ id="path8407-3"
+ d="M 0,0 5,-5 -12.5,0 5,5 0,0 z"
+ style="fill-rule:evenodd;stroke:#000000;stroke-width:1pt"
+ transform="matrix(-0.8,0,0,-0.8,-10,0)" />
+ </marker>
+ <marker
+ inkscape:stockid="Arrow1Lend"
+ orient="auto"
+ refY="0"
+ refX="0"
+ id="Arrow1Lend-6"
+ style="overflow:visible">
+ <path
+ inkscape:connector-curvature="0"
+ id="path8407-0"
+ d="M 0,0 5,-5 -12.5,0 5,5 0,0 z"
+ style="fill-rule:evenodd;stroke:#000000;stroke-width:1pt"
+ transform="matrix(-0.8,0,0,-0.8,-10,0)" />
+ </marker>
+ <marker
+ inkscape:stockid="Arrow1Lstart"
+ orient="auto"
+ refY="0"
+ refX="0"
+ id="Arrow1Lstart-7"
+ style="overflow:visible">
+ <path
+ inkscape:connector-curvature="0"
+ id="path8404-0"
+ d="M 0,0 5,-5 -12.5,0 5,5 0,0 z"
+ style="fill-rule:evenodd;stroke:#000000;stroke-width:1pt"
+ transform="matrix(0.8,0,0,0.8,10,0)" />
+ </marker>
+ <marker
+ inkscape:stockid="Arrow1Lend"
+ orient="auto"
+ refY="0"
+ refX="0"
+ id="Arrow1Lend-51"
+ style="overflow:visible">
+ <path
+ inkscape:connector-curvature="0"
+ id="path8407-1"
+ d="M 0,0 5,-5 -12.5,0 5,5 0,0 z"
+ style="fill-rule:evenodd;stroke:#000000;stroke-width:1pt"
+ transform="matrix(-0.8,0,0,-0.8,-10,0)" />
+ </marker>
+ <marker
+ inkscape:stockid="Arrow1Lend"
+ orient="auto"
+ refY="0"
+ refX="0"
+ id="Arrow1Lend-3"
+ style="overflow:visible">
+ <path
+ inkscape:connector-curvature="0"
+ id="path8407-6"
+ d="M 0,0 5,-5 -12.5,0 5,5 0,0 z"
+ style="fill-rule:evenodd;stroke:#000000;stroke-width:1pt"
+ transform="matrix(-0.8,0,0,-0.8,-10,0)" />
+ </marker>
+ <marker
+ inkscape:stockid="Arrow1Lend"
+ orient="auto"
+ refY="0"
+ refX="0"
+ id="Arrow1Lend-62"
+ style="overflow:visible">
+ <path
+ inkscape:connector-curvature="0"
+ id="path8407-9"
+ d="M 0,0 5,-5 -12.5,0 5,5 0,0 z"
+ style="fill-rule:evenodd;stroke:#000000;stroke-width:1pt"
+ transform="matrix(-0.8,0,0,-0.8,-10,0)" />
+ </marker>
+ <marker
+ inkscape:stockid="Arrow1Lend"
+ orient="auto"
+ refY="0"
+ refX="0"
+ id="Arrow1Lend-2"
+ style="overflow:visible">
+ <path
+ inkscape:connector-curvature="0"
+ id="path8407-7"
+ d="M 0,0 5,-5 -12.5,0 5,5 0,0 z"
+ style="fill-rule:evenodd;stroke:#000000;stroke-width:1pt"
+ transform="matrix(-0.8,0,0,-0.8,-10,0)" />
+ </marker>
+ <marker
+ inkscape:stockid="Arrow1Lstart"
+ orient="auto"
+ refY="0"
+ refX="0"
+ id="Arrow1Lstart-7-9"
+ style="overflow:visible">
+ <path
+ inkscape:connector-curvature="0"
+ id="path8404-0-3"
+ d="M 0,0 5,-5 -12.5,0 5,5 0,0 z"
+ style="fill-rule:evenodd;stroke:#000000;stroke-width:1pt"
+ transform="matrix(0.8,0,0,0.8,10,0)" />
+ </marker>
+ <filter
+ id="filter_2-3-6-1"
+ color-interpolation-filters="sRGB">
+ <feGaussianBlur
+ stdDeviation="2"
+ id="feGaussianBlur15-1-63-8" />
+ </filter>
+ <filter
+ id="filter_2-3-92"
+ color-interpolation-filters="sRGB"
+ x="-0.086178862"
+ y="-0.086178862"
+ width="1.1723577"
+ height="1.1723577">
+ <feGaussianBlur
+ stdDeviation="2"
+ id="feGaussianBlur15-1-2" />
+ </filter>
+ <filter
+ id="filter_2-3-94"
+ color-interpolation-filters="sRGB"
+ x="-0.086178862"
+ y="-0.086178862"
+ width="1.1723577"
+ height="1.1723577">
+ <feGaussianBlur
+ stdDeviation="2"
+ id="feGaussianBlur15-1-7" />
+ </filter>
+ <marker
+ inkscape:stockid="Arrow1Lstart"
+ orient="auto"
+ refY="0"
+ refX="0"
+ id="Arrow1Lstart-7-6"
+ style="overflow:visible">
+ <path
+ inkscape:connector-curvature="0"
+ id="path8404-0-1"
+ d="M 0,0 5,-5 -12.5,0 5,5 0,0 z"
+ style="fill-rule:evenodd;stroke:#000000;stroke-width:1pt"
+ transform="matrix(0.8,0,0,0.8,10,0)" />
+ </marker>
+ <marker
+ inkscape:stockid="Arrow1Lend"
+ orient="auto"
+ refY="0"
+ refX="0"
+ id="Arrow1Lend-55"
+ style="overflow:visible">
+ <path
+ inkscape:connector-curvature="0"
+ id="path8407-4"
+ d="M 0,0 5,-5 -12.5,0 5,5 0,0 z"
+ style="fill-rule:evenodd;stroke:#000000;stroke-width:1pt"
+ transform="matrix(-0.8,0,0,-0.8,-10,0)" />
+ </marker>
+ </defs>
+ <sodipodi:namedview
+ id="base"
+ pagecolor="#ffffff"
+ bordercolor="#666666"
+ borderopacity="1.0"
+ inkscape:pageopacity="0.0"
+ inkscape:pageshadow="2"
+ inkscape:zoom="1"
+ inkscape:cx="233.5"
+ inkscape:cy="288"
+ inkscape:document-units="px"
+ inkscape:current-layer="layer1"
+ showgrid="false"
+ inkscape:window-width="1920"
+ inkscape:window-height="1017"
+ inkscape:window-x="-8"
+ inkscape:window-y="-8"
+ inkscape:window-maximized="1"
+ inkscape:snap-nodes="false"
+ inkscape:showpageshadow="2"
+ inkscape:pagecheckerboard="0"
+ inkscape:deskcolor="#d1d1d1">
+ <inkscape:grid
+ type="xygrid"
+ id="grid13454" />
+ </sodipodi:namedview>
+ <metadata
+ id="metadata13242">
+ <rdf:RDF>
+ <cc:Work
+ rdf:about="">
+ <dc:format>image/svg+xml</dc:format>
+ <dc:type
+ rdf:resource="http://purl.org/dc/dcmitype/StillImage" />
+ <dc:title />
+ </cc:Work>
+ </rdf:RDF>
+ </metadata>
+ <g
+ id="layer1"
+ inkscape:label="Layer 1"
+ inkscape:groupmode="layer">
+ <g
+ style="font-size:12px;fill:none;stroke-linecap:square;stroke-miterlimit:3;overflow:visible"
+ id="shape1-1-2-4"
+ v:mID="1"
+ v:groupContext="shape"
+ transform="matrix(2.1604167,0,0,1.5671361,88.874699,-812.39909)">
+ <title
+ id="title22-7-5">Square</title>
+ <desc
+ id="desc24-7-8">Atomic Queue #1</desc>
+ <v:userDefs>
+ <v:ud
+ v:nameU="visVersion"
+ v:val="VT0(15):26" />
+ </v:userDefs>
+ <v:textBlock
+ v:margins="rect(4,4,4,4)" />
+ <v:textRect
+ cx="30.75"
+ cy="581.25"
+ width="61.5"
+ height="61.5" />
+ <g
+ id="shadow1-2-9-5"
+ v:groupContext="shadow"
+ v:shadowOffsetX="0.345598"
+ v:shadowOffsetY="-1.97279"
+ v:shadowType="1"
+ transform="translate(0.345598,1.97279)"
+ class="st1"
+ style="visibility:visible">
+ <rect
+ x="0"
+ y="550.5"
+ width="61.5"
+ height="61.5"
+ class="st2"
+ id="rect27-8-7"
+ style="fill:#5b9bd5;fill-opacity:0.22000002;stroke:#5b9bd5;stroke-opacity:0.22000002;filter:url(#filter_2-3-91)" />
+ </g>
+ <g
+ id="g13515-33">
+ <g
+ id="g13534-8">
+ <rect
+ x="0"
+ y="550.5"
+ width="61.5"
+ height="61.5"
+ class="st3"
+ id="rect29-1-95"
+ style="fill:#5b9bd5;stroke:#c7c8c8;stroke-width:0.25" />
+ </g>
+ </g>
+ </g>
+ <path
+ style="fill:none;stroke:#000000;stroke-width:0.712265;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:4;stroke-dasharray:none;stroke-opacity:1;marker-start:url(#Arrow1Lstart-7);marker-end:none"
+ d="M 312.28671,240.74335 H 227.99897"
+ id="path17209"
+ inkscape:connector-type="orthogonal"
+ inkscape:connector-curvature="0" />
+ <path
+ style="fill:none;stroke:#000000;stroke-width:0.718986px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1;marker-mid:none;marker-end:url(#Arrow1Lend)"
+ d="m 221.6484,77.57125 h 94.28101"
+ id="path17209-8"
+ inkscape:connector-type="orthogonal"
+ inkscape:connector-curvature="0" />
+ <g
+ style="font-size:12px;fill:none;stroke-linecap:square;stroke-miterlimit:3;overflow:visible"
+ id="shape1-1-2"
+ v:mID="1"
+ v:groupContext="shape"
+ transform="matrix(2.1604167,0,0,1.5671361,314.24227,-811.89589)">
+ <title
+ id="title22-7">Square</title>
+ <desc
+ id="desc24-7">Atomic Queue #1</desc>
+ <v:userDefs>
+ <v:ud
+ v:nameU="visVersion"
+ v:val="VT0(15):26" />
+ </v:userDefs>
+ <v:textBlock
+ v:margins="rect(4,4,4,4)" />
+ <v:textRect
+ cx="30.75"
+ cy="581.25"
+ width="61.5"
+ height="61.5" />
+ <g
+ id="shadow1-2-9"
+ v:groupContext="shadow"
+ v:shadowOffsetX="0.345598"
+ v:shadowOffsetY="-1.97279"
+ v:shadowType="1"
+ transform="translate(0.345598,1.97279)"
+ class="st1"
+ style="visibility:visible">
+ <rect
+ x="0"
+ y="550.5"
+ width="61.5"
+ height="61.5"
+ class="st2"
+ id="rect27-8"
+ style="fill:#5b9bd5;fill-opacity:0.22000002;stroke:#5b9bd5;stroke-opacity:0.22000002;filter:url(#filter_2-3)" />
+ </g>
+ <g
+ id="g13515">
+ <g
+ id="g13534">
+ <rect
+ x="0"
+ y="550.5"
+ width="61.5"
+ height="61.5"
+ class="st3"
+ id="rect29-1"
+ style="fill:#5b9bd5;stroke:#c7c8c8;stroke-width:0.25" />
+ </g>
+ </g>
+ </g>
+ <path
+ style="fill:none;stroke:#000000;stroke-width:0.724714;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:4;stroke-dasharray:none;stroke-opacity:1;marker-start:url(#Arrow1Lstart);marker-end:none"
+ d="M 89.025329,74.39932 H 24.750043"
+ id="path17209-3"
+ inkscape:connector-type="orthogonal"
+ inkscape:connector-curvature="0" />
+ <path
+ transform="matrix(0.73232502,0,0,0.75477602,-4.325033,28.642983)"
+ sodipodi:type="arc"
+ style="fill:#539de6;fill-opacity:1;stroke:#0000ea;stroke-width:1;stroke-linecap:square;stroke-miterlimit:4;stroke-opacity:1;stroke-dasharray:none"
+ id="path29161-3"
+ sodipodi:cx="371"
+ sodipodi:cy="64.5"
+ sodipodi:rx="17"
+ sodipodi:ry="15.5"
+ d="m 388,64.5 a 17,15.5 0 1 1 -34,0 17,15.5 0 1 1 34,0 z" />
+ <path
+ transform="matrix(0.73232502,0,0,0.75477602,-1.93108,192.80833)"
+ sodipodi:type="arc"
+ style="fill:#539de6;fill-opacity:1;stroke:#0000ea;stroke-width:1;stroke-linecap:square;stroke-miterlimit:4;stroke-opacity:1;stroke-dasharray:none"
+ id="path29161-1"
+ sodipodi:cx="371"
+ sodipodi:cy="64.5"
+ sodipodi:rx="17"
+ sodipodi:ry="15.5"
+ d="m 388,64.5 a 17,15.5 0 1 1 -34,0 17,15.5 0 1 1 34,0 z" />
+ <path
+ style="fill:none;stroke:#000000;stroke-width:0.751412;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:4;stroke-dasharray:none;stroke-opacity:1;marker-start:url(#Arrow1Lstart-7);marker-end:none"
+ d="M 18.763392,120.7432 H 87.758545"
+ id="path17209-3-0"
+ inkscape:connector-type="orthogonal"
+ inkscape:connector-curvature="0" />
+ <path
+ sodipodi:type="arc"
+ style="fill:#539de6;fill-opacity:1;stroke:#0000ea;stroke-width:1;stroke-linecap:square;stroke-miterlimit:4;stroke-opacity:1;stroke-dasharray:none"
+ id="path29161"
+ sodipodi:cx="371"
+ sodipodi:cy="64.5"
+ sodipodi:rx="17"
+ sodipodi:ry="15.5"
+ d="m 388,64.5 a 17,15.5 0 1 1 -34,0 17,15.5 0 1 1 34,0 z"
+ transform="matrix(0.73232502,0,0,0.75477602,-218.16394,72.68276)" />
+ <path
+ sodipodi:type="arc"
+ style="fill:#539de6;fill-opacity:1;stroke:#0000ea;stroke-width:1;stroke-linecap:square;stroke-miterlimit:4;stroke-opacity:1;stroke-dasharray:none"
+ id="path29161-2"
+ sodipodi:cx="371"
+ sodipodi:cy="64.5"
+ sodipodi:rx="17"
+ sodipodi:ry="15.5"
+ d="m 388,64.5 a 17,15.5 0 1 1 -34,0 17,15.5 0 1 1 34,0 z"
+ transform="matrix(0.73232502,0,0,0.75477602,-217.40136,26.716271)" />
+ <g
+ id="g29167-4"
+ transform="matrix(0.73232502,0,0,0.75477602,-217.31662,28.007562)">
+ <text
+ id="text29163-9"
+ y="70"
+ x="365"
+ style="font-style:normal;font-weight:normal;line-height:0%;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none"
+ xml:space="preserve"><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="70"
+ x="365"
+ id="tspan29165-9"
+ sodipodi:role="line">1</tspan></text>
+ </g>
+ <g
+ id="g29167-9"
+ transform="matrix(0.73232502,0,0,0.75477602,-4.9726112,28.689051)">
+ <text
+ id="text29163-3"
+ y="70"
+ x="365"
+ style="font-style:normal;font-weight:normal;line-height:0%;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none"
+ xml:space="preserve"><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="70"
+ x="365"
+ id="tspan29165-3"
+ sodipodi:role="line">2</tspan></text>
+ </g>
+ <path
+ style="fill:none;stroke:#000000;stroke-width:0.678033px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1;marker-start:url(#Arrow1Lstart-7);marker-end:none"
+ d="M 181,214.66098 V 145.33902"
+ id="path17211-7-1-6"
+ inkscape:connector-type="orthogonal"
+ inkscape:connector-curvature="0" />
+ <g
+ id="g29167"
+ transform="matrix(0.73232502,0,0,0.75477602,-218.07919,73.10621)">
+ <text
+ id="text29163"
+ y="70"
+ x="365"
+ style="font-style:normal;font-weight:normal;line-height:0%;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none"
+ xml:space="preserve"><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="70"
+ x="365"
+ id="tspan29165"
+ sodipodi:role="line">8</tspan></text>
+ </g>
+ <path
+ style="fill:none;stroke:#000000;stroke-width:0.678033px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1;marker-start:url(#Arrow1Lstart-7);marker-end:none"
+ d="m 131,145.8531 v 69.32197"
+ id="path17211-7-1"
+ inkscape:connector-type="orthogonal"
+ inkscape:connector-curvature="0" />
+ <path
+ transform="matrix(0.73232502,0,0,0.75477602,-140.37076,129.97088)"
+ sodipodi:type="arc"
+ style="fill:#539de6;fill-opacity:1;stroke:#0000ea;stroke-width:1;stroke-linecap:square;stroke-miterlimit:4;stroke-opacity:1;stroke-dasharray:none"
+ id="path29161-8"
+ sodipodi:cx="371"
+ sodipodi:cy="64.5"
+ sodipodi:rx="17"
+ sodipodi:ry="15.5"
+ d="m 388,64.5 a 17,15.5 0 1 1 -34,0 17,15.5 0 1 1 34,0 z" />
+ <g
+ id="g29167-2"
+ transform="matrix(0.73232502,0,0,0.75477602,-140.28602,131.01695)">
+ <text
+ id="text29163-92"
+ y="70"
+ x="365"
+ style="font-style:normal;font-weight:normal;line-height:0%;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none"
+ xml:space="preserve"><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="70"
+ x="365"
+ id="tspan29165-8"
+ sodipodi:role="line">7</tspan></text>
+ </g>
+ <path
+ style="fill:none;stroke:#000000;stroke-width:0.718986px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1;marker-mid:none;marker-end:url(#Arrow1Lend)"
+ d="m 317.1405,116 h -94.281"
+ id="path17209-8-0"
+ inkscape:connector-type="orthogonal"
+ inkscape:connector-curvature="0" />
+ <path
+ transform="matrix(0.73232502,0,0,0.75477602,-3.4914,66.68745)"
+ sodipodi:type="arc"
+ style="fill:#539de6;fill-opacity:1;stroke:#0000ea;stroke-width:1;stroke-linecap:square;stroke-miterlimit:4;stroke-opacity:1;stroke-dasharray:none"
+ id="path29161-6"
+ sodipodi:cx="371"
+ sodipodi:cy="64.5"
+ sodipodi:rx="17"
+ sodipodi:ry="15.5"
+ d="m 388,64.5 a 17,15.5 0 1 1 -34,0 17,15.5 0 1 1 34,0 z" />
+ <g
+ id="g29167-46"
+ transform="matrix(0.73232502,0,0,0.75477602,-4.40666,67.48829)">
+ <text
+ id="text29163-1"
+ y="70"
+ x="365"
+ style="font-style:normal;font-weight:normal;line-height:0%;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none"
+ xml:space="preserve"><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="70"
+ x="365"
+ id="tspan29165-5"
+ sodipodi:role="line">3</tspan></text>
+ </g>
+ <path
+ transform="matrix(0.73232502,0,0,0.75477602,-90.692582,130.31695)"
+ sodipodi:type="arc"
+ style="fill:#539de6;fill-opacity:1;stroke:#0000ea;stroke-width:1;stroke-linecap:square;stroke-miterlimit:4;stroke-opacity:1;stroke-dasharray:none"
+ id="path29161-8-6"
+ sodipodi:cx="371"
+ sodipodi:cy="64.5"
+ sodipodi:rx="17"
+ sodipodi:ry="15.5"
+ d="m 388,64.5 a 17,15.5 0 1 1 -34,0 17,15.5 0 1 1 34,0 z" />
+ <g
+ id="g29167-6"
+ transform="matrix(0.73232502,0,0,0.75477602,-90.84634,131.60918)">
+ <text
+ id="text29163-17"
+ y="70"
+ x="365"
+ style="font-style:normal;font-weight:normal;line-height:0%;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none"
+ xml:space="preserve"><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="70"
+ x="365"
+ id="tspan29165-2"
+ sodipodi:role="line">4</tspan></text>
+ </g>
+ <g
+ id="g29167-2-0"
+ transform="matrix(0.73232502,0,0,0.75477602,-2.424397,194.0216)">
+ <text
+ id="text29163-92-6"
+ y="70"
+ x="365"
+ style="font-style:normal;font-weight:normal;line-height:0%;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none"
+ xml:space="preserve"><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="70"
+ x="365"
+ id="tspan29165-8-2"
+ sodipodi:role="line">5</tspan></text>
+ </g>
+ <g
+ style="font-size:12px;fill:none;stroke-linecap:square;stroke-miterlimit:3;overflow:visible"
+ id="shape1-1-2-8"
+ v:mID="1"
+ v:groupContext="shape"
+ transform="matrix(2.1604167,0,0,1.5671361,93.82055,-648.98949)">
+ <title
+ id="title22-7-97">Square</title>
+ <desc
+ id="desc24-7-3">Atomic Queue #1</desc>
+ <v:userDefs>
+ <v:ud
+ v:nameU="visVersion"
+ v:val="VT0(15):26" />
+ </v:userDefs>
+ <v:textBlock
+ v:margins="rect(4,4,4,4)" />
+ <v:textRect
+ cx="30.75"
+ cy="581.25"
+ width="61.5"
+ height="61.5" />
+ <g
+ id="shadow1-2-9-6"
+ v:groupContext="shadow"
+ v:shadowOffsetX="0.345598"
+ v:shadowOffsetY="-1.97279"
+ v:shadowType="1"
+ transform="translate(0.345598,1.97279)"
+ class="st1"
+ style="visibility:visible">
+ <rect
+ x="0"
+ y="550.5"
+ width="61.5"
+ height="61.5"
+ class="st2"
+ id="rect27-8-12"
+ style="fill:#5b9bd5;fill-opacity:0.22000002;stroke:#5b9bd5;stroke-opacity:0.22000002;filter:url(#filter_2-3-92)" />
+ </g>
+ <g
+ id="g13515-9">
+ <g
+ id="g13534-3">
+ <rect
+ x="0"
+ y="550.5"
+ width="61.5"
+ height="61.5"
+ class="st3"
+ id="rect29-1-1"
+ style="fill:#5b9bd5;stroke:#c7c8c8;stroke-width:0.25" />
+ </g>
+ </g>
+ </g>
+ <g
+ style="font-size:12px;fill:none;stroke-linecap:square;stroke-miterlimit:3;overflow:visible"
+ id="shape1-1-2-84"
+ v:mID="1"
+ v:groupContext="shape"
+ transform="matrix(2.1604167,0,0,1.5671361,314.82055,-648.98949)">
+ <title
+ id="title22-7-50">Square</title>
+ <desc
+ id="desc24-7-36">Atomic Queue #1</desc>
+ <v:userDefs>
+ <v:ud
+ v:nameU="visVersion"
+ v:val="VT0(15):26" />
+ </v:userDefs>
+ <v:textBlock
+ v:margins="rect(4,4,4,4)" />
+ <v:textRect
+ cx="30.75"
+ cy="581.25"
+ width="61.5"
+ height="61.5" />
+ <g
+ id="shadow1-2-9-1"
+ v:groupContext="shadow"
+ v:shadowOffsetX="0.345598"
+ v:shadowOffsetY="-1.97279"
+ v:shadowType="1"
+ transform="translate(0.345598,1.97279)"
+ class="st1"
+ style="visibility:visible">
+ <rect
+ x="0"
+ y="550.5"
+ width="61.5"
+ height="61.5"
+ class="st2"
+ id="rect27-8-0"
+ style="fill:#5b9bd5;fill-opacity:0.22000002;stroke:#5b9bd5;stroke-opacity:0.22000002;filter:url(#filter_2-3-94)" />
+ </g>
+ <g
+ id="g13515-6">
+ <g
+ id="g13534-32">
+ <rect
+ x="0"
+ y="550.5"
+ width="61.5"
+ height="61.5"
+ class="st3"
+ id="rect29-1-0"
+ style="fill:#5b9bd5;stroke:#c7c8c8;stroke-width:0.25" />
+ </g>
+ </g>
+ </g>
+ <path
+ style="fill:none;stroke:#000000;stroke-width:0.712265;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:4;stroke-dasharray:none;stroke-opacity:1;marker-start:none;marker-end:url(#Arrow1Lend)"
+ d="M 313.14387,285 H 228.85613"
+ id="path17209-7"
+ inkscape:connector-type="orthogonal"
+ inkscape:connector-curvature="0" />
+ <path
+ transform="matrix(0.73232502,0,0,0.75477602,-2.692582,236.31695)"
+ sodipodi:type="arc"
+ style="fill:#539de6;fill-opacity:1;stroke:#0000ea;stroke-width:1;stroke-linecap:square;stroke-miterlimit:4;stroke-opacity:1;stroke-dasharray:none"
+ id="path29161-1-6"
+ sodipodi:cx="371"
+ sodipodi:cy="64.5"
+ sodipodi:rx="17"
+ sodipodi:ry="15.5"
+ d="m 388,64.5 a 17,15.5 0 1 1 -34,0 17,15.5 0 1 1 34,0 z" />
+ <g
+ id="g29167-2-0-5"
+ transform="matrix(0.73232502,0,0,0.75477602,-2.424397,237.0216)">
+ <text
+ id="text29163-92-6-6"
+ y="70"
+ x="365"
+ style="font-style:normal;font-weight:normal;line-height:0%;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none"
+ xml:space="preserve"><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="70"
+ x="365"
+ id="tspan29165-8-2-9"
+ sodipodi:role="line">6</tspan></text>
+ </g>
+ <g
+ id="g29167-4-3"
+ transform="matrix(0.73232502,0,0,0.75477602,-154.60784,51.117791)">
+ <text
+ id="text29163-9-6"
+ y="70"
+ x="365"
+ style="font-style:normal;font-weight:normal;line-height:0%;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none"
+ xml:space="preserve"><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:24.2109px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="70"
+ x="365"
+ id="tspan29165-9-7"
+ sodipodi:role="line">Eventdev</tspan></text>
+ </g>
+ <g
+ id="g29167-4-3-5"
+ transform="matrix(0.73232502,0,0,0.75477602,-144.65044,201.97821)">
+ <text
+ id="text29163-9-6-3"
+ y="70"
+ x="412.93716"
+ style="font-style:normal;font-weight:normal;line-height:0%;font-family:sans-serif;text-align:center;letter-spacing:0px;word-spacing:0px;text-anchor:middle;fill:#000000;fill-opacity:1;stroke:none"
+ xml:space="preserve"><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:24.2109px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:center;writing-mode:lr-tb;text-anchor:middle"
+ y="70"
+ x="412.93716"
+ id="tspan29165-9-7-5"
+ sodipodi:role="line">DMA</tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:24.2109px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:center;writing-mode:lr-tb;text-anchor:middle"
+ y="100.26363"
+ x="412.93716"
+ sodipodi:role="line"
+ id="tspan3201">Adapter</tspan></text>
+ </g>
+ <g
+ id="g29167-4-3-5-6"
+ transform="matrix(0.73232502,0,0,0.75477602,79.53518,46.62529)">
+ <text
+ id="text29163-9-6-3-2"
+ y="48.801659"
+ x="412.93716"
+ style="font-style:normal;font-weight:normal;line-height:0%;font-family:sans-serif;text-align:center;letter-spacing:0px;word-spacing:0px;text-anchor:middle;fill:#000000;fill-opacity:1;stroke:none"
+ xml:space="preserve"><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:24.2109px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:center;writing-mode:lr-tb;text-anchor:middle"
+ y="48.801659"
+ x="412.93716"
+ sodipodi:role="line"
+ id="tspan3155">Application</tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:24.2109px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:center;writing-mode:lr-tb;text-anchor:middle"
+ y="98.801659"
+ x="412.93716"
+ sodipodi:role="line"
+ id="tspan3201-1">in ordered</tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:24.2109px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:center;writing-mode:lr-tb;text-anchor:middle"
+ y="148.80167"
+ x="412.93716"
+ sodipodi:role="line"
+ id="tspan3161">stage</tspan></text>
+ </g>
+ <g
+ id="g29167-4-3-5-2"
+ transform="matrix(0.73232502,0,0,0.75477602,77.535182,213.62529)">
+ <text
+ id="text29163-9-6-3-7"
+ y="70"
+ x="412.93716"
+ style="font-style:normal;font-weight:normal;line-height:0%;font-family:sans-serif;text-align:center;letter-spacing:0px;word-spacing:0px;text-anchor:middle;fill:#000000;fill-opacity:1;stroke:none"
+ xml:space="preserve"><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:24.2109px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:center;writing-mode:lr-tb;text-anchor:middle"
+ y="70"
+ x="412.93716"
+ sodipodi:role="line"
+ id="tspan3201-9">DMA Device</tspan></text>
+ </g>
+ <g
+ id="g29167-4-3-5-3"
+ transform="matrix(0.73232502,0,0,0.75477602,188.53518,-3.37471)">
+ <text
+ id="text29163-9-6-3-6"
+ y="70"
+ x="375.65271"
+ style="font-style:normal;font-weight:normal;line-height:0%;font-family:sans-serif;text-align:center;letter-spacing:0px;word-spacing:0px;text-anchor:middle;fill:#000000;fill-opacity:1;stroke:none"
+ xml:space="preserve"><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18.8307px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="70"
+ x="375.65271"
+ sodipodi:role="line"
+ id="tspan3201-6">1. Events from the previous stage.</tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18.8307px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="93.538376"
+ x="375.65271"
+ sodipodi:role="line"
+ id="tspan3260"> </tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18.8307px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="117.07675"
+ x="375.65271"
+ sodipodi:role="line"
+ id="tspan3262">2. Application in ordered stage</tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18.8307px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="140.61513"
+ x="375.65271"
+ sodipodi:role="line"
+ id="tspan3288"> dequeues events from eventdev.</tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18.8307px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="164.1535"
+ x="375.65271"
+ sodipodi:role="line"
+ id="tspan3264"> </tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18.8307px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="187.69188"
+ x="375.65271"
+ sodipodi:role="line"
+ id="tspan3266">3. Application enqueues DMA</tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18.8307px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="211.23026"
+ x="375.65271"
+ sodipodi:role="line"
+ id="tspan3290"> operations as events to eventdev.</tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18.8307px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="234.76863"
+ x="375.65271"
+ sodipodi:role="line"
+ id="tspan3268"> </tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18.8307px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="258.30701"
+ x="375.65271"
+ sodipodi:role="line"
+ id="tspan3270">4. DMA adapter dequeues event</tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18.8307px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="281.84537"
+ x="375.65271"
+ sodipodi:role="line"
+ id="tspan3292"> from eventdev.</tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18.8307px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="305.38376"
+ x="375.65271"
+ sodipodi:role="line"
+ id="tspan3272"> </tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18.8307px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="328.92212"
+ x="375.65271"
+ sodipodi:role="line"
+ id="tspan3274">5. DMA adapter submits DMA</tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18.8307px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="352.46051"
+ x="375.65271"
+ sodipodi:role="line"
+ id="tspan3294"> operations to DMA Device (Atomic</tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18.8307px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="375.99887"
+ x="375.65271"
+ sodipodi:role="line"
+ id="tspan3296"> stage)</tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18.8307px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="399.53726"
+ x="375.65271"
+ sodipodi:role="line"
+ id="tspan3276"> </tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18.8307px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="423.07562"
+ x="375.65271"
+ sodipodi:role="line"
+ id="tspan3278">6. DMA adapter dequeues DMA</tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18.8307px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="446.61401"
+ x="375.65271"
+ sodipodi:role="line"
+ id="tspan3298"> completions from DMA Device</tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18.8307px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="470.15237"
+ x="375.65271"
+ sodipodi:role="line"
+ id="tspan3280"> </tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18.8307px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="493.69073"
+ x="375.65271"
+ sodipodi:role="line"
+ id="tspan3282">7. DMA adapter enqueues events</tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18.8307px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="517.22913"
+ x="375.65271"
+ sodipodi:role="line"
+ id="tspan3300"> to the eventdev</tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18.8307px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="540.76752"
+ x="375.65271"
+ sodipodi:role="line"
+ id="tspan3284"> </tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18.8307px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="564.30585"
+ x="375.65271"
+ sodipodi:role="line"
+ id="tspan3286">8. Events to the next stage</tspan></text>
+ </g>
+ </g>
+</svg>
diff --git a/doc/guides/prog_guide/img/event_dma_adapter_op_new.svg b/doc/guides/prog_guide/img/event_dma_adapter_op_new.svg
new file mode 100644
index 0000000000..e9e8bb2b98
--- /dev/null
+++ b/doc/guides/prog_guide/img/event_dma_adapter_op_new.svg
@@ -0,0 +1,1079 @@
+<?xml version="1.0" encoding="UTF-8" standalone="no"?>
+<!-- SPDX-License-Identifier: BSD-3-Clause -->
+<!-- Copyright (c) 2023 Marvell. -->
+<!-- Created with Inkscape (http://www.inkscape.org/) -->
+
+<svg
+ width="720px"
+ height="486px"
+ id="svg13237"
+ version="1.1"
+ inkscape:version="1.2.1 (9c6d41e410, 2022-07-14)"
+ sodipodi:docname="event_dma_adapter_op_new.svg"
+ xmlns:inkscape="http://www.inkscape.org/namespaces/inkscape"
+ xmlns:sodipodi="http://sodipodi.sourceforge.net/DTD/sodipodi-0.dtd"
+ xmlns="http://www.w3.org/2000/svg"
+ xmlns:svg="http://www.w3.org/2000/svg"
+ xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#"
+ xmlns:cc="http://creativecommons.org/ns#"
+ xmlns:dc="http://purl.org/dc/elements/1.1/"
+ xmlns:v="http://schemas.microsoft.com/visio/2003/SVGExtensions/">
+ <defs
+ id="defs13239">
+ <marker
+ inkscape:stockid="Arrow1Sstart"
+ orient="auto"
+ refY="0.0"
+ refX="0.0"
+ id="Arrow1Sstart"
+ style="overflow:visible">
+ <path
+ id="path8416"
+ d="M 0.0,0.0 L 5.0,-5.0 L -12.5,0.0 L 5.0,5.0 L 0.0,0.0 z "
+ style="fill-rule:evenodd;stroke:#000000;stroke-width:1.0pt"
+ transform="scale(0.2) translate(6,0)" />
+ </marker>
+ <marker
+ inkscape:stockid="Arrow1Send"
+ orient="auto"
+ refY="0.0"
+ refX="0.0"
+ id="Arrow1Send"
+ style="overflow:visible;">
+ <path
+ id="path8419"
+ d="M 0.0,0.0 L 5.0,-5.0 L -12.5,0.0 L 5.0,5.0 L 0.0,0.0 z "
+ style="fill-rule:evenodd;stroke:#000000;stroke-width:1.0pt;"
+ transform="scale(0.2) rotate(180) translate(6,0)" />
+ </marker>
+ <marker
+ inkscape:stockid="DiamondL"
+ orient="auto"
+ refY="0.0"
+ refX="0.0"
+ id="DiamondL"
+ style="overflow:visible">
+ <path
+ id="path8483"
+ d="M 0,-7.0710768 L -7.0710894,0 L 0,7.0710589 L 7.0710462,0 L 0,-7.0710768 z "
+ style="fill-rule:evenodd;stroke:#000000;stroke-width:1.0pt"
+ transform="scale(0.8)" />
+ </marker>
+ <marker
+ inkscape:stockid="DotL"
+ orient="auto"
+ refY="0.0"
+ refX="0.0"
+ id="DotL"
+ style="overflow:visible">
+ <path
+ id="path8465"
+ d="M -2.5,-1.0 C -2.5,1.7600000 -4.7400000,4.0 -7.5,4.0 C -10.260000,4.0 -12.5,1.7600000 -12.5,-1.0 C -12.5,-3.7600000 -10.260000,-6.0 -7.5,-6.0 C -4.7400000,-6.0 -2.5,-3.7600000 -2.5,-1.0 z "
+ style="fill-rule:evenodd;stroke:#000000;stroke-width:1.0pt"
+ transform="scale(0.8) translate(7.4, 1)" />
+ </marker>
+ <marker
+ inkscape:stockid="SquareL"
+ orient="auto"
+ refY="0.0"
+ refX="0.0"
+ id="SquareL"
+ style="overflow:visible">
+ <path
+ id="path8474"
+ d="M -5.0,-5.0 L -5.0,5.0 L 5.0,5.0 L 5.0,-5.0 L -5.0,-5.0 z "
+ style="fill-rule:evenodd;stroke:#000000;stroke-width:1.0pt"
+ transform="scale(0.8)" />
+ </marker>
+ <marker
+ inkscape:stockid="TriangleOutL"
+ orient="auto"
+ refY="0.0"
+ refX="0.0"
+ id="TriangleOutL"
+ style="overflow:visible">
+ <path
+ id="path8546"
+ d="M 5.77,0.0 L -2.88,5.0 L -2.88,-5.0 L 5.77,0.0 z "
+ style="fill-rule:evenodd;stroke:#000000;stroke-width:1.0pt"
+ transform="scale(0.8)" />
+ </marker>
+ <marker
+ inkscape:stockid="Arrow1Lstart"
+ orient="auto"
+ refY="0.0"
+ refX="0.0"
+ id="Arrow1Lstart"
+ style="overflow:visible">
+ <path
+ id="path8404"
+ d="M 0.0,0.0 L 5.0,-5.0 L -12.5,0.0 L 5.0,5.0 L 0.0,0.0 z "
+ style="fill-rule:evenodd;stroke:#000000;stroke-width:1.0pt"
+ transform="scale(0.8) translate(12.5,0)" />
+ </marker>
+ <marker
+ inkscape:stockid="Arrow1Mend"
+ orient="auto"
+ refY="0.0"
+ refX="0.0"
+ id="Arrow1Mend"
+ style="overflow:visible;">
+ <path
+ id="path8413"
+ d="M 0.0,0.0 L 5.0,-5.0 L -12.5,0.0 L 5.0,5.0 L 0.0,0.0 z "
+ style="fill-rule:evenodd;stroke:#000000;stroke-width:1.0pt;"
+ transform="scale(0.4) rotate(180) translate(10,0)" />
+ </marker>
+ <marker
+ inkscape:stockid="Arrow2Lend"
+ orient="auto"
+ refY="0.0"
+ refX="0.0"
+ id="Arrow2Lend"
+ style="overflow:visible;">
+ <path
+ id="path8425"
+ style="fill-rule:evenodd;stroke-width:0.62500000;stroke-linejoin:round;"
+ d="M 8.7185878,4.0337352 L -2.2072895,0.016013256 L 8.7185884,-4.0017078 C 6.9730900,-1.6296469 6.9831476,1.6157441 8.7185878,4.0337352 z "
+ transform="scale(1.1) rotate(180) translate(1,0)" />
+ </marker>
+ <marker
+ inkscape:stockid="Arrow1Lend"
+ orient="auto"
+ refY="0.0"
+ refX="0.0"
+ id="Arrow1Lend"
+ style="overflow:visible;">
+ <path
+ id="path8407"
+ d="M 0.0,0.0 L 5.0,-5.0 L -12.5,0.0 L 5.0,5.0 L 0.0,0.0 z "
+ style="fill-rule:evenodd;stroke:#000000;stroke-width:1.0pt;"
+ transform="scale(0.8) rotate(180) translate(12.5,0)" />
+ </marker>
+ <filter
+ id="filter_2"
+ color-interpolation-filters="sRGB">
+ <feGaussianBlur
+ stdDeviation="2"
+ id="feGaussianBlur15" />
+ </filter>
+ <filter
+ id="filter_2-3"
+ color-interpolation-filters="sRGB"
+ x="-0.086178862"
+ y="-0.086178862"
+ width="1.1723577"
+ height="1.1723577">
+ <feGaussianBlur
+ stdDeviation="2"
+ id="feGaussianBlur15-1" />
+ </filter>
+ <filter
+ id="filter_2-0"
+ color-interpolation-filters="sRGB">
+ <feGaussianBlur
+ stdDeviation="2"
+ id="feGaussianBlur15-7" />
+ </filter>
+ <filter
+ id="filter_2-0-8"
+ color-interpolation-filters="sRGB">
+ <feGaussianBlur
+ stdDeviation="2"
+ id="feGaussianBlur15-7-7" />
+ </filter>
+ <filter
+ id="filter_2-3-9"
+ color-interpolation-filters="sRGB"
+ x="-0.086178862"
+ y="-0.086178862"
+ width="1.1723577"
+ height="1.1723577">
+ <feGaussianBlur
+ stdDeviation="2"
+ id="feGaussianBlur15-1-6" />
+ </filter>
+ <filter
+ id="filter_2-3-6"
+ color-interpolation-filters="sRGB"
+ x="-0.086178862"
+ y="-0.086178862"
+ width="1.1723577"
+ height="1.1723577">
+ <feGaussianBlur
+ stdDeviation="2"
+ id="feGaussianBlur15-1-63" />
+ </filter>
+ <filter
+ id="filter_2-3-91"
+ color-interpolation-filters="sRGB"
+ x="-0.086178862"
+ y="-0.086178862"
+ width="1.1723577"
+ height="1.1723577">
+ <feGaussianBlur
+ stdDeviation="2"
+ id="feGaussianBlur15-1-3" />
+ </filter>
+ <marker
+ inkscape:stockid="Arrow1Lend"
+ orient="auto"
+ refY="0"
+ refX="0"
+ id="Arrow1Lend-5"
+ style="overflow:visible">
+ <path
+ inkscape:connector-curvature="0"
+ id="path8407-3"
+ d="M 0,0 5,-5 -12.5,0 5,5 0,0 z"
+ style="fill-rule:evenodd;stroke:#000000;stroke-width:1pt"
+ transform="matrix(-0.8,0,0,-0.8,-10,0)" />
+ </marker>
+ <marker
+ inkscape:stockid="Arrow1Lend"
+ orient="auto"
+ refY="0"
+ refX="0"
+ id="Arrow1Lend-6"
+ style="overflow:visible">
+ <path
+ inkscape:connector-curvature="0"
+ id="path8407-0"
+ d="M 0,0 5,-5 -12.5,0 5,5 0,0 z"
+ style="fill-rule:evenodd;stroke:#000000;stroke-width:1pt"
+ transform="matrix(-0.8,0,0,-0.8,-10,0)" />
+ </marker>
+ <marker
+ inkscape:stockid="Arrow1Lstart"
+ orient="auto"
+ refY="0"
+ refX="0"
+ id="Arrow1Lstart-7"
+ style="overflow:visible">
+ <path
+ inkscape:connector-curvature="0"
+ id="path8404-0"
+ d="M 0,0 5,-5 -12.5,0 5,5 0,0 z"
+ style="fill-rule:evenodd;stroke:#000000;stroke-width:1pt"
+ transform="matrix(0.8,0,0,0.8,10,0)" />
+ </marker>
+ <marker
+ inkscape:stockid="Arrow1Lend"
+ orient="auto"
+ refY="0"
+ refX="0"
+ id="Arrow1Lend-51"
+ style="overflow:visible">
+ <path
+ inkscape:connector-curvature="0"
+ id="path8407-1"
+ d="M 0,0 5,-5 -12.5,0 5,5 0,0 z"
+ style="fill-rule:evenodd;stroke:#000000;stroke-width:1pt"
+ transform="matrix(-0.8,0,0,-0.8,-10,0)" />
+ </marker>
+ <marker
+ inkscape:stockid="Arrow1Lend"
+ orient="auto"
+ refY="0"
+ refX="0"
+ id="Arrow1Lend-3"
+ style="overflow:visible">
+ <path
+ inkscape:connector-curvature="0"
+ id="path8407-6"
+ d="M 0,0 5,-5 -12.5,0 5,5 0,0 z"
+ style="fill-rule:evenodd;stroke:#000000;stroke-width:1pt"
+ transform="matrix(-0.8,0,0,-0.8,-10,0)" />
+ </marker>
+ <marker
+ inkscape:stockid="Arrow1Lend"
+ orient="auto"
+ refY="0"
+ refX="0"
+ id="Arrow1Lend-62"
+ style="overflow:visible">
+ <path
+ inkscape:connector-curvature="0"
+ id="path8407-9"
+ d="M 0,0 5,-5 -12.5,0 5,5 0,0 z"
+ style="fill-rule:evenodd;stroke:#000000;stroke-width:1pt"
+ transform="matrix(-0.8,0,0,-0.8,-10,0)" />
+ </marker>
+ <marker
+ inkscape:stockid="Arrow1Lend"
+ orient="auto"
+ refY="0"
+ refX="0"
+ id="Arrow1Lend-2"
+ style="overflow:visible">
+ <path
+ inkscape:connector-curvature="0"
+ id="path8407-7"
+ d="M 0,0 5,-5 -12.5,0 5,5 0,0 z"
+ style="fill-rule:evenodd;stroke:#000000;stroke-width:1pt"
+ transform="matrix(-0.8,0,0,-0.8,-10,0)" />
+ </marker>
+ <filter
+ id="filter_2-3-91-3"
+ color-interpolation-filters="sRGB"
+ x="-0.086178862"
+ y="-0.086178862"
+ width="1.1723577"
+ height="1.1723577">
+ <feGaussianBlur
+ stdDeviation="2"
+ id="feGaussianBlur15-1-3-6" />
+ </filter>
+ </defs>
+ <sodipodi:namedview
+ id="base"
+ pagecolor="#ffffff"
+ bordercolor="#666666"
+ borderopacity="1.0"
+ inkscape:pageopacity="0.0"
+ inkscape:pageshadow="2"
+ inkscape:zoom="1"
+ inkscape:cx="385.5"
+ inkscape:cy="234"
+ inkscape:document-units="px"
+ inkscape:current-layer="layer1"
+ showgrid="false"
+ inkscape:window-width="1920"
+ inkscape:window-height="1017"
+ inkscape:window-x="-8"
+ inkscape:window-y="-8"
+ inkscape:window-maximized="1"
+ inkscape:snap-nodes="false"
+ inkscape:showpageshadow="2"
+ inkscape:pagecheckerboard="0"
+ inkscape:deskcolor="#d1d1d1">
+ <inkscape:grid
+ type="xygrid"
+ id="grid13454" />
+ </sodipodi:namedview>
+ <metadata
+ id="metadata13242">
+ <rdf:RDF>
+ <cc:Work
+ rdf:about="">
+ <dc:format>image/svg+xml</dc:format>
+ <dc:type
+ rdf:resource="http://purl.org/dc/dcmitype/StillImage" />
+ <dc:title />
+ </cc:Work>
+ </rdf:RDF>
+ </metadata>
+ <g
+ id="layer1"
+ inkscape:label="Layer 1"
+ inkscape:groupmode="layer">
+ <g
+ style="font-size:12px;fill:none;stroke-linecap:square;stroke-miterlimit:3;overflow:visible"
+ id="shape1-1-2-0"
+ v:mID="1"
+ v:groupContext="shape"
+ transform="matrix(2.1604167,0,0,1.5671361,323.2187,-540.25927)">
+ <title
+ id="title22-7-6">Square</title>
+ <desc
+ id="desc24-7-4">Atomic Queue #1</desc>
+ <v:userDefs>
+ <v:ud
+ v:nameU="visVersion"
+ v:val="VT0(15):26" />
+ </v:userDefs>
+ <v:textBlock
+ v:margins="rect(4,4,4,4)" />
+ <v:textRect
+ cx="30.75"
+ cy="581.25"
+ width="61.5"
+ height="61.5" />
+ <g
+ id="shadow1-2-9-0"
+ v:groupContext="shadow"
+ v:shadowOffsetX="0.345598"
+ v:shadowOffsetY="-1.97279"
+ v:shadowType="1"
+ transform="translate(0.345598,1.97279)"
+ class="st1"
+ style="visibility:visible">
+ <rect
+ x="0"
+ y="550.5"
+ width="61.5"
+ height="61.5"
+ class="st2"
+ id="rect27-8-9"
+ style="fill:#5b9bd5;fill-opacity:0.22000002;stroke:#5b9bd5;stroke-opacity:0.22000002;filter:url(#filter_2-3-9)" />
+ </g>
+ <g
+ id="g13515-4">
+ <g
+ id="g13534-5">
+ <rect
+ x="0"
+ y="550.5"
+ width="61.5"
+ height="61.5"
+ class="st3"
+ id="rect29-1-4"
+ style="fill:#5b9bd5;stroke:#c7c8c8;stroke-width:0.25" />
+ </g>
+ </g>
+ </g>
+ <g
+ style="font-size:12px;fill:none;stroke-linecap:square;stroke-miterlimit:3;overflow:visible"
+ id="shape1-1-2-4"
+ v:mID="1"
+ v:groupContext="shape"
+ transform="matrix(2.1604167,0,0,1.165886,88.874699,-447.8809)">
+ <title
+ id="title22-7-5">Square</title>
+ <desc
+ id="desc24-7-8">Atomic Queue #1</desc>
+ <v:userDefs>
+ <v:ud
+ v:nameU="visVersion"
+ v:val="VT0(15):26" />
+ </v:userDefs>
+ <v:textBlock
+ v:margins="rect(4,4,4,4)" />
+ <v:textRect
+ cx="30.75"
+ cy="581.25"
+ width="61.5"
+ height="61.5" />
+ <g
+ id="shadow1-2-9-5"
+ v:groupContext="shadow"
+ v:shadowOffsetX="0.345598"
+ v:shadowOffsetY="-1.97279"
+ v:shadowType="1"
+ transform="translate(0.345598,1.97279)"
+ class="st1"
+ style="visibility:visible">
+ <rect
+ x="0"
+ y="550.5"
+ width="61.5"
+ height="61.5"
+ class="st2"
+ id="rect27-8-7"
+ style="fill:#5b9bd5;fill-opacity:0.22000002;stroke:#5b9bd5;stroke-opacity:0.22000002;filter:url(#filter_2-3-91)" />
+ </g>
+ <g
+ id="g13515-33">
+ <g
+ id="g13534-8">
+ <rect
+ x="0"
+ y="550.5"
+ width="61.5"
+ height="61.5"
+ class="st3"
+ id="rect29-1-95"
+ style="fill:#5b9bd5;stroke:#c7c8c8;stroke-width:0.25" />
+ </g>
+ </g>
+ </g>
+ <g
+ style="font-size:12px;fill:none;stroke-linecap:square;stroke-miterlimit:3;overflow:visible"
+ id="shape1-1-2-9"
+ v:mID="1"
+ v:groupContext="shape"
+ transform="matrix(2.1604167,0,0,1.5671361,88.874699,-538.24651)">
+ <title
+ id="title22-7-9">Square</title>
+ <desc
+ id="desc24-7-5">Atomic Queue #1</desc>
+ <v:userDefs>
+ <v:ud
+ v:nameU="visVersion"
+ v:val="VT0(15):26" />
+ </v:userDefs>
+ <v:textBlock
+ v:margins="rect(4,4,4,4)" />
+ <v:textRect
+ cx="30.75"
+ cy="581.25"
+ width="61.5"
+ height="61.5" />
+ <g
+ id="shadow1-2-9-2"
+ v:groupContext="shadow"
+ v:shadowOffsetX="0.345598"
+ v:shadowOffsetY="-1.97279"
+ v:shadowType="1"
+ transform="translate(0.345598,1.97279)"
+ class="st1"
+ style="visibility:visible">
+ <rect
+ x="0"
+ y="550.5"
+ width="61.5"
+ height="61.5"
+ class="st2"
+ id="rect27-8-1"
+ style="fill:#5b9bd5;fill-opacity:0.22000002;stroke:#5b9bd5;stroke-opacity:0.22000002;filter:url(#filter_2-3-6)" />
+ </g>
+ <g
+ id="g13515-3">
+ <g
+ id="g13534-0">
+ <rect
+ x="0"
+ y="550.5"
+ width="61.5"
+ height="61.5"
+ class="st3"
+ id="rect29-1-9"
+ style="fill:#5b9bd5;stroke:#c7c8c8;stroke-width:0.25" />
+ </g>
+ </g>
+ </g>
+ <path
+ style="fill:none;stroke:#000000;stroke-width:0.743466px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1;marker-mid:none;marker-end:url(#Arrow1Lend)"
+ d="M 220.66064,98.57125 H 321.88592"
+ id="path17209-8"
+ inkscape:connector-type="orthogonal"
+ inkscape:connector-curvature="0" />
+ <g
+ style="font-size:12px;fill:none;stroke-linecap:square;stroke-miterlimit:3;overflow:visible"
+ id="shape1-1-2"
+ v:mID="1"
+ v:groupContext="shape"
+ transform="matrix(2.1604167,0,0,1.5671361,322.24227,-811.89589)">
+ <title
+ id="title22-7">Square</title>
+ <desc
+ id="desc24-7">Atomic Queue #1</desc>
+ <v:userDefs>
+ <v:ud
+ v:nameU="visVersion"
+ v:val="VT0(15):26" />
+ </v:userDefs>
+ <v:textBlock
+ v:margins="rect(4,4,4,4)" />
+ <v:textRect
+ cx="30.75"
+ cy="581.25"
+ width="61.5"
+ height="61.5" />
+ <g
+ id="shadow1-2-9"
+ v:groupContext="shadow"
+ v:shadowOffsetX="0.345598"
+ v:shadowOffsetY="-1.97279"
+ v:shadowType="1"
+ transform="translate(0.345598,1.97279)"
+ class="st1"
+ style="visibility:visible">
+ <rect
+ x="0"
+ y="550.5"
+ width="61.5"
+ height="61.5"
+ class="st2"
+ id="rect27-8"
+ style="fill:#5b9bd5;fill-opacity:0.22000002;stroke:#5b9bd5;stroke-opacity:0.22000002;filter:url(#filter_2-3)" />
+ </g>
+ <g
+ id="g13515">
+ <g
+ id="g13534">
+ <rect
+ x="0"
+ y="550.5"
+ width="61.5"
+ height="61.5"
+ class="st3"
+ id="rect29-1"
+ style="fill:#5b9bd5;stroke:#c7c8c8;stroke-width:0.25" />
+ </g>
+ </g>
+ </g>
+ <g
+ id="g13518"
+ transform="matrix(0.73232502,0,0,0.75477602,25.29268,348.89752)">
+ <g
+ id="g13526">
+ <flowRoot
+ xml:space="preserve"
+ id="flowRoot13464-9"
+ style="font-style:normal;font-weight:normal;line-height:0.01%;font-family:sans-serif;text-align:center;letter-spacing:0px;word-spacing:0px;text-anchor:middle;fill:#000000;fill-opacity:1;stroke:none"
+ transform="translate(-12.00521,-129.65179)"><flowRegion
+ id="flowRegion13466-1"
+ style="font-family:sans-serif"><rect
+ id="rect13468-2"
+ width="195.99997"
+ height="112.00001"
+ x="273.33334"
+ y="175.33333"
+ style="text-align:center;text-anchor:middle" /></flowRegion><flowPara
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:20px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:center;writing-mode:lr-tb;text-anchor:middle"
+ id="flowPara13511"> </flowPara></flowRoot>
+ </g>
+ </g>
+ <path
+ style="fill:none;stroke:#000000;stroke-width:0.751455;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:4;stroke-dasharray:none;stroke-opacity:1;marker-start:url(#Arrow1Lstart);marker-end:none"
+ d="m 176.26096,124.64833 v 69.24854"
+ id="path17209-3"
+ inkscape:connector-type="orthogonal"
+ inkscape:connector-curvature="0" />
+ <path
+ transform="matrix(0.73232502,0,0,0.75477602,-4.325033,50.642983)"
+ sodipodi:type="arc"
+ style="fill:#539de6;fill-opacity:1;stroke:#0000ea;stroke-width:1;stroke-linecap:square;stroke-miterlimit:4;stroke-opacity:1;stroke-dasharray:none"
+ id="path29161-3"
+ sodipodi:cx="371"
+ sodipodi:cy="64.5"
+ sodipodi:rx="17"
+ sodipodi:ry="15.5"
+ d="m 388,64.5 a 17,15.5 0 1 1 -34,0 17,15.5 0 1 1 34,0 z" />
+ <path
+ style="fill:none;stroke:#000000;stroke-width:0.743466px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1;marker-mid:none;marker-end:url(#Arrow1Lend)"
+ d="M 322.61264,375 H 221.38736"
+ id="path17209-8-0"
+ inkscape:connector-type="orthogonal"
+ inkscape:connector-curvature="0" />
+ <path
+ transform="matrix(0.73232502,0,0,0.75477602,0.0689171,324.80833)"
+ sodipodi:type="arc"
+ style="fill:#539de6;fill-opacity:1;stroke:#0000ea;stroke-width:1;stroke-linecap:square;stroke-miterlimit:4;stroke-opacity:1;stroke-dasharray:none"
+ id="path29161-1"
+ sodipodi:cx="371"
+ sodipodi:cy="64.5"
+ sodipodi:rx="17"
+ sodipodi:ry="15.5"
+ d="m 388,64.5 a 17,15.5 0 1 1 -34,0 17,15.5 0 1 1 34,0 z" />
+ <path
+ style="fill:none;stroke:#000000;stroke-width:0.629082px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1;marker-end:url(#Arrow1Lend)"
+ d="M 155,324.19955 V 264.82863"
+ id="path17211-7-1"
+ inkscape:connector-type="orthogonal"
+ inkscape:connector-curvature="0" />
+ <path
+ transform="matrix(0.73232502,0,0,0.75477602,-116.37076,245.97088)"
+ sodipodi:type="arc"
+ style="fill:#539de6;fill-opacity:1;stroke:#0000ea;stroke-width:1;stroke-linecap:square;stroke-miterlimit:4;stroke-opacity:1;stroke-dasharray:none"
+ id="path29161-8"
+ sodipodi:cx="371"
+ sodipodi:cy="64.5"
+ sodipodi:rx="17"
+ sodipodi:ry="15.5"
+ d="m 388,64.5 a 17,15.5 0 1 1 -34,0 17,15.5 0 1 1 34,0 z" />
+ <path
+ style="fill:none;stroke:#000000;stroke-width:0.75059;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:4;stroke-dasharray:none;stroke-opacity:1;marker-start:url(#Arrow1Lstart-7);marker-end:none"
+ d="m 126.26097,124.99178 v 69.24941"
+ id="path17209-3-0"
+ inkscape:connector-type="orthogonal"
+ inkscape:connector-curvature="0" />
+ <path
+ sodipodi:type="arc"
+ style="fill:#539de6;fill-opacity:1;stroke:#0000ea;stroke-width:1;stroke-linecap:square;stroke-miterlimit:4;stroke-opacity:1;stroke-dasharray:none"
+ id="path29161"
+ sodipodi:cx="371"
+ sodipodi:cy="64.5"
+ sodipodi:rx="17"
+ sodipodi:ry="15.5"
+ d="m 388,64.5 a 17,15.5 0 1 1 -34,0 17,15.5 0 1 1 34,0 z"
+ transform="matrix(0.73232502,0,0,0.75477602,-146.16394,110.68276)" />
+ <path
+ sodipodi:type="arc"
+ style="fill:#539de6;fill-opacity:1;stroke:#0000ea;stroke-width:1;stroke-linecap:square;stroke-miterlimit:4;stroke-opacity:1;stroke-dasharray:none"
+ id="path29161-2"
+ sodipodi:cx="371"
+ sodipodi:cy="64.5"
+ sodipodi:rx="17"
+ sodipodi:ry="15.5"
+ d="m 388,64.5 a 17,15.5 0 1 1 -34,0 17,15.5 0 1 1 34,0 z"
+ transform="matrix(0.73232502,0,0,0.75477602,-95.40136,110.71627)" />
+ <g
+ id="g29167-4"
+ transform="matrix(0.73232502,0,0,0.75477602,-95.31662,112.00756)">
+ <text
+ id="text29163-9"
+ y="70"
+ x="365"
+ style="font-style:normal;font-weight:normal;line-height:0%;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none"
+ xml:space="preserve"><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="70"
+ x="365"
+ id="tspan29165-9"
+ sodipodi:role="line">1</tspan></text>
+ </g>
+ <g
+ id="g29167-9"
+ transform="matrix(0.73232502,0,0,0.75477602,-4.9726112,50.689051)">
+ <text
+ id="text29163-3"
+ y="70"
+ x="365"
+ style="font-style:normal;font-weight:normal;line-height:0%;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none"
+ xml:space="preserve"><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="70"
+ x="365"
+ id="tspan29165-3"
+ sodipodi:role="line">2</tspan></text>
+ </g>
+ <path
+ style="fill:none;stroke:#000000;stroke-width:1.04033px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1;marker-end:url(#Arrow1Lend)"
+ d="M 388.20118,147.93341 V 321.89308"
+ id="path17211-7"
+ inkscape:connector-type="orthogonal"
+ inkscape:connector-curvature="0" />
+ <path
+ transform="matrix(0.73232502,0,0,0.75477602,116.5086,136.68745)"
+ sodipodi:type="arc"
+ style="fill:#539de6;fill-opacity:1;stroke:#0000ea;stroke-width:1;stroke-linecap:square;stroke-miterlimit:4;stroke-opacity:1;stroke-dasharray:none"
+ id="path29161-6"
+ sodipodi:cx="371"
+ sodipodi:cy="64.5"
+ sodipodi:rx="17"
+ sodipodi:ry="15.5"
+ d="m 388,64.5 a 17,15.5 0 1 1 -34,0 17,15.5 0 1 1 34,0 z" />
+ <g
+ id="g29167-46"
+ transform="matrix(0.73232502,0,0,0.75477602,116.59334,137.48829)">
+ <text
+ id="text29163-1"
+ y="70"
+ x="365"
+ style="font-style:normal;font-weight:normal;line-height:0%;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none"
+ xml:space="preserve"><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="70"
+ x="365"
+ id="tspan29165-5"
+ sodipodi:role="line">3</tspan></text>
+ </g>
+ <g
+ id="g29167-6"
+ transform="matrix(0.73232502,0,0,0.75477602,0.1536639,325.60918)">
+ <text
+ id="text29163-17"
+ y="70"
+ x="365"
+ style="font-style:normal;font-weight:normal;line-height:0%;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none"
+ xml:space="preserve"><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="70"
+ x="365"
+ id="tspan29165-2"
+ sodipodi:role="line">4</tspan></text>
+ </g>
+ <g
+ id="g29167"
+ transform="matrix(0.73232502,0,0,0.75477602,-146.07919,111.10621)">
+ <text
+ id="text29163"
+ y="70"
+ x="365"
+ style="font-style:normal;font-weight:normal;line-height:0%;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none"
+ xml:space="preserve"><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="70"
+ x="365"
+ id="tspan29165"
+ sodipodi:role="line">6</tspan></text>
+ </g>
+ <g
+ id="g29167-4-3"
+ transform="matrix(0.73232502,0,0,0.75477602,-117.60784,180.11779)">
+ <text
+ id="text29163-9-6"
+ y="70"
+ x="321.30356"
+ style="font-style:normal;font-weight:normal;line-height:0%;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none"
+ xml:space="preserve"><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:24.2109px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="70"
+ x="321.30356"
+ id="tspan29165-9-7"
+ sodipodi:role="line">Eventdev</tspan></text>
+ </g>
+ <g
+ id="g29167-4-3-5"
+ transform="matrix(0.73232502,0,0,0.75477602,55.34956,26.97821)">
+ <text
+ id="text29163-9-6-3"
+ y="70"
+ x="454.74152"
+ style="font-style:normal;font-weight:normal;line-height:0%;font-family:sans-serif;text-align:center;letter-spacing:0px;word-spacing:0px;text-anchor:middle;fill:#000000;fill-opacity:1;stroke:none"
+ xml:space="preserve"><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:21.5208px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:center;writing-mode:lr-tb;text-anchor:middle"
+ y="70"
+ x="454.74152"
+ id="tspan29165-9-7-5"
+ sodipodi:role="line">A<tspan
+ style="line-height:100%;font-family:sans-serif"
+ id="tspan3374">tomic Stage</tspan></tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:21.5208px;line-height:100%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:center;writing-mode:lr-tb;text-anchor:middle"
+ y="94.210899"
+ x="454.74152"
+ sodipodi:role="line"
+ id="tspan3320">+</tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:21.5208px;line-height:100%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:center;writing-mode:lr-tb;text-anchor:middle"
+ y="115.7317"
+ x="454.74152"
+ sodipodi:role="line"
+ id="tspan3322">Enqueue to</tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:21.5208px;line-height:100%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:center;writing-mode:lr-tb;text-anchor:middle"
+ y="137.2525"
+ x="454.74152"
+ sodipodi:role="line"
+ id="tspan3324">DMA Device</tspan></text>
+ </g>
+ <g
+ id="g29167-2"
+ transform="matrix(0.73232502,0,0,0.75477602,-116.28602,248.01695)">
+ <text
+ id="text29163-92"
+ y="70"
+ x="365"
+ style="font-style:normal;font-weight:normal;line-height:0%;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none"
+ xml:space="preserve"><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="70"
+ x="365"
+ id="tspan29165-8"
+ sodipodi:role="line">5</tspan></text>
+ </g>
+ <flowRoot
+ xml:space="preserve"
+ id="flowRoot3376"
+ style="font-style:normal;font-weight:normal;line-height:0.01%;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none;stroke-width:1px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1"><flowRegion
+ id="flowRegion3378"
+ style="font-family:sans-serif"><rect
+ id="rect3380"
+ width="100"
+ height="37"
+ x="109"
+ y="259" /></flowRegion><flowPara
+ id="flowPara3382"
+ style="font-size:18px;line-height:1.25;font-family:sans-serif"> </flowPara></flowRoot>
+ <g
+ id="g29167-4-3-1"
+ transform="matrix(0.73232502,0,0,0.75477602,109.34956,323.97821)">
+ <text
+ id="text29163-9-6-8"
+ y="70"
+ x="321.30356"
+ style="font-style:normal;font-weight:normal;line-height:0%;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none"
+ xml:space="preserve"><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:24.2109px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="70"
+ x="321.30356"
+ id="tspan29165-9-7-7"
+ sodipodi:role="line">DMA Device</tspan></text>
+ </g>
+ <g
+ id="g29167-4-3-1-9"
+ transform="matrix(0.73232502,0,0,0.75477602,-114.48565,314.20704)">
+ <text
+ id="text29163-9-6-8-2"
+ y="70"
+ x="368.01718"
+ style="font-style:normal;font-weight:normal;line-height:0%;font-family:sans-serif;text-align:center;letter-spacing:0px;word-spacing:0px;text-anchor:middle;fill:#000000;fill-opacity:1;stroke:none"
+ xml:space="preserve"><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:24.2109px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:center;writing-mode:lr-tb;text-anchor:middle"
+ y="70"
+ x="368.01718"
+ id="tspan29165-9-7-7-0"
+ sodipodi:role="line">DMA</tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:24.2109px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:center;writing-mode:lr-tb;text-anchor:middle"
+ y="100.26363"
+ x="368.01718"
+ sodipodi:role="line"
+ id="tspan3488">Adapter</tspan></text>
+ </g>
+ <g
+ id="g29167-4-3-1-9-2"
+ transform="matrix(0.73232502,0,0,0.75477602,250.96804,192.62529)">
+ <text
+ id="text29163-9-6-8-2-3"
+ y="-188.35481"
+ x="318.61978"
+ style="font-style:normal;font-weight:normal;line-height:0%;font-family:sans-serif;text-align:center;letter-spacing:0px;word-spacing:0px;text-anchor:middle;fill:#000000;fill-opacity:1;stroke:none"
+ xml:space="preserve"><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:21.5208px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="-188.35481"
+ x="318.61978"
+ sodipodi:role="line"
+ id="tspan3543">1. Application dequeues</tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:21.5208px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="-161.45381"
+ x="318.61978"
+ sodipodi:role="line"
+ id="tspan3196"> events from the previous</tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:21.5208px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="-134.55281"
+ x="318.61978"
+ sodipodi:role="line"
+ id="tspan3232"> stage</tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:21.5208px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="-107.65182"
+ x="318.61978"
+ sodipodi:role="line"
+ id="tspan3519"> </tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:21.5208px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="-80.750816"
+ x="318.61978"
+ sodipodi:role="line"
+ id="tspan3551">2. Application prepares the</tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:21.5208px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="-53.849815"
+ x="318.61978"
+ sodipodi:role="line"
+ id="tspan3203"> DMA operations.</tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:21.5208px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="-26.948814"
+ x="318.61978"
+ sodipodi:role="line"
+ id="tspan3523"> </tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:21.5208px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="-0.0478158"
+ x="318.61978"
+ sodipodi:role="line"
+ id="tspan3541">3. DMA operations are</tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:21.5208px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="26.853184"
+ x="318.61978"
+ sodipodi:role="line"
+ id="tspan3207"> submitted to dmadev</tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:21.5208px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="53.754185"
+ x="318.61978"
+ sodipodi:role="line"
+ id="tspan3209"> by application.</tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:21.5208px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="80.655182"
+ x="318.61978"
+ sodipodi:role="line"
+ id="tspan3527"> </tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:21.5208px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="107.55618"
+ x="318.61978"
+ sodipodi:role="line"
+ id="tspan3547">4. DMA adapter dequeues</tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:21.5208px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="134.45718"
+ x="318.61978"
+ sodipodi:role="line"
+ id="tspan3216"> DMA completions from</tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:21.5208px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="161.35818"
+ x="318.61978"
+ sodipodi:role="line"
+ id="tspan3218"> DMA device.</tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:21.5208px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="188.25919"
+ x="318.61978"
+ sodipodi:role="line"
+ id="tspan3531"> </tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:21.5208px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="215.16019"
+ x="318.61978"
+ sodipodi:role="line"
+ id="tspan3549">5. DMA adapter enqueues</tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:21.5208px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="242.06119"
+ x="318.61978"
+ sodipodi:role="line"
+ id="tspan3222"> events to the eventdev.</tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:21.5208px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="268.96219"
+ x="318.61978"
+ sodipodi:role="line"
+ id="tspan3535"> </tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:21.5208px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="295.86319"
+ x="318.61978"
+ sodipodi:role="line"
+ id="tspan3537">6. Application dequeues from</tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:21.5208px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="322.76419"
+ x="318.61978"
+ sodipodi:role="line"
+ id="tspan3224"> eventdev and prepare for</tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:21.5208px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="349.66519"
+ x="318.61978"
+ sodipodi:role="line"
+ id="tspan3226"> further processing</tspan></text>
+ </g>
+ <g
+ style="font-size:12px;fill:none;stroke-linecap:square;stroke-miterlimit:3;overflow:visible"
+ id="shape1-1-2-4-7"
+ v:mID="1"
+ v:groupContext="shape"
+ transform="matrix(2.1604167,0,0,1.165886,90.820551,-587.97129)">
+ <title
+ id="title22-7-5-5">Square</title>
+ <desc
+ id="desc24-7-8-3">Atomic Queue #1</desc>
+ <v:userDefs>
+ <v:ud
+ v:nameU="visVersion"
+ v:val="VT0(15):26" />
+ </v:userDefs>
+ <v:textBlock
+ v:margins="rect(4,4,4,4)" />
+ <v:textRect
+ cx="30.75"
+ cy="581.25"
+ width="61.5"
+ height="61.5" />
+ <g
+ id="shadow1-2-9-5-5"
+ v:groupContext="shadow"
+ v:shadowOffsetX="0.345598"
+ v:shadowOffsetY="-1.97279"
+ v:shadowType="1"
+ transform="translate(0.345598,1.97279)"
+ class="st1"
+ style="visibility:visible">
+ <rect
+ x="0"
+ y="550.5"
+ width="61.5"
+ height="61.5"
+ class="st2"
+ id="rect27-8-7-6"
+ style="fill:#5b9bd5;fill-opacity:0.22000002;stroke:#5b9bd5;stroke-opacity:0.22000002;filter:url(#filter_2-3-91-3)" />
+ </g>
+ <g
+ id="g13515-33-2">
+ <g
+ id="g13534-8-9">
+ <rect
+ x="0"
+ y="550.5"
+ width="61.5"
+ height="61.5"
+ class="st3"
+ id="rect29-1-95-1"
+ style="fill:#5b9bd5;stroke:#c7c8c8;stroke-width:0.25" />
+ </g>
+ </g>
+ </g>
+ <g
+ id="g29167-4-3-2"
+ transform="matrix(0.73232502,0,0,0.75477602,-125.66199,44.027402)">
+ <text
+ id="text29163-9-6-7"
+ y="70"
+ x="321.30356"
+ style="font-style:normal;font-weight:normal;line-height:0%;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none"
+ xml:space="preserve"><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:24.2109px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="70"
+ x="321.30356"
+ id="tspan29165-9-7-0"
+ sodipodi:role="line">Application</tspan></text>
+ </g>
+ </g>
+</svg>
diff --git a/doc/guides/prog_guide/index.rst b/doc/guides/prog_guide/index.rst
index 52a6d9e7aa..beaa4b8869 100644
--- a/doc/guides/prog_guide/index.rst
+++ b/doc/guides/prog_guide/index.rst
@@ -60,6 +60,7 @@ Programmer's Guide
event_ethernet_tx_adapter
event_timer_adapter
event_crypto_adapter
+ event_dma_adapter
qos_framework
power_man
packet_classif_access_ctrl
diff --git a/doc/guides/rel_notes/release_23_11.rst b/doc/guides/rel_notes/release_23_11.rst
index b34ddc0860..1a1f337d23 100644
--- a/doc/guides/rel_notes/release_23_11.rst
+++ b/doc/guides/rel_notes/release_23_11.rst
@@ -89,6 +89,11 @@ New Features
* Added support for ``remaining_ticks_get`` timer adapter PMD callback
to get the remaining ticks to expire for a given event timer.
+* **Added event DMA adapter library.**
+
+ * Added the Event DMA Adapter Library. This library extends the event-based
+ model by introducing APIs that allow applications to enqueue/dequeue DMA
+ operations to/from dmadev as events scheduled by an event device.
Removed Items
-------------
diff --git a/lib/eventdev/eventdev_pmd.h b/lib/eventdev/eventdev_pmd.h
index f62f42e140..f7227c0bfd 100644
--- a/lib/eventdev/eventdev_pmd.h
+++ b/lib/eventdev/eventdev_pmd.h
@@ -178,8 +178,12 @@ struct rte_eventdev {
event_tx_adapter_enqueue_t txa_enqueue;
/**< Pointer to PMD eth Tx adapter enqueue function. */
event_crypto_adapter_enqueue_t ca_enqueue;
+ /**< Pointer to PMD crypto adapter enqueue function. */
- uint64_t reserved_64s[4]; /**< Reserved for future fields */
+ event_dma_adapter_enqueue_t dma_enqueue;
+ /**< Pointer to PMD DMA adapter enqueue function. */
+
+ uint64_t reserved_64s[3]; /**< Reserved for future fields */
void *reserved_ptrs[3]; /**< Reserved for future fields */
} __rte_cache_aligned;
@@ -1320,6 +1324,156 @@ typedef int (*eventdev_eth_tx_adapter_queue_stop)
#define eventdev_stop_flush_t rte_eventdev_stop_flush_t
+/**
+ * Retrieve the event device's DMA adapter capabilities for the
+ * specified DMA device
+ *
+ * @param dev
+ * Event device pointer
+ *
+ * @param dma_dev_id
+ * DMA device identifier
+ *
+ * @param[out] caps
+ * A pointer to memory filled with event adapter capabilities.
+ * It is expected to be pre-allocated & initialized by caller.
+ *
+ * @return
+ * - 0: Success, driver provides event adapter capabilities for the
+ * dmadev.
+ * - <0: Error code returned by the driver function.
+ *
+ */
+typedef int (*eventdev_dma_adapter_caps_get_t)(const struct rte_eventdev *dev,
+ const int16_t dma_dev_id, uint32_t *caps);
+
+/**
+ * Add DMA vchan queue to event device. This callback is invoked if
+ * the caps returned from rte_event_dma_adapter_caps_get(, dmadev_id)
+ * has RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_* set.
+ *
+ * @param dev
+ * Event device pointer
+ *
+ * @param dma_dev_id
+ * DMA device identifier
+ *
+ * @param vchan_id
+ * dmadev vchan queue identifier.
+ *
+ * @param event
+ * Event information required for binding dmadev vchan to event queue.
+ * This structure will have a valid value for only those HW PMDs supporting
+ * @see RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_VCHAN_EV_BIND capability.
+ *
+ * @return
+ * - 0: Success, dmadev vchan added successfully.
+ * - <0: Error code returned by the driver function.
+ *
+ */
+typedef int (*eventdev_dma_adapter_vchan_add_t)(const struct rte_eventdev *dev,
+ const int16_t dma_dev_id,
+ uint16_t vchan_id,
+ const struct rte_event *event);
+
+/**
+ * Delete DMA vhcan to event device. This callback is invoked if
+ * the caps returned from rte_event_dma_adapter_caps_get(, dmadev_id)
+ * has RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_* set.
+ *
+ * @param dev
+ * Event device pointer
+ *
+ * @param dma_dev_id
+ * DMA device identifier
+ *
+ * @param vchan_id
+ * dmadev vchan identifier.
+ *
+ * @return
+ * - 0: Success, dmadev vchan deleted successfully.
+ * - <0: Error code returned by the driver function.
+ *
+ */
+typedef int (*eventdev_dma_adapter_vchan_del_t)(const struct rte_eventdev *dev,
+ const int16_t dma_dev_id,
+ uint16_t vchan_id);
+
+/**
+ * Start DMA adapter. This callback is invoked if
+ * the caps returned from rte_event_dma_adapter_caps_get(.., dmadev_id)
+ * has RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_* set and vchan for dmadev_id
+ * have been added to the event device.
+ *
+ * @param dev
+ * Event device pointer
+ *
+ * @param dma_dev_id
+ * DMA device identifier
+ *
+ * @return
+ * - 0: Success, DMA adapter started successfully.
+ * - <0: Error code returned by the driver function.
+ */
+typedef int (*eventdev_dma_adapter_start_t)(const struct rte_eventdev *dev,
+ const int16_t dma_dev_id);
+
+/**
+ * Stop DMA adapter. This callback is invoked if
+ * the caps returned from rte_event_dma_adapter_caps_get(.., dmadev_id)
+ * has RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_* set and vchan for dmadev_id
+ * have been added to the event device.
+ *
+ * @param dev
+ * Event device pointer
+ *
+ * @param dma_dev_id
+ * DMA device identifier
+ *
+ * @return
+ * - 0: Success, DMA adapter stopped successfully.
+ * - <0: Error code returned by the driver function.
+ */
+typedef int (*eventdev_dma_adapter_stop_t)(const struct rte_eventdev *dev,
+ const int16_t dma_dev_id);
+
+struct rte_event_dma_adapter_stats;
+
+/**
+ * Retrieve DMA adapter statistics.
+ *
+ * @param dev
+ * Event device pointer
+ *
+ * @param dma_dev_id
+ * DMA device identifier
+ *
+ * @param[out] stats
+ * Pointer to stats structure
+ *
+ * @return
+ * Return 0 on success.
+ */
+typedef int (*eventdev_dma_adapter_stats_get)(const struct rte_eventdev *dev,
+ const int16_t dma_dev_id,
+ struct rte_event_dma_adapter_stats *stats);
+
+/**
+ * Reset DMA adapter statistics.
+ *
+ * @param dev
+ * Event device pointer
+ *
+ * @param dma_dev_id
+ * DMA device identifier
+ *
+ * @return
+ * Return 0 on success.
+ */
+typedef int (*eventdev_dma_adapter_stats_reset)(const struct rte_eventdev *dev,
+ const int16_t dma_dev_id);
+
+
/** Event device operations function pointer table */
struct eventdev_ops {
eventdev_info_get_t dev_infos_get; /**< Get device info. */
@@ -1440,6 +1594,21 @@ struct eventdev_ops {
eventdev_eth_tx_adapter_queue_stop eth_tx_adapter_queue_stop;
/**< Stop Tx queue assigned to Tx adapter instance */
+ eventdev_dma_adapter_caps_get_t dma_adapter_caps_get;
+ /**< Get DMA adapter capabilities */
+ eventdev_dma_adapter_vchan_add_t dma_adapter_vchan_add;
+ /**< Add vchan queue to DMA adapter */
+ eventdev_dma_adapter_vchan_del_t dma_adapter_vchan_del;
+ /**< Delete vchan queue from DMA adapter */
+ eventdev_dma_adapter_start_t dma_adapter_start;
+ /**< Start DMA adapter */
+ eventdev_dma_adapter_stop_t dma_adapter_stop;
+ /**< Stop DMA adapter */
+ eventdev_dma_adapter_stats_get dma_adapter_stats_get;
+ /**< Get DMA stats */
+ eventdev_dma_adapter_stats_reset dma_adapter_stats_reset;
+ /**< Reset DMA stats */
+
eventdev_selftest dev_selftest;
/**< Start eventdev Selftest */
diff --git a/lib/eventdev/eventdev_private.c b/lib/eventdev/eventdev_private.c
index 1d3d9d357e..18ed8bf3c8 100644
--- a/lib/eventdev/eventdev_private.c
+++ b/lib/eventdev/eventdev_private.c
@@ -81,6 +81,14 @@ dummy_event_crypto_adapter_enqueue(__rte_unused void *port,
return 0;
}
+static uint16_t
+dummy_event_dma_adapter_enqueue(__rte_unused void *port, __rte_unused struct rte_event ev[],
+ __rte_unused uint16_t nb_events)
+{
+ RTE_EDEV_LOG_ERR("event DMA adapter enqueue requested for unconfigured event device");
+ return 0;
+}
+
void
event_dev_fp_ops_reset(struct rte_event_fp_ops *fp_op)
{
@@ -97,6 +105,7 @@ event_dev_fp_ops_reset(struct rte_event_fp_ops *fp_op)
.txa_enqueue_same_dest =
dummy_event_tx_adapter_enqueue_same_dest,
.ca_enqueue = dummy_event_crypto_adapter_enqueue,
+ .dma_enqueue = dummy_event_dma_adapter_enqueue,
.data = dummy_data,
};
@@ -117,5 +126,6 @@ event_dev_fp_ops_set(struct rte_event_fp_ops *fp_op,
fp_op->txa_enqueue = dev->txa_enqueue;
fp_op->txa_enqueue_same_dest = dev->txa_enqueue_same_dest;
fp_op->ca_enqueue = dev->ca_enqueue;
+ fp_op->dma_enqueue = dev->dma_enqueue;
fp_op->data = dev->data->ports;
}
diff --git a/lib/eventdev/meson.build b/lib/eventdev/meson.build
index 6edf98dfa5..21347f7c4c 100644
--- a/lib/eventdev/meson.build
+++ b/lib/eventdev/meson.build
@@ -25,6 +25,7 @@ sources = files(
)
headers = files(
'rte_event_crypto_adapter.h',
+ 'rte_event_dma_adapter.h',
'rte_event_eth_rx_adapter.h',
'rte_event_eth_tx_adapter.h',
'rte_event_ring.h',
diff --git a/lib/eventdev/rte_event_dma_adapter.h b/lib/eventdev/rte_event_dma_adapter.h
new file mode 100644
index 0000000000..e924ab673d
--- /dev/null
+++ b/lib/eventdev/rte_event_dma_adapter.h
@@ -0,0 +1,581 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright (c) 2023 Marvell.
+ */
+
+#ifndef RTE_EVENT_DMA_ADAPTER
+#define RTE_EVENT_DMA_ADAPTER
+
+/**
+ * @file rte_event_dma_adapter.h
+ *
+ * @warning
+ * @b EXPERIMENTAL:
+ * All functions in this file may be changed or removed without prior notice.
+ *
+ * DMA Event Adapter API.
+ *
+ * Eventdev library provides adapters to bridge between various components for providing new
+ * event source. The event DMA adapter is one of those adapters which is intended to bridge
+ * between event devices and DMA devices.
+ *
+ * The DMA adapter adds support to enqueue / dequeue DMA operations to / from event device. The
+ * packet flow between DMA device and the event device can be accomplished using both SW and HW
+ * based transfer mechanisms. The adapter uses an EAL service core function for SW based packet
+ * transfer and uses the eventdev PMD functions to configure HW based packet transfer between the
+ * DMA device and the event device.
+ *
+ * The application can choose to submit a DMA operation directly to an DMA device or send it to the
+ * DMA adapter via eventdev based on RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP_FWD capability. The
+ * first mode is known as the event new (RTE_EVENT_DMA_ADAPTER_OP_NEW) mode and the second as the
+ * event forward (RTE_EVENT_DMA_ADAPTER_OP_FORWARD) mode. The choice of mode can be specified while
+ * creating the adapter. In the former mode, it is an application responsibility to enable ingress
+ * packet ordering. In the latter mode, it is the adapter responsibility to enable the ingress
+ * packet ordering.
+ *
+ *
+ * Working model of RTE_EVENT_DMA_ADAPTER_OP_NEW mode:
+ *
+ * +--------------+ +--------------+
+ * | | | DMA stage |
+ * | Application |---[2]-->| + enqueue to |
+ * | | | dmadev |
+ * +--------------+ +--------------+
+ * ^ ^ |
+ * | | [3]
+ * [6] [1] |
+ * | | |
+ * +--------------+ |
+ * | | |
+ * | Event device | |
+ * | | |
+ * +--------------+ |
+ * ^ |
+ * | |
+ * [5] |
+ * | v
+ * +--------------+ +--------------+
+ * | | | |
+ * | DMA adapter |<--[4]---| dmadev |
+ * | | | |
+ * +--------------+ +--------------+
+ *
+ *
+ * [1] Application dequeues events from the previous stage.
+ * [2] Application prepares the DMA operations.
+ * [3] DMA operations are submitted to dmadev by application.
+ * [4] DMA adapter dequeues DMA completions from dmadev.
+ * [5] DMA adapter enqueues events to the eventdev.
+ * [6] Application dequeues from eventdev for further processing.
+ *
+ * In the RTE_EVENT_DMA_ADAPTER_OP_NEW mode, application submits DMA operations directly to DMA
+ * device. The DMA adapter then dequeues DMA completions from DMA device and enqueue events to the
+ * event device. This mode does not ensure ingress ordering, if the application directly enqueues
+ * to dmadev without going through DMA / atomic stage i.e. removing item [1] and [2].
+ *
+ * Events dequeued from the adapter will be treated as new events. In this mode, application needs
+ * to specify event information (response information) which is needed to enqueue an event after the
+ * DMA operation is completed.
+ *
+ *
+ * Working model of RTE_EVENT_DMA_ADAPTER_OP_FORWARD mode:
+ *
+ * +--------------+ +--------------+
+ * --[1]-->| |---[2]-->| Application |
+ * | Event device | | in |
+ * <--[8]--| |<--[3]---| Ordered stage|
+ * +--------------+ +--------------+
+ * ^ |
+ * | [4]
+ * [7] |
+ * | v
+ * +----------------+ +--------------+
+ * | |--[5]->| |
+ * | DMA adapter | | dmadev |
+ * | |<-[6]--| |
+ * +----------------+ +--------------+
+ *
+ *
+ * [1] Events from the previous stage.
+ * [2] Application in ordered stage dequeues events from eventdev.
+ * [3] Application enqueues DMA operations as events to eventdev.
+ * [4] DMA adapter dequeues event from eventdev.
+ * [5] DMA adapter submits DMA operations to dmadev (Atomic stage).
+ * [6] DMA adapter dequeues DMA completions from dmadev
+ * [7] DMA adapter enqueues events to the eventdev
+ * [8] Events to the next stage
+ *
+ * In the event forward (RTE_EVENT_DMA_ADAPTER_OP_FORWARD) mode, if the HW supports the capability
+ * RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP_FWD, application can directly submit the DMA
+ * operations to the dmadev. If not, application retrieves the event port of the DMA adapter
+ * through the API, rte_event_DMA_adapter_event_port_get(). Then, links its event queue to this
+ * port and starts enqueuing DMA operations as events to the eventdev. The adapter then dequeues
+ * the events and submits the DMA operations to the dmadev. After the DMA completions, the adapter
+ * enqueues events to the event device.
+ *
+ * Application can use this mode, when ingress packet ordering is needed. Events dequeued from the
+ * adapter will be treated as forwarded events. In this mode, the application needs to specify the
+ * dmadev ID and queue pair ID (request information) needed to enqueue an DMA operation in addition
+ * to the event information (response information) needed to enqueue an event after the DMA
+ * operation has completed.
+ *
+ * The event DMA adapter provides common APIs to configure the packet flow from the DMA device to
+ * event devices for both SW and HW based transfers. The DMA event adapter's functions are:
+ *
+ * - rte_event_dma_adapter_create_ext()
+ * - rte_event_dma_adapter_create()
+ * - rte_event_dma_adapter_free()
+ * - rte_event_dma_adapter_vchan_add()
+ * - rte_event_dma_adapter_vchan_del()
+ * - rte_event_dma_adapter_start()
+ * - rte_event_dma_adapter_stop()
+ * - rte_event_dma_adapter_stats_get()
+ * - rte_event_dma_adapter_stats_reset()
+ *
+ * The application creates an instance using rte_event_dma_adapter_create() or
+ * rte_event_dma_adapter_create_ext().
+ *
+ * dmadev queue pair addition / deletion is done using the rte_event_dma_adapter_vchan_add() /
+ * rte_event_dma_adapter_vchan_del() APIs. If HW supports the capability
+ * RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_VCHAN_EV_BIND, event information must be passed to the
+ * add API.
+ *
+ */
+
+#include <stdint.h>
+
+#include <rte_common.h>
+#include <rte_dmadev_pmd.h>
+#include <rte_eventdev.h>
+
+#ifdef __cplusplus
+extern "C" {
+#endif
+
+/**
+ * A structure used to hold event based DMA operation entry. All the information
+ * required for a DMA transfer shall be populated in "struct rte_event_dma_adapter_op"
+ * instance.
+ */
+struct rte_event_dma_adapter_op {
+ struct rte_dma_sge *src_seg;
+ /**< Source segments. */
+ struct rte_dma_sge *dst_seg;
+ /**< Destination segments. */
+ uint16_t nb_src;
+ /**< Number of source segments. */
+ uint16_t nb_dst;
+ /**< Number of destination segments. */
+ uint64_t flags;
+ /**< Flags related to the operation.
+ * @see RTE_DMA_OP_FLAG_*
+ */
+ int16_t dma_dev_id;
+ /**< DMA device ID to be used */
+ uint16_t vchan;
+ /**< DMA vchan ID to be used */
+ struct rte_mempool *op_mp;
+ /**< Mempool from which op is allocated. */
+};
+
+/**
+ * DMA event adapter mode
+ */
+enum rte_event_dma_adapter_mode {
+ RTE_EVENT_DMA_ADAPTER_OP_NEW,
+ /**< Start the DMA adapter in event new mode.
+ * @see RTE_EVENT_OP_NEW.
+ *
+ * Application submits DMA operations to the dmadev. Adapter only dequeues the DMA
+ * completions from dmadev and enqueue events to the eventdev.
+ */
+
+ RTE_EVENT_DMA_ADAPTER_OP_FORWARD,
+ /**< Start the DMA adapter in event forward mode.
+ * @see RTE_EVENT_OP_FORWARD.
+ *
+ * Application submits DMA requests as events to the DMA adapter or DMA device based on
+ * RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP_FWD capability. DMA completions are enqueued
+ * back to the eventdev by DMA adapter.
+ */
+};
+
+/**
+ * Adapter configuration structure that the adapter configuration callback function is expected to
+ * fill out.
+ *
+ * @see rte_event_dma_adapter_conf_cb
+ */
+struct rte_event_dma_adapter_conf {
+ uint8_t event_port_id;
+ /** < Event port identifier, the adapter enqueues events to this port and dequeues DMA
+ * request events in RTE_EVENT_DMA_ADAPTER_OP_FORWARD mode.
+ */
+
+ uint32_t max_nb;
+ /**< The adapter can return early if it has processed at least max_nb DMA ops. This isn't
+ * treated as a requirement; batching may cause the adapter to process more than max_nb DMA
+ * ops.
+ */
+};
+
+/**
+ * Adapter runtime configuration parameters
+ */
+struct rte_event_dma_adapter_runtime_params {
+ uint32_t max_nb;
+ /**< The adapter can return early if it has processed at least max_nb DMA ops. This isn't
+ * treated as a requirement; batching may cause the adapter to process more than max_nb DMA
+ * ops.
+ *
+ * Callback function passed to rte_event_dma_adapter_create_ext() configures the adapter
+ * with default value of max_nb.
+ * rte_event_dma_adapter_runtime_params_set() allows to re-configure max_nb during runtime
+ * (after adding at least one queue pair)
+ *
+ * This is valid for the devices without RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP_FWD or
+ * RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP_NEW capability.
+ */
+
+ uint32_t rsvd[15];
+ /**< Reserved fields for future expansion */
+};
+
+/**
+ * Function type used for adapter configuration callback. The callback is used to fill in members of
+ * the struct rte_event_dma_adapter_conf, this callback is invoked when creating a SW service for
+ * packet transfer from dmadev vchan to the event device. The SW service is created within the
+ * function, rte_event_dma_adapter_vchan_add(), if SW based packet transfers from dmadev vchan
+ * to the event device are required.
+ *
+ * @param id
+ * Adapter identifier.
+ * @param evdev_id
+ * Event device identifier.
+ * @param conf
+ * Structure that needs to be populated by this callback.
+ * @param arg
+ * Argument to the callback. This is the same as the conf_arg passed to the
+ * rte_event_dma_adapter_create_ext().
+ */
+typedef int (*rte_event_dma_adapter_conf_cb)(uint8_t id, uint8_t evdev_id,
+ struct rte_event_dma_adapter_conf *conf, void *arg);
+
+/**
+ * A structure used to retrieve statistics for an event DMA adapter instance.
+ */
+struct rte_event_dma_adapter_stats {
+ uint64_t event_poll_count;
+ /**< Event port poll count */
+
+ uint64_t event_deq_count;
+ /**< Event dequeue count */
+
+ uint64_t dma_enq_count;
+ /**< dmadev enqueue count */
+
+ uint64_t dma_enq_fail_count;
+ /**< dmadev enqueue failed count */
+
+ uint64_t dma_deq_count;
+ /**< dmadev dequeue count */
+
+ uint64_t event_enq_count;
+ /**< Event enqueue count */
+
+ uint64_t event_enq_retry_count;
+ /**< Event enqueue retry count */
+
+ uint64_t event_enq_fail_count;
+ /**< Event enqueue fail count */
+};
+
+/**
+ * Create a new event DMA adapter with the specified identifier.
+ *
+ * @param id
+ * Adapter identifier.
+ * @param evdev_id
+ * Event device identifier.
+ * @param conf_cb
+ * Callback function that fills in members of a struct rte_event_dma_adapter_conf struct passed
+ * into it.
+ * @param mode
+ * Flag to indicate the mode of the adapter.
+ * @see rte_event_dma_adapter_mode
+ * @param conf_arg
+ * Argument that is passed to the conf_cb function.
+ *
+ * @return
+ * - 0: Success
+ * - <0: Error code on failure
+ */
+__rte_experimental
+int rte_event_dma_adapter_create_ext(uint8_t id, uint8_t evdev_id,
+ rte_event_dma_adapter_conf_cb conf_cb,
+ enum rte_event_dma_adapter_mode mode, void *conf_arg);
+
+/**
+ * Create a new event DMA adapter with the specified identifier. This function uses an internal
+ * configuration function that creates an event port. This default function reconfigures the event
+ * device with an additional event port and set up the event port using the port_config parameter
+ * passed into this function. In case the application needs more control in configuration of the
+ * service, it should use the rte_event_dma_adapter_create_ext() version.
+ *
+ * @param id
+ * Adapter identifier.
+ * @param evdev_id
+ * Event device identifier.
+ * @param port_config
+ * Argument of type *rte_event_port_conf* that is passed to the conf_cb function.
+ * @param mode
+ * Flag to indicate the mode of the adapter.
+ * @see rte_event_dma_adapter_mode
+ *
+ * @return
+ * - 0: Success
+ * - <0: Error code on failure
+ */
+__rte_experimental
+int rte_event_dma_adapter_create(uint8_t id, uint8_t evdev_id,
+ struct rte_event_port_conf *port_config,
+ enum rte_event_dma_adapter_mode mode);
+
+/**
+ * Free an event DMA adapter
+ *
+ * @param id
+ * Adapter identifier.
+ * @return
+ * - 0: Success
+ * - <0: Error code on failure, If the adapter still has queue pairs added to it, the function
+ * returns -EBUSY.
+ */
+__rte_experimental
+int rte_event_dma_adapter_free(uint8_t id);
+
+/**
+ * Retrieve the event port of an adapter.
+ *
+ * @param id
+ * Adapter identifier.
+ *
+ * @param [out] event_port_id
+ * Application links its event queue to this adapter port which is used in
+ * RTE_EVENT_DMA_ADAPTER_OP_FORWARD mode.
+ *
+ * @return
+ * - 0: Success
+ * - <0: Error code on failure.
+ */
+__rte_experimental
+int rte_event_dma_adapter_event_port_get(uint8_t id, uint8_t *event_port_id);
+
+/**
+ * Add a vchan to an event DMA adapter.
+ *
+ * @param id
+ * Adapter identifier.
+ * @param dmadev_id
+ * dmadev identifier.
+ * @param vchan
+ * DMA device vchan identifier. If vchan is set -1, adapter adds all the
+ * preconfigured vchan to the instance.
+ * @param event
+ * If HW supports dmadev vchan to event queue binding, application is expected to fill in
+ * event information, else it will be NULL.
+ * @see RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_VCHAN_EV_BIND
+ *
+ * @return
+ * - 0: Success, vchan added correctly.
+ * - <0: Error code on failure.
+ */
+__rte_experimental
+int rte_event_dma_adapter_vchan_add(uint8_t id, int16_t dmadev_id, uint16_t vchan,
+ const struct rte_event *event);
+
+/**
+ * Delete a vchan from an event DMA adapter.
+ *
+ * @param id
+ * Adapter identifier.
+ * @param dmadev_id
+ * DMA device identifier.
+ * @param vchan
+ * DMA device vchan identifier.
+ *
+ * @return
+ * - 0: Success, vchan deleted successfully.
+ * - <0: Error code on failure.
+ */
+__rte_experimental
+int rte_event_dma_adapter_vchan_del(uint8_t id, int16_t dmadev_id, uint16_t vchan);
+
+/**
+ * Retrieve the service ID of an adapter. If the adapter doesn't use a rte_service function, this
+ * function returns -ESRCH.
+ *
+ * @param id
+ * Adapter identifier.
+ * @param [out] service_id
+ * A pointer to a uint32_t, to be filled in with the service id.
+ *
+ * @return
+ * - 0: Success
+ * - <0: Error code on failure, if the adapter doesn't use a rte_service function, this function
+ * returns -ESRCH.
+ */
+__rte_experimental
+int rte_event_dma_adapter_service_id_get(uint8_t id, uint32_t *service_id);
+
+/**
+ * Start event DMA adapter
+ *
+ * @param id
+ * Adapter identifier.
+ *
+ * @return
+ * - 0: Success, adapter started successfully.
+ * - <0: Error code on failure.
+ *
+ * @note The eventdev and dmadev to which the event_dma_adapter is connected should be started
+ * before calling rte_event_dma_adapter_start().
+ */
+__rte_experimental
+int rte_event_dma_adapter_start(uint8_t id);
+
+/**
+ * Stop event DMA adapter
+ *
+ * @param id
+ * Adapter identifier.
+ *
+ * @return
+ * - 0: Success, adapter stopped successfully.
+ * - <0: Error code on failure.
+ */
+__rte_experimental
+int rte_event_dma_adapter_stop(uint8_t id);
+
+/**
+ * Initialize the adapter runtime configuration parameters
+ *
+ * @param params
+ * A pointer to structure of type struct rte_event_dma_adapter_runtime_params
+ *
+ * @return
+ * - 0: Success
+ * - <0: Error code on failure
+ */
+__rte_experimental
+int rte_event_dma_adapter_runtime_params_init(struct rte_event_dma_adapter_runtime_params *params);
+
+/**
+ * Set the adapter runtime configuration parameters
+ *
+ * @param id
+ * Adapter identifier
+ *
+ * @param params
+ * A pointer to structure of type struct rte_event_dma_adapter_runtime_params with configuration
+ * parameter values. The reserved fields of this structure must be initialized to zero and the valid
+ * fields need to be set appropriately. This struct can be initialized using
+ * rte_event_dma_adapter_runtime_params_init() API to default values or application may reset this
+ * struct and update required fields.
+ *
+ * @return
+ * - 0: Success
+ * - <0: Error code on failure
+ */
+__rte_experimental
+int rte_event_dma_adapter_runtime_params_set(uint8_t id,
+ struct rte_event_dma_adapter_runtime_params *params);
+
+/**
+ * Get the adapter runtime configuration parameters
+ *
+ * @param id
+ * Adapter identifier
+ *
+ * @param[out] params
+ * A pointer to structure of type struct rte_event_dma_adapter_runtime_params containing valid
+ * adapter parameters when return value is 0.
+ *
+ * @return
+ * - 0: Success
+ * - <0: Error code on failure
+ */
+__rte_experimental
+int rte_event_dma_adapter_runtime_params_get(uint8_t id,
+ struct rte_event_dma_adapter_runtime_params *params);
+
+/**
+ * Retrieve statistics for an adapter
+ *
+ * @param id
+ * Adapter identifier.
+ * @param [out] stats
+ * A pointer to structure used to retrieve statistics for an adapter.
+ *
+ * @return
+ * - 0: Success, retrieved successfully.
+ * - <0: Error code on failure.
+ */
+__rte_experimental
+int rte_event_dma_adapter_stats_get(uint8_t id, struct rte_event_dma_adapter_stats *stats);
+
+/**
+ * Reset statistics for an adapter.
+ *
+ * @param id
+ * Adapter identifier.
+ *
+ * @return
+ * - 0: Success, statistics reset successfully.
+ * - <0: Error code on failure.
+ */
+__rte_experimental
+int rte_event_dma_adapter_stats_reset(uint8_t id);
+
+/**
+ * Enqueue a burst of DMA operations as event objects supplied in *rte_event* structure on an event
+ * DMA adapter designated by its event *evdev_id* through the event port specified by *port_id*.
+ * This function is supported if the eventdev PMD has the
+ * #RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP_FWD capability flag set.
+ *
+ * The *nb_events* parameter is the number of event objects to enqueue that are supplied in the
+ * *ev* array of *rte_event* structure.
+ *
+ * The rte_event_dma_adapter_enqueue() function returns the number of event objects it actually
+ * enqueued. A return value equal to *nb_events* means that all event objects have been enqueued.
+ *
+ * @param evdev_id
+ * The identifier of the device.
+ * @param port_id
+ * The identifier of the event port.
+ * @param ev
+ * Points to an array of *nb_events* objects of type *rte_event* structure which contain the
+ * event object enqueue operations to be processed.
+ * @param nb_events
+ * The number of event objects to enqueue, typically number of
+ * rte_event_port_attr_get(...RTE_EVENT_PORT_ATTR_ENQ_DEPTH...) available for this port.
+ *
+ * @return
+ * The number of event objects actually enqueued on the event device. The return value can be
+ * less than the value of the *nb_events* parameter when the event devices queue is full or if
+ * invalid parameters are specified in a *rte_event*. If the return value is less than *nb_events*,
+ * the remaining events at the end of ev[] are not consumed and the caller has to take care of them,
+ * and rte_errno is set accordingly. Possible errno values include:
+ * - EINVAL: The port ID is invalid, device ID is invalid, an event's queue ID is invalid, or an
+ * event's sched type doesn't match the capabilities of the destination queue.
+ * - ENOSPC: The event port was backpressured and unable to enqueue one or more events. This
+ * error code is only applicable to closed systems.
+ */
+__rte_experimental
+uint16_t rte_event_dma_adapter_enqueue(uint8_t evdev_id, uint8_t port_id, struct rte_event ev[],
+ uint16_t nb_events);
+
+#ifdef __cplusplus
+}
+#endif
+
+#endif /* RTE_EVENT_DMA_ADAPTER */
diff --git a/lib/eventdev/rte_eventdev.h b/lib/eventdev/rte_eventdev.h
index 2ba8a7b090..41743f91b1 100644
--- a/lib/eventdev/rte_eventdev.h
+++ b/lib/eventdev/rte_eventdev.h
@@ -1197,6 +1197,8 @@ struct rte_event_vector {
*/
#define RTE_EVENT_TYPE_ETH_RX_ADAPTER 0x4
/**< The event generated from event eth Rx adapter */
+#define RTE_EVENT_TYPE_DMADEV 0x5
+/**< The event generated from dma subsystem */
#define RTE_EVENT_TYPE_VECTOR 0x8
/**< Indicates that event is a vector.
* All vector event types should be a logical OR of EVENT_TYPE_VECTOR.
@@ -1462,6 +1464,48 @@ int
rte_event_crypto_adapter_caps_get(uint8_t dev_id, uint8_t cdev_id,
uint32_t *caps);
+/* DMA adapter capability bitmap flag */
+#define RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP_NEW 0x1
+/**< Flag indicates HW is capable of generating events in
+ * RTE_EVENT_OP_NEW enqueue operation. DMADEV will send
+ * packets to the event device as new events using an
+ * internal event port.
+ */
+
+#define RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP_FWD 0x2
+/**< Flag indicates HW is capable of generating events in
+ * RTE_EVENT_OP_FORWARD enqueue operation. DMADEV will send
+ * packets to the event device as forwarded event using an
+ * internal event port.
+ */
+
+#define RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_VCHAN_EV_BIND 0x4
+/**< Flag indicates HW is capable of mapping DMA vchan to event queue. */
+
+/**
+ * Retrieve the event device's DMA adapter capabilities for the
+ * specified dmadev device
+ *
+ * @param dev_id
+ * The identifier of the device.
+ *
+ * @param dmadev_id
+ * The identifier of the dmadev device.
+ *
+ * @param[out] caps
+ * A pointer to memory filled with event adapter capabilities.
+ * It is expected to be pre-allocated & initialized by caller.
+ *
+ * @return
+ * - 0: Success, driver provides event adapter capabilities for the
+ * dmadev device.
+ * - <0: Error code returned by the driver function.
+ *
+ */
+__rte_experimental
+int
+rte_event_dma_adapter_caps_get(uint8_t dev_id, uint8_t dmadev_id, uint32_t *caps);
+
/* Ethdev Tx adapter capability bitmap flags */
#define RTE_EVENT_ETH_TX_ADAPTER_CAP_INTERNAL_PORT 0x1
/**< This flag is sent when the PMD supports a packet transmit callback
diff --git a/lib/eventdev/rte_eventdev_core.h b/lib/eventdev/rte_eventdev_core.h
index c27a52ccc0..83e8736c71 100644
--- a/lib/eventdev/rte_eventdev_core.h
+++ b/lib/eventdev/rte_eventdev_core.h
@@ -42,6 +42,10 @@ typedef uint16_t (*event_crypto_adapter_enqueue_t)(void *port,
uint16_t nb_events);
/**< @internal Enqueue burst of events on crypto adapter */
+typedef uint16_t (*event_dma_adapter_enqueue_t)(void *port, struct rte_event ev[],
+ uint16_t nb_events);
+/**< @internal Enqueue burst of events on DMA adapter */
+
struct rte_event_fp_ops {
void **data;
/**< points to array of internal port data pointers */
@@ -65,7 +69,9 @@ struct rte_event_fp_ops {
/**< PMD Tx adapter enqueue same destination function. */
event_crypto_adapter_enqueue_t ca_enqueue;
/**< PMD Crypto adapter enqueue function. */
- uintptr_t reserved[5];
+ event_dma_adapter_enqueue_t dma_enqueue;
+ /**< PMD DMA adapter enqueue function. */
+ uintptr_t reserved[4];
} __rte_cache_aligned;
extern struct rte_event_fp_ops rte_event_fp_ops[RTE_EVENT_MAX_DEVS];
diff --git a/lib/eventdev/version.map b/lib/eventdev/version.map
index 7ce09a87bb..b81eb2919c 100644
--- a/lib/eventdev/version.map
+++ b/lib/eventdev/version.map
@@ -133,6 +133,22 @@ EXPERIMENTAL {
rte_event_timer_remaining_ticks_get;
# added in 23.11
+ rte_event_dma_adapter_caps_get;
+ rte_event_dma_adapter_create;
+ rte_event_dma_adapter_create_ext;
+ rte_event_dma_adapter_enqueue;
+ rte_event_dma_adapter_event_port_get;
+ rte_event_dma_adapter_free;
+ rte_event_dma_adapter_runtime_params_get;
+ rte_event_dma_adapter_runtime_params_init;
+ rte_event_dma_adapter_runtime_params_set;
+ rte_event_dma_adapter_service_id_get;
+ rte_event_dma_adapter_start;
+ rte_event_dma_adapter_stats_get;
+ rte_event_dma_adapter_stats_reset;
+ rte_event_dma_adapter_stop;
+ rte_event_dma_adapter_vchan_add;
+ rte_event_dma_adapter_vchan_del;
rte_event_eth_rx_adapter_create_ext_with_params;
};
diff --git a/lib/meson.build b/lib/meson.build
index 53155be8e9..f3191f10b6 100644
--- a/lib/meson.build
+++ b/lib/meson.build
@@ -33,6 +33,7 @@ libraries = [
'compressdev',
'cryptodev',
'distributor',
+ 'dmadev',
'efd',
'eventdev',
'gpudev',
@@ -48,7 +49,6 @@ libraries = [
'rawdev',
'regexdev',
'mldev',
- 'dmadev',
'rib',
'reorder',
'sched',
--
2.25.1
^ permalink raw reply [flat|nested] 110+ messages in thread
* [PATCH v5 02/12] eventdev/dma: support adapter capabilities get
2023-09-28 10:36 ` [PATCH v5 00/12] event DMA adapter library support Amit Prakash Shukla
2023-09-28 10:36 ` [PATCH v5 01/12] eventdev/dma: introduce DMA adapter Amit Prakash Shukla
@ 2023-09-28 10:36 ` Amit Prakash Shukla
2023-09-28 10:36 ` [PATCH v5 03/12] eventdev/dma: support adapter create and free Amit Prakash Shukla
` (10 subsequent siblings)
12 siblings, 0 replies; 110+ messages in thread
From: Amit Prakash Shukla @ 2023-09-28 10:36 UTC (permalink / raw)
To: Jerin Jacob
Cc: dev, fengchengwen, kevin.laatz, bruce.richardson, conor.walsh,
vattunuru, g.singh, sachin.saxena, hemant.agrawal, cheng1.jiang,
ndabilpuram, anoobj, mb, Amit Prakash Shukla
Added a new eventdev API rte_event_dma_adapter_caps_get(), to get
DMA adapter capabilities supported by the driver.
Signed-off-by: Amit Prakash Shukla <amitprakashs@marvell.com>
---
lib/eventdev/meson.build | 2 +-
lib/eventdev/rte_eventdev.c | 23 +++++++++++++++++++++++
2 files changed, 24 insertions(+), 1 deletion(-)
diff --git a/lib/eventdev/meson.build b/lib/eventdev/meson.build
index 21347f7c4c..b46bbbc9aa 100644
--- a/lib/eventdev/meson.build
+++ b/lib/eventdev/meson.build
@@ -43,5 +43,5 @@ driver_sdk_headers += files(
'event_timer_adapter_pmd.h',
)
-deps += ['ring', 'ethdev', 'hash', 'mempool', 'mbuf', 'timer', 'cryptodev']
+deps += ['ring', 'ethdev', 'hash', 'mempool', 'mbuf', 'timer', 'cryptodev', 'dmadev']
deps += ['telemetry']
diff --git a/lib/eventdev/rte_eventdev.c b/lib/eventdev/rte_eventdev.c
index 6ab4524332..60509c6efb 100644
--- a/lib/eventdev/rte_eventdev.c
+++ b/lib/eventdev/rte_eventdev.c
@@ -20,6 +20,7 @@
#include <rte_errno.h>
#include <ethdev_driver.h>
#include <rte_cryptodev.h>
+#include <rte_dmadev.h>
#include <cryptodev_pmd.h>
#include <rte_telemetry.h>
@@ -224,6 +225,28 @@ rte_event_eth_tx_adapter_caps_get(uint8_t dev_id, uint16_t eth_port_id,
: 0;
}
+int
+rte_event_dma_adapter_caps_get(uint8_t dev_id, uint8_t dma_dev_id, uint32_t *caps)
+{
+ struct rte_eventdev *dev;
+
+ RTE_EVENTDEV_VALID_DEVID_OR_ERR_RET(dev_id, -EINVAL);
+ if (!rte_dma_is_valid(dma_dev_id))
+ return -EINVAL;
+
+ dev = &rte_eventdevs[dev_id];
+
+ if (caps == NULL)
+ return -EINVAL;
+
+ *caps = 0;
+
+ if (dev->dev_ops->dma_adapter_caps_get)
+ return (*dev->dev_ops->dma_adapter_caps_get)(dev, dma_dev_id, caps);
+
+ return 0;
+}
+
static inline int
event_dev_queue_config(struct rte_eventdev *dev, uint8_t nb_queues)
{
--
2.25.1
^ permalink raw reply [flat|nested] 110+ messages in thread
* [PATCH v5 03/12] eventdev/dma: support adapter create and free
2023-09-28 10:36 ` [PATCH v5 00/12] event DMA adapter library support Amit Prakash Shukla
2023-09-28 10:36 ` [PATCH v5 01/12] eventdev/dma: introduce DMA adapter Amit Prakash Shukla
2023-09-28 10:36 ` [PATCH v5 02/12] eventdev/dma: support adapter capabilities get Amit Prakash Shukla
@ 2023-09-28 10:36 ` Amit Prakash Shukla
2023-09-28 10:36 ` [PATCH v5 04/12] eventdev/dma: support for vchan add and delete Amit Prakash Shukla
` (9 subsequent siblings)
12 siblings, 0 replies; 110+ messages in thread
From: Amit Prakash Shukla @ 2023-09-28 10:36 UTC (permalink / raw)
To: Bruce Richardson, Jerin Jacob, Amit Prakash Shukla
Cc: dev, fengchengwen, kevin.laatz, conor.walsh, vattunuru, g.singh,
sachin.saxena, hemant.agrawal, cheng1.jiang, ndabilpuram, anoobj,
mb
Added API support to create and free DMA adapter. Create function shall be
called with event device to be associated with the adapter and port
configuration to setup an event port.
Signed-off-by: Amit Prakash Shukla <amitprakashs@marvell.com>
---
config/rte_config.h | 1 +
lib/eventdev/meson.build | 1 +
lib/eventdev/rte_event_dma_adapter.c | 335 +++++++++++++++++++++++++++
3 files changed, 337 insertions(+)
create mode 100644 lib/eventdev/rte_event_dma_adapter.c
diff --git a/config/rte_config.h b/config/rte_config.h
index 400e44e3cf..401727703f 100644
--- a/config/rte_config.h
+++ b/config/rte_config.h
@@ -77,6 +77,7 @@
#define RTE_EVENT_ETH_INTR_RING_SIZE 1024
#define RTE_EVENT_CRYPTO_ADAPTER_MAX_INSTANCE 32
#define RTE_EVENT_ETH_TX_ADAPTER_MAX_INSTANCE 32
+#define RTE_EVENT_DMA_ADAPTER_MAX_INSTANCE 32
/* rawdev defines */
#define RTE_RAWDEV_MAX_DEVS 64
diff --git a/lib/eventdev/meson.build b/lib/eventdev/meson.build
index b46bbbc9aa..250abcb154 100644
--- a/lib/eventdev/meson.build
+++ b/lib/eventdev/meson.build
@@ -17,6 +17,7 @@ sources = files(
'eventdev_private.c',
'eventdev_trace_points.c',
'rte_event_crypto_adapter.c',
+ 'rte_event_dma_adapter.c',
'rte_event_eth_rx_adapter.c',
'rte_event_eth_tx_adapter.c',
'rte_event_ring.c',
diff --git a/lib/eventdev/rte_event_dma_adapter.c b/lib/eventdev/rte_event_dma_adapter.c
new file mode 100644
index 0000000000..e57d8407cb
--- /dev/null
+++ b/lib/eventdev/rte_event_dma_adapter.c
@@ -0,0 +1,335 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright (c) 2023 Marvell.
+ */
+
+#include <eventdev_pmd.h>
+
+#include "rte_event_dma_adapter.h"
+
+#define DMA_BATCH_SIZE 32
+#define DMA_DEFAULT_MAX_NB 128
+#define DMA_ADAPTER_NAME_LEN 32
+#define DMA_ADAPTER_BUFFER_SIZE 1024
+
+#define DMA_ADAPTER_OPS_BUFFER_SIZE (DMA_BATCH_SIZE + DMA_BATCH_SIZE)
+
+#define DMA_ADAPTER_ARRAY "event_dma_adapter_array"
+
+/* Macros to check for valid adapter */
+#define EVENT_DMA_ADAPTER_ID_VALID_OR_ERR_RET(id, retval) \
+ do { \
+ if (!edma_adapter_valid_id(id)) { \
+ RTE_EDEV_LOG_ERR("Invalid DMA adapter id = %d\n", id); \
+ return retval; \
+ } \
+ } while (0)
+
+/* DMA ops circular buffer */
+struct dma_ops_circular_buffer {
+ /* Index of head element */
+ uint16_t head;
+
+ /* Index of tail element */
+ uint16_t tail;
+
+ /* Number of elements in buffer */
+ uint16_t count;
+
+ /* Size of circular buffer */
+ uint16_t size;
+
+ /* Pointer to hold rte_event_dma_adapter_op for processing */
+ struct rte_event_dma_adapter_op **op_buffer;
+} __rte_cache_aligned;
+
+/* DMA device information */
+struct dma_device_info {
+ /* Number of vchans configured for a DMA device. */
+ uint16_t num_dma_dev_vchan;
+} __rte_cache_aligned;
+
+struct event_dma_adapter {
+ /* Event device identifier */
+ uint8_t eventdev_id;
+
+ /* Event port identifier */
+ uint8_t event_port_id;
+
+ /* Adapter mode */
+ enum rte_event_dma_adapter_mode mode;
+
+ /* Memory allocation name */
+ char mem_name[DMA_ADAPTER_NAME_LEN];
+
+ /* Socket identifier cached from eventdev */
+ int socket_id;
+
+ /* Lock to serialize config updates with service function */
+ rte_spinlock_t lock;
+
+ /* DMA device structure array */
+ struct dma_device_info *dma_devs;
+
+ /* Circular buffer for processing DMA ops to eventdev */
+ struct dma_ops_circular_buffer ebuf;
+
+ /* Configuration callback for rte_service configuration */
+ rte_event_dma_adapter_conf_cb conf_cb;
+
+ /* Configuration callback argument */
+ void *conf_arg;
+
+ /* Set if default_cb is being used */
+ int default_cb_arg;
+} __rte_cache_aligned;
+
+static struct event_dma_adapter **event_dma_adapter;
+
+static inline int
+edma_adapter_valid_id(uint8_t id)
+{
+ return id < RTE_EVENT_DMA_ADAPTER_MAX_INSTANCE;
+}
+
+static inline struct event_dma_adapter *
+edma_id_to_adapter(uint8_t id)
+{
+ return event_dma_adapter ? event_dma_adapter[id] : NULL;
+}
+
+static int
+edma_array_init(void)
+{
+ const struct rte_memzone *mz;
+ uint32_t sz;
+
+ mz = rte_memzone_lookup(DMA_ADAPTER_ARRAY);
+ if (mz == NULL) {
+ sz = sizeof(struct event_dma_adapter *) * RTE_EVENT_DMA_ADAPTER_MAX_INSTANCE;
+ sz = RTE_ALIGN(sz, RTE_CACHE_LINE_SIZE);
+
+ mz = rte_memzone_reserve_aligned(DMA_ADAPTER_ARRAY, sz, rte_socket_id(), 0,
+ RTE_CACHE_LINE_SIZE);
+ if (mz == NULL) {
+ RTE_EDEV_LOG_ERR("Failed to reserve memzone : %s, err = %d",
+ DMA_ADAPTER_ARRAY, rte_errno);
+ return -rte_errno;
+ }
+ }
+
+ event_dma_adapter = mz->addr;
+
+ return 0;
+}
+
+static inline int
+edma_circular_buffer_init(const char *name, struct dma_ops_circular_buffer *buf, uint16_t sz)
+{
+ buf->op_buffer = rte_zmalloc(name, sizeof(struct rte_event_dma_adapter_op *) * sz, 0);
+ if (buf->op_buffer == NULL)
+ return -ENOMEM;
+
+ buf->size = sz;
+
+ return 0;
+}
+
+static inline void
+edma_circular_buffer_free(struct dma_ops_circular_buffer *buf)
+{
+ rte_free(buf->op_buffer);
+}
+
+static int
+edma_default_config_cb(uint8_t id, uint8_t evdev_id, struct rte_event_dma_adapter_conf *conf,
+ void *arg)
+{
+ struct rte_event_port_conf *port_conf;
+ struct rte_event_dev_config dev_conf;
+ struct event_dma_adapter *adapter;
+ struct rte_eventdev *dev;
+ uint8_t port_id;
+ int started;
+ int ret;
+
+ adapter = edma_id_to_adapter(id);
+ if (adapter == NULL)
+ return -EINVAL;
+
+ dev = &rte_eventdevs[adapter->eventdev_id];
+ dev_conf = dev->data->dev_conf;
+
+ started = dev->data->dev_started;
+ if (started)
+ rte_event_dev_stop(evdev_id);
+
+ port_id = dev_conf.nb_event_ports;
+ dev_conf.nb_event_ports += 1;
+
+ port_conf = arg;
+ if (port_conf->event_port_cfg & RTE_EVENT_PORT_CFG_SINGLE_LINK)
+ dev_conf.nb_single_link_event_port_queues += 1;
+
+ ret = rte_event_dev_configure(evdev_id, &dev_conf);
+ if (ret) {
+ RTE_EDEV_LOG_ERR("Failed to configure event dev %u\n", evdev_id);
+ if (started) {
+ if (rte_event_dev_start(evdev_id))
+ return -EIO;
+ }
+ return ret;
+ }
+
+ ret = rte_event_port_setup(evdev_id, port_id, port_conf);
+ if (ret) {
+ RTE_EDEV_LOG_ERR("Failed to setup event port %u\n", port_id);
+ return ret;
+ }
+
+ conf->event_port_id = port_id;
+ conf->max_nb = DMA_DEFAULT_MAX_NB;
+ if (started)
+ ret = rte_event_dev_start(evdev_id);
+
+ adapter->default_cb_arg = 1;
+ adapter->event_port_id = conf->event_port_id;
+
+ return ret;
+}
+
+int
+rte_event_dma_adapter_create_ext(uint8_t id, uint8_t evdev_id,
+ rte_event_dma_adapter_conf_cb conf_cb,
+ enum rte_event_dma_adapter_mode mode, void *conf_arg)
+{
+ struct rte_event_dev_info dev_info;
+ struct event_dma_adapter *adapter;
+ char name[DMA_ADAPTER_NAME_LEN];
+ struct rte_dma_info info;
+ uint16_t num_dma_dev;
+ int socket_id;
+ uint8_t i;
+ int ret;
+
+ EVENT_DMA_ADAPTER_ID_VALID_OR_ERR_RET(id, -EINVAL);
+ RTE_EVENTDEV_VALID_DEVID_OR_ERR_RET(evdev_id, -EINVAL);
+
+ if (conf_cb == NULL)
+ return -EINVAL;
+
+ if (event_dma_adapter == NULL) {
+ ret = edma_array_init();
+ if (ret)
+ return ret;
+ }
+
+ adapter = edma_id_to_adapter(id);
+ if (adapter != NULL) {
+ RTE_EDEV_LOG_ERR("ML adapter ID %d already exists!", id);
+ return -EEXIST;
+ }
+
+ socket_id = rte_event_dev_socket_id(evdev_id);
+ snprintf(name, DMA_ADAPTER_NAME_LEN, "rte_event_dma_adapter_%d", id);
+ adapter = rte_zmalloc_socket(name, sizeof(struct event_dma_adapter), RTE_CACHE_LINE_SIZE,
+ socket_id);
+ if (adapter == NULL) {
+ RTE_EDEV_LOG_ERR("Failed to get mem for event ML adapter!");
+ return -ENOMEM;
+ }
+
+ if (edma_circular_buffer_init("edma_circular_buffer", &adapter->ebuf,
+ DMA_ADAPTER_BUFFER_SIZE)) {
+ RTE_EDEV_LOG_ERR("Failed to get memory for event adapter circular buffer");
+ rte_free(adapter);
+ return -ENOMEM;
+ }
+
+ ret = rte_event_dev_info_get(evdev_id, &dev_info);
+ if (ret < 0) {
+ RTE_EDEV_LOG_ERR("Failed to get info for eventdev %d: %s", evdev_id,
+ dev_info.driver_name);
+ edma_circular_buffer_free(&adapter->ebuf);
+ rte_free(adapter);
+ return ret;
+ }
+
+ num_dma_dev = rte_dma_count_avail();
+
+ adapter->eventdev_id = evdev_id;
+ adapter->mode = mode;
+ strcpy(adapter->mem_name, name);
+ adapter->socket_id = socket_id;
+ adapter->conf_cb = conf_cb;
+ adapter->conf_arg = conf_arg;
+ adapter->dma_devs = rte_zmalloc_socket(adapter->mem_name,
+ num_dma_dev * sizeof(struct dma_device_info), 0,
+ socket_id);
+ if (adapter->dma_devs == NULL) {
+ RTE_EDEV_LOG_ERR("Failed to get memory for DMA devices\n");
+ edma_circular_buffer_free(&adapter->ebuf);
+ rte_free(adapter);
+ return -ENOMEM;
+ }
+
+ rte_spinlock_init(&adapter->lock);
+ for (i = 0; i < num_dma_dev; i++) {
+ ret = rte_dma_info_get(i, &info);
+ if (ret) {
+ RTE_EDEV_LOG_ERR("Failed to get dma device info\n");
+ edma_circular_buffer_free(&adapter->ebuf);
+ rte_free(adapter);
+ return ret;
+ }
+
+ adapter->dma_devs[i].num_dma_dev_vchan = info.nb_vchans;
+ }
+
+ event_dma_adapter[id] = adapter;
+
+ return 0;
+}
+
+int
+rte_event_dma_adapter_create(uint8_t id, uint8_t evdev_id, struct rte_event_port_conf *port_config,
+ enum rte_event_dma_adapter_mode mode)
+{
+ struct rte_event_port_conf *pc;
+ int ret;
+
+ if (port_config == NULL)
+ return -EINVAL;
+
+ EVENT_DMA_ADAPTER_ID_VALID_OR_ERR_RET(id, -EINVAL);
+
+ pc = rte_malloc(NULL, sizeof(struct rte_event_port_conf), 0);
+ if (pc == NULL)
+ return -ENOMEM;
+
+ rte_memcpy(pc, port_config, sizeof(struct rte_event_port_conf));
+ ret = rte_event_dma_adapter_create_ext(id, evdev_id, edma_default_config_cb, mode, pc);
+ if (ret != 0)
+ rte_free(pc);
+
+ return ret;
+}
+
+int
+rte_event_dma_adapter_free(uint8_t id)
+{
+ struct event_dma_adapter *adapter;
+
+ EVENT_DMA_ADAPTER_ID_VALID_OR_ERR_RET(id, -EINVAL);
+
+ adapter = edma_id_to_adapter(id);
+ if (adapter == NULL)
+ return -EINVAL;
+
+ rte_free(adapter->conf_arg);
+ rte_free(adapter->dma_devs);
+ edma_circular_buffer_free(&adapter->ebuf);
+ rte_free(adapter);
+ event_dma_adapter[id] = NULL;
+
+ return 0;
+}
--
2.25.1
^ permalink raw reply [flat|nested] 110+ messages in thread
* [PATCH v5 04/12] eventdev/dma: support for vchan add and delete
2023-09-28 10:36 ` [PATCH v5 00/12] event DMA adapter library support Amit Prakash Shukla
` (2 preceding siblings ...)
2023-09-28 10:36 ` [PATCH v5 03/12] eventdev/dma: support adapter create and free Amit Prakash Shukla
@ 2023-09-28 10:36 ` Amit Prakash Shukla
2023-09-28 10:36 ` [PATCH v5 05/12] eventdev/dma: support for adapter service function Amit Prakash Shukla
` (8 subsequent siblings)
12 siblings, 0 replies; 110+ messages in thread
From: Amit Prakash Shukla @ 2023-09-28 10:36 UTC (permalink / raw)
To: Amit Prakash Shukla, Jerin Jacob
Cc: dev, fengchengwen, kevin.laatz, bruce.richardson, conor.walsh,
vattunuru, g.singh, sachin.saxena, hemant.agrawal, cheng1.jiang,
ndabilpuram, anoobj, mb
Added API support to add and delete vchan's from the DMA adapter. DMA devid
and vchan are added to the addapter instance by calling
rte_event_dma_adapter_vchan_add and deleted using
rte_event_dma_adapter_vchan_del.
Signed-off-by: Amit Prakash Shukla <amitprakashs@marvell.com>
---
lib/eventdev/rte_event_dma_adapter.c | 204 +++++++++++++++++++++++++++
1 file changed, 204 insertions(+)
diff --git a/lib/eventdev/rte_event_dma_adapter.c b/lib/eventdev/rte_event_dma_adapter.c
index e57d8407cb..ec81281bf8 100644
--- a/lib/eventdev/rte_event_dma_adapter.c
+++ b/lib/eventdev/rte_event_dma_adapter.c
@@ -42,8 +42,31 @@ struct dma_ops_circular_buffer {
struct rte_event_dma_adapter_op **op_buffer;
} __rte_cache_aligned;
+/* Vchan information */
+struct dma_vchan_info {
+ /* Set to indicate vchan queue is enabled */
+ bool vq_enabled;
+
+ /* Circular buffer for batching DMA ops to dma_dev */
+ struct dma_ops_circular_buffer dma_buf;
+} __rte_cache_aligned;
+
/* DMA device information */
struct dma_device_info {
+ /* Pointer to vchan queue info */
+ struct dma_vchan_info *vchanq;
+
+ /* Pointer to vchan queue info.
+ * This holds ops passed by application till the
+ * dma completion is done.
+ */
+ struct dma_vchan_info *tqmap;
+
+ /* If num_vchanq > 0, the start callback will
+ * be invoked if not already invoked
+ */
+ uint16_t num_vchanq;
+
/* Number of vchans configured for a DMA device. */
uint16_t num_dma_dev_vchan;
} __rte_cache_aligned;
@@ -81,6 +104,9 @@ struct event_dma_adapter {
/* Set if default_cb is being used */
int default_cb_arg;
+
+ /* No. of vchan queue configured */
+ uint16_t nb_vchanq;
} __rte_cache_aligned;
static struct event_dma_adapter **event_dma_adapter;
@@ -333,3 +359,181 @@ rte_event_dma_adapter_free(uint8_t id)
return 0;
}
+
+static void
+edma_update_vchanq_info(struct event_dma_adapter *adapter, struct dma_device_info *dev_info,
+ uint16_t vchan, uint8_t add)
+{
+ struct dma_vchan_info *vchan_info;
+ struct dma_vchan_info *tqmap_info;
+ int enabled;
+ uint16_t i;
+
+ if (dev_info->vchanq == NULL)
+ return;
+
+ if (vchan == RTE_DMA_ALL_VCHAN) {
+ for (i = 0; i < dev_info->num_dma_dev_vchan; i++)
+ edma_update_vchanq_info(adapter, dev_info, i, add);
+ } else {
+ tqmap_info = &dev_info->tqmap[vchan];
+ vchan_info = &dev_info->vchanq[vchan];
+ enabled = vchan_info->vq_enabled;
+ if (add) {
+ adapter->nb_vchanq += !enabled;
+ dev_info->num_vchanq += !enabled;
+ } else {
+ adapter->nb_vchanq -= enabled;
+ dev_info->num_vchanq -= enabled;
+ }
+ vchan_info->vq_enabled = !!add;
+ tqmap_info->vq_enabled = !!add;
+ }
+}
+
+int
+rte_event_dma_adapter_vchan_add(uint8_t id, int16_t dma_dev_id, uint16_t vchan,
+ const struct rte_event *event)
+{
+ struct event_dma_adapter *adapter;
+ struct dma_device_info *dev_info;
+ struct rte_eventdev *dev;
+ uint32_t cap;
+ int ret;
+
+ EVENT_DMA_ADAPTER_ID_VALID_OR_ERR_RET(id, -EINVAL);
+
+ if (!rte_dma_is_valid(dma_dev_id)) {
+ RTE_EDEV_LOG_ERR("Invalid dma_dev_id = %" PRIu8, dma_dev_id);
+ return -EINVAL;
+ }
+
+ adapter = edma_id_to_adapter(id);
+ if (adapter == NULL)
+ return -EINVAL;
+
+ dev = &rte_eventdevs[adapter->eventdev_id];
+ ret = rte_event_dma_adapter_caps_get(adapter->eventdev_id, dma_dev_id, &cap);
+ if (ret) {
+ RTE_EDEV_LOG_ERR("Failed to get adapter caps dev %u dma_dev %u", id, dma_dev_id);
+ return ret;
+ }
+
+ if ((cap & RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_VCHAN_EV_BIND) && (event == NULL)) {
+ RTE_EDEV_LOG_ERR("Event can not be NULL for dma_dev_id = %u", dma_dev_id);
+ return -EINVAL;
+ }
+
+ dev_info = &adapter->dma_devs[dma_dev_id];
+ if (vchan != RTE_DMA_ALL_VCHAN && vchan >= dev_info->num_dma_dev_vchan) {
+ RTE_EDEV_LOG_ERR("Invalid vhcan %u", vchan);
+ return -EINVAL;
+ }
+
+ /* In case HW cap is RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP_FWD, no
+ * need of service core as HW supports event forward capability.
+ */
+ if ((cap & RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP_FWD) ||
+ (cap & RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_VCHAN_EV_BIND &&
+ adapter->mode == RTE_EVENT_DMA_ADAPTER_OP_NEW) ||
+ (cap & RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP_NEW &&
+ adapter->mode == RTE_EVENT_DMA_ADAPTER_OP_NEW)) {
+ if (*dev->dev_ops->dma_adapter_vchan_add == NULL)
+ return -ENOTSUP;
+ if (dev_info->vchanq == NULL) {
+ dev_info->vchanq = rte_zmalloc_socket(adapter->mem_name,
+ dev_info->num_dma_dev_vchan *
+ sizeof(struct dma_vchan_info),
+ 0, adapter->socket_id);
+ if (dev_info->vchanq == NULL) {
+ printf("Queue pair add not supported\n");
+ return -ENOMEM;
+ }
+ }
+
+ if (dev_info->tqmap == NULL) {
+ dev_info->tqmap = rte_zmalloc_socket(adapter->mem_name,
+ dev_info->num_dma_dev_vchan *
+ sizeof(struct dma_vchan_info),
+ 0, adapter->socket_id);
+ if (dev_info->tqmap == NULL) {
+ printf("tq pair add not supported\n");
+ return -ENOMEM;
+ }
+ }
+
+ ret = (*dev->dev_ops->dma_adapter_vchan_add)(dev, dma_dev_id, vchan, event);
+ if (ret)
+ return ret;
+
+ else
+ edma_update_vchanq_info(adapter, &adapter->dma_devs[dma_dev_id], vchan, 1);
+ }
+
+ return 0;
+}
+
+int
+rte_event_dma_adapter_vchan_del(uint8_t id, int16_t dma_dev_id, uint16_t vchan)
+{
+ struct event_dma_adapter *adapter;
+ struct dma_device_info *dev_info;
+ struct rte_eventdev *dev;
+ uint32_t cap;
+ int ret;
+
+ EVENT_DMA_ADAPTER_ID_VALID_OR_ERR_RET(id, -EINVAL);
+
+ if (!rte_dma_is_valid(dma_dev_id)) {
+ RTE_EDEV_LOG_ERR("Invalid dma_dev_id = %" PRIu8, dma_dev_id);
+ return -EINVAL;
+ }
+
+ adapter = edma_id_to_adapter(id);
+ if (adapter == NULL)
+ return -EINVAL;
+
+ dev = &rte_eventdevs[adapter->eventdev_id];
+ ret = rte_event_dma_adapter_caps_get(adapter->eventdev_id, dma_dev_id, &cap);
+ if (ret)
+ return ret;
+
+ dev_info = &adapter->dma_devs[dma_dev_id];
+
+ if (vchan != RTE_DMA_ALL_VCHAN && vchan >= dev_info->num_dma_dev_vchan) {
+ RTE_EDEV_LOG_ERR("Invalid vhcan %" PRIu16, vchan);
+ return -EINVAL;
+ }
+
+ if ((cap & RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP_FWD) ||
+ (cap & RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP_NEW &&
+ adapter->mode == RTE_EVENT_DMA_ADAPTER_OP_NEW)) {
+ if (*dev->dev_ops->dma_adapter_vchan_del == NULL)
+ return -ENOTSUP;
+ ret = (*dev->dev_ops->dma_adapter_vchan_del)(dev, dma_dev_id, vchan);
+ if (ret == 0) {
+ edma_update_vchanq_info(adapter, dev_info, vchan, 0);
+ if (dev_info->num_vchanq == 0) {
+ rte_free(dev_info->vchanq);
+ dev_info->vchanq = NULL;
+ }
+ }
+ } else {
+ if (adapter->nb_vchanq == 0)
+ return 0;
+
+ rte_spinlock_lock(&adapter->lock);
+ edma_update_vchanq_info(adapter, dev_info, vchan, 0);
+
+ if (dev_info->num_vchanq == 0) {
+ rte_free(dev_info->vchanq);
+ rte_free(dev_info->tqmap);
+ dev_info->vchanq = NULL;
+ dev_info->tqmap = NULL;
+ }
+
+ rte_spinlock_unlock(&adapter->lock);
+ }
+
+ return ret;
+}
--
2.25.1
^ permalink raw reply [flat|nested] 110+ messages in thread
* [PATCH v5 05/12] eventdev/dma: support for adapter service function
2023-09-28 10:36 ` [PATCH v5 00/12] event DMA adapter library support Amit Prakash Shukla
` (3 preceding siblings ...)
2023-09-28 10:36 ` [PATCH v5 04/12] eventdev/dma: support for vchan add and delete Amit Prakash Shukla
@ 2023-09-28 10:36 ` Amit Prakash Shukla
2023-09-28 10:36 ` [PATCH v5 06/12] eventdev/dma: support for adapter start and stop Amit Prakash Shukla
` (7 subsequent siblings)
12 siblings, 0 replies; 110+ messages in thread
From: Amit Prakash Shukla @ 2023-09-28 10:36 UTC (permalink / raw)
To: Amit Prakash Shukla, Jerin Jacob
Cc: dev, fengchengwen, kevin.laatz, bruce.richardson, conor.walsh,
vattunuru, g.singh, sachin.saxena, hemant.agrawal, cheng1.jiang,
ndabilpuram, anoobj, mb
Added support for DMA adapter service function for event devices.
Enqueue and dequeue of event from eventdev and DMA device are done
based on the adapter mode and the supported HW capabilities.
Signed-off-by: Amit Prakash Shukla <amitprakashs@marvell.com>
---
lib/eventdev/rte_event_dma_adapter.c | 588 +++++++++++++++++++++++++++
1 file changed, 588 insertions(+)
diff --git a/lib/eventdev/rte_event_dma_adapter.c b/lib/eventdev/rte_event_dma_adapter.c
index ec81281bf8..b0845eb415 100644
--- a/lib/eventdev/rte_event_dma_adapter.c
+++ b/lib/eventdev/rte_event_dma_adapter.c
@@ -3,6 +3,7 @@
*/
#include <eventdev_pmd.h>
+#include <rte_service_component.h>
#include "rte_event_dma_adapter.h"
@@ -69,6 +70,10 @@ struct dma_device_info {
/* Number of vchans configured for a DMA device. */
uint16_t num_dma_dev_vchan;
+
+ /* Next queue pair to be processed */
+ uint16_t next_vchan_id;
+
} __rte_cache_aligned;
struct event_dma_adapter {
@@ -90,6 +95,9 @@ struct event_dma_adapter {
/* Lock to serialize config updates with service function */
rte_spinlock_t lock;
+ /* Next dma device to be processed */
+ uint16_t next_dmadev_id;
+
/* DMA device structure array */
struct dma_device_info *dma_devs;
@@ -107,6 +115,26 @@ struct event_dma_adapter {
/* No. of vchan queue configured */
uint16_t nb_vchanq;
+
+ /* Per adapter EAL service ID */
+ uint32_t service_id;
+
+ /* Service initialization state */
+ uint8_t service_initialized;
+
+ /* Max DMA ops processed in any service function invocation */
+ uint32_t max_nb;
+
+ /* Store event port's implicit release capability */
+ uint8_t implicit_release_disabled;
+
+ /* Flag to indicate backpressure at dma_dev
+ * Stop further dequeuing events from eventdev
+ */
+ bool stop_enq_to_dma_dev;
+
+ /* Loop counter to flush dma ops */
+ uint16_t transmit_loop_count;
} __rte_cache_aligned;
static struct event_dma_adapter **event_dma_adapter;
@@ -148,6 +176,18 @@ edma_array_init(void)
return 0;
}
+static inline bool
+edma_circular_buffer_batch_ready(struct dma_ops_circular_buffer *bufp)
+{
+ return bufp->count >= DMA_BATCH_SIZE;
+}
+
+static inline bool
+edma_circular_buffer_space_for_batch(struct dma_ops_circular_buffer *bufp)
+{
+ return (bufp->size - bufp->count) >= DMA_BATCH_SIZE;
+}
+
static inline int
edma_circular_buffer_init(const char *name, struct dma_ops_circular_buffer *buf, uint16_t sz)
{
@@ -166,6 +206,67 @@ edma_circular_buffer_free(struct dma_ops_circular_buffer *buf)
rte_free(buf->op_buffer);
}
+static inline int
+edma_circular_buffer_add(struct dma_ops_circular_buffer *bufp, struct rte_event_dma_adapter_op *op)
+{
+ uint16_t *tail = &bufp->tail;
+
+ bufp->op_buffer[*tail] = op;
+
+ /* circular buffer, go round */
+ *tail = (*tail + 1) % bufp->size;
+ bufp->count++;
+
+ return 0;
+}
+
+static inline int
+edma_circular_buffer_flush_to_dma_dev(struct event_dma_adapter *adapter,
+ struct dma_ops_circular_buffer *bufp, uint8_t dma_dev_id,
+ uint16_t vchan, uint16_t *nb_ops_flushed)
+{
+ struct rte_event_dma_adapter_op *op;
+ struct dma_vchan_info *tq;
+ uint16_t *head = &bufp->head;
+ uint16_t *tail = &bufp->tail;
+ uint16_t n;
+ uint16_t i;
+ int ret;
+
+ if (*tail > *head)
+ n = *tail - *head;
+ else if (*tail < *head)
+ n = bufp->size - *head;
+ else {
+ *nb_ops_flushed = 0;
+ return 0; /* buffer empty */
+ }
+
+ tq = &adapter->dma_devs[dma_dev_id].tqmap[vchan];
+
+ for (i = 0; i < n; i++) {
+ op = bufp->op_buffer[*head];
+ ret = rte_dma_copy_sg(dma_dev_id, vchan, op->src_seg, op->dst_seg,
+ op->nb_src, op->nb_dst, op->flags);
+ if (ret < 0)
+ break;
+
+ /* Enqueue in transaction queue. */
+ edma_circular_buffer_add(&tq->dma_buf, op);
+
+ *head = (*head + 1) % bufp->size;
+ }
+
+ *nb_ops_flushed = i;
+ bufp->count -= *nb_ops_flushed;
+ if (!bufp->count) {
+ *head = 0;
+ *tail = 0;
+ }
+
+ return *nb_ops_flushed == n ? 0 : -1;
+}
+
static int
edma_default_config_cb(uint8_t id, uint8_t evdev_id, struct rte_event_dma_adapter_conf *conf,
void *arg)
@@ -360,6 +461,406 @@ rte_event_dma_adapter_free(uint8_t id)
return 0;
}
+static inline unsigned int
+edma_enq_to_dma_dev(struct event_dma_adapter *adapter, struct rte_event *ev, unsigned int cnt)
+{
+ struct dma_vchan_info *vchan_qinfo = NULL;
+ struct rte_event_dma_adapter_op *dma_op;
+ uint16_t vchan, nb_enqueued = 0;
+ int16_t dma_dev_id;
+ unsigned int i, n;
+ int ret;
+
+ ret = 0;
+ n = 0;
+
+ for (i = 0; i < cnt; i++) {
+ dma_op = ev[i].event_ptr;
+ if (dma_op == NULL)
+ continue;
+
+ /* Expected to have response info appended to dma_op. */
+
+ dma_dev_id = dma_op->dma_dev_id;
+ vchan = dma_op->vchan;
+ vchan_qinfo = &adapter->dma_devs[dma_dev_id].vchanq[vchan];
+ if (!vchan_qinfo->vq_enabled) {
+ if (dma_op != NULL && dma_op->op_mp != NULL)
+ rte_mempool_put(dma_op->op_mp, dma_op);
+ continue;
+ }
+ edma_circular_buffer_add(&vchan_qinfo->dma_buf, dma_op);
+
+ if (edma_circular_buffer_batch_ready(&vchan_qinfo->dma_buf)) {
+ ret = edma_circular_buffer_flush_to_dma_dev(adapter, &vchan_qinfo->dma_buf,
+ dma_dev_id, vchan,
+ &nb_enqueued);
+ n += nb_enqueued;
+
+ /**
+ * If some dma ops failed to flush to dma_dev and
+ * space for another batch is not available, stop
+ * dequeue from eventdev momentarily
+ */
+ if (unlikely(ret < 0 &&
+ !edma_circular_buffer_space_for_batch(&vchan_qinfo->dma_buf)))
+ adapter->stop_enq_to_dma_dev = true;
+ }
+ }
+
+ return n;
+}
+
+static unsigned int
+edma_adapter_dev_flush(struct event_dma_adapter *adapter, int16_t dma_dev_id,
+ uint16_t *nb_ops_flushed)
+{
+ struct dma_vchan_info *vchan_info;
+ struct dma_device_info *dev_info;
+ uint16_t nb = 0, nb_enqueued = 0;
+ uint16_t vchan, nb_vchans;
+
+ dev_info = &adapter->dma_devs[dma_dev_id];
+ nb_vchans = dev_info->num_vchanq;
+
+ for (vchan = 0; vchan < nb_vchans; vchan++) {
+
+ vchan_info = &dev_info->vchanq[vchan];
+ if (unlikely(vchan_info == NULL || !vchan_info->vq_enabled))
+ continue;
+
+ edma_circular_buffer_flush_to_dma_dev(adapter, &vchan_info->dma_buf, dma_dev_id,
+ vchan, &nb_enqueued);
+ *nb_ops_flushed += vchan_info->dma_buf.count;
+ nb += nb_enqueued;
+ }
+
+ return nb;
+}
+
+static unsigned int
+edma_adapter_enq_flush(struct event_dma_adapter *adapter)
+{
+ int16_t dma_dev_id;
+ uint16_t nb_enqueued = 0;
+ uint16_t nb_ops_flushed = 0;
+ uint16_t num_dma_dev = rte_dma_count_avail();
+
+ for (dma_dev_id = 0; dma_dev_id < num_dma_dev; dma_dev_id++)
+ nb_enqueued += edma_adapter_dev_flush(adapter, dma_dev_id, &nb_ops_flushed);
+ /**
+ * Enable dequeue from eventdev if all ops from circular
+ * buffer flushed to dma_dev
+ */
+ if (!nb_ops_flushed)
+ adapter->stop_enq_to_dma_dev = false;
+
+ return nb_enqueued;
+}
+
+/* Flush an instance's enqueue buffers every DMA_ENQ_FLUSH_THRESHOLD
+ * iterations of edma_adapter_enq_run()
+ */
+#define DMA_ENQ_FLUSH_THRESHOLD 1024
+
+static int
+edma_adapter_enq_run(struct event_dma_adapter *adapter, unsigned int max_enq)
+{
+ uint8_t event_port_id = adapter->event_port_id;
+ uint8_t event_dev_id = adapter->eventdev_id;
+ struct rte_event ev[DMA_BATCH_SIZE];
+ unsigned int nb_enq, nb_enqueued;
+ uint16_t n;
+
+ if (adapter->mode == RTE_EVENT_DMA_ADAPTER_OP_NEW)
+ return 0;
+
+ nb_enqueued = 0;
+ for (nb_enq = 0; nb_enq < max_enq; nb_enq += n) {
+
+ if (unlikely(adapter->stop_enq_to_dma_dev)) {
+ nb_enqueued += edma_adapter_enq_flush(adapter);
+
+ if (unlikely(adapter->stop_enq_to_dma_dev))
+ break;
+ }
+
+ n = rte_event_dequeue_burst(event_dev_id, event_port_id, ev, DMA_BATCH_SIZE, 0);
+
+ if (!n)
+ break;
+
+ nb_enqueued += edma_enq_to_dma_dev(adapter, ev, n);
+ }
+
+ if ((++adapter->transmit_loop_count & (DMA_ENQ_FLUSH_THRESHOLD - 1)) == 0)
+ nb_enqueued += edma_adapter_enq_flush(adapter);
+
+ return nb_enqueued;
+}
+
+#define DMA_ADAPTER_MAX_EV_ENQ_RETRIES 100
+
+static inline uint16_t
+edma_ops_enqueue_burst(struct event_dma_adapter *adapter, struct rte_event_dma_adapter_op **ops,
+ uint16_t num)
+{
+ uint8_t event_port_id = adapter->event_port_id;
+ uint8_t event_dev_id = adapter->eventdev_id;
+ struct rte_event events[DMA_BATCH_SIZE];
+ struct rte_event *response_info;
+ uint16_t nb_enqueued, nb_ev;
+ uint8_t retry;
+ uint8_t i;
+
+ nb_ev = 0;
+ retry = 0;
+ nb_enqueued = 0;
+ num = RTE_MIN(num, DMA_BATCH_SIZE);
+ for (i = 0; i < num; i++) {
+ struct rte_event *ev = &events[nb_ev++];
+
+ /* Expected to have response info appended to dma_op. */
+ response_info = (struct rte_event *)((uint8_t *)ops[i] +
+ sizeof(struct rte_event_dma_adapter_op));
+ if (unlikely(response_info == NULL)) {
+ if (ops[i] != NULL && ops[i]->op_mp != NULL)
+ rte_mempool_put(ops[i]->op_mp, ops[i]);
+ continue;
+ }
+
+ rte_memcpy(ev, response_info, sizeof(struct rte_event));
+ ev->event_ptr = ops[i];
+ ev->event_type = RTE_EVENT_TYPE_DMADEV;
+ if (adapter->implicit_release_disabled)
+ ev->op = RTE_EVENT_OP_FORWARD;
+ else
+ ev->op = RTE_EVENT_OP_NEW;
+ }
+
+ do {
+ nb_enqueued += rte_event_enqueue_burst(event_dev_id, event_port_id,
+ &events[nb_enqueued], nb_ev - nb_enqueued);
+
+ } while (retry++ < DMA_ADAPTER_MAX_EV_ENQ_RETRIES && nb_enqueued < nb_ev);
+
+ return nb_enqueued;
+}
+
+static int
+edma_circular_buffer_flush_to_evdev(struct event_dma_adapter *adapter,
+ struct dma_ops_circular_buffer *bufp,
+ uint16_t *enqueue_count)
+{
+ struct rte_event_dma_adapter_op **ops = bufp->op_buffer;
+ uint16_t n = 0, nb_ops_flushed;
+ uint16_t *head = &bufp->head;
+ uint16_t *tail = &bufp->tail;
+
+ if (*tail > *head)
+ n = *tail - *head;
+ else if (*tail < *head)
+ n = bufp->size - *head;
+ else {
+ if (enqueue_count)
+ *enqueue_count = 0;
+ return 0; /* buffer empty */
+ }
+
+ if (enqueue_count && n > *enqueue_count)
+ n = *enqueue_count;
+
+ nb_ops_flushed = edma_ops_enqueue_burst(adapter, &ops[*head], n);
+ if (enqueue_count)
+ *enqueue_count = nb_ops_flushed;
+
+ bufp->count -= nb_ops_flushed;
+ if (!bufp->count) {
+ *head = 0;
+ *tail = 0;
+ return 0; /* buffer empty */
+ }
+
+ *head = (*head + nb_ops_flushed) % bufp->size;
+ return 1;
+}
+
+static void
+edma_ops_buffer_flush(struct event_dma_adapter *adapter)
+{
+ if (likely(adapter->ebuf.count == 0))
+ return;
+
+ while (edma_circular_buffer_flush_to_evdev(adapter, &adapter->ebuf, NULL))
+ ;
+}
+
+static inline unsigned int
+edma_adapter_deq_run(struct event_dma_adapter *adapter, unsigned int max_deq)
+{
+ struct dma_vchan_info *vchan_info;
+ struct dma_ops_circular_buffer *tq_buf;
+ struct rte_event_dma_adapter_op *ops;
+ uint16_t n, nb_deq, nb_enqueued, i;
+ struct dma_device_info *dev_info;
+ uint16_t vchan, num_vchan;
+ uint16_t num_dma_dev;
+ int16_t dma_dev_id;
+ uint16_t index;
+ bool done;
+ bool err;
+
+ nb_deq = 0;
+ edma_ops_buffer_flush(adapter);
+
+ num_dma_dev = rte_dma_count_avail();
+ do {
+ done = true;
+
+ for (dma_dev_id = adapter->next_dmadev_id; dma_dev_id < num_dma_dev; dma_dev_id++) {
+ uint16_t queues = 0;
+ dev_info = &adapter->dma_devs[dma_dev_id];
+ num_vchan = dev_info->num_vchanq;
+
+ for (vchan = dev_info->next_vchan_id; queues < num_vchan;
+ vchan = (vchan + 1) % num_vchan, queues++) {
+
+ vchan_info = &dev_info->vchanq[vchan];
+ if (unlikely(vchan_info == NULL || !vchan_info->vq_enabled))
+ continue;
+
+ n = rte_dma_completed(dma_dev_id, vchan, DMA_BATCH_SIZE,
+ &index, &err);
+ if (!n)
+ continue;
+
+ done = false;
+
+ tq_buf = &dev_info->tqmap[vchan].dma_buf;
+
+ nb_enqueued = n;
+ if (unlikely(!adapter->ebuf.count))
+ edma_circular_buffer_flush_to_evdev(adapter, tq_buf,
+ &nb_enqueued);
+
+ if (likely(nb_enqueued == n))
+ goto check;
+
+ /* Failed to enqueue events case */
+ for (i = nb_enqueued; i < n; i++) {
+ ops = tq_buf->op_buffer[tq_buf->head];
+ edma_circular_buffer_add(&adapter->ebuf, ops);
+ tq_buf->head = (tq_buf->head + 1) % tq_buf->size;
+ }
+
+check:
+ nb_deq += n;
+ if (nb_deq >= max_deq) {
+ if ((vchan + 1) == num_vchan)
+ adapter->next_dmadev_id =
+ (dma_dev_id + 1) % num_dma_dev;
+
+ dev_info->next_vchan_id = (vchan + 1) % num_vchan;
+
+ return nb_deq;
+ }
+ }
+ }
+ adapter->next_dmadev_id = 0;
+
+ } while (done == false);
+
+ return nb_deq;
+}
+
+static int
+edma_adapter_run(struct event_dma_adapter *adapter, unsigned int max_ops)
+{
+ unsigned int ops_left = max_ops;
+
+ while (ops_left > 0) {
+ unsigned int e_cnt, d_cnt;
+
+ e_cnt = edma_adapter_deq_run(adapter, ops_left);
+ ops_left -= RTE_MIN(ops_left, e_cnt);
+
+ d_cnt = edma_adapter_enq_run(adapter, ops_left);
+ ops_left -= RTE_MIN(ops_left, d_cnt);
+
+ if (e_cnt == 0 && d_cnt == 0)
+ break;
+ }
+
+ if (ops_left == max_ops) {
+ rte_event_maintain(adapter->eventdev_id, adapter->event_port_id, 0);
+ return -EAGAIN;
+ } else
+ return 0;
+}
+
+static int
+edma_service_func(void *args)
+{
+ struct event_dma_adapter *adapter = args;
+ int ret;
+
+ if (rte_spinlock_trylock(&adapter->lock) == 0)
+ return 0;
+ ret = edma_adapter_run(adapter, adapter->max_nb);
+ rte_spinlock_unlock(&adapter->lock);
+
+ return ret;
+}
+
+static int
+edma_init_service(struct event_dma_adapter *adapter, uint8_t id)
+{
+ struct rte_event_dma_adapter_conf adapter_conf;
+ struct rte_service_spec service;
+ uint32_t impl_rel;
+ int ret;
+
+ if (adapter->service_initialized)
+ return 0;
+
+ memset(&service, 0, sizeof(service));
+ snprintf(service.name, DMA_ADAPTER_NAME_LEN, "rte_event_dma_adapter_%d", id);
+ service.socket_id = adapter->socket_id;
+ service.callback = edma_service_func;
+ service.callback_userdata = adapter;
+
+ /* Service function handles locking for queue add/del updates */
+ service.capabilities = RTE_SERVICE_CAP_MT_SAFE;
+ ret = rte_service_component_register(&service, &adapter->service_id);
+ if (ret) {
+ RTE_EDEV_LOG_ERR("failed to register service %s err = %" PRId32, service.name, ret);
+ return ret;
+ }
+
+ ret = adapter->conf_cb(id, adapter->eventdev_id, &adapter_conf, adapter->conf_arg);
+ if (ret) {
+ RTE_EDEV_LOG_ERR("configuration callback failed err = %" PRId32, ret);
+ return ret;
+ }
+
+ adapter->max_nb = adapter_conf.max_nb;
+ adapter->event_port_id = adapter_conf.event_port_id;
+
+ if (rte_event_port_attr_get(adapter->eventdev_id, adapter->event_port_id,
+ RTE_EVENT_PORT_ATTR_IMPLICIT_RELEASE_DISABLE, &impl_rel)) {
+ RTE_EDEV_LOG_ERR("Failed to get port info for eventdev %" PRId32,
+ adapter->eventdev_id);
+ edma_circular_buffer_free(&adapter->ebuf);
+ rte_free(adapter);
+ return -EINVAL;
+ }
+
+ adapter->implicit_release_disabled = (uint8_t)impl_rel;
+ adapter->service_initialized = 1;
+
+ return ret;
+}
+
static void
edma_update_vchanq_info(struct event_dma_adapter *adapter, struct dma_device_info *dev_info,
uint16_t vchan, uint8_t add)
@@ -391,6 +892,60 @@ edma_update_vchanq_info(struct event_dma_adapter *adapter, struct dma_device_inf
}
}
+static int
+edma_add_vchan(struct event_dma_adapter *adapter, int16_t dma_dev_id, uint16_t vchan)
+{
+ struct dma_device_info *dev_info = &adapter->dma_devs[dma_dev_id];
+ struct dma_vchan_info *vchanq;
+ struct dma_vchan_info *tqmap;
+ uint16_t nb_vchans;
+ uint32_t i;
+
+ if (dev_info->vchanq == NULL) {
+ nb_vchans = dev_info->num_dma_dev_vchan;
+
+ dev_info->vchanq = rte_zmalloc_socket(adapter->mem_name,
+ nb_vchans * sizeof(struct dma_vchan_info),
+ 0, adapter->socket_id);
+ if (dev_info->vchanq == NULL)
+ return -ENOMEM;
+
+ dev_info->tqmap = rte_zmalloc_socket(adapter->mem_name,
+ nb_vchans * sizeof(struct dma_vchan_info),
+ 0, adapter->socket_id);
+ if (dev_info->tqmap == NULL)
+ return -ENOMEM;
+
+ for (i = 0; i < nb_vchans; i++) {
+ vchanq = &dev_info->vchanq[i];
+
+ if (edma_circular_buffer_init("dma_dev_circular_buffer", &vchanq->dma_buf,
+ DMA_ADAPTER_OPS_BUFFER_SIZE)) {
+ RTE_EDEV_LOG_ERR("Failed to get memory for dma_dev buffer");
+ rte_free(vchanq);
+ return -ENOMEM;
+ }
+
+ tqmap = &dev_info->tqmap[i];
+ if (edma_circular_buffer_init("dma_dev_circular_trans_buf", &tqmap->dma_buf,
+ DMA_ADAPTER_OPS_BUFFER_SIZE)) {
+ RTE_EDEV_LOG_ERR(
+ "Failed to get memory for dma_dev transaction buffer");
+ rte_free(tqmap);
+ return -ENOMEM;
+ }
+ }
+ }
+
+ if (vchan == RTE_DMA_ALL_VCHAN) {
+ for (i = 0; i < dev_info->num_dma_dev_vchan; i++)
+ edma_update_vchanq_info(adapter, dev_info, i, 1);
+ } else
+ edma_update_vchanq_info(adapter, dev_info, vchan, 1);
+
+ return 0;
+}
+
int
rte_event_dma_adapter_vchan_add(uint8_t id, int16_t dma_dev_id, uint16_t vchan,
const struct rte_event *event)
@@ -470,6 +1025,38 @@ rte_event_dma_adapter_vchan_add(uint8_t id, int16_t dma_dev_id, uint16_t vchan,
edma_update_vchanq_info(adapter, &adapter->dma_devs[dma_dev_id], vchan, 1);
}
+ /* In case HW cap is RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP_NEW, or SW adapter, initiate
+ * services so the application can choose which ever way it wants to use the adapter.
+ *
+ * Case 1: RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP_NEW. Application may wants to use one
+ * of below two modes
+ *
+ * a. OP_FORWARD mode -> HW Dequeue + SW enqueue
+ * b. OP_NEW mode -> HW Dequeue
+ *
+ * Case 2: No HW caps, use SW adapter
+ *
+ * a. OP_FORWARD mode -> SW enqueue & dequeue
+ * b. OP_NEW mode -> SW Dequeue
+ */
+ if ((cap & RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP_NEW &&
+ !(cap & RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP_FWD) &&
+ adapter->mode == RTE_EVENT_DMA_ADAPTER_OP_FORWARD) ||
+ (!(cap & RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP_NEW) &&
+ !(cap & RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP_FWD) &&
+ !(cap & RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_VCHAN_EV_BIND))) {
+ rte_spinlock_lock(&adapter->lock);
+ ret = edma_init_service(adapter, id);
+ if (ret == 0)
+ ret = edma_add_vchan(adapter, dma_dev_id, vchan);
+ rte_spinlock_unlock(&adapter->lock);
+
+ if (ret)
+ return ret;
+
+ rte_service_component_runstate_set(adapter->service_id, 1);
+ }
+
return 0;
}
@@ -533,6 +1120,7 @@ rte_event_dma_adapter_vchan_del(uint8_t id, int16_t dma_dev_id, uint16_t vchan)
}
rte_spinlock_unlock(&adapter->lock);
+ rte_service_component_runstate_set(adapter->service_id, adapter->nb_vchanq);
}
return ret;
--
2.25.1
^ permalink raw reply [flat|nested] 110+ messages in thread
* [PATCH v5 06/12] eventdev/dma: support for adapter start and stop
2023-09-28 10:36 ` [PATCH v5 00/12] event DMA adapter library support Amit Prakash Shukla
` (4 preceding siblings ...)
2023-09-28 10:36 ` [PATCH v5 05/12] eventdev/dma: support for adapter service function Amit Prakash Shukla
@ 2023-09-28 10:36 ` Amit Prakash Shukla
2023-09-28 10:36 ` [PATCH v5 07/12] eventdev/dma: support for adapter service ID get Amit Prakash Shukla
` (6 subsequent siblings)
12 siblings, 0 replies; 110+ messages in thread
From: Amit Prakash Shukla @ 2023-09-28 10:36 UTC (permalink / raw)
To: Amit Prakash Shukla, Jerin Jacob
Cc: dev, fengchengwen, kevin.laatz, bruce.richardson, conor.walsh,
vattunuru, g.singh, sachin.saxena, hemant.agrawal, cheng1.jiang,
ndabilpuram, anoobj, mb
Added API support to start and stop DMA adapter.
Signed-off-by: Amit Prakash Shukla <amitprakashs@marvell.com>
---
lib/eventdev/rte_event_dma_adapter.c | 69 ++++++++++++++++++++++++++++
1 file changed, 69 insertions(+)
diff --git a/lib/eventdev/rte_event_dma_adapter.c b/lib/eventdev/rte_event_dma_adapter.c
index b0845eb415..e955f19c68 100644
--- a/lib/eventdev/rte_event_dma_adapter.c
+++ b/lib/eventdev/rte_event_dma_adapter.c
@@ -74,6 +74,13 @@ struct dma_device_info {
/* Next queue pair to be processed */
uint16_t next_vchan_id;
+ /* Set to indicate processing has been started */
+ uint8_t dev_started;
+
+ /* Set to indicate dmadev->eventdev packet
+ * transfer uses a hardware mechanism
+ */
+ uint8_t internal_event_port;
} __rte_cache_aligned;
struct event_dma_adapter {
@@ -1125,3 +1132,65 @@ rte_event_dma_adapter_vchan_del(uint8_t id, int16_t dma_dev_id, uint16_t vchan)
return ret;
}
+
+static int
+edma_adapter_ctrl(uint8_t id, int start)
+{
+ struct event_dma_adapter *adapter;
+ struct dma_device_info *dev_info;
+ struct rte_eventdev *dev;
+ uint16_t num_dma_dev;
+ int stop = !start;
+ int use_service;
+ uint32_t i;
+
+ use_service = 0;
+ EVENT_DMA_ADAPTER_ID_VALID_OR_ERR_RET(id, -EINVAL);
+ adapter = edma_id_to_adapter(id);
+ if (adapter == NULL)
+ return -EINVAL;
+
+ num_dma_dev = rte_dma_count_avail();
+ dev = &rte_eventdevs[adapter->eventdev_id];
+
+ for (i = 0; i < num_dma_dev; i++) {
+ dev_info = &adapter->dma_devs[i];
+ /* start check for num queue pairs */
+ if (start && !dev_info->num_vchanq)
+ continue;
+ /* stop check if dev has been started */
+ if (stop && !dev_info->dev_started)
+ continue;
+ use_service |= !dev_info->internal_event_port;
+ dev_info->dev_started = start;
+ if (dev_info->internal_event_port == 0)
+ continue;
+ start ? (*dev->dev_ops->dma_adapter_start)(dev, i) :
+ (*dev->dev_ops->dma_adapter_stop)(dev, i);
+ }
+
+ if (use_service)
+ rte_service_runstate_set(adapter->service_id, start);
+
+ return 0;
+}
+
+int
+rte_event_dma_adapter_start(uint8_t id)
+{
+ struct event_dma_adapter *adapter;
+
+ EVENT_DMA_ADAPTER_ID_VALID_OR_ERR_RET(id, -EINVAL);
+
+ adapter = edma_id_to_adapter(id);
+ if (adapter == NULL)
+ return -EINVAL;
+
+ return edma_adapter_ctrl(id, 1);
+}
+
+int
+rte_event_dma_adapter_stop(uint8_t id)
+{
+ return edma_adapter_ctrl(id, 0);
+}
--
2.25.1
^ permalink raw reply [flat|nested] 110+ messages in thread
* [PATCH v5 07/12] eventdev/dma: support for adapter service ID get
2023-09-28 10:36 ` [PATCH v5 00/12] event DMA adapter library support Amit Prakash Shukla
` (5 preceding siblings ...)
2023-09-28 10:36 ` [PATCH v5 06/12] eventdev/dma: support for adapter start and stop Amit Prakash Shukla
@ 2023-09-28 10:36 ` Amit Prakash Shukla
2023-09-28 10:36 ` [PATCH v5 08/12] eventdev/dma: support adapter runtime params Amit Prakash Shukla
` (5 subsequent siblings)
12 siblings, 0 replies; 110+ messages in thread
From: Amit Prakash Shukla @ 2023-09-28 10:36 UTC (permalink / raw)
To: Amit Prakash Shukla, Jerin Jacob
Cc: dev, fengchengwen, kevin.laatz, bruce.richardson, conor.walsh,
vattunuru, g.singh, sachin.saxena, hemant.agrawal, cheng1.jiang,
ndabilpuram, anoobj, mb
Added API support to get DMA adapter service ID. Service id
returned in the variable by the API call shall be used by application
to map a service core.
Signed-off-by: Amit Prakash Shukla <amitprakashs@marvell.com>
---
lib/eventdev/rte_event_dma_adapter.c | 17 +++++++++++++++++
1 file changed, 17 insertions(+)
diff --git a/lib/eventdev/rte_event_dma_adapter.c b/lib/eventdev/rte_event_dma_adapter.c
index e955f19c68..63b07cd14e 100644
--- a/lib/eventdev/rte_event_dma_adapter.c
+++ b/lib/eventdev/rte_event_dma_adapter.c
@@ -1133,6 +1133,23 @@ rte_event_dma_adapter_vchan_del(uint8_t id, int16_t dma_dev_id, uint16_t vchan)
return ret;
}
+int
+rte_event_dma_adapter_service_id_get(uint8_t id, uint32_t *service_id)
+{
+ struct event_dma_adapter *adapter;
+
+ EVENT_DMA_ADAPTER_ID_VALID_OR_ERR_RET(id, -EINVAL);
+
+ adapter = edma_id_to_adapter(id);
+ if (adapter == NULL || service_id == NULL)
+ return -EINVAL;
+
+ if (adapter->service_initialized)
+ *service_id = adapter->service_id;
+
+ return adapter->service_initialized ? 0 : -ESRCH;
+}
+
static int
edma_adapter_ctrl(uint8_t id, int start)
{
--
2.25.1
^ permalink raw reply [flat|nested] 110+ messages in thread
* [PATCH v5 08/12] eventdev/dma: support adapter runtime params
2023-09-28 10:36 ` [PATCH v5 00/12] event DMA adapter library support Amit Prakash Shukla
` (6 preceding siblings ...)
2023-09-28 10:36 ` [PATCH v5 07/12] eventdev/dma: support for adapter service ID get Amit Prakash Shukla
@ 2023-09-28 10:36 ` Amit Prakash Shukla
2023-09-28 10:36 ` [PATCH v5 09/12] eventdev/dma: support for adapter stats Amit Prakash Shukla
` (4 subsequent siblings)
12 siblings, 0 replies; 110+ messages in thread
From: Amit Prakash Shukla @ 2023-09-28 10:36 UTC (permalink / raw)
To: Amit Prakash Shukla, Jerin Jacob
Cc: dev, fengchengwen, kevin.laatz, bruce.richardson, conor.walsh,
vattunuru, g.singh, sachin.saxena, hemant.agrawal, cheng1.jiang,
ndabilpuram, anoobj, mb
Added support to set and get runtime params for DMA adapter. The
parameters that can be set/get are defined in
struct rte_event_dma_adapter_runtime_params.
Signed-off-by: Amit Prakash Shukla <amitprakashs@marvell.com>
---
lib/eventdev/rte_event_dma_adapter.c | 93 ++++++++++++++++++++++++++++
1 file changed, 93 insertions(+)
diff --git a/lib/eventdev/rte_event_dma_adapter.c b/lib/eventdev/rte_event_dma_adapter.c
index 63b07cd14e..850b010712 100644
--- a/lib/eventdev/rte_event_dma_adapter.c
+++ b/lib/eventdev/rte_event_dma_adapter.c
@@ -1211,3 +1211,96 @@ rte_event_dma_adapter_stop(uint8_t id)
{
return edma_adapter_ctrl(id, 0);
}
+
+#define DEFAULT_MAX_NB 128
+
+int
+rte_event_dma_adapter_runtime_params_init(struct rte_event_dma_adapter_runtime_params *params)
+{
+ if (params == NULL)
+ return -EINVAL;
+
+ memset(params, 0, sizeof(*params));
+ params->max_nb = DEFAULT_MAX_NB;
+
+ return 0;
+}
+
+static int
+dma_adapter_cap_check(struct event_dma_adapter *adapter)
+{
+ uint32_t caps;
+ int ret;
+
+ if (!adapter->nb_vchanq)
+ return -EINVAL;
+
+ ret = rte_event_dma_adapter_caps_get(adapter->eventdev_id, adapter->next_dmadev_id, &caps);
+ if (ret) {
+ RTE_EDEV_LOG_ERR("Failed to get adapter caps dev %" PRIu8 " cdev %" PRIu8,
+ adapter->eventdev_id, adapter->next_dmadev_id);
+ return ret;
+ }
+
+ if ((caps & RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP_FWD) ||
+ (caps & RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP_NEW))
+ return -ENOTSUP;
+
+ return 0;
+}
+
+int
+rte_event_dma_adapter_runtime_params_set(uint8_t id,
+ struct rte_event_dma_adapter_runtime_params *params)
+{
+ struct event_dma_adapter *adapter;
+ int ret;
+
+ EVENT_DMA_ADAPTER_ID_VALID_OR_ERR_RET(id, -EINVAL);
+
+ if (params == NULL) {
+ RTE_EDEV_LOG_ERR("params pointer is NULL\n");
+ return -EINVAL;
+ }
+
+ adapter = edma_id_to_adapter(id);
+ if (adapter == NULL)
+ return -EINVAL;
+
+ ret = dma_adapter_cap_check(adapter);
+ if (ret)
+ return ret;
+
+ rte_spinlock_lock(&adapter->lock);
+ adapter->max_nb = params->max_nb;
+ rte_spinlock_unlock(&adapter->lock);
+
+ return 0;
+}
+
+int
+rte_event_dma_adapter_runtime_params_get(uint8_t id,
+ struct rte_event_dma_adapter_runtime_params *params)
+{
+ struct event_dma_adapter *adapter;
+ int ret;
+
+ EVENT_DMA_ADAPTER_ID_VALID_OR_ERR_RET(id, -EINVAL);
+
+ if (params == NULL) {
+ RTE_EDEV_LOG_ERR("params pointer is NULL\n");
+ return -EINVAL;
+ }
+
+ adapter = edma_id_to_adapter(id);
+ if (adapter == NULL)
+ return -EINVAL;
+
+ ret = dma_adapter_cap_check(adapter);
+ if (ret)
+ return ret;
+
+ params->max_nb = adapter->max_nb;
+
+ return 0;
+}
--
2.25.1
^ permalink raw reply [flat|nested] 110+ messages in thread
* [PATCH v5 09/12] eventdev/dma: support for adapter stats
2023-09-28 10:36 ` [PATCH v5 00/12] event DMA adapter library support Amit Prakash Shukla
` (7 preceding siblings ...)
2023-09-28 10:36 ` [PATCH v5 08/12] eventdev/dma: support adapter runtime params Amit Prakash Shukla
@ 2023-09-28 10:36 ` Amit Prakash Shukla
2023-09-28 10:36 ` [PATCH v5 10/12] eventdev/dma: support for adapter enqueue Amit Prakash Shukla
` (3 subsequent siblings)
12 siblings, 0 replies; 110+ messages in thread
From: Amit Prakash Shukla @ 2023-09-28 10:36 UTC (permalink / raw)
To: Amit Prakash Shukla, Jerin Jacob
Cc: dev, fengchengwen, kevin.laatz, bruce.richardson, conor.walsh,
vattunuru, g.singh, sachin.saxena, hemant.agrawal, cheng1.jiang,
ndabilpuram, anoobj, mb
Added DMA adapter stats API support to get and reset stats. DMA
SW adapter stats and eventdev driver supported stats for enqueue
and dequeue are reported by get API.
Signed-off-by: Amit Prakash Shukla <amitprakashs@marvell.com>
---
lib/eventdev/rte_event_dma_adapter.c | 95 ++++++++++++++++++++++++++++
1 file changed, 95 insertions(+)
diff --git a/lib/eventdev/rte_event_dma_adapter.c b/lib/eventdev/rte_event_dma_adapter.c
index 850b010712..842fb74734 100644
--- a/lib/eventdev/rte_event_dma_adapter.c
+++ b/lib/eventdev/rte_event_dma_adapter.c
@@ -142,6 +142,9 @@ struct event_dma_adapter {
/* Loop counter to flush dma ops */
uint16_t transmit_loop_count;
+
+ /* Per instance stats structure */
+ struct rte_event_dma_adapter_stats dma_stats;
} __rte_cache_aligned;
static struct event_dma_adapter **event_dma_adapter;
@@ -471,6 +474,7 @@ rte_event_dma_adapter_free(uint8_t id)
static inline unsigned int
edma_enq_to_dma_dev(struct event_dma_adapter *adapter, struct rte_event *ev, unsigned int cnt)
{
+ struct rte_event_dma_adapter_stats *stats = &adapter->dma_stats;
struct dma_vchan_info *vchan_qinfo = NULL;
struct rte_event_dma_adapter_op *dma_op;
uint16_t vchan, nb_enqueued = 0;
@@ -480,6 +484,7 @@ edma_enq_to_dma_dev(struct event_dma_adapter *adapter, struct rte_event *ev, uns
ret = 0;
n = 0;
+ stats->event_deq_count += cnt;
for (i = 0; i < cnt; i++) {
dma_op = ev[i].event_ptr;
@@ -502,6 +507,7 @@ edma_enq_to_dma_dev(struct event_dma_adapter *adapter, struct rte_event *ev, uns
ret = edma_circular_buffer_flush_to_dma_dev(adapter, &vchan_qinfo->dma_buf,
dma_dev_id, vchan,
&nb_enqueued);
+ stats->dma_enq_count += nb_enqueued;
n += nb_enqueued;
/**
@@ -548,6 +554,7 @@ edma_adapter_dev_flush(struct event_dma_adapter *adapter, int16_t dma_dev_id,
static unsigned int
edma_adapter_enq_flush(struct event_dma_adapter *adapter)
{
+ struct rte_event_dma_adapter_stats *stats = &adapter->dma_stats;
int16_t dma_dev_id;
uint16_t nb_enqueued = 0;
uint16_t nb_ops_flushed = 0;
@@ -562,6 +569,8 @@ edma_adapter_enq_flush(struct event_dma_adapter *adapter)
if (!nb_ops_flushed)
adapter->stop_enq_to_dma_dev = false;
+ stats->dma_enq_count += nb_enqueued;
+
return nb_enqueued;
}
@@ -573,6 +582,7 @@ edma_adapter_enq_flush(struct event_dma_adapter *adapter)
static int
edma_adapter_enq_run(struct event_dma_adapter *adapter, unsigned int max_enq)
{
+ struct rte_event_dma_adapter_stats *stats = &adapter->dma_stats;
uint8_t event_port_id = adapter->event_port_id;
uint8_t event_dev_id = adapter->eventdev_id;
struct rte_event ev[DMA_BATCH_SIZE];
@@ -592,6 +602,7 @@ edma_adapter_enq_run(struct event_dma_adapter *adapter, unsigned int max_enq)
break;
}
+ stats->event_poll_count++;
n = rte_event_dequeue_burst(event_dev_id, event_port_id, ev, DMA_BATCH_SIZE, 0);
if (!n)
@@ -612,6 +623,7 @@ static inline uint16_t
edma_ops_enqueue_burst(struct event_dma_adapter *adapter, struct rte_event_dma_adapter_op **ops,
uint16_t num)
{
+ struct rte_event_dma_adapter_stats *stats = &adapter->dma_stats;
uint8_t event_port_id = adapter->event_port_id;
uint8_t event_dev_id = adapter->eventdev_id;
struct rte_event events[DMA_BATCH_SIZE];
@@ -651,6 +663,10 @@ edma_ops_enqueue_burst(struct event_dma_adapter *adapter, struct rte_event_dma_a
} while (retry++ < DMA_ADAPTER_MAX_EV_ENQ_RETRIES && nb_enqueued < nb_ev);
+ stats->event_enq_fail_count += nb_ev - nb_enqueued;
+ stats->event_enq_count += nb_enqueued;
+ stats->event_enq_retry_count += retry - 1;
+
return nb_enqueued;
}
@@ -705,6 +721,7 @@ edma_ops_buffer_flush(struct event_dma_adapter *adapter)
static inline unsigned int
edma_adapter_deq_run(struct event_dma_adapter *adapter, unsigned int max_deq)
{
+ struct rte_event_dma_adapter_stats *stats = &adapter->dma_stats;
struct dma_vchan_info *vchan_info;
struct dma_ops_circular_buffer *tq_buf;
struct rte_event_dma_adapter_op *ops;
@@ -742,6 +759,7 @@ edma_adapter_deq_run(struct event_dma_adapter *adapter, unsigned int max_deq)
continue;
done = false;
+ stats->dma_deq_count += n;
tq_buf = &dev_info->tqmap[vchan].dma_buf;
@@ -1304,3 +1322,80 @@ rte_event_dma_adapter_runtime_params_get(uint8_t id,
return 0;
}
+
+int
+rte_event_dma_adapter_stats_get(uint8_t id, struct rte_event_dma_adapter_stats *stats)
+{
+ struct rte_event_dma_adapter_stats dev_stats_sum = {0};
+ struct rte_event_dma_adapter_stats dev_stats;
+ struct event_dma_adapter *adapter;
+ struct dma_device_info *dev_info;
+ struct rte_eventdev *dev;
+ uint16_t num_dma_dev;
+ uint32_t i;
+ int ret;
+
+ EVENT_DMA_ADAPTER_ID_VALID_OR_ERR_RET(id, -EINVAL);
+
+ adapter = edma_id_to_adapter(id);
+ if (adapter == NULL || stats == NULL)
+ return -EINVAL;
+
+ num_dma_dev = rte_dma_count_avail();
+ dev = &rte_eventdevs[adapter->eventdev_id];
+ memset(stats, 0, sizeof(*stats));
+ for (i = 0; i < num_dma_dev; i++) {
+ dev_info = &adapter->dma_devs[i];
+
+ if (dev_info->internal_event_port == 0 ||
+ dev->dev_ops->dma_adapter_stats_get == NULL)
+ continue;
+
+ ret = (*dev->dev_ops->dma_adapter_stats_get)(dev, i, &dev_stats);
+ if (ret)
+ continue;
+
+ dev_stats_sum.dma_deq_count += dev_stats.dma_deq_count;
+ dev_stats_sum.event_enq_count += dev_stats.event_enq_count;
+ }
+
+ if (adapter->service_initialized)
+ *stats = adapter->dma_stats;
+
+ stats->dma_deq_count += dev_stats_sum.dma_deq_count;
+ stats->event_enq_count += dev_stats_sum.event_enq_count;
+
+ return 0;
+}
+
+int
+rte_event_dma_adapter_stats_reset(uint8_t id)
+{
+ struct event_dma_adapter *adapter;
+ struct dma_device_info *dev_info;
+ struct rte_eventdev *dev;
+ uint16_t num_dma_dev;
+ uint32_t i;
+
+ EVENT_DMA_ADAPTER_ID_VALID_OR_ERR_RET(id, -EINVAL);
+
+ adapter = edma_id_to_adapter(id);
+ if (adapter == NULL)
+ return -EINVAL;
+
+ num_dma_dev = rte_dma_count_avail();
+ dev = &rte_eventdevs[adapter->eventdev_id];
+ for (i = 0; i < num_dma_dev; i++) {
+ dev_info = &adapter->dma_devs[i];
+
+ if (dev_info->internal_event_port == 0 ||
+ dev->dev_ops->dma_adapter_stats_reset == NULL)
+ continue;
+
+ (*dev->dev_ops->dma_adapter_stats_reset)(dev, i);
+ }
+
+ memset(&adapter->dma_stats, 0, sizeof(adapter->dma_stats));
+
+ return 0;
+}
--
2.25.1
^ permalink raw reply [flat|nested] 110+ messages in thread
* [PATCH v5 10/12] eventdev/dma: support for adapter enqueue
2023-09-28 10:36 ` [PATCH v5 00/12] event DMA adapter library support Amit Prakash Shukla
` (8 preceding siblings ...)
2023-09-28 10:36 ` [PATCH v5 09/12] eventdev/dma: support for adapter stats Amit Prakash Shukla
@ 2023-09-28 10:36 ` Amit Prakash Shukla
2023-09-28 10:36 ` [PATCH v5 11/12] eventdev/dma: support for adapter event port get Amit Prakash Shukla
` (2 subsequent siblings)
12 siblings, 0 replies; 110+ messages in thread
From: Amit Prakash Shukla @ 2023-09-28 10:36 UTC (permalink / raw)
To: Amit Prakash Shukla, Jerin Jacob
Cc: dev, fengchengwen, kevin.laatz, bruce.richardson, conor.walsh,
vattunuru, g.singh, sachin.saxena, hemant.agrawal, cheng1.jiang,
ndabilpuram, anoobj, mb
Added API support to enqueue a DMA operation to the DMA driver.
Signed-off-by: Amit Prakash Shukla <amitprakashs@marvell.com>
---
lib/eventdev/rte_event_dma_adapter.c | 13 +++++++++++++
1 file changed, 13 insertions(+)
diff --git a/lib/eventdev/rte_event_dma_adapter.c b/lib/eventdev/rte_event_dma_adapter.c
index 842fb74734..bca2be2731 100644
--- a/lib/eventdev/rte_event_dma_adapter.c
+++ b/lib/eventdev/rte_event_dma_adapter.c
@@ -1399,3 +1399,16 @@ rte_event_dma_adapter_stats_reset(uint8_t id)
return 0;
}
+
+uint16_t
+rte_event_dma_adapter_enqueue(uint8_t dev_id, uint8_t port_id, struct rte_event ev[],
+ uint16_t nb_events)
+{
+ const struct rte_event_fp_ops *fp_ops;
+ void *port;
+
+ fp_ops = &rte_event_fp_ops[dev_id];
+ port = fp_ops->data[port_id];
+
+ return fp_ops->dma_enqueue(port, ev, nb_events);
+}
--
2.25.1
^ permalink raw reply [flat|nested] 110+ messages in thread
* [PATCH v5 11/12] eventdev/dma: support for adapter event port get
2023-09-28 10:36 ` [PATCH v5 00/12] event DMA adapter library support Amit Prakash Shukla
` (9 preceding siblings ...)
2023-09-28 10:36 ` [PATCH v5 10/12] eventdev/dma: support for adapter enqueue Amit Prakash Shukla
@ 2023-09-28 10:36 ` Amit Prakash Shukla
2023-09-28 10:36 ` [PATCH v5 12/12] app/test: add event DMA adapter auto-test Amit Prakash Shukla
2023-09-28 16:49 ` [PATCH v6 00/12] event DMA adapter library support Amit Prakash Shukla
12 siblings, 0 replies; 110+ messages in thread
From: Amit Prakash Shukla @ 2023-09-28 10:36 UTC (permalink / raw)
To: Amit Prakash Shukla, Jerin Jacob
Cc: dev, fengchengwen, kevin.laatz, bruce.richardson, conor.walsh,
vattunuru, g.singh, sachin.saxena, hemant.agrawal, cheng1.jiang,
ndabilpuram, anoobj, mb
Added support for DMA adapter event port get.
Signed-off-by: Amit Prakash Shukla <amitprakashs@marvell.com>
---
lib/eventdev/rte_event_dma_adapter.c | 16 ++++++++++++++++
1 file changed, 16 insertions(+)
diff --git a/lib/eventdev/rte_event_dma_adapter.c b/lib/eventdev/rte_event_dma_adapter.c
index bca2be2731..4899bc5d0f 100644
--- a/lib/eventdev/rte_event_dma_adapter.c
+++ b/lib/eventdev/rte_event_dma_adapter.c
@@ -471,6 +471,22 @@ rte_event_dma_adapter_free(uint8_t id)
return 0;
}
+int
+rte_event_dma_adapter_event_port_get(uint8_t id, uint8_t *event_port_id)
+{
+ struct event_dma_adapter *adapter;
+
+ EVENT_DMA_ADAPTER_ID_VALID_OR_ERR_RET(id, -EINVAL);
+
+ adapter = edma_id_to_adapter(id);
+ if (adapter == NULL || event_port_id == NULL)
+ return -EINVAL;
+
+ *event_port_id = adapter->event_port_id;
+
+ return 0;
+}
+
static inline unsigned int
edma_enq_to_dma_dev(struct event_dma_adapter *adapter, struct rte_event *ev, unsigned int cnt)
{
--
2.25.1
^ permalink raw reply [flat|nested] 110+ messages in thread
* [PATCH v5 12/12] app/test: add event DMA adapter auto-test
2023-09-28 10:36 ` [PATCH v5 00/12] event DMA adapter library support Amit Prakash Shukla
` (10 preceding siblings ...)
2023-09-28 10:36 ` [PATCH v5 11/12] eventdev/dma: support for adapter event port get Amit Prakash Shukla
@ 2023-09-28 10:36 ` Amit Prakash Shukla
2023-09-29 7:20 ` Jerin Jacob
2023-09-28 16:49 ` [PATCH v6 00/12] event DMA adapter library support Amit Prakash Shukla
12 siblings, 1 reply; 110+ messages in thread
From: Amit Prakash Shukla @ 2023-09-28 10:36 UTC (permalink / raw)
To: Amit Prakash Shukla
Cc: dev, jerinj, fengchengwen, kevin.laatz, bruce.richardson,
conor.walsh, vattunuru, g.singh, sachin.saxena, hemant.agrawal,
cheng1.jiang, ndabilpuram, anoobj, mb
Added testsuite to test the dma adapter functionality.
The testsuite detects event and DMA device capability
and accordingly dma adapter is configured and modes are
tested. Test command:
<build dir>/app/test/dpdk-test event_dma_adapter_autotest
Signed-off-by: Amit Prakash Shukla <amitprakashs@marvell.com>
---
app/test/meson.build | 1 +
app/test/test_event_dma_adapter.c | 805 ++++++++++++++++++++++++++++++
2 files changed, 806 insertions(+)
create mode 100644 app/test/test_event_dma_adapter.c
diff --git a/app/test/meson.build b/app/test/meson.build
index 05bae9216d..7caf5ae5fc 100644
--- a/app/test/meson.build
+++ b/app/test/meson.build
@@ -66,6 +66,7 @@ source_file_deps = {
'test_errno.c': [],
'test_ethdev_link.c': ['ethdev'],
'test_event_crypto_adapter.c': ['cryptodev', 'eventdev', 'bus_vdev'],
+ 'test_event_dma_adapter.c': ['dmadev', 'eventdev', 'bus_vdev'],
'test_event_eth_rx_adapter.c': ['ethdev', 'eventdev', 'bus_vdev'],
'test_event_eth_tx_adapter.c': ['bus_vdev', 'ethdev', 'net_ring', 'eventdev'],
'test_event_ring.c': ['eventdev'],
diff --git a/app/test/test_event_dma_adapter.c b/app/test/test_event_dma_adapter.c
new file mode 100644
index 0000000000..1e193f4b52
--- /dev/null
+++ b/app/test/test_event_dma_adapter.c
@@ -0,0 +1,805 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright (c) 2023 Marvell.
+ */
+
+#include "test.h"
+#include <string.h>
+#include <rte_common.h>
+#include <rte_malloc.h>
+#include <rte_mempool.h>
+#include <rte_mbuf.h>
+#include <rte_random.h>
+
+#ifdef RTE_EXEC_ENV_WINDOWS
+static int
+test_event_dma_adapter(void)
+{
+ printf("event_dma_adapter not supported on Windows, skipping test\n");
+ return TEST_SKIPPED;
+}
+
+#else
+
+#include <rte_bus_vdev.h>
+#include <rte_dmadev.h>
+#include <rte_eventdev.h>
+#include <rte_event_dma_adapter.h>
+#include <rte_service.h>
+
+#define NUM_MBUFS (8191)
+#define MBUF_CACHE_SIZE (256)
+#define TEST_APP_PORT_ID 0
+#define TEST_APP_EV_QUEUE_ID 0
+#define TEST_APP_EV_PRIORITY 0
+#define TEST_APP_EV_FLOWID 0xAABB
+#define TEST_DMA_EV_QUEUE_ID 1
+#define TEST_ADAPTER_ID 0
+#define TEST_DMA_DEV_ID 0
+#define TEST_DMA_VCHAN_ID 0
+#define PACKET_LENGTH 1024
+#define NB_TEST_PORTS 1
+#define NB_TEST_QUEUES 2
+#define NUM_CORES 2
+#define DMA_OP_POOL_SIZE 128
+#define TEST_MAX_OP 32
+#define TEST_RINGSIZE 512
+
+#define MBUF_SIZE (RTE_PKTMBUF_HEADROOM + PACKET_LENGTH)
+
+/* Handle log statements in same manner as test macros */
+#define LOG_DBG(...) RTE_LOG(DEBUG, EAL, __VA_ARGS__)
+
+struct event_dma_adapter_test_params {
+ struct rte_mempool *src_mbuf_pool;
+ struct rte_mempool *dst_mbuf_pool;
+ struct rte_mempool *op_mpool;
+ uint8_t dma_event_port_id;
+ uint8_t internal_port_op_fwd;
+};
+
+struct rte_event dma_response_info = {
+ .queue_id = TEST_APP_EV_QUEUE_ID,
+ .sched_type = RTE_SCHED_TYPE_ATOMIC,
+ .flow_id = TEST_APP_EV_FLOWID,
+ .priority = TEST_APP_EV_PRIORITY
+};
+
+static struct event_dma_adapter_test_params params;
+static uint8_t dma_adapter_setup_done;
+static uint32_t slcore_id;
+static int evdev;
+
+static int
+send_recv_ev(struct rte_event *ev)
+{
+ struct rte_event recv_ev[TEST_MAX_OP];
+ uint16_t nb_enqueued = 0;
+ int i = 0;
+
+ if (params.internal_port_op_fwd) {
+ nb_enqueued = rte_event_dma_adapter_enqueue(evdev, TEST_APP_PORT_ID, ev,
+ TEST_MAX_OP);
+ } else {
+ while (nb_enqueued < TEST_MAX_OP) {
+ nb_enqueued += rte_event_enqueue_burst(evdev, TEST_APP_PORT_ID,
+ &ev[nb_enqueued], TEST_MAX_OP -
+ nb_enqueued);
+ }
+ }
+
+ TEST_ASSERT_EQUAL(nb_enqueued, TEST_MAX_OP, "Failed to send event to dma adapter\n");
+
+ while (i < TEST_MAX_OP) {
+ if (rte_event_dequeue_burst(evdev, TEST_APP_PORT_ID, &recv_ev[i], 1, 0) != 1)
+ continue;
+ i++;
+ }
+
+ TEST_ASSERT_EQUAL(i, TEST_MAX_OP, "Test failed. Failed to dequeue events.\n");
+
+ return TEST_SUCCESS;
+}
+
+static int
+test_dma_adapter_stats(void)
+{
+ struct rte_event_dma_adapter_stats stats;
+
+ rte_event_dma_adapter_stats_get(TEST_ADAPTER_ID, &stats);
+ printf(" +------------------------------------------------------+\n");
+ printf(" + DMA adapter stats for instance %u:\n", TEST_ADAPTER_ID);
+ printf(" + Event port poll count 0x%" PRIx64 "\n",
+ stats.event_poll_count);
+ printf(" + Event dequeue count 0x%" PRIx64 "\n",
+ stats.event_deq_count);
+ printf(" + DMA dev enqueue count 0x%" PRIx64 "\n",
+ stats.dma_enq_count);
+ printf(" + DMA dev enqueue failed count 0x%" PRIx64 "\n",
+ stats.dma_enq_fail_count);
+ printf(" + DMA dev dequeue count 0x%" PRIx64 "\n",
+ stats.dma_deq_count);
+ printf(" + Event enqueue count 0x%" PRIx64 "\n",
+ stats.event_enq_count);
+ printf(" + Event enqueue retry count 0x%" PRIx64 "\n",
+ stats.event_enq_retry_count);
+ printf(" + Event enqueue fail count 0x%" PRIx64 "\n",
+ stats.event_enq_fail_count);
+ printf(" +------------------------------------------------------+\n");
+
+ rte_event_dma_adapter_stats_reset(TEST_ADAPTER_ID);
+ return TEST_SUCCESS;
+}
+
+static int
+test_dma_adapter_params(void)
+{
+ struct rte_event_dma_adapter_runtime_params out_params;
+ struct rte_event_dma_adapter_runtime_params in_params;
+ struct rte_event event;
+ uint32_t cap;
+ int err, rc;
+
+ err = rte_event_dma_adapter_caps_get(evdev, TEST_DMA_DEV_ID, &cap);
+ TEST_ASSERT_SUCCESS(err, "Failed to get adapter capabilities\n");
+
+ if (cap & RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_VCHAN_EV_BIND) {
+ err = rte_event_dma_adapter_vchan_add(TEST_ADAPTER_ID, TEST_DMA_DEV_ID,
+ TEST_DMA_VCHAN_ID, &event);
+ } else
+ err = rte_event_dma_adapter_vchan_add(TEST_ADAPTER_ID, TEST_DMA_DEV_ID,
+ TEST_DMA_VCHAN_ID, NULL);
+
+ TEST_ASSERT_SUCCESS(err, "Failed to add vchan\n");
+
+ err = rte_event_dma_adapter_runtime_params_init(&in_params);
+ TEST_ASSERT(err == 0, "Expected 0 got %d", err);
+ err = rte_event_dma_adapter_runtime_params_init(&out_params);
+ TEST_ASSERT(err == 0, "Expected 0 got %d", err);
+
+ /* Case 1: Get the default value of mbufs processed by adapter */
+ err = rte_event_dma_adapter_runtime_params_get(TEST_ADAPTER_ID, &out_params);
+ if (err == -ENOTSUP) {
+ rc = TEST_SKIPPED;
+ goto vchan_del;
+ }
+ TEST_ASSERT(err == 0, "Expected 0 got %d", err);
+
+ /* Case 2: Set max_nb = 32 (=BATCH_SEIZE) */
+ in_params.max_nb = 32;
+
+ err = rte_event_dma_adapter_runtime_params_set(TEST_ADAPTER_ID, &in_params);
+ TEST_ASSERT(err == 0, "Expected 0 got %d", err);
+
+ err = rte_event_dma_adapter_runtime_params_get(TEST_ADAPTER_ID, &out_params);
+ TEST_ASSERT(err == 0, "Expected 0 got %d", err);
+ TEST_ASSERT(in_params.max_nb == out_params.max_nb, "Expected %u got %u",
+ in_params.max_nb, out_params.max_nb);
+
+ /* Case 3: Set max_nb = 192 */
+ in_params.max_nb = 192;
+
+ err = rte_event_dma_adapter_runtime_params_set(TEST_ADAPTER_ID, &in_params);
+ TEST_ASSERT(err == 0, "Expected 0 got %d", err);
+
+ err = rte_event_dma_adapter_runtime_params_get(TEST_ADAPTER_ID, &out_params);
+ TEST_ASSERT(err == 0, "Expected 0 got %d", err);
+ TEST_ASSERT(in_params.max_nb == out_params.max_nb, "Expected %u got %u",
+ in_params.max_nb, out_params.max_nb);
+
+ /* Case 4: Set max_nb = 256 */
+ in_params.max_nb = 256;
+
+ err = rte_event_dma_adapter_runtime_params_set(TEST_ADAPTER_ID, &in_params);
+ TEST_ASSERT(err == 0, "Expected 0 got %d", err);
+
+ err = rte_event_dma_adapter_runtime_params_get(TEST_ADAPTER_ID, &out_params);
+ TEST_ASSERT(err == 0, "Expected 0 got %d", err);
+ TEST_ASSERT(in_params.max_nb == out_params.max_nb, "Expected %u got %u",
+ in_params.max_nb, out_params.max_nb);
+
+ /* Case 5: Set max_nb = 30(<BATCH_SIZE) */
+ in_params.max_nb = 30;
+
+ err = rte_event_dma_adapter_runtime_params_set(TEST_ADAPTER_ID, &in_params);
+ TEST_ASSERT(err == 0, "Expected 0 got %d", err);
+
+ err = rte_event_dma_adapter_runtime_params_get(TEST_ADAPTER_ID, &out_params);
+ TEST_ASSERT(err == 0, "Expected 0 got %d", err);
+ TEST_ASSERT(in_params.max_nb == out_params.max_nb, "Expected %u got %u",
+ in_params.max_nb, out_params.max_nb);
+
+ /* Case 6: Set max_nb = 512 */
+ in_params.max_nb = 512;
+
+ err = rte_event_dma_adapter_runtime_params_set(TEST_ADAPTER_ID, &in_params);
+ TEST_ASSERT(err == 0, "Expected 0 got %d", err);
+
+ err = rte_event_dma_adapter_runtime_params_get(TEST_ADAPTER_ID, &out_params);
+ TEST_ASSERT(err == 0, "Expected 0 got %d", err);
+ TEST_ASSERT(in_params.max_nb == out_params.max_nb, "Expected %u got %u",
+ in_params.max_nb, out_params.max_nb);
+
+ rc = TEST_SUCCESS;
+vchan_del:
+ err = rte_event_dma_adapter_vchan_del(TEST_ADAPTER_ID, TEST_DMA_DEV_ID,
+ TEST_DMA_VCHAN_ID);
+ TEST_ASSERT_SUCCESS(err, "Failed to delete vchan\n");
+
+ return rc;
+}
+
+static int
+test_op_forward_mode(void)
+{
+ struct rte_mbuf *src_mbuf[TEST_MAX_OP];
+ struct rte_mbuf *dst_mbuf[TEST_MAX_OP];
+ struct rte_event_dma_adapter_op *op;
+ struct rte_event ev[TEST_MAX_OP];
+ struct rte_event response_info;
+ int ret, i;
+
+ ret = rte_pktmbuf_alloc_bulk(params.src_mbuf_pool, src_mbuf, TEST_MAX_OP);
+ TEST_ASSERT_SUCCESS(ret, "alloc src mbufs failed.\n");
+
+ ret = rte_pktmbuf_alloc_bulk(params.dst_mbuf_pool, dst_mbuf, TEST_MAX_OP);
+ TEST_ASSERT_SUCCESS(ret, "alloc dst mbufs failed.\n");
+
+ for (i = 0; i < TEST_MAX_OP; i++) {
+ memset(rte_pktmbuf_mtod(src_mbuf[i], void *), rte_rand(), PACKET_LENGTH);
+ memset(rte_pktmbuf_mtod(dst_mbuf[i], void *), 0, PACKET_LENGTH);
+ }
+
+ for (i = 0; i < TEST_MAX_OP; i++) {
+ rte_mempool_get(params.op_mpool, (void **)&op);
+ TEST_ASSERT_NOT_NULL(op, "Failed to allocate dma operation struct\n");
+
+ op->src_seg = rte_malloc(NULL, sizeof(struct rte_dma_sge), 0);
+ op->dst_seg = rte_malloc(NULL, sizeof(struct rte_dma_sge), 0);
+
+ /* Update Op */
+ op->src_seg->addr = rte_pktmbuf_iova(src_mbuf[i]);
+ op->dst_seg->addr = rte_pktmbuf_iova(dst_mbuf[i]);
+ op->src_seg->length = PACKET_LENGTH;
+ op->dst_seg->length = PACKET_LENGTH;
+ op->nb_src = 1;
+ op->nb_dst = 1;
+ op->flags = RTE_DMA_OP_FLAG_SUBMIT;
+ op->op_mp = params.op_mpool;
+ op->dma_dev_id = TEST_DMA_DEV_ID;
+ op->vchan = TEST_DMA_VCHAN_ID;
+
+ response_info.event = dma_response_info.event;
+ rte_memcpy((uint8_t *)op + sizeof(struct rte_event_dma_adapter_op), &response_info,
+ sizeof(struct rte_event));
+
+ /* Fill in event info and update event_ptr with rte_event_dma_adapter_op */
+ memset(&ev[i], 0, sizeof(struct rte_event));
+ ev[i].event = 0;
+ ev[i].event_type = RTE_EVENT_TYPE_DMADEV;
+ ev[i].queue_id = TEST_DMA_EV_QUEUE_ID;
+ ev[i].sched_type = RTE_SCHED_TYPE_ATOMIC;
+ ev[i].flow_id = 0xAABB;
+ ev[i].event_ptr = op;
+ }
+
+ ret = send_recv_ev(ev);
+ TEST_ASSERT_SUCCESS(ret, "Failed to send/receive event to dma adapter\n");
+
+ test_dma_adapter_stats();
+
+ for (i = 0; i < TEST_MAX_OP; i++) {
+ op = ev[i].event_ptr;
+ ret = memcmp(rte_pktmbuf_mtod(src_mbuf[i], void *),
+ rte_pktmbuf_mtod(dst_mbuf[i], void *), PACKET_LENGTH);
+
+ TEST_ASSERT_EQUAL(ret, 0, "Data mismatch for dma adapter\n");
+
+ rte_free(op->src_seg);
+ rte_free(op->dst_seg);
+ rte_mempool_put(op->op_mp, op);
+ }
+
+ rte_pktmbuf_free_bulk(src_mbuf, TEST_MAX_OP);
+ rte_pktmbuf_free_bulk(dst_mbuf, TEST_MAX_OP);
+
+ return TEST_SUCCESS;
+}
+
+static int
+map_adapter_service_core(void)
+{
+ uint32_t adapter_service_id;
+ int ret;
+
+ if (rte_event_dma_adapter_service_id_get(TEST_ADAPTER_ID, &adapter_service_id) == 0) {
+ uint32_t core_list[NUM_CORES];
+
+ ret = rte_service_lcore_list(core_list, NUM_CORES);
+ TEST_ASSERT(ret >= 0, "Failed to get service core list!");
+
+ if (core_list[0] != slcore_id) {
+ TEST_ASSERT_SUCCESS(rte_service_lcore_add(slcore_id),
+ "Failed to add service core");
+ TEST_ASSERT_SUCCESS(rte_service_lcore_start(slcore_id),
+ "Failed to start service core");
+ }
+
+ TEST_ASSERT_SUCCESS(rte_service_map_lcore_set(
+ adapter_service_id, slcore_id, 1),
+ "Failed to map adapter service");
+ }
+
+ return TEST_SUCCESS;
+}
+
+static int
+test_with_op_forward_mode(void)
+{
+ uint32_t cap;
+ int ret;
+
+ ret = rte_event_dma_adapter_caps_get(evdev, TEST_DMA_DEV_ID, &cap);
+ TEST_ASSERT_SUCCESS(ret, "Failed to get adapter capabilities\n");
+
+ if (!(cap & RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP_FWD) &&
+ !(cap & RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP_NEW))
+ map_adapter_service_core();
+ else {
+ if (!(cap & RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP_FWD))
+ return TEST_SKIPPED;
+ }
+
+ TEST_ASSERT_SUCCESS(rte_event_dma_adapter_start(TEST_ADAPTER_ID),
+ "Failed to start event dma adapter");
+
+ ret = test_op_forward_mode();
+ TEST_ASSERT_SUCCESS(ret, "DMA - FORWARD mode test failed\n");
+ return TEST_SUCCESS;
+}
+
+static int
+configure_dmadev(void)
+{
+ const struct rte_dma_conf conf = { .nb_vchans = 1};
+ const struct rte_dma_vchan_conf qconf = {
+ .direction = RTE_DMA_DIR_MEM_TO_MEM,
+ .nb_desc = TEST_RINGSIZE,
+ };
+ struct rte_dma_info info;
+ unsigned int elt_size;
+ int ret;
+
+ ret = rte_dma_count_avail();
+ RTE_TEST_ASSERT_FAIL(ret, "No dma devices found!\n");
+
+ ret = rte_dma_info_get(TEST_DMA_DEV_ID, &info);
+ TEST_ASSERT_SUCCESS(ret, "Error with rte_dma_info_get()\n");
+
+ if (info.max_vchans < 1)
+ RTE_LOG(ERR, USER1, "Error, no channels available on device id %u\n",
+ TEST_DMA_DEV_ID);
+
+ if (rte_dma_configure(TEST_DMA_DEV_ID, &conf) != 0)
+ RTE_LOG(ERR, USER1, "Error with rte_dma_configure()\n");
+
+ if (rte_dma_vchan_setup(TEST_DMA_DEV_ID, TEST_DMA_VCHAN_ID, &qconf) < 0)
+ RTE_LOG(ERR, USER1, "Error with vchan configuration\n");
+
+ ret = rte_dma_info_get(TEST_DMA_DEV_ID, &info);
+ if (ret != 0 || info.nb_vchans != 1)
+ RTE_LOG(ERR, USER1, "Error, no configured vhcan reported on device id %u\n",
+ TEST_DMA_DEV_ID);
+
+ params.src_mbuf_pool = rte_pktmbuf_pool_create("DMA_ADAPTER_SRC_MBUFPOOL", NUM_MBUFS,
+ MBUF_CACHE_SIZE, 0, MBUF_SIZE,
+ rte_socket_id());
+ RTE_TEST_ASSERT_NOT_NULL(params.src_mbuf_pool, "Can't create DMA_SRC_MBUFPOOL\n");
+
+ params.dst_mbuf_pool = rte_pktmbuf_pool_create("DMA_ADAPTER_DST_MBUFPOOL", NUM_MBUFS,
+ MBUF_CACHE_SIZE, 0, MBUF_SIZE,
+ rte_socket_id());
+ RTE_TEST_ASSERT_NOT_NULL(params.dst_mbuf_pool, "Can't create DMA_DST_MBUFPOOL\n");
+
+ elt_size = sizeof(struct rte_event_dma_adapter_op) + sizeof(struct rte_event);
+ params.op_mpool = rte_mempool_create("EVENT_DMA_OP_POOL", DMA_OP_POOL_SIZE, elt_size, 0,
+ 0, NULL, NULL, NULL, NULL, rte_socket_id(), 0);
+ RTE_TEST_ASSERT_NOT_NULL(params.op_mpool, "Can't create DMA_OP_POOL\n");
+
+ return TEST_SUCCESS;
+}
+
+static inline void
+evdev_set_conf_values(struct rte_event_dev_config *dev_conf, struct rte_event_dev_info *info)
+{
+ memset(dev_conf, 0, sizeof(struct rte_event_dev_config));
+ dev_conf->dequeue_timeout_ns = info->min_dequeue_timeout_ns;
+ dev_conf->nb_event_ports = NB_TEST_PORTS;
+ dev_conf->nb_event_queues = NB_TEST_QUEUES;
+ dev_conf->nb_event_queue_flows = info->max_event_queue_flows;
+ dev_conf->nb_event_port_dequeue_depth =
+ info->max_event_port_dequeue_depth;
+ dev_conf->nb_event_port_enqueue_depth =
+ info->max_event_port_enqueue_depth;
+ dev_conf->nb_event_port_enqueue_depth =
+ info->max_event_port_enqueue_depth;
+ dev_conf->nb_events_limit =
+ info->max_num_events;
+}
+
+static int
+configure_eventdev(void)
+{
+ struct rte_event_queue_conf queue_conf;
+ struct rte_event_dev_config devconf;
+ struct rte_event_dev_info info;
+ uint32_t queue_count;
+ uint32_t port_count;
+ uint8_t qid;
+ int ret;
+
+ if (!rte_event_dev_count()) {
+ /* If there is no hardware eventdev, or no software vdev was
+ * specified on the command line, create an instance of
+ * event_sw.
+ */
+ LOG_DBG("Failed to find a valid event device... "
+ "testing with event_sw device\n");
+ TEST_ASSERT_SUCCESS(rte_vdev_init("event_sw0", NULL),
+ "Error creating eventdev");
+ evdev = rte_event_dev_get_dev_id("event_sw0");
+ }
+
+ ret = rte_event_dev_info_get(evdev, &info);
+ TEST_ASSERT_SUCCESS(ret, "Failed to get event dev info\n");
+
+ evdev_set_conf_values(&devconf, &info);
+
+ ret = rte_event_dev_configure(evdev, &devconf);
+ TEST_ASSERT_SUCCESS(ret, "Failed to configure eventdev\n");
+
+ /* Set up event queue */
+ ret = rte_event_dev_attr_get(evdev, RTE_EVENT_DEV_ATTR_QUEUE_COUNT, &queue_count);
+ TEST_ASSERT_SUCCESS(ret, "Queue count get failed\n");
+ TEST_ASSERT_EQUAL(queue_count, 2, "Unexpected queue count\n");
+
+ qid = TEST_APP_EV_QUEUE_ID;
+ ret = rte_event_queue_setup(evdev, qid, NULL);
+ TEST_ASSERT_SUCCESS(ret, "Failed to setup queue=%d\n", qid);
+
+ queue_conf.nb_atomic_flows = info.max_event_queue_flows;
+ queue_conf.nb_atomic_order_sequences = 32;
+ queue_conf.schedule_type = RTE_SCHED_TYPE_ATOMIC;
+ queue_conf.priority = RTE_EVENT_DEV_PRIORITY_HIGHEST;
+ queue_conf.event_queue_cfg = RTE_EVENT_QUEUE_CFG_SINGLE_LINK;
+
+ qid = TEST_DMA_EV_QUEUE_ID;
+ ret = rte_event_queue_setup(evdev, qid, &queue_conf);
+ TEST_ASSERT_SUCCESS(ret, "Failed to setup queue=%u\n", qid);
+
+ /* Set up event port */
+ ret = rte_event_dev_attr_get(evdev, RTE_EVENT_DEV_ATTR_PORT_COUNT,
+ &port_count);
+ TEST_ASSERT_SUCCESS(ret, "Port count get failed\n");
+ TEST_ASSERT_EQUAL(port_count, 1, "Unexpected port count\n");
+
+ ret = rte_event_port_setup(evdev, TEST_APP_PORT_ID, NULL);
+ TEST_ASSERT_SUCCESS(ret, "Failed to setup port=%d\n",
+ TEST_APP_PORT_ID);
+
+ qid = TEST_APP_EV_QUEUE_ID;
+ ret = rte_event_port_link(evdev, TEST_APP_PORT_ID, &qid, NULL, 1);
+ TEST_ASSERT(ret >= 0, "Failed to link queue port=%d\n",
+ TEST_APP_PORT_ID);
+
+ return TEST_SUCCESS;
+}
+
+static void
+test_dma_adapter_free(void)
+{
+ rte_event_dma_adapter_free(TEST_ADAPTER_ID);
+}
+
+static int
+test_dma_adapter_create(void)
+{
+ struct rte_event_dev_info evdev_info = {0};
+ struct rte_event_port_conf conf = {0};
+ int ret;
+
+ ret = rte_event_dev_info_get(evdev, &evdev_info);
+ TEST_ASSERT_SUCCESS(ret, "Failed to create event dma adapter\n");
+
+ conf.new_event_threshold = evdev_info.max_num_events;
+ conf.dequeue_depth = evdev_info.max_event_port_dequeue_depth;
+ conf.enqueue_depth = evdev_info.max_event_port_enqueue_depth;
+
+ /* Create adapter with default port creation callback */
+ ret = rte_event_dma_adapter_create(TEST_ADAPTER_ID, evdev, &conf, 0);
+ TEST_ASSERT_SUCCESS(ret, "Failed to create event dma adapter\n");
+
+ return TEST_SUCCESS;
+}
+
+static int
+test_dma_adapter_vchan_add_del(void)
+{
+ struct rte_event event;
+ uint32_t cap;
+ int ret;
+
+ ret = rte_event_dma_adapter_caps_get(evdev, TEST_DMA_DEV_ID, &cap);
+ TEST_ASSERT_SUCCESS(ret, "Failed to get adapter capabilities\n");
+
+ if (cap & RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_VCHAN_EV_BIND) {
+ ret = rte_event_dma_adapter_vchan_add(TEST_ADAPTER_ID, TEST_DMA_DEV_ID,
+ TEST_DMA_VCHAN_ID, &event);
+ } else
+ ret = rte_event_dma_adapter_vchan_add(TEST_ADAPTER_ID, TEST_DMA_DEV_ID,
+ TEST_DMA_VCHAN_ID, NULL);
+
+ TEST_ASSERT_SUCCESS(ret, "Failed to create add vchan\n");
+
+ ret = rte_event_dma_adapter_vchan_del(TEST_ADAPTER_ID, TEST_DMA_DEV_ID,
+ TEST_DMA_VCHAN_ID);
+ TEST_ASSERT_SUCCESS(ret, "Failed to delete vchan\n");
+
+ return TEST_SUCCESS;
+}
+
+static int
+configure_event_dma_adapter(enum rte_event_dma_adapter_mode mode)
+{
+ struct rte_event_dev_info evdev_info = {0};
+ struct rte_event_port_conf conf = {0};
+ struct rte_event event;
+ uint32_t cap;
+ int ret;
+
+ ret = rte_event_dma_adapter_caps_get(evdev, TEST_DMA_DEV_ID, &cap);
+ TEST_ASSERT_SUCCESS(ret, "Failed to get adapter capabilities\n");
+
+ /* Skip mode and capability mismatch check for SW eventdev */
+ if (!(cap & RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP_NEW) &&
+ !(cap & RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP_FWD) &&
+ !(cap & RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_VCHAN_EV_BIND))
+ goto adapter_create;
+
+ if (mode == RTE_EVENT_DMA_ADAPTER_OP_FORWARD) {
+ if (cap & RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP_FWD)
+ params.internal_port_op_fwd = 1;
+ else
+ return -ENOTSUP;
+ }
+
+adapter_create:
+ ret = rte_event_dev_info_get(evdev, &evdev_info);
+ TEST_ASSERT_SUCCESS(ret, "Failed to create event dma adapter\n");
+
+ conf.new_event_threshold = evdev_info.max_num_events;
+ conf.dequeue_depth = evdev_info.max_event_port_dequeue_depth;
+ conf.enqueue_depth = evdev_info.max_event_port_enqueue_depth;
+
+ /* Create adapter with default port creation callback */
+ ret = rte_event_dma_adapter_create(TEST_ADAPTER_ID, evdev, &conf, mode);
+ TEST_ASSERT_SUCCESS(ret, "Failed to create event dma adapter\n");
+
+ if (cap & RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_VCHAN_EV_BIND) {
+ ret = rte_event_dma_adapter_vchan_add(TEST_ADAPTER_ID, TEST_DMA_DEV_ID,
+ TEST_DMA_VCHAN_ID, &event);
+ } else
+ ret = rte_event_dma_adapter_vchan_add(TEST_ADAPTER_ID, TEST_DMA_DEV_ID,
+ TEST_DMA_VCHAN_ID, NULL);
+
+ TEST_ASSERT_SUCCESS(ret, "Failed to add vchan\n");
+
+ if (!params.internal_port_op_fwd) {
+ ret = rte_event_dma_adapter_event_port_get(TEST_ADAPTER_ID,
+ ¶ms.dma_event_port_id);
+ TEST_ASSERT_SUCCESS(ret, "Failed to get event port\n");
+ }
+
+ return TEST_SUCCESS;
+}
+
+static void
+test_dma_adapter_stop(void)
+{
+ uint32_t evdev_service_id, adapter_service_id;
+
+ /* retrieve service ids & stop services */
+ if (rte_event_dma_adapter_service_id_get(TEST_ADAPTER_ID,
+ &adapter_service_id) == 0) {
+ rte_service_runstate_set(adapter_service_id, 0);
+ rte_service_lcore_stop(slcore_id);
+ rte_service_lcore_del(slcore_id);
+ rte_event_dma_adapter_stop(TEST_ADAPTER_ID);
+ }
+
+ if (rte_event_dev_service_id_get(evdev, &evdev_service_id) == 0) {
+ rte_service_runstate_set(evdev_service_id, 0);
+ rte_service_lcore_stop(slcore_id);
+ rte_service_lcore_del(slcore_id);
+ rte_dma_stop(TEST_DMA_DEV_ID);
+ rte_event_dev_stop(evdev);
+ } else {
+ rte_dma_stop(TEST_DMA_DEV_ID);
+ rte_event_dev_stop(evdev);
+ }
+}
+
+static int
+test_dma_adapter_conf(enum rte_event_dma_adapter_mode mode)
+{
+ uint32_t evdev_service_id;
+ uint8_t qid;
+ int ret;
+
+ if (!dma_adapter_setup_done) {
+ ret = configure_event_dma_adapter(mode);
+ if (ret)
+ return ret;
+ if (!params.internal_port_op_fwd) {
+ qid = TEST_DMA_EV_QUEUE_ID;
+ ret = rte_event_port_link(evdev,
+ params.dma_event_port_id, &qid, NULL, 1);
+ TEST_ASSERT(ret >= 0, "Failed to link queue %d "
+ "port=%u\n", qid,
+ params.dma_event_port_id);
+ }
+ dma_adapter_setup_done = 1;
+ }
+
+ /* retrieve service ids */
+ if (rte_event_dev_service_id_get(evdev, &evdev_service_id) == 0) {
+ /* add a service core and start it */
+ TEST_ASSERT_SUCCESS(rte_service_lcore_add(slcore_id),
+ "Failed to add service core");
+ TEST_ASSERT_SUCCESS(rte_service_lcore_start(slcore_id),
+ "Failed to start service core");
+
+ /* map services to it */
+ TEST_ASSERT_SUCCESS(rte_service_map_lcore_set(evdev_service_id,
+ slcore_id, 1), "Failed to map evdev service");
+
+ /* set services to running */
+ TEST_ASSERT_SUCCESS(rte_service_runstate_set(evdev_service_id,
+ 1), "Failed to start evdev service");
+ }
+
+ /* start the eventdev */
+ TEST_ASSERT_SUCCESS(rte_event_dev_start(evdev),
+ "Failed to start event device");
+
+ /* start the dma dev */
+ TEST_ASSERT_SUCCESS(rte_dma_start(TEST_DMA_DEV_ID),
+ "Failed to start dma device");
+
+ return TEST_SUCCESS;
+}
+
+static int
+test_dma_adapter_conf_op_forward_mode(void)
+{
+ enum rte_event_dma_adapter_mode mode;
+
+ mode = RTE_EVENT_DMA_ADAPTER_OP_FORWARD;
+
+ return test_dma_adapter_conf(mode);
+}
+
+static int
+testsuite_setup(void)
+{
+ int ret;
+
+ slcore_id = rte_get_next_lcore(-1, 1, 0);
+ TEST_ASSERT_NOT_EQUAL(slcore_id, RTE_MAX_LCORE, "At least 2 lcores "
+ "are required to run this autotest\n");
+
+ /* Setup and start event device. */
+ ret = configure_eventdev();
+ TEST_ASSERT_SUCCESS(ret, "Failed to setup eventdev\n");
+
+ /* Setup and start dma device. */
+ ret = configure_dmadev();
+ TEST_ASSERT_SUCCESS(ret, "dmadev initialization failed\n");
+
+ return TEST_SUCCESS;
+}
+
+static void
+dma_adapter_teardown(void)
+{
+ int ret;
+
+ ret = rte_event_dma_adapter_stop(TEST_ADAPTER_ID);
+ if (ret < 0)
+ RTE_LOG(ERR, USER1, "Failed to stop adapter!");
+
+ ret = rte_event_dma_adapter_vchan_del(TEST_ADAPTER_ID, TEST_DMA_DEV_ID,
+ TEST_DMA_VCHAN_ID);
+ if (ret < 0)
+ RTE_LOG(ERR, USER1, "Failed to delete vchan!");
+
+ ret = rte_event_dma_adapter_free(TEST_ADAPTER_ID);
+ if (ret < 0)
+ RTE_LOG(ERR, USER1, "Failed to free adapter!");
+
+ dma_adapter_setup_done = 0;
+}
+
+static void
+dma_teardown(void)
+{
+ /* Free mbuf mempool */
+ if (params.src_mbuf_pool != NULL) {
+ RTE_LOG(DEBUG, USER1, "DMA_ADAPTER_SRC_MBUFPOOL count %u\n",
+ rte_mempool_avail_count(params.src_mbuf_pool));
+ rte_mempool_free(params.src_mbuf_pool);
+ params.src_mbuf_pool = NULL;
+ }
+
+ if (params.dst_mbuf_pool != NULL) {
+ RTE_LOG(DEBUG, USER1, "DMA_ADAPTER_DST_MBUFPOOL count %u\n",
+ rte_mempool_avail_count(params.dst_mbuf_pool));
+ rte_mempool_free(params.dst_mbuf_pool);
+ params.dst_mbuf_pool = NULL;
+ }
+
+ /* Free ops mempool */
+ if (params.op_mpool != NULL) {
+ RTE_LOG(DEBUG, USER1, "EVENT_DMA_OP_POOL count %u\n",
+ rte_mempool_avail_count(params.op_mpool));
+ rte_mempool_free(params.op_mpool);
+ params.op_mpool = NULL;
+ }
+}
+
+static void
+eventdev_teardown(void)
+{
+ rte_event_dev_stop(evdev);
+}
+
+static void
+testsuite_teardown(void)
+{
+ dma_adapter_teardown();
+ dma_teardown();
+ eventdev_teardown();
+}
+
+static struct unit_test_suite functional_testsuite = {
+ .suite_name = "Event dma adapter test suite",
+ .setup = testsuite_setup,
+ .teardown = testsuite_teardown,
+ .unit_test_cases = {
+
+ TEST_CASE_ST(NULL, test_dma_adapter_free, test_dma_adapter_create),
+
+ TEST_CASE_ST(test_dma_adapter_create, test_dma_adapter_free,
+ test_dma_adapter_vchan_add_del),
+
+ TEST_CASE_ST(test_dma_adapter_create, test_dma_adapter_free,
+ test_dma_adapter_stats),
+
+ TEST_CASE_ST(test_dma_adapter_create, test_dma_adapter_free,
+ test_dma_adapter_params),
+
+ TEST_CASE_ST(test_dma_adapter_conf_op_forward_mode, test_dma_adapter_stop,
+ test_with_op_forward_mode),
+
+ TEST_CASES_END() /**< NULL terminate unit test array */
+ }
+};
+
+static int
+test_event_dma_adapter(void)
+{
+ return unit_test_suite_runner(&functional_testsuite);
+}
+
+#endif /* !RTE_EXEC_ENV_WINDOWS */
+
+REGISTER_TEST_COMMAND(event_dma_adapter_autotest, test_event_dma_adapter);
--
2.25.1
^ permalink raw reply [flat|nested] 110+ messages in thread
* Re: [PATCH v5 01/12] eventdev/dma: introduce DMA adapter
2023-09-28 10:36 ` [PATCH v5 01/12] eventdev/dma: introduce DMA adapter Amit Prakash Shukla
@ 2023-09-28 15:33 ` Jerin Jacob
0 siblings, 0 replies; 110+ messages in thread
From: Jerin Jacob @ 2023-09-28 15:33 UTC (permalink / raw)
To: Amit Prakash Shukla
Cc: Thomas Monjalon, Jerin Jacob, dev, fengchengwen, kevin.laatz,
bruce.richardson, conor.walsh, vattunuru, g.singh, sachin.saxena,
hemant.agrawal, cheng1.jiang, ndabilpuram, anoobj, mb
On Thu, Sep 28, 2023 at 4:46 PM Amit Prakash Shukla
<amitprakashs@marvell.com> wrote:
>
> Introduce event dma adapter interface to transfer packets between
> dma device and event device.
>
> Signed-off-by: Amit Prakash Shukla <amitprakashs@marvell.com>
> Acked-by: Jerin Jacob <jerinj@marvell.com>
> ---
>
> diff --git a/MAINTAINERS b/MAINTAINERS
> index a926155f26..e731a75d39 100644
> --- a/MAINTAINERS
> +++ b/MAINTAINERS
> @@ -540,6 +540,13 @@ F: lib/eventdev/*crypto_adapter*
> F: app/test/test_event_crypto_adapter.c
> F: doc/guides/prog_guide/event_crypto_adapter.rst
>
> +Eventdev DMA Adapter API
> +M: Amit Prakash Shukla <amitprakashs@marvell.com>
> +T: git://dpdk.org/next/dpdk-next-eventdev
> +F: lib/eventdev/*dma_adapter*
> +F: app/test/test_event_dma_adapter.c
Add this on last patch, i.e when this file introduced.
> +F: doc/guides/prog_guide/event_dma_adapter.rst
> +
> Raw device API
> M: Sachin Saxena <sachin.saxena@oss.nxp.com>
> M: Hemant Agrawal <hemant.agrawal@nxp.com>
> @@ -548,7 +555,6 @@ F: drivers/raw/skeleton/
> F: app/test/test_rawdev.c
> F: doc/guides/prog_guide/rawdev.rst
>
> -
Don't delete this line
^ permalink raw reply [flat|nested] 110+ messages in thread
* [PATCH v6 00/12] event DMA adapter library support
2023-09-28 10:36 ` [PATCH v5 00/12] event DMA adapter library support Amit Prakash Shukla
` (11 preceding siblings ...)
2023-09-28 10:36 ` [PATCH v5 12/12] app/test: add event DMA adapter auto-test Amit Prakash Shukla
@ 2023-09-28 16:49 ` Amit Prakash Shukla
2023-09-28 16:49 ` [PATCH v6 01/12] eventdev/dma: introduce DMA adapter Amit Prakash Shukla
` (12 more replies)
12 siblings, 13 replies; 110+ messages in thread
From: Amit Prakash Shukla @ 2023-09-28 16:49 UTC (permalink / raw)
Cc: dev, jerinj, fengchengwen, kevin.laatz, bruce.richardson,
conor.walsh, vattunuru, g.singh, sachin.saxena, hemant.agrawal,
cheng1.jiang, ndabilpuram, anoobj, mb, Amit Prakash Shukla
This series adds support for event DMA adapter library. API's defined
as part of this library can be used by the application for DMA transfer
of data using event based mechanism.
v6:
- Resolved review comments.
- Updated git commit message.
v5:
- Resolved review comments.
v4:
- Fixed compilation error.
v3:
- Resolved checkpatch warnings.
- Fixed compilation error on intel.
- Updated git commit message.
v2:
- Resolved review comments.
- Patch split into multiple patches.
Amit Prakash Shukla (12):
eventdev/dma: introduce DMA adapter
eventdev/dma: support adapter capabilities get
eventdev/dma: support adapter create and free
eventdev/dma: support vchan add and delete
eventdev/dma: support adapter service function
eventdev/dma: support adapter start and stop
eventdev/dma: support adapter service ID get
eventdev/dma: support adapter runtime params
eventdev/dma: support adapter stats
eventdev/dma: support adapter enqueue
eventdev/dma: support adapter event port get
app/test: add event DMA adapter auto-test
MAINTAINERS | 7 +
app/test/meson.build | 1 +
app/test/test_event_dma_adapter.c | 805 ++++++++++
config/rte_config.h | 1 +
doc/api/doxy-api-index.md | 1 +
doc/guides/eventdevs/features/default.ini | 8 +
doc/guides/prog_guide/event_dma_adapter.rst | 264 +++
doc/guides/prog_guide/eventdev.rst | 8 +-
.../img/event_dma_adapter_op_forward.svg | 1086 +++++++++++++
.../img/event_dma_adapter_op_new.svg | 1079 +++++++++++++
doc/guides/prog_guide/index.rst | 1 +
doc/guides/rel_notes/release_23_11.rst | 5 +
lib/eventdev/eventdev_pmd.h | 171 +-
lib/eventdev/eventdev_private.c | 10 +
lib/eventdev/meson.build | 4 +-
lib/eventdev/rte_event_dma_adapter.c | 1430 +++++++++++++++++
lib/eventdev/rte_event_dma_adapter.h | 581 +++++++
lib/eventdev/rte_eventdev.c | 23 +
lib/eventdev/rte_eventdev.h | 44 +
lib/eventdev/rte_eventdev_core.h | 8 +-
lib/eventdev/version.map | 16 +
lib/meson.build | 2 +-
22 files changed, 5548 insertions(+), 7 deletions(-)
create mode 100644 app/test/test_event_dma_adapter.c
create mode 100644 doc/guides/prog_guide/event_dma_adapter.rst
create mode 100644 doc/guides/prog_guide/img/event_dma_adapter_op_forward.svg
create mode 100644 doc/guides/prog_guide/img/event_dma_adapter_op_new.svg
create mode 100644 lib/eventdev/rte_event_dma_adapter.c
create mode 100644 lib/eventdev/rte_event_dma_adapter.h
--
2.25.1
^ permalink raw reply [flat|nested] 110+ messages in thread
* [PATCH v6 01/12] eventdev/dma: introduce DMA adapter
2023-09-28 16:49 ` [PATCH v6 00/12] event DMA adapter library support Amit Prakash Shukla
@ 2023-09-28 16:49 ` Amit Prakash Shukla
2023-09-28 16:49 ` [PATCH v6 02/12] eventdev/dma: support adapter capabilities get Amit Prakash Shukla
` (11 subsequent siblings)
12 siblings, 0 replies; 110+ messages in thread
From: Amit Prakash Shukla @ 2023-09-28 16:49 UTC (permalink / raw)
To: Thomas Monjalon, Amit Prakash Shukla, Jerin Jacob
Cc: dev, fengchengwen, kevin.laatz, bruce.richardson, conor.walsh,
vattunuru, g.singh, sachin.saxena, hemant.agrawal, cheng1.jiang,
ndabilpuram, anoobj, mb
Introduce event dma adapter interface to transfer packets between
dma device and event device.
Signed-off-by: Amit Prakash Shukla <amitprakashs@marvell.com>
Acked-by: Jerin Jacob <jerinj@marvell.com>
---
MAINTAINERS | 6 +
doc/api/doxy-api-index.md | 1 +
doc/guides/eventdevs/features/default.ini | 8 +
doc/guides/prog_guide/event_dma_adapter.rst | 264 ++++
doc/guides/prog_guide/eventdev.rst | 8 +-
.../img/event_dma_adapter_op_forward.svg | 1086 +++++++++++++++++
.../img/event_dma_adapter_op_new.svg | 1079 ++++++++++++++++
doc/guides/prog_guide/index.rst | 1 +
doc/guides/rel_notes/release_23_11.rst | 5 +
lib/eventdev/eventdev_pmd.h | 171 ++-
lib/eventdev/eventdev_private.c | 10 +
lib/eventdev/meson.build | 1 +
lib/eventdev/rte_event_dma_adapter.h | 581 +++++++++
lib/eventdev/rte_eventdev.h | 44 +
lib/eventdev/rte_eventdev_core.h | 8 +-
lib/eventdev/version.map | 16 +
lib/meson.build | 2 +-
17 files changed, 3285 insertions(+), 6 deletions(-)
create mode 100644 doc/guides/prog_guide/event_dma_adapter.rst
create mode 100644 doc/guides/prog_guide/img/event_dma_adapter_op_forward.svg
create mode 100644 doc/guides/prog_guide/img/event_dma_adapter_op_new.svg
create mode 100644 lib/eventdev/rte_event_dma_adapter.h
diff --git a/MAINTAINERS b/MAINTAINERS
index a926155f26..4ebbbe8bb3 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -540,6 +540,12 @@ F: lib/eventdev/*crypto_adapter*
F: app/test/test_event_crypto_adapter.c
F: doc/guides/prog_guide/event_crypto_adapter.rst
+Eventdev DMA Adapter API
+M: Amit Prakash Shukla <amitprakashs@marvell.com>
+T: git://dpdk.org/next/dpdk-next-eventdev
+F: lib/eventdev/*dma_adapter*
+F: doc/guides/prog_guide/event_dma_adapter.rst
+
Raw device API
M: Sachin Saxena <sachin.saxena@oss.nxp.com>
M: Hemant Agrawal <hemant.agrawal@nxp.com>
diff --git a/doc/api/doxy-api-index.md b/doc/api/doxy-api-index.md
index fdeda13932..b7df7be4d9 100644
--- a/doc/api/doxy-api-index.md
+++ b/doc/api/doxy-api-index.md
@@ -29,6 +29,7 @@ The public API headers are grouped by topics:
[event_eth_tx_adapter](@ref rte_event_eth_tx_adapter.h),
[event_timer_adapter](@ref rte_event_timer_adapter.h),
[event_crypto_adapter](@ref rte_event_crypto_adapter.h),
+ [event_dma_adapter](@ref rte_event_dma_adapter.h),
[rawdev](@ref rte_rawdev.h),
[metrics](@ref rte_metrics.h),
[bitrate](@ref rte_bitrate.h),
diff --git a/doc/guides/eventdevs/features/default.ini b/doc/guides/eventdevs/features/default.ini
index 00360f60c6..73a52d915b 100644
--- a/doc/guides/eventdevs/features/default.ini
+++ b/doc/guides/eventdevs/features/default.ini
@@ -44,6 +44,14 @@ internal_port_op_fwd =
internal_port_qp_ev_bind =
session_private_data =
+;
+; Features of a default DMA adapter.
+;
+[DMA adapter Features]
+internal_port_op_new =
+internal_port_op_fwd =
+internal_port_vchan_ev_bind =
+
;
; Features of a default Timer adapter.
;
diff --git a/doc/guides/prog_guide/event_dma_adapter.rst b/doc/guides/prog_guide/event_dma_adapter.rst
new file mode 100644
index 0000000000..701e50d042
--- /dev/null
+++ b/doc/guides/prog_guide/event_dma_adapter.rst
@@ -0,0 +1,264 @@
+.. SPDX-License-Identifier: BSD-3-Clause
+ Copyright (c) 2023 Marvell.
+
+Event DMA Adapter Library
+=========================
+
+DPDK :doc:`Eventdev library <eventdev>` provides event driven programming model with features
+to schedule events. :doc:`DMA Device library <dmadev>` provides an interface to DMA poll mode
+drivers that support DMA operations. Event DMA Adapter is intended to bridge between the event
+device and the DMA device.
+
+Packet flow from DMA device to the event device can be accomplished using software and hardware
+based transfer mechanisms. The adapter queries an eventdev PMD to determine which mechanism to
+be used. The adapter uses an EAL service core function for software based packet transfer and
+uses the eventdev PMD functions to configure hardware based packet transfer between DMA device
+and the event device. DMA adapter uses a new event type called ``RTE_EVENT_TYPE_DMADEV`` to
+indicate the source of event.
+
+Application can choose to submit an DMA operation directly to an DMA device or send it to an DMA
+adapter via eventdev based on ``RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP_FWD`` capability. The
+first mode is known as the event new (``RTE_EVENT_DMA_ADAPTER_OP_NEW``) mode and the second as the
+event forward (``RTE_EVENT_DMA_ADAPTER_OP_FORWARD``) mode. Choice of mode can be specified while
+creating the adapter. In the former mode, it is the application's responsibility to enable
+ingress packet ordering. In the latter mode, it is the adapter's responsibility to enable
+ingress packet ordering.
+
+
+Adapter Modes
+-------------
+
+RTE_EVENT_DMA_ADAPTER_OP_NEW mode
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+In the ``RTE_EVENT_DMA_ADAPTER_OP_NEW`` mode, application submits DMA operations directly to an DMA
+device. The adapter then dequeues DMA completions from the DMA device and enqueues them as events
+to the event device. This mode does not ensure ingress ordering as the application directly
+enqueues to the dmadev without going through DMA/atomic stage. In this mode, events dequeued
+from the adapter are treated as new events. The application has to specify event information
+(response information) which is needed to enqueue an event after the DMA operation is completed.
+
+.. _figure_event_dma_adapter_op_new:
+
+.. figure:: img/event_dma_adapter_op_new.*
+
+ Working model of ``RTE_EVENT_DMA_ADAPTER_OP_NEW`` mode
+
+
+RTE_EVENT_DMA_ADAPTER_OP_FORWARD mode
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+In the ``RTE_EVENT_DMA_ADAPTER_OP_FORWARD`` mode, if the event PMD and DMA PMD supports internal
+event port (``RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP_FWD``), the application should use
+``rte_event_dma_adapter_enqueue()`` API to enqueue DMA operations as events to DMA adapter. If
+not, application retrieves DMA adapter's event port using ``rte_event_dma_adapter_event_port_get()``
+API, links its event queue to this port and starts enqueuing DMA operations as events to eventdev
+using ``rte_event_enqueue_burst()``. The adapter then dequeues the events and submits the DMA
+operations to the dmadev. After the DMA operation is complete, the adapter enqueues events to the
+event device.
+
+Applications can use this mode when ingress packet ordering is needed. In this mode, events
+dequeued from the adapter will be treated as forwarded events. Application has to specify event
+information (response information) needed to enqueue the event after the DMA operation has
+completed.
+
+.. _figure_event_dma_adapter_op_forward:
+
+.. figure:: img/event_dma_adapter_op_forward.*
+
+ Working model of ``RTE_EVENT_DMA_ADAPTER_OP_FORWARD`` mode
+
+
+API Overview
+------------
+
+This section has a brief introduction to the event DMA adapter APIs. The application is expected
+to create an adapter which is associated with a single eventdev, then add dmadev and vchan to the
+adapter instance.
+
+
+Create an adapter instance
+~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+An adapter instance is created using ``rte_event_dma_adapter_create()``. This function is called
+with event device to be associated with the adapter and port configuration for the adapter to
+setup an event port (if the adapter needs to use a service function).
+
+Adapter can be started in ``RTE_EVENT_DMA_ADAPTER_OP_NEW`` or ``RTE_EVENT_DMA_ADAPTER_OP_FORWARD``
+mode.
+
+.. code-block:: c
+
+ enum rte_event_dma_adapter_mode mode;
+ struct rte_event_dev_info dev_info;
+ struct rte_event_port_conf conf;
+ uint8_t evdev_id;
+ uint8_t dma_id;
+ int ret;
+
+ ret = rte_event_dev_info_get(dma_id, &dev_info);
+
+ conf.new_event_threshold = dev_info.max_num_events;
+ conf.dequeue_depth = dev_info.max_event_port_dequeue_depth;
+ conf.enqueue_depth = dev_info.max_event_port_enqueue_depth;
+ mode = RTE_EVENT_DMA_ADAPTER_OP_FORWARD;
+ ret = rte_event_dma_adapter_create(dma_id, evdev_id, &conf, mode);
+
+
+``rte_event_dma_adapter_create_ext()`` function can be used by the application to have a finer
+control on eventdev port allocation and setup. The ``rte_event_dma_adapter_create_ext()``
+function is passed a callback function. The callback function is invoked if the adapter creates
+a service function and uses an event port for it. The callback is expected to fill the
+``struct rte_event_dma_adapter_conf`` structure passed to it.
+
+In the ``RTE_EVENT_DMA_ADAPTER_OP_FORWARD`` mode, if the event PMD and DMA PMD supports internal
+event port (``RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP_FWD``), events with DMA operations should
+be enqueued to the DMA adapter using ``rte_event_dma_adapter_enqueue()`` API. If not, the event port
+created by the adapter can be retrieved using ``rte_event_dma_adapter_event_port_get()`` API. An
+application can use this event port to link with an event queue, on which it enqueues events
+towards the DMA adapter using ``rte_event_enqueue_burst()``.
+
+.. code-block:: c
+
+ uint8_t dma_adpt_id, evdev_id, dma_dev_id, dma_ev_port_id, app_qid;
+ struct rte_event ev;
+ uint32_t cap;
+ int ret;
+
+ // Fill in event info and update event_ptr with rte_dma_op
+ memset(&ev, 0, sizeof(ev));
+ .
+ .
+ ev.event_ptr = op;
+
+ ret = rte_event_dma_adapter_caps_get(evdev_id, dma_dev_id, &cap);
+ if (cap & RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP_FWD) {
+ ret = rte_event_dma_adapter_enqueue(evdev_id, app_ev_port_id, ev, nb_events);
+ } else {
+ ret = rte_event_dma_adapter_event_port_get(dma_adpt_id, &dma_ev_port_id);
+ ret = rte_event_queue_setup(evdev_id, app_qid, NULL);
+ ret = rte_event_port_link(evdev_id, dma_ev_port_id, &app_qid, NULL, 1);
+ ev.queue_id = app_qid;
+ ret = rte_event_enqueue_burst(evdev_id, app_ev_port_id, ev, nb_events);
+ }
+
+
+Event device configuration for service based adapter
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+When ``rte_event_dma_adapter_create()`` is used for creating adapter instance,
+``rte_event_dev_config::nb_event_ports`` is automatically incremented, and event device is
+reconfigured with additional event port during service initialization. This event device
+reconfigure logic also increments the ``rte_event_dev_config::nb_single_link_event_port_queues``
+parameter if the adapter event port config is of type ``RTE_EVENT_PORT_CFG_SINGLE_LINK``.
+
+Applications using this mode of adapter creation need not configure the event device with
+``rte_event_dev_config::nb_event_ports`` and
+``rte_event_dev_config::nb_single_link_event_port_queues`` parameters required for DMA adapter when
+the adapter is created using the above-mentioned API.
+
+
+Querying adapter capabilities
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+The ``rte_event_dma_adapter_caps_get()`` function allows the application to query the adapter
+capabilities for an eventdev and dmadev combination. This API provides whether dmadev and eventdev
+are connected using internal HW port or not.
+
+.. code-block:: c
+
+ rte_event_dma_adapter_caps_get(dev_id, dma_dev_id, &cap);
+
+
+Adding vchan to the adapter instance
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+dmadev device id and vchan are configured using dmadev APIs. For more information
+see :doc:`here <dmadev>`.
+
+.. code-block:: c
+
+ struct rte_dma_vchan_conf vchan_conf;
+ struct rte_dma_conf dev_conf;
+ uint8_t dev_id = 0;
+ uint16_t vchan = 0;
+
+ rte_dma_configure(dev_id, &dev_conf);
+ rte_dma_vchan_setup(dev_id, vchan, &vchan_conf);
+
+These dmadev id and vchan are added to the instance using the
+``rte_event_dma_adapter_vchan_add()`` API. The same is removed using
+``rte_event_dma_adapter_vchan_del()`` API. If hardware supports
+``RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_VCHAN_EV_BIND`` capability, event information must be passed to the add API.
+
+.. code-block:: c
+
+ uint32_t cap;
+ int ret;
+
+ ret = rte_event_dma_adapter_caps_get(evdev_id, dma_dev_id, &cap);
+ if (cap & RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_VCHAN_EV_BIND) {
+ struct rte_event event;
+
+ rte_event_dma_adapter_vchan_add(id, dma_dev_id, vchan, &conf);
+ } else
+ rte_event_dma_adapter_vchan_add(id, dma_dev_id, vchan, NULL);
+
+
+Configuring service function
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+If the adapter uses a service function, the application is required to assign a service core to
+the service function as show below.
+
+.. code-block:: c
+
+ uint32_t service_id;
+
+ if (rte_event_dma_adapter_service_id_get(dma_id, &service_id) == 0)
+ rte_service_map_lcore_set(service_id, CORE_ID);
+
+
+Set event response information
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+In the RTE_EVENT_DMA_ADAPTER_OP_FORWARD / RTE_EVENT_DMA_ADAPTER_OP_NEW mode, the application
+specifies the dmadev ID and vchan ID in ``struct rte_event_dma_adapter_op`` and the event
+information (response information) needed to enqueue an event after the DMA operation has
+completed. The response information is specified in ``struct rte_event`` and appended to the
+``struct rte_event_dma_adapter_op``.
+
+
+Start the adapter instance
+~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+The application calls ``rte_event_dma_adapter_start()`` to start the adapter. This function calls
+the start callbacks of the eventdev PMDs for hardware based eventdev-dmadev connections and
+``rte_service_run_state_set()`` to enable the service function if one exists.
+
+.. code-block:: c
+
+ rte_event_dma_adapter_start(id);
+
+.. Note::
+
+ The eventdev to which the event_dma_adapter is connected should be started before calling
+ rte_event_dma_adapter_start().
+
+
+Get adapter statistics
+~~~~~~~~~~~~~~~~~~~~~~
+
+The ``rte_event_dma_adapter_stats_get()`` function reports counters defined in struct
+``rte_event_dma_adapter_stats``. The received packet and enqueued event counts are a sum of the
+counts from the eventdev PMD callbacks if the callback is supported, and the counts maintained by
+the service function, if one exists.
+
+Set/Get adapter runtime configuration parameters
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+The runtime configuration parameters of adapter can be set/get using
+``rte_event_dma_adapter_runtime_params_set()`` and
+``rte_event_dma_adapter_runtime_params_get()`` respectively.
+The parameters that can be set/get are defined in
+``struct rte_event_dma_adapter_runtime_params``.
diff --git a/doc/guides/prog_guide/eventdev.rst b/doc/guides/prog_guide/eventdev.rst
index 2c83176846..ff55115d0d 100644
--- a/doc/guides/prog_guide/eventdev.rst
+++ b/doc/guides/prog_guide/eventdev.rst
@@ -333,7 +333,8 @@ eventdev.
.. Note::
EventDev needs to be started before starting the event producers such
- as event_eth_rx_adapter, event_timer_adapter and event_crypto_adapter.
+ as event_eth_rx_adapter, event_timer_adapter, event_crypto_adapter and
+ event_dma_adapter.
Ingress of New Events
~~~~~~~~~~~~~~~~~~~~~
@@ -445,8 +446,9 @@ using ``rte_event_dev_stop_flush_callback_register()`` function.
.. Note::
The event producers such as ``event_eth_rx_adapter``,
- ``event_timer_adapter`` and ``event_crypto_adapter``
- need to be stopped before stopping the event device.
+ ``event_timer_adapter``, ``event_crypto_adapter`` and
+ ``event_dma_adapter`` need to be stopped before stopping
+ the event device.
Summary
-------
diff --git a/doc/guides/prog_guide/img/event_dma_adapter_op_forward.svg b/doc/guides/prog_guide/img/event_dma_adapter_op_forward.svg
new file mode 100644
index 0000000000..b7fe1fecf2
--- /dev/null
+++ b/doc/guides/prog_guide/img/event_dma_adapter_op_forward.svg
@@ -0,0 +1,1086 @@
+<?xml version="1.0" encoding="UTF-8" standalone="no"?>
+<!-- SPDX-License-Identifier: BSD-3-Clause -->
+<!-- Copyright (c) 2023 Marvell. -->
+<!-- Created with Inkscape (http://www.inkscape.org/) -->
+
+<svg
+ width="720px"
+ height="486px"
+ id="svg13237"
+ version="1.1"
+ inkscape:version="1.2.1 (9c6d41e410, 2022-07-14)"
+ sodipodi:docname="event_dma_adapter_op_forward.svg"
+ xmlns:inkscape="http://www.inkscape.org/namespaces/inkscape"
+ xmlns:sodipodi="http://sodipodi.sourceforge.net/DTD/sodipodi-0.dtd"
+ xmlns="http://www.w3.org/2000/svg"
+ xmlns:svg="http://www.w3.org/2000/svg"
+ xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#"
+ xmlns:cc="http://creativecommons.org/ns#"
+ xmlns:dc="http://purl.org/dc/elements/1.1/"
+ xmlns:v="http://schemas.microsoft.com/visio/2003/SVGExtensions/">
+ <defs
+ id="defs13239">
+ <marker
+ inkscape:stockid="Arrow1Sstart"
+ orient="auto"
+ refY="0.0"
+ refX="0.0"
+ id="Arrow1Sstart"
+ style="overflow:visible">
+ <path
+ id="path8416"
+ d="M 0.0,0.0 L 5.0,-5.0 L -12.5,0.0 L 5.0,5.0 L 0.0,0.0 z "
+ style="fill-rule:evenodd;stroke:#000000;stroke-width:1.0pt"
+ transform="scale(0.2) translate(6,0)" />
+ </marker>
+ <marker
+ inkscape:stockid="Arrow1Send"
+ orient="auto"
+ refY="0.0"
+ refX="0.0"
+ id="Arrow1Send"
+ style="overflow:visible;">
+ <path
+ id="path8419"
+ d="M 0.0,0.0 L 5.0,-5.0 L -12.5,0.0 L 5.0,5.0 L 0.0,0.0 z "
+ style="fill-rule:evenodd;stroke:#000000;stroke-width:1.0pt;"
+ transform="scale(0.2) rotate(180) translate(6,0)" />
+ </marker>
+ <marker
+ inkscape:stockid="DiamondL"
+ orient="auto"
+ refY="0.0"
+ refX="0.0"
+ id="DiamondL"
+ style="overflow:visible">
+ <path
+ id="path8483"
+ d="M 0,-7.0710768 L -7.0710894,0 L 0,7.0710589 L 7.0710462,0 L 0,-7.0710768 z "
+ style="fill-rule:evenodd;stroke:#000000;stroke-width:1.0pt"
+ transform="scale(0.8)" />
+ </marker>
+ <marker
+ inkscape:stockid="DotL"
+ orient="auto"
+ refY="0.0"
+ refX="0.0"
+ id="DotL"
+ style="overflow:visible">
+ <path
+ id="path8465"
+ d="M -2.5,-1.0 C -2.5,1.7600000 -4.7400000,4.0 -7.5,4.0 C -10.260000,4.0 -12.5,1.7600000 -12.5,-1.0 C -12.5,-3.7600000 -10.260000,-6.0 -7.5,-6.0 C -4.7400000,-6.0 -2.5,-3.7600000 -2.5,-1.0 z "
+ style="fill-rule:evenodd;stroke:#000000;stroke-width:1.0pt"
+ transform="scale(0.8) translate(7.4, 1)" />
+ </marker>
+ <marker
+ inkscape:stockid="SquareL"
+ orient="auto"
+ refY="0.0"
+ refX="0.0"
+ id="SquareL"
+ style="overflow:visible">
+ <path
+ id="path8474"
+ d="M -5.0,-5.0 L -5.0,5.0 L 5.0,5.0 L 5.0,-5.0 L -5.0,-5.0 z "
+ style="fill-rule:evenodd;stroke:#000000;stroke-width:1.0pt"
+ transform="scale(0.8)" />
+ </marker>
+ <marker
+ inkscape:stockid="TriangleOutL"
+ orient="auto"
+ refY="0.0"
+ refX="0.0"
+ id="TriangleOutL"
+ style="overflow:visible">
+ <path
+ id="path8546"
+ d="M 5.77,0.0 L -2.88,5.0 L -2.88,-5.0 L 5.77,0.0 z "
+ style="fill-rule:evenodd;stroke:#000000;stroke-width:1.0pt"
+ transform="scale(0.8)" />
+ </marker>
+ <marker
+ inkscape:stockid="Arrow1Lstart"
+ orient="auto"
+ refY="0.0"
+ refX="0.0"
+ id="Arrow1Lstart"
+ style="overflow:visible">
+ <path
+ id="path8404"
+ d="M 0.0,0.0 L 5.0,-5.0 L -12.5,0.0 L 5.0,5.0 L 0.0,0.0 z "
+ style="fill-rule:evenodd;stroke:#000000;stroke-width:1.0pt"
+ transform="scale(0.8) translate(12.5,0)" />
+ </marker>
+ <marker
+ inkscape:stockid="Arrow1Mend"
+ orient="auto"
+ refY="0.0"
+ refX="0.0"
+ id="Arrow1Mend"
+ style="overflow:visible;">
+ <path
+ id="path8413"
+ d="M 0.0,0.0 L 5.0,-5.0 L -12.5,0.0 L 5.0,5.0 L 0.0,0.0 z "
+ style="fill-rule:evenodd;stroke:#000000;stroke-width:1.0pt;"
+ transform="scale(0.4) rotate(180) translate(10,0)" />
+ </marker>
+ <marker
+ inkscape:stockid="Arrow2Lend"
+ orient="auto"
+ refY="0.0"
+ refX="0.0"
+ id="Arrow2Lend"
+ style="overflow:visible;">
+ <path
+ id="path8425"
+ style="fill-rule:evenodd;stroke-width:0.62500000;stroke-linejoin:round;"
+ d="M 8.7185878,4.0337352 L -2.2072895,0.016013256 L 8.7185884,-4.0017078 C 6.9730900,-1.6296469 6.9831476,1.6157441 8.7185878,4.0337352 z "
+ transform="scale(1.1) rotate(180) translate(1,0)" />
+ </marker>
+ <marker
+ inkscape:stockid="Arrow1Lend"
+ orient="auto"
+ refY="0.0"
+ refX="0.0"
+ id="Arrow1Lend"
+ style="overflow:visible;">
+ <path
+ id="path8407"
+ d="M 0.0,0.0 L 5.0,-5.0 L -12.5,0.0 L 5.0,5.0 L 0.0,0.0 z "
+ style="fill-rule:evenodd;stroke:#000000;stroke-width:1.0pt;"
+ transform="scale(0.8) rotate(180) translate(12.5,0)" />
+ </marker>
+ <filter
+ id="filter_2"
+ color-interpolation-filters="sRGB">
+ <feGaussianBlur
+ stdDeviation="2"
+ id="feGaussianBlur15" />
+ </filter>
+ <filter
+ id="filter_2-3"
+ color-interpolation-filters="sRGB"
+ x="-0.086178862"
+ y="-0.086178862"
+ width="1.1723577"
+ height="1.1723577">
+ <feGaussianBlur
+ stdDeviation="2"
+ id="feGaussianBlur15-1" />
+ </filter>
+ <filter
+ id="filter_2-0"
+ color-interpolation-filters="sRGB">
+ <feGaussianBlur
+ stdDeviation="2"
+ id="feGaussianBlur15-7" />
+ </filter>
+ <filter
+ id="filter_2-0-8"
+ color-interpolation-filters="sRGB">
+ <feGaussianBlur
+ stdDeviation="2"
+ id="feGaussianBlur15-7-7" />
+ </filter>
+ <filter
+ id="filter_2-3-9"
+ color-interpolation-filters="sRGB">
+ <feGaussianBlur
+ stdDeviation="2"
+ id="feGaussianBlur15-1-6" />
+ </filter>
+ <filter
+ id="filter_2-3-6"
+ color-interpolation-filters="sRGB">
+ <feGaussianBlur
+ stdDeviation="2"
+ id="feGaussianBlur15-1-63" />
+ </filter>
+ <filter
+ id="filter_2-3-91"
+ color-interpolation-filters="sRGB"
+ x="-0.086178862"
+ y="-0.086178862"
+ width="1.1723577"
+ height="1.1723577">
+ <feGaussianBlur
+ stdDeviation="2"
+ id="feGaussianBlur15-1-3" />
+ </filter>
+ <marker
+ inkscape:stockid="Arrow1Lend"
+ orient="auto"
+ refY="0"
+ refX="0"
+ id="Arrow1Lend-5"
+ style="overflow:visible">
+ <path
+ inkscape:connector-curvature="0"
+ id="path8407-3"
+ d="M 0,0 5,-5 -12.5,0 5,5 0,0 z"
+ style="fill-rule:evenodd;stroke:#000000;stroke-width:1pt"
+ transform="matrix(-0.8,0,0,-0.8,-10,0)" />
+ </marker>
+ <marker
+ inkscape:stockid="Arrow1Lend"
+ orient="auto"
+ refY="0"
+ refX="0"
+ id="Arrow1Lend-6"
+ style="overflow:visible">
+ <path
+ inkscape:connector-curvature="0"
+ id="path8407-0"
+ d="M 0,0 5,-5 -12.5,0 5,5 0,0 z"
+ style="fill-rule:evenodd;stroke:#000000;stroke-width:1pt"
+ transform="matrix(-0.8,0,0,-0.8,-10,0)" />
+ </marker>
+ <marker
+ inkscape:stockid="Arrow1Lstart"
+ orient="auto"
+ refY="0"
+ refX="0"
+ id="Arrow1Lstart-7"
+ style="overflow:visible">
+ <path
+ inkscape:connector-curvature="0"
+ id="path8404-0"
+ d="M 0,0 5,-5 -12.5,0 5,5 0,0 z"
+ style="fill-rule:evenodd;stroke:#000000;stroke-width:1pt"
+ transform="matrix(0.8,0,0,0.8,10,0)" />
+ </marker>
+ <marker
+ inkscape:stockid="Arrow1Lend"
+ orient="auto"
+ refY="0"
+ refX="0"
+ id="Arrow1Lend-51"
+ style="overflow:visible">
+ <path
+ inkscape:connector-curvature="0"
+ id="path8407-1"
+ d="M 0,0 5,-5 -12.5,0 5,5 0,0 z"
+ style="fill-rule:evenodd;stroke:#000000;stroke-width:1pt"
+ transform="matrix(-0.8,0,0,-0.8,-10,0)" />
+ </marker>
+ <marker
+ inkscape:stockid="Arrow1Lend"
+ orient="auto"
+ refY="0"
+ refX="0"
+ id="Arrow1Lend-3"
+ style="overflow:visible">
+ <path
+ inkscape:connector-curvature="0"
+ id="path8407-6"
+ d="M 0,0 5,-5 -12.5,0 5,5 0,0 z"
+ style="fill-rule:evenodd;stroke:#000000;stroke-width:1pt"
+ transform="matrix(-0.8,0,0,-0.8,-10,0)" />
+ </marker>
+ <marker
+ inkscape:stockid="Arrow1Lend"
+ orient="auto"
+ refY="0"
+ refX="0"
+ id="Arrow1Lend-62"
+ style="overflow:visible">
+ <path
+ inkscape:connector-curvature="0"
+ id="path8407-9"
+ d="M 0,0 5,-5 -12.5,0 5,5 0,0 z"
+ style="fill-rule:evenodd;stroke:#000000;stroke-width:1pt"
+ transform="matrix(-0.8,0,0,-0.8,-10,0)" />
+ </marker>
+ <marker
+ inkscape:stockid="Arrow1Lend"
+ orient="auto"
+ refY="0"
+ refX="0"
+ id="Arrow1Lend-2"
+ style="overflow:visible">
+ <path
+ inkscape:connector-curvature="0"
+ id="path8407-7"
+ d="M 0,0 5,-5 -12.5,0 5,5 0,0 z"
+ style="fill-rule:evenodd;stroke:#000000;stroke-width:1pt"
+ transform="matrix(-0.8,0,0,-0.8,-10,0)" />
+ </marker>
+ <marker
+ inkscape:stockid="Arrow1Lstart"
+ orient="auto"
+ refY="0"
+ refX="0"
+ id="Arrow1Lstart-7-9"
+ style="overflow:visible">
+ <path
+ inkscape:connector-curvature="0"
+ id="path8404-0-3"
+ d="M 0,0 5,-5 -12.5,0 5,5 0,0 z"
+ style="fill-rule:evenodd;stroke:#000000;stroke-width:1pt"
+ transform="matrix(0.8,0,0,0.8,10,0)" />
+ </marker>
+ <filter
+ id="filter_2-3-6-1"
+ color-interpolation-filters="sRGB">
+ <feGaussianBlur
+ stdDeviation="2"
+ id="feGaussianBlur15-1-63-8" />
+ </filter>
+ <filter
+ id="filter_2-3-92"
+ color-interpolation-filters="sRGB"
+ x="-0.086178862"
+ y="-0.086178862"
+ width="1.1723577"
+ height="1.1723577">
+ <feGaussianBlur
+ stdDeviation="2"
+ id="feGaussianBlur15-1-2" />
+ </filter>
+ <filter
+ id="filter_2-3-94"
+ color-interpolation-filters="sRGB"
+ x="-0.086178862"
+ y="-0.086178862"
+ width="1.1723577"
+ height="1.1723577">
+ <feGaussianBlur
+ stdDeviation="2"
+ id="feGaussianBlur15-1-7" />
+ </filter>
+ <marker
+ inkscape:stockid="Arrow1Lstart"
+ orient="auto"
+ refY="0"
+ refX="0"
+ id="Arrow1Lstart-7-6"
+ style="overflow:visible">
+ <path
+ inkscape:connector-curvature="0"
+ id="path8404-0-1"
+ d="M 0,0 5,-5 -12.5,0 5,5 0,0 z"
+ style="fill-rule:evenodd;stroke:#000000;stroke-width:1pt"
+ transform="matrix(0.8,0,0,0.8,10,0)" />
+ </marker>
+ <marker
+ inkscape:stockid="Arrow1Lend"
+ orient="auto"
+ refY="0"
+ refX="0"
+ id="Arrow1Lend-55"
+ style="overflow:visible">
+ <path
+ inkscape:connector-curvature="0"
+ id="path8407-4"
+ d="M 0,0 5,-5 -12.5,0 5,5 0,0 z"
+ style="fill-rule:evenodd;stroke:#000000;stroke-width:1pt"
+ transform="matrix(-0.8,0,0,-0.8,-10,0)" />
+ </marker>
+ </defs>
+ <sodipodi:namedview
+ id="base"
+ pagecolor="#ffffff"
+ bordercolor="#666666"
+ borderopacity="1.0"
+ inkscape:pageopacity="0.0"
+ inkscape:pageshadow="2"
+ inkscape:zoom="1"
+ inkscape:cx="233.5"
+ inkscape:cy="288"
+ inkscape:document-units="px"
+ inkscape:current-layer="layer1"
+ showgrid="false"
+ inkscape:window-width="1920"
+ inkscape:window-height="1017"
+ inkscape:window-x="-8"
+ inkscape:window-y="-8"
+ inkscape:window-maximized="1"
+ inkscape:snap-nodes="false"
+ inkscape:showpageshadow="2"
+ inkscape:pagecheckerboard="0"
+ inkscape:deskcolor="#d1d1d1">
+ <inkscape:grid
+ type="xygrid"
+ id="grid13454" />
+ </sodipodi:namedview>
+ <metadata
+ id="metadata13242">
+ <rdf:RDF>
+ <cc:Work
+ rdf:about="">
+ <dc:format>image/svg+xml</dc:format>
+ <dc:type
+ rdf:resource="http://purl.org/dc/dcmitype/StillImage" />
+ <dc:title />
+ </cc:Work>
+ </rdf:RDF>
+ </metadata>
+ <g
+ id="layer1"
+ inkscape:label="Layer 1"
+ inkscape:groupmode="layer">
+ <g
+ style="font-size:12px;fill:none;stroke-linecap:square;stroke-miterlimit:3;overflow:visible"
+ id="shape1-1-2-4"
+ v:mID="1"
+ v:groupContext="shape"
+ transform="matrix(2.1604167,0,0,1.5671361,88.874699,-812.39909)">
+ <title
+ id="title22-7-5">Square</title>
+ <desc
+ id="desc24-7-8">Atomic Queue #1</desc>
+ <v:userDefs>
+ <v:ud
+ v:nameU="visVersion"
+ v:val="VT0(15):26" />
+ </v:userDefs>
+ <v:textBlock
+ v:margins="rect(4,4,4,4)" />
+ <v:textRect
+ cx="30.75"
+ cy="581.25"
+ width="61.5"
+ height="61.5" />
+ <g
+ id="shadow1-2-9-5"
+ v:groupContext="shadow"
+ v:shadowOffsetX="0.345598"
+ v:shadowOffsetY="-1.97279"
+ v:shadowType="1"
+ transform="translate(0.345598,1.97279)"
+ class="st1"
+ style="visibility:visible">
+ <rect
+ x="0"
+ y="550.5"
+ width="61.5"
+ height="61.5"
+ class="st2"
+ id="rect27-8-7"
+ style="fill:#5b9bd5;fill-opacity:0.22000002;stroke:#5b9bd5;stroke-opacity:0.22000002;filter:url(#filter_2-3-91)" />
+ </g>
+ <g
+ id="g13515-33">
+ <g
+ id="g13534-8">
+ <rect
+ x="0"
+ y="550.5"
+ width="61.5"
+ height="61.5"
+ class="st3"
+ id="rect29-1-95"
+ style="fill:#5b9bd5;stroke:#c7c8c8;stroke-width:0.25" />
+ </g>
+ </g>
+ </g>
+ <path
+ style="fill:none;stroke:#000000;stroke-width:0.712265;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:4;stroke-dasharray:none;stroke-opacity:1;marker-start:url(#Arrow1Lstart-7);marker-end:none"
+ d="M 312.28671,240.74335 H 227.99897"
+ id="path17209"
+ inkscape:connector-type="orthogonal"
+ inkscape:connector-curvature="0" />
+ <path
+ style="fill:none;stroke:#000000;stroke-width:0.718986px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1;marker-mid:none;marker-end:url(#Arrow1Lend)"
+ d="m 221.6484,77.57125 h 94.28101"
+ id="path17209-8"
+ inkscape:connector-type="orthogonal"
+ inkscape:connector-curvature="0" />
+ <g
+ style="font-size:12px;fill:none;stroke-linecap:square;stroke-miterlimit:3;overflow:visible"
+ id="shape1-1-2"
+ v:mID="1"
+ v:groupContext="shape"
+ transform="matrix(2.1604167,0,0,1.5671361,314.24227,-811.89589)">
+ <title
+ id="title22-7">Square</title>
+ <desc
+ id="desc24-7">Atomic Queue #1</desc>
+ <v:userDefs>
+ <v:ud
+ v:nameU="visVersion"
+ v:val="VT0(15):26" />
+ </v:userDefs>
+ <v:textBlock
+ v:margins="rect(4,4,4,4)" />
+ <v:textRect
+ cx="30.75"
+ cy="581.25"
+ width="61.5"
+ height="61.5" />
+ <g
+ id="shadow1-2-9"
+ v:groupContext="shadow"
+ v:shadowOffsetX="0.345598"
+ v:shadowOffsetY="-1.97279"
+ v:shadowType="1"
+ transform="translate(0.345598,1.97279)"
+ class="st1"
+ style="visibility:visible">
+ <rect
+ x="0"
+ y="550.5"
+ width="61.5"
+ height="61.5"
+ class="st2"
+ id="rect27-8"
+ style="fill:#5b9bd5;fill-opacity:0.22000002;stroke:#5b9bd5;stroke-opacity:0.22000002;filter:url(#filter_2-3)" />
+ </g>
+ <g
+ id="g13515">
+ <g
+ id="g13534">
+ <rect
+ x="0"
+ y="550.5"
+ width="61.5"
+ height="61.5"
+ class="st3"
+ id="rect29-1"
+ style="fill:#5b9bd5;stroke:#c7c8c8;stroke-width:0.25" />
+ </g>
+ </g>
+ </g>
+ <path
+ style="fill:none;stroke:#000000;stroke-width:0.724714;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:4;stroke-dasharray:none;stroke-opacity:1;marker-start:url(#Arrow1Lstart);marker-end:none"
+ d="M 89.025329,74.39932 H 24.750043"
+ id="path17209-3"
+ inkscape:connector-type="orthogonal"
+ inkscape:connector-curvature="0" />
+ <path
+ transform="matrix(0.73232502,0,0,0.75477602,-4.325033,28.642983)"
+ sodipodi:type="arc"
+ style="fill:#539de6;fill-opacity:1;stroke:#0000ea;stroke-width:1;stroke-linecap:square;stroke-miterlimit:4;stroke-opacity:1;stroke-dasharray:none"
+ id="path29161-3"
+ sodipodi:cx="371"
+ sodipodi:cy="64.5"
+ sodipodi:rx="17"
+ sodipodi:ry="15.5"
+ d="m 388,64.5 a 17,15.5 0 1 1 -34,0 17,15.5 0 1 1 34,0 z" />
+ <path
+ transform="matrix(0.73232502,0,0,0.75477602,-1.93108,192.80833)"
+ sodipodi:type="arc"
+ style="fill:#539de6;fill-opacity:1;stroke:#0000ea;stroke-width:1;stroke-linecap:square;stroke-miterlimit:4;stroke-opacity:1;stroke-dasharray:none"
+ id="path29161-1"
+ sodipodi:cx="371"
+ sodipodi:cy="64.5"
+ sodipodi:rx="17"
+ sodipodi:ry="15.5"
+ d="m 388,64.5 a 17,15.5 0 1 1 -34,0 17,15.5 0 1 1 34,0 z" />
+ <path
+ style="fill:none;stroke:#000000;stroke-width:0.751412;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:4;stroke-dasharray:none;stroke-opacity:1;marker-start:url(#Arrow1Lstart-7);marker-end:none"
+ d="M 18.763392,120.7432 H 87.758545"
+ id="path17209-3-0"
+ inkscape:connector-type="orthogonal"
+ inkscape:connector-curvature="0" />
+ <path
+ sodipodi:type="arc"
+ style="fill:#539de6;fill-opacity:1;stroke:#0000ea;stroke-width:1;stroke-linecap:square;stroke-miterlimit:4;stroke-opacity:1;stroke-dasharray:none"
+ id="path29161"
+ sodipodi:cx="371"
+ sodipodi:cy="64.5"
+ sodipodi:rx="17"
+ sodipodi:ry="15.5"
+ d="m 388,64.5 a 17,15.5 0 1 1 -34,0 17,15.5 0 1 1 34,0 z"
+ transform="matrix(0.73232502,0,0,0.75477602,-218.16394,72.68276)" />
+ <path
+ sodipodi:type="arc"
+ style="fill:#539de6;fill-opacity:1;stroke:#0000ea;stroke-width:1;stroke-linecap:square;stroke-miterlimit:4;stroke-opacity:1;stroke-dasharray:none"
+ id="path29161-2"
+ sodipodi:cx="371"
+ sodipodi:cy="64.5"
+ sodipodi:rx="17"
+ sodipodi:ry="15.5"
+ d="m 388,64.5 a 17,15.5 0 1 1 -34,0 17,15.5 0 1 1 34,0 z"
+ transform="matrix(0.73232502,0,0,0.75477602,-217.40136,26.716271)" />
+ <g
+ id="g29167-4"
+ transform="matrix(0.73232502,0,0,0.75477602,-217.31662,28.007562)">
+ <text
+ id="text29163-9"
+ y="70"
+ x="365"
+ style="font-style:normal;font-weight:normal;line-height:0%;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none"
+ xml:space="preserve"><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="70"
+ x="365"
+ id="tspan29165-9"
+ sodipodi:role="line">1</tspan></text>
+ </g>
+ <g
+ id="g29167-9"
+ transform="matrix(0.73232502,0,0,0.75477602,-4.9726112,28.689051)">
+ <text
+ id="text29163-3"
+ y="70"
+ x="365"
+ style="font-style:normal;font-weight:normal;line-height:0%;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none"
+ xml:space="preserve"><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="70"
+ x="365"
+ id="tspan29165-3"
+ sodipodi:role="line">2</tspan></text>
+ </g>
+ <path
+ style="fill:none;stroke:#000000;stroke-width:0.678033px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1;marker-start:url(#Arrow1Lstart-7);marker-end:none"
+ d="M 181,214.66098 V 145.33902"
+ id="path17211-7-1-6"
+ inkscape:connector-type="orthogonal"
+ inkscape:connector-curvature="0" />
+ <g
+ id="g29167"
+ transform="matrix(0.73232502,0,0,0.75477602,-218.07919,73.10621)">
+ <text
+ id="text29163"
+ y="70"
+ x="365"
+ style="font-style:normal;font-weight:normal;line-height:0%;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none"
+ xml:space="preserve"><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="70"
+ x="365"
+ id="tspan29165"
+ sodipodi:role="line">8</tspan></text>
+ </g>
+ <path
+ style="fill:none;stroke:#000000;stroke-width:0.678033px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1;marker-start:url(#Arrow1Lstart-7);marker-end:none"
+ d="m 131,145.8531 v 69.32197"
+ id="path17211-7-1"
+ inkscape:connector-type="orthogonal"
+ inkscape:connector-curvature="0" />
+ <path
+ transform="matrix(0.73232502,0,0,0.75477602,-140.37076,129.97088)"
+ sodipodi:type="arc"
+ style="fill:#539de6;fill-opacity:1;stroke:#0000ea;stroke-width:1;stroke-linecap:square;stroke-miterlimit:4;stroke-opacity:1;stroke-dasharray:none"
+ id="path29161-8"
+ sodipodi:cx="371"
+ sodipodi:cy="64.5"
+ sodipodi:rx="17"
+ sodipodi:ry="15.5"
+ d="m 388,64.5 a 17,15.5 0 1 1 -34,0 17,15.5 0 1 1 34,0 z" />
+ <g
+ id="g29167-2"
+ transform="matrix(0.73232502,0,0,0.75477602,-140.28602,131.01695)">
+ <text
+ id="text29163-92"
+ y="70"
+ x="365"
+ style="font-style:normal;font-weight:normal;line-height:0%;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none"
+ xml:space="preserve"><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="70"
+ x="365"
+ id="tspan29165-8"
+ sodipodi:role="line">7</tspan></text>
+ </g>
+ <path
+ style="fill:none;stroke:#000000;stroke-width:0.718986px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1;marker-mid:none;marker-end:url(#Arrow1Lend)"
+ d="m 317.1405,116 h -94.281"
+ id="path17209-8-0"
+ inkscape:connector-type="orthogonal"
+ inkscape:connector-curvature="0" />
+ <path
+ transform="matrix(0.73232502,0,0,0.75477602,-3.4914,66.68745)"
+ sodipodi:type="arc"
+ style="fill:#539de6;fill-opacity:1;stroke:#0000ea;stroke-width:1;stroke-linecap:square;stroke-miterlimit:4;stroke-opacity:1;stroke-dasharray:none"
+ id="path29161-6"
+ sodipodi:cx="371"
+ sodipodi:cy="64.5"
+ sodipodi:rx="17"
+ sodipodi:ry="15.5"
+ d="m 388,64.5 a 17,15.5 0 1 1 -34,0 17,15.5 0 1 1 34,0 z" />
+ <g
+ id="g29167-46"
+ transform="matrix(0.73232502,0,0,0.75477602,-4.40666,67.48829)">
+ <text
+ id="text29163-1"
+ y="70"
+ x="365"
+ style="font-style:normal;font-weight:normal;line-height:0%;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none"
+ xml:space="preserve"><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="70"
+ x="365"
+ id="tspan29165-5"
+ sodipodi:role="line">3</tspan></text>
+ </g>
+ <path
+ transform="matrix(0.73232502,0,0,0.75477602,-90.692582,130.31695)"
+ sodipodi:type="arc"
+ style="fill:#539de6;fill-opacity:1;stroke:#0000ea;stroke-width:1;stroke-linecap:square;stroke-miterlimit:4;stroke-opacity:1;stroke-dasharray:none"
+ id="path29161-8-6"
+ sodipodi:cx="371"
+ sodipodi:cy="64.5"
+ sodipodi:rx="17"
+ sodipodi:ry="15.5"
+ d="m 388,64.5 a 17,15.5 0 1 1 -34,0 17,15.5 0 1 1 34,0 z" />
+ <g
+ id="g29167-6"
+ transform="matrix(0.73232502,0,0,0.75477602,-90.84634,131.60918)">
+ <text
+ id="text29163-17"
+ y="70"
+ x="365"
+ style="font-style:normal;font-weight:normal;line-height:0%;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none"
+ xml:space="preserve"><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="70"
+ x="365"
+ id="tspan29165-2"
+ sodipodi:role="line">4</tspan></text>
+ </g>
+ <g
+ id="g29167-2-0"
+ transform="matrix(0.73232502,0,0,0.75477602,-2.424397,194.0216)">
+ <text
+ id="text29163-92-6"
+ y="70"
+ x="365"
+ style="font-style:normal;font-weight:normal;line-height:0%;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none"
+ xml:space="preserve"><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="70"
+ x="365"
+ id="tspan29165-8-2"
+ sodipodi:role="line">5</tspan></text>
+ </g>
+ <g
+ style="font-size:12px;fill:none;stroke-linecap:square;stroke-miterlimit:3;overflow:visible"
+ id="shape1-1-2-8"
+ v:mID="1"
+ v:groupContext="shape"
+ transform="matrix(2.1604167,0,0,1.5671361,93.82055,-648.98949)">
+ <title
+ id="title22-7-97">Square</title>
+ <desc
+ id="desc24-7-3">Atomic Queue #1</desc>
+ <v:userDefs>
+ <v:ud
+ v:nameU="visVersion"
+ v:val="VT0(15):26" />
+ </v:userDefs>
+ <v:textBlock
+ v:margins="rect(4,4,4,4)" />
+ <v:textRect
+ cx="30.75"
+ cy="581.25"
+ width="61.5"
+ height="61.5" />
+ <g
+ id="shadow1-2-9-6"
+ v:groupContext="shadow"
+ v:shadowOffsetX="0.345598"
+ v:shadowOffsetY="-1.97279"
+ v:shadowType="1"
+ transform="translate(0.345598,1.97279)"
+ class="st1"
+ style="visibility:visible">
+ <rect
+ x="0"
+ y="550.5"
+ width="61.5"
+ height="61.5"
+ class="st2"
+ id="rect27-8-12"
+ style="fill:#5b9bd5;fill-opacity:0.22000002;stroke:#5b9bd5;stroke-opacity:0.22000002;filter:url(#filter_2-3-92)" />
+ </g>
+ <g
+ id="g13515-9">
+ <g
+ id="g13534-3">
+ <rect
+ x="0"
+ y="550.5"
+ width="61.5"
+ height="61.5"
+ class="st3"
+ id="rect29-1-1"
+ style="fill:#5b9bd5;stroke:#c7c8c8;stroke-width:0.25" />
+ </g>
+ </g>
+ </g>
+ <g
+ style="font-size:12px;fill:none;stroke-linecap:square;stroke-miterlimit:3;overflow:visible"
+ id="shape1-1-2-84"
+ v:mID="1"
+ v:groupContext="shape"
+ transform="matrix(2.1604167,0,0,1.5671361,314.82055,-648.98949)">
+ <title
+ id="title22-7-50">Square</title>
+ <desc
+ id="desc24-7-36">Atomic Queue #1</desc>
+ <v:userDefs>
+ <v:ud
+ v:nameU="visVersion"
+ v:val="VT0(15):26" />
+ </v:userDefs>
+ <v:textBlock
+ v:margins="rect(4,4,4,4)" />
+ <v:textRect
+ cx="30.75"
+ cy="581.25"
+ width="61.5"
+ height="61.5" />
+ <g
+ id="shadow1-2-9-1"
+ v:groupContext="shadow"
+ v:shadowOffsetX="0.345598"
+ v:shadowOffsetY="-1.97279"
+ v:shadowType="1"
+ transform="translate(0.345598,1.97279)"
+ class="st1"
+ style="visibility:visible">
+ <rect
+ x="0"
+ y="550.5"
+ width="61.5"
+ height="61.5"
+ class="st2"
+ id="rect27-8-0"
+ style="fill:#5b9bd5;fill-opacity:0.22000002;stroke:#5b9bd5;stroke-opacity:0.22000002;filter:url(#filter_2-3-94)" />
+ </g>
+ <g
+ id="g13515-6">
+ <g
+ id="g13534-32">
+ <rect
+ x="0"
+ y="550.5"
+ width="61.5"
+ height="61.5"
+ class="st3"
+ id="rect29-1-0"
+ style="fill:#5b9bd5;stroke:#c7c8c8;stroke-width:0.25" />
+ </g>
+ </g>
+ </g>
+ <path
+ style="fill:none;stroke:#000000;stroke-width:0.712265;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:4;stroke-dasharray:none;stroke-opacity:1;marker-start:none;marker-end:url(#Arrow1Lend)"
+ d="M 313.14387,285 H 228.85613"
+ id="path17209-7"
+ inkscape:connector-type="orthogonal"
+ inkscape:connector-curvature="0" />
+ <path
+ transform="matrix(0.73232502,0,0,0.75477602,-2.692582,236.31695)"
+ sodipodi:type="arc"
+ style="fill:#539de6;fill-opacity:1;stroke:#0000ea;stroke-width:1;stroke-linecap:square;stroke-miterlimit:4;stroke-opacity:1;stroke-dasharray:none"
+ id="path29161-1-6"
+ sodipodi:cx="371"
+ sodipodi:cy="64.5"
+ sodipodi:rx="17"
+ sodipodi:ry="15.5"
+ d="m 388,64.5 a 17,15.5 0 1 1 -34,0 17,15.5 0 1 1 34,0 z" />
+ <g
+ id="g29167-2-0-5"
+ transform="matrix(0.73232502,0,0,0.75477602,-2.424397,237.0216)">
+ <text
+ id="text29163-92-6-6"
+ y="70"
+ x="365"
+ style="font-style:normal;font-weight:normal;line-height:0%;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none"
+ xml:space="preserve"><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="70"
+ x="365"
+ id="tspan29165-8-2-9"
+ sodipodi:role="line">6</tspan></text>
+ </g>
+ <g
+ id="g29167-4-3"
+ transform="matrix(0.73232502,0,0,0.75477602,-154.60784,51.117791)">
+ <text
+ id="text29163-9-6"
+ y="70"
+ x="365"
+ style="font-style:normal;font-weight:normal;line-height:0%;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none"
+ xml:space="preserve"><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:24.2109px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="70"
+ x="365"
+ id="tspan29165-9-7"
+ sodipodi:role="line">Eventdev</tspan></text>
+ </g>
+ <g
+ id="g29167-4-3-5"
+ transform="matrix(0.73232502,0,0,0.75477602,-144.65044,201.97821)">
+ <text
+ id="text29163-9-6-3"
+ y="70"
+ x="412.93716"
+ style="font-style:normal;font-weight:normal;line-height:0%;font-family:sans-serif;text-align:center;letter-spacing:0px;word-spacing:0px;text-anchor:middle;fill:#000000;fill-opacity:1;stroke:none"
+ xml:space="preserve"><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:24.2109px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:center;writing-mode:lr-tb;text-anchor:middle"
+ y="70"
+ x="412.93716"
+ id="tspan29165-9-7-5"
+ sodipodi:role="line">DMA</tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:24.2109px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:center;writing-mode:lr-tb;text-anchor:middle"
+ y="100.26363"
+ x="412.93716"
+ sodipodi:role="line"
+ id="tspan3201">Adapter</tspan></text>
+ </g>
+ <g
+ id="g29167-4-3-5-6"
+ transform="matrix(0.73232502,0,0,0.75477602,79.53518,46.62529)">
+ <text
+ id="text29163-9-6-3-2"
+ y="48.801659"
+ x="412.93716"
+ style="font-style:normal;font-weight:normal;line-height:0%;font-family:sans-serif;text-align:center;letter-spacing:0px;word-spacing:0px;text-anchor:middle;fill:#000000;fill-opacity:1;stroke:none"
+ xml:space="preserve"><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:24.2109px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:center;writing-mode:lr-tb;text-anchor:middle"
+ y="48.801659"
+ x="412.93716"
+ sodipodi:role="line"
+ id="tspan3155">Application</tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:24.2109px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:center;writing-mode:lr-tb;text-anchor:middle"
+ y="98.801659"
+ x="412.93716"
+ sodipodi:role="line"
+ id="tspan3201-1">in ordered</tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:24.2109px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:center;writing-mode:lr-tb;text-anchor:middle"
+ y="148.80167"
+ x="412.93716"
+ sodipodi:role="line"
+ id="tspan3161">stage</tspan></text>
+ </g>
+ <g
+ id="g29167-4-3-5-2"
+ transform="matrix(0.73232502,0,0,0.75477602,77.535182,213.62529)">
+ <text
+ id="text29163-9-6-3-7"
+ y="70"
+ x="412.93716"
+ style="font-style:normal;font-weight:normal;line-height:0%;font-family:sans-serif;text-align:center;letter-spacing:0px;word-spacing:0px;text-anchor:middle;fill:#000000;fill-opacity:1;stroke:none"
+ xml:space="preserve"><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:24.2109px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:center;writing-mode:lr-tb;text-anchor:middle"
+ y="70"
+ x="412.93716"
+ sodipodi:role="line"
+ id="tspan3201-9">DMA Device</tspan></text>
+ </g>
+ <g
+ id="g29167-4-3-5-3"
+ transform="matrix(0.73232502,0,0,0.75477602,188.53518,-3.37471)">
+ <text
+ id="text29163-9-6-3-6"
+ y="70"
+ x="375.65271"
+ style="font-style:normal;font-weight:normal;line-height:0%;font-family:sans-serif;text-align:center;letter-spacing:0px;word-spacing:0px;text-anchor:middle;fill:#000000;fill-opacity:1;stroke:none"
+ xml:space="preserve"><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18.8307px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="70"
+ x="375.65271"
+ sodipodi:role="line"
+ id="tspan3201-6">1. Events from the previous stage.</tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18.8307px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="93.538376"
+ x="375.65271"
+ sodipodi:role="line"
+ id="tspan3260"> </tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18.8307px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="117.07675"
+ x="375.65271"
+ sodipodi:role="line"
+ id="tspan3262">2. Application in ordered stage</tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18.8307px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="140.61513"
+ x="375.65271"
+ sodipodi:role="line"
+ id="tspan3288"> dequeues events from eventdev.</tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18.8307px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="164.1535"
+ x="375.65271"
+ sodipodi:role="line"
+ id="tspan3264"> </tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18.8307px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="187.69188"
+ x="375.65271"
+ sodipodi:role="line"
+ id="tspan3266">3. Application enqueues DMA</tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18.8307px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="211.23026"
+ x="375.65271"
+ sodipodi:role="line"
+ id="tspan3290"> operations as events to eventdev.</tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18.8307px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="234.76863"
+ x="375.65271"
+ sodipodi:role="line"
+ id="tspan3268"> </tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18.8307px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="258.30701"
+ x="375.65271"
+ sodipodi:role="line"
+ id="tspan3270">4. DMA adapter dequeues event</tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18.8307px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="281.84537"
+ x="375.65271"
+ sodipodi:role="line"
+ id="tspan3292"> from eventdev.</tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18.8307px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="305.38376"
+ x="375.65271"
+ sodipodi:role="line"
+ id="tspan3272"> </tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18.8307px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="328.92212"
+ x="375.65271"
+ sodipodi:role="line"
+ id="tspan3274">5. DMA adapter submits DMA</tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18.8307px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="352.46051"
+ x="375.65271"
+ sodipodi:role="line"
+ id="tspan3294"> operations to DMA Device (Atomic</tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18.8307px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="375.99887"
+ x="375.65271"
+ sodipodi:role="line"
+ id="tspan3296"> stage)</tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18.8307px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="399.53726"
+ x="375.65271"
+ sodipodi:role="line"
+ id="tspan3276"> </tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18.8307px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="423.07562"
+ x="375.65271"
+ sodipodi:role="line"
+ id="tspan3278">6. DMA adapter dequeues DMA</tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18.8307px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="446.61401"
+ x="375.65271"
+ sodipodi:role="line"
+ id="tspan3298"> completions from DMA Device</tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18.8307px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="470.15237"
+ x="375.65271"
+ sodipodi:role="line"
+ id="tspan3280"> </tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18.8307px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="493.69073"
+ x="375.65271"
+ sodipodi:role="line"
+ id="tspan3282">7. DMA adapter enqueues events</tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18.8307px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="517.22913"
+ x="375.65271"
+ sodipodi:role="line"
+ id="tspan3300"> to the eventdev</tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18.8307px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="540.76752"
+ x="375.65271"
+ sodipodi:role="line"
+ id="tspan3284"> </tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18.8307px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="564.30585"
+ x="375.65271"
+ sodipodi:role="line"
+ id="tspan3286">8. Events to the next stage</tspan></text>
+ </g>
+ </g>
+</svg>
diff --git a/doc/guides/prog_guide/img/event_dma_adapter_op_new.svg b/doc/guides/prog_guide/img/event_dma_adapter_op_new.svg
new file mode 100644
index 0000000000..e9e8bb2b98
--- /dev/null
+++ b/doc/guides/prog_guide/img/event_dma_adapter_op_new.svg
@@ -0,0 +1,1079 @@
+<?xml version="1.0" encoding="UTF-8" standalone="no"?>
+<!-- SPDX-License-Identifier: BSD-3-Clause -->
+<!-- Copyright (c) 2023 Marvell. -->
+<!-- Created with Inkscape (http://www.inkscape.org/) -->
+
+<svg
+ width="720px"
+ height="486px"
+ id="svg13237"
+ version="1.1"
+ inkscape:version="1.2.1 (9c6d41e410, 2022-07-14)"
+ sodipodi:docname="event_dma_adapter_op_new.svg"
+ xmlns:inkscape="http://www.inkscape.org/namespaces/inkscape"
+ xmlns:sodipodi="http://sodipodi.sourceforge.net/DTD/sodipodi-0.dtd"
+ xmlns="http://www.w3.org/2000/svg"
+ xmlns:svg="http://www.w3.org/2000/svg"
+ xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#"
+ xmlns:cc="http://creativecommons.org/ns#"
+ xmlns:dc="http://purl.org/dc/elements/1.1/"
+ xmlns:v="http://schemas.microsoft.com/visio/2003/SVGExtensions/">
+ <defs
+ id="defs13239">
+ <marker
+ inkscape:stockid="Arrow1Sstart"
+ orient="auto"
+ refY="0.0"
+ refX="0.0"
+ id="Arrow1Sstart"
+ style="overflow:visible">
+ <path
+ id="path8416"
+ d="M 0.0,0.0 L 5.0,-5.0 L -12.5,0.0 L 5.0,5.0 L 0.0,0.0 z "
+ style="fill-rule:evenodd;stroke:#000000;stroke-width:1.0pt"
+ transform="scale(0.2) translate(6,0)" />
+ </marker>
+ <marker
+ inkscape:stockid="Arrow1Send"
+ orient="auto"
+ refY="0.0"
+ refX="0.0"
+ id="Arrow1Send"
+ style="overflow:visible;">
+ <path
+ id="path8419"
+ d="M 0.0,0.0 L 5.0,-5.0 L -12.5,0.0 L 5.0,5.0 L 0.0,0.0 z "
+ style="fill-rule:evenodd;stroke:#000000;stroke-width:1.0pt;"
+ transform="scale(0.2) rotate(180) translate(6,0)" />
+ </marker>
+ <marker
+ inkscape:stockid="DiamondL"
+ orient="auto"
+ refY="0.0"
+ refX="0.0"
+ id="DiamondL"
+ style="overflow:visible">
+ <path
+ id="path8483"
+ d="M 0,-7.0710768 L -7.0710894,0 L 0,7.0710589 L 7.0710462,0 L 0,-7.0710768 z "
+ style="fill-rule:evenodd;stroke:#000000;stroke-width:1.0pt"
+ transform="scale(0.8)" />
+ </marker>
+ <marker
+ inkscape:stockid="DotL"
+ orient="auto"
+ refY="0.0"
+ refX="0.0"
+ id="DotL"
+ style="overflow:visible">
+ <path
+ id="path8465"
+ d="M -2.5,-1.0 C -2.5,1.7600000 -4.7400000,4.0 -7.5,4.0 C -10.260000,4.0 -12.5,1.7600000 -12.5,-1.0 C -12.5,-3.7600000 -10.260000,-6.0 -7.5,-6.0 C -4.7400000,-6.0 -2.5,-3.7600000 -2.5,-1.0 z "
+ style="fill-rule:evenodd;stroke:#000000;stroke-width:1.0pt"
+ transform="scale(0.8) translate(7.4, 1)" />
+ </marker>
+ <marker
+ inkscape:stockid="SquareL"
+ orient="auto"
+ refY="0.0"
+ refX="0.0"
+ id="SquareL"
+ style="overflow:visible">
+ <path
+ id="path8474"
+ d="M -5.0,-5.0 L -5.0,5.0 L 5.0,5.0 L 5.0,-5.0 L -5.0,-5.0 z "
+ style="fill-rule:evenodd;stroke:#000000;stroke-width:1.0pt"
+ transform="scale(0.8)" />
+ </marker>
+ <marker
+ inkscape:stockid="TriangleOutL"
+ orient="auto"
+ refY="0.0"
+ refX="0.0"
+ id="TriangleOutL"
+ style="overflow:visible">
+ <path
+ id="path8546"
+ d="M 5.77,0.0 L -2.88,5.0 L -2.88,-5.0 L 5.77,0.0 z "
+ style="fill-rule:evenodd;stroke:#000000;stroke-width:1.0pt"
+ transform="scale(0.8)" />
+ </marker>
+ <marker
+ inkscape:stockid="Arrow1Lstart"
+ orient="auto"
+ refY="0.0"
+ refX="0.0"
+ id="Arrow1Lstart"
+ style="overflow:visible">
+ <path
+ id="path8404"
+ d="M 0.0,0.0 L 5.0,-5.0 L -12.5,0.0 L 5.0,5.0 L 0.0,0.0 z "
+ style="fill-rule:evenodd;stroke:#000000;stroke-width:1.0pt"
+ transform="scale(0.8) translate(12.5,0)" />
+ </marker>
+ <marker
+ inkscape:stockid="Arrow1Mend"
+ orient="auto"
+ refY="0.0"
+ refX="0.0"
+ id="Arrow1Mend"
+ style="overflow:visible;">
+ <path
+ id="path8413"
+ d="M 0.0,0.0 L 5.0,-5.0 L -12.5,0.0 L 5.0,5.0 L 0.0,0.0 z "
+ style="fill-rule:evenodd;stroke:#000000;stroke-width:1.0pt;"
+ transform="scale(0.4) rotate(180) translate(10,0)" />
+ </marker>
+ <marker
+ inkscape:stockid="Arrow2Lend"
+ orient="auto"
+ refY="0.0"
+ refX="0.0"
+ id="Arrow2Lend"
+ style="overflow:visible;">
+ <path
+ id="path8425"
+ style="fill-rule:evenodd;stroke-width:0.62500000;stroke-linejoin:round;"
+ d="M 8.7185878,4.0337352 L -2.2072895,0.016013256 L 8.7185884,-4.0017078 C 6.9730900,-1.6296469 6.9831476,1.6157441 8.7185878,4.0337352 z "
+ transform="scale(1.1) rotate(180) translate(1,0)" />
+ </marker>
+ <marker
+ inkscape:stockid="Arrow1Lend"
+ orient="auto"
+ refY="0.0"
+ refX="0.0"
+ id="Arrow1Lend"
+ style="overflow:visible;">
+ <path
+ id="path8407"
+ d="M 0.0,0.0 L 5.0,-5.0 L -12.5,0.0 L 5.0,5.0 L 0.0,0.0 z "
+ style="fill-rule:evenodd;stroke:#000000;stroke-width:1.0pt;"
+ transform="scale(0.8) rotate(180) translate(12.5,0)" />
+ </marker>
+ <filter
+ id="filter_2"
+ color-interpolation-filters="sRGB">
+ <feGaussianBlur
+ stdDeviation="2"
+ id="feGaussianBlur15" />
+ </filter>
+ <filter
+ id="filter_2-3"
+ color-interpolation-filters="sRGB"
+ x="-0.086178862"
+ y="-0.086178862"
+ width="1.1723577"
+ height="1.1723577">
+ <feGaussianBlur
+ stdDeviation="2"
+ id="feGaussianBlur15-1" />
+ </filter>
+ <filter
+ id="filter_2-0"
+ color-interpolation-filters="sRGB">
+ <feGaussianBlur
+ stdDeviation="2"
+ id="feGaussianBlur15-7" />
+ </filter>
+ <filter
+ id="filter_2-0-8"
+ color-interpolation-filters="sRGB">
+ <feGaussianBlur
+ stdDeviation="2"
+ id="feGaussianBlur15-7-7" />
+ </filter>
+ <filter
+ id="filter_2-3-9"
+ color-interpolation-filters="sRGB"
+ x="-0.086178862"
+ y="-0.086178862"
+ width="1.1723577"
+ height="1.1723577">
+ <feGaussianBlur
+ stdDeviation="2"
+ id="feGaussianBlur15-1-6" />
+ </filter>
+ <filter
+ id="filter_2-3-6"
+ color-interpolation-filters="sRGB"
+ x="-0.086178862"
+ y="-0.086178862"
+ width="1.1723577"
+ height="1.1723577">
+ <feGaussianBlur
+ stdDeviation="2"
+ id="feGaussianBlur15-1-63" />
+ </filter>
+ <filter
+ id="filter_2-3-91"
+ color-interpolation-filters="sRGB"
+ x="-0.086178862"
+ y="-0.086178862"
+ width="1.1723577"
+ height="1.1723577">
+ <feGaussianBlur
+ stdDeviation="2"
+ id="feGaussianBlur15-1-3" />
+ </filter>
+ <marker
+ inkscape:stockid="Arrow1Lend"
+ orient="auto"
+ refY="0"
+ refX="0"
+ id="Arrow1Lend-5"
+ style="overflow:visible">
+ <path
+ inkscape:connector-curvature="0"
+ id="path8407-3"
+ d="M 0,0 5,-5 -12.5,0 5,5 0,0 z"
+ style="fill-rule:evenodd;stroke:#000000;stroke-width:1pt"
+ transform="matrix(-0.8,0,0,-0.8,-10,0)" />
+ </marker>
+ <marker
+ inkscape:stockid="Arrow1Lend"
+ orient="auto"
+ refY="0"
+ refX="0"
+ id="Arrow1Lend-6"
+ style="overflow:visible">
+ <path
+ inkscape:connector-curvature="0"
+ id="path8407-0"
+ d="M 0,0 5,-5 -12.5,0 5,5 0,0 z"
+ style="fill-rule:evenodd;stroke:#000000;stroke-width:1pt"
+ transform="matrix(-0.8,0,0,-0.8,-10,0)" />
+ </marker>
+ <marker
+ inkscape:stockid="Arrow1Lstart"
+ orient="auto"
+ refY="0"
+ refX="0"
+ id="Arrow1Lstart-7"
+ style="overflow:visible">
+ <path
+ inkscape:connector-curvature="0"
+ id="path8404-0"
+ d="M 0,0 5,-5 -12.5,0 5,5 0,0 z"
+ style="fill-rule:evenodd;stroke:#000000;stroke-width:1pt"
+ transform="matrix(0.8,0,0,0.8,10,0)" />
+ </marker>
+ <marker
+ inkscape:stockid="Arrow1Lend"
+ orient="auto"
+ refY="0"
+ refX="0"
+ id="Arrow1Lend-51"
+ style="overflow:visible">
+ <path
+ inkscape:connector-curvature="0"
+ id="path8407-1"
+ d="M 0,0 5,-5 -12.5,0 5,5 0,0 z"
+ style="fill-rule:evenodd;stroke:#000000;stroke-width:1pt"
+ transform="matrix(-0.8,0,0,-0.8,-10,0)" />
+ </marker>
+ <marker
+ inkscape:stockid="Arrow1Lend"
+ orient="auto"
+ refY="0"
+ refX="0"
+ id="Arrow1Lend-3"
+ style="overflow:visible">
+ <path
+ inkscape:connector-curvature="0"
+ id="path8407-6"
+ d="M 0,0 5,-5 -12.5,0 5,5 0,0 z"
+ style="fill-rule:evenodd;stroke:#000000;stroke-width:1pt"
+ transform="matrix(-0.8,0,0,-0.8,-10,0)" />
+ </marker>
+ <marker
+ inkscape:stockid="Arrow1Lend"
+ orient="auto"
+ refY="0"
+ refX="0"
+ id="Arrow1Lend-62"
+ style="overflow:visible">
+ <path
+ inkscape:connector-curvature="0"
+ id="path8407-9"
+ d="M 0,0 5,-5 -12.5,0 5,5 0,0 z"
+ style="fill-rule:evenodd;stroke:#000000;stroke-width:1pt"
+ transform="matrix(-0.8,0,0,-0.8,-10,0)" />
+ </marker>
+ <marker
+ inkscape:stockid="Arrow1Lend"
+ orient="auto"
+ refY="0"
+ refX="0"
+ id="Arrow1Lend-2"
+ style="overflow:visible">
+ <path
+ inkscape:connector-curvature="0"
+ id="path8407-7"
+ d="M 0,0 5,-5 -12.5,0 5,5 0,0 z"
+ style="fill-rule:evenodd;stroke:#000000;stroke-width:1pt"
+ transform="matrix(-0.8,0,0,-0.8,-10,0)" />
+ </marker>
+ <filter
+ id="filter_2-3-91-3"
+ color-interpolation-filters="sRGB"
+ x="-0.086178862"
+ y="-0.086178862"
+ width="1.1723577"
+ height="1.1723577">
+ <feGaussianBlur
+ stdDeviation="2"
+ id="feGaussianBlur15-1-3-6" />
+ </filter>
+ </defs>
+ <sodipodi:namedview
+ id="base"
+ pagecolor="#ffffff"
+ bordercolor="#666666"
+ borderopacity="1.0"
+ inkscape:pageopacity="0.0"
+ inkscape:pageshadow="2"
+ inkscape:zoom="1"
+ inkscape:cx="385.5"
+ inkscape:cy="234"
+ inkscape:document-units="px"
+ inkscape:current-layer="layer1"
+ showgrid="false"
+ inkscape:window-width="1920"
+ inkscape:window-height="1017"
+ inkscape:window-x="-8"
+ inkscape:window-y="-8"
+ inkscape:window-maximized="1"
+ inkscape:snap-nodes="false"
+ inkscape:showpageshadow="2"
+ inkscape:pagecheckerboard="0"
+ inkscape:deskcolor="#d1d1d1">
+ <inkscape:grid
+ type="xygrid"
+ id="grid13454" />
+ </sodipodi:namedview>
+ <metadata
+ id="metadata13242">
+ <rdf:RDF>
+ <cc:Work
+ rdf:about="">
+ <dc:format>image/svg+xml</dc:format>
+ <dc:type
+ rdf:resource="http://purl.org/dc/dcmitype/StillImage" />
+ <dc:title />
+ </cc:Work>
+ </rdf:RDF>
+ </metadata>
+ <g
+ id="layer1"
+ inkscape:label="Layer 1"
+ inkscape:groupmode="layer">
+ <g
+ style="font-size:12px;fill:none;stroke-linecap:square;stroke-miterlimit:3;overflow:visible"
+ id="shape1-1-2-0"
+ v:mID="1"
+ v:groupContext="shape"
+ transform="matrix(2.1604167,0,0,1.5671361,323.2187,-540.25927)">
+ <title
+ id="title22-7-6">Square</title>
+ <desc
+ id="desc24-7-4">Atomic Queue #1</desc>
+ <v:userDefs>
+ <v:ud
+ v:nameU="visVersion"
+ v:val="VT0(15):26" />
+ </v:userDefs>
+ <v:textBlock
+ v:margins="rect(4,4,4,4)" />
+ <v:textRect
+ cx="30.75"
+ cy="581.25"
+ width="61.5"
+ height="61.5" />
+ <g
+ id="shadow1-2-9-0"
+ v:groupContext="shadow"
+ v:shadowOffsetX="0.345598"
+ v:shadowOffsetY="-1.97279"
+ v:shadowType="1"
+ transform="translate(0.345598,1.97279)"
+ class="st1"
+ style="visibility:visible">
+ <rect
+ x="0"
+ y="550.5"
+ width="61.5"
+ height="61.5"
+ class="st2"
+ id="rect27-8-9"
+ style="fill:#5b9bd5;fill-opacity:0.22000002;stroke:#5b9bd5;stroke-opacity:0.22000002;filter:url(#filter_2-3-9)" />
+ </g>
+ <g
+ id="g13515-4">
+ <g
+ id="g13534-5">
+ <rect
+ x="0"
+ y="550.5"
+ width="61.5"
+ height="61.5"
+ class="st3"
+ id="rect29-1-4"
+ style="fill:#5b9bd5;stroke:#c7c8c8;stroke-width:0.25" />
+ </g>
+ </g>
+ </g>
+ <g
+ style="font-size:12px;fill:none;stroke-linecap:square;stroke-miterlimit:3;overflow:visible"
+ id="shape1-1-2-4"
+ v:mID="1"
+ v:groupContext="shape"
+ transform="matrix(2.1604167,0,0,1.165886,88.874699,-447.8809)">
+ <title
+ id="title22-7-5">Square</title>
+ <desc
+ id="desc24-7-8">Atomic Queue #1</desc>
+ <v:userDefs>
+ <v:ud
+ v:nameU="visVersion"
+ v:val="VT0(15):26" />
+ </v:userDefs>
+ <v:textBlock
+ v:margins="rect(4,4,4,4)" />
+ <v:textRect
+ cx="30.75"
+ cy="581.25"
+ width="61.5"
+ height="61.5" />
+ <g
+ id="shadow1-2-9-5"
+ v:groupContext="shadow"
+ v:shadowOffsetX="0.345598"
+ v:shadowOffsetY="-1.97279"
+ v:shadowType="1"
+ transform="translate(0.345598,1.97279)"
+ class="st1"
+ style="visibility:visible">
+ <rect
+ x="0"
+ y="550.5"
+ width="61.5"
+ height="61.5"
+ class="st2"
+ id="rect27-8-7"
+ style="fill:#5b9bd5;fill-opacity:0.22000002;stroke:#5b9bd5;stroke-opacity:0.22000002;filter:url(#filter_2-3-91)" />
+ </g>
+ <g
+ id="g13515-33">
+ <g
+ id="g13534-8">
+ <rect
+ x="0"
+ y="550.5"
+ width="61.5"
+ height="61.5"
+ class="st3"
+ id="rect29-1-95"
+ style="fill:#5b9bd5;stroke:#c7c8c8;stroke-width:0.25" />
+ </g>
+ </g>
+ </g>
+ <g
+ style="font-size:12px;fill:none;stroke-linecap:square;stroke-miterlimit:3;overflow:visible"
+ id="shape1-1-2-9"
+ v:mID="1"
+ v:groupContext="shape"
+ transform="matrix(2.1604167,0,0,1.5671361,88.874699,-538.24651)">
+ <title
+ id="title22-7-9">Square</title>
+ <desc
+ id="desc24-7-5">Atomic Queue #1</desc>
+ <v:userDefs>
+ <v:ud
+ v:nameU="visVersion"
+ v:val="VT0(15):26" />
+ </v:userDefs>
+ <v:textBlock
+ v:margins="rect(4,4,4,4)" />
+ <v:textRect
+ cx="30.75"
+ cy="581.25"
+ width="61.5"
+ height="61.5" />
+ <g
+ id="shadow1-2-9-2"
+ v:groupContext="shadow"
+ v:shadowOffsetX="0.345598"
+ v:shadowOffsetY="-1.97279"
+ v:shadowType="1"
+ transform="translate(0.345598,1.97279)"
+ class="st1"
+ style="visibility:visible">
+ <rect
+ x="0"
+ y="550.5"
+ width="61.5"
+ height="61.5"
+ class="st2"
+ id="rect27-8-1"
+ style="fill:#5b9bd5;fill-opacity:0.22000002;stroke:#5b9bd5;stroke-opacity:0.22000002;filter:url(#filter_2-3-6)" />
+ </g>
+ <g
+ id="g13515-3">
+ <g
+ id="g13534-0">
+ <rect
+ x="0"
+ y="550.5"
+ width="61.5"
+ height="61.5"
+ class="st3"
+ id="rect29-1-9"
+ style="fill:#5b9bd5;stroke:#c7c8c8;stroke-width:0.25" />
+ </g>
+ </g>
+ </g>
+ <path
+ style="fill:none;stroke:#000000;stroke-width:0.743466px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1;marker-mid:none;marker-end:url(#Arrow1Lend)"
+ d="M 220.66064,98.57125 H 321.88592"
+ id="path17209-8"
+ inkscape:connector-type="orthogonal"
+ inkscape:connector-curvature="0" />
+ <g
+ style="font-size:12px;fill:none;stroke-linecap:square;stroke-miterlimit:3;overflow:visible"
+ id="shape1-1-2"
+ v:mID="1"
+ v:groupContext="shape"
+ transform="matrix(2.1604167,0,0,1.5671361,322.24227,-811.89589)">
+ <title
+ id="title22-7">Square</title>
+ <desc
+ id="desc24-7">Atomic Queue #1</desc>
+ <v:userDefs>
+ <v:ud
+ v:nameU="visVersion"
+ v:val="VT0(15):26" />
+ </v:userDefs>
+ <v:textBlock
+ v:margins="rect(4,4,4,4)" />
+ <v:textRect
+ cx="30.75"
+ cy="581.25"
+ width="61.5"
+ height="61.5" />
+ <g
+ id="shadow1-2-9"
+ v:groupContext="shadow"
+ v:shadowOffsetX="0.345598"
+ v:shadowOffsetY="-1.97279"
+ v:shadowType="1"
+ transform="translate(0.345598,1.97279)"
+ class="st1"
+ style="visibility:visible">
+ <rect
+ x="0"
+ y="550.5"
+ width="61.5"
+ height="61.5"
+ class="st2"
+ id="rect27-8"
+ style="fill:#5b9bd5;fill-opacity:0.22000002;stroke:#5b9bd5;stroke-opacity:0.22000002;filter:url(#filter_2-3)" />
+ </g>
+ <g
+ id="g13515">
+ <g
+ id="g13534">
+ <rect
+ x="0"
+ y="550.5"
+ width="61.5"
+ height="61.5"
+ class="st3"
+ id="rect29-1"
+ style="fill:#5b9bd5;stroke:#c7c8c8;stroke-width:0.25" />
+ </g>
+ </g>
+ </g>
+ <g
+ id="g13518"
+ transform="matrix(0.73232502,0,0,0.75477602,25.29268,348.89752)">
+ <g
+ id="g13526">
+ <flowRoot
+ xml:space="preserve"
+ id="flowRoot13464-9"
+ style="font-style:normal;font-weight:normal;line-height:0.01%;font-family:sans-serif;text-align:center;letter-spacing:0px;word-spacing:0px;text-anchor:middle;fill:#000000;fill-opacity:1;stroke:none"
+ transform="translate(-12.00521,-129.65179)"><flowRegion
+ id="flowRegion13466-1"
+ style="font-family:sans-serif"><rect
+ id="rect13468-2"
+ width="195.99997"
+ height="112.00001"
+ x="273.33334"
+ y="175.33333"
+ style="text-align:center;text-anchor:middle" /></flowRegion><flowPara
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:20px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:center;writing-mode:lr-tb;text-anchor:middle"
+ id="flowPara13511"> </flowPara></flowRoot>
+ </g>
+ </g>
+ <path
+ style="fill:none;stroke:#000000;stroke-width:0.751455;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:4;stroke-dasharray:none;stroke-opacity:1;marker-start:url(#Arrow1Lstart);marker-end:none"
+ d="m 176.26096,124.64833 v 69.24854"
+ id="path17209-3"
+ inkscape:connector-type="orthogonal"
+ inkscape:connector-curvature="0" />
+ <path
+ transform="matrix(0.73232502,0,0,0.75477602,-4.325033,50.642983)"
+ sodipodi:type="arc"
+ style="fill:#539de6;fill-opacity:1;stroke:#0000ea;stroke-width:1;stroke-linecap:square;stroke-miterlimit:4;stroke-opacity:1;stroke-dasharray:none"
+ id="path29161-3"
+ sodipodi:cx="371"
+ sodipodi:cy="64.5"
+ sodipodi:rx="17"
+ sodipodi:ry="15.5"
+ d="m 388,64.5 a 17,15.5 0 1 1 -34,0 17,15.5 0 1 1 34,0 z" />
+ <path
+ style="fill:none;stroke:#000000;stroke-width:0.743466px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1;marker-mid:none;marker-end:url(#Arrow1Lend)"
+ d="M 322.61264,375 H 221.38736"
+ id="path17209-8-0"
+ inkscape:connector-type="orthogonal"
+ inkscape:connector-curvature="0" />
+ <path
+ transform="matrix(0.73232502,0,0,0.75477602,0.0689171,324.80833)"
+ sodipodi:type="arc"
+ style="fill:#539de6;fill-opacity:1;stroke:#0000ea;stroke-width:1;stroke-linecap:square;stroke-miterlimit:4;stroke-opacity:1;stroke-dasharray:none"
+ id="path29161-1"
+ sodipodi:cx="371"
+ sodipodi:cy="64.5"
+ sodipodi:rx="17"
+ sodipodi:ry="15.5"
+ d="m 388,64.5 a 17,15.5 0 1 1 -34,0 17,15.5 0 1 1 34,0 z" />
+ <path
+ style="fill:none;stroke:#000000;stroke-width:0.629082px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1;marker-end:url(#Arrow1Lend)"
+ d="M 155,324.19955 V 264.82863"
+ id="path17211-7-1"
+ inkscape:connector-type="orthogonal"
+ inkscape:connector-curvature="0" />
+ <path
+ transform="matrix(0.73232502,0,0,0.75477602,-116.37076,245.97088)"
+ sodipodi:type="arc"
+ style="fill:#539de6;fill-opacity:1;stroke:#0000ea;stroke-width:1;stroke-linecap:square;stroke-miterlimit:4;stroke-opacity:1;stroke-dasharray:none"
+ id="path29161-8"
+ sodipodi:cx="371"
+ sodipodi:cy="64.5"
+ sodipodi:rx="17"
+ sodipodi:ry="15.5"
+ d="m 388,64.5 a 17,15.5 0 1 1 -34,0 17,15.5 0 1 1 34,0 z" />
+ <path
+ style="fill:none;stroke:#000000;stroke-width:0.75059;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:4;stroke-dasharray:none;stroke-opacity:1;marker-start:url(#Arrow1Lstart-7);marker-end:none"
+ d="m 126.26097,124.99178 v 69.24941"
+ id="path17209-3-0"
+ inkscape:connector-type="orthogonal"
+ inkscape:connector-curvature="0" />
+ <path
+ sodipodi:type="arc"
+ style="fill:#539de6;fill-opacity:1;stroke:#0000ea;stroke-width:1;stroke-linecap:square;stroke-miterlimit:4;stroke-opacity:1;stroke-dasharray:none"
+ id="path29161"
+ sodipodi:cx="371"
+ sodipodi:cy="64.5"
+ sodipodi:rx="17"
+ sodipodi:ry="15.5"
+ d="m 388,64.5 a 17,15.5 0 1 1 -34,0 17,15.5 0 1 1 34,0 z"
+ transform="matrix(0.73232502,0,0,0.75477602,-146.16394,110.68276)" />
+ <path
+ sodipodi:type="arc"
+ style="fill:#539de6;fill-opacity:1;stroke:#0000ea;stroke-width:1;stroke-linecap:square;stroke-miterlimit:4;stroke-opacity:1;stroke-dasharray:none"
+ id="path29161-2"
+ sodipodi:cx="371"
+ sodipodi:cy="64.5"
+ sodipodi:rx="17"
+ sodipodi:ry="15.5"
+ d="m 388,64.5 a 17,15.5 0 1 1 -34,0 17,15.5 0 1 1 34,0 z"
+ transform="matrix(0.73232502,0,0,0.75477602,-95.40136,110.71627)" />
+ <g
+ id="g29167-4"
+ transform="matrix(0.73232502,0,0,0.75477602,-95.31662,112.00756)">
+ <text
+ id="text29163-9"
+ y="70"
+ x="365"
+ style="font-style:normal;font-weight:normal;line-height:0%;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none"
+ xml:space="preserve"><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="70"
+ x="365"
+ id="tspan29165-9"
+ sodipodi:role="line">1</tspan></text>
+ </g>
+ <g
+ id="g29167-9"
+ transform="matrix(0.73232502,0,0,0.75477602,-4.9726112,50.689051)">
+ <text
+ id="text29163-3"
+ y="70"
+ x="365"
+ style="font-style:normal;font-weight:normal;line-height:0%;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none"
+ xml:space="preserve"><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="70"
+ x="365"
+ id="tspan29165-3"
+ sodipodi:role="line">2</tspan></text>
+ </g>
+ <path
+ style="fill:none;stroke:#000000;stroke-width:1.04033px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1;marker-end:url(#Arrow1Lend)"
+ d="M 388.20118,147.93341 V 321.89308"
+ id="path17211-7"
+ inkscape:connector-type="orthogonal"
+ inkscape:connector-curvature="0" />
+ <path
+ transform="matrix(0.73232502,0,0,0.75477602,116.5086,136.68745)"
+ sodipodi:type="arc"
+ style="fill:#539de6;fill-opacity:1;stroke:#0000ea;stroke-width:1;stroke-linecap:square;stroke-miterlimit:4;stroke-opacity:1;stroke-dasharray:none"
+ id="path29161-6"
+ sodipodi:cx="371"
+ sodipodi:cy="64.5"
+ sodipodi:rx="17"
+ sodipodi:ry="15.5"
+ d="m 388,64.5 a 17,15.5 0 1 1 -34,0 17,15.5 0 1 1 34,0 z" />
+ <g
+ id="g29167-46"
+ transform="matrix(0.73232502,0,0,0.75477602,116.59334,137.48829)">
+ <text
+ id="text29163-1"
+ y="70"
+ x="365"
+ style="font-style:normal;font-weight:normal;line-height:0%;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none"
+ xml:space="preserve"><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="70"
+ x="365"
+ id="tspan29165-5"
+ sodipodi:role="line">3</tspan></text>
+ </g>
+ <g
+ id="g29167-6"
+ transform="matrix(0.73232502,0,0,0.75477602,0.1536639,325.60918)">
+ <text
+ id="text29163-17"
+ y="70"
+ x="365"
+ style="font-style:normal;font-weight:normal;line-height:0%;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none"
+ xml:space="preserve"><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="70"
+ x="365"
+ id="tspan29165-2"
+ sodipodi:role="line">4</tspan></text>
+ </g>
+ <g
+ id="g29167"
+ transform="matrix(0.73232502,0,0,0.75477602,-146.07919,111.10621)">
+ <text
+ id="text29163"
+ y="70"
+ x="365"
+ style="font-style:normal;font-weight:normal;line-height:0%;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none"
+ xml:space="preserve"><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="70"
+ x="365"
+ id="tspan29165"
+ sodipodi:role="line">6</tspan></text>
+ </g>
+ <g
+ id="g29167-4-3"
+ transform="matrix(0.73232502,0,0,0.75477602,-117.60784,180.11779)">
+ <text
+ id="text29163-9-6"
+ y="70"
+ x="321.30356"
+ style="font-style:normal;font-weight:normal;line-height:0%;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none"
+ xml:space="preserve"><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:24.2109px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="70"
+ x="321.30356"
+ id="tspan29165-9-7"
+ sodipodi:role="line">Eventdev</tspan></text>
+ </g>
+ <g
+ id="g29167-4-3-5"
+ transform="matrix(0.73232502,0,0,0.75477602,55.34956,26.97821)">
+ <text
+ id="text29163-9-6-3"
+ y="70"
+ x="454.74152"
+ style="font-style:normal;font-weight:normal;line-height:0%;font-family:sans-serif;text-align:center;letter-spacing:0px;word-spacing:0px;text-anchor:middle;fill:#000000;fill-opacity:1;stroke:none"
+ xml:space="preserve"><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:21.5208px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:center;writing-mode:lr-tb;text-anchor:middle"
+ y="70"
+ x="454.74152"
+ id="tspan29165-9-7-5"
+ sodipodi:role="line">A<tspan
+ style="line-height:100%;font-family:sans-serif"
+ id="tspan3374">tomic Stage</tspan></tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:21.5208px;line-height:100%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:center;writing-mode:lr-tb;text-anchor:middle"
+ y="94.210899"
+ x="454.74152"
+ sodipodi:role="line"
+ id="tspan3320">+</tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:21.5208px;line-height:100%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:center;writing-mode:lr-tb;text-anchor:middle"
+ y="115.7317"
+ x="454.74152"
+ sodipodi:role="line"
+ id="tspan3322">Enqueue to</tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:21.5208px;line-height:100%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:center;writing-mode:lr-tb;text-anchor:middle"
+ y="137.2525"
+ x="454.74152"
+ sodipodi:role="line"
+ id="tspan3324">DMA Device</tspan></text>
+ </g>
+ <g
+ id="g29167-2"
+ transform="matrix(0.73232502,0,0,0.75477602,-116.28602,248.01695)">
+ <text
+ id="text29163-92"
+ y="70"
+ x="365"
+ style="font-style:normal;font-weight:normal;line-height:0%;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none"
+ xml:space="preserve"><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="70"
+ x="365"
+ id="tspan29165-8"
+ sodipodi:role="line">5</tspan></text>
+ </g>
+ <flowRoot
+ xml:space="preserve"
+ id="flowRoot3376"
+ style="font-style:normal;font-weight:normal;line-height:0.01%;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none;stroke-width:1px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1"><flowRegion
+ id="flowRegion3378"
+ style="font-family:sans-serif"><rect
+ id="rect3380"
+ width="100"
+ height="37"
+ x="109"
+ y="259" /></flowRegion><flowPara
+ id="flowPara3382"
+ style="font-size:18px;line-height:1.25;font-family:sans-serif"> </flowPara></flowRoot>
+ <g
+ id="g29167-4-3-1"
+ transform="matrix(0.73232502,0,0,0.75477602,109.34956,323.97821)">
+ <text
+ id="text29163-9-6-8"
+ y="70"
+ x="321.30356"
+ style="font-style:normal;font-weight:normal;line-height:0%;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none"
+ xml:space="preserve"><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:24.2109px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="70"
+ x="321.30356"
+ id="tspan29165-9-7-7"
+ sodipodi:role="line">DMA Device</tspan></text>
+ </g>
+ <g
+ id="g29167-4-3-1-9"
+ transform="matrix(0.73232502,0,0,0.75477602,-114.48565,314.20704)">
+ <text
+ id="text29163-9-6-8-2"
+ y="70"
+ x="368.01718"
+ style="font-style:normal;font-weight:normal;line-height:0%;font-family:sans-serif;text-align:center;letter-spacing:0px;word-spacing:0px;text-anchor:middle;fill:#000000;fill-opacity:1;stroke:none"
+ xml:space="preserve"><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:24.2109px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:center;writing-mode:lr-tb;text-anchor:middle"
+ y="70"
+ x="368.01718"
+ id="tspan29165-9-7-7-0"
+ sodipodi:role="line">DMA</tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:24.2109px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:center;writing-mode:lr-tb;text-anchor:middle"
+ y="100.26363"
+ x="368.01718"
+ sodipodi:role="line"
+ id="tspan3488">Adapter</tspan></text>
+ </g>
+ <g
+ id="g29167-4-3-1-9-2"
+ transform="matrix(0.73232502,0,0,0.75477602,250.96804,192.62529)">
+ <text
+ id="text29163-9-6-8-2-3"
+ y="-188.35481"
+ x="318.61978"
+ style="font-style:normal;font-weight:normal;line-height:0%;font-family:sans-serif;text-align:center;letter-spacing:0px;word-spacing:0px;text-anchor:middle;fill:#000000;fill-opacity:1;stroke:none"
+ xml:space="preserve"><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:21.5208px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="-188.35481"
+ x="318.61978"
+ sodipodi:role="line"
+ id="tspan3543">1. Application dequeues</tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:21.5208px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="-161.45381"
+ x="318.61978"
+ sodipodi:role="line"
+ id="tspan3196"> events from the previous</tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:21.5208px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="-134.55281"
+ x="318.61978"
+ sodipodi:role="line"
+ id="tspan3232"> stage</tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:21.5208px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="-107.65182"
+ x="318.61978"
+ sodipodi:role="line"
+ id="tspan3519"> </tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:21.5208px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="-80.750816"
+ x="318.61978"
+ sodipodi:role="line"
+ id="tspan3551">2. Application prepares the</tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:21.5208px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="-53.849815"
+ x="318.61978"
+ sodipodi:role="line"
+ id="tspan3203"> DMA operations.</tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:21.5208px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="-26.948814"
+ x="318.61978"
+ sodipodi:role="line"
+ id="tspan3523"> </tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:21.5208px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="-0.0478158"
+ x="318.61978"
+ sodipodi:role="line"
+ id="tspan3541">3. DMA operations are</tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:21.5208px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="26.853184"
+ x="318.61978"
+ sodipodi:role="line"
+ id="tspan3207"> submitted to dmadev</tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:21.5208px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="53.754185"
+ x="318.61978"
+ sodipodi:role="line"
+ id="tspan3209"> by application.</tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:21.5208px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="80.655182"
+ x="318.61978"
+ sodipodi:role="line"
+ id="tspan3527"> </tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:21.5208px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="107.55618"
+ x="318.61978"
+ sodipodi:role="line"
+ id="tspan3547">4. DMA adapter dequeues</tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:21.5208px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="134.45718"
+ x="318.61978"
+ sodipodi:role="line"
+ id="tspan3216"> DMA completions from</tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:21.5208px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="161.35818"
+ x="318.61978"
+ sodipodi:role="line"
+ id="tspan3218"> DMA device.</tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:21.5208px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="188.25919"
+ x="318.61978"
+ sodipodi:role="line"
+ id="tspan3531"> </tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:21.5208px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="215.16019"
+ x="318.61978"
+ sodipodi:role="line"
+ id="tspan3549">5. DMA adapter enqueues</tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:21.5208px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="242.06119"
+ x="318.61978"
+ sodipodi:role="line"
+ id="tspan3222"> events to the eventdev.</tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:21.5208px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="268.96219"
+ x="318.61978"
+ sodipodi:role="line"
+ id="tspan3535"> </tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:21.5208px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="295.86319"
+ x="318.61978"
+ sodipodi:role="line"
+ id="tspan3537">6. Application dequeues from</tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:21.5208px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="322.76419"
+ x="318.61978"
+ sodipodi:role="line"
+ id="tspan3224"> eventdev and prepare for</tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:21.5208px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="349.66519"
+ x="318.61978"
+ sodipodi:role="line"
+ id="tspan3226"> further processing</tspan></text>
+ </g>
+ <g
+ style="font-size:12px;fill:none;stroke-linecap:square;stroke-miterlimit:3;overflow:visible"
+ id="shape1-1-2-4-7"
+ v:mID="1"
+ v:groupContext="shape"
+ transform="matrix(2.1604167,0,0,1.165886,90.820551,-587.97129)">
+ <title
+ id="title22-7-5-5">Square</title>
+ <desc
+ id="desc24-7-8-3">Atomic Queue #1</desc>
+ <v:userDefs>
+ <v:ud
+ v:nameU="visVersion"
+ v:val="VT0(15):26" />
+ </v:userDefs>
+ <v:textBlock
+ v:margins="rect(4,4,4,4)" />
+ <v:textRect
+ cx="30.75"
+ cy="581.25"
+ width="61.5"
+ height="61.5" />
+ <g
+ id="shadow1-2-9-5-5"
+ v:groupContext="shadow"
+ v:shadowOffsetX="0.345598"
+ v:shadowOffsetY="-1.97279"
+ v:shadowType="1"
+ transform="translate(0.345598,1.97279)"
+ class="st1"
+ style="visibility:visible">
+ <rect
+ x="0"
+ y="550.5"
+ width="61.5"
+ height="61.5"
+ class="st2"
+ id="rect27-8-7-6"
+ style="fill:#5b9bd5;fill-opacity:0.22000002;stroke:#5b9bd5;stroke-opacity:0.22000002;filter:url(#filter_2-3-91-3)" />
+ </g>
+ <g
+ id="g13515-33-2">
+ <g
+ id="g13534-8-9">
+ <rect
+ x="0"
+ y="550.5"
+ width="61.5"
+ height="61.5"
+ class="st3"
+ id="rect29-1-95-1"
+ style="fill:#5b9bd5;stroke:#c7c8c8;stroke-width:0.25" />
+ </g>
+ </g>
+ </g>
+ <g
+ id="g29167-4-3-2"
+ transform="matrix(0.73232502,0,0,0.75477602,-125.66199,44.027402)">
+ <text
+ id="text29163-9-6-7"
+ y="70"
+ x="321.30356"
+ style="font-style:normal;font-weight:normal;line-height:0%;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none"
+ xml:space="preserve"><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:24.2109px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="70"
+ x="321.30356"
+ id="tspan29165-9-7-0"
+ sodipodi:role="line">Application</tspan></text>
+ </g>
+ </g>
+</svg>
diff --git a/doc/guides/prog_guide/index.rst b/doc/guides/prog_guide/index.rst
index 52a6d9e7aa..beaa4b8869 100644
--- a/doc/guides/prog_guide/index.rst
+++ b/doc/guides/prog_guide/index.rst
@@ -60,6 +60,7 @@ Programmer's Guide
event_ethernet_tx_adapter
event_timer_adapter
event_crypto_adapter
+ event_dma_adapter
qos_framework
power_man
packet_classif_access_ctrl
diff --git a/doc/guides/rel_notes/release_23_11.rst b/doc/guides/rel_notes/release_23_11.rst
index b34ddc0860..1a1f337d23 100644
--- a/doc/guides/rel_notes/release_23_11.rst
+++ b/doc/guides/rel_notes/release_23_11.rst
@@ -89,6 +89,11 @@ New Features
* Added support for ``remaining_ticks_get`` timer adapter PMD callback
to get the remaining ticks to expire for a given event timer.
+* **Added event DMA adapter library.**
+
+ * Added the Event DMA Adapter Library. This library extends the event-based
+ model by introducing APIs that allow applications to enqueue/dequeue DMA
+ operations to/from dmadev as events scheduled by an event device.
Removed Items
-------------
diff --git a/lib/eventdev/eventdev_pmd.h b/lib/eventdev/eventdev_pmd.h
index f62f42e140..f7227c0bfd 100644
--- a/lib/eventdev/eventdev_pmd.h
+++ b/lib/eventdev/eventdev_pmd.h
@@ -178,8 +178,12 @@ struct rte_eventdev {
event_tx_adapter_enqueue_t txa_enqueue;
/**< Pointer to PMD eth Tx adapter enqueue function. */
event_crypto_adapter_enqueue_t ca_enqueue;
+ /**< Pointer to PMD crypto adapter enqueue function. */
- uint64_t reserved_64s[4]; /**< Reserved for future fields */
+ event_dma_adapter_enqueue_t dma_enqueue;
+ /**< Pointer to PMD DMA adapter enqueue function. */
+
+ uint64_t reserved_64s[3]; /**< Reserved for future fields */
void *reserved_ptrs[3]; /**< Reserved for future fields */
} __rte_cache_aligned;
@@ -1320,6 +1324,156 @@ typedef int (*eventdev_eth_tx_adapter_queue_stop)
#define eventdev_stop_flush_t rte_eventdev_stop_flush_t
+/**
+ * Retrieve the event device's DMA adapter capabilities for the
+ * specified DMA device
+ *
+ * @param dev
+ * Event device pointer
+ *
+ * @param dma_dev_id
+ * DMA device identifier
+ *
+ * @param[out] caps
+ * A pointer to memory filled with event adapter capabilities.
+ * It is expected to be pre-allocated & initialized by caller.
+ *
+ * @return
+ * - 0: Success, driver provides event adapter capabilities for the
+ * dmadev.
+ * - <0: Error code returned by the driver function.
+ *
+ */
+typedef int (*eventdev_dma_adapter_caps_get_t)(const struct rte_eventdev *dev,
+ const int16_t dma_dev_id, uint32_t *caps);
+
+/**
+ * Add DMA vchan queue to event device. This callback is invoked if
+ * the caps returned from rte_event_dma_adapter_caps_get(, dmadev_id)
+ * has RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_* set.
+ *
+ * @param dev
+ * Event device pointer
+ *
+ * @param dma_dev_id
+ * DMA device identifier
+ *
+ * @param vchan_id
+ * dmadev vchan queue identifier.
+ *
+ * @param event
+ * Event information required for binding dmadev vchan to event queue.
+ * This structure will have a valid value for only those HW PMDs supporting
+ * @see RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_VCHAN_EV_BIND capability.
+ *
+ * @return
+ * - 0: Success, dmadev vchan added successfully.
+ * - <0: Error code returned by the driver function.
+ *
+ */
+typedef int (*eventdev_dma_adapter_vchan_add_t)(const struct rte_eventdev *dev,
+ const int16_t dma_dev_id,
+ uint16_t vchan_id,
+ const struct rte_event *event);
+
+/**
+ * Delete DMA vhcan to event device. This callback is invoked if
+ * the caps returned from rte_event_dma_adapter_caps_get(, dmadev_id)
+ * has RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_* set.
+ *
+ * @param dev
+ * Event device pointer
+ *
+ * @param dma_dev_id
+ * DMA device identifier
+ *
+ * @param vchan_id
+ * dmadev vchan identifier.
+ *
+ * @return
+ * - 0: Success, dmadev vchan deleted successfully.
+ * - <0: Error code returned by the driver function.
+ *
+ */
+typedef int (*eventdev_dma_adapter_vchan_del_t)(const struct rte_eventdev *dev,
+ const int16_t dma_dev_id,
+ uint16_t vchan_id);
+
+/**
+ * Start DMA adapter. This callback is invoked if
+ * the caps returned from rte_event_dma_adapter_caps_get(.., dmadev_id)
+ * has RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_* set and vchan for dmadev_id
+ * have been added to the event device.
+ *
+ * @param dev
+ * Event device pointer
+ *
+ * @param dma_dev_id
+ * DMA device identifier
+ *
+ * @return
+ * - 0: Success, DMA adapter started successfully.
+ * - <0: Error code returned by the driver function.
+ */
+typedef int (*eventdev_dma_adapter_start_t)(const struct rte_eventdev *dev,
+ const int16_t dma_dev_id);
+
+/**
+ * Stop DMA adapter. This callback is invoked if
+ * the caps returned from rte_event_dma_adapter_caps_get(.., dmadev_id)
+ * has RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_* set and vchan for dmadev_id
+ * have been added to the event device.
+ *
+ * @param dev
+ * Event device pointer
+ *
+ * @param dma_dev_id
+ * DMA device identifier
+ *
+ * @return
+ * - 0: Success, DMA adapter stopped successfully.
+ * - <0: Error code returned by the driver function.
+ */
+typedef int (*eventdev_dma_adapter_stop_t)(const struct rte_eventdev *dev,
+ const int16_t dma_dev_id);
+
+struct rte_event_dma_adapter_stats;
+
+/**
+ * Retrieve DMA adapter statistics.
+ *
+ * @param dev
+ * Event device pointer
+ *
+ * @param dma_dev_id
+ * DMA device identifier
+ *
+ * @param[out] stats
+ * Pointer to stats structure
+ *
+ * @return
+ * Return 0 on success.
+ */
+typedef int (*eventdev_dma_adapter_stats_get)(const struct rte_eventdev *dev,
+ const int16_t dma_dev_id,
+ struct rte_event_dma_adapter_stats *stats);
+
+/**
+ * Reset DMA adapter statistics.
+ *
+ * @param dev
+ * Event device pointer
+ *
+ * @param dma_dev_id
+ * DMA device identifier
+ *
+ * @return
+ * Return 0 on success.
+ */
+typedef int (*eventdev_dma_adapter_stats_reset)(const struct rte_eventdev *dev,
+ const int16_t dma_dev_id);
+
+
/** Event device operations function pointer table */
struct eventdev_ops {
eventdev_info_get_t dev_infos_get; /**< Get device info. */
@@ -1440,6 +1594,21 @@ struct eventdev_ops {
eventdev_eth_tx_adapter_queue_stop eth_tx_adapter_queue_stop;
/**< Stop Tx queue assigned to Tx adapter instance */
+ eventdev_dma_adapter_caps_get_t dma_adapter_caps_get;
+ /**< Get DMA adapter capabilities */
+ eventdev_dma_adapter_vchan_add_t dma_adapter_vchan_add;
+ /**< Add vchan queue to DMA adapter */
+ eventdev_dma_adapter_vchan_del_t dma_adapter_vchan_del;
+ /**< Delete vchan queue from DMA adapter */
+ eventdev_dma_adapter_start_t dma_adapter_start;
+ /**< Start DMA adapter */
+ eventdev_dma_adapter_stop_t dma_adapter_stop;
+ /**< Stop DMA adapter */
+ eventdev_dma_adapter_stats_get dma_adapter_stats_get;
+ /**< Get DMA stats */
+ eventdev_dma_adapter_stats_reset dma_adapter_stats_reset;
+ /**< Reset DMA stats */
+
eventdev_selftest dev_selftest;
/**< Start eventdev Selftest */
diff --git a/lib/eventdev/eventdev_private.c b/lib/eventdev/eventdev_private.c
index 1d3d9d357e..18ed8bf3c8 100644
--- a/lib/eventdev/eventdev_private.c
+++ b/lib/eventdev/eventdev_private.c
@@ -81,6 +81,14 @@ dummy_event_crypto_adapter_enqueue(__rte_unused void *port,
return 0;
}
+static uint16_t
+dummy_event_dma_adapter_enqueue(__rte_unused void *port, __rte_unused struct rte_event ev[],
+ __rte_unused uint16_t nb_events)
+{
+ RTE_EDEV_LOG_ERR("event DMA adapter enqueue requested for unconfigured event device");
+ return 0;
+}
+
void
event_dev_fp_ops_reset(struct rte_event_fp_ops *fp_op)
{
@@ -97,6 +105,7 @@ event_dev_fp_ops_reset(struct rte_event_fp_ops *fp_op)
.txa_enqueue_same_dest =
dummy_event_tx_adapter_enqueue_same_dest,
.ca_enqueue = dummy_event_crypto_adapter_enqueue,
+ .dma_enqueue = dummy_event_dma_adapter_enqueue,
.data = dummy_data,
};
@@ -117,5 +126,6 @@ event_dev_fp_ops_set(struct rte_event_fp_ops *fp_op,
fp_op->txa_enqueue = dev->txa_enqueue;
fp_op->txa_enqueue_same_dest = dev->txa_enqueue_same_dest;
fp_op->ca_enqueue = dev->ca_enqueue;
+ fp_op->dma_enqueue = dev->dma_enqueue;
fp_op->data = dev->data->ports;
}
diff --git a/lib/eventdev/meson.build b/lib/eventdev/meson.build
index 6edf98dfa5..21347f7c4c 100644
--- a/lib/eventdev/meson.build
+++ b/lib/eventdev/meson.build
@@ -25,6 +25,7 @@ sources = files(
)
headers = files(
'rte_event_crypto_adapter.h',
+ 'rte_event_dma_adapter.h',
'rte_event_eth_rx_adapter.h',
'rte_event_eth_tx_adapter.h',
'rte_event_ring.h',
diff --git a/lib/eventdev/rte_event_dma_adapter.h b/lib/eventdev/rte_event_dma_adapter.h
new file mode 100644
index 0000000000..e924ab673d
--- /dev/null
+++ b/lib/eventdev/rte_event_dma_adapter.h
@@ -0,0 +1,581 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright (c) 2023 Marvell.
+ */
+
+#ifndef RTE_EVENT_DMA_ADAPTER
+#define RTE_EVENT_DMA_ADAPTER
+
+/**
+ * @file rte_event_dma_adapter.h
+ *
+ * @warning
+ * @b EXPERIMENTAL:
+ * All functions in this file may be changed or removed without prior notice.
+ *
+ * DMA Event Adapter API.
+ *
+ * Eventdev library provides adapters to bridge between various components for providing new
+ * event source. The event DMA adapter is one of those adapters which is intended to bridge
+ * between event devices and DMA devices.
+ *
+ * The DMA adapter adds support to enqueue / dequeue DMA operations to / from event device. The
+ * packet flow between DMA device and the event device can be accomplished using both SW and HW
+ * based transfer mechanisms. The adapter uses an EAL service core function for SW based packet
+ * transfer and uses the eventdev PMD functions to configure HW based packet transfer between the
+ * DMA device and the event device.
+ *
+ * The application can choose to submit a DMA operation directly to an DMA device or send it to the
+ * DMA adapter via eventdev based on RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP_FWD capability. The
+ * first mode is known as the event new (RTE_EVENT_DMA_ADAPTER_OP_NEW) mode and the second as the
+ * event forward (RTE_EVENT_DMA_ADAPTER_OP_FORWARD) mode. The choice of mode can be specified while
+ * creating the adapter. In the former mode, it is an application responsibility to enable ingress
+ * packet ordering. In the latter mode, it is the adapter responsibility to enable the ingress
+ * packet ordering.
+ *
+ *
+ * Working model of RTE_EVENT_DMA_ADAPTER_OP_NEW mode:
+ *
+ * +--------------+ +--------------+
+ * | | | DMA stage |
+ * | Application |---[2]-->| + enqueue to |
+ * | | | dmadev |
+ * +--------------+ +--------------+
+ * ^ ^ |
+ * | | [3]
+ * [6] [1] |
+ * | | |
+ * +--------------+ |
+ * | | |
+ * | Event device | |
+ * | | |
+ * +--------------+ |
+ * ^ |
+ * | |
+ * [5] |
+ * | v
+ * +--------------+ +--------------+
+ * | | | |
+ * | DMA adapter |<--[4]---| dmadev |
+ * | | | |
+ * +--------------+ +--------------+
+ *
+ *
+ * [1] Application dequeues events from the previous stage.
+ * [2] Application prepares the DMA operations.
+ * [3] DMA operations are submitted to dmadev by application.
+ * [4] DMA adapter dequeues DMA completions from dmadev.
+ * [5] DMA adapter enqueues events to the eventdev.
+ * [6] Application dequeues from eventdev for further processing.
+ *
+ * In the RTE_EVENT_DMA_ADAPTER_OP_NEW mode, application submits DMA operations directly to DMA
+ * device. The DMA adapter then dequeues DMA completions from DMA device and enqueue events to the
+ * event device. This mode does not ensure ingress ordering, if the application directly enqueues
+ * to dmadev without going through DMA / atomic stage i.e. removing item [1] and [2].
+ *
+ * Events dequeued from the adapter will be treated as new events. In this mode, application needs
+ * to specify event information (response information) which is needed to enqueue an event after the
+ * DMA operation is completed.
+ *
+ *
+ * Working model of RTE_EVENT_DMA_ADAPTER_OP_FORWARD mode:
+ *
+ * +--------------+ +--------------+
+ * --[1]-->| |---[2]-->| Application |
+ * | Event device | | in |
+ * <--[8]--| |<--[3]---| Ordered stage|
+ * +--------------+ +--------------+
+ * ^ |
+ * | [4]
+ * [7] |
+ * | v
+ * +----------------+ +--------------+
+ * | |--[5]->| |
+ * | DMA adapter | | dmadev |
+ * | |<-[6]--| |
+ * +----------------+ +--------------+
+ *
+ *
+ * [1] Events from the previous stage.
+ * [2] Application in ordered stage dequeues events from eventdev.
+ * [3] Application enqueues DMA operations as events to eventdev.
+ * [4] DMA adapter dequeues event from eventdev.
+ * [5] DMA adapter submits DMA operations to dmadev (Atomic stage).
+ * [6] DMA adapter dequeues DMA completions from dmadev
+ * [7] DMA adapter enqueues events to the eventdev
+ * [8] Events to the next stage
+ *
+ * In the event forward (RTE_EVENT_DMA_ADAPTER_OP_FORWARD) mode, if the HW supports the capability
+ * RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP_FWD, application can directly submit the DMA
+ * operations to the dmadev. If not, application retrieves the event port of the DMA adapter
+ * through the API, rte_event_DMA_adapter_event_port_get(). Then, links its event queue to this
+ * port and starts enqueuing DMA operations as events to the eventdev. The adapter then dequeues
+ * the events and submits the DMA operations to the dmadev. After the DMA completions, the adapter
+ * enqueues events to the event device.
+ *
+ * Application can use this mode, when ingress packet ordering is needed. Events dequeued from the
+ * adapter will be treated as forwarded events. In this mode, the application needs to specify the
+ * dmadev ID and queue pair ID (request information) needed to enqueue an DMA operation in addition
+ * to the event information (response information) needed to enqueue an event after the DMA
+ * operation has completed.
+ *
+ * The event DMA adapter provides common APIs to configure the packet flow from the DMA device to
+ * event devices for both SW and HW based transfers. The DMA event adapter's functions are:
+ *
+ * - rte_event_dma_adapter_create_ext()
+ * - rte_event_dma_adapter_create()
+ * - rte_event_dma_adapter_free()
+ * - rte_event_dma_adapter_vchan_add()
+ * - rte_event_dma_adapter_vchan_del()
+ * - rte_event_dma_adapter_start()
+ * - rte_event_dma_adapter_stop()
+ * - rte_event_dma_adapter_stats_get()
+ * - rte_event_dma_adapter_stats_reset()
+ *
+ * The application creates an instance using rte_event_dma_adapter_create() or
+ * rte_event_dma_adapter_create_ext().
+ *
+ * dmadev queue pair addition / deletion is done using the rte_event_dma_adapter_vchan_add() /
+ * rte_event_dma_adapter_vchan_del() APIs. If HW supports the capability
+ * RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_VCHAN_EV_BIND, event information must be passed to the
+ * add API.
+ *
+ */
+
+#include <stdint.h>
+
+#include <rte_common.h>
+#include <rte_dmadev_pmd.h>
+#include <rte_eventdev.h>
+
+#ifdef __cplusplus
+extern "C" {
+#endif
+
+/**
+ * A structure used to hold event based DMA operation entry. All the information
+ * required for a DMA transfer shall be populated in "struct rte_event_dma_adapter_op"
+ * instance.
+ */
+struct rte_event_dma_adapter_op {
+ struct rte_dma_sge *src_seg;
+ /**< Source segments. */
+ struct rte_dma_sge *dst_seg;
+ /**< Destination segments. */
+ uint16_t nb_src;
+ /**< Number of source segments. */
+ uint16_t nb_dst;
+ /**< Number of destination segments. */
+ uint64_t flags;
+ /**< Flags related to the operation.
+ * @see RTE_DMA_OP_FLAG_*
+ */
+ int16_t dma_dev_id;
+ /**< DMA device ID to be used */
+ uint16_t vchan;
+ /**< DMA vchan ID to be used */
+ struct rte_mempool *op_mp;
+ /**< Mempool from which op is allocated. */
+};
+
+/**
+ * DMA event adapter mode
+ */
+enum rte_event_dma_adapter_mode {
+ RTE_EVENT_DMA_ADAPTER_OP_NEW,
+ /**< Start the DMA adapter in event new mode.
+ * @see RTE_EVENT_OP_NEW.
+ *
+ * Application submits DMA operations to the dmadev. Adapter only dequeues the DMA
+ * completions from dmadev and enqueue events to the eventdev.
+ */
+
+ RTE_EVENT_DMA_ADAPTER_OP_FORWARD,
+ /**< Start the DMA adapter in event forward mode.
+ * @see RTE_EVENT_OP_FORWARD.
+ *
+ * Application submits DMA requests as events to the DMA adapter or DMA device based on
+ * RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP_FWD capability. DMA completions are enqueued
+ * back to the eventdev by DMA adapter.
+ */
+};
+
+/**
+ * Adapter configuration structure that the adapter configuration callback function is expected to
+ * fill out.
+ *
+ * @see rte_event_dma_adapter_conf_cb
+ */
+struct rte_event_dma_adapter_conf {
+ uint8_t event_port_id;
+ /** < Event port identifier, the adapter enqueues events to this port and dequeues DMA
+ * request events in RTE_EVENT_DMA_ADAPTER_OP_FORWARD mode.
+ */
+
+ uint32_t max_nb;
+ /**< The adapter can return early if it has processed at least max_nb DMA ops. This isn't
+ * treated as a requirement; batching may cause the adapter to process more than max_nb DMA
+ * ops.
+ */
+};
+
+/**
+ * Adapter runtime configuration parameters
+ */
+struct rte_event_dma_adapter_runtime_params {
+ uint32_t max_nb;
+ /**< The adapter can return early if it has processed at least max_nb DMA ops. This isn't
+ * treated as a requirement; batching may cause the adapter to process more than max_nb DMA
+ * ops.
+ *
+ * Callback function passed to rte_event_dma_adapter_create_ext() configures the adapter
+ * with default value of max_nb.
+ * rte_event_dma_adapter_runtime_params_set() allows to re-configure max_nb during runtime
+ * (after adding at least one queue pair)
+ *
+ * This is valid for the devices without RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP_FWD or
+ * RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP_NEW capability.
+ */
+
+ uint32_t rsvd[15];
+ /**< Reserved fields for future expansion */
+};
+
+/**
+ * Function type used for adapter configuration callback. The callback is used to fill in members of
+ * the struct rte_event_dma_adapter_conf, this callback is invoked when creating a SW service for
+ * packet transfer from dmadev vchan to the event device. The SW service is created within the
+ * function, rte_event_dma_adapter_vchan_add(), if SW based packet transfers from dmadev vchan
+ * to the event device are required.
+ *
+ * @param id
+ * Adapter identifier.
+ * @param evdev_id
+ * Event device identifier.
+ * @param conf
+ * Structure that needs to be populated by this callback.
+ * @param arg
+ * Argument to the callback. This is the same as the conf_arg passed to the
+ * rte_event_dma_adapter_create_ext().
+ */
+typedef int (*rte_event_dma_adapter_conf_cb)(uint8_t id, uint8_t evdev_id,
+ struct rte_event_dma_adapter_conf *conf, void *arg);
+
+/**
+ * A structure used to retrieve statistics for an event DMA adapter instance.
+ */
+struct rte_event_dma_adapter_stats {
+ uint64_t event_poll_count;
+ /**< Event port poll count */
+
+ uint64_t event_deq_count;
+ /**< Event dequeue count */
+
+ uint64_t dma_enq_count;
+ /**< dmadev enqueue count */
+
+ uint64_t dma_enq_fail_count;
+ /**< dmadev enqueue failed count */
+
+ uint64_t dma_deq_count;
+ /**< dmadev dequeue count */
+
+ uint64_t event_enq_count;
+ /**< Event enqueue count */
+
+ uint64_t event_enq_retry_count;
+ /**< Event enqueue retry count */
+
+ uint64_t event_enq_fail_count;
+ /**< Event enqueue fail count */
+};
+
+/**
+ * Create a new event DMA adapter with the specified identifier.
+ *
+ * @param id
+ * Adapter identifier.
+ * @param evdev_id
+ * Event device identifier.
+ * @param conf_cb
+ * Callback function that fills in members of a struct rte_event_dma_adapter_conf struct passed
+ * into it.
+ * @param mode
+ * Flag to indicate the mode of the adapter.
+ * @see rte_event_dma_adapter_mode
+ * @param conf_arg
+ * Argument that is passed to the conf_cb function.
+ *
+ * @return
+ * - 0: Success
+ * - <0: Error code on failure
+ */
+__rte_experimental
+int rte_event_dma_adapter_create_ext(uint8_t id, uint8_t evdev_id,
+ rte_event_dma_adapter_conf_cb conf_cb,
+ enum rte_event_dma_adapter_mode mode, void *conf_arg);
+
+/**
+ * Create a new event DMA adapter with the specified identifier. This function uses an internal
+ * configuration function that creates an event port. This default function reconfigures the event
+ * device with an additional event port and set up the event port using the port_config parameter
+ * passed into this function. In case the application needs more control in configuration of the
+ * service, it should use the rte_event_dma_adapter_create_ext() version.
+ *
+ * @param id
+ * Adapter identifier.
+ * @param evdev_id
+ * Event device identifier.
+ * @param port_config
+ * Argument of type *rte_event_port_conf* that is passed to the conf_cb function.
+ * @param mode
+ * Flag to indicate the mode of the adapter.
+ * @see rte_event_dma_adapter_mode
+ *
+ * @return
+ * - 0: Success
+ * - <0: Error code on failure
+ */
+__rte_experimental
+int rte_event_dma_adapter_create(uint8_t id, uint8_t evdev_id,
+ struct rte_event_port_conf *port_config,
+ enum rte_event_dma_adapter_mode mode);
+
+/**
+ * Free an event DMA adapter
+ *
+ * @param id
+ * Adapter identifier.
+ * @return
+ * - 0: Success
+ * - <0: Error code on failure, If the adapter still has queue pairs added to it, the function
+ * returns -EBUSY.
+ */
+__rte_experimental
+int rte_event_dma_adapter_free(uint8_t id);
+
+/**
+ * Retrieve the event port of an adapter.
+ *
+ * @param id
+ * Adapter identifier.
+ *
+ * @param [out] event_port_id
+ * Application links its event queue to this adapter port which is used in
+ * RTE_EVENT_DMA_ADAPTER_OP_FORWARD mode.
+ *
+ * @return
+ * - 0: Success
+ * - <0: Error code on failure.
+ */
+__rte_experimental
+int rte_event_dma_adapter_event_port_get(uint8_t id, uint8_t *event_port_id);
+
+/**
+ * Add a vchan to an event DMA adapter.
+ *
+ * @param id
+ * Adapter identifier.
+ * @param dmadev_id
+ * dmadev identifier.
+ * @param vchan
+ * DMA device vchan identifier. If vchan is set -1, adapter adds all the
+ * preconfigured vchan to the instance.
+ * @param event
+ * If HW supports dmadev vchan to event queue binding, application is expected to fill in
+ * event information, else it will be NULL.
+ * @see RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_VCHAN_EV_BIND
+ *
+ * @return
+ * - 0: Success, vchan added correctly.
+ * - <0: Error code on failure.
+ */
+__rte_experimental
+int rte_event_dma_adapter_vchan_add(uint8_t id, int16_t dmadev_id, uint16_t vchan,
+ const struct rte_event *event);
+
+/**
+ * Delete a vchan from an event DMA adapter.
+ *
+ * @param id
+ * Adapter identifier.
+ * @param dmadev_id
+ * DMA device identifier.
+ * @param vchan
+ * DMA device vchan identifier.
+ *
+ * @return
+ * - 0: Success, vchan deleted successfully.
+ * - <0: Error code on failure.
+ */
+__rte_experimental
+int rte_event_dma_adapter_vchan_del(uint8_t id, int16_t dmadev_id, uint16_t vchan);
+
+/**
+ * Retrieve the service ID of an adapter. If the adapter doesn't use a rte_service function, this
+ * function returns -ESRCH.
+ *
+ * @param id
+ * Adapter identifier.
+ * @param [out] service_id
+ * A pointer to a uint32_t, to be filled in with the service id.
+ *
+ * @return
+ * - 0: Success
+ * - <0: Error code on failure, if the adapter doesn't use a rte_service function, this function
+ * returns -ESRCH.
+ */
+__rte_experimental
+int rte_event_dma_adapter_service_id_get(uint8_t id, uint32_t *service_id);
+
+/**
+ * Start event DMA adapter
+ *
+ * @param id
+ * Adapter identifier.
+ *
+ * @return
+ * - 0: Success, adapter started successfully.
+ * - <0: Error code on failure.
+ *
+ * @note The eventdev and dmadev to which the event_dma_adapter is connected should be started
+ * before calling rte_event_dma_adapter_start().
+ */
+__rte_experimental
+int rte_event_dma_adapter_start(uint8_t id);
+
+/**
+ * Stop event DMA adapter
+ *
+ * @param id
+ * Adapter identifier.
+ *
+ * @return
+ * - 0: Success, adapter stopped successfully.
+ * - <0: Error code on failure.
+ */
+__rte_experimental
+int rte_event_dma_adapter_stop(uint8_t id);
+
+/**
+ * Initialize the adapter runtime configuration parameters
+ *
+ * @param params
+ * A pointer to structure of type struct rte_event_dma_adapter_runtime_params
+ *
+ * @return
+ * - 0: Success
+ * - <0: Error code on failure
+ */
+__rte_experimental
+int rte_event_dma_adapter_runtime_params_init(struct rte_event_dma_adapter_runtime_params *params);
+
+/**
+ * Set the adapter runtime configuration parameters
+ *
+ * @param id
+ * Adapter identifier
+ *
+ * @param params
+ * A pointer to structure of type struct rte_event_dma_adapter_runtime_params with configuration
+ * parameter values. The reserved fields of this structure must be initialized to zero and the valid
+ * fields need to be set appropriately. This struct can be initialized using
+ * rte_event_dma_adapter_runtime_params_init() API to default values or application may reset this
+ * struct and update required fields.
+ *
+ * @return
+ * - 0: Success
+ * - <0: Error code on failure
+ */
+__rte_experimental
+int rte_event_dma_adapter_runtime_params_set(uint8_t id,
+ struct rte_event_dma_adapter_runtime_params *params);
+
+/**
+ * Get the adapter runtime configuration parameters
+ *
+ * @param id
+ * Adapter identifier
+ *
+ * @param[out] params
+ * A pointer to structure of type struct rte_event_dma_adapter_runtime_params containing valid
+ * adapter parameters when return value is 0.
+ *
+ * @return
+ * - 0: Success
+ * - <0: Error code on failure
+ */
+__rte_experimental
+int rte_event_dma_adapter_runtime_params_get(uint8_t id,
+ struct rte_event_dma_adapter_runtime_params *params);
+
+/**
+ * Retrieve statistics for an adapter
+ *
+ * @param id
+ * Adapter identifier.
+ * @param [out] stats
+ * A pointer to structure used to retrieve statistics for an adapter.
+ *
+ * @return
+ * - 0: Success, retrieved successfully.
+ * - <0: Error code on failure.
+ */
+__rte_experimental
+int rte_event_dma_adapter_stats_get(uint8_t id, struct rte_event_dma_adapter_stats *stats);
+
+/**
+ * Reset statistics for an adapter.
+ *
+ * @param id
+ * Adapter identifier.
+ *
+ * @return
+ * - 0: Success, statistics reset successfully.
+ * - <0: Error code on failure.
+ */
+__rte_experimental
+int rte_event_dma_adapter_stats_reset(uint8_t id);
+
+/**
+ * Enqueue a burst of DMA operations as event objects supplied in *rte_event* structure on an event
+ * DMA adapter designated by its event *evdev_id* through the event port specified by *port_id*.
+ * This function is supported if the eventdev PMD has the
+ * #RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP_FWD capability flag set.
+ *
+ * The *nb_events* parameter is the number of event objects to enqueue that are supplied in the
+ * *ev* array of *rte_event* structure.
+ *
+ * The rte_event_dma_adapter_enqueue() function returns the number of event objects it actually
+ * enqueued. A return value equal to *nb_events* means that all event objects have been enqueued.
+ *
+ * @param evdev_id
+ * The identifier of the device.
+ * @param port_id
+ * The identifier of the event port.
+ * @param ev
+ * Points to an array of *nb_events* objects of type *rte_event* structure which contain the
+ * event object enqueue operations to be processed.
+ * @param nb_events
+ * The number of event objects to enqueue, typically number of
+ * rte_event_port_attr_get(...RTE_EVENT_PORT_ATTR_ENQ_DEPTH...) available for this port.
+ *
+ * @return
+ * The number of event objects actually enqueued on the event device. The return value can be
+ * less than the value of the *nb_events* parameter when the event devices queue is full or if
+ * invalid parameters are specified in a *rte_event*. If the return value is less than *nb_events*,
+ * the remaining events at the end of ev[] are not consumed and the caller has to take care of them,
+ * and rte_errno is set accordingly. Possible errno values include:
+ * - EINVAL: The port ID is invalid, device ID is invalid, an event's queue ID is invalid, or an
+ * event's sched type doesn't match the capabilities of the destination queue.
+ * - ENOSPC: The event port was backpressured and unable to enqueue one or more events. This
+ * error code is only applicable to closed systems.
+ */
+__rte_experimental
+uint16_t rte_event_dma_adapter_enqueue(uint8_t evdev_id, uint8_t port_id, struct rte_event ev[],
+ uint16_t nb_events);
+
+#ifdef __cplusplus
+}
+#endif
+
+#endif /* RTE_EVENT_DMA_ADAPTER */
diff --git a/lib/eventdev/rte_eventdev.h b/lib/eventdev/rte_eventdev.h
index 2ba8a7b090..41743f91b1 100644
--- a/lib/eventdev/rte_eventdev.h
+++ b/lib/eventdev/rte_eventdev.h
@@ -1197,6 +1197,8 @@ struct rte_event_vector {
*/
#define RTE_EVENT_TYPE_ETH_RX_ADAPTER 0x4
/**< The event generated from event eth Rx adapter */
+#define RTE_EVENT_TYPE_DMADEV 0x5
+/**< The event generated from dma subsystem */
#define RTE_EVENT_TYPE_VECTOR 0x8
/**< Indicates that event is a vector.
* All vector event types should be a logical OR of EVENT_TYPE_VECTOR.
@@ -1462,6 +1464,48 @@ int
rte_event_crypto_adapter_caps_get(uint8_t dev_id, uint8_t cdev_id,
uint32_t *caps);
+/* DMA adapter capability bitmap flag */
+#define RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP_NEW 0x1
+/**< Flag indicates HW is capable of generating events in
+ * RTE_EVENT_OP_NEW enqueue operation. DMADEV will send
+ * packets to the event device as new events using an
+ * internal event port.
+ */
+
+#define RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP_FWD 0x2
+/**< Flag indicates HW is capable of generating events in
+ * RTE_EVENT_OP_FORWARD enqueue operation. DMADEV will send
+ * packets to the event device as forwarded event using an
+ * internal event port.
+ */
+
+#define RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_VCHAN_EV_BIND 0x4
+/**< Flag indicates HW is capable of mapping DMA vchan to event queue. */
+
+/**
+ * Retrieve the event device's DMA adapter capabilities for the
+ * specified dmadev device
+ *
+ * @param dev_id
+ * The identifier of the device.
+ *
+ * @param dmadev_id
+ * The identifier of the dmadev device.
+ *
+ * @param[out] caps
+ * A pointer to memory filled with event adapter capabilities.
+ * It is expected to be pre-allocated & initialized by caller.
+ *
+ * @return
+ * - 0: Success, driver provides event adapter capabilities for the
+ * dmadev device.
+ * - <0: Error code returned by the driver function.
+ *
+ */
+__rte_experimental
+int
+rte_event_dma_adapter_caps_get(uint8_t dev_id, uint8_t dmadev_id, uint32_t *caps);
+
/* Ethdev Tx adapter capability bitmap flags */
#define RTE_EVENT_ETH_TX_ADAPTER_CAP_INTERNAL_PORT 0x1
/**< This flag is sent when the PMD supports a packet transmit callback
diff --git a/lib/eventdev/rte_eventdev_core.h b/lib/eventdev/rte_eventdev_core.h
index c27a52ccc0..83e8736c71 100644
--- a/lib/eventdev/rte_eventdev_core.h
+++ b/lib/eventdev/rte_eventdev_core.h
@@ -42,6 +42,10 @@ typedef uint16_t (*event_crypto_adapter_enqueue_t)(void *port,
uint16_t nb_events);
/**< @internal Enqueue burst of events on crypto adapter */
+typedef uint16_t (*event_dma_adapter_enqueue_t)(void *port, struct rte_event ev[],
+ uint16_t nb_events);
+/**< @internal Enqueue burst of events on DMA adapter */
+
struct rte_event_fp_ops {
void **data;
/**< points to array of internal port data pointers */
@@ -65,7 +69,9 @@ struct rte_event_fp_ops {
/**< PMD Tx adapter enqueue same destination function. */
event_crypto_adapter_enqueue_t ca_enqueue;
/**< PMD Crypto adapter enqueue function. */
- uintptr_t reserved[5];
+ event_dma_adapter_enqueue_t dma_enqueue;
+ /**< PMD DMA adapter enqueue function. */
+ uintptr_t reserved[4];
} __rte_cache_aligned;
extern struct rte_event_fp_ops rte_event_fp_ops[RTE_EVENT_MAX_DEVS];
diff --git a/lib/eventdev/version.map b/lib/eventdev/version.map
index 7ce09a87bb..b81eb2919c 100644
--- a/lib/eventdev/version.map
+++ b/lib/eventdev/version.map
@@ -133,6 +133,22 @@ EXPERIMENTAL {
rte_event_timer_remaining_ticks_get;
# added in 23.11
+ rte_event_dma_adapter_caps_get;
+ rte_event_dma_adapter_create;
+ rte_event_dma_adapter_create_ext;
+ rte_event_dma_adapter_enqueue;
+ rte_event_dma_adapter_event_port_get;
+ rte_event_dma_adapter_free;
+ rte_event_dma_adapter_runtime_params_get;
+ rte_event_dma_adapter_runtime_params_init;
+ rte_event_dma_adapter_runtime_params_set;
+ rte_event_dma_adapter_service_id_get;
+ rte_event_dma_adapter_start;
+ rte_event_dma_adapter_stats_get;
+ rte_event_dma_adapter_stats_reset;
+ rte_event_dma_adapter_stop;
+ rte_event_dma_adapter_vchan_add;
+ rte_event_dma_adapter_vchan_del;
rte_event_eth_rx_adapter_create_ext_with_params;
};
diff --git a/lib/meson.build b/lib/meson.build
index 53155be8e9..f3191f10b6 100644
--- a/lib/meson.build
+++ b/lib/meson.build
@@ -33,6 +33,7 @@ libraries = [
'compressdev',
'cryptodev',
'distributor',
+ 'dmadev',
'efd',
'eventdev',
'gpudev',
@@ -48,7 +49,6 @@ libraries = [
'rawdev',
'regexdev',
'mldev',
- 'dmadev',
'rib',
'reorder',
'sched',
--
2.25.1
^ permalink raw reply [flat|nested] 110+ messages in thread
* [PATCH v6 02/12] eventdev/dma: support adapter capabilities get
2023-09-28 16:49 ` [PATCH v6 00/12] event DMA adapter library support Amit Prakash Shukla
2023-09-28 16:49 ` [PATCH v6 01/12] eventdev/dma: introduce DMA adapter Amit Prakash Shukla
@ 2023-09-28 16:49 ` Amit Prakash Shukla
2023-09-28 16:49 ` [PATCH v6 03/12] eventdev/dma: support adapter create and free Amit Prakash Shukla
` (10 subsequent siblings)
12 siblings, 0 replies; 110+ messages in thread
From: Amit Prakash Shukla @ 2023-09-28 16:49 UTC (permalink / raw)
To: Jerin Jacob
Cc: dev, fengchengwen, kevin.laatz, bruce.richardson, conor.walsh,
vattunuru, g.singh, sachin.saxena, hemant.agrawal, cheng1.jiang,
ndabilpuram, anoobj, mb, Amit Prakash Shukla
Added a new eventdev API rte_event_dma_adapter_caps_get(), to get
DMA adapter capabilities supported by the driver.
Signed-off-by: Amit Prakash Shukla <amitprakashs@marvell.com>
---
lib/eventdev/meson.build | 2 +-
lib/eventdev/rte_eventdev.c | 23 +++++++++++++++++++++++
2 files changed, 24 insertions(+), 1 deletion(-)
diff --git a/lib/eventdev/meson.build b/lib/eventdev/meson.build
index 21347f7c4c..b46bbbc9aa 100644
--- a/lib/eventdev/meson.build
+++ b/lib/eventdev/meson.build
@@ -43,5 +43,5 @@ driver_sdk_headers += files(
'event_timer_adapter_pmd.h',
)
-deps += ['ring', 'ethdev', 'hash', 'mempool', 'mbuf', 'timer', 'cryptodev']
+deps += ['ring', 'ethdev', 'hash', 'mempool', 'mbuf', 'timer', 'cryptodev', 'dmadev']
deps += ['telemetry']
diff --git a/lib/eventdev/rte_eventdev.c b/lib/eventdev/rte_eventdev.c
index 6ab4524332..60509c6efb 100644
--- a/lib/eventdev/rte_eventdev.c
+++ b/lib/eventdev/rte_eventdev.c
@@ -20,6 +20,7 @@
#include <rte_errno.h>
#include <ethdev_driver.h>
#include <rte_cryptodev.h>
+#include <rte_dmadev.h>
#include <cryptodev_pmd.h>
#include <rte_telemetry.h>
@@ -224,6 +225,28 @@ rte_event_eth_tx_adapter_caps_get(uint8_t dev_id, uint16_t eth_port_id,
: 0;
}
+int
+rte_event_dma_adapter_caps_get(uint8_t dev_id, uint8_t dma_dev_id, uint32_t *caps)
+{
+ struct rte_eventdev *dev;
+
+ RTE_EVENTDEV_VALID_DEVID_OR_ERR_RET(dev_id, -EINVAL);
+ if (!rte_dma_is_valid(dma_dev_id))
+ return -EINVAL;
+
+ dev = &rte_eventdevs[dev_id];
+
+ if (caps == NULL)
+ return -EINVAL;
+
+ *caps = 0;
+
+ if (dev->dev_ops->dma_adapter_caps_get)
+ return (*dev->dev_ops->dma_adapter_caps_get)(dev, dma_dev_id, caps);
+
+ return 0;
+}
+
static inline int
event_dev_queue_config(struct rte_eventdev *dev, uint8_t nb_queues)
{
--
2.25.1
^ permalink raw reply [flat|nested] 110+ messages in thread
* [PATCH v6 03/12] eventdev/dma: support adapter create and free
2023-09-28 16:49 ` [PATCH v6 00/12] event DMA adapter library support Amit Prakash Shukla
2023-09-28 16:49 ` [PATCH v6 01/12] eventdev/dma: introduce DMA adapter Amit Prakash Shukla
2023-09-28 16:49 ` [PATCH v6 02/12] eventdev/dma: support adapter capabilities get Amit Prakash Shukla
@ 2023-09-28 16:49 ` Amit Prakash Shukla
2023-09-28 16:49 ` [PATCH v6 04/12] eventdev/dma: support vchan add and delete Amit Prakash Shukla
` (9 subsequent siblings)
12 siblings, 0 replies; 110+ messages in thread
From: Amit Prakash Shukla @ 2023-09-28 16:49 UTC (permalink / raw)
To: Bruce Richardson, Jerin Jacob, Amit Prakash Shukla
Cc: dev, fengchengwen, kevin.laatz, conor.walsh, vattunuru, g.singh,
sachin.saxena, hemant.agrawal, cheng1.jiang, ndabilpuram, anoobj,
mb
Added API support to create and free DMA adapter. Create function shall be
called with event device to be associated with the adapter and port
configuration to setup an event port.
Signed-off-by: Amit Prakash Shukla <amitprakashs@marvell.com>
---
config/rte_config.h | 1 +
lib/eventdev/meson.build | 1 +
lib/eventdev/rte_event_dma_adapter.c | 335 +++++++++++++++++++++++++++
3 files changed, 337 insertions(+)
create mode 100644 lib/eventdev/rte_event_dma_adapter.c
diff --git a/config/rte_config.h b/config/rte_config.h
index 400e44e3cf..401727703f 100644
--- a/config/rte_config.h
+++ b/config/rte_config.h
@@ -77,6 +77,7 @@
#define RTE_EVENT_ETH_INTR_RING_SIZE 1024
#define RTE_EVENT_CRYPTO_ADAPTER_MAX_INSTANCE 32
#define RTE_EVENT_ETH_TX_ADAPTER_MAX_INSTANCE 32
+#define RTE_EVENT_DMA_ADAPTER_MAX_INSTANCE 32
/* rawdev defines */
#define RTE_RAWDEV_MAX_DEVS 64
diff --git a/lib/eventdev/meson.build b/lib/eventdev/meson.build
index b46bbbc9aa..250abcb154 100644
--- a/lib/eventdev/meson.build
+++ b/lib/eventdev/meson.build
@@ -17,6 +17,7 @@ sources = files(
'eventdev_private.c',
'eventdev_trace_points.c',
'rte_event_crypto_adapter.c',
+ 'rte_event_dma_adapter.c',
'rte_event_eth_rx_adapter.c',
'rte_event_eth_tx_adapter.c',
'rte_event_ring.c',
diff --git a/lib/eventdev/rte_event_dma_adapter.c b/lib/eventdev/rte_event_dma_adapter.c
new file mode 100644
index 0000000000..e57d8407cb
--- /dev/null
+++ b/lib/eventdev/rte_event_dma_adapter.c
@@ -0,0 +1,335 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright (c) 2023 Marvell.
+ */
+
+#include <eventdev_pmd.h>
+
+#include "rte_event_dma_adapter.h"
+
+#define DMA_BATCH_SIZE 32
+#define DMA_DEFAULT_MAX_NB 128
+#define DMA_ADAPTER_NAME_LEN 32
+#define DMA_ADAPTER_BUFFER_SIZE 1024
+
+#define DMA_ADAPTER_OPS_BUFFER_SIZE (DMA_BATCH_SIZE + DMA_BATCH_SIZE)
+
+#define DMA_ADAPTER_ARRAY "event_dma_adapter_array"
+
+/* Macros to check for valid adapter */
+#define EVENT_DMA_ADAPTER_ID_VALID_OR_ERR_RET(id, retval) \
+ do { \
+ if (!edma_adapter_valid_id(id)) { \
+ RTE_EDEV_LOG_ERR("Invalid DMA adapter id = %d\n", id); \
+ return retval; \
+ } \
+ } while (0)
+
+/* DMA ops circular buffer */
+struct dma_ops_circular_buffer {
+ /* Index of head element */
+ uint16_t head;
+
+ /* Index of tail element */
+ uint16_t tail;
+
+ /* Number of elements in buffer */
+ uint16_t count;
+
+ /* Size of circular buffer */
+ uint16_t size;
+
+ /* Pointer to hold rte_event_dma_adapter_op for processing */
+ struct rte_event_dma_adapter_op **op_buffer;
+} __rte_cache_aligned;
+
+/* DMA device information */
+struct dma_device_info {
+ /* Number of vchans configured for a DMA device. */
+ uint16_t num_dma_dev_vchan;
+} __rte_cache_aligned;
+
+struct event_dma_adapter {
+ /* Event device identifier */
+ uint8_t eventdev_id;
+
+ /* Event port identifier */
+ uint8_t event_port_id;
+
+ /* Adapter mode */
+ enum rte_event_dma_adapter_mode mode;
+
+ /* Memory allocation name */
+ char mem_name[DMA_ADAPTER_NAME_LEN];
+
+ /* Socket identifier cached from eventdev */
+ int socket_id;
+
+ /* Lock to serialize config updates with service function */
+ rte_spinlock_t lock;
+
+ /* DMA device structure array */
+ struct dma_device_info *dma_devs;
+
+ /* Circular buffer for processing DMA ops to eventdev */
+ struct dma_ops_circular_buffer ebuf;
+
+ /* Configuration callback for rte_service configuration */
+ rte_event_dma_adapter_conf_cb conf_cb;
+
+ /* Configuration callback argument */
+ void *conf_arg;
+
+ /* Set if default_cb is being used */
+ int default_cb_arg;
+} __rte_cache_aligned;
+
+static struct event_dma_adapter **event_dma_adapter;
+
+static inline int
+edma_adapter_valid_id(uint8_t id)
+{
+ return id < RTE_EVENT_DMA_ADAPTER_MAX_INSTANCE;
+}
+
+static inline struct event_dma_adapter *
+edma_id_to_adapter(uint8_t id)
+{
+ return event_dma_adapter ? event_dma_adapter[id] : NULL;
+}
+
+static int
+edma_array_init(void)
+{
+ const struct rte_memzone *mz;
+ uint32_t sz;
+
+ mz = rte_memzone_lookup(DMA_ADAPTER_ARRAY);
+ if (mz == NULL) {
+ sz = sizeof(struct event_dma_adapter *) * RTE_EVENT_DMA_ADAPTER_MAX_INSTANCE;
+ sz = RTE_ALIGN(sz, RTE_CACHE_LINE_SIZE);
+
+ mz = rte_memzone_reserve_aligned(DMA_ADAPTER_ARRAY, sz, rte_socket_id(), 0,
+ RTE_CACHE_LINE_SIZE);
+ if (mz == NULL) {
+ RTE_EDEV_LOG_ERR("Failed to reserve memzone : %s, err = %d",
+ DMA_ADAPTER_ARRAY, rte_errno);
+ return -rte_errno;
+ }
+ }
+
+ event_dma_adapter = mz->addr;
+
+ return 0;
+}
+
+static inline int
+edma_circular_buffer_init(const char *name, struct dma_ops_circular_buffer *buf, uint16_t sz)
+{
+ buf->op_buffer = rte_zmalloc(name, sizeof(struct rte_event_dma_adapter_op *) * sz, 0);
+ if (buf->op_buffer == NULL)
+ return -ENOMEM;
+
+ buf->size = sz;
+
+ return 0;
+}
+
+static inline void
+edma_circular_buffer_free(struct dma_ops_circular_buffer *buf)
+{
+ rte_free(buf->op_buffer);
+}
+
+static int
+edma_default_config_cb(uint8_t id, uint8_t evdev_id, struct rte_event_dma_adapter_conf *conf,
+ void *arg)
+{
+ struct rte_event_port_conf *port_conf;
+ struct rte_event_dev_config dev_conf;
+ struct event_dma_adapter *adapter;
+ struct rte_eventdev *dev;
+ uint8_t port_id;
+ int started;
+ int ret;
+
+ adapter = edma_id_to_adapter(id);
+ if (adapter == NULL)
+ return -EINVAL;
+
+ dev = &rte_eventdevs[adapter->eventdev_id];
+ dev_conf = dev->data->dev_conf;
+
+ started = dev->data->dev_started;
+ if (started)
+ rte_event_dev_stop(evdev_id);
+
+ port_id = dev_conf.nb_event_ports;
+ dev_conf.nb_event_ports += 1;
+
+ port_conf = arg;
+ if (port_conf->event_port_cfg & RTE_EVENT_PORT_CFG_SINGLE_LINK)
+ dev_conf.nb_single_link_event_port_queues += 1;
+
+ ret = rte_event_dev_configure(evdev_id, &dev_conf);
+ if (ret) {
+ RTE_EDEV_LOG_ERR("Failed to configure event dev %u\n", evdev_id);
+ if (started) {
+ if (rte_event_dev_start(evdev_id))
+ return -EIO;
+ }
+ return ret;
+ }
+
+ ret = rte_event_port_setup(evdev_id, port_id, port_conf);
+ if (ret) {
+ RTE_EDEV_LOG_ERR("Failed to setup event port %u\n", port_id);
+ return ret;
+ }
+
+ conf->event_port_id = port_id;
+ conf->max_nb = DMA_DEFAULT_MAX_NB;
+ if (started)
+ ret = rte_event_dev_start(evdev_id);
+
+ adapter->default_cb_arg = 1;
+ adapter->event_port_id = conf->event_port_id;
+
+ return ret;
+}
+
+int
+rte_event_dma_adapter_create_ext(uint8_t id, uint8_t evdev_id,
+ rte_event_dma_adapter_conf_cb conf_cb,
+ enum rte_event_dma_adapter_mode mode, void *conf_arg)
+{
+ struct rte_event_dev_info dev_info;
+ struct event_dma_adapter *adapter;
+ char name[DMA_ADAPTER_NAME_LEN];
+ struct rte_dma_info info;
+ uint16_t num_dma_dev;
+ int socket_id;
+ uint8_t i;
+ int ret;
+
+ EVENT_DMA_ADAPTER_ID_VALID_OR_ERR_RET(id, -EINVAL);
+ RTE_EVENTDEV_VALID_DEVID_OR_ERR_RET(evdev_id, -EINVAL);
+
+ if (conf_cb == NULL)
+ return -EINVAL;
+
+ if (event_dma_adapter == NULL) {
+ ret = edma_array_init();
+ if (ret)
+ return ret;
+ }
+
+ adapter = edma_id_to_adapter(id);
+ if (adapter != NULL) {
+ RTE_EDEV_LOG_ERR("ML adapter ID %d already exists!", id);
+ return -EEXIST;
+ }
+
+ socket_id = rte_event_dev_socket_id(evdev_id);
+ snprintf(name, DMA_ADAPTER_NAME_LEN, "rte_event_dma_adapter_%d", id);
+ adapter = rte_zmalloc_socket(name, sizeof(struct event_dma_adapter), RTE_CACHE_LINE_SIZE,
+ socket_id);
+ if (adapter == NULL) {
+ RTE_EDEV_LOG_ERR("Failed to get mem for event ML adapter!");
+ return -ENOMEM;
+ }
+
+ if (edma_circular_buffer_init("edma_circular_buffer", &adapter->ebuf,
+ DMA_ADAPTER_BUFFER_SIZE)) {
+ RTE_EDEV_LOG_ERR("Failed to get memory for event adapter circular buffer");
+ rte_free(adapter);
+ return -ENOMEM;
+ }
+
+ ret = rte_event_dev_info_get(evdev_id, &dev_info);
+ if (ret < 0) {
+ RTE_EDEV_LOG_ERR("Failed to get info for eventdev %d: %s", evdev_id,
+ dev_info.driver_name);
+ edma_circular_buffer_free(&adapter->ebuf);
+ rte_free(adapter);
+ return ret;
+ }
+
+ num_dma_dev = rte_dma_count_avail();
+
+ adapter->eventdev_id = evdev_id;
+ adapter->mode = mode;
+ strcpy(adapter->mem_name, name);
+ adapter->socket_id = socket_id;
+ adapter->conf_cb = conf_cb;
+ adapter->conf_arg = conf_arg;
+ adapter->dma_devs = rte_zmalloc_socket(adapter->mem_name,
+ num_dma_dev * sizeof(struct dma_device_info), 0,
+ socket_id);
+ if (adapter->dma_devs == NULL) {
+ RTE_EDEV_LOG_ERR("Failed to get memory for DMA devices\n");
+ edma_circular_buffer_free(&adapter->ebuf);
+ rte_free(adapter);
+ return -ENOMEM;
+ }
+
+ rte_spinlock_init(&adapter->lock);
+ for (i = 0; i < num_dma_dev; i++) {
+ ret = rte_dma_info_get(i, &info);
+ if (ret) {
+ RTE_EDEV_LOG_ERR("Failed to get dma device info\n");
+ edma_circular_buffer_free(&adapter->ebuf);
+ rte_free(adapter);
+ return ret;
+ }
+
+ adapter->dma_devs[i].num_dma_dev_vchan = info.nb_vchans;
+ }
+
+ event_dma_adapter[id] = adapter;
+
+ return 0;
+}
+
+int
+rte_event_dma_adapter_create(uint8_t id, uint8_t evdev_id, struct rte_event_port_conf *port_config,
+ enum rte_event_dma_adapter_mode mode)
+{
+ struct rte_event_port_conf *pc;
+ int ret;
+
+ if (port_config == NULL)
+ return -EINVAL;
+
+ EVENT_DMA_ADAPTER_ID_VALID_OR_ERR_RET(id, -EINVAL);
+
+ pc = rte_malloc(NULL, sizeof(struct rte_event_port_conf), 0);
+ if (pc == NULL)
+ return -ENOMEM;
+
+ rte_memcpy(pc, port_config, sizeof(struct rte_event_port_conf));
+ ret = rte_event_dma_adapter_create_ext(id, evdev_id, edma_default_config_cb, mode, pc);
+ if (ret != 0)
+ rte_free(pc);
+
+ return ret;
+}
+
+int
+rte_event_dma_adapter_free(uint8_t id)
+{
+ struct event_dma_adapter *adapter;
+
+ EVENT_DMA_ADAPTER_ID_VALID_OR_ERR_RET(id, -EINVAL);
+
+ adapter = edma_id_to_adapter(id);
+ if (adapter == NULL)
+ return -EINVAL;
+
+ rte_free(adapter->conf_arg);
+ rte_free(adapter->dma_devs);
+ edma_circular_buffer_free(&adapter->ebuf);
+ rte_free(adapter);
+ event_dma_adapter[id] = NULL;
+
+ return 0;
+}
--
2.25.1
^ permalink raw reply [flat|nested] 110+ messages in thread
* [PATCH v6 04/12] eventdev/dma: support vchan add and delete
2023-09-28 16:49 ` [PATCH v6 00/12] event DMA adapter library support Amit Prakash Shukla
` (2 preceding siblings ...)
2023-09-28 16:49 ` [PATCH v6 03/12] eventdev/dma: support adapter create and free Amit Prakash Shukla
@ 2023-09-28 16:49 ` Amit Prakash Shukla
2023-09-28 16:49 ` [PATCH v6 05/12] eventdev/dma: support adapter service function Amit Prakash Shukla
` (8 subsequent siblings)
12 siblings, 0 replies; 110+ messages in thread
From: Amit Prakash Shukla @ 2023-09-28 16:49 UTC (permalink / raw)
To: Amit Prakash Shukla, Jerin Jacob
Cc: dev, fengchengwen, kevin.laatz, bruce.richardson, conor.walsh,
vattunuru, g.singh, sachin.saxena, hemant.agrawal, cheng1.jiang,
ndabilpuram, anoobj, mb
Added API support to add and delete vchan's from the DMA adapter. DMA devid
and vchan are added to the addapter instance by calling
rte_event_dma_adapter_vchan_add and deleted using
rte_event_dma_adapter_vchan_del.
Signed-off-by: Amit Prakash Shukla <amitprakashs@marvell.com>
---
lib/eventdev/rte_event_dma_adapter.c | 204 +++++++++++++++++++++++++++
1 file changed, 204 insertions(+)
diff --git a/lib/eventdev/rte_event_dma_adapter.c b/lib/eventdev/rte_event_dma_adapter.c
index e57d8407cb..ec81281bf8 100644
--- a/lib/eventdev/rte_event_dma_adapter.c
+++ b/lib/eventdev/rte_event_dma_adapter.c
@@ -42,8 +42,31 @@ struct dma_ops_circular_buffer {
struct rte_event_dma_adapter_op **op_buffer;
} __rte_cache_aligned;
+/* Vchan information */
+struct dma_vchan_info {
+ /* Set to indicate vchan queue is enabled */
+ bool vq_enabled;
+
+ /* Circular buffer for batching DMA ops to dma_dev */
+ struct dma_ops_circular_buffer dma_buf;
+} __rte_cache_aligned;
+
/* DMA device information */
struct dma_device_info {
+ /* Pointer to vchan queue info */
+ struct dma_vchan_info *vchanq;
+
+ /* Pointer to vchan queue info.
+ * This holds ops passed by application till the
+ * dma completion is done.
+ */
+ struct dma_vchan_info *tqmap;
+
+ /* If num_vchanq > 0, the start callback will
+ * be invoked if not already invoked
+ */
+ uint16_t num_vchanq;
+
/* Number of vchans configured for a DMA device. */
uint16_t num_dma_dev_vchan;
} __rte_cache_aligned;
@@ -81,6 +104,9 @@ struct event_dma_adapter {
/* Set if default_cb is being used */
int default_cb_arg;
+
+ /* No. of vchan queue configured */
+ uint16_t nb_vchanq;
} __rte_cache_aligned;
static struct event_dma_adapter **event_dma_adapter;
@@ -333,3 +359,181 @@ rte_event_dma_adapter_free(uint8_t id)
return 0;
}
+
+static void
+edma_update_vchanq_info(struct event_dma_adapter *adapter, struct dma_device_info *dev_info,
+ uint16_t vchan, uint8_t add)
+{
+ struct dma_vchan_info *vchan_info;
+ struct dma_vchan_info *tqmap_info;
+ int enabled;
+ uint16_t i;
+
+ if (dev_info->vchanq == NULL)
+ return;
+
+ if (vchan == RTE_DMA_ALL_VCHAN) {
+ for (i = 0; i < dev_info->num_dma_dev_vchan; i++)
+ edma_update_vchanq_info(adapter, dev_info, i, add);
+ } else {
+ tqmap_info = &dev_info->tqmap[vchan];
+ vchan_info = &dev_info->vchanq[vchan];
+ enabled = vchan_info->vq_enabled;
+ if (add) {
+ adapter->nb_vchanq += !enabled;
+ dev_info->num_vchanq += !enabled;
+ } else {
+ adapter->nb_vchanq -= enabled;
+ dev_info->num_vchanq -= enabled;
+ }
+ vchan_info->vq_enabled = !!add;
+ tqmap_info->vq_enabled = !!add;
+ }
+}
+
+int
+rte_event_dma_adapter_vchan_add(uint8_t id, int16_t dma_dev_id, uint16_t vchan,
+ const struct rte_event *event)
+{
+ struct event_dma_adapter *adapter;
+ struct dma_device_info *dev_info;
+ struct rte_eventdev *dev;
+ uint32_t cap;
+ int ret;
+
+ EVENT_DMA_ADAPTER_ID_VALID_OR_ERR_RET(id, -EINVAL);
+
+ if (!rte_dma_is_valid(dma_dev_id)) {
+ RTE_EDEV_LOG_ERR("Invalid dma_dev_id = %" PRIu8, dma_dev_id);
+ return -EINVAL;
+ }
+
+ adapter = edma_id_to_adapter(id);
+ if (adapter == NULL)
+ return -EINVAL;
+
+ dev = &rte_eventdevs[adapter->eventdev_id];
+ ret = rte_event_dma_adapter_caps_get(adapter->eventdev_id, dma_dev_id, &cap);
+ if (ret) {
+ RTE_EDEV_LOG_ERR("Failed to get adapter caps dev %u dma_dev %u", id, dma_dev_id);
+ return ret;
+ }
+
+ if ((cap & RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_VCHAN_EV_BIND) && (event == NULL)) {
+ RTE_EDEV_LOG_ERR("Event can not be NULL for dma_dev_id = %u", dma_dev_id);
+ return -EINVAL;
+ }
+
+ dev_info = &adapter->dma_devs[dma_dev_id];
+ if (vchan != RTE_DMA_ALL_VCHAN && vchan >= dev_info->num_dma_dev_vchan) {
+ RTE_EDEV_LOG_ERR("Invalid vhcan %u", vchan);
+ return -EINVAL;
+ }
+
+ /* In case HW cap is RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP_FWD, no
+ * need of service core as HW supports event forward capability.
+ */
+ if ((cap & RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP_FWD) ||
+ (cap & RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_VCHAN_EV_BIND &&
+ adapter->mode == RTE_EVENT_DMA_ADAPTER_OP_NEW) ||
+ (cap & RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP_NEW &&
+ adapter->mode == RTE_EVENT_DMA_ADAPTER_OP_NEW)) {
+ if (*dev->dev_ops->dma_adapter_vchan_add == NULL)
+ return -ENOTSUP;
+ if (dev_info->vchanq == NULL) {
+ dev_info->vchanq = rte_zmalloc_socket(adapter->mem_name,
+ dev_info->num_dma_dev_vchan *
+ sizeof(struct dma_vchan_info),
+ 0, adapter->socket_id);
+ if (dev_info->vchanq == NULL) {
+ printf("Queue pair add not supported\n");
+ return -ENOMEM;
+ }
+ }
+
+ if (dev_info->tqmap == NULL) {
+ dev_info->tqmap = rte_zmalloc_socket(adapter->mem_name,
+ dev_info->num_dma_dev_vchan *
+ sizeof(struct dma_vchan_info),
+ 0, adapter->socket_id);
+ if (dev_info->tqmap == NULL) {
+ printf("tq pair add not supported\n");
+ return -ENOMEM;
+ }
+ }
+
+ ret = (*dev->dev_ops->dma_adapter_vchan_add)(dev, dma_dev_id, vchan, event);
+ if (ret)
+ return ret;
+
+ else
+ edma_update_vchanq_info(adapter, &adapter->dma_devs[dma_dev_id], vchan, 1);
+ }
+
+ return 0;
+}
+
+int
+rte_event_dma_adapter_vchan_del(uint8_t id, int16_t dma_dev_id, uint16_t vchan)
+{
+ struct event_dma_adapter *adapter;
+ struct dma_device_info *dev_info;
+ struct rte_eventdev *dev;
+ uint32_t cap;
+ int ret;
+
+ EVENT_DMA_ADAPTER_ID_VALID_OR_ERR_RET(id, -EINVAL);
+
+ if (!rte_dma_is_valid(dma_dev_id)) {
+ RTE_EDEV_LOG_ERR("Invalid dma_dev_id = %" PRIu8, dma_dev_id);
+ return -EINVAL;
+ }
+
+ adapter = edma_id_to_adapter(id);
+ if (adapter == NULL)
+ return -EINVAL;
+
+ dev = &rte_eventdevs[adapter->eventdev_id];
+ ret = rte_event_dma_adapter_caps_get(adapter->eventdev_id, dma_dev_id, &cap);
+ if (ret)
+ return ret;
+
+ dev_info = &adapter->dma_devs[dma_dev_id];
+
+ if (vchan != RTE_DMA_ALL_VCHAN && vchan >= dev_info->num_dma_dev_vchan) {
+ RTE_EDEV_LOG_ERR("Invalid vhcan %" PRIu16, vchan);
+ return -EINVAL;
+ }
+
+ if ((cap & RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP_FWD) ||
+ (cap & RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP_NEW &&
+ adapter->mode == RTE_EVENT_DMA_ADAPTER_OP_NEW)) {
+ if (*dev->dev_ops->dma_adapter_vchan_del == NULL)
+ return -ENOTSUP;
+ ret = (*dev->dev_ops->dma_adapter_vchan_del)(dev, dma_dev_id, vchan);
+ if (ret == 0) {
+ edma_update_vchanq_info(adapter, dev_info, vchan, 0);
+ if (dev_info->num_vchanq == 0) {
+ rte_free(dev_info->vchanq);
+ dev_info->vchanq = NULL;
+ }
+ }
+ } else {
+ if (adapter->nb_vchanq == 0)
+ return 0;
+
+ rte_spinlock_lock(&adapter->lock);
+ edma_update_vchanq_info(adapter, dev_info, vchan, 0);
+
+ if (dev_info->num_vchanq == 0) {
+ rte_free(dev_info->vchanq);
+ rte_free(dev_info->tqmap);
+ dev_info->vchanq = NULL;
+ dev_info->tqmap = NULL;
+ }
+
+ rte_spinlock_unlock(&adapter->lock);
+ }
+
+ return ret;
+}
--
2.25.1
^ permalink raw reply [flat|nested] 110+ messages in thread
* [PATCH v6 05/12] eventdev/dma: support adapter service function
2023-09-28 16:49 ` [PATCH v6 00/12] event DMA adapter library support Amit Prakash Shukla
` (3 preceding siblings ...)
2023-09-28 16:49 ` [PATCH v6 04/12] eventdev/dma: support vchan add and delete Amit Prakash Shukla
@ 2023-09-28 16:49 ` Amit Prakash Shukla
2023-09-28 16:49 ` [PATCH v6 06/12] eventdev/dma: support adapter start and stop Amit Prakash Shukla
` (7 subsequent siblings)
12 siblings, 0 replies; 110+ messages in thread
From: Amit Prakash Shukla @ 2023-09-28 16:49 UTC (permalink / raw)
To: Amit Prakash Shukla, Jerin Jacob
Cc: dev, fengchengwen, kevin.laatz, bruce.richardson, conor.walsh,
vattunuru, g.singh, sachin.saxena, hemant.agrawal, cheng1.jiang,
ndabilpuram, anoobj, mb
Added support for DMA adapter service function for event devices.
Enqueue and dequeue of event from eventdev and DMA device are done
based on the adapter mode and the supported HW capabilities.
Signed-off-by: Amit Prakash Shukla <amitprakashs@marvell.com>
---
lib/eventdev/rte_event_dma_adapter.c | 588 +++++++++++++++++++++++++++
1 file changed, 588 insertions(+)
diff --git a/lib/eventdev/rte_event_dma_adapter.c b/lib/eventdev/rte_event_dma_adapter.c
index ec81281bf8..b0845eb415 100644
--- a/lib/eventdev/rte_event_dma_adapter.c
+++ b/lib/eventdev/rte_event_dma_adapter.c
@@ -3,6 +3,7 @@
*/
#include <eventdev_pmd.h>
+#include <rte_service_component.h>
#include "rte_event_dma_adapter.h"
@@ -69,6 +70,10 @@ struct dma_device_info {
/* Number of vchans configured for a DMA device. */
uint16_t num_dma_dev_vchan;
+
+ /* Next queue pair to be processed */
+ uint16_t next_vchan_id;
+
} __rte_cache_aligned;
struct event_dma_adapter {
@@ -90,6 +95,9 @@ struct event_dma_adapter {
/* Lock to serialize config updates with service function */
rte_spinlock_t lock;
+ /* Next dma device to be processed */
+ uint16_t next_dmadev_id;
+
/* DMA device structure array */
struct dma_device_info *dma_devs;
@@ -107,6 +115,26 @@ struct event_dma_adapter {
/* No. of vchan queue configured */
uint16_t nb_vchanq;
+
+ /* Per adapter EAL service ID */
+ uint32_t service_id;
+
+ /* Service initialization state */
+ uint8_t service_initialized;
+
+ /* Max DMA ops processed in any service function invocation */
+ uint32_t max_nb;
+
+ /* Store event port's implicit release capability */
+ uint8_t implicit_release_disabled;
+
+ /* Flag to indicate backpressure at dma_dev
+ * Stop further dequeuing events from eventdev
+ */
+ bool stop_enq_to_dma_dev;
+
+ /* Loop counter to flush dma ops */
+ uint16_t transmit_loop_count;
} __rte_cache_aligned;
static struct event_dma_adapter **event_dma_adapter;
@@ -148,6 +176,18 @@ edma_array_init(void)
return 0;
}
+static inline bool
+edma_circular_buffer_batch_ready(struct dma_ops_circular_buffer *bufp)
+{
+ return bufp->count >= DMA_BATCH_SIZE;
+}
+
+static inline bool
+edma_circular_buffer_space_for_batch(struct dma_ops_circular_buffer *bufp)
+{
+ return (bufp->size - bufp->count) >= DMA_BATCH_SIZE;
+}
+
static inline int
edma_circular_buffer_init(const char *name, struct dma_ops_circular_buffer *buf, uint16_t sz)
{
@@ -166,6 +206,67 @@ edma_circular_buffer_free(struct dma_ops_circular_buffer *buf)
rte_free(buf->op_buffer);
}
+static inline int
+edma_circular_buffer_add(struct dma_ops_circular_buffer *bufp, struct rte_event_dma_adapter_op *op)
+{
+ uint16_t *tail = &bufp->tail;
+
+ bufp->op_buffer[*tail] = op;
+
+ /* circular buffer, go round */
+ *tail = (*tail + 1) % bufp->size;
+ bufp->count++;
+
+ return 0;
+}
+
+static inline int
+edma_circular_buffer_flush_to_dma_dev(struct event_dma_adapter *adapter,
+ struct dma_ops_circular_buffer *bufp, uint8_t dma_dev_id,
+ uint16_t vchan, uint16_t *nb_ops_flushed)
+{
+ struct rte_event_dma_adapter_op *op;
+ struct dma_vchan_info *tq;
+ uint16_t *head = &bufp->head;
+ uint16_t *tail = &bufp->tail;
+ uint16_t n;
+ uint16_t i;
+ int ret;
+
+ if (*tail > *head)
+ n = *tail - *head;
+ else if (*tail < *head)
+ n = bufp->size - *head;
+ else {
+ *nb_ops_flushed = 0;
+ return 0; /* buffer empty */
+ }
+
+ tq = &adapter->dma_devs[dma_dev_id].tqmap[vchan];
+
+ for (i = 0; i < n; i++) {
+ op = bufp->op_buffer[*head];
+ ret = rte_dma_copy_sg(dma_dev_id, vchan, op->src_seg, op->dst_seg,
+ op->nb_src, op->nb_dst, op->flags);
+ if (ret < 0)
+ break;
+
+ /* Enqueue in transaction queue. */
+ edma_circular_buffer_add(&tq->dma_buf, op);
+
+ *head = (*head + 1) % bufp->size;
+ }
+
+ *nb_ops_flushed = i;
+ bufp->count -= *nb_ops_flushed;
+ if (!bufp->count) {
+ *head = 0;
+ *tail = 0;
+ }
+
+ return *nb_ops_flushed == n ? 0 : -1;
+}
+
static int
edma_default_config_cb(uint8_t id, uint8_t evdev_id, struct rte_event_dma_adapter_conf *conf,
void *arg)
@@ -360,6 +461,406 @@ rte_event_dma_adapter_free(uint8_t id)
return 0;
}
+static inline unsigned int
+edma_enq_to_dma_dev(struct event_dma_adapter *adapter, struct rte_event *ev, unsigned int cnt)
+{
+ struct dma_vchan_info *vchan_qinfo = NULL;
+ struct rte_event_dma_adapter_op *dma_op;
+ uint16_t vchan, nb_enqueued = 0;
+ int16_t dma_dev_id;
+ unsigned int i, n;
+ int ret;
+
+ ret = 0;
+ n = 0;
+
+ for (i = 0; i < cnt; i++) {
+ dma_op = ev[i].event_ptr;
+ if (dma_op == NULL)
+ continue;
+
+ /* Expected to have response info appended to dma_op. */
+
+ dma_dev_id = dma_op->dma_dev_id;
+ vchan = dma_op->vchan;
+ vchan_qinfo = &adapter->dma_devs[dma_dev_id].vchanq[vchan];
+ if (!vchan_qinfo->vq_enabled) {
+ if (dma_op != NULL && dma_op->op_mp != NULL)
+ rte_mempool_put(dma_op->op_mp, dma_op);
+ continue;
+ }
+ edma_circular_buffer_add(&vchan_qinfo->dma_buf, dma_op);
+
+ if (edma_circular_buffer_batch_ready(&vchan_qinfo->dma_buf)) {
+ ret = edma_circular_buffer_flush_to_dma_dev(adapter, &vchan_qinfo->dma_buf,
+ dma_dev_id, vchan,
+ &nb_enqueued);
+ n += nb_enqueued;
+
+ /**
+ * If some dma ops failed to flush to dma_dev and
+ * space for another batch is not available, stop
+ * dequeue from eventdev momentarily
+ */
+ if (unlikely(ret < 0 &&
+ !edma_circular_buffer_space_for_batch(&vchan_qinfo->dma_buf)))
+ adapter->stop_enq_to_dma_dev = true;
+ }
+ }
+
+ return n;
+}
+
+static unsigned int
+edma_adapter_dev_flush(struct event_dma_adapter *adapter, int16_t dma_dev_id,
+ uint16_t *nb_ops_flushed)
+{
+ struct dma_vchan_info *vchan_info;
+ struct dma_device_info *dev_info;
+ uint16_t nb = 0, nb_enqueued = 0;
+ uint16_t vchan, nb_vchans;
+
+ dev_info = &adapter->dma_devs[dma_dev_id];
+ nb_vchans = dev_info->num_vchanq;
+
+ for (vchan = 0; vchan < nb_vchans; vchan++) {
+
+ vchan_info = &dev_info->vchanq[vchan];
+ if (unlikely(vchan_info == NULL || !vchan_info->vq_enabled))
+ continue;
+
+ edma_circular_buffer_flush_to_dma_dev(adapter, &vchan_info->dma_buf, dma_dev_id,
+ vchan, &nb_enqueued);
+ *nb_ops_flushed += vchan_info->dma_buf.count;
+ nb += nb_enqueued;
+ }
+
+ return nb;
+}
+
+static unsigned int
+edma_adapter_enq_flush(struct event_dma_adapter *adapter)
+{
+ int16_t dma_dev_id;
+ uint16_t nb_enqueued = 0;
+ uint16_t nb_ops_flushed = 0;
+ uint16_t num_dma_dev = rte_dma_count_avail();
+
+ for (dma_dev_id = 0; dma_dev_id < num_dma_dev; dma_dev_id++)
+ nb_enqueued += edma_adapter_dev_flush(adapter, dma_dev_id, &nb_ops_flushed);
+ /**
+ * Enable dequeue from eventdev if all ops from circular
+ * buffer flushed to dma_dev
+ */
+ if (!nb_ops_flushed)
+ adapter->stop_enq_to_dma_dev = false;
+
+ return nb_enqueued;
+}
+
+/* Flush an instance's enqueue buffers every DMA_ENQ_FLUSH_THRESHOLD
+ * iterations of edma_adapter_enq_run()
+ */
+#define DMA_ENQ_FLUSH_THRESHOLD 1024
+
+static int
+edma_adapter_enq_run(struct event_dma_adapter *adapter, unsigned int max_enq)
+{
+ uint8_t event_port_id = adapter->event_port_id;
+ uint8_t event_dev_id = adapter->eventdev_id;
+ struct rte_event ev[DMA_BATCH_SIZE];
+ unsigned int nb_enq, nb_enqueued;
+ uint16_t n;
+
+ if (adapter->mode == RTE_EVENT_DMA_ADAPTER_OP_NEW)
+ return 0;
+
+ nb_enqueued = 0;
+ for (nb_enq = 0; nb_enq < max_enq; nb_enq += n) {
+
+ if (unlikely(adapter->stop_enq_to_dma_dev)) {
+ nb_enqueued += edma_adapter_enq_flush(adapter);
+
+ if (unlikely(adapter->stop_enq_to_dma_dev))
+ break;
+ }
+
+ n = rte_event_dequeue_burst(event_dev_id, event_port_id, ev, DMA_BATCH_SIZE, 0);
+
+ if (!n)
+ break;
+
+ nb_enqueued += edma_enq_to_dma_dev(adapter, ev, n);
+ }
+
+ if ((++adapter->transmit_loop_count & (DMA_ENQ_FLUSH_THRESHOLD - 1)) == 0)
+ nb_enqueued += edma_adapter_enq_flush(adapter);
+
+ return nb_enqueued;
+}
+
+#define DMA_ADAPTER_MAX_EV_ENQ_RETRIES 100
+
+static inline uint16_t
+edma_ops_enqueue_burst(struct event_dma_adapter *adapter, struct rte_event_dma_adapter_op **ops,
+ uint16_t num)
+{
+ uint8_t event_port_id = adapter->event_port_id;
+ uint8_t event_dev_id = adapter->eventdev_id;
+ struct rte_event events[DMA_BATCH_SIZE];
+ struct rte_event *response_info;
+ uint16_t nb_enqueued, nb_ev;
+ uint8_t retry;
+ uint8_t i;
+
+ nb_ev = 0;
+ retry = 0;
+ nb_enqueued = 0;
+ num = RTE_MIN(num, DMA_BATCH_SIZE);
+ for (i = 0; i < num; i++) {
+ struct rte_event *ev = &events[nb_ev++];
+
+ /* Expected to have response info appended to dma_op. */
+ response_info = (struct rte_event *)((uint8_t *)ops[i] +
+ sizeof(struct rte_event_dma_adapter_op));
+ if (unlikely(response_info == NULL)) {
+ if (ops[i] != NULL && ops[i]->op_mp != NULL)
+ rte_mempool_put(ops[i]->op_mp, ops[i]);
+ continue;
+ }
+
+ rte_memcpy(ev, response_info, sizeof(struct rte_event));
+ ev->event_ptr = ops[i];
+ ev->event_type = RTE_EVENT_TYPE_DMADEV;
+ if (adapter->implicit_release_disabled)
+ ev->op = RTE_EVENT_OP_FORWARD;
+ else
+ ev->op = RTE_EVENT_OP_NEW;
+ }
+
+ do {
+ nb_enqueued += rte_event_enqueue_burst(event_dev_id, event_port_id,
+ &events[nb_enqueued], nb_ev - nb_enqueued);
+
+ } while (retry++ < DMA_ADAPTER_MAX_EV_ENQ_RETRIES && nb_enqueued < nb_ev);
+
+ return nb_enqueued;
+}
+
+static int
+edma_circular_buffer_flush_to_evdev(struct event_dma_adapter *adapter,
+ struct dma_ops_circular_buffer *bufp,
+ uint16_t *enqueue_count)
+{
+ struct rte_event_dma_adapter_op **ops = bufp->op_buffer;
+ uint16_t n = 0, nb_ops_flushed;
+ uint16_t *head = &bufp->head;
+ uint16_t *tail = &bufp->tail;
+
+ if (*tail > *head)
+ n = *tail - *head;
+ else if (*tail < *head)
+ n = bufp->size - *head;
+ else {
+ if (enqueue_count)
+ *enqueue_count = 0;
+ return 0; /* buffer empty */
+ }
+
+ if (enqueue_count && n > *enqueue_count)
+ n = *enqueue_count;
+
+ nb_ops_flushed = edma_ops_enqueue_burst(adapter, &ops[*head], n);
+ if (enqueue_count)
+ *enqueue_count = nb_ops_flushed;
+
+ bufp->count -= nb_ops_flushed;
+ if (!bufp->count) {
+ *head = 0;
+ *tail = 0;
+ return 0; /* buffer empty */
+ }
+
+ *head = (*head + nb_ops_flushed) % bufp->size;
+ return 1;
+}
+
+static void
+edma_ops_buffer_flush(struct event_dma_adapter *adapter)
+{
+ if (likely(adapter->ebuf.count == 0))
+ return;
+
+ while (edma_circular_buffer_flush_to_evdev(adapter, &adapter->ebuf, NULL))
+ ;
+}
+
+static inline unsigned int
+edma_adapter_deq_run(struct event_dma_adapter *adapter, unsigned int max_deq)
+{
+ struct dma_vchan_info *vchan_info;
+ struct dma_ops_circular_buffer *tq_buf;
+ struct rte_event_dma_adapter_op *ops;
+ uint16_t n, nb_deq, nb_enqueued, i;
+ struct dma_device_info *dev_info;
+ uint16_t vchan, num_vchan;
+ uint16_t num_dma_dev;
+ int16_t dma_dev_id;
+ uint16_t index;
+ bool done;
+ bool err;
+
+ nb_deq = 0;
+ edma_ops_buffer_flush(adapter);
+
+ num_dma_dev = rte_dma_count_avail();
+ do {
+ done = true;
+
+ for (dma_dev_id = adapter->next_dmadev_id; dma_dev_id < num_dma_dev; dma_dev_id++) {
+ uint16_t queues = 0;
+ dev_info = &adapter->dma_devs[dma_dev_id];
+ num_vchan = dev_info->num_vchanq;
+
+ for (vchan = dev_info->next_vchan_id; queues < num_vchan;
+ vchan = (vchan + 1) % num_vchan, queues++) {
+
+ vchan_info = &dev_info->vchanq[vchan];
+ if (unlikely(vchan_info == NULL || !vchan_info->vq_enabled))
+ continue;
+
+ n = rte_dma_completed(dma_dev_id, vchan, DMA_BATCH_SIZE,
+ &index, &err);
+ if (!n)
+ continue;
+
+ done = false;
+
+ tq_buf = &dev_info->tqmap[vchan].dma_buf;
+
+ nb_enqueued = n;
+ if (unlikely(!adapter->ebuf.count))
+ edma_circular_buffer_flush_to_evdev(adapter, tq_buf,
+ &nb_enqueued);
+
+ if (likely(nb_enqueued == n))
+ goto check;
+
+ /* Failed to enqueue events case */
+ for (i = nb_enqueued; i < n; i++) {
+ ops = tq_buf->op_buffer[tq_buf->head];
+ edma_circular_buffer_add(&adapter->ebuf, ops);
+ tq_buf->head = (tq_buf->head + 1) % tq_buf->size;
+ }
+
+check:
+ nb_deq += n;
+ if (nb_deq >= max_deq) {
+ if ((vchan + 1) == num_vchan)
+ adapter->next_dmadev_id =
+ (dma_dev_id + 1) % num_dma_dev;
+
+ dev_info->next_vchan_id = (vchan + 1) % num_vchan;
+
+ return nb_deq;
+ }
+ }
+ }
+ adapter->next_dmadev_id = 0;
+
+ } while (done == false);
+
+ return nb_deq;
+}
+
+static int
+edma_adapter_run(struct event_dma_adapter *adapter, unsigned int max_ops)
+{
+ unsigned int ops_left = max_ops;
+
+ while (ops_left > 0) {
+ unsigned int e_cnt, d_cnt;
+
+ e_cnt = edma_adapter_deq_run(adapter, ops_left);
+ ops_left -= RTE_MIN(ops_left, e_cnt);
+
+ d_cnt = edma_adapter_enq_run(adapter, ops_left);
+ ops_left -= RTE_MIN(ops_left, d_cnt);
+
+ if (e_cnt == 0 && d_cnt == 0)
+ break;
+ }
+
+ if (ops_left == max_ops) {
+ rte_event_maintain(adapter->eventdev_id, adapter->event_port_id, 0);
+ return -EAGAIN;
+ } else
+ return 0;
+}
+
+static int
+edma_service_func(void *args)
+{
+ struct event_dma_adapter *adapter = args;
+ int ret;
+
+ if (rte_spinlock_trylock(&adapter->lock) == 0)
+ return 0;
+ ret = edma_adapter_run(adapter, adapter->max_nb);
+ rte_spinlock_unlock(&adapter->lock);
+
+ return ret;
+}
+
+static int
+edma_init_service(struct event_dma_adapter *adapter, uint8_t id)
+{
+ struct rte_event_dma_adapter_conf adapter_conf;
+ struct rte_service_spec service;
+ uint32_t impl_rel;
+ int ret;
+
+ if (adapter->service_initialized)
+ return 0;
+
+ memset(&service, 0, sizeof(service));
+ snprintf(service.name, DMA_ADAPTER_NAME_LEN, "rte_event_dma_adapter_%d", id);
+ service.socket_id = adapter->socket_id;
+ service.callback = edma_service_func;
+ service.callback_userdata = adapter;
+
+ /* Service function handles locking for queue add/del updates */
+ service.capabilities = RTE_SERVICE_CAP_MT_SAFE;
+ ret = rte_service_component_register(&service, &adapter->service_id);
+ if (ret) {
+ RTE_EDEV_LOG_ERR("failed to register service %s err = %" PRId32, service.name, ret);
+ return ret;
+ }
+
+ ret = adapter->conf_cb(id, adapter->eventdev_id, &adapter_conf, adapter->conf_arg);
+ if (ret) {
+ RTE_EDEV_LOG_ERR("configuration callback failed err = %" PRId32, ret);
+ return ret;
+ }
+
+ adapter->max_nb = adapter_conf.max_nb;
+ adapter->event_port_id = adapter_conf.event_port_id;
+
+ if (rte_event_port_attr_get(adapter->eventdev_id, adapter->event_port_id,
+ RTE_EVENT_PORT_ATTR_IMPLICIT_RELEASE_DISABLE, &impl_rel)) {
+ RTE_EDEV_LOG_ERR("Failed to get port info for eventdev %" PRId32,
+ adapter->eventdev_id);
+ edma_circular_buffer_free(&adapter->ebuf);
+ rte_free(adapter);
+ return -EINVAL;
+ }
+
+ adapter->implicit_release_disabled = (uint8_t)impl_rel;
+ adapter->service_initialized = 1;
+
+ return ret;
+}
+
static void
edma_update_vchanq_info(struct event_dma_adapter *adapter, struct dma_device_info *dev_info,
uint16_t vchan, uint8_t add)
@@ -391,6 +892,60 @@ edma_update_vchanq_info(struct event_dma_adapter *adapter, struct dma_device_inf
}
}
+static int
+edma_add_vchan(struct event_dma_adapter *adapter, int16_t dma_dev_id, uint16_t vchan)
+{
+ struct dma_device_info *dev_info = &adapter->dma_devs[dma_dev_id];
+ struct dma_vchan_info *vchanq;
+ struct dma_vchan_info *tqmap;
+ uint16_t nb_vchans;
+ uint32_t i;
+
+ if (dev_info->vchanq == NULL) {
+ nb_vchans = dev_info->num_dma_dev_vchan;
+
+ dev_info->vchanq = rte_zmalloc_socket(adapter->mem_name,
+ nb_vchans * sizeof(struct dma_vchan_info),
+ 0, adapter->socket_id);
+ if (dev_info->vchanq == NULL)
+ return -ENOMEM;
+
+ dev_info->tqmap = rte_zmalloc_socket(adapter->mem_name,
+ nb_vchans * sizeof(struct dma_vchan_info),
+ 0, adapter->socket_id);
+ if (dev_info->tqmap == NULL)
+ return -ENOMEM;
+
+ for (i = 0; i < nb_vchans; i++) {
+ vchanq = &dev_info->vchanq[i];
+
+ if (edma_circular_buffer_init("dma_dev_circular_buffer", &vchanq->dma_buf,
+ DMA_ADAPTER_OPS_BUFFER_SIZE)) {
+ RTE_EDEV_LOG_ERR("Failed to get memory for dma_dev buffer");
+ rte_free(vchanq);
+ return -ENOMEM;
+ }
+
+ tqmap = &dev_info->tqmap[i];
+ if (edma_circular_buffer_init("dma_dev_circular_trans_buf", &tqmap->dma_buf,
+ DMA_ADAPTER_OPS_BUFFER_SIZE)) {
+ RTE_EDEV_LOG_ERR(
+ "Failed to get memory for dma_dev transaction buffer");
+ rte_free(tqmap);
+ return -ENOMEM;
+ }
+ }
+ }
+
+ if (vchan == RTE_DMA_ALL_VCHAN) {
+ for (i = 0; i < dev_info->num_dma_dev_vchan; i++)
+ edma_update_vchanq_info(adapter, dev_info, i, 1);
+ } else
+ edma_update_vchanq_info(adapter, dev_info, vchan, 1);
+
+ return 0;
+}
+
int
rte_event_dma_adapter_vchan_add(uint8_t id, int16_t dma_dev_id, uint16_t vchan,
const struct rte_event *event)
@@ -470,6 +1025,38 @@ rte_event_dma_adapter_vchan_add(uint8_t id, int16_t dma_dev_id, uint16_t vchan,
edma_update_vchanq_info(adapter, &adapter->dma_devs[dma_dev_id], vchan, 1);
}
+ /* In case HW cap is RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP_NEW, or SW adapter, initiate
+ * services so the application can choose which ever way it wants to use the adapter.
+ *
+ * Case 1: RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP_NEW. Application may wants to use one
+ * of below two modes
+ *
+ * a. OP_FORWARD mode -> HW Dequeue + SW enqueue
+ * b. OP_NEW mode -> HW Dequeue
+ *
+ * Case 2: No HW caps, use SW adapter
+ *
+ * a. OP_FORWARD mode -> SW enqueue & dequeue
+ * b. OP_NEW mode -> SW Dequeue
+ */
+ if ((cap & RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP_NEW &&
+ !(cap & RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP_FWD) &&
+ adapter->mode == RTE_EVENT_DMA_ADAPTER_OP_FORWARD) ||
+ (!(cap & RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP_NEW) &&
+ !(cap & RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP_FWD) &&
+ !(cap & RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_VCHAN_EV_BIND))) {
+ rte_spinlock_lock(&adapter->lock);
+ ret = edma_init_service(adapter, id);
+ if (ret == 0)
+ ret = edma_add_vchan(adapter, dma_dev_id, vchan);
+ rte_spinlock_unlock(&adapter->lock);
+
+ if (ret)
+ return ret;
+
+ rte_service_component_runstate_set(adapter->service_id, 1);
+ }
+
return 0;
}
@@ -533,6 +1120,7 @@ rte_event_dma_adapter_vchan_del(uint8_t id, int16_t dma_dev_id, uint16_t vchan)
}
rte_spinlock_unlock(&adapter->lock);
+ rte_service_component_runstate_set(adapter->service_id, adapter->nb_vchanq);
}
return ret;
--
2.25.1
^ permalink raw reply [flat|nested] 110+ messages in thread
* [PATCH v6 06/12] eventdev/dma: support adapter start and stop
2023-09-28 16:49 ` [PATCH v6 00/12] event DMA adapter library support Amit Prakash Shukla
` (4 preceding siblings ...)
2023-09-28 16:49 ` [PATCH v6 05/12] eventdev/dma: support adapter service function Amit Prakash Shukla
@ 2023-09-28 16:49 ` Amit Prakash Shukla
2023-09-28 16:49 ` [PATCH v6 07/12] eventdev/dma: support adapter service ID get Amit Prakash Shukla
` (6 subsequent siblings)
12 siblings, 0 replies; 110+ messages in thread
From: Amit Prakash Shukla @ 2023-09-28 16:49 UTC (permalink / raw)
To: Amit Prakash Shukla, Jerin Jacob
Cc: dev, fengchengwen, kevin.laatz, bruce.richardson, conor.walsh,
vattunuru, g.singh, sachin.saxena, hemant.agrawal, cheng1.jiang,
ndabilpuram, anoobj, mb
Added API support to start and stop DMA adapter.
Signed-off-by: Amit Prakash Shukla <amitprakashs@marvell.com>
---
lib/eventdev/rte_event_dma_adapter.c | 69 ++++++++++++++++++++++++++++
1 file changed, 69 insertions(+)
diff --git a/lib/eventdev/rte_event_dma_adapter.c b/lib/eventdev/rte_event_dma_adapter.c
index b0845eb415..e955f19c68 100644
--- a/lib/eventdev/rte_event_dma_adapter.c
+++ b/lib/eventdev/rte_event_dma_adapter.c
@@ -74,6 +74,13 @@ struct dma_device_info {
/* Next queue pair to be processed */
uint16_t next_vchan_id;
+ /* Set to indicate processing has been started */
+ uint8_t dev_started;
+
+ /* Set to indicate dmadev->eventdev packet
+ * transfer uses a hardware mechanism
+ */
+ uint8_t internal_event_port;
} __rte_cache_aligned;
struct event_dma_adapter {
@@ -1125,3 +1132,65 @@ rte_event_dma_adapter_vchan_del(uint8_t id, int16_t dma_dev_id, uint16_t vchan)
return ret;
}
+
+static int
+edma_adapter_ctrl(uint8_t id, int start)
+{
+ struct event_dma_adapter *adapter;
+ struct dma_device_info *dev_info;
+ struct rte_eventdev *dev;
+ uint16_t num_dma_dev;
+ int stop = !start;
+ int use_service;
+ uint32_t i;
+
+ use_service = 0;
+ EVENT_DMA_ADAPTER_ID_VALID_OR_ERR_RET(id, -EINVAL);
+ adapter = edma_id_to_adapter(id);
+ if (adapter == NULL)
+ return -EINVAL;
+
+ num_dma_dev = rte_dma_count_avail();
+ dev = &rte_eventdevs[adapter->eventdev_id];
+
+ for (i = 0; i < num_dma_dev; i++) {
+ dev_info = &adapter->dma_devs[i];
+ /* start check for num queue pairs */
+ if (start && !dev_info->num_vchanq)
+ continue;
+ /* stop check if dev has been started */
+ if (stop && !dev_info->dev_started)
+ continue;
+ use_service |= !dev_info->internal_event_port;
+ dev_info->dev_started = start;
+ if (dev_info->internal_event_port == 0)
+ continue;
+ start ? (*dev->dev_ops->dma_adapter_start)(dev, i) :
+ (*dev->dev_ops->dma_adapter_stop)(dev, i);
+ }
+
+ if (use_service)
+ rte_service_runstate_set(adapter->service_id, start);
+
+ return 0;
+}
+
+int
+rte_event_dma_adapter_start(uint8_t id)
+{
+ struct event_dma_adapter *adapter;
+
+ EVENT_DMA_ADAPTER_ID_VALID_OR_ERR_RET(id, -EINVAL);
+
+ adapter = edma_id_to_adapter(id);
+ if (adapter == NULL)
+ return -EINVAL;
+
+ return edma_adapter_ctrl(id, 1);
+}
+
+int
+rte_event_dma_adapter_stop(uint8_t id)
+{
+ return edma_adapter_ctrl(id, 0);
+}
--
2.25.1
^ permalink raw reply [flat|nested] 110+ messages in thread
* [PATCH v6 07/12] eventdev/dma: support adapter service ID get
2023-09-28 16:49 ` [PATCH v6 00/12] event DMA adapter library support Amit Prakash Shukla
` (5 preceding siblings ...)
2023-09-28 16:49 ` [PATCH v6 06/12] eventdev/dma: support adapter start and stop Amit Prakash Shukla
@ 2023-09-28 16:49 ` Amit Prakash Shukla
2023-09-28 16:49 ` [PATCH v6 08/12] eventdev/dma: support adapter runtime params Amit Prakash Shukla
` (5 subsequent siblings)
12 siblings, 0 replies; 110+ messages in thread
From: Amit Prakash Shukla @ 2023-09-28 16:49 UTC (permalink / raw)
To: Amit Prakash Shukla, Jerin Jacob
Cc: dev, fengchengwen, kevin.laatz, bruce.richardson, conor.walsh,
vattunuru, g.singh, sachin.saxena, hemant.agrawal, cheng1.jiang,
ndabilpuram, anoobj, mb
Added API support to get DMA adapter service ID. Service id
returned in the variable by the API call shall be used by application
to map a service core.
Signed-off-by: Amit Prakash Shukla <amitprakashs@marvell.com>
---
lib/eventdev/rte_event_dma_adapter.c | 17 +++++++++++++++++
1 file changed, 17 insertions(+)
diff --git a/lib/eventdev/rte_event_dma_adapter.c b/lib/eventdev/rte_event_dma_adapter.c
index e955f19c68..63b07cd14e 100644
--- a/lib/eventdev/rte_event_dma_adapter.c
+++ b/lib/eventdev/rte_event_dma_adapter.c
@@ -1133,6 +1133,23 @@ rte_event_dma_adapter_vchan_del(uint8_t id, int16_t dma_dev_id, uint16_t vchan)
return ret;
}
+int
+rte_event_dma_adapter_service_id_get(uint8_t id, uint32_t *service_id)
+{
+ struct event_dma_adapter *adapter;
+
+ EVENT_DMA_ADAPTER_ID_VALID_OR_ERR_RET(id, -EINVAL);
+
+ adapter = edma_id_to_adapter(id);
+ if (adapter == NULL || service_id == NULL)
+ return -EINVAL;
+
+ if (adapter->service_initialized)
+ *service_id = adapter->service_id;
+
+ return adapter->service_initialized ? 0 : -ESRCH;
+}
+
static int
edma_adapter_ctrl(uint8_t id, int start)
{
--
2.25.1
^ permalink raw reply [flat|nested] 110+ messages in thread
* [PATCH v6 08/12] eventdev/dma: support adapter runtime params
2023-09-28 16:49 ` [PATCH v6 00/12] event DMA adapter library support Amit Prakash Shukla
` (6 preceding siblings ...)
2023-09-28 16:49 ` [PATCH v6 07/12] eventdev/dma: support adapter service ID get Amit Prakash Shukla
@ 2023-09-28 16:49 ` Amit Prakash Shukla
2023-09-28 16:49 ` [PATCH v6 09/12] eventdev/dma: support adapter stats Amit Prakash Shukla
` (4 subsequent siblings)
12 siblings, 0 replies; 110+ messages in thread
From: Amit Prakash Shukla @ 2023-09-28 16:49 UTC (permalink / raw)
To: Amit Prakash Shukla, Jerin Jacob
Cc: dev, fengchengwen, kevin.laatz, bruce.richardson, conor.walsh,
vattunuru, g.singh, sachin.saxena, hemant.agrawal, cheng1.jiang,
ndabilpuram, anoobj, mb
Added support to set and get runtime params for DMA adapter. The
parameters that can be set/get are defined in
struct rte_event_dma_adapter_runtime_params.
Signed-off-by: Amit Prakash Shukla <amitprakashs@marvell.com>
---
lib/eventdev/rte_event_dma_adapter.c | 93 ++++++++++++++++++++++++++++
1 file changed, 93 insertions(+)
diff --git a/lib/eventdev/rte_event_dma_adapter.c b/lib/eventdev/rte_event_dma_adapter.c
index 63b07cd14e..850b010712 100644
--- a/lib/eventdev/rte_event_dma_adapter.c
+++ b/lib/eventdev/rte_event_dma_adapter.c
@@ -1211,3 +1211,96 @@ rte_event_dma_adapter_stop(uint8_t id)
{
return edma_adapter_ctrl(id, 0);
}
+
+#define DEFAULT_MAX_NB 128
+
+int
+rte_event_dma_adapter_runtime_params_init(struct rte_event_dma_adapter_runtime_params *params)
+{
+ if (params == NULL)
+ return -EINVAL;
+
+ memset(params, 0, sizeof(*params));
+ params->max_nb = DEFAULT_MAX_NB;
+
+ return 0;
+}
+
+static int
+dma_adapter_cap_check(struct event_dma_adapter *adapter)
+{
+ uint32_t caps;
+ int ret;
+
+ if (!adapter->nb_vchanq)
+ return -EINVAL;
+
+ ret = rte_event_dma_adapter_caps_get(adapter->eventdev_id, adapter->next_dmadev_id, &caps);
+ if (ret) {
+ RTE_EDEV_LOG_ERR("Failed to get adapter caps dev %" PRIu8 " cdev %" PRIu8,
+ adapter->eventdev_id, adapter->next_dmadev_id);
+ return ret;
+ }
+
+ if ((caps & RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP_FWD) ||
+ (caps & RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP_NEW))
+ return -ENOTSUP;
+
+ return 0;
+}
+
+int
+rte_event_dma_adapter_runtime_params_set(uint8_t id,
+ struct rte_event_dma_adapter_runtime_params *params)
+{
+ struct event_dma_adapter *adapter;
+ int ret;
+
+ EVENT_DMA_ADAPTER_ID_VALID_OR_ERR_RET(id, -EINVAL);
+
+ if (params == NULL) {
+ RTE_EDEV_LOG_ERR("params pointer is NULL\n");
+ return -EINVAL;
+ }
+
+ adapter = edma_id_to_adapter(id);
+ if (adapter == NULL)
+ return -EINVAL;
+
+ ret = dma_adapter_cap_check(adapter);
+ if (ret)
+ return ret;
+
+ rte_spinlock_lock(&adapter->lock);
+ adapter->max_nb = params->max_nb;
+ rte_spinlock_unlock(&adapter->lock);
+
+ return 0;
+}
+
+int
+rte_event_dma_adapter_runtime_params_get(uint8_t id,
+ struct rte_event_dma_adapter_runtime_params *params)
+{
+ struct event_dma_adapter *adapter;
+ int ret;
+
+ EVENT_DMA_ADAPTER_ID_VALID_OR_ERR_RET(id, -EINVAL);
+
+ if (params == NULL) {
+ RTE_EDEV_LOG_ERR("params pointer is NULL\n");
+ return -EINVAL;
+ }
+
+ adapter = edma_id_to_adapter(id);
+ if (adapter == NULL)
+ return -EINVAL;
+
+ ret = dma_adapter_cap_check(adapter);
+ if (ret)
+ return ret;
+
+ params->max_nb = adapter->max_nb;
+
+ return 0;
+}
--
2.25.1
^ permalink raw reply [flat|nested] 110+ messages in thread
* [PATCH v6 09/12] eventdev/dma: support adapter stats
2023-09-28 16:49 ` [PATCH v6 00/12] event DMA adapter library support Amit Prakash Shukla
` (7 preceding siblings ...)
2023-09-28 16:49 ` [PATCH v6 08/12] eventdev/dma: support adapter runtime params Amit Prakash Shukla
@ 2023-09-28 16:49 ` Amit Prakash Shukla
2023-09-28 16:49 ` [PATCH v6 10/12] eventdev/dma: support adapter enqueue Amit Prakash Shukla
` (3 subsequent siblings)
12 siblings, 0 replies; 110+ messages in thread
From: Amit Prakash Shukla @ 2023-09-28 16:49 UTC (permalink / raw)
To: Amit Prakash Shukla, Jerin Jacob
Cc: dev, fengchengwen, kevin.laatz, bruce.richardson, conor.walsh,
vattunuru, g.singh, sachin.saxena, hemant.agrawal, cheng1.jiang,
ndabilpuram, anoobj, mb
Added DMA adapter stats API support to get and reset stats. DMA
SW adapter stats and eventdev driver supported stats for enqueue
and dequeue are reported by get API.
Signed-off-by: Amit Prakash Shukla <amitprakashs@marvell.com>
---
lib/eventdev/rte_event_dma_adapter.c | 95 ++++++++++++++++++++++++++++
1 file changed, 95 insertions(+)
diff --git a/lib/eventdev/rte_event_dma_adapter.c b/lib/eventdev/rte_event_dma_adapter.c
index 850b010712..842fb74734 100644
--- a/lib/eventdev/rte_event_dma_adapter.c
+++ b/lib/eventdev/rte_event_dma_adapter.c
@@ -142,6 +142,9 @@ struct event_dma_adapter {
/* Loop counter to flush dma ops */
uint16_t transmit_loop_count;
+
+ /* Per instance stats structure */
+ struct rte_event_dma_adapter_stats dma_stats;
} __rte_cache_aligned;
static struct event_dma_adapter **event_dma_adapter;
@@ -471,6 +474,7 @@ rte_event_dma_adapter_free(uint8_t id)
static inline unsigned int
edma_enq_to_dma_dev(struct event_dma_adapter *adapter, struct rte_event *ev, unsigned int cnt)
{
+ struct rte_event_dma_adapter_stats *stats = &adapter->dma_stats;
struct dma_vchan_info *vchan_qinfo = NULL;
struct rte_event_dma_adapter_op *dma_op;
uint16_t vchan, nb_enqueued = 0;
@@ -480,6 +484,7 @@ edma_enq_to_dma_dev(struct event_dma_adapter *adapter, struct rte_event *ev, uns
ret = 0;
n = 0;
+ stats->event_deq_count += cnt;
for (i = 0; i < cnt; i++) {
dma_op = ev[i].event_ptr;
@@ -502,6 +507,7 @@ edma_enq_to_dma_dev(struct event_dma_adapter *adapter, struct rte_event *ev, uns
ret = edma_circular_buffer_flush_to_dma_dev(adapter, &vchan_qinfo->dma_buf,
dma_dev_id, vchan,
&nb_enqueued);
+ stats->dma_enq_count += nb_enqueued;
n += nb_enqueued;
/**
@@ -548,6 +554,7 @@ edma_adapter_dev_flush(struct event_dma_adapter *adapter, int16_t dma_dev_id,
static unsigned int
edma_adapter_enq_flush(struct event_dma_adapter *adapter)
{
+ struct rte_event_dma_adapter_stats *stats = &adapter->dma_stats;
int16_t dma_dev_id;
uint16_t nb_enqueued = 0;
uint16_t nb_ops_flushed = 0;
@@ -562,6 +569,8 @@ edma_adapter_enq_flush(struct event_dma_adapter *adapter)
if (!nb_ops_flushed)
adapter->stop_enq_to_dma_dev = false;
+ stats->dma_enq_count += nb_enqueued;
+
return nb_enqueued;
}
@@ -573,6 +582,7 @@ edma_adapter_enq_flush(struct event_dma_adapter *adapter)
static int
edma_adapter_enq_run(struct event_dma_adapter *adapter, unsigned int max_enq)
{
+ struct rte_event_dma_adapter_stats *stats = &adapter->dma_stats;
uint8_t event_port_id = adapter->event_port_id;
uint8_t event_dev_id = adapter->eventdev_id;
struct rte_event ev[DMA_BATCH_SIZE];
@@ -592,6 +602,7 @@ edma_adapter_enq_run(struct event_dma_adapter *adapter, unsigned int max_enq)
break;
}
+ stats->event_poll_count++;
n = rte_event_dequeue_burst(event_dev_id, event_port_id, ev, DMA_BATCH_SIZE, 0);
if (!n)
@@ -612,6 +623,7 @@ static inline uint16_t
edma_ops_enqueue_burst(struct event_dma_adapter *adapter, struct rte_event_dma_adapter_op **ops,
uint16_t num)
{
+ struct rte_event_dma_adapter_stats *stats = &adapter->dma_stats;
uint8_t event_port_id = adapter->event_port_id;
uint8_t event_dev_id = adapter->eventdev_id;
struct rte_event events[DMA_BATCH_SIZE];
@@ -651,6 +663,10 @@ edma_ops_enqueue_burst(struct event_dma_adapter *adapter, struct rte_event_dma_a
} while (retry++ < DMA_ADAPTER_MAX_EV_ENQ_RETRIES && nb_enqueued < nb_ev);
+ stats->event_enq_fail_count += nb_ev - nb_enqueued;
+ stats->event_enq_count += nb_enqueued;
+ stats->event_enq_retry_count += retry - 1;
+
return nb_enqueued;
}
@@ -705,6 +721,7 @@ edma_ops_buffer_flush(struct event_dma_adapter *adapter)
static inline unsigned int
edma_adapter_deq_run(struct event_dma_adapter *adapter, unsigned int max_deq)
{
+ struct rte_event_dma_adapter_stats *stats = &adapter->dma_stats;
struct dma_vchan_info *vchan_info;
struct dma_ops_circular_buffer *tq_buf;
struct rte_event_dma_adapter_op *ops;
@@ -742,6 +759,7 @@ edma_adapter_deq_run(struct event_dma_adapter *adapter, unsigned int max_deq)
continue;
done = false;
+ stats->dma_deq_count += n;
tq_buf = &dev_info->tqmap[vchan].dma_buf;
@@ -1304,3 +1322,80 @@ rte_event_dma_adapter_runtime_params_get(uint8_t id,
return 0;
}
+
+int
+rte_event_dma_adapter_stats_get(uint8_t id, struct rte_event_dma_adapter_stats *stats)
+{
+ struct rte_event_dma_adapter_stats dev_stats_sum = {0};
+ struct rte_event_dma_adapter_stats dev_stats;
+ struct event_dma_adapter *adapter;
+ struct dma_device_info *dev_info;
+ struct rte_eventdev *dev;
+ uint16_t num_dma_dev;
+ uint32_t i;
+ int ret;
+
+ EVENT_DMA_ADAPTER_ID_VALID_OR_ERR_RET(id, -EINVAL);
+
+ adapter = edma_id_to_adapter(id);
+ if (adapter == NULL || stats == NULL)
+ return -EINVAL;
+
+ num_dma_dev = rte_dma_count_avail();
+ dev = &rte_eventdevs[adapter->eventdev_id];
+ memset(stats, 0, sizeof(*stats));
+ for (i = 0; i < num_dma_dev; i++) {
+ dev_info = &adapter->dma_devs[i];
+
+ if (dev_info->internal_event_port == 0 ||
+ dev->dev_ops->dma_adapter_stats_get == NULL)
+ continue;
+
+ ret = (*dev->dev_ops->dma_adapter_stats_get)(dev, i, &dev_stats);
+ if (ret)
+ continue;
+
+ dev_stats_sum.dma_deq_count += dev_stats.dma_deq_count;
+ dev_stats_sum.event_enq_count += dev_stats.event_enq_count;
+ }
+
+ if (adapter->service_initialized)
+ *stats = adapter->dma_stats;
+
+ stats->dma_deq_count += dev_stats_sum.dma_deq_count;
+ stats->event_enq_count += dev_stats_sum.event_enq_count;
+
+ return 0;
+}
+
+int
+rte_event_dma_adapter_stats_reset(uint8_t id)
+{
+ struct event_dma_adapter *adapter;
+ struct dma_device_info *dev_info;
+ struct rte_eventdev *dev;
+ uint16_t num_dma_dev;
+ uint32_t i;
+
+ EVENT_DMA_ADAPTER_ID_VALID_OR_ERR_RET(id, -EINVAL);
+
+ adapter = edma_id_to_adapter(id);
+ if (adapter == NULL)
+ return -EINVAL;
+
+ num_dma_dev = rte_dma_count_avail();
+ dev = &rte_eventdevs[adapter->eventdev_id];
+ for (i = 0; i < num_dma_dev; i++) {
+ dev_info = &adapter->dma_devs[i];
+
+ if (dev_info->internal_event_port == 0 ||
+ dev->dev_ops->dma_adapter_stats_reset == NULL)
+ continue;
+
+ (*dev->dev_ops->dma_adapter_stats_reset)(dev, i);
+ }
+
+ memset(&adapter->dma_stats, 0, sizeof(adapter->dma_stats));
+
+ return 0;
+}
--
2.25.1
^ permalink raw reply [flat|nested] 110+ messages in thread
* [PATCH v6 10/12] eventdev/dma: support adapter enqueue
2023-09-28 16:49 ` [PATCH v6 00/12] event DMA adapter library support Amit Prakash Shukla
` (8 preceding siblings ...)
2023-09-28 16:49 ` [PATCH v6 09/12] eventdev/dma: support adapter stats Amit Prakash Shukla
@ 2023-09-28 16:49 ` Amit Prakash Shukla
2023-09-28 16:49 ` [PATCH v6 11/12] eventdev/dma: support adapter event port get Amit Prakash Shukla
` (2 subsequent siblings)
12 siblings, 0 replies; 110+ messages in thread
From: Amit Prakash Shukla @ 2023-09-28 16:49 UTC (permalink / raw)
To: Amit Prakash Shukla, Jerin Jacob
Cc: dev, fengchengwen, kevin.laatz, bruce.richardson, conor.walsh,
vattunuru, g.singh, sachin.saxena, hemant.agrawal, cheng1.jiang,
ndabilpuram, anoobj, mb
Added API support to enqueue a DMA operation to the DMA driver.
Signed-off-by: Amit Prakash Shukla <amitprakashs@marvell.com>
---
lib/eventdev/rte_event_dma_adapter.c | 13 +++++++++++++
1 file changed, 13 insertions(+)
diff --git a/lib/eventdev/rte_event_dma_adapter.c b/lib/eventdev/rte_event_dma_adapter.c
index 842fb74734..bca2be2731 100644
--- a/lib/eventdev/rte_event_dma_adapter.c
+++ b/lib/eventdev/rte_event_dma_adapter.c
@@ -1399,3 +1399,16 @@ rte_event_dma_adapter_stats_reset(uint8_t id)
return 0;
}
+
+uint16_t
+rte_event_dma_adapter_enqueue(uint8_t dev_id, uint8_t port_id, struct rte_event ev[],
+ uint16_t nb_events)
+{
+ const struct rte_event_fp_ops *fp_ops;
+ void *port;
+
+ fp_ops = &rte_event_fp_ops[dev_id];
+ port = fp_ops->data[port_id];
+
+ return fp_ops->dma_enqueue(port, ev, nb_events);
+}
--
2.25.1
^ permalink raw reply [flat|nested] 110+ messages in thread
* [PATCH v6 11/12] eventdev/dma: support adapter event port get
2023-09-28 16:49 ` [PATCH v6 00/12] event DMA adapter library support Amit Prakash Shukla
` (9 preceding siblings ...)
2023-09-28 16:49 ` [PATCH v6 10/12] eventdev/dma: support adapter enqueue Amit Prakash Shukla
@ 2023-09-28 16:49 ` Amit Prakash Shukla
2023-09-28 16:49 ` [PATCH v6 12/12] app/test: add event DMA adapter auto-test Amit Prakash Shukla
2023-09-29 8:12 ` [PATCH v7 00/12] event DMA adapter library support Amit Prakash Shukla
12 siblings, 0 replies; 110+ messages in thread
From: Amit Prakash Shukla @ 2023-09-28 16:49 UTC (permalink / raw)
To: Amit Prakash Shukla, Jerin Jacob
Cc: dev, fengchengwen, kevin.laatz, bruce.richardson, conor.walsh,
vattunuru, g.singh, sachin.saxena, hemant.agrawal, cheng1.jiang,
ndabilpuram, anoobj, mb
Added support for DMA adapter event port get.
Signed-off-by: Amit Prakash Shukla <amitprakashs@marvell.com>
---
lib/eventdev/rte_event_dma_adapter.c | 16 ++++++++++++++++
1 file changed, 16 insertions(+)
diff --git a/lib/eventdev/rte_event_dma_adapter.c b/lib/eventdev/rte_event_dma_adapter.c
index bca2be2731..4899bc5d0f 100644
--- a/lib/eventdev/rte_event_dma_adapter.c
+++ b/lib/eventdev/rte_event_dma_adapter.c
@@ -471,6 +471,22 @@ rte_event_dma_adapter_free(uint8_t id)
return 0;
}
+int
+rte_event_dma_adapter_event_port_get(uint8_t id, uint8_t *event_port_id)
+{
+ struct event_dma_adapter *adapter;
+
+ EVENT_DMA_ADAPTER_ID_VALID_OR_ERR_RET(id, -EINVAL);
+
+ adapter = edma_id_to_adapter(id);
+ if (adapter == NULL || event_port_id == NULL)
+ return -EINVAL;
+
+ *event_port_id = adapter->event_port_id;
+
+ return 0;
+}
+
static inline unsigned int
edma_enq_to_dma_dev(struct event_dma_adapter *adapter, struct rte_event *ev, unsigned int cnt)
{
--
2.25.1
^ permalink raw reply [flat|nested] 110+ messages in thread
* [PATCH v6 12/12] app/test: add event DMA adapter auto-test
2023-09-28 16:49 ` [PATCH v6 00/12] event DMA adapter library support Amit Prakash Shukla
` (10 preceding siblings ...)
2023-09-28 16:49 ` [PATCH v6 11/12] eventdev/dma: support adapter event port get Amit Prakash Shukla
@ 2023-09-28 16:49 ` Amit Prakash Shukla
2023-09-29 8:12 ` [PATCH v7 00/12] event DMA adapter library support Amit Prakash Shukla
12 siblings, 0 replies; 110+ messages in thread
From: Amit Prakash Shukla @ 2023-09-28 16:49 UTC (permalink / raw)
To: Thomas Monjalon, Amit Prakash Shukla
Cc: dev, jerinj, fengchengwen, kevin.laatz, bruce.richardson,
conor.walsh, vattunuru, g.singh, sachin.saxena, hemant.agrawal,
cheng1.jiang, ndabilpuram, anoobj, mb
Added testsuite to test the dma adapter functionality.
The testsuite detects event and DMA device capability
and accordingly dma adapter is configured and modes are
tested. Test command:
<build dir>/app/test/dpdk-test event_dma_adapter_autotest
Signed-off-by: Amit Prakash Shukla <amitprakashs@marvell.com>
---
MAINTAINERS | 1 +
app/test/meson.build | 1 +
app/test/test_event_dma_adapter.c | 805 ++++++++++++++++++++++++++++++
3 files changed, 807 insertions(+)
create mode 100644 app/test/test_event_dma_adapter.c
diff --git a/MAINTAINERS b/MAINTAINERS
index 4ebbbe8bb3..92c0b47618 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -544,6 +544,7 @@ Eventdev DMA Adapter API
M: Amit Prakash Shukla <amitprakashs@marvell.com>
T: git://dpdk.org/next/dpdk-next-eventdev
F: lib/eventdev/*dma_adapter*
+F: app/test/test_event_dma_adapter.c
F: doc/guides/prog_guide/event_dma_adapter.rst
Raw device API
diff --git a/app/test/meson.build b/app/test/meson.build
index 05bae9216d..7caf5ae5fc 100644
--- a/app/test/meson.build
+++ b/app/test/meson.build
@@ -66,6 +66,7 @@ source_file_deps = {
'test_errno.c': [],
'test_ethdev_link.c': ['ethdev'],
'test_event_crypto_adapter.c': ['cryptodev', 'eventdev', 'bus_vdev'],
+ 'test_event_dma_adapter.c': ['dmadev', 'eventdev', 'bus_vdev'],
'test_event_eth_rx_adapter.c': ['ethdev', 'eventdev', 'bus_vdev'],
'test_event_eth_tx_adapter.c': ['bus_vdev', 'ethdev', 'net_ring', 'eventdev'],
'test_event_ring.c': ['eventdev'],
diff --git a/app/test/test_event_dma_adapter.c b/app/test/test_event_dma_adapter.c
new file mode 100644
index 0000000000..1e193f4b52
--- /dev/null
+++ b/app/test/test_event_dma_adapter.c
@@ -0,0 +1,805 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright (c) 2023 Marvell.
+ */
+
+#include "test.h"
+#include <string.h>
+#include <rte_common.h>
+#include <rte_malloc.h>
+#include <rte_mempool.h>
+#include <rte_mbuf.h>
+#include <rte_random.h>
+
+#ifdef RTE_EXEC_ENV_WINDOWS
+static int
+test_event_dma_adapter(void)
+{
+ printf("event_dma_adapter not supported on Windows, skipping test\n");
+ return TEST_SKIPPED;
+}
+
+#else
+
+#include <rte_bus_vdev.h>
+#include <rte_dmadev.h>
+#include <rte_eventdev.h>
+#include <rte_event_dma_adapter.h>
+#include <rte_service.h>
+
+#define NUM_MBUFS (8191)
+#define MBUF_CACHE_SIZE (256)
+#define TEST_APP_PORT_ID 0
+#define TEST_APP_EV_QUEUE_ID 0
+#define TEST_APP_EV_PRIORITY 0
+#define TEST_APP_EV_FLOWID 0xAABB
+#define TEST_DMA_EV_QUEUE_ID 1
+#define TEST_ADAPTER_ID 0
+#define TEST_DMA_DEV_ID 0
+#define TEST_DMA_VCHAN_ID 0
+#define PACKET_LENGTH 1024
+#define NB_TEST_PORTS 1
+#define NB_TEST_QUEUES 2
+#define NUM_CORES 2
+#define DMA_OP_POOL_SIZE 128
+#define TEST_MAX_OP 32
+#define TEST_RINGSIZE 512
+
+#define MBUF_SIZE (RTE_PKTMBUF_HEADROOM + PACKET_LENGTH)
+
+/* Handle log statements in same manner as test macros */
+#define LOG_DBG(...) RTE_LOG(DEBUG, EAL, __VA_ARGS__)
+
+struct event_dma_adapter_test_params {
+ struct rte_mempool *src_mbuf_pool;
+ struct rte_mempool *dst_mbuf_pool;
+ struct rte_mempool *op_mpool;
+ uint8_t dma_event_port_id;
+ uint8_t internal_port_op_fwd;
+};
+
+struct rte_event dma_response_info = {
+ .queue_id = TEST_APP_EV_QUEUE_ID,
+ .sched_type = RTE_SCHED_TYPE_ATOMIC,
+ .flow_id = TEST_APP_EV_FLOWID,
+ .priority = TEST_APP_EV_PRIORITY
+};
+
+static struct event_dma_adapter_test_params params;
+static uint8_t dma_adapter_setup_done;
+static uint32_t slcore_id;
+static int evdev;
+
+static int
+send_recv_ev(struct rte_event *ev)
+{
+ struct rte_event recv_ev[TEST_MAX_OP];
+ uint16_t nb_enqueued = 0;
+ int i = 0;
+
+ if (params.internal_port_op_fwd) {
+ nb_enqueued = rte_event_dma_adapter_enqueue(evdev, TEST_APP_PORT_ID, ev,
+ TEST_MAX_OP);
+ } else {
+ while (nb_enqueued < TEST_MAX_OP) {
+ nb_enqueued += rte_event_enqueue_burst(evdev, TEST_APP_PORT_ID,
+ &ev[nb_enqueued], TEST_MAX_OP -
+ nb_enqueued);
+ }
+ }
+
+ TEST_ASSERT_EQUAL(nb_enqueued, TEST_MAX_OP, "Failed to send event to dma adapter\n");
+
+ while (i < TEST_MAX_OP) {
+ if (rte_event_dequeue_burst(evdev, TEST_APP_PORT_ID, &recv_ev[i], 1, 0) != 1)
+ continue;
+ i++;
+ }
+
+ TEST_ASSERT_EQUAL(i, TEST_MAX_OP, "Test failed. Failed to dequeue events.\n");
+
+ return TEST_SUCCESS;
+}
+
+static int
+test_dma_adapter_stats(void)
+{
+ struct rte_event_dma_adapter_stats stats;
+
+ rte_event_dma_adapter_stats_get(TEST_ADAPTER_ID, &stats);
+ printf(" +------------------------------------------------------+\n");
+ printf(" + DMA adapter stats for instance %u:\n", TEST_ADAPTER_ID);
+ printf(" + Event port poll count 0x%" PRIx64 "\n",
+ stats.event_poll_count);
+ printf(" + Event dequeue count 0x%" PRIx64 "\n",
+ stats.event_deq_count);
+ printf(" + DMA dev enqueue count 0x%" PRIx64 "\n",
+ stats.dma_enq_count);
+ printf(" + DMA dev enqueue failed count 0x%" PRIx64 "\n",
+ stats.dma_enq_fail_count);
+ printf(" + DMA dev dequeue count 0x%" PRIx64 "\n",
+ stats.dma_deq_count);
+ printf(" + Event enqueue count 0x%" PRIx64 "\n",
+ stats.event_enq_count);
+ printf(" + Event enqueue retry count 0x%" PRIx64 "\n",
+ stats.event_enq_retry_count);
+ printf(" + Event enqueue fail count 0x%" PRIx64 "\n",
+ stats.event_enq_fail_count);
+ printf(" +------------------------------------------------------+\n");
+
+ rte_event_dma_adapter_stats_reset(TEST_ADAPTER_ID);
+ return TEST_SUCCESS;
+}
+
+static int
+test_dma_adapter_params(void)
+{
+ struct rte_event_dma_adapter_runtime_params out_params;
+ struct rte_event_dma_adapter_runtime_params in_params;
+ struct rte_event event;
+ uint32_t cap;
+ int err, rc;
+
+ err = rte_event_dma_adapter_caps_get(evdev, TEST_DMA_DEV_ID, &cap);
+ TEST_ASSERT_SUCCESS(err, "Failed to get adapter capabilities\n");
+
+ if (cap & RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_VCHAN_EV_BIND) {
+ err = rte_event_dma_adapter_vchan_add(TEST_ADAPTER_ID, TEST_DMA_DEV_ID,
+ TEST_DMA_VCHAN_ID, &event);
+ } else
+ err = rte_event_dma_adapter_vchan_add(TEST_ADAPTER_ID, TEST_DMA_DEV_ID,
+ TEST_DMA_VCHAN_ID, NULL);
+
+ TEST_ASSERT_SUCCESS(err, "Failed to add vchan\n");
+
+ err = rte_event_dma_adapter_runtime_params_init(&in_params);
+ TEST_ASSERT(err == 0, "Expected 0 got %d", err);
+ err = rte_event_dma_adapter_runtime_params_init(&out_params);
+ TEST_ASSERT(err == 0, "Expected 0 got %d", err);
+
+ /* Case 1: Get the default value of mbufs processed by adapter */
+ err = rte_event_dma_adapter_runtime_params_get(TEST_ADAPTER_ID, &out_params);
+ if (err == -ENOTSUP) {
+ rc = TEST_SKIPPED;
+ goto vchan_del;
+ }
+ TEST_ASSERT(err == 0, "Expected 0 got %d", err);
+
+ /* Case 2: Set max_nb = 32 (=BATCH_SEIZE) */
+ in_params.max_nb = 32;
+
+ err = rte_event_dma_adapter_runtime_params_set(TEST_ADAPTER_ID, &in_params);
+ TEST_ASSERT(err == 0, "Expected 0 got %d", err);
+
+ err = rte_event_dma_adapter_runtime_params_get(TEST_ADAPTER_ID, &out_params);
+ TEST_ASSERT(err == 0, "Expected 0 got %d", err);
+ TEST_ASSERT(in_params.max_nb == out_params.max_nb, "Expected %u got %u",
+ in_params.max_nb, out_params.max_nb);
+
+ /* Case 3: Set max_nb = 192 */
+ in_params.max_nb = 192;
+
+ err = rte_event_dma_adapter_runtime_params_set(TEST_ADAPTER_ID, &in_params);
+ TEST_ASSERT(err == 0, "Expected 0 got %d", err);
+
+ err = rte_event_dma_adapter_runtime_params_get(TEST_ADAPTER_ID, &out_params);
+ TEST_ASSERT(err == 0, "Expected 0 got %d", err);
+ TEST_ASSERT(in_params.max_nb == out_params.max_nb, "Expected %u got %u",
+ in_params.max_nb, out_params.max_nb);
+
+ /* Case 4: Set max_nb = 256 */
+ in_params.max_nb = 256;
+
+ err = rte_event_dma_adapter_runtime_params_set(TEST_ADAPTER_ID, &in_params);
+ TEST_ASSERT(err == 0, "Expected 0 got %d", err);
+
+ err = rte_event_dma_adapter_runtime_params_get(TEST_ADAPTER_ID, &out_params);
+ TEST_ASSERT(err == 0, "Expected 0 got %d", err);
+ TEST_ASSERT(in_params.max_nb == out_params.max_nb, "Expected %u got %u",
+ in_params.max_nb, out_params.max_nb);
+
+ /* Case 5: Set max_nb = 30(<BATCH_SIZE) */
+ in_params.max_nb = 30;
+
+ err = rte_event_dma_adapter_runtime_params_set(TEST_ADAPTER_ID, &in_params);
+ TEST_ASSERT(err == 0, "Expected 0 got %d", err);
+
+ err = rte_event_dma_adapter_runtime_params_get(TEST_ADAPTER_ID, &out_params);
+ TEST_ASSERT(err == 0, "Expected 0 got %d", err);
+ TEST_ASSERT(in_params.max_nb == out_params.max_nb, "Expected %u got %u",
+ in_params.max_nb, out_params.max_nb);
+
+ /* Case 6: Set max_nb = 512 */
+ in_params.max_nb = 512;
+
+ err = rte_event_dma_adapter_runtime_params_set(TEST_ADAPTER_ID, &in_params);
+ TEST_ASSERT(err == 0, "Expected 0 got %d", err);
+
+ err = rte_event_dma_adapter_runtime_params_get(TEST_ADAPTER_ID, &out_params);
+ TEST_ASSERT(err == 0, "Expected 0 got %d", err);
+ TEST_ASSERT(in_params.max_nb == out_params.max_nb, "Expected %u got %u",
+ in_params.max_nb, out_params.max_nb);
+
+ rc = TEST_SUCCESS;
+vchan_del:
+ err = rte_event_dma_adapter_vchan_del(TEST_ADAPTER_ID, TEST_DMA_DEV_ID,
+ TEST_DMA_VCHAN_ID);
+ TEST_ASSERT_SUCCESS(err, "Failed to delete vchan\n");
+
+ return rc;
+}
+
+static int
+test_op_forward_mode(void)
+{
+ struct rte_mbuf *src_mbuf[TEST_MAX_OP];
+ struct rte_mbuf *dst_mbuf[TEST_MAX_OP];
+ struct rte_event_dma_adapter_op *op;
+ struct rte_event ev[TEST_MAX_OP];
+ struct rte_event response_info;
+ int ret, i;
+
+ ret = rte_pktmbuf_alloc_bulk(params.src_mbuf_pool, src_mbuf, TEST_MAX_OP);
+ TEST_ASSERT_SUCCESS(ret, "alloc src mbufs failed.\n");
+
+ ret = rte_pktmbuf_alloc_bulk(params.dst_mbuf_pool, dst_mbuf, TEST_MAX_OP);
+ TEST_ASSERT_SUCCESS(ret, "alloc dst mbufs failed.\n");
+
+ for (i = 0; i < TEST_MAX_OP; i++) {
+ memset(rte_pktmbuf_mtod(src_mbuf[i], void *), rte_rand(), PACKET_LENGTH);
+ memset(rte_pktmbuf_mtod(dst_mbuf[i], void *), 0, PACKET_LENGTH);
+ }
+
+ for (i = 0; i < TEST_MAX_OP; i++) {
+ rte_mempool_get(params.op_mpool, (void **)&op);
+ TEST_ASSERT_NOT_NULL(op, "Failed to allocate dma operation struct\n");
+
+ op->src_seg = rte_malloc(NULL, sizeof(struct rte_dma_sge), 0);
+ op->dst_seg = rte_malloc(NULL, sizeof(struct rte_dma_sge), 0);
+
+ /* Update Op */
+ op->src_seg->addr = rte_pktmbuf_iova(src_mbuf[i]);
+ op->dst_seg->addr = rte_pktmbuf_iova(dst_mbuf[i]);
+ op->src_seg->length = PACKET_LENGTH;
+ op->dst_seg->length = PACKET_LENGTH;
+ op->nb_src = 1;
+ op->nb_dst = 1;
+ op->flags = RTE_DMA_OP_FLAG_SUBMIT;
+ op->op_mp = params.op_mpool;
+ op->dma_dev_id = TEST_DMA_DEV_ID;
+ op->vchan = TEST_DMA_VCHAN_ID;
+
+ response_info.event = dma_response_info.event;
+ rte_memcpy((uint8_t *)op + sizeof(struct rte_event_dma_adapter_op), &response_info,
+ sizeof(struct rte_event));
+
+ /* Fill in event info and update event_ptr with rte_event_dma_adapter_op */
+ memset(&ev[i], 0, sizeof(struct rte_event));
+ ev[i].event = 0;
+ ev[i].event_type = RTE_EVENT_TYPE_DMADEV;
+ ev[i].queue_id = TEST_DMA_EV_QUEUE_ID;
+ ev[i].sched_type = RTE_SCHED_TYPE_ATOMIC;
+ ev[i].flow_id = 0xAABB;
+ ev[i].event_ptr = op;
+ }
+
+ ret = send_recv_ev(ev);
+ TEST_ASSERT_SUCCESS(ret, "Failed to send/receive event to dma adapter\n");
+
+ test_dma_adapter_stats();
+
+ for (i = 0; i < TEST_MAX_OP; i++) {
+ op = ev[i].event_ptr;
+ ret = memcmp(rte_pktmbuf_mtod(src_mbuf[i], void *),
+ rte_pktmbuf_mtod(dst_mbuf[i], void *), PACKET_LENGTH);
+
+ TEST_ASSERT_EQUAL(ret, 0, "Data mismatch for dma adapter\n");
+
+ rte_free(op->src_seg);
+ rte_free(op->dst_seg);
+ rte_mempool_put(op->op_mp, op);
+ }
+
+ rte_pktmbuf_free_bulk(src_mbuf, TEST_MAX_OP);
+ rte_pktmbuf_free_bulk(dst_mbuf, TEST_MAX_OP);
+
+ return TEST_SUCCESS;
+}
+
+static int
+map_adapter_service_core(void)
+{
+ uint32_t adapter_service_id;
+ int ret;
+
+ if (rte_event_dma_adapter_service_id_get(TEST_ADAPTER_ID, &adapter_service_id) == 0) {
+ uint32_t core_list[NUM_CORES];
+
+ ret = rte_service_lcore_list(core_list, NUM_CORES);
+ TEST_ASSERT(ret >= 0, "Failed to get service core list!");
+
+ if (core_list[0] != slcore_id) {
+ TEST_ASSERT_SUCCESS(rte_service_lcore_add(slcore_id),
+ "Failed to add service core");
+ TEST_ASSERT_SUCCESS(rte_service_lcore_start(slcore_id),
+ "Failed to start service core");
+ }
+
+ TEST_ASSERT_SUCCESS(rte_service_map_lcore_set(
+ adapter_service_id, slcore_id, 1),
+ "Failed to map adapter service");
+ }
+
+ return TEST_SUCCESS;
+}
+
+static int
+test_with_op_forward_mode(void)
+{
+ uint32_t cap;
+ int ret;
+
+ ret = rte_event_dma_adapter_caps_get(evdev, TEST_DMA_DEV_ID, &cap);
+ TEST_ASSERT_SUCCESS(ret, "Failed to get adapter capabilities\n");
+
+ if (!(cap & RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP_FWD) &&
+ !(cap & RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP_NEW))
+ map_adapter_service_core();
+ else {
+ if (!(cap & RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP_FWD))
+ return TEST_SKIPPED;
+ }
+
+ TEST_ASSERT_SUCCESS(rte_event_dma_adapter_start(TEST_ADAPTER_ID),
+ "Failed to start event dma adapter");
+
+ ret = test_op_forward_mode();
+ TEST_ASSERT_SUCCESS(ret, "DMA - FORWARD mode test failed\n");
+ return TEST_SUCCESS;
+}
+
+static int
+configure_dmadev(void)
+{
+ const struct rte_dma_conf conf = { .nb_vchans = 1};
+ const struct rte_dma_vchan_conf qconf = {
+ .direction = RTE_DMA_DIR_MEM_TO_MEM,
+ .nb_desc = TEST_RINGSIZE,
+ };
+ struct rte_dma_info info;
+ unsigned int elt_size;
+ int ret;
+
+ ret = rte_dma_count_avail();
+ RTE_TEST_ASSERT_FAIL(ret, "No dma devices found!\n");
+
+ ret = rte_dma_info_get(TEST_DMA_DEV_ID, &info);
+ TEST_ASSERT_SUCCESS(ret, "Error with rte_dma_info_get()\n");
+
+ if (info.max_vchans < 1)
+ RTE_LOG(ERR, USER1, "Error, no channels available on device id %u\n",
+ TEST_DMA_DEV_ID);
+
+ if (rte_dma_configure(TEST_DMA_DEV_ID, &conf) != 0)
+ RTE_LOG(ERR, USER1, "Error with rte_dma_configure()\n");
+
+ if (rte_dma_vchan_setup(TEST_DMA_DEV_ID, TEST_DMA_VCHAN_ID, &qconf) < 0)
+ RTE_LOG(ERR, USER1, "Error with vchan configuration\n");
+
+ ret = rte_dma_info_get(TEST_DMA_DEV_ID, &info);
+ if (ret != 0 || info.nb_vchans != 1)
+ RTE_LOG(ERR, USER1, "Error, no configured vhcan reported on device id %u\n",
+ TEST_DMA_DEV_ID);
+
+ params.src_mbuf_pool = rte_pktmbuf_pool_create("DMA_ADAPTER_SRC_MBUFPOOL", NUM_MBUFS,
+ MBUF_CACHE_SIZE, 0, MBUF_SIZE,
+ rte_socket_id());
+ RTE_TEST_ASSERT_NOT_NULL(params.src_mbuf_pool, "Can't create DMA_SRC_MBUFPOOL\n");
+
+ params.dst_mbuf_pool = rte_pktmbuf_pool_create("DMA_ADAPTER_DST_MBUFPOOL", NUM_MBUFS,
+ MBUF_CACHE_SIZE, 0, MBUF_SIZE,
+ rte_socket_id());
+ RTE_TEST_ASSERT_NOT_NULL(params.dst_mbuf_pool, "Can't create DMA_DST_MBUFPOOL\n");
+
+ elt_size = sizeof(struct rte_event_dma_adapter_op) + sizeof(struct rte_event);
+ params.op_mpool = rte_mempool_create("EVENT_DMA_OP_POOL", DMA_OP_POOL_SIZE, elt_size, 0,
+ 0, NULL, NULL, NULL, NULL, rte_socket_id(), 0);
+ RTE_TEST_ASSERT_NOT_NULL(params.op_mpool, "Can't create DMA_OP_POOL\n");
+
+ return TEST_SUCCESS;
+}
+
+static inline void
+evdev_set_conf_values(struct rte_event_dev_config *dev_conf, struct rte_event_dev_info *info)
+{
+ memset(dev_conf, 0, sizeof(struct rte_event_dev_config));
+ dev_conf->dequeue_timeout_ns = info->min_dequeue_timeout_ns;
+ dev_conf->nb_event_ports = NB_TEST_PORTS;
+ dev_conf->nb_event_queues = NB_TEST_QUEUES;
+ dev_conf->nb_event_queue_flows = info->max_event_queue_flows;
+ dev_conf->nb_event_port_dequeue_depth =
+ info->max_event_port_dequeue_depth;
+ dev_conf->nb_event_port_enqueue_depth =
+ info->max_event_port_enqueue_depth;
+ dev_conf->nb_event_port_enqueue_depth =
+ info->max_event_port_enqueue_depth;
+ dev_conf->nb_events_limit =
+ info->max_num_events;
+}
+
+static int
+configure_eventdev(void)
+{
+ struct rte_event_queue_conf queue_conf;
+ struct rte_event_dev_config devconf;
+ struct rte_event_dev_info info;
+ uint32_t queue_count;
+ uint32_t port_count;
+ uint8_t qid;
+ int ret;
+
+ if (!rte_event_dev_count()) {
+ /* If there is no hardware eventdev, or no software vdev was
+ * specified on the command line, create an instance of
+ * event_sw.
+ */
+ LOG_DBG("Failed to find a valid event device... "
+ "testing with event_sw device\n");
+ TEST_ASSERT_SUCCESS(rte_vdev_init("event_sw0", NULL),
+ "Error creating eventdev");
+ evdev = rte_event_dev_get_dev_id("event_sw0");
+ }
+
+ ret = rte_event_dev_info_get(evdev, &info);
+ TEST_ASSERT_SUCCESS(ret, "Failed to get event dev info\n");
+
+ evdev_set_conf_values(&devconf, &info);
+
+ ret = rte_event_dev_configure(evdev, &devconf);
+ TEST_ASSERT_SUCCESS(ret, "Failed to configure eventdev\n");
+
+ /* Set up event queue */
+ ret = rte_event_dev_attr_get(evdev, RTE_EVENT_DEV_ATTR_QUEUE_COUNT, &queue_count);
+ TEST_ASSERT_SUCCESS(ret, "Queue count get failed\n");
+ TEST_ASSERT_EQUAL(queue_count, 2, "Unexpected queue count\n");
+
+ qid = TEST_APP_EV_QUEUE_ID;
+ ret = rte_event_queue_setup(evdev, qid, NULL);
+ TEST_ASSERT_SUCCESS(ret, "Failed to setup queue=%d\n", qid);
+
+ queue_conf.nb_atomic_flows = info.max_event_queue_flows;
+ queue_conf.nb_atomic_order_sequences = 32;
+ queue_conf.schedule_type = RTE_SCHED_TYPE_ATOMIC;
+ queue_conf.priority = RTE_EVENT_DEV_PRIORITY_HIGHEST;
+ queue_conf.event_queue_cfg = RTE_EVENT_QUEUE_CFG_SINGLE_LINK;
+
+ qid = TEST_DMA_EV_QUEUE_ID;
+ ret = rte_event_queue_setup(evdev, qid, &queue_conf);
+ TEST_ASSERT_SUCCESS(ret, "Failed to setup queue=%u\n", qid);
+
+ /* Set up event port */
+ ret = rte_event_dev_attr_get(evdev, RTE_EVENT_DEV_ATTR_PORT_COUNT,
+ &port_count);
+ TEST_ASSERT_SUCCESS(ret, "Port count get failed\n");
+ TEST_ASSERT_EQUAL(port_count, 1, "Unexpected port count\n");
+
+ ret = rte_event_port_setup(evdev, TEST_APP_PORT_ID, NULL);
+ TEST_ASSERT_SUCCESS(ret, "Failed to setup port=%d\n",
+ TEST_APP_PORT_ID);
+
+ qid = TEST_APP_EV_QUEUE_ID;
+ ret = rte_event_port_link(evdev, TEST_APP_PORT_ID, &qid, NULL, 1);
+ TEST_ASSERT(ret >= 0, "Failed to link queue port=%d\n",
+ TEST_APP_PORT_ID);
+
+ return TEST_SUCCESS;
+}
+
+static void
+test_dma_adapter_free(void)
+{
+ rte_event_dma_adapter_free(TEST_ADAPTER_ID);
+}
+
+static int
+test_dma_adapter_create(void)
+{
+ struct rte_event_dev_info evdev_info = {0};
+ struct rte_event_port_conf conf = {0};
+ int ret;
+
+ ret = rte_event_dev_info_get(evdev, &evdev_info);
+ TEST_ASSERT_SUCCESS(ret, "Failed to create event dma adapter\n");
+
+ conf.new_event_threshold = evdev_info.max_num_events;
+ conf.dequeue_depth = evdev_info.max_event_port_dequeue_depth;
+ conf.enqueue_depth = evdev_info.max_event_port_enqueue_depth;
+
+ /* Create adapter with default port creation callback */
+ ret = rte_event_dma_adapter_create(TEST_ADAPTER_ID, evdev, &conf, 0);
+ TEST_ASSERT_SUCCESS(ret, "Failed to create event dma adapter\n");
+
+ return TEST_SUCCESS;
+}
+
+static int
+test_dma_adapter_vchan_add_del(void)
+{
+ struct rte_event event;
+ uint32_t cap;
+ int ret;
+
+ ret = rte_event_dma_adapter_caps_get(evdev, TEST_DMA_DEV_ID, &cap);
+ TEST_ASSERT_SUCCESS(ret, "Failed to get adapter capabilities\n");
+
+ if (cap & RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_VCHAN_EV_BIND) {
+ ret = rte_event_dma_adapter_vchan_add(TEST_ADAPTER_ID, TEST_DMA_DEV_ID,
+ TEST_DMA_VCHAN_ID, &event);
+ } else
+ ret = rte_event_dma_adapter_vchan_add(TEST_ADAPTER_ID, TEST_DMA_DEV_ID,
+ TEST_DMA_VCHAN_ID, NULL);
+
+ TEST_ASSERT_SUCCESS(ret, "Failed to create add vchan\n");
+
+ ret = rte_event_dma_adapter_vchan_del(TEST_ADAPTER_ID, TEST_DMA_DEV_ID,
+ TEST_DMA_VCHAN_ID);
+ TEST_ASSERT_SUCCESS(ret, "Failed to delete vchan\n");
+
+ return TEST_SUCCESS;
+}
+
+static int
+configure_event_dma_adapter(enum rte_event_dma_adapter_mode mode)
+{
+ struct rte_event_dev_info evdev_info = {0};
+ struct rte_event_port_conf conf = {0};
+ struct rte_event event;
+ uint32_t cap;
+ int ret;
+
+ ret = rte_event_dma_adapter_caps_get(evdev, TEST_DMA_DEV_ID, &cap);
+ TEST_ASSERT_SUCCESS(ret, "Failed to get adapter capabilities\n");
+
+ /* Skip mode and capability mismatch check for SW eventdev */
+ if (!(cap & RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP_NEW) &&
+ !(cap & RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP_FWD) &&
+ !(cap & RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_VCHAN_EV_BIND))
+ goto adapter_create;
+
+ if (mode == RTE_EVENT_DMA_ADAPTER_OP_FORWARD) {
+ if (cap & RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP_FWD)
+ params.internal_port_op_fwd = 1;
+ else
+ return -ENOTSUP;
+ }
+
+adapter_create:
+ ret = rte_event_dev_info_get(evdev, &evdev_info);
+ TEST_ASSERT_SUCCESS(ret, "Failed to create event dma adapter\n");
+
+ conf.new_event_threshold = evdev_info.max_num_events;
+ conf.dequeue_depth = evdev_info.max_event_port_dequeue_depth;
+ conf.enqueue_depth = evdev_info.max_event_port_enqueue_depth;
+
+ /* Create adapter with default port creation callback */
+ ret = rte_event_dma_adapter_create(TEST_ADAPTER_ID, evdev, &conf, mode);
+ TEST_ASSERT_SUCCESS(ret, "Failed to create event dma adapter\n");
+
+ if (cap & RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_VCHAN_EV_BIND) {
+ ret = rte_event_dma_adapter_vchan_add(TEST_ADAPTER_ID, TEST_DMA_DEV_ID,
+ TEST_DMA_VCHAN_ID, &event);
+ } else
+ ret = rte_event_dma_adapter_vchan_add(TEST_ADAPTER_ID, TEST_DMA_DEV_ID,
+ TEST_DMA_VCHAN_ID, NULL);
+
+ TEST_ASSERT_SUCCESS(ret, "Failed to add vchan\n");
+
+ if (!params.internal_port_op_fwd) {
+ ret = rte_event_dma_adapter_event_port_get(TEST_ADAPTER_ID,
+ ¶ms.dma_event_port_id);
+ TEST_ASSERT_SUCCESS(ret, "Failed to get event port\n");
+ }
+
+ return TEST_SUCCESS;
+}
+
+static void
+test_dma_adapter_stop(void)
+{
+ uint32_t evdev_service_id, adapter_service_id;
+
+ /* retrieve service ids & stop services */
+ if (rte_event_dma_adapter_service_id_get(TEST_ADAPTER_ID,
+ &adapter_service_id) == 0) {
+ rte_service_runstate_set(adapter_service_id, 0);
+ rte_service_lcore_stop(slcore_id);
+ rte_service_lcore_del(slcore_id);
+ rte_event_dma_adapter_stop(TEST_ADAPTER_ID);
+ }
+
+ if (rte_event_dev_service_id_get(evdev, &evdev_service_id) == 0) {
+ rte_service_runstate_set(evdev_service_id, 0);
+ rte_service_lcore_stop(slcore_id);
+ rte_service_lcore_del(slcore_id);
+ rte_dma_stop(TEST_DMA_DEV_ID);
+ rte_event_dev_stop(evdev);
+ } else {
+ rte_dma_stop(TEST_DMA_DEV_ID);
+ rte_event_dev_stop(evdev);
+ }
+}
+
+static int
+test_dma_adapter_conf(enum rte_event_dma_adapter_mode mode)
+{
+ uint32_t evdev_service_id;
+ uint8_t qid;
+ int ret;
+
+ if (!dma_adapter_setup_done) {
+ ret = configure_event_dma_adapter(mode);
+ if (ret)
+ return ret;
+ if (!params.internal_port_op_fwd) {
+ qid = TEST_DMA_EV_QUEUE_ID;
+ ret = rte_event_port_link(evdev,
+ params.dma_event_port_id, &qid, NULL, 1);
+ TEST_ASSERT(ret >= 0, "Failed to link queue %d "
+ "port=%u\n", qid,
+ params.dma_event_port_id);
+ }
+ dma_adapter_setup_done = 1;
+ }
+
+ /* retrieve service ids */
+ if (rte_event_dev_service_id_get(evdev, &evdev_service_id) == 0) {
+ /* add a service core and start it */
+ TEST_ASSERT_SUCCESS(rte_service_lcore_add(slcore_id),
+ "Failed to add service core");
+ TEST_ASSERT_SUCCESS(rte_service_lcore_start(slcore_id),
+ "Failed to start service core");
+
+ /* map services to it */
+ TEST_ASSERT_SUCCESS(rte_service_map_lcore_set(evdev_service_id,
+ slcore_id, 1), "Failed to map evdev service");
+
+ /* set services to running */
+ TEST_ASSERT_SUCCESS(rte_service_runstate_set(evdev_service_id,
+ 1), "Failed to start evdev service");
+ }
+
+ /* start the eventdev */
+ TEST_ASSERT_SUCCESS(rte_event_dev_start(evdev),
+ "Failed to start event device");
+
+ /* start the dma dev */
+ TEST_ASSERT_SUCCESS(rte_dma_start(TEST_DMA_DEV_ID),
+ "Failed to start dma device");
+
+ return TEST_SUCCESS;
+}
+
+static int
+test_dma_adapter_conf_op_forward_mode(void)
+{
+ enum rte_event_dma_adapter_mode mode;
+
+ mode = RTE_EVENT_DMA_ADAPTER_OP_FORWARD;
+
+ return test_dma_adapter_conf(mode);
+}
+
+static int
+testsuite_setup(void)
+{
+ int ret;
+
+ slcore_id = rte_get_next_lcore(-1, 1, 0);
+ TEST_ASSERT_NOT_EQUAL(slcore_id, RTE_MAX_LCORE, "At least 2 lcores "
+ "are required to run this autotest\n");
+
+ /* Setup and start event device. */
+ ret = configure_eventdev();
+ TEST_ASSERT_SUCCESS(ret, "Failed to setup eventdev\n");
+
+ /* Setup and start dma device. */
+ ret = configure_dmadev();
+ TEST_ASSERT_SUCCESS(ret, "dmadev initialization failed\n");
+
+ return TEST_SUCCESS;
+}
+
+static void
+dma_adapter_teardown(void)
+{
+ int ret;
+
+ ret = rte_event_dma_adapter_stop(TEST_ADAPTER_ID);
+ if (ret < 0)
+ RTE_LOG(ERR, USER1, "Failed to stop adapter!");
+
+ ret = rte_event_dma_adapter_vchan_del(TEST_ADAPTER_ID, TEST_DMA_DEV_ID,
+ TEST_DMA_VCHAN_ID);
+ if (ret < 0)
+ RTE_LOG(ERR, USER1, "Failed to delete vchan!");
+
+ ret = rte_event_dma_adapter_free(TEST_ADAPTER_ID);
+ if (ret < 0)
+ RTE_LOG(ERR, USER1, "Failed to free adapter!");
+
+ dma_adapter_setup_done = 0;
+}
+
+static void
+dma_teardown(void)
+{
+ /* Free mbuf mempool */
+ if (params.src_mbuf_pool != NULL) {
+ RTE_LOG(DEBUG, USER1, "DMA_ADAPTER_SRC_MBUFPOOL count %u\n",
+ rte_mempool_avail_count(params.src_mbuf_pool));
+ rte_mempool_free(params.src_mbuf_pool);
+ params.src_mbuf_pool = NULL;
+ }
+
+ if (params.dst_mbuf_pool != NULL) {
+ RTE_LOG(DEBUG, USER1, "DMA_ADAPTER_DST_MBUFPOOL count %u\n",
+ rte_mempool_avail_count(params.dst_mbuf_pool));
+ rte_mempool_free(params.dst_mbuf_pool);
+ params.dst_mbuf_pool = NULL;
+ }
+
+ /* Free ops mempool */
+ if (params.op_mpool != NULL) {
+ RTE_LOG(DEBUG, USER1, "EVENT_DMA_OP_POOL count %u\n",
+ rte_mempool_avail_count(params.op_mpool));
+ rte_mempool_free(params.op_mpool);
+ params.op_mpool = NULL;
+ }
+}
+
+static void
+eventdev_teardown(void)
+{
+ rte_event_dev_stop(evdev);
+}
+
+static void
+testsuite_teardown(void)
+{
+ dma_adapter_teardown();
+ dma_teardown();
+ eventdev_teardown();
+}
+
+static struct unit_test_suite functional_testsuite = {
+ .suite_name = "Event dma adapter test suite",
+ .setup = testsuite_setup,
+ .teardown = testsuite_teardown,
+ .unit_test_cases = {
+
+ TEST_CASE_ST(NULL, test_dma_adapter_free, test_dma_adapter_create),
+
+ TEST_CASE_ST(test_dma_adapter_create, test_dma_adapter_free,
+ test_dma_adapter_vchan_add_del),
+
+ TEST_CASE_ST(test_dma_adapter_create, test_dma_adapter_free,
+ test_dma_adapter_stats),
+
+ TEST_CASE_ST(test_dma_adapter_create, test_dma_adapter_free,
+ test_dma_adapter_params),
+
+ TEST_CASE_ST(test_dma_adapter_conf_op_forward_mode, test_dma_adapter_stop,
+ test_with_op_forward_mode),
+
+ TEST_CASES_END() /**< NULL terminate unit test array */
+ }
+};
+
+static int
+test_event_dma_adapter(void)
+{
+ return unit_test_suite_runner(&functional_testsuite);
+}
+
+#endif /* !RTE_EXEC_ENV_WINDOWS */
+
+REGISTER_TEST_COMMAND(event_dma_adapter_autotest, test_event_dma_adapter);
--
2.25.1
^ permalink raw reply [flat|nested] 110+ messages in thread
* Re: [PATCH v5 12/12] app/test: add event DMA adapter auto-test
2023-09-28 10:36 ` [PATCH v5 12/12] app/test: add event DMA adapter auto-test Amit Prakash Shukla
@ 2023-09-29 7:20 ` Jerin Jacob
0 siblings, 0 replies; 110+ messages in thread
From: Jerin Jacob @ 2023-09-29 7:20 UTC (permalink / raw)
To: Amit Prakash Shukla
Cc: dev, jerinj, fengchengwen, kevin.laatz, bruce.richardson,
conor.walsh, vattunuru, g.singh, sachin.saxena, hemant.agrawal,
cheng1.jiang, ndabilpuram, anoobj, mb
On Fri, Sep 29, 2023 at 9:46 AM Amit Prakash Shukla
<amitprakashs@marvell.com> wrote:
>
> Added testsuite to test the dma adapter functionality.
> The testsuite detects event and DMA device capability
> and accordingly dma adapter is configured and modes are
> tested. Test command:
>
> <build dir>/app/test/dpdk-test event_dma_adapter_autotest
Use the below command with SW driver so that anyone can run it.
>
> Signed-off-by: Amit Prakash Shukla <amitprakashs@marvell.com>
sudo ./build/app/test/dpdk-test --vdev=dma_skeleton event_dma_adapter_autotest
There are failures with above as skelton dmadev does not support SG
and most of remaining drivers. So please change to following.
[for-main]dell[dpdk-next-eventdev] $ git diff
diff --git a/lib/eventdev/rte_event_dma_adapter.c
b/lib/eventdev/rte_event_dma_adapter.c
index 4899bc5d0f..bbdfd3daa6 100644
--- a/lib/eventdev/rte_event_dma_adapter.c
+++ b/lib/eventdev/rte_event_dma_adapter.c
@@ -256,8 +256,13 @@ edma_circular_buffer_flush_to_dma_dev(struct
event_dma_adapter *adapter,
for (i = 0; i < n; i++) {
op = bufp->op_buffer[*head];
- ret = rte_dma_copy_sg(dma_dev_id, vchan, op->src_seg,
op->dst_seg,
- op->nb_src, op->nb_dst, op->flags);
+ if (op->nb_src == 1 && op->nb_dst == 1)
+ ret = rte_dma_copy(dma_dev_id, vchan,
op->src_seg->addr, op->dst_seg->addr,
+ op->src_seg->length, op->flags);
+ else
+ ret = rte_dma_copy_sg(dma_dev_id, vchan,
op->src_seg, op->dst_seg,
+ op->nb_src, op->nb_dst, op->flags);
+
With above change all test cases are pasiing on skelton device.
[for-main]dell[dpdk-next-eventdev] $ sudo ./build/app/test/dpdk-test
--vdev=dma_skeleton event_dma_adapter_autotest
EAL: Detected CPU lcores: 56
EAL: Detected NUMA nodes: 2
EAL: Detected static linkage of DPDK
EAL: Multi-process socket /var/run/dpdk/rte/mp_socket
EAL: Selected IOVA mode 'VA'
skeldma_probe(): Create dma_skeleton dmadev with lcore-id -1
APP: HPET is not enabled, using TSC as default timer
RTE>>event_dma_adapter_autotest
+ ------------------------------------------------------- +
+ Test Suite : Event dma adapter test suite
+ ------------------------------------------------------- +
+ TestCase [ 0] : test_dma_adapter_create succeeded
+ TestCase [ 1] : test_dma_adapter_vchan_add_del succeeded
+------------------------------------------------------+
+ DMA adapter stats for instance 0:
+ Event port poll count 0x0
+ Event dequeue count 0x0
+ DMA dev enqueue count 0x0
+ DMA dev enqueue failed count 0x0
+ DMA dev dequeue count 0x0
+ Event enqueue count 0x0
+ Event enqueue retry count 0x0
+ Event enqueue fail count 0x0
+------------------------------------------------------+
+ TestCase [ 2] : test_dma_adapter_stats succeeded
+ TestCase [ 3] : test_dma_adapter_params succeeded
+------------------------------------------------------+
+ DMA adapter stats for instance 0:
+ Event port poll count 0xc5df
+ Event dequeue count 0x20
+ DMA dev enqueue count 0x20
+ DMA dev enqueue failed count 0x0
+ DMA dev dequeue count 0x20
+ Event enqueue count 0x20
+ Event enqueue retry count 0x0
+ Event enqueue fail count 0x0
+------------------------------------------------------+
+ TestCase [ 4] : test_with_op_forward_mode succeeded
EVENTDEV: rte_event_dev_stop() line 1427: Device with dev_id=0already stopped
+ ------------------------------------------------------- +
+ Test Suite Summary : Event dma adapter test suite
+ ------------------------------------------------------- +
+ Tests Total : 5
+ Tests Skipped : 0
+ Tests Executed : 5
+ Tests Unsupported: 0
+ Tests Passed : 5
+ Tests Failed : 0
+ ------------------------------------------------------- +
Test OK
RTE>>skeldma_remove(): Remove dma_skeleton dmadev
# Please fix the second warning by using rte_strscpy
[for-main]dell[dpdk-next-eventdev] $ ./devtools/checkpatches.sh -n 12
&& ./devtools/check-git-log.sh -n 12
### [PATCH] eventdev/dma: support adapter create and free
WARNING:MACRO_WITH_FLOW_CONTROL: Macros with flow control statements
should be avoided
#60: FILE: lib/eventdev/rte_event_dma_adapter.c:19:
+#define EVENT_DMA_ADAPTER_ID_VALID_OR_ERR_RET(id, retval) \
+ do { \
+ if (!edma_adapter_valid_id(id)) { \
+ RTE_EDEV_LOG_ERR("Invalid DMA adapter id = %d\n", id); \
+ return retval; \
+ } \
+ } while (0)
WARNING:STRCPY: Prefer strscpy over strcpy - see:
https://github.com/KSPP/linux/issues/88
#302: FILE: lib/eventdev/rte_event_dma_adapter.c:261:
+ strcpy(adapter->mem_name, name);
total: 0 errors, 2 warnings, 349 lines checked
Rest of the chages looks good to me. Good to merge next version.
^ permalink raw reply [flat|nested] 110+ messages in thread
* [PATCH v7 00/12] event DMA adapter library support
2023-09-28 16:49 ` [PATCH v6 00/12] event DMA adapter library support Amit Prakash Shukla
` (11 preceding siblings ...)
2023-09-28 16:49 ` [PATCH v6 12/12] app/test: add event DMA adapter auto-test Amit Prakash Shukla
@ 2023-09-29 8:12 ` Amit Prakash Shukla
2023-09-29 8:12 ` [PATCH v7 01/12] eventdev/dma: introduce DMA adapter Amit Prakash Shukla
` (12 more replies)
12 siblings, 13 replies; 110+ messages in thread
From: Amit Prakash Shukla @ 2023-09-29 8:12 UTC (permalink / raw)
Cc: dev, jerinj, fengchengwen, kevin.laatz, bruce.richardson,
conor.walsh, vattunuru, g.singh, sachin.saxena, hemant.agrawal,
cheng1.jiang, ndabilpuram, anoobj, mb, Amit Prakash Shukla
This series adds support for event DMA adapter library. API's defined
as part of this library can be used by the application for DMA transfer
of data using event based mechanism.
v7:
- Resolved review comments.
v6:
- Resolved review comments.
- Updated git commit message.
v5:
- Resolved review comments.
v4:
- Fixed compilation error.
v3:
- Resolved checkpatch warnings.
- Fixed compilation error on intel.
- Updated git commit message.
v2:
- Resolved review comments.
- Patch split into multiple patches.
Amit Prakash Shukla (12):
eventdev/dma: introduce DMA adapter
eventdev/dma: support adapter capabilities get
eventdev/dma: support adapter create and free
eventdev/dma: support vchan add and delete
eventdev/dma: support adapter service function
eventdev/dma: support adapter start and stop
eventdev/dma: support adapter service ID get
eventdev/dma: support adapter runtime params
eventdev/dma: support adapter stats
eventdev/dma: support adapter enqueue
eventdev/dma: support adapter event port get
app/test: add event DMA adapter auto-test
MAINTAINERS | 7 +
app/test/meson.build | 1 +
app/test/test_event_dma_adapter.c | 805 +++++++++
config/rte_config.h | 1 +
doc/api/doxy-api-index.md | 1 +
doc/guides/eventdevs/features/default.ini | 8 +
doc/guides/prog_guide/event_dma_adapter.rst | 264 +++
doc/guides/prog_guide/eventdev.rst | 8 +-
.../img/event_dma_adapter_op_forward.svg | 1086 +++++++++++++
.../img/event_dma_adapter_op_new.svg | 1079 +++++++++++++
doc/guides/prog_guide/index.rst | 1 +
doc/guides/rel_notes/release_23_11.rst | 5 +
lib/eventdev/eventdev_pmd.h | 171 +-
lib/eventdev/eventdev_private.c | 10 +
lib/eventdev/meson.build | 4 +-
lib/eventdev/rte_event_dma_adapter.c | 1434 +++++++++++++++++
lib/eventdev/rte_event_dma_adapter.h | 581 +++++++
lib/eventdev/rte_eventdev.c | 23 +
lib/eventdev/rte_eventdev.h | 44 +
lib/eventdev/rte_eventdev_core.h | 8 +-
lib/eventdev/version.map | 16 +
lib/meson.build | 2 +-
22 files changed, 5552 insertions(+), 7 deletions(-)
create mode 100644 app/test/test_event_dma_adapter.c
create mode 100644 doc/guides/prog_guide/event_dma_adapter.rst
create mode 100644 doc/guides/prog_guide/img/event_dma_adapter_op_forward.svg
create mode 100644 doc/guides/prog_guide/img/event_dma_adapter_op_new.svg
create mode 100644 lib/eventdev/rte_event_dma_adapter.c
create mode 100644 lib/eventdev/rte_event_dma_adapter.h
--
2.25.1
^ permalink raw reply [flat|nested] 110+ messages in thread
* [PATCH v7 01/12] eventdev/dma: introduce DMA adapter
2023-09-29 8:12 ` [PATCH v7 00/12] event DMA adapter library support Amit Prakash Shukla
@ 2023-09-29 8:12 ` Amit Prakash Shukla
2023-09-29 8:12 ` [PATCH v7 02/12] eventdev/dma: support adapter capabilities get Amit Prakash Shukla
` (11 subsequent siblings)
12 siblings, 0 replies; 110+ messages in thread
From: Amit Prakash Shukla @ 2023-09-29 8:12 UTC (permalink / raw)
To: Thomas Monjalon, Amit Prakash Shukla, Jerin Jacob
Cc: dev, fengchengwen, kevin.laatz, bruce.richardson, conor.walsh,
vattunuru, g.singh, sachin.saxena, hemant.agrawal, cheng1.jiang,
ndabilpuram, anoobj, mb
Introduce event dma adapter interface to transfer packets between
dma device and event device.
Signed-off-by: Amit Prakash Shukla <amitprakashs@marvell.com>
Acked-by: Jerin Jacob <jerinj@marvell.com>
---
MAINTAINERS | 6 +
doc/api/doxy-api-index.md | 1 +
doc/guides/eventdevs/features/default.ini | 8 +
doc/guides/prog_guide/event_dma_adapter.rst | 264 ++++
doc/guides/prog_guide/eventdev.rst | 8 +-
.../img/event_dma_adapter_op_forward.svg | 1086 +++++++++++++++++
.../img/event_dma_adapter_op_new.svg | 1079 ++++++++++++++++
doc/guides/prog_guide/index.rst | 1 +
doc/guides/rel_notes/release_23_11.rst | 5 +
lib/eventdev/eventdev_pmd.h | 171 ++-
lib/eventdev/eventdev_private.c | 10 +
lib/eventdev/meson.build | 1 +
lib/eventdev/rte_event_dma_adapter.h | 581 +++++++++
lib/eventdev/rte_eventdev.h | 44 +
lib/eventdev/rte_eventdev_core.h | 8 +-
lib/eventdev/version.map | 16 +
lib/meson.build | 2 +-
17 files changed, 3285 insertions(+), 6 deletions(-)
create mode 100644 doc/guides/prog_guide/event_dma_adapter.rst
create mode 100644 doc/guides/prog_guide/img/event_dma_adapter_op_forward.svg
create mode 100644 doc/guides/prog_guide/img/event_dma_adapter_op_new.svg
create mode 100644 lib/eventdev/rte_event_dma_adapter.h
diff --git a/MAINTAINERS b/MAINTAINERS
index a926155f26..4ebbbe8bb3 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -540,6 +540,12 @@ F: lib/eventdev/*crypto_adapter*
F: app/test/test_event_crypto_adapter.c
F: doc/guides/prog_guide/event_crypto_adapter.rst
+Eventdev DMA Adapter API
+M: Amit Prakash Shukla <amitprakashs@marvell.com>
+T: git://dpdk.org/next/dpdk-next-eventdev
+F: lib/eventdev/*dma_adapter*
+F: doc/guides/prog_guide/event_dma_adapter.rst
+
Raw device API
M: Sachin Saxena <sachin.saxena@oss.nxp.com>
M: Hemant Agrawal <hemant.agrawal@nxp.com>
diff --git a/doc/api/doxy-api-index.md b/doc/api/doxy-api-index.md
index fdeda13932..b7df7be4d9 100644
--- a/doc/api/doxy-api-index.md
+++ b/doc/api/doxy-api-index.md
@@ -29,6 +29,7 @@ The public API headers are grouped by topics:
[event_eth_tx_adapter](@ref rte_event_eth_tx_adapter.h),
[event_timer_adapter](@ref rte_event_timer_adapter.h),
[event_crypto_adapter](@ref rte_event_crypto_adapter.h),
+ [event_dma_adapter](@ref rte_event_dma_adapter.h),
[rawdev](@ref rte_rawdev.h),
[metrics](@ref rte_metrics.h),
[bitrate](@ref rte_bitrate.h),
diff --git a/doc/guides/eventdevs/features/default.ini b/doc/guides/eventdevs/features/default.ini
index 00360f60c6..73a52d915b 100644
--- a/doc/guides/eventdevs/features/default.ini
+++ b/doc/guides/eventdevs/features/default.ini
@@ -44,6 +44,14 @@ internal_port_op_fwd =
internal_port_qp_ev_bind =
session_private_data =
+;
+; Features of a default DMA adapter.
+;
+[DMA adapter Features]
+internal_port_op_new =
+internal_port_op_fwd =
+internal_port_vchan_ev_bind =
+
;
; Features of a default Timer adapter.
;
diff --git a/doc/guides/prog_guide/event_dma_adapter.rst b/doc/guides/prog_guide/event_dma_adapter.rst
new file mode 100644
index 0000000000..701e50d042
--- /dev/null
+++ b/doc/guides/prog_guide/event_dma_adapter.rst
@@ -0,0 +1,264 @@
+.. SPDX-License-Identifier: BSD-3-Clause
+ Copyright (c) 2023 Marvell.
+
+Event DMA Adapter Library
+=========================
+
+DPDK :doc:`Eventdev library <eventdev>` provides event driven programming model with features
+to schedule events. :doc:`DMA Device library <dmadev>` provides an interface to DMA poll mode
+drivers that support DMA operations. Event DMA Adapter is intended to bridge between the event
+device and the DMA device.
+
+Packet flow from DMA device to the event device can be accomplished using software and hardware
+based transfer mechanisms. The adapter queries an eventdev PMD to determine which mechanism to
+be used. The adapter uses an EAL service core function for software based packet transfer and
+uses the eventdev PMD functions to configure hardware based packet transfer between DMA device
+and the event device. DMA adapter uses a new event type called ``RTE_EVENT_TYPE_DMADEV`` to
+indicate the source of event.
+
+Application can choose to submit an DMA operation directly to an DMA device or send it to an DMA
+adapter via eventdev based on ``RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP_FWD`` capability. The
+first mode is known as the event new (``RTE_EVENT_DMA_ADAPTER_OP_NEW``) mode and the second as the
+event forward (``RTE_EVENT_DMA_ADAPTER_OP_FORWARD``) mode. Choice of mode can be specified while
+creating the adapter. In the former mode, it is the application's responsibility to enable
+ingress packet ordering. In the latter mode, it is the adapter's responsibility to enable
+ingress packet ordering.
+
+
+Adapter Modes
+-------------
+
+RTE_EVENT_DMA_ADAPTER_OP_NEW mode
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+In the ``RTE_EVENT_DMA_ADAPTER_OP_NEW`` mode, application submits DMA operations directly to an DMA
+device. The adapter then dequeues DMA completions from the DMA device and enqueues them as events
+to the event device. This mode does not ensure ingress ordering as the application directly
+enqueues to the dmadev without going through DMA/atomic stage. In this mode, events dequeued
+from the adapter are treated as new events. The application has to specify event information
+(response information) which is needed to enqueue an event after the DMA operation is completed.
+
+.. _figure_event_dma_adapter_op_new:
+
+.. figure:: img/event_dma_adapter_op_new.*
+
+ Working model of ``RTE_EVENT_DMA_ADAPTER_OP_NEW`` mode
+
+
+RTE_EVENT_DMA_ADAPTER_OP_FORWARD mode
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+In the ``RTE_EVENT_DMA_ADAPTER_OP_FORWARD`` mode, if the event PMD and DMA PMD supports internal
+event port (``RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP_FWD``), the application should use
+``rte_event_dma_adapter_enqueue()`` API to enqueue DMA operations as events to DMA adapter. If
+not, application retrieves DMA adapter's event port using ``rte_event_dma_adapter_event_port_get()``
+API, links its event queue to this port and starts enqueuing DMA operations as events to eventdev
+using ``rte_event_enqueue_burst()``. The adapter then dequeues the events and submits the DMA
+operations to the dmadev. After the DMA operation is complete, the adapter enqueues events to the
+event device.
+
+Applications can use this mode when ingress packet ordering is needed. In this mode, events
+dequeued from the adapter will be treated as forwarded events. Application has to specify event
+information (response information) needed to enqueue the event after the DMA operation has
+completed.
+
+.. _figure_event_dma_adapter_op_forward:
+
+.. figure:: img/event_dma_adapter_op_forward.*
+
+ Working model of ``RTE_EVENT_DMA_ADAPTER_OP_FORWARD`` mode
+
+
+API Overview
+------------
+
+This section has a brief introduction to the event DMA adapter APIs. The application is expected
+to create an adapter which is associated with a single eventdev, then add dmadev and vchan to the
+adapter instance.
+
+
+Create an adapter instance
+~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+An adapter instance is created using ``rte_event_dma_adapter_create()``. This function is called
+with event device to be associated with the adapter and port configuration for the adapter to
+setup an event port (if the adapter needs to use a service function).
+
+Adapter can be started in ``RTE_EVENT_DMA_ADAPTER_OP_NEW`` or ``RTE_EVENT_DMA_ADAPTER_OP_FORWARD``
+mode.
+
+.. code-block:: c
+
+ enum rte_event_dma_adapter_mode mode;
+ struct rte_event_dev_info dev_info;
+ struct rte_event_port_conf conf;
+ uint8_t evdev_id;
+ uint8_t dma_id;
+ int ret;
+
+ ret = rte_event_dev_info_get(dma_id, &dev_info);
+
+ conf.new_event_threshold = dev_info.max_num_events;
+ conf.dequeue_depth = dev_info.max_event_port_dequeue_depth;
+ conf.enqueue_depth = dev_info.max_event_port_enqueue_depth;
+ mode = RTE_EVENT_DMA_ADAPTER_OP_FORWARD;
+ ret = rte_event_dma_adapter_create(dma_id, evdev_id, &conf, mode);
+
+
+``rte_event_dma_adapter_create_ext()`` function can be used by the application to have a finer
+control on eventdev port allocation and setup. The ``rte_event_dma_adapter_create_ext()``
+function is passed a callback function. The callback function is invoked if the adapter creates
+a service function and uses an event port for it. The callback is expected to fill the
+``struct rte_event_dma_adapter_conf`` structure passed to it.
+
+In the ``RTE_EVENT_DMA_ADAPTER_OP_FORWARD`` mode, if the event PMD and DMA PMD supports internal
+event port (``RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP_FWD``), events with DMA operations should
+be enqueued to the DMA adapter using ``rte_event_dma_adapter_enqueue()`` API. If not, the event port
+created by the adapter can be retrieved using ``rte_event_dma_adapter_event_port_get()`` API. An
+application can use this event port to link with an event queue, on which it enqueues events
+towards the DMA adapter using ``rte_event_enqueue_burst()``.
+
+.. code-block:: c
+
+ uint8_t dma_adpt_id, evdev_id, dma_dev_id, dma_ev_port_id, app_qid;
+ struct rte_event ev;
+ uint32_t cap;
+ int ret;
+
+ // Fill in event info and update event_ptr with rte_dma_op
+ memset(&ev, 0, sizeof(ev));
+ .
+ .
+ ev.event_ptr = op;
+
+ ret = rte_event_dma_adapter_caps_get(evdev_id, dma_dev_id, &cap);
+ if (cap & RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP_FWD) {
+ ret = rte_event_dma_adapter_enqueue(evdev_id, app_ev_port_id, ev, nb_events);
+ } else {
+ ret = rte_event_dma_adapter_event_port_get(dma_adpt_id, &dma_ev_port_id);
+ ret = rte_event_queue_setup(evdev_id, app_qid, NULL);
+ ret = rte_event_port_link(evdev_id, dma_ev_port_id, &app_qid, NULL, 1);
+ ev.queue_id = app_qid;
+ ret = rte_event_enqueue_burst(evdev_id, app_ev_port_id, ev, nb_events);
+ }
+
+
+Event device configuration for service based adapter
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+When ``rte_event_dma_adapter_create()`` is used for creating adapter instance,
+``rte_event_dev_config::nb_event_ports`` is automatically incremented, and event device is
+reconfigured with additional event port during service initialization. This event device
+reconfigure logic also increments the ``rte_event_dev_config::nb_single_link_event_port_queues``
+parameter if the adapter event port config is of type ``RTE_EVENT_PORT_CFG_SINGLE_LINK``.
+
+Applications using this mode of adapter creation need not configure the event device with
+``rte_event_dev_config::nb_event_ports`` and
+``rte_event_dev_config::nb_single_link_event_port_queues`` parameters required for DMA adapter when
+the adapter is created using the above-mentioned API.
+
+
+Querying adapter capabilities
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+The ``rte_event_dma_adapter_caps_get()`` function allows the application to query the adapter
+capabilities for an eventdev and dmadev combination. This API provides whether dmadev and eventdev
+are connected using internal HW port or not.
+
+.. code-block:: c
+
+ rte_event_dma_adapter_caps_get(dev_id, dma_dev_id, &cap);
+
+
+Adding vchan to the adapter instance
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+dmadev device id and vchan are configured using dmadev APIs. For more information
+see :doc:`here <dmadev>`.
+
+.. code-block:: c
+
+ struct rte_dma_vchan_conf vchan_conf;
+ struct rte_dma_conf dev_conf;
+ uint8_t dev_id = 0;
+ uint16_t vchan = 0;
+
+ rte_dma_configure(dev_id, &dev_conf);
+ rte_dma_vchan_setup(dev_id, vchan, &vchan_conf);
+
+These dmadev id and vchan are added to the instance using the
+``rte_event_dma_adapter_vchan_add()`` API. The same is removed using
+``rte_event_dma_adapter_vchan_del()`` API. If hardware supports
+``RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_VCHAN_EV_BIND`` capability, event information must be passed to the add API.
+
+.. code-block:: c
+
+ uint32_t cap;
+ int ret;
+
+ ret = rte_event_dma_adapter_caps_get(evdev_id, dma_dev_id, &cap);
+ if (cap & RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_VCHAN_EV_BIND) {
+ struct rte_event event;
+
+ rte_event_dma_adapter_vchan_add(id, dma_dev_id, vchan, &conf);
+ } else
+ rte_event_dma_adapter_vchan_add(id, dma_dev_id, vchan, NULL);
+
+
+Configuring service function
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+If the adapter uses a service function, the application is required to assign a service core to
+the service function as show below.
+
+.. code-block:: c
+
+ uint32_t service_id;
+
+ if (rte_event_dma_adapter_service_id_get(dma_id, &service_id) == 0)
+ rte_service_map_lcore_set(service_id, CORE_ID);
+
+
+Set event response information
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+In the RTE_EVENT_DMA_ADAPTER_OP_FORWARD / RTE_EVENT_DMA_ADAPTER_OP_NEW mode, the application
+specifies the dmadev ID and vchan ID in ``struct rte_event_dma_adapter_op`` and the event
+information (response information) needed to enqueue an event after the DMA operation has
+completed. The response information is specified in ``struct rte_event`` and appended to the
+``struct rte_event_dma_adapter_op``.
+
+
+Start the adapter instance
+~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+The application calls ``rte_event_dma_adapter_start()`` to start the adapter. This function calls
+the start callbacks of the eventdev PMDs for hardware based eventdev-dmadev connections and
+``rte_service_run_state_set()`` to enable the service function if one exists.
+
+.. code-block:: c
+
+ rte_event_dma_adapter_start(id);
+
+.. Note::
+
+ The eventdev to which the event_dma_adapter is connected should be started before calling
+ rte_event_dma_adapter_start().
+
+
+Get adapter statistics
+~~~~~~~~~~~~~~~~~~~~~~
+
+The ``rte_event_dma_adapter_stats_get()`` function reports counters defined in struct
+``rte_event_dma_adapter_stats``. The received packet and enqueued event counts are a sum of the
+counts from the eventdev PMD callbacks if the callback is supported, and the counts maintained by
+the service function, if one exists.
+
+Set/Get adapter runtime configuration parameters
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+The runtime configuration parameters of adapter can be set/get using
+``rte_event_dma_adapter_runtime_params_set()`` and
+``rte_event_dma_adapter_runtime_params_get()`` respectively.
+The parameters that can be set/get are defined in
+``struct rte_event_dma_adapter_runtime_params``.
diff --git a/doc/guides/prog_guide/eventdev.rst b/doc/guides/prog_guide/eventdev.rst
index 2c83176846..ff55115d0d 100644
--- a/doc/guides/prog_guide/eventdev.rst
+++ b/doc/guides/prog_guide/eventdev.rst
@@ -333,7 +333,8 @@ eventdev.
.. Note::
EventDev needs to be started before starting the event producers such
- as event_eth_rx_adapter, event_timer_adapter and event_crypto_adapter.
+ as event_eth_rx_adapter, event_timer_adapter, event_crypto_adapter and
+ event_dma_adapter.
Ingress of New Events
~~~~~~~~~~~~~~~~~~~~~
@@ -445,8 +446,9 @@ using ``rte_event_dev_stop_flush_callback_register()`` function.
.. Note::
The event producers such as ``event_eth_rx_adapter``,
- ``event_timer_adapter`` and ``event_crypto_adapter``
- need to be stopped before stopping the event device.
+ ``event_timer_adapter``, ``event_crypto_adapter`` and
+ ``event_dma_adapter`` need to be stopped before stopping
+ the event device.
Summary
-------
diff --git a/doc/guides/prog_guide/img/event_dma_adapter_op_forward.svg b/doc/guides/prog_guide/img/event_dma_adapter_op_forward.svg
new file mode 100644
index 0000000000..b7fe1fecf2
--- /dev/null
+++ b/doc/guides/prog_guide/img/event_dma_adapter_op_forward.svg
@@ -0,0 +1,1086 @@
+<?xml version="1.0" encoding="UTF-8" standalone="no"?>
+<!-- SPDX-License-Identifier: BSD-3-Clause -->
+<!-- Copyright (c) 2023 Marvell. -->
+<!-- Created with Inkscape (http://www.inkscape.org/) -->
+
+<svg
+ width="720px"
+ height="486px"
+ id="svg13237"
+ version="1.1"
+ inkscape:version="1.2.1 (9c6d41e410, 2022-07-14)"
+ sodipodi:docname="event_dma_adapter_op_forward.svg"
+ xmlns:inkscape="http://www.inkscape.org/namespaces/inkscape"
+ xmlns:sodipodi="http://sodipodi.sourceforge.net/DTD/sodipodi-0.dtd"
+ xmlns="http://www.w3.org/2000/svg"
+ xmlns:svg="http://www.w3.org/2000/svg"
+ xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#"
+ xmlns:cc="http://creativecommons.org/ns#"
+ xmlns:dc="http://purl.org/dc/elements/1.1/"
+ xmlns:v="http://schemas.microsoft.com/visio/2003/SVGExtensions/">
+ <defs
+ id="defs13239">
+ <marker
+ inkscape:stockid="Arrow1Sstart"
+ orient="auto"
+ refY="0.0"
+ refX="0.0"
+ id="Arrow1Sstart"
+ style="overflow:visible">
+ <path
+ id="path8416"
+ d="M 0.0,0.0 L 5.0,-5.0 L -12.5,0.0 L 5.0,5.0 L 0.0,0.0 z "
+ style="fill-rule:evenodd;stroke:#000000;stroke-width:1.0pt"
+ transform="scale(0.2) translate(6,0)" />
+ </marker>
+ <marker
+ inkscape:stockid="Arrow1Send"
+ orient="auto"
+ refY="0.0"
+ refX="0.0"
+ id="Arrow1Send"
+ style="overflow:visible;">
+ <path
+ id="path8419"
+ d="M 0.0,0.0 L 5.0,-5.0 L -12.5,0.0 L 5.0,5.0 L 0.0,0.0 z "
+ style="fill-rule:evenodd;stroke:#000000;stroke-width:1.0pt;"
+ transform="scale(0.2) rotate(180) translate(6,0)" />
+ </marker>
+ <marker
+ inkscape:stockid="DiamondL"
+ orient="auto"
+ refY="0.0"
+ refX="0.0"
+ id="DiamondL"
+ style="overflow:visible">
+ <path
+ id="path8483"
+ d="M 0,-7.0710768 L -7.0710894,0 L 0,7.0710589 L 7.0710462,0 L 0,-7.0710768 z "
+ style="fill-rule:evenodd;stroke:#000000;stroke-width:1.0pt"
+ transform="scale(0.8)" />
+ </marker>
+ <marker
+ inkscape:stockid="DotL"
+ orient="auto"
+ refY="0.0"
+ refX="0.0"
+ id="DotL"
+ style="overflow:visible">
+ <path
+ id="path8465"
+ d="M -2.5,-1.0 C -2.5,1.7600000 -4.7400000,4.0 -7.5,4.0 C -10.260000,4.0 -12.5,1.7600000 -12.5,-1.0 C -12.5,-3.7600000 -10.260000,-6.0 -7.5,-6.0 C -4.7400000,-6.0 -2.5,-3.7600000 -2.5,-1.0 z "
+ style="fill-rule:evenodd;stroke:#000000;stroke-width:1.0pt"
+ transform="scale(0.8) translate(7.4, 1)" />
+ </marker>
+ <marker
+ inkscape:stockid="SquareL"
+ orient="auto"
+ refY="0.0"
+ refX="0.0"
+ id="SquareL"
+ style="overflow:visible">
+ <path
+ id="path8474"
+ d="M -5.0,-5.0 L -5.0,5.0 L 5.0,5.0 L 5.0,-5.0 L -5.0,-5.0 z "
+ style="fill-rule:evenodd;stroke:#000000;stroke-width:1.0pt"
+ transform="scale(0.8)" />
+ </marker>
+ <marker
+ inkscape:stockid="TriangleOutL"
+ orient="auto"
+ refY="0.0"
+ refX="0.0"
+ id="TriangleOutL"
+ style="overflow:visible">
+ <path
+ id="path8546"
+ d="M 5.77,0.0 L -2.88,5.0 L -2.88,-5.0 L 5.77,0.0 z "
+ style="fill-rule:evenodd;stroke:#000000;stroke-width:1.0pt"
+ transform="scale(0.8)" />
+ </marker>
+ <marker
+ inkscape:stockid="Arrow1Lstart"
+ orient="auto"
+ refY="0.0"
+ refX="0.0"
+ id="Arrow1Lstart"
+ style="overflow:visible">
+ <path
+ id="path8404"
+ d="M 0.0,0.0 L 5.0,-5.0 L -12.5,0.0 L 5.0,5.0 L 0.0,0.0 z "
+ style="fill-rule:evenodd;stroke:#000000;stroke-width:1.0pt"
+ transform="scale(0.8) translate(12.5,0)" />
+ </marker>
+ <marker
+ inkscape:stockid="Arrow1Mend"
+ orient="auto"
+ refY="0.0"
+ refX="0.0"
+ id="Arrow1Mend"
+ style="overflow:visible;">
+ <path
+ id="path8413"
+ d="M 0.0,0.0 L 5.0,-5.0 L -12.5,0.0 L 5.0,5.0 L 0.0,0.0 z "
+ style="fill-rule:evenodd;stroke:#000000;stroke-width:1.0pt;"
+ transform="scale(0.4) rotate(180) translate(10,0)" />
+ </marker>
+ <marker
+ inkscape:stockid="Arrow2Lend"
+ orient="auto"
+ refY="0.0"
+ refX="0.0"
+ id="Arrow2Lend"
+ style="overflow:visible;">
+ <path
+ id="path8425"
+ style="fill-rule:evenodd;stroke-width:0.62500000;stroke-linejoin:round;"
+ d="M 8.7185878,4.0337352 L -2.2072895,0.016013256 L 8.7185884,-4.0017078 C 6.9730900,-1.6296469 6.9831476,1.6157441 8.7185878,4.0337352 z "
+ transform="scale(1.1) rotate(180) translate(1,0)" />
+ </marker>
+ <marker
+ inkscape:stockid="Arrow1Lend"
+ orient="auto"
+ refY="0.0"
+ refX="0.0"
+ id="Arrow1Lend"
+ style="overflow:visible;">
+ <path
+ id="path8407"
+ d="M 0.0,0.0 L 5.0,-5.0 L -12.5,0.0 L 5.0,5.0 L 0.0,0.0 z "
+ style="fill-rule:evenodd;stroke:#000000;stroke-width:1.0pt;"
+ transform="scale(0.8) rotate(180) translate(12.5,0)" />
+ </marker>
+ <filter
+ id="filter_2"
+ color-interpolation-filters="sRGB">
+ <feGaussianBlur
+ stdDeviation="2"
+ id="feGaussianBlur15" />
+ </filter>
+ <filter
+ id="filter_2-3"
+ color-interpolation-filters="sRGB"
+ x="-0.086178862"
+ y="-0.086178862"
+ width="1.1723577"
+ height="1.1723577">
+ <feGaussianBlur
+ stdDeviation="2"
+ id="feGaussianBlur15-1" />
+ </filter>
+ <filter
+ id="filter_2-0"
+ color-interpolation-filters="sRGB">
+ <feGaussianBlur
+ stdDeviation="2"
+ id="feGaussianBlur15-7" />
+ </filter>
+ <filter
+ id="filter_2-0-8"
+ color-interpolation-filters="sRGB">
+ <feGaussianBlur
+ stdDeviation="2"
+ id="feGaussianBlur15-7-7" />
+ </filter>
+ <filter
+ id="filter_2-3-9"
+ color-interpolation-filters="sRGB">
+ <feGaussianBlur
+ stdDeviation="2"
+ id="feGaussianBlur15-1-6" />
+ </filter>
+ <filter
+ id="filter_2-3-6"
+ color-interpolation-filters="sRGB">
+ <feGaussianBlur
+ stdDeviation="2"
+ id="feGaussianBlur15-1-63" />
+ </filter>
+ <filter
+ id="filter_2-3-91"
+ color-interpolation-filters="sRGB"
+ x="-0.086178862"
+ y="-0.086178862"
+ width="1.1723577"
+ height="1.1723577">
+ <feGaussianBlur
+ stdDeviation="2"
+ id="feGaussianBlur15-1-3" />
+ </filter>
+ <marker
+ inkscape:stockid="Arrow1Lend"
+ orient="auto"
+ refY="0"
+ refX="0"
+ id="Arrow1Lend-5"
+ style="overflow:visible">
+ <path
+ inkscape:connector-curvature="0"
+ id="path8407-3"
+ d="M 0,0 5,-5 -12.5,0 5,5 0,0 z"
+ style="fill-rule:evenodd;stroke:#000000;stroke-width:1pt"
+ transform="matrix(-0.8,0,0,-0.8,-10,0)" />
+ </marker>
+ <marker
+ inkscape:stockid="Arrow1Lend"
+ orient="auto"
+ refY="0"
+ refX="0"
+ id="Arrow1Lend-6"
+ style="overflow:visible">
+ <path
+ inkscape:connector-curvature="0"
+ id="path8407-0"
+ d="M 0,0 5,-5 -12.5,0 5,5 0,0 z"
+ style="fill-rule:evenodd;stroke:#000000;stroke-width:1pt"
+ transform="matrix(-0.8,0,0,-0.8,-10,0)" />
+ </marker>
+ <marker
+ inkscape:stockid="Arrow1Lstart"
+ orient="auto"
+ refY="0"
+ refX="0"
+ id="Arrow1Lstart-7"
+ style="overflow:visible">
+ <path
+ inkscape:connector-curvature="0"
+ id="path8404-0"
+ d="M 0,0 5,-5 -12.5,0 5,5 0,0 z"
+ style="fill-rule:evenodd;stroke:#000000;stroke-width:1pt"
+ transform="matrix(0.8,0,0,0.8,10,0)" />
+ </marker>
+ <marker
+ inkscape:stockid="Arrow1Lend"
+ orient="auto"
+ refY="0"
+ refX="0"
+ id="Arrow1Lend-51"
+ style="overflow:visible">
+ <path
+ inkscape:connector-curvature="0"
+ id="path8407-1"
+ d="M 0,0 5,-5 -12.5,0 5,5 0,0 z"
+ style="fill-rule:evenodd;stroke:#000000;stroke-width:1pt"
+ transform="matrix(-0.8,0,0,-0.8,-10,0)" />
+ </marker>
+ <marker
+ inkscape:stockid="Arrow1Lend"
+ orient="auto"
+ refY="0"
+ refX="0"
+ id="Arrow1Lend-3"
+ style="overflow:visible">
+ <path
+ inkscape:connector-curvature="0"
+ id="path8407-6"
+ d="M 0,0 5,-5 -12.5,0 5,5 0,0 z"
+ style="fill-rule:evenodd;stroke:#000000;stroke-width:1pt"
+ transform="matrix(-0.8,0,0,-0.8,-10,0)" />
+ </marker>
+ <marker
+ inkscape:stockid="Arrow1Lend"
+ orient="auto"
+ refY="0"
+ refX="0"
+ id="Arrow1Lend-62"
+ style="overflow:visible">
+ <path
+ inkscape:connector-curvature="0"
+ id="path8407-9"
+ d="M 0,0 5,-5 -12.5,0 5,5 0,0 z"
+ style="fill-rule:evenodd;stroke:#000000;stroke-width:1pt"
+ transform="matrix(-0.8,0,0,-0.8,-10,0)" />
+ </marker>
+ <marker
+ inkscape:stockid="Arrow1Lend"
+ orient="auto"
+ refY="0"
+ refX="0"
+ id="Arrow1Lend-2"
+ style="overflow:visible">
+ <path
+ inkscape:connector-curvature="0"
+ id="path8407-7"
+ d="M 0,0 5,-5 -12.5,0 5,5 0,0 z"
+ style="fill-rule:evenodd;stroke:#000000;stroke-width:1pt"
+ transform="matrix(-0.8,0,0,-0.8,-10,0)" />
+ </marker>
+ <marker
+ inkscape:stockid="Arrow1Lstart"
+ orient="auto"
+ refY="0"
+ refX="0"
+ id="Arrow1Lstart-7-9"
+ style="overflow:visible">
+ <path
+ inkscape:connector-curvature="0"
+ id="path8404-0-3"
+ d="M 0,0 5,-5 -12.5,0 5,5 0,0 z"
+ style="fill-rule:evenodd;stroke:#000000;stroke-width:1pt"
+ transform="matrix(0.8,0,0,0.8,10,0)" />
+ </marker>
+ <filter
+ id="filter_2-3-6-1"
+ color-interpolation-filters="sRGB">
+ <feGaussianBlur
+ stdDeviation="2"
+ id="feGaussianBlur15-1-63-8" />
+ </filter>
+ <filter
+ id="filter_2-3-92"
+ color-interpolation-filters="sRGB"
+ x="-0.086178862"
+ y="-0.086178862"
+ width="1.1723577"
+ height="1.1723577">
+ <feGaussianBlur
+ stdDeviation="2"
+ id="feGaussianBlur15-1-2" />
+ </filter>
+ <filter
+ id="filter_2-3-94"
+ color-interpolation-filters="sRGB"
+ x="-0.086178862"
+ y="-0.086178862"
+ width="1.1723577"
+ height="1.1723577">
+ <feGaussianBlur
+ stdDeviation="2"
+ id="feGaussianBlur15-1-7" />
+ </filter>
+ <marker
+ inkscape:stockid="Arrow1Lstart"
+ orient="auto"
+ refY="0"
+ refX="0"
+ id="Arrow1Lstart-7-6"
+ style="overflow:visible">
+ <path
+ inkscape:connector-curvature="0"
+ id="path8404-0-1"
+ d="M 0,0 5,-5 -12.5,0 5,5 0,0 z"
+ style="fill-rule:evenodd;stroke:#000000;stroke-width:1pt"
+ transform="matrix(0.8,0,0,0.8,10,0)" />
+ </marker>
+ <marker
+ inkscape:stockid="Arrow1Lend"
+ orient="auto"
+ refY="0"
+ refX="0"
+ id="Arrow1Lend-55"
+ style="overflow:visible">
+ <path
+ inkscape:connector-curvature="0"
+ id="path8407-4"
+ d="M 0,0 5,-5 -12.5,0 5,5 0,0 z"
+ style="fill-rule:evenodd;stroke:#000000;stroke-width:1pt"
+ transform="matrix(-0.8,0,0,-0.8,-10,0)" />
+ </marker>
+ </defs>
+ <sodipodi:namedview
+ id="base"
+ pagecolor="#ffffff"
+ bordercolor="#666666"
+ borderopacity="1.0"
+ inkscape:pageopacity="0.0"
+ inkscape:pageshadow="2"
+ inkscape:zoom="1"
+ inkscape:cx="233.5"
+ inkscape:cy="288"
+ inkscape:document-units="px"
+ inkscape:current-layer="layer1"
+ showgrid="false"
+ inkscape:window-width="1920"
+ inkscape:window-height="1017"
+ inkscape:window-x="-8"
+ inkscape:window-y="-8"
+ inkscape:window-maximized="1"
+ inkscape:snap-nodes="false"
+ inkscape:showpageshadow="2"
+ inkscape:pagecheckerboard="0"
+ inkscape:deskcolor="#d1d1d1">
+ <inkscape:grid
+ type="xygrid"
+ id="grid13454" />
+ </sodipodi:namedview>
+ <metadata
+ id="metadata13242">
+ <rdf:RDF>
+ <cc:Work
+ rdf:about="">
+ <dc:format>image/svg+xml</dc:format>
+ <dc:type
+ rdf:resource="http://purl.org/dc/dcmitype/StillImage" />
+ <dc:title />
+ </cc:Work>
+ </rdf:RDF>
+ </metadata>
+ <g
+ id="layer1"
+ inkscape:label="Layer 1"
+ inkscape:groupmode="layer">
+ <g
+ style="font-size:12px;fill:none;stroke-linecap:square;stroke-miterlimit:3;overflow:visible"
+ id="shape1-1-2-4"
+ v:mID="1"
+ v:groupContext="shape"
+ transform="matrix(2.1604167,0,0,1.5671361,88.874699,-812.39909)">
+ <title
+ id="title22-7-5">Square</title>
+ <desc
+ id="desc24-7-8">Atomic Queue #1</desc>
+ <v:userDefs>
+ <v:ud
+ v:nameU="visVersion"
+ v:val="VT0(15):26" />
+ </v:userDefs>
+ <v:textBlock
+ v:margins="rect(4,4,4,4)" />
+ <v:textRect
+ cx="30.75"
+ cy="581.25"
+ width="61.5"
+ height="61.5" />
+ <g
+ id="shadow1-2-9-5"
+ v:groupContext="shadow"
+ v:shadowOffsetX="0.345598"
+ v:shadowOffsetY="-1.97279"
+ v:shadowType="1"
+ transform="translate(0.345598,1.97279)"
+ class="st1"
+ style="visibility:visible">
+ <rect
+ x="0"
+ y="550.5"
+ width="61.5"
+ height="61.5"
+ class="st2"
+ id="rect27-8-7"
+ style="fill:#5b9bd5;fill-opacity:0.22000002;stroke:#5b9bd5;stroke-opacity:0.22000002;filter:url(#filter_2-3-91)" />
+ </g>
+ <g
+ id="g13515-33">
+ <g
+ id="g13534-8">
+ <rect
+ x="0"
+ y="550.5"
+ width="61.5"
+ height="61.5"
+ class="st3"
+ id="rect29-1-95"
+ style="fill:#5b9bd5;stroke:#c7c8c8;stroke-width:0.25" />
+ </g>
+ </g>
+ </g>
+ <path
+ style="fill:none;stroke:#000000;stroke-width:0.712265;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:4;stroke-dasharray:none;stroke-opacity:1;marker-start:url(#Arrow1Lstart-7);marker-end:none"
+ d="M 312.28671,240.74335 H 227.99897"
+ id="path17209"
+ inkscape:connector-type="orthogonal"
+ inkscape:connector-curvature="0" />
+ <path
+ style="fill:none;stroke:#000000;stroke-width:0.718986px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1;marker-mid:none;marker-end:url(#Arrow1Lend)"
+ d="m 221.6484,77.57125 h 94.28101"
+ id="path17209-8"
+ inkscape:connector-type="orthogonal"
+ inkscape:connector-curvature="0" />
+ <g
+ style="font-size:12px;fill:none;stroke-linecap:square;stroke-miterlimit:3;overflow:visible"
+ id="shape1-1-2"
+ v:mID="1"
+ v:groupContext="shape"
+ transform="matrix(2.1604167,0,0,1.5671361,314.24227,-811.89589)">
+ <title
+ id="title22-7">Square</title>
+ <desc
+ id="desc24-7">Atomic Queue #1</desc>
+ <v:userDefs>
+ <v:ud
+ v:nameU="visVersion"
+ v:val="VT0(15):26" />
+ </v:userDefs>
+ <v:textBlock
+ v:margins="rect(4,4,4,4)" />
+ <v:textRect
+ cx="30.75"
+ cy="581.25"
+ width="61.5"
+ height="61.5" />
+ <g
+ id="shadow1-2-9"
+ v:groupContext="shadow"
+ v:shadowOffsetX="0.345598"
+ v:shadowOffsetY="-1.97279"
+ v:shadowType="1"
+ transform="translate(0.345598,1.97279)"
+ class="st1"
+ style="visibility:visible">
+ <rect
+ x="0"
+ y="550.5"
+ width="61.5"
+ height="61.5"
+ class="st2"
+ id="rect27-8"
+ style="fill:#5b9bd5;fill-opacity:0.22000002;stroke:#5b9bd5;stroke-opacity:0.22000002;filter:url(#filter_2-3)" />
+ </g>
+ <g
+ id="g13515">
+ <g
+ id="g13534">
+ <rect
+ x="0"
+ y="550.5"
+ width="61.5"
+ height="61.5"
+ class="st3"
+ id="rect29-1"
+ style="fill:#5b9bd5;stroke:#c7c8c8;stroke-width:0.25" />
+ </g>
+ </g>
+ </g>
+ <path
+ style="fill:none;stroke:#000000;stroke-width:0.724714;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:4;stroke-dasharray:none;stroke-opacity:1;marker-start:url(#Arrow1Lstart);marker-end:none"
+ d="M 89.025329,74.39932 H 24.750043"
+ id="path17209-3"
+ inkscape:connector-type="orthogonal"
+ inkscape:connector-curvature="0" />
+ <path
+ transform="matrix(0.73232502,0,0,0.75477602,-4.325033,28.642983)"
+ sodipodi:type="arc"
+ style="fill:#539de6;fill-opacity:1;stroke:#0000ea;stroke-width:1;stroke-linecap:square;stroke-miterlimit:4;stroke-opacity:1;stroke-dasharray:none"
+ id="path29161-3"
+ sodipodi:cx="371"
+ sodipodi:cy="64.5"
+ sodipodi:rx="17"
+ sodipodi:ry="15.5"
+ d="m 388,64.5 a 17,15.5 0 1 1 -34,0 17,15.5 0 1 1 34,0 z" />
+ <path
+ transform="matrix(0.73232502,0,0,0.75477602,-1.93108,192.80833)"
+ sodipodi:type="arc"
+ style="fill:#539de6;fill-opacity:1;stroke:#0000ea;stroke-width:1;stroke-linecap:square;stroke-miterlimit:4;stroke-opacity:1;stroke-dasharray:none"
+ id="path29161-1"
+ sodipodi:cx="371"
+ sodipodi:cy="64.5"
+ sodipodi:rx="17"
+ sodipodi:ry="15.5"
+ d="m 388,64.5 a 17,15.5 0 1 1 -34,0 17,15.5 0 1 1 34,0 z" />
+ <path
+ style="fill:none;stroke:#000000;stroke-width:0.751412;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:4;stroke-dasharray:none;stroke-opacity:1;marker-start:url(#Arrow1Lstart-7);marker-end:none"
+ d="M 18.763392,120.7432 H 87.758545"
+ id="path17209-3-0"
+ inkscape:connector-type="orthogonal"
+ inkscape:connector-curvature="0" />
+ <path
+ sodipodi:type="arc"
+ style="fill:#539de6;fill-opacity:1;stroke:#0000ea;stroke-width:1;stroke-linecap:square;stroke-miterlimit:4;stroke-opacity:1;stroke-dasharray:none"
+ id="path29161"
+ sodipodi:cx="371"
+ sodipodi:cy="64.5"
+ sodipodi:rx="17"
+ sodipodi:ry="15.5"
+ d="m 388,64.5 a 17,15.5 0 1 1 -34,0 17,15.5 0 1 1 34,0 z"
+ transform="matrix(0.73232502,0,0,0.75477602,-218.16394,72.68276)" />
+ <path
+ sodipodi:type="arc"
+ style="fill:#539de6;fill-opacity:1;stroke:#0000ea;stroke-width:1;stroke-linecap:square;stroke-miterlimit:4;stroke-opacity:1;stroke-dasharray:none"
+ id="path29161-2"
+ sodipodi:cx="371"
+ sodipodi:cy="64.5"
+ sodipodi:rx="17"
+ sodipodi:ry="15.5"
+ d="m 388,64.5 a 17,15.5 0 1 1 -34,0 17,15.5 0 1 1 34,0 z"
+ transform="matrix(0.73232502,0,0,0.75477602,-217.40136,26.716271)" />
+ <g
+ id="g29167-4"
+ transform="matrix(0.73232502,0,0,0.75477602,-217.31662,28.007562)">
+ <text
+ id="text29163-9"
+ y="70"
+ x="365"
+ style="font-style:normal;font-weight:normal;line-height:0%;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none"
+ xml:space="preserve"><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="70"
+ x="365"
+ id="tspan29165-9"
+ sodipodi:role="line">1</tspan></text>
+ </g>
+ <g
+ id="g29167-9"
+ transform="matrix(0.73232502,0,0,0.75477602,-4.9726112,28.689051)">
+ <text
+ id="text29163-3"
+ y="70"
+ x="365"
+ style="font-style:normal;font-weight:normal;line-height:0%;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none"
+ xml:space="preserve"><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="70"
+ x="365"
+ id="tspan29165-3"
+ sodipodi:role="line">2</tspan></text>
+ </g>
+ <path
+ style="fill:none;stroke:#000000;stroke-width:0.678033px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1;marker-start:url(#Arrow1Lstart-7);marker-end:none"
+ d="M 181,214.66098 V 145.33902"
+ id="path17211-7-1-6"
+ inkscape:connector-type="orthogonal"
+ inkscape:connector-curvature="0" />
+ <g
+ id="g29167"
+ transform="matrix(0.73232502,0,0,0.75477602,-218.07919,73.10621)">
+ <text
+ id="text29163"
+ y="70"
+ x="365"
+ style="font-style:normal;font-weight:normal;line-height:0%;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none"
+ xml:space="preserve"><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="70"
+ x="365"
+ id="tspan29165"
+ sodipodi:role="line">8</tspan></text>
+ </g>
+ <path
+ style="fill:none;stroke:#000000;stroke-width:0.678033px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1;marker-start:url(#Arrow1Lstart-7);marker-end:none"
+ d="m 131,145.8531 v 69.32197"
+ id="path17211-7-1"
+ inkscape:connector-type="orthogonal"
+ inkscape:connector-curvature="0" />
+ <path
+ transform="matrix(0.73232502,0,0,0.75477602,-140.37076,129.97088)"
+ sodipodi:type="arc"
+ style="fill:#539de6;fill-opacity:1;stroke:#0000ea;stroke-width:1;stroke-linecap:square;stroke-miterlimit:4;stroke-opacity:1;stroke-dasharray:none"
+ id="path29161-8"
+ sodipodi:cx="371"
+ sodipodi:cy="64.5"
+ sodipodi:rx="17"
+ sodipodi:ry="15.5"
+ d="m 388,64.5 a 17,15.5 0 1 1 -34,0 17,15.5 0 1 1 34,0 z" />
+ <g
+ id="g29167-2"
+ transform="matrix(0.73232502,0,0,0.75477602,-140.28602,131.01695)">
+ <text
+ id="text29163-92"
+ y="70"
+ x="365"
+ style="font-style:normal;font-weight:normal;line-height:0%;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none"
+ xml:space="preserve"><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="70"
+ x="365"
+ id="tspan29165-8"
+ sodipodi:role="line">7</tspan></text>
+ </g>
+ <path
+ style="fill:none;stroke:#000000;stroke-width:0.718986px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1;marker-mid:none;marker-end:url(#Arrow1Lend)"
+ d="m 317.1405,116 h -94.281"
+ id="path17209-8-0"
+ inkscape:connector-type="orthogonal"
+ inkscape:connector-curvature="0" />
+ <path
+ transform="matrix(0.73232502,0,0,0.75477602,-3.4914,66.68745)"
+ sodipodi:type="arc"
+ style="fill:#539de6;fill-opacity:1;stroke:#0000ea;stroke-width:1;stroke-linecap:square;stroke-miterlimit:4;stroke-opacity:1;stroke-dasharray:none"
+ id="path29161-6"
+ sodipodi:cx="371"
+ sodipodi:cy="64.5"
+ sodipodi:rx="17"
+ sodipodi:ry="15.5"
+ d="m 388,64.5 a 17,15.5 0 1 1 -34,0 17,15.5 0 1 1 34,0 z" />
+ <g
+ id="g29167-46"
+ transform="matrix(0.73232502,0,0,0.75477602,-4.40666,67.48829)">
+ <text
+ id="text29163-1"
+ y="70"
+ x="365"
+ style="font-style:normal;font-weight:normal;line-height:0%;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none"
+ xml:space="preserve"><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="70"
+ x="365"
+ id="tspan29165-5"
+ sodipodi:role="line">3</tspan></text>
+ </g>
+ <path
+ transform="matrix(0.73232502,0,0,0.75477602,-90.692582,130.31695)"
+ sodipodi:type="arc"
+ style="fill:#539de6;fill-opacity:1;stroke:#0000ea;stroke-width:1;stroke-linecap:square;stroke-miterlimit:4;stroke-opacity:1;stroke-dasharray:none"
+ id="path29161-8-6"
+ sodipodi:cx="371"
+ sodipodi:cy="64.5"
+ sodipodi:rx="17"
+ sodipodi:ry="15.5"
+ d="m 388,64.5 a 17,15.5 0 1 1 -34,0 17,15.5 0 1 1 34,0 z" />
+ <g
+ id="g29167-6"
+ transform="matrix(0.73232502,0,0,0.75477602,-90.84634,131.60918)">
+ <text
+ id="text29163-17"
+ y="70"
+ x="365"
+ style="font-style:normal;font-weight:normal;line-height:0%;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none"
+ xml:space="preserve"><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="70"
+ x="365"
+ id="tspan29165-2"
+ sodipodi:role="line">4</tspan></text>
+ </g>
+ <g
+ id="g29167-2-0"
+ transform="matrix(0.73232502,0,0,0.75477602,-2.424397,194.0216)">
+ <text
+ id="text29163-92-6"
+ y="70"
+ x="365"
+ style="font-style:normal;font-weight:normal;line-height:0%;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none"
+ xml:space="preserve"><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="70"
+ x="365"
+ id="tspan29165-8-2"
+ sodipodi:role="line">5</tspan></text>
+ </g>
+ <g
+ style="font-size:12px;fill:none;stroke-linecap:square;stroke-miterlimit:3;overflow:visible"
+ id="shape1-1-2-8"
+ v:mID="1"
+ v:groupContext="shape"
+ transform="matrix(2.1604167,0,0,1.5671361,93.82055,-648.98949)">
+ <title
+ id="title22-7-97">Square</title>
+ <desc
+ id="desc24-7-3">Atomic Queue #1</desc>
+ <v:userDefs>
+ <v:ud
+ v:nameU="visVersion"
+ v:val="VT0(15):26" />
+ </v:userDefs>
+ <v:textBlock
+ v:margins="rect(4,4,4,4)" />
+ <v:textRect
+ cx="30.75"
+ cy="581.25"
+ width="61.5"
+ height="61.5" />
+ <g
+ id="shadow1-2-9-6"
+ v:groupContext="shadow"
+ v:shadowOffsetX="0.345598"
+ v:shadowOffsetY="-1.97279"
+ v:shadowType="1"
+ transform="translate(0.345598,1.97279)"
+ class="st1"
+ style="visibility:visible">
+ <rect
+ x="0"
+ y="550.5"
+ width="61.5"
+ height="61.5"
+ class="st2"
+ id="rect27-8-12"
+ style="fill:#5b9bd5;fill-opacity:0.22000002;stroke:#5b9bd5;stroke-opacity:0.22000002;filter:url(#filter_2-3-92)" />
+ </g>
+ <g
+ id="g13515-9">
+ <g
+ id="g13534-3">
+ <rect
+ x="0"
+ y="550.5"
+ width="61.5"
+ height="61.5"
+ class="st3"
+ id="rect29-1-1"
+ style="fill:#5b9bd5;stroke:#c7c8c8;stroke-width:0.25" />
+ </g>
+ </g>
+ </g>
+ <g
+ style="font-size:12px;fill:none;stroke-linecap:square;stroke-miterlimit:3;overflow:visible"
+ id="shape1-1-2-84"
+ v:mID="1"
+ v:groupContext="shape"
+ transform="matrix(2.1604167,0,0,1.5671361,314.82055,-648.98949)">
+ <title
+ id="title22-7-50">Square</title>
+ <desc
+ id="desc24-7-36">Atomic Queue #1</desc>
+ <v:userDefs>
+ <v:ud
+ v:nameU="visVersion"
+ v:val="VT0(15):26" />
+ </v:userDefs>
+ <v:textBlock
+ v:margins="rect(4,4,4,4)" />
+ <v:textRect
+ cx="30.75"
+ cy="581.25"
+ width="61.5"
+ height="61.5" />
+ <g
+ id="shadow1-2-9-1"
+ v:groupContext="shadow"
+ v:shadowOffsetX="0.345598"
+ v:shadowOffsetY="-1.97279"
+ v:shadowType="1"
+ transform="translate(0.345598,1.97279)"
+ class="st1"
+ style="visibility:visible">
+ <rect
+ x="0"
+ y="550.5"
+ width="61.5"
+ height="61.5"
+ class="st2"
+ id="rect27-8-0"
+ style="fill:#5b9bd5;fill-opacity:0.22000002;stroke:#5b9bd5;stroke-opacity:0.22000002;filter:url(#filter_2-3-94)" />
+ </g>
+ <g
+ id="g13515-6">
+ <g
+ id="g13534-32">
+ <rect
+ x="0"
+ y="550.5"
+ width="61.5"
+ height="61.5"
+ class="st3"
+ id="rect29-1-0"
+ style="fill:#5b9bd5;stroke:#c7c8c8;stroke-width:0.25" />
+ </g>
+ </g>
+ </g>
+ <path
+ style="fill:none;stroke:#000000;stroke-width:0.712265;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:4;stroke-dasharray:none;stroke-opacity:1;marker-start:none;marker-end:url(#Arrow1Lend)"
+ d="M 313.14387,285 H 228.85613"
+ id="path17209-7"
+ inkscape:connector-type="orthogonal"
+ inkscape:connector-curvature="0" />
+ <path
+ transform="matrix(0.73232502,0,0,0.75477602,-2.692582,236.31695)"
+ sodipodi:type="arc"
+ style="fill:#539de6;fill-opacity:1;stroke:#0000ea;stroke-width:1;stroke-linecap:square;stroke-miterlimit:4;stroke-opacity:1;stroke-dasharray:none"
+ id="path29161-1-6"
+ sodipodi:cx="371"
+ sodipodi:cy="64.5"
+ sodipodi:rx="17"
+ sodipodi:ry="15.5"
+ d="m 388,64.5 a 17,15.5 0 1 1 -34,0 17,15.5 0 1 1 34,0 z" />
+ <g
+ id="g29167-2-0-5"
+ transform="matrix(0.73232502,0,0,0.75477602,-2.424397,237.0216)">
+ <text
+ id="text29163-92-6-6"
+ y="70"
+ x="365"
+ style="font-style:normal;font-weight:normal;line-height:0%;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none"
+ xml:space="preserve"><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="70"
+ x="365"
+ id="tspan29165-8-2-9"
+ sodipodi:role="line">6</tspan></text>
+ </g>
+ <g
+ id="g29167-4-3"
+ transform="matrix(0.73232502,0,0,0.75477602,-154.60784,51.117791)">
+ <text
+ id="text29163-9-6"
+ y="70"
+ x="365"
+ style="font-style:normal;font-weight:normal;line-height:0%;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none"
+ xml:space="preserve"><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:24.2109px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="70"
+ x="365"
+ id="tspan29165-9-7"
+ sodipodi:role="line">Eventdev</tspan></text>
+ </g>
+ <g
+ id="g29167-4-3-5"
+ transform="matrix(0.73232502,0,0,0.75477602,-144.65044,201.97821)">
+ <text
+ id="text29163-9-6-3"
+ y="70"
+ x="412.93716"
+ style="font-style:normal;font-weight:normal;line-height:0%;font-family:sans-serif;text-align:center;letter-spacing:0px;word-spacing:0px;text-anchor:middle;fill:#000000;fill-opacity:1;stroke:none"
+ xml:space="preserve"><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:24.2109px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:center;writing-mode:lr-tb;text-anchor:middle"
+ y="70"
+ x="412.93716"
+ id="tspan29165-9-7-5"
+ sodipodi:role="line">DMA</tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:24.2109px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:center;writing-mode:lr-tb;text-anchor:middle"
+ y="100.26363"
+ x="412.93716"
+ sodipodi:role="line"
+ id="tspan3201">Adapter</tspan></text>
+ </g>
+ <g
+ id="g29167-4-3-5-6"
+ transform="matrix(0.73232502,0,0,0.75477602,79.53518,46.62529)">
+ <text
+ id="text29163-9-6-3-2"
+ y="48.801659"
+ x="412.93716"
+ style="font-style:normal;font-weight:normal;line-height:0%;font-family:sans-serif;text-align:center;letter-spacing:0px;word-spacing:0px;text-anchor:middle;fill:#000000;fill-opacity:1;stroke:none"
+ xml:space="preserve"><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:24.2109px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:center;writing-mode:lr-tb;text-anchor:middle"
+ y="48.801659"
+ x="412.93716"
+ sodipodi:role="line"
+ id="tspan3155">Application</tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:24.2109px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:center;writing-mode:lr-tb;text-anchor:middle"
+ y="98.801659"
+ x="412.93716"
+ sodipodi:role="line"
+ id="tspan3201-1">in ordered</tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:24.2109px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:center;writing-mode:lr-tb;text-anchor:middle"
+ y="148.80167"
+ x="412.93716"
+ sodipodi:role="line"
+ id="tspan3161">stage</tspan></text>
+ </g>
+ <g
+ id="g29167-4-3-5-2"
+ transform="matrix(0.73232502,0,0,0.75477602,77.535182,213.62529)">
+ <text
+ id="text29163-9-6-3-7"
+ y="70"
+ x="412.93716"
+ style="font-style:normal;font-weight:normal;line-height:0%;font-family:sans-serif;text-align:center;letter-spacing:0px;word-spacing:0px;text-anchor:middle;fill:#000000;fill-opacity:1;stroke:none"
+ xml:space="preserve"><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:24.2109px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:center;writing-mode:lr-tb;text-anchor:middle"
+ y="70"
+ x="412.93716"
+ sodipodi:role="line"
+ id="tspan3201-9">DMA Device</tspan></text>
+ </g>
+ <g
+ id="g29167-4-3-5-3"
+ transform="matrix(0.73232502,0,0,0.75477602,188.53518,-3.37471)">
+ <text
+ id="text29163-9-6-3-6"
+ y="70"
+ x="375.65271"
+ style="font-style:normal;font-weight:normal;line-height:0%;font-family:sans-serif;text-align:center;letter-spacing:0px;word-spacing:0px;text-anchor:middle;fill:#000000;fill-opacity:1;stroke:none"
+ xml:space="preserve"><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18.8307px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="70"
+ x="375.65271"
+ sodipodi:role="line"
+ id="tspan3201-6">1. Events from the previous stage.</tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18.8307px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="93.538376"
+ x="375.65271"
+ sodipodi:role="line"
+ id="tspan3260"> </tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18.8307px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="117.07675"
+ x="375.65271"
+ sodipodi:role="line"
+ id="tspan3262">2. Application in ordered stage</tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18.8307px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="140.61513"
+ x="375.65271"
+ sodipodi:role="line"
+ id="tspan3288"> dequeues events from eventdev.</tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18.8307px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="164.1535"
+ x="375.65271"
+ sodipodi:role="line"
+ id="tspan3264"> </tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18.8307px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="187.69188"
+ x="375.65271"
+ sodipodi:role="line"
+ id="tspan3266">3. Application enqueues DMA</tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18.8307px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="211.23026"
+ x="375.65271"
+ sodipodi:role="line"
+ id="tspan3290"> operations as events to eventdev.</tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18.8307px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="234.76863"
+ x="375.65271"
+ sodipodi:role="line"
+ id="tspan3268"> </tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18.8307px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="258.30701"
+ x="375.65271"
+ sodipodi:role="line"
+ id="tspan3270">4. DMA adapter dequeues event</tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18.8307px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="281.84537"
+ x="375.65271"
+ sodipodi:role="line"
+ id="tspan3292"> from eventdev.</tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18.8307px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="305.38376"
+ x="375.65271"
+ sodipodi:role="line"
+ id="tspan3272"> </tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18.8307px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="328.92212"
+ x="375.65271"
+ sodipodi:role="line"
+ id="tspan3274">5. DMA adapter submits DMA</tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18.8307px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="352.46051"
+ x="375.65271"
+ sodipodi:role="line"
+ id="tspan3294"> operations to DMA Device (Atomic</tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18.8307px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="375.99887"
+ x="375.65271"
+ sodipodi:role="line"
+ id="tspan3296"> stage)</tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18.8307px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="399.53726"
+ x="375.65271"
+ sodipodi:role="line"
+ id="tspan3276"> </tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18.8307px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="423.07562"
+ x="375.65271"
+ sodipodi:role="line"
+ id="tspan3278">6. DMA adapter dequeues DMA</tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18.8307px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="446.61401"
+ x="375.65271"
+ sodipodi:role="line"
+ id="tspan3298"> completions from DMA Device</tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18.8307px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="470.15237"
+ x="375.65271"
+ sodipodi:role="line"
+ id="tspan3280"> </tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18.8307px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="493.69073"
+ x="375.65271"
+ sodipodi:role="line"
+ id="tspan3282">7. DMA adapter enqueues events</tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18.8307px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="517.22913"
+ x="375.65271"
+ sodipodi:role="line"
+ id="tspan3300"> to the eventdev</tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18.8307px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="540.76752"
+ x="375.65271"
+ sodipodi:role="line"
+ id="tspan3284"> </tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18.8307px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="564.30585"
+ x="375.65271"
+ sodipodi:role="line"
+ id="tspan3286">8. Events to the next stage</tspan></text>
+ </g>
+ </g>
+</svg>
diff --git a/doc/guides/prog_guide/img/event_dma_adapter_op_new.svg b/doc/guides/prog_guide/img/event_dma_adapter_op_new.svg
new file mode 100644
index 0000000000..e9e8bb2b98
--- /dev/null
+++ b/doc/guides/prog_guide/img/event_dma_adapter_op_new.svg
@@ -0,0 +1,1079 @@
+<?xml version="1.0" encoding="UTF-8" standalone="no"?>
+<!-- SPDX-License-Identifier: BSD-3-Clause -->
+<!-- Copyright (c) 2023 Marvell. -->
+<!-- Created with Inkscape (http://www.inkscape.org/) -->
+
+<svg
+ width="720px"
+ height="486px"
+ id="svg13237"
+ version="1.1"
+ inkscape:version="1.2.1 (9c6d41e410, 2022-07-14)"
+ sodipodi:docname="event_dma_adapter_op_new.svg"
+ xmlns:inkscape="http://www.inkscape.org/namespaces/inkscape"
+ xmlns:sodipodi="http://sodipodi.sourceforge.net/DTD/sodipodi-0.dtd"
+ xmlns="http://www.w3.org/2000/svg"
+ xmlns:svg="http://www.w3.org/2000/svg"
+ xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#"
+ xmlns:cc="http://creativecommons.org/ns#"
+ xmlns:dc="http://purl.org/dc/elements/1.1/"
+ xmlns:v="http://schemas.microsoft.com/visio/2003/SVGExtensions/">
+ <defs
+ id="defs13239">
+ <marker
+ inkscape:stockid="Arrow1Sstart"
+ orient="auto"
+ refY="0.0"
+ refX="0.0"
+ id="Arrow1Sstart"
+ style="overflow:visible">
+ <path
+ id="path8416"
+ d="M 0.0,0.0 L 5.0,-5.0 L -12.5,0.0 L 5.0,5.0 L 0.0,0.0 z "
+ style="fill-rule:evenodd;stroke:#000000;stroke-width:1.0pt"
+ transform="scale(0.2) translate(6,0)" />
+ </marker>
+ <marker
+ inkscape:stockid="Arrow1Send"
+ orient="auto"
+ refY="0.0"
+ refX="0.0"
+ id="Arrow1Send"
+ style="overflow:visible;">
+ <path
+ id="path8419"
+ d="M 0.0,0.0 L 5.0,-5.0 L -12.5,0.0 L 5.0,5.0 L 0.0,0.0 z "
+ style="fill-rule:evenodd;stroke:#000000;stroke-width:1.0pt;"
+ transform="scale(0.2) rotate(180) translate(6,0)" />
+ </marker>
+ <marker
+ inkscape:stockid="DiamondL"
+ orient="auto"
+ refY="0.0"
+ refX="0.0"
+ id="DiamondL"
+ style="overflow:visible">
+ <path
+ id="path8483"
+ d="M 0,-7.0710768 L -7.0710894,0 L 0,7.0710589 L 7.0710462,0 L 0,-7.0710768 z "
+ style="fill-rule:evenodd;stroke:#000000;stroke-width:1.0pt"
+ transform="scale(0.8)" />
+ </marker>
+ <marker
+ inkscape:stockid="DotL"
+ orient="auto"
+ refY="0.0"
+ refX="0.0"
+ id="DotL"
+ style="overflow:visible">
+ <path
+ id="path8465"
+ d="M -2.5,-1.0 C -2.5,1.7600000 -4.7400000,4.0 -7.5,4.0 C -10.260000,4.0 -12.5,1.7600000 -12.5,-1.0 C -12.5,-3.7600000 -10.260000,-6.0 -7.5,-6.0 C -4.7400000,-6.0 -2.5,-3.7600000 -2.5,-1.0 z "
+ style="fill-rule:evenodd;stroke:#000000;stroke-width:1.0pt"
+ transform="scale(0.8) translate(7.4, 1)" />
+ </marker>
+ <marker
+ inkscape:stockid="SquareL"
+ orient="auto"
+ refY="0.0"
+ refX="0.0"
+ id="SquareL"
+ style="overflow:visible">
+ <path
+ id="path8474"
+ d="M -5.0,-5.0 L -5.0,5.0 L 5.0,5.0 L 5.0,-5.0 L -5.0,-5.0 z "
+ style="fill-rule:evenodd;stroke:#000000;stroke-width:1.0pt"
+ transform="scale(0.8)" />
+ </marker>
+ <marker
+ inkscape:stockid="TriangleOutL"
+ orient="auto"
+ refY="0.0"
+ refX="0.0"
+ id="TriangleOutL"
+ style="overflow:visible">
+ <path
+ id="path8546"
+ d="M 5.77,0.0 L -2.88,5.0 L -2.88,-5.0 L 5.77,0.0 z "
+ style="fill-rule:evenodd;stroke:#000000;stroke-width:1.0pt"
+ transform="scale(0.8)" />
+ </marker>
+ <marker
+ inkscape:stockid="Arrow1Lstart"
+ orient="auto"
+ refY="0.0"
+ refX="0.0"
+ id="Arrow1Lstart"
+ style="overflow:visible">
+ <path
+ id="path8404"
+ d="M 0.0,0.0 L 5.0,-5.0 L -12.5,0.0 L 5.0,5.0 L 0.0,0.0 z "
+ style="fill-rule:evenodd;stroke:#000000;stroke-width:1.0pt"
+ transform="scale(0.8) translate(12.5,0)" />
+ </marker>
+ <marker
+ inkscape:stockid="Arrow1Mend"
+ orient="auto"
+ refY="0.0"
+ refX="0.0"
+ id="Arrow1Mend"
+ style="overflow:visible;">
+ <path
+ id="path8413"
+ d="M 0.0,0.0 L 5.0,-5.0 L -12.5,0.0 L 5.0,5.0 L 0.0,0.0 z "
+ style="fill-rule:evenodd;stroke:#000000;stroke-width:1.0pt;"
+ transform="scale(0.4) rotate(180) translate(10,0)" />
+ </marker>
+ <marker
+ inkscape:stockid="Arrow2Lend"
+ orient="auto"
+ refY="0.0"
+ refX="0.0"
+ id="Arrow2Lend"
+ style="overflow:visible;">
+ <path
+ id="path8425"
+ style="fill-rule:evenodd;stroke-width:0.62500000;stroke-linejoin:round;"
+ d="M 8.7185878,4.0337352 L -2.2072895,0.016013256 L 8.7185884,-4.0017078 C 6.9730900,-1.6296469 6.9831476,1.6157441 8.7185878,4.0337352 z "
+ transform="scale(1.1) rotate(180) translate(1,0)" />
+ </marker>
+ <marker
+ inkscape:stockid="Arrow1Lend"
+ orient="auto"
+ refY="0.0"
+ refX="0.0"
+ id="Arrow1Lend"
+ style="overflow:visible;">
+ <path
+ id="path8407"
+ d="M 0.0,0.0 L 5.0,-5.0 L -12.5,0.0 L 5.0,5.0 L 0.0,0.0 z "
+ style="fill-rule:evenodd;stroke:#000000;stroke-width:1.0pt;"
+ transform="scale(0.8) rotate(180) translate(12.5,0)" />
+ </marker>
+ <filter
+ id="filter_2"
+ color-interpolation-filters="sRGB">
+ <feGaussianBlur
+ stdDeviation="2"
+ id="feGaussianBlur15" />
+ </filter>
+ <filter
+ id="filter_2-3"
+ color-interpolation-filters="sRGB"
+ x="-0.086178862"
+ y="-0.086178862"
+ width="1.1723577"
+ height="1.1723577">
+ <feGaussianBlur
+ stdDeviation="2"
+ id="feGaussianBlur15-1" />
+ </filter>
+ <filter
+ id="filter_2-0"
+ color-interpolation-filters="sRGB">
+ <feGaussianBlur
+ stdDeviation="2"
+ id="feGaussianBlur15-7" />
+ </filter>
+ <filter
+ id="filter_2-0-8"
+ color-interpolation-filters="sRGB">
+ <feGaussianBlur
+ stdDeviation="2"
+ id="feGaussianBlur15-7-7" />
+ </filter>
+ <filter
+ id="filter_2-3-9"
+ color-interpolation-filters="sRGB"
+ x="-0.086178862"
+ y="-0.086178862"
+ width="1.1723577"
+ height="1.1723577">
+ <feGaussianBlur
+ stdDeviation="2"
+ id="feGaussianBlur15-1-6" />
+ </filter>
+ <filter
+ id="filter_2-3-6"
+ color-interpolation-filters="sRGB"
+ x="-0.086178862"
+ y="-0.086178862"
+ width="1.1723577"
+ height="1.1723577">
+ <feGaussianBlur
+ stdDeviation="2"
+ id="feGaussianBlur15-1-63" />
+ </filter>
+ <filter
+ id="filter_2-3-91"
+ color-interpolation-filters="sRGB"
+ x="-0.086178862"
+ y="-0.086178862"
+ width="1.1723577"
+ height="1.1723577">
+ <feGaussianBlur
+ stdDeviation="2"
+ id="feGaussianBlur15-1-3" />
+ </filter>
+ <marker
+ inkscape:stockid="Arrow1Lend"
+ orient="auto"
+ refY="0"
+ refX="0"
+ id="Arrow1Lend-5"
+ style="overflow:visible">
+ <path
+ inkscape:connector-curvature="0"
+ id="path8407-3"
+ d="M 0,0 5,-5 -12.5,0 5,5 0,0 z"
+ style="fill-rule:evenodd;stroke:#000000;stroke-width:1pt"
+ transform="matrix(-0.8,0,0,-0.8,-10,0)" />
+ </marker>
+ <marker
+ inkscape:stockid="Arrow1Lend"
+ orient="auto"
+ refY="0"
+ refX="0"
+ id="Arrow1Lend-6"
+ style="overflow:visible">
+ <path
+ inkscape:connector-curvature="0"
+ id="path8407-0"
+ d="M 0,0 5,-5 -12.5,0 5,5 0,0 z"
+ style="fill-rule:evenodd;stroke:#000000;stroke-width:1pt"
+ transform="matrix(-0.8,0,0,-0.8,-10,0)" />
+ </marker>
+ <marker
+ inkscape:stockid="Arrow1Lstart"
+ orient="auto"
+ refY="0"
+ refX="0"
+ id="Arrow1Lstart-7"
+ style="overflow:visible">
+ <path
+ inkscape:connector-curvature="0"
+ id="path8404-0"
+ d="M 0,0 5,-5 -12.5,0 5,5 0,0 z"
+ style="fill-rule:evenodd;stroke:#000000;stroke-width:1pt"
+ transform="matrix(0.8,0,0,0.8,10,0)" />
+ </marker>
+ <marker
+ inkscape:stockid="Arrow1Lend"
+ orient="auto"
+ refY="0"
+ refX="0"
+ id="Arrow1Lend-51"
+ style="overflow:visible">
+ <path
+ inkscape:connector-curvature="0"
+ id="path8407-1"
+ d="M 0,0 5,-5 -12.5,0 5,5 0,0 z"
+ style="fill-rule:evenodd;stroke:#000000;stroke-width:1pt"
+ transform="matrix(-0.8,0,0,-0.8,-10,0)" />
+ </marker>
+ <marker
+ inkscape:stockid="Arrow1Lend"
+ orient="auto"
+ refY="0"
+ refX="0"
+ id="Arrow1Lend-3"
+ style="overflow:visible">
+ <path
+ inkscape:connector-curvature="0"
+ id="path8407-6"
+ d="M 0,0 5,-5 -12.5,0 5,5 0,0 z"
+ style="fill-rule:evenodd;stroke:#000000;stroke-width:1pt"
+ transform="matrix(-0.8,0,0,-0.8,-10,0)" />
+ </marker>
+ <marker
+ inkscape:stockid="Arrow1Lend"
+ orient="auto"
+ refY="0"
+ refX="0"
+ id="Arrow1Lend-62"
+ style="overflow:visible">
+ <path
+ inkscape:connector-curvature="0"
+ id="path8407-9"
+ d="M 0,0 5,-5 -12.5,0 5,5 0,0 z"
+ style="fill-rule:evenodd;stroke:#000000;stroke-width:1pt"
+ transform="matrix(-0.8,0,0,-0.8,-10,0)" />
+ </marker>
+ <marker
+ inkscape:stockid="Arrow1Lend"
+ orient="auto"
+ refY="0"
+ refX="0"
+ id="Arrow1Lend-2"
+ style="overflow:visible">
+ <path
+ inkscape:connector-curvature="0"
+ id="path8407-7"
+ d="M 0,0 5,-5 -12.5,0 5,5 0,0 z"
+ style="fill-rule:evenodd;stroke:#000000;stroke-width:1pt"
+ transform="matrix(-0.8,0,0,-0.8,-10,0)" />
+ </marker>
+ <filter
+ id="filter_2-3-91-3"
+ color-interpolation-filters="sRGB"
+ x="-0.086178862"
+ y="-0.086178862"
+ width="1.1723577"
+ height="1.1723577">
+ <feGaussianBlur
+ stdDeviation="2"
+ id="feGaussianBlur15-1-3-6" />
+ </filter>
+ </defs>
+ <sodipodi:namedview
+ id="base"
+ pagecolor="#ffffff"
+ bordercolor="#666666"
+ borderopacity="1.0"
+ inkscape:pageopacity="0.0"
+ inkscape:pageshadow="2"
+ inkscape:zoom="1"
+ inkscape:cx="385.5"
+ inkscape:cy="234"
+ inkscape:document-units="px"
+ inkscape:current-layer="layer1"
+ showgrid="false"
+ inkscape:window-width="1920"
+ inkscape:window-height="1017"
+ inkscape:window-x="-8"
+ inkscape:window-y="-8"
+ inkscape:window-maximized="1"
+ inkscape:snap-nodes="false"
+ inkscape:showpageshadow="2"
+ inkscape:pagecheckerboard="0"
+ inkscape:deskcolor="#d1d1d1">
+ <inkscape:grid
+ type="xygrid"
+ id="grid13454" />
+ </sodipodi:namedview>
+ <metadata
+ id="metadata13242">
+ <rdf:RDF>
+ <cc:Work
+ rdf:about="">
+ <dc:format>image/svg+xml</dc:format>
+ <dc:type
+ rdf:resource="http://purl.org/dc/dcmitype/StillImage" />
+ <dc:title />
+ </cc:Work>
+ </rdf:RDF>
+ </metadata>
+ <g
+ id="layer1"
+ inkscape:label="Layer 1"
+ inkscape:groupmode="layer">
+ <g
+ style="font-size:12px;fill:none;stroke-linecap:square;stroke-miterlimit:3;overflow:visible"
+ id="shape1-1-2-0"
+ v:mID="1"
+ v:groupContext="shape"
+ transform="matrix(2.1604167,0,0,1.5671361,323.2187,-540.25927)">
+ <title
+ id="title22-7-6">Square</title>
+ <desc
+ id="desc24-7-4">Atomic Queue #1</desc>
+ <v:userDefs>
+ <v:ud
+ v:nameU="visVersion"
+ v:val="VT0(15):26" />
+ </v:userDefs>
+ <v:textBlock
+ v:margins="rect(4,4,4,4)" />
+ <v:textRect
+ cx="30.75"
+ cy="581.25"
+ width="61.5"
+ height="61.5" />
+ <g
+ id="shadow1-2-9-0"
+ v:groupContext="shadow"
+ v:shadowOffsetX="0.345598"
+ v:shadowOffsetY="-1.97279"
+ v:shadowType="1"
+ transform="translate(0.345598,1.97279)"
+ class="st1"
+ style="visibility:visible">
+ <rect
+ x="0"
+ y="550.5"
+ width="61.5"
+ height="61.5"
+ class="st2"
+ id="rect27-8-9"
+ style="fill:#5b9bd5;fill-opacity:0.22000002;stroke:#5b9bd5;stroke-opacity:0.22000002;filter:url(#filter_2-3-9)" />
+ </g>
+ <g
+ id="g13515-4">
+ <g
+ id="g13534-5">
+ <rect
+ x="0"
+ y="550.5"
+ width="61.5"
+ height="61.5"
+ class="st3"
+ id="rect29-1-4"
+ style="fill:#5b9bd5;stroke:#c7c8c8;stroke-width:0.25" />
+ </g>
+ </g>
+ </g>
+ <g
+ style="font-size:12px;fill:none;stroke-linecap:square;stroke-miterlimit:3;overflow:visible"
+ id="shape1-1-2-4"
+ v:mID="1"
+ v:groupContext="shape"
+ transform="matrix(2.1604167,0,0,1.165886,88.874699,-447.8809)">
+ <title
+ id="title22-7-5">Square</title>
+ <desc
+ id="desc24-7-8">Atomic Queue #1</desc>
+ <v:userDefs>
+ <v:ud
+ v:nameU="visVersion"
+ v:val="VT0(15):26" />
+ </v:userDefs>
+ <v:textBlock
+ v:margins="rect(4,4,4,4)" />
+ <v:textRect
+ cx="30.75"
+ cy="581.25"
+ width="61.5"
+ height="61.5" />
+ <g
+ id="shadow1-2-9-5"
+ v:groupContext="shadow"
+ v:shadowOffsetX="0.345598"
+ v:shadowOffsetY="-1.97279"
+ v:shadowType="1"
+ transform="translate(0.345598,1.97279)"
+ class="st1"
+ style="visibility:visible">
+ <rect
+ x="0"
+ y="550.5"
+ width="61.5"
+ height="61.5"
+ class="st2"
+ id="rect27-8-7"
+ style="fill:#5b9bd5;fill-opacity:0.22000002;stroke:#5b9bd5;stroke-opacity:0.22000002;filter:url(#filter_2-3-91)" />
+ </g>
+ <g
+ id="g13515-33">
+ <g
+ id="g13534-8">
+ <rect
+ x="0"
+ y="550.5"
+ width="61.5"
+ height="61.5"
+ class="st3"
+ id="rect29-1-95"
+ style="fill:#5b9bd5;stroke:#c7c8c8;stroke-width:0.25" />
+ </g>
+ </g>
+ </g>
+ <g
+ style="font-size:12px;fill:none;stroke-linecap:square;stroke-miterlimit:3;overflow:visible"
+ id="shape1-1-2-9"
+ v:mID="1"
+ v:groupContext="shape"
+ transform="matrix(2.1604167,0,0,1.5671361,88.874699,-538.24651)">
+ <title
+ id="title22-7-9">Square</title>
+ <desc
+ id="desc24-7-5">Atomic Queue #1</desc>
+ <v:userDefs>
+ <v:ud
+ v:nameU="visVersion"
+ v:val="VT0(15):26" />
+ </v:userDefs>
+ <v:textBlock
+ v:margins="rect(4,4,4,4)" />
+ <v:textRect
+ cx="30.75"
+ cy="581.25"
+ width="61.5"
+ height="61.5" />
+ <g
+ id="shadow1-2-9-2"
+ v:groupContext="shadow"
+ v:shadowOffsetX="0.345598"
+ v:shadowOffsetY="-1.97279"
+ v:shadowType="1"
+ transform="translate(0.345598,1.97279)"
+ class="st1"
+ style="visibility:visible">
+ <rect
+ x="0"
+ y="550.5"
+ width="61.5"
+ height="61.5"
+ class="st2"
+ id="rect27-8-1"
+ style="fill:#5b9bd5;fill-opacity:0.22000002;stroke:#5b9bd5;stroke-opacity:0.22000002;filter:url(#filter_2-3-6)" />
+ </g>
+ <g
+ id="g13515-3">
+ <g
+ id="g13534-0">
+ <rect
+ x="0"
+ y="550.5"
+ width="61.5"
+ height="61.5"
+ class="st3"
+ id="rect29-1-9"
+ style="fill:#5b9bd5;stroke:#c7c8c8;stroke-width:0.25" />
+ </g>
+ </g>
+ </g>
+ <path
+ style="fill:none;stroke:#000000;stroke-width:0.743466px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1;marker-mid:none;marker-end:url(#Arrow1Lend)"
+ d="M 220.66064,98.57125 H 321.88592"
+ id="path17209-8"
+ inkscape:connector-type="orthogonal"
+ inkscape:connector-curvature="0" />
+ <g
+ style="font-size:12px;fill:none;stroke-linecap:square;stroke-miterlimit:3;overflow:visible"
+ id="shape1-1-2"
+ v:mID="1"
+ v:groupContext="shape"
+ transform="matrix(2.1604167,0,0,1.5671361,322.24227,-811.89589)">
+ <title
+ id="title22-7">Square</title>
+ <desc
+ id="desc24-7">Atomic Queue #1</desc>
+ <v:userDefs>
+ <v:ud
+ v:nameU="visVersion"
+ v:val="VT0(15):26" />
+ </v:userDefs>
+ <v:textBlock
+ v:margins="rect(4,4,4,4)" />
+ <v:textRect
+ cx="30.75"
+ cy="581.25"
+ width="61.5"
+ height="61.5" />
+ <g
+ id="shadow1-2-9"
+ v:groupContext="shadow"
+ v:shadowOffsetX="0.345598"
+ v:shadowOffsetY="-1.97279"
+ v:shadowType="1"
+ transform="translate(0.345598,1.97279)"
+ class="st1"
+ style="visibility:visible">
+ <rect
+ x="0"
+ y="550.5"
+ width="61.5"
+ height="61.5"
+ class="st2"
+ id="rect27-8"
+ style="fill:#5b9bd5;fill-opacity:0.22000002;stroke:#5b9bd5;stroke-opacity:0.22000002;filter:url(#filter_2-3)" />
+ </g>
+ <g
+ id="g13515">
+ <g
+ id="g13534">
+ <rect
+ x="0"
+ y="550.5"
+ width="61.5"
+ height="61.5"
+ class="st3"
+ id="rect29-1"
+ style="fill:#5b9bd5;stroke:#c7c8c8;stroke-width:0.25" />
+ </g>
+ </g>
+ </g>
+ <g
+ id="g13518"
+ transform="matrix(0.73232502,0,0,0.75477602,25.29268,348.89752)">
+ <g
+ id="g13526">
+ <flowRoot
+ xml:space="preserve"
+ id="flowRoot13464-9"
+ style="font-style:normal;font-weight:normal;line-height:0.01%;font-family:sans-serif;text-align:center;letter-spacing:0px;word-spacing:0px;text-anchor:middle;fill:#000000;fill-opacity:1;stroke:none"
+ transform="translate(-12.00521,-129.65179)"><flowRegion
+ id="flowRegion13466-1"
+ style="font-family:sans-serif"><rect
+ id="rect13468-2"
+ width="195.99997"
+ height="112.00001"
+ x="273.33334"
+ y="175.33333"
+ style="text-align:center;text-anchor:middle" /></flowRegion><flowPara
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:20px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:center;writing-mode:lr-tb;text-anchor:middle"
+ id="flowPara13511"> </flowPara></flowRoot>
+ </g>
+ </g>
+ <path
+ style="fill:none;stroke:#000000;stroke-width:0.751455;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:4;stroke-dasharray:none;stroke-opacity:1;marker-start:url(#Arrow1Lstart);marker-end:none"
+ d="m 176.26096,124.64833 v 69.24854"
+ id="path17209-3"
+ inkscape:connector-type="orthogonal"
+ inkscape:connector-curvature="0" />
+ <path
+ transform="matrix(0.73232502,0,0,0.75477602,-4.325033,50.642983)"
+ sodipodi:type="arc"
+ style="fill:#539de6;fill-opacity:1;stroke:#0000ea;stroke-width:1;stroke-linecap:square;stroke-miterlimit:4;stroke-opacity:1;stroke-dasharray:none"
+ id="path29161-3"
+ sodipodi:cx="371"
+ sodipodi:cy="64.5"
+ sodipodi:rx="17"
+ sodipodi:ry="15.5"
+ d="m 388,64.5 a 17,15.5 0 1 1 -34,0 17,15.5 0 1 1 34,0 z" />
+ <path
+ style="fill:none;stroke:#000000;stroke-width:0.743466px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1;marker-mid:none;marker-end:url(#Arrow1Lend)"
+ d="M 322.61264,375 H 221.38736"
+ id="path17209-8-0"
+ inkscape:connector-type="orthogonal"
+ inkscape:connector-curvature="0" />
+ <path
+ transform="matrix(0.73232502,0,0,0.75477602,0.0689171,324.80833)"
+ sodipodi:type="arc"
+ style="fill:#539de6;fill-opacity:1;stroke:#0000ea;stroke-width:1;stroke-linecap:square;stroke-miterlimit:4;stroke-opacity:1;stroke-dasharray:none"
+ id="path29161-1"
+ sodipodi:cx="371"
+ sodipodi:cy="64.5"
+ sodipodi:rx="17"
+ sodipodi:ry="15.5"
+ d="m 388,64.5 a 17,15.5 0 1 1 -34,0 17,15.5 0 1 1 34,0 z" />
+ <path
+ style="fill:none;stroke:#000000;stroke-width:0.629082px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1;marker-end:url(#Arrow1Lend)"
+ d="M 155,324.19955 V 264.82863"
+ id="path17211-7-1"
+ inkscape:connector-type="orthogonal"
+ inkscape:connector-curvature="0" />
+ <path
+ transform="matrix(0.73232502,0,0,0.75477602,-116.37076,245.97088)"
+ sodipodi:type="arc"
+ style="fill:#539de6;fill-opacity:1;stroke:#0000ea;stroke-width:1;stroke-linecap:square;stroke-miterlimit:4;stroke-opacity:1;stroke-dasharray:none"
+ id="path29161-8"
+ sodipodi:cx="371"
+ sodipodi:cy="64.5"
+ sodipodi:rx="17"
+ sodipodi:ry="15.5"
+ d="m 388,64.5 a 17,15.5 0 1 1 -34,0 17,15.5 0 1 1 34,0 z" />
+ <path
+ style="fill:none;stroke:#000000;stroke-width:0.75059;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:4;stroke-dasharray:none;stroke-opacity:1;marker-start:url(#Arrow1Lstart-7);marker-end:none"
+ d="m 126.26097,124.99178 v 69.24941"
+ id="path17209-3-0"
+ inkscape:connector-type="orthogonal"
+ inkscape:connector-curvature="0" />
+ <path
+ sodipodi:type="arc"
+ style="fill:#539de6;fill-opacity:1;stroke:#0000ea;stroke-width:1;stroke-linecap:square;stroke-miterlimit:4;stroke-opacity:1;stroke-dasharray:none"
+ id="path29161"
+ sodipodi:cx="371"
+ sodipodi:cy="64.5"
+ sodipodi:rx="17"
+ sodipodi:ry="15.5"
+ d="m 388,64.5 a 17,15.5 0 1 1 -34,0 17,15.5 0 1 1 34,0 z"
+ transform="matrix(0.73232502,0,0,0.75477602,-146.16394,110.68276)" />
+ <path
+ sodipodi:type="arc"
+ style="fill:#539de6;fill-opacity:1;stroke:#0000ea;stroke-width:1;stroke-linecap:square;stroke-miterlimit:4;stroke-opacity:1;stroke-dasharray:none"
+ id="path29161-2"
+ sodipodi:cx="371"
+ sodipodi:cy="64.5"
+ sodipodi:rx="17"
+ sodipodi:ry="15.5"
+ d="m 388,64.5 a 17,15.5 0 1 1 -34,0 17,15.5 0 1 1 34,0 z"
+ transform="matrix(0.73232502,0,0,0.75477602,-95.40136,110.71627)" />
+ <g
+ id="g29167-4"
+ transform="matrix(0.73232502,0,0,0.75477602,-95.31662,112.00756)">
+ <text
+ id="text29163-9"
+ y="70"
+ x="365"
+ style="font-style:normal;font-weight:normal;line-height:0%;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none"
+ xml:space="preserve"><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="70"
+ x="365"
+ id="tspan29165-9"
+ sodipodi:role="line">1</tspan></text>
+ </g>
+ <g
+ id="g29167-9"
+ transform="matrix(0.73232502,0,0,0.75477602,-4.9726112,50.689051)">
+ <text
+ id="text29163-3"
+ y="70"
+ x="365"
+ style="font-style:normal;font-weight:normal;line-height:0%;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none"
+ xml:space="preserve"><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="70"
+ x="365"
+ id="tspan29165-3"
+ sodipodi:role="line">2</tspan></text>
+ </g>
+ <path
+ style="fill:none;stroke:#000000;stroke-width:1.04033px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1;marker-end:url(#Arrow1Lend)"
+ d="M 388.20118,147.93341 V 321.89308"
+ id="path17211-7"
+ inkscape:connector-type="orthogonal"
+ inkscape:connector-curvature="0" />
+ <path
+ transform="matrix(0.73232502,0,0,0.75477602,116.5086,136.68745)"
+ sodipodi:type="arc"
+ style="fill:#539de6;fill-opacity:1;stroke:#0000ea;stroke-width:1;stroke-linecap:square;stroke-miterlimit:4;stroke-opacity:1;stroke-dasharray:none"
+ id="path29161-6"
+ sodipodi:cx="371"
+ sodipodi:cy="64.5"
+ sodipodi:rx="17"
+ sodipodi:ry="15.5"
+ d="m 388,64.5 a 17,15.5 0 1 1 -34,0 17,15.5 0 1 1 34,0 z" />
+ <g
+ id="g29167-46"
+ transform="matrix(0.73232502,0,0,0.75477602,116.59334,137.48829)">
+ <text
+ id="text29163-1"
+ y="70"
+ x="365"
+ style="font-style:normal;font-weight:normal;line-height:0%;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none"
+ xml:space="preserve"><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="70"
+ x="365"
+ id="tspan29165-5"
+ sodipodi:role="line">3</tspan></text>
+ </g>
+ <g
+ id="g29167-6"
+ transform="matrix(0.73232502,0,0,0.75477602,0.1536639,325.60918)">
+ <text
+ id="text29163-17"
+ y="70"
+ x="365"
+ style="font-style:normal;font-weight:normal;line-height:0%;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none"
+ xml:space="preserve"><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="70"
+ x="365"
+ id="tspan29165-2"
+ sodipodi:role="line">4</tspan></text>
+ </g>
+ <g
+ id="g29167"
+ transform="matrix(0.73232502,0,0,0.75477602,-146.07919,111.10621)">
+ <text
+ id="text29163"
+ y="70"
+ x="365"
+ style="font-style:normal;font-weight:normal;line-height:0%;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none"
+ xml:space="preserve"><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="70"
+ x="365"
+ id="tspan29165"
+ sodipodi:role="line">6</tspan></text>
+ </g>
+ <g
+ id="g29167-4-3"
+ transform="matrix(0.73232502,0,0,0.75477602,-117.60784,180.11779)">
+ <text
+ id="text29163-9-6"
+ y="70"
+ x="321.30356"
+ style="font-style:normal;font-weight:normal;line-height:0%;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none"
+ xml:space="preserve"><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:24.2109px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="70"
+ x="321.30356"
+ id="tspan29165-9-7"
+ sodipodi:role="line">Eventdev</tspan></text>
+ </g>
+ <g
+ id="g29167-4-3-5"
+ transform="matrix(0.73232502,0,0,0.75477602,55.34956,26.97821)">
+ <text
+ id="text29163-9-6-3"
+ y="70"
+ x="454.74152"
+ style="font-style:normal;font-weight:normal;line-height:0%;font-family:sans-serif;text-align:center;letter-spacing:0px;word-spacing:0px;text-anchor:middle;fill:#000000;fill-opacity:1;stroke:none"
+ xml:space="preserve"><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:21.5208px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:center;writing-mode:lr-tb;text-anchor:middle"
+ y="70"
+ x="454.74152"
+ id="tspan29165-9-7-5"
+ sodipodi:role="line">A<tspan
+ style="line-height:100%;font-family:sans-serif"
+ id="tspan3374">tomic Stage</tspan></tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:21.5208px;line-height:100%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:center;writing-mode:lr-tb;text-anchor:middle"
+ y="94.210899"
+ x="454.74152"
+ sodipodi:role="line"
+ id="tspan3320">+</tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:21.5208px;line-height:100%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:center;writing-mode:lr-tb;text-anchor:middle"
+ y="115.7317"
+ x="454.74152"
+ sodipodi:role="line"
+ id="tspan3322">Enqueue to</tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:21.5208px;line-height:100%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:center;writing-mode:lr-tb;text-anchor:middle"
+ y="137.2525"
+ x="454.74152"
+ sodipodi:role="line"
+ id="tspan3324">DMA Device</tspan></text>
+ </g>
+ <g
+ id="g29167-2"
+ transform="matrix(0.73232502,0,0,0.75477602,-116.28602,248.01695)">
+ <text
+ id="text29163-92"
+ y="70"
+ x="365"
+ style="font-style:normal;font-weight:normal;line-height:0%;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none"
+ xml:space="preserve"><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="70"
+ x="365"
+ id="tspan29165-8"
+ sodipodi:role="line">5</tspan></text>
+ </g>
+ <flowRoot
+ xml:space="preserve"
+ id="flowRoot3376"
+ style="font-style:normal;font-weight:normal;line-height:0.01%;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none;stroke-width:1px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1"><flowRegion
+ id="flowRegion3378"
+ style="font-family:sans-serif"><rect
+ id="rect3380"
+ width="100"
+ height="37"
+ x="109"
+ y="259" /></flowRegion><flowPara
+ id="flowPara3382"
+ style="font-size:18px;line-height:1.25;font-family:sans-serif"> </flowPara></flowRoot>
+ <g
+ id="g29167-4-3-1"
+ transform="matrix(0.73232502,0,0,0.75477602,109.34956,323.97821)">
+ <text
+ id="text29163-9-6-8"
+ y="70"
+ x="321.30356"
+ style="font-style:normal;font-weight:normal;line-height:0%;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none"
+ xml:space="preserve"><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:24.2109px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="70"
+ x="321.30356"
+ id="tspan29165-9-7-7"
+ sodipodi:role="line">DMA Device</tspan></text>
+ </g>
+ <g
+ id="g29167-4-3-1-9"
+ transform="matrix(0.73232502,0,0,0.75477602,-114.48565,314.20704)">
+ <text
+ id="text29163-9-6-8-2"
+ y="70"
+ x="368.01718"
+ style="font-style:normal;font-weight:normal;line-height:0%;font-family:sans-serif;text-align:center;letter-spacing:0px;word-spacing:0px;text-anchor:middle;fill:#000000;fill-opacity:1;stroke:none"
+ xml:space="preserve"><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:24.2109px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:center;writing-mode:lr-tb;text-anchor:middle"
+ y="70"
+ x="368.01718"
+ id="tspan29165-9-7-7-0"
+ sodipodi:role="line">DMA</tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:24.2109px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:center;writing-mode:lr-tb;text-anchor:middle"
+ y="100.26363"
+ x="368.01718"
+ sodipodi:role="line"
+ id="tspan3488">Adapter</tspan></text>
+ </g>
+ <g
+ id="g29167-4-3-1-9-2"
+ transform="matrix(0.73232502,0,0,0.75477602,250.96804,192.62529)">
+ <text
+ id="text29163-9-6-8-2-3"
+ y="-188.35481"
+ x="318.61978"
+ style="font-style:normal;font-weight:normal;line-height:0%;font-family:sans-serif;text-align:center;letter-spacing:0px;word-spacing:0px;text-anchor:middle;fill:#000000;fill-opacity:1;stroke:none"
+ xml:space="preserve"><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:21.5208px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="-188.35481"
+ x="318.61978"
+ sodipodi:role="line"
+ id="tspan3543">1. Application dequeues</tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:21.5208px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="-161.45381"
+ x="318.61978"
+ sodipodi:role="line"
+ id="tspan3196"> events from the previous</tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:21.5208px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="-134.55281"
+ x="318.61978"
+ sodipodi:role="line"
+ id="tspan3232"> stage</tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:21.5208px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="-107.65182"
+ x="318.61978"
+ sodipodi:role="line"
+ id="tspan3519"> </tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:21.5208px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="-80.750816"
+ x="318.61978"
+ sodipodi:role="line"
+ id="tspan3551">2. Application prepares the</tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:21.5208px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="-53.849815"
+ x="318.61978"
+ sodipodi:role="line"
+ id="tspan3203"> DMA operations.</tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:21.5208px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="-26.948814"
+ x="318.61978"
+ sodipodi:role="line"
+ id="tspan3523"> </tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:21.5208px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="-0.0478158"
+ x="318.61978"
+ sodipodi:role="line"
+ id="tspan3541">3. DMA operations are</tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:21.5208px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="26.853184"
+ x="318.61978"
+ sodipodi:role="line"
+ id="tspan3207"> submitted to dmadev</tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:21.5208px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="53.754185"
+ x="318.61978"
+ sodipodi:role="line"
+ id="tspan3209"> by application.</tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:21.5208px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="80.655182"
+ x="318.61978"
+ sodipodi:role="line"
+ id="tspan3527"> </tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:21.5208px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="107.55618"
+ x="318.61978"
+ sodipodi:role="line"
+ id="tspan3547">4. DMA adapter dequeues</tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:21.5208px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="134.45718"
+ x="318.61978"
+ sodipodi:role="line"
+ id="tspan3216"> DMA completions from</tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:21.5208px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="161.35818"
+ x="318.61978"
+ sodipodi:role="line"
+ id="tspan3218"> DMA device.</tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:21.5208px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="188.25919"
+ x="318.61978"
+ sodipodi:role="line"
+ id="tspan3531"> </tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:21.5208px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="215.16019"
+ x="318.61978"
+ sodipodi:role="line"
+ id="tspan3549">5. DMA adapter enqueues</tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:21.5208px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="242.06119"
+ x="318.61978"
+ sodipodi:role="line"
+ id="tspan3222"> events to the eventdev.</tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:21.5208px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="268.96219"
+ x="318.61978"
+ sodipodi:role="line"
+ id="tspan3535"> </tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:21.5208px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="295.86319"
+ x="318.61978"
+ sodipodi:role="line"
+ id="tspan3537">6. Application dequeues from</tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:21.5208px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="322.76419"
+ x="318.61978"
+ sodipodi:role="line"
+ id="tspan3224"> eventdev and prepare for</tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:21.5208px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="349.66519"
+ x="318.61978"
+ sodipodi:role="line"
+ id="tspan3226"> further processing</tspan></text>
+ </g>
+ <g
+ style="font-size:12px;fill:none;stroke-linecap:square;stroke-miterlimit:3;overflow:visible"
+ id="shape1-1-2-4-7"
+ v:mID="1"
+ v:groupContext="shape"
+ transform="matrix(2.1604167,0,0,1.165886,90.820551,-587.97129)">
+ <title
+ id="title22-7-5-5">Square</title>
+ <desc
+ id="desc24-7-8-3">Atomic Queue #1</desc>
+ <v:userDefs>
+ <v:ud
+ v:nameU="visVersion"
+ v:val="VT0(15):26" />
+ </v:userDefs>
+ <v:textBlock
+ v:margins="rect(4,4,4,4)" />
+ <v:textRect
+ cx="30.75"
+ cy="581.25"
+ width="61.5"
+ height="61.5" />
+ <g
+ id="shadow1-2-9-5-5"
+ v:groupContext="shadow"
+ v:shadowOffsetX="0.345598"
+ v:shadowOffsetY="-1.97279"
+ v:shadowType="1"
+ transform="translate(0.345598,1.97279)"
+ class="st1"
+ style="visibility:visible">
+ <rect
+ x="0"
+ y="550.5"
+ width="61.5"
+ height="61.5"
+ class="st2"
+ id="rect27-8-7-6"
+ style="fill:#5b9bd5;fill-opacity:0.22000002;stroke:#5b9bd5;stroke-opacity:0.22000002;filter:url(#filter_2-3-91-3)" />
+ </g>
+ <g
+ id="g13515-33-2">
+ <g
+ id="g13534-8-9">
+ <rect
+ x="0"
+ y="550.5"
+ width="61.5"
+ height="61.5"
+ class="st3"
+ id="rect29-1-95-1"
+ style="fill:#5b9bd5;stroke:#c7c8c8;stroke-width:0.25" />
+ </g>
+ </g>
+ </g>
+ <g
+ id="g29167-4-3-2"
+ transform="matrix(0.73232502,0,0,0.75477602,-125.66199,44.027402)">
+ <text
+ id="text29163-9-6-7"
+ y="70"
+ x="321.30356"
+ style="font-style:normal;font-weight:normal;line-height:0%;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none"
+ xml:space="preserve"><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:24.2109px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="70"
+ x="321.30356"
+ id="tspan29165-9-7-0"
+ sodipodi:role="line">Application</tspan></text>
+ </g>
+ </g>
+</svg>
diff --git a/doc/guides/prog_guide/index.rst b/doc/guides/prog_guide/index.rst
index 52a6d9e7aa..beaa4b8869 100644
--- a/doc/guides/prog_guide/index.rst
+++ b/doc/guides/prog_guide/index.rst
@@ -60,6 +60,7 @@ Programmer's Guide
event_ethernet_tx_adapter
event_timer_adapter
event_crypto_adapter
+ event_dma_adapter
qos_framework
power_man
packet_classif_access_ctrl
diff --git a/doc/guides/rel_notes/release_23_11.rst b/doc/guides/rel_notes/release_23_11.rst
index b34ddc0860..1a1f337d23 100644
--- a/doc/guides/rel_notes/release_23_11.rst
+++ b/doc/guides/rel_notes/release_23_11.rst
@@ -89,6 +89,11 @@ New Features
* Added support for ``remaining_ticks_get`` timer adapter PMD callback
to get the remaining ticks to expire for a given event timer.
+* **Added event DMA adapter library.**
+
+ * Added the Event DMA Adapter Library. This library extends the event-based
+ model by introducing APIs that allow applications to enqueue/dequeue DMA
+ operations to/from dmadev as events scheduled by an event device.
Removed Items
-------------
diff --git a/lib/eventdev/eventdev_pmd.h b/lib/eventdev/eventdev_pmd.h
index f62f42e140..f7227c0bfd 100644
--- a/lib/eventdev/eventdev_pmd.h
+++ b/lib/eventdev/eventdev_pmd.h
@@ -178,8 +178,12 @@ struct rte_eventdev {
event_tx_adapter_enqueue_t txa_enqueue;
/**< Pointer to PMD eth Tx adapter enqueue function. */
event_crypto_adapter_enqueue_t ca_enqueue;
+ /**< Pointer to PMD crypto adapter enqueue function. */
- uint64_t reserved_64s[4]; /**< Reserved for future fields */
+ event_dma_adapter_enqueue_t dma_enqueue;
+ /**< Pointer to PMD DMA adapter enqueue function. */
+
+ uint64_t reserved_64s[3]; /**< Reserved for future fields */
void *reserved_ptrs[3]; /**< Reserved for future fields */
} __rte_cache_aligned;
@@ -1320,6 +1324,156 @@ typedef int (*eventdev_eth_tx_adapter_queue_stop)
#define eventdev_stop_flush_t rte_eventdev_stop_flush_t
+/**
+ * Retrieve the event device's DMA adapter capabilities for the
+ * specified DMA device
+ *
+ * @param dev
+ * Event device pointer
+ *
+ * @param dma_dev_id
+ * DMA device identifier
+ *
+ * @param[out] caps
+ * A pointer to memory filled with event adapter capabilities.
+ * It is expected to be pre-allocated & initialized by caller.
+ *
+ * @return
+ * - 0: Success, driver provides event adapter capabilities for the
+ * dmadev.
+ * - <0: Error code returned by the driver function.
+ *
+ */
+typedef int (*eventdev_dma_adapter_caps_get_t)(const struct rte_eventdev *dev,
+ const int16_t dma_dev_id, uint32_t *caps);
+
+/**
+ * Add DMA vchan queue to event device. This callback is invoked if
+ * the caps returned from rte_event_dma_adapter_caps_get(, dmadev_id)
+ * has RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_* set.
+ *
+ * @param dev
+ * Event device pointer
+ *
+ * @param dma_dev_id
+ * DMA device identifier
+ *
+ * @param vchan_id
+ * dmadev vchan queue identifier.
+ *
+ * @param event
+ * Event information required for binding dmadev vchan to event queue.
+ * This structure will have a valid value for only those HW PMDs supporting
+ * @see RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_VCHAN_EV_BIND capability.
+ *
+ * @return
+ * - 0: Success, dmadev vchan added successfully.
+ * - <0: Error code returned by the driver function.
+ *
+ */
+typedef int (*eventdev_dma_adapter_vchan_add_t)(const struct rte_eventdev *dev,
+ const int16_t dma_dev_id,
+ uint16_t vchan_id,
+ const struct rte_event *event);
+
+/**
+ * Delete DMA vhcan to event device. This callback is invoked if
+ * the caps returned from rte_event_dma_adapter_caps_get(, dmadev_id)
+ * has RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_* set.
+ *
+ * @param dev
+ * Event device pointer
+ *
+ * @param dma_dev_id
+ * DMA device identifier
+ *
+ * @param vchan_id
+ * dmadev vchan identifier.
+ *
+ * @return
+ * - 0: Success, dmadev vchan deleted successfully.
+ * - <0: Error code returned by the driver function.
+ *
+ */
+typedef int (*eventdev_dma_adapter_vchan_del_t)(const struct rte_eventdev *dev,
+ const int16_t dma_dev_id,
+ uint16_t vchan_id);
+
+/**
+ * Start DMA adapter. This callback is invoked if
+ * the caps returned from rte_event_dma_adapter_caps_get(.., dmadev_id)
+ * has RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_* set and vchan for dmadev_id
+ * have been added to the event device.
+ *
+ * @param dev
+ * Event device pointer
+ *
+ * @param dma_dev_id
+ * DMA device identifier
+ *
+ * @return
+ * - 0: Success, DMA adapter started successfully.
+ * - <0: Error code returned by the driver function.
+ */
+typedef int (*eventdev_dma_adapter_start_t)(const struct rte_eventdev *dev,
+ const int16_t dma_dev_id);
+
+/**
+ * Stop DMA adapter. This callback is invoked if
+ * the caps returned from rte_event_dma_adapter_caps_get(.., dmadev_id)
+ * has RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_* set and vchan for dmadev_id
+ * have been added to the event device.
+ *
+ * @param dev
+ * Event device pointer
+ *
+ * @param dma_dev_id
+ * DMA device identifier
+ *
+ * @return
+ * - 0: Success, DMA adapter stopped successfully.
+ * - <0: Error code returned by the driver function.
+ */
+typedef int (*eventdev_dma_adapter_stop_t)(const struct rte_eventdev *dev,
+ const int16_t dma_dev_id);
+
+struct rte_event_dma_adapter_stats;
+
+/**
+ * Retrieve DMA adapter statistics.
+ *
+ * @param dev
+ * Event device pointer
+ *
+ * @param dma_dev_id
+ * DMA device identifier
+ *
+ * @param[out] stats
+ * Pointer to stats structure
+ *
+ * @return
+ * Return 0 on success.
+ */
+typedef int (*eventdev_dma_adapter_stats_get)(const struct rte_eventdev *dev,
+ const int16_t dma_dev_id,
+ struct rte_event_dma_adapter_stats *stats);
+
+/**
+ * Reset DMA adapter statistics.
+ *
+ * @param dev
+ * Event device pointer
+ *
+ * @param dma_dev_id
+ * DMA device identifier
+ *
+ * @return
+ * Return 0 on success.
+ */
+typedef int (*eventdev_dma_adapter_stats_reset)(const struct rte_eventdev *dev,
+ const int16_t dma_dev_id);
+
+
/** Event device operations function pointer table */
struct eventdev_ops {
eventdev_info_get_t dev_infos_get; /**< Get device info. */
@@ -1440,6 +1594,21 @@ struct eventdev_ops {
eventdev_eth_tx_adapter_queue_stop eth_tx_adapter_queue_stop;
/**< Stop Tx queue assigned to Tx adapter instance */
+ eventdev_dma_adapter_caps_get_t dma_adapter_caps_get;
+ /**< Get DMA adapter capabilities */
+ eventdev_dma_adapter_vchan_add_t dma_adapter_vchan_add;
+ /**< Add vchan queue to DMA adapter */
+ eventdev_dma_adapter_vchan_del_t dma_adapter_vchan_del;
+ /**< Delete vchan queue from DMA adapter */
+ eventdev_dma_adapter_start_t dma_adapter_start;
+ /**< Start DMA adapter */
+ eventdev_dma_adapter_stop_t dma_adapter_stop;
+ /**< Stop DMA adapter */
+ eventdev_dma_adapter_stats_get dma_adapter_stats_get;
+ /**< Get DMA stats */
+ eventdev_dma_adapter_stats_reset dma_adapter_stats_reset;
+ /**< Reset DMA stats */
+
eventdev_selftest dev_selftest;
/**< Start eventdev Selftest */
diff --git a/lib/eventdev/eventdev_private.c b/lib/eventdev/eventdev_private.c
index 1d3d9d357e..18ed8bf3c8 100644
--- a/lib/eventdev/eventdev_private.c
+++ b/lib/eventdev/eventdev_private.c
@@ -81,6 +81,14 @@ dummy_event_crypto_adapter_enqueue(__rte_unused void *port,
return 0;
}
+static uint16_t
+dummy_event_dma_adapter_enqueue(__rte_unused void *port, __rte_unused struct rte_event ev[],
+ __rte_unused uint16_t nb_events)
+{
+ RTE_EDEV_LOG_ERR("event DMA adapter enqueue requested for unconfigured event device");
+ return 0;
+}
+
void
event_dev_fp_ops_reset(struct rte_event_fp_ops *fp_op)
{
@@ -97,6 +105,7 @@ event_dev_fp_ops_reset(struct rte_event_fp_ops *fp_op)
.txa_enqueue_same_dest =
dummy_event_tx_adapter_enqueue_same_dest,
.ca_enqueue = dummy_event_crypto_adapter_enqueue,
+ .dma_enqueue = dummy_event_dma_adapter_enqueue,
.data = dummy_data,
};
@@ -117,5 +126,6 @@ event_dev_fp_ops_set(struct rte_event_fp_ops *fp_op,
fp_op->txa_enqueue = dev->txa_enqueue;
fp_op->txa_enqueue_same_dest = dev->txa_enqueue_same_dest;
fp_op->ca_enqueue = dev->ca_enqueue;
+ fp_op->dma_enqueue = dev->dma_enqueue;
fp_op->data = dev->data->ports;
}
diff --git a/lib/eventdev/meson.build b/lib/eventdev/meson.build
index 6edf98dfa5..21347f7c4c 100644
--- a/lib/eventdev/meson.build
+++ b/lib/eventdev/meson.build
@@ -25,6 +25,7 @@ sources = files(
)
headers = files(
'rte_event_crypto_adapter.h',
+ 'rte_event_dma_adapter.h',
'rte_event_eth_rx_adapter.h',
'rte_event_eth_tx_adapter.h',
'rte_event_ring.h',
diff --git a/lib/eventdev/rte_event_dma_adapter.h b/lib/eventdev/rte_event_dma_adapter.h
new file mode 100644
index 0000000000..e924ab673d
--- /dev/null
+++ b/lib/eventdev/rte_event_dma_adapter.h
@@ -0,0 +1,581 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright (c) 2023 Marvell.
+ */
+
+#ifndef RTE_EVENT_DMA_ADAPTER
+#define RTE_EVENT_DMA_ADAPTER
+
+/**
+ * @file rte_event_dma_adapter.h
+ *
+ * @warning
+ * @b EXPERIMENTAL:
+ * All functions in this file may be changed or removed without prior notice.
+ *
+ * DMA Event Adapter API.
+ *
+ * Eventdev library provides adapters to bridge between various components for providing new
+ * event source. The event DMA adapter is one of those adapters which is intended to bridge
+ * between event devices and DMA devices.
+ *
+ * The DMA adapter adds support to enqueue / dequeue DMA operations to / from event device. The
+ * packet flow between DMA device and the event device can be accomplished using both SW and HW
+ * based transfer mechanisms. The adapter uses an EAL service core function for SW based packet
+ * transfer and uses the eventdev PMD functions to configure HW based packet transfer between the
+ * DMA device and the event device.
+ *
+ * The application can choose to submit a DMA operation directly to an DMA device or send it to the
+ * DMA adapter via eventdev based on RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP_FWD capability. The
+ * first mode is known as the event new (RTE_EVENT_DMA_ADAPTER_OP_NEW) mode and the second as the
+ * event forward (RTE_EVENT_DMA_ADAPTER_OP_FORWARD) mode. The choice of mode can be specified while
+ * creating the adapter. In the former mode, it is an application responsibility to enable ingress
+ * packet ordering. In the latter mode, it is the adapter responsibility to enable the ingress
+ * packet ordering.
+ *
+ *
+ * Working model of RTE_EVENT_DMA_ADAPTER_OP_NEW mode:
+ *
+ * +--------------+ +--------------+
+ * | | | DMA stage |
+ * | Application |---[2]-->| + enqueue to |
+ * | | | dmadev |
+ * +--------------+ +--------------+
+ * ^ ^ |
+ * | | [3]
+ * [6] [1] |
+ * | | |
+ * +--------------+ |
+ * | | |
+ * | Event device | |
+ * | | |
+ * +--------------+ |
+ * ^ |
+ * | |
+ * [5] |
+ * | v
+ * +--------------+ +--------------+
+ * | | | |
+ * | DMA adapter |<--[4]---| dmadev |
+ * | | | |
+ * +--------------+ +--------------+
+ *
+ *
+ * [1] Application dequeues events from the previous stage.
+ * [2] Application prepares the DMA operations.
+ * [3] DMA operations are submitted to dmadev by application.
+ * [4] DMA adapter dequeues DMA completions from dmadev.
+ * [5] DMA adapter enqueues events to the eventdev.
+ * [6] Application dequeues from eventdev for further processing.
+ *
+ * In the RTE_EVENT_DMA_ADAPTER_OP_NEW mode, application submits DMA operations directly to DMA
+ * device. The DMA adapter then dequeues DMA completions from DMA device and enqueue events to the
+ * event device. This mode does not ensure ingress ordering, if the application directly enqueues
+ * to dmadev without going through DMA / atomic stage i.e. removing item [1] and [2].
+ *
+ * Events dequeued from the adapter will be treated as new events. In this mode, application needs
+ * to specify event information (response information) which is needed to enqueue an event after the
+ * DMA operation is completed.
+ *
+ *
+ * Working model of RTE_EVENT_DMA_ADAPTER_OP_FORWARD mode:
+ *
+ * +--------------+ +--------------+
+ * --[1]-->| |---[2]-->| Application |
+ * | Event device | | in |
+ * <--[8]--| |<--[3]---| Ordered stage|
+ * +--------------+ +--------------+
+ * ^ |
+ * | [4]
+ * [7] |
+ * | v
+ * +----------------+ +--------------+
+ * | |--[5]->| |
+ * | DMA adapter | | dmadev |
+ * | |<-[6]--| |
+ * +----------------+ +--------------+
+ *
+ *
+ * [1] Events from the previous stage.
+ * [2] Application in ordered stage dequeues events from eventdev.
+ * [3] Application enqueues DMA operations as events to eventdev.
+ * [4] DMA adapter dequeues event from eventdev.
+ * [5] DMA adapter submits DMA operations to dmadev (Atomic stage).
+ * [6] DMA adapter dequeues DMA completions from dmadev
+ * [7] DMA adapter enqueues events to the eventdev
+ * [8] Events to the next stage
+ *
+ * In the event forward (RTE_EVENT_DMA_ADAPTER_OP_FORWARD) mode, if the HW supports the capability
+ * RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP_FWD, application can directly submit the DMA
+ * operations to the dmadev. If not, application retrieves the event port of the DMA adapter
+ * through the API, rte_event_DMA_adapter_event_port_get(). Then, links its event queue to this
+ * port and starts enqueuing DMA operations as events to the eventdev. The adapter then dequeues
+ * the events and submits the DMA operations to the dmadev. After the DMA completions, the adapter
+ * enqueues events to the event device.
+ *
+ * Application can use this mode, when ingress packet ordering is needed. Events dequeued from the
+ * adapter will be treated as forwarded events. In this mode, the application needs to specify the
+ * dmadev ID and queue pair ID (request information) needed to enqueue an DMA operation in addition
+ * to the event information (response information) needed to enqueue an event after the DMA
+ * operation has completed.
+ *
+ * The event DMA adapter provides common APIs to configure the packet flow from the DMA device to
+ * event devices for both SW and HW based transfers. The DMA event adapter's functions are:
+ *
+ * - rte_event_dma_adapter_create_ext()
+ * - rte_event_dma_adapter_create()
+ * - rte_event_dma_adapter_free()
+ * - rte_event_dma_adapter_vchan_add()
+ * - rte_event_dma_adapter_vchan_del()
+ * - rte_event_dma_adapter_start()
+ * - rte_event_dma_adapter_stop()
+ * - rte_event_dma_adapter_stats_get()
+ * - rte_event_dma_adapter_stats_reset()
+ *
+ * The application creates an instance using rte_event_dma_adapter_create() or
+ * rte_event_dma_adapter_create_ext().
+ *
+ * dmadev queue pair addition / deletion is done using the rte_event_dma_adapter_vchan_add() /
+ * rte_event_dma_adapter_vchan_del() APIs. If HW supports the capability
+ * RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_VCHAN_EV_BIND, event information must be passed to the
+ * add API.
+ *
+ */
+
+#include <stdint.h>
+
+#include <rte_common.h>
+#include <rte_dmadev_pmd.h>
+#include <rte_eventdev.h>
+
+#ifdef __cplusplus
+extern "C" {
+#endif
+
+/**
+ * A structure used to hold event based DMA operation entry. All the information
+ * required for a DMA transfer shall be populated in "struct rte_event_dma_adapter_op"
+ * instance.
+ */
+struct rte_event_dma_adapter_op {
+ struct rte_dma_sge *src_seg;
+ /**< Source segments. */
+ struct rte_dma_sge *dst_seg;
+ /**< Destination segments. */
+ uint16_t nb_src;
+ /**< Number of source segments. */
+ uint16_t nb_dst;
+ /**< Number of destination segments. */
+ uint64_t flags;
+ /**< Flags related to the operation.
+ * @see RTE_DMA_OP_FLAG_*
+ */
+ int16_t dma_dev_id;
+ /**< DMA device ID to be used */
+ uint16_t vchan;
+ /**< DMA vchan ID to be used */
+ struct rte_mempool *op_mp;
+ /**< Mempool from which op is allocated. */
+};
+
+/**
+ * DMA event adapter mode
+ */
+enum rte_event_dma_adapter_mode {
+ RTE_EVENT_DMA_ADAPTER_OP_NEW,
+ /**< Start the DMA adapter in event new mode.
+ * @see RTE_EVENT_OP_NEW.
+ *
+ * Application submits DMA operations to the dmadev. Adapter only dequeues the DMA
+ * completions from dmadev and enqueue events to the eventdev.
+ */
+
+ RTE_EVENT_DMA_ADAPTER_OP_FORWARD,
+ /**< Start the DMA adapter in event forward mode.
+ * @see RTE_EVENT_OP_FORWARD.
+ *
+ * Application submits DMA requests as events to the DMA adapter or DMA device based on
+ * RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP_FWD capability. DMA completions are enqueued
+ * back to the eventdev by DMA adapter.
+ */
+};
+
+/**
+ * Adapter configuration structure that the adapter configuration callback function is expected to
+ * fill out.
+ *
+ * @see rte_event_dma_adapter_conf_cb
+ */
+struct rte_event_dma_adapter_conf {
+ uint8_t event_port_id;
+ /** < Event port identifier, the adapter enqueues events to this port and dequeues DMA
+ * request events in RTE_EVENT_DMA_ADAPTER_OP_FORWARD mode.
+ */
+
+ uint32_t max_nb;
+ /**< The adapter can return early if it has processed at least max_nb DMA ops. This isn't
+ * treated as a requirement; batching may cause the adapter to process more than max_nb DMA
+ * ops.
+ */
+};
+
+/**
+ * Adapter runtime configuration parameters
+ */
+struct rte_event_dma_adapter_runtime_params {
+ uint32_t max_nb;
+ /**< The adapter can return early if it has processed at least max_nb DMA ops. This isn't
+ * treated as a requirement; batching may cause the adapter to process more than max_nb DMA
+ * ops.
+ *
+ * Callback function passed to rte_event_dma_adapter_create_ext() configures the adapter
+ * with default value of max_nb.
+ * rte_event_dma_adapter_runtime_params_set() allows to re-configure max_nb during runtime
+ * (after adding at least one queue pair)
+ *
+ * This is valid for the devices without RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP_FWD or
+ * RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP_NEW capability.
+ */
+
+ uint32_t rsvd[15];
+ /**< Reserved fields for future expansion */
+};
+
+/**
+ * Function type used for adapter configuration callback. The callback is used to fill in members of
+ * the struct rte_event_dma_adapter_conf, this callback is invoked when creating a SW service for
+ * packet transfer from dmadev vchan to the event device. The SW service is created within the
+ * function, rte_event_dma_adapter_vchan_add(), if SW based packet transfers from dmadev vchan
+ * to the event device are required.
+ *
+ * @param id
+ * Adapter identifier.
+ * @param evdev_id
+ * Event device identifier.
+ * @param conf
+ * Structure that needs to be populated by this callback.
+ * @param arg
+ * Argument to the callback. This is the same as the conf_arg passed to the
+ * rte_event_dma_adapter_create_ext().
+ */
+typedef int (*rte_event_dma_adapter_conf_cb)(uint8_t id, uint8_t evdev_id,
+ struct rte_event_dma_adapter_conf *conf, void *arg);
+
+/**
+ * A structure used to retrieve statistics for an event DMA adapter instance.
+ */
+struct rte_event_dma_adapter_stats {
+ uint64_t event_poll_count;
+ /**< Event port poll count */
+
+ uint64_t event_deq_count;
+ /**< Event dequeue count */
+
+ uint64_t dma_enq_count;
+ /**< dmadev enqueue count */
+
+ uint64_t dma_enq_fail_count;
+ /**< dmadev enqueue failed count */
+
+ uint64_t dma_deq_count;
+ /**< dmadev dequeue count */
+
+ uint64_t event_enq_count;
+ /**< Event enqueue count */
+
+ uint64_t event_enq_retry_count;
+ /**< Event enqueue retry count */
+
+ uint64_t event_enq_fail_count;
+ /**< Event enqueue fail count */
+};
+
+/**
+ * Create a new event DMA adapter with the specified identifier.
+ *
+ * @param id
+ * Adapter identifier.
+ * @param evdev_id
+ * Event device identifier.
+ * @param conf_cb
+ * Callback function that fills in members of a struct rte_event_dma_adapter_conf struct passed
+ * into it.
+ * @param mode
+ * Flag to indicate the mode of the adapter.
+ * @see rte_event_dma_adapter_mode
+ * @param conf_arg
+ * Argument that is passed to the conf_cb function.
+ *
+ * @return
+ * - 0: Success
+ * - <0: Error code on failure
+ */
+__rte_experimental
+int rte_event_dma_adapter_create_ext(uint8_t id, uint8_t evdev_id,
+ rte_event_dma_adapter_conf_cb conf_cb,
+ enum rte_event_dma_adapter_mode mode, void *conf_arg);
+
+/**
+ * Create a new event DMA adapter with the specified identifier. This function uses an internal
+ * configuration function that creates an event port. This default function reconfigures the event
+ * device with an additional event port and set up the event port using the port_config parameter
+ * passed into this function. In case the application needs more control in configuration of the
+ * service, it should use the rte_event_dma_adapter_create_ext() version.
+ *
+ * @param id
+ * Adapter identifier.
+ * @param evdev_id
+ * Event device identifier.
+ * @param port_config
+ * Argument of type *rte_event_port_conf* that is passed to the conf_cb function.
+ * @param mode
+ * Flag to indicate the mode of the adapter.
+ * @see rte_event_dma_adapter_mode
+ *
+ * @return
+ * - 0: Success
+ * - <0: Error code on failure
+ */
+__rte_experimental
+int rte_event_dma_adapter_create(uint8_t id, uint8_t evdev_id,
+ struct rte_event_port_conf *port_config,
+ enum rte_event_dma_adapter_mode mode);
+
+/**
+ * Free an event DMA adapter
+ *
+ * @param id
+ * Adapter identifier.
+ * @return
+ * - 0: Success
+ * - <0: Error code on failure, If the adapter still has queue pairs added to it, the function
+ * returns -EBUSY.
+ */
+__rte_experimental
+int rte_event_dma_adapter_free(uint8_t id);
+
+/**
+ * Retrieve the event port of an adapter.
+ *
+ * @param id
+ * Adapter identifier.
+ *
+ * @param [out] event_port_id
+ * Application links its event queue to this adapter port which is used in
+ * RTE_EVENT_DMA_ADAPTER_OP_FORWARD mode.
+ *
+ * @return
+ * - 0: Success
+ * - <0: Error code on failure.
+ */
+__rte_experimental
+int rte_event_dma_adapter_event_port_get(uint8_t id, uint8_t *event_port_id);
+
+/**
+ * Add a vchan to an event DMA adapter.
+ *
+ * @param id
+ * Adapter identifier.
+ * @param dmadev_id
+ * dmadev identifier.
+ * @param vchan
+ * DMA device vchan identifier. If vchan is set -1, adapter adds all the
+ * preconfigured vchan to the instance.
+ * @param event
+ * If HW supports dmadev vchan to event queue binding, application is expected to fill in
+ * event information, else it will be NULL.
+ * @see RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_VCHAN_EV_BIND
+ *
+ * @return
+ * - 0: Success, vchan added correctly.
+ * - <0: Error code on failure.
+ */
+__rte_experimental
+int rte_event_dma_adapter_vchan_add(uint8_t id, int16_t dmadev_id, uint16_t vchan,
+ const struct rte_event *event);
+
+/**
+ * Delete a vchan from an event DMA adapter.
+ *
+ * @param id
+ * Adapter identifier.
+ * @param dmadev_id
+ * DMA device identifier.
+ * @param vchan
+ * DMA device vchan identifier.
+ *
+ * @return
+ * - 0: Success, vchan deleted successfully.
+ * - <0: Error code on failure.
+ */
+__rte_experimental
+int rte_event_dma_adapter_vchan_del(uint8_t id, int16_t dmadev_id, uint16_t vchan);
+
+/**
+ * Retrieve the service ID of an adapter. If the adapter doesn't use a rte_service function, this
+ * function returns -ESRCH.
+ *
+ * @param id
+ * Adapter identifier.
+ * @param [out] service_id
+ * A pointer to a uint32_t, to be filled in with the service id.
+ *
+ * @return
+ * - 0: Success
+ * - <0: Error code on failure, if the adapter doesn't use a rte_service function, this function
+ * returns -ESRCH.
+ */
+__rte_experimental
+int rte_event_dma_adapter_service_id_get(uint8_t id, uint32_t *service_id);
+
+/**
+ * Start event DMA adapter
+ *
+ * @param id
+ * Adapter identifier.
+ *
+ * @return
+ * - 0: Success, adapter started successfully.
+ * - <0: Error code on failure.
+ *
+ * @note The eventdev and dmadev to which the event_dma_adapter is connected should be started
+ * before calling rte_event_dma_adapter_start().
+ */
+__rte_experimental
+int rte_event_dma_adapter_start(uint8_t id);
+
+/**
+ * Stop event DMA adapter
+ *
+ * @param id
+ * Adapter identifier.
+ *
+ * @return
+ * - 0: Success, adapter stopped successfully.
+ * - <0: Error code on failure.
+ */
+__rte_experimental
+int rte_event_dma_adapter_stop(uint8_t id);
+
+/**
+ * Initialize the adapter runtime configuration parameters
+ *
+ * @param params
+ * A pointer to structure of type struct rte_event_dma_adapter_runtime_params
+ *
+ * @return
+ * - 0: Success
+ * - <0: Error code on failure
+ */
+__rte_experimental
+int rte_event_dma_adapter_runtime_params_init(struct rte_event_dma_adapter_runtime_params *params);
+
+/**
+ * Set the adapter runtime configuration parameters
+ *
+ * @param id
+ * Adapter identifier
+ *
+ * @param params
+ * A pointer to structure of type struct rte_event_dma_adapter_runtime_params with configuration
+ * parameter values. The reserved fields of this structure must be initialized to zero and the valid
+ * fields need to be set appropriately. This struct can be initialized using
+ * rte_event_dma_adapter_runtime_params_init() API to default values or application may reset this
+ * struct and update required fields.
+ *
+ * @return
+ * - 0: Success
+ * - <0: Error code on failure
+ */
+__rte_experimental
+int rte_event_dma_adapter_runtime_params_set(uint8_t id,
+ struct rte_event_dma_adapter_runtime_params *params);
+
+/**
+ * Get the adapter runtime configuration parameters
+ *
+ * @param id
+ * Adapter identifier
+ *
+ * @param[out] params
+ * A pointer to structure of type struct rte_event_dma_adapter_runtime_params containing valid
+ * adapter parameters when return value is 0.
+ *
+ * @return
+ * - 0: Success
+ * - <0: Error code on failure
+ */
+__rte_experimental
+int rte_event_dma_adapter_runtime_params_get(uint8_t id,
+ struct rte_event_dma_adapter_runtime_params *params);
+
+/**
+ * Retrieve statistics for an adapter
+ *
+ * @param id
+ * Adapter identifier.
+ * @param [out] stats
+ * A pointer to structure used to retrieve statistics for an adapter.
+ *
+ * @return
+ * - 0: Success, retrieved successfully.
+ * - <0: Error code on failure.
+ */
+__rte_experimental
+int rte_event_dma_adapter_stats_get(uint8_t id, struct rte_event_dma_adapter_stats *stats);
+
+/**
+ * Reset statistics for an adapter.
+ *
+ * @param id
+ * Adapter identifier.
+ *
+ * @return
+ * - 0: Success, statistics reset successfully.
+ * - <0: Error code on failure.
+ */
+__rte_experimental
+int rte_event_dma_adapter_stats_reset(uint8_t id);
+
+/**
+ * Enqueue a burst of DMA operations as event objects supplied in *rte_event* structure on an event
+ * DMA adapter designated by its event *evdev_id* through the event port specified by *port_id*.
+ * This function is supported if the eventdev PMD has the
+ * #RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP_FWD capability flag set.
+ *
+ * The *nb_events* parameter is the number of event objects to enqueue that are supplied in the
+ * *ev* array of *rte_event* structure.
+ *
+ * The rte_event_dma_adapter_enqueue() function returns the number of event objects it actually
+ * enqueued. A return value equal to *nb_events* means that all event objects have been enqueued.
+ *
+ * @param evdev_id
+ * The identifier of the device.
+ * @param port_id
+ * The identifier of the event port.
+ * @param ev
+ * Points to an array of *nb_events* objects of type *rte_event* structure which contain the
+ * event object enqueue operations to be processed.
+ * @param nb_events
+ * The number of event objects to enqueue, typically number of
+ * rte_event_port_attr_get(...RTE_EVENT_PORT_ATTR_ENQ_DEPTH...) available for this port.
+ *
+ * @return
+ * The number of event objects actually enqueued on the event device. The return value can be
+ * less than the value of the *nb_events* parameter when the event devices queue is full or if
+ * invalid parameters are specified in a *rte_event*. If the return value is less than *nb_events*,
+ * the remaining events at the end of ev[] are not consumed and the caller has to take care of them,
+ * and rte_errno is set accordingly. Possible errno values include:
+ * - EINVAL: The port ID is invalid, device ID is invalid, an event's queue ID is invalid, or an
+ * event's sched type doesn't match the capabilities of the destination queue.
+ * - ENOSPC: The event port was backpressured and unable to enqueue one or more events. This
+ * error code is only applicable to closed systems.
+ */
+__rte_experimental
+uint16_t rte_event_dma_adapter_enqueue(uint8_t evdev_id, uint8_t port_id, struct rte_event ev[],
+ uint16_t nb_events);
+
+#ifdef __cplusplus
+}
+#endif
+
+#endif /* RTE_EVENT_DMA_ADAPTER */
diff --git a/lib/eventdev/rte_eventdev.h b/lib/eventdev/rte_eventdev.h
index 2ba8a7b090..41743f91b1 100644
--- a/lib/eventdev/rte_eventdev.h
+++ b/lib/eventdev/rte_eventdev.h
@@ -1197,6 +1197,8 @@ struct rte_event_vector {
*/
#define RTE_EVENT_TYPE_ETH_RX_ADAPTER 0x4
/**< The event generated from event eth Rx adapter */
+#define RTE_EVENT_TYPE_DMADEV 0x5
+/**< The event generated from dma subsystem */
#define RTE_EVENT_TYPE_VECTOR 0x8
/**< Indicates that event is a vector.
* All vector event types should be a logical OR of EVENT_TYPE_VECTOR.
@@ -1462,6 +1464,48 @@ int
rte_event_crypto_adapter_caps_get(uint8_t dev_id, uint8_t cdev_id,
uint32_t *caps);
+/* DMA adapter capability bitmap flag */
+#define RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP_NEW 0x1
+/**< Flag indicates HW is capable of generating events in
+ * RTE_EVENT_OP_NEW enqueue operation. DMADEV will send
+ * packets to the event device as new events using an
+ * internal event port.
+ */
+
+#define RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP_FWD 0x2
+/**< Flag indicates HW is capable of generating events in
+ * RTE_EVENT_OP_FORWARD enqueue operation. DMADEV will send
+ * packets to the event device as forwarded event using an
+ * internal event port.
+ */
+
+#define RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_VCHAN_EV_BIND 0x4
+/**< Flag indicates HW is capable of mapping DMA vchan to event queue. */
+
+/**
+ * Retrieve the event device's DMA adapter capabilities for the
+ * specified dmadev device
+ *
+ * @param dev_id
+ * The identifier of the device.
+ *
+ * @param dmadev_id
+ * The identifier of the dmadev device.
+ *
+ * @param[out] caps
+ * A pointer to memory filled with event adapter capabilities.
+ * It is expected to be pre-allocated & initialized by caller.
+ *
+ * @return
+ * - 0: Success, driver provides event adapter capabilities for the
+ * dmadev device.
+ * - <0: Error code returned by the driver function.
+ *
+ */
+__rte_experimental
+int
+rte_event_dma_adapter_caps_get(uint8_t dev_id, uint8_t dmadev_id, uint32_t *caps);
+
/* Ethdev Tx adapter capability bitmap flags */
#define RTE_EVENT_ETH_TX_ADAPTER_CAP_INTERNAL_PORT 0x1
/**< This flag is sent when the PMD supports a packet transmit callback
diff --git a/lib/eventdev/rte_eventdev_core.h b/lib/eventdev/rte_eventdev_core.h
index c27a52ccc0..83e8736c71 100644
--- a/lib/eventdev/rte_eventdev_core.h
+++ b/lib/eventdev/rte_eventdev_core.h
@@ -42,6 +42,10 @@ typedef uint16_t (*event_crypto_adapter_enqueue_t)(void *port,
uint16_t nb_events);
/**< @internal Enqueue burst of events on crypto adapter */
+typedef uint16_t (*event_dma_adapter_enqueue_t)(void *port, struct rte_event ev[],
+ uint16_t nb_events);
+/**< @internal Enqueue burst of events on DMA adapter */
+
struct rte_event_fp_ops {
void **data;
/**< points to array of internal port data pointers */
@@ -65,7 +69,9 @@ struct rte_event_fp_ops {
/**< PMD Tx adapter enqueue same destination function. */
event_crypto_adapter_enqueue_t ca_enqueue;
/**< PMD Crypto adapter enqueue function. */
- uintptr_t reserved[5];
+ event_dma_adapter_enqueue_t dma_enqueue;
+ /**< PMD DMA adapter enqueue function. */
+ uintptr_t reserved[4];
} __rte_cache_aligned;
extern struct rte_event_fp_ops rte_event_fp_ops[RTE_EVENT_MAX_DEVS];
diff --git a/lib/eventdev/version.map b/lib/eventdev/version.map
index 7ce09a87bb..b81eb2919c 100644
--- a/lib/eventdev/version.map
+++ b/lib/eventdev/version.map
@@ -133,6 +133,22 @@ EXPERIMENTAL {
rte_event_timer_remaining_ticks_get;
# added in 23.11
+ rte_event_dma_adapter_caps_get;
+ rte_event_dma_adapter_create;
+ rte_event_dma_adapter_create_ext;
+ rte_event_dma_adapter_enqueue;
+ rte_event_dma_adapter_event_port_get;
+ rte_event_dma_adapter_free;
+ rte_event_dma_adapter_runtime_params_get;
+ rte_event_dma_adapter_runtime_params_init;
+ rte_event_dma_adapter_runtime_params_set;
+ rte_event_dma_adapter_service_id_get;
+ rte_event_dma_adapter_start;
+ rte_event_dma_adapter_stats_get;
+ rte_event_dma_adapter_stats_reset;
+ rte_event_dma_adapter_stop;
+ rte_event_dma_adapter_vchan_add;
+ rte_event_dma_adapter_vchan_del;
rte_event_eth_rx_adapter_create_ext_with_params;
};
diff --git a/lib/meson.build b/lib/meson.build
index 53155be8e9..f3191f10b6 100644
--- a/lib/meson.build
+++ b/lib/meson.build
@@ -33,6 +33,7 @@ libraries = [
'compressdev',
'cryptodev',
'distributor',
+ 'dmadev',
'efd',
'eventdev',
'gpudev',
@@ -48,7 +49,6 @@ libraries = [
'rawdev',
'regexdev',
'mldev',
- 'dmadev',
'rib',
'reorder',
'sched',
--
2.25.1
^ permalink raw reply [flat|nested] 110+ messages in thread
* [PATCH v7 02/12] eventdev/dma: support adapter capabilities get
2023-09-29 8:12 ` [PATCH v7 00/12] event DMA adapter library support Amit Prakash Shukla
2023-09-29 8:12 ` [PATCH v7 01/12] eventdev/dma: introduce DMA adapter Amit Prakash Shukla
@ 2023-09-29 8:12 ` Amit Prakash Shukla
2023-09-29 8:13 ` [PATCH v7 03/12] eventdev/dma: support adapter create and free Amit Prakash Shukla
` (10 subsequent siblings)
12 siblings, 0 replies; 110+ messages in thread
From: Amit Prakash Shukla @ 2023-09-29 8:12 UTC (permalink / raw)
To: Jerin Jacob
Cc: dev, fengchengwen, kevin.laatz, bruce.richardson, conor.walsh,
vattunuru, g.singh, sachin.saxena, hemant.agrawal, cheng1.jiang,
ndabilpuram, anoobj, mb, Amit Prakash Shukla
Added a new eventdev API rte_event_dma_adapter_caps_get(), to get
DMA adapter capabilities supported by the driver.
Signed-off-by: Amit Prakash Shukla <amitprakashs@marvell.com>
---
lib/eventdev/meson.build | 2 +-
lib/eventdev/rte_eventdev.c | 23 +++++++++++++++++++++++
2 files changed, 24 insertions(+), 1 deletion(-)
diff --git a/lib/eventdev/meson.build b/lib/eventdev/meson.build
index 21347f7c4c..b46bbbc9aa 100644
--- a/lib/eventdev/meson.build
+++ b/lib/eventdev/meson.build
@@ -43,5 +43,5 @@ driver_sdk_headers += files(
'event_timer_adapter_pmd.h',
)
-deps += ['ring', 'ethdev', 'hash', 'mempool', 'mbuf', 'timer', 'cryptodev']
+deps += ['ring', 'ethdev', 'hash', 'mempool', 'mbuf', 'timer', 'cryptodev', 'dmadev']
deps += ['telemetry']
diff --git a/lib/eventdev/rte_eventdev.c b/lib/eventdev/rte_eventdev.c
index 6ab4524332..60509c6efb 100644
--- a/lib/eventdev/rte_eventdev.c
+++ b/lib/eventdev/rte_eventdev.c
@@ -20,6 +20,7 @@
#include <rte_errno.h>
#include <ethdev_driver.h>
#include <rte_cryptodev.h>
+#include <rte_dmadev.h>
#include <cryptodev_pmd.h>
#include <rte_telemetry.h>
@@ -224,6 +225,28 @@ rte_event_eth_tx_adapter_caps_get(uint8_t dev_id, uint16_t eth_port_id,
: 0;
}
+int
+rte_event_dma_adapter_caps_get(uint8_t dev_id, uint8_t dma_dev_id, uint32_t *caps)
+{
+ struct rte_eventdev *dev;
+
+ RTE_EVENTDEV_VALID_DEVID_OR_ERR_RET(dev_id, -EINVAL);
+ if (!rte_dma_is_valid(dma_dev_id))
+ return -EINVAL;
+
+ dev = &rte_eventdevs[dev_id];
+
+ if (caps == NULL)
+ return -EINVAL;
+
+ *caps = 0;
+
+ if (dev->dev_ops->dma_adapter_caps_get)
+ return (*dev->dev_ops->dma_adapter_caps_get)(dev, dma_dev_id, caps);
+
+ return 0;
+}
+
static inline int
event_dev_queue_config(struct rte_eventdev *dev, uint8_t nb_queues)
{
--
2.25.1
^ permalink raw reply [flat|nested] 110+ messages in thread
* [PATCH v7 03/12] eventdev/dma: support adapter create and free
2023-09-29 8:12 ` [PATCH v7 00/12] event DMA adapter library support Amit Prakash Shukla
2023-09-29 8:12 ` [PATCH v7 01/12] eventdev/dma: introduce DMA adapter Amit Prakash Shukla
2023-09-29 8:12 ` [PATCH v7 02/12] eventdev/dma: support adapter capabilities get Amit Prakash Shukla
@ 2023-09-29 8:13 ` Amit Prakash Shukla
2023-09-29 8:13 ` [PATCH v7 04/12] eventdev/dma: support vchan add and delete Amit Prakash Shukla
` (9 subsequent siblings)
12 siblings, 0 replies; 110+ messages in thread
From: Amit Prakash Shukla @ 2023-09-29 8:13 UTC (permalink / raw)
To: Bruce Richardson, Jerin Jacob, Amit Prakash Shukla
Cc: dev, fengchengwen, kevin.laatz, conor.walsh, vattunuru, g.singh,
sachin.saxena, hemant.agrawal, cheng1.jiang, ndabilpuram, anoobj,
mb
Added API support to create and free DMA adapter. Create function shall be
called with event device to be associated with the adapter and port
configuration to setup an event port.
Signed-off-by: Amit Prakash Shukla <amitprakashs@marvell.com>
---
config/rte_config.h | 1 +
lib/eventdev/meson.build | 1 +
lib/eventdev/rte_event_dma_adapter.c | 335 +++++++++++++++++++++++++++
3 files changed, 337 insertions(+)
create mode 100644 lib/eventdev/rte_event_dma_adapter.c
diff --git a/config/rte_config.h b/config/rte_config.h
index 400e44e3cf..401727703f 100644
--- a/config/rte_config.h
+++ b/config/rte_config.h
@@ -77,6 +77,7 @@
#define RTE_EVENT_ETH_INTR_RING_SIZE 1024
#define RTE_EVENT_CRYPTO_ADAPTER_MAX_INSTANCE 32
#define RTE_EVENT_ETH_TX_ADAPTER_MAX_INSTANCE 32
+#define RTE_EVENT_DMA_ADAPTER_MAX_INSTANCE 32
/* rawdev defines */
#define RTE_RAWDEV_MAX_DEVS 64
diff --git a/lib/eventdev/meson.build b/lib/eventdev/meson.build
index b46bbbc9aa..250abcb154 100644
--- a/lib/eventdev/meson.build
+++ b/lib/eventdev/meson.build
@@ -17,6 +17,7 @@ sources = files(
'eventdev_private.c',
'eventdev_trace_points.c',
'rte_event_crypto_adapter.c',
+ 'rte_event_dma_adapter.c',
'rte_event_eth_rx_adapter.c',
'rte_event_eth_tx_adapter.c',
'rte_event_ring.c',
diff --git a/lib/eventdev/rte_event_dma_adapter.c b/lib/eventdev/rte_event_dma_adapter.c
new file mode 100644
index 0000000000..241327d2a7
--- /dev/null
+++ b/lib/eventdev/rte_event_dma_adapter.c
@@ -0,0 +1,335 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright (c) 2023 Marvell.
+ */
+
+#include <eventdev_pmd.h>
+
+#include "rte_event_dma_adapter.h"
+
+#define DMA_BATCH_SIZE 32
+#define DMA_DEFAULT_MAX_NB 128
+#define DMA_ADAPTER_NAME_LEN 32
+#define DMA_ADAPTER_BUFFER_SIZE 1024
+
+#define DMA_ADAPTER_OPS_BUFFER_SIZE (DMA_BATCH_SIZE + DMA_BATCH_SIZE)
+
+#define DMA_ADAPTER_ARRAY "event_dma_adapter_array"
+
+/* Macros to check for valid adapter */
+#define EVENT_DMA_ADAPTER_ID_VALID_OR_ERR_RET(id, retval) \
+ do { \
+ if (!edma_adapter_valid_id(id)) { \
+ RTE_EDEV_LOG_ERR("Invalid DMA adapter id = %d\n", id); \
+ return retval; \
+ } \
+ } while (0)
+
+/* DMA ops circular buffer */
+struct dma_ops_circular_buffer {
+ /* Index of head element */
+ uint16_t head;
+
+ /* Index of tail element */
+ uint16_t tail;
+
+ /* Number of elements in buffer */
+ uint16_t count;
+
+ /* Size of circular buffer */
+ uint16_t size;
+
+ /* Pointer to hold rte_event_dma_adapter_op for processing */
+ struct rte_event_dma_adapter_op **op_buffer;
+} __rte_cache_aligned;
+
+/* DMA device information */
+struct dma_device_info {
+ /* Number of vchans configured for a DMA device. */
+ uint16_t num_dma_dev_vchan;
+} __rte_cache_aligned;
+
+struct event_dma_adapter {
+ /* Event device identifier */
+ uint8_t eventdev_id;
+
+ /* Event port identifier */
+ uint8_t event_port_id;
+
+ /* Adapter mode */
+ enum rte_event_dma_adapter_mode mode;
+
+ /* Memory allocation name */
+ char mem_name[DMA_ADAPTER_NAME_LEN];
+
+ /* Socket identifier cached from eventdev */
+ int socket_id;
+
+ /* Lock to serialize config updates with service function */
+ rte_spinlock_t lock;
+
+ /* DMA device structure array */
+ struct dma_device_info *dma_devs;
+
+ /* Circular buffer for processing DMA ops to eventdev */
+ struct dma_ops_circular_buffer ebuf;
+
+ /* Configuration callback for rte_service configuration */
+ rte_event_dma_adapter_conf_cb conf_cb;
+
+ /* Configuration callback argument */
+ void *conf_arg;
+
+ /* Set if default_cb is being used */
+ int default_cb_arg;
+} __rte_cache_aligned;
+
+static struct event_dma_adapter **event_dma_adapter;
+
+static inline int
+edma_adapter_valid_id(uint8_t id)
+{
+ return id < RTE_EVENT_DMA_ADAPTER_MAX_INSTANCE;
+}
+
+static inline struct event_dma_adapter *
+edma_id_to_adapter(uint8_t id)
+{
+ return event_dma_adapter ? event_dma_adapter[id] : NULL;
+}
+
+static int
+edma_array_init(void)
+{
+ const struct rte_memzone *mz;
+ uint32_t sz;
+
+ mz = rte_memzone_lookup(DMA_ADAPTER_ARRAY);
+ if (mz == NULL) {
+ sz = sizeof(struct event_dma_adapter *) * RTE_EVENT_DMA_ADAPTER_MAX_INSTANCE;
+ sz = RTE_ALIGN(sz, RTE_CACHE_LINE_SIZE);
+
+ mz = rte_memzone_reserve_aligned(DMA_ADAPTER_ARRAY, sz, rte_socket_id(), 0,
+ RTE_CACHE_LINE_SIZE);
+ if (mz == NULL) {
+ RTE_EDEV_LOG_ERR("Failed to reserve memzone : %s, err = %d",
+ DMA_ADAPTER_ARRAY, rte_errno);
+ return -rte_errno;
+ }
+ }
+
+ event_dma_adapter = mz->addr;
+
+ return 0;
+}
+
+static inline int
+edma_circular_buffer_init(const char *name, struct dma_ops_circular_buffer *buf, uint16_t sz)
+{
+ buf->op_buffer = rte_zmalloc(name, sizeof(struct rte_event_dma_adapter_op *) * sz, 0);
+ if (buf->op_buffer == NULL)
+ return -ENOMEM;
+
+ buf->size = sz;
+
+ return 0;
+}
+
+static inline void
+edma_circular_buffer_free(struct dma_ops_circular_buffer *buf)
+{
+ rte_free(buf->op_buffer);
+}
+
+static int
+edma_default_config_cb(uint8_t id, uint8_t evdev_id, struct rte_event_dma_adapter_conf *conf,
+ void *arg)
+{
+ struct rte_event_port_conf *port_conf;
+ struct rte_event_dev_config dev_conf;
+ struct event_dma_adapter *adapter;
+ struct rte_eventdev *dev;
+ uint8_t port_id;
+ int started;
+ int ret;
+
+ adapter = edma_id_to_adapter(id);
+ if (adapter == NULL)
+ return -EINVAL;
+
+ dev = &rte_eventdevs[adapter->eventdev_id];
+ dev_conf = dev->data->dev_conf;
+
+ started = dev->data->dev_started;
+ if (started)
+ rte_event_dev_stop(evdev_id);
+
+ port_id = dev_conf.nb_event_ports;
+ dev_conf.nb_event_ports += 1;
+
+ port_conf = arg;
+ if (port_conf->event_port_cfg & RTE_EVENT_PORT_CFG_SINGLE_LINK)
+ dev_conf.nb_single_link_event_port_queues += 1;
+
+ ret = rte_event_dev_configure(evdev_id, &dev_conf);
+ if (ret) {
+ RTE_EDEV_LOG_ERR("Failed to configure event dev %u\n", evdev_id);
+ if (started) {
+ if (rte_event_dev_start(evdev_id))
+ return -EIO;
+ }
+ return ret;
+ }
+
+ ret = rte_event_port_setup(evdev_id, port_id, port_conf);
+ if (ret) {
+ RTE_EDEV_LOG_ERR("Failed to setup event port %u\n", port_id);
+ return ret;
+ }
+
+ conf->event_port_id = port_id;
+ conf->max_nb = DMA_DEFAULT_MAX_NB;
+ if (started)
+ ret = rte_event_dev_start(evdev_id);
+
+ adapter->default_cb_arg = 1;
+ adapter->event_port_id = conf->event_port_id;
+
+ return ret;
+}
+
+int
+rte_event_dma_adapter_create_ext(uint8_t id, uint8_t evdev_id,
+ rte_event_dma_adapter_conf_cb conf_cb,
+ enum rte_event_dma_adapter_mode mode, void *conf_arg)
+{
+ struct rte_event_dev_info dev_info;
+ struct event_dma_adapter *adapter;
+ char name[DMA_ADAPTER_NAME_LEN];
+ struct rte_dma_info info;
+ uint16_t num_dma_dev;
+ int socket_id;
+ uint8_t i;
+ int ret;
+
+ EVENT_DMA_ADAPTER_ID_VALID_OR_ERR_RET(id, -EINVAL);
+ RTE_EVENTDEV_VALID_DEVID_OR_ERR_RET(evdev_id, -EINVAL);
+
+ if (conf_cb == NULL)
+ return -EINVAL;
+
+ if (event_dma_adapter == NULL) {
+ ret = edma_array_init();
+ if (ret)
+ return ret;
+ }
+
+ adapter = edma_id_to_adapter(id);
+ if (adapter != NULL) {
+ RTE_EDEV_LOG_ERR("ML adapter ID %d already exists!", id);
+ return -EEXIST;
+ }
+
+ socket_id = rte_event_dev_socket_id(evdev_id);
+ snprintf(name, DMA_ADAPTER_NAME_LEN, "rte_event_dma_adapter_%d", id);
+ adapter = rte_zmalloc_socket(name, sizeof(struct event_dma_adapter), RTE_CACHE_LINE_SIZE,
+ socket_id);
+ if (adapter == NULL) {
+ RTE_EDEV_LOG_ERR("Failed to get mem for event ML adapter!");
+ return -ENOMEM;
+ }
+
+ if (edma_circular_buffer_init("edma_circular_buffer", &adapter->ebuf,
+ DMA_ADAPTER_BUFFER_SIZE)) {
+ RTE_EDEV_LOG_ERR("Failed to get memory for event adapter circular buffer");
+ rte_free(adapter);
+ return -ENOMEM;
+ }
+
+ ret = rte_event_dev_info_get(evdev_id, &dev_info);
+ if (ret < 0) {
+ RTE_EDEV_LOG_ERR("Failed to get info for eventdev %d: %s", evdev_id,
+ dev_info.driver_name);
+ edma_circular_buffer_free(&adapter->ebuf);
+ rte_free(adapter);
+ return ret;
+ }
+
+ num_dma_dev = rte_dma_count_avail();
+
+ adapter->eventdev_id = evdev_id;
+ adapter->mode = mode;
+ rte_strscpy(adapter->mem_name, name, DMA_ADAPTER_NAME_LEN);
+ adapter->socket_id = socket_id;
+ adapter->conf_cb = conf_cb;
+ adapter->conf_arg = conf_arg;
+ adapter->dma_devs = rte_zmalloc_socket(adapter->mem_name,
+ num_dma_dev * sizeof(struct dma_device_info), 0,
+ socket_id);
+ if (adapter->dma_devs == NULL) {
+ RTE_EDEV_LOG_ERR("Failed to get memory for DMA devices\n");
+ edma_circular_buffer_free(&adapter->ebuf);
+ rte_free(adapter);
+ return -ENOMEM;
+ }
+
+ rte_spinlock_init(&adapter->lock);
+ for (i = 0; i < num_dma_dev; i++) {
+ ret = rte_dma_info_get(i, &info);
+ if (ret) {
+ RTE_EDEV_LOG_ERR("Failed to get dma device info\n");
+ edma_circular_buffer_free(&adapter->ebuf);
+ rte_free(adapter);
+ return ret;
+ }
+
+ adapter->dma_devs[i].num_dma_dev_vchan = info.nb_vchans;
+ }
+
+ event_dma_adapter[id] = adapter;
+
+ return 0;
+}
+
+int
+rte_event_dma_adapter_create(uint8_t id, uint8_t evdev_id, struct rte_event_port_conf *port_config,
+ enum rte_event_dma_adapter_mode mode)
+{
+ struct rte_event_port_conf *pc;
+ int ret;
+
+ if (port_config == NULL)
+ return -EINVAL;
+
+ EVENT_DMA_ADAPTER_ID_VALID_OR_ERR_RET(id, -EINVAL);
+
+ pc = rte_malloc(NULL, sizeof(struct rte_event_port_conf), 0);
+ if (pc == NULL)
+ return -ENOMEM;
+
+ rte_memcpy(pc, port_config, sizeof(struct rte_event_port_conf));
+ ret = rte_event_dma_adapter_create_ext(id, evdev_id, edma_default_config_cb, mode, pc);
+ if (ret != 0)
+ rte_free(pc);
+
+ return ret;
+}
+
+int
+rte_event_dma_adapter_free(uint8_t id)
+{
+ struct event_dma_adapter *adapter;
+
+ EVENT_DMA_ADAPTER_ID_VALID_OR_ERR_RET(id, -EINVAL);
+
+ adapter = edma_id_to_adapter(id);
+ if (adapter == NULL)
+ return -EINVAL;
+
+ rte_free(adapter->conf_arg);
+ rte_free(adapter->dma_devs);
+ edma_circular_buffer_free(&adapter->ebuf);
+ rte_free(adapter);
+ event_dma_adapter[id] = NULL;
+
+ return 0;
+}
--
2.25.1
^ permalink raw reply [flat|nested] 110+ messages in thread
* [PATCH v7 04/12] eventdev/dma: support vchan add and delete
2023-09-29 8:12 ` [PATCH v7 00/12] event DMA adapter library support Amit Prakash Shukla
` (2 preceding siblings ...)
2023-09-29 8:13 ` [PATCH v7 03/12] eventdev/dma: support adapter create and free Amit Prakash Shukla
@ 2023-09-29 8:13 ` Amit Prakash Shukla
2023-09-29 8:13 ` [PATCH v7 05/12] eventdev/dma: support adapter service function Amit Prakash Shukla
` (8 subsequent siblings)
12 siblings, 0 replies; 110+ messages in thread
From: Amit Prakash Shukla @ 2023-09-29 8:13 UTC (permalink / raw)
To: Amit Prakash Shukla, Jerin Jacob
Cc: dev, fengchengwen, kevin.laatz, bruce.richardson, conor.walsh,
vattunuru, g.singh, sachin.saxena, hemant.agrawal, cheng1.jiang,
ndabilpuram, anoobj, mb
Added API support to add and delete vchan's from the DMA adapter. DMA devid
and vchan are added to the addapter instance by calling
rte_event_dma_adapter_vchan_add and deleted using
rte_event_dma_adapter_vchan_del.
Signed-off-by: Amit Prakash Shukla <amitprakashs@marvell.com>
---
lib/eventdev/rte_event_dma_adapter.c | 204 +++++++++++++++++++++++++++
1 file changed, 204 insertions(+)
diff --git a/lib/eventdev/rte_event_dma_adapter.c b/lib/eventdev/rte_event_dma_adapter.c
index 241327d2a7..fa2e29b9d3 100644
--- a/lib/eventdev/rte_event_dma_adapter.c
+++ b/lib/eventdev/rte_event_dma_adapter.c
@@ -42,8 +42,31 @@ struct dma_ops_circular_buffer {
struct rte_event_dma_adapter_op **op_buffer;
} __rte_cache_aligned;
+/* Vchan information */
+struct dma_vchan_info {
+ /* Set to indicate vchan queue is enabled */
+ bool vq_enabled;
+
+ /* Circular buffer for batching DMA ops to dma_dev */
+ struct dma_ops_circular_buffer dma_buf;
+} __rte_cache_aligned;
+
/* DMA device information */
struct dma_device_info {
+ /* Pointer to vchan queue info */
+ struct dma_vchan_info *vchanq;
+
+ /* Pointer to vchan queue info.
+ * This holds ops passed by application till the
+ * dma completion is done.
+ */
+ struct dma_vchan_info *tqmap;
+
+ /* If num_vchanq > 0, the start callback will
+ * be invoked if not already invoked
+ */
+ uint16_t num_vchanq;
+
/* Number of vchans configured for a DMA device. */
uint16_t num_dma_dev_vchan;
} __rte_cache_aligned;
@@ -81,6 +104,9 @@ struct event_dma_adapter {
/* Set if default_cb is being used */
int default_cb_arg;
+
+ /* No. of vchan queue configured */
+ uint16_t nb_vchanq;
} __rte_cache_aligned;
static struct event_dma_adapter **event_dma_adapter;
@@ -333,3 +359,181 @@ rte_event_dma_adapter_free(uint8_t id)
return 0;
}
+
+static void
+edma_update_vchanq_info(struct event_dma_adapter *adapter, struct dma_device_info *dev_info,
+ uint16_t vchan, uint8_t add)
+{
+ struct dma_vchan_info *vchan_info;
+ struct dma_vchan_info *tqmap_info;
+ int enabled;
+ uint16_t i;
+
+ if (dev_info->vchanq == NULL)
+ return;
+
+ if (vchan == RTE_DMA_ALL_VCHAN) {
+ for (i = 0; i < dev_info->num_dma_dev_vchan; i++)
+ edma_update_vchanq_info(adapter, dev_info, i, add);
+ } else {
+ tqmap_info = &dev_info->tqmap[vchan];
+ vchan_info = &dev_info->vchanq[vchan];
+ enabled = vchan_info->vq_enabled;
+ if (add) {
+ adapter->nb_vchanq += !enabled;
+ dev_info->num_vchanq += !enabled;
+ } else {
+ adapter->nb_vchanq -= enabled;
+ dev_info->num_vchanq -= enabled;
+ }
+ vchan_info->vq_enabled = !!add;
+ tqmap_info->vq_enabled = !!add;
+ }
+}
+
+int
+rte_event_dma_adapter_vchan_add(uint8_t id, int16_t dma_dev_id, uint16_t vchan,
+ const struct rte_event *event)
+{
+ struct event_dma_adapter *adapter;
+ struct dma_device_info *dev_info;
+ struct rte_eventdev *dev;
+ uint32_t cap;
+ int ret;
+
+ EVENT_DMA_ADAPTER_ID_VALID_OR_ERR_RET(id, -EINVAL);
+
+ if (!rte_dma_is_valid(dma_dev_id)) {
+ RTE_EDEV_LOG_ERR("Invalid dma_dev_id = %" PRIu8, dma_dev_id);
+ return -EINVAL;
+ }
+
+ adapter = edma_id_to_adapter(id);
+ if (adapter == NULL)
+ return -EINVAL;
+
+ dev = &rte_eventdevs[adapter->eventdev_id];
+ ret = rte_event_dma_adapter_caps_get(adapter->eventdev_id, dma_dev_id, &cap);
+ if (ret) {
+ RTE_EDEV_LOG_ERR("Failed to get adapter caps dev %u dma_dev %u", id, dma_dev_id);
+ return ret;
+ }
+
+ if ((cap & RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_VCHAN_EV_BIND) && (event == NULL)) {
+ RTE_EDEV_LOG_ERR("Event can not be NULL for dma_dev_id = %u", dma_dev_id);
+ return -EINVAL;
+ }
+
+ dev_info = &adapter->dma_devs[dma_dev_id];
+ if (vchan != RTE_DMA_ALL_VCHAN && vchan >= dev_info->num_dma_dev_vchan) {
+ RTE_EDEV_LOG_ERR("Invalid vhcan %u", vchan);
+ return -EINVAL;
+ }
+
+ /* In case HW cap is RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP_FWD, no
+ * need of service core as HW supports event forward capability.
+ */
+ if ((cap & RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP_FWD) ||
+ (cap & RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_VCHAN_EV_BIND &&
+ adapter->mode == RTE_EVENT_DMA_ADAPTER_OP_NEW) ||
+ (cap & RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP_NEW &&
+ adapter->mode == RTE_EVENT_DMA_ADAPTER_OP_NEW)) {
+ if (*dev->dev_ops->dma_adapter_vchan_add == NULL)
+ return -ENOTSUP;
+ if (dev_info->vchanq == NULL) {
+ dev_info->vchanq = rte_zmalloc_socket(adapter->mem_name,
+ dev_info->num_dma_dev_vchan *
+ sizeof(struct dma_vchan_info),
+ 0, adapter->socket_id);
+ if (dev_info->vchanq == NULL) {
+ printf("Queue pair add not supported\n");
+ return -ENOMEM;
+ }
+ }
+
+ if (dev_info->tqmap == NULL) {
+ dev_info->tqmap = rte_zmalloc_socket(adapter->mem_name,
+ dev_info->num_dma_dev_vchan *
+ sizeof(struct dma_vchan_info),
+ 0, adapter->socket_id);
+ if (dev_info->tqmap == NULL) {
+ printf("tq pair add not supported\n");
+ return -ENOMEM;
+ }
+ }
+
+ ret = (*dev->dev_ops->dma_adapter_vchan_add)(dev, dma_dev_id, vchan, event);
+ if (ret)
+ return ret;
+
+ else
+ edma_update_vchanq_info(adapter, &adapter->dma_devs[dma_dev_id], vchan, 1);
+ }
+
+ return 0;
+}
+
+int
+rte_event_dma_adapter_vchan_del(uint8_t id, int16_t dma_dev_id, uint16_t vchan)
+{
+ struct event_dma_adapter *adapter;
+ struct dma_device_info *dev_info;
+ struct rte_eventdev *dev;
+ uint32_t cap;
+ int ret;
+
+ EVENT_DMA_ADAPTER_ID_VALID_OR_ERR_RET(id, -EINVAL);
+
+ if (!rte_dma_is_valid(dma_dev_id)) {
+ RTE_EDEV_LOG_ERR("Invalid dma_dev_id = %" PRIu8, dma_dev_id);
+ return -EINVAL;
+ }
+
+ adapter = edma_id_to_adapter(id);
+ if (adapter == NULL)
+ return -EINVAL;
+
+ dev = &rte_eventdevs[adapter->eventdev_id];
+ ret = rte_event_dma_adapter_caps_get(adapter->eventdev_id, dma_dev_id, &cap);
+ if (ret)
+ return ret;
+
+ dev_info = &adapter->dma_devs[dma_dev_id];
+
+ if (vchan != RTE_DMA_ALL_VCHAN && vchan >= dev_info->num_dma_dev_vchan) {
+ RTE_EDEV_LOG_ERR("Invalid vhcan %" PRIu16, vchan);
+ return -EINVAL;
+ }
+
+ if ((cap & RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP_FWD) ||
+ (cap & RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP_NEW &&
+ adapter->mode == RTE_EVENT_DMA_ADAPTER_OP_NEW)) {
+ if (*dev->dev_ops->dma_adapter_vchan_del == NULL)
+ return -ENOTSUP;
+ ret = (*dev->dev_ops->dma_adapter_vchan_del)(dev, dma_dev_id, vchan);
+ if (ret == 0) {
+ edma_update_vchanq_info(adapter, dev_info, vchan, 0);
+ if (dev_info->num_vchanq == 0) {
+ rte_free(dev_info->vchanq);
+ dev_info->vchanq = NULL;
+ }
+ }
+ } else {
+ if (adapter->nb_vchanq == 0)
+ return 0;
+
+ rte_spinlock_lock(&adapter->lock);
+ edma_update_vchanq_info(adapter, dev_info, vchan, 0);
+
+ if (dev_info->num_vchanq == 0) {
+ rte_free(dev_info->vchanq);
+ rte_free(dev_info->tqmap);
+ dev_info->vchanq = NULL;
+ dev_info->tqmap = NULL;
+ }
+
+ rte_spinlock_unlock(&adapter->lock);
+ }
+
+ return ret;
+}
--
2.25.1
^ permalink raw reply [flat|nested] 110+ messages in thread
* [PATCH v7 05/12] eventdev/dma: support adapter service function
2023-09-29 8:12 ` [PATCH v7 00/12] event DMA adapter library support Amit Prakash Shukla
` (3 preceding siblings ...)
2023-09-29 8:13 ` [PATCH v7 04/12] eventdev/dma: support vchan add and delete Amit Prakash Shukla
@ 2023-09-29 8:13 ` Amit Prakash Shukla
2023-09-29 8:13 ` [PATCH v7 06/12] eventdev/dma: support adapter start and stop Amit Prakash Shukla
` (7 subsequent siblings)
12 siblings, 0 replies; 110+ messages in thread
From: Amit Prakash Shukla @ 2023-09-29 8:13 UTC (permalink / raw)
To: Amit Prakash Shukla, Jerin Jacob
Cc: dev, fengchengwen, kevin.laatz, bruce.richardson, conor.walsh,
vattunuru, g.singh, sachin.saxena, hemant.agrawal, cheng1.jiang,
ndabilpuram, anoobj, mb
Added support for DMA adapter service function for event devices.
Enqueue and dequeue of event from eventdev and DMA device are done
based on the adapter mode and the supported HW capabilities.
Signed-off-by: Amit Prakash Shukla <amitprakashs@marvell.com>
---
lib/eventdev/rte_event_dma_adapter.c | 592 +++++++++++++++++++++++++++
1 file changed, 592 insertions(+)
diff --git a/lib/eventdev/rte_event_dma_adapter.c b/lib/eventdev/rte_event_dma_adapter.c
index fa2e29b9d3..1d8bae0422 100644
--- a/lib/eventdev/rte_event_dma_adapter.c
+++ b/lib/eventdev/rte_event_dma_adapter.c
@@ -3,6 +3,7 @@
*/
#include <eventdev_pmd.h>
+#include <rte_service_component.h>
#include "rte_event_dma_adapter.h"
@@ -69,6 +70,10 @@ struct dma_device_info {
/* Number of vchans configured for a DMA device. */
uint16_t num_dma_dev_vchan;
+
+ /* Next queue pair to be processed */
+ uint16_t next_vchan_id;
+
} __rte_cache_aligned;
struct event_dma_adapter {
@@ -90,6 +95,9 @@ struct event_dma_adapter {
/* Lock to serialize config updates with service function */
rte_spinlock_t lock;
+ /* Next dma device to be processed */
+ uint16_t next_dmadev_id;
+
/* DMA device structure array */
struct dma_device_info *dma_devs;
@@ -107,6 +115,26 @@ struct event_dma_adapter {
/* No. of vchan queue configured */
uint16_t nb_vchanq;
+
+ /* Per adapter EAL service ID */
+ uint32_t service_id;
+
+ /* Service initialization state */
+ uint8_t service_initialized;
+
+ /* Max DMA ops processed in any service function invocation */
+ uint32_t max_nb;
+
+ /* Store event port's implicit release capability */
+ uint8_t implicit_release_disabled;
+
+ /* Flag to indicate backpressure at dma_dev
+ * Stop further dequeuing events from eventdev
+ */
+ bool stop_enq_to_dma_dev;
+
+ /* Loop counter to flush dma ops */
+ uint16_t transmit_loop_count;
} __rte_cache_aligned;
static struct event_dma_adapter **event_dma_adapter;
@@ -148,6 +176,18 @@ edma_array_init(void)
return 0;
}
+static inline bool
+edma_circular_buffer_batch_ready(struct dma_ops_circular_buffer *bufp)
+{
+ return bufp->count >= DMA_BATCH_SIZE;
+}
+
+static inline bool
+edma_circular_buffer_space_for_batch(struct dma_ops_circular_buffer *bufp)
+{
+ return (bufp->size - bufp->count) >= DMA_BATCH_SIZE;
+}
+
static inline int
edma_circular_buffer_init(const char *name, struct dma_ops_circular_buffer *buf, uint16_t sz)
{
@@ -166,6 +206,71 @@ edma_circular_buffer_free(struct dma_ops_circular_buffer *buf)
rte_free(buf->op_buffer);
}
+static inline int
+edma_circular_buffer_add(struct dma_ops_circular_buffer *bufp, struct rte_event_dma_adapter_op *op)
+{
+ uint16_t *tail = &bufp->tail;
+
+ bufp->op_buffer[*tail] = op;
+
+ /* circular buffer, go round */
+ *tail = (*tail + 1) % bufp->size;
+ bufp->count++;
+
+ return 0;
+}
+
+static inline int
+edma_circular_buffer_flush_to_dma_dev(struct event_dma_adapter *adapter,
+ struct dma_ops_circular_buffer *bufp, uint8_t dma_dev_id,
+ uint16_t vchan, uint16_t *nb_ops_flushed)
+{
+ struct rte_event_dma_adapter_op *op;
+ struct dma_vchan_info *tq;
+ uint16_t *head = &bufp->head;
+ uint16_t *tail = &bufp->tail;
+ uint16_t n;
+ uint16_t i;
+ int ret;
+
+ if (*tail > *head)
+ n = *tail - *head;
+ else if (*tail < *head)
+ n = bufp->size - *head;
+ else {
+ *nb_ops_flushed = 0;
+ return 0; /* buffer empty */
+ }
+
+ tq = &adapter->dma_devs[dma_dev_id].tqmap[vchan];
+
+ for (i = 0; i < n; i++) {
+ op = bufp->op_buffer[*head];
+ if (op->nb_src == 1 && op->nb_dst == 1)
+ ret = rte_dma_copy(dma_dev_id, vchan, op->src_seg->addr, op->dst_seg->addr,
+ op->src_seg->length, op->flags);
+ else
+ ret = rte_dma_copy_sg(dma_dev_id, vchan, op->src_seg, op->dst_seg,
+ op->nb_src, op->nb_dst, op->flags);
+ if (ret < 0)
+ break;
+
+ /* Enqueue in transaction queue. */
+ edma_circular_buffer_add(&tq->dma_buf, op);
+
+ *head = (*head + 1) % bufp->size;
+ }
+
+ *nb_ops_flushed = i;
+ bufp->count -= *nb_ops_flushed;
+ if (!bufp->count) {
+ *head = 0;
+ *tail = 0;
+ }
+
+ return *nb_ops_flushed == n ? 0 : -1;
+}
+
static int
edma_default_config_cb(uint8_t id, uint8_t evdev_id, struct rte_event_dma_adapter_conf *conf,
void *arg)
@@ -360,6 +465,406 @@ rte_event_dma_adapter_free(uint8_t id)
return 0;
}
+static inline unsigned int
+edma_enq_to_dma_dev(struct event_dma_adapter *adapter, struct rte_event *ev, unsigned int cnt)
+{
+ struct dma_vchan_info *vchan_qinfo = NULL;
+ struct rte_event_dma_adapter_op *dma_op;
+ uint16_t vchan, nb_enqueued = 0;
+ int16_t dma_dev_id;
+ unsigned int i, n;
+ int ret;
+
+ ret = 0;
+ n = 0;
+
+ for (i = 0; i < cnt; i++) {
+ dma_op = ev[i].event_ptr;
+ if (dma_op == NULL)
+ continue;
+
+ /* Expected to have response info appended to dma_op. */
+
+ dma_dev_id = dma_op->dma_dev_id;
+ vchan = dma_op->vchan;
+ vchan_qinfo = &adapter->dma_devs[dma_dev_id].vchanq[vchan];
+ if (!vchan_qinfo->vq_enabled) {
+ if (dma_op != NULL && dma_op->op_mp != NULL)
+ rte_mempool_put(dma_op->op_mp, dma_op);
+ continue;
+ }
+ edma_circular_buffer_add(&vchan_qinfo->dma_buf, dma_op);
+
+ if (edma_circular_buffer_batch_ready(&vchan_qinfo->dma_buf)) {
+ ret = edma_circular_buffer_flush_to_dma_dev(adapter, &vchan_qinfo->dma_buf,
+ dma_dev_id, vchan,
+ &nb_enqueued);
+ n += nb_enqueued;
+
+ /**
+ * If some dma ops failed to flush to dma_dev and
+ * space for another batch is not available, stop
+ * dequeue from eventdev momentarily
+ */
+ if (unlikely(ret < 0 &&
+ !edma_circular_buffer_space_for_batch(&vchan_qinfo->dma_buf)))
+ adapter->stop_enq_to_dma_dev = true;
+ }
+ }
+
+ return n;
+}
+
+static unsigned int
+edma_adapter_dev_flush(struct event_dma_adapter *adapter, int16_t dma_dev_id,
+ uint16_t *nb_ops_flushed)
+{
+ struct dma_vchan_info *vchan_info;
+ struct dma_device_info *dev_info;
+ uint16_t nb = 0, nb_enqueued = 0;
+ uint16_t vchan, nb_vchans;
+
+ dev_info = &adapter->dma_devs[dma_dev_id];
+ nb_vchans = dev_info->num_vchanq;
+
+ for (vchan = 0; vchan < nb_vchans; vchan++) {
+
+ vchan_info = &dev_info->vchanq[vchan];
+ if (unlikely(vchan_info == NULL || !vchan_info->vq_enabled))
+ continue;
+
+ edma_circular_buffer_flush_to_dma_dev(adapter, &vchan_info->dma_buf, dma_dev_id,
+ vchan, &nb_enqueued);
+ *nb_ops_flushed += vchan_info->dma_buf.count;
+ nb += nb_enqueued;
+ }
+
+ return nb;
+}
+
+static unsigned int
+edma_adapter_enq_flush(struct event_dma_adapter *adapter)
+{
+ int16_t dma_dev_id;
+ uint16_t nb_enqueued = 0;
+ uint16_t nb_ops_flushed = 0;
+ uint16_t num_dma_dev = rte_dma_count_avail();
+
+ for (dma_dev_id = 0; dma_dev_id < num_dma_dev; dma_dev_id++)
+ nb_enqueued += edma_adapter_dev_flush(adapter, dma_dev_id, &nb_ops_flushed);
+ /**
+ * Enable dequeue from eventdev if all ops from circular
+ * buffer flushed to dma_dev
+ */
+ if (!nb_ops_flushed)
+ adapter->stop_enq_to_dma_dev = false;
+
+ return nb_enqueued;
+}
+
+/* Flush an instance's enqueue buffers every DMA_ENQ_FLUSH_THRESHOLD
+ * iterations of edma_adapter_enq_run()
+ */
+#define DMA_ENQ_FLUSH_THRESHOLD 1024
+
+static int
+edma_adapter_enq_run(struct event_dma_adapter *adapter, unsigned int max_enq)
+{
+ uint8_t event_port_id = adapter->event_port_id;
+ uint8_t event_dev_id = adapter->eventdev_id;
+ struct rte_event ev[DMA_BATCH_SIZE];
+ unsigned int nb_enq, nb_enqueued;
+ uint16_t n;
+
+ if (adapter->mode == RTE_EVENT_DMA_ADAPTER_OP_NEW)
+ return 0;
+
+ nb_enqueued = 0;
+ for (nb_enq = 0; nb_enq < max_enq; nb_enq += n) {
+
+ if (unlikely(adapter->stop_enq_to_dma_dev)) {
+ nb_enqueued += edma_adapter_enq_flush(adapter);
+
+ if (unlikely(adapter->stop_enq_to_dma_dev))
+ break;
+ }
+
+ n = rte_event_dequeue_burst(event_dev_id, event_port_id, ev, DMA_BATCH_SIZE, 0);
+
+ if (!n)
+ break;
+
+ nb_enqueued += edma_enq_to_dma_dev(adapter, ev, n);
+ }
+
+ if ((++adapter->transmit_loop_count & (DMA_ENQ_FLUSH_THRESHOLD - 1)) == 0)
+ nb_enqueued += edma_adapter_enq_flush(adapter);
+
+ return nb_enqueued;
+}
+
+#define DMA_ADAPTER_MAX_EV_ENQ_RETRIES 100
+
+static inline uint16_t
+edma_ops_enqueue_burst(struct event_dma_adapter *adapter, struct rte_event_dma_adapter_op **ops,
+ uint16_t num)
+{
+ uint8_t event_port_id = adapter->event_port_id;
+ uint8_t event_dev_id = adapter->eventdev_id;
+ struct rte_event events[DMA_BATCH_SIZE];
+ struct rte_event *response_info;
+ uint16_t nb_enqueued, nb_ev;
+ uint8_t retry;
+ uint8_t i;
+
+ nb_ev = 0;
+ retry = 0;
+ nb_enqueued = 0;
+ num = RTE_MIN(num, DMA_BATCH_SIZE);
+ for (i = 0; i < num; i++) {
+ struct rte_event *ev = &events[nb_ev++];
+
+ /* Expected to have response info appended to dma_op. */
+ response_info = (struct rte_event *)((uint8_t *)ops[i] +
+ sizeof(struct rte_event_dma_adapter_op));
+ if (unlikely(response_info == NULL)) {
+ if (ops[i] != NULL && ops[i]->op_mp != NULL)
+ rte_mempool_put(ops[i]->op_mp, ops[i]);
+ continue;
+ }
+
+ rte_memcpy(ev, response_info, sizeof(struct rte_event));
+ ev->event_ptr = ops[i];
+ ev->event_type = RTE_EVENT_TYPE_DMADEV;
+ if (adapter->implicit_release_disabled)
+ ev->op = RTE_EVENT_OP_FORWARD;
+ else
+ ev->op = RTE_EVENT_OP_NEW;
+ }
+
+ do {
+ nb_enqueued += rte_event_enqueue_burst(event_dev_id, event_port_id,
+ &events[nb_enqueued], nb_ev - nb_enqueued);
+
+ } while (retry++ < DMA_ADAPTER_MAX_EV_ENQ_RETRIES && nb_enqueued < nb_ev);
+
+ return nb_enqueued;
+}
+
+static int
+edma_circular_buffer_flush_to_evdev(struct event_dma_adapter *adapter,
+ struct dma_ops_circular_buffer *bufp,
+ uint16_t *enqueue_count)
+{
+ struct rte_event_dma_adapter_op **ops = bufp->op_buffer;
+ uint16_t n = 0, nb_ops_flushed;
+ uint16_t *head = &bufp->head;
+ uint16_t *tail = &bufp->tail;
+
+ if (*tail > *head)
+ n = *tail - *head;
+ else if (*tail < *head)
+ n = bufp->size - *head;
+ else {
+ if (enqueue_count)
+ *enqueue_count = 0;
+ return 0; /* buffer empty */
+ }
+
+ if (enqueue_count && n > *enqueue_count)
+ n = *enqueue_count;
+
+ nb_ops_flushed = edma_ops_enqueue_burst(adapter, &ops[*head], n);
+ if (enqueue_count)
+ *enqueue_count = nb_ops_flushed;
+
+ bufp->count -= nb_ops_flushed;
+ if (!bufp->count) {
+ *head = 0;
+ *tail = 0;
+ return 0; /* buffer empty */
+ }
+
+ *head = (*head + nb_ops_flushed) % bufp->size;
+ return 1;
+}
+
+static void
+edma_ops_buffer_flush(struct event_dma_adapter *adapter)
+{
+ if (likely(adapter->ebuf.count == 0))
+ return;
+
+ while (edma_circular_buffer_flush_to_evdev(adapter, &adapter->ebuf, NULL))
+ ;
+}
+
+static inline unsigned int
+edma_adapter_deq_run(struct event_dma_adapter *adapter, unsigned int max_deq)
+{
+ struct dma_vchan_info *vchan_info;
+ struct dma_ops_circular_buffer *tq_buf;
+ struct rte_event_dma_adapter_op *ops;
+ uint16_t n, nb_deq, nb_enqueued, i;
+ struct dma_device_info *dev_info;
+ uint16_t vchan, num_vchan;
+ uint16_t num_dma_dev;
+ int16_t dma_dev_id;
+ uint16_t index;
+ bool done;
+ bool err;
+
+ nb_deq = 0;
+ edma_ops_buffer_flush(adapter);
+
+ num_dma_dev = rte_dma_count_avail();
+ do {
+ done = true;
+
+ for (dma_dev_id = adapter->next_dmadev_id; dma_dev_id < num_dma_dev; dma_dev_id++) {
+ uint16_t queues = 0;
+ dev_info = &adapter->dma_devs[dma_dev_id];
+ num_vchan = dev_info->num_vchanq;
+
+ for (vchan = dev_info->next_vchan_id; queues < num_vchan;
+ vchan = (vchan + 1) % num_vchan, queues++) {
+
+ vchan_info = &dev_info->vchanq[vchan];
+ if (unlikely(vchan_info == NULL || !vchan_info->vq_enabled))
+ continue;
+
+ n = rte_dma_completed(dma_dev_id, vchan, DMA_BATCH_SIZE,
+ &index, &err);
+ if (!n)
+ continue;
+
+ done = false;
+
+ tq_buf = &dev_info->tqmap[vchan].dma_buf;
+
+ nb_enqueued = n;
+ if (unlikely(!adapter->ebuf.count))
+ edma_circular_buffer_flush_to_evdev(adapter, tq_buf,
+ &nb_enqueued);
+
+ if (likely(nb_enqueued == n))
+ goto check;
+
+ /* Failed to enqueue events case */
+ for (i = nb_enqueued; i < n; i++) {
+ ops = tq_buf->op_buffer[tq_buf->head];
+ edma_circular_buffer_add(&adapter->ebuf, ops);
+ tq_buf->head = (tq_buf->head + 1) % tq_buf->size;
+ }
+
+check:
+ nb_deq += n;
+ if (nb_deq >= max_deq) {
+ if ((vchan + 1) == num_vchan)
+ adapter->next_dmadev_id =
+ (dma_dev_id + 1) % num_dma_dev;
+
+ dev_info->next_vchan_id = (vchan + 1) % num_vchan;
+
+ return nb_deq;
+ }
+ }
+ }
+ adapter->next_dmadev_id = 0;
+
+ } while (done == false);
+
+ return nb_deq;
+}
+
+static int
+edma_adapter_run(struct event_dma_adapter *adapter, unsigned int max_ops)
+{
+ unsigned int ops_left = max_ops;
+
+ while (ops_left > 0) {
+ unsigned int e_cnt, d_cnt;
+
+ e_cnt = edma_adapter_deq_run(adapter, ops_left);
+ ops_left -= RTE_MIN(ops_left, e_cnt);
+
+ d_cnt = edma_adapter_enq_run(adapter, ops_left);
+ ops_left -= RTE_MIN(ops_left, d_cnt);
+
+ if (e_cnt == 0 && d_cnt == 0)
+ break;
+ }
+
+ if (ops_left == max_ops) {
+ rte_event_maintain(adapter->eventdev_id, adapter->event_port_id, 0);
+ return -EAGAIN;
+ } else
+ return 0;
+}
+
+static int
+edma_service_func(void *args)
+{
+ struct event_dma_adapter *adapter = args;
+ int ret;
+
+ if (rte_spinlock_trylock(&adapter->lock) == 0)
+ return 0;
+ ret = edma_adapter_run(adapter, adapter->max_nb);
+ rte_spinlock_unlock(&adapter->lock);
+
+ return ret;
+}
+
+static int
+edma_init_service(struct event_dma_adapter *adapter, uint8_t id)
+{
+ struct rte_event_dma_adapter_conf adapter_conf;
+ struct rte_service_spec service;
+ uint32_t impl_rel;
+ int ret;
+
+ if (adapter->service_initialized)
+ return 0;
+
+ memset(&service, 0, sizeof(service));
+ snprintf(service.name, DMA_ADAPTER_NAME_LEN, "rte_event_dma_adapter_%d", id);
+ service.socket_id = adapter->socket_id;
+ service.callback = edma_service_func;
+ service.callback_userdata = adapter;
+
+ /* Service function handles locking for queue add/del updates */
+ service.capabilities = RTE_SERVICE_CAP_MT_SAFE;
+ ret = rte_service_component_register(&service, &adapter->service_id);
+ if (ret) {
+ RTE_EDEV_LOG_ERR("failed to register service %s err = %" PRId32, service.name, ret);
+ return ret;
+ }
+
+ ret = adapter->conf_cb(id, adapter->eventdev_id, &adapter_conf, adapter->conf_arg);
+ if (ret) {
+ RTE_EDEV_LOG_ERR("configuration callback failed err = %" PRId32, ret);
+ return ret;
+ }
+
+ adapter->max_nb = adapter_conf.max_nb;
+ adapter->event_port_id = adapter_conf.event_port_id;
+
+ if (rte_event_port_attr_get(adapter->eventdev_id, adapter->event_port_id,
+ RTE_EVENT_PORT_ATTR_IMPLICIT_RELEASE_DISABLE, &impl_rel)) {
+ RTE_EDEV_LOG_ERR("Failed to get port info for eventdev %" PRId32,
+ adapter->eventdev_id);
+ edma_circular_buffer_free(&adapter->ebuf);
+ rte_free(adapter);
+ return -EINVAL;
+ }
+
+ adapter->implicit_release_disabled = (uint8_t)impl_rel;
+ adapter->service_initialized = 1;
+
+ return ret;
+}
+
static void
edma_update_vchanq_info(struct event_dma_adapter *adapter, struct dma_device_info *dev_info,
uint16_t vchan, uint8_t add)
@@ -391,6 +896,60 @@ edma_update_vchanq_info(struct event_dma_adapter *adapter, struct dma_device_inf
}
}
+static int
+edma_add_vchan(struct event_dma_adapter *adapter, int16_t dma_dev_id, uint16_t vchan)
+{
+ struct dma_device_info *dev_info = &adapter->dma_devs[dma_dev_id];
+ struct dma_vchan_info *vchanq;
+ struct dma_vchan_info *tqmap;
+ uint16_t nb_vchans;
+ uint32_t i;
+
+ if (dev_info->vchanq == NULL) {
+ nb_vchans = dev_info->num_dma_dev_vchan;
+
+ dev_info->vchanq = rte_zmalloc_socket(adapter->mem_name,
+ nb_vchans * sizeof(struct dma_vchan_info),
+ 0, adapter->socket_id);
+ if (dev_info->vchanq == NULL)
+ return -ENOMEM;
+
+ dev_info->tqmap = rte_zmalloc_socket(adapter->mem_name,
+ nb_vchans * sizeof(struct dma_vchan_info),
+ 0, adapter->socket_id);
+ if (dev_info->tqmap == NULL)
+ return -ENOMEM;
+
+ for (i = 0; i < nb_vchans; i++) {
+ vchanq = &dev_info->vchanq[i];
+
+ if (edma_circular_buffer_init("dma_dev_circular_buffer", &vchanq->dma_buf,
+ DMA_ADAPTER_OPS_BUFFER_SIZE)) {
+ RTE_EDEV_LOG_ERR("Failed to get memory for dma_dev buffer");
+ rte_free(vchanq);
+ return -ENOMEM;
+ }
+
+ tqmap = &dev_info->tqmap[i];
+ if (edma_circular_buffer_init("dma_dev_circular_trans_buf", &tqmap->dma_buf,
+ DMA_ADAPTER_OPS_BUFFER_SIZE)) {
+ RTE_EDEV_LOG_ERR(
+ "Failed to get memory for dma_dev transaction buffer");
+ rte_free(tqmap);
+ return -ENOMEM;
+ }
+ }
+ }
+
+ if (vchan == RTE_DMA_ALL_VCHAN) {
+ for (i = 0; i < dev_info->num_dma_dev_vchan; i++)
+ edma_update_vchanq_info(adapter, dev_info, i, 1);
+ } else
+ edma_update_vchanq_info(adapter, dev_info, vchan, 1);
+
+ return 0;
+}
+
int
rte_event_dma_adapter_vchan_add(uint8_t id, int16_t dma_dev_id, uint16_t vchan,
const struct rte_event *event)
@@ -470,6 +1029,38 @@ rte_event_dma_adapter_vchan_add(uint8_t id, int16_t dma_dev_id, uint16_t vchan,
edma_update_vchanq_info(adapter, &adapter->dma_devs[dma_dev_id], vchan, 1);
}
+ /* In case HW cap is RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP_NEW, or SW adapter, initiate
+ * services so the application can choose which ever way it wants to use the adapter.
+ *
+ * Case 1: RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP_NEW. Application may wants to use one
+ * of below two modes
+ *
+ * a. OP_FORWARD mode -> HW Dequeue + SW enqueue
+ * b. OP_NEW mode -> HW Dequeue
+ *
+ * Case 2: No HW caps, use SW adapter
+ *
+ * a. OP_FORWARD mode -> SW enqueue & dequeue
+ * b. OP_NEW mode -> SW Dequeue
+ */
+ if ((cap & RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP_NEW &&
+ !(cap & RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP_FWD) &&
+ adapter->mode == RTE_EVENT_DMA_ADAPTER_OP_FORWARD) ||
+ (!(cap & RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP_NEW) &&
+ !(cap & RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP_FWD) &&
+ !(cap & RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_VCHAN_EV_BIND))) {
+ rte_spinlock_lock(&adapter->lock);
+ ret = edma_init_service(adapter, id);
+ if (ret == 0)
+ ret = edma_add_vchan(adapter, dma_dev_id, vchan);
+ rte_spinlock_unlock(&adapter->lock);
+
+ if (ret)
+ return ret;
+
+ rte_service_component_runstate_set(adapter->service_id, 1);
+ }
+
return 0;
}
@@ -533,6 +1124,7 @@ rte_event_dma_adapter_vchan_del(uint8_t id, int16_t dma_dev_id, uint16_t vchan)
}
rte_spinlock_unlock(&adapter->lock);
+ rte_service_component_runstate_set(adapter->service_id, adapter->nb_vchanq);
}
return ret;
--
2.25.1
^ permalink raw reply [flat|nested] 110+ messages in thread
* [PATCH v7 06/12] eventdev/dma: support adapter start and stop
2023-09-29 8:12 ` [PATCH v7 00/12] event DMA adapter library support Amit Prakash Shukla
` (4 preceding siblings ...)
2023-09-29 8:13 ` [PATCH v7 05/12] eventdev/dma: support adapter service function Amit Prakash Shukla
@ 2023-09-29 8:13 ` Amit Prakash Shukla
2023-09-29 8:13 ` [PATCH v7 07/12] eventdev/dma: support adapter service ID get Amit Prakash Shukla
` (6 subsequent siblings)
12 siblings, 0 replies; 110+ messages in thread
From: Amit Prakash Shukla @ 2023-09-29 8:13 UTC (permalink / raw)
To: Amit Prakash Shukla, Jerin Jacob
Cc: dev, fengchengwen, kevin.laatz, bruce.richardson, conor.walsh,
vattunuru, g.singh, sachin.saxena, hemant.agrawal, cheng1.jiang,
ndabilpuram, anoobj, mb
Added API support to start and stop DMA adapter.
Signed-off-by: Amit Prakash Shukla <amitprakashs@marvell.com>
---
lib/eventdev/rte_event_dma_adapter.c | 69 ++++++++++++++++++++++++++++
1 file changed, 69 insertions(+)
diff --git a/lib/eventdev/rte_event_dma_adapter.c b/lib/eventdev/rte_event_dma_adapter.c
index 1d8bae0422..be6c2623e9 100644
--- a/lib/eventdev/rte_event_dma_adapter.c
+++ b/lib/eventdev/rte_event_dma_adapter.c
@@ -74,6 +74,13 @@ struct dma_device_info {
/* Next queue pair to be processed */
uint16_t next_vchan_id;
+ /* Set to indicate processing has been started */
+ uint8_t dev_started;
+
+ /* Set to indicate dmadev->eventdev packet
+ * transfer uses a hardware mechanism
+ */
+ uint8_t internal_event_port;
} __rte_cache_aligned;
struct event_dma_adapter {
@@ -1129,3 +1136,65 @@ rte_event_dma_adapter_vchan_del(uint8_t id, int16_t dma_dev_id, uint16_t vchan)
return ret;
}
+
+static int
+edma_adapter_ctrl(uint8_t id, int start)
+{
+ struct event_dma_adapter *adapter;
+ struct dma_device_info *dev_info;
+ struct rte_eventdev *dev;
+ uint16_t num_dma_dev;
+ int stop = !start;
+ int use_service;
+ uint32_t i;
+
+ use_service = 0;
+ EVENT_DMA_ADAPTER_ID_VALID_OR_ERR_RET(id, -EINVAL);
+ adapter = edma_id_to_adapter(id);
+ if (adapter == NULL)
+ return -EINVAL;
+
+ num_dma_dev = rte_dma_count_avail();
+ dev = &rte_eventdevs[adapter->eventdev_id];
+
+ for (i = 0; i < num_dma_dev; i++) {
+ dev_info = &adapter->dma_devs[i];
+ /* start check for num queue pairs */
+ if (start && !dev_info->num_vchanq)
+ continue;
+ /* stop check if dev has been started */
+ if (stop && !dev_info->dev_started)
+ continue;
+ use_service |= !dev_info->internal_event_port;
+ dev_info->dev_started = start;
+ if (dev_info->internal_event_port == 0)
+ continue;
+ start ? (*dev->dev_ops->dma_adapter_start)(dev, i) :
+ (*dev->dev_ops->dma_adapter_stop)(dev, i);
+ }
+
+ if (use_service)
+ rte_service_runstate_set(adapter->service_id, start);
+
+ return 0;
+}
+
+int
+rte_event_dma_adapter_start(uint8_t id)
+{
+ struct event_dma_adapter *adapter;
+
+ EVENT_DMA_ADAPTER_ID_VALID_OR_ERR_RET(id, -EINVAL);
+
+ adapter = edma_id_to_adapter(id);
+ if (adapter == NULL)
+ return -EINVAL;
+
+ return edma_adapter_ctrl(id, 1);
+}
+
+int
+rte_event_dma_adapter_stop(uint8_t id)
+{
+ return edma_adapter_ctrl(id, 0);
+}
--
2.25.1
^ permalink raw reply [flat|nested] 110+ messages in thread
* [PATCH v7 07/12] eventdev/dma: support adapter service ID get
2023-09-29 8:12 ` [PATCH v7 00/12] event DMA adapter library support Amit Prakash Shukla
` (5 preceding siblings ...)
2023-09-29 8:13 ` [PATCH v7 06/12] eventdev/dma: support adapter start and stop Amit Prakash Shukla
@ 2023-09-29 8:13 ` Amit Prakash Shukla
2023-09-29 8:13 ` [PATCH v7 08/12] eventdev/dma: support adapter runtime params Amit Prakash Shukla
` (5 subsequent siblings)
12 siblings, 0 replies; 110+ messages in thread
From: Amit Prakash Shukla @ 2023-09-29 8:13 UTC (permalink / raw)
To: Amit Prakash Shukla, Jerin Jacob
Cc: dev, fengchengwen, kevin.laatz, bruce.richardson, conor.walsh,
vattunuru, g.singh, sachin.saxena, hemant.agrawal, cheng1.jiang,
ndabilpuram, anoobj, mb
Added API support to get DMA adapter service ID. Service id
returned in the variable by the API call shall be used by application
to map a service core.
Signed-off-by: Amit Prakash Shukla <amitprakashs@marvell.com>
---
lib/eventdev/rte_event_dma_adapter.c | 17 +++++++++++++++++
1 file changed, 17 insertions(+)
diff --git a/lib/eventdev/rte_event_dma_adapter.c b/lib/eventdev/rte_event_dma_adapter.c
index be6c2623e9..c3b014aaf9 100644
--- a/lib/eventdev/rte_event_dma_adapter.c
+++ b/lib/eventdev/rte_event_dma_adapter.c
@@ -1137,6 +1137,23 @@ rte_event_dma_adapter_vchan_del(uint8_t id, int16_t dma_dev_id, uint16_t vchan)
return ret;
}
+int
+rte_event_dma_adapter_service_id_get(uint8_t id, uint32_t *service_id)
+{
+ struct event_dma_adapter *adapter;
+
+ EVENT_DMA_ADAPTER_ID_VALID_OR_ERR_RET(id, -EINVAL);
+
+ adapter = edma_id_to_adapter(id);
+ if (adapter == NULL || service_id == NULL)
+ return -EINVAL;
+
+ if (adapter->service_initialized)
+ *service_id = adapter->service_id;
+
+ return adapter->service_initialized ? 0 : -ESRCH;
+}
+
static int
edma_adapter_ctrl(uint8_t id, int start)
{
--
2.25.1
^ permalink raw reply [flat|nested] 110+ messages in thread
* [PATCH v7 08/12] eventdev/dma: support adapter runtime params
2023-09-29 8:12 ` [PATCH v7 00/12] event DMA adapter library support Amit Prakash Shukla
` (6 preceding siblings ...)
2023-09-29 8:13 ` [PATCH v7 07/12] eventdev/dma: support adapter service ID get Amit Prakash Shukla
@ 2023-09-29 8:13 ` Amit Prakash Shukla
2023-09-29 8:13 ` [PATCH v7 09/12] eventdev/dma: support adapter stats Amit Prakash Shukla
` (4 subsequent siblings)
12 siblings, 0 replies; 110+ messages in thread
From: Amit Prakash Shukla @ 2023-09-29 8:13 UTC (permalink / raw)
To: Amit Prakash Shukla, Jerin Jacob
Cc: dev, fengchengwen, kevin.laatz, bruce.richardson, conor.walsh,
vattunuru, g.singh, sachin.saxena, hemant.agrawal, cheng1.jiang,
ndabilpuram, anoobj, mb
Added support to set and get runtime params for DMA adapter. The
parameters that can be set/get are defined in
struct rte_event_dma_adapter_runtime_params.
Signed-off-by: Amit Prakash Shukla <amitprakashs@marvell.com>
---
lib/eventdev/rte_event_dma_adapter.c | 93 ++++++++++++++++++++++++++++
1 file changed, 93 insertions(+)
diff --git a/lib/eventdev/rte_event_dma_adapter.c b/lib/eventdev/rte_event_dma_adapter.c
index c3b014aaf9..632169a7c2 100644
--- a/lib/eventdev/rte_event_dma_adapter.c
+++ b/lib/eventdev/rte_event_dma_adapter.c
@@ -1215,3 +1215,96 @@ rte_event_dma_adapter_stop(uint8_t id)
{
return edma_adapter_ctrl(id, 0);
}
+
+#define DEFAULT_MAX_NB 128
+
+int
+rte_event_dma_adapter_runtime_params_init(struct rte_event_dma_adapter_runtime_params *params)
+{
+ if (params == NULL)
+ return -EINVAL;
+
+ memset(params, 0, sizeof(*params));
+ params->max_nb = DEFAULT_MAX_NB;
+
+ return 0;
+}
+
+static int
+dma_adapter_cap_check(struct event_dma_adapter *adapter)
+{
+ uint32_t caps;
+ int ret;
+
+ if (!adapter->nb_vchanq)
+ return -EINVAL;
+
+ ret = rte_event_dma_adapter_caps_get(adapter->eventdev_id, adapter->next_dmadev_id, &caps);
+ if (ret) {
+ RTE_EDEV_LOG_ERR("Failed to get adapter caps dev %" PRIu8 " cdev %" PRIu8,
+ adapter->eventdev_id, adapter->next_dmadev_id);
+ return ret;
+ }
+
+ if ((caps & RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP_FWD) ||
+ (caps & RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP_NEW))
+ return -ENOTSUP;
+
+ return 0;
+}
+
+int
+rte_event_dma_adapter_runtime_params_set(uint8_t id,
+ struct rte_event_dma_adapter_runtime_params *params)
+{
+ struct event_dma_adapter *adapter;
+ int ret;
+
+ EVENT_DMA_ADAPTER_ID_VALID_OR_ERR_RET(id, -EINVAL);
+
+ if (params == NULL) {
+ RTE_EDEV_LOG_ERR("params pointer is NULL\n");
+ return -EINVAL;
+ }
+
+ adapter = edma_id_to_adapter(id);
+ if (adapter == NULL)
+ return -EINVAL;
+
+ ret = dma_adapter_cap_check(adapter);
+ if (ret)
+ return ret;
+
+ rte_spinlock_lock(&adapter->lock);
+ adapter->max_nb = params->max_nb;
+ rte_spinlock_unlock(&adapter->lock);
+
+ return 0;
+}
+
+int
+rte_event_dma_adapter_runtime_params_get(uint8_t id,
+ struct rte_event_dma_adapter_runtime_params *params)
+{
+ struct event_dma_adapter *adapter;
+ int ret;
+
+ EVENT_DMA_ADAPTER_ID_VALID_OR_ERR_RET(id, -EINVAL);
+
+ if (params == NULL) {
+ RTE_EDEV_LOG_ERR("params pointer is NULL\n");
+ return -EINVAL;
+ }
+
+ adapter = edma_id_to_adapter(id);
+ if (adapter == NULL)
+ return -EINVAL;
+
+ ret = dma_adapter_cap_check(adapter);
+ if (ret)
+ return ret;
+
+ params->max_nb = adapter->max_nb;
+
+ return 0;
+}
--
2.25.1
^ permalink raw reply [flat|nested] 110+ messages in thread
* [PATCH v7 09/12] eventdev/dma: support adapter stats
2023-09-29 8:12 ` [PATCH v7 00/12] event DMA adapter library support Amit Prakash Shukla
` (7 preceding siblings ...)
2023-09-29 8:13 ` [PATCH v7 08/12] eventdev/dma: support adapter runtime params Amit Prakash Shukla
@ 2023-09-29 8:13 ` Amit Prakash Shukla
2023-09-29 8:13 ` [PATCH v7 10/12] eventdev/dma: support adapter enqueue Amit Prakash Shukla
` (3 subsequent siblings)
12 siblings, 0 replies; 110+ messages in thread
From: Amit Prakash Shukla @ 2023-09-29 8:13 UTC (permalink / raw)
To: Amit Prakash Shukla, Jerin Jacob
Cc: dev, fengchengwen, kevin.laatz, bruce.richardson, conor.walsh,
vattunuru, g.singh, sachin.saxena, hemant.agrawal, cheng1.jiang,
ndabilpuram, anoobj, mb
Added DMA adapter stats API support to get and reset stats. DMA
SW adapter stats and eventdev driver supported stats for enqueue
and dequeue are reported by get API.
Signed-off-by: Amit Prakash Shukla <amitprakashs@marvell.com>
---
lib/eventdev/rte_event_dma_adapter.c | 95 ++++++++++++++++++++++++++++
1 file changed, 95 insertions(+)
diff --git a/lib/eventdev/rte_event_dma_adapter.c b/lib/eventdev/rte_event_dma_adapter.c
index 632169a7c2..6c67e6d499 100644
--- a/lib/eventdev/rte_event_dma_adapter.c
+++ b/lib/eventdev/rte_event_dma_adapter.c
@@ -142,6 +142,9 @@ struct event_dma_adapter {
/* Loop counter to flush dma ops */
uint16_t transmit_loop_count;
+
+ /* Per instance stats structure */
+ struct rte_event_dma_adapter_stats dma_stats;
} __rte_cache_aligned;
static struct event_dma_adapter **event_dma_adapter;
@@ -475,6 +478,7 @@ rte_event_dma_adapter_free(uint8_t id)
static inline unsigned int
edma_enq_to_dma_dev(struct event_dma_adapter *adapter, struct rte_event *ev, unsigned int cnt)
{
+ struct rte_event_dma_adapter_stats *stats = &adapter->dma_stats;
struct dma_vchan_info *vchan_qinfo = NULL;
struct rte_event_dma_adapter_op *dma_op;
uint16_t vchan, nb_enqueued = 0;
@@ -484,6 +488,7 @@ edma_enq_to_dma_dev(struct event_dma_adapter *adapter, struct rte_event *ev, uns
ret = 0;
n = 0;
+ stats->event_deq_count += cnt;
for (i = 0; i < cnt; i++) {
dma_op = ev[i].event_ptr;
@@ -506,6 +511,7 @@ edma_enq_to_dma_dev(struct event_dma_adapter *adapter, struct rte_event *ev, uns
ret = edma_circular_buffer_flush_to_dma_dev(adapter, &vchan_qinfo->dma_buf,
dma_dev_id, vchan,
&nb_enqueued);
+ stats->dma_enq_count += nb_enqueued;
n += nb_enqueued;
/**
@@ -552,6 +558,7 @@ edma_adapter_dev_flush(struct event_dma_adapter *adapter, int16_t dma_dev_id,
static unsigned int
edma_adapter_enq_flush(struct event_dma_adapter *adapter)
{
+ struct rte_event_dma_adapter_stats *stats = &adapter->dma_stats;
int16_t dma_dev_id;
uint16_t nb_enqueued = 0;
uint16_t nb_ops_flushed = 0;
@@ -566,6 +573,8 @@ edma_adapter_enq_flush(struct event_dma_adapter *adapter)
if (!nb_ops_flushed)
adapter->stop_enq_to_dma_dev = false;
+ stats->dma_enq_count += nb_enqueued;
+
return nb_enqueued;
}
@@ -577,6 +586,7 @@ edma_adapter_enq_flush(struct event_dma_adapter *adapter)
static int
edma_adapter_enq_run(struct event_dma_adapter *adapter, unsigned int max_enq)
{
+ struct rte_event_dma_adapter_stats *stats = &adapter->dma_stats;
uint8_t event_port_id = adapter->event_port_id;
uint8_t event_dev_id = adapter->eventdev_id;
struct rte_event ev[DMA_BATCH_SIZE];
@@ -596,6 +606,7 @@ edma_adapter_enq_run(struct event_dma_adapter *adapter, unsigned int max_enq)
break;
}
+ stats->event_poll_count++;
n = rte_event_dequeue_burst(event_dev_id, event_port_id, ev, DMA_BATCH_SIZE, 0);
if (!n)
@@ -616,6 +627,7 @@ static inline uint16_t
edma_ops_enqueue_burst(struct event_dma_adapter *adapter, struct rte_event_dma_adapter_op **ops,
uint16_t num)
{
+ struct rte_event_dma_adapter_stats *stats = &adapter->dma_stats;
uint8_t event_port_id = adapter->event_port_id;
uint8_t event_dev_id = adapter->eventdev_id;
struct rte_event events[DMA_BATCH_SIZE];
@@ -655,6 +667,10 @@ edma_ops_enqueue_burst(struct event_dma_adapter *adapter, struct rte_event_dma_a
} while (retry++ < DMA_ADAPTER_MAX_EV_ENQ_RETRIES && nb_enqueued < nb_ev);
+ stats->event_enq_fail_count += nb_ev - nb_enqueued;
+ stats->event_enq_count += nb_enqueued;
+ stats->event_enq_retry_count += retry - 1;
+
return nb_enqueued;
}
@@ -709,6 +725,7 @@ edma_ops_buffer_flush(struct event_dma_adapter *adapter)
static inline unsigned int
edma_adapter_deq_run(struct event_dma_adapter *adapter, unsigned int max_deq)
{
+ struct rte_event_dma_adapter_stats *stats = &adapter->dma_stats;
struct dma_vchan_info *vchan_info;
struct dma_ops_circular_buffer *tq_buf;
struct rte_event_dma_adapter_op *ops;
@@ -746,6 +763,7 @@ edma_adapter_deq_run(struct event_dma_adapter *adapter, unsigned int max_deq)
continue;
done = false;
+ stats->dma_deq_count += n;
tq_buf = &dev_info->tqmap[vchan].dma_buf;
@@ -1308,3 +1326,80 @@ rte_event_dma_adapter_runtime_params_get(uint8_t id,
return 0;
}
+
+int
+rte_event_dma_adapter_stats_get(uint8_t id, struct rte_event_dma_adapter_stats *stats)
+{
+ struct rte_event_dma_adapter_stats dev_stats_sum = {0};
+ struct rte_event_dma_adapter_stats dev_stats;
+ struct event_dma_adapter *adapter;
+ struct dma_device_info *dev_info;
+ struct rte_eventdev *dev;
+ uint16_t num_dma_dev;
+ uint32_t i;
+ int ret;
+
+ EVENT_DMA_ADAPTER_ID_VALID_OR_ERR_RET(id, -EINVAL);
+
+ adapter = edma_id_to_adapter(id);
+ if (adapter == NULL || stats == NULL)
+ return -EINVAL;
+
+ num_dma_dev = rte_dma_count_avail();
+ dev = &rte_eventdevs[adapter->eventdev_id];
+ memset(stats, 0, sizeof(*stats));
+ for (i = 0; i < num_dma_dev; i++) {
+ dev_info = &adapter->dma_devs[i];
+
+ if (dev_info->internal_event_port == 0 ||
+ dev->dev_ops->dma_adapter_stats_get == NULL)
+ continue;
+
+ ret = (*dev->dev_ops->dma_adapter_stats_get)(dev, i, &dev_stats);
+ if (ret)
+ continue;
+
+ dev_stats_sum.dma_deq_count += dev_stats.dma_deq_count;
+ dev_stats_sum.event_enq_count += dev_stats.event_enq_count;
+ }
+
+ if (adapter->service_initialized)
+ *stats = adapter->dma_stats;
+
+ stats->dma_deq_count += dev_stats_sum.dma_deq_count;
+ stats->event_enq_count += dev_stats_sum.event_enq_count;
+
+ return 0;
+}
+
+int
+rte_event_dma_adapter_stats_reset(uint8_t id)
+{
+ struct event_dma_adapter *adapter;
+ struct dma_device_info *dev_info;
+ struct rte_eventdev *dev;
+ uint16_t num_dma_dev;
+ uint32_t i;
+
+ EVENT_DMA_ADAPTER_ID_VALID_OR_ERR_RET(id, -EINVAL);
+
+ adapter = edma_id_to_adapter(id);
+ if (adapter == NULL)
+ return -EINVAL;
+
+ num_dma_dev = rte_dma_count_avail();
+ dev = &rte_eventdevs[adapter->eventdev_id];
+ for (i = 0; i < num_dma_dev; i++) {
+ dev_info = &adapter->dma_devs[i];
+
+ if (dev_info->internal_event_port == 0 ||
+ dev->dev_ops->dma_adapter_stats_reset == NULL)
+ continue;
+
+ (*dev->dev_ops->dma_adapter_stats_reset)(dev, i);
+ }
+
+ memset(&adapter->dma_stats, 0, sizeof(adapter->dma_stats));
+
+ return 0;
+}
--
2.25.1
^ permalink raw reply [flat|nested] 110+ messages in thread
* [PATCH v7 10/12] eventdev/dma: support adapter enqueue
2023-09-29 8:12 ` [PATCH v7 00/12] event DMA adapter library support Amit Prakash Shukla
` (8 preceding siblings ...)
2023-09-29 8:13 ` [PATCH v7 09/12] eventdev/dma: support adapter stats Amit Prakash Shukla
@ 2023-09-29 8:13 ` Amit Prakash Shukla
2023-09-29 8:13 ` [PATCH v7 11/12] eventdev/dma: support adapter event port get Amit Prakash Shukla
` (2 subsequent siblings)
12 siblings, 0 replies; 110+ messages in thread
From: Amit Prakash Shukla @ 2023-09-29 8:13 UTC (permalink / raw)
To: Amit Prakash Shukla, Jerin Jacob
Cc: dev, fengchengwen, kevin.laatz, bruce.richardson, conor.walsh,
vattunuru, g.singh, sachin.saxena, hemant.agrawal, cheng1.jiang,
ndabilpuram, anoobj, mb
Added API support to enqueue a DMA operation to the DMA driver.
Signed-off-by: Amit Prakash Shukla <amitprakashs@marvell.com>
---
lib/eventdev/rte_event_dma_adapter.c | 13 +++++++++++++
1 file changed, 13 insertions(+)
diff --git a/lib/eventdev/rte_event_dma_adapter.c b/lib/eventdev/rte_event_dma_adapter.c
index 6c67e6d499..f299914dec 100644
--- a/lib/eventdev/rte_event_dma_adapter.c
+++ b/lib/eventdev/rte_event_dma_adapter.c
@@ -1403,3 +1403,16 @@ rte_event_dma_adapter_stats_reset(uint8_t id)
return 0;
}
+
+uint16_t
+rte_event_dma_adapter_enqueue(uint8_t dev_id, uint8_t port_id, struct rte_event ev[],
+ uint16_t nb_events)
+{
+ const struct rte_event_fp_ops *fp_ops;
+ void *port;
+
+ fp_ops = &rte_event_fp_ops[dev_id];
+ port = fp_ops->data[port_id];
+
+ return fp_ops->dma_enqueue(port, ev, nb_events);
+}
--
2.25.1
^ permalink raw reply [flat|nested] 110+ messages in thread
* [PATCH v7 11/12] eventdev/dma: support adapter event port get
2023-09-29 8:12 ` [PATCH v7 00/12] event DMA adapter library support Amit Prakash Shukla
` (9 preceding siblings ...)
2023-09-29 8:13 ` [PATCH v7 10/12] eventdev/dma: support adapter enqueue Amit Prakash Shukla
@ 2023-09-29 8:13 ` Amit Prakash Shukla
2023-09-29 8:13 ` [PATCH v7 12/12] app/test: add event DMA adapter auto-test Amit Prakash Shukla
2023-09-29 11:50 ` [PATCH v8 00/12] event DMA adapter library support Amit Prakash Shukla
12 siblings, 0 replies; 110+ messages in thread
From: Amit Prakash Shukla @ 2023-09-29 8:13 UTC (permalink / raw)
To: Amit Prakash Shukla, Jerin Jacob
Cc: dev, fengchengwen, kevin.laatz, bruce.richardson, conor.walsh,
vattunuru, g.singh, sachin.saxena, hemant.agrawal, cheng1.jiang,
ndabilpuram, anoobj, mb
Added support for DMA adapter event port get.
Signed-off-by: Amit Prakash Shukla <amitprakashs@marvell.com>
---
lib/eventdev/rte_event_dma_adapter.c | 16 ++++++++++++++++
1 file changed, 16 insertions(+)
diff --git a/lib/eventdev/rte_event_dma_adapter.c b/lib/eventdev/rte_event_dma_adapter.c
index f299914dec..af4b5ad388 100644
--- a/lib/eventdev/rte_event_dma_adapter.c
+++ b/lib/eventdev/rte_event_dma_adapter.c
@@ -475,6 +475,22 @@ rte_event_dma_adapter_free(uint8_t id)
return 0;
}
+int
+rte_event_dma_adapter_event_port_get(uint8_t id, uint8_t *event_port_id)
+{
+ struct event_dma_adapter *adapter;
+
+ EVENT_DMA_ADAPTER_ID_VALID_OR_ERR_RET(id, -EINVAL);
+
+ adapter = edma_id_to_adapter(id);
+ if (adapter == NULL || event_port_id == NULL)
+ return -EINVAL;
+
+ *event_port_id = adapter->event_port_id;
+
+ return 0;
+}
+
static inline unsigned int
edma_enq_to_dma_dev(struct event_dma_adapter *adapter, struct rte_event *ev, unsigned int cnt)
{
--
2.25.1
^ permalink raw reply [flat|nested] 110+ messages in thread
* [PATCH v7 12/12] app/test: add event DMA adapter auto-test
2023-09-29 8:12 ` [PATCH v7 00/12] event DMA adapter library support Amit Prakash Shukla
` (10 preceding siblings ...)
2023-09-29 8:13 ` [PATCH v7 11/12] eventdev/dma: support adapter event port get Amit Prakash Shukla
@ 2023-09-29 8:13 ` Amit Prakash Shukla
2023-09-29 11:50 ` [PATCH v8 00/12] event DMA adapter library support Amit Prakash Shukla
12 siblings, 0 replies; 110+ messages in thread
From: Amit Prakash Shukla @ 2023-09-29 8:13 UTC (permalink / raw)
To: Thomas Monjalon, Amit Prakash Shukla
Cc: dev, jerinj, fengchengwen, kevin.laatz, bruce.richardson,
conor.walsh, vattunuru, g.singh, sachin.saxena, hemant.agrawal,
cheng1.jiang, ndabilpuram, anoobj, mb
Added testsuite to test the dma adapter functionality.
The testsuite detects event and DMA device capability
and accordingly dma adapter is configured and modes are
tested. Test command:
sudo <build dir>/app/test/dpdk-test --vdev=dma_skeleton \
event_dma_adapter_autotest
Signed-off-by: Amit Prakash Shukla <amitprakashs@marvell.com>
---
MAINTAINERS | 1 +
app/test/meson.build | 1 +
app/test/test_event_dma_adapter.c | 805 ++++++++++++++++++++++++++++++
3 files changed, 807 insertions(+)
create mode 100644 app/test/test_event_dma_adapter.c
diff --git a/MAINTAINERS b/MAINTAINERS
index 4ebbbe8bb3..92c0b47618 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -544,6 +544,7 @@ Eventdev DMA Adapter API
M: Amit Prakash Shukla <amitprakashs@marvell.com>
T: git://dpdk.org/next/dpdk-next-eventdev
F: lib/eventdev/*dma_adapter*
+F: app/test/test_event_dma_adapter.c
F: doc/guides/prog_guide/event_dma_adapter.rst
Raw device API
diff --git a/app/test/meson.build b/app/test/meson.build
index 05bae9216d..7caf5ae5fc 100644
--- a/app/test/meson.build
+++ b/app/test/meson.build
@@ -66,6 +66,7 @@ source_file_deps = {
'test_errno.c': [],
'test_ethdev_link.c': ['ethdev'],
'test_event_crypto_adapter.c': ['cryptodev', 'eventdev', 'bus_vdev'],
+ 'test_event_dma_adapter.c': ['dmadev', 'eventdev', 'bus_vdev'],
'test_event_eth_rx_adapter.c': ['ethdev', 'eventdev', 'bus_vdev'],
'test_event_eth_tx_adapter.c': ['bus_vdev', 'ethdev', 'net_ring', 'eventdev'],
'test_event_ring.c': ['eventdev'],
diff --git a/app/test/test_event_dma_adapter.c b/app/test/test_event_dma_adapter.c
new file mode 100644
index 0000000000..1e193f4b52
--- /dev/null
+++ b/app/test/test_event_dma_adapter.c
@@ -0,0 +1,805 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright (c) 2023 Marvell.
+ */
+
+#include "test.h"
+#include <string.h>
+#include <rte_common.h>
+#include <rte_malloc.h>
+#include <rte_mempool.h>
+#include <rte_mbuf.h>
+#include <rte_random.h>
+
+#ifdef RTE_EXEC_ENV_WINDOWS
+static int
+test_event_dma_adapter(void)
+{
+ printf("event_dma_adapter not supported on Windows, skipping test\n");
+ return TEST_SKIPPED;
+}
+
+#else
+
+#include <rte_bus_vdev.h>
+#include <rte_dmadev.h>
+#include <rte_eventdev.h>
+#include <rte_event_dma_adapter.h>
+#include <rte_service.h>
+
+#define NUM_MBUFS (8191)
+#define MBUF_CACHE_SIZE (256)
+#define TEST_APP_PORT_ID 0
+#define TEST_APP_EV_QUEUE_ID 0
+#define TEST_APP_EV_PRIORITY 0
+#define TEST_APP_EV_FLOWID 0xAABB
+#define TEST_DMA_EV_QUEUE_ID 1
+#define TEST_ADAPTER_ID 0
+#define TEST_DMA_DEV_ID 0
+#define TEST_DMA_VCHAN_ID 0
+#define PACKET_LENGTH 1024
+#define NB_TEST_PORTS 1
+#define NB_TEST_QUEUES 2
+#define NUM_CORES 2
+#define DMA_OP_POOL_SIZE 128
+#define TEST_MAX_OP 32
+#define TEST_RINGSIZE 512
+
+#define MBUF_SIZE (RTE_PKTMBUF_HEADROOM + PACKET_LENGTH)
+
+/* Handle log statements in same manner as test macros */
+#define LOG_DBG(...) RTE_LOG(DEBUG, EAL, __VA_ARGS__)
+
+struct event_dma_adapter_test_params {
+ struct rte_mempool *src_mbuf_pool;
+ struct rte_mempool *dst_mbuf_pool;
+ struct rte_mempool *op_mpool;
+ uint8_t dma_event_port_id;
+ uint8_t internal_port_op_fwd;
+};
+
+struct rte_event dma_response_info = {
+ .queue_id = TEST_APP_EV_QUEUE_ID,
+ .sched_type = RTE_SCHED_TYPE_ATOMIC,
+ .flow_id = TEST_APP_EV_FLOWID,
+ .priority = TEST_APP_EV_PRIORITY
+};
+
+static struct event_dma_adapter_test_params params;
+static uint8_t dma_adapter_setup_done;
+static uint32_t slcore_id;
+static int evdev;
+
+static int
+send_recv_ev(struct rte_event *ev)
+{
+ struct rte_event recv_ev[TEST_MAX_OP];
+ uint16_t nb_enqueued = 0;
+ int i = 0;
+
+ if (params.internal_port_op_fwd) {
+ nb_enqueued = rte_event_dma_adapter_enqueue(evdev, TEST_APP_PORT_ID, ev,
+ TEST_MAX_OP);
+ } else {
+ while (nb_enqueued < TEST_MAX_OP) {
+ nb_enqueued += rte_event_enqueue_burst(evdev, TEST_APP_PORT_ID,
+ &ev[nb_enqueued], TEST_MAX_OP -
+ nb_enqueued);
+ }
+ }
+
+ TEST_ASSERT_EQUAL(nb_enqueued, TEST_MAX_OP, "Failed to send event to dma adapter\n");
+
+ while (i < TEST_MAX_OP) {
+ if (rte_event_dequeue_burst(evdev, TEST_APP_PORT_ID, &recv_ev[i], 1, 0) != 1)
+ continue;
+ i++;
+ }
+
+ TEST_ASSERT_EQUAL(i, TEST_MAX_OP, "Test failed. Failed to dequeue events.\n");
+
+ return TEST_SUCCESS;
+}
+
+static int
+test_dma_adapter_stats(void)
+{
+ struct rte_event_dma_adapter_stats stats;
+
+ rte_event_dma_adapter_stats_get(TEST_ADAPTER_ID, &stats);
+ printf(" +------------------------------------------------------+\n");
+ printf(" + DMA adapter stats for instance %u:\n", TEST_ADAPTER_ID);
+ printf(" + Event port poll count 0x%" PRIx64 "\n",
+ stats.event_poll_count);
+ printf(" + Event dequeue count 0x%" PRIx64 "\n",
+ stats.event_deq_count);
+ printf(" + DMA dev enqueue count 0x%" PRIx64 "\n",
+ stats.dma_enq_count);
+ printf(" + DMA dev enqueue failed count 0x%" PRIx64 "\n",
+ stats.dma_enq_fail_count);
+ printf(" + DMA dev dequeue count 0x%" PRIx64 "\n",
+ stats.dma_deq_count);
+ printf(" + Event enqueue count 0x%" PRIx64 "\n",
+ stats.event_enq_count);
+ printf(" + Event enqueue retry count 0x%" PRIx64 "\n",
+ stats.event_enq_retry_count);
+ printf(" + Event enqueue fail count 0x%" PRIx64 "\n",
+ stats.event_enq_fail_count);
+ printf(" +------------------------------------------------------+\n");
+
+ rte_event_dma_adapter_stats_reset(TEST_ADAPTER_ID);
+ return TEST_SUCCESS;
+}
+
+static int
+test_dma_adapter_params(void)
+{
+ struct rte_event_dma_adapter_runtime_params out_params;
+ struct rte_event_dma_adapter_runtime_params in_params;
+ struct rte_event event;
+ uint32_t cap;
+ int err, rc;
+
+ err = rte_event_dma_adapter_caps_get(evdev, TEST_DMA_DEV_ID, &cap);
+ TEST_ASSERT_SUCCESS(err, "Failed to get adapter capabilities\n");
+
+ if (cap & RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_VCHAN_EV_BIND) {
+ err = rte_event_dma_adapter_vchan_add(TEST_ADAPTER_ID, TEST_DMA_DEV_ID,
+ TEST_DMA_VCHAN_ID, &event);
+ } else
+ err = rte_event_dma_adapter_vchan_add(TEST_ADAPTER_ID, TEST_DMA_DEV_ID,
+ TEST_DMA_VCHAN_ID, NULL);
+
+ TEST_ASSERT_SUCCESS(err, "Failed to add vchan\n");
+
+ err = rte_event_dma_adapter_runtime_params_init(&in_params);
+ TEST_ASSERT(err == 0, "Expected 0 got %d", err);
+ err = rte_event_dma_adapter_runtime_params_init(&out_params);
+ TEST_ASSERT(err == 0, "Expected 0 got %d", err);
+
+ /* Case 1: Get the default value of mbufs processed by adapter */
+ err = rte_event_dma_adapter_runtime_params_get(TEST_ADAPTER_ID, &out_params);
+ if (err == -ENOTSUP) {
+ rc = TEST_SKIPPED;
+ goto vchan_del;
+ }
+ TEST_ASSERT(err == 0, "Expected 0 got %d", err);
+
+ /* Case 2: Set max_nb = 32 (=BATCH_SEIZE) */
+ in_params.max_nb = 32;
+
+ err = rte_event_dma_adapter_runtime_params_set(TEST_ADAPTER_ID, &in_params);
+ TEST_ASSERT(err == 0, "Expected 0 got %d", err);
+
+ err = rte_event_dma_adapter_runtime_params_get(TEST_ADAPTER_ID, &out_params);
+ TEST_ASSERT(err == 0, "Expected 0 got %d", err);
+ TEST_ASSERT(in_params.max_nb == out_params.max_nb, "Expected %u got %u",
+ in_params.max_nb, out_params.max_nb);
+
+ /* Case 3: Set max_nb = 192 */
+ in_params.max_nb = 192;
+
+ err = rte_event_dma_adapter_runtime_params_set(TEST_ADAPTER_ID, &in_params);
+ TEST_ASSERT(err == 0, "Expected 0 got %d", err);
+
+ err = rte_event_dma_adapter_runtime_params_get(TEST_ADAPTER_ID, &out_params);
+ TEST_ASSERT(err == 0, "Expected 0 got %d", err);
+ TEST_ASSERT(in_params.max_nb == out_params.max_nb, "Expected %u got %u",
+ in_params.max_nb, out_params.max_nb);
+
+ /* Case 4: Set max_nb = 256 */
+ in_params.max_nb = 256;
+
+ err = rte_event_dma_adapter_runtime_params_set(TEST_ADAPTER_ID, &in_params);
+ TEST_ASSERT(err == 0, "Expected 0 got %d", err);
+
+ err = rte_event_dma_adapter_runtime_params_get(TEST_ADAPTER_ID, &out_params);
+ TEST_ASSERT(err == 0, "Expected 0 got %d", err);
+ TEST_ASSERT(in_params.max_nb == out_params.max_nb, "Expected %u got %u",
+ in_params.max_nb, out_params.max_nb);
+
+ /* Case 5: Set max_nb = 30(<BATCH_SIZE) */
+ in_params.max_nb = 30;
+
+ err = rte_event_dma_adapter_runtime_params_set(TEST_ADAPTER_ID, &in_params);
+ TEST_ASSERT(err == 0, "Expected 0 got %d", err);
+
+ err = rte_event_dma_adapter_runtime_params_get(TEST_ADAPTER_ID, &out_params);
+ TEST_ASSERT(err == 0, "Expected 0 got %d", err);
+ TEST_ASSERT(in_params.max_nb == out_params.max_nb, "Expected %u got %u",
+ in_params.max_nb, out_params.max_nb);
+
+ /* Case 6: Set max_nb = 512 */
+ in_params.max_nb = 512;
+
+ err = rte_event_dma_adapter_runtime_params_set(TEST_ADAPTER_ID, &in_params);
+ TEST_ASSERT(err == 0, "Expected 0 got %d", err);
+
+ err = rte_event_dma_adapter_runtime_params_get(TEST_ADAPTER_ID, &out_params);
+ TEST_ASSERT(err == 0, "Expected 0 got %d", err);
+ TEST_ASSERT(in_params.max_nb == out_params.max_nb, "Expected %u got %u",
+ in_params.max_nb, out_params.max_nb);
+
+ rc = TEST_SUCCESS;
+vchan_del:
+ err = rte_event_dma_adapter_vchan_del(TEST_ADAPTER_ID, TEST_DMA_DEV_ID,
+ TEST_DMA_VCHAN_ID);
+ TEST_ASSERT_SUCCESS(err, "Failed to delete vchan\n");
+
+ return rc;
+}
+
+static int
+test_op_forward_mode(void)
+{
+ struct rte_mbuf *src_mbuf[TEST_MAX_OP];
+ struct rte_mbuf *dst_mbuf[TEST_MAX_OP];
+ struct rte_event_dma_adapter_op *op;
+ struct rte_event ev[TEST_MAX_OP];
+ struct rte_event response_info;
+ int ret, i;
+
+ ret = rte_pktmbuf_alloc_bulk(params.src_mbuf_pool, src_mbuf, TEST_MAX_OP);
+ TEST_ASSERT_SUCCESS(ret, "alloc src mbufs failed.\n");
+
+ ret = rte_pktmbuf_alloc_bulk(params.dst_mbuf_pool, dst_mbuf, TEST_MAX_OP);
+ TEST_ASSERT_SUCCESS(ret, "alloc dst mbufs failed.\n");
+
+ for (i = 0; i < TEST_MAX_OP; i++) {
+ memset(rte_pktmbuf_mtod(src_mbuf[i], void *), rte_rand(), PACKET_LENGTH);
+ memset(rte_pktmbuf_mtod(dst_mbuf[i], void *), 0, PACKET_LENGTH);
+ }
+
+ for (i = 0; i < TEST_MAX_OP; i++) {
+ rte_mempool_get(params.op_mpool, (void **)&op);
+ TEST_ASSERT_NOT_NULL(op, "Failed to allocate dma operation struct\n");
+
+ op->src_seg = rte_malloc(NULL, sizeof(struct rte_dma_sge), 0);
+ op->dst_seg = rte_malloc(NULL, sizeof(struct rte_dma_sge), 0);
+
+ /* Update Op */
+ op->src_seg->addr = rte_pktmbuf_iova(src_mbuf[i]);
+ op->dst_seg->addr = rte_pktmbuf_iova(dst_mbuf[i]);
+ op->src_seg->length = PACKET_LENGTH;
+ op->dst_seg->length = PACKET_LENGTH;
+ op->nb_src = 1;
+ op->nb_dst = 1;
+ op->flags = RTE_DMA_OP_FLAG_SUBMIT;
+ op->op_mp = params.op_mpool;
+ op->dma_dev_id = TEST_DMA_DEV_ID;
+ op->vchan = TEST_DMA_VCHAN_ID;
+
+ response_info.event = dma_response_info.event;
+ rte_memcpy((uint8_t *)op + sizeof(struct rte_event_dma_adapter_op), &response_info,
+ sizeof(struct rte_event));
+
+ /* Fill in event info and update event_ptr with rte_event_dma_adapter_op */
+ memset(&ev[i], 0, sizeof(struct rte_event));
+ ev[i].event = 0;
+ ev[i].event_type = RTE_EVENT_TYPE_DMADEV;
+ ev[i].queue_id = TEST_DMA_EV_QUEUE_ID;
+ ev[i].sched_type = RTE_SCHED_TYPE_ATOMIC;
+ ev[i].flow_id = 0xAABB;
+ ev[i].event_ptr = op;
+ }
+
+ ret = send_recv_ev(ev);
+ TEST_ASSERT_SUCCESS(ret, "Failed to send/receive event to dma adapter\n");
+
+ test_dma_adapter_stats();
+
+ for (i = 0; i < TEST_MAX_OP; i++) {
+ op = ev[i].event_ptr;
+ ret = memcmp(rte_pktmbuf_mtod(src_mbuf[i], void *),
+ rte_pktmbuf_mtod(dst_mbuf[i], void *), PACKET_LENGTH);
+
+ TEST_ASSERT_EQUAL(ret, 0, "Data mismatch for dma adapter\n");
+
+ rte_free(op->src_seg);
+ rte_free(op->dst_seg);
+ rte_mempool_put(op->op_mp, op);
+ }
+
+ rte_pktmbuf_free_bulk(src_mbuf, TEST_MAX_OP);
+ rte_pktmbuf_free_bulk(dst_mbuf, TEST_MAX_OP);
+
+ return TEST_SUCCESS;
+}
+
+static int
+map_adapter_service_core(void)
+{
+ uint32_t adapter_service_id;
+ int ret;
+
+ if (rte_event_dma_adapter_service_id_get(TEST_ADAPTER_ID, &adapter_service_id) == 0) {
+ uint32_t core_list[NUM_CORES];
+
+ ret = rte_service_lcore_list(core_list, NUM_CORES);
+ TEST_ASSERT(ret >= 0, "Failed to get service core list!");
+
+ if (core_list[0] != slcore_id) {
+ TEST_ASSERT_SUCCESS(rte_service_lcore_add(slcore_id),
+ "Failed to add service core");
+ TEST_ASSERT_SUCCESS(rte_service_lcore_start(slcore_id),
+ "Failed to start service core");
+ }
+
+ TEST_ASSERT_SUCCESS(rte_service_map_lcore_set(
+ adapter_service_id, slcore_id, 1),
+ "Failed to map adapter service");
+ }
+
+ return TEST_SUCCESS;
+}
+
+static int
+test_with_op_forward_mode(void)
+{
+ uint32_t cap;
+ int ret;
+
+ ret = rte_event_dma_adapter_caps_get(evdev, TEST_DMA_DEV_ID, &cap);
+ TEST_ASSERT_SUCCESS(ret, "Failed to get adapter capabilities\n");
+
+ if (!(cap & RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP_FWD) &&
+ !(cap & RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP_NEW))
+ map_adapter_service_core();
+ else {
+ if (!(cap & RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP_FWD))
+ return TEST_SKIPPED;
+ }
+
+ TEST_ASSERT_SUCCESS(rte_event_dma_adapter_start(TEST_ADAPTER_ID),
+ "Failed to start event dma adapter");
+
+ ret = test_op_forward_mode();
+ TEST_ASSERT_SUCCESS(ret, "DMA - FORWARD mode test failed\n");
+ return TEST_SUCCESS;
+}
+
+static int
+configure_dmadev(void)
+{
+ const struct rte_dma_conf conf = { .nb_vchans = 1};
+ const struct rte_dma_vchan_conf qconf = {
+ .direction = RTE_DMA_DIR_MEM_TO_MEM,
+ .nb_desc = TEST_RINGSIZE,
+ };
+ struct rte_dma_info info;
+ unsigned int elt_size;
+ int ret;
+
+ ret = rte_dma_count_avail();
+ RTE_TEST_ASSERT_FAIL(ret, "No dma devices found!\n");
+
+ ret = rte_dma_info_get(TEST_DMA_DEV_ID, &info);
+ TEST_ASSERT_SUCCESS(ret, "Error with rte_dma_info_get()\n");
+
+ if (info.max_vchans < 1)
+ RTE_LOG(ERR, USER1, "Error, no channels available on device id %u\n",
+ TEST_DMA_DEV_ID);
+
+ if (rte_dma_configure(TEST_DMA_DEV_ID, &conf) != 0)
+ RTE_LOG(ERR, USER1, "Error with rte_dma_configure()\n");
+
+ if (rte_dma_vchan_setup(TEST_DMA_DEV_ID, TEST_DMA_VCHAN_ID, &qconf) < 0)
+ RTE_LOG(ERR, USER1, "Error with vchan configuration\n");
+
+ ret = rte_dma_info_get(TEST_DMA_DEV_ID, &info);
+ if (ret != 0 || info.nb_vchans != 1)
+ RTE_LOG(ERR, USER1, "Error, no configured vhcan reported on device id %u\n",
+ TEST_DMA_DEV_ID);
+
+ params.src_mbuf_pool = rte_pktmbuf_pool_create("DMA_ADAPTER_SRC_MBUFPOOL", NUM_MBUFS,
+ MBUF_CACHE_SIZE, 0, MBUF_SIZE,
+ rte_socket_id());
+ RTE_TEST_ASSERT_NOT_NULL(params.src_mbuf_pool, "Can't create DMA_SRC_MBUFPOOL\n");
+
+ params.dst_mbuf_pool = rte_pktmbuf_pool_create("DMA_ADAPTER_DST_MBUFPOOL", NUM_MBUFS,
+ MBUF_CACHE_SIZE, 0, MBUF_SIZE,
+ rte_socket_id());
+ RTE_TEST_ASSERT_NOT_NULL(params.dst_mbuf_pool, "Can't create DMA_DST_MBUFPOOL\n");
+
+ elt_size = sizeof(struct rte_event_dma_adapter_op) + sizeof(struct rte_event);
+ params.op_mpool = rte_mempool_create("EVENT_DMA_OP_POOL", DMA_OP_POOL_SIZE, elt_size, 0,
+ 0, NULL, NULL, NULL, NULL, rte_socket_id(), 0);
+ RTE_TEST_ASSERT_NOT_NULL(params.op_mpool, "Can't create DMA_OP_POOL\n");
+
+ return TEST_SUCCESS;
+}
+
+static inline void
+evdev_set_conf_values(struct rte_event_dev_config *dev_conf, struct rte_event_dev_info *info)
+{
+ memset(dev_conf, 0, sizeof(struct rte_event_dev_config));
+ dev_conf->dequeue_timeout_ns = info->min_dequeue_timeout_ns;
+ dev_conf->nb_event_ports = NB_TEST_PORTS;
+ dev_conf->nb_event_queues = NB_TEST_QUEUES;
+ dev_conf->nb_event_queue_flows = info->max_event_queue_flows;
+ dev_conf->nb_event_port_dequeue_depth =
+ info->max_event_port_dequeue_depth;
+ dev_conf->nb_event_port_enqueue_depth =
+ info->max_event_port_enqueue_depth;
+ dev_conf->nb_event_port_enqueue_depth =
+ info->max_event_port_enqueue_depth;
+ dev_conf->nb_events_limit =
+ info->max_num_events;
+}
+
+static int
+configure_eventdev(void)
+{
+ struct rte_event_queue_conf queue_conf;
+ struct rte_event_dev_config devconf;
+ struct rte_event_dev_info info;
+ uint32_t queue_count;
+ uint32_t port_count;
+ uint8_t qid;
+ int ret;
+
+ if (!rte_event_dev_count()) {
+ /* If there is no hardware eventdev, or no software vdev was
+ * specified on the command line, create an instance of
+ * event_sw.
+ */
+ LOG_DBG("Failed to find a valid event device... "
+ "testing with event_sw device\n");
+ TEST_ASSERT_SUCCESS(rte_vdev_init("event_sw0", NULL),
+ "Error creating eventdev");
+ evdev = rte_event_dev_get_dev_id("event_sw0");
+ }
+
+ ret = rte_event_dev_info_get(evdev, &info);
+ TEST_ASSERT_SUCCESS(ret, "Failed to get event dev info\n");
+
+ evdev_set_conf_values(&devconf, &info);
+
+ ret = rte_event_dev_configure(evdev, &devconf);
+ TEST_ASSERT_SUCCESS(ret, "Failed to configure eventdev\n");
+
+ /* Set up event queue */
+ ret = rte_event_dev_attr_get(evdev, RTE_EVENT_DEV_ATTR_QUEUE_COUNT, &queue_count);
+ TEST_ASSERT_SUCCESS(ret, "Queue count get failed\n");
+ TEST_ASSERT_EQUAL(queue_count, 2, "Unexpected queue count\n");
+
+ qid = TEST_APP_EV_QUEUE_ID;
+ ret = rte_event_queue_setup(evdev, qid, NULL);
+ TEST_ASSERT_SUCCESS(ret, "Failed to setup queue=%d\n", qid);
+
+ queue_conf.nb_atomic_flows = info.max_event_queue_flows;
+ queue_conf.nb_atomic_order_sequences = 32;
+ queue_conf.schedule_type = RTE_SCHED_TYPE_ATOMIC;
+ queue_conf.priority = RTE_EVENT_DEV_PRIORITY_HIGHEST;
+ queue_conf.event_queue_cfg = RTE_EVENT_QUEUE_CFG_SINGLE_LINK;
+
+ qid = TEST_DMA_EV_QUEUE_ID;
+ ret = rte_event_queue_setup(evdev, qid, &queue_conf);
+ TEST_ASSERT_SUCCESS(ret, "Failed to setup queue=%u\n", qid);
+
+ /* Set up event port */
+ ret = rte_event_dev_attr_get(evdev, RTE_EVENT_DEV_ATTR_PORT_COUNT,
+ &port_count);
+ TEST_ASSERT_SUCCESS(ret, "Port count get failed\n");
+ TEST_ASSERT_EQUAL(port_count, 1, "Unexpected port count\n");
+
+ ret = rte_event_port_setup(evdev, TEST_APP_PORT_ID, NULL);
+ TEST_ASSERT_SUCCESS(ret, "Failed to setup port=%d\n",
+ TEST_APP_PORT_ID);
+
+ qid = TEST_APP_EV_QUEUE_ID;
+ ret = rte_event_port_link(evdev, TEST_APP_PORT_ID, &qid, NULL, 1);
+ TEST_ASSERT(ret >= 0, "Failed to link queue port=%d\n",
+ TEST_APP_PORT_ID);
+
+ return TEST_SUCCESS;
+}
+
+static void
+test_dma_adapter_free(void)
+{
+ rte_event_dma_adapter_free(TEST_ADAPTER_ID);
+}
+
+static int
+test_dma_adapter_create(void)
+{
+ struct rte_event_dev_info evdev_info = {0};
+ struct rte_event_port_conf conf = {0};
+ int ret;
+
+ ret = rte_event_dev_info_get(evdev, &evdev_info);
+ TEST_ASSERT_SUCCESS(ret, "Failed to create event dma adapter\n");
+
+ conf.new_event_threshold = evdev_info.max_num_events;
+ conf.dequeue_depth = evdev_info.max_event_port_dequeue_depth;
+ conf.enqueue_depth = evdev_info.max_event_port_enqueue_depth;
+
+ /* Create adapter with default port creation callback */
+ ret = rte_event_dma_adapter_create(TEST_ADAPTER_ID, evdev, &conf, 0);
+ TEST_ASSERT_SUCCESS(ret, "Failed to create event dma adapter\n");
+
+ return TEST_SUCCESS;
+}
+
+static int
+test_dma_adapter_vchan_add_del(void)
+{
+ struct rte_event event;
+ uint32_t cap;
+ int ret;
+
+ ret = rte_event_dma_adapter_caps_get(evdev, TEST_DMA_DEV_ID, &cap);
+ TEST_ASSERT_SUCCESS(ret, "Failed to get adapter capabilities\n");
+
+ if (cap & RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_VCHAN_EV_BIND) {
+ ret = rte_event_dma_adapter_vchan_add(TEST_ADAPTER_ID, TEST_DMA_DEV_ID,
+ TEST_DMA_VCHAN_ID, &event);
+ } else
+ ret = rte_event_dma_adapter_vchan_add(TEST_ADAPTER_ID, TEST_DMA_DEV_ID,
+ TEST_DMA_VCHAN_ID, NULL);
+
+ TEST_ASSERT_SUCCESS(ret, "Failed to create add vchan\n");
+
+ ret = rte_event_dma_adapter_vchan_del(TEST_ADAPTER_ID, TEST_DMA_DEV_ID,
+ TEST_DMA_VCHAN_ID);
+ TEST_ASSERT_SUCCESS(ret, "Failed to delete vchan\n");
+
+ return TEST_SUCCESS;
+}
+
+static int
+configure_event_dma_adapter(enum rte_event_dma_adapter_mode mode)
+{
+ struct rte_event_dev_info evdev_info = {0};
+ struct rte_event_port_conf conf = {0};
+ struct rte_event event;
+ uint32_t cap;
+ int ret;
+
+ ret = rte_event_dma_adapter_caps_get(evdev, TEST_DMA_DEV_ID, &cap);
+ TEST_ASSERT_SUCCESS(ret, "Failed to get adapter capabilities\n");
+
+ /* Skip mode and capability mismatch check for SW eventdev */
+ if (!(cap & RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP_NEW) &&
+ !(cap & RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP_FWD) &&
+ !(cap & RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_VCHAN_EV_BIND))
+ goto adapter_create;
+
+ if (mode == RTE_EVENT_DMA_ADAPTER_OP_FORWARD) {
+ if (cap & RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP_FWD)
+ params.internal_port_op_fwd = 1;
+ else
+ return -ENOTSUP;
+ }
+
+adapter_create:
+ ret = rte_event_dev_info_get(evdev, &evdev_info);
+ TEST_ASSERT_SUCCESS(ret, "Failed to create event dma adapter\n");
+
+ conf.new_event_threshold = evdev_info.max_num_events;
+ conf.dequeue_depth = evdev_info.max_event_port_dequeue_depth;
+ conf.enqueue_depth = evdev_info.max_event_port_enqueue_depth;
+
+ /* Create adapter with default port creation callback */
+ ret = rte_event_dma_adapter_create(TEST_ADAPTER_ID, evdev, &conf, mode);
+ TEST_ASSERT_SUCCESS(ret, "Failed to create event dma adapter\n");
+
+ if (cap & RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_VCHAN_EV_BIND) {
+ ret = rte_event_dma_adapter_vchan_add(TEST_ADAPTER_ID, TEST_DMA_DEV_ID,
+ TEST_DMA_VCHAN_ID, &event);
+ } else
+ ret = rte_event_dma_adapter_vchan_add(TEST_ADAPTER_ID, TEST_DMA_DEV_ID,
+ TEST_DMA_VCHAN_ID, NULL);
+
+ TEST_ASSERT_SUCCESS(ret, "Failed to add vchan\n");
+
+ if (!params.internal_port_op_fwd) {
+ ret = rte_event_dma_adapter_event_port_get(TEST_ADAPTER_ID,
+ ¶ms.dma_event_port_id);
+ TEST_ASSERT_SUCCESS(ret, "Failed to get event port\n");
+ }
+
+ return TEST_SUCCESS;
+}
+
+static void
+test_dma_adapter_stop(void)
+{
+ uint32_t evdev_service_id, adapter_service_id;
+
+ /* retrieve service ids & stop services */
+ if (rte_event_dma_adapter_service_id_get(TEST_ADAPTER_ID,
+ &adapter_service_id) == 0) {
+ rte_service_runstate_set(adapter_service_id, 0);
+ rte_service_lcore_stop(slcore_id);
+ rte_service_lcore_del(slcore_id);
+ rte_event_dma_adapter_stop(TEST_ADAPTER_ID);
+ }
+
+ if (rte_event_dev_service_id_get(evdev, &evdev_service_id) == 0) {
+ rte_service_runstate_set(evdev_service_id, 0);
+ rte_service_lcore_stop(slcore_id);
+ rte_service_lcore_del(slcore_id);
+ rte_dma_stop(TEST_DMA_DEV_ID);
+ rte_event_dev_stop(evdev);
+ } else {
+ rte_dma_stop(TEST_DMA_DEV_ID);
+ rte_event_dev_stop(evdev);
+ }
+}
+
+static int
+test_dma_adapter_conf(enum rte_event_dma_adapter_mode mode)
+{
+ uint32_t evdev_service_id;
+ uint8_t qid;
+ int ret;
+
+ if (!dma_adapter_setup_done) {
+ ret = configure_event_dma_adapter(mode);
+ if (ret)
+ return ret;
+ if (!params.internal_port_op_fwd) {
+ qid = TEST_DMA_EV_QUEUE_ID;
+ ret = rte_event_port_link(evdev,
+ params.dma_event_port_id, &qid, NULL, 1);
+ TEST_ASSERT(ret >= 0, "Failed to link queue %d "
+ "port=%u\n", qid,
+ params.dma_event_port_id);
+ }
+ dma_adapter_setup_done = 1;
+ }
+
+ /* retrieve service ids */
+ if (rte_event_dev_service_id_get(evdev, &evdev_service_id) == 0) {
+ /* add a service core and start it */
+ TEST_ASSERT_SUCCESS(rte_service_lcore_add(slcore_id),
+ "Failed to add service core");
+ TEST_ASSERT_SUCCESS(rte_service_lcore_start(slcore_id),
+ "Failed to start service core");
+
+ /* map services to it */
+ TEST_ASSERT_SUCCESS(rte_service_map_lcore_set(evdev_service_id,
+ slcore_id, 1), "Failed to map evdev service");
+
+ /* set services to running */
+ TEST_ASSERT_SUCCESS(rte_service_runstate_set(evdev_service_id,
+ 1), "Failed to start evdev service");
+ }
+
+ /* start the eventdev */
+ TEST_ASSERT_SUCCESS(rte_event_dev_start(evdev),
+ "Failed to start event device");
+
+ /* start the dma dev */
+ TEST_ASSERT_SUCCESS(rte_dma_start(TEST_DMA_DEV_ID),
+ "Failed to start dma device");
+
+ return TEST_SUCCESS;
+}
+
+static int
+test_dma_adapter_conf_op_forward_mode(void)
+{
+ enum rte_event_dma_adapter_mode mode;
+
+ mode = RTE_EVENT_DMA_ADAPTER_OP_FORWARD;
+
+ return test_dma_adapter_conf(mode);
+}
+
+static int
+testsuite_setup(void)
+{
+ int ret;
+
+ slcore_id = rte_get_next_lcore(-1, 1, 0);
+ TEST_ASSERT_NOT_EQUAL(slcore_id, RTE_MAX_LCORE, "At least 2 lcores "
+ "are required to run this autotest\n");
+
+ /* Setup and start event device. */
+ ret = configure_eventdev();
+ TEST_ASSERT_SUCCESS(ret, "Failed to setup eventdev\n");
+
+ /* Setup and start dma device. */
+ ret = configure_dmadev();
+ TEST_ASSERT_SUCCESS(ret, "dmadev initialization failed\n");
+
+ return TEST_SUCCESS;
+}
+
+static void
+dma_adapter_teardown(void)
+{
+ int ret;
+
+ ret = rte_event_dma_adapter_stop(TEST_ADAPTER_ID);
+ if (ret < 0)
+ RTE_LOG(ERR, USER1, "Failed to stop adapter!");
+
+ ret = rte_event_dma_adapter_vchan_del(TEST_ADAPTER_ID, TEST_DMA_DEV_ID,
+ TEST_DMA_VCHAN_ID);
+ if (ret < 0)
+ RTE_LOG(ERR, USER1, "Failed to delete vchan!");
+
+ ret = rte_event_dma_adapter_free(TEST_ADAPTER_ID);
+ if (ret < 0)
+ RTE_LOG(ERR, USER1, "Failed to free adapter!");
+
+ dma_adapter_setup_done = 0;
+}
+
+static void
+dma_teardown(void)
+{
+ /* Free mbuf mempool */
+ if (params.src_mbuf_pool != NULL) {
+ RTE_LOG(DEBUG, USER1, "DMA_ADAPTER_SRC_MBUFPOOL count %u\n",
+ rte_mempool_avail_count(params.src_mbuf_pool));
+ rte_mempool_free(params.src_mbuf_pool);
+ params.src_mbuf_pool = NULL;
+ }
+
+ if (params.dst_mbuf_pool != NULL) {
+ RTE_LOG(DEBUG, USER1, "DMA_ADAPTER_DST_MBUFPOOL count %u\n",
+ rte_mempool_avail_count(params.dst_mbuf_pool));
+ rte_mempool_free(params.dst_mbuf_pool);
+ params.dst_mbuf_pool = NULL;
+ }
+
+ /* Free ops mempool */
+ if (params.op_mpool != NULL) {
+ RTE_LOG(DEBUG, USER1, "EVENT_DMA_OP_POOL count %u\n",
+ rte_mempool_avail_count(params.op_mpool));
+ rte_mempool_free(params.op_mpool);
+ params.op_mpool = NULL;
+ }
+}
+
+static void
+eventdev_teardown(void)
+{
+ rte_event_dev_stop(evdev);
+}
+
+static void
+testsuite_teardown(void)
+{
+ dma_adapter_teardown();
+ dma_teardown();
+ eventdev_teardown();
+}
+
+static struct unit_test_suite functional_testsuite = {
+ .suite_name = "Event dma adapter test suite",
+ .setup = testsuite_setup,
+ .teardown = testsuite_teardown,
+ .unit_test_cases = {
+
+ TEST_CASE_ST(NULL, test_dma_adapter_free, test_dma_adapter_create),
+
+ TEST_CASE_ST(test_dma_adapter_create, test_dma_adapter_free,
+ test_dma_adapter_vchan_add_del),
+
+ TEST_CASE_ST(test_dma_adapter_create, test_dma_adapter_free,
+ test_dma_adapter_stats),
+
+ TEST_CASE_ST(test_dma_adapter_create, test_dma_adapter_free,
+ test_dma_adapter_params),
+
+ TEST_CASE_ST(test_dma_adapter_conf_op_forward_mode, test_dma_adapter_stop,
+ test_with_op_forward_mode),
+
+ TEST_CASES_END() /**< NULL terminate unit test array */
+ }
+};
+
+static int
+test_event_dma_adapter(void)
+{
+ return unit_test_suite_runner(&functional_testsuite);
+}
+
+#endif /* !RTE_EXEC_ENV_WINDOWS */
+
+REGISTER_TEST_COMMAND(event_dma_adapter_autotest, test_event_dma_adapter);
--
2.25.1
^ permalink raw reply [flat|nested] 110+ messages in thread
* [PATCH v8 00/12] event DMA adapter library support
2023-09-29 8:12 ` [PATCH v7 00/12] event DMA adapter library support Amit Prakash Shukla
` (11 preceding siblings ...)
2023-09-29 8:13 ` [PATCH v7 12/12] app/test: add event DMA adapter auto-test Amit Prakash Shukla
@ 2023-09-29 11:50 ` Amit Prakash Shukla
2023-09-29 11:50 ` [PATCH v8 01/12] eventdev/dma: introduce DMA adapter Amit Prakash Shukla
` (12 more replies)
12 siblings, 13 replies; 110+ messages in thread
From: Amit Prakash Shukla @ 2023-09-29 11:50 UTC (permalink / raw)
Cc: dev, jerinj, fengchengwen, kevin.laatz, bruce.richardson,
conor.walsh, vattunuru, g.singh, sachin.saxena, hemant.agrawal,
cheng1.jiang, ndabilpuram, anoobj, mb, Amit Prakash Shukla
This series adds support for event DMA adapter library. API's defined
as part of this library can be used by the application for DMA transfer
of data using event based mechanism.
v8:
- Re-arranged DMA adapter section in release notes.
v7:
- Resolved review comments.
v6:
- Resolved review comments.
- Updated git commit message.
v5:
- Resolved review comments.
v4:
- Fixed compilation error.
v3:
- Resolved checkpatch warnings.
- Fixed compilation error on intel.
- Updated git commit message.
v2:
- Resolved review comments.
- Patch split into multiple patches.
Amit Prakash Shukla (12):
eventdev/dma: introduce DMA adapter
eventdev/dma: support adapter capabilities get
eventdev/dma: support adapter create and free
eventdev/dma: support vchan add and delete
eventdev/dma: support adapter service function
eventdev/dma: support adapter start and stop
eventdev/dma: support adapter service ID get
eventdev/dma: support adapter runtime params
eventdev/dma: support adapter stats
eventdev/dma: support adapter enqueue
eventdev/dma: support adapter event port get
app/test: add event DMA adapter auto-test
MAINTAINERS | 7 +
app/test/meson.build | 1 +
app/test/test_event_dma_adapter.c | 805 +++++++++
config/rte_config.h | 1 +
doc/api/doxy-api-index.md | 1 +
doc/guides/eventdevs/features/default.ini | 8 +
doc/guides/prog_guide/event_dma_adapter.rst | 264 +++
doc/guides/prog_guide/eventdev.rst | 8 +-
.../img/event_dma_adapter_op_forward.svg | 1086 +++++++++++++
.../img/event_dma_adapter_op_new.svg | 1079 +++++++++++++
doc/guides/prog_guide/index.rst | 1 +
doc/guides/rel_notes/release_23_11.rst | 6 +
lib/eventdev/eventdev_pmd.h | 171 +-
lib/eventdev/eventdev_private.c | 10 +
lib/eventdev/meson.build | 4 +-
lib/eventdev/rte_event_dma_adapter.c | 1434 +++++++++++++++++
lib/eventdev/rte_event_dma_adapter.h | 581 +++++++
lib/eventdev/rte_eventdev.c | 23 +
lib/eventdev/rte_eventdev.h | 44 +
lib/eventdev/rte_eventdev_core.h | 8 +-
lib/eventdev/version.map | 16 +
lib/meson.build | 2 +-
22 files changed, 5553 insertions(+), 7 deletions(-)
create mode 100644 app/test/test_event_dma_adapter.c
create mode 100644 doc/guides/prog_guide/event_dma_adapter.rst
create mode 100644 doc/guides/prog_guide/img/event_dma_adapter_op_forward.svg
create mode 100644 doc/guides/prog_guide/img/event_dma_adapter_op_new.svg
create mode 100644 lib/eventdev/rte_event_dma_adapter.c
create mode 100644 lib/eventdev/rte_event_dma_adapter.h
--
2.25.1
^ permalink raw reply [flat|nested] 110+ messages in thread
* [PATCH v8 01/12] eventdev/dma: introduce DMA adapter
2023-09-29 11:50 ` [PATCH v8 00/12] event DMA adapter library support Amit Prakash Shukla
@ 2023-09-29 11:50 ` Amit Prakash Shukla
2023-09-29 11:50 ` [PATCH v8 02/12] eventdev/dma: support adapter capabilities get Amit Prakash Shukla
` (11 subsequent siblings)
12 siblings, 0 replies; 110+ messages in thread
From: Amit Prakash Shukla @ 2023-09-29 11:50 UTC (permalink / raw)
To: Thomas Monjalon, Amit Prakash Shukla, Jerin Jacob
Cc: dev, fengchengwen, kevin.laatz, bruce.richardson, conor.walsh,
vattunuru, g.singh, sachin.saxena, hemant.agrawal, cheng1.jiang,
ndabilpuram, anoobj, mb
Introduce event dma adapter interface to transfer packets between
dma device and event device.
Signed-off-by: Amit Prakash Shukla <amitprakashs@marvell.com>
Acked-by: Jerin Jacob <jerinj@marvell.com>
---
MAINTAINERS | 6 +
doc/api/doxy-api-index.md | 1 +
doc/guides/eventdevs/features/default.ini | 8 +
doc/guides/prog_guide/event_dma_adapter.rst | 264 ++++
doc/guides/prog_guide/eventdev.rst | 8 +-
.../img/event_dma_adapter_op_forward.svg | 1086 +++++++++++++++++
.../img/event_dma_adapter_op_new.svg | 1079 ++++++++++++++++
doc/guides/prog_guide/index.rst | 1 +
doc/guides/rel_notes/release_23_11.rst | 6 +
lib/eventdev/eventdev_pmd.h | 171 ++-
lib/eventdev/eventdev_private.c | 10 +
lib/eventdev/meson.build | 1 +
lib/eventdev/rte_event_dma_adapter.h | 581 +++++++++
lib/eventdev/rte_eventdev.h | 44 +
lib/eventdev/rte_eventdev_core.h | 8 +-
lib/eventdev/version.map | 16 +
lib/meson.build | 2 +-
17 files changed, 3286 insertions(+), 6 deletions(-)
create mode 100644 doc/guides/prog_guide/event_dma_adapter.rst
create mode 100644 doc/guides/prog_guide/img/event_dma_adapter_op_forward.svg
create mode 100644 doc/guides/prog_guide/img/event_dma_adapter_op_new.svg
create mode 100644 lib/eventdev/rte_event_dma_adapter.h
diff --git a/MAINTAINERS b/MAINTAINERS
index a926155f26..4ebbbe8bb3 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -540,6 +540,12 @@ F: lib/eventdev/*crypto_adapter*
F: app/test/test_event_crypto_adapter.c
F: doc/guides/prog_guide/event_crypto_adapter.rst
+Eventdev DMA Adapter API
+M: Amit Prakash Shukla <amitprakashs@marvell.com>
+T: git://dpdk.org/next/dpdk-next-eventdev
+F: lib/eventdev/*dma_adapter*
+F: doc/guides/prog_guide/event_dma_adapter.rst
+
Raw device API
M: Sachin Saxena <sachin.saxena@oss.nxp.com>
M: Hemant Agrawal <hemant.agrawal@nxp.com>
diff --git a/doc/api/doxy-api-index.md b/doc/api/doxy-api-index.md
index fdeda13932..b7df7be4d9 100644
--- a/doc/api/doxy-api-index.md
+++ b/doc/api/doxy-api-index.md
@@ -29,6 +29,7 @@ The public API headers are grouped by topics:
[event_eth_tx_adapter](@ref rte_event_eth_tx_adapter.h),
[event_timer_adapter](@ref rte_event_timer_adapter.h),
[event_crypto_adapter](@ref rte_event_crypto_adapter.h),
+ [event_dma_adapter](@ref rte_event_dma_adapter.h),
[rawdev](@ref rte_rawdev.h),
[metrics](@ref rte_metrics.h),
[bitrate](@ref rte_bitrate.h),
diff --git a/doc/guides/eventdevs/features/default.ini b/doc/guides/eventdevs/features/default.ini
index 00360f60c6..73a52d915b 100644
--- a/doc/guides/eventdevs/features/default.ini
+++ b/doc/guides/eventdevs/features/default.ini
@@ -44,6 +44,14 @@ internal_port_op_fwd =
internal_port_qp_ev_bind =
session_private_data =
+;
+; Features of a default DMA adapter.
+;
+[DMA adapter Features]
+internal_port_op_new =
+internal_port_op_fwd =
+internal_port_vchan_ev_bind =
+
;
; Features of a default Timer adapter.
;
diff --git a/doc/guides/prog_guide/event_dma_adapter.rst b/doc/guides/prog_guide/event_dma_adapter.rst
new file mode 100644
index 0000000000..701e50d042
--- /dev/null
+++ b/doc/guides/prog_guide/event_dma_adapter.rst
@@ -0,0 +1,264 @@
+.. SPDX-License-Identifier: BSD-3-Clause
+ Copyright (c) 2023 Marvell.
+
+Event DMA Adapter Library
+=========================
+
+DPDK :doc:`Eventdev library <eventdev>` provides event driven programming model with features
+to schedule events. :doc:`DMA Device library <dmadev>` provides an interface to DMA poll mode
+drivers that support DMA operations. Event DMA Adapter is intended to bridge between the event
+device and the DMA device.
+
+Packet flow from DMA device to the event device can be accomplished using software and hardware
+based transfer mechanisms. The adapter queries an eventdev PMD to determine which mechanism to
+be used. The adapter uses an EAL service core function for software based packet transfer and
+uses the eventdev PMD functions to configure hardware based packet transfer between DMA device
+and the event device. DMA adapter uses a new event type called ``RTE_EVENT_TYPE_DMADEV`` to
+indicate the source of event.
+
+Application can choose to submit an DMA operation directly to an DMA device or send it to an DMA
+adapter via eventdev based on ``RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP_FWD`` capability. The
+first mode is known as the event new (``RTE_EVENT_DMA_ADAPTER_OP_NEW``) mode and the second as the
+event forward (``RTE_EVENT_DMA_ADAPTER_OP_FORWARD``) mode. Choice of mode can be specified while
+creating the adapter. In the former mode, it is the application's responsibility to enable
+ingress packet ordering. In the latter mode, it is the adapter's responsibility to enable
+ingress packet ordering.
+
+
+Adapter Modes
+-------------
+
+RTE_EVENT_DMA_ADAPTER_OP_NEW mode
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+In the ``RTE_EVENT_DMA_ADAPTER_OP_NEW`` mode, application submits DMA operations directly to an DMA
+device. The adapter then dequeues DMA completions from the DMA device and enqueues them as events
+to the event device. This mode does not ensure ingress ordering as the application directly
+enqueues to the dmadev without going through DMA/atomic stage. In this mode, events dequeued
+from the adapter are treated as new events. The application has to specify event information
+(response information) which is needed to enqueue an event after the DMA operation is completed.
+
+.. _figure_event_dma_adapter_op_new:
+
+.. figure:: img/event_dma_adapter_op_new.*
+
+ Working model of ``RTE_EVENT_DMA_ADAPTER_OP_NEW`` mode
+
+
+RTE_EVENT_DMA_ADAPTER_OP_FORWARD mode
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+In the ``RTE_EVENT_DMA_ADAPTER_OP_FORWARD`` mode, if the event PMD and DMA PMD supports internal
+event port (``RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP_FWD``), the application should use
+``rte_event_dma_adapter_enqueue()`` API to enqueue DMA operations as events to DMA adapter. If
+not, application retrieves DMA adapter's event port using ``rte_event_dma_adapter_event_port_get()``
+API, links its event queue to this port and starts enqueuing DMA operations as events to eventdev
+using ``rte_event_enqueue_burst()``. The adapter then dequeues the events and submits the DMA
+operations to the dmadev. After the DMA operation is complete, the adapter enqueues events to the
+event device.
+
+Applications can use this mode when ingress packet ordering is needed. In this mode, events
+dequeued from the adapter will be treated as forwarded events. Application has to specify event
+information (response information) needed to enqueue the event after the DMA operation has
+completed.
+
+.. _figure_event_dma_adapter_op_forward:
+
+.. figure:: img/event_dma_adapter_op_forward.*
+
+ Working model of ``RTE_EVENT_DMA_ADAPTER_OP_FORWARD`` mode
+
+
+API Overview
+------------
+
+This section has a brief introduction to the event DMA adapter APIs. The application is expected
+to create an adapter which is associated with a single eventdev, then add dmadev and vchan to the
+adapter instance.
+
+
+Create an adapter instance
+~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+An adapter instance is created using ``rte_event_dma_adapter_create()``. This function is called
+with event device to be associated with the adapter and port configuration for the adapter to
+setup an event port (if the adapter needs to use a service function).
+
+Adapter can be started in ``RTE_EVENT_DMA_ADAPTER_OP_NEW`` or ``RTE_EVENT_DMA_ADAPTER_OP_FORWARD``
+mode.
+
+.. code-block:: c
+
+ enum rte_event_dma_adapter_mode mode;
+ struct rte_event_dev_info dev_info;
+ struct rte_event_port_conf conf;
+ uint8_t evdev_id;
+ uint8_t dma_id;
+ int ret;
+
+ ret = rte_event_dev_info_get(dma_id, &dev_info);
+
+ conf.new_event_threshold = dev_info.max_num_events;
+ conf.dequeue_depth = dev_info.max_event_port_dequeue_depth;
+ conf.enqueue_depth = dev_info.max_event_port_enqueue_depth;
+ mode = RTE_EVENT_DMA_ADAPTER_OP_FORWARD;
+ ret = rte_event_dma_adapter_create(dma_id, evdev_id, &conf, mode);
+
+
+``rte_event_dma_adapter_create_ext()`` function can be used by the application to have a finer
+control on eventdev port allocation and setup. The ``rte_event_dma_adapter_create_ext()``
+function is passed a callback function. The callback function is invoked if the adapter creates
+a service function and uses an event port for it. The callback is expected to fill the
+``struct rte_event_dma_adapter_conf`` structure passed to it.
+
+In the ``RTE_EVENT_DMA_ADAPTER_OP_FORWARD`` mode, if the event PMD and DMA PMD supports internal
+event port (``RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP_FWD``), events with DMA operations should
+be enqueued to the DMA adapter using ``rte_event_dma_adapter_enqueue()`` API. If not, the event port
+created by the adapter can be retrieved using ``rte_event_dma_adapter_event_port_get()`` API. An
+application can use this event port to link with an event queue, on which it enqueues events
+towards the DMA adapter using ``rte_event_enqueue_burst()``.
+
+.. code-block:: c
+
+ uint8_t dma_adpt_id, evdev_id, dma_dev_id, dma_ev_port_id, app_qid;
+ struct rte_event ev;
+ uint32_t cap;
+ int ret;
+
+ // Fill in event info and update event_ptr with rte_dma_op
+ memset(&ev, 0, sizeof(ev));
+ .
+ .
+ ev.event_ptr = op;
+
+ ret = rte_event_dma_adapter_caps_get(evdev_id, dma_dev_id, &cap);
+ if (cap & RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP_FWD) {
+ ret = rte_event_dma_adapter_enqueue(evdev_id, app_ev_port_id, ev, nb_events);
+ } else {
+ ret = rte_event_dma_adapter_event_port_get(dma_adpt_id, &dma_ev_port_id);
+ ret = rte_event_queue_setup(evdev_id, app_qid, NULL);
+ ret = rte_event_port_link(evdev_id, dma_ev_port_id, &app_qid, NULL, 1);
+ ev.queue_id = app_qid;
+ ret = rte_event_enqueue_burst(evdev_id, app_ev_port_id, ev, nb_events);
+ }
+
+
+Event device configuration for service based adapter
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+When ``rte_event_dma_adapter_create()`` is used for creating adapter instance,
+``rte_event_dev_config::nb_event_ports`` is automatically incremented, and event device is
+reconfigured with additional event port during service initialization. This event device
+reconfigure logic also increments the ``rte_event_dev_config::nb_single_link_event_port_queues``
+parameter if the adapter event port config is of type ``RTE_EVENT_PORT_CFG_SINGLE_LINK``.
+
+Applications using this mode of adapter creation need not configure the event device with
+``rte_event_dev_config::nb_event_ports`` and
+``rte_event_dev_config::nb_single_link_event_port_queues`` parameters required for DMA adapter when
+the adapter is created using the above-mentioned API.
+
+
+Querying adapter capabilities
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+The ``rte_event_dma_adapter_caps_get()`` function allows the application to query the adapter
+capabilities for an eventdev and dmadev combination. This API provides whether dmadev and eventdev
+are connected using internal HW port or not.
+
+.. code-block:: c
+
+ rte_event_dma_adapter_caps_get(dev_id, dma_dev_id, &cap);
+
+
+Adding vchan to the adapter instance
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+dmadev device id and vchan are configured using dmadev APIs. For more information
+see :doc:`here <dmadev>`.
+
+.. code-block:: c
+
+ struct rte_dma_vchan_conf vchan_conf;
+ struct rte_dma_conf dev_conf;
+ uint8_t dev_id = 0;
+ uint16_t vchan = 0;
+
+ rte_dma_configure(dev_id, &dev_conf);
+ rte_dma_vchan_setup(dev_id, vchan, &vchan_conf);
+
+These dmadev id and vchan are added to the instance using the
+``rte_event_dma_adapter_vchan_add()`` API. The same is removed using
+``rte_event_dma_adapter_vchan_del()`` API. If hardware supports
+``RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_VCHAN_EV_BIND`` capability, event information must be passed to the add API.
+
+.. code-block:: c
+
+ uint32_t cap;
+ int ret;
+
+ ret = rte_event_dma_adapter_caps_get(evdev_id, dma_dev_id, &cap);
+ if (cap & RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_VCHAN_EV_BIND) {
+ struct rte_event event;
+
+ rte_event_dma_adapter_vchan_add(id, dma_dev_id, vchan, &conf);
+ } else
+ rte_event_dma_adapter_vchan_add(id, dma_dev_id, vchan, NULL);
+
+
+Configuring service function
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+If the adapter uses a service function, the application is required to assign a service core to
+the service function as show below.
+
+.. code-block:: c
+
+ uint32_t service_id;
+
+ if (rte_event_dma_adapter_service_id_get(dma_id, &service_id) == 0)
+ rte_service_map_lcore_set(service_id, CORE_ID);
+
+
+Set event response information
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+In the RTE_EVENT_DMA_ADAPTER_OP_FORWARD / RTE_EVENT_DMA_ADAPTER_OP_NEW mode, the application
+specifies the dmadev ID and vchan ID in ``struct rte_event_dma_adapter_op`` and the event
+information (response information) needed to enqueue an event after the DMA operation has
+completed. The response information is specified in ``struct rte_event`` and appended to the
+``struct rte_event_dma_adapter_op``.
+
+
+Start the adapter instance
+~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+The application calls ``rte_event_dma_adapter_start()`` to start the adapter. This function calls
+the start callbacks of the eventdev PMDs for hardware based eventdev-dmadev connections and
+``rte_service_run_state_set()`` to enable the service function if one exists.
+
+.. code-block:: c
+
+ rte_event_dma_adapter_start(id);
+
+.. Note::
+
+ The eventdev to which the event_dma_adapter is connected should be started before calling
+ rte_event_dma_adapter_start().
+
+
+Get adapter statistics
+~~~~~~~~~~~~~~~~~~~~~~
+
+The ``rte_event_dma_adapter_stats_get()`` function reports counters defined in struct
+``rte_event_dma_adapter_stats``. The received packet and enqueued event counts are a sum of the
+counts from the eventdev PMD callbacks if the callback is supported, and the counts maintained by
+the service function, if one exists.
+
+Set/Get adapter runtime configuration parameters
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+The runtime configuration parameters of adapter can be set/get using
+``rte_event_dma_adapter_runtime_params_set()`` and
+``rte_event_dma_adapter_runtime_params_get()`` respectively.
+The parameters that can be set/get are defined in
+``struct rte_event_dma_adapter_runtime_params``.
diff --git a/doc/guides/prog_guide/eventdev.rst b/doc/guides/prog_guide/eventdev.rst
index 2c83176846..ff55115d0d 100644
--- a/doc/guides/prog_guide/eventdev.rst
+++ b/doc/guides/prog_guide/eventdev.rst
@@ -333,7 +333,8 @@ eventdev.
.. Note::
EventDev needs to be started before starting the event producers such
- as event_eth_rx_adapter, event_timer_adapter and event_crypto_adapter.
+ as event_eth_rx_adapter, event_timer_adapter, event_crypto_adapter and
+ event_dma_adapter.
Ingress of New Events
~~~~~~~~~~~~~~~~~~~~~
@@ -445,8 +446,9 @@ using ``rte_event_dev_stop_flush_callback_register()`` function.
.. Note::
The event producers such as ``event_eth_rx_adapter``,
- ``event_timer_adapter`` and ``event_crypto_adapter``
- need to be stopped before stopping the event device.
+ ``event_timer_adapter``, ``event_crypto_adapter`` and
+ ``event_dma_adapter`` need to be stopped before stopping
+ the event device.
Summary
-------
diff --git a/doc/guides/prog_guide/img/event_dma_adapter_op_forward.svg b/doc/guides/prog_guide/img/event_dma_adapter_op_forward.svg
new file mode 100644
index 0000000000..b7fe1fecf2
--- /dev/null
+++ b/doc/guides/prog_guide/img/event_dma_adapter_op_forward.svg
@@ -0,0 +1,1086 @@
+<?xml version="1.0" encoding="UTF-8" standalone="no"?>
+<!-- SPDX-License-Identifier: BSD-3-Clause -->
+<!-- Copyright (c) 2023 Marvell. -->
+<!-- Created with Inkscape (http://www.inkscape.org/) -->
+
+<svg
+ width="720px"
+ height="486px"
+ id="svg13237"
+ version="1.1"
+ inkscape:version="1.2.1 (9c6d41e410, 2022-07-14)"
+ sodipodi:docname="event_dma_adapter_op_forward.svg"
+ xmlns:inkscape="http://www.inkscape.org/namespaces/inkscape"
+ xmlns:sodipodi="http://sodipodi.sourceforge.net/DTD/sodipodi-0.dtd"
+ xmlns="http://www.w3.org/2000/svg"
+ xmlns:svg="http://www.w3.org/2000/svg"
+ xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#"
+ xmlns:cc="http://creativecommons.org/ns#"
+ xmlns:dc="http://purl.org/dc/elements/1.1/"
+ xmlns:v="http://schemas.microsoft.com/visio/2003/SVGExtensions/">
+ <defs
+ id="defs13239">
+ <marker
+ inkscape:stockid="Arrow1Sstart"
+ orient="auto"
+ refY="0.0"
+ refX="0.0"
+ id="Arrow1Sstart"
+ style="overflow:visible">
+ <path
+ id="path8416"
+ d="M 0.0,0.0 L 5.0,-5.0 L -12.5,0.0 L 5.0,5.0 L 0.0,0.0 z "
+ style="fill-rule:evenodd;stroke:#000000;stroke-width:1.0pt"
+ transform="scale(0.2) translate(6,0)" />
+ </marker>
+ <marker
+ inkscape:stockid="Arrow1Send"
+ orient="auto"
+ refY="0.0"
+ refX="0.0"
+ id="Arrow1Send"
+ style="overflow:visible;">
+ <path
+ id="path8419"
+ d="M 0.0,0.0 L 5.0,-5.0 L -12.5,0.0 L 5.0,5.0 L 0.0,0.0 z "
+ style="fill-rule:evenodd;stroke:#000000;stroke-width:1.0pt;"
+ transform="scale(0.2) rotate(180) translate(6,0)" />
+ </marker>
+ <marker
+ inkscape:stockid="DiamondL"
+ orient="auto"
+ refY="0.0"
+ refX="0.0"
+ id="DiamondL"
+ style="overflow:visible">
+ <path
+ id="path8483"
+ d="M 0,-7.0710768 L -7.0710894,0 L 0,7.0710589 L 7.0710462,0 L 0,-7.0710768 z "
+ style="fill-rule:evenodd;stroke:#000000;stroke-width:1.0pt"
+ transform="scale(0.8)" />
+ </marker>
+ <marker
+ inkscape:stockid="DotL"
+ orient="auto"
+ refY="0.0"
+ refX="0.0"
+ id="DotL"
+ style="overflow:visible">
+ <path
+ id="path8465"
+ d="M -2.5,-1.0 C -2.5,1.7600000 -4.7400000,4.0 -7.5,4.0 C -10.260000,4.0 -12.5,1.7600000 -12.5,-1.0 C -12.5,-3.7600000 -10.260000,-6.0 -7.5,-6.0 C -4.7400000,-6.0 -2.5,-3.7600000 -2.5,-1.0 z "
+ style="fill-rule:evenodd;stroke:#000000;stroke-width:1.0pt"
+ transform="scale(0.8) translate(7.4, 1)" />
+ </marker>
+ <marker
+ inkscape:stockid="SquareL"
+ orient="auto"
+ refY="0.0"
+ refX="0.0"
+ id="SquareL"
+ style="overflow:visible">
+ <path
+ id="path8474"
+ d="M -5.0,-5.0 L -5.0,5.0 L 5.0,5.0 L 5.0,-5.0 L -5.0,-5.0 z "
+ style="fill-rule:evenodd;stroke:#000000;stroke-width:1.0pt"
+ transform="scale(0.8)" />
+ </marker>
+ <marker
+ inkscape:stockid="TriangleOutL"
+ orient="auto"
+ refY="0.0"
+ refX="0.0"
+ id="TriangleOutL"
+ style="overflow:visible">
+ <path
+ id="path8546"
+ d="M 5.77,0.0 L -2.88,5.0 L -2.88,-5.0 L 5.77,0.0 z "
+ style="fill-rule:evenodd;stroke:#000000;stroke-width:1.0pt"
+ transform="scale(0.8)" />
+ </marker>
+ <marker
+ inkscape:stockid="Arrow1Lstart"
+ orient="auto"
+ refY="0.0"
+ refX="0.0"
+ id="Arrow1Lstart"
+ style="overflow:visible">
+ <path
+ id="path8404"
+ d="M 0.0,0.0 L 5.0,-5.0 L -12.5,0.0 L 5.0,5.0 L 0.0,0.0 z "
+ style="fill-rule:evenodd;stroke:#000000;stroke-width:1.0pt"
+ transform="scale(0.8) translate(12.5,0)" />
+ </marker>
+ <marker
+ inkscape:stockid="Arrow1Mend"
+ orient="auto"
+ refY="0.0"
+ refX="0.0"
+ id="Arrow1Mend"
+ style="overflow:visible;">
+ <path
+ id="path8413"
+ d="M 0.0,0.0 L 5.0,-5.0 L -12.5,0.0 L 5.0,5.0 L 0.0,0.0 z "
+ style="fill-rule:evenodd;stroke:#000000;stroke-width:1.0pt;"
+ transform="scale(0.4) rotate(180) translate(10,0)" />
+ </marker>
+ <marker
+ inkscape:stockid="Arrow2Lend"
+ orient="auto"
+ refY="0.0"
+ refX="0.0"
+ id="Arrow2Lend"
+ style="overflow:visible;">
+ <path
+ id="path8425"
+ style="fill-rule:evenodd;stroke-width:0.62500000;stroke-linejoin:round;"
+ d="M 8.7185878,4.0337352 L -2.2072895,0.016013256 L 8.7185884,-4.0017078 C 6.9730900,-1.6296469 6.9831476,1.6157441 8.7185878,4.0337352 z "
+ transform="scale(1.1) rotate(180) translate(1,0)" />
+ </marker>
+ <marker
+ inkscape:stockid="Arrow1Lend"
+ orient="auto"
+ refY="0.0"
+ refX="0.0"
+ id="Arrow1Lend"
+ style="overflow:visible;">
+ <path
+ id="path8407"
+ d="M 0.0,0.0 L 5.0,-5.0 L -12.5,0.0 L 5.0,5.0 L 0.0,0.0 z "
+ style="fill-rule:evenodd;stroke:#000000;stroke-width:1.0pt;"
+ transform="scale(0.8) rotate(180) translate(12.5,0)" />
+ </marker>
+ <filter
+ id="filter_2"
+ color-interpolation-filters="sRGB">
+ <feGaussianBlur
+ stdDeviation="2"
+ id="feGaussianBlur15" />
+ </filter>
+ <filter
+ id="filter_2-3"
+ color-interpolation-filters="sRGB"
+ x="-0.086178862"
+ y="-0.086178862"
+ width="1.1723577"
+ height="1.1723577">
+ <feGaussianBlur
+ stdDeviation="2"
+ id="feGaussianBlur15-1" />
+ </filter>
+ <filter
+ id="filter_2-0"
+ color-interpolation-filters="sRGB">
+ <feGaussianBlur
+ stdDeviation="2"
+ id="feGaussianBlur15-7" />
+ </filter>
+ <filter
+ id="filter_2-0-8"
+ color-interpolation-filters="sRGB">
+ <feGaussianBlur
+ stdDeviation="2"
+ id="feGaussianBlur15-7-7" />
+ </filter>
+ <filter
+ id="filter_2-3-9"
+ color-interpolation-filters="sRGB">
+ <feGaussianBlur
+ stdDeviation="2"
+ id="feGaussianBlur15-1-6" />
+ </filter>
+ <filter
+ id="filter_2-3-6"
+ color-interpolation-filters="sRGB">
+ <feGaussianBlur
+ stdDeviation="2"
+ id="feGaussianBlur15-1-63" />
+ </filter>
+ <filter
+ id="filter_2-3-91"
+ color-interpolation-filters="sRGB"
+ x="-0.086178862"
+ y="-0.086178862"
+ width="1.1723577"
+ height="1.1723577">
+ <feGaussianBlur
+ stdDeviation="2"
+ id="feGaussianBlur15-1-3" />
+ </filter>
+ <marker
+ inkscape:stockid="Arrow1Lend"
+ orient="auto"
+ refY="0"
+ refX="0"
+ id="Arrow1Lend-5"
+ style="overflow:visible">
+ <path
+ inkscape:connector-curvature="0"
+ id="path8407-3"
+ d="M 0,0 5,-5 -12.5,0 5,5 0,0 z"
+ style="fill-rule:evenodd;stroke:#000000;stroke-width:1pt"
+ transform="matrix(-0.8,0,0,-0.8,-10,0)" />
+ </marker>
+ <marker
+ inkscape:stockid="Arrow1Lend"
+ orient="auto"
+ refY="0"
+ refX="0"
+ id="Arrow1Lend-6"
+ style="overflow:visible">
+ <path
+ inkscape:connector-curvature="0"
+ id="path8407-0"
+ d="M 0,0 5,-5 -12.5,0 5,5 0,0 z"
+ style="fill-rule:evenodd;stroke:#000000;stroke-width:1pt"
+ transform="matrix(-0.8,0,0,-0.8,-10,0)" />
+ </marker>
+ <marker
+ inkscape:stockid="Arrow1Lstart"
+ orient="auto"
+ refY="0"
+ refX="0"
+ id="Arrow1Lstart-7"
+ style="overflow:visible">
+ <path
+ inkscape:connector-curvature="0"
+ id="path8404-0"
+ d="M 0,0 5,-5 -12.5,0 5,5 0,0 z"
+ style="fill-rule:evenodd;stroke:#000000;stroke-width:1pt"
+ transform="matrix(0.8,0,0,0.8,10,0)" />
+ </marker>
+ <marker
+ inkscape:stockid="Arrow1Lend"
+ orient="auto"
+ refY="0"
+ refX="0"
+ id="Arrow1Lend-51"
+ style="overflow:visible">
+ <path
+ inkscape:connector-curvature="0"
+ id="path8407-1"
+ d="M 0,0 5,-5 -12.5,0 5,5 0,0 z"
+ style="fill-rule:evenodd;stroke:#000000;stroke-width:1pt"
+ transform="matrix(-0.8,0,0,-0.8,-10,0)" />
+ </marker>
+ <marker
+ inkscape:stockid="Arrow1Lend"
+ orient="auto"
+ refY="0"
+ refX="0"
+ id="Arrow1Lend-3"
+ style="overflow:visible">
+ <path
+ inkscape:connector-curvature="0"
+ id="path8407-6"
+ d="M 0,0 5,-5 -12.5,0 5,5 0,0 z"
+ style="fill-rule:evenodd;stroke:#000000;stroke-width:1pt"
+ transform="matrix(-0.8,0,0,-0.8,-10,0)" />
+ </marker>
+ <marker
+ inkscape:stockid="Arrow1Lend"
+ orient="auto"
+ refY="0"
+ refX="0"
+ id="Arrow1Lend-62"
+ style="overflow:visible">
+ <path
+ inkscape:connector-curvature="0"
+ id="path8407-9"
+ d="M 0,0 5,-5 -12.5,0 5,5 0,0 z"
+ style="fill-rule:evenodd;stroke:#000000;stroke-width:1pt"
+ transform="matrix(-0.8,0,0,-0.8,-10,0)" />
+ </marker>
+ <marker
+ inkscape:stockid="Arrow1Lend"
+ orient="auto"
+ refY="0"
+ refX="0"
+ id="Arrow1Lend-2"
+ style="overflow:visible">
+ <path
+ inkscape:connector-curvature="0"
+ id="path8407-7"
+ d="M 0,0 5,-5 -12.5,0 5,5 0,0 z"
+ style="fill-rule:evenodd;stroke:#000000;stroke-width:1pt"
+ transform="matrix(-0.8,0,0,-0.8,-10,0)" />
+ </marker>
+ <marker
+ inkscape:stockid="Arrow1Lstart"
+ orient="auto"
+ refY="0"
+ refX="0"
+ id="Arrow1Lstart-7-9"
+ style="overflow:visible">
+ <path
+ inkscape:connector-curvature="0"
+ id="path8404-0-3"
+ d="M 0,0 5,-5 -12.5,0 5,5 0,0 z"
+ style="fill-rule:evenodd;stroke:#000000;stroke-width:1pt"
+ transform="matrix(0.8,0,0,0.8,10,0)" />
+ </marker>
+ <filter
+ id="filter_2-3-6-1"
+ color-interpolation-filters="sRGB">
+ <feGaussianBlur
+ stdDeviation="2"
+ id="feGaussianBlur15-1-63-8" />
+ </filter>
+ <filter
+ id="filter_2-3-92"
+ color-interpolation-filters="sRGB"
+ x="-0.086178862"
+ y="-0.086178862"
+ width="1.1723577"
+ height="1.1723577">
+ <feGaussianBlur
+ stdDeviation="2"
+ id="feGaussianBlur15-1-2" />
+ </filter>
+ <filter
+ id="filter_2-3-94"
+ color-interpolation-filters="sRGB"
+ x="-0.086178862"
+ y="-0.086178862"
+ width="1.1723577"
+ height="1.1723577">
+ <feGaussianBlur
+ stdDeviation="2"
+ id="feGaussianBlur15-1-7" />
+ </filter>
+ <marker
+ inkscape:stockid="Arrow1Lstart"
+ orient="auto"
+ refY="0"
+ refX="0"
+ id="Arrow1Lstart-7-6"
+ style="overflow:visible">
+ <path
+ inkscape:connector-curvature="0"
+ id="path8404-0-1"
+ d="M 0,0 5,-5 -12.5,0 5,5 0,0 z"
+ style="fill-rule:evenodd;stroke:#000000;stroke-width:1pt"
+ transform="matrix(0.8,0,0,0.8,10,0)" />
+ </marker>
+ <marker
+ inkscape:stockid="Arrow1Lend"
+ orient="auto"
+ refY="0"
+ refX="0"
+ id="Arrow1Lend-55"
+ style="overflow:visible">
+ <path
+ inkscape:connector-curvature="0"
+ id="path8407-4"
+ d="M 0,0 5,-5 -12.5,0 5,5 0,0 z"
+ style="fill-rule:evenodd;stroke:#000000;stroke-width:1pt"
+ transform="matrix(-0.8,0,0,-0.8,-10,0)" />
+ </marker>
+ </defs>
+ <sodipodi:namedview
+ id="base"
+ pagecolor="#ffffff"
+ bordercolor="#666666"
+ borderopacity="1.0"
+ inkscape:pageopacity="0.0"
+ inkscape:pageshadow="2"
+ inkscape:zoom="1"
+ inkscape:cx="233.5"
+ inkscape:cy="288"
+ inkscape:document-units="px"
+ inkscape:current-layer="layer1"
+ showgrid="false"
+ inkscape:window-width="1920"
+ inkscape:window-height="1017"
+ inkscape:window-x="-8"
+ inkscape:window-y="-8"
+ inkscape:window-maximized="1"
+ inkscape:snap-nodes="false"
+ inkscape:showpageshadow="2"
+ inkscape:pagecheckerboard="0"
+ inkscape:deskcolor="#d1d1d1">
+ <inkscape:grid
+ type="xygrid"
+ id="grid13454" />
+ </sodipodi:namedview>
+ <metadata
+ id="metadata13242">
+ <rdf:RDF>
+ <cc:Work
+ rdf:about="">
+ <dc:format>image/svg+xml</dc:format>
+ <dc:type
+ rdf:resource="http://purl.org/dc/dcmitype/StillImage" />
+ <dc:title />
+ </cc:Work>
+ </rdf:RDF>
+ </metadata>
+ <g
+ id="layer1"
+ inkscape:label="Layer 1"
+ inkscape:groupmode="layer">
+ <g
+ style="font-size:12px;fill:none;stroke-linecap:square;stroke-miterlimit:3;overflow:visible"
+ id="shape1-1-2-4"
+ v:mID="1"
+ v:groupContext="shape"
+ transform="matrix(2.1604167,0,0,1.5671361,88.874699,-812.39909)">
+ <title
+ id="title22-7-5">Square</title>
+ <desc
+ id="desc24-7-8">Atomic Queue #1</desc>
+ <v:userDefs>
+ <v:ud
+ v:nameU="visVersion"
+ v:val="VT0(15):26" />
+ </v:userDefs>
+ <v:textBlock
+ v:margins="rect(4,4,4,4)" />
+ <v:textRect
+ cx="30.75"
+ cy="581.25"
+ width="61.5"
+ height="61.5" />
+ <g
+ id="shadow1-2-9-5"
+ v:groupContext="shadow"
+ v:shadowOffsetX="0.345598"
+ v:shadowOffsetY="-1.97279"
+ v:shadowType="1"
+ transform="translate(0.345598,1.97279)"
+ class="st1"
+ style="visibility:visible">
+ <rect
+ x="0"
+ y="550.5"
+ width="61.5"
+ height="61.5"
+ class="st2"
+ id="rect27-8-7"
+ style="fill:#5b9bd5;fill-opacity:0.22000002;stroke:#5b9bd5;stroke-opacity:0.22000002;filter:url(#filter_2-3-91)" />
+ </g>
+ <g
+ id="g13515-33">
+ <g
+ id="g13534-8">
+ <rect
+ x="0"
+ y="550.5"
+ width="61.5"
+ height="61.5"
+ class="st3"
+ id="rect29-1-95"
+ style="fill:#5b9bd5;stroke:#c7c8c8;stroke-width:0.25" />
+ </g>
+ </g>
+ </g>
+ <path
+ style="fill:none;stroke:#000000;stroke-width:0.712265;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:4;stroke-dasharray:none;stroke-opacity:1;marker-start:url(#Arrow1Lstart-7);marker-end:none"
+ d="M 312.28671,240.74335 H 227.99897"
+ id="path17209"
+ inkscape:connector-type="orthogonal"
+ inkscape:connector-curvature="0" />
+ <path
+ style="fill:none;stroke:#000000;stroke-width:0.718986px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1;marker-mid:none;marker-end:url(#Arrow1Lend)"
+ d="m 221.6484,77.57125 h 94.28101"
+ id="path17209-8"
+ inkscape:connector-type="orthogonal"
+ inkscape:connector-curvature="0" />
+ <g
+ style="font-size:12px;fill:none;stroke-linecap:square;stroke-miterlimit:3;overflow:visible"
+ id="shape1-1-2"
+ v:mID="1"
+ v:groupContext="shape"
+ transform="matrix(2.1604167,0,0,1.5671361,314.24227,-811.89589)">
+ <title
+ id="title22-7">Square</title>
+ <desc
+ id="desc24-7">Atomic Queue #1</desc>
+ <v:userDefs>
+ <v:ud
+ v:nameU="visVersion"
+ v:val="VT0(15):26" />
+ </v:userDefs>
+ <v:textBlock
+ v:margins="rect(4,4,4,4)" />
+ <v:textRect
+ cx="30.75"
+ cy="581.25"
+ width="61.5"
+ height="61.5" />
+ <g
+ id="shadow1-2-9"
+ v:groupContext="shadow"
+ v:shadowOffsetX="0.345598"
+ v:shadowOffsetY="-1.97279"
+ v:shadowType="1"
+ transform="translate(0.345598,1.97279)"
+ class="st1"
+ style="visibility:visible">
+ <rect
+ x="0"
+ y="550.5"
+ width="61.5"
+ height="61.5"
+ class="st2"
+ id="rect27-8"
+ style="fill:#5b9bd5;fill-opacity:0.22000002;stroke:#5b9bd5;stroke-opacity:0.22000002;filter:url(#filter_2-3)" />
+ </g>
+ <g
+ id="g13515">
+ <g
+ id="g13534">
+ <rect
+ x="0"
+ y="550.5"
+ width="61.5"
+ height="61.5"
+ class="st3"
+ id="rect29-1"
+ style="fill:#5b9bd5;stroke:#c7c8c8;stroke-width:0.25" />
+ </g>
+ </g>
+ </g>
+ <path
+ style="fill:none;stroke:#000000;stroke-width:0.724714;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:4;stroke-dasharray:none;stroke-opacity:1;marker-start:url(#Arrow1Lstart);marker-end:none"
+ d="M 89.025329,74.39932 H 24.750043"
+ id="path17209-3"
+ inkscape:connector-type="orthogonal"
+ inkscape:connector-curvature="0" />
+ <path
+ transform="matrix(0.73232502,0,0,0.75477602,-4.325033,28.642983)"
+ sodipodi:type="arc"
+ style="fill:#539de6;fill-opacity:1;stroke:#0000ea;stroke-width:1;stroke-linecap:square;stroke-miterlimit:4;stroke-opacity:1;stroke-dasharray:none"
+ id="path29161-3"
+ sodipodi:cx="371"
+ sodipodi:cy="64.5"
+ sodipodi:rx="17"
+ sodipodi:ry="15.5"
+ d="m 388,64.5 a 17,15.5 0 1 1 -34,0 17,15.5 0 1 1 34,0 z" />
+ <path
+ transform="matrix(0.73232502,0,0,0.75477602,-1.93108,192.80833)"
+ sodipodi:type="arc"
+ style="fill:#539de6;fill-opacity:1;stroke:#0000ea;stroke-width:1;stroke-linecap:square;stroke-miterlimit:4;stroke-opacity:1;stroke-dasharray:none"
+ id="path29161-1"
+ sodipodi:cx="371"
+ sodipodi:cy="64.5"
+ sodipodi:rx="17"
+ sodipodi:ry="15.5"
+ d="m 388,64.5 a 17,15.5 0 1 1 -34,0 17,15.5 0 1 1 34,0 z" />
+ <path
+ style="fill:none;stroke:#000000;stroke-width:0.751412;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:4;stroke-dasharray:none;stroke-opacity:1;marker-start:url(#Arrow1Lstart-7);marker-end:none"
+ d="M 18.763392,120.7432 H 87.758545"
+ id="path17209-3-0"
+ inkscape:connector-type="orthogonal"
+ inkscape:connector-curvature="0" />
+ <path
+ sodipodi:type="arc"
+ style="fill:#539de6;fill-opacity:1;stroke:#0000ea;stroke-width:1;stroke-linecap:square;stroke-miterlimit:4;stroke-opacity:1;stroke-dasharray:none"
+ id="path29161"
+ sodipodi:cx="371"
+ sodipodi:cy="64.5"
+ sodipodi:rx="17"
+ sodipodi:ry="15.5"
+ d="m 388,64.5 a 17,15.5 0 1 1 -34,0 17,15.5 0 1 1 34,0 z"
+ transform="matrix(0.73232502,0,0,0.75477602,-218.16394,72.68276)" />
+ <path
+ sodipodi:type="arc"
+ style="fill:#539de6;fill-opacity:1;stroke:#0000ea;stroke-width:1;stroke-linecap:square;stroke-miterlimit:4;stroke-opacity:1;stroke-dasharray:none"
+ id="path29161-2"
+ sodipodi:cx="371"
+ sodipodi:cy="64.5"
+ sodipodi:rx="17"
+ sodipodi:ry="15.5"
+ d="m 388,64.5 a 17,15.5 0 1 1 -34,0 17,15.5 0 1 1 34,0 z"
+ transform="matrix(0.73232502,0,0,0.75477602,-217.40136,26.716271)" />
+ <g
+ id="g29167-4"
+ transform="matrix(0.73232502,0,0,0.75477602,-217.31662,28.007562)">
+ <text
+ id="text29163-9"
+ y="70"
+ x="365"
+ style="font-style:normal;font-weight:normal;line-height:0%;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none"
+ xml:space="preserve"><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="70"
+ x="365"
+ id="tspan29165-9"
+ sodipodi:role="line">1</tspan></text>
+ </g>
+ <g
+ id="g29167-9"
+ transform="matrix(0.73232502,0,0,0.75477602,-4.9726112,28.689051)">
+ <text
+ id="text29163-3"
+ y="70"
+ x="365"
+ style="font-style:normal;font-weight:normal;line-height:0%;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none"
+ xml:space="preserve"><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="70"
+ x="365"
+ id="tspan29165-3"
+ sodipodi:role="line">2</tspan></text>
+ </g>
+ <path
+ style="fill:none;stroke:#000000;stroke-width:0.678033px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1;marker-start:url(#Arrow1Lstart-7);marker-end:none"
+ d="M 181,214.66098 V 145.33902"
+ id="path17211-7-1-6"
+ inkscape:connector-type="orthogonal"
+ inkscape:connector-curvature="0" />
+ <g
+ id="g29167"
+ transform="matrix(0.73232502,0,0,0.75477602,-218.07919,73.10621)">
+ <text
+ id="text29163"
+ y="70"
+ x="365"
+ style="font-style:normal;font-weight:normal;line-height:0%;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none"
+ xml:space="preserve"><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="70"
+ x="365"
+ id="tspan29165"
+ sodipodi:role="line">8</tspan></text>
+ </g>
+ <path
+ style="fill:none;stroke:#000000;stroke-width:0.678033px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1;marker-start:url(#Arrow1Lstart-7);marker-end:none"
+ d="m 131,145.8531 v 69.32197"
+ id="path17211-7-1"
+ inkscape:connector-type="orthogonal"
+ inkscape:connector-curvature="0" />
+ <path
+ transform="matrix(0.73232502,0,0,0.75477602,-140.37076,129.97088)"
+ sodipodi:type="arc"
+ style="fill:#539de6;fill-opacity:1;stroke:#0000ea;stroke-width:1;stroke-linecap:square;stroke-miterlimit:4;stroke-opacity:1;stroke-dasharray:none"
+ id="path29161-8"
+ sodipodi:cx="371"
+ sodipodi:cy="64.5"
+ sodipodi:rx="17"
+ sodipodi:ry="15.5"
+ d="m 388,64.5 a 17,15.5 0 1 1 -34,0 17,15.5 0 1 1 34,0 z" />
+ <g
+ id="g29167-2"
+ transform="matrix(0.73232502,0,0,0.75477602,-140.28602,131.01695)">
+ <text
+ id="text29163-92"
+ y="70"
+ x="365"
+ style="font-style:normal;font-weight:normal;line-height:0%;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none"
+ xml:space="preserve"><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="70"
+ x="365"
+ id="tspan29165-8"
+ sodipodi:role="line">7</tspan></text>
+ </g>
+ <path
+ style="fill:none;stroke:#000000;stroke-width:0.718986px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1;marker-mid:none;marker-end:url(#Arrow1Lend)"
+ d="m 317.1405,116 h -94.281"
+ id="path17209-8-0"
+ inkscape:connector-type="orthogonal"
+ inkscape:connector-curvature="0" />
+ <path
+ transform="matrix(0.73232502,0,0,0.75477602,-3.4914,66.68745)"
+ sodipodi:type="arc"
+ style="fill:#539de6;fill-opacity:1;stroke:#0000ea;stroke-width:1;stroke-linecap:square;stroke-miterlimit:4;stroke-opacity:1;stroke-dasharray:none"
+ id="path29161-6"
+ sodipodi:cx="371"
+ sodipodi:cy="64.5"
+ sodipodi:rx="17"
+ sodipodi:ry="15.5"
+ d="m 388,64.5 a 17,15.5 0 1 1 -34,0 17,15.5 0 1 1 34,0 z" />
+ <g
+ id="g29167-46"
+ transform="matrix(0.73232502,0,0,0.75477602,-4.40666,67.48829)">
+ <text
+ id="text29163-1"
+ y="70"
+ x="365"
+ style="font-style:normal;font-weight:normal;line-height:0%;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none"
+ xml:space="preserve"><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="70"
+ x="365"
+ id="tspan29165-5"
+ sodipodi:role="line">3</tspan></text>
+ </g>
+ <path
+ transform="matrix(0.73232502,0,0,0.75477602,-90.692582,130.31695)"
+ sodipodi:type="arc"
+ style="fill:#539de6;fill-opacity:1;stroke:#0000ea;stroke-width:1;stroke-linecap:square;stroke-miterlimit:4;stroke-opacity:1;stroke-dasharray:none"
+ id="path29161-8-6"
+ sodipodi:cx="371"
+ sodipodi:cy="64.5"
+ sodipodi:rx="17"
+ sodipodi:ry="15.5"
+ d="m 388,64.5 a 17,15.5 0 1 1 -34,0 17,15.5 0 1 1 34,0 z" />
+ <g
+ id="g29167-6"
+ transform="matrix(0.73232502,0,0,0.75477602,-90.84634,131.60918)">
+ <text
+ id="text29163-17"
+ y="70"
+ x="365"
+ style="font-style:normal;font-weight:normal;line-height:0%;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none"
+ xml:space="preserve"><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="70"
+ x="365"
+ id="tspan29165-2"
+ sodipodi:role="line">4</tspan></text>
+ </g>
+ <g
+ id="g29167-2-0"
+ transform="matrix(0.73232502,0,0,0.75477602,-2.424397,194.0216)">
+ <text
+ id="text29163-92-6"
+ y="70"
+ x="365"
+ style="font-style:normal;font-weight:normal;line-height:0%;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none"
+ xml:space="preserve"><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="70"
+ x="365"
+ id="tspan29165-8-2"
+ sodipodi:role="line">5</tspan></text>
+ </g>
+ <g
+ style="font-size:12px;fill:none;stroke-linecap:square;stroke-miterlimit:3;overflow:visible"
+ id="shape1-1-2-8"
+ v:mID="1"
+ v:groupContext="shape"
+ transform="matrix(2.1604167,0,0,1.5671361,93.82055,-648.98949)">
+ <title
+ id="title22-7-97">Square</title>
+ <desc
+ id="desc24-7-3">Atomic Queue #1</desc>
+ <v:userDefs>
+ <v:ud
+ v:nameU="visVersion"
+ v:val="VT0(15):26" />
+ </v:userDefs>
+ <v:textBlock
+ v:margins="rect(4,4,4,4)" />
+ <v:textRect
+ cx="30.75"
+ cy="581.25"
+ width="61.5"
+ height="61.5" />
+ <g
+ id="shadow1-2-9-6"
+ v:groupContext="shadow"
+ v:shadowOffsetX="0.345598"
+ v:shadowOffsetY="-1.97279"
+ v:shadowType="1"
+ transform="translate(0.345598,1.97279)"
+ class="st1"
+ style="visibility:visible">
+ <rect
+ x="0"
+ y="550.5"
+ width="61.5"
+ height="61.5"
+ class="st2"
+ id="rect27-8-12"
+ style="fill:#5b9bd5;fill-opacity:0.22000002;stroke:#5b9bd5;stroke-opacity:0.22000002;filter:url(#filter_2-3-92)" />
+ </g>
+ <g
+ id="g13515-9">
+ <g
+ id="g13534-3">
+ <rect
+ x="0"
+ y="550.5"
+ width="61.5"
+ height="61.5"
+ class="st3"
+ id="rect29-1-1"
+ style="fill:#5b9bd5;stroke:#c7c8c8;stroke-width:0.25" />
+ </g>
+ </g>
+ </g>
+ <g
+ style="font-size:12px;fill:none;stroke-linecap:square;stroke-miterlimit:3;overflow:visible"
+ id="shape1-1-2-84"
+ v:mID="1"
+ v:groupContext="shape"
+ transform="matrix(2.1604167,0,0,1.5671361,314.82055,-648.98949)">
+ <title
+ id="title22-7-50">Square</title>
+ <desc
+ id="desc24-7-36">Atomic Queue #1</desc>
+ <v:userDefs>
+ <v:ud
+ v:nameU="visVersion"
+ v:val="VT0(15):26" />
+ </v:userDefs>
+ <v:textBlock
+ v:margins="rect(4,4,4,4)" />
+ <v:textRect
+ cx="30.75"
+ cy="581.25"
+ width="61.5"
+ height="61.5" />
+ <g
+ id="shadow1-2-9-1"
+ v:groupContext="shadow"
+ v:shadowOffsetX="0.345598"
+ v:shadowOffsetY="-1.97279"
+ v:shadowType="1"
+ transform="translate(0.345598,1.97279)"
+ class="st1"
+ style="visibility:visible">
+ <rect
+ x="0"
+ y="550.5"
+ width="61.5"
+ height="61.5"
+ class="st2"
+ id="rect27-8-0"
+ style="fill:#5b9bd5;fill-opacity:0.22000002;stroke:#5b9bd5;stroke-opacity:0.22000002;filter:url(#filter_2-3-94)" />
+ </g>
+ <g
+ id="g13515-6">
+ <g
+ id="g13534-32">
+ <rect
+ x="0"
+ y="550.5"
+ width="61.5"
+ height="61.5"
+ class="st3"
+ id="rect29-1-0"
+ style="fill:#5b9bd5;stroke:#c7c8c8;stroke-width:0.25" />
+ </g>
+ </g>
+ </g>
+ <path
+ style="fill:none;stroke:#000000;stroke-width:0.712265;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:4;stroke-dasharray:none;stroke-opacity:1;marker-start:none;marker-end:url(#Arrow1Lend)"
+ d="M 313.14387,285 H 228.85613"
+ id="path17209-7"
+ inkscape:connector-type="orthogonal"
+ inkscape:connector-curvature="0" />
+ <path
+ transform="matrix(0.73232502,0,0,0.75477602,-2.692582,236.31695)"
+ sodipodi:type="arc"
+ style="fill:#539de6;fill-opacity:1;stroke:#0000ea;stroke-width:1;stroke-linecap:square;stroke-miterlimit:4;stroke-opacity:1;stroke-dasharray:none"
+ id="path29161-1-6"
+ sodipodi:cx="371"
+ sodipodi:cy="64.5"
+ sodipodi:rx="17"
+ sodipodi:ry="15.5"
+ d="m 388,64.5 a 17,15.5 0 1 1 -34,0 17,15.5 0 1 1 34,0 z" />
+ <g
+ id="g29167-2-0-5"
+ transform="matrix(0.73232502,0,0,0.75477602,-2.424397,237.0216)">
+ <text
+ id="text29163-92-6-6"
+ y="70"
+ x="365"
+ style="font-style:normal;font-weight:normal;line-height:0%;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none"
+ xml:space="preserve"><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="70"
+ x="365"
+ id="tspan29165-8-2-9"
+ sodipodi:role="line">6</tspan></text>
+ </g>
+ <g
+ id="g29167-4-3"
+ transform="matrix(0.73232502,0,0,0.75477602,-154.60784,51.117791)">
+ <text
+ id="text29163-9-6"
+ y="70"
+ x="365"
+ style="font-style:normal;font-weight:normal;line-height:0%;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none"
+ xml:space="preserve"><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:24.2109px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="70"
+ x="365"
+ id="tspan29165-9-7"
+ sodipodi:role="line">Eventdev</tspan></text>
+ </g>
+ <g
+ id="g29167-4-3-5"
+ transform="matrix(0.73232502,0,0,0.75477602,-144.65044,201.97821)">
+ <text
+ id="text29163-9-6-3"
+ y="70"
+ x="412.93716"
+ style="font-style:normal;font-weight:normal;line-height:0%;font-family:sans-serif;text-align:center;letter-spacing:0px;word-spacing:0px;text-anchor:middle;fill:#000000;fill-opacity:1;stroke:none"
+ xml:space="preserve"><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:24.2109px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:center;writing-mode:lr-tb;text-anchor:middle"
+ y="70"
+ x="412.93716"
+ id="tspan29165-9-7-5"
+ sodipodi:role="line">DMA</tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:24.2109px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:center;writing-mode:lr-tb;text-anchor:middle"
+ y="100.26363"
+ x="412.93716"
+ sodipodi:role="line"
+ id="tspan3201">Adapter</tspan></text>
+ </g>
+ <g
+ id="g29167-4-3-5-6"
+ transform="matrix(0.73232502,0,0,0.75477602,79.53518,46.62529)">
+ <text
+ id="text29163-9-6-3-2"
+ y="48.801659"
+ x="412.93716"
+ style="font-style:normal;font-weight:normal;line-height:0%;font-family:sans-serif;text-align:center;letter-spacing:0px;word-spacing:0px;text-anchor:middle;fill:#000000;fill-opacity:1;stroke:none"
+ xml:space="preserve"><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:24.2109px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:center;writing-mode:lr-tb;text-anchor:middle"
+ y="48.801659"
+ x="412.93716"
+ sodipodi:role="line"
+ id="tspan3155">Application</tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:24.2109px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:center;writing-mode:lr-tb;text-anchor:middle"
+ y="98.801659"
+ x="412.93716"
+ sodipodi:role="line"
+ id="tspan3201-1">in ordered</tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:24.2109px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:center;writing-mode:lr-tb;text-anchor:middle"
+ y="148.80167"
+ x="412.93716"
+ sodipodi:role="line"
+ id="tspan3161">stage</tspan></text>
+ </g>
+ <g
+ id="g29167-4-3-5-2"
+ transform="matrix(0.73232502,0,0,0.75477602,77.535182,213.62529)">
+ <text
+ id="text29163-9-6-3-7"
+ y="70"
+ x="412.93716"
+ style="font-style:normal;font-weight:normal;line-height:0%;font-family:sans-serif;text-align:center;letter-spacing:0px;word-spacing:0px;text-anchor:middle;fill:#000000;fill-opacity:1;stroke:none"
+ xml:space="preserve"><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:24.2109px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:center;writing-mode:lr-tb;text-anchor:middle"
+ y="70"
+ x="412.93716"
+ sodipodi:role="line"
+ id="tspan3201-9">DMA Device</tspan></text>
+ </g>
+ <g
+ id="g29167-4-3-5-3"
+ transform="matrix(0.73232502,0,0,0.75477602,188.53518,-3.37471)">
+ <text
+ id="text29163-9-6-3-6"
+ y="70"
+ x="375.65271"
+ style="font-style:normal;font-weight:normal;line-height:0%;font-family:sans-serif;text-align:center;letter-spacing:0px;word-spacing:0px;text-anchor:middle;fill:#000000;fill-opacity:1;stroke:none"
+ xml:space="preserve"><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18.8307px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="70"
+ x="375.65271"
+ sodipodi:role="line"
+ id="tspan3201-6">1. Events from the previous stage.</tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18.8307px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="93.538376"
+ x="375.65271"
+ sodipodi:role="line"
+ id="tspan3260"> </tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18.8307px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="117.07675"
+ x="375.65271"
+ sodipodi:role="line"
+ id="tspan3262">2. Application in ordered stage</tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18.8307px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="140.61513"
+ x="375.65271"
+ sodipodi:role="line"
+ id="tspan3288"> dequeues events from eventdev.</tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18.8307px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="164.1535"
+ x="375.65271"
+ sodipodi:role="line"
+ id="tspan3264"> </tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18.8307px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="187.69188"
+ x="375.65271"
+ sodipodi:role="line"
+ id="tspan3266">3. Application enqueues DMA</tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18.8307px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="211.23026"
+ x="375.65271"
+ sodipodi:role="line"
+ id="tspan3290"> operations as events to eventdev.</tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18.8307px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="234.76863"
+ x="375.65271"
+ sodipodi:role="line"
+ id="tspan3268"> </tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18.8307px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="258.30701"
+ x="375.65271"
+ sodipodi:role="line"
+ id="tspan3270">4. DMA adapter dequeues event</tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18.8307px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="281.84537"
+ x="375.65271"
+ sodipodi:role="line"
+ id="tspan3292"> from eventdev.</tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18.8307px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="305.38376"
+ x="375.65271"
+ sodipodi:role="line"
+ id="tspan3272"> </tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18.8307px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="328.92212"
+ x="375.65271"
+ sodipodi:role="line"
+ id="tspan3274">5. DMA adapter submits DMA</tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18.8307px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="352.46051"
+ x="375.65271"
+ sodipodi:role="line"
+ id="tspan3294"> operations to DMA Device (Atomic</tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18.8307px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="375.99887"
+ x="375.65271"
+ sodipodi:role="line"
+ id="tspan3296"> stage)</tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18.8307px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="399.53726"
+ x="375.65271"
+ sodipodi:role="line"
+ id="tspan3276"> </tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18.8307px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="423.07562"
+ x="375.65271"
+ sodipodi:role="line"
+ id="tspan3278">6. DMA adapter dequeues DMA</tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18.8307px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="446.61401"
+ x="375.65271"
+ sodipodi:role="line"
+ id="tspan3298"> completions from DMA Device</tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18.8307px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="470.15237"
+ x="375.65271"
+ sodipodi:role="line"
+ id="tspan3280"> </tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18.8307px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="493.69073"
+ x="375.65271"
+ sodipodi:role="line"
+ id="tspan3282">7. DMA adapter enqueues events</tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18.8307px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="517.22913"
+ x="375.65271"
+ sodipodi:role="line"
+ id="tspan3300"> to the eventdev</tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18.8307px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="540.76752"
+ x="375.65271"
+ sodipodi:role="line"
+ id="tspan3284"> </tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18.8307px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="564.30585"
+ x="375.65271"
+ sodipodi:role="line"
+ id="tspan3286">8. Events to the next stage</tspan></text>
+ </g>
+ </g>
+</svg>
diff --git a/doc/guides/prog_guide/img/event_dma_adapter_op_new.svg b/doc/guides/prog_guide/img/event_dma_adapter_op_new.svg
new file mode 100644
index 0000000000..e9e8bb2b98
--- /dev/null
+++ b/doc/guides/prog_guide/img/event_dma_adapter_op_new.svg
@@ -0,0 +1,1079 @@
+<?xml version="1.0" encoding="UTF-8" standalone="no"?>
+<!-- SPDX-License-Identifier: BSD-3-Clause -->
+<!-- Copyright (c) 2023 Marvell. -->
+<!-- Created with Inkscape (http://www.inkscape.org/) -->
+
+<svg
+ width="720px"
+ height="486px"
+ id="svg13237"
+ version="1.1"
+ inkscape:version="1.2.1 (9c6d41e410, 2022-07-14)"
+ sodipodi:docname="event_dma_adapter_op_new.svg"
+ xmlns:inkscape="http://www.inkscape.org/namespaces/inkscape"
+ xmlns:sodipodi="http://sodipodi.sourceforge.net/DTD/sodipodi-0.dtd"
+ xmlns="http://www.w3.org/2000/svg"
+ xmlns:svg="http://www.w3.org/2000/svg"
+ xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#"
+ xmlns:cc="http://creativecommons.org/ns#"
+ xmlns:dc="http://purl.org/dc/elements/1.1/"
+ xmlns:v="http://schemas.microsoft.com/visio/2003/SVGExtensions/">
+ <defs
+ id="defs13239">
+ <marker
+ inkscape:stockid="Arrow1Sstart"
+ orient="auto"
+ refY="0.0"
+ refX="0.0"
+ id="Arrow1Sstart"
+ style="overflow:visible">
+ <path
+ id="path8416"
+ d="M 0.0,0.0 L 5.0,-5.0 L -12.5,0.0 L 5.0,5.0 L 0.0,0.0 z "
+ style="fill-rule:evenodd;stroke:#000000;stroke-width:1.0pt"
+ transform="scale(0.2) translate(6,0)" />
+ </marker>
+ <marker
+ inkscape:stockid="Arrow1Send"
+ orient="auto"
+ refY="0.0"
+ refX="0.0"
+ id="Arrow1Send"
+ style="overflow:visible;">
+ <path
+ id="path8419"
+ d="M 0.0,0.0 L 5.0,-5.0 L -12.5,0.0 L 5.0,5.0 L 0.0,0.0 z "
+ style="fill-rule:evenodd;stroke:#000000;stroke-width:1.0pt;"
+ transform="scale(0.2) rotate(180) translate(6,0)" />
+ </marker>
+ <marker
+ inkscape:stockid="DiamondL"
+ orient="auto"
+ refY="0.0"
+ refX="0.0"
+ id="DiamondL"
+ style="overflow:visible">
+ <path
+ id="path8483"
+ d="M 0,-7.0710768 L -7.0710894,0 L 0,7.0710589 L 7.0710462,0 L 0,-7.0710768 z "
+ style="fill-rule:evenodd;stroke:#000000;stroke-width:1.0pt"
+ transform="scale(0.8)" />
+ </marker>
+ <marker
+ inkscape:stockid="DotL"
+ orient="auto"
+ refY="0.0"
+ refX="0.0"
+ id="DotL"
+ style="overflow:visible">
+ <path
+ id="path8465"
+ d="M -2.5,-1.0 C -2.5,1.7600000 -4.7400000,4.0 -7.5,4.0 C -10.260000,4.0 -12.5,1.7600000 -12.5,-1.0 C -12.5,-3.7600000 -10.260000,-6.0 -7.5,-6.0 C -4.7400000,-6.0 -2.5,-3.7600000 -2.5,-1.0 z "
+ style="fill-rule:evenodd;stroke:#000000;stroke-width:1.0pt"
+ transform="scale(0.8) translate(7.4, 1)" />
+ </marker>
+ <marker
+ inkscape:stockid="SquareL"
+ orient="auto"
+ refY="0.0"
+ refX="0.0"
+ id="SquareL"
+ style="overflow:visible">
+ <path
+ id="path8474"
+ d="M -5.0,-5.0 L -5.0,5.0 L 5.0,5.0 L 5.0,-5.0 L -5.0,-5.0 z "
+ style="fill-rule:evenodd;stroke:#000000;stroke-width:1.0pt"
+ transform="scale(0.8)" />
+ </marker>
+ <marker
+ inkscape:stockid="TriangleOutL"
+ orient="auto"
+ refY="0.0"
+ refX="0.0"
+ id="TriangleOutL"
+ style="overflow:visible">
+ <path
+ id="path8546"
+ d="M 5.77,0.0 L -2.88,5.0 L -2.88,-5.0 L 5.77,0.0 z "
+ style="fill-rule:evenodd;stroke:#000000;stroke-width:1.0pt"
+ transform="scale(0.8)" />
+ </marker>
+ <marker
+ inkscape:stockid="Arrow1Lstart"
+ orient="auto"
+ refY="0.0"
+ refX="0.0"
+ id="Arrow1Lstart"
+ style="overflow:visible">
+ <path
+ id="path8404"
+ d="M 0.0,0.0 L 5.0,-5.0 L -12.5,0.0 L 5.0,5.0 L 0.0,0.0 z "
+ style="fill-rule:evenodd;stroke:#000000;stroke-width:1.0pt"
+ transform="scale(0.8) translate(12.5,0)" />
+ </marker>
+ <marker
+ inkscape:stockid="Arrow1Mend"
+ orient="auto"
+ refY="0.0"
+ refX="0.0"
+ id="Arrow1Mend"
+ style="overflow:visible;">
+ <path
+ id="path8413"
+ d="M 0.0,0.0 L 5.0,-5.0 L -12.5,0.0 L 5.0,5.0 L 0.0,0.0 z "
+ style="fill-rule:evenodd;stroke:#000000;stroke-width:1.0pt;"
+ transform="scale(0.4) rotate(180) translate(10,0)" />
+ </marker>
+ <marker
+ inkscape:stockid="Arrow2Lend"
+ orient="auto"
+ refY="0.0"
+ refX="0.0"
+ id="Arrow2Lend"
+ style="overflow:visible;">
+ <path
+ id="path8425"
+ style="fill-rule:evenodd;stroke-width:0.62500000;stroke-linejoin:round;"
+ d="M 8.7185878,4.0337352 L -2.2072895,0.016013256 L 8.7185884,-4.0017078 C 6.9730900,-1.6296469 6.9831476,1.6157441 8.7185878,4.0337352 z "
+ transform="scale(1.1) rotate(180) translate(1,0)" />
+ </marker>
+ <marker
+ inkscape:stockid="Arrow1Lend"
+ orient="auto"
+ refY="0.0"
+ refX="0.0"
+ id="Arrow1Lend"
+ style="overflow:visible;">
+ <path
+ id="path8407"
+ d="M 0.0,0.0 L 5.0,-5.0 L -12.5,0.0 L 5.0,5.0 L 0.0,0.0 z "
+ style="fill-rule:evenodd;stroke:#000000;stroke-width:1.0pt;"
+ transform="scale(0.8) rotate(180) translate(12.5,0)" />
+ </marker>
+ <filter
+ id="filter_2"
+ color-interpolation-filters="sRGB">
+ <feGaussianBlur
+ stdDeviation="2"
+ id="feGaussianBlur15" />
+ </filter>
+ <filter
+ id="filter_2-3"
+ color-interpolation-filters="sRGB"
+ x="-0.086178862"
+ y="-0.086178862"
+ width="1.1723577"
+ height="1.1723577">
+ <feGaussianBlur
+ stdDeviation="2"
+ id="feGaussianBlur15-1" />
+ </filter>
+ <filter
+ id="filter_2-0"
+ color-interpolation-filters="sRGB">
+ <feGaussianBlur
+ stdDeviation="2"
+ id="feGaussianBlur15-7" />
+ </filter>
+ <filter
+ id="filter_2-0-8"
+ color-interpolation-filters="sRGB">
+ <feGaussianBlur
+ stdDeviation="2"
+ id="feGaussianBlur15-7-7" />
+ </filter>
+ <filter
+ id="filter_2-3-9"
+ color-interpolation-filters="sRGB"
+ x="-0.086178862"
+ y="-0.086178862"
+ width="1.1723577"
+ height="1.1723577">
+ <feGaussianBlur
+ stdDeviation="2"
+ id="feGaussianBlur15-1-6" />
+ </filter>
+ <filter
+ id="filter_2-3-6"
+ color-interpolation-filters="sRGB"
+ x="-0.086178862"
+ y="-0.086178862"
+ width="1.1723577"
+ height="1.1723577">
+ <feGaussianBlur
+ stdDeviation="2"
+ id="feGaussianBlur15-1-63" />
+ </filter>
+ <filter
+ id="filter_2-3-91"
+ color-interpolation-filters="sRGB"
+ x="-0.086178862"
+ y="-0.086178862"
+ width="1.1723577"
+ height="1.1723577">
+ <feGaussianBlur
+ stdDeviation="2"
+ id="feGaussianBlur15-1-3" />
+ </filter>
+ <marker
+ inkscape:stockid="Arrow1Lend"
+ orient="auto"
+ refY="0"
+ refX="0"
+ id="Arrow1Lend-5"
+ style="overflow:visible">
+ <path
+ inkscape:connector-curvature="0"
+ id="path8407-3"
+ d="M 0,0 5,-5 -12.5,0 5,5 0,0 z"
+ style="fill-rule:evenodd;stroke:#000000;stroke-width:1pt"
+ transform="matrix(-0.8,0,0,-0.8,-10,0)" />
+ </marker>
+ <marker
+ inkscape:stockid="Arrow1Lend"
+ orient="auto"
+ refY="0"
+ refX="0"
+ id="Arrow1Lend-6"
+ style="overflow:visible">
+ <path
+ inkscape:connector-curvature="0"
+ id="path8407-0"
+ d="M 0,0 5,-5 -12.5,0 5,5 0,0 z"
+ style="fill-rule:evenodd;stroke:#000000;stroke-width:1pt"
+ transform="matrix(-0.8,0,0,-0.8,-10,0)" />
+ </marker>
+ <marker
+ inkscape:stockid="Arrow1Lstart"
+ orient="auto"
+ refY="0"
+ refX="0"
+ id="Arrow1Lstart-7"
+ style="overflow:visible">
+ <path
+ inkscape:connector-curvature="0"
+ id="path8404-0"
+ d="M 0,0 5,-5 -12.5,0 5,5 0,0 z"
+ style="fill-rule:evenodd;stroke:#000000;stroke-width:1pt"
+ transform="matrix(0.8,0,0,0.8,10,0)" />
+ </marker>
+ <marker
+ inkscape:stockid="Arrow1Lend"
+ orient="auto"
+ refY="0"
+ refX="0"
+ id="Arrow1Lend-51"
+ style="overflow:visible">
+ <path
+ inkscape:connector-curvature="0"
+ id="path8407-1"
+ d="M 0,0 5,-5 -12.5,0 5,5 0,0 z"
+ style="fill-rule:evenodd;stroke:#000000;stroke-width:1pt"
+ transform="matrix(-0.8,0,0,-0.8,-10,0)" />
+ </marker>
+ <marker
+ inkscape:stockid="Arrow1Lend"
+ orient="auto"
+ refY="0"
+ refX="0"
+ id="Arrow1Lend-3"
+ style="overflow:visible">
+ <path
+ inkscape:connector-curvature="0"
+ id="path8407-6"
+ d="M 0,0 5,-5 -12.5,0 5,5 0,0 z"
+ style="fill-rule:evenodd;stroke:#000000;stroke-width:1pt"
+ transform="matrix(-0.8,0,0,-0.8,-10,0)" />
+ </marker>
+ <marker
+ inkscape:stockid="Arrow1Lend"
+ orient="auto"
+ refY="0"
+ refX="0"
+ id="Arrow1Lend-62"
+ style="overflow:visible">
+ <path
+ inkscape:connector-curvature="0"
+ id="path8407-9"
+ d="M 0,0 5,-5 -12.5,0 5,5 0,0 z"
+ style="fill-rule:evenodd;stroke:#000000;stroke-width:1pt"
+ transform="matrix(-0.8,0,0,-0.8,-10,0)" />
+ </marker>
+ <marker
+ inkscape:stockid="Arrow1Lend"
+ orient="auto"
+ refY="0"
+ refX="0"
+ id="Arrow1Lend-2"
+ style="overflow:visible">
+ <path
+ inkscape:connector-curvature="0"
+ id="path8407-7"
+ d="M 0,0 5,-5 -12.5,0 5,5 0,0 z"
+ style="fill-rule:evenodd;stroke:#000000;stroke-width:1pt"
+ transform="matrix(-0.8,0,0,-0.8,-10,0)" />
+ </marker>
+ <filter
+ id="filter_2-3-91-3"
+ color-interpolation-filters="sRGB"
+ x="-0.086178862"
+ y="-0.086178862"
+ width="1.1723577"
+ height="1.1723577">
+ <feGaussianBlur
+ stdDeviation="2"
+ id="feGaussianBlur15-1-3-6" />
+ </filter>
+ </defs>
+ <sodipodi:namedview
+ id="base"
+ pagecolor="#ffffff"
+ bordercolor="#666666"
+ borderopacity="1.0"
+ inkscape:pageopacity="0.0"
+ inkscape:pageshadow="2"
+ inkscape:zoom="1"
+ inkscape:cx="385.5"
+ inkscape:cy="234"
+ inkscape:document-units="px"
+ inkscape:current-layer="layer1"
+ showgrid="false"
+ inkscape:window-width="1920"
+ inkscape:window-height="1017"
+ inkscape:window-x="-8"
+ inkscape:window-y="-8"
+ inkscape:window-maximized="1"
+ inkscape:snap-nodes="false"
+ inkscape:showpageshadow="2"
+ inkscape:pagecheckerboard="0"
+ inkscape:deskcolor="#d1d1d1">
+ <inkscape:grid
+ type="xygrid"
+ id="grid13454" />
+ </sodipodi:namedview>
+ <metadata
+ id="metadata13242">
+ <rdf:RDF>
+ <cc:Work
+ rdf:about="">
+ <dc:format>image/svg+xml</dc:format>
+ <dc:type
+ rdf:resource="http://purl.org/dc/dcmitype/StillImage" />
+ <dc:title />
+ </cc:Work>
+ </rdf:RDF>
+ </metadata>
+ <g
+ id="layer1"
+ inkscape:label="Layer 1"
+ inkscape:groupmode="layer">
+ <g
+ style="font-size:12px;fill:none;stroke-linecap:square;stroke-miterlimit:3;overflow:visible"
+ id="shape1-1-2-0"
+ v:mID="1"
+ v:groupContext="shape"
+ transform="matrix(2.1604167,0,0,1.5671361,323.2187,-540.25927)">
+ <title
+ id="title22-7-6">Square</title>
+ <desc
+ id="desc24-7-4">Atomic Queue #1</desc>
+ <v:userDefs>
+ <v:ud
+ v:nameU="visVersion"
+ v:val="VT0(15):26" />
+ </v:userDefs>
+ <v:textBlock
+ v:margins="rect(4,4,4,4)" />
+ <v:textRect
+ cx="30.75"
+ cy="581.25"
+ width="61.5"
+ height="61.5" />
+ <g
+ id="shadow1-2-9-0"
+ v:groupContext="shadow"
+ v:shadowOffsetX="0.345598"
+ v:shadowOffsetY="-1.97279"
+ v:shadowType="1"
+ transform="translate(0.345598,1.97279)"
+ class="st1"
+ style="visibility:visible">
+ <rect
+ x="0"
+ y="550.5"
+ width="61.5"
+ height="61.5"
+ class="st2"
+ id="rect27-8-9"
+ style="fill:#5b9bd5;fill-opacity:0.22000002;stroke:#5b9bd5;stroke-opacity:0.22000002;filter:url(#filter_2-3-9)" />
+ </g>
+ <g
+ id="g13515-4">
+ <g
+ id="g13534-5">
+ <rect
+ x="0"
+ y="550.5"
+ width="61.5"
+ height="61.5"
+ class="st3"
+ id="rect29-1-4"
+ style="fill:#5b9bd5;stroke:#c7c8c8;stroke-width:0.25" />
+ </g>
+ </g>
+ </g>
+ <g
+ style="font-size:12px;fill:none;stroke-linecap:square;stroke-miterlimit:3;overflow:visible"
+ id="shape1-1-2-4"
+ v:mID="1"
+ v:groupContext="shape"
+ transform="matrix(2.1604167,0,0,1.165886,88.874699,-447.8809)">
+ <title
+ id="title22-7-5">Square</title>
+ <desc
+ id="desc24-7-8">Atomic Queue #1</desc>
+ <v:userDefs>
+ <v:ud
+ v:nameU="visVersion"
+ v:val="VT0(15):26" />
+ </v:userDefs>
+ <v:textBlock
+ v:margins="rect(4,4,4,4)" />
+ <v:textRect
+ cx="30.75"
+ cy="581.25"
+ width="61.5"
+ height="61.5" />
+ <g
+ id="shadow1-2-9-5"
+ v:groupContext="shadow"
+ v:shadowOffsetX="0.345598"
+ v:shadowOffsetY="-1.97279"
+ v:shadowType="1"
+ transform="translate(0.345598,1.97279)"
+ class="st1"
+ style="visibility:visible">
+ <rect
+ x="0"
+ y="550.5"
+ width="61.5"
+ height="61.5"
+ class="st2"
+ id="rect27-8-7"
+ style="fill:#5b9bd5;fill-opacity:0.22000002;stroke:#5b9bd5;stroke-opacity:0.22000002;filter:url(#filter_2-3-91)" />
+ </g>
+ <g
+ id="g13515-33">
+ <g
+ id="g13534-8">
+ <rect
+ x="0"
+ y="550.5"
+ width="61.5"
+ height="61.5"
+ class="st3"
+ id="rect29-1-95"
+ style="fill:#5b9bd5;stroke:#c7c8c8;stroke-width:0.25" />
+ </g>
+ </g>
+ </g>
+ <g
+ style="font-size:12px;fill:none;stroke-linecap:square;stroke-miterlimit:3;overflow:visible"
+ id="shape1-1-2-9"
+ v:mID="1"
+ v:groupContext="shape"
+ transform="matrix(2.1604167,0,0,1.5671361,88.874699,-538.24651)">
+ <title
+ id="title22-7-9">Square</title>
+ <desc
+ id="desc24-7-5">Atomic Queue #1</desc>
+ <v:userDefs>
+ <v:ud
+ v:nameU="visVersion"
+ v:val="VT0(15):26" />
+ </v:userDefs>
+ <v:textBlock
+ v:margins="rect(4,4,4,4)" />
+ <v:textRect
+ cx="30.75"
+ cy="581.25"
+ width="61.5"
+ height="61.5" />
+ <g
+ id="shadow1-2-9-2"
+ v:groupContext="shadow"
+ v:shadowOffsetX="0.345598"
+ v:shadowOffsetY="-1.97279"
+ v:shadowType="1"
+ transform="translate(0.345598,1.97279)"
+ class="st1"
+ style="visibility:visible">
+ <rect
+ x="0"
+ y="550.5"
+ width="61.5"
+ height="61.5"
+ class="st2"
+ id="rect27-8-1"
+ style="fill:#5b9bd5;fill-opacity:0.22000002;stroke:#5b9bd5;stroke-opacity:0.22000002;filter:url(#filter_2-3-6)" />
+ </g>
+ <g
+ id="g13515-3">
+ <g
+ id="g13534-0">
+ <rect
+ x="0"
+ y="550.5"
+ width="61.5"
+ height="61.5"
+ class="st3"
+ id="rect29-1-9"
+ style="fill:#5b9bd5;stroke:#c7c8c8;stroke-width:0.25" />
+ </g>
+ </g>
+ </g>
+ <path
+ style="fill:none;stroke:#000000;stroke-width:0.743466px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1;marker-mid:none;marker-end:url(#Arrow1Lend)"
+ d="M 220.66064,98.57125 H 321.88592"
+ id="path17209-8"
+ inkscape:connector-type="orthogonal"
+ inkscape:connector-curvature="0" />
+ <g
+ style="font-size:12px;fill:none;stroke-linecap:square;stroke-miterlimit:3;overflow:visible"
+ id="shape1-1-2"
+ v:mID="1"
+ v:groupContext="shape"
+ transform="matrix(2.1604167,0,0,1.5671361,322.24227,-811.89589)">
+ <title
+ id="title22-7">Square</title>
+ <desc
+ id="desc24-7">Atomic Queue #1</desc>
+ <v:userDefs>
+ <v:ud
+ v:nameU="visVersion"
+ v:val="VT0(15):26" />
+ </v:userDefs>
+ <v:textBlock
+ v:margins="rect(4,4,4,4)" />
+ <v:textRect
+ cx="30.75"
+ cy="581.25"
+ width="61.5"
+ height="61.5" />
+ <g
+ id="shadow1-2-9"
+ v:groupContext="shadow"
+ v:shadowOffsetX="0.345598"
+ v:shadowOffsetY="-1.97279"
+ v:shadowType="1"
+ transform="translate(0.345598,1.97279)"
+ class="st1"
+ style="visibility:visible">
+ <rect
+ x="0"
+ y="550.5"
+ width="61.5"
+ height="61.5"
+ class="st2"
+ id="rect27-8"
+ style="fill:#5b9bd5;fill-opacity:0.22000002;stroke:#5b9bd5;stroke-opacity:0.22000002;filter:url(#filter_2-3)" />
+ </g>
+ <g
+ id="g13515">
+ <g
+ id="g13534">
+ <rect
+ x="0"
+ y="550.5"
+ width="61.5"
+ height="61.5"
+ class="st3"
+ id="rect29-1"
+ style="fill:#5b9bd5;stroke:#c7c8c8;stroke-width:0.25" />
+ </g>
+ </g>
+ </g>
+ <g
+ id="g13518"
+ transform="matrix(0.73232502,0,0,0.75477602,25.29268,348.89752)">
+ <g
+ id="g13526">
+ <flowRoot
+ xml:space="preserve"
+ id="flowRoot13464-9"
+ style="font-style:normal;font-weight:normal;line-height:0.01%;font-family:sans-serif;text-align:center;letter-spacing:0px;word-spacing:0px;text-anchor:middle;fill:#000000;fill-opacity:1;stroke:none"
+ transform="translate(-12.00521,-129.65179)"><flowRegion
+ id="flowRegion13466-1"
+ style="font-family:sans-serif"><rect
+ id="rect13468-2"
+ width="195.99997"
+ height="112.00001"
+ x="273.33334"
+ y="175.33333"
+ style="text-align:center;text-anchor:middle" /></flowRegion><flowPara
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:20px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:center;writing-mode:lr-tb;text-anchor:middle"
+ id="flowPara13511"> </flowPara></flowRoot>
+ </g>
+ </g>
+ <path
+ style="fill:none;stroke:#000000;stroke-width:0.751455;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:4;stroke-dasharray:none;stroke-opacity:1;marker-start:url(#Arrow1Lstart);marker-end:none"
+ d="m 176.26096,124.64833 v 69.24854"
+ id="path17209-3"
+ inkscape:connector-type="orthogonal"
+ inkscape:connector-curvature="0" />
+ <path
+ transform="matrix(0.73232502,0,0,0.75477602,-4.325033,50.642983)"
+ sodipodi:type="arc"
+ style="fill:#539de6;fill-opacity:1;stroke:#0000ea;stroke-width:1;stroke-linecap:square;stroke-miterlimit:4;stroke-opacity:1;stroke-dasharray:none"
+ id="path29161-3"
+ sodipodi:cx="371"
+ sodipodi:cy="64.5"
+ sodipodi:rx="17"
+ sodipodi:ry="15.5"
+ d="m 388,64.5 a 17,15.5 0 1 1 -34,0 17,15.5 0 1 1 34,0 z" />
+ <path
+ style="fill:none;stroke:#000000;stroke-width:0.743466px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1;marker-mid:none;marker-end:url(#Arrow1Lend)"
+ d="M 322.61264,375 H 221.38736"
+ id="path17209-8-0"
+ inkscape:connector-type="orthogonal"
+ inkscape:connector-curvature="0" />
+ <path
+ transform="matrix(0.73232502,0,0,0.75477602,0.0689171,324.80833)"
+ sodipodi:type="arc"
+ style="fill:#539de6;fill-opacity:1;stroke:#0000ea;stroke-width:1;stroke-linecap:square;stroke-miterlimit:4;stroke-opacity:1;stroke-dasharray:none"
+ id="path29161-1"
+ sodipodi:cx="371"
+ sodipodi:cy="64.5"
+ sodipodi:rx="17"
+ sodipodi:ry="15.5"
+ d="m 388,64.5 a 17,15.5 0 1 1 -34,0 17,15.5 0 1 1 34,0 z" />
+ <path
+ style="fill:none;stroke:#000000;stroke-width:0.629082px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1;marker-end:url(#Arrow1Lend)"
+ d="M 155,324.19955 V 264.82863"
+ id="path17211-7-1"
+ inkscape:connector-type="orthogonal"
+ inkscape:connector-curvature="0" />
+ <path
+ transform="matrix(0.73232502,0,0,0.75477602,-116.37076,245.97088)"
+ sodipodi:type="arc"
+ style="fill:#539de6;fill-opacity:1;stroke:#0000ea;stroke-width:1;stroke-linecap:square;stroke-miterlimit:4;stroke-opacity:1;stroke-dasharray:none"
+ id="path29161-8"
+ sodipodi:cx="371"
+ sodipodi:cy="64.5"
+ sodipodi:rx="17"
+ sodipodi:ry="15.5"
+ d="m 388,64.5 a 17,15.5 0 1 1 -34,0 17,15.5 0 1 1 34,0 z" />
+ <path
+ style="fill:none;stroke:#000000;stroke-width:0.75059;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:4;stroke-dasharray:none;stroke-opacity:1;marker-start:url(#Arrow1Lstart-7);marker-end:none"
+ d="m 126.26097,124.99178 v 69.24941"
+ id="path17209-3-0"
+ inkscape:connector-type="orthogonal"
+ inkscape:connector-curvature="0" />
+ <path
+ sodipodi:type="arc"
+ style="fill:#539de6;fill-opacity:1;stroke:#0000ea;stroke-width:1;stroke-linecap:square;stroke-miterlimit:4;stroke-opacity:1;stroke-dasharray:none"
+ id="path29161"
+ sodipodi:cx="371"
+ sodipodi:cy="64.5"
+ sodipodi:rx="17"
+ sodipodi:ry="15.5"
+ d="m 388,64.5 a 17,15.5 0 1 1 -34,0 17,15.5 0 1 1 34,0 z"
+ transform="matrix(0.73232502,0,0,0.75477602,-146.16394,110.68276)" />
+ <path
+ sodipodi:type="arc"
+ style="fill:#539de6;fill-opacity:1;stroke:#0000ea;stroke-width:1;stroke-linecap:square;stroke-miterlimit:4;stroke-opacity:1;stroke-dasharray:none"
+ id="path29161-2"
+ sodipodi:cx="371"
+ sodipodi:cy="64.5"
+ sodipodi:rx="17"
+ sodipodi:ry="15.5"
+ d="m 388,64.5 a 17,15.5 0 1 1 -34,0 17,15.5 0 1 1 34,0 z"
+ transform="matrix(0.73232502,0,0,0.75477602,-95.40136,110.71627)" />
+ <g
+ id="g29167-4"
+ transform="matrix(0.73232502,0,0,0.75477602,-95.31662,112.00756)">
+ <text
+ id="text29163-9"
+ y="70"
+ x="365"
+ style="font-style:normal;font-weight:normal;line-height:0%;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none"
+ xml:space="preserve"><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="70"
+ x="365"
+ id="tspan29165-9"
+ sodipodi:role="line">1</tspan></text>
+ </g>
+ <g
+ id="g29167-9"
+ transform="matrix(0.73232502,0,0,0.75477602,-4.9726112,50.689051)">
+ <text
+ id="text29163-3"
+ y="70"
+ x="365"
+ style="font-style:normal;font-weight:normal;line-height:0%;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none"
+ xml:space="preserve"><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="70"
+ x="365"
+ id="tspan29165-3"
+ sodipodi:role="line">2</tspan></text>
+ </g>
+ <path
+ style="fill:none;stroke:#000000;stroke-width:1.04033px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1;marker-end:url(#Arrow1Lend)"
+ d="M 388.20118,147.93341 V 321.89308"
+ id="path17211-7"
+ inkscape:connector-type="orthogonal"
+ inkscape:connector-curvature="0" />
+ <path
+ transform="matrix(0.73232502,0,0,0.75477602,116.5086,136.68745)"
+ sodipodi:type="arc"
+ style="fill:#539de6;fill-opacity:1;stroke:#0000ea;stroke-width:1;stroke-linecap:square;stroke-miterlimit:4;stroke-opacity:1;stroke-dasharray:none"
+ id="path29161-6"
+ sodipodi:cx="371"
+ sodipodi:cy="64.5"
+ sodipodi:rx="17"
+ sodipodi:ry="15.5"
+ d="m 388,64.5 a 17,15.5 0 1 1 -34,0 17,15.5 0 1 1 34,0 z" />
+ <g
+ id="g29167-46"
+ transform="matrix(0.73232502,0,0,0.75477602,116.59334,137.48829)">
+ <text
+ id="text29163-1"
+ y="70"
+ x="365"
+ style="font-style:normal;font-weight:normal;line-height:0%;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none"
+ xml:space="preserve"><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="70"
+ x="365"
+ id="tspan29165-5"
+ sodipodi:role="line">3</tspan></text>
+ </g>
+ <g
+ id="g29167-6"
+ transform="matrix(0.73232502,0,0,0.75477602,0.1536639,325.60918)">
+ <text
+ id="text29163-17"
+ y="70"
+ x="365"
+ style="font-style:normal;font-weight:normal;line-height:0%;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none"
+ xml:space="preserve"><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="70"
+ x="365"
+ id="tspan29165-2"
+ sodipodi:role="line">4</tspan></text>
+ </g>
+ <g
+ id="g29167"
+ transform="matrix(0.73232502,0,0,0.75477602,-146.07919,111.10621)">
+ <text
+ id="text29163"
+ y="70"
+ x="365"
+ style="font-style:normal;font-weight:normal;line-height:0%;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none"
+ xml:space="preserve"><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="70"
+ x="365"
+ id="tspan29165"
+ sodipodi:role="line">6</tspan></text>
+ </g>
+ <g
+ id="g29167-4-3"
+ transform="matrix(0.73232502,0,0,0.75477602,-117.60784,180.11779)">
+ <text
+ id="text29163-9-6"
+ y="70"
+ x="321.30356"
+ style="font-style:normal;font-weight:normal;line-height:0%;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none"
+ xml:space="preserve"><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:24.2109px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="70"
+ x="321.30356"
+ id="tspan29165-9-7"
+ sodipodi:role="line">Eventdev</tspan></text>
+ </g>
+ <g
+ id="g29167-4-3-5"
+ transform="matrix(0.73232502,0,0,0.75477602,55.34956,26.97821)">
+ <text
+ id="text29163-9-6-3"
+ y="70"
+ x="454.74152"
+ style="font-style:normal;font-weight:normal;line-height:0%;font-family:sans-serif;text-align:center;letter-spacing:0px;word-spacing:0px;text-anchor:middle;fill:#000000;fill-opacity:1;stroke:none"
+ xml:space="preserve"><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:21.5208px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:center;writing-mode:lr-tb;text-anchor:middle"
+ y="70"
+ x="454.74152"
+ id="tspan29165-9-7-5"
+ sodipodi:role="line">A<tspan
+ style="line-height:100%;font-family:sans-serif"
+ id="tspan3374">tomic Stage</tspan></tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:21.5208px;line-height:100%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:center;writing-mode:lr-tb;text-anchor:middle"
+ y="94.210899"
+ x="454.74152"
+ sodipodi:role="line"
+ id="tspan3320">+</tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:21.5208px;line-height:100%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:center;writing-mode:lr-tb;text-anchor:middle"
+ y="115.7317"
+ x="454.74152"
+ sodipodi:role="line"
+ id="tspan3322">Enqueue to</tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:21.5208px;line-height:100%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:center;writing-mode:lr-tb;text-anchor:middle"
+ y="137.2525"
+ x="454.74152"
+ sodipodi:role="line"
+ id="tspan3324">DMA Device</tspan></text>
+ </g>
+ <g
+ id="g29167-2"
+ transform="matrix(0.73232502,0,0,0.75477602,-116.28602,248.01695)">
+ <text
+ id="text29163-92"
+ y="70"
+ x="365"
+ style="font-style:normal;font-weight:normal;line-height:0%;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none"
+ xml:space="preserve"><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="70"
+ x="365"
+ id="tspan29165-8"
+ sodipodi:role="line">5</tspan></text>
+ </g>
+ <flowRoot
+ xml:space="preserve"
+ id="flowRoot3376"
+ style="font-style:normal;font-weight:normal;line-height:0.01%;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none;stroke-width:1px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1"><flowRegion
+ id="flowRegion3378"
+ style="font-family:sans-serif"><rect
+ id="rect3380"
+ width="100"
+ height="37"
+ x="109"
+ y="259" /></flowRegion><flowPara
+ id="flowPara3382"
+ style="font-size:18px;line-height:1.25;font-family:sans-serif"> </flowPara></flowRoot>
+ <g
+ id="g29167-4-3-1"
+ transform="matrix(0.73232502,0,0,0.75477602,109.34956,323.97821)">
+ <text
+ id="text29163-9-6-8"
+ y="70"
+ x="321.30356"
+ style="font-style:normal;font-weight:normal;line-height:0%;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none"
+ xml:space="preserve"><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:24.2109px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="70"
+ x="321.30356"
+ id="tspan29165-9-7-7"
+ sodipodi:role="line">DMA Device</tspan></text>
+ </g>
+ <g
+ id="g29167-4-3-1-9"
+ transform="matrix(0.73232502,0,0,0.75477602,-114.48565,314.20704)">
+ <text
+ id="text29163-9-6-8-2"
+ y="70"
+ x="368.01718"
+ style="font-style:normal;font-weight:normal;line-height:0%;font-family:sans-serif;text-align:center;letter-spacing:0px;word-spacing:0px;text-anchor:middle;fill:#000000;fill-opacity:1;stroke:none"
+ xml:space="preserve"><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:24.2109px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:center;writing-mode:lr-tb;text-anchor:middle"
+ y="70"
+ x="368.01718"
+ id="tspan29165-9-7-7-0"
+ sodipodi:role="line">DMA</tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:24.2109px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:center;writing-mode:lr-tb;text-anchor:middle"
+ y="100.26363"
+ x="368.01718"
+ sodipodi:role="line"
+ id="tspan3488">Adapter</tspan></text>
+ </g>
+ <g
+ id="g29167-4-3-1-9-2"
+ transform="matrix(0.73232502,0,0,0.75477602,250.96804,192.62529)">
+ <text
+ id="text29163-9-6-8-2-3"
+ y="-188.35481"
+ x="318.61978"
+ style="font-style:normal;font-weight:normal;line-height:0%;font-family:sans-serif;text-align:center;letter-spacing:0px;word-spacing:0px;text-anchor:middle;fill:#000000;fill-opacity:1;stroke:none"
+ xml:space="preserve"><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:21.5208px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="-188.35481"
+ x="318.61978"
+ sodipodi:role="line"
+ id="tspan3543">1. Application dequeues</tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:21.5208px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="-161.45381"
+ x="318.61978"
+ sodipodi:role="line"
+ id="tspan3196"> events from the previous</tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:21.5208px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="-134.55281"
+ x="318.61978"
+ sodipodi:role="line"
+ id="tspan3232"> stage</tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:21.5208px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="-107.65182"
+ x="318.61978"
+ sodipodi:role="line"
+ id="tspan3519"> </tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:21.5208px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="-80.750816"
+ x="318.61978"
+ sodipodi:role="line"
+ id="tspan3551">2. Application prepares the</tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:21.5208px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="-53.849815"
+ x="318.61978"
+ sodipodi:role="line"
+ id="tspan3203"> DMA operations.</tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:21.5208px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="-26.948814"
+ x="318.61978"
+ sodipodi:role="line"
+ id="tspan3523"> </tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:21.5208px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="-0.0478158"
+ x="318.61978"
+ sodipodi:role="line"
+ id="tspan3541">3. DMA operations are</tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:21.5208px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="26.853184"
+ x="318.61978"
+ sodipodi:role="line"
+ id="tspan3207"> submitted to dmadev</tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:21.5208px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="53.754185"
+ x="318.61978"
+ sodipodi:role="line"
+ id="tspan3209"> by application.</tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:21.5208px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="80.655182"
+ x="318.61978"
+ sodipodi:role="line"
+ id="tspan3527"> </tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:21.5208px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="107.55618"
+ x="318.61978"
+ sodipodi:role="line"
+ id="tspan3547">4. DMA adapter dequeues</tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:21.5208px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="134.45718"
+ x="318.61978"
+ sodipodi:role="line"
+ id="tspan3216"> DMA completions from</tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:21.5208px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="161.35818"
+ x="318.61978"
+ sodipodi:role="line"
+ id="tspan3218"> DMA device.</tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:21.5208px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="188.25919"
+ x="318.61978"
+ sodipodi:role="line"
+ id="tspan3531"> </tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:21.5208px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="215.16019"
+ x="318.61978"
+ sodipodi:role="line"
+ id="tspan3549">5. DMA adapter enqueues</tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:21.5208px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="242.06119"
+ x="318.61978"
+ sodipodi:role="line"
+ id="tspan3222"> events to the eventdev.</tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:21.5208px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="268.96219"
+ x="318.61978"
+ sodipodi:role="line"
+ id="tspan3535"> </tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:21.5208px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="295.86319"
+ x="318.61978"
+ sodipodi:role="line"
+ id="tspan3537">6. Application dequeues from</tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:21.5208px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="322.76419"
+ x="318.61978"
+ sodipodi:role="line"
+ id="tspan3224"> eventdev and prepare for</tspan><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:21.5208px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="349.66519"
+ x="318.61978"
+ sodipodi:role="line"
+ id="tspan3226"> further processing</tspan></text>
+ </g>
+ <g
+ style="font-size:12px;fill:none;stroke-linecap:square;stroke-miterlimit:3;overflow:visible"
+ id="shape1-1-2-4-7"
+ v:mID="1"
+ v:groupContext="shape"
+ transform="matrix(2.1604167,0,0,1.165886,90.820551,-587.97129)">
+ <title
+ id="title22-7-5-5">Square</title>
+ <desc
+ id="desc24-7-8-3">Atomic Queue #1</desc>
+ <v:userDefs>
+ <v:ud
+ v:nameU="visVersion"
+ v:val="VT0(15):26" />
+ </v:userDefs>
+ <v:textBlock
+ v:margins="rect(4,4,4,4)" />
+ <v:textRect
+ cx="30.75"
+ cy="581.25"
+ width="61.5"
+ height="61.5" />
+ <g
+ id="shadow1-2-9-5-5"
+ v:groupContext="shadow"
+ v:shadowOffsetX="0.345598"
+ v:shadowOffsetY="-1.97279"
+ v:shadowType="1"
+ transform="translate(0.345598,1.97279)"
+ class="st1"
+ style="visibility:visible">
+ <rect
+ x="0"
+ y="550.5"
+ width="61.5"
+ height="61.5"
+ class="st2"
+ id="rect27-8-7-6"
+ style="fill:#5b9bd5;fill-opacity:0.22000002;stroke:#5b9bd5;stroke-opacity:0.22000002;filter:url(#filter_2-3-91-3)" />
+ </g>
+ <g
+ id="g13515-33-2">
+ <g
+ id="g13534-8-9">
+ <rect
+ x="0"
+ y="550.5"
+ width="61.5"
+ height="61.5"
+ class="st3"
+ id="rect29-1-95-1"
+ style="fill:#5b9bd5;stroke:#c7c8c8;stroke-width:0.25" />
+ </g>
+ </g>
+ </g>
+ <g
+ id="g29167-4-3-2"
+ transform="matrix(0.73232502,0,0,0.75477602,-125.66199,44.027402)">
+ <text
+ id="text29163-9-6-7"
+ y="70"
+ x="321.30356"
+ style="font-style:normal;font-weight:normal;line-height:0%;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none"
+ xml:space="preserve"><tspan
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:24.2109px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start"
+ y="70"
+ x="321.30356"
+ id="tspan29165-9-7-0"
+ sodipodi:role="line">Application</tspan></text>
+ </g>
+ </g>
+</svg>
diff --git a/doc/guides/prog_guide/index.rst b/doc/guides/prog_guide/index.rst
index 52a6d9e7aa..beaa4b8869 100644
--- a/doc/guides/prog_guide/index.rst
+++ b/doc/guides/prog_guide/index.rst
@@ -60,6 +60,7 @@ Programmer's Guide
event_ethernet_tx_adapter
event_timer_adapter
event_crypto_adapter
+ event_dma_adapter
qos_framework
power_man
packet_classif_access_ctrl
diff --git a/doc/guides/rel_notes/release_23_11.rst b/doc/guides/rel_notes/release_23_11.rst
index b34ddc0860..b66c364e21 100644
--- a/doc/guides/rel_notes/release_23_11.rst
+++ b/doc/guides/rel_notes/release_23_11.rst
@@ -84,6 +84,12 @@ New Features
for creating Rx adapter instance for the applications desire to
control both the event port allocation and event buffer size.
+* **Added event DMA adapter library.**
+
+ * Added the Event DMA Adapter Library. This library extends the event-based
+ model by introducing APIs that allow applications to enqueue/dequeue DMA
+ operations to/from dmadev as events scheduled by an event device.
+
* **Updated Marvell cnxk eventdev driver.**
* Added support for ``remaining_ticks_get`` timer adapter PMD callback
diff --git a/lib/eventdev/eventdev_pmd.h b/lib/eventdev/eventdev_pmd.h
index f62f42e140..f7227c0bfd 100644
--- a/lib/eventdev/eventdev_pmd.h
+++ b/lib/eventdev/eventdev_pmd.h
@@ -178,8 +178,12 @@ struct rte_eventdev {
event_tx_adapter_enqueue_t txa_enqueue;
/**< Pointer to PMD eth Tx adapter enqueue function. */
event_crypto_adapter_enqueue_t ca_enqueue;
+ /**< Pointer to PMD crypto adapter enqueue function. */
- uint64_t reserved_64s[4]; /**< Reserved for future fields */
+ event_dma_adapter_enqueue_t dma_enqueue;
+ /**< Pointer to PMD DMA adapter enqueue function. */
+
+ uint64_t reserved_64s[3]; /**< Reserved for future fields */
void *reserved_ptrs[3]; /**< Reserved for future fields */
} __rte_cache_aligned;
@@ -1320,6 +1324,156 @@ typedef int (*eventdev_eth_tx_adapter_queue_stop)
#define eventdev_stop_flush_t rte_eventdev_stop_flush_t
+/**
+ * Retrieve the event device's DMA adapter capabilities for the
+ * specified DMA device
+ *
+ * @param dev
+ * Event device pointer
+ *
+ * @param dma_dev_id
+ * DMA device identifier
+ *
+ * @param[out] caps
+ * A pointer to memory filled with event adapter capabilities.
+ * It is expected to be pre-allocated & initialized by caller.
+ *
+ * @return
+ * - 0: Success, driver provides event adapter capabilities for the
+ * dmadev.
+ * - <0: Error code returned by the driver function.
+ *
+ */
+typedef int (*eventdev_dma_adapter_caps_get_t)(const struct rte_eventdev *dev,
+ const int16_t dma_dev_id, uint32_t *caps);
+
+/**
+ * Add DMA vchan queue to event device. This callback is invoked if
+ * the caps returned from rte_event_dma_adapter_caps_get(, dmadev_id)
+ * has RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_* set.
+ *
+ * @param dev
+ * Event device pointer
+ *
+ * @param dma_dev_id
+ * DMA device identifier
+ *
+ * @param vchan_id
+ * dmadev vchan queue identifier.
+ *
+ * @param event
+ * Event information required for binding dmadev vchan to event queue.
+ * This structure will have a valid value for only those HW PMDs supporting
+ * @see RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_VCHAN_EV_BIND capability.
+ *
+ * @return
+ * - 0: Success, dmadev vchan added successfully.
+ * - <0: Error code returned by the driver function.
+ *
+ */
+typedef int (*eventdev_dma_adapter_vchan_add_t)(const struct rte_eventdev *dev,
+ const int16_t dma_dev_id,
+ uint16_t vchan_id,
+ const struct rte_event *event);
+
+/**
+ * Delete DMA vhcan to event device. This callback is invoked if
+ * the caps returned from rte_event_dma_adapter_caps_get(, dmadev_id)
+ * has RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_* set.
+ *
+ * @param dev
+ * Event device pointer
+ *
+ * @param dma_dev_id
+ * DMA device identifier
+ *
+ * @param vchan_id
+ * dmadev vchan identifier.
+ *
+ * @return
+ * - 0: Success, dmadev vchan deleted successfully.
+ * - <0: Error code returned by the driver function.
+ *
+ */
+typedef int (*eventdev_dma_adapter_vchan_del_t)(const struct rte_eventdev *dev,
+ const int16_t dma_dev_id,
+ uint16_t vchan_id);
+
+/**
+ * Start DMA adapter. This callback is invoked if
+ * the caps returned from rte_event_dma_adapter_caps_get(.., dmadev_id)
+ * has RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_* set and vchan for dmadev_id
+ * have been added to the event device.
+ *
+ * @param dev
+ * Event device pointer
+ *
+ * @param dma_dev_id
+ * DMA device identifier
+ *
+ * @return
+ * - 0: Success, DMA adapter started successfully.
+ * - <0: Error code returned by the driver function.
+ */
+typedef int (*eventdev_dma_adapter_start_t)(const struct rte_eventdev *dev,
+ const int16_t dma_dev_id);
+
+/**
+ * Stop DMA adapter. This callback is invoked if
+ * the caps returned from rte_event_dma_adapter_caps_get(.., dmadev_id)
+ * has RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_* set and vchan for dmadev_id
+ * have been added to the event device.
+ *
+ * @param dev
+ * Event device pointer
+ *
+ * @param dma_dev_id
+ * DMA device identifier
+ *
+ * @return
+ * - 0: Success, DMA adapter stopped successfully.
+ * - <0: Error code returned by the driver function.
+ */
+typedef int (*eventdev_dma_adapter_stop_t)(const struct rte_eventdev *dev,
+ const int16_t dma_dev_id);
+
+struct rte_event_dma_adapter_stats;
+
+/**
+ * Retrieve DMA adapter statistics.
+ *
+ * @param dev
+ * Event device pointer
+ *
+ * @param dma_dev_id
+ * DMA device identifier
+ *
+ * @param[out] stats
+ * Pointer to stats structure
+ *
+ * @return
+ * Return 0 on success.
+ */
+typedef int (*eventdev_dma_adapter_stats_get)(const struct rte_eventdev *dev,
+ const int16_t dma_dev_id,
+ struct rte_event_dma_adapter_stats *stats);
+
+/**
+ * Reset DMA adapter statistics.
+ *
+ * @param dev
+ * Event device pointer
+ *
+ * @param dma_dev_id
+ * DMA device identifier
+ *
+ * @return
+ * Return 0 on success.
+ */
+typedef int (*eventdev_dma_adapter_stats_reset)(const struct rte_eventdev *dev,
+ const int16_t dma_dev_id);
+
+
/** Event device operations function pointer table */
struct eventdev_ops {
eventdev_info_get_t dev_infos_get; /**< Get device info. */
@@ -1440,6 +1594,21 @@ struct eventdev_ops {
eventdev_eth_tx_adapter_queue_stop eth_tx_adapter_queue_stop;
/**< Stop Tx queue assigned to Tx adapter instance */
+ eventdev_dma_adapter_caps_get_t dma_adapter_caps_get;
+ /**< Get DMA adapter capabilities */
+ eventdev_dma_adapter_vchan_add_t dma_adapter_vchan_add;
+ /**< Add vchan queue to DMA adapter */
+ eventdev_dma_adapter_vchan_del_t dma_adapter_vchan_del;
+ /**< Delete vchan queue from DMA adapter */
+ eventdev_dma_adapter_start_t dma_adapter_start;
+ /**< Start DMA adapter */
+ eventdev_dma_adapter_stop_t dma_adapter_stop;
+ /**< Stop DMA adapter */
+ eventdev_dma_adapter_stats_get dma_adapter_stats_get;
+ /**< Get DMA stats */
+ eventdev_dma_adapter_stats_reset dma_adapter_stats_reset;
+ /**< Reset DMA stats */
+
eventdev_selftest dev_selftest;
/**< Start eventdev Selftest */
diff --git a/lib/eventdev/eventdev_private.c b/lib/eventdev/eventdev_private.c
index 1d3d9d357e..18ed8bf3c8 100644
--- a/lib/eventdev/eventdev_private.c
+++ b/lib/eventdev/eventdev_private.c
@@ -81,6 +81,14 @@ dummy_event_crypto_adapter_enqueue(__rte_unused void *port,
return 0;
}
+static uint16_t
+dummy_event_dma_adapter_enqueue(__rte_unused void *port, __rte_unused struct rte_event ev[],
+ __rte_unused uint16_t nb_events)
+{
+ RTE_EDEV_LOG_ERR("event DMA adapter enqueue requested for unconfigured event device");
+ return 0;
+}
+
void
event_dev_fp_ops_reset(struct rte_event_fp_ops *fp_op)
{
@@ -97,6 +105,7 @@ event_dev_fp_ops_reset(struct rte_event_fp_ops *fp_op)
.txa_enqueue_same_dest =
dummy_event_tx_adapter_enqueue_same_dest,
.ca_enqueue = dummy_event_crypto_adapter_enqueue,
+ .dma_enqueue = dummy_event_dma_adapter_enqueue,
.data = dummy_data,
};
@@ -117,5 +126,6 @@ event_dev_fp_ops_set(struct rte_event_fp_ops *fp_op,
fp_op->txa_enqueue = dev->txa_enqueue;
fp_op->txa_enqueue_same_dest = dev->txa_enqueue_same_dest;
fp_op->ca_enqueue = dev->ca_enqueue;
+ fp_op->dma_enqueue = dev->dma_enqueue;
fp_op->data = dev->data->ports;
}
diff --git a/lib/eventdev/meson.build b/lib/eventdev/meson.build
index 6edf98dfa5..21347f7c4c 100644
--- a/lib/eventdev/meson.build
+++ b/lib/eventdev/meson.build
@@ -25,6 +25,7 @@ sources = files(
)
headers = files(
'rte_event_crypto_adapter.h',
+ 'rte_event_dma_adapter.h',
'rte_event_eth_rx_adapter.h',
'rte_event_eth_tx_adapter.h',
'rte_event_ring.h',
diff --git a/lib/eventdev/rte_event_dma_adapter.h b/lib/eventdev/rte_event_dma_adapter.h
new file mode 100644
index 0000000000..e924ab673d
--- /dev/null
+++ b/lib/eventdev/rte_event_dma_adapter.h
@@ -0,0 +1,581 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright (c) 2023 Marvell.
+ */
+
+#ifndef RTE_EVENT_DMA_ADAPTER
+#define RTE_EVENT_DMA_ADAPTER
+
+/**
+ * @file rte_event_dma_adapter.h
+ *
+ * @warning
+ * @b EXPERIMENTAL:
+ * All functions in this file may be changed or removed without prior notice.
+ *
+ * DMA Event Adapter API.
+ *
+ * Eventdev library provides adapters to bridge between various components for providing new
+ * event source. The event DMA adapter is one of those adapters which is intended to bridge
+ * between event devices and DMA devices.
+ *
+ * The DMA adapter adds support to enqueue / dequeue DMA operations to / from event device. The
+ * packet flow between DMA device and the event device can be accomplished using both SW and HW
+ * based transfer mechanisms. The adapter uses an EAL service core function for SW based packet
+ * transfer and uses the eventdev PMD functions to configure HW based packet transfer between the
+ * DMA device and the event device.
+ *
+ * The application can choose to submit a DMA operation directly to an DMA device or send it to the
+ * DMA adapter via eventdev based on RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP_FWD capability. The
+ * first mode is known as the event new (RTE_EVENT_DMA_ADAPTER_OP_NEW) mode and the second as the
+ * event forward (RTE_EVENT_DMA_ADAPTER_OP_FORWARD) mode. The choice of mode can be specified while
+ * creating the adapter. In the former mode, it is an application responsibility to enable ingress
+ * packet ordering. In the latter mode, it is the adapter responsibility to enable the ingress
+ * packet ordering.
+ *
+ *
+ * Working model of RTE_EVENT_DMA_ADAPTER_OP_NEW mode:
+ *
+ * +--------------+ +--------------+
+ * | | | DMA stage |
+ * | Application |---[2]-->| + enqueue to |
+ * | | | dmadev |
+ * +--------------+ +--------------+
+ * ^ ^ |
+ * | | [3]
+ * [6] [1] |
+ * | | |
+ * +--------------+ |
+ * | | |
+ * | Event device | |
+ * | | |
+ * +--------------+ |
+ * ^ |
+ * | |
+ * [5] |
+ * | v
+ * +--------------+ +--------------+
+ * | | | |
+ * | DMA adapter |<--[4]---| dmadev |
+ * | | | |
+ * +--------------+ +--------------+
+ *
+ *
+ * [1] Application dequeues events from the previous stage.
+ * [2] Application prepares the DMA operations.
+ * [3] DMA operations are submitted to dmadev by application.
+ * [4] DMA adapter dequeues DMA completions from dmadev.
+ * [5] DMA adapter enqueues events to the eventdev.
+ * [6] Application dequeues from eventdev for further processing.
+ *
+ * In the RTE_EVENT_DMA_ADAPTER_OP_NEW mode, application submits DMA operations directly to DMA
+ * device. The DMA adapter then dequeues DMA completions from DMA device and enqueue events to the
+ * event device. This mode does not ensure ingress ordering, if the application directly enqueues
+ * to dmadev without going through DMA / atomic stage i.e. removing item [1] and [2].
+ *
+ * Events dequeued from the adapter will be treated as new events. In this mode, application needs
+ * to specify event information (response information) which is needed to enqueue an event after the
+ * DMA operation is completed.
+ *
+ *
+ * Working model of RTE_EVENT_DMA_ADAPTER_OP_FORWARD mode:
+ *
+ * +--------------+ +--------------+
+ * --[1]-->| |---[2]-->| Application |
+ * | Event device | | in |
+ * <--[8]--| |<--[3]---| Ordered stage|
+ * +--------------+ +--------------+
+ * ^ |
+ * | [4]
+ * [7] |
+ * | v
+ * +----------------+ +--------------+
+ * | |--[5]->| |
+ * | DMA adapter | | dmadev |
+ * | |<-[6]--| |
+ * +----------------+ +--------------+
+ *
+ *
+ * [1] Events from the previous stage.
+ * [2] Application in ordered stage dequeues events from eventdev.
+ * [3] Application enqueues DMA operations as events to eventdev.
+ * [4] DMA adapter dequeues event from eventdev.
+ * [5] DMA adapter submits DMA operations to dmadev (Atomic stage).
+ * [6] DMA adapter dequeues DMA completions from dmadev
+ * [7] DMA adapter enqueues events to the eventdev
+ * [8] Events to the next stage
+ *
+ * In the event forward (RTE_EVENT_DMA_ADAPTER_OP_FORWARD) mode, if the HW supports the capability
+ * RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP_FWD, application can directly submit the DMA
+ * operations to the dmadev. If not, application retrieves the event port of the DMA adapter
+ * through the API, rte_event_DMA_adapter_event_port_get(). Then, links its event queue to this
+ * port and starts enqueuing DMA operations as events to the eventdev. The adapter then dequeues
+ * the events and submits the DMA operations to the dmadev. After the DMA completions, the adapter
+ * enqueues events to the event device.
+ *
+ * Application can use this mode, when ingress packet ordering is needed. Events dequeued from the
+ * adapter will be treated as forwarded events. In this mode, the application needs to specify the
+ * dmadev ID and queue pair ID (request information) needed to enqueue an DMA operation in addition
+ * to the event information (response information) needed to enqueue an event after the DMA
+ * operation has completed.
+ *
+ * The event DMA adapter provides common APIs to configure the packet flow from the DMA device to
+ * event devices for both SW and HW based transfers. The DMA event adapter's functions are:
+ *
+ * - rte_event_dma_adapter_create_ext()
+ * - rte_event_dma_adapter_create()
+ * - rte_event_dma_adapter_free()
+ * - rte_event_dma_adapter_vchan_add()
+ * - rte_event_dma_adapter_vchan_del()
+ * - rte_event_dma_adapter_start()
+ * - rte_event_dma_adapter_stop()
+ * - rte_event_dma_adapter_stats_get()
+ * - rte_event_dma_adapter_stats_reset()
+ *
+ * The application creates an instance using rte_event_dma_adapter_create() or
+ * rte_event_dma_adapter_create_ext().
+ *
+ * dmadev queue pair addition / deletion is done using the rte_event_dma_adapter_vchan_add() /
+ * rte_event_dma_adapter_vchan_del() APIs. If HW supports the capability
+ * RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_VCHAN_EV_BIND, event information must be passed to the
+ * add API.
+ *
+ */
+
+#include <stdint.h>
+
+#include <rte_common.h>
+#include <rte_dmadev_pmd.h>
+#include <rte_eventdev.h>
+
+#ifdef __cplusplus
+extern "C" {
+#endif
+
+/**
+ * A structure used to hold event based DMA operation entry. All the information
+ * required for a DMA transfer shall be populated in "struct rte_event_dma_adapter_op"
+ * instance.
+ */
+struct rte_event_dma_adapter_op {
+ struct rte_dma_sge *src_seg;
+ /**< Source segments. */
+ struct rte_dma_sge *dst_seg;
+ /**< Destination segments. */
+ uint16_t nb_src;
+ /**< Number of source segments. */
+ uint16_t nb_dst;
+ /**< Number of destination segments. */
+ uint64_t flags;
+ /**< Flags related to the operation.
+ * @see RTE_DMA_OP_FLAG_*
+ */
+ int16_t dma_dev_id;
+ /**< DMA device ID to be used */
+ uint16_t vchan;
+ /**< DMA vchan ID to be used */
+ struct rte_mempool *op_mp;
+ /**< Mempool from which op is allocated. */
+};
+
+/**
+ * DMA event adapter mode
+ */
+enum rte_event_dma_adapter_mode {
+ RTE_EVENT_DMA_ADAPTER_OP_NEW,
+ /**< Start the DMA adapter in event new mode.
+ * @see RTE_EVENT_OP_NEW.
+ *
+ * Application submits DMA operations to the dmadev. Adapter only dequeues the DMA
+ * completions from dmadev and enqueue events to the eventdev.
+ */
+
+ RTE_EVENT_DMA_ADAPTER_OP_FORWARD,
+ /**< Start the DMA adapter in event forward mode.
+ * @see RTE_EVENT_OP_FORWARD.
+ *
+ * Application submits DMA requests as events to the DMA adapter or DMA device based on
+ * RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP_FWD capability. DMA completions are enqueued
+ * back to the eventdev by DMA adapter.
+ */
+};
+
+/**
+ * Adapter configuration structure that the adapter configuration callback function is expected to
+ * fill out.
+ *
+ * @see rte_event_dma_adapter_conf_cb
+ */
+struct rte_event_dma_adapter_conf {
+ uint8_t event_port_id;
+ /** < Event port identifier, the adapter enqueues events to this port and dequeues DMA
+ * request events in RTE_EVENT_DMA_ADAPTER_OP_FORWARD mode.
+ */
+
+ uint32_t max_nb;
+ /**< The adapter can return early if it has processed at least max_nb DMA ops. This isn't
+ * treated as a requirement; batching may cause the adapter to process more than max_nb DMA
+ * ops.
+ */
+};
+
+/**
+ * Adapter runtime configuration parameters
+ */
+struct rte_event_dma_adapter_runtime_params {
+ uint32_t max_nb;
+ /**< The adapter can return early if it has processed at least max_nb DMA ops. This isn't
+ * treated as a requirement; batching may cause the adapter to process more than max_nb DMA
+ * ops.
+ *
+ * Callback function passed to rte_event_dma_adapter_create_ext() configures the adapter
+ * with default value of max_nb.
+ * rte_event_dma_adapter_runtime_params_set() allows to re-configure max_nb during runtime
+ * (after adding at least one queue pair)
+ *
+ * This is valid for the devices without RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP_FWD or
+ * RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP_NEW capability.
+ */
+
+ uint32_t rsvd[15];
+ /**< Reserved fields for future expansion */
+};
+
+/**
+ * Function type used for adapter configuration callback. The callback is used to fill in members of
+ * the struct rte_event_dma_adapter_conf, this callback is invoked when creating a SW service for
+ * packet transfer from dmadev vchan to the event device. The SW service is created within the
+ * function, rte_event_dma_adapter_vchan_add(), if SW based packet transfers from dmadev vchan
+ * to the event device are required.
+ *
+ * @param id
+ * Adapter identifier.
+ * @param evdev_id
+ * Event device identifier.
+ * @param conf
+ * Structure that needs to be populated by this callback.
+ * @param arg
+ * Argument to the callback. This is the same as the conf_arg passed to the
+ * rte_event_dma_adapter_create_ext().
+ */
+typedef int (*rte_event_dma_adapter_conf_cb)(uint8_t id, uint8_t evdev_id,
+ struct rte_event_dma_adapter_conf *conf, void *arg);
+
+/**
+ * A structure used to retrieve statistics for an event DMA adapter instance.
+ */
+struct rte_event_dma_adapter_stats {
+ uint64_t event_poll_count;
+ /**< Event port poll count */
+
+ uint64_t event_deq_count;
+ /**< Event dequeue count */
+
+ uint64_t dma_enq_count;
+ /**< dmadev enqueue count */
+
+ uint64_t dma_enq_fail_count;
+ /**< dmadev enqueue failed count */
+
+ uint64_t dma_deq_count;
+ /**< dmadev dequeue count */
+
+ uint64_t event_enq_count;
+ /**< Event enqueue count */
+
+ uint64_t event_enq_retry_count;
+ /**< Event enqueue retry count */
+
+ uint64_t event_enq_fail_count;
+ /**< Event enqueue fail count */
+};
+
+/**
+ * Create a new event DMA adapter with the specified identifier.
+ *
+ * @param id
+ * Adapter identifier.
+ * @param evdev_id
+ * Event device identifier.
+ * @param conf_cb
+ * Callback function that fills in members of a struct rte_event_dma_adapter_conf struct passed
+ * into it.
+ * @param mode
+ * Flag to indicate the mode of the adapter.
+ * @see rte_event_dma_adapter_mode
+ * @param conf_arg
+ * Argument that is passed to the conf_cb function.
+ *
+ * @return
+ * - 0: Success
+ * - <0: Error code on failure
+ */
+__rte_experimental
+int rte_event_dma_adapter_create_ext(uint8_t id, uint8_t evdev_id,
+ rte_event_dma_adapter_conf_cb conf_cb,
+ enum rte_event_dma_adapter_mode mode, void *conf_arg);
+
+/**
+ * Create a new event DMA adapter with the specified identifier. This function uses an internal
+ * configuration function that creates an event port. This default function reconfigures the event
+ * device with an additional event port and set up the event port using the port_config parameter
+ * passed into this function. In case the application needs more control in configuration of the
+ * service, it should use the rte_event_dma_adapter_create_ext() version.
+ *
+ * @param id
+ * Adapter identifier.
+ * @param evdev_id
+ * Event device identifier.
+ * @param port_config
+ * Argument of type *rte_event_port_conf* that is passed to the conf_cb function.
+ * @param mode
+ * Flag to indicate the mode of the adapter.
+ * @see rte_event_dma_adapter_mode
+ *
+ * @return
+ * - 0: Success
+ * - <0: Error code on failure
+ */
+__rte_experimental
+int rte_event_dma_adapter_create(uint8_t id, uint8_t evdev_id,
+ struct rte_event_port_conf *port_config,
+ enum rte_event_dma_adapter_mode mode);
+
+/**
+ * Free an event DMA adapter
+ *
+ * @param id
+ * Adapter identifier.
+ * @return
+ * - 0: Success
+ * - <0: Error code on failure, If the adapter still has queue pairs added to it, the function
+ * returns -EBUSY.
+ */
+__rte_experimental
+int rte_event_dma_adapter_free(uint8_t id);
+
+/**
+ * Retrieve the event port of an adapter.
+ *
+ * @param id
+ * Adapter identifier.
+ *
+ * @param [out] event_port_id
+ * Application links its event queue to this adapter port which is used in
+ * RTE_EVENT_DMA_ADAPTER_OP_FORWARD mode.
+ *
+ * @return
+ * - 0: Success
+ * - <0: Error code on failure.
+ */
+__rte_experimental
+int rte_event_dma_adapter_event_port_get(uint8_t id, uint8_t *event_port_id);
+
+/**
+ * Add a vchan to an event DMA adapter.
+ *
+ * @param id
+ * Adapter identifier.
+ * @param dmadev_id
+ * dmadev identifier.
+ * @param vchan
+ * DMA device vchan identifier. If vchan is set -1, adapter adds all the
+ * preconfigured vchan to the instance.
+ * @param event
+ * If HW supports dmadev vchan to event queue binding, application is expected to fill in
+ * event information, else it will be NULL.
+ * @see RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_VCHAN_EV_BIND
+ *
+ * @return
+ * - 0: Success, vchan added correctly.
+ * - <0: Error code on failure.
+ */
+__rte_experimental
+int rte_event_dma_adapter_vchan_add(uint8_t id, int16_t dmadev_id, uint16_t vchan,
+ const struct rte_event *event);
+
+/**
+ * Delete a vchan from an event DMA adapter.
+ *
+ * @param id
+ * Adapter identifier.
+ * @param dmadev_id
+ * DMA device identifier.
+ * @param vchan
+ * DMA device vchan identifier.
+ *
+ * @return
+ * - 0: Success, vchan deleted successfully.
+ * - <0: Error code on failure.
+ */
+__rte_experimental
+int rte_event_dma_adapter_vchan_del(uint8_t id, int16_t dmadev_id, uint16_t vchan);
+
+/**
+ * Retrieve the service ID of an adapter. If the adapter doesn't use a rte_service function, this
+ * function returns -ESRCH.
+ *
+ * @param id
+ * Adapter identifier.
+ * @param [out] service_id
+ * A pointer to a uint32_t, to be filled in with the service id.
+ *
+ * @return
+ * - 0: Success
+ * - <0: Error code on failure, if the adapter doesn't use a rte_service function, this function
+ * returns -ESRCH.
+ */
+__rte_experimental
+int rte_event_dma_adapter_service_id_get(uint8_t id, uint32_t *service_id);
+
+/**
+ * Start event DMA adapter
+ *
+ * @param id
+ * Adapter identifier.
+ *
+ * @return
+ * - 0: Success, adapter started successfully.
+ * - <0: Error code on failure.
+ *
+ * @note The eventdev and dmadev to which the event_dma_adapter is connected should be started
+ * before calling rte_event_dma_adapter_start().
+ */
+__rte_experimental
+int rte_event_dma_adapter_start(uint8_t id);
+
+/**
+ * Stop event DMA adapter
+ *
+ * @param id
+ * Adapter identifier.
+ *
+ * @return
+ * - 0: Success, adapter stopped successfully.
+ * - <0: Error code on failure.
+ */
+__rte_experimental
+int rte_event_dma_adapter_stop(uint8_t id);
+
+/**
+ * Initialize the adapter runtime configuration parameters
+ *
+ * @param params
+ * A pointer to structure of type struct rte_event_dma_adapter_runtime_params
+ *
+ * @return
+ * - 0: Success
+ * - <0: Error code on failure
+ */
+__rte_experimental
+int rte_event_dma_adapter_runtime_params_init(struct rte_event_dma_adapter_runtime_params *params);
+
+/**
+ * Set the adapter runtime configuration parameters
+ *
+ * @param id
+ * Adapter identifier
+ *
+ * @param params
+ * A pointer to structure of type struct rte_event_dma_adapter_runtime_params with configuration
+ * parameter values. The reserved fields of this structure must be initialized to zero and the valid
+ * fields need to be set appropriately. This struct can be initialized using
+ * rte_event_dma_adapter_runtime_params_init() API to default values or application may reset this
+ * struct and update required fields.
+ *
+ * @return
+ * - 0: Success
+ * - <0: Error code on failure
+ */
+__rte_experimental
+int rte_event_dma_adapter_runtime_params_set(uint8_t id,
+ struct rte_event_dma_adapter_runtime_params *params);
+
+/**
+ * Get the adapter runtime configuration parameters
+ *
+ * @param id
+ * Adapter identifier
+ *
+ * @param[out] params
+ * A pointer to structure of type struct rte_event_dma_adapter_runtime_params containing valid
+ * adapter parameters when return value is 0.
+ *
+ * @return
+ * - 0: Success
+ * - <0: Error code on failure
+ */
+__rte_experimental
+int rte_event_dma_adapter_runtime_params_get(uint8_t id,
+ struct rte_event_dma_adapter_runtime_params *params);
+
+/**
+ * Retrieve statistics for an adapter
+ *
+ * @param id
+ * Adapter identifier.
+ * @param [out] stats
+ * A pointer to structure used to retrieve statistics for an adapter.
+ *
+ * @return
+ * - 0: Success, retrieved successfully.
+ * - <0: Error code on failure.
+ */
+__rte_experimental
+int rte_event_dma_adapter_stats_get(uint8_t id, struct rte_event_dma_adapter_stats *stats);
+
+/**
+ * Reset statistics for an adapter.
+ *
+ * @param id
+ * Adapter identifier.
+ *
+ * @return
+ * - 0: Success, statistics reset successfully.
+ * - <0: Error code on failure.
+ */
+__rte_experimental
+int rte_event_dma_adapter_stats_reset(uint8_t id);
+
+/**
+ * Enqueue a burst of DMA operations as event objects supplied in *rte_event* structure on an event
+ * DMA adapter designated by its event *evdev_id* through the event port specified by *port_id*.
+ * This function is supported if the eventdev PMD has the
+ * #RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP_FWD capability flag set.
+ *
+ * The *nb_events* parameter is the number of event objects to enqueue that are supplied in the
+ * *ev* array of *rte_event* structure.
+ *
+ * The rte_event_dma_adapter_enqueue() function returns the number of event objects it actually
+ * enqueued. A return value equal to *nb_events* means that all event objects have been enqueued.
+ *
+ * @param evdev_id
+ * The identifier of the device.
+ * @param port_id
+ * The identifier of the event port.
+ * @param ev
+ * Points to an array of *nb_events* objects of type *rte_event* structure which contain the
+ * event object enqueue operations to be processed.
+ * @param nb_events
+ * The number of event objects to enqueue, typically number of
+ * rte_event_port_attr_get(...RTE_EVENT_PORT_ATTR_ENQ_DEPTH...) available for this port.
+ *
+ * @return
+ * The number of event objects actually enqueued on the event device. The return value can be
+ * less than the value of the *nb_events* parameter when the event devices queue is full or if
+ * invalid parameters are specified in a *rte_event*. If the return value is less than *nb_events*,
+ * the remaining events at the end of ev[] are not consumed and the caller has to take care of them,
+ * and rte_errno is set accordingly. Possible errno values include:
+ * - EINVAL: The port ID is invalid, device ID is invalid, an event's queue ID is invalid, or an
+ * event's sched type doesn't match the capabilities of the destination queue.
+ * - ENOSPC: The event port was backpressured and unable to enqueue one or more events. This
+ * error code is only applicable to closed systems.
+ */
+__rte_experimental
+uint16_t rte_event_dma_adapter_enqueue(uint8_t evdev_id, uint8_t port_id, struct rte_event ev[],
+ uint16_t nb_events);
+
+#ifdef __cplusplus
+}
+#endif
+
+#endif /* RTE_EVENT_DMA_ADAPTER */
diff --git a/lib/eventdev/rte_eventdev.h b/lib/eventdev/rte_eventdev.h
index 2ba8a7b090..41743f91b1 100644
--- a/lib/eventdev/rte_eventdev.h
+++ b/lib/eventdev/rte_eventdev.h
@@ -1197,6 +1197,8 @@ struct rte_event_vector {
*/
#define RTE_EVENT_TYPE_ETH_RX_ADAPTER 0x4
/**< The event generated from event eth Rx adapter */
+#define RTE_EVENT_TYPE_DMADEV 0x5
+/**< The event generated from dma subsystem */
#define RTE_EVENT_TYPE_VECTOR 0x8
/**< Indicates that event is a vector.
* All vector event types should be a logical OR of EVENT_TYPE_VECTOR.
@@ -1462,6 +1464,48 @@ int
rte_event_crypto_adapter_caps_get(uint8_t dev_id, uint8_t cdev_id,
uint32_t *caps);
+/* DMA adapter capability bitmap flag */
+#define RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP_NEW 0x1
+/**< Flag indicates HW is capable of generating events in
+ * RTE_EVENT_OP_NEW enqueue operation. DMADEV will send
+ * packets to the event device as new events using an
+ * internal event port.
+ */
+
+#define RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP_FWD 0x2
+/**< Flag indicates HW is capable of generating events in
+ * RTE_EVENT_OP_FORWARD enqueue operation. DMADEV will send
+ * packets to the event device as forwarded event using an
+ * internal event port.
+ */
+
+#define RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_VCHAN_EV_BIND 0x4
+/**< Flag indicates HW is capable of mapping DMA vchan to event queue. */
+
+/**
+ * Retrieve the event device's DMA adapter capabilities for the
+ * specified dmadev device
+ *
+ * @param dev_id
+ * The identifier of the device.
+ *
+ * @param dmadev_id
+ * The identifier of the dmadev device.
+ *
+ * @param[out] caps
+ * A pointer to memory filled with event adapter capabilities.
+ * It is expected to be pre-allocated & initialized by caller.
+ *
+ * @return
+ * - 0: Success, driver provides event adapter capabilities for the
+ * dmadev device.
+ * - <0: Error code returned by the driver function.
+ *
+ */
+__rte_experimental
+int
+rte_event_dma_adapter_caps_get(uint8_t dev_id, uint8_t dmadev_id, uint32_t *caps);
+
/* Ethdev Tx adapter capability bitmap flags */
#define RTE_EVENT_ETH_TX_ADAPTER_CAP_INTERNAL_PORT 0x1
/**< This flag is sent when the PMD supports a packet transmit callback
diff --git a/lib/eventdev/rte_eventdev_core.h b/lib/eventdev/rte_eventdev_core.h
index c27a52ccc0..83e8736c71 100644
--- a/lib/eventdev/rte_eventdev_core.h
+++ b/lib/eventdev/rte_eventdev_core.h
@@ -42,6 +42,10 @@ typedef uint16_t (*event_crypto_adapter_enqueue_t)(void *port,
uint16_t nb_events);
/**< @internal Enqueue burst of events on crypto adapter */
+typedef uint16_t (*event_dma_adapter_enqueue_t)(void *port, struct rte_event ev[],
+ uint16_t nb_events);
+/**< @internal Enqueue burst of events on DMA adapter */
+
struct rte_event_fp_ops {
void **data;
/**< points to array of internal port data pointers */
@@ -65,7 +69,9 @@ struct rte_event_fp_ops {
/**< PMD Tx adapter enqueue same destination function. */
event_crypto_adapter_enqueue_t ca_enqueue;
/**< PMD Crypto adapter enqueue function. */
- uintptr_t reserved[5];
+ event_dma_adapter_enqueue_t dma_enqueue;
+ /**< PMD DMA adapter enqueue function. */
+ uintptr_t reserved[4];
} __rte_cache_aligned;
extern struct rte_event_fp_ops rte_event_fp_ops[RTE_EVENT_MAX_DEVS];
diff --git a/lib/eventdev/version.map b/lib/eventdev/version.map
index 7ce09a87bb..b81eb2919c 100644
--- a/lib/eventdev/version.map
+++ b/lib/eventdev/version.map
@@ -133,6 +133,22 @@ EXPERIMENTAL {
rte_event_timer_remaining_ticks_get;
# added in 23.11
+ rte_event_dma_adapter_caps_get;
+ rte_event_dma_adapter_create;
+ rte_event_dma_adapter_create_ext;
+ rte_event_dma_adapter_enqueue;
+ rte_event_dma_adapter_event_port_get;
+ rte_event_dma_adapter_free;
+ rte_event_dma_adapter_runtime_params_get;
+ rte_event_dma_adapter_runtime_params_init;
+ rte_event_dma_adapter_runtime_params_set;
+ rte_event_dma_adapter_service_id_get;
+ rte_event_dma_adapter_start;
+ rte_event_dma_adapter_stats_get;
+ rte_event_dma_adapter_stats_reset;
+ rte_event_dma_adapter_stop;
+ rte_event_dma_adapter_vchan_add;
+ rte_event_dma_adapter_vchan_del;
rte_event_eth_rx_adapter_create_ext_with_params;
};
diff --git a/lib/meson.build b/lib/meson.build
index 53155be8e9..8330d90820 100644
--- a/lib/meson.build
+++ b/lib/meson.build
@@ -33,6 +33,7 @@ libraries = [
'compressdev',
'cryptodev',
'distributor',
+ 'dmadev', # eventdev depends on this
'efd',
'eventdev',
'gpudev',
@@ -48,7 +49,6 @@ libraries = [
'rawdev',
'regexdev',
'mldev',
- 'dmadev',
'rib',
'reorder',
'sched',
--
2.25.1
^ permalink raw reply [flat|nested] 110+ messages in thread
* [PATCH v8 02/12] eventdev/dma: support adapter capabilities get
2023-09-29 11:50 ` [PATCH v8 00/12] event DMA adapter library support Amit Prakash Shukla
2023-09-29 11:50 ` [PATCH v8 01/12] eventdev/dma: introduce DMA adapter Amit Prakash Shukla
@ 2023-09-29 11:50 ` Amit Prakash Shukla
2023-09-29 11:50 ` [PATCH v8 03/12] eventdev/dma: support adapter create and free Amit Prakash Shukla
` (10 subsequent siblings)
12 siblings, 0 replies; 110+ messages in thread
From: Amit Prakash Shukla @ 2023-09-29 11:50 UTC (permalink / raw)
To: Jerin Jacob
Cc: dev, fengchengwen, kevin.laatz, bruce.richardson, conor.walsh,
vattunuru, g.singh, sachin.saxena, hemant.agrawal, cheng1.jiang,
ndabilpuram, anoobj, mb, Amit Prakash Shukla
Added a new eventdev API rte_event_dma_adapter_caps_get(), to get
DMA adapter capabilities supported by the driver.
Signed-off-by: Amit Prakash Shukla <amitprakashs@marvell.com>
---
lib/eventdev/meson.build | 2 +-
lib/eventdev/rte_eventdev.c | 23 +++++++++++++++++++++++
2 files changed, 24 insertions(+), 1 deletion(-)
diff --git a/lib/eventdev/meson.build b/lib/eventdev/meson.build
index 21347f7c4c..b46bbbc9aa 100644
--- a/lib/eventdev/meson.build
+++ b/lib/eventdev/meson.build
@@ -43,5 +43,5 @@ driver_sdk_headers += files(
'event_timer_adapter_pmd.h',
)
-deps += ['ring', 'ethdev', 'hash', 'mempool', 'mbuf', 'timer', 'cryptodev']
+deps += ['ring', 'ethdev', 'hash', 'mempool', 'mbuf', 'timer', 'cryptodev', 'dmadev']
deps += ['telemetry']
diff --git a/lib/eventdev/rte_eventdev.c b/lib/eventdev/rte_eventdev.c
index 6ab4524332..60509c6efb 100644
--- a/lib/eventdev/rte_eventdev.c
+++ b/lib/eventdev/rte_eventdev.c
@@ -20,6 +20,7 @@
#include <rte_errno.h>
#include <ethdev_driver.h>
#include <rte_cryptodev.h>
+#include <rte_dmadev.h>
#include <cryptodev_pmd.h>
#include <rte_telemetry.h>
@@ -224,6 +225,28 @@ rte_event_eth_tx_adapter_caps_get(uint8_t dev_id, uint16_t eth_port_id,
: 0;
}
+int
+rte_event_dma_adapter_caps_get(uint8_t dev_id, uint8_t dma_dev_id, uint32_t *caps)
+{
+ struct rte_eventdev *dev;
+
+ RTE_EVENTDEV_VALID_DEVID_OR_ERR_RET(dev_id, -EINVAL);
+ if (!rte_dma_is_valid(dma_dev_id))
+ return -EINVAL;
+
+ dev = &rte_eventdevs[dev_id];
+
+ if (caps == NULL)
+ return -EINVAL;
+
+ *caps = 0;
+
+ if (dev->dev_ops->dma_adapter_caps_get)
+ return (*dev->dev_ops->dma_adapter_caps_get)(dev, dma_dev_id, caps);
+
+ return 0;
+}
+
static inline int
event_dev_queue_config(struct rte_eventdev *dev, uint8_t nb_queues)
{
--
2.25.1
^ permalink raw reply [flat|nested] 110+ messages in thread
* [PATCH v8 03/12] eventdev/dma: support adapter create and free
2023-09-29 11:50 ` [PATCH v8 00/12] event DMA adapter library support Amit Prakash Shukla
2023-09-29 11:50 ` [PATCH v8 01/12] eventdev/dma: introduce DMA adapter Amit Prakash Shukla
2023-09-29 11:50 ` [PATCH v8 02/12] eventdev/dma: support adapter capabilities get Amit Prakash Shukla
@ 2023-09-29 11:50 ` Amit Prakash Shukla
2023-09-29 11:50 ` [PATCH v8 04/12] eventdev/dma: support vchan add and delete Amit Prakash Shukla
` (9 subsequent siblings)
12 siblings, 0 replies; 110+ messages in thread
From: Amit Prakash Shukla @ 2023-09-29 11:50 UTC (permalink / raw)
To: Bruce Richardson, Jerin Jacob, Amit Prakash Shukla
Cc: dev, fengchengwen, kevin.laatz, conor.walsh, vattunuru, g.singh,
sachin.saxena, hemant.agrawal, cheng1.jiang, ndabilpuram, anoobj,
mb
Added API support to create and free DMA adapter. Create function shall be
called with event device to be associated with the adapter and port
configuration to setup an event port.
Signed-off-by: Amit Prakash Shukla <amitprakashs@marvell.com>
---
config/rte_config.h | 1 +
lib/eventdev/meson.build | 1 +
lib/eventdev/rte_event_dma_adapter.c | 335 +++++++++++++++++++++++++++
3 files changed, 337 insertions(+)
create mode 100644 lib/eventdev/rte_event_dma_adapter.c
diff --git a/config/rte_config.h b/config/rte_config.h
index 400e44e3cf..401727703f 100644
--- a/config/rte_config.h
+++ b/config/rte_config.h
@@ -77,6 +77,7 @@
#define RTE_EVENT_ETH_INTR_RING_SIZE 1024
#define RTE_EVENT_CRYPTO_ADAPTER_MAX_INSTANCE 32
#define RTE_EVENT_ETH_TX_ADAPTER_MAX_INSTANCE 32
+#define RTE_EVENT_DMA_ADAPTER_MAX_INSTANCE 32
/* rawdev defines */
#define RTE_RAWDEV_MAX_DEVS 64
diff --git a/lib/eventdev/meson.build b/lib/eventdev/meson.build
index b46bbbc9aa..250abcb154 100644
--- a/lib/eventdev/meson.build
+++ b/lib/eventdev/meson.build
@@ -17,6 +17,7 @@ sources = files(
'eventdev_private.c',
'eventdev_trace_points.c',
'rte_event_crypto_adapter.c',
+ 'rte_event_dma_adapter.c',
'rte_event_eth_rx_adapter.c',
'rte_event_eth_tx_adapter.c',
'rte_event_ring.c',
diff --git a/lib/eventdev/rte_event_dma_adapter.c b/lib/eventdev/rte_event_dma_adapter.c
new file mode 100644
index 0000000000..241327d2a7
--- /dev/null
+++ b/lib/eventdev/rte_event_dma_adapter.c
@@ -0,0 +1,335 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright (c) 2023 Marvell.
+ */
+
+#include <eventdev_pmd.h>
+
+#include "rte_event_dma_adapter.h"
+
+#define DMA_BATCH_SIZE 32
+#define DMA_DEFAULT_MAX_NB 128
+#define DMA_ADAPTER_NAME_LEN 32
+#define DMA_ADAPTER_BUFFER_SIZE 1024
+
+#define DMA_ADAPTER_OPS_BUFFER_SIZE (DMA_BATCH_SIZE + DMA_BATCH_SIZE)
+
+#define DMA_ADAPTER_ARRAY "event_dma_adapter_array"
+
+/* Macros to check for valid adapter */
+#define EVENT_DMA_ADAPTER_ID_VALID_OR_ERR_RET(id, retval) \
+ do { \
+ if (!edma_adapter_valid_id(id)) { \
+ RTE_EDEV_LOG_ERR("Invalid DMA adapter id = %d\n", id); \
+ return retval; \
+ } \
+ } while (0)
+
+/* DMA ops circular buffer */
+struct dma_ops_circular_buffer {
+ /* Index of head element */
+ uint16_t head;
+
+ /* Index of tail element */
+ uint16_t tail;
+
+ /* Number of elements in buffer */
+ uint16_t count;
+
+ /* Size of circular buffer */
+ uint16_t size;
+
+ /* Pointer to hold rte_event_dma_adapter_op for processing */
+ struct rte_event_dma_adapter_op **op_buffer;
+} __rte_cache_aligned;
+
+/* DMA device information */
+struct dma_device_info {
+ /* Number of vchans configured for a DMA device. */
+ uint16_t num_dma_dev_vchan;
+} __rte_cache_aligned;
+
+struct event_dma_adapter {
+ /* Event device identifier */
+ uint8_t eventdev_id;
+
+ /* Event port identifier */
+ uint8_t event_port_id;
+
+ /* Adapter mode */
+ enum rte_event_dma_adapter_mode mode;
+
+ /* Memory allocation name */
+ char mem_name[DMA_ADAPTER_NAME_LEN];
+
+ /* Socket identifier cached from eventdev */
+ int socket_id;
+
+ /* Lock to serialize config updates with service function */
+ rte_spinlock_t lock;
+
+ /* DMA device structure array */
+ struct dma_device_info *dma_devs;
+
+ /* Circular buffer for processing DMA ops to eventdev */
+ struct dma_ops_circular_buffer ebuf;
+
+ /* Configuration callback for rte_service configuration */
+ rte_event_dma_adapter_conf_cb conf_cb;
+
+ /* Configuration callback argument */
+ void *conf_arg;
+
+ /* Set if default_cb is being used */
+ int default_cb_arg;
+} __rte_cache_aligned;
+
+static struct event_dma_adapter **event_dma_adapter;
+
+static inline int
+edma_adapter_valid_id(uint8_t id)
+{
+ return id < RTE_EVENT_DMA_ADAPTER_MAX_INSTANCE;
+}
+
+static inline struct event_dma_adapter *
+edma_id_to_adapter(uint8_t id)
+{
+ return event_dma_adapter ? event_dma_adapter[id] : NULL;
+}
+
+static int
+edma_array_init(void)
+{
+ const struct rte_memzone *mz;
+ uint32_t sz;
+
+ mz = rte_memzone_lookup(DMA_ADAPTER_ARRAY);
+ if (mz == NULL) {
+ sz = sizeof(struct event_dma_adapter *) * RTE_EVENT_DMA_ADAPTER_MAX_INSTANCE;
+ sz = RTE_ALIGN(sz, RTE_CACHE_LINE_SIZE);
+
+ mz = rte_memzone_reserve_aligned(DMA_ADAPTER_ARRAY, sz, rte_socket_id(), 0,
+ RTE_CACHE_LINE_SIZE);
+ if (mz == NULL) {
+ RTE_EDEV_LOG_ERR("Failed to reserve memzone : %s, err = %d",
+ DMA_ADAPTER_ARRAY, rte_errno);
+ return -rte_errno;
+ }
+ }
+
+ event_dma_adapter = mz->addr;
+
+ return 0;
+}
+
+static inline int
+edma_circular_buffer_init(const char *name, struct dma_ops_circular_buffer *buf, uint16_t sz)
+{
+ buf->op_buffer = rte_zmalloc(name, sizeof(struct rte_event_dma_adapter_op *) * sz, 0);
+ if (buf->op_buffer == NULL)
+ return -ENOMEM;
+
+ buf->size = sz;
+
+ return 0;
+}
+
+static inline void
+edma_circular_buffer_free(struct dma_ops_circular_buffer *buf)
+{
+ rte_free(buf->op_buffer);
+}
+
+static int
+edma_default_config_cb(uint8_t id, uint8_t evdev_id, struct rte_event_dma_adapter_conf *conf,
+ void *arg)
+{
+ struct rte_event_port_conf *port_conf;
+ struct rte_event_dev_config dev_conf;
+ struct event_dma_adapter *adapter;
+ struct rte_eventdev *dev;
+ uint8_t port_id;
+ int started;
+ int ret;
+
+ adapter = edma_id_to_adapter(id);
+ if (adapter == NULL)
+ return -EINVAL;
+
+ dev = &rte_eventdevs[adapter->eventdev_id];
+ dev_conf = dev->data->dev_conf;
+
+ started = dev->data->dev_started;
+ if (started)
+ rte_event_dev_stop(evdev_id);
+
+ port_id = dev_conf.nb_event_ports;
+ dev_conf.nb_event_ports += 1;
+
+ port_conf = arg;
+ if (port_conf->event_port_cfg & RTE_EVENT_PORT_CFG_SINGLE_LINK)
+ dev_conf.nb_single_link_event_port_queues += 1;
+
+ ret = rte_event_dev_configure(evdev_id, &dev_conf);
+ if (ret) {
+ RTE_EDEV_LOG_ERR("Failed to configure event dev %u\n", evdev_id);
+ if (started) {
+ if (rte_event_dev_start(evdev_id))
+ return -EIO;
+ }
+ return ret;
+ }
+
+ ret = rte_event_port_setup(evdev_id, port_id, port_conf);
+ if (ret) {
+ RTE_EDEV_LOG_ERR("Failed to setup event port %u\n", port_id);
+ return ret;
+ }
+
+ conf->event_port_id = port_id;
+ conf->max_nb = DMA_DEFAULT_MAX_NB;
+ if (started)
+ ret = rte_event_dev_start(evdev_id);
+
+ adapter->default_cb_arg = 1;
+ adapter->event_port_id = conf->event_port_id;
+
+ return ret;
+}
+
+int
+rte_event_dma_adapter_create_ext(uint8_t id, uint8_t evdev_id,
+ rte_event_dma_adapter_conf_cb conf_cb,
+ enum rte_event_dma_adapter_mode mode, void *conf_arg)
+{
+ struct rte_event_dev_info dev_info;
+ struct event_dma_adapter *adapter;
+ char name[DMA_ADAPTER_NAME_LEN];
+ struct rte_dma_info info;
+ uint16_t num_dma_dev;
+ int socket_id;
+ uint8_t i;
+ int ret;
+
+ EVENT_DMA_ADAPTER_ID_VALID_OR_ERR_RET(id, -EINVAL);
+ RTE_EVENTDEV_VALID_DEVID_OR_ERR_RET(evdev_id, -EINVAL);
+
+ if (conf_cb == NULL)
+ return -EINVAL;
+
+ if (event_dma_adapter == NULL) {
+ ret = edma_array_init();
+ if (ret)
+ return ret;
+ }
+
+ adapter = edma_id_to_adapter(id);
+ if (adapter != NULL) {
+ RTE_EDEV_LOG_ERR("ML adapter ID %d already exists!", id);
+ return -EEXIST;
+ }
+
+ socket_id = rte_event_dev_socket_id(evdev_id);
+ snprintf(name, DMA_ADAPTER_NAME_LEN, "rte_event_dma_adapter_%d", id);
+ adapter = rte_zmalloc_socket(name, sizeof(struct event_dma_adapter), RTE_CACHE_LINE_SIZE,
+ socket_id);
+ if (adapter == NULL) {
+ RTE_EDEV_LOG_ERR("Failed to get mem for event ML adapter!");
+ return -ENOMEM;
+ }
+
+ if (edma_circular_buffer_init("edma_circular_buffer", &adapter->ebuf,
+ DMA_ADAPTER_BUFFER_SIZE)) {
+ RTE_EDEV_LOG_ERR("Failed to get memory for event adapter circular buffer");
+ rte_free(adapter);
+ return -ENOMEM;
+ }
+
+ ret = rte_event_dev_info_get(evdev_id, &dev_info);
+ if (ret < 0) {
+ RTE_EDEV_LOG_ERR("Failed to get info for eventdev %d: %s", evdev_id,
+ dev_info.driver_name);
+ edma_circular_buffer_free(&adapter->ebuf);
+ rte_free(adapter);
+ return ret;
+ }
+
+ num_dma_dev = rte_dma_count_avail();
+
+ adapter->eventdev_id = evdev_id;
+ adapter->mode = mode;
+ rte_strscpy(adapter->mem_name, name, DMA_ADAPTER_NAME_LEN);
+ adapter->socket_id = socket_id;
+ adapter->conf_cb = conf_cb;
+ adapter->conf_arg = conf_arg;
+ adapter->dma_devs = rte_zmalloc_socket(adapter->mem_name,
+ num_dma_dev * sizeof(struct dma_device_info), 0,
+ socket_id);
+ if (adapter->dma_devs == NULL) {
+ RTE_EDEV_LOG_ERR("Failed to get memory for DMA devices\n");
+ edma_circular_buffer_free(&adapter->ebuf);
+ rte_free(adapter);
+ return -ENOMEM;
+ }
+
+ rte_spinlock_init(&adapter->lock);
+ for (i = 0; i < num_dma_dev; i++) {
+ ret = rte_dma_info_get(i, &info);
+ if (ret) {
+ RTE_EDEV_LOG_ERR("Failed to get dma device info\n");
+ edma_circular_buffer_free(&adapter->ebuf);
+ rte_free(adapter);
+ return ret;
+ }
+
+ adapter->dma_devs[i].num_dma_dev_vchan = info.nb_vchans;
+ }
+
+ event_dma_adapter[id] = adapter;
+
+ return 0;
+}
+
+int
+rte_event_dma_adapter_create(uint8_t id, uint8_t evdev_id, struct rte_event_port_conf *port_config,
+ enum rte_event_dma_adapter_mode mode)
+{
+ struct rte_event_port_conf *pc;
+ int ret;
+
+ if (port_config == NULL)
+ return -EINVAL;
+
+ EVENT_DMA_ADAPTER_ID_VALID_OR_ERR_RET(id, -EINVAL);
+
+ pc = rte_malloc(NULL, sizeof(struct rte_event_port_conf), 0);
+ if (pc == NULL)
+ return -ENOMEM;
+
+ rte_memcpy(pc, port_config, sizeof(struct rte_event_port_conf));
+ ret = rte_event_dma_adapter_create_ext(id, evdev_id, edma_default_config_cb, mode, pc);
+ if (ret != 0)
+ rte_free(pc);
+
+ return ret;
+}
+
+int
+rte_event_dma_adapter_free(uint8_t id)
+{
+ struct event_dma_adapter *adapter;
+
+ EVENT_DMA_ADAPTER_ID_VALID_OR_ERR_RET(id, -EINVAL);
+
+ adapter = edma_id_to_adapter(id);
+ if (adapter == NULL)
+ return -EINVAL;
+
+ rte_free(adapter->conf_arg);
+ rte_free(adapter->dma_devs);
+ edma_circular_buffer_free(&adapter->ebuf);
+ rte_free(adapter);
+ event_dma_adapter[id] = NULL;
+
+ return 0;
+}
--
2.25.1
^ permalink raw reply [flat|nested] 110+ messages in thread
* [PATCH v8 04/12] eventdev/dma: support vchan add and delete
2023-09-29 11:50 ` [PATCH v8 00/12] event DMA adapter library support Amit Prakash Shukla
` (2 preceding siblings ...)
2023-09-29 11:50 ` [PATCH v8 03/12] eventdev/dma: support adapter create and free Amit Prakash Shukla
@ 2023-09-29 11:50 ` Amit Prakash Shukla
2023-09-29 11:50 ` [PATCH v8 05/12] eventdev/dma: support adapter service function Amit Prakash Shukla
` (8 subsequent siblings)
12 siblings, 0 replies; 110+ messages in thread
From: Amit Prakash Shukla @ 2023-09-29 11:50 UTC (permalink / raw)
To: Amit Prakash Shukla, Jerin Jacob
Cc: dev, fengchengwen, kevin.laatz, bruce.richardson, conor.walsh,
vattunuru, g.singh, sachin.saxena, hemant.agrawal, cheng1.jiang,
ndabilpuram, anoobj, mb
Added API support to add and delete vchan's from the DMA adapter. DMA devid
and vchan are added to the addapter instance by calling
rte_event_dma_adapter_vchan_add and deleted using
rte_event_dma_adapter_vchan_del.
Signed-off-by: Amit Prakash Shukla <amitprakashs@marvell.com>
---
lib/eventdev/rte_event_dma_adapter.c | 204 +++++++++++++++++++++++++++
1 file changed, 204 insertions(+)
diff --git a/lib/eventdev/rte_event_dma_adapter.c b/lib/eventdev/rte_event_dma_adapter.c
index 241327d2a7..fa2e29b9d3 100644
--- a/lib/eventdev/rte_event_dma_adapter.c
+++ b/lib/eventdev/rte_event_dma_adapter.c
@@ -42,8 +42,31 @@ struct dma_ops_circular_buffer {
struct rte_event_dma_adapter_op **op_buffer;
} __rte_cache_aligned;
+/* Vchan information */
+struct dma_vchan_info {
+ /* Set to indicate vchan queue is enabled */
+ bool vq_enabled;
+
+ /* Circular buffer for batching DMA ops to dma_dev */
+ struct dma_ops_circular_buffer dma_buf;
+} __rte_cache_aligned;
+
/* DMA device information */
struct dma_device_info {
+ /* Pointer to vchan queue info */
+ struct dma_vchan_info *vchanq;
+
+ /* Pointer to vchan queue info.
+ * This holds ops passed by application till the
+ * dma completion is done.
+ */
+ struct dma_vchan_info *tqmap;
+
+ /* If num_vchanq > 0, the start callback will
+ * be invoked if not already invoked
+ */
+ uint16_t num_vchanq;
+
/* Number of vchans configured for a DMA device. */
uint16_t num_dma_dev_vchan;
} __rte_cache_aligned;
@@ -81,6 +104,9 @@ struct event_dma_adapter {
/* Set if default_cb is being used */
int default_cb_arg;
+
+ /* No. of vchan queue configured */
+ uint16_t nb_vchanq;
} __rte_cache_aligned;
static struct event_dma_adapter **event_dma_adapter;
@@ -333,3 +359,181 @@ rte_event_dma_adapter_free(uint8_t id)
return 0;
}
+
+static void
+edma_update_vchanq_info(struct event_dma_adapter *adapter, struct dma_device_info *dev_info,
+ uint16_t vchan, uint8_t add)
+{
+ struct dma_vchan_info *vchan_info;
+ struct dma_vchan_info *tqmap_info;
+ int enabled;
+ uint16_t i;
+
+ if (dev_info->vchanq == NULL)
+ return;
+
+ if (vchan == RTE_DMA_ALL_VCHAN) {
+ for (i = 0; i < dev_info->num_dma_dev_vchan; i++)
+ edma_update_vchanq_info(adapter, dev_info, i, add);
+ } else {
+ tqmap_info = &dev_info->tqmap[vchan];
+ vchan_info = &dev_info->vchanq[vchan];
+ enabled = vchan_info->vq_enabled;
+ if (add) {
+ adapter->nb_vchanq += !enabled;
+ dev_info->num_vchanq += !enabled;
+ } else {
+ adapter->nb_vchanq -= enabled;
+ dev_info->num_vchanq -= enabled;
+ }
+ vchan_info->vq_enabled = !!add;
+ tqmap_info->vq_enabled = !!add;
+ }
+}
+
+int
+rte_event_dma_adapter_vchan_add(uint8_t id, int16_t dma_dev_id, uint16_t vchan,
+ const struct rte_event *event)
+{
+ struct event_dma_adapter *adapter;
+ struct dma_device_info *dev_info;
+ struct rte_eventdev *dev;
+ uint32_t cap;
+ int ret;
+
+ EVENT_DMA_ADAPTER_ID_VALID_OR_ERR_RET(id, -EINVAL);
+
+ if (!rte_dma_is_valid(dma_dev_id)) {
+ RTE_EDEV_LOG_ERR("Invalid dma_dev_id = %" PRIu8, dma_dev_id);
+ return -EINVAL;
+ }
+
+ adapter = edma_id_to_adapter(id);
+ if (adapter == NULL)
+ return -EINVAL;
+
+ dev = &rte_eventdevs[adapter->eventdev_id];
+ ret = rte_event_dma_adapter_caps_get(adapter->eventdev_id, dma_dev_id, &cap);
+ if (ret) {
+ RTE_EDEV_LOG_ERR("Failed to get adapter caps dev %u dma_dev %u", id, dma_dev_id);
+ return ret;
+ }
+
+ if ((cap & RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_VCHAN_EV_BIND) && (event == NULL)) {
+ RTE_EDEV_LOG_ERR("Event can not be NULL for dma_dev_id = %u", dma_dev_id);
+ return -EINVAL;
+ }
+
+ dev_info = &adapter->dma_devs[dma_dev_id];
+ if (vchan != RTE_DMA_ALL_VCHAN && vchan >= dev_info->num_dma_dev_vchan) {
+ RTE_EDEV_LOG_ERR("Invalid vhcan %u", vchan);
+ return -EINVAL;
+ }
+
+ /* In case HW cap is RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP_FWD, no
+ * need of service core as HW supports event forward capability.
+ */
+ if ((cap & RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP_FWD) ||
+ (cap & RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_VCHAN_EV_BIND &&
+ adapter->mode == RTE_EVENT_DMA_ADAPTER_OP_NEW) ||
+ (cap & RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP_NEW &&
+ adapter->mode == RTE_EVENT_DMA_ADAPTER_OP_NEW)) {
+ if (*dev->dev_ops->dma_adapter_vchan_add == NULL)
+ return -ENOTSUP;
+ if (dev_info->vchanq == NULL) {
+ dev_info->vchanq = rte_zmalloc_socket(adapter->mem_name,
+ dev_info->num_dma_dev_vchan *
+ sizeof(struct dma_vchan_info),
+ 0, adapter->socket_id);
+ if (dev_info->vchanq == NULL) {
+ printf("Queue pair add not supported\n");
+ return -ENOMEM;
+ }
+ }
+
+ if (dev_info->tqmap == NULL) {
+ dev_info->tqmap = rte_zmalloc_socket(adapter->mem_name,
+ dev_info->num_dma_dev_vchan *
+ sizeof(struct dma_vchan_info),
+ 0, adapter->socket_id);
+ if (dev_info->tqmap == NULL) {
+ printf("tq pair add not supported\n");
+ return -ENOMEM;
+ }
+ }
+
+ ret = (*dev->dev_ops->dma_adapter_vchan_add)(dev, dma_dev_id, vchan, event);
+ if (ret)
+ return ret;
+
+ else
+ edma_update_vchanq_info(adapter, &adapter->dma_devs[dma_dev_id], vchan, 1);
+ }
+
+ return 0;
+}
+
+int
+rte_event_dma_adapter_vchan_del(uint8_t id, int16_t dma_dev_id, uint16_t vchan)
+{
+ struct event_dma_adapter *adapter;
+ struct dma_device_info *dev_info;
+ struct rte_eventdev *dev;
+ uint32_t cap;
+ int ret;
+
+ EVENT_DMA_ADAPTER_ID_VALID_OR_ERR_RET(id, -EINVAL);
+
+ if (!rte_dma_is_valid(dma_dev_id)) {
+ RTE_EDEV_LOG_ERR("Invalid dma_dev_id = %" PRIu8, dma_dev_id);
+ return -EINVAL;
+ }
+
+ adapter = edma_id_to_adapter(id);
+ if (adapter == NULL)
+ return -EINVAL;
+
+ dev = &rte_eventdevs[adapter->eventdev_id];
+ ret = rte_event_dma_adapter_caps_get(adapter->eventdev_id, dma_dev_id, &cap);
+ if (ret)
+ return ret;
+
+ dev_info = &adapter->dma_devs[dma_dev_id];
+
+ if (vchan != RTE_DMA_ALL_VCHAN && vchan >= dev_info->num_dma_dev_vchan) {
+ RTE_EDEV_LOG_ERR("Invalid vhcan %" PRIu16, vchan);
+ return -EINVAL;
+ }
+
+ if ((cap & RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP_FWD) ||
+ (cap & RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP_NEW &&
+ adapter->mode == RTE_EVENT_DMA_ADAPTER_OP_NEW)) {
+ if (*dev->dev_ops->dma_adapter_vchan_del == NULL)
+ return -ENOTSUP;
+ ret = (*dev->dev_ops->dma_adapter_vchan_del)(dev, dma_dev_id, vchan);
+ if (ret == 0) {
+ edma_update_vchanq_info(adapter, dev_info, vchan, 0);
+ if (dev_info->num_vchanq == 0) {
+ rte_free(dev_info->vchanq);
+ dev_info->vchanq = NULL;
+ }
+ }
+ } else {
+ if (adapter->nb_vchanq == 0)
+ return 0;
+
+ rte_spinlock_lock(&adapter->lock);
+ edma_update_vchanq_info(adapter, dev_info, vchan, 0);
+
+ if (dev_info->num_vchanq == 0) {
+ rte_free(dev_info->vchanq);
+ rte_free(dev_info->tqmap);
+ dev_info->vchanq = NULL;
+ dev_info->tqmap = NULL;
+ }
+
+ rte_spinlock_unlock(&adapter->lock);
+ }
+
+ return ret;
+}
--
2.25.1
^ permalink raw reply [flat|nested] 110+ messages in thread
* [PATCH v8 05/12] eventdev/dma: support adapter service function
2023-09-29 11:50 ` [PATCH v8 00/12] event DMA adapter library support Amit Prakash Shukla
` (3 preceding siblings ...)
2023-09-29 11:50 ` [PATCH v8 04/12] eventdev/dma: support vchan add and delete Amit Prakash Shukla
@ 2023-09-29 11:50 ` Amit Prakash Shukla
2023-09-29 11:50 ` [PATCH v8 06/12] eventdev/dma: support adapter start and stop Amit Prakash Shukla
` (7 subsequent siblings)
12 siblings, 0 replies; 110+ messages in thread
From: Amit Prakash Shukla @ 2023-09-29 11:50 UTC (permalink / raw)
To: Amit Prakash Shukla, Jerin Jacob
Cc: dev, fengchengwen, kevin.laatz, bruce.richardson, conor.walsh,
vattunuru, g.singh, sachin.saxena, hemant.agrawal, cheng1.jiang,
ndabilpuram, anoobj, mb
Added support for DMA adapter service function for event devices.
Enqueue and dequeue of event from eventdev and DMA device are done
based on the adapter mode and the supported HW capabilities.
Signed-off-by: Amit Prakash Shukla <amitprakashs@marvell.com>
---
lib/eventdev/rte_event_dma_adapter.c | 592 +++++++++++++++++++++++++++
1 file changed, 592 insertions(+)
diff --git a/lib/eventdev/rte_event_dma_adapter.c b/lib/eventdev/rte_event_dma_adapter.c
index fa2e29b9d3..1d8bae0422 100644
--- a/lib/eventdev/rte_event_dma_adapter.c
+++ b/lib/eventdev/rte_event_dma_adapter.c
@@ -3,6 +3,7 @@
*/
#include <eventdev_pmd.h>
+#include <rte_service_component.h>
#include "rte_event_dma_adapter.h"
@@ -69,6 +70,10 @@ struct dma_device_info {
/* Number of vchans configured for a DMA device. */
uint16_t num_dma_dev_vchan;
+
+ /* Next queue pair to be processed */
+ uint16_t next_vchan_id;
+
} __rte_cache_aligned;
struct event_dma_adapter {
@@ -90,6 +95,9 @@ struct event_dma_adapter {
/* Lock to serialize config updates with service function */
rte_spinlock_t lock;
+ /* Next dma device to be processed */
+ uint16_t next_dmadev_id;
+
/* DMA device structure array */
struct dma_device_info *dma_devs;
@@ -107,6 +115,26 @@ struct event_dma_adapter {
/* No. of vchan queue configured */
uint16_t nb_vchanq;
+
+ /* Per adapter EAL service ID */
+ uint32_t service_id;
+
+ /* Service initialization state */
+ uint8_t service_initialized;
+
+ /* Max DMA ops processed in any service function invocation */
+ uint32_t max_nb;
+
+ /* Store event port's implicit release capability */
+ uint8_t implicit_release_disabled;
+
+ /* Flag to indicate backpressure at dma_dev
+ * Stop further dequeuing events from eventdev
+ */
+ bool stop_enq_to_dma_dev;
+
+ /* Loop counter to flush dma ops */
+ uint16_t transmit_loop_count;
} __rte_cache_aligned;
static struct event_dma_adapter **event_dma_adapter;
@@ -148,6 +176,18 @@ edma_array_init(void)
return 0;
}
+static inline bool
+edma_circular_buffer_batch_ready(struct dma_ops_circular_buffer *bufp)
+{
+ return bufp->count >= DMA_BATCH_SIZE;
+}
+
+static inline bool
+edma_circular_buffer_space_for_batch(struct dma_ops_circular_buffer *bufp)
+{
+ return (bufp->size - bufp->count) >= DMA_BATCH_SIZE;
+}
+
static inline int
edma_circular_buffer_init(const char *name, struct dma_ops_circular_buffer *buf, uint16_t sz)
{
@@ -166,6 +206,71 @@ edma_circular_buffer_free(struct dma_ops_circular_buffer *buf)
rte_free(buf->op_buffer);
}
+static inline int
+edma_circular_buffer_add(struct dma_ops_circular_buffer *bufp, struct rte_event_dma_adapter_op *op)
+{
+ uint16_t *tail = &bufp->tail;
+
+ bufp->op_buffer[*tail] = op;
+
+ /* circular buffer, go round */
+ *tail = (*tail + 1) % bufp->size;
+ bufp->count++;
+
+ return 0;
+}
+
+static inline int
+edma_circular_buffer_flush_to_dma_dev(struct event_dma_adapter *adapter,
+ struct dma_ops_circular_buffer *bufp, uint8_t dma_dev_id,
+ uint16_t vchan, uint16_t *nb_ops_flushed)
+{
+ struct rte_event_dma_adapter_op *op;
+ struct dma_vchan_info *tq;
+ uint16_t *head = &bufp->head;
+ uint16_t *tail = &bufp->tail;
+ uint16_t n;
+ uint16_t i;
+ int ret;
+
+ if (*tail > *head)
+ n = *tail - *head;
+ else if (*tail < *head)
+ n = bufp->size - *head;
+ else {
+ *nb_ops_flushed = 0;
+ return 0; /* buffer empty */
+ }
+
+ tq = &adapter->dma_devs[dma_dev_id].tqmap[vchan];
+
+ for (i = 0; i < n; i++) {
+ op = bufp->op_buffer[*head];
+ if (op->nb_src == 1 && op->nb_dst == 1)
+ ret = rte_dma_copy(dma_dev_id, vchan, op->src_seg->addr, op->dst_seg->addr,
+ op->src_seg->length, op->flags);
+ else
+ ret = rte_dma_copy_sg(dma_dev_id, vchan, op->src_seg, op->dst_seg,
+ op->nb_src, op->nb_dst, op->flags);
+ if (ret < 0)
+ break;
+
+ /* Enqueue in transaction queue. */
+ edma_circular_buffer_add(&tq->dma_buf, op);
+
+ *head = (*head + 1) % bufp->size;
+ }
+
+ *nb_ops_flushed = i;
+ bufp->count -= *nb_ops_flushed;
+ if (!bufp->count) {
+ *head = 0;
+ *tail = 0;
+ }
+
+ return *nb_ops_flushed == n ? 0 : -1;
+}
+
static int
edma_default_config_cb(uint8_t id, uint8_t evdev_id, struct rte_event_dma_adapter_conf *conf,
void *arg)
@@ -360,6 +465,406 @@ rte_event_dma_adapter_free(uint8_t id)
return 0;
}
+static inline unsigned int
+edma_enq_to_dma_dev(struct event_dma_adapter *adapter, struct rte_event *ev, unsigned int cnt)
+{
+ struct dma_vchan_info *vchan_qinfo = NULL;
+ struct rte_event_dma_adapter_op *dma_op;
+ uint16_t vchan, nb_enqueued = 0;
+ int16_t dma_dev_id;
+ unsigned int i, n;
+ int ret;
+
+ ret = 0;
+ n = 0;
+
+ for (i = 0; i < cnt; i++) {
+ dma_op = ev[i].event_ptr;
+ if (dma_op == NULL)
+ continue;
+
+ /* Expected to have response info appended to dma_op. */
+
+ dma_dev_id = dma_op->dma_dev_id;
+ vchan = dma_op->vchan;
+ vchan_qinfo = &adapter->dma_devs[dma_dev_id].vchanq[vchan];
+ if (!vchan_qinfo->vq_enabled) {
+ if (dma_op != NULL && dma_op->op_mp != NULL)
+ rte_mempool_put(dma_op->op_mp, dma_op);
+ continue;
+ }
+ edma_circular_buffer_add(&vchan_qinfo->dma_buf, dma_op);
+
+ if (edma_circular_buffer_batch_ready(&vchan_qinfo->dma_buf)) {
+ ret = edma_circular_buffer_flush_to_dma_dev(adapter, &vchan_qinfo->dma_buf,
+ dma_dev_id, vchan,
+ &nb_enqueued);
+ n += nb_enqueued;
+
+ /**
+ * If some dma ops failed to flush to dma_dev and
+ * space for another batch is not available, stop
+ * dequeue from eventdev momentarily
+ */
+ if (unlikely(ret < 0 &&
+ !edma_circular_buffer_space_for_batch(&vchan_qinfo->dma_buf)))
+ adapter->stop_enq_to_dma_dev = true;
+ }
+ }
+
+ return n;
+}
+
+static unsigned int
+edma_adapter_dev_flush(struct event_dma_adapter *adapter, int16_t dma_dev_id,
+ uint16_t *nb_ops_flushed)
+{
+ struct dma_vchan_info *vchan_info;
+ struct dma_device_info *dev_info;
+ uint16_t nb = 0, nb_enqueued = 0;
+ uint16_t vchan, nb_vchans;
+
+ dev_info = &adapter->dma_devs[dma_dev_id];
+ nb_vchans = dev_info->num_vchanq;
+
+ for (vchan = 0; vchan < nb_vchans; vchan++) {
+
+ vchan_info = &dev_info->vchanq[vchan];
+ if (unlikely(vchan_info == NULL || !vchan_info->vq_enabled))
+ continue;
+
+ edma_circular_buffer_flush_to_dma_dev(adapter, &vchan_info->dma_buf, dma_dev_id,
+ vchan, &nb_enqueued);
+ *nb_ops_flushed += vchan_info->dma_buf.count;
+ nb += nb_enqueued;
+ }
+
+ return nb;
+}
+
+static unsigned int
+edma_adapter_enq_flush(struct event_dma_adapter *adapter)
+{
+ int16_t dma_dev_id;
+ uint16_t nb_enqueued = 0;
+ uint16_t nb_ops_flushed = 0;
+ uint16_t num_dma_dev = rte_dma_count_avail();
+
+ for (dma_dev_id = 0; dma_dev_id < num_dma_dev; dma_dev_id++)
+ nb_enqueued += edma_adapter_dev_flush(adapter, dma_dev_id, &nb_ops_flushed);
+ /**
+ * Enable dequeue from eventdev if all ops from circular
+ * buffer flushed to dma_dev
+ */
+ if (!nb_ops_flushed)
+ adapter->stop_enq_to_dma_dev = false;
+
+ return nb_enqueued;
+}
+
+/* Flush an instance's enqueue buffers every DMA_ENQ_FLUSH_THRESHOLD
+ * iterations of edma_adapter_enq_run()
+ */
+#define DMA_ENQ_FLUSH_THRESHOLD 1024
+
+static int
+edma_adapter_enq_run(struct event_dma_adapter *adapter, unsigned int max_enq)
+{
+ uint8_t event_port_id = adapter->event_port_id;
+ uint8_t event_dev_id = adapter->eventdev_id;
+ struct rte_event ev[DMA_BATCH_SIZE];
+ unsigned int nb_enq, nb_enqueued;
+ uint16_t n;
+
+ if (adapter->mode == RTE_EVENT_DMA_ADAPTER_OP_NEW)
+ return 0;
+
+ nb_enqueued = 0;
+ for (nb_enq = 0; nb_enq < max_enq; nb_enq += n) {
+
+ if (unlikely(adapter->stop_enq_to_dma_dev)) {
+ nb_enqueued += edma_adapter_enq_flush(adapter);
+
+ if (unlikely(adapter->stop_enq_to_dma_dev))
+ break;
+ }
+
+ n = rte_event_dequeue_burst(event_dev_id, event_port_id, ev, DMA_BATCH_SIZE, 0);
+
+ if (!n)
+ break;
+
+ nb_enqueued += edma_enq_to_dma_dev(adapter, ev, n);
+ }
+
+ if ((++adapter->transmit_loop_count & (DMA_ENQ_FLUSH_THRESHOLD - 1)) == 0)
+ nb_enqueued += edma_adapter_enq_flush(adapter);
+
+ return nb_enqueued;
+}
+
+#define DMA_ADAPTER_MAX_EV_ENQ_RETRIES 100
+
+static inline uint16_t
+edma_ops_enqueue_burst(struct event_dma_adapter *adapter, struct rte_event_dma_adapter_op **ops,
+ uint16_t num)
+{
+ uint8_t event_port_id = adapter->event_port_id;
+ uint8_t event_dev_id = adapter->eventdev_id;
+ struct rte_event events[DMA_BATCH_SIZE];
+ struct rte_event *response_info;
+ uint16_t nb_enqueued, nb_ev;
+ uint8_t retry;
+ uint8_t i;
+
+ nb_ev = 0;
+ retry = 0;
+ nb_enqueued = 0;
+ num = RTE_MIN(num, DMA_BATCH_SIZE);
+ for (i = 0; i < num; i++) {
+ struct rte_event *ev = &events[nb_ev++];
+
+ /* Expected to have response info appended to dma_op. */
+ response_info = (struct rte_event *)((uint8_t *)ops[i] +
+ sizeof(struct rte_event_dma_adapter_op));
+ if (unlikely(response_info == NULL)) {
+ if (ops[i] != NULL && ops[i]->op_mp != NULL)
+ rte_mempool_put(ops[i]->op_mp, ops[i]);
+ continue;
+ }
+
+ rte_memcpy(ev, response_info, sizeof(struct rte_event));
+ ev->event_ptr = ops[i];
+ ev->event_type = RTE_EVENT_TYPE_DMADEV;
+ if (adapter->implicit_release_disabled)
+ ev->op = RTE_EVENT_OP_FORWARD;
+ else
+ ev->op = RTE_EVENT_OP_NEW;
+ }
+
+ do {
+ nb_enqueued += rte_event_enqueue_burst(event_dev_id, event_port_id,
+ &events[nb_enqueued], nb_ev - nb_enqueued);
+
+ } while (retry++ < DMA_ADAPTER_MAX_EV_ENQ_RETRIES && nb_enqueued < nb_ev);
+
+ return nb_enqueued;
+}
+
+static int
+edma_circular_buffer_flush_to_evdev(struct event_dma_adapter *adapter,
+ struct dma_ops_circular_buffer *bufp,
+ uint16_t *enqueue_count)
+{
+ struct rte_event_dma_adapter_op **ops = bufp->op_buffer;
+ uint16_t n = 0, nb_ops_flushed;
+ uint16_t *head = &bufp->head;
+ uint16_t *tail = &bufp->tail;
+
+ if (*tail > *head)
+ n = *tail - *head;
+ else if (*tail < *head)
+ n = bufp->size - *head;
+ else {
+ if (enqueue_count)
+ *enqueue_count = 0;
+ return 0; /* buffer empty */
+ }
+
+ if (enqueue_count && n > *enqueue_count)
+ n = *enqueue_count;
+
+ nb_ops_flushed = edma_ops_enqueue_burst(adapter, &ops[*head], n);
+ if (enqueue_count)
+ *enqueue_count = nb_ops_flushed;
+
+ bufp->count -= nb_ops_flushed;
+ if (!bufp->count) {
+ *head = 0;
+ *tail = 0;
+ return 0; /* buffer empty */
+ }
+
+ *head = (*head + nb_ops_flushed) % bufp->size;
+ return 1;
+}
+
+static void
+edma_ops_buffer_flush(struct event_dma_adapter *adapter)
+{
+ if (likely(adapter->ebuf.count == 0))
+ return;
+
+ while (edma_circular_buffer_flush_to_evdev(adapter, &adapter->ebuf, NULL))
+ ;
+}
+
+static inline unsigned int
+edma_adapter_deq_run(struct event_dma_adapter *adapter, unsigned int max_deq)
+{
+ struct dma_vchan_info *vchan_info;
+ struct dma_ops_circular_buffer *tq_buf;
+ struct rte_event_dma_adapter_op *ops;
+ uint16_t n, nb_deq, nb_enqueued, i;
+ struct dma_device_info *dev_info;
+ uint16_t vchan, num_vchan;
+ uint16_t num_dma_dev;
+ int16_t dma_dev_id;
+ uint16_t index;
+ bool done;
+ bool err;
+
+ nb_deq = 0;
+ edma_ops_buffer_flush(adapter);
+
+ num_dma_dev = rte_dma_count_avail();
+ do {
+ done = true;
+
+ for (dma_dev_id = adapter->next_dmadev_id; dma_dev_id < num_dma_dev; dma_dev_id++) {
+ uint16_t queues = 0;
+ dev_info = &adapter->dma_devs[dma_dev_id];
+ num_vchan = dev_info->num_vchanq;
+
+ for (vchan = dev_info->next_vchan_id; queues < num_vchan;
+ vchan = (vchan + 1) % num_vchan, queues++) {
+
+ vchan_info = &dev_info->vchanq[vchan];
+ if (unlikely(vchan_info == NULL || !vchan_info->vq_enabled))
+ continue;
+
+ n = rte_dma_completed(dma_dev_id, vchan, DMA_BATCH_SIZE,
+ &index, &err);
+ if (!n)
+ continue;
+
+ done = false;
+
+ tq_buf = &dev_info->tqmap[vchan].dma_buf;
+
+ nb_enqueued = n;
+ if (unlikely(!adapter->ebuf.count))
+ edma_circular_buffer_flush_to_evdev(adapter, tq_buf,
+ &nb_enqueued);
+
+ if (likely(nb_enqueued == n))
+ goto check;
+
+ /* Failed to enqueue events case */
+ for (i = nb_enqueued; i < n; i++) {
+ ops = tq_buf->op_buffer[tq_buf->head];
+ edma_circular_buffer_add(&adapter->ebuf, ops);
+ tq_buf->head = (tq_buf->head + 1) % tq_buf->size;
+ }
+
+check:
+ nb_deq += n;
+ if (nb_deq >= max_deq) {
+ if ((vchan + 1) == num_vchan)
+ adapter->next_dmadev_id =
+ (dma_dev_id + 1) % num_dma_dev;
+
+ dev_info->next_vchan_id = (vchan + 1) % num_vchan;
+
+ return nb_deq;
+ }
+ }
+ }
+ adapter->next_dmadev_id = 0;
+
+ } while (done == false);
+
+ return nb_deq;
+}
+
+static int
+edma_adapter_run(struct event_dma_adapter *adapter, unsigned int max_ops)
+{
+ unsigned int ops_left = max_ops;
+
+ while (ops_left > 0) {
+ unsigned int e_cnt, d_cnt;
+
+ e_cnt = edma_adapter_deq_run(adapter, ops_left);
+ ops_left -= RTE_MIN(ops_left, e_cnt);
+
+ d_cnt = edma_adapter_enq_run(adapter, ops_left);
+ ops_left -= RTE_MIN(ops_left, d_cnt);
+
+ if (e_cnt == 0 && d_cnt == 0)
+ break;
+ }
+
+ if (ops_left == max_ops) {
+ rte_event_maintain(adapter->eventdev_id, adapter->event_port_id, 0);
+ return -EAGAIN;
+ } else
+ return 0;
+}
+
+static int
+edma_service_func(void *args)
+{
+ struct event_dma_adapter *adapter = args;
+ int ret;
+
+ if (rte_spinlock_trylock(&adapter->lock) == 0)
+ return 0;
+ ret = edma_adapter_run(adapter, adapter->max_nb);
+ rte_spinlock_unlock(&adapter->lock);
+
+ return ret;
+}
+
+static int
+edma_init_service(struct event_dma_adapter *adapter, uint8_t id)
+{
+ struct rte_event_dma_adapter_conf adapter_conf;
+ struct rte_service_spec service;
+ uint32_t impl_rel;
+ int ret;
+
+ if (adapter->service_initialized)
+ return 0;
+
+ memset(&service, 0, sizeof(service));
+ snprintf(service.name, DMA_ADAPTER_NAME_LEN, "rte_event_dma_adapter_%d", id);
+ service.socket_id = adapter->socket_id;
+ service.callback = edma_service_func;
+ service.callback_userdata = adapter;
+
+ /* Service function handles locking for queue add/del updates */
+ service.capabilities = RTE_SERVICE_CAP_MT_SAFE;
+ ret = rte_service_component_register(&service, &adapter->service_id);
+ if (ret) {
+ RTE_EDEV_LOG_ERR("failed to register service %s err = %" PRId32, service.name, ret);
+ return ret;
+ }
+
+ ret = adapter->conf_cb(id, adapter->eventdev_id, &adapter_conf, adapter->conf_arg);
+ if (ret) {
+ RTE_EDEV_LOG_ERR("configuration callback failed err = %" PRId32, ret);
+ return ret;
+ }
+
+ adapter->max_nb = adapter_conf.max_nb;
+ adapter->event_port_id = adapter_conf.event_port_id;
+
+ if (rte_event_port_attr_get(adapter->eventdev_id, adapter->event_port_id,
+ RTE_EVENT_PORT_ATTR_IMPLICIT_RELEASE_DISABLE, &impl_rel)) {
+ RTE_EDEV_LOG_ERR("Failed to get port info for eventdev %" PRId32,
+ adapter->eventdev_id);
+ edma_circular_buffer_free(&adapter->ebuf);
+ rte_free(adapter);
+ return -EINVAL;
+ }
+
+ adapter->implicit_release_disabled = (uint8_t)impl_rel;
+ adapter->service_initialized = 1;
+
+ return ret;
+}
+
static void
edma_update_vchanq_info(struct event_dma_adapter *adapter, struct dma_device_info *dev_info,
uint16_t vchan, uint8_t add)
@@ -391,6 +896,60 @@ edma_update_vchanq_info(struct event_dma_adapter *adapter, struct dma_device_inf
}
}
+static int
+edma_add_vchan(struct event_dma_adapter *adapter, int16_t dma_dev_id, uint16_t vchan)
+{
+ struct dma_device_info *dev_info = &adapter->dma_devs[dma_dev_id];
+ struct dma_vchan_info *vchanq;
+ struct dma_vchan_info *tqmap;
+ uint16_t nb_vchans;
+ uint32_t i;
+
+ if (dev_info->vchanq == NULL) {
+ nb_vchans = dev_info->num_dma_dev_vchan;
+
+ dev_info->vchanq = rte_zmalloc_socket(adapter->mem_name,
+ nb_vchans * sizeof(struct dma_vchan_info),
+ 0, adapter->socket_id);
+ if (dev_info->vchanq == NULL)
+ return -ENOMEM;
+
+ dev_info->tqmap = rte_zmalloc_socket(adapter->mem_name,
+ nb_vchans * sizeof(struct dma_vchan_info),
+ 0, adapter->socket_id);
+ if (dev_info->tqmap == NULL)
+ return -ENOMEM;
+
+ for (i = 0; i < nb_vchans; i++) {
+ vchanq = &dev_info->vchanq[i];
+
+ if (edma_circular_buffer_init("dma_dev_circular_buffer", &vchanq->dma_buf,
+ DMA_ADAPTER_OPS_BUFFER_SIZE)) {
+ RTE_EDEV_LOG_ERR("Failed to get memory for dma_dev buffer");
+ rte_free(vchanq);
+ return -ENOMEM;
+ }
+
+ tqmap = &dev_info->tqmap[i];
+ if (edma_circular_buffer_init("dma_dev_circular_trans_buf", &tqmap->dma_buf,
+ DMA_ADAPTER_OPS_BUFFER_SIZE)) {
+ RTE_EDEV_LOG_ERR(
+ "Failed to get memory for dma_dev transaction buffer");
+ rte_free(tqmap);
+ return -ENOMEM;
+ }
+ }
+ }
+
+ if (vchan == RTE_DMA_ALL_VCHAN) {
+ for (i = 0; i < dev_info->num_dma_dev_vchan; i++)
+ edma_update_vchanq_info(adapter, dev_info, i, 1);
+ } else
+ edma_update_vchanq_info(adapter, dev_info, vchan, 1);
+
+ return 0;
+}
+
int
rte_event_dma_adapter_vchan_add(uint8_t id, int16_t dma_dev_id, uint16_t vchan,
const struct rte_event *event)
@@ -470,6 +1029,38 @@ rte_event_dma_adapter_vchan_add(uint8_t id, int16_t dma_dev_id, uint16_t vchan,
edma_update_vchanq_info(adapter, &adapter->dma_devs[dma_dev_id], vchan, 1);
}
+ /* In case HW cap is RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP_NEW, or SW adapter, initiate
+ * services so the application can choose which ever way it wants to use the adapter.
+ *
+ * Case 1: RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP_NEW. Application may wants to use one
+ * of below two modes
+ *
+ * a. OP_FORWARD mode -> HW Dequeue + SW enqueue
+ * b. OP_NEW mode -> HW Dequeue
+ *
+ * Case 2: No HW caps, use SW adapter
+ *
+ * a. OP_FORWARD mode -> SW enqueue & dequeue
+ * b. OP_NEW mode -> SW Dequeue
+ */
+ if ((cap & RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP_NEW &&
+ !(cap & RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP_FWD) &&
+ adapter->mode == RTE_EVENT_DMA_ADAPTER_OP_FORWARD) ||
+ (!(cap & RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP_NEW) &&
+ !(cap & RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP_FWD) &&
+ !(cap & RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_VCHAN_EV_BIND))) {
+ rte_spinlock_lock(&adapter->lock);
+ ret = edma_init_service(adapter, id);
+ if (ret == 0)
+ ret = edma_add_vchan(adapter, dma_dev_id, vchan);
+ rte_spinlock_unlock(&adapter->lock);
+
+ if (ret)
+ return ret;
+
+ rte_service_component_runstate_set(adapter->service_id, 1);
+ }
+
return 0;
}
@@ -533,6 +1124,7 @@ rte_event_dma_adapter_vchan_del(uint8_t id, int16_t dma_dev_id, uint16_t vchan)
}
rte_spinlock_unlock(&adapter->lock);
+ rte_service_component_runstate_set(adapter->service_id, adapter->nb_vchanq);
}
return ret;
--
2.25.1
^ permalink raw reply [flat|nested] 110+ messages in thread
* [PATCH v8 06/12] eventdev/dma: support adapter start and stop
2023-09-29 11:50 ` [PATCH v8 00/12] event DMA adapter library support Amit Prakash Shukla
` (4 preceding siblings ...)
2023-09-29 11:50 ` [PATCH v8 05/12] eventdev/dma: support adapter service function Amit Prakash Shukla
@ 2023-09-29 11:50 ` Amit Prakash Shukla
2023-09-29 11:50 ` [PATCH v8 07/12] eventdev/dma: support adapter service ID get Amit Prakash Shukla
` (6 subsequent siblings)
12 siblings, 0 replies; 110+ messages in thread
From: Amit Prakash Shukla @ 2023-09-29 11:50 UTC (permalink / raw)
To: Amit Prakash Shukla, Jerin Jacob
Cc: dev, fengchengwen, kevin.laatz, bruce.richardson, conor.walsh,
vattunuru, g.singh, sachin.saxena, hemant.agrawal, cheng1.jiang,
ndabilpuram, anoobj, mb
Added API support to start and stop DMA adapter.
Signed-off-by: Amit Prakash Shukla <amitprakashs@marvell.com>
---
lib/eventdev/rte_event_dma_adapter.c | 69 ++++++++++++++++++++++++++++
1 file changed, 69 insertions(+)
diff --git a/lib/eventdev/rte_event_dma_adapter.c b/lib/eventdev/rte_event_dma_adapter.c
index 1d8bae0422..be6c2623e9 100644
--- a/lib/eventdev/rte_event_dma_adapter.c
+++ b/lib/eventdev/rte_event_dma_adapter.c
@@ -74,6 +74,13 @@ struct dma_device_info {
/* Next queue pair to be processed */
uint16_t next_vchan_id;
+ /* Set to indicate processing has been started */
+ uint8_t dev_started;
+
+ /* Set to indicate dmadev->eventdev packet
+ * transfer uses a hardware mechanism
+ */
+ uint8_t internal_event_port;
} __rte_cache_aligned;
struct event_dma_adapter {
@@ -1129,3 +1136,65 @@ rte_event_dma_adapter_vchan_del(uint8_t id, int16_t dma_dev_id, uint16_t vchan)
return ret;
}
+
+static int
+edma_adapter_ctrl(uint8_t id, int start)
+{
+ struct event_dma_adapter *adapter;
+ struct dma_device_info *dev_info;
+ struct rte_eventdev *dev;
+ uint16_t num_dma_dev;
+ int stop = !start;
+ int use_service;
+ uint32_t i;
+
+ use_service = 0;
+ EVENT_DMA_ADAPTER_ID_VALID_OR_ERR_RET(id, -EINVAL);
+ adapter = edma_id_to_adapter(id);
+ if (adapter == NULL)
+ return -EINVAL;
+
+ num_dma_dev = rte_dma_count_avail();
+ dev = &rte_eventdevs[adapter->eventdev_id];
+
+ for (i = 0; i < num_dma_dev; i++) {
+ dev_info = &adapter->dma_devs[i];
+ /* start check for num queue pairs */
+ if (start && !dev_info->num_vchanq)
+ continue;
+ /* stop check if dev has been started */
+ if (stop && !dev_info->dev_started)
+ continue;
+ use_service |= !dev_info->internal_event_port;
+ dev_info->dev_started = start;
+ if (dev_info->internal_event_port == 0)
+ continue;
+ start ? (*dev->dev_ops->dma_adapter_start)(dev, i) :
+ (*dev->dev_ops->dma_adapter_stop)(dev, i);
+ }
+
+ if (use_service)
+ rte_service_runstate_set(adapter->service_id, start);
+
+ return 0;
+}
+
+int
+rte_event_dma_adapter_start(uint8_t id)
+{
+ struct event_dma_adapter *adapter;
+
+ EVENT_DMA_ADAPTER_ID_VALID_OR_ERR_RET(id, -EINVAL);
+
+ adapter = edma_id_to_adapter(id);
+ if (adapter == NULL)
+ return -EINVAL;
+
+ return edma_adapter_ctrl(id, 1);
+}
+
+int
+rte_event_dma_adapter_stop(uint8_t id)
+{
+ return edma_adapter_ctrl(id, 0);
+}
--
2.25.1
^ permalink raw reply [flat|nested] 110+ messages in thread
* [PATCH v8 07/12] eventdev/dma: support adapter service ID get
2023-09-29 11:50 ` [PATCH v8 00/12] event DMA adapter library support Amit Prakash Shukla
` (5 preceding siblings ...)
2023-09-29 11:50 ` [PATCH v8 06/12] eventdev/dma: support adapter start and stop Amit Prakash Shukla
@ 2023-09-29 11:50 ` Amit Prakash Shukla
2023-09-29 11:50 ` [PATCH v8 08/12] eventdev/dma: support adapter runtime params Amit Prakash Shukla
` (5 subsequent siblings)
12 siblings, 0 replies; 110+ messages in thread
From: Amit Prakash Shukla @ 2023-09-29 11:50 UTC (permalink / raw)
To: Amit Prakash Shukla, Jerin Jacob
Cc: dev, fengchengwen, kevin.laatz, bruce.richardson, conor.walsh,
vattunuru, g.singh, sachin.saxena, hemant.agrawal, cheng1.jiang,
ndabilpuram, anoobj, mb
Added API support to get DMA adapter service ID. Service id
returned in the variable by the API call shall be used by application
to map a service core.
Signed-off-by: Amit Prakash Shukla <amitprakashs@marvell.com>
---
lib/eventdev/rte_event_dma_adapter.c | 17 +++++++++++++++++
1 file changed, 17 insertions(+)
diff --git a/lib/eventdev/rte_event_dma_adapter.c b/lib/eventdev/rte_event_dma_adapter.c
index be6c2623e9..c3b014aaf9 100644
--- a/lib/eventdev/rte_event_dma_adapter.c
+++ b/lib/eventdev/rte_event_dma_adapter.c
@@ -1137,6 +1137,23 @@ rte_event_dma_adapter_vchan_del(uint8_t id, int16_t dma_dev_id, uint16_t vchan)
return ret;
}
+int
+rte_event_dma_adapter_service_id_get(uint8_t id, uint32_t *service_id)
+{
+ struct event_dma_adapter *adapter;
+
+ EVENT_DMA_ADAPTER_ID_VALID_OR_ERR_RET(id, -EINVAL);
+
+ adapter = edma_id_to_adapter(id);
+ if (adapter == NULL || service_id == NULL)
+ return -EINVAL;
+
+ if (adapter->service_initialized)
+ *service_id = adapter->service_id;
+
+ return adapter->service_initialized ? 0 : -ESRCH;
+}
+
static int
edma_adapter_ctrl(uint8_t id, int start)
{
--
2.25.1
^ permalink raw reply [flat|nested] 110+ messages in thread
* [PATCH v8 08/12] eventdev/dma: support adapter runtime params
2023-09-29 11:50 ` [PATCH v8 00/12] event DMA adapter library support Amit Prakash Shukla
` (6 preceding siblings ...)
2023-09-29 11:50 ` [PATCH v8 07/12] eventdev/dma: support adapter service ID get Amit Prakash Shukla
@ 2023-09-29 11:50 ` Amit Prakash Shukla
2023-09-29 11:50 ` [PATCH v8 09/12] eventdev/dma: support adapter stats Amit Prakash Shukla
` (4 subsequent siblings)
12 siblings, 0 replies; 110+ messages in thread
From: Amit Prakash Shukla @ 2023-09-29 11:50 UTC (permalink / raw)
To: Amit Prakash Shukla, Jerin Jacob
Cc: dev, fengchengwen, kevin.laatz, bruce.richardson, conor.walsh,
vattunuru, g.singh, sachin.saxena, hemant.agrawal, cheng1.jiang,
ndabilpuram, anoobj, mb
Added support to set and get runtime params for DMA adapter. The
parameters that can be set/get are defined in
struct rte_event_dma_adapter_runtime_params.
Signed-off-by: Amit Prakash Shukla <amitprakashs@marvell.com>
---
lib/eventdev/rte_event_dma_adapter.c | 93 ++++++++++++++++++++++++++++
1 file changed, 93 insertions(+)
diff --git a/lib/eventdev/rte_event_dma_adapter.c b/lib/eventdev/rte_event_dma_adapter.c
index c3b014aaf9..632169a7c2 100644
--- a/lib/eventdev/rte_event_dma_adapter.c
+++ b/lib/eventdev/rte_event_dma_adapter.c
@@ -1215,3 +1215,96 @@ rte_event_dma_adapter_stop(uint8_t id)
{
return edma_adapter_ctrl(id, 0);
}
+
+#define DEFAULT_MAX_NB 128
+
+int
+rte_event_dma_adapter_runtime_params_init(struct rte_event_dma_adapter_runtime_params *params)
+{
+ if (params == NULL)
+ return -EINVAL;
+
+ memset(params, 0, sizeof(*params));
+ params->max_nb = DEFAULT_MAX_NB;
+
+ return 0;
+}
+
+static int
+dma_adapter_cap_check(struct event_dma_adapter *adapter)
+{
+ uint32_t caps;
+ int ret;
+
+ if (!adapter->nb_vchanq)
+ return -EINVAL;
+
+ ret = rte_event_dma_adapter_caps_get(adapter->eventdev_id, adapter->next_dmadev_id, &caps);
+ if (ret) {
+ RTE_EDEV_LOG_ERR("Failed to get adapter caps dev %" PRIu8 " cdev %" PRIu8,
+ adapter->eventdev_id, adapter->next_dmadev_id);
+ return ret;
+ }
+
+ if ((caps & RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP_FWD) ||
+ (caps & RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP_NEW))
+ return -ENOTSUP;
+
+ return 0;
+}
+
+int
+rte_event_dma_adapter_runtime_params_set(uint8_t id,
+ struct rte_event_dma_adapter_runtime_params *params)
+{
+ struct event_dma_adapter *adapter;
+ int ret;
+
+ EVENT_DMA_ADAPTER_ID_VALID_OR_ERR_RET(id, -EINVAL);
+
+ if (params == NULL) {
+ RTE_EDEV_LOG_ERR("params pointer is NULL\n");
+ return -EINVAL;
+ }
+
+ adapter = edma_id_to_adapter(id);
+ if (adapter == NULL)
+ return -EINVAL;
+
+ ret = dma_adapter_cap_check(adapter);
+ if (ret)
+ return ret;
+
+ rte_spinlock_lock(&adapter->lock);
+ adapter->max_nb = params->max_nb;
+ rte_spinlock_unlock(&adapter->lock);
+
+ return 0;
+}
+
+int
+rte_event_dma_adapter_runtime_params_get(uint8_t id,
+ struct rte_event_dma_adapter_runtime_params *params)
+{
+ struct event_dma_adapter *adapter;
+ int ret;
+
+ EVENT_DMA_ADAPTER_ID_VALID_OR_ERR_RET(id, -EINVAL);
+
+ if (params == NULL) {
+ RTE_EDEV_LOG_ERR("params pointer is NULL\n");
+ return -EINVAL;
+ }
+
+ adapter = edma_id_to_adapter(id);
+ if (adapter == NULL)
+ return -EINVAL;
+
+ ret = dma_adapter_cap_check(adapter);
+ if (ret)
+ return ret;
+
+ params->max_nb = adapter->max_nb;
+
+ return 0;
+}
--
2.25.1
^ permalink raw reply [flat|nested] 110+ messages in thread
* [PATCH v8 09/12] eventdev/dma: support adapter stats
2023-09-29 11:50 ` [PATCH v8 00/12] event DMA adapter library support Amit Prakash Shukla
` (7 preceding siblings ...)
2023-09-29 11:50 ` [PATCH v8 08/12] eventdev/dma: support adapter runtime params Amit Prakash Shukla
@ 2023-09-29 11:50 ` Amit Prakash Shukla
2023-09-29 11:50 ` [PATCH v8 10/12] eventdev/dma: support adapter enqueue Amit Prakash Shukla
` (3 subsequent siblings)
12 siblings, 0 replies; 110+ messages in thread
From: Amit Prakash Shukla @ 2023-09-29 11:50 UTC (permalink / raw)
To: Amit Prakash Shukla, Jerin Jacob
Cc: dev, fengchengwen, kevin.laatz, bruce.richardson, conor.walsh,
vattunuru, g.singh, sachin.saxena, hemant.agrawal, cheng1.jiang,
ndabilpuram, anoobj, mb
Added DMA adapter stats API support to get and reset stats. DMA
SW adapter stats and eventdev driver supported stats for enqueue
and dequeue are reported by get API.
Signed-off-by: Amit Prakash Shukla <amitprakashs@marvell.com>
---
lib/eventdev/rte_event_dma_adapter.c | 95 ++++++++++++++++++++++++++++
1 file changed, 95 insertions(+)
diff --git a/lib/eventdev/rte_event_dma_adapter.c b/lib/eventdev/rte_event_dma_adapter.c
index 632169a7c2..6c67e6d499 100644
--- a/lib/eventdev/rte_event_dma_adapter.c
+++ b/lib/eventdev/rte_event_dma_adapter.c
@@ -142,6 +142,9 @@ struct event_dma_adapter {
/* Loop counter to flush dma ops */
uint16_t transmit_loop_count;
+
+ /* Per instance stats structure */
+ struct rte_event_dma_adapter_stats dma_stats;
} __rte_cache_aligned;
static struct event_dma_adapter **event_dma_adapter;
@@ -475,6 +478,7 @@ rte_event_dma_adapter_free(uint8_t id)
static inline unsigned int
edma_enq_to_dma_dev(struct event_dma_adapter *adapter, struct rte_event *ev, unsigned int cnt)
{
+ struct rte_event_dma_adapter_stats *stats = &adapter->dma_stats;
struct dma_vchan_info *vchan_qinfo = NULL;
struct rte_event_dma_adapter_op *dma_op;
uint16_t vchan, nb_enqueued = 0;
@@ -484,6 +488,7 @@ edma_enq_to_dma_dev(struct event_dma_adapter *adapter, struct rte_event *ev, uns
ret = 0;
n = 0;
+ stats->event_deq_count += cnt;
for (i = 0; i < cnt; i++) {
dma_op = ev[i].event_ptr;
@@ -506,6 +511,7 @@ edma_enq_to_dma_dev(struct event_dma_adapter *adapter, struct rte_event *ev, uns
ret = edma_circular_buffer_flush_to_dma_dev(adapter, &vchan_qinfo->dma_buf,
dma_dev_id, vchan,
&nb_enqueued);
+ stats->dma_enq_count += nb_enqueued;
n += nb_enqueued;
/**
@@ -552,6 +558,7 @@ edma_adapter_dev_flush(struct event_dma_adapter *adapter, int16_t dma_dev_id,
static unsigned int
edma_adapter_enq_flush(struct event_dma_adapter *adapter)
{
+ struct rte_event_dma_adapter_stats *stats = &adapter->dma_stats;
int16_t dma_dev_id;
uint16_t nb_enqueued = 0;
uint16_t nb_ops_flushed = 0;
@@ -566,6 +573,8 @@ edma_adapter_enq_flush(struct event_dma_adapter *adapter)
if (!nb_ops_flushed)
adapter->stop_enq_to_dma_dev = false;
+ stats->dma_enq_count += nb_enqueued;
+
return nb_enqueued;
}
@@ -577,6 +586,7 @@ edma_adapter_enq_flush(struct event_dma_adapter *adapter)
static int
edma_adapter_enq_run(struct event_dma_adapter *adapter, unsigned int max_enq)
{
+ struct rte_event_dma_adapter_stats *stats = &adapter->dma_stats;
uint8_t event_port_id = adapter->event_port_id;
uint8_t event_dev_id = adapter->eventdev_id;
struct rte_event ev[DMA_BATCH_SIZE];
@@ -596,6 +606,7 @@ edma_adapter_enq_run(struct event_dma_adapter *adapter, unsigned int max_enq)
break;
}
+ stats->event_poll_count++;
n = rte_event_dequeue_burst(event_dev_id, event_port_id, ev, DMA_BATCH_SIZE, 0);
if (!n)
@@ -616,6 +627,7 @@ static inline uint16_t
edma_ops_enqueue_burst(struct event_dma_adapter *adapter, struct rte_event_dma_adapter_op **ops,
uint16_t num)
{
+ struct rte_event_dma_adapter_stats *stats = &adapter->dma_stats;
uint8_t event_port_id = adapter->event_port_id;
uint8_t event_dev_id = adapter->eventdev_id;
struct rte_event events[DMA_BATCH_SIZE];
@@ -655,6 +667,10 @@ edma_ops_enqueue_burst(struct event_dma_adapter *adapter, struct rte_event_dma_a
} while (retry++ < DMA_ADAPTER_MAX_EV_ENQ_RETRIES && nb_enqueued < nb_ev);
+ stats->event_enq_fail_count += nb_ev - nb_enqueued;
+ stats->event_enq_count += nb_enqueued;
+ stats->event_enq_retry_count += retry - 1;
+
return nb_enqueued;
}
@@ -709,6 +725,7 @@ edma_ops_buffer_flush(struct event_dma_adapter *adapter)
static inline unsigned int
edma_adapter_deq_run(struct event_dma_adapter *adapter, unsigned int max_deq)
{
+ struct rte_event_dma_adapter_stats *stats = &adapter->dma_stats;
struct dma_vchan_info *vchan_info;
struct dma_ops_circular_buffer *tq_buf;
struct rte_event_dma_adapter_op *ops;
@@ -746,6 +763,7 @@ edma_adapter_deq_run(struct event_dma_adapter *adapter, unsigned int max_deq)
continue;
done = false;
+ stats->dma_deq_count += n;
tq_buf = &dev_info->tqmap[vchan].dma_buf;
@@ -1308,3 +1326,80 @@ rte_event_dma_adapter_runtime_params_get(uint8_t id,
return 0;
}
+
+int
+rte_event_dma_adapter_stats_get(uint8_t id, struct rte_event_dma_adapter_stats *stats)
+{
+ struct rte_event_dma_adapter_stats dev_stats_sum = {0};
+ struct rte_event_dma_adapter_stats dev_stats;
+ struct event_dma_adapter *adapter;
+ struct dma_device_info *dev_info;
+ struct rte_eventdev *dev;
+ uint16_t num_dma_dev;
+ uint32_t i;
+ int ret;
+
+ EVENT_DMA_ADAPTER_ID_VALID_OR_ERR_RET(id, -EINVAL);
+
+ adapter = edma_id_to_adapter(id);
+ if (adapter == NULL || stats == NULL)
+ return -EINVAL;
+
+ num_dma_dev = rte_dma_count_avail();
+ dev = &rte_eventdevs[adapter->eventdev_id];
+ memset(stats, 0, sizeof(*stats));
+ for (i = 0; i < num_dma_dev; i++) {
+ dev_info = &adapter->dma_devs[i];
+
+ if (dev_info->internal_event_port == 0 ||
+ dev->dev_ops->dma_adapter_stats_get == NULL)
+ continue;
+
+ ret = (*dev->dev_ops->dma_adapter_stats_get)(dev, i, &dev_stats);
+ if (ret)
+ continue;
+
+ dev_stats_sum.dma_deq_count += dev_stats.dma_deq_count;
+ dev_stats_sum.event_enq_count += dev_stats.event_enq_count;
+ }
+
+ if (adapter->service_initialized)
+ *stats = adapter->dma_stats;
+
+ stats->dma_deq_count += dev_stats_sum.dma_deq_count;
+ stats->event_enq_count += dev_stats_sum.event_enq_count;
+
+ return 0;
+}
+
+int
+rte_event_dma_adapter_stats_reset(uint8_t id)
+{
+ struct event_dma_adapter *adapter;
+ struct dma_device_info *dev_info;
+ struct rte_eventdev *dev;
+ uint16_t num_dma_dev;
+ uint32_t i;
+
+ EVENT_DMA_ADAPTER_ID_VALID_OR_ERR_RET(id, -EINVAL);
+
+ adapter = edma_id_to_adapter(id);
+ if (adapter == NULL)
+ return -EINVAL;
+
+ num_dma_dev = rte_dma_count_avail();
+ dev = &rte_eventdevs[adapter->eventdev_id];
+ for (i = 0; i < num_dma_dev; i++) {
+ dev_info = &adapter->dma_devs[i];
+
+ if (dev_info->internal_event_port == 0 ||
+ dev->dev_ops->dma_adapter_stats_reset == NULL)
+ continue;
+
+ (*dev->dev_ops->dma_adapter_stats_reset)(dev, i);
+ }
+
+ memset(&adapter->dma_stats, 0, sizeof(adapter->dma_stats));
+
+ return 0;
+}
--
2.25.1
^ permalink raw reply [flat|nested] 110+ messages in thread
* [PATCH v8 10/12] eventdev/dma: support adapter enqueue
2023-09-29 11:50 ` [PATCH v8 00/12] event DMA adapter library support Amit Prakash Shukla
` (8 preceding siblings ...)
2023-09-29 11:50 ` [PATCH v8 09/12] eventdev/dma: support adapter stats Amit Prakash Shukla
@ 2023-09-29 11:50 ` Amit Prakash Shukla
2023-09-29 11:50 ` [PATCH v8 11/12] eventdev/dma: support adapter event port get Amit Prakash Shukla
` (2 subsequent siblings)
12 siblings, 0 replies; 110+ messages in thread
From: Amit Prakash Shukla @ 2023-09-29 11:50 UTC (permalink / raw)
To: Amit Prakash Shukla, Jerin Jacob
Cc: dev, fengchengwen, kevin.laatz, bruce.richardson, conor.walsh,
vattunuru, g.singh, sachin.saxena, hemant.agrawal, cheng1.jiang,
ndabilpuram, anoobj, mb
Added API support to enqueue a DMA operation to the DMA driver.
Signed-off-by: Amit Prakash Shukla <amitprakashs@marvell.com>
---
lib/eventdev/rte_event_dma_adapter.c | 13 +++++++++++++
1 file changed, 13 insertions(+)
diff --git a/lib/eventdev/rte_event_dma_adapter.c b/lib/eventdev/rte_event_dma_adapter.c
index 6c67e6d499..f299914dec 100644
--- a/lib/eventdev/rte_event_dma_adapter.c
+++ b/lib/eventdev/rte_event_dma_adapter.c
@@ -1403,3 +1403,16 @@ rte_event_dma_adapter_stats_reset(uint8_t id)
return 0;
}
+
+uint16_t
+rte_event_dma_adapter_enqueue(uint8_t dev_id, uint8_t port_id, struct rte_event ev[],
+ uint16_t nb_events)
+{
+ const struct rte_event_fp_ops *fp_ops;
+ void *port;
+
+ fp_ops = &rte_event_fp_ops[dev_id];
+ port = fp_ops->data[port_id];
+
+ return fp_ops->dma_enqueue(port, ev, nb_events);
+}
--
2.25.1
^ permalink raw reply [flat|nested] 110+ messages in thread
* [PATCH v8 11/12] eventdev/dma: support adapter event port get
2023-09-29 11:50 ` [PATCH v8 00/12] event DMA adapter library support Amit Prakash Shukla
` (9 preceding siblings ...)
2023-09-29 11:50 ` [PATCH v8 10/12] eventdev/dma: support adapter enqueue Amit Prakash Shukla
@ 2023-09-29 11:50 ` Amit Prakash Shukla
2023-09-29 11:50 ` [PATCH v8 12/12] app/test: add event DMA adapter auto-test Amit Prakash Shukla
2023-10-03 5:13 ` [PATCH v8 00/12] event DMA adapter library support Jerin Jacob
12 siblings, 0 replies; 110+ messages in thread
From: Amit Prakash Shukla @ 2023-09-29 11:50 UTC (permalink / raw)
To: Amit Prakash Shukla, Jerin Jacob
Cc: dev, fengchengwen, kevin.laatz, bruce.richardson, conor.walsh,
vattunuru, g.singh, sachin.saxena, hemant.agrawal, cheng1.jiang,
ndabilpuram, anoobj, mb
Added support for DMA adapter event port get.
Signed-off-by: Amit Prakash Shukla <amitprakashs@marvell.com>
---
lib/eventdev/rte_event_dma_adapter.c | 16 ++++++++++++++++
1 file changed, 16 insertions(+)
diff --git a/lib/eventdev/rte_event_dma_adapter.c b/lib/eventdev/rte_event_dma_adapter.c
index f299914dec..af4b5ad388 100644
--- a/lib/eventdev/rte_event_dma_adapter.c
+++ b/lib/eventdev/rte_event_dma_adapter.c
@@ -475,6 +475,22 @@ rte_event_dma_adapter_free(uint8_t id)
return 0;
}
+int
+rte_event_dma_adapter_event_port_get(uint8_t id, uint8_t *event_port_id)
+{
+ struct event_dma_adapter *adapter;
+
+ EVENT_DMA_ADAPTER_ID_VALID_OR_ERR_RET(id, -EINVAL);
+
+ adapter = edma_id_to_adapter(id);
+ if (adapter == NULL || event_port_id == NULL)
+ return -EINVAL;
+
+ *event_port_id = adapter->event_port_id;
+
+ return 0;
+}
+
static inline unsigned int
edma_enq_to_dma_dev(struct event_dma_adapter *adapter, struct rte_event *ev, unsigned int cnt)
{
--
2.25.1
^ permalink raw reply [flat|nested] 110+ messages in thread
* [PATCH v8 12/12] app/test: add event DMA adapter auto-test
2023-09-29 11:50 ` [PATCH v8 00/12] event DMA adapter library support Amit Prakash Shukla
` (10 preceding siblings ...)
2023-09-29 11:50 ` [PATCH v8 11/12] eventdev/dma: support adapter event port get Amit Prakash Shukla
@ 2023-09-29 11:50 ` Amit Prakash Shukla
2023-10-03 5:13 ` [PATCH v8 00/12] event DMA adapter library support Jerin Jacob
12 siblings, 0 replies; 110+ messages in thread
From: Amit Prakash Shukla @ 2023-09-29 11:50 UTC (permalink / raw)
To: Thomas Monjalon, Amit Prakash Shukla
Cc: dev, jerinj, fengchengwen, kevin.laatz, bruce.richardson,
conor.walsh, vattunuru, g.singh, sachin.saxena, hemant.agrawal,
cheng1.jiang, ndabilpuram, anoobj, mb
Added testsuite to test the dma adapter functionality.
The testsuite detects event and DMA device capability
and accordingly dma adapter is configured and modes are
tested. Test command:
sudo <build dir>/app/test/dpdk-test --vdev=dma_skeleton \
event_dma_adapter_autotest
Signed-off-by: Amit Prakash Shukla <amitprakashs@marvell.com>
---
MAINTAINERS | 1 +
app/test/meson.build | 1 +
app/test/test_event_dma_adapter.c | 805 ++++++++++++++++++++++++++++++
3 files changed, 807 insertions(+)
create mode 100644 app/test/test_event_dma_adapter.c
diff --git a/MAINTAINERS b/MAINTAINERS
index 4ebbbe8bb3..92c0b47618 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -544,6 +544,7 @@ Eventdev DMA Adapter API
M: Amit Prakash Shukla <amitprakashs@marvell.com>
T: git://dpdk.org/next/dpdk-next-eventdev
F: lib/eventdev/*dma_adapter*
+F: app/test/test_event_dma_adapter.c
F: doc/guides/prog_guide/event_dma_adapter.rst
Raw device API
diff --git a/app/test/meson.build b/app/test/meson.build
index 05bae9216d..7caf5ae5fc 100644
--- a/app/test/meson.build
+++ b/app/test/meson.build
@@ -66,6 +66,7 @@ source_file_deps = {
'test_errno.c': [],
'test_ethdev_link.c': ['ethdev'],
'test_event_crypto_adapter.c': ['cryptodev', 'eventdev', 'bus_vdev'],
+ 'test_event_dma_adapter.c': ['dmadev', 'eventdev', 'bus_vdev'],
'test_event_eth_rx_adapter.c': ['ethdev', 'eventdev', 'bus_vdev'],
'test_event_eth_tx_adapter.c': ['bus_vdev', 'ethdev', 'net_ring', 'eventdev'],
'test_event_ring.c': ['eventdev'],
diff --git a/app/test/test_event_dma_adapter.c b/app/test/test_event_dma_adapter.c
new file mode 100644
index 0000000000..1e193f4b52
--- /dev/null
+++ b/app/test/test_event_dma_adapter.c
@@ -0,0 +1,805 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright (c) 2023 Marvell.
+ */
+
+#include "test.h"
+#include <string.h>
+#include <rte_common.h>
+#include <rte_malloc.h>
+#include <rte_mempool.h>
+#include <rte_mbuf.h>
+#include <rte_random.h>
+
+#ifdef RTE_EXEC_ENV_WINDOWS
+static int
+test_event_dma_adapter(void)
+{
+ printf("event_dma_adapter not supported on Windows, skipping test\n");
+ return TEST_SKIPPED;
+}
+
+#else
+
+#include <rte_bus_vdev.h>
+#include <rte_dmadev.h>
+#include <rte_eventdev.h>
+#include <rte_event_dma_adapter.h>
+#include <rte_service.h>
+
+#define NUM_MBUFS (8191)
+#define MBUF_CACHE_SIZE (256)
+#define TEST_APP_PORT_ID 0
+#define TEST_APP_EV_QUEUE_ID 0
+#define TEST_APP_EV_PRIORITY 0
+#define TEST_APP_EV_FLOWID 0xAABB
+#define TEST_DMA_EV_QUEUE_ID 1
+#define TEST_ADAPTER_ID 0
+#define TEST_DMA_DEV_ID 0
+#define TEST_DMA_VCHAN_ID 0
+#define PACKET_LENGTH 1024
+#define NB_TEST_PORTS 1
+#define NB_TEST_QUEUES 2
+#define NUM_CORES 2
+#define DMA_OP_POOL_SIZE 128
+#define TEST_MAX_OP 32
+#define TEST_RINGSIZE 512
+
+#define MBUF_SIZE (RTE_PKTMBUF_HEADROOM + PACKET_LENGTH)
+
+/* Handle log statements in same manner as test macros */
+#define LOG_DBG(...) RTE_LOG(DEBUG, EAL, __VA_ARGS__)
+
+struct event_dma_adapter_test_params {
+ struct rte_mempool *src_mbuf_pool;
+ struct rte_mempool *dst_mbuf_pool;
+ struct rte_mempool *op_mpool;
+ uint8_t dma_event_port_id;
+ uint8_t internal_port_op_fwd;
+};
+
+struct rte_event dma_response_info = {
+ .queue_id = TEST_APP_EV_QUEUE_ID,
+ .sched_type = RTE_SCHED_TYPE_ATOMIC,
+ .flow_id = TEST_APP_EV_FLOWID,
+ .priority = TEST_APP_EV_PRIORITY
+};
+
+static struct event_dma_adapter_test_params params;
+static uint8_t dma_adapter_setup_done;
+static uint32_t slcore_id;
+static int evdev;
+
+static int
+send_recv_ev(struct rte_event *ev)
+{
+ struct rte_event recv_ev[TEST_MAX_OP];
+ uint16_t nb_enqueued = 0;
+ int i = 0;
+
+ if (params.internal_port_op_fwd) {
+ nb_enqueued = rte_event_dma_adapter_enqueue(evdev, TEST_APP_PORT_ID, ev,
+ TEST_MAX_OP);
+ } else {
+ while (nb_enqueued < TEST_MAX_OP) {
+ nb_enqueued += rte_event_enqueue_burst(evdev, TEST_APP_PORT_ID,
+ &ev[nb_enqueued], TEST_MAX_OP -
+ nb_enqueued);
+ }
+ }
+
+ TEST_ASSERT_EQUAL(nb_enqueued, TEST_MAX_OP, "Failed to send event to dma adapter\n");
+
+ while (i < TEST_MAX_OP) {
+ if (rte_event_dequeue_burst(evdev, TEST_APP_PORT_ID, &recv_ev[i], 1, 0) != 1)
+ continue;
+ i++;
+ }
+
+ TEST_ASSERT_EQUAL(i, TEST_MAX_OP, "Test failed. Failed to dequeue events.\n");
+
+ return TEST_SUCCESS;
+}
+
+static int
+test_dma_adapter_stats(void)
+{
+ struct rte_event_dma_adapter_stats stats;
+
+ rte_event_dma_adapter_stats_get(TEST_ADAPTER_ID, &stats);
+ printf(" +------------------------------------------------------+\n");
+ printf(" + DMA adapter stats for instance %u:\n", TEST_ADAPTER_ID);
+ printf(" + Event port poll count 0x%" PRIx64 "\n",
+ stats.event_poll_count);
+ printf(" + Event dequeue count 0x%" PRIx64 "\n",
+ stats.event_deq_count);
+ printf(" + DMA dev enqueue count 0x%" PRIx64 "\n",
+ stats.dma_enq_count);
+ printf(" + DMA dev enqueue failed count 0x%" PRIx64 "\n",
+ stats.dma_enq_fail_count);
+ printf(" + DMA dev dequeue count 0x%" PRIx64 "\n",
+ stats.dma_deq_count);
+ printf(" + Event enqueue count 0x%" PRIx64 "\n",
+ stats.event_enq_count);
+ printf(" + Event enqueue retry count 0x%" PRIx64 "\n",
+ stats.event_enq_retry_count);
+ printf(" + Event enqueue fail count 0x%" PRIx64 "\n",
+ stats.event_enq_fail_count);
+ printf(" +------------------------------------------------------+\n");
+
+ rte_event_dma_adapter_stats_reset(TEST_ADAPTER_ID);
+ return TEST_SUCCESS;
+}
+
+static int
+test_dma_adapter_params(void)
+{
+ struct rte_event_dma_adapter_runtime_params out_params;
+ struct rte_event_dma_adapter_runtime_params in_params;
+ struct rte_event event;
+ uint32_t cap;
+ int err, rc;
+
+ err = rte_event_dma_adapter_caps_get(evdev, TEST_DMA_DEV_ID, &cap);
+ TEST_ASSERT_SUCCESS(err, "Failed to get adapter capabilities\n");
+
+ if (cap & RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_VCHAN_EV_BIND) {
+ err = rte_event_dma_adapter_vchan_add(TEST_ADAPTER_ID, TEST_DMA_DEV_ID,
+ TEST_DMA_VCHAN_ID, &event);
+ } else
+ err = rte_event_dma_adapter_vchan_add(TEST_ADAPTER_ID, TEST_DMA_DEV_ID,
+ TEST_DMA_VCHAN_ID, NULL);
+
+ TEST_ASSERT_SUCCESS(err, "Failed to add vchan\n");
+
+ err = rte_event_dma_adapter_runtime_params_init(&in_params);
+ TEST_ASSERT(err == 0, "Expected 0 got %d", err);
+ err = rte_event_dma_adapter_runtime_params_init(&out_params);
+ TEST_ASSERT(err == 0, "Expected 0 got %d", err);
+
+ /* Case 1: Get the default value of mbufs processed by adapter */
+ err = rte_event_dma_adapter_runtime_params_get(TEST_ADAPTER_ID, &out_params);
+ if (err == -ENOTSUP) {
+ rc = TEST_SKIPPED;
+ goto vchan_del;
+ }
+ TEST_ASSERT(err == 0, "Expected 0 got %d", err);
+
+ /* Case 2: Set max_nb = 32 (=BATCH_SEIZE) */
+ in_params.max_nb = 32;
+
+ err = rte_event_dma_adapter_runtime_params_set(TEST_ADAPTER_ID, &in_params);
+ TEST_ASSERT(err == 0, "Expected 0 got %d", err);
+
+ err = rte_event_dma_adapter_runtime_params_get(TEST_ADAPTER_ID, &out_params);
+ TEST_ASSERT(err == 0, "Expected 0 got %d", err);
+ TEST_ASSERT(in_params.max_nb == out_params.max_nb, "Expected %u got %u",
+ in_params.max_nb, out_params.max_nb);
+
+ /* Case 3: Set max_nb = 192 */
+ in_params.max_nb = 192;
+
+ err = rte_event_dma_adapter_runtime_params_set(TEST_ADAPTER_ID, &in_params);
+ TEST_ASSERT(err == 0, "Expected 0 got %d", err);
+
+ err = rte_event_dma_adapter_runtime_params_get(TEST_ADAPTER_ID, &out_params);
+ TEST_ASSERT(err == 0, "Expected 0 got %d", err);
+ TEST_ASSERT(in_params.max_nb == out_params.max_nb, "Expected %u got %u",
+ in_params.max_nb, out_params.max_nb);
+
+ /* Case 4: Set max_nb = 256 */
+ in_params.max_nb = 256;
+
+ err = rte_event_dma_adapter_runtime_params_set(TEST_ADAPTER_ID, &in_params);
+ TEST_ASSERT(err == 0, "Expected 0 got %d", err);
+
+ err = rte_event_dma_adapter_runtime_params_get(TEST_ADAPTER_ID, &out_params);
+ TEST_ASSERT(err == 0, "Expected 0 got %d", err);
+ TEST_ASSERT(in_params.max_nb == out_params.max_nb, "Expected %u got %u",
+ in_params.max_nb, out_params.max_nb);
+
+ /* Case 5: Set max_nb = 30(<BATCH_SIZE) */
+ in_params.max_nb = 30;
+
+ err = rte_event_dma_adapter_runtime_params_set(TEST_ADAPTER_ID, &in_params);
+ TEST_ASSERT(err == 0, "Expected 0 got %d", err);
+
+ err = rte_event_dma_adapter_runtime_params_get(TEST_ADAPTER_ID, &out_params);
+ TEST_ASSERT(err == 0, "Expected 0 got %d", err);
+ TEST_ASSERT(in_params.max_nb == out_params.max_nb, "Expected %u got %u",
+ in_params.max_nb, out_params.max_nb);
+
+ /* Case 6: Set max_nb = 512 */
+ in_params.max_nb = 512;
+
+ err = rte_event_dma_adapter_runtime_params_set(TEST_ADAPTER_ID, &in_params);
+ TEST_ASSERT(err == 0, "Expected 0 got %d", err);
+
+ err = rte_event_dma_adapter_runtime_params_get(TEST_ADAPTER_ID, &out_params);
+ TEST_ASSERT(err == 0, "Expected 0 got %d", err);
+ TEST_ASSERT(in_params.max_nb == out_params.max_nb, "Expected %u got %u",
+ in_params.max_nb, out_params.max_nb);
+
+ rc = TEST_SUCCESS;
+vchan_del:
+ err = rte_event_dma_adapter_vchan_del(TEST_ADAPTER_ID, TEST_DMA_DEV_ID,
+ TEST_DMA_VCHAN_ID);
+ TEST_ASSERT_SUCCESS(err, "Failed to delete vchan\n");
+
+ return rc;
+}
+
+static int
+test_op_forward_mode(void)
+{
+ struct rte_mbuf *src_mbuf[TEST_MAX_OP];
+ struct rte_mbuf *dst_mbuf[TEST_MAX_OP];
+ struct rte_event_dma_adapter_op *op;
+ struct rte_event ev[TEST_MAX_OP];
+ struct rte_event response_info;
+ int ret, i;
+
+ ret = rte_pktmbuf_alloc_bulk(params.src_mbuf_pool, src_mbuf, TEST_MAX_OP);
+ TEST_ASSERT_SUCCESS(ret, "alloc src mbufs failed.\n");
+
+ ret = rte_pktmbuf_alloc_bulk(params.dst_mbuf_pool, dst_mbuf, TEST_MAX_OP);
+ TEST_ASSERT_SUCCESS(ret, "alloc dst mbufs failed.\n");
+
+ for (i = 0; i < TEST_MAX_OP; i++) {
+ memset(rte_pktmbuf_mtod(src_mbuf[i], void *), rte_rand(), PACKET_LENGTH);
+ memset(rte_pktmbuf_mtod(dst_mbuf[i], void *), 0, PACKET_LENGTH);
+ }
+
+ for (i = 0; i < TEST_MAX_OP; i++) {
+ rte_mempool_get(params.op_mpool, (void **)&op);
+ TEST_ASSERT_NOT_NULL(op, "Failed to allocate dma operation struct\n");
+
+ op->src_seg = rte_malloc(NULL, sizeof(struct rte_dma_sge), 0);
+ op->dst_seg = rte_malloc(NULL, sizeof(struct rte_dma_sge), 0);
+
+ /* Update Op */
+ op->src_seg->addr = rte_pktmbuf_iova(src_mbuf[i]);
+ op->dst_seg->addr = rte_pktmbuf_iova(dst_mbuf[i]);
+ op->src_seg->length = PACKET_LENGTH;
+ op->dst_seg->length = PACKET_LENGTH;
+ op->nb_src = 1;
+ op->nb_dst = 1;
+ op->flags = RTE_DMA_OP_FLAG_SUBMIT;
+ op->op_mp = params.op_mpool;
+ op->dma_dev_id = TEST_DMA_DEV_ID;
+ op->vchan = TEST_DMA_VCHAN_ID;
+
+ response_info.event = dma_response_info.event;
+ rte_memcpy((uint8_t *)op + sizeof(struct rte_event_dma_adapter_op), &response_info,
+ sizeof(struct rte_event));
+
+ /* Fill in event info and update event_ptr with rte_event_dma_adapter_op */
+ memset(&ev[i], 0, sizeof(struct rte_event));
+ ev[i].event = 0;
+ ev[i].event_type = RTE_EVENT_TYPE_DMADEV;
+ ev[i].queue_id = TEST_DMA_EV_QUEUE_ID;
+ ev[i].sched_type = RTE_SCHED_TYPE_ATOMIC;
+ ev[i].flow_id = 0xAABB;
+ ev[i].event_ptr = op;
+ }
+
+ ret = send_recv_ev(ev);
+ TEST_ASSERT_SUCCESS(ret, "Failed to send/receive event to dma adapter\n");
+
+ test_dma_adapter_stats();
+
+ for (i = 0; i < TEST_MAX_OP; i++) {
+ op = ev[i].event_ptr;
+ ret = memcmp(rte_pktmbuf_mtod(src_mbuf[i], void *),
+ rte_pktmbuf_mtod(dst_mbuf[i], void *), PACKET_LENGTH);
+
+ TEST_ASSERT_EQUAL(ret, 0, "Data mismatch for dma adapter\n");
+
+ rte_free(op->src_seg);
+ rte_free(op->dst_seg);
+ rte_mempool_put(op->op_mp, op);
+ }
+
+ rte_pktmbuf_free_bulk(src_mbuf, TEST_MAX_OP);
+ rte_pktmbuf_free_bulk(dst_mbuf, TEST_MAX_OP);
+
+ return TEST_SUCCESS;
+}
+
+static int
+map_adapter_service_core(void)
+{
+ uint32_t adapter_service_id;
+ int ret;
+
+ if (rte_event_dma_adapter_service_id_get(TEST_ADAPTER_ID, &adapter_service_id) == 0) {
+ uint32_t core_list[NUM_CORES];
+
+ ret = rte_service_lcore_list(core_list, NUM_CORES);
+ TEST_ASSERT(ret >= 0, "Failed to get service core list!");
+
+ if (core_list[0] != slcore_id) {
+ TEST_ASSERT_SUCCESS(rte_service_lcore_add(slcore_id),
+ "Failed to add service core");
+ TEST_ASSERT_SUCCESS(rte_service_lcore_start(slcore_id),
+ "Failed to start service core");
+ }
+
+ TEST_ASSERT_SUCCESS(rte_service_map_lcore_set(
+ adapter_service_id, slcore_id, 1),
+ "Failed to map adapter service");
+ }
+
+ return TEST_SUCCESS;
+}
+
+static int
+test_with_op_forward_mode(void)
+{
+ uint32_t cap;
+ int ret;
+
+ ret = rte_event_dma_adapter_caps_get(evdev, TEST_DMA_DEV_ID, &cap);
+ TEST_ASSERT_SUCCESS(ret, "Failed to get adapter capabilities\n");
+
+ if (!(cap & RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP_FWD) &&
+ !(cap & RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP_NEW))
+ map_adapter_service_core();
+ else {
+ if (!(cap & RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP_FWD))
+ return TEST_SKIPPED;
+ }
+
+ TEST_ASSERT_SUCCESS(rte_event_dma_adapter_start(TEST_ADAPTER_ID),
+ "Failed to start event dma adapter");
+
+ ret = test_op_forward_mode();
+ TEST_ASSERT_SUCCESS(ret, "DMA - FORWARD mode test failed\n");
+ return TEST_SUCCESS;
+}
+
+static int
+configure_dmadev(void)
+{
+ const struct rte_dma_conf conf = { .nb_vchans = 1};
+ const struct rte_dma_vchan_conf qconf = {
+ .direction = RTE_DMA_DIR_MEM_TO_MEM,
+ .nb_desc = TEST_RINGSIZE,
+ };
+ struct rte_dma_info info;
+ unsigned int elt_size;
+ int ret;
+
+ ret = rte_dma_count_avail();
+ RTE_TEST_ASSERT_FAIL(ret, "No dma devices found!\n");
+
+ ret = rte_dma_info_get(TEST_DMA_DEV_ID, &info);
+ TEST_ASSERT_SUCCESS(ret, "Error with rte_dma_info_get()\n");
+
+ if (info.max_vchans < 1)
+ RTE_LOG(ERR, USER1, "Error, no channels available on device id %u\n",
+ TEST_DMA_DEV_ID);
+
+ if (rte_dma_configure(TEST_DMA_DEV_ID, &conf) != 0)
+ RTE_LOG(ERR, USER1, "Error with rte_dma_configure()\n");
+
+ if (rte_dma_vchan_setup(TEST_DMA_DEV_ID, TEST_DMA_VCHAN_ID, &qconf) < 0)
+ RTE_LOG(ERR, USER1, "Error with vchan configuration\n");
+
+ ret = rte_dma_info_get(TEST_DMA_DEV_ID, &info);
+ if (ret != 0 || info.nb_vchans != 1)
+ RTE_LOG(ERR, USER1, "Error, no configured vhcan reported on device id %u\n",
+ TEST_DMA_DEV_ID);
+
+ params.src_mbuf_pool = rte_pktmbuf_pool_create("DMA_ADAPTER_SRC_MBUFPOOL", NUM_MBUFS,
+ MBUF_CACHE_SIZE, 0, MBUF_SIZE,
+ rte_socket_id());
+ RTE_TEST_ASSERT_NOT_NULL(params.src_mbuf_pool, "Can't create DMA_SRC_MBUFPOOL\n");
+
+ params.dst_mbuf_pool = rte_pktmbuf_pool_create("DMA_ADAPTER_DST_MBUFPOOL", NUM_MBUFS,
+ MBUF_CACHE_SIZE, 0, MBUF_SIZE,
+ rte_socket_id());
+ RTE_TEST_ASSERT_NOT_NULL(params.dst_mbuf_pool, "Can't create DMA_DST_MBUFPOOL\n");
+
+ elt_size = sizeof(struct rte_event_dma_adapter_op) + sizeof(struct rte_event);
+ params.op_mpool = rte_mempool_create("EVENT_DMA_OP_POOL", DMA_OP_POOL_SIZE, elt_size, 0,
+ 0, NULL, NULL, NULL, NULL, rte_socket_id(), 0);
+ RTE_TEST_ASSERT_NOT_NULL(params.op_mpool, "Can't create DMA_OP_POOL\n");
+
+ return TEST_SUCCESS;
+}
+
+static inline void
+evdev_set_conf_values(struct rte_event_dev_config *dev_conf, struct rte_event_dev_info *info)
+{
+ memset(dev_conf, 0, sizeof(struct rte_event_dev_config));
+ dev_conf->dequeue_timeout_ns = info->min_dequeue_timeout_ns;
+ dev_conf->nb_event_ports = NB_TEST_PORTS;
+ dev_conf->nb_event_queues = NB_TEST_QUEUES;
+ dev_conf->nb_event_queue_flows = info->max_event_queue_flows;
+ dev_conf->nb_event_port_dequeue_depth =
+ info->max_event_port_dequeue_depth;
+ dev_conf->nb_event_port_enqueue_depth =
+ info->max_event_port_enqueue_depth;
+ dev_conf->nb_event_port_enqueue_depth =
+ info->max_event_port_enqueue_depth;
+ dev_conf->nb_events_limit =
+ info->max_num_events;
+}
+
+static int
+configure_eventdev(void)
+{
+ struct rte_event_queue_conf queue_conf;
+ struct rte_event_dev_config devconf;
+ struct rte_event_dev_info info;
+ uint32_t queue_count;
+ uint32_t port_count;
+ uint8_t qid;
+ int ret;
+
+ if (!rte_event_dev_count()) {
+ /* If there is no hardware eventdev, or no software vdev was
+ * specified on the command line, create an instance of
+ * event_sw.
+ */
+ LOG_DBG("Failed to find a valid event device... "
+ "testing with event_sw device\n");
+ TEST_ASSERT_SUCCESS(rte_vdev_init("event_sw0", NULL),
+ "Error creating eventdev");
+ evdev = rte_event_dev_get_dev_id("event_sw0");
+ }
+
+ ret = rte_event_dev_info_get(evdev, &info);
+ TEST_ASSERT_SUCCESS(ret, "Failed to get event dev info\n");
+
+ evdev_set_conf_values(&devconf, &info);
+
+ ret = rte_event_dev_configure(evdev, &devconf);
+ TEST_ASSERT_SUCCESS(ret, "Failed to configure eventdev\n");
+
+ /* Set up event queue */
+ ret = rte_event_dev_attr_get(evdev, RTE_EVENT_DEV_ATTR_QUEUE_COUNT, &queue_count);
+ TEST_ASSERT_SUCCESS(ret, "Queue count get failed\n");
+ TEST_ASSERT_EQUAL(queue_count, 2, "Unexpected queue count\n");
+
+ qid = TEST_APP_EV_QUEUE_ID;
+ ret = rte_event_queue_setup(evdev, qid, NULL);
+ TEST_ASSERT_SUCCESS(ret, "Failed to setup queue=%d\n", qid);
+
+ queue_conf.nb_atomic_flows = info.max_event_queue_flows;
+ queue_conf.nb_atomic_order_sequences = 32;
+ queue_conf.schedule_type = RTE_SCHED_TYPE_ATOMIC;
+ queue_conf.priority = RTE_EVENT_DEV_PRIORITY_HIGHEST;
+ queue_conf.event_queue_cfg = RTE_EVENT_QUEUE_CFG_SINGLE_LINK;
+
+ qid = TEST_DMA_EV_QUEUE_ID;
+ ret = rte_event_queue_setup(evdev, qid, &queue_conf);
+ TEST_ASSERT_SUCCESS(ret, "Failed to setup queue=%u\n", qid);
+
+ /* Set up event port */
+ ret = rte_event_dev_attr_get(evdev, RTE_EVENT_DEV_ATTR_PORT_COUNT,
+ &port_count);
+ TEST_ASSERT_SUCCESS(ret, "Port count get failed\n");
+ TEST_ASSERT_EQUAL(port_count, 1, "Unexpected port count\n");
+
+ ret = rte_event_port_setup(evdev, TEST_APP_PORT_ID, NULL);
+ TEST_ASSERT_SUCCESS(ret, "Failed to setup port=%d\n",
+ TEST_APP_PORT_ID);
+
+ qid = TEST_APP_EV_QUEUE_ID;
+ ret = rte_event_port_link(evdev, TEST_APP_PORT_ID, &qid, NULL, 1);
+ TEST_ASSERT(ret >= 0, "Failed to link queue port=%d\n",
+ TEST_APP_PORT_ID);
+
+ return TEST_SUCCESS;
+}
+
+static void
+test_dma_adapter_free(void)
+{
+ rte_event_dma_adapter_free(TEST_ADAPTER_ID);
+}
+
+static int
+test_dma_adapter_create(void)
+{
+ struct rte_event_dev_info evdev_info = {0};
+ struct rte_event_port_conf conf = {0};
+ int ret;
+
+ ret = rte_event_dev_info_get(evdev, &evdev_info);
+ TEST_ASSERT_SUCCESS(ret, "Failed to create event dma adapter\n");
+
+ conf.new_event_threshold = evdev_info.max_num_events;
+ conf.dequeue_depth = evdev_info.max_event_port_dequeue_depth;
+ conf.enqueue_depth = evdev_info.max_event_port_enqueue_depth;
+
+ /* Create adapter with default port creation callback */
+ ret = rte_event_dma_adapter_create(TEST_ADAPTER_ID, evdev, &conf, 0);
+ TEST_ASSERT_SUCCESS(ret, "Failed to create event dma adapter\n");
+
+ return TEST_SUCCESS;
+}
+
+static int
+test_dma_adapter_vchan_add_del(void)
+{
+ struct rte_event event;
+ uint32_t cap;
+ int ret;
+
+ ret = rte_event_dma_adapter_caps_get(evdev, TEST_DMA_DEV_ID, &cap);
+ TEST_ASSERT_SUCCESS(ret, "Failed to get adapter capabilities\n");
+
+ if (cap & RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_VCHAN_EV_BIND) {
+ ret = rte_event_dma_adapter_vchan_add(TEST_ADAPTER_ID, TEST_DMA_DEV_ID,
+ TEST_DMA_VCHAN_ID, &event);
+ } else
+ ret = rte_event_dma_adapter_vchan_add(TEST_ADAPTER_ID, TEST_DMA_DEV_ID,
+ TEST_DMA_VCHAN_ID, NULL);
+
+ TEST_ASSERT_SUCCESS(ret, "Failed to create add vchan\n");
+
+ ret = rte_event_dma_adapter_vchan_del(TEST_ADAPTER_ID, TEST_DMA_DEV_ID,
+ TEST_DMA_VCHAN_ID);
+ TEST_ASSERT_SUCCESS(ret, "Failed to delete vchan\n");
+
+ return TEST_SUCCESS;
+}
+
+static int
+configure_event_dma_adapter(enum rte_event_dma_adapter_mode mode)
+{
+ struct rte_event_dev_info evdev_info = {0};
+ struct rte_event_port_conf conf = {0};
+ struct rte_event event;
+ uint32_t cap;
+ int ret;
+
+ ret = rte_event_dma_adapter_caps_get(evdev, TEST_DMA_DEV_ID, &cap);
+ TEST_ASSERT_SUCCESS(ret, "Failed to get adapter capabilities\n");
+
+ /* Skip mode and capability mismatch check for SW eventdev */
+ if (!(cap & RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP_NEW) &&
+ !(cap & RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP_FWD) &&
+ !(cap & RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_VCHAN_EV_BIND))
+ goto adapter_create;
+
+ if (mode == RTE_EVENT_DMA_ADAPTER_OP_FORWARD) {
+ if (cap & RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP_FWD)
+ params.internal_port_op_fwd = 1;
+ else
+ return -ENOTSUP;
+ }
+
+adapter_create:
+ ret = rte_event_dev_info_get(evdev, &evdev_info);
+ TEST_ASSERT_SUCCESS(ret, "Failed to create event dma adapter\n");
+
+ conf.new_event_threshold = evdev_info.max_num_events;
+ conf.dequeue_depth = evdev_info.max_event_port_dequeue_depth;
+ conf.enqueue_depth = evdev_info.max_event_port_enqueue_depth;
+
+ /* Create adapter with default port creation callback */
+ ret = rte_event_dma_adapter_create(TEST_ADAPTER_ID, evdev, &conf, mode);
+ TEST_ASSERT_SUCCESS(ret, "Failed to create event dma adapter\n");
+
+ if (cap & RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_VCHAN_EV_BIND) {
+ ret = rte_event_dma_adapter_vchan_add(TEST_ADAPTER_ID, TEST_DMA_DEV_ID,
+ TEST_DMA_VCHAN_ID, &event);
+ } else
+ ret = rte_event_dma_adapter_vchan_add(TEST_ADAPTER_ID, TEST_DMA_DEV_ID,
+ TEST_DMA_VCHAN_ID, NULL);
+
+ TEST_ASSERT_SUCCESS(ret, "Failed to add vchan\n");
+
+ if (!params.internal_port_op_fwd) {
+ ret = rte_event_dma_adapter_event_port_get(TEST_ADAPTER_ID,
+ ¶ms.dma_event_port_id);
+ TEST_ASSERT_SUCCESS(ret, "Failed to get event port\n");
+ }
+
+ return TEST_SUCCESS;
+}
+
+static void
+test_dma_adapter_stop(void)
+{
+ uint32_t evdev_service_id, adapter_service_id;
+
+ /* retrieve service ids & stop services */
+ if (rte_event_dma_adapter_service_id_get(TEST_ADAPTER_ID,
+ &adapter_service_id) == 0) {
+ rte_service_runstate_set(adapter_service_id, 0);
+ rte_service_lcore_stop(slcore_id);
+ rte_service_lcore_del(slcore_id);
+ rte_event_dma_adapter_stop(TEST_ADAPTER_ID);
+ }
+
+ if (rte_event_dev_service_id_get(evdev, &evdev_service_id) == 0) {
+ rte_service_runstate_set(evdev_service_id, 0);
+ rte_service_lcore_stop(slcore_id);
+ rte_service_lcore_del(slcore_id);
+ rte_dma_stop(TEST_DMA_DEV_ID);
+ rte_event_dev_stop(evdev);
+ } else {
+ rte_dma_stop(TEST_DMA_DEV_ID);
+ rte_event_dev_stop(evdev);
+ }
+}
+
+static int
+test_dma_adapter_conf(enum rte_event_dma_adapter_mode mode)
+{
+ uint32_t evdev_service_id;
+ uint8_t qid;
+ int ret;
+
+ if (!dma_adapter_setup_done) {
+ ret = configure_event_dma_adapter(mode);
+ if (ret)
+ return ret;
+ if (!params.internal_port_op_fwd) {
+ qid = TEST_DMA_EV_QUEUE_ID;
+ ret = rte_event_port_link(evdev,
+ params.dma_event_port_id, &qid, NULL, 1);
+ TEST_ASSERT(ret >= 0, "Failed to link queue %d "
+ "port=%u\n", qid,
+ params.dma_event_port_id);
+ }
+ dma_adapter_setup_done = 1;
+ }
+
+ /* retrieve service ids */
+ if (rte_event_dev_service_id_get(evdev, &evdev_service_id) == 0) {
+ /* add a service core and start it */
+ TEST_ASSERT_SUCCESS(rte_service_lcore_add(slcore_id),
+ "Failed to add service core");
+ TEST_ASSERT_SUCCESS(rte_service_lcore_start(slcore_id),
+ "Failed to start service core");
+
+ /* map services to it */
+ TEST_ASSERT_SUCCESS(rte_service_map_lcore_set(evdev_service_id,
+ slcore_id, 1), "Failed to map evdev service");
+
+ /* set services to running */
+ TEST_ASSERT_SUCCESS(rte_service_runstate_set(evdev_service_id,
+ 1), "Failed to start evdev service");
+ }
+
+ /* start the eventdev */
+ TEST_ASSERT_SUCCESS(rte_event_dev_start(evdev),
+ "Failed to start event device");
+
+ /* start the dma dev */
+ TEST_ASSERT_SUCCESS(rte_dma_start(TEST_DMA_DEV_ID),
+ "Failed to start dma device");
+
+ return TEST_SUCCESS;
+}
+
+static int
+test_dma_adapter_conf_op_forward_mode(void)
+{
+ enum rte_event_dma_adapter_mode mode;
+
+ mode = RTE_EVENT_DMA_ADAPTER_OP_FORWARD;
+
+ return test_dma_adapter_conf(mode);
+}
+
+static int
+testsuite_setup(void)
+{
+ int ret;
+
+ slcore_id = rte_get_next_lcore(-1, 1, 0);
+ TEST_ASSERT_NOT_EQUAL(slcore_id, RTE_MAX_LCORE, "At least 2 lcores "
+ "are required to run this autotest\n");
+
+ /* Setup and start event device. */
+ ret = configure_eventdev();
+ TEST_ASSERT_SUCCESS(ret, "Failed to setup eventdev\n");
+
+ /* Setup and start dma device. */
+ ret = configure_dmadev();
+ TEST_ASSERT_SUCCESS(ret, "dmadev initialization failed\n");
+
+ return TEST_SUCCESS;
+}
+
+static void
+dma_adapter_teardown(void)
+{
+ int ret;
+
+ ret = rte_event_dma_adapter_stop(TEST_ADAPTER_ID);
+ if (ret < 0)
+ RTE_LOG(ERR, USER1, "Failed to stop adapter!");
+
+ ret = rte_event_dma_adapter_vchan_del(TEST_ADAPTER_ID, TEST_DMA_DEV_ID,
+ TEST_DMA_VCHAN_ID);
+ if (ret < 0)
+ RTE_LOG(ERR, USER1, "Failed to delete vchan!");
+
+ ret = rte_event_dma_adapter_free(TEST_ADAPTER_ID);
+ if (ret < 0)
+ RTE_LOG(ERR, USER1, "Failed to free adapter!");
+
+ dma_adapter_setup_done = 0;
+}
+
+static void
+dma_teardown(void)
+{
+ /* Free mbuf mempool */
+ if (params.src_mbuf_pool != NULL) {
+ RTE_LOG(DEBUG, USER1, "DMA_ADAPTER_SRC_MBUFPOOL count %u\n",
+ rte_mempool_avail_count(params.src_mbuf_pool));
+ rte_mempool_free(params.src_mbuf_pool);
+ params.src_mbuf_pool = NULL;
+ }
+
+ if (params.dst_mbuf_pool != NULL) {
+ RTE_LOG(DEBUG, USER1, "DMA_ADAPTER_DST_MBUFPOOL count %u\n",
+ rte_mempool_avail_count(params.dst_mbuf_pool));
+ rte_mempool_free(params.dst_mbuf_pool);
+ params.dst_mbuf_pool = NULL;
+ }
+
+ /* Free ops mempool */
+ if (params.op_mpool != NULL) {
+ RTE_LOG(DEBUG, USER1, "EVENT_DMA_OP_POOL count %u\n",
+ rte_mempool_avail_count(params.op_mpool));
+ rte_mempool_free(params.op_mpool);
+ params.op_mpool = NULL;
+ }
+}
+
+static void
+eventdev_teardown(void)
+{
+ rte_event_dev_stop(evdev);
+}
+
+static void
+testsuite_teardown(void)
+{
+ dma_adapter_teardown();
+ dma_teardown();
+ eventdev_teardown();
+}
+
+static struct unit_test_suite functional_testsuite = {
+ .suite_name = "Event dma adapter test suite",
+ .setup = testsuite_setup,
+ .teardown = testsuite_teardown,
+ .unit_test_cases = {
+
+ TEST_CASE_ST(NULL, test_dma_adapter_free, test_dma_adapter_create),
+
+ TEST_CASE_ST(test_dma_adapter_create, test_dma_adapter_free,
+ test_dma_adapter_vchan_add_del),
+
+ TEST_CASE_ST(test_dma_adapter_create, test_dma_adapter_free,
+ test_dma_adapter_stats),
+
+ TEST_CASE_ST(test_dma_adapter_create, test_dma_adapter_free,
+ test_dma_adapter_params),
+
+ TEST_CASE_ST(test_dma_adapter_conf_op_forward_mode, test_dma_adapter_stop,
+ test_with_op_forward_mode),
+
+ TEST_CASES_END() /**< NULL terminate unit test array */
+ }
+};
+
+static int
+test_event_dma_adapter(void)
+{
+ return unit_test_suite_runner(&functional_testsuite);
+}
+
+#endif /* !RTE_EXEC_ENV_WINDOWS */
+
+REGISTER_TEST_COMMAND(event_dma_adapter_autotest, test_event_dma_adapter);
--
2.25.1
^ permalink raw reply [flat|nested] 110+ messages in thread
* Re: [PATCH v8 00/12] event DMA adapter library support
2023-09-29 11:50 ` [PATCH v8 00/12] event DMA adapter library support Amit Prakash Shukla
` (11 preceding siblings ...)
2023-09-29 11:50 ` [PATCH v8 12/12] app/test: add event DMA adapter auto-test Amit Prakash Shukla
@ 2023-10-03 5:13 ` Jerin Jacob
12 siblings, 0 replies; 110+ messages in thread
From: Jerin Jacob @ 2023-10-03 5:13 UTC (permalink / raw)
To: Amit Prakash Shukla
Cc: dev, jerinj, fengchengwen, kevin.laatz, bruce.richardson,
conor.walsh, vattunuru, g.singh, sachin.saxena, hemant.agrawal,
cheng1.jiang, ndabilpuram, anoobj, mb
On Sat, Sep 30, 2023 at 1:06 AM Amit Prakash Shukla
<amitprakashs@marvell.com> wrote:
>
> This series adds support for event DMA adapter library. API's defined
> as part of this library can be used by the application for DMA transfer
> of data using event based mechanism.
>
> v8:
> - Re-arranged DMA adapter section in release notes.
Updated the git commit as follows and applied series to
dpdk-next-net-eventdev/for-main. Thanks
commit 9f4e84563c6e27fa6c17662f46a0fd7927228801 (HEAD -> for-main,
origin/for-main, origin/HEAD)
Author: Amit Prakash Shukla <amitprakashs@marvell.com>
Date: Fri Sep 29 17:20:51 2023 +0530
app/test: add event DMA adapter auto-test
Added testsuite to test the dma adapter functionality.
The testsuite detects event and DMA device capability
and accordingly dma adapter is configured and modes are
tested. Test command:
sudo <build dir>/app/test/dpdk-test --vdev=dma_skeleton \
event_dma_adapter_autotest
Signed-off-by: Amit Prakash Shukla <amitprakashs@marvell.com>
commit 8c35a6be9d5a2017563b7cf3e3285711ea7b4b3a
Author: Amit Prakash Shukla <amitprakashs@marvell.com>
Date: Fri Sep 29 17:20:50 2023 +0530
eventdev/dma: support adapter event port get
Added support for DMA adapter event port get.
Signed-off-by: Amit Prakash Shukla <amitprakashs@marvell.com>
commit 72124a5ada9743113bcebeefdfa2cc7bfd01f1ba
Author: Amit Prakash Shukla <amitprakashs@marvell.com>
Date: Fri Sep 29 17:20:49 2023 +0530
eventdev/dma: support adapter enqueue
Added API support to enqueue a DMA operation to the DMA driver.
Signed-off-by: Amit Prakash Shukla <amitprakashs@marvell.com>
commit 60254867e10d7ceaff14dd1468bcb8ebfc536384
Author: Amit Prakash Shukla <amitprakashs@marvell.com>
Date: Fri Sep 29 17:20:48 2023 +0530
eventdev/dma: support adapter stats
Added DMA adapter stats API support to get and reset stats.
DMA SW adapter stats and eventdev driver supported stats for
enqueue and dequeue are reported by get API.
Signed-off-by: Amit Prakash Shukla <amitprakashs@marvell.com>
commit 1582c998adb0c39b297c14bd26d920dc3ffbfe7e
Author: Amit Prakash Shukla <amitprakashs@marvell.com>
Date: Fri Sep 29 17:20:47 2023 +0530
eventdev/dma: support adapter runtime params
Added support to set and get runtime params for DMA adapter.
The parameters that can be set/get are defined in
struct rte_event_dma_adapter_runtime_params.
Signed-off-by: Amit Prakash Shukla <amitprakashs@marvell.com>
commit cacb21e2a5c30221a1327a8fccacde36836c31c2
Author: Amit Prakash Shukla <amitprakashs@marvell.com>
Date: Fri Sep 29 17:20:46 2023 +0530
eventdev/dma: support adapter service ID get
Added API support to get DMA adapter service ID. Service ID
returned in the variable by the API call shall be used by
application to map a service core.
Signed-off-by: Amit Prakash Shukla <amitprakashs@marvell.com>
commit c2d22cda05dd842e8966408880d3d1416820519d
Author: Amit Prakash Shukla <amitprakashs@marvell.com>
Date: Fri Sep 29 17:20:45 2023 +0530
eventdev/dma: support adapter start and stop
Added API support to start and stop DMA adapter.
Signed-off-by: Amit Prakash Shukla <amitprakashs@marvell.com>
commit 8e54ab5c44cc41995e0f09cbe00ff8423ad19283
Author: Amit Prakash Shukla <amitprakashs@marvell.com>
Date: Fri Sep 29 17:20:44 2023 +0530
eventdev/dma: support adapter service function
Added support for DMA adapter service function for event devices.
Enqueue and dequeue of event from eventdev and DMA device are done
based on the adapter mode and the supported HW capabilities.
Signed-off-by: Amit Prakash Shukla <amitprakashs@marvell.com>
commit cc0a6f2373e1e10471d9a8ffb4d9bcb1f48d68e5
Author: Amit Prakash Shukla <amitprakashs@marvell.com>
Date: Fri Sep 29 17:20:43 2023 +0530
eventdev/dma: support vchan add and delete
Added API support to add and delete vchan's from the DMA adapter.
DMA devid and vchan are added to the adapter instance by calling
rte_event_dma_adapter_vchan_add() and deleted using
rte_event_dma_adapter_vchan_del().
Signed-off-by: Amit Prakash Shukla <amitprakashs@marvell.com>
commit c70b5e254a8f72811865944186e83b267d958804
Author: Amit Prakash Shukla <amitprakashs@marvell.com>
Date: Fri Sep 29 17:20:41 2023 +0530
eventdev/dma: support adapter capabilities get
Implement rte_event_dma_adapter_caps_get() API to get
DMA adapter capabilities supported by the driver.
Signed-off-by: Amit Prakash Shukla <amitprakashs@marvell.com>
commit abde4c7f93da4bb436908f62bc1fe98f8af2c347
Author: Amit Prakash Shukla <amitprakashs@marvell.com>
Date: Fri Sep 29 17:20:40 2023 +0530
eventdev/dma: introduce DMA adapter
Introduce event dma adapter interface to transfer packets between
dma device and event device.
Signed-off-by: Amit Prakash Shukla <amitprakashs@marvell.com>
Acked-by: Jerin Jacob <jerinj@marvell.com>
^ permalink raw reply [flat|nested] 110+ messages in thread
end of thread, other threads:[~2023-10-03 5:14 UTC | newest]
Thread overview: 110+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2023-09-19 13:42 [PATCH v1 1/7] eventdev: introduce DMA event adapter library Amit Prakash Shukla
2023-09-19 13:42 ` [PATCH v1 2/7] eventdev: api to get DMA capabilities Amit Prakash Shukla
2023-09-19 13:42 ` [PATCH v1 3/7] eventdev: add DMA adapter implementation Amit Prakash Shukla
2023-09-19 13:42 ` [PATCH v1 4/7] app/test: add event DMA adapter auto-test Amit Prakash Shukla
2023-09-19 13:42 ` [PATCH v1 5/7] common/cnxk: dma result to an offset of the event Amit Prakash Shukla
2023-09-19 13:42 ` [PATCH v1 6/7] dma/cnxk: support for DMA event enqueue dequeue Amit Prakash Shukla
2023-09-19 13:42 ` [PATCH v1 7/7] event/cnxk: support DMA event functions Amit Prakash Shukla
2023-09-21 2:41 ` [PATCH v1 1/7] eventdev: introduce DMA event adapter library Jerin Jacob
2023-09-21 6:42 ` [EXT] " Amit Prakash Shukla
2023-09-22 20:13 ` [PATCH v2 00/12] event DMA adapter library support Amit Prakash Shukla
2023-09-22 20:13 ` [PATCH v2 01/12] eventdev: introduce event DMA adapter library Amit Prakash Shukla
2023-09-22 20:13 ` [PATCH v2 02/12] eventdev: api to get DMA adapter capabilities Amit Prakash Shukla
2023-09-22 20:13 ` [PATCH v2 03/12] eventdev: add DMA adapter API to create and free Amit Prakash Shukla
2023-09-22 20:13 ` [PATCH v2 04/12] eventdev: api support for vchan add and delete Amit Prakash Shukla
2023-09-22 20:13 ` [PATCH v2 05/12] eventdev: add support for service function Amit Prakash Shukla
2023-09-22 20:13 ` [PATCH v2 06/12] eventdev: api support for DMA adapter start stop Amit Prakash Shukla
2023-09-22 20:13 ` [PATCH v2 07/12] eventdev: api support to get DMA adapter service ID Amit Prakash Shukla
2023-09-22 20:13 ` [PATCH v2 08/12] eventdev: add DMA adapter support for runtime params Amit Prakash Shukla
2023-09-22 20:13 ` [PATCH v2 09/12] eventdev: add support for DMA adapter stats Amit Prakash Shukla
2023-09-22 20:13 ` [PATCH v2 10/12] eventdev: add support for DMA adapter enqueue Amit Prakash Shukla
2023-09-22 20:13 ` [PATCH v2 11/12] eventdev: add DMA adapter port get Amit Prakash Shukla
2023-09-22 20:13 ` [PATCH v2 12/12] app/test: add event DMA adapter auto-test Amit Prakash Shukla
2023-09-23 13:34 ` [PATCH v3 00/12] event DMA adapter library support Amit Prakash Shukla
2023-09-23 13:34 ` [PATCH v3 01/12] eventdev: introduce event DMA adapter library Amit Prakash Shukla
2023-09-23 13:34 ` [PATCH v3 02/12] eventdev: api to get DMA adapter capabilities Amit Prakash Shukla
2023-09-23 13:34 ` [PATCH v3 03/12] eventdev: create and free API for DMA adapter Amit Prakash Shukla
2023-09-23 13:34 ` [PATCH v3 04/12] eventdev: add API support for vchan add and delete Amit Prakash Shukla
2023-09-23 13:34 ` [PATCH v3 05/12] eventdev: add support for DMA adapter service function Amit Prakash Shukla
2023-09-23 13:34 ` [PATCH v3 06/12] eventdev: add support for DMA adapter start and stop Amit Prakash Shukla
2023-09-23 13:34 ` [PATCH v3 07/12] eventdev: add support for DMA adapter service ID get Amit Prakash Shukla
2023-09-23 13:34 ` [PATCH v3 08/12] eventdev: add DMA adapter support for runtime params Amit Prakash Shukla
2023-09-23 13:34 ` [PATCH v3 09/12] eventdev: add support for DMA adapter stats Amit Prakash Shukla
2023-09-23 13:34 ` [PATCH v3 10/12] eventdev: add support for DMA adapter enqueue Amit Prakash Shukla
2023-09-23 13:34 ` [PATCH v3 11/12] eventdev: add DMA adapter event port get Amit Prakash Shukla
2023-09-23 13:34 ` [PATCH v3 12/12] app/test: add event DMA adapter auto-test Amit Prakash Shukla
2023-09-26 10:32 ` [PATCH v4 00/12] event DMA adapter library support Amit Prakash Shukla
2023-09-26 10:32 ` [PATCH v4 01/12] eventdev: introduce event DMA adapter library Amit Prakash Shukla
2023-09-27 18:12 ` Jerin Jacob
2023-09-27 20:45 ` Thomas Monjalon
2023-09-28 4:04 ` Jerin Jacob
2023-09-26 10:32 ` [PATCH v4 02/12] eventdev: api to get DMA adapter capabilities Amit Prakash Shukla
2023-09-27 18:20 ` Jerin Jacob
2023-09-26 10:32 ` [PATCH v4 03/12] eventdev: create and free API for DMA adapter Amit Prakash Shukla
2023-09-27 18:23 ` Jerin Jacob
2023-09-26 10:32 ` [PATCH v4 04/12] eventdev: add API support for vchan add and delete Amit Prakash Shukla
2023-09-26 10:32 ` [PATCH v4 05/12] eventdev: add support for DMA adapter service function Amit Prakash Shukla
2023-09-26 10:32 ` [PATCH v4 06/12] eventdev: add support for DMA adapter start and stop Amit Prakash Shukla
2023-09-26 10:32 ` [PATCH v4 07/12] eventdev: add support for DMA adapter service ID get Amit Prakash Shukla
2023-09-26 10:32 ` [PATCH v4 08/12] eventdev: add DMA adapter support for runtime params Amit Prakash Shukla
2023-09-26 10:32 ` [PATCH v4 09/12] eventdev: add support for DMA adapter stats Amit Prakash Shukla
2023-09-26 10:32 ` [PATCH v4 10/12] eventdev: add support for DMA adapter enqueue Amit Prakash Shukla
2023-09-26 10:32 ` [PATCH v4 11/12] eventdev: add DMA adapter event port get Amit Prakash Shukla
2023-09-26 10:32 ` [PATCH v4 12/12] app/test: add event DMA adapter auto-test Amit Prakash Shukla
2023-09-27 18:27 ` Jerin Jacob
2023-09-28 10:36 ` [PATCH v5 00/12] event DMA adapter library support Amit Prakash Shukla
2023-09-28 10:36 ` [PATCH v5 01/12] eventdev/dma: introduce DMA adapter Amit Prakash Shukla
2023-09-28 15:33 ` Jerin Jacob
2023-09-28 10:36 ` [PATCH v5 02/12] eventdev/dma: support adapter capabilities get Amit Prakash Shukla
2023-09-28 10:36 ` [PATCH v5 03/12] eventdev/dma: support adapter create and free Amit Prakash Shukla
2023-09-28 10:36 ` [PATCH v5 04/12] eventdev/dma: support for vchan add and delete Amit Prakash Shukla
2023-09-28 10:36 ` [PATCH v5 05/12] eventdev/dma: support for adapter service function Amit Prakash Shukla
2023-09-28 10:36 ` [PATCH v5 06/12] eventdev/dma: support for adapter start and stop Amit Prakash Shukla
2023-09-28 10:36 ` [PATCH v5 07/12] eventdev/dma: support for adapter service ID get Amit Prakash Shukla
2023-09-28 10:36 ` [PATCH v5 08/12] eventdev/dma: support adapter runtime params Amit Prakash Shukla
2023-09-28 10:36 ` [PATCH v5 09/12] eventdev/dma: support for adapter stats Amit Prakash Shukla
2023-09-28 10:36 ` [PATCH v5 10/12] eventdev/dma: support for adapter enqueue Amit Prakash Shukla
2023-09-28 10:36 ` [PATCH v5 11/12] eventdev/dma: support for adapter event port get Amit Prakash Shukla
2023-09-28 10:36 ` [PATCH v5 12/12] app/test: add event DMA adapter auto-test Amit Prakash Shukla
2023-09-29 7:20 ` Jerin Jacob
2023-09-28 16:49 ` [PATCH v6 00/12] event DMA adapter library support Amit Prakash Shukla
2023-09-28 16:49 ` [PATCH v6 01/12] eventdev/dma: introduce DMA adapter Amit Prakash Shukla
2023-09-28 16:49 ` [PATCH v6 02/12] eventdev/dma: support adapter capabilities get Amit Prakash Shukla
2023-09-28 16:49 ` [PATCH v6 03/12] eventdev/dma: support adapter create and free Amit Prakash Shukla
2023-09-28 16:49 ` [PATCH v6 04/12] eventdev/dma: support vchan add and delete Amit Prakash Shukla
2023-09-28 16:49 ` [PATCH v6 05/12] eventdev/dma: support adapter service function Amit Prakash Shukla
2023-09-28 16:49 ` [PATCH v6 06/12] eventdev/dma: support adapter start and stop Amit Prakash Shukla
2023-09-28 16:49 ` [PATCH v6 07/12] eventdev/dma: support adapter service ID get Amit Prakash Shukla
2023-09-28 16:49 ` [PATCH v6 08/12] eventdev/dma: support adapter runtime params Amit Prakash Shukla
2023-09-28 16:49 ` [PATCH v6 09/12] eventdev/dma: support adapter stats Amit Prakash Shukla
2023-09-28 16:49 ` [PATCH v6 10/12] eventdev/dma: support adapter enqueue Amit Prakash Shukla
2023-09-28 16:49 ` [PATCH v6 11/12] eventdev/dma: support adapter event port get Amit Prakash Shukla
2023-09-28 16:49 ` [PATCH v6 12/12] app/test: add event DMA adapter auto-test Amit Prakash Shukla
2023-09-29 8:12 ` [PATCH v7 00/12] event DMA adapter library support Amit Prakash Shukla
2023-09-29 8:12 ` [PATCH v7 01/12] eventdev/dma: introduce DMA adapter Amit Prakash Shukla
2023-09-29 8:12 ` [PATCH v7 02/12] eventdev/dma: support adapter capabilities get Amit Prakash Shukla
2023-09-29 8:13 ` [PATCH v7 03/12] eventdev/dma: support adapter create and free Amit Prakash Shukla
2023-09-29 8:13 ` [PATCH v7 04/12] eventdev/dma: support vchan add and delete Amit Prakash Shukla
2023-09-29 8:13 ` [PATCH v7 05/12] eventdev/dma: support adapter service function Amit Prakash Shukla
2023-09-29 8:13 ` [PATCH v7 06/12] eventdev/dma: support adapter start and stop Amit Prakash Shukla
2023-09-29 8:13 ` [PATCH v7 07/12] eventdev/dma: support adapter service ID get Amit Prakash Shukla
2023-09-29 8:13 ` [PATCH v7 08/12] eventdev/dma: support adapter runtime params Amit Prakash Shukla
2023-09-29 8:13 ` [PATCH v7 09/12] eventdev/dma: support adapter stats Amit Prakash Shukla
2023-09-29 8:13 ` [PATCH v7 10/12] eventdev/dma: support adapter enqueue Amit Prakash Shukla
2023-09-29 8:13 ` [PATCH v7 11/12] eventdev/dma: support adapter event port get Amit Prakash Shukla
2023-09-29 8:13 ` [PATCH v7 12/12] app/test: add event DMA adapter auto-test Amit Prakash Shukla
2023-09-29 11:50 ` [PATCH v8 00/12] event DMA adapter library support Amit Prakash Shukla
2023-09-29 11:50 ` [PATCH v8 01/12] eventdev/dma: introduce DMA adapter Amit Prakash Shukla
2023-09-29 11:50 ` [PATCH v8 02/12] eventdev/dma: support adapter capabilities get Amit Prakash Shukla
2023-09-29 11:50 ` [PATCH v8 03/12] eventdev/dma: support adapter create and free Amit Prakash Shukla
2023-09-29 11:50 ` [PATCH v8 04/12] eventdev/dma: support vchan add and delete Amit Prakash Shukla
2023-09-29 11:50 ` [PATCH v8 05/12] eventdev/dma: support adapter service function Amit Prakash Shukla
2023-09-29 11:50 ` [PATCH v8 06/12] eventdev/dma: support adapter start and stop Amit Prakash Shukla
2023-09-29 11:50 ` [PATCH v8 07/12] eventdev/dma: support adapter service ID get Amit Prakash Shukla
2023-09-29 11:50 ` [PATCH v8 08/12] eventdev/dma: support adapter runtime params Amit Prakash Shukla
2023-09-29 11:50 ` [PATCH v8 09/12] eventdev/dma: support adapter stats Amit Prakash Shukla
2023-09-29 11:50 ` [PATCH v8 10/12] eventdev/dma: support adapter enqueue Amit Prakash Shukla
2023-09-29 11:50 ` [PATCH v8 11/12] eventdev/dma: support adapter event port get Amit Prakash Shukla
2023-09-29 11:50 ` [PATCH v8 12/12] app/test: add event DMA adapter auto-test Amit Prakash Shukla
2023-10-03 5:13 ` [PATCH v8 00/12] event DMA adapter library support Jerin Jacob
2023-09-26 5:06 ` [PATCH v2 " Jerin Jacob
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).