From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 678A842660; Thu, 28 Sep 2023 12:36:50 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 5581140A6F; Thu, 28 Sep 2023 12:36:50 +0200 (CEST) Received: from mx0b-0016f401.pphosted.com (mx0a-0016f401.pphosted.com [67.231.148.174]) by mails.dpdk.org (Postfix) with ESMTP id B49AB4029D for ; Thu, 28 Sep 2023 12:36:48 +0200 (CEST) Received: from pps.filterd (m0045849.ppops.net [127.0.0.1]) by mx0a-0016f401.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 38SAATfA022971; Thu, 28 Sep 2023 03:36:48 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-type : content-transfer-encoding; s=pfpt0220; bh=ru75LCftSjxiLM+NCoC6DarAoF7dPhhg3ZMvIAQ6IUA=; b=eRyQZi5TgrTIwyd42agt9PBKwe3ccgc+/4NBB7WH83e1HLyVmFq4EtVMBi+JYrRQjRYk IWkatddrcKljL8H1GJKCw5ouy5/ZV8ImUEs9pcb9l9hKD9lDHdLoSs9sruUw9hVir/BK D2BWeCAqfBvgMmWTJyFQhhWfuT9WzjrV4xk5eyPmHRMhiaOBCvjN89G1wUu9wn8QYonN H4lnkUOm/8acYYYS9nnOn1OMjg7j57mJyc/bdoKs+r3SMlDFPWNeyFObi1T6qMt1i9mT rWEvYW6WSGNZ2EcMyAGsA4AXttZV17klENQRWoW8UBVDRb84hg3YjIDjoL80HGU1CemW Zg== Received: from dc5-exch02.marvell.com ([199.233.59.182]) by mx0a-0016f401.pphosted.com (PPS) with ESMTPS id 3tcu1qthft-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT); Thu, 28 Sep 2023 03:36:47 -0700 Received: from DC5-EXCH02.marvell.com (10.69.176.39) by DC5-EXCH02.marvell.com (10.69.176.39) with Microsoft SMTP Server (TLS) id 15.0.1497.48; Thu, 28 Sep 2023 03:36:44 -0700 Received: from maili.marvell.com (10.69.176.80) by DC5-EXCH02.marvell.com (10.69.176.39) with Microsoft SMTP Server id 15.0.1497.48 via Frontend Transport; Thu, 28 Sep 2023 03:36:44 -0700 Received: from localhost.localdomain (unknown [10.28.36.157]) by maili.marvell.com (Postfix) with ESMTP id CC6C45C68EF; Thu, 28 Sep 2023 03:36:39 -0700 (PDT) From: Amit Prakash Shukla To: Thomas Monjalon , Amit Prakash Shukla , Jerin Jacob CC: , , , , , , , , , , , , Subject: [PATCH v5 01/12] eventdev/dma: introduce DMA adapter Date: Thu, 28 Sep 2023 16:06:12 +0530 Message-ID: <20230928103623.216287-2-amitprakashs@marvell.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20230928103623.216287-1-amitprakashs@marvell.com> References: <20230926103233.3956431-1-amitprakashs@marvell.com> <20230928103623.216287-1-amitprakashs@marvell.com> MIME-Version: 1.0 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 8bit X-Proofpoint-GUID: crNFViidAzuvODkDRnkQwCoHAJI877AM X-Proofpoint-ORIG-GUID: crNFViidAzuvODkDRnkQwCoHAJI877AM X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.267,Aquarius:18.0.980,Hydra:6.0.619,FMLib:17.11.176.26 definitions=2023-09-28_08,2023-09-28_01,2023-05-22_02 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Introduce event dma adapter interface to transfer packets between dma device and event device. Signed-off-by: Amit Prakash Shukla Acked-by: Jerin Jacob --- MAINTAINERS | 8 +- doc/api/doxy-api-index.md | 1 + doc/guides/eventdevs/features/default.ini | 8 + doc/guides/prog_guide/event_dma_adapter.rst | 264 ++++ doc/guides/prog_guide/eventdev.rst | 8 +- .../img/event_dma_adapter_op_forward.svg | 1086 +++++++++++++++++ .../img/event_dma_adapter_op_new.svg | 1079 ++++++++++++++++ doc/guides/prog_guide/index.rst | 1 + doc/guides/rel_notes/release_23_11.rst | 5 + lib/eventdev/eventdev_pmd.h | 171 ++- lib/eventdev/eventdev_private.c | 10 + lib/eventdev/meson.build | 1 + lib/eventdev/rte_event_dma_adapter.h | 581 +++++++++ lib/eventdev/rte_eventdev.h | 44 + lib/eventdev/rte_eventdev_core.h | 8 +- lib/eventdev/version.map | 16 + lib/meson.build | 2 +- 17 files changed, 3286 insertions(+), 7 deletions(-) create mode 100644 doc/guides/prog_guide/event_dma_adapter.rst create mode 100644 doc/guides/prog_guide/img/event_dma_adapter_op_forward.svg create mode 100644 doc/guides/prog_guide/img/event_dma_adapter_op_new.svg create mode 100644 lib/eventdev/rte_event_dma_adapter.h diff --git a/MAINTAINERS b/MAINTAINERS index a926155f26..e731a75d39 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -540,6 +540,13 @@ F: lib/eventdev/*crypto_adapter* F: app/test/test_event_crypto_adapter.c F: doc/guides/prog_guide/event_crypto_adapter.rst +Eventdev DMA Adapter API +M: Amit Prakash Shukla +T: git://dpdk.org/next/dpdk-next-eventdev +F: lib/eventdev/*dma_adapter* +F: app/test/test_event_dma_adapter.c +F: doc/guides/prog_guide/event_dma_adapter.rst + Raw device API M: Sachin Saxena M: Hemant Agrawal @@ -548,7 +555,6 @@ F: drivers/raw/skeleton/ F: app/test/test_rawdev.c F: doc/guides/prog_guide/rawdev.rst - Memory Pool Drivers ------------------- diff --git a/doc/api/doxy-api-index.md b/doc/api/doxy-api-index.md index fdeda13932..b7df7be4d9 100644 --- a/doc/api/doxy-api-index.md +++ b/doc/api/doxy-api-index.md @@ -29,6 +29,7 @@ The public API headers are grouped by topics: [event_eth_tx_adapter](@ref rte_event_eth_tx_adapter.h), [event_timer_adapter](@ref rte_event_timer_adapter.h), [event_crypto_adapter](@ref rte_event_crypto_adapter.h), + [event_dma_adapter](@ref rte_event_dma_adapter.h), [rawdev](@ref rte_rawdev.h), [metrics](@ref rte_metrics.h), [bitrate](@ref rte_bitrate.h), diff --git a/doc/guides/eventdevs/features/default.ini b/doc/guides/eventdevs/features/default.ini index 00360f60c6..73a52d915b 100644 --- a/doc/guides/eventdevs/features/default.ini +++ b/doc/guides/eventdevs/features/default.ini @@ -44,6 +44,14 @@ internal_port_op_fwd = internal_port_qp_ev_bind = session_private_data = +; +; Features of a default DMA adapter. +; +[DMA adapter Features] +internal_port_op_new = +internal_port_op_fwd = +internal_port_vchan_ev_bind = + ; ; Features of a default Timer adapter. ; diff --git a/doc/guides/prog_guide/event_dma_adapter.rst b/doc/guides/prog_guide/event_dma_adapter.rst new file mode 100644 index 0000000000..701e50d042 --- /dev/null +++ b/doc/guides/prog_guide/event_dma_adapter.rst @@ -0,0 +1,264 @@ +.. SPDX-License-Identifier: BSD-3-Clause + Copyright (c) 2023 Marvell. + +Event DMA Adapter Library +========================= + +DPDK :doc:`Eventdev library ` provides event driven programming model with features +to schedule events. :doc:`DMA Device library ` provides an interface to DMA poll mode +drivers that support DMA operations. Event DMA Adapter is intended to bridge between the event +device and the DMA device. + +Packet flow from DMA device to the event device can be accomplished using software and hardware +based transfer mechanisms. The adapter queries an eventdev PMD to determine which mechanism to +be used. The adapter uses an EAL service core function for software based packet transfer and +uses the eventdev PMD functions to configure hardware based packet transfer between DMA device +and the event device. DMA adapter uses a new event type called ``RTE_EVENT_TYPE_DMADEV`` to +indicate the source of event. + +Application can choose to submit an DMA operation directly to an DMA device or send it to an DMA +adapter via eventdev based on ``RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP_FWD`` capability. The +first mode is known as the event new (``RTE_EVENT_DMA_ADAPTER_OP_NEW``) mode and the second as the +event forward (``RTE_EVENT_DMA_ADAPTER_OP_FORWARD``) mode. Choice of mode can be specified while +creating the adapter. In the former mode, it is the application's responsibility to enable +ingress packet ordering. In the latter mode, it is the adapter's responsibility to enable +ingress packet ordering. + + +Adapter Modes +------------- + +RTE_EVENT_DMA_ADAPTER_OP_NEW mode +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +In the ``RTE_EVENT_DMA_ADAPTER_OP_NEW`` mode, application submits DMA operations directly to an DMA +device. The adapter then dequeues DMA completions from the DMA device and enqueues them as events +to the event device. This mode does not ensure ingress ordering as the application directly +enqueues to the dmadev without going through DMA/atomic stage. In this mode, events dequeued +from the adapter are treated as new events. The application has to specify event information +(response information) which is needed to enqueue an event after the DMA operation is completed. + +.. _figure_event_dma_adapter_op_new: + +.. figure:: img/event_dma_adapter_op_new.* + + Working model of ``RTE_EVENT_DMA_ADAPTER_OP_NEW`` mode + + +RTE_EVENT_DMA_ADAPTER_OP_FORWARD mode +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +In the ``RTE_EVENT_DMA_ADAPTER_OP_FORWARD`` mode, if the event PMD and DMA PMD supports internal +event port (``RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP_FWD``), the application should use +``rte_event_dma_adapter_enqueue()`` API to enqueue DMA operations as events to DMA adapter. If +not, application retrieves DMA adapter's event port using ``rte_event_dma_adapter_event_port_get()`` +API, links its event queue to this port and starts enqueuing DMA operations as events to eventdev +using ``rte_event_enqueue_burst()``. The adapter then dequeues the events and submits the DMA +operations to the dmadev. After the DMA operation is complete, the adapter enqueues events to the +event device. + +Applications can use this mode when ingress packet ordering is needed. In this mode, events +dequeued from the adapter will be treated as forwarded events. Application has to specify event +information (response information) needed to enqueue the event after the DMA operation has +completed. + +.. _figure_event_dma_adapter_op_forward: + +.. figure:: img/event_dma_adapter_op_forward.* + + Working model of ``RTE_EVENT_DMA_ADAPTER_OP_FORWARD`` mode + + +API Overview +------------ + +This section has a brief introduction to the event DMA adapter APIs. The application is expected +to create an adapter which is associated with a single eventdev, then add dmadev and vchan to the +adapter instance. + + +Create an adapter instance +~~~~~~~~~~~~~~~~~~~~~~~~~~ + +An adapter instance is created using ``rte_event_dma_adapter_create()``. This function is called +with event device to be associated with the adapter and port configuration for the adapter to +setup an event port (if the adapter needs to use a service function). + +Adapter can be started in ``RTE_EVENT_DMA_ADAPTER_OP_NEW`` or ``RTE_EVENT_DMA_ADAPTER_OP_FORWARD`` +mode. + +.. code-block:: c + + enum rte_event_dma_adapter_mode mode; + struct rte_event_dev_info dev_info; + struct rte_event_port_conf conf; + uint8_t evdev_id; + uint8_t dma_id; + int ret; + + ret = rte_event_dev_info_get(dma_id, &dev_info); + + conf.new_event_threshold = dev_info.max_num_events; + conf.dequeue_depth = dev_info.max_event_port_dequeue_depth; + conf.enqueue_depth = dev_info.max_event_port_enqueue_depth; + mode = RTE_EVENT_DMA_ADAPTER_OP_FORWARD; + ret = rte_event_dma_adapter_create(dma_id, evdev_id, &conf, mode); + + +``rte_event_dma_adapter_create_ext()`` function can be used by the application to have a finer +control on eventdev port allocation and setup. The ``rte_event_dma_adapter_create_ext()`` +function is passed a callback function. The callback function is invoked if the adapter creates +a service function and uses an event port for it. The callback is expected to fill the +``struct rte_event_dma_adapter_conf`` structure passed to it. + +In the ``RTE_EVENT_DMA_ADAPTER_OP_FORWARD`` mode, if the event PMD and DMA PMD supports internal +event port (``RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP_FWD``), events with DMA operations should +be enqueued to the DMA adapter using ``rte_event_dma_adapter_enqueue()`` API. If not, the event port +created by the adapter can be retrieved using ``rte_event_dma_adapter_event_port_get()`` API. An +application can use this event port to link with an event queue, on which it enqueues events +towards the DMA adapter using ``rte_event_enqueue_burst()``. + +.. code-block:: c + + uint8_t dma_adpt_id, evdev_id, dma_dev_id, dma_ev_port_id, app_qid; + struct rte_event ev; + uint32_t cap; + int ret; + + // Fill in event info and update event_ptr with rte_dma_op + memset(&ev, 0, sizeof(ev)); + . + . + ev.event_ptr = op; + + ret = rte_event_dma_adapter_caps_get(evdev_id, dma_dev_id, &cap); + if (cap & RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP_FWD) { + ret = rte_event_dma_adapter_enqueue(evdev_id, app_ev_port_id, ev, nb_events); + } else { + ret = rte_event_dma_adapter_event_port_get(dma_adpt_id, &dma_ev_port_id); + ret = rte_event_queue_setup(evdev_id, app_qid, NULL); + ret = rte_event_port_link(evdev_id, dma_ev_port_id, &app_qid, NULL, 1); + ev.queue_id = app_qid; + ret = rte_event_enqueue_burst(evdev_id, app_ev_port_id, ev, nb_events); + } + + +Event device configuration for service based adapter +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +When ``rte_event_dma_adapter_create()`` is used for creating adapter instance, +``rte_event_dev_config::nb_event_ports`` is automatically incremented, and event device is +reconfigured with additional event port during service initialization. This event device +reconfigure logic also increments the ``rte_event_dev_config::nb_single_link_event_port_queues`` +parameter if the adapter event port config is of type ``RTE_EVENT_PORT_CFG_SINGLE_LINK``. + +Applications using this mode of adapter creation need not configure the event device with +``rte_event_dev_config::nb_event_ports`` and +``rte_event_dev_config::nb_single_link_event_port_queues`` parameters required for DMA adapter when +the adapter is created using the above-mentioned API. + + +Querying adapter capabilities +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +The ``rte_event_dma_adapter_caps_get()`` function allows the application to query the adapter +capabilities for an eventdev and dmadev combination. This API provides whether dmadev and eventdev +are connected using internal HW port or not. + +.. code-block:: c + + rte_event_dma_adapter_caps_get(dev_id, dma_dev_id, &cap); + + +Adding vchan to the adapter instance +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +dmadev device id and vchan are configured using dmadev APIs. For more information +see :doc:`here `. + +.. code-block:: c + + struct rte_dma_vchan_conf vchan_conf; + struct rte_dma_conf dev_conf; + uint8_t dev_id = 0; + uint16_t vchan = 0; + + rte_dma_configure(dev_id, &dev_conf); + rte_dma_vchan_setup(dev_id, vchan, &vchan_conf); + +These dmadev id and vchan are added to the instance using the +``rte_event_dma_adapter_vchan_add()`` API. The same is removed using +``rte_event_dma_adapter_vchan_del()`` API. If hardware supports +``RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_VCHAN_EV_BIND`` capability, event information must be passed to the add API. + +.. code-block:: c + + uint32_t cap; + int ret; + + ret = rte_event_dma_adapter_caps_get(evdev_id, dma_dev_id, &cap); + if (cap & RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_VCHAN_EV_BIND) { + struct rte_event event; + + rte_event_dma_adapter_vchan_add(id, dma_dev_id, vchan, &conf); + } else + rte_event_dma_adapter_vchan_add(id, dma_dev_id, vchan, NULL); + + +Configuring service function +~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +If the adapter uses a service function, the application is required to assign a service core to +the service function as show below. + +.. code-block:: c + + uint32_t service_id; + + if (rte_event_dma_adapter_service_id_get(dma_id, &service_id) == 0) + rte_service_map_lcore_set(service_id, CORE_ID); + + +Set event response information +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +In the RTE_EVENT_DMA_ADAPTER_OP_FORWARD / RTE_EVENT_DMA_ADAPTER_OP_NEW mode, the application +specifies the dmadev ID and vchan ID in ``struct rte_event_dma_adapter_op`` and the event +information (response information) needed to enqueue an event after the DMA operation has +completed. The response information is specified in ``struct rte_event`` and appended to the +``struct rte_event_dma_adapter_op``. + + +Start the adapter instance +~~~~~~~~~~~~~~~~~~~~~~~~~~ + +The application calls ``rte_event_dma_adapter_start()`` to start the adapter. This function calls +the start callbacks of the eventdev PMDs for hardware based eventdev-dmadev connections and +``rte_service_run_state_set()`` to enable the service function if one exists. + +.. code-block:: c + + rte_event_dma_adapter_start(id); + +.. Note:: + + The eventdev to which the event_dma_adapter is connected should be started before calling + rte_event_dma_adapter_start(). + + +Get adapter statistics +~~~~~~~~~~~~~~~~~~~~~~ + +The ``rte_event_dma_adapter_stats_get()`` function reports counters defined in struct +``rte_event_dma_adapter_stats``. The received packet and enqueued event counts are a sum of the +counts from the eventdev PMD callbacks if the callback is supported, and the counts maintained by +the service function, if one exists. + +Set/Get adapter runtime configuration parameters +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +The runtime configuration parameters of adapter can be set/get using +``rte_event_dma_adapter_runtime_params_set()`` and +``rte_event_dma_adapter_runtime_params_get()`` respectively. +The parameters that can be set/get are defined in +``struct rte_event_dma_adapter_runtime_params``. diff --git a/doc/guides/prog_guide/eventdev.rst b/doc/guides/prog_guide/eventdev.rst index 2c83176846..ff55115d0d 100644 --- a/doc/guides/prog_guide/eventdev.rst +++ b/doc/guides/prog_guide/eventdev.rst @@ -333,7 +333,8 @@ eventdev. .. Note:: EventDev needs to be started before starting the event producers such - as event_eth_rx_adapter, event_timer_adapter and event_crypto_adapter. + as event_eth_rx_adapter, event_timer_adapter, event_crypto_adapter and + event_dma_adapter. Ingress of New Events ~~~~~~~~~~~~~~~~~~~~~ @@ -445,8 +446,9 @@ using ``rte_event_dev_stop_flush_callback_register()`` function. .. Note:: The event producers such as ``event_eth_rx_adapter``, - ``event_timer_adapter`` and ``event_crypto_adapter`` - need to be stopped before stopping the event device. + ``event_timer_adapter``, ``event_crypto_adapter`` and + ``event_dma_adapter`` need to be stopped before stopping + the event device. Summary ------- diff --git a/doc/guides/prog_guide/img/event_dma_adapter_op_forward.svg b/doc/guides/prog_guide/img/event_dma_adapter_op_forward.svg new file mode 100644 index 0000000000..b7fe1fecf2 --- /dev/null +++ b/doc/guides/prog_guide/img/event_dma_adapter_op_forward.svg @@ -0,0 +1,1086 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + image/svg+xml + + + + + + + + Square + Atomic Queue #1 + + + + + + + + + + + + + + + + + + Square + Atomic Queue #1 + + + + + + + + + + + + + + + + + + + + + + 1 + + + 2 + + + + 8 + + + + + 7 + + + + + 3 + + + + 4 + + + 5 + + + Square + Atomic Queue #1 + + + + + + + + + + + + + + + + Square + Atomic Queue #1 + + + + + + + + + + + + + + + + + + 6 + + + Eventdev + + + DMAAdapter + + + Applicationin orderedstage + + + DMA Device + + + 1. Events from the previous stage. 2. Application in ordered stage dequeues events from eventdev. 3. Application enqueues DMA operations as events to eventdev. 4. DMA adapter dequeues event from eventdev. 5. DMA adapter submits DMA operations to DMA Device (Atomic stage) 6. DMA adapter dequeues DMA completions from DMA Device 7. DMA adapter enqueues events to the eventdev 8. Events to the next stage + + + diff --git a/doc/guides/prog_guide/img/event_dma_adapter_op_new.svg b/doc/guides/prog_guide/img/event_dma_adapter_op_new.svg new file mode 100644 index 0000000000..e9e8bb2b98 --- /dev/null +++ b/doc/guides/prog_guide/img/event_dma_adapter_op_new.svg @@ -0,0 +1,1079 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + image/svg+xml + + + + + + + + Square + Atomic Queue #1 + + + + + + + + + + + + + + + + Square + Atomic Queue #1 + + + + + + + + + + + + + + + + Square + Atomic Queue #1 + + + + + + + + + + + + + + + + + Square + Atomic Queue #1 + + + + + + + + + + + + + + + + +   + + + + + + + + + + + + + 1 + + + 2 + + + + + 3 + + + 4 + + + 6 + + + Eventdev + + + Atomic Stage+Enqueue toDMA Device + + + 5 + +   + + DMA Device + + + DMAAdapter + + + 1. Application dequeues events from the previous stage 2. Application prepares the DMA operations. 3. DMA operations are submitted to dmadev by application. 4. DMA adapter dequeues DMA completions from DMA device. 5. DMA adapter enqueues events to the eventdev. 6. Application dequeues from eventdev and prepare for further processing + + + Square + Atomic Queue #1 + + + + + + + + + + + + + + + + Application + + + diff --git a/doc/guides/prog_guide/index.rst b/doc/guides/prog_guide/index.rst index 52a6d9e7aa..beaa4b8869 100644 --- a/doc/guides/prog_guide/index.rst +++ b/doc/guides/prog_guide/index.rst @@ -60,6 +60,7 @@ Programmer's Guide event_ethernet_tx_adapter event_timer_adapter event_crypto_adapter + event_dma_adapter qos_framework power_man packet_classif_access_ctrl diff --git a/doc/guides/rel_notes/release_23_11.rst b/doc/guides/rel_notes/release_23_11.rst index b34ddc0860..1a1f337d23 100644 --- a/doc/guides/rel_notes/release_23_11.rst +++ b/doc/guides/rel_notes/release_23_11.rst @@ -89,6 +89,11 @@ New Features * Added support for ``remaining_ticks_get`` timer adapter PMD callback to get the remaining ticks to expire for a given event timer. +* **Added event DMA adapter library.** + + * Added the Event DMA Adapter Library. This library extends the event-based + model by introducing APIs that allow applications to enqueue/dequeue DMA + operations to/from dmadev as events scheduled by an event device. Removed Items ------------- diff --git a/lib/eventdev/eventdev_pmd.h b/lib/eventdev/eventdev_pmd.h index f62f42e140..f7227c0bfd 100644 --- a/lib/eventdev/eventdev_pmd.h +++ b/lib/eventdev/eventdev_pmd.h @@ -178,8 +178,12 @@ struct rte_eventdev { event_tx_adapter_enqueue_t txa_enqueue; /**< Pointer to PMD eth Tx adapter enqueue function. */ event_crypto_adapter_enqueue_t ca_enqueue; + /**< Pointer to PMD crypto adapter enqueue function. */ - uint64_t reserved_64s[4]; /**< Reserved for future fields */ + event_dma_adapter_enqueue_t dma_enqueue; + /**< Pointer to PMD DMA adapter enqueue function. */ + + uint64_t reserved_64s[3]; /**< Reserved for future fields */ void *reserved_ptrs[3]; /**< Reserved for future fields */ } __rte_cache_aligned; @@ -1320,6 +1324,156 @@ typedef int (*eventdev_eth_tx_adapter_queue_stop) #define eventdev_stop_flush_t rte_eventdev_stop_flush_t +/** + * Retrieve the event device's DMA adapter capabilities for the + * specified DMA device + * + * @param dev + * Event device pointer + * + * @param dma_dev_id + * DMA device identifier + * + * @param[out] caps + * A pointer to memory filled with event adapter capabilities. + * It is expected to be pre-allocated & initialized by caller. + * + * @return + * - 0: Success, driver provides event adapter capabilities for the + * dmadev. + * - <0: Error code returned by the driver function. + * + */ +typedef int (*eventdev_dma_adapter_caps_get_t)(const struct rte_eventdev *dev, + const int16_t dma_dev_id, uint32_t *caps); + +/** + * Add DMA vchan queue to event device. This callback is invoked if + * the caps returned from rte_event_dma_adapter_caps_get(, dmadev_id) + * has RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_* set. + * + * @param dev + * Event device pointer + * + * @param dma_dev_id + * DMA device identifier + * + * @param vchan_id + * dmadev vchan queue identifier. + * + * @param event + * Event information required for binding dmadev vchan to event queue. + * This structure will have a valid value for only those HW PMDs supporting + * @see RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_VCHAN_EV_BIND capability. + * + * @return + * - 0: Success, dmadev vchan added successfully. + * - <0: Error code returned by the driver function. + * + */ +typedef int (*eventdev_dma_adapter_vchan_add_t)(const struct rte_eventdev *dev, + const int16_t dma_dev_id, + uint16_t vchan_id, + const struct rte_event *event); + +/** + * Delete DMA vhcan to event device. This callback is invoked if + * the caps returned from rte_event_dma_adapter_caps_get(, dmadev_id) + * has RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_* set. + * + * @param dev + * Event device pointer + * + * @param dma_dev_id + * DMA device identifier + * + * @param vchan_id + * dmadev vchan identifier. + * + * @return + * - 0: Success, dmadev vchan deleted successfully. + * - <0: Error code returned by the driver function. + * + */ +typedef int (*eventdev_dma_adapter_vchan_del_t)(const struct rte_eventdev *dev, + const int16_t dma_dev_id, + uint16_t vchan_id); + +/** + * Start DMA adapter. This callback is invoked if + * the caps returned from rte_event_dma_adapter_caps_get(.., dmadev_id) + * has RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_* set and vchan for dmadev_id + * have been added to the event device. + * + * @param dev + * Event device pointer + * + * @param dma_dev_id + * DMA device identifier + * + * @return + * - 0: Success, DMA adapter started successfully. + * - <0: Error code returned by the driver function. + */ +typedef int (*eventdev_dma_adapter_start_t)(const struct rte_eventdev *dev, + const int16_t dma_dev_id); + +/** + * Stop DMA adapter. This callback is invoked if + * the caps returned from rte_event_dma_adapter_caps_get(.., dmadev_id) + * has RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_* set and vchan for dmadev_id + * have been added to the event device. + * + * @param dev + * Event device pointer + * + * @param dma_dev_id + * DMA device identifier + * + * @return + * - 0: Success, DMA adapter stopped successfully. + * - <0: Error code returned by the driver function. + */ +typedef int (*eventdev_dma_adapter_stop_t)(const struct rte_eventdev *dev, + const int16_t dma_dev_id); + +struct rte_event_dma_adapter_stats; + +/** + * Retrieve DMA adapter statistics. + * + * @param dev + * Event device pointer + * + * @param dma_dev_id + * DMA device identifier + * + * @param[out] stats + * Pointer to stats structure + * + * @return + * Return 0 on success. + */ +typedef int (*eventdev_dma_adapter_stats_get)(const struct rte_eventdev *dev, + const int16_t dma_dev_id, + struct rte_event_dma_adapter_stats *stats); + +/** + * Reset DMA adapter statistics. + * + * @param dev + * Event device pointer + * + * @param dma_dev_id + * DMA device identifier + * + * @return + * Return 0 on success. + */ +typedef int (*eventdev_dma_adapter_stats_reset)(const struct rte_eventdev *dev, + const int16_t dma_dev_id); + + /** Event device operations function pointer table */ struct eventdev_ops { eventdev_info_get_t dev_infos_get; /**< Get device info. */ @@ -1440,6 +1594,21 @@ struct eventdev_ops { eventdev_eth_tx_adapter_queue_stop eth_tx_adapter_queue_stop; /**< Stop Tx queue assigned to Tx adapter instance */ + eventdev_dma_adapter_caps_get_t dma_adapter_caps_get; + /**< Get DMA adapter capabilities */ + eventdev_dma_adapter_vchan_add_t dma_adapter_vchan_add; + /**< Add vchan queue to DMA adapter */ + eventdev_dma_adapter_vchan_del_t dma_adapter_vchan_del; + /**< Delete vchan queue from DMA adapter */ + eventdev_dma_adapter_start_t dma_adapter_start; + /**< Start DMA adapter */ + eventdev_dma_adapter_stop_t dma_adapter_stop; + /**< Stop DMA adapter */ + eventdev_dma_adapter_stats_get dma_adapter_stats_get; + /**< Get DMA stats */ + eventdev_dma_adapter_stats_reset dma_adapter_stats_reset; + /**< Reset DMA stats */ + eventdev_selftest dev_selftest; /**< Start eventdev Selftest */ diff --git a/lib/eventdev/eventdev_private.c b/lib/eventdev/eventdev_private.c index 1d3d9d357e..18ed8bf3c8 100644 --- a/lib/eventdev/eventdev_private.c +++ b/lib/eventdev/eventdev_private.c @@ -81,6 +81,14 @@ dummy_event_crypto_adapter_enqueue(__rte_unused void *port, return 0; } +static uint16_t +dummy_event_dma_adapter_enqueue(__rte_unused void *port, __rte_unused struct rte_event ev[], + __rte_unused uint16_t nb_events) +{ + RTE_EDEV_LOG_ERR("event DMA adapter enqueue requested for unconfigured event device"); + return 0; +} + void event_dev_fp_ops_reset(struct rte_event_fp_ops *fp_op) { @@ -97,6 +105,7 @@ event_dev_fp_ops_reset(struct rte_event_fp_ops *fp_op) .txa_enqueue_same_dest = dummy_event_tx_adapter_enqueue_same_dest, .ca_enqueue = dummy_event_crypto_adapter_enqueue, + .dma_enqueue = dummy_event_dma_adapter_enqueue, .data = dummy_data, }; @@ -117,5 +126,6 @@ event_dev_fp_ops_set(struct rte_event_fp_ops *fp_op, fp_op->txa_enqueue = dev->txa_enqueue; fp_op->txa_enqueue_same_dest = dev->txa_enqueue_same_dest; fp_op->ca_enqueue = dev->ca_enqueue; + fp_op->dma_enqueue = dev->dma_enqueue; fp_op->data = dev->data->ports; } diff --git a/lib/eventdev/meson.build b/lib/eventdev/meson.build index 6edf98dfa5..21347f7c4c 100644 --- a/lib/eventdev/meson.build +++ b/lib/eventdev/meson.build @@ -25,6 +25,7 @@ sources = files( ) headers = files( 'rte_event_crypto_adapter.h', + 'rte_event_dma_adapter.h', 'rte_event_eth_rx_adapter.h', 'rte_event_eth_tx_adapter.h', 'rte_event_ring.h', diff --git a/lib/eventdev/rte_event_dma_adapter.h b/lib/eventdev/rte_event_dma_adapter.h new file mode 100644 index 0000000000..e924ab673d --- /dev/null +++ b/lib/eventdev/rte_event_dma_adapter.h @@ -0,0 +1,581 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright (c) 2023 Marvell. + */ + +#ifndef RTE_EVENT_DMA_ADAPTER +#define RTE_EVENT_DMA_ADAPTER + +/** + * @file rte_event_dma_adapter.h + * + * @warning + * @b EXPERIMENTAL: + * All functions in this file may be changed or removed without prior notice. + * + * DMA Event Adapter API. + * + * Eventdev library provides adapters to bridge between various components for providing new + * event source. The event DMA adapter is one of those adapters which is intended to bridge + * between event devices and DMA devices. + * + * The DMA adapter adds support to enqueue / dequeue DMA operations to / from event device. The + * packet flow between DMA device and the event device can be accomplished using both SW and HW + * based transfer mechanisms. The adapter uses an EAL service core function for SW based packet + * transfer and uses the eventdev PMD functions to configure HW based packet transfer between the + * DMA device and the event device. + * + * The application can choose to submit a DMA operation directly to an DMA device or send it to the + * DMA adapter via eventdev based on RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP_FWD capability. The + * first mode is known as the event new (RTE_EVENT_DMA_ADAPTER_OP_NEW) mode and the second as the + * event forward (RTE_EVENT_DMA_ADAPTER_OP_FORWARD) mode. The choice of mode can be specified while + * creating the adapter. In the former mode, it is an application responsibility to enable ingress + * packet ordering. In the latter mode, it is the adapter responsibility to enable the ingress + * packet ordering. + * + * + * Working model of RTE_EVENT_DMA_ADAPTER_OP_NEW mode: + * + * +--------------+ +--------------+ + * | | | DMA stage | + * | Application |---[2]-->| + enqueue to | + * | | | dmadev | + * +--------------+ +--------------+ + * ^ ^ | + * | | [3] + * [6] [1] | + * | | | + * +--------------+ | + * | | | + * | Event device | | + * | | | + * +--------------+ | + * ^ | + * | | + * [5] | + * | v + * +--------------+ +--------------+ + * | | | | + * | DMA adapter |<--[4]---| dmadev | + * | | | | + * +--------------+ +--------------+ + * + * + * [1] Application dequeues events from the previous stage. + * [2] Application prepares the DMA operations. + * [3] DMA operations are submitted to dmadev by application. + * [4] DMA adapter dequeues DMA completions from dmadev. + * [5] DMA adapter enqueues events to the eventdev. + * [6] Application dequeues from eventdev for further processing. + * + * In the RTE_EVENT_DMA_ADAPTER_OP_NEW mode, application submits DMA operations directly to DMA + * device. The DMA adapter then dequeues DMA completions from DMA device and enqueue events to the + * event device. This mode does not ensure ingress ordering, if the application directly enqueues + * to dmadev without going through DMA / atomic stage i.e. removing item [1] and [2]. + * + * Events dequeued from the adapter will be treated as new events. In this mode, application needs + * to specify event information (response information) which is needed to enqueue an event after the + * DMA operation is completed. + * + * + * Working model of RTE_EVENT_DMA_ADAPTER_OP_FORWARD mode: + * + * +--------------+ +--------------+ + * --[1]-->| |---[2]-->| Application | + * | Event device | | in | + * <--[8]--| |<--[3]---| Ordered stage| + * +--------------+ +--------------+ + * ^ | + * | [4] + * [7] | + * | v + * +----------------+ +--------------+ + * | |--[5]->| | + * | DMA adapter | | dmadev | + * | |<-[6]--| | + * +----------------+ +--------------+ + * + * + * [1] Events from the previous stage. + * [2] Application in ordered stage dequeues events from eventdev. + * [3] Application enqueues DMA operations as events to eventdev. + * [4] DMA adapter dequeues event from eventdev. + * [5] DMA adapter submits DMA operations to dmadev (Atomic stage). + * [6] DMA adapter dequeues DMA completions from dmadev + * [7] DMA adapter enqueues events to the eventdev + * [8] Events to the next stage + * + * In the event forward (RTE_EVENT_DMA_ADAPTER_OP_FORWARD) mode, if the HW supports the capability + * RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP_FWD, application can directly submit the DMA + * operations to the dmadev. If not, application retrieves the event port of the DMA adapter + * through the API, rte_event_DMA_adapter_event_port_get(). Then, links its event queue to this + * port and starts enqueuing DMA operations as events to the eventdev. The adapter then dequeues + * the events and submits the DMA operations to the dmadev. After the DMA completions, the adapter + * enqueues events to the event device. + * + * Application can use this mode, when ingress packet ordering is needed. Events dequeued from the + * adapter will be treated as forwarded events. In this mode, the application needs to specify the + * dmadev ID and queue pair ID (request information) needed to enqueue an DMA operation in addition + * to the event information (response information) needed to enqueue an event after the DMA + * operation has completed. + * + * The event DMA adapter provides common APIs to configure the packet flow from the DMA device to + * event devices for both SW and HW based transfers. The DMA event adapter's functions are: + * + * - rte_event_dma_adapter_create_ext() + * - rte_event_dma_adapter_create() + * - rte_event_dma_adapter_free() + * - rte_event_dma_adapter_vchan_add() + * - rte_event_dma_adapter_vchan_del() + * - rte_event_dma_adapter_start() + * - rte_event_dma_adapter_stop() + * - rte_event_dma_adapter_stats_get() + * - rte_event_dma_adapter_stats_reset() + * + * The application creates an instance using rte_event_dma_adapter_create() or + * rte_event_dma_adapter_create_ext(). + * + * dmadev queue pair addition / deletion is done using the rte_event_dma_adapter_vchan_add() / + * rte_event_dma_adapter_vchan_del() APIs. If HW supports the capability + * RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_VCHAN_EV_BIND, event information must be passed to the + * add API. + * + */ + +#include + +#include +#include +#include + +#ifdef __cplusplus +extern "C" { +#endif + +/** + * A structure used to hold event based DMA operation entry. All the information + * required for a DMA transfer shall be populated in "struct rte_event_dma_adapter_op" + * instance. + */ +struct rte_event_dma_adapter_op { + struct rte_dma_sge *src_seg; + /**< Source segments. */ + struct rte_dma_sge *dst_seg; + /**< Destination segments. */ + uint16_t nb_src; + /**< Number of source segments. */ + uint16_t nb_dst; + /**< Number of destination segments. */ + uint64_t flags; + /**< Flags related to the operation. + * @see RTE_DMA_OP_FLAG_* + */ + int16_t dma_dev_id; + /**< DMA device ID to be used */ + uint16_t vchan; + /**< DMA vchan ID to be used */ + struct rte_mempool *op_mp; + /**< Mempool from which op is allocated. */ +}; + +/** + * DMA event adapter mode + */ +enum rte_event_dma_adapter_mode { + RTE_EVENT_DMA_ADAPTER_OP_NEW, + /**< Start the DMA adapter in event new mode. + * @see RTE_EVENT_OP_NEW. + * + * Application submits DMA operations to the dmadev. Adapter only dequeues the DMA + * completions from dmadev and enqueue events to the eventdev. + */ + + RTE_EVENT_DMA_ADAPTER_OP_FORWARD, + /**< Start the DMA adapter in event forward mode. + * @see RTE_EVENT_OP_FORWARD. + * + * Application submits DMA requests as events to the DMA adapter or DMA device based on + * RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP_FWD capability. DMA completions are enqueued + * back to the eventdev by DMA adapter. + */ +}; + +/** + * Adapter configuration structure that the adapter configuration callback function is expected to + * fill out. + * + * @see rte_event_dma_adapter_conf_cb + */ +struct rte_event_dma_adapter_conf { + uint8_t event_port_id; + /** < Event port identifier, the adapter enqueues events to this port and dequeues DMA + * request events in RTE_EVENT_DMA_ADAPTER_OP_FORWARD mode. + */ + + uint32_t max_nb; + /**< The adapter can return early if it has processed at least max_nb DMA ops. This isn't + * treated as a requirement; batching may cause the adapter to process more than max_nb DMA + * ops. + */ +}; + +/** + * Adapter runtime configuration parameters + */ +struct rte_event_dma_adapter_runtime_params { + uint32_t max_nb; + /**< The adapter can return early if it has processed at least max_nb DMA ops. This isn't + * treated as a requirement; batching may cause the adapter to process more than max_nb DMA + * ops. + * + * Callback function passed to rte_event_dma_adapter_create_ext() configures the adapter + * with default value of max_nb. + * rte_event_dma_adapter_runtime_params_set() allows to re-configure max_nb during runtime + * (after adding at least one queue pair) + * + * This is valid for the devices without RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP_FWD or + * RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP_NEW capability. + */ + + uint32_t rsvd[15]; + /**< Reserved fields for future expansion */ +}; + +/** + * Function type used for adapter configuration callback. The callback is used to fill in members of + * the struct rte_event_dma_adapter_conf, this callback is invoked when creating a SW service for + * packet transfer from dmadev vchan to the event device. The SW service is created within the + * function, rte_event_dma_adapter_vchan_add(), if SW based packet transfers from dmadev vchan + * to the event device are required. + * + * @param id + * Adapter identifier. + * @param evdev_id + * Event device identifier. + * @param conf + * Structure that needs to be populated by this callback. + * @param arg + * Argument to the callback. This is the same as the conf_arg passed to the + * rte_event_dma_adapter_create_ext(). + */ +typedef int (*rte_event_dma_adapter_conf_cb)(uint8_t id, uint8_t evdev_id, + struct rte_event_dma_adapter_conf *conf, void *arg); + +/** + * A structure used to retrieve statistics for an event DMA adapter instance. + */ +struct rte_event_dma_adapter_stats { + uint64_t event_poll_count; + /**< Event port poll count */ + + uint64_t event_deq_count; + /**< Event dequeue count */ + + uint64_t dma_enq_count; + /**< dmadev enqueue count */ + + uint64_t dma_enq_fail_count; + /**< dmadev enqueue failed count */ + + uint64_t dma_deq_count; + /**< dmadev dequeue count */ + + uint64_t event_enq_count; + /**< Event enqueue count */ + + uint64_t event_enq_retry_count; + /**< Event enqueue retry count */ + + uint64_t event_enq_fail_count; + /**< Event enqueue fail count */ +}; + +/** + * Create a new event DMA adapter with the specified identifier. + * + * @param id + * Adapter identifier. + * @param evdev_id + * Event device identifier. + * @param conf_cb + * Callback function that fills in members of a struct rte_event_dma_adapter_conf struct passed + * into it. + * @param mode + * Flag to indicate the mode of the adapter. + * @see rte_event_dma_adapter_mode + * @param conf_arg + * Argument that is passed to the conf_cb function. + * + * @return + * - 0: Success + * - <0: Error code on failure + */ +__rte_experimental +int rte_event_dma_adapter_create_ext(uint8_t id, uint8_t evdev_id, + rte_event_dma_adapter_conf_cb conf_cb, + enum rte_event_dma_adapter_mode mode, void *conf_arg); + +/** + * Create a new event DMA adapter with the specified identifier. This function uses an internal + * configuration function that creates an event port. This default function reconfigures the event + * device with an additional event port and set up the event port using the port_config parameter + * passed into this function. In case the application needs more control in configuration of the + * service, it should use the rte_event_dma_adapter_create_ext() version. + * + * @param id + * Adapter identifier. + * @param evdev_id + * Event device identifier. + * @param port_config + * Argument of type *rte_event_port_conf* that is passed to the conf_cb function. + * @param mode + * Flag to indicate the mode of the adapter. + * @see rte_event_dma_adapter_mode + * + * @return + * - 0: Success + * - <0: Error code on failure + */ +__rte_experimental +int rte_event_dma_adapter_create(uint8_t id, uint8_t evdev_id, + struct rte_event_port_conf *port_config, + enum rte_event_dma_adapter_mode mode); + +/** + * Free an event DMA adapter + * + * @param id + * Adapter identifier. + * @return + * - 0: Success + * - <0: Error code on failure, If the adapter still has queue pairs added to it, the function + * returns -EBUSY. + */ +__rte_experimental +int rte_event_dma_adapter_free(uint8_t id); + +/** + * Retrieve the event port of an adapter. + * + * @param id + * Adapter identifier. + * + * @param [out] event_port_id + * Application links its event queue to this adapter port which is used in + * RTE_EVENT_DMA_ADAPTER_OP_FORWARD mode. + * + * @return + * - 0: Success + * - <0: Error code on failure. + */ +__rte_experimental +int rte_event_dma_adapter_event_port_get(uint8_t id, uint8_t *event_port_id); + +/** + * Add a vchan to an event DMA adapter. + * + * @param id + * Adapter identifier. + * @param dmadev_id + * dmadev identifier. + * @param vchan + * DMA device vchan identifier. If vchan is set -1, adapter adds all the + * preconfigured vchan to the instance. + * @param event + * If HW supports dmadev vchan to event queue binding, application is expected to fill in + * event information, else it will be NULL. + * @see RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_VCHAN_EV_BIND + * + * @return + * - 0: Success, vchan added correctly. + * - <0: Error code on failure. + */ +__rte_experimental +int rte_event_dma_adapter_vchan_add(uint8_t id, int16_t dmadev_id, uint16_t vchan, + const struct rte_event *event); + +/** + * Delete a vchan from an event DMA adapter. + * + * @param id + * Adapter identifier. + * @param dmadev_id + * DMA device identifier. + * @param vchan + * DMA device vchan identifier. + * + * @return + * - 0: Success, vchan deleted successfully. + * - <0: Error code on failure. + */ +__rte_experimental +int rte_event_dma_adapter_vchan_del(uint8_t id, int16_t dmadev_id, uint16_t vchan); + +/** + * Retrieve the service ID of an adapter. If the adapter doesn't use a rte_service function, this + * function returns -ESRCH. + * + * @param id + * Adapter identifier. + * @param [out] service_id + * A pointer to a uint32_t, to be filled in with the service id. + * + * @return + * - 0: Success + * - <0: Error code on failure, if the adapter doesn't use a rte_service function, this function + * returns -ESRCH. + */ +__rte_experimental +int rte_event_dma_adapter_service_id_get(uint8_t id, uint32_t *service_id); + +/** + * Start event DMA adapter + * + * @param id + * Adapter identifier. + * + * @return + * - 0: Success, adapter started successfully. + * - <0: Error code on failure. + * + * @note The eventdev and dmadev to which the event_dma_adapter is connected should be started + * before calling rte_event_dma_adapter_start(). + */ +__rte_experimental +int rte_event_dma_adapter_start(uint8_t id); + +/** + * Stop event DMA adapter + * + * @param id + * Adapter identifier. + * + * @return + * - 0: Success, adapter stopped successfully. + * - <0: Error code on failure. + */ +__rte_experimental +int rte_event_dma_adapter_stop(uint8_t id); + +/** + * Initialize the adapter runtime configuration parameters + * + * @param params + * A pointer to structure of type struct rte_event_dma_adapter_runtime_params + * + * @return + * - 0: Success + * - <0: Error code on failure + */ +__rte_experimental +int rte_event_dma_adapter_runtime_params_init(struct rte_event_dma_adapter_runtime_params *params); + +/** + * Set the adapter runtime configuration parameters + * + * @param id + * Adapter identifier + * + * @param params + * A pointer to structure of type struct rte_event_dma_adapter_runtime_params with configuration + * parameter values. The reserved fields of this structure must be initialized to zero and the valid + * fields need to be set appropriately. This struct can be initialized using + * rte_event_dma_adapter_runtime_params_init() API to default values or application may reset this + * struct and update required fields. + * + * @return + * - 0: Success + * - <0: Error code on failure + */ +__rte_experimental +int rte_event_dma_adapter_runtime_params_set(uint8_t id, + struct rte_event_dma_adapter_runtime_params *params); + +/** + * Get the adapter runtime configuration parameters + * + * @param id + * Adapter identifier + * + * @param[out] params + * A pointer to structure of type struct rte_event_dma_adapter_runtime_params containing valid + * adapter parameters when return value is 0. + * + * @return + * - 0: Success + * - <0: Error code on failure + */ +__rte_experimental +int rte_event_dma_adapter_runtime_params_get(uint8_t id, + struct rte_event_dma_adapter_runtime_params *params); + +/** + * Retrieve statistics for an adapter + * + * @param id + * Adapter identifier. + * @param [out] stats + * A pointer to structure used to retrieve statistics for an adapter. + * + * @return + * - 0: Success, retrieved successfully. + * - <0: Error code on failure. + */ +__rte_experimental +int rte_event_dma_adapter_stats_get(uint8_t id, struct rte_event_dma_adapter_stats *stats); + +/** + * Reset statistics for an adapter. + * + * @param id + * Adapter identifier. + * + * @return + * - 0: Success, statistics reset successfully. + * - <0: Error code on failure. + */ +__rte_experimental +int rte_event_dma_adapter_stats_reset(uint8_t id); + +/** + * Enqueue a burst of DMA operations as event objects supplied in *rte_event* structure on an event + * DMA adapter designated by its event *evdev_id* through the event port specified by *port_id*. + * This function is supported if the eventdev PMD has the + * #RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP_FWD capability flag set. + * + * The *nb_events* parameter is the number of event objects to enqueue that are supplied in the + * *ev* array of *rte_event* structure. + * + * The rte_event_dma_adapter_enqueue() function returns the number of event objects it actually + * enqueued. A return value equal to *nb_events* means that all event objects have been enqueued. + * + * @param evdev_id + * The identifier of the device. + * @param port_id + * The identifier of the event port. + * @param ev + * Points to an array of *nb_events* objects of type *rte_event* structure which contain the + * event object enqueue operations to be processed. + * @param nb_events + * The number of event objects to enqueue, typically number of + * rte_event_port_attr_get(...RTE_EVENT_PORT_ATTR_ENQ_DEPTH...) available for this port. + * + * @return + * The number of event objects actually enqueued on the event device. The return value can be + * less than the value of the *nb_events* parameter when the event devices queue is full or if + * invalid parameters are specified in a *rte_event*. If the return value is less than *nb_events*, + * the remaining events at the end of ev[] are not consumed and the caller has to take care of them, + * and rte_errno is set accordingly. Possible errno values include: + * - EINVAL: The port ID is invalid, device ID is invalid, an event's queue ID is invalid, or an + * event's sched type doesn't match the capabilities of the destination queue. + * - ENOSPC: The event port was backpressured and unable to enqueue one or more events. This + * error code is only applicable to closed systems. + */ +__rte_experimental +uint16_t rte_event_dma_adapter_enqueue(uint8_t evdev_id, uint8_t port_id, struct rte_event ev[], + uint16_t nb_events); + +#ifdef __cplusplus +} +#endif + +#endif /* RTE_EVENT_DMA_ADAPTER */ diff --git a/lib/eventdev/rte_eventdev.h b/lib/eventdev/rte_eventdev.h index 2ba8a7b090..41743f91b1 100644 --- a/lib/eventdev/rte_eventdev.h +++ b/lib/eventdev/rte_eventdev.h @@ -1197,6 +1197,8 @@ struct rte_event_vector { */ #define RTE_EVENT_TYPE_ETH_RX_ADAPTER 0x4 /**< The event generated from event eth Rx adapter */ +#define RTE_EVENT_TYPE_DMADEV 0x5 +/**< The event generated from dma subsystem */ #define RTE_EVENT_TYPE_VECTOR 0x8 /**< Indicates that event is a vector. * All vector event types should be a logical OR of EVENT_TYPE_VECTOR. @@ -1462,6 +1464,48 @@ int rte_event_crypto_adapter_caps_get(uint8_t dev_id, uint8_t cdev_id, uint32_t *caps); +/* DMA adapter capability bitmap flag */ +#define RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP_NEW 0x1 +/**< Flag indicates HW is capable of generating events in + * RTE_EVENT_OP_NEW enqueue operation. DMADEV will send + * packets to the event device as new events using an + * internal event port. + */ + +#define RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP_FWD 0x2 +/**< Flag indicates HW is capable of generating events in + * RTE_EVENT_OP_FORWARD enqueue operation. DMADEV will send + * packets to the event device as forwarded event using an + * internal event port. + */ + +#define RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_VCHAN_EV_BIND 0x4 +/**< Flag indicates HW is capable of mapping DMA vchan to event queue. */ + +/** + * Retrieve the event device's DMA adapter capabilities for the + * specified dmadev device + * + * @param dev_id + * The identifier of the device. + * + * @param dmadev_id + * The identifier of the dmadev device. + * + * @param[out] caps + * A pointer to memory filled with event adapter capabilities. + * It is expected to be pre-allocated & initialized by caller. + * + * @return + * - 0: Success, driver provides event adapter capabilities for the + * dmadev device. + * - <0: Error code returned by the driver function. + * + */ +__rte_experimental +int +rte_event_dma_adapter_caps_get(uint8_t dev_id, uint8_t dmadev_id, uint32_t *caps); + /* Ethdev Tx adapter capability bitmap flags */ #define RTE_EVENT_ETH_TX_ADAPTER_CAP_INTERNAL_PORT 0x1 /**< This flag is sent when the PMD supports a packet transmit callback diff --git a/lib/eventdev/rte_eventdev_core.h b/lib/eventdev/rte_eventdev_core.h index c27a52ccc0..83e8736c71 100644 --- a/lib/eventdev/rte_eventdev_core.h +++ b/lib/eventdev/rte_eventdev_core.h @@ -42,6 +42,10 @@ typedef uint16_t (*event_crypto_adapter_enqueue_t)(void *port, uint16_t nb_events); /**< @internal Enqueue burst of events on crypto adapter */ +typedef uint16_t (*event_dma_adapter_enqueue_t)(void *port, struct rte_event ev[], + uint16_t nb_events); +/**< @internal Enqueue burst of events on DMA adapter */ + struct rte_event_fp_ops { void **data; /**< points to array of internal port data pointers */ @@ -65,7 +69,9 @@ struct rte_event_fp_ops { /**< PMD Tx adapter enqueue same destination function. */ event_crypto_adapter_enqueue_t ca_enqueue; /**< PMD Crypto adapter enqueue function. */ - uintptr_t reserved[5]; + event_dma_adapter_enqueue_t dma_enqueue; + /**< PMD DMA adapter enqueue function. */ + uintptr_t reserved[4]; } __rte_cache_aligned; extern struct rte_event_fp_ops rte_event_fp_ops[RTE_EVENT_MAX_DEVS]; diff --git a/lib/eventdev/version.map b/lib/eventdev/version.map index 7ce09a87bb..b81eb2919c 100644 --- a/lib/eventdev/version.map +++ b/lib/eventdev/version.map @@ -133,6 +133,22 @@ EXPERIMENTAL { rte_event_timer_remaining_ticks_get; # added in 23.11 + rte_event_dma_adapter_caps_get; + rte_event_dma_adapter_create; + rte_event_dma_adapter_create_ext; + rte_event_dma_adapter_enqueue; + rte_event_dma_adapter_event_port_get; + rte_event_dma_adapter_free; + rte_event_dma_adapter_runtime_params_get; + rte_event_dma_adapter_runtime_params_init; + rte_event_dma_adapter_runtime_params_set; + rte_event_dma_adapter_service_id_get; + rte_event_dma_adapter_start; + rte_event_dma_adapter_stats_get; + rte_event_dma_adapter_stats_reset; + rte_event_dma_adapter_stop; + rte_event_dma_adapter_vchan_add; + rte_event_dma_adapter_vchan_del; rte_event_eth_rx_adapter_create_ext_with_params; }; diff --git a/lib/meson.build b/lib/meson.build index 53155be8e9..f3191f10b6 100644 --- a/lib/meson.build +++ b/lib/meson.build @@ -33,6 +33,7 @@ libraries = [ 'compressdev', 'cryptodev', 'distributor', + 'dmadev', 'efd', 'eventdev', 'gpudev', @@ -48,7 +49,6 @@ libraries = [ 'rawdev', 'regexdev', 'mldev', - 'dmadev', 'rib', 'reorder', 'sched', -- 2.25.1