From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 640E2425B7; Wed, 27 Sep 2023 20:13:11 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 48D90402E6; Wed, 27 Sep 2023 20:13:11 +0200 (CEST) Received: from mail-vk1-f173.google.com (mail-vk1-f173.google.com [209.85.221.173]) by mails.dpdk.org (Postfix) with ESMTP id 4CCA540271 for ; Wed, 27 Sep 2023 20:13:09 +0200 (CEST) Received: by mail-vk1-f173.google.com with SMTP id 71dfb90a1353d-49352207f33so4474866e0c.2 for ; Wed, 27 Sep 2023 11:13:09 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1695838388; x=1696443188; darn=dpdk.org; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=GUcZYhBXR6WIeGpMyc9t04n1cfZ44/Ui6ShTOG27AjQ=; b=MDl/NeJ19Rujun0qw/9Ml77EZ0kLKvXtzTd3kOhU2HPrmZG7s+Rj6rPvycTVayXFpu iryitevhG+P3Lbj+g084EoJ99/aXxm8xyN+iYbrRs4PRwR3ODBkU/HfgzJnPLERJNlKZ 3VJ1LILG7elhJJm2bJQunVkQtsLtU5HzRESwEmZaVYDu8CoIXNV+xLRgoww/whS8WD4G DCxeLnKj6cWL617VO3HVN/DQvx02iT+56u/q6CQoKgTNbr5CPXE4hHvVBvIXU4Wl/0ld 0e85QdJPhwatvVC3LVR9I/znxob1wmqr8VArxK5WmQ2ZWUbTZw2mc8dUxj7tf5vDE7AT Qa1g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1695838388; x=1696443188; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=GUcZYhBXR6WIeGpMyc9t04n1cfZ44/Ui6ShTOG27AjQ=; b=CpUDfakn5Ke4G5xWW0+1xCmKEWNmaZhlxM7uaXWm6xRHCvH696Jz4VWcaYUYIXcl2f EmuVjiZsBCxOZt7v2Gy+2zcexG5oK8g+XPbdOwMpZehFx1Pr/BgHhi/Ssd7zo2C3+Phi HMJMeLWVrSvlfnKz3lx8PgpNQjyEYDg7HAymRO/2kyeSMIw9abpNuQe2IUrRRg2btOzS 4Tw//b0z32IJ0kyXTUFpF+9VB4yYXbsq6wApFex9CnnRTbhhVJKy4uJedWpnMKGX84e1 igi05cUYnvMwdOEh/nZOc6S3atASdDeAZhxm37MS5m5jovIQKt+RnoCdIxcnkIpaUIpI STTQ== X-Gm-Message-State: AOJu0YwEFpD7hZJcZahALO1vMKCaR78vY2BHSWZLhvKG9XTjvC8WgAeQ fdZndWdRl84UAgGs/+yCM933Nj/zUVYyeYS18og= X-Google-Smtp-Source: AGHT+IF/AsCk/kEDc0NdKy8IwMftjc0yZT3z9UEKCPsT+r0JKI+fTRi9YAo+qWb6yS0lx6G8TDxVU5/7jMZhP/jy9hk= X-Received: by 2002:a1f:4e87:0:b0:49a:36e4:5565 with SMTP id c129-20020a1f4e87000000b0049a36e45565mr2359523vkb.16.1695838388248; Wed, 27 Sep 2023 11:13:08 -0700 (PDT) MIME-Version: 1.0 References: <20230923133449.3780841-1-amitprakashs@marvell.com> <20230926103233.3956431-1-amitprakashs@marvell.com> <20230926103233.3956431-2-amitprakashs@marvell.com> In-Reply-To: <20230926103233.3956431-2-amitprakashs@marvell.com> From: Jerin Jacob Date: Wed, 27 Sep 2023 23:42:42 +0530 Message-ID: Subject: Re: [PATCH v4 01/12] eventdev: introduce event DMA adapter library To: Amit Prakash Shukla Cc: Thomas Monjalon , Jerin Jacob , dev@dpdk.org, fengchengwen@huawei.com, kevin.laatz@intel.com, bruce.richardson@intel.com, conor.walsh@intel.com, vattunuru@marvell.com, g.singh@nxp.com, sachin.saxena@oss.nxp.com, hemant.agrawal@nxp.com, cheng1.jiang@intel.com, ndabilpuram@marvell.com, anoobj@marvell.com, mb@smartsharesystems.com Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org On Tue, Sep 26, 2023 at 4:03=E2=80=AFPM Amit Prakash Shukla wrote: > > Introduce event DMA adapter APIs. The change provides information > on adapter modes and usage. Application can use this event adapter > interface to transfer packets between DMA device and event device. With below changes, Acked-by: Jerin Jacob Use ethdev/dma for all patches like other adapters. Rewrite suggestion: eventdev/dma: introduce event dma adapter Introduce event dma adapter interface to transfer packets between dma device and event device. > > Signed-off-by: Amit Prakash Shukla > --- > > diff --git a/MAINTAINERS b/MAINTAINERS > index a926155f26..a6b8fc88d0 100644 > --- a/MAINTAINERS > +++ b/MAINTAINERS > @@ -548,6 +548,11 @@ F: drivers/raw/skeleton/ > F: app/test/test_rawdev.c > F: doc/guides/prog_guide/rawdev.rst > > +Eventdev DMA Adapter API > +M: Amit Prakash Shukla > +T: git://dpdk.org/next/dpdk-next-eventdev > +F: lib/eventdev/*dma_adapter* > +F: doc/guides/prog_guide/event_dma_adapter.rst > Move after "Eventdev Crypto Adapter API" to group all adapter frameworks. > diff --git a/doc/guides/prog_guide/event_dma_adapter.rst b/doc/guides/pro= g_guide/event_dma_adapter.rst > new file mode 100644 > index 0000000000..eeb9ce6dfd > --- /dev/null > +++ b/doc/guides/prog_guide/event_dma_adapter.rst > @@ -0,0 +1,264 @@ > +.. SPDX-License-Identifier: BSD-3-Clause > + Copyright (c) 2023 Marvell. > + > +Event DMA Adapter Library > +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D > + > +DPDK :doc:`Eventdev library ` provides event driven programmin= g model with features > +to schedule events. :doc:`DMA Device library ` provides an inter= face to DMA poll mode > +drivers that support DMA operations. Event DMA Adapter is intended to br= idge between the event > +device and the DMA device. > + > +Packet flow from DMA device to the event device can be accomplished usin= g software and hardware > +based transfer mechanisms. The adapter queries an eventdev PMD to determ= ine which mechanism to > +be used. The adapter uses an EAL service core function for software base= d packet transfer and > +uses the eventdev PMD functions to configure hardware based packet trans= fer between DMA device > +and the event device. DMA adapter uses a new event type called ``RTE_EVE= NT_TYPE_DMADEV`` to > +indicate the source of event. > + > +Application can choose to submit an DMA operation directly to an DMA dev= ice or send it to an DMA > +adapter via eventdev based on RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP= _FWD capability. The Change as ``RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP_FWD`` > +first mode is known as the event new (RTE_EVENT_DMA_ADAPTER_OP_NEW) mode= and the second as the Change as ``RTE_EVENT_DMA_ADAPTER_OP_NEW`` > +event forward (RTE_EVENT_DMA_ADAPTER_OP_FORWARD) mode. Choice of mode ca= n be specified while Change as ``RTE_EVENT_DMA_ADAPTER_OP_FORWARD`` > +creating the adapter. In the former mode, it is the application's respon= sibility to enable > +ingress packet ordering. In the latter mode, it is the adapter's respons= ibility to enable > +ingress packet ordering. > + > + > +Adapter Modes > +------------- > + > +RTE_EVENT_DMA_ADAPTER_OP_NEW mode > +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ > + > +In the RTE_EVENT_DMA_ADAPTER_OP_NEW mode, application submits DMA operat= ions directly to an DMA Change as ``RTE_EVENT_DMA_ADAPTER_OP_NEW`` > +device. The adapter then dequeues DMA completions from the DMA device an= d enqueues them as events > +to the event device. This mode does not ensure ingress ordering as the a= pplication directly > +enqueues to the dmadev without going through DMA/atomic stage. In this m= ode, events dequeued > +from the adapter are treated as new events. The application has to speci= fy event information > +(response information) which is needed to enqueue an event after the DMA= operation is completed. > + > +.. _figure_event_dma_adapter_op_new: > + > +.. figure:: img/event_dma_adapter_op_new.* > + > + Working model of ``RTE_EVENT_DMA_ADAPTER_OP_NEW`` mode > + > + > +RTE_EVENT_DMA_ADAPTER_OP_FORWARD mode > +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ > + > +In the ``RTE_EVENT_DMA_ADAPTER_OP_FORWARD`` mode, if the event PMD and D= MA PMD supports internal > +event port (``RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP_FWD``), the app= lication should use > +``rte_event_dma_adapter_enqueue()`` API to enqueue DMA operations as eve= nts to DMA adapter. If > +not, application retrieves DMA adapter's event port using ``rte_event_dm= a_adapter_event_port_get()`` > +API, links its event queue to this port and starts enqueuing DMA operati= ons as events to eventdev > +using ``rte_event_enqueue_burst()``. The adapter then dequeues the event= s and submits the DMA > +operations to the dmadev. After the DMA operation is complete, the adapt= er enqueues events to the > +event device. > + > +Applications can use this mode when ingress packet ordering is needed. I= n this mode, events > +dequeued from the adapter will be treated as forwarded events. Applicati= on has to specify event > +information (response information) needed to enqueue the event after the= DMA operation has > +completed. > + > +.. _figure_event_dma_adapter_op_forward: > + > +.. figure:: img/event_dma_adapter_op_forward.* > + > + Working model of ``RTE_EVENT_DMA_ADAPTER_OP_FORWARD`` mode > + > + > +API Overview > +------------ > + > +This section has a brief introduction to the event DMA adapter APIs. The= application is expected > +to create an adapter which is associated with a single eventdev, then ad= d dmadev and vchan to the > +adapter instance. > + > + > +Create an adapter instance > +~~~~~~~~~~~~~~~~~~~~~~~~~~ > + > +An adapter instance is created using ``rte_event_dma_adapter_create()``.= This function is called > +with event device to be associated with the adapter and port configurati= on for the adapter to > +setup an event port (if the adapter needs to use a service function). > + > +Adapter can be started in ``RTE_EVENT_DMA_ADAPTER_OP_NEW`` or ``RTE_EVEN= T_DMA_ADAPTER_OP_FORWARD`` > +mode. > + > +.. code-block:: c > + > + enum rte_event_dma_adapter_mode mode; > + struct rte_event_dev_info dev_info; > + struct rte_event_port_conf conf; > + uint8_t evdev_id; > + uint8_t dma_id; > + int ret; > + > + ret =3D rte_event_dev_info_get(dma_id, &dev_info); > + > + conf.new_event_threshold =3D dev_info.max_num_events; > + conf.dequeue_depth =3D dev_info.max_event_port_dequeue_depth; > + conf.enqueue_depth =3D dev_info.max_event_port_enqueue_depth; > + mode =3D RTE_EVENT_DMA_ADAPTER_OP_FORWARD; > + ret =3D rte_event_dma_adapter_create(dma_id, evdev_id, &conf, mo= de); > + > + > +``rte_event_dma_adapter_create_ext()`` function can be used by the appli= cation to have a finer > +control on eventdev port allocation and setup. The ``rte_event_dma_adapt= er_create_ext()`` > +function is passed a callback function. The callback function is invoked= if the adapter creates > +a service function and uses an event port for it. The callback is expect= ed to fill the > +``struct rte_event_dma_adapter_conf`` structure passed to it. > + > +In the ``RTE_EVENT_DMA_ADAPTER_OP_FORWARD`` mode, if the event PMD and D= MA PMD supports internal > +event port (``RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP_FWD``), events = with DMA operations should > +be enqueued to the DMA adapter using ``rte_event_dma_adapter_enqueue()``= API. If not, the event port > +created by the adapter can be retrieved using ``rte_event_dma_adapter_ev= ent_port_get()`` API. An > +application can use this event port to link with an event queue, on whic= h it enqueues events > +towards the DMA adapter using ``rte_event_enqueue_burst()``. > + > +.. code-block:: c > + > + uint8_t dma_adpt_id, evdev_id, dma_dev_id, dma_ev_port_id, app_q= id; > + struct rte_event ev; > + uint32_t cap; > + int ret; > + > + // Fill in event info and update event_ptr with rte_dma_op > + memset(&ev, 0, sizeof(ev)); > + . > + . > + ev.event_ptr =3D op; > + > + ret =3D rte_event_dma_adapter_caps_get(evdev_id, dma_dev_id, &ca= p); > + if (cap & RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP_FWD) { > + ret =3D rte_event_dma_adapter_enqueue(evdev_id, app_ev_p= ort_id, ev, nb_events); > + } else { > + ret =3D rte_event_dma_adapter_event_port_get(dma_adpt_id= , &dma_ev_port_id); > + ret =3D rte_event_queue_setup(evdev_id, app_qid, NULL); > + ret =3D rte_event_port_link(evdev_id, dma_ev_port_id, &a= pp_qid, NULL, 1); > + ev.queue_id =3D app_qid; > + ret =3D rte_event_enqueue_burst(evdev_id, app_ev_port_id= , ev, nb_events); > + } > + > + > +Event device configuration for service based adapter > +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ > + > +When ``rte_event_dma_adapter_create()`` is used for creating adapter ins= tance, > +``rte_event_dev_config::nb_event_ports`` is automatically incremented, a= nd event device is > +reconfigured with additional event port during service initialization. T= his event device > +reconfigure logic also increments the ``rte_event_dev_config::nb_single_= link_event_port_queues`` > +parameter if the adapter event port config is of type ``RTE_EVENT_PORT_C= FG_SINGLE_LINK``. > + > +Applications using this mode of adapter creation need not configure the = event device with > +``rte_event_dev_config::nb_event_ports`` and > +``rte_event_dev_config::nb_single_link_event_port_queues`` parameters re= quired for DMA adapter when > +the adapter is created using the above-mentioned API. > + > + > +Querying adapter capabilities > +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ > + > +The ``rte_event_dma_adapter_caps_get()`` function allows the application= to query the adapter > +capabilities for an eventdev and dmadev combination. This API provides w= hether dmadev and eventdev > +are connected using internal HW port or not. > + > +.. code-block:: c > + > + rte_event_dma_adapter_caps_get(dev_id, dma_dev_id, &cap); > + > + > +Adding vchan to the adapter instance > +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ > + > +dmadev device id and vchan are configured using dmadev APIs. For more in= formation > +see :doc:`here `. > + > +.. code-block:: c > + > + struct rte_dma_vchan_conf vchan_conf; > + struct rte_dma_conf dev_conf; > + uint8_t dev_id =3D 0; > + uint16_t vchan =3D 0; > + > + rte_dma_configure(dev_id, &dev_conf); > + rte_dma_vchan_setup(dev_id, vhcan, &vchan_conf); vhcan -> vchan > + > +These dmadev id and vchan are added to the instance using the > +``rte_event_dma_adapter_vchan_add()`` API. The same is removed using > +``rte_event_dma_adapter_vchan_del()`` API. If hardware supports > +``RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_VCHAN_EV_BIND`` capability, ev= ent information must be passed to the add API. > + > +.. code-block:: c > + > + uint32_t cap; > + int ret; > + > + ret =3D rte_event_dma_adapter_caps_get(evdev_id, dma_dev_id, &ca= p); > + if (cap & RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_VCHAN_EV_BIND)= { > + struct rte_event event; > + > + rte_event_dma_adapter_vchan_add(id, dma_dev_id, vchan, &= conf); > + } else > + rte_event_dma_adapter_vchan_add(id, dma_dev_id, vchan, N= ULL); > + > + > +Configuring service function > +~~~~~~~~~~~~~~~~~~~~~~~~~~~~ > + > +If the adapter uses a service function, the application is required to a= ssign a service core to > +the service function as show below. > + > +.. code-block:: c > + > + uint32_t service_id; > + > + if (rte_event_dma_adapter_service_id_get(dma_id, &service_id) = =3D=3D 0) > + rte_service_map_lcore_set(service_id, CORE_ID); > + > + > +Set event response information > +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ > + > +In the RTE_EVENT_DMA_ADAPTER_OP_FORWARD / RTE_EVENT_DMA_ADAPTER_OP_NEW m= ode, the application > +specifies the dmadev ID and vchan ID in ``struct rte_event_dma_adapter_o= p`` and the event > +information (response information) needed to enqueue an event after the = DMA operation has > +completed. The response information is specified in ``struct rte_event``= and appended to the > +``struct rte_event_dma_adapter_op``. > + > + > +Start the adapter instance > +~~~~~~~~~~~~~~~~~~~~~~~~~~ > + > +The application calls ``rte_event_dma_adapter_start()`` to start the ada= pter. This function calls > +the start callbacks of the eventdev PMDs for hardware based eventdev-dma= dev connections and > +``rte_service_run_state_set()`` to enable the service function if one ex= ists. > + > +.. code-block:: c > + > + rte_event_dma_adapter_start(id); > + > +.. Note:: > + > + The eventdev to which the event_dma_adapter is connected should= be started before calling > + rte_event_dma_adapter_start(). > + > + > +Get adapter statistics > +~~~~~~~~~~~~~~~~~~~~~~ > + > +The ``rte_event_dma_adapter_stats_get()`` function reports counters def= ined in struct > +``rte_event_dma_adapter_stats``. The received packet and enqueued event = counts are a sum of the > +counts from the eventdev PMD callbacks if the callback is supported, and= the counts maintained by > +the service function, if one exists. > + > +Set/Get adapter runtime configuration parameters > +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ > + > +The runtime configuration parameters of adapter can be set/get using > +``rte_event_dma_adapter_runtime_params_set()`` and > +``rte_event_dma_adapter_runtime_params_get()`` respectively. > +The parameters that can be set/get are defined in > +``struct rte_event_dma_adapter_runtime_params``. > diff --git a/doc/guides/prog_guide/eventdev.rst b/doc/guides/prog_guide/e= ventdev.rst > index 2c83176846..ff55115d0d 100644 > --- a/doc/guides/prog_guide/eventdev.rst > +++ b/doc/guides/prog_guide/eventdev.rst > @@ -333,7 +333,8 @@ eventdev. > .. Note:: > > EventDev needs to be started before starting the event producer= s such > - as event_eth_rx_adapter, event_timer_adapter and event_crypto_a= dapter. > + as event_eth_rx_adapter, event_timer_adapter, event_crypto_adap= ter and > + event_dma_adapter. > > Ingress of New Events > ~~~~~~~~~~~~~~~~~~~~~ > @@ -445,8 +446,9 @@ using ``rte_event_dev_stop_flush_callback_register()`= ` function. > .. Note:: > > The event producers such as ``event_eth_rx_adapter``, > - ``event_timer_adapter`` and ``event_crypto_adapter`` > - need to be stopped before stopping the event device. > + ``event_timer_adapter``, ``event_crypto_adapter`` and > + ``event_dma_adapter`` need to be stopped before stopping > + the event device. > > diff --git a/doc/guides/prog_guide/index.rst b/doc/guides/prog_guide/inde= x.rst > index 52a6d9e7aa..beaa4b8869 100644 > --- a/doc/guides/prog_guide/index.rst > +++ b/doc/guides/prog_guide/index.rst > @@ -60,6 +60,7 @@ Programmer's Guide > event_ethernet_tx_adapter > event_timer_adapter > event_crypto_adapter > + event_dma_adapter > qos_framework > power_man > packet_classif_access_ctrl > diff --git a/doc/guides/rel_notes/release_23_11.rst b/doc/guides/rel_note= s/release_23_11.rst > index b34ddc0860..70f2e7e03c 100644 > --- a/doc/guides/rel_notes/release_23_11.rst > +++ b/doc/guides/rel_notes/release_23_11.rst > @@ -44,7 +44,9 @@ New Features > .. This section should contain new features added in this release. > Sample format: > > - * **Add a title in the past tense with a full stop.** Dont delete this. See other commits which update the release notes > + * **Added eventdev DMA adapter library.** > + > + Added API to DMA transfer data using event mechanism. Pull the text which is add for Crypto adapter when it was addded. > > Add a short 1-2 sentence description in the past tense. > The description should be enough to allow someone scanning > diff --git a/lib/eventdev/eventdev_pmd.h b/lib/eventdev/eventdev_pmd.h > index f62f42e140..8e5fd420f6 100644 > --- a/lib/eventdev/eventdev_pmd.h > +++ b/lib/eventdev/eventdev_pmd.h > @@ -178,8 +178,12 @@ struct rte_eventdev { > event_tx_adapter_enqueue_t txa_enqueue; > /**< Pointer to PMD eth Tx adapter enqueue function. */ > event_crypto_adapter_enqueue_t ca_enqueue; > + /**< Pointer to PMD crypto adapter enqueue function. */ > > - uint64_t reserved_64s[4]; /**< Reserved for future fields */ > + event_dma_adapter_enqueue_t dma_enqueue; > + /**< Pointer to PMD DMA adapter enqueue function. */ > + > + uint64_t reserved_64s[3]; /**< Reserved for future fields */ > void *reserved_ptrs[3]; /**< Reserved for future fields */ > } __rte_cache_aligned; > > @@ -1320,6 +1324,160 @@ typedef int (*eventdev_eth_tx_adapter_queue_stop) > > #define eventdev_stop_flush_t rte_eventdev_stop_flush_t > > +/** > + * Retrieve the event device's DMA adapter capabilities for the > + * specified DMA device > + * > + * @param dev > + * Event device pointer > + * > + * @param dma_dev_id > + * DMA device identification > + * > + * @param[out] caps > + * A pointer to memory filled with event adapter capabilities. > + * It is expected to be pre-allocated & initialized by caller. > + * > + * @return > + * - 0: Success, driver provides event adapter capabilities for the > + * DMADEV. > + * - <0: Error code returned by the driver function. > + * > + */ > +typedef int (*eventdev_dma_adapter_caps_get_t)(const struct rte_eventdev= *dev, > + const int16_t dma_dev_id, = uint32_t *caps); > + > +/** > + * This API may change without prior notice Remove this. > + * > + * Add DMA vchan queue to event device. This callback is invoked if > + * the caps returned from rte_event_dma_adapter_caps_get(, dmadev_id) > + * has RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_* set. > + * > + * @param dev > + * Event device pointer > + * > + * @param dma_dev_id > + * DMADEV identification > + * > + * @param vchan_id > + * DMADEV vchan queue identifier. > + * > + * @param event > + * Event information required for binding dmadev queue pair to event qu= eue. > + * This structure will have a valid value for only those HW PMDs support= ing > + * @see RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_VCHAN_EV_BIND capability= . > + * > + * @return > + * - 0: Success, dmadev vchan added successfully. > + * - <0: Error code returned by the driver function. > + * > + */ > +typedef int (*eventdev_dma_adapter_vchan_add_t)(const struct rte_eventde= v *dev, > + const int16_t dma_dev_id, > + uint16_t vchan_id, > + const struct rte_event *e= vent); > + > +/** > + * This API may change without prior notice Remove this. > + * > + * Delete DMA vhcan to event device. This callback is invoked if > + * the caps returned from rte_event_dma_adapter_caps_get(, dmadev_id) > + * has RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_* set. > + * > + * @param dev > + * Event device pointer > + * > + * @param dma_dev_id > + * DMADEV identification identification -> identifier. Change DMADEV as dmadev across this patch for documentation. > + * > + * @param vchan_id > + * dmadev vchan identifier. > + * > + * @return > + * - 0: Success, dmadev vchan deleted successfully. > + * - <0: Error code returned by the driver function. > + * > + */ > +typedef int (*eventdev_dma_adapter_vchan_del_t)(const struct rte_eventde= v *dev, > + const int16_t dma_dev_id, > + uint16_t vchan_id); > + > +/** > + * Start DMA adapter. This callback is invoked if > + * the caps returned from rte_event_dma_adapter_caps_get(.., dmadev_id) > + * has RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_* set and vchan for dmade= v_id > + * have been added to the event device. > + * > + * @param dev > + * Event device pointer > + * > + * @param dma_dev_id > + * DMA device identification > + * > + * @return > + * - 0: Success, DMA adapter started successfully. > + * - <0: Error code returned by the driver function. > + */ > +typedef int (*eventdev_dma_adapter_start_t)(const struct rte_eventdev *d= ev, > + const int16_t dma_dev_id); > + > +/** > + * Stop DMA adapter. This callback is invoked if > + * the caps returned from rte_event_dma_adapter_caps_get(.., dmadev_id) > + * has RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_* set and vchan for dmade= v_id > + * have been added to the event device. > + * > + * @param dev > + * Event device pointer > + * > + * @param dma_dev_id > + * DMA device identification > + * > + * @return > + * - 0: Success, DMA adapter stopped successfully. > + * - <0: Error code returned by the driver function. > + */ > +typedef int (*eventdev_dma_adapter_stop_t)(const struct rte_eventdev *de= v, > + const int16_t dma_dev_id); > + > +struct rte_event_dma_adapter_stats; > + > +/** > + * Retrieve DMA adapter statistics. > + * > + * @param dev > + * Event device pointer > + * > + * @param dma_dev_id > + * DMA device identification > + * > + * @param[out] stats > + * Pointer to stats structure > + * > + * @return > + * Return 0 on success. > + */ > +typedef int (*eventdev_dma_adapter_stats_get)(const struct rte_eventdev = *dev, > + const int16_t dma_dev_id, > + struct rte_event_dma_adapte= r_stats *stats); > + > +/** > + * Reset DMA adapter statistics. > + * > + * @param dev > + * Event device pointer > + * > + * @param dma_dev_id > + * DMA device identification > + * > + * @return > + * Return 0 on success. > + */ > +typedef int (*eventdev_dma_adapter_stats_reset)(const struct rte_eventde= v *dev, > + const int16_t dma_dev_id)= ; > + > + > /** Event device operations function pointer table */ > struct eventdev_ops { > eventdev_info_get_t dev_infos_get; /**< Get device info. */ > @@ -1440,6 +1598,21 @@ struct eventdev_ops { > eventdev_eth_tx_adapter_queue_stop eth_tx_adapter_queue_stop; > /**< Stop Tx queue assigned to Tx adapter instance */ > > + eventdev_dma_adapter_caps_get_t dma_adapter_caps_get; > + /**< Get DMA adapter capabilities */ > + eventdev_dma_adapter_vchan_add_t dma_adapter_vchan_add; > + /**< Add vchan queue to DMA adapter */ > + eventdev_dma_adapter_vchan_del_t dma_adapter_vchan_del; > + /**< Delete vchan queue from DMA adapter */ > + eventdev_dma_adapter_start_t dma_adapter_start; > + /**< Start DMA adapter */ > + eventdev_dma_adapter_stop_t dma_adapter_stop; > + /**< Stop DMA adapter */ > + eventdev_dma_adapter_stats_get dma_adapter_stats_get; > + /**< Get DMA stats */ > + eventdev_dma_adapter_stats_reset dma_adapter_stats_reset; > + /**< Reset DMA stats */ > + > eventdev_selftest dev_selftest; > /**< Start eventdev Selftest */ > > diff --git a/lib/eventdev/eventdev_private.c b/lib/eventdev/eventdev_priv= ate.c > index 1d3d9d357e..18ed8bf3c8 100644 > --- a/lib/eventdev/eventdev_private.c > +++ b/lib/eventdev/eventdev_private.c > @@ -81,6 +81,14 @@ dummy_event_crypto_adapter_enqueue(__rte_unused void *= port, > return 0; > } > > +static uint16_t > +dummy_event_dma_adapter_enqueue(__rte_unused void *port, __rte_unused st= ruct rte_event ev[], > + __rte_unused uint16_t nb_events) > +{ > + RTE_EDEV_LOG_ERR("event DMA adapter enqueue requested for unconfi= gured event device"); > + return 0; > +} > + > void > event_dev_fp_ops_reset(struct rte_event_fp_ops *fp_op) > { > @@ -97,6 +105,7 @@ event_dev_fp_ops_reset(struct rte_event_fp_ops *fp_op) > .txa_enqueue_same_dest =3D > dummy_event_tx_adapter_enqueue_same_dest, > .ca_enqueue =3D dummy_event_crypto_adapter_enqueue, > + .dma_enqueue =3D dummy_event_dma_adapter_enqueue, > .data =3D dummy_data, > }; > > @@ -117,5 +126,6 @@ event_dev_fp_ops_set(struct rte_event_fp_ops *fp_op, > fp_op->txa_enqueue =3D dev->txa_enqueue; > fp_op->txa_enqueue_same_dest =3D dev->txa_enqueue_same_dest; > fp_op->ca_enqueue =3D dev->ca_enqueue; > + fp_op->dma_enqueue =3D dev->dma_enqueue; > fp_op->data =3D dev->data->ports; > } > diff --git a/lib/eventdev/meson.build b/lib/eventdev/meson.build > index 6edf98dfa5..21347f7c4c 100644 > --- a/lib/eventdev/meson.build > +++ b/lib/eventdev/meson.build > @@ -25,6 +25,7 @@ sources =3D files( > ) > headers =3D files( > 'rte_event_crypto_adapter.h', > + 'rte_event_dma_adapter.h', > 'rte_event_eth_rx_adapter.h', > 'rte_event_eth_tx_adapter.h', > 'rte_event_ring.h', > diff --git a/lib/eventdev/rte_event_dma_adapter.h b/lib/eventdev/rte_even= t_dma_adapter.h > new file mode 100644 > index 0000000000..d413d31a61 > --- /dev/null > +++ b/lib/eventdev/rte_event_dma_adapter.h > @@ -0,0 +1,582 @@ > +/* SPDX-License-Identifier: BSD-3-Clause > + * Copyright (c) 2023 Marvell. > + */ > + > +#ifndef RTE_EVENT_DMA_ADAPTER > +#define RTE_EVENT_DMA_ADAPTER > + > +/** > + * @file rte_event_dma_adapter.h > + * > + * @warning > + * @b EXPERIMENTAL: > + * All functions in this file may be changed or removed without prior no= tice. > + * > + * DMA Event Adapter API. > + * > + * Eventdev library provides adapters to bridge between various componen= ts for providing new > + * event source. The event DMA adapter is one of those adapters which is= intended to bridge > + * between event devices and DMA devices. > + * > + * The DMA adapter adds support to enqueue / dequeue DMA operations to /= from event device. The > + * packet flow between DMA device and the event device can be accomplish= ed using both SW and HW > + * based transfer mechanisms. The adapter uses an EAL service core funct= ion for SW based packet > + * transfer and uses the eventdev PMD functions to configure HW based pa= cket transfer between the > + * DMA device and the event device. > + * > + * The application can choose to submit a DMA operation directly to an D= MA device or send it to the > + * DMA adapter via eventdev based on RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_= PORT_OP_FWD capability. The > + * first mode is known as the event new (RTE_EVENT_DMA_ADAPTER_OP_NEW) m= ode and the second as the > + * event forward (RTE_EVENT_DMA_ADAPTER_OP_FORWARD) mode. The choice of = mode can be specified while > + * creating the adapter. In the former mode, it is an application respon= sibility to enable ingress > + * packet ordering. In the latter mode, it is the adapter responsibility= to enable the ingress > + * packet ordering. > + * > + * > + * Working model of RTE_EVENT_DMA_ADAPTER_OP_NEW mode: > + * > + * +--------------+ +--------------+ > + * | | | DMA stage | > + * | Application |---[2]-->| + enqueue to | > + * | | | dmadev | > + * +--------------+ +--------------+ > + * ^ ^ | > + * | | [3] > + * [6] [1] | > + * | | | > + * +--------------+ | > + * | | | > + * | Event device | | > + * | | | > + * +--------------+ | > + * ^ | > + * | | > + * [5] | > + * | v > + * +--------------+ +--------------+ > + * | | | | > + * | DMA adapter |<--[4]---| dmadev | > + * | | | | > + * +--------------+ +--------------+ > + * > + * > + * [1] Application dequeues events from the previous stage. > + * [2] Application prepares the DMA operations. > + * [3] DMA operations are submitted to dmadev by application. > + * [4] DMA adapter dequeues DMA completions from dmadev. > + * [5] DMA adapter enqueues events to the eventdev. > + * [6] Application dequeues from eventdev for further processing= . > + * > + * In the RTE_EVENT_DMA_ADAPTER_OP_NEW mode, application submits DMA ope= rations directly to DMA > + * device. The DMA adapter then dequeues DMA completions from DMA device= and enqueue events to the > + * event device. This mode does not ensure ingress ordering, if the appl= ication directly enqueues > + * to dmadev without going through DMA / atomic stage i.e. removing item= [1] and [2]. > + * > + * Events dequeued from the adapter will be treated as new events. In th= is mode, application needs > + * to specify event information (response information) which is needed t= o enqueue an event after the > + * DMA operation is completed. > + * > + * > + * Working model of RTE_EVENT_DMA_ADAPTER_OP_FORWARD mode: > + * > + * +--------------+ +--------------+ > + * --[1]-->| |---[2]-->| Application | > + * | Event device | | in | > + * <--[8]--| |<--[3]---| Ordered stage| > + * +--------------+ +--------------+ > + * ^ | > + * | [4] > + * [7] | > + * | v > + * +----------------+ +--------------+ > + * | |--[5]->| | > + * | DMA adapter | | dmadev | > + * | |<-[6]--| | > + * +----------------+ +--------------+ > + * > + * > + * [1] Events from the previous stage. > + * [2] Application in ordered stage dequeues events from eventde= v. > + * [3] Application enqueues DMA operations as events to eventdev= . > + * [4] DMA adapter dequeues event from eventdev. > + * [5] DMA adapter submits DMA operations to dmadev (Atomic stag= e). > + * [6] DMA adapter dequeues DMA completions from dmadev > + * [7] DMA adapter enqueues events to the eventdev > + * [8] Events to the next stage > + * > + * In the event forward (RTE_EVENT_DMA_ADAPTER_OP_FORWARD) mode, if the = HW supports the capability > + * RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP_FWD, application can direc= tly submit the DMA > + * operations to the dmadev. If not, application retrieves the event por= t of the DMA adapter > + * through the API, rte_event_DMA_adapter_event_port_get(). Then, links = its event queue to this > + * port and starts enqueuing DMA operations as events to the eventdev. T= he adapter then dequeues > + * the events and submits the DMA operations to the dmadev. After the DM= A completions, the adapter > + * enqueues events to the event device. > + * > + * Application can use this mode, when ingress packet ordering is needed= . Events dequeued from the > + * adapter will be treated as forwarded events. In this mode, the applic= ation needs to specify the > + * dmadev ID and queue pair ID (request information) needed to enqueue a= n DMA operation in addition > + * to the event information (response information) needed to enqueue an = event after the DMA > + * operation has completed. > + * > + * The event DMA adapter provides common APIs to configure the packet fl= ow from the DMA device to > + * event devices for both SW and HW based transfers. The DMA event adapt= er's functions are: > + * > + * - rte_event_dma_adapter_create_ext() > + * - rte_event_dma_adapter_create() > + * - rte_event_dma_adapter_free() > + * - rte_event_dma_adapter_vchan_add() > + * - rte_event_dma_adapter_vchan_del() > + * - rte_event_dma_adapter_start() > + * - rte_event_dma_adapter_stop() > + * - rte_event_dma_adapter_stats_get() > + * - rte_event_dma_adapter_stats_reset() > + * > + * The application creates an instance using rte_event_dma_adapter_creat= e() or > + * rte_event_dma_adapter_create_ext(). > + * > + * dmadev queue pair addition / deletion is done using the rte_event_dma= _adapter_vchan_add() / > + * rte_event_dma_adapter_vchan_del() APIs. If HW supports the capability > + * RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_VCHAN_EV_BIND, event informat= ion must be passed to the > + * add API. > + * > + */ > + > +#include > + > +#include > +#include > +#include Alphabetical order. > + > +#ifdef __cplusplus > +extern "C" { > +#endif > + > + > +/** > + * Enqueue a burst of DMA operations as event objects supplied in *rte_e= vent* structure on an event > + * DMA adapter designated by its event *evdev_id* through the event port= specified by *port_id*. > + * This function is supported if the eventdev PMD has the > + * #RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP_FWD capability flag set. > + * > + * The *nb_events* parameter is the number of event objects to enqueue t= hat are supplied in the > + * *ev* array of *rte_event* structure. > + * > + * The rte_event_dma_adapter_enqueue() function returns the number of ev= ent objects it actually > + * enqueued. A return value equal to *nb_events* means that all event ob= jects have been enqueued. > + * > + * @param evdev_id > + * The identifier of the device. > + * @param port_id > + * The identifier of the event port. > + * @param ev > + * Points to an array of *nb_events* objects of type *rte_event* str= ucture which contain the > + * event object enqueue operations to be processed. > + * @param nb_events > + * The number of event objects to enqueue, typically number of > + * rte_event_port_attr_get(...RTE_EVENT_PORT_ATTR_ENQ_DEPTH...) availabl= e for this port. > + * > + * @return > + * The number of event objects actually enqueued on the event device= . The return value can be > + * less than the value of the *nb_events* parameter when the event devic= es queue is full or if > + * invalid parameters are specified in a *rte_event*. If the return valu= e is less than *nb_events*, > + * the remaining events at the end of ev[] are not consumed and the call= er has to take care of them, > + * and rte_errno is set accordingly. Possible errno values include: > + * Due to this newline, generated doxygen coming bad. Please check generated doxygen html page in browser. > + * - EINVAL: The port ID is invalid, device ID is invalid, an event'= s queue ID is invalid, or an > + * event's sched type doesn't match the capabilities of the destination = queue. > + * - ENOSPC: The event port was backpressured and unable to enqueue = one or more events. This > + * error code is only applicable to closed systems. > + */ > +__rte_experimental > +uint16_t rte_event_dma_adapter_enqueue(uint8_t evdev_id, uint8_t port_id= , struct rte_event ev[], > + uint16_t nb_events); > + > +#ifdef __cplusplus > > > extern struct rte_event_fp_ops rte_event_fp_ops[RTE_EVENT_MAX_DEVS]; > diff --git a/lib/eventdev/version.map b/lib/eventdev/version.map > index 7ce09a87bb..5d644b1759 100644 > --- a/lib/eventdev/version.map > +++ b/lib/eventdev/version.map > @@ -133,6 +133,22 @@ EXPERIMENTAL { > rte_event_timer_remaining_ticks_get; > > # added in 23.11 > + rte_event_dma_adapter_caps_get; > + rte_event_dma_adapter_create_ext; > + rte_event_dma_adapter_create; > + rte_event_dma_adapter_event_port_get; > + rte_event_dma_adapter_enqueue; > + rte_event_dma_adapter_free; > + rte_event_dma_adapter_runtime_params_init; > + rte_event_dma_adapter_runtime_params_set; > + rte_event_dma_adapter_runtime_params_get; Sort this list in alphabetical order. > + rte_event_dma_adapter_service_id_get; > + rte_event_dma_adapter_start; > + rte_event_dma_adapter_stop; > + rte_event_dma_adapter_stats_get; > + rte_event_dma_adapter_stats_reset; > + rte_event_dma_adapter_vchan_add; > + rte_event_dma_adapter_vchan_del; > rte_event_eth_rx_adapter_create_ext_with_params; > }; > > -- > 2.25.1 >