From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id E3ED743A4C; Fri, 2 Feb 2024 09:52:22 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id A78ED4026E; Fri, 2 Feb 2024 09:52:22 +0100 (CET) Received: from mail-qt1-f169.google.com (mail-qt1-f169.google.com [209.85.160.169]) by mails.dpdk.org (Postfix) with ESMTP id B941D402BF for ; Fri, 2 Feb 2024 09:52:21 +0100 (CET) Received: by mail-qt1-f169.google.com with SMTP id d75a77b69052e-42ab7522c75so10812981cf.0 for ; Fri, 02 Feb 2024 00:52:21 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1706863941; x=1707468741; darn=dpdk.org; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=Po1qXcbEk+iG2PexCG2NOKf0Im9G53Gg2GgMv4Uf+YE=; b=mCeALuBdCcs+nnpVTQ5mXtgXtN4k2KoMS9rn4m8XNbvgh1lLiH7RUEIOZk6TQGn5IZ yiMWbuSqKn+EEL9lA1fGWpSz329uzn0aDgdPzR+G6oLrYPv3jTTEHMttJQHWTP6SbW2R NTFiIssKUEyWNrnmfa3cx6/D8dd9jAm0Exd5lt7g2NvhM3jrzAbpcJuq4bIzZyQ/pI5n cU+n9YoD8xGn8fN4K7VpvmjBgUq02j6C24cFeC+v2x7KGCSvaTEgmoFWQUlmHusTY7Km YA7ortb9aHqOpuf0S7+6JOoOyuGV7ILA9DxL0JpIyWBFHrqRD+ObBFlsWwkvOfvKKO/1 3bAA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1706863941; x=1707468741; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=Po1qXcbEk+iG2PexCG2NOKf0Im9G53Gg2GgMv4Uf+YE=; b=Bl0i5XYqHerIuRs0JKv4WgrbIBrvNwKNg8mQeuVSx38uP3DNvCfxvOHIqDkQ6t+Ww1 p93zoRMrb/ibv7q1vx9MXvvsK4KYPZgYM1cO5DLOoWMot3zfbe0MT+/dp8gFARIBa3UE fHPPh9VbarfklOSoWWOCagqRzA4ZINIt41VxtXypTmpbHKWnZZdTUb3Y4ZazYANOCILc 5RyYHECPNoqO8g6UNxHZ2qSemaE80wSky/kj68phi1mkOd9AK7x50JRctOeaUOBpU7TD t23+pI4qQNqgl1ukGOQjjKFrh1sryVaWSl5YWAWayrVIkE0Op+W1RTNxoeZGFTQvoOoe XRzg== X-Gm-Message-State: AOJu0YwNqVwA0gAvvxQsrozEqSEH1O3nmAvo1CqGe+9bEUQL+OgxksUH 2SZSyx/LhmkhLrI++7XWp4P5Jxxs4OqrcG5hpRzw29WjaglRQ6IpbgMkdWEtmkYCvbqgvi5bNpZ rnh0Ksxs4cSdZg8IL9XHVFrn/efE= X-Google-Smtp-Source: AGHT+IHPe20uZvAkEi+ZieWydnaiLZ/3DciUVY4XfRhbqli8+nKNd2mLZsk7P5xlFTT94xs/M/UJyznUJ481HYl3roU= X-Received: by 2002:ac8:7c4e:0:b0:42a:b2b8:21c with SMTP id o14-20020ac87c4e000000b0042ab2b8021cmr1732160qtv.13.1706863940946; Fri, 02 Feb 2024 00:52:20 -0800 (PST) MIME-Version: 1.0 References: <20240107153454.3909-1-syalavarthi@marvell.com> <20240107153454.3909-2-syalavarthi@marvell.com> In-Reply-To: <20240107153454.3909-2-syalavarthi@marvell.com> From: Jerin Jacob Date: Fri, 2 Feb 2024 14:21:54 +0530 Message-ID: Subject: Re: [PATCH 01/11] eventdev: introduce ML event adapter library To: Srikanth Yalavarthi , "Naga Harish K, S V" , Erik Gabriel Carrillo , "Gujjar, Abhinandan S" , Amit Prakash Shukla Cc: Thomas Monjalon , Bruce Richardson , Jerin Jacob , dev@dpdk.org, aprabhu@marvell.com, sshankarnara@marvell.com, ptakkar@marvell.com Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org On Sun, Jan 7, 2024 at 9:05=E2=80=AFPM Srikanth Yalavarthi wrote: > > Introduce event ML adapter APIs. This patch provides > information on adapter modes and usage. Application > can use this event adapter interface to transfer > packets between ML device and event device. > > Signed-off-by: Srikanth Yalavarthi 1) Please --thread ---in-reply-to on reply with RFC patch 2) Pleaes add change log 3)In order to merge this series, We need to have UT. it can be based on SW adapter. 4) Added other event adapter maintainers for review in case if anyone interested. > --- > MAINTAINERS | 6 + > config/rte_config.h | 1 + > doc/api/doxy-api-index.md | 1 + > doc/guides/prog_guide/event_ml_adapter.rst | 268 ++++ > doc/guides/prog_guide/eventdev.rst | 10 +- > .../img/event_ml_adapter_op_forward.svg | 1086 +++++++++++++++++ > .../img/event_ml_adapter_op_new.svg | 1079 ++++++++++++++++ > doc/guides/prog_guide/index.rst | 1 + > lib/eventdev/meson.build | 4 +- > lib/eventdev/rte_event_ml_adapter.c | 6 + > lib/eventdev/rte_event_ml_adapter.h | 594 +++++++++ > lib/eventdev/rte_eventdev.h | 45 + > lib/meson.build | 2 +- > lib/mldev/rte_mldev.h | 6 + I perfer this change to come via main tree. Is it possible to move that change as sepereate patch. > diff --git a/MAINTAINERS b/MAINTAINERS > index 0d1c8126e3e..a1125e93621 100644 > --- a/MAINTAINERS > +++ b/MAINTAINERS > @@ -554,6 +554,12 @@ F: drivers/raw/skeleton/ > F: app/test/test_rawdev.c > F: doc/guides/prog_guide/rawdev.rst > > +Eventdev ML Adapter API Add after "Eventdev DMA Adapter API" section > +M: Srikanth Yalavarthi > +T: git://dpdk.org/next/dpdk-next-eventdev > +F: lib/eventdev/*ml_adapter* > +F: doc/guides/prog_guide/event_ml_adapter.rst > + > > index 00000000000..71f6c4b5974 > --- /dev/null > +++ b/doc/guides/prog_guide/event_ml_adapter.rst > @@ -0,0 +1,268 @@ > +.. SPDX-License-Identifier: BSD-3-Clause > + Copyright (c) 2024 Marvell. > + > +Event ML Adapter Library > +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D > + > +DPDK :doc:`Eventdev library ` provides event driven programmin= g model with features > +to schedule events. :doc:`ML Device library ` provides an interfa= ce to ML poll mode > +drivers that support Machine Learning inference operations. Event ML Ada= pter is intended to > +bridge between the event device and the ML device. > + > +Packet flow from ML device to the event device can be accomplished using= software and hardware > +based transfer mechanisms. The adapter queries an eventdev PMD to determ= ine which mechanism to > +be used. The adapter uses an EAL service core function for software base= d packet transfer and > +uses the eventdev PMD functions to configure hardware based packet trans= fer between ML device > +and the event device. ML adapter uses a new event type called ``RTE_EVEN= T_TYPE_MLDEV`` to > +indicate the source of event. > + > +Application can choose to submit an ML operation directly to an ML devic= e or send it to an ML > +adapter via eventdev based on RTE_EVENT_ML_ADAPTER_CAP_INTERNAL_PORT_OP_= FWD capability. The Use ``...`` everywhere accros the this document when using the symbols like RTE_EVENT_ML_ADAPTER_CAP_INTERNAL_PORT_OP_FWD > +first mode is known as the event new (RTE_EVENT_ML_ADAPTER_OP_NEW) mode = and the second as the Same as above.. Please follow the same for rest of the file., > +event forward (RTE_EVENT_ML_ADAPTER_OP_FORWARD) mode. Choice of mode can= be specified while > +creating the adapter. In the former mode, it is the application's respon= sibility to enable > +ingress packet ordering. In the latter mode, it is the adapter's respons= ibility to enable > +ingress packet ordering. > + > + > +Adapter Modes > +------------- > + > +RTE_EVENT_ML_ADAPTER_OP_NEW mode > +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ > + > +In the RTE_EVENT_ML_ADAPTER_OP_NEW mode, application submits ML operatio= ns directly to an ML > +device. The adapter then dequeues ML completions from the ML device and = enqueues them as events > +to the event device. This mode does not ensure ingress ordering as the a= pplication directly > +enqueues to the mldev without going through ML/atomic stage. In this mod= e, events dequeued Is this ML/atomic or atomic ? > +from the adapter are treated as new events. The application has to speci= fy event information > +(response information) which is needed to enqueue an event after the ML = operation is completed. > + > + > + > +API Overview > +------------ > + > +This section has a brief introduction to the event ML adapter APIs. The = application is expected > +to create an adapter which is associated with a single eventdev, then ad= d mldev and queue pair > +to the adapter instance. > + > + > +Create an adapter instance > +~~~~~~~~~~~~~~~~~~~~~~~~~~ > + > +An adapter instance is created using ``rte_event_ml_adapter_create()``. = This function is called > +with event device to be associated with the adapter and port configurati= on for the adapter to > +setup an event port (if the adapter needs to use a service function). > + > +Adapter can be started in ``RTE_EVENT_ML_ADAPTER_OP_NEW`` or ``RTE_EVENT= _ML_ADAPTER_OP_FORWARD`` > +mode. > + > + */ > + > +#ifndef RTE_EVENT_ML_ADAPTER > +#define RTE_EVENT_ML_ADAPTER > + > +/** > + * @file rte_event_ml_adapter.h > + * > + * @warning > + * @b EXPERIMENTAL: > + * All functions in this file may be changed or removed without prior no= tice. > + * > + * ML (Machine Learning) Event Adapter API. > + * > + * Eventdev library provides adapters to bridge between various componen= ts for providing new > + * event source. The event ML adapter is one of those adapters which is = intended to bridge > + * between event devices and ML devices. > + * > + * The ML adapter adds support to enqueue / dequeue ML operations to / f= rom event device. The packet > + * flow between ML device and the event device can be accomplished using= both SW and HW based > + * transfer mechanisms. The adapter uses an EAL service core function fo= r SW based packet transfer > + * and uses the eventdev PMD functions to configure HW based packet tran= sfer between the ML device > + * and the event device. > + * > + * The application can choose to submit a ML operation directly to an ML= device or send it to the ML > + * adapter via eventdev based on RTE_EVENT_ML_ADAPTER_CAP_INTERNAL_PORT_= OP_FWD capability. The first Please check genereated doxygen output. Use @ref to get proper link. Same comment across this file. > + * mode is known as the event new (RTE_EVENT_ML_ADAPTER_OP_NEW) mode and= the second as the event > + * forward (RTE_EVENT_ML_ADAPTER_OP_FORWARD) mode. The choice of mode ca= n be specified while > + * creating the adapter. In the former mode, it is an application respon= sibility to enable ingress > + * packet ordering. In the latter mode, it is the adapter responsibility= to enable the ingress > + * packet ordering. > + * > + * > + * Working model of RTE_EVENT_ML_ADAPTER_OP_NEW mode: > + * > + * +--------------+ +--------------+ > + * | | | ML stage | > + * | Application |---[2]-->| + enqueue to | > + * | | | mldev | > + * +--------------+ +--------------+ > + * ^ ^ | > + * | | [3] > + * [6] [1] | > + * | | | > + * +--------------+ | > + * | | | > + * | Event device | | > + * | | | > + * +--------------+ | > + * ^ | > + * | | > + * [5] | > + * | v > + * +--------------+ +--------------+ > + * | | | | > + * | ML adapter |<--[4]---| mldev | > + * | | | | > + * +--------------+ +--------------+ > + * > + * > + * [1] Application dequeues events from the previous stage. > + * [2] Application prepares the ML operations. > + * [3] ML operations are submitted to mldev by application. > + * [4] ML adapter dequeues ML completions from mldev. > + * [5] ML adapter enqueues events to the eventdev. > + * [6] Application dequeues from eventdev for further processing= . > + * > + * In the RTE_EVENT_ML_ADAPTER_OP_NEW mode, application submits ML opera= tions directly to ML device. > + * The ML adapter then dequeues ML completions from ML device and enqueu= es events to the event > + * device. This mode does not ensure ingress ordering, if the applicatio= n directly enqueues to mldev > + * without going through ML / atomic stage i.e. removing item [1] and [2= ]. ML/atomic or atomic ? > + * > + * Events dequeued from the adapter will be treated as new events. In th= is mode, application needs > + * to specify event information (response information) which is needed t= o enqueue an event after the > + * ML operation is completed. > + * > + * > + */ > + > +#include > + > +#include "rte_eventdev.h" > +#include > + > +#ifdef __cplusplus > +extern "C" { > +#endif > + > +/** > + * ML event adapter mode > + */ > +enum rte_event_ml_adapter_mode { > + RTE_EVENT_ML_ADAPTER_OP_NEW, > + /**< Start the ML adapter in event new mode. > + * @see RTE_EVENT_OP_NEW. > + * > + * Application submits ML operations to the mldev. Adapter only d= equeues the ML completions > + * from mldev and enqueue events to the eventdev. > + */ > + > + RTE_EVENT_ML_ADAPTER_OP_FORWARD, > + /**< Start the ML adapter in event forward mode. > + * @see RTE_EVENT_OP_FORWARD. > + * > + * Application submits ML requests as events to the ML adapter or= ML device based on > + * RTE_EVENT_ML_ADAPTER_CAP_INTERNAL_PORT_OP_FWD capability. ML c= ompletions are enqueued > + * back to the eventdev by ML adapter. > + */ > +}; > + > +}; > + > + > +/** > + * Free an event ML adapter > + * > + * @param id > + * Adapter identifier. > + * @return > + * - 0: Success > + * - <0: Error code on failure, If the adapter still has queue pairs= added to it, the function > + * returns -EBUSY. > + */ > +int > +rte_event_ml_adapter_free(uint8_t id); > + > + > +/** > + * Enqueue a burst of ML operations as event objects supplied in *rte_ev= ent* structure on an event > + * ML adapter designated by its event *evdev_id* through the event port = specified by *port_id*. This > + * function is supported if the eventdev PMD has the #RTE_EVENT_ML_ADAPT= ER_CAP_INTERNAL_PORT_OP_FWD > + * capability flag set. > + * > + * The *nb_events* parameter is the number of event objects to enqueue t= hat are supplied in the > + * *ev* array of *rte_event* structure. > + * > + * The rte_event_ml_adapter_enqueue() function returns the number of eve= nt objects it actually > + * enqueued. A return value equal to *nb_events* means that all event ob= jects have been enqueued. > + * > + * @param evdev_id > + * The identifier of the device. > + * @param port_id > + * The identifier of the event port. > + * @param ev > + * Points to an array of *nb_events* objects of type *rte_event* str= ucture which contain the > + * event object enqueue operations to be processed. > + * @param nb_events > + * The number of event objects to enqueue, typically number of > + * rte_event_port_attr_get(...RTE_EVENT_PORT_ATTR_ENQ_DEPTH...) availabl= e for this port. > + * > + * @return > + * The number of event objects actually enqueued on the event device= . The return value can be > + * less than the value of the *nb_events* parameter when the event devic= es queue is full or if > + * invalid parameters are specified in a *rte_event*. If the return valu= e is less than *nb_events*, > + * the remaining events at the end of ev[] are not consumed and the call= er has to take care of them, > + * and rte_errno is set accordingly. Possible errno values include: > + * > + * - EINVAL: The port ID is invalid, device ID is invalid, an event'= s queue ID is invalid, or an > + * event's sched type doesn't match the capabilities of the destination = queue. Html rendering of return values are not correct. > + * - ENOSPC: The event port was backpressured and unable to enqueue = one or more events. This > + * error code is only applicable to closed systems. , > diff --git a/lib/mldev/rte_mldev.h b/lib/mldev/rte_mldev.h I perfer this change to come via main tree. Is it possible to move that change as sepereate patch. > index 27e372fbcf1..a031592de61 100644 > --- a/lib/mldev/rte_mldev.h > +++ b/lib/mldev/rte_mldev.h > @@ -469,6 +469,12 @@ struct rte_ml_op { > * dequeue and enqueue operation. > * The application should not modify this field. > */ > + uint32_t private_data_offset; > + /**< Offset to indicate start of private data (if any). > + * The offset is counted from the start of the rte_ml_op. > + * The offset provides an offset to locate the request / > + * response information in the rte_ml_op. > + */ > } __rte_cache_aligned; > > /* Enqueue/Dequeue operations */ > -- > 2.42.0 >