DPDK patches and discussions
 help / color / mirror / Atom feed
From: "Mattias Rönnblom" <hofors@lysator.liu.se>
To: Bruce Richardson <bruce.richardson@intel.com>, dev@dpdk.org
Cc: jerinj@marvell.com, mattias.ronnblom@ericsson.com,
	abdullah.sevincer@intel.com, sachin.saxena@oss.nxp.com,
	hemant.agrawal@nxp.com, pbhagavatula@marvell.com,
	pravin.pathak@intel.com
Subject: Re: [PATCH v2 01/11] eventdev: improve doxygen introduction text
Date: Tue, 23 Jan 2024 09:57:58 +0100	[thread overview]
Message-ID: <e86fb56c-20cb-450b-8915-f0684a20cb9b@lysator.liu.se> (raw)
In-Reply-To: <20240119174346.108905-2-bruce.richardson@intel.com>

On 2024-01-19 18:43, Bruce Richardson wrote:
> Make some textual improvements to the introduction to eventdev and event
> devices in the eventdev header file. This text appears in the doxygen
> output for the header file, and introduces the key concepts, for
> example: events, event devices, queues, ports and scheduling.
> 

Great stuff, Bruce.

> This patch makes the following improvements:
> * small textual fixups, e.g. correcting use of singular/plural
> * rewrites of some sentences to improve clarity
> * using doxygen markdown to split the whole large block up into
>    sections, thereby making it easier to read.
> 
> No large-scale changes are made, and blocks are not reordered
> 
> Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
> ---
>   lib/eventdev/rte_eventdev.h | 112 +++++++++++++++++++++---------------
>   1 file changed, 66 insertions(+), 46 deletions(-)
> 
> diff --git a/lib/eventdev/rte_eventdev.h b/lib/eventdev/rte_eventdev.h
> index ec9b02455d..a36c89c7a4 100644
> --- a/lib/eventdev/rte_eventdev.h
> +++ b/lib/eventdev/rte_eventdev.h
> @@ -12,12 +12,13 @@
>    * @file
>    *
>    * RTE Event Device API
> + * ====================
>    *
>    * In a polling model, lcores poll ethdev ports and associated rx queues

"In a polling model, lcores pick up packets from Ethdev ports and 
associated RX queues, runs the processing to completion, and enqueues 
the completed packets to a TX queue. NIC-level receive-side scaling 
(RSS) may be used to balance the load across multiple CPU cores."

I thought it might be worth to be a little more verbose on what is the 
reference model Eventdev is compared to. Maybe you can add "traditional" 
or "archetypal", or "simple" as a prefix to the "polling model". (I 
think I would call this a "simple run-to-completion model" rather than 
"polling model".)

"By contrast, in Eventdev, ingressing* packets are fed into an event 
device, which schedules packets across available lcores, in accordance 
to its configuration. This event-driven programming model offers 
applications automatic multicore scaling, dynamic load balancing, 
pipelining, packet order maintenance, synchronization, and quality of 
service."

* Is this a word?

> - * directly to look for packet. In an event driven model, by contrast, lcores
> - * call the scheduler that selects packets for them based on programmer
> - * specified criteria. Eventdev library adds support for event driven
> - * programming model, which offer applications automatic multicore scaling,
> + * directly to look for packets. In an event driven model, in contrast, lcores
> + * call a scheduler that selects packets for them based on programmer
> + * specified criteria. The eventdev library adds support for the event driven
> + * programming model, which offers applications automatic multicore scaling,
>    * dynamic load balancing, pipelining, packet ingress order maintenance and
>    * synchronization services to simplify application packet processing.
>    *
> @@ -25,12 +26,15 @@
>    *
>    * - The application-oriented Event API that includes functions to setup
>    *   an event device (configure it, setup its queues, ports and start it), to
> - *   establish the link between queues to port and to receive events, and so on.
> + *   establish the links between queues and ports to receive events, and so on.
>    *
>    * - The driver-oriented Event API that exports a function allowing
> - *   an event poll Mode Driver (PMD) to simultaneously register itself as
> + *   an event poll Mode Driver (PMD) to register itself as
>    *   an event device driver.
>    *
> + * Application-oriented Event API
> + * ------------------------------
> + *
>    * Event device components:
>    *
>    *                     +-----------------+
> @@ -75,27 +79,33 @@
>    *            |                                                           |
>    *            +-----------------------------------------------------------+
>    *
> - * Event device: A hardware or software-based event scheduler.
> + * **Event device**: A hardware or software-based event scheduler.
>    *
> - * Event: A unit of scheduling that encapsulates a packet or other datatype
> - * like SW generated event from the CPU, Crypto work completion notification,
> - * Timer expiry event notification etc as well as metadata.
> - * The metadata includes flow ID, scheduling type, event priority, event_type,
> + * **Event**: A unit of scheduling that encapsulates a packet or other datatype,

"Event: Represents an item of work and is the smallest unit of 
scheduling. An event carries metadata, such as queue ID, scheduling 
type, and event priority, and data such as one or more packets or other 
kinds of buffers. Examples of events are a software-generated item of 
work originating from a lcore carrying a packet to be processed, a 
crypto work completion notification and a timer expiry notification."

I've found "work scheduler" as helpful term describing what role an 
event device serve in the system, and thus an event represent an item of 
work. "Event" and "Event device" are also good names, but lead some 
people to think libevent or event loop, which is not exactly right.

> + * such as: SW generated event from the CPU, crypto work completion notification,
> + * timer expiry event notification etc., as well as metadata about the packet or data.
> + * The metadata includes a flow ID (if any), scheduling type, event priority, event_type,
>    * sub_event_type etc.
>    *
> - * Event queue: A queue containing events that are scheduled by the event dev.
> + * **Event queue**: A queue containing events that are scheduled by the event device.
>    * An event queue contains events of different flows associated with scheduling
>    * types, such as atomic, ordered, or parallel.
> + * Each event given to an eventdev must have a valid event queue id field in the metadata,
"eventdev" -> "event device"

> + * to specify on which event queue in the device the event must be placed,
> + * for later scheduling to a core.

Events aren't nessarily scheduled to cores, so remove the last part.

>    *
> - * Event port: An application's interface into the event dev for enqueue and
> + * **Event port**: An application's interface into the event dev for enqueue and
>    * dequeue operations. Each event port can be linked with one or more
>    * event queues for dequeue operations.
> - *
> - * By default, all the functions of the Event Device API exported by a PMD
> - * are lock-free functions which assume to not be invoked in parallel on
> - * different logical cores to work on the same target object. For instance,
> - * the dequeue function of a PMD cannot be invoked in parallel on two logical
> - * cores to operates on same  event port. Of course, this function
> + * Each port should be associated with a single core (enqueue and dequeue is not thread-safe).

Should, or must?

Either it's a MT safety issue, and any lcore can access the port with 
the proper serialization, or it's something where the lcore id used to 
store state between invocations, or some other mechanism that prevents a 
port from being used by multiple threads (lcore or not).

> + * To schedule events to a core, the event device will schedule them to the event port(s)
> + * being polled by that core.

"core" -> "lcore" ?

> + *
> + * *NOTE*: By default, all the functions of the Event Device API exported by a PMD
> + * are lock-free functions, which must not be invoked on the same object in parallel on
> + * different logical cores.

This is a one-sentence contradiction. The term "lock free" implies a 
data structure which is MT safe, achieving this goal without the use of 
locks. A lock-free object thus *may* be called from different threads, 
including different lcore threads.

Ports are not MT safe, and thus one port should not be acted upon by 
more than one thread (either in parallel, or throughout the lifetime of 
the event device/port; see above).

The event device is MT safe, provided the different parallel callers use 
different ports.

A more subtle question and one with a less obvious answer is if the 
caller of also *must* be an EAL thread, or if a registered non-EAL 
thread or even an unregistered non-EAL thread may call the "fast path" 
functions (enqueue, dequeue etc).

For EAL threads, the event device implementation may safely use 
non-preemption safe constructs (like the default ring variant and spin 
locks).

If the caller is a registered non-EAL thread or an EAL thread, the lcore 
id may be used to index various data structures.

If "lcore id"-less threads may call the fast path APIs, what are the MT 
safety guarantees in that case? Like rte_random.h, or something else.

> + * For instance, the dequeue function of a PMD cannot be invoked in parallel on two logical
> + * cores to operate on same  event port. Of course, this function
>    * can be invoked in parallel by different logical cores on different ports.
>    * It is the responsibility of the upper level application to enforce this rule.
>    *
> @@ -107,22 +117,19 @@
>    *
>    * Event devices are dynamically registered during the PCI/SoC device probing
>    * phase performed at EAL initialization time.
> - * When an Event device is being probed, a *rte_event_dev* structure and
> - * a new device identifier are allocated for that device. Then, the
> - * event_dev_init() function supplied by the Event driver matching the probed
> - * device is invoked to properly initialize the device.
> + * When an Event device is being probed, an *rte_event_dev* structure is allocated
> + * for it and the event_dev_init() function supplied by the Event driver
> + * is invoked to properly initialize the device.
>    *
> - * The role of the device init function consists of resetting the hardware or
> - * software event driver implementations.
> + * The role of the device init function is to reset the device hardware or
> + * to initialize the software event driver implementation.
>    *
> - * If the device init operation is successful, the correspondence between
> - * the device identifier assigned to the new device and its associated
> - * *rte_event_dev* structure is effectively registered.
> - * Otherwise, both the *rte_event_dev* structure and the device identifier are
> - * freed.
> + * If the device init operation is successful, the device is assigned a device
> + * id (dev_id) for application use.
> + * Otherwise, the *rte_event_dev* structure is freed.
>    *
>    * The functions exported by the application Event API to setup a device
> - * designated by its device identifier must be invoked in the following order:
> + * must be invoked in the following order:
>    *     - rte_event_dev_configure()
>    *     - rte_event_queue_setup()
>    *     - rte_event_port_setup()
> @@ -130,10 +137,15 @@
>    *     - rte_event_dev_start()
>    *
>    * Then, the application can invoke, in any order, the functions
> - * exported by the Event API to schedule events, dequeue events, enqueue events,
> - * change event queue(s) to event port [un]link establishment and so on.
> - *
> - * Application may use rte_event_[queue/port]_default_conf_get() to get the
> + * exported by the Event API to dequeue events, enqueue events,
> + * and link and unlink event queue(s) to event ports.
> + *
> + * Before configuring a device, an application should call rte_event_dev_info_get()
> + * to determine the capabilities of the event device, and any queue or port
> + * limits of that device. The parameters set in the various device configuration
> + * structures may need to be adjusted based on the max values provided in the
> + * device information structure returned from the info_get API.
> + * An application may use rte_event_[queue/port]_default_conf_get() to get the
>    * default configuration to set up an event queue or event port by
>    * overriding few default values.
>    *
> @@ -145,7 +157,11 @@
>    * when the device is stopped.
>    *
>    * Finally, an application can close an Event device by invoking the
> - * rte_event_dev_close() function.
> + * rte_event_dev_close() function. Once closed, a device cannot be
> + * reconfigured or restarted.
> + *
> + * Driver-Oriented Event API
> + * -------------------------
>    *
>    * Each function of the application Event API invokes a specific function
>    * of the PMD that controls the target device designated by its device
> @@ -164,10 +180,13 @@
>    * supplied in the *event_dev_ops* structure of the *rte_event_dev* structure.
>    *
>    * For performance reasons, the address of the fast-path functions of the
> - * Event driver is not contained in the *event_dev_ops* structure.
> + * Event driver are not contained in the *event_dev_ops* structure.

It's one address, so it should remain "is"?

>    * Instead, they are directly stored at the beginning of the *rte_event_dev*
>    * structure to avoid an extra indirect memory access during their invocation.
>    *
> + * Event Enqueue, Dequeue and Scheduling
> + * -------------------------------------
> + *
>    * RTE event device drivers do not use interrupts for enqueue or dequeue
>    * operation. Instead, Event drivers export Poll-Mode enqueue and dequeue
>    * functions to applications.
> @@ -179,21 +198,22 @@
>    * crypto work completion notification etc
>    *
>    * The *dequeue* operation gets one or more events from the event ports.
> - * The application process the events and send to downstream event queue through
> - * rte_event_enqueue_burst() if it is an intermediate stage of event processing,
> - * on the final stage, the application may use Tx adapter API for maintaining
> - * the ingress order and then send the packet/event on the wire.
> + * The application processes the events and sends them to a downstream event queue through
> + * rte_event_enqueue_burst(), if it is an intermediate stage of event processing.
> + * On the final stage of processing, the application may use the Tx adapter API for maintaining
> + * the event ingress order while sending the packet/event on the wire via NIC Tx.
>    *
>    * The point at which events are scheduled to ports depends on the device.
>    * For hardware devices, scheduling occurs asynchronously without any software
>    * intervention. Software schedulers can either be distributed
>    * (each worker thread schedules events to its own port) or centralized
>    * (a dedicated thread schedules to all ports). Distributed software schedulers
> - * perform the scheduling in rte_event_dequeue_burst(), whereas centralized
> - * scheduler logic need a dedicated service core for scheduling.
> - * The RTE_EVENT_DEV_CAP_DISTRIBUTED_SCHED capability flag is not set
> - * indicates the device is centralized and thus needs a dedicated scheduling
> - * thread that repeatedly calls software specific scheduling function.
> + * perform the scheduling inside the enqueue or dequeue functions, whereas centralized
> + * software schedulers need a dedicated service core for scheduling.
> + * The absence of the RTE_EVENT_DEV_CAP_DISTRIBUTED_SCHED capability flag
> + * indicates that the device is centralized and thus needs a dedicated scheduling
> + * thread, generally a service core,
> + * that repeatedly calls the software specific scheduling function.

In the SW case, what you have is a service that needs to be mapped to a 
service lcore.

"generally a RTE service that should be mapped to one or more service 
lcores"

>    *
>    * An event driven worker thread has following typical workflow on fastpath:
>    * \code{.c}

  reply	other threads:[~2024-01-23  8:58 UTC|newest]

Thread overview: 123+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2024-01-18 13:45 [PATCH v1 0/7] improve eventdev API specification/documentation Bruce Richardson
2024-01-18 13:45 ` [PATCH v1 1/7] eventdev: improve doxygen introduction text Bruce Richardson
2024-01-18 13:45 ` [PATCH v1 2/7] eventdev: move text on driver internals to proper section Bruce Richardson
2024-01-18 13:45 ` [PATCH v1 3/7] eventdev: update documentation on device capability flags Bruce Richardson
2024-01-18 13:45 ` [PATCH v1 4/7] eventdev: cleanup doxygen comments on info structure Bruce Richardson
2024-01-18 13:49   ` Bruce Richardson
2024-01-18 13:45 ` [PATCH v1 5/7] eventdev: improve function documentation for query fns Bruce Richardson
2024-01-18 13:45 ` [PATCH v1 6/7] eventdev: improve doxygen comments on configure struct Bruce Richardson
2024-01-18 13:45 ` [PATCH v1 7/7] eventdev: fix documentation for counting single-link ports Bruce Richardson
2024-01-19 17:43 ` [PATCH v2 00/11] improve eventdev API specification/documentation Bruce Richardson
2024-01-19 17:43   ` [PATCH v2 01/11] eventdev: improve doxygen introduction text Bruce Richardson
2024-01-23  8:57     ` Mattias Rönnblom [this message]
2024-01-23  9:06       ` Bruce Richardson
2024-01-24 11:37         ` Mattias Rönnblom
2024-01-31 13:45       ` Bruce Richardson
2024-01-19 17:43   ` [PATCH v2 02/11] eventdev: move text on driver internals to proper section Bruce Richardson
2024-01-19 17:43   ` [PATCH v2 03/11] eventdev: update documentation on device capability flags Bruce Richardson
2024-01-23  9:18     ` Mattias Rönnblom
2024-01-23  9:34       ` Bruce Richardson
2024-01-31 14:09       ` Bruce Richardson
2024-02-02  8:58         ` Mattias Rönnblom
2024-02-02 11:20           ` Bruce Richardson
2024-01-19 17:43   ` [PATCH v2 04/11] eventdev: cleanup doxygen comments on info structure Bruce Richardson
2024-01-23  9:35     ` Mattias Rönnblom
2024-01-23  9:43       ` Bruce Richardson
2024-01-24 11:51         ` Mattias Rönnblom
2024-01-31 14:37           ` Bruce Richardson
2024-02-02  9:24             ` Mattias Rönnblom
2024-02-02 10:30               ` Bruce Richardson
2024-01-19 17:43   ` [PATCH v2 05/11] eventdev: improve function documentation for query fns Bruce Richardson
2024-01-23  9:40     ` Mattias Rönnblom
2024-01-19 17:43   ` [PATCH v2 06/11] eventdev: improve doxygen comments on configure struct Bruce Richardson
2024-01-23  9:46     ` Mattias Rönnblom
2024-01-31 16:15       ` Bruce Richardson
2024-01-19 17:43   ` [PATCH v2 07/11] eventdev: fix documentation for counting single-link ports Bruce Richardson
2024-01-23  9:48     ` Mattias Rönnblom
2024-01-23  9:56       ` Bruce Richardson
2024-01-31 16:18         ` Bruce Richardson
2024-01-19 17:43   ` [PATCH v2 08/11] eventdev: improve doxygen comments on config fns Bruce Richardson
2024-01-23 10:00     ` Mattias Rönnblom
2024-01-23 10:07       ` Bruce Richardson
2024-01-19 17:43   ` [PATCH v2 09/11] eventdev: improve doxygen comments for control APIs Bruce Richardson
2024-01-23 10:10     ` Mattias Rönnblom
2024-01-19 17:43   ` [PATCH v2 10/11] eventdev: RFC clarify comments on scheduling types Bruce Richardson
2024-01-23 16:19     ` Mattias Rönnblom
2024-01-24 11:21       ` Bruce Richardson
2024-01-31 17:54       ` Bruce Richardson
2024-01-19 17:43   ` [PATCH v2 11/11] eventdev: RFC clarify docs on event object fields Bruce Richardson
2024-01-24 11:34     ` Mattias Rönnblom
2024-02-01 16:59       ` Bruce Richardson
2024-02-02  9:38         ` Mattias Rönnblom
2024-02-02 11:33           ` Bruce Richardson
2024-02-02 12:02             ` Bruce Richardson
2024-02-01 17:02       ` Bruce Richardson
2024-02-02  9:14         ` Bruce Richardson
2024-02-02  9:22         ` Jerin Jacob
2024-02-02  9:36           ` Bruce Richardson
2024-02-02  9:45         ` Mattias Rönnblom
2024-02-02 10:32           ` Bruce Richardson
2024-02-01  9:35     ` Bruce Richardson
2024-02-01 15:00       ` Jerin Jacob
2024-02-01 15:24         ` Bruce Richardson
2024-02-01 16:20           ` Jerin Jacob
2024-02-02 12:39   ` [PATCH v3 00/11] improve eventdev API specification/documentation Bruce Richardson
2024-02-02 12:39     ` [PATCH v3 01/11] eventdev: improve doxygen introduction text Bruce Richardson
2024-02-07 10:14       ` Jerin Jacob
2024-02-08  9:50         ` Mattias Rönnblom
2024-02-09  8:43           ` Jerin Jacob
2024-02-10  7:24             ` Mattias Rönnblom
2024-02-20 16:28               ` Bruce Richardson
2024-02-20 16:33         ` Bruce Richardson
2024-02-02 12:39     ` [PATCH v3 02/11] eventdev: move text on driver internals to proper section Bruce Richardson
2024-02-02 12:39     ` [PATCH v3 03/11] eventdev: update documentation on device capability flags Bruce Richardson
2024-02-07 10:30       ` Jerin Jacob
2024-02-20 16:42         ` Bruce Richardson
2024-02-02 12:39     ` [PATCH v3 04/11] eventdev: cleanup doxygen comments on info structure Bruce Richardson
2024-02-02 12:39     ` [PATCH v3 05/11] eventdev: improve function documentation for query fns Bruce Richardson
2024-02-02 12:39     ` [PATCH v3 06/11] eventdev: improve doxygen comments on configure struct Bruce Richardson
2024-02-02 12:39     ` [PATCH v3 07/11] eventdev: improve doxygen comments on config fns Bruce Richardson
2024-02-02 12:39     ` [PATCH v3 08/11] eventdev: improve doxygen comments for control APIs Bruce Richardson
2024-02-02 12:39     ` [PATCH v3 09/11] eventdev: improve comments on scheduling types Bruce Richardson
2024-02-08  9:18       ` Jerin Jacob
2024-02-08 10:04         ` Mattias Rönnblom
2024-02-20 17:23           ` Bruce Richardson
2024-02-02 12:39     ` [PATCH v3 10/11] eventdev: clarify docs on event object fields and op types Bruce Richardson
2024-02-09  9:14       ` Jerin Jacob
2024-02-20 17:39         ` Bruce Richardson
2024-02-21  9:31           ` Jerin Jacob
2024-02-21 10:28             ` Bruce Richardson
2024-02-20 17:50         ` Bruce Richardson
2024-02-20 18:03         ` Bruce Richardson
2024-02-02 12:39     ` [PATCH v3 11/11] eventdev: drop comment for anon union from doxygen Bruce Richardson
2024-02-21 10:32   ` [PATCH v4 00/12] improve eventdev API specification/documentation Bruce Richardson
2024-02-21 10:32     ` [PATCH v4 01/12] eventdev: improve doxygen introduction text Bruce Richardson
2024-02-26  4:51       ` [EXT] " Pavan Nikhilesh Bhagavatula
2024-02-26  9:59         ` Bruce Richardson
2024-02-29 16:13           ` Jerin Jacob
2024-02-21 10:32     ` [PATCH v4 02/12] eventdev: move text on driver internals to proper section Bruce Richardson
2024-02-26  5:01       ` [EXT] " Pavan Nikhilesh Bhagavatula
2024-02-21 10:32     ` [PATCH v4 03/12] eventdev: update documentation on device capability flags Bruce Richardson
2024-02-26  5:07       ` [EXT] " Pavan Nikhilesh Bhagavatula
2024-02-21 10:32     ` [PATCH v4 04/12] eventdev: cleanup doxygen comments on info structure Bruce Richardson
2024-02-26  5:18       ` [EXT] " Pavan Nikhilesh Bhagavatula
2024-02-21 10:32     ` [PATCH v4 05/12] eventdev: improve function documentation for query fns Bruce Richardson
2024-02-26  5:18       ` [EXT] " Pavan Nikhilesh Bhagavatula
2024-02-21 10:32     ` [PATCH v4 06/12] eventdev: improve doxygen comments on configure struct Bruce Richardson
2024-02-26  6:36       ` [EXT] " Pavan Nikhilesh Bhagavatula
2024-02-21 10:32     ` [PATCH v4 07/12] eventdev: improve doxygen comments on config fns Bruce Richardson
2024-02-26  6:43       ` [EXT] " Pavan Nikhilesh Bhagavatula
2024-02-26  6:44         ` Pavan Nikhilesh Bhagavatula
2024-02-21 10:32     ` [PATCH v4 08/12] eventdev: improve doxygen comments for control APIs Bruce Richardson
2024-02-26  6:44       ` [EXT] " Pavan Nikhilesh Bhagavatula
2024-02-21 10:32     ` [PATCH v4 09/12] eventdev: improve comments on scheduling types Bruce Richardson
2024-02-26  6:49       ` [EXT] " Pavan Nikhilesh Bhagavatula
2024-02-21 10:32     ` [PATCH v4 10/12] eventdev: clarify docs on event object fields and op types Bruce Richardson
2024-02-26  6:52       ` [EXT] " Pavan Nikhilesh Bhagavatula
2024-02-21 10:32     ` [PATCH v4 11/12] eventdev: drop comment for anon union from doxygen Bruce Richardson
2024-02-26  6:52       ` [EXT] " Pavan Nikhilesh Bhagavatula
2024-02-21 10:32     ` [PATCH v4 12/12] eventdev: fix doxygen processing of event vector struct Bruce Richardson
2024-02-26  6:53       ` [EXT] " Pavan Nikhilesh Bhagavatula
2024-03-04 15:35       ` Thomas Monjalon
2024-03-04 15:49         ` Bruce Richardson
2024-02-23 12:36     ` [PATCH v4 00/12] improve eventdev API specification/documentation Jerin Jacob

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=e86fb56c-20cb-450b-8915-f0684a20cb9b@lysator.liu.se \
    --to=hofors@lysator.liu.se \
    --cc=abdullah.sevincer@intel.com \
    --cc=bruce.richardson@intel.com \
    --cc=dev@dpdk.org \
    --cc=hemant.agrawal@nxp.com \
    --cc=jerinj@marvell.com \
    --cc=mattias.ronnblom@ericsson.com \
    --cc=pbhagavatula@marvell.com \
    --cc=pravin.pathak@intel.com \
    --cc=sachin.saxena@oss.nxp.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).