DPDK patches and discussions
 help / color / mirror / Atom feed
From: Tyler Retzlaff <roretzla@linux.microsoft.com>
To: Bruce Richardson <bruce.richardson@intel.com>
Cc: dev@dpdk.org, Ajit Khaparde <ajit.khaparde@broadcom.com>,
	Andrew Boyer <andrew.boyer@amd.com>,
	Andrew Rybchenko <andrew.rybchenko@oktetlabs.ru>,
	Chenbo Xia <chenbox@nvidia.com>,
	Chengwen Feng <fengchengwen@huawei.com>,
	Dariusz Sosnowski <dsosnowski@nvidia.com>,
	David Christensen <drc@linux.vnet.ibm.com>,
	Hyong Youb Kim <hyonkim@cisco.com>,
	Jerin Jacob <jerinj@marvell.com>, Jie Hai <haijie1@huawei.com>,
	Jingjing Wu <jingjing.wu@intel.com>,
	John Daley <johndale@cisco.com>,
	Kevin Laatz <kevin.laatz@intel.com>,
	Kiran Kumar K <kirankumark@marvell.com>,
	Konstantin Ananyev <konstantin.v.ananyev@yandex.ru>,
	Maciej Czekaj <mczekaj@marvell.com>,
	Matan Azrad <matan@nvidia.com>,
	Maxime Coquelin <maxime.coquelin@redhat.com>,
	Nithin Dabilpuram <ndabilpuram@marvell.com>,
	Ori Kam <orika@nvidia.com>, Ruifeng Wang <ruifeng.wang@arm.com>,
	Satha Rao <skoteshwar@marvell.com>,
	Somnath Kotur <somnath.kotur@broadcom.com>,
	Suanming Mou <suanmingm@nvidia.com>,
	Sunil Kumar Kori <skori@marvell.com>,
	Viacheslav Ovsiienko <viacheslavo@nvidia.com>,
	Yisen Zhuang <yisen.zhuang@huawei.com>,
	Yuying Zhang <Yuying.Zhang@intel.com>,
	mb@smartsharesystems.com
Subject: Re: [PATCH v7 2/4] mbuf: remove rte marker fields
Date: Thu, 21 Mar 2024 08:31:23 -0700	[thread overview]
Message-ID: <20240321153123.GA8873@linuxonhyperv3.guj3yctzbm1etfxqx2vob5hsef.xx.internal.cloudapp.net> (raw)
In-Reply-To: <ZfwMoln6d5q6jdE0@bricha3-mobl1.ger.corp.intel.com>

On Thu, Mar 21, 2024 at 10:32:02AM +0000, Bruce Richardson wrote:
> On Wed, Mar 20, 2024 at 03:01:36PM -0700, Tyler Retzlaff wrote:
> > RTE_MARKER typedefs are a GCC extension unsupported by MSVC. Remove
> > RTE_MARKER fields from rte_mbuf struct.
> > 
> > Maintain alignment of fields after removed cacheline1 marker by placing
> > C11 alignas(RTE_CACHE_LINE_MIN_SIZE).
> > 
> > Provide new rearm_data and rx_descriptor_fields1 fields in anonymous
> > unions as single element arrays of with types matching the original
> > markers to maintain API compatibility.
> > 
> > Signed-off-by: Tyler Retzlaff <roretzla@linux.microsoft.com>
> > ---
> >  doc/guides/rel_notes/release_24_03.rst |   2 +
> >  lib/mbuf/rte_mbuf.h                    |   4 +-
> >  lib/mbuf/rte_mbuf_core.h               | 188 ++++++++++++++++++---------------
> >  3 files changed, 104 insertions(+), 90 deletions(-)
> > 
> > diff --git a/doc/guides/rel_notes/release_24_03.rst b/doc/guides/rel_notes/release_24_03.rst
> > index 14826ea..4f18cca 100644
> > --- a/doc/guides/rel_notes/release_24_03.rst
> > +++ b/doc/guides/rel_notes/release_24_03.rst
> > @@ -216,6 +216,8 @@ Removed Items
> >  
> >  * acc101: Removed obsolete code for non productized HW variant.
> >  
> > +* mbuf: ``RTE_MARKER`` fields ``cacheline0`` and ``cacheline1``
> > +  have been removed from ``struct rte_mbuf``.
> >  
> >  API Changes
> >  -----------
> > diff --git a/lib/mbuf/rte_mbuf.h b/lib/mbuf/rte_mbuf.h
> > index 286b32b..4c4722e 100644
> > --- a/lib/mbuf/rte_mbuf.h
> > +++ b/lib/mbuf/rte_mbuf.h
> > @@ -108,7 +108,7 @@
> >  static inline void
> >  rte_mbuf_prefetch_part1(struct rte_mbuf *m)
> >  {
> > -	rte_prefetch0(&m->cacheline0);
> > +	rte_prefetch0(m);
> >  }
> >  
> >  /**
> > @@ -126,7 +126,7 @@
> >  rte_mbuf_prefetch_part2(struct rte_mbuf *m)
> >  {
> >  #if RTE_CACHE_LINE_SIZE == 64
> > -	rte_prefetch0(&m->cacheline1);
> > +	rte_prefetch0(RTE_PTR_ADD(m, RTE_CACHE_LINE_MIN_SIZE));
> >  #else
> >  	RTE_SET_USED(m);
> >  #endif
> > diff --git a/lib/mbuf/rte_mbuf_core.h b/lib/mbuf/rte_mbuf_core.h
> > index 9f58076..665213c 100644
> > --- a/lib/mbuf/rte_mbuf_core.h
> > +++ b/lib/mbuf/rte_mbuf_core.h
> > @@ -465,8 +465,6 @@ enum {
> >   * The generic rte_mbuf, containing a packet mbuf.
> >   */
> >  struct __rte_cache_aligned rte_mbuf {
> > -	RTE_MARKER cacheline0;
> > -
> >  	void *buf_addr;           /**< Virtual address of segment buffer. */
> >  #if RTE_IOVA_IN_MBUF
> >  	/**
> > @@ -488,116 +486,130 @@ struct __rte_cache_aligned rte_mbuf {
> >  #endif
> >  
> >  	/* next 8 bytes are initialised on RX descriptor rearm */
> > -	RTE_MARKER64 rearm_data;
> > -	uint16_t data_off;
> > -
> > -	/**
> > -	 * Reference counter. Its size should at least equal to the size
> > -	 * of port field (16 bits), to support zero-copy broadcast.
> > -	 * It should only be accessed using the following functions:
> > -	 * rte_mbuf_refcnt_update(), rte_mbuf_refcnt_read(), and
> > -	 * rte_mbuf_refcnt_set(). The functionality of these functions (atomic,
> > -	 * or non-atomic) is controlled by the RTE_MBUF_REFCNT_ATOMIC flag.
> > -	 */
> > -	RTE_ATOMIC(uint16_t) refcnt;
> > +	union {
> > +		uint64_t rearm_data[1];
> > +		__extension__
> > +		struct {
> > +			uint16_t data_off;
> > +
> > +			/**
> > +			 * Reference counter. Its size should at least equal to the size
> > +			 * of port field (16 bits), to support zero-copy broadcast.
> > +			 * It should only be accessed using the following functions:
> > +			 * rte_mbuf_refcnt_update(), rte_mbuf_refcnt_read(), and
> > +			 * rte_mbuf_refcnt_set(). The functionality of these functions (atomic,
> > +			 * or non-atomic) is controlled by the RTE_MBUF_REFCNT_ATOMIC flag.
> > +			 */
> > +			RTE_ATOMIC(uint16_t) refcnt;
> >  
> > -	/**
> > -	 * Number of segments. Only valid for the first segment of an mbuf
> > -	 * chain.
> > -	 */
> > -	uint16_t nb_segs;
> > +			/**
> > +			 * Number of segments. Only valid for the first segment of an mbuf
> > +			 * chain.
> > +			 */
> > +			uint16_t nb_segs;
> >  
> > -	/** Input port (16 bits to support more than 256 virtual ports).
> > -	 * The event eth Tx adapter uses this field to specify the output port.
> > -	 */
> > -	uint16_t port;
> > +			/** Input port (16 bits to support more than 256 virtual ports).
> > +			 * The event eth Tx adapter uses this field to specify the output port.
> > +			 */
> > +			uint16_t port;
> > +		};
> > +	};
> >  
> >  	uint64_t ol_flags;        /**< Offload features. */
> >  
> >  	/* remaining bytes are set on RX when pulling packet from descriptor */
> > -	RTE_MARKER rx_descriptor_fields1;
> > -
> > -	/*
> > -	 * The packet type, which is the combination of outer/inner L2, L3, L4
> > -	 * and tunnel types. The packet_type is about data really present in the
> > -	 * mbuf. Example: if vlan stripping is enabled, a received vlan packet
> > -	 * would have RTE_PTYPE_L2_ETHER and not RTE_PTYPE_L2_VLAN because the
> > -	 * vlan is stripped from the data.
> > -	 */
> >  	union {
> > -		uint32_t packet_type; /**< L2/L3/L4 and tunnel information. */
> > +		void *rx_descriptor_fields1[1];
> 
> Can we make this array the actual size of all the fields, rather than just
> an 8-byte value? That would allow the right think to be done if assigning
> the descriptor fields from one mbuf to another, or when using memset or
> memcpy on them.

Morten pointed out in a previous version that the marker being an array of
void * was a bug to begin with.

The other field of the union is 24 bytes. I suppose it would be possible
to conditionally compile the array to be either 3 or 6 elements. I guess
this would be an improvement over what the marker is doing now.

Just a reminder that we cannot 'correct' the type since that would
require adaptation of calling code.

What do others think? Keep it as a single element array or conditional
compile based on sizeof(void *)?

> 
> /Bruce

  reply	other threads:[~2024-03-21 15:31 UTC|newest]

Thread overview: 171+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2024-01-30 23:26 [PATCH] replace GCC marker extension with C11 anonymous unions Tyler Retzlaff
2024-01-30 23:26 ` [PATCH] mbuf: " Tyler Retzlaff
2024-01-31  9:18   ` Morten Brørup
2024-01-31 21:09     ` Tyler Retzlaff
2024-01-31 22:39       ` Morten Brørup
2024-01-31 13:49   ` Bruce Richardson
2024-01-31 20:45     ` Tyler Retzlaff
2024-01-31 22:55       ` Morten Brørup
2024-02-13  6:45   ` [PATCH v2] RFC: " Tyler Retzlaff
2024-02-13  6:45     ` [PATCH v2] mbuf: " Tyler Retzlaff
2024-02-13 16:58       ` Morten Brørup
2024-02-13 18:48         ` Tyler Retzlaff
2024-02-13 19:27           ` Morten Brørup
2024-02-13 20:00             ` Tyler Retzlaff
2024-02-13  8:57     ` [PATCH v2] RFC: " Bruce Richardson
2024-02-13 17:09     ` Morten Brørup
2024-02-13 23:33   ` [PATCH v3] RFC deprecate RTE_MARKER in struct rte_mbuf Tyler Retzlaff
2024-02-13 23:33     ` [PATCH v3] mbuf: deprecate GCC marker in rte mbuf struct Tyler Retzlaff
2024-02-14 10:46       ` Morten Brørup
2024-02-14 20:16         ` Tyler Retzlaff
2024-02-14 10:49     ` [PATCH v3] RFC deprecate RTE_MARKER in struct rte_mbuf Morten Brørup
2024-02-26  1:18     ` Stephen Hemminger
2024-02-15  6:21 ` [PATCH v4 00/18] stop using zero sized marker fields Tyler Retzlaff
2024-02-15  6:21   ` [PATCH v4 01/18] mbuf: deprecate GCC marker in rte mbuf struct Tyler Retzlaff
2024-02-18  2:28     ` fengchengwen
2024-02-18 12:39     ` Thomas Monjalon
2024-02-18 13:07       ` Morten Brørup
2024-02-18 15:22         ` Thomas Monjalon
2024-02-18 16:20           ` Morten Brørup
2024-02-20 17:24           ` Tyler Retzlaff
2024-02-20 17:20       ` Tyler Retzlaff
2024-02-20 17:53         ` Thomas Monjalon
2024-02-20 19:16           ` Thomas Monjalon
2024-02-20 19:37             ` Tyler Retzlaff
2024-02-15  6:21   ` [PATCH v4 02/18] mbuf: stop using zero sized marker fields Tyler Retzlaff
2024-02-18  2:38     ` fengchengwen
2024-02-15  6:21   ` [PATCH v4 03/18] net/i40e: " Tyler Retzlaff
2024-02-15  6:21   ` [PATCH v4 04/18] net/iavf: " Tyler Retzlaff
2024-02-15  6:21   ` [PATCH v4 05/18] net/ice: " Tyler Retzlaff
2024-02-15  6:21   ` [PATCH v4 06/18] net/ixgbe: " Tyler Retzlaff
2024-02-15  6:21   ` [PATCH v4 07/18] net/mlx5: " Tyler Retzlaff
2024-02-15  6:21   ` [PATCH v4 08/18] net/sfc: " Tyler Retzlaff
2024-02-15  6:21   ` [PATCH v4 09/18] net/bnxt: " Tyler Retzlaff
2024-02-15  6:21   ` [PATCH v4 10/18] net/enic: " Tyler Retzlaff
2024-02-15  6:21   ` [PATCH v4 11/18] net/fm10k: " Tyler Retzlaff
2024-02-15  6:21   ` [PATCH v4 12/18] net/hns3: " Tyler Retzlaff
2024-02-18  3:00     ` fengchengwen
2024-02-15  6:21   ` [PATCH v4 13/18] net/ionic: " Tyler Retzlaff
2024-02-15  6:21   ` [PATCH v4 14/18] net/thunderx: " Tyler Retzlaff
2024-02-15  6:21   ` [PATCH v4 15/18] net/virtio: " Tyler Retzlaff
2024-02-15  6:21   ` [PATCH v4 16/18] net/cnxk: " Tyler Retzlaff
2024-02-15  6:21   ` [PATCH v4 17/18] common/idpf: " Tyler Retzlaff
2024-02-15  6:21   ` [PATCH v4 18/18] examples/dma: " Tyler Retzlaff
2024-02-15  9:37   ` [PATCH v4 00/18] " Morten Brørup
2024-02-24  8:21 ` [PATCH v5 00/22] stop using RTE_MARKER extensions Tyler Retzlaff
2024-02-24  8:21   ` [PATCH v5 01/22] eal: provide macro to expand marker extensions Tyler Retzlaff
2024-02-24 10:51     ` Thomas Monjalon
2024-02-24 11:18       ` Thomas Monjalon
2024-02-24  8:21   ` [PATCH v5 02/22] mbuf: expand rte markers empty when building with MSVC Tyler Retzlaff
2024-02-24  8:21   ` [PATCH v5 03/22] security: " Tyler Retzlaff
2024-02-24  8:21   ` [PATCH v5 04/22] cryptodev: " Tyler Retzlaff
2024-02-24  8:21   ` [PATCH v5 05/22] mbuf: stop using mbuf cacheline marker fields Tyler Retzlaff
2024-02-24 10:58     ` Thomas Monjalon
2024-02-26 18:20       ` Stephen Hemminger
2024-02-24  8:21   ` [PATCH v5 06/22] mbuf: add mbuf descriptor accessors Tyler Retzlaff
2024-02-24 11:01     ` Thomas Monjalon
2024-02-24  8:21   ` [PATCH v5 07/22] common/idpf: use " Tyler Retzlaff
2024-02-24  8:21   ` [PATCH v5 08/22] net/bnxt: " Tyler Retzlaff
2024-02-24  8:21   ` [PATCH v5 09/22] net/cnxk: " Tyler Retzlaff
2024-02-24  8:21   ` [PATCH v5 10/22] net/enic: " Tyler Retzlaff
2024-02-24  8:21   ` [PATCH v5 11/22] net/fm10k: " Tyler Retzlaff
2024-02-24  8:21   ` [PATCH v5 12/22] net/hns3: " Tyler Retzlaff
2024-02-24  8:21   ` [PATCH v5 13/22] net/i40e: " Tyler Retzlaff
2024-02-24  8:21   ` [PATCH v5 14/22] net/iavf: " Tyler Retzlaff
2024-02-24  8:22   ` [PATCH v5 15/22] net/ice: " Tyler Retzlaff
2024-02-24  8:22   ` [PATCH v5 16/22] net/ionic: " Tyler Retzlaff
2024-02-24  8:22   ` [PATCH v5 17/22] net/ixgbe: " Tyler Retzlaff
2024-02-24  8:22   ` [PATCH v5 18/22] net/mlx5: " Tyler Retzlaff
2024-02-24  8:22   ` [PATCH v5 19/22] net/octeon_ep: " Tyler Retzlaff
2024-02-24  8:22   ` [PATCH v5 20/22] net/sfc: " Tyler Retzlaff
2024-02-24  8:22   ` [PATCH v5 21/22] net/thunderx: " Tyler Retzlaff
2024-02-24  8:22   ` [PATCH v5 22/22] net/virtio: " Tyler Retzlaff
2024-02-24 10:42   ` [PATCH v5 00/22] stop using RTE_MARKER extensions Morten Brørup
2024-02-24 11:13     ` Thomas Monjalon
2024-02-24 11:23       ` Morten Brørup
2024-02-27  5:41 ` [PATCH v6 00/23] stop and remove RTE_MARKER typedefs Tyler Retzlaff
2024-02-27  5:41   ` [PATCH v6 01/23] mbuf: add accessors for rearm and Rx descriptor fields Tyler Retzlaff
2024-02-27  9:10     ` Morten Brørup
2024-02-27 17:17       ` Tyler Retzlaff
2024-02-28  8:28         ` Morten Brørup
2024-02-28 16:20           ` Tyler Retzlaff
2024-02-27  5:41   ` [PATCH v6 02/23] mbuf: consolidate driver asserts for mbuf struct Tyler Retzlaff
2024-02-27 10:02     ` Konstantin Ananyev
2024-03-14 16:51     ` Tyler Retzlaff
2024-02-27  5:41   ` [PATCH v6 03/23] common/idpf: use mbuf descriptor accessors Tyler Retzlaff
2024-02-27  5:41   ` [PATCH v6 04/23] net/bnxt: " Tyler Retzlaff
2024-02-27  5:41   ` [PATCH v6 05/23] net/cnxk: " Tyler Retzlaff
2024-02-27  5:41   ` [PATCH v6 06/23] net/enic: " Tyler Retzlaff
2024-02-27  5:41   ` [PATCH v6 07/23] net/fm10k: " Tyler Retzlaff
2024-02-27  5:41   ` [PATCH v6 08/23] net/hns3: " Tyler Retzlaff
2024-02-27  5:41   ` [PATCH v6 09/23] net/i40e: " Tyler Retzlaff
2024-02-27  5:41   ` [PATCH v6 10/23] net/iavf: " Tyler Retzlaff
2024-02-27  5:41   ` [PATCH v6 11/23] net/ice: " Tyler Retzlaff
2024-02-27  5:41   ` [PATCH v6 12/23] net/ionic: " Tyler Retzlaff
2024-02-27  5:41   ` [PATCH v6 13/23] net/ixgbe: " Tyler Retzlaff
2024-02-27  5:41   ` [PATCH v6 14/23] net/mlx5: " Tyler Retzlaff
2024-02-27  5:41   ` [PATCH v6 15/23] net/octeon_ep: " Tyler Retzlaff
2024-02-27  5:41   ` [PATCH v6 16/23] net/sfc: " Tyler Retzlaff
2024-02-27  5:41   ` [PATCH v6 17/23] net/thunderx: " Tyler Retzlaff
2024-02-27  5:41   ` [PATCH v6 18/23] net/virtio: " Tyler Retzlaff
2024-02-27  5:41   ` [PATCH v6 19/23] examples/dma: use mbuf descriptor accessor Tyler Retzlaff
2024-02-27  5:41   ` [PATCH v6 20/23] mbuf: remove and stop using rte marker fields Tyler Retzlaff
2024-02-27  9:15     ` Morten Brørup
2024-02-27 10:03     ` Konstantin Ananyev
2024-02-27 15:18     ` David Marchand
2024-02-27 16:04       ` Morten Brørup
2024-02-27 17:23       ` Tyler Retzlaff
2024-02-28 10:42         ` David Marchand
2024-02-28 14:03       ` Dodji Seketeli
2024-02-28 14:43         ` David Marchand
2024-02-29 14:50           ` Dodji Seketeli
2024-02-28 14:18     ` David Marchand
2024-02-28 15:01       ` Morten Brørup
2024-02-28 15:33         ` David Marchand
2024-02-28 17:20       ` Tyler Retzlaff
2024-02-27  5:41   ` [PATCH v6 21/23] security: remove " Tyler Retzlaff
2024-02-27  5:41   ` [PATCH v6 22/23] cryptodev: " Tyler Retzlaff
2024-02-27  5:41   ` [PATCH v6 23/23] devtools: forbid new use of rte marker typedefs Tyler Retzlaff
2024-03-20 22:01 ` [PATCH v7 0/4] remove use of RTE_MARKER fields in libraries Tyler Retzlaff
2024-03-20 22:01   ` [PATCH v7 1/4] net/i40e: use inline prefetch function Tyler Retzlaff
2024-03-26 10:16     ` Morten Brørup
2024-03-27 18:14       ` Tyler Retzlaff
2024-03-27 19:45         ` Morten Brørup
2024-03-20 22:01   ` [PATCH v7 2/4] mbuf: remove rte marker fields Tyler Retzlaff
2024-03-21 10:32     ` Bruce Richardson
2024-03-21 15:31       ` Tyler Retzlaff [this message]
2024-03-21 16:19         ` Morten Brørup
2024-03-26 11:12     ` Morten Brørup
2024-03-20 22:01   ` [PATCH v7 3/4] security: " Tyler Retzlaff
2024-03-26 10:28     ` Morten Brørup
2024-03-27 19:59       ` Tyler Retzlaff
2024-03-20 22:01   ` [PATCH v7 4/4] cryptodev: " Tyler Retzlaff
2024-03-26 10:31     ` Morten Brørup
2024-03-27 19:56 ` [PATCH v8 0/4] remove use of RTE_MARKER fields in libraries Tyler Retzlaff
2024-03-27 19:56   ` [PATCH v8 1/4] net/i40e: use inline prefetch function Tyler Retzlaff
2024-03-27 19:56   ` [PATCH v8 2/4] mbuf: remove rte marker fields Tyler Retzlaff
2024-03-27 19:56   ` [PATCH v8 3/4] security: " Tyler Retzlaff
2024-03-27 19:56   ` [PATCH v8 4/4] cryptodev: " Tyler Retzlaff
2024-04-02 20:08 ` [PATCH v9 0/4] remove use of RTE_MARKER fields in libraries Tyler Retzlaff
2024-04-02 20:08   ` [PATCH v9 1/4] net/i40e: use inline prefetch function Tyler Retzlaff
2024-04-02 20:08   ` [PATCH v9 2/4] mbuf: remove rte marker fields Tyler Retzlaff
2024-04-02 20:45     ` Stephen Hemminger
2024-04-02 20:51       ` Tyler Retzlaff
2024-04-02 20:08   ` [PATCH v9 3/4] security: " Tyler Retzlaff
2024-04-02 20:08   ` [PATCH v9 4/4] cryptodev: " Tyler Retzlaff
2024-04-03 17:53 ` [PATCH v10 0/4] remove use of RTE_MARKER fields in libraries Tyler Retzlaff
2024-04-03 17:53   ` [PATCH v10 1/4] net/i40e: use inline prefetch function Tyler Retzlaff
2024-04-03 21:49     ` Stephen Hemminger
2024-04-03 17:53   ` [PATCH v10 2/4] mbuf: remove rte marker fields Tyler Retzlaff
2024-04-03 19:32     ` Morten Brørup
2024-04-03 22:45       ` Tyler Retzlaff
2024-04-03 21:49     ` Stephen Hemminger
2024-04-03 17:53   ` [PATCH v10 3/4] security: " Tyler Retzlaff
2024-04-03 21:50     ` Stephen Hemminger
2024-04-03 17:53   ` [PATCH v10 4/4] cryptodev: " Tyler Retzlaff
2024-04-03 21:50     ` Stephen Hemminger
2024-04-04 17:51 ` [PATCH v11 0/4] remove use of RTE_MARKER fields in libraries Tyler Retzlaff
2024-04-04 17:51   ` [PATCH v11 1/4] net/i40e: use inline prefetch function Tyler Retzlaff
2024-04-04 17:51   ` [PATCH v11 2/4] mbuf: remove rte marker fields Tyler Retzlaff
2024-04-04 17:51   ` [PATCH v11 3/4] security: " Tyler Retzlaff
2024-04-04 17:51   ` [PATCH v11 4/4] cryptodev: " Tyler Retzlaff

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20240321153123.GA8873@linuxonhyperv3.guj3yctzbm1etfxqx2vob5hsef.xx.internal.cloudapp.net \
    --to=roretzla@linux.microsoft.com \
    --cc=Yuying.Zhang@intel.com \
    --cc=ajit.khaparde@broadcom.com \
    --cc=andrew.boyer@amd.com \
    --cc=andrew.rybchenko@oktetlabs.ru \
    --cc=bruce.richardson@intel.com \
    --cc=chenbox@nvidia.com \
    --cc=dev@dpdk.org \
    --cc=drc@linux.vnet.ibm.com \
    --cc=dsosnowski@nvidia.com \
    --cc=fengchengwen@huawei.com \
    --cc=haijie1@huawei.com \
    --cc=hyonkim@cisco.com \
    --cc=jerinj@marvell.com \
    --cc=jingjing.wu@intel.com \
    --cc=johndale@cisco.com \
    --cc=kevin.laatz@intel.com \
    --cc=kirankumark@marvell.com \
    --cc=konstantin.v.ananyev@yandex.ru \
    --cc=matan@nvidia.com \
    --cc=maxime.coquelin@redhat.com \
    --cc=mb@smartsharesystems.com \
    --cc=mczekaj@marvell.com \
    --cc=ndabilpuram@marvell.com \
    --cc=orika@nvidia.com \
    --cc=ruifeng.wang@arm.com \
    --cc=skori@marvell.com \
    --cc=skoteshwar@marvell.com \
    --cc=somnath.kotur@broadcom.com \
    --cc=suanmingm@nvidia.com \
    --cc=viacheslavo@nvidia.com \
    --cc=yisen.zhuang@huawei.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).