From: "Morten Brørup" <mb@smartsharesystems.com>
To: "David Marchand" <david.marchand@redhat.com>,
"Tyler Retzlaff" <roretzla@linux.microsoft.com>
Cc: <dev@dpdk.org>, "Ajit Khaparde" <ajit.khaparde@broadcom.com>,
"Andrew Boyer" <andrew.boyer@amd.com>,
"Andrew Rybchenko" <andrew.rybchenko@oktetlabs.ru>,
"Bruce Richardson" <bruce.richardson@intel.com>,
"Chenbo Xia" <chenbox@nvidia.com>,
"Chengwen Feng" <fengchengwen@huawei.com>,
"Dariusz Sosnowski" <dsosnowski@nvidia.com>,
"David Christensen" <drc@linux.vnet.ibm.com>,
"Hyong Youb Kim" <hyonkim@cisco.com>,
"Jerin Jacob" <jerinj@marvell.com>,
"Jie Hai" <haijie1@huawei.com>,
"Jingjing Wu" <jingjing.wu@intel.com>,
"John Daley" <johndale@cisco.com>,
"Kevin Laatz" <kevin.laatz@intel.com>,
"Kiran Kumar K" <kirankumark@marvell.com>,
"Konstantin Ananyev" <konstantin.v.ananyev@yandex.ru>,
"Maciej Czekaj" <mczekaj@marvell.com>,
"Matan Azrad" <matan@nvidia.com>,
"Maxime Coquelin" <maxime.coquelin@redhat.com>,
"Nithin Dabilpuram" <ndabilpuram@marvell.com>,
"Ori Kam" <orika@nvidia.com>,
"Ruifeng Wang" <ruifeng.wang@arm.com>,
"Satha Rao" <skoteshwar@marvell.com>,
"Somnath Kotur" <somnath.kotur@broadcom.com>,
"Suanming Mou" <suanmingm@nvidia.com>,
"Sunil Kumar Kori" <skori@marvell.com>,
"Viacheslav Ovsiienko" <viacheslavo@nvidia.com>,
"Yisen Zhuang" <yisen.zhuang@huawei.com>,
"Yuying Zhang" <Yuying.Zhang@intel.com>,
"Thomas Monjalon" <thomas@monjalon.net>
Subject: RE: [PATCH v6 20/23] mbuf: remove and stop using rte marker fields
Date: Wed, 28 Feb 2024 16:01:34 +0100 [thread overview]
Message-ID: <98CBD80474FA8B44BF855DF32C47DC35E9F283@smartserver.smartshare.dk> (raw)
In-Reply-To: <CAJFAV8xF+Bt0DReeX69CJcg6WMxYvcU+SVJskFnBzdAGn9ckYw@mail.gmail.com>
> From: David Marchand [mailto:david.marchand@redhat.com]
> Sent: Wednesday, 28 February 2024 15.19
>
> On Tue, Feb 27, 2024 at 6:44 AM Tyler Retzlaff
> <roretzla@linux.microsoft.com> wrote:
> >
> > RTE_MARKER typedefs are a GCC extension unsupported by MSVC. Remove
> > RTE_MARKER fields from rte_mbuf struct.
> >
> > Maintain alignment of fields after removed cacheline1 marker by placing
> > C11 alignas(RTE_CACHE_LINE_MIN_SIZE).
> >
> > Update implementation of rte_mbuf_prefetch_part1() and
> > rte_mbuf_prefetch_part2() inline functions calculate pointer for
> > prefetch of cachline0 and cachline1 without using removed markers.
> >
> > Update static_assert of rte_mbuf struct fields to reference data_off and
> > packet_type fields that occupy the original offsets of the marker
> > fields.
> >
> > Signed-off-by: Tyler Retzlaff <roretzla@linux.microsoft.com>
> > ---
> > doc/guides/rel_notes/release_24_03.rst | 9 ++++++++
> > lib/mbuf/rte_mbuf.h | 4 ++--
> > lib/mbuf/rte_mbuf_core.h | 39 +++++++++++++------------------
> ---
> > 3 files changed, 26 insertions(+), 26 deletions(-)
> >
> > diff --git a/doc/guides/rel_notes/release_24_03.rst
> b/doc/guides/rel_notes/release_24_03.rst
> > index 879bb49..67750f2 100644
> > --- a/doc/guides/rel_notes/release_24_03.rst
> > +++ b/doc/guides/rel_notes/release_24_03.rst
> > @@ -156,6 +156,15 @@ Removed Items
> > The application reserved statically defined logtypes
> ``RTE_LOGTYPE_USER1..RTE_LOGTYPE_USER8``
> > are still defined.
> >
> > +* mbuf: ``RTE_MARKER`` fields ``cacheline0`` ``cacheline1``
> > + ``rx_descriptor_fields1`` and ``RTE_MARKER64`` field ``rearm_data``
> > + have been removed from ``struct rte_mbuf``.
> > + Prefetch of ``cacheline0`` and ``cacheline1`` may be achieved through
> > + ``rte_mbuf_prefetch_part1()`` and ``rte_mbuf_prefetch_part2()`` inline
> > + functions respectively.
> > + Access to ``rearm_data`` and ``rx_descriptor_fields1`` should be
> > + through new inline functions ``rte_mbuf_rearm_data()`` and
> > + ``rte_mbuf_rx_descriptor_fields1()`` respectively.
> >
> > API Changes
> > -----------
> > diff --git a/lib/mbuf/rte_mbuf.h b/lib/mbuf/rte_mbuf.h
> > index aa7495b..61cda20 100644
> > --- a/lib/mbuf/rte_mbuf.h
> > +++ b/lib/mbuf/rte_mbuf.h
> > @@ -108,7 +108,7 @@
> > static inline void
> > rte_mbuf_prefetch_part1(struct rte_mbuf *m)
> > {
> > - rte_prefetch0(&m->cacheline0);
> > + rte_prefetch0(m);
> > }
> >
> > /**
> > @@ -126,7 +126,7 @@
> > rte_mbuf_prefetch_part2(struct rte_mbuf *m)
> > {
> > #if RTE_CACHE_LINE_SIZE == 64
> > - rte_prefetch0(&m->cacheline1);
> > + rte_prefetch0(RTE_PTR_ADD(m, RTE_CACHE_LINE_MIN_SIZE));
> > #else
> > RTE_SET_USED(m);
> > #endif
> > diff --git a/lib/mbuf/rte_mbuf_core.h b/lib/mbuf/rte_mbuf_core.h
> > index 36551c2..4e06f15 100644
> > --- a/lib/mbuf/rte_mbuf_core.h
> > +++ b/lib/mbuf/rte_mbuf_core.h
> > @@ -18,6 +18,7 @@
> >
> > #include <assert.h>
> > #include <stddef.h>
> > +#include <stdalign.h>
> > #include <stdint.h>
> >
> > #include <rte_common.h>
> > @@ -467,8 +468,6 @@ enum {
> > * The generic rte_mbuf, containing a packet mbuf.
> > */
> > struct rte_mbuf {
> > - RTE_MARKER cacheline0;
> > -
> > void *buf_addr; /**< Virtual address of segment buffer. */
> > #if RTE_IOVA_IN_MBUF
> > /**
> > @@ -495,7 +494,6 @@ struct rte_mbuf {
> > * To obtain a pointer to rearm_data use the rte_mbuf_rearm_data()
> > * accessor instead of directly referencing through the data_off
> field.
> > */
> > - RTE_MARKER64 rearm_data;
> > uint16_t data_off;
>
> One subtile change of removing the marker is that fields may not be
> aligned as before.
>
> #if RTE_IOVA_IN_MBUF
> /**
> * Physical address of segment buffer.
> * This field is undefined if the build is configured to use only
> * virtual address as IOVA (i.e. RTE_IOVA_IN_MBUF is 0).
> * Force alignment to 8-bytes, so as to ensure we have the exact
> * same mbuf cacheline0 layout for 32-bit and 64-bit. This makes
> * working on vector drivers easier.
> */
> rte_iova_t buf_iova __rte_aligned(sizeof(rte_iova_t));
> #else
> /**
> * Next segment of scattered packet.
> * This field is valid when physical address field is undefined.
> * Otherwise next pointer in the second cache line will be used.
> */
> struct rte_mbuf *next;
> #endif
>
> When building ! RTE_IOVA_IN_MBUF on a 32 bits arch, the next pointer
> is not force aligned to 64bits.
> Which has a cascade effect on data_off alignement.
>
> In file included from ../lib/mbuf/rte_mbuf_core.h:19,
> from ../lib/mbuf/rte_mbuf.h:42,
> from ../lib/mbuf/rte_mbuf_dyn.c:18:
> ../lib/mbuf/rte_mbuf_core.h:676:1: error: static assertion failed: "data_off"
> 676 | static_assert(!(offsetof(struct rte_mbuf, data_off) !=
> | ^~~~~~~~~~~~~
>
>
> I hope reviewers pay attention to the alignment changes when removing
> those markers.
> This is not trivial to catch in the CI.
Good catch, David.
I wonder about the reason for 64 bit aligning the rearm_data group of fields? Perhaps it's there for (64 bit arch) vector instruction purposes?
Regardless, it's an ABI break, so padding or an alignment attribute must be added to avoid ABI breakage. If there is no valid reason for the 64 bit alignment, it could be noted that the padding (or alignment attribute) is there for 32 bit arch ABI compatibility reasons only.
Please note that only RTE_MARKER64 is affected by this. The other marker types have arch bit-width (or smaller) alignment, i.e. RTE_MARKER is 8 byte aligned on 64 bit arch and 4 byte aligned on 32 bit arch.
And RTE_MARKER64 is only used in the rte_mbuf structure.
next prev parent reply other threads:[~2024-02-28 15:01 UTC|newest]
Thread overview: 171+ messages / expand[flat|nested] mbox.gz Atom feed top
2024-01-30 23:26 [PATCH] replace GCC marker extension with C11 anonymous unions Tyler Retzlaff
2024-01-30 23:26 ` [PATCH] mbuf: " Tyler Retzlaff
2024-01-31 9:18 ` Morten Brørup
2024-01-31 21:09 ` Tyler Retzlaff
2024-01-31 22:39 ` Morten Brørup
2024-01-31 13:49 ` Bruce Richardson
2024-01-31 20:45 ` Tyler Retzlaff
2024-01-31 22:55 ` Morten Brørup
2024-02-13 6:45 ` [PATCH v2] RFC: " Tyler Retzlaff
2024-02-13 6:45 ` [PATCH v2] mbuf: " Tyler Retzlaff
2024-02-13 16:58 ` Morten Brørup
2024-02-13 18:48 ` Tyler Retzlaff
2024-02-13 19:27 ` Morten Brørup
2024-02-13 20:00 ` Tyler Retzlaff
2024-02-13 8:57 ` [PATCH v2] RFC: " Bruce Richardson
2024-02-13 17:09 ` Morten Brørup
2024-02-13 23:33 ` [PATCH v3] RFC deprecate RTE_MARKER in struct rte_mbuf Tyler Retzlaff
2024-02-13 23:33 ` [PATCH v3] mbuf: deprecate GCC marker in rte mbuf struct Tyler Retzlaff
2024-02-14 10:46 ` Morten Brørup
2024-02-14 20:16 ` Tyler Retzlaff
2024-02-14 10:49 ` [PATCH v3] RFC deprecate RTE_MARKER in struct rte_mbuf Morten Brørup
2024-02-26 1:18 ` Stephen Hemminger
2024-02-15 6:21 ` [PATCH v4 00/18] stop using zero sized marker fields Tyler Retzlaff
2024-02-15 6:21 ` [PATCH v4 01/18] mbuf: deprecate GCC marker in rte mbuf struct Tyler Retzlaff
2024-02-18 2:28 ` fengchengwen
2024-02-18 12:39 ` Thomas Monjalon
2024-02-18 13:07 ` Morten Brørup
2024-02-18 15:22 ` Thomas Monjalon
2024-02-18 16:20 ` Morten Brørup
2024-02-20 17:24 ` Tyler Retzlaff
2024-02-20 17:20 ` Tyler Retzlaff
2024-02-20 17:53 ` Thomas Monjalon
2024-02-20 19:16 ` Thomas Monjalon
2024-02-20 19:37 ` Tyler Retzlaff
2024-02-15 6:21 ` [PATCH v4 02/18] mbuf: stop using zero sized marker fields Tyler Retzlaff
2024-02-18 2:38 ` fengchengwen
2024-02-15 6:21 ` [PATCH v4 03/18] net/i40e: " Tyler Retzlaff
2024-02-15 6:21 ` [PATCH v4 04/18] net/iavf: " Tyler Retzlaff
2024-02-15 6:21 ` [PATCH v4 05/18] net/ice: " Tyler Retzlaff
2024-02-15 6:21 ` [PATCH v4 06/18] net/ixgbe: " Tyler Retzlaff
2024-02-15 6:21 ` [PATCH v4 07/18] net/mlx5: " Tyler Retzlaff
2024-02-15 6:21 ` [PATCH v4 08/18] net/sfc: " Tyler Retzlaff
2024-02-15 6:21 ` [PATCH v4 09/18] net/bnxt: " Tyler Retzlaff
2024-02-15 6:21 ` [PATCH v4 10/18] net/enic: " Tyler Retzlaff
2024-02-15 6:21 ` [PATCH v4 11/18] net/fm10k: " Tyler Retzlaff
2024-02-15 6:21 ` [PATCH v4 12/18] net/hns3: " Tyler Retzlaff
2024-02-18 3:00 ` fengchengwen
2024-02-15 6:21 ` [PATCH v4 13/18] net/ionic: " Tyler Retzlaff
2024-02-15 6:21 ` [PATCH v4 14/18] net/thunderx: " Tyler Retzlaff
2024-02-15 6:21 ` [PATCH v4 15/18] net/virtio: " Tyler Retzlaff
2024-02-15 6:21 ` [PATCH v4 16/18] net/cnxk: " Tyler Retzlaff
2024-02-15 6:21 ` [PATCH v4 17/18] common/idpf: " Tyler Retzlaff
2024-02-15 6:21 ` [PATCH v4 18/18] examples/dma: " Tyler Retzlaff
2024-02-15 9:37 ` [PATCH v4 00/18] " Morten Brørup
2024-02-24 8:21 ` [PATCH v5 00/22] stop using RTE_MARKER extensions Tyler Retzlaff
2024-02-24 8:21 ` [PATCH v5 01/22] eal: provide macro to expand marker extensions Tyler Retzlaff
2024-02-24 10:51 ` Thomas Monjalon
2024-02-24 11:18 ` Thomas Monjalon
2024-02-24 8:21 ` [PATCH v5 02/22] mbuf: expand rte markers empty when building with MSVC Tyler Retzlaff
2024-02-24 8:21 ` [PATCH v5 03/22] security: " Tyler Retzlaff
2024-02-24 8:21 ` [PATCH v5 04/22] cryptodev: " Tyler Retzlaff
2024-02-24 8:21 ` [PATCH v5 05/22] mbuf: stop using mbuf cacheline marker fields Tyler Retzlaff
2024-02-24 10:58 ` Thomas Monjalon
2024-02-26 18:20 ` Stephen Hemminger
2024-02-24 8:21 ` [PATCH v5 06/22] mbuf: add mbuf descriptor accessors Tyler Retzlaff
2024-02-24 11:01 ` Thomas Monjalon
2024-02-24 8:21 ` [PATCH v5 07/22] common/idpf: use " Tyler Retzlaff
2024-02-24 8:21 ` [PATCH v5 08/22] net/bnxt: " Tyler Retzlaff
2024-02-24 8:21 ` [PATCH v5 09/22] net/cnxk: " Tyler Retzlaff
2024-02-24 8:21 ` [PATCH v5 10/22] net/enic: " Tyler Retzlaff
2024-02-24 8:21 ` [PATCH v5 11/22] net/fm10k: " Tyler Retzlaff
2024-02-24 8:21 ` [PATCH v5 12/22] net/hns3: " Tyler Retzlaff
2024-02-24 8:21 ` [PATCH v5 13/22] net/i40e: " Tyler Retzlaff
2024-02-24 8:21 ` [PATCH v5 14/22] net/iavf: " Tyler Retzlaff
2024-02-24 8:22 ` [PATCH v5 15/22] net/ice: " Tyler Retzlaff
2024-02-24 8:22 ` [PATCH v5 16/22] net/ionic: " Tyler Retzlaff
2024-02-24 8:22 ` [PATCH v5 17/22] net/ixgbe: " Tyler Retzlaff
2024-02-24 8:22 ` [PATCH v5 18/22] net/mlx5: " Tyler Retzlaff
2024-02-24 8:22 ` [PATCH v5 19/22] net/octeon_ep: " Tyler Retzlaff
2024-02-24 8:22 ` [PATCH v5 20/22] net/sfc: " Tyler Retzlaff
2024-02-24 8:22 ` [PATCH v5 21/22] net/thunderx: " Tyler Retzlaff
2024-02-24 8:22 ` [PATCH v5 22/22] net/virtio: " Tyler Retzlaff
2024-02-24 10:42 ` [PATCH v5 00/22] stop using RTE_MARKER extensions Morten Brørup
2024-02-24 11:13 ` Thomas Monjalon
2024-02-24 11:23 ` Morten Brørup
2024-02-27 5:41 ` [PATCH v6 00/23] stop and remove RTE_MARKER typedefs Tyler Retzlaff
2024-02-27 5:41 ` [PATCH v6 01/23] mbuf: add accessors for rearm and Rx descriptor fields Tyler Retzlaff
2024-02-27 9:10 ` Morten Brørup
2024-02-27 17:17 ` Tyler Retzlaff
2024-02-28 8:28 ` Morten Brørup
2024-02-28 16:20 ` Tyler Retzlaff
2024-02-27 5:41 ` [PATCH v6 02/23] mbuf: consolidate driver asserts for mbuf struct Tyler Retzlaff
2024-02-27 10:02 ` Konstantin Ananyev
2024-03-14 16:51 ` Tyler Retzlaff
2024-02-27 5:41 ` [PATCH v6 03/23] common/idpf: use mbuf descriptor accessors Tyler Retzlaff
2024-02-27 5:41 ` [PATCH v6 04/23] net/bnxt: " Tyler Retzlaff
2024-02-27 5:41 ` [PATCH v6 05/23] net/cnxk: " Tyler Retzlaff
2024-02-27 5:41 ` [PATCH v6 06/23] net/enic: " Tyler Retzlaff
2024-02-27 5:41 ` [PATCH v6 07/23] net/fm10k: " Tyler Retzlaff
2024-02-27 5:41 ` [PATCH v6 08/23] net/hns3: " Tyler Retzlaff
2024-02-27 5:41 ` [PATCH v6 09/23] net/i40e: " Tyler Retzlaff
2024-02-27 5:41 ` [PATCH v6 10/23] net/iavf: " Tyler Retzlaff
2024-02-27 5:41 ` [PATCH v6 11/23] net/ice: " Tyler Retzlaff
2024-02-27 5:41 ` [PATCH v6 12/23] net/ionic: " Tyler Retzlaff
2024-02-27 5:41 ` [PATCH v6 13/23] net/ixgbe: " Tyler Retzlaff
2024-02-27 5:41 ` [PATCH v6 14/23] net/mlx5: " Tyler Retzlaff
2024-02-27 5:41 ` [PATCH v6 15/23] net/octeon_ep: " Tyler Retzlaff
2024-02-27 5:41 ` [PATCH v6 16/23] net/sfc: " Tyler Retzlaff
2024-02-27 5:41 ` [PATCH v6 17/23] net/thunderx: " Tyler Retzlaff
2024-02-27 5:41 ` [PATCH v6 18/23] net/virtio: " Tyler Retzlaff
2024-02-27 5:41 ` [PATCH v6 19/23] examples/dma: use mbuf descriptor accessor Tyler Retzlaff
2024-02-27 5:41 ` [PATCH v6 20/23] mbuf: remove and stop using rte marker fields Tyler Retzlaff
2024-02-27 9:15 ` Morten Brørup
2024-02-27 10:03 ` Konstantin Ananyev
2024-02-27 15:18 ` David Marchand
2024-02-27 16:04 ` Morten Brørup
2024-02-27 17:23 ` Tyler Retzlaff
2024-02-28 10:42 ` David Marchand
2024-02-28 14:03 ` Dodji Seketeli
2024-02-28 14:43 ` David Marchand
2024-02-29 14:50 ` Dodji Seketeli
2024-02-28 14:18 ` David Marchand
2024-02-28 15:01 ` Morten Brørup [this message]
2024-02-28 15:33 ` David Marchand
2024-02-28 17:20 ` Tyler Retzlaff
2024-02-27 5:41 ` [PATCH v6 21/23] security: remove " Tyler Retzlaff
2024-02-27 5:41 ` [PATCH v6 22/23] cryptodev: " Tyler Retzlaff
2024-02-27 5:41 ` [PATCH v6 23/23] devtools: forbid new use of rte marker typedefs Tyler Retzlaff
2024-03-20 22:01 ` [PATCH v7 0/4] remove use of RTE_MARKER fields in libraries Tyler Retzlaff
2024-03-20 22:01 ` [PATCH v7 1/4] net/i40e: use inline prefetch function Tyler Retzlaff
2024-03-26 10:16 ` Morten Brørup
2024-03-27 18:14 ` Tyler Retzlaff
2024-03-27 19:45 ` Morten Brørup
2024-03-20 22:01 ` [PATCH v7 2/4] mbuf: remove rte marker fields Tyler Retzlaff
2024-03-21 10:32 ` Bruce Richardson
2024-03-21 15:31 ` Tyler Retzlaff
2024-03-21 16:19 ` Morten Brørup
2024-03-26 11:12 ` Morten Brørup
2024-03-20 22:01 ` [PATCH v7 3/4] security: " Tyler Retzlaff
2024-03-26 10:28 ` Morten Brørup
2024-03-27 19:59 ` Tyler Retzlaff
2024-03-20 22:01 ` [PATCH v7 4/4] cryptodev: " Tyler Retzlaff
2024-03-26 10:31 ` Morten Brørup
2024-03-27 19:56 ` [PATCH v8 0/4] remove use of RTE_MARKER fields in libraries Tyler Retzlaff
2024-03-27 19:56 ` [PATCH v8 1/4] net/i40e: use inline prefetch function Tyler Retzlaff
2024-03-27 19:56 ` [PATCH v8 2/4] mbuf: remove rte marker fields Tyler Retzlaff
2024-03-27 19:56 ` [PATCH v8 3/4] security: " Tyler Retzlaff
2024-03-27 19:56 ` [PATCH v8 4/4] cryptodev: " Tyler Retzlaff
2024-04-02 20:08 ` [PATCH v9 0/4] remove use of RTE_MARKER fields in libraries Tyler Retzlaff
2024-04-02 20:08 ` [PATCH v9 1/4] net/i40e: use inline prefetch function Tyler Retzlaff
2024-04-02 20:08 ` [PATCH v9 2/4] mbuf: remove rte marker fields Tyler Retzlaff
2024-04-02 20:45 ` Stephen Hemminger
2024-04-02 20:51 ` Tyler Retzlaff
2024-04-02 20:08 ` [PATCH v9 3/4] security: " Tyler Retzlaff
2024-04-02 20:08 ` [PATCH v9 4/4] cryptodev: " Tyler Retzlaff
2024-04-03 17:53 ` [PATCH v10 0/4] remove use of RTE_MARKER fields in libraries Tyler Retzlaff
2024-04-03 17:53 ` [PATCH v10 1/4] net/i40e: use inline prefetch function Tyler Retzlaff
2024-04-03 21:49 ` Stephen Hemminger
2024-04-03 17:53 ` [PATCH v10 2/4] mbuf: remove rte marker fields Tyler Retzlaff
2024-04-03 19:32 ` Morten Brørup
2024-04-03 22:45 ` Tyler Retzlaff
2024-04-03 21:49 ` Stephen Hemminger
2024-04-03 17:53 ` [PATCH v10 3/4] security: " Tyler Retzlaff
2024-04-03 21:50 ` Stephen Hemminger
2024-04-03 17:53 ` [PATCH v10 4/4] cryptodev: " Tyler Retzlaff
2024-04-03 21:50 ` Stephen Hemminger
2024-04-04 17:51 ` [PATCH v11 0/4] remove use of RTE_MARKER fields in libraries Tyler Retzlaff
2024-04-04 17:51 ` [PATCH v11 1/4] net/i40e: use inline prefetch function Tyler Retzlaff
2024-04-04 17:51 ` [PATCH v11 2/4] mbuf: remove rte marker fields Tyler Retzlaff
2024-04-04 17:51 ` [PATCH v11 3/4] security: " Tyler Retzlaff
2024-04-04 17:51 ` [PATCH v11 4/4] cryptodev: " Tyler Retzlaff
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=98CBD80474FA8B44BF855DF32C47DC35E9F283@smartserver.smartshare.dk \
--to=mb@smartsharesystems.com \
--cc=Yuying.Zhang@intel.com \
--cc=ajit.khaparde@broadcom.com \
--cc=andrew.boyer@amd.com \
--cc=andrew.rybchenko@oktetlabs.ru \
--cc=bruce.richardson@intel.com \
--cc=chenbox@nvidia.com \
--cc=david.marchand@redhat.com \
--cc=dev@dpdk.org \
--cc=drc@linux.vnet.ibm.com \
--cc=dsosnowski@nvidia.com \
--cc=fengchengwen@huawei.com \
--cc=haijie1@huawei.com \
--cc=hyonkim@cisco.com \
--cc=jerinj@marvell.com \
--cc=jingjing.wu@intel.com \
--cc=johndale@cisco.com \
--cc=kevin.laatz@intel.com \
--cc=kirankumark@marvell.com \
--cc=konstantin.v.ananyev@yandex.ru \
--cc=matan@nvidia.com \
--cc=maxime.coquelin@redhat.com \
--cc=mczekaj@marvell.com \
--cc=ndabilpuram@marvell.com \
--cc=orika@nvidia.com \
--cc=roretzla@linux.microsoft.com \
--cc=ruifeng.wang@arm.com \
--cc=skori@marvell.com \
--cc=skoteshwar@marvell.com \
--cc=somnath.kotur@broadcom.com \
--cc=suanmingm@nvidia.com \
--cc=thomas@monjalon.net \
--cc=viacheslavo@nvidia.com \
--cc=yisen.zhuang@huawei.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).