From: "Morten Brørup" <mb@smartsharesystems.com>
To: "Mattias Rönnblom" <hofors@lysator.liu.se>,
"Stephen Hemminger" <stephen@networkplumber.org>,
dev@dpdk.org
Subject: RE: [PATCH v2 00/71] replace use of fixed size rte_mempcy
Date: Sat, 2 Mar 2024 14:05:45 +0100 [thread overview]
Message-ID: <98CBD80474FA8B44BF855DF32C47DC35E9F294@smartserver.smartshare.dk> (raw)
In-Reply-To: <bca732d8-28e1-48c5-abc2-f0aa9f3748ad@lysator.liu.se>
> From: Mattias Rönnblom [mailto:hofors@lysator.liu.se]
> Sent: Saturday, 2 March 2024 13.02
>
> On 2024-03-02 12:14, Mattias Rönnblom wrote:
> > On 2024-03-01 18:14, Stephen Hemminger wrote:
> >> The DPDK has a lot of "cargo cult" usage of rte_memcpy.
> >> This patch set replaces cases where rte_memcpy is used with a fixed
> >> size constant size.
> >>
> >> Typical example is:
> >> rte_memcpy(mac_addrs, mac.addr_bytes, RTE_ETHER_ADDR_LEN);
> >> which can be replaced with:
> >> memcpy(mac_addrs, mac.addr_bytes, RTE_ETHER_ADDR_LEN);
> >>
> >> This has two benefits. Gcc (and clang) are smart enough that for
> >> all small fixed size values, they just generate the necessary
> >> instructions
> >> to do it inline. It also means that fortify, Coverity, and ASAN
> >> analyzers can check these memcpy's.
> >>
> >
> > Instead of smearing out the knowledge of when to use rte_memcpy(), and
> > when to use memcpy() across the code base, wouldn't it be better to
> > *always* call rte_memcpy() in the fast path, and leave the policy
> > decision to the rte_memcpy() implementation?
> >
> > In rte_memcpy(), add:
> > if (__builtin_constant_p(n) && n < RTE_LIBC_MEMCPY_SIZE_THRESHOLD)
> > memcpy(/../);
> > ..or something to that effect.
> >
> > Could you have a #ifdef for dumb static analysis tools? To make it
> look
> > like you are always using memcpy()?
> >
> >> So faster, better, safer.
> >>
> >
> > What is "faster" based on?
> >
>
> I ran some DSW benchmarks, and if you add
>
> diff --git a/lib/eal/x86/include/rte_memcpy.h
> b/lib/eal/x86/include/rte_memcpy.h
> index 72a92290e0..64cd82d78d 100644
> --- a/lib/eal/x86/include/rte_memcpy.h
> +++ b/lib/eal/x86/include/rte_memcpy.h
> @@ -862,6 +862,11 @@ rte_memcpy_aligned(void *dst, const void *src,
> size_t n)
> static __rte_always_inline void *
> rte_memcpy(void *dst, const void *src, size_t n)
> {
> + if (__builtin_constant_p(n) && n <= 32) {
> + memcpy(dst, src, n);
> + return dst;
> + }
> +
> if (!(((uintptr_t)dst | (uintptr_t)src) & ALIGNMENT_MASK))
> return rte_memcpy_aligned(dst, src, n);
> else
>
> ...the overhead increases from roughly 48 core clock cycles/event to 59
> cc/event. The same for "n < 128". (I'm not sure what counts as "small"
> here.)
Thank you for digging deep into this, Mattias.
Your performance data are very interesting.
>
> So something rte_memcpy() does for small and constant memory copies does
> make things go *significantly* faster, at least in certain cases.
Interesting.
Perhaps something with aligned copies...
The performance benefits of well known alignment was something I was looking into when working on non-temporal memcpy functions, because non-temporal load/store has some alignment requirements. (NB: NT memcpy development is hold, until I get more time to work on it again.)
Passing alignment information as a flag to an extended memcpy, to be used by __builtin_constant_p(n), could speed up copying when alignment is known by the developer, but impossible for the compiler to determine at build time.
The rte_memcpy() checks for one specific alignment criteria at runtime. I suppose the branch predictor makes it execute nearly as fast as if determined at build time, but it still consumes a lot more instruction memory.
Perhaps something else...?
>
> (Linux, GCC 11.4, Intel Gracemont.)
>
> > My experience with replacing rte_memcpy() with memcpy() (or vice
> versa)
> > is mixed.
> >
> > I've also tried just dropping the DPDK-custom memcpy() implementation
> > altogether, and that caused a performance drop (in a particular app,
> on
> > a particular compiler and CPU).
I guess the compilers are just not where we want them to be yet.
I don't mind generally replacing rte_memcpy() with memcpy() in the control plane.
But we should use whatever is more efficient in the data plane.
We must also keep in mind that DPDK supports old distros with old compilers. We should not remove a superfluous hand crafted optimization if a supported old compiler hasn't caught up with it yet, i.e. if it isn't superfluous on some of the old compilers supported by DPDK.
next prev parent reply other threads:[~2024-03-02 13:05 UTC|newest]
Thread overview: 205+ messages / expand[flat|nested] mbox.gz Atom feed top
2024-02-29 22:57 [PATCH 00/71] replace use of fixed size rte_memcpy Stephen Hemminger
2024-02-29 22:57 ` [PATCH 01/71] cocci/rte_memcpy: add script to eliminate " Stephen Hemminger
2024-03-01 8:07 ` Morten Brørup
2024-02-29 22:57 ` [PATCH 02/71] eal: replace use of " Stephen Hemminger
2024-02-29 22:57 ` [PATCH 03/71] ethdev: " Stephen Hemminger
2024-02-29 22:57 ` [PATCH 04/71] eventdev: " Stephen Hemminger
2024-02-29 22:57 ` [PATCH 05/71] cryptodev: " Stephen Hemminger
2024-02-29 22:57 ` [PATCH 06/71] ip_frag: " Stephen Hemminger
2024-02-29 22:57 ` [PATCH 07/71] net: " Stephen Hemminger
2024-02-29 22:57 ` [PATCH 08/71] lpm: " Stephen Hemminger
2024-02-29 22:57 ` [PATCH 09/71] node: " Stephen Hemminger
2024-02-29 22:57 ` [PATCH 10/71] pdcp: " Stephen Hemminger
2024-02-29 22:57 ` [PATCH 11/71] pipeline: " Stephen Hemminger
2024-02-29 22:57 ` [PATCH 12/71] rib: " Stephen Hemminger
2024-02-29 22:58 ` [PATCH 13/71] security: " Stephen Hemminger
2024-02-29 22:58 ` [PATCH 14/71] net/mlx5: " Stephen Hemminger
2024-02-29 22:58 ` [PATCH 15/71] net/nfp: " Stephen Hemminger
2024-02-29 22:58 ` [PATCH 16/71] net/ngbe: " Stephen Hemminger
2024-02-29 22:58 ` [PATCH 17/71] net/null: " Stephen Hemminger
2024-02-29 22:58 ` [PATCH 18/71] net/pcap: " Stephen Hemminger
2024-02-29 22:58 ` [PATCH 19/71] net/sfc: " Stephen Hemminger
2024-02-29 22:58 ` [PATCH 20/71] net/tap: " Stephen Hemminger
2024-02-29 22:58 ` [PATCH 21/71] net/txgbe: " Stephen Hemminger
2024-02-29 22:58 ` [PATCH 22/71] raw/ifpga: " Stephen Hemminger
2024-02-29 22:58 ` [PATCH 23/71] raw/skeleton: " Stephen Hemminger
2024-02-29 22:58 ` [PATCH 24/71] net/hns3: " Stephen Hemminger
2024-02-29 22:58 ` [PATCH 25/71] net/i40e: " Stephen Hemminger
2024-02-29 22:58 ` [PATCH 26/71] net/iavf: " Stephen Hemminger
2024-02-29 22:58 ` [PATCH 27/71] net/ice: " Stephen Hemminger
2024-02-29 22:58 ` [PATCH 28/71] net/idpf: " Stephen Hemminger
2024-02-29 22:58 ` [PATCH 29/71] net/ipn3ke: " Stephen Hemminger
2024-02-29 22:58 ` [PATCH 30/71] net/ixgbe: " Stephen Hemminger
2024-02-29 22:58 ` [PATCH 31/71] net/memif: " Stephen Hemminger
2024-02-29 22:58 ` [PATCH 32/71] net/qede: " Stephen Hemminger
2024-02-29 22:58 ` [PATCH 33/71] baseband/acc: " Stephen Hemminger
2024-02-29 22:58 ` [PATCH 34/71] baseband/la12xx: " Stephen Hemminger
2024-02-29 22:58 ` [PATCH 35/71] common/idpf: " Stephen Hemminger
2024-02-29 22:58 ` [PATCH 36/71] common/qat: " Stephen Hemminger
2024-02-29 22:58 ` [PATCH 37/71] compress/qat: " Stephen Hemminger
2024-02-29 22:58 ` [PATCH 38/71] crypto/ccp: " Stephen Hemminger
2024-02-29 22:58 ` [PATCH 39/71] crypto/cnxk: " Stephen Hemminger
2024-02-29 22:58 ` [PATCH 40/71] crypto/dpaa_sec: " Stephen Hemminger
2024-02-29 22:58 ` [PATCH 41/71] crypto/ipsec_mb: " Stephen Hemminger
2024-02-29 22:58 ` [PATCH 42/71] crypto/qat: " Stephen Hemminger
2024-02-29 22:58 ` [PATCH 43/71] crypto/scheduler: " Stephen Hemminger
2024-02-29 22:58 ` [PATCH 44/71] event/cnxk: " Stephen Hemminger
2024-02-29 22:58 ` [PATCH 45/71] event/dlb2: " Stephen Hemminger
2024-02-29 22:58 ` [PATCH 46/71] event/dpaa2: " Stephen Hemminger
2024-02-29 22:58 ` [PATCH 47/71] event/octeontx: " Stephen Hemminger
2024-02-29 22:58 ` [PATCH 48/71] mempool/dpaa: " Stephen Hemminger
2024-02-29 22:58 ` [PATCH 49/71] mempool/dpaa2: " Stephen Hemminger
2024-02-29 22:58 ` [PATCH 50/71] ml/cnxk: " Stephen Hemminger
2024-02-29 22:58 ` [PATCH 51/71] net/af_xdp: " Stephen Hemminger
2024-02-29 22:58 ` [PATCH 52/71] net/avp: " Stephen Hemminger
2024-02-29 22:58 ` [PATCH 53/71] net/axgbe: " Stephen Hemminger
2024-02-29 22:58 ` [PATCH 54/71] net/bnx2x: " Stephen Hemminger
2024-02-29 22:58 ` [PATCH 55/71] net/bnxt: " Stephen Hemminger
2024-03-01 2:54 ` Somnath Kotur
2024-02-29 22:58 ` [PATCH 56/71] net/bonding: " Stephen Hemminger
2024-02-29 22:58 ` [PATCH 57/71] net/cnxk: " Stephen Hemminger
2024-02-29 22:58 ` [PATCH 58/71] net/cpfl: " Stephen Hemminger
2024-02-29 22:58 ` [PATCH 59/71] net/cxgbe: " Stephen Hemminger
2024-02-29 22:58 ` [PATCH 60/71] net/dpaa2: " Stephen Hemminger
2024-02-29 22:58 ` [PATCH 61/71] net/e1000: " Stephen Hemminger
2024-02-29 22:58 ` [PATCH 62/71] net/enic: " Stephen Hemminger
2024-02-29 22:58 ` [PATCH 63/71] net/failsafe: " Stephen Hemminger
2024-02-29 22:58 ` [PATCH 64/71] net/gve/base: " Stephen Hemminger
2024-02-29 22:58 ` [PATCH 65/71] net/hinic: " Stephen Hemminger
2024-02-29 22:58 ` [PATCH 66/71] net/mvpp2: " Stephen Hemminger
2024-02-29 22:58 ` [PATCH 67/71] app/test-pmd: " Stephen Hemminger
2024-02-29 22:58 ` [PATCH 68/71] app/graph: " Stephen Hemminger
2024-02-29 22:58 ` [PATCH 69/71] app/test-eventdev: " Stephen Hemminger
2024-02-29 22:58 ` [PATCH 70/71] app/test: " Stephen Hemminger
2024-02-29 22:58 ` [PATCH 71/71] examples: " Stephen Hemminger
2024-03-01 8:38 ` [PATCH 00/71] " Bruce Richardson
2024-03-01 11:06 ` Hemant Agrawal
2024-03-01 12:55 ` Konstantin Ananyev
2024-03-01 18:55 ` Ajit Khaparde
2024-03-01 13:53 ` David Marchand
2024-03-01 17:14 ` [PATCH v2 00/71] replace use of fixed size rte_mempcy Stephen Hemminger
2024-03-01 17:14 ` [PATCH v2 01/71] cocci/rte_memcpy: add script to eliminate fixed size rte_memcpy Stephen Hemminger
2024-03-02 11:19 ` Mattias Rönnblom
2024-03-02 17:02 ` Stephen Hemminger
2024-03-02 17:39 ` Morten Brørup
2024-03-01 17:14 ` [PATCH v2 02/71] eal: replace use of " Stephen Hemminger
2024-03-01 17:14 ` [PATCH v2 03/71] ethdev: " Stephen Hemminger
2024-03-01 17:15 ` [PATCH v2 04/71] eventdev: " Stephen Hemminger
2024-03-01 17:15 ` [PATCH v2 05/71] cryptodev: " Stephen Hemminger
2024-03-01 17:15 ` [PATCH v2 06/71] ip_frag: " Stephen Hemminger
2024-03-01 17:15 ` [PATCH v2 07/71] net: " Stephen Hemminger
2024-03-01 17:15 ` [PATCH v2 08/71] lpm: " Stephen Hemminger
2024-03-01 17:15 ` [PATCH v2 09/71] node: " Stephen Hemminger
2024-03-01 17:15 ` [PATCH v2 10/71] pdcp: " Stephen Hemminger
2024-03-01 17:15 ` [PATCH v2 11/71] pipeline: " Stephen Hemminger
2024-03-01 17:15 ` [PATCH v2 12/71] rib: " Stephen Hemminger
2024-03-01 17:15 ` [PATCH v2 13/71] security: " Stephen Hemminger
2024-03-01 17:15 ` [PATCH v2 14/71] net/mlx5: " Stephen Hemminger
2024-03-08 18:16 ` Dariusz Sosnowski
2024-03-01 17:15 ` [PATCH v2 15/71] net/nfp: " Stephen Hemminger
2024-03-01 17:15 ` [PATCH v2 16/71] net/ngbe: " Stephen Hemminger
2024-03-01 17:15 ` [PATCH v2 17/71] net/null: " Stephen Hemminger
2024-03-01 17:15 ` [PATCH v2 18/71] net/pcap: " Stephen Hemminger
2024-03-01 17:15 ` [PATCH v2 19/71] net/sfc: " Stephen Hemminger
2024-03-01 17:15 ` [PATCH v2 20/71] net/tap: " Stephen Hemminger
2024-03-01 17:15 ` [PATCH v2 21/71] net/txgbe: " Stephen Hemminger
2024-03-01 17:15 ` [PATCH v2 22/71] raw/ifpga: " Stephen Hemminger
2024-03-05 9:13 ` Xu, Rosen
2024-03-01 17:15 ` [PATCH v2 23/71] raw/skeleton: " Stephen Hemminger
2024-03-01 17:15 ` [PATCH v2 24/71] net/hns3: " Stephen Hemminger
2024-03-01 17:15 ` [PATCH v2 25/71] net/i40e: " Stephen Hemminger
2024-03-01 17:15 ` [PATCH v2 26/71] net/iavf: " Stephen Hemminger
2024-03-01 17:15 ` [PATCH v2 27/71] net/ice: " Stephen Hemminger
2024-03-01 17:15 ` [PATCH v2 28/71] net/idpf: " Stephen Hemminger
2024-03-01 17:15 ` [PATCH v2 29/71] net/ipn3ke: " Stephen Hemminger
2024-03-01 17:15 ` [PATCH v2 30/71] net/ixgbe: " Stephen Hemminger
2024-03-01 17:15 ` [PATCH v2 31/71] net/memif: " Stephen Hemminger
2024-03-01 17:15 ` [PATCH v2 32/71] net/qede: " Stephen Hemminger
2024-03-01 17:15 ` [PATCH v2 33/71] baseband/acc: " Stephen Hemminger
2024-03-02 1:01 ` Chautru, Nicolas
2024-03-02 3:23 ` Stephen Hemminger
2024-03-04 18:39 ` Chautru, Nicolas
2024-03-01 17:15 ` [PATCH v2 34/71] baseband/la12xx: " Stephen Hemminger
2024-03-01 17:15 ` [PATCH v2 35/71] common/idpf: " Stephen Hemminger
2024-03-01 17:15 ` [PATCH v2 36/71] common/qat: " Stephen Hemminger
2024-03-01 17:15 ` [PATCH v2 37/71] compress/qat: " Stephen Hemminger
2024-03-01 17:15 ` [PATCH v2 38/71] crypto/ccp: " Stephen Hemminger
2024-03-01 17:15 ` [PATCH v2 39/71] crypto/cnxk: " Stephen Hemminger
2024-03-01 17:15 ` [PATCH v2 40/71] crypto/dpaa_sec: " Stephen Hemminger
2024-03-01 17:15 ` [PATCH v2 41/71] crypto/ipsec_mb: " Stephen Hemminger
2024-03-01 17:15 ` [PATCH v2 42/71] crypto/qat: " Stephen Hemminger
2024-03-01 17:15 ` [PATCH v2 43/71] crypto/scheduler: " Stephen Hemminger
2024-03-01 17:15 ` [PATCH v2 44/71] event/cnxk: " Stephen Hemminger
2024-03-01 17:15 ` [PATCH v2 45/71] event/dlb2: " Stephen Hemminger
2024-03-05 17:07 ` Sevincer, Abdullah
2024-03-05 18:30 ` Stephen Hemminger
2024-03-06 15:24 ` Sevincer, Abdullah
2024-03-01 17:15 ` [PATCH v2 46/71] event/dpaa2: " Stephen Hemminger
2024-03-01 17:15 ` [PATCH v2 47/71] event/octeontx: " Stephen Hemminger
2024-03-01 17:15 ` [PATCH v2 48/71] mempool/dpaa: " Stephen Hemminger
2024-03-01 17:15 ` [PATCH v2 49/71] mempool/dpaa2: " Stephen Hemminger
2024-03-01 17:15 ` [PATCH v2 50/71] ml/cnxk: " Stephen Hemminger
2024-03-01 17:15 ` [PATCH v2 51/71] net/af_xdp: " Stephen Hemminger
2024-03-01 17:15 ` [PATCH v2 52/71] net/avp: " Stephen Hemminger
2024-03-01 17:15 ` [PATCH v2 53/71] net/axgbe: " Stephen Hemminger
2024-03-01 17:15 ` [PATCH v2 54/71] net/bnx2x: " Stephen Hemminger
2024-03-01 17:15 ` [PATCH v2 55/71] net/bnxt: " Stephen Hemminger
2024-03-01 17:15 ` [PATCH v2 56/71] net/bonding: " Stephen Hemminger
2024-03-01 17:15 ` [PATCH v2 57/71] net/cnxk: " Stephen Hemminger
2024-03-01 17:15 ` [PATCH v2 58/71] net/cpfl: " Stephen Hemminger
2024-03-01 17:15 ` [PATCH v2 59/71] net/cxgbe: " Stephen Hemminger
2024-03-01 17:15 ` [PATCH v2 60/71] net/dpaa2: " Stephen Hemminger
2024-03-01 17:15 ` [PATCH v2 61/71] net/e1000: " Stephen Hemminger
2024-03-01 17:15 ` [PATCH v2 62/71] net/enic: " Stephen Hemminger
2024-03-01 17:15 ` [PATCH v2 63/71] net/failsafe: " Stephen Hemminger
2024-03-01 17:16 ` [PATCH v2 64/71] net/gve/base: " Stephen Hemminger
2024-03-01 17:16 ` [PATCH v2 65/71] net/hinic: " Stephen Hemminger
2024-03-01 17:16 ` [PATCH v2 66/71] net/mvpp2: " Stephen Hemminger
2024-03-01 17:16 ` [PATCH v2 67/71] app/test-pmd: " Stephen Hemminger
2024-03-01 17:16 ` [PATCH v2 68/71] app/graph: " Stephen Hemminger
2024-03-01 17:16 ` [PATCH v2 69/71] app/test-eventdev: " Stephen Hemminger
2024-03-01 17:16 ` [PATCH v2 70/71] app/test: " Stephen Hemminger
2024-03-01 17:16 ` [PATCH v2 71/71] examples: " Stephen Hemminger
2024-03-01 17:50 ` [PATCH v2 00/71] replace use of fixed size rte_mempcy Tyler Retzlaff
2024-03-02 11:14 ` Mattias Rönnblom
2024-03-02 12:01 ` Mattias Rönnblom
2024-03-02 13:05 ` Morten Brørup [this message]
2024-03-02 16:37 ` Stephen Hemminger
2024-03-02 17:32 ` Morten Brørup
2024-03-02 16:35 ` Stephen Hemminger
2024-04-02 21:50 ` [PATCH v3 00/30] replace rte_memcpy of fixed size with memcpy Stephen Hemminger
2024-04-02 21:50 ` [PATCH v3 01/30] cocci/rte_memcpy: add script to eliminate fixed size rte_memcpy Stephen Hemminger
2024-04-02 21:50 ` [PATCH v3 02/30] eal: replace use of " Stephen Hemminger
2024-04-02 21:50 ` [PATCH v3 03/30] ethdev: " Stephen Hemminger
2024-04-02 21:50 ` [PATCH v3 04/30] eventdev: " Stephen Hemminger
2024-04-02 21:50 ` [PATCH v3 05/30] cryptodev: " Stephen Hemminger
2024-04-02 21:50 ` [PATCH v3 06/30] ip_frag: " Stephen Hemminger
2024-04-02 21:50 ` [PATCH v3 07/30] net: " Stephen Hemminger
2024-04-02 21:50 ` [PATCH v3 08/30] lpm: " Stephen Hemminger
2024-04-02 21:50 ` [PATCH v3 09/30] node: " Stephen Hemminger
2024-04-02 21:50 ` [PATCH v3 10/30] pdcp: " Stephen Hemminger
2024-04-02 21:50 ` [PATCH v3 11/30] pipeline: " Stephen Hemminger
2024-04-02 21:50 ` [PATCH v3 12/30] rib: " Stephen Hemminger
2024-04-02 21:50 ` [PATCH v3 13/30] security: " Stephen Hemminger
2024-04-02 21:50 ` [PATCH v3 14/30] bus: remove unneeded rte_memcpy.h include Stephen Hemminger
2024-04-02 21:50 ` [PATCH v3 15/30] net: replace use of fixed size rte_memcpy Stephen Hemminger
2024-04-04 11:07 ` Slava Ovsiienko
2024-04-02 21:50 ` [PATCH v3 16/30] raw: " Stephen Hemminger
2024-04-02 21:50 ` [PATCH v3 17/30] baseband: " Stephen Hemminger
2024-04-05 1:22 ` Chautru, Nicolas
2024-04-05 15:02 ` Stephen Hemminger
2024-04-02 21:50 ` [PATCH v3 18/30] common: " Stephen Hemminger
2024-04-02 21:50 ` [PATCH v3 19/30] crypto: " Stephen Hemminger
2024-04-02 21:50 ` [PATCH v3 20/30] " Stephen Hemminger
2024-04-02 21:50 ` [PATCH v3 21/30] event: " Stephen Hemminger
2024-04-02 21:50 ` [PATCH v3 22/30] mempool: " Stephen Hemminger
2024-04-02 21:50 ` [PATCH v3 23/30] ml/cnxk: " Stephen Hemminger
2024-04-02 21:50 ` [PATCH v3 24/30] app/test-pmd: " Stephen Hemminger
2024-04-02 21:50 ` [PATCH v3 25/30] app/graph: " Stephen Hemminger
2024-04-02 21:50 ` [PATCH v3 26/30] app/test-eventdev: " Stephen Hemminger
2024-04-02 21:50 ` [PATCH v3 27/30] app/test: " Stephen Hemminger
2024-04-02 21:50 ` [PATCH v3 28/30] app/test-pipeline: remove unused rte_memcpy.h include Stephen Hemminger
2024-04-02 21:50 ` [PATCH v3 29/30] app/test-bbdev: remove unnecessary include of rte_memcpy.h Stephen Hemminger
2024-04-02 21:50 ` [PATCH v3 30/30] examples: replace use of fixed size rte_memcpy Stephen Hemminger
2024-04-04 7:50 ` [PATCH v3 00/30] replace rte_memcpy of fixed size with memcpy Konstantin Ananyev
2024-04-04 15:52 ` Tyler Retzlaff
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=98CBD80474FA8B44BF855DF32C47DC35E9F294@smartserver.smartshare.dk \
--to=mb@smartsharesystems.com \
--cc=dev@dpdk.org \
--cc=hofors@lysator.liu.se \
--cc=stephen@networkplumber.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).