From: "Mattias Rönnblom" <hofors@lysator.liu.se>
To: "Morten Brørup" <mb@smartsharesystems.com>,
"Stephen Hemminger" <stephen@networkplumber.org>
Cc: "Bruce Richardson" <bruce.richardson@intel.com>,
"Ferruh Yigit" <ferruh.yigit@amd.com>,
"John W. Linville" <linville@tuxdriver.com>,
"Thomas Monjalon" <thomas@monjalon.net>,
dev@dpdk.org, "Mattias Rönnblom" <mattias.ronnblom@ericsson.com>
Subject: Re: [RFC v3] net/af_packet: make stats reset reliable
Date: Sun, 26 May 2024 09:10:12 +0200 [thread overview]
Message-ID: <1cc502d9-2afd-47fb-a340-bb3d9ad0d573@lysator.liu.se> (raw)
In-Reply-To: <98CBD80474FA8B44BF855DF32C47DC35E9F434@smartserver.smartshare.dk>
On 2024-05-10 11:14, Morten Brørup wrote:
>> From: Stephen Hemminger [mailto:stephen@networkplumber.org]
>> Sent: Friday, 10 May 2024 06.56
>>
>> On Thu, 9 May 2024 16:19:08 +0200
>> Morten Brørup <mb@smartsharesystems.com> wrote:
>>
>>>> From: Morten Brørup [mailto:mb@smartsharesystems.com]
>>>> Sent: Thursday, 9 May 2024 13.37
>>>>
>>>>> From: Bruce Richardson [mailto:bruce.richardson@intel.com]
>>>>> Sent: Thursday, 9 May 2024 11.30
>>>>>
>>>>> On Thu, May 09, 2024 at 09:43:16AM +0200, Morten Brørup wrote:
>>>>>>> From: Stephen Hemminger [mailto:stephen@networkplumber.org]
>>>>>>> Sent: Wednesday, 8 May 2024 22.54
>>>>>>>
>>>>>>> On Wed, 8 May 2024 20:48:06 +0100
>>>>>>> Ferruh Yigit <ferruh.yigit@amd.com> wrote:
>>>>>>>
>>>>>>>>>
>>>>>>>>> The idea of load tearing is crazy talk of integral types.
>> It
>>>>> would
>>>>>>> break so many things.
>>>>>>>>> It is the kind of stupid compiler thing that would send
>> Linus
>>>> on
>>>>> a
>>>>>>> rant and get
>>>>>>>>> the GCC compiler writers in trouble.
>>>>>>>>>
>>>>>>>>> The DPDK has always favored performance over strict safety
>>>> guard
>>>>>>> rails everywhere.
>>>>>>>>> Switching to making every statistic an atomic operation is
>> not
>>>>> in
>>>>>>> the spirit of
>>>>>>>>> what is required. There is no strict guarantee necessary
>> here.
>>>>>>>>>
>>>>>>>>
>>>>>>>> I kind of agree with Stephen.
>>>>>>>>
>>>>>>>> Thanks Mattias, Morten & Stephen, it was informative
>> discussion.
>>>>> But
>>>>>>> for
>>>>>>>> *SW drivers* stats update and reset is not core
>> functionality
>>>> and
>>>>> I
>>>>>>>> think we can be OK to get hit on corner cases, instead of
>>>>>>>> over-engineering or making code more complex.
>>>>>>>
>>>>>>>
>>>>>>> I forgot the case of 64 bit values on 32 bit platforms!
>>>>>>> Mostly because haven't cared about 32 bit for years...
>>>>>>>
>>>>>>> The Linux kernel uses some wrappers to handle this.
>>>>>>> On 64 bit platforms they become noop.
>>>>>>> On 32 bit platform, they are protected by a seqlock and
>> updates
>>>> are
>>>>>>> wrapped by the sequence count.
>>>>>>>
>>>>>>> If we go this way, then doing similar Noop on 64 bit and
>> atomic or
>>>>>>> seqlock
>>>>>>> on 32 bit should be done, but in common helper.
>>>>>>>
>>>>>>> Looking inside FreeBSD, it looks like that has changed over
>> the
>>>>> years as
>>>>>>> well.
>>>>>>>
>>>>>>> if_inc_counter
>>>>>>> counter_u64_add
>>>>>>> atomic_add_64
>>>>>>> But the counters are always per-cpu in this case. So although
>> it
>>>>> does
>>>>>>> use
>>>>>>> locked operation, will always be uncontended.
>>>>>>>
>>>>>>>
>>>>>>> PS: Does DPDK still actually support 32 bit on x86? Can it be
>>>>> dropped
>>>>>>> this cycle?
>>>>>>
>>>>>> We cannot drop 32 bit architecture support altogether.
>>>>>>
>>>>>> But, unlike the Linux kernel, DPDK doesn't need to support
>> ancient
>>>> 32
>>>>> bit architectures.
>>>>>> If the few 32 bit architectures supported by DPDK provide non-
>>>> tearing
>>>>> 64 bit loads/stores, we don't need locks (in the fast path) for 64
>> bit
>>>>> counters.
>>>>>>
>>>>>> In addition to 32 bit x86, DPDK supports ARMv7-A (a 32 bit
>>>>> architecture) and 32 bit ARMv8.
>>>>>> I don't think DPDK support any other 32 bit architectures.
>>>>>>
>>>>>>
>>>>>> As Mattias mentioned, 32 bit x86 can use xmm registers to
>> provide 64
>>>>> bit non-tearing load/store.
>>>>>>
>>>>>
>>>>> Testing this a little in godbolt, I see gcc using xmm registers on
>> 32-
>>>>> bit
>>>>> when updating 64-bit counters, but clang doesn't seem to do so,
>> but
>>>>> instead
>>>>> does 2 stores when writing back the 64 value. (I tried with both
>>>>> volatile
>>>>> and non-volatile 64-bit values, just to see if volatile would
>>>> encourage
>>>>> clang to do a single store).
>>>>>
>>>>> GCC: https://godbolt.org/z/9eqKfT3hz
>>>>> Clang: https://godbolt.org/z/PT5EqKn4c
>>>>
>>>> Interesting.
>>>> I guess this can be fixed by manually implementing what GCC does.
>>>>
>>>> I'm more concerned about finding a high-performance (in the fast
>> path)
>>>> 64 bit counter solution for 32 bit ARM.
>>>
>>> Reading up on the topic, and continuing Bruce's experiment on Godbolt,
>> it is possible on 32 bit ARMv7-A too, using LDRD/STRD (Load/Store
>> Register Dual) instructions, which load/store 64 bit from memory into
>> two registers at once.
>>>
>>> Clang is emits more efficient code without volatile.
>>> GCC requires volatile to use STRD.
>>>
>>> Clang: https://godbolt.org/z/WjdTq6EKh
>>> GCC: https://godbolt.org/z/qq9j7d4Ea
>>>
>>> Summing it up, it is possible to implement non-tearing 64 bit high-
>> performance (lockless, barrier-free) counters on the 32 bit
>> architectures supported by DPDK.
>>>
>>> But the implementation is both architecture and compiler specific.
>>> So it seems a "64 bit counters" library would be handy. (Or a "non-
>> tearing 64 bit integers" library, for support of the signed variant too;
>> but I don't think we need that.)
>>> We can use uint64_t as the underlying type and type cast in the
>> library (where needed by the specific compiler/architecture), or
>> introduce a new rte_ctr64_t type to ensure that accessor functions are
>> always used and the developer doesn't have to worry about tearing on 32
>> bit architectures.
>>>
>>> The most simple variant of such a library only provides load and store
>> functions. The API would look like this:
>>>
>>> uint64_t inline
>>> rte_ctr64_get(const rte_ctr64_t *const ctr);
>>>
>>> void inline
>>> rte_ctr64_set(rte_ctr64_t *const ctr, const uint64_t value);
>>>
>>> And if some CPU offers special instructions for increment or addition,
>> faster (regarding performance) and/or more compact (regarding
>> instruction memory) than a sequence of load-add-store instructions:
>>>
>>> void inline
>>> rte_ctr64_inc(rte_ctr64_t *const ctr);
>>>
>>> void inline
>>> rte_ctr64_add(rte_ctr64_t *const ctr, const uint64_t value);
>
> Note: 32 bit architectures might achieve higher performance if the "value" parameter to rte_ctr64_add() is unsigned long (or unsigned int) instead of uint64_t.
>
If the rte_ctr64_add() implementation is kept in the header file (or,
always in case of LTO), I don't see how that could make a difference,
assuming the operand is of a smaller (<64 bits) type.
>>>
>>> <feature creep>
>>> And perhaps atomic variants of all these functions, with explicit
>> and/or relaxed memory ordering, for counters shared by multiple threads.
>>> </feature creep>
>>>
>>
>>
>> This kind of what I am experimenting with but...
>
> Excellent!
>
>> Intend to keep the details of the counters read and update in one file
>> and not as inlines.
>
> Please note that traffic management applications maintain many counters (e.g. per-flow, per-subscriber and per-link packet and byte counters, some also per QoS class), so rte_ctr64_add() must have the highest performance technically possible.
>
> For reference, the packet scheduling code in our application updates 28 statistics counters per burst of packets. (In addition to internal state variables for e.g. queue lenghts.)
>
> Furthermore, our application processes and displays live statistics with one second granularity, using a separate thread. Although statistics processing is not part of the fast path, the sheer number of counters processed requires high performance read access to those counters.
>
>
> <more feature creep>
> Some groups of counters are maintained locally in the inner loop, and then added in bulk to the "public" statistics afterwards. Conceptually:
>
> struct stats_per_prio {
> uint64_t forwarded_bytes;
> uint64_t forwarded_packets;
> uint64_t marked_bytes;
> uint64_t marked_packets;
> uint64_t dropped_bytes;
> uint64_t dropped_packets;
> };
>
> If this is a common design pattern in DPDK (drivers, libraries and/or applications), perhaps also provide a performance optimized function for architectures offering vector instructions:
>
> void inline
> rte_ctr64_add_bulk(
> rte_ctr64_t *const ctrs,
> const unsigned long *const values,
> const unsigned int n /* compile time constant */);
>
> This slightly resembles the Linux kernel's design pattern, where counters are updated in bulk, protected by a common lock for the bulk update. (However, DPDK has no lock, so the motivation for optimizing for this design pattern is only "nice to have".)
>
> PS:
> Many DPDK applications are 64 bit only, including the SmartShare appliances, and can easily manage 64 bit counters without all this.
> However, if the DPDK project still considers 32 bit architectures first class citizens, 64 bit counters should have high performance on 32 bit architectures too.
>
next prev parent reply other threads:[~2024-05-26 7:10 UTC|newest]
Thread overview: 178+ messages / expand[flat|nested] mbox.gz Atom feed top
2024-04-25 17:46 [RFC] " Ferruh Yigit
2024-04-26 11:33 ` Morten Brørup
2024-04-26 13:37 ` Ferruh Yigit
2024-04-26 14:56 ` Morten Brørup
2024-04-28 15:42 ` Mattias Rönnblom
2024-04-26 14:38 ` [RFC v2] " Ferruh Yigit
2024-04-26 14:47 ` Morten Brørup
2024-04-28 15:11 ` Mattias Rönnblom
2024-05-01 16:19 ` Ferruh Yigit
2024-05-02 5:51 ` Mattias Rönnblom
2024-05-02 14:22 ` Ferruh Yigit
2024-05-02 15:59 ` Stephen Hemminger
2024-05-02 18:20 ` Ferruh Yigit
2024-05-02 17:37 ` Mattias Rönnblom
2024-05-02 18:26 ` Stephen Hemminger
2024-05-02 21:26 ` Mattias Rönnblom
2024-05-02 21:46 ` Stephen Hemminger
2024-05-07 7:23 ` Mattias Rönnblom
2024-05-07 13:49 ` Ferruh Yigit
2024-05-07 14:51 ` Stephen Hemminger
2024-05-07 16:00 ` Morten Brørup
2024-05-07 16:54 ` Ferruh Yigit
2024-05-07 18:47 ` Stephen Hemminger
2024-05-08 7:48 ` Mattias Rönnblom
2024-05-08 6:28 ` Mattias Rönnblom
2024-05-08 6:25 ` Mattias Rönnblom
2024-05-07 19:19 ` Morten Brørup
2024-05-08 6:34 ` Mattias Rönnblom
2024-05-08 7:10 ` Morten Brørup
2024-05-08 7:23 ` Mattias Rönnblom
2024-04-26 21:28 ` [RFC] " Patrick Robb
2024-05-03 15:45 ` [RFC v3] " Ferruh Yigit
2024-05-03 22:00 ` Stephen Hemminger
2024-05-07 13:48 ` Ferruh Yigit
2024-05-07 14:52 ` Stephen Hemminger
2024-05-07 17:27 ` Ferruh Yigit
2024-05-08 7:19 ` Mattias Rönnblom
2024-05-08 15:23 ` Stephen Hemminger
2024-05-08 19:48 ` Ferruh Yigit
2024-05-08 20:54 ` Stephen Hemminger
2024-05-09 7:43 ` Morten Brørup
2024-05-09 9:29 ` Bruce Richardson
2024-05-09 11:37 ` Morten Brørup
2024-05-09 14:19 ` Morten Brørup
2024-05-10 4:56 ` Stephen Hemminger
2024-05-10 9:14 ` Morten Brørup
2024-05-26 7:10 ` Mattias Rönnblom [this message]
2024-05-26 7:07 ` Mattias Rönnblom
2024-05-26 7:03 ` Mattias Rönnblom
2024-05-26 7:21 ` Mattias Rönnblom
2024-05-07 15:27 ` Morten Brørup
2024-05-07 17:40 ` Ferruh Yigit
2024-05-10 5:01 ` [RFC 0/3] generic sw counters Stephen Hemminger
2024-05-10 5:01 ` [RFC 1/3] ethdev: add internal helper of SW driver statistics Stephen Hemminger
2024-05-10 5:01 ` [RFC 2/3] net/af_packet: use SW stats helper Stephen Hemminger
2024-05-10 5:01 ` [RFC 3/3] net/tap: use generic SW stats Stephen Hemminger
2024-05-10 17:29 ` [RFC 0/3] generic sw counters Morten Brørup
2024-05-10 19:30 ` Stephen Hemminger
2024-05-13 18:52 ` [RFC v2 0/7] generic SW counters Stephen Hemminger
2024-05-13 18:52 ` [RFC v2 1/7] eal: generic 64 bit counter Stephen Hemminger
2024-05-13 19:36 ` Morten Brørup
2024-05-13 18:52 ` [RFC v2 2/7] ethdev: add internal helper of SW driver statistics Stephen Hemminger
2024-05-13 18:52 ` [RFC v2 3/7] net/af_packet: use SW stats helper Stephen Hemminger
2024-05-13 18:52 ` [RFC v2 4/7] net/tap: use generic SW stats Stephen Hemminger
2024-05-13 18:52 ` [RFC v2 5/7] net/pcap: " Stephen Hemminger
2024-05-13 18:52 ` [RFC v2 6/7] net/af_xdp: " Stephen Hemminger
2024-05-13 18:52 ` [RFC v2 7/7] net/ring: " Stephen Hemminger
2024-05-14 15:35 ` [PATCH v3 0/7] Generic SW counters Stephen Hemminger
2024-05-14 15:35 ` [PATCH v3 1/7] eal: generic 64 bit counter Stephen Hemminger
2024-05-15 9:30 ` Morten Brørup
2024-05-15 15:03 ` Stephen Hemminger
2024-05-15 16:18 ` Morten Brørup
2024-05-26 7:34 ` Mattias Rönnblom
2024-05-26 6:45 ` Mattias Rönnblom
2024-05-14 15:35 ` [PATCH v3 2/7] ethdev: add internal helper of SW driver statistics Stephen Hemminger
2024-05-14 15:35 ` [PATCH v3 3/7] net/af_packet: use SW stats helper Stephen Hemminger
2024-05-14 15:35 ` [PATCH v3 4/7] net/af_xdp: use generic SW stats Stephen Hemminger
2024-05-14 15:35 ` [PATCH v3 5/7] net/pcap: " Stephen Hemminger
2024-05-14 15:35 ` [PATCH v3 6/7] net/ring: " Stephen Hemminger
2024-05-14 15:35 ` [PATCH v3 7/7] net/tap: " Stephen Hemminger
2024-05-15 23:40 ` [PATCH v4 0/8] Generic 64 bit counters for SW PMD's Stephen Hemminger
2024-05-15 23:40 ` [PATCH v4 1/8] eal: generic 64 bit counter Stephen Hemminger
2024-05-15 23:40 ` [PATCH v4 2/8] ethdev: add common counters for statistics Stephen Hemminger
2024-05-15 23:40 ` [PATCH v4 3/8] net/af_packet: use generic SW stats Stephen Hemminger
2024-05-15 23:40 ` [PATCH v4 4/8] net/af_xdp: " Stephen Hemminger
2024-05-15 23:40 ` [PATCH v4 5/8] net/pcap: " Stephen Hemminger
2024-05-15 23:40 ` [PATCH v4 6/8] net/ring: " Stephen Hemminger
2024-05-15 23:40 ` [PATCH v4 7/8] net/tap: " Stephen Hemminger
2024-05-15 23:41 ` [PATCH v4 8/8] net/null: " Stephen Hemminger
2024-05-16 15:40 ` [PATCH v5 0/9] Generic 64 bit counters Stephen Hemminger
2024-05-16 15:40 ` [PATCH v5 1/9] eal: generic 64 bit counter Stephen Hemminger
2024-05-16 18:22 ` Wathsala Wathawana Vithanage
2024-05-16 21:42 ` Stephen Hemminger
2024-05-17 2:39 ` Honnappa Nagarahalli
2024-05-17 3:29 ` Stephen Hemminger
2024-05-17 4:39 ` Honnappa Nagarahalli
2024-05-16 15:40 ` [PATCH v5 2/9] ethdev: add common counters for statistics Stephen Hemminger
2024-05-16 18:30 ` Wathsala Wathawana Vithanage
2024-05-17 0:19 ` Stephen Hemminger
2024-05-16 15:40 ` [PATCH v5 3/9] net/af_packet: use generic SW stats Stephen Hemminger
2024-05-16 15:40 ` [PATCH v5 4/9] net/af_xdp: " Stephen Hemminger
2024-05-16 15:40 ` [PATCH v5 5/9] net/pcap: " Stephen Hemminger
2024-05-16 15:40 ` [PATCH v5 6/9] test/pmd_ring: initialize mbufs Stephen Hemminger
2024-05-16 15:40 ` [PATCH v5 7/9] net/ring: use generic SW stats Stephen Hemminger
2024-05-16 15:40 ` [PATCH v5 8/9] net/tap: " Stephen Hemminger
2024-05-16 15:40 ` [PATCH v5 9/9] net/null: " Stephen Hemminger
2024-05-17 0:12 ` [PATCH v6 0/9] Generic 64 bit counters for SW drivers Stephen Hemminger
2024-05-17 0:12 ` [PATCH v6 1/9] eal: generic 64 bit counter Stephen Hemminger
2024-05-17 2:45 ` Honnappa Nagarahalli
2024-05-17 3:30 ` Stephen Hemminger
2024-05-17 4:26 ` Honnappa Nagarahalli
2024-05-17 6:44 ` Morten Brørup
2024-05-17 15:05 ` Stephen Hemminger
2024-05-17 16:18 ` Stephen Hemminger
2024-05-18 14:00 ` Morten Brørup
2024-05-19 15:13 ` Stephen Hemminger
2024-05-19 17:10 ` Morten Brørup
2024-05-19 22:49 ` Stephen Hemminger
2024-05-20 7:57 ` Morten Brørup
2024-05-17 15:07 ` Stephen Hemminger
2024-05-17 0:12 ` [PATCH v6 2/9] ethdev: add common counters for statistics Stephen Hemminger
2024-05-17 0:12 ` [PATCH v6 3/9] net/af_packet: use generic SW stats Stephen Hemminger
2024-05-17 0:12 ` [PATCH v6 4/9] net/af_xdp: " Stephen Hemminger
2024-05-17 13:34 ` Loftus, Ciara
2024-05-17 14:54 ` Stephen Hemminger
2024-05-17 0:12 ` [PATCH v6 5/9] net/pcap: " Stephen Hemminger
2024-05-17 0:12 ` [PATCH v6 6/9] test/pmd_ring: initialize mbufs Stephen Hemminger
2024-05-17 0:12 ` [PATCH v6 7/9] net/ring: use generic SW stats Stephen Hemminger
2024-05-17 0:12 ` [PATCH v6 8/9] net/tap: " Stephen Hemminger
2024-05-17 0:12 ` [PATCH v6 9/9] net/null: " Stephen Hemminger
2024-05-17 17:35 ` [PATCH v7 0/9] Use weak atomic operations for SW PMD counters Stephen Hemminger
2024-05-17 17:35 ` [PATCH v7 1/9] eal: generic 64 bit counter Stephen Hemminger
2024-05-17 17:35 ` [PATCH v7 2/9] ethdev: add common counters for statistics Stephen Hemminger
2024-05-17 17:35 ` [PATCH v7 3/9] net/af_packet: use generic SW stats Stephen Hemminger
2024-05-17 17:35 ` [PATCH v7 4/9] net/af_xdp: " Stephen Hemminger
2024-05-17 17:35 ` [PATCH v7 5/9] net/pcap: " Stephen Hemminger
2024-05-17 17:35 ` [PATCH v7 6/9] test/pmd_ring: initialize mbufs Stephen Hemminger
2024-05-17 17:35 ` [PATCH v7 7/9] net/ring: use generic SW stats Stephen Hemminger
2024-05-17 17:35 ` [PATCH v7 8/9] net/tap: " Stephen Hemminger
2024-05-17 17:35 ` [PATCH v7 9/9] net/null: " Stephen Hemminger
2024-05-21 17:00 ` [PATCH v8 0/8] Common statistics routines for SW based PMD's Stephen Hemminger
2024-05-21 17:00 ` [PATCH v8 1/8] eal: generic 64 bit counter Stephen Hemminger
2024-05-21 17:00 ` [PATCH v8 2/8] ethdev: add common counters for statistics Stephen Hemminger
2024-05-21 17:00 ` [PATCH v8 3/8] net/af_packet: use generic SW stats Stephen Hemminger
2024-05-21 17:00 ` [PATCH v8 4/8] net/af_xdp: " Stephen Hemminger
2024-05-21 17:00 ` [PATCH v8 5/8] net/pcap: " Stephen Hemminger
2024-05-21 17:00 ` [PATCH v8 6/8] net/ring: " Stephen Hemminger
2024-05-21 17:00 ` [PATCH v8 7/8] net/tap: " Stephen Hemminger
2024-05-21 17:00 ` [PATCH v8 8/8] net/null: " Stephen Hemminger
2024-05-21 20:16 ` [PATCH v9 0/8] Common statistics for SW PMD's Stephen Hemminger
2024-05-21 20:16 ` [PATCH v9 1/8] eal: generic 64 bit counter Stephen Hemminger
2024-05-22 8:31 ` Morten Brørup
2024-05-22 15:33 ` Stephen Hemminger
2024-05-22 18:09 ` Morten Brørup
2024-05-22 19:53 ` Stephen Hemminger
2024-05-22 20:56 ` Morten Brørup
2024-05-22 15:37 ` Stephen Hemminger
2024-05-22 17:57 ` Morten Brørup
2024-05-22 19:01 ` Tyler Retzlaff
2024-05-22 19:51 ` Stephen Hemminger
2024-05-26 14:46 ` Mattias Rönnblom
2024-05-26 14:39 ` Mattias Rönnblom
2024-05-21 20:16 ` [PATCH v9 2/8] ethdev: add common counters for statistics Stephen Hemminger
2024-05-21 20:16 ` [PATCH v9 3/8] net/af_packet: use generic SW stats Stephen Hemminger
2024-05-21 20:16 ` [PATCH v9 4/8] net/af_xdp: " Stephen Hemminger
2024-05-21 20:16 ` [PATCH v9 5/8] net/pcap: " Stephen Hemminger
2024-05-21 20:16 ` [PATCH v9 6/8] net/ring: " Stephen Hemminger
2024-05-21 20:16 ` [PATCH v9 7/8] net/tap: " Stephen Hemminger
2024-05-21 20:16 ` [PATCH v9 8/8] net/null: " Stephen Hemminger
2024-05-22 16:12 ` [PATCH v10 0/8] Common statistics for software PMD's Stephen Hemminger
2024-05-22 16:12 ` [PATCH v10 1/8] eal: generic 64 bit counter Stephen Hemminger
2024-05-22 16:12 ` [PATCH v10 2/8] ethdev: add common counters for statistics Stephen Hemminger
2024-05-22 16:12 ` [PATCH v10 3/8] net/af_packet: use generic SW stats Stephen Hemminger
2024-05-22 16:12 ` [PATCH v10 4/8] net/af_xdp: " Stephen Hemminger
2024-05-22 16:12 ` [PATCH v10 5/8] net/pcap: " Stephen Hemminger
2024-05-22 16:12 ` [PATCH v10 6/8] net/ring: " Stephen Hemminger
2024-05-22 16:12 ` [PATCH v10 7/8] net/tap: " Stephen Hemminger
2024-05-22 16:12 ` [PATCH v10 8/8] net/null: " Stephen Hemminger
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=1cc502d9-2afd-47fb-a340-bb3d9ad0d573@lysator.liu.se \
--to=hofors@lysator.liu.se \
--cc=bruce.richardson@intel.com \
--cc=dev@dpdk.org \
--cc=ferruh.yigit@amd.com \
--cc=linville@tuxdriver.com \
--cc=mattias.ronnblom@ericsson.com \
--cc=mb@smartsharesystems.com \
--cc=stephen@networkplumber.org \
--cc=thomas@monjalon.net \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).