DPDK patches and discussions
 help / color / mirror / Atom feed
From: Raslan Darawsheh <rasland@nvidia.com>
To: Suanming Mou <suanmingm@nvidia.com>,
	Slava Ovsiienko <viacheslavo@nvidia.com>,
	Matan Azrad <matan@nvidia.com>
Cc: Ori Kam <orika@nvidia.com>, "dev@dpdk.org" <dev@dpdk.org>
Subject: Re: [dpdk-dev] [PATCH v6 00/26] net/mlx5: insertion rate optimization
Date: Tue, 13 Jul 2021 15:18:52 +0000	[thread overview]
Message-ID: <CH0PR12MB50432678280175834E989D2ACF149@CH0PR12MB5043.namprd12.prod.outlook.com> (raw)
In-Reply-To: <20210713084500.19964-1-suanmingm@nvidia.com>

Hi,

> -----Original Message-----
> From: Suanming Mou <suanmingm@nvidia.com>
> Sent: Tuesday, July 13, 2021 11:45 AM
> To: Slava Ovsiienko <viacheslavo@nvidia.com>; Matan Azrad
> <matan@nvidia.com>
> Cc: Raslan Darawsheh <rasland@nvidia.com>; Ori Kam <orika@nvidia.com>;
> dev@dpdk.org
> Subject: [PATCH v6 00/26] net/mlx5: insertion rate optimization
> 
> This patch series optimize the flow insertion rate with adding local cache to
> index pool and list.
> 
> For object which wants efficient index allocate and free, local cache will be
> very helpful.
> 
> For index pool, two level cache is added, one as local and another as global.
> The global cache is able to save all the allocated index. That means all the
> allocated index will not be freed. Once the local cache is full, the extra index
> will be flushed to the global cache. Once local cache is empty, first try to fetch
> more index from global, if global is still empty, allocate new trunk and more
> index.
> 
> For list, sub local core list is introduced. The allocated objects will be added
> and released only from local list without any locks.
> Only the objects need to be shared will be synced from global list.
> 
> ---
> 
> v6: fix compilation issue in "common/mlx5: add list lcore share" patch.
> 
> v5: code rebase to the latest version.
> 
> v4:
>    - split the list utility to commom patch
>    - fix one ipool crash issue in mlx5_ipool_get_cache() function.
>    - reorganize the common and mlx5 patches.
>    - split the doc update to an individual patch.
>    - update and rebase to the latest version.
> 
> v3: fix Windows compilation issue
> 
> v2: add the list per-lcore cache optimization
> 
> ---
> 
> Matan Azrad (9):
>   net/mlx5: optimize modify header action memory
>   net/mlx5: remove cache term from the list utility
>   net/mlx5: add per lcore cache to the list utility
>   net/mlx5: minimize list critical sections
>   net/mlx5: manage list cache entries release
>   net/mlx5: relax the list utility atomic operations
>   net/mlx5: allocate list memory by the create API
>   common/mlx5: add per-lcore cache to hash list utility
>   net/mlx5: move modify header allocator to ipool
> 
> Suanming Mou (17):
>   net/mlx5: allow limiting the index pool maximum index
>   net/mlx5: add indexed pool local cache
>   net/mlx5: add index pool foreach define
>   net/mlx5: support index pool non-lcore operations
>   net/mlx5: replace flow list with index pool
>   common/mlx5: move list utility to common
>   common/mlx5: add list lcore share
>   common/mlx5: call list callbacks with context
>   common/mlx5: allocate cache list memory individually
>   common/mlx5: optimize cache list object memory
>   common/mlx5: support list non-lcore operations
>   net/mlx5: adjust the hash bucket size
>   net/mlx5: enable index pool per-core cache
>   net/mlx5: optimize hash list table allocate on demand
>   net/mlx5: change memory release configuration
>   net/mlx5: optimize Rx queue match
>   doc: add mlx5 multiple-thread flow insertion optimization
> 
>  doc/guides/nics/mlx5.rst                |    5 +
>  doc/guides/rel_notes/release_21_08.rst  |    1 +
>  drivers/common/mlx5/linux/mlx5_glue.h   |    1 +
>  drivers/common/mlx5/mlx5_common.h       |    2 +
>  drivers/common/mlx5/mlx5_common_utils.c |  569 ++++++++---
> drivers/common/mlx5/mlx5_common_utils.h |  294 ++++--
>  drivers/common/mlx5/version.map         |    7 +
>  drivers/net/mlx5/linux/mlx5_flow_os.h   |    3 +-
>  drivers/net/mlx5/linux/mlx5_os.c        |  209 ++--
>  drivers/net/mlx5/mlx5.c                 |   34 +-
>  drivers/net/mlx5/mlx5.h                 |   46 +-
>  drivers/net/mlx5/mlx5_defs.h            |   12 +-
>  drivers/net/mlx5/mlx5_flow.c            |  308 +++---
>  drivers/net/mlx5/mlx5_flow.h            |  209 ++--
>  drivers/net/mlx5/mlx5_flow_dv.c         | 1206 +++++++++++++++--------
>  drivers/net/mlx5/mlx5_rx.h              |   14 +-
>  drivers/net/mlx5/mlx5_rxq.c             |  136 ++-
>  drivers/net/mlx5/mlx5_trigger.c         |    8 +-
>  drivers/net/mlx5/mlx5_utils.c           |  627 ++++++++----
>  drivers/net/mlx5/mlx5_utils.h           |  255 ++---
>  drivers/net/mlx5/windows/mlx5_os.c      |   11 +-
>  21 files changed, 2543 insertions(+), 1414 deletions(-)
> 
> --
> 2.25.1

Series applied to next-net-mlx,

Kindest regards,
Raslan Darawsheh

      parent reply	other threads:[~2021-07-13 15:18 UTC|newest]

Thread overview: 135+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2021-05-27  9:33 [dpdk-dev] [PATCH 0/4] net/mlx5: add indexed pool local cache Suanming Mou
2021-05-27  9:34 ` [dpdk-dev] [PATCH 1/4] net/mlx5: add index allocate with up limit Suanming Mou
2021-05-27  9:34 ` [dpdk-dev] [PATCH 2/4] net/mlx5: add indexed pool local cache Suanming Mou
2021-05-27  9:34 ` [dpdk-dev] [PATCH 3/4] net/mlx5: add index pool cache flush Suanming Mou
2021-05-27  9:34 ` [dpdk-dev] [PATCH 4/4] net/mlx5: replace flow list with index pool Suanming Mou
2021-06-30 12:45 ` [dpdk-dev] [PATCH v2 00/22] net/mlx5: insertion rate optimization Suanming Mou
2021-06-30 12:45   ` [dpdk-dev] [PATCH v2 01/22] net/mlx5: allow limiting the index pool maximum index Suanming Mou
2021-06-30 12:45   ` [dpdk-dev] [PATCH v2 02/22] net/mlx5: add indexed pool local cache Suanming Mou
2021-06-30 12:45   ` [dpdk-dev] [PATCH v2 03/22] net/mlx5: add index pool foreach define Suanming Mou
2021-06-30 12:45   ` [dpdk-dev] [PATCH v2 04/22] net/mlx5: replace flow list with index pool Suanming Mou
2021-06-30 12:45   ` [dpdk-dev] [PATCH v2 05/22] net/mlx5: optimize modify header action memory Suanming Mou
2021-06-30 12:45   ` [dpdk-dev] [PATCH v2 06/22] net/mlx5: remove cache term from the list utility Suanming Mou
2021-06-30 12:45   ` [dpdk-dev] [PATCH v2 07/22] net/mlx5: add per lcore cache to " Suanming Mou
2021-06-30 12:45   ` [dpdk-dev] [PATCH v2 08/22] net/mlx5: minimize list critical sections Suanming Mou
2021-06-30 12:45   ` [dpdk-dev] [PATCH v2 09/22] net/mlx5: manage list cache entries release Suanming Mou
2021-06-30 12:45   ` [dpdk-dev] [PATCH v2 10/22] net/mlx5: relax the list utility atomic operations Suanming Mou
2021-06-30 12:45   ` [dpdk-dev] [PATCH v2 11/22] net/mlx5: allocate list memory by the create API Suanming Mou
2021-06-30 12:45   ` [dpdk-dev] [PATCH v2 12/22] common/mlx5: add per-lcore cache to hash list utility Suanming Mou
2021-06-30 12:46   ` [dpdk-dev] [PATCH v2 13/22] net/mlx5: move modify header allocator to ipool Suanming Mou
2021-06-30 12:46   ` [dpdk-dev] [PATCH v2 14/22] net/mlx5: adjust the hash bucket size Suanming Mou
2021-06-30 12:46   ` [dpdk-dev] [PATCH v2 15/22] common/mlx5: allocate cache list memory individually Suanming Mou
2021-06-30 12:46   ` [dpdk-dev] [PATCH v2 16/22] net/mlx5: enable index pool per-core cache Suanming Mou
2021-06-30 12:46   ` [dpdk-dev] [PATCH v2 17/22] net/mlx5: optimize hash list table allocate on demand Suanming Mou
2021-06-30 12:46   ` [dpdk-dev] [PATCH v2 18/22] common/mlx5: optimize cache list object memory Suanming Mou
2021-06-30 12:46   ` [dpdk-dev] [PATCH v2 19/22] net/mlx5: change memory release configuration Suanming Mou
2021-06-30 12:46   ` [dpdk-dev] [PATCH v2 20/22] net/mlx5: support index pool none local core operations Suanming Mou
2021-06-30 12:46   ` [dpdk-dev] [PATCH v2 21/22] net/mlx5: support list " Suanming Mou
2021-06-30 12:46   ` [dpdk-dev] [PATCH v2 22/22] net/mlx5: optimize Rx queue match Suanming Mou
2021-07-02  6:17 ` [dpdk-dev] [PATCH v3 00/22] net/mlx5: insertion rate optimization Suanming Mou
2021-07-02  6:17   ` [dpdk-dev] [PATCH v3 01/22] net/mlx5: allow limiting the index pool maximum index Suanming Mou
2021-07-02  6:17   ` [dpdk-dev] [PATCH v3 02/22] net/mlx5: add indexed pool local cache Suanming Mou
2021-07-02  6:17   ` [dpdk-dev] [PATCH v3 03/22] net/mlx5: add index pool foreach define Suanming Mou
2021-07-02  6:17   ` [dpdk-dev] [PATCH v3 04/22] net/mlx5: replace flow list with index pool Suanming Mou
2021-07-02  6:17   ` [dpdk-dev] [PATCH v3 05/22] net/mlx5: optimize modify header action memory Suanming Mou
2021-07-02  6:18   ` [dpdk-dev] [PATCH v3 06/22] net/mlx5: remove cache term from the list utility Suanming Mou
2021-07-02  6:18   ` [dpdk-dev] [PATCH v3 07/22] net/mlx5: add per lcore cache to " Suanming Mou
2021-07-02  6:18   ` [dpdk-dev] [PATCH v3 08/22] net/mlx5: minimize list critical sections Suanming Mou
2021-07-02  6:18   ` [dpdk-dev] [PATCH v3 09/22] net/mlx5: manage list cache entries release Suanming Mou
2021-07-02  6:18   ` [dpdk-dev] [PATCH v3 10/22] net/mlx5: relax the list utility atomic operations Suanming Mou
2021-07-02  6:18   ` [dpdk-dev] [PATCH v3 11/22] net/mlx5: allocate list memory by the create API Suanming Mou
2021-07-02  6:18   ` [dpdk-dev] [PATCH v3 12/22] common/mlx5: add per-lcore cache to hash list utility Suanming Mou
2021-07-02  6:18   ` [dpdk-dev] [PATCH v3 13/22] net/mlx5: move modify header allocator to ipool Suanming Mou
2021-07-02  6:18   ` [dpdk-dev] [PATCH v3 14/22] net/mlx5: adjust the hash bucket size Suanming Mou
2021-07-02  6:18   ` [dpdk-dev] [PATCH v3 15/22] common/mlx5: allocate cache list memory individually Suanming Mou
2021-07-02  6:18   ` [dpdk-dev] [PATCH v3 16/22] net/mlx5: enable index pool per-core cache Suanming Mou
2021-07-02  6:18   ` [dpdk-dev] [PATCH v3 17/22] net/mlx5: optimize hash list table allocate on demand Suanming Mou
2021-07-02  6:18   ` [dpdk-dev] [PATCH v3 18/22] common/mlx5: optimize cache list object memory Suanming Mou
2021-07-02  6:18   ` [dpdk-dev] [PATCH v3 19/22] net/mlx5: change memory release configuration Suanming Mou
2021-07-02  6:18   ` [dpdk-dev] [PATCH v3 20/22] net/mlx5: support index pool none local core operations Suanming Mou
2021-07-02  6:18   ` [dpdk-dev] [PATCH v3 21/22] net/mlx5: support list " Suanming Mou
2021-07-02  6:18   ` [dpdk-dev] [PATCH v3 22/22] net/mlx5: optimize Rx queue match Suanming Mou
2021-07-06 13:32 ` [dpdk-dev] [PATCH v4 00/26] net/mlx5: insertion rate optimization Suanming Mou
2021-07-06 13:32   ` [dpdk-dev] [PATCH v4 01/26] net/mlx5: allow limiting the index pool maximum index Suanming Mou
2021-07-06 13:32   ` [dpdk-dev] [PATCH v4 02/26] net/mlx5: add indexed pool local cache Suanming Mou
2021-07-06 13:32   ` [dpdk-dev] [PATCH v4 03/26] net/mlx5: add index pool foreach define Suanming Mou
2021-07-06 13:32   ` [dpdk-dev] [PATCH v4 04/26] net/mlx5: support index pool non-lcore operations Suanming Mou
2021-07-06 13:32   ` [dpdk-dev] [PATCH v4 05/26] net/mlx5: replace flow list with index pool Suanming Mou
2021-07-06 13:32   ` [dpdk-dev] [PATCH v4 06/26] net/mlx5: optimize modify header action memory Suanming Mou
2021-07-06 13:32   ` [dpdk-dev] [PATCH v4 07/26] net/mlx5: remove cache term from the list utility Suanming Mou
2021-07-06 13:32   ` [dpdk-dev] [PATCH v4 08/26] net/mlx5: add per lcore cache to " Suanming Mou
2021-07-06 13:32   ` [dpdk-dev] [PATCH v4 09/26] net/mlx5: minimize list critical sections Suanming Mou
2021-07-06 13:32   ` [dpdk-dev] [PATCH v4 10/26] net/mlx5: manage list cache entries release Suanming Mou
2021-07-06 13:32   ` [dpdk-dev] [PATCH v4 11/26] net/mlx5: relax the list utility atomic operations Suanming Mou
2021-07-06 13:32   ` [dpdk-dev] [PATCH v4 12/26] net/mlx5: allocate list memory by the create API Suanming Mou
2021-07-06 13:32   ` [dpdk-dev] [PATCH v4 13/26] common/mlx5: move list utility to common Suanming Mou
2021-07-06 13:32   ` [dpdk-dev] [PATCH v4 14/26] common/mlx5: add list lcore share Suanming Mou
2021-07-06 13:32   ` [dpdk-dev] [PATCH v4 15/26] common/mlx5: call list callbacks with context Suanming Mou
2021-07-06 13:32   ` [dpdk-dev] [PATCH v4 16/26] common/mlx5: add per-lcore cache to hash list utility Suanming Mou
2021-07-06 13:32   ` [dpdk-dev] [PATCH v4 17/26] common/mlx5: allocate cache list memory individually Suanming Mou
2021-07-06 13:32   ` [dpdk-dev] [PATCH v4 18/26] common/mlx5: optimize cache list object memory Suanming Mou
2021-07-06 13:32   ` [dpdk-dev] [PATCH v4 19/26] common/mlx5: support list non-lcore operations Suanming Mou
2021-07-06 13:32   ` [dpdk-dev] [PATCH v4 20/26] net/mlx5: move modify header allocator to ipool Suanming Mou
2021-07-06 13:32   ` [dpdk-dev] [PATCH v4 21/26] net/mlx5: adjust the hash bucket size Suanming Mou
2021-07-06 13:32   ` [dpdk-dev] [PATCH v4 22/26] net/mlx5: enable index pool per-core cache Suanming Mou
2021-07-06 13:32   ` [dpdk-dev] [PATCH v4 23/26] net/mlx5: optimize hash list table allocate on demand Suanming Mou
2021-07-06 13:32   ` [dpdk-dev] [PATCH v4 24/26] net/mlx5: change memory release configuration Suanming Mou
2021-07-06 13:32   ` [dpdk-dev] [PATCH v4 25/26] net/mlx5: optimize Rx queue match Suanming Mou
2021-07-06 13:32   ` [dpdk-dev] [PATCH v4 26/26] doc: add mlx5 multiple-thread flow insertion optimization Suanming Mou
2021-07-12  1:46 ` [dpdk-dev] [PATCH v5 00/26] net/mlx5: insertion rate optimization Suanming Mou
2021-07-12  1:46   ` [dpdk-dev] [PATCH v5 01/26] net/mlx5: allow limiting the index pool maximum index Suanming Mou
2021-07-12  1:46   ` [dpdk-dev] [PATCH v5 02/26] net/mlx5: add indexed pool local cache Suanming Mou
2021-07-12  1:46   ` [dpdk-dev] [PATCH v5 03/26] net/mlx5: add index pool foreach define Suanming Mou
2021-07-12  1:46   ` [dpdk-dev] [PATCH v5 04/26] net/mlx5: support index pool non-lcore operations Suanming Mou
2021-07-12  1:46   ` [dpdk-dev] [PATCH v5 05/26] net/mlx5: replace flow list with index pool Suanming Mou
2021-07-12  1:46   ` [dpdk-dev] [PATCH v5 06/26] net/mlx5: optimize modify header action memory Suanming Mou
2021-07-12  1:46   ` [dpdk-dev] [PATCH v5 07/26] net/mlx5: remove cache term from the list utility Suanming Mou
2021-07-12  1:46   ` [dpdk-dev] [PATCH v5 08/26] net/mlx5: add per lcore cache to " Suanming Mou
2021-07-12  1:46   ` [dpdk-dev] [PATCH v5 09/26] net/mlx5: minimize list critical sections Suanming Mou
2021-07-12  1:46   ` [dpdk-dev] [PATCH v5 10/26] net/mlx5: manage list cache entries release Suanming Mou
2021-07-12  1:46   ` [dpdk-dev] [PATCH v5 11/26] net/mlx5: relax the list utility atomic operations Suanming Mou
2021-07-12  1:46   ` [dpdk-dev] [PATCH v5 12/26] net/mlx5: allocate list memory by the create API Suanming Mou
2021-07-12  1:46   ` [dpdk-dev] [PATCH v5 13/26] common/mlx5: move list utility to common Suanming Mou
2021-07-12  1:46   ` [dpdk-dev] [PATCH v5 14/26] common/mlx5: add list lcore share Suanming Mou
2021-07-12 14:59     ` Raslan Darawsheh
2021-07-12 23:26       ` Suanming Mou
2021-07-12  1:46   ` [dpdk-dev] [PATCH v5 15/26] common/mlx5: call list callbacks with context Suanming Mou
2021-07-12  1:46   ` [dpdk-dev] [PATCH v5 16/26] common/mlx5: add per-lcore cache to hash list utility Suanming Mou
2021-07-12  1:46   ` [dpdk-dev] [PATCH v5 17/26] common/mlx5: allocate cache list memory individually Suanming Mou
2021-07-12  1:46   ` [dpdk-dev] [PATCH v5 18/26] common/mlx5: optimize cache list object memory Suanming Mou
2021-07-12  1:46   ` [dpdk-dev] [PATCH v5 19/26] common/mlx5: support list non-lcore operations Suanming Mou
2021-07-12  1:46   ` [dpdk-dev] [PATCH v5 20/26] net/mlx5: move modify header allocator to ipool Suanming Mou
2021-07-12  1:46   ` [dpdk-dev] [PATCH v5 21/26] net/mlx5: adjust the hash bucket size Suanming Mou
2021-07-12  1:46   ` [dpdk-dev] [PATCH v5 22/26] net/mlx5: enable index pool per-core cache Suanming Mou
2021-07-12  1:46   ` [dpdk-dev] [PATCH v5 23/26] net/mlx5: optimize hash list table allocate on demand Suanming Mou
2021-07-12  1:46   ` [dpdk-dev] [PATCH v5 24/26] net/mlx5: change memory release configuration Suanming Mou
2021-07-12  1:46   ` [dpdk-dev] [PATCH v5 25/26] net/mlx5: optimize Rx queue match Suanming Mou
2021-07-12  1:46   ` [dpdk-dev] [PATCH v5 26/26] doc: add mlx5 multiple-thread flow insertion optimization Suanming Mou
2021-07-13  8:44 ` [dpdk-dev] [PATCH v6 00/26] net/mlx5: insertion rate optimization Suanming Mou
2021-07-13  8:44   ` [dpdk-dev] [PATCH v6 01/26] net/mlx5: allow limiting the index pool maximum index Suanming Mou
2021-07-13  8:44   ` [dpdk-dev] [PATCH v6 02/26] net/mlx5: add indexed pool local cache Suanming Mou
2021-07-13  8:44   ` [dpdk-dev] [PATCH v6 03/26] net/mlx5: add index pool foreach define Suanming Mou
2021-07-13  8:44   ` [dpdk-dev] [PATCH v6 04/26] net/mlx5: support index pool non-lcore operations Suanming Mou
2021-07-13  8:44   ` [dpdk-dev] [PATCH v6 05/26] net/mlx5: replace flow list with index pool Suanming Mou
2021-07-13  8:44   ` [dpdk-dev] [PATCH v6 06/26] net/mlx5: optimize modify header action memory Suanming Mou
2021-07-13  8:44   ` [dpdk-dev] [PATCH v6 07/26] net/mlx5: remove cache term from the list utility Suanming Mou
2021-07-13  8:44   ` [dpdk-dev] [PATCH v6 08/26] net/mlx5: add per lcore cache to " Suanming Mou
2021-07-13  8:44   ` [dpdk-dev] [PATCH v6 09/26] net/mlx5: minimize list critical sections Suanming Mou
2021-07-13  8:44   ` [dpdk-dev] [PATCH v6 10/26] net/mlx5: manage list cache entries release Suanming Mou
2021-07-13  8:44   ` [dpdk-dev] [PATCH v6 11/26] net/mlx5: relax the list utility atomic operations Suanming Mou
2021-07-13  8:44   ` [dpdk-dev] [PATCH v6 12/26] net/mlx5: allocate list memory by the create API Suanming Mou
2021-07-13  8:44   ` [dpdk-dev] [PATCH v6 13/26] common/mlx5: move list utility to common Suanming Mou
2021-07-13  8:44   ` [dpdk-dev] [PATCH v6 14/26] common/mlx5: add list lcore share Suanming Mou
2021-07-13  8:44   ` [dpdk-dev] [PATCH v6 15/26] common/mlx5: call list callbacks with context Suanming Mou
2021-07-13  8:44   ` [dpdk-dev] [PATCH v6 16/26] common/mlx5: add per-lcore cache to hash list utility Suanming Mou
2021-07-13  8:44   ` [dpdk-dev] [PATCH v6 17/26] common/mlx5: allocate cache list memory individually Suanming Mou
2021-07-13  8:44   ` [dpdk-dev] [PATCH v6 18/26] common/mlx5: optimize cache list object memory Suanming Mou
2021-07-13  8:44   ` [dpdk-dev] [PATCH v6 19/26] common/mlx5: support list non-lcore operations Suanming Mou
2021-07-13  8:44   ` [dpdk-dev] [PATCH v6 20/26] net/mlx5: move modify header allocator to ipool Suanming Mou
2021-07-13  8:44   ` [dpdk-dev] [PATCH v6 21/26] net/mlx5: adjust the hash bucket size Suanming Mou
2021-07-13  8:44   ` [dpdk-dev] [PATCH v6 22/26] net/mlx5: enable index pool per-core cache Suanming Mou
2021-07-13  8:44   ` [dpdk-dev] [PATCH v6 23/26] net/mlx5: optimize hash list table allocate on demand Suanming Mou
2021-07-13  8:44   ` [dpdk-dev] [PATCH v6 24/26] net/mlx5: change memory release configuration Suanming Mou
2021-07-13  8:44   ` [dpdk-dev] [PATCH v6 25/26] net/mlx5: optimize Rx queue match Suanming Mou
2021-07-13  8:45   ` [dpdk-dev] [PATCH v6 26/26] doc: add mlx5 multiple-thread flow insertion optimization Suanming Mou
2021-07-13 15:18   ` Raslan Darawsheh [this message]

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=CH0PR12MB50432678280175834E989D2ACF149@CH0PR12MB5043.namprd12.prod.outlook.com \
    --to=rasland@nvidia.com \
    --cc=dev@dpdk.org \
    --cc=matan@nvidia.com \
    --cc=orika@nvidia.com \
    --cc=suanmingm@nvidia.com \
    --cc=viacheslavo@nvidia.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).