DPDK patches and discussions
 help / color / mirror / Atom feed
From: Akhil Goyal <akhil.goyal@nxp.com>
To: "Ananyev, Konstantin" <konstantin.ananyev@intel.com>,
	"dev@dpdk.org" <dev@dpdk.org>
Cc: "Nicolau, Radu" <radu.nicolau@intel.com>,
	"Horton, Remy" <remy.horton@intel.com>
Subject: Re: [dpdk-dev] [PATCH v4 1/9] examples/ipsec-secgw: avoid to request unused TX offloads
Date: Mon, 24 Dec 2018 12:31:33 +0000	[thread overview]
Message-ID: <0cc8fd94-8530-9ce0-e03b-1e37d65ecb25@nxp.com> (raw)
In-Reply-To: <2601191342CEEE43887BDE71AB977258010D8BE8EE@IRSMSX106.ger.corp.intel.com>



On 12/24/2018 5:07 PM, Ananyev, Konstantin wrote:
>
>> -----Original Message-----
>> From: Akhil Goyal [mailto:akhil.goyal@nxp.com]
>> Sent: Monday, December 24, 2018 11:25 AM
>> To: Ananyev, Konstantin <konstantin.ananyev@intel.com>; dev@dpdk.org
>> Cc: Nicolau, Radu <radu.nicolau@intel.com>; Horton, Remy <remy.horton@intel.com>
>> Subject: Re: [dpdk-dev] [PATCH v4 1/9] examples/ipsec-secgw: avoid to request unused TX offloads
>>
>>
>>
>> On 12/24/2018 4:52 PM, Ananyev, Konstantin wrote:
>>>> -----Original Message-----
>>>> From: Akhil Goyal [mailto:akhil.goyal@nxp.com]
>>>> Sent: Monday, December 24, 2018 10:54 AM
>>>> To: Ananyev, Konstantin <konstantin.ananyev@intel.com>; dev@dpdk.org
>>>> Cc: Nicolau, Radu <radu.nicolau@intel.com>; Horton, Remy <remy.horton@intel.com>
>>>> Subject: Re: [dpdk-dev] [PATCH v4 1/9] examples/ipsec-secgw: avoid to request unused TX offloads
>>>>
>>>>
>>>>
>>>> On 12/24/2018 3:49 PM, Ananyev, Konstantin wrote:
>>>>>>>> On 12/14/2018 10:10 PM, Konstantin Ananyev wrote:
>>>>>>>>> ipsec-secgw always enables TX offloads
>>>>>>>>> (DEV_TX_OFFLOAD_MULTI_SEGS, DEV_TX_OFFLOAD_SECURITY),
>>>>>>>>> even when they are not requested by the config.
>>>>>>>>> That causes many PMD to choose full-featured TX function,
>>>>>>>>> which in many cases is much slower then one without offloads.
>>>>>>>>> That patch adds checks to enabled extra HW offloads, only when
>>>>>>>>> they were requested.
>>>>>>>>> Plus it enables DEV_TX_OFFLOAD_IPV4_CKSUM,
>>>>>>>>> only when other HW TX ofloads are going to be enabled.
>>>>>>>>> Otherwise SW version of ip cksum calculation is used.
>>>>>>>>> That allows to use vector TX function, when inline-ipsec is not
>>>>>>>>> requested.
>>>>>>>>>
>>>>>>>>> Signed-off-by: Remy Horton <remy.horton@intel.com>
>>>>>>>>> Signed-off-by: Konstantin Ananyev <konstantin.ananyev@intel.com>
>>>>>>>>> Acked-by: Radu Nicolau <radu.nicolau@intel.com>
>>>>>>>>> ---
>>>>>>>>>       examples/ipsec-secgw/ipsec-secgw.c | 44 +++++++++++++++--------
>>>>>>>>>       examples/ipsec-secgw/ipsec.h       |  6 ++++
>>>>>>>>>       examples/ipsec-secgw/sa.c          | 56 ++++++++++++++++++++++++++++++
>>>>>>>>>       3 files changed, 91 insertions(+), 15 deletions(-)
>>>>>>>>>
>>>>>>>>> diff --git a/examples/ipsec-secgw/ipsec-secgw.c b/examples/ipsec-secgw/ipsec-secgw.c
>>>>>>>>> index 1bc0b5b50..cfc2b05e5 100644
>>>>>>>>> --- a/examples/ipsec-secgw/ipsec-secgw.c
>>>>>>>>> +++ b/examples/ipsec-secgw/ipsec-secgw.c
>>>>>>>>> @@ -208,8 +208,6 @@ static struct rte_eth_conf port_conf = {
>>>>>>>>>       	},
>>>>>>>>>       	.txmode = {
>>>>>>>>>       		.mq_mode = ETH_MQ_TX_NONE,
>>>>>>>>> -		.offloads = (DEV_TX_OFFLOAD_IPV4_CKSUM |
>>>>>>>>> -			     DEV_TX_OFFLOAD_MULTI_SEGS),
>>>>>>>> I believe this is disabling checksum offload for all cases and then
>>>>>>>> enabling only for inline crypto and inline proto.
>>>>>>> Yes.
>>>>>>>
>>>>>>>> This is breaking lookaside proto and lookaside none cases. Please
>>>>>>>> correct me if I am wrong.
>>>>>>> Why breaking?
>>>>>> reduction in performance is kind of breaking the code.
>>>>> I didn’t observe any performance drop with that patch.
>>>>> In fact there was a tiny improvement (see below).
>>>>> Did you see any regression with this patch on your HW?
>>>> NXP hardware are low -end to mid end devices and we are always
>>>> bottleneck by core cycles.
>>>> So we would like to have as much offloads to HW as possible.
>>> Ok, then I suppose we need to introduce new cmd-line options,
>>> Something like: --txoffloads=<tx_offload_mask> --rx_offloads=<rx_offload_mask>
>>> to keep everyone happy.
>>> Are you ok with that?
>> I think it should be taken from the PMD capabilities.
> Don't see how?
> Let say, Intel NICs do support HW IPv4 cksum offload, but we don't want
> to enable it on its own - only if IPsec offload is also enabled.
>  From other side you want HW IPv4 cksum offload to be always enabled
> if present.
> As I can see, to fulfill everyone needs we need to provide user ability
> to specify which HW offloads to use.
if there is no other way to resolve it, then probably you can add an 
optional parameter to disable offloads when it is required.
I believe default behavior should not be changed.
>
>> cmd line for every
>> parameter will make it very complex.
>>> Konstantin
>>>
>>>>>>> For cases when HW cksum offload is disabled, IPv4 cksum calculation
>>>>>>> will be done in SW, see below:
>>>>>>> prepare_tx_pkt(...)
>>>>>>> {
>>>>>>>        ...
>>>>>>>         +
>>>>>>>         +		/* calculate IPv4 cksum in SW */
>>>>>>>         +		if ((pkt->ol_flags & PKT_TX_IP_CKSUM) == 0)
>>>>>>>         +			ip->ip_sum = rte_ipv4_cksum((struct ipv4_hdr *)ip);
>>>>>>>
>>>>>>>
>>>>>>> We tested lookaside-none case quite extensively - all works well,
>>>>>>> in fact on Intel NICs it became even a bit faster because of that change
>>>>>>> (though not much).
>>>>>> yes, it may work well on one hardware, but may not perform good in other
>>>>>> hardware where cores are limited.
>>>>> Could you elaborate a bit more what do you mean by 'cores are limited' here?
>>>> we have single core devices as well on which we run ipsec-secgw.
>>>>> Do you mean that for some low end cpus calculating IPv4 cksum in SW is too expensive?
>>>> yes, limited by core cycles and not by HW
>>>>> Note that prepare_tx_pkts() and friends read/write L2/L3 packet headers anyway -
>>>>> so IPv4 header will be in L1 cache already.
>>>> Agreed, but still it will consume some cycles which are more than that
>>>> of HW.
>>>>>>> Disabling HW offloads when they are not really required has 2 benefits:
>>>>>>>      1) allows app to be run on NICs without HW offloads support.
>>>>>>>      2) allows dev_configure() for TX path to select simple/vector TX functions
>>>>>>>          which for many NICs are significantly faster.
>>>>>>>
>>>>>>> Konstantin
>>>>>>>
>>>>>>>> So a NACK for this if my understanding is correct.
>>>>>>>>


  reply	other threads:[~2018-12-24 12:31 UTC|newest]

Thread overview: 132+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2018-11-22 18:49 [dpdk-dev] [PATCH 0/7] examples/ipsec-secgw: make app to use ipsec library Konstantin Ananyev
2018-11-22 18:49 ` [dpdk-dev] [PATCH 1/7] examples/ipsec-secgw: avoid to request unused TX offloads Konstantin Ananyev
2018-11-30 17:04   ` [dpdk-dev] [PATCH v2 0/7] examples/ipsec-secgw: make app to use ipsec library Konstantin Ananyev
2018-11-30 17:04   ` [dpdk-dev] [PATCH v2 1/7] examples/ipsec-secgw: avoid to request unused TX offloads Konstantin Ananyev
2018-12-06 15:54     ` [dpdk-dev] [PATCH v3 0/8] examples/ipsec-secgw: make app to use ipsec library Konstantin Ananyev
2018-12-07 10:01       ` Radu Nicolau
2018-12-14 16:40       ` [dpdk-dev] [PATCH v4 0/9] " Konstantin Ananyev
2018-12-14 16:40       ` [dpdk-dev] [PATCH v4 1/9] examples/ipsec-secgw: avoid to request unused TX offloads Konstantin Ananyev
2018-12-21 13:57         ` Akhil Goyal
2018-12-21 15:58           ` Ananyev, Konstantin
2018-12-24  9:45             ` Akhil Goyal
2018-12-24 10:19               ` Ananyev, Konstantin
2018-12-24 10:54                 ` Akhil Goyal
2018-12-24 10:55                   ` Akhil Goyal
2018-12-24 11:22                   ` Ananyev, Konstantin
2018-12-24 11:24                     ` Akhil Goyal
2018-12-24 11:37                       ` Ananyev, Konstantin
2018-12-24 12:31                         ` Akhil Goyal [this message]
2018-12-28 15:33         ` [dpdk-dev] [PATCH v5 00/10] examples/ipsec-secgw: make app to use ipsec library Konstantin Ananyev
2019-01-02  8:48           ` Akhil Goyal
2018-12-28 15:33         ` [dpdk-dev] [PATCH v5 01/10] examples/ipsec-secgw: allow user to disable some RX/TX offloads Konstantin Ananyev
2019-01-02 13:42           ` Akhil Goyal
2019-01-02 15:29             ` Ananyev, Konstantin
2019-01-03 20:25           ` [dpdk-dev] [PATCH v6 00/10] examples/ipsec-secgw: make app to use ipsec library Konstantin Ananyev
2019-01-04 11:37             ` Akhil Goyal
2019-01-04 12:29               ` Ananyev, Konstantin
2019-01-04 14:40                 ` Akhil Goyal
2019-01-04 15:02                   ` Akhil Goyal
2019-01-04 17:04                   ` Ananyev, Konstantin
2019-01-04 17:38                     ` Akhil Goyal
2019-01-03 20:25           ` [dpdk-dev] [PATCH v6 01/10] examples/ipsec-secgw: allow user to disable some RX/TX offloads Konstantin Ananyev
2019-01-09 11:44             ` [dpdk-dev] [PATCH v7 00/10] examples/ipsec-secgw: make app to use ipsec library Konstantin Ananyev
2019-01-10 15:20               ` Akhil Goyal
2019-01-11  1:08               ` Xu, Yanjie
2019-01-09 11:44             ` [dpdk-dev] [PATCH v7 01/10] examples/ipsec-secgw: allow user to disable some RX/TX offloads Konstantin Ananyev
2019-01-10 21:09               ` [dpdk-dev] [PATCH v8 00/10] examples/ipsec-secgw: make app to use ipsec library Konstantin Ananyev
2019-01-11  0:00                 ` De Lara Guarch, Pablo
2019-01-11  0:16                   ` Ananyev, Konstantin
2019-01-10 21:09               ` [dpdk-dev] [PATCH v8 01/10] examples/ipsec-secgw: allow user to disable some Rx/Tx offloads Konstantin Ananyev
2019-01-10 21:09               ` [dpdk-dev] [PATCH v8 02/10] examples/ipsec-secgw: allow to specify neighbour MAC address Konstantin Ananyev
2019-01-10 21:09               ` [dpdk-dev] [PATCH v8 03/10] examples/ipsec-secgw: fix crypto-op might never get dequeued Konstantin Ananyev
2019-01-10 21:09               ` [dpdk-dev] [PATCH v8 04/10] examples/ipsec-secgw: fix outbound codepath for single SA Konstantin Ananyev
2019-01-10 21:09               ` [dpdk-dev] [PATCH v8 05/10] examples/ipsec-secgw: make local variables static Konstantin Ananyev
2019-01-10 21:09               ` [dpdk-dev] [PATCH v8 06/10] examples/ipsec-secgw: fix inbound SA checking Konstantin Ananyev
2019-01-10 21:09               ` [dpdk-dev] [PATCH v8 07/10] examples/ipsec-secgw: make app to use ipsec library Konstantin Ananyev
2019-01-10 21:09               ` [dpdk-dev] [PATCH v8 08/10] examples/ipsec-secgw: make data-path " Konstantin Ananyev
2019-01-10 21:09               ` [dpdk-dev] [PATCH v8 09/10] examples/ipsec-secgw: add scripts for functional test Konstantin Ananyev
2019-01-10 21:09               ` [dpdk-dev] [PATCH v8 10/10] doc: update ipsec-secgw guide and relelase notes Konstantin Ananyev
2019-01-11  2:49                 ` Varghese, Vipin
2019-01-11  6:56                   ` Akhil Goyal
2019-01-11  8:11                     ` Varghese, Vipin
2019-01-12 23:49                 ` Thomas Monjalon
2019-01-09 11:44             ` [dpdk-dev] [PATCH v7 02/10] examples/ipsec-secgw: allow to specify neighbour mac address Konstantin Ananyev
2019-01-09 11:44             ` [dpdk-dev] [PATCH v7 03/10] examples/ipsec-secgw: fix crypto-op might never get dequeued Konstantin Ananyev
2019-01-09 11:44             ` [dpdk-dev] [PATCH v7 04/10] examples/ipsec-secgw: fix outbound codepath for single SA Konstantin Ananyev
2019-01-09 11:44             ` [dpdk-dev] [PATCH v7 05/10] examples/ipsec-secgw: make local variables static Konstantin Ananyev
2019-01-09 11:44             ` [dpdk-dev] [PATCH v7 06/10] examples/ipsec-secgw: fix inbound SA checking Konstantin Ananyev
2019-01-09 11:44             ` [dpdk-dev] [PATCH v7 07/10] examples/ipsec-secgw: make app to use ipsec library Konstantin Ananyev
2019-01-09 11:44             ` [dpdk-dev] [PATCH v7 08/10] examples/ipsec-secgw: make data-path " Konstantin Ananyev
2019-01-09 11:44             ` [dpdk-dev] [PATCH v7 09/10] examples/ipsec-secgw: add scripts for functional test Konstantin Ananyev
2019-01-09 11:44             ` [dpdk-dev] [PATCH v7 10/10] doc: update ipsec-secgw guide and relelase notes Konstantin Ananyev
2019-01-03 20:25           ` [dpdk-dev] [PATCH v6 02/10] examples/ipsec-secgw: allow to specify neighbour mac address Konstantin Ananyev
2019-01-03 20:25           ` [dpdk-dev] [PATCH v6 03/10] examples/ipsec-secgw: fix crypto-op might never get dequeued Konstantin Ananyev
2019-01-03 20:25           ` [dpdk-dev] [PATCH v6 04/10] examples/ipsec-secgw: fix outbound codepath for single SA Konstantin Ananyev
2019-01-03 20:25           ` [dpdk-dev] [PATCH v6 05/10] examples/ipsec-secgw: make local variables static Konstantin Ananyev
2019-01-03 20:25           ` [dpdk-dev] [PATCH v6 06/10] examples/ipsec-secgw: fix inbound SA checking Konstantin Ananyev
2019-01-03 20:25           ` [dpdk-dev] [PATCH v6 07/10] examples/ipsec-secgw: make app to use ipsec library Konstantin Ananyev
2019-01-03 20:25           ` [dpdk-dev] [PATCH v6 08/10] examples/ipsec-secgw: make data-path " Konstantin Ananyev
2019-01-04 14:58             ` Akhil Goyal
2019-01-04 16:25               ` Ananyev, Konstantin
2019-01-03 20:25           ` [dpdk-dev] [PATCH v6 09/10] examples/ipsec-secgw: add scripts for functional test Konstantin Ananyev
2019-01-03 20:25           ` [dpdk-dev] [PATCH v6 10/10] doc: update ipsec-secgw guide and relelase notes Konstantin Ananyev
2019-01-04  2:42             ` Varghese, Vipin
2018-12-28 15:33         ` [dpdk-dev] [PATCH v5 02/10] examples/ipsec-secgw: allow to specify neighbour mac address Konstantin Ananyev
2018-12-28 15:33         ` [dpdk-dev] [PATCH v5 03/10] examples/ipsec-secgw: fix crypto-op might never get dequeued Konstantin Ananyev
2019-01-02 11:44           ` Akhil Goyal
2019-01-02 13:43             ` Ananyev, Konstantin
2019-01-02 13:50               ` Akhil Goyal
2019-01-02 15:06                 ` Ananyev, Konstantin
2019-01-03 20:36                 ` Ananyev, Konstantin
2018-12-28 15:33         ` [dpdk-dev] [PATCH v5 04/10] examples/ipsec-secgw: fix outbound codepath for single SA Konstantin Ananyev
2018-12-28 15:33         ` [dpdk-dev] [PATCH v5 05/10] examples/ipsec-secgw: make local variables static Konstantin Ananyev
2018-12-28 15:33         ` [dpdk-dev] [PATCH v5 06/10] examples/ipsec-secgw: fix inbound SA checking Konstantin Ananyev
2018-12-28 15:33         ` [dpdk-dev] [PATCH v5 07/10] examples/ipsec-secgw: make app to use ipsec library Konstantin Ananyev
2018-12-28 15:33         ` [dpdk-dev] [PATCH v5 08/10] examples/ipsec-secgw: make data-path " Konstantin Ananyev
2018-12-28 15:33         ` [dpdk-dev] [PATCH v5 09/10] examples/ipsec-secgw: add scripts for functional test Konstantin Ananyev
2018-12-28 15:33         ` [dpdk-dev] [PATCH v5 10/10] doc: update ipsec-secgw guide and relelase notes Konstantin Ananyev
2018-12-14 16:40       ` [dpdk-dev] [PATCH v4 2/9] examples/ipsec-secgw: allow to specify neighbor mac address Konstantin Ananyev
2018-12-21 14:05         ` Akhil Goyal
2018-12-14 16:40       ` [dpdk-dev] [PATCH v4 3/9] examples/ipsec-secgw: fix crypto-op might never get dequeued Konstantin Ananyev
2018-12-21 14:12         ` Akhil Goyal
2018-12-21 14:49           ` Ananyev, Konstantin
2018-12-21 14:57             ` Akhil Goyal
2018-12-21 15:01               ` Ananyev, Konstantin
2018-12-14 16:40       ` [dpdk-dev] [PATCH v4 4/9] examples/ipsec-secgw: fix outbound codepath for single SA Konstantin Ananyev
2018-12-21 14:25         ` Akhil Goyal
2018-12-21 14:54           ` Ananyev, Konstantin
2018-12-14 16:40       ` [dpdk-dev] [PATCH v4 5/9] examples/ipsec-secgw: make local variables static Konstantin Ananyev
2018-12-14 16:40       ` [dpdk-dev] [PATCH v4 6/9] examples/ipsec-secgw: make app to use ipsec library Konstantin Ananyev
2018-12-21 15:15         ` Akhil Goyal
2018-12-24 12:29           ` Ananyev, Konstantin
2018-12-24 12:32             ` Akhil Goyal
2018-12-24 12:37               ` Ananyev, Konstantin
2018-12-24 13:21                 ` Ananyev, Konstantin
2018-12-24 13:50                   ` Akhil Goyal
2018-12-24 15:01                     ` Ananyev, Konstantin
2018-12-26  9:02                       ` Akhil Goyal
2018-12-27 11:06                         ` Ananyev, Konstantin
2018-12-14 16:40       ` [dpdk-dev] [PATCH v4 7/9] examples/ipsec-secgw: make data-path " Konstantin Ananyev
2018-12-21 15:23         ` Akhil Goyal
2018-12-14 16:40       ` [dpdk-dev] [PATCH v4 8/9] examples/ipsec-secgw: add scripts for functional test Konstantin Ananyev
2018-12-14 16:40       ` [dpdk-dev] [PATCH v4 9/9] doc: update ipsec-secgw guide and relelase notes Konstantin Ananyev
2018-12-06 15:54     ` [dpdk-dev] [PATCH v3 1/8] examples/ipsec-secgw: avoid to request unused TX offloads Konstantin Ananyev
2018-12-06 15:54     ` [dpdk-dev] [PATCH v3 2/8] examples/ipsec-secgw: allow to specify neighbor mac address Konstantin Ananyev
2018-12-06 15:54     ` [dpdk-dev] [PATCH v3 3/8] examples/ipsec-secgw: fix crypto-op might never get dequeued Konstantin Ananyev
2018-12-06 15:54     ` [dpdk-dev] [PATCH v3 4/8] examples/ipsec-secgw: fix outbound codepath for single SA Konstantin Ananyev
2018-12-06 15:54     ` [dpdk-dev] [PATCH v3 5/8] examples/ipsec-secgw: make local variables static Konstantin Ananyev
2018-12-06 15:54     ` [dpdk-dev] [PATCH v3 6/8] examples/ipsec-secgw: make app to use ipsec library Konstantin Ananyev
2018-12-06 15:54     ` [dpdk-dev] [PATCH v3 7/8] examples/ipsec-secgw: make data-path " Konstantin Ananyev
2018-12-06 15:54     ` [dpdk-dev] [PATCH v3 8/8] examples/ipsec-secgw: add scripts for functional test Konstantin Ananyev
2018-11-30 17:04   ` [dpdk-dev] [PATCH v2 2/7] examples/ipsec-secgw: allow to specify neighbor mac address Konstantin Ananyev
2018-11-30 17:04   ` [dpdk-dev] [PATCH v2 3/7] examples/ipsec-secgw: fix crypto-op might never get dequeued Konstantin Ananyev
2018-11-30 17:04   ` [dpdk-dev] [PATCH v2 4/7] examples/ipsec-secgw: fix outbound codepath for single SA Konstantin Ananyev
2018-11-30 17:04   ` [dpdk-dev] [PATCH v2 5/7] examples/ipsec-secgw: make app to use ipsec library Konstantin Ananyev
2018-11-30 17:04   ` [dpdk-dev] [PATCH v2 6/7] examples/ipsec-secgw: make data-path " Konstantin Ananyev
2018-11-30 17:04   ` [dpdk-dev] [PATCH v2 7/7] examples/ipsec-secgw: add scripts for functional test Konstantin Ananyev
2018-11-22 18:49 ` [dpdk-dev] [PATCH 2/7] examples/ipsec-secgw: allow to specify neighbor mac address Konstantin Ananyev
2018-11-22 18:49 ` [dpdk-dev] [PATCH 3/7] examples/ipsec-secgw: fix crypto-op might never get dequeued Konstantin Ananyev
2018-11-22 18:49 ` [dpdk-dev] [PATCH 4/7] examples/ipsec-secgw: fix outbound codepath for single SA Konstantin Ananyev
2018-11-22 18:49 ` [dpdk-dev] [PATCH 5/7] examples/ipsec-secgw: make app to use ipsec library Konstantin Ananyev
2018-11-22 18:49 ` [dpdk-dev] [PATCH 6/7] examples/ipsec-secgw: make data-path " Konstantin Ananyev
2018-11-22 18:49 ` [dpdk-dev] [PATCH 7/7] examples/ipsec-secgw: add scripts for functional test Konstantin Ananyev

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=0cc8fd94-8530-9ce0-e03b-1e37d65ecb25@nxp.com \
    --to=akhil.goyal@nxp.com \
    --cc=dev@dpdk.org \
    --cc=konstantin.ananyev@intel.com \
    --cc=radu.nicolau@intel.com \
    --cc=remy.horton@intel.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).