From: "Lu, Wenzhuo" <wenzhuo.lu@intel.com>
To: Maxime Coquelin <maxime.coquelin@redhat.com>,
"dev@dpdk.org" <dev@dpdk.org>
Subject: Re: [dpdk-dev] [PATCH v5 6/8] net/ice: support Rx AVX2 vector
Date: Thu, 28 Mar 2019 01:56:13 +0000 [thread overview]
Message-ID: <6A0DE07E22DDAD4C9103DF62FEBC0909407F15A0@shsmsx102.ccr.corp.intel.com> (raw)
In-Reply-To: <b9989400-d38d-c4a8-10e1-514304f25114@redhat.com>
Hi Maxime,
> -----Original Message-----
> From: Maxime Coquelin [mailto:maxime.coquelin@redhat.com]
> Sent: Wednesday, March 27, 2019 3:50 PM
> To: Lu, Wenzhuo <wenzhuo.lu@intel.com>; dev@dpdk.org
> Subject: Re: [dpdk-dev] [PATCH v5 6/8] net/ice: support Rx AVX2 vector
>
>
>
> On 3/27/19 1:56 AM, Lu, Wenzhuo wrote:
> > Hi Maxime,
> >
> >> -----Original Message-----
> >> From: Maxime Coquelin [mailto:maxime.coquelin@redhat.com]
> >> Sent: Tuesday, March 26, 2019 5:29 PM
> >> To: Lu, Wenzhuo <wenzhuo.lu@intel.com>; dev@dpdk.org
> >> Subject: Re: [dpdk-dev] [PATCH v5 6/8] net/ice: support Rx AVX2
> >> vector
> >>
> >> Hi,
> >>
> >> On 3/26/19 2:00 AM, Lu, Wenzhuo wrote:
> >>> Hi Maxime,
> >>>
> >>>> -----Original Message-----
> >>>> From: Maxime Coquelin [mailto:maxime.coquelin@redhat.com]
> >>>> Sent: Monday, March 25, 2019 4:26 PM
> >>>> To: Lu, Wenzhuo <wenzhuo.lu@intel.com>; dev@dpdk.org
> >>>> Subject: Re: [dpdk-dev] [PATCH v5 6/8] net/ice: support Rx AVX2
> >>>> vector
> >>>>
> >>>> Hi,
> >>>>
> >>>> On 3/25/19 3:22 AM, Lu, Wenzhuo wrote:
> >>>>> Hi Maxime,
> >>>>>
> >>>>>
> >>>>>> -----Original Message-----
> >>>>>> From: Maxime Coquelin [mailto:maxime.coquelin@redhat.com]
> >>>>>> Sent: Friday, March 22, 2019 6:12 PM
> >>>>>> To: Lu, Wenzhuo <wenzhuo.lu@intel.com>; dev@dpdk.org
> >>>>>> Subject: Re: [dpdk-dev] [PATCH v5 6/8] net/ice: support Rx AVX2
> >>>>>> vector
> >>>>>
> >>>>>
> >>>>>>> +#ifndef RTE_LIBRTE_ICE_16BYTE_RX_DESC
> >>>>>>
> >>>>>> I see same is done for other Intel NICs, but I wonder what would
> >>>>>> be the performance cost of making it dynamic, if any cost?
> >>>>> Currently we don't have a good idea to make it dynamic. If we use
> >>>>> pointer
> >>>> to point to different functions for 16 byte and 32 byte, there's
> >>>> too much duplicate code to make it hard to maintain. If we use the
> >>>> same function, and check the configure in it. It impacts the
> performance.
> >>>>
> >>>> Have you done some measurements, what would be the performance
> >>>> impact?
> >>> I mean if we check the configuration is 16 byte or 32 byte, this
> >>> check will
> >> consume extra CPU cycles.
> >>> That why I think the better way is to have different paths for 16
> >>> byte and
> >> 32 byte. We should choose the appropriate path at the beginning.
> >>>
> >>>>
> >>>>> As HW does not support to change the configuration dynamically.
> >>>>> The
> >>>> device must be stopped and restarted if the configuration is changed.
> >>>> It's not very helpful to make it a dynamic configuration. We assume
> >>>> that the users can make their choice at the beginning and will not
> >>>> change
> >> it.
> >>>>
> >>>> The problem is that the user has to recompile to switch between the
> >>>> two configurations. And it may not be an option for the user if he
> >>>> uses dpdk packaged by a distribution, for example.
> >>>>
> >>>> Maybe I was not clear, but I don't mean to be able to switch mode
> >>>> while the port is started. I think it would be better to make it
> >>>> possible to switch mode at application startup time.
> >>> Yes, I understand the problem is the recompiling. But we think the
> >>> users
> >> will not change it after they made decision. That's why's acceptable
> >> in previous drivers.
> >>
> >> The problem is that the user may not be able to change it, if he does
> >> not get DPDK from source but from a distribution like Debian, Ubuntu or
> Red Hat.
> >>
> >> In this case, it means the user has no choice than sticking to 32
> >> bytes descriptors.
> > Normally using 32 bytes is the default behavior and it's good to do that.
> > But I have to say I don't quite understand the scenario. DPDK is open
> source, whatever OS that users are using, nothing prevents them going to
> dpdk website to get the code and customize it.
>
> The user may prefer to use the distribution package for several reasons.
> Like not loosing the support he pays to the distributor by recompiling the
> package, or also not benefiting from the validation done by the distributor
> on the pre-built package.
Thanks for sharing the info about the deployment. Good to know that. How to say I'm always looking at the problem from the developer's point of view.
>
> For example, would it make sense to fix the queue size at build time instead
> of using the --txd/--rxd run-time paramaters to save a few cycles here and
> there? I think not.
Agree, we have to balance them if we cannot make it perfect. We'll think about how to optimize it.
next prev parent reply other threads:[~2019-03-28 1:56 UTC|newest]
Thread overview: 219+ messages / expand[flat|nested] mbox.gz Atom feed top
2019-02-28 7:48 [dpdk-dev] [PATCH 0/8] Support vector instructions on ICE Wenzhuo Lu
2019-02-28 7:48 ` [dpdk-dev] [PATCH 1/8] net/ice: fix TX function setting Wenzhuo Lu
2019-02-28 7:48 ` [dpdk-dev] [PATCH 2/8] net/ice: add pointer for queue buffer release Wenzhuo Lu
2019-02-28 7:48 ` [dpdk-dev] [PATCH 3/8] net/ice: support RX SSE vector Wenzhuo Lu
2019-03-01 3:44 ` Zhang, Qi Z
2019-03-04 1:27 ` Lu, Wenzhuo
2019-02-28 7:48 ` [dpdk-dev] [PATCH 4/8] net/ice: support RX scatter " Wenzhuo Lu
2019-02-28 7:48 ` [dpdk-dev] [PATCH 5/8] net/ice: support TX " Wenzhuo Lu
2019-02-28 7:48 ` [dpdk-dev] [PATCH 6/8] net/ice: support RX AVX2 vector Wenzhuo Lu
2019-02-28 7:48 ` [dpdk-dev] [PATCH 7/8] net/ice: support RX scatter " Wenzhuo Lu
2019-02-28 7:48 ` [dpdk-dev] [PATCH 8/8] net/ice: support TX " Wenzhuo Lu
2019-03-01 3:41 ` [dpdk-dev] [PATCH 0/8] Support vector instructions on ICE Zhang, Qi Z
2019-03-04 1:24 ` Lu, Wenzhuo
2019-03-04 6:53 ` [dpdk-dev] [PATCH v2 " Wenzhuo Lu
2019-03-04 6:53 ` [dpdk-dev] [PATCH v2 1/8] net/ice: fix Tx function setting Wenzhuo Lu
2019-03-04 6:53 ` [dpdk-dev] [PATCH v2 2/8] net/ice: add pointer for queue buffer release Wenzhuo Lu
2019-03-04 6:53 ` [dpdk-dev] [PATCH v2 3/8] net/ice: support vector SSE in RX Wenzhuo Lu
2019-03-11 3:26 ` Zhang, Qi Z
2019-03-15 1:50 ` Lu, Wenzhuo
2019-03-15 1:50 ` Lu, Wenzhuo
2019-03-04 6:53 ` [dpdk-dev] [PATCH v2 4/8] net/ice: support Rx scatter SSE vector Wenzhuo Lu
2019-03-04 6:53 ` [dpdk-dev] [PATCH v2 5/8] net/ice: support Tx " Wenzhuo Lu
2019-03-04 6:53 ` [dpdk-dev] [PATCH v2 6/8] net/ice: support Rx AVX2 vector Wenzhuo Lu
2019-03-04 6:53 ` [dpdk-dev] [PATCH v2 7/8] net/ice: support Rx scatter " Wenzhuo Lu
2019-03-04 6:53 ` [dpdk-dev] [PATCH v2 8/8] net/ice: support vector AVX2 in TX Wenzhuo Lu
2019-03-15 6:22 ` [dpdk-dev] [PATCH v3 0/8] Support vector instructions on ICE Wenzhuo Lu
2019-03-15 6:22 ` Wenzhuo Lu
2019-03-15 6:22 ` [dpdk-dev] [PATCH v3 1/8] net/ice: fix Tx function setting Wenzhuo Lu
2019-03-15 6:22 ` Wenzhuo Lu
2019-03-15 17:52 ` Ferruh Yigit
2019-03-15 17:52 ` Ferruh Yigit
2019-03-18 1:08 ` Lu, Wenzhuo
2019-03-18 1:08 ` Lu, Wenzhuo
2019-03-20 17:22 ` Ferruh Yigit
2019-03-20 17:22 ` Ferruh Yigit
2019-03-21 2:29 ` Lu, Wenzhuo
2019-03-21 2:29 ` Lu, Wenzhuo
2019-03-15 6:22 ` [dpdk-dev] [PATCH v3 2/8] net/ice: add pointer for queue buffer release Wenzhuo Lu
2019-03-15 6:22 ` Wenzhuo Lu
2019-03-15 17:52 ` Ferruh Yigit
2019-03-15 17:52 ` Ferruh Yigit
2019-03-18 1:15 ` Lu, Wenzhuo
2019-03-18 1:15 ` Lu, Wenzhuo
2019-03-15 6:22 ` [dpdk-dev] [PATCH v3 3/8] net/ice: support vector SSE in RX Wenzhuo Lu
2019-03-15 6:22 ` Wenzhuo Lu
2019-03-15 17:53 ` Ferruh Yigit
2019-03-15 17:53 ` Ferruh Yigit
2019-03-18 1:22 ` Lu, Wenzhuo
2019-03-18 1:22 ` Lu, Wenzhuo
2019-03-20 17:35 ` Ferruh Yigit
2019-03-20 17:35 ` Ferruh Yigit
2019-03-21 2:48 ` Lu, Wenzhuo
2019-03-21 2:48 ` Lu, Wenzhuo
2019-03-15 6:22 ` [dpdk-dev] [PATCH v3 4/8] net/ice: support Rx scatter SSE vector Wenzhuo Lu
2019-03-15 6:22 ` Wenzhuo Lu
2019-03-15 6:22 ` [dpdk-dev] [PATCH v3 5/8] net/ice: support Tx " Wenzhuo Lu
2019-03-15 6:22 ` Wenzhuo Lu
2019-03-15 6:22 ` [dpdk-dev] [PATCH v3 6/8] net/ice: support Rx AVX2 vector Wenzhuo Lu
2019-03-15 6:22 ` Wenzhuo Lu
2019-03-15 17:54 ` Ferruh Yigit
2019-03-15 17:54 ` Ferruh Yigit
2019-03-18 1:37 ` Lu, Wenzhuo
2019-03-18 1:37 ` Lu, Wenzhuo
2019-03-20 17:37 ` Ferruh Yigit
2019-03-20 17:37 ` Ferruh Yigit
2019-03-21 2:31 ` Lu, Wenzhuo
2019-03-21 2:31 ` Lu, Wenzhuo
2019-03-15 6:22 ` [dpdk-dev] [PATCH v3 7/8] net/ice: support Rx scatter " Wenzhuo Lu
2019-03-15 6:22 ` Wenzhuo Lu
2019-03-15 6:22 ` [dpdk-dev] [PATCH v3 8/8] net/ice: support vector AVX2 in TX Wenzhuo Lu
2019-03-15 6:22 ` Wenzhuo Lu
2019-03-15 17:54 ` Ferruh Yigit
2019-03-15 17:54 ` Ferruh Yigit
2019-03-18 1:38 ` Lu, Wenzhuo
2019-03-18 1:38 ` Lu, Wenzhuo
2019-03-15 8:08 ` [dpdk-dev] [PATCH v3 0/8] Support vector instructions on ICE Zhang, Qi Z
2019-03-15 8:08 ` Zhang, Qi Z
2019-03-21 6:26 ` [dpdk-dev] [PATCH v4 " Wenzhuo Lu
2019-03-21 6:26 ` Wenzhuo Lu
2019-03-21 6:26 ` [dpdk-dev] [PATCH v4 1/8] net/ice: fix Tx function setting Wenzhuo Lu
2019-03-21 6:26 ` Wenzhuo Lu
2019-03-22 8:46 ` Maxime Coquelin
2019-03-22 8:46 ` Maxime Coquelin
2019-03-22 9:01 ` Maxime Coquelin
2019-03-22 9:01 ` Maxime Coquelin
2019-03-21 6:26 ` [dpdk-dev] [PATCH v4 2/8] net/ice: add pointer for queue buffer release Wenzhuo Lu
2019-03-21 6:26 ` Wenzhuo Lu
2019-03-22 8:59 ` Maxime Coquelin
2019-03-22 8:59 ` Maxime Coquelin
2019-03-21 6:26 ` [dpdk-dev] [PATCH v4 3/8] net/ice: support vector SSE in RX Wenzhuo Lu
2019-03-21 6:26 ` Wenzhuo Lu
2019-03-21 19:02 ` Ferruh Yigit
2019-03-21 19:02 ` Ferruh Yigit
2019-03-22 1:46 ` Lu, Wenzhuo
2019-03-22 1:46 ` Lu, Wenzhuo
2019-03-21 6:26 ` [dpdk-dev] [PATCH v4 4/8] net/ice: support Rx scatter SSE vector Wenzhuo Lu
2019-03-21 6:26 ` Wenzhuo Lu
2019-03-21 6:26 ` [dpdk-dev] [PATCH v4 5/8] net/ice: support Tx " Wenzhuo Lu
2019-03-21 6:26 ` Wenzhuo Lu
2019-03-21 6:26 ` [dpdk-dev] [PATCH v4 6/8] net/ice: support Rx AVX2 vector Wenzhuo Lu
2019-03-21 6:26 ` Wenzhuo Lu
2019-03-21 6:26 ` [dpdk-dev] [PATCH v4 7/8] net/ice: support Rx scatter " Wenzhuo Lu
2019-03-21 6:26 ` Wenzhuo Lu
2019-03-21 6:26 ` [dpdk-dev] [PATCH v4 8/8] net/ice: support vector AVX2 in TX Wenzhuo Lu
2019-03-21 6:26 ` Wenzhuo Lu
2019-03-21 19:20 ` Ferruh Yigit
2019-03-21 19:20 ` Ferruh Yigit
2019-03-22 1:45 ` Lu, Wenzhuo
2019-03-22 1:45 ` Lu, Wenzhuo
2019-03-22 2:58 ` [dpdk-dev] [PATCH v5 0/8] Support vector instructions on ICE Wenzhuo Lu
2019-03-22 2:58 ` Wenzhuo Lu
2019-03-22 2:58 ` [dpdk-dev] [PATCH v5 1/8] net/ice: fix Tx function setting Wenzhuo Lu
2019-03-22 2:58 ` Wenzhuo Lu
2019-03-22 2:58 ` [dpdk-dev] [PATCH v5 2/8] net/ice: add pointer for queue buffer release Wenzhuo Lu
2019-03-22 2:58 ` Wenzhuo Lu
2019-03-22 2:58 ` [dpdk-dev] [PATCH v5 3/8] net/ice: support vector SSE in RX Wenzhuo Lu
2019-03-22 2:58 ` Wenzhuo Lu
2019-03-22 9:42 ` Maxime Coquelin
2019-03-22 9:42 ` Maxime Coquelin
2019-03-25 1:56 ` Lu, Wenzhuo
2019-03-25 1:56 ` Lu, Wenzhuo
2019-03-22 2:58 ` [dpdk-dev] [PATCH v5 4/8] net/ice: support Rx scatter SSE vector Wenzhuo Lu
2019-03-22 2:58 ` Wenzhuo Lu
2019-03-22 2:58 ` [dpdk-dev] [PATCH v5 5/8] net/ice: support Tx " Wenzhuo Lu
2019-03-22 2:58 ` Wenzhuo Lu
2019-03-22 9:58 ` Maxime Coquelin
2019-03-22 9:58 ` Maxime Coquelin
2019-03-25 2:02 ` Lu, Wenzhuo
2019-03-25 2:02 ` Lu, Wenzhuo
2019-03-22 2:58 ` [dpdk-dev] [PATCH v5 6/8] net/ice: support Rx AVX2 vector Wenzhuo Lu
2019-03-22 2:58 ` Wenzhuo Lu
2019-03-22 10:12 ` Maxime Coquelin
2019-03-22 10:12 ` Maxime Coquelin
2019-03-25 2:22 ` Lu, Wenzhuo
2019-03-25 2:22 ` Lu, Wenzhuo
2019-03-25 8:26 ` Maxime Coquelin
2019-03-25 8:26 ` Maxime Coquelin
2019-03-26 1:00 ` Lu, Wenzhuo
2019-03-26 1:00 ` Lu, Wenzhuo
2019-03-26 9:28 ` Maxime Coquelin
2019-03-26 9:28 ` Maxime Coquelin
2019-03-27 0:56 ` Lu, Wenzhuo
2019-03-27 0:56 ` Lu, Wenzhuo
2019-03-27 7:50 ` Maxime Coquelin
2019-03-27 7:50 ` Maxime Coquelin
2019-03-28 1:56 ` Lu, Wenzhuo [this message]
2019-03-28 1:56 ` Lu, Wenzhuo
2019-03-22 2:58 ` [dpdk-dev] [PATCH v5 7/8] net/ice: support Rx scatter " Wenzhuo Lu
2019-03-22 2:58 ` Wenzhuo Lu
2019-03-22 2:58 ` [dpdk-dev] [PATCH v5 8/8] net/ice: support vector AVX2 in TX Wenzhuo Lu
2019-03-22 2:58 ` Wenzhuo Lu
2019-03-25 6:06 ` [dpdk-dev] [PATCH v6 0/8] Support vector instructions on ICE Wenzhuo Lu
2019-03-25 6:06 ` Wenzhuo Lu
2019-03-25 6:06 ` [dpdk-dev] [PATCH v6 1/8] net/ice: fix Tx function setting Wenzhuo Lu
2019-03-25 6:06 ` Wenzhuo Lu
2019-03-25 6:06 ` [dpdk-dev] [PATCH v6 2/8] net/ice: add pointer for queue buffer release Wenzhuo Lu
2019-03-25 6:06 ` Wenzhuo Lu
2019-03-25 13:23 ` Maxime Coquelin
2019-03-25 13:23 ` Maxime Coquelin
2019-03-26 1:15 ` Lu, Wenzhuo
2019-03-26 1:15 ` Lu, Wenzhuo
2019-03-25 6:06 ` [dpdk-dev] [PATCH v6 3/8] net/ice: support vector SSE in RX Wenzhuo Lu
2019-03-25 6:06 ` Wenzhuo Lu
2019-03-25 6:06 ` [dpdk-dev] [PATCH v6 4/8] net/ice: support Rx scatter SSE vector Wenzhuo Lu
2019-03-25 6:06 ` Wenzhuo Lu
2019-03-25 6:06 ` [dpdk-dev] [PATCH v6 5/8] net/ice: support Tx " Wenzhuo Lu
2019-03-25 6:06 ` Wenzhuo Lu
2019-03-25 6:06 ` [dpdk-dev] [PATCH v6 6/8] net/ice: support Rx AVX2 vector Wenzhuo Lu
2019-03-25 6:06 ` Wenzhuo Lu
2019-03-25 6:06 ` [dpdk-dev] [PATCH v6 7/8] net/ice: support Rx scatter " Wenzhuo Lu
2019-03-25 6:06 ` Wenzhuo Lu
2019-03-25 6:06 ` [dpdk-dev] [PATCH v6 8/8] net/ice: support vector AVX2 in TX Wenzhuo Lu
2019-03-25 6:06 ` Wenzhuo Lu
2019-03-25 7:56 ` [dpdk-dev] [PATCH v6 0/8] Support vector instructions on ICE Zhang, Qi Z
2019-03-25 7:56 ` Zhang, Qi Z
2019-03-26 6:16 ` [dpdk-dev] [PATCH v7 " Wenzhuo Lu
2019-03-26 6:16 ` Wenzhuo Lu
2019-03-26 6:16 ` [dpdk-dev] [PATCH v7 1/8] net/ice: fix Tx function setting Wenzhuo Lu
2019-03-26 6:16 ` Wenzhuo Lu
2019-03-26 6:16 ` [dpdk-dev] [PATCH v7 2/8] net/ice: add pointer for queue buffer release Wenzhuo Lu
2019-03-26 6:16 ` Wenzhuo Lu
2019-03-26 6:16 ` [dpdk-dev] [PATCH v7 3/8] net/ice: support vector SSE in RX Wenzhuo Lu
2019-03-26 6:16 ` Wenzhuo Lu
2019-03-26 6:16 ` [dpdk-dev] [PATCH v7 4/8] net/ice: support Rx scatter SSE vector Wenzhuo Lu
2019-03-26 6:16 ` Wenzhuo Lu
2019-03-26 6:16 ` [dpdk-dev] [PATCH v7 5/8] net/ice: support Tx " Wenzhuo Lu
2019-03-26 6:16 ` Wenzhuo Lu
2019-03-26 6:16 ` [dpdk-dev] [PATCH v7 6/8] net/ice: support Rx AVX2 vector Wenzhuo Lu
2019-03-26 6:16 ` Wenzhuo Lu
2019-03-26 6:16 ` [dpdk-dev] [PATCH v7 7/8] net/ice: support Rx scatter " Wenzhuo Lu
2019-03-26 6:16 ` Wenzhuo Lu
2019-03-26 6:16 ` [dpdk-dev] [PATCH v7 8/8] net/ice: support vector AVX2 in TX Wenzhuo Lu
2019-03-26 6:16 ` Wenzhuo Lu
2019-03-26 9:50 ` [dpdk-dev] [PATCH v7 0/8] Support vector instructions on ICE Ferruh Yigit
2019-03-26 9:50 ` Ferruh Yigit
2019-03-31 15:52 ` Thomas Monjalon
2019-03-31 15:52 ` Thomas Monjalon
2019-04-01 5:46 ` Lu, Wenzhuo
2019-04-01 5:46 ` Lu, Wenzhuo
2019-04-01 12:51 ` Ferruh Yigit
2019-04-01 12:51 ` Ferruh Yigit
2019-04-01 13:27 ` Thomas Monjalon
2019-04-01 13:27 ` Thomas Monjalon
2019-04-01 15:12 ` Ferruh Yigit
2019-04-01 15:12 ` Ferruh Yigit
2019-04-01 15:14 ` Thomas Monjalon
2019-04-01 15:14 ` Thomas Monjalon
2019-04-02 1:01 ` Lu, Wenzhuo
2019-04-02 1:01 ` Lu, Wenzhuo
2019-04-02 7:12 ` Thomas Monjalon
2019-04-02 7:12 ` Thomas Monjalon
2019-04-01 14:39 ` Bruce Richardson
2019-04-01 14:39 ` Bruce Richardson
2019-04-01 14:56 ` Ferruh Yigit
2019-04-01 14:56 ` Ferruh Yigit
2019-04-01 15:09 ` Ferruh Yigit
2019-04-01 15:09 ` Ferruh Yigit
2019-04-01 15:13 ` Thomas Monjalon
2019-04-01 15:13 ` Thomas Monjalon
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=6A0DE07E22DDAD4C9103DF62FEBC0909407F15A0@shsmsx102.ccr.corp.intel.com \
--to=wenzhuo.lu@intel.com \
--cc=dev@dpdk.org \
--cc=maxime.coquelin@redhat.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).