DPDK patches and discussions
 help / color / mirror / Atom feed
From: Jerin Jacob <jerinjacobk@gmail.com>
To: Slava Ovsiienko <viacheslavo@nvidia.com>
Cc: dpdk-dev <dev@dpdk.org>,
	NBU-Contact-Thomas Monjalon <thomas@monjalon.net>,
	Stephen Hemminger <stephen@networkplumber.org>,
	Ferruh Yigit <ferruh.yigit@intel.com>,
	 Olivier Matz <olivier.matz@6wind.com>,
	Maxime Coquelin <maxime.coquelin@redhat.com>,
	 David Marchand <david.marchand@redhat.com>,
	Andrew Rybchenko <arybchenko@solarflare.com>
Subject: Re: [dpdk-dev] [PATCH v6 1/6] ethdev: introduce Rx buffer split
Date: Thu, 15 Oct 2020 14:57:30 +0530	[thread overview]
Message-ID: <CALBAE1MHs077qDT3FomyVjqNUEc1ddmxxh4QEW4=OUvNtr-O8w@mail.gmail.com> (raw)
In-Reply-To: <MWHPR12MB1360EECBBCBF64E432D89807DF020@MWHPR12MB1360.namprd12.prod.outlook.com>

On Thu, Oct 15, 2020 at 1:13 PM Slava Ovsiienko <viacheslavo@nvidia.com> wrote:
>
> Hi, Jerin

Hi Slava,

>
> > -----Original Message-----
> > From: Jerin Jacob <jerinjacobk@gmail.com>
> > Sent: Wednesday, October 14, 2020 21:57
> > To: Slava Ovsiienko <viacheslavo@nvidia.com>
> > Cc: dpdk-dev <dev@dpdk.org>; NBU-Contact-Thomas Monjalon
> > <thomas@monjalon.net>; Stephen Hemminger
> > <stephen@networkplumber.org>; Ferruh Yigit <ferruh.yigit@intel.com>;
> > Olivier Matz <olivier.matz@6wind.com>; Maxime Coquelin
> > <maxime.coquelin@redhat.com>; David Marchand
> > <david.marchand@redhat.com>; Andrew Rybchenko
> > <arybchenko@solarflare.com>
> > Subject: Re: [PATCH v6 1/6] ethdev: introduce Rx buffer split
> >
> > On Wed, Oct 14, 2020 at 11:42 PM Viacheslav Ovsiienko
> > <viacheslavo@nvidia.com> wrote:
> > >
> > > The DPDK datapath in the transmit direction is very flexible.
> > > An application can build the multi-segment packet and manages almost
> > > all data aspects - the memory pools where segments are allocated from,
> > > the segment lengths, the memory attributes like external buffers,
> > > registered for DMA, etc.
> > >
>
> [..snip..]
>
> > > For example, let's suppose we configured the Rx queue with the
> > > following segments:
> > >     seg0 - pool0, len0=14B, off0=2
> > >     seg1 - pool1, len1=20B, off1=128B
> > >     seg2 - pool2, len2=20B, off2=0B
> > >     seg3 - pool3, len3=512B, off3=0B
> >
> >
> > Sorry for chime in late. This API lookout looks good to me.
> > But, I am wondering how the application can know the capability or "limits" of
> > struct rte_eth_rxseg structure for the specific PMD. The other descriptor limit,
> > it's being exposed with struct rte_eth_dev_info::rx_desc_lim; If PMD can
> > support a specific pattern rather than returning the blanket error, the
> > application should know the limit.
> > IMO, it is better to add
> > struct rte_eth_rxseg *rxsegs;
> > unint16_t nb_max_rxsegs
> > in rte_eth_dev_info structure to express the capablity.
> > Where the en and offset can define the max offset.
> >
> > Thoughts?
>
> Moreover, there might be implied a lot of various limitations - offsets might be not supported at all or
> have some requirements for alignment, the similar requirements might be applied to segment size
> (say, ask for some granularity). Currently it is not obvious how to report all nuances, and it is supposed
> the limitations of this kind must be documented in PMD chapter. As for mlx5 - it has no special
> limitations besides common requirements to the regular segments.

Reporting the limitation in the documentation will not help for the
generic applications.

>
> One more point - the split feature might be considered as just one of possible cases of using
> these segment descriptions, other features might impose other (unknown for now) limitations.
> If we see some of the features of such kind or other PMDs adopts the split feature - we'll try to find
> the common root and consider the way how to report it.

My only concern with that approach will be ABI break again if
something needs to exposed over rte_eth_dev_info().
IMO, if we featured needs to completed only when its capabilities are
exposed in a programmatic manner.
As of mlx5, if there not limitation then info
rte_eth_dev_info::rxsegs[x].len, offset etc as UINT16_MAX so
that application is aware of the state.

>
> With best regards, Slava
>

  reply	other threads:[~2020-10-15  9:27 UTC|newest]

Thread overview: 172+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2020-08-17 17:49 [dpdk-dev] [RFC] " Slava Ovsiienko
2020-09-17 16:55 ` Andrew Rybchenko
2020-10-01  8:54   ` Slava Ovsiienko
2020-10-12  8:45     ` Andrew Rybchenko
2020-10-12  9:56       ` Slava Ovsiienko
2020-10-12 15:14         ` Thomas Monjalon
2020-10-12 15:28           ` Ananyev, Konstantin
2020-10-12 15:34             ` Slava Ovsiienko
2020-10-12 15:56               ` Ananyev, Konstantin
2020-10-12 15:59                 ` Slava Ovsiienko
2020-10-12 16:52                 ` Thomas Monjalon
2020-10-12 16:03           ` Andrew Rybchenko
2020-10-12 16:10             ` Slava Ovsiienko
2020-10-13 21:59         ` Ferruh Yigit
2020-10-14  7:17           ` Thomas Monjalon
2020-10-14  7:37           ` Slava Ovsiienko
2020-10-05  6:26 ` [dpdk-dev] [PATCH 0/5] " Viacheslav Ovsiienko
2020-10-05  6:26   ` [dpdk-dev] [PATCH 1/5] " Viacheslav Ovsiienko
2020-10-05  6:26   ` [dpdk-dev] [PATCH 2/5] app/testpmd: add multiple pools per core creation Viacheslav Ovsiienko
2020-10-05  6:26   ` [dpdk-dev] [PATCH 3/5] app/testpmd: add buffer split offload configuration Viacheslav Ovsiienko
2020-10-05  6:26   ` [dpdk-dev] [PATCH 4/5] app/testpmd: add rxpkts commands and parameters Viacheslav Ovsiienko
2020-10-05  6:26   ` [dpdk-dev] [PATCH 5/5] app/testpmd: add extended Rx queue setup Viacheslav Ovsiienko
2020-10-07 15:06 ` [dpdk-dev] [PATCH v2 0/9] ethdev: introduce Rx buffer split Viacheslav Ovsiienko
2020-10-07 15:06   ` [dpdk-dev] [PATCH v2 1/9] " Viacheslav Ovsiienko
2020-10-11 22:17     ` Thomas Monjalon
2020-10-12  9:40       ` Slava Ovsiienko
2020-10-12 10:09         ` Thomas Monjalon
2020-10-07 15:06   ` [dpdk-dev] [PATCH v2 2/9] app/testpmd: add multiple pools per core creation Viacheslav Ovsiienko
2020-10-07 15:06   ` [dpdk-dev] [PATCH v2 3/9] app/testpmd: add buffer split offload configuration Viacheslav Ovsiienko
2020-10-07 15:06   ` [dpdk-dev] [PATCH v2 4/9] app/testpmd: add rxpkts commands and parameters Viacheslav Ovsiienko
2020-10-07 15:06   ` [dpdk-dev] [PATCH v2 5/9] app/testpmd: add extended Rx queue setup Viacheslav Ovsiienko
2020-10-07 15:06   ` [dpdk-dev] [PATCH v2 6/9] net/mlx5: add extended Rx queue setup routine Viacheslav Ovsiienko
2020-10-07 15:06   ` [dpdk-dev] [PATCH v2 7/9] net/mlx5: configure Rx queue to support split Viacheslav Ovsiienko
2020-10-07 15:06   ` [dpdk-dev] [PATCH v2 8/9] net/mlx5: register multiple pool for Rx queue Viacheslav Ovsiienko
2020-10-07 15:06   ` [dpdk-dev] [PATCH v2 9/9] net/mlx5: update Rx datapath to support split Viacheslav Ovsiienko
2020-10-12 16:19 ` [dpdk-dev] [PATCH v3 0/9] ethdev: introduce Rx buffer split Viacheslav Ovsiienko
2020-10-12 16:19   ` [dpdk-dev] [PATCH v3 1/9] " Viacheslav Ovsiienko
2020-10-12 16:38     ` Andrew Rybchenko
2020-10-12 17:03       ` Thomas Monjalon
2020-10-12 17:11         ` Andrew Rybchenko
2020-10-12 20:22           ` Slava Ovsiienko
2020-10-12 17:11         ` Slava Ovsiienko
2020-10-12 16:19   ` [dpdk-dev] [PATCH v3 2/9] app/testpmd: add multiple pools per core creation Viacheslav Ovsiienko
2020-10-12 16:19   ` [dpdk-dev] [PATCH v3 3/9] app/testpmd: add buffer split offload configuration Viacheslav Ovsiienko
2020-10-12 16:19   ` [dpdk-dev] [PATCH v3 4/9] app/testpmd: add rxpkts commands and parameters Viacheslav Ovsiienko
2020-10-12 16:19   ` [dpdk-dev] [PATCH v3 5/9] app/testpmd: add extended Rx queue setup Viacheslav Ovsiienko
2020-10-12 16:19   ` [dpdk-dev] [PATCH v3 6/9] net/mlx5: add extended Rx queue setup routine Viacheslav Ovsiienko
2020-10-12 16:19   ` [dpdk-dev] [PATCH v3 7/9] net/mlx5: configure Rx queue to support split Viacheslav Ovsiienko
2020-10-12 16:19   ` [dpdk-dev] [PATCH v3 8/9] net/mlx5: register multiple pool for Rx queue Viacheslav Ovsiienko
2020-10-12 16:19   ` [dpdk-dev] [PATCH v3 9/9] net/mlx5: update Rx datapath to support split Viacheslav Ovsiienko
2020-10-12 20:09 ` [dpdk-dev] [PATCH v4 0/9] ethdev: introduce Rx buffer split Viacheslav Ovsiienko
2020-10-12 20:09   ` [dpdk-dev] [PATCH v4 1/9] " Viacheslav Ovsiienko
2020-10-12 20:09   ` [dpdk-dev] [PATCH v4 2/9] app/testpmd: add multiple pools per core creation Viacheslav Ovsiienko
2020-10-12 20:09   ` [dpdk-dev] [PATCH v4 3/9] app/testpmd: add buffer split offload configuration Viacheslav Ovsiienko
2020-10-12 20:09   ` [dpdk-dev] [PATCH v4 4/9] app/testpmd: add rxpkts commands and parameters Viacheslav Ovsiienko
2020-10-12 20:09   ` [dpdk-dev] [PATCH v4 5/9] app/testpmd: add extended Rx queue setup Viacheslav Ovsiienko
2020-10-12 20:09   ` [dpdk-dev] [PATCH v4 6/9] net/mlx5: add extended Rx queue setup routine Viacheslav Ovsiienko
2020-10-12 20:10   ` [dpdk-dev] [PATCH v4 7/9] net/mlx5: configure Rx queue to support split Viacheslav Ovsiienko
2020-10-12 20:10   ` [dpdk-dev] [PATCH v4 8/9] net/mlx5: register multiple pool for Rx queue Viacheslav Ovsiienko
2020-10-12 20:10   ` [dpdk-dev] [PATCH v4 9/9] net/mlx5: update Rx datapath to support split Viacheslav Ovsiienko
2020-10-13 19:21 ` [dpdk-dev] [PATCH v5 0/6] ethdev: introduce Rx buffer split Viacheslav Ovsiienko
2020-10-13 19:21   ` [dpdk-dev] [PATCH v5 1/6] " Viacheslav Ovsiienko
2020-10-13 22:34     ` Ferruh Yigit
2020-10-14 13:31       ` Olivier Matz
2020-10-14 14:42       ` Slava Ovsiienko
2020-10-13 19:21   ` [dpdk-dev] [PATCH v5 2/6] app/testpmd: add multiple pools per core creation Viacheslav Ovsiienko
2020-10-13 19:21   ` [dpdk-dev] [PATCH v5 3/6] app/testpmd: add buffer split offload configuration Viacheslav Ovsiienko
2020-10-13 19:21   ` [dpdk-dev] [PATCH v5 4/6] app/testpmd: add rxpkts commands and parameters Viacheslav Ovsiienko
2020-10-13 19:21   ` [dpdk-dev] [PATCH v5 5/6] app/testpmd: add rxoffs " Viacheslav Ovsiienko
2020-10-13 19:21   ` [dpdk-dev] [PATCH v5 6/6] app/testpmd: add extended Rx queue setup Viacheslav Ovsiienko
2020-10-14 18:11 ` [dpdk-dev] [PATCH v6 0/6] ethdev: introduce Rx buffer split Viacheslav Ovsiienko
2020-10-14 18:11   ` [dpdk-dev] [PATCH v6 1/6] " Viacheslav Ovsiienko
2020-10-14 18:57     ` Jerin Jacob
2020-10-15  7:43       ` Slava Ovsiienko
2020-10-15  9:27         ` Jerin Jacob [this message]
2020-10-15 10:27           ` Jerin Jacob
2020-10-15 10:51             ` Slava Ovsiienko
2020-10-15 11:26               ` Jerin Jacob
2020-10-15 11:36                 ` Ferruh Yigit
2020-10-15 11:49                   ` Slava Ovsiienko
2020-10-15 12:49                     ` Thomas Monjalon
2020-10-15 13:07                       ` Andrew Rybchenko
2020-10-15 13:57                         ` Slava Ovsiienko
2020-10-15 20:22                         ` Slava Ovsiienko
2020-10-15  9:49         ` Andrew Rybchenko
2020-10-15 10:34           ` Slava Ovsiienko
2020-10-15 11:09             ` Andrew Rybchenko
2020-10-15 14:39               ` Slava Ovsiienko
2020-10-14 22:13     ` Thomas Monjalon
2020-10-14 22:50     ` Ajit Khaparde
2020-10-15 10:11     ` Andrew Rybchenko
2020-10-15 10:19       ` Thomas Monjalon
2020-10-14 18:11   ` [dpdk-dev] [PATCH v6 2/6] app/testpmd: add multiple pools per core creation Viacheslav Ovsiienko
2020-10-14 18:11   ` [dpdk-dev] [PATCH v6 3/6] app/testpmd: add buffer split offload configuration Viacheslav Ovsiienko
2020-10-14 18:12   ` [dpdk-dev] [PATCH v6 4/6] app/testpmd: add rxpkts commands and parameters Viacheslav Ovsiienko
2020-10-14 18:12   ` [dpdk-dev] [PATCH v6 5/6] app/testpmd: add rxoffs " Viacheslav Ovsiienko
2020-10-14 18:12   ` [dpdk-dev] [PATCH v6 6/6] app/testpmd: add extended Rx queue setup Viacheslav Ovsiienko
2020-10-15  0:55 ` [dpdk-dev] [PATCH v2] eal/rte_malloc: add alloc_size() attribute to allocation functions Stephen Hemminger
2020-10-19 14:13   ` Thomas Monjalon
2020-10-19 14:22     ` Thomas Monjalon
2020-10-15 20:17 ` [dpdk-dev] [PATCH v7 0/6] ethdev: introduce Rx buffer split Viacheslav Ovsiienko
2020-10-15 20:17   ` [dpdk-dev] [PATCH v7 1/6] " Viacheslav Ovsiienko
2020-10-15 20:30     ` Jerin Jacob
2020-10-15 20:33     ` Thomas Monjalon
2020-10-15 22:01       ` Ajit Khaparde
2020-10-15 20:17   ` [dpdk-dev] [PATCH v7 2/6] app/testpmd: add multiple pools per core creation Viacheslav Ovsiienko
2020-10-15 20:17   ` [dpdk-dev] [PATCH v7 3/6] app/testpmd: add buffer split offload configuration Viacheslav Ovsiienko
2020-10-15 20:17   ` [dpdk-dev] [PATCH v7 4/6] app/testpmd: add rxpkts commands and parameters Viacheslav Ovsiienko
2020-10-15 20:17   ` [dpdk-dev] [PATCH v7 5/6] app/testpmd: add rxoffs " Viacheslav Ovsiienko
2020-10-15 20:17   ` [dpdk-dev] [PATCH v7 6/6] app/testpmd: add extended Rx queue setup Viacheslav Ovsiienko
2020-10-16  7:48 ` [dpdk-dev] [PATCH v8 0/6] ethdev: introduce Rx buffer split Viacheslav Ovsiienko
2020-10-16  7:48   ` [dpdk-dev] [PATCH v8 1/6] " Viacheslav Ovsiienko
2020-10-16  8:51     ` Andrew Rybchenko
2020-10-16  8:58       ` Andrew Rybchenko
2020-10-16  9:15       ` Slava Ovsiienko
2020-10-16  9:27         ` Andrew Rybchenko
2020-10-16  9:34           ` Slava Ovsiienko
2020-10-16  9:37         ` Thomas Monjalon
2020-10-16  9:38           ` Slava Ovsiienko
2020-10-16  9:19     ` Ferruh Yigit
2020-10-16  9:21       ` Andrew Rybchenko
2020-10-16  9:22       ` Slava Ovsiienko
2020-10-16  7:48   ` [dpdk-dev] [PATCH v8 2/6] app/testpmd: add multiple pools per core creation Viacheslav Ovsiienko
2020-10-16  7:48   ` [dpdk-dev] [PATCH v8 3/6] app/testpmd: add buffer split offload configuration Viacheslav Ovsiienko
2020-10-16  7:48   ` [dpdk-dev] [PATCH v8 4/6] app/testpmd: add rxpkts commands and parameters Viacheslav Ovsiienko
2020-10-16  7:48   ` [dpdk-dev] [PATCH v8 5/6] app/testpmd: add rxoffs " Viacheslav Ovsiienko
2020-10-16  7:48   ` [dpdk-dev] [PATCH v8 6/6] app/testpmd: add extended Rx queue setup Viacheslav Ovsiienko
2020-10-16 10:22 ` [dpdk-dev] [PATCH v9 0/6] ethdev: introduce Rx buffer split Viacheslav Ovsiienko
2020-10-16 10:22   ` [dpdk-dev] [PATCH v9 1/6] " Viacheslav Ovsiienko
2020-10-16 11:21     ` Ferruh Yigit
2020-10-16 13:08       ` Slava Ovsiienko
2020-10-16 10:22   ` [dpdk-dev] [PATCH v9 2/6] app/testpmd: add multiple pools per core creation Viacheslav Ovsiienko
2020-10-16 10:22   ` [dpdk-dev] [PATCH v9 3/6] app/testpmd: add buffer split offload configuration Viacheslav Ovsiienko
2020-10-16 10:22   ` [dpdk-dev] [PATCH v9 4/6] app/testpmd: add rxpkts commands and parameters Viacheslav Ovsiienko
2020-10-16 10:22   ` [dpdk-dev] [PATCH v9 5/6] app/testpmd: add rxoffs " Viacheslav Ovsiienko
2020-10-16 10:22   ` [dpdk-dev] [PATCH v9 6/6] app/testpmd: add extended Rx queue setup Viacheslav Ovsiienko
2020-10-16 12:41 ` [dpdk-dev] [PATCH v10 0/6] ethdev: introduce Rx buffer split Viacheslav Ovsiienko
2020-10-16 12:41   ` [dpdk-dev] [PATCH v10 1/6] " Viacheslav Ovsiienko
2020-10-16 12:41   ` [dpdk-dev] [PATCH v10 2/6] app/testpmd: add multiple pools per core creation Viacheslav Ovsiienko
2020-10-16 12:41   ` [dpdk-dev] [PATCH v10 3/6] app/testpmd: add buffer split offload configuration Viacheslav Ovsiienko
2020-10-16 12:41   ` [dpdk-dev] [PATCH v10 4/6] app/testpmd: add rxpkts commands and parameters Viacheslav Ovsiienko
2020-10-16 12:41   ` [dpdk-dev] [PATCH v10 5/6] app/testpmd: add rxoffs " Viacheslav Ovsiienko
2020-10-16 12:41   ` [dpdk-dev] [PATCH v10 6/6] app/testpmd: add extended Rx queue setup Viacheslav Ovsiienko
2020-10-16 13:39 ` [dpdk-dev] [PATCH v11 0/6] ethdev: introduce Rx buffer split Viacheslav Ovsiienko
2020-10-16 13:39   ` [dpdk-dev] [PATCH v11 1/6] " Viacheslav Ovsiienko
2020-10-16 15:14     ` Thomas Monjalon
2020-10-16 16:18       ` Slava Ovsiienko
2020-10-16 15:47     ` Ferruh Yigit
2020-10-16 16:05       ` Thomas Monjalon
2020-10-16 16:06         ` Ferruh Yigit
2020-10-16 13:39   ` [dpdk-dev] [PATCH v11 2/6] app/testpmd: add multiple pools per core creation Viacheslav Ovsiienko
2020-10-16 15:05     ` Ferruh Yigit
2020-10-16 15:38       ` Ferruh Yigit
2020-10-16 15:48         ` Slava Ovsiienko
2020-10-16 15:52           ` Ferruh Yigit
2020-10-16 15:55             ` Slava Ovsiienko
2020-10-16 15:57               ` Ferruh Yigit
2020-10-16 13:39   ` [dpdk-dev] [PATCH v11 3/6] app/testpmd: add buffer split offload configuration Viacheslav Ovsiienko
2020-10-16 13:39   ` [dpdk-dev] [PATCH v11 4/6] app/testpmd: add rxpkts commands and parameters Viacheslav Ovsiienko
2020-10-16 13:39   ` [dpdk-dev] [PATCH v11 5/6] app/testpmd: add rxoffs " Viacheslav Ovsiienko
2020-10-16 13:39   ` [dpdk-dev] [PATCH v11 6/6] app/testpmd: add extended Rx queue setup Viacheslav Ovsiienko
2020-10-16 16:44 ` [dpdk-dev] [PATCH v12 0/6] ethdev: introduce Rx buffer split Viacheslav Ovsiienko
2020-10-16 16:44   ` [dpdk-dev] [PATCH v12 1/6] " Viacheslav Ovsiienko
2020-10-16 19:22     ` Ferruh Yigit
2020-10-16 21:36       ` Ferruh Yigit
2020-10-16 16:44   ` [dpdk-dev] [PATCH v12 2/6] app/testpmd: add multiple pools per core creation Viacheslav Ovsiienko
2020-10-16 16:44   ` [dpdk-dev] [PATCH v12 3/6] app/testpmd: add buffer split offload configuration Viacheslav Ovsiienko
2020-10-16 16:44   ` [dpdk-dev] [PATCH v12 4/6] app/testpmd: add rxpkts commands and parameters Viacheslav Ovsiienko
2020-10-16 16:44   ` [dpdk-dev] [PATCH v12 5/6] app/testpmd: add rxoffs " Viacheslav Ovsiienko
2020-10-16 16:44   ` [dpdk-dev] [PATCH v12 6/6] app/testpmd: add extended Rx queue setup Viacheslav Ovsiienko
2020-10-16 17:05   ` [dpdk-dev] [PATCH v12 0/6] ethdev: introduce Rx buffer split Ferruh Yigit
2020-10-16 17:07     ` Slava Ovsiienko

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to='CALBAE1MHs077qDT3FomyVjqNUEc1ddmxxh4QEW4=OUvNtr-O8w@mail.gmail.com' \
    --to=jerinjacobk@gmail.com \
    --cc=arybchenko@solarflare.com \
    --cc=david.marchand@redhat.com \
    --cc=dev@dpdk.org \
    --cc=ferruh.yigit@intel.com \
    --cc=maxime.coquelin@redhat.com \
    --cc=olivier.matz@6wind.com \
    --cc=stephen@networkplumber.org \
    --cc=thomas@monjalon.net \
    --cc=viacheslavo@nvidia.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).