From: "Dumitrescu, Cristian" <cristian.dumitrescu@intel.com>
To: "jerinj@marvell.com" <jerinj@marvell.com>,
"dev@dpdk.org" <dev@dpdk.org>,
Thomas Monjalon <thomas@monjalon.net>,
Ferruh Yigit <ferruh.yigit@amd.com>,
Andrew Rybchenko <andrew.rybchenko@oktetlabs.ru>
Cc: "ferruh.yigit@xilinx.com" <ferruh.yigit@xilinx.com>,
"ajit.khaparde@broadcom.com" <ajit.khaparde@broadcom.com>,
"aboyer@pensando.io" <aboyer@pensando.io>,
"Xing, Beilei" <beilei.xing@intel.com>,
"Richardson, Bruce" <bruce.richardson@intel.com>,
"chas3@att.com" <chas3@att.com>,
"chenbo.xia@intel.com" <chenbo.xia@intel.com>,
"Loftus, Ciara" <ciara.loftus@intel.com>,
"dsinghrawat@marvell.com" <dsinghrawat@marvell.com>,
"Czeck, Ed" <ed.czeck@atomicrules.com>,
"evgenys@amazon.com" <evgenys@amazon.com>,
"grive@u256.net" <grive@u256.net>,
"g.singh@nxp.com" <g.singh@nxp.com>,
"zhouguoyang@huawei.com" <zhouguoyang@huawei.com>,
"Wang, Haiyue" <haiyue.wang@intel.com>,
"hkalra@marvell.com" <hkalra@marvell.com>,
"heinrich.kuhn@corigine.com" <heinrich.kuhn@corigine.com>,
"hemant.agrawal@nxp.com" <hemant.agrawal@nxp.com>,
"hyonkim@cisco.com" <hyonkim@cisco.com>,
"igorch@amazon.com" <igorch@amazon.com>,
"irusskikh@marvell.com" <irusskikh@marvell.com>,
"jgrajcia@cisco.com" <jgrajcia@cisco.com>,
"Singh, Jasvinder" <jasvinder.singh@intel.com>,
"jianwang@trustnetic.com" <jianwang@trustnetic.com>,
"jiawenwu@trustnetic.com" <jiawenwu@trustnetic.com>,
"Wu, Jingjing" <jingjing.wu@intel.com>,
"johndale@cisco.com" <johndale@cisco.com>,
"john.miller@atomicrules.com" <john.miller@atomicrules.com>,
"linville@tuxdriver.com" <linville@tuxdriver.com>,
"Wiles, Keith" <keith.wiles@intel.com>,
"kirankumark@marvell.com" <kirankumark@marvell.com>,
"oulijun@huawei.com" <oulijun@huawei.com>,
"lironh@marvell.com" <lironh@marvell.com>,
"longli@microsoft.com" <longli@microsoft.com>,
"mw@semihalf.com" <mw@semihalf.com>,
"spinler@cesnet.cz" <spinler@cesnet.cz>,
"matan@nvidia.com" <matan@nvidia.com>,
"Peters, Matt" <matt.peters@windriver.com>,
"maxime.coquelin@redhat.com" <maxime.coquelin@redhat.com>,
"mk@semihalf.com" <mk@semihalf.com>,
"humin29@huawei.com" <humin29@huawei.com>,
"pnalla@marvell.com" <pnalla@marvell.com>,
"ndabilpuram@marvell.com" <ndabilpuram@marvell.com>,
"Yang, Qiming" <qiming.yang@intel.com>,
"Zhang, Qi Z" <qi.z.zhang@intel.com>,
"radhac@marvell.com" <radhac@marvell.com>,
"rahul.lakkireddy@chelsio.com" <rahul.lakkireddy@chelsio.com>,
"rmody@marvell.com" <rmody@marvell.com>,
"Xu, Rosen" <rosen.xu@intel.com>,
"sachin.saxena@oss.nxp.com" <sachin.saxena@oss.nxp.com>,
"skoteshwar@marvell.com" <skoteshwar@marvell.com>,
"shshaikh@marvell.com" <shshaikh@marvell.com>,
"shaibran@amazon.com" <shaibran@amazon.com>,
"Siegel, Shepard" <shepard.siegel@atomicrules.com>,
"asomalap@amd.com" <asomalap@amd.com>,
"somnath.kotur@broadcom.com" <somnath.kotur@broadcom.com>,
"sthemmin@microsoft.com" <sthemmin@microsoft.com>,
"Webster, Steven" <steven.webster@windriver.com>,
"skori@marvell.com" <skori@marvell.com>,
"mtetsuyah@gmail.com" <mtetsuyah@gmail.com>,
"vburru@marvell.com" <vburru@marvell.com>,
"viacheslavo@nvidia.com" <viacheslavo@nvidia.com>,
"Wang, Xiao W" <xiao.w.wang@intel.com>,
"cloud.wangxiaoyun@huawei.com" <cloud.wangxiaoyun@huawei.com>,
"yisen.zhuang@huawei.com" <yisen.zhuang@huawei.com>,
"Wang, Yong" <yongwang@vmware.com>,
"xuanziyang2@huawei.com" <xuanziyang2@huawei.com>
Subject: RE: [dpdk-dev] [RFC] ethdev: support Tx queue free descriptor query
Date: Thu, 4 Jan 2024 13:16:00 +0000 [thread overview]
Message-ID: <DS0PR11MB74427526A1388284F70625C0EB672@DS0PR11MB7442.namprd11.prod.outlook.com> (raw)
In-Reply-To: <20231219172948.3909749-1-jerinj@marvell.com>
> -----Original Message-----
> From: jerinj@marvell.com <jerinj@marvell.com>
> Sent: Tuesday, December 19, 2023 5:30 PM
> To: dev@dpdk.org; Thomas Monjalon <thomas@monjalon.net>; Ferruh Yigit
> <ferruh.yigit@amd.com>; Andrew Rybchenko <andrew.rybchenko@oktetlabs.ru>
> Cc: ferruh.yigit@xilinx.com; ajit.khaparde@broadcom.com;
> aboyer@pensando.io; Xing, Beilei <beilei.xing@intel.com>; Richardson, Bruce
> <bruce.richardson@intel.com>; chas3@att.com; chenbo.xia@intel.com; Loftus,
> Ciara <ciara.loftus@intel.com>; dsinghrawat@marvell.com; Czeck, Ed
> <ed.czeck@atomicrules.com>; evgenys@amazon.com; grive@u256.net;
> g.singh@nxp.com; zhouguoyang@huawei.com; Wang, Haiyue
> <haiyue.wang@intel.com>; hkalra@marvell.com; heinrich.kuhn@corigine.com;
> hemant.agrawal@nxp.com; hyonkim@cisco.com; igorch@amazon.com;
> irusskikh@marvell.com; jgrajcia@cisco.com; Singh, Jasvinder
> <jasvinder.singh@intel.com>; jianwang@trustnetic.com;
> jiawenwu@trustnetic.com; Wu, Jingjing <jingjing.wu@intel.com>;
> johndale@cisco.com; john.miller@atomicrules.com; linville@tuxdriver.com;
> Wiles, Keith <keith.wiles@intel.com>; kirankumark@marvell.com;
> oulijun@huawei.com; lironh@marvell.com; longli@microsoft.com;
> mw@semihalf.com; spinler@cesnet.cz; matan@nvidia.com; Peters, Matt
> <matt.peters@windriver.com>; maxime.coquelin@redhat.com;
> mk@semihalf.com; humin29@huawei.com; pnalla@marvell.com;
> ndabilpuram@marvell.com; Yang, Qiming <qiming.yang@intel.com>; Zhang, Qi Z
> <qi.z.zhang@intel.com>; radhac@marvell.com; rahul.lakkireddy@chelsio.com;
> rmody@marvell.com; Xu, Rosen <rosen.xu@intel.com>;
> sachin.saxena@oss.nxp.com; skoteshwar@marvell.com; shshaikh@marvell.com;
> shaibran@amazon.com; Siegel, Shepard <shepard.siegel@atomicrules.com>;
> asomalap@amd.com; somnath.kotur@broadcom.com;
> sthemmin@microsoft.com; Webster, Steven <steven.webster@windriver.com>;
> skori@marvell.com; mtetsuyah@gmail.com; vburru@marvell.com;
> viacheslavo@nvidia.com; Wang, Xiao W <xiao.w.wang@intel.com>;
> cloud.wangxiaoyun@huawei.com; yisen.zhuang@huawei.com; Wang, Yong
> <yongwang@vmware.com>; xuanziyang2@huawei.com; Dumitrescu, Cristian
> <cristian.dumitrescu@intel.com>; Jerin Jacob <jerinj@marvell.com>
> Subject: [dpdk-dev] [RFC] ethdev: support Tx queue free descriptor query
>
> From: Jerin Jacob <jerinj@marvell.com>
>
> Introduce a new API to retrieve the number of available free descriptors
> in a Tx queue. Applications can leverage this API in the fast path to
> inspect the Tx queue occupancy and take appropriate actions based on the
> available free descriptors.
>
> A notable use case could be implementing Random Early Discard (RED)
> in software based on Tx queue occupancy.
>
> Signed-off-by: Jerin Jacob <jerinj@marvell.com>
> ---
> doc/guides/nics/features.rst | 10 ++++
> doc/guides/nics/features/default.ini | 1 +
> lib/ethdev/ethdev_trace_points.c | 3 ++
> lib/ethdev/rte_ethdev.h | 78 ++++++++++++++++++++++++++++
> lib/ethdev/rte_ethdev_core.h | 7 ++-
> lib/ethdev/rte_ethdev_trace_fp.h | 8 +++
> 6 files changed, 106 insertions(+), 1 deletion(-)
>
> diff --git a/doc/guides/nics/features.rst b/doc/guides/nics/features.rst
> index f7d9980849..9d6655473a 100644
> --- a/doc/guides/nics/features.rst
> +++ b/doc/guides/nics/features.rst
> @@ -962,6 +962,16 @@ management (see :doc:`../prog_guide/power_man` for
> more details).
>
> * **[implements] eth_dev_ops**: ``get_monitor_addr``
>
> +.. _nic_features_tx_queue_free_desc_query:
> +
> +Tx queue free descriptor query
> +------------------------------
> +
> +Supports to get the number of free descriptors in a Tx queue.
> +
> +* **[implements] eth_dev_ops**: ``tx_queue_free_desc_get``.
> +* **[related] API**: ``rte_eth_tx_queue_free_desc_get()``.
> +
> .. _nic_features_other:
>
> Other dev ops not represented by a Feature
> diff --git a/doc/guides/nics/features/default.ini
> b/doc/guides/nics/features/default.ini
> index 806cb033ff..b30002b1c1 100644
> --- a/doc/guides/nics/features/default.ini
> +++ b/doc/guides/nics/features/default.ini
> @@ -59,6 +59,7 @@ Packet type parsing =
> Timesync =
> Rx descriptor status =
> Tx descriptor status =
> +Tx free descriptor query =
> Basic stats =
> Extended stats =
> Stats per queue =
> diff --git a/lib/ethdev/ethdev_trace_points.c b/lib/ethdev/ethdev_trace_points.c
> index 91f71d868b..346f37f2e4 100644
> --- a/lib/ethdev/ethdev_trace_points.c
> +++ b/lib/ethdev/ethdev_trace_points.c
> @@ -481,6 +481,9 @@
> RTE_TRACE_POINT_REGISTER(rte_eth_trace_count_aggr_ports,
> RTE_TRACE_POINT_REGISTER(rte_eth_trace_map_aggr_tx_affinity,
> lib.ethdev.map_aggr_tx_affinity)
>
> +RTE_TRACE_POINT_REGISTER(rte_eth_trace_tx_queue_free_desc_get,
> + lib.ethdev.tx_queue_free_desc_get)
> +
> RTE_TRACE_POINT_REGISTER(rte_flow_trace_copy,
> lib.ethdev.flow.copy)
>
> diff --git a/lib/ethdev/rte_ethdev.h b/lib/ethdev/rte_ethdev.h
> index 77331ce652..033fcb8c9b 100644
> --- a/lib/ethdev/rte_ethdev.h
> +++ b/lib/ethdev/rte_ethdev.h
> @@ -6802,6 +6802,84 @@ rte_eth_recycle_mbufs(uint16_t rx_port_id, uint16_t
> rx_queue_id,
> __rte_experimental
> int rte_eth_buffer_split_get_supported_hdr_ptypes(uint16_t port_id, uint32_t
> *ptypes, int num);
>
> +/**
> + * @warning
> + * @b EXPERIMENTAL: this API may change, or be removed, without prior notice
> + *
> + * Get the number of free descriptors in a Tx queue.
> + *
> + * This function retrieves the number of available free descriptors in a
> + * transmit queue. Applications can use this API in the fast path to inspect
> + * Tx queue occupancy and take appropriate actions based on the available
> + * free descriptors. An example action could be implementing the
> + * Random Early Discard (RED).
> + *
> + * If there are no packets in the Tx queue, the function returns the value
> + * of `nb_tx_desc` provided during the initialization of the Tx queue using
> + * rte_eth_tx_queue_setup(), signifying that all descriptors are free.
> + *
> + * @param port_id
> + * The port identifier of the device.
> + * @param tx_queue_id
> + * The index of the transmit queue.
> + * The value must be in the range [0, nb_tx_queue - 1] previously supplied
> + * to rte_eth_dev_configure().
> + * @return
> + * - (<= UINT16_MAX) Number of free descriptors in a Tx queue
> + * - (> UINT16_MAX) if error. Enabled only when RTE_ETHDEV_DEBUG_TX is
> enabled
> + *
> + * @note This function is designed for fast-path use.
> + *
> + */
> +__rte_experimental
> +static inline uint32_t
> +rte_eth_tx_queue_free_desc_get(uint16_t port_id, uint16_t tx_queue_id)
> +{
> + struct rte_eth_fp_ops *fops;
> + uint32_t rc;
> + void *qd;
> +
> +#ifdef RTE_ETHDEV_DEBUG_TX
> + rc = UINT32_MAX;
> + if (port_id >= RTE_MAX_ETHPORTS || tx_queue_id >=
> RTE_MAX_QUEUES_PER_PORT) {
> + RTE_ETHDEV_LOG(ERR, "Invalid port_id=%u or
> tx_queue_id=%u\n",
> + port_id, tx_queue_id);
> +
> + rte_eth_trace_tx_queue_free_desc_get(port_id, tx_queue_id,
> rc);
> + return rc;
> + }
> +#endif
> +
> + /* Fetch pointer to Tx queue data */
> + fops = &rte_eth_fp_ops[port_id];
> + qd = fops->txq.data[tx_queue_id];
> +
> +#ifdef RTE_ETHDEV_DEBUG_TX
> + RTE_ETH_VALID_PORTID_OR_ERR_RET(port_id, 0);
> +
> + if (qd == NULL) {
> + RTE_ETHDEV_LOG(ERR, "Invalid Tx queue_id=%u for
> port_id=%u\n",
> + tx_queue_id, port_id);
> +
> + rte_eth_trace_tx_queue_free_desc_get(port_id, tx_queue_id,
> rc);
> + return rc;
> + }
> +
> + if (fops->tx_queue_free_desc_get == NULL) {
> + RTE_ETHDEV_LOG(ERR, "tx_queue_free_desc_get callback not
> implementedd Tx queue_id=%u for port_id=%u\n",
> + tx_queue_id, port_id);
> +
> + rte_eth_trace_tx_queue_free_desc_get(port_id, tx_queue_id,
> rc);
> + return rc;
> + }
> +#endif
> + rc = fops->tx_queue_free_desc_get(qd);
> +
> + rte_eth_trace_tx_queue_free_desc_get(port_id, tx_queue_id, rc);
> +
> + return rc;
> +}
> +
> #ifdef __cplusplus
> }
> #endif
> diff --git a/lib/ethdev/rte_ethdev_core.h b/lib/ethdev/rte_ethdev_core.h
> index 4bfaf79c6c..5b7ee66ee7 100644
> --- a/lib/ethdev/rte_ethdev_core.h
> +++ b/lib/ethdev/rte_ethdev_core.h
> @@ -60,6 +60,9 @@ typedef uint16_t (*eth_recycle_tx_mbufs_reuse_t)(void
> *txq,
> /** @internal Refill Rx descriptors with the recycling mbufs */
> typedef void (*eth_recycle_rx_descriptors_refill_t)(void *rxq, uint16_t nb);
>
> +/** @internal Get the number of free descriptors count of a Tx queue */
> +typedef uint16_t (*eth_tx_queue_free_desc_get_t)(void *txq);
> +
> /**
> * @internal
> * Structure used to hold opaque pointers to internal ethdev Rx/Tx
> @@ -116,7 +119,9 @@ struct rte_eth_fp_ops {
> eth_tx_descriptor_status_t tx_descriptor_status;
> /** Copy used mbufs from Tx mbuf ring into Rx. */
> eth_recycle_tx_mbufs_reuse_t recycle_tx_mbufs_reuse;
> - uintptr_t reserved2[2];
> + /** Get the number of free descriptors count of a Tx queue. */
> + eth_tx_queue_free_desc_get_t tx_queue_free_desc_get;
> + uintptr_t reserved2[1];
> /**@}*/
>
> } __rte_cache_aligned;
> diff --git a/lib/ethdev/rte_ethdev_trace_fp.h b/lib/ethdev/rte_ethdev_trace_fp.h
> index 186271c9ff..2c57b39bd2 100644
> --- a/lib/ethdev/rte_ethdev_trace_fp.h
> +++ b/lib/ethdev/rte_ethdev_trace_fp.h
> @@ -73,6 +73,14 @@ RTE_TRACE_POINT_FP(
> rte_trace_point_emit_u64(count);
> )
>
> +RTE_TRACE_POINT_FP(
> + rte_eth_trace_tx_queue_free_desc_get,
> + RTE_TRACE_POINT_ARGS(uint16_t port_id, uint16_t tx_queue_id,
> uint32_t nb_free_desc),
> + rte_trace_point_emit_u16(port_id);
> + rte_trace_point_emit_u16(tx_queue_id);
> + rte_trace_point_emit_u32(nb_free_desc);
> +)
> +
> #ifdef __cplusplus
> }
> #endif
> --
> 2.43.0
Hi Jerin,
I think having an API to get the number of free descriptors per queue is a good idea. Why have it only for TX queues and not for RX queues as well?
Regards,
Cristian
next prev parent reply other threads:[~2024-01-04 13:16 UTC|newest]
Thread overview: 46+ messages / expand[flat|nested] mbox.gz Atom feed top
2023-12-19 17:29 jerinj
2024-01-04 13:16 ` Dumitrescu, Cristian [this message]
2024-01-04 13:35 ` Jerin Jacob
2024-01-04 14:21 ` Konstantin Ananyev
2024-01-04 18:29 ` Thomas Monjalon
2024-01-05 9:57 ` Jerin Jacob
2024-01-05 10:03 ` Thomas Monjalon
2024-01-05 11:12 ` Konstantin Ananyev
2024-01-08 20:54 ` Morten Brørup
2024-01-09 14:45 ` Jerin Jacob
2024-01-04 21:17 ` Thomas Monjalon
2024-01-05 9:54 ` Jerin Jacob
2024-01-05 10:02 ` Thomas Monjalon
2024-01-08 10:54 ` Bruce Richardson
2024-01-08 21:15 ` Morten Brørup
2024-01-09 8:47 ` Bruce Richardson
2024-01-12 10:56 ` Ferruh Yigit
2024-01-11 15:17 ` [dpdk-dev] [v1] ethdev: support Tx queue used count jerinj
2024-01-11 16:17 ` Andrew Rybchenko
2024-01-12 6:56 ` Jerin Jacob
2024-01-11 16:20 ` Morten Brørup
2024-01-12 6:59 ` Jerin Jacob
2024-01-11 17:00 ` Stephen Hemminger
2024-01-12 7:01 ` Jerin Jacob
2024-01-12 16:30 ` Stephen Hemminger
2024-01-12 8:02 ` David Marchand
2024-01-12 9:29 ` Jerin Jacob
2024-01-12 11:34 ` Ferruh Yigit
2024-01-12 12:11 ` David Marchand
2024-01-12 14:25 ` Ferruh Yigit
2024-01-12 12:29 ` Morten Brørup
2024-01-12 14:29 ` Ferruh Yigit
2024-01-18 9:06 ` Jerin Jacob
2024-01-12 12:33 ` Konstantin Ananyev
2024-01-16 6:37 ` Jerin Jacob
2024-01-18 10:17 ` Konstantin Ananyev
2024-01-18 11:21 ` Jerin Jacob
2024-01-18 13:36 ` Morten Brørup
2024-01-19 9:52 ` Konstantin Ananyev
2024-01-19 10:32 ` Morten Brørup
2024-01-12 16:52 ` Stephen Hemminger
2024-01-18 9:47 ` [dpdk-dev] [v2] " jerinj
2024-01-22 13:00 ` Konstantin Ananyev
2024-01-23 11:46 ` Ferruh Yigit
2024-02-07 20:30 ` Ferruh Yigit
2024-01-29 15:03 ` Ferruh Yigit
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=DS0PR11MB74427526A1388284F70625C0EB672@DS0PR11MB7442.namprd11.prod.outlook.com \
--to=cristian.dumitrescu@intel.com \
--cc=aboyer@pensando.io \
--cc=ajit.khaparde@broadcom.com \
--cc=andrew.rybchenko@oktetlabs.ru \
--cc=asomalap@amd.com \
--cc=beilei.xing@intel.com \
--cc=bruce.richardson@intel.com \
--cc=chas3@att.com \
--cc=chenbo.xia@intel.com \
--cc=ciara.loftus@intel.com \
--cc=cloud.wangxiaoyun@huawei.com \
--cc=dev@dpdk.org \
--cc=dsinghrawat@marvell.com \
--cc=ed.czeck@atomicrules.com \
--cc=evgenys@amazon.com \
--cc=ferruh.yigit@amd.com \
--cc=ferruh.yigit@xilinx.com \
--cc=g.singh@nxp.com \
--cc=grive@u256.net \
--cc=haiyue.wang@intel.com \
--cc=heinrich.kuhn@corigine.com \
--cc=hemant.agrawal@nxp.com \
--cc=hkalra@marvell.com \
--cc=humin29@huawei.com \
--cc=hyonkim@cisco.com \
--cc=igorch@amazon.com \
--cc=irusskikh@marvell.com \
--cc=jasvinder.singh@intel.com \
--cc=jerinj@marvell.com \
--cc=jgrajcia@cisco.com \
--cc=jianwang@trustnetic.com \
--cc=jiawenwu@trustnetic.com \
--cc=jingjing.wu@intel.com \
--cc=john.miller@atomicrules.com \
--cc=johndale@cisco.com \
--cc=keith.wiles@intel.com \
--cc=kirankumark@marvell.com \
--cc=linville@tuxdriver.com \
--cc=lironh@marvell.com \
--cc=longli@microsoft.com \
--cc=matan@nvidia.com \
--cc=matt.peters@windriver.com \
--cc=maxime.coquelin@redhat.com \
--cc=mk@semihalf.com \
--cc=mtetsuyah@gmail.com \
--cc=mw@semihalf.com \
--cc=ndabilpuram@marvell.com \
--cc=oulijun@huawei.com \
--cc=pnalla@marvell.com \
--cc=qi.z.zhang@intel.com \
--cc=qiming.yang@intel.com \
--cc=radhac@marvell.com \
--cc=rahul.lakkireddy@chelsio.com \
--cc=rmody@marvell.com \
--cc=rosen.xu@intel.com \
--cc=sachin.saxena@oss.nxp.com \
--cc=shaibran@amazon.com \
--cc=shepard.siegel@atomicrules.com \
--cc=shshaikh@marvell.com \
--cc=skori@marvell.com \
--cc=skoteshwar@marvell.com \
--cc=somnath.kotur@broadcom.com \
--cc=spinler@cesnet.cz \
--cc=steven.webster@windriver.com \
--cc=sthemmin@microsoft.com \
--cc=thomas@monjalon.net \
--cc=vburru@marvell.com \
--cc=viacheslavo@nvidia.com \
--cc=xiao.w.wang@intel.com \
--cc=xuanziyang2@huawei.com \
--cc=yisen.zhuang@huawei.com \
--cc=yongwang@vmware.com \
--cc=zhouguoyang@huawei.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).