DPDK patches and discussions
 help / color / mirror / Atom feed
From: Bing Zhao <bingz@nvidia.com>
To: <viacheslavo@nvidia.com>, <matan@nvidia.com>, <thomas@monjalon.net>
Cc: <dev@dpdk.org>, <orika@nvidia.com>, <rasland@nvidia.com>
Subject: [dpdk-dev] [PATCH v3 10/17] net/mlx5: add ASO CT destroy handling
Date: Wed, 5 May 2021 09:50:01 +0300	[thread overview]
Message-ID: <20210505065008.30680-11-bingz@nvidia.com> (raw)
In-Reply-To: <20210505065008.30680-1-bingz@nvidia.com>

When trying to destroy an ASO connection tracking context, the DR
action created on this context should also be destroyed. Before
inserting the related software object into the management free list,
the reference count should be checked.

Right now, the context object will not be freed to the system and
will be reused directly from the free list.

Signed-off-by: Bing Zhao <bingz@nvidia.com>
---
 drivers/net/mlx5/mlx5_flow_dv.c | 16 +++++++++++++++-
 1 file changed, 15 insertions(+), 1 deletion(-)

diff --git a/drivers/net/mlx5/mlx5_flow_dv.c b/drivers/net/mlx5/mlx5_flow_dv.c
index 84e7f0b3d3..0fa0671ace 100644
--- a/drivers/net/mlx5/mlx5_flow_dv.c
+++ b/drivers/net/mlx5/mlx5_flow_dv.c
@@ -11136,9 +11136,15 @@ flow_dv_aso_ct_release(struct rte_eth_dev *dev, uint32_t idx)
 {
 	struct mlx5_priv *priv = dev->data->dev_private;
 	struct mlx5_aso_ct_pools_mng *mng = priv->sh->ct_mng;
+	uint32_t ret;
 	struct mlx5_aso_ct_action *ct = flow_aso_ct_get_by_idx(dev, idx);
-	uint32_t ret = __atomic_sub_fetch(&ct->refcnt, 1, __ATOMIC_RELAXED);
+	enum mlx5_aso_ct_state state =
+			__atomic_load_n(&ct->state, __ATOMIC_RELAXED);
 
+	/* Cannot release when CT is in the ASO SQ. */
+	if (state == ASO_CONNTRACK_WAIT || state == ASO_CONNTRACK_QUERY)
+		return -1;
+	ret = __atomic_sub_fetch(&ct->refcnt, 1, __ATOMIC_RELAXED);
 	if (!ret) {
 		if (ct->dr_action_orig) {
 #ifdef HAVE_MLX5_DR_ACTION_ASO_CT
@@ -11154,6 +11160,8 @@ flow_dv_aso_ct_release(struct rte_eth_dev *dev, uint32_t idx)
 #endif
 			ct->dr_action_rply = NULL;
 		}
+		/* Clear the state to free, no need in 1st allocation. */
+		MLX5_ASO_CT_UPDATE_STATE(ct, ASO_CONNTRACK_FREE);
 		rte_spinlock_lock(&mng->ct_sl);
 		LIST_INSERT_HEAD(&mng->free_cts, ct, next);
 		rte_spinlock_unlock(&mng->ct_sl);
@@ -13648,6 +13656,12 @@ flow_dv_action_destroy(struct rte_eth_dev *dev,
 			DRV_LOG(DEBUG, "Indirect age action %" PRIu32 " was"
 				" released with references %d.", idx, ret);
 		return 0;
+	case MLX5_INDIRECT_ACTION_TYPE_CT:
+		ret = flow_dv_aso_ct_release(dev, idx);
+		if (ret)
+			DRV_LOG(DEBUG, "Connection tracking object %u still "
+				"has references %d.", idx, ret);
+		return 0;
 	default:
 		return rte_flow_error_set(error, ENOTSUP,
 					  RTE_FLOW_ERROR_TYPE_ACTION,
-- 
2.27.0


  parent reply	other threads:[~2021-05-05  6:52 UTC|newest]

Thread overview: 147+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2021-04-27 15:37 [dpdk-dev] [PATCH 00/17] conntrack support in mlx5 PMD Bing Zhao
2021-04-27 15:37 ` [dpdk-dev] [PATCH 01/17] common/mlx5: add connection tracking object definition Bing Zhao
2021-04-27 15:37 ` [dpdk-dev] [PATCH 02/17] common/mlx5: add CT offload capability checking Bing Zhao
2021-04-27 15:37 ` [dpdk-dev] [PATCH 03/17] net/mlx5: use meter color reg for CT Bing Zhao
2021-04-27 15:37 ` [dpdk-dev] [PATCH 04/17] net/mlx5: initialization of CT management Bing Zhao
2021-04-27 15:37 ` [dpdk-dev] [PATCH 05/17] common/mlx5: add Dexv CT objects creation Bing Zhao
2021-04-27 15:38 ` [dpdk-dev] [PATCH 06/17] net/mlx5: add modify support for CT Bing Zhao
2021-04-27 15:38 ` [dpdk-dev] [PATCH 07/17] net/mlx5: add actions creating " Bing Zhao
2021-04-27 15:38 ` [dpdk-dev] [PATCH 08/17] net/mlx5: close CT management structure Bing Zhao
2021-04-27 15:38 ` [dpdk-dev] [PATCH 09/17] net/mlx5: add ASO CT query implementation Bing Zhao
2021-04-27 15:38 ` [dpdk-dev] [PATCH 10/17] net/mlx5: add ASO CT destroy handling Bing Zhao
2021-04-27 15:38 ` [dpdk-dev] [PATCH 11/17] net/mlx5: add translation for CT action Bing Zhao
2021-04-27 15:38 ` [dpdk-dev] [PATCH 12/17] net/mlx5: add translation of CT item Bing Zhao
2021-04-27 15:38 ` [dpdk-dev] [PATCH 13/17] net/mlx5: add CT context update Bing Zhao
2021-04-27 15:38 ` [dpdk-dev] [PATCH 14/17] net/mlx5: validation of CT action Bing Zhao
2021-04-27 15:38 ` [dpdk-dev] [PATCH 15/17] net/mlx5: validation of CT item Bing Zhao
2021-04-27 15:38 ` [dpdk-dev] [PATCH 16/17] net/mlx5: reduce the reference count of CT Bing Zhao
2021-04-27 15:38 ` [dpdk-dev] [PATCH 17/17] net/mlx5: add support of CT between two ports Bing Zhao
2021-05-05  4:19 ` [dpdk-dev] [PATCH v2 00/17] conntrack support in mlx5 PMD Bing Zhao
2021-05-05  4:19   ` [dpdk-dev] [PATCH v2 01/17] common/mlx5: add connection tracking object definition Bing Zhao
2021-05-05  4:19   ` [dpdk-dev] [PATCH v2 02/17] common/mlx5: add CT offload capability checking Bing Zhao
2021-05-05  4:19   ` [dpdk-dev] [PATCH v2 03/17] net/mlx5: use meter color reg for CT Bing Zhao
2021-05-05  4:19   ` [dpdk-dev] [PATCH v2 04/17] net/mlx5: initialization of CT management Bing Zhao
2021-05-05  4:19   ` [dpdk-dev] [PATCH v2 05/17] common/mlx5: add Dexv CT objects creation Bing Zhao
2021-05-05  4:19   ` [dpdk-dev] [PATCH v2 06/17] net/mlx5: add modify support for CT Bing Zhao
2021-05-05  4:19   ` [dpdk-dev] [PATCH v2 07/17] net/mlx5: add actions creating " Bing Zhao
2021-05-05  4:19   ` [dpdk-dev] [PATCH v2 08/17] net/mlx5: close CT management structure Bing Zhao
2021-05-05  4:19   ` [dpdk-dev] [PATCH v2 09/17] net/mlx5: add ASO CT query implementation Bing Zhao
2021-05-05  4:19   ` [dpdk-dev] [PATCH v2 10/17] net/mlx5: add ASO CT destroy handling Bing Zhao
2021-05-05  4:19   ` [dpdk-dev] [PATCH v2 11/17] net/mlx5: add translation of CT action Bing Zhao
2021-05-05  4:19   ` [dpdk-dev] [PATCH v2 12/17] net/mlx5: add translation of CT item Bing Zhao
2021-05-05  4:20   ` [dpdk-dev] [PATCH v2 13/17] net/mlx5: add CT context update Bing Zhao
2021-05-05  4:20   ` [dpdk-dev] [PATCH v2 14/17] net/mlx5: validation of CT action Bing Zhao
2021-05-05  4:20   ` [dpdk-dev] [PATCH v2 15/17] net/mlx5: validation of CT item Bing Zhao
2021-05-05  4:20   ` [dpdk-dev] [PATCH v2 16/17] net/mlx5: add support of CT between two ports Bing Zhao
2021-05-05  4:20   ` [dpdk-dev] [PATCH v2 17/17] doc: update mlx5 support for conntrack Bing Zhao
2021-05-05  6:05   ` [dpdk-dev] [PATCH v2 00/17] conntrack support in mlx5 PMD Slava Ovsiienko
2021-05-05  6:40 ` [dpdk-dev] [PATCH v3 " Bing Zhao
2021-05-05  6:40   ` [dpdk-dev] [PATCH v3 01/17] common/mlx5: add connection tracking object definition Bing Zhao
2021-05-05  6:40   ` [dpdk-dev] [PATCH v3 02/17] common/mlx5: add CT offload capability checking Bing Zhao
2021-05-05  6:40   ` [dpdk-dev] [PATCH v3 03/17] net/mlx5: use meter color reg for CT Bing Zhao
2021-05-05  6:40   ` [dpdk-dev] [PATCH v3 04/17] net/mlx5: initialization of CT management Bing Zhao
2021-05-05  6:40   ` [dpdk-dev] [PATCH v3 05/17] common/mlx5: add Dexv CT objects creation Bing Zhao
2021-05-05  6:40   ` [dpdk-dev] [PATCH v3 06/17] net/mlx5: add modify support for CT Bing Zhao
2021-05-05  6:40   ` [dpdk-dev] [PATCH v3 07/17] net/mlx5: add actions creating " Bing Zhao
2021-05-05  6:40   ` [dpdk-dev] [PATCH v3 08/17] net/mlx5: close CT management structure Bing Zhao
2021-05-05  6:40   ` [dpdk-dev] [PATCH v3 09/17] net/mlx5: add ASO CT query implementation Bing Zhao
2021-05-05  6:40   ` [dpdk-dev] [PATCH v3 10/17] net/mlx5: add ASO CT destroy handling Bing Zhao
2021-05-05  6:40   ` [dpdk-dev] [PATCH v3 11/17] net/mlx5: add translation of CT action Bing Zhao
2021-05-05  6:40   ` [dpdk-dev] [PATCH v3 12/17] net/mlx5: add translation of CT item Bing Zhao
2021-05-05  6:41   ` [dpdk-dev] [PATCH v3 13/17] net/mlx5: add CT context update Bing Zhao
2021-05-05  6:41   ` [dpdk-dev] [PATCH v3 14/17] net/mlx5: validation of CT action Bing Zhao
2021-05-05  6:41   ` [dpdk-dev] [PATCH v3 15/17] net/mlx5: validation of CT item Bing Zhao
2021-05-05  6:41   ` [dpdk-dev] [PATCH v3 16/17] net/mlx5: add support of CT between two ports Bing Zhao
2021-05-05  6:49 ` [dpdk-dev] [PATCH v3 00/17] conntrack support in mlx5 PMD Bing Zhao
2021-05-05  6:49   ` [dpdk-dev] [PATCH v3 01/17] common/mlx5: add connection tracking object definition Bing Zhao
2021-05-05  6:49   ` [dpdk-dev] [PATCH v3 02/17] common/mlx5: add CT offload capability checking Bing Zhao
2021-05-05  6:49   ` [dpdk-dev] [PATCH v3 03/17] net/mlx5: use meter color reg for CT Bing Zhao
2021-05-05  6:49   ` [dpdk-dev] [PATCH v3 04/17] net/mlx5: initialization of CT management Bing Zhao
2021-05-05  6:49   ` [dpdk-dev] [PATCH v3 05/17] common/mlx5: add Dexv CT objects creation Bing Zhao
2021-05-05  6:49   ` [dpdk-dev] [PATCH v3 06/17] net/mlx5: add modify support for CT Bing Zhao
2021-05-05  6:49   ` [dpdk-dev] [PATCH v3 07/17] net/mlx5: add actions creating " Bing Zhao
2021-05-05  6:49   ` [dpdk-dev] [PATCH v3 08/17] net/mlx5: close CT management structure Bing Zhao
2021-05-05  6:50   ` [dpdk-dev] [PATCH v3 09/17] net/mlx5: add ASO CT query implementation Bing Zhao
2021-05-05  6:50   ` Bing Zhao [this message]
2021-05-05  6:50   ` [dpdk-dev] [PATCH v3 11/17] net/mlx5: add translation of CT action Bing Zhao
2021-05-05  6:50   ` [dpdk-dev] [PATCH v3 12/17] net/mlx5: add translation of CT item Bing Zhao
2021-05-05  6:50   ` [dpdk-dev] [PATCH v3 13/17] net/mlx5: add CT context update Bing Zhao
2021-05-05  6:50   ` [dpdk-dev] [PATCH v3 14/17] net/mlx5: validation of CT action Bing Zhao
2021-05-05  6:50   ` [dpdk-dev] [PATCH v3 15/17] net/mlx5: validation of CT item Bing Zhao
2021-05-05  6:50   ` [dpdk-dev] [PATCH v3 16/17] net/mlx5: add support of CT between two ports Bing Zhao
2021-05-05  6:50   ` [dpdk-dev] [PATCH v3 17/17] doc: update mlx5 support for conntrack Bing Zhao
2021-05-05  7:19 ` [dpdk-dev] [PATCH v4 00/17] conntrack support in mlx5 PMD Bing Zhao
2021-05-05  7:19   ` [dpdk-dev] [PATCH v4 01/17] common/mlx5: add connection tracking object definition Bing Zhao
2021-05-05  7:19   ` [dpdk-dev] [PATCH v4 02/17] common/mlx5: add CT offload capability checking Bing Zhao
2021-05-05  7:19   ` [dpdk-dev] [PATCH v4 03/17] net/mlx5: use meter color reg for CT Bing Zhao
2021-05-05  7:19   ` [dpdk-dev] [PATCH v4 04/17] net/mlx5: initialization of CT management Bing Zhao
2021-05-05  7:19   ` [dpdk-dev] [PATCH v4 05/17] common/mlx5: add Dexv CT objects creation Bing Zhao
2021-05-05  7:19   ` [dpdk-dev] [PATCH v4 06/17] net/mlx5: add modify support for CT Bing Zhao
2021-05-05  7:19   ` [dpdk-dev] [PATCH v4 07/17] net/mlx5: add actions creating " Bing Zhao
2021-05-05  7:19   ` [dpdk-dev] [PATCH v4 08/17] net/mlx5: close CT management structure Bing Zhao
2021-05-05  7:19   ` [dpdk-dev] [PATCH v4 09/17] net/mlx5: add ASO CT query implementation Bing Zhao
2021-05-05  7:19   ` [dpdk-dev] [PATCH v4 10/17] net/mlx5: add ASO CT destroy handling Bing Zhao
2021-05-05  7:19   ` [dpdk-dev] [PATCH v4 11/17] net/mlx5: add translation of CT action Bing Zhao
2021-05-05  7:19   ` [dpdk-dev] [PATCH v4 12/17] net/mlx5: add translation of CT item Bing Zhao
2021-05-05  7:19   ` [dpdk-dev] [PATCH v4 13/17] net/mlx5: add CT context update Bing Zhao
2021-05-05  7:19   ` [dpdk-dev] [PATCH v4 14/17] net/mlx5: validation of CT action Bing Zhao
2021-05-05  7:19   ` [dpdk-dev] [PATCH v4 15/17] net/mlx5: validation of CT item Bing Zhao
2021-05-05  7:19   ` [dpdk-dev] [PATCH v4 16/17] net/mlx5: add support of CT between two ports Bing Zhao
2021-05-05  7:19   ` [dpdk-dev] [PATCH v4 17/17] doc: update mlx5 support for conntrack Bing Zhao
2021-05-05  8:05 ` [dpdk-dev] [PATCH v5 00/17] conntrack support in mlx5 PMD Bing Zhao
2021-05-05  8:05   ` [dpdk-dev] [PATCH v5 01/17] common/mlx5: add connection tracking object definition Bing Zhao
2021-05-05  8:05   ` [dpdk-dev] [PATCH v5 02/17] common/mlx5: add CT offload capability checking Bing Zhao
2021-05-05  8:05   ` [dpdk-dev] [PATCH v5 03/17] net/mlx5: use meter color reg for CT Bing Zhao
2021-05-05  8:05   ` [dpdk-dev] [PATCH v5 04/17] net/mlx5: initialization of CT management Bing Zhao
2021-05-05  8:05   ` [dpdk-dev] [PATCH v5 05/17] common/mlx5: add Dexv CT objects creation Bing Zhao
2021-05-05  8:05   ` [dpdk-dev] [PATCH v5 06/17] net/mlx5: add modify support for CT Bing Zhao
2021-05-05  8:05   ` [dpdk-dev] [PATCH v5 07/17] net/mlx5: add actions creating " Bing Zhao
2021-05-05  8:05   ` [dpdk-dev] [PATCH v5 08/17] net/mlx5: close CT management structure Bing Zhao
2021-05-05  8:05   ` [dpdk-dev] [PATCH v5 09/17] net/mlx5: add ASO CT query implementation Bing Zhao
2021-05-05  8:05   ` [dpdk-dev] [PATCH v5 10/17] net/mlx5: add ASO CT destroy handling Bing Zhao
2021-05-05  8:05   ` [dpdk-dev] [PATCH v5 11/17] net/mlx5: add translation of CT action Bing Zhao
2021-05-05  8:05   ` [dpdk-dev] [PATCH v5 12/17] net/mlx5: add translation of CT item Bing Zhao
2021-05-05  8:05   ` [dpdk-dev] [PATCH v5 13/17] net/mlx5: add CT context update Bing Zhao
2021-05-05  8:05   ` [dpdk-dev] [PATCH v5 14/17] net/mlx5: validation of CT action Bing Zhao
2021-05-05  8:05   ` [dpdk-dev] [PATCH v5 15/17] net/mlx5: validation of CT item Bing Zhao
2021-05-05  8:05   ` [dpdk-dev] [PATCH v5 16/17] net/mlx5: add support of CT between two ports Bing Zhao
2021-05-05  8:06   ` [dpdk-dev] [PATCH v5 17/17] doc: update mlx5 support for conntrack Bing Zhao
2021-05-05  9:49 ` [dpdk-dev] [PATCH v6 00/17] conntrack support in mlx5 PMD Bing Zhao
2021-05-05  9:49   ` [dpdk-dev] [PATCH v6 01/17] common/mlx5: add connection tracking object definition Bing Zhao
2021-05-05  9:49   ` [dpdk-dev] [PATCH v6 02/17] common/mlx5: add CT offload capability checking Bing Zhao
2021-05-05  9:49   ` [dpdk-dev] [PATCH v6 03/17] net/mlx5: use meter color reg for CT Bing Zhao
2021-05-05  9:49   ` [dpdk-dev] [PATCH v6 04/17] net/mlx5: initialization of CT management Bing Zhao
2021-05-05  9:49   ` [dpdk-dev] [PATCH v6 05/17] common/mlx5: add Dexv CT objects creation Bing Zhao
2021-05-05  9:49   ` [dpdk-dev] [PATCH v6 06/17] net/mlx5: add modify support for CT Bing Zhao
2021-05-05  9:49   ` [dpdk-dev] [PATCH v6 07/17] net/mlx5: add actions creating " Bing Zhao
2021-05-05  9:50   ` [dpdk-dev] [PATCH v6 08/17] net/mlx5: close CT management structure Bing Zhao
2021-05-05  9:50   ` [dpdk-dev] [PATCH v6 09/17] net/mlx5: add ASO CT query implementation Bing Zhao
2021-05-05  9:50   ` [dpdk-dev] [PATCH v6 10/17] net/mlx5: add ASO CT destroy handling Bing Zhao
2021-05-05  9:50   ` [dpdk-dev] [PATCH v6 11/17] net/mlx5: add translation of CT action Bing Zhao
2021-05-05  9:50   ` [dpdk-dev] [PATCH v6 12/17] net/mlx5: add translation of CT item Bing Zhao
2021-05-05  9:50   ` [dpdk-dev] [PATCH v6 13/17] net/mlx5: add CT context update Bing Zhao
2021-05-05  9:50   ` [dpdk-dev] [PATCH v6 14/17] net/mlx5: validation of CT action Bing Zhao
2021-05-05  9:50   ` [dpdk-dev] [PATCH v6 15/17] net/mlx5: validation of CT item Bing Zhao
2021-05-05  9:50   ` [dpdk-dev] [PATCH v6 16/17] net/mlx5: add support of CT between two ports Bing Zhao
2021-05-05  9:50   ` [dpdk-dev] [PATCH v6 17/17] doc: update mlx5 support for conntrack Bing Zhao
2021-05-05 12:23 ` [dpdk-dev] [PATCH v7 00/17] conntrack support in mlx5 PMD Bing Zhao
2021-05-05 12:23   ` [dpdk-dev] [PATCH v7 01/17] common/mlx5: add connection tracking object definition Bing Zhao
2021-05-05 12:23   ` [dpdk-dev] [PATCH v7 02/17] common/mlx5: add CT offload capability checking Bing Zhao
2021-05-05 12:23   ` [dpdk-dev] [PATCH v7 03/17] net/mlx5: use meter color reg for CT Bing Zhao
2021-05-05 12:23   ` [dpdk-dev] [PATCH v7 04/17] net/mlx5: initialization of CT management Bing Zhao
2021-05-05 12:23   ` [dpdk-dev] [PATCH v7 05/17] common/mlx5: add Dexv CT objects creation Bing Zhao
2021-05-05 12:23   ` [dpdk-dev] [PATCH v7 06/17] net/mlx5: add modify support for CT Bing Zhao
2021-05-05 12:23   ` [dpdk-dev] [PATCH v7 07/17] net/mlx5: add actions creating " Bing Zhao
2021-05-05 12:23   ` [dpdk-dev] [PATCH v7 08/17] net/mlx5: close CT management structure Bing Zhao
2021-05-05 12:23   ` [dpdk-dev] [PATCH v7 09/17] net/mlx5: add ASO CT query implementation Bing Zhao
2021-05-05 12:23   ` [dpdk-dev] [PATCH v7 10/17] net/mlx5: add ASO CT destroy handling Bing Zhao
2021-05-05 12:23   ` [dpdk-dev] [PATCH v7 11/17] net/mlx5: add translation of CT action Bing Zhao
2021-05-05 12:23   ` [dpdk-dev] [PATCH v7 12/17] net/mlx5: add translation of CT item Bing Zhao
2021-05-05 12:23   ` [dpdk-dev] [PATCH v7 13/17] net/mlx5: add CT context update Bing Zhao
2021-05-05 12:23   ` [dpdk-dev] [PATCH v7 14/17] net/mlx5: validation of CT action Bing Zhao
2021-05-05 12:23   ` [dpdk-dev] [PATCH v7 15/17] net/mlx5: validation of CT item Bing Zhao
2021-05-05 12:23   ` [dpdk-dev] [PATCH v7 16/17] net/mlx5: add support of CT between two ports Bing Zhao
2021-05-05 12:23   ` [dpdk-dev] [PATCH v7 17/17] doc: update mlx5 support for conntrack Bing Zhao
2021-05-05 18:21     ` Ferruh Yigit
2021-05-18 13:05       ` Bing Zhao
2021-05-05 17:35   ` [dpdk-dev] [PATCH v7 00/17] conntrack support in mlx5 PMD Raslan Darawsheh

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20210505065008.30680-11-bingz@nvidia.com \
    --to=bingz@nvidia.com \
    --cc=dev@dpdk.org \
    --cc=matan@nvidia.com \
    --cc=orika@nvidia.com \
    --cc=rasland@nvidia.com \
    --cc=thomas@monjalon.net \
    --cc=viacheslavo@nvidia.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).